(503) 567-2314 info@opiniator.com

Disadvantages of Mystery Shopping – Twelve Major Pitfalls

Jul 21, 2019 | Blog

Why it simply doesn’t work and what to use instead.

What is Mystery Shopping?

Mystery shopping, also known as secret shopping, is a method used externally by market research companies or internally by companies themselves to measure the quality of service, or compliance with regulation, or to gather specific information about products and services.

More simply, it refers to the practice of anonymously evaluating the quality of services and products offered by a company through the eyes of regular customers.

When Did it Start?

First established in the early 1940’s, mystery shopping has been around for a while. But just because it’s been in action for almost 70 years, doesn’t mean that it’s an effective method of data gathering, performance audit or performance improvement. In fact, we argue, that at best, it is a high cost, low success employee training tool. Nowadays, there are much better alternatives for a company to consider.

How Big is the Mystery Shopping Industry?

According to MarketForce:

It is currently a $1.5 billion industry, with well over a million active mystery shoppers worldwide. In every industry—retail, restaurant, grocery, convenience store, banking, hospitality, entertainment, etc. —brands want their managers, front-line employees and franchisees to deliver their brand promise to customers.

The purpose of mystery shopper program is to measure employee integrity, customer service and operating standards. Together these form, ‘Customer Experience’.

What is the Purpose of Mystery Shopping?

A secret shopper gauges the collective performance of employees within a business. The shopping method is predominantly used by market research companies, watchdog organizations, and by brands themselves. They measure the quality of service, across the entire customer journey and to confirm whether employees are complying with regulations and policies.

In addition, it can also often used to gather specific information about products and services and deliver valuable insights. In either case, the information is then fed back to the organization and used to improve internal processes and audit employees.

So what is the best way to do this?

Perhaps by engaging a third-party provider. They will send in undercover customers so you can find out everything you need to know about the customer experience, and keep an eye on your staff at the same time.

Sounds great, right?

Unfortunately, it’s not quite as clear-cut as that. Not only is this old-school approach ineffective for judging customer experience, but it has at least a dozen reasons pitfalls. These should cause a business to challenge the wisdom of having a mystery shopping program.

While it offers valuable feedback, it’s essential to consider the disadvantages of mystery shopping. This article delves into the pitfalls associated with mystery shopping, providing a comprehensive understanding of its drawbacks and offering actionable recommendations.

Who Uses Mystery Shopping?

Clearly only for those with brick and mortar channels, many industries use mystery shopping, but the most common of these brands tend to be in retail, healthcare, and hospitality. These include:

  • High-street stores
  • Hotels and accommodation providers
  • Movie theaters
  • Restaurants, cafes and fast-food chains
  • Health care facilities
  • Independent shops

Does Mystery Shopping Work?

Some academics see that Mystery Shopping is effective. For example, in Hospitality and Retail: Mystery shopping has been successfully used to evaluate and improve service processes in hospitality and retail. It has provided valuable feedback on employee behavior, service environment, and customer interactions, leading to better service delivery (Šebová et al., 2021), (Rood & Dziadkowiec, 2010).

Others however see successes as harder to quantify. For example: in Multi-Site Firms mystery shopping serves as a strategic management tool to monitor and allocate resources effectively across different locations. However, the correlation between mystery shopping scores and sales performance is low, indicating its primary value in maintaining service standards rather than directly boosting sales (Block et al., 2022).

Therefore while much debated by many but, in a nutshell:

Secret shopping isn’t an effective method of measurement – in any form. And many agree:

In recent months we’ve had the opportunity to talk to over a hundred retail executives about their mystery shopping programs. The majority believe that the return on investment is poor but they continue to invest in them due to a perceived lack of alternatives. Maze – Australia

 

It’s time to say goodbye to mystery shopping. It is this old-school approach that is ineffective for judging customer experience. Customer Radar Retail Blog

Perhaps this result is not a surprise. After all:

  • Asking fake customers (professional secret shoppers)
  • Paid by a third party (secret shopping agency)
  • To assess unfamiliar processes and procedures (mystery shop program)
  • Using a synthetic transaction
  • At most, once a month
  • With the resultant poor analysis (shopping reports)

Who are we kidding people?

It is not going to provide valuable and actionable data This is not going to provide objective feedback and actionable data – rather deliver only corporate defined customer experiences.

Is there Data?

The judgment that mystery shopping doesn’t actually work comes down to the data from several studies, many from the retail industry.

Don’t believe me? A detailed article from The Journal of Retailing from May 2019 reviewed the following question:

Do Mystery Shoppers Really Predict Customer Satisfaction and Sales Performance?

Their conclusions are startling:

  • We observe a low correlation between Mystery Shopper assessments and evaluations of customers
  • In contrast to customer evaluations, Mystery Shopper assessments cannot predict sales.
  • These findings put the informative value of Mystery Shopper data into question
  • The results show that Secret Shoppers are not good proxies for real customers

Our research reveals that the level of agreement between the assessments of mystery shoppers and those of real customers is low and that MS assessments are not effective in predicting customer satisfaction.

In short, a resounding – NO!

Academics say No to mystery shopping

Mystery Shopping Disadvantages – 12 Pitfalls

Here are some of the reasons that mystery shopping should be replaced with something better. We think there are twelve.

12 Rules to get actionable customer feedback

Mystery Shopping:

  1. Is expensive
  2. Is inaccurate and biased
  3. Has limited scope
  4. Has poor data quality
  5. With illegitimate data
  6. All feedback is too delayed to be useful
  7. Causes stress on employees
  8. Heavy shopper training requirements
  9. Significant ethical concerns
  10. Some potential for fraud
  11. High psychological impact on shoppers
  12. Carries some legal risks
Lets dive into the details

1) Expensive

Mystery shopping is a considerable expense to the business. Each visit from a mystery shopper costs between $30 – $100. Specifically these costs include fees paid to shoppers and administrative overheads related to managing the programs.

Moreover, not only is it expensive, but there is little evidence that mystery shopping brings in much return on investment.

Example: Large-scale programs or monthly visits can strain the budget of small businesses.

Solution: Weigh costs against potential benefits and consider complementary feedback mechanisms like customer feedback at the store using the cell phone.

2) Inaccuracy and Bias

Mystery shoppers bring their biases and inaccuracies into evaluations. Personal experiences, expectations, and subjective opinions can affect the objectivity of their reports.

Example: A mystery shopper with a preconceived notion about a brand may influence their evaluation.

Solution: Implement multiple feedback methods to cross-verify data collected through mystery shopping. Rotate shoppers across locations and brands.

3) Limited Scope

Mystery shopping focuses on specific aspects of the customer experience, which may not provide a comprehensive view of overall service quality. Remember also that Not Enough Data to be Useful – Unlike in-store surveys and exit polls, mystery shopping is a one-man, one data point deal. Therefore, the level of data that comes from just one mystery shopping experience is always going to be very minimal. Mystery shopping involves a limited number of visits, which may not represent the overall customer experience. This the data becomes unreliable for operating improvements.

Example: Evaluations may only assess customer service, ignoring product quality and pricing.

Solution: Use mystery shopping with other evaluation methods for a holistic view.

4) Poor Data Quality

Mystery Shopping is only as good as the Mystery Shopper. These people themselves vary enormously in capability. This is partly driven by the high turnover of the industry. They may also not represent the demographics of the real business. All this means that any result that one shopper produces may be entirely inconsistent with another.

Example: Mystery shopper turnover is high, which means high acquisition, recruiting and training costs. Not all shoppers are good, which adds to the variability in data quality. The motivations of mystery shoppers include enjoyment, excitement, and personal development, but also compensation and discretionary benefits. These diverse motivations might impact their commitment to long-term engagement with a single company (Allison & Severt, 2010).

Solution: The motivations of mystery shoppers include enjoyment, excitement, and personal development, but also compensation and discretionary benefits. These diverse motivations might impact their commitment to long-term engagement with a single company (Allison & Severt, 2010).

5) Illegitimate Data

Mystery Shopping is a false premise. While mystery shoppers should be unbiased, remember they are not real customers. Consequently, their overall experience cannot be genuine. They bring their own preferences, habits, and shopping senses to the job. Moreover, they are looking for issues and a level of service of a synthetic transaction. This means mystery shoppers have a built-in negative bias. Perhaps a better term would be “Mystery Observers”.

Example: Mystery shoppers maybe shoppers in their own right but they are not their to shop or browse, rather to observe and score.

Solution: Many businesses encourage their own staff to shop its other locations, rather than rely on third parties.

6) Delayed Feedback

Conducting mystery shopping and compiling reports can lead to delays in receiving feedback, hampering timely improvements. The data is therefore Not Time-Sensitive – Shopper findings take a while to reach the relevant management. This means they become irrelevant quite quickly. Even worse, the actions taken by staff are seldom tracked so the business cannot determine if the staff actions have closed out the original issue.

Example: Feedback received weeks later might not be relevant to current service conditions.

Solution: Streamline reporting processes for quicker turnaround times.

So far, not much good news….

Bad news everywhere

7) Stress on Employees

Knowing they might be evaluated at any time can create stress among employees, leading to decreased job satisfaction and morale.

Example: Employees feeling constantly scrutinized can affect their natural behavior.

Solution: Balance mystery shopping with positive reinforcement and support for employees.

8) Heavy Shopper Training Requirements

Mystery shoppers need thorough training to ensure they know what to look for and how to report findings accurately. As reported, turnover is high for both shoppers and management.

Example: Inadequate training can lead to inconsistent and unreliable reports.

Solution: Invest in comprehensive training programs for mystery shoppers.

9) Ethical Concerns

Ethical issues arise around deception and privacy with mystery shopping.

Example: Employees may feel deceived when they learn they were evaluated by a mystery shopper.

Solution: Be transparent about using mystery shopping within legal and ethical boundaries.

10) Potential for Fraud

There is a risk of fraud if mystery shoppers collude with employees or provide falsified reports. Employees often look out for certain behaviors indicating a mystery shopper and / or simply recognize them from multiple visits

Example: Shoppers and employees could manipulate evaluation results for mutual benefit.

Solution: Implement checks and balances to verify the authenticity of reports.

11) Psychological Impact on Shoppers

Mystery shoppers can experience stress and pressure to perform evaluations accurately, affecting their well-being.

Example: The pressure to provide detailed and accurate reports can be overwhelming.

Solution: Provide support and resources to help mystery shoppers manage stress.

12) Legal Risks

Legal implications can arise if mystery shopping violates labor laws or regulations.

Example: Conducting mystery shopping without proper disclosure can lead to legal challenges.

Solution: Ensure compliance with all relevant laws and regulations when implementing programs.

So it seems that the disadvantages of Mystery shopping are just to many.

A business fires mystery shopping

The Alternative to the Mystery Shopping program – Use Your Own Customers

Your business already has a team of hundreds or thousands of ‘mystery shoppers’ – the customers that purchase from you every day – so use them! Moreover, real customers are authentic – they are the real thing, not a proxy. Customer feedback technology exists to take the mystery out of these shoppers by providing you with validated, instant feedback. Consider using real-time customer feedback software to continuously improve.. Armed with this, you can now continuously improve the aspects of your business that you know your customers care about.

This instant opportunity to loop with customers helps deliver customer loyalty.

There are solutions out there that provide customer feedback which captures comments, ratings, and feedback from real customers using their mobile phone. This on the spot  (at the point of experience) data arrives in real-time and Feedback is generated throughout the customers’ experience and delivered to the business in real-time. This means it is used to assess any part of the offer eg. Staff courtesy, queuing, product availability, cleanliness, product quality – even restroom cleanliness.

This allows you, as a business owner, to track all feedback and immediately respond to any issue – within seconds. Consequently, your team can fix the problem and connect with the customer – while they are still in your location. Perhaps this is the biggest challenge that mystery shopping fails to address.

The big benefits here are:
  1. Improved customer retention
  2. Increased operational standards
  3. Fewer complaints on social media
  4. Lower maintenance and HSE exposure
  5. Lower overall feedback costs
  6. More data and detailed analysis

In this modern-day and age of technology-driven opportunities, there are far more responsive and results-focused ways in which to find out how your organization is operating. Many of these leverage the ever-present, cell phone. This means that even without a mystery shopper, it’s entirely feasible to measure the performance of your business from customer entrance to exit and everything in between.

More to the point, it is a more effective strategy to take. In fact, this is the conclusion from a recent CustomerThink article published in August, that concludes Mystery Shopping is ineffective.

It’s a mystery no longer.

Opiniator is a customer feedback and recovery platform, delivered as SaaS.  It captures on location comments, ratings and feedback from real customers using their mobile phone. This powerful technology allows you, as a business owner, to track all feedback and monitor a resolution to any issue. This closed-loop feedback system will help improve business operations and customer satisfaction.

This form of real-time feedback capture means that a business respond to a negative experience and connect with a disgruntled customer before they take their business elsewhere.

It’s time to give mystery shopping up and use a digital alternative instead ie. a real-time data capturing and actionable customer feedback system using their own cell phone.

Mystery Shopping Gravestone

Conclusion

In summary, while mystery shopping can be valuable, it has significant disadvantages. High costs, inaccuracy, ethical concerns, and stress on employees are considerable drawbacks. Businesses should weigh these against benefits and consider complementing mystery shopping with other feedback methods for a comprehensive evaluation. Exploring alternatives and addressing the disadvantages can help businesses make informed decisions and improve their customer service strategy.

FAQs:

Q1: Why is mystery shopping considered expensive?
A1: Each visit from a mystery shopper costs between $30 to $100, making it a significant expense with little return on investment.

Q2: What is a major drawback of the data collected from mystery shopping?
A2: The data from mystery shopping is often minimal and unreliable, making it less useful for operational improvements.

Q3: How does the timing of mystery shopping findings affect their usefulness?
A3: Mystery shopping findings take a while to reach management, making them irrelevant quickly and hindering effective response to issues.

Q4: Why are the results from mystery shoppers inconsistent?
A4: The inconsistency is due to the high variability in the capabilities and experiences of different mystery shoppers.

Q5: What is a key limitation of using mystery shoppers instead of real customers?
A5: Mystery shoppers are not real customers, so their feedback lacks authenticity and can be biased.

Q6: What is a better alternative to mystery shopping for gathering customer feedback?
A6: Real-time customer feedback from actual customers using mobile technology is a more reliable and actionable alternative.

 

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *