Mystery Shopping Demystified

Dries Noeth

“Mystery shopping is only a mystery if you don’t know how to properly execute it” ~ Andries Noeth, Head: Business Intelligence and Research Insights

Mystery shopping is one of the “forgotten” methodologies that a lot of companies either don’t use or if they use it they don’t execute properly. As a result, mystery shopping either becomes too much of an effort to execute or the results do not provide real tangible insights.

Especially in a retail setting, mystery shopping can be a powerful methodology to differentiate you from your competitors, if used correctly. The intent of this article is to shed some light on the mystery surrounding mystery shopping and put forth some suggestions for using mystery shopping as an effective customer experience improvement methodology.

What is mystery shopping?

One of the first questions I get asked when consulting clients on mystery shopping is whether it is a quantitative or qualitative methodology. In my opinion, it is one of the few methodologies that can be seen as qualitative and quantitative. Because mystery shopping is in essence a numbers game (interviewers capture numerically what they see and observe) it can be viewed and should also be analysed as a quantitative study. However, because it is an observational technique (interviewers observe what is going on) and there is a level of interpretation from the interviewer it also has a qualitative aspect.

In essence, mystery shopping is a research methodology that companies can use to evaluate their products, services and processes, from a customer’s point of view. It provides you with an objective view and detailed insights into what actual customers experience when engaging with you at various touch points in their engagement journey. It is a reliable way to understand what is really happening at branches and outlets instead of what you think and hope is happening. C-suite executives might be very clear about what they strategically want to achieve, but how do you test the clarity through which this is executed operationally?

Quote 1
Another characteristic of mystery shopping that sets it apart from other methodologies is its usefulness to gain insights into what your competitors are doing. As products become increasingly similar it will become more and more important to differentiate yourself from your competitors through the service that you offer.

And while it is sometimes extremely difficult (if not impossible) to interview competitors’ customers, mystery shoppers can visit your competitors just as easily as visiting your own branches, gaining valuable insights into their operating model.

Mystery shopping as part of a customer experience management program

An aspect that most companies struggle with is understanding the difference between mystery shopping and customer satisfaction or customer experience survey-based research. Some companies feel that if they do customer satisfaction surveys they don’t need to do mystery shopping and those that do mystery shopping often feel that its’s sufficient and they do not need additional customer satisfaction surveys.

The truth is that these two methodologies measure different aspects of the business and each has their own advantages. Although either methodology on its own can provide useful insights for improving your customer experience, combining them into one customer experience management program yields synergistic results that can take your company to unrivalled customer satisfaction and delight.
Improving your customer experience is a 3 step process:

  • Understand your customers’ “ideal state”. What is their ideal experience with your company?
  • Use your Customer Experience Management (CEM) program to develop this ideal state.
  • Use mystery shopping to evaluate how closely you are delivering this ideal experience to clients.

Customer experience surveys vs. mystery shopping

Customer experience surveys – Gaining feedback from actual customers about how they experience your services and products compared to what they expect. It also measures the problems that customers experience, the likelihood to recommend your company or product and the likelihood to repurchase your product or increase the usage of a product or service. There are generally two levels of measurement in this category – the real-time closest to interaction, a quick short survey and the relational survey within a proprietary or benchmark model that measures the cumulative experience over a set period of time (i.e. last six months).

Mystery shopping – Focuses more on how well service standards and new initiatives are implemented across the entire organisation in different branches and outlets. It also measures tangible aspects like store layout, signage, processes and staff interaction.

So while customer experience feedback can shed some light on what (perceived review against customer expectations) is happening, mystery shopping provides some insight into what was the cause (the factual context against a standard/guideline) of what is happening. I personally experienced this less than a month ago when I walked into a branch and to my dismay saw that of the six possible counters only one was open to assist customers. While personally frustrated, the researcher in me watched with fascination how the queue became longer and longer, clients becoming increasingly upset and the single employee, the only one trying to help, taking the brunt of all these dissatisfied customers.

Customer experience survey results might indicate that the waiting time is too long and that staff are unfriendly and unprofessional. Mystery shopping on the other hand would have revealed that the branch is understaffed. While both are correct, neither methodology on its own would have provided you with the complete picture. Even the most jubilant employee cannot remain friendly under such circumstances.

Combining CEM with mystery shopping will give you the best of both worlds. Through customer experience surveys customers tell you what they experience compared to what they expect and mystery shoppers will review the factual compliance to aspects that have been put in place to ensure the meeting and even exceeding of customer expectations.

Another great advantage when combining the two methodologies is that they feed from each other and improve each other. Results from a CEM survey can inform you of standards that should be revised, while results from mystery shopping might highlight aspects that should be included in the CEM methodology.

It is common knowledge that you need to “inspect what you expect” and combining the two methodologies does exactly that.

The advantages and disadvantages of mystery shopping

In the following section I will discuss some of the most important advantages and disadvantages of mystery shopping. This will not only tell you what you can achieve but also what you should guard against when developing and implementing your mystery shopping program.

Quote 2

The advantages:

  • Objectively measure what typical customers experience when engaging with your company
  • What improvements can be made in the physical environment (displays, signage, queues etc.)
  • Effectively measures how staff engage with customers and how it can be improved (identifying training needs in terms of technical knowledge and/or soft skills)
  • To what extent are customers service standards and policies adhered to
  • Comparison between different branches, outlets, areas or regions and identifying those areas that are not performing optimally
  • Understanding how your competitors behave and what you can do to gain a differentiated advantage
  • Valuable insights into how the customer engagement journey can be improved

The disadvantages:

Since one mystery shopper can only collect a limited number of mystery shopping surveys in a given time period you might end up with insufficient data affecting the reliability of the results. You will need a large team of mystery shoppers to effectively conduct a mystery shopper program.

  • Day to day differences can have a significant impact on overall results. Aspects such as the time of day/week/month, system down time, staff compliment on a specific day will all have an effect on the results. Mystery shopping programs should be carefully planned and executed to account for these operational challenges.
  • In most instances professional interviewers are used to conduct the surveys. The disadvantage of this is that these interviewers complete a large number of these interactions and might be biased in their view (heightened more critical evaluation compared to a real customer). They can become either overly critical or insensitive towards circumstantial or staff related issues.
  • Mystery shopping does not provide feedback on how actual customers experience the products and service. Customers’ expectations and their perception of the product and service is not measured. This is one of the primary reasons why it is important to make mystery shopping part of customer experience program and not to use it as a stand-alone, single metric.
  • Mystery shopping can be an expensive methodology if you don’t have the right tools and people to manage the program. Using professional interviewers also contributes to the cost of the methodology, especially if you have numerous branches that you want to measure.
  • Finally, managing the entire mystery shopping process, tracking changes and implementing corrective actions will be very difficult if you don’t have an experienced team managing the program.

Expected outcomes of using mystery shopping as a methodology:

  • Identify shortcomings in your operating model and highlight ways to improve it
  • Identify staff training needs or the effectiveness of current training programs
  • Aspects in the physical environment that can be improved
  • Enable you to create a differentiated experience giving you a competitive advantage
  • Ensure that standards are adhered to across all touch points and develop benchmarking metrics
  • Ultimately improve your customer experience by understanding what delights customers and driving the staff behaviours that lead to customer delight
  • Finally, delighted customers will lead to improved customer retention and repeat business, having a direct effect on your bottom line and improved profits

World class mystery shopping at your doorstep

One of the key attributes of a well-managed mystery shopper program is to be able to report in real time on the performance of any branch or outlet. This will allow managers to implement corrective actions timeously. After all, one day of substandard engagements with customers is one day too many.

Consulta has partnered with ETHOS, a Dubai based consulting firm, to bring you a state-of-the-art mystery shopping solution. KnowledgeTRAK is a new innovative cloud based and crowd-sourced mystery shopping program that provides a solution to many of the disadvantages discussed in this article.

Image 1

  • Recruitment of mystery shoppers:
    One of the most important advantages of using KnowledgeTRAK is that we don’t use professional interviewers to conduct the mystery shopping surveys. Mystery shoppers are crowd-sourced - we use real consumers that are recruited through our online platform to perform the mystery shopper reviews as part of their daily interaction with companies. We have detailed profiles on each of our “consumer interviewers” and recruit and select those that closely fit your own client base or target audience.

  • Self-service or managed services:
    Providing you with a customised solution is what we do and your involvement can be as little or as much as you require:
    Managed service – We do everything for you; design the mystery shopper review form (survey), manage the fieldwork, do the analysis (mostly automated) and provide you with online reports that you can view and distribute as you like. We also use the same platform (app) to do customer surveys by our trained interviewers that also belong to the KnowledgeTRAK community.
    Self-service – We enable you to do everything on your own. We will setup the software for you, design the reports and once everything is in place we then transfer the knowledge of the solution to you to manage the process yourself.

  • Reporting system:
    KnowledgeTRAK is an exclusive online and app-based system. Questionnaires and mystery reviews are developed on a web-based platform; data is collected using an app for smart phones or tablets and pushed through automatically to your reporting dashboard. The dashboard interface allows you to filter and view results according to predetermined criteria such as specific areas or types of branches.Our innovative online system allows for immediate real-time reporting of results that can be accessed by anyone in your organisation. This dynamic, rapid reporting allows you to reduce turnaround times between data collection and implementing initiatives based on the results from mystery shopping interactions.

  • The Process:
    Consulta follows a very detailed process when it comes to mystery shopping involving seven steps as displayed below:
    Image 2

  • Use and distribution of the results:
    The results from the mystery shopping project provides you with the following outputs:

In-depth analysis – An in-depth understanding of all the aspects included in your mystery review/questionnaire.

Trend analysis – As a standard output you can look at the trends of various aspects on a daily basis. We usually recommend looking at weekly and monthly trends that will allow you to visually see the outcomes of actions that you have implemented.

Qualitative and quantitative data – The results contain both quantitative and qualitative information. This allows you to have numerical results that shows performance and trends and also some qualitative feedback that provides additional insights. The combination of quantitative and qualitative results puts you in a unique position to understand what is happening in your business but also why these things are happening and how to improve on them.

Query drill down reports – The system allows you to look at various aspects on the project and develop reports for each of them. Some examples are:

  1. Trend results by mystery shopping question.
  2. Weekly, monthly or quarterly reports by outlet, area or region.
  3. Results of one question for a specific region or the entire organisation as a whole.
  4. Monthly summary reports.
  5. When you combine the mystery shopping with a journey map solution you can view results by each macro or micro moment of truth.
  6. Our system is so flexible that all reports are customised according to your needs and thus any type of report or view that you would like can be developed.

Report distribution – Reports can be viewed and distributed in various ways. Each user in your organisation will have a unique profile allowing different levels of access. For example, a specific branch manager will only have access to their own results while an area or regional manager will have access to all outlets within their area of responsibility. Reports can also be downloaded as PDF documents and distributed in this manner.

Actionable insights – By providing companies with individual reports for each outlet, managers can implement results on a real-time basis and track how their results improve as a direct consequence of the changes that they implement. It has never been this easy to identify and manage changes in your company and actually see the return on your investment.

What Consulta offers

Consulta can provide you with a managed solution offering where we do everything or we can offer a self-service solution where we help you get started, transfer all our knowledge and empower you to run the mystery shopping program on your own. Irrespective of the option that you choose, Consulta offers the following support.

Quote 3

  • Questionnaire & Shopper Review development – Consulta will assist you in developing a questionnaire that will provide you with insights on all business levels and also ensure that the results are valid and actionable.
  • Recruitment – Consulta manages the entire recruitment process seamlessly on our KnowledgeTRAK system ensuring that mystery shoppers fulfil the profile of your customers or any other pre-determined criteria that you want to use. The performance of each mystery shopper is also managed and tracked on the system itself.
  • Data collection – All data is collected using the KnowledgeTRAK app and automatically captured on the KnowledgeTRAK Web interface.
  • Reporting – Consulta will develop dynamic reports with your input, ensuring that the results you see are the key drivers that you want to track.

So whether you require a fully managed mystery shopper program or just needs some advice on how to get started and manage it yourself, Consulta is available to design a customised solution that will fulfil all your mystery shopping needs.

ESOMAR guidelines

Conducting mystery shopping can become quite tricky and there are some stringent regulations to adhere to. The full ESOMAR Codes and Guidelines can be found on the ESOMAR site. Consulta will ensure that all these are adhered to when developing and conducting a mystery shopping program.

Key takeout’s
final image