Selection bias in marketing

Almost every decision we make tends to be influenced by subconscious biases. The technical term for this psychological phenomenon is “selection bias,” also known as sampling bias. The cognitive bias describes how inaccurate results are caused by selection errors (e.g. in the selection of sample units). Selection bias affects market research and should, therefore, be taken into account by brands and companies.

Selection bias significantly affects the selection of information – and not just in market research or scientific disciplines but also in everyday situations. Only those who are aware of their own infallibility can benefit from the use of biases (also referred to as distortions) and make use of their effects. Being aware of such errors in judgement can yield significant benefits for companies. By consciously trying to counteract selection biases, businesses can obtain more meaningful results from their consumer studies.

Selection bias is a statistical bias in the selection of sample units. This bias needs to be avoided to obtain meaningful data and results. For example, in marketing, selection bias jeopardizes the objectivity of customer surveys and other market research methods. There are numerous reasons why selection bias may occur, but they often involve either the participants or the individuals selecting the sample units. For example, one such bias involves people’s willingness to participate in research, provided that this is not randomly conducted. Some well-known selection biases include “participation bias,” “self-selection bias,”, and “survivorship bias.” An example of survivorship bias is when an analysis of successes or failures unintentionally only includes the data from the successes (i.e. the “survivors”) in the results.

The precautions and countermeasures required to best minimize or completely avoid selection bias are relatively complex. Statistical techniques can be useful here, such as the Heckman correction, to obtain accurate results in empirical social research or market research.

Selection bias is not just important in research. Businesses and people in their private lives also tend to select information in a biased manner or are presented with biased data due to selection. Cognitive biases contribute to us making mistakes in the selection process which then inevitably skews the results.

Such constantly recurring selection biases clearly show that we are not impartial and that impartiality requires a lot of effort. The following examples of selection bias illustrate the far-reaching consequences of sampling bias.

In the first example, a survey on general brand awareness for a health dietary supplement is to be conducted. If the survey is conducted in a gym, health food store, or organic supermarket, the products’ target audiences are being surveyed. This can be useful. However, the results of such market research must be interpreted with care because selection bias has already occurred. People who frequent gyms, health food stores, and organic supermarkets are generally more open to the effectiveness and benefits of health products. Therefore, it can be assumed that brand awareness is higher among this group of people because they were not surveyed in a neutral environment.

The second example of selection bias illustrates the far-reaching consequences of not making a truly random selection. When studying the economy, economic researchers should use a sample that is representative of all the companies in a country or region under study. If data selection is based on a register of limited and trading companies, for example, the final sample will exclude small businesses and successful freelancers (e.g. lawyers, doctors, architects), artists, and part-time workers.

It is an obvious example and experienced researchers are unlikely to make such a mistake. However, numerous smaller sampling biases can add up and quickly skew a country’s economic forecast.

Selection bias is mainly a challenge in market research, and less so in active marketing campaigns. But brands should consider potential selection biases when assessing the success of an advertising campaign if they want to be able to show accurate marketing results, since campaigns are also a form of market research.

In customer and user surveys, one type of selection bias called self-selection bias plays an important role. This bias occurs where participants are given the choice to participate in a survey. If the non-participants and the participants have a significantly different opinion (e.g. dissatisfied versus satisfied with the company, respectively), the selection bias can lead to customer satisfaction being overestimated. The problem of selection bias can and must be minimized here using statistical weighting techniques.

Selection bias is tricky in marketing because the sampling bias often occurs in conjunction with other kinds of biases. To be able to take the necessary statistical countermeasures, it is necessary to understand the combination of biases at work. These may include publication bias (only positive results are published) or the previously mentioned self-selection bias (only certain groups participate).

The smaller the amount of available data and/or the smaller the sample taken from it, the more likely it is that selection bias will make data-driven marketing impossible. Any undetected errors can skew studies and surveys and lead to disastrous decision-making errors for marketers. A properly designed study should always ensure that subconscious effects, unintentional biases, and potential tampering are avoided.

Click here for important legal disclaimers.