Tel: +27 11 463-5340/1/2 | Fax: 086 635-3592 | Mail: saarf@saarf.co.za  Contact  Map
The new senator Guerra Souty original series, with the new 36 production movement, and after glashutte original certification, it includes not only the high requirements on the movement stability and running time, even to watch the appearance of design aesthetics. To test the swiss watches assembled, can better guarantee the performance of the whole table and watch as the quality of each one assembled in the swiss rolex factory before to watch after 24 days, 6 different directions, 3 different temperature comprehensive test, after the test statistics at replica fake least ten data indicators to assess.

GUIDELINES FOR READERSHIP RESEARCH

By
PIET SMIT
Technical Director

All enquiries to be directed to :
SOUTH AFRICAN ADVERTISING RESEARCH FOUNDATION
BOX 98874, SLOANE PARK, 2152
54 QUEENS ROAD, BRYANSTON
TELEPHONE : (011) 463-5340/1/2
FACSIMILE : (011) 463-5010
E-MAIL :
saarf@saarf.co.za

CONTENTS

1. INTRODUCTION

2. GENERAL ASPECTS THAT CAN HAVE AN EFFECT ON READERSHIP LEVELS

2.1 Level of measurement
2.1.1 Vehicle distribution
2.1.2 Vehicle contact
2.1.3 Contact with the advertisement
2.1.4 Ad noting, ad perception and sales response

2.2 Definition of reading

2.3 The use of a filter/screening question

2.4 The use of a deflator

2.5 Which other questions to ask?

2.6 What to use as the currency to estimate readership

2.7 Should I prompt and if so, how?

2.8 Rotation of titles

2.9 Learning effect

3. WHICH METHODS ARE AVAILABLE TO ESTIOMATE NEWSPAPER AND MAGAZINE AUDIENCES

3.1 Readership Diary

3.2 Telephone interviewing

3.2.1 Advantages of telephone interviewing

3.2.2 Disadvantages of telephone interviewing

3.3 Mail/postal surveys

3.3.1 Advantages of postal surveys

3.3.2 Disadvantages of mail surveys

3.4 Personal face-to-face interviewing

3.4.1 Advantages of f-t-f interviewing
3.4.2 Disadvantages of f-t-f interviewing

3.5 Online and Email Surveys

3.5.1 Advantages of online and email surveys
3.5.2 Disadvantages of online and email surveys


4. WHICH METHODS ARE NOT ACCEPTABLE?

5. RESEARCH DESIGN

5.1 The Universe
The sample
Random sampling
Stratification
Systematic sampling
Cluster sampling
Disproportionate sampling
Sample size
Substitution
5.3 Questionnaire Design
5.4 Fieldwork Standards

5. CALCULATION OF REACH AND FREQUENCY

6. MARGIN OF ERROR

7. ETHICAL ASPECTS

8. THE ROLE OF SAARF

9. SOURCES CONSULTED

1. INTRODUCTION

There is a general danger that users of Market Research data will misinterpret results and thereby create confusion in the minds of interested parties. It is also possible for surveys to be conducted in such a manner that the results will not provide data that can be reliably used for their intended purpose. This scenario is especially true for estimating the audiences of newspapers and magazines and this document is an attempt to produce a summary of Best Demonstrated Research Practice that can provide authoritative guidelines to the marketing, media and advertising industries.

Before discussing the requirements for good research, one first has to distinguish between research done on behalf of the entire media, marketing and advertising industry by an independent organisation such as SAARF that has been formed to fulfil this purpose, and media owners own research.

SAARF is doing research on behalf of all media, while in most other countries research for the different media types (television, radio, print, etc) are done by different industry bodies, known as joint industry committees (JICs). Industry research is used as a currency for buying and selling of advertising time in electronic media and advertising space in printed media. The idea of industry research is to create a level playing field so that the readership levels between titles are comparable. The same apply to radio listening, television viewing, etc.

Unfortunately, AMPS does not provide sufficient details for programme and editorial planning, for targeting, positioning and other purposes. It also only provides quantitative information, while media owners and marketers also need more qualitative information about the needs, likes, preferences, etc. of the audiences. Marketers also have to do their own product development and product positioning research, research on branding, packaging, etc.

The need for this document has increased in recent times as the number of newspaper and magazine titles is growing, thereby creating the potential for a much more fragmented audience than hitherto. Naturally each title wishes to determine the size and demographic characteristics of its audience and therefore it is vital that not only the credibility of SAARF's AMPS "currency" is maintained, but that the many newcomers to the print industry are made aware of the pitfalls of conducting inappropriate media audience research. These comments are written to serve as guidelines for publishers who want to do their own research.

The AMPS samples are designed to measure national newspaper and magazine audiences down to rural level. A criterion for release into the AMPS database is that at least 40 respondents should claim readership of a specific title during the issue period before demographic details of the audience can be published. Since SAARF started reporting 12-months’ rolling data, the above minimum requirements apply to a 12-months’ sample.

SAARF invites all publishers who plan their own surveys to consult the SAARF Technical Director or the Technical Support Executive for advice before they finalise the research brief.

2. GENERAL ASPECTS THAT CAN HAVE AN EFFECT ON READERSHIP LEVELS

2.1 Level of measurement

Depending on how accurate the required audience estimate(s) are expected to be, an estimate at one of the following levels can be used:

ARF = Advertising Research Foundation in the USA

2.1.1 Vehicle distribution

This is a count of the number of advertising carrying vehicles that are distributed into the market place or the circulation figure (ABC)/ verified free distribution (VFD).

To use circulation figures on its own to estimate the audience size, one has to work with an assumption regarding the number of readers that an average copy would produce. This is extremely

difficult as it differs from publication to publication and it could change without notice due to a myriad of possible changes in the publication such as its price, title or masthead, editorial format, etc. A further shortcoming of using circulation figures only is that no details of the demographic profile of the audience are available for purposes of targeting.

2.1.2 Vehicle contact

This is the level at which AMPS estimates audiences of newspapers and magazines. AMPS determine opportunities to see (OTS) an advertisement in a certain publication, as reflected by the number of readers.

2.1.3 Contact with the advertisement

At this level one would determine opportunities to see an advertisement, which means determining whether a specific page was opened or not, which is very complicated. This is also such an involved process that it does not lend itself to surveys such as AMPS where many titles are measured. Most of the readership research done at this level around the world focuses on one or only a few titles.

For surveys such as AMPS, other variables can be used to estimate page traffic, for example thoroughness of reading, assuming that a thorough reader will have a better opportunity to see an advertisement anywhere in a publication than a person who has just paged through it. Similar assumptions using other variables can be used e.g.:

• A regular reader would also be a more thorough reader than an occasional reader and, thus, has a better opportunity to see and advertisement.

• A person who reads his own purchased or subscribed copy would read more thorough than somebody who reads a friend or family member’s copy

2.1.4 Ad noting, ad perception and sales response

The ideal for the advertiser would be to determine what impact every advertisement has on the sales of the product, which is the final level in the above diagram. Unfortunately, as is evident from the diagram, qualitative factors are at play from the ad noting level further on that make it impossible in a survey such as AMPS, to obtain a noting, perception or sales response measure. This has to be done for each advertisement individually.

2.2 Definition of reading

Before readership can be estimated, one has to decide on a definition of what reading entails.

AMPS uses the following definition:

“Average issue readership” (AIR) means the number of people who claim to personally read or paged through all or part of a copy of a publication for the first time during the issue period prior to the interview. It can be anywhere, anybody’s copy and include both current and old issues.

It will be noted that this definition is quite liberal because one would like to include all potential readers and maybe later on filter them out on other questions, rather than to exclude persons who could have been regarded as readers.

The National Readership Survey (NRS) in the UK’s definition is even wider by also including respondents who claim to only ‘glanced at’ a copy, even without touching it. In the UK environment, this can happen for instance when a person reads over the shoulder of somebody else in the underground train.

2.3 The use of a filter or screening question

Most readership surveys use a filter or screening question to get the irrelevant titles out of the way and then ask more specific questions only of the titles relevant to a specific respondent. This can be done in different ways and one has to decide on whether a time related or open-ended (ever) filter is going to be used. Also what the time span (6-months; 12-months, etc of the filter is going to be and how many answer options to provide (just a Yes/No or also an ‘Unsure’ option).

SAARF experimented with the inclusion of an ‘Unsure’ option during AMPS 2001. After having given respondents a second opportunity to group the ‘Unsure” titles into either the ‘Yes’ or ‘No’ category, we came to the conclusion that it did not assist in any way to help respondents to decide and it was deleted.

2.4 The use of a deflator

Some readership surveys also make use of a so-called readership-deflating question. The purpose of such a question is, as the name indicate, try to limit over-claiming by asking an additional question before the filter question is asked. There are lots of evidence from around the world of over-claiming (or status claiming) of particularly glossy magazines to impress the interviewer. In our environment it also happens in the lower LSM groups, where very little reading takes place and where respondents want to save themselves the embarrassment of indicating that they don’t read.

This is an example of a status deflating question:

Please indicate which statement applies best to each publication:

• I have never heard of it
• I heard of it but have never read it
• I read it in the past but no longer read it
• I still read it from time to time
• I read it regularly

Only the last three categories would then be subjected to further questions. If a normal filter follows, there are then two opportunities to filter non-readers out. Dr Valentine Appel in the USA has done a lot of work to prove that the more people are screened in, the higher the readership level, and vice versa.

However, it should be re-emphasised that one should attempt to obtain a balance between screening non-readers out and screening potential readers in.

2.5 Which other questions to ask?

Normally users of readership research results want for purposes of audience estimates to know the size of the audience, sometimes referred to as the reach or coverage and frequency of reading. In other words, if an advertisement is placed in a title, how many OTS’s it will create and if it is placed in consecutive issues (which is standard procedure for display advertisements), how the audience would accumulate over time. To determine this, you need a measure of frequency of reading (6/6; 5/6; 4/6, etc.)

Normally at least one indicator of quality of contact is asked. On the current AMPS survey, the following quality of contact question is used:

• Origin of copy

A high correlation has been found over the years between different quality of contact questions, such as the origin of the copy and thoroughness of reading (from read cover-to-cover to just flipped through it), consequently only one is regarded as sufficient. The reason why AMPS uses the origin of copy question is because it can also be used to validate the readers per copy (rpc) figures.

2.6 What to use as the currency to estimate readership

It is evident from a Summary of Readership Research Results which was launched in Venice during a Worldwide Readership Research Symposium in 2001 that almost all countries that do industry surveys are using the so-called “recent reading” method to estimate readership.

The recent reading method determines the number of people that read an issue during an average issue period, the so-called average issue readers (AIR-readers). In practice, this reflects the number of readers of a daily newspaper on an average day and yesterday is used during the interview to determine readership. The reason why yesterday is used and not today is because by the time of the interview (normally during the late afternoon/early evening), today is not complete and potential readers can still read after the interview. For weekly titles past 7-days is regarded as the issue period, for fortnightlies 14-days, and so on.

AMPS is the only study around the world that uses the First Reading in Period of Issue method, dubbed FRIPI over the years. The difference between FRIPI and recent reading is that FRIPI only uses respondents who claimed to have read a specific copy, for instance of a daily, for the first time yesterday. If the person has read the same copy that was read yesterday also the day before, he or she would be filtered out and not be included as an average issue reader of that title. This is done to eliminate statistical replication of readers by giving the same person only one opportunity to qualify with a specific copy.

Another method that is used by a few countries, for example in Germany, is the so-called frequency method. The claimed frequency of reading (6/6; 5/6; etc.) is used to allocate a reading probability to each respondent and by summarising these probabilities for the entire sample, the number of readers is estimated. A person that indicated that he/she read 6 out of 6 issues would be allocated a probability of 1, a person that reads 5 out of 6 would obtain a probability of 0.83 and so on.

Another method that was developed in the Netherlands was the First Reading Yesterday or FRY method. It used for more than a decade in Holland as well as in the Scandinavian countries and has just died because it was not a direct measure but relied on modelling.

2.7 Should I prompt and if so, how?

It has been proved that prompting improves respondents’ ability to recall reading and all significant readership surveys use prompting.

When face-to-face interviewing or mail surveys are done, a variety of ways can be used to prompt the respondent to make a reading claim. Normally reduced mastheads are used when asking the filter question. For personal interviewing two options are available. Mastheads can be grouped together (normally up to 6) on a prompt page or they can be shown on cards each carrying only one masthead. The advantage of grouped masthead is that similar titles that can be confused one with the other (e.g. home and garden titles, automotive or financial publications) can be grouped together to limit possible confusion. The weakness is that not all mastheads are good prompts and grouping them together can highlight the stronger ones to the detriment of weaker ones. They are also not all evenly prominent. Just to look at a few mastheads will illustrate this point. One the other hand, single title cards are the most objective and least biased way to prompt, but because they are not grouped cannot be used to limit title confusion.

However, the experiment to include an ‘unsure’ category in the filter question mentioned in Par 2.2 showed that title confusion on AMPS that uses single title cards was not a problem. The greatest source that causes uncertainty in the present AMPS methodology is the fact that some respondents are unsure whether they have read a specific publication within the filter period or outside of it.

During telephone interviewing, obviously only verbal prompting is possible which has limitations of its own.

2.8 Rotation of titles

It is well known that there is a rotational or position effect in the number of positive reading claims, depending on how early or how late the title occurs in the list of titles. The longer the list of titles, the larger the rotational effect becomes. When single title cards are used, it is normal practise to shuffle them in a similar way that playing cards are shuffled before every interview. This randomises the position of each title and evens out the rotation effect across all titles. However, with a lengthy list of titles, this effect may lead to a general reduction in reading claims.

The NRS in the UK is currently experimenting with using personalized title list. They delete some titles for specific respondents, based on the likelihood of reading it. A simple example to demonstrate this method is that publication with a high proportion of male readers will be asked of all male respondents, but only of a sample of female respondents. The

results of the Female sample will then be used to estimate total female readership.

When prompt pages are used the pages must be rotated between interviews to also even out the rotation effect.


2.9 Learning effect

When more than one publication group (e.g. daily and weekly newspapers) are included in the study, the filter should preferably be asked for all groups before any further questions are asked.

If the groups are separated for the filter question, and further questions are asked before moving to the second and further sections to ask the filter question, respondents learn from the group that is first in the rotation that they only get further questions for publication that they claim to have read during the filter period. This artificially reduce the levels claimed for the second and further groups in the rotation because respondents tend to claim fewer titles to reduce the number of extra questions.

3. WHICH METHODS ARE AVAILABLE TO ESTIMATE NEWSPAPER AND MAGAZINE AUDIENCES ?

3.1 Readership Diary

The use of a reading diary has been tested and reported on at many worldwide readership research symposia. Michael Brown discussed the work done in this regard in his book “Effective Print Media Measurement”

Most of the work was experimental and by using panels recruited for some other purpose than measuring readership. He summarises the status as follows:

“Setting up and running a diary panel is not an inexpensive operation. Excluding the measurement of readership among panellists, primarily recruited for other purposes, there is no case, at present, of the use of a panel in place of repeated sample surveys or continuous interviewing, but there is a considerable number of examples of experimental panels”. Some of the obstacles that one has to bear in mind in a readership panel are the frequency at which the panel is used, which might have an effect on the relative levels of titles with different publication intervals. Also those panellists get used to the questions and fatigue can quickly affect the results. It is different from a peoplemeter panel, where the same remote control unit that has to be used to switch the set on or off or to change channels is used to log-in, with multiple prompting opportunities.

3.2 Telephone interviewing

Telephone research is an acceptable method to obtain readership audience estimates, but it has limitations. Firstly, it can only be used when the telephone penetration is acceptably high. As mentioned earlier, show material cannot be used and prompting can only be done verbally, which has limitations. It works quite well for daily newspapers provided that a balanced sample by day of the week is used, but its not that accurate for publications with a longer publication intervals. It is known that the bulk of reading of a weekly, fortnightly, monthly, etc. publication takes place during the few days immediately after publication. Consequently, interviewing on other days will water down the readership to levels that are artificially low. In a survey such as AMPS, and with different titles published on different days, this method would not work.


3.2.1 Advantages of telephone interviews

• It is cheaper than personal interviewing. However, it is not advisable for a seven-day period (or any longer period for that matter). Surveys which use telephone interviewing should use balanced samples for each day of the week. Consequently, seven interviews are required to obtain seven-day information compared to one face-to-face interview and therefore telephone samples normally need to be larger.

• A telephone survey can be completed in a relatively short time. To reduce the impact of atypical events, interviewing should, thus, be spread across a number of weeks to obtain an average figure as opposed to one week only.

3.2.2 Disadvantages of telephone interviewing

• The reliance on telephone penetration has already been pointed out, as well as the larger sample requirements. Currently, only 27% of households in South Africa have access to a home telephone.

• It is recommended that the duration of telephone interviews should not exceed 15 minutes due to respondent fatigue.

• The lack of the opportunity to use prompt material is an important factor to keep in mind when telephone interviewing is done. Together with the reliance on recall, the researcher is obliged to rely on unprompted recall or to read out all the possible titles. In certain areas it would be almost impossible because of the large number of titles. Prompted recall has proven to be the more reliable method.

3.3 Mail/postal surveys

This research method entails mailing questionnaires to a sample of ‘potential' respondents. It is relatively inexpensive, assuming a sufficient delivery and response rate is obtained.

3.3.1 Advantages of Postal Surveys

• Possible interviewer biases do not exist.

• The geographic distribution of the sample does not affect the cost. Mail surveys can reach a geographically dispersed sample at the same cost that would apply to a smaller area.

• As has been mentioned already, mail surveys are a relatively inexpensive way to collect data.

3.3.2 Disadvantages of Mail surveys

• The universe is limited to respondents with a reasonable degree of literacy.

• The most important shortcoming of mail surveys is a low response rate. Furthermore, it is usually almost impossible to determine the representativeness of the response because of possible differential response rates, even if information about the universe is available. There is no way to determine whether respondents and non-respondents are similar. Very often the samples realized by using this approach are “self selected”.

• Because the questionnaire is the only means of communication, misunderstandings can easily occur.

• The physical appearance of the questionnaire, its format, layout and the wording of the questions are extremely important.

• One is reliant on the efficiency of the postal service.

• A number of areas have no or only a rudimentary postal service.

3.4 Personal face-to-face (f-t-f) interviewing

F-t-f interviewing is by far the most important data collection method and of the 64 national surveys that were reported on at the Worldwide Readership Research Symposium in 2001 in Venice, 46 used the traditional f-t-f pen and paper interviewing, while a further 4 (including AMPS) used Computer Assisted Personal Interviewing (CAPI). Only 22% there national surveys were conducted in other ways, notably by telephone in developed countries with high levels of literacy and high penetration of telephones.

3.4.1 Advantages of f-t-f interviewing

The most important advantage is the fact that prompting can be used. The different ways that can be used to prompt and some of the pitfalls are discussed in Paragraph 2. Nevertheless, most of the shortcomings of survey research can be minimized in a f-t-f interview which is not always possible when the interviewer and respondent are not physically together.

3.4.2 Disadvantages of f-t-f interviewing

Possible interviewer effects/biases is always a risk because people are not perfect. Supervision, testing, back checking and other measures should be used to reduce these possible influences to a minimum. The use of CAPI as such is also making it easier for interviewers and reduces the risk of human errors.

3.5 Online and Email Surveys

Internet based surveys allow respondents to complete a set of questions online or via email. A respondent may be invited to complete the form by visiting a specific site. Survey’s may also be administered by emailing an invitation or questionnaire to respondents to complete in their own time.

3.5.1 Advantages of online and email surveys
The advantage of both internet and email surveys is that they have the potential to reach a relatively large number of people very quickly and cost effectively. The respondent can complete and return their responses very quickly as well. Both methods offer a reduction of interviewer bias, since the respondent completes the questionnaire without the assistance of an interviewer.

For email surveys, there is a degree of flexibility in the manner in which the response is captured, as they can be completed electronically or printed out and completed on paper.

In instances where the questionnaire is completed via a website, analysis time is reduced because all responses are captured electronically.

3.5.2 Disadvantages of internet questionnaires

Unfortunately the potential of internet and email surveys to reach a large number of respondents is compromised. In South Africa particularly, internet access is available to only a small segment of the population.

Furthermore, methods of sampling for both internet and email surveys mean that the findings cannot be generalised to the population as whole. For email surveys, respondents are chosen from lists. These lists can be difficult and expensive to source and require that the respondents have agreed to put out their contact details for such surveys. There is thus an element of bias inherent to the lists. For internet-based surveys that are linked to web-sites, the sample is also self-selecting. Internet usage itself is a niche activity, and even internationally samples of internet based surveys have been found to skew towards a younger sample.

In addition, similar to postal surveys response rate is not guaranteed. International research suggests that email and internet surveys have quite low response rates.

4. WHICH METHODS ARE NOT ACCEPTABLE ?

In most of the queries that SAARF receives regarding audience levels, a reference is made to the number of competition entrants, telephone calls or letters received, or people who made financial or other contributions.

All these results are based on “self-selected” samples and cannot be grossed up or generalised to the universe. Such samples are statistically known as non-probability samples and conclusions only apply the sample. For instance, it is valid to conclude that people who entered for a competition were reading the issue (or one of the issues) in which the competition was announced. Whether they also read other issues cannot be deduced from the fact that they have entered for the competition. The same applies for people who phoned or wrote to the publication. Such information can only be used qualitatively.

Because of the invalidity of such sources as the above to estimate quantitative audience sizes, this problem is discussed at almost every international media audience research conference. Even for more qualitative purposes such as views expressed regarding the quality or liking of the editorial, the results cannot be generalised. It is again the views of some readers and others may not agree. At best, letters, phone calls and competitions can be used to assist in the design of research.

Another source of concern is that audience estimates are sometimes based on unproven, unrelated or unclear assumptions. One such example is to obtain information about the population size, and then assume that a certain percentage would read a specific title.

5. RESEARCH DESIGN

5.1 The Universe

For any research for which a sample is used to estimate certain aspects of the population, it is essential that the universe that is researched be defined and described. Before this is done, it is impossible to design a sample that represents that population.

Many sources of information on the South African population exist. The most detailed source is the latest Population Census of Statistics South Africa or StatsSA. However, censuses are only conducted about every five years, while it can take as long as three years after the census before the results are published. Because of population growth, deaths and migration, the information becomes outdated quickly. Institutions such as the Development Bank of Southern Africa (DBSA), the Human Sciences Research Council (HSRC), market research agencies, and many others regularly update population estimates by using mortality, fertility and migration trends. All AMPS surveys use population data from an international demographic contractor for universe purposes. Due to the short-term stability of fertility and mortality trend and the availability of secondary information on migration, these estimates are normally very accurate.

It is recommended that the different available sources of estimates of the population be investigated, and the best one selected to obtain details for a specific target market. Such information would make it possible not only to define the population that forms the target market properly, but also describe them by means of demographic and geographic characteristics. Such information will also make it possible to design a sample with high precision.

5.2 Sample

Apart from the size of the sample, the way in which it is designed is extremely important. In this regard, it is important that a reliable and credible source or sources of population information be used to design the sample. When detailed information is available on the universe or population that forms the target market, it is possible to design a sample that can be grossed-up to estimate the readership of the defined population.

There are a variety of standard sampling methods available that can be used for this purpose. In statistical terms they are referred to as probability samples. A probability sample implies that each person in the universe should have a fixed probability, which can also be calculated, to be included in the sample.

Other techniques, commonly referred to as non-probability sample designs, should be avoided when the aim of the research is to estimate, or quantify the audience. Quota sampling is one such method that should be avoided.

The following probability sampling methods can be considered, either separately or in tandem.

5.2.1 Random sampling

A random sample is basically a sample that is selected by chance. The principle involved is the same as when competition entries are placed in a container and winners blindly drawn.

Statistical handbooks or computer programmes can be used to generate random numbers. If a sample of 1 000 has to be drawn from a universe of 10 000, then 1 000 numbers which fall between 1 and 10 000 are drawn after each unit in the population has been numbered. The units which each of these numbers represent are then identified and they form the sample.

Although this method is statistically regarded as a probability sampling technique, it does not guarantee perfect representativeness, particularly in relatively small samples. The representativeness of the sample can be improved by using stratification or systematic sampling, which are described below.

5.2.2 Stratification

If it is known (or expected) that reading levels will or may be different among different sub-populations, for instance in different towns, cities, suburbs, or for different demographic sub-groups, eg. males or females or different age groups, then the incidence of such variables can be controlled to coincide with their incidence in the population. For example, if the geographical distribution of the target market of a city/town is known, the sample can be designed to represent these suburbs proportionately. Or, if 40% of the population is in the age group 16-24 years, the sample can be designed in such a way that 40% of the sample also falls in this group. This procedure is known as stratification.


More than one variable can be used simultaneously or interlaced to stratify the sample, in which case it is referred to as multistage-stratification, such as in the fictitious example below.

AGE

16-24 25-34 35-49 50+ Total

% % % % %

Suburb A 10 8 6 4 28

Suburb B 9 10 6 5 30

Suburb C 8 7 4 4 23

Suburb D 6 6 5 2 19

Total 33 31 21 15 100

In the above example that covers four suburbs, 10% of the universe lives in Suburb A and is in the age group 16-24. When stratification by suburb and age is used, 10% of the sample will also be 16-24 years old and live in Suburb A, and so on. More variables can be added to the above matrix, in which case the number of cells will increase. The product of the number of categories that is used for the individual variables determines the number of cells. In the above example, 4 suburbs and 4 age groups are used and provide a matrix with 4 x 4 = 16 cells. Care should be taken that the number of respondents per cell should not be too small. Statisticians suggest that the smallest cell should have at least 10 respondents, but preferably 20. The maximum number of cells that can be used, thus, goes hand in hand with the total sample size. In the above example, the smallest cell represents 2% of the population (suburb D, 50+), which means that to obtain the absolute minimum sample size in the smallest cell, a minimum sample of 500 must be used.

5.2.3 Systematic sampling

Systematic sampling is another way to ensure that the sample is spread across the entire population, and not biased towards certain sub-populations and under-representing others.

If a systematic sample of 1 000 has to be drawn from a universe of 10 000, the following procedure will be followed:


Determine the sampling interval :

If 1 000 numbers have to be drawn from 10 000, every 10th number (universe ÷ sample) has to be used to ensure that the sample is spread evenly/systematically between 1 and 10 000. The sampling interval in this case is 10.


Choose a random starting point :

For the above example, the sampling interval is 10, which means that every 10th number must be selected, starting from a random starting point from 1 to 10. This point is chosen following the procedure as is described under random sampling.

Selecting the sample:

Say that 7 is selected as the starting point, then numbers 7, 17, 27, 37, .......through to 9 997 will be used, resulting in 1 000 sampling points as required.

5.2.4 Cluster sampling

Cluster sampling is usually used for economic reasons. If any of the previous three methods are used alone, particularly if a large geographic area has to be covered, the sampling points will be spread individually across the entire area, which would increase the cost. Cluster sampling can be used in such cases to reduce the cost of travel.

Cluster sampling implies that fewer than the required points be selected and that more than one respondent is selected in the vicinity of each point. Such points are referred to as clusters.

However, it should be remembered that cluster sampling decreases the precision of the sample to the sample, if there is a possibility that more than one respondent in the cluster can fall in the same weighting cell. Clusters of 2 are used in AMPS, but a male and a female respondent is selected in each cluster, and because gender is a weighting variable, it is impossible that they would fall in the same weighting cell.

5.2.5 Disproportionate sampling

When it is important to obtain a large enough sample for separate analysis of one or more small sub-populations, for instance to report separately for a community radio station with a relatively small footprint area, that area can be over-sampled. Before reporting the results, such over-sampling should be down-weighted to reflect the correct proportion in the total population. To retain statistical validity, the level of over-sampling should not exceed a ratio of 2:1, particularly in small samples.

5.2.6 Sample size

There is a general misconception that the size of the sample is determined by the size of the universe and that large populations can only be researched by using large samples. Statistically, there is no direct relationship between the size of the universe and the size of the sample required to estimate certain aspects of that universe accurately. The size of the sample is determined by the following factors:

• The number of factors that can cause variation in the results. If it is, for instance, expected that everybody in the population will respond more or less the same, a small sample will accurately estimate this response. However, if it is, for instance, expected that males and females will respond differently, a larger sample is required. If it is expected that males and females as well as people of different ages will behave differently, an even larger sample will be required, and so on.

• The level of detail in which the results will be reported. If the aim of the survey is only to report the behaviour of the total population, using a small sample can do it relatively accurately. If, however, separate reporting of details of certain sub-populations e.g. geographical regions, demographic sub-groups such as gender and age, is important, a much larger sample is required.

• The level of accuracy of the results that is required. Because there is an inversely proportional correlation between the size of the sample and the accuracy of the results, a large sample should provide more accurate results than a smaller one.

Given the above criteria, the following minimum requirements are usually set:

• The total number of sampling points in each reporting cell should not be less than 140, but preferably not less than 200



• Each weighting cell should preferably contain at least 20 respondents.

• The ratio of weights should not be more than 2:1 if disproportionate sampling is used.

• Each reporting cell should have at least 40 respondents who claimed to have read a specific publication before demographic profiling can be done.

5.2.7 Substitution

For a variety of reasons, it always happens that not all of the initially selected respondents are available to form part of the final sample. Some respondents will refuse to participate, while others might be difficult to contact. To compensate for this and to ensure that the final sample size and structure will be the same as the selected sample, substitution can be used. However, if the level of substitution is high and if it is not controlled properly, it can bias the results. The following requirements are usually set:

• Substitutes must be selected by using a probability sampling technique, similar to the procedures that were used to select the initial sample.

• Substitution should only be allowed when it is absolutely sure that the initially selected respondent cannot be included. This implies that every attempt should be made to include persons who are difficult to contact. On AMPS, three callbacks are required on different days of the week and at different times of the day before substitution is allowed.

• Substitution should, preferably, not be allowed within the household of the initially selected respondent.

• When reporting the results, the level of substitution should be provided and, if it is high, the possible effect on the results should be indicated.

Finally, it is recommended that a statistician be consulted.

5.3 Questionnaire design

A questionnaire that is used to collect valid information is more than just a list of questions. Therefore, it is important that attention be given to not only the formulation of every question, but also to how they are arranged to ensure a logical, simple, understandable and unbiased interview. Furthermore, it is important that the questionnaire be tested and improved in a mini-sample before it is used in the final survey. The following guidelines can be used:

• Use the language of the target group. This does not only imply that the interview should be conducted in the respondent's home language or language of preference, but also that the level of the language should be simple and easily understandable. Researchers frequently tend to communicate using marketing language rather than commonly spoken language.

• Keep the questions - and the questionnaire - as short as possible. Avoid asking ‘nice to know' questions which do not really contribute to the aim of the survey.

• Biases through asking ‘leading' questions must be avoided. Even a statement such as "I am doing this research on behalf of....(name of a publisher).." can bias the results.

• Care should be taken to avoid general research problems such as over-claiming on certain questions, under-claiming on others, the probability of a rotation effect, and many others. Consultation with experienced researchers in media audience research will help to reduce such possible biases.

• Remember that media audience research is a highly specialized field and only a hand full of market researchers in South Africa are actually qualified to do it well.

5.4 Fieldwork Standards

Data collection is the most crucial aspect of all research, because mistakes that are made when collecting the information, very often cannot be corrected later. Therefore, it is important that:

• Interviewers are properly selected during recruitment to ensure that they have the abilities that are required to do high quality work;

• All selected interviewers should be trained properly in the basics of scientific data collecting;

• Interviewers should be briefed on the application of every specific questionnaire, and pilot interviews should be done before they commence with the actual interviews;


• Strict control measures should be applied to ensure that respondent errors, interviewer mistakes, misunderstanding of the questions and situation errors are limited.

• To ensure a high level of accuracy of the results, a minimum of 10% of all interviews are usually checked back. During the back checking, both the selection of the sampling point and of the correct respondent must be checked, as well as on a spot check basis using selected questions, that the information in the questionnaire has been recorded accurately. The check-backs should include the work of all interviewers.

6. CALCULATION OF REACH AND FREQUENCY

The aim of media planning basically is to select those titles which reach as many as possible persons of a specific target market a given number of times (called the frequency) in a certain time period (e.g. seven days). Therefore, research that does not provide a reach and frequency estimate will be of little value for buying and selling of advertising time.

It is, thus, important to provide users, not only with an estimate of the number of readers, but also with an estimate of their frequency of reading. In AMPS, we ask about the number of issues normally read out of five for dailies and out of 6 for weekly and longer publication interval titles. The time span covered by the number of issues in the frequency question should, however, not exceed the filter period. For instance, we use a 6-month filter question in AMPS, and for the newly included bi-monthlies we only ask frequency out of three (3x2=6), similarly quarterlies are ask out of 2.

This information is used to estimate the gross rating points (GRPs) of a schedule. If you reach a given number of people on average a certain number of times, the GRP or total number of opportunities to see is the product of the reach and frequency.

7. MARGIN OF ERROR

All research for which samples are used to estimate the behaviour, attitudes, etc., of the population is subject to sampling (or statistical) errors. If probability samples such as the samples that are described in Paragraph 5.2 are used, the size (or margin) of the error can be calculated. The margin of error can be calculated for different levels of accuracy, but in most research, the 95% confidence level is used. If the margin of error is calculated at the 95% confidence level, it means that if 100 similar samples are used, the error would for 95 of them be within the relevant margin, whilst 5 could fall outside this figure.

Two variables determine the size of the margin of error, namely:

• The size of the sample and
• The degree of unanimity of the response.

The latter refers to the ratio of the proportion of the sample who responded positively (in this instance the readers) and those who responded negatively (non-readers).

The formula (Bravis-Pearson’s product-moment coefficient) for calculating the error is:

s =  p x (100-p) x 1.96
n

Where:

s = Standard Error

p = Penetration (% who read)

n = Sample Size

8. ETHICAL ASPECTS

As in many other countries, the South African Market Research Industry strives for a high standard of research, as well as to protect the interests of the different stakeholders. Most of the leading researchers are members of the Southern African Market Research Association (SAMRA). SAMRA is a professional association and has a code of conduct, to which its members subscribe. The stakeholders are the general public who provide the information, the client who pays for it, and the researcher. The code stipulates the responsibilities of all these parties.

Furthermore, the organisation that represents market research suppliers, Research Suppliers South Africa or the RSSA is an umbrella body to which many research providers belong. This body also sets standards to which its members subscribe.

Individual researchers who are members of SAMRA and research providers who are members of the RSSA are obliged to the best of their ability, to ensure that the research practitioner(s) with which they are associated and the people conducting research on their behalf adhere to this Code of Conduct. More information on SAMRA and the RSSA can be obtained from them at:

P O Box 91879 Tel: (011) 482-1419
AUCKLAND PARK Fax: (011) 482-4609
2006

9. THE ROLE OF SAARF

The South African Advertising Research Foundation was founded in 1974 as a non-profit joint industry committee. SAARF was formed because of a need in the marketing and advertising communities for an independent, comprehensive, unbiased, reliable, regular and technically excellent media audience and products survey. Its purpose is to provide information about the population's use of the media, products, and services so as to enable reliable targeting for advertising purposes. The data are in such a format that it is used, among others, for the buying and selling of advertising time and space in the media and for strategic editorial and programme planning.

Over the years, the All Media and Products Surveys (AMPS™), the SAARF Radio Audience Measurement Survey (RAMS™) and the SAARF Television Audience Measurement Survey (TAMS™) have established themselves as reliable, valid and credible research vehicles. Apart from commissioning these surveys, SAARF also assists media owners, advertisers and advertising agencies in a series of other areas such as training in the use of AMPS data.

SAARF's mission is to serve its members, and interested persons and bodies are invited to liase with SAARF about any aspect related to media audience, market and marketing and advertising research.

SAARF will present and issue a press release when new AMPS results are published. Such press releases will comment on readership levels and trends in general terms, not on the performance of specific titles. Publishers who wish to issue press releases or newsletters with information about their specific titles are invited to consult with SAARF in this regard to ensure that the information is interpreted correctly. SAARF will only highlight significant changes, upward and downward.

The members of the SAARF Print Council, who are representative of the Print, advertising and marketing industries, have also committed themselves to assist SAARF in ensuring that the SAARF readership audience data are used correctly.

















10. SOURCES CONSULTED

a. AMPS Technical Report, South African Advertising Research Foundation (SAARF), Johannesburg, 2002.

b. ARF Guidelines Handbook, Advertising Research Foundation (ARF), New York, 1990.

c. SAMRA Yearbook, Southern African Market Research Association, Johannesburg, 2001.

d. Michael Brown, Effective Print Media Measurement: audiences ……… and more, London, 2001

e. Michael Brown, Dear Reader. Some readership questions and some answers, London, 1991

f. Erhard Meier, Summary of Readership Research, prepared for the 10th Worldwide Readership Research Symposium, Venice, November 2001

The above publications can be consulted in the SAARF Library.
xx1. INTRODUCTION
1

2. GENERAL ASPECTS THAT CAN HAVE AN EFFECT ON READERSHIP LEVELS 2

2.1 Level of measurement 2

2.1.1 Vehicle distribution 2

2.1.2 Vehicle contact 3

2.1.3 Contact with the advertisement 3

2.1.4 Ad noting, ad perception and sales response 3

2.2 Definition of reading 3

2.3 The use of a filter/screening question 4

2.4 The use of a deflator 4

2.5 Which other questions to ask? 5

2.6 What to use as the currency to estimate readership 5

2.7 Should I prompt and if so, how? 5

2.8 Rotation of titles 7

2.9 Learning effect 7

3. WHICH METHODS ARE AVAILABLE TO ESTIOMATE NEWSPAPER AND MAGAZINE AUDIENCES 7

3.1 Readership Diary 7

3.2 Telephone interviewing 8

3.2.1 Advantages of telephone interviewing 8

3.2.2 Disadvantages of telephone interviewing 9

3.3 Mail/postal surveys 9

3.3.1 Advantages of postal surveys 9

3.3.2 Disadvantages of mail surveys 9

3.4 Personal face-to-face interviewing 10

3.4.1 Advantages of f-t-f interviewing 10

3.4.2 Disadvantages of f-t-f interviewing 10

4. WHICH METHODS ARE NOT ACCEPTABLE? 10

5. RESEARCH DESIGN 11

5.1 The Universe 11

5.2 The sample 12

5.2.1 Random sampling 12

5.2.2 Stratification 13

5.2.3 Systematic sampling 14

5.2.4 Cluster sampling 14

5.2.5 Disproportionate sampling 15

5.2.6 Sample size 15

5.2.7 Substitution 16

5.3 Questionnaire Design 16

5.4 Fieldwork Standards 17

5. CALCULATION OF REACH AND FREQUENCY 18

6. MARGIN OF ERROR 18

7. ETHICAL ASPECTS 19

8. THE ROLE OF SAARF 19

9. SOURCES CONSULTED 21

1. INTRODUCTION

There is a general danger that users of Market Research data will misinterpret results and thereby create confusion in the minds of interested parties. It is also possible for surveys to be conducted in such a manner that the results will not provide data that can be reliably used for their intended purpose. This scenario is especially true for estimating the audiences of newspapers and magazines and this document is an attempt to produce a summary of Best Demonstrated Research Practice that can provide authoritative guidelines to the marketing, media and advertising industries.

Before discussing the requirements for good research, one first has to distinguish between research done on behalf of the entire media, marketing and advertising industry by an independent organisation such as SAARF that has been formed to fulfil this purpose, and media owners own research.

SAARF is doing research on behalf of all media, while in most other countries research for the different media types (television, radio, print, etc) are done by different industry bodies, known as joint industry committees (JICs). Industry research is used as a currency for buying and selling of advertising time in electronic media and advertising space in printed media. The idea of industry research is to create a level playing field so that the readership levels between titles are comparable. The same apply to radio listening, television viewing, etc.

Unfortunately, AMPS does not provide sufficient details for programme and editorial planning, for targeting, positioning and other purposes. It also only provides quantitative information, while media owners and marketers also need more qualitative information about the needs, likes, preferences, etc. of the audiences. Marketers also have to do their own product development and product positioning research, research on branding, packaging, etc.

The need for this document has increased in recent times as the number of newspaper and magazine titles is growing, thereby creating the potential for a much more fragmented audience than hitherto. Naturally each title wishes to determine the size and demographic characteristics of its audience and therefore it is vital that not only the credibility of SAARF's AMPS "currency" is maintained, but that the many newcomers to the print industry are made aware of the pitfalls of conducting inappropriate media audience research. These comments are written to serve as guidelines for publishers who want to do their own research.

The AMPS samples are designed to measure magazines (national) and national, regional and community papers’ audiences down to suburban level. Most of the Community Newspapers distribution areas are at suburban level, and consequently the AMPS sample has to, in many cases, be boosted to measure some of them. Before the inclusion of community newspapers in AMPS in the late nineties, SAARF had required a minimum of 200 AMPS respondents in the footprint area of a community newspaper. In an attempt to report information for as many community newspapers as possible, this criterion has for the interim been lowered to a minimum of 140 interviews. A further criterion is that at least 40 respondents should claim readership of a specific title during the issue period before demographic details of the audience can be published. Since SAARF started reporting 12-months’ rolling data, the above minimum requirements apply to a 12-months’ sample.

SAARF invites all publishers who plan their own surveys to consult the SAARF Technical Director or the Technical Support Executive for advice before they finalise the research brief.

2. GENERAL ASPECTS THAT CAN HAVE AN EFFECT ON READERSHIP LEVELS

2.1 Level of measurement

Depending on how accurate the required audience estimate(s) are expected to be, an estimate at one of the following levels can be used:

 

ARF = Advertising Research Foundation in the USA

2.1.1 Vehicle distribution

This is a count of the number of advertising carrying vehicles that are distributed into the market place or the circulation figure (ABC)/ verified free distribution (VFD).

To use circulation figures on its own to estimate the audience size, one has to work with an assumption regarding the number of readers that an average copy would produce. This is extremely

difficult as it differs from publication to publication and it could change without notice due to a myriad of possible changes in the publication such as its price, title or masthead, editorial format, etc. A further shortcoming of using circulation figures only is that no details of the demographic profile of the audience are available for purposes of targeting.

2.1.2 Vehicle contact

This is the level at which AMPS estimates audiences of newspapers and magazines. AMPS determine opportunities to see (OTS) an advertisement in a certain publication, as reflected by the number of readers.

2.1.3 Contact with the advertisement

At this level one would determine opportunities to see an advertisement, which means determining whether a specific page was opened or not, which is very complicated. This is also such an involved process that it does not lend itself to surveys such as AMPS where many titles are measured. Most of the readership research done at this level around the world focuses on one or only a few titles.

For surveys such as AMPS, other variables can be used to estimate page traffic, for example thoroughness of reading, assuming that a thorough reader will have a better opportunity to see an advertisement anywhere in a publication than a person who has just paged through it. Similar assumptions using other variables can be used e.g.:

2.1.4 Ad noting, ad perception and sales response

The ideal for the advertiser would be to determine what impact every advertisement has on the sales of the product, which is the final level in the above diagram. Unfortunately, as is evident from the diagram, qualitative factors are at play from the ad noting level further on that make it impossible in a survey such as AMPS, to obtain a noting, perception or sales response measure. This has to be done for each advertisement individually.

2.2 Definition of reading

Before readership can be estimated, one has to decide on a definition of what reading entails.

AMPS uses the following definition:

“Average issue readership” (AIR) means the number of people who claim to personally read or paged through all or part of a copy of a publication for the first time during the issue period prior to the interview. It can be anywhere, anybody’s copy and include both current and old issues.

It will be noted that this definition is quite liberal because one would like to include all potential readers and maybe later on filter them out on other questions, rather than to exclude persons who could have been regarded as readers.

The National Readership Survey (NRS) in the UK’s definition is even wider by also including respondents who claim to only ‘glanced at’ a copy, even without touching it. In the UK environment, this can happen for instance when a person reads over the shoulder of somebody else in the underground train.

2.3 The use of a filter or screening question

Most readership surveys use a filter or screening question to get the irrelevant titles out of the way and then ask more specific questions only of the titles relevant to a specific respondent. This can be done in different ways and one has to decide on whether a time related or open-ended (ever) filter is going to be used. Also what the time span (6-months; 12-months, etc of the filter is going to be and how many answer options to provide (just a Yes/No or also an ‘Unsure’ option).

SAARF experimented with the inclusion of an ‘Unsure’ option during AMPS 2001. After having given respondents a second opportunity to group the ‘Unsure” titles into either the ‘Yes’ or ‘No’ category, we came to the conclusion that it did not assist in any way to help respondents to decide and it was deleted.

2.4 The use of a deflator

Some readership surveys also make use of a so-called readership-deflating question. The purpose of such a question is, as the name indicate, try to limit over-claiming by asking an additional question before the filter question is asked. There are lots of evidence from around the world of over-claiming (or status claiming) of particularly glossy magazines to impress the interviewer. In our environment it also happens in the lower LSM groups, where very little reading takes place and where respondents want to save themselves the embarrassment of indicating that they don’t read.

This is an example of a status deflating question:

Please indicate which statement applies best to each publication:

Only the last three categories would then be subjected to further questions. If a normal filter follows, there are then two opportunities to filter non-readers out. Dr Valentine Appel in the USA has done a lot of work to prove that the more people are screened in, the higher the readership level, and vice versa.

However, it should be re-emphasised that one should attempt to obtain a balance between screening non-readers out and screening potential readers in.

2.5 Which other questions to ask?

Normally users of readership research results want for purposes of audience estimates to know the size of the audience, sometimes referred to as the reach or coverage and frequency of reading. In other words, if an advertisement is placed in a title, how many OTS’s it will create and if it is placed in consecutive issues (which is standard procedure for display advertisements), how the audience would accumulate over time. To determine this, you need a measure of frequency of reading (6/6; 5/6; 4/6, etc.)

Normally at least one indicator of quality of contact is asked. On the current AMPS survey, the following quality of contact question is used:

A high correlation has been found over the years between different quality of contact questions, such as the origin of the copy and thoroughness of reading (from read cover-to-cover to just flipped through it), consequently only one is regarded as sufficient. The reason why AMPS uses the origin of copy question is because it can also be used to validate the readers per copy (rpc) figures.

2.6 What to use as the currency to estimate readership

It is evident from a Summary of Readership Research Results which was launched in Venice during a Worldwide Readership Research Symposium in 2001 that almost all countries that do industry surveys are using the so-called “recent reading” method to estimate readership.

The recent reading method determines the number of people that read an issue during an average issue period, the so-called average issue readers (AIR-readers). In practice, this reflects the number of readers of a daily newspaper on an average day and yesterday is used during the interview to determine readership. The reason why yesterday is used and not today is because by the time of the interview (normally during the late afternoon/early evening), today is not complete and potential readers can still read after the interview. For weekly titles past 7-days is regarded as the issue period, for fortnightlies 14-days, and so on.

AMPS is the only study around the world that uses the First Reading in Period of Issue method, dubbed FRIPI over the years. The difference between FRIPI and recent reading is that FRIPI only uses respondents who claimed to have read a specific copy, for instance of a daily, for the first time yesterday. If the person has read the same copy that was read yesterday also the day before, he or she would be filtered out and not be included as an average issue reader of that title. This is done to eliminate statistical replication of readers by giving the same person only one opportunity to qualify with a specific copy.

Another method that is used by a few countries, for example in Germany, is the so-called frequency method. The claimed frequency of reading (6/6; 5/6; etc.) is used to allocate a reading probability to each respondent and by summarising these probabilities for the entire sample, the number of readers is estimated. A person that indicated that he/she read 6 out of 6 issues would be allocated a probability of 1, a person that reads 5 out of 6 would obtain a probability of 0.83 and so on.

Another method that was developed in the Netherlands was the First Reading Yesterday or FRY method. It used for more than a decade in Holland as well as in the Scandinavian countries and has just died because it was not a direct measure but relied on modelling.

2.7 Should I prompt and if so, how?

It has been proved that prompting improves respondents’ ability to recall reading and all significant readership surveys use prompting.

When face-to-face interviewing or mail surveys are done, a variety of ways can be used to prompt the respondent to make a reading claim. Normally reduced mastheads are used when asking the filter question. For personal interviewing two options are available. Mastheads can be grouped together (normally up to 6) on a prompt page or they can be shown on cards each carrying only one masthead. The advantage of grouped masthead is that similar titles that can be confused one with the other (e.g. home and garden titles, automotive or financial publications) can be grouped together to limit possible confusion. The weakness is that not all mastheads are good prompts and grouping them together can highlight the stronger ones to the detriment of weaker ones. They are also not all evenly prominent. Just to look at a few mastheads will illustrate this point. One the other hand, single title cards are the most objective and least biased way to prompt, but because they are not grouped cannot be used to limit title confusion.

However, the experiment to include an ‘unsure’ category in the filter question mentioned in Par 2.2 showed that title confusion on AMPS that uses single title cards was not a problem. The greatest source that causes uncertainty in the present AMPS methodology is the fact that some respondents are unsure whether they have read a specific publication within the filter period or outside of it.

During telephone interviewing, obviously only verbal prompting is possible which has limitations of its own.

2.8 Rotation of titles

It is well known that there is a rotational or position effect in the number of positive reading claims, depending on how early or how late the title occurs in the list of titles. The longer the list of titles, the larger the rotational effect becomes. When single title cards are used, it is normal practise to shuffle them in a similar way that playing cards are shuffled before every interview. This randomises the position of each title and evens out the rotation effect across all titles. However, with a lengthy list of titles, this effect may lead to a general reduction in reading claims.

The NRS in the UK is currently experimenting with using personalized title list. They delete some titles for specific respondents, based on the likelihood of reading it. A simple example to demonstrate this method is that publication with a high proportion of male readers will be asked of all male respondents, but only of a sample of female respondents. The

results of the Female sample will then be used to estimate total female readership.

When prompt pages are used the pages must be rotated between interviews to also even out the rotation effect.

2.9 Learning effect

When more than one publication group (e.g. daily and weekly newspapers) are included in the study, the filter should preferably be asked for all groups before any further questions are asked.

If the groups are separated for the filter question, and further questions are asked before moving to the second and further sections to ask the filter question, respondents learn from the group that is first in the rotation that they only get further questions for publication that they claim to have read during the filter period. This artificially reduce the levels claimed for the second and further groups in the rotation because respondents tend to claim fewer titles to reduce the number of extra questions.

3. WHICH METHODS ARE AVAILABLE TO ESTIMATE NEWSPAPER AND MAGAZINE AUDIENCES ?

3.1 Readership Diary

The use of a reading diary has been tested and reported on at many worldwide readership research symposia. Michael Brown discussed the work done in this regard in his book “Effective Print Media Measurement”

Most of the work was experimental and by using panels recruited for some other purpose than measuring readership. He summarises the status as follows:

“Setting up and running a diary panel is not an inexpensive operation. Excluding the measurement of readership among panellists, primarily recruited for other purposes, there is no case, at present, of the use of a panel in place of repeated sample surveys or continuous interviewing, but there is a considerable number of examples of experimental panels”. Some of the obstacles that one has to bear in mind in a readership panel are the frequency at which the panel is used, which might have an effect on the relative levels of titles with different publication intervals. Also those panellists get used to the questions and fatigue can quickly affect the results. It is different from a peoplemeter panel, where the same remote control unit that has to be used to switch the set on or off or to change channels is used to log-in, with multiple prompting opportunities.

3.2 Telephone interviewing

Telephone research is an acceptable method to obtain readership audience estimates, but it has limitations. Firstly, it can only be used when the telephone penetration is acceptably high. As mentioned earlier, show material cannot be used and prompting can only be done verbally, which has limitations. It works quite well for daily newspapers provided that a balanced sample by day of the week is used, but its not that accurate for publications with a longer publication intervals. It is known that the bulk of reading of a weekly, fortnightly, monthly, etc. publication takes place during the few days immediately after publication. Consequently, interviewing on other days will water down the readership to levels that are artificially low. In a survey such as AMPS, and with different titles published on different days, this method would not work.

3.2.1 Advantages of telephone interviews

3.2.2 Disadvantages of telephone interviewing

3.3 Mail/postal surveys

This research method entails mailing questionnaires to a sample of ‘potential' respondents. It is relatively inexpensive, assuming a sufficient delivery and response rate is obtained.

3.3.1 Advantages of Postal Surveys

3.3.2 Disadvantages of Mail surveys

3.4 Personal face-to-face (f-t-f) interviewing

F-t-f interviewing is by far the most important data collection method and of the 64 national surveys that were reported on at the Worldwide Readership Research Symposium in 2001 in Venice, 46 used the traditional f-t-f pen and paper interviewing, while a further 4 (including AMPS) used Computer Assisted Personal Interviewing (CAPI). Only 22% there national surveys were conducted in other ways, notably by telephone in developed countries with high levels of literacy and high penetration of telephones.

3.4.1 Advantages of f-t-f interviewing

The most important advantage is the fact that prompting can be used. The different ways that can be used to prompt and some of the pitfalls are discussed in Paragraph 2. Nevertheless, most of the shortcomings of survey research can be minimized in a f-t-f interview which is not always possible when the interviewer and respondent are not physically together.

3.4.2 Disadvantages of f-t-f interviewing

Possible interviewer effects/biases is always a risk because people are not perfect. Supervision, testing, back checking and other measures should be used to reduce these possible influences to a minimum. The use of CAPI as such is also making it easier for interviewers and reduces the risk of human errors.

4. WHICH METHODS ARE NOT ACCEPTABLE ?

In most of the queries that SAARF receives regarding audience levels, a reference is made to the number of competition entrants, telephone calls or letters received, or people who made financial or other contributions.

All these results are based on “self-selected” samples and cannot be grossed up or generalised to the universe. Such samples are statistically known as non-probability samples and conclusions only apply the sample. For instance, it is valid to conclude that people who entered for a competition were reading the issue (or one of the issues) in which the competition was announced. Whether they also read other issues cannot be deduced from the fact that they have entered for the competition. The same applies for people who phoned or wrote to the publication. Such information can only be used qualitatively.

Because of the invalidity of such sources as the above to estimate quantitative audience sizes, this problem is discussed at almost every international media audience research conference. Even for more qualitative purposes such as views expressed regarding the quality or liking of the editorial, the results cannot be generalised. It is again the views of some readers and others may not agree. At best, letters, phone calls and competitions can be used to assist in the design of research.

Another source of concern is that audience estimates are sometimes based on unproven, unrelated or unclear assumptions. One such example is to obtain information about the population size, and then assume that a certain percentage would read a specific title.

5. RESEARCH DESIGN

5.1 The Universe

For any research for which a sample is used to estimate certain aspects of the population, it is essential that the universe that is researched be defined and described. Before this is done, it is impossible to design a sample that represents that population.

Many sources of information on the South African population exist. The most detailed source is the latest Population Census of Statistics South Africa or StatsSA. However, censuses are only conducted about every five years, while it can take as long as three years after the census before the results are published. Because of population growth, deaths and migration, the information becomes outdated quickly. Institutions such as the Development Bank of Southern Africa (DBSA), the Human Sciences Research Council (HSRC), the Bureau of Market Research (BMR of Unisa), market research agencies, and many others regularly update population estimates by using mortality, fertility and migration trends. The BMR, for instance, annually publishes a report entitled: "Population Estimates for the RSA by Magisterial District", which provides information for each district by race and gender. The BMR estimates also now provide for adjustment due to HIV/AIDs. All AMPS surveys use the BMR population data for universe purposes. Due to the short-term stability of fertility and mortality trend and the availability of secondary information on migration, these estimates are normally very accurate.

It is recommended that the different available sources of estimates of the population be investigated, and the best one selected to obtain details for a specific target market. Such information would make it possible not only to define the population that forms the target market properly, but also describe them by means of demographic and geographic characteristics. Such information will also make it possible to design a sample with high precision.

5.2 Sample

Apart from the size of the sample, the way in which it is designed is extremely important. In this regard, it is important that a reliable and credible source or sources of population information be used to design the sample. When detailed information is available on the universe or population that forms the target market, it is possible to design a sample that can be grossed-up to estimate the readership of the defined population.

There are a variety of standard sampling methods available that can be used for this purpose. In statistical terms they are referred to as probability samples. A probability sample implies that each person in the universe should have a fixed probability, which can also be calculated, to be included in the sample.

Other techniques, commonly referred to as non-probability sample designs, should be avoided when the aim of the research is to estimate, or quantify the audience. Quota sampling is one such method that should be avoided.

The following probability sampling methods can be considered, either separately or in tandem.

5.2.1 Random sampling

A random sample is basically a sample that is selected by chance. The principle involved is the same as when competition entries are placed in a container and winners blindly drawn.

Statistical handbooks or computer programmes can be used to generate random numbers. If a sample of 1 000 has to be drawn from a universe of 10 000, then 1 000 numbers which fall between 1 and 10 000 are drawn after each unit in the population has been numbered. The units which each of these numbers represent are then identified and they form the sample.

Although this method is statistically regarded as a probability sampling technique, it does not guarantee perfect representativeness, particularly in relatively small samples. The representativeness of the sample can be improved by using stratification or systematic sampling, which are described below.

5.2.2 Stratification

If it is known (or expected) that reading levels will or may be different among different sub-populations, for instance in different towns, cities, suburbs, or for different demographic sub-groups, eg. males or females or different age groups, then the incidence of such variables can be controlled to coincide with their incidence in the population. For example, if the geographical distribution of the target market of a city/town is known, the sample can be designed to represent these suburbs proportionately. Or, if 40% of the population is in the age group 16-24 years, the sample can be designed in such a way that 40% of the sample also falls in this group. This procedure is known as stratification.

More than one variable can be used simultaneously or interlaced to stratify the sample, in which case it is referred to as multistage-stratification, such as in the fictitious example below.

In the above example that covers four suburbs, 10% of the universe lives in Suburb A and is in the age group 16-24. When stratification by suburb and age is used, 10% of the sample will also be 16-24 years old and live in Suburb A, and so on. More variables can be added to the above matrix, in which case the number of cells will increase. The product of the number of categories that is used for the individual variables determines the number of cells. In the above example, 4 suburbs and 4 age groups are used and provide a matrix with 4 x 4 = 16 cells. Care should be taken that the number of respondents per cell should not be too small. Statisticians suggest that the smallest cell should have at least 10 respondents, but preferably 20. The maximum number of cells that can be used, thus, goes hand in hand with the total sample size. In the above example, the smallest cell represents 2% of the population (suburb D, 50+), which means that to obtain the absolute minimum sample size in the smallest cell, a minimum sample of 500 must be used.

5.2.3 Systematic sampling

Systematic sampling is another way to ensure that the sample is spread across the entire population, and not biased towards certain sub-populations and under-representing others.

If a systematic sample of 1 000 has to be drawn from a universe of 10 000, the following procedure will be followed:

Determine the sampling interval :

If 1 000 numbers have to be drawn from 10 000, every 10th number (universe ÷ sample) has to be used to ensure that the sample is spread evenly/systematically between 1 and 10 000. The sampling interval in this case is 10.

Choose a random starting point :

For the above example, the sampling interval is 10, which means that every 10th number must be selected, starting from a random starting point from 1 to 10. This point is chosen following the procedure as is described under random sampling.

Selecting the sample:

Say that 7 is selected as the starting point, then numbers 7, 17, 27, 37, .......through to 9 997 will be used, resulting in 1 000 sampling points as required.

5.2.4 Cluster sampling

Cluster sampling is usually used for economic reasons. If any of the previous three methods are used alone, particularly if a large geographic area has to be covered, the sampling points will be spread individually across the entire area, which would increase the cost. Cluster sampling can be used in such cases to reduce the cost of travel.

Cluster sampling implies that fewer than the required points be selected and that more than one respondent is selected in the vicinity of each point. Such points are referred to as clusters.

However, it should be remembered that cluster sampling decreases the precision of the sample to the sample, if there is a possibility that more than one respondent in the cluster can fall in the same weighting cell. Clusters of 2 are used in AMPS, but a male and a female respondent is selected in each cluster, and because gender is a weighting variable, it is impossible that they would fall in the same weighting cell.

5.2.5 Disproportionate sampling

When it is important to obtain a large enough sample for separate analysis of one or more small sub-populations, for instance to report separately for a community radio station with a relatively small footprint area, that area can be over-sampled. Before reporting the results, such over-sampling should be down-weighted to reflect the correct proportion in the total population. To retain statistical validity, the level of over-sampling should not exceed a ratio of 2:1, particularly in small samples.

5.2.6 Sample size

There is a general misconception that the size of the sample is determined by the size of the universe and that large populations can only be researched by using large samples. Statistically, there is no direct relationship between the size of the universe and the size of the sample required to estimate certain aspects of that universe accurately. The size of the sample is determined by the following factors:

Given the above criteria, the following minimum requirements are usually set:

5.2.7 Substitution

For a variety of reasons, it always happens that not all of the initially selected respondents are available to form part of the final sample. Some respondents will refuse to participate, while others might be difficult to contact. To compensate for this and to ensure that the final sample size and structure will be the same as the selected sample, substitution can be used. However, if the level of substitution is high and if it is not controlled properly, it can bias the results. The following requirements are usually set:

Finally, it is recommended that a statistician be consulted.

5.3 Questionnaire design

A questionnaire that is used to collect valid information is more than just a list of questions. Therefore, it is important that attention be given to not only the formulation of every question, but also to how they are arranged to ensure a logical, simple, understandable and unbiased interview. Furthermore, it is important that the questionnaire be tested and improved in a mini-sample before it is used in the final survey. The following guidelines can be used:

5.4 Fieldwork Standards

Data collection is the most crucial aspect of all research, because mistakes that are made when collecting the information, very often cannot be corrected later. Therefore, it is important that:

6. CALCULATION OF REACH AND FREQUENCY

The aim of media planning basically is to select those titles which reach as many as possible persons of a specific target market a given number of times (called the frequency) in a certain time period (e.g. seven days). Therefore, research that does not provide a reach and frequency estimate will be of little value for buying and selling of advertising time.

It is, thus, important to provide users, not only with an estimate of the number of readers, but also with an estimate of their frequency of reading. In AMPS, we ask about the number of issues normally read out of five for dailies and out of 6 for weekly and longer publication interval titles. The time span covered by the number of issues in the frequency question should, however, not exceed the filter period. For instance, we use a 6-month filter question in AMPS, and for the newly included bi-monthlies we only ask frequency out of three (3x2=6), similarly quarterlies are ask out of 2.

This information is used to estimate the gross rating points (GRPs) of a schedule. If you reach a given number of people on average a certain number of times, the GRP or total number of opportunities to see is the product of the reach and frequency.

7. MARGIN OF ERROR

All research for which samples are used to estimate the behaviour, attitudes, etc., of the population is subject to sampling (or statistical) errors. If probability samples such as the samples that are described in Paragraph 5.2 are used, the size (or margin) of the error can be calculated. The margin of error can be calculated for different levels of accuracy, but in most research, the 95% confidence level is used. If the margin of error is calculated at the 95% confidence level, it means that if 100 similar samples are used, the error would for 95 of them be within the relevant margin, whilst 5 could fall outside this figure.

Two variables determine the size of the margin of error, namely:

The latter refers to the ratio of the proportion of the sample who responded positively (in this instance the readers) and those who responded negatively (non-readers).

The formula (Bravis-Pearson’s product-moment coefficient) for calculating the error is:

s = p x (100-p) x 1.96
            n

Where:

s = Standard Error
p = Penetration (% who read)
n = Sample Size

8. ETHICAL ASPECTS

As in many other countries, the South African Market Research Industry strives for a high standard of research, as well as to protect the interests of the different stakeholders. Most of the leading researchers are members of the Southern African Market Research Association (SAMRA). SAMRA is a professional association and has a code of conduct, to which its members subscribe. The stakeholders are the general public who provide the information, the client who pays for it, and the researcher. The code stipulates the responsibilities of all these parties.

Furthermore, the organisation that represents market research suppliers, Research Suppliers South Africa or the RSSA is an umbrella body to which many research providers belong. This body also sets standards to which its members subscribe.

Individual researchers who are members of SAMRA and research providers who are members of the RSSA are obliged to the best of their ability, to ensure that the research practitioner(s) with which they are associated and the people conducting research on their behalf adhere to this Code of Conduct. More information on SAMRA and the RSSA can be obtained from them at :

P O Box 91879 Tel: (011) 482-1419
AUCKLAND PARK Fax: (011) 482-4609
2006

9. THE ROLE OF SAARF

The South African Advertising Research Foundation was founded in 1974 as a non-profit joint industry committee. SAARF was formed because of a need in the marketing and advertising communities for an independent, comprehensive, unbiased, reliable, regular and technically excellent media audience and products survey. Its purpose is to provide information about the population's use of the media, products, and services so as to enable reliable targeting for advertising purposes. The data are in such a format that it is used, among others, for the buying and selling of advertising time and space in the media and for strategic editorial and programme planning.

Over the years, the All Media and Products Surveys (AMPS®), the SAARF Radio Audience Measurement Survey (RAMS®) and the SAARF Television Audience Measurement Survey (TAMS®) have established themselves as reliable, valid and credible research vehicles. Apart from commissioning these surveys, SAARF also assists media owners, advertisers and advertising agencies in a series of other areas such as training in the use of AMPS data.

SAARF's mission is to serve its members, and interested persons and bodies are invited to liase with SAARF about any aspect related to media audience, market and marketing and advertising research.

SAARF will present and issue a press release when new AMPS results are published. Such press releases will comment on readership levels and trends in general terms, not on the performance of specific titles. Publishers who wish to issue press releases or newsletters with information about their specific titles are invited to consult with SAARF in this regard to ensure that the information is interpreted correctly. SAARF will only highlight significant changes, upward and downward.

The members of the SAARF Print Council, who are representative of the Print, advertising and marketing industries, have also committed themselves to assist SAARF in ensuring that the SAARF readership audience data are used correctly.

10. SOURCES CONSULTED

a. AMPS Technical Report, South African Advertising Research Foundation (SAARF), Johannesburg, 2002.

b. ARF Guidelines Handbook, Advertising Research Foundation (ARF), New York, 1990.

c. SAMRA Yearbook, Southern African Market Research Association, Johannesburg, 2001.

d. Michael Brown, Effective Print Media Measurement: audiences ……… and more, London, 2001

e. Michael Brown, Dear Reader. Some readership questions and some answers, London, 1991

f. Erhard Meier, Summary of Readership Research, prepared for the 10th Worldwide Readership Research Symposium, Venice, November 2001

The above publications can be consulted in the SAARF Library.

 
Copyright 2017 saarf.co.za