Nielsen & Local People Meters.
May 22, 2004
Nielsen Media Research released the following interim report on the results of an ongoing MRC audit of Nielsen’s Local People Meter service in New York. While the MRC accreditation process is continuing, certain portions of the audit report have been leaked to the press, and opponents of the LPM service have launched an aggressive advertising campaign of disinformation. Nielsen responded to inquiries as follows:
More Accurate TV Ratings
* Nielsen Media Research, the TV ratings company, is in the process of introducing more accurate and reliable ratings technology to local markets, including New York, Los Angeles and Chicago.
* This technology involves Local People Meters, which are boxes that rest on top of your TV and automatically record what every person in the house watches. Local People Meters do a better job of tracking ratings than traditional paper diaries, which require people to remember to write down what they’ve watched after they’ve watched it.
Independent Auditing
* Nielsen’s ratings services are independently audited on an ongoing basis by the Media Rating Council (MRC). Recently, the MRC raised several concerns about Nielsen’s Local People Meter service in the New York market. As we always have, Nielsen is working with the MRC to address the four areas – out of the 85 audited areas – where Nielsen was found not to be in material compliance with MRC standards.
* Some information in the confidential preliminary audit was leaked to the press, so Nielsen is taking the unusual step of publicly addressing the concerns raised while we continue to work with the MRC to resolve them.
Major Issues Raised by the MRC Audit
* The MRC found a discrepancy on race in two of the 30 audited homes (of which twelve were identified as African American). In both cases, this was due to differences in the way different people in the household reported their race. Racial identification is a complex issue for many American families, particularly mix-ed race households, and we are working with the MRC on an alternative method to ask this question in mixed race households. We have also double-checked all mixed race
households in the sample to ensure they are properly identified.
* In two of the 27 Latino households that were audited, the audit found a difference in the household language of the residents. In one case, Nielsen’s interviewer mistook a Brazilian household for Argentinean- an error that we will correct through intensified training of field staff. In the other household, a change in answer by the owner/renter failed
to be recorded by Nielsen’s representative.
* The audit found that fault rates — the percentage of homes with communications errors, unplugged meters or improper usage of the meter — rose in some areas. At one point, the fault rate for African American households temporarily reached 25%, compared to an overall average of 16%. Some of this may have been due to the active campaign against Nielsen in communities of color. We have added nine field representatives to the New York market to stabilize the fault rates at ordinary levels. Since late May, fault rates are have been well within target levels and over the past seven days (June 10-16) fault rates for African Americans have averaged 18%.
The Most Accurate and Reliable TV Ratings Ever
* Nielsen’s New York Local People Meter service is currently providing the most accurate and reliable TV ratings that have ever been reported for this marketplace. We are working hard to address all issues raised by the MRC and are confident that we will receive accreditation for this sample shortly.
For more details, please see the attached summary.
Nielsen’s Response to Issues Raised by the Media Rating Council
On June 3, 2004, Nielsen Media Research launched a Local People Meter (LPM) service in the New York market for the measurement of local television audiences.
Nielsen’s national and local services are audited annually by an independent auditor Ernst & Young (E&Y), and accredited by the Media Rating Council (MRC) — a body of television and cable networks, station groups, advertisers, agencies and media buyers.
As a result of the recent audit of the new NY LPM service, the MRC withheld accreditation, citing certain outstanding issues. Nielsen will continue to work with the MRC so that the NYC LPM service receives accreditation as soon as possible.
Nielsen goes into the accreditation process from a position of considerable experience in television audience measurement. For example:
* Nielsen has been measuring TV audiences in the US for 54 years
* The Nielsen ratings are the currency for more than $60 billion in ad sales in 2004.
* The US television industry is the best measured most effective medium for commercial expression compared to any other media in the world.
* Nielsen is independent from the measured media; we are not owned or controlled by the media outlets; we provide objective, independent estimates of audience viewing behavior. We have no financial stake in the outcome.
* Nielsen measures the largest, most diverse, most competitive, most technically challenging television marketplace in the world. And we issue reports on that marketplace every day. No other TV ratings service, anywhere, can match that.
* The Nielsen People Meter samples have the highest levels of cooperation and are the most representative of the populations they measure than any other TV research company in the world.
Before commenting on what the MRC audit report did say, here’s what it did not say:
* The audit said nothing about undercounting persons of color.
* The audit said nothing about flaws in the People Meter technology.
* The audit said nothing about delaying the rollout of Local People Meters.
Here, specifically, is what the audit report did say:
* Out of approximately 85 categories audited, 4 items were tagged by the MRC. They were:
— Cable TV information (15 of 602 homes audited were affected by cable rebuilds);
— Race identification (discrepancy in 2 homes out of 30 audited);
— Ethnic identification (discrepancy in 2 of 27 homes audited for ethnicity);
— Incomplete documentation (applies to more complete disclosure in Nielsen’s standard reports).
* The MRC also expressed concern about higher-than-normal fault rates.
Cable Information
The audit cited a lack of sufficient quality control in identifying changing cable lineups due to “cable rebuilds” (e.g., when cable systems replace analog with digital service resulting in significant lineup changes).
In these cases, Nielsen is supposed to contact the affected homes within 60 days to determine whether there has been a change in program line-up.
* The audit found that 15 of 602 cable homes audited were affected by a cable rebuild situation.
* Of these 15 homes, the audit found that five homes were not successfully contacted within the proscribed 60-day period despite numerous attempts by Nielsen to contact the homes. (Subsequent contacts by Nielsen have shown that four of the five households showed no change in cable lineups, and the fifth home has not yet been successfully contacted).
Race and Ethnicity
Two findings by E&Y concerned the appropriate recording of Race of Owner/Renter information, and Owner/Renter Hispanic Identity information. In each case, the category reported by the household and documented by Nielsen was different from that documented during the audit.
Race of Owner/Renter — Of the 30 households audited — 12 were classified as “Black” by Nielsen — the audit collected different information for 2 households.
a. In the first sample home, the female of the house identified herself as the owner/renter and each family member, including herself, as “Black.” In the audit, the male of the house was interviewed. He identified himself and their child as “Black” but identified his wife as “Other/Pacific Islander.”
b. In the second household, the female of the house identified herself as owner/renter and each family member, including herself, as “Black.” In this instance, the household was also classified as Hispanic, so Country of Origin was asked as well. She said she was from Puerto Rico, and identified herself as “White.” But all the other members of the family were “Black,” she said, and they were from the Dominican Republic.
Hispanic Identity of Owner/Renter — In 2 of 27 households audited in this category, there was a discrepancy between the categorizations of Nielsen and the audit.
a. The audit showed one household to be Brazilian, and the language used to be Only English, since both household members speak mostly Portuguese and English, but do not speak Spanish. Nielsen’s records show this household to be classified as Argentine, speaking mostly Spanish. (Nielsen believes this to be a straightforward example of interviewer error requiring improved training and monitoring.)
b. In the second house, the owner/renter initially responded, “I don’t know” to the auditor, since her parents were “white and black and her grandfather was Cuban.” When pressed by the auditor for a yes/no answer, the owner/renter replied “no, I’m African American.” Nielsen field records show that the household member changed her answer for the Nielsen representative from Hispanic to non-Hispanic, but that the change was not recorded in the field data base. This represents a failure to record a change in ethnicity in the field system. (This shows how complex it can be to gather accurate data from multi-racial, multi-ethnic, or Black-Hispanic households.)
Documentation
Nielsen was cited as non-compliant for failure to disclose certain weighting and data adjustment procedures in its local reference supplements. These disclosures are not limited to NY LPM, but to all or most accredited samples that Nielsen maintains.
Faulting
Fault rates are the percentage of homes with communications errors, unplugged meters or improper usage of the meter. While not an instance of non- compliance, the MRC expressed concern about Fault rates that characterized the NY LPM sample between April 1 and May 15. A people meter “faults” for three reasons:
* Technology — Something goes wrong with the equipment.
* Behavior — People don’t push their buttons, unplug the equipment or buy a new attachment (DVR, VCR, video game, etc.)
* Environmental — Power outages
Some homes fault more than others. Larger homes with multiple TV sets and multiple People Meters, tend to fault more frequently than homes with fewer TV sets.
The MRC has created a “Faulting Committee,” and Nielsen is working with this committee to find ways to reduce faults.
The audit report identified a progression of higher and higher fault rates for the New York LPM sample leading up to the original launch date. At one point, the fault rate for African American households temporarily reached 25%, compared to an overall average of 16%. There appear to be two possible reasons why fault rates might have climbed higher during the month of April, one procedural, the other “environmental.”
1. Procedural: the sample is presently over-installed on African-American homes. Because we prioritize our field visits based on in-tab representation, a category that is over-represented will receive lower field priorities if homes in that category fault but still maintain a close relationship to the universe estimates when evaluated on an in-tab basis (which is currently the case for Black representation). We are considering a modification of this process, and will discuss the issue with the MRC and our clients shortly.
2. Environmental: This could be explained by some unusual events occurring during April in the New York market. Starting in late March, the News Corp.-backed advocacy groups took a strong public position against the use of People Meters. Broad-based mailings, telephone messaging, television coverage, and possibly door-to-door canvassing may have led respondents to become less cooperative. We are in the process of assessing the impact of the controversy on our sample quality.
We have added nine field representatives to the New York market to stabilize the fault rates at ordinary levels. Since late May, fault rates are have been well within target levels and over the past seven days (June 10-16) fault rates for African Americans have averaged 18%.
Other Concerns by the MRC:
* Scheduled Calls: (Nielsen procedures require that every People Meter home must be contacted by Nielsen once every six months.) Scheduled visits did not occur in 6 of 61 cases. The MRC called for tighter controls to improve this rate. Nielsen agrees and will be providing procedure changes to ensure improvement.
* Bypassed technology: (Homes with TV equipment that Nielsen can’t meter are bypassed.) This affects the current set meters and people meters in all markets although NY may have a higher rate of technically difficult homes. Half of all technically difficult homes are PVR/DVR homes for which measurement solutions will be available next April. MRC requests that Nielsen prominently disclose this in reports.
* Metering Issues: (In some instances cellular technology can interfere with Nielsen’s metering equipment.) This affects current set meters and people meters in all markets. Three conditions were noted in the audit report, and the MRC asked Nielsen about the status of installing shields on meter probes to protect against cellular interference.
* Characteristics of Refusing Basics: (Our procedures require that alternate households match certain characteristics of the “basic” (first choice) home which has been randomly selected. These matching characteristics include cable status, presence of child, etc.) In 522 alternate households audited, 15 households had changes in the characteristics of the basic refusing households without a proper explanation.
* Alternates Passed: MRC requested that we report the number of households that we contact before recruiting a substitute household. Nielsen will include this information in its published reports by the end of this year.
* Sample Distribution: Asians are under-represented by 3 points and HH Size 2 is over by 3 points. Nielsen has implemented special Asian procedures in the NY market. However, the demographic representation of the NY LPM sample far exceeds the representation of the current set meter sample.
Summary
Nielsen is in the process of addressing the non-compliance issues cited by the MRC as well as the other areas of concern. It is important to note that surfacing problems with the sample is part of the normal, ongoing effort to get as accurate data as possible. Nielsen plans to:
* Maintain fault rates at their current, improved levels
* Complete a re-audit of Race and Ethnicity;
* Review collection procedures;
* Broaden our disclosure policies as indicated.
Nielsen has asked the MRC to convene another accreditation meeting for mid-August at which point we will have addressed all items raised.



























