ErinMedia Exposes Flaws In Nielsen Media Methodology.

Television ratings provider erinMedia published information that exposes fundamental flaws in Nielsen Media Research’s (NMR) television program ratings, based on NMR’s June 23 press release, which measured African American viewership in Washington, D.C., and attached dual panel data, which compares Local People Meter (LPM) and set meter/diary ratings.

The NMR release and dual panel data purported to show that data generated by NMR’s LPMs were more accurate than NMR’s set meter/diary-based data, because the LPM data reflected higher viewership of African American themed programs. However, analysis conducted by Dr. Michael Vinson, erinMedia’s Senior Research Scientist, using industry-accepted methods including the time-honored Student t-Test approach, reveals with a certainty in excess of 99% that at least one set of Washington, D.C. data, if not both, are “systemically flawed, and apparently not both sets of data are being derived from the same representative population.”

erinMedia’s detailed analysis was conducted after a simple examination of the data obtained through NMR’s website revealed an average discrepancy in excess of 40% between ratings of the top 30 shows among African Americans in Washington, D.C., between April 28 and May 25, 2005, compared with set meter/diary-based results during the same timeframe. These massive and unexplained swings led to further examination, which revealed that 90% of these programs experienced a dramatic decrease in audience size when compared to LPM measurement.

“NMR’s news release unwittingly reveals, with 99% certainty, that at least one set of data, if not both, is systemically flawed,” said Frank Foster, president of erinMedia. “The two measurement systems are simply not pointing to the same viewing picture and it is highly unlikely that both data sets accurately represent how African Americans view television in Washington, D.C. With more data, I may come to a different conclusion. Keep in mind that we are talking about the most popular television shows; if we move down through the top 200 programs, we would expect to see even greater swings between the data sets.”

“If we accept NMR’s own assessment that LPMs are the ‘gold standard’ that provides the superior methodology, then, at least as it relates to African American viewers in highly populated African American metropolitan areas like Washington, D.C., the continued use of set meters may be severely over-reporting the results,” said Frank Maggio, Chairman and Chief Executive Officer of erinMedia. “Can it really be, as Nielsen is implicitly suggesting, that a sponsor placing an advertisement to reach the African American community during a program such as ‘Oprah’ was actually getting almost 44% less of the audience, as shown by Nielsen’s set meter/diary ratings? At a time when advertisers are re-evaluating their TV spending anyway, this discovery represents a potential loss in confidence that the medium can ill afford.”

The other logical possibility supported by the analysis, Maggio noted, is that both of the NMR methodologies cited in its June 23 release are fundamentally flawed. “And remember, the data NMR released on June 23 are the data that it considers the most reassuring and helpful to its case. What does the rest of the data show? Only NMR knows for sure.”

To address the massive discrepancies and statistical uncertainties exposed by NMR’s own selective release of data, Maggio said, NMR needs to release all its dual panel data and allow advertisers, the television industry and expert observers to make up their own minds. “With this new data revealing that one or both of NMR’s metered methodologies are fatally flawed at accurately measuring the African American audience in our nation’s capital, the only way for NMR to address the industry’s concerns is to make ALL of its dual panel viewership data publicly available, immediately,” Maggio said. “If it’s self-regulation the industry wants, then that is the only solution. Without that level of openness and scientific scrutiny, we are essentially asking for government intervention.”

Calling the limitations of NMR’s data “startling,” Maggio said the discrepancy provides timely and urgent evidence that a change to more inclusive methods of measurement must be immediately adopted by the industry.

“Given that the technology already exists to utilize digital set-top box data to directly measure the second-by-second viewing habits of more than 25 percent of the U.S. population, it’s inexcusable that NMR is asking the industry to rely on its selective releases of data, which in turn are based on a miniscule sample of less than 7,800 households nationwide,” Maggio said. Based on this new NMR data, erinMedia and reactive cable network ReacTV also announced the launch of an ongoing “watchdog” campaign as part of their joint “WeShouldALLCount” initiative. The campaign will dig deeper into NMR’s press releases, particularly as it relates to the ongoing LPM rollout across up to 210 designated market areas (DMAs). It will expose program ratings fluctuations and attempt to quantify the levels of advertising dollars potentially being misspent on viewers previously counted by NMR but later found, by NMR’s newest and “best” technology, not actually to be watching. The site will also call on the industry to look beyond NMR’s headlines and spin, while considering a significantly more accurate and sophisticated ratings system.

To view charts CLICK above on ‘More Images’.