The attached material is posted on regulation2point0.org with permission.THE OBERHOLZER-GEE/STRUMPF FILE-SHARIG ISTRUMET FAILS THE LAUGH TEST STA J LIEBOWITZ SCHOOL OF MAAGEMET UIVERSITY OF TEXAS AT DALLAS MAY 2010 I examine the key instrument (German kids on vacation) used by Professors Oberholzer-Gee and Strumpf. Their measured relationship between the instrument and the variable that it is instrumenting for, American downloading, is seen to have outlandish implications, indicating an important error. The instrument is also shown to be related to American record sales, contrary to the requirements of their analysis. The data set used by O/S is biased, considerably overstating the share of German files. Finally, I demonstrate that the instrument must have a de minimus impact on American downloading (and thus American record sales) negating its potential usefulness and implying that their approach could never have worked.1 [In February of 2007 the Journal of Political Economy published an influential article by Professors Felix Oberholzer-Gee and Koleman Strumpf (O/S) examining whether file-sharing by Americans reduced sound recording sales in the United States. The JPE website currently lists their paper as the most heavily downloaded paper on the site and Google Scholar shows that the citations to this paper are mounting in an impressive manner. I have previously (2007) examined this paper in some detail.1 In my earlier work I noted that numerous factual assertions found in the O/S paper were incorrect and that several tests made to support their main regressions were improperly conducted, leading to erroneous conclusions. I suggested that the pattern of error was so strong that a reader should be wary of believing anything in their paper. I submitted that paper to the JPE, but although an independent referee recommended publication, Oberholzer-Gee and Strumpf, in the guise of an anonymous referee, recommended against publication and the editor went along with them. Their strongest argument, in fact their only real argument, was that I did not criticize their main regressions and so I couldn’t prove their main result was wrong.2 Of course, I couldn’t directly examine or try to replicate their main result because they refused to make their data available. Although they have told two reporters that they could not provide their data due to a non-disclosure agreement, they were unwilling to provide a copy of that agreement.3 Further, their claims are contradicted by their former public statements where they said that the data would be made public sometime in the not-too-distant future.4 A few months ago I was invited to a conference in Vienna where Oberholzer-Gee and I were to be the only two presenters in a morning session. In anticipation of the quasi-debate, I reread the O/S paper. I had always found their main instrument, the number of students on vacation in Germany, questionable, but I had never looked at the details of its construction. I also discovered that right out in the open, in their own words, was a central empirical finding that was quite preposterous. These discoveries have led to this new comment which does focus on the main regression analysis. I refer to their preposterous empirical result as a “smoking gun” since it provides unambiguous evidence that something is terribly amiss in their paper. As was the case in my first comment, I find numerous new instances of variables constructed without documentation, errors in simple statistical presentations, misreported facts, and questionable assertions. Alas, the quasi-debate will not take place as anticipated. Oberholzer-Gee’s presentation has moved to the afternoon. Still, I am sure there will be fireworks. ] 1 Liebowitz, Stan J., How Reliable is the Oberholzer-Gee and Strumpf Paper on File-Sharing? (September 2007). Available at SSRN: http://ssrn.com/abstract=1014399. 2 More information on the history of that submission, including their response (their referee report) can be found at Liebowitz, Stan J., Sequel to Liebowitz's Comment on the Oberholzer-Gee and Strumpf Paper on Filesharing (July 5, 2008). Available at: http://ssrn.com/abstract=1155764. 3 Go to http://chronicle.com/article/Dispute-Over-the-Economics-/989/ and search for “confidentiality” 4 Go to this website and search for “share numbers”: http://web.archive.org/web/20040804095120/http://cdbaby.net/fom/000004.html .2 Who are you going to believe, me or your own eyes? Chico Marx in Duck Soup 1. Introduction Professors Oberholzer-Gee and Strumpf (O/S) compare sales of individual sound recordings to the unauthorized downloads of those sound recordings. It is clear, as O/S acknowledge, that comparing the sales of songs to the number of unauthorized downloads of those songs suffers from a severe simultaneity problem because the popularity of a song influences both its sales and downloads. In an attempt to overcome this simultaneity problem, O/S chose a dazzlingly recondite instrument for filesharing—the number of students on school holidays each week in Germany (or Number of German Students Vacationing: NGSV) during the 17-week period (the last third of 2002) of their analysis. They refer to this as their “most important instrument” because it allows them to use the time element in the panel data analysis. Using this instrument, O/S conclude that file-sharing has no impact on record sales.5 What rationale do Professors Oberholzer-Gee and Strumpf provide for their use of NGSV? First, they note that if file-sharing influences sound recording sales then a shock to the amount of American file-sharing will induce a change in American sound recording sales. Next, they claim that the amount of American file-sharing responds to changes in the number of computers using file-sharing services. Finally, they look for a shock to the number of computers on file-sharing networks and claim to have found one in NGSV. O/S assume, although without any supporting evidence, that German students are more likely to keep their computers attached to file-sharing networks during