"Challenging the Empirical Contribution of Thomas Piketty's Capital in the 21st Century" by Phillip W. Magness and Robert P. Murphy is one of the first deep reviews of the facts that I have seen. I haven't read it yet, but the abstract looks promising:
Thomas Piketty's Capital in the 21st Century has been widely debated on theoretical grounds, yet continues to attract acclaim for its historically-infused data analysis. In this study we conduct a closer scrutiny of Piketty's empirics than has appeared thus far, focusing upon his treatment of the United States. We find evidence of pervasive errors of historical fact, opaque methodological choices, and the cherry-picking of sources to construct favorable patterns from ambiguous data. Additional evidence suggests that Piketty used a highly distortive data assumption from the Soviet Union to accentuate one of his main historical claims about global “capitalism” in the 20th century. Taken together, these problems suggest that Piketty’s highly praised and historically-driven empirical work may actually be the book’s greatest weakness.Comments on the paper welcome. If I get a chance to read it I'll post some.
"opaque methodological choices"
ReplyDeleteConsidering he put all of his data and documentation online, it seems pretty transparent to me. I wish more finance papers were so opaque! [noting that JC is an admirable outlier on the transparency front: http://faculty.chicagobooth.edu/john.cochrane/research/Data_and_Programs/]
If Figures 3 and 6 are supposed to be smoking guns, the rhetoric seems a lot hotter than the substance.
Addressed on p. 8: "While Piketty has received praise for following the increasingly common practice of making his source data available to other scholars by posting them online, this act of courtesy should not be mistaken for a license to impute validity to their contents."
DeleteNo posting data and documentation certainly doesn't make that data true, but it is the very opposite of opaque.
DeleteYeah, cause that documentation is always soooooo precise and clear:
Delete"Piketty’s derivation of Figure 5.8/12.4 poses a challenge as it is sourced and annotated in ways that are even less transparent than his U.S. wealth inequality example. His technical appendix annotates Figure 5.8, the first iteration of this graph, by noting:
The series used to construct figure 5.8, replicated in the book on p.196, are available in Table S12.4 (see appendix to chapter 12). All the details about the assumptions on which the series are build, especially for the period 2010-2100, are specified in the book, as well as in the corresponding excel file. (Piketty, Technical Appendix, p. 34).
When one turns to figure 12.4 and the corresponding excel Table S12.4, the accompanying note sends the reader on a circular return to the original note for Figure 5.8: “These estimates are based on assumptions already presented in chapter 5 and in the appendix to chapter 5” (p. 71)"
Let's not let the perfect be the enemy of the good. Let's identify the best and most clearly annotated work of historical economics and compare Piketty's to that.
Delete...It's quite possible the comparison would be to Piketty.
Have you read the critique? From Magness/Murphey pg 15 "reveals that most of its root data is actually taken from the Kopczuk-Saez study, yet it is also “augmented” and extended through 2010 with other studies based on the SCF as well as a number of opaque adjustments that are simply hard-coded into Piketty’s source files. Unfortunately neither Piketty’s annotation nor the supplemental document that he released in response to data criticisms contain an adequate or transparent explanation of how he performs these “augmentations”.
DeleteM/M are saying he is opaque because while he supplies the numbers that he uses them it is often in question where he gets them from, and he offers no explanation of why he chooses one series or another or why he switches between them- "Without citing the change and in fact suggesting otherwise" also pg 15.
It seems, Charlie, that you're going to pretty silly lengths to defend Piketty. He arbitrarily and without explanation adds and subtracts seemingly random numbers from individual data points throughout his data. Further, he refuses to address any of these issues and simply pooh-poohs them as "typos." This is opacity of the highest degree.
DeleteRe-read for clarity. Methodological opacity says nothing about source transparency. If I use birth sign as a predictor of income in my model, no amount of data disclosure will make more transparent my methods.
ReplyDeleteGlenn, I don't see what is opaque in your example. You would have to us birth sign as a predictor of income and hide it in some way to be opaque.
DeleteThe example has little to do with the actual authors complaint also. Piketty expressly is writing a book combining a lot of imperfect data sources and expressly making decisions about how to combine them. He provides pages and pages of spreadsheets as well as pages and pages of spreadsheets of documentation. Not to mention, the citations that link to the original papers that build/use the data sources.
The authors say that they found a case, where Piketty made an adjustment and/or combination of data sources and doesn't document why. Even if the authors are right that they found an instance that isn't documented, I still contend that Piketty has had incredible transparency in sources and methodology--and contrary to the authors view--the praise he has received from imminent economists on this front has been well-deserved. Standard practice would be just linking to the original citations with no links to the underlying data and no documentation on how it was combined other than as described in the original citations.
[Additionally, to my eye, the "corrections" they make in figures 3 and 6 look pretty tame.]
There's something strawmanish about your portrayal. Figure 3 is openly acknowledged as just one of many smaller errors uncovered - "At this point it is fair to query whether the aforementioned issues, even if acknowledged as problematic to Piketty’s historical narrative, pose a larger challenge to his theoretical argument." Figure 6 isn't even theirs - it's a reprint of the FT story that made a splash last year.
DeleteSo why are you focusing on those while you leave out the real meat of the paper which goes after Piketty's main charts on inequality in the U.S.?
Sorry, Charlie, let me try be more clear.
DeleteThe authors attack Piketty for not being transparent in his methods. You reply in Piketty's defense that he has made his data publicly available (a fact the authors are clearly aware of, as his data factors heavily in their critique).
This is, as Francisco said, a straw man. The authors never argued that Piketty failed to share his sources. They argue that his use of that data in the context of the book is controversial (unorthodox), requiring defense, yet none is offered. The authors themselves argue that he cherry picks in support of a pre-ordained narrative, and this is an argument I find compelling given the evidence they present.
They may still be wrong, but your critique - while easy and convenient - doesn't make any sense (in my humble opinion).
"They argue that his use of that data in the context of the book is controversial (unorthodox), requiring defense, yet none is offered."
DeleteGlenn, do you agree with this claim when made holistically about the book? Taken holistically, do you think Piketty providing the hundreds of pages of documentation, the source files for all the figures (and supplemental figures), and the underlying data with which the figures were constructed amounts to opaque methodological choices?
Further, do you agree with the authors that accumulating, sourcing and making transparently available all of this data (for which Piketty has been praised) is actually the book's weakest contribution?
I'm not sure I follow - is it promising because it claims to find errors, or promising for some other reason?
ReplyDeleteThe paper itself is a promising because it essentially dispels much of Piketty's statistical claims. This be received much attention because he appeared to have a silver bullet for inequality. However, upon inspection, the data do not support his claims.The paper points out ways in which Piketty not only flagrantly politicizes his interpretation of history (taxes were raised only under Roosevelt, but not under Hoover - false. Minimum wages were raised under Carter/Clinton, but not under Bush/Reagan - false), but legitimately makes up data in order to make bold claims such as having "130 years worth of [tax] data," "inequality has been rising at an alarming rate over the past 30 years," (a statement which he is unable to support statistically), et al. He has violated some pretty standard statistical practices in order to make grandiose claims about the nature of inequality, tax policy history/effectiveness, and capitalism.
ReplyDeleteIn the midst of halving extreme world poverty over the last 25 years, the recent development of new economic fields (i.e behavioral economics), and a challenging demographic/economic shift taking place in two of the world's largest economies (aging workforces); he is distracting the world from some very interesting and meaningful discussions about economics in the name of politics. So this review is interesting because with all good research, it requires time and thorough examination in order to critique the claims made by the research. Debates on research usual concern the ideas drawn from the data or the data itself. For some reason, Piketty has just made stuff up or misled the reader, and the places where he makes things up lie at the basis of his arguments. In other words, this book should not have been taken as seriously as it was and we should be studying topics we can discuss with more certainty.
Murphy is an Austrian, i.e. he has a) never done proper empirical work like Piketty for years and b) naturally has issues with the positive effects of equality upon democracy, social stability and so on so he naturally tries to smear Piketty.
ReplyDeleteFurthermore you can pick any data source and see the inequality has been rising since the 80s and is about to reach 19th century levels.
You're right. How dare they smear Piketty by pointing out all of his factual inaccuracies
DeleteSo your defense of Piketty is an ad hominem attack on the second author? How crude.
DeleteIt was easy to predict the politics of the authors prior to googling them.
ReplyDeleteMy last attempted posting appears to have disappeared into the ether, I assume censored, so I will try again.
ReplyDeleteIt is possible that Magness & Murphy are absolutely correct and Piketty is blatantly dishonest, an incompetent, a liar and a charlatan but I really would not bank on it. A few months ago the FT published their attempted takedown, still easy to access via Google et al despite the polite but humiliating point by point rebuttals by Piketty amongst (many) others. The most embarrassing point for the FT was their reliance on the ONS's UK Wealth Survey despite that resource carrying a health warning from the ONS itself regarding its limited statistical reliability. The FT has not apologised but recently awarded "Capital" their business book of the year award.
Something tells me that in the fullness of time Magness and Murphy will be embarking on the same walk of shame.
I will be fascinated to see whether the "Grumpy Economist" censors this as well, but maybe websites like this are meant to be private spaces for free market fundamentalists who would rather not know just how discredited their beliefs are.