Synthetic Controls, Political Axe-grinding, and Historical Foolishness

This time the use of quantitative methods in economics really has gone too far.

The Journal of Political Economy—one of the prestigious “top five” journals in economics—recently published a paper by Phil Magness and Michael Makovi arguing that Karl Marx would have remained a “relatively obscure figure” were it not for the Russian Revolution.¹ Their claim rests on evidence that Marx’s relative influence massively increased only after 1917, supported by a marginally significant p-value of 0.047. But this raises an immediate question: relative to whom?

This question gave rise to the synthetic Marx controversy. To demonstrate Marx’s supposed irrelevance without the Russian Revolution, Magness and Makovi (2023) use Google Ngram data to track changes in his citation frequency, comparing it to that of a group of ostensibly similar authors. This group, often called the donor pool in the literature, forms the synthetic control. In theory, such a pool should be carefully constructed to closely resemble the observable and unobservable characteristics of the treated unit—in this case, Marx. But as Joseph Francis has compellingly explained, the authors fail to meet this standard:

The treated unit is Karl Marx, who Magness and Makovi (2023, 1509) describe as ”an occasionally acknowledged but relatively minor figure between his death and the events of 1917.” Following Abadie’s (2021) guide [on how to use the synthetic control method that developed in 2003], it would therefore appear sensible to include similarly minor figures in a donor pool designed to test the effects of the Russian Revolution on Marx’s intellectual influence, which Magness and Makovi (2023) proxy by his share of n-grams in the Google Ngram Viewer. Similarly minor figures would, it can be assumed, be close to Marx in terms of both 𝑍𝑗 and 𝜇𝑗 [observed and unobserved characteristics]. Nonetheless, without any explanation why, Magness and Makovi’s (2023, 1520–1521, Table 1) donor pool includes Abraham Lincoln, the world-famous American president; Adam Smith, the world’s most famous economist; Aeschylus, the ancient Greek tragedian; Aesop, the ancient Greek fabulist and storyteller; Alessandro Manzoni, the widely known Italian poet and philosopher; Alexander Hamilton, an American Founding Father; and so on. These are the various names whom Magness and Makovi have subjectively decided resembled Marx in terms of 𝑍𝑗 and 𝜇𝑗 [observed and unobserved characteristics] in the pretreatment period.²

Furthermore, when the donor pool is limited to a more comparable list of German authors the results fail to achieve statistical significance. Francis shows that the authors’ broader and less appropriate donor pool artificially lowers their p-value, creating a false impression of precision. In synthetic control analysis, it is not the size of the control group that matters, but the quality of its composition. Poorly matched donors undermine the credibility of the estimate, no matter how large the pool.³

The econometric flaws in this paper are serious enough on their own, but they are only the beginning. The core research question—Did Karl Marx’s relative influence increase after 1917?—is ridiculous, because the answer is obviously yes. It’s like asking whether recent Israeli airstrikes have increased the relative mentions of Iran in the news. Of course they have—but that says nothing about Iran’s prior level of significance. Similarly, the fact that Marx’s citations increased after the Russian Revolution does not mean he was irrelevant beforehand, as Magness and Makovi repeatedly suggest. This conflates rates of change with absolute levels of influence. Their analysis focuses solely on the growth in citation frequency, yet they draw conclusions about Marx’s overall significance—as if a sharp increase in his influence implies that his prior influence was minimal:

Our answer is that Marx’s influence is adventitious, i.e., that it occurred by chance. Marx was not destined by any inherent quality of his writings or personality to become one of the most influential thinkers of all time. Instead, a chance event propelled Marx to a level of fame that he would not have come close to achieving in the absence of that event. Without the Russian Revolution, Marx would likely have remained a niche socialist thinker rather than a leading influence across multiple academic disciplines.⁴

This claim might hold within certain academic circles (though, as Magness and Makovi themselves acknowledge, not in economic ones), but it overlooks Marx’s broader historical role. By the late 19th century, he was already a prominent figure in radical political movements and revolutionary thought. To assert that Marx was “not destined by any inherent quality of his writings or personality” to become influential in a subset of social-scientific scholarship is historically naïve. Who is? Virtually every world-historical figure rises to prominence through a convergence of personal contributions and historical circumstance. Would Oppenheimer have achieved global renown without the U.S. wartime push to develop the atom bomb? Would Burke have become a defining political thinker absent the French Revolution? The question isn’t whether the course of history played a role in Marx’s rise —it plays a role in every historic figure’s rise—but whether Marx’s trajectory was unusually dependent on it. But that question is never deeply considered by the authors, who instead seem focused on linking Marxist scholarship to “the Soviet Union’s troublesome historical record.”⁵ But this is a difficult task for the authors given that their strong trend in favor of citing Marx occurs primarily in immediate response to the 1917 Bolshevik Revolution, not in response to Stalinist terror.⁶

Without a rigorously constructed control group, these kinds of historical counterfactuals drift into pure speculation—more appropriate for a second-rate historical novel than for academic publication. For a sense of what rigor looks like, consider developments in modern economic history which emphasize causal inference through unexpected institutional shocks—such as sudden leader deaths or national partitions. It makes sense to compare the economic development of North and South Korea before and after their partition; it does not make sense to compare the citation levels of Karl Marx and Abraham Lincoln. Yet these superficial comparisons persist, favored by economists who are methodologically careless and historically lazy. By reducing complex historical trajectories to tidy event studies, they erase the rich, qualitative detail that earlier pioneers of the new economic history took so seriously.⁷ And this problem abounds. Recent papers use synthetic controls to study the effect of Pakistan’s nuclear program on GDP growth (by comparing Pakistan to other similar countries that didn’t develop a nuclear weapon)⁸; the effect of the Russian Revolution on economic growth⁹; and the impact of the Iranian Revolution on development. The latter paper came to the following conclusion:

If Iran had not faced such a revolution, the accumulated per capita GDP would have been $6,479 higher, which amounts to an average annual loss of about $2,159 over [the three months between the Iranian revolution and the beginning of the Iran-Iraq war.]¹⁰

But consider the absurdity of this measurement. The author attempts to estimate the effect of the Iranian Revolution over just three months—because any longer time frame would be confounded by the onset of the Iran–Iraq War. Yet this short window merely underscores a deeper point: revolutions are often followed by retaliatory policies, economic punishment, and exogenous shocks. How can one assess the economic consequences of the Russian Revolution without acknowledging the subsequent invasion by the American, British, French, and allied troops? How can one evaluate the outcomes of the Cuban or Iranian revolutions without addressing the severe sanctions imposed in their aftermath? One might argue that such retaliations are a consequence of the revolutions—but then these synthetic control papers are tackling a fundamentally different issue. They are positing not that revolutions themselves cause relative economic decline or stagnation, but that they invite coercive and vindictive responses from hostile superpowers.

Another example comes from a recent paper on the economic effects of Hugo Chávez. The authors construct five different synthetic control groups to evaluate Venezuela’s performance across five quality-of-life metrics. For income comparisons, Venezuela is compared with oil-exporting countries like Iran, Canada, Peru, and Mexico. Unsurprisingly, the results show that Venezuela’s relative growth slowed under Chávez, especially during periods of declining oil prices—an outcome that is both directionally correct and unsurprising given Venezuela’s far greater dependence on oil (with the exception of Iran).¹¹

But the paper’s findings on poverty are more striking. During Chávez’s presidency, Venezuela recorded some of the largest percentage declines in its poverty rate anywhere in the world. Yet the authors claim that Venezuela would have achieved similar reductions without Chávez. How? By constructing a synthetic control using only two countries and assigning 80 percent of the weight to Colombia. Conveniently, at the time Chávez came to power, Colombia was just beginning to recover from its worst economic downturn in decades, during which poverty had spiked. The author’s null result is thus largely driven by Colombia’s rapid reduction in poverty from a temporarily elevated baseline.

Assessing the extent of the bias in these results is difficult, because the synthetic controls method does not simply compare the poverty levels of Venezuela and Colombia side-by-side. Instead, it relies on an event study that models changes in the rate of poverty reduction in both countries. Typically that’s a sound practice, but in this context the parallel trends assumption (that is required for an unbiased event study) is entirely neglected so the event study method only obscures just how much faster Venezuela’s poverty fell compared to Colombia’s, and makes it nearly impossible to gauge the true impact of Chávez on poverty alleviation.

The synthetic control groups used by (Grier & Maynard, 2016). Source: Grier, Kevin, and Norman Maynard. "The economic consequences of Hugo Chavez: A synthetic control analysis." Journal of Economic Behavior & Organization 125 (2016): 1-21.

By now, the underlying problem should be clear: authors have wide latitude to select and shape their control groups, often with minimal justification—and academic journals are frequently willing to accept these choices at face value. This methodological flexibility makes synthetic controls a tempting tool for political axe-grinding. It’s unsurprising that the authors of the “synthetic Marx” paper are long-time critics of Marx and Marxist academics, or that the lead author of the Chávez paper is affiliated with the Free Market Institute and has consistently championed the Washington Consensus.¹² These affiliations don’t automatically discredit their findings, but they do raise significant concerns when the same flexible method is repeatedly applied to politically-charged questions by authors with clear ideological commitments.

But the problem with synthetic controls goes even deeper. By reducing complex historical developments to simple event studies, economists can conveniently sidestep the messy, qualitative richness of the past.¹³ In this framework, Karl Marx and Abraham Lincoln can be made to satisfy the parallel trends assumption. Comparing the relative income levels of Canada and Venezuela is rendered methodologically sound. And the economic cost of building a nuclear weapon can be estimated down to the last dollar in terms of foregone growth. This is not a breakthrough in causal inference—it’s an exercise in historical fantasy disguised as rigor.

1. Magness, Phillip W., and Michael Makovi. "The Mainstreaming of Marx: Measuring the effect of the Russian Revolution on Karl Marx’s influence." Journal of Political Economy 131, no. 6 (2023): 1507-1545.

2. Francis, Joseph. "A p-Hacker’s Guide to the Synthetic Control Method." Unpublished paper, v1. 0, June 2 (2025).

3. Ibid.

4. Magness, Phillip W., and Michael Makovi. "In Defense of Synthetic Karl Marx: A Reply to Joseph Francis." Econ Journal Watch 21, no. 2 (2024): 385-414.

5. Magness, Phillip W., and Michael Makovi. "The Mainstreaming of Marx: Measuring the effect of the Russian Revolution on Karl Marx’s influence." Journal of Political Economy 131, no. 6 (2023): 1507-1545.

6. A pedant might night that the Soviet Union was not formed until 1922.

7. I’m currently reading Robert Fogel’s Without Contract or Consent: The Rise and Fall of American Slavery which is a great example of economic history done right.

8. Mayberry, Anthony A. "The economic cost of a nuclear weapon: a synthetic control approach." Defence and Peace Economics 34, no. 6 (2023): 747-766.

9. Korolev, Ivan. "How Could Russia Have Developed without the Revolution of 1917?." Annals of Economics and Statistics 144 (2021): 75-112.

10. Hasancebi, Serhat. "The economic cost of revolution: The Iranian case. A synthetic control analysis." The Singapore Economic Review 67, no. 01 (2022): 267-287.

11. Grier, Kevin, and Norman Maynard. "The economic consequences of Hugo Chavez: A synthetic control analysis." Journal of Economic Behavior & Organization 125 (2016): 1-21.

12. See https://www.kevinbgrier.com/research particularly his paper Grier, Kevin B., and Robin M. Grier. "The Washington consensus works: Causal effects of reform, 1970-2015." Journal of Comparative Economics 49, no. 1 (2021): 59-72.

13. There are, of course, times when these “simple event studies” enrich our understanding of the past! But these are instances where historical developments create natural experiments that are just waiting to be analyzed by an economist’s methods (like sudden leader deaths). Conversely, our understanding of the past is distorted when we force our modern econometric methods upon a begrudging past, ignoring whether things like the parallel trends assumption holds, in order to make a clever comparison.