Last week the IMF published a paper that it claims shows a “strong association between gender inequality and income inequality“, along with an accompanying blog post and tweet. However, the study presented in the paper itself in no way supports the claims the IMF makes. Indeed, the paper itself is so shoddy and full of holes that it would shame even a first year undergrad had they submitted it as part of their coursework.
The paper published by the IMF uses data regarding income inequality and gender inequality (both measured as Gini coefficients) across a group of countries for the past two decades, controls for other factors that can affect income inequality, and finds that there is a statistically significant impact of gender inequality on income inequality. That is to say that the paper’s results suggest that reducing gender inequality will reduce income inequality.
However, this result is based on a methodology that is fundamentally flawed. In particular, although their main result is based on a regression analysis that does control for factors other than gender inequality that might affect income inequality, one of their main pieces of “evidence” is the supposed simple correlation between income and gender inequality. Indeed, they claim that the fact that this correlation is positive supports the idea that gender inequality is associated with income inequality see the first graph in the IMF’s blog post). However, the study fails to acknowledge that this correlation is, in fact, weak at best – the claimed positive relationship is barely above the horizontal.
Even worse is the fact that their claimed “strong association” between gender and income inequality is based on a regression analysis that finds that gender inequality is only statistically significant at the 10% level. In other words, there is a a 5%-10% chance that the results of this study are not due to any actual impact of gender inequality on income inequality, but are instead due to random chance. That can hardly be the basis for a claim of a “strong association” between the two, yet the IMF claims exactly that.
Furthermore, their treatment of the obvious simultaneity between income and gender inequality is not valid. Instead of using the accepted methods of two-stage least squares or GMM, the study instead just uses its instruments directly in the main regression and yet still claims that those results represent the impact of gender inequality on income inequality. This is patently false. In order to properly account for the simultaneity between income and gender inequality while still estimating the impact of gender inequality on income inequality, the study should include the instruments in a first-stage regression in which gender inequality is the dependent variable, before using the fitted values from that regression in the main second-stage regression in which income inequality is the dependent variable. The fact that the study has not done this, yet still claims that “[t]he estimation results are robust to concerns about the direction of causality” is, at best, disingenuous. (And, as an aside, the instruments the study uses are unlikely to be valid – it is easy to see how, despite the study’s claims, variables such as legal rights and labour force participation are likely to affect income inequality independent of their impact on gender inequality.)
Moreover, there are a number of substantial criticisms regarding the use of Gini coefficients to measure inequality, but these are too numerous to detail here. Suffice it to say, however, that the IMF’s study does not even attempt to ameliorate any of these problems.
It is baffling as to why the IMF thinks that a study so incomplete and open to justified criticism is worthy of publication. If this paper reflects the true standards the IMF require for publication, then this is a very dark day indeed.