Gravity, Gobbledygook, and Government Reports

Mark Reckless is an idiot. Now, if I stopped there, it wouldn’t make much of a blog post (although it would, as usual, be factually accurate). So, you might be asking, “Why is Mark Reckless an idiot?” But that is the wrong question. Instead, what you should be asking is “How has Mark Reckless demonstrated his idiocy this time?” And that would be a very good question indeed.

The answer to that question rests in the tweet below. This was Reckless’ comment regarding a description of one part of the methodology that the Treasury used to estimate the costs of Brexit – the equation in question was contained in a technical annex (i.e. where one would expect to find a detailed explanation of the approach used).

Reckless appears to be claiming that the equation in the picture he posted is equivalent to the fraudulent claims of a fortune teller. That could not be further from the truth.

Instead, the equation posted by Reckless is an algebraic representation of the “Gravity Equation” as applied to international trade. Emanating from Newtonian physics, this equation relates trade between two countries to the relative sizes of those countries (in terms of output and population) and the distance between them, plus some other controls for whether or not the countries in question share a common border / language / colonial history.

This is not a controversial method to estimate the impact of those factors on trade between two countries. In fact, the use of gravity equations is widespread in the assessment of international trade. A priori, one would expect larger countries to trade more with each other, but countries that are further away to trade less with each other and this is indeed reflected in the Treasury’s results.

The main point of this exercise, however, was to estimate the impact of being in the EU on the UK’s trade, and the Treasury does this by including a variable to capture that. The main result is that being in the EU increases trade in goods by about 100% (i.e. leaving the EU would result in a decrease in trade in goods of 53%) and increases trade in services by about 22%. Hence, being in the EU increases trade in goods and services overall by about 75%.

However, although the main approach used by the Treasury is reasonable, there are some areas in which it could be refined further. First, the Treasury’s analysis uses data covering the period 1948-2013, yet does not really try to control for factors that change over time (other than GDP and population). For example, there have been substantial changes to exchange rates and barriers to trade during the period covered by the Treasury’s data, both of which would have had substantial impacts on trade between two countries. The Treasury’s attempt to control for these changes over time consists solely of using dummy variables for each year (that do not vary across countries), which cannot even begin to capture the changes in exchange rates, trade barriers etc that would have occurred over the time period. This means that the estimated impact of being in the EU could well be incorrect.

Second, the Treasury’s approach assumes that the impact of being in the EU is the same for all countries. However, it is possible that the EU has an heterogeneous impact across countries – for some countries the impact of being in the EU might be larger than it is for other countries. By assuming away this possibility, the Treasury is likely to have under or over-estimated the impact of Brexit on trade.

Third, and on a more technical note, the Treasury does not specify what standard errors it has used. If the Treasury has used incorrect standard errors (for example, ones that do not correct for serial correlation or heteroscedasticity), that means that the statistical significance of its estimates is incorrect and, more importantly, that the error bounds (i.e. the upper and lower ends of their estimate) are likely to be incorrect.

Nonetheless, these minor potential refinements of the Treasury’s approach do not detract from the fact that Mark Reckless has been remarkably foolhardy in his response to the Treasury’s assessment of the impact of Brexit.


The cost of Brexit

How much does the UK’s membership of the EU actually cost? And, in fact, does being in the EU represent a net economic benefit, rather than a net cost?

If you were to believe the information provided by Vote Leave, you might think you’d know that the answer. Vote Leave has claimed that membership of the EU costs the UK about £18 billion per year, the equivalent of about £280 per person per year. However, this figure does not include the substantial rebates and public/private sector receipts that the UK receives from the EU – once these are taken into account the actual direct budgetary cost of the UK’s membership of the EU is about £8.4 billion, or £131 per person, per year (i.e. less than half of the original Vote Leave claim).

Moreover, the Vote Leave figure only includes the direct budgetary costs of being part of the EU. Importantly, it does not include, nor does the Vote Leave campaign attempt to include, any “indirect” benefits that result from EU membership. Such indirect benefits include, for example, any jobs or exports resulting from trade with EU countries that would not otherwise occur absent EU membership. If membership of the EU increases UK output above what it would have been if the UK was not part of the EU (which is likely to be the case), then leaving the EU would result in a decrease in UK output.

This could happen for such wide-ranging reasons as EU consumers have more diverse tastes than just those in the UK allowing a larger number of different firms to flourish in the UK and export their output to the EU than would otherwise prevail if the UK left the EU and UK firms would have reduced demand from EU countries; or collaboration between EU and UK firms enables a wider spread of technology that would not be possible after Brexit such that UK productivity is higher than it would be outside the EU; or membership of the EU encourages investment not just from EU firms but from firms located in the Rest of the World that would not occur if the UK were to leave the EU. There are plenty of other potential mechanisms through which EU membership increases UK output.

Importantly, although Vote Leave has not attempted to include such factors, the Centre for Economic Performance (CEP) has done so and finds that leaving the EU would reduce the UK’s output by at least £850 per household per year. That is the best case scenario for the Vote Leave supporters. Note, too, that this only includes “static trade consequences” – i.e. the impact that can be attributed just to losing the ability to trade freely with EU countries; it does not include any of the costs associated with reduced migration, technology transfer, investment etc that would also result from leaving the EU. In fact, once these factors are taken into account, the cost of leaving the EU could be as high as £6,400 per household per year.

As such, the £850 figure likely underestimates the true cost of leaving the EU.

Nonetheless, that is not to say that every part of the CEP analysis is beyond criticism. For example, the study assumes that intra-EU trade costs will continue to fall as they have done in the past, but does not provide any evidence to suggest that such an assumption is reasonable. If, in fact, intra-EU trade costs were to fall less quickly than assumed by the study, then the costs to leaving the EU would be reduced.

Moreover, little information is provided regarding how the estimates of the cost of leaving the EU that account for the aforementioned “dynamic” factors (such as migration and investment) are obtained. Given that those estimates are likely to be based (at least in part) on complex (albeit commonly used) statistical methods, a higher level of transparency regarding the approach used would be welcome so as to enable a higher degree of confidence that the estimates have been obtained via a reasonable approach.

Overall, therefore, although the Vote Leave figure regarding the benefits of leaving the EU is an egregious over-estimation, and it is actually highly likely that there would be a large net cost to leaving the EU, it is unclear what exactly the cost per household per year is. However, this uncertainty regarding the exact cost should not detract from the fact that the cost to leaving the EU is large.

Glassdoor’s “contribution” to gender wage gap research

In a whirlwind of publicity and self-promotion, Glassdoor recently released the results of a “study” that claimed to prove the existence of a gender pay gap even when potential differences in areas such as “personal characteristics, job title, company, industry and other factors” are accounted for. As a result, Glassdoor boldly claims that men are paid about 5% more than women.

However, the approach used by Glassdoor is subject to a major problem. In particular, Glassdoor’s approach relies on them being able fully to control for all other factors (such as experience, qualifications etc) that might determine someone’s wage. Although Glassdoor notes this themselves (in a single paragraph relegated to the back of their report), they do not qualify any of their headline results with an acknowledgement of this.

Seeing as Glassdoor only includes controls for a few personal characteristics (such as age, qualifications, and experience) and some factors relating to a person’s occupation and industry (such as job title and company name). In other words, Glassdoor excludes a number of relevant factors that are likely to be relevant when it comes to explaining someone’s wage.

Indeed, other studies have found that factors such as ethnicity, whether or not someone is a member of a trade uniona person’s mental and physical health, and even language skills can be important determinants of a person’s wage. The Glassdoor study does not account for any of these, and thereby erroneously attributes differences in wages that could be due to these (or other factors) to the gender pay gap.

In addition, the Glassdoor study does not seem to account for whether or not someone is working part-time or full-time – as part-time workers are likely to be paid less than full-time workers, even on an hourly basis, Glassdoor’s apparent failure to include such a distinction in their analysis could bias their results substantially. Similarly, the Glassdoor study does not even try to account for potential unobservable differences (such as personal preferences regarding careers), and this failure further biases Glassdoor’s estimate of the gender pay gap.

Finally, the data used by Glassdoor are from self-reported salaries and characteristics that are recorded by members of the Glassdoor website. There are plenty of reasons to suspect that these data are unreliable – at the very least, it is widely recognised that figures that are self-reported are likely to be subject to considerable bias, such that relying on them for a study such as this is nonsensical.

Therefore, it is clear that Glassdoor’s “study” into the gender wage gap is merely an exercise in self-promotion rather than a useful contribution to the substantial amount of past research that has been conducted on this issue.