Scotland's Renewable Energy Statistics

Summary

Full Fact have concluded that senior SNP politicians have been guilty of making false statements to suggest that Scotland is close to self-sufficient in renewables (see here and here).

Nowadays - to avoid being accused of making blatantly false claims - the SNP are generally careful to include the qualifying phrase "the equivalent of" when making statements about the extent to which Scottish renewables meet Scotland's electricity demand. But as Full Fact have also made clear, this phraseology is still misleading:

The SNP are clearly wedded to this misleading statistic - take just three recent examples:

• Responding to Full Fact: "A Scottish Government Spokesperson said: Scotland has a hugely positive story to tell in renewables, which provided the equivalent of 96% of Scotland's gross electricity consumption in 2020"
  
  
• Nicola Sturgeon correcting the record in Holyrood: "Corrected text: Under this Government, the equivalent of 98.8% of our gross electricity consumption is already provided by renewable energy sources."
  
  
• Ian Blackford in Parliamentary Debate: "and nearly 100% of the equivalent of our electricity consumption already comes from renewables <interruption> ... I ... you know I've said equivalent on many occasions but I thank the right honourable gentleman for that [...] I think if the honourable gentleman checks Hansard he'll find that I've said that on a number of occassions"
  
  

---

As an aside: it's easy to check Hansard and in fact Ian Blackford has never used the "equivalent" qualifier in this context (see here) but he did make the false, unqualifed claim in September (see here); "almost 100% of our entire electricity production comes from renewables. That is not attention-seeking, I would say to the Prime Minister; these are the facts."

---

To illustrate why quoting only the "equivalent of " statistic is highly misleading, the following figures all come from the same Scottish Government source for 2020 and are internally consistent:

• Scotland was capable of meeting all of its electricity demand from renewable generation just 51.1% of the time
  
  
• 55.7% of electricity consumed in Scotland came from renewables
  
  
• The equivalent of 98.8% of Scotland's gross electricity consumption was generated by renewables [this is the figure the SNP keep quoting]

Using the same source we also know the figures for 2021 (with one caveat) and in all cases the most recent year's figures are lower:

• Scotland was capable of meeting all of its electricity demand from renewable generation just 37.8% of the time [this is the figure quoted by Full Fact above]
  
  
• 52.8% of electricity consumed in Scotland came from renewables
  
  
• The equivalent of 82.9% of Scotland's gross electricity consumption was generated by renewables [a figure which was briefly published and subsequently deleted, see below]

Looking at the figures above it's obvious why the SNP are so fond of the 2020 "equivalent of" figure. What is perhaps less obvious is how the first two measures above can be consistent with that figure. This blog seeks to clarify the confusion caused by these different measures and to explain why the first two measures give a clearer "real world" picture.

It should be clear from these figures that anybody with the impression that Scotland is close to being self-sufficient in renewables has been seriously misled. 

Explaining the Figures

Scotland's First Minister was recently compelled to correct the official record following an FOI request from Sam Taylor of These Islands. The correction (found here) was as follows:

Original text: Under this Government, we have a position where our net energy consumption is already provided by renewable energy sources.

Corrected text: Under this Government, the equivalent of 98.8% of our gross electricity consumption is already provided by renewable energy sources.

The introduction of 98.8% and the switch from net to gross are distractions, the key amendment doing all the heavy lifting is the introduction of the words "the equivalent of".

To explain what's going on here we need to understand three different measures, each of which tells us something different about the extent to which Scotland's renewables energy generation matches Scotland's electricity demand

1. The equivalent of gross electricity consumption met from renewables: this simply takes total renewables electricity produced through the year (including electricity exported) and divides it by total electricity consumed in Scotland in the same year; it takes no account of whether the electricity was being produced when it was needed in Scotland
   
   
2. Proportion of electricity consumed from renewables: this shows how much electricity consumed in Scotland actually came from renewables. It's a far lower figure because it reflects the reality of demand vs supply variation; if Scotland needs more electricity than renewables can provide at the time it's needed then non-renewable generation is used 
   
   
3. Proportion of time when Scotland is capable of meeting demand from renewables: this is the harshest measure of "self-sufficiency" as it counts only those half-hour periods where Scotland's renewables generation exceeds demand.

[If these measure still seem confusing, the appendix below provides some simple worked examples] 

We can look at each of these measures as shown on the Scottish Government's "Energy Statistics Hub".

Measure 1: in 2020 the equivalent of  98.8% of gross electricity consumption came from renewable sources [found here]

This is the figure relied upon by the First Minister's correction above. Note this figure is for 2020 despite it now being November 2022.  Interestingly the figure for 2021 was briefly published but has since been deleted - credit to @PantoPolitics for grabbing the screencap below before the 2021 data was removed

That this showed a material decline in 2021 is not a surprise: the same data source shows us that significantly less electricity was generated from renewables in 2021 than in 2020 (it simply wasn't as windy).

The decline in the headline "equivalent of" percentage is certainly politically unhelpful to the SNP, but as the figure remains unpublished, they are free to keep using the out-of-date 2020 figure. What is curious is that an initial view of the 2020 data (showing an increase on the prior year) was made available in March 2021:

But in November 2022 the data for 2021 (which we know will show a decrease on the prior year) has not been published. Readers are free to draw their own inference from this observation.

Measure 2: in the 12 months to September 2022, 63.1% of electricity that Scotland consumed came from renewable resources [found here]

That figure is also available for past calendar years so we can see that in 2021 it was 52.8% and in 2020 it was 55.7%

Measure 3: in 2021 Scotland was capable of meeting demand from Scottish renewables generation 37.8% of the time [found here]

As we can see from the green line on the graph above, that figure was down from 2020 (when it was 51.1%).

For the avoidance of doubt: the light orange line shows that when Scotland's Nuclear and gas generation capacity is included, Scotland is effectively self-sufficient in electricity generation - but we are most definitely not close to self-sufficient when it comes to generation from renewables.

It is perhaps also worth noting that the SNP are resolutely opposed to nuclear generation (nuclear being the gap between the green and blue lines above).

So to summarise using the figures for 2021:

• Scotland was capable of meeting all its electricity demand from renewable generation just 37.8% of the time 
  
  
• 52.8% of electricity consumed in Scotland came from renewables
  
  
• The equivalent of [roughly] 82.9% of Scotland's gross electricity consumption was generated by renewables [a figure which was briefly published and subsequently deleted]

Appendix: Explaining how the measures differ

Having discussed these measures with many smart and well-informed people, it's clear that it's easy to get confused about what the different measures mean. To explain how the three different measures vary, it's helpful to illustrate with some simple theoretical examples of supply/demand matching.

Let's start with a very simple extreme example where, over a 12-period timescale, renewables always meets 50% of demand:

In this illustration:

• Measure 1: the equivalent of 50% of electricity consumption is met by renewables
• Measure 2: 50% of electricity consumed came from renewables
• Measure 3: renewables were capable of meeting demand 0% of the time
  
  

Now let's try another extreme example, where half the time renewables fully meet demand, half the time there is no renewables generation:

In this illustration:

• Measure 1: the equivalent of 50% of electricity consumption is met by renewables
• Measure 2: 50% of electricity consumed came from renewables
• Measure 3: renewables were capable of meeting demand 50% of the time
  
  

Now let's consider something a bit more variable (but with supply still never exceeding demand):

In this illustration, 73% of the graph is green so:

• Measure 1: the equivalent of 73% of electricity consumption is met by renewables
• Measure 2: 73% of electricity consumed came from renewables

In 6 of the 12 periods renewables fully met demand so:

• Measure 3: renewables were capable of meeting demand 50% of the time

As a final illustration, lets consider the more realistic scenario where renewables generation exceeds demand in those periods where 100% of demand can be met from renewables:

In this illustration

• Measure 1: the equivalent of 83% of electricity consumption is met by renewables (the total green area would fill in 83% of the area under the 100% of demand line)
  
  
• Measure 2: 73% of electricity consumed came from renewables (this is the same as the previous illustration because renewables production which exceeds demand does not contribute to this figure)
  
  
• Measure 3: renewables were capable of meeting demand 50% of the time (6 out of 12 periods)
  
  

In reality, each time period would have a different absolute level of demand, so the maths gets a little more complicated - meeting demand when demand is low has less impact on measures 1 and 2 than meeting demand when demand is high (although measure 3 is unaffected by this issue). I could draw another graph, but I think we've probably gone far enough to explain the dynamics at play here.

ADDENDUM [added 04/11/2022]

The difference between "when the wind blows" and "when we need electricity" is clearly an important factor in this debate, so I went hunting for the data and found what I wanted on the National Grid Data Explorer.

The "Demand Data Update" provides actual half-hourly data for (amongst other things) National Demand and Embedded Wind Generation.  That Embedded Wind Generation is not all wind generation*, but it provides us with data on variability of wind generation for the same half-hourly periods that we have data for National Demand. 

Armed with this data we can create indices of electricity demand and wind generation where both average to 100.  This gives us an illustrative graph where the equivalent of all demand is met by wind generation while reflecting the actual variability of that supply and demand during the month of October 2022:

So when the graph is brown demand is met by wind supply, when it's green wind supply exceeds demand, when it's red demand is not met by wind supply. Note: while this is based on real UK demand and wind supply profiles (on a half-hourly basis) it is indexed to illustrate a position where the equivalent of demand would be met by wind supply.

The Graph illustrates quite neatly that electricity demand and wind energy supply are not correlated - having the equivalent of enough wind generation to meet demand is not the same as supply actually matching demand when it's needed.

This is of course why developing economically viable and scaleable storage solutions for periods of excess reneweables generation is so important (e.g. pumped, battery or potentially Hydrogen storage - see here). But it also shows why wind generators in Scotland benefit from direct access to the larger pool of demand that comes from being part of a fully-integrated GB energy market - placing a border between excess renewables supply and demand for that renewables capacity helps nobody (and can only hinder progress to Net Zero). 

******

* "This is an estimate of the GB wind generation from wind farms which do not have Transmission System metering installed. These wind farms are embedded in the distribution network and invisible to National Grid. Their effect is to suppress the electricity demand during periods of high wind. The true output of these generators is not known so an estimate is provided based on National Grid’s best model. Note that embedded wind farms which do have Transmission System metering are not included in this total."

Deficits, Deficit Gaps and Fiscal Transfers

To understand what is going on when we talk about implied fiscal transfers between different parts of the UK (as discussed here), it's perhaps easiest to think of what happens when we split the bill in a restaurant.

To know whether or not we benefit from splitting the bill, we only need to know two things:

1. How much of the bill are we responsible for creating?
2. How much of the bill do we actually have to pay?

If the first figure is greater than the second, we benefit from splitting the bill (we receive an implied transfer from the others we're splitting the bill with).

In the context of the debate around Scottish independence, the first of these questions is answered by the Scottish Government's own GERS report. This tells us, based on a series of explicit assumptions, how much of the UK's deficit (the bill) Scotland is responsbile for.

The second question is more contentious, as there are no "official figures" as to how responsibility for the national debt (the cumulation of annual deficits) is shared¹  - so how much of the bill does Scotland have to pay?

Fortunately there is broad consensus around the view that the UK's debt should (or at the very least reasonably could) be shared on a population basis.

1. The GERS figures include a population share of the interest charge generated by the UK's debt - given that debt is merely the accumulaton of the UK's deficits over time, that is effectively a population share of the UK's (cumulative) deficit
2. The Independence White Paper in 2014 stated "Scotland and the rest of the UK will agree a share of the national debt. This could be by reference to the historical contribution made to the UK’s public finances by Scotland. An alternative approach would be to use our population share."
3. The SNP's own Sustainable Growth Commission danced around this question, but eventually assumed a population share of UK debt interest within their proposed "solidarity payment"

So it's really pretty simple: the difference between the share of the UK's deficit Scotland is responsible for creating (see GERS) and the share of the UK's deficit Scotland pays for (assume population share) is the implied fiscal transfer.

As we'll come on to see, you can make different assumptions about the share of the UK's deficit Scotland will ultimately have to pay, and conclude a different figure for the implied fiscal transfer.

***

At the risk of labouring the restaurant analogy, let's run through an example with some illustrative figures to help us explain the differences between three terms that often get confused: the deficit, the deficit gap and the effective fiscal transfer:

• Group A: 18 people go for a meal, the bill comes to £1,800 so they have spent £100/head
• Group B: 2 different people go for a meal, their bill is £300 so they have spent £150/head

If Group A and Group B decide to get together and split the bill (to "pool & share the deficit"), what happens? The total bill would be £2,100 which split equally between 20 people would be £105/head. Here's a simple summary:

The spending gap between the groups is £50/head (Group B spent £50/head more than Group A), but the benefit of pooling and sharing - the effective transfer Group B receives - is £45/head².

The total transfer from Group A to Group B is £45x2 = £90

Now let's replace the figures in our analogy with the fiscal reality (per GERS 2018-19) - the "bill" is the deficit, Group B is Scotland and Group A is the rest of the UK.

So Scotland's GERS deficit is £12.6bn, the deficit gap is £11.6bn and the effective fiscal transfer to Scotland is £10.7bn. 

A huge amount of confusion is caused by people failing to understand the conceptual the differences between these figures - if you've followed what's going on up to here, give yourself a pat on the back.

***

So armed with this understanding, let's take a look at the most common mistake made when people debate the "£10bn fiscal transfer". To illustrate, let me use the following screen-capture which (incredibly) is taken from Stuart Campbell's own "Wings Over Scotland" blog:

If you've been following this blog post so far, you will realise who the twit is in the exchange above (hint: it's not Paul). To walk through this carefully, per the figures above:

• Scotland's deficit is £12.6bn
• We assume Scotland bears a population share of the UK's deficit - so in this year Scotland takes on an additional "loan" of just £1.9bn
• The difference of £10.7bn is the effective fiscal transfer Scotland receives - it's the amount over and above the "loan" Scotland takes on

Fun Fact: this means that those who argue Scotland should assume less than our population share of the UK's debt are - whether they realise it or not - arguing that the effective fiscal transfer in Scotland's favour is in fact larger than £10.7bn.

***

A common reaction to these figures is "how can Scotland's 8% of the UK population possibly be responsible for a third of the UK's deficit - that seems unbelievable". This is what is technically known as an "argument from incredulity" and is perhaps best summarised by this quote from Professor Richard Murphy:

"I have been continually bemused by the fact that GERS says that Scotland runs a deficit so  much larger in proportionate terms than that for the UK as a whole."

Here our restaurant bill analogy falls short, because what we're dealing with when we're sharing the deficit is not how much we've spent but the net effect of how much revenue we've generated less the amount we've spent. I've explained the dynamics involved here in this brief video (with apologies for my exasperated tone and the figures being a year out-of-date)




Another way to help understand this point is to look at fiscal transfers across the UK (including the English regions) as this blog has recently done here. There is nothing surprising or hard to fathom going on here - it's just simple fiscal arithemetic.

As I've pointed out before: it's not hard to imagine a situation where Scotland runs a small deficit while the the UK overall is in fiscal balance - in that scenario Scotland would be responsible for an infinite (or more accurately: a "divide by zero error") share of the UK's deficit. It's just maths.

***

When we use the GERS figures to scale the effective fiscal transfer, we have to recognise that these are only pro-forma figures, they represent what Scotland's stand-alone defict would be if we kept generating revenues and incurring spending as shown in GERS.

---

In case it's not already dead, let me flog the restaurant analogy one more time: "if we weren't sharing the bill, maybe we wouldn't have tipped the waiter 15% and perhaps we wouldn't have ordered the bottled water for the table."

---

This is a fair point. Even before we consider the likely economic shock impacts on revenue or spending that separation from the UK would cause (see Brexit), the scale of deficit that the GERS figures reveal means that current levels of spending would be unsustainable for a newly independent Scotland, particularly if trying to launch a new currency.

It's true that some of that spending in GERS is costs allocated from the rest of the UK on a simple population basis (defence, debt interest and international aid being the vast majority of these), so any case for independence needs to start by working out what an independent Scotland would replace these costs with. For reference: relative to that £10.7bn fiscal transfer, the notoriously optimistic White Paper on independence assumed a net saving of £0.6bn.

What typically happens at this point is that some of the more blindly-committed supporters of independence start suggesting that the GERS figures are all made up anyway as part of some vast conspiracy by which Westminster has managed to get the Scottish Government's own economists to pull the wool over the eyes of the SNP (and their Sustainable Growth Commission, their Fiscal Commission Working Group, the IFS, Fraser of Allandar, NIESR, UK Statistics Authority, etc. etc.).

This is of course a ridiculous position to adopt (which, to be fair, is why only those flakier members of the independence movement attempt to adopt it). Alex Salmond was certainly very clear about what the GERS figures told us when he thought he could spin them in his favour:



Salmond is the man who once proudly boasted of his ability to put “a gloss on statistics or any economic figure” to build a political case, and he certainly did his best to do that with the 2010-11 GERS figures. He made the highly dubious claim that they showed an independent Scotland could have been spending £2.7bn more and therefore should have been running an even higher deficit than that shown in GERS!




Still: desite the fact that he used a different method for "splitting the bill" (based on a GDP share not a population share), he was recognising the principle of the fiscal transfer³.

Unfortunately for independence supporters, taking the logic Salmond applied to the 2011-12 figures and applying them to the 2018-19 figures produces a massive fiscal transfer now in Scotland's favour - so by his own logic, an independent Scotland should now be spending £10bn less⁴.

At this point, most of those arguing for independence ignore how wedded they used to be to the figures and return to straight-froward "GERS denial" - fortunately this blog has already comprehensively dealt with those denials here > GERS Deniers.

Ah but wait: what about "this is just a snapshot"?

OK, well we can do this analysis over time and plot the size of the deficit gap⁵ for the last 21 years:

You can see why Alex Salmond was so excited about the 2008/09 to 2010/11 figures⁶.

The reason for the dramatic reversal and growth in that gap will be familiar to regular readers of Chokkablog - they are most easily summarised by this graph:

• The gap closed when North Sea revenues boomed, but has grown massively as North Sea revenues have plummeted
• Scotland has not only continued to spend more per head than the rest of the UK, that spending gap itself has actually grown (thanks to the Barnett Formula and low levels of absolute spending growth⁷) 
• Scotland's onshore revenue performance has declined relative to rUK⁸

For completeness, we can plot the onshore deficit gap over time (i.e. to see what happens if we strip out North Sea revenue effects from these figures):

Without oil revenues, there would never have been a prime facie economic case for Scottish independence - and the vagaries of the Barnett Formula (plus perhaps the impact of the SNP's tax rises) have led to the scale of the fiscal transfer that Scotland benefits from within the UK actually increasing in recent years.

/Ends/

---

Notes

---

1. This very carefully worded FoI response is sometimes in debates around the fiscal transfer:

"Official figures for any fiscal transfer are not available.

The reason this information is not available is that such a figure requires a number of assumptions to be made. For example, as the UK as a whole spends more than is raised in revenue, an assumption would need to be made about which parts of the UK borrowing is undertaken for, or which types of public spending are financed by borrowing as opposed to taxation. This information is not available as, for example, some taxes are ringfenced to fund particular services; for example, some national insurance contributions are ring-fenced to fund the NHS. As such, any figure for a fiscal transfer from the rest of the UK to Scotland would rely on a number of assumptions."

this is entirely consistent with what this blog (and others) have always said - to calculate the implied fiscal transfer, we have to make some assumptions. In fact, argue we can calculate and implied fiscal transfer by only making one assumption: that the burden of the UK's deficit (and associated debt) is borne on a population share basis

2. People used to dealing with numbers will have spotted that the transfer = [(1-population share) x the gap] - something easily proved if you care for such things

This matters only insofar as we need to understand that, in the case of Scotland in the UK, the fiscal transfer is 92% of the deficit gap

3. The IFS implicitly use that same assumption when referring to the fiscal tranfer here

The most recent figures (2016–17) imply a budget deficit for Scotland of 8.3% of GDP. Managing this is the UK Government’s responsibility as it is part of the UK’s deficit, which was 2.3% of UK-wide GDP in the same year. Therefore there was a fiscal transfer from the rest of the UK to Scotland of about 6% of Scotland’s GDP (equivalent to around £1,750 per person in Scotland).

Because GDP/Capita is now about the same for Scotland and rUK, allocating the deficit on a per capita basis or per GDP basis makes no material difference - but I would still argue that per capta is the right way to do the analysis as long as GERS uses per capita allocations for all shared UK-wide costs

4. To be completely accurate: if we used his GDP share rather than population share method then the figure would be £9.8bn (rather than the £10.7bn we get using population share) - but the broader point stands

5. Remember: the implied fiscal transfer = [(1-population share) x the gap] = 92% of this figure

6. These are the latest available restated historical figures - when first released the figures showed a significantly more favourable position for Scotland, but later revisions lowered Scotland's apparent fiscal advantage vs rUK - covered in some here: The SNP: Living in the Past

7. A dynamic most easily understood if you imagine a scenario where UK spend (and therefore Scotland's spend) doesn't change, but Scotland's population grows more slowly than rUK's - under that scenario it is inevitable that the gap between Scotland's spend/head and rUK's must increase

8. Due to some combination of historically over-estimating the number of top-rate tax payers in Scotland and/or the increase in the Scotttish Rate of Income Tax causing some of those tax payers to redomicile

Pooling and Sharing: The English Regions

In a few weeks time I'll be chairing a conference in Newcastle - These Islands: Our Past, Present & Future

The conference will feature an impressive array of speakers and panelists including: Douglas Alexander, Philippe Auclair, Gordon Brown, Andy Burnham, Frances Coppola, Sir John Curtice, Simon Evans, Sophia Gaston, Ayesha Hazarika, Gerry Hassan, Fiona Hill, Henry Hill, Colin Kidd, Carwyn Jones, David Lidington, Ian Murray, Baronesss Quin, Mark Reckless, Willie Rennie, Lord Salisbury and many, many more. If you're interested in coming along, you can find more details and ticket booking information here > eventbrite page.

Needless to say: just as Bob Geldof wasn't going to go to all the trouble of organising Live Aid and not get on stage to perform with the Boomtown Rats, so I will not be passing up the opportunity to put some graphs in front of this captive audience¹ .

We're holding the conference in Newcastle to highlight the importance of the English regions in any debate about the future of the UK. To this end I've been doing fresh analysis on the fiscal economics of the English regions and - because I won't have time to present detailed analysis at the conference - I though I'd quickly blog about it here.

Chokkablog regulars will be familiar with the concept of the implied fiscal transfer, but to recap: if a devolved nation runs a deficit per head (aka "per capita deficit") higher than the UK average, then that nation is benefitting from an implied fiscal transfer from the rest of the UK².

The same principle can be applied to the English regions. Fortunately the data now exists to allow us to calculate and understand these regional fiscal transfers just as the (notorious?) Government Expenditure and Revenue Scotland (aka GERS) figures do for Scotland³.

The source data is Country and Region Public Sector Finances analysis as produced by the ONS⁴. All the figures we use here are those that allocate a "geographical share" of oil & gas revenues (i.e. Scotland gets to keep the oil & gas revenues generated by oil in Scottish waters).

The only other thing we have to remember before diving into this analysis is that the per capita deficit difference to the UK average is made up of two distinct parts: the per capita revenue difference and the per capita spending difference. The former reflects the economic performance of the region in terms of tax revenue generation, the latter reflects the cost of delivering public services to that region⁵.

So let's look first at per capita revenue generation differences by region:

Remember that what we're seeing here is how well these regions generate tax revenue versus the UK average. There has been a lot of talk in recent weeks of "levelling up" - that's basically about getting these bars to shrink back towards zero, so the size of the red bars is a decent guide to which areas are in greatest need of "levelling up".

It should come as no surprise that London and the South East are the areas which "out-perform" and - unless you've been deceived by the SNP's grievance rehetoric - it should also be no surprise to see Scotland (like the East of England) performing as per the UK average and significantly out-performing Wales, Northern Ireland and all other English regions.

So what's Scotland's problem?

Well let's look at per capita spending per region⁴:

Here's where we see the areas that enjoy (or require) higher spending per capita: Northern Ireland and Scotland most significantly, with London, Wales and the North East as the other "relatively high spend" areas. The extent to which this is based on greater need (e.g. to deliver equivalent services in areas of lower population density and/or with remote/island communities and/or to reflecting higher social costs driven by demographic factors and/or due to areas of endemic poverty) or greater investment (for better services than the UK average or to stimulate economic development) is the subject of some debate.

If we combine the per capita revenue difference with the per capita spending difference, we get to the per capita fiscal balance difference (and hence the implied fiscal transfer):

It's quite a striking picture isn't it? The Devolved Administrations in Wales and Northern Ireland receive far greater per capita fiscal transfers than Scotland, as does the North East.  London, the South East and the East of England are responsible for generating fiscal transfers that go to the rest of the UK.

It's perhaps helpful to summarise all of this data on one exhibit:

What this shows us is the extent to which the fiscal transfer to Scotland (caused by the much debated higher notional Scottish deficit) is a function of higher spending, not lower revenue generation (i.e. not "weaker economic performance"). This contrasts dramatically with Wales the Midlands and the North of England, where relatively poor fiscal performance is explained by weaker revenue generation (i.e. "weaker economic performance") far more than by any spending differences.

It's clear that any debate about the future of the UK has to grapple with these two related questons

• How are resources most fairly and efficiently distributed between and administered in the devolved nations and English regions?
• What practical steps can be taken to "level up" economic performance across the UK?

***

For those who like to see the figures behind the pictures:

For those who wonder about regional difference within Scotland, I offer the following GDP/capita chart - suffice to say we could expect there to be similar fiscal transfers happening within Scotland, and the variance of economic performance within Scotland appears not dissimilar to the variance across the UK (or indeed in continental Europe):

---

Notes

---

1. Yes, I'm aware how ridiculous it is to imply that this conference is some sort of constitutional Live Aid

2. We share the burden of the UK's deficit UK-wide - both in these analyses (i.e. the cost of the total UK debt is allocated to the regions and devolved administrations on a per capita basis) and as widely assumed and accepted in the case of inherited liabilities (i.e. were Scotland to separate from the UK, it would inherit a population share of he UK's debt, as accepted by the Independence White Paper and the SNP's more recent Sustainable Growth Commission)

3. This figures differ from GERS, but not materially so at the deficit per head level - comparing 2018-19 ONS and GERS, it looks like there is a different approach to what is taken as revenue vs what is netted off against cost (but I'm guessing here) - all that really matters is that the figures used for the exhibits on this blog post are all compiled on a comparable basis

• Spend/head: ONS = £14.5k; GERS = £13.9k
• Revenue/head: ONS = £12.0k; GERS = £11.5k
• Deficit per head: ONS = £2.5k; GERS = £2.6k

4. It is worth noting that these are not qualified as National Statistics, but rather Experimental Statistics

5. It's worth noting that those costs which are shared on a population basis (mainly debt interest, defence and international aid) have no impact on this analysis - there is by definition zero difference between per capita costs allocated on a per capita basis!

Judging the SNP on Their Education Record

*************************

Note: this blog post is now more neatly summarised here: Scottish Education: A Generation, Failed

*************************

"Let me be clear – I want to be judged on this. If you are not, as First Minister, prepared to put your neck on the line on the education of our young people then what are you prepared to. It really matters." 
Nicola Sturgeon, August 2015

"the statistics also show a drop in writing performance for S2 pupils which is of particular concern [..] the SSLN statistics are disappointing"
John Swinney, May 2016

"The figures for Scotland do not make comfortable reading [..] compared to 2012, our performance in science and reading has fallen.  In science and maths we are now below the levels at which we performed in 2006, and more countries have outperformed Scotland in all three areas than at any time since PISA began."
John Swinney December 2016

Nicola Sturgeon has asked to be judged on her record on education and the SNP has been in charge of our fully devolved Scottish education system for roughly 10 years. Responsibility for the educational outcomes we're now seeing must lie with the SNP - and the trend in those outcomes is simply terrible, as Cabinet Secretary for Education and Skills John Swinney has had to admit.

The most recent international benchmark data provided by PISA¹ covers the period from 2006 to 2015 (it's a survey that happens only once every three years). PISA surveys 15-year-old pupils, so the most recent data shows results for pupils who had been in the Scottish education system for 7 years under an SNP administration.

The graphs below are pretty clear. Since the SNP came to power, in each of the three main disciplines as measured by PISA (mathematics, reading & science), Scotland's performance has declined significantly. This is true in absolute terms, relative to the whole of the UK and relative to the average for all OECD countries.

So in maths and reading, where Scottish pupils used to perform significantly better than rUK students, latest available data shows we've dropped back to average at best. In science, where Scottish pupils used to perform in line with the UK average, their performance in the most recent survey is very significantly worse. In all three disciplines, we've gone from out-performing the OECD average to being - well - distinctly average.

Looking at our performance in international league-table format: in all three subject areas we've dropped below the OECD average and into the bottom half of the table.

As the Scottish Government themselves make clear, "PISA is the major international study of pupil performance in which Scotland participates" - it is their chosen method of benchmarking our educational attainment.

In fact, having withdrawn from the other globally recognised assessment body (TIMSS & PIRLS) in 2011, it's now the only data we have to compare our performance on an international basis. It's good quality data¹ and the results are a damning indictment of the SNP's performance as custodians of our education system.

***

There is one other way we can objectively assess the performance over time of the Scottish education system and that's the Scottish Government run Scottish Survey of Literacy and Numeracy² (SSLN). This doesn't offer international comparisons, but it does allow us to observe trends over time on a like-for-like basis and - among other things - look at the "Attainment Gap" (the extent by which pupils from the most deprived backgrounds under-perform those from the least deprived).

SSLN focuses on numeracy and literacy in alternate years, so we have numeracy data for 2011, 2013 & 2015 and literacy data for 2012, 2014 & 2016. Pupils are assessed at three different stages of education: P4 (age 7-8), P7 (age 10-11) and S2 (age 13-14). For each of these groups the graphs below show trend in the percentage of pupils assessed as performing "well" or "very well" in the subject, and splits these by deprivation status (most deprived, middle, least deprived).

Taking numeracy first:

---

To read these graphs: the dotted black line shows the overall performance over time, the gap between the red (least deprived) and green (most deprived) lines illustrates the Attainment Gap.

---

So the available data for numeracy shows overall standards declining slightly, although stabilised (at a very poor level) for S2. The largest Attainment Gap exists at the S2 level and there's no evidence of it closing. The Attainment Gaps at both P4 and P7 are growing significantly, and the decrease in attainment for the most deprived pupils at P4 level (the earliest measured stage) is dramatic and concerning.

Now looking at literacy performance:

---

**
[03/06/2017] NB: as has been helpfully pointed out by James McEnaney, the graphs below are for the "reading" element of literacy only (result of me not paying enough attention to the data table structure, my bad: assumed would be a single measure as the numeracy data is). When I have time I'll crunch the "writing" data ["listening & talking" data only exists for two years]. Full correctly labelled set of graphs is now >here
**

---

Overall attainment levels are broadly stable, although there is some evidence of decline at P4. The Attainment Gaps for literacy are less severe than for numeracy. There is no evidence of any sustained reduction in the Attainment Gap at S2 or P7 - and at P4 the Attainment Gap is widening (primarily due to a decline in performance of pupils from the most deprived areas).

Having looked at all these graphs now, the poor level of overall numeracy attainment at S2 level is striking: is our education system producing a generation soon to vote who aren't great at adding up?

Combine the SSLN findings with the PISA results and a clear picture emerges: on the SNP's watch, overall standards in Scottish education have dropped and pupils from the most deprived areas have been the ones to suffer the most.

You might think it's going to be interesting to see what the next SSLN survey data shows, particularly as it would be on numeracy again, the area where the most worrying trend has been observed. Well it turns out we'll be unable to objectively judge Sturgeon's record on this because - with a spectacular disregard for the value of consistently audit-trailed performance data - her government have chosen to stop the SSLN survey altogether. To coin a rather poignant phrase in this context: you do the math.

***

In her recent grilling by Andrew Neil on BBC1, Nicola Sturgeon delivered a masterclass in obfuscation and misdirection when pressed on this topic. She brushed off references to the latest available international benchmark data as "from two years ago" and airily dismissed her own Government's survey of over 10,000 pupils as somehow irrelevant "because it's a sample survey". She even had the brass neck to go on to argue that "there's real progress been made".

---

The Spectator's Fraser Nelson has written an excellent blog unpicking the detail of Sturgeon's blustering in that interview: "Fact-checking what Nicola Sturgeon told Andrew Neil about education" - I heartily recommend it.

---

Sturgeon's argument for dismissing the SSLN data and discontinuing the survey was that it doesn't give meaningful data at a school-by-school level. That is a frankly ridiculous argument that would only make sense if anybody was trying to use the data to judge specific schools or local authorities - they're not.

SSLN was designed to give an overall assessment of the quality of literacy and numeracy education in Scotland, and it does that well. The (Scottish Government commissioned) OECD report "Improving Schools in Scotland: An OECD Perspective"cites the SSLN data fully 27 times and at no point suggests the data is in any way not fit-for-purpose³.

It's hard not to conclude that the SNP's decision to stop the SSLN is motivated by the fact that it exposes failings on their part - and that they're fearful of what the next ones would show.  Put it like this: do you think they'd scrap it if they thought the upcoming surveys were going to show how successful their reforms have been?

It gets worse. The Scottish Government states "New statistics on literacy and numeracy performance will be available annually from the teacher professional judgement data collection". So not only are we losing the ability to track performance versus prior years on a like-for-like basis, we're moving from largely objective test-based measurements to subjective measures "based on teachers' professional judgements". It's almost as if the SNP are going out of the way to avoid being judged objectively on their performance.

***

So how does the SNP attempt to put a positive spin on this?

In that Andrew Neil interview Sturgeon used the following three arguments, none of which withstand scrutiny

1. "If you take Level-5 qualifications which are broadly equivalent to the old O-Grades and Standard Grades [..] we've seen the gap between the richest pupils and the poorest pupils almost half"

I assume here she's referring to the National Improvement Framework: 2016 Evidence Report which includes this analysis

If I'm right, then she's dismissing robust PISA data from 2 years ago but seizing on a snippet of data from - er - 2 years ago. Either way, the measure of "one or more qualifications as SCQF Level 5 or better" is laughably crude compared to the SSLN data.


2. "If you take the tariff scores in Scottish education which measure not just the quantity of qualifications that young people get but the quality as well, it shows that performance in the top 20% has improved by about 9% but the performance in the bottom 20% has improved by 26%"

The data she quotes clearly comes from the National Benchmarking Overview Report 2015/16:

This is certainly a measure of something, and quite possibly of something to be applauded. But the report itself offers some pretty heavy caveats about the crudeness of the measure: "measures of both attainment and outcome for education are presently crude and need developed [..] “Total tariff score” needs controlled for the year in which young people left school so that we compare like-with-like". So this crude measure that confesses not to be offering like-for-like figures is in no way an alternative to the SSLN data, which it appears to directly contradict.

The Tariff Score is explained as "a summary measure of the number, level and grade of qualifications children had achieved during the senior phase". A bit more digging tells us its based on "the attainment of all pupils who left school that year after either S4 S5 or S6 by level of deprivation. Each qualification attained by a pupil is awarded tariff points based on its SCQF level and credit points. Points are also based on the grade of award achieved."

There are two glaringly obvious weaknesses with this measure compared to the SSLN data

• It only offers a crude measure of what happens at the end of the education pipeline, it offers no insight into issues in the pipeline. Take the P4 numeracy issues exposed by the SSLN survey: these won't translate into "tariff points" for another 8 years or so!
• The impacts of mix (for example if students are encouraged to take easier subjects) and grade inflation (which may differ by grade within subject as well as across subjects) will be near impossible to allow for

To seize on this tariff data as proof of something good happening (which it might be) while simultaneously dismissing the transparently more robust and strategically valuable  SSLN data is simply indefensible.

3. "we're seeing more young people coming out with Highers and Advanced Highers, we're seeing more young people going in to University, we're seeing the positive destinations of young people continuing to increase"

Claims on numbers of Higher and admissions to University (which I don't have the energy to check, but believe) in no way counter the core issue of declining educational standards highlighted by PISA and the chronic Attainment Gap problem as shown in the SSLN data. Those surveys provide like-for-like comparisons over time, crudely counting number of Higher passes or number of people getting in to University obviously doesn't.

As for "positive destinations", this appears to be little more than a proxy for "not being unemployed" given the term is defined as including "higher education, further education, training, voluntary work, employment and activity agreements"

***

So Sturgeon did a good job of spinning her way through the interview, but the underlying issue can't be spun away so easily.

Don't get me wrong: I believe that Sturgeon and her colleagues care about education and I believe that after nearly 10 years in power they've finally really started trying to improve things. But I also believe that it is not and never will be their main priority because - as the party's constitution and Sturgeon herself has made very clear - for them, independence transcends all.

If you want proof of the SNP's culpability (beyond that provided by the PISA and SSNL data), read this report on John Swinney's appearance in front of Holyrood's Education Committee, by Angus Howarth in the Scotsman:

Cuts in the number of teacher training places in Scotland “probably” went too far, the Education Secretary has conceded. John Swinney said that with hindsight, the target intake for student teachers was “probably over-corrected too far” in 2011.

In 2005-06, before the SNP came to power, the target number was 4,437, but it then went from 3,857 in 2009-10 to 2,307 in 2010-11 - a drop of 1,550 places.

Alternatively take a look at our old friend the Scottish Government's GERS figures. As I've pointed out before, the SNP relatively reduced per capita spending on education (relative to the rest of the UK) in their early years. This is all about priorities: in other areas like policing they chose to spend more.

Of course this graph shows that we've continued to spend more per capita on education than the rest of the UK - but our population density is 80% lower than the rest of the UK and we have extensive island communities to serve, so we'd expect that. What's undeniable is that the SNP reduced spend relative to the rest of the UK (so "Tory austerity" can't be blamed) and have only very recently looked to return to the levels of spend they inherited (in a move some of us applauded at the time).

[A more compete analysis of spend, teacher numbers, class sizes etc. can now be found >here]

Ultimately its about priorities and education simply hasn't been a priority for the SNP until very recently. I fear that hollow sound you can hear is a stable door being shut on an empty stable.

---

It would be unfair to suggest the SNP have neglected our children entirely. Children are future voters after all, so when it comes to marketing "brand SNP" in schools, we should give them their due: they do a great job

---

***

There can be few responsibilities greater for a party of government than to ensure a country's children receive the best possible education, to provide the next generation with the best chances to succeed in life.

Tony Blair swept to power in 1997 promising to make "education, education, education" his top priorities in office; the Scottish education system appears to be suffering the effects of having an SNP government in power with "independence, independence, independence" as theirs.

Young Scots emerging from education with reduced life-chances are the ones paying the price for the SNP's failings today. The question is, will the SNP start paying the price at the ballot box tomorrow?

***

---

Notes

---

1. Programme for International Student Assessment (PISA)

The global benchmark standard for education is provided by the OECD's PISA:

The Programme for International Student Assessment (PISA) is a triennial international survey which aims to evaluate education systems worldwide by testing the skills and knowledge of 15-year-old students. In 2015 over half a million students, representing 28 million 15-year-olds in 72 countries and economies, took the internationally agreed two-hour test.

As the Scottish Government website explains:

PISA seeks to measure skills which are necessary for participation in society. Accordingly, it assesses how students apply the skills they have gained to the types of problem they may encounter in work or elsewhere. Pupils are assessed at the age of 15 as this is regarded as a reasonable point at which to test the impact of compulsory education throughout the developed world (most PISA 2012 participants in Scotland were attending S4).

I have used the PISA "data explorer" tool to download and play with the data that's available as a consistent time-series. Fortuitously when it comes to judging the SNP's record, this means our data set starts in 2006, the year before the SNP came to power.

The data is survey based and provided as score averages. As with all good statistical analysis, standard error figures are given. This means our graphs can include vertical bars to show the range within which we can be 95% confident the true figure lies.

A minor irritation is that I've not been able to clarify if the UK data includes Scotland - by implication it does, but either way it doesn't change the overall message (in fact at worst if UK does include Scotland then it dampens the severity of the observed relative trend).

More detail on Scotland's PISA results can be found on the Scottish Government website.

2. Scottish Survey of Literacy & Numeracy (SSLN)

As the Scottish Government website explains:

The SSLN was a sample-based survey which monitored national performance over time in literacy and numeracy at P4, P7 and S2. It provided a snapshot of Scotland's achievement in literacy and numeracy at a specific point in time and allowed for comparisons over time to be made. The information from the survey informed the development of dedicated resources to facilitate improvements in learning and teaching.

Whilst the data is survey based, it qualifies for National Statistics certification and rigorous statistical techniques are used to to ensure the data is presented with meaningful confidence intervals. As the latest report explains in helpful detail

As in all sample surveys, as the SSLN is based on a sample of pupils rather than on the
whole population, the results shown are estimates. Therefore there is an element of
uncertainty within the results because the pupils sampled may not reflect the population
exactly.

Uncertainty around the results is estimated using standard errors. Standard errors are a
measure of the variation in the data i.e. how each observation differs from the mean. As
the SSLN sample design is not a simple random sample - pupils at small schools have a
higher probability of being selected than pupils at large schools - this means that standard formulae used to calculate the standard error from a simple random sample would not be appropriate. Standard errors are therefore calculated empirically using the jackknife procedure.

Standard errors are in turn used to produce confidence intervals around the estimates.
Confidence intervals show the range of values within which one can be reasonably
confident that the actual value would lie if all pupils were assessed. Ninety-five per cent confidence intervals for the main national estimates were calculated and were around ± two percentage points. This means that the true value of each estimate is likely to lie within two percentage points either side of the given estimate.

Where appropriate, confidence intervals are represented on charts by error bars to help
demonstrate this level of uncertainty. Where the estimates are different but the error bars
overlap we cannot be sure that the true values of each estimate are statistically
significantly different from each other. Significance tests (t-tests) are used to assess the
statistical significance of comparisons made.

3. The only observation the OECD make relating to limitations of the SSLN data is "The SSLN sample size is too small to be disaggregated to the local authority level." . This is true, but a reason to gather more comprehensive data as well as the SSLN, not instead of.