Sunday 11 June 2017

General Election 2017: The Scottish Constituencies

A quick crunch of the figures and some throwaway observations:
  • The "unionist" parties (Conservatives, Labour and Lib Dems) had a 62.4 % overall vote share and greater than a 50% vote share in every constituency in Scotland

  • The SNP's 36.9% share in Scotland was lower than Labour's 40.0% vote share across the UK -- but won them 59% of seats

  • With 27% of the vote, Labour won just 12% of seats; the Conservatives with 29% of the vote won 22% of seats, the Lib Dems with 7% of vote won 7% of seats

  • The Scottish Conservatives received just a 1.5% higher vote share than Scottish Labour by dint of getting 5.7% more votes [757k vs 717k]

  • Scottish Labour were second in 24 constituencies, Conservatives in 10 and Lib Dems in 1 (by just 2 votes)

  • The SNP came second in every seat they lost

  • There were four constituencies with winning margins of less than 100 votes -- these were all won by the SNP

  • Across the 6 constituencies which the SNP won with the lowest margin of victory, they cumulatively won by just 619 votes (or an average of 102 votes per constituency) -- a swing of just 310 votes nationwide would have seen the SNP lose their claim to a majority of Scottish seats at Westminster

  • The lowest winning constituency vote share was the SNP's 32.6% in Lanark & Hamilton East where the Tories polled 32.1%, Labour 31.9%

  • The largest winning margin was comfortably Labour's Ian Murray's 15,514 majority in Edinburgh South; with a 54.9% vote share he was 32.4% ahead of  the SNP who polled just 22.5%

  • The only other constituency won with a majority vote share was the Scottish Conservative's John Lamont who won Berwickshire, Roxburgh and Selkirk with 53.9% of the vote, a majority of 11,060 over the SNP who polled 32.8%

  • The SNP's Vote share dropped from 50.0% in 2015 to 36.9% in 2017; their total votes dropped by a third [from 1,454,436 to 977,569]

Here's the raw data in tabular and graph format - if I've made any errors let me know and I will of course correct









Tuesday 6 June 2017

Scottish Education: A Generation, Failed

A summary of my recent analysis on the data around the SNP's performance on education appeared in today's Daily Record, along with a broader perspective and some caveats (which generally I agree with) by James McEnaney


Unfortunately one graph appeared mislabelled in print (graph 5 is "writing" not "numeracy") and the online version get's a little confused with graphs and sources - so (with some very minor tweaks) the below is the text of the article with full source links, exhibits we didn't have room for in the paper and references to James' caveats.


*********
Judge SNP on Education? They Flunked it

"Let me be clear – I want to be judged on this. If you are not, as First Minister, prepared to put your neck on the line on the education of our young people then what are you prepared to. It really matters."
Nicola Sturgeon, August 2015 
Nicola Sturgeon has asked to be judged on her record on education and the SNP has been in charge of our fully devolved Scottish education system for roughly 10 years now. Responsibility for the educational outcomes we're now seeing must lie with the SNP - and the trend in those outcomes is simply terrible, as Cabinet Secretary for Education and Skills John Swinney has had to admit.

"the statistics also show a drop in writing performance for S2 pupils which is of particular concern [..] the SSLN [literacy and numeracy] statistics are disappointing"
John Swinney, May 2016
"The figures for Scotland do not make comfortable reading [..] compared to 2012, our performance in science and reading has fallen.  In science and maths we are now below the levels at which we performed in 2006, and more countries have outperformed Scotland in all three areas than at any time since PISA began." John Swinney December 2016

The Evidence of Falling Standards

Scotland participates in one international benchmark survey on educational attainment, the three-yearly PISA study1. PISA surveys 15-year-old pupils, so the most recent data (for 2015) was the first chance to see how pupils who had spent significant time in an SNP-run education system were faring, as these pupils had been in the Scottish education system for 7 years under an SNP administration.

Since the SNP came to power, in each of the three main disciplines as measured by PISA (mathematics, reading & science), Scotland's performance has declined significantly. This is true in absolute terms, relative to the whole of the UK and relative to the average for all OECD countries.

In maths and reading, where Scottish pupils used to perform significantly better than UK students overall, latest available data shows we've dropped back to average at best. In science, where Scottish pupils used to perform in line with the UK average, their performance in the most recent survey is very significantly worse. In all three disciplines, we've gone from out-performing the OECD average to being - well - distinctly average.




If were to you look at our performance in international league-table format: in all three subject areas we've dropped below the UK and OECD averages and into the bottom half of the table2.

As the Scottish Government themselves make clear, "PISA is the major international study of pupil performance in which Scotland participates" - it is their chosen method of benchmarking our educational attainment.

In fact, having withdrawn from the other globally recognised assessment body (TIMSS & PIRLS) in 2011, it's now the only data we have to compare our performance on an international basis. This is good quality data and the results are a damning indictment of the SNP's performance as custodians of our education system.


Caveat: James McEnaney argues that PISA data "is certainly not without its problems", concluding "although PISA has done good things for global education [..] it is a very long way way from being perfect data". I don't dispute that the data is far from perfect, but it is the only international benchmark data we have and the trends observed are consistent across all three subject areas and broadly consistent with the directional message from the SSLN data below. I don't think the data can be ignored (and don't think James is suggesting we ignore it).


The other way we have been able to objectively assess the performance over time of the Scottish education system is the Scottish Government run Scottish Survey of Literacy and Numeracy3 (SSLN).

This data provides worrying evidence of declining performance and wide (and in some cases widening) Attainment Gaps between the performance of pupils from the most deprived and least deprived backgrounds.

To illustrate, let's looks at numeracy performance and the Attainment Gap for P4 (7-8 year-old) pupils. The graph clearly shows standards are dropping and the Attainment Gap is growing.


Another striking example is writing performance for S2 (13-14 year-old) pupils. Here the graph shows overall performance is declining dramatically and the Attainment Gap remains disturbingly large.


There’s a lot more in the SSLN data4. For example it also shows not just a worryingly large Attainment Gap, but also that only 40% of S2 pupils are judged to be performing “well” or “very well” in numeracy. Is our education system producing a generation, soon to vote, who aren't great at adding up?



Caveat: James McEnaney rightly points out that the S2 numeracy data is judging pupils against S3 student's benchmarks, so I guess my cheap line on "not great at adding up" is rather harsh. He also flags concerns with the writing assessment methodology. He knows the data (and the subject) better than I do, so I'm in no position to question his judgement on that.

He does however go on to say "problems with some of the information don't change the downward trends. The SSLN data certainly suggests that the gap between the most-deprived and least-deprived children is, at best, not being closed. It also raises real concerns, shared by teachers, about literacy and numeracy in schools". I think that's a fair summary of the point I'm trying to highlight with this analysis. The full set of graphs is listed under note 4 below - so you can judge for yourself.


But there’s no danger of the SNP being embarrassed by the next set of SSLN data, because – incredibly - they’ve simply decided to stop the survey altogether. They justify this by saying that, because it’s survey based, the data doesn’t provide useful information at a school or local authority level. That’s a frankly ridiculous argument, because that was never what the survey was intended to do.

SSLN gives an overall assessment of the quality of literacy and numeracy education in Scotland, and it does that well. The Scottish Government commissioned the OECD to write a report "Improving Schools in Scotland: An OECD Perspective". That report cites the SSLN data fully 27 times and at no point suggests the data is in any way not fit-for-purpose.

It's hard not to conclude that the SNP's decision to stop the SSLN survey is motivated by the fact that it exposes failings on their part - and that they're fearful of what the next ones might show. Do you think they'd scrap it if they thought the upcoming surveys were going to show how successful their reforms have been? To coin a rather poignant phrase in this context: you do the math.

It gets worse. The Scottish Government states "New statistics on literacy and numeracy performance will be available annually from the teacher professional judgement data collection". So not only are we losing the ability to track performance versus prior years on a like-for-like basis, we're moving from largely objective test-based measurements to subjective measures "based on teachers' professional judgements". It's almost as if the SNP are going out of the way to avoid being judged objectively on their performance.

Combine the SSLN findings with the PISA results and a clear picture emerges: on the SNP's watch, overall standards in Scottish education have dropped and pupils from the most deprived areas are the ones suffering the most.


The Case for the Defence

When grilled on this topic by Andrew Neil during a recent BBC interview, Sturgeon offered a master-class in political spin. She brushed off references to the latest available international benchmark data (reflecting 7 years of SNP control) as "from two years ago" and airily dismissed her own Government's regular survey of over 10,000 pupils as somehow irrelevant.

She even went on to argue that "there's real progress been made". To justify this she fired out a barrage of stats on exam passes and “tariff scores”, both crude measures which suffer from obvious weaknesses when used to judge the performance of our education system. There is no robust way to adjust for the issue of “grade-inflation” (lowering pass marks) and the mix of exams taken, but more importantly these are measures of the end of the educational pipeline. If, for example, there are issues with numeracy at P4 (as the SSLN survey suggests), we need to know about it now, not wait until those pupils leave school in 8 or 9 years’ time.

The other measure seized on by Sturgeon is the number of pupils leaving school for "positive destinations”. This basically means not ending up unemployed or in jail - so if your kids leave school for a minimum wage job on a zero-hours contract, our First Minister will celebrate that as having achieved a “positive destination”.


Who's Really to Blame?

The SNP would of course like to blame “Westminster austerity” for these failings, but that doesn’t wash. The PISA scores show this is an issue specific to Scotland.

The reality, as revealed in the Scottish Government’s own GERS figures, is that the SNP have cut spending on education & training in Scotland over the last 9 years by on average 8% more than it's been cut in the rest of the UK. That was a political choice.


Given the education & training budget has to fund expensive flagship policies like free university tuition, it's not surprising that schools have suffered badly over this period. Primary schools have been particularly badly hit5.

Action has only recently been taken to redress the balance, but the SNP’s response is too little, too late for the generation that has been disadvantaged as a result of education spending  being low on the SNP's priority list.


Over the period the SNP have been in power, teacher numbers in Scotland have declined by 8% and class sizes have increased by 6% - but the trend in England is reversed, with both teacher numbers and class sizes improving6.


The data is clear: the SNP - not “Westminster austerity” - are to blame for failures to invest appropriately in our Scottish education system.


Caveat: James McEnaney rightly argues that - while the data is valid - "we mustn't lose sight of the that fact that it is just one part of a very large and almost uniquely complex picture". I don't disagree with that: wider socio-economic issues are certainly a factor, in particular the broader issue of poverty. I agree with James when he says "the big picture of Scottish education is about much more than just the failures of Nicola Sturgeon and Alex Salmond". It's about more than the SNP's bad choices around specific issues relating to education - but those bad choices are a significant factor


Reflection

There can be few greater responsibilities for a party of government than to ensure its country's children receive the best possible education, to provide the next generation with the best chances to succeed in life. We know that for the SNP, educating our children will never be their top priority because - as Sturgeon herself and the party's constitution make clear - independence transcends all.

Tony Blair swept to power in 1997 promising to make "education, education, education" his top priorities in office; the Scottish education system appears to be suffering the effects of having an SNP government in power with "independence, independence, independence" as theirs.

Unfortunately it's young Scots emerging from education with reduced life-chances who are the ones paying the price for the SNP's failings today. The question is, will the SNP start paying the price themselves at the ballot box tomorrow?

*******


Notes


1. Programme for International Student Assessment (PISA)

The global benchmark standard for education is provided by the OECD's PISA:
The Programme for International Student Assessment (PISA) is a triennial international survey which aims to evaluate education systems worldwide by testing the skills and knowledge of 15-year-old students. In 2015 over half a million students, representing 28 million 15-year-olds in 72 countries and economies, took the internationally agreed two-hour test.
As the Scottish Government website explains:
PISA seeks to measure skills which are necessary for participation in society. Accordingly, it assesses how students apply the skills they have gained to the types of problem they may encounter in work or elsewhere. Pupils are assessed at the age of 15 as this is regarded as a reasonable point at which to test the impact of compulsory education throughout the developed world (most PISA 2012 participants in Scotland were attending S4).
I have used the PISA "data explorer" tool to download and play with the data that's available as a consistent time-series. Fortuitously when it comes to judging the SNP's record, this means our data set starts in 2006, the year before the SNP came to power.

The data is survey based and provided as score averages. As with all good statistical analysis, standard error figures are given. This means our graphs can include vertical bars to show the range within which we can be 95% confident the true figure lies.

A minor irritation is that I've not been able to clarify if the UK data includes Scotland - by implication it does, but either way it doesn't change the overall message (in fact at worst if UK does include Scotland then it dampens the severity of the observed relative trend).

More detail on Scotland's PISA results can be found on the Scottish Government website.


2. PISA Rankings



3.. Scottish Survey of Literacy & Numeracy (SSLN)

As the Scottish Government website explains:
The SSLN was a sample-based survey which monitored national performance over time in literacy and numeracy at P4, P7 and S2. It provided a snapshot of Scotland's achievement in literacy and numeracy at a specific point in time and allowed for comparisons over time to be made. The information from the survey informed the development of dedicated resources to facilitate improvements in learning and teaching.
Whilst the data is survey based, it qualifies for National Statistics certification and rigorous statistical techniques are used to to ensure the data is presented with meaningful confidence intervals. As the latest report explains in helpful detail

As in all sample surveys, as the SSLN is based on a sample of pupils rather than on the
whole population, the results shown are estimates. Therefore there is an element of
uncertainty within the results because the pupils sampled may not reflect the population
exactly.
Uncertainty around the results is estimated using standard errors. Standard errors are a
measure of the variation in the data i.e. how each observation differs from the mean. As
the SSLN sample design is not a simple random sample - pupils at small schools have a
higher probability of being selected than pupils at large schools - this means that standard formulae used to calculate the standard error from a simple random sample would not be appropriate. Standard errors are therefore calculated empirically using the jackknife procedure.

Standard errors are in turn used to produce confidence intervals around the estimates.
Confidence intervals show the range of values within which one can be reasonably
confident that the actual value would lie if all pupils were assessed. Ninety-five per cent confidence intervals for the main national estimates were calculated and were around ± two percentage points. This means that the true value of each estimate is likely to lie within two percentage points either side of the given estimate.
Where appropriate, confidence intervals are represented on charts by error bars to help
demonstrate this level of uncertainty. Where the estimates are different but the error bars
overlap we cannot be sure that the true values of each estimate are statistically
significantly different from each other. Significance tests (t-tests) are used to assess the
statistical significance of comparisons made.

4. Full set of SSLN graphs

Reading



Writing




Numeracy





5. Data on Spend/Pupil
Spend per pupil is highlighted in the National Benchmarking Overview Report 2015/16:




6. Data on Teacher Numbers

There's an excellent fact-check by Ferret Scotland on this topic which shows teacher numbers in Scotland have declined by 4,000 since the SNP came to power - that's a 7.5% decline.

The most recent stats on school teacher numbers for England only go up to 2015: over the 2007 to 2015 period they show FTE "regular teacher" numbers increasing by 4%, or nearly 16 thousand teachers. The same period in Scotland saw teacher numbers decline by 8%.

Looking at pupil:teacher ratios back to 2007 takes a bit of work. Summary Statistics for Schools in Scotland shows the data from 2010 and shows pupil:teacher ratios increasing (ie getting worse, as we might expect given the scale of the teacher number decline)


It's a bit of a faff to go back to 2007, but I've done the work so here it is:


I've struggled to get fully comparable stats for England, but this data suggest (as we'd expect from the teacher number stats) that pupil:teach ratios (PTR) are decreasing (improving) in England. This table may be hard to read, but it shows
  • LA Maintained Primary PTR decreased from 21.9 in 2007 to 21.0 in 2015
  • LA Maintained Secondary PTR decreased from 16.5 in 2007 to 15.8 in 2015


Sunday 4 June 2017

Scottish Education: SNP Neglect or Tory Austerity?

*************************
Note: this blog post is now more neatly summarised here: Scottish Education: A Generation, Failed
*************************



This is not a carefully crafted blog post, it's simply me sharing "work-in-progress" as I dig a little further into the issues underlying the record of failure in Scottish education. I'm sharing my work as I ask the following question: to what extent is it fair to blame the SNP for the worrying trends in Scottish pupils' educational attainment?

There are issues that pre-date the SNP's leadership and there are issue that are UK-wide, so I want to see if there is sound evidence that the SNP themselves have made decisions to de-prioritise education.

First of all a bit of house-keeping. In my recent blog Judging the SNP on Their Education Record, I'd labelled some SSLN graphs as being "Literacy" attainment when in fact they were just the Reading element of Literacy. For completeness then, here is the full set of graphs for Numeracy, Reading and Writing*.

* the stats for "Listening & Talking" were only introduced 2014, so I haven't taken the time to graph these

To read these graphs: the dotted black line shows the overall performance over time, the gap between the red (least deprived) and green (most deprived) lines illustrates the Attainment Gap.


P4 (age 7-8): Reading, Writing & Numeracy
Little evidence of sustained improvements in Reading & Writing and a worrying drop in Numeracy performance particularly among children from the most deprived backgrounds.







P7 (age 10 - 11): Reading, Writing & Numeracy
Reading performance stable and with a relatively small Attainment Gap, but declines in Writing and Numeracy performance, along with widening Attainment Gaps.






S2 (age 13 - 14): Reading, Writing & Numeracy
Reading performance relatively stable, with no sustained progress on closing the Attainment Gap. Dramatic decline in Writing performance, with the Attainment Gap only slightly narrowing because of faster decline in performance by pupils from least deprived backgrounds. Numeracy overall stable at a low level of performance and with no progress in closing what is the largest Attainment Gap of all.




OK, so that's tidied up the evidence on performance. Combine the PISA data from the previous blog post with the SSLN data and we can fairly conclude that, overall, standards are slipping and Attainment Gaps remain high and are growing more than they're closing. Something's going badly wrong.

So what evidence do we have that the SNP have taken actions that have caused this? There are of course questions about the wider strategic approach with the Curriculum for Excellence, but I'm focusing here on hard, objective data. In particular I'm looking to see differences between Scotland and England, as that provides evidence that what we're seeing here can't simply be dismissed as the knock-on effect of "Tory Austerity".

Spending

Using our old friend the GERS figures, we can look at the different trends in spending on education and training (i.e. more than just schools) between Scotland and the rest of the UK. Given the Barnett Formula ensures that Scotland has the capacity to maintain spending in this area in line with the rest of the UK, any decision to pursue a different path is the result of SNP policy decisions.

If we look at indexed spend per capita we can illustrate the trend differences between spending on Education & Training in Scotland and the rest of the UK. 

This shows there is little difference between Scottish and rUK indexed spend in 2015-16 (both are down about 7.5% vs 2006/7). However, in the intervening years on average Scotland lagged rUK spend by 8% - Scotland didn't increase spending when rUK did, and it's just in the last year that the SNP has finally taken action to increase real-terms spend and close the gap


It's not just how much that has been spent of course, but what it's been spent on. We know the Scottish Government chooses to spend some education & training money to support their flagship "no tuition fees" policy for example - so let's looking at schooling specific spend.

Unsurprisingly in this context, spend per pupil is down in primary and secondary education in Scotland, as highlighted in the National Benchmarking Overview Report 2015/16:

This data shows Scottish spend/pupil between 2010-11 and 2015-16 declining by 9.4% in Primary and 2.2% in Secondary. In both cases there has been a "shutting-the-stable-door-after-the-horse-has-bolted" looking increase in spend just in the last year (consistent with the overall education & training spend data above)



Teacher Numbers

There's an excellent fact-check by Ferret Scotland on this topic which shows teacher numbers in Scotland have declined by 4,000 since the SNP came to power - that's a 7.5% decline.

The most recent stats on school teacher numbers for England only go up to 2015: over the 2007 to 2015 period they show FTE "regular teacher" numbers increasing by 4%, or nearly 16 thousand teachers. The same period in Scotland saw teacher numbers decline by 8%.

Looking at pupil:teacher ratios back to 2007 takes a bit of work. Summary Statistics for Schools in Scotland shows the data from 2010 and shows pupil:teacher ratios increasing (ie getting worse, as we might expect given the scale of the teacher number decline)


It's a bit of a faff to go back to 2007, but I've done the work so here it is:


I've struggled to get fully comparable stats for England, but this data suggest (as we'd expect from the teacher number stats) that pupil:teach ratios (PTR) are decreasing (improving) in England. This table may be hard to read, but it shows
  • LA Maintained Primary PTR decreased from 21.9 in 2007 to 21.0 in 2015
  • LA Maintained Secondary PTR decreased from 16.5 in 2007 to 15.8 in 2015


***

To summarise: the SNP have cut spending on education & training in Scotland over the last 9 years by on average 8% more than it's been cut in the rest of the UK.

Unsurprisingly, over the same period teacher numbers in Scotland have declined (by 8%) and pupil:teacher ratios (a proxy for class sizes) have increased (by 6%). The reverse has happened in England, with teacher numbers up (by 4%) and pupil:teacher ratios improving.

The Barnett formula ensures Scotland has the option to follow the rest of the UK when investing in schooling; the hard evidence shows that the SNP has simply chosen not to. 

The area worst hit in terms of teacher numbers and pupil:teacher ratios has been Primary school education. If we rely on school-leaver "tariff scores" and exam passes to measure our education system's performance, it will be a long while before the true cost of these cuts is seen.

One of the factors driving the decrease in teacher numbers in Scotland has clearly been a failure to train enough teachers. Witness this recent exchange between Daniel Johnson and John Swinney at the recent Education and Skills Committee (31 May 2017).


In summary: Daniel points out the target numbers for student teachers nearly halved from 4,437 in 2005/6 to 2,300 in 20011/12 and we currently have 700 teacher vacancies - the SNP has simply failed to train enough teachers.

In response John Swinney admits:
"Clearly with the benefit of hindsight the intake number in 2011 was probably over-corrected too far ..."
***

I started this rambling blog by asking the question: to what extent is it fair to blame the SNP for the worrying trends in Scottish pupils' educational attainment?

Based on the evidence above I'd suggest the conclusion is pretty clear: the SNP (and not "Tory austerity") are to blame for failing to invest appropriately in our Scottish education system