Notes on LocalGov spending data
Local authorities (LA) in England are encouraged by central government to publish details of all expenditure over £500. This data is downloaded, converted to a standardised format on a central database and republished on the AppGov website.
Data loading
Data is being loaded from January 2011 onwards for each LA but it will be some time before the data for all 350 English authorities has been loaded. You can see which authorities have data available from the LA selection page. Not all LAs provide historical data back to 2011.
Data format
Each LA publishes its spending data in a different format (and may itself use several different formats over time!) but these are all converted into a single standard layout for use on the AppGov website.
The type of data supplied by each authority also varies widely - from 2 to over 20 separate data fields for a transaction record. The standard set of data fields on the AppGov website encompasses most of the possible data types. Apart from Amount, Supplier and Date there is no guarantee that data fields are comparable across LAs.
Data values
Care has been taken to ensure that the data is accurately transcribed from the LA download files to the AppGov database but there are a few cases where this is not possible -
Occasionally a default value has to be assumed e.g where no invoice date or an invalid date is provided then the first of the month is used. Where an unrecognisable character encoding has been employed (e.g for e acute) then an appropriate character is inserted or it is left blank. Leading/trailing spaces in text fields and non-numeric characters in numeric fields are deleted. Header, footer and total rows from the input file are ignored. Blank Amount fields are set to £0.0.
A few authorities include VAT in their amount values. In these cases the VAT amount is deducted so that amount values on AppGov are comparable across LAs. This process results in some small rounding errors with the consequence that the total amounts in any one month for these authorities might be slightly inaccurate. Where VAT has been deducted the Vatinex field in the Spend record is set to "20% VAT removed". Where an LA publishes separate values for unrecoverable VAT this has been ignored.
Cases of particularly anomalous data (such as invoice dates in the far future or more than 6 years in the past, very large payment amounts or missing supplier names) are sometimes queried with the relevant local authority and the data is corrected if necessary.
LAs may use different names for the same supplier.
Personal names for individuals may be redacted (although the date and amount will still be shown) although some LAs do not publish redacted records at all. In addition, some personal names have been redacted by the AppGov system. The number of redacted records for each LA can be seen from the Type of LA pages eg London Boroughs.
Data fields - notes
Date: This is normally the date that the invoice was paid. The first of the month is assumed if the date is not provided. Excel 5 digit date numbers are converted to calendar dates. Date errors are probably the result of incorrect OCR processing and are usually left uncorrected.
GSS & SNAC codes: See ONS coding system.
File formats
Most authorities make their data available as CSV, some only supply XLS/XLSX and a few (notably Enfield & Southwark) used to publish only in PDF format. Spend data is only loaded from CSV or XLS/XLSX files.
Guidance documentation
DCLG has published a Local authority data transparency code, the LGA has detailed guidance in its Local transparency guidance and a summary used to be available on data.gov.uk
Licensing for data reuse
Authorities are encouraged to license their spending data for reuse under the Open Government Licence and most of them do so while a few authorities have their own licences with similar terms. AppGov is pleased to acknowledge the relevant licence details for each authority wherever possible and does not claim any endorsement or official recognition in its use of their data.
For more information or to make suggestions please contact info@appgov.org