Immutable raw
Provider-native payload, source URL, retrieval time, response metadata, licence and SHA-256 receipt. Never overwritten.
Data strategy
Record once. Rebuild transparently.
A three-layer architecture for current monitoring and the 2016–2025 historical programme.
Bronze · Silver · Gold
Preserve the source; make every derived layer rebuildable.
Canonical observations
Station identity, pollutant, value, unit, observation period, quality/status flags and retained provider identifiers. Partitioned Parquet/GeoParquet.
Reviewed products
Monthly and annual aggregates, completeness, trend tables, contextual comparisons and public report datasets, versioned by release.
Historical acquisition
Ten complete years first.
Prioritise 2016–2025 as one reproducible unit. Discover provider-native interfaces, download in resumable chunks, preserve originals, normalise monthly, reconcile station identities and publish annual receipts.
AURN / UK-AIR
Core regulated ambient measurements and first historical priority.
NPL particle networks
Particle number, black carbon and composition where available; separate from mass-only PM reporting.
NAEI / compliance
Emissions and statutory context maintained as separate layers, never substituted for ambient observations.
Meteorology / models
Time-aligned contextual joins with observed-versus-modelled semantics kept explicit.
Operational rule
Current and backfill pipelines should not be coupled.
Daily current evidence must remain fast and fail-closed. Decade-scale recovery should run independently, with resumable provider-aware chunks and explicit gap ledgers.
Google Drive is suitable as backup/archive storage for reviewed artefacts and raw historical payloads, but should not be treated as an evidential source or mandatory dependency for core validation.