CSO & Area Data Sources

Last updated: 2026-03-30 Map URL: http://cathaldev3.fcrweb.ie/ Data location: data/map/ (local) + fcrwatcher S3 bucket (roamworkflow/map-data/)

1. Data Sources

1.1 Census 2022 Small Areas (CSO)

Field	Detail
Source	Central Statistics Office — Small Area Population Statistics (SAPS) 2022
Download	`data/cso/SAPS_2022_Small_Area_270923.csv` (793 columns, 18,919 rows)
Granularity	Small Area — smallest census geography (~80–150 households each)
Coverage	All 26 counties, 18,919 small areas
Freshness	Census 2022 (static until Census 2027)
Licence	Open Data, CSO Open Data Licence

Raw fields used:

CSO Field	Description	Table
`T1_1AGETM` / `T1_1AGETF`	Total male/female population	T1 — Age
`T6_1_TH`	Total households	T6 — Housing
`T6_1_HB_H` / `T6_1_FA_H`	Houses / Flats (household count)	T6 — Housing
`T6_3_OMLH`	Own with mortgage/loan (households)	T6 — Tenure
`T6_3_OOH`	Own outright (households)	T6 — Tenure
`T6_3_TH`	Total households (tenure base)	T6 — Tenure
`T8_1_WT`	At work (persons)	T8 — Employment
`T8_1_TT`	Total 15+ population (employment base)	T8 — Employment
`T9_2_HA`	Professional workers (households)	T9 — Social class
`T9_2_HB`	Managerial & technical (households)	T9 — Social class
`T9_2_HT`	Total households (social class base)	T9 — Social class
`T10_4_HDPQT`	Honours degree / professional qualification (persons)	T10 — Education
`T10_4_PDT`	Postgraduate (persons)	T10 — Education
`T10_4_DT`	Doctorate (persons)	T10 — Education
`T10_4_TT`	Total 15+ population (education base)	T10 — Education
`T15_1_2C` / `T15_1_3C` / `T15_1_GE4C`	Households with 2/3/4+ cars	T15 — Cars
`T15_1_TC`	Total households (cars base)	T15 — Cars

1.2 CSO Small Area Boundaries (GeoJSON)

Field	Detail
Source	Ordnance Survey Ireland via CSO
File	`data/cso/small_areas_boundaries.geojson`
Used for	Centroid coordinates (lat/lng per small area)
Coordinate system	Irish Transverse Mercator (ITM) — converted to WGS84
Conversion	`data/cso/fix_coords.py` — custom ITM→WGS84 projection using GRS80 ellipsoid

1.3 Pobal HP Deprivation Index (2022)

Field	Detail
Source	Pobal (funded by Dept. of Rural & Community Development)
File	`data/cso/pobal_2022.csv` (3,418 rows)
Granularity	Electoral Division (ED) — ~5–6 small areas per ED
Coverage	3,418 EDs (3,099 unique names, matches 18,853 of 18,919 small areas)
Key field	`Index22_ED_std_rel_wt` — relative deprivation score
Categories	1=Extremely Disadvantaged, 2=Very Disadvantaged, 3=Disadvantaged, 4=Marginally Below Average, 5=Marginally Above Average, 6=Affluent
Methodology	Weighted composite of: age dependency, lone parents, low education, high education, professional share, lower social class, unemployment (M/F), owner-occupied, private rent, LA rent, persons per room

1.4 Commercial Properties — Tailte Éireann Valuation Office

Field	Detail
Source	Tailte Éireann (formerly Valuation Office) — opendata.tailte.ie
Download script	`data/cso/download_valuation.py`
BQ table	`fcr_operations.valuation_properties`
Records	153,410 rated commercial properties
Fields	property_number, category, uses, county, local_authority, address, eircode, centroid_lat, centroid_lng
Categories	RETAIL (SHOPS), OFFICE, INDUSTRIAL USES, HOSPITALITY, FUEL/DEPOT, LEISURE, MISCELLANEOUS, RETAIL (WAREHOUSE), UTILITY, HEALTH, MINERALS
Coordinates	Originally ITM from Valuation Office, converted to WGS84

1.5 Companies Registration Office (CRO)

Field	Detail
Source	Companies Registration Office bulk data
File	`data/cso/companies.csv` (810,769 rows raw)
BQ table	`fcr_operations.cro_companies_agg` (16,782 rows aggregated)
Granularity	County + NACE sector + registration year
Note	Registered address, not trading address. Aggregated to county level to avoid “1,000 companies at one accountancy firm” problem
NACE codes	2-digit EU economic activity classification (e.g. 69=Legal & accounting, 56=Food & beverage)

1.6 New Dwelling Completions (CSO NDQ07)

Field	Detail
Source	CSO PxStat table NDQ07
API	`https://ws.cso.ie/public/api.restful/PxStat.Data.Cube_API.ReadDataset/NDQ07/CSV/1.0/en`
Granularity	Eircode routing key (3-character, e.g. “X91” = Waterford)
Coverage	139 Eircode areas
Period	Q1 2012 – Q4 2025 (we use 2023–2025)
Total (2023–2025)	98,917 completions
Freshness	Updated quarterly by CSO, ~2 month lag

1.7 Commencement Notices (CSO HSM14)

Field	Detail
Source	Dept. of Housing via CSO PxStat table HSM14
API	`https://ws.cso.ie/public/api.restful/PxStat.Data.Cube_API.ReadDataset/HSM14/CSV/1.0/en`
Granularity	Local Authority
Coverage	31 Local Authorities (mapped to 29 county-level entries)
Period	Jan 2021 – Dec 2025 (we use 2023–2025)
Total (2023–2025)	112,015 units commenced
Freshness	Updated monthly by CSO

1.8 Property Price Register (NOT YET INTEGRATED)

Field	Detail
Source	Property Services Regulatory Authority (PSRA)
URL	propertypriceregister.ie
Granularity	Individual property sale (address + Eircode + price)
Status	Not integrated — no public API or direct CSV download. Form-based download only (Lotus Domino).
Future plan	Monitor for API availability. County+Year download via form POST if automatable. Would provide trailing 12-month transaction volumes and median prices by Eircode.

2. BigQuery Storage

Project: listingmanager-1529856313699 Dataset: fcr_operations

2.1 Tables

Table	Rows	Description
`cso_small_areas`	18,919	Original small areas (geo + basic counts only)
`cso_small_areas_v2`	18,919	Enriched — includes purchasing power fields + Pobal join
`pobal_deprivation`	3,418	Pobal deprivation index at ED level
`valuation_properties`	153,410	Commercial properties with coordinates
`cro_companies_agg`	16,782	Company counts by county + NACE sector + year

2.2 cso_small_areas_v2 Schema

CREATE TABLE fcr_operations.cso_small_areas_v2 (
  sa_geogid        STRING,     -- CSO small area ID (e.g. "A017010016")
  sa_pub           STRING,     -- Public ID (e.g. "017010016")
  county           STRING,     -- County name (UPPERCASE, e.g. "WATERFORD")
  ed               STRING,     -- Electoral Division name
  urban_flag       INT64,      -- 1=urban, 0=rural
  urban_area_name  STRING,     -- Town/city name (blank for rural)
  total_households INT64,      -- Occupied dwellings (Census 2022)
  total_population INT64,      -- Resident population
  houses           INT64,      -- Houses/bungalows
  flats            INT64,      -- Flats/apartments
  centroid_lat     FLOAT64,    -- WGS84 latitude
  centroid_lng     FLOAT64,    -- WGS84 longitude
  -- Purchasing power (derived)
  pct_professional FLOAT64,    -- % professional/managerial social class
  pct_degree_plus  FLOAT64,    -- % honours degree or higher
  pct_homeowner    FLOAT64,    -- % owner-occupied households
  pct_multicar     FLOAT64,    -- % households with 2+ cars
  pct_employed     FLOAT64,    -- % at work (of 15+ population)
  pp_score         FLOAT64,    -- Composite purchasing power 0–71
  -- Pobal deprivation (joined from ED level)
  pobal_index      FLOAT64,    -- Pobal relative deprivation index
  pobal_category   INT64,      -- 1–6 scale
  pobal_label      STRING      -- Human-readable label
);

3. Transformations

3.1 Purchasing Power Score (`pp_score`)

Script: data/cso/build_purchasing_power.py

Input: SAPS 2022 CSV (793 columns) → 5 percentage fields → 1 composite score

Component	Formula	Weight
`pct_professional`	(T9_2_HA + T9_2_HB) / T9_2_HT × 100	25%
`pct_degree_plus`	(T10_4_HDPQT + T10_4_PDT + T10_4_DT) / T10_4_TT × 100	25%
`pct_homeowner`	(T6_3_OMLH + T6_3_OOH) / T6_3_TH × 100	20%
`pct_multicar`	(T15_1_2C + T15_1_3C + T15_1_GE4C) / T15_1_TC × 100	15%
`pct_employed`	T8_1_WT / T8_1_TT × 100	15%

Composite: pp_score = Σ(component × weight) / Σ(weights for non-null components)

Distribution:

Percentile	Score	Interpretation
Min	2.55	Most deprived
P10	23.03
P25	31.18
Median	39.55	Middle Ireland
P75	45.02
P90	49.69
Max	71.26	Most affluent

Important: This is our own derived metric, not an official index. It uses real census counts but the weighting is ours. Pobal’s deprivation index uses similar inputs but different methodology (factor analysis).

3.2 Pobal Deprivation Join

Script: data/cso/build_purchasing_power.py

Join key: cso_small_areas.ed (Electoral Division name) → pobal_deprivation.ED_ENGLISH (case-insensitive UPPER match)

Match rate: 18,853 of 18,919 small areas matched (99.65%). 66 small areas across 3 EDs unmatched due to naming discrepancies:

CASTLETOWN (NORTH)
DUNDALK RURAL (SOUTH)
ST. MARY’S (EAST)

Granularity mismatch: Pobal is at ED level (~3,418 EDs), small areas are finer (~18,919). All small areas within an ED get the same Pobal score.

3.3 Coordinate Conversion (ITM → WGS84)

Script: data/cso/fix_coords.py

CSO GeoJSON boundaries use Irish Transverse Mercator (ITM). Centroids computed as average of boundary polygon vertices, then converted to WGS84 lat/lng using GRS80 ellipsoid parameters:

Semi-major axis: 6,378,137.0m
Origin: 53.5°N, 8°W
False easting: 600,000m, false northing: 750,000m

Sanity check: all 18,919 centroids fall within 51–56°N, 5–11°W (Ireland bounding box).

3.4 Eircode → County Mapping (Growth Data)

Script: data/map/build_growth_data.py

CSO NDQ07 reports completions by 3-character Eircode routing key (e.g. “X91”). We map these to county names using a hardcoded lookup table (139 Eircodes → 26 counties).

Mapping rate: 98,724 of 98,917 completions mapped (99.8%). 193 completions in Eircodes not in our lookup.

Known limitation: Dublin Eircodes (D01–D24, A94, A96, K78, etc.) all map to “DUBLIN”. The map’s county selector splits Dublin into Dublin City, Fingal, South Dublin, Dún Laoghaire-Rathdown. Growth data shows the combined Dublin figure for any Dublin sub-county. Same applies to Cork/Cork City, Galway/Galway City, Limerick/Limerick City, Waterford/Waterford City.

3.5 Local Authority → County Mapping (Commencements)

Script: data/map/build_growth_data.py

CSO HSM14 uses Local Authority names (e.g. “Dún Laoghaire-Rathdown”). We map these to our county names using a string-match lookup. 29 of 31 LAs mapped.

4. JSON Data Files (Map Layer Data)

Local path: data/map/ S3 path: s3://fcrwatcher/roamworkflow/map-data/ Public URL: https://fcrwatcher.s3.eu-west-1.amazonaws.com/roamworkflow/map-data/

File	Rows	Size	Description
`small_areas.json`	18,919	5.0 MB	All small areas with PP + Pobal
`cro_companies.json`	1,938	115 KB	CRO by county + NACE sector
`growth_data.json`	—	20 KB	Completions + commencements (2023–2025)
`properties_index.json`	26	2 KB	County → property file mapping
`properties_{COUNTY}.json`	varies	80 KB–3 MB	Commercial properties per county

Total: 29 files, 16 MB

4.1 small_areas.json Record Format

{
  "id": "A017010016",
  "ed": "CARLOW RURAL",
  "county": "CARLOW",
  "area": "Carlow",
  "lat": 52.844252,
  "lng": -6.918703,
  "hh": 114,
  "pop": 242,
  "houses": 105,
  "flats": 9,
  "pp": 36.87,
  "pProf": 14.91,
  "pDeg": 25.95,
  "pHome": 70.18,
  "pCar": 36.84,
  "pEmp": 47.27,
  "pi": -2.21,
  "pc": 4,
  "pl": "Marginally Below Average"
}

4.2 growth_data.json Structure

{
  "meta": { "sources": [...], "updated": "2026-03-30" },
  "completions_eircode": { "X91": { "2023": 560, "2024": 739, "2025": 742, "total": 2041 }, ... },
  "completions_county": { "WATERFORD": { "2023": 560, ... }, ... },
  "commencements_county": { "WATERFORD": { "2023": 700, ... }, ... },
  "completions_lea": { "Dungarvan, Waterford": { "2023": 48, ... }, ... }
}

5. Data Pipeline

5.1 Initial Load (one-time)

Download SAPS 2022 CSV from CSO
Download GeoJSON boundaries from OSi
Run fix_coords.py — converts ITM→WGS84, joins SAPS, outputs cso_small_areas_centroids.ndjson
Run build_purchasing_power.py — adds PP fields + Pobal join, outputs cso_small_areas_enriched.ndjson + BQ batch SQL files
Create cso_small_areas_v2 table in BQ (manual — CREATE TABLE DDL)
Load batches via n8n workflow cso-bq-insert (48 batches of 400 rows)

5.2 Map Data Generation

Run data/map/pull_all_data.py — pulls from BQ via worker API into local JSON files
Run data/map/build_growth_data.py — processes CSO NDQ07 + HSM14 CSVs into growth_data.json
Upload JSON files to S3 via n8n workflow map-data-upload (webhook: POST /webhook/map-data-upload)
Upload JSON files to server: scp to admin@52.48.145.217:/var/www/cathaldev3/data/
Run data/map/generate_national_map_s3.py — generates Ireland_Area_Intelligence.html
Deploy HTML: scp to /var/www/cathaldev3/index.html

5.3 Refresh Schedule

Data	How to refresh	Frequency
Census 2022	No refresh needed until Census 2027	Static
Pobal Index	Re-download if Pobal publishes update	~5 yearly
Valuation properties	Re-run `download_valuation.py`, rebuild batches	Annually
CRO companies	Re-download from CRO, rebuild aggregations	Annually
New dwelling completions (NDQ07)	Re-download CSV from CSO API, run `build_growth_data.py`	Quarterly
Commencement notices (HSM14)	Re-download CSV from CSO API, run `build_growth_data.py`	Monthly

5.4 n8n Workflows

Workflow	ID	Webhook	Purpose
CSO Data Loader	`ygmVyBXSoRQO5yUP`	`/webhook/cso-bq-insert`	Load SQL batches into BQ (deactivated, activate for reloads)
Map Data S3 Upload	`UVpwFXjgLS8TTYR2`	`/webhook/map-data-upload`	Upload JSON files to S3 `fcrwatcher/roamworkflow/map-data/`

6. Known Limitations & Caveats

Census 2022 is 4 years old. New housing estates built 2023–2026 have zero population in census data. Growth data (NDQ07/HSM14) partially compensates but doesn’t have demographic detail.
Purchasing power score is derived, not official. Our weighting (25/25/20/15/15) is reasonable but arbitrary. Pobal uses factor analysis with different weights. Both use the same underlying census data.
Pobal at ED level, PP at small area level. Pobal assigns one score to ~5-6 small areas. Our PP score varies within an ED. They may appear to contradict — e.g. a small area with PP 40 inside an ED labelled “Marginally Below Average”. This is expected: the PP score is more granular.
CRO data is county-level only. No coordinates for individual companies. Registered address ≠ trading address. Useful for macro density but not for “where exactly are the accountants”.
Growth data eircode→county mapping loses Dublin granularity. All Dublin Eircodes map to one “DUBLIN” entry. Can’t split completions between Dublin City, Fingal, South Dublin, DLR.
Valuation properties may be stale. Tailte Éireann updates periodically but we have a point-in-time snapshot. New builds won’t appear until rated.
Property Price Register not integrated. No public API. Form-based download only (Lotus Domino). Would provide trailing 12-month transaction volumes and median prices by Eircode — valuable for showing where money is being spent.
3 Electoral Divisions unmatched in Pobal join (66 small areas). CASTLETOWN (NORTH), DUNDALK RURAL (SOUTH), ST. MARY’S (EAST) — naming differences between CSO and Pobal datasets.

7. Verification Queries

Business Planning can verify key numbers using these BQ queries via the worker API:

API_KEY=$(grep VITE_N8N_API_KEY .env.local | cut -d= -f2)
BASE="https://fcr-dashboard-api.cathaldempsey.workers.dev/dashboard-bq-execute"

# Total small areas and population
curl -s -X POST -H "x-api-key: $API_KEY" -H "Content-Type: application/json" "$BASE" \
  -d '{"sql": "SELECT COUNT(*) as areas, SUM(total_population) as pop, SUM(total_households) as hh, ROUND(AVG(pp_score),1) as avg_pp FROM `listingmanager-1529856313699.fcr_operations.cso_small_areas_v2`"}'

# Purchasing power by county
curl -s -X POST -H "x-api-key: $API_KEY" -H "Content-Type: application/json" "$BASE" \
  -d '{"sql": "SELECT county, COUNT(*) as areas, ROUND(AVG(pp_score),1) as avg_pp, SUM(total_households) as hh FROM `listingmanager-1529856313699.fcr_operations.cso_small_areas_v2` GROUP BY county ORDER BY avg_pp DESC"}'

# Pobal match rate
curl -s -X POST -H "x-api-key: $API_KEY" -H "Content-Type: application/json" "$BASE" \
  -d '{"sql": "SELECT COUNTIF(pobal_index IS NOT NULL) as matched, COUNTIF(pobal_index IS NULL) as unmatched FROM `listingmanager-1529856313699.fcr_operations.cso_small_areas_v2`"}'

# Commercial properties by county
curl -s -X POST -H "x-api-key: $API_KEY" -H "Content-Type: application/json" "$BASE" \
  -d '{"sql": "SELECT county, COUNT(*) as properties FROM `listingmanager-1529856313699.fcr_operations.valuation_properties` WHERE centroid_lat IS NOT NULL GROUP BY county ORDER BY properties DESC", "limit": 50}'

Ask the docsRAG over this site

Ask anything about the FCR Dashboard platform — architecture, BigQuery, the worker routes, billing rules, the LRC stack, scoring… Answers are grounded in this documentation, with source links.

How does the deal-brief refresh work? Which routes are Worker vs n8n? How is account health scored?