Data Feeds & Knowledge Sources

The “where does X come from and how do I change it” reference: the customer site index, the company knowledge base, the deal-history index, the product portfolio, and the outreach email templates. Verified 2026-05-20 (deal-history index + its refresh cron added 2026-05-23).

1. The customer site index (`fcr-site-portfolio`)

This is the Vectorize index behind “similar FCR-built sites / similar clients” in prospect intel and the deal brief.

Feed chain:

MegaDoc (Google Sheet, maintained by the build team)
   │  weekly Tue 04:00 — n8n "Dashboard - SitePro MegaDoc" (CkAXUJQhwdT07aGr)
   ▼
fcr_operations.sitepro_megadoc_cache   (one row per built site; is_live flag)
   │  manual — handlers/vector-reindex.js (route dashboard-vector-reindex),
   │  driven by n8n "Dashboard - Vector Reindex (Manual)" (gwg0ZtoHjlRlQNf1)
   │  JOINs active_clients (latest snapshot) to attach product/MRR metadata
   ▼
Vectorize  fcr-site-portfolio

Does active_clients touch it? Yes — vector-reindex.js pulls Live sites from sitepro_megadoc_cache (WHERE is_live = "Live") and joins active_clients (latest snapshot_date) to embed each site’s active product lines + MRR into the vector metadata. So the index carries billing/product flags — but as a frozen snapshot from reindex time. Reindex is manual (no cron), so:

those metadata flags drift as clients churn/upsell, and
consumers must re-hydrate money/product state from active_clients live (prospect-intel does this; never trust the vector’s total_mrr/product flags directly — see hubspot-data-architecture.md).
The index is GP-listing-wide, so it also contains free listings / churned — consumers must split paying vs not, not just hydrate.

To refresh after MegaDoc/site changes: run the n8n “Vector Reindex (Manual)” workflow (loops POST /dashboard-vector-reindex in batches of 50 until remaining=0). It’s inactive by design — run on demand.

2. The company knowledge base (`fcr-company-knowledge`)

The Vectorize index behind the advisor’s search_company_knowledge (“what is SayMore”, “FCR’s heritage”, product descriptions, case studies).

Four sources now feed this one index (distinguished by content_type):

Source	`content_type`	Where it’s curated	Route
Crawled public sites	(page)	`worker/src/data/company-sites.js` → `COMPANY_SITES` (`fcrmedia.ie`, `saymore.ie`)	`dashboard-company-knowledge-reindex` (crawls sitemaps → chunks ~600 tokens → embeds)
Case studies (CCO-authored, verbatim)	`case_study`	`worker/src/data/case-studies.js`	`dashboard-case-studies-reindex`
Marketing knowledge docs (positioning, FAQs, proof points)	`marketing_doc`	`fcr_operations.marketing_knowledge_docs` — marketing self-service	`dashboard-marketing-knowledge` (`/marketing-knowledge` skill)
Product descriptions (from the catalogue)	`product`	`fcr_operations.marketing_products` — marketing self-service	`dashboard-marketing-products` (`/marketing-products` skill)

To add to the knowledge base:

A marketing doc (no deploy): /marketing-knowledge --add — chunks, embeds, and indexes a doc (content_type=marketing_doc, site_id=marketing) live; --remove de-indexes it (soft-delete + deleteByIds). This is the “marketing canonical source seeds the company-knowledge index” path, now built.
A new site/section: add an entry to COMPANY_SITES in company-sites.js (and tune EXCLUDE_PATH_PATTERNS), commit + deploy, then call dashboard-company-knowledge-reindex repeatedly (paginated).
A new case study: add it to case-studies.js (preserve the Commercial Director’s wording verbatim — never copy-edit), deploy, run dashboard-case-studies-reindex.

Keep COMPANY_SITES short and curated. Marketing-authored reference content now goes through /marketing-knowledge (no code edit) rather than the crawl.

3. The deal-history index (`fcr-deal-history`)

The Vectorize index behind the advisor’s “deals like this one / what happened” retrieval (deals-vector-search / find_similar_deals) and the deal brief’s similar-deals block. Separate from fcr-site-portfolio — that index is sites, this one is deals (open + 18 months of closed won/lost).

Feed chain:

hubspot_deals (open)  +  hubspot_deals_backcatalogue_18m (closed won/lost)
   │  + hubspot_engagements (last ~5 NOTE/EMAIL/CALL bodies per deal)
   │  handlers/deals-vector-reindex.js  (route dashboard-deals-vector-reindex)
   │  embeds: business · category · county · channel · source · outcome ·
   │  won/lost reason · products pitched · cycle metrics · owner · engagement
   │  content · (2026-05-23) buddy split · attribution · phase forecast
   ▼
Vectorize  fcr-deal-history   (1536-d cosine, text-embedding-3-small)

Refresh — daily incremental cron (added 2026-05-23; was manual-only):

deals-vector-reindex-cron.js fires daily 06:30 UTC, FCR-only, finds deals modified in the last 2 days, and enqueues ≤100-deal batches (capped 80/run) to fcr-deal-vector-queue; the consumer re-embeds each via an explicit deal_ids[] list (a literal IN on the engagements cluster key, so each batch prunes the scan instead of full-scanning). Keeps the index fresh with no manual pass. DMs Ro.am on read/enqueue failure, cap overflow, or a dead-lettered batch.
Full rebuild (only needed after changing the embed shape) stays manual: drive POST /dashboard-deals-vector-reindex with cohort=all in batches until remaining=0 (~12.8k deals). Manual trigger for the incremental path: POST /dashboard-deals-vector-reindex-cron?lookback_days=N.

Metadata filters: outcome, close_year, county, service_trade, owner_name are indexed (filterable). products_pitched + amount_band are embedded and slated for indexing; is_buddy / utm_source / pipeline_status are stored & returned but not yet filterable (a metadata index must be created per account + a full reindex run — 5-10 min async propagation). Like §1, the metadata is a frozen snapshot from embed time — hydrate volatile money/live-state from active_clients live; don’t trust the vector copy.

4. The FCR product portfolio

The canonical price list / product catalogue. Marketing owns it (as of 2026-05-24) — the hard-coded file is now only a fallback.

Source of truth: fcr_operations.marketing_products (+ pricing guardrails in marketing_pricing_rules), curated via /marketing-products (dashboard-marketing-products). Each product: id, name, setup_eur, monthly_eur, term_months, description, includes[], sort_order.
How the advisor reads it: on every edit the endpoint re-renders the exact PRODUCT CATALOGUE block to KV (marketing:products-prompt-block); ai-advisor.js reads that block per request, falling back to worker/src/data/fcr-products.js (FCR_PRODUCTS_PROMPT_BLOCK) if KV is empty or unreachable — so the advisor never loses its price list. The renderer (lib/render-product-catalogue.js) is byte-for-byte compatible with the legacy block, so output is unchanged except for marketing’s edits.
Each product is also embedded into fcr-company-knowledge (content_type=product) so descriptive product questions retrieve it.

To update a price: /marketing-products --update <id> (confirm-first; preview the rendered line with --dry-run). No deploy, no code edit — the advisor quotes the new price on its next turn. Pricing is client-facing, so the confirm-first + marketing_kb_audit trail replaces the old “sales-ops reviews the PR” gate.

One-time bootstrap: /marketing-products seed action imports today’s fcr-products.js into the tables + renders the block. After that, fcr-products.js survives only as the KV-unreachable fallback.

5. Outreach email templates

Where the AM outreach templates the advisor drafts from are stored.

Tier	Storage (BQ, `fcr_operations`)	Managed via
Personal (per AM)	`am_email_templates` (key: `roam_user_id` + `label`)	`dashboard-am-email-templates` (GET / POST upsert / DELETE soft-delete)
Team / peer	`am_team_email_templates`	`dashboard-am-team-email-templates`
CCO house (universal fallback)	same tables, the CCO’s templates	the firm-wide standing wording every rep falls back to
Marketing-approved (house outreach copy)	`marketing_outreach_templates` + `fcr-outreach-templates` Vectorize index	`dashboard-marketing-outreach` (`/marketing-outreach` skill); advisor retrieves by intent via `find_outreach_template`

The first three tiers are per-AM voice (the template-first picker resolves personal → team → CCO). The marketing-approved tier is a standalone, semantically-indexed store of house copy the advisor can pull by intent — added 2026-05-24, distinct from the AM voice tiers above.

Each row carries template_body (verbatim), label (slug like audit_followup_website), and when_to_use (the context hint the advisor’s template picker matches against — semantically, not by label string).

To add/edit a template:

Easiest: ask the advisor to “save this as my template” — it calls save_my_email_template / save_team_email_template under the hood.
Or POST /dashboard-am-email-templates with roam_user_id, label, template_body, when_to_use.

The template-first policy (why this matters): when asked for outreach, the advisor MUST match an existing template by when_to_use and use its body verbatim (only substituting {{tokens}}) before ever drafting fresh — personal → own-team → peer-team → CCO house → only then Claude’s own voice. Templates carry the brand voice the CCO approved.

Summary: how to change each thing

Want to change…	Do this
A built site shown as a “similar client”	update the MegaDoc → it flows to `sitepro_megadoc_cache` weekly → run the manual Vector Reindex
FCR’s about-us / product knowledge	edit `company-sites.js` (or `case-studies.js`) → deploy → run the matching reindex route
Add/remove a marketing knowledge doc	`/marketing-knowledge --add` / `--remove` (no deploy)
“Similar deals” the advisor surfaces	self-refreshes daily (deals-vector-reindex cron); after an embed-shape change, run a full `cohort=all` reindex once
A product price	`/marketing-products --update <id>` (no deploy; `fcr-products.js` is now only the fallback)
An AM outreach template	save via the advisor, or `POST /dashboard-am-email-templates`
A marketing-approved outreach template	`/marketing-outreach --add` / `--remove`

Ask the docsRAG over this site

Ask anything about the FCR Dashboard platform — architecture, BigQuery, the worker routes, billing rules, the LRC stack, scoring… Answers are grounded in this documentation, with source links.

How does the deal-brief refresh work? Which routes are Worker vs n8n? How is account health scored?