Methodology

Every number Fealty publishes traces to a SEC filing. Here is how the instrument is measured — and, honestly, what it doesn’t yet publish.

Extraction eval gate

PASSgemma4-12b-says-v2@ab1241d7 · local · 39 hand-labeled filings

95.9%Value precisiongate ≥ 95.0% · met

91.1%Detection recallgate ≥ 85.0% · met

97.2%Basis accuracy

The extractor is a single pinned model (temperature 0, strict schema, quote-relocation check). A wrong number on a named company is worse than a missed one, so precision dominates. Changing the model or prompt bumps the version and re-runs this gate; CI blocks a regression.

Extractor changelog

gemma4-12b-says-v2@ab1241d7local · pinned 2026-07-11 · the model that ships — clears the eval gate, enforced in CI

production

gemma-4-31b@ab1241d7cerebras · distillation teacher + eval reference — NOT production (free host, unpinnable weights, daily-quota-bound; see 04b §1, §5.1)

reference

The production extractor is a local, fine-tuned open model. An earlier cloud model is retained only as a distillation teacher and eval reference — never a shipping path. Changing the production pin bumps its version and re-runs the gate.

Deterministic gates

Disposition versiongates/v1

Active gatesR1G1G4G6

Quarantine rate2.0% of model candidates held back

After the model, deterministic gates run on every candidate: a repair for ± ranges, and quarantines for a value not in its quote, a redundant growth-rate, or an out-of-scope comparable-sales KPI. A gated candidate is never published — precision is a publishing threshold, and staying silent beats a wrong number on a named company.

Versions

Extractorgemma4-12b-says-v2@ab1241d7

Resolverresolver-v3

Scorescore-v1

Resolution rules are deterministic and versioned; a threshold change regenerates outcomes under a new resolver version, old rows retained (bitemporal). Version bumps are changelog entries.

Coverage

Buyback49 graded · 476 resolved

Dividend477 graded · 12672 resolved

Guidance149 graded · 2309 resolved

A grade publishes only above a minimum resolved-promise coverage — below it we show the honest “not enough resolved promises” state, never a fabricated letter. 675 companies graded across 15457 resolved promises.

Guidance basis drop-rate

54.3% of gradeable guidance pairs were dropped for basis mismatch (3046 of 5611).

XBRL actuals are GAAP. A guidance figure is graded against a same-basis actual or DROPPED and counted here — we never grade a non-GAAP-intent number against a GAAP actual. Publishing the drop-rate is the point: a hidden drop is a hidden bias.

Not yet published (in progress)

· Universe inclusion rate, including delisted/acquired names since 2015
· Coverage histogram and extractor P/R by stratum (era, form type, size)
· Resolving percent-growth guidance against the reported prior-year base

These publish once the full backfill runs. We list what’s missing rather than imply completeness — that gap is the difference between an instrument and AI slop.