Shows how much of the model is in the code, as 3 separate numbers: fields in the types, fields in real data, and fields the screens mention. Also catches the model contradicting itself: duplicate ids, links to things that do not exist, and a journey marked built while its questions are open or with no note of who saw it work.
Add "Contract coverage" from ntent to this repo.
Fetch https://ntent.app/r/f/check-contract-coverage as plain text. Write it verbatim to scripts/check-contract-coverage.mjs. Then make exactly these edits and no others: The maps at the top of the file describe an example billing app. Replace CONFIG.entityTypeMap, CONFIG.entityUiMap and CONFIG.fieldAliases with this product’s own entities, and point typeDirs/fixtureDirs/uiDirs at its directories. It needs contract.json and TypeScript. If this repo does not have them, stop and tell me.
Then check it: It prints a table with a percentage per entity, names the gaps, and says how many of the contract’s fields the number is even about. Under --check an unmapped entity fails until you map it or excuse it with a reason. Set minCoverage just below your current number so it fails when coverage goes backwards. `--refs` is the gate half: it runs in milliseconds, exits non-zero on a duplicate id or a broken reference, and every finding names its fix. None of it says a journey works: that is the journey’s state, set by hand, and `--refs` refuses `built` unless an acceptance record sits under it: who exercised it, when, at which commit, against what evidence.Reads https://ntent.app/r/f/check-contract-coverage
Read the code
#!/usr/bin/env node
/**
* Contract coverage: of every field the contract declares, which ones can the
* build show evidence for, at three layers?
*
* This is a diagnostic, and it is named precisely because the question it gets
* asked in service of, "are we about halfway?", is one it cannot answer. A
* burndown measures tickets closed. This measures three narrower things:
*
* type DECLARED FIELD PRESENCE. The field is a property of an app type
* mapped to this entity, and its primitive does not contradict the
* contract's. Missing = not declared.
* fixture FIXTURE KEY PRESENCE. A captured payload for this entity carries
* the key with a value in it. Missing = never seen with data.
* ui UI SOURCE MENTIONS. The name appears in the files mapped to this
* entity's screens, comments stripped. Missing = not surfaced.
*
* None of these says a journey works. A field can be declared, captured and
* mentioned on a page that renders null, and this tool will credit all three.
* "Is it built" is answered by a journey's state, which a person sets, and by
* the acceptance record under it: who exercised it, when, at which commit,
* against what evidence. The reference pass refuses `built` without that
* record (see ACCEPTANCE below). It cannot tell whether the evidence is true;
* it can tell that somebody wrote it down. Report the three numbers under
* their names, next to the scope that was NOT measured, and keep readiness a
* separate sentence.
*
* A field that is in the type but in no fixture is a field nobody has ever seen
* with data in it. A field in the fixture but on no screen is data the product
* collects and never shows. Those are different problems with different fixes,
* and one combined percentage hides both.
*
* HOW STRONG EACH NUMBER IS, because a coverage tool that overstates itself is
* worse than no coverage tool. Type is read from the AST, inherited members
* included: it is a fact about the declaration. Fixture is read from PARSED
* payloads, keyed to the entity, counting only keys that hold a value: it is a
* fact about data that exists. UI is a mention count inside the files you map
* to the entity, with comments stripped, and it is a POINTER, not proof: it
* says the name appears where that entity's screens live, not that anything
* renders it.
*
* NOT EVERY FIELD BELONGS ON A SCREEN. A field can be server-only, derived,
* sensitive, or irrelevant to this consumer, and counting it against the UI
* layer pushes people to render data that should stay off the page. Mark it in
* the contract as `{ "name": "...", "ui": false, "why": "..." }` and it leaves
* the UI denominator, with the reason printed. An omission with a reason is a
* decision; an omission without one is a gap.
*
* All three used to be one grep over one concatenated string. An invoice with
* fields `id` and `total_amount`, a fixture file containing the comment
* `// id total`, and a page containing the same comment above `return null`
* scored 100% fixture and 100% UI. No payload, no screen. Names like id, total
* and items are common enough that a whole-repo text search credits an entity
* for anything.
*
* Two design decisions worth understanding before you copy this.
*
* FIELD ALIASES. The contract is domain-shaped and usually snake_case. The app
* is UI-shaped and usually camelCase, and it abbreviates: hbl_number becomes
* ref, fee_amount becomes total. You have three options. Force the app to
* rename, which is a fight you will lose. Report the mismatches as gaps, which
* makes the number a lie. Or write the mapping down. A mapping table is an
* honest artifact; a silent mismatch is a bug.
*
* BLIND SPOTS ARE REPORTED. Any contract entity with no mapping into the app is
* printed at the bottom as unmeasured, not silently skipped. A coverage tool
* that quietly ignores what it cannot see reports a number that goes up when
* you add entities it does not understand.
*
* Uses the TypeScript compiler API, which is already installed in any TS
* project. Regex over type declarations was the first attempt and it breaks on
* generics, unions and nested objects, which is most real types.
*
* TWO AXES, AND ONLY ONE OF THEM IS A GATE. Coverage is completeness: it moves
* as you build, and a low number is information rather than a defect, so it
* only fails under --check. References are structural integrity: a rule that
* binds an entity nobody declared, or a journey standing on an open question
* that does not exist, is a broken file whatever the percentage says. Those
* fail always. You can commit an incomplete contract; you cannot commit a
* contract that contradicts itself.
*
* Without the reference pass, rules, constraints and open questions are read by
* people and by nothing else, which is how they rot. Every rule has a gate or
* it is not a rule.
*
* Usage:
* node scripts/check-contract-coverage.mjs
* node scripts/check-contract-coverage.mjs --json > coverage.json
* node scripts/check-contract-coverage.mjs --check (also fails below the floor)
* node scripts/check-contract-coverage.mjs --check --min 0.6 (floor for one run)
* node scripts/check-contract-coverage.mjs --refs (references only, no TS parse)
*/
import { readFileSync, readdirSync, statSync, existsSync } from 'node:fs'
import { execFileSync } from 'node:child_process'
import { join, extname } from 'node:path'
import { createRequire } from 'node:module'
import { pathToFileURL } from 'node:url'
const require = createRequire(import.meta.url)
// Loaded on first use, not at import. --refs never reaches this, so the
// pre-commit gate runs in a repo that has not installed TypeScript yet, and a
// repo that never will still gets its references checked.
let _ts
const typescript = () => {
if (_ts) return _ts
try {
_ts = require('typescript')
} catch {
// A clear sentence, not a module resolution stack. The type layer is the
// one part of this that needs a compiler, and --refs deliberately does not,
// so "install TypeScript or run --refs" is the whole message.
console.error(`\n Type coverage needs TypeScript, and it is not installed here.`)
console.error(` fix: pnpm add -D typescript, or run --refs, which checks the`)
console.error(` contract's own references and never loads a compiler.\n`)
process.exit(2)
}
return _ts
}
// ---------------------------------------------------------------- CONFIG ---
const CONFIG = {
// The spec. See the sample at docs/playbook/examples/contract.json.
contractFile: 'contract.json',
// Where the app's types live.
typeDirs: ['src/types', 'src/lib', 'src/features'],
// Where fixtures and seeds live. fixtures/ first, because that is where the
// install plan puts captured payloads and where the message below points.
fixtureDirs: ['fixtures', 'src/data', 'prisma'],
// Where UI lives.
uiDirs: ['src/app', 'src/components'],
ignoreDirs: ['node_modules', '.next', 'dist', '.git'],
// Contract entity id -> the app type(s) that implement it. An entity absent
// from this map is reported as unmeasured rather than skipped, and under
// --check it fails unless it is excused below.
entityTypeMap: {
invoice: ['Invoice', 'InvoiceDetail'],
customer: ['Customer'],
subscription: ['Subscription'],
},
// Contract entity id -> the fixture files that carry its payloads, by stem.
// fixtures/invoice.json and fixtures/invoice.overdue.json both have the stem
// `invoice`. An entity with no fixture is reported unmeasured at that layer
// rather than scored zero: nobody has captured its data yet, which is a
// different statement from "the data is wrong".
entityFixtureMap: {
invoice: ['invoice'],
customer: ['customer'],
subscription: ['subscription'],
},
// Contract entity id -> the paths whose files are that entity's UI. Prefixes,
// matched against the repo-relative path.
//
// Explicit, because the alternative is searching the whole app for the word
// `total` and calling every hit invoice coverage. Writing the mapping down is
// two lines per entity and it is the difference between a number and a guess.
entityUiMap: {
invoice: ['src/app/invoices', 'src/components/invoice'],
customer: ['src/app/customers', 'src/components/customer'],
subscription: ['src/app/subscriptions'],
},
// Entities deliberately outside this consumer's measurement, each with a
// reason. Under --check an entity that is neither mapped nor listed here is
// a failure: the alternative is a 100% that survives every entity you add.
unmeasured: {
// payment: 'owned by the billing service; this app only reads invoice.paid_at.',
},
// Contract field name -> what the app calls it.
fieldAliases: {
invoice_number: ['ref', 'number'],
total_amount: ['total', 'amount'],
customer_id: ['customerId'],
created_at: ['createdAt'],
due_date: ['dueDate'],
line_items: ['lines', 'items'],
},
// Fail --check below this fraction of type-layer coverage. Start at your
// current number so the gate is green today, then ratchet it up. A threshold
// nobody can meet is a threshold people delete.
minCoverage: 0.0,
}
// --------------------------------------------------------------- HELPERS ---
function walk(dir, exts, acc = []) {
let entries
try {
entries = readdirSync(dir)
} catch {
return acc
}
for (const entry of entries) {
if (CONFIG.ignoreDirs.includes(entry)) continue
const full = join(dir, entry)
if (statSync(full).isDirectory()) walk(full, exts, acc)
else if (exts.includes(extname(full))) acc.push(full)
}
return acc
}
const snakeToCamel = (s) => s.replace(/_([a-z])/g, (_, c) => c.toUpperCase())
/**
* A field is a bare string, or an object once a type, a required flag or a PII
* marker starts earning its keep. Everything downstream wants the name, so the
* progressive shape costs exactly this one line.
*/
export const fieldName = (f) => (typeof f === 'string' ? f : f?.name)
/** Every name the app might use for this contract field. */
function candidateNames(field) {
return [field, snakeToCamel(field), ...(CONFIG.fieldAliases[field] || [])]
}
/**
* Every interface and type alias declared across the type files, by name, so a
* type can be followed to the ones it extends. Parsed once per run.
*/
let _declarations
function declarationsIn(files) {
if (_declarations) return _declarations
const ts = typescript()
_declarations = new Map()
for (const file of files) {
const source = ts.createSourceFile(file, readFileSync(file, 'utf8'), ts.ScriptTarget.Latest, true)
ts.forEachChild(source, (node) => {
if (ts.isInterfaceDeclaration(node) || ts.isTypeAliasDeclaration(node)) {
// First declaration wins. Two types with one name in different files
// is a problem this tool is not the place to report.
if (!_declarations.has(node.name.text)) _declarations.set(node.name.text, node)
}
})
}
return _declarations
}
/**
* The primitive a property is declared as, when it is one: `boolean`, `string`
* or `number`, with `| null` and `| undefined` ignored. Anything else, a
* reference, an object, a union of two real types, is `null`: not a primitive,
* so nothing here will argue with it.
*/
function primitiveOf(ts, typeNode) {
if (!typeNode) return null
const parts = ts.isUnionTypeNode(typeNode) ? typeNode.types : [typeNode]
const real = parts.filter(
(t) => !(t.kind === ts.SyntaxKind.UndefinedKeyword || (ts.isLiteralTypeNode(t) && t.literal.kind === ts.SyntaxKind.NullKeyword))
)
if (real.length !== 1) return null
const k = real[0].kind
if (k === ts.SyntaxKind.BooleanKeyword) return 'boolean'
if (k === ts.SyntaxKind.StringKeyword) return 'string'
if (k === ts.SyntaxKind.NumberKeyword) return 'number'
return null
}
/**
* Property name -> declared primitive (or null) for the named types, read from
* the AST rather than the text. Handles interfaces, type aliases with object
* literals, intersections, and INHERITANCE: `interface Invoice extends Base`
* carries Base's members, and `type Invoice = Base & { ... }` carries them too.
*
* The first version collected direct members only, so a field that lived on a
* base interface scored as missing from a type that plainly had it. Following
* heritage clauses through the declarations in the scanned files is not the
* full type checker, and does not need to be: generics and mapped types stay
* out of reach and score as absent, which is the honest direction to be wrong.
*/
function propertiesOfTypes(files, typeNames) {
const ts = typescript()
const declared = declarationsIn(files)
const props = new Map()
const seen = new Set()
const collectMembers = (members) => {
for (const member of members) {
if (ts.isPropertySignature(member) && member.name) {
const name = member.name.getText()
if (!props.has(name)) props.set(name, primitiveOf(ts, member.type))
}
}
}
const visit = (name) => {
if (seen.has(name)) return
seen.add(name)
const node = declared.get(name)
if (!node) return
if (ts.isInterfaceDeclaration(node)) {
collectMembers(node.members)
for (const clause of node.heritageClauses || []) {
for (const t of clause.types) visit(t.expression.getText())
}
} else {
const t = node.type
if (ts.isTypeLiteralNode(t)) collectMembers(t.members)
else if (ts.isIntersectionTypeNode(t)) {
for (const part of t.types) {
if (ts.isTypeLiteralNode(part)) collectMembers(part.members)
else if (ts.isTypeReferenceNode(part)) visit(part.typeName.getText())
}
} else if (ts.isTypeReferenceNode(t)) visit(t.typeName.getText())
}
}
for (const name of typeNames) visit(name)
return props
}
/**
* The primitive a contract type has to be declared as, when there is only one
* defensible answer. `money` is a string in some codebases and a number in
* others, so it is not listed; `boolean` is a boolean everywhere. Deliberately
* short: a mismatch this reports is a mismatch, not a style opinion.
*/
const CONTRACT_PRIMITIVES = { boolean: 'boolean', bool: 'boolean' }
/**
* A declared field whose primitive contradicts the contract's. A `money` field
* declared `boolean` is not "declared": it is a placeholder with the right
* name, and the name is the only thing a presence check reads. So a contract
* field with a type, declared as a bare primitive that no reading of that type
* allows, does not count, and is listed separately so the fix is obvious.
*/
function mismatch(field, declaredPrimitive) {
const contractType = typeof field === 'string' ? undefined : field?.type
if (!contractType || !declaredPrimitive) return null
const expected = CONTRACT_PRIMITIVES[contractType]
if (expected) return declaredPrimitive === expected ? null : { contract: contractType, declared: declaredPrimitive }
// Every non-boolean contract type: text, money, date, id, enum, url. None of
// them is a boolean.
return declaredPrimitive === 'boolean' ? { contract: contractType, declared: declaredPrimitive } : null
}
/** The fixture file's stem: invoice.overdue.json -> invoice. */
const stemOf = (file) => file.split('/').pop().replace(/\.json$/, '').split('.')[0]
/**
* Every key that holds a value anywhere in a parsed payload, at any depth.
*
* Depth matters because a real payload nests: line_items[0].unit_price is a
* field of the contract's invoice as far as anyone reading a screen is
* concerned. Arrays are walked into rather than counted, so a 400-row fixture
* costs the same as a 1-row one.
*
* A key whose value is null does not count. The column this feeds is read as
* "seen with data in it", and `"total_amount": null` is the key without the
* data: it proves the name was typed, which the type layer already knows.
* A field that is legitimately absent on one variant gets its credit from the
* variant that carries it.
*/
function keysDeep(value, into = new Set(), depth = 0) {
if (depth > 12 || value == null || typeof value !== 'object') return into
if (Array.isArray(value)) {
for (const item of value.slice(0, 50)) keysDeep(item, into, depth + 1)
return into
}
for (const [k, v] of Object.entries(value)) {
if (v != null) into.add(k)
keysDeep(v, into, depth + 1)
}
return into
}
/**
* Entity id -> the set of keys its captured payloads actually contain.
*
* Parsed, not grepped. A fixture that does not parse is reported rather than
* counted: a broken JSON file silently contributing zero keys reads as an
* entity with no data, which sends someone to write a fixture that already
* exists.
*/
function fixtureKeys() {
const files = CONFIG.fixtureDirs.filter(existsSync).flatMap((d) => walk(d, ['.json']))
const byStem = new Map()
const unparsable = []
for (const file of files) {
let parsed
try {
parsed = JSON.parse(readFileSync(file, 'utf8'))
} catch (e) {
unparsable.push({ file, why: e.message.split('\n')[0] })
continue
}
const stem = stemOf(file)
if (!byStem.has(stem)) byStem.set(stem, new Set())
keysDeep(parsed, byStem.get(stem))
}
const byEntity = new Map()
for (const entity of Object.keys(CONFIG.entityTypeMap)) {
const stems = CONFIG.entityFixtureMap[entity] ?? [entity]
const keys = new Set()
let found = false
for (const stem of stems) {
const hit = byStem.get(stem)
if (!hit) continue
found = true
for (const k of hit) keys.add(k)
}
if (found) byEntity.set(entity, keys)
}
return { byEntity, unparsable }
}
/**
* Comments out, so a TODO listing three field names stops scoring as UI.
*
* A scanner rather than a regex, because the regex only removed `//` at the
* start of a line: `return null } // total_amount` scored the field as on
* screen. Strings are skipped so `'https://...'` keeps its slashes, and a
* regex literal is left alone by the cheap rule that `/` after an identifier
* or a closing bracket is division. Good enough for a mention count.
*/
export function stripComments(text) {
let out = ''
let i = 0
const n = text.length
let last = '' // last significant character, for the regex-versus-division call
while (i < n) {
const c = text[i]
const next = text[i + 1]
if (c === '/' && next === '*') {
const end = text.indexOf('*/', i + 2)
i = end === -1 ? n : end + 2
out += ' '
continue
}
if (c === '/' && next === '/') {
const end = text.indexOf('\n', i)
i = end === -1 ? n : end
out += ' '
continue
}
if (c === '"' || c === "'" || c === '`') {
let j = i + 1
while (j < n && text[j] !== c) {
if (text[j] === '\\') j++
else if (c !== '`' && text[j] === '\n') break
j++
}
out += text.slice(i, j + 1)
i = j + 1
last = c
continue
}
if (c === '/' && !/[\w)\]]/.test(last)) {
// A regex literal. Skip to its end so a `//` inside it is not a comment.
let j = i + 1
let inClass = false
while (j < n && text[j] !== '\n') {
if (text[j] === '\\') j++
else if (text[j] === '[') inClass = true
else if (text[j] === ']') inClass = false
else if (text[j] === '/' && !inClass) break
j++
}
out += text.slice(i, j + 1)
i = j + 1
last = '/'
continue
}
out += c
if (!/\s/.test(c)) last = c
i++
}
return out
}
/**
* Entity id -> the text of the files mapped to that entity's screens.
*
* Nothing is searched that was not mapped. An entity with no mapping is
* unmeasured at the UI layer and says so, which is the only honest answer:
* without a mapping the tool does not know where that entity's screens are.
*/
function uiText() {
const byEntity = new Map()
for (const [entity, prefixes] of Object.entries(CONFIG.entityUiMap)) {
const files = CONFIG.uiDirs
.filter(existsSync)
.flatMap((d) => walk(d, ['.ts', '.tsx', '.js', '.jsx']))
.filter((f) => prefixes.some((prefix) => f.startsWith(prefix)))
if (!files.length) continue
byEntity.set(entity, stripComments(files.map((f) => readFileSync(f, 'utf8')).join('\n')))
}
return byEntity
}
const mentions = (haystack, name) =>
candidateNames(name).some((n) => new RegExp(`\\b${n}\\b`).test(haystack))
const pct = (a, b) => (b === 0 ? 0 : Math.round((a / b) * 100))
function bar(n) {
const filled = Math.round(n / 10)
return '█'.repeat(filled) + '░'.repeat(10 - filled)
}
function status(typePct) {
if (typePct >= 80) return 'green'
if (typePct >= 50) return 'amber'
return 'red'
}
// ------------------------------------------------------ REFERENCE CHECKS ---
/**
* Three checks, all of the same shape: does a name written in one pillar exist
* in the pillar it points at?
*
* The bar is deliberately low. This does not read semantics and cannot tell you
* a rule is wrong. It tells you a rule binds an entity nobody declared, which is
* the failure that actually happens, because ids are typed by hand and pillars
* are edited weeks apart.
*
* Ids carry their own namespace by prefix, so most references are bare:
* BR-x is a rule, OQ-x a question, ENT-x an entitlement gate, C-x a constraint,
* and anything else is an entity. Journeys and actors have no prefix in
* practice, so they are written journey:<id> and actor:<id> where they appear
* as a target.
*
* Every finding carries a fix. Chapter 3 asks an error message to name the
* remedy; making `fix` a field of the finding is how you guarantee one exists
* rather than hoping each check author writes a good message.
*/
const PREFIXES = { 'BR-': 'rule', 'OQ-': 'question', 'ENT-': 'entitlement', 'C-': 'constraint' }
/** "journey:pay-invoice" -> [journey, pay-invoice]. "BR-014" -> [rule, BR-014]. */
export function refTarget(ref) {
if (typeof ref !== 'string') return ['?', String(ref)]
const colon = ref.indexOf(':')
if (colon > 0) return [ref.slice(0, colon), ref.slice(colon + 1)]
const prefix = Object.keys(PREFIXES).find((pre) => ref.startsWith(pre))
return [prefix ? PREFIXES[prefix] : 'entity', ref]
}
/**
* The words a status field may hold. Anything else is a typo that every check
* downstream reads as "not built" or "not resolved", silently, in whichever
* direction happens to be lenient.
*/
export const VOCABULARY = {
'journeys[].state': ['unbuilt', 'partial', 'built'],
'open_questions[].status': ['open', 'resolved', 'deferred'],
'rules[].status': ['proposed', 'agreed', 'retired'],
'actors[].kind': ['user', 'system', 'schedule'],
}
export function checkReferences(contract) {
const findings = []
const add = (where, message, fix) => findings.push({ where, message, fix })
/**
* 0. Identity and vocabulary. Two entities called `invoice` is not a
* reference problem, it is worse: every reference to `invoice` resolves,
* to whichever one the reader picked. And a journey in state `done` is a
* journey no check will ever ask about.
*/
const pillars = {
actors: contract.actors,
entities: contract.entities,
journeys: contract.journeys,
rules: contract.rules,
constraints: contract.constraints,
'entitlements.gates': contract.entitlements?.gates,
open_questions: contract.open_questions,
}
for (const [pillar, list] of Object.entries(pillars)) {
if (!Array.isArray(list)) continue
const seen = new Set()
for (const [i, item] of list.entries()) {
if (!item || typeof item.id !== 'string' || !item.id) {
add(`${pillar}[${i}]`, 'has no id', 'Give it an id. Everything else in the file points at things by id.')
continue
}
if (seen.has(item.id)) {
add(`${pillar}.${item.id}`, `id "${item.id}" is declared twice in ${pillar}`, 'Merge the two, or rename one. A reference to a duplicated id resolves to whichever the reader picked.')
}
seen.add(item.id)
}
}
const vocab = (pillar, list, key) => {
const allowed = VOCABULARY[`${pillar}[].${key}`]
for (const item of list || []) {
if (item?.[key] === undefined || allowed.includes(item[key])) continue
add(`${pillar}.${item.id}.${key}`, `"${item[key]}" is not a ${key} this file knows`, `Use one of: ${allowed.join(', ')}.`)
}
}
vocab('journeys', contract.journeys, 'state')
vocab('open_questions', contract.open_questions, 'status')
vocab('rules', contract.rules, 'status')
vocab('actors', contract.actors, 'kind')
const ids = (list, key = 'id') => new Set((list || []).map((x) => x[key]))
const entities = ids(contract.entities)
const actors = ids(contract.actors)
const journeys = ids(contract.journeys)
const rules = ids(contract.rules)
const constraints = ids(contract.constraints)
const entitlements = ids(contract.entitlements?.gates)
const questions = new Map((contract.open_questions || []).map((q) => [q.id, q]))
const known = {
entity: entities,
actor: actors,
journey: journeys,
rule: rules,
constraint: constraints,
entitlement: entitlements,
question: new Set(questions.keys()),
}
const declared = {
entity: 'entities',
actor: 'actors',
journey: 'journeys',
rule: 'rules',
constraint: 'constraints',
entitlement: 'entitlements.gates',
question: 'open_questions',
}
/** 1. Every id one pillar points at exists in the pillar it points at. */
const resolve = (where, ref, expect) => {
const [kind, id] = expect ? [expect, ref] : refTarget(ref)
const set = known[kind]
if (!set) return add(where, `"${ref}" names no pillar this file has`, `Use an id, or namespace it as journey:<id> or actor:<id>.`)
if (!set.has(id)) {
add(
where,
`"${ref}" does not resolve to ${/^[aeiou]/.test(kind) ? 'an' : 'a'} ${kind}`,
set.size === 0
? `There is no ${declared[kind]} array in the contract. Add one, or drop the reference.`
: `Add "${id}" to ${declared[kind]}, or correct the spelling. Known: ${[...set].join(', ')}.`
)
}
}
for (const e of contract.entities || []) {
for (const [i, f] of (e.fields || []).entries()) {
if (!fieldName(f)) {
add(`entities.${e.id}.fields[${i}]`, 'a field object with no name', 'Give it a "name", or write it as a bare string.')
}
}
}
for (const r of contract.rules || []) {
for (const ref of r.applies_to || []) resolve(`rules.${r.id}.applies_to`, ref)
}
for (const j of contract.journeys || []) {
if (j.actor) resolve(`journeys.${j.id}.actor`, j.actor, 'actor')
for (const ref of j.entities || []) resolve(`journeys.${j.id}.entities`, ref, 'entity')
for (const ref of j.depends_on || []) resolve(`journeys.${j.id}.depends_on`, ref)
}
for (const [i, row] of (contract.access?.matrix || []).entries()) {
resolve(`access.matrix[${i}].actor`, row.actor, 'actor')
resolve(`access.matrix[${i}].entity`, row.entity, 'entity')
}
if (contract.access?.tenancy?.scope_entity) {
resolve('access.tenancy.scope_entity', contract.access.tenancy.scope_entity, 'entity')
}
for (const g of contract.entitlements?.gates || []) {
for (const ref of g.gates || []) resolve(`entitlements.${g.id}.gates`, ref)
}
for (const c of contract.constraints || []) {
for (const ref of c.affects || []) resolve(`constraints.${c.id}.affects`, ref)
}
for (const q of questions.values()) {
for (const ref of q.blocks || []) resolve(`open_questions.${q.id}.blocks`, ref)
}
/**
* 2 and 3. Ids cited in prose. A rule id inside a journey's gap text, or an
* open question named in a note, is the most common way a contract cites
* something: as a sentence, not as a field. Those citations are the ones that
* silently stop resolving when an id is renamed, so scan every string.
*/
const prose = JSON.stringify(contract)
// A retired id still resolves, to the rule that absorbed it. This is the
// machine-readable half of "never let an id die silently": the sentence in
// `source` tells a person, `supersedes` tells the check, and a ticket citing
// BR-007 two years from now still lands somewhere.
const retired = new Set((contract.rules || []).flatMap((r) => r.supersedes || []))
for (const id of new Set(prose.match(/\bBR-\d+\b/g) || [])) {
if (!rules.has(id) && !retired.has(id)) {
add('cited in prose', `${id} is referred to but is not in rules`, `Add ${id} to rules, or name it in the supersedes list of the rule that absorbed it. Never let an id die silently: tickets and code comments still resolve against it.`)
}
}
for (const id of new Set(prose.match(/\bOQ-\d+\b/g) || [])) {
if (!questions.has(id)) {
add('cited in prose', `${id} is referred to but is not in open_questions`, `Add ${id} to open_questions with a status, or stop citing it. A question nothing can find is not a question anyone will answer.`)
}
}
/**
* The one check that is about honesty rather than spelling. source_policy says
* an unknown blocks the affected implementation. This is that sentence with a
* gate under it: a journey cannot be finished while it stands on a question
* nobody has answered.
*
* WHAT A JOURNEY STANDS ON. The question can name the journey, or the journey
* can cite the question. It can also block something the journey depends on,
* and that dependency has to be written down somewhere in the file:
*
* - an entitlement gate whose `gates` names the journey
* - a constraint whose `affects` names the journey
* - an entity the journey lists in `entities`
* - anything the journey lists in `depends_on`, which is where a rule goes
*
* A question on BR-014, which applies to invoices, does NOT block every
* journey that touches an invoice. That would be inference, and inference
* blocks unrelated work until somebody deletes the check. If a journey
* stands on a rule, say so in depends_on and the check will hold it to it.
*/
const dependenciesOf = (j) => {
const deps = new Set()
for (const e of j.entities || []) deps.add(`entity:${e}`)
for (const ref of j.depends_on || []) deps.add(refTarget(ref).join(':'))
for (const g of contract.entitlements?.gates || []) {
if ((g.gates || []).includes(`journey:${j.id}`)) deps.add(`entitlement:${g.id}`)
}
for (const c of contract.constraints || []) {
if ((c.affects || []).includes(`journey:${j.id}`)) deps.add(`constraint:${c.id}`)
}
return deps
}
for (const j of contract.journeys || []) {
if (j.state !== 'built') continue
const deps = dependenciesOf(j)
const standsOn = new Map() // question id -> what it blocks that this journey needs
for (const id of JSON.stringify(j).match(/\bOQ-\d+\b/g) || []) standsOn.set(id, 'cited by the journey')
for (const q of questions.values()) {
for (const ref of q.blocks || []) {
const [kind, id] = refTarget(ref)
if (kind === 'journey' && id === j.id) standsOn.set(q.id, 'names the journey')
else if (deps.has(`${kind}:${id}`)) standsOn.set(q.id, `blocks ${ref}, which the journey stands on`)
}
}
for (const [id, via] of standsOn) {
const q = questions.get(id)
if (q && q.blocking && q.status !== 'resolved') {
add(
`journeys.${j.id}`,
`marked built while ${id} is open and blocking (${via}): "${q.question}"`,
`Answer ${id} and set its status to resolved, drop its blocking flag with a reason, or move the journey back to partial and name the gap.`
)
}
}
}
/**
* ACCEPTANCE. `built` used to be a word anyone could type. The coverage
* number, the progress task and "how far along are we" all stood on it, and
* nothing stood under it. A built journey now carries the record of how it
* was found to work:
*
* "accepted": {
* "on": "2026-09-04",
* "by": "R. Nair, finance lead",
* "commit": "8c1f2a9",
* "contract": "0.4.0",
* "evidence": ["pnpm test -- pay-invoice",
* "manual: paid a real-shaped invoice on a phone, card-declined path included"]
* }
*
* Four fields, written once per journey, not per commit. A test command or a
* test file is evidence; so is a line starting "manual:" with a reviewer's
* name behind it, because some journeys are checked by a person and saying
* so beats pretending a script did it. This check reads that the record is
* there and complete. Whether the evidence is true is what the name on the
* line is for. `contract` is optional and is how the report notices that the
* model moved on after acceptance: stale is printed, never failed.
*/
const ACCEPTANCE_FIELDS = ['on', 'by', 'commit', 'evidence']
for (const j of contract.journeys || []) {
if (j.state !== 'built') continue
const a = j.accepted
if (!a || typeof a !== 'object' || Array.isArray(a)) {
add(
`journeys.${j.id}`,
'marked built with no acceptance record',
'Add "accepted": { "on", "by", "commit", "evidence": [...] } saying who exercised it, when, at which commit and against what. Or move it back to partial and name the gap.'
)
continue
}
const missing = ACCEPTANCE_FIELDS.filter((k) => {
const v = a[k]
if (k === 'evidence') return !Array.isArray(v) || v.length === 0 || v.some((x) => typeof x !== 'string' || !x.trim())
return typeof v !== 'string' || !v.trim()
})
if (missing.length) {
add(
`journeys.${j.id}.accepted`,
`acceptance record is missing ${missing.join(', ')}`,
'on is the date, by is the person, commit is the implementation revision, evidence is a non-empty list of test commands, test files, or "manual: what was seen".'
)
} else if (!/^\d{4}-\d{2}-\d{2}$/.test(a.on)) {
add(`journeys.${j.id}.accepted.on`, `"${a.on}" is not a date`, 'Write it as YYYY-MM-DD, so the record can be read against the commit log.')
}
}
return findings
}
/**
* Whether a commit exists in the repository this runs in. null when there is no
* git to ask, which is different from "no": a contract read from a tarball
* still has its records.
*/
function commitKnown(sha) {
const git = (...args) => {
try {
execFileSync('git', args, { stdio: 'ignore' })
return true
} catch {
return false
}
}
if (!git('rev-parse', '--git-dir')) return null
return git('cat-file', '-e', `${sha}^{commit}`)
}
function reportReferences(findings) {
if (!findings.length) return
console.log(` BROKEN REFERENCES. ${findings.length} in the contract itself:\n`)
for (const f of findings) {
console.log(` x ${f.where}`)
console.log(` ${f.message}`)
console.log(` fix: ${f.fix}`)
}
console.log(`\n These fail whatever the coverage number is. A contract that`)
console.log(` contradicts itself scores nothing worth reading.\n`)
}
// ------------------------------------------------------------------ MAIN ---
function main() {
const asJson = process.argv.includes('--json')
const strict = process.argv.includes('--check')
const refsOnly = process.argv.includes('--refs')
// A floor for this run only, as a fraction. CI ratchets with a flag; the
// committed CONFIG.minCoverage stays the number the repo agreed on.
const minIndex = process.argv.indexOf('--min')
const minCoverage = minIndex === -1 ? CONFIG.minCoverage : Number(process.argv[minIndex + 1])
if (Number.isNaN(minCoverage)) {
console.error(`--min needs a fraction, such as --min 0.6`)
process.exit(2)
}
if (!existsSync(CONFIG.contractFile)) {
console.error(`No contract at ${CONFIG.contractFile}.`)
process.exit(1)
}
const contract = JSON.parse(readFileSync(CONFIG.contractFile, 'utf8'))
const broken = checkReferences(contract)
// --refs is what the pre-commit hook calls. No TypeScript parse, no walking
// the app, so it runs in milliseconds on a commit that only edits the
// contract. The full run is a nudge; this one is a gate.
if (refsOnly) {
if (asJson) console.log(JSON.stringify({ broken }, null, 2))
else if (broken.length) {
console.log()
reportReferences(broken)
} else console.log(` Contract references resolve.`)
process.exit(broken.length ? 1 : 0)
}
const typeFiles = CONFIG.typeDirs.filter(existsSync).flatMap((d) => walk(d, ['.ts', '.tsx']))
const { byEntity: payloadKeys, unparsable } = fixtureKeys()
const uiByEntity = uiText()
const rows = []
const unmeasured = []
const noFixture = []
const noUiMap = []
for (const entity of contract.entities) {
const typeNames = CONFIG.entityTypeMap[entity.id]
if (!typeNames) {
unmeasured.push(entity.id)
continue
}
const declared = propertiesOfTypes(typeFiles, typeNames)
const fieldObjects = (entity.fields || []).filter(fieldName)
const fields = fieldObjects.map(fieldName)
// Declared, and not declared as something the contract type cannot be.
const mismatched = []
const inType = fields.filter((f, i) => {
const hit = candidateNames(f).find((n) => declared.has(n))
if (hit === undefined) return false
const bad = mismatch(fieldObjects[i], declared.get(hit))
if (bad) mismatched.push({ field: f, ...bad })
return !bad
})
// Fields the contract says are not for this consumer's screens, with the
// reason. They leave the UI denominator and are printed, so a decision
// reads as a decision rather than as a gap.
const offScreen = fieldObjects.filter((f) => typeof f === 'object' && f.ui === false)
const uiFields = fields.filter((f) => !offScreen.some((o) => o.name === f))
// null, not 0. "No payload has ever been captured for this entity" and
// "the payload is missing every field" are different findings with
// different fixes, and a zero in the table cannot tell you which you have.
const keys = payloadKeys.get(entity.id)
if (!keys) noFixture.push(entity.id)
const inFix = keys ? fields.filter((f) => candidateNames(f).some((n) => keys.has(n))) : null
const ui = uiByEntity.get(entity.id)
if (!ui) noUiMap.push(entity.id)
const inUiL = ui ? uiFields.filter((f) => mentions(ui, f)) : null
rows.push({
id: entity.id,
total: fields.length,
type: pct(inType.length, fields.length),
fixture: inFix ? pct(inFix.length, fields.length) : null,
ui: inUiL ? pct(inUiL.length, uiFields.length) : null,
missingFromType: fields.filter((f) => !inType.includes(f)),
mismatched,
missingFromFixture: inFix ? fields.filter((f) => !inFix.includes(f)) : [],
missingFromUi: inUiL ? uiFields.filter((f) => !inUiL.includes(f)) : [],
offScreen: offScreen.map((f) => ({ field: f.name, why: f.why || '' })),
})
}
// Journeys, if the contract declares them, are scored by hand and just
// reported. Nothing can infer "is this flow built" from source. What CAN be
// read is whether a built journey's record still describes this repository:
// a commit nobody here can find, or a contract that has moved on since the
// acceptance, is printed beside the journey. Stale is a note, not a failure.
const journeys = contract.journeys || []
const acceptance = journeys
.filter((j) => j.state === 'built' && j.accepted && typeof j.accepted === 'object')
.map((j) => ({
id: j.id,
...j.accepted,
commitKnown: commitKnown(String(j.accepted.commit)),
contractMoved: j.accepted.contract != null && j.accepted.contract !== contract.version,
}))
const totalFields = rows.reduce((n, r) => n + r.total, 0)
const weightedType = rows.reduce((n, r) => n + (r.type / 100) * r.total, 0)
const overall = pct(weightedType, totalFields)
/*
* THE DENOMINATOR, SAID OUT LOUD. `overall` is over the entities this run
* could measure. An entity with no mapping is outside it, and a percentage
* that goes up when you add an entity the tool cannot see is the exact
* number this file's header promises not to print. So: how much of the
* contract's fields the number is even about, and, under --check, an
* unmapped entity that nobody has excused is a failure rather than a
* footnote.
*/
const unmeasuredFields = contract.entities
.filter((e) => unmeasured.includes(e.id))
.reduce((n, e) => n + (e.fields || []).filter(fieldName).length, 0)
const measured = { fields: totalFields, of: totalFields + unmeasuredFields }
const unexcused = unmeasured.filter((id) => !CONFIG.unmeasured[id])
const excused = unmeasured.filter((id) => CONFIG.unmeasured[id]).map((id) => ({ id, why: CONFIG.unmeasured[id] }))
/*
* ONE VERDICT, USED BY BOTH OUTPUTS.
*
* The JSON branch used to return before the threshold comparison, so a CI job
* asking for a machine-readable report AND enforcement got a green exit below
* its own floor. Asking for a different output format is not a request for a
* different answer.
*/
const belowFloor = strict && overall < minCoverage * 100
const unmappedScope = strict && unexcused.length > 0
const failed = broken.length > 0 || belowFloor || unmappedScope
if (asJson) {
console.log(
JSON.stringify(
{
overall,
measured,
minCoverage,
belowFloor,
unmappedScope,
ok: !failed,
rows,
journeys,
acceptance,
unmeasured,
excused,
unmeasuredFixtures: noFixture,
unmeasuredUi: noUiMap,
unparsableFixtures: unparsable,
broken,
},
null,
2
)
)
process.exit(failed ? 1 : 0)
}
// -------------------------------------------------------------- REPORT ---
const asPct = (n) => (n == null ? '-' : `${n}%`)
console.log(`\nContract coverage (contract v${contract.version || '?'})\n`)
console.log(` Entity Type Fixture UI* Status`)
console.log(` ${'-'.repeat(52)}`)
for (const r of rows.sort((a, b) => a.type - b.type)) {
console.log(
` ${r.id.padEnd(16)} ${String(r.type + '%').padStart(4)} ` +
`${asPct(r.fixture).padStart(5)} ${asPct(r.ui).padStart(4)} ${status(r.type)}`
)
}
console.log(`\n Declared fields: ${overall}% of the ${measured.fields} measured ${bar(overall)}`)
if (measured.of > measured.fields) {
console.log(` That is over ${measured.fields} of the contract's ${measured.of} fields. See UNMEASURED below.`)
}
console.log()
console.log(` Type is read from the AST and Fixture from parsed payloads: both are facts`)
console.log(` about declarations and data, not about a journey working.`)
console.log(` * UI counts field names mentioned in the files mapped to that entity, with`)
console.log(` comments stripped. It says the name appears where those screens live, not`)
console.log(` that anything renders it. Treat it as a pointer and open the screen.\n`)
const gaps = rows.filter((r) => r.missingFromType.length)
if (gaps.length) {
console.log(` Not declared:\n`)
for (const r of gaps) {
console.log(` ${r.id}: ${r.missingFromType.join(', ')}`)
}
console.log()
}
const wrong = rows.filter((r) => r.mismatched.length)
if (wrong.length) {
console.log(` Declared as something the contract says it is not:\n`)
for (const r of wrong) {
for (const m of r.mismatched) console.log(` ${r.id}.${m.field}: contract says ${m.contract}, the type says ${m.declared}`)
}
console.log(` A placeholder with the right name is not a declaration. These are not counted.\n`)
}
const offScreen = rows.filter((r) => r.offScreen.length)
if (offScreen.length) {
console.log(` Off screen by decision, outside the UI column:\n`)
for (const r of offScreen) {
for (const o of r.offScreen) console.log(` ${r.id}.${o.field}: ${o.why || '(no reason given: add a "why")'}`)
}
console.log()
}
const dataGaps = rows.filter((r) => r.missingFromFixture.length)
if (dataGaps.length) {
console.log(` In the contract, in no captured payload:\n`)
for (const r of dataGaps) {
console.log(` ${r.id}: ${r.missingFromFixture.join(', ')}`)
}
console.log(` Nobody has seen these with data in them.\n`)
}
if (journeys.length) {
const built = journeys.filter((j) => j.state === 'built').length
const partial = journeys.filter((j) => j.state === 'partial').length
console.log(` Journeys: ${built} built, ${partial} partial, ${journeys.length - built - partial} unbuilt\n`)
for (const a of acceptance) {
const notes = []
if (a.commitKnown === false) notes.push('commit not in this repository')
if (a.contractMoved) notes.push(`accepted against contract v${a.contract}, now v${contract.version}`)
const count = Array.isArray(a.evidence) ? a.evidence.length : 0
console.log(
` + ${a.id.padEnd(28)} accepted ${a.on} by ${a.by} at ${a.commit}, ${count} piece(s) of evidence` +
(notes.length ? ` (${notes.join('; ')})` : '')
)
}
for (const j of journeys.filter((j) => j.state !== 'built')) {
const mark = j.state === 'partial' ? '~' : 'x'
console.log(` ${mark} ${j.id.padEnd(28)} ${j.gap || ''}`)
}
console.log()
}
if (unexcused.length) {
console.log(` UNMEASURED. ${unexcused.length} contract entit(ies) have no mapping in`)
console.log(` CONFIG.entityTypeMap, so they are not in the number above:`)
console.log(` ${unexcused.join(', ')}`)
console.log(` Map them, or excuse each in CONFIG.unmeasured with a reason. Under`)
console.log(` --check this is a failure: a percentage that ignores what it cannot`)
console.log(` see rises whenever the contract grows.\n`)
}
if (excused.length) {
console.log(` Outside this consumer's measurement, by decision:\n`)
for (const e of excused) console.log(` ${e.id}: ${e.why}`)
console.log()
}
if (noFixture.length) {
console.log(` NO CAPTURED DATA. ${noFixture.length} entit(ies) have no payload to read, so`)
console.log(` their Fixture column is '-' rather than 0%:`)
console.log(` ${noFixture.join(', ')}`)
console.log(` Capture one into ${CONFIG.fixtureDirs[0]}/<entity>.json, or map its stem in`)
console.log(` CONFIG.entityFixtureMap.\n`)
}
if (noUiMap.length) {
console.log(` NO SCREENS MAPPED. ${noUiMap.length} entit(ies) have no paths in`)
console.log(` CONFIG.entityUiMap, so their UI column is '-':`)
console.log(` ${noUiMap.join(', ')}`)
console.log(` Name the directories their screens live in.\n`)
}
if (unparsable.length) {
console.log(` UNREADABLE FIXTURES. ${unparsable.length} file(s) did not parse, so they`)
console.log(` contributed nothing and would otherwise look like missing data:`)
for (const u of unparsable) console.log(` ${u.file}: ${u.why}`)
console.log()
}
reportReferences(broken)
if (belowFloor) {
console.log(` FAIL: ${overall}% is below the ${minCoverage * 100}% floor.\n`)
}
if (unmappedScope) {
console.log(` FAIL: ${unexcused.length} entit(ies) unmapped and unexcused, so the number above is not about the whole contract.\n`)
}
process.exit(failed ? 1 : 0)
}
// Run when invoked, importable when tested. The reference checks are pure
// functions over a parsed contract, so they are worth testing directly rather
// than by reading the output of a subprocess.
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) main()
Success check: It prints a table with a percentage per entity, names the gaps, and says how many of the contract’s fields the number is even about. Under --check an unmapped entity fails until you map it or excuse it with a reason. Set minCoverage just below your current number so it fails when coverage goes backwards. `--refs` is the gate half: it runs in milliseconds, exits non-zero on a duplicate id or a broken reference, and every finding names its fix. None of it says a journey works: that is the journey’s state, set by hand, and `--refs` refuses `built` unless an acceptance record sits under it: who exercised it, when, at which commit, against what evidence.