A table any signed-in user can read in full, a policy that lets one customer read another customer’s rows, and tables that contradict the model: a required field in a column that allows empty, or a model entity with no table. It builds the database from the migrations in-process and tries the reads the model says must fail. The table checks run on Postgres, Supabase and SQLite; the read tests run on Postgres and Supabase only, because SQLite has no row security.
Add "Database access checks" from ntent to this repo.
Fetch https://ntent.app/r/f/check-backend as plain text. Write it verbatim to scripts/check-backend.mjs. Then make exactly these edits and no others: First run node scripts/check-backend.mjs --print-config. It searches the repo at any depth and prints the detected platform (and why), clientAccess, enforcement and, when it finds no inputs, ranked candidate lines. Accept the detected platform, or paste the top candidate as CONFIG.sources (a candidate line is a ready CONFIG sources line; where migrations apply in file order, use order: 'listed'). CONFIG.platform and CONFIG.enforcement default to auto, so accept what it detects: it ignores templates and examples, and sets postgres or supabase for you. On plain Postgres with no RLS anywhere in the schema, auto treats access as enforced in app code: exit 0, a note, and 'access NOT checked' in the summary line; set enforcement: 'app' to confirm. It never does this on Supabase. If the schema does have RLS on plain Postgres, set CONFIG.actAs to the role your app connects as, and the migrations must create that role and grant it the tables. A candidate for a SQLite or D1 schema carries dialect: 'sqlite'. If no client reaches the database directly (a server holds the only connection), set CONFIG.clientAccess to server-only. Access enforced in app code (a missing owner filter, an IDOR) is not checked by this script. In contract.json, give each entity you want checked a table, and set access.tenancy.column (and membership, if users join a tenant through a table). Before you set an entity's read scope, read the table's SELECT policies: a policy that is using (true), is named for the public or everyone, or filters only on a published-style column (is_published, active, status = 'published') means the table is probably public on purpose, so ask, or decide from the product, whether readers get scope all, and add a visitor actor (id visitor) with a read row at scope all when anon can read it. Take owner from the policy or foreign key that enforces ownership, not from the column name, and mark a field required only when the schema says NOT NULL. For Prisma, set CONFIG.sources to [{ kind: 'command', preset: 'prisma' }] (Prisma 7 also needs datasource.url set to process.env.DATABASE_URL in prisma.config.ts, even to print SQL), then let install fetch its schema engine: with pnpm 10, add "@prisma/engines" to pnpm.onlyBuiltDependencies in package.json; with pnpm 11, which ignores that key, add @prisma/engines: true under allowBuilds in pnpm-workspace.yaml; then run the install once online. For Drizzle set [{ kind: 'command', preset: 'drizzle' }], for Alembic [{ kind: 'command', preset: 'alembic', dialect: 'postgres' }] (alembic must be in .venv or on PATH, and the migrations must run offline), and for SQLite migration files [{ kind: 'sql', paths: ['db/migrations/*.sql'], dialect: 'sqlite' }]. SQLite and Turso have no row security, so the access gate says it did not check and the schema gate runs as normal. Add a package.json script "backend:check": "node scripts/check-backend.mjs --check", a row for it in the AGENTS.md enforcement table, and a CI step that runs node scripts/check-backend.mjs --check --allow-empty, so CI passes before the first migration exists. Then run node scripts/check-backend.mjs --check once. If the migrations do not load (a table made in the dashboard, a dump that does not run, a real SQL error), do not leave the commit blocked and do not drop the check: first try the escapes its LOAD-FAILED lines name (a base schema file first in CONFIG.sources, CONFIG.preamble, late: true, CONFIG.skipStatements). If it still does not load, set loadGate: 'warn' in CONFIG and tell the person plainly that the database is not being checked yet. Warn mode prints every load failure on every run and does not block. The first clean load exits 1 with LOAD-GATE-READY; set loadGate back to 'gate' and commit that, and from then on a migration set that does not load blocks the commit. Never set 'warn' on a repo that loads. It needs a Postgres, SQLite or Supabase database, now or planned. If this repo does not have it, stop and tell me.
Then check it: Write a migration that creates a table in public without enabling row level security, and the commit is refused naming RLS-OFF and the table. Enable RLS with a policy on auth.uid() and it passes. Map an entity to that table with scope own, change the policy to using (true), and the refusal says LEAK-READ: user A read a row that belongs to user B. A table the check cannot fill with test rows refuses the commit with the column it could not fill, never a pass.Reads https://ntent.app/r/f/check-backend
Read the code
#!/usr/bin/env node
/**
* Backend gate: the database keeps the model's promises.
*
* The contract makes promises about the server. `access.tenancy` says there is
* no cross-customer read at any layer, `access.matrix` scopes rows to their
* owner, and every entity lists the fields it holds and which of them are
* required. Nothing else in this repo checks any of that against a database.
* This builds the database from its SQL migrations, in-process, and holds it
* to them.
*
* Backend architecture is still yours. This checks that the server keeps the
* model's promises, and nothing about how it is built.
*
* HOW IT BUILDS THE DATABASE. PGlite: Postgres 17 compiled to WebAssembly and
* run inside Node, so there is no Docker, no network and no running server.
* SQLite schemas load into node:sqlite instead (Node 22.13 or later).
* For Supabase a shim loads first: the roles, the `auth` schema and its
* functions, and the grants Supabase gives `anon`, `authenticated` and
* `service_role` on every new table. Without those grants a table with row
* level security off reads as "permission denied", and a naive test counts
* that as a pass. Then every migration runs, in order, and the result is read
* back from the catalog into one JSON snapshot. Every check reads the
* snapshot, never the SQL.
*
* WHERE THE SQL COMES FROM. SQL migration files, or a command that prints SQL:
* Prisma (`migrate diff --from-empty --to-schema`), drizzle-kit (`export`),
* Alembic (`upgrade head --sql`), or any command you name. The command runs
* only when the cache misses, without a shell and with CONFIG.commandTimeoutMs,
* and its output loads like a migration file. No database is contacted: none of
* them needs one to print SQL.
*
* TWO GATES, ONE BOOT.
*
* schema The tables agree with the model's entities. A required field in a
* column that allows null, an entity mapped to a table the
* migrations never create, a column whose type cannot hold what the
* model means. Contradictions only: a field with no column is
* reported, and gated only if CONFIG.minFieldCoverage says so.
* access Row level security, linted and then tested: a user in one tenant
* tries to read, change, delete and create another tenant's rows.
*
* The check runs in a child process with a hard timeout, because PGlite does
* not enforce statement_timeout and a migration that sleeps holds the event
* loop for as long as it likes. Results are cached by a hash of everything
* they depend on, so a commit that touches nothing the database reads boots
* nothing.
*
* EXIT CODES, and exit 2 is never a pass.
*
* 0 every check ran and nothing failed.
* 1 at least one finding. Anything that could not be checked still prints.
* 2 no findings, and something could not be checked: no input, PGlite
* missing, a migration that does not load, an extension PGlite lacks, a
* malformed CONFIG or contract, a table the seeder cannot fill, a failed
* positive control, a probe error, or the timeout. A command that is not
* installed, fails, times out or prints no SQL, Prisma's schema engine not
* being installed, and a dialect this version cannot load (MySQL, MongoDB)
* are exit 2 too. With --allow-empty, NO-INPUT alone exits 0.
*
* A migration that does not load does not stop the check: every failing
* statement is reported, with file:line, classed as a real Postgres error, an
* ordering problem, something created outside the migrations, a duplicate, or
* a PGlite gap.
*
* Every exit-2 cause has a CONFIG escape with a reason: CONFIG.preamble for a
* few objects the migrations use and never create, a SQL file listed first in
* CONFIG.sources for a whole base schema, CONFIG.skipStatements for a
* statement PGlite cannot run, CONFIG.stubExtensions for an extension it does
* not have, CONFIG.untested for a table the test cannot fill. The escape is
* printed on every run, so an accepted gap stays visible.
*
* WHAT IT DOES NOT PROVE
*
* That the live database matches these migrations. Anyone with dashboard or
* SQL access can change it without a migration.
* Anything done with the service_role key, which skips RLS by design: Edge
* Functions, API routes and server code that use it.
* That a user cannot read a teammate's row inside their own tenant. It tests
* one tenant against another.
* That the matrix's verbs hold: that finance_admin cannot update an invoice.
* It tests isolation, not each permission.
* Claims your auth hook adds in production. The test only has the claims
* CONFIG.actors gives it.
* Column exposure. A row the user may read can still carry a column they
* should not see.
* Storage buckets and Realtime channels.
* What a SECURITY DEFINER function returns when called over RPC. A lint
* reads its body for the tables it names and whether it looks at the caller,
* and a finding needs both to be clear. Dynamic SQL is not followed.
* Writes as anon. Only what anon can read is probed.
* Updates and deletes through an RPC function. The probes use plain
* statements, as the API does.
* Speed. A correct policy that scans the table passes.
* Parity with the hosted Postgres. PGlite is Postgres 17 in WebAssembly;
* something it lacks is exit 2, never a pass.
* What a CONFIG.skipStatements entry, a CONFIG.preamble object or a base
* schema file stands in for. The checks run on the stand-in.
* Row level security on SQLite. It has none, so the access gate prints "not
* checked" and the schema gate is all that runs. Access lives in app code.
* What a command reads that CONFIG.sources does not list in `watch`. The
* cache is keyed on the watched files and the tool's version, so an input
* outside them (an env file, a generated file) goes stale until --no-cache.
* An Alembic migration that calls op.get_bind(): offline mode cannot run it.
* That the SQL a tool prints is the SQL it would run. It is the tool's
* from-empty output, not your migration history, and MySQL, SQL Server and
* MongoDB schemas are not loaded at all.
* SQLite's type rules. They are loose, so only a boolean in a REAL or BLOB
* column, or a number in a BLOB column, contradicts the model.
* Access enforced in app code. A query that forgets its tenant or owner
* filter (an IDOR) is not caught, and with enforcement 'app', or 'auto' on
* plain Postgres with no RLS, no access test runs. The summary then says
* "access NOT checked", and exit 0 is not "access is safe". Roadmap R1.
* Anything, when it exits 2.
*
* Usage:
* node scripts/check-backend.mjs both gates, full report with notes
* node scripts/check-backend.mjs --check hook and CI: one line on a pass
* node scripts/check-backend.mjs --schema the schema gate only (or --access)
* node scripts/check-backend.mjs --json machine output on stdout
* node scripts/check-backend.mjs --timing phase durations on stderr
* node scripts/check-backend.mjs --no-cache ignore the cache and do not write it
* node scripts/check-backend.mjs --print-config what the hook needs to wire it, no PGlite
* node scripts/check-backend.mjs --print-snapshot the schema snapshot as JSON
* node scripts/check-backend.mjs --root <dir> the repo root, default the current directory
* node scripts/check-backend.mjs --allow-empty no migrations yet is exit 0, for CI
*/
import { createHash } from 'node:crypto'
import { spawnSync } from 'node:child_process'
import {
existsSync,
mkdirSync,
readFileSync,
readdirSync,
realpathSync,
renameSync,
statSync,
unlinkSync,
utimesSync,
writeFileSync,
writeSync,
} from 'node:fs'
import { createRequire } from 'node:module'
import { delimiter, dirname, isAbsolute, join, relative, resolve, sep } from 'node:path'
import { fileURLToPath } from 'node:url'
// ---------------------------------------------------------------- CONFIG ---
export const CONFIG = {
// 'supabase' loads the shim below before any migration. 'postgres' loads
// nothing first, and the deny test acts as CONFIG.actAs. 'auto' reads the
// repo and picks one: Supabase when it finds supabase/config.toml, a
// @supabase/supabase-js or @supabase/ssr dependency, a supabase.co URL in an
// env example, or auth.uid() in the migrations; plain Postgres when it finds
// none. It skips templates, examples, docs and fixtures, and does not count a
// peer or optional dependency. The report header and --print-config say which,
// and why.
platform: 'auto',
// Who reaches the database directly. 'direct': a client does (Supabase's Data
// API, PostgREST, Hasura, a connection string in a browser or phone), so a
// table with RLS off is open to it and RLS-OFF is a finding. 'server-only':
// only your own server connects, so RLS-OFF is a note, and the deny test still
// fails any table the contract scopes. 'auto' is 'direct' on Supabase and
// 'server-only' on plain Postgres.
clientAccess: 'auto',
// Where the schema comes from, in load order. One database per run, so every
// source has the same dialect.
//
// { kind: 'sql' } SQL files.
// paths: files, directories, or one-level globs ('dir/*.sql', 'dir/*/migration.sql').
// order: 'name' sorts by path; 'natural' sorts V2 before V10 (Flyway);
// 'listed' keeps the order of paths, sorting only inside each one
// (['schema.sql', 'patches/*.sql'] loads schema.sql first).
// sections: null, 'dbmate' or 'goose'. Keeps only the up section.
// late: true loads these files after every other source. A statement that
// failed because something they add was missing is run again after them.
// dialect: 'postgres' (default, PGlite) or 'sqlite' (node:sqlite).
//
// { kind: 'command' } a tool that prints the schema as SQL. Runs only when
// the cache misses, with no shell and no database.
// preset: 'prisma' schema (default 'prisma/schema.prisma', a file or a folder).
// 'drizzle' config (default 'drizzle.config.ts').
// 'alembic' dialect is required; config (default 'alembic.ini').
// The tool is found from the schema's or config's own folder, up to
// the repo root: node_modules/.bin, or for alembic a .venv or venv.
// drizzle and alembic run from the config's folder.
// run: ['node_modules/.bin/tool', 'print-sql'] instead of a preset. dialect
// and watch are required.
// dialect: read from the Prisma datasource or the Drizzle config; set it to
// override, or when it cannot be read.
// watch: paths the command reads, beyond the preset's own. The cache is keyed
// on their bytes, so list everything the command reads.
// cwd: 'snapshot' (default; in the pre-commit hook, the staged files) or
// 'root' (the worktree; the default for alembic).
//
// Not sure what goes here? node scripts/check-backend.mjs --print-config looks
// for the schema at any depth and prints the line to paste for each find.
//
// e.g. { kind: 'command', preset: 'prisma', schema: 'prisma/schema.prisma' }
// { kind: 'command', preset: 'drizzle', config: 'drizzle.config.ts' }
// { kind: 'command', preset: 'alembic', dialect: 'postgres' }
// { kind: 'sql', paths: ['db/migrations/*.sql'], dialect: 'sqlite' }
sources: [{ kind: 'sql', paths: ['supabase/migrations/*.sql'], order: 'name', sections: null }],
// Hard limit for one command source. The check's own limit is timeoutMs.
commandTimeoutMs: 60000,
// What a migration set that does not load does. 'gate' (the default, and what
// a missing key means): the check cannot read the database, so it is exit 2
// and blocks the commit. 'warn': for a repo that has never loaded cleanly. A
// load failure is printed in full on every run and does not block, and the
// summary says the database was not checked. The first clean load in 'warn'
// is a finding that says to set 'gate', so the switch is a commit, and from
// then on a load failure blocks. Set 'warn' only while the migrations cannot
// rebuild the database yet; it is a ratchet, not a way to turn the check off.
loadGate: 'gate',
// How access is enforced.
// 'auto': 'rls' on Supabase. On plain Postgres, 'app' when the loaded schema
// has no RLS anywhere (no enable row level security, no policy), and 'rls'
// when it has some. On SQLite, 'app'. It never reads a Supabase repo as 'app',
// because RLS off on an exposed table is the hole this check exists to catch.
// 'rls': access is enforced by row level security, so the RLS gate runs.
// 'app': access is enforced in app code. The RLS gate prints "not checked"
// and says what that leaves open: no RLS lint and no cross-tenant test runs,
// so a query that forgets its tenant filter (an IDOR) is not caught. The
// schema gate still runs. App-code checks are roadmap (R1).
// 'auto' that lands on 'app' says so in the report. Set 'app' to confirm it
// and quiet the note, or 'rls' to hold the schema to having RLS.
enforcement: 'auto',
// Schemas the API exposes. Supabase's default. Lints only look here.
exposedSchemas: ['public'],
supabase: {
// Supabase grants anon, authenticated and service_role everything on new
// tables in public. If this project turned that off, set false, and the
// check stops assuming the grants.
defaultGrants: true,
},
// Postgres platform only. The role the app's queries run as, and the
// settings its policies read. {user} and {tenant} are filled per test.
actAs: { role: 'app_user', settings: { 'app.user_id': '{user}' } },
// Per actor in contract.json, what the app gives that actor in production:
// claims extra JWT claims (Supabase). claims.app_metadata also seeds
// auth.users.raw_app_meta_data, claims.user_metadata seeds
// raw_user_meta_data.
// role the database role to act as. Default 'authenticated'
// (Supabase) or actAs.role (postgres).
// settings extra set_config values (postgres), {user} and {tenant} filled.
// e.g. customer: { claims: { app_metadata: { role: 'customer' } } }
actors: {},
// Seed values the synthesiser cannot guess. Literal, or {user}, {tenant}, {n}.
// e.g. 'public.ledger': { currency: 'GBP' }
seed: {},
// How a field finds its column without fields[].column:
// 'exact'; 'camel' (field customer_id, column customerId);
// 'snake' (field customerId, column customer_id).
columnNames: 'exact',
// Lowest share of mapped fields that must have a column. null reports only.
minFieldCoverage: null,
// Tables the SCHEMA gate ignores. Prefix match with a trailing *. The RLS
// lints still read them: with default grants, public.alembic_version is
// readable by anon, and RLS-OFF says so, because it is. The fix is
// revoke all on <t> from anon, authenticated, or publicTables below.
ignoreTables: [
'public._prisma_migrations', 'public.__drizzle_migrations', 'drizzle.*',
'public.schema_migrations', 'public.ar_internal_metadata',
'public.alembic_version', 'public.auth_*', 'public.django_*',
'public.knex_migrations*', 'public.flyway_schema_history',
'supabase_migrations.*',
],
// ALLOWLISTS. Every entry needs a reason and an until date (YYYY-MM-DD).
// An entry without both refuses to run. An expired entry stops applying
// and is named. An entry that matches nothing is named as stale.
// Exposed tables allowed to have RLS off.
// 'public.countries': { reason: 'reference data, no user rows', until: '2027-03-31' }
publicTables: {},
// Extensions PGlite does not have, loaded as a no-op. Only safe when no
// table, policy or function the checks read depends on the extension.
stubExtensions: {},
// Statements PGlite cannot run, skipped one by one. The key is text that
// appears in the statement (case and spacing ignored); every statement that
// contains it is replaced by `select 1`. Safe only when nothing the checks
// read depends on the statement. It is printed on every run.
// e.g. 'timescaledb.compress': { reason: 'storage parameter, no table shape change', until: '2027-03-31' }
skipStatements: {},
// Tables, or 'table as actor', the deny test may leave untested.
untested: {},
// SQL run after the shim and before the migrations, for objects your
// migrations use that PGlite and the shim lack, and for `create extension`
// of one a migration uses but never creates. A reason is required; an
// entry without one is CONFIG-INVALID. No until date: it describes the
// platform, not a debt. For more than a few objects (a base schema made
// outside the migrations), list a SQL file as the first entry of
// CONFIG.sources instead.
// e.g. { sql: 'create table auth.identities (id uuid primary key, user_id uuid)', reason: 'FK target only' }
preamble: [],
// Schema findings accepted for now, by 'schema.table.column' (or
// 'schema.table' for ENTITY-NO-TABLE).
schemaWaivers: {},
// Hard limit for the whole run. PGlite does not enforce statement_timeout.
timeoutMs: 60000,
cacheDir: 'node_modules/.cache/ntent/backend',
}
const deepFreeze = (value) => {
if (value && typeof value === 'object') {
for (const v of Object.values(value)) deepFreeze(v)
Object.freeze(value)
}
return value
}
/**
* CONFIG as shipped, frozen, written out again on purpose. The install steps
* tell a consumer to edit CONFIG (sources, platform, actAs), so a copy taken of
* CONFIG would follow the edit. Tests build their config from this and never
* from CONFIG, so the shipped test still passes after a consumer customises it.
* Keep it equal to the shipped CONFIG above; the package's own tests check that.
*/
export const DEFAULT_CONFIG = deepFreeze({
platform: 'auto',
clientAccess: 'auto',
sources: [{ kind: 'sql', paths: ['supabase/migrations/*.sql'], order: 'name', sections: null }],
commandTimeoutMs: 60000,
loadGate: 'gate',
enforcement: 'auto',
exposedSchemas: ['public'],
supabase: { defaultGrants: true },
actAs: { role: 'app_user', settings: { 'app.user_id': '{user}' } },
actors: {},
seed: {},
columnNames: 'exact',
minFieldCoverage: null,
ignoreTables: [
'public._prisma_migrations', 'public.__drizzle_migrations', 'drizzle.*',
'public.schema_migrations', 'public.ar_internal_metadata',
'public.alembic_version', 'public.auth_*', 'public.django_*',
'public.knex_migrations*', 'public.flyway_schema_history',
'supabase_migrations.*',
],
publicTables: {},
stubExtensions: {},
skipStatements: {},
untested: {},
preamble: [],
schemaWaivers: {},
timeoutMs: 60000,
cacheDir: 'node_modules/.cache/ntent/backend',
})
// ------------------------------------------------------------- CONSTANTS ---
export const SNAPSHOT_FORMAT = 'ntent-schema/1'
/** Any change to the shim bumps this, which invalidates every cache. */
export const SHIM_VERSION = 'supabase/2'
/**
* Schemas whose tables are the platform's, not the app's, and are left out of
* the snapshot. auth.users is the exception, kept with `platform: true`,
* because lints and seeding refer to it. Anything starting pg_ is left out too.
*/
const PLATFORM_SCHEMAS = [
'information_schema', 'auth', 'storage', 'extensions', 'net', 'cron', 'vault',
'realtime', 'supabase_functions', 'supabase_migrations',
]
const KINDS = { r: 'table', v: 'view', m: 'materialized_view', p: 'partitioned_table', f: 'foreign_table' }
/** Relation kinds that carry NOT NULL. A view's columns always read as nullable. */
const NULLABLE_KNOWN = new Set(['table', 'partitioned_table', 'foreign_table'])
const ISO_DATE = /^\d{4}-\d{2}-\d{2}$/
const SELF = fileURLToPath(import.meta.url)
// ------------------------------------------------------------------ SHIM ---
// Not Supabase: the parts a migration and a policy touch. Verified on PGlite
// 0.5.8 to load, to give authenticated SELECT on a table created afterwards
// (and not on auth.users), and to let auth.uid(), auth.role() and auth.jwt()
// read the claims.
const SHIM_BASE = `-- ntent supabase shim 2. Not Supabase: the parts a migration and a policy touch.
do $$
declare r text;
begin
foreach r in array array['anon','authenticated','service_role','authenticator',
'supabase_admin','supabase_auth_admin','supabase_storage_admin','dashboard_user','pgbouncer']
loop
if not exists (select 1 from pg_roles where rolname = r) then
execute format('create role %I nologin noinherit', r);
end if;
end loop;
end $$;
alter role service_role bypassrls;
alter role authenticator login;
grant anon, authenticated, service_role to authenticator;
create schema if not exists auth;
create schema if not exists extensions;
create schema if not exists storage;
create table if not exists auth.users (
instance_id uuid,
id uuid primary key,
aud varchar(255),
role varchar(255),
email varchar(255) unique,
encrypted_password varchar(255),
email_confirmed_at timestamptz,
phone text unique,
raw_app_meta_data jsonb default '{}'::jsonb,
raw_user_meta_data jsonb default '{}'::jsonb,
is_super_admin boolean,
is_anonymous boolean not null default false,
created_at timestamptz default now(),
updated_at timestamptz default now(),
last_sign_in_at timestamptz,
deleted_at timestamptz
);
create or replace function auth.uid() returns uuid language sql stable as $$
select coalesce(
nullif(current_setting('request.jwt.claim.sub', true), ''),
(nullif(current_setting('request.jwt.claims', true), '')::jsonb ->> 'sub')
)::uuid
$$;
create or replace function auth.role() returns text language sql stable as $$
select coalesce(
nullif(current_setting('request.jwt.claim.role', true), ''),
(nullif(current_setting('request.jwt.claims', true), '')::jsonb ->> 'role')
)::text
$$;
create or replace function auth.email() returns text language sql stable as $$
select coalesce(
nullif(current_setting('request.jwt.claim.email', true), ''),
(nullif(current_setting('request.jwt.claims', true), '')::jsonb ->> 'email')
)::text
$$;
create or replace function auth.jwt() returns jsonb language sql stable as $$
select coalesce(
nullif(current_setting('request.jwt.claim', true), ''),
nullif(current_setting('request.jwt.claims', true), '')
)::jsonb
$$;
create table if not exists storage.buckets (
id text primary key, name text not null unique, owner uuid, public boolean default false,
created_at timestamptz default now(), updated_at timestamptz default now()
);
create table if not exists storage.objects (
id uuid primary key default gen_random_uuid(), bucket_id text references storage.buckets(id),
name text, owner uuid, owner_id text, metadata jsonb, path_tokens text[],
created_at timestamptz default now(), updated_at timestamptz default now()
);
alter table storage.objects enable row level security;
alter table storage.buckets enable row level security;
create or replace function storage.foldername(name text) returns text[] language sql immutable as $$
select (string_to_array(name, '/'))[1:array_length(string_to_array(name, '/'), 1) - 1]
$$;
create or replace function storage.filename(name text) returns text language sql immutable as $$
select (string_to_array(name, '/'))[array_length(string_to_array(name, '/'), 1)]
$$;
create or replace function storage.extension(name text) returns text language sql immutable as $$
select reverse(split_part(reverse(storage.filename(name)), '.', 1))
$$;
create publication supabase_realtime;
grant usage on schema public, auth, extensions, storage to anon, authenticated, service_role;
grant all on storage.objects, storage.buckets to anon, authenticated, service_role;
grant all on auth.users to service_role;
-- Added in shim 2: objects that routinely appear in Supabase migrations and
-- \`supabase db dump\`, and failed against shim 1.
do $$
declare r text;
begin
foreach r in array array['supabase_realtime_admin','supabase_replication_admin','supabase_read_only_user','supabase_functions_admin'] loop
if not exists (select 1 from pg_roles where rolname = r) then execute format('create role %I nologin noinherit', r); end if;
end loop;
end $$;
alter table storage.buckets
add column if not exists owner_id text,
add column if not exists file_size_limit bigint,
add column if not exists allowed_mime_types text[],
add column if not exists avif_autodetection boolean default false;
alter table storage.objects
add column if not exists version text,
add column if not exists user_metadata jsonb,
add column if not exists last_accessed_at timestamptz default now();
alter table auth.users
add column if not exists phone_confirmed_at timestamptz,
add column if not exists confirmed_at timestamptz,
add column if not exists banned_until timestamptz,
add column if not exists is_sso_user boolean not null default false;
create schema if not exists supabase_functions;
create or replace function supabase_functions.http_request() returns trigger language plpgsql as $$ begin return new; end $$;
create schema if not exists realtime;
create table if not exists realtime.messages (
id uuid primary key default gen_random_uuid(), topic text not null, extension text not null,
payload jsonb, event text, private boolean default false,
inserted_at timestamptz default now(), updated_at timestamptz default now()
);
alter table realtime.messages enable row level security;
create or replace function realtime.topic() returns text language sql stable as $$
select nullif(current_setting('realtime.topic', true), '')::text
$$;
grant usage on schema realtime to anon, authenticated, service_role;
grant all on realtime.messages to anon, authenticated, service_role;
`
// The default grants. Without these a table with RLS off reads as
// "permission denied" and the deny test passes it. Emitted only when
// CONFIG.supabase.defaultGrants is true.
const SHIM_DEFAULT_GRANTS = `
grant all on all tables in schema public to anon, authenticated, service_role;
grant all on all sequences in schema public to anon, authenticated, service_role;
grant all on all functions in schema public to anon, authenticated, service_role;
alter default privileges for role postgres in schema public grant all on tables to anon, authenticated, service_role;
alter default privileges for role postgres in schema public grant all on sequences to anon, authenticated, service_role;
alter default privileges for role postgres in schema public grant all on functions to anon, authenticated, service_role;
`
/** The shim SQL for a config: shim 2, and the default grants unless turned off. */
export const shimSql = (config) => SHIM_BASE + (config.supabase?.defaultGrants === false ? '' : SHIM_DEFAULT_GRANTS)
// PGlite 0.5.8 needs pgcrypto and uuid_ossp passed to PGlite.create before
// these can run (verified).
const AFTER_SHIM =
'create extension if not exists pgcrypto with schema extensions;\n' +
'create extension if not exists "uuid-ossp" with schema extensions;\n' +
'set search_path = "$user", public, extensions;'
// pg_dump-style migrations set search_path = '' and row_security = off.
// Every check after the load runs with neither.
const RESET_SESSION = 'reset all; set role none; set search_path = "$user", public, extensions; set row_security = on;'
/** Extensions the shim has already created, on platform 'supabase'. */
const SHIMMED_EXTENSIONS = new Set(['pgcrypto', 'uuid-ossp', 'plpgsql'])
/**
* Built-in stubs, always allowed, emitted in place of the matching create
* extension. One statement per line, because the replacement has to keep the
* migration's line count so error positions still map to its lines.
*/
const STUBS = {
pg_net: [
'create schema if not exists net;',
"create or replace function net.http_post(url text, body jsonb default '{}', params jsonb default '{}', headers jsonb default '{}', timeout_milliseconds int default 5000) returns bigint language sql as $$ select 0::bigint $$;",
"create or replace function net.http_get(url text, params jsonb default '{}', headers jsonb default '{}', timeout_milliseconds int default 5000) returns bigint language sql as $$ select 0::bigint $$;",
],
pg_cron: [
'create schema if not exists cron;',
'create or replace function cron.schedule(job_name text, schedule text, command text) returns bigint language sql as $$ select 0::bigint $$;',
'create or replace function cron.schedule(schedule text, command text) returns bigint language sql as $$ select 0::bigint $$;',
'create or replace function cron.unschedule(job_name text) returns boolean language sql as $$ select true $$;',
],
supabase_vault: [
'create schema if not exists vault;',
'create table if not exists vault.secrets (id uuid primary key default gen_random_uuid(), name text unique, secret text, description text);',
'create or replace view vault.decrypted_secrets as select *, secret as decrypted_secret from vault.secrets;',
"create or replace function vault.create_secret(new_secret text, new_name text default null, new_description text default '') returns uuid language sql as $$ insert into vault.secrets(name, secret, description) values (new_name, new_secret, new_description) returning id $$;",
],
pg_graphql: ['select 1;'],
pgsodium: ['select 1;'],
pgjwt: ['select 1;'],
pg_stat_monitor: ['select 1;'],
}
// ---------------------------------------------------------------- ERRORS ---
/**
* Something could not be checked. Always exit 2, never a pass. Carries what
* the report prints: a code, the object it is about, what happened and the fix.
*/
export class CannotCheck extends Error {
constructor(code, { object = '', message, fix = '', ...extra } = {}) {
super(message)
this.code = code
this.object = object
this.fix = fix
this.extra = extra
}
toJSON() {
return { code: this.code, object: this.object, message: this.message, fix: this.fix, ...this.extra }
}
}
// ------------------------------------------------------------------- CLI ---
const FLAGS = new Set([
'--check', '--schema', '--access', '--json', '--timing', '--no-cache',
'--print-config', '--print-snapshot', '--allow-empty',
])
/** Flags into options. Anything unknown refuses: a typo of --check that ran as a report would pass a commit. */
export function parseArgs(argv) {
const opts = { root: null }
const rest = [...argv]
while (rest.length) {
const arg = rest.shift()
if (arg === '--root') {
const value = rest.shift()
if (value === undefined || value.startsWith('--')) {
throw new CannotCheck('USAGE', { message: '--root needs a value.', fix: 'Pass the repo root: --root <dir>.' })
}
opts.root = value
} else if (FLAGS.has(arg)) {
opts[arg.slice(2).replace(/-(\w)/g, (_, c) => c.toUpperCase())] = true
} else {
throw new CannotCheck('USAGE', {
object: arg,
message: `unknown argument: ${arg}`,
fix: 'See the usage at the top of scripts/check-backend.mjs.',
})
}
}
if (opts.schema && opts.access) {
throw new CannotCheck('USAGE', {
message: '--schema and --access each run one gate. Pass neither to run both.',
fix: 'Drop one of them.',
})
}
opts.gates = { schema: !opts.access, access: !opts.schema }
return opts
}
// ------------------------------------------------------------ VALIDATION ---
const isObject = (v) => v !== null && typeof v === 'object' && !Array.isArray(v)
const nonEmpty = (v) => typeof v === 'string' && v.trim() !== ''
/** Whether an entity's owner sits on a parent row: a ref field, up to 3 hops, to an entity that has an owner. */
const reachesOwner = (entity, byId, depth = 0, seen = new Set()) => {
if (!isObject(entity) || depth >= 3 || seen.has(entity.id)) return false
for (const raw of Array.isArray(entity.fields) ? entity.fields : []) {
const f = typeof raw === 'string' ? { name: raw } : raw
const parent = isObject(f) && f.ref !== undefined ? byId.get(f.ref) : null
if (!parent) continue
if (parent.owner !== undefined || reachesOwner(parent, byId, depth + 1, new Set([...seen, entity.id]))) return true
}
return false
}
const validDate = (s) => {
if (!ISO_DATE.test(s)) return false
const d = new Date(`${s}T00:00:00Z`)
return !Number.isNaN(d.getTime()) && d.toISOString().startsWith(s)
}
const ALLOWLISTS = ['publicTables', 'stubExtensions', 'untested', 'schemaWaivers']
// Added later than the others, so a CONFIG written before it still validates.
const OPTIONAL_ALLOWLISTS = ['skipStatements']
const PRESETS = ['prisma', 'drizzle', 'alembic']
const DIALECTS = ['postgres', 'postgresql', 'sqlite']
/** Dialects a tool can print that this version does not load, and the roadmap item that will. */
const UNSUPPORTED = {
mysql: 'roadmap R3, MySQL',
mariadb: 'roadmap R3, MySQL',
singlestore: 'roadmap R3, MySQL',
mongodb: 'roadmap R4, MongoDB',
sqlserver: 'no roadmap item yet',
mssql: 'no roadmap item yet',
cockroachdb: 'no roadmap item yet',
}
/**
* Everything wrong with a config, as CONFIG-INVALID problems. A malformed
* allowlist entry refuses to run: an exception with no reason or no date is
* the one thing an allowlist that demands them exists to stop.
*/
export function validateConfig(config) {
const problems = []
const bad = (object, message, fix) => problems.push({ code: 'CONFIG-INVALID', object, message, fix })
if (!isObject(config)) {
bad('CONFIG', 'is not an object', 'Restore the CONFIG block from the shipped file.')
return problems
}
const is = (v) => `is ${JSON.stringify(v)}`
const inList = (v, list) => list.includes(v)
const inRange = (v, lo, hi) => v >= lo && v <= hi
const listOf = (v, test = nonEmpty) => Array.isArray(v) && v.every(test)
// [object, ok, message, fix]
const rules = [
['platform', inList(config.platform, ['auto', 'supabase', 'postgres']), is(config.platform), "Set it to 'auto', 'supabase' or 'postgres'."],
[
'clientAccess',
config.clientAccess === undefined || inList(config.clientAccess, ['auto', 'direct', 'server-only']),
is(config.clientAccess),
"Set 'auto', 'direct' (a client reaches the database) or 'server-only' (only your server does).",
],
[
'loadGate',
config.loadGate === undefined || inList(config.loadGate, ['warn', 'gate']),
is(config.loadGate),
"Set 'gate' (a migration set that does not load blocks the commit) or 'warn' (it is reported and does not block, until the first clean load).",
],
[
'sources',
listOf(config.sources, isObject) && config.sources.length > 0,
'must be a non-empty list of sources',
"Point it at the migrations: [{ kind: 'sql', paths: ['supabase/migrations/*.sql'], order: 'name', sections: null }].",
],
['enforcement', inList(config.enforcement, ['auto', 'rls', 'app']), is(config.enforcement), "Set 'auto', 'rls' or 'app'."],
['exposedSchemas', listOf(config.exposedSchemas), 'must be a list of schema names', "Supabase's default is ['public']."],
[
'supabase.defaultGrants',
typeof config.supabase?.defaultGrants === 'boolean',
'must be true or false',
'Leave it true unless this project turned the default grants off.',
],
[
'actAs',
isObject(config.actAs) && nonEmpty(config.actAs.role) && isObject(config.actAs.settings ?? {}),
'needs a role name and a settings object',
"Write { role: 'app_user', settings: { 'app.user_id': '{user}' } }.",
],
['actors', isObject(config.actors), 'must be an object keyed by actor id', 'Write {} or { customer: { claims: { ... } } }.'],
['seed', isObject(config.seed), "must be an object keyed by 'schema.table'", "Write {} or { 'public.ledger': { currency: 'GBP' } }."],
['columnNames', inList(config.columnNames, ['exact', 'camel', 'snake']), is(config.columnNames), "Set 'exact', 'camel' or 'snake'."],
[
'minFieldCoverage',
config.minFieldCoverage === null || (typeof config.minFieldCoverage === 'number' && inRange(config.minFieldCoverage, 0, 1)),
is(config.minFieldCoverage),
'Set null, or a share between 0 and 1.',
],
['ignoreTables', listOf(config.ignoreTables), "must be a list of 'schema.table' names", "Prefix match with a trailing *, e.g. 'drizzle.*'."],
['preamble', Array.isArray(config.preamble), 'must be a list', "Write [] or [{ sql: '...', reason: '...' }]."],
[
'timeoutMs',
typeof config.timeoutMs === 'number' && config.timeoutMs > 0,
is(config.timeoutMs),
'Set a number of milliseconds. The default is 60000.',
],
['cacheDir', nonEmpty(config.cacheDir), 'must be a path', "The default is 'node_modules/.cache/ntent/backend'."],
]
for (const [object, ok, message, fix] of rules) if (!ok) bad(object, message, fix)
if (config.commandTimeoutMs !== undefined && !(typeof config.commandTimeoutMs === 'number' && config.commandTimeoutMs > 0)) {
bad('commandTimeoutMs', is(config.commandTimeoutMs), 'Set a number of milliseconds. The default is 60000.')
}
for (const [i, source] of (listOf(config.sources, isObject) ? config.sources : []).entries()) {
const at = `sources[${i}]`
if (!inList(source.kind, ['sql', 'command'])) {
bad(`${at}.kind`, is(source.kind), "Set kind: 'sql' for SQL files, or 'command' for a tool that prints SQL.")
continue
}
if (source.dialect !== undefined && !inList(source.dialect, DIALECTS)) {
const later = UNSUPPORTED[String(source.dialect).toLowerCase()]
bad(
`${at}.dialect`,
later ? `${is(source.dialect)}, which this version does not load (${later})` : is(source.dialect),
"Set 'postgres' or 'sqlite'.",
)
}
if (source.kind === 'sql') {
if (!listOf(source.paths) || !source.paths.length) {
bad(`${at}.paths`, 'must be a non-empty list of paths', "List files, directories or one-level globs, e.g. 'supabase/migrations/*.sql'.")
}
if (source.order !== undefined && !inList(source.order, ['name', 'natural', 'listed'])) {
bad(`${at}.order`, is(source.order), "Set 'name', 'natural' or 'listed'.")
}
if (source.sections !== undefined && !inList(source.sections, [null, 'dbmate', 'goose'])) {
bad(`${at}.sections`, is(source.sections), "Set null, 'dbmate' or 'goose'.")
}
if (source.late !== undefined && typeof source.late !== 'boolean') {
bad(`${at}.late`, is(source.late), 'Set true or false.')
}
continue
}
const hasPreset = source.preset !== undefined
const hasRun = source.run !== undefined
if (hasPreset === hasRun) {
bad(at, hasPreset ? 'has both preset and run' : 'has neither preset nor run', "Name a preset ('prisma', 'drizzle', 'alembic') or give run: ['tool', 'arg'].")
}
if (hasPreset && !inList(source.preset, PRESETS)) {
bad(`${at}.preset`, is(source.preset), "Use 'prisma', 'drizzle' or 'alembic', or give run: [...] for another tool.")
}
if (hasRun) {
if (!listOf(source.run) || !source.run.length) bad(`${at}.run`, 'must be a non-empty list: the command, then its arguments', "Write run: ['node_modules/.bin/tool', 'print-sql'].")
if (source.dialect === undefined) bad(`${at}.dialect`, 'is required with run', "Set dialect: 'postgres' or 'sqlite'.")
if (!listOf(source.watch) || !source.watch?.length) {
bad(`${at}.watch`, 'is required with run', "List what the command reads, e.g. watch: ['db/schema/']. The cache is keyed on those files.")
}
}
if (source.preset === 'alembic' && source.dialect === undefined) {
bad(`${at}.dialect`, 'is required for alembic', "Set dialect: 'postgres' or 'sqlite'. Alembic's offline SQL does not say which.")
}
if (source.watch !== undefined && !listOf(source.watch)) bad(`${at}.watch`, 'must be a list of paths', "Write watch: ['db/schema/'].")
if (source.cwd !== undefined && !inList(source.cwd, ['snapshot', 'root'])) bad(`${at}.cwd`, is(source.cwd), "Set 'snapshot' or 'root'.")
for (const key of ['schema', 'config']) {
if (source[key] !== undefined && !nonEmpty(source[key])) bad(`${at}.${key}`, 'must be a path', 'Write the path relative to the repo root.')
}
}
for (const list of [...ALLOWLISTS, ...OPTIONAL_ALLOWLISTS]) {
const entries = config[list]
if (entries === undefined && OPTIONAL_ALLOWLISTS.includes(list)) continue
if (!isObject(entries)) {
bad(list, 'must be an object', `Write {} or { 'key': { reason: '...', until: 'YYYY-MM-DD' } }.`)
continue
}
for (const [key, entry] of Object.entries(entries)) {
const at = `${list}['${key}']`
if (!isObject(entry) || !nonEmpty(entry.reason)) {
bad(at, 'has no reason', 'Say why this is acceptable, in a sentence someone can challenge, or delete the entry.')
}
if (!isObject(entry) || entry.until === undefined) {
bad(at, 'has no until date', 'Add until: YYYY-MM-DD. An exception with no date never leaves.')
} else if (typeof entry.until !== 'string' || !validDate(entry.until)) {
bad(at, `has until: ${JSON.stringify(entry.until)}`, 'Write the date as YYYY-MM-DD.')
}
}
}
for (const [i, entry] of (Array.isArray(config.preamble) ? config.preamble : []).entries()) {
if (!isObject(entry) || !nonEmpty(entry.sql)) {
bad(`preamble[${i}]`, 'has no sql', 'Write the statement that creates what the migrations need.')
}
if (!isObject(entry) || !nonEmpty(entry.reason)) {
bad(`preamble[${i}]`, 'has no reason', 'Say what the migrations use that PGlite or the shim lacks.')
}
}
return problems
}
const TABLE_NAME = /^[A-Za-z_][\w$]*(\.[A-Za-z_][\w$]*)?$/
/** A field in its object form. A bare string is `{ name }`, untyped and optional. */
export const fieldOf = (f) => (typeof f === 'string' ? { name: f } : f)
/** 'invoices' is 'public.invoices'. */
export const qualify = (table) => (table.includes('.') ? table : `public.${table}`)
/**
* The contract fields this check reads, validated here rather than trusted to
* --refs: a consumer's installed check-contract-coverage is never upgraded.
* Errors are CONTRACT-SHAPE (exit 2); warnings are notes.
*/
export function validateContract(contract) {
const errors = []
const warnings = []
const err = (where, message, fix) => errors.push({ code: 'CONTRACT-SHAPE', object: where, message, fix })
const warn = (where, message, fix) => warnings.push({ code: 'CONTRACT-WARNING', object: where, message, fix })
if (!isObject(contract)) {
err('contract.json', 'is not a JSON object', 'Restore the file, or delete it if this repo has no contract yet.')
return { errors, warnings }
}
if (contract.actors !== undefined && !Array.isArray(contract.actors)) {
err('actors', 'must be a list', 'Write actors as a list of { id, kind }.')
}
if (contract.entities !== undefined && !Array.isArray(contract.entities)) {
err('entities', 'must be a list', 'Write entities as a list of { id, fields }.')
return { errors, warnings }
}
const entities = contract.entities ?? []
const ids = new Set(entities.filter((e) => isObject(e) && nonEmpty(e.id)).map((e) => e.id))
const tables = new Map()
entities.forEach((e, i) => {
if (!isObject(e) || !nonEmpty(e.id)) return err(`entities[${i}]`, 'has no id', 'Give every entity a string id.')
const at = `entities.${e.id}`
if (e.table !== undefined) {
if (typeof e.table !== 'string' || !TABLE_NAME.test(e.table)) {
err(`${at}.table`, 'must be a table name, "table" or "schema.table"', 'Write the table the rows live in, e.g. "public.invoices".')
} else {
const key = qualify(e.table)
if (tables.has(key)) {
warn(`${at}.table`, `"${key}" is also the table of ${tables.get(key)}`, 'Fine for single-table inheritance. Otherwise one of them is wrong.')
}
else tables.set(key, e.id)
}
}
if (e.fields !== undefined && !Array.isArray(e.fields)) {
return err(`${at}.fields`, 'must be a list', 'Write fields as a list of names or { name, type } objects.')
}
const fields = (e.fields ?? []).map(fieldOf)
fields.forEach((f, j) => {
if (!isObject(f) || !nonEmpty(f.name)) {
return err(`${at}.fields[${j}]`, 'has no name', 'Write the field as a name, or an object with a name.')
}
const fat = `${at}.fields.${f.name}`
if (f.ref !== undefined) {
if (typeof f.ref !== 'string' || !ids.has(f.ref)) {
err(`${fat}.ref`, `"${f.ref}" is not an entity`, 'Point ref at an entity id, or remove it.')
}
if (f.type !== 'id') err(`${fat}.ref`, 'ref is only valid on an id field', 'Set type to "id", or remove ref.')
}
if (f.column !== undefined && !nonEmpty(f.column)) {
err(`${fat}.column`, 'must be a column name', "Write the column's name as it is in the database.")
}
})
if (e.owner !== undefined && !fields.some((f) => isObject(f) && f.name === e.owner)) {
err(`${at}.owner`, `"${e.owner}" is not a field of ${e.id}`, "Name the field that holds the owning user's id, or remove owner.")
}
// New vocabulary: it warns and never fails, so an older script and a newer contract still agree.
if (e.owner_via !== undefined && !fields.some((f) => isObject(f) && f.name === e.owner_via && f.ref !== undefined)) {
warn(`${at}.owner_via`, `"${e.owner_via}" is not a ref field of ${e.id}, so it is ignored`, 'Name the id field whose ref leads to the entity that has an owner, or remove owner_via.')
}
})
const access = contract.access
if (access !== undefined && !isObject(access)) err('access', 'must be an object', 'Write access as { tenancy, matrix }.')
const tenancy = isObject(access) ? access.tenancy : undefined
if (tenancy !== undefined && !isObject(tenancy)) {
err('access.tenancy', 'must be an object', 'Write tenancy as { scope_entity, column, membership, rule }.')
}
if (isObject(tenancy)) {
if (tenancy.column !== undefined && !nonEmpty(tenancy.column)) {
err('access.tenancy.column', 'must be a column name', 'Name the column every tenant-scoped table carries, e.g. "customer_id".')
}
if (tenancy.membership !== undefined) {
const m = tenancy.membership
if (!isObject(m) || ![m.table, m.user_column, m.tenant_column].every(nonEmpty)) {
err('access.tenancy.membership', 'needs table, user_column and tenant_column', 'Name the table linking users to tenants, and its two columns.')
} else if (!TABLE_NAME.test(m.table)) {
err('access.tenancy.membership.table', 'must be a table name, "table" or "schema.table"', 'Name the table linking users to tenants, e.g. "public.members".')
}
}
}
const matrix = isObject(access) ? access.matrix : undefined
if (matrix !== undefined && !Array.isArray(matrix)) {
err('access.matrix', 'must be a list', 'Write the matrix as a list of { actor, entity, can, scope }.')
}
if (Array.isArray(matrix)) {
const byId = new Map(entities.filter(isObject).map((e) => [e.id, e]))
matrix.forEach((row, i) => {
if (!isObject(row) || row.scope === undefined) return
const at = `access.matrix[${i}].scope`
if (!['own', 'tenant', 'all'].includes(row.scope)) {
warn(at, `"${row.scope}" is not a scope the checks know, so this row is not tested`, 'Use own, tenant or all. Free text belongs in why.')
} else if (row.scope === 'tenant' && !tenancy?.column && !tenancy?.membership) {
warn(at, 'tenant scope with no way to find the tenant', 'Set access.tenancy.column.')
} else if (row.scope === 'own' && !byId.get(row.entity)?.owner && !tenancy?.column && !reachesOwner(byId.get(row.entity), byId)) {
warn(at, 'own scope with no owner column', `Set owner on ${row.entity}, or access.tenancy.column.`)
}
})
}
return { errors, warnings }
}
// ---------------------------------------------------------------- INPUTS ---
const posix = (p) => p.split(sep).join('/')
const hasGlob = (s) => /[*?]/.test(s)
const globRe = (seg) => new RegExp(`^${seg.replace(/[.+^${}()|[\]\\]/g, '\\$&').replace(/\*/g, '.*').replace(/\?/g, '.')}$`)
const isDir = (p) => {
try {
return statSync(p).isDirectory()
} catch {
return false
}
}
/** One CONFIG.sources path to the files and directories it names, relative to root. */
function globBases(root, pattern) {
let bases = ['']
for (const seg of pattern.split('/').filter(Boolean)) {
const next = []
for (const base of bases) {
if (!hasGlob(seg)) {
next.push(base ? `${base}/${seg}` : seg)
continue
}
let entries
try {
entries = readdirSync(join(root, base))
} catch {
continue
}
const re = globRe(seg)
for (const e of entries) if (!e.startsWith('.') && re.test(e)) next.push(base ? `${base}/${e}` : e)
}
bases = next
}
return bases
}
/** One CONFIG.sources path to the SQL files it names, relative to root. */
function expandPath(root, pattern) {
const files = []
for (const base of globBases(root, pattern)) {
const full = join(root, base)
let st
try {
st = statSync(full)
} catch {
continue
}
if (st.isFile()) files.push(base)
else if (st.isDirectory()) {
for (const e of readdirSync(full)) {
if (e.endsWith('.sql') && statSync(join(full, e)).isFile()) files.push(`${base}/${e}`)
}
}
}
return files
}
const byName = (a, b) => (a < b ? -1 : a > b ? 1 : 0)
const byNatural = (a, b) => a.localeCompare(b, 'en', { numeric: true })
/** Every input file, in load order: [{ path, sections, late }]. */
export function resolveInputs(root, sources) {
const seen = new Set()
const out = []
for (const source of sources) {
if (source.kind === 'command') continue
const sort = (list) => list.sort(source.order === 'natural' ? byNatural : byName)
// 'listed' keeps the order of `paths`: a base schema first, then its patches.
const found = source.order === 'listed' ? [...new Set(source.paths.flatMap((p) => sort(expandPath(root, p))))] : sort([...new Set(source.paths.flatMap((p) => expandPath(root, p)))])
for (const path of found) {
if (seen.has(path)) continue
seen.add(path)
out.push({ path, sections: source.sections ?? null, late: source.late === true })
}
}
return out
}
/** What the hook watches for one source path: a glob's directory, a directory with a trailing /, or the file. */
function watchPaths(root, pattern) {
const segs = pattern.split('/').filter(Boolean)
const firstGlob = segs.findIndex(hasGlob)
if (firstGlob > 0) return [`${segs.slice(0, firstGlob).join('/')}/`]
if (firstGlob === 0) return expandPath(root, pattern)
const path = segs.join('/')
if (pattern.endsWith('/') || isDir(join(root, path))) return [`${path}/`]
return [path]
}
/**
* Keep only the up section of a dbmate or goose migration. The rest becomes
* blank lines, so a line number still points at the right line of the file.
*/
function upSection(text, sections) {
if (!sections) return text
const [up, down] =
sections === 'dbmate'
? [/^--\s*migrate:up\b/, /^--\s*migrate:down\b/]
: [/^--\s*\+goose\s+Up\b/i, /^--\s*\+goose\s+Down\b/i]
const lines = text.split('\n')
if (!lines.some((l) => up.test(l.trim()))) return text
let inside = false
return lines
.map((line) => {
const t = line.trim()
if (up.test(t)) {
inside = true
return ''
}
if (down.test(t)) inside = false
return inside ? line : ''
})
.join('\n')
}
// -------------------------------------------------------- COMMAND SOURCES ---
// A command source runs a tool that prints the schema as SQL, then loads the
// SQL like a migration file. The parent plans it (which files it reads, which
// dialect, which tool version) so the cache key needs no run, and runs it only
// when the cache misses.
const normalDialect = (d) => (d === 'postgresql' ? 'postgres' : d)
const cleanPath = (p) => p.replace(/^(\.\/)+/, '')
const stripComments = (sql) => sql.replace(/--.*$/gm, '').replace(/\/\*[\s\S]*?\*\//g, '').trim()
/** Every file under each watch path (globs allowed), relative to base, with its bytes. Skips dot entries and node_modules. */
function watchedFiles(base, patterns) {
const found = new Set()
const walk = (rel) => {
let st
try {
st = statSync(join(base, rel))
} catch {
return
}
if (st.isFile()) found.add(rel)
else if (st.isDirectory()) {
for (const e of readdirSync(join(base, rel)).sort()) {
if (found.size > 5000) return
if (!e.startsWith('.') && e !== 'node_modules') walk(`${rel}/${e}`)
}
}
}
for (const pattern of patterns) for (const rel of globBases(base, cleanPath(pattern))) walk(rel)
return [...found].sort(byName).map((path) => ({ path, bytes: readFileSync(join(base, path)) }))
}
const packageVersion = (modules, name) => {
try {
return JSON.parse(readFileSync(join(modules, name, 'package.json'), 'utf8')).version ?? null
} catch {
return null
}
}
/** 'apps/api/prisma' to ['apps/api/prisma', 'apps/api', 'apps', '']: a folder and every one above it, up to the root. */
const dirsUp = (rel) => {
const out = []
let dir = cleanPath(rel ?? '').replace(/\/+$/, '')
while (dir && dir !== '.') {
out.push(dir)
dir = dir.includes('/') ? dir.slice(0, dir.lastIndexOf('/')) : ''
}
return [...out, '']
}
const at = (dir, ...parts) => posix(join(dir, ...parts))
const dirOf = (rel) => (rel.includes('/') ? rel.slice(0, rel.lastIndexOf('/')) : '')
/**
* The folders above `base` up to the git top level, as '..', '../..'. A package
* install keeps its tools in a hoisted workspace root, so the walk goes past
* the install folder when it holds none of its own.
*/
const aboveBase = (base) => {
const out = []
let dir = resolve(base)
let rel = ''
while (!existsSync(join(dir, '.git')) && dirname(dir) !== dir) {
dir = dirname(dir)
rel = rel ? `${rel}/..` : '..'
out.push(rel)
}
return existsSync(join(dir, '.git')) ? out : []
}
const where = (dirs, tail) => dirs.map((d) => `${d || '.'}/${tail}`).join(', ')
/**
* A tool installed with the project's JS packages. Looks in `from`, then every
* folder above it up to the repo root, because a monorepo installs prisma in
* the package that uses it, or hoists it to the top. `file` is relative to
* base. Where nothing is found it names the root's path and lists what was
* searched, so the error can say where the checker looked.
*/
function findNodeTool(base, from, bin, pkg) {
const looked = dirsUp(from)
if (!existsSync(join(base, 'node_modules'))) looked.push(...aboveBase(base))
for (const dir of looked) {
const file = at(dir, 'node_modules', '.bin', bin)
if (existsSync(join(base, file))) {
const modules = join(base, dir, 'node_modules')
return { file, looked, modules, version: packageVersion(modules, pkg) }
}
}
// No bin link. The package may still be there (a version for the cache key, a place for the engine check).
for (const dir of looked) {
const modules = join(base, dir, 'node_modules')
if (existsSync(join(modules, pkg, 'package.json'))) return { file: `node_modules/.bin/${bin}`, looked, modules, version: packageVersion(modules, pkg) }
}
return { file: `node_modules/.bin/${bin}`, looked, modules: join(base, from, 'node_modules'), version: null }
}
/** Alembic is a Python package: its version is the dist-info folder in the venv, so nothing runs to read it. */
const alembicVersion = (venv) => {
try {
const lib = join(venv, 'lib')
for (const py of readdirSync(lib)) {
for (const e of readdirSync(join(lib, py, 'site-packages'))) {
const m = /^alembic-(.+)\.dist-info$/.exec(e)
if (m) return m[1]
}
}
} catch {
// No venv, or a layout this does not know. The key then has no version.
}
return null
}
/** A .venv or venv in `from` or any folder above it that has alembic. Its bin goes first on PATH; none found leaves it to PATH. */
function findAlembic(base, from) {
const looked = dirsUp(from)
looked.push(...aboveBase(base))
for (const dir of looked) {
for (const name of ['.venv', 'venv']) {
if (existsSync(join(base, dir, name, 'bin', 'alembic'))) {
const venv = join(base, dir, name)
return { looked, pathPrefix: join(venv, 'bin'), version: alembicVersion(venv) }
}
}
}
return { looked, pathPrefix: null, version: null }
}
const readText = (file) => {
try {
return readFileSync(file, 'utf8')
} catch {
return ''
}
}
const dialectProblem = (index, what, name, fix) => {
const later = UNSUPPORTED[String(name).toLowerCase()]
if (later) {
return new CannotCheck('DIALECT-UNSUPPORTED', {
object: `sources[${index}]`,
message: `${what} is ${name}, which this version does not load (${later}).`,
fix: 'Nothing is checked for this database yet. The schema gate for it is on the roadmap.',
})
}
return new CannotCheck('DIALECT-UNKNOWN', {
object: `sources[${index}]`,
message: name ? `${what} is ${name}, which this version does not know.` : `the dialect could not be read from ${what}.`,
fix,
})
}
const SET_DIALECT = (index) => `Set dialect: 'postgres' or 'sqlite' on CONFIG.sources[${index}].`
function planPrisma(source, index, { base }, lenient) {
const schema = cleanPath(source.schema ?? 'prisma/schema.prisma')
const full = join(base, schema)
if (!existsSync(full)) return null
const folder = isDir(full)
let dialect = source.dialect === undefined ? null : normalDialect(source.dialect)
if (!dialect) {
const text = (folder ? watchedFiles(base, [schema]).filter((f) => f.path.endsWith('.prisma')).map((f) => f.bytes.toString('utf8')) : [readText(full)]).join('\n')
const provider = /\bdatasource\s+\w+\s*\{[^}]*?\bprovider\s*=\s*"([^"]+)"/.exec(text.replace(/\/\/.*$/gm, ''))?.[1]
const known = { postgresql: 'postgres', postgres: 'postgres', sqlite: 'sqlite' }
dialect = known[provider] ?? null
if (!dialect && !lenient) {
throw dialectProblem(index, `the datasource provider in ${schema}`, provider, `Set provider = "postgresql" or "sqlite" in the datasource block, or ${SET_DIALECT(index)}`)
}
}
// The project is the folder above `prisma/`: where its package.json,
// node_modules and prisma.config.ts live. Prisma runs from there.
const segments = (folder ? schema : dirOf(schema)).split('/').filter(Boolean)
const k = segments.lastIndexOf('prisma')
const project = k >= 0 ? segments.slice(0, k).join('/') : segments.join('/')
const inProject = (p) => (project && p.startsWith(`${project}/`) ? p.slice(project.length + 1) : p)
const tool = findNodeTool(base, project, 'prisma', 'prisma')
const major = Number.parseInt(tool.version ?? '', 10)
// Prisma 7 removed --to-schema-datamodel, and --to-schema takes a file or a folder since 5.x.
const flag = Number.isFinite(major) && major <= 6 ? '--to-schema-datamodel' : '--to-schema'
const configs = ['prisma.config.ts', 'prisma.config.mts', 'prisma.config.js', 'prisma.config.mjs']
return {
args: ['migrate', 'diff', '--from-empty', flag, inProject(schema), '--script'],
file: tool.file,
cwd: project,
dialect,
version: tool.version,
watch: [folder ? `${schema.replace(/\/+$/, '')}/` : schema, ...(project ? configs.map((c) => at(project, c)) : []), ...configs],
tool: 'prisma',
toolDirs: tool.looked,
modules: tool.modules,
project,
// migrate diff never connects, but Prisma 7 reads these when prisma.config.ts does. Set only when unset.
defaults: (() => {
const url = dialect === 'sqlite' ? 'file:./ntent.db' : 'postgresql://ntent:ntent@127.0.0.1:1/ntent'
return { DATABASE_URL: url, DIRECT_URL: url }
})(),
// No update check and no ping: this runs offline, in a hook.
force: { CHECKPOINT_DISABLE: '1', PRISMA_HIDE_UPDATE_MESSAGE: '1' },
}
}
/**
* Stand-ins for the variables a drizzle config reads, so it loads with no .env.
* drizzle-kit export never connects. These are never a real value: they replace
* what the caller has set, so a real URL in the environment cannot reach it.
*/
const DB_WORD = /DATABASE|POSTGRES|PG|DB|SQL|NEON|SUPABASE|TURSO|LIBSQL/
export function placeholderFor(name, dialect) {
if (!DB_WORD.test(name)) return null
if (/(^|_)(URL|URI|DSN)(_\w+)?$|CONNECTION(_STRING)?$/.test(name)) return dialect === 'sqlite' ? 'file:./ntent.db' : 'postgresql://ntent:ntent@127.0.0.1:1/ntent'
if (/_HOST(NAME)?$/.test(name)) return '127.0.0.1'
if (/_PORT$/.test(name)) return '5432'
if (/_(USER|USERNAME|PASSWORD|PASS|NAME|DB|DATABASE|TOKEN)$/.test(name)) return 'ntent'
return null
}
function planDrizzle(source, index, { base }, lenient) {
const config = cleanPath(source.config ?? 'drizzle.config.ts')
const full = join(base, config)
if (!existsSync(full)) return null
const text = readText(full).replace(/\/\/.*$/gm, '')
let dialect = source.dialect === undefined ? null : normalDialect(source.dialect)
if (!dialect) {
const name = /\bdialect\s*:\s*['"`](\w+)['"`]/.exec(text)?.[1]
dialect = { postgresql: 'postgres', sqlite: 'sqlite', turso: 'sqlite' }[name] ?? null
if (!dialect && !lenient) {
throw dialectProblem(index, `the dialect in ${config}`, name, `Write dialect: 'postgresql' or 'sqlite' as a string in ${config}, or ${SET_DIALECT(index)}`)
}
}
// drizzle-kit reads the schema paths relative to where it runs, so it runs in
// the config's own folder.
const project = dirOf(config)
// The schema path is a string or a list of strings. Read with a regex: the
// config is TypeScript and is not run here.
const schemaValue = /\bschema\s*:\s*(\[[^\]]*\]|['"`][^'"`]+['"`])/.exec(text)?.[1]
const schemas = schemaValue ? [...schemaValue.matchAll(/['"`]([^'"`]+)['"`]/g)].map((m) => at(project, cleanPath(m[1]))) : []
if (!schemas.length && !source.watch?.length && !lenient) {
throw new CannotCheck('CONFIG-INVALID', {
object: `sources[${index}].watch`,
message: `the schema path could not be read from ${config}, so the check does not know which files the cache depends on.`,
fix: `Set watch: ['src/db/'] (the files drizzle-kit reads) on CONFIG.sources[${index}].`,
})
}
const tool = findNodeTool(base, project, 'drizzle-kit', 'drizzle-kit')
// Every variable the config reads, so one that is not a URL (DB_HOST) is set too.
const names = new Set(['DATABASE_URL', 'POSTGRES_URL', 'DIRECT_URL', 'DB_URL'])
for (const m of text.matchAll(/\b(?:env|Bun\.env)\s*(?:\.\s*([A-Z][A-Z0-9_]*)|\[\s*['"`]([A-Z][A-Z0-9_]*)['"`]\s*\])|\benv\.get\(\s*['"`]([A-Z][A-Z0-9_]*)['"`]/g)) names.add(m[1] ?? m[2] ?? m[3])
const placeholders = {}
for (const name of names) {
const value = placeholderFor(name, dialect)
if (value) placeholders[name] = value
}
return {
args: ['export', '--config', config.slice(project.length ? project.length + 1 : 0)],
file: tool.file,
cwd: project,
dialect,
version: tool.version,
watch: [config, ...schemas],
tool: 'drizzle-kit',
toolDirs: tool.looked,
modules: tool.modules,
project,
force: placeholders,
}
}
/** Where an alembic.ini is, what runs from, and where its migrations are. script_location is relative to the ini's folder; %(here)s is that folder. */
function alembicLayout(base, source) {
const ini = cleanPath(source.config ?? 'alembic.ini')
const home = dirOf(ini)
const location = /^\s*script_location\s*=\s*(.+?)\s*$/m.exec(readText(join(base, ini)))?.[1]
const rel = cleanPath((location ?? 'alembic').replace(/^%\(here\)s\/?/, '')).replace(/\/+$/, '') || 'alembic'
return { ini, home, dir: at(home, rel) }
}
function planAlembic(source, index, { base }) {
const { ini, home, dir } = alembicLayout(base, source)
if (!existsSync(join(base, ini))) return null
const found = findAlembic(base, home)
const name = ini.slice(home.length ? home.length + 1 : 0)
return {
args: [...(name === 'alembic.ini' ? [] : ['-c', name]), 'upgrade', 'head', '--sql'],
file: 'alembic',
cwd: home,
dialect: normalDialect(source.dialect),
version: found.version,
watch: [`${dir}/`, ini],
tool: 'alembic',
toolDirs: found.looked,
project: home,
alembicDir: dir,
pathPrefix: found.pathPrefix,
}
}
function planRun(source) {
const [file, ...args] = source.run
return { args, file, dialect: normalDialect(source.dialect), version: null, watch: [], tool: file }
}
const PLANNERS = { prisma: planPrisma, drizzle: planDrizzle, alembic: planAlembic }
/**
* One command source, planned without running it: the command, the files it
* reads (and their bytes, for the cache key), its dialect and its tool's
* version. null when the file it starts from is not there, like a glob that
* matches nothing. lenient (for --print-config) never throws over a dialect.
*/
export function planCommand(source, index, { root, repoRoot = root, env = process.env }, { lenient = false } = {}) {
const mode = (source.cwd ?? (source.preset === 'alembic' ? 'root' : 'snapshot')) === 'root' ? 'root' : 'snapshot'
const base = mode === 'root' ? repoRoot : root
const ctx = { base, root, repoRoot }
const planned = source.preset ? PLANNERS[source.preset](source, index, ctx, lenient) : planRun(source)
if (!planned) return null
const watch = [...new Set([...planned.watch, ...(source.watch ?? []).map(cleanPath)].map((p) => (p.endsWith('/') || hasGlob(p) ? p : isDir(join(base, p)) ? `${p}/` : p)))]
const files = watchedFiles(base, watch)
// Nothing to read: the file the preset starts from is absent, or a generic run's watch matched nothing.
if (!files.length) return null
const label = `command:${source.preset ?? 'run'}`
const file = planned.file.includes('/') && !isAbsolute(planned.file) ? join(base, planned.file) : planned.file
const childEnv = { ...env }
for (const [k, v] of Object.entries(planned.defaults ?? {})) childEnv[k] = env[k] || v
Object.assign(childEnv, planned.force)
if (planned.pathPrefix) childEnv.PATH = `${planned.pathPrefix}${delimiter}${env.PATH ?? ''}`
return {
kind: 'command',
index,
label,
preset: source.preset ?? null,
mode,
base,
file,
cwd: planned.cwd ? join(base, planned.cwd) : base,
args: planned.args,
display: [planned.file, ...planned.args].join(' '),
env: childEnv,
// Where the tool was looked for, for the error that says it is missing.
project: planned.project ?? '',
toolDirs: planned.toolDirs ?? [],
modules: planned.modules ?? null,
alembicDir: planned.alembicDir ?? null,
dialect: planned.dialect,
tool: planned.tool,
version: planned.version,
watch: watch.map((p) => (hasGlob(p) ? watchPaths(base, p)[0] : p)),
files,
hash: sha256(canonical([label, planned.file, planned.args, planned.dialect, planned.version, mode, planned.cwd ?? '']), ...files.flatMap((f) => [f.path, '\0', sha256(f.bytes), '\0'])),
}
}
/**
* CONFIG.sources to what the run reads, in load order: SQL files and planned
* commands. One dialect for the lot, because one run builds one database.
*/
export function planSources(config, { root, repoRoot = root, env = process.env }, { lenient = false } = {}) {
const items = []
const seen = new Set()
for (const [index, source] of config.sources.entries()) {
if (source.kind === 'command') {
const unit = planCommand(source, index, { root, repoRoot, env }, { lenient })
if (unit) items.push(unit)
continue
}
for (const f of resolveInputs(root, [source])) {
if (seen.has(f.path)) continue
seen.add(f.path)
items.push({ kind: 'file', ...f, dialect: normalDialect(source.dialect ?? 'postgres') })
}
}
const dialects = [...new Set(items.map((i) => i.dialect).filter(Boolean))]
if (dialects.length > 1 && !lenient) {
throw new CannotCheck('CONFIG-INVALID', {
object: 'sources',
message: `mix ${dialects.join(' and ')}, and one run builds one database.`,
fix: 'Keep one dialect in CONFIG.sources.',
})
}
return { items, dialect: dialects[0] ?? 'postgres' }
}
/**
* What the hook should watch for a command source, whether or not its files
* exist yet: staging the first schema file must still run the check.
*/
export function commandWatch(source, { root, repoRoot = root }) {
const mode = (source.cwd ?? (source.preset === 'alembic' ? 'root' : 'snapshot')) === 'root' ? 'root' : 'snapshot'
const base = mode === 'root' ? repoRoot : root
const prismaConfigs = ['prisma.config.ts', 'prisma.config.mts', 'prisma.config.js', 'prisma.config.mjs']
const schema = cleanPath(source.schema ?? 'prisma/schema.prisma')
const segments = dirOf(schema).split('/').filter(Boolean)
const k = segments.lastIndexOf('prisma')
const prismaProject = k > 0 ? segments.slice(0, k).join('/') : ''
const alembic = alembicLayout(base, source)
const own = {
prisma: [schema, ...(prismaProject ? prismaConfigs.map((c) => at(prismaProject, c)) : []), ...prismaConfigs],
drizzle: [source.config ?? 'drizzle.config.ts'],
alembic: [`${alembic.dir}/`, alembic.ini],
}[source.preset] ?? []
return [...own, ...(source.watch ?? [])].map(cleanPath).flatMap((p) => (hasGlob(p) ? watchPaths(base, p) : [p.endsWith('/') || !isDir(join(base, p)) ? p : `${p}/`]))
}
/** Command sources that read the worktree report a file that differs from the index (R B2). */
export function unstagedNotes(items) {
const notes = []
for (const unit of items) {
if (unit.kind !== 'command' || unit.mode !== 'root' || !unit.watch.length) continue
const r = spawnSync('git', ['diff', '--name-only', '--', ...unit.watch.map((w) => w.replace(/\/$/, ''))], {
cwd: unit.base,
encoding: 'utf8',
stdio: ['ignore', 'pipe', 'ignore'],
timeout: 10000,
})
const changed = r.status === 0 ? String(r.stdout).split('\n').filter(Boolean) : []
if (changed.length) {
notes.push({
code: 'UNSTAGED-INPUT',
object: unit.label,
message: `note: ${changed.join(', ')} has unstaged changes; this check read the worktree, not the commit.`,
})
}
}
return notes
}
/**
* Sources listed one file at a time load only those files. A migration that
* lands beside them is never read, and the gate stays green without it (R33).
* Names that look like migrations (a number or V<number> first) are reported;
* a helper or a seed beside them is not.
*/
export function unlistedMigrationNotes(root, sources, items) {
const loaded = new Set(items.filter((i) => i.kind === 'file').map((i) => i.path))
const folders = new Set()
for (const src of sources) {
if (src.kind !== 'sql') continue
for (const p of src.paths ?? []) {
const rel = posix(p)
if (rel.endsWith('.sql') && !rel.includes('*') && loaded.has(rel)) folders.add(rel.includes('/') ? rel.slice(0, rel.lastIndexOf('/')) : '')
}
}
const notes = []
for (const folder of folders) {
let names = []
try {
names = readdirSync(join(root, folder)).sort()
} catch {
continue
}
const missed = names
.filter((n) => /^[vV]?\d.*\.sql$/.test(n) && !/(?:^|[._-])(?:down|rollback)\.sql$/i.test(n))
.map((n) => (folder ? `${folder}/${n}` : n))
.filter((n) => !loaded.has(n))
if (!missed.length) continue
const shown = missed.length > 5 ? `${missed.slice(0, 5).join(', ')} and ${missed.length - 5} more` : missed.join(', ')
notes.push({
code: 'UNLISTED-MIGRATION',
object: folder || '.',
// Printed as it is on every run, so it carries its own fix.
message:
`note: ${shown} ${missed.length === 1 ? 'sits' : 'sit'} beside migrations that CONFIG.sources lists one file at a time, and ${missed.length === 1 ? 'is' : 'are'} not loaded. ` +
`If ${missed.length === 1 ? 'it is a migration' : 'they are migrations'}, add ${missed.length === 1 ? 'it' : 'them'} to CONFIG.sources in order, or list the folder as one glob (${folder ? `${folder}/` : ''}*.sql) so the next one is read too.`,
})
}
return notes
}
/** Where a missing tool was looked for, and what to do about it. Names the folders so a monorepo owner can see which one was meant. */
function notInstalledFix(unit) {
const home = unit.project || 'the repo root'
if (unit.tool === 'alembic') {
return `Looked for a .venv or venv with alembic in ${where(unit.toolDirs, '{.venv,venv}/bin')}, then on PATH. Install it in the folder that owns alembic.ini (${home}): python -m venv .venv && .venv/bin/pip install alembic, or put alembic on PATH.`
}
if (unit.preset === 'prisma' || unit.preset === 'drizzle') {
return `Looked for node_modules/.bin/${unit.tool} in ${where(unit.toolDirs, 'node_modules/.bin')}. Add ${unit.tool} as a devDependency of the package that owns the schema (${home}) and install.`
}
return `Install ${unit.tool}, or correct CONFIG.sources[${unit.index}].run.`
}
/**
* Which JS package manager installed this repo, from the project folder up to
* the root: the packageManager field, the lockfile, then (for pnpm) the settings
* file that holds its build allowlist. pnpm 11 reads allowBuilds from
* pnpm-workspace.yaml and ignores pnpm.onlyBuiltDependencies, which pnpm 10 reads.
*/
export function packageManagerOf(base, from = '') {
let manager = null
let major = null
let style = null
for (const dir of dirsUp(from)) {
const pkg = (() => {
try {
return JSON.parse(readFileSync(join(base, dir, 'package.json'), 'utf8'))
} catch {
return {}
}
})()
const field = /^(pnpm|npm|yarn|bun)@(\d+)/.exec(String(pkg.packageManager ?? ''))
if (field && !manager) [manager, major] = [field[1], Number(field[2])]
if (!manager) {
const locks = { 'pnpm-lock.yaml': 'pnpm', 'package-lock.json': 'npm', 'yarn.lock': 'yarn', 'bun.lock': 'bun', 'bun.lockb': 'bun' }
for (const [file, name] of Object.entries(locks)) if (existsSync(join(base, dir, file))) manager = manager ?? name
}
if (/\ballowBuilds\s*:/.test(readText(join(base, dir, 'pnpm-workspace.yaml')))) style ??= 'allowBuilds'
if (pkg.pnpm?.onlyBuiltDependencies) style ??= 'onlyBuiltDependencies'
}
if (manager === 'pnpm' && major === null && style === null) {
// Nothing in the repo says which pnpm. Ask the one on PATH, once, only on this failure path.
const r = spawnSync('pnpm', ['--version'], { encoding: 'utf8', timeout: 5000, stdio: ['ignore', 'pipe', 'ignore'] })
const m = r.status === 0 ? /^(\d+)/.exec(String(r.stdout).trim()) : null
if (m) major = Number(m[1])
}
return { manager, major, style }
}
/** The fix for PRISMA-ENGINE-MISSING: the engine is a postinstall download, and each package manager blocks or allows that in its own way. */
export function engineFix(base, from = '') {
const { manager, major, style } = packageManagerOf(base, from)
const pnpm11 = "In pnpm-workspace.yaml add allowBuilds: { '@prisma/engines': true, prisma: true }, then pnpm install. If pnpm says 'Already up to date', run pnpm rebuild @prisma/engines prisma, or delete node_modules and install again."
const pnpm10 = 'Add @prisma/engines and prisma to pnpm.onlyBuiltDependencies in package.json, then pnpm install. If pnpm says \'Already up to date\', run pnpm rebuild @prisma/engines prisma.'
if (manager === 'pnpm') {
// The repo's own settings say more than the pnpm on PATH, which may not be the one it uses.
if (style === 'allowBuilds' || (style === null && major !== null && major >= 11)) return `This repo uses pnpm ${major ?? '11'}. ${pnpm11}`
if (style === 'onlyBuiltDependencies' || (style === null && major !== null && major >= 10)) return `This repo uses pnpm ${major ?? '10'}. ${pnpm10}`
return `pnpm 9 and earlier run the download at install: pnpm rebuild @prisma/engines prisma. pnpm 10: ${pnpm10} pnpm 11: ${pnpm11}`
}
if (manager === 'npm') return 'This repo uses npm. The download runs at install, so it was skipped: run npm rebuild @prisma/engines prisma, and remove ignore-scripts=true from .npmrc if it is set.'
if (manager === 'yarn') return 'This repo uses yarn. Run yarn install again without --ignore-scripts, or run npx prisma --version once online.'
if (manager === 'bun') return 'This repo uses bun. Run bun pm trust @prisma/engines prisma, then bun install.'
return `Allow the engine download, once, online. pnpm 11: add allowBuilds: { '@prisma/engines': true, prisma: true } to pnpm-workspace.yaml. pnpm 10: add @prisma/engines to pnpm.onlyBuiltDependencies. npm: npm rebuild @prisma/engines prisma. Then install again.`
}
/** The Prisma schema engine is a download that pnpm 10 skips at install (review 2, M11). */
function prismaEngineFound(unit) {
const explicit = unit.env.PRISMA_SCHEMA_ENGINE_BINARY
if (explicit) return existsSync(explicit)
const modules = unit.modules ?? join(unit.base, 'node_modules')
const dirs = [join(modules, '@prisma', 'engines')]
try {
// pnpm keeps @prisma/engines next to prisma in its store, not at the top of node_modules.
dirs.push(join(dirname(realpathSync(join(modules, 'prisma'))), '@prisma', 'engines'))
} catch {
// prisma is not installed; the not-installed error comes first.
}
return dirs.some((dir) => {
try {
return readdirSync(dir).some((e) => e.startsWith('schema-engine'))
} catch {
return false
}
})
}
const lastLines = (text, n = 20) => text.split('\n').filter((l) => l.trim()).slice(-n).join('\n')
/**
* What alembic's offline mode cannot do, read from the migrations. Offline there
* is no database: op.get_bind() is a stand-in whose execute() returns None, and
* a bound parameter prints as NULL. Each hit is { file, line, text, binds } with
* binds true when the statement passes parameters.
*/
export function alembicBindUses(dir) {
const hits = []
const walk = (d, depth) => {
let entries
try {
entries = readdirSync(d, { withFileTypes: true })
} catch {
return
}
for (const e of entries) {
if (e.name.startsWith('.') || e.name === '__pycache__') continue
if (e.isDirectory() && depth < 3) walk(join(d, e.name), depth + 1)
else if (e.isFile() && e.name.endsWith('.py') && hits.length < 50) scan(join(d, e.name))
}
}
const scan = (file) => {
const lines = readText(file).split('\n')
// Names bound to the connection: conn = op.get_bind().
const names = new Set(['bind', 'conn', 'connection'])
for (const l of lines) {
const m = /^\s*(\w+)\s*=\s*(?:op\.)?get_bind\(\)/.exec(l)
if (m) names.add(m[1])
}
const uses = new RegExp(`\\b(?:(?:op\\.)?get_bind\\(\\)|${[...names].join('|')})\\s*\\.\\s*execute\\s*\\(`)
for (const [i, l] of lines.entries()) {
if (/^\s*#/.test(l) || !uses.test(l)) continue
// The call may span lines: read to its closing line for a second argument or .bindparams.
const call = lines.slice(i, i + 8).join(' ')
const binds = /\.execute\s*\([^()]*(?:\([^()]*\)[^()]*)*,\s*[{\w*]/.test(call) || /\bbindparams?\s*\(/.test(call) || /(?<![:\w]):[A-Za-z_]\w*/.test(call)
hits.push({ file, line: i + 1, text: l.trim().slice(0, 100), binds })
}
for (const [i, l] of lines.entries()) {
if (/^\s*#/.test(l) || !/\bop\.execute\s*\(/.test(l)) continue
const call = lines.slice(i, i + 8).join(' ')
if (/\bbindparams?\s*\(/.test(call)) hits.push({ file, line: i + 1, text: l.trim().slice(0, 100), binds: true })
}
}
walk(dir, 0)
return hits
}
/** The files alembicBindUses found, named relative to base, as one sentence. */
const bindSentence = (hits, base) => hits.slice(0, 5).map((h) => `${posix(relative(base, h.file))}:${h.line}`).join(', ')
/**
* App logging that reaches stdout lands in the SQL: a banner before the first
* statement, or a log line between two. Keep only SQL. Everything before the
* first line that starts a statement or a comment is dropped, and after that a
* line that is plainly a log record (a timestamp, a level, a JSON object) is
* dropped unless it sits inside a $$ body, where it could be the function's.
* Returns the SQL and how many lines it dropped.
*/
export function stripNonSql(text) {
const START = /^\s*(?:--|\/\*|(?:begin|commit|start|set|reset|create|alter|drop|insert|update|delete|grant|revoke|comment|do|with|select|truncate|call|execute|copy|lock|notify|refresh|reindex|cluster|vacuum|analyze|savepoint|release|rollback|end|values|table|declare|prepare|listen|import|security|explain)\b)/i
const LOG = /^\s*(?:\d{4}-\d\d-\d\d[T ]\d\d:\d\d|\[?\s*(?:debug|info|warn(?:ing)?|error|critical|trace)\s*\]?[\s:\]]|\{\s*"|\{'|\x1b\[)/i
const lines = text.split('\n')
const out = []
let started = false
let dropped = 0
let dollar = false
for (const line of lines) {
if (!started) {
if (START.test(line) || !line.trim()) {
if (line.trim()) started = true
else continue
} else {
dropped++
continue
}
}
if (!dollar && LOG.test(line) && !START.test(line)) {
dropped++
continue
}
// A $$ or $tag$ quote opens and the next one closes: an odd count flips.
const quotes = line.match(/\$[A-Za-z_]*\$/g)
if (quotes && quotes.length % 2 === 1) dollar = !dollar
out.push(line)
}
return { sql: out.join('\n'), dropped }
}
/** Run a planned command and return the SQL it printed, or refuse with the fix. */
export function runCommand(unit, { timeoutMs = 60000 } = {}) {
const notInstalled = () =>
new CannotCheck('COMMAND-FAILED', {
object: unit.display,
message: `${unit.tool} is not installed.`,
fix: notInstalledFix(unit),
})
if (unit.preset === 'prisma') {
if (!existsSync(unit.file)) throw notInstalled()
if (!prismaEngineFound(unit)) {
throw new CannotCheck('PRISMA-ENGINE-MISSING', {
object: `${posix(relative(unit.base, unit.modules ?? join(unit.base, 'node_modules')))}/@prisma/engines`,
message: "Prisma's schema engine is not installed, so prisma migrate diff would have to download it.",
fix: engineFix(unit.base, unit.project),
})
}
}
const r = spawnSync(unit.file, unit.args, {
cwd: unit.cwd ?? unit.base,
env: unit.env,
encoding: 'utf8',
timeout: timeoutMs,
maxBuffer: 64 * 1024 * 1024,
stdio: ['ignore', 'pipe', 'pipe'],
})
if (r.error?.code === 'ENOENT') throw notInstalled()
if (r.error?.code === 'ETIMEDOUT') {
throw new CannotCheck('COMMAND-FAILED', {
object: unit.display,
message: `ran past ${timeoutMs} ms and was stopped.`,
fix: 'Raise CONFIG.commandTimeoutMs. Run the command by hand to see whether it hangs.',
})
}
if (r.error) {
throw new CannotCheck('COMMAND-FAILED', { object: unit.display, message: `could not run: ${r.error.message}`, fix: 'Run the command by hand to see why.' })
}
const stderr = String(r.stderr ?? '')
if (r.status === 127) {
if (unit.preset || !/^(?:.*\/)?(?:ba|z|da|k)?sh$/.test(unit.tool)) throw notInstalled()
// A run source goes through sh or the tool itself, so the missing command is named in stderr.
const name = /^(?:\S*sh: (?:\d+: )?)?([^\s:]+): (?:command )?not found\s*$/m.exec(stderr)?.[1]
throw new CannotCheck('COMMAND-FAILED', {
object: unit.display,
message: name ? `${name} is not installed.` : `the command exited 127 (command not found).${stderr.trim() ? `\n${lastLines(stderr)}` : ''}`,
fix: name ? `Install ${name}, or put it on the PATH the check runs with. Run the command by hand to confirm.` : 'Run the command by hand to see which program is missing.',
})
}
const alembic = unit.preset === 'alembic' ? alembicBindUses(join(unit.base, unit.alembicDir ?? 'alembic')) : []
if (r.status !== 0) {
// get_bind() in offline mode fails in one of two shapes: the word itself in a
// traceback, or execute() returning None and the next call on it raising.
if (unit.preset === 'alembic' && (/get_bind/.test(stderr) || /AttributeError: 'NoneType' object has no attribute/.test(stderr))) {
const uses = alembic.length ? ` Found in ${bindSentence(alembic, unit.base)}.` : ''
throw new CannotCheck('COMMAND-FAILED', {
object: unit.display,
message: `exited ${r.status}.\n${lastLines(stderr)}`,
fix: `A migration calls op.get_bind(), which offline mode cannot run: execute() returns None there.${uses} Point sources at SQL files instead, or use a run source that prints the schema another way.`,
})
}
throw new CannotCheck('COMMAND-FAILED', {
object: unit.display,
message: `exited ${r.status ?? r.signal}.\n${lastLines(stderr || String(r.stdout ?? ''))}`,
fix: 'Run the command by hand. It needs to print SQL without a database.',
})
}
let sql = String(r.stdout ?? '')
// App logging on stdout is not SQL (alembic). Only SQL reaches the loader.
if (unit.preset === 'alembic') sql = stripNonSql(sql).sql
// Comments alone load nothing: Prisma prints "-- This is an empty migration." for an empty schema.
if (!stripComments(sql)) {
if (unit.preset === 'prisma') {
throw new CannotCheck('PRISMA-EMPTY', {
object: unit.display,
message: 'printed no SQL.',
fix: 'prisma printed no SQL. Prisma 7 needs datasource.url in prisma.config.ts even to print SQL: set it to process.env.DATABASE_URL.',
})
}
throw new CannotCheck('COMMAND-EMPTY', {
object: unit.display,
message: 'printed no SQL.',
fix: `Run ${unit.display} by hand. If it prints SQL there, it needs something this run lacks: an environment variable or a config file.`,
})
}
// Offline, a bound parameter prints as NULL, so the SQL is not what the
// migration does. Both signs are needed: a migration that binds, and a NULL
// where a value was meant. A migration that binds and loses nothing passes.
const binding = alembic.filter((h) => h.binds)
if (binding.length && /\binsert\s+into\b[^;]*\bvalues\s*\([^;]*\bnull\b|\bupdate\b[^;]*\bset\b[^;]*=\s*null\b/i.test(sql)) {
throw new CannotCheck('COMMAND-FAILED', {
object: unit.display,
message: `alembic printed NULL where a migration binds a parameter (${bindSentence(binding, unit.base)}). Offline mode has no database to bind to, so the SQL is not what the migration does.`,
fix: `Pass literals instead: sa.text(...).bindparams(code='x') with context.configure(literal_binds=True) in env.py, or write the value into the statement. Or print the schema another way with a run source: { kind: 'command', run: ['.venv/bin/python', 'scripts/print_schema.py'], dialect: 'postgres', watch: ['${unit.alembicDir ?? 'alembic'}/'] }.`,
})
}
return sql
}
// -------------------------------------------------------------- DISCOVERY ---
// Finding the schema and the platform from the repo, for a first run and for
// --print-config. Nothing here picks for the caller: a source decides what the
// gate checks, and a pick that moved when someone added a folder would move the
// gate. It prints the line to paste, ranked, and CONFIG stays the one place that
// says what is read.
// Folders that hold dependencies or build output, never a schema. Dot folders
// are skipped too, and so is any folder with a pyvenv.cfg (a Python venv).
const SKIP_DIRS = new Set(['node_modules', 'dist', 'build', 'out', 'vendor', 'target', 'coverage', '__pycache__', 'venv', 'site-packages', 'bower_components', 'Pods'])
/** Every folder under root to maxDepth, with the names of its files. Capped, so a huge repo costs a bounded walk. */
export function scanRepo(root, { maxDepth = 8, maxEntries = 30000 } = {}) {
const dirs = new Map()
let seen = 0
let truncated = false
const walk = (rel, depth) => {
let entries
try {
entries = readdirSync(join(root, rel), { withFileTypes: true })
} catch {
return
}
if (entries.some((e) => e.name === 'pyvenv.cfg')) return
const files = []
const subdirs = []
for (const e of entries) {
if (++seen > maxEntries) {
truncated = true
return
}
if (e.isDirectory()) {
if (!e.name.startsWith('.') && !SKIP_DIRS.has(e.name) && depth < maxDepth) subdirs.push(e.name)
} else if (e.isFile()) files.push(e.name)
}
dirs.set(rel, files)
for (const d of subdirs.sort()) walk(rel ? `${rel}/${d}` : d, depth + 1)
}
walk('', 0)
return { dirs, truncated }
}
/** A value as the one-line JavaScript CONFIG is written in: single quotes, no semicolons. */
const js = (v) =>
Array.isArray(v)
? `[${v.map(js).join(', ')}]`
: isObject(v)
? `{ ${Object.entries(v).map(([k, x]) => `${k}: ${js(x)}`).join(', ')} }`
: typeof v === 'string'
? `'${v.replace(/\\/g, '\\\\').replace(/'/g, "\\'")}'`
: String(v)
/** The exact line to paste into CONFIG. */
export const sourcesLine = (sources) => `sources: ${js(sources)},`
// A folder that is probably not the live schema: an archive, a fixture, a seed.
const NOT_LIVE = /(^|\/)(_?archive[sd]?|old|backups?|bak|legacy|deprecated|examples?|samples?|templates?|demos?|tests?|__tests__|e2e|fixtures?|seeds?|docs?|rollbacks?|downs?)(\/|$)/i
const BASE_FILE = /(^|[_-])(schema|init|initial|base|dump|setup|database|structure)([_.-]|\.sql$)/i
// SQLite and D1 SQL, and what Drizzle prints for them: backtick names. Not its
// statement-breakpoint comment: drizzle-kit writes that for every dialect.
const SQLITE_TELL = /\b(?:PRAGMA\b|AUTOINCREMENT\b|WITHOUT\s+ROWID\b|sqlite_\w+|unixepoch\s*\(|strftime\s*\(|INSERT\s+OR\s+(?:IGNORE|REPLACE)\b)|`\w+`/i
// Only what SQLite cannot run. A column called uuid, or a RETURNING clause, proves nothing.
const POSTGRES_TELL = /\b(?:jsonb|timestamptz|serial|bigserial|gen_random_uuid|plpgsql|create\s+policy|create\s+extension|create\s+or\s+replace\s+function|enable\s+row\s+level)\b|\$\$|::\w/i
// A desktop or mobile app's own database, not the server's.
const DESKTOP_DIR = /(^|[\/-])(tauri|desktop|electron|mobile|ios|android)(\/|$)/i
const SERVER_DIR = /(^|[\/-])(server|backend|api)(\/|$)/i
/** The common prefix of some names. */
const commonPrefix = (names) => names.reduce((a, b) => { let i = 0; while (i < a.length && a[i] === b[i]) i++; return a.slice(0, i) }, names[0] ?? '')
/** The dialect a drizzle-kit journal names: 'postgres', 'sqlite' (also turso), or null when absent or another one. */
function journalDialect(file) {
try {
const d = JSON.parse(readFileSync(file, 'utf8')).dialect
return d === 'postgresql' ? 'postgres' : d === 'sqlite' || d === 'turso' ? 'sqlite' : null
} catch {
return null
}
}
// A script somebody runs by hand with their own values filled in, not a migration: it raises when it still holds a placeholder.
const PLACEHOLDER = /raise\s+(?:exception|'|notice)[^;]*(?:\w_HERE\b|<[a-z_ -]+>|\bplaceholder\b|\bYOUR_\w+|\bREPLACE\b|\bCHANGE_?ME\b)/i
const MANUAL_NAME = /(^|[._-])(helper|manual|adhoc|ad-hoc|one-?off|run-?once|scratch)([._-]|$)/i
const isManualScript = (name, file) => MANUAL_NAME.test(name) || PLACEHOLDER.test(readText(file))
/** One folder of SQL files, as a candidate source. null when it holds no up migration. */
function sqlCandidate(root, dir, allNames) {
// Only .sql files: a folder with a README and a Dockerfile beside its schema is not 12 migrations.
const names = allNames.filter((n) => n.endsWith('.sql'))
const isDown = (n) => /(?:^|[._-])(?:down|rollback)\.sql$/i.test(n) || /(?:^|[._-])down[._-]/i.test(n)
const up = names.filter((n) => n.endsWith('.up.sql'))
const sql = (up.length ? up : names.filter((n) => !isDown(n))).sort(byName)
if (!sql.length) return null
const at0 = (n) => (dir ? `${dir}/${n}` : n)
// The first three and the last two: a base schema is often first, and the newest file is the one in the tool's style.
const head = [...new Set([...sql.slice(0, 3), ...sql.slice(-2)])].map((n) => readFileText(join(root, at0(n)))).join('\n')
const sections = /^--\s*migrate:up\b/m.test(head) ? 'dbmate' : /^--\s*\+goose\s+Up\b/im.test(head) ? 'goose' : null
// drizzle-kit names the dialect in meta/_journal.json, which beats any guess from the SQL.
const journal = journalDialect(join(root, at0('meta/_journal.json')))
const hard = POSTGRES_TELL.test(head)
const guessed = !hard && (SQLITE_TELL.test(head) || /(^|[\/-])(d1|sqlite|turso|tauri)(\/|$)/i.test(dir))
const dialect = (journal ? journal === 'sqlite' : guessed) ? 'sqlite' : null
const withDialect = (src) => (dialect ? { ...src, dialect } : src)
// A manual helper is never a base schema, and is left out of the list a glob would have loaded it with.
const manual = sql.filter((n) => isManualScript(n, join(root, at0(n))))
const base = sql.filter((n) => BASE_FILE.test(n) && !/^[vV]?\d/.test(n) && !manual.includes(n))
const rest = sql.filter((n) => !base.includes(n) && !manual.includes(n))
// A base schema that would not load first: it sorts after its patches. Say so
// in the order, one source with 'listed', rather than three entries.
if (base.length && rest.length && !base.includes([...sql].sort(byName)[0])) {
// 'migration_add_b' and 'migration_add_c' share 'migration_': cut at the first separator, so the glob reads as a name.
const shared = commonPrefix(rest)
const prefix = /^[^_.-]*[_-]/.exec(shared)?.[0] ?? shared
const glob = prefix.length >= 3 && sql.filter((n) => n.startsWith(prefix) && n.endsWith('.sql')).length === rest.length ? at0(`${prefix}*.sql`) : null
const paths = [...base.map(at0), ...(glob ? [glob] : rest.map(at0))]
return { sources: [withDialect({ kind: 'sql', paths, order: 'listed', sections })], files: sql.length, listed: true, dialect }
}
if (manual.length) {
const paths = sql.filter((n) => !manual.includes(n)).map(at0)
if (!paths.length) return null
return { sources: [withDialect({ kind: 'sql', paths, order: 'name', sections })], files: paths.length, listed: false, dialect, skipped: manual }
}
// A folder named for migrations gets a glob even with one file in it: the next one lands there.
const pattern = up.length ? at0('*.up.sql') : sql.length === 1 && !/^(migrations?|migrate|db)$/.test(dir.split('/').at(-1)) ? at0(sql[0]) : at0('*.sql')
const sorted = [...sql].sort(byName)
const natural = [...sql].sort(byNatural)
const order = sorted.some((n, i) => n !== natural[i]) ? 'natural' : 'name'
return { sources: [withDialect({ kind: 'sql', paths: [pattern], order, sections })], files: sql.length, listed: false, dialect }
}
const readFileText = (file) => {
try {
return readFileSync(file, 'utf8').slice(0, 4000)
} catch {
return ''
}
}
/**
* Where the schema might be, best first. Each is { rank, kind, path, note,
* sources }: sources is what CONFIG.sources would hold. Command presets rank
* above SQL because they are the declared source of truth; SQL in a folder
* called supabase/migrations ranks above all, because the shipped default
* already reads it. Archives, fixtures and seeds rank last.
*/
export function discoverSources(root, scan = scanRepo(root)) {
const found = []
const add = (c) => found.push({ lowPriority: NOT_LIVE.test(c.path), ...c })
const depth = (p) => (p ? p.split('/').length : 0)
const prismaDirs = new Set()
for (const [dir, names] of scan.dirs) {
const at0 = (n) => (dir ? `${dir}/${n}` : n)
// Prisma: schema.prisma, or a folder of .prisma files with a datasource in one.
const prismas = names.filter((n) => n.endsWith('.prisma'))
if (prismas.length) {
const withSource = prismas.filter((n) => /\bdatasource\s+\w+\s*\{/.test(readText(join(root, at0(n)))))
if (prismas.includes('schema.prisma') || withSource.length) {
const multi = prismas.length > 1 && !prismas.includes('schema.prisma')
const schema = multi ? dir : at0(prismas.includes('schema.prisma') ? 'schema.prisma' : withSource[0])
prismaDirs.add(dir)
add({ rank: 90 - depth(dir), kind: 'prisma', path: schema, note: `Prisma schema${multi ? ' folder' : ''}`, sources: [{ kind: 'command', preset: 'prisma', schema }] })
}
}
// drizzle.config.ts, or a named one (drizzle.db.config.ts, drizzle-do.config.ts): one candidate each, the default name first.
const isDefault = (x) => /^drizzle\.config\./.test(x)
const drizzles = names.filter((x) => /^drizzle(?:[.-][\w-]+)?\.config\.(?:ts|js|mjs|cjs|mts|cts)$/.test(x)).sort((a, b) => Number(isDefault(b)) - Number(isDefault(a)) || byName(a, b))
for (const [i, n] of drizzles.entries()) {
add({ rank: 90 - depth(dir) - i * 0.1, kind: 'drizzle', path: at0(n), note: 'drizzle config', sources: [{ kind: 'command', preset: 'drizzle', config: at0(n) }] })
}
if (names.includes('alembic.ini')) {
const url = /^\s*sqlalchemy\.url\s*=\s*(\w+)/m.exec(readText(join(root, at0('alembic.ini'))))?.[1] ?? ''
const dialect = /^sqlite/i.test(url) ? 'sqlite' : 'postgres'
add({
rank: 90 - depth(dir),
kind: 'alembic',
path: at0('alembic.ini'),
note: `alembic.ini${url ? '' : ', dialect guessed: check it'}`,
sources: [{ kind: 'command', preset: 'alembic', ...(dir ? { config: at0('alembic.ini') } : {}), dialect }],
})
}
}
// Prisma's own migration folders: <parent>/*/migration.sql.
const migrationParents = new Map()
for (const [dir, names] of scan.dirs) {
// Prisma's folder holds the one migration.sql. A folder with other SQL beside it is a plain SQL folder.
if (names.includes('migration.sql') && dir.includes('/') && names.filter((n) => n.endsWith('.sql')).length === 1) {
const parent = dir.slice(0, dir.lastIndexOf('/'))
migrationParents.set(parent, (migrationParents.get(parent) ?? 0) + 1)
}
}
for (const [parent, count] of migrationParents) {
add({
rank: 60 - depth(parent) * 1.5 + Math.min(count, 20) * 0.5,
kind: 'sql',
path: parent,
note: `${count} migration folder${count === 1 ? '' : 's'}. Reads committed SQL, so no tool needs installing`,
sources: [{ kind: 'sql', paths: [`${parent}/*/migration.sql`], order: 'name', sections: null }],
})
}
for (const [dir, names] of scan.dirs) {
if (!names.some((n) => n.endsWith('.sql'))) continue
if (names.includes('migration.sql') && names.filter((n) => n.endsWith('.sql')).length === 1 && migrationParents.has(dir.slice(0, Math.max(dir.lastIndexOf('/'), 0)))) continue
const c = sqlCandidate(root, dir, names)
if (!c) continue
const last = dir.split('/').at(-1)
const supabase = /(^|\/)supabase\/migrations$/.test(dir)
const named = ['migrations', 'migrate', 'db'].includes(last)
const baseRank = supabase ? 100 : named ? 70 : c.listed || names.some((n) => BASE_FILE.test(n)) ? 55 : 45
// A desktop app's local database ranks below the server's; a server folder ranks a little above.
const desktop = DESKTOP_DIR.test(dir)
const rank = baseRank - depth(dir) * 1.5 + Math.min(c.files, 20) * 0.5 - (desktop ? 20 : 0) + (SERVER_DIR.test(dir) && !desktop ? 6 : 0)
const note = `${c.files} SQL file${c.files === 1 ? '' : 's'}${c.dialect ? `, ${c.dialect}` : ''}${c.listed ? ', a base schema first, then patches' : ''}${c.skipped ? `, ${c.skipped.join(', ')} left out: a script to run by hand, not a migration` : ''}${desktop ? ", a desktop or mobile app's own database" : ''}`
add({ rank, kind: supabase ? 'supabase' : 'sql', path: dir || '.', note, sources: c.sources })
// Patches with no base schema beside them, and one above: the base schema loads first, then the patches.
if (named && !c.listed && !names.some((n) => n.endsWith('.sql') && BASE_FILE.test(n) && !/^[vV]?\d/.test(n))) {
const parts = dir.split('/')
for (let i = parts.length - 1; i >= 0; i--) {
const up = parts.slice(0, i).join('/')
const base = (scan.dirs.get(up) ?? []).filter((n) => n.endsWith('.sql') && BASE_FILE.test(n) && !/^[vV]?\d/.test(n)).sort(byName)
if (!base.length) continue
const [patches] = c.sources[0].paths
const src = { ...c.sources[0], paths: [...base.map((n) => (up ? `${up}/${n}` : n)), patches], order: 'listed' }
add({ rank: rank + 1, kind: 'sql', path: dir, note: `${base.join(', ')} in ${up || 'the repo root'} first, then the ${c.files} patch${c.files === 1 ? '' : 'es'} here`, sources: [src] })
break
}
}
}
return found
.map((c) => ({ ...c, rank: c.lowPriority ? c.rank - 40 : c.rank }))
.sort((a, b) => b.rank - a.rank || a.path.length - b.path.length || byName(a.path, b.path))
.slice(0, 12)
.map((c, i) => ({ ...c, n: i + 1, line: sourcesLine(c.sources) }))
}
/** Candidates as text for NO-INPUT: what was found, best first, and the line to paste for each. */
export function candidatesText(candidates, limit = 5, heading = 'Found in this repo, best first. Paste the line you want over `sources:` in CONFIG:') {
const shown = candidates.slice(0, limit)
const lines = [heading]
for (const c of shown) lines.push(` ${c.n}. ${c.path} (${c.note}${c.lowPriority ? '; looks like an archive, a fixture or a seed' : ''})`, ` ${c.line}`)
if (candidates.length > shown.length) lines.push(` ...and ${candidates.length - shown.length} more. node scripts/check-backend.mjs --print-config lists them all.`)
if (shown.filter((c) => !c.lowPriority).length > 1) {
lines.push('More than one place holds SQL. This check reads one database per run: pick the live one, or list several in order with order: \'listed\'. It does not choose for you.')
}
return lines.join('\n')
}
// Signals that a repo is on Supabase, each as [what it is, how it is found].
const SUPABASE_DEP = /["']@supabase\/(?:supabase-js|ssr|auth-helpers[\w-]*|auth-ui[\w-]*)["']/
const SUPABASE_PY = /^\s*["']?supabase\b/im
const ENV_EXAMPLE = /^(?:\.env(?:\.[\w-]+)?\.(?:example|sample|template|dist)|env\.(?:example|sample)|example\.env|sample\.env)$/i
const PROVIDER_DIR = /(?:^|[-_.])(?:auth|storage|provider|adapter|plugin|sso|oauth)s?[-_.]supabase|supabase[-_.](?:auth|storage|provider|adapter|plugin|sso|oauth)/i
const SUPABASE_URL = /supabase\.(?:co|in)\b|\bSUPABASE_(?:URL|ANON_KEY|SERVICE_ROLE_KEY)\b/
const SUPABASE_SQL = /\bauth\.(?:uid|jwt|role)\s*\(|\bauth\.users\b|\bto\s+(?:authenticated|anon)\b|\bservice_role\b/i
const NO_RLS_HINT =
"no migration enables row level security or creates a policy. If access is enforced in app code, set enforcement: 'app' in CONFIG, and read what that leaves unchecked in its comment."
const ANY_RLS = /\benable\s+row\s+level\s+security\b|\bcreate\s+policy\b/i
/** True when no table in the loaded schema has row level security on or a policy: access is left to app code. */
export const snapshotNoRls = (snapshot) => !Object.values(snapshot?.tables ?? {}).some((t) => !t.platform && (t.rls?.enabled || t.policies?.length))
/** The one-line summary text for access that was not checked, so an exit 0 is not read as "access is safe". */
const NOT_CHECKED_APP = 'access NOT checked: enforced in app code'
/** Where in a package.json the Supabase client is named: a dependency says Supabase, a peer or optional one does not. */
function supabaseInPackageJson(text) {
let pkg
try {
pkg = JSON.parse(text)
} catch {
return SUPABASE_DEP.test(text) ? 'strong' : null
}
const named = (key) => Object.keys(pkg?.[key] ?? {}).some((n) => SUPABASE_DEP.test(`"${n}"`))
if (named('dependencies') || named('devDependencies')) return 'strong'
return named('peerDependencies') || named('optionalDependencies') ? 'weak' : null
}
/** The same for Python: an optional extra, or a dev or test requirements file, is a weak signal. */
function supabaseInPython(name, text) {
if (!name.startsWith('pyproject')) return SUPABASE_PY.test(text) ? (/(?:dev|test|extra|optional)/i.test(name) ? 'weak' : 'strong') : null
let found = null
for (const section of text.split(/^(?=\[)/m)) {
if (!SUPABASE_PY.test(section)) continue
if (!/^\[(?:project\.optional-dependencies|tool\.poetry\.extras)\]/.test(section)) return 'strong'
found = 'weak'
}
return found
}
/**
* Supabase or plain Postgres, from what the repo holds. A strong signal says
* Supabase; none says Postgres. A folder that is not the live app (templates,
* examples, docs, fixtures) is not read, and a weak signal (a peer or optional
* dependency, an optional extra, a dev requirements file) is named but never
* decides. `sqlTexts` is the migrations' text, read by the caller. Returns
* { platform, reasons, weak }: the reasons are what it found, or what it looked
* for and did not find; weak is what it saw and did not count.
*/
export function detectPlatform(root, scan, sqlTexts = []) {
const reasons = []
const weak = []
const look = (rel, name, test, what, weakNote = 'optional') => {
const hit = test(readFileText(join(root, rel ? `${rel}/${name}` : name)))
const where = `${what} in ${rel ? `${rel}/` : ''}${name}`
if (hit === 'weak') weak.push(`${where} (${weakNote})`)
else if (hit && reasons.length < 4) reasons.push(where)
}
for (const [dir, names] of scan.dirs) {
if (dir.split('/').length > 4 || NOT_LIVE.test(dir)) continue
if (dir.split('/').at(-1) === 'supabase' && names.includes('config.toml')) reasons.push(`${dir}/config.toml`)
// A package that adapts one provider (packages/auth-supabase) is an option the app may not use.
if (names.includes('package.json')) look(dir, 'package.json', (t) => (PROVIDER_DIR.test(dir.split('/').at(-1)) && supabaseInPackageJson(t) ? 'weak' : supabaseInPackageJson(t)), 'a @supabase dependency')
for (const n of names.filter((x) => /^(?:requirements[\w.-]*\.txt|pyproject\.toml)$/.test(x))) look(dir, n, (t) => supabaseInPython(n, t), 'the supabase package')
// An env example lists every provider the app can be pointed at, and a comment or an 'or clerk' is not a choice: a hint, never the decision.
for (const n of names.filter((x) => ENV_EXAMPLE.test(x))) look(dir, n, (t) => (SUPABASE_URL.test(t) ? 'weak' : null), 'a Supabase URL or key', 'an env example lists options, it does not choose')
}
for (const [path, text] of sqlTexts) {
if (NOT_LIVE.test(path)) continue
const m = SUPABASE_SQL.exec(text)
if (m) {
reasons.push(`${m[0].trim().replace(/\($/, '()')} in ${path}`)
break
}
}
if (reasons.length) return { platform: 'supabase', reasons, weak }
return {
platform: 'postgres',
reasons: [
`no supabase/config.toml, no @supabase dependency, no Supabase URL in an env example, no auth.uid() or authenticated role in the migrations${weak.length ? `. Seen and not counted: ${weak.join(', ')}` : ''}`,
],
weak,
}
}
/** The text of the plan's SQL files, as [path, text], capped. */
function sqlTextsOf(root, items) {
const out = []
let total = 0
for (const i of items) {
if (i.kind !== 'file' || out.length >= 300 || total > 8e6) continue
const text = readText(join(root, i.path))
total += text.length
out.push([i.path, text])
}
return out
}
/**
* CONFIG.platform and CONFIG.clientAccess with 'auto' read from the repo.
* Returns the config with both resolved, and where each came from, for the
* report header and --print-config. An explicit value is kept and says so.
*/
export function resolveSettings(config, { root, items = [], scan = null }) {
const sqlite = items.some((i) => i.dialect === 'sqlite')
let platform = config.platform
let platformWhy = 'set in CONFIG'
const sqlTexts = sqlTextsOf(root, items)
if (platform === 'auto' || platform === undefined) {
if (sqlite) {
platform = 'postgres'
platformWhy = 'auto: the schema is SQLite, where the platform does not apply'
} else {
const found = detectPlatform(root, scan ?? scanRepo(root, { maxDepth: 4 }), sqlTexts)
platform = found.platform
platformWhy = `auto: ${found.platform === 'supabase' ? `found ${found.reasons.join(', ')}` : found.reasons[0]}`
}
}
let clientAccess = config.clientAccess ?? 'auto'
let clientWhy = 'set in CONFIG'
if (clientAccess === 'auto') {
clientAccess = platform === 'supabase' ? 'direct' : 'server-only'
clientWhy = platform === 'supabase' ? "auto: Supabase's Data API serves the database to clients" : 'auto: plain Postgres is reached by your server only. Set clientAccess: \'direct\' if a client reaches it (PostgREST, Hasura, a connection string in an app)'
}
const hasRls = sqlTexts.some(([, t]) => ANY_RLS.test(t))
return { config: { ...config, platform, clientAccess }, platformWhy, clientWhy, noRls: sqlTexts.length > 0 && !hasRls }
}
// ---------------------------------------------------------------- LOADER ---
const lineAt = (text, index) => text.slice(0, index).split('\n').length
const newlines = (s) => '\n'.repeat((s.match(/\n/g) ?? []).length)
let pgliteModule = null
/** Import PGlite, or refuse with the fix. Resolved from this file, so the repo's node_modules. */
export async function loadPGlite() {
if (pgliteModule) return pgliteModule
try {
pgliteModule = await import('@electric-sql/pglite')
return pgliteModule
} catch (e) {
throw new CannotCheck('PGLITE-MISSING', {
object: '@electric-sql/pglite',
message: `@electric-sql/pglite could not be imported (${e.code ?? e.message}).`,
fix: 'Run the backend-deps step: add @electric-sql/pglite 0.5.8 to devDependencies and install.',
})
}
}
/** The PGlite extension object for a SQL extension name, or null when PGlite has none. */
async function extensionModule(name) {
if (name === 'vector') {
try {
const mod = await import('@electric-sql/pglite-pgvector')
return mod.vector ?? mod.default ?? null
} catch {
return null
}
}
const id = name.replace(/-/g, '_')
if (!/^[a-z0-9_]+$/.test(id)) return null
try {
const mod = await import(`@electric-sql/pglite/contrib/${id}`)
return mod[id] ?? Object.values(mod)[0] ?? null
} catch {
return null
}
}
// The closing semicolon is optional: a CONFIG.preamble entry is usually one statement without one.
const CREATE_EXTENSION = /\bcreate\s+extension\s+(?:if\s+not\s+exists\s+)?("?)([A-Za-z0-9_-]+)\1[^;]*(?:;|$)/gi
const COMMENT_ON_EXTENSION = /\bcomment\s+on\s+extension\s+("?)([A-Za-z0-9_-]+)\1\s+is\s+(?:'(?:[^']|'')*'|null)\s*;/gi
const CONCURRENTLY = /(\b(?:create\s+(?:unique\s+)?|drop\s+)index)(\s+)concurrently\b/gi
/**
* A migration file's bytes as text. A UTF-8 byte order mark is dropped, and
* UTF-16 (a BOM, or every other byte NUL) is decoded. Windows editors and
* PowerShell redirects write both, and Postgres reads neither.
* Returns { text, encoding } with encoding null for plain UTF-8.
*/
export function decodeSql(bytes) {
const b = Buffer.from(bytes)
if (b[0] === 0xef && b[1] === 0xbb && b[2] === 0xbf) return { text: b.subarray(3).toString('utf8'), encoding: 'utf-8 BOM' }
if (b[0] === 0xff && b[1] === 0xfe) return { text: b.subarray(2).toString('utf16le'), encoding: 'utf-16le' }
if (b[0] === 0xfe && b[1] === 0xff) {
const swapped = Buffer.from(b.subarray(2))
swapped.swap16()
return { text: swapped.toString('utf16le'), encoding: 'utf-16be' }
}
if (b.length >= 4 && b[1] === 0 && b[3] === 0 && b[0] !== 0 && b[2] !== 0) return { text: b.toString('utf16le'), encoding: 'utf-16le' }
return { text: b.toString('utf8'), encoding: null }
}
/**
* Split SQL into statements, for the two places that need them: the statement
* CONFIG.skipStatements names, and a file that failed as a whole. Quotes,
* dollar quotes, comments, parentheses and `begin atomic` bodies hold their
* semicolons. Returns [{ text, index, line }] with text including its own
* semicolon, and line the first line with SQL on it. A run of comments with no
* SQL is not a statement.
*/
export function splitStatements(text) {
const out = []
let start = 0
let depth = 0
let atomic = null
let prev = ''
let i = 0
const n = text.length
const flush = (end) => {
const raw = text.slice(start, end)
// First character that is not whitespace or comment.
let k = 0
for (;;) {
while (k < raw.length && /\s/.test(raw[k])) k++
if (raw.startsWith('--', k)) {
const nl = raw.indexOf('\n', k)
k = nl === -1 ? raw.length : nl + 1
} else if (raw.startsWith('/*', k)) {
let d = 0
let j = k
do {
if (raw.startsWith('/*', j)) {
d++
j += 2
} else if (raw.startsWith('*/', j)) {
d--
j += 2
} else j++
} while (d > 0 && j < raw.length)
k = j
} else break
}
if (k < raw.length && raw.slice(k).replace(/;/g, '').trim() !== '') {
out.push({ text: raw.slice(k), index: start + k, line: lineAt(text, start + k) })
}
start = end
}
while (i < n) {
const c = text[i]
if (c === '-' && text[i + 1] === '-') {
const nl = text.indexOf('\n', i)
i = nl === -1 ? n : nl
} else if (c === '/' && text[i + 1] === '*') {
let d = 0
do {
if (text.startsWith('/*', i)) {
d++
i += 2
} else if (text.startsWith('*/', i)) {
d--
i += 2
} else i++
} while (d > 0 && i < n)
} else if (c === "'") {
const escapes = /[eE]/.test(text[i - 1] ?? '') && !/[\w$]/.test(text[i - 2] ?? '')
i++
while (i < n) {
if (escapes && text[i] === '\\') i += 2
else if (text[i] === "'" && text[i + 1] === "'") i += 2
else if (text[i] === "'") break
else i++
}
i++
} else if (c === '"') {
i++
while (i < n && !(text[i] === '"' && text[i + 1] !== '"')) i += text[i] === '"' ? 2 : 1
i++
} else if (c === '$' && !/[\w$]/.test(text[i - 1] ?? '')) {
const tag = /^\$(?:[A-Za-z_][\w]*)?\$/.exec(text.slice(i, i + 80))
if (tag) {
const close = text.indexOf(tag[0], i + tag[0].length)
i = close === -1 ? n : close + tag[0].length
} else i++
} else if (c === '(') {
depth++
i++
} else if (c === ')') {
depth = Math.max(0, depth - 1)
i++
} else if (/[A-Za-z_]/.test(c) && !/[\w$]/.test(text[i - 1] ?? '')) {
const word = /^[A-Za-z_]\w*/.exec(text.slice(i, i + 40))[0].toLowerCase()
if (word === 'atomic' && prev === 'begin') atomic = 0
else if (atomic !== null && word === 'case') atomic++
else if (atomic !== null && word === 'end') {
if (atomic === 0) atomic = null
else atomic--
}
prev = word
i += word.length
} else if (c === ';') {
i++
if (depth === 0 && atomic === null) flush(i)
} else i++
}
flush(n)
return out
}
const normalise = (s) => s.replace(/\s+/g, ' ').trim().toLowerCase()
/**
* Pre-scan and rewrite the migrations before anything boots: which extensions
* PGlite has to be created with, which are stubbed, and the statements PGlite
* cannot run as written. Line counts are kept, so an error position still maps
* to the line in the file. Every rewrite is recorded in the snapshot.
* CONFIG.preamble goes through the same scan, so it may create an extension.
*
* inputs: [{ path, text, sections, encoding? }]. Returns
* { files, preamble, extensions, stubbed, rewritten, empty, skipped }.
*/
export async function prepareInputs(inputs, config, { expiredStubs = {} } = {}) {
const extensions = new Set(config.platform === 'supabase' ? ['pgcrypto', 'uuid-ossp'] : [])
const stubbed = []
const rewritten = []
const stubbedNames = new Set()
const available = new Map()
const skips = Object.keys(config.skipStatements ?? {}).map((key) => ({ key, needle: normalise(key) }))
const skipped = new Set()
// Every extension that cannot be had, so one run names them all.
const unavailable = new Map()
const has = async (name) => {
if (!available.has(name)) available.set(name, (await extensionModule(name)) !== null)
return available.get(name)
}
const prepare = async (path, source) => {
let text = source
// Collect first, replace after: the decision for each statement is async.
const edits = []
const covered = []
if (skips.length) {
for (const st of splitStatements(text)) {
const flat = normalise(st.text)
const hit = skips.find((s) => flat.includes(s.needle))
if (!hit) continue
skipped.add(hit.key)
covered.push([st.index, st.index + st.text.length])
edits.push({
index: st.index,
length: st.text.length,
replacement: 'select 1;',
what: `statement skipped -> select 1 (CONFIG.skipStatements '${hit.key}')`,
line: st.line,
})
}
}
const inside = (index) => covered.some(([a, b]) => index >= a && index < b)
for (const m of text.matchAll(CREATE_EXTENSION)) {
if (inside(m.index)) continue
const name = m[2].toLowerCase()
const line = lineAt(text, m.index)
const at = `${path}:${line}`
if (config.platform === 'supabase' && SHIMMED_EXTENSIONS.has(name)) {
edits.push({
index: m.index,
length: m[0].length,
replacement: 'select 1;',
what: `create extension ${name} -> select 1 (the shim created it)`,
line,
})
} else if (name === 'plpgsql') {
// Built in. Kept, so it behaves as it would on Postgres.
} else if (await has(name)) {
extensions.add(name)
} else if (STUBS[name]) {
edits.push({
index: m.index,
length: m[0].length,
replacement: STUBS[name].join(' '),
what: `create extension ${name} -> built in stub`,
line,
})
if (!stubbedNames.has(name)) stubbed.push({ name, reason: 'built in stub', by: 'builtin' })
stubbedNames.add(name)
} else if (config.stubExtensions[name]) {
edits.push({
index: m.index,
length: m[0].length,
replacement: 'select 1;',
what: `create extension ${name} -> select 1 (CONFIG.stubExtensions)`,
line,
})
if (!stubbedNames.has(name)) stubbed.push({ name, reason: config.stubExtensions[name].reason, by: 'config' })
stubbedNames.add(name)
} else if (!unavailable.has(name)) {
const expired = expiredStubs[name]
const waiver = expired ? ` (waiver expired ${expired})` : ''
unavailable.set(
name,
name === 'vector'
? {
code: 'EXTENSION-UNAVAILABLE',
object: 'vector',
message: `${at} creates extension vector, and @electric-sql/pglite-pgvector is not installed.${waiver}`,
fix: 'Add @electric-sql/pglite-pgvector 0.0.9 as a devDependency, or stub vector in CONFIG.stubExtensions if no table the checks read uses it. A stubbed vector has no vector type, so skip the statements that use it with CONFIG.skipStatements.',
}
: {
code: 'EXTENSION-UNAVAILABLE',
object: name,
message: `${at} creates extension ${name}, which PGlite does not have.${waiver}`,
fix: `PGlite has no ${name}. If no table the checks read uses it, add it to CONFIG.stubExtensions with a reason and an until date.`,
},
)
}
}
for (const m of text.matchAll(COMMENT_ON_EXTENSION)) {
if (inside(m.index)) continue
if (!stubbedNames.has(m[2].toLowerCase())) continue
edits.push({
index: m.index,
length: m[0].length,
replacement: 'select 1;',
what: `comment on extension ${m[2]} -> select 1 (stubbed)`,
line: lineAt(text, m.index),
})
}
for (const m of text.matchAll(CONCURRENTLY)) {
if (inside(m.index)) continue
edits.push({
index: m.index,
length: m[0].length,
replacement: `${m[1]}${m[2]}`,
what: `${m[1].replace(/\s+/g, ' ').toLowerCase()} concurrently -> ${m[1].replace(/\s+/g, ' ').toLowerCase()}`,
line: lineAt(text, m.index),
})
}
edits.sort((a, b) => b.index - a.index)
for (const e of edits) {
const original = text.slice(e.index, e.index + e.length)
text = text.slice(0, e.index) + e.replacement + newlines(original) + text.slice(e.index + e.length)
}
for (const e of edits.reverse()) rewritten.push({ file: path, line: e.line, what: e.what })
return text
}
const preamble = []
for (const [i, entry] of config.preamble.entries()) preamble.push(await prepare(`CONFIG.preamble[${i}]`, entry.sql))
const files = []
const empty = []
for (const input of inputs) {
const source = upSection(input.text, input.sections)
if (input.encoding) {
rewritten.push({
file: input.path,
line: 1,
what: input.encoding === 'utf-8 BOM' ? 'removed a UTF-8 byte order mark' : `decoded ${input.encoding} to text`,
})
}
// A file with no statement runs nothing. A comment-only one is usually a
// migration somebody meant to write, or one that was stripped.
if (!input.command && !splitStatements(source).length) empty.push({ file: input.path, comments: source.trim() !== '' })
const text = await prepare(input.path, source)
files.push({ path: input.path, text, ...(input.late ? { late: true } : {}), ...(input.command ? { command: input.command } : {}) })
}
if (unavailable.size) {
const all = [...unavailable.values()]
const [first] = all
throw new CannotCheck('EXTENSION-UNAVAILABLE', {
object: first.object,
message: all.length === 1 ? first.message : `${all.length} extensions are not available: ${all.map((p) => p.object).join(', ')}. ${first.message}`,
fix: first.fix,
...(all.length > 1 ? { problems: all } : {}),
})
}
return { files, preamble, extensions: [...extensions], stubbed, rewritten, empty, skipped: [...skipped] }
}
/** A clock for --timing. Each phase() ends the one before it. */
export const makeClock = (emit = () => {}) => {
const times = {}
let current = null
let since = 0
const stop = () => {
if (current) times[current] = (times[current] ?? 0) + (performance.now() - since)
}
return {
phase(name) {
stop()
current = name
since = performance.now()
emit(name)
},
end() {
stop()
current = null
return Object.fromEntries(Object.entries(times).map(([k, v]) => [k, Math.round(v)]))
},
}
}
/**
* Boot PGlite. Fresh: with the extensions the migrations need, then the shim
* on platform 'supabase'. From a datadir (a Blob from dumpDataDir): the same
* extensions again, then the session reset, because loadDataDir brings back
* roles and default privileges but not search_path (verified).
*/
export async function openDatabase({ config, extensions = [], dataDir = null, clock = makeClock() }) {
const { PGlite } = await loadPGlite()
const names = [...new Set([...(config.platform === 'supabase' ? ['pgcrypto', 'uuid-ossp'] : []), ...extensions])]
const ext = {}
for (const name of names) {
const mod = await extensionModule(name)
if (!mod) {
throw new CannotCheck('EXTENSION-UNAVAILABLE', {
object: name,
message: `PGlite has no ${name}, which this database was built with.`,
fix: `Install the package that provides ${name} for PGlite, or clear the cache with --no-cache.`,
})
}
ext[name.replace(/-/g, '_')] = mod
}
const db = await PGlite.create({ extensions: ext, ...(dataDir ? { loadDataDir: dataDir } : {}) })
if (dataDir) {
await db.exec(RESET_SESSION)
return db
}
if (config.platform === 'supabase') {
clock.phase('shim')
await db.exec(shimSql(config))
await db.exec(AFTER_SHIM)
}
return db
}
/** How a load failure reads, by class: the heading, and the way out. */
export const LOAD_CLASSES = {
real: {
title: 'a real Postgres error',
fix: 'Postgres refuses this SQL too, so fix the migration. For a statement that inserts or updates rows, the migration relies on data that exists only in production.',
},
ordering: {
title: 'an ordering problem',
fix: 'Real Postgres refuses these in this order too: what they use is created later. Move the creating migration earlier, or load files in the right order with one entry per file in CONFIG.sources (order: name sorts by path, natural sorts V2 before V10). If a file that runs last adds what they use (alter table ... add column), list it as its own entry with late: true.',
},
outside: {
title: 'created outside the migrations',
fix: 'No migration creates what these statements use: it was made in the dashboard, by an ORM or by another script. Put its definition in a base schema file (for example pg_dump --schema-only, trimmed to what is used) as the FIRST entry of CONFIG.sources, so it loads before the migrations. For one or two objects, CONFIG.preamble with a reason is enough. If an ORM makes the tables when the app starts (SQLAlchemy create_all, Prisma db push), a command source can print its DDL: { kind: \'command\', run: [\'python\', \'ntent/base_schema.py\'], dialect: \'postgres\', watch: [\'app/models/\'] }, as the first entry, or the only one when the migrations do not build on their own.',
},
duplicate: {
title: 'a duplicate',
fix: 'The object already exists: two migrations create it, or a migration is repeated. Rename the second policy or object, or make it idempotent (drop ... if exists first). If the same schema is listed twice in CONFIG.sources, list it once.',
},
engine: {
title: 'a PGlite or extension gap',
fix: 'PGlite cannot run this, and Postgres could. If nothing the checks read depends on the statement, skip it with CONFIG.skipStatements (a key that appears in the statement, with a reason and an until date). For an extension, stub it in CONFIG.stubExtensions or create it in CONFIG.preamble.',
},
cascade: {
title: 'a result of another failure',
fix: 'These fail because an object an earlier failure should have created is missing. Fix that one first.',
},
}
const DUPLICATE_CODES = new Set(['42710', '42P07', '42723', '42P06', '42701', '42P04', '42712', '42P16'])
const MISSING_CODES = new Set(['42P01', '42703', '42883', '3F000', '42704', '0P000'])
const ENGINE_MESSAGE =
/timescaledb|pg_cron|pgsodium|supabase_vault|pg_net|extension "[^"]+" is not available|could not open extension control file|could not access file|unrecognized parameter namespace|not supported in (?:PGlite|WASM)|shared_preload_libraries/i
const ENGINE_SCHEMAS = /^(?:cron|net|vault|pgsodium|timescaledb_information|_timescaledb\w*)\./i
// A function comes with its argument types, which hold spaces: function f(uuid, text) does not exist.
const MISSING_OBJECT =
/\b(relation|schema|function|type|role|column|table|extension)\s+("(?:[^"]|"")+"|[^\s,"(]+(?:\([^)]*\))?)(?:\s+of relation\s+"([^"]+)")?\s+does not exist/i
const CREATES =
/\bcreate\s+(?:or\s+replace\s+)?(?:unlogged\s+|temp(?:orary)?\s+|materialized\s+)?(?:table|view|function|procedure|type|schema|sequence|extension|domain)\s+(?:if\s+not\s+exists\s+)?("?[\w$]+"?(?:\."?[\w$]+"?)?)/gi
const lastName = (name) => name.replace(/\(.*$/, '').split('.').pop().replace(/"/g, '').toLowerCase()
/** The object a missing-object error names, as { kind, name, relation }; null when it names none. */
const missingObject = (message) => {
const m = MISSING_OBJECT.exec(message)
if (!m) return null
return { kind: m[1].toLowerCase(), name: m[2].replace(/^"|"$/g, '').replace(/""/g, '"'), relation: m[3] ?? null }
}
/** Where a later statement or file creates a name, as 'file:line'; null when none does. */
function createdAt(name, files, after) {
const want = lastName(name)
for (const [i, file] of files.entries()) {
if (i < after.file) continue
for (const m of file.text.matchAll(CREATES)) {
if (lastName(m[1]) !== want) continue
const line = lineAt(file.text, m.index)
if (i === after.file && line <= after.line) continue
return `${file.path}:${line}`
}
const col = new RegExp(`\\badd\\s+column\\s+(?:if\\s+not\\s+exists\\s+)?"?${want.replace(/[^\w]/g, '')}"?\\b`, 'i').exec(file.text)
if (col) return `${file.path}:${lineAt(file.text, col.index)}`
}
return null
}
const OBJECT_KINDS = 'table|view|materialized\\s+view|function|procedure|type|schema|sequence|domain|policy|trigger|index|extension'
const OBJECT_NAME = '"?[\\w$]+"?(?:\\."?[\\w$]+"?)?'
const LEADING_COMMENTS = /^(?:\s+|--[^\n]*(?:\n|$)|\/\*[\s\S]*?\*\/)*/
const DROP_OBJECT = new RegExp(`^drop\\s+(${OBJECT_KINDS})\\s+(?:if\\s+exists\\s+)?(${OBJECT_NAME})(?:\\s+on\\s+(${OBJECT_NAME}))?`, 'i')
const CREATE_OBJECT = new RegExp(`^create\\s+(?:or\\s+replace\\s+)?(?:unique\\s+|unlogged\\s+|temp(?:orary)?\\s+)?(${OBJECT_KINDS})\\s+(?:concurrently\\s+)?(?:if\\s+not\\s+exists\\s+)?(${OBJECT_NAME})`, 'i')
/** What a drop or create statement names, as { kind, name, table }; null for any other statement. */
export function objectOf(text, create) {
const body = text.slice(LEADING_COMMENTS.exec(text)?.[0].length ?? 0)
const m = (create ? CREATE_OBJECT : DROP_OBJECT).exec(body)
if (!m) return null
const table = create ? new RegExp(`\\bon\\s+(?:only\\s+)?(${OBJECT_NAME})`, 'i').exec(body.slice(m[0].length))?.[1] : m[3]
return { kind: m[1].toLowerCase().replace(/\s+/g, ' '), name: lastName(m[2]), table: table ? lastName(table) : null }
}
const sameObject = (a, b) => a.kind === b.kind && a.name === b.name && (!a.table || !b.table || a.table === b.table)
// PGlite leaks a little stack on every error, and after about 2,000 of them
// every statement fails with 54001 "stack depth limit exceeded", even a
// create index. The load starts a fresh database from a dump well before that.
const RECYCLE_AFTER = 1200
// A statement that fails with 54001 on a fresh database is too deep for PGlite
// for real. Two of those prove it; a third restart would only cost time.
const MAX_STACK_RECYCLES = 2
/**
* CONFIG.preamble, then every migration in order, then the session reset.
* A failure does not stop the load. A file that fails whole is run again
* statement by statement, every failed statement that Postgres would accept
* once something later exists is tried again after the rest has loaded, and
* what is left is classified (LOAD_CLASSES). The result is one LOAD-FAILED,
* exit 2, with every failure in `problems`, each naming file:line. A
* statement at line N of a file is exact; the line inside a DO block is the
* line of the DO.
*
* Returns the database. It is a new one when the load had to start over from
* a dump (see RECYCLE_AFTER), so the caller uses what comes back.
*/
export async function loadSchema(db, prepared, config, clock = makeClock()) {
let errors = 0
let recycles = 0
let stackRecycles = 0
const recycle = async () => {
const dump = await db.dumpDataDir('gzip')
await db.close()
db = await openDatabase({ config, extensions: prepared.extensions ?? [], dataDir: dump })
errors = 0
recycles++
}
/** One run of some SQL. A 54001 is PGlite's, not the SQL's: once on a fresh database settles it. */
const exec = async (sql) => {
if (errors >= RECYCLE_AFTER) await recycle()
try {
await db.exec(sql)
} catch (e) {
errors++
await db.exec('rollback').catch(() => {})
if (e.code !== '54001' || stackRecycles >= MAX_STACK_RECYCLES) throw e
stackRecycles++
await recycle()
try {
await db.exec(sql)
} catch (again) {
errors++
await db.exec('rollback').catch(() => {})
throw again
}
}
}
if (config.preamble.length) {
clock.phase('preamble')
for (const [i, entry] of config.preamble.entries()) {
try {
await exec(prepared.preamble?.[i] ?? entry.sql)
} catch (e) {
throw new CannotCheck('CONFIG-INVALID', {
object: `preamble[${i}]`,
message: `${e.code ?? ''} ${e.message}`.trim(),
fix: `Fix the SQL in CONFIG.preamble[${i}]. It runs after the shim and before the first migration.`,
})
}
}
}
clock.phase('migrate')
const failures = []
// Every statement that ran on its own, in load order, for the retry below.
// A drop is kept (and run again) only when a failed statement re-creates what it drops.
let seq = 0
const drops = []
const loadFile = async (file, fileIndex) => {
try {
await exec(file.text)
return
} catch {
// Whole file failed, and exec has rolled it back.
}
// Its transaction control is dropped: each statement is its own
// transaction now, so a failed one leaves the rest running.
for (const st of splitStatements(file.text)) {
if (/^\s*(?:begin|commit|end|start\s+transaction|rollback|abort)\b[^;]*;?\s*$/i.test(st.text)) continue
seq++
try {
await exec(st.text)
const dropped = objectOf(st.text, false)
if (dropped) drops.push({ seq, st, dropped })
} catch (e) {
failures.push({ file, fileIndex, st, e, seq })
}
}
}
// A late file (CONFIG.sources late: true) holds what the migrations rely on
// but never create, so it runs after all of them.
for (const [i, file] of prepared.files.entries()) if (!file.late) await loadFile(file, i)
const hasLate = prepared.files.some((f) => f.late)
for (const [i, file] of prepared.files.entries()) if (file.late) await loadFile(file, i)
if (failures.length) {
// A statement that works once the rest has loaded is an ordering problem.
// Only a missing object (or a row whose parent is not there yet) can be
// one: a syntax error or a clash does not go away because of what comes
// later, and a statement that happens to pass after a later migration
// dropped the thing it clashed with still fails on a real database.
// Repeat while that unblocks more, because one late table can hold up a chain.
let pending = failures.filter((f) => MISSING_CODES.has(f.e.code) || f.e.code === '23503')
for (const f of pending) f.creates = objectOf(f.st.text, true)
for (let round = 0; round < 6 && pending.length; round++) {
const next = []
// In the order they first ran, with each drop that a later failed
// statement undoes: a migration that drops a view and creates it again
// needs the drop to run again, or the create finds the first one.
const steps = [...pending, ...drops.filter((d) => pending.some((f) => f.creates && f.seq > d.seq && sameObject(d.dropped, f.creates)))].sort((a, b) => a.seq - b.seq)
for (const f of steps) {
if (!f.e) {
await exec(f.st.text).catch(() => {})
continue
}
try {
await exec(f.st.text)
f.ordering = true
// With a late file the statement was waiting for it: that is the cure, not a finding.
f.settled = hasLate
} catch (e) {
f.e = e
next.push(f)
}
}
if (next.length === pending.length) break
pending = next
}
// The caller's handle is the first database, which a recycle closed.
if (recycles && failures.some((f) => !f.settled)) await db.close().catch(() => {})
const left = failures.filter((f) => !f.settled)
if (left.length) throw loadFailed(left, prepared.files, prepared.stubbed ?? [])
}
clock.phase('reset')
await db.exec(RESET_SESSION)
return db
}
/** The extension a missing function or type comes from, for the ones PGlite can load. */
const EXTENSION_OF = {
uuid_generate_v1: 'uuid-ossp',
uuid_generate_v1mc: 'uuid-ossp',
uuid_generate_v3: 'uuid-ossp',
uuid_generate_v4: 'uuid-ossp',
uuid_generate_v5: 'uuid-ossp',
uuid_nil: 'uuid-ossp',
crypt: 'pgcrypto',
gen_salt: 'pgcrypto',
digest: 'pgcrypto',
hmac: 'pgcrypto',
gen_random_bytes: 'pgcrypto',
pgp_sym_encrypt: 'pgcrypto',
pgp_sym_decrypt: 'pgcrypto',
similarity: 'pg_trgm',
word_similarity: 'pg_trgm',
show_trgm: 'pg_trgm',
unaccent: 'unaccent',
levenshtein: 'fuzzystrmatch',
soundex: 'fuzzystrmatch',
citext: 'citext',
hstore: 'hstore',
ltree: 'ltree',
}
const STUB_GAP = /operator class "[^"]*" does not exist for access method|access method "[^"]*" does not exist/i
const FAILING_ROW = /^Failing row contains \((.*)\)\.?$/s
const FAILING_KEY = /^Key \([^)]*\)=\((.*?)\)(?: is not present| already exists)/s
/**
* The line inside a statement that holds the row a constraint refused, as an
* offset from its first line. Postgres gives no position for a constraint, so
* the row it names is looked for in the text: a line that holds every value
* of it, when exactly one does. 0 when none or several do.
*/
function rowOffset(text, detail) {
const m = FAILING_ROW.exec(detail ?? '') ?? FAILING_KEY.exec(detail ?? '')
if (!m) return 0
const values = m[1].split(/,\s*/).filter((v) => v !== '')
if (!values.length || values.length > 16) return 0
const lines = text.split('\n')
const tokens = values.map((v) => new RegExp(`(?<![\\w.])'?${esc(v)}'?(?![\\w.])`, 'i'))
// A value the text does not hold is a default the table filled in.
const present = tokens.filter((t) => t.test(text))
if (!present.length) return 0
const at = lines.flatMap((line, i) => (present.every((t) => t.test(line)) ? [i] : []))
return at.length === 1 ? at[0] : 0
}
/** Whether the file drops what this create statement makes, earlier in the same file. */
function alreadyDropped(file, st) {
const made = objectOf(st.text, true)
if (!made) return false
return splitStatements(file.text).some((other) => {
if (other.line >= st.line) return false
const dropped = objectOf(other.text, false)
return !!dropped && sameObject(dropped, made)
})
}
/** Classify every failure and build the one LOAD-FAILED that carries them. */
export function loadFailed(failures, files, stubbed = []) {
const real = failures.filter((f) => !f.ordering)
const failedCreates = new Map()
for (const f of real) {
for (const m of f.st.text.matchAll(CREATES)) failedCreates.set(lastName(m[1]), `${f.file.path}:${f.st.line}`)
}
const stubNames = stubbed.map((s) => s.name)
// Where a failure that is the engine's or a stub's gap sits, so what depends on it can say to skip it too.
const engineAt = new Set(
real.flatMap((f) => {
const message = String(f.e.message)
const missing = missingObject(message)
const gap = ENGINE_MESSAGE.test(message) || ENGINE_SCHEMAS.test(missing?.name ?? '') || STUB_GAP.test(message) || (stubNames.length && missing && ['function', 'type'].includes(missing.kind) && !EXTENSION_OF[lastName(missing.name)])
return gap ? [`${f.file.path}:${f.st.line}`] : []
}),
)
const problems = failures.map((f) => {
const { e, st, file } = f
const position = Number(e.position)
const hasPosition = Number.isFinite(position) && position > 0
const line = st.line + (hasPosition ? st.text.slice(0, position - 1).split('\n').length - 1 : rowOffset(st.text, e.detail))
const code = e.code ?? ''
const missing = missingObject(String(e.message))
let cls = 'real'
let by = null
let fix = null
if (f.ordering) {
cls = 'ordering'
const target = missing?.relation ?? missing?.name
by = target ? createdAt(target, files, { file: f.fileIndex, line: st.line }) : null
} else if (DUPLICATE_CODES.has(code)) cls = 'duplicate'
else if (ENGINE_MESSAGE.test(String(e.message)) || ENGINE_SCHEMAS.test(missing?.name ?? '') || ['58P01', '58P02', '54001'].includes(code)) cls = 'engine'
else if (stubNames.length && STUB_GAP.test(String(e.message))) cls = 'engine'
else if (MISSING_CODES.has(code) && missing) {
const owner = missing.relation ?? missing.name
const cause = failedCreates.get(lastName(owner))
if (cause && cause !== `${file.path}:${st.line}`) {
cls = 'cascade'
by = cause
} else if (stubNames.length && ['function', 'type'].includes(missing.kind) && !EXTENSION_OF[lastName(missing.name)]) cls = 'engine'
else cls = 'outside'
}
const where = line ? `${file.path}:${line}` : file.path
if (cls === 'outside') {
const name = missing.kind === 'column' ? `column ${missing.name}${missing.relation ? ` of ${missing.relation}` : ''}` : `${missing.kind} ${missing.name}`
const ext = ['function', 'type'].includes(missing.kind) ? EXTENSION_OF[lastName(missing.name)] : null
// The table a missing column belongs to: named by the error, or one the statement reads that a migration creates.
const tables = missing.relation ? [missing.relation] : st.text.match(/(?<=\b(?:from|join|update|into|table)\s+)[\w."]+/gi) ?? []
const onMigrationTable = missing.kind === 'column' && tables.map((r) => createdAt(r, files, { file: 0, line: -1 })).find(Boolean)
fix = onMigrationTable
? `${name} is missing from a table the migrations create (${onMigrationTable}), so it was added outside them. List a SQL file that adds it (alter table ... add column if not exists) as its own CONFIG.sources entry with late: true, so it runs after the migrations and the statements that failed are run again. Or fix the migration that creates the table.`
: ext
? `${name} comes from the ${ext} extension, and nothing creates the extension. Add { sql: 'create extension if not exists "${ext}"', reason: '...' } to CONFIG.preamble, or create it in a migration.`
: `Nothing in the migrations creates ${name}. ${LOAD_CLASSES.outside.fix}`
} else if (cls === 'ordering' && by?.startsWith(`${file.path}:`)) {
// Inside one file there is no source order to change, and Postgres runs the file top to bottom.
fix = `Real Postgres refuses this too: what it uses is created later in the same file (${by}), and a file runs top to bottom. Move the creating statement above this one. For a foreign key, create both tables first and add the constraint after (alter table ... add constraint ... foreign key). No CONFIG entry can reorder inside a file. While the file does not load, CONFIG.loadGate: 'warn' keeps this from blocking commits.`
} else if (cls === 'ordering') {
fix = `${by ? `What this uses is created at ${by}, after it. ` : ''}${LOAD_CLASSES.ordering.fix}`
} else if (cls === 'duplicate' && alreadyDropped(file, st)) {
fix = 'This file already drops the object first (drop ... if exists), so it was created again between that drop and this statement: by another statement here or in a file that loads later, or by a statement that ran again after a late source. Remove the extra create, or give one of the two a new name.'
} else if (cls === 'cascade') {
fix = `${by} failed, and this uses what it should have created. Fix ${by} first.${engineAt.has(by) ? ' That one is a PGlite or stubbed-extension gap: if you skip it with CONFIG.skipStatements, use a key that also appears here (the table name), and this is skipped with it.' : ''}`
} else if (cls === 'engine' && code === '54001') {
fix = 'PGlite ran out of stack on this statement, again on a fresh database. Postgres may run it. If nothing the checks read depends on it, skip it with CONFIG.skipStatements (a key that appears in the statement, with a reason and an until date).'
} else if (cls === 'engine' && stubNames.length && missing && ['function', 'type'].includes(missing.kind)) {
fix = `${stubNames.join(', ')} ${stubNames.length === 1 ? 'is' : 'are'} stubbed, and a stub has none of the extension's functions or types. If ${missing.kind} ${missing.name} comes from ${stubNames.length === 1 ? 'it' : 'one of them'}, skip the statement with CONFIG.skipStatements (a key that appears in it, with a reason and an until date). Otherwise it was made outside the migrations: define it in a base schema file listed first in CONFIG.sources.`
} else if (cls === 'real' && file.command) {
fix = `Postgres refuses the SQL that ${file.command} printed, at line ${line} of its output. Run it by hand to see the statement. If it is a real error, fix the schema source.`
} else if (cls === 'real' && code === '42P13' && /return type/i.test(String(e.message))) {
const fn = /\bcreate\s+or\s+replace\s+function\s+([\w."$]+)/i.exec(st.text)?.[1]
fix = `create or replace cannot change what a function returns, and Postgres refuses it here too. Put drop function if exists ${fn ?? '<name>'}(<argument types>) before it, or give the function a new name.`
} else if (cls === 'real' && code.startsWith('23')) {
fix = `This statement puts rows into the database, and the constraint refuses them on a fresh one. The migration relies on rows that exist only in production. Guard the insert, or remove it from the migration.`
} else fix = LOAD_CLASSES[cls].fix
return {
code: 'LOAD-FAILED',
object: where,
message: `${code} ${e.message}`.trim() + (!hasPosition && e.where ? `\n${e.where}` : ''),
fix,
class: cls,
sqlstate: code || null,
file: file.path,
line,
statement: st.text.replace(/\s+/g, ' ').slice(0, 160),
...(by ? { by } : {}),
}
})
// A flood of 54001 is one cause. Keep the first and say how many followed.
const stack = problems.filter((p) => p.sqlstate === '54001')
const kept = stack.length > 1 ? problems.filter((p) => p.sqlstate !== '54001' || p === stack[0]) : problems
if (stack.length > 1) stack[0].message += `\n${stack.length - 1} more statements failed the same way and are not listed.`
const first = kept.find((p) => p.class !== 'cascade') ?? kept[0]
const rest = { ...first }
delete rest.code
return new CannotCheck('LOAD-FAILED', { ...rest, problems: kept })
}
// ----------------------------------------------------------------- SQLITE ---
const NODE_SQLITE = [22, 13]
/**
* Import node:sqlite, or refuse with the fix. It needs Node 22.13 or later
* (earlier 22.x had it behind a flag). It prints an ExperimentalWarning when
* imported; the child is started with --disable-warning for that, and an
* in-process caller (the tests) has the one warning dropped here.
*/
export async function loadSqlite({ nodeVersion = process.versions.node } = {}) {
const [major, minor] = nodeVersion.split('.').map(Number)
const unavailable = (why) =>
new CannotCheck('SQLITE-UNAVAILABLE', {
object: 'node:sqlite',
message: `node:sqlite is not available (${why}).`,
fix: `The SQLite loader needs Node ${NODE_SQLITE.join('.')} or later; this is ${nodeVersion}. Upgrade Node.`,
})
if (major < NODE_SQLITE[0] || (major === NODE_SQLITE[0] && minor < NODE_SQLITE[1])) throw unavailable(`Node ${nodeVersion}`)
const emit = process.emitWarning
process.emitWarning = (warning, ...rest) => {
const type = typeof rest[0] === 'string' ? rest[0] : rest[0]?.type
if (type === 'ExperimentalWarning' && /SQLite/i.test(String(warning))) return
return emit.call(process, warning, ...rest)
}
try {
return await import('node:sqlite')
} catch (e) {
throw unavailable(e.code ?? e.message)
} finally {
process.emitWarning = emit
}
}
/** The SQL files as they will run. SQLite needs none of the Postgres rewrites. */
export function prepareSqlite(inputs) {
return {
files: inputs.map((i) => ({ path: i.path, text: upSection(i.text, i.sections), ...(i.late ? { late: true } : {}), ...(i.command ? { command: i.command } : {}) })),
extensions: [],
stubbed: [],
rewritten: [],
}
}
/** CONFIG.preamble, then each file in order. SQLite gives no line for an error, so the file is named. */
export function loadSqliteSchema(db, prepared, config, clock = makeClock()) {
if (config.preamble.length) {
clock.phase('preamble')
for (const [i, entry] of config.preamble.entries()) {
try {
db.exec(entry.sql)
} catch (e) {
throw new CannotCheck('CONFIG-INVALID', {
object: `preamble[${i}]`,
message: e.message,
fix: `Fix the SQL in CONFIG.preamble[${i}]. It runs before the first migration.`,
})
}
}
}
clock.phase('migrate')
for (const file of prepared.files) {
try {
db.exec(file.text)
} catch (e) {
throw new CannotCheck('LOAD-FAILED', {
object: file.path,
message: e.message,
fix: file.command
? `SQLite could not run the SQL that ${file.command} printed. Run it by hand to see the statement. If it is valid on your database, define what it needs in CONFIG.preamble with a reason; otherwise fix the schema source.`
: 'SQLite could not run this statement. If it is valid on your database and creates or uses something the checks do not read, define the missing object in CONFIG.preamble with a reason. If it is a real error, fix the migration.',
file: file.path,
line: null,
})
}
}
}
// SQLite's own rules for a declared type's affinity (https://sqlite.org/datatype3.html, 3.1), in order.
const affinityOf = (declared) => {
const t = String(declared ?? '').toUpperCase()
if (!t.trim()) return 'NONE'
if (t.includes('INT')) return 'INTEGER'
if (/CHAR|CLOB|TEXT/.test(t)) return 'TEXT'
if (t.includes('BLOB')) return 'BLOB'
if (/REAL|FLOA|DOUB/.test(t)) return 'REAL'
return 'NUMERIC'
}
const FAMILY_BY_AFFINITY = { INTEGER: 'num', REAL: 'num', TEXT: 'text', BLOB: 'bytes', NUMERIC: 'num', NONE: 'other' }
/**
* Read a SQLite database into the snapshot's shape. Tables are keyed
* public.<name>, although SQLite has no schemas, so that a contract's
* `table: "invoices"` and CONFIG.ignoreTables read the same as on Postgres.
* `rls` is null: SQLite has no row security. Each column also carries its
* `affinity`, which the loose type check reads.
*/
export function introspectSqlite(db) {
const all = (sql) => db.prepare(sql).all()
const q = (name) => `"${String(name).replace(/"/g, '""')}"`
const tables = {}
const objects = all("select name, type from sqlite_schema where type in ('table', 'view') and name not like 'sqlite\\_%' escape '\\' order by name")
for (const o of objects) {
const view = o.type === 'view'
const columns = {}
const pk = []
for (const c of all(`pragma table_xinfo(${q(o.name)})`)) {
if (c.hidden === 1) continue
const affinity = affinityOf(c.type)
// A primary key is required whether or not it says NOT NULL: INTEGER PRIMARY KEY never says so.
columns[c.name] = {
type: c.type || 'any',
family: FAMILY_BY_AFFINITY[affinity],
affinity,
nullable: c.notnull === 0 && c.pk === 0,
default: c.hidden >= 2 ? null : (c.dflt_value ?? null),
identity: null,
generated: c.hidden >= 2 ? 'generated' : null,
array: false,
enum: null,
}
if (c.pk > 0) pk[c.pk - 1] = c.name
}
const foreignKeys = []
const fks = new Map()
for (const f of all(`pragma foreign_key_list(${q(o.name)})`)) {
if (!fks.has(f.id)) fks.set(f.id, { table: f.table, from: [], to: [] })
const fk = fks.get(f.id)
fk.from[f.seq] = f.from
fk.to[f.seq] = f.to
}
for (const fk of fks.values()) {
foreignKeys.push({ name: `${o.name}_${fk.from.join('_')}_fkey`, columns: fk.from, references: `public.${fk.table}`, referencedColumns: fk.to })
}
const unique = []
for (const i of view ? [] : all(`pragma index_list(${q(o.name)})`)) {
if (!i.unique || i.partial || i.origin === 'pk') continue
const cols = all(`pragma index_info(${q(i.name)})`).map((c) => c.name)
// An expression index has no column name for that key, so it says nothing about a column.
if (cols.length && cols.every((c) => c !== null)) unique.push(cols)
}
tables[`public.${o.name}`] = {
kind: view ? 'view' : 'table',
owner: null,
columns,
primaryKey: pk.filter(Boolean),
foreignKeys,
unique,
checks: [],
rls: null,
policies: [],
grants: {},
securityInvoker: view ? false : null,
dependsOn: [],
}
}
// A foreign key with no column list points at the parent's primary key.
for (const t of Object.values(tables)) {
for (const fk of t.foreignKeys) {
fk.referencedColumns = fk.referencedColumns.map((c, i) => c ?? tables[fk.references]?.primaryKey[i] ?? null).filter((c) => c !== null)
}
}
return { tables, enums: {}, functions: [] }
}
// -------------------------------------------------------------- SNAPSHOT ---
// Relations the snapshot holds: every table, view and friends outside the
// platform's schemas, partitions folded into their parent, plus auth.users.
const RELS = `with rels as (
select c.oid, n.nspname, c.relname from pg_class c join pg_namespace n on n.oid = c.relnamespace
where c.relkind in ('r', 'v', 'm', 'p', 'f') and not c.relispartition
and ((n.nspname !~ '^pg_' and n.nspname <> all ($1::text[])) or (n.nspname = 'auth' and c.relname = 'users'))
)`
const FAMILY_BY_CATEGORY = { B: 'bool', N: 'num', S: 'text', D: 'time', T: 'time' }
const FAMILY_BY_NAME = { uuid: 'uuid', json: 'json', jsonb: 'json', bytea: 'bytes' }
/**
* Read the catalog into the snapshot's tables, enums and functions. Only
* pg_catalog is read, so a migration cannot shadow what this sees.
*/
export async function introspect(db, config) {
const rows = async (sql, params = [PLATFORM_SCHEMAS]) => (await db.query(sql, params)).rows
const roles = ['anon', 'authenticated', 'service_role', config.actAs?.role].filter(Boolean)
const execRoles = ['anon', 'authenticated', config.actAs?.role].filter(Boolean)
const rels = await rows(`${RELS}
select c.oid::int as oid, n.nspname as schema, c.relname as name, c.relkind as kind, pg_get_userbyid(c.relowner) as owner,
c.relrowsecurity as rls, c.relforcerowsecurity as forced, c.reloptions as options
from rels r join pg_class c on c.oid = r.oid join pg_namespace n on n.oid = c.relnamespace
order by n.nspname, c.relname`)
const cols = await rows(`${RELS}
select a.attrelid::int as rel, a.attname as name, format_type(a.atttypid, a.atttypmod) as type, a.atttypid::int as typid,
a.attnotnull as notnull, a.attidentity as identity, a.attgenerated as generated, pg_get_expr(d.adbin, d.adrelid) as expr
from pg_attribute a left join pg_attrdef d on d.adrelid = a.attrelid and d.adnum = a.attnum
where a.attrelid in (select oid from rels) and a.attnum > 0 and not a.attisdropped
order by a.attrelid, a.attnum`)
const types = new Map(
(await rows('select oid::int as oid, typname as name, typtype, typcategory as category, typbasetype::int as base, typelem::int as elem from pg_type', []))
.map((t) => [t.oid, t]),
)
const enumRows = await rows(
`select e.enumtypid::int as oid, n.nspname as schema, t.typname as name, array_agg(e.enumlabel::text order by e.enumsortorder) as labels
from pg_enum e join pg_type t on t.oid = e.enumtypid join pg_namespace n on n.oid = t.typnamespace
group by 1, 2, 3 order by 2, 3`,
[],
)
const labelsOf = new Map(enumRows.map((e) => [e.oid, e.labels]))
const cons = await rows(`${RELS}
select con.conrelid::int as rel, con.conname as name, con.contype as type, pg_get_constraintdef(con.oid) as def,
array(select a.attname::text from unnest(con.conkey) with ordinality k(n, i) join pg_attribute a on a.attrelid = con.conrelid and a.attnum = k.n order by k.i) as cols,
fn.nspname as ref_schema, fc.relname as ref_name,
array(select a.attname::text from unnest(con.confkey) with ordinality k(n, i) join pg_attribute a on a.attrelid = con.confrelid and a.attnum = k.n order by k.i) as ref_cols
from pg_constraint con left join pg_class fc on fc.oid = con.confrelid left join pg_namespace fn on fn.oid = fc.relnamespace
where con.conrelid in (select oid from rels) and con.contype in ('p', 'f', 'u', 'c')
order by con.conrelid, con.conname`)
const uniqueIndexes = await rows(`${RELS}
select i.indrelid::int as rel,
array(select a.attname::text from unnest((i.indkey::int2[])[0:i.indnkeyatts - 1]) with ordinality k(n, o) join pg_attribute a on a.attrelid = i.indrelid and a.attnum = k.n order by k.o) as cols
from pg_index i
where i.indisunique and not i.indisprimary and i.indrelid in (select oid from rels)
and not (0 = any (i.indkey::int2[]))
and not exists (select 1 from pg_constraint con where con.conindid = i.indexrelid)`)
const policies = await rows(`${RELS}
select p.polrelid::int as rel, p.polname as name, p.polcmd as cmd, p.polpermissive as permissive,
array(select case when r = 0 then 'public' else pg_get_userbyid(r)::text end from unnest(p.polroles) r) as roles,
pg_get_expr(p.polqual, p.polrelid) as "using", pg_get_expr(p.polwithcheck, p.polrelid) as "check"
from pg_policy p where p.polrelid in (select oid from rels)
order by p.polrelid, p.polname`)
const grants = await rows(`${RELS}
select r.oid::int as rel, ro.rolname as role,
array_remove(array[
case when has_table_privilege(ro.oid, r.oid, 'DELETE') then 'DELETE' end,
case when has_table_privilege(ro.oid, r.oid, 'INSERT') then 'INSERT' end,
case when has_table_privilege(ro.oid, r.oid, 'SELECT') then 'SELECT' end,
case when has_table_privilege(ro.oid, r.oid, 'UPDATE') then 'UPDATE' end
], null) as privileges
from rels r cross join pg_roles ro where ro.rolname = any ($2::text[])
order by ro.rolname`, [PLATFORM_SCHEMAS, roles])
const deps = await rows(`${RELS}
select distinct rw.ev_class::int as rel, n.nspname as schema, c.relname as name
from pg_rewrite rw join pg_depend d on d.classid = 'pg_rewrite'::regclass and d.objid = rw.oid and d.refclassid = 'pg_class'::regclass
join pg_class c on c.oid = d.refobjid join pg_namespace n on n.oid = c.relnamespace
where rw.ev_class in (select oid from rels) and d.refobjid <> rw.ev_class and c.relkind in ('r', 'v', 'm', 'p', 'f')
order by 1, 2, 3`)
const functions = await rows(
`select n.nspname as schema, p.proname as name, pg_get_function_identity_arguments(p.oid) as args, p.prosecdef as definer,
array(select ro.rolname::text from pg_roles ro where ro.rolname = any ($2::text[]) and has_function_privilege(ro.oid, p.oid, 'EXECUTE') order by ro.rolname) as exec
from pg_proc p join pg_namespace n on n.oid = p.pronamespace
where n.nspname !~ '^pg_' and n.nspname <> all ($1::text[]) and p.prokind in ('f', 'p')
and not exists (select 1 from pg_depend d where d.classid = 'pg_proc'::regclass and d.objid = p.oid and d.deptype = 'e')
order by 1, 2, 3`,
[PLATFORM_SCHEMAS, execRoles],
)
// A column's family comes from its base type, after domains and enums are
// resolved. An array keeps its element's family.
const describe = (typid) => {
let t = types.get(typid)
let array = false
for (let guard = 0; t && guard < 20; guard++) {
if (t.typtype === 'd') t = types.get(t.base)
else if (t.category === 'A' && t.elem) {
array = true
t = types.get(t.elem)
} else break
}
if (!t) return { family: 'other', array, enum: null }
if (t.typtype === 'e') return { family: 'text', array, enum: labelsOf.get(t.oid) ?? [] }
return { family: FAMILY_BY_NAME[t.name] ?? FAMILY_BY_CATEGORY[t.category] ?? 'other', array, enum: null }
}
const tables = {}
const keyOf = new Map()
for (const r of rels) {
const key = `${r.schema}.${r.name}`
keyOf.set(r.oid, key)
const kind = KINDS[r.kind]
const isView = kind === 'view'
const option = (r.options ?? []).map((o) => o.split('=')).find(([k]) => k === 'security_invoker')
tables[key] = {
kind,
owner: r.owner,
...(r.schema === 'auth' ? { platform: true } : {}),
columns: {},
primaryKey: [],
foreignKeys: [],
unique: [],
checks: [],
rls: NULLABLE_KNOWN.has(kind) ? { enabled: r.rls, forced: r.forced } : null,
policies: [],
grants: {},
securityInvoker: isView ? (option ? ['true', 'on', '1', 'yes'].includes(option[1]?.toLowerCase()) : false) : null,
dependsOn: [],
}
}
for (const c of cols) {
const t = tables[keyOf.get(c.rel)]
const d = describe(c.typid)
t.columns[c.name] = {
type: c.type,
family: d.family,
nullable: !c.notnull,
default: c.generated ? null : (c.expr ?? null),
identity: c.identity === 'a' ? 'always' : c.identity === 'd' ? 'by default' : null,
generated: c.generated ? (c.expr ?? c.generated) : null,
array: d.array,
enum: d.enum,
}
}
for (const c of cons) {
const t = tables[keyOf.get(c.rel)]
if (c.type === 'p') t.primaryKey = c.cols
else if (c.type === 'u') t.unique.push(c.cols)
else if (c.type === 'c') t.checks.push({ name: c.name, def: c.def })
else if (c.type === 'f') {
t.foreignKeys.push({ name: c.name, columns: c.cols, references: `${c.ref_schema}.${c.ref_name}`, referencedColumns: c.ref_cols })
}
}
for (const u of uniqueIndexes) {
const t = tables[keyOf.get(u.rel)]
if (u.cols.length && !t.unique.some((set) => set.join('\0') === u.cols.join('\0'))) t.unique.push(u.cols)
}
const COMMANDS = { r: 'SELECT', a: 'INSERT', w: 'UPDATE', d: 'DELETE', '*': 'ALL' }
for (const p of policies) {
tables[keyOf.get(p.rel)].policies.push({
name: p.name,
command: COMMANDS[p.cmd],
permissive: p.permissive,
roles: [...p.roles].sort(),
using: p.using ?? null,
check: p.check ?? null,
})
}
for (const g of grants) {
if (g.privileges.length) tables[keyOf.get(g.rel)].grants[g.role] = g.privileges
}
for (const d of deps) tables[keyOf.get(d.rel)].dependsOn.push(`${d.schema}.${d.name}`)
const enums = {}
for (const e of enumRows) if (!e.schema.startsWith('pg_') && e.schema !== 'information_schema') enums[`${e.schema}.${e.name}`] = e.labels
return {
tables,
enums,
functions: functions.map((f) => ({ name: `${f.schema}.${f.name}`, args: f.args, securityDefiner: f.definer, executableBy: f.exec })),
}
}
// ----------------------------------------------------------------- LINTS ---
// ntent's own SQL, written from each Splinter lint's documented intent and
// never copied: supabase/splinter has no licence (checked at fccca4b). Each
// query names the lint it is modelled on. Scope is CONFIG.exposedSchemas, the
// schemas the API serves.
/**
* The roles an API request runs as. Supabase: anon and authenticated.
* Postgres: the role the app connects as. The lints read these only, so the
* snapshot key (which holds actAs.role) covers them.
*/
const lintRoles = (config) => (config.platform === 'supabase' ? ['anon', 'authenticated'] : [config.actAs.role])
// Every relation a view in an exposed schema reads, through other views.
const VIEW_READS = `with recursive reads (view, rel) as (
select rw.ev_class, d.refobjid
from pg_rewrite rw
join pg_class c on c.oid = rw.ev_class join pg_namespace n on n.oid = c.relnamespace
join pg_depend d on d.classid = 'pg_rewrite'::regclass and d.objid = rw.oid and d.refclassid = 'pg_class'::regclass
where n.nspname = any ($1::text[]) and d.refobjid <> rw.ev_class
union
select r.view, d.refobjid
from reads r
join pg_rewrite rw on rw.ev_class = r.rel
join pg_depend d on d.classid = 'pg_rewrite'::regclass and d.objid = rw.oid and d.refclassid = 'pg_class'::regclass
where d.refobjid <> rw.ev_class
)`
const READABLE = `exists (select 1 from pg_roles ro where ro.rolname = any ($2::text[]) and has_table_privilege(ro.oid, c.oid, 'SELECT'))`
const LINT_SQL = {
// Splinter 0013 rls_disabled_in_public, and 0007 policy_exists_rls_disabled
// when the table has policies. Partitions count: the API reads each one
// directly, and the parent's RLS does not cover that.
rlsOff: `select n.nspname as schema, c.relname as object,
exists (select 1 from pg_policy p where p.polrelid = c.oid) as "hasPolicies",
array(select ro.rolname::text from pg_roles ro where ro.rolname = any ($2::text[])
and (has_table_privilege(ro.oid, c.oid, 'SELECT, INSERT, UPDATE, DELETE, TRUNCATE')
or has_any_column_privilege(ro.oid, c.oid, 'SELECT, INSERT, UPDATE')) order by 1) as grants
from pg_class c join pg_namespace n on n.oid = c.relnamespace
where c.relkind in ('r', 'p') and n.nspname = any ($1::text[]) and not c.relrowsecurity
order by 1, 2`,
// Splinter 0008 rls_enabled_no_policy.
noPolicy: `select n.nspname as schema, c.relname as object
from pg_class c join pg_namespace n on n.oid = c.relnamespace
where c.relkind in ('r', 'p') and n.nspname = any ($1::text[]) and c.relrowsecurity
and not exists (select 1 from pg_policy p where p.polrelid = c.oid)
order by 1, 2`,
// Splinter 0002 auth_users_exposed: a view the API serves that reads
// auth.users, directly or through another view.
authUsers: `${VIEW_READS}
select n.nspname as schema, c.relname as object, case c.relkind when 'm' then 'materialized view' else 'view' end as type
from pg_class c join pg_namespace n on n.oid = c.relnamespace
where c.relkind in ('v', 'm') and n.nspname = any ($1::text[])
and exists (select 1 from reads r where r.view = c.oid and r.rel = to_regclass('auth.users'))
and ${READABLE}
order by 1, 2`,
// Splinter 0010 security_definer_view: a view without security_invoker runs
// as its owner, so the RLS of what it reads does not apply to the caller.
definerViews: `${VIEW_READS}
select n.nspname as schema, c.relname as object,
array(select rn.nspname || '.' || rc.relname from reads r join pg_class rc on rc.oid = r.rel join pg_namespace rn on rn.oid = rc.relnamespace
where r.view = c.oid and rc.relkind in ('r', 'p', 'f', 'm') order by 1) as reads,
array(select rn.nspname || '.' || rc.relname from reads r join pg_class rc on rc.oid = r.rel join pg_namespace rn on rn.oid = rc.relnamespace
where r.view = c.oid and rc.relkind in ('r', 'p') and rc.relrowsecurity order by 1) as "readsRls"
from pg_class c join pg_namespace n on n.oid = c.relnamespace
where c.relkind = 'v' and n.nspname = any ($1::text[])
and not exists (select 1 from unnest(coalesce(c.reloptions, '{}'::text[])) o
where lower(o) in ('security_invoker=true', 'security_invoker=on', 'security_invoker=1', 'security_invoker=yes'))
and ${READABLE}
order by 1, 2`,
// Every policy in an exposed schema. Splinter 0015, 0024 and ntent's
// SCOPED-READ-TRUE are read from these.
policies: `select n.nspname as schema, c.relname as object, p.polname as policy, p.polcmd as cmd, p.polpermissive as permissive,
array(select case when r = 0 then 'public' else pg_get_userbyid(r)::text end from unnest(p.polroles) r order by 1) as roles,
pg_get_expr(p.polqual, p.polrelid) as "using", pg_get_expr(p.polwithcheck, p.polrelid) as "check"
from pg_policy p join pg_class c on c.oid = p.polrelid join pg_namespace n on n.oid = c.relnamespace
where n.nspname = any ($1::text[])
order by 1, 2, 3`,
// Splinter 0028 and 0029: a security definer function the API roles can
// call over RPC. Trigger functions cannot be called that way.
definerFunctions: `select n.nspname as schema, p.proname as object, pg_get_function_identity_arguments(p.oid) as args,
pg_get_function_result(p.oid) as returns, p.prosrc as src,
array(select ro.rolname::text from pg_roles ro where ro.rolname = any ($2::text[]) and has_function_privilege(ro.oid, p.oid, 'EXECUTE') order by 1) as "executableBy"
from pg_proc p join pg_namespace n on n.oid = p.pronamespace
where n.nspname = any ($1::text[]) and p.prosecdef and p.prokind = 'f'
and p.prorettype not in ('trigger'::regtype, 'event_trigger'::regtype)
and exists (select 1 from pg_roles ro where ro.rolname = any ($2::text[]) and has_function_privilege(ro.oid, p.oid, 'EXECUTE'))
order by 1, 2, 3`,
// The tables a definer function body might read, with the columns that look
// like secrets. $2 is the pattern for those column names.
readableTables: `select n.nspname as schema, c.relname as object,
array(select a.attname::text from pg_attribute a where a.attrelid = c.oid and a.attnum > 0 and not a.attisdropped and a.attname ~* $2::text order by a.attnum) as secrets
from pg_class c join pg_namespace n on n.oid = c.relnamespace
where c.relkind in ('r', 'p') and ((n.nspname !~ '^pg_' and n.nspname <> all ($1::text[])) or (n.nspname = 'auth' and c.relname = 'users'))
order by 1, 2`,
// No Splinter lint. On plain Postgres the app's role may own its tables,
// and an owner skips RLS unless it is forced. Membership counts: a role
// with the owner's privileges skips it too.
notForced: `select n.nspname as schema, c.relname as object, pg_get_userbyid(c.relowner) as owner
from pg_class c join pg_namespace n on n.oid = c.relnamespace join pg_roles ro on ro.rolname = ($2::text[])[1]
where c.relkind in ('r', 'p') and n.nspname = any ($1::text[]) and c.relrowsecurity and not c.relforcerowsecurity
and pg_has_role(ro.oid, c.relowner, 'USAGE')
order by 1, 2`,
}
// Column names that read as secrets, for a definer function that returns them.
const SECRET_COLUMN = '(^|_)(secret|password|passwd|token|api_?key|private_?key|credential|encrypted)s?(_|$)'
const NOT_SECRET_COLUMN = /_(count|used|limit|at|type|id)$/
// A function body that looks at who is calling.
const CALLER_CHECK =
/\bauth\.(uid|jwt|role|email)\s*\(|\b(current_user|session_user)\b|current_setting\s*\(\s*'request\.jwt|\bis_(admin|member|owner|staff|team|org)\w*\s*\(|\bhas_(role|permission|access)\w*\s*\(|\bcheck_(access|member|permission)\w*\s*\(/i
/**
* Every plain SQL or plpgsql function outside the system schemas, by name, so
* a definer function that calls a helper can be read through it.
*/
async function helperSources(db) {
const rows = (
await db.query(
`select p.proname as name, p.prosrc as src from pg_proc p join pg_namespace n on n.oid = p.pronamespace join pg_language l on l.oid = p.prolang
where p.prokind = 'f' and n.nspname !~ '^pg_' and n.nspname <> 'information_schema' and l.lanname in ('sql', 'plpgsql')`,
)
).rows
const out = new Map()
for (const r of rows) out.set(r.name, [...(out.get(r.name) ?? []), stripComments(typeof r.src === 'string' ? r.src : '')])
return out
}
const NOT_A_CALL = new Set(['if', 'elsif', 'while', 'and', 'or', 'not', 'in', 'exists', 'select', 'from', 'where', 'values', 'return', 'returns', 'case', 'when', 'then', 'else', 'end', 'coalesce', 'count', 'sum', 'any', 'all', 'array', 'lower', 'upper', 'now', 'length', 'cast', 'on', 'as', 'using', 'with', 'set', 'into', 'raise', 'perform', 'loop', 'for', 'insert', 'update', 'delete', 'join', 'limit', 'order', 'by', 'group', 'having', 'over', 'filter', 'extract', 'unnest', 'row', 'least', 'greatest', 'nullif', 'jsonb_build_object', 'json_build_object', 'to_jsonb', 'to_json', 'jsonb_agg', 'json_agg', 'array_agg', 'string_agg', 'gen_random_uuid', 'md5', 'format'])
/**
* Does a function body look at who is calling, itself or through a helper it
* calls (one or two levels down)? `via` names the first helper that does.
*/
export function callerCheck(body, helpers, depth = 0, seen = new Set()) {
if (CALLER_CHECK.test(body)) return { checksCaller: true }
if (depth >= 2) return { checksCaller: false }
for (const m of body.matchAll(/(?<![\w$])"?([A-Za-z_][\w$]*)"?\s*\(/g)) {
const name = m[1].toLowerCase()
if (NOT_A_CALL.has(name) || seen.has(name) || !helpers.has(name)) continue
seen.add(name)
if (helpers.get(name).some((src) => callerCheck(src, helpers, depth + 1, seen).checksCaller)) return { checksCaller: true, checksCallerVia: name }
}
return { checksCaller: false }
}
/** The tables a function body names after from, join, into, update, using or table. */
const tablesReadBy = (src, schema, tables) => {
const body = stripComments(src)
const out = []
for (const t of tables) {
const name = t.object.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
const qualified = `"?${t.schema}"?\\s*\\.\\s*"?${name}"?`
const bare = t.schema === schema || t.schema === 'public' ? `|"?${name}"?` : ''
if (new RegExp(`\\b(?:from|join|into|update|using|table)\\s+(?:only\\s+)?(?:${qualified}${bare})(?![\\w"])`, 'i').test(body)) out.push(t)
}
return out
}
/** A policy expression that is the literal true. */
const isTrue = (expr) => typeof expr === 'string' && /^[\s(]*true[\s)]*$/i.test(expr)
const PUBLIC_NAME = /\b(public|everyone|anyone|anon|anonymous|published)\b/i
const PUBLIC_FLAG = '(?:is_)?(?:published|public|visible|active)'
const PUBLIC_TERM = new RegExp(
`^\\(*\\s*(?:[\\w"]+\\.)?"?(?:(${PUBLIC_FLAG})"?(?:\\s*=\\s*true|\\s+is\\s+true)?|(status)"?\\s*=\\s*'published'(?:::\\w+)?)\\s*\\)*$`,
'i',
)
/**
* Whether a SELECT policy looks public on purpose, and why (null when it
* does not). Only the policy itself is read, never the table. Three signals,
* in this order:
* 1. its USING is literally true;
* 2. its name says public, everyone, anyone, anon or published (not
* visible: "visible to team members" is a scoped policy);
* 3. its USING only tests a boolean or status column with a published-style
* name: is_published, published, is_public, public, visible, active, or
* status = 'published' (terms joined by and).
* Write policies never qualify: a public read does not excuse an open write.
* A hit does not make the finding a note. The contract still disagrees with
* the database, and only a person can say which is right.
*/
export function publicPolicyReason(policy) {
if (!policy || policy.command !== 'SELECT' || policy.permissive === false) return null
if (isTrue(policy.using)) return 'it is using (true), so every row is readable'
if (PUBLIC_NAME.test(policy.name ?? '')) return `its name, "${policy.name}", says it is for everyone`
const terms = String(policy.using ?? '').split(/\band\b/i)
if (policy.using && terms.every((t) => PUBLIC_TERM.test(t.trim()))) {
const column = PUBLIC_TERM.exec(terms[0].trim()).slice(1).find(Boolean)
return `it filters only on ${column}, which reads as a published flag`
}
return null
}
/**
* The fix text for a read finding caused by a policy that looks public on
* purpose: the contract change first, then the existing fix as the other
* reading. `who` and `entity` name the matrix row; `extra` adds another way
* to say the table is public.
*/
export function withPublicHint(reason, fix, { actor, entity, visitor, extra = '' }) {
const who = visitor ? `a visitor actor (id 'visitor', or kind 'public')` : actor
const row = `{ actor: '${visitor ? 'visitor' : actor}', entity: '${entity}', can: ['read'], scope: 'all' }`
const rest = fix.charAt(0).toLowerCase() + fix.slice(1)
return `This policy looks public on purpose: ${reason}. If the table is meant to be public, change the contract: give ${who} scope 'all' on ${entity} for read (${row})${extra}. If it is not, the policy is the leak: ${rest}`
}
// What a definer function may return without handing back rows: a flag, or a whole-number count. A numeric is a value (a total), so it is not here.
const SCALAR_RETURN = /^(void|bool|boolean|trigger|smallint|integer|int|int2|int4|int8|bigint)$/i
const COMMAND = { r: 'SELECT', a: 'INSERT', w: 'UPDATE', d: 'DELETE', '*': 'ALL' }
// Roles only a server uses. A policy written to one of them is not open to the API.
const SERVER_ROLES = new Set(['service_role', 'postgres', 'supabase_admin', 'supabase_auth_admin', 'supabase_storage_admin', 'dashboard_user'])
/** True when only the app's own server connects, so RLS-OFF is a note. 'auto' follows the platform. */
export const serverOnly = (config) => {
const access = config.clientAccess ?? 'auto'
return access === 'server-only' || (access === 'auto' && config.platform === 'postgres')
}
/**
* The RLS lints (section 6), as raw rows. Whether a row is a finding depends
* on the contract, so it is decided at report time (accessGate), and a
* contract edit never invalidates the snapshot.
*/
export async function runLints(db, config, _found) {
const params = [config.exposedSchemas, lintRoles(config)]
// Each query gets the parameters it names: PGlite refuses extras.
const q = async (sql) => (await db.query(sql, sql.includes('$2') ? params : params.slice(0, 1))).rows
const lints = []
const row = (code, modelledOn, schema, object, type, extra = {}) => lints.push({ code, modelledOn, schema, object, type, ...extra })
for (const r of await q(LINT_SQL.rlsOff)) {
if (r.hasPolicies) row('POLICY-BUT-RLS-OFF', '0007', r.schema, r.object, 'table', { grants: r.grants })
else row('RLS-OFF', '0013', r.schema, r.object, 'table', { grants: r.grants })
}
for (const r of await q(LINT_SQL.noPolicy)) row('RLS-NO-POLICY', '0008', r.schema, r.object, 'table')
for (const r of await q(LINT_SQL.authUsers)) row('AUTH-USERS-EXPOSED', '0002', r.schema, r.object, r.type)
for (const r of await q(LINT_SQL.definerViews)) row('DEFINER-VIEW', '0010', r.schema, r.object, 'view', { reads: r.reads, readsRls: r.readsRls })
for (const p of await q(LINT_SQL.policies)) {
const command = COMMAND[p.cmd]
const at = { policy: p.policy, command, roles: p.roles, permissive: p.permissive }
if (/user_metadata|raw_user_meta_data/.test(`${p.using ?? ''} ${p.check ?? ''}`)) {
row('USER-METADATA-IN-POLICY', '0015', p.schema, p.object, 'policy', at)
}
if (!p.permissive) continue
// Splinter 0024 skips SELECT on purpose. An UPDATE with no WITH CHECK
// uses its USING for both.
const alwaysTrue =
(command === 'INSERT' && isTrue(p.check)) ||
(['UPDATE', 'ALL'].includes(command) && (isTrue(p.using) || isTrue(p.check))) ||
(command === 'DELETE' && isTrue(p.using))
if (alwaysTrue) row('POLICY-ALWAYS-TRUE', '0024', p.schema, p.object, 'policy', at)
// D18. A SELECT policy with no USING shows no rows (verified on PGlite
// 0.5.8), so only the literal true is a read of every row.
if (['SELECT', 'ALL'].includes(command) && isTrue(p.using)) row('SCOPED-READ-TRUE', null, p.schema, p.object, 'policy', at)
}
const definers = await q(LINT_SQL.definerFunctions)
const helpers = definers.length ? await helperSources(db) : new Map()
const readable = definers.length ? (await db.query(LINT_SQL.readableTables, [PLATFORM_SCHEMAS, SECRET_COLUMN])).rows : []
for (const r of definers) {
const src = typeof r.src === 'string' ? r.src : ''
const reads = tablesReadBy(src, r.schema, readable).map((t) => ({
table: `${t.schema}.${t.object}`,
secrets: t.secrets.filter((c) => !NOT_SECRET_COLUMN.test(c)),
}))
row('DEFINER-FUNCTION-EXECUTABLE', '0028', r.schema, r.object, 'function', {
args: r.args,
executableBy: r.executableBy,
returns: r.returns,
bodyKnown: src.trim() !== '',
reads,
...callerCheck(stripComments(src), helpers),
})
}
if (config.platform === 'postgres') {
for (const r of await q(LINT_SQL.notForced)) row('RLS-NOT-FORCED', null, r.schema, r.object, 'table', { owner: r.owner })
}
return lints
}
/**
* The snapshot every check reads. `--print-snapshot` prints it. For SQL files
* on Postgres the shape is the one section 2 gives. A command source changes
* `source.kind` (and adds `loader.via`); SQLite changes `source.platform` and
* `loader.name`.
*/
export function makeSnapshot({ found, lints = [], prepared, config, inputsHash, pgliteVersion, dialect = 'postgres', sourceKind = 'sql', via = [] }) {
const sqlite = dialect === 'sqlite'
return {
format: SNAPSHOT_FORMAT,
source: { kind: sourceKind, platform: sqlite ? 'sqlite' : config.platform, inputs: prepared.files.map((f) => f.path), hash: `sha256:${inputsHash}` },
loader: {
name: sqlite ? 'node:sqlite' : 'pglite',
version: pgliteVersion,
shim: !sqlite && config.platform === 'supabase' ? SHIM_VERSION : null,
extensions: prepared.extensions,
stubbed: prepared.stubbed,
rewritten: prepared.rewritten,
...(prepared.empty?.length ? { empty: prepared.empty } : {}),
...(prepared.skipped?.length ? { skipped: prepared.skipped } : {}),
...(via.length ? { via } : {}),
},
tables: found.tables,
enums: found.enums,
functions: found.functions,
lints,
notes: [],
}
}
// ------------------------------------------------------------- DENY PLAN ---
// Two users, always the same ids, so a failure names the same rows every run.
export const USERS = { a: '00000000-0000-4000-8000-00000000000a', b: '00000000-0000-4000-8000-00000000000b' }
const EMAILS = { a: 'a@ntent.test', b: 'b@ntent.test', c: 'c@ntent.test' }
// A third user, made only when a CHECK needs two different rows of one table (requester <> addressee).
const BYSTANDER = '00000000-0000-4000-8000-00000000000c'
const list = (v) => (Array.isArray(v) ? v : [])
/** An identifier in generated SQL, always quoted, so a table called "order" still works. */
const qi = (s) => `"${String(s).replace(/"/g, '""')}"`
/** 'public.invoices' as SQL. */
const qt = (key) => {
const dot = key.indexOf('.')
return `${qi(key.slice(0, dot))}.${qi(key.slice(dot + 1))}`
}
/** Fill {user}, {tenant} and {n} in every string inside a value. */
const fillIn = (value, vars) => {
if (typeof value === 'string') {
return value.replaceAll('{user}', vars.user ?? '').replaceAll('{tenant}', vars.tenant ?? '').replaceAll('{n}', String(vars.n ?? ''))
}
if (Array.isArray(value)) return value.map((v) => fillIn(v, vars))
if (isObject(value)) return Object.fromEntries(Object.entries(value).map(([k, v]) => [k, fillIn(v, vars)]))
return value
}
const deepMerge = (a, b) => {
const out = { ...a }
for (const [k, v] of Object.entries(b ?? {})) out[k] = isObject(v) && isObject(out[k]) ? deepMerge(out[k], v) : v
return out
}
/** The owning field's column per mapped table, for every entity that names an owner. */
const ownerColumns = (snapshot, contract, config) => {
const out = new Map()
for (const e of list(contract?.entities)) {
if (!isObject(e) || e.table === undefined || !e.owner) continue
const field = list(e.fields).map(fieldOf).find((f) => isObject(f) && f.name === e.owner)
const key = qualify(e.table)
const col = field ? columnOf(field, config.columnNames) : null
if (col && snapshot.tables[key]?.columns[col]) out.set(key, col)
}
return out
}
/**
* How a table's rows reach a tenant or an owner (section 7, "Seeding" step 4):
* the tenancy column on the row; else ref fields, up to 3 hops, to a table
* that has it; the entity's owner; or the table is the tenant or membership
* table itself. null when nothing links it, which is NO-TENANT-PATH.
*/
function tenantPath({ snapshot, entity, entities, tenancy, membership, config, owners }) {
const table = qualify(entity.table)
const t = snapshot.tables[table]
const column = nonEmpty(tenancy.column) ? tenancy.column : null
const owner = owners.get(table) ?? null
const direct = column && t.columns[column] ? column : null
// Up to 3 hops along ref fields to a table that `goal` accepts.
const findChain = (e, depth, seen, goal, via = null) => {
if (depth >= 3) return null
const from = qualify(e.table)
const ft = snapshot.tables[from]
for (const raw of list(e.fields)) {
const f = fieldOf(raw)
if (!isObject(f) || !f.ref) continue
if (via && f.name !== via) continue
const parent = entities.get(f.ref)
if (!parent || parent.table === undefined || seen.has(parent.id)) continue
const to = qualify(parent.table)
const pt = snapshot.tables[to]
const col = columnOf(f, config.columnNames)
if (!pt || !ft.columns[col]) continue
const fk = ft.foreignKeys.find((k) => k.columns.length === 1 && k.columns[0] === col && k.references === to)
const parentColumn = fk?.referencedColumns[0] ?? (pt.primaryKey.length === 1 ? pt.primaryKey[0] : null)
if (!parentColumn) continue
const hop = { table: from, column: col, parent: to, parentColumn }
if (goal(pt, to)) return [hop]
const rest = findChain(parent, depth + 1, new Set([...seen, e.id]), goal)
if (rest) return [hop, ...rest]
}
return null
}
const tenantChain = !direct && column ? findChain(entity, 0, new Set([entity.id]), (pt) => !!pt.columns[column]) : null
// No tenancy column and no owner on the row itself: the owner may sit on a
// parent row, reached the same way (entities[].owner_via picks the ref).
const ownerChain =
!direct && !tenantChain && !owner
? findChain(entity, 0, new Set([entity.id]), (pt, to) => owners.has(to), nonEmpty(entity.owner_via) ? entity.owner_via : null)
: null
const chain = tenantChain ?? ownerChain
const chainOwner = ownerChain ? owners.get(ownerChain.at(-1).parent) : null
// The tenant table: the foreign key target of the tenancy column on the
// table that carries it; else the table of tenancy.scope_entity; else none,
// and the tenant id is a bare value.
const holder = direct ? table : tenantChain ? tenantChain.at(-1).parent : null
const fk = holder ? snapshot.tables[holder].foreignKeys.find((k) => k.columns.length === 1 && k.columns[0] === column) : null
let tenantTable = fk && snapshot.tables[fk.references] ? fk.references : null
let tenantKey = tenantTable ? fk.referencedColumns[0] : null
if (!tenantTable) {
const scope = entities.get(tenancy.scope_entity)
const key = scope?.table !== undefined ? qualify(scope.table) : null
const pk = key ? snapshot.tables[key]?.primaryKey : null
if (pk?.length === 1) {
tenantTable = key
tenantKey = pk[0]
}
}
const self = table === tenantTable ? 'tenant' : membership && table === membership.table ? 'membership' : null
if (!direct && !chain && !owner && !self) return null
// The columns that say whose a row is. P5 copies them from B's row, and
// counts rows that carry any of B's values.
const pathColumns = [direct, owner, chain?.[0].column]
if (self === 'membership') pathColumns.push(membership.user_column, membership.tenant_column)
if (self === 'tenant') pathColumns.push(membership?.table === table ? membership.user_column : tenantKey)
return {
column: direct,
chain,
chainOwner,
owner,
tenantTable,
tenantKey,
self,
pathColumns: [...new Set(pathColumns.filter(Boolean))],
}
}
/**
* Which matrix rows are deny-tested, and how each table reaches a tenant. Reads
* the snapshot and the contract, never the database. Two rows with the same
* actor and entity are one test.
*/
export function planDenyTests(snapshot, contract, config) {
const tests = []
const notes = []
const problems = []
const access = isObject(contract?.access) ? contract.access : {}
const tenancy = isObject(access.tenancy) ? access.tenancy : {}
const entities = new Map(list(contract?.entities).filter((e) => isObject(e) && nonEmpty(e.id)).map((e) => [e.id, e]))
const kinds = new Map(list(contract?.actors).filter(isObject).map((a) => [a.id, a.kind]))
const owners = ownerColumns(snapshot, contract, config)
const skip = (row, reason) => notes.push({ code: 'NOT-DENY-TESTED', object: `${row.actor} on ${row.entity}`, message: reason })
const shape = (object, message, fix) => problems.push({ code: 'CONTRACT-SHAPE', object, message, fix })
let membership = null
if (isObject(tenancy.membership)) {
membership = { ...tenancy.membership, table: qualify(tenancy.membership.table) }
const mt = snapshot.tables[membership.table]
if (!mt) {
shape('access.tenancy.membership.table', `names ${membership.table}, which the migrations never create`, 'Create the table in a migration, or correct access.tenancy.membership.')
} else {
for (const k of ['user_column', 'tenant_column']) {
if (!mt.columns[membership[k]]) {
shape(`access.tenancy.membership.${k}`, `names ${membership[k]}, which ${membership.table} does not have`, 'Name the column as it is in the database.')
}
}
}
}
const groups = new Map()
for (const row of list(access.matrix)) {
if (!isObject(row)) continue
// Only system and schedule actors are skipped. An absent, "user" or
// unrecognised kind is tested: a kind nobody knows must not hide a table.
const kind = kinds.get(row.actor)
if (kind === 'system' || kind === 'schedule') {
skip(row, `${row.actor} is a ${kind} actor, and the test acts as signed-in users`)
continue
}
if (row.scope === undefined) {
skip(row, 'the row has no scope')
continue
}
if (row.scope === 'all') {
skip(row, 'scope all has no row filter')
continue
}
if (!['own', 'tenant'].includes(row.scope)) {
skip(row, `"${row.scope}" is not a scope the checks know`)
continue
}
const e = entities.get(row.entity)
if (!e) {
skip(row, `${row.entity} is not an entity`)
continue
}
if (e.table === undefined) {
skip(row, `${row.entity} is not mapped to a table`)
continue
}
const key = `${row.actor}\0${e.id}`
const group = groups.get(key) ?? { actor: row.actor, entity: e, read: false }
// `can` picks the control only (D5). No `can` at all reads as read, so
// the read probes run.
group.read ||= !Array.isArray(row.can) || row.can.includes('read')
groups.set(key, group)
}
for (const g of groups.values()) {
const table = qualify(g.entity.table)
const test = { actor: g.actor, entity: g.entity.id, table, read: g.read, path: null, problem: null }
const t = snapshot.tables[table]
const untested = `add '${table}' to CONFIG.untested with a reason and an until date.`
if (!t) {
test.problem = {
code: 'CONTRACT-SHAPE',
object: `entities.${g.entity.id}.table`,
message: `names ${table}, which the migrations never create`,
fix: `Create the table in a migration, correct entities[].table, or ${untested}`,
}
} else if (!['table', 'partitioned_table'].includes(t.kind)) {
test.problem = {
code: 'SEED-FAILED',
object: table,
message: `${table} is a ${t.kind.replace('_', ' ')}. The test seeds rows into tables, so it cannot give user B a row here.`,
fix: `Map ${g.entity.id} to the table the rows live in, or ${untested}`,
}
} else if (!t.primaryKey.length) {
test.problem = {
code: 'PROBE-ERROR',
object: table,
message: `${table} has no primary key, so the test cannot address a row.`,
fix: `Add a primary key, or ${untested}`,
}
} else {
test.path = tenantPath({ snapshot, entity: g.entity, entities, tenancy, membership, config, owners })
if (!test.path) {
test.problem = {
code: 'NO-TENANT-PATH',
object: table,
message: `${g.entity.id} -> ${table} has no tenancy column, no ref field that leads to one, no owner, and no ref field that leads to an entity with an owner, so the test cannot give user B a row.`,
fix: `Nothing links ${table} to a tenant or owner. Set access.tenancy.column on it, a ref field that leads to a table that has it, entities[].owner, or a ref field (entities[].owner_via names it) to an entity that has an owner; or ${untested}`,
}
}
}
tests.push(test)
}
return { tests, notes, problems, membership, tenancy, owners }
}
/** How a policy should read the user: auth.uid(), or the setting actAs fills with {user}. */
const userExpr = (config) =>
config.platform === 'supabase'
? 'auth.uid()'
: `current_setting('${Object.entries(config.actAs.settings ?? {}).find(([, v]) => v === '{user}')?.[0] ?? 'app.user_id'}')`
/**
* The user, for comparing with `column` of `key`. auth.uid() is a uuid, so a
* column that holds a text id (a Clerk user id) is compared with the token's
* sub claim instead.
*/
const userFor = (config, snapshot, key, column) =>
config.platform === 'supabase' && /^(text|character varying|varchar|character|citext)\b/i.test(String(snapshot.tables[key]?.columns[column]?.type ?? ''))
? "(select auth.jwt()->>'sub')"
: userExpr(config)
/**
* The condition a policy on this table should have, for the fix line: the
* tenancy column through the membership table, the ref chain, or the owner.
*/
export function tenantCondition({ table, path, snapshot, tenancy, membership, config }) {
const uid = (key, column) => userFor(config, snapshot, key, column)
const on = (key, hops) => {
if (hops?.length) {
const [hop, ...rest] = hops
return `${quoteIdent(hop.column)} in (select ${quoteIdent(hop.parentColumn)} from ${hop.parent} where ${on(hop.parent, rest)})`
}
if (path.chainOwner && !hops?.length) return `${quoteIdent(path.chainOwner)} = ${uid(key, path.chainOwner)}`
if (path.self === 'membership' || (path.self === 'tenant' && membership?.table === key)) return `${quoteIdent(membership.user_column)} = ${uid(key, membership.user_column)}`
if (path.self === 'tenant') return `${quoteIdent(path.tenantKey)} = ${uid(key, path.tenantKey)}`
if (tenancy.column && snapshot.tables[key]?.columns[tenancy.column]) {
return membership
? `${quoteIdent(tenancy.column)} in (select ${quoteIdent(membership.tenant_column)} from ${membership.table} where ${quoteIdent(membership.user_column)} = ${uid(membership.table, membership.user_column)})`
: `${quoteIdent(tenancy.column)} = ${uid(key, tenancy.column)}`
}
return `${quoteIdent(path.owner)} = ${uid(key, path.owner)}`
}
return on(table, path.column ? null : path.chain)
}
/**
* What a policy on the membership table itself should check on INSERT. The
* owner condition (user_id = auth.uid()) alone lets any user add themselves to
* any team, in any role.
*/
export function membershipInsertCondition({ membership, snapshot, config }) {
const t = snapshot.tables[membership.table]
const uid = userFor(config, snapshot, membership.table, membership.user_column)
const role = Object.keys(t?.columns ?? {}).find((c) => /^(role|roles|member_role|permission|permissions)$/i.test(c))
const admin = role ? ` and ${quoteIdent(role)} in ('owner', 'admin')` : ''
return `${quoteIdent(membership.tenant_column)} in (select ${quoteIdent(membership.tenant_column)} from ${membership.table} where ${quoteIdent(membership.user_column)} = ${uid}${admin})`
}
// ---------------------------------------------------------------- SEEDER ---
/**
* Column types, with domains resolved: what the synthesiser needs and the
* snapshot does not carry (a domain's base type and CHECK rules, a length).
*/
async function loadTypes(db) {
const types = new Map(
(await db.query('select oid::int as oid, typname as name, typtype as type, typcategory as category, typbasetype::int as base, typtypmod as typmod, typelem::int as elem from pg_type'))
.rows.map((t) => [t.oid, t]),
)
const domainChecks = new Map()
for (const c of (await db.query("select contypid::int as oid, pg_get_constraintdef(oid) as def from pg_constraint where contypid <> 0 and contype = 'c'")).rows) {
domainChecks.set(c.oid, [...(domainChecks.get(c.oid) ?? []), c.def])
}
const labels = new Map(
(await db.query('select enumtypid::int as oid, array_agg(enumlabel::text order by enumsortorder) as labels from pg_enum group by 1')).rows.map((e) => [e.oid, e.labels]),
)
const columns = new Map(
(await db.query(
`select n.nspname || '.' || c.relname as tbl, a.attname as col, a.atttypid::int as typid, a.atttypmod as typmod
from pg_attribute a join pg_class c on c.oid = a.attrelid join pg_namespace n on n.oid = c.relnamespace
where c.relkind in ('r', 'p') and a.attnum > 0 and not a.attisdropped and n.nspname !~ '^pg_' and n.nspname <> 'information_schema'`,
)).rows.map((r) => [`${r.tbl}\0${r.col}`, r]),
)
return (table, column) => {
const at = columns.get(`${table}\0${column}`)
if (!at) return null
let t = types.get(at.typid)
let typmod = at.typmod
const checks = []
for (let guard = 0; t?.type === 'd' && guard < 20; guard++) {
checks.push(...(domainChecks.get(t.oid) ?? []))
if (typmod === -1) typmod = t.typmod
t = types.get(t.base)
}
if (!t) return null
return {
name: t.name,
array: t.category === 'A' && t.elem !== 0,
labels: t.type === 'e' ? (labels.get(t.oid) ?? []) : null,
composite: t.type === 'c',
length: ['varchar', 'bpchar'].includes(t.name) && typmod > 4 ? typmod - 4 : null,
domainChecks: checks,
}
}
}
const TEXT_TYPES = new Set(['text', 'varchar', 'bpchar', 'citext', 'name'])
const INT_TYPES = new Set(['int2', 'int4', 'int8'])
const NUM_TYPES = new Set(['int2', 'int4', 'int8', 'numeric', 'float4', 'float8', 'money'])
const esc = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')
/**
* Bend a synthesised value to the CHECK rules the seeder can read (section 7,
* step 5): an IN list takes its first literal; > N takes N+1, >= N takes N,
* < N takes N-1; length(col) > N pads the text. Anything else keeps the value
* and the insert decides.
*/
function applyChecks(value, { defs, ref, kind, unique, n }) {
const col = `(?<![\\w"$])${ref}(?![\\w"$])\\)?(?:::[\\w ]+(?:\\[\\])?)?`
let lo = null
let hi = null
for (const def of defs) {
const any = new RegExp(`${col}\\s*=\\s*ANY\\s*\\(+ARRAY\\[\\s*\\(*('(?:[^']|'')*'|-?\\d+(?:\\.\\d+)?)`).exec(def)
if (any) {
value = any[1].startsWith("'") ? any[1].slice(1, -1).replace(/''/g, "'") : any[1]
continue
}
if (kind === 'num') {
for (const m of def.matchAll(new RegExp(`${col}\\s*(>=|<=|>|<)\\s*\\(?'?(-?\\d+(?:\\.\\d+)?)(?=['\\)\\s:]|$)`, 'g'))) {
const v = Number(m[2])
if (m[1] === '>=') lo = Math.max(lo ?? -Infinity, v)
else if (m[1] === '>') lo = Math.max(lo ?? -Infinity, v + 1)
else if (m[1] === '<=') hi = Math.min(hi ?? Infinity, v)
else hi = Math.min(hi ?? Infinity, v - 1)
}
}
if (kind === 'text') {
const len = new RegExp(`(?:char_length|length)\\(${col}\\)\\s*(>=|>)\\s*\\(?(\\d+)`).exec(def)
if (len) value = String(value).padEnd(Number(len[2]) + (len[1] === '>' ? 1 : 0), 'x')
}
}
if (lo !== null) value = String(unique ? lo + n : lo)
else if (hi !== null && (unique || Number(value) > hi)) value = String(unique ? hi - n : hi)
return value
}
const CHECK_COLUMN = '("(?:[^"]|"")+"|[A-Za-z_][\\w$]*)'
const COMPARES_COLUMNS = new RegExp(`\\(*${CHECK_COLUMN}\\)*(?:::[A-Za-z ]+)?\\s*(>=|<=|>|<)\\s*\\(*${CHECK_COLUMN}`, 'g')
const DIFFERS = new RegExp(`\\(*${CHECK_COLUMN}\\)*(?:::[A-Za-z ]+)?\\s*(?:<>|!=)\\s*\\(*${CHECK_COLUMN}`, 'g')
const DAY = 86400000
/**
* The foreign key columns a CHECK says must differ from another column that
* points at the same table (requester <> addressee). Those need a row of that
* table that is not the principal's, so the seeder makes one for a third user.
*/
function apartColumns(tt) {
const unquote = (c) => c.replace(/^"|"$/g, '').replace(/""/g, '"')
const target = (c) => tt.foreignKeys.find((k) => k.columns.length === 1 && k.columns[0] === c)?.references
const out = new Set()
for (const check of tt.checks) {
for (const m of check.def.matchAll(DIFFERS)) {
const [left, right] = [unquote(m[1]), unquote(m[2])]
if (target(left) && target(left) === target(right)) out.add(right)
}
}
return out
}
const instant = (v) => Date.parse(String(v).replace(' ', 'T').replace(/\+00(:00)?$/, 'Z'))
/**
* Bend a row to a CHECK that compares two of its columns, like end > start
* (section 7, step 5). The larger side moves just past the smaller one when it
* does not already hold. A column CONFIG.seed or the caller fixed is left
* alone, and the insert decides.
*/
function applyCrossChecks(tt, vals, nameOf, locked) {
const unquote = (c) => c.replace(/^"|"$/g, '').replace(/""/g, '"')
for (const check of tt.checks) {
for (const m of check.def.matchAll(COMPARES_COLUMNS)) {
const [left, right] = [unquote(m[1]), unquote(m[3])]
if (!tt.columns[left] || !tt.columns[right] || vals[left] == null || vals[right] == null) continue
const [big, small, strict] = m[2][0] === '>' ? [left, right, m[2] === '>'] : [right, left, m[2] === '<']
if (locked.has(big)) continue
const name = nameOf(big)
if (NUM_TYPES.has(name)) {
const [b, s] = [Number(vals[big]), Number(vals[small])]
if (Number.isFinite(b) && Number.isFinite(s) && !(strict ? b > s : b >= s)) vals[big] = String(strict ? s + 1 : s)
} else if (['date', 'timestamp', 'timestamptz'].includes(name)) {
const [b, s] = [instant(vals[big]), instant(vals[small])]
if (Number.isFinite(b) && Number.isFinite(s) && !(strict ? b > s : b >= s)) {
const next = new Date(strict ? s + DAY : s).toISOString()
vals[big] = name === 'date' ? next.slice(0, 10) : next
}
}
}
}
}
/** Refused while seeding: exit 2 SEED-FAILED, never a pass. `detail` says what the fix text needs. */
class SeedFailure extends Error {
constructor(table, column, reason, detail = null) {
super(reason)
this.table = table
this.column = column
this.detail = detail
}
}
/** Why an insert failed while seeding, as the report says it (section 7). */
function seedFailure(table, e, vals, types, tt = null) {
const column = e.column ?? Object.keys(vals).find((c) => vals[c] != null && String(e.message).includes(`"${vals[c]}"`)) ?? null
const type = /for type (\S+)/.exec(e.message)?.[1] ?? (column ? types[column] : null) ?? 'its type'
const reason = {
23502: `${column} is NOT NULL and the seeder had no value for it`,
23503: `foreign key ${e.constraint} points at a row the seeder did not create`,
23505: `unique ${e.constraint} collided`,
23514: `check ${e.constraint} refused the synthesised value`,
'22P02': `${column ?? 'a column'}: the synthesised value is not valid for ${type}`,
22007: `${column ?? 'a column'}: the synthesised value is not valid for ${type}`,
22008: `${column ?? 'a column'}: the synthesised value is not valid for ${type}`,
P0001: `a trigger raised: ${e.message}`,
}[e.code]
let detail = null
if (e.code === '23514') {
const check = tt?.checks.find((c) => c.name === e.constraint)
if (check) detail = { check: { name: check.name, def: check.def, columns: Object.keys(tt.columns).filter((c) => new RegExp(`(?<![\\w"$])"?${esc(c)}"?(?![\\w"$])`).test(check.def)) } }
} else if (e.code === '23503') {
const fk = tt?.foreignKeys.find((k) => k.name === e.constraint)
if (fk) detail = { fk }
} else if (e.code === '23505') detail = { unique: e.constraint }
return new SeedFailure(table, column, reason ?? `${e.code} ${e.message}`, detail)
}
/** What to put in CONFIG.seed, or in the contract, for a failed seed. The default is a value for the column. */
function seedFix(e, test) {
const untested = `add '${test.table}' to CONFIG.untested with a reason and an until date`
const seed = `CONFIG.seed['${e.table}']`
const d = e.detail
if (d?.check) {
const cols = d.check.columns
const shape = cols.length ? `{ ${cols.map((c) => `${c}: '<a value that passes>'`).join(', ')} }` : '{ <column>: <value> }'
return `The seeder only reads simple checks (a column against a number, an IN list, one column against another). ${d.check.name} is ${d.check.def}. Set ${seed} = ${shape} to values it accepts, or ${untested}.`
}
if (d?.fk) {
return `${e.table}.${d.fk.columns.join(', ')} references ${d.fk.references}, and the seeder did not make that row. ${seed} can set a value, but it cannot create a parent row. If ${d.fk.references} holds the app's users, point access.tenancy.membership.user_column or the entity's owner at the column that references it, and the seeder makes the user's row first. Otherwise ${untested}.`
}
if (d?.unique && /_pkey$/.test(d.unique)) {
return `${d.unique} is the primary key of ${e.table}, and a row with the key the seeder chose was already there (usually a trigger on a parent's insert makes it) but the seeder could not find it by that key. CONFIG.seed cannot help: the key comes from the rows it links to, not from a value. Check the trigger writes the same key columns the table's foreign keys give, or ${untested}.`
}
if (d?.unique) {
return `${d.unique} needs a different value for user A's row and user B's. Give ${seed} a value with {user} or {n} in it for each column the constraint covers, or ${untested}.`
}
const column = e.column ? ` a value for ${e.column}` : ' the value it needs'
return `Give ${seed}${column}, or ${untested}.`
}
// ------------------------------------------------------------- DENY TEST ---
const RLS_REFUSED = /new row violates row-level security policy(?: "(?:[^"]|"")*")? for table "((?:[^"]|"")+)"/
/**
* What a probe's error means (the classification table in section 7):
* refused the policy refused the row, on this table
* recursion 42P17
* constraint a 23xxx on this table with no dataType: Postgres checks the
* policy first, so the row got past it (D22)
* error anything else, naming what it reached when the error says
*/
function errorKind(e, schema, name) {
const code = e?.code ?? ''
const message = String(e?.message ?? e)
if (code === '42P17') return { kind: 'recursion' }
const refused = RLS_REFUSED.exec(message)
if (code === '42501' && refused) {
return refused[1].replace(/""/g, '"') === name ? { kind: 'refused' } : { kind: 'error', object: refused[1] }
}
if (code.startsWith('23')) {
if (e.table === name && (!e.schema || e.schema === schema) && !e.dataType) return { kind: 'constraint', constraint: e.constraint ?? null }
return { kind: 'error', object: e.dataType ? `domain ${e.dataType}` : e.table ? `${e.schema ?? schema}.${e.table}` : (e.constraint ?? null) }
}
const denied = /permission denied for (.+)$/.exec(message)
return { kind: 'error', object: denied ? denied[1] : null }
}
/** A policy expression inside using ( ... ), without doubling the parentheses. */
const wrap = (expr) => {
if (expr == null) return '()'
let depth = 0
const single = expr.startsWith('(') && [...expr].every((ch, i) => {
depth += ch === '(' ? 1 : ch === ')' ? -1 : 0
return depth > 0 || i === expr.length - 1
})
return single ? expr : `(${expr})`
}
/**
* The cross-tenant deny test (section 7). User A, in tenant A, tries to read,
* change, delete and create a row of tenant B. It receives a database booted
* from the snapshot's datadir, or the one that built the snapshot. Each test
* is one transaction that ends in rollback, so nothing it does survives.
*/
export async function denyTests(db, snapshot, job) {
const { config, contract } = job
const out = { built: true, tests: [], findings: [], untested: [], notes: [] }
const plan = planDenyTests(snapshot, contract, config)
out.notes.push(...plan.notes)
out.untested.push(...plan.problems)
const supabase = config.platform === 'supabase'
const superuser = (await db.query('select current_user as u')).rows[0].u
const roles = new Set((await db.query('select rolname from pg_roles')).rows.map((r) => r.rolname))
const roleOf = (actor) => config.actors?.[actor]?.role ?? (supabase ? 'authenticated' : config.actAs.role)
const missingRoles = [...new Set(plan.tests.map((t) => roleOf(t.actor)))].filter((r) => !roles.has(r))
for (const role of missingRoles) {
const own = role !== (supabase ? 'authenticated' : config.actAs.role)
out.untested.push({
code: 'CONFIG-INVALID',
object: own ? 'actors' : 'actAs.role',
message: `the role ${role} does not exist after the migrations, so the test cannot act as it.`,
fix: `Name the role your app connects as. The migrations must create it. If this app has no row level security and checks access in app code, set enforcement: 'app' in CONFIG instead.`,
})
}
// A contract or config problem that touches every test stops them all: a
// test run against a guessed shape proves nothing.
if (plan.problems.length || missingRoles.length) {
for (const t of plan.tests) out.tests.push({ actor: t.actor, entity: t.entity, table: t.table, probes: {} })
return out
}
const typeOf = await loadTypes(db)
for (const test of plan.tests) {
const record = { actor: test.actor, entity: test.entity, table: test.table, probes: {} }
out.tests.push(record)
if (test.problem) {
out.untested.push({ ...test.problem, table: test.table, actor: test.actor })
continue
}
await db.exec('begin')
try {
await runDenyTest({ db, snapshot, config, plan, test, record, out, typeOf, superuser, role: roleOf(test.actor) })
} catch (caught) {
// A database error outside a probe (a lookup of a row the seeder made
// with a value its column refuses) is a seed that failed, not a crash:
// the lints that already ran must still be reported.
const fromDatabase = /^[0-9A-Z]{5}$/.test(String(caught?.code ?? ''))
if (!(caught instanceof SeedFailure) && !fromDatabase) throw caught
const e = caught instanceof SeedFailure ? caught : new SeedFailure(test.table, null, `${caught.code} ${caught.message}`)
out.untested.push({
code: 'SEED-FAILED',
table: test.table,
actor: test.actor,
object: test.table,
message: `seeding ${e.table}: ${e.message}`,
fix: seedFix(e, test),
})
} finally {
await db.exec('rollback')
}
}
return out
}
/**
* The anon probe: as the anon role, with no user, select from a table the
* test seeded. A row back is open to anyone holding the public key. Columns a
* policy reads are filled on B's row first, since the seeder leaves nullable
* ones empty and a branch like `auth.uid() is null and session_id is not null`
* needs them. Every change is inside the test's transaction and rolls back.
*/
async function anonRead({ db, snapshot, config, table, extra, row, fill, byKey, castOf, seen, out, supabase, superuser }) {
if (serverOnly(config)) return
if (!(await db.query("select 1 from pg_roles where rolname = 'anon'")).rows.length) return
const savepoint = async (fn) => {
await db.exec('savepoint anon_probe')
try {
return await fn()
} finally {
await db.exec('rollback to savepoint anon_probe; release savepoint anon_probe')
}
}
for (const key of new Set([table, ...extra])) {
const t = snapshot.tables[key]
const target = key === table && row
if (seen.get(key) === true || (seen.has(key) && !target)) continue
if (!t || t.kind !== 'table' || !t.rls?.enabled || !config.exposedSchemas.includes(key.slice(0, key.indexOf('.')))) continue
if (!t.grants.anon?.includes('SELECT')) continue
const reaching = t.policies.filter((p) => p.permissive && ['SELECT', 'ALL'].includes(p.command) && (p.roles.includes('public') || p.roles.includes('anon')))
if (!reaching.length) continue
seen.set(key, !!target)
const asAnon = async () => {
await db.exec('set local role "anon"')
if (supabase) {
await db.query("select set_config('request.jwt.claims', $1, true), set_config('request.jwt.claim.sub', '', true), set_config('request.jwt.claim.role', 'anon', true)", [
JSON.stringify({ role: 'anon' }),
])
}
return (await db.query(`select count(*)::int as n from ${qt(key)}`)).rows[0].n
}
const count = (policies = null) =>
savepoint(async () => {
if (policies) {
await db.exec(`set local role ${qi(superuser)}`)
for (const p of reaching.filter((x) => !policies.includes(x))) await db.exec(`drop policy ${qi(p.name)} on ${qt(key)}`)
}
return asAnon()
}).catch(() => 0)
await db.exec('savepoint anon_fill')
try {
if (target) {
const text = reaching.map((p) => `${p.using ?? ''} ${p.check ?? ''}`).join(' ')
const pk = snapshot.tables[key].primaryKey
const fk = new Set(t.foreignKeys.flatMap((k) => k.columns))
const empty = Object.entries(t.columns)
.filter(([c, col]) => row[c] == null && !pk.includes(c) && !fk.has(c) && !col.generated && col.identity !== 'always' && new RegExp(`\\b${c}\\b`).test(text))
.map(([c]) => c)
if (empty.length) {
const k = byKey(row, empty.length + 1)
try {
await db.exec(`set local role ${qi(superuser)}`)
await db.query(
`update ${qt(key)} set ${empty.map((c, i) => `${qi(c)} = $${i + 1}${castOf(t, c)}`).join(', ')} where ${k.sql}`,
[...empty.map((c) => fill(c)), ...k.params],
)
} catch {
await db.exec('rollback to savepoint anon_fill')
}
}
}
const n = await count()
if (n < 1) continue
let culprits = reaching
if (reaching.length > 1) {
const alone = []
for (const p of reaching) if ((await count([p])) > 0) alone.push(p)
if (alone.length) culprits = alone
}
const show = (p) => `"${p.name}" using ${(p.using ?? '()').replace(/\s+/g, ' ')}`
const names = culprits.map((p) => `"${p.name}"`).join(' and ')
const publicReason = culprits.map(publicPolicyReason).find(Boolean)
out.findings.push({
code: 'LEAK-ANON-READ',
table: key,
object: key,
message: `anon, with no signed-in user, read ${n} row${n === 1 ? '' : 's'} of ${key}.\npolicies for SELECT that reach anon: ${culprits.map(show).join('; ')}`,
fix: `Policy ${names} on ${key} lets a caller with no user read rows. A test like auth.uid() is null is true for anon, so remove that branch, or limit the policy to authenticated. If every row is meant to be public, add ${key} to CONFIG.publicTables with a reason and an until date.`,
// Read back by accessGate, which knows the contract's entity and actor names.
...(publicReason ? { publicReason } : {}),
})
} finally {
await db.exec('rollback to savepoint anon_fill; release savepoint anon_fill')
}
}
}
/** One test: seed A and B, check the grants, then the probes. Runs inside the test's transaction. */
async function runDenyTest({ db, snapshot, config, plan, test, record, out, typeOf, superuser, role }) {
const { table, actor, path } = test
const { tenancy, membership, owners } = plan
const t = snapshot.tables[table]
const [schema, name] = [table.slice(0, table.indexOf('.')), table.slice(table.indexOf('.') + 1)]
const supabase = config.platform === 'supabase'
const actorConfig = config.actors?.[actor] ?? {}
const tenant = {}
const parents = new Map()
const ID = { ...USERS, c: BYSTANDER }
const signedUp = new Set()
let n = 0
let seedingAs = null
const vars = (who) => ({ user: ID[who], tenant: tenant[who] ?? '', n: ++n })
const claimsFor = (who) =>
deepMerge(
{ sub: ID[who], role, aud: 'authenticated', email: EMAILS[who], is_anonymous: false, app_metadata: { provider: 'email' }, user_metadata: {} },
fillIn(actorConfig.claims ?? {}, vars(who)),
)
const settingsFor = (who) => fillIn({ ...(config.actAs.settings ?? {}), ...(actorConfig.settings ?? {}) }, vars(who))
const session = async (who) => {
if (supabase) {
const claims = who ? JSON.stringify(claimsFor(who)) : ''
await db.query("select set_config('request.jwt.claims', $1, true), set_config('request.jwt.claim.sub', $2, true)", [claims, who ? ID[who] : ''])
} else if (who) {
for (const [k, v] of Object.entries(settingsFor(who))) await db.query('select set_config($1, $2, true)', [k, String(v)])
}
}
const returning = (tt) => Object.keys(tt.columns).map((c) => `${qi(c)}::text as ${qi(c)}`).join(', ')
const castOf = (tt, c) => `::text::${tt.columns[c].type}`
const typesOf = (tt) => Object.fromEntries(Object.entries(tt.columns).map(([c, col]) => [c, col.type]))
/** A value for one column, from its type and the CHECK rules on it. */
const synth = (key, column, who) => {
const tt = snapshot.tables[key]
const info = typeOf(key, column)
const type = tt.columns[column].type
if (!info || info.composite) throw new SeedFailure(key, column, `${column} has type ${type}, which the seeder cannot synthesise`)
const count = ++n
const unique = tt.primaryKey.includes(column) || tt.unique.some((set) => set.includes(column))
let value
let kind = 'other'
if (info.array) value = '{}'
else if (info.labels) {
value = info.labels[0]
kind = 'text'
} else if (info.name === 'uuid') value = crypto.randomUUID()
else if (TEXT_TYPES.has(info.name)) {
value = `ntent-${who}-${column}-${count}`
kind = 'text'
} else if (INT_TYPES.has(info.name)) {
value = String(count)
kind = 'num'
} else if (NUM_TYPES.has(info.name)) {
value = unique ? String(count) : '1'
kind = 'num'
} else {
value = {
bool: 'false', date: '2026-01-01', timestamp: '2026-01-01 00:00:00+00', timestamptz: '2026-01-01 00:00:00+00',
time: '12:00', timetz: '12:00', interval: '1 day', json: '{}', jsonb: '{}', bytea: '\\x00',
inet: '127.0.0.1', cidr: '127.0.0.1', tsvector: '',
}[info.name]
if (value === undefined) throw new SeedFailure(key, column, `${column} has type ${type}, which the seeder cannot synthesise`)
}
if (!info.array) {
const defs = tt.checks.map((c) => c.def)
value = applyChecks(value, { defs, ref: esc(quoteIdent(column)), kind, unique, n: count })
value = applyChecks(value, { defs: info.domainChecks, ref: 'VALUE', kind, unique, n: count })
}
if (kind === 'text' && info.length) value = value.slice(0, info.length)
return value
}
/** CONFIG.seed's value for a column, filled for the principal; undefined when it has none. */
const seeded = (key, column, who) => {
const v = config.seed?.[key]?.[column]
if (v === undefined) return undefined
if (v === null) return null
return fillIn(typeof v === 'string' ? v : isObject(v) || Array.isArray(v) ? JSON.stringify(v) : String(v), vars(who))
}
/** The id a user has in a column: their uuid, or a whole number when the column is an integer. */
const principal = (key, column, who) => {
const name = column ? typeOf(key, column)?.name : null
if (!INT_TYPES.has(name) && name !== 'numeric') return ID[who]
return String({ int2: 30000, int4: 900000, int8: 900000, numeric: 900000 }[name] + 'abc'.indexOf(who) + 1)
}
/**
* The value a column that names a user takes. The user's id when it is the
* auth user (or has no foreign key); when it points at the app's own users
* or profiles table, the key of that table's row for the user.
*/
const userValue = async (key, column, who, depth = 0) => {
const fk = snapshot.tables[key]?.foreignKeys.find((k) => k.columns.length === 1 && k.columns[0] === column)
if (!fk || (fk.references === 'auth.users' && supabase) || !snapshot.tables[fk.references]) return principal(key, column, who)
return (await userRow(fk.references, who, depth + 1))[fk.referencedColumns[0]]
}
/**
* The user's row in the app's own users table. Keyed by the auth user's id,
* it is made with that id. With its own key and a column that references
* auth.users, that column takes the user and the key is whatever it gets.
* Either way a row a trigger already made is the row.
*/
const userRow = async (key, who, depth) => {
const cached = parents.get(`${who}\0user\0${key}`)
if (cached) return cached
const tp = snapshot.tables[key]
const authFk = supabase ? tp.foreignKeys.find((k) => k.references === 'auth.users' && k.columns.length === 1) : null
const row =
authFk || tp.primaryKey.length !== 1
? await seedRow(key, who, { depth })
: await seedRow(key, who, { fixed: { [tp.primaryKey[0]]: await userValue(key, tp.primaryKey[0], who, depth) }, depth })
parents.set(`${who}\0user\0${key}`, row)
return row
}
/**
* One row for a principal (steps 2 and 5), as text by column. Parents come
* first, for the same principal. A row a trigger already made for the
* principal is reused.
*/
const seedRow = async (key, who, { fixed = {}, depth = 0, seen = [] } = {}) => {
if (key === 'auth.users' && supabase) {
await signUp(who)
return { id: ID[who] }
}
const tt = snapshot.tables[key]
if (!tt || !['table', 'partitioned_table'].includes(tt.kind)) {
throw new SeedFailure(key, null, `${key} is not a table the snapshot holds, so the seeder cannot fill it`)
}
if (depth > 5) throw new SeedFailure(key, null, 'the foreign keys go more than 5 tables deep')
const vals = { ...fixed }
const apart = apartColumns(tt)
if (tenancy.column && tt.columns[tenancy.column] && !(tenancy.column in vals) && tenant[who] !== undefined) vals[tenancy.column] = tenant[who]
const owner = owners.get(key)
if (owner && !(owner in vals)) vals[owner] = await userValue(key, owner, who, depth)
if (membership && key === membership.table && !(membership.user_column in vals)) {
vals[membership.user_column] = await userValue(key, membership.user_column, who, depth)
}
const toUsers = supabase ? tt.foreignKeys.filter((k) => k.references === 'auth.users' && k.columns.length === 1) : []
if (toUsers.length) await signUp(who)
for (const k of toUsers) if (!(k.columns[0] in vals)) vals[k.columns[0]] = ID[who]
// A row that is already there is the row: one a trigger made when the
// user signed up, found by its key or by the user it belongs to.
const keyed = tt.primaryKey.length > 0 && tt.primaryKey.every((c) => vals[c] != null)
if (keyed || toUsers.length) {
const where = keyed ? tt.primaryKey.map((c) => [c, vals[c]]) : Object.entries(vals).filter(([, v]) => v !== null)
const found = (await db.query(
`select ${returning(tt)} from ${qt(key)} where ${where.map(([c], i) => `${qi(c)} = $${i + 1}${castOf(tt, c)}`).join(' and ')} limit 1`,
where.map(([, v]) => v),
)).rows[0]
if (found) return found
}
for (const [column, col] of Object.entries(tt.columns)) {
if (column in vals) continue
const value = seeded(key, column, who)
if (value !== undefined) {
vals[column] = value
continue
}
if (col.nullable || col.default !== null || col.identity || col.generated) continue
const fk = tt.foreignKeys.find((k) => k.columns.includes(column))
if (fk) {
if (fk.references === key || seen.includes(fk.references)) {
throw new SeedFailure(key, column, `${column} is NOT NULL and closes a cycle of foreign keys through ${fk.references}`)
}
// A CHECK that wants this column to differ from another one into the same table gets a row of someone else's.
const of = apart.has(column) ? 'c' : who
const cached = parents.get(`${of}\0${fk.references}`)
const parent = cached ?? (await seedRow(fk.references, of, { depth: depth + 1, seen: [...seen, key] }))
fk.columns.forEach((c, i) => {
if (!(c in vals)) vals[c] = parent[fk.referencedColumns[i]] ?? null
})
continue
}
vals[column] = synth(key, column, who)
}
// The parents are made now, so a key that was not known before is known. A
// row a trigger made when a parent was inserted (the owner's membership row)
// has that key: it is the row, as one made at signup is.
const findByKey = async () => {
if (!tt.primaryKey.length || !tt.primaryKey.every((c) => vals[c] != null)) return null
return (await db.query(
`select ${returning(tt)} from ${qt(key)} where ${tt.primaryKey.map((c, i) => `${qi(c)} = $${i + 1}${castOf(tt, c)}`).join(' and ')} limit 1`,
tt.primaryKey.map((c) => vals[c]),
)).rows[0] ?? null
}
if (!keyed) {
const found = await findByKey()
if (found) return found
}
applyCrossChecks(tt, vals, (c) => typeOf(key, c)?.name, new Set([...Object.keys(fixed), ...Object.keys(config.seed?.[key] ?? {})]))
if (seedingAs !== who) {
await session(who)
seedingAs = who
}
const cols = Object.keys(vals)
const sql = cols.length
? `insert into ${qt(key)} (${cols.map(qi).join(', ')}) values (${cols.map((c, i) => `$${i + 1}${castOf(tt, c)}`).join(', ')}) returning ${returning(tt)}`
: `insert into ${qt(key)} default values returning ${returning(tt)}`
await db.exec('savepoint seed')
try {
const row = (await db.query(sql, cols.map((c) => vals[c]))).rows[0]
await db.exec('release savepoint seed')
if (!parents.has(`${who}\0${key}`)) parents.set(`${who}\0${key}`, row)
return row
} catch (e) {
// A trigger that fires on this insert can make a row with the same key first.
if (e.code === '23505') {
await db.exec('rollback to savepoint seed')
const found = await findByKey()
if (found) {
await db.exec('release savepoint seed')
if (!parents.has(`${who}\0${key}`)) parents.set(`${who}\0${key}`, found)
return found
}
}
throw seedFailure(key, e, vals, typesOf(tt), tt)
}
}
// 1. Users. Signup runs with no session, so a trigger on auth.users sees
// no auth.uid(), as it would in production.
const signUp = async (who) => {
if (!supabase || signedUp.has(who)) return
signedUp.add(who)
await session(null)
seedingAs = null
const claims = fillIn(actorConfig.claims ?? {}, vars(who))
await db.exec('savepoint seed')
try {
await db.query(
"insert into auth.users (id, aud, role, email, raw_app_meta_data, raw_user_meta_data) values ($1::text::uuid, 'authenticated', 'authenticated', $2, $3::text::jsonb, $4::text::jsonb)",
[ID[who], EMAILS[who], JSON.stringify(deepMerge({ provider: 'email', providers: ['email'] }, claims.app_metadata ?? {})), JSON.stringify(claims.user_metadata ?? {})],
)
await db.exec('release savepoint seed')
} catch (e) {
throw seedFailure('auth.users', e, {}, {})
}
}
for (const who of ['a', 'b']) await signUp(who)
// 3. Tenants and membership. A membership row a trigger made is the tenant.
const { tenantTable, tenantKey } = path
for (const who of ['a', 'b']) {
if (membership) {
const mt = snapshot.tables[membership.table]
// The auth user, or the user's row in the app's users table.
const uv = await userValue(membership.table, membership.user_column, who)
const found = (await db.query(
`select ${qi(membership.tenant_column)}::text as t from ${qt(membership.table)} where ${qi(membership.user_column)} = $1${castOf(mt, membership.user_column)} limit 1`,
[uv],
)).rows[0]
if (found) {
tenant[who] = found.t
continue
}
if (membership.table === tenantTable) {
const row = await seedRow(membership.table, who, { fixed: { [membership.user_column]: uv } })
tenant[who] = row[membership.tenant_column]
continue
}
const fixed = { [membership.user_column]: uv }
if (tenantTable && tenantTable !== 'auth.users') fixed[membership.tenant_column] = (await seedRow(tenantTable, who))[tenantKey]
tenant[who] = (await seedRow(membership.table, who, { fixed }))[membership.tenant_column]
} else {
// No membership: the actor is the tenant. A tenant table row, if there
// is one, gets the user's id as its key.
tenant[who] = tenantTable ? principal(tenantTable, tenantKey, who) : principal(table, tenancy.column, who)
if (tenantTable && !(tenantTable === 'auth.users' && supabase)) await seedRow(tenantTable, who, { fixed: { [tenantKey]: tenant[who] } })
}
}
// Claims can name {tenant}; now there is one.
seedingAs = null
if (supabase && JSON.stringify(actorConfig.claims ?? {}).includes('{tenant}')) {
for (const who of ['a', 'b']) {
const claims = fillIn(actorConfig.claims ?? {}, vars(who))
await db.query('update auth.users set raw_app_meta_data = $2::text::jsonb, raw_user_meta_data = $3::text::jsonb where id = $1::text::uuid', [
ID[who],
JSON.stringify(deepMerge({ provider: 'email', providers: ['email'] }, claims.app_metadata ?? {})),
JSON.stringify(claims.user_metadata ?? {}),
])
}
}
// 4. The target rows, one for A and one for B.
const target = async (who) => {
if (path.self) {
const where =
path.self === 'membership'
? { [membership.user_column]: await userValue(table, membership.user_column, who), [membership.tenant_column]: tenant[who] }
: { [tenantKey]: tenant[who] }
const w = Object.entries(where)
const found = (await db.query(
`select ${returning(t)} from ${qt(table)} where ${w.map(([c], i) => `${qi(c)} = $${i + 1}${castOf(t, c)}`).join(' and ')} limit 1`,
w.map(([, v]) => v),
)).rows[0]
if (found) return found
return seedRow(table, who, { fixed: where })
}
const fixed = {}
if (path.chain) {
// Top-down: the table that carries the tenancy column first.
let value = null
for (let i = path.chain.length - 1; i >= 0; i--) {
const hop = path.chain[i]
const above = path.chain[i + 1]
const row = await seedRow(hop.parent, who, { fixed: above ? { [above.column]: value } : {} })
value = row[hop.parentColumn]
}
fixed[path.chain[0].column] = value
}
return seedRow(table, who, { fixed })
}
const own = await target('a')
const other = await target('b')
// The grants, read as postgres before acting (review 1, B1). A command the
// role has no grant for is denied by grant, not by policy, and not probed.
const [grant] = (await db.query(
`select has_any_column_privilege($1::text::name, $2::text::regclass, 'SELECT') as "SELECT",
has_any_column_privilege($1::text::name, $2::text::regclass, 'INSERT') as "INSERT",
has_any_column_privilege($1::text::name, $2::text::regclass, 'UPDATE') as "UPDATE",
has_table_privilege($1::text::name, $2::text::regclass, 'DELETE') as "DELETE"`,
[role, qt(table)],
)).rows
const columnGrant = new Map((await db.query(
`select attname::text as name, has_column_privilege($1::text::name, $2::text::regclass, attname, 'SELECT') as sel,
has_column_privilege($1::text::name, $2::text::regclass, attname, 'UPDATE') as upd
from pg_attribute where attrelid = $2::text::regclass and attnum > 0 and not attisdropped order by attnum`,
[role, qt(table)],
)).rows.map((r) => [r.name, r]))
for (const command of ['SELECT', 'UPDATE', 'DELETE', 'INSERT']) {
if (grant[command]) continue
record.probes[command] = 'denied by grant'
out.notes.push({
code: 'DENIED-BY-GRANT',
object: `${table} as ${actor}`,
message: `${role} has no ${command} grant on ${table}, so ${command} is denied by grant, not by policy, and was not probed.`,
})
}
// Finding and could-not-check lines.
const byTenant = !!(path.column || (path.chain && !path.chainOwner) || path.self)
const noun = byTenant ? (tenancy.scope_entity ?? 'tenant') : 'user'
const scopeWord = byTenant ? 'tenant' : 'owner'
const condition = tenantCondition({ table, path, snapshot, tenancy, membership, config })
const onMembership = Boolean(membership && table === membership.table && byTenant)
const policyLines = (command, clause) => {
if (!t.rls?.enabled) return `policies for ${command}: none apply, row level security is off on ${table}`
const ps = t.policies.filter((p) => (p.command === command || p.command === 'ALL') && (p.roles.includes('public') || p.roles.includes(role)))
if (!ps.length) return `policies for ${command}: none for ${role}`
const expr = (p) => (clause === 'check' ? `with check ${wrap(p.check ?? p.using)}` : `using ${wrap(p.using)}`).replace(/\s+/g, ' ')
return `policies for ${command}: ${ps.map((p) => `"${p.name}"${p.permissive ? '' : ' (restrictive)'} ${expr(p)}`).join('; ')}`
}
const finding = (code, message, fix) => out.findings.push({ code, table, actor, object: table, message, fix })
const aRow = `${/^[aeiou]/i.test(test.entity) ? 'an' : 'a'} ${test.entity} row`
// Columns the triggers on this table read as OLD.x or NEW.x that the table
// does not have. plpgsql binds late, so the load passes and the first
// statement that fires the trigger finds them, one at a time.
const triggerGaps = async () => {
const { rows } = await db.query(
'select p.prosrc as src from pg_trigger g join pg_proc p on p.oid = g.tgfoid where g.tgrelid = $1::text::regclass and not g.tgisinternal',
[qt(table)],
)
const named = new Set()
for (const { src } of rows) {
for (const m of String(src).matchAll(/\b(?:old|new)\.("(?:[^"]|"")+"|[A-Za-z_][\w$]*)/gi)) {
const column = m[1].startsWith('"') ? m[1].slice(1, -1).replace(/""/g, '"') : m[1].toLowerCase()
if (!t.columns[column]) named.add(column)
}
}
return [...named].sort()
}
const probeError = async (probe, e, object) => {
const what = `${e?.code ?? ''} ${e?.message ?? e}`.trim()
// A trigger or function that reads something the migrations never made
// (a column, a table, a function). Granting or leaving the table untested
// is not the fix.
const missing = ['42703', '42P01', '42883'].includes(e?.code) && (e?.where || /record "(?:new|old)"/.test(String(e?.message)))
const gaps = missing && /record "(?:new|old)"/.test(String(e?.message)) ? await triggerGaps().catch(() => []) : []
out.untested.push({
code: 'PROBE-ERROR',
table,
actor,
object: table,
message: `${probe} failed: ${what}${object ? `. It reaches ${object}.` : ''}${missing && e.where ? `\n${String(e.where).split('\n')[0]}` : ''}${gaps.length ? `\nThe triggers on ${table} read ${gaps.join(', ')}, which the table does not have.` : ''}`,
fix: missing
? `A trigger or function that runs on ${table} reads something the migrations never create, so any statement that fires it fails. This is not a grant problem. Add the missing ${gaps.length ? `column${gaps.length === 1 ? '' : 's'} (${gaps.join(', ')})` : 'column, table or function'} in a migration, or fix the trigger. If it was made outside the migrations, put its definition in a SQL file in CONFIG.sources: with late: true for a column of a table the migrations create (alter table ... add column if not exists), listed first for a table or function they use but never create. With late: true the statements that failed are run again once it has loaded.`
: `The probe failed for a reason that is not a denial (${what}). If the policy calls something the role cannot reach, grant it; otherwise add '${table} as ${actor}' to CONFIG.untested with a reason and an until date.`,
})
record.probes[probe] = 'error'
}
const recursion = (probe) => {
finding(
'POLICY-RECURSES',
`${probe} raised 42P17: the policies on ${table} reach back to themselves.`,
`The policies on ${table} reference each other in a loop. Move the lookup into a security definer function, or read a table whose policy does not read back.`,
)
record.probes[probe] = 'recursion'
}
// Acting as a user: a savepoint, the role, the claims. Rolling back to the
// savepoint returns the role and the settings to postgres (verified).
const actAs = async (who) => {
await db.exec(`set local role ${qi(role)}`)
if (supabase) {
const claims = claimsFor(who)
await db.query(
"select set_config('request.jwt.claims', $1, true), set_config('request.jwt.claim.sub', $2, true), set_config('request.jwt.claim.role', $3, true)",
[JSON.stringify(claims), ID[who], String(claims.role ?? role)],
)
} else {
for (const [k, v] of Object.entries(settingsFor(who))) await db.query('select set_config($1, $2, true)', [k, String(v)])
}
}
const attempt = async (fn) => {
await db.exec('savepoint probe')
try {
await actAs('a')
return { value: await fn() }
} catch (error) {
return { error }
} finally {
await db.exec('rollback to savepoint probe; release savepoint probe')
}
}
const asPostgres = () => db.exec(`set local role ${qi(superuser)}`)
const pk = t.primaryKey
// Columns that hold a user, so a retried probe can put A's user in them.
const userColumns = [
...new Set(
[
membership && table === membership.table ? membership.user_column : null,
path.owner,
...t.foreignKeys.filter((k) => k.references === 'auth.users' && k.columns.length === 1).map((k) => k.columns[0]),
].filter((c) => c && t.columns[c]),
),
]
const byKey = (row, from = 1) => ({
sql: pk.map((c, i) => `${qi(c)} = $${from + i}${castOf(t, c)}`).join(' and '),
params: pk.map((c) => row[c]),
})
const count = async (row) => {
const k = byKey(row)
return (await db.query(`select count(*)::int as n from ${qt(table)} where ${k.sql}`, k.params)).rows[0].n
}
/**
* A fresh row: the principal's path and parents copied from their target
* row, synthesised values elsewhere. `swap` names columns that take another
* row's value instead, for the retry in P5.
*/
const freshRow = (who, from, swap = {}) => {
const vals = {}
for (const [column, col] of Object.entries(t.columns)) {
if (col.generated || col.identity === 'always') continue
const isFk = t.foreignKeys.some((k) => k.columns.includes(column))
if (path.pathColumns.includes(column) || isFk) {
if (column in swap) vals[column] = swap[column]
else if (from[column] != null) vals[column] = from[column]
continue
}
const value = seeded(table, column, who)
if (value !== undefined) vals[column] = value
else if (!col.nullable && col.default === null && !col.identity) vals[column] = synth(table, column, who)
}
applyCrossChecks(t, vals, (c) => typeOf(table, c)?.name, new Set([...path.pathColumns, ...Object.keys(config.seed?.[table] ?? {})]))
const cols = Object.keys(vals)
return {
sql: cols.length
? `insert into ${qt(table)} (${cols.map(qi).join(', ')}) values (${cols.map((c, i) => `$${i + 1}${castOf(t, c)}`).join(', ')})`
: `insert into ${qt(table)} default values`,
params: cols.map((c) => vals[c]),
}
}
await anonRead({
db, snapshot, config, table, supabase, superuser, row: other, byKey, castOf, out,
extra: [...parents.keys()].map((k) => k.slice(k.lastIndexOf('\0') + 1)),
fill: (column) => synth(table, column, 'b'),
seen: (plan.anonSeen ??= new Map()),
})
// The control. A test that cannot see or make the user's own row proves
// nothing about anyone else's.
const readProbes = test.read && grant.SELECT
if (readProbes) {
const hidden = pk.filter((c) => !columnGrant.get(c)?.sel)
if (hidden.length) {
out.untested.push({
code: 'PROBE-ERROR',
table,
actor,
object: table,
message: `the role cannot read ${table}.${hidden.join(', ')}, so the test cannot address a row.`,
fix: `Add '${table}' to CONFIG.untested with a reason and an until date.`,
})
return
}
const p1 = await attempt(() => count(own))
if (p1.error) {
const k = errorKind(p1.error, schema, name)
if (k.kind === 'recursion') recursion('P1')
else await probeError('P1', p1.error, k.object)
return
}
if (p1.value !== 1) {
record.probes.P1 = 'hidden'
// Policies that reach this role for SELECT. With none, the rows are hidden by default, not by a claim.
const reaching = t.rls?.enabled ? t.policies.filter((p) => p.permissive && ['SELECT', 'ALL'].includes(p.command) && (p.roles.includes('public') || p.roles.includes(role))) : [null]
out.untested.push({
code: 'OWN-ROW-HIDDEN',
table,
actor,
object: table,
message: `user A cannot read their own ${test.entity} row, so the test proves nothing about other rows.\n${policyLines('SELECT')}`,
fix: reaching.length
? `User A cannot read their own ${table} row. Either the policy is wrong for everyone, or it reads a claim the test does not set: add it to CONFIG.actors.${actor}.claims.`
: `No SELECT policy on ${table} applies to ${role}: ${t.policies.length ? 'every policy is for another role or another command' : 'there is no policy'}, so row level security denies every read, and no claim would change that. If the table is only read by the server or through a security definer function, add '${table} as ${actor}' to CONFIG.untested with a reason and an until date. If ${role} should read its own rows, add a SELECT policy to ${role}: using (${condition}).`,
})
return
}
record.probes.P1 = 'pass'
const p2 = await attempt(() => count(other))
if (p2.error) {
const k = errorKind(p2.error, schema, name)
if (k.kind === 'recursion') return recursion('P2')
await probeError('P2', p2.error, k.object)
} else if (p2.value > 0) {
record.probes.P2 = 'leak'
// A SELECT policy that looks public on purpose: offer the contract change first.
const reason = t.rls?.enabled
? t.policies
.filter((p) => p.command === 'SELECT' && (p.roles.includes('public') || p.roles.includes(role)))
.map(publicPolicyReason)
.find(Boolean)
: null
const fix = `scope the policy to the ${scopeWord}: using (${condition}).`
finding(
'LEAK-READ',
`user A read ${p2.value} ${test.entity} row${p2.value === 1 ? '' : 's'} that belong${p2.value === 1 ? 's' : ''} to another ${noun}.\n${policyLines('SELECT')}`,
reason ? withPublicHint(reason, fix, { actor, entity: test.entity }) : fix,
)
} else record.probes.P2 = 'pass'
} else if (grant.INSERT) {
// Write-only (`can` has no read), or no SELECT grant: the control is a
// fresh row for A's own tenant, which must go in.
const control = await attempt(async () => {
const row = freshRow('a', own)
await db.query(row.sql, row.params)
})
if (control.error) {
const k = errorKind(control.error, schema, name)
if (k.kind === 'recursion') return recursion('control')
if (k.kind === 'refused') {
record.probes.control = 'refused'
out.untested.push({
code: 'OWN-INSERT-REFUSED',
table,
actor,
object: table,
message: `user A cannot create a row in their own ${noun}, so the test proves nothing about other rows.\n${policyLines('INSERT', 'check')}`,
fix: `User A cannot insert their own ${table} row. Either the policy is wrong for everyone, or it reads a claim the test does not set: add it to CONFIG.actors.${actor}.claims.`,
})
return
}
// A 23xxx on this table means the policy let A's row through (D22),
// which is all the control asks.
if (k.kind !== 'constraint') {
await probeError('control', control.error, k.object)
return
}
}
record.probes.control = 'pass'
} else if (grant.UPDATE || grant.DELETE) {
out.untested.push({
code: 'PROBE-ERROR',
table,
actor,
object: table,
message: `${role} can neither read nor insert ${table}, so no own-row control can run, and a refused update or delete would prove nothing.`,
fix: `Grant ${role} SELECT or INSERT on ${table} if the app uses them; otherwise add '${table} as ${actor}' to CONFIG.untested with a reason and an until date.`,
})
return
}
// P3, blind (D27): no WHERE, so no SELECT policy applies, then compare B's
// row as postgres. A's own row is written back with its own value, so a
// WITH CHECK refusal can only come from B's row.
if (grant.UPDATE) {
const unique = new Set(t.unique.flat())
const column = Object.entries(t.columns).find(
([c, col]) => !pk.includes(c) && !unique.has(c) && !col.generated && !col.identity && columnGrant.get(c)?.upd,
)?.[0]
const narrow = column
? null
: Object.entries(t.columns).find(([c, col]) => !col.generated && col.identity !== 'always' && columnGrant.get(c)?.upd)?.[0]
const before = (await db.query(`select ctid::text as c from ${qt(table)} where ${byKey(other).sql}`, byKey(other).params)).rows[0]?.c
if (column || narrow) {
if (narrow) out.notes.push({ code: 'UPDATE-PROBE-NARROW', object: `${table} as ${actor}`, message: 'only updates that read a column were tested' })
const p3 = await attempt(async () => {
if (column) await db.query(`update ${qt(table)} set ${qi(column)} = $1${castOf(t, column)}`, [own[column]])
else {
const k = byKey(other)
await db.query(`update ${qt(table)} set ${qi(narrow)} = ${qi(narrow)} where ${k.sql}`, k.params)
}
await asPostgres()
return (await db.query(`select ctid::text as c from ${qt(table)} where ${byKey(other).sql}`, byKey(other).params)).rows[0]?.c
})
const leak = (how) => {
record.probes.P3 = 'leak'
finding(
'LEAK-UPDATE',
`user A ${how} ${aRow} that belongs to another ${noun}.\n${policyLines('UPDATE')}`,
`scope the UPDATE policy's USING to the ${scopeWord}: using (${condition}). A WITH CHECK alone does not keep users out of another ${noun}'s rows.`,
)
}
if (p3.error) {
const k = errorKind(p3.error, schema, name)
if (k.kind === 'recursion') recursion('P3')
else if (k.kind === 'refused') {
leak(
`reached, with an update that reads no column,${column && column !== tenancy.column && tenancy.column && t.columns[tenancy.column] ? ` and could move to their own ${noun} by setting ${tenancy.column},` : ''}`,
)
} else await probeError('P3', p3.error, k.object)
} else if (p3.value !== before) leak('changed, with an update that reads no column,')
else record.probes.P3 = 'pass'
} else record.probes.P3 = 'no updatable column'
}
// P4, blind: delete with no WHERE, then count B's row as postgres.
if (grant.DELETE) {
const p4 = await attempt(async () => {
await db.query(`delete from ${qt(table)}`)
await asPostgres()
return count(other)
})
if (p4.error) {
const k = errorKind(p4.error, schema, name)
if (k.kind === 'recursion') recursion('P4')
else await probeError('P4', p4.error, k.object)
} else if (p4.value === 0) {
record.probes.P4 = 'leak'
finding(
'LEAK-DELETE',
`user A deleted ${aRow} that belongs to another ${noun}, with a delete that reads no column.\n${policyLines('DELETE')}`,
`scope the DELETE policy's USING to the ${scopeWord}: using (${condition}).`,
)
} else record.probes.P4 = 'pass'
}
// P5: a row for B's tenant, B's owner and B's parents, with no RETURNING,
// which would apply the SELECT policy and hide a with check (true) leak.
if (grant.INSERT) {
const marks = path.pathColumns.filter((c) => other[c] != null)
if (!marks.length) {
await probeError('P5', `B's row has no value in ${path.pathColumns.join(', ')}, so a new row of B's cannot be counted`)
return
}
const mine = { sql: marks.map((c, i) => `${qi(c)} = $${i + 1}${castOf(t, c)}`).join(' or '), params: marks.map((c) => other[c]) }
const countB = async () => (await db.query(`select count(*)::int as n from ${qt(table)} where ${mine.sql}`, mine.params)).rows[0].n
const before = await countB()
const row = freshRow('b', other)
const p5 = await attempt(async () => {
await db.query(row.sql, row.params)
await asPostgres()
return countB()
})
const leak = (extra = '') => {
record.probes.P5 = 'leak'
finding(
'LEAK-INSERT',
`user A created ${aRow} in another ${noun}${extra}.\n${policyLines('INSERT', 'check')}`,
onMembership
? `A row of ${table} is a user joining a ${noun}, so check the ${noun}, not only the user: with check (${membershipInsertCondition({ membership, snapshot, config })}), so a user can only add people to a ${noun} they already administer. with check (${condition}) alone lets anyone join any ${noun}, in any role. The first row, and joining by invitation, go through a security definer function.`
: `scope the INSERT policy's WITH CHECK to the ${scopeWord}: with check (${condition}).`,
)
}
// A unique or primary key error comes after the policy passed, and proves
// nothing about whether a row can be made: the key B's row already has
// refuses the copy. Try again as A: B's tenant and parents, A's user in
// the columns that name a user. Only a row that lands is a leak.
const retry = async () => {
const swap = {}
for (const c of userColumns) if (own[c] != null && other[c] != null && own[c] !== other[c]) swap[c] = own[c]
if (!Object.keys(swap).length) return null
const again = freshRow('b', other, swap)
return attempt(async () => {
await db.query(again.sql, again.params)
await asPostgres()
return countB()
})
}
const unproven = (constraint) => {
record.probes.P5 = 'unproven'
out.notes.push({
code: 'INSERT-UNPROVEN',
object: `${table} as ${actor}`,
message: `the insert broke ${constraint ?? 'a unique key'} after the policy passed, and a row with other key values could not be tried, so whether user A can create ${aRow} in another ${noun} is not proven.\n${policyLines('INSERT', 'check')}`,
fix: `Read the INSERT policy: its WITH CHECK should hold ${condition}. Or give CONFIG.seed values that make the new row unique.`,
})
}
// The user columns are unique too (one row per user), so neither A's nor
// B's id is free. Give the row to a third user, who has no row, and count
// theirs. The policy refusing it is the pass: A may not make rows for others.
const stranger = async () => {
const cols = userColumns.filter((c) => other[c] != null)
if (!cols.length) return null
let swap
try {
swap = {}
for (const c of cols) swap[c] = await userValue(table, c, 'c')
} catch {
return null
}
const theirs = { sql: cols.map((c, i) => `${qi(c)} = $${i + 1}${castOf(t, c)}`).join(' or '), params: cols.map((c) => swap[c]) }
const countTheirs = async () => (await db.query(`select count(*)::int as n from ${qt(table)} where ${theirs.sql}`, theirs.params)).rows[0].n
const was = await countTheirs()
const again = freshRow('b', other, swap)
const result = await attempt(async () => {
await db.query(again.sql, again.params)
await asPostgres()
return countTheirs()
})
return { ...result, was }
}
// After a unique error with no row to retry as A: the third user's.
const lastTry = async (constraint) => {
const result = await stranger()
if (!result) return unproven(constraint)
if (result.error) {
const kind = errorKind(result.error, schema, name)
if (kind.kind === 'recursion') return recursion('P5')
if (kind.kind === 'refused') {
record.probes.P5 = 'pass'
return
}
if (kind.kind === 'constraint') return unproven(constraint)
return await probeError('P5', result.error, kind.object)
}
if (result.value > result.was) return leak('; the row belongs to a third user, because the ids of A and B are each held by a unique key already')
return unproven(constraint)
}
if (p5.error) {
const k = errorKind(p5.error, schema, name)
if (k.kind === 'recursion') recursion('P5')
else if (k.kind === 'refused') {
record.probes.P5 = 'pass'
// B's row is refused by a policy that checks who the user is. Then A as the owner of a row in B's tenant:
// a check on the user alone (user_id = auth.uid()) lets that through, and the tenant is never looked at.
if (byTenant) {
const own5 = await retry()
if (own5 && !own5.error && own5.value > before) {
leak(', with their own user as its owner: the policy checks who the user is and never the ' + noun)
}
}
} else if (k.kind === 'constraint' && p5.error.code === '23505') {
const again = await retry()
if (!again) await lastTry(k.constraint)
else if (again.error) {
const kind = errorKind(again.error, schema, name)
if (kind.kind === 'recursion') recursion('P5')
else if (kind.kind === 'refused' || kind.kind === 'constraint') await lastTry(k.constraint)
else await probeError('P5', again.error, kind.object)
} else if (again.value > before) leak()
else await lastTry(k.constraint)
} else if (k.kind === 'constraint') {
leak(`; the row also broke ${k.constraint ?? 'a constraint'}, which Postgres checks after the policy`)
} else await probeError('P5', p5.error, k.object)
} else if (p5.value > before) leak()
else {
record.probes.P5 = 'rewritten'
out.notes.push({
code: 'INSERT-REWRITTEN',
object: `${table} as ${actor}`,
message: "a trigger set the tenant, so the policy was not what refused B's row",
})
}
}
}
// ----------------------------------------------------------- SCHEMA GATE ---
/** Column families a contract type cannot live in. Conservative on purpose. */
const FORBIDDEN = {}
for (const [types, families] of [
[['boolean'], ['num', 'text', 'time', 'uuid', 'json', 'bytes', 'other']],
[['integer', 'number', 'decimal', 'money', 'percent'], ['bool', 'time', 'uuid', 'json']],
[['date', 'datetime', 'timestamp'], ['bool', 'num', 'uuid', 'json']],
[['id'], ['bool', 'time', 'json']],
[['text', 'email', 'url', 'phone'], ['bool', 'num', 'time', 'uuid']],
[['enum'], ['bool', 'time', 'uuid', 'json']],
]) {
for (const t of types) FORBIDDEN[t] = new Set(families)
}
const NUMERIC_TYPES = new Set(['integer', 'number', 'decimal', 'money', 'percent'])
/**
* SQLite's looser table (section 18). Its types are affinities, and INTEGER
* holds bool and time, TEXT holds time, uuid and json, so only a boolean in a
* REAL or BLOB column, or a number in a BLOB column, contradicts the model. A
* column declared with no type holds anything.
*/
const sqliteContradicts = (type, column) =>
(type === 'boolean' && ['REAL', 'BLOB'].includes(column.affinity)) || (NUMERIC_TYPES.has(type) && column.affinity === 'BLOB')
const camel = (s) => s.replace(/_+([a-z0-9])/g, (_, c) => c.toUpperCase())
const snake = (s) => s.replace(/([a-z0-9])([A-Z])/g, '$1_$2').toLowerCase()
/** The column a field lives in. */
export const columnOf = (field, columnNames) =>
field.column ?? (columnNames === 'camel' ? camel(field.name) : columnNames === 'snake' ? snake(field.name) : field.name)
const quoteIdent = (s) => (/^[a-z_][a-z0-9_$]*$/.test(s) ? s : `"${s.replace(/"/g, '""')}"`)
/** True when CONFIG.ignoreTables covers a table. Prefix match with a trailing *. */
export const ignored = (key, patterns) => patterns.some((p) => (p.endsWith('*') ? key.startsWith(p.slice(0, -1)) : key === p))
const percent = (n) => `${Math.round(n * 100)}%`
/**
* The tables against the model's entities. Contradictions are findings;
* absence is reported, and gated only through CONFIG.minFieldCoverage.
*/
export function schemaGate(snapshot, contract, config) {
const findings = []
const notes = []
const sqlite = snapshot.loader?.name === 'node:sqlite'
const entities = (contract?.entities ?? []).filter(isObject)
const byId = new Map(entities.map((e) => [e.id, e]))
const mapped = new Set()
const unmapped = []
let total = 0
let withColumn = 0
for (const e of entities) {
if (e.table === undefined) {
unmapped.push(e.id)
continue
}
const key = qualify(e.table)
mapped.add(key)
const t = snapshot.tables[key]
if (!t) {
const [schema] = key.split('.')
const near = Object.keys(snapshot.tables).filter((k) => k !== key && k.toLowerCase() === key.toLowerCase())
const hint = near.length === 1 ? ` (did you mean ${schema}.${quoteIdent(near[0].slice(schema.length + 1))}?)` : ''
findings.push({
code: 'ENTITY-NO-TABLE',
table: key,
object: key,
message: `${e.id} maps to ${key}, which the migrations never create${hint}`,
fix: 'Create the table in a migration, or correct entities[].table.',
})
continue
}
const missing = []
const relations = []
for (const raw of e.fields ?? []) {
const f = fieldOf(raw)
if (f.many === true || (byId.has(f.type) && f.type !== e.id)) {
relations.push(f.name)
continue
}
total++
const col = columnOf(f, config.columnNames)
const c = t.columns[col]
if (!c) {
missing.push(f.name)
continue
}
withColumn++
const object = `${key}.${col}`
if (f.required === true && c.nullable && NULLABLE_KNOWN.has(t.kind)) {
findings.push({
code: 'REQUIRED-BUT-NULLABLE',
table: key,
object,
message: `${e.id}.${f.name} is required in the model; ${object} allows null`,
fix: 'Add NOT NULL in a migration (backfill first), or make the field optional in the model if it can be absent.',
})
}
const forbidden = FORBIDDEN[f.type]
if (sqlite ? sqliteContradicts(f.type, c) : forbidden && (c.array || forbidden.has(c.family))) {
findings.push({
code: 'TYPE-CONTRADICTS',
table: key,
object,
message: `${e.id}.${f.name} is ${f.type} in the model; the column is ${c.type}`,
fix: "Change the column type in a migration, or correct the field's type.",
})
}
const target = f.ref && byId.get(f.ref)?.table !== undefined ? qualify(byId.get(f.ref).table) : null
if (target && !t.foreignKeys.some((fk) => fk.columns.includes(col) && fk.references === target)) {
notes.push({ code: 'REF-NO-FK', message: `${e.id}.${f.name} points at ${f.ref}, and ${object} has no foreign key to ${target}.` })
}
}
const fieldCount = (e.fields ?? []).length - relations.length
if (missing.length) {
notes.push({
code: 'MISSING-COLUMN',
message: `${e.id} -> ${key}: ${fieldCount - missing.length} of ${fieldCount} fields have a column (missing: ${missing.join(', ')})`,
})
}
if (relations.length) notes.push({ code: 'RELATION', message: `${e.id}: ${relations.join(', ')}: relation, not checked` })
}
if (config.minFieldCoverage !== null && total > 0 && withColumn / total < config.minFieldCoverage) {
findings.push({
code: 'FIELD-COVERAGE-BELOW-FLOOR',
table: null,
object: 'minFieldCoverage',
message: `${percent(withColumn / total)} of mapped fields have a column; the floor is ${percent(config.minFieldCoverage)}`,
fix: 'Add the missing columns, set fields[].column where the name differs, or lower the floor with a reason in the commit.',
})
}
if (unmapped.length) notes.push({ code: 'NOT-MAPPED', message: `not mapped to a table, so not checked: ${unmapped.join(', ')}` })
const unlisted = Object.entries(snapshot.tables)
.filter(([key, t]) => !t.platform && !mapped.has(key) && !ignored(key, config.ignoreTables))
.map(([key]) => key)
if (unlisted.length) {
notes.push({
code: 'UNLISTED-TABLE',
count: unlisted.length,
tables: unlisted,
message: `${unlisted.length} table(s) no entity maps: ${unlisted.join(', ')}`,
})
}
return { findings, notes }
}
// --------------------------------------------------------- ACCESS REPORT ---
/**
* The scoped tables (section 6), each with the scope the model gives it and
* its tenant path: tables mapped from an entity with an own or tenant row for
* a user actor, plus the tenant tables and the membership table. An actor
* whose kind is absent or unknown counts as a user, so a typo never unscopes
* a table.
*/
export function scopedTables(snapshot, contract, config) {
const plan = planDenyTests(snapshot, contract, config)
const scoped = new Map()
const add = (table, scope, path) => {
if (!snapshot.tables[table]) return
const had = scoped.get(table)
scoped.set(table, { scope: had?.scope ?? scope, path: had?.path ?? path })
}
const scopes = new Map()
for (const row of list(contract?.access?.matrix)) {
if (isObject(row) && ['own', 'tenant'].includes(row.scope)) scopes.set(`${row.actor}\0${row.entity}`, row.scope)
}
// Tables the matrix scopes first, so their own scope wins over the
// tenant-table default.
for (const t of plan.tests) add(t.table, scopes.get(`${t.actor}\0${t.entity}`) ?? 'tenant', t.path)
for (const t of plan.tests) if (t.path?.tenantTable) add(t.path.tenantTable, 'tenant', null)
if (plan.membership) add(plan.membership.table, 'tenant', { self: 'membership', pathColumns: [] })
return { scoped, plan }
}
const ANON_ACTOR = /^(anon|anonymous|public|guest|visitor)$/i
/**
* Tables the contract marks as public: an entity an anonymous actor (kind
* public, or an id like anon or guest) reads with scope all.
*/
export function contractPublicTables(contract) {
const out = new Set()
const actors = new Map(list(contract?.actors).filter(isObject).map((a) => [a.id, a]))
const entities = new Map(list(contract?.entities).filter(isObject).map((e) => [e.id, e]))
for (const row of list(contract?.access?.matrix)) {
if (!isObject(row) || row.scope !== 'all') continue
const actor = actors.get(row.actor)
if (!(actor?.kind === 'public' || ANON_ACTOR.test(String(row.actor)))) continue
if (list(row.can).length && !list(row.can).some((v) => /^(read|list|view|get|browse)/i.test(String(v)))) continue
const table = entities.get(row.entity)?.table
if (table !== undefined) out.add(qualify(table))
}
return out
}
const policyCovers = (restrictive, command, roles) =>
!isTrue(restrictive.using) &&
(restrictive.command === command || restrictive.command === 'ALL') &&
roles.every((r) => restrictive.roles.includes('public') || restrictive.roles.includes(r))
/**
* The access gate at report time (sections 6 and 7): the lints held against
* the contract, the deny test's results, and the publicTables and untested
* allowlists.
*/
export function accessGate({ snapshot, contract, config, deny, today }) {
let hollow = false
let closedByGrant = false
// Every deny test ended in an untested entry the allowlist waived, so no policy ran and the exit is 0 all the same.
let waivedAll = false
const findings = []
const untested = []
const notes = []
const waived = []
const { scoped, plan } = scopedTables(snapshot, contract, config)
const key = (l) => `${l.schema}.${l.object}`
const apiRoles = config.platform === 'supabase' ? ['anon', 'authenticated'] : [config.actAs.role]
const actorRoles = Object.values(config.actors ?? {}).map((a) => a?.role).filter(Boolean)
const readers = new Set(['public', ...apiRoles, ...actorRoles])
// The roles a policy reaches that the API can act as. 'public' reaches them all.
const reach = (roles) => (roles.includes('public') ? [...new Set([...apiRoles, ...actorRoles])] : roles.filter((r) => readers.has(r)))
const policiesOf = (l) => snapshot.tables[key(l)]?.policies ?? []
// A RESTRICTIVE policy for the same command and every role the true
// policy reaches leaves the call to the deny test.
const restricted = (l, command) => {
const roles = reach(l.roles).length ? reach(l.roles) : l.roles
return policiesOf(l).some((p) => !p.permissive && policyCovers(p, command, roles))
}
const conditionOf = (table) => {
const s = scoped.get(table)
if (!s?.path) return '<the owner or tenant condition>'
return tenantCondition({ table, path: s.path, snapshot, tenancy: plan.tenancy, membership: plan.membership, config })
}
// Tables meant to be read by anyone: CONFIG.publicTables in force, or the contract says so.
const contractPublic = contractPublicTables(contract)
const isPublic = (table) => contractPublic.has(table) || (config.publicTables?.[table] && !(today > config.publicTables[table].until))
// The entity id and the actors the contract names for a table, for the "change the contract" half of a public hint.
const entityOf = (table) => list(contract?.entities).find((e) => isObject(e) && e.table !== undefined && qualify(e.table) === table)?.id ?? table
const readerOf = (table) => {
const id = entityOf(table)
const rows = list(contract?.access?.matrix).filter((r) => isObject(r) && r.entity === id && !ANON_ACTOR.test(String(r.actor)))
return (rows.find((r) => ['own', 'tenant'].includes(r.scope)) ?? rows[0])?.actor ?? 'the reader'
}
const hintFor = (l) => publicPolicyReason(policiesOf(l).find((p) => p.name === l.policy && p.command === 'SELECT'))
// Anon cannot read what its grants or the platform do not give it.
const anonReads = (table) => snapshot.tables[table]?.grants?.anon?.includes('SELECT') && !serverOnly(config)
// Tables a lint already flags as open to anon, so the deny probe does not say it twice.
const anonLinted = new Set()
const anonOpen = []
const rlsOff = []
for (const l of snapshot.lints ?? []) {
const table = key(l)
const f = (message, fix) => ({ code: l.code, object: table, message, fix })
switch (l.code) {
case 'RLS-OFF': {
const ignoredTable = ignored(table, config.ignoreTables)
// A framework table the API cannot reach leaks nothing, and the fix
// this lint gives for one (revoke) has to be able to clear it.
if (ignoredTable && !l.grants?.length) break
rlsOff.push(
f(
`row level security is off on ${table}${l.grants?.length ? `, and ${l.grants.join(' and ')} can reach it` : ''}, so every row is open to anyone the grants let in.`,
ignoredTable
? `${table} is in CONFIG.ignoreTables, which only quiets the schema gate. Revoke what the API roles hold (revoke all on ${table} from ${apiRoles.join(', ')}), or, if every row is public, add it to CONFIG.publicTables with a reason and an until date.`
: `Enable row level security on ${table} and add a policy, or add it to CONFIG.publicTables with a reason and an until date if every row is public.`,
),
)
break
}
case 'POLICY-BUT-RLS-OFF':
findings.push(
f(
`${table} has policies, but row level security is off, so none of them apply.`,
`The policies on ${table} do nothing until RLS is enabled: alter table ${table} enable row level security.`,
),
)
break
case 'AUTH-USERS-EXPOSED':
findings.push(
f(
`the ${l.type} ${table} reads auth.users, and the API serves it.`,
'A view in public exposes auth.users. Drop it, or read the columns you need through a security invoker view over a profiles table.',
),
)
break
case 'USER-METADATA-IN-POLICY':
findings.push({
code: l.code,
object: table,
message: `policy "${l.policy}" on ${table} reads user_metadata, which the user can set.`,
fix: `Policy ${l.policy} reads user_metadata, which the user can set. Use app_metadata or a table.`,
})
break
case 'DEFINER-VIEW': {
const readers = apiRoles.filter((r) => snapshot.tables[table]?.grants?.[r]?.includes('SELECT'))
// A table every reader of the view can already read as it is adds nothing through the view.
const open = (t) =>
isPublic(t) ||
(!scoped.has(t) &&
readers.length > 0 &&
readers.every((r) =>
(snapshot.tables[t]?.policies ?? []).some(
(p) => p.permissive && ['SELECT', 'ALL'].includes(p.command) && isTrue(p.using) && (p.roles.includes('public') || p.roles.includes(r)),
),
))
const wanted = [...new Set([...(l.reads ?? []).filter((t) => scoped.has(t)), ...(l.readsRls ?? [])])]
const hits = wanted.filter((t) => !open(t))
if (wanted.length && !hits.length) break
const item = f(
`view ${table} has no security_invoker, so it reads ${hits.length ? hits.join(', ') : (l.reads ?? []).join(', ') || 'its tables'} as its owner.`,
`View ${table} runs as its owner and skips the RLS on ${hits.join(', ') || (l.reads ?? []).join(', ')}. Add with (security_invoker = true).`,
)
if (hits.length) findings.push(item)
else notes.push(item)
break
}
case 'POLICY-ALWAYS-TRUE': {
// A policy only a server role can use ("service role has full access") opens nothing to the API.
if (l.roles.length && l.roles.every((r) => SERVER_ROLES.has(r))) break
if (restricted(l, l.command)) break
const who = l.roles.map((r) => (r === 'public' ? 'every role' : r)).join(', ')
const clause = l.command === 'INSERT' ? 'with check' : 'using'
const item = {
code: l.code,
object: table,
message: `policy "${l.policy}" on ${table} is true for ${l.command}, for ${who}.`,
fix: `Policy ${l.policy} on ${table} allows ${l.command} on every row for ${who}. Scope it: ${clause} (${conditionOf(table)}). If only the service role should use it, add to service_role.${l.command === 'INSERT' && config.platform === 'supabase' ? ' If anyone may create a row here on purpose (a sign-up), tie it to the caller: with check ((select auth.uid()) is not null).' : ''}`,
}
if (scoped.has(table)) findings.push(item)
else notes.push(item)
break
}
case 'SCOPED-READ-TRUE': {
if (!scoped.has(table) && !contractPublic.has(table) && anonReads(table) && (l.roles.includes('public') || l.roles.includes('anon')) && !restricted(l, 'SELECT')) {
anonLinted.add(table)
anonOpen.push({
code: 'ANON-READ-OPEN',
object: table,
message: `policy "${l.policy}" on ${table} is using (true) for ${l.roles.map((r) => (r === 'public' ? 'every role' : r)).join(', ')}, and anon can select from it, so anyone with the public key reads every row.`,
fix: (() => {
const fix = `If every row of ${table} is meant to be public, add it to CONFIG.publicTables with a reason and an until date. Otherwise scope policy ${l.policy}: using (${conditionOf(table)}), and keep anon out with to authenticated.`
const reason = hintFor(l)
if (!reason) return fix
const rest = `Scope policy ${l.policy}: using (${conditionOf(table)}), and keep anon out with to authenticated.`
return withPublicHint(reason, rest, { entity: entityOf(table), visitor: true, extra: `, or add ${table} to CONFIG.publicTables with a reason and an until date` })
})(),
})
break
}
if (!scoped.has(table) || !reach(l.roles).length || restricted(l, 'SELECT')) break
// Reaching anon is worse than reaching a signed-in user: no account is needed. The anon probe still runs and says what it read.
const toAnon = anonReads(table) && (l.roles.includes('public') || l.roles.includes('anon'))
const who = l.roles.map((r) => (r === 'public' ? 'every role' : r)).join(', ')
const reason = hintFor(l)
const fix = `Policy ${l.policy} lets ${toAnon ? 'anyone, signed in or not,' : 'any signed-in user'} read every row of ${table}, which the model scopes ${scoped.get(table).scope}. Replace using (true) with the owner or tenant condition: using (${conditionOf(table)})${toAnon ? ', and limit the policy to authenticated (to authenticated). If it was meant for the service role only, say so: to service_role' : ''}.`
findings.push({
code: l.code,
object: table,
message: `policy "${l.policy}" on ${table} is using (true) for ${who}${toAnon ? ', and anon can select from it, so anyone with the public key reads every row, signed in or not' : ''}.`,
fix: reason ? withPublicHint(reason, fix, { actor: readerOf(table), entity: entityOf(table), visitor: toAnon }) : fix,
})
break
}
case 'RLS-NO-POLICY':
notes.push({ code: l.code, object: table, message: 'row level security is on and no policy exists: safe, it denies everything.' })
break
case 'DEFINER-FUNCTION-EXECUTABLE': {
const fn = `${table}(${l.args ?? ''})`
const callers = list(l.executableBy).join(', ')
const base = `security definer function ${fn} runs as its owner, and ${callers} can call it over RPC`
const revoke = `Revoke execute on ${fn} from anon, authenticated, or make it security invoker.`
// Tables it reads that hold what a caller must not get: tenant rows, secret-looking columns, or anything outside the exposed schemas.
const sensitive = list(l.reads).filter(
(r) => !isPublic(r.table) && (scoped.has(r.table) || r.secrets?.length || !config.exposedSchemas.includes(r.table.slice(0, r.table.indexOf('.')))),
)
const names = sensitive.map((r) => (r.secrets?.length ? `${r.table} (${r.secrets.join(', ')})` : r.table)).join(', ')
// A flag or a count hands back no rows: void, boolean, a number.
const returnsData = l.returns !== undefined && !SCALAR_RETURN.test(String(l.returns).trim())
let why = null
if (l.bodyKnown === false) why = 'its body is not plain text, so what it reads was not checked'
else if (!sensitive.length) why = 'its body names no tenant table, no secret-looking column and nothing outside the exposed schemas'
else if (!returnsData) why = `it returns ${l.returns}, so it hands back no rows`
else if (l.checksCaller) {
why = `it reads ${names} but ${l.checksCallerVia ? `its body calls ${l.checksCallerVia}(), which checks the caller` : 'its body checks the caller (auth.uid() or similar)'}, and whether that check is right needs a human`
}
if (why === null && !serverOnly(config)) {
findings.push({
code: 'DEFINER-FUNCTION-LEAKS',
object: fn,
message: `${base}. It returns ${l.returns} read from ${names}, and nothing in its body checks who is calling, so any caller gets those rows.`,
fix: `${fn} skips RLS and does not check the caller. Compare auth.uid() to the row's owner or tenant inside it, or ${revoke.charAt(0).toLowerCase()}${revoke.slice(1)}`,
})
break
}
notes.push({
code: l.code,
object: fn,
message: `${base}. Kept as a note: ${why ?? 'only your server connects (CONFIG.clientAccess is server-only)'}.`,
fix: revoke,
})
break
}
case 'RLS-NOT-FORCED':
if (config.platform !== 'postgres' || !scoped.has(table)) break
findings.push(
f(
`${config.actAs.role} owns ${table} (through ${l.owner}), and row level security is not forced, so the app's queries skip it.`,
`${config.actAs.role} owns ${table}, and owners skip RLS. alter table ${table} force row level security, or run the app as a role that does not own the table.`,
),
)
break
default:
notes.push({ code: l.code, object: table, message: 'a lint this version does not know how to read.' })
}
}
// The anon probe's findings go through the same allowlist, unless a lint already said it.
const anonProbed = (deny?.findings ?? [])
.filter((f) => f.code === 'LEAK-ANON-READ' && anonReads(f.object))
.map(({ publicReason, ...f }) => (publicReason ? { ...f, fix: withPublicHint(publicReason, f.fix, { entity: entityOf(f.object), visitor: true }) } : f))
anonOpen.push(...anonProbed.filter((f) => !anonLinted.has(f.object) && !anonOpen.some((o) => o.object === f.object)))
const pub = applyAllowlist('publicTables', config.publicTables, [...rlsOff, ...anonOpen].filter((f) => !contractPublic.has(f.object)), (f) => f.object, today)
// Only the server connects: an open table is a note, and the deny test still fails any the contract scopes.
if (serverOnly(config)) {
// One note for all of them: sixty tables must not print sixty times.
if (pub.live.length) {
const names = pub.live.map((n) => n.object)
const shown = names.length > 12 ? `${names.slice(0, 12).join(', ')} and ${names.length - 12} more` : names.join(', ')
notes.push({
code: 'RLS-OFF',
object: plural(names.length, 'table'),
message: `row level security is off on ${shown}. Only your server connects (CONFIG.clientAccess is server-only), so this is a note, not a finding. The deny test still fails any of them the contract scopes.`,
fix: `If a client can reach this database (PostgREST, Hasura, a connection string in an app), set clientAccess: 'direct' in CONFIG, and these become findings. Otherwise enable row level security on the tables that hold tenant data.`,
})
}
} else findings.push(...pub.live)
waived.push(...pub.waived)
notes.push(...pub.stale)
if (contract && !deny?.built) {
untested.push({
code: 'DENY-NOT-RUN',
object: 'deny test',
message: 'the cross-tenant deny test did not run, so no tenant isolation was checked.',
fix: 'Run the check again with --no-cache. If it recurs, this is a bug in the check.',
})
} else if (deny?.built) {
findings.push(...deny.findings.filter((f) => f.code !== 'LEAK-ANON-READ'))
notes.push(...deny.notes)
const entries = config.untested ?? {}
const u = applyAllowlist(
'untested',
entries,
deny.untested,
(x) => (!x.table ? null : entries[`${x.table} as ${x.actor}`] ? `${x.table} as ${x.actor}` : x.table),
today,
)
untested.push(...u.live)
waived.push(...u.waived)
notes.push(...u.stale)
const ran = deny.tests.some((t) => Object.entries(t.probes).some(([probe, result]) => /^P\d$/.test(probe) && ['pass', 'leak', 'rewritten', 'unproven'].includes(result)))
waivedAll = deny.tests.length > 0 && !ran && u.waived.length > 0 && !u.live.length
// A pass nobody earned: tests were planned and every probe was stopped by a grant or an error, so no policy ran.
// A bare `create role`, or a table the role cannot touch, looks exactly like this. It is not a pass.
const exercised = deny.tests.some((t) => Object.entries(t.probes).some(([probe, result]) => /^P\d$/.test(probe) && EXERCISED.has(result)))
if (deny.tests.length && !exercised) {
// Closed by grant is a result when the roles are real: Supabase's anon and authenticated, or a role the repo's own
// migrations create. A role CONFIG.preamble made up for the check proves nothing by being denied.
const supabase = config.platform === 'supabase'
const roles = [...new Set(deny.tests.map((t) => config.actors?.[t.actor]?.role ?? (supabase ? 'authenticated' : config.actAs.role)))]
const invented = roles.filter((r) => (config.preamble ?? []).some((e) => new RegExp(`create\\s+role\\s+(?:if\\s+not\\s+exists\\s+)?"?${r.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}\\b`, 'i').test(e?.sql ?? '')))
const closed = deny.tests.every((t) => Object.values(t.probes).length > 0 && Object.values(t.probes).every((v) => v === 'denied by grant'))
if (closed && !invented.length) closedByGrant = true
else {
hollow = true
if (!deny.untested.length) {
untested.push({
code: 'ACCESS-NOT-EXERCISED',
object: 'deny tests',
message: `${plural(deny.tests.length, 'deny test')} ran and none exercised a policy: every probe was denied by a missing grant, or failed before a policy ran. That is not a pass, and it looks the same as a role with no privileges.`,
fix: `Give the role the table privileges your app has (grant select, insert, update, delete on the tables to ${config.platform === 'supabase' ? 'authenticated' : config.actAs.role}), so the policies are what decide. If access is enforced in app code and these tables have no RLS, set enforcement: 'app' in CONFIG. If these tables are only for the service role, take them out of the contract's access scoping.${invented.length ? ` ${invented.join(', ')} is created in CONFIG.preamble, not in your migrations, so a grant denial says nothing about it.` : ''}`,
})
}
}
}
}
return { findings, untested, notes, waived, hollow, closedByGrant, waivedAll }
}
// -------------------------------------------------------------- EVALUATE ---
/** Probe results that mean a policy decided: it let the own row through, hid or leaked the other's. */
const EXERCISED = new Set(['pass', 'hidden', 'leak', 'rewritten', 'unproven'])
/**
* Apply one CONFIG allowlist to findings. An entry in force turns its finding
* into a printed waiver line; one past its date stops applying and the
* finding says so; one that matched nothing is named as stale.
*/
export function applyAllowlist(list, entries, findings, keyOf, today) {
const live = []
const waived = []
const used = new Set()
for (const f of findings) {
const key = keyOf(f)
const entry = key == null ? undefined : entries?.[key]
if (!entry) {
live.push(f)
continue
}
used.add(key)
if (today > entry.until) live.push({ ...f, message: `${f.message} (waiver expired ${entry.until})` })
else waived.push({ list, key, code: f.code, until: entry.until, reason: entry.reason })
}
const stale = Object.keys(entries ?? {})
.filter((k) => !used.has(k))
.map((k) => ({ code: 'STALE-WAIVER', message: `stale waiver: ${k} in CONFIG.${list}. Delete it.` }))
return { live, waived, stale }
}
/**
* Everything the report says, from a snapshot, the contract and the config.
* Allowlists and their dates apply here, never in the cache.
*/
export function evaluate({ snapshot, contract, config, gates = { schema: true, access: true }, today, deny = null, contractNotes = [], enforcementAuto = false }) {
const findings = []
const untested = []
const notes = [...contractNotes]
const waived = []
if (!contract) notes.push({ code: 'NO-CONTRACT', message: 'no contract.json, so no entity was held to a table and no deny test ran.' })
if (gates.schema) {
const s = schemaGate(snapshot, contract, config)
const a = applyAllowlist('schemaWaivers', config.schemaWaivers, s.findings, (f) => f.object, today)
findings.push(...a.live)
waived.push(...a.waived)
notes.push(...s.notes, ...a.stale)
}
// Extensions stubbed by CONFIG print as waivers on every run.
const configStubs = new Set(snapshot.loader.stubbed.filter((s) => s.by === 'config').map((s) => s.name))
for (const [name, entry] of Object.entries(config.stubExtensions)) {
if (configStubs.has(name)) {
waived.push({ list: 'stubExtensions', key: name, code: 'EXTENSION-UNAVAILABLE', until: entry.until, reason: entry.reason })
}
else notes.push({ code: 'STALE-WAIVER', message: `stale waiver: ${name} in CONFIG.stubExtensions. Delete it.` })
}
for (const f of snapshot.loader.empty ?? []) {
notes.push({
code: 'EMPTY-MIGRATION',
object: f.file,
message: f.comments ? 'has comments and no SQL, so it changes nothing.' : 'is empty, so it changes nothing.',
fix: 'If it was meant to hold SQL, the SQL is missing (a literal \\n instead of a newline, or a stripped file). If it is a placeholder, delete it.',
})
}
const skipped = new Set(snapshot.loader.skipped ?? [])
for (const [key, entry] of Object.entries(config.skipStatements ?? {})) {
if (skipped.has(key)) waived.push({ list: 'skipStatements', key, code: 'LOAD-FAILED', until: entry.until, reason: entry.reason })
else if (today > entry.until) notes.push({ code: 'STALE-WAIVER', message: `stale waiver: '${key}' in CONFIG.skipStatements expired on ${entry.until}. Delete it.` })
else notes.push({ code: 'STALE-WAIVER', message: `stale waiver: '${key}' in CONFIG.skipStatements. Delete it.` })
}
// What the one-line summary says when access was not checked. null when it was.
let accessNote = null
if (gates.access) {
if (snapshot.loader?.name === 'node:sqlite') {
accessNote = 'access NOT checked: SQLite has no row security'
notes.push({
code: 'ACCESS-NOT-CHECKED',
message: 'not checked: SQLite has no row security. Access lives in app code (roadmap R1).',
})
} else if (config.enforcement === 'app') {
accessNote = NOT_CHECKED_APP
const left =
'Left unchecked: every RLS lint (RLS-OFF, always-true policies, security definer views and functions) and the cross-tenant deny test. ' +
"A query that forgets its tenant or owner filter (an IDOR), and a table that holds another tenant's rows, are not caught: app-code access is out of scope here (roadmap R1). " +
'Still checked: the schema gate (entities against tables, columns and types).'
notes.push({
code: 'ACCESS-NOT-CHECKED',
message: enforcementAuto
? "access is enforced in app code; deny tests skipped; set enforcement: 'app' to confirm. " +
`This is plain Postgres and no migration enables row level security or creates a policy, so CONFIG.enforcement 'auto' reads access as app code. ${left}`
: `not checked: CONFIG.enforcement is 'app', so access is enforced in app code, which this check does not run (roadmap R1). ${left}`,
})
} else {
const a = accessGate({ snapshot, contract, config, deny, today })
findings.push(...a.findings)
untested.push(...a.untested)
notes.push(...a.notes)
waived.push(...a.waived)
if (a.hollow) accessNote = 'access NOT checked: no deny test exercised a policy'
else if (a.waivedAll) accessNote = 'access NOT checked: every deny test was waived in CONFIG.untested, so no policy was exercised'
else if (a.closedByGrant) accessNote = 'access checked: every scoped table is closed to client roles by grants'
if (config.platform === 'postgres' && snapshotNoRls(snapshot)) notes.push({ code: 'NO-RLS-FOUND', message: NO_RLS_HINT })
}
}
return { findings, untested, notes, waived, accessNote }
}
// ---------------------------------------------------------------- REPORT ---
const plural = (n, word, many = `${word}s`) => `${n} ${n === 1 ? word : many}`
const block = (p) => {
const lines = [` ${p.code} ${[p.object, p.actor ? `as ${p.actor}` : null].filter(Boolean).join(' ')}`.trimEnd()]
for (const line of String(p.message).split('\n')) lines.push(` ${line}`)
if (p.fix) lines.push(` fix: ${p.fix}`)
return lines.join('\n')
}
/**
* Findings with one cause read as one finding. Two kinds fold together:
* rows that differ only in the object or the actor (RLS off on 60 tables),
* and deny-test LEAK rows on a table whose RLS is off, which that finding
* already explains. Nothing is dropped: each member keeps a line, and --json
* still lists every row, with a `group` on those that share one.
*/
const shapeOf = (f) => {
const generic = (text) => String(text ?? '').split(f.object).join('<object>').split(f.actor ?? '\0').join('<actor>')
return `${f.code}\0${generic(f.message)}\0${generic(f.fix)}`
}
const RLS_OFF_CODES = new Set(['RLS-OFF', 'POLICY-BUT-RLS-OFF'])
export function groupFindings(rows) {
const roots = new Map()
const byShape = new Map()
const groups = []
for (const f of rows) {
if (f.code.startsWith('LEAK-')) continue
const shape = shapeOf(f)
let g = byShape.get(shape)
if (!g) {
g = { head: f, members: [], children: [] }
byShape.set(shape, g)
}
if (RLS_OFF_CODES.has(f.code)) roots.set(f.object, g)
}
const seen = new Set()
for (const f of rows) {
if (f.code.startsWith('LEAK-')) {
const root = roots.get(f.table ?? f.object)
if (root) {
root.children.push(f)
if (!seen.has(root)) {
seen.add(root)
groups.push(root)
}
continue
}
}
const shape = shapeOf(f)
const g = byShape.get(shape) ?? { head: f, members: [], children: [] }
byShape.set(shape, g)
g.members.push(f)
if (!seen.has(g)) {
seen.add(g)
groups.push(g)
}
}
return groups
}
const asObject = (f) => [f.object, f.actor ? `as ${f.actor}` : null].filter(Boolean).join(' ')
const LOAD_ORDER = ['real', 'engine', 'duplicate', 'outside', 'ordering', 'cascade']
/** One group as text. A group of one with nothing under it prints exactly as block() does. */
function groupBlock(g) {
const { head, members, children } = g
if (members.length === 1 && !children.length) return block(head)
const generic = (text) => String(text ?? '').split(head.object).join('<object>')
const lines = [` ${head.code} ${members.length > 1 ? `${members.length} objects, one cause` : asObject(head)}`.trimEnd()]
for (const line of (members.length > 1 ? generic(head.message) : String(head.message)).split('\n')) lines.push(` ${line}`)
const fix = members.length > 1 ? generic(head.fix) : head.fix
if (fix) lines.push(` fix: ${fix}`)
const byObject = new Map(members.map((m) => [m.object, []]))
for (const c of children) byObject.set(c.table ?? c.object, [...(byObject.get(c.table ?? c.object) ?? []), c])
if (members.length > 1 || children.length) lines.push(` ${members.length > 1 ? 'affected' : 'the deny test finds the same cause'}:`)
for (const [object, kids] of byObject) {
const actors = [...new Set(kids.map((k) => k.actor).filter(Boolean))]
const codes = [...new Set(kids.map((k) => k.code))]
const member = members.find((m) => m.object === object)
const label = member ? asObject(member) : object
lines.push(` ${label}${codes.length ? ` ${codes.join(', ')}${actors.length ? ` as ${actors.join(', ')}` : ''}` : ''}`)
}
return lines.join('\n')
}
/** Load failures grouped by class, then everything else grouped by cause. Returns blocks, one per group. */
function problemBlocks(rows) {
const out = []
const loads = rows.filter((r) => r.code === 'LOAD-FAILED' && r.class)
const others = rows.filter((r) => !(r.code === 'LOAD-FAILED' && r.class))
for (const g of groupFindings(others)) out.push(groupBlock(g))
if (loads.length === 1) out.push(block(loads[0]))
else for (const cls of LOAD_ORDER) {
const items = loads.filter((r) => r.class === cls)
if (!items.length) continue
const lines = [` LOAD-FAILED ${LOAD_CLASSES[cls].title}, ${plural(items.length, 'statement')}`, ` fix: ${LOAD_CLASSES[cls].fix}`]
for (const r of items) {
const where = r.class === 'ordering' ? (r.by ? ` (created at ${r.by})` : '') : r.class === 'cascade' ? ` (after ${r.by})` : ''
lines.push(` ${r.object} ${String(r.message).split('\n')[0]}${where}`)
if (r.fix && !Object.values(LOAD_CLASSES).some((c) => r.fix.endsWith(c.fix)) && r.class === 'real') lines.push(` fix: ${r.fix}`)
}
out.push(lines.join('\n'))
}
return out
}
/** The same grouping as a `group` on each row that shares one, for --json. Rows are never removed. */
function withGroups(rows) {
const tag = new Map()
for (const g of groupFindings(rows.filter((r) => !(r.code === 'LOAD-FAILED' && r.class)))) {
if (g.members.length < 2 && !g.children.length) continue
const id = `${g.head.code}:${g.head.object}`
for (const r of [...g.members, ...g.children]) tag.set(r, id)
}
return rows.map((r) => (r.code === 'LOAD-FAILED' && r.class ? { ...r, group: `LOAD-FAILED:${r.class}` } : tag.has(r) ? { ...r, group: tag.get(r) } : r))
}
/**
* The groups themselves, for --json: one entry per cause, so a reader does not
* have to fold the rows by their `group` tag. Added beside the rows, never in
* place of them: every row keeps its fields and its `group`.
*/
export function jsonGroups({ findings = [], untested = [] }) {
const out = []
for (const [section, rows] of [['findings', findings], ['untested', untested]]) {
const loads = rows.filter((r) => r.code === 'LOAD-FAILED' && r.class)
for (const cls of LOAD_ORDER) {
const items = loads.filter((r) => r.class === cls)
if (items.length) out.push({ id: `LOAD-FAILED:${cls}`, section, code: 'LOAD-FAILED', class: cls, count: items.length, objects: items.map((r) => r.object), message: LOAD_CLASSES[cls].title, fix: LOAD_CLASSES[cls].fix })
}
for (const g of groupFindings(rows.filter((r) => !(r.code === 'LOAD-FAILED' && r.class)))) {
if (g.members.length < 2 && !g.children.length) continue
const { head } = g
out.push({
id: `${head.code}:${head.object}`,
section,
code: head.code,
count: g.members.length + g.children.length,
objects: [...new Set([...g.members, ...g.children].map((r) => r.table ?? r.object))],
message: head.message,
fix: head.fix ?? null,
})
}
}
return out
}
const waiverLine = (w) => ` waived ${w.key} ${w.code} until ${w.until} ${w.reason}`
/** The problems that mean the migrations do not replay: the only ones CONFIG.loadGate: 'warn' softens. */
const LOAD_CODES = new Set(['LOAD-FAILED', 'EXTENSION-UNAVAILABLE'])
const LOAD_WARN_LINE = 'database NOT checked: the migrations do not load yet (warn mode). Fix them or add escapes; the first clean load turns this into a gate.'
const exitOf = ({ findings, untested }) => (findings.length ? 1 : untested.length ? 2 : 0)
/** The summary's deny-test count. When every deny test was waived none ran, and the count would read as coverage. */
export const denyTestsPart = (meta) =>
meta.accessNote?.startsWith('access NOT checked: every deny test was waived') ? '0 deny tests ran' : plural(meta.denyTests, 'deny test')
const stripPublic = (v) => (typeof v === 'string' ? v.replace(/(?<![\w.])public\.(?=["\w])/g, '') : v)
/** The outcome with the invented public. schema taken off every name it shows, for a SQLite database. */
function unqualified(outcome) {
const row = (r) => (isObject(r) ? { ...r, ...Object.fromEntries(['object', 'message', 'fix', 'table', 'key'].filter((k) => typeof r[k] === 'string').map((k) => [k, stripPublic(r[k])])) } : r)
return { ...outcome, findings: outcome.findings.map(row), untested: outcome.untested.map(row), notes: outcome.notes.map(row), waived: outcome.waived.map(row) }
}
/** The report, in text or JSON. Returns the exit code. */
function report({ outcome: raw, opts, log, meta }) {
// SQLite has no schemas: the tables are keyed public.<name> inside, and shown as <name>.
const outcome = meta.platform === 'sqlite' ? unqualified(raw) : raw
const { findings, untested, notes, waived } = outcome
const code = outcome.exit ?? exitOf(outcome)
if (opts.json) {
const { booted, cached, platform } = meta
log(JSON.stringify({ exit: code, booted, cached, platform, ...(meta.loadWarn ? { loadWarn: LOAD_WARN_LINE } : {}), findings: withGroups(findings), untested: withGroups(untested), groups: jsonGroups({ findings, untested }), notes, waived }, null, 2))
return code
}
// Printed on every run, --check included: a stale waiver, and a read of the worktree instead of the commit.
const loud = (n) => n.code === 'STALE-WAIVER' || n.code === 'UNSTAGED-INPUT' || n.code === 'OTHER-SOURCES' || n.code === 'UNLISTED-MIGRATION'
const stale = notes.filter(loud)
const other = notes.filter((n) => !loud(n))
const cachedNote = meta.cached === 'none' ? '' : ' (cached)'
if (meta.loadWarn) {
// Not a pass and not a block: everything the load found, on every run, and the state in one line.
const out = [`check-backend: ${meta.platform}${meta.platformNote ? ` (${meta.platformNote})` : ''}, ${plural(meta.inputs, 'migration')}, load failed (warn mode, not blocking)`]
out.push('', ` could not load ${plural(untested.length, 'thing')}:`, ...problemBlocks(untested).flatMap((b) => ['', b]))
for (const s of notes.filter(loud)) out.push('', ` ${s.message}`)
out.push('', ` backend: ${LOAD_WARN_LINE}`)
log(out.join('\n'))
return code
}
if (opts.check && code === 0) {
for (const w of waived) log(waiverLine(w))
for (const s of stale) log(` ${s.message}`)
const parts = [plural(meta.inputs, 'migration'), plural(meta.tables, 'table')]
if (meta.denyTests !== null) parts.push(denyTestsPart(meta))
if (meta.accessNote) parts.push(meta.accessNote)
log(`backend: ${parts.join(', ')}, 0 findings${cachedNote}`)
return code
}
const out = [`check-backend: ${meta.platform}${meta.platformNote ? ` (${meta.platformNote})` : ''}, ${plural(meta.inputs, 'migration')}, ${plural(meta.tables, 'table')}${cachedNote}`]
if (findings.length) out.push('', ` ${plural(findings.length, 'finding')}:`, ...problemBlocks(findings).flatMap((b) => ['', b]))
if (untested.length) {
out.push('', ` could not check ${plural(untested.length, 'thing')}. Exit 2 is never a pass:`, ...problemBlocks(untested).flatMap((b) => ['', b]))
}
if (waived.length || stale.length) out.push('', ...waived.map(waiverLine), ...stale.map((n) => ` ${n.message}`))
if (!opts.check && other.length) {
out.push('', ' notes:')
for (const n of other) out.push(` note ${n.code} ${n.object ? `${n.object}: ` : ''}${n.message}${n.fix ? `\n ${n.fix}` : ''}`)
}
const notChecked = meta.accessNote ? ` ${meta.accessNote[0].toUpperCase()}${meta.accessNote.slice(1)}.` : ''
if (code === 0) out.push('', ` backend: ok. 0 findings${cachedNote}.${notChecked}`)
else out.push('', ` backend: ${plural(findings.length, 'finding')}, ${untested.length} could not be checked.${notChecked}`)
log(out.join('\n'))
return code
}
// ----------------------------------------------------------------- CACHE ---
const sha256 = (...parts) => {
const h = createHash('sha256')
for (const p of parts) h.update(typeof p === 'string' ? p : Buffer.from(p))
return h.digest('hex')
}
/** JSON with sorted keys, so the same value always hashes the same. */
const canonical = (v) => {
if (Array.isArray(v)) return `[${v.map(canonical).join(',')}]`
if (isObject(v)) return `{${Object.keys(v).sort().map((k) => `${JSON.stringify(k)}:${canonical(v[k])}`).join(',')}}`
return JSON.stringify(v ?? null)
}
/** This file without its CONFIG block, so a waiver edit never forces a cold boot. */
const selfWithoutConfig = () => readFileSync(SELF, 'utf8').replace(/^export const CONFIG = \{[\s\S]*?^\}$/m, '')
/**
* The installed PGlite version, without importing it. Its ./package.json is not
* exported, so walk up from the resolved main entry.
*/
export function pgliteVersion() {
try {
let dir = dirname(createRequire(import.meta.url).resolve('@electric-sql/pglite'))
for (let i = 0; i < 6; i++) {
const pkg = join(dir, 'package.json')
if (existsSync(pkg)) {
const json = JSON.parse(readFileSync(pkg, 'utf8'))
if (json.name === '@electric-sql/pglite') return json.version
}
dir = dirname(dir)
}
} catch {
// Not installed. The child says so, with the fix.
}
return 'missing'
}
const readJson = (file) => {
try {
const value = JSON.parse(readFileSync(file, 'utf8'))
const now = new Date()
utimesSync(file, now, now)
return value
} catch {
return null
}
}
/** Write through a temp file and a rename, so two runs at once never read half a file. */
const writeAtomic = (file, data) => {
const tmp = `${file}.tmp-${process.pid}`
writeFileSync(tmp, data)
renameSync(tmp, file)
}
/** Keep the 5 newest cache files of each kind by mtime. */
export const prune = (dir) => {
let entries
try {
entries = readdirSync(dir)
} catch {
return
}
for (const kind of ['.snapshot.json', '.datadir.tgz', '.deny.json']) {
const files = entries
.filter((e) => e.endsWith(kind))
.map((e) => {
try {
return { file: join(dir, e), mtime: statSync(join(dir, e)).mtimeMs }
} catch {
return null // A run sharing this directory deleted it between the listing and now.
}
})
.filter(Boolean)
.sort((a, b) => b.mtime - a.mtime)
for (const { file } of files.slice(5)) {
try {
unlinkSync(file)
} catch {
// Another run got there first.
}
}
}
}
// --------------------------------------------------------------- PROCESS ---
/** Run the engine in a child with a hard timeout. Returns the child's result, or a problem. */
function spawnChild(job, { config, opts, err, root }) {
const r = spawnSync(process.execPath, ['--disable-warning=ExperimentalWarning', SELF, '--child'], {
input: JSON.stringify(job),
timeout: config.timeoutMs,
stdio: ['pipe', 'pipe', 'pipe'],
maxBuffer: 64 * 1024 * 1024,
cwd: root,
})
const stderr = String(r.stderr ?? '')
const phases = stderr.split('\n').filter((l) => l.startsWith('phase ')).map((l) => l.slice(6).trim())
const last = phases.at(-1) ?? 'start'
const relay = stderr.split('\n').filter((l) => l.trim() && (opts.timing || !l.startsWith('phase ')))
for (const line of relay) err(line)
if (r.error?.code === 'ETIMEDOUT') {
return {
ok: false,
problem: {
code: 'TIMEOUT',
object: last,
message: `The check was stopped after ${config.timeoutMs} ms, during ${last}.`,
fix: `The check ran past CONFIG.timeoutMs during ${last}. Raise timeoutMs if the migrations are large; a migration that sleeps or loops must not run here.`,
},
}
}
if (r.signal || r.error) {
return {
ok: false,
problem: {
code: 'CHILD-FAILED',
object: last,
message: `The check process ${r.signal ? `was killed by ${r.signal}` : `failed (${r.error.code ?? r.error.message})`} during ${last}.`,
fix: 'If memory ran out, give Node more room (the check needs about 1.1 GB); otherwise run it again with --timing to see where it stopped.',
},
}
}
try {
return JSON.parse(String(r.stdout))
} catch {
return {
ok: false,
problem: {
code: 'CHILD-FAILED',
object: last,
message: `The check process exited ${r.status} during ${last} without a result.\n${stderr.split('\n').slice(-20).join('\n')}`.trim(),
fix: 'Run it again with --timing to see where it stopped.',
},
}
}
}
/**
* The child: boot once, run every phase against the one database, close it,
* and exit explicitly (PGlite keeps the process alive otherwise). Everything
* comes from the job on stdin; CONFIG and contract.json are never read here.
*/
async function engine(job) {
const clock = makeClock((name) => {
try {
writeSync(2, `phase ${name}\n`)
} catch {
// A full stderr pipe never stops the check.
}
})
const { config } = job
let db = null
try {
clock.phase('boot')
if (job.dialect === 'sqlite') return await engineSqlite(job, clock)
await loadPGlite()
let snapshot = job.snapshot
let dumped = false
if (!snapshot) {
const prepared = await prepareInputs(job.inputs, config, { expiredStubs: job.expiredStubs })
db = await openDatabase({ config, extensions: prepared.extensions, clock })
db = await loadSchema(db, prepared, config, clock)
clock.phase('introspect')
const found = await introspect(db, config)
clock.phase('lints')
const lints = await runLints(db, config, found)
snapshot = makeSnapshot({
found, lints, prepared, config, inputsHash: job.inputsHash, pgliteVersion: job.pgliteVersion, sourceKind: job.sourceKind, via: job.via,
})
if (job.dumpTo) {
clock.phase('dump')
const blob = await db.dumpDataDir('gzip')
writeFileSync(job.dumpTo, Buffer.from(await blob.arrayBuffer()))
dumped = true
}
} else {
db = await openDatabase({ config, extensions: snapshot.loader.extensions, dataDir: new Blob([readFileSync(job.datadir)]), clock })
}
let deny = null
// 'auto' on plain Postgres with no RLS anywhere is app-code access: nothing to deny-test.
if (job.wantDeny && !(config.enforcement === 'auto' && snapshotNoRls(snapshot))) {
clock.phase('deny')
deny = await denyTests(db, snapshot, job)
}
clock.phase('close')
await db.close()
db = null
return { ok: true, snapshot: job.snapshot ? null : snapshot, deny, dumped, timing: clock.end() }
} catch (e) {
if (e instanceof CannotCheck) return { ok: false, problem: e.toJSON(), timing: clock.end() }
return {
ok: false,
problem: {
code: 'CHILD-FAILED',
object: '',
message: String(e?.stack ?? e),
fix: 'This is a bug in the check, or PGlite failed. Run it again with --timing to see where.',
},
timing: clock.end(),
}
} finally {
if (db) await db.close().catch(() => {})
}
}
/** The SQLite path of the child: node:sqlite in memory, no shim, no lints, no deny test. */
async function engineSqlite(job, clock) {
const { DatabaseSync } = await loadSqlite()
const db = new DatabaseSync(':memory:')
try {
const prepared = prepareSqlite(job.inputs)
loadSqliteSchema(db, prepared, job.config, clock)
clock.phase('introspect')
const found = introspectSqlite(db)
const snapshot = makeSnapshot({
found,
prepared,
config: job.config,
inputsHash: job.inputsHash,
pgliteVersion: process.versions.sqlite ?? null,
dialect: 'sqlite',
sourceKind: job.sourceKind,
via: job.via,
})
return { ok: true, snapshot, deny: null, dumped: false, timing: clock.end() }
} finally {
db.close()
}
}
async function childMain() {
const job = JSON.parse(readFileSync(0, 'utf8'))
const result = await engine(job)
process.stdout.write(JSON.stringify(result), () => process.exit(0))
}
// ------------------------------------------------------------------- RUN ---
const isoDay = (today) => (today instanceof Date ? today.toISOString().slice(0, 10) : String(today ?? new Date().toISOString()).slice(0, 10))
/**
* The whole check, as a function returning its exit code. The CLI below is a
* thin wrapper, so the tests can hand it a config, a contract and a date.
*/
export async function run({
argv = [],
cwd = process.cwd(),
config = CONFIG,
contract,
today = new Date(),
env = process.env,
log = console.log,
err = (s) => process.stderr.write(`${s}\n`),
} = {}) {
const started = performance.now()
const day = isoDay(today)
let opts
try {
opts = parseArgs(argv)
} catch (e) {
log(block(e.toJSON()))
return 2
}
const root = resolve(cwd, opts.root ?? '.')
// Where a command source with cwd: 'root' runs: --root, else the hook's NTENT_REPO_ROOT, else here.
const repoRoot = opts.root || !env.NTENT_REPO_ROOT ? root : resolve(env.NTENT_REPO_ROOT)
const meta = { booted: false, cached: 'none', platform: config?.platform ?? null, inputs: 0, tables: 0, denyTests: null }
const stop = (problems, exit = 2, notes = []) => report({ outcome: { findings: [], untested: problems, notes, waived: [], exit }, opts, log, meta })
// After a failed load: the other places this repo keeps a schema. The sources it read may be a patch folder without its base.
const elsewhere = (problems) => {
if (!problems.some((p) => p.code === 'LOAD-FAILED')) return []
const mine = canonical(config.sources)
const others = discoverSources(root).filter((c) => canonical(c.sources) !== mine)
if (!others.length) return []
const heading = 'The sources in CONFIG did not load. This repo keeps SQL in other places too, and the schema may live there, or need a base schema first. Paste a line over `sources:` in CONFIG to try it:'
return [{ code: 'OTHER-SOURCES', message: candidatesText(others, 3, heading) }]
}
// CONFIG first. A malformed allowlist entry refuses before anything else runs.
const configProblems = validateConfig(config)
if (configProblems.length) {
if (opts.printConfig) {
for (const p of configProblems) err(block(p))
return 2
}
return stop(configProblems)
}
if (opts.printConfig) {
let plan
try {
plan = planSources(config, { root, repoRoot, env }, { lenient: true })
} catch (e) {
if (!(e instanceof CannotCheck)) throw e
err(block(e.toJSON()))
return 2
}
const self = posix(relative(root, SELF))
const sourcePaths = config.sources.flatMap((s) => (s.kind === 'sql' ? s.paths.flatMap((p) => watchPaths(root, p)) : []))
const commandPaths = [
...config.sources.flatMap((source) => (source.kind === 'command' ? commandWatch(source, { root, repoRoot }) : [])),
...plan.items.flatMap((i) => (i.kind === 'command' ? i.watch : [])),
]
const watch = [...new Set([...sourcePaths, ...commandPaths, 'contract.json', self.startsWith('..') ? 'scripts/check-backend.mjs' : self])]
// With no input yet, the answer comes from the best candidate's SQL, so pasting that candidate does not change it (R34).
const candidates = plan.items.length ? [] : discoverSources(root)
const top = candidates.find((c) => !c.lowPriority && c.sources.every((s) => s.kind === 'sql'))
let items = plan.items
if (top) {
try {
items = planSources({ ...config, sources: top.sources }, { root, repoRoot, env }, { lenient: true }).items
} catch (e) {
if (!(e instanceof CannotCheck)) throw e
}
}
const settings = resolveSettings(config, { root, items })
if (top && items !== plan.items && config.platform === 'auto') settings.platformWhy += ` (read from candidate ${top.n}, ${top.path})`
log(`platform\t${items.some((i) => i.dialect === 'sqlite') ? 'sqlite' : settings.config.platform}`)
log(`platformWhy\t${settings.platformWhy}`)
log(`clientAccess\t${settings.config.clientAccess}`)
log(`clientAccessWhy\t${items.some((i) => i.dialect === 'sqlite') ? 'not used: SQLite has no row security, so access is not checked' : settings.clientWhy}`)
// What 'auto' will do, as far as the repo says now. Command sources load at the run, so they stay 'auto'.
let enforcement = config.enforcement
let hint = null
const platform = settings.config.platform
if (enforcement === 'auto') {
const fileOnly = items.length > 0 && items.every((i) => i.kind === 'file')
if (items.some((i) => i.dialect === 'sqlite')) {
enforcement = 'app'
hint = 'auto: SQLite has no row security, so access is not checked.'
} else if (platform === 'supabase') enforcement = 'rls'
else if (fileOnly && settings.noRls) {
enforcement = 'app'
hint = "auto: plain Postgres, and no migration enables row level security or creates a policy, so access is treated as enforced in app code and the deny tests are skipped. Set enforcement: 'app' in CONFIG to confirm."
} else if (fileOnly) enforcement = 'rls'
else if (items.length) hint = 'auto: decided when the schema loads. On plain Postgres with no row level security in it, access is treated as enforced in app code and the deny tests are skipped.'
} else if (enforcement === 'rls' && platform === 'postgres' && settings.noRls) hint = NO_RLS_HINT
log(`enforcement\t${enforcement}`)
if (hint) log(`enforcementHint\t${hint}`)
log(`loadGate\t${config.loadGate ?? 'gate'}`)
for (const w of watch) log(`watch\t${w}`)
for (const source of config.sources) log(`source\t${JSON.stringify(source)}`)
// With no input, look for one. With input, the hook runs this on every commit and the walk is not needed.
if (!plan.items.length) for (const c of candidates) log(`candidate\t${c.n}\t${c.path}\t${c.note}\t${c.line}`)
log(`inputCount\t${plan.items.length}`)
return 0
}
// The contract. Every field this reads is validated here.
if (contract === undefined) {
const file = join(root, 'contract.json')
if (existsSync(file)) {
try {
contract = JSON.parse(readFileSync(file, 'utf8'))
} catch (e) {
return stop([{
code: 'CONTRACT-SHAPE',
object: 'contract.json',
message: `is not valid JSON: ${e.message}`,
fix: 'Fix the JSON, or delete the file if this repo has no contract yet.',
}])
}
} else contract = null
}
let contractNotes = []
if (contract) {
const { errors, warnings } = validateContract(contract)
if (errors.length) return stop(errors)
contractNotes = warnings
}
// The inputs, in load order.
let plan
try {
plan = planSources(config, { root, repoRoot, env })
} catch (e) {
if (e instanceof CannotCheck) return stop([e.toJSON()])
throw e
}
const sqlite = plan.dialect === 'sqlite'
// 'auto' becomes supabase or postgres here, once, so nothing below sees it.
const settings = resolveSettings(config, { root, items: plan.items })
meta.platformNote = sqlite || config.platform !== 'auto' ? null : settings.platformWhy
config = settings.config
// SQLite names are shown without the public. schema it is keyed by, so a key copied from the report has to match too.
if (sqlite) {
const keyed = (k) => (k.startsWith('public.') ? k : `public.${k}`)
config = {
...config,
schemaWaivers: Object.fromEntries(Object.entries(config.schemaWaivers ?? {}).map(([k, v]) => [keyed(k), v])),
ignoreTables: (config.ignoreTables ?? []).map(keyed),
}
}
meta.platform = sqlite ? 'sqlite' : config.platform
meta.inputs = plan.items.length
// 'auto' on Supabase is 'rls', on SQLite 'app'. On plain Postgres it waits for the loaded schema: no RLS anywhere is 'app'.
let enforcementAuto = false
if (config.enforcement === 'auto') {
enforcementAuto = true
if (sqlite) config = { ...config, enforcement: 'app' }
else if (config.platform === 'supabase') config = { ...config, enforcement: 'rls' }
}
if (!plan.items.length) {
const paths = config.sources.flatMap((s) => (s.kind === 'sql' ? s.paths : commandWatch(s, { root, repoRoot }))).join(', ')
const candidates = discoverSources(root)
const problem = {
code: 'NO-INPUT',
object: paths,
message: `No migration files, so nothing was checked.${candidates.length ? `\n${candidatesText(candidates)}` : ''}`,
fix: candidates.length
? `No migration files at ${paths}. Paste one of the sources lines above into CONFIG, or write the first migration.`
: `No migration files at ${paths}, and none found anywhere in the repo. Point CONFIG.sources at them, or write the first one.`,
}
if (opts.allowEmpty) return report({ outcome: { findings: [], untested: [], notes: [problem], waived: [], exit: 0 }, opts, log, meta })
return stop([problem])
}
const files = plan.items.map((i) => (i.kind === 'file' ? { ...i, bytes: readFileSync(join(root, i.path)) } : i))
contractNotes = [...contractNotes, ...unstagedNotes(plan.items), ...unlistedMigrationNotes(root, config.sources, plan.items)]
// Allowlist dates are evaluated now, never cached. An expired stub is not
// passed to the child, so the extension it covered is unavailable again.
const stubExtensions = {}
const expiredStubs = {}
for (const [name, entry] of Object.entries(config.stubExtensions)) {
if (day > entry.until) expiredStubs[name] = entry.until
else stubExtensions[name] = entry
}
// Same for a skipped statement: past its date it runs, and fails as it would.
const skipStatements = Object.fromEntries(Object.entries(config.skipStatements ?? {}).filter(([, entry]) => day <= entry.until))
const childConfig = { ...config, stubExtensions, skipStatements }
// Cache keys.
const version = sqlite ? `node:sqlite ${process.versions.sqlite ?? ''}` : pgliteVersion()
const inputsHash = sha256(...files.flatMap((f) => (f.kind === 'file' ? [f.path, '\0', sha256(f.bytes), '\0'] : [f.label, '\0', f.hash, '\0'])))
const snapshotKey = sha256(
canonical([
SNAPSHOT_FORMAT, SHIM_VERSION, version, sha256(selfWithoutConfig()), plan.dialect,
config.platform, config.sources, config.exposedSchemas, config.supabase, stubExtensions, skipStatements, config.preamble,
// The snapshot's grants include actAs.role, so it is part of the key.
config.actAs.role,
]),
inputsHash,
)
const denyKey = sha256(
snapshotKey,
canonical([
contract?.access ?? null,
contract?.entities ?? null,
(contract?.actors ?? []).map((a) => (isObject(a) ? { id: a.id ?? null, kind: a.kind ?? null } : a)),
config.actors, config.actAs, config.seed, config.columnNames,
]),
)
const cacheDir = resolve(root, config.cacheDir)
const snapshotFile = join(cacheDir, `${snapshotKey}.snapshot.json`)
const datadirFile = join(cacheDir, `${snapshotKey}.datadir.tgz`)
const denyFile = join(cacheDir, `${denyKey}.deny.json`)
let snapshot = opts.noCache ? null : readJson(snapshotFile)
if (snapshot?.format !== SNAPSHOT_FORMAT) snapshot = null
// A cached snapshot settles 'auto' here. A cold run settles it after the child has loaded the schema.
const autoPending = () => config.enforcement === 'auto'
if (snapshot && autoPending()) config = { ...config, enforcement: snapshotNoRls(snapshot) ? 'app' : 'rls' }
const wantDeny = !sqlite && opts.gates.access && (config.enforcement === 'rls' || autoPending()) && !!contract && !opts.printSnapshot
let deny = wantDeny && !opts.noCache && snapshot ? readJson(denyFile) : null
const readMs = performance.now() - started
let timing = {}
if (!snapshot || (wantDeny && !deny)) {
const fromDatadir = !!snapshot && existsSync(datadirFile)
if (snapshot && !fromDatadir) snapshot = null
// The commands run only here, on a miss: the cache key already holds their
// inputs. One that fails is exit 2 before anything boots or is cached.
let inputTexts = []
let commandMs = null
if (!snapshot) {
const since = performance.now()
try {
inputTexts = files.map((f) =>
f.kind === 'file'
? { path: f.path, sections: f.sections, late: f.late, ...decodeSql(f.bytes) }
: { path: f.label, sections: null, text: runCommand(f, { timeoutMs: config.commandTimeoutMs }), command: f.display },
)
} catch (e) {
if (!(e instanceof CannotCheck)) throw e
if (opts.printSnapshot) {
err(block(e.toJSON()))
return 2
}
return stop([e.toJSON()])
}
if (files.some((f) => f.kind === 'command')) commandMs = Math.round(performance.now() - since)
}
// No cache directory is made for a run that cannot boot: a repo without
// PGlite gets PGLITE-MISSING, not a node_modules folder.
const dumpTo = !sqlite && !snapshot && !opts.noCache && version !== 'missing' ? `${datadirFile}.tmp-${process.pid}` : null
if (dumpTo) mkdirSync(cacheDir, { recursive: true })
const job = {
argv,
root,
config: childConfig,
contract,
today: day,
inputs: inputTexts,
dialect: plan.dialect,
sourceKind: [...new Set(files.map((f) => f.kind))].length > 1 ? 'mixed' : files[0].kind === 'command' ? 'command' : 'sql',
via: [...new Set(files.filter((f) => f.kind === 'command').map((f) => f.label))],
inputsHash,
pgliteVersion: version,
expiredStubs,
snapshot,
datadir: snapshot ? datadirFile : null,
dumpTo,
wantDeny,
}
const result = spawnChild(job, { config, opts, err, root })
meta.booted = true
meta.cached = snapshot ? 'datadir' : 'none'
timing = { ...(commandMs === null ? {} : { command: commandMs }), ...(result.timing ?? {}) }
if (!result.ok) {
try {
if (dumpTo && existsSync(dumpTo)) unlinkSync(dumpTo)
} catch {
// Already gone.
}
if (opts.timing) err(timingLine(readMs, timing, started))
if (opts.printSnapshot) {
for (const b of problemBlocks(result.problem.problems ?? [result.problem])) err(b)
return 2
}
// A load failure carries every failure it found, one problem each.
const problems = result.problem.problems ?? [result.problem]
// Warn mode softens a load that fails and nothing else. Any other problem in the list keeps exit 2.
if (config.loadGate === 'warn' && problems.every((p) => LOAD_CODES.has(p.code))) {
meta.loadWarn = true
return stop(problems, 0, elsewhere(problems))
}
return stop(problems, 2, elsewhere(problems))
}
if (!snapshot) snapshot = result.snapshot
if (autoPending()) config = { ...config, enforcement: snapshotNoRls(snapshot) ? 'app' : 'rls' }
deny = result.deny
if (!opts.noCache) {
// The cache is only a speed-up: a write or prune that fails (a run
// sharing the directory pruned a file, a read-only disk) never changes
// the result.
try {
mkdirSync(cacheDir, { recursive: true })
if (result.snapshot) writeAtomic(snapshotFile, JSON.stringify(result.snapshot))
if (result.dumped && dumpTo) renameSync(dumpTo, datadirFile)
if (wantDeny && deny) writeAtomic(denyFile, JSON.stringify(deny))
prune(cacheDir)
} catch {
// Skip the cache this run.
}
}
} else {
meta.cached = 'full'
}
if (opts.printSnapshot) {
log(JSON.stringify(snapshot, null, 2))
if (opts.timing) err(timingLine(readMs, timing, started))
return 0
}
meta.tables = Object.values(snapshot.tables).filter((t) => !t.platform).length
if (wantDeny && deny?.built) meta.denyTests = deny.tests.length
const outcome = evaluate({ snapshot, contract, config, gates: opts.gates, today: day, deny, contractNotes, enforcementAuto })
meta.accessNote = outcome.accessNote ?? null
if (config.loadGate === 'warn' && !opts.printSnapshot) {
// It loads now. Exit 1 until the switch is committed, so warn mode cannot outlive the problem it was for.
outcome.findings = [
{
code: 'LOAD-GATE-READY',
object: 'CONFIG.loadGate',
message: "the migrations load now, and CONFIG.loadGate is still 'warn', so a later load failure would not block.",
fix: "Set CONFIG.loadGate to 'gate' in scripts/check-backend.mjs and commit it. From then on a migration set that does not load is exit 2.",
},
...outcome.findings,
]
outcome.exit = Math.max(outcome.exit ?? exitOf(outcome), 1)
}
const code = report({ outcome, opts, log, meta })
if (opts.timing) err(timingLine(readMs, timing, started))
return code
}
const timingLine = (readMs, timing, started) => {
const parts = [`read=${Math.round(readMs)}ms`, ...Object.entries(timing).map(([k, v]) => `${k}=${v}ms`)]
return `timing ${parts.join(' ')} total=${((performance.now() - started) / 1000).toFixed(2)}s`
}
// ------------------------------------------------------------ ENTRYPOINT ---
const invokedDirectly = (() => {
try {
return realpathSync(process.argv[1] ?? '') === realpathSync(SELF)
} catch {
return false
}
})()
if (invokedDirectly) {
if (process.argv.includes('--child')) {
await childMain()
} else {
// A crash is exit 2, never 1: a stack trace is not a finding about the database.
const code = await run({ argv: process.argv.slice(2), cwd: process.cwd(), config: CONFIG, today: new Date() }).catch((e) => {
console.error(e?.stack ?? e)
return 2
})
// Exit once stdout has drained: on a pipe, exiting at once cuts a long
// --json or --print-snapshot off at 64 KB.
process.stdout.write('', () => process.exit(code))
}
}
How to tell it works: Write a migration that creates a table in public without enabling row level security, and the commit is refused naming RLS-OFF and the table. Enable RLS with a policy on auth.uid() and it passes. Map an entity to that table with scope own, change the policy to using (true), and the refusal says LEAK-READ: user A read a row that belongs to user B. A table the check cannot fill with test rows refuses the commit with the column it could not fill, never a pass.
Included in
- Any setup that adds Database, for when it stores data in postgres, sqlite or supabase