Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions .changeset/27705-searchable-credential-fields.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
'@objectstack/lint': minor
'@objectstack/spec': patch
'@objectstack/objectql': patch
---

feat(lint)!: `searchable-field-unsearchable` reports a `searchableFields` entry that names a `secret` or `password` field, which the engine leaves out of every search without a word

Clause-②: no (narrowing) — `os validate`, `os build`, `os lint` and the metadata publish door refuse a `searchableFields` entry whose field type is `secret` or `password`, on the object's own set, on every list view surface and on a react page's ListView `searchableFields` prop, which they accepted before. The finding takes the severity of the `formula`-entry finding (`error`). No rule id is added or removed.
Metadata-protocol: narrowed

<!-- adr-0087: not-required (no-migration-prescription) No key of any schema is added, removed, renamed or re-shaped: a searchableFields entry the engine already withheld from every search (ADR-0100) is now refused where it is authored. The one remedy is to delete the entry, which a conversion cannot decide for the author, so there is no key for a migration to rewrite and no tombstone. -->

**BREAKING** accept-set narrowing at every authoring door for `searchableFields`.

- **Before.** The engine's search expansion leaves every credential column (`secret`, `password`) out of the set it scans (ADR-0100), an author-declared one included, and says nothing. A declaration naming one passed every gate and searched a narrower set than written; a declaration naming only credential fields fell through to the auto-default set.
- **Now.** The rule refuses the entry at `error`, naming the entry, its type and the object. It is judged on every object, a better-auth `password` included: the read mask spares that one, the search withhold does not.
- **Who sees it.** An object or list view whose `searchableFields` names a `secret` or `password` field. No metadata shipped in this repository does.

**The remedy.** Drop the entry: credential fields are never searched. `os explain searchable-field-unsearchable` states what the engine does with it on each surface.

`@objectstack/spec` exports `isCredentialFieldType(fieldType)` from `@objectstack/spec/data`: whether a field type is masked on read on at least one object (`secret`, `password`), the exemption-free question. `@objectstack/objectql`'s `collectCredentialFields` now reads it instead of its own type literal; its answer is unchanged.
23 changes: 18 additions & 5 deletions packages/lint/src/rule-explanations.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2465,11 +2465,24 @@ const SEARCHABLE_FIELD_UNSEARCHABLE_EXPLANATION: RuleExplanation = {
VIRTUAL_FIELD_STORAGE,
'A formula entry is refused on EVERY surface, the object\'s own set included — the REST ingress ' +
'answers `400 INVALID_FIELD`, the same answer a stale entry gets: measured, it ' +
'matched 0 rows on driver-memory, and 0 rows with no error on driver-sql. Otherwise the ' +
'object\'s own `searchableFields` is existence-only: its declared branch scans any stored ' +
'column, so a json or lookup column declared there is a narrow choice the engine executes, not ' +
'a finding. A system column outside the allowed set is not judged either: its runtime ' +
'metadata is registry-owned and invisible here.',
'matched 0 rows on driver-memory, and 0 rows with no error on driver-sql. Otherwise, apart ' +
'from a credential entry, the object\'s own `searchableFields` is existence-only: its declared ' +
'branch scans any stored column, so a json or lookup column declared there is a narrow choice ' +
'the engine executes, not a finding. A system column outside the allowed set is not judged ' +
'either: its runtime metadata is registry-owned and invisible here.',
'A `secret` or `password` entry is refused on EVERY surface as well, the object\'s own set ' +
'included. The engine leaves every credential column out of the set a search scans (ADR-0100), ' +
'an author-declared one included, and says nothing: a declaration naming one scans a narrower ' +
'set than written, and one naming only credential fields falls through to the AUTO-DEFAULT ' +
'set. A list view naming one is refused with `400 INVALID_FIELD` when the object\'s set is the ' +
'auto-default, which never admits a credential type; when the object declares the entry, the ' +
'REST ingress admits it and the engine withholds it, so a view naming only credential fields ' +
'searches the object\'s whole set. Credential types are judged by the spec\'s ' +
'`isCredentialFieldType`, the predicate the engine\'s credential collector reads, with no ' +
'better-auth exemption: the read mask spares a better-auth `password`, the search withhold ' +
'does not. An error, as a formula entry is: the entry can never match, and the narrowing is ' +
'otherwise silent. The one fix is to drop it — mirroring the value onto a searchable column ' +
'would publish what the withhold protects.',
],
};

Expand Down
16 changes: 16 additions & 0 deletions packages/lint/src/runtime-gate.object-writes.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -301,6 +301,22 @@ describe('the object write door dispatches at the adjudicated scope (#4716)', ()
expect(f.message).toContain('gone_field');
});

it('REFUSES an object whose searchableFields names a credential field the engine never searches', () => {
// The same rule's credential arm, at the publish door: the engine withholds
// a `secret` / `password` column from every search (ADR-0100), so the
// declaration is refused where a Studio tenant writes it.
const result = expectSingleRefusal(
cleanObject({
fields: { owner: { type: 'text' }, api_key: { type: 'secret' } },
searchableFields: ['owner', 'api_key'],
}),
'searchable-field-unsearchable',
);
const f = result.errors.find((e) => e.rule === 'searchable-field-unsearchable')!;
expect(f.path).toBe('objects.leave_request.searchableFields[1]');
expect(f.message).toContain("is a 'secret' field, a credential");
});

it('REFUSES an object whose built-in list view names a column it does not have', () => {
const result = expectSingleRefusal(
cleanObject({ listViews: { all: { label: 'All', columns: ['owner', 'gone_column'] } } }),
Expand Down
18 changes: 18 additions & 0 deletions packages/lint/src/validate-react-page-props.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -520,6 +520,24 @@ describe('validateReactPageProps — <ListView> searchableFields (#4329)', () =>
expect(f[0].message).toContain('400 INVALID_FIELD');
});

it('flags a secret or password entry under the metadata verdict — the shared core\'s credential check', () => {
const vault = {
name: 'crm_account',
fields: { name: { type: 'text' }, api_key: { type: 'secret' }, portal_password: { type: 'password' } },
};
const f = pastDeprecation(validateReactPageProps(
listPage(
list(`data={{ provider: 'object', object: 'crm_account' }} searchableFields={['name', 'api_key', 'portal_password']}`),
[vault],
),
));
expect(f.map((x) => [x.rule, x.severity, x.path])).toEqual([
[SEARCHABLE_FIELD_UNSEARCHABLE, 'error', 'pages[0].source › searchableFields[1]'],
[SEARCHABLE_FIELD_UNSEARCHABLE, 'error', 'pages[0].source › searchableFields[2]'],
]);
expect(f[0].hint).toBe('Drop "api_key": credential fields are never searched');
});

it('[#8404] warns on an unprovisioned anchor, and PINS that this surface threads the index', () => {
// Load-bearing beyond the warning itself. `validateReactPageProps` builds
// `unprovisionedAnchors` for its other field-prop checks, so if this
Expand Down
126 changes: 124 additions & 2 deletions packages/lint/src/validate-searchable-fields.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -939,6 +939,127 @@ describe('[#6674] validateSearchableFields — a virtual formula entry', () => {
});
});

/**
* A credential entry. Since the engine's ADR-0100 search withhold, the search
* expansion leaves every `secret` / `password` column out of the field map it
* resolves over, an author-declared one included, and says nothing — so the
* declared set is silently narrower than written. The finding is that
* withhold's build-time voice, at the virtual entry's severity.
*/
describe('validateSearchableFields — a secret or password entry', () => {
const accountFields = {
name: { type: 'text' },
billing_email: { type: 'email' },
api_key: { type: 'secret' },
portal_password: { type: 'password' },
};

it.each([
['secret', 'api_key'],
['password', 'portal_password'],
])("flags a '%s' entry on the OBJECT's own searchableFields — previously clean", (ctype, field) => {
const findings = validateSearchableFields({
objects: [
{ name: 'crm_account', fields: accountFields, searchableFields: ['name', field, 'billing_email'] },
],
});

// The text and email entries beside it stay clean: the check reads the TYPE.
expect(findings).toHaveLength(1);
expect(findings[0].rule).toBe(SEARCHABLE_FIELD_UNSEARCHABLE);
expect(findings[0].severity).toBe('error');
expect(findings[0].path).toBe('objects[0].searchableFields[1]');
expect(findings[0].where).toBe('object "crm_account"');
expect(findings[0].message).toContain(`"${field}"`);
expect(findings[0].message).toContain(`is a '${ctype}' field, a credential`);
expect(findings[0].hint).toBe(`Drop "${field}": credential fields are never searched`);
});

it.each([
['secret', 'api_key'],
['password', 'portal_password'],
])("flags a '%s' entry on a list view narrowing, under the same verdict", (ctype, field) => {
const findings = validateSearchableFields({
objects: [
{
name: 'crm_account',
fields: accountFields,
listViews: { all: { type: 'grid', searchableFields: ['name', field] } },
},
],
});

// Auto-default object: without the credential check the admissibility pass
// would explain this entry by its type alone and offer a text-like field.
expect(findings).toHaveLength(1);
expect(findings[0].rule).toBe(SEARCHABLE_FIELD_UNSEARCHABLE);
expect(findings[0].severity).toBe('error');
expect(findings[0].path).toBe('objects[0].listViews.all.searchableFields[1]');
expect(findings[0].message).toContain(`is a '${ctype}' field, a credential`);
expect(findings[0].hint).toBe(`Drop "${field}": credential fields are never searched`);
});

it('flags a view entry the object ITSELF declares — the declared set does not make it searchable', () => {
// The object's declaration admits the entry at the REST ingress, so the
// #4830 admissibility pass passes it; the engine still withholds it.
const findings = validateSearchableFields({
objects: [
{
name: 'crm_account',
fields: accountFields,
searchableFields: ['name', 'api_key'],
listViews: { all: { type: 'grid', searchableFields: ['api_key'] } },
},
],
});

expect(findings.map((f) => [f.rule, f.path])).toEqual([
[SEARCHABLE_FIELD_UNSEARCHABLE, 'objects[0].searchableFields[1]'],
[SEARCHABLE_FIELD_UNSEARCHABLE, 'objects[0].listViews.all.searchableFields[0]'],
]);
});

it('is exemption-free: a `password` on a better-auth object is reported too', () => {
// The read mask spares a better-auth `password`; the search withhold does not.
const findings = validateSearchableFields({
objects: [
{
name: 'crm_account',
managedBy: 'better-auth',
fields: accountFields,
searchableFields: ['name', 'portal_password'],
},
],
});

expect(findings).toHaveLength(1);
expect(findings[0].path).toBe('objects[0].searchableFields[1]');
expect(findings[0].message).toContain("is a 'password' field, a credential");
});

it('CONTROL — the same declaration with text and email entries only stays clean', () => {
expect(
validateSearchableFields({
objects: [
{
name: 'crm_account',
fields: accountFields,
searchableFields: ['name', 'billing_email'],
listViews: { all: { type: 'grid', searchableFields: ['name', 'billing_email'] } },
},
],
}),
).toEqual([]);
});

it('`os explain` names the silent withhold and the exemption-free predicate', () => {
const text = explanationOf(SEARCHABLE_FIELD_UNSEARCHABLE);
expect(text).toContain('A `secret` or `password` entry is refused on EVERY surface');
expect(text).toContain('`isCredentialFieldType`');
expect(text).toContain('no better-auth exemption');
});
});

/**
* [#8404] The FIFTH blanket-`SYSTEM_FIELDS` read site. Existence and provenance
* are different questions: on an ADR-0015 `external` object the platform
Expand Down Expand Up @@ -1132,13 +1253,14 @@ describe('[#22161] one-line verdicts — the three searchable ids', () => {
describe('[#22161] one-line fixes — the three searchable ids', () => {
it('every `fix:` line the cases above fired is one line of at most 200 characters', () => {
// Coverage control: every arm fired — both unknown arms, the anchor, and
// the four unsearchable arms (virtual, outside a declared set, a reference
// column and any other type outside the auto-default set).
// the five unsearchable arms (virtual, credential, outside a declared set,
// a reference column and any other type outside the auto-default set).
const hints = fired.map((f) => f.hint);
expect(hints.some((h) => h.startsWith('Fix the name, or add "')), 'unknown name').toBe(true);
expect(hints.some((h) => h.startsWith("'search' scans this object's own columns only")), 'dotted').toBe(true);
expect(hints.some((h) => h.startsWith("If the remote table carries '")), 'anchor').toBe(true);
expect(hints.some((h) => h.startsWith('Mirror the computed value')), 'virtual').toBe(true);
expect(hints.some((h) => h.endsWith(': credential fields are never searched')), 'credential').toBe(true);
expect(hints.some((h) => h.startsWith('Add "') && h.includes('.searchableFields')), 'declared set').toBe(true);
expect(hints.some((h) => h.includes("to search by the related record's title")), 'reference').toBe(true);
expect(hints.some((h) => h.endsWith('or target a text-like field instead')), 'other type').toBe(true);
Expand Down
56 changes: 54 additions & 2 deletions packages/lint/src/validate-searchable-fields.ts
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,30 @@
* WITH NO ERROR on driver-sql. Since #6674 the spec resolution drops such
* an entry and both enforcement faces refuse it by name.
*
* 4. CREDENTIALS, on EVERY surface — the object's own set included
* (`searchable-field-unsearchable`): a `secret` or `password` entry names a
* real, stored field, so checks 1 and 3 pass it — but since the engine's
* ADR-0100 search withhold the expansion leaves every credential column out
* of the field map it resolves over (`expandSearchOnAst`,
* `@objectstack/objectql`, by `collectCredentialFields`), an author-declared
* one included, and says nothing. The declared set is silently narrower
* than written, and a declaration naming ONLY credential fields falls
* through to the auto-default set — #4254's "asked narrower, answered
* wider" again. Judged by the spec's own predicate
* ({@link isCredentialFieldType}, `@objectstack/spec/data`), the one that
* engine collector reads, so linter and engine cannot disagree about which
* types are credentials. The withhold is exemption-free (a better-auth
* `password` is withheld too), and so is this check.
*
* `error`, the virtual entry's severity, deliberately and for the virtual
* entry's reason: the entry can never match wherever it is declared, and a
* narrowing the runtime makes in silence gets its word at build time. A
* warning would leave `os validate` passing a declaration the runtime does
* not honour, which is the gap this check exists to close. The remedy has
* one form — drop the entry — since no mirror of a credential onto a
* searchable column is a fix: that would publish the very value the
* withhold protects.
*
* The OBJECT's own `searchableFields` stays existence-only OTHERWISE, and the
* dividing line is STORAGE, not search quality: the runtime's declared branch
* filters by existence and scannability, never by type taste, so a json or
Expand Down Expand Up @@ -121,6 +145,7 @@

import {
resolveSearchFieldResolution,
isCredentialFieldType,
isVirtualSearchField,
SEARCH_AUTO_EXCLUDED_FIELDS,
type SearchFieldMeta,
Expand Down Expand Up @@ -159,8 +184,9 @@ export interface SearchableFieldFinding {
*
* - `'canonical'` — the object's own `searchableFields`. The runtime honors
* any entry that exists and has a stored column to scan (existence- and
* scannability-filtered, never type-filtered), so existence and virtuality
* are checked and nothing else (#6674).
* scannability-filtered, never type-filtered), apart from the credential
* columns the engine withholds, so existence, virtuality and credentials
* are checked and nothing else (#6674, module note check 4).
* - `'narrowing'` — a list view's `searchableFields` (metadata or react
* surface). Clients echo it as the `$searchFields` override, which the
* #4254 ingress gate intersects with the object's allowed set — entries the
Expand Down Expand Up @@ -430,6 +456,32 @@ export function checkSearchableFieldList(
continue;
}

// ── Credential entries — EVERY surface, canonical included ──
//
// The engine's search expansion withholds every `secret` / `password`
// column from the set it scans (ADR-0100), a declared one included, without
// a word; this is that withhold's build-time voice (module note, check 4).
// Judged by the spec predicate the engine's collector reads. Ahead of the
// admissibility pass on purpose: on a view over an auto-default object
// that pass would otherwise explain the entry by its type alone, and the
// remedy it offers ("target a text-like field") is not this one.
const ctype = target.fields[name]?.type;
if (isCredentialFieldType(ctype)) {
findings.push({
severity: 'error',
rule: SEARCHABLE_FIELD_UNSEARCHABLE,
where,
path: `${path}[${i}]`,
message:
`${subject} entry "${name}" on object "${objectName}" is a '${ctype}' field, a ` +
`credential: 'search' never scans it`,
// One remedy line; what the engine does with the entry on each surface
// is the id's `os explain` entry.
hint: `Drop "${name}": credential fields are never searched`,
});
continue;
}

// ── Runtime admissibility (#4830) — view-level narrowings only ──
if (!resolution || resolution.allowed.has(name)) continue;
// ③ System column outside the allowed set: its runtime metadata is
Expand Down
4 changes: 2 additions & 2 deletions packages/objectql/src/secret-fields.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
* ruling of 2026-08-12 on #7728.
*/

import { isMaskedOnReadFieldType, type ServiceObject } from '@objectstack/spec/data';
import { isCredentialFieldType, isMaskedOnReadFieldType, type ServiceObject } from '@objectstack/spec/data';

/**
* Prefix marking a persisted field value as a `sys_secret` handle ref rather
Expand Down Expand Up @@ -291,7 +291,7 @@ export function collectCredentialFields(schema: ServiceObject | undefined | null
if (!fields) return [];
const out: string[] = [];
for (const [name, def] of Object.entries(fields)) {
if (def && (def.type === 'secret' || def.type === 'password')) out.push(name);
if (def && isCredentialFieldType(def.type)) out.push(name);
}
return out;
}
Expand Down
1 change: 1 addition & 0 deletions packages/spec/api-surface/data.json
Original file line number Diff line number Diff line change
Expand Up @@ -909,6 +909,7 @@
"isComposedReadableIds (function)",
"isContextToken (function)",
"isControlledByParentMasterField (function)",
"isCredentialFieldType (function)",
"isCurrentUserDefaultToken (function)",
"isDateMacroToken (function)",
"isDateRangePresetName (function)",
Expand Down
Loading
Loading