Skip to content

docs(flagship): update API envelope docs and add A/B/n rollout pattern#59

Merged
threepointone merged 5 commits into
cloudflare:mainfrom
akshitsinha:asinha/flagship-abn-testing-docs
Jun 11, 2026
Merged

docs(flagship): update API envelope docs and add A/B/n rollout pattern#59
threepointone merged 5 commits into
cloudflare:mainfrom
akshitsinha:asinha/flagship-abn-testing-docs

Conversation

@akshitsinha

@akshitsinha akshitsinha commented Jun 4, 2026

Copy link
Copy Markdown
Member

Updates Flagship Cloudflare skill references for current control-plane behavior and adds guidance for multi-variant A/B/n experiments.

Changes:

  • Document current Cloudflare v4 response envelope for management endpoints:
    • payloads under result
    • pagination cursor under result_info.cursor
    • errors under errors[].message
  • Remove unsupported legacy response-field usage from examples.
  • Update REST examples to use jq '.result'.
  • Clarify management API response casing:
    • snake_case fields such as created_at, updated_at, updated_by, default_variation, serve_variation, and changelog flag_key
  • Clarify /evaluate response behavior:
    • not wrapped in management envelope
    • OpenFeature-style camelCase (flagKey, variant, reason)
  • Add response-shape examples for apps, flags, and changelog entries.
  • Add gotchas for envelope access and casing differences between management responses and evaluation responses.
  • Add A/B/n multi-variant testing pattern:
    • one rule per variant
    • cumulative rollout thresholds
    • example 30% / 40% / 30% split across variants A/B/C
    • guidance to use conditions: [] and set final threshold to 100

Validation:

  • Reviewed branch diff against origin/main.
  • Verified no unsupported .data, nextCursor, or single .error references remain in Flagship docs.

@akshitsinha akshitsinha marked this pull request as draft June 4, 2026 09:31
@akshitsinha akshitsinha changed the title feat(flagship): add A/B/n multi-variant testing pattern with cumulative rollout example docs(flagship): update API envelope docs and add A/B/n rollout pattern Jun 9, 2026
@akshitsinha akshitsinha marked this pull request as ready for review June 9, 2026 07:57

@kale-stew kale-stew left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this — the envelope and casing cleanup is the kind of correctness fix the skill needed. The .data.result, nextCursorresult_info.cursor, and errors[].message changes are consistent across all three files, and I confirmed nothing stale lingers elsewhere in the flagship dir (including the untouched configuration.md). That part looks ready.

My main hesitation is that the two new factual claims — the A/B/n rollout semantics and the /evaluate/response shapes — don't seem to have been exercised against the live API yet. A few of them are also presented pretty authoritatively, so I'd love to confirm them before they become the reference people trust. Left specifics inline. The envelope fixes feel safe to ship regardless; I'd just hold the A/B/n section until we can verify the bucketing behavior with a real multi-rule test.

Two nits not worth inlining: the SPLIT reason (it exists in the details type but isn't mentioned in the new A/B/n section), and "30 %" spacing being inconsistent with 10%/25%/50% elsewhere in the file.

Comment thread skills/cloudflare/references/flagship/patterns.md Outdated
Comment thread skills/cloudflare/references/flagship/patterns.md Outdated
Comment thread skills/cloudflare/references/flagship/api.md Outdated
Comment thread skills/cloudflare/references/flagship/api.md Outdated
Comment thread skills/cloudflare/references/flagship/api.md

@moshahin19841984-coder moshahin19841984-coder left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok

@kale-stew kale-stew left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work on this — the mechanical envelope migration is clean and I was able to verify the key claims against the live Cloudflare API docs.

What's solid:

  • All .data.result, nextCursorresult_info.cursor, and .errorerrors[].message conversions are accurate and match the current v4 envelope.
  • The management response shape examples (app, flag, changelog) line up with the live API reference.
  • The /evaluate endpoint description is correct: not enveloped, camelCase, with the right field names and reason values (TARGETING_MATCH, SPLIT, DEFAULT, DISABLED).
  • The A/B/n multi-variant pattern is well-explained and matches the cumulative-threshold guidance in the official docs (e.g. 30/70/100).
  • Good addition of the source-of-truth API reference link.

Questions / suggestions:

  • The evaluate endpoint permission wording changed from flagship:evaluate to Flagship read permissions. I couldn't confirm the exact token scope name from the public API docs — if the product currently uses a specific permission like flagship:evaluate, keeping the precise name avoids users creating tokens with the wrong scope.
  • In the A/B/n example, every rule uses conditions: [{ attribute: "targetingKey", operator: "not_equals", value: "" }]. The API docs note that an empty conditions: [] matches all contexts. Using [] would make the catch-all intent clearer and remove the redundant not_equals "" guard.

Nothing blocking — mostly minor nits and one verification question.

Comment thread skills/cloudflare/references/flagship/api.md Outdated
Comment thread skills/cloudflare/references/flagship/patterns.md Outdated

@kale-stew kale-stew left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approved w/ comments (left above)

@threepointone threepointone merged commit 12520fd into cloudflare:main Jun 11, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants