Skip to content

feat(context): track context tokens#2009

Open
lizradway wants to merge 2 commits intostrands-agents:mainfrom
lizradway:messages-token-size
Open

feat(context): track context tokens#2009
lizradway wants to merge 2 commits intostrands-agents:mainfrom
lizradway:messages-token-size

Conversation

@lizradway
Copy link
Copy Markdown
Member

@lizradway lizradway commented Mar 30, 2026

Description

needs api review

Add a latest_context_tokens property to EventLoopMetrics that surfaces the most recent context window size (in tokens) as reported by the model.

Strands currently has no visibility into how full the context window is. The LLM already reports inputTokens on every call, but that data was buried in per-cycle usage metrics. This property exposes it as a simple, top-level read:

agent = Agent(tools=[...])
result = agent("Do something with tools")
context_size = agent.event_loop_metrics.latest_context_tokens

This is a lagging indicator (post-call), not a pre-call estimate. It enables downstream features like compression and externalization to make threshold-based decisions between invocations.

  • Zero overhead: reuses inputTokens already returned by every LLM call
  • Zero latency impact: no additional API calls
  • Provider agnostic: all providers normalize to the Usage TypedDict with inputTokens
  • Fully backward compatible: adds a read-only property, no existing behavior changes

Related Issues

Closes #1197

Documentation PR

N/A — no user-facing documentation changes needed for this property addition.

Type of Change

New feature

Testing

How have you tested the change?

  • I ran hatch run prepare
  • Added unit tests covering: no invocations, invocation with no cycles, single invocation with multiple cycles, multiple invocations (returns latest)

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@lizradway lizradway added the area-context Session or context related label Mar 30, 2026
@lizradway lizradway temporarily deployed to manual-approval March 30, 2026 20:48 — with GitHub Actions Inactive
@lizradway lizradway changed the title feat(context): track agent.messages token size feat(context): track context tokens Mar 30, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 30, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@github-actions
Copy link
Copy Markdown

Assessment: Approve ✅

Clean, minimal implementation that addresses issue #1197 with zero overhead. The tests are comprehensive and cover all edge cases well.

Review Details
  • API Design: Read-only property that follows existing patterns (latest_agent_invocation); intuitive naming
  • Testing: Good coverage including no-invocations, no-cycles, multi-cycle, multi-invocation, and defensive provider-bug scenarios
  • Documentation: PR description is thorough with clear usage example

Nice work keeping this focused and backward-compatible! 🎉

@github-actions github-actions bot added size/s and removed size/s labels Mar 30, 2026
@lizradway lizradway temporarily deployed to manual-approval March 30, 2026 20:55 — with GitHub Actions Inactive
@lizradway lizradway marked this pull request as ready for review March 30, 2026 20:56
@lizradway lizradway temporarily deployed to manual-approval March 30, 2026 20:56 — with GitHub Actions Inactive
@github-actions
Copy link
Copy Markdown

Thanks for addressing the docstring feedback! The Returns section looks good.

Assessment: ✅ Approve - Ready to merge.

@kevmyung
Copy link
Copy Markdown
Contributor

Hi @lizradway, Thanks for this PR! I've been hoping for this kind of context token tracking in Strands for a while.

One concern: this works well when the agent instance stays in memory, but when the agent is restored from a session (e.g., via RepositorySessionManager), EventLoopMetrics is re-initialized from scratch — so latest_context_tokens returns 0 until the next LLM call completes. This could cause issues for smart compaction strategies that rely on this value to make threshold-based decisions immediately after restoration.

To address this, I think we'd also need per-message metrics metadata that gets persisted and restored alongside the messages themselves. I opened #1532 with a proposal for adding a _metadata field to messages for this purpose — it could complement this PR nicely.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-context Session or context related size/s

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CONTEXT] [FEATURE] Track context token size

2 participants