Skip to content

Commit a7fad9b

Browse files
committed
feat: add TokenLimiterProcessor to various agents and update README
- Integrated TokenLimiterProcessor into multiple agents to manage output token limits effectively. - Updated README.md with new Wakatime badge link. - Adjusted sampling rates for structure and creativity scorers in Content Strategist Agent. - Enhanced Upstash configuration for improved vector search and memory management.
1 parent 8c36544 commit a7fad9b

28 files changed

Lines changed: 129 additions & 57 deletions

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@
2929
<!-- Repository Links -->
3030
[![GitHub](https://img.shields.io/badge/GitHub-ssdeanx/AgentStack-181717?logo=github)](https://github.com/ssdeanx/AgentStack)
3131
[![GitMCP](https://img.shields.io/endpoint?url=https://gitmcp.io/badge/ssdeanx/AgentStack)](https://gitmcp.io/ssdeanx/AgentStack)
32-
[![wakatime](https://wakatime.com/badge/user/7a2fb9a0-188b-4568-887f-7645f9249e62/project/e44412f3-9bcc-4661-b79d-23160d90dfe0.svg)](https://wakatime.com/badge/user/7a2fb9a0-188b-4568-887f-7645f9249e62/project/e44412f3-9bcc-4661-b79d-23160d90dfe0)
32+
[![wakatime](https://wakatime.com/badge/user/7a2fb9a0-188b-4568-887f-7645f9249e62/project/e52d02a1-f64a-4f8d-bc13-caaa2dc37461.svg)](https://wakatime.com/badge/user/7a2fb9a0-188b-4568-887f-7645f9249e62/project/e52d02a1-f64a-4f8d-bc13-caaa2dc37461)
3333

3434
**AgentStack** is a **production-grade multi-agent framework** built on Mastra, delivering **60+ enterprise tools**, **30+ specialized agents**, **14 workflows**, **4 agent networks**, **65 UI components** (30+ AI Elements + 35+ base), and **A2A/MCP orchestration** for scalable AI systems. Focuses on **financial intelligence**, **RAG pipelines**, **observability**, **secure governance**, and **AI chat interfaces**.
3535

@@ -71,7 +71,7 @@
7171
## **Core Capabilities**
7272

7373
- **💰 Financial Intelligence**: 30+ tools (Polygon quotes/aggs/fundamentals, Finnhub analysis, AlphaVantage indicators)
74-
- **🔍 Semantic RAG**: PgVector (3072D embeddings) + MDocument chunking + rerank + graph traversal
74+
- **🔍 Semantic RAG**: PgVector (3072D embeddings) + MDocument chunking + rerank + graph traversal
7575
- **🤖 30+ Agents**: Research → Learn → Report → Edit → Analyze (stock/crypto/copywriter/evaluator/data pipeline/business-legal/charting/image/coding/dane)
7676
- **📋 14 Workflows**: Weather, content, financial reports, document processing, research synthesis, learning extraction, governed RAG (index + answer), spec generation, repo ingestion, stock analysis
7777
- **🌐 4 Agent Networks**: Coding team, data pipeline, report generation, research pipeline

src/mastra/agents/acpAgent.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ import { batchWebScraperTool, contentCleanerTool, htmlToMarkdownTool, linkExtrac
1919
import type { GoogleGenerativeAIProviderOptions } from '@ai-sdk/google';
2020
import type { RequestContext } from '@mastra/core/request-context';
2121
import { PGVECTOR_PROMPT } from '@mastra/pg';
22+
import { TokenLimiterProcessor } from '@mastra/core/processors';
2223

2324
export interface ACPContext {
2425
userId?: string
@@ -129,7 +130,7 @@ export const acpAgent = new Agent({
129130
// containerTags: ['acp-agent']
130131
// }),
131132
},
132-
133+
outputProcessors: [new TokenLimiterProcessor(1048576)],
133134
workflows: {},
134135
scorers: {
135136
relevancy: {

src/mastra/agents/businessLegalAgents.ts

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ import { GoogleGenerativeAIProviderMetadata } from '@ai-sdk/google';
44
import { googleTools } from '@ai-sdk/google/internal';
55
import { Agent } from '@mastra/core/agent';
66

7-
import { BatchPartsProcessor, UnicodeNormalizer } from '@mastra/core/processors';
7+
import { BatchPartsProcessor, TokenLimiterProcessor, UnicodeNormalizer } from '@mastra/core/processors';
88
import type { RequestContext } from '@mastra/core/request-context';
99
import {
1010
createAnswerRelevancyScorer,
@@ -156,7 +156,8 @@ ${PGVECTOR_PROMPT}
156156
stripControlChars: true,
157157
collapseWhitespace: true,
158158
}),
159-
]
159+
],
160+
outputProcessors: [new TokenLimiterProcessor(1048576)]
160161
})
161162

162163
export const contractAnalysisAgent = new Agent({
@@ -239,7 +240,8 @@ You are a Senior Contract Analyst. Analyze legal documents for risks, obligation
239240
stripControlChars: true,
240241
collapseWhitespace: true,
241242
}),
242-
]
243+
],
244+
outputProcessors: [new TokenLimiterProcessor(1048576)]
243245
})
244246

245247
export const complianceMonitoringAgent = new Agent({
@@ -325,7 +327,8 @@ You are a Compliance Officer. Monitor regulatory compliance and identify risks a
325327
stripControlChars: true,
326328
collapseWhitespace: true,
327329
}),
328-
]
330+
],
331+
outputProcessors: [new TokenLimiterProcessor(1048576)]
329332
})
330333

331334
export const businessStrategyAgent = new Agent({
@@ -406,7 +409,8 @@ You are a Chief Strategy Officer with legal expertise. Align business strategy w
406409
stripControlChars: true,
407410
collapseWhitespace: true,
408411
}),
409-
]
412+
],
413+
outputProcessors: [new TokenLimiterProcessor(1048576)]
410414
})
411415

412416

src/mastra/agents/calendarAgent.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ import { listEvents, getTodayEvents, getUpcomingEvents, findFreeSlots } from '..
33
import { googleAIFlashLite, pgMemory } from '../config';
44
import type { GoogleGenerativeAIProviderOptions } from '@ai-sdk/google';
55
import type { RequestContext } from '@mastra/core/request-context';
6+
import { TokenLimiterProcessor } from '@mastra/core/processors';
67

78
export interface CalendarContext {
89
userId?: string
@@ -54,5 +55,5 @@ Current user: ${userId ?? 'anonymous'}`,
5455
getUpcomingEvents,
5556
findFreeSlots,
5657
},
57-
58+
outputProcessors: [new TokenLimiterProcessor(1048576)]
5859
});

src/mastra/agents/codingAgents.ts

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import { Agent } from '@mastra/core/agent'
22

3-
import { BatchPartsProcessor, UnicodeNormalizer } from '@mastra/core/processors'
3+
import { BatchPartsProcessor, TokenLimiterProcessor, UnicodeNormalizer } from '@mastra/core/processors'
44
import type { RequestContext } from '@mastra/core/request-context'
55
import { createAnswerRelevancyScorer, createToxicityScorer } from '@mastra/evals/scorers/prebuilt'
66
import type { GoogleGenerativeAIProviderOptions } from '@ai-sdk/google';
@@ -128,8 +128,7 @@ Always consider maintainability, scalability, and testability in your recommenda
128128
inputProcessors: [
129129
new UnicodeNormalizer({ stripControlChars: true, collapseWhitespace: true }),
130130
],
131-
outputProcessors: [
132-
],
131+
outputProcessors: [new TokenLimiterProcessor(1048576)]
133132
})
134133

135134
//log.info('Cached tokens:', providerMetadata.google?.usageMetadata);
@@ -239,6 +238,7 @@ Be constructive and educational in feedback.`,
239238
inputProcessors: [
240239
new UnicodeNormalizer({ stripControlChars: true, collapseWhitespace: true }),
241240
],
241+
outputProcessors: [new TokenLimiterProcessor(1048576)]
242242
})
243243

244244
/**
@@ -340,7 +340,7 @@ Always use Vitest syntax: describe, it, expect, vi.mock, vi.fn.`,
340340
inputProcessors: [
341341
new UnicodeNormalizer({ stripControlChars: true, collapseWhitespace: true }),
342342
],
343-
343+
outputProcessors: [new TokenLimiterProcessor(1048576)]
344344
})
345345

346346
/**
@@ -448,7 +448,7 @@ For each refactoring:
448448
inputProcessors: [
449449
new UnicodeNormalizer({ stripControlChars: true, collapseWhitespace: true }),
450450
],
451-
451+
outputProcessors: [new TokenLimiterProcessor(1048576)]
452452
})
453453

454454
log.info('Coding Team Agents initialized: codeArchitectAgent, codeReviewerAgent, testEngineerAgent, refactoringAgent')

src/mastra/agents/contentStrategistAgent.ts

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import { structureScorer, creativityScorer } from '../scorers';
66
import { chartSupervisorTool } from '../tools/financial-chart-tools';
77
import type { GoogleGenerativeAIProviderOptions } from '@ai-sdk/google';
88
import type { RequestContext } from '@mastra/core/request-context'
9+
import { TokenLimiterProcessor } from '@mastra/core/processors';
910

1011
export type UserTier = 'free' | 'pro' | 'enterprise'
1112
export interface ContentAgentContext {
@@ -99,11 +100,12 @@ EXAMPLE FLOW:
99100
scorers: {
100101
structure: {
101102
scorer: structureScorer,
102-
sampling: { type: 'ratio', rate: 1.0 },
103+
sampling: { type: 'ratio', rate: 0.3 },
103104
},
104105
creativity: {
105106
scorer: creativityScorer,
106-
sampling: { type: 'ratio', rate: 1.0 },
107+
sampling: { type: 'ratio', rate: 0.3 },
107108
},
108-
}
109+
},
110+
outputProcessors: [new TokenLimiterProcessor(1048576)]
109111
});

src/mastra/agents/copywriterAgent.ts

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ import { structureScorer } from '../scorers'
1414
import { chartSupervisorTool } from '../tools/financial-chart-tools'
1515
import type { GoogleGenerativeAIProviderOptions } from '@ai-sdk/google'
1616
import type { RequestContext } from '@mastra/core/request-context'
17+
import { TokenLimiterProcessor } from '@mastra/core/processors'
1718

1819
// Define runtime context for this agent
1920
export interface CopywriterAgentContext {
@@ -142,6 +143,7 @@ Provide the final content in a clear, well-structured format appropriate for the
142143
},
143144
},
144145
workflows: {},
145-
maxRetries: 5
146+
maxRetries: 5,
147+
outputProcessors: [new TokenLimiterProcessor(1048576)]
146148
})
147149

src/mastra/agents/csv_to_excalidraw.ts

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ import { googleAI, pgMemory } from "../config";
33
import { structureScorer } from "../scorers";
44
import type { GoogleGenerativeAIProviderOptions } from "@ai-sdk/google";
55
import type { RequestContext } from '@mastra/core/request-context'
6+
import { TokenLimiterProcessor } from "@mastra/core/processors";
67

78
export interface CsvToExcalidrawRuntimeContext {
89
userId?: string
@@ -182,5 +183,6 @@ Structure:
182183
},
183184
},
184185
workflows: {},
185-
maxRetries: 5
186+
maxRetries: 5,
187+
outputProcessors: [new TokenLimiterProcessor(1048576)]
186188
});

src/mastra/agents/dane.ts

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ import { listEvents } from '../tools/calendar-tool'
55
import { execaTool } from '../tools/execa-tool'
66
import type { GoogleGenerativeAIProviderOptions } from '@ai-sdk/google'
77
import type { RequestContext } from '@mastra/core/request-context'
8+
import { TokenLimiterProcessor } from '@mastra/core/processors'
89

910
export interface DaneContext {
1011
userId?: string
@@ -39,6 +40,7 @@ export const daneCommitMessage = new Agent({
3940
},
4041
model: googleAIFlashLite,
4142
memory: pgMemory,
43+
outputProcessors: [new TokenLimiterProcessor(1048576)]
4244
})
4345

4446
export const daneIssueLabeler = new Agent({
@@ -67,6 +69,7 @@ export const daneIssueLabeler = new Agent({
6769
},
6870
model: googleAIFlashLite,
6971
memory: pgMemory,
72+
outputProcessors: [new TokenLimiterProcessor(1048576)]
7073
})
7174

7275
export const daneLinkChecker = new Agent({
@@ -100,6 +103,7 @@ export const daneLinkChecker = new Agent({
100103
},
101104
model: googleAIFlashLite,
102105
memory: pgMemory,
106+
outputProcessors: [new TokenLimiterProcessor(1048576)]
103107
})
104108

105109
export const daneChangeLog = new Agent({
@@ -135,6 +139,7 @@ export const daneChangeLog = new Agent({
135139
},
136140
model: googleAIFlashLite,
137141
memory: pgMemory,
142+
outputProcessors: [new TokenLimiterProcessor(1048576)]
138143
})
139144

140145
export const dane = new Agent({
@@ -201,4 +206,5 @@ export const dane = new Agent({
201206
googleSearch,
202207
listEvents,
203208
},
209+
outputProcessors: [new TokenLimiterProcessor(1048576)]
204210
})

src/mastra/agents/dataExportAgent.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ import {
1212
listDataDirTool,
1313
} from '../tools/data-file-manager'
1414
import type { RequestContext } from '@mastra/core/request-context'
15+
import { TokenLimiterProcessor } from '@mastra/core/processors'
1516

1617
export interface DataExportContext {
1718
userId?: string
@@ -85,6 +86,7 @@ export const dataExportAgent = new Agent({
8586
backupDataTool,
8687
listDataDirTool,
8788
},
89+
outputProcessors: [new TokenLimiterProcessor(1048576)]
8890
})
8991

9092
log.info('Data Export Agent initialized')

0 commit comments

Comments
 (0)