fix(memory): persist summary buffer checkpoints#6622
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces persistent checkpointing for ConversationSummaryBufferMemory by adding a new ConversationSummaryBufferState entity and corresponding database migrations. This allows the memory component to store and restore conversation summaries efficiently without re-processing the entire history. The review feedback highlights three key areas for optimization: batching the mapping of chat messages to base messages to avoid performance bottlenecks, explicitly updating the updatedDate column during state persistence since TypeORM's update method does not do so automatically, and using a DISTINCT query to retrieve unique session IDs during message deletion to reduce memory overhead.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| private async mapMessages(prependMessages: IMessage[], storedMessages: StoredChatMessage[]): Promise<BufferEntry[]> { | ||
| const inputs = [ | ||
| ...prependMessages.map((message) => ({ message, storedMessage: undefined })), | ||
| ...storedMessages.map((message) => ({ message, storedMessage: message })) | ||
| ] | ||
|
|
||
| const entries: Array<BufferEntry | undefined> = await Promise.all( | ||
| inputs.map(async ({ message, storedMessage }): Promise<BufferEntry | undefined> => { | ||
| const [baseMessage] = await mapChatMessageToBaseMessage([message], this.orgId) | ||
| if (!baseMessage) return undefined | ||
| return storedMessage ? { message: baseMessage, storedMessage } : { message: baseMessage } | ||
| }) | ||
| ) | ||
|
|
||
| return entries.filter((entry): entry is BufferEntry => entry !== undefined) | ||
| } |
There was a problem hiding this comment.
Calling mapChatMessageToBaseMessage individually for each message inside Promise.all is highly inefficient and can lead to performance bottlenecks, especially for long chat histories. Since mapChatMessageToBaseMessage accepts an array of messages, we should batch the mapping operation in a single call.
private async mapMessages(prependMessages: IMessage[], storedMessages: StoredChatMessage[]): Promise<BufferEntry[]> {
const inputs = [
...prependMessages.map((message) => ({ message, storedMessage: undefined })),
...storedMessages.map((message) => ({ message, storedMessage: message }))
]
const baseMessages = await mapChatMessageToBaseMessage(
inputs.map((i) => i.message),
this.orgId
)
const entries: BufferEntry[] = []
for (let i = 0; i < inputs.length; i += 1) {
const baseMessage = baseMessages[i]
if (baseMessage) {
const { storedMessage } = inputs[i]
entries.push(storedMessage ? { message: baseMessage, storedMessage } : { message: baseMessage })
}
}
return entries
}| private async persistState( | ||
| sessionId: string, | ||
| stateKey: string, | ||
| state: SummaryBufferState | null, | ||
| summary: string, | ||
| checkpoint: StoredChatMessage | ||
| ): Promise<boolean> { | ||
| const stateRepository = this.appDataSource.getRepository(this.databaseEntities['ConversationSummaryBufferState']) | ||
| const nextVersion = (state?.version ?? 0) + 1 | ||
| const values = { | ||
| chatflowid: this.chatflowid, | ||
| sessionId, | ||
| nodeId: this.nodeId, | ||
| summary, | ||
| cursorCreatedDate: checkpoint.createdDate, | ||
| cursorMessageId: checkpoint.id, | ||
| version: nextVersion | ||
| } | ||
|
|
||
| if (state) { | ||
| const result = await stateRepository.update({ stateKey, version: state.version }, values) | ||
| return result.affected === 1 | ||
| } | ||
|
|
||
| try { | ||
| await stateRepository.insert({ stateKey, ...values }) | ||
| return true | ||
| } catch (error) { | ||
| const existingState = await stateRepository.findOneBy({ stateKey }) | ||
| if (existingState) return false | ||
| throw error | ||
| } | ||
| } |
There was a problem hiding this comment.
In TypeORM, Repository.update executes a direct SQL UPDATE query and does not automatically update @UpdateDateColumn columns (like updatedDate) unless explicitly passed in the values object or handled by database-level triggers (which SQLite and Postgres do not have for this column). To ensure updatedDate is correctly updated across all databases, we should explicitly include updatedDate: new Date() in the values object.
private async persistState(
sessionId: string,
stateKey: string,
state: SummaryBufferState | null,
summary: string,
checkpoint: StoredChatMessage
): Promise<boolean> {
const stateRepository = this.appDataSource.getRepository(this.databaseEntities['ConversationSummaryBufferState'])
const nextVersion = (state?.version ?? 0) + 1
const values = {
chatflowid: this.chatflowid,
sessionId,
nodeId: this.nodeId,
summary,
cursorCreatedDate: checkpoint.createdDate,
cursorMessageId: checkpoint.id,
version: nextVersion,
updatedDate: new Date()
}
if (state) {
const result = await stateRepository.update({ stateKey, version: state.version }, values)
return result.affected === 1
}
try {
await stateRepository.insert({ stateKey, ...values })
return true
} catch (error) {
const existingState = await stateRepository.findOneBy({ stateKey })
if (existingState) return false
throw error
}
}| const chatMessageRepository = appServer.AppDataSource.getRepository(ChatMessage) | ||
| const affectedMessages = await chatMessageRepository.find({ | ||
| select: ['sessionId'], | ||
| where: deleteOptions | ||
| }) | ||
| const sessionIds = Array.from( | ||
| new Set(affectedMessages.map((message) => message.sessionId).filter((id): id is string => Boolean(id))) | ||
| ) |
There was a problem hiding this comment.
Fetching all matching messages from the database just to extract unique sessionIds is highly inefficient and can cause significant memory overhead and performance degradation when deleting a large number of messages. We should use a DISTINCT query directly in the database to retrieve only the unique sessionIds.
const chatMessageRepository = appServer.AppDataSource.getRepository(ChatMessage)
const affectedSessions = await chatMessageRepository
.createQueryBuilder('chat_message')
.select('DISTINCT chat_message.sessionId', 'sessionId')
.where(deleteOptions)
.getRawMany()
const sessionIds = affectedSessions.map((row) => row.sessionId).filter((id): id is string => Boolean(id))
What changed
ConversationSummaryBufferMemorykept its moving summary on the short-lived node instance, so every new prediction loaded and summarized the full database history again.This adds a private checkpoint table keyed by chatflow, session, and node. Each checkpoint stores the summary and a
createdDate + message idcursor, so later requests only load messages after the exact checkpoint, including messages with identical timestamps. The summary remains separate from the unsummarized tail and is added to the model context once.Writes use a version check and retry after conflicts. Temporary prepended history does not advance the checkpoint, and failed summary generation leaves the previous state unchanged. Message/session clears and chatflow deletion invalidate affected checkpoints. Migrations are included for SQLite, MySQL, MariaDB, and Postgres.
Tests
pnpm --filter flowise-components test -- nodes/memory/ConversationSummaryBufferMemory/ConversationSummaryBufferMemory.test.ts --runInBandpnpm --dir packages/server test -- src/utils/conversationSummaryBufferState.test.ts src/database/migrations/sqlite/1783987200000-AddConversationSummaryBufferState.test.ts --runInBandpnpm lintNODE_OPTIONS=--max_old_space_size=4096 pnpm buildI also ran
pnpm test:coverage; the workspace run hit the unrelated existinguseAsyncOptions › starts in loading statefailure inpackages/agentflow. The focused component and server suites above pass.Closes #5873