Skip to content

fix(memory): persist summary buffer checkpoints#6622

Open
zzz100868 wants to merge 1 commit into
FlowiseAI:mainfrom
zzz100868:bugfix/conversation-summary-buffer-state
Open

fix(memory): persist summary buffer checkpoints#6622
zzz100868 wants to merge 1 commit into
FlowiseAI:mainfrom
zzz100868:bugfix/conversation-summary-buffer-state

Conversation

@zzz100868

Copy link
Copy Markdown

What changed

ConversationSummaryBufferMemory kept its moving summary on the short-lived node instance, so every new prediction loaded and summarized the full database history again.

This adds a private checkpoint table keyed by chatflow, session, and node. Each checkpoint stores the summary and a createdDate + message id cursor, so later requests only load messages after the exact checkpoint, including messages with identical timestamps. The summary remains separate from the unsummarized tail and is added to the model context once.

Writes use a version check and retry after conflicts. Temporary prepended history does not advance the checkpoint, and failed summary generation leaves the previous state unchanged. Message/session clears and chatflow deletion invalidate affected checkpoints. Migrations are included for SQLite, MySQL, MariaDB, and Postgres.

Tests

  • pnpm --filter flowise-components test -- nodes/memory/ConversationSummaryBufferMemory/ConversationSummaryBufferMemory.test.ts --runInBand
  • pnpm --dir packages/server test -- src/utils/conversationSummaryBufferState.test.ts src/database/migrations/sqlite/1783987200000-AddConversationSummaryBufferState.test.ts --runInBand
  • pnpm lint
  • NODE_OPTIONS=--max_old_space_size=4096 pnpm build

I also ran pnpm test:coverage; the workspace run hit the unrelated existing useAsyncOptions › starts in loading state failure in packages/agentflow. The focused component and server suites above pass.

Closes #5873

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces persistent checkpointing for ConversationSummaryBufferMemory by adding a new ConversationSummaryBufferState entity and corresponding database migrations. This allows the memory component to store and restore conversation summaries efficiently without re-processing the entire history. The review feedback highlights three key areas for optimization: batching the mapping of chat messages to base messages to avoid performance bottlenecks, explicitly updating the updatedDate column during state persistence since TypeORM's update method does not do so automatically, and using a DISTINCT query to retrieve unique session IDs during message deletion to reduce memory overhead.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +291 to +306
private async mapMessages(prependMessages: IMessage[], storedMessages: StoredChatMessage[]): Promise<BufferEntry[]> {
const inputs = [
...prependMessages.map((message) => ({ message, storedMessage: undefined })),
...storedMessages.map((message) => ({ message, storedMessage: message }))
]

const entries: Array<BufferEntry | undefined> = await Promise.all(
inputs.map(async ({ message, storedMessage }): Promise<BufferEntry | undefined> => {
const [baseMessage] = await mapChatMessageToBaseMessage([message], this.orgId)
if (!baseMessage) return undefined
return storedMessage ? { message: baseMessage, storedMessage } : { message: baseMessage }
})
)

return entries.filter((entry): entry is BufferEntry => entry !== undefined)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Calling mapChatMessageToBaseMessage individually for each message inside Promise.all is highly inefficient and can lead to performance bottlenecks, especially for long chat histories. Since mapChatMessageToBaseMessage accepts an array of messages, we should batch the mapping operation in a single call.

    private async mapMessages(prependMessages: IMessage[], storedMessages: StoredChatMessage[]): Promise<BufferEntry[]> {
        const inputs = [
            ...prependMessages.map((message) => ({ message, storedMessage: undefined })),
            ...storedMessages.map((message) => ({ message, storedMessage: message }))
        ]

        const baseMessages = await mapChatMessageToBaseMessage(
            inputs.map((i) => i.message),
            this.orgId
        )

        const entries: BufferEntry[] = []
        for (let i = 0; i < inputs.length; i += 1) {
            const baseMessage = baseMessages[i]
            if (baseMessage) {
                const { storedMessage } = inputs[i]
                entries.push(storedMessage ? { message: baseMessage, storedMessage } : { message: baseMessage })
            }
        }

        return entries
    }

Comment on lines +327 to 359
private async persistState(
sessionId: string,
stateKey: string,
state: SummaryBufferState | null,
summary: string,
checkpoint: StoredChatMessage
): Promise<boolean> {
const stateRepository = this.appDataSource.getRepository(this.databaseEntities['ConversationSummaryBufferState'])
const nextVersion = (state?.version ?? 0) + 1
const values = {
chatflowid: this.chatflowid,
sessionId,
nodeId: this.nodeId,
summary,
cursorCreatedDate: checkpoint.createdDate,
cursorMessageId: checkpoint.id,
version: nextVersion
}

if (state) {
const result = await stateRepository.update({ stateKey, version: state.version }, values)
return result.affected === 1
}

try {
await stateRepository.insert({ stateKey, ...values })
return true
} catch (error) {
const existingState = await stateRepository.findOneBy({ stateKey })
if (existingState) return false
throw error
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In TypeORM, Repository.update executes a direct SQL UPDATE query and does not automatically update @UpdateDateColumn columns (like updatedDate) unless explicitly passed in the values object or handled by database-level triggers (which SQLite and Postgres do not have for this column). To ensure updatedDate is correctly updated across all databases, we should explicitly include updatedDate: new Date() in the values object.

    private async persistState(
        sessionId: string,
        stateKey: string,
        state: SummaryBufferState | null,
        summary: string,
        checkpoint: StoredChatMessage
    ): Promise<boolean> {
        const stateRepository = this.appDataSource.getRepository(this.databaseEntities['ConversationSummaryBufferState'])
        const nextVersion = (state?.version ?? 0) + 1
        const values = {
            chatflowid: this.chatflowid,
            sessionId,
            nodeId: this.nodeId,
            summary,
            cursorCreatedDate: checkpoint.createdDate,
            cursorMessageId: checkpoint.id,
            version: nextVersion,
            updatedDate: new Date()
        }

        if (state) {
            const result = await stateRepository.update({ stateKey, version: state.version }, values)
            return result.affected === 1
        }

        try {
            await stateRepository.insert({ stateKey, ...values })
            return true
        } catch (error) {
            const existingState = await stateRepository.findOneBy({ stateKey })
            if (existingState) return false
            throw error
        }
    }

Comment on lines +121 to +128
const chatMessageRepository = appServer.AppDataSource.getRepository(ChatMessage)
const affectedMessages = await chatMessageRepository.find({
select: ['sessionId'],
where: deleteOptions
})
const sessionIds = Array.from(
new Set(affectedMessages.map((message) => message.sessionId).filter((id): id is string => Boolean(id)))
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Fetching all matching messages from the database just to extract unique sessionIds is highly inefficient and can cause significant memory overhead and performance degradation when deleting a large number of messages. We should use a DISTINCT query directly in the database to retrieve only the unique sessionIds.

        const chatMessageRepository = appServer.AppDataSource.getRepository(ChatMessage)
        const affectedSessions = await chatMessageRepository
            .createQueryBuilder('chat_message')
            .select('DISTINCT chat_message.sessionId', 'sessionId')
            .where(deleteOptions)
            .getRawMany()
        const sessionIds = affectedSessions.map((row) => row.sessionId).filter((id): id is string => Boolean(id))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Conversational Summary Buffer Memory gets all messages from db

1 participant