Feature request
I propose modernizing the existing LangChain representation interface and adding native structured multi aspect output. The updated interface would accept custom system and user prompts while BERTopic internally prepares and supplies keywords and representative documents. Users could declare an output schema as a mapping of field names to descriptions. Each topic would then produce one validated JSON response containing fields such as label and description.
A designated main field would update topic_representations_, while the remaining fields would populate topic_aspects_ and appear as columns in get_topic_info(). Existing features such as Runnable support, representative document selection, truncation, diversity, batch execution, and configurable concurrency would be preserved.
Motivation
I am an algorithm engineer currently building a topic discovery project where BERTopic is an important part of our technical stack. Through intensive use with a large volume of data, we found that generating multiple topic aspects is less efficient and flexible than it could be.
The current LangChain representation supports batch execution, but expects each result to contain a single output_text value. Multi aspect representations are executed separately. For example, generating a label and description for 100 topics requires 200 model requests. Returning both fields from one structured response would reduce this to 100 requests. This avoids repeated input context, reduces latency and rate limit pressure, and improves consistency because all fields are generated from the same documents, keywords, and model response.
Custom system and user prompts combined with a declared output schema would also make the interface more flexible and allow users to obtain application ready results with less configuration. This proposal is related to #2187 and #2188. Those discussions explore updating LangChain support and structured output, but the current implementation does not preserve multiple named fields or map one response into multiple BERTopic aspects.
Your contribution
I am willing to implement this feature. I can update the existing LangChain representation while preserving backward compatibility and its current batch and concurrency capabilities. I plan to add custom system and user prompts, structured output validation, support for multiple named fields, and integration with topic_representations_ and topic_aspects_.
I will also add unit tests and documentation covering the existing single label behavior, structured multi field output, custom prompts, concurrency, invalid responses, and compatibility with custom LangChain Runnables. I would appreciate feedback on the proposed interface and return protocol before beginning the implementation.
:)
Feature request
I propose modernizing the existing LangChain representation interface and adding native structured multi aspect output. The updated interface would accept custom system and user prompts while BERTopic internally prepares and supplies keywords and representative documents. Users could declare an output schema as a mapping of field names to descriptions. Each topic would then produce one validated JSON response containing fields such as label and description.
A designated main field would update
topic_representations_, while the remaining fields would populatetopic_aspects_and appear as columns inget_topic_info(). Existing features such as Runnable support, representative document selection, truncation, diversity, batch execution, and configurable concurrency would be preserved.Motivation
I am an algorithm engineer currently building a topic discovery project where BERTopic is an important part of our technical stack. Through intensive use with a large volume of data, we found that generating multiple topic aspects is less efficient and flexible than it could be.
The current LangChain representation supports batch execution, but expects each result to contain a single
output_textvalue. Multi aspect representations are executed separately. For example, generating a label and description for 100 topics requires 200 model requests. Returning both fields from one structured response would reduce this to 100 requests. This avoids repeated input context, reduces latency and rate limit pressure, and improves consistency because all fields are generated from the same documents, keywords, and model response.Custom system and user prompts combined with a declared output schema would also make the interface more flexible and allow users to obtain application ready results with less configuration. This proposal is related to #2187 and #2188. Those discussions explore updating LangChain support and structured output, but the current implementation does not preserve multiple named fields or map one response into multiple BERTopic aspects.
Your contribution
I am willing to implement this feature. I can update the existing LangChain representation while preserving backward compatibility and its current batch and concurrency capabilities. I plan to add custom system and user prompts, structured output validation, support for multiple named fields, and integration with
topic_representations_andtopic_aspects_.I will also add unit tests and documentation covering the existing single label behavior, structured multi field output, custom prompts, concurrency, invalid responses, and compatibility with custom LangChain Runnables. I would appreciate feedback on the proposed interface and return protocol before beginning the implementation.
:)