Feature request
_extract_representative_docs() runs the full computation pipeline (sampling → c-TF-IDF → cosine similarity → MMR selection) every time it is invoked. In interactive workflows, users often call a sequence of visualization methods that each trigger this computation:
topic_model.visualize_topics() # triggers representative docs computation
topic_model.visualize_hierarchy() # triggers it again
topic_model.visualize_documents() # triggers it again
Cache representative_docs_ after first computation and invalidate when topic assignments change, following the same pattern already used for topic_embeddings_.
Motivation
Each call repeats the same expensive similarity computations even though topic assignments haven't changed. This is also the root cause behind #2367 — KeyBERTInspired recomputes representative docs embeddings even when precomputed embeddings are available.
BERTopic already caches topic_embeddings_ with compute-once-store-invalidate-on-change semantics. Representative docs should follow the same pattern.
Your contribution
I can submit a PR that caches representative_docs_ after first computation and invalidates the cache when topic assignments change (i.e., after update_topics(), reduce_outliers(), merge_topics()). Subsequent calls return the cached result without recomputation.
Note: This is most effective when the underlying _extract_representative_docs sampling and indexing bugs are fixed first (#2495). The caching mechanism itself is independent, but caching correct results is obviously preferable.
I've already been prototyping this in my fork, so I can open a PR quickly if the approach looks good to you.
Feature request
_extract_representative_docs()runs the full computation pipeline (sampling → c-TF-IDF → cosine similarity → MMR selection) every time it is invoked. In interactive workflows, users often call a sequence of visualization methods that each trigger this computation:Cache
representative_docs_after first computation and invalidate when topic assignments change, following the same pattern already used fortopic_embeddings_.Motivation
Each call repeats the same expensive similarity computations even though topic assignments haven't changed. This is also the root cause behind #2367 — KeyBERTInspired recomputes representative docs embeddings even when precomputed embeddings are available.
BERTopic already caches
topic_embeddings_with compute-once-store-invalidate-on-change semantics. Representative docs should follow the same pattern.Your contribution
I can submit a PR that caches
representative_docs_after first computation and invalidates the cache when topic assignments change (i.e., afterupdate_topics(),reduce_outliers(),merge_topics()). Subsequent calls return the cached result without recomputation.Note: This is most effective when the underlying
_extract_representative_docssampling and indexing bugs are fixed first (#2495). The caching mechanism itself is independent, but caching correct results is obviously preferable.I've already been prototyping this in my fork, so I can open a PR quickly if the approach looks good to you.