Feature request
_save_representative_docs hardcodes nr_samples=500 and nr_repr_docs=3. Users cannot control how many representative documents are stored per topic, nor the sampling pool size.
Add a public method to recalculate representative documents after training with a configurable count, following the API direction suggested in #1767:
# Recalculate with more representative docs after training
topic_model.recalculate_representative_docs(nr_repr_docs=10, nr_samples=1000)
Motivation
This is a frequently requested feature:
- #552 — "get more than 3 representative docs"
- #848 — "Ability to change # of documents pulled per topic"
- #1767 — "How to get more than 3 representative docs per topic"
In #1767, @MaartenGr noted:
"It might be nice to have an additional function that re-calculates the representative documents since this question seems to appear frequently."
PR #2130 attempted an nr_repr_docs constructor parameter, but @MaartenGr's feedback was that he prefers a standalone public function that recalculates representative docs after training.
Use cases: production users who need more docs for human review, small datasets where 500 exceeds topic size, users who want to iterate on representative doc quality post-training.
Your contribution
I can submit a PR following @MaartenGr's preferred API direction:
- New public method:
recalculate_representative_docs(nr_repr_docs=3, nr_samples=500) — recalculates and replaces representative_docs_ after training.
- Constructor defaults
nr_repr_docs (default 3) and nr_repr_docs_nr_samples (default 500) — used during fit_transform().
I've already been prototyping this in my fork along the lines you suggested, so I can open a PR quickly if this direction works for you.
Backward compatible: defaults reproduce the current behavior exactly.
Note: Works standalone. Pairs well with the sampling/indexing fix for _extract_representative_docs (#2495) — that fix corrects the sampling this method configures.
Feature request
_save_representative_docshardcodesnr_samples=500andnr_repr_docs=3. Users cannot control how many representative documents are stored per topic, nor the sampling pool size.Add a public method to recalculate representative documents after training with a configurable count, following the API direction suggested in #1767:
Motivation
This is a frequently requested feature:
In #1767, @MaartenGr noted:
PR #2130 attempted an
nr_repr_docsconstructor parameter, but @MaartenGr's feedback was that he prefers a standalone public function that recalculates representative docs after training.Use cases: production users who need more docs for human review, small datasets where 500 exceeds topic size, users who want to iterate on representative doc quality post-training.
Your contribution
I can submit a PR following @MaartenGr's preferred API direction:
recalculate_representative_docs(nr_repr_docs=3, nr_samples=500)— recalculates and replacesrepresentative_docs_after training.nr_repr_docs(default 3) andnr_repr_docs_nr_samples(default 500) — used duringfit_transform().I've already been prototyping this in my fork along the lines you suggested, so I can open a PR quickly if this direction works for you.
Backward compatible: defaults reproduce the current behavior exactly.
Note: Works standalone. Pairs well with the sampling/indexing fix for
_extract_representative_docs(#2495) — that fix corrects the sampling this method configures.