Feature request
approximate_distribution() requires manually specifying min_similarity. Like the threshold parameter in reduce_outliers, users must iterate to find a value that produces an acceptable number of documents with zero topic distribution.
Add an outliers_nb_target parameter that auto-searches for the optimal min_similarity:
# Current: manual trial-and-error
distributions = topic_model.approximate_distribution(docs, min_similarity=0.1)
# Count zeros... too many. Try 0.05... too few. Try 0.08...
# Proposed: specify the target directly
distributions = topic_model.approximate_distribution(docs, outliers_nb_target=50)
Motivation
Users know their target — "I want at most 50 documents without any topic assignment" — not the internal min_similarity value that achieves it. The same trial-and-error UX problem as reduce_outliers (see related issue).
Your contribution
I can submit a PR that adds:
outliers_nb_target (int): target number of zero-distribution documents
- Performance optimization: refactor the internals so that the expensive similarity matrix is computed once and reused across all binary search iterations
Cannot set both min_similarity and outliers_nb_target. Backward compatible: defaults to None.
I've already been prototyping this in my fork, so I can open a PR quickly if this looks like a good direction.
Feature request
approximate_distribution()requires manually specifyingmin_similarity. Like thethresholdparameter inreduce_outliers, users must iterate to find a value that produces an acceptable number of documents with zero topic distribution.Add an
outliers_nb_targetparameter that auto-searches for the optimalmin_similarity:Motivation
Users know their target — "I want at most 50 documents without any topic assignment" — not the internal
min_similarityvalue that achieves it. The same trial-and-error UX problem asreduce_outliers(see related issue).Your contribution
I can submit a PR that adds:
outliers_nb_target(int): target number of zero-distribution documentsCannot set both
min_similarityandoutliers_nb_target. Backward compatible: defaults toNone.I've already been prototyping this in my fork, so I can open a PR quickly if this looks like a good direction.