Introduce MemoryAccountingBitsetCollectorManager#15795
Introduce MemoryAccountingBitsetCollectorManager#15795gaobinlong wants to merge 4 commits intoapache:mainfrom
Conversation
Signed-off-by: Binlong Gao <gbinlong@amazon.com>
Signed-off-by: Binlong Gao <gbinlong@amazon.com>
| protected void doSetNextReader(LeafReaderContext context) throws IOException { | ||
| docBase = context.docBase; | ||
| length += context.reader().maxDoc(); | ||
| length = docBase + context.reader().maxDoc(); |
There was a problem hiding this comment.
This change works for both single-thread case and concurrency search case, the original code doesn't work for concurrency search case since length always starts from 0 for each collector.
There was a problem hiding this comment.
Does this mean we're creating a maxDoc-length FixedBitSet for every thread? I wonder if there's a way of being more memory efficient here?
There was a problem hiding this comment.
I've refactored the code to make it memory efficient for concurrent search case, please take a look, thanks!
|
Hi @romseygeek , would you mind taking look into this PR? Thank you~ |
Signed-off-by: Binlong Gao <gbinlong@amazon.com>
…setCollectorManager
|
This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the dev@lucene.apache.org list. Thank you for your contribution! |
2 participants
Description
This pr introduces MemoryAccountingBitsetCollectorManager and switches TestMemoryAccountingBitsetCollector.java to use search concurrency and move away from the deprecated search(Query, Collector) method.
Relates to #12892.