fix(store): include summarized news in active_incidents window check#49
Merged
Conversation
active_incidents() excluded any incident whose news had been moved into a dated log, by short-circuiting _pending_news_max_dates() on the news→log map. Once generate-logs consumed all pending news for an incident, it dropped out of the active set, and search-news --repoll never queried DDG for it again — so no fresh news could ever arrive. This silently excluded 20 of 25 incidents with news within the 7-day tracking window (Venezuela M7.2, Typhoon BAVI-26, Mexico M6.0, Ebola DRC, China flood, Bangladesh measles, etc.). The 5 still in the active set were only there because they had a couple of straggler pending items. Fix: rename _pending_news_max_dates → _recent_news_max_dates and drop the _news_log short-circuit. The window cutoff already ages old news out, so the summarized-news exclusion was both redundant and harmful. The extended_monitoring flag remains as the override for incidents older than the window that still need polling. Tests: invert test_active_ignores_summarized_news to test_active_keeps_incident_with_recent_summarized_news, and add test_active_drops_incident_when_summarized_news_outside_window to preserve the window contract. Mirrored in the .pyi.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
active_incidents(window)silently excluded any incident whose news had been moved into a dated log. Oncegenerate-logsconsumed all pending news for an incident, it dropped out of the active set, andsearch-news --repollnever queried DDG for it again — so no fresh news could ever arrive for it.Root cause
_pending_news_max_dates()short-circuited on the news→log map (self._news_log.get(nuuid) is not None), so summarized news was invisible to the window check. The window cutoff already ages old news out, making the summarized-news exclusion both redundant and harmful.Impact (production data, before fix)
20 of 25 incidents with news within the 7-day tracking window were silently excluded from repolling. The 5 still active were only there because of straggler pending items.
Notable victims: Venezuela M7.2 (135 news, last 2026-07-18), Typhoon BAVI-26 (97), Mexico M6.0 (81), Ebola DRC (60), China flood (43), Bangladesh measles (29), plus 14 others.
User-visible symptom: dashboard digests for Venezuela stopped at 2026-07-18 (
surpasses 5,000/ 5,069 deaths) while the actual toll climbed to 5,278 by 2026-07-21 — the system was alive but had stopped asking.Fix
Rename
_pending_news_max_dates→_recent_news_max_datesand drop the_news_logshort-circuit.extended_monitoringremains as the override for incidents aged past the window.Tests (TDD)
test_active_ignores_summarized_news→test_active_keeps_incident_with_recent_summarized_news(RED → GREEN).test_active_drops_incident_when_summarized_news_outside_windowto preserve the window contract.content_store_test.pyi.Validation
Simulated
repolllable_incidents(7)against production data: 5 → 25 incidents. Manual catch-up run for the 20 affected incidents yielded 4 new 07-22 logs (Venezuela, Spain forest fire, Ebola Bundibugyo, Dengue) + 7 new news items across 4 more incidents (COVID, Bangladesh measles, China flood, France forest fire green). Venezuela's 07-22 log summary correctly carries the updated toll:…risen to 5,278. DP World… 27 tonnes of humanitarian aid.