diff --git a/app/README.md b/app/README.md index dfa615f78..c6bfe8a6b 100644 --- a/app/README.md +++ b/app/README.md @@ -68,10 +68,11 @@ The application module owns the Android app shell: `BoxLoreApplication`, `MainAc - `LegacyWorkerFactory` maps legacy worker class names to current worker implementations for WorkManager continuity. - `MainActivity` hosts theme, edge-to-edge setup, app update hooks, surveys, library import state (full-screen `OpmlImportDialog` overlay so onboarding welcome never shows through), the selected floating or classic navigation presentation, and the player overlay. Its start/stop lifecycle informs the application-scoped playback repository whether UI position polling is needed; background audio, persistence, notifications, Cast, and MediaSession behavior remain service-owned. - **Feed fix off switches** (`sync/FeedFixSwitches`): PostHog kill switches for the feed and episode fixes, passed into core modules as lambdas. A fix runs unless its flag is on; no answer (PostHog not ready, flag missing or inactive, an error) keeps it on. `feed-fix-match-baseline-off` sends the feed-only extras' comparison list back from the 1,000-item `episodes/match` list to the 100-item page (`PodcastRepository.useEpisodeMatchBaseline`). `feed-fix-address-recheck-off`, `feed-fix-sync-fallback-off` and `feed-fix-moved-feed-off` turn off the subscription sync's dead-address re-check, its `/sync` fallback for failed feeds, and following moved feeds (`FeedFixToggles`). `FeedFixSwitchesTest` covers the default. The same fail-open switches pause the v2 build's jobs: `upgrade-repair-off` (the one-time repair of saved feed addresses after the update), `address-check-off` (the daily address check of followed shows), both without marking the work done, and `v2-discovery-off` (discovery back on the v1 routes). These three are read through `sync/SavedKillSwitch`, because the repair can start on a headless launch before PostHog is set up: once PostHog is ready its answer decides and is saved; while it is still starting, the last saved answer is used, and the job runs only when none was saved. -- **Feed-first rollout** (`sync/FeedFirstSwitch`): the `feed_first` PostHog flag picks where show pages read episodes (on: the show's own feed; off: the server's list). Unlike the off switches it fails closed: with no boolean answer from PostHog it uses the last saved answer, else off. The first read fixes the value for the process, so a page's source never switches while the app runs. `$feature_flag_called` is recorded once per process, only when the value came from a live PostHog answer, and the process uses the value that exposure read returned. `sync/RolloutFlagMemory` saves `feed_first` and the three switches above for the next process whenever PostHog reports loaded flags (`PostHogAndroidConfig.onFeatureFlags`: the startup preload, `identify` and `reset`; explicit `reloadFeatureFlags` calls don't reach it, so a mid-session change is saved at the next of those loads). To roll back, keep `feed_first` active with a 0% release condition so phones get and save `false`; never disable or delete it before the flag's removal, because a disabled flag is missing from PostHog's answer and a phone that saved `true` would stay on. `AppContainer.rolloutSwitches` hands `feed_first` and the three switches above to `:core:catalog` as `RolloutSwitches` on `PodcastRepository`. Discovery reads `v2Discovery` (`PodcastRepository` and `DiscoveryRouting`, below); the daily address check and broken-feed reports read `addressCheck` (`address-check-off`, through `SubscriptionForegroundSync`, below); `feed_first` and `upgradeRepair` have no reader yet. +- **Feed-first rollout** (`sync/FeedFirstSwitch`): the `feed_first` PostHog flag picks where show pages read episodes (on: the show's own feed; off: the server's list). Unlike the off switches it fails closed: with no boolean answer from PostHog it uses the last saved answer, else off. The first read fixes the value for the process, so a page's source never switches while the app runs. `$feature_flag_called` is recorded once per process, only when the value came from a live PostHog answer, and the process uses the value that exposure read returned. `sync/RolloutFlagMemory` saves `feed_first` and the three switches above for the next process whenever PostHog reports loaded flags (`PostHogAndroidConfig.onFeatureFlags`: the startup preload, `identify` and `reset`; explicit `reloadFeatureFlags` calls don't reach it, so a mid-session change is saved at the next of those loads). To roll back, keep `feed_first` active with a 0% release condition so phones get and save `false`; never disable or delete it before the flag's removal, because a disabled flag is missing from PostHog's answer and a phone that saved `true` would stay on. `AppContainer.rolloutSwitches` hands `feed_first` and the three switches above to `:core:catalog` as `RolloutSwitches` on `PodcastRepository`. Discovery reads `v2Discovery` (`PodcastRepository` and `DiscoveryRouting`, below); the daily address check and broken-feed reports read `addressCheck` (`address-check-off`, through `SubscriptionForegroundSync`, below); show pages read `feed_first` (`PodcastRepository.isFeedFirstShow`, below); `upgradeRepair` has no reader yet. - **v2 discovery wiring**: `AppContainer.discoveryEpisodeIds` (`RoomDiscoveryEpisodeIds` over the main database, opened on first use) goes to `PodcastRepository` with `rolloutSwitches`, and the container's `init` installs the same `v2-discovery-off` switch and translator in `DiscoveryRouting`, so the briefing link builders in features and playback and `ChapterRepository` follow the repository: `/v2` routes with episode IDs shown under this phone's own IDs, or the v1 routes while the switch is set ([`core/catalog/README.md`](../core/catalog/README.md)). - **Feed reporting** (`sync/AnalyticsFeedTelemetry`): `AppContainer.feedTelemetry` is the `FeedTelemetryPort` handed to catalog code, so feed reads and address swaps reach `feed_fetch_result` and `feed_url_corrected` without catalog depending on analytics. A reporting failure never reaches the caller (`AnalyticsFeedTelemetryTest`). `SubscriptionRepository` gets it through `FeedAddressWiring`, so every saved-address correction of a followed show sends one `feed_url_corrected`. - **Saved feed addresses** (P4 PR 14): `AppContainer` wires the one address writer. `subscriptionRepository` gets `FeedAddressWiring` (re-registration through `fcm/PodcastNotificationRegistrationWorker`, `feedTelemetry`, and pending registrations in SharedPreferences `boxlore_notification_reregistrations`); `subscribedEpisodeCatalog` follows moved feeds on every refresh path (`FeedMoveFollower`, behind `feed-fix-moved-feed-off`); `autoDownloadCoordinator` saves recovered addresses through `ensureHttpsFeedUrl`; cloud pulls save addresses through `subscriptionRepository.feedAddresses`; `subscriptionForegroundSync` gets `FeedAddressUpkeep` for the daily address check and broken-feed reports (report times in SharedPreferences `boxlore_feed_reports`, never sent while `connectivityStatus` says offline), behind `address-check-off`. `PodcastNotificationRegistrationWorker` registers a Podcast Index show's notifications again after its address changed and waits for Firebase to acknowledge the node and topic: unique work `podcast-notification-registration-`, REPLACE (it reads the row when it runs), network-gated, exponential backoff from 30 s, at most 8 runs. It records each outcome (`settle`): a show it gives up on stays pending and the next foreground pass queues it again; `rss:` shows keep their own journal worker. +- **Feed-first show pages** (P4 PR 16): `AppContainer.podcastRepository` gets `FeedFirstWiring`: the stored copies (`localEpisodeCatalogRepository.showPages`), `connectivityStatus` for the offline rung, one broken-feed report limiter (`feedReportLimiter` over the `boxlore_feed_reports` stamps, the same instance `FeedAddressUpkeep.reportLimiter` gives the daily address check, so both claim inside one daily limit), `feedTelemetry` for `feed_fetch_result`, and `isFollowed` (`podcasts.isSubscribed`), so the show page leaves followed shows' reports to their own refresh. The page reads the feed only while `feed_first` is on (`RolloutSwitches.feedFirst`, fixed per process). - NPS survey UI (`surveys/`) applies `NpsSurveyBranching` after each answer so detractor / passive / promoter open-text paths end instead of falling through into the next score band (PostHog open-question `end` branching is not reliably persisted via API). - JVM `ArtworkPredictiveBackBackgroundTest` checks actual native rendering of the Back wrapper with changing artwork backgrounds beneath the saved app theme. - JVM `ArtworkNavigationHostThemeTest` uses a real NavHostController and the production slide transitions to verify source capture before first loader composition, stable saved-theme fallback, Back loader inheritance and separation of Home/regular-page colors from transient loader accents. diff --git a/app/src/main/java/cx/aswin/boxlore/AppContainer.kt b/app/src/main/java/cx/aswin/boxlore/AppContainer.kt index ad145acd5..67623470c 100644 --- a/app/src/main/java/cx/aswin/boxlore/AppContainer.kt +++ b/app/src/main/java/cx/aswin/boxlore/AppContainer.kt @@ -9,8 +9,10 @@ import cx.aswin.boxlore.core.catalog.EpisodeSharePreparer import cx.aswin.boxlore.core.catalog.EpisodeShareRegistrar import cx.aswin.boxlore.core.catalog.FeedAddressUpkeep import cx.aswin.boxlore.core.catalog.FeedAddressWiring +import cx.aswin.boxlore.core.catalog.FeedFirstWiring import cx.aswin.boxlore.core.catalog.FeedFixToggles import cx.aswin.boxlore.core.catalog.FeedMoveFollower +import cx.aswin.boxlore.core.catalog.FeedReportLimiter import cx.aswin.boxlore.core.catalog.FolderRepository import cx.aswin.boxlore.core.catalog.InstallReferrerManager import cx.aswin.boxlore.core.catalog.LegacyRssRepair @@ -227,6 +229,15 @@ class AppContainer( rolloutSwitches = rolloutSwitches, episodeAliases = episodeAliasReads, discoveryEpisodeIds = discoveryEpisodeIds, + // Feed-first show pages (P4 PR 16): stored copies, the shared report limiter and feed_fetch_result. + feedFirstWiring = + FeedFirstWiring( + copies = localEpisodeCatalogRepository.showPages, + isOnline = { connectivityStatus.isOnline() }, + reportLimiter = feedReportLimiter, + telemetry = feedTelemetry, + isFollowed = { id -> database.podcastDao().getPodcast(id)?.isSubscribed == true }, + ), ) } @@ -429,15 +440,29 @@ class AppContainer( // Daily address check and broken-feed reports (P4 PR 14), behind address-check-off. addressUpkeep = FeedAddressUpkeep( - reportStamps = - SharedPreferencesFeedReportStamps( - appContext.getSharedPreferences(SharedPreferencesFeedReportStamps.FILE_NAME, Context.MODE_PRIVATE), - ), + reportStamps = feedReportStamps, isOnline = connectivityStatus::isOnline, + reportLimiter = feedReportLimiter, ), ) } + /** + * Broken-feed report times, shared by the daily address check and the show page so both stay + * inside one daily limit (P4 PR 14 and PR 16). + */ + private val feedReportStamps: SharedPreferencesFeedReportStamps by lazy { + SharedPreferencesFeedReportStamps( + appContext.getSharedPreferences(SharedPreferencesFeedReportStamps.FILE_NAME, Context.MODE_PRIVATE), + ) + } + + /** + * The app's one broken-feed report limiter over [feedReportStamps]: the daily address check and + * the show page claim through the same instance, so they never pass the caps together (P4 PR 16). + */ + private val feedReportLimiter: FeedReportLimiter by lazy { FeedReportLimiter(feedReportStamps) } + override val folderRepository: FolderRepository by lazy { RoomFolderRepository( folderDao = database.folderDao(), diff --git a/core/catalog/README.md b/core/catalog/README.md index d6c80564a..5d3f89186 100644 --- a/core/catalog/README.md +++ b/core/catalog/README.md @@ -14,11 +14,12 @@ Failed consent writes keep activation off, and local notification flags prevent - **Saved feed addresses (one writer).** `FeedAddressUpdater` (`SubscriptionRepository.feedAddresses`, `updateFeedAddress(FeedAddressChange)`, `ensureHttpsFeedUrl`, `adoptMovedFeed`) is the only writer of a show's saved address. In one Room transaction (`PodcastDao.replaceFeedAddress`) it writes the HTTPS publisher address and the transport address, keeps the old address in `previousFeedUrl` for undo, records where the address came from (`feedUrlSource`: a `FeedUrlSources` value or the registry's `feedSource`) and when the registry confirmed it (`feedUrlCheckedAt`), clears every stored ETag and Last-Modified of the show (the row, its catalog feed and its feed-only extras), and marks a followed show for the next sync push (not for an address that came from the cloud, and never an unfollowed row, whose push could end the follow on another device). A write the registry didn't confirm keeps the stored daily-check stamp. It is a compare-and-set on the address it read, and `FeedAddressChange.onlyReplacing` narrows it to the address a caller read: a move is adopted only from the saved address that was read (a push or a restore may read another one, `FeedMoveFollower`), and a recovered address replaces only the address that failed. It records the show as pending (`PendingNotificationRegistrations`, the app's `SharedPreferencesPendingNotificationRegistrations`) before the write, and writes nothing (`NOT_SAVED`) when that record couldn't be saved, which no caller treats as done: a cloud pull fails before its pull time is saved (`FeedAddressNotSavedException`), the daily check leaves the show due for the next pass, and a restore reports the show, asks the app to register the show's notifications again when they are on (`FeedAddressWiring.requestNotificationRegistration`; any failure of that request is caught), and clears the mark only once Firebase acknowledged (`registrationSettled`), and only the request the job read before registering (`pendingMark`; each request gets a new mark), so an address change made while a job ran stays pending for the job that replaced it; each foreground pass asks again for every pending show (`resumePendingRegistrations`), so one lost with the process or given up on by the background job is never dropped. It reports a correction of a followed show's address as `feed_url_corrected` (`FeedAddressWiring.telemetry`). `rss:` shows and addresses that aren't HTTPS are refused. A cloud pull never puts back the address a confirmed correction (`FeedUrlSources.isConfirmedCorrection`: the registry, a feed move, a broken-feed report or the `/podcast` re-check) replaced: it returns `KEPT_CORRECTION` and marks the follow for a push, so the account gets the corrected address back. Merges of pulled rows write through `PodcastDao.upsertKeepingFeedAddress`, so an older copy never writes its address over one saved meanwhile. A follow keeps a saved HTTPS address (`FeedUrlSources.followedFeedUrl`), and a missing address from a show's details is filled through the writer. Every writer goes through it: missing-address recovery, the dead-address re-check, moved feeds, broken-feed reports, the daily check, auto-download, backup restores and cloud pulls (`SubscriptionSyncResolver(feedAddresses = …)` saves a pulled address for a Podcast Index show this way and leaves the row clean; without it, as in older tests, the row is written as before). - `PodcastNotificationRegistrar` (`SubscriptionRepository.registerPodcastNotifications`) registers a Podcast Index show's notifications again and waits for Firebase to acknowledge both the `tracked_podcasts/` node and the `new_ep_` topic, the same node and topic as turning notifications on. The node is one per show for every follower and its write replaces it whole, so it is written only with an address (once the show has a latest episode); otherwise only the topic is subscribed. `NotificationRegistrationOutcome.RETRY` is retried by the app's capped background job; rules that refuse the write give `DROPPED`. - **Daily address check.** `FeedAddressDailyCheck` runs at the start of each foreground pass (behind `RolloutSwitches.addressCheck`). It asks `POST /v2/shows/feeds` for every followed Podcast Index show with a saved address not checked in the last day, in chunks of at most 500, and swaps an address only for a confirmed move: the registry flags the show for address repair, the address is the registry's own (`registry:` source), HTTPS (an http-only address is never adopted, since background reads need HTTPS), different from the saved one, and not the address the saved one replaced (whatever replaced it: a feed move, dead-address recovery or a cloud pull). `fetchUrl: null` keeps the saved address. The new address is read first (`DirectFeedRecovery.addressRead`, a catalog refresh from it) and saved only when that read works, so an address the phone can't read never replaces one it can. Each show the registry answered for is stamped (`podcasts.feedUrlCheckedAt`): a swap writes the stamp with the address, and any other show is stamped after its swap had its chance. A failed call or `registryAvailable: false` stamps and swaps nothing. Follows with a change made on this phone that the account hasn't received yet (dirty and synced before) wait for the push; follows never synced to an account are checked. A different `epNs` is only logged (D12). -- **Broken-feed reports.** `FeedReporter` sends `POST /v2/feed-report` (`{id, status}`, never an address) for every failed background read of a Podcast Index show (`RefreshOutcome.Failure.reportStatus`), behind the same switch, never while offline, at most once per show a day and 15 a day in all, below the API's 20 so the show page has room (`FeedReportLimiter`; times kept in `FeedReportStamps`, the app's `SharedPreferencesFeedReportStamps`). Only a report the API answered counts: a failed send or a 429 uses up nothing. Its answer is an address to try only when it differs, is HTTPS and the saved address looks dead. +- **Broken-feed reports.** `FeedReporter` sends `POST /v2/feed-report` (`{id, status}`, never an address) for every failed background read of a Podcast Index show (`RefreshOutcome.Failure.reportStatus`), behind the same switch, never while offline, at most once per show a day and 20 a day in all, the API's own limit (`FeedReportLimiter`, one per app, shared with the show page, which stops 5 short so followed shows always have room; times kept in `FeedReportStamps`, the app's `SharedPreferencesFeedReportStamps`). Only a report the API answered counts: a failed send or a 429 uses up nothing. Its answer is an address to try only when it differs, is HTTPS and the saved address looks dead. - RSS notification activation requires explicit public-feed disclosure acceptance through `setNotificationsEnabled(..., acceptRssDisclosure = true, disclosedFeedUrl = savedUrl)`. `DeviceRssNotificationConsent` stores only URL fingerprints in `noBackupFilesDir`; changed URLs, restore and remote cloud settings cannot supply acceptance. Canonical `rss:` IDs require disclosure even with stale source metadata. Refused or revoked activation also removes this device's old registration; a library cleared during acceptance cannot publish a deleted show. Notification settings remain independent of auto-download. The shared checker receives only explicitly accepted public feeds. `TrackedPodcastRtdbLogic.topic` maps `rss:` to FCM-safe `rss_` while retaining canonical IDs in payloads. JSON/cloud RSS restore retains the original saved show ID across redirects. - RSS topics and per-device registration keys include a SHA-256 hash of the exact trimmed accepted URL. `DeviceRssNotificationRegistrations` keeps an atomic publication/cleanup journal in `noBackupFilesDir`, containing fingerprints and keys rather than URLs. Activation fails closed if the journal cannot be saved. `RssNotificationReconciler` awaits both RTDB and FCM acknowledgement, retains active entries and failed cleanup obligations, and retires old URL scopes or deleted/disabled shows. Live RTDB rules refuse whole-node deletes, so cleanup keeps the node and removes only its `feedUrl` child, which the checker needs to poll an RSS scope. The rules refuse that delete only when no node exists for the key (for example a legacy key this device never published), so a refusal still unsubscribes the FCM topic and clears the journal entry; any other failure keeps both. Migration scans disabled and unsubscribed canonical RSS rows too, removes legacy device keys/topics, and rejects restored notification flags without local consent. `canPresentEpisodeNotification` verifies the current saved URL, subscription, enabled flag and device-local acceptance. - `PodcastRepository` coordinates Podcast Index calls, recommendation endpoints, curated vibes, home bootstrap, similar episodes (with content languages), content catalog/v3, user feedback submission (`submitFeedback` transmitting categorized feedback, repro steps, optional email, and diagnostics to proxy), and RSS delegation. Canonical `PodcastEntity.toPodcast()` mapper standardizes entity conversion across data and feature modules, with module-boundary isolation between `:core:catalog` and `:core:rss`. Subscribed PI shows whose local catalog is **ready** (`LocalEpisodeCatalogPort.isReady`) serve list/search/`getEpisode` from Room only. Unsubscribe stamps a 14-day TTL; that catalog stays on disk but is **not** ready, so unsubscribed reads keep today’s PI pages plus cached extras merge. True `rss:` rows are unchanged. `getEpisodes` uses a bounded window (200), not `getAllNewest`. Followed shows' catalog refreshes never read a Podcast Index list: new episodes take recipe IDs in `:core:rss`. `loadPiEpisodesForSupplement` is the Podcast Index list feed-only extras (`EpisodeSupplementPort`) are compared with: up to 1,000 newest episodes from `episodes/match` (match fields only), falling back to the oldest page (at most 100) when that route fails, returns nothing, or its `useEpisodeMatchBaseline` off switch is set. It is strict, so a failure of both stops the extras refresh instead of comparing the feed with an empty list. Its callers are the show page's "Missing episodes?" refresh, the legacy subscription sync and push paths without a local catalog, and the restore of extras opt-ins; they stay until show pages read feeds for every listener. Full pages keep the 100 cap. `getEpisode` resolves an ID in `EpisodeLinkResolver`'s order without reading a feed: a stored row (local catalog, then the feed-only rows for a negative ID), then the phone's row of the episode the ID is a trusted alias of, then the network. It sends a revcipe-ry canonical positivge ID (`[2^52, 2^53 − 1]`) that no local row or alias holds to `/v2/episode` (with the show when a caller passes `getEpisode(episodeId, showId)` and the show is below 2^52): a recipe-range ID (`[2^52, 2^53 − 1]`)d never reachesto the v1 Podcast Index lookup, and an old Podcast Index ID falls back; to v1 `/episode` only when the v2 route can't answer; other legacy IDs keep v1. The episode keeps the ID asked for and carries the recipe ID and catalog key. A `needsFeed` answer, a 400/404/410 or an error is not found. Podcast Index IDs keep v1 `/episode`. `invalidateEpisodesCache` drops the 5-minute PI page cache for a feed (Podcast Info pull-to-refresh). `PodcastRepository` also owns the `/v2` client (`apiV2`, a `BoxLoreApiV2` built on first use): same base URL and the same cached OkHttp client as `api`, so the process keeps one HTTP cache. Tests can pass `boxLoreApiV2` the same way they pass `boxLoreApi`. `getEpisode` and the discovery routes below use its only caller so far. In-memory caching and pagination safety are managed by `PodcastRepositoryMemoryPolicy` (`clampPageLimit` caps requests to 100, and `TimedLruCache` bounds cache entries with LRU and TTL eviction to prevent OutOfMemoryError). +- **Show page episode source (P4 PR 16).** `PodcastRepository.showPageSource(feedId)` is `RSS` (`rss:` shows), `FOLLOWED_CATALOG` (a ready followed catalog), `FEED` (`RolloutSwitches.feedFirst` on, a Podcast Index show ID, and its feed not found unreadable on this page) or `SERVER_LIST`. Only the show page reads the stored copy: `getShowPageEpisodes` pages a `FEED` show from it (`FeedFirstWiring.copies`, a `ShowPageFeedPort`), more pages following while the copy has rows or isn't the whole archive, and an oldest-first page of a partial copy comes from the server's list. `getEpisodesPaginated` (Smart Queue, Android Auto, Home, downloads, the episode page) never reads the copy, which only the show page reads again. For a show whose catalog isn't ready it reads `/v2/episodes`, its recipe IDs mapped to the phone's owners (`DiscoveryEpisodeIds.toLocalOwners`, so a rollback of the flag shows the same IDs), and v1 `/episodes` only when `/v2/episodes` fails (decision D1). `getEpisodeWindow` and `getEpisodes` use the same IDs: around an episode the feed-first copy holds, the copy's window; else the newest 200 of `/v2/episodes` under local owners; v1 only when that fails (`LEGACY(own-ids)`). `FeedFirstShowPages` (`openFeedShowPage`, `readShowPageFeed`) reads the feed into the copy, `NEWEST` or `ARCHIVE` deep, down the failure ladder: offline shows the copy with no read and no report (`OFFLINE`); the address is the registry's (`/v2/podcast`), else the saved one, and an http-only feed goes through `/v2/feed/` (`v2Transport`: the copy's ETag as `If-None-Match`, the body capped like a direct read with `RssFeedClient.readFeedBody`, a `404 not_proxied` with an HTTPS `fetchUrl` read directly, under the download gate and per-show lock); a failed read is tried once more (not after a 404 or 410), then reported through `FeedReporter.reportTarget` (the app's one `FeedReportLimiter`, leaving `FOLLOWED_SHOWS_RESERVE` of the daily cap to followed shows; never for a followed show, whose own refresh reports and saves addresses), and a different address in the answer is read, through the transport when it is http-only; then the page uses the server's list with a notice (`FALLBACK`) until it is opened again. One `feed_fetch_result` per read (source `feed` or `transport`, `partial` or `complete`). Once a day per process, in the background so a page's first paint never waits, it removes unfollowed copies not opened for 30 days, under a size cap (decision D10). In-show search always runs on the phone: the stored copy (`LIMIT 200` in the query), else the newest 1,000 of `/v2/episodes` matched by title or description; v1 `episodes/search` is never called (decision D16). `invalidateEpisodesCache` also drops pages cached under the Podcast Index ID a `url:`, `guid:` or `itunes:` ID resolved to. - Episode links, pushes and Android Auto items that name an episode by ID and maybe its show resolve through `resolveEpisodeLink(episodeId, showId, onReadingFeed)` (`EpisodeLinkResolver`, `PodcastRepositoryEpisodeLinks.kt`). Order: a stored row; a trusted alias of the ID, or for a recipe ID with its show the one episode of that show whose trusted aliases carry it, answered with the phone's own row and ID; the API (an answer for an episode stored under another trusted ID or the same key is translated to the phone's row; a v1 answer, which has neither, by its enclosure address); and, for a `needsFeed` answer, one read of the show's feed through the existing reader (`RssEpisodeFeedReader`: the publisher's HTTPS address with `RssFeedClient`, an http-only address only through `/v2/feed/`), matched by recipe ID in the show's stored namespace. Only items whose key follows the contract match; a feed listing the same item twice gives its first copy in document order, and a recipe ID two different keys share matches nothing. Results are `EpisodeLinkResult.Found`, `NotFound` (nothing lists it: 400, 404 or 410) or `Unavailable` (a source couldn't be reached, including 401, 403, 408 and 429). `storedEpisodeForKey(show, key)` finds a stored row by catalog key (the local catalog by GUID or enclosure, then the show's feed-only rows by enclosure); alias, push and link lookups use it. Ambiguous and unresolved aliases never resolve. `EpisodeAliasReads` / `RoomEpisodeAliasReads` are the alias reads (`trustedKeyOf`, `trustedKeyForRecipe`, `namespaceOf`); `PodcastRepository(episodeAliases = …)` takes them (none by default, as in older tests) and `feedReader` is the test seam for the feed read. `localEpisodeForRecipe(show, recipeId)` is the push match by (show, recipe ID). - Episode share links: `EpisodeSharePreparer` (an `EpisodeShareTargetSource`) names the episode by its trusted alias's recipe ID and Podcast Index show (also for an RSS show linked to that show), else by `EpisodeShareIds.choose` on the episode's fields; phone-only negative or reserved IDs are never shared: an episode held only under one is shared by the recipe ID of its catalog key in the show's namespace, else as the show's link, and `rss:` shows keep today's link. With a known catalog key it registers the episode first through `EpisodeShareRegistrar.registerOnce` (`POST /v2/episodes/register`, `{show, guid, enclosureUrl}` with the audio address so the API can match a GUID that differs from Podcast Index's, or `{show, enclosureUrl}` when the key is the audio address), waiting at most `DEFAULT_TIMEOUT_MS` (2 s); the link goes out whatever the answer, and a timeout, offline, 401, 429 or server error hands the request to `scheduleRetry` (the app's background worker). 400/404 are dropped. - `syncSubscriptions` (`POST /sync`) is PI-only. `SubscriptionForegroundSync` sends it the shows that are **not** catalog-ready and have no saved HTTPS feed address, plus the shows whose own feed failed this pass. The PI ids go out in chunks of 20 (`PI_SYNC_CHUNK_SIZE`), at most two calls at a time (`PI_SYNC_PARALLELISM`), because the API serves only the first 40 ids of a call: whole-library callers (Smart Downloads, backup restore) used to lose every show past the 40th. A failed chunk drops only its own shows' tips. Ready ids contribute the Room newest tip; opted-in not-ready ids contribute the cached extras tip. Live feed freshen stays on `SubscriptionForegroundSync`. `getEpisodeWindow` is the bounded Smart Queue / Auto entry (never `getAllNewest`). @@ -165,7 +166,7 @@ Main Kotlin files should remain below 1000 lines; extracted helpers keep reposit ## Testing notes -- Saved feed addresses: `FeedAddressUpdaterTest` (Robolectric Room: one update writes both addresses, keeps the old one, resets every validator, marks the row dirty, re-registers and reports once; undo; `rss:` and non-HTTPS refused; a changed row left alone; fills, cloud addresses and unfollowed shows aren't corrections and an unfollowed row stays clean; a failed registration request of any kind keeps the address; a registration stays pending until acknowledged and is asked for again, also after the process died; no address written when its request couldn't be saved; a redirect from an address other than the saved one is never adopted; a re-follow keeps a corrected address), `FeedAddressDailyCheckTest` (501 followed shows make two calls, nothing again the same day; only confirmed moves, read before they are saved; an unreadable registry address never saved and tried again the next day; a swapped show stamped by its swap; never back to the address the saved one replaced, whatever replaced it; follows changed on the phone wait; a failed call or an unavailable registry stamps nothing, so the `/podcast` fallback runs once, and never for a show the check reached; the switch), `FeedAddressRecordedAnswersTest` (the API's own `/v2/shows/feeds` and `/v2/feed-report` answers, `fixtures/v2`, through the app's client: a registry move, a dump address, an unknown show, an http-only show, an unavailable registry and a refused report), `FeedReportsTest` and `SharedPreferencesFeedReportStampsTest` (once per show a day, 15 a day in all, a failed or refused send counting nothing, kept across processes, never offline, for `rss:` shows, without a read or with the switch off; only a readable new address offered back), `PodcastNotificationRegistrarTest` (the same node and topic, the shared node never written without its address, nothing without notifications, retry and refused outcomes, a cloud pull that changes only the address re-registering on the same node and topic, a pulled previous address never undoing a registry correction, and a merge never writing an older copy's address over a newer one), `SubscribedEpisodeCatalogTest` (moves followed on every refresh path, never for `rss:` shows or with the switch off, caller-handled requests left alone), and `DirectFeedRecoveryTest` (every failed read reported, the once-a-day `/podcast` re-check sharing the daily stamp, the report's address tried before the `/podcast` re-check, a passing failure keeping its address, a candidate replacing only the address that failed). +- Saved feed addresses: `FeedAddressUpdaterTest` (Robolectric Room: one update writes both addresses, keeps the old one, resets every validator, marks the row dirty, re-registers and reports once; undo; `rss:` and non-HTTPS refused; a changed row left alone; fills, cloud addresses and unfollowed shows aren't corrections and an unfollowed row stays clean; a failed registration request of any kind keeps the address; a registration stays pending until acknowledged and is asked for again, also after the process died; no address written when its request couldn't be saved; a redirect from an address other than the saved one is never adopted; a re-follow keeps a corrected address), `FeedAddressDailyCheckTest` (501 followed shows make two calls, nothing again the same day; only confirmed moves, read before they are saved; an unreadable registry address never saved and tried again the next day; a swapped show stamped by its swap; never back to the address the saved one replaced, whatever replaced it; follows changed on the phone wait; a failed call or an unavailable registry stamps nothing, so the `/podcast` fallback runs once, and never for a show the check reached; the switch), `FeedAddressRecordedAnswersTest` (the API's own `/v2/shows/feeds` and `/v2/feed-report` answers, `fixtures/v2`, through the app's client: a registry move, a dump address, an unknown show, an http-only show, an unavailable registry and a refused report), `FeedReportsTest` and `SharedPreferencesFeedReportStampsTest` (once per show a day, 20 a day in all, limiters sharing one store counting each other's reports, a failed or refused send counting nothing, kept across processes, never offline, for `rss:` shows, without a read or with the switch off; only a readable new address offered back), `PodcastNotificationRegistrarTest` (the same node and topic, the shared node never written without its address, nothing without notifications, retry and refused outcomes, a cloud pull that changes only the address re-registering on the same node and topic, a pulled previous address never undoing a registry correction, and a merge never writing an older copy's address over a newer one), `SubscribedEpisodeCatalogTest` (moves followed on every refresh path, never for `rss:` shows or with the switch off, caller-handled requests left alone), and `DirectFeedRecoveryTest` (every failed read reported, the once-a-day `/podcast` re-check sharing the daily stamp, the report's address tried before the `/podcast` re-check, a passing failure keeping its address, a candidate replacing only the address that failed). - `PodcastRepositoryPlayerPrefsTest` checks the player preferences (and their legacy-file migration) open on first use, not while the repository is constructed on the main thread at startup. - Unit tests live under `core/catalog/src/test`. - `PodcastRepositoryApiV2Test` (Robolectric, MockWebServer): the production `api` and `apiV2` reach the same base URL and open the HTTP cache once; test overrides open none. @@ -188,6 +189,7 @@ Main Kotlin files should remain below 1000 lines; extracted helpers keep reposit - `ShowNotesParserTest`, `CrossPromotionDetectorTest`, and `CrossPromotionResolverTest` cover the Blood Trails announcement, artwork-corroborated presenters, own-series/premium/topic/event/mix refusals, new-show and friends-at naming, colon, host and parenthetical title matches, shorter-name search, malformed/relative anchors, platform-label/handle separation, reserved profile routes, destination purpose and spoof domains, URL deduplication, safe timestamp linking, chapter boundaries, weekday-scheduled full-length introductions, false promotions and shorter-name prefixes, full-title matching, ambiguous/host matches, verified unnamed Apple links, retry, cancellation, and cache expiry. - `RoomFolderRepositoryTest` covers linking shows to a deleted folder. - Cloud sync identity (Robolectric, in-memory Room): `SyncEpisodeIdentityTest` covers which identity a pushed row and a rich removal carry (trusted, unresolved, ambiguous and linked RSS rows; a recipe ID two keys share; a recipe ID whose local ID another show's alias or catalog row holds; a served ID that isn't its recipe ID) and which aliases a pull records (unknown rows, matches by key and by a unique recipe ID, an unknown key winning over a recipe ID, settling an unresolved alias, rows left alone) and the episode tokens. `CloudSyncIdentityTest` runs `UserSyncCoordinator` against `FakeSyncServer`, which applies the sync API's rules (recipe fields kept when omitted, a recipe-range ID naming another show or key refused while the push answers ok, rich removals expanded and stored only when sent): the push and pull fields, a Podcast Index row from an older build plus a recipe owner read as one episode with the owner unchanged, an unknown row owning nothing, a row whose recipe ID names another episode here pushed without it so another device's removal of that episode by recipe ID can't reach its queue row, a served ID stored without a key, a pulled row of the playing episode (stored without touching the owner row; identified before the merge so a stored row keeps its progress), a large backlog pushed one bounded batch per sync, newest first, and the version-3 catch-up: a pull from the start first, then the cloud's own copies with the identity added (the newer progress and its own time, a completion the cloud has at the same last play kept, nothing new in another device's delta pull), every batch in the same sync, a run cut short by a failed push marking no row and finishing on the next sync without overwriting newer progress, a row removed here before the update staying removed, nothing again on the next sync, and an account switch purging the rich removals and catching up the new account. It also covers queue removals across IDs: removing an episode queued here under its RSS ID removes the copy an older build queued under the Podcast Index ID, a removal reaches the cloud copy the merge folded into the phone's item and it never comes back, and a stale rich removal never reaching an episode another device queued again. `QueueSyncIdentityTest` covers a removal under a Podcast Index ID removing the recipe ID's item, this phone's removal dropping the pulled item under another ID, the united plain and rich lists, a stale rich removal, the server's rich list replacing the phone's older one on a clean pull, a removal made stale by the identity a pulled item carries, a removal its own side still queues, one item per episode, the playing ID, and hydration from another ID's catalog row and from `/v2/episode` (MockWebServer). `SyncEpisodeIdentityTest` also covers the catch-up builder: only copies lacking a known identity, only the missing field, another device's key or recipe ID never overwritten. +- Feed-first show pages: `FeedFirstShowPagesTest` (the failure ladder against a scripted copy: offline, one retry but none after a 404 or 410, the report and the reported address, an http-only reported address read through the transport, no report for a followed show, the reserve for followed shows, the fallback and the way back, the registry or saved address and the transport, one event per read, opening state, the daily cleanup that never holds up the opening) and `PodcastRepositoryShowPagesTest` (Robolectric Room, MockWebServer and a real feed parser: the router per state, `/v2/episodes` under local owners and v1 only on failure, paging that stays stable while the copy grows from 100 to 150 episodes, the fallback switching the source, search on the phone that never calls v1 `episodes/search`, a flag rollback that changes no stored ID, alias or saved show row and calls no registration route, other callers reading the server's list while a followed http-only show's page reads its transported copy with its ETag, oldest-first pages of a partial copy, playback windows under the page's IDs, recorded `/v2/episodes`, `/v2/podcast` and `/v2/feed` answers (`src/test/resources/fixtures/v2`), and dropping the pages of a resolved `url:` show). ## CI relevance diff --git a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/FeedFirstShowPages.kt b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/FeedFirstShowPages.kt new file mode 100644 index 000000000..5234e760d --- /dev/null +++ b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/FeedFirstShowPages.kt @@ -0,0 +1,291 @@ +package cx.aswin.boxlore.core.catalog + +import cx.aswin.boxlore.core.domain.ports.FeedTelemetryPort +import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort.PodcastMeta +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.ReadOutcome +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.ReadRequest +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.Scope +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.Transported +import cx.aswin.boxlore.core.model.Episode +import cx.aswin.boxlore.core.model.FeedFetchCompleteness +import cx.aswin.boxlore.core.model.FeedFetchOutcome +import cx.aswin.boxlore.core.model.FeedFetchReport +import cx.aswin.boxlore.core.model.FeedFetchSource +import cx.aswin.boxlore.core.network.model.V2EpisodeIds +import cx.aswin.boxlore.core.network.model.V2FeedReadTarget +import cx.aswin.boxlore.core.network.model.V2PodcastResponse +import cx.aswin.boxlore.core.rss.ShowPageFeedRules +import java.util.concurrent.ConcurrentHashMap +import kotlinx.coroutines.CancellationException +import kotlinx.coroutines.CoroutineScope +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.SupervisorJob +import kotlinx.coroutines.launch + +/** Where a show page's episodes come from. */ +enum class ShowPageSource { + /** A show added by its feed address: the RSS table. */ + RSS, + + /** A followed show whose stored catalog is complete: Room only. */ + FOLLOWED_CATALOG, + + /** The show's own feed, through the phone's stored copy (`feed_first` on). */ + FEED, + + /** The server's list of the show's episodes (`/v2/episodes`). */ + SERVER_LIST, +} + +/** + * What a feed-first show page holds when it opens: whether a stored copy exists and is the whole + * archive, whether it is old enough to read again ([ShowPageFeedRules.RECHECK_AFTER_MS]), and + * whether the feed lists its oldest items first. + */ +data class ShowPageFeedState( + val cached: Boolean, + val archiveComplete: Boolean, + val recheckDue: Boolean, + val oldestFirst: Boolean, +) + +/** How a show page's feed read ended. */ +enum class ShowPageFeedResult { + /** The feed was read and stored. */ + UPDATED, + + /** The feed answered that the stored copy is current. */ + UNCHANGED, + + /** The phone is offline: the stored copy is shown as it is. */ + OFFLINE, + + /** The feed couldn't be read after a retry and a report: the page shows the server's list with a notice. */ + FALLBACK, +} + +/** + * [result] of a read, whether the stored copy now holds the whole archive, and whether the phone + * has a copy at all ([hasCopy]): an offline page without one has nothing saved to show. + */ +data class ShowPageFeedRead(val result: ShowPageFeedResult, val archiveComplete: Boolean, val hasCopy: Boolean = false,) + +/** + * The feed-first show page of a Podcast Index show: it reads the show's own feed into the phone's + * stored copy ([ShowPageFeedPort]) and decides when the page falls back to the server's list. + * + * The failure ladder of one read ([read]): + * 1. Offline: nothing is read and nothing is reported; the stored copy is shown as it is. + * 2. The address comes from `/v2/podcast` (the registry), else the saved one. An http-only address + * is read through boxlore's transport (`/v2/feed/`), with the copy's ETag. + * 3. A failed read is tried once more, except after a 404 or 410. + * 4. Then the failure is reported (`/v2/feed-report`, rate-limited, never offline, and not for a + * followed show, whose own refresh reports and saves addresses); when the answer names a + * different address, that one is read, through the transport when it is http-only. + * 5. Then the page uses `/v2/episodes` with a notice ([ShowPageFeedResult.FALLBACK]) until the page + * is opened again ([open]). + * + * Each read sends one `feed_fetch_result` (decision D11). Once a day, away from the page's first + * paint ([background]), it also removes the stored copies of unfollowed shows not opened for 30 + * days, under a size cap (decision D10). + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page). + * COMPAT(own-ids): show pages read the show's own feed. Permanent: never remove (MIGRATION.md §5). + */ +@Suppress("LongParameterList", "kotlin:S107") // Each dependency is one rung of the ladder or a test seam. +class FeedFirstShowPages( + private val copies: ShowPageFeedPort?, + private val registry: suspend (Long) -> V2PodcastResponse?, + private val transport: suspend (showId: Long, ifNoneMatch: String?) -> Transported, + private val reporter: FeedReporter?, + private val isOnline: () -> Boolean, + private val telemetry: FeedTelemetryPort = FeedTelemetryPort.NONE, + private val nowMillis: () -> Long = System::currentTimeMillis, + private val background: CoroutineScope = CoroutineScope(SupervisorJob() + Dispatchers.IO), + private val isFollowed: suspend (String) -> Boolean = { false }, +) { + private val fallbackShows = ConcurrentHashMap.newKeySet() + private val metas = ConcurrentHashMap() + + @Volatile + private var lastCleanUpAt: Long? = null + + /** True while [podcastId]'s page uses the server's list because its feed couldn't be read. */ + fun isInFallback(podcastId: String): Boolean = podcastId in fallbackShows + + /** Whether the phone is online now, as the ladder sees it. */ + fun online(): Boolean = isOnline() + + /** The show details last given to [open] or [read], for episodes listed from the stored copy. */ + fun metaFor(podcastId: String): PodcastMeta? = metas[podcastId] + + suspend fun cached(podcastId: String): ShowPageFeedPort.CachedShow? = safely(null) { copies?.cached(podcastId) } + + /** Bounded in-show search over [podcastId]'s stored copy; null when the phone has none. */ + suspend fun search(podcastId: String, query: String, meta: PodcastMeta, limit: Int): List? { + val port = copies ?: return null + cached(podcastId) ?: return null + return safely(null) { port.search(podcastId, query, limit, meta) } + } + + /** + * What the page holds as it opens; notes the opening and starts the daily cleanup in the + * background. A page that opens tries the feed again, even after an earlier page fell back to + * the server's list. + */ + suspend fun open(podcastId: String, meta: PodcastMeta): ShowPageFeedState { + remember(podcastId, meta) + fallbackShows.remove(podcastId) + val cached = cached(podcastId) + val now = nowMillis() + if (cached != null) safely(Unit) { copies?.markOpened(podcastId, now) } + scheduleCleanUp(now, podcastId) + return ShowPageFeedState( + cached = cached != null, + archiveComplete = cached?.archiveComplete == true, + recheckDue = cached == null || now - cached.fetchedAt !in 0 until ShowPageFeedRules.RECHECK_AFTER_MS, + oldestFirst = cached?.oldestFirst == true, + ) + } + + /** One read of the show's feed, [scope] deep, down the failure ladder. [savedFeedUrl]: the address the phone has, if any. */ + suspend fun read(podcastId: String, meta: PodcastMeta, scope: Scope, savedFeedUrl: String?): ShowPageFeedRead { + remember(podcastId, meta) + val showId = V2EpisodeIds.parseShowId(podcastId) + if (copies == null || showId == null) return finish(podcastId, ShowPageFeedResult.FALLBACK) + if (!isOnline()) return finish(podcastId, ShowPageFeedResult.OFFLINE) + val answer = safely(null) { registry(showId) } + val saved = savedFeedUrl?.takeIf(String::isNotBlank) + val target = + answer?.readTarget(showId) + ?: saved?.let(V2FeedReadTarget::Direct) + ?: return finish(podcastId, ShowPageFeedResult.FALLBACK) + // The publisher's own address, for the read's request; a transport read never records it. + val publisher = answer?.fetchUrl?.takeIf(String::isNotBlank) ?: saved.orEmpty() + val request = ReadRequest(podcastId, publisher, scope, meta, answer?.validEpNs) + val attempt = afterReport(podcastId, request, withRetry(request, target), saved) ?: return finish(podcastId, ShowPageFeedResult.OFFLINE) + report(podcastId, attempt) + return when (attempt.outcome) { + is ReadOutcome.Failed -> finish(podcastId, ShowPageFeedResult.FALLBACK) + is ReadOutcome.Stored -> finish(podcastId, ShowPageFeedResult.UPDATED) + ReadOutcome.NotModified -> finish(podcastId, ShowPageFeedResult.UNCHANGED) + } + } + + /** The read, and once more when it failed in a way a second try may not (not a 404 or 410). */ + private suspend fun withRetry(request: ReadRequest, target: V2FeedReadTarget): Attempt { + val first = Attempt(target, readOnce(request, target)) + return if (first.outcome is ReadOutcome.Failed && first.retryable) first.copy(outcome = readOnce(request, target)) else first + } + + /** + * After a failed read, the report and a read of the address it names, when it names a + * different one. Null when the phone went offline meanwhile: nothing is reported then. A + * followed show isn't reported here: its own refresh reports it and saves a working address + * (P4 PR 14), and the one report a day it may send stays for that. + */ + private suspend fun afterReport(podcastId: String, request: ReadRequest, attempt: Attempt, savedFeedUrl: String?): Attempt? { + val failed = attempt.outcome as? ReadOutcome.Failed ?: return attempt + if (!isOnline()) return null + if (safely(false) { isFollowed(podcastId) }) return attempt + val used = (attempt.target as? V2FeedReadTarget.Direct)?.url ?: savedFeedUrl + val next = safely(null) { reporter?.reportTarget(podcastId, failed.reportStatus, used) } ?: return attempt + if (next == attempt.target) return attempt + return Attempt(next, readOnce(request, next)) + } + + private data class Attempt(val target: V2FeedReadTarget, val outcome: ReadOutcome) { + /** A 404 or 410 says the address is gone: a second try would see the same. */ + val retryable: Boolean get() = (outcome as? ReadOutcome.Failed)?.reportStatus !in GONE_STATUSES + } + + private suspend fun readOnce(request: ReadRequest, target: V2FeedReadTarget): ReadOutcome { + val port = copies ?: return ReadOutcome.Failed(OTHER) + return safely(ReadOutcome.Failed(OTHER)) { + when (target) { + is V2FeedReadTarget.Direct -> port.read(request.copy(feedUrl = target.url)) + is V2FeedReadTarget.ViaApi -> port.readTransport(request) { etag -> transport(target.showId, etag) } + } + } + } + + /** + * The read's result, with whether the copy is now the whole archive. A fallback puts the page on + * the server's list; a read that worked takes it back to the feed; offline changes neither. + */ + private suspend fun finish(podcastId: String, result: ShowPageFeedResult): ShowPageFeedRead { + when (result) { + ShowPageFeedResult.FALLBACK -> fallbackShows.add(podcastId) + ShowPageFeedResult.UPDATED, ShowPageFeedResult.UNCHANGED -> fallbackShows.remove(podcastId) + ShowPageFeedResult.OFFLINE -> Unit + } + val cached = cached(podcastId) + return ShowPageFeedRead(result, archiveComplete = cached?.archiveComplete == true, hasCopy = cached != null) + } + + private fun remember(podcastId: String, meta: PodcastMeta) { + if (meta.title != null || meta.imageUrl != null) metas[podcastId] = meta + } + + /** Starts the daily cleanup in [background], at most once a day, keeping the show [keep] being opened. */ + private fun scheduleCleanUp(now: Long, keep: String) { + val last = lastCleanUpAt + if (last != null && now - last in 0 until CLEANUP_INTERVAL_MS) return + lastCleanUpAt = now + background.launch { safely(0) { copies?.cleanUp(now, keep) ?: 0 } } + } + + /** The one `feed_fetch_result` of this read (decision D11). Never carries an address. */ + private fun report(podcastId: String, attempt: Attempt) { + try { + telemetry.feedFetched(showPageFetchReport(podcastId, attempt.target, attempt.outcome)) + } catch (error: IllegalStateException) { + android.util.Log.w(TAG, "feed_fetch_result not sent", error) + } + } + + companion object { + private const val OTHER = "other" + private val GONE_STATUSES = setOf("http_404", "http_410") + private const val CLEANUP_INTERVAL_MS = 24L * 60L * 60L * 1000L + + /** The `feed_fetch_result` outcome of a failed read, from its `/v2/feed-report` status. */ + fun failureOutcome(reportStatus: String): FeedFetchOutcome = when { + reportStatus in GONE_STATUSES -> FeedFetchOutcome.GONE + reportStatus.startsWith(HTTP_PREFIX) -> FeedFetchOutcome.HTTP_ERROR + reportStatus == "timeout" -> FeedFetchOutcome.TIMEOUT + reportStatus == "parse" -> FeedFetchOutcome.PARSE_ERROR + else -> FeedFetchOutcome.NETWORK_ERROR + } + } +} + +private const val TAG = "FeedFirstShowPages" +private const val HTTP_PREFIX = "http_" + +/** The `feed_fetch_result` of one show-page read of [target] that ended in [outcome]. */ +internal fun showPageFetchReport(podcastId: String, target: V2FeedReadTarget, outcome: ReadOutcome): FeedFetchReport.ShowPage { + val source = if (target is V2FeedReadTarget.ViaApi) FeedFetchSource.TRANSPORT else FeedFetchSource.FEED + return when (outcome) { + is ReadOutcome.Stored -> { + val completeness = if (outcome.complete) FeedFetchCompleteness.COMPLETE else FeedFetchCompleteness.PARTIAL + FeedFetchReport.ShowPage(podcastId, source, FeedFetchOutcome.OK, completeness) + } + ReadOutcome.NotModified -> FeedFetchReport.ShowPage(podcastId, source, FeedFetchOutcome.NOT_MODIFIED) + is ReadOutcome.Failed -> { + val status = outcome.reportStatus.takeIf { it.startsWith(HTTP_PREFIX) }?.removePrefix(HTTP_PREFIX)?.toIntOrNull() + FeedFetchReport.ShowPage(podcastId, source, FeedFirstShowPages.failureOutcome(outcome.reportStatus), httpStatus = status) + } + } +} + +/** [block]'s answer, or [fallback] when it fails; cancellation propagates. */ +private suspend fun safely(fallback: T, block: suspend () -> T): T = try { + block() +} catch (error: CancellationException) { + throw error +} catch (error: Exception) { + android.util.Log.w(TAG, "Show page feed step failed", error) + fallback +} diff --git a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/FeedReports.kt b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/FeedReports.kt index 947b7d856..669bcc200 100644 --- a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/FeedReports.kt +++ b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/FeedReports.kt @@ -4,6 +4,7 @@ import android.content.SharedPreferences import android.util.Log import cx.aswin.boxlore.core.network.BoxLoreApiV2 import cx.aswin.boxlore.core.network.model.V2EpisodeIds +import cx.aswin.boxlore.core.network.model.V2FeedReadTarget import cx.aswin.boxlore.core.network.model.V2FeedReportRequest import cx.aswin.boxlore.core.network.model.V2FeedReportResponse import cx.aswin.boxlore.core.network.model.V2FeedReportStatus @@ -61,23 +62,27 @@ class SharedPreferencesFeedReportStamps(private val prefs: SharedPreferences) : /** * Limits broken-feed reports: one per show per [PER_SHOW_INTERVAL_MS], and at most [DAILY_CAP] a - * day in all. The API takes 20 a day from one phone; the background pass keeps 5 of them for the - * show page. Only a report the API answered counts ([record]): a failed or refused send uses up - * nothing. + * day in all, the API's own limit. Followed shows may use the whole day; the show page stops + * [FOLLOWED_SHOWS_RESERVE] short, so followed shows always have room. Only a report the API + * answered counts ([record]): a failed or refused send uses up nothing. Two sends at the same moment + * can pass [allows] together; the API then answers the extra one with a 429, which counts nothing. * * Added in P4 PR 14 of the own-episode-IDs plan (plan section 11, followed shows: reports). */ class FeedReportLimiter(private val stamps: FeedReportStamps) { - /** True when a report of [podcastId] may be sent at [nowMillis]. */ - @Synchronized - fun allows(podcastId: String, nowMillis: Long): Boolean { + /** + * True when a report of [podcastId] may be sent at [nowMillis]: the show wasn't reported in the + * last day, and the day's reports leave [keepFree] of [DAILY_CAP] for other callers (the show + * page keeps [FOLLOWED_SHOWS_RESERVE] free for followed shows, P4 PR 16). Limiters sharing one + * store lock on it, so each one counts the reports the others recorded. + */ + fun allows(podcastId: String, nowMillis: Long, keepFree: Int = 0): Boolean = synchronized(stamps) { val recent = recent(nowMillis) - return podcastId !in recent && recent.size < DAILY_CAP + podcastId !in recent && recent.size < DAILY_CAP - keepFree.coerceIn(0, DAILY_CAP) } /** Counts a report of [podcastId] the API answered at [nowMillis]. */ - @Synchronized - fun record(podcastId: String, nowMillis: Long) { + fun record(podcastId: String, nowMillis: Long) = synchronized(stamps) { stamps.mark(podcastId, nowMillis, keepSince = nowMillis - PER_SHOW_INTERVAL_MS) } @@ -89,8 +94,11 @@ class FeedReportLimiter(private val stamps: FeedReportStamps) { companion object { const val PER_SHOW_INTERVAL_MS = 24L * 60L * 60L * 1000L - /** Below the API's 20 a day per phone, so the show page's own reports still go through. */ - const val DAILY_CAP = 15 + /** The API's limit: 20 reports a day from one phone. */ + const val DAILY_CAP = 20 + + /** Reports of the day the show page leaves to followed shows (P4 PR 16). */ + const val FOLLOWED_SHOWS_RESERVE = 5 } } @@ -110,15 +118,36 @@ class FeedReporter( private val isOnline: () -> Boolean, private val enabled: () -> Boolean, private val nowMillis: () -> Long = System::currentTimeMillis, + private val keepFree: Int = 0, ) { /** * @param status what the failed read saw, as the route takes it; null when no read happened * @return the API's HTTPS address when it differs from [savedFeedUrl], else null */ - suspend fun report(podcastId: String, status: String?, savedFeedUrl: String?): String? { - if (status == null || !V2FeedReportStatus.isValid(status) || !enabled()) return null + suspend fun report(podcastId: String, status: String?, savedFeedUrl: String?): String? = + (reportTarget(podcastId, status, savedFeedUrl) as? V2FeedReadTarget.Direct)?.url + + /** + * [report] for a caller that can also read through boxlore's transport: the answer's address + * as a read target, [V2FeedReadTarget.ViaApi] when it is http-only (`fetchVia: proxy`, + * `V2-ROUTES.md` `/v2/feed-report`), or a different HTTPS address. Null when nothing was sent or + * the answer names the address already read. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, the show page's failure ladder). + */ + suspend fun reportTarget(podcastId: String, status: String?, savedFeedUrl: String?): V2FeedReadTarget? { val showId = V2EpisodeIds.parseShowId(podcastId) ?: return null - if (!isOnline() || !limiter.allows(podcastId, nowMillis())) return null + val answer = sendReport(podcastId, showId, status) ?: return null + if (answer.fetchVia == V2FetchVia.PROXY) return answer.readTarget(showId) + if (answer.fetchVia != null && answer.fetchVia != V2FetchVia.DIRECT) return null + val https = TrackedPodcastRtdbLogic.httpsFeedUrl(answer.fetchUrl) ?: return null + return https.takeIf { it != savedFeedUrl?.trim() }?.let(V2FeedReadTarget::Direct) + } + + /** One report within the limits; null when none was sent or it got no answer. */ + private suspend fun sendReport(podcastId: String, showId: Long, status: String?): V2FeedReportResponse? { + if (status == null || !V2FeedReportStatus.isValid(status) || !enabled()) return null + if (!isOnline() || !limiter.allows(podcastId, nowMillis(), keepFree)) return null val answer = try { send(V2FeedReportRequest(id = showId, status = status)) @@ -129,9 +158,7 @@ class FeedReporter( null } ?: return null limiter.record(podcastId, nowMillis()) - if (answer.fetchVia != null && answer.fetchVia != V2FetchVia.DIRECT) return null - val https = TrackedPodcastRtdbLogic.httpsFeedUrl(answer.fetchUrl) ?: return null - return https.takeIf { it != savedFeedUrl?.trim() } + return answer } companion object { diff --git a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepository.kt b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepository.kt index 66c36e9d2..2016375d0 100644 --- a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepository.kt +++ b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepository.kt @@ -60,8 +60,8 @@ class PodcastRepository( /** * The v2 build's rollout switches (feed-first show pages, v2 discovery, address check, upgrade * repair), from the app's PostHog flags. Added in P4 (plan section 11, Rollback). Discovery - * reads [RolloutSwitches.v2Discovery] per request ([usesV2Discovery]); the other code paths - * that read them land later. + * reads [RolloutSwitches.v2Discovery] per request ([usesV2Discovery]) and + * [RolloutSwitches.feedFirst] for the show-page source ([showPageSource], P4 PR 16). */ val rolloutSwitches: RolloutSwitches = RolloutSwitches(), /** Trusted alias reads for links, pushes, shares and Android Auto (P4 PR 13); none in older tests. */ @@ -74,6 +74,11 @@ class PodcastRepository( * Added in P4 PR 12 (own-IDs plan section 9, v2 discovery routes). */ internal val discoveryEpisodeIds: DiscoveryEpisodeIds = DiscoveryEpisodeIds.Unmapped, + /** + * The feed-first show page's stored copies, connectivity, report limits and telemetry. The + * default stores nothing, so show pages read the server's list (P4 PR 16, plan section 11). + */ + feedFirstWiring: FeedFirstWiring = FeedFirstWiring(), ) : cx.aswin.boxlore.core.domain.ports.PodcastCatalogPort { internal val episodeFeedReader: EpisodeFeedReader by lazy { feedReader ?: RssEpisodeFeedReader(cx.aswin.boxlore.core.rss.RssFeedClient()) { apiV2 } } @@ -88,6 +93,29 @@ class PodcastRepository( */ val apiV2: BoxLoreApiV2 by lazy { boxLoreApiV2 ?: NetworkModule.createBoxLoreApiV2(baseUrl, cachedClient) } + /** + * The feed-first show page: reads a show's own feed into the phone's stored copy, with its + * failure ladder (P4 PR 16, own-IDs plan section 11). + */ + val feedFirst: FeedFirstShowPages by lazy { + FeedFirstShowPages( + copies = feedFirstWiring.copies, + registry = { showId -> v2ShowAddress(showId) }, + transport = { showId, etag -> v2Transport(showId, etag) }, + reporter = + FeedReporter( + send = { request -> FeedReporter.sendWith(apiV2, request) }, + limiter = feedFirstWiring.reportLimiter, + isOnline = feedFirstWiring.isOnline, + enabled = rolloutSwitches.addressCheck, + keepFree = FeedReportLimiter.FOLLOWED_SHOWS_RESERVE, + ), + isOnline = feedFirstWiring.isOnline, + telemetry = feedFirstWiring.telemetry, + isFollowed = feedFirstWiring.isFollowed, + ) + } + /** Last good `/v2` discovery answers of this process (P4 PR 12, plan section 9). */ internal val v2LastGood = V2DiscoveryLastGood() @@ -279,6 +307,12 @@ class PodcastRepository( } } + /** + * In-show search, always on the phone: the RSS table, a followed show's catalog, the feed-first + * stored copy (the show page completes its archive first), or else the server's list of the + * show (`/v2/episodes`). Never v1 `episodes/search`, whose answers carry served IDs that would + * mix with recipe IDs (P4 PR 16, decision D16). + */ suspend fun searchEpisodes(feedId: String, query: String,): List = withContext(Dispatchers.IO) { if (feedId.startsWith("rss:")) { return@withContext searchRssEpisodes(feedId, query) @@ -286,7 +320,55 @@ class PodcastRepository( if (isLocalCatalogReady(feedId)) { return@withContext searchLocalCatalog(feedId, query) } - searchNetworkEpisodes(feedId, query) + if (showPageSource(feedId) == ShowPageSource.FEED) { + searchFeedCopy(feedId, query)?.let { return@withContext it } + } + searchServerList(feedId, query) + } + + /** + * Where [feedId]'s show page reads its episodes: the RSS table, a followed show's complete + * catalog, the show's own feed (`feed_first` on, a Podcast Index show whose feed was not found + * unreadable on this page), or the server's list. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page; decision D1). + */ + suspend fun showPageSource(feedId: String): ShowPageSource = withContext(Dispatchers.IO) { + when { + feedId.startsWith("rss:") -> ShowPageSource.RSS + isLocalCatalogReady(feedId) -> ShowPageSource.FOLLOWED_CATALOG + isFeedFirstShow(feedId) && !feedFirst.isInFallback(feedId) -> ShowPageSource.FEED + else -> ShowPageSource.SERVER_LIST + } + } + + /** + * What the feed-first show page of [feedId] holds as it opens ([FeedFirstShowPages.open]), or + * null when the page reads another source (`rss:` shows, a followed show's ready catalog, + * `feed_first` off). A page that opens tries the feed again after an earlier fallback. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page). + */ + suspend fun openFeedShowPage(feedId: String, meta: cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort.PodcastMeta,): ShowPageFeedState? = + withContext(Dispatchers.IO) { + if (feedId.startsWith("rss:") || !isFeedFirstShow(feedId) || isLocalCatalogReady(feedId)) return@withContext null + feedFirst.open(feedId, meta) + } + + /** + * Reads [feedId]'s feed into its stored copy, [scope] deep, down the failure ladder + * ([FeedFirstShowPages.read]), then drops cached pages of the show so the next page reads the + * new copy, or the server's list after a fallback. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page). + */ + suspend fun readShowPageFeed( + feedId: String, + meta: cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort.PodcastMeta, + scope: cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.Scope, + savedFeedUrl: String?, + ): ShowPageFeedRead = withContext(Dispatchers.IO) { + feedFirst.read(feedId, meta, scope, savedFeedUrl).also { invalidateEpisodesCache(feedId) } } // Opened on first use: the migration reads the prefs file from disk, and this repository is @@ -337,13 +419,46 @@ class PodcastRepository( if (isLocalCatalogReady(feedId)) { return@withContext getLocalCatalogPage(feedId, safeLimit, offset, sort) } - getNetworkEpisodesPaginated(feedId, safeLimit, offset, sort, mergeSupplements) + getServerListPage(feedId, safeLimit, offset, sort, mergeSupplements) + } + + /** + * [getEpisodesPaginated] for the show page itself: the feed-first stored copy when the page + * reads the feed ([ShowPageSource.FEED]), else the same sources. Only the show page reads the + * copy, because only it reads the feed again (on open, pull to refresh and archive actions); + * Smart Queue, Android Auto, Home and downloads keep reading the server's list + * ([getEpisodesPaginated]), so a copy that stops being read never goes stale under them. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page; decision D1). + */ + suspend fun getShowPageEpisodes( + feedId: String, + limit: Int = 20, + offset: Int = 0, + sort: String = "newest", + ): EpisodePage = withContext(Dispatchers.IO) { + if (showPageSource(feedId) == ShowPageSource.FEED) { + feedCopyPage(feedId, clampPageLimit(limit), offset, sort)?.let { return@withContext it } + } + getEpisodesPaginated(feedId, limit, offset, sort) } - /** Drop cached PI episode pages for [feedId] so the next read hits the network. */ + /** + * True when [feedId]'s show page reads its feed first (`feed_first` on, a Podcast Index show), + * whether or not this page has fallen back: such a page never offers "Missing episodes?". + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, refresh replaces the chip). + */ + fun isFeedFirstShowPage(feedId: String): Boolean = !feedId.startsWith("rss:") && isFeedFirstShow(feedId) + + /** + * Drop cached episode pages for [feedId] so the next read hits the network. Pages are cached + * under the Podcast Index ID a `url:`, `guid:` or `itunes:` ID resolved to, so that ID's pages + * go too (before P4 PR 16 only pages under [feedId] itself were dropped). + */ fun invalidateEpisodesCache(feedId: String) { - val prefix = "$feedId|" - episodesCache.invalidateIf { it.startsWith(prefix) } + val prefixes = setOfNotNull(feedId, resolvedFeedIds[feedId]).map { "$it|" } + episodesCache.invalidateIf { key -> prefixes.any(key::startsWith) } } /** @@ -389,7 +504,13 @@ class PodcastRepository( EpisodePage(emptyList(), false) } - private suspend fun getNetworkEpisodesPaginated( + /** + * v1 `/episodes` pages of a Podcast Index show, merged with its cached feed extras. + * + * LEGACY(own-ids): the last rung of the show page's list, used only when `/v2/episodes` fails. + * Remove at sunset step S7 (MIGRATION.md §6) once `feed_first` has been at 100% for 30 days. + */ + internal suspend fun getNetworkEpisodesPaginated( feedId: String, limit: Int, offset: Int, @@ -397,6 +518,7 @@ class PodcastRepository( mergeSupplements: Boolean, ): EpisodePage { val resolvedId = resolvePodcastIndexFeedId(feedId) + rememberResolvedFeedId(feedId, resolvedId) val cacheKey = "$resolvedId|$limit|$offset|$sort" val cached = episodesCache[cacheKey] if (cached != null) { @@ -438,7 +560,7 @@ class PodcastRepository( } } - private suspend fun maybeMergeSupplements( + internal suspend fun maybeMergeSupplements( podcastId: String, page: EpisodePage, offset: Int, @@ -927,7 +1049,8 @@ class PodcastRepository( if (isLocalCatalogReady(feedId)) { return@withContext getLocalCatalogWindow(feedId, sort, aroundEpisodeId) } - getNetworkEpisodeWindow(feedId) + // The show page's IDs (P4 PR 16): the feed-first copy around its episode, else /v2/episodes. + getShowEpisodeWindow(feedId, aroundEpisodeId, sort) } override suspend fun getEpisode(episodeId: String): Episode? = getEpisodeImpl(episodeId, showId = null) @@ -948,7 +1071,14 @@ class PodcastRepository( // Catalog caches are endpoint-versioned so an otherwise-valid v1/v2 response cannot // suppress the v3 fetch and then invalidate grouped v3 section responses. - private val episodesCache = TimedLruCache(maxEntries = 30, ttlMillis = DEFAULT_EPISODE_CACHE_TTL_MS) + internal val episodesCache = TimedLruCache(maxEntries = 30, ttlMillis = DEFAULT_EPISODE_CACHE_TTL_MS) + + /** The Podcast Index ID each `url:`, `guid:` or `itunes:` show ID resolved to, for [invalidateEpisodesCache]. */ + internal val resolvedFeedIds = java.util.concurrent.ConcurrentHashMap() + + internal fun rememberResolvedFeedId(feedId: String, resolvedId: String) { + if (feedId != resolvedId) resolvedFeedIds[feedId] = resolvedId + } internal val recommendationsCache = TimedLruCache>(maxEntries = 20, ttlMillis = DEFAULT_RECOMMENDATIONS_CACHE_TTL_MS) internal val becauseYouLikeCache = TimedLruCache(maxEntries = 20, ttlMillis = DEFAULT_RECOMMENDATIONS_CACHE_TTL_MS) } diff --git a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryEpisodeLinks.kt b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryEpisodeLinks.kt index b528d7036..f5630109d 100644 --- a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryEpisodeLinks.kt +++ b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryEpisodeLinks.kt @@ -201,7 +201,8 @@ internal class RssEpisodeFeedReader( is V2FeedReadTarget.ViaApi -> { val response = api().getFeed(target.showId).awaitResponse() val body = response.body()?.takeIf { response.isSuccessful } ?: error("feed proxy answered ${response.code()}") - val bytes = body.use { it.bytes() } + // The same size cap as a direct read (P4 PR 16): a huge document never fills memory. + val bytes = body.use { RssFeedClient.readFeedBody(it.byteStream()) } feedClient.parse(requireNotNull(address.transportUrl), bytes) } null -> null diff --git a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryNetworkLookups.kt b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryNetworkLookups.kt index 9635dd15a..d355b4f5a 100644 --- a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryNetworkLookups.kt +++ b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryNetworkLookups.kt @@ -14,22 +14,6 @@ internal suspend fun PodcastRepository.searchRssEpisodes(feedId: String, query: emptyList() } -internal suspend fun PodcastRepository.searchNetworkEpisodes(feedId: String, query: String,): List = try { - val resolvedId = resolvePodcastIndexFeedId(feedId) - val response = api.searchEpisodes(publicKey, resolvedId, query).execute() - val network = - if (response.isSuccessful && response.body() != null) { - response.body()!!.items.mapNotNull { mapToEpisode(it) } - } else { - emptyList() - } - unionCachedSupplementSearch(resolvedId, query, network) -} catch (e: kotlinx.coroutines.CancellationException) { - throw e -} catch (e: Exception) { - emptyList() -} - internal suspend fun PodcastRepository.getEpisodesImpl(feedId: String): List = withContext(Dispatchers.IO) { if (feedId.startsWith("rss:")) { return@withContext getRssEpisodeWindow(feedId) @@ -37,7 +21,8 @@ internal suspend fun PodcastRepository.getEpisodesImpl(feedId: String): List = try { @@ -49,6 +34,10 @@ internal suspend fun PodcastRepository.getRssEpisodeWindow(feedId: String, aroun emptyList() } +/** + * LEGACY(own-ids): v1 `/episodes` window, served IDs. Since P4 PR 16 only when `/v2/episodes` fails + * ([getShowEpisodeWindow]). Remove at sunset step S7 (MIGRATION.md §6). + */ internal suspend fun PodcastRepository.getNetworkEpisodeWindow(feedId: String): List = try { val resolvedId = resolvePodcastIndexFeedId(feedId) val response = @@ -75,6 +64,10 @@ internal suspend fun PodcastRepository.getAllRssEpisodes(feedId: String): List = try { val resolvedId = resolvePodcastIndexFeedId(feedId) val page1Response = diff --git a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryShowPageTransport.kt b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryShowPageTransport.kt new file mode 100644 index 000000000..f702fb32a --- /dev/null +++ b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryShowPageTransport.kt @@ -0,0 +1,53 @@ +package cx.aswin.boxlore.core.catalog + +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort +import cx.aswin.boxlore.core.rss.FeedTooLargeException +import cx.aswin.boxlore.core.rss.RssFeedClient +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.contentOrNull +import kotlinx.serialization.json.jsonObject +import kotlinx.serialization.json.jsonPrimitive +import retrofit2.awaitResponse + +/** + * One read of an http-only show's feed through boxlore's transport (`/v2/feed/`), with the + * copy's ETag ([ifNoneMatch]): the document, read at most [RssFeedClient.readFeedBody]'s cap; a + * 304; a `404 not_proxied` naming an HTTPS address to read directly; or what failed, as + * `/v2/feed-report` words it. Network errors propagate to the ladder, which words them. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page; `V2-ROUTES.md` `/v2/feed/`). + */ +internal suspend fun PodcastRepository.v2Transport(showId: Long, ifNoneMatch: String?): ShowPageFeedPort.Transported { + val response = apiV2.getFeed(showId, ifNoneMatch).awaitResponse() + return when { + response.code() == HTTP_NOT_MODIFIED -> ShowPageFeedPort.Transported.NotModified + response.isSuccessful -> { + val body = response.body() ?: return ShowPageFeedPort.Transported.Failed(OTHER_STATUS) + try { + ShowPageFeedPort.Transported.Document( + bytes = body.use { RssFeedClient.readFeedBody(it.byteStream()) }, + etag = response.headers()["ETag"], + ) + } catch (_: FeedTooLargeException) { + ShowPageFeedPort.Transported.Failed(OTHER_STATUS) + } + } + else -> transportFailure(response.code(), response.errorBody()?.use { it.string() }) + } +} + +/** A failed transport answer: a `404 not_proxied` with an HTTPS `fetchUrl` is read directly; anything else is `http_`. */ +internal fun transportFailure(code: Int, errorBody: String?): ShowPageFeedPort.Transported { + if (code == HTTP_NOT_FOUND && errorBody != null) { + val answer = runCatching { Json.parseToJsonElement(errorBody).jsonObject }.getOrNull() + val error = answer?.get("error")?.jsonPrimitive?.contentOrNull + val fetchUrl = TrackedPodcastRtdbLogic.httpsFeedUrl(answer?.get("fetchUrl")?.jsonPrimitive?.contentOrNull) + if (error == "not_proxied" && fetchUrl != null) return ShowPageFeedPort.Transported.ReadDirect(fetchUrl) + } + return ShowPageFeedPort.Transported.Failed(if (code in HTTP_CODES) "http_$code" else OTHER_STATUS) +} + +private const val HTTP_NOT_MODIFIED = 304 +private const val HTTP_NOT_FOUND = 404 +private val HTTP_CODES = 100..599 +private const val OTHER_STATUS = "other" diff --git a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryShowPages.kt b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryShowPages.kt new file mode 100644 index 000000000..b5c09163a --- /dev/null +++ b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryShowPages.kt @@ -0,0 +1,196 @@ +package cx.aswin.boxlore.core.catalog + +import cx.aswin.boxlore.core.domain.ports.FeedTelemetryPort +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort +import cx.aswin.boxlore.core.model.Episode +import cx.aswin.boxlore.core.network.model.V2EpisodeIds +import cx.aswin.boxlore.core.network.model.V2EpisodesPaging +import cx.aswin.boxlore.core.network.model.V2PodcastResponse +import java.util.Locale +import kotlinx.coroutines.CancellationException +import retrofit2.Response +import retrofit2.awaitResponse + +/** + * What [PodcastRepository]'s feed-first show page needs from the app: the stored copies + * ([ShowPageFeedPort]; null stores nothing), whether the phone is online, the app's one broken-feed + * report limiter (shared with the daily address check, [FeedAddressUpkeep.reportLimiter]), + * `feed_fetch_result` reporting, and whether the listener follows a show ([isFollowed]: followed + * shows are reported by their own refresh, not by the show page). + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page). + */ +data class FeedFirstWiring( + val copies: ShowPageFeedPort? = null, + val isOnline: () -> Boolean = { true }, + val reportLimiter: FeedReportLimiter = FeedReportLimiter(MemoryFeedReportStamps()), + val telemetry: FeedTelemetryPort = FeedTelemetryPort.NONE, + val isFollowed: suspend (String) -> Boolean = { false }, +) + +/** In-show search answers at most this many episodes. */ +internal const val MAX_SHOW_SEARCH_RESULTS = 200 + +private const val SHOW_PAGES_TAG = "PodcastRepository" +private const val OLDEST_SORT = "oldest" +private const val NEWEST_SORT = "newest" + +/** + * True when [feedId]'s show page reads its feed first: `feed_first` is on and [feedId] is a Podcast + * Index show ID (not `rss:`, nor a `url:`, `guid:` or `itunes:` lookup that hasn't resolved yet). + * + * LEGACY(own-ids): the staged feed-first rollout. Remove the switch at sunset step S7 (MIGRATION.md §6) + * once `feed_first` has been at 100% for 30 days; the show-ID check stays. + */ +internal fun PodcastRepository.isFeedFirstShow(feedId: String): Boolean = V2EpisodeIds.parseShowId(feedId) != null && rolloutSwitches.feedFirst() + +/** + * A page of [feedId]'s stored feed copy, or null when the phone has none yet, or when an oldest-first + * page is asked of a copy that holds only the newest items (its oldest rows aren't the show's first + * episodes). More pages follow while the copy has more rows, and while it is not the whole archive: + * the show page then reads the rest of the feed before it pages on. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page). + */ +internal suspend fun PodcastRepository.feedCopyPage(feedId: String, limit: Int, offset: Int, sort: String,): PodcastRepository.EpisodePage? { + val cached = feedFirst.cached(feedId) ?: return null + if (sort == OLDEST_SORT && !cached.archiveComplete) return null + val catalog = localEpisodeCatalog ?: return null + val episodes = catalog.getPage(feedId, limit, offset, sort, feedFirst.metaFor(feedId) ?: localCatalogMeta(feedId)) + return PodcastRepository.EpisodePage( + episodes = episodes, + hasMore = offset + episodes.size < cached.itemCount || !cached.archiveComplete, + sourceCount = episodes.size, + ) +} + +/** + * Bounded search over [feedId]'s stored feed copy ([MAX_SHOW_SEARCH_RESULTS] rows, limited in the + * query); null when the phone has none. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, in-show search on the phone). + */ +internal suspend fun PodcastRepository.searchFeedCopy(feedId: String, query: String,): List? = + feedFirst.search(feedId, query, feedFirst.metaFor(feedId) ?: localCatalogMeta(feedId), MAX_SHOW_SEARCH_RESULTS) + +/** + * The bounded episode window (Smart Queue, same-show continuation, Android Auto) of a Podcast Index + * show whose catalog isn't ready, under the same IDs its show page lists: + * - around an episode the feed-first copy holds ([aroundEpisodeId]), the copy's window, so playback + * that started on a feed-first page continues under the copy's IDs; + * - else the newest [LOCAL_CATALOG_WINDOW_BOUND] of `/v2/episodes`, mapped to the IDs this phone + * stores, as the show page's server list is; + * - v1 `/episodes` only when `/v2/episodes` fails. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page; decision D1). + */ +internal suspend fun PodcastRepository.getShowEpisodeWindow(feedId: String, aroundEpisodeId: String?, sort: String,): List { + if (aroundEpisodeId != null && isFeedFirstShow(feedId) && feedFirst.cached(feedId) != null) { + val catalog = localEpisodeCatalog + val window = catalog?.getWindow(feedId, sort, LOCAL_CATALOG_WINDOW_BOUND, aroundEpisodeId, feedFirst.metaFor(feedId) ?: localCatalogMeta(feedId)) + if (window != null && window.any { it.id == aroundEpisodeId }) return window + } + return serverListWindow(feedId) ?: getNetworkEpisodeWindow(feedId) +} + +/** + * The newest [LOCAL_CATALOG_WINDOW_BOUND] episodes of the show for callers that list a whole show + * ([PodcastRepository.getEpisodes]): `/v2/episodes` mapped to the IDs this phone stores, then v1. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11; decision D1). + */ +internal suspend fun PodcastRepository.getShowEpisodesNewest(feedId: String): List = serverListWindow(feedId) ?: getAllNetworkEpisodes(feedId) + +/** The newest [LOCAL_CATALOG_WINDOW_BOUND] of `/v2/episodes` under local owners, with cached feed extras; null when `/v2` can't answer. */ +private suspend fun PodcastRepository.serverListWindow(feedId: String): List? { + val resolvedId = resolvePodcastIndexFeedId(feedId) + PodcastRepository.rememberResolvedFeedId(feedId, resolvedId) + val page = v2EpisodesPage(resolvedId, LOCAL_CATALOG_WINDOW_BOUND, 0, NEWEST_SORT) ?: return null + return mergeCachedSupplementsNewest(resolvedId, page.episodes).take(LOCAL_CATALOG_WINDOW_BOUND) +} + +/** + * A page of the server's list of the show (`/v2/episodes`), its recipe IDs mapped to the stored IDs + * that own them on this phone, merged with cached feed extras; v1 `/episodes` only when `/v2` fails. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, rollback: `feed_first` off reads + * `/v2/episodes`; decision D1). + * COMPAT(own-ids): the server's list in recipe IDs. Permanent: never remove (MIGRATION.md §5). + */ +internal suspend fun PodcastRepository.getServerListPage( + feedId: String, + limit: Int, + offset: Int, + sort: String, + mergeSupplements: Boolean, +): PodcastRepository.EpisodePage { + val resolvedId = resolvePodcastIndexFeedId(feedId) + PodcastRepository.rememberResolvedFeedId(feedId, resolvedId) + val page = v2EpisodesPage(resolvedId, limit, offset, sort) ?: return getNetworkEpisodesPaginated(feedId, limit, offset, sort, mergeSupplements) + return maybeMergeSupplements(resolvedId, page, offset, sort, mergeSupplements) +} + +/** One `/v2/episodes` page under local owners, cached with the v1 pages; null when the route can't answer. */ +private suspend fun PodcastRepository.v2EpisodesPage(resolvedId: String, limit: Int, offset: Int, sort: String,): PodcastRepository.EpisodePage? { + val showId = V2EpisodeIds.parseShowId(resolvedId)?.takeIf { V2EpisodesPaging.isValid(it, limit, offset) } ?: return null + val cacheKey = "$resolvedId|v2|$limit|$offset|$sort" + PodcastRepository.episodesCache[cacheKey]?.let { return it } + val body = v2Answer("episodes") { apiV2.getEpisodes(showId, limit, offset, sort).awaitResponse() } ?: return null + return PodcastRepository + .EpisodePage( + episodes = discoveryEpisodeIds.toLocalOwners(body.items.mapNotNull(::mapToEpisode)), + hasMore = body.hasMore, + sourceCount = body.items.size, + ).also { PodcastRepository.episodesCache[cacheKey] = it } +} + +/** + * In-show search over the server's list of the show, on the phone: the newest + * [V2EpisodesPaging.LIMIT_MAX] episodes of `/v2/episodes`, matched by title or description, under + * local owners, with matching cached feed extras. Empty when `/v2` fails: never v1 `episodes/search`. + * + * Added in P4 PR 16 of the own-episode-IDs plan (decision D16: search with `feed_first` off). + * COMPAT(own-ids): no served IDs mix with recipe IDs. Permanent: never remove (MIGRATION.md §5). + */ +internal suspend fun PodcastRepository.searchServerList(feedId: String, query: String,): List { + val trimmed = query.trim() + if (trimmed.isEmpty()) return emptyList() + val resolvedId = resolvePodcastIndexFeedId(feedId) + val list = serverSearchList(resolvedId) + val matches = list.filter { it.matches(trimmed) }.take(MAX_SHOW_SEARCH_RESULTS) + return unionCachedSupplementSearch(resolvedId, trimmed, matches) +} + +private suspend fun PodcastRepository.serverSearchList(resolvedId: String): List { + val showId = V2EpisodeIds.parseShowId(resolvedId)?.takeIf(V2EpisodeIds::isEpisodeShowId) ?: return emptyList() + val cacheKey = "$resolvedId|v2search" + PodcastRepository.episodesCache[cacheKey]?.let { return it.episodes } + val body = + v2Answer("episodes search") { + apiV2.getEpisodes(showId, V2EpisodesPaging.LIMIT_MAX, 0, cx.aswin.boxlore.core.network.model.V2EpisodeSort.NEWEST).awaitResponse() + } ?: return emptyList() + val episodes = discoveryEpisodeIds.toLocalOwners(body.items.mapNotNull(::mapToEpisode)) + PodcastRepository.episodesCache[cacheKey] = PodcastRepository.EpisodePage(episodes, body.hasMore, body.items.size) + return episodes +} + +private fun Episode.matches(query: String): Boolean { + val needle = query.lowercase(Locale.ROOT) + return title.lowercase(Locale.ROOT).contains(needle) || description.lowercase(Locale.ROOT).contains(needle) +} + +/** The show's registry entry (`/v2/podcast`): its address, transport and namespace. Null when it can't be read. */ +internal suspend fun PodcastRepository.v2ShowAddress(showId: Long): V2PodcastResponse? = v2Answer("podcast") { apiV2.getPodcast(showId).awaitResponse() } + +/** The body of a successful `/v2` answer, or null; cancellation propagates. */ +private inline fun v2Answer(route: String, call: () -> Response): T? = try { + val response = call() + response.body().takeIf { response.isSuccessful }.also { + if (it == null) android.util.Log.w(SHOW_PAGES_TAG, "v2 $route failed: HTTP ${response.code()}") + } +} catch (error: CancellationException) { + throw error +} catch (error: Exception) { + android.util.Log.w(SHOW_PAGES_TAG, "v2 $route failed", error) + null +} diff --git a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/RolloutSwitches.kt b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/RolloutSwitches.kt index 75e31142e..0a7aa2efa 100644 --- a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/RolloutSwitches.kt +++ b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/RolloutSwitches.kt @@ -5,8 +5,9 @@ package cx.aswin.boxlore.core.catalog * module never reads PostHog itself (`ARCHITECTURE.md`: catalog does not depend on analytics). * * Added in P4 (own episode IDs, plan section 11 "P4: App v2 build", Rollback). [PodcastRepository] - * reads [v2Discovery] for every discovery request (P4 PR 12); the show-page source, the daily - * address check and the one-time upgrade repair each read their switch when they land. The + * reads [v2Discovery] for every discovery request (P4 PR 12) and [feedFirst] for the show-page + * source (P4 PR 16, [PodcastRepository.showPageSource]); the daily address check and the one-time + * upgrade repair read theirs. The * defaults are what a build gets without the app's wiring (tests, previews): the feed-first show * page off, every job on. */ diff --git a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/SubscriptionForegroundSync.kt b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/SubscriptionForegroundSync.kt index 792d54727..022bab62b 100644 --- a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/SubscriptionForegroundSync.kt +++ b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/SubscriptionForegroundSync.kt @@ -306,7 +306,7 @@ class SubscriptionForegroundSync( reporter = FeedReporter( send = { request -> FeedReporter.sendWith(podcastRepository.apiV2, request) }, - limiter = FeedReportLimiter(upkeep.reportStamps), + limiter = upkeep.reportLimiter, isOnline = upkeep.isOnline, enabled = enabled, ), @@ -931,10 +931,13 @@ internal data class DirectFeedSyncSeams( /** * What the daily address check and broken-feed reports need from the app (P4 PR 14): where report * times are kept across launches, and whether the phone is online (reports are never sent offline). + * [reportLimiter] is the app's one limiter, shared with the show page (P4 PR 16) so both stay inside + * one daily limit; the default builds one on [reportStamps]. */ data class FeedAddressUpkeep( val reportStamps: FeedReportStamps = MemoryFeedReportStamps(), val isOnline: () -> Boolean = { true }, + val reportLimiter: FeedReportLimiter = FeedReportLimiter(reportStamps), ) /** The daily address check and the broken-feed reporter of one [SubscriptionForegroundSync] (P4 PR 14). */ diff --git a/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/FeedFirstShowPagesTest.kt b/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/FeedFirstShowPagesTest.kt new file mode 100644 index 000000000..57c084729 --- /dev/null +++ b/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/FeedFirstShowPagesTest.kt @@ -0,0 +1,284 @@ +package cx.aswin.boxlore.core.catalog + +import cx.aswin.boxlore.core.domain.ports.FeedTelemetryPort +import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort.PodcastMeta +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.ReadOutcome +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.Scope +import cx.aswin.boxlore.core.model.Episode +import cx.aswin.boxlore.core.model.FeedFetchCompleteness +import cx.aswin.boxlore.core.model.FeedFetchOutcome +import cx.aswin.boxlore.core.model.FeedFetchReport +import cx.aswin.boxlore.core.model.FeedFetchSource +import cx.aswin.boxlore.core.model.FeedUrlCorrection +import cx.aswin.boxlore.core.network.model.V2FeedReportRequest +import cx.aswin.boxlore.core.network.model.V2FeedReportResponse +import cx.aswin.boxlore.core.network.model.V2PodcastResponse +import kotlinx.coroutines.CompletableDeferred +import kotlinx.coroutines.CoroutineScope +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.runBlocking +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.annotation.Config + +/** + * The feed-first show page's failure ladder against a scripted stored copy: offline shows the copy + * without reading or reporting, a failed read is tried once more (never after a 404 or 410), then + * reported, then the reported address is read, and only then the page falls back to the server's + * list. One `feed_fetch_result` per read, without addresses. + */ +@RunWith(RobolectricTestRunner::class) +@Config(sdk = [34]) +class FeedFirstShowPagesTest { + private var online = true + private var registry: V2PodcastResponse? = V2PodcastResponse(fetchUrl = PUBLISHER, fetchVia = "direct", epNs = SHOW.toLong()) + private val reports = mutableListOf() + private var reportAnswer: V2FeedReportResponse? = null + private val telemetry = RecordingTelemetry() + private val copies = ScriptedCopies() + private var now = 1_000_000L + + private val pages by lazy { + FeedFirstShowPages( + copies = copies, + registry = { registry }, + transport = { _, _ -> ShowPageFeedPort.Transported.Document("".encodeToByteArray(), null) }, + reporter = + FeedReporter( + send = { request -> + reports += request + reportAnswer + }, + limiter = FeedReportLimiter(MemoryFeedReportStamps()), + isOnline = { online }, + enabled = { true }, + ), + isOnline = { online }, + telemetry = telemetry, + nowMillis = { now }, + background = CoroutineScope(Dispatchers.Unconfined), + isFollowed = { it in followed }, + ) + } + private val followed = mutableSetOf() + + @Test + fun offlineShowsTheStoredCopyWithoutReadingOrReporting() = runBlocking { + online = false + + assertEquals(ShowPageFeedResult.OFFLINE, pages.read(SHOW, META, Scope.NEWEST, SAVED).result) + assertTrue(copies.reads.isEmpty()) + assertTrue(reports.isEmpty()) + assertFalse(pages.isInFallback(SHOW)) + assertTrue("no read, no event", telemetry.reports.isEmpty()) + } + + @Test + fun theRegistryAddressIsReadAndOneEventSaysHowMuch() = runBlocking { + copies.answers[PUBLISHER] = mutableListOf(ReadOutcome.Stored(100, complete = false)) + + val read = pages.read(SHOW, META, Scope.NEWEST, SAVED) + + assertEquals(ShowPageFeedResult.UPDATED, read.result) + assertEquals(listOf(PUBLISHER to Scope.NEWEST), copies.reads) + assertEquals("the registry's namespace is passed on to be compared", SHOW.toLong(), copies.lastRequest?.serverEpNs) + assertEquals( + listOf(FeedFetchReport.ShowPage(SHOW, FeedFetchSource.FEED, FeedFetchOutcome.OK, FeedFetchCompleteness.PARTIAL)), + telemetry.reports, + ) + } + + @Test + fun aFailedReadIsRetriedOnceThenReportedThenTheReportedAddressIsRead() = runBlocking { + copies.answers[PUBLISHER] = mutableListOf(ReadOutcome.Failed("timeout"), ReadOutcome.Failed("timeout")) + copies.answers[REPORTED] = mutableListOf(ReadOutcome.Stored(40, complete = true)) + reportAnswer = V2FeedReportResponse(queued = true, fetchUrl = REPORTED, fetchVia = "direct") + + val read = pages.read(SHOW, META, Scope.NEWEST, SAVED) + + assertEquals(ShowPageFeedResult.UPDATED, read.result) + assertEquals(listOf(PUBLISHER, PUBLISHER, REPORTED), copies.reads.map { it.first }) + assertEquals(listOf(V2FeedReportRequest(SHOW.toLong(), "timeout")), reports) + assertEquals(1, telemetry.reports.size) + } + + @Test + fun aGoneAddressIsNotRetriedAndWithoutAnotherAddressThePageFallsBack() = runBlocking { + copies.answers[PUBLISHER] = mutableListOf(ReadOutcome.Failed("http_404")) + reportAnswer = V2FeedReportResponse(queued = true, fetchUrl = PUBLISHER, fetchVia = "direct") + + val read = pages.read(SHOW, META, Scope.NEWEST, SAVED) + + assertEquals(ShowPageFeedResult.FALLBACK, read.result) + assertEquals(listOf(PUBLISHER), copies.reads.map { it.first }) + assertEquals(1, reports.size) + assertTrue(pages.isInFallback(SHOW)) + assertEquals( + listOf(FeedFetchReport.ShowPage(SHOW, FeedFetchSource.FEED, FeedFetchOutcome.GONE, httpStatus = 404)), + telemetry.reports, + ) + + // Opening the page again tries the feed again; a read that works keeps it there. + pages.open(SHOW, META) + assertFalse(pages.isInFallback(SHOW)) + copies.answers[PUBLISHER] = mutableListOf(ReadOutcome.NotModified) + assertEquals(ShowPageFeedResult.UNCHANGED, pages.read(SHOW, META, Scope.NEWEST, SAVED).result) + assertFalse(pages.isInFallback(SHOW)) + } + + @Test + fun withoutTheRegistryTheSavedAddressIsReadAndAnHttpOnlyShowGoesThroughTheTransport() = runBlocking { + registry = null + copies.answers[SAVED] = mutableListOf(ReadOutcome.Stored(10, complete = true)) + assertEquals(ShowPageFeedResult.UPDATED, pages.read(SHOW, META, Scope.ARCHIVE, SAVED).result) + assertEquals(listOf(SAVED to Scope.ARCHIVE), copies.reads) + + registry = V2PodcastResponse(fetchUrl = "http://plain.example/feed", fetchVia = "proxy", transportUrl = "https://api.example/v2/feed/$SHOW") + assertEquals(ShowPageFeedResult.UPDATED, pages.read(SHOW, META, Scope.NEWEST, SAVED).result) + assertEquals(1, copies.stores) + assertEquals(FeedFetchSource.TRANSPORT, (telemetry.reports.last() as FeedFetchReport.ShowPage).source) + + registry = null + assertEquals("no address at all", ShowPageFeedResult.FALLBACK, pages.read(SHOW, META, Scope.NEWEST, null).result) + assertEquals("rss: shows never read through here", ShowPageFeedResult.FALLBACK, pages.read("rss:abc", META, Scope.NEWEST, SAVED).result) + } + + @Test + fun openingSaysWhatTheCopyHoldsAndWhenItIsDueAgain() = runBlocking { + assertEquals(ShowPageFeedState(cached = false, archiveComplete = false, recheckDue = true, oldestFirst = false), pages.open(SHOW, META)) + + copies.cachedShow = ShowPageFeedPort.CachedShow(SHOW, PUBLISHER, fetchedAt = now - 10 * 60_000L, itemCount = 100, archiveComplete = false, oldestFirst = false) + assertEquals(ShowPageFeedState(cached = true, archiveComplete = false, recheckDue = false, oldestFirst = false), pages.open(SHOW, META)) + now += 60 * 60_000L + assertTrue("an hour old: read again", pages.open(SHOW, META).recheckDue) + assertEquals(META, pages.metaFor(SHOW)) + assertEquals("the daily cleanup ran once, keeping the show being opened", listOf(SHOW), copies.cleanups) + assertEquals(2, copies.opened.size) + } + + @Test + fun aReportNamingAnHttpOnlyAddressIsReadThroughTheTransport() = runBlocking { + copies.answers[PUBLISHER] = mutableListOf(ReadOutcome.Failed("http_404")) + reportAnswer = V2FeedReportResponse(queued = true, fetchUrl = "http://plain.example/feed", fetchVia = "proxy", transportUrl = "https://api.example/v2/feed/$SHOW") + + val read = pages.read(SHOW, META, Scope.NEWEST, SAVED) + + assertEquals(ShowPageFeedResult.UPDATED, read.result) + assertEquals(1, copies.stores) + assertEquals(FeedFetchSource.TRANSPORT, (telemetry.reports.single() as FeedFetchReport.ShowPage).source) + } + + @Test + fun aFollowedShowIsLeftToItsOwnRefreshToReport() = runBlocking { + followed += SHOW + copies.answers[PUBLISHER] = mutableListOf(ReadOutcome.Failed("http_404")) + reportAnswer = V2FeedReportResponse(queued = true, fetchUrl = REPORTED, fetchVia = "direct") + + assertEquals(ShowPageFeedResult.FALLBACK, pages.read(SHOW, META, Scope.NEWEST, SAVED).result) + assertTrue("the followed show's one report a day stays for its refresh", reports.isEmpty()) + } + + @Test + fun theShowPageLeavesPartOfTheDailyReportsToFollowedShows() = runBlocking { + val limiter = FeedReportLimiter(MemoryFeedReportStamps()) + val reporter = FeedReporter({ reportAnswer }, limiter, { true }, { true }, { now }, keepFree = FeedReportLimiter.FOLLOWED_SHOWS_RESERVE) + val showPageCap = FeedReportLimiter.DAILY_CAP - FeedReportLimiter.FOLLOWED_SHOWS_RESERVE + repeat(showPageCap) { limiter.record("${100 + it}", now) } + + assertEquals(null, reporter.reportTarget("920700", "timeout", SAVED)) + assertTrue("followed shows still have theirs", limiter.allows("920701", now)) + } + + @Test + fun theOpeningShowsCopyNeverWaitsForTheDailyCleanup() = runBlocking { + val held = CompletableDeferred() + copies.cleanupGate = held + + val opened = pages.open(SHOW, META) + + assertTrue("the page opened while the cleanup still runs", opened.recheckDue) + assertEquals(listOf(SHOW), copies.cleanups) + held.complete(Unit) + Unit + } + + @Test + fun readFailuresMapToTheEventOutcomes() { + assertEquals(FeedFetchOutcome.GONE, FeedFirstShowPages.failureOutcome("http_410")) + assertEquals(FeedFetchOutcome.HTTP_ERROR, FeedFirstShowPages.failureOutcome("http_503")) + assertEquals(FeedFetchOutcome.TIMEOUT, FeedFirstShowPages.failureOutcome("timeout")) + assertEquals(FeedFetchOutcome.PARSE_ERROR, FeedFirstShowPages.failureOutcome("parse")) + assertEquals(FeedFetchOutcome.NETWORK_ERROR, FeedFirstShowPages.failureOutcome("dns")) + } + + private class ScriptedCopies : ShowPageFeedPort { + val answers = mutableMapOf>() + val reads = mutableListOf>() + var lastRequest: ShowPageFeedPort.ReadRequest? = null + var stores = 0 + val transportRequests = mutableListOf() + var transportAnswer: ShowPageFeedPort.Transported? = null + var cachedShow: ShowPageFeedPort.CachedShow? = null + val opened = mutableListOf() + val cleanups = mutableListOf() + var cleanupGate: CompletableDeferred? = null + + override suspend fun cached(podcastId: String): ShowPageFeedPort.CachedShow? = cachedShow + + override suspend fun read(request: ShowPageFeedPort.ReadRequest): ReadOutcome { + reads += request.feedUrl to request.scope + lastRequest = request + val queue = answers[request.feedUrl] ?: return ReadOutcome.Failed("other") + return if (queue.size > 1) queue.removeAt(0) else queue.first() + } + + override suspend fun readTransport( + request: ShowPageFeedPort.ReadRequest, + fetch: suspend (ifNoneMatch: String?) -> ShowPageFeedPort.Transported, + ): ReadOutcome { + stores++ + transportRequests += request + return when (val answer = transportAnswer ?: fetch(null)) { + is ShowPageFeedPort.Transported.Document -> ReadOutcome.Stored(5, complete = true) + ShowPageFeedPort.Transported.NotModified -> ReadOutcome.NotModified + is ShowPageFeedPort.Transported.ReadDirect -> read(request.copy(feedUrl = answer.url)) + is ShowPageFeedPort.Transported.Failed -> ReadOutcome.Failed(answer.reportStatus) + } + } + + override suspend fun search(podcastId: String, query: String, limit: Int, meta: PodcastMeta): List = emptyList() + + override suspend fun markOpened(podcastId: String, atMillis: Long) { + opened += podcastId + } + + override suspend fun cleanUp(nowMillis: Long, keepPodcastId: String?): Int { + cleanups += keepPodcastId + cleanupGate?.await() + return 0 + } + } + + private class RecordingTelemetry : FeedTelemetryPort { + val reports = mutableListOf() + + override fun feedFetched(report: FeedFetchReport) { + reports += report + } + + override fun feedUrlCorrected(correction: FeedUrlCorrection) = Unit + } + + private companion object { + const val SHOW = "920666" + const val PUBLISHER = "https://feeds.example/920666.xml" + const val SAVED = "https://saved.example/920666.xml" + const val REPORTED = "https://moved.example/920666.xml" + val META = PodcastMeta(title = "Show", imageUrl = "https://img.example/a.jpg") + } +} diff --git a/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/FeedReportsTest.kt b/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/FeedReportsTest.kt index 858c47a77..ecf1d0351 100644 --- a/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/FeedReportsTest.kt +++ b/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/FeedReportsTest.kt @@ -1,5 +1,6 @@ package cx.aswin.boxlore.core.catalog +import cx.aswin.boxlore.core.network.model.V2FeedReadTarget import cx.aswin.boxlore.core.network.model.V2FeedReportRequest import cx.aswin.boxlore.core.network.model.V2FeedReportResponse import kotlinx.coroutines.test.runTest @@ -44,10 +45,10 @@ class FeedReportsTest { } @Test - fun `at most fifteen reports a day in all, leaving the show page room under the API's twenty`() = runTest { + fun `at most twenty reports a day in all, the API's own limit`() = runTest { repeat(25) { reporter.report((it + 1).toString(), "dns", OLD) } - assertEquals(15, FeedReportLimiter.DAILY_CAP) + assertEquals(20, FeedReportLimiter.DAILY_CAP) assertEquals(FeedReportLimiter.DAILY_CAP, sent.size) now += DAY reporter.report("25", "dns", OLD) @@ -94,6 +95,25 @@ class FeedReportsTest { assertNull(failing.report("4", "http_404", OLD)) } + @Test + fun `the show page reads an http-only answer through the transport`() = runTest { + answer = V2FeedReportResponse(queued = true, fetchUrl = "http://new.example/feed.xml", fetchVia = "proxy", transportUrl = "https://api.example/v2/feed/2") + assertEquals(V2FeedReadTarget.ViaApi(2), reporter.reportTarget("2", "http_404", OLD)) + answer = V2FeedReportResponse(queued = true, fetchUrl = NEW, fetchVia = "direct") + assertEquals(V2FeedReadTarget.Direct(NEW), reporter.reportTarget("3", "http_404", OLD)) + } + + @Test + fun `limiters sharing one store count each other's reports`() { + val first = FeedReportLimiter(stamps) + val second = FeedReportLimiter(stamps) + repeat(FeedReportLimiter.DAILY_CAP - 1) { first.record("${100 + it}", now) } + + assertTrue(second.allows("1", now)) + second.record("1", now) + assertFalse(first.allows("2", now)) + } + @Test fun `old report times are dropped as new ones are written`() { val limiter = FeedReportLimiter(stamps) diff --git a/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryEpisodeSupplementTest.kt b/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryEpisodeSupplementTest.kt index 949e525d6..b31b35a97 100644 --- a/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryEpisodeSupplementTest.kt +++ b/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryEpisodeSupplementTest.kt @@ -71,6 +71,7 @@ class PodcastRepositoryEpisodeSupplementTest { episodeSupplementRepository = fakePort, ioDispatcher = testDispatcher, boxLoreApi = api, + boxLoreApiV2 = NetworkModule.createBoxLoreApiV2(server.url("/").toString(), client), ) } @@ -240,7 +241,7 @@ class PodcastRepositoryEpisodeSupplementTest { } @Test - fun `searchEpisodes unions PI hits with supplement matches`() = runTest(testDispatcher) { + fun `searchEpisodes matches the server's list on the phone and unions supplement matches`() = runTest(testDispatcher) { val podcastId = "900003" fakePort.searchByPodcast[podcastId] = listOf( @@ -275,6 +276,8 @@ class PodcastRepositoryEpisodeSupplementTest { val results = repository.searchEpisodes(podcastId, "match") assertEquals(listOf("-9", "44"), results.map { it.id }) + // Search runs on the phone over `/v2/episodes`; v1 `episodes/search` is never called (P4 PR 16). + assertEquals("/v2/episodes?id=900003&limit=1000&offset=0&sort=newest", server.takeRequest().path) } @Test diff --git a/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryShowPagesTest.kt b/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryShowPagesTest.kt new file mode 100644 index 000000000..3adeac7a1 --- /dev/null +++ b/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/PodcastRepositoryShowPagesTest.kt @@ -0,0 +1,385 @@ +package cx.aswin.boxlore.core.catalog + +import android.content.Context +import androidx.room.Room +import androidx.test.core.app.ApplicationProvider +import cx.aswin.boxlore.core.database.BoxLoreDatabase +import cx.aswin.boxlore.core.database.LocalEpisodeEntity +import cx.aswin.boxlore.core.database.PodcastEntity +import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort.PodcastMeta +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.Scope +import cx.aswin.boxlore.core.model.EpisodeRecipeId +import cx.aswin.boxlore.core.network.NetworkModule +import cx.aswin.boxlore.core.rss.LocalEpisodeCatalogRepository +import cx.aswin.boxlore.core.rss.RssFeedClient +import cx.aswin.boxlore.core.rss.RssPodcastRepository +import java.time.Instant +import java.time.ZoneOffset +import java.time.format.DateTimeFormatter +import java.util.concurrent.CopyOnWriteArrayList +import java.util.concurrent.TimeUnit +import kotlinx.coroutines.runBlocking +import okhttp3.MediaType.Companion.toMediaType +import okhttp3.OkHttpClient +import okhttp3.Protocol +import okhttp3.Response +import okhttp3.ResponseBody.Companion.toResponseBody +import okhttp3.mockwebserver.Dispatcher +import okhttp3.mockwebserver.MockResponse +import okhttp3.mockwebserver.MockWebServer +import okhttp3.mockwebserver.RecordedRequest +import org.junit.After +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Before +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.annotation.Config + +/** + * The show page's episode source through [PodcastRepository] on a real Room database: the + * feed-first stored copy (`feed_first` on), the server's list in recipe IDs mapped to the phone's + * owners (`feed_first` off, and after a fallback), v1 `/episodes` only when `/v2/episodes` fails, + * search on the phone that never calls v1 `episodes/search`, paging that stays stable while the copy + * grows, and a rollback that changes no stored ID or alias. + */ +@RunWith(RobolectricTestRunner::class) +@Config(sdk = [34]) +class PodcastRepositoryShowPagesTest { + private lateinit var server: MockWebServer + private lateinit var context: Context + private lateinit var database: BoxLoreDatabase + private val seen = CopyOnWriteArrayList() + private val ifNoneMatch = CopyOnWriteArrayList() + private val answers = mutableMapOf MockResponse>() + private var feedFirst = true + private var feedItems = 150 + + @Before + fun setUp() { + context = ApplicationProvider.getApplicationContext() + database = Room.inMemoryDatabaseBuilder(context, BoxLoreDatabase::class.java).allowMainThreadQueries().build() + PodcastRepository.episodesCache.clear() + server = MockWebServer() + server.dispatcher = + object : Dispatcher() { + override fun dispatch(request: RecordedRequest): MockResponse { + seen += request.path.orEmpty() + request.getHeader("If-None-Match")?.let { ifNoneMatch += it } + return answers[request.path.orEmpty().substringBefore('?')]?.invoke() ?: MockResponse().setResponseCode(404) + } + } + server.start() + answers["/v2/podcast"] = { json("""{"status":"true","fetchUrl":"$FEED_URL","fetchVia":"direct","epNs":$SHOW}""") } + answers["/v2/episodes"] = { json(v2Episodes(listOf("g0", "g1", "Moon g2"))) } + } + + @After + fun tearDown() { + server.shutdown() + database.close() + RssPodcastRepository.clearInstanceForTests() + PodcastRepository.episodesCache.clear() + } + + @Test + fun withTheFlagOffThePageReadsTheServersListUnderLocalOwnersAndV1OnlyWhenItFails() = runBlocking { + feedFirst = false + val repository = repository(FakeIds(mapOf(recipe("g1") to "5001"))) + + assertEquals(ShowPageSource.SERVER_LIST, repository.showPageSource(SHOW)) + val page = repository.getEpisodesPaginated(SHOW, limit = 20, offset = 0, sort = "newest") + + assertEquals(listOf(recipe("g0").toString(), "5001", recipe("Moon g2").toString()), page.episodes.map { it.id }) + assertEquals("the recipe ID stays with its owner", recipe("g1"), page.episodes[1].recipeId) + assertEquals("/v2/episodes?id=$SHOW&limit=20&offset=0&sort=newest", seen.single()) + + answers["/v2/episodes"] = { MockResponse().setResponseCode(503) } + answers["/episodes"] = { json("""{"items":[{"id":777,"title":"v1","enclosureUrl":"https://cdn.example/v1.mp3","feedId":$SHOW}],"hasMore":false}""") } + repository.invalidateEpisodesCache(SHOW) + assertEquals(listOf("777"), repository.getEpisodesPaginated(SHOW, 20, 0, "newest").episodes.map { it.id }) + assertTrue(seen.last().startsWith("/episodes?")) + } + + @Test + fun withTheFlagOnThePageShowsTheStoredCopyAndPagesStayStableWhileItGrows() = runBlocking { + val repository = repository() + assertEquals(ShowPageSource.FEED, repository.showPageSource(SHOW)) + assertFalse(requireNotNull(repository.openFeedShowPage(SHOW, META)).cached) + + val newest = repository.readShowPageFeed(SHOW, META, Scope.NEWEST, savedFeedUrl = null) + assertEquals(ShowPageFeedRead(ShowPageFeedResult.UPDATED, archiveComplete = false, hasCopy = true), newest) + + val shown = mutableListOf() + var offset = 0 + repeat(5) { + val page = repository.getShowPageEpisodes(SHOW, 20, offset, "newest") + assertTrue(page.hasMore) + shown += page.episodes.map { it.title } + offset += page.sourceCount + } + val pastCopy = repository.getShowPageEpisodes(SHOW, 20, offset, "newest") + assertTrue("the copy ran out, but the archive isn't read yet", pastCopy.episodes.isEmpty() && pastCopy.hasMore) + + assertEquals(ShowPageFeedResult.UPDATED, repository.readShowPageFeed(SHOW, META, Scope.ARCHIVE, null).result) + while (true) { + val page = repository.getShowPageEpisodes(SHOW, 20, offset, "newest") + shown += page.episodes.map { it.title } + offset += page.sourceCount + if (!page.hasMore) break + } + assertEquals("every episode once, in order, across the growth", (0 until 150).map { "Episode $it" }, shown) + assertTrue("no network page was read", seen.none { it.startsWith("/v2/episodes") || it.startsWith("/episodes") }) + } + + @Test + fun searchRunsOnThePhoneAndNeverCallsV1EpisodeSearch() = runBlocking { + feedFirst = false + val offRepository = repository() + assertEquals(listOf("Moon g2"), offRepository.searchEpisodes(SHOW, "moon").map { it.title }) + assertTrue(seen.single().startsWith("/v2/episodes?id=$SHOW&limit=1000")) + + feedFirst = true + feedItems = 30 + val onRepository = repository() + onRepository.readShowPageFeed(SHOW, META, Scope.ARCHIVE, null) + seen.clear() + assertEquals(listOf("Episode 12"), onRepository.searchEpisodes(SHOW, "Episode 12").map { it.title }) + assertTrue(seen.isEmpty()) + assertTrue(seen.none { it.startsWith("/episodes/search") }) + } + + @Test + fun aFeedThatCantBeReadSwitchesThePageToTheServersList() = runBlocking { + val repository = repository() + answers["/v2/podcast"] = { json("""{"status":"true","fetchUrl":"https://feeds.example/gone.xml","fetchVia":"direct"}""") } + + val read = repository.readShowPageFeed(SHOW, META, Scope.NEWEST, null) + + assertEquals(ShowPageFeedResult.FALLBACK, read.result) + assertEquals(ShowPageSource.SERVER_LIST, repository.showPageSource(SHOW)) + assertTrue("still a feed-first page: no Missing episodes? chip", repository.isFeedFirstShowPage(SHOW)) + assertEquals(3, repository.getShowPageEpisodes(SHOW, 20, 0, "newest").episodes.size) + assertTrue(seen.any { it.startsWith("/v2/feed-report") }) + + // Opening the page again tries the feed again. + requireNotNull(repository.openFeedShowPage(SHOW, META)) + assertEquals(ShowPageSource.FEED, repository.showPageSource(SHOW)) + feedFirst = false + assertEquals("with the flag off the page never reads the feed", null, repository.openFeedShowPage(SHOW, META)) + } + + @Test + fun rollingTheFlagBackChangesNoStoredIdAndTheServersListMapsToTheSameOwners() = runBlocking { + // An episode an older build stored under its Podcast Index ID while the show was followed. + database.localEpisodeCatalogDao().upsertEpisodes(listOf(row("5001", "g1"))) + // The show's saved row, which notification registration and the tracked-shows sync read. + database.podcastDao().upsert(PodcastEntity(podcastId = SHOW, title = "Show", author = "", imageUrl = "", description = null, isSubscribed = false, feedUrl = SAVED_URL)) + val savedRow = database.podcastDao().getPodcast(SHOW) + val repository = repository(RoomDiscoveryEpisodeIds { database }) + repository.readShowPageFeed(SHOW, META, Scope.ARCHIVE, null) + val storedIds = database.localEpisodeCatalogDao().listIdentities(SHOW).associate { it.guid to it.episodeId } + val aliases = database.episodeIdentityDao().aliasesForShow(SHOW).sortedBy { it.aliasId } + assertEquals("5001", storedIds["g1"]) + assertEquals(recipe("g0").toString(), storedIds["g0"]) + + feedFirst = false + repository.invalidateEpisodesCache(SHOW) + val serverList = repository.getEpisodesPaginated(SHOW, 20, 0, "newest").episodes.map { it.id } + + assertEquals(listOf(storedIds["g0"], "5001", storedIds["Moon g2"] ?: recipe("Moon g2").toString()), serverList) + assertEquals(storedIds, database.localEpisodeCatalogDao().listIdentities(SHOW).associate { it.guid to it.episodeId }) + assertEquals(aliases, database.episodeIdentityDao().aliasesForShow(SHOW).sortedBy { it.aliasId }) + val after = requireNotNull(database.podcastDao().getPodcast(SHOW)) + // The namespace the read kept (decision D12) is the one column it may fill. + assertEquals("no address, transport or sync state of the show changed", savedRow?.copy(epNs = after.epNs), after) + assertTrue("no registration or address route was called", seen.none { it.startsWith("/v2/episodes/register") || it.startsWith("/v2/shows/feeds") }) + } + + @Test + fun onlyTheShowPageReadsTheCopyAndAFollowedHttpOnlyShowKeepsReadingTheServersList() = runBlocking { + answers["/v2/podcast"] = { fixture("fixtures/v2/podcast-proxy.json") } + answers["/v2/feed/$PROXIED"] = { MockResponse().setHeader("ETag", "\"t1\"").setBody(feedXml(30)) } + val repository = repository() + // Followed, but its catalog can never be ready: the address is http-only. + database.podcastDao().upsert(PodcastEntity(podcastId = PROXIED, title = "Proxied", author = "", imageUrl = "", description = null, isSubscribed = true)) + + assertEquals(ShowPageFeedResult.UPDATED, repository.readShowPageFeed(PROXIED, META, Scope.NEWEST, null).result) + assertEquals(ShowPageSource.FEED, repository.showPageSource(PROXIED)) + seen.clear() + + assertEquals(20, repository.getShowPageEpisodes(PROXIED, 20, 0, "newest").episodes.size) + assertTrue("the show page reads the copy", seen.isEmpty()) + repository.getEpisodesPaginated(PROXIED, 20, 0, "newest") + assertEquals("Smart Queue, Auto, Home and downloads read the server's list", "/v2/episodes?id=$PROXIED&limit=20&offset=0&sort=newest", seen.single()) + + // The next read sends the copy's ETag; the 304 keeps it. + answers["/v2/feed/$PROXIED"] = { MockResponse().setResponseCode(304) } + assertEquals(ShowPageFeedResult.UNCHANGED, repository.readShowPageFeed(PROXIED, META, Scope.ARCHIVE, null).result) + assertEquals(listOf("\"t1\""), ifNoneMatch.toList()) + } + + @Test + fun anOldestFirstPageOfAPartialCopyComesFromTheServersList() = runBlocking { + val repository = repository() + repository.readShowPageFeed(SHOW, META, Scope.NEWEST, null) + seen.clear() + + repository.getShowPageEpisodes(SHOW, 50, 0, "oldest") + + assertEquals("/v2/episodes?id=$SHOW&limit=50&offset=0&sort=oldest", seen.single()) + } + + @Test + fun playbackWindowsListTheEpisodesUnderTheIdsTheShowPageShowed() = runBlocking { + val repository = repository() + repository.readShowPageFeed(SHOW, META, Scope.ARCHIVE, null) + val playing = recipe("g1").toString() + val next = recipe("g0").toString() + + val fromCopy = repository.getEpisodeWindow(SHOW, aroundEpisodeId = playing).map { it.id } + assertTrue(playing in fromCopy && next in fromCopy) + assertTrue("no v1 window", seen.none { it.startsWith("/episodes") }) + + feedFirst = false + val fromServerList = repository.getEpisodeWindow(SHOW, aroundEpisodeId = playing).map { it.id } + assertTrue(playing in fromServerList && next in fromServerList) + assertEquals(listOf(next, playing, recipe("Moon g2").toString()), repository.getEpisodes(SHOW).map { it.id }) + assertTrue("v1 /episodes only when /v2/episodes fails", seen.none { it.startsWith("/episodes") }) + } + + @Test + fun recordedAnswersOfTheV2RoutesReadAsTheRoutesSendThem() = runBlocking { + feedFirst = false + answers["/v2/episodes"] = { fixture("fixtures/v2/episodes.json") } + val page = repository().getEpisodesPaginated(SHOW, 20, 0, "newest") + + assertEquals(listOf("5626175219222083", "7579913942623881"), page.episodes.map { it.id }) + assertEquals(5626175219222083L, page.episodes.first().recipeId) + + feedFirst = true + answers["/v2/podcast"] = { fixture("fixtures/v2/podcast-direct.json") } + assertEquals(ShowPageFeedResult.UPDATED, repository().readShowPageFeed(SHOW, META, Scope.NEWEST, null).result) + + assertEquals( + ShowPageFeedPort.Transported.ReadDirect(RECORDED_FEED_URL), + transportFailure(404, fixtureText("fixtures/v2/feed-not-proxied.json")), + ) + assertEquals(ShowPageFeedPort.Transported.Failed("http_503"), transportFailure(503, "{\"error\":\"feed_unavailable\"}")) + } + + @Test + fun droppingAResolvedShowsPagesAlsoDropsThePagesStoredUnderItsShowId() = runBlocking { + feedFirst = false + answers["/podcast"] = { json("""{"status":"true","feed":{"id":$SHOW,"title":"Show","url":"$FEED_URL"}}""") } + val repository = repository() + val byUrl = "url:" + java.net.URLEncoder.encode(FEED_URL, "UTF-8") + + repository.getEpisodesPaginated(byUrl, 20, 0, "newest") + repository.getEpisodesPaginated(byUrl, 20, 0, "newest") + assertEquals("the second page came from the cache", 1, seen.count { it.startsWith("/v2/episodes") }) + repository.invalidateEpisodesCache(byUrl) + repository.getEpisodesPaginated(byUrl, 20, 0, "newest") + assertEquals(2, seen.count { it.startsWith("/v2/episodes") }) + } + + private fun repository(ids: DiscoveryEpisodeIds = DiscoveryEpisodeIds.Unmapped): PodcastRepository { + val client = OkHttpClient.Builder().readTimeout(5, TimeUnit.SECONDS).build() + val base = server.url("/").toString() + val catalog = LocalEpisodeCatalogRepository.create(database, feedClient()) + return PodcastRepository( + baseUrl = base, + publicKey = "test-app-key", + context = context, + rssRepository = RssPodcastRepository.createForTests(context, database), + localEpisodeCatalog = catalog, + boxLoreApi = NetworkModule.createBoxLoreApi(base, client), + boxLoreApiV2 = NetworkModule.createBoxLoreApiV2(base, client), + rolloutSwitches = RolloutSwitches(feedFirst = { feedFirst }), + discoveryEpisodeIds = ids, + feedFirstWiring = FeedFirstWiring(copies = catalog.showPages), + ) + } + + /** Publisher feeds over an interceptor: [FEED_URL] lists [feedItems] episodes, newest first; any other address is gone. */ + private fun feedClient() = RssFeedClient( + OkHttpClient.Builder().addInterceptor { chain -> + val url = chain.request().url.toString() + val code = if (url == FEED_URL || url == RECORDED_FEED_URL) 200 else 404 + Response.Builder().request(chain.request()).protocol(Protocol.HTTP_1_1).code(code).message("test") + .body(feedXml(feedItems).toResponseBody("application/rss+xml".toMediaType())).build() + }.build(), + ) + + private fun row(id: String, key: String) = LocalEpisodeEntity( + episodeId = id, + podcastId = SHOW, + guid = key, + title = "Episode 1", + description = "", + audioUrl = "https://cdn.example/$key.mp3", + imageUrl = null, + duration = 60, + publishedDate = 1_790_000_000_000L - 86_400_000L, + chaptersUrl = null, + transcriptUrl = null, + transcripts = null, + persons = null, + seasonNumber = null, + episodeNumber = null, + episodeType = null, + enclosureType = "audio/mpeg", + ) + + private class FakeIds(private val owners: Map) : DiscoveryEpisodeIds { + override suspend fun recipeIds(episodeIds: Collection): Map = DiscoveryEpisodeIds.ownRecipeIds(episodeIds) + + override suspend fun localOwners(refs: Collection): Map = + refs.mapNotNull { ref -> owners[ref.recipeId]?.let { ref to DiscoveryEpisodeOwner(it, ref.podcastId) } }.toMap() + } + + private companion object { + const val SHOW = "920666" + const val FEED_URL = "https://feeds.example/920666.xml" + const val SAVED_URL = "https://saved.example/920666.xml" + + /** The address `fixtures/v2/podcast-direct.json` names. */ + const val RECORDED_FEED_URL = "https://feeds.podcastindex.org/pc20.xml" + + /** The http-only show of `fixtures/v2/podcast-proxy.json`. */ + const val PROXIED = "1031614" + + fun fixtureText(path: String): String = requireNotNull(PodcastRepositoryShowPagesTest::class.java.classLoader?.getResource(path)) { path }.readText() + + fun fixture(path: String) = json(fixtureText(path)) + val META = PodcastMeta(title = "Show") + + fun recipe(key: String): Long = EpisodeRecipeId.forNamespace(SHOW.toLong(), key) + + fun json(body: String) = MockResponse().setHeader("Content-Type", "application/json").setBody(body) + + /** A `/v2/episodes` answer naming [keys] under their recipe IDs, as the route re-keys Podcast Index's list. */ + fun v2Episodes(keys: List): String { + val items = + keys.mapIndexed { index, key -> + """{"id":${recipe(key)},"title":"$key","guid":"$key","catalogKey":"$key","enclosureUrl":"https://cdn.example/$index.mp3",""" + + """"feedId":$SHOW,"datePublished":${1_790_000_000L - index * 86_400L}}""" + } + return """{"status":"true","fallback":true,"source":"pi","feedId":$SHOW,"items":[${items.joinToString(",")}],"hasMore":false,"offset":0,"limit":20}""" + } + + fun feedXml(count: Int): String { + val items = + (0 until count).joinToString("") { index -> + val key = if (index == 2) "Moon g2" else "g$index" + val date = DateTimeFormatter.RFC_1123_DATE_TIME.format(Instant.ofEpochSecond(1_790_000_000L - index * 86_400L).atZone(ZoneOffset.UTC)) + "Episode $index$key$date" + + "" + } + return "Show$items" + } + } +} diff --git a/core/catalog/src/test/resources/fixtures/v2/README.md b/core/catalog/src/test/resources/fixtures/v2/README.md index 6309be98f..79af7de01 100644 --- a/core/catalog/src/test/resources/fixtures/v2/README.md +++ b/core/catalog/src/test/resources/fixtures/v2/README.md @@ -24,3 +24,13 @@ Worker answers with (the same fields, types and nesting as `shows-feeds.json`): |---|---| | `shows-feeds-registry-unavailable.json` | `POST /v2/shows/feeds` while the address registry is down: `registryAvailable: false`, addresses from the dump | | `feed-report.json` | `POST /v2/feed-report` for show 3391039: the registry's address. A live report would queue a real re-check | + +Show page routes, recorded the same way on 2026-10-27 (episode lists cut to two items, long text +shortened, nested lists cut to their first entry): + +| File | Request | +|---|---| +| `episodes.json` | `GET /v2/episodes?id=920666&limit=3&offset=0&sort=newest` (`source: pi`, items with `servedId` and `catalogKey`) | +| `podcast-direct.json` | `GET /v2/podcast?id=920666` (`fetchVia: direct`) | +| `podcast-proxy.json` | `GET /v2/podcast?id=1031614` (`fetchVia: proxy` with `transportUrl`) | +| `feed-not-proxied.json` | `GET /v2/feed/920666`, a show that isn't http-only: 404 `not_proxied` with `fetchUrl` | diff --git a/core/catalog/src/test/resources/fixtures/v2/episodes.json b/core/catalog/src/test/resources/fixtures/v2/episodes.json new file mode 100644 index 000000000..593740114 --- /dev/null +++ b/core/catalog/src/test/resources/fixtures/v2/episodes.json @@ -0,0 +1,155 @@ +{ + "status": "true", + "fallback": true, + "source": "pi", + "feedId": 920666, + "items": [ + { + "id": 5626175219222083, + "title": "Episode 276: \"Jackleg leechers\"", + "link": "https://www.podpage.com/podcasting-20/jackleg-leechers/", + "description": "

Podcasting 2.0 October 23d 2026 Episode 276 - \"Jackleg Leechers\"Podcasting 2.0 October 16th 2026 Episode 275 - \"Off The Rails\" + + /** + * Stored copies the automatic cleanup may remove: not followed by either record (the feed row's + * [LocalEpisodeFeedEntity.followed] and the `podcasts` row), and without an unsubscribe TTL, + * which [listExpiredFeedIds] handles. With each show's stored episode count. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, cleanup of unfollowed cached shows; decision D10). + * COMPAT(own-ids): bounded storage for show pages. Permanent: never remove (MIGRATION.md §5). + */ + @Query( + """ + SELECT f.podcastId AS podcastId, f.lastOpenedAt AS lastOpenedAt, f.fetchedAt AS fetchedAt, + (SELECT COUNT(*) FROM local_episodes e WHERE e.podcastId = f.podcastId) AS episodeCount + FROM local_episode_feeds f + WHERE f.followed = 0 AND f.ttlExpiresAt IS NULL + AND NOT EXISTS (SELECT 1 FROM podcasts p WHERE p.podcastId = f.podcastId AND p.isSubscribed = 1) + """, + ) + suspend fun listUnfollowedCopies(): List + + /** + * Deletes [podcastId]'s stored copy when it is still unfollowed, checked again inside the + * transaction so a follow that lands meanwhile keeps it. Only `local_episodes` and + * `local_episode_feeds` rows go. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, cleanup of unfollowed cached shows; decision D10). + */ + @Transaction + suspend fun deleteCatalogIfUnfollowed(podcastId: String): Boolean { + if (!isUnfollowedCopy(podcastId)) return false + deleteCatalog(podcastId) + return true + } + + /** True when [podcastId]'s stored copy is one [listUnfollowedCopies] lists. */ + @Query( + """ + SELECT EXISTS( + SELECT 1 FROM local_episode_feeds f + WHERE f.podcastId = :podcastId AND f.followed = 0 AND f.ttlExpiresAt IS NULL + AND NOT EXISTS (SELECT 1 FROM podcasts p WHERE p.podcastId = f.podcastId AND p.isSubscribed = 1) + ) + """, + ) + suspend fun isUnfollowedCopy(podcastId: String): Boolean + @Query("DELETE FROM local_episodes WHERE podcastId = :podcastId") suspend fun deleteEpisodes(podcastId: String) diff --git a/core/database/src/main/java/cx/aswin/boxlore/core/database/LocalEpisodeFeedEntity.kt b/core/database/src/main/java/cx/aswin/boxlore/core/database/LocalEpisodeFeedEntity.kt index 6ad8fa1b7..1b00ac19d 100644 --- a/core/database/src/main/java/cx/aswin/boxlore/core/database/LocalEpisodeFeedEntity.kt +++ b/core/database/src/main/java/cx/aswin/boxlore/core/database/LocalEpisodeFeedEntity.kt @@ -13,9 +13,9 @@ import androidx.room.PrimaryKey * A non-null TTL means the catalog is not ready for live lists: unsubscribed * reads use Podcast Index until resubscribe clears the TTL. * - * Room v39 adds the feed-first columns ([followed] through [showMetadataAt]). Nothing reads them - * yet. [LocalEpisodeCatalogDao.upsertFeed] keeps them when it replaces a stored row, so they change - * only through their own updates. + * Room v39 adds the feed-first columns ([followed] through [showMetadataAt]); the feed-first show + * page (P4 PR 16) reads and writes them. [LocalEpisodeCatalogDao.upsertFeed] keeps them when it + * replaces a stored row, so they change only through their own updates. */ @Entity(tableName = "local_episode_feeds") data class LocalEpisodeFeedEntity( @@ -33,8 +33,8 @@ data class LocalEpisodeFeedEntity( val ready: Boolean, val feedUrlLookupAt: Long, /** - * The show is followed. Seeded from `podcasts.isSubscribed` at the v39 upgrade; follow and - * unfollow don't update it yet, so whatever reads it first must also write it. + * The show is followed. Seeded from `podcasts.isSubscribed` at the v39 upgrade; since P4 PR 16 + * follow and unfollow write it with the unsubscribe TTL ([LocalEpisodeCatalogDao.setTtlAndFollowed]). * * Added in P4 of the own-episode-IDs plan (plan section 8, Room v39 feed columns). * COMPAT(own-ids): feed-first catalog state. Permanent: never remove (MIGRATION.md §5). @@ -103,3 +103,16 @@ object LocalFeedOrder { const val OLDEST_FIRST = "oldest_first" const val MIXED = "mixed" } + +/** + * A stored copy of an unfollowed show's feed ([LocalEpisodeCatalogDao.listUnfollowedCopies]), with + * when the show page last opened it and how many episodes it holds. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, cleanup of unfollowed cached shows; decision D10). + */ +data class UnfollowedCatalogCopy( + val podcastId: String, + val lastOpenedAt: Long, + val fetchedAt: Long, + val episodeCount: Int, +) diff --git a/core/domain/README.md b/core/domain/README.md index 15050fd72..f6aaf9233 100644 --- a/core/domain/README.md +++ b/core/domain/README.md @@ -19,6 +19,7 @@ Owns thin domain ports and small result types used by ViewModels, repositories, - `ConnectivityStatusPort` and connectivity status types. - `DeviceIdentityPort` for cross-device sync attribution. - `FeedTelemetryPort` reports feed reads (`feedFetched`) and saved-address swaps (`feedUrlCorrected`) without a dependency on `:core:analytics`; `AppContainer` wires the analytics-backed one and `NONE` reports nothing. Implementations must not throw. +- `ShowPageFeedPort` is the feed-first show page's stored feed copy of a Podcast Index show, followed or not (P4 PR 16): `cached` (episode count, whether it is the whole archive, when it was read, oldest-first), `read` (`NEWEST` or `ARCHIVE` of an HTTPS address; `Stored`, `NotModified` or `Failed` with the `/v2/feed-report` status, never a throw), `readTransport` (an http-only feed through boxlore's transport: the caller's fetch gets the copy's ETag and answers `Transported` `Document`, `NotModified`, `ReadDirect` or `Failed`; it runs under the read's lock and download gate), `search` (bounded in the query), `markOpened` and `cleanUp` (unfollowed copies not opened in 30 days, then a size cap; never history, downloads, the queue or aliases). Pages of the copy go through `LocalEpisodeCatalogPort.getPage`. `:core:rss` implements it (`LocalEpisodeCatalogRepository.showPages`). - `LocalEpisodeCatalogPort.RefreshRequest.reason` defaults to `NORMAL`; automatic refresh and `AUTO_DOWNLOAD` share the persisted six-hour freshness gate. `MANUAL` and `NEW_RELEASE` bypass quiet/HEAD shortcuts. `canProceed` rechecks authorization before network/persistence, `runPostPersistCallback` prevents background metadata recovery from invoking ungated ingest callbacks, and `isRefreshDue` supports bounded foreground batches. `RefreshOutcome.Failure.addressGone` says the saved address looks dead rather than temporarily failing, and `RefreshOutcome.Failure.reportStatus` says what the read saw in the words `/v2/feed-report` takes (`http_404`, `timeout`, `dns`, `tls`, `parse`, `network`, `other`; null when no read happened); with `RefreshRequest.detectMove` (and a `PodcastMeta.title` to compare), `RefreshOutcome.Success.movedTo` carries a confirmed new HTTPS address. `RefreshRequest.serverEpNs` passes the episode namespace the server reported for the show; it is only compared with the stored one and never changes an ID. Publisher RSS and Room remain the source of subscribed episodes. diff --git a/core/domain/src/main/java/cx/aswin/boxlore/core/domain/ports/ShowPageFeedPort.kt b/core/domain/src/main/java/cx/aswin/boxlore/core/domain/ports/ShowPageFeedPort.kt new file mode 100644 index 000000000..024ca106c --- /dev/null +++ b/core/domain/src/main/java/cx/aswin/boxlore/core/domain/ports/ShowPageFeedPort.kt @@ -0,0 +1,117 @@ +package cx.aswin.boxlore.core.domain.ports + +import cx.aswin.boxlore.core.model.Episode + +/** + * The phone's stored copy of a Podcast Index show's feed for its show page, followed or not, and + * the reads that fill it. Pages read the copy through [LocalEpisodeCatalogPort.getPage]; this port + * says what the copy holds, adds to it and searches it ([search], bounded). + * + * A read never makes an unfollowed show's copy the followed live list ([LocalEpisodeCatalogPort.isReady] + * stays false), and it never runs the post-write work of followed shows (auto-download, new-episode + * delivery). New items take the same IDs a followed show's refresh gives them; stored rows keep theirs. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page). + * COMPAT(own-ids): show pages read the show's own feed. Permanent: never remove (MIGRATION.md §5). + */ +interface ShowPageFeedPort { + /** + * What the phone holds for a show. + * + * - [itemCount]: stored episodes. + * - [archiveComplete]: every item of the feed has been read, not only the newest part. + * - [fetchedAt]: when the copy was last read or confirmed current (epoch ms). + * - [oldestFirst]: the feed lists its oldest items first, so a read of its start is no use. + */ + data class CachedShow( + val podcastId: String, + val feedUrl: String, + val fetchedAt: Long, + val itemCount: Int, + val archiveComplete: Boolean, + val oldestFirst: Boolean, + ) + + /** How much of the feed a read takes: its newest items, or the whole document. */ + enum class Scope { NEWEST, ARCHIVE } + + /** + * One read of [feedUrl] (HTTPS; for [readTransport] the publisher's http address, if known) for show [podcastId]. [serverEpNs] is only compared with the + * stored namespace (decision D12). + */ + data class ReadRequest( + val podcastId: String, + val feedUrl: String, + val scope: Scope, + val meta: LocalEpisodeCatalogPort.PodcastMeta = LocalEpisodeCatalogPort.PodcastMeta(), + val serverEpNs: Long? = null, + ) + + /** + * One answer of boxlore's transport (`/v2/feed/`), the only way to read an http-only feed. + * Transport documents are always the whole feed. + */ + sealed interface Transported { + /** The feed document; [etag] answers the next read's `If-None-Match`. */ + class Document(val bytes: ByteArray, val etag: String?) : Transported + + /** The stored copy is current (304 to the ETag the read sent). */ + data object NotModified : Transported + + /** The show's address is HTTPS after all (`404 not_proxied`): read [url] directly. */ + data class ReadDirect(val url: String) : Transported + + /** The transport couldn't answer; [reportStatus] as in [ReadOutcome.Failed]. */ + data class Failed(val reportStatus: String) : Transported + } + + sealed interface ReadOutcome { + /** The read was stored; [complete] when it held the whole feed. */ + data class Stored(val itemCount: Int, val complete: Boolean) : ReadOutcome + + /** The publisher said the copy is current (304). */ + data object NotModified : ReadOutcome + + /** + * The read failed. [reportStatus] is what it saw, in the words `/v2/feed-report` takes + * (`http_404`, `timeout`, `dns`, `tls`, `parse`, `network`, `other`). + */ + data class Failed(val reportStatus: String) : ReadOutcome + } + + suspend fun cached(podcastId: String): CachedShow? + + /** Reads [ReadRequest.feedUrl] and stores what it holds. Never throws for a failed read. */ + suspend fun read(request: ReadRequest): ReadOutcome + + /** + * Reads [request]'s show through boxlore's transport: [fetch] is given the stored ETag (only + * when the copy is the whole archive) and answers. It runs under the same per-show lock and + * download gate as [read]. Neither the transport address nor [ReadRequest.feedUrl] (the + * publisher's own http address) is recorded as the copy's address. A [Transported.ReadDirect] + * answer is read directly ([read]). + */ + suspend fun readTransport(request: ReadRequest, fetch: suspend (ifNoneMatch: String?) -> Transported): ReadOutcome + + /** + * In-show search over [podcastId]'s copy: titles and descriptions containing [query], newest + * first, at most [limit] episodes. + */ + suspend fun search( + podcastId: String, + query: String, + limit: Int, + meta: LocalEpisodeCatalogPort.PodcastMeta = LocalEpisodeCatalogPort.PodcastMeta(), + ): List + + /** Notes that the show page opened this show now, which keeps its copy from the automatic cleanup. */ + suspend fun markOpened(podcastId: String, atMillis: Long) + + /** + * Removes the copies of unfollowed shows not opened in 30 days, then the least recently opened + * ones over the size cap, never [keepPodcastId]. Only `local_episodes` and `local_episode_feeds` + * rows go: listening history, downloads, the queue and episode aliases stay. Returns how many + * shows were removed. + */ + suspend fun cleanUp(nowMillis: Long, keepPodcastId: String? = null): Int +} diff --git a/core/downloads/README.md b/core/downloads/README.md index 81ab6e863..5aec85753 100644 --- a/core/downloads/README.md +++ b/core/downloads/README.md @@ -84,6 +84,7 @@ Files under `core/data` are compatibility stubs for previously scheduled WorkMan ./gradlew :core:downloads:testDebugUnitTest ``` - `DownloadAudioUriTest` covers episodes whose audio URL is null or blank. +- `AutoDownloadCoordinatorTest.anUnfollowedCachedShowNeverAutoDownloads`: a show page's stored copy of an unfollowed show (P4 PR 16) schedules no post-write work, claims no release and enqueues nothing, even with auto-download left on from when it was followed. ## CI relevance diff --git a/core/downloads/src/test/java/cx/aswin/boxlore/core/downloads/AutoDownloadCoordinatorTest.kt b/core/downloads/src/test/java/cx/aswin/boxlore/core/downloads/AutoDownloadCoordinatorTest.kt index 62cbfdebb..79061e299 100644 --- a/core/downloads/src/test/java/cx/aswin/boxlore/core/downloads/AutoDownloadCoordinatorTest.kt +++ b/core/downloads/src/test/java/cx/aswin/boxlore/core/downloads/AutoDownloadCoordinatorTest.kt @@ -7,10 +7,12 @@ import cx.aswin.boxlore.core.database.AutoDownloadReleaseEntity import cx.aswin.boxlore.core.database.BoxLoreDatabase import cx.aswin.boxlore.core.database.PodcastEntity import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort import cx.aswin.boxlore.core.prefs.UserPreferencesRepository import cx.aswin.boxlore.core.rss.LocalEpisodeCatalogRepository import cx.aswin.boxlore.core.rss.RssFeedClient import cx.aswin.boxlore.core.rss.RssFetchResult +import cx.aswin.boxlore.core.rss.RssPrefixFetch import java.time.Instant import java.time.ZoneOffset import java.time.format.DateTimeFormatter @@ -144,6 +146,31 @@ class AutoDownloadCoordinatorTest { assertNull(database.autoDownloadDao().getShow("100")) } + /** + * A show page's stored copy of an unfollowed show (P4 PR 16) never reaches auto-download: the + * read runs no post-write work, and the show is not eligible however its old settings read. + */ + @Test fun anUnfollowedCachedShowNeverAutoDownloads() = runBlocking { + val persisted = mutableListOf() + val showPages = LocalEpisodeCatalogRepository.create(database, feed, onCatalogPersisted = { persisted += it }) + database.podcastDao().upsert( + PodcastEntity("200", "Cached", "Author", "", null, isSubscribed = false, autoDownloadEnabled = true, feedUrl = URL), + ) + val coordinator = coordinator() + coordinator.synchronizeSubscriptions() + feed.items = listOf("cached-new" to now + 60, "archive" to now - 86_400) + + val read = showPages.showPages.read(ShowPageFeedPort.ReadRequest("200", URL, ShowPageFeedPort.Scope.NEWEST)) + coordinator.scanCached("200") + coordinator.acceptRelease("200", catalog.findByCatalogKey("200", "cached-new", null)!!) + + assertTrue(read is ShowPageFeedPort.ReadOutcome.Stored) + assertTrue("a show page read schedules nothing", persisted.isEmpty()) + assertTrue(enqueued.isEmpty()) + assertNull(database.autoDownloadDao().getShow("200")) + assertFalse("the copy is never the followed live list", catalog.isReady("200")) + } + @Test fun publisherFailureKeepsPendingForRetry() = runBlocking { val coordinator = coordinator() coordinator.synchronizeSubscriptions() @@ -266,6 +293,7 @@ class AutoDownloadCoordinatorTest { var fail = false var afterFetch: () -> Unit = {} override suspend fun fetchConditional(url: String, etag: String?, lastModified: String?) = fetch(url) + override suspend fun fetchNewest(url: String, maxItems: Int, etag: String?, lastModified: String?) = RssPrefixFetch(fetch(url), complete = true) override suspend fun fetch(url: String): RssFetchResult { fetches++ if (fail) throw java.io.IOException("offline") diff --git a/core/rss/README.md b/core/rss/README.md index bb80f54fa..d3f12bf31 100644 --- a/core/rss/README.md +++ b/core/rss/README.md @@ -15,6 +15,7 @@ Failed automatic fetches record a six-hour retry boundary while keeping catalogs - `RssPodcastRepository` implements `RssSubscriptionPort` and manages RSS podcast and episode catalog operations. Refresh and subscribe remap parsed rows through `StickyRssEpisodeRemap.prepare` before writing `latestEpisode` and `rss_episodes`, so stored `episodeId`s never remint. `getEpisodesAround` queries forward-continuation episodes via `RssEpisodeDao.getEpisodesAfter` for same-show queue playback. - `RssPodcastRepository.legacySubscriptionRepair` exposes `LegacyRssSubscriptionRepair`. `inspect` returns a `LegacyRssFeedSnapshot` without mutating Room; `upgrade` returns a `LegacyRssUpgradeOutcome` and atomically copies a verified RSS catalog under a Podcast Index id without re-keying episodes, and records a trusted alias (`rss_follow_upgrade`) for every copied ID whose key follows the identity contract, so the IDs survive eviction. The copied IDs own their episodes: a trusted owner of the same (show, key) under another ID, left by an earlier read of the show's catalog, is demoted in the same transaction and stays trusted, so a read after eviction comes back under the copied IDs that hold the listener state. When such an owner's ID holds history, a download or a queue entry under any show, the upgrade writes nothing and returns `TARGET_ALREADY_IN_USE`. Both share the per-podcast refresh lock with `refreshCatalog`. - `LocalEpisodeCatalogRepository` implements `LocalEpisodeCatalogPort`: full publisher-feed catalog under a Podcast Index id, sticky upsert by guid, and a structural per-show ready gate (complete persisted count; a non-null unsubscribe `ttlExpiresAt` is not ready so unsubscribed reads stay on Podcast Index). `getWindow` with `aroundEpisodeId` uses `continuationRows` (`getEpisodesAfter`) to fetch the anchor and subsequent forward chronological episodes. A not-ready catalog always bypasses HEAD validators and performs the repair GET. Listener-id or cached PI-tip mismatches never permanently hide a successfully persisted publisher feed. HEAD/quiet skip and mega GET (capped at 2) remain for ready catalogs. It never creates `rss:` rows; incremental upsert never shrinks existing rows. HEAD validators are not reused after the publisher `feedUrl` changes. `sweepExpired` rechecks TTL before delete so a refresh that cleared `ttlExpiresAt` is kept. `:core:catalog` `SubscriptionForegroundSync` runs that sweep on each launch / Subscriptions refresh pass. +- `LocalEpisodeCatalogRepository.showPages` implements `ShowPageFeedPort` (`ShowPageFeedCatalog`) for the feed-first show page: a stored copy of a Podcast Index show's feed, followed or not, in the same `local_episodes` rows and through the same write (sticky IDs, recipe IDs for new items, aliases, the collision guard) under the same per-show lock. A first read takes the newest 100 items with `RssFeedClient.fetchNewest`, a streaming read that stops early (`FeedPrefixReader`, RSS `` or Atom ``, cut documents closed so they parse); the archive read takes the whole feed. `FeedRowPolicy` records `archiveComplete` apart from `ready`: a show-page read never makes a copy the followed live list and never runs the post-write work (auto-download, delivery). An oldest-first feed (classified in document order) is always read whole. `ShowPageFeedRules`: the newest read sends the copy's validators; an archive read of a partial copy goes out unconditional, so a 304 that covers only the stored prefix is never taken as the archive; a copy older than an hour is read again; cleanup removes unfollowed copies not opened in 30 days, then the least recently opened over 40 shows or 20,000 episodes. A document read through boxlore's transport (`readTransport`, with the copy's ETag, under the download gate) is parsed under one fixed namespace per show, and the row never records the transport or http address. A show-page write keeps the stored row's unsubscribe TTL and `ready`, read again inside its transaction, so a recently unfollowed show keeps its cheap resubscribe and TTL sweep and an unfollow during the read stays. A newest read that meets no stored item leaves a gap, so `archiveComplete` goes back to false and the next archive read goes out without validators. A new feed row takes `followed` from `podcasts.isSubscribed`. In-show search of a copy is limited in the query (`searchLimited`). `RssFeedClient.readFeedBody` is the 25 MB cap every feed read keeps (`FeedTooLargeException`), also for transport documents. Followed shows' refreshes now also set `archiveComplete`. P4 PR 16. - `StickyEpisodeIdentity` / `LocalEpisodeCatalogPersist` / `FeedOrderLogic` / `LocalCatalogReadyLogic` / `LocalCatalogOrphanRematch` are the identity and freshness helpers. `LocalCatalogListenerStateReconciler` (the orphan rematch) safely re-keys orphaned negative restore ids after a full persist (unique enclosure, then unique title within 48 hours measured in epoch seconds), merging history, sessions/rollups, and queue state into the chosen catalog id. It moves only unresolved legacy negative IDs, after consulting aliases. An ID that a catalog row of any show still holds (an older duplicate row, a changed GUID, an item the feed dropped) or whose alias is trusted or ambiguous (an evicted row's ID, for example) is a known episode, not an orphan, and never moves. An orphan that a new feed item provably owns is never re-keyed either: the read already gave that item the orphan's ID (`ListenerIdAdoption`, below). When a catalog row now holds a trusted orphan's (show, key) under another ID (its row was evicted and the feed read again, for example with a Podcast Index match), the row's ID joins that episode as a trusted alias (`EpisodeAliasProvenance.CATALOG_KEY_MATCH`), so both IDs read as one episode and nothing moves. Only phone-minted IDs with no alias or an unresolved one, whose episode already has a stored row or that match no new item uniquely, are rematched and moved; their history merges into the row the catalog row's episode reads (its owner when the episode is stored under several IDs); merged into another ID's row, it keeps that row's show and episode details, so an owner row of a show switched to its feed stays under that show. The merged like is the latest like decision of the two rows and the episode's group settings (`EpisodeListenerProjection.likeDecision`), so an older like can't undo a later unlike; completion is kept from either row unless the group settings cleared it later. When the match was exact (its enclosure address names one catalog row) and the catalog row has a trusted alias, the moved ID becomes a trusted alias of that episode (`EpisodeAliasProvenance.ORPHAN_REMATCH`), so a later copy of it counts as the same episode and isn't rematched again; a title and date match records no alias. Completed downloads move only when `DownloadCacheRelinker` confirms cached bytes reached the new key; otherwise the legacy row/cache is retained. Ambiguous or metadata-poor rows are retained rather than guessed and no longer block source cutover. - `ListenerStateMoves` holds the rules for the one path that still moves listener state between stored IDs, the orphan rematch (which IDs are settled, the catalog row that joins a trusted orphan's episode by key, the alias a moved ID gets, the merged completion), and the links that replace the former move when a Podcast Index show switches to its RSS feed (`switchedShowLinks`, `linkWrite`). `SwitchedShowLinker` writes those links. `RssPodcastRepository.confirmPodcastIndexLink`, and adding the feed of a followed Podcast Index show by its address, switch the show without moving anything: history, likes, completion, the queue, downloads, sessions and rollups stay under their stored IDs and show, and no download cache is relinked. Instead, each RSS episode that matches an episode those rows hold gets an alias under the Podcast Index show (`EpisodeAliasProvenance.RSS_SOURCE_LINK`), recorded as a non-owner; ownership then follows the owner rule like any trusted ID. The link is trusted, so the RSS episode reads the old IDs' projected state, its progress goes to the episode's owner and offline playback uses the ID that holds the completed download, when the old ID's trusted key is the RSS episode's key and no other RSS episode has it (for a show the listener confirmed as a lookalike, without the same feed address or podcast GUID, only with the same audio address or title on one of the old ID's rows as well), or when an old ID with no alias or an unresolved one names exactly one RSS episode with a GUID by its exact audio address and no other old ID points at it: that old ID is settled first, trusted under the Podcast Index show with the episode's key (`EpisodeAliasProvenance.RSS_SOURCE_AUDIO_MATCH`, the owner only when the episode has none). Any weaker match (title or date only, a key several RSS episodes share) is ambiguous; a weaker match from another old ID never undoes an exact one. An old ID whose alias is ambiguous links nothing, and an RSS episode whose ID the show's rows already hold gets no alias. Linking is idempotent and runs again after a library backup restore (`RssPodcastRepository.relinkSwitchedShow`), when the RSS show is followed again or restored with its link (`restoreSubscription`), and after every refresh of its feed (`refreshCatalog`, `refreshCatalogIfNeeded`, and each persisted read of `RssEpisodeCatalog`): it adds links for rows that arrived since and turns an ambiguous `rss_source_link` alias trusted once exact evidence exists, never downgrading a trusted alias or touching another alias. A Podcast Index show's namespace for the aliases' recipe IDs is its catalog's stored `epNs`, then the show's (decision D12). Every ID stays as it is, so following the Podcast Index show again keeps everything; its catalog read never reuses a linked RSS ID (`NewEpisodeIds` skips `rss_source_link` aliases). - `RssIdGenerator` creates deterministic `rss:` podcast IDs and negative episode IDs (mint only, not refresh). @@ -113,6 +114,7 @@ src/main/java/cx/aswin/boxlore/core/rss/ - The guard returns the rows to write, the stored keys it corrected with where each old key came from (`KeyRepair`: re-trimmed, synthetic, audio address), the reserved IDs it handed out, and any row with no free ID after 8 reserved tries. The refresh stores the corrected keys' provenance and the reserved IDs on aliases in the same transaction, and logs both. A row with no free ID makes the refresh write nothing and fail, so a partial catalog is never marked complete. - The legacy RSS upgrade keeps its IDs for listener state, so it stays on RSS instead when another show holds one, as a catalog row or as an evicted episode's alias. It takes ownership of an episode only from an owner whose ID holds no listener state. - Package root is `cx.aswin.boxlore.core.rss`. +- Feed-first copies (P4 PR 16) live in the existing `local_episode_feeds`/`local_episodes` tables; no schema change. Their v39 columns are written through their own updates: `archiveComplete` after a whole-feed read, `lastOpenedAt` on every show-page read or open, and `followed` with the unsubscribe TTL (`setUnsubscribedTtl` writes both). Cleanup deletes only those two tables' rows of shows followed by neither record: history, downloads, the queue and `episode_aliases` stay. ## Testing notes @@ -128,6 +130,7 @@ src/main/java/cx/aswin/boxlore/core/rss/ - `RssParsePolicyTest` checks the size limit and that a large valid feed never reaches the library parser. - `FeedDocumentOrderTest` and the `feed_identity.xml` fixture (`IdentityFixtureFeed`, with the rows older builds stored for it, read by both parsers and by the streaming parser alone): a duplicate GUID whose later copy is newer keeps the first copy; GUID-less items where the parsers pick different addresses are one episode keyed on the enclosure; a feed crossing 4 MB keeps every stored ID in both directions, from rows the new code wrote and from rows older builds wrote on either side; a library-only read finds the rows a joined read stored; NBSP and U+180E padding trimmed with the recipe set (written as `\u` escapes); which item updates which stored row (ID and key per title), and which older rows are no longer refreshed, for local catalogs; every stored ID kept for true RSS rows; exact keys win over older keys. `feed_shared_enclosure.xml`: GUID-less items sharing a cover-image or trailer enclosure stay apart. `LocalEpisodeCatalogRepositoryTest` runs both fixtures through the refresh and the collision guard (rows from older builds on either side of 4 MB with the leftover duplicates unchanged; shared enclosures through repeated refreshes, a new release and a read over 4 MB). `EpisodeSupplementRepositoryTest` refreshes supplements small, large, small from older rows and from none: no ID is deleted or changes. `LegacyRssSubscriptionUpgradeLogicTest` checks the upgrade's copied catalog keeps every stored RSS ID and gives a shared key to the feed item, which owners the copied IDs displace (trusted owners of copied contract keys under other IDs), and the ownership hand-over (demoted once a copied ID owns the key, the copied ID's trusted alias promoted, an owner kept when no copied ID can own). - `EpisodeRecipeIdTest` runs every case in the shared recipe fixture against `:core:model`'s `EpisodeRecipeId` and asserts the fixture's SHA-256, so a changed copy fails here and in the scripts test. +- `FeedPrefixReaderTest` (early stop, tags split across reads, Atom, byte cap, `fetchNewest` validators and 304), `ShowPageFeedRulesTest` (the prefix 304 rule, oldest-first, cleanup age and caps) and `ShowPageFeedCatalogTest` (real Room and parser: newest 100 then the archive with the same IDs, a 304 to the stored newest never taken as the archive, oldest-first feeds read whole, transport documents with their ETag and a `not_proxied` answer read directly, a show-page read that keeps an unsubscribe TTL so a resubscribe stays ready, a newest read past a gap, `followed` for a show followed before its first read, bounded search, cleanup that keeps history, downloads, the queue and aliases, and follow/unfollow writing `followed`). ## CI relevance diff --git a/core/rss/src/main/java/cx/aswin/boxlore/core/rss/FeedPrefixReader.kt b/core/rss/src/main/java/cx/aswin/boxlore/core/rss/FeedPrefixReader.kt new file mode 100644 index 000000000..a8c11082c --- /dev/null +++ b/core/rss/src/main/java/cx/aswin/boxlore/core/rss/FeedPrefixReader.kt @@ -0,0 +1,107 @@ +package cx.aswin.boxlore.core.rss + +import java.io.InputStream + +/** + * The start of a feed document read from a stream, for [RssFeedClient.fetchNewest]. + * + * [bytes] is a parseable document: when the read stopped early it is cut after the last whole item + * and closed with the root's end tags. [complete] is true when the stream ended on its own, so + * [bytes] is the whole document. [itemCount] counts the items whose end tag was read. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page: the newest + * 100 come from a streaming read that can stop early). + */ +class FeedPrefix(val bytes: ByteArray, val complete: Boolean, val itemCount: Int) + +/** + * Reads a feed document until [read]'s `maxItems` items have ended, then stops, so a show page can + * show a show's newest episodes without downloading a feed of thousands. Items are RSS `` or + * Atom `` elements, matched by their end tag without regard to case. A document that ends + * before that is returned whole. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page). + */ +internal object FeedPrefixReader { + private val ITEM_END = "".toByteArray(Charsets.US_ASCII) + private val ENTRY_END = "".toByteArray(Charsets.US_ASCII) + private val RSS_CLOSE = "\n\n\n".toByteArray(Charsets.US_ASCII) + private val ATOM_CLOSE = "\n\n".toByteArray(Charsets.US_ASCII) + private const val CHUNK = 16 * 1024 + + /** Reads at most [maxBytes]; throws [IllegalArgumentException] past it, as a whole read does. */ + fun read(input: InputStream, maxItems: Int, maxBytes: Long): FeedPrefix { + require(maxItems > 0) { "maxItems must be positive" } + val buffer = GrowableBytes() + val chunk = ByteArray(CHUNK) + val scan = ItemScan() + while (true) { + val read = input.read(chunk) + if (read < 0) return FeedPrefix(buffer.toArray(), complete = true, itemCount = scan.count) + buffer.append(chunk, read) + require(buffer.size <= maxBytes) { "Feed is larger than ${maxBytes / (1024 * 1024)} MB" } + val cut = scan.advance(buffer, maxItems) + if (cut != null) { + val close = if (scan.atom) ATOM_CLOSE else RSS_CLOSE + return FeedPrefix(buffer.toArray(cut) + close, complete = false, itemCount = scan.count) + } + } + } + + /** Finds item end tags across chunk boundaries; [advance] returns the cut offset once [count] reaches the limit. */ + private class ItemScan { + var count = 0 + var atom = false + private var position = 0 + + fun advance(buffer: GrowableBytes, maxItems: Int): Int? { + // The last few bytes wait for the next chunk, so a tag split across chunks is still found. + while (position <= buffer.size - ENTRY_END.size) { + val end = + when { + buffer.matchesAt(position, ITEM_END) -> position + ITEM_END.size + buffer.matchesAt(position, ENTRY_END) -> { + atom = true + position + ENTRY_END.size + } + else -> null + } + if (end == null) { + position++ + continue + } + count++ + position = end + if (count >= maxItems) return end + } + return null + } + } + + private class GrowableBytes { + private var data = ByteArray(CHUNK) + var size = 0 + private set + + fun append(source: ByteArray, length: Int) { + if (size + length > data.size) data = data.copyOf(maxOf(data.size * 2, size + length)) + System.arraycopy(source, 0, data, size, length) + size += length + } + + fun matchesAt(offset: Int, tag: ByteArray): Boolean { + for (index in tag.indices) { + if (lower(data[offset + index]) != tag[index]) return false + } + return true + } + + fun toArray(length: Int = size): ByteArray = data.copyOf(length) + + private fun lower(byte: Byte): Byte = if (byte in UPPER_A..UPPER_Z) (byte + CASE_OFFSET).toByte() else byte + } + + private const val UPPER_A: Byte = 'A'.code.toByte() + private const val UPPER_Z: Byte = 'Z'.code.toByte() + private const val CASE_OFFSET = 'a'.code - 'A'.code +} diff --git a/core/rss/src/main/java/cx/aswin/boxlore/core/rss/FeedReadFailure.kt b/core/rss/src/main/java/cx/aswin/boxlore/core/rss/FeedReadFailure.kt index 34c9da9da..19dc52eb9 100644 --- a/core/rss/src/main/java/cx/aswin/boxlore/core/rss/FeedReadFailure.kt +++ b/core/rss/src/main/java/cx/aswin/boxlore/core/rss/FeedReadFailure.kt @@ -11,6 +11,12 @@ import javax.net.ssl.SSLPeerUnverifiedException /** A publisher feed answered with an HTTP error status ([RssFeedClient.fetch]). */ class FeedHttpException(val code: Int) : IllegalArgumentException("Feed returned HTTP $code") +/** + * A feed document over the size cap ([RssFeedClient.readFeedBody]). An [IllegalArgumentException], + * as the cap's `require` threw before P4 PR 16, so callers that caught that still do. + */ +class FeedTooLargeException(maxBytes: Long) : IllegalArgumentException("Feed is larger than ${maxBytes / (1024 * 1024)} MB") + /** * Whether a failed publisher-feed read says the saved address no longer works, so it's worth asking * the server for a newer one: gone or refused (401, 403, 404, 410, 451), no such host, a certificate diff --git a/core/rss/src/main/java/cx/aswin/boxlore/core/rss/LocalEpisodeCatalogReads.kt b/core/rss/src/main/java/cx/aswin/boxlore/core/rss/LocalEpisodeCatalogReads.kt index a484b7da3..2ac312320 100644 --- a/core/rss/src/main/java/cx/aswin/boxlore/core/rss/LocalEpisodeCatalogReads.kt +++ b/core/rss/src/main/java/cx/aswin/boxlore/core/rss/LocalEpisodeCatalogReads.kt @@ -119,9 +119,13 @@ internal class LocalEpisodeCatalogReads( dao.getFeed(podcastId)?.feedUrlLookupAt ?: 0L } + /** + * Follow clears the TTL and unfollow sets it; since P4 PR 16 the same write keeps + * `local_episode_feeds.followed` in step, which the cleanup of unfollowed copies reads. + */ override suspend fun setUnsubscribedTtl(podcastId: String, ttlExpiresAt: Long?,) = withContext(Dispatchers.IO) { if (dao.getFeed(podcastId) == null) return@withContext - dao.setTtl(podcastId, ttlExpiresAt) + dao.setTtlAndFollowed(podcastId, ttlExpiresAt, followed = ttlExpiresAt == null) } override suspend fun sweepExpired(nowMillis: Long) = withContext(Dispatchers.IO) { diff --git a/core/rss/src/main/java/cx/aswin/boxlore/core/rss/LocalEpisodeCatalogRefresh.kt b/core/rss/src/main/java/cx/aswin/boxlore/core/rss/LocalEpisodeCatalogRefresh.kt index 1f8c5f833..be10baf4d 100644 --- a/core/rss/src/main/java/cx/aswin/boxlore/core/rss/LocalEpisodeCatalogRefresh.kt +++ b/core/rss/src/main/java/cx/aswin/boxlore/core/rss/LocalEpisodeCatalogRefresh.kt @@ -146,13 +146,86 @@ internal fun olderNamespaces( .filter { it != rssNamespaceId } .distinct() -private suspend fun persistParsed( +/** + * How a write records the show's feed row besides its episodes. + * + * - [complete]: the read held the whole feed, so the copy is the complete archive + * ([LocalEpisodeFeedEntity.archiveComplete]); a read of the newest items only leaves it as it was. + * - [showPage]: a show-page read ([ShowPageFeedCatalog]). It never makes a copy the followed live + * list, and it keeps what follow and unfollow wrote: the stored row's `ready` and unsubscribe TTL + * are read again inside the write's transaction and kept, so a recently unfollowed show keeps its + * cheap resubscribe and its TTL sweep, and an unfollow that lands during the read stays. + * - [documentOrder]: the feed's order classified in document order, for a row that has none yet. + * - [openedAt]: when the show page opened the show (epoch ms), kept from the automatic cleanup. + * - [recordedAddress]: the address the row records instead of the one read; boxlore's transport + * address of an http-only feed is never recorded. + * + * The defaults are a followed show's refresh, which reads the whole feed. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page: `archiveComplete` + * is separate from ready). + * COMPAT(own-ids): feed-first catalog state. Permanent: never remove (MIGRATION.md §5). + */ +internal data class FeedRowPolicy( + val complete: Boolean = true, + val showPage: Boolean = false, + val documentOrder: String? = null, + val openedAt: Long? = null, + val recordedAddress: String? = null, +) { + /** A stored order other than mixed stays; else the document order, else the parsed dates. */ + fun feedOrder(existing: LocalEpisodeFeedEntity?, parsed: ParsedRssFeed): String = existing + ?.feedOrder + ?.takeIf { it != LocalFeedOrder.MIXED } + ?: documentOrder + ?: FeedOrderLogic.classify(parsed.episodes.map { it.publishedDate }) + + /** + * A show-page read keeps the stored row's `ready` ([current], read in the write's transaction); + * a followed show's refresh flips once the count is complete. + */ + fun ready(current: LocalEpisodeFeedEntity?, storedCount: Int, copied: Int): Boolean = if (showPage) { + current?.ready == true + } else { + LocalCatalogReadyLogic.isReadyToFlip(feedReady = storedCount >= copied) + } + + /** A show-page read keeps the stored unsubscribe TTL ([current]); a followed show's refresh clears it. */ + fun ttlExpiresAt(current: LocalEpisodeFeedEntity?): Long? = if (showPage) current?.ttlExpiresAt else null + + /** + * Whether a read of the newest items left a gap: the copy had rows, yet none of the items read + * was one of them, so more items than the read holds came out since the last read. The copy is + * then no longer the whole archive. + */ + fun leavesGap(existingRows: Int, planned: List): Boolean = + !complete && existingRows > 0 && planned.none { it.source == PlannedIdSource.STORED } + + /** A read of the newest items leaves a pending backfill pending. */ + fun needsFullBackfill(existing: LocalEpisodeFeedEntity?): Boolean = !complete && (existing?.needsFullBackfill ?: true) + + /** + * The v39 columns [LocalEpisodeCatalogDao.upsertFeed] keeps, written on their own: the archive + * state ([gap]: see [leavesGap]), the opening time, and `followed` for a row this write created + * ([created]), from `podcasts.isSubscribed`. + */ + suspend fun writeFeedFirstColumns(dao: LocalEpisodeCatalogDao, podcastId: String, created: Boolean, gap: Boolean) { + if (complete) dao.setArchiveComplete(podcastId, true) + if (gap) dao.setArchiveComplete(podcastId, false) + openedAt?.let { dao.setLastOpenedAt(podcastId, it) } + if (created) dao.seedFollowed(podcastId) + } +} + +@Suppress("LongParameterList", "kotlin:S107") // One write: the read, its parse and how the row is recorded. +internal suspend fun persistParsed( deps: LocalCatalogRefreshDeps, request: RefreshRequest, existing: LocalEpisodeFeedEntity?, fetched: RssFetchResult, rssNamespaceId: String, parsed: ParsedRssFeed, + policy: FeedRowPolicy = FeedRowPolicy(), ): RefreshOutcome { val identities = deps.dao.listIdentities(request.podcastIndexId) val newIds = newEpisodeIds(deps, request, existing) @@ -174,11 +247,7 @@ private suspend fun persistParsed( return RefreshOutcome.Failure(LocalEpisodeCatalogRepository.FEED_LOAD_FAILED_MESSAGE, addressGone = true, reportStatus = FeedReadFailure.PARSE) } val copied = existing?.copiedExtrasCount ?: 0 - val feedOrder = - existing - ?.feedOrder - ?.takeIf { it != LocalFeedOrder.MIXED } - ?: FeedOrderLogic.classify(parsed.episodes.map { it.publishedDate }) + val feedOrder = policy.feedOrder(existing, parsed) var written = rows var resolution: LocalEpisodeCollisionGuard.Resolution? = null var storedCount = rows.size @@ -207,19 +276,20 @@ private suspend fun persistParsed( deps.dao.upsertEpisodes(written) recordAliases(deps, request.podcastIndexId, planned, resolved, newIds) deps.reconcileListenerState(request.podcastIndexId, written) + val current = deps.dao.getFeed(request.podcastIndexId) storedCount = maxOf(deps.dao.count(request.podcastIndexId), written.size) - ready = LocalCatalogReadyLogic.isReadyToFlip(feedReady = storedCount >= copied) + ready = policy.ready(current, storedCount, copied) deps.dao.upsertFeed( LocalEpisodeFeedEntity( podcastId = request.podcastIndexId, - feedUrl = fetched.finalUrl, + feedUrl = policy.recordedAddress ?: fetched.finalUrl, feedEtag = fetched.etag, feedLastModified = fetched.lastModified, fetchedAt = System.currentTimeMillis(), itemCount = storedCount, feedOrder = feedOrder, - ttlExpiresAt = null, - needsFullBackfill = false, + ttlExpiresAt = policy.ttlExpiresAt(current), + needsFullBackfill = policy.needsFullBackfill(current ?: existing), copiedExtrasCount = copied, ready = ready, feedUrlLookupAt = existing?.feedUrlLookupAt ?: 0L, @@ -231,6 +301,12 @@ private suspend fun persistParsed( deps.dao.setEpNsIfUnset(request.podcastIndexId, epNs) deps.identity.keepShowNamespace(request.podcastIndexId, epNs) } + policy.writeFeedFirstColumns( + deps.dao, + request.podcastIndexId, + created = current == null, + gap = policy.leavesGap(identities.size, planned), + ) } resolution?.let { logResolution(request.podcastIndexId, it, written = !incomplete) } if (incomplete) return RefreshOutcome.Failure(LocalEpisodeCatalogRepository.FEED_LOAD_FAILED_MESSAGE) diff --git a/core/rss/src/main/java/cx/aswin/boxlore/core/rss/LocalEpisodeCatalogRepository.kt b/core/rss/src/main/java/cx/aswin/boxlore/core/rss/LocalEpisodeCatalogRepository.kt index bd6f41471..505a2862f 100644 --- a/core/rss/src/main/java/cx/aswin/boxlore/core/rss/LocalEpisodeCatalogRepository.kt +++ b/core/rss/src/main/java/cx/aswin/boxlore/core/rss/LocalEpisodeCatalogRepository.kt @@ -9,6 +9,7 @@ import cx.aswin.boxlore.core.database.LocalFeedOrder import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort.RefreshOutcome import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort.RefreshRequest +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort import cx.aswin.boxlore.core.rss.ports.DownloadCacheRelinker import java.util.concurrent.ConcurrentHashMap import kotlinx.coroutines.Dispatchers @@ -92,6 +93,17 @@ class LocalEpisodeCatalogRepository private constructor( private val refreshLocks = ConcurrentHashMap() + /** + * The show page's stored feed copies, followed or not, written under the same per-show lock as + * [refresh] ([ShowPageFeedCatalog]). + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page). + * COMPAT(own-ids): show pages read the show's own feed. Permanent: never remove (MIGRATION.md §5). + */ + val showPages: ShowPageFeedPort by lazy { + ShowPageFeedCatalog(refreshDeps) { podcastId -> refreshLocks.getOrPut(podcastId) { Mutex() } } + } + override suspend fun refresh(request: RefreshRequest): RefreshOutcome = withContext(Dispatchers.IO) { val lock = refreshLocks.getOrPut(request.podcastIndexId) { Mutex() } val outcome = lock.withLock { refreshLocalCatalogLocked(refreshDeps, request) } diff --git a/core/rss/src/main/java/cx/aswin/boxlore/core/rss/RssFeedClient.kt b/core/rss/src/main/java/cx/aswin/boxlore/core/rss/RssFeedClient.kt index 6210e2361..eec90a3f6 100644 --- a/core/rss/src/main/java/cx/aswin/boxlore/core/rss/RssFeedClient.kt +++ b/core/rss/src/main/java/cx/aswin/boxlore/core/rss/RssFeedClient.kt @@ -57,6 +57,14 @@ data class RssFetchResult( val movedTo: String? = null, ) +/** + * A read of the start of a feed ([RssFeedClient.fetchNewest]). [complete]: the whole document was + * read, so [result] holds every item. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page). + */ +class RssPrefixFetch(val result: RssFetchResult, val complete: Boolean) + sealed interface RssFreshnessResult { data class Unchanged(val etag: String?, val lastModified: String?,) : RssFreshnessResult @@ -116,6 +124,33 @@ open class RssFeedClient(private val httpClient: OkHttpClient = defaultHttpClien ) } + /** + * Reads the first [maxItems] items of a feed and stops, instead of downloading the whole + * document ([FeedPrefixReader]). Sends [etag] and [lastModified] when given; null means a 304, + * so the stored copy is current. [RssPrefixFetch.complete] says whether the whole document was + * read. The validators in the answer describe the whole document, not the part read. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page). + */ + open suspend fun fetchNewest(url: String, maxItems: Int, etag: String? = null, lastModified: String? = null,): RssPrefixFetch? = + execute(conditionalGetRequest(url, etag, lastModified)) { response -> + if (response.code == HTTP_NOT_MODIFIED) return@execute null + if (!response.isSuccessful) throw FeedHttpException(response.code) + validateContentType(response) + val prefix = FeedPrefixReader.read(response.body.byteStream(), maxItems, MAX_FEED_BYTES) + RssPrefixFetch( + result = + RssFetchResult( + finalUrl = response.request.url.toString(), + etag = response.header("ETag"), + lastModified = response.header(LAST_MODIFIED_HEADER), + body = prefix.bytes, + movedTo = permanentMoveTarget(response), + ), + complete = prefix.complete, + ) + } + suspend fun confirmHeadValidators(url: String, etag: String?, lastModified: String?,): Boolean = confirmUnchanged(url, etag, lastModified) /** @@ -650,21 +685,7 @@ open class RssFeedClient(private val httpClient: OkHttpClient = defaultHttpClien }.getOrDefault("") } - private fun readBounded(input: java.io.InputStream): ByteArray { - val output = ByteArrayOutputStream() - val buffer = ByteArray(DEFAULT_BUFFER_SIZE) - var total = 0L - while (true) { - val read = input.read(buffer) - if (read < 0) break - total += read - require(total <= MAX_FEED_BYTES) { - "Feed is larger than ${MAX_FEED_BYTES / (1024 * 1024)} MB" - } - output.write(buffer, 0, read) - } - return output.toByteArray() - } + private fun readBounded(input: java.io.InputStream): ByteArray = readFeedBody(input) private fun validateContentType(response: Response) { val contentType = @@ -767,6 +788,27 @@ open class RssFeedClient(private val httpClient: OkHttpClient = defaultHttpClien } companion object { + /** + * Reads a feed document from [input], at most [MAX_FEED_BYTES]: the cap every feed read + * keeps, also for documents boxlore's transport (`/v2/feed/`) serves. A larger + * document throws [FeedTooLargeException] before it fills memory. + * + * Public since P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page). + */ + fun readFeedBody(input: java.io.InputStream): ByteArray { + val output = ByteArrayOutputStream() + val buffer = ByteArray(DEFAULT_BUFFER_SIZE) + var total = 0L + while (true) { + val read = input.read(buffer) + if (read < 0) break + total += read + if (total > MAX_FEED_BYTES) throw FeedTooLargeException(MAX_FEED_BYTES) + output.write(buffer, 0, read) + } + return output.toByteArray() + } + private const val MAX_FEED_BYTES = 25L * 1024L * 1024L private const val MAX_DESCRIPTION_LENGTH = 20_000 private const val HTTP_NOT_MODIFIED = 304 diff --git a/core/rss/src/main/java/cx/aswin/boxlore/core/rss/ShowPageFeedCatalog.kt b/core/rss/src/main/java/cx/aswin/boxlore/core/rss/ShowPageFeedCatalog.kt new file mode 100644 index 000000000..9bced9221 --- /dev/null +++ b/core/rss/src/main/java/cx/aswin/boxlore/core/rss/ShowPageFeedCatalog.kt @@ -0,0 +1,217 @@ +package cx.aswin.boxlore.core.rss + +import cx.aswin.boxlore.core.database.LocalEpisodeFeedEntity +import cx.aswin.boxlore.core.database.LocalFeedOrder +import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort.PodcastMeta +import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort.RefreshOutcome +import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort.RefreshRequest +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.ReadOutcome +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.ReadRequest +import cx.aswin.boxlore.core.model.Episode +import kotlinx.coroutines.CancellationException +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.sync.Mutex +import kotlinx.coroutines.sync.withLock +import kotlinx.coroutines.sync.withPermit +import kotlinx.coroutines.withContext + +/** + * [ShowPageFeedPort] on the local episode catalog: a show page's reads go into the same + * `local_episodes` rows a followed show's refresh writes, through the same write + * ([persistParsed]: sticky IDs, recipe IDs for new items, aliases, the collision guard), under the + * same per-show lock. What differs is recorded in the feed row ([FeedRowPolicy]): a read of the + * newest items leaves the archive incomplete, and a show-page read never makes an unfollowed copy + * the followed live list. No post-write work runs (auto-download, delivery). + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page; decision D10). + * COMPAT(own-ids): show pages read the show's own feed. Permanent: never remove (MIGRATION.md §5). + */ +internal class ShowPageFeedCatalog( + private val deps: LocalCatalogRefreshDeps, + private val lockFor: (String) -> Mutex, +) : ShowPageFeedPort { + override suspend fun cached(podcastId: String): ShowPageFeedPort.CachedShow? = withContext(Dispatchers.IO) { + val feed = deps.dao.getFeed(podcastId) ?: return@withContext null + val count = deps.dao.count(podcastId) + if (count <= 0) return@withContext null + ShowPageFeedPort.CachedShow( + podcastId = podcastId, + feedUrl = feed.feedUrl, + fetchedAt = feed.fetchedAt, + itemCount = count, + archiveComplete = feed.archiveComplete, + oldestFirst = feed.feedOrder == LocalFeedOrder.OLDEST_FIRST, + ) + } + + override suspend fun read(request: ReadRequest): ReadOutcome = withContext(Dispatchers.IO) { + if (!isFeedShow(request.podcastId)) return@withContext ReadOutcome.Failed(OTHER) + val url = LocalEpisodeCatalogRepository.resolveHttps(request.feedUrl, null) ?: return@withContext ReadOutcome.Failed(OTHER) + lockFor(request.podcastId).withLock { + failSafe { + val existing = deps.dao.getFeed(request.podcastId) + val wholeFeed = ShowPageFeedRules.readsWholeFeed(request.scope, existing) + val fetched = fetch(url, wholeFeed, existing) ?: return@failSafe notModified(request.podcastId, existing) + persist(request, existing, fetched, url) + } + } + } + + override suspend fun readTransport( + request: ReadRequest, + fetch: suspend (ifNoneMatch: String?) -> ShowPageFeedPort.Transported, + ): ReadOutcome = withContext(Dispatchers.IO) { + if (!isFeedShow(request.podcastId)) return@withContext ReadOutcome.Failed(OTHER) + var direct: String? = null + val outcome = + lockFor(request.podcastId).withLock { + failSafe { + val existing = deps.dao.getFeed(request.podcastId) + // Transport documents are the whole feed: the ETag goes out only for a whole copy. + val etag = ShowPageFeedRules.validators(wholeFeed = true, existing)?.first + when (val answer = deps.megaGetGate.withPermit { fetch(etag) }) { + is ShowPageFeedPort.Transported.Document -> storeTransported(request, existing, answer) + ShowPageFeedPort.Transported.NotModified -> + if (etag != null) notModified(request.podcastId, existing) else ReadOutcome.Failed(OTHER) + is ShowPageFeedPort.Transported.ReadDirect -> { + direct = answer.url + ReadOutcome.Failed(OTHER) + } + is ShowPageFeedPort.Transported.Failed -> ReadOutcome.Failed(answer.reportStatus) + } + } + } + // Read outside the lock: [read] takes it again. + direct?.let { url -> read(request.copy(feedUrl = url)) } ?: outcome + } + + override suspend fun search(podcastId: String, query: String, limit: Int, meta: PodcastMeta,): List = withContext(Dispatchers.IO) { + val trimmed = query.trim() + if (trimmed.isEmpty() || limit <= 0) return@withContext emptyList() + deps.dao.searchLimited(podcastId, trimmed.escapeForSqlLike(), limit).map { it.toCatalogEpisode(meta) } + } + + /** + * A transport document. The phone never mints IDs from an http address ([RssIdGenerator] + * refuses them), so no older ID of the show can be under its publisher address; its namespace + * is one fixed address per show ([transportNamespace]), which never changes with the API's + * host. The row keeps the address it had, if any: the transport address needs the API session + * and is never recorded. + */ + private suspend fun storeTransported( + request: ReadRequest, + existing: LocalEpisodeFeedEntity?, + document: ShowPageFeedPort.Transported.Document, + ): ReadOutcome { + val result = RssFetchResult(transportNamespace(request.podcastId), document.etag, null, document.bytes) + return persist(request, existing, Fetched(result, complete = true), url = null, recordedAddress = existing?.feedUrl.orEmpty()) + } + + override suspend fun markOpened(podcastId: String, atMillis: Long) = withContext(Dispatchers.IO) { + deps.dao.setLastOpenedAt(podcastId, atMillis) + } + + override suspend fun cleanUp(nowMillis: Long, keepPodcastId: String?): Int = withContext(Dispatchers.IO) { + val victims = ShowPageFeedRules.cleanupVictims(deps.dao.listUnfollowedCopies(), nowMillis, keepPodcastId) + victims.count { podcastId -> lockFor(podcastId).withLock { deps.dao.deleteCatalogIfUnfollowed(podcastId) } } + } + + private class Fetched(val result: RssFetchResult, val complete: Boolean) + + /** The read, or null for a 304 that answers it. */ + private suspend fun fetch(url: String, wholeFeed: Boolean, existing: LocalEpisodeFeedEntity?): Fetched? { + val validators = ShowPageFeedRules.validators(wholeFeed, existing) + if (!wholeFeed) { + val prefix = deps.feedClient.fetchNewest(url, ShowPageFeedRules.NEWEST_ITEMS, validators?.first, validators?.second) ?: return null + return Fetched(prefix.result, prefix.complete) + } + val conditional = deps.megaGetGate.withPermit { deps.feedClient.fetchConditional(url, validators?.first, validators?.second) } + if (conditional != null) return Fetched(conditional, complete = true) + if (ShowPageFeedRules.acceptsNotModified(wholeFeed = true, cachedArchiveComplete = existing?.archiveComplete == true)) return null + // A 304 that covers only the stored newest items is not the archive: read it whole. + return Fetched(deps.megaGetGate.withPermit { deps.feedClient.fetch(url) }, complete = true) + } + + private suspend fun notModified(podcastId: String, existing: LocalEpisodeFeedEntity?): ReadOutcome { + existing?.let { deps.dao.upsertFeed(it.copy(fetchedAt = deps.nowMillis())) } + deps.dao.setLastOpenedAt(podcastId, deps.nowMillis()) + return ReadOutcome.NotModified + } + + private suspend fun persist( + request: ReadRequest, + existing: LocalEpisodeFeedEntity?, + first: Fetched, + url: String?, + recordedAddress: String? = null, + ): ReadOutcome { + var fetched = first + var parsed = parseOrNull(fetched) + if (parsed == null && !fetched.complete && url != null) { + // The cut-short document didn't parse (an unusual root): read the whole feed instead. + fetched = Fetched(deps.megaGetGate.withPermit { deps.feedClient.fetch(url) }, complete = true) + parsed = parseOrNull(fetched) + } + if (parsed == null) return ReadOutcome.Failed(FeedReadFailure.PARSE) + var order = documentOrder(parsed) + if (!fetched.complete && order == LocalFeedOrder.OLDEST_FIRST && url != null) { + // The start of an oldest-first feed holds its oldest items, not its newest. + fetched = Fetched(deps.megaGetGate.withPermit { deps.feedClient.fetch(url) }, complete = true) + parsed = parseOrNull(fetched) ?: return ReadOutcome.Failed(FeedReadFailure.PARSE) + order = documentOrder(parsed) + } + val refresh = + RefreshRequest( + podcastIndexId = request.podcastId, + feedUrl = url ?: fetched.result.finalUrl, + meta = request.meta, + runPostPersistCallback = false, + serverEpNs = request.serverEpNs, + ) + val policy = + FeedRowPolicy( + complete = fetched.complete, + showPage = true, + documentOrder = order, + openedAt = deps.nowMillis(), + recordedAddress = recordedAddress, + ) + val namespace = RssIdGenerator.podcastId(fetched.result.finalUrl) + return when (val outcome = persistParsed(deps, refresh, existing, fetched.result, namespace, parsed, policy)) { + is RefreshOutcome.Success -> ReadOutcome.Stored(outcome.itemCount, fetched.complete) + is RefreshOutcome.Unchanged -> ReadOutcome.NotModified + is RefreshOutcome.Failure -> ReadOutcome.Failed(outcome.reportStatus ?: OTHER) + } + } + + private suspend fun parseOrNull(fetched: Fetched): ParsedRssFeed? = try { + deps.feedClient.parse(fetched.result.finalUrl, fetched.result.body, RssIdGenerator.podcastId(fetched.result.finalUrl)) + } catch (error: CancellationException) { + throw error + } catch (_: Exception) { + null + } + + private suspend fun failSafe(block: suspend () -> ReadOutcome): ReadOutcome = try { + block() + } catch (error: CancellationException) { + throw error + } catch (error: Exception) { + ReadOutcome.Failed(FeedReadFailure.reportStatus(error)) + } + + private companion object { + const val OTHER = "other" + + /** The parse and namespace address of [podcastId]'s transport documents. */ + fun transportNamespace(podcastId: String): String = "https://transport.boxlore.invalid/feed/$podcastId" + + /** Show pages store copies for Podcast Index shows; `rss:` shows have their own table. */ + fun isFeedShow(podcastId: String): Boolean = podcastId.isNotBlank() && !podcastId.startsWith("rss:") + + /** The feed's order as its document lists the items, not as the parser sorted them. */ + fun documentOrder(parsed: ParsedRssFeed): String = + FeedOrderLogic.classify(parsed.items.sortedBy(ParsedFeedItem::documentIndex).map { it.episode.publishedDate }) + } +} diff --git a/core/rss/src/main/java/cx/aswin/boxlore/core/rss/ShowPageFeedRules.kt b/core/rss/src/main/java/cx/aswin/boxlore/core/rss/ShowPageFeedRules.kt new file mode 100644 index 000000000..23170a049 --- /dev/null +++ b/core/rss/src/main/java/cx/aswin/boxlore/core/rss/ShowPageFeedRules.kt @@ -0,0 +1,69 @@ +package cx.aswin.boxlore.core.rss + +import cx.aswin.boxlore.core.database.LocalEpisodeFeedEntity +import cx.aswin.boxlore.core.database.LocalFeedOrder +import cx.aswin.boxlore.core.database.UnfollowedCatalogCopy +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.Scope + +/** + * The rules of a show page's stored feed copy, kept apart from I/O so they can be tested alone. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page; decision D10). + * COMPAT(own-ids): show pages read the show's own feed. Permanent: never remove (MIGRATION.md §5). + */ +object ShowPageFeedRules { + /** How many of a feed's newest items a first read takes. */ + const val NEWEST_ITEMS = 100 + + /** A stored copy is shown at once and read again with its validators once it is this old. */ + const val RECHECK_AFTER_MS = 60L * 60L * 1000L + + /** An unfollowed show's copy not opened for this long is removed. */ + const val UNUSED_AFTER_MS = 30L * 24L * 60L * 60L * 1000L + + /** At most this many unfollowed shows keep a copy… */ + const val MAX_UNFOLLOWED_SHOWS = 40 + + /** …and at most this many episodes between them. */ + const val MAX_UNFOLLOWED_EPISODES = 20_000 + + /** True when a read of the start of the feed can't give its newest items (oldest-first feeds). */ + fun readsWholeFeed(scope: Scope, cached: LocalEpisodeFeedEntity?): Boolean = scope == Scope.ARCHIVE || cached?.feedOrder == LocalFeedOrder.OLDEST_FIRST + + /** + * The validators to send with a read, or null to read without them. A copy that holds only the + * newest items has validators for the whole document, so a 304 to an archive read would only + * say the start is current: archive reads of such a copy go out without validators, and a 304 + * to them is never taken as the archive ([acceptsNotModified]). + */ + fun validators(wholeFeed: Boolean, cached: LocalEpisodeFeedEntity?): Pair? { + if (cached == null || cached.itemCount <= 0) return null + if (wholeFeed && !cached.archiveComplete) return null + if (cached.feedEtag.isNullOrBlank() && cached.feedLastModified.isNullOrBlank()) return null + return cached.feedEtag to cached.feedLastModified + } + + /** Whether a 304 answers this read: always for the newest items, for the archive only when the copy is already complete. */ + fun acceptsNotModified(wholeFeed: Boolean, cachedArchiveComplete: Boolean): Boolean = !wholeFeed || cachedArchiveComplete + + /** Copies to remove: not opened in [UNUSED_AFTER_MS], then the least recently opened over the caps. Never [keep]. */ + fun cleanupVictims(copies: List, nowMillis: Long, keep: String?): List { + val candidates = copies.filter { it.podcastId != keep } + val (stale, fresh) = candidates.partition { nowMillis - lastUse(it) > UNUSED_AFTER_MS } + val keptEpisodes = copies.filter { it.podcastId == keep }.sumOf { it.episodeCount } + var shows = copies.size - candidates.size + var episodes = keptEpisodes + val overCap = mutableListOf() + for (copy in fresh.sortedByDescending(::lastUse)) { + if (shows + 1 > MAX_UNFOLLOWED_SHOWS || episodes + copy.episodeCount > MAX_UNFOLLOWED_EPISODES) { + overCap += copy.podcastId + } else { + shows++ + episodes += copy.episodeCount + } + } + return stale.map { it.podcastId } + overCap + } + + private fun lastUse(copy: UnfollowedCatalogCopy): Long = maxOf(copy.lastOpenedAt, copy.fetchedAt) +} diff --git a/core/rss/src/test/java/cx/aswin/boxlore/core/rss/FeedPrefixReaderTest.kt b/core/rss/src/test/java/cx/aswin/boxlore/core/rss/FeedPrefixReaderTest.kt new file mode 100644 index 000000000..90a3f3ebe --- /dev/null +++ b/core/rss/src/test/java/cx/aswin/boxlore/core/rss/FeedPrefixReaderTest.kt @@ -0,0 +1,119 @@ +package cx.aswin.boxlore.core.rss + +import java.io.ByteArrayInputStream +import java.io.InputStream +import kotlinx.coroutines.test.runTest +import okhttp3.MediaType.Companion.toMediaType +import okhttp3.OkHttpClient +import okhttp3.Protocol +import okhttp3.Request +import okhttp3.Response +import okhttp3.ResponseBody.Companion.toResponseBody +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Assertions.assertFalse +import org.junit.jupiter.api.Assertions.assertNull +import org.junit.jupiter.api.Assertions.assertThrows +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.Test + +/** The streaming read of a feed's newest items stops early and leaves a document that still parses as a whole. */ +class FeedPrefixReaderTest { + @Test + fun stopsAfterTheItemLimitAndClosesTheRssDocument() { + val prefix = FeedPrefixReader.read(input(rss(150)), maxItems = 100, maxBytes = MAX) + + assertFalse(prefix.complete) + assertEquals(100, prefix.itemCount) + val text = prefix.bytes.decodeToString() + assertTrue(text.contains("g99")) + assertFalse(text.contains("g100")) + assertTrue(text.trimEnd().endsWith("\n")) + } + + @Test + fun aShortFeedIsReadWhole() { + val document = rss(30) + val prefix = FeedPrefixReader.read(input(document), maxItems = 100, maxBytes = MAX) + + assertTrue(prefix.complete) + assertEquals(30, prefix.itemCount) + assertEquals(document, prefix.bytes.decodeToString()) + } + + @Test + fun tagsSplitAcrossReadsAndAnyCaseAreFound() { + val document = rss(5).replace("", "") + val prefix = FeedPrefixReader.read(TrickleStream(document.encodeToByteArray(), chunk = 3), maxItems = 3, maxBytes = MAX) + + assertFalse(prefix.complete) + assertEquals(3, prefix.itemCount) + assertTrue(prefix.bytes.decodeToString().contains("g2")) + assertFalse(prefix.bytes.decodeToString().contains("g3")) + } + + @Test + fun atomEntriesCloseTheFeedElement() { + val entries = (0 until 4).joinToString("") { "e$it" } + val prefix = FeedPrefixReader.read(input("$entries"), maxItems = 2, maxBytes = MAX) + + assertEquals(2, prefix.itemCount) + assertTrue(prefix.bytes.decodeToString().trimEnd().endsWith("e1\n")) + } + + @Test + fun aDocumentPastTheByteCapFails() { + assertThrows(IllegalArgumentException::class.java) { + FeedPrefixReader.read(input(rss(50)), maxItems = 100, maxBytes = 200) + } + } + + @Test + fun fetchNewestSendsValidatorsTreatsA304AsCurrentAndReportsCompleteness() = runTest { + val requests = mutableListOf() + var code = 304 + val client = + RssFeedClient( + OkHttpClient.Builder().addInterceptor { chain -> + requests += chain.request() + Response.Builder().request(chain.request()).protocol(Protocol.HTTP_1_1).code(code).message("test") + .header("ETag", "whole-doc").body(rss(150).toResponseBody("application/rss+xml".toMediaType())).build() + }.build(), + ) + + assertNull(client.fetchNewest("https://example.com/feed", 100, "etag-1", null)) + assertEquals("etag-1", requests.single().header("If-None-Match")) + + code = 200 + val read = requireNotNull(client.fetchNewest("https://example.com/feed", 100)) + assertFalse(read.complete) + assertEquals("whole-doc", read.result.etag) + assertNull(requests.last().header("If-None-Match")) + assertTrue(requireNotNull(client.fetchNewest("https://example.com/feed", 500)).complete) + } + + private fun input(text: String): InputStream = ByteArrayInputStream(text.encodeToByteArray()) + + /** Hands out at most [chunk] bytes per read, like a slow connection. */ + private class TrickleStream(private val bytes: ByteArray, private val chunk: Int) : InputStream() { + private var position = 0 + + override fun read(): Int = if (position < bytes.size) bytes[position++].toInt() and 0xFF else -1 + + override fun read(buffer: ByteArray, offset: Int, length: Int): Int { + if (position >= bytes.size) return -1 + val count = minOf(chunk, length, bytes.size - position) + System.arraycopy(bytes, position, buffer, offset, count) + position += count + return count + } + } + + private companion object { + const val MAX = 25L * 1024L * 1024L + + fun rss(items: Int): String { + val body = (0 until items).joinToString("") { "T$itg$it" } + return "Show$body" + } + } +} diff --git a/core/rss/src/test/java/cx/aswin/boxlore/core/rss/LocalEpisodeCatalogRepositoryTest.kt b/core/rss/src/test/java/cx/aswin/boxlore/core/rss/LocalEpisodeCatalogRepositoryTest.kt index 7f0f4e04a..b1b4d2e88 100644 --- a/core/rss/src/test/java/cx/aswin/boxlore/core/rss/LocalEpisodeCatalogRepositoryTest.kt +++ b/core/rss/src/test/java/cx/aswin/boxlore/core/rss/LocalEpisodeCatalogRepositoryTest.kt @@ -811,6 +811,28 @@ class LocalEpisodeCatalogRepositoryTest { feeds[podcastId] = feeds.getValue(podcastId).copy(feedUrlLookupAt = atMillis) } + override suspend fun setArchiveComplete(podcastId: String, archiveComplete: Boolean,) { + feeds[podcastId]?.let { feeds[podcastId] = it.copy(archiveComplete = archiveComplete) } + } + + override suspend fun setLastOpenedAt(podcastId: String, atMillis: Long,) { + feeds[podcastId]?.let { feeds[podcastId] = it.copy(lastOpenedAt = atMillis) } + } + + override suspend fun setTtlAndFollowed(podcastId: String, ttlExpiresAt: Long?, followed: Boolean,) { + feeds[podcastId]?.let { feeds[podcastId] = it.copy(ttlExpiresAt = ttlExpiresAt, followed = followed) } + } + + override suspend fun seedFollowed(podcastId: String) = Unit + + override suspend fun searchLimited(podcastId: String, query: String, limit: Int,): List = emptyList() + + override suspend fun listUnfollowedCopies(): List = feeds.values + .filter { !it.followed && it.ttlExpiresAt == null } + .map { feed -> cx.aswin.boxlore.core.database.UnfollowedCatalogCopy(feed.podcastId, feed.lastOpenedAt, feed.fetchedAt, episodes.count { it.podcastId == feed.podcastId }) } + + override suspend fun isUnfollowedCopy(podcastId: String): Boolean = feeds[podcastId]?.let { !it.followed && it.ttlExpiresAt == null } == true + override suspend fun setEpNsIfUnset(podcastId: String, epNs: Long,) { feeds[podcastId]?.takeIf { it.epNs == null }?.let { feeds[podcastId] = it.copy(epNs = epNs) } } diff --git a/core/rss/src/test/java/cx/aswin/boxlore/core/rss/ShowPageFeedCatalogTest.kt b/core/rss/src/test/java/cx/aswin/boxlore/core/rss/ShowPageFeedCatalogTest.kt new file mode 100644 index 000000000..2a431598f --- /dev/null +++ b/core/rss/src/test/java/cx/aswin/boxlore/core/rss/ShowPageFeedCatalogTest.kt @@ -0,0 +1,279 @@ +package cx.aswin.boxlore.core.rss + +import cx.aswin.boxlore.core.database.LocalFeedOrder +import cx.aswin.boxlore.core.database.PodcastEntity +import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.ReadOutcome +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.Scope +import cx.aswin.boxlore.core.model.EpisodeRecipeId +import cx.aswin.boxlore.core.rss.ports.DownloadCacheRelinker +import java.time.Instant +import java.time.ZoneOffset +import java.time.format.DateTimeFormatter +import java.util.concurrent.CopyOnWriteArrayList +import kotlinx.coroutines.runBlocking +import okhttp3.MediaType.Companion.toMediaType +import okhttp3.OkHttpClient +import okhttp3.Protocol +import okhttp3.Request +import okhttp3.Response +import okhttp3.ResponseBody.Companion.toResponseBody +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertNotNull +import org.junit.Assert.assertNull +import org.junit.Assert.assertTrue +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.annotation.Config + +/** + * The show page's stored feed copy through the real catalog write on a real Room database, with + * feed documents served over an interceptor and parsed by the real parser: the newest 100 first, + * the archive later with the same IDs, the prefix 304 rule, oldest-first feeds, and the cleanup that + * never touches history, downloads, the queue or aliases. + */ +@RunWith(RobolectricTestRunner::class) +@Config(sdk = [34]) +class ShowPageFeedCatalogTest : RoomCatalogTestBase() { + private val requests = CopyOnWriteArrayList() + private var document: String = feedXml(150) + private var status = 200 + private var etag = ETAG + private val persisted = mutableListOf() + + private val client = + RssFeedClient( + OkHttpClient.Builder().addInterceptor { chain -> + requests += chain.request() + val answer = if (chain.request().header("If-None-Match") == etag && status == 200) 304 else status + Response.Builder().request(chain.request()).protocol(Protocol.HTTP_1_1).code(answer).message("test") + .header("ETag", etag).body(document.toResponseBody("application/rss+xml".toMediaType())).build() + }.build(), + ) + + private val catalog by lazy { + LocalEpisodeCatalogRepository(database, client, DownloadCacheRelinker { _, _ -> false }, { persisted += it }, RecipeIdHash.RECIPE) + } + + private val pages: ShowPageFeedPort get() = catalog.showPages + + @Test + fun theNewestHundredComeFirstAndTheArchiveKeepsTheirIds() = runBlocking { + val newest = pages.read(request(Scope.NEWEST)) + + assertEquals(ReadOutcome.Stored(100, complete = false), newest) + val first = rows(SHOW) + assertEquals(100, first.size) + assertEquals("new items take their recipe IDs", EpisodeRecipeId.forNamespace(SHOW.toLong(), "g0").toString(), first["g0"]) + val cached = requireNotNull(pages.cached(SHOW)) + assertFalse(cached.archiveComplete) + assertFalse("an unfollowed copy is never the live list", catalog.isReady(SHOW)) + assertTrue("a show page read schedules nothing", persisted.isEmpty()) + + val archive = pages.read(request(Scope.ARCHIVE)) + + assertEquals(ReadOutcome.Stored(150, complete = true), archive) + val all = rows(SHOW) + assertEquals(150, all.size) + for ((key, id) in first) assertEquals("$key keeps its ID", id, all[key]) + assertTrue(requireNotNull(pages.cached(SHOW)).archiveComplete) + assertFalse(catalog.isReady(SHOW)) + assertTrue(database.localEpisodeCatalogDao().getFeed(SHOW)!!.lastOpenedAt > 0L) + } + + @Test + fun a304ToTheStoredNewestIsNeverTakenAsTheArchive() = runBlocking { + pages.read(request(Scope.NEWEST)) + assertEquals("the validators of the whole document were kept", ETAG, database.localEpisodeCatalogDao().getFeed(SHOW)!!.feedEtag) + + assertEquals(ReadOutcome.NotModified, pages.read(request(Scope.NEWEST))) + assertEquals(ETAG, requests.last().header("If-None-Match")) + + val archive = pages.read(request(Scope.ARCHIVE)) + + assertNull("an archive read of a partial copy goes out without validators", requests.last().header("If-None-Match")) + assertEquals(ReadOutcome.Stored(150, complete = true), archive) + // Once the archive is whole, its validators answer an archive read too. + assertEquals(ReadOutcome.NotModified, pages.read(request(Scope.ARCHIVE))) + } + + @Test + fun theStartOfAnOldestFirstFeedIsNotItsNewestSoTheWholeFeedIsRead() = runBlocking { + document = feedXml(150, oldestFirst = true) + + assertEquals(ReadOutcome.Stored(150, complete = true), pages.read(request(Scope.NEWEST))) + assertEquals(LocalFeedOrder.OLDEST_FIRST, database.localEpisodeCatalogDao().getFeed(SHOW)!!.feedOrder) + assertTrue(requireNotNull(pages.cached(SHOW)).oldestFirst) + } + + @Test + fun aFailedReadSaysWhatItSawAndStoresNothing() = runBlocking { + status = 404 + assertEquals(ReadOutcome.Failed("http_404"), pages.read(request(Scope.NEWEST))) + assertNull(pages.cached(SHOW)) + assertEquals(ReadOutcome.Failed("other"), pages.read(request(Scope.NEWEST).copy(podcastId = "rss:abc"))) + } + + @Test + fun aTransportDocumentIsStoredWithItsEtagAndNeverRecordsAnAddress() = runBlocking { + val etags = mutableListOf() + + val stored = + pages.readTransport(request(Scope.NEWEST).copy(feedUrl = PLAIN_URL)) { etag -> + etags += etag + ShowPageFeedPort.Transported.Document(feedXml(20).encodeToByteArray(), "t-etag") + } + + assertEquals(ReadOutcome.Stored(20, complete = true), stored) + assertEquals(EpisodeRecipeId.forNamespace(SHOW.toLong(), "g1").toString(), rows(SHOW)["g1"]) + assertEquals("neither the transport nor the http address is recorded", "", database.localEpisodeCatalogDao().getFeed(SHOW)!!.feedUrl) + assertEquals(listOf(null), etags) + + // The whole copy's ETag goes with the next read, and a 304 to it keeps the copy. + val again = + pages.readTransport(request(Scope.ARCHIVE).copy(feedUrl = PLAIN_URL)) { etag -> + etags += etag + ShowPageFeedPort.Transported.NotModified + } + assertEquals(ReadOutcome.NotModified, again) + assertEquals("t-etag", etags.last()) + } + + @Test + fun aTransportThatSaysTheAddressIsHttpsReadsItDirectly() = runBlocking { + val outcome = pages.readTransport(request(Scope.NEWEST).copy(feedUrl = PLAIN_URL)) { ShowPageFeedPort.Transported.ReadDirect(feedUrl(SHOW)) } + + assertEquals(ReadOutcome.Stored(100, complete = false), outcome) + assertEquals(feedUrl(SHOW), requests.single().url.toString()) + assertEquals( + ReadOutcome.Failed("http_503"), + pages.readTransport(request(Scope.NEWEST)) { ShowPageFeedPort.Transported.Failed("http_503") }, + ) + } + + @Test + fun aShowPageReadKeepsTheUnsubscribeTtlSoAResubscribeStaysCheap() = runBlocking { + // A followed show's complete catalog, then unfollowed: TTL set, ready kept for a resubscribe. + follow(SHOW) + assertTrue(catalog.refresh(LocalEpisodeCatalogPort.RefreshRequest(SHOW, feedUrl(SHOW))) is LocalEpisodeCatalogPort.RefreshOutcome.Success) + assertTrue(catalog.isReady(SHOW)) + val ttl = System.currentTimeMillis() + 14L * DAY + catalog.setUnsubscribedTtl(SHOW, ttl) + assertFalse(catalog.isReady(SHOW)) + + pages.read(request(Scope.NEWEST)) + + val feed = database.localEpisodeCatalogDao().getFeed(SHOW)!! + assertEquals("the TTL sweep still owns the copy", ttl, feed.ttlExpiresAt) + assertTrue("ready is kept for the resubscribe", feed.ready) + assertFalse("still not the live list while unfollowed", catalog.isReady(SHOW)) + catalog.setUnsubscribedTtl(SHOW, null) + assertTrue("resubscribing uses the stored catalog at once", catalog.isReady(SHOW)) + } + + @Test + fun aNewestReadWithNoStoredItemLeavesTheArchiveIncomplete() = runBlocking { + document = feedXml(150) + pages.read(request(Scope.ARCHIVE)) + assertTrue(requireNotNull(pages.cached(SHOW)).archiveComplete) + + // More than a newest read holds came out since: none of the 100 items read is stored. + document = feedXml(150, firstIndex = -120) + etag = "\"v2\"" + assertEquals(ReadOutcome.Stored(250, complete = false), pages.read(request(Scope.NEWEST))) + + assertFalse("a gap below the new items", requireNotNull(pages.cached(SHOW)).archiveComplete) + pages.read(request(Scope.ARCHIVE)) + assertNull("the archive read goes out without validators", requests.last().header("If-None-Match")) + + // A newest read that meets a stored item leaves a whole archive whole. + document = feedXml(150, firstIndex = -125) + etag = "\"v3\"" + pages.read(request(Scope.NEWEST)) + assertTrue(requireNotNull(pages.cached(SHOW)).archiveComplete) + } + + @Test + fun aShowFollowedBeforeItsFirstReadGetsItsFollowedColumnFromTheFollow() = runBlocking { + follow(SHOW) + assertNull(database.localEpisodeCatalogDao().getFeed(SHOW)) + + pages.read(request(Scope.NEWEST)) + + assertTrue(database.localEpisodeCatalogDao().getFeed(SHOW)!!.followed) + pages.read(request(Scope.NEWEST, show = OTHER_SHOW)) + assertFalse(database.localEpisodeCatalogDao().getFeed(OTHER_SHOW)!!.followed) + } + + @Test + fun inShowSearchIsLimitedInTheQuery() = runBlocking { + pages.read(request(Scope.ARCHIVE)) + + val found = pages.search(SHOW, "Episode", limit = 7) + + assertEquals(7, found.size) + assertEquals("Episode 0", found.first().title) + assertTrue(pages.search(SHOW, " ", limit = 7).isEmpty()) + } + + @Test + fun cleanupRemovesOnlyUnfollowedCopiesAndKeepsListenerState() = runBlocking { + document = feedXml(5) + pages.read(request(Scope.ARCHIVE)) + pages.read(request(Scope.ARCHIVE, show = OTHER_SHOW)) + pages.read(request(Scope.ARCHIVE, show = THIRD_SHOW)) + follow(THIRD_SHOW) + val played = rows(SHOW).getValue("g1") + history(played, SHOW) + completedDownload(played, SHOW, "Ep 1") + queue(played, SHOW, "https://cdn.example/g1.mp3", "Ep 1") + val aliasesBefore = database.episodeIdentityDao().aliasesForShow(SHOW).size + + val removed = pages.cleanUp(nowMillis = System.currentTimeMillis() + 31L * DAY, keepPodcastId = OTHER_SHOW) + + assertEquals(1, removed) + assertNull(pages.cached(SHOW)) + assertNotNull("the show being opened stays", pages.cached(OTHER_SHOW)) + assertNotNull("a followed show's catalog stays", pages.cached(THIRD_SHOW)) + assertNotNull(database.listeningHistoryDao().getHistoryItem(played)) + assertNotNull(database.downloadedEpisodeDao().getDownload(played)) + assertNotNull(database.queueDao().getQueueItemByEpisodeId(played)) + assertEquals("aliases outlive the copy", aliasesBefore, database.episodeIdentityDao().aliasesForShow(SHOW).size) + } + + @Test + fun followAndUnfollowWriteTheFollowedColumn() = runBlocking { + pages.read(request(Scope.NEWEST)) + catalog.setUnsubscribedTtl(SHOW, null) + assertTrue(database.localEpisodeCatalogDao().getFeed(SHOW)!!.followed) + catalog.setUnsubscribedTtl(SHOW, 5L) + assertFalse(database.localEpisodeCatalogDao().getFeed(SHOW)!!.followed) + database.podcastDao().upsert(PodcastEntity(podcastId = SHOW, title = "Show", author = "", imageUrl = "", description = null, isSubscribed = false)) + assertEquals("a copy under an unsubscribe TTL is left to the TTL sweep", 0, pages.cleanUp(Long.MAX_VALUE / 2)) + } + + private fun request(scope: Scope, show: String = SHOW) = ShowPageFeedPort.ReadRequest(show, feedUrl(show), scope, LocalEpisodeCatalogPort.PodcastMeta(title = "Show")) + + private companion object { + const val ETAG = "\"whole-doc\"" + const val DAY = 24L * 60L * 60L * 1000L + + const val PLAIN_URL = "http://plain.example/920666.xml" + + /** [count] items dated a day apart from [firstIndex] (negative: newer), newest first unless [oldestFirst]. */ + fun feedXml(count: Int, oldestFirst: Boolean = false, firstIndex: Int = 0): String { + val last = firstIndex + count - 1 + val order = if (oldestFirst) (last downTo firstIndex) else (firstIndex..last) + val items = + order.joinToString("") { index -> + val date = DateTimeFormatter.RFC_1123_DATE_TIME.format(Instant.ofEpochSecond(1_790_000_000L - index * 86_400L).atZone(ZoneOffset.UTC)) + "Episode $indexg$index$date" + + "" + } + return "Show$items" + } + } +} diff --git a/core/rss/src/test/java/cx/aswin/boxlore/core/rss/ShowPageFeedRulesTest.kt b/core/rss/src/test/java/cx/aswin/boxlore/core/rss/ShowPageFeedRulesTest.kt new file mode 100644 index 000000000..afd068681 --- /dev/null +++ b/core/rss/src/test/java/cx/aswin/boxlore/core/rss/ShowPageFeedRulesTest.kt @@ -0,0 +1,101 @@ +package cx.aswin.boxlore.core.rss + +import cx.aswin.boxlore.core.database.LocalEpisodeFeedEntity +import cx.aswin.boxlore.core.database.LocalFeedOrder +import cx.aswin.boxlore.core.database.UnfollowedCatalogCopy +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.Scope +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Assertions.assertFalse +import org.junit.jupiter.api.Assertions.assertNull +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.Test + +class ShowPageFeedRulesTest { + @Test + fun aCopyOfTheNewestItemsNeverAnswersAnArchiveReadWithA304() { + val partial = copy(archiveComplete = false) + + assertEquals("etag" to "lm", ShowPageFeedRules.validators(wholeFeed = false, cached = partial)) + assertNull(ShowPageFeedRules.validators(wholeFeed = true, cached = partial), "an archive read of a partial copy goes out unconditional") + assertFalse(ShowPageFeedRules.acceptsNotModified(wholeFeed = true, cachedArchiveComplete = false)) + assertTrue(ShowPageFeedRules.acceptsNotModified(wholeFeed = false, cachedArchiveComplete = false)) + } + + @Test + fun aCompleteCopySendsItsValidatorsForEitherRead() { + val complete = copy(archiveComplete = true) + + assertEquals("etag" to "lm", ShowPageFeedRules.validators(wholeFeed = true, cached = complete)) + assertTrue(ShowPageFeedRules.acceptsNotModified(wholeFeed = true, cachedArchiveComplete = true)) + } + + @Test + fun noCopyOrNoValidatorsReadsUnconditionally() { + assertNull(ShowPageFeedRules.validators(wholeFeed = false, cached = null)) + assertNull(ShowPageFeedRules.validators(wholeFeed = false, cached = copy(itemCount = 0))) + assertNull(ShowPageFeedRules.validators(wholeFeed = false, cached = copy(etag = null, lastModified = null))) + } + + @Test + fun anOldestFirstFeedIsAlwaysReadWhole() { + assertTrue(ShowPageFeedRules.readsWholeFeed(Scope.NEWEST, copy(order = LocalFeedOrder.OLDEST_FIRST))) + assertFalse(ShowPageFeedRules.readsWholeFeed(Scope.NEWEST, copy(order = LocalFeedOrder.NEWEST_FIRST))) + assertTrue(ShowPageFeedRules.readsWholeFeed(Scope.ARCHIVE, null)) + } + + @Test + fun cleanupRemovesCopiesUnusedForThirtyDaysButNeverTheOneOpening() { + val now = 100L * DAY + val copies = + listOf( + UnfollowedCatalogCopy("old", lastOpenedAt = now - 31 * DAY, fetchedAt = now - 40 * DAY, episodeCount = 10), + UnfollowedCatalogCopy("recent", lastOpenedAt = now - 2 * DAY, fetchedAt = now - 2 * DAY, episodeCount = 10), + UnfollowedCatalogCopy("refetched", lastOpenedAt = 0L, fetchedAt = now - DAY, episodeCount = 10), + UnfollowedCatalogCopy("opening", lastOpenedAt = now - 60 * DAY, fetchedAt = now - 60 * DAY, episodeCount = 10), + ) + + assertEquals(listOf("old"), ShowPageFeedRules.cleanupVictims(copies, now, keep = "opening")) + } + + @Test + fun cleanupKeepsTheMostRecentlyOpenedCopiesUnderTheCaps() { + val now = 100L * DAY + val many = (0 until ShowPageFeedRules.MAX_UNFOLLOWED_SHOWS + 3).map { UnfollowedCatalogCopy("s$it", now - it * 1_000L, 0L, 1) } + assertEquals(listOf("s40", "s41", "s42"), ShowPageFeedRules.cleanupVictims(many, now, keep = null)) + + val big = + listOf( + UnfollowedCatalogCopy("newest", now - 1, 0L, 15_000), + UnfollowedCatalogCopy("middle", now - 2, 0L, 6_000), + UnfollowedCatalogCopy("small", now - 3, 0L, 4_000), + ) + // 15,000 + 6,000 is over the 20,000 episode cap; the next copy that fits still stays. + assertEquals(listOf("middle"), ShowPageFeedRules.cleanupVictims(big, now, keep = null)) + } + + private fun copy( + archiveComplete: Boolean = false, + itemCount: Int = 100, + etag: String? = "etag", + lastModified: String? = "lm", + order: String = LocalFeedOrder.NEWEST_FIRST, + ) = LocalEpisodeFeedEntity( + podcastId = "920666", + feedUrl = "https://feeds.example/920666.xml", + feedEtag = etag, + feedLastModified = lastModified, + fetchedAt = 1L, + itemCount = itemCount, + feedOrder = order, + ttlExpiresAt = null, + needsFullBackfill = !archiveComplete, + copiedExtrasCount = 0, + ready = false, + feedUrlLookupAt = 0L, + archiveComplete = archiveComplete, + ) + + private companion object { + const val DAY = 24L * 60L * 60L * 1000L + } +} diff --git a/feature/info/README.md b/feature/info/README.md index d46580456..82d2f83a0 100644 --- a/feature/info/README.md +++ b/feature/info/README.md @@ -29,6 +29,8 @@ Disclosure recognizes canonical RSS IDs even with stale source metadata and uses - Detail UI uses centralized Google Sans Flex weight tokens from `:core:designsystem`. - `EpisodeInfoSeekLogic` builds the progress-save payload when seeking an episode that is not the current player item. - For PI-owned shows whose local catalog is **not ready**, unsubscribed Podcast Info may still show **Missing episodes?** (opt-in extras path). The confirm dialog explains that publisher-feed access is allowed once and refreshes on future visits, while subscribing automatically keeps the latest episodes up to date. Subscribe kicks `SubscriptionForegroundSync.requestCatalogIngest` on the application scope so leaving Podcast Info does not cancel the publisher-feed persist (no PI-vs-feed extras compare). The screen only waits on `catalogIngestFinished` to remount if it is still open. Unsubscribe stamps a 14-day Room TTL; that catalog is **not** ready, so the page uses Podcast Index again (a new PI episode still appears). Resubscribe clears the TTL. Once `LocalEpisodeCatalogPort.isReady` is true, the pill is hidden and the episode list comes from Room only. Successful publisher-feed refreshes treat that feed's newest row as authoritative, including same-date/different-id and older-than-PI cross-promo cases. Pull-to-refresh: true RSS uses `refreshCatalog`; subscribed non-pure RSS shows route to a direct-feed catalog refresh via `LocalEpisodeCatalogPort.refresh` (preserving notification, auto-download, skip overrides, artwork fallback, and sort preferences); not-ready opted-in unsubscribed PI still runs the extras refresh; other unsubscribed PI shows reload from `PodcastRepository` (Room or PI). NEW badges still follow newer `publishedDate` only. +- **Feed-first show pages (P4 PR 16, behind `feed_first`).** When `PodcastRepository.showPageSource` is `FEED`, `PodcastInfoFeedFirst` runs the page: a stored copy shows at once (status `CACHED`, no line); without one, boxlore's list of the show shows while the feed is read, under the same IDs, so the list then grows without moving. Opening reads the newest episodes when the copy is missing or an hour old (`READING_NEWEST`: "Checking the feed for new episodes…"), and the whole archive for an oldest-first page (serial shows) or an oldest-first feed (`COMPLETING_ARCHIVE`: "Loading all episodes from the feed…"). Search, switching to oldest first and scrolling past the stored newest episodes read the archive first; a search the listener keeps typing into waits on the same read. Opening a serial show reads its whole archive even when the page opens newest first. When the feed can't be read the list becomes boxlore's list of the show with `FALLBACK` ("Can’t reach this show’s feed. Showing boxlore’s episode list instead."), and search, sorting and scrolling read no feed again until the page is opened again (`readsFeed`); pull to refresh still tries it. Offline keeps the stored copy with `OFFLINE` ("You’re offline. Showing saved episodes."), or with no copy `OFFLINE_NO_COPY` ("You’re offline. Connect to load this show’s episodes."); scrolling past the copy while offline waits for a connection instead of retrying, and the list stays open for it. The line (`ShowFeedStatusLine`, a solid `surfaceContainerHigh` card, polite live region) sits above the episodes. After each read the page reloads as many episodes as showed (plus a page when scrolling on), and on until the episode that was last is back when newer ones came in above, so rows keep their IDs and the scroll position and selection stay while the list grows or switches source. The show page reads its list through `PodcastRepository.getShowPageEpisodes`. Pull-to-refresh on a show that isn't followed reads the newest episodes at the registry's address (`Target.FEED_FIRST`); once followed, the show's own catalog refresh takes over and the page stops reading the feed (`PodcastInfoFeedFirst.stop`). A page that opens tries the feed again after an earlier fallback. These pages never offer **Missing episodes?**, also after a fallback (`isFeedFirstShowPage`): the chip and its dialog stay hidden. Search over the feed's copy skips the feed extras; after a fallback it searches the server's list with them. Decisions live in `logic/ShowFeedLogic`. With `feed_first` off, a show that isn't followed lists `/v2/episodes` and in-show search filters that list on the phone; v1 `episodes/search` is no longer used. +- `loadMoreEpisodes` and `toggleSort` rethrow `CancellationException` instead of swallowing it. - Show auto-download can be enabled with notifications off or system notification permission denied. Turning the bell off preserves auto-download; the cloud-dirty preference and activation boundary are persisted by the catalog repository. Pure RSS auto-download eligibility remains unchanged. @@ -92,7 +94,9 @@ src/main/java/cx/aswin/boxlore/feature/info/ - Tablet detail content is centered in the shared Reading width (720dp); the app palette still paints the full viewport. Poster rails keep fixed card sizes and Podroll rail items use the shared RailItem cap. Phone modifiers return unchanged. - `PodcastInfoAdaptiveTransitionTest` drives one live `PodcastInfoScreen` (Robolectric/Compose JVM, mocked `PodcastInfoViewModel`) across 599↔600dp width, 479↔480dp height, 839↔840dp in landscape, portrait↔landscape and a flat↔tabletop fold, in both directions, and recreates it with `StateRestorationTester` while the window changes, which is also what a back-stack return does. It covers search query, results and scroll anchor, and the query run again after process death; sort, hide-completed and the visible refresh indicator (no reload or refresh from a window change); selection, including a full-range selection rebuilt after process death, an oldest-first selection that waits for the reloaded page and is looked up in its own order, actions held while the lookup runs, a failed lookup that keeps every ID and retries, a lookup cut short by a reload, Clear during a lookup, and IDs dropped when the page can no longer find them; each of the five dialogs and sheets on both the compact and the tablet host, keeping its content across window changes and recreation, with confirm or save, cancel, Back and outside tap each running its action once or not at all; a restored Missing episodes? dialog that waits for the offer; and the loading indicator, error, retry and empty states, also after recreation. The episode list, loading state and refresh indicator carry test tags. `PodcastEpisodeSelectionLogicTest` covers the restore lookup and its gate; `PodcastInfoLoadGuardTest` covers the repeat-load check for `rss:`-linked shows and catalog IDs that differ from the route. +- `PodcastInfoAdaptiveTransitionTest` also covers the feed status line in each state across every window gate and recreation, and a list that grows (older episodes below, newer ones above, the archive read) and switches source (stored copy → server's list → back) in the same frame as each window change, keeping the scrolled-to row, the selection and the episodes it plays. `ShowFeedLogicTest` covers read depth (serial shows too), the status line (offline with and without a copy), archive gating, reload size and the reload anchor, keeping the list open past the copy, show details on stored-copy episodes and the `FEED_FIRST` pull target. `PodcastInfoFeedFirstTest` drives `PodcastInfoFeedFirst` against a scripted source: no feed read after a fallback until the page opens again, newer episodes above keeping the last row, offline without a copy, the copy's end waiting for a connection, concurrent searches sharing one archive read, a read that ends after the page moved on, a serial show's opening read, and following stopping the page. - **Manual acceptance (show page).** Open a show with many episodes. (1) Phone: tap Search, type a word, scroll the results, rotate twice; the field, query, results and scroll stay. (2) Long-press an episode, choose Select all, rotate; the count stays and Play acts on the same episodes. (3) Open each overflow item (Mark all as played, Mark all as unplayed, Playback for this show, Change tag / genre with a typed draft) and the Missing episodes? chip on an unfollowed show; rotate, then confirm, press Back or tap outside, and check each acts once. (4) Tablet or desktop window: repeat (1) to (3) while resizing across 600dp and 840dp wide and 480dp tall, and in split-screen. (5) Foldable: repeat while folding to tabletop and unfolding, which recreates the activity. (6) Developer options → Don't keep activities: leave and return; open search with its query, dialogs and selection come back. (7) Repeat (1), (2) and (6) on a show you follow through its feed, and on a serial show sorted oldest first. Device runs are not part of CI. +- **Manual acceptance (feed-first show page).** Turn `feed_first` on for the device (PostHog), restart the app, then open a show you don't follow. (1) Phone: the line "Checking the feed for new episodes…" shows, then disappears as the list fills; scroll to the end of the first 100 and "Loading all episodes from the feed…" shows while older episodes appear below without the list jumping. (2) Long-press an episode, then rotate and resize split-screen while the archive loads: the selection count and scroll stay. (3) Tap Search and type a word on a show opened for the first time: the archive line shows, then the results. (4) Sort oldest first on a long show: the oldest episode appears after the archive line. (5) Open the show again within an hour: the saved list shows with no line; pull to refresh reads the feed. (6) Airplane mode, reopen: the saved list shows with "You’re offline. Showing saved episodes."; a show never opened before says "You’re offline. Connect to load this show’s episodes." (7) A show whose feed is down: after a moment the line says "Can’t reach this show’s feed. Showing boxlore’s episode list instead." and episodes still play. Repeat (1)–(2) on a tablet (landscape split at 840dp) and a foldable (flat ↔ tabletop). With `feed_first` off, the same show lists its episodes from boxlore's list, and search still finds them. - `InfoPhoneGoldenTest` captures the actual podcast and episode screens at both portrait phone sizes and short landscape in light/dark. Mocked ViewModel flows avoid constructing Application-backed ViewModels or making network requests. These local-only captures use `-PphoneGoldens=verify --tests '*PhoneGoldenTest'`; see [capture instructions](../../docs/screenshots/README.md). diff --git a/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoEpisodeList.kt b/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoEpisodeList.kt index fdf7c6287..6f22a75df 100644 --- a/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoEpisodeList.kt +++ b/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoEpisodeList.kt @@ -36,7 +36,9 @@ import cx.aswin.boxlore.feature.info.components.EpisodeListIndicators import cx.aswin.boxlore.feature.info.components.EpisodeSelectionUi import cx.aswin.boxlore.feature.info.components.EpisodeToolbar import cx.aswin.boxlore.feature.info.components.PodcastInfoEpisodeSearch +import cx.aswin.boxlore.feature.info.components.ShowFeedStatusLine import cx.aswin.boxlore.feature.info.components.ToolbarWarningBanner +import cx.aswin.boxlore.feature.info.components.showFeedStatusText import cx.aswin.boxlore.feature.info.logic.FeedItem import cx.aswin.boxlore.feature.info.logic.ToolbarWarning import cx.aswin.boxlore.feature.info.logic.podcastSearchHasNoResults @@ -156,6 +158,8 @@ internal fun PodcastInfoEpisodeList( onAction = callbacks.onWarningAction, ) + feedStatusItem(contentState.state.feedStatus) + episodesSection( contentState = contentState, callbacks = callbacks, @@ -190,6 +194,15 @@ internal fun PodcastInfoEpisodeList( } } +/** Feed-first show pages (P4 PR 16): reading the feed, the server's list instead, or offline. */ +private fun LazyListScope.feedStatusItem(status: ShowFeedStatus) { + if (showFeedStatusText(status) != null) { + item(key = "feed_status") { + ShowFeedStatusLine(status = status, modifier = Modifier.animateItem()) + } + } +} + private fun LazyListScope.toolbarWarningItem( warning: ToolbarWarning, onDismiss: () -> Unit, diff --git a/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoFeedFirst.kt b/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoFeedFirst.kt new file mode 100644 index 000000000..2e7d55d2e --- /dev/null +++ b/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoFeedFirst.kt @@ -0,0 +1,205 @@ +package cx.aswin.boxlore.feature.info + +import cx.aswin.boxlore.core.catalog.PodcastRepository +import cx.aswin.boxlore.core.catalog.ShowPageFeedRead +import cx.aswin.boxlore.core.catalog.ShowPageFeedResult +import cx.aswin.boxlore.core.catalog.ShowPageFeedState +import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort.PodcastMeta +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.Scope +import cx.aswin.boxlore.core.model.Episode +import cx.aswin.boxlore.core.model.Podcast +import cx.aswin.boxlore.feature.info.logic.ShowFeedLogic +import kotlinx.coroutines.CoroutineScope +import kotlinx.coroutines.Deferred +import kotlinx.coroutines.async + +/** + * The feed-first part of [PodcastInfoViewModel]: which show reads its feed, the reads (with the + * status line) and the page reloads after them. The rows keep their IDs, so a reload grows or + * switches the list (stored copy → feed → server's list) without moving the scroll position or + * dropping a selection. + * + * After a read falls back to the server's list, this page stops reading the feed for search, + * sorting and scrolling until it is opened again ([readsFeed]); pull to refresh still tries it. + * Following the show hands the page to the show's own catalog refresh ([stop]). + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page). + * COMPAT(own-ids): show pages read the show's own feed. Permanent: never remove (MIGRATION.md §5). + */ +internal class PodcastInfoFeedFirst( + private val source: Source, + private val ui: Ui, +) { + /** What the page reads through, so tests can script it ([sourceOf] in the app). */ + interface Source { + suspend fun open(podcastId: String, meta: PodcastMeta): ShowPageFeedState? + + suspend fun read(podcastId: String, meta: PodcastMeta, scope: Scope, savedFeedUrl: String?): ShowPageFeedRead + + suspend fun page(podcastId: String, limit: Int, offset: Int, sort: String): PodcastRepository.EpisodePage + + fun online(): Boolean + } + + /** How the helper reads and writes the page it serves. */ + interface Ui { + fun state(): PodcastInfoUiState.Success? + + fun set(state: PodcastInfoUiState.Success) + + fun currentPodcastId(): String + + fun setOffset(offset: Int) + + suspend fun remount(state: PodcastInfoUiState.Success, episodes: List, hasMore: Boolean): PodcastInfoUiState.Success + } + + /** The show whose page reads its feed; null when the page reads another source. */ + @Volatile + var podcastId: String? = null + private set + + @Volatile + var archiveComplete: Boolean = false + private set + + /** The last read of this page fell back to the server's list. */ + @Volatile + private var fellBack: Boolean = false + + /** Scrolling reached the end of the copy while offline: it waits for a connection. */ + @Volatile + private var waitingAtCopyEnd: Boolean = false + + @Volatile + private var archiveRead: Deferred? = null + + fun isActive(id: String?): Boolean = id != null && id == podcastId + + /** True while [id]'s page reads its feed: it is feed-first and its last read didn't fall back. */ + fun readsFeed(id: String?): Boolean = isActive(id) && !fellBack + + /** Whether [podcastId]'s page reads the feed, and what it holds; null for any other source. */ + suspend fun open(podcastId: String, podcast: Podcast?): ShowPageFeedState? { + val opened = source.open(podcastId, podcast?.let(ShowFeedLogic::metaOf) ?: PodcastMeta()) + this.podcastId = podcastId.takeIf { opened != null } + fellBack = false + waitingAtCopyEnd = false + archiveRead = null + if (opened == null) return null + archiveComplete = opened.archiveComplete + return opened + } + + /** The listener followed the show: its own catalog refresh takes over, and this page reads no more. */ + fun stop() { + podcastId = null + archiveRead = null + } + + /** The status line and the read a page starts with once its first page shows. */ + suspend fun begin(podcast: Podcast, sort: EpisodeSort, opened: ShowPageFeedState) { + val scope = ShowFeedLogic.openingScope(sort, opened, podcast.type) + if (!ShowFeedLogic.readsOnOpen(opened, scope)) { + setStatus(podcast.id, ShowFeedLogic.openingStatus(opened)) + return + } + read(podcast, scope) + } + + /** + * Reads [podcast]'s feed [scope] deep, then reloads as many episodes as were showing plus + * [extra]. [showStatus] false keeps the status line out of a pull-to-refresh, which has its own + * indicator. Null when the page moved to another show meanwhile. + */ + suspend fun read(podcast: Podcast, scope: Scope, showStatus: Boolean = true, extra: Int = 0): ShowPageFeedRead? { + if (showStatus) setStatus(podcast.id, ShowFeedLogic.readingStatus(scope)) + val read = source.read(podcast.id, ShowFeedLogic.metaOf(podcast), scope, podcast.feedUrl) + if (ui.currentPodcastId() != podcast.id) return null + archiveComplete = read.archiveComplete + when (read.result) { + ShowPageFeedResult.FALLBACK -> fellBack = true + ShowPageFeedResult.UPDATED, ShowPageFeedResult.UNCHANGED -> { + fellBack = false + waitingAtCopyEnd = false + } + ShowPageFeedResult.OFFLINE -> Unit + } + reload(podcast, extra) + setStatus(podcast.id, ShowFeedLogic.statusAfter(read)) + return read + } + + /** + * Reads the rest of the feed before an action that needs every episode, when the copy holds + * only the newest. The read runs in [scope], so a search the listener keeps typing into waits + * on the same read instead of starting it again. Nothing is read once the page fell back: the + * action uses the server's list until the page opens again. [atCopyEnd]: scrolling reached the + * end of the copy, which then waits for a connection when the read finds the phone offline. + */ + suspend fun completeArchive(scope: CoroutineScope, podcast: Podcast, extra: Int = 0, atCopyEnd: Boolean = false): ShowPageFeedRead? { + if (archiveComplete || fellBack) return null + val running = archiveRead?.takeIf { it.isActive } ?: scope.async { read(podcast, Scope.ARCHIVE, extra = extra) }.also { archiveRead = it } + val read = running.await() + if (atCopyEnd && read?.result == ShowPageFeedResult.OFFLINE) waitingAtCopyEnd = true + return read + } + + /** + * True while scrolling past the copy waits for a connection: loading more does nothing until + * the phone is online again, so the end of the list doesn't keep retrying. + */ + fun holdsAtCopyEnd(): Boolean = waitingAtCopyEnd && !source.online() + + /** + * Pages from the start until the list holds as many episodes as it showed, plus [extra], and + * down to the episode that was last ([ShowFeedLogic.keepsLoading]), so newer episodes added + * above never push the rows the listener was on out of the list. + */ + suspend fun reload(podcast: Podcast, extra: Int = 0) { + val state = ui.state()?.takeIf { it.podcast.id == podcast.id } ?: return + val oldest = state.currentSort == EpisodeSort.OLDEST + val pageSize = if (oldest) OLDEST_PAGE_SIZE else PAGE_SIZE + val wanted = ShowFeedLogic.reloadCount(state.piEpisodes.size, pageSize, extra) + val anchorId = state.piEpisodes.lastOrNull()?.id + val episodes = mutableListOf() + var offset = 0 + var more = true + var hasMore = true + while (more && ShowFeedLogic.keepsLoading(episodes, wanted, anchorId)) { + val page = source.page(podcast.id, minOf(MAX_PAGE, wanted - episodes.size).coerceAtLeast(MIN_PAGE), offset, if (oldest) "oldest" else "newest") + episodes += page.episodes + offset += page.sourceCount + more = page.hasMore && page.sourceCount > 0 + hasMore = ShowFeedLogic.hasMoreAfterPage(page.hasMore, page.sourceCount, archivePending = readsFeed(podcast.id) && !archiveComplete) + } + val latest = ui.state()?.takeIf { it.podcast.id == podcast.id && it.currentSort == state.currentSort } ?: return + ui.setOffset(offset) + ui.set(ui.remount(latest, ShowFeedLogic.withShow(episodes, latest.podcast), hasMore).copy(isLoadingMore = false)) + } + + private fun setStatus(podcastId: String, status: ShowFeedStatus) { + val state = ui.state()?.takeIf { it.podcast.id == podcastId } ?: return + if (state.feedStatus != status) ui.set(state.copy(feedStatus = status)) + } + + companion object { + const val PAGE_SIZE = 20 + const val OLDEST_PAGE_SIZE = 50 + private const val MAX_PAGE = 100 + private const val MIN_PAGE = 20 + + /** The app's [Source]: the repository's feed-first calls and the show page's episode list. */ + fun sourceOf(repository: PodcastRepository): Source = object : Source { + override suspend fun open(podcastId: String, meta: PodcastMeta): ShowPageFeedState? = repository.openFeedShowPage(podcastId, meta) + + override suspend fun read(podcastId: String, meta: PodcastMeta, scope: Scope, savedFeedUrl: String?): ShowPageFeedRead = + repository.readShowPageFeed(podcastId, meta, scope, savedFeedUrl) + + override suspend fun page(podcastId: String, limit: Int, offset: Int, sort: String): PodcastRepository.EpisodePage = + repository.getShowPageEpisodes(podcastId, limit, offset, sort) + + override fun online(): Boolean = repository.feedFirst.online() + } + } +} diff --git a/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoSupplementSupport.kt b/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoSupplementSupport.kt index e369ad05b..9cd01a2a0 100644 --- a/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoSupplementSupport.kt +++ b/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoSupplementSupport.kt @@ -52,10 +52,14 @@ internal class PodcastInfoSupplementSupport( ) : this( episodeSupplementPort, { id, limit, offset, sort -> - repository.getEpisodesPaginated(id, limit, offset, sort) + repository.getShowPageEpisodes(id, limit, offset, sort) }, { id -> repository.loadPiEpisodesForSupplement(id) }, - { id -> repository.localEpisodeCatalog?.isReady(id) == true }, + // A feed-first page reads the feed itself (pull to refresh), so it never offers "Missing + // episodes?", also after it fell back to the server's list (P4 PR 16). + { id -> + repository.localEpisodeCatalog?.isReady(id) == true || repository.isFeedFirstShowPage(id) + }, ) data class MissingEpisodesRefresh( diff --git a/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoUiModels.kt b/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoUiModels.kt index 70c1538ce..ab18405f5 100644 --- a/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoUiModels.kt +++ b/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoUiModels.kt @@ -21,6 +21,34 @@ enum class DirectFeedChipState { Updated, } +/** + * The status line of a feed-first show page, above its episodes. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page). + */ +enum class ShowFeedStatus { + /** Nothing to say: the list is current, or the page doesn't read the feed. */ + NONE, + + /** The phone's stored copy shows; it is recent, so the feed isn't read again yet. No line. */ + CACHED, + + /** The newest episodes are being read from the feed. */ + READING_NEWEST, + + /** Every episode is being read from the feed (search, oldest first, scrolling on). */ + COMPLETING_ARCHIVE, + + /** The feed couldn't be read: the list is boxlore's list of the show. */ + FALLBACK, + + /** No connection: the list is the stored copy, as it is. */ + OFFLINE, + + /** No connection and no stored copy: nothing saved to show yet. */ + OFFLINE_NO_COPY, +} + @Immutable sealed interface PodcastInfoUiState { data object Loading : PodcastInfoUiState @@ -41,6 +69,8 @@ sealed interface PodcastInfoUiState { val directFeedChip: DirectFeedChipState = DirectFeedChipState.Hidden, /** One-shot toast/snackbar text; cleared via [PodcastInfoViewModel.consumeUserMessage]. */ val userMessage: String? = null, + /** The feed-first show page's status line (P4 PR 16). */ + val feedStatus: ShowFeedStatus = ShowFeedStatus.NONE, ) : PodcastInfoUiState data object Error : PodcastInfoUiState diff --git a/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoViewModel.kt b/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoViewModel.kt index bacbda09d..862856c45 100644 --- a/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoViewModel.kt +++ b/feature/info/src/main/java/cx/aswin/boxlore/feature/info/PodcastInfoViewModel.kt @@ -24,6 +24,7 @@ import cx.aswin.boxlore.core.prefs.HomePinnedShows import cx.aswin.boxlore.feature.info.logic.PodcastInfoAsyncResultLogic import cx.aswin.boxlore.feature.info.logic.PodcastInfoFolderSyncLogic import cx.aswin.boxlore.feature.info.logic.PodcastInfoPullRefreshLogic +import cx.aswin.boxlore.feature.info.logic.ShowFeedLogic import cx.aswin.boxlore.feature.info.logic.enablePodcastNotifications import cx.aswin.boxlore.feature.info.logic.podcastSearchResponseIsCurrent import cx.aswin.boxlore.feature.info.logic.togglePodcastAutoDownload @@ -77,6 +78,28 @@ class PodcastInfoViewModel( private val scrollDepth = routeArgs.scrollDepth private val searchQuery = routeArgs.searchQuery private val supplementSupport = PodcastInfoSupplementSupport(repository, episodeSupplementPort) + + /** The feed-first show page (P4 PR 16): reads, the status line and reloads as the list grows. */ + private val feedFirst = + PodcastInfoFeedFirst( + PodcastInfoFeedFirst.sourceOf(repository), + object : PodcastInfoFeedFirst.Ui { + override fun state(): PodcastInfoUiState.Success? = _uiState.value as? PodcastInfoUiState.Success + + override fun set(state: PodcastInfoUiState.Success) { + _uiState.value = state + } + + override fun currentPodcastId(): String = currentPodcastId + + override fun setOffset(offset: Int) { + currentOffset = offset + } + + override suspend fun remount(state: PodcastInfoUiState.Success, episodes: List, hasMore: Boolean): PodcastInfoUiState.Success = + supplementSupport.remountWithSupplements(state = state, piEpisodes = episodes, hasMoreEpisodes = hasMore, isLoadingMore = false) + }, + ) private val customGenreMutex = Mutex() private val _uiState = MutableStateFlow(PodcastInfoUiState.Loading) @@ -467,7 +490,7 @@ class PodcastInfoViewModel( } val sortParam = if (newestFirst) "newest" else "oldest" val page = - repository.getEpisodesPaginated( + repository.getShowPageEpisodes( feedId = currentPodcastId, limit = MAX_SELECTION_EPISODES, offset = 0, @@ -639,7 +662,7 @@ class PodcastInfoViewModel( if (currentPodcast?.isRss == true) { val cachedPage = - repository.getEpisodesPaginated(effectivePodcastId, limit, 0, sortParam) + repository.getShowPageEpisodes(effectivePodcastId, limit, 0, sortParam) currentOffset = cachedPage.sourceCount val successState = PodcastInfoUiState.Success( @@ -659,6 +682,9 @@ class PodcastInfoViewModel( rssRepository.refreshCatalogIfNeeded(effectivePodcastId) } + // Feed-first show pages (P4 PR 16): without a stored copy, boxlore's list shows while the + // feed is read; its rows carry the same IDs, so the list then grows without moving. + val feedOpened = if (currentPodcast?.isRss == true) null else feedFirst.open(effectivePodcastId, currentPodcast) val (apiPodcast, page) = fetchPodcastAndEpisodes(effectivePodcastId, limit, sortParam) @@ -684,11 +710,12 @@ class PodcastInfoViewModel( } currentOffset = page.sourceCount + val pageEpisodes = if (feedOpened != null) ShowFeedLogic.withShow(page.episodes, apiPodcastWithFallback) else page.episodes val baseSuccess = PodcastInfoUiState.Success( podcast = apiPodcastWithFallback, - episodes = page.episodes, - piEpisodes = page.episodes, + episodes = pageEpisodes, + piEpisodes = pageEpisodes, isSubscribed = latestIsSubscribed, hasMoreEpisodes = page.hasMore, currentSort = initialSort, @@ -697,10 +724,11 @@ class PodcastInfoViewModel( _uiState.value = supplementSupport.remountWithSupplements( state = baseSuccess, - piEpisodes = page.episodes, + piEpisodes = pageEpisodes, hasMoreEpisodes = page.hasMore, isLoadingMore = false, ) + feedOpened?.let { opened -> launch { beginFeedFirst(apiPodcastWithFallback, initialSort, opened) } } if (supplementSupport.shouldRefreshOnOpen( apiPodcastWithFallback.id, @@ -796,6 +824,8 @@ class PodcastInfoViewModel( } else { recovered } + val podcastShown = currentPodcast + feedOpened?.let { opened -> launch { beginFeedFirst(podcastShown, initialSort, opened) } } } } } @@ -834,7 +864,7 @@ class PodcastInfoViewModel( apiPodcast = repository.getPodcastDetails(podcastId) if (apiPodcast != null) { val realId = apiPodcast.id - val episodesDeferred = async { repository.getEpisodesPaginated(realId, limit, 0, sortParam) } + val episodesDeferred = async { repository.getShowPageEpisodes(realId, limit, 0, sortParam) } page = episodesDeferred.await() } else { page = @@ -843,13 +873,48 @@ class PodcastInfoViewModel( } } else { val podcastDeferred = async { repository.getPodcastDetails(podcastId) } - val episodesDeferred = async { repository.getEpisodesPaginated(podcastId, limit, 0, sortParam) } + val episodesDeferred = async { repository.getShowPageEpisodes(podcastId, limit, 0, sortParam) } apiPodcast = podcastDeferred.await() page = episodesDeferred.await() } Pair(apiPodcast, page) } + /** + * True when [feedId]'s search runs over the feed's own copy (P4 PR 16): the page reads its feed, + * so a copy that holds only the newest is completed first. False once the page fell back: search + * then runs over the server's list, with cached feed extras, and reads no feed. + */ + private suspend fun prepareFeedFirstSearch(feedId: String, podcast: Podcast): Boolean { + if (!feedFirst.readsFeed(feedId)) return false + if (ShowFeedLogic.needsArchive(feedFirst.archiveComplete, ShowFeedLogic.ArchiveAction.SEARCH)) { + feedFirst.completeArchive(viewModelScope, podcast) + } + return feedFirst.readsFeed(feedId) && repository.showPageSource(feedId) == cx.aswin.boxlore.core.catalog.ShowPageSource.FEED + } + + /** Clears the loading row when a feed-first read ended without reloading the page (it moved on, or the archive was already whole). */ + private fun stopLoadingMore() { + val state = _uiState.value as? PodcastInfoUiState.Success ?: return + if (state.isLoadingMore) _uiState.value = state.copy(isLoadingMore = false) + } + + /** + * Starts the feed-first page's opening read once its first page shows. A failure leaves the + * stored copy showing and clears the status line. + */ + private suspend fun beginFeedFirst(podcast: Podcast, sort: EpisodeSort, opened: cx.aswin.boxlore.core.catalog.ShowPageFeedState) { + try { + feedFirst.begin(podcast, sort, opened) + } catch (e: CancellationException) { + throw e + } catch (e: Exception) { + Log.e(TAG, "Feed-first read failed for ${podcast.id}", e) + val state = _uiState.value as? PodcastInfoUiState.Success ?: return + if (state.podcast.id == podcast.id) _uiState.value = state.copy(isLoadingMore = false, feedStatus = ShowFeedStatus.NONE) + } + } + private fun trackScreenViewed( podcastId: String, podcastName: String?, @@ -913,9 +978,7 @@ class PodcastInfoViewModel( fun loadMoreEpisodes() { val currentState = _uiState.value - if (currentState !is PodcastInfoUiState.Success) return - if (currentState.isLoadingMore || !currentState.hasMoreEpisodes) return - if (currentState.searchResults != null) return // Don't load more when searching + if (currentState !is PodcastInfoUiState.Success || !canLoadMore(currentState)) return _uiState.value = currentState.copy(isLoadingMore = true) @@ -923,7 +986,13 @@ class PodcastInfoViewModel( try { val limit = if (currentState.currentSort == EpisodeSort.OLDEST) OLDEST_PAGE_SIZE else PAGE_SIZE val sortParam = if (currentState.currentSort == EpisodeSort.OLDEST) "oldest" else "newest" - val page = repository.getEpisodesPaginated(currentPodcastId, limit, currentOffset, sortParam) + val page = repository.getShowPageEpisodes(currentPodcastId, limit, currentOffset, sortParam) + if (needsArchiveToLoadMore(limit, page.episodes.size)) { + // Past the stored newest episodes: read the whole feed, then show one more page. + feedFirst.completeArchive(viewModelScope, currentState.podcast, extra = limit, atCopyEnd = true) + stopLoadingMore() + return@launch + } currentOffset += page.sourceCount val latest = _uiState.value as? PodcastInfoUiState.Success ?: return@launch val nextPi = latest.piEpisodes + page.episodes @@ -934,6 +1003,8 @@ class PodcastInfoViewModel( hasMoreEpisodes = page.hasMore, isLoadingMore = false, ) + } catch (e: CancellationException) { + throw e } catch (e: Exception) { e.printStackTrace() _uiState.value = currentState.copy(isLoadingMore = false) @@ -941,6 +1012,21 @@ class PodcastInfoViewModel( } } + /** + * Whether another page may load: not already loading, more to come, not searching, and (for a + * feed-first page, P4 PR 16) not past the stored copy while offline, which waits for a connection. + */ + private fun canLoadMore(state: PodcastInfoUiState.Success): Boolean = + !state.isLoadingMore && + state.hasMoreEpisodes && + state.searchResults == null && + !(feedFirst.readsFeed(currentPodcastId) && feedFirst.holdsAtCopyEnd()) + + /** A feed-first page past its stored newest episodes reads the whole feed before the next page (P4 PR 16). */ + private fun needsArchiveToLoadMore(limit: Int, loaded: Int): Boolean = + feedFirst.readsFeed(currentPodcastId) && + ShowFeedLogic.needsArchive(feedFirst.archiveComplete, ShowFeedLogic.ArchiveAction.LOAD_MORE, limit, loaded) + fun refreshRssFeed() { val currentState = _uiState.value as? PodcastInfoUiState.Success ?: return if (currentState.isRssRefreshing) return @@ -949,8 +1035,11 @@ class PodcastInfoViewModel( isRss = currentState.podcast.isRss, isSubscribed = currentState.isSubscribed, chip = currentState.directFeedChip, + // Once followed, the show's own catalog refresh (with auto-download) takes over. + feedFirst = feedFirst.isActive(currentState.podcast.id) && !currentState.isSubscribed, ) ) { + PodcastInfoPullRefreshLogic.Target.FEED_FIRST -> refreshFeedFirstFromPull(currentState) PodcastInfoPullRefreshLogic.Target.RSS_CATALOG -> refreshTrueRssCatalog(currentState) PodcastInfoPullRefreshLogic.Target.SUBSCRIBED_DIRECT_FEED -> refreshSubscribedDirectFeedCatalogFromPull(currentState) PodcastInfoPullRefreshLogic.Target.DIRECT_FEED -> refreshDirectFeedFromPull(currentState) @@ -959,6 +1048,25 @@ class PodcastInfoViewModel( } } + /** + * Pull-to-refresh of a feed-first page: reads the newest episodes at the registry's address + * into the stored copy (whether it is partial or the whole archive), then reloads the pages. + */ + private fun refreshFeedFirstFromPull(currentState: PodcastInfoUiState.Success) { + val podcast = currentState.podcast + _uiState.value = currentState.copy(isRssRefreshing = true) + viewModelScope.launch { + try { + feedFirst.read(podcast, cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.Scope.NEWEST, showStatus = false) + } catch (e: CancellationException) { + throw e + } catch (e: Exception) { + Log.e(TAG, "Pull-to-refresh feed read failed", e) + } + clearPullRefreshing(podcast.id) + } + } + private fun refreshTrueRssCatalog(currentState: PodcastInfoUiState.Success) { val targetPodcastId = currentState.podcast.id _uiState.value = currentState.copy(isLoadingMore = true, isRssRefreshing = true) @@ -1093,7 +1201,7 @@ class PodcastInfoViewModel( val sortParam = if (activeInitialState.currentSort == EpisodeSort.OLDEST) "oldest" else "newest" val podcast = repository.getPodcastDetails(targetPodcastId) ?: activeInitialState.podcast - val page = repository.getEpisodesPaginated(targetPodcastId, limit, 0, sortParam) + val page = repository.getShowPageEpisodes(targetPodcastId, limit, 0, sortParam) if (!PodcastInfoPullRefreshLogic.shouldApply(currentPodcastId, targetPodcastId)) { return } @@ -1237,25 +1345,40 @@ class PodcastInfoViewModel( val sortString = if (newSort == EpisodeSort.OLDEST) "oldest" else "newest" subscriptionRepository.updatePreferredSort(currentPodcastId, sortString) } + loadFirstSortedPage(currentState, newSort) + } + } - try { - val limit = if (newSort == EpisodeSort.OLDEST) OLDEST_PAGE_SIZE else PAGE_SIZE - val sortParam = if (newSort == EpisodeSort.OLDEST) "oldest" else "newest" - val page = repository.getEpisodesPaginated(currentPodcastId, limit, 0, sortParam) - currentOffset = page.sourceCount - val latestState = _uiState.value as? PodcastInfoUiState.Success ?: return@launch - _uiState.value = - supplementSupport.remountWithSupplements( - state = latestState, - piEpisodes = page.episodes, - hasMoreEpisodes = page.hasMore, - isLoadingMore = false, - ) - } catch (e: Exception) { - e.printStackTrace() - val latestState = _uiState.value as? PodcastInfoUiState.Success ?: return@launch - _uiState.value = latestState.copy(isLoadingMore = false) + /** The first page in [sort] after the sort changed; a feed-first page's oldest wait for the whole feed (P4 PR 16). */ + private suspend fun loadFirstSortedPage(currentState: PodcastInfoUiState.Success, sort: EpisodeSort) { + try { + if (sort == EpisodeSort.OLDEST && + feedFirst.readsFeed(currentPodcastId) && + ShowFeedLogic.needsArchive(feedFirst.archiveComplete, ShowFeedLogic.ArchiveAction.SORT_OLDEST) + ) { + // The oldest episodes are only known once the whole feed is read. + feedFirst.completeArchive(viewModelScope, currentState.podcast) + stopLoadingMore() + return } + val limit = if (sort == EpisodeSort.OLDEST) OLDEST_PAGE_SIZE else PAGE_SIZE + val sortParam = if (sort == EpisodeSort.OLDEST) "oldest" else "newest" + val page = repository.getShowPageEpisodes(currentPodcastId, limit, 0, sortParam) + currentOffset = page.sourceCount + val latestState = _uiState.value as? PodcastInfoUiState.Success ?: return + _uiState.value = + supplementSupport.remountWithSupplements( + state = latestState, + piEpisodes = page.episodes, + hasMoreEpisodes = page.hasMore, + isLoadingMore = false, + ) + } catch (e: CancellationException) { + throw e + } catch (e: Exception) { + e.printStackTrace() + val latestState = _uiState.value as? PodcastInfoUiState.Success ?: return + _uiState.value = latestState.copy(isLoadingMore = false) } } @@ -1291,12 +1414,13 @@ class PodcastInfoViewModel( val feedId = podcastContext?.podcast?.id ?: return@launch val podcast = podcastContext.podcast + val readsFeed = prepareFeedFirstSearch(feedId, podcast) val networkResults = repository.searchEpisodes(feedId, query) val results = supplementSupport.unionSearch( feedId = feedId, query = query, - networkResults = networkResults, + networkResults = ShowFeedLogic.withShow(networkResults, podcast), meta = PodcastListMeta( title = podcast.title, @@ -1304,7 +1428,9 @@ class PodcastInfoViewModel( genre = podcast.genre, artist = podcast.artist, ), - isRss = podcast.isRss, + // The feed's own copy holds every episode the feed has: no extras to add. + // After a fallback the server's list gets them, as without the flag. + isRss = podcast.isRss || readsFeed, ) // Ensure we are still in a valid state to update @@ -1370,6 +1496,9 @@ class PodcastInfoViewModel( podcast = updatedPodcast, isSubscribed = isSubscribed, ) + // Feed-first (P4 PR 16): once followed, the show's own catalog refresh (with auto-download) + // reads the feed; this page's reads would store its validators first. + if (isSubscribed && feedFirst.isActive(latestState.podcast.id)) feedFirst.stop() cx.aswin.boxlore.core.analytics.AnalyticsHelper.trackPodcastSubscriptionToggled( podcastId = latestState.podcast.id, podcastName = latestState.podcast.title, @@ -1499,7 +1628,7 @@ class PodcastInfoViewModel( } val oldest = pageState.currentSort == EpisodeSort.OLDEST val page = - repository.getEpisodesPaginated( + repository.getShowPageEpisodes( feedId = targetPodcastId, limit = if (oldest) OLDEST_PAGE_SIZE else PAGE_SIZE, offset = 0, diff --git a/feature/info/src/main/java/cx/aswin/boxlore/feature/info/components/ShowFeedStatusLine.kt b/feature/info/src/main/java/cx/aswin/boxlore/feature/info/components/ShowFeedStatusLine.kt new file mode 100644 index 000000000..1ce778bd1 --- /dev/null +++ b/feature/info/src/main/java/cx/aswin/boxlore/feature/info/components/ShowFeedStatusLine.kt @@ -0,0 +1,83 @@ +package cx.aswin.boxlore.feature.info.components + +import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Row +import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.layout.size +import androidx.compose.foundation.shape.RoundedCornerShape +import androidx.compose.material.icons.Icons +import androidx.compose.material.icons.rounded.CloudOff +import androidx.compose.material.icons.rounded.Info +import androidx.compose.material3.CircularProgressIndicator +import androidx.compose.material3.Icon +import androidx.compose.material3.MaterialTheme +import androidx.compose.material3.Surface +import androidx.compose.material3.Text +import androidx.compose.runtime.Composable +import androidx.compose.ui.Alignment +import androidx.compose.ui.Modifier +import androidx.compose.ui.platform.testTag +import androidx.compose.ui.res.stringResource +import androidx.compose.ui.semantics.LiveRegionMode +import androidx.compose.ui.semantics.liveRegion +import androidx.compose.ui.semantics.semantics +import androidx.compose.ui.unit.dp +import cx.aswin.boxlore.core.designsystem.layout.BoxloreCardFamily +import cx.aswin.boxlore.core.designsystem.layout.boxloreCardWidth +import cx.aswin.boxlore.feature.info.R +import cx.aswin.boxlore.feature.info.ShowFeedStatus + +internal const val SHOW_FEED_STATUS_TAG = "podcast_info_feed_status" + +/** The text of [status]'s line, or null when the page shows no line. */ +internal fun showFeedStatusText(status: ShowFeedStatus): Int? = when (status) { + ShowFeedStatus.NONE, ShowFeedStatus.CACHED -> null + ShowFeedStatus.READING_NEWEST -> R.string.podcast_info_feed_reading + ShowFeedStatus.COMPLETING_ARCHIVE -> R.string.podcast_info_feed_completing + ShowFeedStatus.FALLBACK -> R.string.podcast_info_feed_fallback + ShowFeedStatus.OFFLINE -> R.string.podcast_info_feed_offline + ShowFeedStatus.OFFLINE_NO_COPY -> R.string.podcast_info_feed_offline_empty +} + +/** + * The feed-first show page's status line above its episodes: a feed read in progress, the + * server's list in place of an unreadable feed, or no connection. A solid tonal surface, announced + * politely to screen readers when it changes. Nothing for a current list or a recent stored copy. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page). + */ +@Composable +internal fun ShowFeedStatusLine(status: ShowFeedStatus, modifier: Modifier = Modifier) { + val text = showFeedStatusText(status) ?: return + val reading = status == ShowFeedStatus.READING_NEWEST || status == ShowFeedStatus.COMPLETING_ARCHIVE + Surface( + color = MaterialTheme.colorScheme.surfaceContainerHigh, + contentColor = MaterialTheme.colorScheme.onSurfaceVariant, + shape = RoundedCornerShape(16.dp), + modifier = + modifier + .fillMaxWidth() + .boxloreCardWidth(BoxloreCardFamily.Utility) + .padding(horizontal = 16.dp) + .testTag(SHOW_FEED_STATUS_TAG) + .semantics(mergeDescendants = true) { liveRegion = LiveRegionMode.Polite }, + ) { + Row( + modifier = Modifier.padding(horizontal = 16.dp, vertical = 12.dp), + horizontalArrangement = Arrangement.spacedBy(12.dp), + verticalAlignment = Alignment.CenterVertically, + ) { + if (reading) { + CircularProgressIndicator(modifier = Modifier.size(18.dp), strokeWidth = 2.dp) + } else { + Icon( + imageVector = if (status == ShowFeedStatus.OFFLINE || status == ShowFeedStatus.OFFLINE_NO_COPY) Icons.Rounded.CloudOff else Icons.Rounded.Info, + contentDescription = null, + modifier = Modifier.size(20.dp), + ) + } + Text(text = stringResource(text), style = MaterialTheme.typography.bodyMedium) + } + } +} diff --git a/feature/info/src/main/java/cx/aswin/boxlore/feature/info/logic/PodcastInfoPullRefreshLogic.kt b/feature/info/src/main/java/cx/aswin/boxlore/feature/info/logic/PodcastInfoPullRefreshLogic.kt index 1de4985fe..c6166f949 100644 --- a/feature/info/src/main/java/cx/aswin/boxlore/feature/info/logic/PodcastInfoPullRefreshLogic.kt +++ b/feature/info/src/main/java/cx/aswin/boxlore/feature/info/logic/PodcastInfoPullRefreshLogic.kt @@ -7,18 +7,24 @@ import cx.aswin.boxlore.feature.info.PodcastInfoUiState internal object PodcastInfoPullRefreshLogic { enum class Target { RSS_CATALOG, + + /** A feed-first show page: read the newest episodes into its stored copy (P4 PR 16). */ + FEED_FIRST, SUBSCRIBED_DIRECT_FEED, DIRECT_FEED, PI_CATALOG, NONE, } + /** [feedFirst]: the page reads the show's own feed (P4 PR 16); it replaces the Missing episodes? path. */ fun target( isRss: Boolean, isSubscribed: Boolean = false, chip: DirectFeedChipState, + feedFirst: Boolean = false, ): Target = when { isRss -> Target.RSS_CATALOG + feedFirst -> Target.FEED_FIRST chip == DirectFeedChipState.Fetching -> Target.NONE isSubscribed -> Target.SUBSCRIBED_DIRECT_FEED chip == DirectFeedChipState.Updated -> Target.DIRECT_FEED diff --git a/feature/info/src/main/java/cx/aswin/boxlore/feature/info/logic/ShowFeedLogic.kt b/feature/info/src/main/java/cx/aswin/boxlore/feature/info/logic/ShowFeedLogic.kt new file mode 100644 index 000000000..6877e65ad --- /dev/null +++ b/feature/info/src/main/java/cx/aswin/boxlore/feature/info/logic/ShowFeedLogic.kt @@ -0,0 +1,110 @@ +package cx.aswin.boxlore.feature.info.logic + +import cx.aswin.boxlore.core.catalog.ShowPageFeedRead +import cx.aswin.boxlore.core.catalog.ShowPageFeedResult +import cx.aswin.boxlore.core.catalog.ShowPageFeedState +import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort.PodcastMeta +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.Scope +import cx.aswin.boxlore.core.model.Episode +import cx.aswin.boxlore.core.model.Podcast +import cx.aswin.boxlore.feature.info.EpisodeSort +import cx.aswin.boxlore.feature.info.ShowFeedStatus + +/** + * Decisions of the feed-first show page, apart from the ViewModel so they can be tested alone: + * how deep a read goes, what the status line says, and how pages join as the list grows. + * + * Added in P4 PR 16 of the own-episode-IDs plan (plan section 11, feed-first show page). + * COMPAT(own-ids): show pages read the show's own feed. Permanent: never remove (MIGRATION.md §5). + */ +internal object ShowFeedLogic { + /** + * The read a page starts as it opens: the whole archive for an oldest-first page, a serial show + * ([podcastType] `serial`, even when the page opens newest first because the phone has no saved + * sort for it) and a feed that lists its oldest items first, else the newest items. + */ + fun openingScope(sort: EpisodeSort, opened: ShowPageFeedState, podcastType: String? = null): Scope = + if (sort == EpisodeSort.OLDEST || opened.oldestFirst || podcastType.equals(SERIAL, ignoreCase = true)) Scope.ARCHIVE else Scope.NEWEST + + /** + * Whether opening reads the feed: always without a stored copy or once it is an hour old, and + * for an archive read the copy doesn't hold yet. + */ + fun readsOnOpen(opened: ShowPageFeedState, scope: Scope): Boolean = opened.recheckDue || (scope == Scope.ARCHIVE && !opened.archiveComplete) + + /** The status line while a read of [scope] runs. */ + fun readingStatus(scope: Scope): ShowFeedStatus = if (scope == Scope.ARCHIVE) ShowFeedStatus.COMPLETING_ARCHIVE else ShowFeedStatus.READING_NEWEST + + /** The status line once [read] ended: offline says whether saved episodes show at all. */ + fun statusAfter(read: ShowPageFeedRead): ShowFeedStatus = when (read.result) { + ShowPageFeedResult.UPDATED, ShowPageFeedResult.UNCHANGED -> ShowFeedStatus.NONE + ShowPageFeedResult.OFFLINE -> if (read.hasCopy) ShowFeedStatus.OFFLINE else ShowFeedStatus.OFFLINE_NO_COPY + ShowPageFeedResult.FALLBACK -> ShowFeedStatus.FALLBACK + } + + /** The status line as a page opens without reading: the stored copy, or nothing yet. */ + fun openingStatus(opened: ShowPageFeedState): ShowFeedStatus = if (opened.cached) ShowFeedStatus.CACHED else ShowFeedStatus.NONE + + /** + * Whether an action must wait for the whole archive: in-show search, the oldest-first sort, and + * a page that ran past the stored newest items ([pageSize] asked, [pageCount] came back). + */ + fun needsArchive(archiveComplete: Boolean, action: ArchiveAction, pageSize: Int = 0, pageCount: Int = 0): Boolean { + if (archiveComplete) return false + return when (action) { + ArchiveAction.SEARCH, ArchiveAction.SORT_OLDEST -> true + ArchiveAction.LOAD_MORE -> pageCount < pageSize + } + } + + /** + * How many episodes a page reloaded after a read holds: as many as were showing (so the rows, + * the scroll position and any selection stay), plus [extra] when the listener was scrolling on. + */ + fun reloadCount(showing: Int, pageSize: Int, extra: Int = 0): Int = maxOf(showing + extra, pageSize) + + /** + * Whether a reload pages on: until it holds [wanted] episodes, and then until the episode that + * was last on the page ([anchorId]) is back, so episodes added above (at most [maxNewAbove], one + * read of the newest items) never push the rows the listener was on out of the list. + */ + fun keepsLoading(loaded: List, wanted: Int, anchorId: String?, maxNewAbove: Int = NEWEST_READ_ITEMS): Boolean { + if (loaded.size < wanted) return true + if (anchorId == null || loaded.size >= wanted + maxNewAbove) return false + return loaded.none { it.id == anchorId } + } + + /** + * Whether the reloaded page has more: what the last page said, except that a stored copy that + * ran out while the archive is still unread ([archivePending]) has more to come from the feed. + */ + fun hasMoreAfterPage(pageHasMore: Boolean, pageCount: Int, archivePending: Boolean): Boolean = + if (pageCount > 0) pageHasMore else pageHasMore && archivePending + + /** Stored-copy episodes with the show's details filled in where the copy has none. */ + fun withShow(episodes: List, podcast: Podcast): List = episodes.map { episode -> + if (!episode.podcastTitle.isNullOrBlank() && !episode.podcastImageUrl.isNullOrBlank()) return@map episode + episode.copy( + podcastTitle = episode.podcastTitle?.takeIf(String::isNotBlank) ?: podcast.title, + podcastImageUrl = episode.podcastImageUrl?.takeIf(String::isNotBlank) ?: podcast.imageUrl.takeIf(String::isNotBlank), + podcastId = episode.podcastId ?: podcast.id, + podcastGenre = episode.podcastGenre ?: podcast.genre, + podcastArtist = episode.podcastArtist ?: podcast.artist, + ) + } + + /** The show details stored-copy episodes are listed with. */ + fun metaOf(podcast: Podcast): PodcastMeta = PodcastMeta( + title = podcast.title.takeIf(String::isNotBlank), + imageUrl = podcast.imageUrl.takeIf(String::isNotBlank), + genre = podcast.genre, + artist = podcast.artist, + ) + + enum class ArchiveAction { SEARCH, SORT_OLDEST, LOAD_MORE } + + private const val SERIAL = "serial" + + /** The most episodes one read of the newest items adds above the list. */ + private const val NEWEST_READ_ITEMS = 100 +} diff --git a/feature/info/src/main/res/values/strings.xml b/feature/info/src/main/res/values/strings.xml index 156aa1a14..9efa29150 100644 --- a/feature/info/src/main/res/values/strings.xml +++ b/feature/info/src/main/res/values/strings.xml @@ -118,6 +118,11 @@ Subscribed shows automatically get the latest episodes from the publisher’s feed. This check is only needed if you prefer to stay unsubscribed.\n\nAllow boxlore to look for missing episodes and refresh the list whenever you open this show. Large feeds may use more data. Allow feed checks Not now + Checking the feed for new episodes… + Loading all episodes from the feed… + Can’t reach this show’s feed. Showing boxlore’s episode list instead. + You’re offline. Showing saved episodes. + You’re offline. Connect to load this show’s episodes. Play trailer Support Select episode diff --git a/feature/info/src/test/java/cx/aswin/boxlore/feature/info/PodcastInfoAdaptiveTransitionTest.kt b/feature/info/src/test/java/cx/aswin/boxlore/feature/info/PodcastInfoAdaptiveTransitionTest.kt index daa77ce09..38eb7c02c 100644 --- a/feature/info/src/test/java/cx/aswin/boxlore/feature/info/PodcastInfoAdaptiveTransitionTest.kt +++ b/feature/info/src/test/java/cx/aswin/boxlore/feature/info/PodcastInfoAdaptiveTransitionTest.kt @@ -86,6 +86,7 @@ import org.robolectric.shadows.ShadowDialog @RunWith(RobolectricTestRunner::class) @Config(sdk = [33], application = Application::class) @GraphicsMode(GraphicsMode.Mode.NATIVE) +@Suppress("LargeClass") // One live page fixture drives every window gate; splitting it would copy the fixture. class PodcastInfoAdaptiveTransitionTest { @get:Rule val fixture = PhoneGoldenRule(PhoneGoldenConfiguration(1_280, 800, false)) @@ -524,6 +525,79 @@ class PodcastInfoAdaptiveTransitionTest { } } + @Test + fun `the feed status line shows each state across window changes and recreation`() { + uiState.value = success().copy(feedStatus = ShowFeedStatus.READING_NEWEST) + render() + val statuses = + listOf( + ShowFeedStatus.READING_NEWEST to "Checking the feed for new episodes…", + ShowFeedStatus.COMPLETING_ARCHIVE to "Loading all episodes from the feed…", + ShowFeedStatus.FALLBACK to "Can’t reach this show’s feed. Showing boxlore’s episode list instead.", + ShowFeedStatus.OFFLINE to "You’re offline. Showing saved episodes.", + ) + for ((status, text) in statuses) { + uiState.update { (it as PodcastInfoUiState.Success).copy(feedStatus = status) } + acrossWindowChanges { + // On a short window the hero fills the screen: the line sits right below it. + fixture.compose.onNodeWithTag(LIST_TAG).performScrollToIndex(RESULT_ROW) + fixture.compose.onNodeWithText(text).assertIsDisplayed() + } + } + recreate(FOLD_TABLETOP) + fixture.compose.onNodeWithTag(LIST_TAG).performScrollToIndex(RESULT_ROW) + fixture.compose.onNodeWithText("You’re offline. Showing saved episodes.").assertIsDisplayed() + fixture.compose.onNodeWithTag(FEED_STATUS_TAG).assertIsDisplayed() + + // A current list and a recent stored copy show no line. + for (quiet in listOf(ShowFeedStatus.NONE, ShowFeedStatus.CACHED)) { + uiState.update { (it as PodcastInfoUiState.Success).copy(feedStatus = quiet) } + acrossWindowChanges { fixture.compose.onNodeWithTag(FEED_STATUS_TAG).assertDoesNotExist() } + } + verify(model, never()).refreshRssFeed() + } + + @Test + fun `scroll and selection hold while the list grows and switches source during window changes`() { + uiState.value = success().copy(feedStatus = ShowFeedStatus.READING_NEWEST) + render() + // Rows: hero, toolbar, the status line, then the episodes. + fixture.compose.onNodeWithTag(LIST_TAG).performScrollToIndex(RESULT_ROW + 1 + 5) + fixture.compose.onNodeWithText("Episode 5").performTouchInput { longClick() } + selectionCount("1") + fixture.compose.onNodeWithTag(LIST_TAG).performScrollToIndex(RESULT_ROW + 1 + 12) + val anchor = "Episode 12" + fixture.compose.onNodeWithText(anchor).assertIsDisplayed() + + val newer = List(3) { index -> + TestFixtures.episode(id = "n$index", title = "New $index", podcastId = "1", podcastTitle = podcast.title, publishedDate = 1_700_000_000L + (index + 1) * 86_400L) + } + // Each window step lands one stage of a feed-first page: the stored copy grows with older + // episodes, newer ones arrive above, the archive is read, and the source switches to the + // server's list and back, every row keeping its ID. + val stages: List<(PodcastInfoUiState.Success) -> PodcastInfoUiState.Success> = + listOf( + { it.copy(episodes = episodes.take(40), piEpisodes = episodes.take(40), feedStatus = ShowFeedStatus.NONE) }, + { it.copy(feedStatus = ShowFeedStatus.COMPLETING_ARCHIVE, isLoadingMore = true) }, + { it.copy(episodes = newer + episodes, piEpisodes = newer + episodes, feedStatus = ShowFeedStatus.NONE, isLoadingMore = false) }, + { it.copy(feedStatus = ShowFeedStatus.FALLBACK) }, + { it.copy(episodes = episodes, piEpisodes = episodes, feedStatus = ShowFeedStatus.READING_NEWEST) }, + { it.copy(feedStatus = ShowFeedStatus.OFFLINE) }, + ) + acrossWindowChanges(onStep = { step -> uiState.update { stages[step % stages.size](it as PodcastInfoUiState.Success) } }) { + fixture.compose.onNodeWithText(anchor).assertIsDisplayed() + selectionCount("1") + } + recreate(FOLD_TABLETOP) + fixture.compose.onNodeWithText(anchor).assertIsDisplayed() + selectionCount("1") + + fixture.compose.onNodeWithContentDescription("Play selected episodes").performClick() + assertEquals(listOf(listOf("e5")), playedIds()) + verify(model, times(2)).loadPodcast("1") + verify(model, never()).refreshRssFeed() + } + @Test fun `loading, error, retry and empty states hold across window changes and recreation`() { uiState.value = PodcastInfoUiState.Loading @@ -682,10 +756,19 @@ class PodcastInfoAdaptiveTransitionTest { private fun playedIds(): List> = played.toList() /** Moves the same live page across every gate and back, asserting after each step. */ - private fun acrossWindowChanges(assertion: () -> Unit) { + private fun acrossWindowChanges(assertion: () -> Unit) = acrossWindowChanges(onStep = {}, assertion = assertion) + + /** + * [acrossWindowChanges], with [onStep] changing the page's state in the same frame as each + * window change, as a feed read that lands mid-resize does. + */ + private fun acrossWindowChanges(onStep: (Int) -> Unit, assertion: () -> Unit) { val start = window.value (TRANSITIONS + start).forEachIndexed { step, size -> - fixture.compose.runOnIdle { window.value = size } + fixture.compose.runOnIdle { + window.value = size + onStep(step) + } fixture.compose.runOnUiThread { Snapshot.sendApplyNotifications() } fixture.compose.waitForIdle() try { @@ -783,6 +866,7 @@ class PodcastInfoAdaptiveTransitionTest { const val LIST_TAG = "podcast_info_episode_list" const val LOADING_TAG = "podcast_info_loading" const val REFRESH_TAG = "podcast_info_refresh_indicator" + const val FEED_STATUS_TAG = "podcast_info_feed_status" const val RESULT_ROW = 2 val ALERT_EXITS = listOf(Exit.CONFIRM, Exit.CANCEL, Exit.BACK, Exit.OUTSIDE) val SELECTION_ACTIONS = listOf( diff --git a/feature/info/src/test/java/cx/aswin/boxlore/feature/info/PodcastInfoFeedFirstTest.kt b/feature/info/src/test/java/cx/aswin/boxlore/feature/info/PodcastInfoFeedFirstTest.kt new file mode 100644 index 000000000..f58aa843b --- /dev/null +++ b/feature/info/src/test/java/cx/aswin/boxlore/feature/info/PodcastInfoFeedFirstTest.kt @@ -0,0 +1,208 @@ +package cx.aswin.boxlore.feature.info + +import cx.aswin.boxlore.core.catalog.PodcastRepository +import cx.aswin.boxlore.core.catalog.ShowPageFeedRead +import cx.aswin.boxlore.core.catalog.ShowPageFeedResult +import cx.aswin.boxlore.core.catalog.ShowPageFeedState +import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort.PodcastMeta +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.Scope +import cx.aswin.boxlore.core.model.Episode +import cx.aswin.boxlore.core.testing.TestFixtures +import kotlinx.coroutines.CompletableDeferred +import kotlinx.coroutines.async +import kotlinx.coroutines.test.runTest +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Assertions.assertFalse +import org.junit.jupiter.api.Assertions.assertNull +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.Test + +/** + * The feed-first show page's reads and reloads ([PodcastInfoFeedFirst]) against a scripted source: + * a fallback stops further feed reads until the page opens again, newer episodes above keep the + * rows the listener was on, offline without a copy says so, the end of the copy waits for a + * connection, concurrent searches share one archive read, and following hands the page over. + */ +class PodcastInfoFeedFirstTest { + private val source = FakeSource() + private val ui = FakeUi() + private val feedFirst = PodcastInfoFeedFirst(source, ui) + private val podcast = TestFixtures.podcast(id = SHOW, title = "Show") + + @Test + fun `after a fallback, search, the oldest sort and scrolling read no feed until the page opens again`() = runTest { + source.list = episodes(0 until 20) + feedFirst.open(SHOW, podcast) + ui.show(podcast, source.list) + source.results += ShowPageFeedResult.FALLBACK + + assertEquals(ShowPageFeedResult.FALLBACK, feedFirst.completeArchive(this, podcast)?.result) + repeat(3) { assertNull(feedFirst.completeArchive(this, podcast), "no read after the fallback") } + + assertEquals(1, source.reads.size) + assertTrue(feedFirst.isActive(SHOW), "pull to refresh still reads the feed") + assertFalse(feedFirst.readsFeed(SHOW)) + assertEquals(ShowFeedStatus.FALLBACK, ui.current?.feedStatus) + + feedFirst.open(SHOW, podcast) + assertTrue(feedFirst.readsFeed(SHOW), "opening the page tries the feed again") + } + + @Test + fun `newer episodes above keep the rows the listener was on`() = runTest { + source.list = episodes(0 until 100) + feedFirst.open(SHOW, podcast) + ui.show(podcast, source.list.take(40)) + // The read finds five newer episodes, which go above the list. + source.onRead = { source.list = episodes(-5 until 0) + source.list } + + feedFirst.read(podcast, Scope.NEWEST) + + val shown = requireNotNull(ui.current).piEpisodes.map { it.id } + assertEquals("e-5", shown.first()) + assertEquals(44, shown.indexOf("e39"), "the row that was last is still in the list, below the five new ones") + assertEquals(shown.size, ui.lastOffset) + } + + @Test + fun `offline without a stored copy never says saved episodes show`() = runTest { + feedFirst.open(SHOW, podcast) + ui.show(podcast, emptyList()) + source.results += ShowPageFeedResult.OFFLINE + source.hasCopy = false + + feedFirst.read(podcast, Scope.NEWEST) + + assertEquals(ShowFeedStatus.OFFLINE_NO_COPY, ui.current?.feedStatus) + } + + @Test + fun `the end of the copy waits for a connection, and the list stays open for it`() = runTest { + source.list = episodes(0 until 20) + source.archivePending = true + feedFirst.open(SHOW, podcast) + ui.show(podcast, source.list) + source.results += ShowPageFeedResult.OFFLINE + source.online = false + + feedFirst.completeArchive(this, podcast, extra = 20, atCopyEnd = true) + + assertTrue(feedFirst.holdsAtCopyEnd()) + assertEquals(ShowFeedStatus.OFFLINE, ui.current?.feedStatus) + assertTrue(requireNotNull(ui.current).hasMoreEpisodes, "scrolling picks up again once online") + source.online = true + assertFalse(feedFirst.holdsAtCopyEnd()) + } + + @Test + fun `searches typed while the archive reads wait on that one read`() = runTest { + source.list = episodes(0 until 20) + feedFirst.open(SHOW, podcast) + ui.show(podcast, source.list) + val gate = CompletableDeferred() + source.gate = gate + source.results += ShowPageFeedResult.UPDATED + source.archiveAfterRead = true + + val first = async { feedFirst.completeArchive(this@runTest, podcast) } + val second = async { feedFirst.completeArchive(this@runTest, podcast) } + testScheduler.advanceUntilIdle() + gate.complete(Unit) + + assertEquals(ShowPageFeedResult.UPDATED, first.await()?.result) + assertEquals(ShowPageFeedResult.UPDATED, second.await()?.result) + assertEquals(listOf(Scope.ARCHIVE), source.reads) + assertTrue(feedFirst.archiveComplete) + assertNull(feedFirst.completeArchive(this, podcast), "a whole archive is never read for search again") + } + + @Test + fun `a read that ends after the page moved to another show changes nothing`() = runTest { + source.list = episodes(0 until 20) + feedFirst.open(SHOW, podcast) + ui.show(podcast, source.list) + val before = ui.current + source.onRead = { ui.podcastId = "other" } + + assertNull(feedFirst.read(podcast, Scope.NEWEST)) + assertEquals(before?.piEpisodes, ui.current?.piEpisodes) + } + + @Test + fun `a serial show reads its archive as it opens, and following hands the page over`() = runTest { + source.list = episodes(0 until 20) + val serial = podcast.copy(type = "serial") + val opened = requireNotNull(feedFirst.open(SHOW, serial)) + ui.show(serial, source.list) + source.results += ShowPageFeedResult.UPDATED + + feedFirst.begin(serial, EpisodeSort.NEWEST, opened) + + assertEquals(listOf(Scope.ARCHIVE), source.reads) + feedFirst.stop() + assertFalse(feedFirst.isActive(SHOW)) + assertFalse(feedFirst.readsFeed(SHOW)) + } + + private class FakeSource : PodcastInfoFeedFirst.Source { + var list: List = emptyList() + var archivePending = false + var archiveAfterRead = false + var hasCopy = true + var online = true + var gate: CompletableDeferred? = null + var onRead: () -> Unit = {} + val results = ArrayDeque() + val reads = mutableListOf() + private var archiveComplete = false + + override suspend fun open(podcastId: String, meta: PodcastMeta): ShowPageFeedState = + ShowPageFeedState(cached = list.isNotEmpty(), archiveComplete = archiveComplete, recheckDue = false, oldestFirst = false) + + override suspend fun read(podcastId: String, meta: PodcastMeta, scope: Scope, savedFeedUrl: String?): ShowPageFeedRead { + reads += scope + gate?.await() + onRead() + if (archiveAfterRead && scope == Scope.ARCHIVE) archiveComplete = true + return ShowPageFeedRead(results.removeFirstOrNull() ?: ShowPageFeedResult.UPDATED, archiveComplete, hasCopy) + } + + override suspend fun page(podcastId: String, limit: Int, offset: Int, sort: String): PodcastRepository.EpisodePage { + val page = list.drop(offset).take(limit) + return PodcastRepository.EpisodePage(page, hasMore = offset + page.size < list.size || archivePending, sourceCount = page.size) + } + + override fun online(): Boolean = online + } + + private class FakeUi : PodcastInfoFeedFirst.Ui { + var current: PodcastInfoUiState.Success? = null + var podcastId = SHOW + var lastOffset = -1 + + fun show(podcast: cx.aswin.boxlore.core.model.Podcast, episodes: List) { + current = PodcastInfoUiState.Success(podcast = podcast, episodes = episodes, piEpisodes = episodes, isSubscribed = false) + } + + override fun state(): PodcastInfoUiState.Success? = current + + override fun set(state: PodcastInfoUiState.Success) { + current = state + } + + override fun currentPodcastId(): String = podcastId + + override fun setOffset(offset: Int) { + lastOffset = offset + } + + override suspend fun remount(state: PodcastInfoUiState.Success, episodes: List, hasMore: Boolean): PodcastInfoUiState.Success = + state.copy(episodes = episodes, piEpisodes = episodes, hasMoreEpisodes = hasMore) + } + + private companion object { + const val SHOW = "920666" + + fun episodes(range: IntProgression): List = range.map { TestFixtures.episode(id = "e$it", podcastId = SHOW) } + } +} diff --git a/feature/info/src/test/java/cx/aswin/boxlore/feature/info/logic/ShowFeedLogicTest.kt b/feature/info/src/test/java/cx/aswin/boxlore/feature/info/logic/ShowFeedLogicTest.kt new file mode 100644 index 000000000..79e0cd22b --- /dev/null +++ b/feature/info/src/test/java/cx/aswin/boxlore/feature/info/logic/ShowFeedLogicTest.kt @@ -0,0 +1,141 @@ +package cx.aswin.boxlore.feature.info.logic + +import cx.aswin.boxlore.core.catalog.ShowPageFeedRead +import cx.aswin.boxlore.core.catalog.ShowPageFeedResult +import cx.aswin.boxlore.core.catalog.ShowPageFeedState +import cx.aswin.boxlore.core.domain.ports.ShowPageFeedPort.Scope +import cx.aswin.boxlore.core.testing.TestFixtures +import cx.aswin.boxlore.feature.info.DirectFeedChipState +import cx.aswin.boxlore.feature.info.EpisodeSort +import cx.aswin.boxlore.feature.info.R +import cx.aswin.boxlore.feature.info.ShowFeedStatus +import cx.aswin.boxlore.feature.info.components.showFeedStatusText +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Assertions.assertFalse +import org.junit.jupiter.api.Assertions.assertNull +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.Test + +/** The feed-first show page's decisions: read depth, the status line, archive gating and page growth. */ +class ShowFeedLogicTest { + private val fresh = ShowPageFeedState(cached = true, archiveComplete = false, recheckDue = false, oldestFirst = false) + + private fun read(result: ShowPageFeedResult, hasCopy: Boolean = true) = ShowPageFeedRead(result, archiveComplete = false, hasCopy = hasCopy) + + @Test + fun `a page opens on the newest items unless it lists oldest first`() { + assertEquals(Scope.NEWEST, ShowFeedLogic.openingScope(EpisodeSort.NEWEST, fresh)) + assertEquals(Scope.ARCHIVE, ShowFeedLogic.openingScope(EpisodeSort.OLDEST, fresh), "serial shows open oldest first") + assertEquals(Scope.ARCHIVE, ShowFeedLogic.openingScope(EpisodeSort.NEWEST, fresh.copy(oldestFirst = true))) + } + + @Test + fun `a serial show reads its whole archive even when its page opens newest first`() { + // An unfollowed show has no saved sort, so its page opens newest first. + assertEquals(Scope.ARCHIVE, ShowFeedLogic.openingScope(EpisodeSort.NEWEST, fresh, podcastType = "serial")) + assertEquals(Scope.ARCHIVE, ShowFeedLogic.openingScope(EpisodeSort.NEWEST, fresh, podcastType = "Serial")) + assertEquals(Scope.NEWEST, ShowFeedLogic.openingScope(EpisodeSort.NEWEST, fresh, podcastType = "episodic")) + } + + @Test + fun `a recent copy is shown without a read, an old one or a missing archive is read`() { + assertFalse(ShowFeedLogic.readsOnOpen(fresh, Scope.NEWEST)) + assertEquals(ShowFeedStatus.CACHED, ShowFeedLogic.openingStatus(fresh)) + assertTrue(ShowFeedLogic.readsOnOpen(fresh.copy(recheckDue = true), Scope.NEWEST)) + assertTrue(ShowFeedLogic.readsOnOpen(fresh, Scope.ARCHIVE)) + assertFalse(ShowFeedLogic.readsOnOpen(fresh.copy(archiveComplete = true), Scope.ARCHIVE)) + assertEquals(ShowFeedStatus.NONE, ShowFeedLogic.openingStatus(fresh.copy(cached = false))) + } + + @Test + fun `the status line follows the read`() { + assertEquals(ShowFeedStatus.READING_NEWEST, ShowFeedLogic.readingStatus(Scope.NEWEST)) + assertEquals(ShowFeedStatus.COMPLETING_ARCHIVE, ShowFeedLogic.readingStatus(Scope.ARCHIVE)) + assertEquals(ShowFeedStatus.NONE, ShowFeedLogic.statusAfter(read(ShowPageFeedResult.UPDATED))) + assertEquals(ShowFeedStatus.NONE, ShowFeedLogic.statusAfter(read(ShowPageFeedResult.UNCHANGED))) + assertEquals(ShowFeedStatus.OFFLINE, ShowFeedLogic.statusAfter(read(ShowPageFeedResult.OFFLINE))) + assertEquals(ShowFeedStatus.FALLBACK, ShowFeedLogic.statusAfter(read(ShowPageFeedResult.FALLBACK))) + } + + @Test + fun `offline without a stored copy never says saved episodes show`() { + assertEquals(ShowFeedStatus.OFFLINE_NO_COPY, ShowFeedLogic.statusAfter(read(ShowPageFeedResult.OFFLINE, hasCopy = false))) + assertEquals(R.string.podcast_info_feed_offline_empty, showFeedStatusText(ShowFeedStatus.OFFLINE_NO_COPY)) + } + + @Test + fun `only the reading, fallback and offline states show a line`() { + assertNull(showFeedStatusText(ShowFeedStatus.NONE)) + assertNull(showFeedStatusText(ShowFeedStatus.CACHED)) + assertEquals(R.string.podcast_info_feed_reading, showFeedStatusText(ShowFeedStatus.READING_NEWEST)) + assertEquals(R.string.podcast_info_feed_completing, showFeedStatusText(ShowFeedStatus.COMPLETING_ARCHIVE)) + assertEquals(R.string.podcast_info_feed_fallback, showFeedStatusText(ShowFeedStatus.FALLBACK)) + assertEquals(R.string.podcast_info_feed_offline, showFeedStatusText(ShowFeedStatus.OFFLINE)) + } + + @Test + fun `search, the oldest sort and scrolling past the stored newest wait for the archive`() { + assertTrue(ShowFeedLogic.needsArchive(false, ShowFeedLogic.ArchiveAction.SEARCH)) + assertTrue(ShowFeedLogic.needsArchive(false, ShowFeedLogic.ArchiveAction.SORT_OLDEST)) + assertTrue(ShowFeedLogic.needsArchive(false, ShowFeedLogic.ArchiveAction.LOAD_MORE, pageSize = 20, pageCount = 0)) + assertFalse(ShowFeedLogic.needsArchive(false, ShowFeedLogic.ArchiveAction.LOAD_MORE, pageSize = 20, pageCount = 20)) + assertFalse(ShowFeedLogic.needsArchive(true, ShowFeedLogic.ArchiveAction.SEARCH)) + } + + @Test + fun `a reload after a read keeps every row that showed`() { + assertEquals(20, ShowFeedLogic.reloadCount(showing = 0, pageSize = 20)) + assertEquals(60, ShowFeedLogic.reloadCount(showing = 60, pageSize = 20)) + assertEquals(120, ShowFeedLogic.reloadCount(showing = 100, pageSize = 20, extra = 20)) + } + + @Test + fun `a reload pages on until the episode that was last is back, at most one newest read further`() { + val loaded = (0 until 40).map { TestFixtures.episode(id = "e$it") } + assertTrue(ShowFeedLogic.keepsLoading(loaded.take(10), wanted = 40, anchorId = "e39")) + assertFalse(ShowFeedLogic.keepsLoading(loaded, wanted = 40, anchorId = "e39")) + // Five newer episodes arrived above: the old last row is now 45th. + assertTrue(ShowFeedLogic.keepsLoading(loaded, wanted = 40, anchorId = "e44")) + assertFalse(ShowFeedLogic.keepsLoading(loaded, wanted = 40, anchorId = "gone", maxNewAbove = 0), "at most one newest read past the rows that showed") + assertFalse(ShowFeedLogic.keepsLoading(loaded, wanted = 40, anchorId = null)) + } + + @Test + fun `a copy that ran out keeps the list open while its archive is unread`() { + assertTrue(ShowFeedLogic.hasMoreAfterPage(pageHasMore = true, pageCount = 0, archivePending = true)) + assertFalse(ShowFeedLogic.hasMoreAfterPage(pageHasMore = true, pageCount = 0, archivePending = false)) + assertTrue(ShowFeedLogic.hasMoreAfterPage(pageHasMore = true, pageCount = 20, archivePending = false)) + assertFalse(ShowFeedLogic.hasMoreAfterPage(pageHasMore = false, pageCount = 20, archivePending = true)) + } + + @Test + fun `stored-copy episodes take the show's details only where they have none`() { + val podcast = TestFixtures.podcast(id = "920666", title = "Curious world", imageUrl = "https://img.example/show.jpg") + val bare = TestFixtures.episode(id = "e1", title = "One").copy(podcastTitle = null, podcastImageUrl = null, podcastId = null) + val own = TestFixtures.episode(id = "e2", title = "Two").copy(podcastTitle = "Own", podcastImageUrl = "https://img.example/own.jpg") + + val filled = ShowFeedLogic.withShow(listOf(bare, own), podcast) + + assertEquals("Curious world", filled[0].podcastTitle) + assertEquals("https://img.example/show.jpg", filled[0].podcastImageUrl) + assertEquals("920666", filled[0].podcastId) + assertEquals(own, filled[1]) + assertEquals("Curious world", ShowFeedLogic.metaOf(podcast).title) + } + + @Test + fun `pull to refresh on a feed-first page reads the feed`() { + assertEquals( + PodcastInfoPullRefreshLogic.Target.FEED_FIRST, + PodcastInfoPullRefreshLogic.target(isRss = false, chip = DirectFeedChipState.Hidden, feedFirst = true), + ) + assertEquals( + PodcastInfoPullRefreshLogic.Target.FEED_FIRST, + PodcastInfoPullRefreshLogic.target(isRss = false, isSubscribed = true, chip = DirectFeedChipState.Hidden, feedFirst = true), + ) + assertEquals( + PodcastInfoPullRefreshLogic.Target.RSS_CATALOG, + PodcastInfoPullRefreshLogic.target(isRss = true, chip = DirectFeedChipState.Hidden, feedFirst = true), + ) + } +}