diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 0c03df78d..bbd8f560d 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -29,6 +29,7 @@ These values are part of the shipping product. Renames or recreations break upgr | Deep links | `boxlore://` and `boxcast://`, plus both HTTPS share-path prefixes. Patterns are only added: episode links may also carry `show=`, which fills the route's `podcastId` | | Episode share registration | Unique work names `episode-share-registration-` (`EpisodeShareRegistrationWorker`); never renamed | | Notification re-registration | Unique work names `podcast-notification-registration-` (`PodcastNotificationRegistrationWorker`, after a saved feed address changes); never renamed | +| Feed address repair after the update | Unique work name `feed-address-upgrade-repair` (`UpgradeRepairWorker`); step 4 and sync catch-up record `noBackupFilesDir/upgrade_repair_notifications`; per-show progress in Room `upgrade_repair_shows`; never renamed | | BuildConfig | Prefer `BOXLORE_*`; Gradle still dual-reads `BOXCAST_*` fallbacks | | Episode / media IDs | `rss:` prefixes, negative RSS IDs, mediaId prefixes, and `customCacheKey` schemes stay as implemented. PI **episode supplements** also use negative episode IDs (generated under an `rss:` feed namespace) but keep `Episode.podcastId` as the Podcast Index show id — they are not `rss:` library rows. Episode identity (show ID, catalog key), recipe IDs, collisions and aliases follow [`docs/EPISODE_IDS.md`](docs/EPISODE_IDS.md) | | Playback instance | One UI-scoped `PlaybackRepository`; routes and workers must not construct a second one | diff --git a/app/README.md b/app/README.md index c6bfe8a6b..11e6573e1 100644 --- a/app/README.md +++ b/app/README.md @@ -68,11 +68,12 @@ The application module owns the Android app shell: `BoxLoreApplication`, `MainAc - `LegacyWorkerFactory` maps legacy worker class names to current worker implementations for WorkManager continuity. - `MainActivity` hosts theme, edge-to-edge setup, app update hooks, surveys, library import state (full-screen `OpmlImportDialog` overlay so onboarding welcome never shows through), the selected floating or classic navigation presentation, and the player overlay. Its start/stop lifecycle informs the application-scoped playback repository whether UI position polling is needed; background audio, persistence, notifications, Cast, and MediaSession behavior remain service-owned. - **Feed fix off switches** (`sync/FeedFixSwitches`): PostHog kill switches for the feed and episode fixes, passed into core modules as lambdas. A fix runs unless its flag is on; no answer (PostHog not ready, flag missing or inactive, an error) keeps it on. `feed-fix-match-baseline-off` sends the feed-only extras' comparison list back from the 1,000-item `episodes/match` list to the 100-item page (`PodcastRepository.useEpisodeMatchBaseline`). `feed-fix-address-recheck-off`, `feed-fix-sync-fallback-off` and `feed-fix-moved-feed-off` turn off the subscription sync's dead-address re-check, its `/sync` fallback for failed feeds, and following moved feeds (`FeedFixToggles`). `FeedFixSwitchesTest` covers the default. The same fail-open switches pause the v2 build's jobs: `upgrade-repair-off` (the one-time repair of saved feed addresses after the update), `address-check-off` (the daily address check of followed shows), both without marking the work done, and `v2-discovery-off` (discovery back on the v1 routes). These three are read through `sync/SavedKillSwitch`, because the repair can start on a headless launch before PostHog is set up: once PostHog is ready its answer decides and is saved; while it is still starting, the last saved answer is used, and the job runs only when none was saved. -- **Feed-first rollout** (`sync/FeedFirstSwitch`): the `feed_first` PostHog flag picks where show pages read episodes (on: the show's own feed; off: the server's list). Unlike the off switches it fails closed: with no boolean answer from PostHog it uses the last saved answer, else off. The first read fixes the value for the process, so a page's source never switches while the app runs. `$feature_flag_called` is recorded once per process, only when the value came from a live PostHog answer, and the process uses the value that exposure read returned. `sync/RolloutFlagMemory` saves `feed_first` and the three switches above for the next process whenever PostHog reports loaded flags (`PostHogAndroidConfig.onFeatureFlags`: the startup preload, `identify` and `reset`; explicit `reloadFeatureFlags` calls don't reach it, so a mid-session change is saved at the next of those loads). To roll back, keep `feed_first` active with a 0% release condition so phones get and save `false`; never disable or delete it before the flag's removal, because a disabled flag is missing from PostHog's answer and a phone that saved `true` would stay on. `AppContainer.rolloutSwitches` hands `feed_first` and the three switches above to `:core:catalog` as `RolloutSwitches` on `PodcastRepository`. Discovery reads `v2Discovery` (`PodcastRepository` and `DiscoveryRouting`, below); the daily address check and broken-feed reports read `addressCheck` (`address-check-off`, through `SubscriptionForegroundSync`, below); show pages read `feed_first` (`PodcastRepository.isFeedFirstShow`, below); `upgradeRepair` has no reader yet. +- **Feed-first rollout** (`sync/FeedFirstSwitch`): the `feed_first` PostHog flag picks where show pages read episodes (on: the show's own feed; off: the server's list). Unlike the off switches it fails closed: with no boolean answer from PostHog it uses the last saved answer, else off. The first read fixes the value for the process, so a page's source never switches while the app runs. `$feature_flag_called` is recorded once per process, only when the value came from a live PostHog answer, and the process uses the value that exposure read returned. `sync/RolloutFlagMemory` saves `feed_first` and the three switches above for the next process whenever PostHog reports loaded flags (`PostHogAndroidConfig.onFeatureFlags`: the startup preload, `identify` and `reset`; explicit `reloadFeatureFlags` calls don't reach it, so a mid-session change is saved at the next of those loads). To roll back, keep `feed_first` active with a 0% release condition so phones get and save `false`; never disable or delete it before the flag's removal, because a disabled flag is missing from PostHog's answer and a phone that saved `true` would stay on. `AppContainer.rolloutSwitches` hands `feed_first` and the three switches above to `:core:catalog` as `RolloutSwitches` on `PodcastRepository`. Discovery reads `v2Discovery` (`PodcastRepository` and `DiscoveryRouting`, below); the daily address check and broken-feed reports read `addressCheck` (`address-check-off`, through `SubscriptionForegroundSync`, below); show pages read `feed_first` (`PodcastRepository.isFeedFirstShow`, below); the feed address repair reads `upgradeRepair`. - **v2 discovery wiring**: `AppContainer.discoveryEpisodeIds` (`RoomDiscoveryEpisodeIds` over the main database, opened on first use) goes to `PodcastRepository` with `rolloutSwitches`, and the container's `init` installs the same `v2-discovery-off` switch and translator in `DiscoveryRouting`, so the briefing link builders in features and playback and `ChapterRepository` follow the repository: `/v2` routes with episode IDs shown under this phone's own IDs, or the v1 routes while the switch is set ([`core/catalog/README.md`](../core/catalog/README.md)). - **Feed reporting** (`sync/AnalyticsFeedTelemetry`): `AppContainer.feedTelemetry` is the `FeedTelemetryPort` handed to catalog code, so feed reads and address swaps reach `feed_fetch_result` and `feed_url_corrected` without catalog depending on analytics. A reporting failure never reaches the caller (`AnalyticsFeedTelemetryTest`). `SubscriptionRepository` gets it through `FeedAddressWiring`, so every saved-address correction of a followed show sends one `feed_url_corrected`. - **Saved feed addresses** (P4 PR 14): `AppContainer` wires the one address writer. `subscriptionRepository` gets `FeedAddressWiring` (re-registration through `fcm/PodcastNotificationRegistrationWorker`, `feedTelemetry`, and pending registrations in SharedPreferences `boxlore_notification_reregistrations`); `subscribedEpisodeCatalog` follows moved feeds on every refresh path (`FeedMoveFollower`, behind `feed-fix-moved-feed-off`); `autoDownloadCoordinator` saves recovered addresses through `ensureHttpsFeedUrl`; cloud pulls save addresses through `subscriptionRepository.feedAddresses`; `subscriptionForegroundSync` gets `FeedAddressUpkeep` for the daily address check and broken-feed reports (report times in SharedPreferences `boxlore_feed_reports`, never sent while `connectivityStatus` says offline), behind `address-check-off`. `PodcastNotificationRegistrationWorker` registers a Podcast Index show's notifications again after its address changed and waits for Firebase to acknowledge the node and topic: unique work `podcast-notification-registration-`, REPLACE (it reads the row when it runs), network-gated, exponential backoff from 30 s, at most 8 runs. It records each outcome (`settle`): a show it gives up on stays pending and the next foreground pass queues it again; `rss:` shows keep their own journal worker. - **Feed-first show pages** (P4 PR 16): `AppContainer.podcastRepository` gets `FeedFirstWiring`: the stored copies (`localEpisodeCatalogRepository.showPages`), `connectivityStatus` for the offline rung, one broken-feed report limiter (`feedReportLimiter` over the `boxlore_feed_reports` stamps, the same instance `FeedAddressUpkeep.reportLimiter` gives the daily address check, so both claim inside one daily limit), `feedTelemetry` for `feed_fetch_result`, and `isFollowed` (`podcasts.isSubscribed`), so the show page leaves followed shows' reports to their own refresh. The page reads the feed only while `feed_first` is on (`RolloutSwitches.feedFirst`, fixed per process). +- **Feed address repair after the update** (P4 PR 15, `sync/UpgradeRepairWorker`): a thin WorkManager shell around `:core:catalog` `UpgradeAddressRepair` (`AppContainer.upgradeAddressRepair`, reached through `SharedAppDependenciesHolder`). Every launch, headless ones included, `BoxLoreApplication` calls `UpgradeRepairWorker.scheduleIfNeeded`, which enqueues unique work `feed-address-upgrade-repair` (KEEP, network-gated, exponential backoff from 30 s) only while the stored state says work is left: followed shows with repair steps left, or step 4 not yet recorded in `noBackupFilesDir/upgrade_repair_notifications`. A fresh install, or one with only `rss:` follows, settles without network and enqueues nothing. A restore brings Room but not WorkManager's queue or the step 4 record, so the next launch enqueues the repair again and every followed show's notifications are registered again. A run maps `DONE` and `PAUSED` (`upgrade-repair-off`) to success and `RETRY` to a retry; a launch's chain stops after 10 runs and the next launch enqueues it again. `AppContainer` wires `/v2/shows/feeds` (`AddressCheckAnswer.fetchWith`), the one address writer, the catalog re-read, `PodcastNotificationRegistrationWorker.request` (returns once WorkManager has saved the request, so step 4 never records one lost to process death), `userSyncCoordinator.requestIdentityCatchUp` (once every followed show was read again, the next sync runs the identity catch-up again), `connectivityStatus` and `feedTelemetry`. - NPS survey UI (`surveys/`) applies `NpsSurveyBranching` after each answer so detractor / passive / promoter open-text paths end instead of falling through into the next score band (PostHog open-question `end` branching is not reliably persisted via API). - JVM `ArtworkPredictiveBackBackgroundTest` checks actual native rendering of the Back wrapper with changing artwork backgrounds beneath the saved app theme. - JVM `ArtworkNavigationHostThemeTest` uses a real NavHostController and the production slide transitions to verify source capture before first loader composition, stable saved-theme fallback, Back loader inheritance and separation of Home/regular-page colors from transient loader accents. @@ -164,7 +165,7 @@ Routes include onboarding, home, learn, briefing, settings, debug, explore, libr Unacknowledged tablet launches first show an opaque alpha notice with the Home feedback icon and OK. The Activity latches this initial-window decision through recreation and resize. It needs both the tablet window gate and a display whose shortest side is at least 600dp, so phones never show the notice or read its acknowledgment, even when a landscape window at a small display size passes the tablet gate. The app root, Welcome/onboarding owners and sign-in portal are not composed until OK, which persists a separate local acknowledgment. Cached appearance settings theme the notice, and its body scrolls in short fold-safe regions. NPS, queued SDK surveys and update presentation wait behind this gate; post-OK prompts resume after app frames establish playback state. The visible notice releases ordinary deferred startup work after its own first frames without creating Welcome. -`StartupWorkGate` is process-scoped and releases optional cloud coordinator initialization, periodic sync scheduling, Smart Downloads scheduling/catch-up, foreground subscription sync and legacy RSS repair after the initial screen commits. Optional initialization uses a suspending mutex so ready does not release a burst of initializers. Home signals after loaded local content and two frames; other initial routes release after their first two frames. A headless worker/widget launch releases at the first idle main queue; `MainActivity` identifies a UI launch before creating content. If the first Activity stops before content is ready, background startup also releases; configuration recreation waits for the replacement UI. Recreated roots reuse the same gate. Repeated signals and cancelled initializers are safe (`StartupWorkGateTest`). Adaptive ranking telemetry also waits for readiness. Appearance flows use restored fast-cache values while background hydration fills missing DataStore keys; existing choices are never overwritten. Play update checks begin after the first two Compose frames. Playback/session restoration and preference identity remain owned by the existing composition root. +`StartupWorkGate` is process-scoped and releases optional cloud coordinator initialization, periodic sync scheduling, Smart Downloads scheduling/catch-up, the feed address repair after the update (`UpgradeRepairWorker.scheduleIfNeeded`), foreground subscription sync and legacy RSS repair after the initial screen commits. Optional initialization uses a suspending mutex so ready does not release a burst of initializers. Home signals after loaded local content and two frames; other initial routes release after their first two frames. A headless worker/widget launch releases at the first idle main queue; `MainActivity` identifies a UI launch before creating content. If the first Activity stops before content is ready, background startup also releases; configuration recreation waits for the replacement UI. Recreated roots reuse the same gate. Repeated signals and cancelled initializers are safe (`StartupWorkGateTest`). Adaptive ranking telemetry also waits for readiness. Appearance flows use restored fast-cache values while background hydration fills missing DataStore keys; existing choices are never overwritten. Play update checks begin after the first two Compose frames. Playback/session restoration and preference identity remain owned by the existing composition root. - Release runtime dependency snapshots include Jsoup through `:core:catalog`, which owns shared show-notes resource and chapter parsing. No direct parser-library dependency is added here. `BoxLoreApplication.onCreate` never builds `PlaybackRepository` for Cast state. `AppContainer.synchronizeCastSession` keeps the latest state in `DeferredCastSessionState` and applies it when playback is first created, so widget, worker and notification starts no longer construct playback (prefs migration, MediaController bind) on the main thread. PostHog setup runs on a background thread through `:core:analytics` `PostHogStartup`, and environment super-properties are registered before queued calls replay. Navigation string arguments encode the literal text `null` as the `_` placeholder, because Navigation reads it as a null argument. @@ -190,7 +191,8 @@ Unacknowledged tablet launches first show an opaque alpha notice with the Home f - `applicationId` is `cx.aswin.boxlore`. - Manifest service, receiver, activity, and worker class names are system-facing identities. - `LegacyWorkerFactory` preserves WorkManager upgrades from legacy worker class names. -- The unique work names `episode-share-registration-` (`EpisodeShareRegistrationWorker`) and `podcast-notification-registration-` (`fcm/PodcastNotificationRegistrationWorker`) are permanent. +- The unique work names `episode-share-registration-` (`EpisodeShareRegistrationWorker`), `podcast-notification-registration-` (`fcm/PodcastNotificationRegistrationWorker`) and `feed-address-upgrade-repair` (`sync/UpgradeRepairWorker`) are permanent. +- The feed address repair's step 4 record lives in `noBackupFilesDir/upgrade_repair_notifications` (`FileUpgradeNotificationMarker`), so a restore never brings it and the restored phone registers notifications again; stable name. - SharedPreferences file `boxlore_feed_reports` (`SharedPreferencesFeedReportStamps`, opened by `AppContainer`) holds broken-feed report times; stable name. - Build config reads `BOXLORE_API_BASE_URL`, `BOXLORE_PUBLIC_KEY`, and the public `BOXLORE_CAST_RECEIVER_ID`, with legacy API/key local-property names still accepted for existing developer environments. - Core modules own Room filenames, DataStore names, SharedPreferences names, ranking identity, RSS IDs, and playback media IDs. @@ -202,7 +204,7 @@ Episode push artwork keeps its six-second total deadline and uses 700ms per rout ## Testing notes -- `EpisodeShowLinksTest` opens every episode link on the real route and arguments: `?show=` from the share page and `&show=` share links on each share address, with and without `t`, a clip or `autoplay`, links the app builds, older links without a show, and the notifier's route. `WebAppLinksManifestTest` also checks every episode share pattern, with and without the show, is on a claimed host and path. `FcmPayloadParserTest`, `NewEpisodePushHydrationTest` and `NewEpisodeFcmLogicTest` cover the push fields, GUID/enclosure before catalog key and recipe ID, and the recipe-ID route. `EpisodeShareRegistrationWorkerTest` covers the unique name (unchanged by the audio address a GUID request carries), constraints, backoff, the attempt cap and invalid input. `PodcastNotificationRegistrationWorkerTest` covers its unique name per show (none for `rss:` shows, a newer request replacing the waiting one), constraints, backoff, the attempt cap, errors counted as retries and work without a show. `NewEpisodeDeliveryWorkerTest` covers delivery names with and without a recipe ID or catalog key. `EpisodeShowLinksTest` also opens install-referrer routes with and without a show, and `DeepLinkAutoplayPlanTest` covers autoplay and seek for a link that resolves to the phone's own ID and a stale page for another link. +- `EpisodeShowLinksTest` opens every episode link on the real route and arguments: `?show=` from the share page and `&show=` share links on each share address, with and without `t`, a clip or `autoplay`, links the app builds, older links without a show, and the notifier's route. `WebAppLinksManifestTest` also checks every episode share pattern, with and without the show, is on a claimed host and path. `FcmPayloadParserTest`, `NewEpisodePushHydrationTest` and `NewEpisodeFcmLogicTest` cover the push fields, GUID/enclosure before catalog key and recipe ID, and the recipe-ID route. `EpisodeShareRegistrationWorkerTest` covers the unique name (unchanged by the audio address a GUID request carries), constraints, backoff, the attempt cap and invalid input. `UpgradeRepairWorkerTest` covers the repair's permanent unique name, constraints and backoff, KEEP across launches, no work for a fresh install or `rss:`-only follows, result mapping with the per-launch cap, errors counted as retries, and a run of the real repair to the end; `AppContainerSmokeTest` checks the container hands workers one repair and settles a fresh install. `PodcastNotificationRegistrationWorkerTest` covers its unique name per show (none for `rss:` shows, a newer request replacing the waiting one, a request returning once the work is saved), constraints, backoff, the attempt cap, errors counted as retries and work without a show. `NewEpisodeDeliveryWorkerTest` covers delivery names with and without a recipe ID or catalog key. `EpisodeShowLinksTest` also opens install-referrer routes with and without a show, and `DeepLinkAutoplayPlanTest` covers autoplay and seek for a link that resolves to the phone's own ID and a stale page for another link. - Tablet context details use their native Back toolbar and keep docked playback in the browsing pane. The context player can be minimized independently. Fullscreen media requests share an app-owned window host across docked/context presentations, restoring the originating presentation when closed. diff --git a/app/src/main/java/cx/aswin/boxlore/AppContainer.kt b/app/src/main/java/cx/aswin/boxlore/AppContainer.kt index 67623470c..406dce16d 100644 --- a/app/src/main/java/cx/aswin/boxlore/AppContainer.kt +++ b/app/src/main/java/cx/aswin/boxlore/AppContainer.kt @@ -2,6 +2,7 @@ package cx.aswin.boxlore import android.content.Context import cx.aswin.boxlore.connectivity.AndroidConnectivityObserver +import cx.aswin.boxlore.core.catalog.AddressCheckAnswer import cx.aswin.boxlore.core.catalog.DiscoveryEpisodeIds import cx.aswin.boxlore.core.catalog.DiscoveryRouting import cx.aswin.boxlore.core.catalog.EpisodeAliasReads @@ -13,6 +14,7 @@ import cx.aswin.boxlore.core.catalog.FeedFirstWiring import cx.aswin.boxlore.core.catalog.FeedFixToggles import cx.aswin.boxlore.core.catalog.FeedMoveFollower import cx.aswin.boxlore.core.catalog.FeedReportLimiter +import cx.aswin.boxlore.core.catalog.FileUpgradeNotificationMarker import cx.aswin.boxlore.core.catalog.FolderRepository import cx.aswin.boxlore.core.catalog.InstallReferrerManager import cx.aswin.boxlore.core.catalog.LegacyRssRepair @@ -31,6 +33,8 @@ import cx.aswin.boxlore.core.catalog.SharedPreferencesFeedReportStamps import cx.aswin.boxlore.core.catalog.SharedPreferencesPendingNotificationRegistrations import cx.aswin.boxlore.core.catalog.SubscriptionForegroundSync import cx.aswin.boxlore.core.catalog.SubscriptionRepository +import cx.aswin.boxlore.core.catalog.UpgradeAddressRepair +import cx.aswin.boxlore.core.catalog.UpgradeRepairPorts import cx.aswin.boxlore.core.catalog.ports.SmartDownloadSyncPort import cx.aswin.boxlore.core.catalog.privacy.ConsentManager import cx.aswin.boxlore.core.catalog.sync.HistorySyncResolver @@ -447,6 +451,33 @@ class AppContainer( ) } + /** + * The one-time repair of saved feed addresses after the update, run by + * [cx.aswin.boxlore.sync.UpgradeRepairWorker] behind `upgrade-repair-off`. Step 4's record lives + * in `noBackupFilesDir/upgrade_repair_notifications`, so a restored phone registers again. + * Once every show was read again, the sync's identity catch-up runs once more. + * Added in P4 PR 15 of the own-episode-IDs plan (plan section 11, upgrade repair). + * COMPAT(own-ids): someone can upgrade from an old build at any time. Permanent: never remove (MIGRATION.md §5). + */ + override val upgradeAddressRepair: UpgradeAddressRepair by lazy { + UpgradeAddressRepair( + repairDao = database.upgradeRepairDao(), + podcastDao = database.podcastDao(), + ports = + UpgradeRepairPorts( + fetchAddresses = AddressCheckAnswer.fetchWith(podcastRepository.apiV2), + replaceAddress = subscriptionRepository::updateFeedAddress, + reread = UpgradeAddressRepair.rereadWith(subscribedEpisodeCatalog), + requestNotificationRegistration = { id -> PodcastNotificationRegistrationWorker.request(appContext, id) }, + onAddressStepsFinished = { userSyncCoordinator.requestIdentityCatchUp() }, + isOnline = connectivityStatus::isOnline, + telemetry = feedTelemetry, + ), + marker = FileUpgradeNotificationMarker(java.io.File(appContext.noBackupFilesDir, FileUpgradeNotificationMarker.FILE_NAME)), + enabled = rolloutSwitches.upgradeRepair, + ) + } + /** * Broken-feed report times, shared by the daily address check and the show page so both stay * inside one daily limit (P4 PR 14 and PR 16). diff --git a/app/src/main/java/cx/aswin/boxlore/BoxLoreApplication.kt b/app/src/main/java/cx/aswin/boxlore/BoxLoreApplication.kt index 829d8686f..05731152d 100644 --- a/app/src/main/java/cx/aswin/boxlore/BoxLoreApplication.kt +++ b/app/src/main/java/cx/aswin/boxlore/BoxLoreApplication.kt @@ -294,6 +294,24 @@ class BoxLoreApplication : container.smartDownloadManager.reconcileScheduleWithPreferences() } } + applicationScope.launch { + startupWorkGate.runWhenReady { scheduleUpgradeRepair() } + } + } + + /** + * Enqueues the one-time repair of saved feed addresses on every launch while it has work left, + * headless launches included; it reads stored state, so a restore starts it again. + * Added in P4 PR 15 of the own-episode-IDs plan (plan section 11, upgrade repair). + * COMPAT(own-ids): someone can upgrade from an old build at any time. Permanent: never remove (MIGRATION.md §5). + */ + private suspend fun scheduleUpgradeRepair() { + runCatching { + cx.aswin.boxlore.sync.UpgradeRepairWorker.scheduleIfNeeded(this, container.upgradeAddressRepair) + }.onFailure { error -> + if (error is kotlinx.coroutines.CancellationException) throw error + android.util.Log.w("BoxLoreApplication", "Feed address repair scheduling failed", error) + } } /** diff --git a/app/src/main/java/cx/aswin/boxlore/fcm/PodcastNotificationRegistrationWorker.kt b/app/src/main/java/cx/aswin/boxlore/fcm/PodcastNotificationRegistrationWorker.kt index 38d7f8a0b..2d28807f3 100644 --- a/app/src/main/java/cx/aswin/boxlore/fcm/PodcastNotificationRegistrationWorker.kt +++ b/app/src/main/java/cx/aswin/boxlore/fcm/PodcastNotificationRegistrationWorker.kt @@ -9,8 +9,10 @@ import androidx.work.ExistingWorkPolicy import androidx.work.NetworkType import androidx.work.OneTimeWorkRequest import androidx.work.OneTimeWorkRequestBuilder +import androidx.work.Operation import androidx.work.WorkManager import androidx.work.WorkerParameters +import androidx.work.await import cx.aswin.boxlore.core.catalog.NotificationRegistrationOutcome import cx.aswin.boxlore.core.catalog.SharedAppDependenciesHolder import java.util.concurrent.TimeUnit @@ -87,8 +89,21 @@ class PodcastNotificationRegistrationWorker(context: Context, params: WorkerPara /** Asks for [podcastId]'s notifications to be registered again; `rss:` shows have their own journal. */ fun enqueue(context: Context, podcastId: String) { - if (podcastId.isBlank() || podcastId.startsWith("rss:")) return - WorkManager.getInstance(context).enqueueUniqueWork(workName(podcastId), ExistingWorkPolicy.REPLACE, workRequest(podcastId)) + enqueueWork(context, podcastId) + } + + /** + * [enqueue], returning once WorkManager has saved the request, so a caller that records it + * (the upgrade repair's step 4) never records one lost to process death. Throws when the + * request couldn't be saved. Added in P4 PR 15 of the own-episode-IDs plan (plan section 11). + */ + suspend fun request(context: Context, podcastId: String) { + enqueueWork(context, podcastId)?.await() + } + + private fun enqueueWork(context: Context, podcastId: String): Operation? { + if (podcastId.isBlank() || podcastId.startsWith("rss:")) return null + return WorkManager.getInstance(context).enqueueUniqueWork(workName(podcastId), ExistingWorkPolicy.REPLACE, workRequest(podcastId)) } } } diff --git a/app/src/main/java/cx/aswin/boxlore/sync/UpgradeRepairWorker.kt b/app/src/main/java/cx/aswin/boxlore/sync/UpgradeRepairWorker.kt new file mode 100644 index 000000000..143c71643 --- /dev/null +++ b/app/src/main/java/cx/aswin/boxlore/sync/UpgradeRepairWorker.kt @@ -0,0 +1,86 @@ +package cx.aswin.boxlore.sync + +import android.content.Context +import android.util.Log +import androidx.work.BackoffPolicy +import androidx.work.Constraints +import androidx.work.CoroutineWorker +import androidx.work.ExistingWorkPolicy +import androidx.work.NetworkType +import androidx.work.OneTimeWorkRequest +import androidx.work.OneTimeWorkRequestBuilder +import androidx.work.WorkManager +import androidx.work.WorkerParameters +import cx.aswin.boxlore.core.catalog.SharedAppDependenciesHolder +import cx.aswin.boxlore.core.catalog.UpgradeAddressRepair +import cx.aswin.boxlore.core.catalog.UpgradeRepairRun +import java.util.concurrent.TimeUnit +import kotlinx.coroutines.CancellationException + +/** + * Runs the one-time repair of saved feed addresses after the update ([UpgradeAddressRepair] in + * `:core:catalog`, reached through [SharedAppDependenciesHolder]). The logic and its stored + * progress live there; this shell only maps a run to a WorkManager result. + * + * Enqueued on every launch while the repair has work left ([scheduleIfNeeded], from + * `BoxLoreApplication`), so a restore, which never brings WorkManager's queue, starts it again. + * Unique work [WORK_NAME] (a permanent identity), KEEP, network-gated, exponential backoff from + * 30 s. A launch's chain ends after [MAX_RUNS] runs; the next launch enqueues it again while + * work is left. + * + * Added in P4 PR 15 of the own-episode-IDs plan (plan section 11, upgrade repair). + * COMPAT(own-ids): someone can upgrade from an old build at any time. Permanent: never remove (MIGRATION.md §5). + */ +class UpgradeRepairWorker(context: Context, params: WorkerParameters) : CoroutineWorker(context, params) { + internal var repair: suspend () -> UpgradeRepairRun = { + SharedAppDependenciesHolder.require().upgradeAddressRepair?.run() ?: UpgradeRepairRun.DONE + } + + override suspend fun doWork(): Result { + val run = + try { + repair() + } catch (error: CancellationException) { + throw error + } catch (error: Exception) { + Log.w(TAG, "Feed address repair failed", error) + UpgradeRepairRun.RETRY + } + return when (run) { + UpgradeRepairRun.DONE, UpgradeRepairRun.PAUSED -> Result.success() + UpgradeRepairRun.RETRY -> if (runAttemptCount + 1 < MAX_RUNS) Result.retry() else Result.success() + } + } + + companion object { + private const val TAG = "UpgradeRepairWorker" + + /** Permanent unique-work name. Never rename. */ + const val WORK_NAME = "feed-address-upgrade-repair" + + /** Runs per launch, the first included. */ + const val MAX_RUNS = 10 + + private const val BACKOFF_SECONDS = 30L + + fun workRequest(): OneTimeWorkRequest = OneTimeWorkRequestBuilder() + .setConstraints(Constraints.Builder().setRequiredNetworkType(NetworkType.CONNECTED).build()) + .setBackoffCriteria(BackoffPolicy.EXPONENTIAL, BACKOFF_SECONDS, TimeUnit.SECONDS) + .build() + + /** Enqueues the repair unless it is already waiting or running. */ + fun enqueue(context: Context) { + WorkManager.getInstance(context).enqueueUniqueWork(WORK_NAME, ExistingWorkPolicy.KEEP, workRequest()) + } + + /** + * Enqueues the repair when it has work left. A fresh install, or one with only `rss:` + * follows, settles here without network and enqueues nothing. + */ + suspend fun scheduleIfNeeded(context: Context, repair: UpgradeAddressRepair): Boolean { + if (!repair.needsWork()) return false + enqueue(context) + return true + } + } +} diff --git a/app/src/test/java/cx/aswin/boxlore/AppContainerSmokeTest.kt b/app/src/test/java/cx/aswin/boxlore/AppContainerSmokeTest.kt index 97dc46e94..d1764133a 100644 --- a/app/src/test/java/cx/aswin/boxlore/AppContainerSmokeTest.kt +++ b/app/src/test/java/cx/aswin/boxlore/AppContainerSmokeTest.kt @@ -138,4 +138,21 @@ class AppContainerSmokeTest { assertTrue(next.addressCheck()) assertTrue(next.v2Discovery()) } + + @Test + fun upgradeRepairIsOneInstanceReachableByWorkersAndSettlesAFreshInstall() = kotlinx.coroutines.runBlocking { + val container = + AppContainer( + context = context, + apiBaseUrl = "https://example.test", + publicKey = "test-key", + sharedUserPreferences = UserPreferencesRepository(context), + ) + val repair = container.upgradeAddressRepair + assertSame(repair, container.upgradeAddressRepair) + assertSame(repair, (container as cx.aswin.boxlore.core.catalog.SharedAppDependencies).upgradeAddressRepair) + + assertFalse("nothing followed: settled without network", repair.needsWork()) + assertTrue(java.io.File(context.noBackupFilesDir, cx.aswin.boxlore.core.catalog.FileUpgradeNotificationMarker.FILE_NAME).exists()) + } } diff --git a/app/src/test/java/cx/aswin/boxlore/fcm/PodcastNotificationRegistrationWorkerTest.kt b/app/src/test/java/cx/aswin/boxlore/fcm/PodcastNotificationRegistrationWorkerTest.kt index ac731847f..ced5ed3bc 100644 --- a/app/src/test/java/cx/aswin/boxlore/fcm/PodcastNotificationRegistrationWorkerTest.kt +++ b/app/src/test/java/cx/aswin/boxlore/fcm/PodcastNotificationRegistrationWorkerTest.kt @@ -54,6 +54,17 @@ class PodcastNotificationRegistrationWorkerTest { assertTrue(WorkManager.getInstance(context).getWorkInfosForUniqueWork(PodcastNotificationRegistrationWorker.workName("rss:abc")).get().isEmpty()) } + @Test + fun `a request returns once the work is saved, and never saves one for rss shows`() = runBlocking { + PodcastNotificationRegistrationWorker.request(context, "920666") + PodcastNotificationRegistrationWorker.request(context, "rss:abc") + + val manager = WorkManager.getInstance(context) + val work = manager.getWorkInfosForUniqueWork(PodcastNotificationRegistrationWorker.workName("920666")).get() + assertEquals(1, work.count { it.state == WorkInfo.State.ENQUEUED }) + assertTrue(manager.getWorkInfosForUniqueWork(PodcastNotificationRegistrationWorker.workName("rss:abc")).get().isEmpty()) + } + @Test fun `the work waits for a network and backs off exponentially`() { val work = PodcastNotificationRegistrationWorker.workRequest("920666").workSpec diff --git a/app/src/test/java/cx/aswin/boxlore/sync/UpgradeRepairWorkerTest.kt b/app/src/test/java/cx/aswin/boxlore/sync/UpgradeRepairWorkerTest.kt new file mode 100644 index 000000000..5cc869aba --- /dev/null +++ b/app/src/test/java/cx/aswin/boxlore/sync/UpgradeRepairWorkerTest.kt @@ -0,0 +1,147 @@ +package cx.aswin.boxlore.sync + +import android.app.Application +import android.content.Context +import androidx.room.Room +import androidx.test.core.app.ApplicationProvider +import androidx.work.BackoffPolicy +import androidx.work.Configuration +import androidx.work.ListenableWorker +import androidx.work.NetworkType +import androidx.work.WorkInfo +import androidx.work.WorkManager +import androidx.work.testing.SynchronousExecutor +import androidx.work.testing.TestListenableWorkerBuilder +import androidx.work.testing.WorkManagerTestInitHelper +import cx.aswin.boxlore.core.catalog.AddressCheckAnswer +import cx.aswin.boxlore.core.catalog.MemoryUpgradeNotificationMarker +import cx.aswin.boxlore.core.catalog.UpgradeAddressRepair +import cx.aswin.boxlore.core.catalog.UpgradeRepairPorts +import cx.aswin.boxlore.core.catalog.UpgradeRepairRun +import cx.aswin.boxlore.core.database.BoxLoreDatabase +import cx.aswin.boxlore.core.database.PodcastEntity +import cx.aswin.boxlore.core.database.UpgradeRepairShowEntity +import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort +import cx.aswin.boxlore.core.network.model.V2ShowAddress +import cx.aswin.boxlore.core.network.model.V2ShowsFeedsResponse +import kotlinx.coroutines.runBlocking +import org.junit.After +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertTrue +import org.junit.Before +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.annotation.Config + +/** + * The shell of the one-time feed address repair (own-episode-IDs P4 PR 15, plan section 11): one + * permanent unique work item, KEEP, network-gated with backoff, enqueued only while work is left, + * and a run mapped to a WorkManager result with a per-launch cap. + */ +@RunWith(RobolectricTestRunner::class) +@Config(sdk = [34], application = Application::class) +class UpgradeRepairWorkerTest { + private val context get() = ApplicationProvider.getApplicationContext() + private lateinit var database: BoxLoreDatabase + + @Before + fun setUp() { + WorkManagerTestInitHelper.initializeTestWorkManager(context, Configuration.Builder().setExecutor(SynchronousExecutor()).build()) + database = Room.inMemoryDatabaseBuilder(context, BoxLoreDatabase::class.java).allowMainThreadQueries().build() + } + + @After + fun tearDown() { + database.close() + } + + @Test + fun `the work has one permanent name, waits for a network and backs off exponentially`() { + assertEquals("feed-address-upgrade-repair", UpgradeRepairWorker.WORK_NAME) + val work = UpgradeRepairWorker.workRequest().workSpec + assertEquals(NetworkType.CONNECTED, work.constraints.requiredNetworkType) + assertEquals(BackoffPolicy.EXPONENTIAL, work.backoffPolicy) + assertEquals(30_000L, work.backoffDelayDuration) + } + + @Test + fun `every launch with work left keeps the one waiting item`() = runBlocking { + database.podcastDao().upsertRow(show("920666")) + database.upgradeRepairDao().upsert(UpgradeRepairShowEntity("920666")) + val repair = repair() + + assertTrue(UpgradeRepairWorker.scheduleIfNeeded(context, repair)) + val first = waiting().single().id + assertTrue(UpgradeRepairWorker.scheduleIfNeeded(context, repair)) + + assertEquals("KEEP leaves the waiting run in place", listOf(first), waiting().map { it.id }) + } + + @Test + fun `a fresh install or rss-only follows enqueue nothing`() = runBlocking { + assertFalse(UpgradeRepairWorker.scheduleIfNeeded(context, repair())) + database.podcastDao().upsertRow(show("rss:abc").copy(sourceType = PodcastEntity.SOURCE_RSS)) + assertFalse(UpgradeRepairWorker.scheduleIfNeeded(context, repair())) + + assertTrue(waiting().isEmpty()) + } + + @Test + fun `a run maps to a result, and retries stop at the per-launch cap`() = runBlocking { + assertEquals(ListenableWorker.Result.success(), worker(UpgradeRepairRun.DONE).doWork()) + assertEquals("paused work is enqueued again by the next launch", ListenableWorker.Result.success(), worker(UpgradeRepairRun.PAUSED).doWork()) + assertEquals(ListenableWorker.Result.retry(), worker(UpgradeRepairRun.RETRY).doWork()) + assertEquals(ListenableWorker.Result.retry(), worker(UpgradeRepairRun.RETRY, attempt = UpgradeRepairWorker.MAX_RUNS - 2).doWork()) + assertEquals(ListenableWorker.Result.success(), worker(UpgradeRepairRun.RETRY, attempt = UpgradeRepairWorker.MAX_RUNS - 1).doWork()) + } + + @Test + fun `an error counts as a retry`() = runBlocking { + val failing = TestListenableWorkerBuilder(context).build().apply { repair = { error("database closed") } } + assertEquals(ListenableWorker.Result.retry(), failing.doWork()) + } + + @Test + fun `the worker runs the real repair to the end`() = runBlocking { + database.podcastDao().upsertRow(show("920666")) + database.upgradeRepairDao().upsert(UpgradeRepairShowEntity("920666")) + val repair = repair() + val worker = TestListenableWorkerBuilder(context).build().apply { this.repair = { repair.run() } } + + assertEquals(ListenableWorker.Result.success(), worker.doWork()) + assertEquals(0, database.upgradeRepairDao().pendingCount()) + assertFalse(repair.needsWork()) + } + + private fun worker(run: UpgradeRepairRun, attempt: Int = 0) = + TestListenableWorkerBuilder(context).setRunAttemptCount(attempt).build().apply { repair = { run } } + + private fun waiting() = + WorkManager.getInstance(context).getWorkInfosForUniqueWork(UpgradeRepairWorker.WORK_NAME).get().filter { it.state == WorkInfo.State.ENQUEUED } + + private fun repair() = UpgradeAddressRepair( + repairDao = database.upgradeRepairDao(), + podcastDao = database.podcastDao(), + ports = + UpgradeRepairPorts( + fetchAddresses = { ids -> AddressCheckAnswer.Answered(V2ShowsFeedsResponse(registryAvailable = true, results = ids.map { V2ShowAddress(id = it) })) }, + replaceAddress = { error("no swap expected") }, + reread = { _, _ -> LocalEpisodeCatalogPort.RefreshOutcome.Unchanged(null) }, + requestNotificationRegistration = {}, + ), + marker = MemoryUpgradeNotificationMarker(), + enabled = { true }, + ) + + private fun show(id: String) = PodcastEntity( + podcastId = id, + title = "Show", + author = "Author", + imageUrl = "", + description = null, + isSubscribed = true, + feedUrl = "https://example.com/feed.xml", + ) +} diff --git a/core/analytics/README.md b/core/analytics/README.md index c53fe390f..c091938f6 100644 --- a/core/analytics/README.md +++ b/core/analytics/README.md @@ -13,7 +13,7 @@ Owns analytics event capture and non-fatal error reporting for Boxlore. The modu - `trackApiSessionStatus(ApiSessionStatus)` emits `api_session_status` for API sessions (`ApiSessionAnalyticsTracks`); null properties are left out. One event per `/auth/session` exchange (`kind=exchange`: `trigger`, `success`, `duration_ms`, `attestation`, `limited`, `app_check_attempted`, `failure_reason`, `server_reason`), per first renewal of a token or renewal problem (`kind=renew`, `renew_outcome`) and per Turnstile step-up (`kind=step_up`, `step_up_outcome`). Properties are categories only, never tokens, ids or keys. `trackAppCheckStatus` takes an optional `reason` (a short failure category such as `play_integrity_-9`); App Check now runs only inside a session exchange, so `app_check_status` arrives once per attempt rather than once per launch. -- `trackFeedFetchResult(FeedFetchReport)` emits `feed_fetch_result`: once per show-page read (`context=show_page`, `show_id`, `source`, `result`, `completeness` on successful reads, `http_status` only on `http_error` and `gone`) and once per background pass and source as a summary (`context=background_pass`, `result` `ok`/`mixed`/`failed`, `shows_count`, `ok_count`, `not_modified_count`, `failed_count`; a pass that read nothing sends nothing). `trackFeedUrlCorrected(FeedUrlCorrection)` emits `feed_url_corrected` once per saved-address swap (`show_id`, `reason`, `step`). Neither carries an address: a show ID that looks like one is left out, and `feed_url_corrected` is then dropped (`FeedAnalyticsTracks`, `FeedAnalyticsTracksTest`). Catalog code reaches them through `:core:domain`'s `FeedTelemetryPort`, which `:app` wires; nothing emits them yet. +- `trackFeedFetchResult(FeedFetchReport)` emits `feed_fetch_result`: once per show-page read (`context=show_page`, `show_id`, `source`, `result`, `completeness` on successful reads, `http_status` only on `http_error` and `gone`) and once per background pass and source as a summary (`context=background_pass`, `result` `ok`/`mixed`/`failed`, `shows_count`, `ok_count`, `not_modified_count`, `failed_count`, and `namespace_mismatch_count` only when the server reported another episode namespace for shows read in the pass; a pass that read nothing sends nothing). `trackFeedUrlCorrected(FeedUrlCorrection)` emits `feed_url_corrected` once per saved-address swap (`show_id`, `reason`, `step`). Neither carries an address: a show ID that looks like one is left out, and `feed_url_corrected` is then dropped (`FeedAnalyticsTracks`, `FeedAnalyticsTracksTest`). Catalog code reaches them through `:core:domain`'s `FeedTelemetryPort`, which `:app` wires. - `AnalyticsGlossary.EPISODE_ID_EVENTS` lists the 25 events that carry `episode_id`; each also declares an optional `recipe_id` (`RECIPE_ID_PROPERTY`), and `home_surface_impression` declares `recipe_ids` (`RECIPE_IDS_PROPERTY`). The façades do not send them yet. `GlossaryRecipeIdTest` keeps the set, the CSV and the Markdown glossary in step. - `Analytics` is the interface used by production code and tests. - `AnalyticsHelper` is the PostHog-backed production singleton. diff --git a/core/analytics/src/main/java/cx/aswin/boxlore/core/analytics/FeedAnalyticsTracks.kt b/core/analytics/src/main/java/cx/aswin/boxlore/core/analytics/FeedAnalyticsTracks.kt index 032e35c1d..e899c0a52 100644 --- a/core/analytics/src/main/java/cx/aswin/boxlore/core/analytics/FeedAnalyticsTracks.kt +++ b/core/analytics/src/main/java/cx/aswin/boxlore/core/analytics/FeedAnalyticsTracks.kt @@ -55,15 +55,17 @@ internal object FeedAnalyticsTracks { } } - private fun backgroundPassProperties(report: FeedFetchReport.BackgroundPass): Map = mapOf( - "context" to CONTEXT_BACKGROUND_PASS, - "source" to report.source.wireName, - "result" to report.passResult.wireName, - "shows_count" to report.showsCount, - "ok_count" to report.okCount.coerceAtLeast(0), - "not_modified_count" to report.notModifiedCount.coerceAtLeast(0), - "failed_count" to report.failedCount.coerceAtLeast(0), - ) + private fun backgroundPassProperties(report: FeedFetchReport.BackgroundPass): Map = buildMap { + put("context", CONTEXT_BACKGROUND_PASS) + put("source", report.source.wireName) + put("result", report.passResult.wireName) + put("shows_count", report.showsCount) + put("ok_count", report.okCount.coerceAtLeast(0)) + put("not_modified_count", report.notModifiedCount.coerceAtLeast(0)) + put("failed_count", report.failedCount.coerceAtLeast(0)) + // Sent only when a show's namespace differs (decision D12), so ordinary passes stay as they were. + if (report.namespaceMismatchCount > 0) put("namespace_mismatch_count", report.namespaceMismatchCount) + } /** Show IDs are Podcast Index numbers or hashed `rss:` IDs; anything with a scheme or host is dropped. */ private fun safeShowId(showId: String): String? = showId.trim().takeIf { it.isNotEmpty() && "/" !in it && "." !in it } diff --git a/core/analytics/src/test/java/cx/aswin/boxlore/core/analytics/FeedAnalyticsTracksTest.kt b/core/analytics/src/test/java/cx/aswin/boxlore/core/analytics/FeedAnalyticsTracksTest.kt index 76b552f3a..0a7ec8cb1 100644 --- a/core/analytics/src/test/java/cx/aswin/boxlore/core/analytics/FeedAnalyticsTracksTest.kt +++ b/core/analytics/src/test/java/cx/aswin/boxlore/core/analytics/FeedAnalyticsTracksTest.kt @@ -134,6 +134,15 @@ class FeedAnalyticsTracksTest { ) } + @Test + fun `a background pass counts shows whose namespace differs only when there are some`() { + AnalyticsHelper.trackFeedFetchResult(FeedFetchReport.BackgroundPass(FeedFetchSource.FEED, 2, 0, 0, namespaceMismatchCount = 1)) + AnalyticsHelper.trackFeedFetchResult(FeedFetchReport.BackgroundPass(FeedFetchSource.FEED, 2, 0, 0)) + + assertEquals(1, recorder[0].second["namespace_mismatch_count"]) + assertFalse("namespace_mismatch_count" in recorder[1].second) + } + @Test fun `a background pass result is ok without failures and failed when every read failed`() { AnalyticsHelper.trackFeedFetchResult(FeedFetchReport.BackgroundPass(FeedFetchSource.FEED, 3, 4, 0)) diff --git a/core/catalog/README.md b/core/catalog/README.md index 5d3f89186..e0be6eac3 100644 --- a/core/catalog/README.md +++ b/core/catalog/README.md @@ -15,6 +15,7 @@ Failed consent writes keep activation off, and local notification flags prevent - `PodcastNotificationRegistrar` (`SubscriptionRepository.registerPodcastNotifications`) registers a Podcast Index show's notifications again and waits for Firebase to acknowledge both the `tracked_podcasts/` node and the `new_ep_` topic, the same node and topic as turning notifications on. The node is one per show for every follower and its write replaces it whole, so it is written only with an address (once the show has a latest episode); otherwise only the topic is subscribed. `NotificationRegistrationOutcome.RETRY` is retried by the app's capped background job; rules that refuse the write give `DROPPED`. - **Daily address check.** `FeedAddressDailyCheck` runs at the start of each foreground pass (behind `RolloutSwitches.addressCheck`). It asks `POST /v2/shows/feeds` for every followed Podcast Index show with a saved address not checked in the last day, in chunks of at most 500, and swaps an address only for a confirmed move: the registry flags the show for address repair, the address is the registry's own (`registry:` source), HTTPS (an http-only address is never adopted, since background reads need HTTPS), different from the saved one, and not the address the saved one replaced (whatever replaced it: a feed move, dead-address recovery or a cloud pull). `fetchUrl: null` keeps the saved address. The new address is read first (`DirectFeedRecovery.addressRead`, a catalog refresh from it) and saved only when that read works, so an address the phone can't read never replaces one it can. Each show the registry answered for is stamped (`podcasts.feedUrlCheckedAt`): a swap writes the stamp with the address, and any other show is stamped after its swap had its chance. A failed call or `registryAvailable: false` stamps and swaps nothing. Follows with a change made on this phone that the account hasn't received yet (dirty and synced before) wait for the push; follows never synced to an account are checked. A different `epNs` is only logged (D12). - **Broken-feed reports.** `FeedReporter` sends `POST /v2/feed-report` (`{id, status}`, never an address) for every failed background read of a Podcast Index show (`RefreshOutcome.Failure.reportStatus`), behind the same switch, never while offline, at most once per show a day and 20 a day in all, the API's own limit (`FeedReportLimiter`, one per app, shared with the show page, which stops 5 short so followed shows always have room; times kept in `FeedReportStamps`, the app's `SharedPreferencesFeedReportStamps`). Only a report the API answered counts: a failed send or a 429 uses up nothing. Its answer is an address to try only when it differs, is HTTPS and the saved address looks dead. +- **Feed address repair after the update.** `UpgradeAddressRepair` (`SharedAppDependencies.upgradeAddressRepair`, run by the app's `UpgradeRepairWorker`) repairs the saved address of every Podcast Index show followed when the phone upgraded; the v39 upgrade seeds one pending `upgrade_repair_shows` row each, a fresh install none. Per show: (1) `POST /v2/shows/feeds` in chunks of at most 500 (`UpgradeAddressCheckStep`, production request `AddressCheckAnswer.fetchWith`); the answer stamps the daily check and records `checkedFetchUrl` only for a confirmed move (`FeedAddressDailyCheck.confirmedMove`), plus the transport address, source and the server's `epNs`. A chunk the server couldn't answer (503, 429, another 5xx, no connection, or `registryAvailable: false`) stays at step 0 without counting a try while the other chunks go on; one it refused (another 4xx) or whose answer couldn't be read counts a try for each show. A show refused 5 times, or unanswered for 7 days since its first failed check (`MAX_UNANSWERED_MS`), goes on without an answer: no swap, and the daily check looks after its address. (2) The swap through the one address writer (`feed_url_corrected`, step `upgrade_repair`; skipped for follows changed on the phone and not pushed yet, which the daily check handles once pushed). An answer more than a day old is asked again at the start of a run (a repair paused or stopped for a while never applies an old answer), and one the daily check has since replaced with a newer answer is not applied. (3) A catalog re-read of the saved address (`rereadWith`; the server's namespace is passed only for comparison, so issued IDs never change, and moves found are followed by the catalog). It never starts the foreground auto-download pass (`runPostPersistCallback = false`): the regular background pass applies the listener's background download rules. (4) Notification re-registration for every followed show with notifications on whose own steps are over, recorded in `UpgradeNotificationMarker` (`FileUpgradeNotificationMarker`, `noBackupFilesDir`) only once the app reports the request saved, so a restored phone registers again even when Room says steps 1–3 are done. Once no show has steps 1–3 left, `UpgradeRepairPorts.onAddressStepsFinished` runs once per install (recorded in the same file; never on a fresh install, while shows wait, or while paused), which the app wires to `UserSyncCoordinator.requestIdentityCatchUp`, so history and queue rows whose episodes the reads settled reach the cloud with their identity. Each step is written as it finishes, so process death resumes at the next one; the row is read again before each step, and a show no longer followed ends `done`. A failed re-read while online counts against a cap of 5 tries (`gave_up`); failures while offline never count and stop the run. Shows followed after the upgrade get no row. `needsWork()` settles a fresh install, or one with only `rss:` follows, without network. Behind `RolloutSwitches.upgradeRepair`: off returns `PAUSED` without marking anything done. One `feed_fetch_result` background summary (`source=feed`) per run that read feeds, with `namespaceMismatchCount` for reads of shows whose server namespace differs from the phone's (decision D12). - RSS notification activation requires explicit public-feed disclosure acceptance through `setNotificationsEnabled(..., acceptRssDisclosure = true, disclosedFeedUrl = savedUrl)`. `DeviceRssNotificationConsent` stores only URL fingerprints in `noBackupFilesDir`; changed URLs, restore and remote cloud settings cannot supply acceptance. Canonical `rss:` IDs require disclosure even with stale source metadata. Refused or revoked activation also removes this device's old registration; a library cleared during acceptance cannot publish a deleted show. Notification settings remain independent of auto-download. The shared checker receives only explicitly accepted public feeds. `TrackedPodcastRtdbLogic.topic` maps `rss:` to FCM-safe `rss_` while retaining canonical IDs in payloads. JSON/cloud RSS restore retains the original saved show ID across redirects. - RSS topics and per-device registration keys include a SHA-256 hash of the exact trimmed accepted URL. `DeviceRssNotificationRegistrations` keeps an atomic publication/cleanup journal in `noBackupFilesDir`, containing fingerprints and keys rather than URLs. Activation fails closed if the journal cannot be saved. `RssNotificationReconciler` awaits both RTDB and FCM acknowledgement, retains active entries and failed cleanup obligations, and retires old URL scopes or deleted/disabled shows. Live RTDB rules refuse whole-node deletes, so cleanup keeps the node and removes only its `feedUrl` child, which the checker needs to poll an RSS scope. The rules refuse that delete only when no node exists for the key (for example a legacy key this device never published), so a refusal still unsubscribes the FCM topic and clears the journal entry; any other failure keeps both. Migration scans disabled and unsubscribed canonical RSS rows too, removes legacy device keys/topics, and rejects restored notification flags without local consent. `canPresentEpisodeNotification` verifies the current saved URL, subscription, enabled flag and device-local acceptance. @@ -34,7 +35,7 @@ Failed consent writes keep activation off, and local notification flags prevent - **Caches.** `/v2` recommendation and because-you-like answers are cached under `v2|` keys, so a v1 answer never serves a v2 request or the reverse. - `RoomDiscoveryEpisodeIds` is the alias-table implementation (`episode_aliases`, trusted aliases only; a recipe ID two keys of one show share stays as it is; the audio-address match above); the app passes it as `discoveryEpisodeIds`. `DiscoveryRouting.install` gives the same switch and translator to code outside the repository: `BriefingLinks` builds the briefing chapters link on `/v2/briefings/chapters/` while v2 discovery is on (the transcript link keeps `/briefings/transcript/`, as the `/v2` briefing answer links it), and `ChapterRepository` translates related episodes of the chapters it fetches. - `RecommendationsV2Request` always sends `contractVersion: 2`, which the `/v2` recommendation bodies need. With the switch off, personal picks go straight to contract-1 `/recommendations` (`v1PersonalizedRecommendations`), exactly what shipped builds received: their v1 `recommendations/v2` request was always refused, so the switch restores the shipped behaviour. -- `SubscriptionForegroundSync` refreshes subscribed latest episodes (`syncSubscriptions` in chunks of 10). Cold start from Home still waits 2s for first paint; **Subscriptions appear** call `requestRefresh()`. Each pass starts with `sweepExpired` so unsubscribed catalogs past the 14-day TTL are deleted. PI `/sync` runs for shows that are not catalog-ready. Every subscribed PI show with an HTTPS `feedUrl` refreshes the local catalog (HEAD/quiet skip for ready rows; not-ready rows force the repair GET; one-time PI rematch only while `needsFullBackfill`). Missing HTTPS URLs are recovered via `GET /podcast` at most once per day, with bounded concurrency; one failed recovery does not abort the rest of the pass. A show whose own feed failed this pass is handled after the feed reads (`DirectFeedRecovery.recover`): the failure is reported (above), and when it says the saved address looks dead (`RefreshOutcome.Failure.addressGone`: 401/403/404/410/451, no such host, a failing certificate, or not a feed), the report's address is tried, else `DirectFeedRecovery.recheckGoneFeedAddress` asks `GET /podcast` for the server's confirmed address. That re-check is the daily check's fallback: it shares its per-show stamp (`feedUrlCheckedAt`), so it runs only for shows the daily check couldn't reach and never twice a day (the lookup counts only once the server answers); a different HTTPS address is read at once and saved only when that read works; shows still failing take their latest episode from `/sync` through the ordinary save, which never replaces a newer tip. A feed that moved for good (`RefreshOutcome.Success.movedTo`) has its new address saved. Never for `rss:` shows. `FeedFixToggles` (`addressRecheck`, `syncFallback`, `movedFeed`) are the app's PostHog kill switches; each stays on without an answer. `PodcastRepository.rolloutSwitches` (`RolloutSwitches`) carries the v2 build's switches from the app: `feedFirst` (show pages read the show's own feed instead of the server's list; off by default, and the app fixes it per process), and `upgradeRepair`, `addressCheck` and `v2Discovery` (on by default, and the app uses the last saved answer while PostHog is starting; off pauses the upgrade repair or the daily address check without marking them done, or sends discovery back to the v1 routes). `v2Discovery` is read by the discovery calls above and `addressCheck` by the daily check and the broken-feed reports; `upgradeRepair` has no reader in this module yet. Automatic foreground feed work selects at most 10 due shows per pass, rotates past failures to prevent starvation, prioritizes the visible show, and caps concurrency at 2. Ready feeds share the RSS layer's persisted six-hour cooldown. The 15-minute timer pauses outside the foreground and leaving foreground cancels its in-flight automatic pass; the five-minute pass cooldown still coalesces repeated screen requests. A successful persist emits `directFeedRefreshed`. Subscribe calls `requestCatalogIngest` on the same application scope so the first publisher-feed persist is not cancelled when Podcast Info is popped; it ignores `requestRefresh` cooldown / in-flight gates. `catalogIngestFinished` emits when that one-shot ends (success, skip, or failure). The same pass patches RTDB `feedUrl` for notifying HTTPS shows. `:app` starts it after onboarding; Library `onScreenResume` calls `requestRefresh`. +- `SubscriptionForegroundSync` refreshes subscribed latest episodes (`syncSubscriptions` in chunks of 10). Cold start from Home still waits 2s for first paint; **Subscriptions appear** call `requestRefresh()`. Each pass starts with `sweepExpired` so unsubscribed catalogs past the 14-day TTL are deleted. PI `/sync` runs for shows that are not catalog-ready. Every subscribed PI show with an HTTPS `feedUrl` refreshes the local catalog (HEAD/quiet skip for ready rows; not-ready rows force the repair GET; one-time PI rematch only while `needsFullBackfill`). Missing HTTPS URLs are recovered via `GET /podcast` at most once per day, with bounded concurrency; one failed recovery does not abort the rest of the pass. A show whose own feed failed this pass is handled after the feed reads (`DirectFeedRecovery.recover`): the failure is reported (above), and when it says the saved address looks dead (`RefreshOutcome.Failure.addressGone`: 401/403/404/410/451, no such host, a failing certificate, or not a feed), the report's address is tried, else `DirectFeedRecovery.recheckGoneFeedAddress` asks `GET /podcast` for the server's confirmed address. That re-check is the daily check's fallback: it shares its per-show stamp (`feedUrlCheckedAt`), so it runs only for shows the daily check couldn't reach and never twice a day (the lookup counts only once the server answers); a different HTTPS address is read at once and saved only when that read works; shows still failing take their latest episode from `/sync` through the ordinary save, which never replaces a newer tip. A feed that moved for good (`RefreshOutcome.Success.movedTo`) has its new address saved. Never for `rss:` shows. `FeedFixToggles` (`addressRecheck`, `syncFallback`, `movedFeed`) are the app's PostHog kill switches; each stays on without an answer. `PodcastRepository.rolloutSwitches` (`RolloutSwitches`) carries the v2 build's switches from the app: `feedFirst` (show pages read the show's own feed instead of the server's list; off by default, and the app fixes it per process), and `upgradeRepair`, `addressCheck` and `v2Discovery` (on by default, and the app uses the last saved answer while PostHog is starting; off pauses the upgrade repair or the daily address check without marking them done, or sends discovery back to the v1 routes). `v2Discovery` is read by the discovery calls above and `addressCheck` by the daily check and the broken-feed reports; `upgradeRepair` by the upgrade repair (`UpgradeAddressRepair`). Automatic foreground feed work selects at most 10 due shows per pass, rotates past failures to prevent starvation, prioritizes the visible show, and caps concurrency at 2. Ready feeds share the RSS layer's persisted six-hour cooldown. The 15-minute timer pauses outside the foreground and leaving foreground cancels its in-flight automatic pass; the five-minute pass cooldown still coalesces repeated screen requests. A successful persist emits `directFeedRefreshed`. Subscribe calls `requestCatalogIngest` on the same application scope so the first publisher-feed persist is not cancelled when Podcast Info is popped; it ignores `requestRefresh` cooldown / in-flight gates. `catalogIngestFinished` emits when that one-shot ends (success, skip, or failure). The same pass patches RTDB `feedUrl` for notifying HTTPS shows. `:app` starts it after onboarding; Library `onScreenResume` calls `requestRefresh`. - `LegacyRssRepair` runs once after onboarding for pre-fix OPML rows when enabled by the app-shell build/install gate. It fetches each subscribed `sourceType=rss` feed and accepts only an exact Podcast Index feed-URL variant or podcast GUID; title/author/artwork are never match inputs. Rows with `linkedPodcastIndexId` are skipped because they represent an intentional RSS-source choice. URL/GUID conflicts, lookup failures, active target subscriptions, or incomplete local catalogs leave the RSS row untouched. Successful rows adopt the PI id while retaining RSS episode ids, journaling DataStore id references for crash-safe completion, restoring notification routing for shows that had alerts enabled, and clearing any recommendation cache seeded by the old id. A pass starts only when an eligible row and connectivity are present, publishes `inProgress` for persistent app-shell status UI, and is admitted by a process-local one-shot gate. Three consecutive transient lookup/inspect failures stop the remaining sequential work. Fresh installs can persist the completed marker without starting a pass or showing progress. Once a pass settles without a pending ID transaction, its version marker is written even if individual feeds had transient failures, preventing foreground/banner loops; only a process killed mid-transaction can resume from the pending journal on a later launch. - `content.ContentOrchestrator`, `ServerGroupedSectionProvider`, `ContentContextEngine`, and related content contracts assemble discovery slates from catalog/v3 (no live `content/sections/v1` client). - `content.CuratedMoods` is the shared catalog of curated-mood IDs/titles used by Home daypart rails and Explore For You chips (`getCuratedVibe`). @@ -53,6 +54,7 @@ Failed consent writes keep activation off, and local notification flags prevent - **Pull.** Pulls send `richRemovals: true`. Before the resolvers merge anything, every pulled history and queue row whose ID has no alias gets one under its show: trusted when it names, by catalog key (else by a recipe ID exactly one key of the show has), an episode this phone already holds under a trusted ID, so it joins that episode's projected state; unresolved otherwise, so it is never a new owner and the show's next feed read can still attribute it. An unresolved alias of the same show is settled the same way. Rows of `rss:` shows, Brief IDs, IDs a catalog row holds and IDs this phone's history stores under another show get none. A pulled row of the playing episode under an ID without a row here is stored as the cloud has it (playback writes only the owner row, which is the newest again from its next save). The idle mini-player is handed the pulled episode's owner ID, and `IdleSessionHandoff` in `:core:playback` leaves it alone when it already holds that episode under another ID. A pull from the start by a device that synced before (`executePull`'s `deliveredUntil`, the last sync time) doesn't restore a history row removed here: history removal is local, and a cloud row stored before the last sync that this device no longer has was removed by the listener. - **Queue merge.** `QueueSyncResolver` works per episode: a pulled item of an episode this phone queues keeps the phone's item and ID, each episode is queued once, and the playing episode keeps the ID it plays under. This phone's unsynced removals also drop the pulled item of the same episode under another ID, and the server's plain and rich removals (united; a rich removal counts only while the plain list still holds its ID) also drop this phone's unsynced item of the same episode. A removal whose episode the same side still queues under another ID is stale and matches only its own ID; a pulled item counts by the identity it carries too, so the cloud's re-queue under an ID this phone can't identify yet still makes the cloud's removal stale. The stored rich list stays in step with the plain one: without unsynced changes here it is the server's own list (not merged with this phone's older one), and no stored rich removal names an episode the merged queue holds, by this phone's aliases or by a pulled item's identity, because the server removes every queue row of a rich removal's episode. The merge keeps one entry per episode, so the cloud can still hold an episode under an ID this phone folded into its own entry; a removal on this phone therefore records every ID of the episode (`QueueRepository` through `EpisodeListenerStore.queueRemovalIds`), which also reaches a copy an older build pushed without identity. `QueueItemHydration` builds the row of a pulled item this phone doesn't queue: the catalog and history tiers read the episode under any of its trusted IDs, and the network tier asks `/v2/episode` (through `PodcastRepository.getEpisode` with the show) for the recipe ID the server sent before the stored ID. - **One-time catch-up.** Sync metadata version 3 (`UserSyncCoordinator.IDENTITY_SYNC_METADATA_VERSION`): the first sync pulls from the start, then pushes back the cloud copies that lack an identity this phone knows (`identityCatchUp` in `SyncIdentityCatchUp.kt`): each pulled history row exactly as the cloud sent it, with its own `updatedAt`, plus only the missing recipe ID or catalog key, and the pulled queue the same way (without rich removals, so the server keeps its list) when one of its items gains an identity. The server keeps every other value, other devices' delta pulls receive nothing, and no local row is marked for the push. A copy that names another key or recipe ID, or a row this phone can't identify, is left as it is. The pushes go batch after batch in the same sync (at most `MAX_CATCH_UP_PUSHES`, 100 batches of 100 rows), and version 3 is recorded only after all of them went through; a failed or cut-short run starts again with a pull from the start on the next sync, and copies already caught up are skipped. + - `requestIdentityCatchUp()` sets the version back below 3 on a phone that already caught up (under the sync lock, so a running sync can't record the catch-up done over it); the feed address repair calls it once every followed show was read again, so episodes those reads settled get their identity in the cloud. - The account-switch purge also clears the queue's rich removals, and the switch sets the sync metadata version below 3 and runs the catch-up for the new account right after its pull from the start. Cloud sync doesn't carry the link of an RSS show to the Podcast Index show it replaced yet, so a phone restored from the cloud links no switched show. - `FolderRepository` and `RoomFolderRepository` manage library subscription folders: creating iconless or icon-assigned folders with optional `showPodcastGrid` 1×1 cover style, updating display sizes, restoring folders from JSON backups (`restoreFolder`), auto-organizing subscribed shows into genre folders (with selectable display sizes and cover styles, while preserving custom and existing folders), auto-syncing subscribed shows matching `linkedGenre` (with canonical genre token matching and Tech/Technology synonym normalization), moving shows between folders on genre customization, adding/removing podcasts from folders, and cascading folder deletions without unsubscribing shows. - `:core:rss`, `:core:domain`, `:core:database`, and `:core:prefs` are re-exported where existing public signatures require those types. @@ -156,6 +158,7 @@ Main Kotlin files should remain below 1000 lines; extracted helpers keep reposit - Recommendation interests are trimmed, de-duplicated and capped at 12 (`boundedRecommendationInterests`) on `/recommendations/v2`, the legacy `/recommendations` fallback and the bootstrap recommendations request, matching the API's caps. - `PodcastRepository.getHomeRecommendations` replaces the old `getHomeBootstrapData`: Home's Picked for you and Explore's For you only used its recommendations, so it now sends `include = ["recommendations"]` on `POST /home/bootstrap` and returns `HomeRecommendations(items, isFallback)` without the Brief fallback fetch. Workers without `include` support still reply with every part, which the app ignores. +- `FileUpgradeNotificationMarker` keeps the feed address repair's step 4 record (show IDs registered again, a completion flag and whether the sync catch-up was asked for; a properties file written atomically) in the file the app opens under `noBackupFilesDir` (`upgrade_repair_notifications`), so Android restore never brings it. The repair's per-show progress lives in Room `upgrade_repair_shows` with the stored `UpgradeRepairState` values. Stable storage names and values: never rename. - `SharedPreferencesFeedReportStamps` keeps each show's last broken-feed report time in SharedPreferences file `boxlore_feed_reports` (key: show ID, value: epoch ms; entries older than a day are dropped). `FeedUrlSources` values (`feed_move`, `podcast_lookup`, `cloud_sync`, `feed_report`, `backup_restore`, `registry`) are stored in `podcasts.feedUrlSource`. Stable storage names and values: never rename. - `RecommendationsLastGood` stores the last good `/v2` picks in SharedPreferences file `boxlore_discovery_last_good`, keys `v2_recommendations|personal|` and `v2_recommendations|home|` (episode JSON as the answer named them) plus a `|fallback` key each. Stable storage names: never rename. - Main Room database identity is owned by `:core:database` and exposed through catalog APIs. @@ -174,6 +177,7 @@ Main Kotlin files should remain below 1000 lines; extracted helpers keep reposit - Episode links and shares: `EpisodeLinkResolverTest` (resolution order, old IDs to the phone's own row, evicted aliases, recipe IDs with and without a show, API answers translated to local rows, an old Podcast Index link answered with its recipe ID opening the recipe-ID row, a v1 answer matched by enclosure, missing vs unreachable and the HTTP status mapping, `needsFeed` hit and miss, stored namespace, contract keys only, a duplicated feed item), `EpisodeLinkRoomTest` (Room aliases: trusted only, recipe collisions, namespace; recorded `/v2/episode` `needsFeed` answer read through `/v2/feed/` and parsed; a recorded `/v2/episode` Podcast Index answer resolving to a stored recipe-ID row by key and by alias; an enclosure-keyed row and the v1 fallback when v2 is unavailable; `getEpisode` never reads the feed; 404 vs 429 vs 503; push match by recipe ID), `EpisodeSharePreparerTest` (alias targets, RSS shows linked to a Podcast Index show, enclosure keys, phone-only IDs shared by their key's recipe ID, the offline and slow paths going out at once with the retry scheduled, dropped requests, `rss:` and keyless episodes, the recorded register body and every answer class). `InstallReferrerManagerTest` covers a share link's `show` in the install referrer. - Existing coverage includes `PodcastRepositoryCatalogTest` (including the extras' 1,000-item `episodes/match` list with 150 episodes, its page fallback when the route fails or returns nothing, its off switch, and recipe-range lookups on `/v2/episode` with and without a show, served IDs kept as asked, `needsFeed`/404/errors never falling back to Podcast Index, and Podcast Index IDs on `/v2/episode` with v1 only when the v2 route can't answer), `PodcastRepositoryEpisodeSupplementTest` (offset-0 merge, later-page PI-only, `mergeSupplements = false` PI baseline, search union, opted-in `/sync` skip), `LegacyRssRepairLogicTest` (URL/GUID-only acceptance, conflicts, transient failures), `LatestEpisodeTipLogicTest`, `TrackedPodcastRtdbLogicTest` (HTTPS `feedUrl` on notification RTDB rows), `ShowSearchMergeTest`, `DirectFeedSyncOrderTest`, `LibraryBackupDirectFeedLogicTest` / `LibraryBackupDirectFeedRestoreTest` (JSON v6 opt-in export targeting, Gson-null skip, restore-before-`/sync` plan, RTDB patch before feed refresh), `LibraryBackupManualOrderPinsTest` (Manual sort + id lists round-trip; older JSON without lists), `LibraryBackupImportLogicTest` (restore/OPML rethrow cancellation), `OpmlImportLogicTest` (catalog URL/title/feed match before `rss:` fallback, URL scheme/slash/`www` candidates, HTTPS rewrite for feed peek), `SubscriptionForegroundSyncLogicTest` (cooldown / subscriptions-first fetch / catalog ingest id gates), `SubscriptionRepositoryTest` (subscription tombstoning with `unsubscribedAt`, resubscription tombstone clearing, and dirty tracking on settings updates), `InstallReferrerManager` channel derivation / attribution callback seams, `SubscriptionForegroundSync` delay / `requestRefresh` / periodic / failed-feed recovery (dead-address re-check and retry, `/sync` fallback and its switch, once-a-day re-check that counts only answered lookups, moved-feed adoption) / `SubscriptionForegroundSyncIngestTest` (`requestCatalogIngest` cooldown bypass, RSS skip, finished emit, unsubscribe TTL, expiry sweep isolation) / direct-feed concurrency and failure isolation (including missing-feed URL recovery), content orchestration tests, content signal enrichment, grouped sections, recent section intent storage, cross-promotion detection, transcript behavior, and dependency-holder behavior. - `PodcastRepositoryV2DiscoveryTest` (Robolectric, MockWebServer, answers recorded from the live `/v2` routes under `fixtures/v2`, see its README): recommendation seeds and exclusions as recipe IDs (Podcast Index, phone-minted and RSS-linked history; an unplaced Podcast Index ID excluded as itself), at most four languages, picks back under local owners with their recipe IDs and without episodes the history holds; through Room, a played Podcast Index episode with no alias matched by its audio address and not offered again; the last good picks after a failure, after a restart and for an answer without picks, never a v1 request, nothing for another country, unreadable stored picks ignored; Home picks keeping their fallback flag, country-default picks for a listener with no seed or interest, interest-only picks personal, and `failed` with nothing to keep; every discovery route on `/v2` without the app key and translated (trending, vibes, semantic, because-you-like, similar with its recipe-ID seed or `"0"` and without the seed itself, curiosity, briefing metadata, bootstrap with related episodes); show search and typeahead on v1; the trending stream and its last good list; an empty page staying empty; the switch sending everything to v1, personal picks on contract 1; v1 and v2 answers never sharing a cache entry. `RoomDiscoveryEpisodeIdsTest` (Robolectric Room) covers both directions through the alias table (trusted only, owner by listening, collisions and other shows left alone), an owner stored under the show's own feed, and the audio-address match (unique rows only, never a row an alias places). `ChapterRepositoryDiscoveryTest` reads the recorded briefing chapters through `ChapterRepository` from the signed `/v2` link, the cache and a downloaded briefing's file. `BriefingLinksTest` covers the chapters and transcript links, the installed switch, briefing related episodes and a failing alias table. `PodcastRecommendationSeedsTest` covers seed IDs and the `/v2` history scope; `PodcastRepositoryRequestBoundsTest` the contract version, the v1 contract-1 path and the `/v2` request bodies. +- Feed address repair: `UpgradeAddressRepairTest` (Robolectric Room with the real address writer): a fresh install and `rss:`-only follows settle without network and never ask for the sync catch-up; check, confirmed swap, re-read and re-registration per show with one summary event; 501 shows in calls of 500 and 1; a partial batch kept at step 0 with no try counted; offline checks and reads counting no try; process death mid-step resuming without asking or swapping again; a restore with Room done but no step 4 record registering every followed show; unfollow between steps; a follow added mid-run left out; a dead show giving up at the cap; a different server `epNs` recorded, counted on the summary and only compared by the read; the production read keeping the saved address, passing the namespace and starting no auto-download pass; a follow changed on the phone keeping its address; the switch pausing without marking done; an answer left waiting by a pause asked again two days later and never applied; a failed swap keeping its answer's time; a refused check reaching the cap and going on without an answer; an unanswered check (503, registry down) counting no try for 7 days, then going on; the sync catch-up asked for once after the last show and not while shows wait or the run is paused, and retried until it could be asked; a registration request that couldn't be saved not recorded and asked again; the step 4 file surviving a new process. `UpgradeAddressRepairApiTest` runs step 1 through the real `/v2` client against MockWebServer with `/v2/shows/feeds` answers in the route's shape (`fixtures/v2`): a confirmed direct move swapped, an http-only show kept with its transport address, an unknown show kept, another `epNs` counted; a 503 and `registryAvailable: false` counting no try; a 400 and an unreadable answer each counting one. `CloudSyncIdentityTest` covers the catch-up asked for again after the repair settled an episode, and a request leaving a phone that hasn't caught up as it is. - `RolloutSwitchesTest` pins the defaults without the app's wiring (`feedFirst` off, every job on) and that each switch reads its own lambda. - `RssNotificationReconcilerTest` runs `FirebaseRssNotificationRemote` against a fake that applies the live `tracked_podcasts` write rule: cleanup keeps the node and drops only `feedUrl`, a never-published key still leaves its topic, and any other failure keeps the topic and the journal entry. - RSS ID and matcher tests live in `:core:rss`; smart-queue tests live in `:core:playback`. diff --git a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/SharedAppDependencies.kt b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/SharedAppDependencies.kt index fc69acf45..37dfd3a9d 100644 --- a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/SharedAppDependencies.kt +++ b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/SharedAppDependencies.kt @@ -35,6 +35,13 @@ interface SharedAppDependencies { val folderRepository: FolderRepository val deviceIdentityPort: cx.aswin.boxlore.core.domain.ports.DeviceIdentityPort? get() = null val userSyncCoordinator: cx.aswin.boxlore.core.catalog.sync.UserSyncCoordinator? get() = null + + /** + * The one-time repair of saved feed addresses after the update, run by the app's + * `UpgradeRepairWorker`; null when the composition root doesn't provide it (tests). + * Added in P4 PR 15 of the own-episode-IDs plan (plan section 11, upgrade repair). + */ + val upgradeAddressRepair: UpgradeAddressRepair? get() = null } /** diff --git a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/UpgradeAddressCheck.kt b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/UpgradeAddressCheck.kt new file mode 100644 index 000000000..0d3bccfd7 --- /dev/null +++ b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/UpgradeAddressCheck.kt @@ -0,0 +1,215 @@ +package cx.aswin.boxlore.core.catalog + +import android.util.Log +import cx.aswin.boxlore.core.database.PodcastDao +import cx.aswin.boxlore.core.database.PodcastEntity +import cx.aswin.boxlore.core.database.UpgradeRepairDao +import cx.aswin.boxlore.core.database.UpgradeRepairShowEntity +import cx.aswin.boxlore.core.network.BoxLoreApiV2 +import cx.aswin.boxlore.core.network.model.V2EpisodeIds +import cx.aswin.boxlore.core.network.model.V2ShowAddress +import cx.aswin.boxlore.core.network.model.V2ShowsFeedsRequest +import cx.aswin.boxlore.core.network.model.V2ShowsFeedsResponse +import java.io.IOException +import kotlinx.coroutines.CancellationException +import retrofit2.awaitResponse + +/** + * What one `POST /v2/shows/feeds` request of the upgrade repair's step 1 got. + * + * Added in P4 PR 15 of the own-episode-IDs plan (plan section 11, upgrade repair). + * COMPAT(own-ids): someone can upgrade from an old build at any time. Permanent: never remove (MIGRATION.md §5). + */ +sealed interface AddressCheckAnswer { + /** A 200 answer. One with `registryAvailable: false` counts as [Unavailable]. */ + data class Answered(val response: V2ShowsFeedsResponse) : AddressCheckAnswer + + /** + * The server couldn't answer now (503 `lookup_unavailable`, 429, another 5xx) or the request + * never got through. Never counts as a try; a show unanswered for + * [UpgradeAddressRepair.MAX_UNANSWERED_MS] goes on without an answer. + */ + data object Unavailable : AddressCheckAnswer + + /** + * The server refused the request (another 4xx) or its answer couldn't be read: the same + * request would fail again, so it counts as a try for each show asked. + */ + data object Rejected : AddressCheckAnswer + + companion object { + private const val TOO_MANY_REQUESTS = 429 + private const val SERVER_ERRORS = 500 + + /** The production step 1 request: [api]'s `/v2/shows/feeds`, its status sorted as above. */ + fun fetchWith(api: BoxLoreApiV2): suspend (List) -> AddressCheckAnswer = { ids -> + try { + val response = api.getShowFeeds(V2ShowsFeedsRequest(ids)).awaitResponse() + val body = response.body() + when { + response.isSuccessful && body != null -> Answered(body) + response.isSuccessful -> Rejected + response.code() == TOO_MANY_REQUESTS || response.code() >= SERVER_ERRORS -> Unavailable + else -> Rejected + } + } catch (e: CancellationException) { + throw e + } catch (_: IOException) { + Unavailable + } catch (e: Exception) { + Log.w("UpgradeAddressRepair", "Address check answer couldn't be read", e) + Rejected + } + } + } +} + +/** + * What one [UpgradeAddressRepair.run] did, and why it stopped early. + * + * Added in P4 PR 15 of the own-episode-IDs plan (plan section 11, upgrade repair). + */ +internal class UpgradeRepairPass { + var offline = false + var paused = false + var read = 0 + var notModified = 0 + var failed = 0 + + /** Reads of shows whose namespace on the server differs from the one the phone keeps (D12). */ + var namespaceMismatches = 0 + + val stopped: Boolean get() = offline || paused +} + +/** + * Step 1 of [UpgradeAddressRepair]: asks the server for the current address of every followed + * show still at step 0, in chunks of at most [V2ShowsFeedsRequest.MAX_IDS], and records each + * answer (see the repair's class description). + * + * A chunk the server couldn't answer ([AddressCheckAnswer.Unavailable], or the registry down) + * counts no try; one it refused ([AddressCheckAnswer.Rejected]) counts a try for each show. A + * show unanswered for [UpgradeAddressRepair.MAX_UNANSWERED_MS] since its first failed check, or + * refused [UpgradeAddressRepair.MAX_ATTEMPTS] times, goes on to steps 2–4 with no answer: no swap, + * and the daily check looks after its address. While a show is at step 0, `lastAttemptAt` holds + * its first failed check. Offline, nothing is recorded and the run stops. + * + * An answer older than [FeedAddressDailyCheck.CHECK_INTERVAL_MS] that still holds a move to swap + * to is dropped at the start of a run ([expireStaleAnswers]), so a repair paused or stopped for a + * while asks again instead of applying an old answer. + * + * Added in P4 PR 15 of the own-episode-IDs plan (plan section 11, upgrade repair; decision D12). + * COMPAT(own-ids): someone can upgrade from an old build at any time. Permanent: never remove (MIGRATION.md §5). + */ +internal class UpgradeAddressCheckStep( + private val repairDao: UpgradeRepairDao, + private val podcastDao: PodcastDao, + private val ports: UpgradeRepairPorts, + private val nowMillis: () -> Long, + private val followedRow: suspend (UpgradeRepairShowEntity) -> PodcastEntity?, + private val stopsHere: (UpgradeRepairPass) -> Boolean, +) { + /** Sends shows at step 1 whose move to swap to is too old back to step 0. */ + suspend fun expireStaleAnswers() { + val now = nowMillis() + for (entry in repairDao.pendingShows()) { + if (entry.completedStep != 1 || entry.checkedFetchUrl == null || !answerExpired(entry.lastAttemptAt, now)) continue + Log.i(TAG, "Address check of ${entry.podcastId} is out of date; asking again") + repairDao.upsert(entry.withoutAnswer().copy(completedStep = 0, lastAttemptAt = 0L, lastError = null)) + } + } + + /** Step 1 for every followed show still at step 0. */ + suspend fun check(pass: UpgradeRepairPass) { + val askable = LinkedHashMap() + for (entry in repairDao.pendingShows().filter { it.completedStep == 0 }) { + val row = followedRow(entry) ?: continue + val showId = V2EpisodeIds.parseShowId(row.podcastId) + if (showId == null) recordAnswer(entry, row, null) else askable[showId] = entry + } + for (chunk in askable.keys.chunked(V2ShowsFeedsRequest.MAX_IDS)) { + if (stopsHere(pass)) return + val answer = fetch(chunk) + if (answer is AddressCheckAnswer.Answered && answer.response.registryAvailable) { + recordChunk(chunk, askable, answer.response) + } else { + unanswered(chunk.map(askable::getValue), counts = answer is AddressCheckAnswer.Rejected, pass) + } + } + } + + private suspend fun fetch(ids: List): AddressCheckAnswer = try { + ports.fetchAddresses(ids) + } catch (e: CancellationException) { + throw e + } catch (e: IOException) { + Log.w(TAG, "Address check of ${ids.size} shows failed", e) + AddressCheckAnswer.Unavailable + } catch (e: Exception) { + Log.w(TAG, "Address check of ${ids.size} shows failed", e) + AddressCheckAnswer.Rejected + } + + private suspend fun recordChunk(chunk: List, askable: Map, answer: V2ShowsFeedsResponse) { + val byId = answer.results.associateBy(V2ShowAddress::id) + for (showId in chunk) { + val entry = askable.getValue(showId) + podcastDao.getPodcast(entry.podcastId)?.let { row -> recordAnswer(entry, row, byId[showId]) } + } + } + + /** Records step 1 for one show; the registry's answer also counts as today's daily check. */ + private suspend fun recordAnswer(entry: UpgradeRepairShowEntity, row: PodcastEntity, result: V2ShowAddress?) { + val now = nowMillis() + val served = result?.validEpNs + if (UpgradeAddressRepair.namespaceDiffers(row, served)) { + Log.w(TAG, "Show ${row.podcastId} has another namespace on the server ($served); keeping the phone's") + } + if (result != null && !FeedAddressDailyCheck.changedOnPhone(row)) podcastDao.markFeedUrlChecked(row.podcastId, now) + repairDao.upsert( + entry.copy( + completedStep = 1, + attempts = 0, + checkedFetchUrl = result?.let { FeedAddressDailyCheck.confirmedMove(row, it) }, + checkedTransportUrl = result?.transportUrl, + checkedFeedSource = result?.feedSource, + checkedEpNs = served, + lastAttemptAt = now, + lastError = null, + ), + ) + } + + /** A chunk without an answer: counted against its shows ([counts]) unless offline, which stops the run. */ + private suspend fun unanswered(entries: List, counts: Boolean, pass: UpgradeRepairPass) { + if (!ports.isOnline()) { + pass.offline = true + return + } + val now = nowMillis() + for (entry in entries) { + val current = repairDao.getShow(entry.podcastId)?.takeIf { it.completedStep == 0 } ?: continue + val since = current.lastAttemptAt.takeIf { it in 1..now } ?: now + val attempts = if (counts) current.attempts + 1 else current.attempts + if (attempts >= UpgradeAddressRepair.MAX_ATTEMPTS || now - since >= UpgradeAddressRepair.MAX_UNANSWERED_MS) { + Log.w(TAG, "No address answer for ${current.podcastId}; going on without one") + val reason = if (counts) REFUSED else UNANSWERED + repairDao.upsert(current.withoutAnswer().copy(completedStep = 1, attempts = 0, lastAttemptAt = now, lastError = reason)) + } else { + repairDao.upsert(current.copy(attempts = attempts, lastAttemptAt = since, lastError = if (counts) REFUSED else UNAVAILABLE)) + } + } + } + + private fun UpgradeRepairShowEntity.withoutAnswer() = copy(checkedFetchUrl = null, checkedTransportUrl = null, checkedFeedSource = null, checkedEpNs = null) + + companion object { + private const val TAG = "UpgradeAddressRepair" + private const val UNAVAILABLE = "address_unavailable" + private const val UNANSWERED = "address_unanswered" + private const val REFUSED = "address_refused" + + /** Whether a step 1 answer recorded at [answeredAt] is too old to swap to at [now]. */ + fun answerExpired(answeredAt: Long, now: Long): Boolean = answeredAt <= 0L || answeredAt > now || now - answeredAt >= FeedAddressDailyCheck.CHECK_INTERVAL_MS + } +} diff --git a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/UpgradeAddressRepair.kt b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/UpgradeAddressRepair.kt new file mode 100644 index 000000000..a8e65aec4 --- /dev/null +++ b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/UpgradeAddressRepair.kt @@ -0,0 +1,324 @@ +package cx.aswin.boxlore.core.catalog + +import android.util.Log +import cx.aswin.boxlore.core.database.PodcastDao +import cx.aswin.boxlore.core.database.PodcastEntity +import cx.aswin.boxlore.core.database.UpgradeRepairDao +import cx.aswin.boxlore.core.database.UpgradeRepairShowEntity +import cx.aswin.boxlore.core.database.UpgradeRepairState +import cx.aswin.boxlore.core.domain.ports.FeedTelemetryPort +import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort +import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort.RefreshOutcome +import cx.aswin.boxlore.core.model.EpisodeRecipeId +import cx.aswin.boxlore.core.model.FeedFetchReport +import cx.aswin.boxlore.core.model.FeedFetchSource +import cx.aswin.boxlore.core.model.FeedUrlCorrectionReason +import cx.aswin.boxlore.core.model.FeedUrlCorrectionStep +import cx.aswin.boxlore.core.network.model.V2EpisodeIds +import cx.aswin.boxlore.core.network.model.V2FetchVia +import cx.aswin.boxlore.core.network.model.V2ShowAddress +import cx.aswin.boxlore.core.network.model.V2ShowFlags +import cx.aswin.boxlore.core.network.model.V2ShowsFeedsRequest +import kotlinx.coroutines.CancellationException +import kotlinx.coroutines.sync.Mutex +import kotlinx.coroutines.sync.withLock + +/** How one [UpgradeAddressRepair.run] ended. */ +enum class UpgradeRepairRun { + /** Every step finished, or there was nothing to do. */ + DONE, + + /** Steps are left: offline, the server didn't answer, or a show failed below its cap. */ + RETRY, + + /** `upgrade-repair-off` is on: the repair stopped without marking anything done. */ + PAUSED, +} + +/** + * What [UpgradeAddressRepair] reaches outside Room. + * + * - [fetchAddresses]: one `POST /v2/shows/feeds` for at most [V2ShowsFeedsRequest.MAX_IDS] show + * IDs ([AddressCheckAnswer.fetchWith]). + * - [replaceAddress]: the one address writer ([FeedAddressUpdater.update]). + * - [reread]: reads a followed show's feed again into the local catalog, passing the namespace the + * server reported ([rereadWith]); moves it finds are followed by the catalog itself. + * - [requestNotificationRegistration]: asks for a show's notifications to be registered again by + * the app's capped, acknowledged job, and returns once the request is saved; throws when it + * couldn't be saved. + * - [onAddressStepsFinished]: called once, after every show's steps 1–3 are over, so the cloud + * sync's identity catch-up runs again for the episodes the reads settled. + * - [isOnline]: failures while offline never count against a show's cap. + * - [telemetry]: one `feed_fetch_result` summary per run that read feeds. + * + * Added in P4 PR 15 of the own-episode-IDs plan (plan section 11, upgrade repair). + */ +data class UpgradeRepairPorts( + val fetchAddresses: suspend (List) -> AddressCheckAnswer, + val replaceAddress: suspend (FeedAddressChange) -> FeedAddressUpdate, + val reread: suspend (show: PodcastEntity, serverEpNs: Long?) -> RefreshOutcome, + val requestNotificationRegistration: suspend (podcastId: String) -> Unit, + val onAddressStepsFinished: suspend () -> Unit = {}, + val isOnline: () -> Boolean = { true }, + val telemetry: FeedTelemetryPort = FeedTelemetryPort.NONE, +) + +/** + * The one-time repair of saved feed addresses after the update, for every Podcast Index show + * followed when the phone upgraded (the v39 upgrade seeds one pending `upgrade_repair_shows` row + * each; a fresh install has none). Per show: + * + * 1. Ask the server for the show's current address ([UpgradeAddressCheckStep]): `POST + * /v2/shows/feeds`, every waiting show in chunks of at most [V2ShowsFeedsRequest.MAX_IDS]. The + * answer is recorded: `checkedFetchUrl` holds the address to swap to, set only for a confirmed + * move ([FeedAddressDailyCheck.confirmedMove]: the registry flags the show for repair and its + * own HTTPS address differs from the saved one); `fetchUrl: null` keeps the saved address. A + * chunk the server couldn't answer stays at step 0 without counting a try, for at most + * [MAX_UNANSWERED_MS]; one it refused counts a try. Either way the show then goes on without + * an answer. A namespace (`epNs`) the server reports differently from the phone's is recorded + * in `checkedEpNs`, counted on `feed_fetch_result` when the show is read, and never applied to + * issued IDs (decision D12). + * 2. Swap the saved address through the one address writer, reported as `feed_url_corrected` + * (step `upgrade_repair`) and registered again by the writer. A follow changed on this phone and + * not pushed yet keeps its address (decision 2); the daily check swaps it once it is pushed. An + * answer older than a day is asked again first, and one the daily check has since replaced + * with a newer answer is not applied. + * 3. Read the feed again, without the auto-download pass a foreground read starts (the background + * pass applies the listener's rules). A failure while online counts against the show's cap of + * [MAX_ATTEMPTS]; the show then ends [UpgradeRepairState.GAVE_UP], and the daily check keeps + * watching it. + * 4. Register notifications again for every followed show with notifications on, once its own + * steps are over, recorded in [UpgradeNotificationMarker] (`noBackupFilesDir`), so a restored + * phone registers again even when Room says steps 1–3 are done. + * + * Once no show has steps 1–3 left, [UpgradeRepairPorts.onAddressStepsFinished] runs once per + * install (recorded in the same marker), so the sync's one-time identity catch-up runs again for + * the episodes the reads settled. A fresh install never calls it. + * + * Each finished step is written at once, so process death resumes at the next step. Before each + * step the show's row is read again: a show no longer followed ends [UpgradeRepairState.DONE] + * without further steps. Shows followed after the upgrade have no repair row and are left to the + * daily check. With [enabled] off (`upgrade-repair-off`) the repair stops without marking anything + * done. [needsWork] settles a fresh install, or one with only `rss:` follows, without network. + * + * Added in P4 PR 15 of the own-episode-IDs plan (plan section 11, upgrade repair; decisions 2 and D12). + * COMPAT(own-ids): someone can upgrade from an old build at any time. Permanent: never remove (MIGRATION.md §5). + */ +class UpgradeAddressRepair( + private val repairDao: UpgradeRepairDao, + private val podcastDao: PodcastDao, + private val ports: UpgradeRepairPorts, + private val marker: UpgradeNotificationMarker, + private val enabled: () -> Boolean, + private val nowMillis: () -> Long = System::currentTimeMillis, +) { + private val runLock = Mutex() + private val addresses = UpgradeAddressCheckStep(repairDao, podcastDao, ports, nowMillis, ::followedRow, ::stopsHere) + private val notifications = UpgradeNotificationStep(repairDao, podcastDao, marker, ports.requestNotificationRegistration) + + /** + * Whether a run has anything to do: shows with steps left, the sync catch-up not asked for yet, + * or step 4 not finished with followed shows to register. With none, step 4 is recorded as + * finished here, without network. + */ + suspend fun needsWork(): Boolean { + if (repairDao.pendingCount() > 0 || catchUpDue()) return true + if (marker.isComplete) return false + if (notifications.targets().isNotEmpty()) return true + marker.markComplete() + return false + } + + /** Runs every step that is due; see the class description. */ + suspend fun run(): UpgradeRepairRun = runLock.withLock { + if (!enabled()) return@withLock UpgradeRepairRun.PAUSED + val pass = UpgradeRepairPass() + addresses.expireStaleAnswers() + addresses.check(pass) + repairShows(pass) + if (!pass.stopped) { + askForSyncCatchUp() + notifications.register() + } + report(pass) + when { + pass.paused -> UpgradeRepairRun.PAUSED + repairDao.pendingCount() == 0 && marker.isComplete && !catchUpDue() -> UpgradeRepairRun.DONE + else -> UpgradeRepairRun.RETRY + } + } + + /** Steps 1–3 are over for an install that had shows to repair, and the sync catch-up wasn't asked for yet. */ + private suspend fun catchUpDue(): Boolean = !marker.syncCatchUpRequested && repairDao.pendingCount() == 0 && repairDao.hasShows() + + private suspend fun askForSyncCatchUp() { + if (!catchUpDue()) return + try { + ports.onAddressStepsFinished() + marker.markSyncCatchUpRequested() + } catch (e: CancellationException) { + throw e + } catch (e: Exception) { + Log.w(TAG, "Couldn't ask for the sync catch-up", e) + } + } + + /** One `feed_fetch_result` summary for the feeds this run read, if any. */ + private fun report(pass: UpgradeRepairPass) { + if (pass.read + pass.notModified + pass.failed == 0) return + ports.telemetry.feedFetched( + FeedFetchReport.BackgroundPass(FeedFetchSource.FEED, pass.read, pass.notModified, pass.failed, namespaceMismatchCount = pass.namespaceMismatches), + ) + } + + /** Steps 2 and 3 for every show past step 1, one show at a time. */ + private suspend fun repairShows(pass: UpgradeRepairPass) { + for (entry in repairDao.pendingShows().filter { it.completedStep != 0 }) { + if (stopsHere(pass)) return + try { + val current = if (entry.completedStep == 1) swapAddress(entry) else entry + if (current?.completedStep == 2) readAgain(current, pass) + } catch (e: CancellationException) { + throw e + } catch (e: Exception) { + Log.w(TAG, "Repair of ${entry.podcastId} failed", e) + recordFailure(entry.podcastId, e.javaClass.simpleName, pass) + } + } + } + + /** Step 2; null when the show is no longer followed. */ + private suspend fun swapAddress(entry: UpgradeRepairShowEntity): UpgradeRepairShowEntity? { + val row = followedRow(entry) ?: return null + val target = swapTarget(row, entry, nowMillis())?.takeUnless { FeedAddressDailyCheck.changedOnPhone(row) } + if (target != null) { + val change = + FeedAddressChange( + podcastId = row.podcastId, + feedUrl = target, + source = entry.checkedFeedSource ?: FeedUrlSources.REGISTRY, + checkedAt = nowMillis(), + onlyReplacing = row.feedUrl, + reason = FeedUrlCorrectionReason.REGISTRY, + step = FeedUrlCorrectionStep.UPGRADE_REPAIR, + ) + Log.i(TAG, "Address swap for ${row.podcastId}: ${ports.replaceAddress(change)}") + } + return entry.copy(completedStep = 2, attempts = 0, lastAttemptAt = nowMillis(), lastError = null).also { repairDao.upsert(it) } + } + + /** Step 3. */ + private suspend fun readAgain(entry: UpgradeRepairShowEntity, pass: UpgradeRepairPass) { + val row = followedRow(entry) ?: return + if (row.feedUrl.isNullOrBlank()) { + finish(entry) + return + } + val outcome = ports.reread(row, entry.checkedEpNs) + if (namespaceDiffers(row, entry.checkedEpNs)) pass.namespaceMismatches++ + when (outcome) { + is RefreshOutcome.Success -> { + pass.read++ + finish(entry) + } + is RefreshOutcome.Unchanged -> { + pass.notModified++ + finish(entry) + } + is RefreshOutcome.Failure -> { + pass.failed++ + recordFailure(entry.podcastId, outcome.reportStatus ?: READ_FAILED, pass) + } + } + } + + private suspend fun finish(entry: UpgradeRepairShowEntity) { + repairDao.upsert(entry.copy(completedStep = 3, state = UpgradeRepairState.DONE, lastAttemptAt = nowMillis(), lastError = null)) + } + + /** + * Counts a failed try against the show's cap, unless the phone is offline (then the run + * stops). A show at step 1 keeps the time of its answer, so an old answer still expires. + */ + private suspend fun recordFailure(podcastId: String, reason: String, pass: UpgradeRepairPass) { + if (!ports.isOnline()) { + pass.offline = true + return + } + val current = repairDao.getShow(podcastId) ?: return + val attempts = current.attempts + 1 + val state = if (attempts >= MAX_ATTEMPTS) UpgradeRepairState.GAVE_UP else current.state + if (state == UpgradeRepairState.GAVE_UP) Log.w(TAG, "Giving up the repair of $podcastId after $attempts tries ($reason)") + val triedAt = if (current.completedStep <= 1) current.lastAttemptAt else nowMillis() + repairDao.upsert(current.copy(attempts = attempts, state = state, lastAttemptAt = triedAt, lastError = reason.take(MAX_ERROR_LENGTH))) + } + + /** The show's row while it is still a followed Podcast Index show; otherwise its repair ends done. */ + private suspend fun followedRow(entry: UpgradeRepairShowEntity): PodcastEntity? { + val row = podcastDao.getPodcast(entry.podcastId) + if (row != null && row.isSubscribed && !row.isRss) return row + repairDao.upsert(entry.copy(state = UpgradeRepairState.DONE, lastAttemptAt = nowMillis())) + return null + } + + private fun stopsHere(pass: UpgradeRepairPass): Boolean { + if (!pass.stopped && !enabled()) pass.paused = true + return pass.stopped + } + + companion object { + private const val TAG = "UpgradeAddressRepair" + private const val READ_FAILED = "read_failed" + private const val MAX_ERROR_LENGTH = 80 + + /** Failed tries of one show (while online) before its repair gives up, or goes on without an address answer. */ + const val MAX_ATTEMPTS = 5 + + /** How long a show waits for an address answer the server couldn't give before it goes on without one. */ + const val MAX_UNANSWERED_MS = 7L * FeedAddressDailyCheck.CHECK_INTERVAL_MS + + /** + * The address step 2 swaps to: step 1's confirmed move, checked again against the row as it + * is now (the saved address may have changed since). Step 1 kept it only when the registry + * flagged the show for repair with its own direct address, so the same rule applies. None + * when the answer is older than a day at [now], or the daily check has a newer one. + */ + internal fun swapTarget(row: PodcastEntity, entry: UpgradeRepairShowEntity, now: Long): String? { + val fetchUrl = entry.checkedFetchUrl ?: return null + if (UpgradeAddressCheckStep.answerExpired(entry.lastAttemptAt, now) || row.feedUrlCheckedAt > entry.lastAttemptAt) return null + val confirmed = + V2ShowAddress( + id = V2EpisodeIds.parseShowId(row.podcastId) ?: return null, + fetchUrl = fetchUrl, + fetchVia = V2FetchVia.DIRECT, + feedSource = entry.checkedFeedSource, + flags = V2ShowFlags(addressRepair = true), + ) + return FeedAddressDailyCheck.confirmedMove(row, confirmed) + } + + /** True when [served], the namespace the server reported, differs from the one the phone keeps for [row]. */ + internal fun namespaceDiffers(row: PodcastEntity, served: Long?): Boolean { + val kept = row.epNs ?: EpisodeRecipeId.showIdOrNull(row.podcastId) + return served != null && kept != null && served != kept + } + + /** + * The production step 3: a catalog refresh of the saved address, with the server's namespace + * for comparison only. It never runs the auto-download pass a foreground read starts: the + * repair runs in the background, where the regular pass applies the listener's background + * download rules. + */ + fun rereadWith(catalog: LocalEpisodeCatalogPort): suspend (PodcastEntity, Long?) -> RefreshOutcome = { row, serverEpNs -> + catalog.refresh( + LocalEpisodeCatalogPort.RefreshRequest( + podcastIndexId = row.podcastId, + feedUrl = row.feedUrl.orEmpty(), + meta = LocalEpisodeCatalogPort.PodcastMeta(title = row.title, imageUrl = row.imageUrl, genre = row.genre, artist = row.author), + runPostPersistCallback = false, + serverEpNs = serverEpNs, + ), + ) + } + } +} diff --git a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/UpgradeNotificationMarker.kt b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/UpgradeNotificationMarker.kt new file mode 100644 index 000000000..8793e9a55 --- /dev/null +++ b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/UpgradeNotificationMarker.kt @@ -0,0 +1,132 @@ +package cx.aswin.boxlore.core.catalog + +import android.util.AtomicFile +import java.io.File +import java.io.FileNotFoundException +import java.util.Properties + +/** + * Step 4 of the one-time feed address repair ([UpgradeAddressRepair]): which followed shows had + * their new-episode notifications registered again on this install, and whether the step finished. + * It also records that the sync's identity catch-up was asked for once steps 1–3 were over. + * + * Steps 1–3 are kept in Room (`upgrade_repair_shows`), which an Android restore brings back; this + * marker lives in `noBackupFilesDir`, which a restore never brings, so a restored phone (a new + * messaging token, no topic subscriptions) registers every followed show again even when Room says + * the repair is done. The catch-up is then asked for again too, which only makes the next sync + * pull from the start once more. + * + * Added in P4 PR 15 of the own-episode-IDs plan (plan section 11, upgrade repair; "Android backup"). + * COMPAT(own-ids): someone can upgrade from an old build at any time. Permanent: never remove (MIGRATION.md §5). + */ +interface UpgradeNotificationMarker { + /** True once every followed show was registered again after the repair. */ + val isComplete: Boolean + + /** Shows whose registration was already requested on this install. */ + fun registered(): Set + + /** Records [podcastIds] as requested; false when the record couldn't be saved. */ + fun addRegistered(podcastIds: Collection): Boolean + + /** Records that step 4 finished; false when the record couldn't be saved. */ + fun markComplete(): Boolean + + /** True once the sync's identity catch-up was asked for after steps 1–3. */ + val syncCatchUpRequested: Boolean + + /** Records that the sync's identity catch-up was asked for; false when the record couldn't be saved. */ + fun markSyncCatchUpRequested(): Boolean +} + +/** In-memory [UpgradeNotificationMarker] for tests and previews. */ +class MemoryUpgradeNotificationMarker : UpgradeNotificationMarker { + private val values = linkedSetOf() + + @Volatile + override var isComplete: Boolean = false + private set + + @Synchronized override fun registered(): Set = values.toSet() + + @Synchronized override fun addRegistered(podcastIds: Collection): Boolean { + values += podcastIds + return true + } + + @Synchronized override fun markComplete(): Boolean { + isComplete = true + return true + } + + @Volatile + override var syncCatchUpRequested: Boolean = false + private set + + @Synchronized override fun markSyncCatchUpRequested(): Boolean { + syncCatchUpRequested = true + return true + } +} + +/** + * [UpgradeNotificationMarker] in one file under `noBackupFilesDir` (the app opens + * [FILE_NAME]); written atomically, so process death mid-write keeps the previous record. + * A file that can't be read counts as no record: the shows are registered again, which is harmless. + */ +class FileUpgradeNotificationMarker internal constructor(private val storage: AtomicFile) : UpgradeNotificationMarker { + constructor(file: File) : this(AtomicFile(file)) + + override val isComplete: Boolean + @Synchronized get() = read().getProperty(COMPLETE_KEY) == "true" + + override val syncCatchUpRequested: Boolean + @Synchronized get() = read().getProperty(SYNC_CATCH_UP_KEY) == "true" + + @Synchronized override fun registered(): Set = read().stringPropertyNames().filterNot { it.startsWith(RECORD_KEY_PREFIX) }.toSet() + + @Synchronized override fun addRegistered(podcastIds: Collection): Boolean = + podcastIds.isEmpty() || change { values -> podcastIds.forEach { values.setProperty(it, "1") } } + + @Synchronized override fun markComplete(): Boolean = change { it.setProperty(COMPLETE_KEY, "true") } + + @Synchronized override fun markSyncCatchUpRequested(): Boolean = change { it.setProperty(SYNC_CATCH_UP_KEY, "true") } + + private fun read(): Properties = Properties().apply { + try { + storage.openRead().use { load(it) } + } catch (_: FileNotFoundException) { + // No record yet. + } catch (_: java.io.IOException) { + clear() + } catch (_: IllegalArgumentException) { + clear() + } + } + + private fun change(edit: (Properties) -> Unit): Boolean = try { + val values = read() + edit(values) + val output = storage.startWrite() + try { + values.store(output, "Feed address repair notifications v1") + storage.finishWrite(output) + } catch (error: java.io.IOException) { + storage.failWrite(output) + throw error + } + true + } catch (_: java.io.IOException) { + false + } + + companion object { + /** File name under `noBackupFilesDir`. Stable storage name: never rename. */ + const val FILE_NAME = "upgrade_repair_notifications" + + /** Keys of the file's own records start with this; show IDs never do. */ + private const val RECORD_KEY_PREFIX = "__" + private const val COMPLETE_KEY = "__complete" + private const val SYNC_CATCH_UP_KEY = "__sync_catch_up" + } +} diff --git a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/UpgradeNotificationStep.kt b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/UpgradeNotificationStep.kt new file mode 100644 index 000000000..70fba5976 --- /dev/null +++ b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/UpgradeNotificationStep.kt @@ -0,0 +1,63 @@ +package cx.aswin.boxlore.core.catalog + +import android.util.Log +import cx.aswin.boxlore.core.database.PodcastDao +import cx.aswin.boxlore.core.database.UpgradeRepairDao +import kotlinx.coroutines.CancellationException + +/** + * Step 4 of [UpgradeAddressRepair]: asks for every followed Podcast Index show with notifications + * on, whose own repair steps are over, to be registered again, records each request in [marker] + * once the request is saved ([requestRegistration] returns), and records the step done once no + * show has steps left and every request was saved. A request that couldn't be saved is not + * recorded, so the next run asks again. + * + * Added in P4 PR 15 of the own-episode-IDs plan (plan section 11, upgrade repair; "Android backup"). + * COMPAT(own-ids): someone can upgrade from an old build at any time. Permanent: never remove (MIGRATION.md §5). + */ +internal class UpgradeNotificationStep( + private val repairDao: UpgradeRepairDao, + private val podcastDao: PodcastDao, + private val marker: UpgradeNotificationMarker, + private val requestRegistration: suspend (podcastId: String) -> Unit, +) { + suspend fun register() { + if (marker.isComplete) return + val requested = mutableListOf() + var allRequested = true + for (podcastId in targets()) { + if (request(podcastId)) requested += podcastId else allRequested = false + if (requested.size >= MARKER_BATCH) { + marker.addRegistered(requested) + requested.clear() + } + } + marker.addRegistered(requested) + if (allRequested && repairDao.pendingCount() == 0) marker.markComplete() + } + + /** Followed Podcast Index shows with notifications on, not yet registered here, whose repair steps are over. */ + suspend fun targets(): List { + val registered = marker.registered() + val waiting = repairDao.pendingShows().mapTo(HashSet()) { it.podcastId } + return podcastDao + .getSubscribedPodcastsList() + .filter { !it.isRss && it.notificationsEnabled && it.podcastId !in registered && it.podcastId !in waiting } + .map { it.podcastId } + } + + private suspend fun request(podcastId: String): Boolean = try { + requestRegistration(podcastId) + true + } catch (e: CancellationException) { + throw e + } catch (e: Exception) { + Log.e(TAG, "Couldn't ask to register notifications again for $podcastId", e) + false + } + + private companion object { + const val TAG = "UpgradeAddressRepair" + const val MARKER_BATCH = 50 + } +} diff --git a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/sync/UserSyncCoordinator.kt b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/sync/UserSyncCoordinator.kt index 6158ca489..eaa6ffc10 100644 --- a/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/sync/UserSyncCoordinator.kt +++ b/core/catalog/src/main/java/cx/aswin/boxlore/core/catalog/sync/UserSyncCoordinator.kt @@ -116,6 +116,24 @@ open class UserSyncCoordinator( } } + /** + * Runs the one-time identity catch-up again on the next sync (a pull from the start, then the + * pushes of cloud copies that gain an identity) when it already ran: sets the sync metadata + * version back below [IDENTITY_SYNC_METADATA_VERSION]. A phone that never synced, or one still + * waiting for the catch-up, is left as it is. Waits for a running sync, so that sync can't + * record the catch-up done over this request. + * + * Called once the one-time feed address repair has read every followed show again, so history + * and queue rows whose episodes those reads settled reach the cloud with their identity. Added + * in P4 PR 15 of the own-episode-IDs plan (plan sections 8 and 11). + * COMPAT(own-ids): someone can upgrade from an old build at any time. Permanent: never remove (MIGRATION.md §5). + */ + suspend fun requestIdentityCatchUp() = syncMutex.withLock { + if (requireBoxcastPrefs.getSyncMetadataVersion() >= IDENTITY_SYNC_METADATA_VERSION) { + requireBoxcastPrefs.setSyncMetadataVersion(IDENTITY_SYNC_METADATA_VERSION - 1) + } + } + private suspend fun handleAccountSwitch(userId: String, token: String?): SyncResult { purgeLocalDataForAccountSwitch() requireBoxcastPrefs.setLastSyncTimestamp(0L) @@ -445,28 +463,7 @@ open class UserSyncCoordinator( if (!queueDirty || queueMeta == null) return null val identities = episodeIdentity?.outgoing(queueItems.map { SyncEpisodeIdentity.OutgoingRow(it.episodeId, it.podcastId) }).orEmpty() return QueueSyncDto( - items = queueItems.map { q -> - val identity = identities[SyncEpisodeIdentity.OutgoingRow(q.episodeId, q.podcastId)] - QueueItemSyncDto( - episodeId = q.episodeId, - podcastId = q.podcastId, - title = q.title.takeIf { it.isNotBlank() }, - podcastTitle = q.podcastTitle.takeIf { it.isNotBlank() }, - imageUrl = persistableArtworkUrl(q.imageUrl), - podcastImageUrl = persistableArtworkUrl(q.podcastImageUrl), - audioUrl = q.audioUrl.takeIf { it.isNotBlank() }, - duration = q.duration, - pubDate = q.pubDate, - description = q.description?.takeIf { it.isNotBlank() }, - position = q.position, - addedAt = q.addedAt, - contextType = q.contextType, - contextSourceId = q.contextSourceId, - updatedAt = queueMeta.queueUpdatedAt, - recipeId = identity?.recipeId, - catalogKey = identity?.catalogKey, - ) - }, + items = queueItems.map { q -> queueItemDto(q, queueMeta, identities[SyncEpisodeIdentity.OutgoingRow(q.episodeId, q.podcastId)]) }, queueUpdatedAt = queueMeta.queueUpdatedAt, queueSequence = queueMeta.queueSequence, lastModifiedDeviceId = queueMeta.lastModifiedDeviceId, @@ -475,6 +472,31 @@ open class UserSyncCoordinator( ) } + private fun queueItemDto( + q: QueueItem, + queueMeta: QueueMetadataEntity, + identity: SyncEpisodeIdentity.WireIdentity?, + ): QueueItemSyncDto = + QueueItemSyncDto( + episodeId = q.episodeId, + podcastId = q.podcastId, + title = q.title.takeIf { it.isNotBlank() }, + podcastTitle = q.podcastTitle.takeIf { it.isNotBlank() }, + imageUrl = persistableArtworkUrl(q.imageUrl), + podcastImageUrl = persistableArtworkUrl(q.podcastImageUrl), + audioUrl = q.audioUrl.takeIf { it.isNotBlank() }, + duration = q.duration, + pubDate = q.pubDate, + description = q.description?.takeIf { it.isNotBlank() }, + position = q.position, + addedAt = q.addedAt, + contextType = q.contextType, + contextSourceId = q.contextSourceId, + updatedAt = queueMeta.queueUpdatedAt, + recipeId = identity?.recipeId, + catalogKey = identity?.catalogKey, + ) + /** * The queue's rich removals as pushed: the stored list, always sent (an empty list too), because * the server replaces its list with the one sent. Recipe IDs that name another episode of the diff --git a/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/UpgradeAddressRepairApiTest.kt b/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/UpgradeAddressRepairApiTest.kt new file mode 100644 index 000000000..8708e340c --- /dev/null +++ b/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/UpgradeAddressRepairApiTest.kt @@ -0,0 +1,173 @@ +package cx.aswin.boxlore.core.catalog + +import android.content.Context +import androidx.room.Room +import androidx.test.core.app.ApplicationProvider +import cx.aswin.boxlore.core.database.BoxLoreDatabase +import cx.aswin.boxlore.core.database.PodcastDao +import cx.aswin.boxlore.core.database.PodcastEntity +import cx.aswin.boxlore.core.database.UpgradeRepairDao +import cx.aswin.boxlore.core.database.UpgradeRepairShowEntity +import cx.aswin.boxlore.core.domain.ports.FeedTelemetryPort +import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort.RefreshOutcome +import cx.aswin.boxlore.core.model.FeedFetchReport +import cx.aswin.boxlore.core.model.FeedFetchSource +import cx.aswin.boxlore.core.model.FeedUrlCorrection +import cx.aswin.boxlore.core.network.NetworkModule +import kotlinx.coroutines.test.runTest +import okhttp3.OkHttpClient +import okhttp3.mockwebserver.MockResponse +import okhttp3.mockwebserver.MockWebServer +import org.junit.After +import org.junit.Assert.assertEquals +import org.junit.Assert.assertNull +import org.junit.Assert.assertTrue +import org.junit.Before +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.annotation.Config + +/** + * Step 1 of the one-time address repair (own-episode-IDs P4 PR 15, plan section 11) through the + * real `/v2` client against MockWebServer serving `/v2/shows/feeds` answers in the route's shape + * (`fixtures/v2`, see its README): how each kind of answer is recorded, and which failures count + * as a try. + */ +@RunWith(RobolectricTestRunner::class) +@Config(sdk = [34]) +class UpgradeAddressRepairApiTest { + private lateinit var server: MockWebServer + private lateinit var database: BoxLoreDatabase + private lateinit var podcasts: PodcastDao + private lateinit var repairs: UpgradeRepairDao + private val responses = ArrayDeque() + private val rereads = mutableListOf>() + private val reports = mutableListOf() + + private val telemetry = + object : FeedTelemetryPort { + override fun feedFetched(report: FeedFetchReport) { + reports += report + } + + override fun feedUrlCorrected(correction: FeedUrlCorrection) = Unit + } + + @Before + fun setUp() { + server = MockWebServer() + server.dispatcher = + object : okhttp3.mockwebserver.Dispatcher() { + override fun dispatch(request: okhttp3.mockwebserver.RecordedRequest): MockResponse = responses.removeFirstOrNull() ?: MockResponse().setResponseCode(404) + } + server.start() + val context = ApplicationProvider.getApplicationContext() + database = Room.inMemoryDatabaseBuilder(context, BoxLoreDatabase::class.java).allowMainThreadQueries().build() + podcasts = database.podcastDao() + repairs = database.upgradeRepairDao() + } + + @After + fun tearDown() { + database.close() + server.shutdown() + } + + @Test + fun `a recorded answer swaps the confirmed move only and records the rest`() = runTest { + seed("920666", OLD) + seed("920667", "http://plain.example.com/feed.xml") + seed("920668", OTHER) + seed("920669", OTHER) + responses += fixture("fixtures/v2/shows-feeds-upgrade-cases.json") + + assertEquals(UpgradeRepairRun.DONE, repair().run()) + + val sent = server.takeRequest() + assertEquals("POST", sent.method) + assertEquals("/v2/shows/feeds", sent.path) + assertEquals("{\"ids\":[920666,920667,920668,920669]}", sent.body.readUtf8()) + + assertEquals("a confirmed direct move is swapped", NEW, podcasts.getPodcast("920666")!!.feedUrl) + assertEquals("an http-only show keeps its address", "http://plain.example.com/feed.xml", podcasts.getPodcast("920667")!!.feedUrl) + assertEquals("https://api.example.test/v2/feed/920667", repairs.getShow("920667")!!.checkedTransportUrl) + assertNull(repairs.getShow("920667")!!.checkedFetchUrl) + assertEquals("a show the lookup doesn't know keeps its address", OTHER, podcasts.getPodcast("920668")!!.feedUrl) + assertEquals("an unflagged address is no move", OTHER, podcasts.getPodcast("920669")!!.feedUrl) + assertEquals(4242L, repairs.getShow("920669")!!.checkedEpNs) + assertNull("the server's namespace is never stored for the show", podcasts.getPodcast("920669")!!.epNs) + assertEquals(listOf("920666" to 920666L, "920667" to 920667L, "920668" to 920668L, "920669" to 4242L), rereads) + assertEquals(listOf(FeedFetchReport.BackgroundPass(FeedFetchSource.FEED, 4, 0, 0, namespaceMismatchCount = 1)), reports) + } + + @Test + fun `a 503 and a registry that is down keep the chunk waiting without counting a try`() = runTest { + seed("920666", OLD) + responses += json("""{"error":"lookup_unavailable"}""", code = 503) + responses += fixture("fixtures/v2/shows-feeds-registry-down.json") + val repair = repair() + + assertEquals(UpgradeRepairRun.RETRY, repair.run()) + assertEquals(UpgradeRepairRun.RETRY, repair.run()) + + val row = repairs.getShow("920666")!! + assertEquals(0, row.completedStep) + assertEquals(0, row.attempts) + assertEquals("address_unavailable", row.lastError) + assertEquals(OLD, podcasts.getPodcast("920666")!!.feedUrl) + assertTrue(rereads.isEmpty()) + assertEquals(2, server.requestCount) + } + + @Test + fun `a refused request and an unreadable answer each count a try`() = runTest { + seed("920666", OLD) + responses += json("""{"error":"invalid_ids","max":500}""", code = 400) + responses += json("") + val repair = repair() + + assertEquals(UpgradeRepairRun.RETRY, repair.run()) + assertEquals(1, repairs.getShow("920666")!!.attempts) + assertEquals(UpgradeRepairRun.RETRY, repair.run()) + assertEquals(2, repairs.getShow("920666")!!.attempts) + assertEquals("address_refused", repairs.getShow("920666")!!.lastError) + } + + private fun repair() = UpgradeAddressRepair( + repairDao = repairs, + podcastDao = podcasts, + ports = + UpgradeRepairPorts( + fetchAddresses = AddressCheckAnswer.fetchWith(NetworkModule.createBoxLoreApiV2(server.url("/").toString(), OkHttpClient())), + replaceAddress = FeedAddressUpdater(podcasts, requestNotificationRegistration = {})::update, + reread = { row, epNs -> + rereads += row.podcastId to epNs + RefreshOutcome.Success(null, 1, true) + }, + requestNotificationRegistration = {}, + telemetry = telemetry, + ), + marker = MemoryUpgradeNotificationMarker(), + enabled = { true }, + nowMillis = { NOW }, + ) + + private suspend fun seed(id: String, feedUrl: String) { + podcasts.upsertRow( + PodcastEntity(podcastId = id, title = "Show", author = "Author", imageUrl = "", description = null, isSubscribed = true, feedUrl = feedUrl), + ) + repairs.upsert(UpgradeRepairShowEntity(id)) + } + + private fun fixture(path: String): MockResponse = json(requireNotNull(javaClass.classLoader?.getResource(path)) { path }.readText()) + + private fun json(body: String, code: Int = 200): MockResponse = MockResponse().setResponseCode(code).setHeader("Content-Type", "application/json").setBody(body) + + private companion object { + const val OLD = "https://old.example.com/feed.xml" + const val NEW = "https://new.example.com/feed.xml" + const val OTHER = "https://other.example.com/feed.xml" + const val NOW = 1_800_000_000_000L + } +} diff --git a/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/UpgradeAddressRepairTest.kt b/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/UpgradeAddressRepairTest.kt new file mode 100644 index 000000000..d861549bd --- /dev/null +++ b/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/UpgradeAddressRepairTest.kt @@ -0,0 +1,655 @@ +package cx.aswin.boxlore.core.catalog + +import android.content.Context +import androidx.room.Room +import androidx.test.core.app.ApplicationProvider +import cx.aswin.boxlore.core.database.BoxLoreDatabase +import cx.aswin.boxlore.core.database.PodcastDao +import cx.aswin.boxlore.core.database.PodcastEntity +import cx.aswin.boxlore.core.database.UpgradeRepairDao +import cx.aswin.boxlore.core.database.UpgradeRepairShowEntity +import cx.aswin.boxlore.core.database.UpgradeRepairState +import cx.aswin.boxlore.core.domain.ports.FeedTelemetryPort +import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort +import cx.aswin.boxlore.core.domain.ports.LocalEpisodeCatalogPort.RefreshOutcome +import cx.aswin.boxlore.core.model.FeedFetchReport +import cx.aswin.boxlore.core.model.FeedFetchSource +import cx.aswin.boxlore.core.model.FeedUrlCorrection +import cx.aswin.boxlore.core.model.FeedUrlCorrectionReason +import cx.aswin.boxlore.core.model.FeedUrlCorrectionStep +import cx.aswin.boxlore.core.network.model.V2ShowAddress +import cx.aswin.boxlore.core.network.model.V2ShowFlags +import cx.aswin.boxlore.core.network.model.V2ShowsFeedsResponse +import cx.aswin.boxlore.core.testing.fakes.FakeLocalEpisodeCatalogPort +import java.io.File +import java.io.IOException +import kotlinx.coroutines.CancellationException +import kotlinx.coroutines.test.runTest +import org.junit.After +import org.junit.Assert.assertEquals +import org.junit.Assert.assertFalse +import org.junit.Assert.assertNull +import org.junit.Assert.assertTrue +import org.junit.Assert.fail +import org.junit.Before +import org.junit.Rule +import org.junit.Test +import org.junit.rules.TemporaryFolder +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.annotation.Config + +/** + * The one-time repair of saved feed addresses after the update (own-episode-IDs P4 PR 15, plan + * section 11): steps 1–3 per show in Room, step 4 in a marker a restore never brings, resumable + * after process death, offline-safe, capped per show, and silent on fresh installs. + */ +@RunWith(RobolectricTestRunner::class) +@Config(sdk = [34]) +class UpgradeAddressRepairTest { + @get:Rule + val folder = TemporaryFolder() + + private lateinit var database: BoxLoreDatabase + private lateinit var podcasts: PodcastDao + private lateinit var repairs: UpgradeRepairDao + private lateinit var updater: FeedAddressUpdater + private val calls = mutableListOf>() + private val answers = mutableMapOf() + private val rereads = mutableListOf>() + private val registrations = mutableListOf() + private val corrections = mutableListOf() + private val reports = mutableListOf() + private var marker: UpgradeNotificationMarker = MemoryUpgradeNotificationMarker() + private var online = true + private var enabled = true + private var now = NOW + private var catchUps = 0 + private var fetch: suspend (List) -> AddressCheckAnswer = { ids -> answer(ids) } + private var register: suspend (String) -> Unit = { registrations += it } + private var reread: suspend (PodcastEntity, Long?) -> RefreshOutcome = { _, _ -> RefreshOutcome.Success(null, 1, true) } + + private val telemetry = + object : FeedTelemetryPort { + override fun feedFetched(report: FeedFetchReport) { + reports += report + } + + override fun feedUrlCorrected(correction: FeedUrlCorrection) { + corrections += correction + } + } + + @Before + fun setUp() { + val context = ApplicationProvider.getApplicationContext() + database = Room.inMemoryDatabaseBuilder(context, BoxLoreDatabase::class.java).allowMainThreadQueries().build() + podcasts = database.podcastDao() + repairs = database.upgradeRepairDao() + updater = FeedAddressUpdater(podcasts, requestNotificationRegistration = { registrations += it }, telemetry = telemetry) + } + + @After + fun tearDown() { + database.close() + } + + @Test + fun `a fresh install and rss-only follows settle without network`() = runTest { + val repair = repair() + assertFalse(repair.needsWork()) + assertTrue(marker.isComplete) + + marker = MemoryUpgradeNotificationMarker() + podcasts.upsertRow(show("rss:abc", "https://example.com/rss.xml", notifications = true).copy(sourceType = PodcastEntity.SOURCE_RSS)) + val rssOnly = repair() + assertFalse(rssOnly.needsWork()) + assertEquals(UpgradeRepairRun.DONE, rssOnly.run()) + + assertTrue(calls.isEmpty()) + assertTrue(rereads.isEmpty()) + assertTrue(registrations.isEmpty()) + assertEquals("nothing was repaired, so the sync is left alone", 0, catchUps) + } + + @Test + fun `each show is checked, a confirmed move swapped, read again and registered again`() = runTest { + seed(show("920666", OLD, notifications = true)) + seed(show("920667", OTHER)) + answers[920666] = moved(920666) + + val repair = repair() + assertTrue(repair.needsWork()) + assertEquals(UpgradeRepairRun.DONE, repair.run()) + + val swapped = podcasts.getPodcast("920666")!! + assertEquals(NEW, swapped.feedUrl) + assertEquals(OLD, swapped.previousFeedUrl) + assertEquals("registry:apple", swapped.feedUrlSource) + assertEquals(NOW, swapped.feedUrlCheckedAt) + assertEquals("a show the server doesn't know keeps its address", OTHER, podcasts.getPodcast("920667")!!.feedUrl) + assertEquals(NOW, podcasts.getPodcast("920667")!!.feedUrlCheckedAt) + assertEquals(listOf(FeedUrlCorrection("920666", FeedUrlCorrectionReason.REGISTRY, FeedUrlCorrectionStep.UPGRADE_REPAIR)), corrections) + assertEquals(listOf("920666" to 920666L, "920667" to null), rereads) + for (id in listOf("920666", "920667")) { + val row = repairs.getShow(id)!! + assertEquals(3, row.completedStep) + assertEquals(UpgradeRepairState.DONE, row.state) + } + assertEquals("the writer and step 4 both ask; shows with notifications off never", listOf("920666", "920666"), registrations) + assertTrue(marker.isComplete) + assertEquals(1, catchUps) + assertEquals(listOf(FeedFetchReport.BackgroundPass(FeedFetchSource.FEED, okCount = 2, notModifiedCount = 0, failedCount = 0)), reports) + assertFalse(repair.needsWork()) + } + + @Test + fun `more than 500 followed shows are asked in chunks of at most 500`() = runTest { + repeat(501) { seed(show((1_000 + it).toString(), OLD)) } + + assertEquals(UpgradeRepairRun.DONE, repair().run()) + + assertEquals(listOf(500, 1), calls.map { it.size }) + assertEquals(0, repairs.pendingCount()) + } + + @Test + fun `a partial batch keeps the failed chunk at step 0 without counting a try`() = runTest { + repeat(501) { seed(show((1_000 + it).toString(), OLD)) } + fetch = { ids -> if (calls.isEmpty()) answer(ids) else AddressCheckAnswer.Unavailable.also { calls += ids } } + + assertEquals(UpgradeRepairRun.RETRY, repair().run()) + + assertEquals(1, repairs.pendingCount()) + val waiting = repairs.getShow("1500")!! + assertEquals(0, waiting.completedStep) + assertEquals(0, waiting.attempts) + assertEquals(3, repairs.getShow("1499")!!.completedStep) + assertFalse("step 4 waits for every show", marker.isComplete) + + calls.clear() + fetch = { ids -> answer(ids) } + assertEquals(UpgradeRepairRun.DONE, repair().run()) + assertEquals(listOf(listOf(1500L)), calls) + } + + @Test + fun `offline the address check advances nothing and counts no try`() = runTest { + seed(show("920666", OLD, notifications = true)) + online = false + fetch = { ids -> + calls += ids + throw IOException("offline") + } + + assertEquals(UpgradeRepairRun.RETRY, repair().run()) + + val row = repairs.getShow("920666")!! + assertEquals(0, row.completedStep) + assertEquals(0, row.attempts) + assertTrue(rereads.isEmpty()) + assertTrue(registrations.isEmpty()) + assertFalse(marker.isComplete) + } + + @Test + fun `a read that fails while offline counts no try and stops the run`() = runTest { + seed(show("920666", OLD), completedStep = 2) + seed(show("920667", OTHER), completedStep = 2) + online = false + reread = { _, _ -> RefreshOutcome.Failure("offline", reportStatus = "network") } + + assertEquals(UpgradeRepairRun.RETRY, repair().run()) + + assertEquals(listOf("920666"), rereads.map { it.first }) + for (id in listOf("920666", "920667")) { + assertEquals(0, repairs.getShow(id)!!.attempts) + assertEquals(UpgradeRepairState.PENDING, repairs.getShow(id)!!.state) + } + } + + @Test + fun `process death mid-step resumes at the next step without asking or swapping again`() = runTest { + seed(show("920666", OLD, notifications = true)) + answers[920666] = moved(920666) + reread = { _, _ -> throw CancellationException("process killed") } + + try { + repair().run() + fail("the run was stopped") + } catch (_: CancellationException) { + // Expected: the process died during step 3. + } + assertEquals(2, repairs.getShow("920666")!!.completedStep) + assertEquals(NEW, podcasts.getPodcast("920666")!!.feedUrl) + + calls.clear() + rereads.clear() + reread = { _, _ -> RefreshOutcome.Unchanged(null) } + assertEquals(UpgradeRepairRun.DONE, repair().run()) + + assertTrue("step 1 is not asked again", calls.isEmpty()) + assertEquals("the address is swapped once", 1, corrections.size) + assertEquals(listOf("920666" to 920666L), rereads) + assertEquals(UpgradeRepairState.DONE, repairs.getShow("920666")!!.state) + } + + @Test + fun `a restore with Room done but no step 4 record registers every followed show again`() = runTest { + podcasts.upsertRow(show("920666", OLD, notifications = true)) + podcasts.upsertRow(show("920667", OTHER, notifications = true)) + podcasts.upsertRow(show("920668", OTHER)) + podcasts.upsertRow(show("rss:abc", OTHER, notifications = true).copy(sourceType = PodcastEntity.SOURCE_RSS)) + repairs.upsert(UpgradeRepairShowEntity("920666", completedStep = 3, state = UpgradeRepairState.DONE)) + repairs.upsert(UpgradeRepairShowEntity("920667", completedStep = 1, state = UpgradeRepairState.GAVE_UP)) + + val repair = repair() + assertTrue(repair.needsWork()) + assertEquals(UpgradeRepairRun.DONE, repair.run()) + + assertEquals(listOf("920666", "920667"), registrations.sorted()) + assertEquals("the restored phone's next sync catches up again", 1, catchUps) + assertTrue(calls.isEmpty()) + assertTrue(rereads.isEmpty()) + assertTrue(marker.isComplete) + assertFalse(repair.needsWork()) + } + + @Test + fun `a show unfollowed between steps ends done without a swap or a read`() = runTest { + seed(show("920666", OLD, notifications = true)) + answers[920666] = moved(920666) + fetch = { ids -> + answer(ids).also { podcasts.upsertRow(podcasts.getPodcast("920666")!!.copy(isSubscribed = false)) } + } + + assertEquals(UpgradeRepairRun.DONE, repair().run()) + + val row = repairs.getShow("920666")!! + assertEquals(UpgradeRepairState.DONE, row.state) + assertEquals(1, row.completedStep) + assertEquals(OLD, podcasts.getPodcast("920666")!!.feedUrl) + assertTrue(rereads.isEmpty()) + assertTrue(registrations.isEmpty()) + } + + @Test + fun `a follow added mid-run is left to the daily check and doesn't hold the repair`() = runTest { + seed(show("920666", OLD)) + answers[920666] = moved(920666) + answers[920700] = moved(920700) + fetch = { ids -> answer(ids).also { podcasts.upsertRow(show("920700", OLD, notifications = true)) } } + + assertEquals(UpgradeRepairRun.DONE, repair().run()) + + assertEquals(listOf(listOf(920666L)), calls) + assertNull(repairs.getShow("920700")) + assertEquals(OLD, podcasts.getPodcast("920700")!!.feedUrl) + assertEquals(listOf("920666"), rereads.map { it.first }) + assertTrue(marker.isComplete) + } + + @Test + fun `a dead show gives up at the cap and the repair then finishes`() = runTest { + seed(show("920666", OLD, notifications = true)) + reread = { _, _ -> RefreshOutcome.Failure("gone", addressGone = true, reportStatus = "http_404") } + val repair = repair() + + repeat(UpgradeAddressRepair.MAX_ATTEMPTS - 1) { assertEquals(UpgradeRepairRun.RETRY, repair.run()) } + assertEquals(UpgradeRepairState.PENDING, repairs.getShow("920666")!!.state) + assertTrue("step 4 waits for the show", registrations.isEmpty()) + + assertEquals(UpgradeRepairRun.DONE, repair.run()) + val row = repairs.getShow("920666")!! + assertEquals(UpgradeRepairState.GAVE_UP, row.state) + assertEquals(UpgradeAddressRepair.MAX_ATTEMPTS, row.attempts) + assertEquals("http_404", row.lastError) + assertEquals(UpgradeAddressRepair.MAX_ATTEMPTS, rereads.size) + assertEquals("notifications are registered again even for a show that gave up", listOf("920666"), registrations) + assertEquals(UpgradeRepairRun.DONE, repair.run()) + assertEquals(UpgradeAddressRepair.MAX_ATTEMPTS, rereads.size) + } + + @Test + fun `a different server namespace is recorded, reported and handed to the read only to compare`() = runTest { + seed(show("920666", OLD).copy(epNs = 920666L)) + seed(show("920667", OTHER)) + answers[920666] = V2ShowAddress(id = 920666, fetchUrl = OLD, fetchVia = "direct", feedSource = "registry:apple", epNs = 4242) + answers[920667] = V2ShowAddress(id = 920667, epNs = 920667) + + assertEquals(UpgradeRepairRun.DONE, repair().run()) + + assertEquals(4242L, repairs.getShow("920666")!!.checkedEpNs) + assertEquals("the phone's own namespace is kept", 920666L, podcasts.getPodcast("920666")!!.epNs) + assertEquals("the read gets it only to compare", listOf("920666" to 4242L, "920667" to 920667L), rereads) + assertTrue(corrections.isEmpty()) + assertEquals(listOf(FeedFetchReport.BackgroundPass(FeedFetchSource.FEED, 2, 0, 0, namespaceMismatchCount = 1)), reports) + } + + @Test + fun `the production read keeps the saved address, passes the namespace and starts no auto-download`() = runTest { + val requests = mutableListOf() + val catalog = + object : LocalEpisodeCatalogPort by FakeLocalEpisodeCatalogPort() { + override suspend fun refresh(request: LocalEpisodeCatalogPort.RefreshRequest): RefreshOutcome { + requests += request + return RefreshOutcome.Unchanged(null) + } + } + + val outcome = UpgradeAddressRepair.rereadWith(catalog)(show("920666", OLD).copy(epNs = 920666L), 4242L) + + assertEquals(RefreshOutcome.Unchanged(null), outcome) + val request = requests.single() + assertEquals("920666", request.podcastIndexId) + assertEquals(OLD, request.feedUrl) + assertEquals(4242L, request.serverEpNs) + assertEquals(LocalEpisodeCatalogPort.RefreshReason.NORMAL, request.reason) + assertFalse("the background repair never starts the foreground auto-download pass", request.runPostPersistCallback) + } + + @Test + fun `a follow changed on the phone keeps its address`() = runTest { + seed(show("920666", OLD).copy(isDirty = true, syncedAt = 5L)) + answers[920666] = moved(920666) + + assertEquals(UpgradeRepairRun.DONE, repair().run()) + + val row = podcasts.getPodcast("920666")!! + assertEquals(OLD, row.feedUrl) + assertEquals("the daily check asks again once it is pushed", 0L, row.feedUrlCheckedAt) + assertTrue(corrections.isEmpty()) + assertEquals(UpgradeRepairState.DONE, repairs.getShow("920666")!!.state) + } + + @Test + fun `with the switch off nothing runs and nothing is marked done`() = runTest { + seed(show("920666", OLD, notifications = true)) + enabled = false + + val repair = repair() + assertEquals(UpgradeRepairRun.PAUSED, repair.run()) + + assertTrue(calls.isEmpty()) + assertEquals(1, repairs.pendingCount()) + assertFalse(marker.isComplete) + assertTrue(repair.needsWork()) + + enabled = true + assertEquals(UpgradeRepairRun.DONE, repair.run()) + } + + @Test + fun `the step 4 record survives a new process in its file and starts empty without one`() { + val file = File(folder.root, FileUpgradeNotificationMarker.FILE_NAME) + val first = FileUpgradeNotificationMarker(file) + assertFalse(first.isComplete) + assertTrue(first.registered().isEmpty()) + + assertTrue(first.addRegistered(listOf("920666", "920667"))) + assertTrue(first.markComplete()) + assertFalse(first.syncCatchUpRequested) + assertTrue(first.markSyncCatchUpRequested()) + + val next = FileUpgradeNotificationMarker(file) + assertEquals("its own records are no shows", setOf("920666", "920667"), next.registered()) + assertTrue(next.isComplete) + assertTrue(next.syncCatchUpRequested) + + val restored = FileUpgradeNotificationMarker(File(folder.root, "restored")) + assertFalse(restored.isComplete) + } + + @Test + fun `the swap target is checked again against the row as it is now`() { + val entry = UpgradeRepairShowEntity("920666", completedStep = 1, checkedFetchUrl = NEW, checkedFeedSource = "registry:apple", lastAttemptAt = NOW) + assertEquals(NEW, UpgradeAddressRepair.swapTarget(show("920666", OLD), entry, NOW)) + assertNull("already there", UpgradeAddressRepair.swapTarget(show("920666", NEW), entry, NOW)) + assertNull( + "a confirmed feed move left that address", + UpgradeAddressRepair.swapTarget(show("920666", OLD).copy(feedUrlSource = FeedUrlSources.FEED_MOVE, previousFeedUrl = NEW), entry, NOW), + ) + assertNull("no confirmed move", UpgradeAddressRepair.swapTarget(show("920666", OLD), entry.copy(checkedFetchUrl = null), NOW)) + assertNull("a day old", UpgradeAddressRepair.swapTarget(show("920666", OLD), entry, NOW + FeedAddressDailyCheck.CHECK_INTERVAL_MS)) + assertNull("the daily check has a newer answer", UpgradeAddressRepair.swapTarget(show("920666", OLD).copy(feedUrlCheckedAt = NOW + 1), entry, NOW + 1)) + } + + @Test + fun `an answer left waiting by a pause is asked again later and never applied`() = runTest { + seed(show("920666", OLD, notifications = true)) + answers[920666] = moved(920666) + fetch = { ids -> answer(ids).also { enabled = false } } + + assertEquals("switched off after step 1", UpgradeRepairRun.PAUSED, repair().run()) + assertEquals(1, repairs.getShow("920666")!!.completedStep) + assertEquals(NEW, repairs.getShow("920666")!!.checkedFetchUrl) + + now = NOW + 2 * FeedAddressDailyCheck.CHECK_INTERVAL_MS + enabled = true + answers.remove(920666) + calls.clear() + fetch = { ids -> answer(ids) } + assertEquals(UpgradeRepairRun.DONE, repair().run()) + + assertEquals("asked again", listOf(listOf(920666L)), calls) + assertEquals("the old answer is never applied", OLD, podcasts.getPodcast("920666")!!.feedUrl) + assertTrue(corrections.isEmpty()) + assertEquals(UpgradeRepairState.DONE, repairs.getShow("920666")!!.state) + } + + @Test + fun `a failed swap keeps the time of its answer, so the answer still expires`() = runTest { + seed(show("920666", OLD)) + answers[920666] = moved(920666) + var swaps = 0 + val failingSwap: suspend (FeedAddressChange) -> FeedAddressUpdate = { + swaps++ + error("database busy") + } + + assertEquals(UpgradeRepairRun.RETRY, repair(replace = failingSwap).run()) + val waiting = repairs.getShow("920666")!! + assertEquals(1, waiting.completedStep) + assertEquals(1, waiting.attempts) + assertEquals(NOW, waiting.lastAttemptAt) + + now = NOW + FeedAddressDailyCheck.CHECK_INTERVAL_MS + answers.remove(920666) + assertEquals(UpgradeRepairRun.DONE, repair().run()) + assertEquals(OLD, podcasts.getPodcast("920666")!!.feedUrl) + assertEquals(1, swaps) + } + + @Test + fun `a refused address check counts a try and the show goes on without an answer at the cap`() = runTest { + seed(show("920666", OLD, notifications = true)) + fetch = { ids -> AddressCheckAnswer.Rejected.also { calls += ids } } + val repair = repair() + + repeat(UpgradeAddressRepair.MAX_ATTEMPTS - 1) { assertEquals(UpgradeRepairRun.RETRY, repair.run()) } + val waiting = repairs.getShow("920666")!! + assertEquals(0, waiting.completedStep) + assertEquals(UpgradeAddressRepair.MAX_ATTEMPTS - 1, waiting.attempts) + assertTrue(rereads.isEmpty()) + + assertEquals(UpgradeRepairRun.DONE, repair.run()) + val row = repairs.getShow("920666")!! + assertEquals(UpgradeRepairState.DONE, row.state) + assertNull("no answer, so no swap", row.checkedFetchUrl) + assertEquals(OLD, podcasts.getPodcast("920666")!!.feedUrl) + assertEquals("the read and step 4 still run", listOf("920666" to null), rereads) + assertEquals(listOf("920666"), registrations) + assertEquals(UpgradeAddressRepair.MAX_ATTEMPTS, calls.size) + } + + @Test + fun `an address check the server can't answer counts no try and goes on without an answer after a week`() = runTest { + seed(show("920666", OLD)) + seed(show("920667", OTHER)) + var registryDown = false + fetch = { ids -> + calls += ids + if (registryDown) { + AddressCheckAnswer.Answered(V2ShowsFeedsResponse(registryAvailable = false, results = ids.map { V2ShowAddress(id = it, fetchUrl = OLD) })) + } else { + AddressCheckAnswer.Unavailable + } + } + val repair = repair() + + assertEquals(UpgradeRepairRun.RETRY, repair.run()) + now = NOW + 6 * FeedAddressDailyCheck.CHECK_INTERVAL_MS + registryDown = true + assertEquals(UpgradeRepairRun.RETRY, repair.run()) + for (id in listOf("920666", "920667")) { + val row = repairs.getShow(id)!! + assertEquals(0, row.completedStep) + assertEquals("never counted as a try", 0, row.attempts) + assertEquals("the first failed check", NOW, row.lastAttemptAt) + } + assertTrue(rereads.isEmpty()) + + now = NOW + UpgradeAddressRepair.MAX_UNANSWERED_MS + assertEquals(UpgradeRepairRun.DONE, repair.run()) + assertEquals(listOf("920666" to null, "920667" to null), rereads) + assertEquals(OTHER, podcasts.getPodcast("920667")!!.feedUrl) + assertTrue(corrections.isEmpty()) + assertEquals(0, repairs.pendingCount()) + } + + @Test + fun `the sync catch-up is asked for once, after the last show, never while shows wait or the run is paused`() = runTest { + repeat(501) { seed(show((1_000 + it).toString(), OLD)) } + fetch = { ids -> if (calls.isEmpty()) answer(ids) else AddressCheckAnswer.Unavailable.also { calls += ids } } + val repair = repair() + + assertEquals(UpgradeRepairRun.RETRY, repair.run()) + assertEquals("one show still waits", 0, catchUps) + + enabled = false + assertEquals(UpgradeRepairRun.PAUSED, repair.run()) + assertEquals(0, catchUps) + + enabled = true + fetch = { ids -> answer(ids) } + assertEquals(UpgradeRepairRun.DONE, repair.run()) + assertEquals(1, catchUps) + assertTrue(marker.syncCatchUpRequested) + + assertEquals(UpgradeRepairRun.DONE, repair.run()) + assertFalse(repair.needsWork()) + assertEquals("never again on this install", 1, catchUps) + } + + @Test + fun `a sync catch-up that couldn't be asked for keeps the repair going until it is`() = runTest { + seed(show("920666", OLD)) + var failing = true + val repair = + UpgradeAddressRepair( + repairDao = repairs, + podcastDao = podcasts, + ports = + UpgradeRepairPorts( + fetchAddresses = { ids -> answer(ids) }, + replaceAddress = updater::update, + reread = { _, _ -> RefreshOutcome.Unchanged(null) }, + requestNotificationRegistration = { registrations += it }, + onAddressStepsFinished = { + check(!failing) { "prefs unavailable" } + catchUps++ + }, + ), + marker = marker, + enabled = { true }, + nowMillis = { now }, + ) + + assertEquals(UpgradeRepairRun.RETRY, repair.run()) + assertFalse(marker.syncCatchUpRequested) + assertTrue(repair.needsWork()) + + failing = false + assertEquals(UpgradeRepairRun.DONE, repair.run()) + assertEquals(1, catchUps) + assertFalse(repair.needsWork()) + } + + @Test + fun `a registration request that couldn't be saved is not recorded and is asked again`() = runTest { + seed(show("920666", OLD, notifications = true)) + var saved = false + register = { id -> + check(saved) { "the request wasn't saved" } + registrations += id + } + + assertEquals(UpgradeRepairRun.RETRY, repair().run()) + assertFalse("920666" in marker.registered()) + assertFalse(marker.isComplete) + + saved = true + assertEquals(UpgradeRepairRun.DONE, repair().run()) + assertEquals(listOf("920666"), registrations) + assertTrue("920666" in marker.registered()) + assertTrue(marker.isComplete) + } + + private fun repair(replace: suspend (FeedAddressChange) -> FeedAddressUpdate = updater::update) = UpgradeAddressRepair( + repairDao = repairs, + podcastDao = podcasts, + ports = + UpgradeRepairPorts( + fetchAddresses = { ids -> fetch(ids) }, + replaceAddress = replace, + reread = { row, epNs -> + rereads += row.podcastId to epNs + reread(row, epNs) + }, + requestNotificationRegistration = { register(it) }, + onAddressStepsFinished = { catchUps++ }, + isOnline = { online }, + telemetry = telemetry, + ), + marker = marker, + enabled = { enabled }, + nowMillis = { now }, + ) + + private fun answer(ids: List): AddressCheckAnswer { + calls += ids + return AddressCheckAnswer.Answered(V2ShowsFeedsResponse(registryAvailable = true, results = ids.map { answers[it] ?: V2ShowAddress(id = it) })) + } + + private suspend fun seed(row: PodcastEntity, completedStep: Int = 0) { + podcasts.upsertRow(row) + repairs.upsert(UpgradeRepairShowEntity(row.podcastId, completedStep = completedStep)) + } + + private fun moved(id: Long) = V2ShowAddress( + id = id, + fetchUrl = NEW, + fetchVia = "direct", + feedSource = "registry:apple", + epNs = id, + flags = V2ShowFlags(addressRepair = true), + ) + + private fun show(id: String, feedUrl: String?, notifications: Boolean = false) = PodcastEntity( + podcastId = id, + title = "Show", + author = "Author", + imageUrl = "", + description = null, + isSubscribed = true, + notificationsEnabled = notifications, + feedUrl = feedUrl, + ) + + private companion object { + const val OLD = "https://old.example.com/feed.xml" + const val NEW = "https://new.example.com/feed.xml" + const val OTHER = "https://other.example.com/feed.xml" + const val NOW = 1_800_000_000_000L + } +} diff --git a/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/sync/CloudSyncIdentityTest.kt b/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/sync/CloudSyncIdentityTest.kt index bc8f121bf..4ae42f3ca 100644 --- a/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/sync/CloudSyncIdentityTest.kt +++ b/core/catalog/src/test/java/cx/aswin/boxlore/core/catalog/sync/CloudSyncIdentityTest.kt @@ -295,6 +295,40 @@ class CloudSyncIdentityTest { assertEquals(1, server.pushes.size) } + @Test + fun aCatchUpAskedForAfterTheAddressRepairSendsTheIdentityOfEpisodesItsReadsSettled() = runTest(dispatcher) { + // Own-episode-IDs P4 PR 15: the repair's reads settle aliases after the first catch-up ran. + upgradedDevice(history(PI_ID, dirty = false, syncedAt = 100L)) + server.push(SyncPushRequest(history = listOf(oldBuildRow(PI_ID)))) + server.pushes.clear() + assertTrue(coordinator.syncNow() is SyncResult.Success) + assertEquals(UserSyncCoordinator.IDENTITY_SYNC_METADATA_VERSION, prefs.getSyncMetadataVersion()) + assertNull("nothing known yet", server.history.getValue(PI_ID).recipeId) + + // The repair's read of the feed settles the episode: its alias becomes trusted with a key. + database.episodeIdentityDao().upsertAlias(alias(PI_ID, owner = true, provenance = EpisodeAliasProvenance.FEED_CONFIRMED_KEY)) + assertTrue(coordinator.syncNow() is SyncResult.Success) + assertNull("a delta sync doesn't send it", server.history.getValue(PI_ID).recipeId) + + coordinator.requestIdentityCatchUp() + assertEquals(UserSyncCoordinator.IDENTITY_SYNC_METADATA_VERSION - 1, prefs.getSyncMetadataVersion()) + assertTrue(coordinator.syncNow() is SyncResult.Success) + + assertEquals(0L, server.pulls.last().since) + assertEquals(RECIPE to KEY, server.history.getValue(PI_ID).let { it.recipeId to it.catalogKey }) + assertEquals(UserSyncCoordinator.IDENTITY_SYNC_METADATA_VERSION, prefs.getSyncMetadataVersion()) + } + + @Test + fun aCatchUpRequestLeavesAPhoneThatHasntCaughtUpYetAsItIs() = runTest(dispatcher) { + coordinator.requestIdentityCatchUp() + assertEquals("never synced", 0, prefs.getSyncMetadataVersion()) + + prefs.setSyncMetadataVersion(2) + coordinator.requestIdentityCatchUp() + assertEquals(2, prefs.getSyncMetadataVersion()) + } + @Test fun theCatchUpKeepsEachCloudRowsTimeSoOtherDevicesPullNothingNew() = runTest(dispatcher) { aliases(alias(PI_ID, owner = true)) diff --git a/core/catalog/src/test/resources/fixtures/v2/README.md b/core/catalog/src/test/resources/fixtures/v2/README.md index 79af7de01..e2338b77b 100644 --- a/core/catalog/src/test/resources/fixtures/v2/README.md +++ b/core/catalog/src/test/resources/fixtures/v2/README.md @@ -24,6 +24,8 @@ Worker answers with (the same fields, types and nesting as `shows-feeds.json`): |---|---| | `shows-feeds-registry-unavailable.json` | `POST /v2/shows/feeds` while the address registry is down: `registryAvailable: false`, addresses from the dump | | `feed-report.json` | `POST /v2/feed-report` for show 3391039: the registry's address. A live report would queue a real re-check | +| `shows-feeds-upgrade-cases.json` | `POST /v2/shows/feeds` written from the route's contract (not recorded) for the upgrade repair: a confirmed move (`registry:` source, `flags.addressRepair`), an http-only show (`fetchVia: proxy` with its `transportUrl`), a show the lookup doesn't know (`fetchUrl: null`) and one with another `epNs` | +| `shows-feeds-registry-down.json` | the same route while the address registry is unavailable (`registryAvailable: false`, dump addresses only), with the upgrade-case show ID | Show page routes, recorded the same way on 2026-10-27 (episode lists cut to two items, long text shortened, nested lists cut to their first entry): diff --git a/core/catalog/src/test/resources/fixtures/v2/shows-feeds-registry-down.json b/core/catalog/src/test/resources/fixtures/v2/shows-feeds-registry-down.json new file mode 100644 index 000000000..54a0695d8 --- /dev/null +++ b/core/catalog/src/test/resources/fixtures/v2/shows-feeds-registry-down.json @@ -0,0 +1,15 @@ +{ + "registryAvailable": false, + "results": [ + { + "id": 920666, + "fetchUrl": "https://old.example.com/feed.xml", + "fetchVia": "direct", + "transportUrl": null, + "feedSource": "pidump", + "epNs": 920666, + "flags": { "addressRepair": false, "episodeStale": false }, + "verifiedAt": null + } + ] +} diff --git a/core/catalog/src/test/resources/fixtures/v2/shows-feeds-upgrade-cases.json b/core/catalog/src/test/resources/fixtures/v2/shows-feeds-upgrade-cases.json new file mode 100644 index 000000000..96834d16a --- /dev/null +++ b/core/catalog/src/test/resources/fixtures/v2/shows-feeds-upgrade-cases.json @@ -0,0 +1,45 @@ +{ + "registryAvailable": true, + "results": [ + { + "id": 920666, + "fetchUrl": "https://new.example.com/feed.xml", + "fetchVia": "direct", + "transportUrl": null, + "feedSource": "registry:apple", + "epNs": 920666, + "flags": { "addressRepair": true, "episodeStale": false }, + "verifiedAt": 1792700000 + }, + { + "id": 920667, + "fetchUrl": "http://plain.example.com/feed.xml", + "fetchVia": "proxy", + "transportUrl": "https://api.example.test/v2/feed/920667", + "feedSource": "registry:apple", + "epNs": 920667, + "flags": { "addressRepair": true, "episodeStale": false }, + "verifiedAt": 1792700000 + }, + { + "id": 920668, + "fetchUrl": null, + "fetchVia": null, + "transportUrl": null, + "feedSource": null, + "epNs": 920668, + "flags": { "addressRepair": false, "episodeStale": false }, + "verifiedAt": null + }, + { + "id": 920669, + "fetchUrl": "https://other.example.com/feed.xml", + "fetchVia": "direct", + "transportUrl": null, + "feedSource": "pidump", + "epNs": 4242, + "flags": { "addressRepair": false, "episodeStale": false }, + "verifiedAt": null + } + ] +} diff --git a/core/database/README.md b/core/database/README.md index aebc4d0b0..9bde7a300 100644 --- a/core/database/README.md +++ b/core/database/README.md @@ -13,7 +13,7 @@ Owns the main Room database, entities, DAOs, type converters, and migrations for - `BoxLoreDatabase` and its `getDatabase` factory. - Entities: `PodcastEntity`, `ListeningHistoryEntity`, `ListeningSessionEntity`, `ListeningRollupEntity`, `DownloadedEpisodeEntity`, `RssEpisodeEntity`, `EpisodeSupplementEntity`, `EpisodeSupplementItemEntity`, `LocalEpisodeFeedEntity`, `LocalEpisodeEntity`, `FolderEntity`, `PodcastFolderCrossRef`, `AutoDownloadShowEntity`, `AutoDownloadReleaseEntity`, `EpisodeAliasEntity`, `EpisodeGroupStateEntity`, `UpgradeRepairShowEntity`, `entities.QueueItem`, and `entities.QueueMetadataEntity`. String values for alias states, alias provenance and repair states live in `EpisodeAliasState`, `EpisodeAliasProvenance` and `UpgradeRepairState`; stored values are never renamed. - `EpisodeListenerStore` (from `BoxLoreDatabase.episodeListenerStore()`, so the playback service reaches it without `AppContainer`) keeps one listener state per episode across its stored IDs ([`docs/EPISODE_IDS.md`](../../docs/EPISODE_IDS.md) §5). An `EpisodeGroup` is every trusted alias of one (show, catalog key), owner first; any other ID is a group of its own and reads and writes exactly as before aliases existed. It works out the owner (the stored mark while that ID holds listener state, else the v39 upgrade's order through `EpisodeIdentityDao.chooseOwner`) and stores it in the transaction of the write that needs it (`historyId`). Reads project (`EpisodeListenerProjection`): the owner's row is the base, and progress, position and play time come from the most recently played row (the owner's, unless a pull, a restore or older-build data put later listening under another ID; an owner that holds only a like never hides another ID's progress); liked and completed are the latest decision across the rows and `episode_group_state`, a group setting winning a tie, so an older alias row can't bring back a cleared like or completion; an episode with one stored ID reads its own row unchanged. `projectRows`, `projectedRows` and `projectCompletedIds` count each episode once (the shared-alias query is watched as a Flow, so episodes with one ID cost nothing extra per emission); `projectCompletedIds` also counts an episode that only its group settings mark completed. `projectRowsByMember`, `projectedRowsByMember` and `projectedLikedIds` list an episode once per stored ID, for screens that look an episode up by the ID they show. `lastPlayedSession` is the projected last session for restores and Android Auto. Writes: `setLiked` writes an absolute like to every trusted ID and the group settings (an episode with one stored ID keeps the clock's time on its row, as before); `recordLike`, `recordCompletion` and the batched `recordCompletions` record a decision after the caller wrote the owner's row (an episode with one ID and no row records nothing); `historyIds` works out owners in batches; `removeGroupState`, `restoreGroupState` and `clearGroupStates` remove an episode's settings with its history and put them back on Undo; `downloadHolder(s)` names the ID whose completed download plays an episode; `memberIds`, `withAllAliases`, `groupTokens` and `queueIdentities` serve the queue, Smart Queue, Smart Downloads, sync and ranking; `queueRemovalIds` names the IDs a queue removal records: the removed ones and every other trusted ID of their episodes, queued here or not (a cloud sync pull keeps one queue entry per episode, so the cloud can still hold another), except the IDs of an episode that stays queued or plays; `withAllAliasesFlow`, `memberIdsFlow` and `aliasGroupsFlow` expand IDs to every trusted ID of their episodes in memory from the watched shared aliases, so download and queue indicators read an episode under the ID a screen shows. `ownerRow` gives a new history row written under another ID of the playing episode (its owner) the owner's own show (its stored row, else its catalog row, else its alias), and `keepingOwnerMetadata` keeps a stored owner row's show and episode metadata when another ID's page saves progress, so the owner row of a show switched to its feed stays under the Podcast Index show it belongs to. `showReplacementsFlow` maps each Podcast Index show a followed RSS show replaced (and the listener no longer follows) to that RSS show, for listening read by show. Like times in the group settings stay monotonic within an episode even when the clock reads earlier than a recorded decision. -- DAOs: `EpisodeIdentityDao` (alias and group-state reads and writes, `upsertGroupStates`, `getGroupStates`, `completedGroupStates`, `deleteGroupState`, `deleteAllGroupStates`, `sharedTrustedAliases` / `sharedTrustedAliasesFlow`, `chooseOwner` (the upgrade's owner order, read at runtime), `setOwner` and `holdsListenerState`; when ownership may move and which states resolve a lookup belong to `EpisodeListenerStore`), `UpgradeRepairDao` (pending shows and per-show progress), `PodcastDao`, `ListeningHistoryDao`, `ListeningSessionDao`, `ListeningRollupDao`, `DownloadedEpisodeDao`, `RssEpisodeDao`, `EpisodeSupplementDao`, `LocalEpisodeCatalogDao`, `FolderDao`, and `dao.QueueDao`. `PodcastDao.upsert` and `LocalEpisodeCatalogDao.upsertFeed` keep a stored row's Room v39 columns (see Version 39 below). `ListeningHistoryDao.insertIfAbsent` lets service-only playback establish a row without replacing concurrent UI/history state, `enrichMetadataIfMissing` fills incomplete service seeds without touching progress or likes, `completeFromPlayback` atomically updates only playback-owned completion columns, `reopenProgress` resets completion and advances progress for intentional replays, and `updateLastPlayedAt` refreshes session recency. `ListeningHistoryDao.getHistoryItems`, `setLikeStatusForIds` and `setCompletionForIds` and `DownloadedEpisodeDao.getCompletedDownloadIds`, `getDownloads`, `countCompletedFlow` and `countActiveFlow` read and write an episode's rows under all its IDs. `PodcastDao.observeSwitchedRssShows` watches the followed RSS shows that replaced a Podcast Index show the listener no longer follows. `PodcastDao.markPodcastSyncedIfUnchanged`, `ListeningHistoryDao.markHistorySyncedIfUnchanged`, and `QueueDao.markQueueSyncedIfSequenceMatches` provide multi-column optimistic concurrency flag clearing during cloud sync. `ListeningHistoryDao.getDirtyListeningHistoryBatch` reads one push batch: up to the given number of unsynced rows, most recently played first, without episode descriptions, so a large backlog is never read in full. `PodcastDao.getDirtyCountFlow` and `ListeningHistoryDao.getDirtyCountFlow` observe pending unsynced changes to drive reactive cloud sync ignition. `PodcastDao.clearAllSubscriptionsForAccountSwitch` cleanly purges subscriptions for account switches without wiping cached episodes. Routine progress updates ignore already-completed rows so a delayed snapshot cannot undo terminal state. `PodcastDao.setFeedUrl` writes a publisher HTTPS URL found on the device and marks the row dirty so the next sync push sends it; an unchanged address writes nothing. App code replaces a saved address only through `PodcastDao.replaceFeedAddress(FeedAddressWrite)` (P4 PR 14, used by `:core:catalog`'s `FeedAddressUpdater`): one transaction that writes `feedUrl` and `transportUrl`, keeps the old address in `previousFeedUrl`, records `feedUrlSource` and `feedUrlCheckedAt`, clears the show's ETag and Last-Modified on its row, its `local_episode_feeds` row and its `episode_supplements` row, and marks the row dirty unless the address came from the cloud. It is a compare-and-set on the address the caller read (`feedUrl IS :expected`) and never touches `rss:` rows. `PodcastDao.markFeedUrlChecked` stamps the daily address check without marking the row dirty. No schema change. Listening sessions can reassign an episode id while preserving stable session UUIDs; catalog restore repair merges rollup primary-key collisions before deleting the legacy episode id. Session/rollup DAOs can also reassign a podcast id for the exact legacy RSS repair. `EpisodeSupplementDao.listSupplements` is the export source for Missing episodes? opt-ins during cutover. `LocalEpisodeCatalogDao` pages the first-class PI local catalog (sticky `episodeId`, unique `(podcastId, guid)`); `findOwners` reports which (show, key) holds each episode ID so writers check before `upsertEpisodes` replaces a row. `LocalEpisodeCatalogDao.getEpisodesAfter` and `RssEpisodeDao.getEpisodesAfter` query chronological forward-continuation episodes for same-show queue playback. Unsubscribe sets `ttlExpiresAt`; a non-null TTL is not catalog-ready for live lists. `deleteCatalogIfExpired` rechecks `ttlExpiresAt` in the same transaction so a refresh that cleared TTL after the expiry scan is not deleted. Migration 31→32 copies extras **keeping episodeIds** and does not drop supplement tables. Migration 32→33 adds `customGenre` and `customGenreIcon` columns to `podcasts` for subscribed podcast tag and icon overrides. Migration 33→34 creates `folders` and `podcast_folder_cross_ref` tables with `CASCADE` on `folderId` so deleting folders never deletes shows. Migration 34→35 adds `showPodcastGrid` column to `folders`. Migration 33→35 provides direct upgrade from version 33. Migration 35→36 adds `chaptersUrl` and `transcriptUrl` columns to `downloaded_episodes` for offline chapter and transcript persistence. Migration 36→37 adds `unsubscribedAt`, `isDirty`, and `syncedAt` columns to `podcasts` with index on `isDirty` for subscription tombstoning; adds `likedAt` column to `listening_history` with indices on `isDirty` and `lastPlayedAt` for decoupled likes and delta tracking; and creates `queue_metadata` table for monotonic queue sequence versioning, device origin tracking, dirty sync state, and recent removed episode tombstones. `LocalEpisodeCatalogDao.previousFeedUrl` reads `podcasts.previousFeedUrl`, so a catalog read still heals legacy rows minted under the address the saved one replaced (own-IDs P4 PR 14). +- DAOs: `EpisodeIdentityDao` (alias and group-state reads and writes, `upsertGroupStates`, `getGroupStates`, `completedGroupStates`, `deleteGroupState`, `deleteAllGroupStates`, `sharedTrustedAliases` / `sharedTrustedAliasesFlow`, `chooseOwner` (the upgrade's owner order, read at runtime), `setOwner` and `holdsListenerState`; when ownership may move and which states resolve a lookup belong to `EpisodeListenerStore`), `UpgradeRepairDao` (pending shows, per-show progress, and `hasShows` for whether the upgrade seeded any), `PodcastDao`, `ListeningHistoryDao`, `ListeningSessionDao`, `ListeningRollupDao`, `DownloadedEpisodeDao`, `RssEpisodeDao`, `EpisodeSupplementDao`, `LocalEpisodeCatalogDao`, `FolderDao`, and `dao.QueueDao`. `PodcastDao.upsert` and `LocalEpisodeCatalogDao.upsertFeed` keep a stored row's Room v39 columns (see Version 39 below). `ListeningHistoryDao.insertIfAbsent` lets service-only playback establish a row without replacing concurrent UI/history state, `enrichMetadataIfMissing` fills incomplete service seeds without touching progress or likes, `completeFromPlayback` atomically updates only playback-owned completion columns, `reopenProgress` resets completion and advances progress for intentional replays, and `updateLastPlayedAt` refreshes session recency. `ListeningHistoryDao.getHistoryItems`, `setLikeStatusForIds` and `setCompletionForIds` and `DownloadedEpisodeDao.getCompletedDownloadIds`, `getDownloads`, `countCompletedFlow` and `countActiveFlow` read and write an episode's rows under all its IDs. `PodcastDao.observeSwitchedRssShows` watches the followed RSS shows that replaced a Podcast Index show the listener no longer follows. `PodcastDao.markPodcastSyncedIfUnchanged`, `ListeningHistoryDao.markHistorySyncedIfUnchanged`, and `QueueDao.markQueueSyncedIfSequenceMatches` provide multi-column optimistic concurrency flag clearing during cloud sync. `ListeningHistoryDao.getDirtyListeningHistoryBatch` reads one push batch: up to the given number of unsynced rows, most recently played first, without episode descriptions, so a large backlog is never read in full. `PodcastDao.getDirtyCountFlow` and `ListeningHistoryDao.getDirtyCountFlow` observe pending unsynced changes to drive reactive cloud sync ignition. `PodcastDao.clearAllSubscriptionsForAccountSwitch` cleanly purges subscriptions for account switches without wiping cached episodes. Routine progress updates ignore already-completed rows so a delayed snapshot cannot undo terminal state. `PodcastDao.setFeedUrl` writes a publisher HTTPS URL found on the device and marks the row dirty so the next sync push sends it; an unchanged address writes nothing. App code replaces a saved address only through `PodcastDao.replaceFeedAddress(FeedAddressWrite)` (P4 PR 14, used by `:core:catalog`'s `FeedAddressUpdater`): one transaction that writes `feedUrl` and `transportUrl`, keeps the old address in `previousFeedUrl`, records `feedUrlSource` and `feedUrlCheckedAt`, clears the show's ETag and Last-Modified on its row, its `local_episode_feeds` row and its `episode_supplements` row, and marks the row dirty unless the address came from the cloud. It is a compare-and-set on the address the caller read (`feedUrl IS :expected`) and never touches `rss:` rows. `PodcastDao.markFeedUrlChecked` stamps the daily address check without marking the row dirty. No schema change. Listening sessions can reassign an episode id while preserving stable session UUIDs; catalog restore repair merges rollup primary-key collisions before deleting the legacy episode id. Session/rollup DAOs can also reassign a podcast id for the exact legacy RSS repair. `EpisodeSupplementDao.listSupplements` is the export source for Missing episodes? opt-ins during cutover. `LocalEpisodeCatalogDao` pages the first-class PI local catalog (sticky `episodeId`, unique `(podcastId, guid)`); `findOwners` reports which (show, key) holds each episode ID so writers check before `upsertEpisodes` replaces a row. `LocalEpisodeCatalogDao.getEpisodesAfter` and `RssEpisodeDao.getEpisodesAfter` query chronological forward-continuation episodes for same-show queue playback. Unsubscribe sets `ttlExpiresAt`; a non-null TTL is not catalog-ready for live lists. `deleteCatalogIfExpired` rechecks `ttlExpiresAt` in the same transaction so a refresh that cleared TTL after the expiry scan is not deleted. Migration 31→32 copies extras **keeping episodeIds** and does not drop supplement tables. Migration 32→33 adds `customGenre` and `customGenreIcon` columns to `podcasts` for subscribed podcast tag and icon overrides. Migration 33→34 creates `folders` and `podcast_folder_cross_ref` tables with `CASCADE` on `folderId` so deleting folders never deletes shows. Migration 34→35 adds `showPodcastGrid` column to `folders`. Migration 33→35 provides direct upgrade from version 33. Migration 35→36 adds `chaptersUrl` and `transcriptUrl` columns to `downloaded_episodes` for offline chapter and transcript persistence. Migration 36→37 adds `unsubscribedAt`, `isDirty`, and `syncedAt` columns to `podcasts` with index on `isDirty` for subscription tombstoning; adds `likedAt` column to `listening_history` with indices on `isDirty` and `lastPlayedAt` for decoupled likes and delta tracking; and creates `queue_metadata` table for monotonic queue sequence versioning, device origin tracking, dirty sync state, and recent removed episode tombstones. `LocalEpisodeCatalogDao.previousFeedUrl` reads `podcasts.previousFeedUrl`, so a catalog read still heals legacy rows minted under the address the saved one replaced (own-IDs P4 PR 14). - `EpisodeSupplementDao.replaceAll` / `upsertSupplementAndOptionalItems` keep validator rows and tip items in one Room transaction. - `LocalEpisodeCatalogDao` writes the v39 feed-first columns on their own (P4 PR 16; no schema change): `setArchiveComplete`, `setLastOpenedAt`, and `setTtlAndFollowed`, which follow and unfollow use so `followed` stays in step with the unsubscribe TTL. `listUnfollowedCopies` (`UnfollowedCatalogCopy`: last opened, last read, episode count) lists the copies the show page's cleanup may remove: followed by neither `local_episode_feeds.followed` nor `podcasts.isSubscribed`, and without an unsubscribe TTL. `deleteCatalogIfUnfollowed` checks that again in its transaction and deletes only the show's `local_episodes` and `local_episode_feeds` rows. `seedFollowed` sets a new feed row's `followed` from `podcasts.isSubscribed` (a show followed before its first read), and `searchLimited` is `search` with a `LIMIT` for the show page. - `QueueDao.bumpQueueVersion` keeps `queue_metadata.recentRemovedEpisodes` (a JSON list of `QueueRemovalIdentity`: ID, recipe ID, show and catalog key, written with hand-named fields) in step with the plain `recentRemovedEpisodeIds`. A removal adds one entry per removed ID, with the episode's identity when the caller passes it; restoring the same episode under any ID (same ID, or same show and key) drops its entry, and a removal of an episode that stays queued under another ID isn't recorded there. Both lists keep the newest 50. `QueueDao.richRemovalsWithin` keeps the rich list in step after a pulled queue state replaced the plain list: only entries the plain list still holds and whose episode isn't queued again. Cloud sync pushes the rich list with the queue and pulls the server's (`:core:catalog` `sync.QueueSyncResolver`). diff --git a/core/database/src/main/java/cx/aswin/boxlore/core/database/UpgradeRepairDao.kt b/core/database/src/main/java/cx/aswin/boxlore/core/database/UpgradeRepairDao.kt index 3fe6d82e1..64fcd2409 100644 --- a/core/database/src/main/java/cx/aswin/boxlore/core/database/UpgradeRepairDao.kt +++ b/core/database/src/main/java/cx/aswin/boxlore/core/database/UpgradeRepairDao.kt @@ -20,6 +20,13 @@ interface UpgradeRepairDao { @Query("SELECT COUNT(*) FROM upgrade_repair_shows WHERE state = 'pending'") suspend fun pendingCount(): Int + /** + * Whether the upgrade seeded any show to repair; false on a fresh install. Added in P4 PR 15 of + * the own-episode-IDs plan (plan section 11, upgrade repair). + */ + @Query("SELECT EXISTS(SELECT 1 FROM upgrade_repair_shows)") + suspend fun hasShows(): Boolean + /** One show's repair progress, or null when the show was never part of the repair. */ @Query("SELECT * FROM upgrade_repair_shows WHERE podcastId = :podcastId LIMIT 1") suspend fun getShow(podcastId: String): UpgradeRepairShowEntity? diff --git a/core/model/README.md b/core/model/README.md index 488dc0a53..02ad1bb1a 100644 --- a/core/model/README.md +++ b/core/model/README.md @@ -24,7 +24,7 @@ Owns shared domain models, enums, and pure value helpers used across network, da - `ListeningHistoryRemoval` holds what removing an episode from history took away, for Undo: the chosen row, its sessions and daily totals, and for an episode stored under several IDs every removed row exactly as stored (`ListeningHistoryRowSnapshot`, with like and completion times and how it was completed) and its shared settings (`ListeningGroupStateSnapshot`), so Undo puts the episode back unchanged. - `PodcastGenres` and `RankingAggregateTelemetry`. - `Episode.recipeId` (boxlore's own episode ID) and `Episode.catalogKey` (trimmed GUID, else enclosure URL) are optional and null until a source fills them; `id` stays the local ID. Unset values are not written to JSON, and JSON without them still decodes (`EpisodeIdentityFieldsTest`). -- `FeedTelemetry.kt`: values for the feed analytics events. `FeedFetchReport.ShowPage` (one show-page read) and `FeedFetchReport.BackgroundPass` (one summary per pass and source, with `showsCount` and `passResult`), `FeedFetchSource`, `FeedFetchOutcome`, `FeedFetchCompleteness`, `FeedPassResult`, and `FeedUrlCorrection` with its `FeedUrlCorrectionReason` and `FeedUrlCorrectionStep`. Each enum's `wireName` is its glossary value. None of them holds an address (`FeedTelemetryTest`). +- `FeedTelemetry.kt`: values for the feed analytics events. `FeedFetchReport.ShowPage` (one show-page read) and `FeedFetchReport.BackgroundPass` (one summary per pass and source, with `showsCount`, `passResult` and `namespaceMismatchCount`), `FeedFetchSource`, `FeedFetchOutcome`, `FeedFetchCompleteness`, `FeedPassResult`, and `FeedUrlCorrection` with its `FeedUrlCorrectionReason` and `FeedUrlCorrectionStep`. Each enum's `wireName` is its glossary value. None of them holds an address (`FeedTelemetryTest`). - Cross-promotion model types: `CrossPromotionResult` / `ResolvedCrossPromotion`, `CrossPromotionConfidence` and `CrossPromotionIndicator`, whose `DISTINCT_ARTWORK` marks an episode whose own cover differs from its show's (evidence `:core:catalog`'s detector uses for trailers of another show). Indicators live only in UI state. - `ShowNotes`, `EpisodeLink`, and `EpisodeLinkKind` carry locally parsed plain text, publisher HTML, chapters, resource purpose, destination, and publisher labels without UI or parser-library dependencies. - `Podcast.isLatestEpisodeNew`: shared NEW badge. Room `rssHasNewEpisodes` is true for true-RSS freshness **and** for Podcast Index direct-feed tip promotions (`updateLatestEpisode(..., markAsNew = true)`). Opening the show clears the flag. Otherwise the 48h window / last-seen id rules apply. diff --git a/core/model/src/main/java/cx/aswin/boxlore/core/model/FeedTelemetry.kt b/core/model/src/main/java/cx/aswin/boxlore/core/model/FeedTelemetry.kt index e9f922a52..2e586550c 100644 --- a/core/model/src/main/java/cx/aswin/boxlore/core/model/FeedTelemetry.kt +++ b/core/model/src/main/java/cx/aswin/boxlore/core/model/FeedTelemetry.kt @@ -87,12 +87,17 @@ sealed interface FeedFetchReport { val httpStatus: Int? = null, ) : FeedFetchReport - /** A background pass over many shows (daily check, refresh pass, upgrade repair). */ + /** + * A background pass over many shows (daily check, refresh pass, upgrade repair). + * [namespaceMismatchCount]: reads in the pass of shows whose episode namespace on the server + * differs from the one the phone keeps (decision D12; added in P4 PR 15). + */ data class BackgroundPass( override val source: FeedFetchSource, val okCount: Int, val notModifiedCount: Int, val failedCount: Int, + val namespaceMismatchCount: Int = 0, ) : FeedFetchReport { /** Shows read in the pass; negative counts are treated as zero. */ val showsCount: Int diff --git a/docs/analytics/event_glossary.csv b/docs/analytics/event_glossary.csv index f68470653..9b78ccc8f 100644 --- a/docs/analytics/event_glossary.csv +++ b/docs/analytics/event_glossary.csv @@ -74,7 +74,7 @@ rss_refresh_failed,RSS refresh failed,RSS feed refresh failed,C,RSS reliability, progress_sync_anomaly,Progress sync anomaly,Progress sync inconsistency detected,C,Sync trust,anomaly_type:string:Y;episode_id:string:N;recipe_id:string:N,,playback,,none,Shipped PR9 late_night_safeguard_decision,Late-night safeguard decision,Late-night listening safeguard fired,C,Late-night UX,decision:string:Y,late_night_safeguard_decision,playback;delight,,none, proxy_fallback_triggered,Image proxy fallback,Image load fell back to proxy,C,CDN/proxy health,reason:string:N,proxy_fallback_triggered,infra,,none,Emitted by designsystem OptimizedImage through the AnalyticsHelper façade -feed_fetch_result,Feed fetch result,A show page read its episodes or a background pass read followed feeds,C,Feed reliability by source; partial vs complete reads; failure mix,context:enum:Y;source:enum:Y;result:enum:Y;show_id:string:N;completeness:enum:N;http_status:int:N;shows_count:int:N;ok_count:int:N;not_modified_count:int:N;failed_count:int:N,,catalog;rss,,none,context: show_page|background_pass; source: feed|transport|v2_episodes|cache; result on show_page: ok|not_modified|network_error|timeout|http_error|gone|parse_error; result on background_pass: ok|mixed|failed; completeness: partial|complete (successful show-page reads); http_status on http_error|gone only; show_id on show_page only; counts on background_pass only (one event per pass and source); never a URL +feed_fetch_result,Feed fetch result,A show page read its episodes or a background pass read followed feeds,C,Feed reliability by source; partial vs complete reads; failure mix,context:enum:Y;source:enum:Y;result:enum:Y;show_id:string:N;completeness:enum:N;http_status:int:N;shows_count:int:N;ok_count:int:N;not_modified_count:int:N;failed_count:int:N;namespace_mismatch_count:int:N,,catalog;rss,,none,context: show_page|background_pass; source: feed|transport|v2_episodes|cache; result on show_page: ok|not_modified|network_error|timeout|http_error|gone|parse_error; result on background_pass: ok|mixed|failed; completeness: partial|complete (successful show-page reads); http_status on http_error|gone only; show_id on show_page only; counts on background_pass only (one event per pass and source); namespace_mismatch_count only when the server reports another episode namespace for shows read in the pass; never a URL feed_url_corrected,Feed address corrected,A followed show's saved feed address was replaced,C,How often saved addresses go stale and which path repairs them,show_id:string:Y;reason:enum:Y;step:enum:Y,,catalog;rss,,none,reason: dead|moved|registry; step: upgrade_repair|daily_check|feed_refresh|feed_report; one event per swap; never a URL support_page_viewed,Support page viewed,Support boxlore development screen opened,B,Support discovery; conversion; support-world experiment exposure,support_world:enum:N;assignment_source:enum:N;entry_point:string:N;local_hour:int:N;weekday:int:N,,settings;support,support_view,none,Dual-emits canonical settings_interaction (action=support_page_viewed) support_tier_toggled,Support tier toggled,Support tier artifact swiped or lore inspected,B,Tier preference; lore engagement,tier_title:string:Y;codename:string:Y;amount:string:Y;is_lore_inspect:bool:N;support_world:enum:N;assignment_source:enum:N;tier_index:int:N,,settings;support,support_tier_select,none,Dual-emits canonical settings_interaction (action=support_tier_toggled)