Conversation
Implements on-demand and scheduled integrity verification for backups. - New verification service (src/services/storage/verification-service.ts) that reads .meta.json sidecars, attempts adapter-native verification (S3/Cloudflare R2/Hetzner via metadata, Google Drive MD5, OneDrive SHA-256, local filesystem via direct hash) and falls back to download+SHA-256 when needed. Writes verification results back to the .meta.json sidecar. - Exposes a POST API route /api/storage/[id]/verify to trigger manual verifications. - UI/UX: adds an Integrity column, a Verify action button, toasts and client handlers (dashboard columns, actions cell, storage client) to support manual verification and display results. - Adapters: added verifyChecksum implementations for S3 (uses HeadObject metadata), Google Drive (md5Checksum), OneDrive (sha256Hash) and local filesystem (direct hash). S3 upload now attaches dbackup-sha256 metadata when provided. - Core changes: added UploadOptions and optional verifyChecksum to StorageAdapter interface; upload flows now compute SHA-256+MD5 in a single pass and pass checksums to adapters; withStorageSession shim updated to forward options. - Runner: post-upload integrity verification integrated via verificationService; post-upload verification is opt-in via system setting (local filesystem remains verified). IntegrityService refactored to use verificationService for scheduled runs and now aggregates results. - StorageService and types updated to include verification info in returned file metadata. - Added calculateFileChecksums utility for efficient dual-hash computation. - Updated changelog to document vNEXT integrity features.
Add horizontal scrolling to the DataTable component by applying overflow-x-auto and responsive max-width constraints to the table container so columns can be scrolled when they overflow. Also update the changelog to document the UI improvement: all data tables (Storage, Jobs, Sources, Destinations, Notifications) now support horizontal scrolling to match the Database Explorer behavior.
Add a new docs page describing the Backup Verification feature (docs/user-guide/features/backup-verification.md). The page explains checksum generation, adapter-specific verification methods, triggers (manual, post-upload, scheduled), scheduler behavior, interpretation of results, and limitations. Also update the VitePress sidebar (docs/.vitepress/config.mts) to include a link to the new Backup Verification page under Storage Explorer so it appears in the documentation navigation.
Introduce a dedicated Integrity modal for storage items and wire up checksum support across the stack. UI: new IntegrityModal component (copyable SHA-256/MD5 display, last verification status, Verify Now flow) and the StorageClient now opens the modal instead of performing immediate fetches. Data model: FileInfo/RichFileInfo extended with checksum and checksumMd5, and StorageService surfaces these values from sidecar files. Runner: upload step now logs MD5 alongside SHA-256. Also removed the previous inline Integrity column from the table and updated the changelog entry to reflect the modal and MD5 history logging. The Verify action refreshes the file list after verification completes.
Truncate long backup names in the Storage Explorer and show the full name on hover. Update NameCell to add min-w-0 and shrink-0 layout tweaks, wrap the name in a Tooltip with a truncated max width (max-w-64), and keep the path display unchanged. Also update the changelog to document the improvement.
Enable filtering for scheduled integrity checks (skip already-passed, max age, max file size). Adds a server action to persist settings (src/app/actions/settings/integrity-settings.ts), a client modal to edit them (IntegrityCheckSettingsModal), and a gear button on the Integrity Check task row in SystemTasksSettings to open the modal. The IntegrityService now reads these settings and applies age/size/skip filters when enumerating files, and VerificationService gains an option to skip verification for already-passed files and a new 'skipped' status. Also load initial integrity settings in the Settings page and update the changelog ordering.
Introduce a SystemTaskRunner and integrate integrity check execution and UI. Added src/lib/runner/system-task-runner.ts to manage system-task executions, streaming logs, stages, and progress. Extended core logs with INTEGRITY_CHECK stages, order and progress map. Updated IntegrityService to perform a two-pass scan/verify flow, expose progress callbacks, and return work items for verification. Wire runner into SystemTaskService to start, stream, and finalize IntegrityCheck executions (including handling trigger type/label). Frontend: add system task columns and a System Tasks tab in history, handle IntegrityCheck labeling and proper live-updating of logs. Also pass the invoking user name from the settings API when manually triggering a task. Minor adjustments to log viewer to support integrity check stage ordering.
Use adapter-provided file.path (full relative path) as remotePath so downloads/read operations resolve correctly for adapters that return flat recursive listings. Replace folder-based listing logic with a single aggregated file list (falling back to per-job listings when root listing fails) and filter out .meta.json files. Add human-friendly skip reason mapping for verifyResult.status to improve log messages and keep existing age/size filtering and error handling.
Introduce async execution for single-file verification and scheduled integrity checks so callers receive an executionId immediately and the work runs in the background with live progress and history entries. Added a new POST /api/storage/[id]/verify-async route and VERIFICATION stage order/progress mapping. systemTaskService.runTask now returns the execution id and runs IntegrityCheck tasks asynchronously (returns runner.id). UI changes: Integrity Modal now uses verify-async and respects the "Auto-redirect on job start" preference (redirects to live History view), System Tasks settings will redirect if executionId is returned, History view treats "Verification" as a system task type, and LogViewer supports VERIFICATION stages and auto-expands the active running stage. Updated docs/changelog and small preference copy tweak to include system tasks.
Introduce a cached Storage Explorer listing stored in SQLite to speed up repeated views and reduce remote adapter calls. Adds a new Prisma model and migration (StorageListCache), storage-service methods (invalidate/append/remove/update/reconcile, listFilesWithMetadata with bypassCache), and background reconcile (stale-while-revalidate, 2h staleness). Wire cache updates into upload, retention, verification, delete and download flows (surgical updates instead of full invalidation). Add a system task (system.warmup_storage_cache) to pre-warm caches on startup and daily at 3 AM, and expose a ?refresh=true API flag plus UI refresh to bypass cache. Update package.json to auto-apply migrations on dev start and add database:reset script. Documentation and changelog updated with developer guidance and storage-cache docs.
Change the storage cache warmup task to run every hour and perform reconciliation: it now detects files deleted outside DBackup and pre-populates caches for adapters without a cache. Implementation: updated system task schedule and description, added per-adapter logic to check for an existing cache and either call reconcileStorageListCache or listFilesWithMetadata, adjusted logging and error messages, and made reconcileStorageListCache non-private. Documentation (changelog and developer guide) updated to reflect the new hourly reconcile behavior and the per-adapter logic.
Update unit tests and minor UI/test fixes after refactoring storage verification and checksum APIs. Tests now mock and use verificationService.verifyFile and calculateFileChecksums instead of previous checksum helpers; adjust expectations accordingly across runner and service tests. Also import startTransition in LogViewer and use it when updating active stages, add a SystemTaskRunner mock for system-task tests, and ensure storageListCache.upsert is treated as async in storage-service tests. Changelog updated to note the test changes.
Introduce per-job and per-destination verification opt-outs and a configurable integrity scan mode. Adds a Prisma migration and schema change (Job.skipVerification) and persists integrity.scanMode system setting. The jobs API and JobService accept and store skipVerification; the job form UI exposes a Skip Verification switch. Adapter forms accept metadata.skipVerification and show a Disable Verification switch for storage destinations. IntegrityService now supports two scan modes (jobs vs destinations), implements gatherFilesFromJobs (respects job and destination skip flags, filters, and dedupes), and skips destinations with metadata.skipVerification. Settings UI and save actions updated to read/save scanMode; changelog updated accordingly.
Add prisma generate to the dev script so pnpm dev runs migrations, regenerates the Prisma client, then starts Next. Also update the changelog entry to note that pnpm dev now regenerates the Prisma client on startup.
Add live token validation and re-authorize UI for OAuth adapters, fix a Google Drive path bug, and improve integrity scanning.
- Add POST validation endpoints: /api/adapters/{dropbox,google-drive,onedrive}/validate-token to verify stored refresh tokens and return a friendly expiry message when needed.
- Update Dropbox, Google Drive and OneDrive OAuth buttons to perform live validation on Connection tab open, show a loading state while checking, display an expiry alert with a Re-authorize button when expired, and otherwise keep the green authorized state. Import AlertTriangle and useEffect for these flows; network failures fail-open to avoid blocking UX.
- Fix Google Drive adapter bug where resolveOrCreatePath used the wrong rootFolderId argument when navigating into a subdir.
- Revamp integrity-service: deduplicate destinations across jobs, list each destination once, filter listed files by job prefix, apply max-age and max-file-size filters per-file, and build work items without duplicate entries. Improved logging and error handling for adapter/config resolution and listing failures.
- Update changelog to document the bug fix: OAuth token expiry now shows a re-authorize option instead of a misleading authorized state.
These changes make OAuth status more accurate for users and make integrity scans more efficient and correct.
Stop filtering skipVerification at the DB query and instead fetch all enabled jobs so we can report which jobs have verification disabled and skip them explicitly. Introduces eligibleJobs/skippedJobs separation and logs skipped jobs. Tracks per-job file counts (jobCounts) while building work items and logs counts per job. Also restricts destination/job collection to eligible jobs and performs minor cleanup.
Enable copying and downloading execution logs from the History dialog. Updated docs/changelog.md and modified the history page to add Copy and Download buttons (hidden for Running/Pending logs) and handlers that sanitize and format logs before export. Added src/lib/logs/sanitize.ts to redact IPs, connection strings and keys from SENSITIVE_KEYS, and src/lib/logs/format.ts to produce a human-readable .log text (metadata header, UTC timestamps, stage grouping, durations, details) and to generate a filename from job name and start time. Export uses navigator.clipboard for copy and a Blob/URL download for .log files; handlers rely on existing parseLogs.
Add downgrade detection and presentation for database version history. Introduces an isDowngrade flag in notification types and recordVersionIfChanged; uses compareVersions to mark and persist downgrades. UI: explorer shows up/down arrows and distinct warning styling for downgrades. Notifications: templates now emit separate Upgrade/Downgrade titles, badges, and colors. Tests and changelog updated accordingly.
Add comprehensive unit tests covering log formatting and sanitization, the SystemTaskRunner lifecycle, and the VerificationService (native and download-based checksum paths). Extend IntegrityService tests to cover jobs-mode scanning and maxAge filtering, and fix JobService test payload to include skipVerification. Update docs/changelog.md to mention the new/updated tests.
Refactor OAuth button components to avoid stale/incorrect token state when credentialId or authorized props change: introduce a tokenCheck object, derive tokenState from it, and add an active cleanup flag to prevent state updates after unmount or race conditions. Preserve fail-open behavior on fetch errors. Also remove an unused `path` import from integrity-service.ts.
Update unit tests to satisfy TypeScript types and add missing storage-service mocks. Cast call.data.metadata/logs to string before JSON.parse in SystemTaskRunner tests, add vi.mock stubs for storageService (append/update/remove) in upload and retention tests, refine prisma.systemSetting.findUnique mock typing and remove an any cast, and add verifyChecksum field to the adapter mock in verification-service tests.
Add client-side search, column sorting, and refresh support to the Database Explorer table list. Introduces a search input with clear button, clickable column headers to sort by Name/Type/Rows/Size (with visual sort icons), and a refresh button that shows loading state. Uses useMemo for filtered/sorted results and preserves existing empty/missing-size handling. Also updates the changelog to document the new Explorer table list search and sorting features.
Bump project and docs to v2.7.0 for the v2.7.0 release. Updated version fields in package.json and docs/package.json, updated OpenAPI metadata in api-docs/openapi.yaml and public/openapi.yaml, and added the v2.7.0 release entry (released June 14, 2026) to docs/changelog.md, including updated Docker image tags.
Add a vi.mock for '@/services/storage/storage-service' in tests/unit/runner/multi-destination-upload.test.ts to stub storageService.appendStorageListCacheEntry (mockResolvedValue undefined). This prevents real storage cache modifications during the unit test run.
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.