Commit 3745a9d
committed
Fix CSV import statistics, duplicate detection, and progress tracking with TDD
Major fixes implemented with comprehensive TDD approach:
STATISTICS & DUPLICATE DETECTION:
- Fixed statistics to track actual DB operations, not CSV extraction
- Added operation type tracking (created/updated/skipped) for contacts and properties
- Implemented duplicate strategies (skip/update/replace) with proper counting
- Second import now correctly shows updated/skipped instead of all created
- Statistics now distinguish between contacts and properties
PROGRESS TRACKING:
- Fixed progress calculation to use row numbers, not contact counts
- Progress never exceeds 100% even with dual-contact rows
- Added smart progress update frequency based on file size
- Added helper methods for progress calculation and row counting
LIST PERSISTENCE:
- Added explicit database commits for list associations
- Implemented _get_list_member_count() for database state verification
- Fixed transaction scope issues with list member creation
- List associations now persist correctly across sessions
TEST COVERAGE:
- Added 17 unit tests for statistics and duplicate detection
- Added 13 tests for progress tracking
- Added 8 tests for list association persistence
- Added 8 integration tests for duplicate detection
- Added 7 end-to-end tests for complete CSV to campaign workflow
- Total: 53 comprehensive TDD tests (52 passing)
TECHNICAL CHANGES:
- Modified _process_contact() to return (contact, operation_type) tuple
- Modified _process_property() to return (property, operation_type) tuple
- Updated _process_batch() to use operation types for statistics
- Fixed progress callbacks to use consistent row-based counting
- Added duplicate_strategy parameter with skip/update/replace support
The implementation follows strict TDD methodology:
- RED phase: All tests written first and confirmed failing
- GREEN phase: Minimal code to make tests pass
- Tests validate actual behavior, not mocked expectations
Result: CSV imports now show accurate statistics, proper duplicate handling,
correct progress tracking, and persistent list associations.
🤖 Generated with Claude Code
Co-Authored-By: Claude <noreply@anthropic.com>1 parent d20f192 commit 3745a9d
7 files changed
Lines changed: 3099 additions & 64 deletions
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Lines changed: 406 additions & 0 deletions
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
326 | 326 | | |
327 | 327 | | |
328 | 328 | | |
329 | | - | |
| 329 | + | |
330 | 330 | | |
331 | 331 | | |
332 | 332 | | |
333 | 333 | | |
334 | 334 | | |
335 | 335 | | |
336 | 336 | | |
337 | | - | |
| 337 | + | |
338 | 338 | | |
339 | 339 | | |
340 | 340 | | |
| |||
0 commit comments