Add evidence URL type detection and validation for submissions#558
Open
Add evidence URL type detection and validation for submissions#558
Conversation
Introduce an EvidenceURLType model that categorizes evidence URLs (X Post, GitHub Repo, GitHub File, GitHub PR/Issue, YouTube Video, Studio Contract, Other) with regex-based pattern matching. On submission, URLs are auto-detected, validated against the contribution type's accepted set, checked for duplicates via an indexed normalized_url field, and verified for handle ownership (X and GitHub). The frontend shows inline URL type badges, client-side mismatch warnings, and embedded social account linking when ownership verification requires it. Both SubmitContribution and EditSubmission pages share the same evidence validation UX. ## Claude Implementation Notes - backend/contributions/models.py: Add EvidenceURLType model with url_patterns JSONField, handle_extract_pattern, ownership_social_account. Add M2M on ContributionType, FK on Evidence, normalized_url indexed field with auto-population on save() - backend/contributions/url_utils.py: New module with normalize_url, detect_url_type, extract_handle, validate_handle_ownership, check_duplicate_url. Duplicate check uses indexed normalized_url for O(1) lookups - backend/contributions/serializers.py: Add LightEvidenceURLTypeSerializer, EvidenceURLTypeSerializer. Extend ContributionTypeSerializer with accepted_evidence_url_types (falls back to all types when M2M empty) and has_explicit_accepted_types flag. Add evidence validation pipeline in _validate_evidence_items (type detection, mismatch rejection, duplicate check, ownership check) - backend/contributions/admin.py: Register EvidenceURLTypeAdmin, add filter_horizontal for accepted types on ContributionType, add url_type to Evidence inline/admin - backend/contributions/views.py: Add prefetch_related for accepted_evidence_url_types and evidence url_type - backend/contributions/management/commands/review_submissions.py: Replace local _normalize_url with shared utility, add _normalize_url_for_blocklist to prevent bypass via query params - backend/contributions/migrations/0050-0053: Schema migration, seed data (8 URL types), normalized_url field, backfill migration - backend/contributions/tests/test_url_utils.py: Tests for normalization, detection, handle extraction, ownership validation, duplicate checking - frontend/src/components/portal/submit-contribution/SubmitContribution.svelte: Client-side URL type detection via regex, green/red type badges, accepted types hint, pre-submit mismatch validation, inline SocialLink for X/GitHub account linking, evidenceRequiredAccounts derived state disabling submit - frontend/src/routes/EditSubmission.svelte: Mirror all evidence URL type features from SubmitContribution -- detection, badges, social linking, type validation, structured error handling - backend/CLAUDE.md: Document new model, serializers, and url_utils module
Remove YouTube Video, Medium Article, and Blog Post evidence URL types since their authorship cannot be verified deterministically. Merge all four migrations (schema, seed, normalized_url, backfill) into a single 0050 migration. The seed function also explicitly deletes removed types from databases where they were previously applied. ## Claude Implementation Notes - backend/contributions/migrations/0050_evidence_url_types.py: Single migration combining schema creation, seed data, normalized_url field, and backfill. Adds REMOVED_SLUGS list to clean up stale types - backend/contributions/migrations/0050-0053: Deleted individual migrations - backend/contributions/tests/test_url_utils.py: YouTube/medium/blog tests now assert generic fallback, ownership test uses github-pr instead of youtube-video
Each evidence slot now has a dropdown where the user selects the evidence type (X Post, GitHub Repo, etc.) before entering the URL. Auto-detection still runs on URL blur to pre-select the type as a convenience. A mismatch warning appears if the URL doesn't match the selected type. Removed the "expected evidence" hint, has_explicit_accepted_types flag, and isUrlTypeAccepted function since the selector makes these unnecessary. ## Claude Implementation Notes - frontend/src/components/portal/submit-contribution/SubmitContribution.svelte: Replace URL type badges with evidence type <select> dropdown per slot. Auto-detect pre-selects on blur. Mismatch warning instead of rejection badge. Remove expected evidence hint. Add handleEvidenceTypeChange. evidenceRequiredAccounts uses selectedType instead of detectedType - frontend/src/routes/EditSubmission.svelte: Same evidence type selector pattern. Existing evidence pre-selects type from auto-detection on load. Remove unused hasExplicitAcceptedTypes and isUrlTypeAccepted - backend/contributions/serializers.py: Remove has_explicit_accepted_types field and getter from ContributionTypeSerializer
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan
python manage.py test contributions.tests.test_url_utils