feat(action_log): Allow GroupDerivedData rows to be regenerated without deleting them

[kcons]

Rather than just delete and regenerate, this updates our GroupDerivedData update scheme to allow for safe in-place updates after regeneration.

This is done by introducing a 'generated_at' timestamp and using it as an indicator of the log state included.
We can check generated_at when updating a GroupDerivedData row, and if it is different we know that it may be working with a different log than us.
This field allows us to trigger generation runs safely and ensure the latest one wins.
Using this new capacity, we introduce introduce replacement regeneration tasks that efficiently and safely update the GroupDerivedData rows in place.

Fixes ISWF-3055.

[github-actions]

This PR has a migration; here is the generated SQL for src/sentry/migrations/1141_add_is_live_to_group_derived_data.py

for 1141_add_is_live_to_group_derived_data in sentry

--
-- Remove index sentry_grou_progres_f1627f_idx from groupderiveddata
--
DROP INDEX CONCURRENTLY IF EXISTS "sentry_grou_progres_f1627f_idx";
--
-- Remove index sentry_grou_last_pr_8c8185_idx from groupderiveddata
--
DROP INDEX CONCURRENTLY IF EXISTS "sentry_grou_last_pr_8c8185_idx";
--
-- Add field is_live to groupderiveddata
--
ALTER TABLE "sentry_groupderiveddata" ADD COLUMN "is_live" boolean DEFAULT false NOT NULL;
--
-- Add field generation_id to groupderiveddata
--
ALTER TABLE "sentry_groupderiveddata" ADD COLUMN "generation_id" bigint DEFAULT 0 NOT NULL;
--
-- Alter field group on groupderiveddata
--
SET CONSTRAINTS "sentry_groupderivedd_group_id_5f3f73c7_fk_sentry_gr" IMMEDIATE; ALTER TABLE "sentry_groupderiveddata" DROP CONSTRAINT "sentry_groupderivedd_group_id_5f3f73c7_fk_sentry_gr";
ALTER TABLE "sentry_groupderiveddata" DROP CONSTRAINT "sentry_groupderiveddata_group_id_key";
CREATE INDEX CONCURRENTLY "sentry_groupderiveddata_group_id_5f3f73c7" ON "sentry_groupderiveddata" ("group_id");
ALTER TABLE "sentry_groupderiveddata" ADD CONSTRAINT "sentry_groupderivedd_group_id_5f3f73c7_fk_sentry_gr" FOREIGN KEY ("group_id") REFERENCES "sentry_groupedmessage" ("id") DEFERRABLE INITIALLY DEFERRED NOT VALID;
ALTER TABLE "sentry_groupderiveddata" VALIDATE CONSTRAINT "sentry_groupderivedd_group_id_5f3f73c7_fk_sentry_gr";
--
-- Create index sentry_gdd_progress_live on field(s) progress, group of model groupderiveddata
--
CREATE INDEX CONCURRENTLY "sentry_gdd_progress_live" ON "sentry_groupderiveddata" ("progress", "group_id") WHERE "is_live";
--
-- Create index sentry_gdd_lastprog_live on field(s) last_progressed_at, group of model groupderiveddata
--
CREATE INDEX CONCURRENTLY "sentry_gdd_lastprog_live" ON "sentry_groupderiveddata" ("last_progressed_at", "group_id") WHERE "is_live";
--
-- Create index sentry_grou_group_i_2bcdf6_idx on field(s) group, is_live of model groupderiveddata
--
CREATE INDEX CONCURRENTLY "sentry_grou_group_i_2bcdf6_idx" ON "sentry_groupderiveddata" ("group_id", "is_live");
--
-- Create index sentry_gdd_stale_cleanup on field(s) date_added of model groupderiveddata
--
CREATE INDEX CONCURRENTLY "sentry_gdd_stale_cleanup" ON "sentry_groupderiveddata" ("date_added") WHERE NOT "is_live";
--
-- Create constraint uniq_live_gdd_per_group on model groupderiveddata
--
CREATE UNIQUE INDEX CONCURRENTLY "uniq_live_gdd_per_group" ON "sentry_groupderiveddata" ("group_id") WHERE "is_live";

[github-actions]

This PR has a migration; here is the generated SQL for src/sentry/migrations/1141_add_invalidated_at_to_group_derived_data.py

for 1141_add_invalidated_at_to_group_derived_data in sentry

--
-- Add field invalidated_at to groupderiveddata
--
ALTER TABLE "sentry_groupderiveddata" ADD COLUMN "invalidated_at" timestamp with time zone NULL;
--
-- Create index sentry_gdd_needs_rebuild on field(s) invalidated_at of model groupderiveddata
--
CREATE INDEX CONCURRENTLY "sentry_gdd_needs_rebuild" ON "sentry_groupderiveddata" ("invalidated_at") WHERE "invalidated_at" IS NOT NULL;

[github-actions]

This PR has a migration; here is the generated SQL for src/sentry/migrations/1141_add_generated_at_to_group_derived_data.py

for 1141_add_generated_at_to_group_derived_data in sentry

--
-- Add field generated_at to groupderiveddata
--
ALTER TABLE "sentry_groupderiveddata" ADD COLUMN "generated_at" timestamp with time zone DEFAULT (STATEMENT_TIMESTAMP()) NOT NULL;

[linear-code]

ISWF-3055

[kcons]

FWIW, once this is considered acceptable I'll pull out the migration and do it first.

[sentry-warden]

db_default references nonexistent models.functions.Now()

Import Now from django.db.models.functions and change db_default to Now(). django.db.models does not expose a functions attribute, so this line raises AttributeError at class definition time on every startup.

Evidence

• groupderiveddata.py:62 evaluates db_default=models.functions.Now() while executing the GroupDerivedData class body.
• models is django.db.models, which does not have a functions attribute.
• Every other model usage in the codebase (e.g., groupactionlogentry.py:6) imports Now from django.db.models.functions.
• This causes an AttributeError on every application startup before any request is handled.

_{Identified by Warden sentry-backend-bugs · YQJ-6WZ}

[sentry-warden]

promote_to_live regresses cursor when concurrent incremental processing advances ahead

The update guard in promote_to_live matches rows with an older generated_at regardless of cursor position. When a full regeneration runs concurrently with incremental processing, the generation can overwrite a more advanced cursor, causing duplicate processing and incorrect derived state.

Evidence

• _process_batch increments the live row's cursor but does not update generated_at, so the DB row retains its older generation timestamp.
• build_and_promote_derived_data drains the log with a new generated_at = timezone.now() and then calls promote_to_live.
• promote_to_live filters with Q(generated_at__lt=generated_at), which matches the live row even when its cursor is ahead of the candidate's cursor.
• .update(**values) then stamps the candidate's older cursor into the row, regressing progress so future incremental passes re-process already-handled entries.
• The CURSOR_BEHIND reload-and-retry block is bypassed because promote_to_live returns PROMOTED instead of CURSOR_BEHIND.

_{Identified by Warden sentry-backend-bugs · GPH-MJ7}