feat: [COSMO2-846] Audit Expiry Urgency enrollment-time assignment with persisted expiry by santhosh-apphelix-2u · Pull Request #197 · edx/edx-platform

santhosh-apphelix-2u · 2026-03-26T03:31:07Z

Summary

This PR implements the Audit Expiry Urgency experiment in edx-platform.

The experiment assigns eligible audit learners to one of two variants at enrollment time and persists the backend decision so that access expiration behavior remains stable across sessions and configured target courses.

Variants:

control_5_7_weeks
expiry_7_days

The backend remains the source of truth for:

learner assignment
persisted audit expiry
experiment metadata
backend experiment analytics

What This PR Does

Enrollment-time assignment and persistence

When an eligible learner enrolls in a target audit course, the backend:

determines the learner’s experiment variant
reuses an existing learner-level sticky assignment across configured target courses
computes the correct audit expiry datetime
persists the experiment state on the enrollment

Persisted experiment attributes:

experiment_key
variant
expiry_days
assigned_at
decision_source
audit_expiry_at

All metadata is stored in CourseEnrollmentAttribute under the audit_expiry_experiment namespace.

Read-side access expiration support

The existing access_expiration response is extended to include experiment context:

experiment_key
variant
expiry_days

This allows frontend consumers to use the same backend response for both expiration behavior and experiment-specific display or analytics needs.

Backend analytics

The backend emits Optimizely events for:

experiment exposure:
- audit_expiry_urgency_exposed
verified upgrade conversion:
- audit_expiry_urgency_upgraded_to_verified

Sticky-assignment behavior

Assignment is learner-level and sticky across the configured target courses.

If a learner already has a persisted experiment assignment, that assignment is reused instead of creating a new one.

Rollback and fallback behavior

audit_expiry_at remains the source of truth for access enforcement
if Optimizely is unavailable or returns an unexpected variation, the backend falls back to control_5_7_weeks
existing waffle-flag-based rollback behavior remains intact

Configuration and Selection Flow

Course target list selection

The experiment uses AUDIT_EXPIRY_EXPERIMENT_COURSES to determine which course runs are eligible.

Course list resolution happens in this order:

Site Configuration value
local/settings fallback value

So if AUDIT_EXPIRY_EXPERIMENT_COURSES is configured in Site Configuration, that value is used. If it is not configured there, the backend falls back to the locally configured settings value.

Variant selection order

Once an enrollment is eligible for the experiment, backend variant selection happens in this order:

forced local variant from AUDIT_EXPIRY_FORCE_VARIANT
existing sticky variant already persisted for the learner across configured target courses
Optimizely assignment
fallback to control_5_7_weeks

This means Site Configuration controls course eligibility, while learner variant selection is handled separately by the experiment helper logic.

Post-Deployment Steps

Enable waffle flag

Enable:

experiments.audit_expiry_urgency_v1.enabled

Add Site Configuration

Configure AUDIT_EXPIRY_EXPERIMENT_COURSES with the target course IDs:

{
  "AUDIT_EXPIRY_EXPERIMENT_COURSES": [
    "<target-course-run-1>",
    "<target-course-run-2>",
    "<target-course-run-3>"
  ]
}

…ment and persisted expiry

Copilot

Pull request overview

Implements the “Audit Expiry Urgency (v1)” experiment by assigning an Optimizely variant at enrollment time, persisting the chosen variant + audit_expiry_at on the enrollment, and updating Course Duration Limits (CDL) reads to prefer the persisted expiry behind a waffle-flag kill switch.

Changes:

Add enrollment post_save signal handling to persist experiment attributes (variant, audit_expiry_at) via CourseEnrollmentAttribute.
Add AUDIT_EXPIRY_URGENCY_V1_ENABLED waffle flag and update CDL expiration-date reads to use persisted audit_expiry_at when enabled.
Add test coverage for both persistence/stickiness behavior and the CDL read-path behavior with flag on/off.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
openedx/features/course_duration_limits/access.py	Reads persisted `audit_expiry_at` when the waffle flag is enabled, otherwise falls back to computed CDL logic.
openedx/features/course_duration_limits/tests/test_access.py	Adds tests asserting persisted expiry is used only when the kill switch is enabled.
lms/djangoapps/experiments/audit_expiry_urgency.py	Implements eligibility checks, Optimizely activation/reuse logic, expiry computation, and persistence via enrollment attributes.
lms/djangoapps/experiments/signals.py	Hooks enrollment `post_save` to run persistence logic safely (never blocking enrollment).
lms/djangoapps/experiments/apps.py	Ensures signals are imported during app initialization.
lms/djangoapps/experiments/flags.py	Introduces the `AUDIT_EXPIRY_URGENCY_V1_ENABLED` waffle flag with annotations.
lms/djangoapps/experiments/tests/test_audit_expiry_urgency.py	Adds tests for allowlisting, Optimizely failure fallback, cross-course stickiness, idempotency, and forced-variant behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-26T03:37:45Z

    access_duration = get_user_course_duration(user, course)
    if access_duration is None:
        return None

    enrollment = enrollment or CourseEnrollment.get_enrollment(user, course.id)
    if enrollment is None or enrollment.mode != CourseMode.AUDIT:
        return None

+    # Audit Expiry Urgency (v1) experiment: if an explicit persisted expiry exists,
+    # it is the single source of truth for UI/API reads.
+    # Kill switch behavior: when disabled, ignore any persisted experiment values
+    # and fall back to the default computed CDL logic.
+    if AUDIT_EXPIRY_URGENCY_V1_ENABLED.is_enabled():
+        persisted_expiry_attr = (
+            CourseEnrollmentAttribute.objects.filter(


get_user_course_expiration_date always calls get_user_course_duration() before checking for a persisted audit_expiry_at. That means even when a persisted expiry exists (and the flag is on), we still do the CDL duration computation (including a potential Catalog API lookup via get_expected_duration). Consider reordering so the persisted-expiry lookup happens first (after verifying audit enrollment), and only fall back to get_user_course_duration() when no persisted expiry is present.

Copilot · 2026-03-26T03:37:45Z

 from openedx.features.course_duration_limits.models import CourseDurationLimitConfig
+from common.djangoapps.student.models import CourseEnrollmentAttribute
+from edx_toggles.toggles.testutils import override_waffle_flag
+from lms.djangoapps.experiments.flags import AUDIT_EXPIRY_URGENCY_V1_ENABLED


This import section is likely not isort-compliant: CourseEnrollmentAttribute (from common.djangoapps...) is inserted after openedx.features... imports, which will typically fail import-order linting. Consider regrouping/sorting imports so all common.djangoapps.* imports stay together with the other common.* imports at the top of the local-import block.

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-30T07:45:46Z

+        assert computed is not None
+        assert computed != persisted


This test asserts computed != persisted, which can become brittle if the default CDL-computed duration ever happens to equal the chosen persisted value (e.g., if discovery returns 6 weeks). To make the intent deterministic, assert that the returned value matches the expected computed CDL expiration (recompute it the same way as the function does) rather than relying on inequality with an arbitrary timestamp.

Suggested change

assert computed is not None

assert computed != persisted

# Recompute the expected CDL-based expiration date deterministically,

# to verify that the persisted audit_expiry_at value is ignored.

content_availability_date = max(enrollment.created, overview.start)

access_duration = get_user_course_duration(user, overview)

expected_course_expiration_date = content_availability_date + access_duration

assert computed is not None

assert computed == expected_course_expiration_date

Copilot · 2026-03-30T07:45:47Z

+        user = None
+        enrollment_1 = CourseEnrollmentFactory.create(course=self.course_1, mode=CourseMode.AUDIT, is_active=True)
+        user = enrollment_1.user
+        enrollment_2 = CourseEnrollmentFactory.create(user=user, course=self.course_2, mode=CourseMode.AUDIT, is_active=True)


This line exceeds the repository’s configured max line length (120) and will trigger line-too-long in pylint (see pylintrc max-line-length). Please wrap the CourseEnrollmentFactory.create(...) call across multiple lines.

Suggested change

enrollment_2 = CourseEnrollmentFactory.create(user=user, course=self.course_2, mode=CourseMode.AUDIT, is_active=True)

enrollment_2 = CourseEnrollmentFactory.create(

user=user,

course=self.course_2,

mode=CourseMode.AUDIT,

is_active=True,

)

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Refactor code for better readability by formatting the creation of CourseEnrollment objects.

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Removed unnecessary blank lines in test_audit_expiry_urgency.py.

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Refactor eligibility checks for audit expiry urgency.

Removed comments to simplify the code and address pylint warnings.

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Refactor enrollment processing logic to use a dedicated function for eligibility checks, improving code clarity and maintainability.

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-07T14:46:45Z

+    if AUDIT_EXPIRY_URGENCY_V1_ENABLED.is_enabled():
+        experiment_attributes = {
+            attr.name: attr.value
+            for attr in enrollment.attributes.filter(namespace='audit_expiry_experiment')


experiment_attributes is built from an unordered queryset, so if there are multiple CourseEnrollmentAttribute rows for the same name (which this PR explicitly notes can happen), the chosen value can be nondeterministic and may not reflect the latest persisted assignment. Consider ordering by id (or -id) and/or fetching the latest value per attribute name (e.g., order the queryset and let later rows overwrite earlier ones, or query last() per name) so reads are stable and consistent.

Suggested change

for attr in enrollment.attributes.filter(namespace='audit_expiry_experiment')

for attr in enrollment.attributes.filter(

namespace='audit_expiry_experiment'

).order_by('id')

Copilot · 2026-04-07T14:46:46Z

+        configured = []
+    if isinstance(configured, str):
+        configured = [configured]
+    if not isinstance(configured, list):


_get_configured_target_course_id_strings() only accepts list for the allowlist value. If AUDIT_EXPIRY_EXPERIMENT_COURSES is configured as a tuple (a common pattern in Django settings) or any other sequence type, this code will silently treat it as invalid and behave as if no courses are allowlisted. Consider accepting tuple (and possibly other iterables) in addition to list (e.g., isinstance(configured, (list, tuple))).

Suggested change

if not isinstance(configured, list):

if not isinstance(configured, (list, tuple)):

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-07T15:21:49Z

+        experiment_attributes = {
+            attr.name: attr.value
+            for attr in enrollment.attributes.filter(namespace='audit_expiry_experiment')
+        }


get_access_expiration_data builds a dict from enrollment.attributes.filter(namespace='audit_expiry_experiment') without an explicit ordering. Since CourseEnrollmentAttribute has no uniqueness constraint (duplicates are possible), this can yield nondeterministic/stale values for experiment_key/variant/expiry_days depending on DB ordering. Consider ordering by id and ensuring the latest attribute per name wins (or fetching the latest row per attribute name explicitly).

Suggested change

experiment_attributes = {

attr.name: attr.value

for attr in enrollment.attributes.filter(namespace='audit_expiry_experiment')

}

experiment_attributes = {}

for attr in enrollment.attributes.filter(

namespace='audit_expiry_experiment'

).order_by('id'):

experiment_attributes[attr.name] = attr.value

Copilot · 2026-04-07T15:21:50Z

+    if any((
+        not enrollment.is_active,
+        enrollment.mode != CourseMode.AUDIT,
+        not is_target_course(enrollment.course_id),
+    )):
+        return
+
+    # Idempotency: do not overwrite once set.
+    if get_persisted_audit_expiry_at(enrollment):
+        return
+
+    access_duration = get_user_course_duration(enrollment.user, enrollment.course_overview)
+    if access_duration is None:
+        # Only apply if Course Duration Limits would normally apply.
+        return
+
+    target_course_ids = _get_configured_target_course_id_strings()
+    variant, decision_source = choose_variant(enrollment.user, target_course_ids)


maybe_persist_audit_expiry_urgency_attributes calls is_target_course(enrollment.course_id) (which reads site config) and then later calls _get_configured_target_course_id_strings() again to choose/reuse a variant. This results in duplicate config lookups per save when the flag is on; consider loading the configured target course IDs once and reusing it for both the eligibility check and choose_variant().

feat: add Audit Expiry Urgency experiment with enrollment-time assign…

adf0569

…ment and persisted expiry

Copilot AI review requested due to automatic review settings March 26, 2026 03:31

Copilot started reviewing on behalf of santhosh-apphelix-2u March 26, 2026 03:31 View session

Copilot AI reviewed Mar 26, 2026

View reviewed changes

santhosh-apphelix-2u added 4 commits March 26, 2026 08:54

chore: remove debug logs and prints from audit expiry urgency experiment

541a127

fix: default to control variant when Optimizely is unavailable

a06391c

fix: Lint fix

6595796

fix: Lint fix

a19e5a8

Copilot AI review requested due to automatic review settings March 30, 2026 07:19

Copilot started reviewing on behalf of santhosh-apphelix-2u March 30, 2026 07:19 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

Comment thread lms/djangoapps/experiments/audit_expiry_urgency.py Outdated

santhosh-apphelix-2u added 2 commits March 30, 2026 07:26

fix: Lint fix

f0a4bf7

fix: Lint fix

32e3883

Copilot AI review requested due to automatic review settings March 30, 2026 07:39

Copilot started reviewing on behalf of santhosh-apphelix-2u March 30, 2026 07:39 View session

fix: Lint fix

7caa8d3

Copilot AI reviewed Mar 30, 2026

View reviewed changes

santhosh-apphelix-2u added 3 commits March 30, 2026 07:59

fix: Lint fix

e8da82f

fix: Lint fix

9439d50

fix: Remove unnecessary blank lines in apps.py

eff28a8

Copilot AI review requested due to automatic review settings March 30, 2026 08:19

Copilot started reviewing on behalf of santhosh-apphelix-2u March 30, 2026 08:20 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

fix: Refactor CourseEnrollment creation for clarity

610f6e5

Refactor code for better readability by formatting the creation of CourseEnrollment objects.

Copilot AI review requested due to automatic review settings March 30, 2026 08:32

Copilot started reviewing on behalf of santhosh-apphelix-2u March 30, 2026 08:33 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

fix: Clean up blank lines in test_audit_expiry_urgency.py

53ab604

Removed unnecessary blank lines in test_audit_expiry_urgency.py.

Copilot AI review requested due to automatic review settings March 30, 2026 08:39

Copilot started reviewing on behalf of santhosh-apphelix-2u March 30, 2026 08:40 View session

edx deleted a comment from Copilot AI Mar 31, 2026

fix: Refactor enrollment processing logic for clarity

b34868c

Copilot AI review requested due to automatic review settings March 31, 2026 02:12

Copilot started reviewing on behalf of santhosh-apphelix-2u March 31, 2026 02:12 View session

Copilot AI reviewed Mar 31, 2026

View reviewed changes

Comment thread lms/djangoapps/experiments/audit_expiry_urgency.py Outdated

Comment thread lms/djangoapps/experiments/audit_expiry_urgency.py Outdated

fix: Consolidate eligibility checks in compute_audit_expiry_at

37bafc4

Refactor eligibility checks for audit expiry urgency.

Copilot AI review requested due to automatic review settings March 31, 2026 02:40

Copilot started reviewing on behalf of santhosh-apphelix-2u March 31, 2026 02:41 View session

fix: Remove comments to simplify eligibility checks

a0dd0db

Removed comments to simplify the code and address pylint warnings.

Copilot AI reviewed Mar 31, 2026

View reviewed changes

Comment thread lms/djangoapps/experiments/signals.py

Comment thread lms/djangoapps/experiments/flags.py

fix: Refactor enrollment eligibility checks

71e9c9d

Refactor enrollment processing logic to use a dedicated function for eligibility checks, improving code clarity and maintainability.

Copilot AI review requested due to automatic review settings March 31, 2026 02:49

Copilot started reviewing on behalf of santhosh-apphelix-2u March 31, 2026 02:49 View session

Copilot AI reviewed Mar 31, 2026

View reviewed changes

Comment thread lms/djangoapps/experiments/audit_expiry_urgency.py Outdated

Comment thread lms/djangoapps/experiments/audit_expiry_urgency.py Outdated

Comment thread lms/djangoapps/experiments/flags.py

fix: refactor

b944e43

jcapphelix approved these changes Mar 31, 2026

View reviewed changes

feat: add audit expiry experiment metadata and analytics

39174c9

Copilot AI review requested due to automatic review settings April 7, 2026 14:40

Copilot started reviewing on behalf of santhosh-apphelix-2u April 7, 2026 14:40 View session

Copilot AI reviewed Apr 7, 2026

View reviewed changes

santhosh-apphelix-2u added 2 commits April 7, 2026 14:48

feat: add Optimizely tracking to audit expiry banner

1187c34

fix: add missing experiment helper docstrings

4a6d251

Copilot AI review requested due to automatic review settings April 7, 2026 15:16

Copilot started reviewing on behalf of santhosh-apphelix-2u April 7, 2026 15:17 View session

Copilot AI reviewed Apr 7, 2026

View reviewed changes

santhosh-apphelix-2u merged commit aa1b064 into release-ulmo Apr 9, 2026
97 of 107 checks passed

santhosh-apphelix-2u deleted the feature/audit-expiry-urgency branch April 9, 2026 08:15

-        assert computed is not None
-        assert computed != persisted
+        # Recompute the expected CDL-based expiration date deterministically,
+        # to verify that the persisted audit_expiry_at value is ignored.
+        content_availability_date = max(enrollment.created, overview.start)
+        access_duration = get_user_course_duration(user, overview)
+        expected_course_expiration_date = content_availability_date + access_duration
+        assert computed is not None
+        assert computed == expected_course_expiration_date

	if not isinstance(configured, list):
	if not isinstance(configured, (list, tuple)):

Conversation

santhosh-apphelix-2u commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What This PR Does

Enrollment-time assignment and persistence

Read-side access expiration support

Backend analytics

Sticky-assignment behavior

Rollback and fallback behavior

Configuration and Selection Flow

Course target list selection

Variant selection order

Post-Deployment Steps

Enable waffle flag

Add Site Configuration

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 26, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

santhosh-apphelix-2u commented Mar 26, 2026 •

edited

Loading