fix k_course fanout (#150)

* fix k_course fanout

Because of the longitudinal nature of course transcripts, we get copies of the same records in each school year -- e.g. a student's 2019 course transcript will appear in 2019, 2020, 2021, etc.

Ideally we would link the course metadata from the year the course was taken to the transcript record, however:
1. We do not have a reliable way to backfill course metadata for arbitrary years of history
2. This is not really how Ed-Fi works either: transcripts are directly linked to the contemporary year's course transcript.

Because dim_course is annualized, our key generation uses the year the transcript was _submitted_ to generate `k_course`. This means that each subsequent copy of the transcript record is unique when we include `k_course` in the grain.

This code changes the grain to the course's non-annualized observables, which has the effect of choosing the course metadata from the year the transcript was submitted (and preferring the most recent submission), which avoids fanning out the transcript record by the `k_course` corresponding to each unique year in which we received it.

* apply 2 dedupes to retrieve all most recent non-deleted records

* rename to correct CTEs

* update filter for is_deleted

* duplicated code

---------

Co-authored-by: gnguyen87 <gnguyen@macalester.edu>
Co-authored-by: rlittle08 <rlittle@edanalytics.org>

Author: 3 people

models/staging/edfi_3/stage/stg_ef3__course_transcripts.sql

@@ -19,17 +19,28 @@ keyed as (
         {{ extract_extension(model_name=this.name, flatten=True) }}
     from base_course_transcripts
 ),
-deduped as (
+first_deduped as (
     {{
         dbt_utils.deduplicate(
             relation='keyed',
             partition_by='k_course, k_student_academic_record, course_attempt_result',
             order_by='api_year desc, last_modified_timestamp desc, pull_timestamp desc'
         )
+    }}      
+),
+no_deletes as (
+    select * from first_deduped
+    {% if not is_incremental() %}
+    where not is_deleted
+    {% endif %}
+),
+final_deduped as (
+    {{
+        dbt_utils.deduplicate(
+            relation='no_deletes',
+            partition_by='course_code, course_ed_org_id, k_student_academic_record, course_attempt_result',
+            order_by='api_year desc, last_modified_timestamp desc, pull_timestamp desc'
+        )
     }}
 )
-select * from deduped
-{% if not is_incremental() %}
-where not is_deleted
-{% endif %}
-
+select * from final_deduped