Fix pseudo-irregular time series direct reads

[krowvin]

Summary

Fixes the time-series direct-read path for pseudo-irregular old-style local-regular IDs such as *.~15Minutes.* when X-CWMS-LRTS-Formatting: true is present.

Root cause

The direct-read path could resolve pseudo-irregular old-style IDs through new LRTS formatting, producing local-regular metadata and returning no values. It also treated ~ intervals as regular without first honoring interval_utc_offset = UTC_OFFSET_IRREGULAR, which could incorrectly invoke gap-fill behavior for pseudo-irregular data.

Changes

• Detect old-style ~ IDs whose old-format metadata has irregular interval offset.
• Resolve/read those IDs under old LRTS formatting for direct reads.
• Suppress regular gap-fill and report zero interval duration for truly irregular offset metadata.
• Add an integration regression test that seeds a pseudo-irregular ~15Minutes series and verifies the CDA response matches retrieve_ts under the LRTS formatting header.

Validation

• ./gradlew.bat :cwms-data-api:compileJava :cwms-data-api:compileTestJava --no-daemon
• ./gradlew.bat :cwms-data-api:integrationTests --tests "*TimeSeriesDirectReadParityIT.pseudoIrregularReadWithLrtsHeaderMatchesRetrieveTs" "-PCDA_POOL_INIT_SIZE=5" "-PCDA_POOL_MAX_ACTIVE=10" "-PCDA_POOL_MAX_IDLE=5" "-PCDA_POOL_MIN_IDLE=2" --no-daemon
• ./gradlew.bat :cwms-data-api:checkstyleMain :cwms-data-api:checkstyleTest --no-daemon

Note: broader TimeSeriesDirectReadParityIT runs were not clean in this local container due fixture/setup issues before the relevant endpoint path was exercised, including Oracle seed partition check failures and a default-data county error.

Closes #1745

[krowvin]

pseudo-irregular ~15Minutes series were being handled like true local-regular series in the direct-read path, which caused incorrect LRTS/session behavior and large window reads

changes

• This fix uses database authoritative LRTS detection via CWMS_TS_PACKAGE.call_IS_LRTS__2(...).
• Pseudo-irregular series with interval_utc_offset == UTC_OFFSET_IRREGULAR are treated as irregular and return observed rows only.
• True local-regular/LRTS series still use expected-time generation and gap-fill behavior.
• Removed the old workaround that forced old LRTS formatting through recursive DAO construction and raw JDBC row reads.
• Kept the direct row read in jOOQ against AV_TSV_DQU.
• Row reads now filter by requested UNIT_ID and normalize quality in Java.
• Added/adapted regression coverage for ITPARPIRR.Flow.Inst.~15Minutes.0.BENCH with X-CWMS-LRTS-Formatting: true.
• Verified the pseudo-irregular regression test passes.
• Verified full TimeSeriesDirectReadParityIT passes locally.
• Verified compile and checkstyle tasks pass locally with JDK 21.
• Tested MVP timeseries Baldhill_Dam.Flow-Out.Inst.~15Minutes.0.best.

tests

• For a 70,177-row production sized window (2024-01-01 to 2026-01-01), local CDA returned the same row count, timestamps, qualities, and pseudo-irregular metadata.
• Local timing for that 70k-row window was about 2 seconds, compared with production runs around 29-38 seconds.
• pulled data from each instance, local and prod, to confirm no drift in datasets.

timings

For this command

localhost

curl -sS -w '\nstatus=%{http_code} time=%{time_total}s bytes=%{size_download}\n' \
  -H 'Accept: application/json;version=2' \
  -H 'X-CWMS-LRTS-Formatting: true' \
  'http://localhost:8081/cwms-data/timeseries?name=Baldhill_Dam.Flow-Out.Inst.~15Minutes.0.best&office=MVP&unit=cfs&begin=2025-05-01T00:00:00Z&end=2025-06-01T00:00:00Z&page-size=100000'

national

curl -sS -w '\nstatus=%{http_code} time=%{time_total}s bytes=%{size_download}\n' \
  -H 'Accept: application/json;version=2' \
  'https://cwms-data.usace.army.mil/cwms-data/timeseries?name=Baldhill_Dam.Flow-Out.Inst.~15Minutes.0.best&office=MVP&unit=cfs&begin=2025-05-01T00:00:00Z&end=2025-06-01T00:00:00Z&page-size=100000'

(LRTS header not in Prod yet)

Target	Window	Rows	Runs	Min	Median	Mean	Max	Next Page
Local CDA	2024-01-01 to 2026-01-01	70,177	5	1.815s	1.903s	2.075s	2.668s	No
Production CDA	2024-01-01 to 2026-01-01	70,177	3	29.088s	37.805s

[MikeNeilson]

Note possible conflicts with #1780 depending on which gets merged in first.

[MikeNeilson]

TSV PR was merged in, conflicts need to be fixed. Also see my comments about LRTS methods and usage, feels like something is stuck down a rabbit hole of over thinking and needs to get dug out.

Ask database if this TSID is LRTS or REGULAR interval
yes -> have expected interval set in some way shape or form
no -> just return the data from the tsv query as it, the timestamps provided are by definition correct.

[MikeNeilson]

Still some concerns with the overall logic, but some of that wasn't touched in this PR.

Actual concerns address in specific comments.

Otherwise, looks reasonable, and the tests show expected behavior correctly.

[MikeNeilson]

why are we spending time formatting a timestamp that can be used as-is in the query?

[krowvin]

When I was trying to run the test for TimeSeriesDirectReadParityIT it started failing/saying missing.

I agree converting from one and back to the other is wasted cycles though. Any ideas on how to make it so it'll match up with AV_TSV_DQU.DATE_TIME?

[MikeNeilson]

Hmm. that's, unexpected, especially since it's a begin/end time stamp so a few seconds on either side shouldn't matter much. Might have something to do with the columns being Date instead of Timestamp in the database.

Presumably the performance is still improved overall? If so, I'd argue that challenge is reasonable cause to leave the extra conversions in. Not like it happens for every row. Just put a comment in about why for future readers.

[adamkorynta]

Was it failing in your dev environment, or on GitHub Actions? I'm curious if your local time zone is getting in the mix somehow

[krowvin]

This was failing in my local integration test run. The GitHub checks were green with the current to_date(bindText) comparison.

When I switched the AV_TSV_DQU.DATE_TIME to direct Timestamp locally TimeSeriesDirectReadParityIT started returning missing/zero direct read rows compared to retrieve_ts. I left the request conversion and added a comment explaining. Probably onto something about the local TZ though.

[MikeNeilson]

is your local database a docker instance or something else?

[MikeNeilson]

Given the conversion only happens once, it's not a huge performance hit, some I'm okay with this explanation for the time being. We'll want to look into it more, but we don't think we should hold up this change just for that.

[krowvin]

Spoke in CDA biweekly if RMA would also be able to review and ensure nothing was missed.