Skip to content

Pluvio#1191

Draft
burlingamet wants to merge 39 commits into
masterfrom
pluvio
Draft

Pluvio#1191
burlingamet wants to merge 39 commits into
masterfrom
pluvio

Conversation

@burlingamet
Copy link
Copy Markdown
Contributor

No description provided.

burlingamet and others added 30 commits July 23, 2025 15:07
Merge branch 'master' into pluvio

# Conflicts:
#	pipe/pluvio/pluvio_calibration_loader.yaml
Merge branch 'master' of github.com:NEONScience/NEON-IS-data-processing into pluvio

# Conflicts:
#	pipe/pluvio/pluvio_cron_daily_and_date_control.yaml
Copilot AI review requested due to automatic review settings May 11, 2026 18:28
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Pluvio precipitation processing for precipWeighingv2 by switching to Pluvio-specific QA/QC + stats containers, adding a publication (group/format/package/egress) pipeline chain, and extending the Pluvio R wrappers with new unit tests and minor logic changes.

Changes:

  • Replace the legacy precipWeighingv2_stats_group_and_compute pipeline (removed) with updated Pluvio QM/stats processing and update downstream consolidation to consume it.
  • Add monthly publication control and new pub pipelines for precipWeighingv2 (group → format/package → egress/publish).
  • Update Pluvio calibration loader/assignment plumbing and add new wrap.precip.pluvio.flags / wrap.precip.pluvio.stats unit tests + wrapper logic tweaks.

Reviewed changes

Copilot reviewed 17 out of 22 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
pipe/precipWeighingv2/precipWeighingv2_stats_group_and_compute.yaml Removes the old stats group/compute pipeline (superseded by QM stats path).
pipe/precipWeighingv2/precipWeighingv2_qm_stats_group_and_compute.yaml Updates the Pluvio stats container image tag used for QM/stats computation.
pipe/precipWeighingv2/precipWeighingv2_analyze_pad_and_qaqc.yaml Updates Pluvio flags image tag and removes passing a (now-optional) plausibility schema.
pipe/precipWeighingv2/precipWeighingv2_level1_group_consolidate_srf.yaml Updates group prefix and points consolidation at the QM stats pipeline output.
pipe/precipWeighingv2/precipWeighingv2_cron_monthly_and_pub_control.yaml Adds monthly cron/pub control pipeline to drive publication timing.
pipe/precipWeighingv2/precipWeighingv2_pub_group.yaml Adds publication grouping pipeline for monthly publication staging.
pipe/precipWeighingv2/precipWeighingv2_pub_format_and_package.yaml Adds formatting + packaging workflow prior to egress/publish.
pipe/precipWeighingv2/precipWeighingv2_pub_egress_and_publish.yaml Adds egress-to-bucket and publish/sync workflow (includes MDP logic).
pipe/precipWeighingv2/pipe_list_precipWeighingv2.txt Updates pipeline list to include new pub chain and consolidation.
pipe/pluvio/pluvio_cron_daily_and_date_control.yaml Adjusts start/end dates and disables cron via @never (consistent with repo patterns).
pipe/pluvio/pluvio_calibration_loader.yaml Simplifies loader command structure and updates env vars/output path.
pipe/pluvio/pluvio_calibration_assignment.yaml Switches assignment input repo from test files to the real calibration loader output.
flow/flow.precip.pluvio.stats/wrap.precip.pluvio.stats.R Adjusts NA aggregation logic and updates finalQF composition behavior.
flow/flow.precip.pluvio.flags/wrap.precip.pluvio.flags.R Changes heater flagging logic to use inlet temperature and tweaks schema handling on write.
flow/tests/testthat/test-wrap-precip-pluvio-stats.R Adds unit tests for Pluvio stats wrapper output and behavior.
flow/tests/testthat/test-wrap-precip-pluvio-flags.R Adds unit tests for Pluvio flags wrapper output and behavior.
flow/tests/testthat/pfs/precipPluvioStats/.../pluvio_*_uncertaintyCoef.json Adds test fixture uncertainty coefficient JSON.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread flow/flow.precip.pluvio.stats/wrap.precip.pluvio.stats.R
Comment thread flow/flow.precip.pluvio.stats/wrap.precip.pluvio.stats.R
Comment thread flow/flow.precip.pluvio.flags/wrap.precip.pluvio.flags.R
Comment thread pipe/precipWeighingv2/precipWeighingv2_pub_format_and_package.yaml
Comment thread pipe/precipWeighingv2/precipWeighingv2_pub_egress_and_publish.yaml
Comment thread pipe/precipWeighingv2/precipWeighingv2_pub_egress_and_publish.yaml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants