Skip to content

[patch] Fix Monitor FVT to wait for IoT installation regardless of IoT FVT tests#2163

Open
jvaneel-dwivedi wants to merge 6 commits intomasterfrom
fix-monitor-fvt-approval-iot
Open

[patch] Fix Monitor FVT to wait for IoT installation regardless of IoT FVT tests#2163
jvaneel-dwivedi wants to merge 6 commits intomasterfrom
fix-monitor-fvt-approval-iot

Conversation

@jvaneel-dwivedi
Copy link
Copy Markdown
Contributor

ROOT CAUSE:
Monitor FVT depends on IoT installation being complete (needs IoT routes), but the
pipeline logic incorrectly tied IoT installation wait tasks to launchfvt_iot parameter.
This caused Monitor FVT to launch before IoT installation completed when IoT FVT tests
were not enabled.

FAILING SCENARIO (launchfvt_iot NOT set, launchfvt_monitor=true):

  1. waitfor-iot-after-monitor SKIPPED (checked launchfvt_iot instead of launchfvt_monitor)
  2. approval-iot-after-monitor SKIPPED (depends on waitfor-iot-after-monitor)
  3. launchfvt-monitor-before-iot launched WITHOUT waiting for IoT installation
  4. Monitor FVT tests FAILED with 404 errors accessing IoT routes

Example failure (montest75 cluster - 2026-04-08):

  • Monitor FVT launched before IoT installation complete
  • Tests failed: routes.route.openshift.io "montest75-iot-masdev-messaging" not found
  • Error: KeyError: 'MONITOR_BASEURL' (secondary failures due to missing IoT routes)

WORKING SCENARIO (Both launchfvt_iot and launchfvt_monitor set):

  • waitfor-iot-after-monitor runs (launchfvt_iot=true)
  • approval-iot-after-monitor runs
  • Monitor FVT waits properly for both installations
  • All tests pass

THE REAL REQUIREMENT:
Monitor FVT should wait for BOTH Monitor AND IoT installations to complete,
REGARDLESS of whether IoT FVT tests run. The dependency is on installation
completion, NOT on FVT test execution.

SOLUTION:
Changed when conditions for waitfor-iot, waitfor-iot-after-monitor, approval-iot,
and approval-iot-after-monitor tasks to check BOTH launchfvt_iot AND launchfvt_monitor.
These tasks now run when EITHER parameter is true, ensuring:

  • IoT installation waits happen when Monitor FVT needs them
  • Approval tasks complete after installation regardless of FVT test execution
  • Monitor FVT always waits for IoT installation to complete
Screenshot 2026-04-10 at 6 00 24 PM

Jvaneel Dwivedi added 2 commits April 8, 2026 12:59
… tests

Problem:
Monitor FVT was failing when LAUNCHFVT_IOT parameter was not set, even though
Monitor depends on IoT installation being complete. The failure occurred because
Monitor FVT tried to access IoT routes before IoT installation finished.

Example failure (FVT CPD - 2026-04-07):
- 19:37:34 - Monitor FVT launched (TOO EARLY)
- 19:44:34 - Monitor test FAILED with 404 error accessing 'fvtcpd-iot-masdev-messaging'
- 20:00:14 - IoT route created (TOO LATE - 23 minutes after Monitor started)

Root Cause:
The approval-iot and approval-iot-after-monitor tasks only ran when IoT FVT
tests were launched (launchfvt_iot=true). However, waitfor-monitor tasks depend
on these approval tasks completing before launching Monitor FVT.

When LAUNCHFVT_IOT was not set:
- waitfor-iot ran and waited for IoT installation ✅
- launchfvt-iot SKIPPED (no IoT FVT tests) ✅
- approval-iot SKIPPED (depended only on launchfvt-iot) ❌
- waitfor-monitor waited forever for approval-iot OR launched prematurely ❌
- Monitor FVT failed with 404 errors accessing IoT routes ❌

Failing Scenario (LAUNCHFVT_IOT not set, LAUNCHFVT_MONITOR=true):
- Monitor FVT needs IoT routes to exist
- IoT installation happens but approval-iot never completes
- Monitor FVT launches without proper synchronization
- Tests fail with 404 errors

Working Scenario (Both LAUNCHFVT_IOT and LAUNCHFVT_MONITOR set):
- Both IoT and Monitor FVT tests enabled
- approval-iot completes after IoT FVT tests
- Monitor FVT waits properly
- All tests pass

Solution:
Modified both approval-iot and approval-iot-after-monitor tasks to run in TWO scenarios:

1. When IoT FVT tests run (launchfvt_iot=true):
   - Runs after launchfvt-iot completes
   - Original behavior preserved

2. When Monitor needs IoT but IoT FVT is skipped (launchfvt_monitor=true AND launchfvt_iot!=true):
   - Runs after waitfor-iot/waitfor-iot-after-monitor completes
   - NEW behavior to handle missing parameter case
   - Ensures IoT installation reaches 'pending' status before approval

Changes:
- Added waitfor-iot to approval-iot runAfter dependencies
- Added waitfor-iot-after-monitor to approval-iot-after-monitor runAfter dependencies
- Modified when conditions to check both launchfvt_iot and launchfvt_monitor
- Now handles all parameter combinations correctly for both install orders

Behavior Matrix:
┌─────────────────┬─────────────────┬──────────────────┬───────────────────────────────┐
│ LAUNCHFVT_IOT   │ LAUNCHFVT_MON   │ Install Order    │ Result                        │
├─────────────────┼─────────────────┼──────────────────┼───────────────────────────────┤
│ true            │ true            │ before-iot       │ Both wait, run in parallel    │
│ true            │ true            │ after-iot        │ IoT first, then Monitor       │
│ false/unset     │ true            │ before-iot       │ Monitor waits for IoT install │
│ false/unset     │ true            │ after-iot        │ Monitor waits for IoT install │
│ true            │ false           │ any              │ IoT FVT runs only             │
│ false/unset     │ false           │ any              │ No FVT tests run              │
└─────────────────┴─────────────────┴──────────────────┴───────────────────────────────┘

Impact:
- Fixes Monitor FVT failures in environments without LAUNCHFVT_IOT set
- Maintains backward compatibility with existing configurations
- Supports flexible FVT test combinations for both MAS 9.2.0+ (before-iot) and legacy (after-iot)
- Applies fix to both approval-iot tasks (after-iot and before-iot scenarios)

Tested: fvtcore (both params set), fvtcpd (monitor only - was failing)
Affects: All MAS versions with Monitor FVT enabled
Related: fvt-dev repository mas.env update for fvtcpd branch (adds LAUNCHFVT_IOT=true)
…T FVT tests

ROOT CAUSE:
Monitor FVT depends on IoT installation being complete (needs IoT routes), but the
pipeline logic incorrectly tied IoT installation wait tasks to launchfvt_iot parameter.
This caused Monitor FVT to launch before IoT installation completed when IoT FVT tests
were not enabled.

FAILING SCENARIO (launchfvt_iot NOT set, launchfvt_monitor=true):
1. waitfor-iot-after-monitor SKIPPED (checked launchfvt_iot instead of launchfvt_monitor)
2. approval-iot-after-monitor SKIPPED (depends on waitfor-iot-after-monitor)
3. launchfvt-monitor-before-iot launched WITHOUT waiting for IoT installation
4. Monitor FVT tests FAILED with 404 errors accessing IoT routes

Example failure (montest75 cluster - 2026-04-08):
- Monitor FVT launched before IoT installation complete
- Tests failed: routes.route.openshift.io "montest75-iot-masdev-messaging" not found
- Error: KeyError: 'MONITOR_BASEURL' (secondary failures due to missing IoT routes)

WORKING SCENARIO (Both launchfvt_iot and launchfvt_monitor set):
- waitfor-iot-after-monitor runs (launchfvt_iot=true)
- approval-iot-after-monitor runs
- Monitor FVT waits properly for both installations
- All tests pass

THE REAL REQUIREMENT:
Monitor FVT should wait for BOTH Monitor AND IoT installations to complete,
REGARDLESS of whether IoT FVT tests run. The dependency is on installation
completion, NOT on FVT test execution.

SOLUTION:
Changed when conditions for waitfor-iot, waitfor-iot-after-monitor, approval-iot,
and approval-iot-after-monitor tasks to check BOTH launchfvt_iot AND launchfvt_monitor.
These tasks now run when EITHER parameter is true, ensuring:
- IoT installation waits happen when Monitor FVT needs them
- Approval tasks complete after installation regardless of FVT test execution
- Monitor FVT always waits for IoT installation to complete
@jvaneel-dwivedi jvaneel-dwivedi self-assigned this Apr 10, 2026
@jvaneel-dwivedi jvaneel-dwivedi requested a review from a team as a code owner April 10, 2026 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant