[AI Generated] test(custom_script): mark boot validation test maturity as preview#4606
Open
LiliDeng wants to merge 2 commits into
Open
[AI Generated] test(custom_script): mark boot validation test maturity as preview#4606LiliDeng wants to merge 2 commits into
LiliDeng wants to merge 2 commits into
Conversation
LiliDeng
requested review from
johnsongeorge-w and
mayankdaruka-msft
as code owners
July 17, 2026 06:24
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the Custom Script VM extension boot validation test’s metadata to classify it as experimental, so it won’t be included in default (stable-only) test selection unless explicitly requested.
Changes:
- Set
maturity="experimental"onmicrosoft_azure_extensions_customscript_boot_validation_testvia@TestCaseMetadata.
Collaborator
Author
AI Test Case SelectionSelected 13 test case(s): microsoft_azure_extensions_customscript_boot_validation_test,verify_public_script_run,verify_second_public_script_run,verify_script_in_both_settings_failed,verify_public_script_protected_settings_run,verify_public_script_without_command_run,verify_base64_script_with_command_run,verify_public_script_with_base64_script_run,verify_public_script_with_gzip_base64_script_run,verify_private_script_without_sas_run_failed,verify_private_script_with_storage_credentials_run,verify_private_sas_script_run,verify_public_python_script_run Marketplace image: Result: Failed |
Collaborator
|
@copilot update this pr to change the maturity to preview |
Contributor
Collaborator
Author
AI Test Case SelectionSelected 13 test case(s): microsoft_azure_extensions_customscript_boot_validation_test,verify_public_script_run,verify_second_public_script_run,verify_script_in_both_settings_failed,verify_public_script_protected_settings_run,verify_public_script_without_command_run,verify_base64_script_with_command_run,verify_public_script_with_base64_script_run,verify_public_script_with_gzip_base64_script_run,verify_private_script_without_sas_run_failed,verify_private_script_with_storage_credentials_run,verify_private_sas_script_run,verify_public_python_script_run Marketplace image: Result: Failed |
LiliDeng
changed the title
tarunsingh-msft
added a commit
to tarunsingh-msft/lisa
that referenced
this pull request
Jul 20, 2026
… maturity Set maturity="preview" on the RCv1/RCv2 boot validation test cases so they are excluded from default (stable-only) runs until fully validated, consistent with the CustomScript boot validation test (microsoft#4606) and the test maturity model (microsoft#4586). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3c8bcd2c-8f65-4576-a13b-870fb7163f0f
tarunsingh-msft
added a commit
to tarunsingh-msft/lisa
that referenced
this pull request
Jul 20, 2026
…review Address review feedback on the RunCommand boot validation tests: - Coerce runbook variables to str before strip(), so a non-string YAML value (e.g. extension_version: 1.3 parsed as a float) does not crash the helper. - Use the normalized Major.Minor value from normalize_type_handler_version() for type_handler_version, so a Major.Minor.Patch runbook value installs correctly instead of being passed through raw. - Set maturity="preview" on both boot validation cases so they are excluded from default stable-only runs until fully validated, consistent with the CustomScript boot validation test (microsoft#4606) and the test maturity model (microsoft#4586). Re-validated on real Azure VMs after these changes: both cases PASSED (v1 centralus, v2 eastus2). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3c8bcd2c-8f65-4576-a13b-870fb7163f0f
LiliDeng
pushed a commit
that referenced
this pull request
Jul 20, 2026
…4599) * test(vm_extensions): add RunCommand v1/v2 boot validation test cases Adds minimal, self-contained boot validation test cases for the Run Command v1 and v2 VM extensions, mirroring the CustomScript boot validation test (PR #4580). - RunCommandV1Tests.microsoft_azure_extensions_runcommandv1_boot_validation_test - RunCommandV2Tests.microsoft_azure_extensions_runcommandv2_boot_validation_test Each installs the extension with a single inline command and no file/script uris (no storage account or public blob access needed), asserts provisioning_state == Succeeded, then removes the extension. Publisher/type/version are read from runbook variables; the test is skipped when extension_version is unset. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3c8bcd2c-8f65-4576-a13b-870fb7163f0f * style(vm_extensions): fix Black formatting in run_commandv1 Collapse the extension_type assignment onto a single line to satisfy the Black formatter (BLK100), matching the flake8 CI check. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3c8bcd2c-8f65-4576-a13b-870fb7163f0f * docs(vm_extensions): clarify extension_version is required in boot validation tests Reword the RCv1/RCv2 boot validation docstrings so the version handling matches the implementation: publisher and type default to the Run Command extension, while extension_version must be supplied via the runbook (the test is skipped otherwise). Addresses PR review feedback. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3c8bcd2c-8f65-4576-a13b-870fb7163f0f * test(vm_extensions): rename boot validation tests to match RunCommand publisher/type The certification functional-validation feature parses the test method name using the pattern <Publisher>_<Type>_boot_validation_test. RunCommand is published under Microsoft.CPlat.Core (types RunCommandLinux / RunCommandHandlerLinux), so the method names now reflect the actual publisher/type instead of the CustomScript publisher prefix: - microsoft_azure_extensions_runcommandv1_boot_validation_test -> microsoft_cplat_core_runcommandlinux_boot_validation_test - microsoft_azure_extensions_runcommandv2_boot_validation_test -> microsoft_cplat_core_runcommandhandlerlinux_boot_validation_test Publisher/type are unchanged in code (Microsoft.CPlat.Core, since 2023 PRs #2809/#2778) and match the deployed extension instance name. Re-validated on real Azure VMs: both cases PASSED. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3c8bcd2c-8f65-4576-a13b-870fb7163f0f * refactor(vm_extensions): extract shared boot validation helper Consolidate the duplicated RunCommand v1/v2 boot validation flow into a single reusable helper run_extension_boot_validation() in common.py, as suggested in review. The helper reads publisher/type from runbook variables (with caller-provided defaults), requires extension_version (skips otherwise), installs the extension, asserts provisioning succeeds, and optionally deletes it (cleanup=False for CRP-managed RunCommand v2). A protected_settings parameter is included so future extensions can reuse it. Both cases re-validated on real Azure VMs after the refactor: PASSED (v1 eastus2, v2 eastus). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3c8bcd2c-8f65-4576-a13b-870fb7163f0f * test(vm_extensions): skip boot validation on malformed extension_version Add a guard in run_extension_boot_validation that validates extension_version is a 'Major.Minor' or 'Major.Minor.Patch' value (reusing AzureExtension.normalize_type_handler_version) and raises SkippedException instead of failing with a confusing Azure error when it is malformed. Addresses review feedback on version validation. Validated on Azure: valid versions (1.0/1.3) PASSED; malformed versions (1.0.beta / abc) SKIPPED with a clear message. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3c8bcd2c-8f65-4576-a13b-870fb7163f0f * docs(vm_extensions): capitalize URIs in boot validation docstrings Address review nit: 'uris' -> 'URIs' in the RCv1/RCv2 boot validation test descriptions. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3c8bcd2c-8f65-4576-a13b-870fb7163f0f * test(vm_extensions): harden version handling and mark boot tests as preview Address review feedback on the RunCommand boot validation tests: - Coerce runbook variables to str before strip(), so a non-string YAML value (e.g. extension_version: 1.3 parsed as a float) does not crash the helper. - Use the normalized Major.Minor value from normalize_type_handler_version() for type_handler_version, so a Major.Minor.Patch runbook value installs correctly instead of being passed through raw. - Set maturity="preview" on both boot validation cases so they are excluded from default stable-only runs until fully validated, consistent with the CustomScript boot validation test (#4606) and the test maturity model (#4586). Re-validated on real Azure VMs after these changes: both cases PASSED (v1 centralus, v2 eastus2). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3c8bcd2c-8f65-4576-a13b-870fb7163f0f * refactor(vm_extensions): drop unused protected_settings from boot helper Address review feedback (LiliDeng): the boot validation callers never pass protected_settings, so remove the unused parameter from run_extension_boot_validation(). It can be reintroduced if a future extension actually needs protected settings. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3c8bcd2c-8f65-4576-a13b-870fb7163f0f --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Set the
maturityofmicrosoft_azure_extensions_customscript_boot_validation_testtopreviewin its@TestCaseMetadata. Same-repo branch so the internal Github PR Validation pipeline triggers.