'az backup protection undelete' command fails for AzureStorage type backups

[AyoubIssaad]

Describe the bug

The az backup protection undelete command is not working as expected for AzureStorage type backups (specifically Azure File Shares). The command executes without error but does not actually perform the undelete operation. The same command works correctly for AzureIaasVM type backups.

Related command

az backup protection undelete --item-name <item_name> --resource-group <resource_group> --vault-name <vault_name> --container-name <container_name> --backup-management-type AzureStorage

az backup protection undelete --ids <backup_id> --backup-management-type AzureStorage

Errors

No error message is displayed. The command appears to execute successfully, but the backup item remains in a soft-deleted state.

Issue script & Debug output

az backup protection undelete --item-name "AzureFileShare;df2f837a1cc800d52b230a4fbe0e14313830811814d82562c99d317500d341ca" --resource-group "z-ago-sup-pp55-en1-01" --vault-name "zagosuppp55en1rsv33" --container-name "StorageContainer;storage;z-ago-sup-pp55-en1-01;vzagosup55en1sto33" --backup-management-type AzureStorage --debug

-- OUTPUT

cli.knack.cli: Command arguments: ['backup', 'protection', 'undelete', '--item-name', 'AzureFileShare;df2f837a1cc800d52b230a4fbe0e14313830811814d82562c99d317500d341ca', '--resource-group', 'z-ago-sup-pp55-en1-01', '--vault-name', 'zagosuppp55en1rsv33', '--container-name', 'StorageContainer;storage;z-ago-sup-pp55-en1-01;vzagosup55en1sto33', '--backup-management-type', 'AzureStorage', '--debug']
cli.knack.cli: init debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7f1fcb52e0d0>, <function OutputProducer.on_global_arguments at 0x7f1fcb448ca0>, <function CLIQuery.on_global_arguments at 0x7f1fcb3de280>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'backup': ['azure.cli.command_modules.backup']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name Load Time Groups Commands
cli.azure.cli.core: backup 0.007 16 60
cli.azure.cli.core: Total (1) 0.007 16 60
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name Load Time Groups Commands Directory
cli.azure.cli.core: ai-examples 0.199 1 1 /usr/lib/python3.9/site-packages/azure-cli-extensions/ai-examples
cli.azure.cli.core: Total (1) 0.199 1 1
cli.azure.cli.core: Loaded 17 groups, 61 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command : backup protection undelete
cli.azure.cli.core: Command table: backup protection undelete
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7f1fca834820>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/home/alexo/.azure/commands/2024-09-06.17-03-09.backup_protection_undelete.2705.log'.
az_command_data_logger: command args: backup protection undelete --item-name {} --resource-group {} --vault-name {} --container-name {} --backup-management-type {} --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0x7f1fca85e550>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument..add_ids_arguments at 0x7f1fca805550>, <function register_cache_arguments..add_cache_arguments at 0x7f1fca824430>, <function register_upcoming_breaking_change_info..update_breaking_change_info at 0x7f1fca8244c0>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7f1fcb448d30>, <function CLIQuery.handle_query_parameter at 0x7f1fcb3de310>, <function register_ids_argument..parse_ids_arguments at 0x7f1fca8243a0>]
cli.azure.cli.core.commands.client_factory: Getting management service client client_type=RecoveryServicesBackupClient
urllib3.connectionpool: Starting new HTTP connection (1): localhost:50342
urllib3.connectionpool: http://localhost:50342 "POST /oauth2/token HTTP/1.1" 200 4789
msrestazure.azure_active_directory: MSI: Retrieving a token from http://localhost:50342/oauth2/token, with payload {'resource': 'https://management.core.windows.net/'}
cli.azure.cli.core.commands.client_factory: Getting management service client client_type=RecoveryServicesBackupClient
urllib3.connectionpool: Starting new HTTP connection (1): localhost:50342
urllib3.connectionpool: http://localhost:50342 "POST /oauth2/token HTTP/1.1" 200 4789
msrestazure.azure_active_directory: MSI: Retrieving a token from http://localhost:50342/oauth2/token, with payload {'resource': 'https://management.core.windows.net/'}
cli.azure.cli.core.commands.client_factory: Getting management service client client_type=RecoveryServicesBackupClient
urllib3.connectionpool: Starting new HTTP connection (1): localhost:50342
urllib3.connectionpool: http://localhost:50342 "POST /oauth2/token HTTP/1.1" 200 4789
msrestazure.azure_active_directory: MSI: Retrieving a token from http://localhost:50342/oauth2/token, with payload {'resource': 'https://management.core.windows.net/'}
cli.azure.cli.core.auth.adal_authentication: MSIAuthenticationWrapper.get_token: scopes=('https://management.core.windows.net//.default',), kwargs={}
urllib3.connectionpool: Starting new HTTP connection (1): localhost:50342
urllib3.connectionpool: http://localhost:50342 "POST /oauth2/token HTTP/1.1" 200 4789
msrestazure.azure_active_directory: MSI: Retrieving a token from http://localhost:50342/oauth2/token, with payload {'resource': 'https://management.core.windows.net/'}
cli.azure.cli.core.auth.adal_authentication: Normalize expires_on: '1725642808' -> 1725642808
cli.azure.cli.core.sdk.policies: Request URL: 'https://management.azure.com/subscriptions/83434f9d-4080-4c3a-87f1-6c6018d3ced2/resourceGroups/z-ago-sup-pp55-en1-01/providers/Microsoft.RecoveryServices/vaults/zagosuppp55en1rsv33/backupFabrics/Azure/protectionContainers/StorageContainer%3Bstorage%3Bz-ago-sup-pp55-en1-01%3Bvzagosup55en1sto33/protectedItems/AzureFileShare%3Bdf2f837a1cc800d52b230a4fbe0e14313830811814d82562c99d317500d341ca?api-version=2024-04-01'
cli.azure.cli.core.sdk.policies: Request method: 'GET'
cli.azure.cli.core.sdk.policies: Request headers:
cli.azure.cli.core.sdk.policies: 'Accept': 'application/json'
cli.azure.cli.core.sdk.policies: 'x-ms-client-request-id': 'e44f34c0-6c71-12ef-adfd-001267d799cb4'
cli.azure.cli.core.sdk.policies: 'CommandName': 'backup protection undelete'
cli.azure.cli.core.sdk.policies: 'ParameterSetName': '--item-name --resource-group --vault-name --container-name --backup-management-type --debug'
cli.azure.cli.core.sdk.policies: 'User-Agent': 'AZURECLI/2.64.0 (RPM) azsdk-python-core/1.28.0 Python/3.9.19 (Linux-5.10.102.2-microsoft-standard-x86_64-with-glibc2.35) cloud-shell/1.0'
cli.azure.cli.core.sdk.policies: 'Authorization': '*****'
cli.azure.cli.core.sdk.policies: Request body:
cli.azure.cli.core.sdk.policies: This request has no body
urllib3.connectionpool: Starting new HTTPS connection (1): management.azure.com:443
urllib3.connectionpool: https://management.azure.com:443 "GET /subscriptions/83434f9d-4080-4c3a-87f1-6c6018d3ced2/resourceGroups/z-ago-sup-pp55-en1-01/providers/Microsoft.RecoveryServices/vaults/zagosuppp55en1rsv33/backupFabrics/Azure/protectionContainers/StorageContainer%3Bstorage%3Bz-ago-sup-pp55-en1-01%3Bvzagosup55en1sto33/protectedItems/AzureFileShare%3Bdf2f837a1cc800d52b230a4fbe0e14313830811814d82562c99d317500d341ca?api-version=2024-04-01 HTTP/1.1" 200 1667
cli.azure.cli.core.sdk.policies: Response status: 200
cli.azure.cli.core.sdk.policies: Response headers:
cli.azure.cli.core.sdk.policies: 'Cache-Control': 'no-cache'
cli.azure.cli.core.sdk.policies: 'Pragma': 'no-cache'
cli.azure.cli.core.sdk.policies: 'Content-Length': '1667'
cli.azure.cli.core.sdk.policies: 'Content-Type': 'application/json; charset=utf-8'
cli.azure.cli.core.sdk.policies: 'Expires': '-1'
cli.azure.cli.core.sdk.policies: 'X-Content-Type-Options': 'nosniff'
cli.azure.cli.core.sdk.policies: 'x-ms-request-id': 'f6a73a8a-4254-4bbe-b464-9b36519029ert'
cli.azure.cli.core.sdk.policies: 'x-ms-client-request-id': 'e44f34c0-6c71-12ef-adfd-001267d799cb4'
cli.azure.cli.core.sdk.policies: 'x-ms-ratelimit-remaining-subscription-resource-requests': '149'
cli.azure.cli.core.sdk.policies: 'x-ms-correlation-request-id': 'f6a73a8a-4254-4bbe-b464-9b36519029ert'
cli.azure.cli.core.sdk.policies: 'x-ms-routing-request-id': 'WESTEUROPE:20240906T170309Z:f6a73a8a-4254-4bbe-b464-9b36519029ert'
cli.azure.cli.core.sdk.policies: 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
cli.azure.cli.core.sdk.policies: 'X-Cache': 'CONFIG_NOCACHE'
cli.azure.cli.core.sdk.policies: 'X-MSEdge-Ref': 'Ref A: 9C6B5BF8665C4BA1917345AEA74D665C Ref B: AMS231022012047 Ref C: 2024-09-06T17:03:09Z'
cli.azure.cli.core.sdk.policies: 'Date': 'Fri, 06 Sep 2024 17:03:09 GMT'
cli.azure.cli.core.sdk.policies: Response content:
cli.azure.cli.core.sdk.policies: {"id":"/subscriptions/83434f9d-4080-4c3a-87f1-6c6018d3ced2/resourceGroups/z-ago-sup-pp55-en1-01/providers/Microsoft.RecoveryServices/vaults/zagosuppp55en1rsv33/backupFabrics/Azure/protectionContainers/StorageContainer;storage;z-ago-sup-pp55-en1-01;vzagosup55en1sto33/protectedItems/AzureFileShare;df2f837a1cc800d52b230a4fbe0e14313830811814d82562c99d317500d341ca","name":"AzureFileShare;df2f837a1cc800d52b230a4fbe0e14313830811814d82562c99d317500d341ca","type":"Microsoft.RecoveryServices/vaults/backupFabrics/protectionContainers/protectedItems","properties":{"friendlyName":"actionsredhat855501","protectionStatus":"Healthy","protectionState":"ProtectionStopped","lastBackupStatus":"IRPending","extendedInfo":{"recoveryPointCount":0,"policyState":"Consistent","resourceState":"Active","resourceStateSyncTime":"2024-08-07T11:15:43.8663789Z"},"protectedItemType":"AzureFileShareProtectedItem","backupManagementType":"AzureStorage","workloadType":"AzureFileShare","containerName":"StorageContainer;storage;z-ago-sup-pp55-en1-01;vzagosup55en1sto33","sourceResourceId":"/subscriptions/83434f9d-4080-4c3a-87f1-6c6018d3ced2/resourceGroups/z-ago-sup-pp55-en1-01/providers/Microsoft.storage/storageAccounts/vzagosup55en1sto33","vaultId":"https://management.azure.com/subscriptions/83434f9d-4080-4c3a-87f1-6c6018d3ced2/resourceGroups/z-ago-sup-pp55-en1-01/providers/Microsoft.RecoveryServices/vaults/zagosuppp55en1rsv33","deferredDeleteTimeInUTC":"2024-08-07T17:19:17.4768461Z","isScheduledForDeferredDelete":true,"deferredDeleteTimeRemaining":"5.00:16:07.6080615","isArchiveEnabled":false,"softDeleteRetentionPeriod":35}}
cli.knack.cli: Event: CommandInvoker.OnTransformResult [<function _resource_group_transform at 0x7f1fca800820>, <function _x509_from_base64_to_hex_transform at 0x7f1fca8008b0>]
cli.knack.cli: Event: CommandInvoker.OnFilterResult []
cli.knack.cli: Event: Cli.SuccessfulExecute []
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7f1fca834a60>]
az_command_data_logger: exit code: 0
cli.main: Command ran in 1.201 seconds (init: 0.187, invoke: 1.013)
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 3968 in cache file under /home/alexo/.azure/telemetry/20240906170309995
telemetry.main: Begin creating telemetry upload process.
telemetry.process: Creating upload process: "/usr/bin/python3.9 /usr/lib/az/lib/python3.9/site-packages/azure/cli/telemetry/init.py /home/alexo/.azure /home/alexo/.azure/telemetry/20240906170309995"
telemetry.process: Return from creating process 2716
telemetry.main: Finish creating telemetry upload process.

Expected behavior

The command should initiate the undelete operation for the specified AzureStorage backup item, removing it from the soft-deleted state and making it available for further backup operations.

Environment Summary

azure-cli 2.64.0

core 2.64.0
telemetry 1.1.0

Extensions:
account 0.2.5
ai-examples 0.2.5
ml 2.29.0
ssh 2.0.5

Dependencies:
msal 1.30.0
azure-mgmt-resource 23.1.1

Python location '/usr/bin/python3.9'
Extensions directory '/home/issaad/.azure/cliextensions'
Extensions system directory '/usr/lib/python3.9/site-packages/azure-cli-extensions'

Python (Linux) 3.9.19 (main, Aug 23 2024, 00:07:48)
[GCC 11.2.0]

Legal docs and information: aka.ms/AzureCliLegal
Your CLI is up-to-date.

Additional context

1. The same command structure works correctly for AzureIaasVM type backups, suggesting that the issue is specific to AzureStorage type.
2. The Azure portal successfully performs the undelete operation for these items, indicating that the backend API supports this functionality.
3. Debug output shows that the CLI is only making a GET request to retrieve item details, instead of a POST or PUT request to initiate the undelete operation.
4. This issue has been reproduced in multiple environments:
   • Azure Cloud Shell
   • Ubuntu server

[yonzhan]

Thank you for opening this issue, we will look into it.

[microsoft-github-policy-service]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @pvrk, @adityabalaji-msft.

[jaredbrogan]

Seeing this behavior as well. It's as if it just never tries to make the request, but only performs a GET request to get a list of protected items.

[AyoubIssaad]

Seeing this behavior as well. It's as if it just never tries to make the request, but only performs a GET request to get a list of protected items.

In my case I was automating the process of removing orphaned backup items, as a workaround to the az cli issue mentioned here, I used the REST API and it works fine, maybe this can be helpful to you as well.

[jaredbrogan]

Seeing this behavior as well. It's as if it just never tries to make the request, but only performs a GET request to get a list of protected items.

In my case I was automating the process of removing orphaned backup items, as a workaround to the az cli issue mentioned here, I used the REST API and it works fine, maybe this can be helpful to you as well.

Would you mind sharing how you're making the calls using their REST API? I believe I have a working solution for undeleting the protected item (the file share backup), but undeleting it's protection container has proven to be a challenge. I really wish Microsoft's docs for File Share backups was as thorough as the VM backup portion, but can't say I'm too surprised either.

[AyoubIssaad]

Would you mind sharing how you're making the calls using their REST API? I believe I have a working solution for undeleting the protected item (the file share backup), but undeleting it's protection container has proven to be a challenge. I really wish Microsoft's docs for File Share backups was as thorough as the VM backup portion, but can't say I'm too surprised either.

Sorry for my late response, here is how I'm making the calls for the REST API, (the documentation wasn't clear for me, so I used the az cli to undelete an AzureIaasVM backup item, added the --debug flag and copied what I saw there and it worked for me

urlencode() {
            local string="${1}"
            local strlen=${#string}
            local encoded=""
            local pos c o

            for (( pos=0 ; pos<strlen ; pos++ )); do
                c=${string:$pos:1}
                case "$c" in
                    [-_.~a-zA-Z0-9] ) o="${c}" ;;
                    * )               printf -v o '%%%02x' "'$c"
                esac
                encoded+="${o}"
            done
            echo "${encoded}"
          }
          
undelete_azurestorage_api() {
              local resource_group="$1"
              local vault_name="$2"
              local container_name="$3"
              local backup_name="$4"
              local resource_id="$5"

              # URL encode the parameters for the URL
              local encoded_container_name=$(urlencode "$container_name")
              local encoded_backup_name=$(urlencode "$backup_name")

              # Get access token access_token

              
              # Get the subscription ID
              local subscription_id=$(az account show --query id -o tsv)
              
              # Construct the API URL with encoded parameters and explicit subscription ID
              local api_url="https://management.azure.com/subscriptions/$subscription_id/resourceGroups/$resource_group/providers/Microsoft.RecoveryServices/vaults/$vault_name/backupFabrics/Azure/protectionContainers/$encoded_container_name/protectedItems/$encoded_backup_name?api-version=2024-04-01"
              
              # Construct the request body using jq for proper JSON formatting and escaping
              local request_body=$(jq -n \
                                    --arg type "AzureStorage" \
                                    --arg id "$resource_id" \
                                    '{properties: {protectedItemType: $type, sourceResourceId: $id, policyId: "", isRehydrate: true, protectionState: "ProtectionStopped"}}')

              # Make the REST API call
              local response=$(curl -s -X PUT -H "Authorization: Bearer $access_token" -H "Content-Type: application/json" -d "$request_body" "$api_url")

              # Return the response
              echo "$response"
          }
          ```