Skip to content

Commit bbfdd97

Browse files
committed
fix(tests): overhaul staging test infrastructure and update fixtures
Rewrite test infrastructure for staging environment safety. Update OpenAPI-generated API client with new models and endpoints. Regenerate gooddata-sdk test cassettes against staging. Remove obsolete cassettes from gooddata-pandas, gooddata-fdw, and gooddata-sdk packages. - Add staging-tests.yaml workflow triggered by label, PR comment (/test-staging), or manual dispatch - Concurrency blocking ensures only one staging test runs at a time - Unify token passing: all staging make targets now use TOKEN= argument - Replace GD_STAGING_TOKEN env var with --gd-test-token pytest CLI argument - Remove TOKEN from tox pass_env (no longer needed as env var) - Document staging test workflow in CONTRIBUTING.md JIRA: DX-431 risk: low
1 parent 9659a3e commit bbfdd97

154 files changed

Lines changed: 7020 additions & 3915 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
name: Staging tests
2+
3+
on:
4+
pull_request:
5+
types: [labeled]
6+
branches:
7+
- master
8+
- 'rel/**'
9+
issue_comment:
10+
types: [created]
11+
workflow_dispatch:
12+
inputs:
13+
test_envs:
14+
description: 'Tox test environments to run (e.g. py312)'
15+
required: false
16+
default: 'py312'
17+
test_filter:
18+
description: 'Pytest filter expression (-k flag)'
19+
required: false
20+
default: ''
21+
22+
concurrency:
23+
group: staging-tests
24+
cancel-in-progress: false
25+
26+
jobs:
27+
staging-tests:
28+
name: Staging tests
29+
if: >-
30+
github.event_name == 'workflow_dispatch' ||
31+
(github.event_name == 'pull_request' && github.event.label.name == 'test-staging') ||
32+
(github.event_name == 'issue_comment' && github.event.issue.pull_request && contains(github.event.comment.body, '/test-staging'))
33+
runs-on:
34+
group: infra1-runners-arc
35+
labels: runners-small
36+
steps:
37+
- name: Get PR head SHA (comment trigger)
38+
if: github.event_name == 'issue_comment'
39+
id: pr
40+
run: |
41+
PR_DATA=$(gh api repos/${{ github.repository }}/pulls/${{ github.event.issue.number }})
42+
echo "sha=$(echo "$PR_DATA" | jq -r .head.sha)" >> "$GITHUB_OUTPUT"
43+
echo "ref=$(echo "$PR_DATA" | jq -r .head.ref)" >> "$GITHUB_OUTPUT"
44+
env:
45+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
46+
47+
- name: Checkout
48+
uses: actions/checkout@v4
49+
with:
50+
ref: ${{ steps.pr.outputs.sha || github.event.pull_request.head.sha || github.sha }}
51+
52+
- name: Set up Python
53+
uses: astral-sh/setup-uv@v6
54+
with:
55+
python-version: '3.12'
56+
57+
- name: Install dependencies
58+
run: uv sync --all-groups --locked
59+
60+
- name: Clean staging environment
61+
run: make clean-staging TOKEN=${{ secrets.PYTHON_SDK_STG_API_KEY }}
62+
63+
- name: Load staging environment
64+
run: make load-staging TOKEN=${{ secrets.PYTHON_SDK_STG_API_KEY }}
65+
66+
- name: Run staging tests
67+
run: |
68+
make test-staging \
69+
TOKEN=${{ secrets.PYTHON_SDK_STG_API_KEY }} \
70+
TEST_ENVS=${{ github.event.inputs.test_envs || 'py312' }} \
71+
ADD_ARGS="${{ github.event.inputs.test_filter && format('-k {0}', github.event.inputs.test_filter) || '' }}"

CONTRIBUTING.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -292,6 +292,42 @@ docker compose --profile fdw up -d
292292
293293
This starts a PostgreSQL instance with the gooddata-fdw extension on port 2543.
294294
295+
## Run staging tests
296+
297+
Staging tests run against a shared GoodData staging environment instead of a local docker-compose stack.
298+
They are useful for validating changes against a real deployment.
299+
300+
### Triggers
301+
302+
The staging tests workflow (`.github/workflows/staging-tests.yaml`) can be triggered in three ways:
303+
304+
1. **Label** — Add the `test-staging` label to a PR targeting `master` or `rel/**`.
305+
2. **PR comment** — Post `/test-staging` as a comment on a PR.
306+
3. **Manual dispatch** — Trigger from the Actions tab with optional `test_envs` and `test_filter` inputs.
307+
308+
Only one staging test run executes at a time (concurrency group `staging-tests`, non-cancelling).
309+
310+
### Running staging tests locally
311+
312+
You need a staging API token (`TOKEN`). The workflow uses the `PYTHON_SDK_STG_API_KEY` secret; locally you
313+
pass it via the `TOKEN=` make argument:
314+
315+
```bash
316+
# 1. Clean the staging workspace (removes previous test data)
317+
make clean-staging TOKEN=<your-staging-token>
318+
319+
# 2. Load the demo layout into staging
320+
make load-staging TOKEN=<your-staging-token>
321+
322+
# 3. Run the tests
323+
make test-staging TOKEN=<your-staging-token>
324+
325+
# Optionally limit python version and test filter:
326+
make test-staging TOKEN=<your-staging-token> TEST_ENVS=py312 ADD_ARGS="-k test_catalog"
327+
```
328+
329+
The token is passed as a CLI argument (`--gd-test-token`) to pytest, **not** as an environment variable.
330+
295331
## Run continuous integration tests
296332
Tests in pull request (PR) are executed using docker. The following is done to make test environment as close
297333
to reproducible as possible:

Makefile

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,18 @@ test:
8282
for project in $(NO_CLIENT_GD_PROJECTS_DIRS); do $(MAKE) -C packages/$${project} test || RESULT=$$?; done; \
8383
exit $$RESULT
8484

85+
.PHONY: test-staging
86+
test-staging:
87+
$(MAKE) -C packages/gooddata-sdk test-staging TOKEN=$(TOKEN)
88+
89+
.PHONY: clean-staging
90+
clean-staging:
91+
cd packages/tests-support && STAGING=1 TOKEN="$(TOKEN)" python clean_staging.py
92+
93+
.PHONY: load-staging
94+
load-staging:
95+
cd packages/tests-support && STAGING=1 TOKEN="$(TOKEN)" python upload_demo_layout.py
96+
8597
.PHONY: release
8698
release:
8799
if [ -z "$(VERSION)" ]; then echo "Usage: 'make release VERSION=X.Y.Z'"; false; else \

gooddata-api-client/.openapi-generator/FILES

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -384,6 +384,7 @@ docs/GeographicDataApi.md
384384
docs/GetAiLakeOperation200Response.md
385385
docs/GetImageExport202ResponseInner.md
386386
docs/GetQualityIssuesResponse.md
387+
docs/GetServiceStatusResponse.md
387388
docs/GrainIdentifier.md
388389
docs/GrantedPermission.md
389390
docs/GranularitiesFormatting.md
@@ -1014,6 +1015,9 @@ docs/ListDatabaseInstancesResponse.md
10141015
docs/ListKnowledgeDocumentsResponseDto.md
10151016
docs/ListLinks.md
10161017
docs/ListLinksAllOf.md
1018+
docs/ListLlmProviderModelsRequest.md
1019+
docs/ListLlmProviderModelsRequestProviderConfig.md
1020+
docs/ListLlmProviderModelsResponse.md
10171021
docs/ListServicesResponse.md
10181022
docs/LlmModel.md
10191023
docs/LlmProviderAuth.md
@@ -1193,8 +1197,8 @@ docs/TabularExportRequest.md
11931197
docs/TestConnectionApi.md
11941198
docs/TestDefinitionRequest.md
11951199
docs/TestDestinationRequest.md
1200+
docs/TestLlmProviderByIdRequest.md
11961201
docs/TestLlmProviderDefinitionRequest.md
1197-
docs/TestLlmProviderDefinitionRequestProviderConfig.md
11981202
docs/TestLlmProviderResponse.md
11991203
docs/TestNotification.md
12001204
docs/TestNotificationAllOf.md
@@ -1207,6 +1211,8 @@ docs/TotalDimension.md
12071211
docs/TotalExecutionResultHeader.md
12081212
docs/TotalResultHeader.md
12091213
docs/TranslationsApi.md
1214+
docs/TrendingObjectItem.md
1215+
docs/TrendingObjectsResult.md
12101216
docs/TriggerAutomationRequest.md
12111217
docs/TriggerQualityIssuesCalculationResponse.md
12121218
docs/UpsertKnowledgeDocumentRequestDto.md
@@ -1708,6 +1714,7 @@ gooddata_api_client/model/geo_collection_identifier.py
17081714
gooddata_api_client/model/get_ai_lake_operation200_response.py
17091715
gooddata_api_client/model/get_image_export202_response_inner.py
17101716
gooddata_api_client/model/get_quality_issues_response.py
1717+
gooddata_api_client/model/get_service_status_response.py
17111718
gooddata_api_client/model/grain_identifier.py
17121719
gooddata_api_client/model/granted_permission.py
17131720
gooddata_api_client/model/granularities_formatting.py
@@ -2328,6 +2335,9 @@ gooddata_api_client/model/list_database_instances_response.py
23282335
gooddata_api_client/model/list_knowledge_documents_response_dto.py
23292336
gooddata_api_client/model/list_links.py
23302337
gooddata_api_client/model/list_links_all_of.py
2338+
gooddata_api_client/model/list_llm_provider_models_request.py
2339+
gooddata_api_client/model/list_llm_provider_models_request_provider_config.py
2340+
gooddata_api_client/model/list_llm_provider_models_response.py
23312341
gooddata_api_client/model/list_services_response.py
23322342
gooddata_api_client/model/llm_model.py
23332343
gooddata_api_client/model/llm_provider_auth.py
@@ -2487,8 +2497,8 @@ gooddata_api_client/model/table_warning.py
24872497
gooddata_api_client/model/tabular_export_request.py
24882498
gooddata_api_client/model/test_definition_request.py
24892499
gooddata_api_client/model/test_destination_request.py
2500+
gooddata_api_client/model/test_llm_provider_by_id_request.py
24902501
gooddata_api_client/model/test_llm_provider_definition_request.py
2491-
gooddata_api_client/model/test_llm_provider_definition_request_provider_config.py
24922502
gooddata_api_client/model/test_llm_provider_response.py
24932503
gooddata_api_client/model/test_notification.py
24942504
gooddata_api_client/model/test_notification_all_of.py
@@ -2500,6 +2510,8 @@ gooddata_api_client/model/total.py
25002510
gooddata_api_client/model/total_dimension.py
25012511
gooddata_api_client/model/total_execution_result_header.py
25022512
gooddata_api_client/model/total_result_header.py
2513+
gooddata_api_client/model/trending_object_item.py
2514+
gooddata_api_client/model/trending_objects_result.py
25032515
gooddata_api_client/model/trigger_automation_request.py
25042516
gooddata_api_client/model/trigger_quality_issues_calculation_response.py
25052517
gooddata_api_client/model/upsert_knowledge_document_request_dto.py

gooddata-api-client/README.md

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,7 @@ Class | Method | HTTP request | Description
106106
*AILakeApi* | [**deprovision_ai_lake_database_instance**](docs/AILakeApi.md#deprovision_ai_lake_database_instance) | **DELETE** /api/v1/ailake/database/instances/{instanceId} | (BETA) Delete an existing AILake Database instance
107107
*AILakeApi* | [**get_ai_lake_database_instance**](docs/AILakeApi.md#get_ai_lake_database_instance) | **GET** /api/v1/ailake/database/instances/{instanceId} | (BETA) Get the specified AILake Database instance
108108
*AILakeApi* | [**get_ai_lake_operation**](docs/AILakeApi.md#get_ai_lake_operation) | **GET** /api/v1/ailake/operations/{operationId} | (BETA) Get Long Running Operation details
109+
*AILakeApi* | [**get_ai_lake_service_status**](docs/AILakeApi.md#get_ai_lake_service_status) | **GET** /api/v1/ailake/services/{serviceId}/status | (BETA) Get AI Lake service status
109110
*AILakeApi* | [**list_ai_lake_database_instances**](docs/AILakeApi.md#list_ai_lake_database_instances) | **GET** /api/v1/ailake/database/instances | (BETA) List AI Lake Database instances
110111
*AILakeApi* | [**list_ai_lake_services**](docs/AILakeApi.md#list_ai_lake_services) | **GET** /api/v1/ailake/services | (BETA) List AI Lake services
111112
*AILakeApi* | [**provision_ai_lake_database_instance**](docs/AILakeApi.md#provision_ai_lake_database_instance) | **POST** /api/v1/ailake/database/instances | (BETA) Create a new AILake Database instance
@@ -416,11 +417,14 @@ Class | Method | HTTP request | Description
416417
*SmartFunctionsApi* | [**generate_title**](docs/SmartFunctionsApi.md#generate_title) | **POST** /api/v1/actions/workspaces/{workspaceId}/ai/analyticsCatalog/generateTitle | Generate Title for Analytics Object
417418
*SmartFunctionsApi* | [**get_quality_issues**](docs/SmartFunctionsApi.md#get_quality_issues) | **GET** /api/v1/actions/workspaces/{workspaceId}/ai/issues | Get Quality Issues
418419
*SmartFunctionsApi* | [**get_quality_issues_calculation_status**](docs/SmartFunctionsApi.md#get_quality_issues_calculation_status) | **GET** /api/v1/actions/workspaces/{workspaceId}/ai/issues/status/{processId} | Get Quality Issues Calculation Status
420+
*SmartFunctionsApi* | [**list_llm_provider_models**](docs/SmartFunctionsApi.md#list_llm_provider_models) | **POST** /api/v1/actions/ai/llmProvider/listModels | List LLM Provider Models
421+
*SmartFunctionsApi* | [**list_llm_provider_models_by_id**](docs/SmartFunctionsApi.md#list_llm_provider_models_by_id) | **POST** /api/v1/actions/ai/llmProvider/{llmProviderId}/listModels | List LLM Provider Models By Id
419422
*SmartFunctionsApi* | [**memory_created_by_users**](docs/SmartFunctionsApi.md#memory_created_by_users) | **GET** /api/v1/actions/workspaces/{workspaceId}/ai/memory/createdBy | Get AI Memory CreatedBy Users
420423
*SmartFunctionsApi* | [**resolve_llm_endpoints**](docs/SmartFunctionsApi.md#resolve_llm_endpoints) | **GET** /api/v1/actions/workspaces/{workspaceId}/ai/resolveLlmEndpoints | Get Active LLM Endpoints for this workspace
421424
*SmartFunctionsApi* | [**tags**](docs/SmartFunctionsApi.md#tags) | **GET** /api/v1/actions/workspaces/{workspaceId}/ai/analyticsCatalog/tags | Get Analytics Catalog Tags
422425
*SmartFunctionsApi* | [**test_llm_provider**](docs/SmartFunctionsApi.md#test_llm_provider) | **POST** /api/v1/actions/ai/llmProvider/test | Test LLM Provider
423426
*SmartFunctionsApi* | [**test_llm_provider_by_id**](docs/SmartFunctionsApi.md#test_llm_provider_by_id) | **POST** /api/v1/actions/ai/llmProvider/{llmProviderId}/test | Test LLM Provider By Id
427+
*SmartFunctionsApi* | [**trending_objects**](docs/SmartFunctionsApi.md#trending_objects) | **GET** /api/v1/actions/workspaces/{workspaceId}/ai/analyticsCatalog/trendingObjects | Get Trending Analytics Catalog Objects
424428
*SmartFunctionsApi* | [**trigger_quality_issues_calculation**](docs/SmartFunctionsApi.md#trigger_quality_issues_calculation) | **POST** /api/v1/actions/workspaces/{workspaceId}/ai/issues/triggerCheck | Trigger Quality Issues Calculation
425429
*SmartFunctionsApi* | [**validate_llm_endpoint**](docs/SmartFunctionsApi.md#validate_llm_endpoint) | **POST** /api/v1/actions/ai/llmEndpoint/test | Validate LLM Endpoint
426430
*SmartFunctionsApi* | [**validate_llm_endpoint_by_id**](docs/SmartFunctionsApi.md#validate_llm_endpoint_by_id) | **POST** /api/v1/actions/ai/llmEndpoint/{llmEndpointId}/test | Validate LLM Endpoint By Id
@@ -577,6 +581,8 @@ Class | Method | HTTP request | Description
577581
*ActionsApi* | [**key_driver_analysis**](docs/ActionsApi.md#key_driver_analysis) | **POST** /api/v1/actions/workspaces/{workspaceId}/execution/computeKeyDrivers | (EXPERIMENTAL) Compute key driver analysis
578582
*ActionsApi* | [**key_driver_analysis_result**](docs/ActionsApi.md#key_driver_analysis_result) | **GET** /api/v1/actions/workspaces/{workspaceId}/execution/computeKeyDrivers/result/{resultId} | (EXPERIMENTAL) Get key driver analysis result
579583
*ActionsApi* | [**list_documents**](docs/ActionsApi.md#list_documents) | **GET** /api/v1/actions/workspaces/{workspaceId}/ai/knowledge/documents |
584+
*ActionsApi* | [**list_llm_provider_models**](docs/ActionsApi.md#list_llm_provider_models) | **POST** /api/v1/actions/ai/llmProvider/listModels | List LLM Provider Models
585+
*ActionsApi* | [**list_llm_provider_models_by_id**](docs/ActionsApi.md#list_llm_provider_models_by_id) | **POST** /api/v1/actions/ai/llmProvider/{llmProviderId}/listModels | List LLM Provider Models By Id
580586
*ActionsApi* | [**list_workspace_user_groups**](docs/ActionsApi.md#list_workspace_user_groups) | **GET** /api/v1/actions/workspaces/{workspaceId}/userGroups |
581587
*ActionsApi* | [**list_workspace_users**](docs/ActionsApi.md#list_workspace_users) | **GET** /api/v1/actions/workspaces/{workspaceId}/users |
582588
*ActionsApi* | [**manage_dashboard_permissions**](docs/ActionsApi.md#manage_dashboard_permissions) | **POST** /api/v1/actions/workspaces/{workspaceId}/analyticalDashboards/{dashboardId}/managePermissions | Manage Permissions for a Dashboard
@@ -618,6 +624,7 @@ Class | Method | HTTP request | Description
618624
*ActionsApi* | [**test_llm_provider**](docs/ActionsApi.md#test_llm_provider) | **POST** /api/v1/actions/ai/llmProvider/test | Test LLM Provider
619625
*ActionsApi* | [**test_llm_provider_by_id**](docs/ActionsApi.md#test_llm_provider_by_id) | **POST** /api/v1/actions/ai/llmProvider/{llmProviderId}/test | Test LLM Provider By Id
620626
*ActionsApi* | [**test_notification_channel**](docs/ActionsApi.md#test_notification_channel) | **POST** /api/v1/actions/notificationChannels/test | Test notification channel.
627+
*ActionsApi* | [**trending_objects**](docs/ActionsApi.md#trending_objects) | **GET** /api/v1/actions/workspaces/{workspaceId}/ai/analyticsCatalog/trendingObjects | Get Trending Analytics Catalog Objects
621628
*ActionsApi* | [**trigger_automation**](docs/ActionsApi.md#trigger_automation) | **POST** /api/v1/actions/workspaces/{workspaceId}/automations/trigger | Trigger automation.
622629
*ActionsApi* | [**trigger_existing_automation**](docs/ActionsApi.md#trigger_existing_automation) | **POST** /api/v1/actions/workspaces/{workspaceId}/automations/{automationId}/trigger | Trigger existing automation.
623630
*ActionsApi* | [**trigger_quality_issues_calculation**](docs/ActionsApi.md#trigger_quality_issues_calculation) | **POST** /api/v1/actions/workspaces/{workspaceId}/ai/issues/triggerCheck | Trigger Quality Issues Calculation
@@ -1527,6 +1534,7 @@ Class | Method | HTTP request | Description
15271534
- [GetAiLakeOperation200Response](docs/GetAiLakeOperation200Response.md)
15281535
- [GetImageExport202ResponseInner](docs/GetImageExport202ResponseInner.md)
15291536
- [GetQualityIssuesResponse](docs/GetQualityIssuesResponse.md)
1537+
- [GetServiceStatusResponse](docs/GetServiceStatusResponse.md)
15301538
- [GrainIdentifier](docs/GrainIdentifier.md)
15311539
- [GrantedPermission](docs/GrantedPermission.md)
15321540
- [GranularitiesFormatting](docs/GranularitiesFormatting.md)
@@ -2147,6 +2155,9 @@ Class | Method | HTTP request | Description
21472155
- [ListKnowledgeDocumentsResponseDto](docs/ListKnowledgeDocumentsResponseDto.md)
21482156
- [ListLinks](docs/ListLinks.md)
21492157
- [ListLinksAllOf](docs/ListLinksAllOf.md)
2158+
- [ListLlmProviderModelsRequest](docs/ListLlmProviderModelsRequest.md)
2159+
- [ListLlmProviderModelsRequestProviderConfig](docs/ListLlmProviderModelsRequestProviderConfig.md)
2160+
- [ListLlmProviderModelsResponse](docs/ListLlmProviderModelsResponse.md)
21502161
- [ListServicesResponse](docs/ListServicesResponse.md)
21512162
- [LlmModel](docs/LlmModel.md)
21522163
- [LlmProviderAuth](docs/LlmProviderAuth.md)
@@ -2306,8 +2317,8 @@ Class | Method | HTTP request | Description
23062317
- [TabularExportRequest](docs/TabularExportRequest.md)
23072318
- [TestDefinitionRequest](docs/TestDefinitionRequest.md)
23082319
- [TestDestinationRequest](docs/TestDestinationRequest.md)
2320+
- [TestLlmProviderByIdRequest](docs/TestLlmProviderByIdRequest.md)
23092321
- [TestLlmProviderDefinitionRequest](docs/TestLlmProviderDefinitionRequest.md)
2310-
- [TestLlmProviderDefinitionRequestProviderConfig](docs/TestLlmProviderDefinitionRequestProviderConfig.md)
23112322
- [TestLlmProviderResponse](docs/TestLlmProviderResponse.md)
23122323
- [TestNotification](docs/TestNotification.md)
23132324
- [TestNotificationAllOf](docs/TestNotificationAllOf.md)
@@ -2319,6 +2330,8 @@ Class | Method | HTTP request | Description
23192330
- [TotalDimension](docs/TotalDimension.md)
23202331
- [TotalExecutionResultHeader](docs/TotalExecutionResultHeader.md)
23212332
- [TotalResultHeader](docs/TotalResultHeader.md)
2333+
- [TrendingObjectItem](docs/TrendingObjectItem.md)
2334+
- [TrendingObjectsResult](docs/TrendingObjectsResult.md)
23222335
- [TriggerAutomationRequest](docs/TriggerAutomationRequest.md)
23232336
- [TriggerQualityIssuesCalculationResponse](docs/TriggerQualityIssuesCalculationResponse.md)
23242337
- [UpsertKnowledgeDocumentRequestDto](docs/UpsertKnowledgeDocumentRequestDto.md)

0 commit comments

Comments
 (0)