You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: modules/ROOT/pages/development-and-deployment.adoc
+87-11Lines changed: 87 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,31 +12,99 @@ ThoughtSpot instances act as a constantly running service, so xref:development-a
12
12
13
13
ThoughtSpot provides numerous tools for building a structured deployment process, built around the link:https://docs.thoughtspot.com/cloud/latest/tml[ThoughtSpot Modeling Language (TML), window=_blank] format for representing the xref:intro-thoughtspot-objects.adoc[objects within ThoughtSpot].
14
14
15
-
== Best practices
16
-
The primary tool for structured development and deployment in ThoughtSpot is called xref:orgs.adoc[Orgs].
15
+
== Overview
16
+
ThoughtSpot may provision your organization one or more separate *instances*, each with an individual URL.
17
17
18
-
Each Org in ThoughtSpot can be xref:version_control.adoc[paired] to a link:https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-branches[branch, target=_blank] in a link:https://docs.github.com/en/repositories/creating-and-managing-repositories/about-repositories[Git repository, target=_blank] as a single *environment*.
18
+
Within a ThoughtSpot instance, the administrator of the Primary Org can create any number of logical tenants called xref:orgs.adoc[Orgs].
19
19
20
20
xref:orgs.adoc[Orgs] are fully separated tenants on a single ThoughtSpot instance. For TSE customers and others who need a structured development and deployment process, Orgs should always be enabled.
21
21
22
-
Once your environments are configured, you can xref:version_control.adoc#moving-tml-changes-between-environments[move data models and content] from the initial "dev environment" to any other environment using the xref:git-rest-api-guide.adoc[Git integration REST APIs].
22
+
image::./images/instances_and_orgs.png[Instances and Orgs]
23
+
24
+
By setting the `xref:intro-thoughtspot-objects.adoc#object-identifiers[obj_id]` property of objects, objects in various Orgs that are related copies of one another will have the same `obj_id`, allowing for tracking related objects and updating them without concern for each object's unique GUID.
23
25
24
26
[NOTE]
25
27
====
26
-
ThoughtSpot does not recommend TML export and import across different versions of ThoughtSpot application because the TML syntax, supported features, and object schemas can vary between releases and can sometimes lead to compatibility issues and validation errors.
28
+
ThoughtSpot does not recommend TML export and import from a *newer version* of ThoughtSpot to an instance on a *previous version*, because the TML syntax, supported features, and object schemas can vary between releases and can sometimes lead to compatibility issues and validation errors.
27
29
====
28
30
31
+
== Version Control
32
+
*Version control* is the process of tracking changes that occur to objects in ThoughtSpot.
33
+
34
+
A single branch in Git can be used for version control of a single Org in ThoughtSpot.
35
+
36
+
ThoughtSpot provides a GitHub-based link:https://docs.thoughtspot.com/cloud/10.15.0.cl/git-version-control[automated version control in the UI] for Liveboards and answers or a customized process can be built using the TML Export API to any Git provider.
37
+
38
+
It is best to use separate branches or even repositories for the UI automated version control and direct REST API processes.
39
+
40
+
When using the TML REST APIs and a Git provider, you can also implement version control branches, but they should be separate from the "deploy branches":
41
+
42
+
image::./images/version-control-branches.png[version control branches diagram]
43
+
44
+
It is important to have a version control branch for any "prod" Orgs with end user created content, which otherwise will not be archived in any way.
45
+
46
+
== Deployment
47
+
*Deployment* is the process of *making copies of objects* from one Org to another Org.
48
+
49
+
Deployment is used in the process of building a *release* from a *dev Org* and then deploying copies of the release objects via TML to *test*, *uat*, and eventual *prod* Orgs.
50
+
51
+
The TML Export and Import APIs allow customizable release and deployment processes to integrate with any Git provider.
52
+
53
+
The standard deployment pattern for a xref:multi-tenancy-intro.adoc[multi-tenanted prod database] is shown below. RLS rules will filter the shared data models on the "prod" Org so that standard LBs and Answers only show the right data for each end customer, who are all only added as users to the "prod" Org.
In a scenario where xref:single-tenant-data-models.adoc[end customer databases are single-tenanted], an Org can be created programmatically matching with the level of tenant separation, so that there is an Org representing each separate logical database.
58
+
59
+
There are two techniques for managing Orgs for single-tenanted databases:
60
+
61
+
* Publishing [beta betaBackground]^Beta^
62
+
* Deploy to each Org using TML Import
63
+
64
+
== Publishing
65
+
66
+
*xref:publishing-overview.adoc[Publishing]* makes objects available in other Orgs without making copies.
67
+
68
+
Variables can be set at the Org level to override the Connection and Table object details for Publishing objects when they are accessed in a specific Org. Variables for connections and tables work with both Publishing and TML Import.
69
+
70
+
image::./images/publishing_diagram.png[Publishing and Variables]
71
+
72
+
With single-tenanted prod Orgs, the *Publishing* feature allows the final deployment step to import TML into the *publisher* Org, updating the *Published* objects, which instantly updates the objects in every Org the objects are published to.
73
+
74
+
image::./images/single_tenant_publishing.png[Publishing to Single-Tenants]
75
+
76
+
If there are structural differences within the various databases that make Publishing unviable, the TML Import process can be used to deploy unique copies of the release TML into each Org. This process may also include modifying the release TML to introduce variation into the objects that are deployed.
77
+
78
+
image::./images/single_tenant_deployment.png[Individual Deployment to Single-Tenants]
79
+
80
+
81
+
82
+
83
+
////
84
+
Each Org in ThoughtSpot can be xref:version_control.adoc[paired] to a link:https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-branches[branch, target=_blank] in a link:https://docs.github.com/en/repositories/creating-and-managing-repositories/about-repositories[Git repository, target=_blank] as a single *environment*.
85
+
////
86
+
87
+
////
88
+
Once your environments are configured, you can xref:version_control.adoc#moving-tml-changes-between-environments[move data models and content] from the initial "dev environment" to any other environment using the xref:git-rest-api-guide.adoc[Git integration REST APIs].
89
+
////
90
+
91
+
92
+
29
93
////
30
94
If you have used ThoughtSpot for a long time and are enabling Orgs for the first time, please see xref:moving-to-orgs.adoc[moving to Orgs from single-tenant ThoughtSpot instance].
31
95
////
32
-
== Standard structure of Orgs
96
+
////
97
+
== Typical Orgs pattern for SDLC
33
98
When Orgs are enabled, the *Primary Org* (`org_id: 0`) should be treated as the "root" or "super-admin" Org, and all "real content" should exist on Orgs other than the Primary.
34
99
100
+
Additionally, the *Primary Org* is the only Org where *Published* objects can originate, so *Primary* serves as the *Publisher* Org within SDLC flows that involve Publishing.
101
+
35
102
The most basic standard configuration for separated development and deployment, with a xref:multi-tenancy-best-practices.adoc[multi-tenanted "prod" environment], has the following environments:
36
103
37
104
- *Primary*: only used for administration of instance and Orgs themselves
38
-
- *Dev*: ThoughtSpot customer internal development team builds data models and standard Liveboards and Answers
39
-
- *Prod*: Data models and standard content are published to this environment by service account. All end customers get READ-ONLY access to data models, standard Liveboards, and Answers.
105
+
- *dev*: ThoughtSpot customer internal development team builds data models and standard Liveboards and Answers
106
+
- *test*:
107
+
- *prod*: Data models and standard content are published to this environment by service account. All end customers get READ-ONLY access to data models, standard Liveboards, and Answers.
@@ -47,20 +115,25 @@ RLS filters the shared data models on the "prod" Org so that standard LBs and An
47
115
If the production end customer data models are xref:single-tenant-data-models.adoc[single-tenanted (logical separation for each end customer)], such that a different connection is necessary), then the standard deployment involves a "release"/"pre-prod" Org and then one prod Org per end customer.
48
116
49
117
image::./images/single-tenant_prod_per_customer.png[Single-tenant final deployment model]
50
-
118
+
////
119
+
////
51
120
=== Additional "typical" Orgs
52
121
53
122
- *Test*, *UAT*: Additional steps in the publishing process between Dev and Prod, for verification before changes are deployed into Prod
54
123
- *Internal Org(s)*: Org(s) for ThoughtSpot customer's own business work, never to be shared with end customers
55
124
- Separate *'dev' instance*: Some people want a fully separated environment for dev, especially if doing MAJOR changes, even to cluster config. Recommend there be a 'Test Org' on the 'Prod Instance' to push to first, before pushing to 'Prod'
56
125
- Separate *regional instances*: prod per customer Orgs may extend to multiple ThoughtSpot instances in different regional cloud data centers
126
+
////
57
127
128
+
////
58
129
[#deploy-between-environments]
59
130
== Deploying content between environments
60
-
////
131
+
61
132
The workflow for a very simple "dev" to "prod" flow on the same environment shown here, is the same pattern for any source-to-destination environment flow:
62
133
63
134
image::./images/development-deployment-process.png[Development and deployment workflow]
135
+
////
136
+
64
137
////
65
138
=== Git deployment
66
139
The xref:version_control.adoc[Git REST APIs] are designed for the best practice pattern of building on a Dev Org and then deploying to any number of Orgs through Prod(s).
@@ -74,9 +147,12 @@ You will need a xref:guid-mapping.adoc[GUID Mapping file] that records the `orig
74
147
75
148
== Multi-tenancy and data security
76
149
The exact deployment pattern chosen will depend on the design of your data warehouse. Please see the xref:multi-tenancy-intro.adoc[full documentation on multi-tenancy within ThoughtSpot] to determine which deployment pattern best fits your needs.
150
+
////
77
151
152
+
////
78
153
== Publishing content to Orgs within a ThoughtSpot instance
79
154
80
155
Starting with the 10.10.0.cl release, ThoughtSpot provides the ability to parameterize object properties using variables for each Org and publish objects directly from the Primary Org to other Orgs on a multi-tenant instance. The publishing feature [beta betaBackground]^Beta^ enables administrators to create a single main object in the Primary Org and distribute it to other Orgs within the instance.
81
156
82
-
For more information, see xref:publishing-overview.adoc[Publishing content to Orgs].
157
+
For more information, see xref:publishing-overview.adoc[Publishing content to Orgs].
Copy file name to clipboardExpand all lines: modules/ROOT/pages/git-configuration.adoc
+20-16Lines changed: 20 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,12 +1,17 @@
1
-
= Configure Git integration
1
+
= Configure GitHub integration
2
2
:toc: true
3
3
:toclevels: 2
4
4
5
-
:page-title: Configure Git integration
5
+
:page-title: Configure GitHub integration
6
6
:page-pageid: git-configuration
7
-
:page-description: Configuring the Git integration for a ThoughtSpot instance
7
+
:page-description: Configuring the GitHub integration for a ThoughtSpot instance
8
8
9
-
Git integration requires configuration within both ThoughtSpot, accomplished via the V2.0 REST APIs, and your Git provider (GitHub currently).
9
+
[NOTE]
10
+
====
11
+
The legacy GitHub REST APIs referenced here are tied to GitHub exclusively. The newer xref:git-provider-integration.adoc[Git provider integration pattern] provides more flexibility and control and should be utilized if you are starting out or having issues with the GitHub APIs.
12
+
====
13
+
14
+
GitHub integration requires configuration within both ThoughtSpot, accomplished via the V2.0 REST APIs, and your Git provider (GitHub currently).
10
15
11
16
== Configure Git repository
12
17
@@ -44,7 +49,7 @@ tscli git-integration enable
44
49
////
45
50
46
51
== Confirm permissions within ThoughtSpot Orgs
47
-
* To commit objects from Thoughtspot to a Git repository, your ThoughtSpot user account requires at least view permission for all objects that will be committed as part of the operation.
52
+
* To commit objects from ThoughtSpot to a Git repository, your ThoughtSpot user account requires at least view permission for all objects that will be committed as part of the operation.
48
53
* To deploy or revert objects from a Git repository to ThoughtSpot, you require edit access to all objects that will be updated as part of the deployment. If the deployment contains Models, Views, or Tables, users require **Can manage data** (`DATAMANAGEMENT`) privilege for deploy, commit, and revert operations.
49
54
50
55
[#guid-map-and-config-files]
@@ -80,17 +85,16 @@ For more advanced Git development patterns, list multiple `branch_names` in the
80
85
|===
81
86
|Parameter|Description
82
87
|`repository_url`|__String__. The HTTPS URL of the Git repository; for example, `\https://github.com/user/repo.git`.
83
-
|`username`
84
-
|__String__. Username to authenticate to the Git repository.
88
+
|`username`|__String__. Username to authenticate to the Git repository.
85
89
|`access_token`|__String__. Access token to authenticate to the Git repository.
86
90
|`org_identifier`|__String__. ID of the Org. Define this parameter only if the Orgs feature is enabled on your ThoughtSpot cluster and separate Orgs are configured for development and production environments.
87
91
|`branch_names`|__Array of strings__. List of Git branches to configure.
88
-
|`commit_branch_name` |__String__. Name of the remote branch where objects committed from this Thoughtspot instance will be versioned. Replaces `default_branch_name`, which is deprecated in 9.10.5.cl.
92
+
|`commit_branch_name` |__String__. Name of the remote branch where objects committed from this ThoughtSpot instance will be versioned. Replaces `default_branch_name`, which is deprecated in 9.10.5.cl.
89
93
|`default_branch_name` +
90
94
__Optional__|__String__. Deprecated in 9.10.5.cl. In earlier versions, this parameter was used to configure the name of the default Git branch to use for all operations on the cluster.
91
95
|`enable_guid_mapping` |__Boolean__. Enables GUID mapping and generates a GUID mapping file. Starting from 9.7.0.cl, this attribute is set to `true` by default.
92
96
To know more about GUID mapping, see xref:version_control.adoc#_guid_mapping_and_configuration_files[GUID mapping].
93
-
|`configuration_branch_name` a|__String__. Name of the branch where the configuration files related to operations between Thoughtspot and the version control repository should be maintained. Replaces `guid_mapping_branch_name`, which is deprecated in 9.10.5.cl.
97
+
|`configuration_branch_name` a|__String__. Name of the branch where the configuration files related to operations between ThoughtSpot and the version control repository should be maintained. Replaces `guid_mapping_branch_name`, which is deprecated in 9.10.5.cl.
94
98
95
99
[NOTE]
96
100
====
@@ -109,14 +113,14 @@ The following example shows the API request format for connecting ThoughtSpot to
If the API request is successful, the ThoughtSpot instance will be connected to the Git repository.
130
134
131
-
Go into each Org an issue the `config/create` call to link the appropriate branch to establish all environments.
135
+
Go into each Org and issue the `config/create` call to link the appropriate branch to establish all environments.
132
136
133
137
The following example shows the API request parameters to connect a ThoughtSpot `Prod` Org to the Git repo. Note that GUID mapping is enabled in the API request.
134
138
@@ -138,15 +142,15 @@ The `Bearer_token` value must be requested for the desired Org, specified throug
@@ -162,9 +166,9 @@ Version control and xref:guid-mapping.adoc[GUID mapping] utilize files stored in
162
166
163
167
You can initialize these files by immediately using the xref:git-rest-api-guide.adoc#deploy-commits[deploy commits REST API] after configuring the Org for Git.
164
168
165
-
See the link:https://github.com/thoughtspot/thoughtspot_rest_api_v1_python/blob/main/examples_v2/create_orgs_with_linked_git_branch.py[create_orgs_with_linked_git_branch.py script] for an example of deploying a full Orgs or branches setup for structured development and deployment.
169
+
See the link:https://github.com/thoughtspot/thoughtspot_rest_api_v1_python/blob/main/examples_v2/create_orgs_with_linked_git_branch.py[create_orgs_with_linked_git_branch.py script, window=_blank] for an example of deploying a full Orgs or branches setup for structured development and deployment.
166
170
167
-
After the Orgs are set up, you can link:https://github.com/thoughtspot/thoughtspot_rest_api_v1_python/blob/main/examples_v2/create_connection_on_orgs.py[create connections across the orgs] and xref:guid-mapping.adoc#using-mapping-for-table-tml-properties[add any necessary entries to the GUID mapping files].
171
+
After the Orgs are set up, you can link:https://github.com/thoughtspot/thoughtspot_rest_api_v1_python/blob/main/examples_v2/create_connection_on_orgs.py[create connections across Orgs, window=_blank] and xref:guid-mapping.adoc#using-mapping-for-table-tml-properties[add any necessary entries to the GUID mapping files].
0 commit comments