Skip to content

Commit 81c29df

Browse files
ManangkaTeamcityJoerivanEngelen
authored
Speed up TeamCity build (#1803)
Fixes #1804 In an effort to speed up the CI-pipeline 2 changes have been made: - After some investigation a bottleneck has been identified during the CI-pipeline. The CI build steps are run in docker containers. In this containers a volume is mounted which contains the checkout repository. This is the way TeamCity works. When installing the pixi environment the environment the environment is installed into this mounted volume. By moving the environment out of the mounted volume to another location in the container a noticeable speedup is achieved. - The editable installation of imod has been changed from a task to a pypi-dependency. This way it happens at the same time as when the pixi environment is being installed, which results in a slight speedup **Unrelated** The mypy daemon kept crashing in vscode. The reason is that it encountered some partial types when analyzing pyvista. I've disabled following that import. Now its possible to use the daemon i.c.w. the mypy extension in vscode, which should give you faster feedback when new mypy errors popup # Description <!--- Thanks for opening a PR! Please add your description here of changes made and how they are going to resolve the linked issue --> # Checklist <!--- Before requesting review, please go through this checklist: --> - [ ] Links to correct issue - [ ] Update changelog, if changes affect users - [ ] PR title starts with ``Issue #nr``, e.g. ``Issue #737`` - [ ] Unit tests were added - [ ] **If feature added**: Added/extended example - [ ] **If feature added**: Added feature to API documentation - [ ] **If pixi.lock was changed**: Ran `pixi run generate-sbom` and committed changes --------- Co-authored-by: Teamcity <teamcity@deltares.nl> Co-authored-by: Joeri van Engelen <joerivanengelen@hotmail.com>
1 parent 01fd9bc commit 81c29df

11 files changed

Lines changed: 5042 additions & 17 deletions

.teamcity/Templates/AcceptanceTestsTemplate.kt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,9 @@ object AcceptanceTestsTemplate : Template({
3030
workingDir = "imod-python"
3131
scriptContent = """
3232
SET PATH=%%PATH%%;%system.teamcity.build.checkoutDir%\modflow6
33+
34+
pixi config set --local detached-environments "C:\pixi_envs"
35+
pixi install --environment user-acceptance --frozen
3336
3437
pixi run --environment user-acceptance --frozen dvc remote modify --local minio access_key_id %env.access_key%
3538
pixi run --environment user-acceptance --frozen dvc remote modify --local minio secret_access_key %env.secret_access_key%

.teamcity/Templates/ExamplesTemplate.kt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,8 @@ object ExamplesTemplate : Template({
2828
workingDir = "imod-python"
2929
scriptContent = """
3030
SET PATH=%%PATH%%;%system.teamcity.build.checkoutDir%\modflow6
31+
pixi config set --local detached-environments "C:\pixi_envs"
32+
pixi install --environment default --frozen
3133
pixi run --environment default --frozen examples
3234
""".trimIndent()
3335
formatStderrAsError = true

.teamcity/Templates/LintTemplate.kt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,8 @@ object LintTemplate : Template({
2323
id = "Static_code_analysis"
2424
workingDir = "imod-python"
2525
scriptContent = """
26+
pixi config set --local detached-environments "C:\pixi_envs"
27+
pixi install --environment default --frozen
2628
pixi run --environment default --frozen lint
2729
""".trimIndent()
2830
formatStderrAsError = true

.teamcity/Templates/MyPyTemplate.kt

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,13 +27,15 @@ object MyPyTemplate : Template({
2727
id = "MyPy_analysis"
2828
workingDir = "imod-python"
2929
scriptContent = """
30+
pixi config set --local detached-environments "C:\pixi_envs"
31+
pixi install --environment default --frozen
3032
pixi run --environment default --frozen mypy_report
31-
pixi run --environment default --frozen mypy
33+
pixi run --environment default --frozen mypy_lint
3234
""".trimIndent()
3335
formatStderrAsError = true
3436
dockerImage = "%DockerContainer%:%DockerVersion%"
3537
dockerImagePlatform = ScriptBuildStep.ImagePlatform.Windows
36-
dockerRunParameters = """--cpus=4 --memory=16g"""
38+
dockerRunParameters = """--cpus=8 --memory=16g"""
3739
dockerPull = false
3840
}
3941
}

.teamcity/Templates/PipPythonTemplate.kt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,8 @@ object PipPythonTemplate : Template({
2424
id = "pip_install"
2525
workingDir = "imod-python"
2626
scriptContent = """
27+
pixi config set --local detached-environments "C:\pixi_envs"
28+
pixi install --environment %python_env% --frozen
2729
pixi run --environment %python_env% --frozen test_import
2830
""".trimIndent()
2931
formatStderrAsError = true

.teamcity/Templates/UnitTestsTemplate.kt

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,8 @@ object UnitTestsTemplate : Template({
3131
workingDir = "imod-python"
3232
scriptContent = """
3333
SET PATH=%%PATH%%;%system.teamcity.build.checkoutDir%\modflow6
34+
pixi config set --local detached-environments "C:\pixi_envs"
35+
pixi install --environment default --frozen
3436
pixi run --environment default --frozen unittests
3537
""".trimIndent()
3638
formatStderrAsError = true
@@ -45,7 +47,9 @@ object UnitTestsTemplate : Template({
4547
workingDir = "imod-python/imod/tests"
4648
scriptMode = script {
4749
content = """
48-
${'$'}REPORT = echo "coverage report" | pixi shell --environment default
50+
pixi config set --local detached-environments "C:\pixi_envs"
51+
52+
${'$'}REPORT = echo "coverage report" | pixi shell --environment default --frozen
4953
5054
${'$'}TOTALS = ${'$'}REPORT | Select-String -Pattern 'TOTAL' -CaseSensitive -SimpleMatch
5155
${'$'}STATISTICS = ${'$'}TOTALS -split "\s+"

docs/faq/python.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -128,7 +128,7 @@ practical sense.)
128128
To create a Pixi environment open a command prompt and ``cd`` to the imod folder.
129129
In the imod folder use the following command::
130130

131-
pixi run install
131+
pixi install
132132

133133
This will create a conda environment inside the imod folder. To activate the environment run::
134134

docs/installation.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -170,7 +170,7 @@ With pixi you can install the latest development version of imod::
170170

171171
git clone https://github.com/Deltares/imod-python.git
172172
cd imod-python
173-
pixi run install
173+
pixi install
174174

175175
This will install the same python installation the iMOD Python developers work
176176
with, so it should work (otherwise we couldn't do our work!). This contains an

0 commit comments

Comments
 (0)