Update content processing for new website (#848)

scottgigante-immunai · web-flow · commit 637163fba7d7 · 2023-03-10T10:25:04.000-05:00
* parse metadata in new format

* typo # ci skip

* ignore workdir

* remove selectively

* temp

* bugfix # ci skip

* fix ref # ci skip

* more temp # ci skip

* don't remove tasks # ci skip # publish

* pretty print # ci skip # publish

* move get_sha to utils

* add get_sha to workflow_utils

* push meta to experimental website

* monitor separate;y

* don't build images for website

* id -&gt; task_id

* add description if missing

* only publish explicitly

* test website # publish

* fix ref # publish

* untemp

* remove patch # publish

* clean up readmes # publish

* also publish the bib

* remove experimental website repo

* remove unnecessary diff

* add code url to function metadata # publish

* set base to main # publish

* code_url was already taken # publish
diff --git a/.github/workflows/run_tests.yml b/.github/workflows/run_tests.yml
@@ -26,6 +26,7 @@ jobs:
     if: |
       !endsWith(github.event.head_commit.message, '# ci skip') &&
       !startsWith(github.ref, 'refs/heads/test_process') &&
+      !startsWith(github.ref, 'refs/heads/test_website') &&
       (
         (github.event_name != 'pull_request_review') ||
         (
diff --git a/.github/workflows/update_website_content.yml b/.github/workflows/update_website_content.yml
@@ -31,6 +31,7 @@ jobs:
       with:
         fetch-depth: 1
         repository: openproblems-bio/website
+        ref: main
         path: website
         token: ${{ secrets.GH_ACTIONS_WEBSITE_PAT }}
 
@@ -51,18 +52,18 @@ jobs:
         pip install -U --editable ./openproblems[process]
         python -c "import openproblems"
 
-    - name: Parse results
+    - name: Parse metadata
       run: |
-        rm -r website/content/benchmarks/*/
-        python openproblems/workflow/generate_website_markdown.py website/content/benchmarks
-        cp openproblems/main.bib website/static/bibliography
+        python openproblems/workflow/parse_metadata.py website/results
+        cp openproblems/main.bib website/bibliography/library.bib
         cd website
         git diff --exit-code --quiet || echo "CHANGED=true" >> $GITHUB_ENV
 
-    - name: Upload markdown
+
+    - name: Upload json
       uses: actions/upload-artifact@main
       with:
-        name: markdown
+        name: json
         path: website/content/benchmarks
 
     - name: Push to openproblems-bio/website
@@ -88,11 +89,11 @@ jobs:
         )
       uses: peter-evans/create-pull-request@v4
       with:
+        base: main
         branch: ${{ env.UPDATE_BRANCH_NAME }}
         delete-branch: true
-        base: main
         title: '[auto] Update benchmark content'
-        reviewers: scottgigante-immunai,rcannood,dburkhardt
+        reviewers: scottgigante-immunai,rcannood
         path: './website'
         token: ${{ secrets.GH_ACTIONS_WEBSITE_PAT }}
         author: "openproblems-bio <singlecellopenproblems@protonmail.com>"
diff --git a/main.bib b/main.bib
@@ -773,7 +773,7 @@ @article{schober2018correlation
   author        = {Patrick Schober and Christa Boer and Lothar A. Schwarte},
   year          = {2018},
   month         = may,
-  journal       = {Anesthesia {\&}amp$\mathsemicolon$ Analgesia},
+  journal       = {Anesthesia {\&} Analgesia},
   publisher     = {Ovid Technologies (Wolters Kluwer Health)},
   volume        = {126},
   number        = {5},
diff --git a/openproblems/tasks/_batch_integration/batch_integration_embed/README.md b/openproblems/tasks/_batch_integration/batch_integration_embed/README.md
@@ -1,9 +1,5 @@
-<!--- TODO: add links --->
-
 # Batch integration embedding
 
-## The task
-
 This is a sub-task of the overall batch integration task. Batch (or data) integration
 integrates datasets across batches that arise from various biological and technical
 sources. Methods that integrate batches typically have three different types of output:
diff --git a/openproblems/tasks/_batch_integration/batch_integration_feature/README.md b/openproblems/tasks/_batch_integration/batch_integration_feature/README.md
@@ -1,9 +1,5 @@
-<!--- TODO: add links --->
-
 # Batch integration feature
 
-## The task
-
 This is a sub-task of the overall batch integration task. Batch (or data) integration
 integrates datasets across batches that arise from various biological and technical
 sources. Methods that integrate batches typically have three different types of output:
diff --git a/openproblems/tasks/_batch_integration/batch_integration_graph/README.md b/openproblems/tasks/_batch_integration/batch_integration_graph/README.md
@@ -1,9 +1,5 @@
-<!--- TODO: add links --->
-
 # Batch integration (graph)
 
-## The task
-
 This is a sub-task of the overall batch integration task. Batch (or data) integration
 methods integrate datasets across batches that arise from various biological and
 technical sources. Methods that integrate batches typically have three different types
diff --git a/openproblems/tasks/_cell_cell_communication/README.md b/openproblems/tasks/_cell_cell_communication/README.md
@@ -1,7 +1,5 @@
 # Cell-cell Communication
 
-## The task
-
 The growing availability of single-cell data has sparked an increased
 interest in the inference of cell-cell communication (CCC),
 with an ever-growing number of computational tools developed for this purpose.
diff --git a/openproblems/tasks/_cell_cell_communication/cell_cell_communication_ligand_target/README.md b/openproblems/tasks/_cell_cell_communication/cell_cell_communication_ligand_target/README.md
@@ -1,6 +1,4 @@
-# Cell-cell Communication
-
-## The task
+# Cell-cell Communication (ligand-target)
 
 The growing availability of single-cell data has sparked an increased
 interest in the inference of cell-cell communication (CCC),
diff --git a/openproblems/tasks/_cell_cell_communication/cell_cell_communication_source_target/README.md b/openproblems/tasks/_cell_cell_communication/cell_cell_communication_source_target/README.md
@@ -1,6 +1,4 @@
-# Cell-cell Communication
-
-## The task
+# Cell-cell Communication (source-target)
 
 The growing availability of single-cell data has sparked an increased
 interest in the inference of cell-cell communication (CCC),
diff --git a/openproblems/tasks/denoising/README.md b/openproblems/tasks/denoising/README.md
@@ -1,7 +1,5 @@
 # Denoising
 
-## The task
-
 Single-cell RNA-Seq protocols only detect a fraction of the mRNA molecules present
 in each cell. As a result, the measurements (UMI counts) observed for each gene and each
 cell are associated with generally high levels of technical noise ([Grün et al.,
diff --git a/openproblems/tasks/dimensionality_reduction/README.md b/openproblems/tasks/dimensionality_reduction/README.md
@@ -1,7 +1,5 @@
 # Dimensionality reduction for visualisation
 
-## The task
-
 Dimensionality reduction is one of the key challenges in single-cell data
 representation. Routine single-cell RNA sequencing (scRNA-seq) experiments measure cells
 in roughly 20,000-30,000 dimensions (i.e., features - mostly gene transcripts but also
diff --git a/openproblems/tasks/label_projection/README.md b/openproblems/tasks/label_projection/README.md
@@ -1,7 +1,5 @@
 # Label Projection
 
-## The task
-
 A major challenge for integrating single cell datasets is creating matching cell type
 annotations for each cell. One of the most common strategies for annotating cell types
 is referred to as
diff --git a/openproblems/tasks/matching_modalities/README.md b/openproblems/tasks/matching_modalities/README.md
@@ -1,7 +1,5 @@
 # Matching modalities
 
-## The task
-
 Cellular function is regulated by the complex interplay of different types of biological
 molecules (DNA, RNA, proteins, etc.), which determine the state of a cell. Several
 recently described technologies allow for simultaneous measurement of different aspects
diff --git a/openproblems/tasks/spatial_decomposition/README.md b/openproblems/tasks/spatial_decomposition/README.md
@@ -1,7 +1,5 @@
 # Spatial Decomposition/Deconvolution
 
-## The task
-
 Spatial decomposition (also often referred to as Spatial deconvolution) is
 applicable to spatial transcriptomics data where the transcription profile of
 each capture location (spot, voxel, bead, etc.) do not share a bijective
diff --git a/openproblems/utils.py b/openproblems/utils.py
@@ -48,3 +48,8 @@ def get_members(module):
 def get_callable_members(module):
     """Get all callable public members from a module."""
     return [member for member in get_members(module) if callable(member)]
+
+
+def get_member_id(member):
+    """Get the submodule or function name for a task, dataset, method or metric"""
+    return member.__name__.split(".")[-1]
diff --git a/scripts/generate_test_matrix.py b/scripts/generate_test_matrix.py
@@ -8,7 +8,7 @@
 def generate_matrix():
     suites = _CORE_TEST_SUITES.copy()
     for task in openproblems.TASKS:
-        task_name = task.__name__.split(".")[-1]
+        task_name = openproblems.utils.get_member_id(task)
         suites.extend([f"{suite} and {task_name}" for suite in _TASK_TEST_SUITES])
     return suites
 
diff --git a/setup.py b/setup.py
@@ -29,7 +29,7 @@
 
 evaluate_requires = ["snakemake>=7.8,<7.17", "tabulate<0.9"]
 
-process_requires = ["numpyencoder==0.3.*"]
+process_requires = ["numpyencoder==0.3.*", "gitpython==3.1.*"]
 
 test_requires = [
     "pytest==7.1.*",
diff --git a/test/test_core_cli.py b/test/test_core_cli.py
@@ -24,11 +24,13 @@ def test_print(capsys):
 def test_tasks(capsys):
     """Test task listing."""
     result = np.array(main(["tasks"], do_print=False))
-    expected = np.array([task.__name__.split(".")[-1] for task in openproblems.TASKS])
+    expected = np.array(
+        [openproblems.utils.get_member_id(task) for task in openproblems.TASKS]
+    )
     assert np.all(result == expected)
     result = np.array(main(["tasks"], do_print=True))
-    expected = (
-        "\n".join([task.__name__.split(".")[-1] for task in openproblems.TASKS]) + "\n"
+    expected = "\n".join(
+        [openproblems.utils.get_member_id(task) for task in openproblems.TASKS] + [""]
     )
     captured = capsys.readouterr()
     assert captured.out == expected
@@ -42,7 +44,7 @@ def test_list(task):
     """Test function listing."""
     result = np.array(
         main(
-            ["list", "--task", task.__name__.split(".")[-1], "--datasets"],
+            ["list", "--task", openproblems.utils.get_member_id(task), "--datasets"],
             do_print=False,
         )
     )
@@ -51,7 +53,7 @@ def test_list(task):
 
     result = np.array(
         main(
-            ["list", "--task", task.__name__.split(".")[-1], "--methods"],
+            ["list", "--task", openproblems.utils.get_member_id(task), "--methods"],
             do_print=False,
         )
     )
@@ -60,7 +62,7 @@ def test_list(task):
 
     result = np.array(
         main(
-            ["list", "--task", task.__name__.split(".")[-1], "--metrics"],
+            ["list", "--task", openproblems.utils.get_member_id(task), "--metrics"],
             do_print=False,
         )
     )
@@ -73,7 +75,7 @@ def _test_image(task, function_type, function):
         [
             "image",
             "--task",
-            task.__name__.split(".")[-1],
+            openproblems.utils.get_member_id(task),
             function_type,
             function.__name__,
         ],
@@ -204,7 +206,7 @@ def __zero_metric(*args):
             [
                 "evaluate",
                 "--task",
-                task.__name__.split(".")[-1],
+                openproblems.utils.get_member_id(task),
                 "--input",
                 dataset_file,
                 metric_name,
diff --git a/test/test_task_1_load_data.py b/test/test_task_1_load_data.py
@@ -7,7 +7,7 @@
 @parameterized.parameterized.expand(
     [
         (
-            task.__name__.split(".")[-1],
+            openproblems.utils.get_member_id(task),
             dataset.__name__,
             test,
             utils.TEMPDIR.name,
diff --git a/test/test_task_methods.py b/test/test_task_methods.py
@@ -14,7 +14,7 @@
 @parameterized.parameterized.expand(
     [
         (
-            task.__name__.split(".")[-1],
+            openproblems.utils.get_member_id(task),
             method.__name__,
             method.metadata["image"],
         )
diff --git a/test/test_task_metrics.py b/test/test_task_metrics.py
@@ -8,7 +8,7 @@
 @parameterized.parameterized.expand(
     [
         (
-            task.__name__.split(".")[-1],
+            openproblems.utils.get_member_id(task),
             metric.__name__,
             metric.metadata["image"],
         )
diff --git a/test/utils/cache.py b/test/utils/cache.py
@@ -1,10 +1,11 @@
 import anndata
+import openproblems
 import os
 
 
 def _cache_name(tempdir, task, dataset, test=None, method=None):
     if not isinstance(task, str):
-        task = task.__name__.split(".")[-1]
+        task = openproblems.utils.get_member_id(task)
     if not isinstance(dataset, str):
         dataset = dataset.__name__
     if method is not None:
diff --git a/workflow/generate_website_markdown.py b/workflow/generate_website_markdown.py
diff --git a/workflow/parse_metadata.py b/workflow/parse_metadata.py
diff --git a/workflow/workflow_utils.py b/workflow/workflow_utils.py

Original file line number	Diff line number	Diff line change
`@@ -26,6 +26,7 @@ jobs:`
`26`	`26`	`if: \|`
`27`	`27`	`!endsWith(github.event.head_commit.message, '# ci skip') &&`
`28`	`28`	`!startsWith(github.ref, 'refs/heads/test_process') &&`
	`29`	`+ !startsWith(github.ref, 'refs/heads/test_website') &&`
`29`	`30`	`(`
`30`	`31`	`(github.event_name != 'pull_request_review') \|\|`
`31`	`32`	`(`