Release preparation for v0.5 (#10)

Author: chrzyki

.gitignore

@@ -87,6 +87,7 @@ raw/othanieljen/
 raw/polyglottaafricana/
 raw/robbeetstriangulation/
 raw/robinsonap/
+raw/rutulbasiclexicon/
 raw/servamalagasy/
 raw/sohartmannchin/
 raw/spagnolmaltese/
@@ -105,3 +106,6 @@ raw/yuchinese/
 raw/zgraggenmadang/
 raw/zhoubizic/
 
+.DS_Store
+.idea/
+

.zenodo.json

@@ -1,5 +1,5 @@
 {
-    "title": "CLICS\u2074",
+    "title": "CLICS 4",
     "access_right": "open",
     "keywords": [
         "cldf:Wordlist",
@@ -10,10 +10,10 @@
             "name": "Annika Tjuka"
         },
         {
-            "name": "Christoph Rzymski"
+            "name": "Robert Forkel"
         },
         {
-            "name": "Robert Forkel"
+            "name": "Christoph Rzymski"
         },
         {
             "name": "Johann-Mattis List"
@@ -29,5 +29,5 @@
     "license": {
         "id": "CC-BY-4.0"
     },
-    "description": "<p>Cite the source of the dataset as:</p>\n\n<blockquote>\n<p>Tjuka, Annika; Forkel, Robert; Rzymski, Christoph; and List, Johann-Mattis (2025): CLICS\u2074: An Improved Database of Cross-Linguistic Colexifications [Dataset, Version 0.4]. Passau: MCL Chair at the University of Passau.</p>\n</blockquote>"
+    "description": "<p>Cite the source of the dataset as:</p>\n\n<blockquote>\n<p>Tjuka, Annika; Forkel, Robert; Rzymski, Christoph; and List, Johann-Mattis (2025): CLICS 4: An Improved Database of Cross-Linguistic Colexifications [Dataset, Version 0.5]. Passau: MCL Chair at the University of Passau.</p>\n</blockquote>"
 }

NOTES.md

@@ -31,7 +31,7 @@ All you need to install the packages required is to install the current package
 ```
 $ git clone https://github.com/clics/clics4.git
 $ cd clics4
-$ git checkout v0.4
+$ git checkout v0.5
 $ pip install -e .
 ```
 

README.md

@@ -1,75 +1,85 @@
-# CLICS⁴
+# CLICS 4
 
 ## How to cite
 
 If you use these data please cite
 - the original source
-  > Tjuka, Annika; Forkel, Robert; Rzymski, Christoph; and List, Johann-Mattis (2025): CLICS⁴: An Improved Database of Cross-Linguistic Colexifications [Dataset, Version 0.4]. Passau: MCL Chair at the University of Passau.
+  > Tjuka, Annika; Forkel, Robert; Rzymski, Christoph; and List, Johann-Mattis (2025): CLICS 4: An Improved Database of Cross-Linguistic Colexifications [Dataset, Version 0.5]. Passau: MCL Chair at the University of Passau.
 - the derived dataset using the DOI of the [particular released version](../../releases/) you were using
 
 ## Description
 
 
 This dataset is licensed under a CC-BY-4.0 license
 
-Available online at https://clics.clld.org
+Available online at https://github.com/clics/clics4
 
 ## Notes
 
-# CLICS4: Workflow
+# CLICS 4: Workflow for Data Generation
 
-The CLICS4 workflow differs slightly from the workflow we have used in [CLICS3](https://github.com/clics/clics3). We now have drastically increased the number of datasets, but we have also made sure to use stricter selection criteria for the languages to be included. This also results in different numbers with respect to the number of concepts and the number of language varieties. 
+The CLICS 4 workflow differs slightly from the workflow we have used in [CLICS3](https://github.com/clics/clics3). We now have drastically increased the number of datasets, but we have also made sure to use stricter selection criteria for the languages to be included. This also results in different numbers with respect to the number of concepts and the number of language varieties. 
 
-## How to Cite CLICS⁴?
+## How to Cite CLICS 4?
 
-> Tjuka, Annika; Forkel, Robert; Rzymski, Christoph; and List, Johann-Mattis (2025): CLICS⁴: An Improved Database of Cross-Linguistic Colexifications [Dataset Version 0.4]. Passau: MCL Chair at the University of Passau.
+If you use the data in your work, make sure to cite the correct version that you are using. For the currently most recent version, we recommend to cite it as follows:
 
-## W1: What is New in Comparison with CLICS³?
+> Tjuka, Annika; Forkel, Robert; Rzymski, Christoph; and List, Johann-Mattis (2025): CLICS 4: An Improved Database of Cross-Linguistic Colexifications [Dataset Version 0.5]. Passau: MCL Chair at the University of Passau. https://github.com/clics/clics4/
+
+Since the whole workflow underlying CLICS 4 regardless of the individual versions will be presented in a freely available publication, we also appreciate if you cite this forthcoming paper (already available as preprint):
+
+> Tjuka, Annika; Forkel, Robert; Rzymski, Christoph; and List, Johann-Mattis (forthcoming): Advancing the Database of Cross-Linguistic Colexifications with New Workflows and Data. Proceedings of the 16th International Conference on Computational Semantics (IWCS). Düsseldorf: Association for Computational Linguistics. 1-15. Preprint: https://doi.org/10.48550/arXiv.2503.11377
+
+## What is New in Comparison with CLICS³?
 
 The following points summarize major differences between CLICS³ and CLICS⁴:
 
-- more datasets in CLICS⁴: CLICS⁴ now uses 98 datasets, while CLICS³ used 30
-- fully transcribed data instead of data in orthography: CLICS⁴ now uses data fully transcribed to IPA, ignoring all datasets that only offer orthography (this results in fewer languages at times, despite the increase in datasets)
-- 
+- more datasets in CLICS 4: CLICS 4 now uses 98 datasets, while CLICS³ used 30
+- fully transcribed data instead of data in orthography: CLICS 4 now uses data fully transcribed to IPA, ignoring all datasets that only offer orthography (this results in fewer languages at times, despite the increase in datasets)
+- treatment of concepts: we now model some "hidden" colexifications that have been ignored before, since the concept identifiers in Concepticon cover two separate concepts that are frequently colexified as one single concept, as separate concepts (these are marked in the CLDF representation)
+- we provide a full-fledged CLDF dataset now, in which the concept network is also modeled with the help of CLDF
 
+## Workflow for Data Aggregation
 
-## W.1: Install Packages
+In the following, we run those interested in trying the workflow that we applied to construct CLICS 4 on their own machines through the workflow in due detail. To run the workflow, we assume that users have enough experience with Python in order to know how to create their own fresh virtual environment and know how to run commands in the terminal. 
+
+### W1: Install Packages
 
 All you need to install the packages required is to install the current package with [PIP](https://pypi.org/project/pip) as follows (using a fresh virtual environment), after having downloaded the `clics4` package with [GIT](https://git-scm.com). The following lines also obtain the version that we used in this demo.
 ```
 $ git clone https://github.com/clics/clics4.git
 $ cd clics4
-$ git checkout v0.4
+$ git checkout v0.5
 $ pip install -e .
 ```
 
 ## W2: Download Data
 
-In order to do a fresh download of all the data that we use in CLICS⁴, you need to run the following command:
+In order to do a fresh download of all the data that we use in CLICS 4, you need to run the following command:
 
 ```
 $ cldfbench download lexibank_clics4.py
 ```
 
-## W3: Create CLICS4 Dataset
+## W3: Create CLICS 4 Dataset
 
 Before you can run the code, you must make sure to have downloaded all data and also obtained actual copies of Glottolog, Concepticon, and CLTS. An easy way to obtain these with the help of `cldfbench` is to run the command `cldfbench catconfig` and follow instructions there. If you use a Windows machine, you will need some additional preparations (see [Snee 2024](https://calc.hypotheses.org/7852)), so we kindly ask you to follow the respective instructions in Snee (2024).
 
 If you have successfully run the `catconfig` subcommand, just type:
 
 ```
-$ cldfbench lexibank.makecldf --glottolog-version=v5.1 --concepticon-version=v3.3.0 --clts-version=v2.3.0 lexibank_clics4.py
+$ cldfbench lexibank.makecldf --glottolog-version=v5.2.1 --concepticon-version=v3.4.0 --clts-version=v2.3.0 lexibank_clics4.py
 ```
 
 In the other case, specify the explicit locations of the repositories for Glottolog, Concepticon, and CLTS as follwo.
 
 ```
-cldfbench lexibank.makecldf --glottolog-repos=Path2Glottolog --concepticon-repos=Path2Concepticon --clts-repos=Path2Clics --glottolog-version=v5.1 --concepticon-version=v3.3.0 --clts-version=v2.3.0 lexibank_clics4.py
+cldfbench lexibank.makecldf --glottolog-repos=Path2Glottolog --concepticon-repos=Path2Concepticon --clts-repos=Path2Clics --glottolog-version=v5.2.1 --concepticon-version=v3.4.0 --clts-version=v2.3.0 lexibank_clics4.py
 ```
 
-## W4: What needs to be done before we publish CLICS4 as Version 1.0
+## W4: CLLD Version of CLICS 4
 
-This release is a CLICS⁴ dataset that we consider generally good enough with respect to the data to be used in publications (small errors would always be possible with such large numbers of data aggregated from different sources). However, we emphasize that there are a couple of shortcomings for now that we will try to handle before publishing a new version of CLICS that succeeds the current version 3.0 at https://clics.clld.org. Before publishing this new CLLD version of CLICS⁴, we will implement a new representation of the data in order to adhere to the representation of ParameterNetworks in the new [CLDF specification](https://cldf.clld.org).  
+This release is a CLICS 4 dataset that we consider generally good enough with respect to the data to be used in publications (small errors would always be possible with such large numbers of data aggregated from different sources). However, we emphasize that there are a couple of shortcomings for now that we will try to handle before publishing a new web-based version of CLICS that succeeds the current version 3.0 at https://clics.clld.org. Before publishing this new CLLD version of CLICS 4, we will implement a new representation of the data in order to adhere to the representation of ParameterNetworks in the new [CLDF specification](https://cldf.clld.org).  
 
 
 
@@ -112,8 +122,8 @@ This release is a CLICS⁴ dataset that we consider generally good enough with r
 Name               | GitHub user | Description | Role
 ---                | ---         | --- | --- 
 Annika Tjuka | @annikatjuka | maintainer | Author
-Christoph Rzymski   | @chrzyki         | maintainer   | Author
 Robert Forkel | @xrotwang | maintainer | Author
+Christoph Rzymski   | @chrzyki         | maintainer   | Author
 Johann-Mattis List | @LinguList  | maintainer | Author
 
 

cldf/README.md

@@ -1,25 +1,25 @@
 # CLDF datasets
 
-- [Wordlist: CLICS⁴](#ds-wordlistmetadatajson)
-- [StructureDataset: CLICS⁴](#ds-structuredatasetmetadatajson)
+- [Wordlist: CLICS 4](#ds-wordlistmetadatajson)
+- [StructureDataset: CLICS 4](#ds-structuredatasetmetadatajson)
 
 <a name="ds-wordlistmetadatajson"> </a>
 
-# Wordlist CLICS⁴
+# Wordlist CLICS 4
 
 **CLDF Metadata**: [Wordlist-metadata.json](./Wordlist-metadata.json)
 
 **Sources**: [sources.bib](./sources.bib)
 
 property | value
  --- | ---
-[dc:bibliographicCitation](http://purl.org/dc/terms/bibliographicCitation) | Tjuka, Annika; Forkel, Robert; Rzymski, Christoph; and List, Johann-Mattis (2025): CLICS⁴: An Improved Database of Cross-Linguistic Colexifications [Dataset, Version 0.4]. Passau: MCL Chair at the University of Passau.
+[dc:bibliographicCitation](http://purl.org/dc/terms/bibliographicCitation) | Tjuka, Annika; Forkel, Robert; Rzymski, Christoph; and List, Johann-Mattis (2025): CLICS 4: An Improved Database of Cross-Linguistic Colexifications [Dataset, Version 0.5]. Passau: MCL Chair at the University of Passau.
 [dc:conformsTo](http://purl.org/dc/terms/conformsTo) | [CLDF Wordlist](http://cldf.clld.org/v1.0/terms.rdf#Wordlist)
-[dc:identifier](http://purl.org/dc/terms/identifier) | https://clics.clld.org
+[dc:identifier](http://purl.org/dc/terms/identifier) | https://github.com/clics/clics4
 [dc:license](http://purl.org/dc/terms/license) | https://creativecommons.org/licenses/by/4.0/
-[dcat:accessURL](http://www.w3.org/ns/dcat#accessURL) | git@github.com:clics/clics4
-[prov:wasDerivedFrom](http://www.w3.org/ns/prov#wasDerivedFrom) | <ol><li><a href="git@github.com:clics/clics4/tree/dd245b2">git@github.com:clics/clics4 v0.3-10-gdd245b2</a></li><li><a href="git@github.com:glottolog/glottolog/tree/v5.2.1">Glottolog v5.2.1</a></li><li><a href="git@github.com:concepticon/concepticon-data/tree/v3.4.0">Concepticon v3.4.0</a></li><li><a href="https://github.com/cldf-clts/clts/tree/v2.3.0">CLTS v2.3.0</a></li></ol>
-[prov:wasGeneratedBy](http://www.w3.org/ns/prov#wasGeneratedBy) | <ol><li><strong>lingpy-rcParams</strong>: <a href="./lingpy-rcParams.json">lingpy-rcParams.json</a></li><li><strong>python</strong>: 3.13.3</li><li><strong>python-packages</strong>: <a href="./requirements.txt">requirements.txt</a></li></ol>
+[dcat:accessURL](http://www.w3.org/ns/dcat#accessURL) | https://github.com/clics/clics4
+[prov:wasDerivedFrom](http://www.w3.org/ns/prov#wasDerivedFrom) | <ol><li><a href="https://github.com/clics/clics4/tree/e86adf3">clics/clics4 v0.4-10-ge86adf3</a></li><li><a href="https://github.com/glottolog/glottolog/tree/v5.2.1">Glottolog v5.2.1</a></li><li><a href="https://github.com/concepticon/concepticon-data/tree/v3.4.0">Concepticon v3.4.0</a></li><li><a href="https://github.com/cldf-clts/clts/tree/v2.3.0">CLTS v2.3.0</a></li></ol>
+[prov:wasGeneratedBy](http://www.w3.org/ns/prov#wasGeneratedBy) | <ol><li><strong>lingpy-rcParams</strong>: <a href="./lingpy-rcParams.json">lingpy-rcParams.json</a></li><li><strong>python</strong>: 3.13.6</li><li><strong>python-packages</strong>: <a href="./requirements.txt">requirements.txt</a></li></ol>
 [rdf:ID](http://www.w3.org/1999/02/22-rdf-syntax-ns#ID) | clics4
 [rdf:type](http://www.w3.org/1999/02/22-rdf-syntax-ns#type) | http://www.w3.org/ns/dcat#Distribution
 
@@ -134,21 +134,21 @@ Name/Property | Datatype | Description
 
 <a name="ds-structuredatasetmetadatajson"> </a>
 
-# StructureDataset CLICS⁴
+# StructureDataset CLICS 4
 
 **CLDF Metadata**: [StructureDataset-metadata.json](./StructureDataset-metadata.json)
 
 **Sources**: [sources.bib](./sources.bib)
 
 property | value
  --- | ---
-[dc:bibliographicCitation](http://purl.org/dc/terms/bibliographicCitation) | Tjuka, Annika; Forkel, Robert; Rzymski, Christoph; and List, Johann-Mattis (2025): CLICS⁴: An Improved Database of Cross-Linguistic Colexifications [Dataset, Version 0.4]. Passau: MCL Chair at the University of Passau.
+[dc:bibliographicCitation](http://purl.org/dc/terms/bibliographicCitation) | Tjuka, Annika; Forkel, Robert; Rzymski, Christoph; and List, Johann-Mattis (2025): CLICS 4: An Improved Database of Cross-Linguistic Colexifications [Dataset, Version 0.5]. Passau: MCL Chair at the University of Passau.
 [dc:conformsTo](http://purl.org/dc/terms/conformsTo) | [CLDF StructureDataset](http://cldf.clld.org/v1.0/terms.rdf#StructureDataset)
-[dc:identifier](http://purl.org/dc/terms/identifier) | https://clics.clld.org
+[dc:identifier](http://purl.org/dc/terms/identifier) | https://github.com/clics/clics4
 [dc:license](http://purl.org/dc/terms/license) | https://creativecommons.org/licenses/by/4.0/
-[dcat:accessURL](http://www.w3.org/ns/dcat#accessURL) | git@github.com:clics/clics4
-[prov:wasDerivedFrom](http://www.w3.org/ns/prov#wasDerivedFrom) | <ol><li><a href="git@github.com:clics/clics4/tree/dd245b2">git@github.com:clics/clics4 v0.3-10-gdd245b2</a></li><li><a href="git@github.com:glottolog/glottolog/tree/v5.2.1">Glottolog v5.2.1</a></li><li><a href="git@github.com:concepticon/concepticon-data/tree/v3.4.0">Concepticon v3.4.0</a></li><li><a href="https://github.com/cldf-clts/clts/tree/v2.3.0">CLTS v2.3.0</a></li></ol>
-[prov:wasGeneratedBy](http://www.w3.org/ns/prov#wasGeneratedBy) | <ol><li><strong>python</strong>: 3.13.3</li><li><strong>python-packages</strong>: <a href="./requirements.txt">requirements.txt</a></li></ol>
+[dcat:accessURL](http://www.w3.org/ns/dcat#accessURL) | https://github.com/clics/clics4
+[prov:wasDerivedFrom](http://www.w3.org/ns/prov#wasDerivedFrom) | <ol><li><a href="https://github.com/clics/clics4/tree/e86adf3">clics/clics4 v0.4-10-ge86adf3</a></li><li><a href="https://github.com/glottolog/glottolog/tree/v5.2.1">Glottolog v5.2.1</a></li><li><a href="https://github.com/concepticon/concepticon-data/tree/v3.4.0">Concepticon v3.4.0</a></li><li><a href="https://github.com/cldf-clts/clts/tree/v2.3.0">CLTS v2.3.0</a></li></ol>
+[prov:wasGeneratedBy](http://www.w3.org/ns/prov#wasGeneratedBy) | <ol><li><strong>python</strong>: 3.13.6</li><li><strong>python-packages</strong>: <a href="./requirements.txt">requirements.txt</a></li></ol>
 [rdf:ID](http://www.w3.org/1999/02/22-rdf-syntax-ns#ID) | clics4
 [rdf:type](http://www.w3.org/1999/02/22-rdf-syntax-ns#type) | http://www.w3.org/ns/dcat#Distribution
 

cldf/StructureDataset-metadata.json

@@ -6,30 +6,30 @@
         }
     ],
     "aboutUrl": null,
-    "dc:bibliographicCitation": "Tjuka, Annika; Forkel, Robert; Rzymski, Christoph; and List, Johann-Mattis (2025): CLICS\u2074: An Improved Database of Cross-Linguistic Colexifications [Dataset, Version 0.4]. Passau: MCL Chair at the University of Passau.",
+    "dc:bibliographicCitation": "Tjuka, Annika; Forkel, Robert; Rzymski, Christoph; and List, Johann-Mattis (2025): CLICS 4: An Improved Database of Cross-Linguistic Colexifications [Dataset, Version 0.5]. Passau: MCL Chair at the University of Passau.",
     "dc:conformsTo": "http://cldf.clld.org/v1.0/terms.rdf#StructureDataset",
-    "dc:identifier": "https://clics.clld.org",
+    "dc:identifier": "https://github.com/clics/clics4",
     "dc:isVersionOf": null,
     "dc:license": "https://creativecommons.org/licenses/by/4.0/",
     "dc:related": null,
     "dc:source": "sources.bib",
-    "dc:title": "CLICS\u2074",
-    "dcat:accessURL": "git@github.com:clics/clics4",
+    "dc:title": "CLICS 4",
+    "dcat:accessURL": "https://github.com/clics/clics4",
     "prov:wasDerivedFrom": [
         {
-            "rdf:about": "git@github.com:clics/clics4",
+            "rdf:about": "https://github.com/clics/clics4",
             "rdf:type": "prov:Entity",
-            "dc:created": "v0.3-10-gdd245b2",
+            "dc:created": "v0.4-10-ge86adf3",
             "dc:title": "Repository"
         },
         {
-            "rdf:about": "git@github.com:glottolog/glottolog",
+            "rdf:about": "https://github.com/glottolog/glottolog",
             "rdf:type": "prov:Entity",
             "dc:created": "v5.2.1",
             "dc:title": "Glottolog"
         },
         {
-            "rdf:about": "git@github.com:concepticon/concepticon-data",
+            "rdf:about": "https://github.com/concepticon/concepticon-data",
             "rdf:type": "prov:Entity",
             "dc:created": "v3.4.0",
             "dc:title": "Concepticon"
@@ -44,7 +44,7 @@
     "prov:wasGeneratedBy": [
         {
             "dc:title": "python",
-            "dc:description": "3.13.3"
+            "dc:description": "3.13.6"
         },
         {
             "dc:title": "python-packages",