You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The following steps are necessary for creating a multi-dataset data loader for model training.
47
47
Step 1 and 2 generate persistent JSON files based on the original dataset (the `image` and `label` pairs; without the additional pseudo label or supervoxel-based label), and only need to be run once when the JSON files don't exist.
48
-
Step 3 is optional for generating overall data analysis stats.
49
48
50
49
##### 1. Generate data list JSON file
51
50
```
@@ -73,34 +72,16 @@ creates a JSON file in a format:
73
72
```
74
73
75
74
This step includes a 5-fold cross validation splitting and
76
-
some logic for 80-20 training/testing splitting.
75
+
some logic for 80-20 training/testing splitting. User need to modify the code in make_datalists.py for their own dataset. Meanwhile, the "training_transform" should manually added for each dataset.
77
76
78
77
The `original_label_dict` corresponds to the original dataset label definitions.
79
78
The `label_dict` modifies `original_label_dict` by simply rephrasing the terms.
80
79
For example in Task06, `cancer` is renamed to `lung tumor`.
81
80
The output of this step is multiple JSON files, each file corresponds
82
81
to one dataset.
83
82
84
-
85
-
##### 2. Verify data pairs and generate a global label dictionary
86
-
```
87
-
python -m data.datasets
88
-
```
89
-
90
-
This script computes a super set of labels from all the dataset JSON files.
91
-
The output of this step is a `jsons/label_dict.json` file,
92
-
representing the global label dictionary mapping, from class names to globally unique class indices (integers).
93
-
94
-
95
-
##### 3. Compute class frequencies, data transform utilities
96
-
```
97
-
python -m data.analyzer ...
98
-
```
99
-
100
-
This file (`data/analyzer.py`) contains useful transforms for reading images
101
-
and labels, converting labels from dataset-specific labels to the global labels
102
-
according to `jsons/label_dict.json`.
103
-
83
+
##### 2. Add label_dict.json and label_mapping.json
84
+
Add new class indexes to `label_dict.json` and the local to global mapping to `label_mapping.json`.
104
85
105
86
## SupverVoxel Generation
106
87
1. Download the segment anything repo and download the ViT-H weights
0 commit comments