Skip to content

Commit b195f0b

Browse files
committed
update shapefile docs
1 parent 3a741da commit b195f0b

2 files changed

Lines changed: 92 additions & 44 deletions

File tree

docs/CHANGELOG.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,36 @@ The list of changes is generated automatically based on the commit messages them
1414

1515
All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.
1616

17+
## [3.0.0](https://github.com/simwrapper/simwrapper/compare/v2.5.4...v3.0.0) (2023-12-06)
18+
19+
20+
### ⚠ BREAKING CHANGES
21+
22+
* Project sites which use custom.css, header.md, and
23+
footer.md should review their dashboards to ensure they still fit with
24+
the new layout and UI design. In many cases, custom.css is now
25+
completely unnecessary, as styling can now be included in
26+
simwrapper-config.yaml
27+
28+
Merge branch 'staging'
29+
30+
### Features
31+
32+
* Dashboard subtabs ([b02322e](https://github.com/simwrapper/simwrapper/commit/b02322e3a6255fdc29594f06bd14d41d5cd37e2a))
33+
* Improved file navigation with breadcrumbs, and better dashboard layout ([2fbf085](https://github.com/simwrapper/simwrapper/commit/2fbf0854d6df62acc690bd4cfc973ac807fdcf2c))
34+
* New "grid map" visualization for point-data ([65e5f7d](https://github.com/simwrapper/simwrapper/commit/65e5f7d472a5c3a3e8ec59669d69a0d34b937335))
35+
* Pinned favorite folders ([c015ef3](https://github.com/simwrapper/simwrapper/commit/c015ef31ba6cd3422cd189e942c38e52d7886742))
36+
* Project site left-side navigation and dashboard layout improvements ([14f528d](https://github.com/simwrapper/simwrapper/commit/14f528d19f5f2752fbf0c4aee6b987748802e146))
37+
* Subtabs YAML update for dashboard sections with subtabs ([1dda4b3](https://github.com/simwrapper/simwrapper/commit/1dda4b3a676d1917818bfb8230b077b1598a7acb))
38+
* Top navigation bar for project pages via simwrapper-config.yaml ([dd51785](https://github.com/simwrapper/simwrapper/commit/dd517859dbf076ad131de46561acb3cff3a2cfec))
39+
* Updated UI, cleaner dashboard looks, new project site features ([6e71517](https://github.com/simwrapper/simwrapper/commit/6e71517b4f13d8fbfa93edb5a8d26a4f9654a479))
40+
41+
42+
### Bug Fixes
43+
44+
* **map:** Map doesn't consistently switch themes ([ccc09c7](https://github.com/simwrapper/simwrapper/commit/ccc09c7557510c18346ba5d55cd8938278224bb7)), closes [#284](https://github.com/simwrapper/simwrapper/issues/284)
45+
* **text:** Text/Markdown panels could be the wrong size ([ce44e1b](https://github.com/simwrapper/simwrapper/commit/ce44e1be8bbf8702f5b16e2c47b11b2d2340ca97))
46+
1747
### [2.5.4](https://github.com/simwrapper/simwrapper/compare/v2.5.3...v2.5.4) (2023-11-17)
1848

1949

docs/ref-shapefiles.md

Lines changed: 62 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -32,11 +32,11 @@ Properties are written in either a standalone `viz-map*.yaml` file, or in a dash
3232
## Properties
3333

3434

35-
### Dashboard-specific properties
35+
### Dashboard panel properties
3636

3737
|Property | Usage|
3838
| --- | --- |
39-
| `type` | In `dashboard-*.yaml` config files, MUST be set to **"map"** |
39+
|`type` | MUST be set to **"map"** in `dashboard-*.yaml` config files |
4040
|`height`| Relative height. Larger numbers create taller panels. (default: 5) |
4141
|`width`| Relative width. The widths of all panels on a single row are summed, and the layout of each panel is then relative to that total width. (default: 1) |
4242

@@ -52,41 +52,57 @@ Properties are written in either a standalone `viz-map*.yaml` file, or in a dash
5252
| `pitch`| Map pitch (default: 0) |
5353
| `bearing`| Map bearing/direction (default:0) |
5454

55-
### **shapes:** the boundaries/areas to be drawn
56-
57-
There are **two separate data types** required for an area map: the boundaries/shapes, and one the dataset (unless the shapefile self-contains all of the required data).
55+
---
5856

59-
Both files must contain an matching identification column in order to **join the two datasets** together. In other words, the boundary IDs must be present (somewhere) in both datafiles. The names of the columns do not have to be identical, but it helps legibility. See below.
57+
### **"shapes:"** the features (links, areas, etc) to be drawn
6058

6159
```yaml
6260
shapes:
6361
file: my-taz-shapefile.shp
6462
join: id
63+
keep: AB,LENGTH,TAZ,SQMILES
6564
```
65+
There are **two separate input types** required for a shapefile/geojson map: (1) the boundaries or shapes themselves, which can be geojson or shapefile format; and (2) additional datasets in CSV/DBF tabular format which contain columns of data that can be attached to the features. These datasets are optional, if the shapefile self-contains all of the required data.
66+
67+
All inputs must contain an _identification column_ in order to **join the datasets** together. In other words, the feature IDs must be present in both datafiles. The names of the columns do not have to be identical, but it helps legibility. See below.
68+
69+
70+
**shapes** section can have the following properties:
6671
67-
Contains two subentries:
72+
|Property | Usage|
73+
| --- | --- |
74+
|`file` | String. The filepath of the feature dataset. May include wildcards \* and ?. File can be in _geojson_ format, or a _shapefile_. File type is determined by the filename extension, which must end in either `.geojson` or `.shp` When loading shapefiles, an identically-named `.dbf` and `.prj` file will also be read from the same folder. Be sure to supply a .prj file containing a valid EPSG code if your data is not in lat/long format!|
75+
|`join` | String. The name of the property containing unique shape IDs, or set to `id` if it is in the id field of the geojson itself.|
76+
|`keep` | String (optional). Comma-separated list of fields to KEEP in memory from the shapefile properties. Some files are very large with many properties that you are not interested in. Use the `keep` property to list the only ones you actually need, in order to save memory and improve performance. If `keep` is not specified then all properties are retained.|
6877

69-
- **file:** String. The filepath containing the data. May include wildcards \* and ?. File can be in _geojson_ format, or a _shapefile_. File type is determined by suffix, so must end in either `.geojson` or `.shp` When loading shapefiles, an identically-named .dbf and .prj file will also be read from the same folder. Be sure to supply a .prj file containing a valid EPSG code if your data is not in lat/long format.
70-
- **join:** String. The name of the data column containing shape IDs, or 'id' if it is in the id field of geojson.
7178

72-
### **datasets:** the dataset to be joined to the shapefile
79+
### **datasets:** datasets to be joined to the features
7380

7481
```yaml
7582
datasets:
83+
population: "../pop/2025/population.csv"
7684
transit-trips:
7785
file: .summaries/transit-outputs.csv
78-
join: TAZ
86+
drop: bike,ped,MTCTAZ
87+
keep: TAZ,TIME,pm_vol,orig,dest
7988
```
8089

81-
Contains an object naming the dataset and providing its filename and join column:
90+
`datasets:` defines any additional tabular datasets that are linked to the feature dataset from above. Every dataset is defined by a short keyword name followed by a colon, and then the needed properties are provided. In the example above, `population` and `transit-trips` keywords are defined. Use the keyword to refer to the dataset when defining colors and widths (see below).
91+
92+
- In simple cases a filename can be given directly, as in the `population` example above.
93+
- Other properties can be defined to keep/drop certain columns, as in `transit-trips`.
94+
95+
**name of dataset:** Give the dataset a simple name, which will be used in the display settings below. e.g. `tazdata:`
8296

83-
- **name of dataset:** Give the dataset a simple name, which will be used in the display settings below. e.g. `tazdata`
84-
- **file:** String. The filepath containing the data. May include wildcards \* and ?. File can be in _CSV or DBF_ format. Any filename not ending in `.dbf` will be parsed as a CSV file, using either commas, tabs, or spaces as delimiters.
85-
- **join:** String. The name of the data column containing the shape IDs for joining.
97+
Each subsection can have the following properties:
8698

87-
### **display:** define the color and value details
99+
|Property | Usage|
100+
| --- | --- |
101+
|`file`| String. The filepath containing the data. May include wildcards \* and ?. File can be in _CSV or DBF_ format. Any filename not ending in `.dbf` will be parsed as a CSV file, using either commas, tabs, or spaces as delimiters.|
102+
|`drop`| String (optional). Comma-separated list of column names to be dropped from memory; these columns will be unavailable to this visualization. This can save memory and improve performance.|
103+
|`keep`| String (optional). Comma-separated list of column names to be kept in memory for this visualization. This can save memory and improve performance.|
88104

89-
For area maps, the `fill` section defines the color fill, and is (currently) the only section that is required. At a later date we may include borders, etc.
105+
### **display:** define the symbology details: colors, widths, etc
90106

91107
```yaml
92108
display:
@@ -101,38 +117,49 @@ display:
101117
steps: 5
102118
```
103119

104-
**dataset:** Name of the dataset from above which includes the data.
120+
The `display` section is where you define the details of your map symbology: the line and fill colors, line widths, circle radii, difference calculations, and so on.
105121

106-
**filters:** (optional) List of any columns which can be used as category filters by the user interactively. Note that _active filters_ will be shown in the URL bar, so curated maps can be shared via URL.
122+
Different map types allow different properties. This is an attempt at summarizing the capabilities; not all combinations work for every map type. There are many, many configuration options listed here, and they mimic the user interface options available in the configuration panel.
107123

108-
**columnName:** The column name (or names) containing values to be plotted. If multiple rows have a matching shape ID, all values will be summed together. (Other stats to be added)
124+
**Instead of trying to write this YAML from scratch,** try creating a map interactively on the SimWrapper site, and then choosing `EXPORT` from the configuration panel. SimWrapper will produce a YAML file with most/all of your settings and save it in your browser's Downloads folder. You can move this to your data folder and modify as necessary.
109125

110-
**colorRamp:** Describe the colors themselves:
126+
|Property | Usage|
127+
| --- | --- |
128+
|`dataset`| Name of the dataset from above which includes the data.|
129+
|`columnName`| The column name (or names) containing values to be plotted. If multiple rows have a matching shape ID, all values will be summed together.|
130+
|`colorRamp`| Describe the colors to be used:<br/>**ramp:** Name of the color ramp to use.<br/>Sequential: `Viridis` `Plasma` `Blues` `Greens` `Purples` `Oranges`<br/>Diverging: `PRGn` `RdBu`<br/>Categorical: `Tableau10` `Paired`. Note categoricals only have ten or twelve categories.|
131+
| | **reversed:** true/false|
132+
| | **steps:** Number of steps in the ramp.|
133+
| | **exponentColors:** Optional true/false. If true, values will be scaled exponentially before being drawn. This is often useful if values are concentrated in small areas, and much higher in value than in typical areas.|
134+
| | **diff:** Example `col1 - col2` will activate diff mode|
135+
| | **breakpoints:** Work in progress, comma-separated list of manual breakpoints for data values.|
111136

112-
- **ramp:** Name of the color ramp to use.
113-
- Sequential: `Viridis` `Plasma` `Blues` `Greens` `Purples` `Oranges`
114-
- Diverging: `PRGn` `RdBu`
115-
- Categorical: `Tableau10` `Paired`. Note categoricals only have ten or twelve categories.
116-
- **reversed:** true/false
117-
- **steps:** Number of steps in the ramp.
118-
- **exponentColors:** Optional true/false. If true, values will be scaled exponentially before being drawn. This is often useful if values are concentrated in small areas, and much higher in value than in typical areas.
119-
- **diff:** Example `col1 - col2` will activate diff mode
120-
- _(todo: more examples needed )_
137+
<!-- |`filters`| (optional) List of any columns which can be used as category filters by the user interactively. Note that _active filters_ will be shown in the URL bar, so curated maps can be shared via URL.|
138+
-->
121139

122-
### filters
123140

124-
In the `filter` section, you can filter dataset with multiple expressions on numerical columns in any dataset. Keys are of the format `dataset.column: "filter"` and filter can be `==, !=, <, <=, >, >=`
141+
### filters
125142

126143
```yaml
127144
filters:
128145
dataset1.trips: "> 0"
129146
shapes.TAZ: "!=0"
130147
```
131148

132-
_TODO: more filter examples needed_
149+
In the `filter` section, you can filter dataset with multiple expressions on numerical columns in any dataset. Keys are of the format `dataset.column: "filter"` and filter can be `==, !=, <, <=, >, >=`
150+
151+
_More filter examples needed!_
133152

134153
### Tooltip
135154

155+
```yaml
156+
tooltip:
157+
- AM:TOT_VOL
158+
- AM:TNC
159+
- freeflow.shp:FACILITY_TYPE
160+
- freeflow.shp:SPEED
161+
```
162+
136163
By default, the tooltip shows all columns in the shapefile, as well as any columns that are actively being displayed as either a color or a width.
137164

138165
You can customize the tooltip to just show what you are interested as follows:
@@ -141,15 +168,6 @@ You can customize the tooltip to just show what you are interested as follows:
141168
- Each entry is of format `datasetname:columnname`, so for example `AM_FLOWS:VEHICLE_VOL` will display the AM_FLOWS dataset and VEHICLE_VOL column.
142169
- Use the shapefile/network filename for its columnar data, or the dataset "key" for joined datasets.
143170

144-
Custom tooltip example:
145-
146-
```yaml
147-
tooltip:
148-
- AM:TOT_VOL
149-
- AM:TNC
150-
- freeflow.shp:FACILITY_TYPE
151-
- freeflow.shp:SPEED
152-
```
153171

154172
## Visualization hints
155173

@@ -174,10 +192,10 @@ shapes:
174192
datasets:
175193
transit-trips:
176194
file: .dashboard/transit-data.csv
177-
join: geoid
178195
display:
179196
fill:
180197
dataset: transit-trips
198+
join: geoid
181199
filters: operator, income
182200
columnName: trip_origins, trip_boards, trip_reslocs
183201
colorRamp:
@@ -206,10 +224,10 @@ layout:
206224
datasets:
207225
transit-trips:
208226
file: .dashboard/transit-data.csv
209-
join: geoid
210227
display:
211228
fill:
212229
dataset: transit-trips
230+
join: geoid
213231
filters: operator, income
214232
columnName: trip_origins, trip_boards, trip_reslocs
215233
colorRamp:

0 commit comments

Comments
 (0)