Skip to content

Commit 138bc22

Browse files
authored
adding files for France autorefresh (#1894)
* reorg folder structure * adding autorefresh artifacts * adding autorefresh artifacts * modified download script * modified download script * adding readme file * Add newline at end of manifest.json * using util_download script * added failure exception * using utility script * modified resource limits * file renamin
1 parent dc2aa66 commit 138bc22

41 files changed

Lines changed: 888 additions & 5007 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# France Demographics Data Import
2+
3+
This directory contains the scripts and configuration to import France demographics data from [INSEE](https://www.insee.fr/en/statistiques/8333211?sommaire=8333329).
4+
5+
## Data Source
6+
- **Provider**: INSEE (Institut national de la statistique et des études économiques)
7+
- **Description**: Population estimates, age structures, and demographic components.
8+
- **URL**: [INSEE Population Statistics](https://www.insee.fr/en/statistiques/8333211?sommaire=8333329)
9+
10+
## Directory Structure
11+
- `download_input_data.py`: Script to download source Excel files from INSEE.
12+
- `*_pvmap.csv`: Property-Value mapping files for the statvar processor.
13+
- `france_demographics_metadata.csv`: Metadata configuration for the import.
14+
- `manifest.json`: Import specification.
15+
16+
## Usage
17+
18+
### 1. Download Data
19+
Run the download script to fetch the latest data files into `input_files/`.
20+
21+
```bash
22+
python3 download_input_data.py
23+
```
24+
25+
### 2. Process Data
26+
Run the `stat_var_processor.py` tool for each dataset to generate MCF and CSV files.
27+
28+
**Annual Population Components:**
29+
```bash
30+
python3 ../../tools/statvar_importer/stat_var_processor.py \
31+
--input_data='./input_files/annual_population_components.xlsx' \
32+
--pv_map='annual_population_components_pvmap.csv' \
33+
--config_file='france_demographics_metadata.csv' \
34+
--output_path='output/annual_population_components_output'
35+
```
36+
37+
**Population by Sex and Detailed Age:**
38+
```bash
39+
python3 ../../tools/statvar_importer/stat_var_processor.py \
40+
--input_data='./input_files/population_sex_detailed_age.xlsx' \
41+
--pv_map='population_sex_detailed_age_pvmap.csv' \
42+
--config_file='france_demographics_metadata.csv' \
43+
--output_path='output/population_sex_detailed_age_output'
44+
```
45+
46+
**Population by Sex and Age Groups:**
47+
```bash
48+
python3 ../../tools/statvar_importer/stat_var_processor.py \
49+
--input_data='./input_files/population_sex_age_groups.xlsx' \
50+
--pv_map='population_sex_age_groups_pvmap.csv' \
51+
--config_file='france_demographics_metadata.csv' \
52+
--output_path='output/population_sex_age_groups_output'
53+
```
54+
55+
**Average and Median Age:**
56+
```bash
57+
python3 ../../tools/statvar_importer/stat_var_processor.py \
58+
--input_data='./input_files/average_median_age.xlsx' \
59+
--pv_map='average_median_age_pvmap.csv' \
60+
--config_file='france_demographics_metadata.csv' \
61+
--output_path='output/average_median_age_output'
62+
```

0 commit comments

Comments
 (0)