All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
- BundleRegistry: bundle multiple modules + hooks in a yaml registry.
- Update CDSE module to get B08 (IR)
- Change logging from name->module
- SYSU_topo dataset
- Add utility functions for dataset_id parsing
- read_timeout in fetchez.core from None->120
- Descriptions in tqdm updated to be more useful
- Allow for POST in fetch_file
- fetchez_cache records empty results
- Added PresetRegistry into registry
- Add audit_full builtin preset
- Add GSHHG module (shorelines)
- Add CUSP module (shorelines (US))
- Update recipe.py to use the new PresetRegistry
- Update nos_hydro for new API
- Update CLI for PresetRegistry
- Removed obsolete presets.py
- Added parse_source_string and parse_hook_string in utils
- Added spatial.regions_intersect from cudem.regions
- re-arranged cli help
- spatial region parsing of geojson files, returns all regions.
- Add "lidarbc" fetchez modules (canada)
- Add .fetchez_cache to save module results for re-use
- Add RecipeRegistry to registry.py
- Add --list-recipes to cli
- FetchModule is now in fetchez.modules
- fetchez.registry combines all registries (modules/hooks for now)
- Hooks are moved out of builtins to flat directory
- Now load extensions/plugins automatically
- Module metadata is prefixed with "meta_"
- Stage names: "pre" -> "manifest"; "post" -> "collection"
- Schema is now Schemas and uses global registry.
- Update to vdatum.geojson fred index of the vdatum module
- Update to fbt reading, accounting for heading; fix!
- Breaks hooks into individual files, out of topical ones.
- Hooks are now auto-detected from 'builtins', so we don't have to maintain a registry.
- Adds 'focus' and 'datatype' builtin hooks.
- The 'unzip' hook now supports tar and gz.
- Post-hooks in fetchez.core was ignoring entry changes, this fixes that.
- We now use yaml files for config, including presets
- Fixed https (now url_fetcher) module bug.
- Add 'schemas' and 'recipe'.
- All builtin hooks in hooks.builtins.
- Add documentation.
- Some new modules.
- Hook system for fetchez! (--list, --inventory, --pipe-path are now hooks)
- Users can add their own hooks in ~/.fetchez/hooks
- 'file' module to send local data through hooks
- --outdir option in CLI (global and per-module).
- Each entry now gets a 'history' key that keeps track of the hooks it passed through.
- groupded parsers in argparse
- updated pyproject.toml for optional deps.
- pyproj/pyshp error msg in dav.py
- name conflict with cudem/coned/dav
- double path.join in core fixed. (this resulted in duplicated outdirs)
- unzip hook would send a bad entry record if the unzip files already existed
- fetchez.spatial region_from_place centered on place
- Add TIGER
- Add arcticdem
- Add DAV
- fetchez.utils p_unzip from cudem.utils
- examples dir for examples, workflows, scripts using fetchez
- bing and tides examples
- sphinx auto-docs
- inventory option in the cli
- Most old fetches modules are now ported to fetchez
- README updates
- CLI description (geospatial vs elevation)
- concurent.futures testing for threads
- STOP_EVENT in fetchez.core threads
- logger uses tqdm.write to not clobber progress bars
- spatial.parse_regions will now output all the regions found in a geojson
- Initial standalone release of Fetchez.
- Decoupled from CUDEM project.
- New
fetchez.spatialmodule for lightweight region parsing. - New
fetchez.registryfor lazy module loading. - Modernized CLI with logging support.
- FRED index now uses GeoJSON and Shapely directly (removed OGR dependency).
- csb module
- fetchez.spatial 'region_to_wkt' method
- fetchez.core fetch_req now supports 'method' arg
- fetchez.spatial 'region_center' method
- buouy module
- gmrt module
- fetchez.spatial 'region_to_bbox' method
- waterservices module
- etopo module
- fetchez.spatial 'region_to_geojson_geom'
- chs module
- bluetopo module
- user plugins
- add emodnet
- Renamed project from
fetchestofetchez. - Refactored some old cudem.fetches modules to inherit from
fetchez .core.FetchModule. - In fetchez.core, allow for transparent gzip (local size is larger than remote size)