Merge pull request #166 from vlahm/master

mostly EML updates

Author: westonslaughter

.RData

@@ -27,4 +27,7 @@ data/general/nhd_hr
 output/
 old_dataset_dirs.tar.gz
 old_logs
+eml/data_links
+eml/eml_out
 vault/*
+.Rdata

.gitignore

@@ -0,0 +1,46 @@
+Glossary of Terms (*as used in this documentation)
+
+watershed
+    All land area contributing runoff to a point of interest along a stream.
+    Does not necessarily account for inputs from subsurface flow or human-constructed diversions.
+    We avoid the terms "catchment" and "basin," though they are sometimes used in this way.
+site*
+    An individual gauging station or stream sampling location and its watershed.
+domain*
+    One or more sites under common management.
+network*
+    One or more domains under common funding/leadership.
+provider*
+    Also sometimes referred to as a "source" -- a primary source of data assimilated into MacroSheds.
+    May be a network, domain, or third party.
+product*
+    A collection of data, possibly including multiple datasets/tables. Providers may separate products by
+    temporal extent/interval, scientific category, detection method, and/or sampling location.
+prodname
+    One of the 7 core product categories included in the MacroSheds dataset. See below.
+prodcode
+    An alphanumeric string associated with a product. Providers have their own. MacroSheds uses its own scheme internally.
+    This won't be relevant for most users, but see detailed documentation included with core data downloads for more information.
+site-product, site-year, etc.
+    Terms like these are used to designate various subdivisions of the overall MacroSheds dataset.
+    A site-product, for example, is the collection of all data for a single MacroSheds product, available at a single site.
+
+MacroSheds data are organized into the following products:
+
+discharge
+    Streamflow; water volume over time; reported in L/s.
+stream chemistry
+    Concentration of chemical constituents in stream water; reported in mg/L or mEq/L.
+stream flux
+    Mass of chemical constituents in stream water, per watershed area, over time; reported in kg/ha/d.
+    (not currently included with this dataset, but can be generated via the macrosheds package for R)
+precipitation
+    Rainfall, snowfall, or both combined; reported per watershed in mm.
+precipitation chemistry
+    Concentration of chemical constituents in precipitation; reported in mg/L or mEq/L; averaged across watershed area.
+precipitation flux
+    Mass of chemical constituents in precipitation, per watershed area, over time; reported in kd/ha/d.
+    (not currently included with this dataset, but can be generated via the macrosheds package for R)
+watershed attributes
+    Areal watershed summary statistics, variables available are common to all MacroSheds sites.
+

eml/additional_info.txt

@@ -0,0 +1 @@
+There exist hundreds of long-term watershed ecosystem monitoring efforts from which solute fluxes can be calculated. While details of instrumentation and sampling methods vary between studies, the types of data collected and the questions that motivate their analysis are often remarkably similar. Nevertheless, little effort toward the compilation of these datasets has previously been made, and comparative watershed analyses have remained limited in scale. The MacroSheds project has developed a flexible, future-friendly system for continually harmonizing daily time series of streamflow, precipitation, and solute chemistry from 168+ watershed studies throughout the U.S. and beyond, and supplementing each with a comprehensive set of predictive watershed attributes. The MacroSheds dataset is an unprecedented resource for watershed ecosystem science, and for hydrology, as a small-watershed supplement to existing collections of streamflow predictors, like CAMELS and GAGES-II. Macrosheds is accompanied by a web dashboard for visualization (macrosheds.org) and an R package (https://github.com/MacroSHEDS/macrosheds) for local analysis. 

eml/eml_templates/abstract.txt

@@ -0,0 +1,40 @@
+-----
+Glossary of Terms (*as used in this documentation)
+-----
+
+watershed -  All land area contributing runoff to a point of interest along a stream. Does not necessarily account for inputs from subsurface flow or human-constructed diversions. We avoid the terms "catchment" and "basin," though they are sometimes used in this way.
+
+site* - An individual gauging station or stream sampling location and its watershed.
+
+domain* - One or more sites under common management.
+
+network* - One or more domains under common funding/leadership.
+
+provider* - Also sometimes referred to as a "source" -- a primary source of data assimilated into MacroSheds. May be a network, domain, or third party.
+
+product* - A collection of data, possibly including multiple datasets/tables. Providers may separate products by temporal extent/interval, scientific category, detection method, and/or sampling location.
+
+prodname - One of the 7 core product categories included in the MacroSheds dataset. See below.
+
+prodcode - An alphanumeric string associated with a product. Providers have their own. MacroSheds uses its own scheme internally.  This won't be relevant for most users, but see detailed documentation included with core data downloads for more information.
+
+site-product, site-year, etc. - Terms like these are used to designate various subdivisions of the overall MacroSheds dataset. A site-product, for example, is the collection of all data for a single MacroSheds product, available at a single site.
+
+-----
+MacroSheds data are organized into the following products:
+-----
+
+discharge - Streamflow; water volume over time; reported in L/s.
+
+stream chemistry - Concentration of chemical constituents in stream water; reported in mg/L or mEq/L.
+
+stream flux - Mass of chemical constituents in stream water, per watershed area, over time; reported in kg/ha/d. (not currently included with this dataset, but can be generated via the macrosheds package for R)
+
+precipitation - Rainfall, snowfall, or both combined; reported per watershed in mm.
+
+precipitation chemistry - Concentration of chemical constituents in precipitation; reported in mg/L or mEq/L; averaged across watershed area.
+
+precipitation flux - Mass of chemical constituents in precipitation, per watershed area, over time; reported in kd/ha/d. (not currently included with this dataset, but can be generated via the macrosheds package for R)
+
+watershed attributes - Areal watershed summary statistics, variables available are common to all MacroSheds sites.
+

eml/eml_templates/additional_info.txt

@@ -0,0 +1,11 @@
+"attributeName"	"attributeDefinition"	"class"	"unit"	"dateTimeFormatString"	"missingValueCode"	"missingValueCodeExplanation"
+"date"	"date as UTC timestamp with 00:00:00 time"	"Date"		"Y-M-DTh:m:sZ"		
+"site_code"	"Short name for MacroSheds site. See site_metadata.csv"	"character"				
+"dayl(s)"	"Watershed average seconds of daylight"	"numeric"	"second"		"NA"	"missing value"
+"prcp(mm/day)"	"Watershed average precipitation"	"numeric"	"millimetersPerDay"		"NA"	"missing value"
+"srad(W/m2)"	"Watershed average solar radiation"	"numeric"	"wattPerMeterSquared"		"NA"	"missing value"
+"swe(mm)"	"Watershed average snow-water equivalent"	"numeric"	"millimeter"		"NA"	"missing value"
+"tmax(C)"	"Watershed average maximum air temperature"	"numeric"	"celsius"		"NA"	"missing value"
+"tmin(C)"	"Watershed average minimum air temperature"	"numeric"	"celsius"		"NA"	"missing value"
+"vp(Pa)"	"Watershed average vapor pressure"	"numeric"	"pascal"		"NA"	"missing value"
+"pet(mm)"	"Watershed average potential evapotranspiration"	"numeric"	"millimeter"		"NA"	"missing value"

eml/eml_templates/attributes_CAMELS_compliant_Daymet_forcings.txt

@@ -0,0 +1,34 @@
+"attributeName"	"attributeDefinition"	"class"	"unit"	"dateTimeFormatString"	"missingValueCode"	"missingValueCodeExplanation"
+"site_code"	"Short name for MacroSheds site. See site_metadata.csv"	"character"				
+"p_mean"	"mean daily precipitation 1989-10-01 to 2009-09-30 (Daymet)"	"numeric"	"millimeterPerDay"		"NA"	"missing value"
+"pet_mean"	"mean daily PET (estimated using Priestley-Taylor formulation with gridded alpha product from Aschonitis et al. 2017) (Daymet)"	"numeric"	"millimeterPerDay"		"NA"	"missing value"
+"aridity"	"aridity (ratio of pet_mean to p_mean) (Daymet)"	"numeric"	"dimensionless"		"NA"	"missing value"
+"p_seasonality"	"seasonality and timing of precipitation (estimated using sine curves to represent the annual temperature and preciptiation cycles, positive [negative] values indicate that precipitation peaks in summer [winter], values close to 0 indicate uniform precipitation throughout the year) (Daymet)"	"numeric"	"dimensionless"		"NA"	"missing value"
+"frac_snow"	"fraction of precipitation falling as snow (i.e., on days colder than 0°C) (Daymet)"	"numeric"	"dimensionless"		"NA"	"missing value"
+"high_prec_freq"	"frequency of high precipitation days ( >= 5 times mean daily precipitation) (Daymet)"	"numeric"	"daysPerYear"		"NA"	"missing value"
+"high_prec_dur"	"average duration of high precipitation events (number of consecutive days >= 5 times mean daily precipitation) (Daymet)"	"numeric"	"day"		"NA"	"missing value"
+"high_prec_timing"	"season during which most high precipitation days ( >= 5 times mean daily precip.) occur (Daymet)"	"categorical"			"NA"	"missing value"
+"low_prec_freq"	"frequency of dry days ( <1 mm/day) (Daymet)"	"numeric"	"daysPerYear"		"NA"	"missing value"
+"low_prec_dur"	"average duration of dry periods (number of consecutive days <1 mm/day) (Daymet)"	"numeric"	"day"		"NA"	"missing value"
+"low_prec_timing"	"season during which most dry days ( <1 mm/day) occur (Daymet)"	"categorical"			"NA"	"missing value"
+"geol_1st_class"	"most common geologic class in the catchment (GLiM)"	"categorical"			"NA"	"missing value"
+"glim_1st_class_frac"	"fraction of the catchment area associated with its most common geologic class (GLiM)"	"numeric"	"dimensionless"		"NA"	"missing value"
+"geol_2nd_class"	"2nd most common geologic class in the catchment (GLiM)"	"categorical"			"NA"	"missing value"
+"glim_2nd_class_frac"	"fraction of the catchment area associated with its 2nd most common geologic class (GLiM)"	"numeric"	"dimensionless"		"NA"	"missing value"
+"carbonate_rocks_frac"	"fraction of the catchment area characterized as ""Carbonate sedimentary rocks"" (GLiM)"	"numeric"	"dimensionless"		"NA"	"missing value"
+"geol_porosity"	"subsurface porosity (GLHYMPS)"	"numeric"	"dimensionless"		"NA"	"missing value"
+"geol_permeability"	"subsurface permeability (log10) (GLHYMPS)"	"numeric"	"squareMeter"		"NA"	"missing value"
+"sand_frac"	"sand fraction (of the soil material smaller than 2 mm, layers marked as oragnic material, water, bedrock and ""other"" were excluded) (gSSURGO)"	"numeric"	"percent"		"NA"	"missing value"
+"silt_frac"	"silt fraction (of the soil material smaller than 2 mm, layers marked as oragnic material, water, bedrock and ""other"" were excluded) (gSSURGO)"	"numeric"	"percent"		"NA"	"missing value"
+"clay_frac"	"clay fraction (of the soil material smaller than 2 mm, layers marked as oragnic material, water, bedrock and ""other"" were excluded) (gSSURGO)"	"numeric"	"percent"		"NA"	"missing value"
+"organic_frac"	"fraction of soil_depth_statsgo marked as organic material (class 13) (gSSURGO)"	"numeric"	"percent"		"NA"	"missing value"
+"gauge_lat"	"gauge latitude "	"numeric"	"degreesNorth"		"NA"	"missing value"
+"gauge_lon"	"gauge longitude"	"numeric"	"degreesEast"		"NA"	"missing value"
+"area"	"watershed area"	"numeric"	"squareKilometers"		"NA"	"missing value"
+"elev_mean"	"watershed mean elevation"	"numeric"	"meter"		"NA"	"missing value"
+"slope_mean"	"watershed mean slope"	"numeric"	"metersPerKilometer"		"NA"	"missing value"
+"frac_forest"	"forest fraction"	"numeric"	"!Add units here!"		"NA"	"missing value"
+"dom_land_cover_frac"	"fraction of the catchment area associated with the dominant land cover"	"numeric"	"dimensionless"		"NA"	"missing value"
+"dom_land_cover"	"dominant land cover type (Noah-modified 20-category IGBP-MODIS land cover)"	"categorical"			"NA"	"missing value"
+"root_depth_50"	"root depth (percentile 50% extracted from a root depth distribution based on IGBP land cover) (MODIS)"	"numeric"	"meter"		"NA"	"missing value"
+"root_depth_99"	"root depth (percentile 99% extracted from a root depth distribution based on IGBP land cover) (MODIS)"	"numeric"	"meter"		"NA"	"missing value"

eml/eml_templates/attributes_CAMELS_compliant_ws_attr.txt

@@ -0,0 +1,10 @@
+"attributeName"	"attributeDefinition"	"class"	"unit"	"dateTimeFormatString"	"missingValueCode"	"missingValueCodeExplanation"
+"network"	"Short name for the data network associated with a MacroSheds site. A network includes one or more domains under common funding/leadership."	"character"				
+"domain"	"Short name for the data domain associated with a MacroSheds site. A domain includes one or more sites under common management."	"character"				
+"site_name"	"May be a site_code or a stream_name. Cross-reference with sites.csv."	"character"				
+"start_date_of_concern"	"The first date on which the irregularity is applicable. May be “whole_record”"	"character"				
+"end_date_of_concern"	"The last date on which the irregularity is applicable. May be “whole_record”"	"character"				
+"macrosheds_product_affected"	"The MacroSheds data product to which this irregularity pertains"	"character"				
+"macrosheds_variables_affected"	"The MacroSheds variable(s) to which this irregularity pertains"	"character"				
+"concern"	"Details of the irregularity"	"character"				
+"included_in_current_dataset"	"If FALSE, this irregularity is not yet present in the published MacroSheds dataset."	"character"				

eml/eml_templates/attributes_data_irregularities.txt

@@ -0,0 +1,11 @@
+"attributeName"	"attributeDefinition"	"class"	"unit"	"dateTimeFormatString"	"missingValueCode"	"missingValueCodeExplanation"
+"domain"	"Short name for the data domain associated with a MacroSheds site. A domain includes one or more sites under common management."	"character"				
+"prodcode"	"One or more MacroSheds product codes, separated by |. These codes are mostly for internal use, but they may be relevant for distinguishing primary data sources here, e.g. if a detection limit differs between stream and precip chemistry within the same domain."	"character"				
+"variable_converted"	"The MacroSheds variable code after it has been converted (only relevant for molecule-to-atomic-constituent conversions, e.g. NO3 to NO3-N)"	"character"				
+"variable_original"	"The MacroSheds variable code before any molecule-to-atomic-constituent conversions have taken place (e.g. NO3 to NO3-N)"	"character"				
+"detection_limit_converted"	"The detection limit after conversion to MacroSheds standard units (see variables_time_series.csv)"	"numeric"	"variable_unit"			
+"detection_limit_original"	"The detection limit before conversion to MacroSheds standard units (see variables_time_series.csv)"	"numeric"	"variable_unit"			
+"unit_converted"	"The units of the detection limit after conversion to MacroSheds standard units (see variables_time_series.csv)"	"character"				
+"unit_original"	"The units of the detection limit before conversion to MacroSheds standard units (see variables_time_series.csv)"	"character"				
+"start_date"	"The first known date on which the detection limit is applicable"	"Date"		"Y-M-D"		
+"end_date"	"The last known date on which the detection limit is applicable"	"Date"		"Y-M-D"		

eml/eml_templates/attributes_detection_limits.txt

@@ -0,0 +1,6 @@
+"attributeName"	"attributeDefinition"	"class"	"unit"	"dateTimeFormatString"	"missingValueCode"	"missingValueCodeExplanation"
+"variable_code"	"MacroSheds variable short name"	"character"				
+"variable_name"	"Full name of MacroSheds variable"	"character"				
+"unit"	"The standard unit of the variable within the MacroSheds dataset"	"character"				
+"range_check_minimum"	"The minimum value that is allowed through the range filter"	"numeric"	"variable_unit"			
+"range_check_maximum"	"The maximum value that is allowed through the range filter"	"numeric"	"variable_unit"