Skip to content

toml.rst

Latest commit

 

History

History
660 lines (485 loc) · 26.6 KB

toml.rst

File metadata and controls

660 lines (485 loc) · 26.6 KB

Simulation settings

.. currentmodule:: muse

This section details the TOML input for MUSE. The format for TOML files is described in this :ref:`previous section <toml-primer>`. Here, however, we focus on sections and attributes that are specific to MUSE.

The TOML file can be read using :py:func:`~readers.toml.read_settings`. The resulting data is used to construct the market clearing algorithm directly in the :py:meth:`MCA's factory function <mca.MCA.factory>`.

Main section

This is the topmost section. It contains settings relevant to the simulation as a whole.

time_framework = [2020, 2025, 2030, 2035, 2040, 2045, 2050]
regions = ["USA"]
interest_rate = 0.1
interpolation_mode = 'linear'
log_level = 'info'

equilibrium_variable = 'demand'
maximum_iterations = 100
tolerance = 0.1
tolerance_unmet_demand = -0.1
time_framework
Required. List of years for which the simulation will run.
region
Subset of regions to consider. If not given, defaults to all regions found in the simulation data.
interpolation_mode

interpolation when reading the initial market. One of linear, nearest, zero, slinear, quadratic, cubic. Defaults to linear.

  • linear interpolates using a linear function.
  • nearest uses nearest-neighbour interpolation. That is, the closest known data point is used as the prediction for the data point to interpolate.
  • zero assumes that the data point to interpolate is zero.
  • slinear uses a spline of order 1. This is a similar method to linear interpolation.
  • quadratic uses a quadratic equation for interpolation. This should be used if you expect your data to take a quadratic form.
  • cubic interpolation is similar to quadratic interpolation, but uses a cubic function for interpolation.
log_level
verbosity of the output. Valid options, from the highest to the lowest level of verbosity, are: "DEBUG", "INFO", "WARNING", "ERROR", "CRITICAL".
equilibirum_variable
whether equilibrium of demand or prices should be sought. Defaults to demand.
maximum_iterations
Maximum number of iterations when searching for equilibrium. Defaults to 100.
tolerance
Tolerance criteria when checking for equilibrium. Defaults to 0.1. 0.1 signifies that 10% of a deviation is allowed among the iterative value of either demand or price over a year per region.
tolerance_unmet_demand
Criteria checking whether the demand has been met. Defaults to -0.1.
excluded_commodities
List of commodities excluded from the equilibrium considerations.
plugins
Path or list of paths to extra python plugins, i.e. files with registered functions such as :py:meth:`~muse.outputs.register_output_quantity`.

Carbon market

This section contains the settings related to the modelling of the carbon market. If omitted, it defaults to not including the carbon market in the simulation.

Example

[carbon_budget_control]
budget = []
budget
Yearly budget. There should be one item for each year the simulation will run. In other words, if given and not empty, this is a list with the same length as time_framework from the main section. If not given or an empty list, then the carbon market feature is disabled. Defaults to an empty list.
commodities
Commodities that make up the carbon market. Defaults to an empty list.
control_undershoot
Whether to control carbon budget undershoots. This parameter allows for carbon tax credit from one year to be passed to the next in the case of less carbon being emitted than the budget. Defaults to False.
control_overshoot
Whether to control carbon budget overshoots. If the amount of carbon emitted is above the carbon budget, this parameter specifies whether this deficit is carried over to the next year. Defaults to False.
method

Method used to equilibrate the carbon market. Available options are fitting and bisection, however this can be expanded with the @register_carbon_budget_method hook in muse.carbon_budget.

These methods solve the market with a number of different carbon prices, aiming to find the carbon price at which emissions (pooled across all regions) are equal to the carbon budget. The obtained carbon price applies to all regions.

The fitting method samples a number of different carbon prices to build a regression model (linear or exponential) of emissions as a function of carbon price. This regression model is then used to estimate the carbon price at which the carbon budget is met.

The bisection method uses an iterative approach to settle on a carbon price. Starting with a lower and upper-bound carbon price, it iteratively halves this price interval until the carbon budget is met to within a user-defined tolerance, or until the maximum number of iterations is reached. Generally, this method is more robust for markets with a complex, nonlinear relationship between emissions and carbon price, but may be slower to converge than the fitting method.

Defaults to bisection.

method_options

Additional options for the specified carbon method.

Parameters for the bisection method:

  • max_iterations: maximum number of iterations. Defaults to 5.
  • tolerance: tolerance for convergence. E.g. 0.1 means that the algorithm will terminate when emissions are within 10% of the carbon budget. Defaults to 0.1.
  • early_termination_count: number of iterations with no change in the carbon price before the algorithm will terminate. Defaults to 5.

Parameters for the fitting method:

  • fitter: the regression model used to approximate model emissions. Predefined options are linear (default) and exponential. Further options can be defined using the @register_carbon_budget_fitter hook in muse.carbon_budget.
  • sample_size: number of price samples used. Defaults to 5.

Shared parameters:

  • refine_price: If True, applies an upper limit on the carbon price. Defaults to False.
  • price_too_high_threshold: upper limit on the carbon price. Defaults to 10.
  • resolution: Number of decimal places to solve the carbon price to. When using the bisection method, increasing this value may increase the time taken to solve the carbon market. Defaults to 2.

Global input files

Defines the paths specific simulation data files. The paths can be formatted as explained in the :ref:`toml-primer`.

[global_input_files]
projections = '{path}/inputs/Projections.csv'
regions = '{path}/inputs/Regions.csv'
global_commodities = '{path}/inputs/MUSEGlobalCommodities.csv'
projections
Path to a csv file giving initial market projection. See :ref:`inputs-projection`.
global_commodities
Path to a csv file describing the comodities in the simulation. See :ref:`inputs-commodities`.
base_year_import
Optional. Path to a file describing fixed amounts of commodities imported by region. A CSV file containing the consumption in the same format as :ref:`inputs-projection`.
base_year_export
Optional. Path to a file describing fixed amounts of commodities imported. A CSV file containing the consumption in the same format as :ref:`inputs-projection`.

Timeslices

Time-slices represent a sub-year disaggregation of commodity demand. Generally, timeslices are expected to introduce several levels, e.g. season, day, or hour. The simplest is to show the TOML for the default timeslice:

[timeslices]
winter.weekday.night = 396
winter.weekday.morning = 396
winter.weekday.afternoon = 264
winter.weekday.early-peak = 66
winter.weekday.late-peak = 66
winter.weekday.evening = 396
winter.weekend.night = 156
winter.weekend.morning = 156
winter.weekend.afternoon = 156
winter.weekend.evening = 156
spring-autumn.weekday.night = 792
spring-autumn.weekday.morning = 792
spring-autumn.weekday.afternoon = 528
spring-autumn.weekday.early-peak = 132
spring-autumn.weekday.late-peak = 132
spring-autumn.weekday.evening = 792
spring-autumn.weekend.night = 300
spring-autumn.weekend.morning = 300
spring-autumn.weekend.afternoon = 300
spring-autumn.weekend.evening = 300
summer.weekday.night = 396
summer.weekday.morning  = 396
summer.weekday.afternoon = 264
summer.weekday.early-peak = 66
summer.weekday.late-peak = 66
summer.weekday.evening = 396
summer.weekend.night = 150
summer.weekend.morning = 150
summer.weekend.afternoon = 150
summer.weekend.evening = 150
level_names = ["month", "day", "hour"]

This input introduces three levels, via level_names: month, day, hours. Other simulations may want fewer or more levels. The month level is split into three points of data, winter, spring-autumn, summer. Then day splits out weekdays from weekends, and so on. Each line indicates the number of hours for the relevant slice. It should be noted that the slices are not a cartesian products of each levels. For instance, there no peak periods during weekends. All that matters is that the relative weights (i.e. the number of hours) are consistent and sum up to a year.

outputs_cache

This option behaves exactly like outputs for sectors and accepts the same options but controls the output of cached quantities instead. This option is NOT available for sectors themselves (i.e using [[sector.commercial.outputs_cache]] will have no effect). See :py:mod:`muse.outputs.cache` for more details.

A single row looks like this:

[[outputs_cache]]
quantity = "production"
sink = "aggregate"
filename = "{cwd}/{default_output_dir}/Cache{Quantity}.csv"

Standard sectors

A MUSE model requires at least one sector. Sectors are declared in the TOML file by adding a subsection to the sectors section:

[sectors.residential]
type = 'default'
[sectors.power]
type = 'default'

Above, we've added two sectors, residential and power. The name of the subsection is only used for identification. In other words, it should be chosen to be meaningful to the user, since it will not affect the model itself.

Sectors are defined in :py:class:`~muse.sectors.Sector`.

A sector accepts these attributes:

type
Defines the kind of sector this is. Standard sectors are those with type "default". This value corresponds to the name with which a sector class is registered with MUSE, via :py:meth:`~muse.sectors.register_sector`. [INSERT OTHER OPTIONS HERE]
priority

An integer denoting which sectors runs when. Lower values imply the sector will run earlier. Later sectors can depend on earlier sectors for the their input. If two sectors share the same priority, then their order is not defined. Indeed, it should indicate that they can run in parallel. For simplicity, the keyword also accepts standard values:

  • "preset": 0
  • "demand": 10
  • "conversion": 20
  • "supply": 30
  • "last": 100

Defaults to "last".

interpolation
Interpolation method used to fill missing years in the technodata (defaults to "linear"). Available interpolation methods depend on the underlying scipy method's kind attribute. Years outside the data range will always be back/forward filled with the closest available data.
dispatch_production

The method used to calculate supply of commodities after investments have been made.

MUSE provides two methods in :py:mod:`muse.production`:

  • share: assets each supply a proportion of demand based on their share of total
    capacity
  • maximum: the production is the maximum production for the existing capacity and
    the technology's utilization factor. See :py:func:`muse.production.maximum_production`.

Defaults to "share".

Additional methods can be registered with :py:func:`muse.production.register_production`

technodata
Path to a csv file containing the characterization of the technologies involved in the sector, e.g. lifetime, capital costs, etc... See :ref:`inputs-technodata`.
technodata_timeslices
Optional. Path to a csv file describing the utilization factor and minimum service factor of each technology in each timeslice. See :ref:`user_guide/inputs/technodata_timeslices`.
commodities_in
Path to a csv file describing the inputs of each technology involved in the sector. See :ref:`inputs-iocomms`.
commodities_out
Path to a csv file describing the outputs of each technology involved in the sector. See :ref:`inputs-iocomms`.
timeslice_level
Optional. This represents the level of timeslice granularity over which commodity flows out of the sector are balanced (e.g. if "day", the sector will aim to meet commodity demands on a daily basis, rather than an hourly basis). If not given, defaults to the finest level defined in the global timeslices section. Note: If technodata_timeslices is used, the data in this file must match the timeslice level of the sector (e.g. with global timeslice levels "month", "day" and "hour", if a sector has "day" as the timeslice level, then technodata_timeslices must have columns "month" and "day", but not "hour")

Sectors contain a number of subsections: interactions

Defines interactions between agents. These interactions take place right before new investments are computed. The interactions can be anything. They are expected to modify the agents and their assets. MUSE provides a default set of interactions that have new agents pass on their assets to the corresponding retro agent, and the retro agents pass on the make-up of their assets to the corresponding new agents.

interactions are specified as a :ref:`TOML array<toml-array>`, e.g. with double brackets. Each sector can specify an arbitrary number of interactaction, simply by adding an extra interaction row.

There are two orthogonal concepts to interactions:

In practice, we always consider sequences of nets (i.e. more than one net) that interact using the same interaction function.

Hence, the input looks something like the following:

[[sectors.commercial.interactions]]
net = 'new_to_retro'
interaction = 'transfer'

"new_to_retro" is a function that figures out all "new/retro" pairs of agents. Whereas "transfer" is a function that performs the transfer of assets and information between each pair.

Furthermore, it is possible to pass parameters to either the net of the interaction as follows:

[[sectors.commercial.interactions]]
net = {"name": "some_net", "param": "some value"}
interaction = {"name": "some_interaction", "param": "some other value"}

The parameters will depend on the net and interaction functions. Neither "new_to_retro" nor "transfer" take any arguments at this point. MUSE interaction facilities are defined in :py:mod:`muse.interactions`.

An error is raised if and empty network is found. This is the case, for example, if a "new_to_retro" type of network has been defined but no retro agents are included in the sector.

subsectors

Subsectors group together agents into separate groups servicing the demand for different commodities. There should be at least one subsector. And there can be as many as required. For instance, a one-subsector setup would look like:

[sectors.gas.subsectors.all]
agents = '{path}/technodata/Agents.csv'
existing_capacity = '{path}/technodata/gas/Existing.csv'

A two-subsector could look like:

[sectors.gas.subsectors.methane_and_ethanol]
agents = '{path}/technodata/me_agents.csv'
existing_capacity = '{path}/technodata/gas/me_existing.csv'
commodities = ["methane", "ethanol"]

[sectors.gas.subsectors.natural]
agents = '{path}/technodata/nat_agents.csv'
existing_capacity = '{path}/technodata/gas/nat_existing.csv'
commodities = ["refined", "crude"]

In the case of multiple subsectors, it is important to specify disjoint sets of commodities so that each subsector can service a separate demand. The subsectors accept the following keywords:

agents
Path to a csv file describing the agents in the sector. See :ref:`user_guide/inputs/agents:agents`.
existing_capacity
Path to a csv file describing the initial capacity of the sector. See :ref:`user_guide/inputs/existing_capacity:existing sectoral capacity`.
lpsolver

The solver for linear problems to use when figuring out investments. The solvers are registered via :py:func:`~muse.investments.register_investment`. At time of writing, three are available:

  • "scipy" solver (default from v1.3): Formulates investment as a true LP problem and solves it using the `scipy solver`_.
  • an "adhoc" solver: Simple in-house solver that ranks the technologies according to cost and service the demand incrementally.
  • "cvxopt" solver: Formulates investment as a true LP problem and solves it using the python package `cvxopt`_. `cvxopt`_ is not installed by default. Users can install it with pip install cvxopt or conda install cvxopt.
demand_share

A method used to split the MCA demand into separate parts to be serviced by specific agents. The appropriate choice depends on the type of agents being used in the simulation. There are currently two options:

  • :py:func:`~muse.demand_share.standard_demand` (default): The input demand is split amongst new agents. New agents get a share of the increase in demand over the investment period, as well as the demand that occurs from decommissioned assets.
  • :py:func:`~muse.demand_share.new_and_retro`: The input demand is split amongst both new and retrofit agents. New agents get a share of the increase in demand over the investment period, whereas retrofit agents are assigned a share of the demand that occurs from decommissioned assets.
constraints

The list of constraints to apply to the LP problem solved by the sector. By default all of the following are included:

output

Outputs are made up of several components. MUSE is designed to allow users to mix-and-match both how and what to save.

output is specified as a TOML array, e.g. with double brackets. Each sector can specify an arbitrary number of outputs, simply by adding an extra output row.

A single row looks like this:

[[sectors.commercial.outputs]]
filename = '{cwd}/Results/{Sector}/{Quantity}/{year}{suffix}'
quantity = "capacity"
sink = 'csv'
overwrite = true

The following attributes are available:

  • quantity: Name of the quantity to save. Currently, capacity exists,

    referring to :py:func:`muse.outputs.capacity`. However, users can customize and create further output quantities by registering with MUSE via :py:func:`muse.outputs.register_output_quantity`. See :py:mod:`muse.outputs` for more details.

  • sink: the sink is the place (disk, cloud, database, etc...) and format with which

    the computed quantity is saved. Currently only sinks that save to files are implemented. The filename can specified via filename, as given below. The following sinks are available: "csv", "netcfd", "excel". However, more sinks can be added by interested users, and registered with MUSE via :py:func:`muse.outputs.register_output_sink`. See :py:mod:`muse.outputs` for more details.

  • filename: defines the format of the file where to save the data. There are several

    standard values that are automatically substituted:

    • cwd: current working directory, where MUSE was started
    • path: directory where the TOML file resides
    • sector: name of the current sector (.e.g. "commercial" above)
    • Sector: capitalized name of the current sector
    • quantity: name of the quantity to save (as given by the quantity attribute)
    • Quantity: capitablized name of the quantity to save
    • year: current year
    • suffix: standard suffix/file extension of the sink

    Defaults to {cwd}/{default_output_dir}/{Sector}/{Quantity}/{year}{suffix}.

  • overwrite: If False MUSE will issue an error and abort, instead of

    overwriting an existing file. Defaults to False. This prevents important output files from being overwritten.

There is a special output sink for aggregating over years. It can be invoked as follows:

[[sectors.commercial.outputs]]
quantity = "capacity"
sink.aggregate = 'csv'

Or, if specifying additional output, where ... can be any parameter for the final sink:

[[sectors.commercial.outputs]]
quantity = "capacity"
sink.aggregate.name = { ... }

Note that the aggregate sink always overwrites the final file, since it will overwrite itself.

Preset sectors

The commodity production, commodity consumption and product prices of preset sectors are determined exogeneously. They are know from the start of the simulation and are not affected by the simulation.

Preset sectors are defined in :py:class:`~muse.sectors.PresetSector`.

The three components, production, consumption, and prices, can be set independently and not all three need to be set. Production and consumption default to zero, and prices default to leaving things unchanged.

The following defines a standard preset sector where consumption is defined as a function of macro-economic data, i.e. population and gdp.

[sectors.commercial_presets]
type = 'presets'
priority = 'presets'
timeslice_shares_path = '{path}/technodata/TimesliceShareCommercial.csv'
macrodrivers_path = '{path}/technodata/Macrodrivers.csv'
regression_path = '{path}/technodata/regressionparameters.csv'
timeslices_levels = {'day': ['all-day']}

The following attributes are accepted:

type
See the attribute in the standard mode, :ref:`type<sector-type>`. Preset sectors are those with type "presets".
priority
See the attribute in the standard mode, :ref:`priority<sector-priority>`.
timeslices_levels
See the attribute in the standard mode, Timeslices.
consumption_path

CSV output files, one per year. This attribute can include wild cards, i.e. '*', which can match anything. For instance: consumption_path = "{cwd}/Consumption*.csv" will match any csv file starting with "Consumption" in the current working directory. The file names must include the year for which it defines the consumption, e.g. Consumption2015.csv.

The CSV format should follow the following format:

Consumption
RegionName Timeslice electricity diesel algae
USA 1 1.9 0 0
USA 2 1.8 0 0

The "RegionName" and "Timeslice" columns must be present. Further columns are reserved for commodities. "Timeslice" refers to the index of the timeslice. Timeslices should be defined consistently to the sectoral level timeslices.

supply_path
CSV file, one per year, indicating the amount of commodities produced. It follows the same format as :ref:`consumption_path <preset-consumption>`.
prices_path
CSV file indicating the amount of a commodities produced. The format of the CSV files follows that of :ref:`inputs-projection`.
demand_path
Incompatible with :ref:`consumption_path<preset-consumption>` or :ref:`macrodrivers_path<preset-macro>`. A CSV file containing the consumption in the same format as :ref:`inputs-projection`.
macrodrivers_path
Incompatible with :ref:`consumption_path<preset-consumption>` or :ref:`demand_path<preset-demand>`. Path to a CSV file giving the profile of the macrodrivers. Also requires :ref:`regression_path<preset-regression>`.
regression_path
Incompatible with :ref:`consumption_path<preset-consumption>`. Path to a CSV file giving the regression parameters with respect to the macrodrivers. Also requires :ref:`macrodrivers_path<preset-macro>`.
timeslice_shares_path
Incompatible with :ref:`consumption_path<preset-consumption>` or :ref:`demand_path<preset-demand>`. Optional csv file giving shares per timeslice. The timeslice share definition needs to have a consistent number of timeslices as the sectoral level time slices. Requires :ref:`macrodrivers_path<preset-consumption>`.
filters

Optional dictionary of entries by which to filter the consumption. Requires :ref:`macrodrivers_path<preset-consumption>`. For instance,

filters.region = ["USA", "ASEA"]
filters.commodity = ["algae", "fluorescent light"]