Merge branch 'main' into v1.3

Author: tsmbland

.pre-commit-config.yaml

@@ -9,7 +9,7 @@ repos:
       - id: trailing-whitespace
       - id: end-of-file-fixer
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.8.0
+    rev: v0.8.1
     hooks:
       - id: ruff-format
         types_or: [python, pyi, jupyter]

docs/conf.py

@@ -5,6 +5,9 @@
 # http://www.sphinx-doc.org/en/master/config
 # -- Project information -----------------------------------------------------
 
+from docutils import nodes
+from docutils.parsers.rst import roles
+
 project = "MUSE"
 copyright = "2024, Imperial College London"
 author = "Imperial College London"
@@ -61,3 +64,29 @@
 # -- Options for HTML output -------------------------------------------------
 
 html_theme = "sphinx_rtd_theme"
+
+# -- Render GitHub links -------------------------------------------------
+
+
+def github_role(name, rawtext, text, lineno, inliner, options=None, content=None):
+    """Creates links to issues/pull requests on the MUSE_OS GitHub site.
+
+    To use in markdown:
+    {github}`ISSUE_NUMBER` (e.g. {github}`123`)
+
+    To use in rst:
+    :github:`ISSUE_NUMBER` (e.g. :github:`123`)
+
+    In both cases this will create a clickable link (visible as #123) to the relevant
+    GitHub page (i.e. https://github.com/EnergySystemsModellingLab/MUSE_OS/issues/123)
+
+    The base URL is for the issues page, but this will also work for pull requests and
+    discussions, as GitHub will automatically redirect to the appropriate page.
+    """
+    base_url = "https://github.com/EnergySystemsModellingLab/MUSE_OS/issues/"
+    url = f"{base_url}{text}"
+    node = nodes.reference(rawtext, f"#{text}", refuri=url, **(options or {}))
+    return [node], []
+
+
+roles.register_canonical_role("github", github_role)

docs/index.rst

@@ -25,6 +25,7 @@ ModUlar energy system Simulation Environment: MUSE
    advanced-guide/index
    faq
    api
+   release-notes/index
 
 
 Indices and tables

docs/inputs/technodata.rst

@@ -136,33 +136,32 @@ InterestRate
    is the technology interest rate (called hurdle rates in other models).
    This is used for the interest used in the discount rate and corresponds to the interest built when borrowing money.
 
-Agent_0, ..., Agent_N
-   represent the allocation of the initial capacity to the each agent.
-   The column heading refers each retrofit agent "AgentShare" as defined in the agents' definition (see :ref:`inputs-agents`).
+Agent0, ..., AgentN
+   represent the proportion of initial capacity allocated to each agent.
+   Must match AgentShare names specified in the :ref:`inputs-agents` file.
+   All agents must be represented in the table.
+   If using "New" and "Retrofit" agents, you should create a column with the name of each "Retrofit" agent share.
+   If only using "New" agents, you should create a column with the name of each "New" agent share.
    The value corresponds to the ownership of the initial stock, as defined in the :ref:`inputs-existing-capacity` for the starting year of the simulation.
-   For example, if an initial boiler stock of 10 PJ is available, this is allocated to each agent according to the "AgentShare".
 
-   In a one-agent simulation, assuming that the *AgentShare* equals to *Agent_2* for the retrofit agent, the technodata should indicate the stock ownership as follows.
-   The modelled agent would own the total 10 PJ of the initial stock of boilers.
+   For example, in a one-agent simulation, you should specify the following to indicate full ownership of existing capacity by the agent (assuming an agent share name of "Agent1"):
 
    .. csv-table:: Techno-data: AgentShare - 1 agent
-      :header: ProcessName, RegionName, Time, ..., Agent_2
+      :header: ProcessName, RegionName, Time, ..., Agent1
 
       resBoilerElectric, region1, 2010, ..., 1
       resBoilerElectric, region1, 2030, ..., 1
 
-   In a two-agent simulation, a new column needs to be added for each retrofit agent belonging to the new-retrofit agent pair.
-   The column heading refers each retrofit agent "AgentShare" as defined in the agents' definition (see :ref:`inputs-agents`).
-   Assuming a split of the initial capacity into 30 \% and 70 \% for each retrofit agent, the model table would be setup as follows.
-   The values of the "AgentShare" needs to reflect the demand split represented by the "Quantity" attribute (see :ref:`inputs-agents`),
-   to make sure that the initial demand is fulfilled with the initial stock.
+   In a two-agent simulation, assuming a 30\% / 70\% split of initial capacity between the two agents, the table would be as follows:
 
    .. csv-table:: Techno-data: AgentShare - 2 agents
-      :header: ProcessName, RegionName, Time, ..., Agent_2, Agent_4
+      :header: ProcessName, RegionName, Time, ..., Agent1, Agent2
 
       resBoilerElectric, region1, 2010, ..., 0.3, 0.7
       resBoilerElectric, region1, 2030, ..., 0.3, 0.7
 
+   Values must sum to 1 for each row of the table.
+
 The input data has to be provided for the base year. Additional years within the time
 framework of the overall simulation can be defined. In this case, MUSE would interpolate
 the values between the provided periods and assume a constant value afterwards. The additional

docs/release-notes/index.rst

@@ -0,0 +1,12 @@
+Release notes
+=============
+
+This is the list of changes to MUSE between each release.
+
+.. toctree::
+   :maxdepth: 1
+
+   v1.2.3
+   v1.2.2
+   v1.2.1
+   v1.2.0

docs/release-notes/v1.2.0.md

@@ -0,0 +1,46 @@
+# Release notes for MUSE v1.2.0 (September 19, 2024)
+
+These are the main changes in MUSE v1.2.0.
+
+Please read carefully if upgrading from an earlier version, as some of the changes may require you to modify your model input files or downstream analysis scripts.
+
+## Dropped support for Python 3.8
+
+- MUSE is now compatible with Python versions 3.9 to 3.12
+- If you were previously using Python 3.8, you must create a new environment using 3.9 or later. There are instructions on setting up a new environment with the appropriate Python version [here](https://muse-os.readthedocs.io/en/latest/installation/virtual-env-based.html)
+
+## Model settings
+
+- The default `demand_share` has changed from "new_and_retro" to "standard_demand" ({github}`349`). If your model uses retrofit agents, you MUST explicitly specify `demand_share = "new_and_retro"` for all relevant sectors. If this is left out, the model will try to use the "standard_demand" share and will fail. See updated documentation about this parameter [here](https://muse-os.readthedocs.io/en/latest/inputs/toml.html#standard-sectors).
+- The default value for `maximum_iterations` has changed from 3 to 100 ({github}`386`)
+
+## Input files
+
+- The `Level` column is no longer required in the `Technodata`  and `CommOut` files, as this parameter isn't used and never has been ({github}`374` and {github}`377`). See updated documentation about this parameter [here](https://muse-os.readthedocs.io/en/latest/inputs/commodities_io.html).
+- The `ProcessName` column is no longer required in consumption files specified in the _consumption_path_ format. See updated documentation [here](https://muse-os.readthedocs.io/en/latest/inputs/toml.html#preset-sectors). Users are encouraged to remove this column from their files (summing rows for different processes if necessary), as this more accurately depicts how this data is used in the model.
+
+## Output files
+
+- We have changed how timeslices are represented in some of the output files ({github}`412`), so this now follows a consistent format across all files. Some of your downstream scripts for loading and analysing these output files may need to be updated.
+- Previously it was necessary to specify `index = true` (in `settings.toml`) for some of the outputs to prevent some important columns from being dropped. This is no longer required, and users should no longer require the `index` parameter for any reason ({github}`412`).
+
+## Model
+
+- Fixed a bug with the calculation of commodity prices ({github}`418`), which affects all models that have multiple timeslices and `maximum_iterations` > 1.
+- Fixed a bug which was preventing the convergence criteria in the MCA algorithm from being properly checked ({github}`407`). This will likely affect the results of most models that have `maximum_iterations` > 1.
+- The `minimum_service_factor` parameter was previously being applied incorrectly - this has been fixed ({github}`388`).
+- Constraints specified in the `settings.toml` file (with the `constraints` key) were previously being ignored (the model would always revert to the default list of constraints). This has been fixed ({github}`354`).
+- We have added a constraint that limits installed capacity to be no greater than that required to meet peak demand ({github}`355`). This is applied by default, however if you are manually overriding the defaults in the settings file with the `constraints` key, you MUST include "demand_limiting_capacity" in this list otherwise this constraint won't be used. Note that this only applies to the scipy solver (`lpsolver = "scipy"` in `settings.toml`), not the adhoc solver.
+- Fixed a bug with the calculation of LCOE in models that have a utilization factor of zero for some timeslices ({github}`304`)
+- Fixed a bug with the weighted_sum decision method ({github}`449`)
+
+## Other notes
+
+- Please don't use the "metric_supply", "metricy_supply", "timeslice_supply", "yearly_supply", "metric_consumption", "metricy_consumption", "timeslice_consumption" or "yearly_consumption" outputs, as these sometimes give incorrect values and will likely be deleted in the future. You should be able to get everything you need by using the "supply" and "consumption" outputs within each sector, for example (replacing `SECTOR_NAME` with the name of the sector):
+
+  ```toml
+  [[sectors.SECTOR_NAME.outputs]]
+  filename = '{cwd}/{default_output_dir}/{Sector}/{Quantity}.csv'
+  quantity = "supply"
+  sink = "aggregate"
+  ```

docs/release-notes/v1.2.1.md

@@ -0,0 +1,26 @@
+# Release notes for MUSE v1.2.1 (October 9, 2024)
+
+These are the main changes in MUSE v1.2.1.
+
+Please read carefully if upgrading from an earlier version, as some of the changes may require you to modify your model input files.
+
+## Carbon budget module
+
+- The bisection method has been fixed, as this was not working as expected before ({github}`483`, {github}`484`). Additionally, a number of default settings have been changed, and parameters renamed. Users must carefully check which parameters are specified in their settings files, as any unspecified parameters will revert to the new defaults. The main changes are as follows:
+  - `control_undershoot` / `control_overshoot`: The default has been changed from True to False
+  - `method`: The default has changed from `fitting` to `bisection`
+  - `method_options.refine_price`: The default has changed from True to False
+  - `method_options.resolution`: New parameter
+  - `method_options.sample_size`: This parameter no longer applies to the bisection algorithm, and won't be permitted when using that method. Please use `method_options.max_iterations` instead
+  - `method_options.tolerance` and `method_options.early_termination_count`: New parameters for the bisection method
+
+__Please read the [new documentation page](https://muse-os.readthedocs.io/en/documentation/inputs/toml.html#carbon-market) in full before using a carbon budget__
+
+## Default model
+
+- A number of changes have been made to the default model that is generated with `muse --model default --copy PATH`. This is mostly to simplify the outputs ({github}`461`)
+
+## Tutorials
+
+- The tutorials have been simplified and re-ordered ({github}`470`)
+- A tutorial has been added explaining the use of a carbon budget ({github}`486`)

docs/release-notes/v1.2.2.md

@@ -0,0 +1,13 @@
+# Release notes for MUSE v1.2.2 (October 28, 2024)
+
+These are the main changes in MUSE v1.2.2.
+
+## Bug fixes
+
+This release fixes several errors in calculations related to the utilization factor, minimum service factor and commodity prices. Interested readers can follow the discussions in the following pull requests:
+
+- {github}`368`
+- {github}`518`
+- {github}`534`
+
+Developers should pay particular attention to the latter two PRs, to avoid introducing similar mistakes in the future.

docs/release-notes/v1.2.3.md

@@ -0,0 +1,25 @@
+# Release notes for MUSE v1.2.3 (November 19, 2024)
+
+These are the main changes in MUSE v1.2.0.
+
+Please read carefully if upgrading from an earlier version, as some of the changes may require you to modify your model input files.
+
+## Log files ({github}`560`)
+
+- MUSE will now output two log files to the results folder whenever a simulation is run: one (`muse_info.log`) containing info and debug messages, and the other (`muse_warning.log`) containing warning messages.
+- Users are particularly encouraged to consult the `muse_warnings.log` file after each run, and pay attention to any messages shown.
+
+## `MaxCapacityGrowth` parameter ({github}`565`)
+
+- The definition of this parameter has changed to compound growth rather than linear growth. It can now be properly interpreted as the maximum growth of the technology per year. See the documentation for this parameter [here](https://muse-os.readthedocs.io/en/latest/inputs/technodata.html)
+- To give an example, if a 5 year time period is being used, a value of 0.5 previously meant that capacity was allowed to grow by 250% over the time period (0.5 x 5 x 100), but with compound growth the limit is now 660% (((1.5 ** 5) - 1) x 100)
+- Users may see results change as a result of this.
+
+## Material and Variable costs ({github}`563`)
+
+- Fixes errors in the calculation of material and variable costs, particularly in the case of technologies with output quantities not equal to 1. See #563 for details
+- This effects decision metrics, but doesn't directly effect commodity prices. There are known issues remaining in the calculation of commodity prices ({github}`551`, {github}`552`), which will be fixed separately
+
+## Minor bug fixes
+
+- Fixed an error introduced in v1.2.1 which caused carbon budget simulations to fail if no method options were specified ({github}`539`)

src/muse/agents/factories.py

@@ -7,7 +7,7 @@
 import xarray as xr
 
 from muse.agents.agent import Agent, InvestingAgent
-from muse.errors import RetrofitAgentNotDefined, TechnologyNotDefined
+from muse.errors import AgentShareNotDefined, TechnologyNotDefined
 
 
 def create_standard_agent(
@@ -251,7 +251,7 @@ def _shared_capacity(
     try:
         shares = technologies[share]
     except KeyError:
-        raise RetrofitAgentNotDefined
+        raise AgentShareNotDefined
 
     try:
         shares = shares.sel(technology=capacity.technology)