Remove remains of the forecast feature (#645)

tsmbland · web-flow · commit f9ad220da856 · 2025-02-07T09:04:18.000Z
* Fix outputs - save data for investment year * Revert change to fix tests * Update results files * Restore test * Revert change to outputs cache * Get outputs for first year * Update mca.py * Remove foresight parameter (#641) * Move broadcast_techs out of quantities module * Update _inner_split * Fix trade model * Fix constraints tests * Fix demand_share tests * Fix quantities tests * Fix trade tests * Fix adhoc model * Rename broadcast_techs to broadcast_over_assets * Better docstring, rename argument * Delete filter_with_template * Remove `broadcast_over_assets` from `capacity_in_use` * Fix trade tests * Edit TODO * Remove foresight parameter * Update settings and documentation * Temporarily suppress test * Fix carbon budget framework * Revert change to settings * Update settings and documentation * Revert change to settings * Update results * Small changes to fix tests * Add warning about deprecated parameter * Clarify log messages * Better final message * Update results * Actually update results * And more * Add kwargs arguments to prevent complaints * Add more kwargs arguments * Update trade result * Remove more forecast features * Change docstring * Remove forecast settings * Simplify _interpolate * Update docs * Remove remaining references * Warning about deprecated parameter * Tidy run function
diff --git a/docs/application-flow.rst b/docs/application-flow.rst
@@ -414,7 +414,7 @@ This deeper level of the process is where most of the input options of MUSE are
 Investment
 ~~~~~~~~~~
 
-In the investment step is where new capacity is added to the different assets managed by the agents. This investment might be needed to cover an increase in demand (between now and forecast) or to match decommissioned assets, typically to do both.
+In the investment step is where new capacity is added to the different assets managed by the agents. This investment might be needed to cover an increase in demand (between now and the investment year) or to match decommissioned assets, typically to do both.
 
 The following graph summarises the process.
 
@@ -455,8 +455,8 @@ The following graph summarises the process.
 
 First the demand is distributed among the available agents as requested by the ``demand_share`` argument of each ``subsector`` in the ``settings.toml`` file. This distribution can be done based on any attribute or property of the agents, as included in the ``Agents.csv`` file. Demand can also be shared across multiple agents, depending on the "quantity" attribute (defined in ``Agents.csv``). The two built-in options in MUSE are:
 
-- `standard_demand` (default): The demand is split only amongst *new* agents (indeed there will be an error if a *retro* agent is found for this subsector). *New* agents get a share of the increase in demand for the forecast years well as the demand that occurs from decommissioned assets.
-- `new_and_retro`: The input demand is split amongst both *new* and *retro* agents. *New* agents get a share of the increase in demand for the forecast year, whereas *retrofit* agents are assigned a share of the demand that occurs from decommissioned assets.
+- `standard_demand` (default): The demand is split only amongst *new* agents (indeed there will be an error if a *retro* agent is found for this subsector). *New* agents get a share of the increase in demand over the investment period as well as the demand that occurs from decommissioned assets.
+- `new_and_retro`: The input demand is split amongst both *new* and *retro* agents. *New* agents get a share of the increase in demand for the investment period, whereas *retrofit* agents are assigned a share of the demand that occurs from decommissioned assets.
 
 Then, each agent select the technologies it can invest in based on what is needed and the **search rules** defined for it in the ``Agents.csv`` file. The possible search rules are described in :py:mod:`muse.filters`. These determine the search rules for each replacement technology.
 
diff --git a/docs/inputs/toml.rst b/docs/inputs/toml.rst
@@ -455,11 +455,11 @@ Sectors contain a number of subsections:
 
         - :py:func:`~muse.demand_share.standard_demand` (default): The input demand is
           split amongst *new* agents. *New* agents get a share of the increase in demand
-          for the forecast years, as well as the demand that occurs from decommissioned
+          over the investment period, as well as the demand that occurs from decommissioned
           assets.
         - :py:func:`~muse.demand_share.new_and_retro`: The input demand is split amongst
           both *new* and *retrofit* agents. *New* agents get a share of the increase in
-          demand for the forecast year, whereas *retrofit* agents are assigned a share
+          demand over the investment period, whereas *retrofit* agents are assigned a share
           of the demand that occurs from decommissioned assets.
 
     *constraints*
@@ -577,7 +577,6 @@ function of macro-economic data, i.e. population and gdp.
     macrodrivers_path = '{path}/technodata/Macrodrivers.csv'
     regression_path = '{path}/technodata/regressionparameters.csv'
     timeslices_levels = {'day': ['all-day']}
-    forecast = [0, 5]
 
 The following attributes are accepted:
 
diff --git a/docs/tutorial-code/add-agent/1-single-objective/settings.toml b/docs/tutorial-code/add-agent/1-single-objective/settings.toml
@@ -51,7 +51,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "standard_demand"  # Optional, default to standard_demand
-forecast = 5  # Optional, defaults to 5
 
 [sectors.power]
 type = 'default'
diff --git a/docs/tutorial-code/add-agent/2-multiple-objective/settings.toml b/docs/tutorial-code/add-agent/2-multiple-objective/settings.toml
@@ -51,7 +51,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "standard_demand"  # Optional, default to standard_demand
-forecast = 5  # Optional, defaults to 5
 
 [sectors.power]
 type = 'default'
diff --git a/docs/tutorial-code/add-correlation-demand/1-correlation/settings.toml b/docs/tutorial-code/add-correlation-demand/1-correlation/settings.toml
@@ -51,7 +51,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "standard_demand"  # Optional, default to standard_demand
-forecast = 5  # Optional, defaults to 5
 
 [sectors.power]
 type = 'default'
diff --git a/docs/tutorial-code/add-new-technology/1-introduction/settings.toml b/docs/tutorial-code/add-new-technology/1-introduction/settings.toml
@@ -51,7 +51,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "standard_demand"  # Optional, default to standard_demand
-forecast = 5  # Optional, defaults to 5
 
 [sectors.power]
 type = 'default'
diff --git a/docs/tutorial-code/add-new-technology/2-scenario/settings.toml b/docs/tutorial-code/add-new-technology/2-scenario/settings.toml
@@ -51,7 +51,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "standard_demand"  # Optional, default to standard_demand
-forecast = 5  # Optional, defaults to 5
 
 [sectors.power]
 type = 'default'
diff --git a/docs/tutorial-code/add-region/1-new-region/settings.toml b/docs/tutorial-code/add-region/1-new-region/settings.toml
@@ -51,7 +51,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "standard_demand"  # Optional, default to standard_demand
-forecast = 5  # Optional, defaults to 5
 
 [sectors.power]
 type = 'default'
diff --git a/docs/tutorial-code/add-service-demand/1-exogenous-demand/settings.toml b/docs/tutorial-code/add-service-demand/1-exogenous-demand/settings.toml
@@ -51,7 +51,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "standard_demand"  # Optional, default to standard_demand
-forecast = 5  # Optional, defaults to 5
 
 [sectors.power]
 type = 'default'
diff --git a/docs/tutorial-code/carbon-budget/1-carbon-budget/settings.toml b/docs/tutorial-code/carbon-budget/1-carbon-budget/settings.toml
@@ -64,7 +64,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "standard_demand"  # Optional, default to standard_demand
-forecast = 5  # Optional, defaults to 5
 
 [sectors.power]
 type = 'default'
diff --git a/docs/tutorial-code/min-max-timeslice-constraints/1-min-constraint/settings.toml b/docs/tutorial-code/min-max-timeslice-constraints/1-min-constraint/settings.toml
@@ -51,7 +51,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "standard_demand"  # Optional, default to standard_demand
-forecast = 5  # Optional, defaults to 5
 
 [sectors.power]
 type = 'default'
diff --git a/docs/tutorial-code/min-max-timeslice-constraints/2-max-constraint/settings.toml b/docs/tutorial-code/min-max-timeslice-constraints/2-max-constraint/settings.toml
@@ -51,7 +51,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "standard_demand"  # Optional, default to standard_demand
-forecast = 5  # Optional, defaults to 5
 
 [sectors.power]
 type = 'default'
diff --git a/docs/tutorial-code/modify-timing-data/1-modify-timeslices/settings.toml b/docs/tutorial-code/modify-timing-data/1-modify-timeslices/settings.toml
@@ -51,7 +51,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "standard_demand"  # Optional, default to standard_demand
-forecast = 5  # Optional, defaults to 5
 
 [sectors.power]
 type = 'default'
diff --git a/docs/tutorial-code/modify-timing-data/2-modify-time-framework/settings.toml b/docs/tutorial-code/modify-timing-data/2-modify-time-framework/settings.toml
@@ -51,7 +51,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "standard_demand"  # Optional, default to standard_demand
-forecast = 2  # Optional, defaults to 5
 
 [sectors.power]
 type = 'default'
@@ -66,7 +65,6 @@ agents = '{path}/technodata/Agents.csv'
 existing_capacity = '{path}/technodata/power/ExistingCapacity.csv'
 lpsolver = "scipy"
 demand_share = "standard_demand"
-forecast = 2
 
 [sectors.gas]
 type = 'default'
@@ -81,7 +79,6 @@ agents = '{path}/technodata/Agents.csv'
 existing_capacity = '{path}/technodata/gas/ExistingCapacity.csv'
 lpsolver = "scipy"
 demand_share = "standard_demand"
-forecast = 2
 
 [sectors.residential_presets]
 type = 'presets'
diff --git a/docs/tutorial-code/modify-timing-data/generate_models.py b/docs/tutorial-code/modify-timing-data/generate_models.py
@@ -6,7 +6,7 @@
 from tomlkit import dumps, parse
 
 from muse import examples
-from muse.wizard import add_timeslice, get_sectors, modify_toml
+from muse.wizard import add_timeslice, modify_toml
 
 parent_path = Path(__file__).parent
 
@@ -80,11 +80,6 @@ def generate_model_2():
     settings_file = model_path / "settings.toml"
     time_framework = [2020, 2022, 2024, 2026, 2028, 2030, 2032, 2034, 2036, 2038, 2040]
     modify_toml(settings_file, lambda x: x.update({"time_framework": time_framework}))
-    for sector in get_sectors(model_path):
-        modify_toml(
-            settings_file,
-            lambda x: x["sectors"][sector]["subsectors"]["all"].update({"forecast": 2}),
-        )
 
 
 if __name__ == "__main__":
diff --git a/docs/tutorial-code/new-decision-metric/settings.toml b/docs/tutorial-code/new-decision-metric/settings.toml
@@ -49,7 +49,6 @@ constraints = [
     "search_space"
 ]
 demand_share = "standard_demand"  # Optional, default to standard_demand
-forecast = 5  # Optional, defaults to 5
 
 [sectors.power]
 type = 'default'
diff --git a/docs/user-guide/modify-timing-data.ipynb b/docs/user-guide/modify-timing-data.ipynb
@@ -149,21 +149,6 @@
     "```"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "We also have to modity the `forecast` value for each sector. For example, for the power sector the `sectors.power.subsectors.all` section should now look as follows:\n",
-    "\n",
-    "```toml\n",
-    "[sectors.power.subsectors.all]\n",
-    "agents = '{path}/technodata/Agents.csv'\n",
-    "existing_capacity = '{path}/technodata/power/ExistingCapacity.csv'\n",
-    "lpsolver = \"scipy\"\n",
-    "forecast = 2\n",
-    "```"
-   ]
-  },
   {
    "cell_type": "markdown",
    "metadata": {},
diff --git a/src/muse/agents/agent.py b/src/muse/agents/agent.py
@@ -113,7 +113,6 @@ def __init__(
         decision: Callable | None = None,
         year: int = 2010,
         maturity_threshold: float = 0,
-        forecast: int = 5,
         housekeeping: Callable | None = None,
         merge_transform: Callable | None = None,
         demand_threshold: float | None = None,
@@ -137,7 +136,6 @@ def __init__(
             year: year the agent is created / current year
             maturity_threshold: threshold when filtering replacement
                 technologies with respect to market share
-            forecast: Number of years the agent will forecast
             housekeeping: transform applied to the assets at the start of
                 iteration. Defaults to doing nothing.
             merge_transform: transform merging current and newly invested assets
@@ -170,9 +168,6 @@ def __init__(
         """ Current year. Incremented by one every time next is called."""
         self.year = year
 
-        """Number of years to look into the future for forecating purposed."""
-        self.forecast = forecast
-
         """Search rule(s) determining potential replacement technologies.
 
         This is a string referring to a filter, or a sequence of strings
@@ -234,11 +229,6 @@ def __init__(
         """Threshold below which assets are not added."""
         self.asset_threshold = asset_threshold
 
-    @property
-    def forecast_year(self):
-        """Year to consider when forecasting."""
-        return self.year + self.forecast
-
     def asset_housekeeping(self):
         """Reduces memory footprint of assets.
 
diff --git a/src/muse/carbon_budget.py b/src/muse/carbon_budget.py
@@ -334,7 +334,7 @@ def bisection(
     # Create cache for emissions at different price points
     emissions_cache = EmissionsCache(market, equilibrium, commodities)
 
-    # Carbon price and emissions threshold in the forecast year
+    # Carbon price and emissions threshold in the investment year
     future = market.year[-1]
     target = carbon_budget.sel(year=future).values.item()
     price = market.prices.sel(year=future, commodity=commodities).mean().values.item()
diff --git a/src/muse/data/example/default/settings.toml b/src/muse/data/example/default/settings.toml
@@ -51,7 +51,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "standard_demand"  # Optional, default to standard_demand
-forecast = 5  # Optional, defaults to 5
 
 [sectors.power]
 type = 'default'
diff --git a/src/muse/data/example/default_retro/settings.toml b/src/muse/data/example/default_retro/settings.toml
@@ -51,7 +51,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "new_and_retro"  # Optional, default to standard_demand
-forecast = 5  # Optional, defaults to 5
 
 [[sectors.residential.interactions]]
 net = 'new_to_retro'
diff --git a/src/muse/data/example/default_timeslice/settings.toml b/src/muse/data/example/default_timeslice/settings.toml
@@ -51,7 +51,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "standard_demand"  # Optional, default to standard_demand
-forecast = 5  # Optional, defaults to 5
 
 [sectors.power]
 type = 'default'
diff --git a/src/muse/data/example/minimum_service/settings.toml b/src/muse/data/example/minimum_service/settings.toml
@@ -56,7 +56,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "standard_demand"  # Optional, default to standard_demand
-forecast = 5  # Optional, defaults to 5
 
 [sectors.industry_presets]
 type = 'presets'
diff --git a/src/muse/data/example/trade/settings.toml b/src/muse/data/example/trade/settings.toml
@@ -58,7 +58,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "new_and_retro"
-forecast = 5
 asset_threshold = 1e-4
 
 [[sectors.residential.interactions]]
@@ -90,7 +89,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "unmet_forecasted_demand"
-forecast = 5
 asset_threshold = 1e-4
 
 [sectors.gas]
@@ -118,7 +116,6 @@ constraints = [
     "demand_limiting_capacity"
 ]
 demand_share = "unmet_forecasted_demand"
-forecast = 5
 asset_threshold = 1e-4
 
 [timeslices]
diff --git a/src/muse/readers/toml.py b/src/muse/readers/toml.py
@@ -737,7 +737,7 @@ def read_technodata(
             technologies[name] = data
 
     # make sure technologies includes the requisite years
-    maxyear = getattr(settings, "forecast", 5) + max(time_framework)
+    maxyear = max(time_framework)
     if technologies.year.max() < maxyear:
         msg = "Forward-filling technodata to fit simulation timeframe"
         getLogger(__name__).info(msg)
diff --git a/src/muse/sectors/preset_sector.py b/src/muse/sectors/preset_sector.py
@@ -143,29 +143,9 @@ def next(self, mca_market: Dataset) -> Dataset:
         consumption = self._interpolate(presets.consumption, mca_market.year)
         costs = self._interpolate(presets.costs, mca_market.year)
 
-        result = Dataset({"supply": supply, "consumption": consumption})
-        result["costs"] = drop_timeslice(costs)
+        result = Dataset({"supply": supply, "consumption": consumption, "costs": costs})
         assert isinstance(result, Dataset)
         return result
 
     def _interpolate(self, data: DataArray, years: DataArray) -> DataArray:
-        """Chooses interpolation depending on whether forecast is available."""
-        if "forecast" in data.dims:
-            baseyear = int(years.min())
-            forecasted = (years - baseyear).values
-            result = (
-                data.interp(
-                    year=baseyear,
-                    method=self.interpolation_mode,
-                    kwargs={"fill_value": "extrapolate"},
-                )
-                .interp(
-                    forecast=forecasted,
-                    method=self.interpolation_mode,
-                    kwargs={"fill_value": "extrapolate"},
-                )
-                .drop_vars(("year", "forecast"))
-            )
-            result["year"] = "forecast", years.values
-            return result.set_index(forecast="year").rename(forecast="year")
         return data.interp(year=years, method=self.interpolation_mode).ffill("year")
diff --git a/src/muse/sectors/sector.py b/src/muse/sectors/sector.py
@@ -169,15 +169,6 @@ def __init__(
         """Full supply, consumption and costs data for the most recent year."""
         self.output_data: xr.Dataset
 
-    @property
-    def forecast(self):
-        """Maximum forecast horizon across agents.
-
-        It cannot be lower than 1 year.
-        """
-        forecasts = [getattr(agent, "forecast") for agent in self.agents]
-        return max(1, max(forecasts))
-
     def next(
         self,
         mca_market: xr.Dataset,
diff --git a/src/muse/sectors/subsector.py b/src/muse/sectors/subsector.py

Original file line number	Diff line number	Diff line change
`@@ -51,7 +51,6 @@ constraints = [`
`51`	`51`	`"demand_limiting_capacity"`
`52`	`52`	`]`
`53`	`53`	`demand_share = "standard_demand" # Optional, default to standard_demand`
`54`		`-forecast = 5 # Optional, defaults to 5`
`55`	`54`
`56`	`55`	`[sectors.power]`
`57`	`56`	`type = 'default'`
Original file line number	Diff line number	Diff line change
`@@ -64,7 +64,6 @@ constraints = [`
`64`	`64`	`"demand_limiting_capacity"`
`65`	`65`	`]`
`66`	`66`	`demand_share = "standard_demand" # Optional, default to standard_demand`
`67`		`-forecast = 5 # Optional, defaults to 5`
`68`	`67`
`69`	`68`	`[sectors.power]`
`70`	`69`	`type = 'default'`