datasets¶
- DSM_cts_ind
- calculate_dlr
- ch4_prod
- ch4_storages
- chp_etrago
- database
- electrical_neighbours
- electricity_demand_etrago
- era5
- etrago_helpers
- etrago_setup
- fill_etrago_gen
- fix_ehv_subnetworks
- gas_areas
- gas_grid
- generate_voronoi
- heat_demand_europe
- industrial_gas_demand
- mastr
- mv_grid_districts
- renewable_feedin
- sanity_checks
- scenario_capacities
- society_prognosis
- substation_voronoi
- tyndp
- vg250_mv_grid_districts
- zensus_mv_grid_districts
- zensus_vg250
- chp
- data_bundle
- demandregio
- district_heating_areas
- electricity_demand
- electricity_demand_timeseries
- gas_neighbours
- heat_demand
- heat_demand_timeseries
- heat_etrago
- heat_supply
- hydrogen_etrago
- industrial_sites
- industry
- loadarea
- low_flex_scenario
- osm
- osm_buildings_streets
- osmtgmod
- power_etrago
- power_plants
- pypsaeursec
- re_potential_areas
- saltcavern
- scenario_parameters
- storages
- storages_etrago
- substation
- vg250
- zensus
The API for configuring datasets.
-
class
Dataset
(name: 'str', version: 'str', dependencies: 'Iterable[Union[Dataset, Task]]' = (), tasks: 'Union[Tasks, TaskGraph]' = ())[source]¶ Bases:
object
-
dependencies
= ()¶ The first task(s) of this
Dataset
will be marked as downstream of any of the listed dependencies. In case of bareTask
, a direct link will be created whereas for aDataset
the link will be made to all of its last tasks.
-
name
= None¶ The name of the Dataset
-
version
= None¶ The
Dataset
’s version. Can be anything from a simple semantic versioning string like “2.1.3”, to a more complex string, like for example “2021-01-01.schleswig-holstein.0” for OpenStreetMap data. Note that the latter encodes theDataset
’s date, region and a sequential number in case the data changes without the date or region changing, for example due to implementation changes.
-
-
class
Model
(**kwargs)[source]¶ Bases:
sqlalchemy.ext.declarative.api.Base
-
dependencies
¶
-
epoch
¶
-
id
¶
-
name
¶
-
version
¶
-
-
Task
= typing.Union[typing.Callable[[], NoneType], airflow.models.baseoperator.BaseOperator]¶ A
Task
is an AirflowOperator
or anyCallable
taking no arguments and returningNone
.Callables
will be converted toOperators
by wrapping them in aPythonOperator
and setting thetask_id
to theCallable
’s__name__
, with underscores replaced with hyphens. If theCallable
’s __module__ attribute contains the string"egon.data.datasets."
, thetask_id
is also prefixed with the module name, followed by a dot and with"egon.data.datasets."
removed.
-
TaskGraph
= typing.Union[typing.Callable[[], NoneType], airflow.models.baseoperator.BaseOperator, typing.Set[ForwardRef('TaskGraph')], typing.Tuple[ForwardRef('TaskGraph'), ...]]¶ A graph of tasks is, in its simplest form, just a single node, i.e. a single
Task
. More complex graphs can be specified by nestingsets
andtuples
ofTaskGraphs
. A set ofTaskGraphs
means that they are unordered and can be executed in parallel. Atuple
specifies an implicit ordering so atuple
ofTaskGraphs
will be executed sequentially in the given order.