Difference between revisions of "Energy data"

From embargo
Jump to navigation Jump to search
 
(15 intermediate revisions by 5 users not shown)
Line 4: Line 4:
  
 
== Trading Hub Europe ==
 
== Trading Hub Europe ==
−
 
+
* {{Citation | vauthors=((THE: Market area monitor)) | title=Germany: Import / Export / Consumption / Storage | url=https://www.tradinghub.eu/en-gb/Publications/Transparency/Market-area-monitor}}
−
{{Citation | vauthors=((THE)) | title=Germany: Daily gas consumption | url=https://www.tradinghub.eu/en-gb/Publications/Transparency/Aggregated-consumption-data#}}
+
* {{Citation | vauthors=((THE)) | title=Germany: Daily gas consumption | url=https://www.tradinghub.eu/en-gb/Publications/Transparency/Aggregated-consumption-data#}}
 +
* Historical data (before Oct. 2021): {{Citation | title=GASPOOL: Aggregated gas consumption | url=https://www.tradinghub.eu/Portals/0/Archiv%20GASPOOL/verbrauchsdaten-gesamt[1].csv?ver=BY7wJ3jSq7CqEWv7Qtka1w%3d%3d}} / {{Citation | title=NetConnect: Aggregated gas consumption | url=https://www.tradinghub.eu/Portals/0/Archiv%20NCG/AggregatedConsumptionData[1].csv?ver=QvIUEDkSVZrhNqQ9YEsu8w%3d%3d}}
 +
* API documentation: https://datenservice.tradinghub.eu/Dokumente/The_XML_Interface_V1.0_en.pdf
 +
** e.g. query on consumption <code> https://datenservice.tradinghub.eu/XmlInterface/getXML.ashx?ReportId=AggregatedConsumptionData&Start=01-08-2022&End=15-08-2022</code>
 +
* Aggregated data from 2011 to 2022: https://files.embargo.energy/the-analysis/the_gas_consumption_2011_to_2022.ods
 +
* Dashboard visualizing daily gas consumption in Germany see https://embargo.energy:3000/d/bRM7QBZVk/the?orgId=1&from=1504453425838&to=1662219825839 , this is is using downloads of the aforementioned XML interface as well as the aforemented historical data, python scripts: git clone https://git.embargo.energy/entsog.git.
 +
* Interpretation at zeit.de (Split consumer/producer):  <nowiki>https://twitter.com/c_endt/status/1566689441992810497</nowiki>
  
 
== Bundesnetzagentur ==
 
== Bundesnetzagentur ==
  
 
{{Citation | vauthors=BNetzA | title=Germany: Daily briefings about gas imports, storage, and consumption | url=https://www.bundesnetzagentur.de/DE/Fachthemen/ElektrizitaetundGas/Versorgungssicherheit/aktuelle_gasversorgung/start.html}}
 
{{Citation | vauthors=BNetzA | title=Germany: Daily briefings about gas imports, storage, and consumption | url=https://www.bundesnetzagentur.de/DE/Fachthemen/ElektrizitaetundGas/Versorgungssicherheit/aktuelle_gasversorgung/start.html}}
−
 
−
:
 
  
 
{| class="wikitable" style=";text-align:right;width:50%"
 
{| class="wikitable" style=";text-align:right;width:50%"
Line 46: Line 50:
 
| TOTAL || 1016.4 ||  || -84.6 ||  
 
| TOTAL || 1016.4 ||  || -84.6 ||  
 
|}
 
|}
 +
 +
 +
https://www.bundesnetzagentur.de/DE/Fachthemen/ElektrizitaetundGas/Versorgungssicherheit/aktuelle_gasversorgung/_downloads/08_August/220825_gaslage.pdf?__blob=publicationFile&v=3#page=5
 +
https://www.bundesnetzagentur.de/DE/Fachthemen/ElektrizitaetundGas/Versorgungssicherheit/aktuelle_gasversorgung/start.html
  
 
== BDEW ==
 
== BDEW ==
Line 76: Line 84:
  
 
* {{Citation | vauthors=Bruegel | title=EU: Natural gas imports (plots) | url=https://www.bruegel.org/dataset/european-natural-gas-imports | access-date=12 July 2022}}
 
* {{Citation | vauthors=Bruegel | title=EU: Natural gas imports (plots) | url=https://www.bruegel.org/dataset/european-natural-gas-imports | access-date=12 July 2022}}
 +
* {{Citation | vauthors=Bruegel | title=Energy policy responses to crisis | url=https://www.bruegel.org/dataset/national-energy-policy-responses-energy-crisis | access-date=19 October 2022}}
 +
* https://www.bruegel.org/dataset/european-natural-gas-demand-tracker
  
 
== ENTSOG gas data ==
 
== ENTSOG gas data ==
Line 87: Line 97:
 
* https://files.embargo.energy/entsog-analysis/
 
* https://files.embargo.energy/entsog-analysis/
  
−
=== ENTSOG data as relational database  ===
+
[[ENTSOG data analysis]]
−
 
 
−
(Experimental, physical flow data from 01 Jan 2021 to 05 Aug 2022, where flow value is not null)
 
−
 
 
−
Postgres access: <code> psql -h embargo.energy -U embargo entsog </code>; password: 9mTc_9hPjk}&D27/F,VHC8qG
 
−
 
 
−
Some fields:
 
−
 
 
−
====== Table operator: ======
 
−
 
 
−
* adjacentcountry: non-TSO country (includes non-EU countries such as RU, MK, NO)
 
−
* crossborderpointtype, examples: 'In-country EU', 'Cross-Border EU|EU', 'Cross-Border EU|Non-EU', 'Cross-Border Non-EU|Non-EU', 'In-country Non-EU'
 
−
* directionkey, examples: 'exit', 'entry'
 
−
* eurelationship, examples: 'within EU', 'between EU and Non-EU', 'Non-EU'
 
−
* hasdata, examples: 'True', 'False'
 
−
* id, examples: '5DE-TSO-0014VTP-00045exitDE-TSO-0010', '5DE-TSO-0002VTP-00012exitDE-TSO-0007'
 
−
* isdoublereporting, examples: 'None', 'False', 'True'
 
−
* ispipeinpipe, examples: 'False', 'True'
 
−
* isvirtualizedcommercially, examples: 'False', 'True'
 
−
* isvirtualizedoperationally, examples: 'False', 'True'
 
−
* lastupdatedatetime, examples: '2022-07-05T19:07:06+02:00'
 
−
* operatorkey, examples: 'UK-TSO-0001', 'NL-TSO-0001', 'DE-TSO-0005'
 
−
* operatorlabel, examples: 'National Grid Gas', 'GTS', 'Gasunie Deutschland '
 
−
* pointkey, examples: 'VTP-00045', 'VTP-00011', 'VTP-00012'
 
−
* pointlabel, examples: 'THE VTP (DE)', 'VHP GASPOOL', 'VHP NetConnectGermany', 'Wallbach'
 
−
* pointtype, examples: 'Trading Point', 'Cross-Border Transmission IP within EU', 'Storage point', 'Distribution Point', 'Cross-Border Transmission IP between EU and Non-EU (import)', 'Cross-Border Transmission IP between EU and Non-EU'
 
−
* tsobalancingzone, examples: 'DE THE BZ', <nowiki>''</nowiki>, 'UK', 'Netherlands', 'TRF',
 
−
* tsocountry: country of TSO (includes non-EU countries such as RU, MK, NO)
 
−
* virtualreverseflow, examples: 'No', 'Yes'
 
−
 
 
−
===== Table: flows =====
 
−
* bookingplatformkey, examples: '', 'PRISMA', 'RBP', 'GSA', 'None'
 
−
* directionkey, examples: 'exit', 'entry'
 
−
* flowstatus, examples: 'Provisional', 'Confirmed'
 
−
* id, examples: '1Physical Flowday2020-12-312021-01-01CZ-TSO-0001ITP-00535exitkWh/d', '1Physical Flowday2020-12-312021-01-01CZ-TSO-0001ITP-00537entrykWh/d', '1Physical Flowday2020-12-312021-01-01CZ-TSO-0001ITP-00537exitkWh/d'
 
−
* idpointtype, examples: '0', '6', '17', '1', '18', '5', '15', '14', '3', '2'
 
−
* indicator, examples: 'Physical Flow'
 
−
* itemremarks, examples: 'None', 'Physical Flow Remark', 'No comment', 'According to Annex 1 Chapter 3 of Regulation CE n. 715/2009, physical flow data are published in aggregate. Capacity information per point are available at www.snamretegas.it at the following path: Business & Services/Thermal year/Transportation capacity.'
 
−
* lastupdatedatetime, examples: '2021-12-24T15:35:23+01:00', '2022-04-28T02:05:53+02:00', '2021-08-24T07:55:09+02:00', '2022-05-25T08:59:09+02:00', '2022-03-17T07:54:09+01:00', '2021-03-31T07:53:09+02:00', '2022-01-26T10:47:08+01:00'
 
−
* operatorkey, examples: 'UK-TSO-0001', 'DE-TSO-0009', 'NL-TSO-0001', 'GR-TSO-0001', 'DE-TSO-0005', 'BE-TSO-0001', 'PL-TSO-0002', 'DE-TSO-0001', 'CZ-TSO-0001', 'IT-TSO-0001'
 
−
* operatorlabel, examples: 'National Grid Gas', 'Open Grid Europe', 'GTS', 'DESFA', 'Gasunie Deutschland ', 'Fluxys Belgium', 'GAZ-SYSTEM', 'GASCADE Gastransport', 'NET4GAS', 'Snam Rete Gas'
 
−
* periodfrom, examples: '2021-04-20 06:00:00', '2021-09-23 06:00:00', '2021-09-24 06:00:00', '2021-09-25 06:00:00', '2021-09-26 06:00:00', '2021-09-27 06:00:00', '2021-09-28 06:00:00', '2021-09-29 06:00:00', '2021-04-08 06:00:00'
 
−
* periodto, examples: '2021-04-21 06:00:00', '2021-09-24 06:00:00', '2021-09-25 06:00:00', '2021-09-26 06:00:00', '2021-09-27 06:00:00', '2021-09-28 06:00:00', '2021-09-29 06:00:00', '2021-09-30 06:00:00', '2021-04-09 06:00:00'
 
−
* periodtype, examples: 'day'
 
−
* pointkey, examples: 'ITP-00535', 'ITP-00019', 'ITP-00537', 'ITP-00538', 'ITP-00015', 'ITP-00027', 'ITP-00049', 'ITP-00051', 'ITP-00061', 'ITP-00076'
 
−
* pointlabel, examples: 'Deutschneudorf EUGAL Brandov ', 'Überackern ABG (AT) / Überackern (DE)', 'Balassagyarmat (HU) / Velké Zlievce (SK)', 'Bunde (DE) / Oude Statenzijl (H) (NL) (GASCADE)', 'Cieszyn (PL) / Český Těšín (CZ)'
 
−
* pointtype, examples: 'Cross-Border Transmission IP within EU', 'Storage point', 'Distribution Point', 'Cross-Border Transmission IP between EU and Non-EU (import)', 'Aggregated Point - Final Consumers'
 
−
* tsoeiccode, examples: '21X-GB-A-A0A0A-7', '21X-DE-C-A0A0A-T', '21X-NL-A-A0A0A-Z', '21X-GR-A-A0A0A-G', '21X-DE-D-A0A0A-K', '21X-BE-A-A0A0A-Y', '21X-PL-A-A0A0A-B', '21X-DE-H-A0A0A-L', '21X000000001304L', '21X-IT-A-A0A0A-7'
 
−
* tsoitemidentifier, examples: '21Z0000000004839', '21Z000000000002E', '21Z0000000004863', '21Z000000000489Y', '21Z0000000000074', '21Z0000000000228', '21Z000000000044Z', '21Z000000000056S', '21Z000000000061Z', '21Z000000000074Q'
 
−
* unit, examples: 'kWh/d'
 
−
* value, examples: '0', '9048000', '1', '168000', '1000', '7', '2', '6', '39730478', '278'
 
−
 
 
−
===== Some tentative examples =====
 
−
 
 
−
Emulate German THE data for Denmark exports 16 July 2022 https://www.tradinghub.eu/en-gb/Publications/Transparency/Market-area-monitor
 
−
<code>
 
−
select '====', operator.operatorlabel, value, flows.directionkey, ispipeinpipe, * from flows left outer join operator on flows.pointkey = operator.pointkey and flows.directionkey = operator.directionkey where tsocountry = 'DE' and adjacentcountry = 'DK' and date(periodfrom)='2022-07-16' and value > 0 order by operator.operatorlabel;
 
−
</code>
 
−
Output still needs to be cleaned for a duplicated directions and tsoitemidentifier.
 
−
 
 
−
Confirm no flow through Nordstream 1 on 15 July 2022:
 
−
<code>
 
−
select '======================', ispipeinpipe, operator.operatorlabel, * from flows left join operator on flows.pointkey = operator.pointkey where tsocountry = 'DE' and adjacentcountry = 'RU' and date(periodto)='2022-07-15' order by operator.operatorlabel;
 
−
</code>
 
−
 
 
−
 
 
−
Give out consumption (apparently underestimation)
 
−
 
 
−
<code>
 
−
select left(operatorkey,2), date_trunc('month', periodto), round(sum(value)) from flows where pointkey in (select pointKey from operator where pointtype = 'Aggregated Point - Final Consumers') and flowstatus = 'Confirmed' group by left(operatorkey,2), date_trunc('month', periodto) order by left(operatorkey,2), date_trunc('month', periodto);
 
−
</code>
 
−
 
 
−
Sort by distribution points
 
−
<code>
 
−
select pointlabel, operatorlabel, sum(value) from flows where pointkey in (select pointKey from operator where pointtype = 'Aggregated Point - Final Consumers') and periodto > '2022-07-01' and periodto <= '2022-08-01' and left(operatorkey,2) = 'DE' group by operatorlabel, pointlabel order by operatorlabel ;
 
−
</code>
 
−
 
 
−
German cross-border transmissions
 
−
<code>
 
−
select left(operatorkey,2), date_trunc('month', periodto), round(sum(value)) from flows where pointkey in (select pointKey from operator where pointtype like 'Cross-Border Transmission IP%' and tsoCountry = 'DE' and isPipeInPipe = 'False') group by left(operatorkey,2), date_trunc('month', periodto) order by left(operatorkey,2), date_trunc('month', periodto);
 
−
</code>
 
−
 
 
−
Python scripts: <code>git clone https://git.embargo.energy/entsog.git</code>
 
  
−
Comments spotted: "Please note that the German dual system in VIP implementation leads to a Co-existence of Virtual Interconnection Points and Interconnection Points and may result in publication of the same physical flow at several points"
 
  
 
== EU: Energy prices ==
 
== EU: Energy prices ==
  
 
* {{Citation | vauthors=((European Commission)) | title=EU: Energy price dashboard - historical data | url=https://energy.ec.europa.eu/data-and-analysis/energy-prices-and-costs-europe/dashboard-energy-prices-eu-and-main-trading-partners_en | access-date=23 July 2022}}
 
* {{Citation | vauthors=((European Commission)) | title=EU: Energy price dashboard - historical data | url=https://energy.ec.europa.eu/data-and-analysis/energy-prices-and-costs-europe/dashboard-energy-prices-eu-and-main-trading-partners_en | access-date=23 July 2022}}

Latest revision as of 21:55, 25 October 2022

Here we list energy open data:

Germany[edit | edit source]

Trading Hub Europe[edit | edit source]

Bundesnetzagentur[edit | edit source]

BNetzA, Germany: Daily briefings about gas imports, storage, and consumption

Realized gas savings in 2022
Month 2021 2022 Diff. Diff.
TWh TWh TWh %
Jan 142.3 129 -13.3 -9.3 %
Feb 121.2 106 -15.2 -12.5 %
Mar 109.7 92.5 -17.2 -15.7 %
Apr 95.9 82.0 -13.9 -14.5 %
May 68.2 51.0 -17.2 -25.2 %
Jun 46.1 39.6 -6.5 -14.1 %
Jul 40.4 39.1 -1.3 -3.2 %
Aug 38.8
Sep 45.3
Oct 73.2
Nov 109.7
Dec 125.6
TOTAL 1016.4 -84.6


https://www.bundesnetzagentur.de/DE/Fachthemen/ElektrizitaetundGas/Versorgungssicherheit/aktuelle_gasversorgung/_downloads/08_August/220825_gaslage.pdf?__blob=publicationFile&v=3#page=5 https://www.bundesnetzagentur.de/DE/Fachthemen/ElektrizitaetundGas/Versorgungssicherheit/aktuelle_gasversorgung/start.html

BDEW[edit | edit source]

BDEW: Monthly gas consumption in Germany 2022 vs. 2021

Eurostat[edit | edit source]

EU[edit | edit source]

Gas storage inventory[edit | edit source]

Data not in AGSI summary:

Bruegel[edit | edit source]

ENTSOG gas data[edit | edit source]

ENTSOG example query to get physical flows on 01 March 2022:

Some analyses (experimental, feedback welcome):

ENTSOG data analysis


EU: Energy prices[edit | edit source]

Retrieved from "https://embargo.energy/index.php?title=Energy_data&oldid=1342"