{"id":10294,"date":"2022-07-17T16:49:53","date_gmt":"2022-07-17T14:49:53","guid":{"rendered":"https:\/\/www.swissfluxnet.ethz.ch\/?page_id=10294"},"modified":"2025-10-05T12:21:57","modified_gmt":"2025-10-05T10:21:57","slug":"database","status":"publish","type":"page","link":"https:\/\/swissfluxnet.ethz.ch\/index.php\/data\/database\/","title":{"rendered":"Database"},"content":{"rendered":"\n<p>&nbsp;<\/p>\n<h1>Visualizations<\/h1>\n<ul>\n<li>We use Grafana to visualize data from our stations in dashboards here: <a href=\"https:\/\/dataviews.swissfluxnet.ethz.ch\/d\/jvoH7Qxnk\/home?orgId=1&amp;from=now-30d&amp;to=now&amp;timezone=Africa%2FAlgiers\" target=\"_blank\" rel=\"noopener\">dataviews.swissfluxnet.ethz.ch<\/a><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h1>Info<\/h1>\n<ul>\n<li>The Grassland Sciences group uses the time series database (TSDB) <code>InfluxDB<\/code> to store eddy covariance fluxes and auxiliary (meteorological) measurements.<\/li>\n<li>We are currently (Mar 2024) working to make data stored in the database accessible to all.<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h1>Server<\/h1>\n<ul>\n<li>The server is running the&nbsp;<code>Red Hat Enterprise Linux (RHEL) version 8.9<\/code> (<a href=\"https:\/\/www.redhat.com\/en\/technologies\/linux-platforms\/enterprise-linux\">link<\/a>) operating system.<\/li>\n<li><code>InfluxDB<\/code> v2 (<a href=\"https:\/\/www.influxdata.com\/\" target=\"_blank\" rel=\"noopener\">link<\/a>) is running on the server and used to <span style=\"text-decoration: underline;\">store<\/span> data.<\/li>\n<li><code>Grafana<\/code> v10 (<a href=\"https:\/\/grafana.com\/\" target=\"_blank\" rel=\"noopener\">link<\/a>) is running on the server and used to <span style=\"text-decoration: underline;\">visualize<\/span> data. <a href=\"https:\/\/dataviews.swissfluxnet.ethz.ch\" target=\"_blank\" rel=\"noopener\">Recent data can be viewed here<\/a>.<\/li>\n<li><code>POET<\/code> scripts (see below) are running on the server automatically (cronjobs), but can be started from a local machine on demand (e.g. to upload legacy data).<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h1>Scripts<\/h1>\n<p>Several scripts are necessary to upload data to the database. The scripts are part of the <code>POET<\/code> project.<\/p>\n<ul>\n<li><code>dataflow<\/code>: <a href=\"https:\/\/github.com\/holukas\/dataflow\" target=\"_blank\" rel=\"noopener\">source code<\/a> | Automatically (cronjob) scan server for files and upload data to database. Uses <code>dbc-influxdb<\/code> to upload files.<\/li>\n<li><code>dbc-influxdb<\/code>: <a href=\"https:\/\/github.com\/holukas\/dbc-influxdb\" target=\"_blank\" rel=\"noopener\">source code<\/a> | Database communication: show, download and upload data. Works with InfluxDB 2.x.<\/li>\n<li><code>configs<\/code>: <a href=\"https:\/\/github.com\/holukas\/configs\" target=\"_blank\" rel=\"noopener\">source code<\/a> | Configuration files for <code>POET<\/code> scripts.<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h1>Timestamps<\/h1>\n<ul>\n<li>All timestamps refer to the <strong>END<\/strong> of the averaging period.<\/li>\n<li>All data are stored in <code>UTC<\/code> time.<\/li>\n<li>Data are originally logged in <code>CET<\/code>, but converted to <code>UTC<\/code> during upload.<\/li>\n<li>Data can be downloaded from the database using the script <code>dbc-dataflow<\/code>. During download, timestamps are converted to the desired timezone, normally to <code>CET<\/code> because data were originally logged in <code>CET<\/code>.<\/li>\n<li><code>Grafana<\/code> by default converts <code>UTC<\/code> time from the database to local browser time (during European summer the browser time would typically show <code>CEST<\/code>, which is UTC+02:00 hours) when a visualization is created. However, the desired timezone in which data should be shown can be selected via the drop-down menu.<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h1>General Dataflow<\/h1>\n<p>The general dataflow in Swiss FluxNet can be seen on the page <a href=\"https:\/\/www.swissfluxnet.ethz.ch\/index.php\/data\/general-dataflow-2\/\">General Dataflow<\/a>.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>&nbsp; Visualizations We use Grafana to visualize data from our stations in dashboards here: dataviews.swissfluxnet.ethz.ch &nbsp; Info The Grassland Sciences group uses the time series database (TSDB) InfluxDB to store eddy covariance fluxes and auxiliary (meteorological) measurements. We are currently (Mar 2024) working to make data stored in the database . . . <a class=\"readmore-link\" href=\"https:\/\/swissfluxnet.ethz.ch\/index.php\/data\/database\/\">Read more<\/a><\/p>\n","protected":false},"author":3,"featured_media":0,"parent":3324,"menu_order":2,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"_lmt_disableupdate":"no","_lmt_disable":"no","ngg_post_thumbnail":0,"footnotes":""},"coauthors":[314],"class_list":["post-10294","page","type-page","status-publish","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/swissfluxnet.ethz.ch\/index.php\/wp-json\/wp\/v2\/pages\/10294","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/swissfluxnet.ethz.ch\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/swissfluxnet.ethz.ch\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/swissfluxnet.ethz.ch\/index.php\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/swissfluxnet.ethz.ch\/index.php\/wp-json\/wp\/v2\/comments?post=10294"}],"version-history":[{"count":19,"href":"https:\/\/swissfluxnet.ethz.ch\/index.php\/wp-json\/wp\/v2\/pages\/10294\/revisions"}],"predecessor-version":[{"id":13214,"href":"https:\/\/swissfluxnet.ethz.ch\/index.php\/wp-json\/wp\/v2\/pages\/10294\/revisions\/13214"}],"up":[{"embeddable":true,"href":"https:\/\/swissfluxnet.ethz.ch\/index.php\/wp-json\/wp\/v2\/pages\/3324"}],"wp:attachment":[{"href":"https:\/\/swissfluxnet.ethz.ch\/index.php\/wp-json\/wp\/v2\/media?parent=10294"}],"wp:term":[{"taxonomy":"author","embeddable":true,"href":"https:\/\/swissfluxnet.ethz.ch\/index.php\/wp-json\/wp\/v2\/coauthors?post=10294"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}