We historically used the sonicread<\/a> script for logging eddy covariance raw data files, a practice that began in 2005. In 2024, we transitioned the first site (CH-FRU) to rECord, a new data logging script based on Each EddyPro needs to know in what sequence data are coming in, i.e. the sequence in which data are stored in a file. Whenever an instrument is added or removed at the site, the data stored in the raw binary files changes. For example, most of the time the regular structure of a raw binary file consists of (a) sonic data and (b) IRGA data. When a QCL system is installed at the site, the data of this new instrument are added to the sonic and IRGA data, e.g. after 6 columns of sonic data and 8 columns of IRGA data follow 5 columns of QCL data <\/p>\n\r\n\r\n\r\n Time periods with different instrumental setup have to be calculated separately.<\/strong> Therefore, there is a subpage called EC Raw Binary Data (XX-XXX) for each site with tables, showing the name of the first\/last raw binary file with the instrumental setting of the respective time period.<\/p>\r\n\r\n\r\n\r\n Data from the instruments arrive in blocks at the logger. These datablocks arrive sequentially: first the datablock from instrument 1, then from instrument 2, etc … Datablocks are then stored to data files in the order as they arrive at the logger.<\/p>\r\n\r\n\r\n\r\n See<\/strong>: Datablocks in bico<\/a><\/p>\r\n\r\n\r\n\r\n In the binary files these datablocks are compressed, which means that one line of data records (i.e., all records for a specific timestamp) can consist of a varying number of Bytes and data columns. For example, if all data are available at a specific timestamp, one line of data records might consist of Let’s say we have a site with one sonic anemometer (Solent R3-50) and one gas analyzer (LI-COR 7200, i.e. IRGA72). Measurements are performed at 20Hz.<\/p>\r\n\r\n\r\n\r\n In this example, the setup consists of 1 sonic anemometer and 2 different gas analyzers.<\/p>\r\n\r\n\r\n\r\n The HEADER is at the start of the file. After the HEADER, data from the instruments are stored sequentially. One record consists of the sonic data block + IRGA75 data block + IRGA72 data block.<\/p>\r\n When converted to an ASCII file, the following columns are found in the converted file. Missing data columns (i.e. missing in the compressed raw binary files) have the numeric value -9999 instead of the (missing) measurement.<\/p>\r\nsonicread<\/code>.<\/p>\r\nsonicread<\/code> eddy covariance raw data binary file contains a header section, and then data from the sonic anemometer and the gas analyzer(s) over a time period of 6 hours. EddyPro cannot handle our EC raw data files because they are compressed, and therefore they need to be converted into an uncompressed format that EddyPro can understand. In our case, the compressed raw binary data files are converted to uncompressed ASCII csv files using the Python script bico (Binary Converter)<\/a>.<\/p>\r\n\r\n\r\nWhat Does Compressed vs Uncompressed Mean?<\/h2>\r\n\r\n\r\n\r\n
26 Bytes<\/code> and 12 data columns. If the data for one instrument are missing, then one line of data records may only consist of 14 Bytes<\/code> and 8 data columns. After bico<\/code> converted the raw data file to ASCII, the files are uncompressed and each line of data records, no matter if data are missing or not, consists of the full 12 data columns: the format is therefore regular.<\/p>\r\n\r\n\r\n\r\nFor example<\/h3>\r\n\r\n\r\n\r\n
\r\n
12 Bytes<\/code>.<\/li>\r\n\r\n\r\n\r\n26 Bytes<\/code>. In this case, the 26 Bytes are added to the raw data file right after the 12-Bytes data block from the sonic anemometer. However, sometimes the IRGA72 data is missing, and in this case the data block from the IRGA72 is only 2 Bytes<\/code> in size. In this case, the 2 Bytes (instead of 26 Bytes) are added to the 12-Bytes data block from the sonic.<\/li>\r\n\r\n\r\n\r\n38 Bytes<\/code> (12 Bytes + 26 Bytes). When sonic data are available but IRGA72 data are missing, then the record is 14 Bytes<\/code> (12 Bytes + 2 Bytes).<\/li>\r\n\r\n\r\n\r\nExample Structure<\/h2>\r\n\r\n\r\n\r\n
\r\n
12 Bytes<\/code> (big endian short integer).<\/li>\r\n2-16 Bytes<\/code>. In case of 2 Bytes, data from the IRGA75 are missing.<\/li>\r\n2-25 Bytes<\/code>. In case of 2 Bytes, data from the IRGA72 are missing.<\/li>\r\n<\/ol>\r\n\r\n
53 Bytes<\/code> (12 + 16 + 25 Bytes).<\/li>\r\n39 Bytes<\/code> (12 + 2 + 25 Bytes)<\/li>\r\n30 Bytes<\/code> (12 + 16 + 2 Bytes)<\/li>\r\n16 Bytes<\/code> (12 + 2 + 2 Bytes)<\/li>\r\n<\/ul>\r\n<\/div>\r\n\r\n\r\n\r\n