GruanToolRS92 (gt92)

Command line tool for handling radiosonde data files

Details

ID

gt92

Title

GruanToolRs92

Author

Michael Sommer (GRUAN Lead Centre)

Brief description

Command line tool which can be used to handle a couple of radiosounding-specific file formats. Files can be tested, converted, extracted and more.

Version

0.4.11_07 (2019-09-12)

Status

Stable

 

Description

  • GRUAN-IN-4: Brief Description of GruanToolRs92 (gt92), DRAFT v0.8 (2018-01-04)

History

  • 0.4.11_07 (2019-09-12)
    • bug fixed: crash during CSV export
  • 0.4.11_06 (2019-08-28)
    • add options --export-col-filter and --reverse-col-filter and --csv-delimiter
  • 0.4.11_05 (2019-07-31)
    • change: update MWX file definition for MW41 v2.15
  • 0.4.11_04 (2019-05-07)
    • bug fixed: handling of wrong date format in GSFZ files
  • 0.4.11_03 (2019-04-02)
    • bug fixed: add 'missing_value' to several columns of XDATA type 'SKYDEW_3F'
  • 0.4.11_02 (2019-03-29)
    • bug fixed: repaired column names of XDATA type 'SKYDEW_3F'
    • rename file type SGNC-RAW to IGNC-RAW
  • 0.4.11_01 (2019-03-28)
    • bug fixed: decode abnormal package length 35 of XDATA type 'SKYDEW_3F'
  • 0.4.11    (2019-03-28)
    • add handling of XDATA type 'SKYDEW_3F' (SKYDEW) packets
    • compiled with Java 8
  • 0.4.10_09 (2019-03-19)
    • add handling of file type SGNC-RAW
  • 0.4.10_08 (2019-02-07)
    • better handling of maybe corrupt system-wide installed netcdf libraries (print messages)
    • compiled with Java SE 11
  • 0.4.10_07 (2018-12-17)
    • bug fixed: wrong XDATA match if both PCFH and CFH are flight together using one data stream (MWX file)
  • 0.4.10_06 (2018-09-27)
    • change: update MWX file definition for MW41 v2.11
  • 0.4.10_05 (2018-09-27)
    • change: SYNC_* columns added/activated for table EDT_XDATA_GRIDDED of file type MWX
  • 0.4.10_04 (2018-07-24)
    • bug fixed: correct global attribute key 'g.Instrument.PCFH.MirNum' to '...SerialNo' in case of handling of XDATA type 'PCFH_3C' (PCFH)
    • bug fixed: add columns PCFH_Trsurf1, PCFH_Trsurf2, PCFH_2_Trsurf1, PCFH_2_Trsurf2 to table 'EDT_XDATA_GRIDDED' in case of handling of XDATA type 'PCFH_3C' (PCFH)
  • 0.4.10_03 (2018-07-13)
    • change: handling of XDATA type `PCFH_3C' (PCFH) packets 0x01 to 0x07, add TimestampXX columns
  • 0.4.10_02 (2018-05-31)
    • add option --checksum-type
      • with following possibilities: CRC-32, MD2, MD5, SHA-1, SHA-224, SHA-256, SHA-384, SHA-512
      • with CRC-32 as default (like before)
  • 0.4.10_01 (2018-05-31)
    • change: update MWX file definition for MW41 v2.9
  • 0.4.10    (2018-05-19)
    • change: handling of XDATA type 'PCFH_3C' (PCFH) packets 0x00 to 0x07 (follow doc v1.2, 2018-04-05)
  • 0.4.9_02  (2018-04-16)
    • bug fixed: handle 'NULL' value in column DataSrvTime, table AdditionalSensorData of MWX file
  • 0.4.9_01  (2018-03-06)
    • add two hidden options for MWX
  • 0.4.9     (2018-02-28)
    • add handling of XDATA type 'PCFH_3C' (PCFH) packets 0x00 to 0x07
  • 0.4.8_03  (2018-01-05)
    • change: option --no-info-key is now also related to test messages (use of -t)
    • change: option --no-info-key is now also related to identify messages (use of -y)
    • change: option --no-info-key is now also related to checksum messages (use of -k)
    • add option --no-source for combination with -x (extract) and file type GNC-RAW
  • 0.4.8_02  (2017-12-08)
    • change: improved file check possibilities (value options added to attribute description)
    • bug fixed: test messages with file path now (instead 'null')
  • 0.4.8_01  (2017-11-01)
    • bug fixed: better error handling during identify DC3DB (in case of corrupt files)
  • 0.4.8     (2017-10-23)
    • change: enable options --no-meta-root and --meta-sep for file type DC3DB
  • 0.4.7_03  (2017-03-17)
    • bug fixed: CFH optics temperature fixed
  • 0.4.7_02  (2017-02-15)
    • bug fixed: using attribute 'g_format_format' if type BYTE or SHORT during converting GNC-DATA to LDF (crash fixed)
  • 0.4.7_01  (2016-11-24) - bug fixed: decoding TINT and ILAMP of XDATA type 'FLASH_3D_0' (FLASH-B)
    • add option --no-csv-nan for MWX, DC3DB, GNC-RAW, GNC-DATA
  • 0.4.7     (2016-11-23)
    • add handling of non-valid xml-chars (e.g. �) for internal XML file 'AdditionalSensorData.xml' of MWX file
  • 0.4.6_13  (2016-11-23)
    • bug fixed: wrong frame length of XDATA type 'FLASH_3D_0' (FLASH-B)
  • 0.4.6_12  (2016-11-22)
    • change handling of XDATA type 'FLASH_3D_0' (FLASH-B)
  • 0.4.6_11  (2016-11-16)
    • add handling of XDATA type 'FLASH_3D_0' (FLASH-B)
  • 0.4.6_10  (2016-11-16)
    • add handling of XDATA type 'FLASH_3D' (FLASH-B)
  • 0.4.6_09  (2016-11-10)
    • bug fixed: handling of data type BYTE with file type MWX
    • change: update MWX file definition for MW41 v2.5
    • add option --no-info-key
  • 0.4.6_08  (2016-11-09)
    • bug fixed: handling of XDATA with file type STRATO-ZIP
  • 0.4.6_07  (2016-11-08)
    • bug fixed: wrong results with a list of INFO keys in combination with HISTORY, HISTORY_LAST, HISTORY_FULL
    • bug fixed: wrong results with -y in case of file types without version
    • change: include tool version in converted GNC-RAW files (gatts: history, g.Product.ProducerTool)
    • add handling of XDATA type 'COVRAD_05_0110' (Cover of radiation chamber)
    • change: improved automatical detection of number type for MWX columns (FLOAT, DOUBLE, LONG)
  • 0.4.6_06  (2016-09-21)
    • bug fixed: always add global attribute 'g.Aliases' during converting to GNC-RAW
    • add new INFO keys: HISTORY_LAST, HISTORY, HISTORY_FULL
    • bug fixed: convert STRATO-ZIP (SkySonde) to GNC-RAW
    • bug fixed: use correct data type INT for COBALD counters
    • add additional meta-data (time) for STRATO RAW decoding & gidding
  • 0.4.6_05  (2016-09-13)
    • bug fixed: handling GSFZ files with extension *.zip
    • bug fixed: handling GNC-DATA files with missing ALIASES
  • 0.4.6_04  (2016-09-12)
    • bug fixed: handling several XData sensors of same type with MWX (CFH, CFH_2, ...)
  • 0.4.6_03  (2016-09-07)
    • bug fixed: handling GSFZ files
  • 0.4.6_02  (2016-08-29)
    • add BOM handling for internal XML files of MWX file
  • 0.4.6_01  (2016-08-01)
  • bug fixed: extract first 'correct' value with calculation
  • 0.4.6     (2016-08-01)
    • change options
      • add --nc-old (enable old version, default before)
      • remove --nc4, --nc4-v2 (--nc4-v2 is default now)
      • add --no-meta-valid, --meta-osep, --no-meta-root, --no-meta-group (use defaults for each file type)
  • 0.4.5_11  (2016-07-29)
    • add calculation to table columns (-> store result as attribute)
    • change gridded table of MWX
      • add additional columns for GPS
      • remove 'meta-data' columns, e.g. CFH, OIF411
      • add calc to gridded table, e.g. CFH, OIF411 (-> store result as global attributes)
  • 0.4.5_10  (2016-07-26)
    • bug fixed: use XML version 1.1 for parsing internal MWX files
  • 0.4.5_09  (2016-06-28)
    • bug fixed: use correct base time for all columns to grid (DC3DB and MWX)
    • add a gridded table to DC3DB
  • 0.4.5_08  (2016-06-22)
    • bug fixed: use correct path to write LDF files
    • bug fixed: first try to handle too short formats for LDF converting (negative ascent speed)
  • 0.4.5_07  (2016-06-10)
    • bug fixed: static reading of file descriptions of MWX, DC3DB, STRATO, GSFZ
  • 0.4.5_06  (2016-05-23)
    • new compiled
  • 0.4.5_05  (2016-02-26)
    • add XData specific meta-data (MWX only)
      • g.Instrument.XData, g.Instrument.XDataSensors
    • rename XData instrument RSA411 to OIF411
  • 0.4.5_04  (2016-02-25)
    • change some internal MWX things
      • e.g. add correct positions to a couple of tables
    • change base of gridded table of MWX to RawPtu
  • 0.4.5_03  (2016-02-23)
    • change column names of XDATA columns of MWX
      • Cfh_... to CFH_...
      • several identical instruments possible, like CFH_..., CFH_2_...
    • bug fixed: rename meta-data keys correctly to original once (if required)
  • 0.4.5_02  (2016-02-23)
    • add functionality of '--json-result' to option --info / -i
    • change print result (without --json-result) to <file/name> = <result>
    • add result line for option --extract / -x (EXTRACTED or NOT_EXTRACTED)
  • 0.4.5_01  (2016-02-22)
    • bug fixed: data type changed from BYTE to SHORT of column 'PumpCurrent' of map 'RawOzone' of format 'MWX'
    • bug fixed: unit 'seconds since ...' of column 'RadioRxTimePk' is now correct in case of converting MWX to GNC-RAW
    • add argument --export-table-filter
    • add argument --import-table-filter
    • add argument --reverse-table-filter
    • mark argument --table-filter as 'deprecated'
  • 0.4.5     (2016-02-18)
    • some changes at file descriptions (MWX, DC3DB, STRATO-ZIP)
    • with option -t (--test) a file can be tested against a file description
      • this is very helpful to test created GDP files (GRUAN data product NetCDF file)
      • all test result messages are printed like other results (at console 'out' without/with JSON format)
  • 0.4.4_05  (2016-02-16)
    • add automatically extracting of relevant native libraries to sub-dir 'gt92-native-libs' of temp dir of system
      • in case of no rights to write in own program folder
  • 0.4.4_04  (2016-02-16)
    • add argument --test-description=<FILE> (for -t)
    • bug fixed: use now argument --no-xdata (true/false) with --convert
  • 0.4.4_03  (2016-02-03)
    • optimization of messages (reading MWX)
    • add new table 'StabilityIndex' to MWX file description
    • change start time for gridded MWX tables to Soundings.BeginTime or if not available Soundings.RadioResetTime
    • bug fixed: wrong name of one V7 value
  • 0.4.4_02  (2016-02-02)
    • bug fixed: optimize gridded data map EDT_XDATA_GRIDDED of MWX file
      • set grid base to AdditionalSensors
      • add columns from RawPtu, GSupport, SynchronizedSoundingData
      • optimize messages if columns/maps are missing
  • 0.4.4_01  (2016-02-01)
    • add gridded data map EDT_XDATA_GRIDDED to MWX extracting (including data from PtuResults & GpsResults & AdditionalSensors)
    • bug fixed: wrong decoding of XDATA instrument type (use HEX decoding now)
  • 0.4.4     (2016-01-13)
    • include native netcdf libraries
      • for os-arch: linux-x86, linux-x86-64, win32-x86, win32-x86-64
      • automatically extracting of relevant native libraries to sub-dir `lib' (in case of bin to '../lib')
    • optimise netcdf logging (bind to log4j)
  • 0.4.3     (2016-01-06)
    • add argument --itemize / -z
    • add argument --identify / -y
    • add argument --no-msg-header
    • add argument --no-silent (for -t, -z, -k, -y)
    • add argument --json-result
    • improved automatically silent mode for -t, -z, -k, -y
    • restructured code of processing routines
  • 0.4.2     (2015-12-16)
    • add argument --corrupt-dir=<DIR>
    • add argument --copy-corrupt
    • add argument --no-msg-error
    • add argument --no-msg-info
  • 0.4.1_01  (2015-12-15)
    • improve source documentation
    • bug fixed: remove tmp file in case of an error
    • add final info about corrupt files
  • 0.4.1     (2015-12-14)
    • add argument --skip-convert
    • add argument -i <KEY> / --info=<KEY> with a couple of possible keys:
      • VERSION, VERSION_NUMBER, VERSION_DATE,
      • NAME, NAME_SHORT, NAME_LONG,
      • COPYRIGHT,
      • AUTHOR, AUTHOR_NAME, AUTHOR_EMAIL,
      • FILE_TYPES
    • new error message in case of missing required option
    • add argument -w <DIR> / --work-dir=<DIR>
    • add argument --no-file-type-check
  • 0.4.0     (2015-12-11)
    • add argument --separate-dir
    • improve handling with DC3DB files (e.g. wrong SN, additional log info, ...)
    • use working directory to create NC files

References