Public:MDF Format - Revision history

Nebain: /* 2.3 The parameter identifier line */

Nebain — Wed, 16 Dec 2009 22:11:28 GMT

2.3 The parameter identifier line

←Older revision		Revision as of 22:11, 16 December 2009
Line 53:		Line 53:
	This line indicates all the parameters stored in the data records. The first three identifiers in the parameter identifiers line must be STID, STNM, and TIME, and they must be in that order. They are not counted in the count of parameters used in the MDF file. Therefore, the number of identifiers is always 3 more than the number of parameters indicated in the number of parameters and time stamp line.		This line indicates all the parameters stored in the data records. The first three identifiers in the parameter identifiers line must be STID, STNM, and TIME, and they must be in that order. They are not counted in the count of parameters used in the MDF file. Therefore, the number of identifiers is always 3 more than the number of parameters indicated in the number of parameters and time stamp line.

-	Parameter IDs are typically four characters, and have always been separated by four spaces (except for STID and STNM, which are separate by three spaces). Most of our software is dependent on the four-character restriction, and it's possible that some older software is dependent on the spacing.	+	Parameter IDs are typically four characters, and are typically separated by four spaces (except for STID and STNM, which are separate by three spaces). Certain files may use longer or shorter parameter IDs, and parsers should not assume that parameter IDs are always 4 characters, nor that there will always be 4 spaces between parameters.

	----		----

Jpbostic: Note that compressed MDF is retired

Jpbostic — Thu, 19 Feb 2009 15:03:45 GMT

Note that compressed MDF is retired

←Older revision		Revision as of 15:03, 19 February 2009
Line 29:		Line 29:
	101 !Copyright (c) 1995 Oklahoma Climatological Survey.		101 !Copyright (c) 1995 Oklahoma Climatological Survey.

-	In an MDF file, the version number is always 101 since there have been no modifications of the MDF format since the use of version numbers began. MDF and MTS files are defined internally to have an initial version of 100. Another number is added to indicate what format the MDF or MTS file is created with, in this case the number is 1, which indicates that it is an uncompressed version of the file. Compressed MDF/MTS files are currently labeled with an even number (currently 102). If the file formats were ever modified, the version numbers on both files would be upgraded making sure that the uncompressed version was odd (e.g. 103, 105, 139, etc.) and the compressed version was even (e.g. 104, 106, 140, etc.). The key to remember is '''odd numbers indicate uncompressed files while even numbers indicate compression was used'''.	+	In an MDF file, the version number is always 101 since there have been no modifications of the MDF format since the use of version numbers began. MDF and MTS files are defined internally to have an initial version of 100. Another number is added to indicate what format the MDF or MTS file is created with, in this case the number is 1, which indicates that it is an uncompressed version of the file. Compressed MDF/MTS files are currently labeled with an even number (currently 102). If the file formats were ever modified, the version numbers on both files would be upgraded making sure that the uncompressed version was odd (e.g. 103, 105, 139, etc.) and the compressed version was even (e.g. 104, 106, 140, etc.). The key to remember is '''odd numbers indicate uncompressed files while even numbers indicate compression was used'''. Also, note that OCS/Mesonet ceased production/usage of the compressed MDF format around 2001.

	The copyright statement is ignored by our current software since it is superfluous to data gathering. Currently there are no plans to parse or display any portion of the copyright statement. It is there solely to protect OCS and OU intellectual property.		The copyright statement is ignored by our current software since it is superfluous to data gathering. Currently there are no plans to parse or display any portion of the copyright statement. It is there solely to protect OCS and OU intellectual property.

Jpbostic: remove the redundant references to "uncompressed" MDF - that's all we use now

Jpbostic — Thu, 19 Feb 2009 14:53:38 GMT

remove the redundant references to "uncompressed" MDF - that's all we use now

←Older revision		Revision as of 14:53, 19 February 2009
Line 8:		Line 8:
	=== 1. Introduction ===		=== 1. Introduction ===

-	Uncompressed Mesonet data files (MDF) and Mesonet time series (MTS) files were the first type of files generated for mass distribution to customers. Their tabular layout is excellent for both printing and importing into spreadsheets and/or word processing documents. The specification for this particular kind of MDF file has undergone considerable modification since it's inception, but generally holds true to it's original format.	+	Mesonet data files (MDF) and Mesonet time series (MTS) files were the first type of files generated for mass distribution to customers. Their tabular layout is excellent for both printing and importing into spreadsheets and/or word processing documents. The specification for this particular kind of MDF file has undergone considerable modification since it's inception, but generally holds true to it's original format.

	----		----
Line 14:		Line 14:
	=== 2. The Format ===		=== 2. The Format ===

-	Uncompressed MDF files should only contain standard, printable ASCII characters. Line breaks can be either DOS (carraige return and line feed characters), Mac (carraige return character only), or Unix (linefeed character only) format, but should be consistent. Uncompressed files should be transmittable in FTP or HTTP text mode without character conversions, aside from the line breaks.	+	MDF files should only contain standard, printable ASCII characters. Line breaks can be either DOS (carraige return and line feed characters), Mac (carraige return character only), or Unix (linefeed character only) format, but should be consistent. They should be transmittable in FTP or HTTP text mode without character conversions, aside from the line breaks.

-	An uncompressed MDF file consists of four key elements:	+	An MDF file consists of four key elements:

	# A version number which identifies the general format as well as whether the file is compressed or uncompressed.		# A version number which identifies the general format as well as whether the file is compressed or uncompressed.
Line 25:		Line 25:
	==== 2.1 The Version Number ====		==== 2.1 The Version Number ====

-	This is a version number line from an uncompressed MDF file downloaded from the Mesonet BBS:	+	This is a version number line from an MDF file:

	101 !Copyright (c) 1995 Oklahoma Climatological Survey.		101 !Copyright (c) 1995 Oklahoma Climatological Survey.

-	In an uncompressed MDF file, the version number is always 101 since there have been no modifications of the uncompressed MDF format since the use of version numbers began. MDF and MTS files are defined internally to have an initial version of 100. Another number is added to indicate what format the MDF or MTS file is created with, in this case the number is 1, which indicates that it is an uncompressed version of the file. Compressed MDF/MTS files are currently labeled with an even number (currently 102). If the file formats were ever modified, the version numbers on both files would be upgraded making sure that the uncompressed version was odd (e.g. 103, 105, 139, etc.) and the compressed version was even (e.g. 104, 106, 140, etc.). The key to remember is '''odd numbers indicate uncompressed files while even numbers indicate compression was used'''.	+	In an MDF file, the version number is always 101 since there have been no modifications of the MDF format since the use of version numbers began. MDF and MTS files are defined internally to have an initial version of 100. Another number is added to indicate what format the MDF or MTS file is created with, in this case the number is 1, which indicates that it is an uncompressed version of the file. Compressed MDF/MTS files are currently labeled with an even number (currently 102). If the file formats were ever modified, the version numbers on both files would be upgraded making sure that the uncompressed version was odd (e.g. 103, 105, 139, etc.) and the compressed version was even (e.g. 104, 106, 140, etc.). The key to remember is '''odd numbers indicate uncompressed files while even numbers indicate compression was used'''.

	The copyright statement is ignored by our current software since it is superfluous to data gathering. Currently there are no plans to parse or display any portion of the copyright statement. It is there solely to protect OCS and OU intellectual property.		The copyright statement is ignored by our current software since it is superfluous to data gathering. Currently there are no plans to parse or display any portion of the copyright statement. It is there solely to protect OCS and OU intellectual property.
Line 47:		Line 47:
	==== 2.3 The parameter identifier line ====		==== 2.3 The parameter identifier line ====

-	The next line in an uncompressed MDF/MTS file contains a list of parameter identifiers delimited by spaces:	+	The next line in an MDF/MTS file contains a list of parameter identifiers delimited by spaces:

	STID STNM TIME RELH TAIR WSPD WVEC WDIR		STID STNM TIME RELH TAIR WSPD WVEC WDIR
Line 188:		Line 188:
	=== 4. Conclusion ===		=== 4. Conclusion ===

-	The uncompressed MDF and MTS format is the one most used by those wishing to supply their own data for certain parameters as it lends itself to both use in software and for viewing in text editors and/or spreadsheets. The current revisions of the plugins (3.1 as of this writing) allow for user-defined parameter identifiers and values, however conversion and units are not supported for these user-defined identifiers. Therefore, if one wishes to plot some data using software developed within OCS, it is highly recommended to use the uncompressed MDF/MTS specification.	+	The MDF and MTS format is the one most used by those wishing to supply their own data for certain parameters as it lends itself to both use in software and for viewing in text editors and/or spreadsheets. The current revisions of the plugins (3.1 as of this writing) allow for user-defined parameter identifiers and values, however conversion and units are not supported for these user-defined identifiers. Therefore, if one wishes to plot some data using software developed within OCS, it is highly recommended to use the MDF/MTS specification.

Nebain: Precipitation is given in millimeters, not inches.

Nebain — Tue, 02 Dec 2008 14:55:51 GMT

Precipitation is given in millimeters, not inches.

←Older revision		Revision as of 14:55, 2 December 2008
Line 116:		Line 116:
	Precipitation		Precipitation
	\| valign="top" \|		\| valign="top" \|
-	inches	+	millimeters
	\|-		\|-
	\| valign="top" \|		\| valign="top" \|

Jpbostic: remove email address to avoid spam to developer

Jpbostic — Sat, 08 Nov 2008 06:01:11 GMT

remove email address to avoid spam to developer

←Older revision		Revision as of 06:01, 8 November 2008
Line 4:		Line 4:
	{\| width="470"		{\| width="470"
	\|		\|
-	'''original: '''2/1/99''' author: '''[mailto:brad@mesonet.org Brad]	+	'''original: '''2/1/99'''

	=== 1. Introduction ===		=== 1. Introduction ===

Jpbostic: /* 2.3 The parameter identifier line */

Jpbostic — Fri, 23 May 2008 15:52:53 GMT

2.3 The parameter identifier line

←Older revision		Revision as of 15:52, 23 May 2008
Line 53:		Line 53:
	This line indicates all the parameters stored in the data records. The first three identifiers in the parameter identifiers line must be STID, STNM, and TIME, and they must be in that order. They are not counted in the count of parameters used in the MDF file. Therefore, the number of identifiers is always 3 more than the number of parameters indicated in the number of parameters and time stamp line.		This line indicates all the parameters stored in the data records. The first three identifiers in the parameter identifiers line must be STID, STNM, and TIME, and they must be in that order. They are not counted in the count of parameters used in the MDF file. Therefore, the number of identifiers is always 3 more than the number of parameters indicated in the number of parameters and time stamp line.

-	Parameter IDs must be four characters, and have always been separated by four spaces (except for STID and SNUM, which are separate by three spaces). Most of our software is dependent on the four-character restriction, and it's possible that some older software is dependent on the spacing.	+	Parameter IDs are typically four characters, and have always been separated by four spaces (except for STID and STNM, which are separate by three spaces). Most of our software is dependent on the four-character restriction, and it's possible that some older software is dependent on the spacing.

	----		----

Jpbostic: /* 2.2 The number of parameters and time stamp line */

Jpbostic — Fri, 23 May 2008 15:44:45 GMT

2.2 The number of parameters and time stamp line

←Older revision		Revision as of 15:44, 23 May 2008
Line 39:		Line 39:
	5 1994 07 07 00 00 00		5 1994 07 07 00 00 00

-	The first number (13) indicates the number of measured parameters that can be found in the MDF or MTS file. This number does not include the first three parameters listed in the parameter identifier line or in the data records (station identifier, station number, and time).	+	The first number (5) indicates the number of measured parameters that can be found in the MDF or MTS file. This number does not include the first three parameters listed in the parameter identifier line or in the data records (station identifier, station number, and time).

	The other numbers in the line indicate year, month, day, hour, minute, and second respectively. This time is a '''base time''' for the file and may or may not represent any actual time used in the data itself. More on this later.		The other numbers in the line indicate year, month, day, hour, minute, and second respectively. This time is a '''base time''' for the file and may or may not represent any actual time used in the data itself. More on this later.

Ddemko: /* 3. Handling Missing Data */

Ddemko — Wed, 09 Apr 2008 05:47:31 GMT

3. Handling Missing Data

←Older revision		Revision as of 05:47, 9 April 2008
Line 173:		Line 173:
	-994		-994
	\| valign="top" \|		\| valign="top" \|
-	Data witheld due to out-of-range value overflowing the maximum column width.	+	Data withheld due to out-of-range value overflowing the maximum column width.
	\|}		\|}

Ddemko: /* 3. Handling Missing Data */

Ddemko — Wed, 09 Apr 2008 05:38:10 GMT

3. Handling Missing Data

←Older revision		Revision as of 05:38, 9 April 2008
Line 136:		Line 136:
	Whenever a record has no value for a certain parameter, for any reason, a missing data value is substituted for that value. This missing data value must be less than -900. Any value greater than -900 (e.g. -888 or -777) will be interpreted as an actual data value by the current revisions of the software. This could lead to undesirable results.		Whenever a record has no value for a certain parameter, for any reason, a missing data value is substituted for that value. This missing data value must be less than -900. Any value greater than -900 (e.g. -888 or -777) will be interpreted as an actual data value by the current revisions of the software. This could lead to undesirable results.

-	Currently, there are four missing data values that are defined explicitly by OCS:	+	Currently, there are six missing data values that are defined explicitly by OCS:

	{\| width="70%" bgcolor="#C0C0C0"		{\| width="70%" bgcolor="#C0C0C0"
Line 158:		Line 158:
	-997		-997
	\| valign="top" \|		\| valign="top" \|
-	Sensor temporarily off-line	+	Sensor temporarily off-line (i.e., missing calibration coefficients).
	\|-		\|-
	\| width="30%" valign="top" \|		\| width="30%" valign="top" \|
	-996		-996
	\| valign="top" \|		\| valign="top" \|
-	Station did not report	+	Station did not report.
		+	\|-
		+	\| width="30%" valign="top" \|
		+	-995
		+	\| valign="top" \|
		+	Parameter not reported on this time interval.
		+	\|-
		+	\| width="30%" valign="top" \|
		+	-994
		+	\| valign="top" \|
		+	Data witheld due to out-of-range value overflowing the maximum column width.
	\|}		\|}

Jpbostic: /* 2.4 The data records */

Jpbostic — Tue, 01 Apr 2008 19:38:57 GMT

2.4 The data records

←Older revision		Revision as of 19:38, 1 April 2008
Line 61:		Line 61:
	The last section of an MDF/MTS file contains a line of data for each station/time stored in the file:		The last section of an MDF/MTS file contains a line of data for each station/time stored in the file:

-	ADAX 1 1020 57 32.8 8.4 8.1 188	+	ADAX 1 1020 57 32.8 8.4 8.1 188
	ALTU 2 1020 45 35.1 5.4 5.3 271		ALTU 2 1020 45 35.1 5.4 5.3 271
	ALVA 3 1020 70 25.0 9.1 9.1 51		ALVA 3 1020 70 25.0 9.1 9.1 51