Meta file:
This is an xml file used to define the structure of the data file for logQL to read. This file contains:
The file stars with <logQLConfig> and ends with </logQLConfig>.
If there is more than one meta defined in the file, you can optionally use
<defaultConfig
name="apache-common"/>
There are three types of configurations:
1. Standard: This configuration is defined using the tag <stdConfig>.
It is used for data files where the field separator changes from one field to another. For example:
<lineFormat variableChar="$">$a$-$b$ $c$</lineFormat>
In the above example, we have three fields: a, b and c. a and b are separated by '-'(hyphen). Where as b and c are separated by ' '(space).
The separator can be a single character or a sequence of characters:
$msg$ from=$fadd$ to=$tadd$
In the example above, the fields are 'msg', 'fadd' and 'tad'. Here we see 'from=' and 'to=' are separators.
2. CSV: This configuration is defined using the tag <csvConfig>.
It is used on files where the fields are separated by comma character ','
CSV files commonly contain a header followed by the data, for this reason, the number of lines to skip can be specified in the configuration tag as follows:
<csvConfig name="nfsData" skipLines = "7">
The fields defined in the Meta (instructions below) should be in the same order as they appear in the data file.
3. Delimiter/separator: This configuration is defined using the tag <sepConfig>
It is used on files where the fields are separated by as standard separated/delimiter. The separator along with the number of lines to skip can be number of lines to skip can be specified as follows:
<sepConfig name="sep" skipLines = "2" seperator="\t">
The separator can be a single character or a sequence of characters.
The fields defined in the Meta (instructions below) should be in the same order as they appear in the data file.
Field definition: the fields in the file can be defined using the <field> tag.
This tag requires the following attributes:
a. name: the name for the field, this should not contain any spaces.
b. type: this can be of type
i. string
ii. long: 64 bit number.
iii. double/number/decimal: 64 bit real number.
iv. date
c. format: required for type date. This specifies the date format in the file. For example 'yyyy-MM-dd'. To know more please see: http://java.sun.com/j2se/1.5.0/docs/api/java/text/SimpleDateFormat.html
For examples, please see 'meta.xml' this is packaged with logQL.