Meta file:

This is an xml file used to define the structure of the data file for logQL to read. This file contains:

1. Configuration (standard/csv/delimited).
2. Field names.
3. Field types (and optionally formatting)

The file stars with <logQLConfig> and ends with </logQLConfig>.

If there is more than one meta defined in the file, you can optionally use

<defaultConfig
name="apache-common"/>

There are three types of configurations:

 

1.       Standard: This configuration is defined using the tag <stdConfig>.

It is used for data files where the field separator changes from one field to another. For example:

<lineFormat variableChar="$">$a$-$b$ $c$</lineFormat>

In the above example, we have three fields: a, b and c. a and b are separated by '-'(hyphen). Where as b and c are separated by ' '(space).

The separator can be a single character or a sequence of characters:

$msg$ from=$fadd$ to=$tadd$

In the example above, the fields are 'msg', 'fadd' and 'tad'.  Here we see 'from=' and 'to=' are separators.

 

2.       CSV: This configuration is defined using the tag <csvConfig>.

 It is used on files where the fields are separated by comma character ','

 CSV files commonly contain a header followed by the data, for this reason, the number of lines to skip can be specified in the configuration tag as follows:

 <csvConfig name="nfsData" skipLines = "7">

The fields defined in the Meta (instructions below) should be in the same order as they appear in the data file.

 

3.       Delimiter/separator: This configuration is defined using the tag <sepConfig>

It is used on files where the fields are separated by as standard separated/delimiter. The separator along with the number of lines to skip can be number of lines to skip can be specified as follows:

<sepConfig name="sep" skipLines = "2" seperator="\t">

The separator can be a single character or a sequence of characters.

The fields defined in the Meta (instructions below) should be in the same order as they appear in the data file.

 

Field definition: the fields in the file can be defined using the <field> tag.

This tag requires the following attributes:

a.       name: the name for the field, this should not contain any spaces.

b.      type: this can be of type

                                                   i.      string

                                                 ii.      long: 64 bit number.

                                                iii.      double/number/decimal: 64 bit real number.

                                               iv.      date

c.       format: required for type date. This specifies the date format in the file. For example 'yyyy-MM-dd'. To know more please see: http://java.sun.com/j2se/1.5.0/docs/api/java/text/SimpleDateFormat.html

 

For examples, please see 'meta.xml' this is packaged with logQL.