Module Name: omelasticsearch
Author:Rainer Gerhards <rgerhards@adiscon.com>
Available since:6.4.0+
Description:
This module provides native support for logging to Elasticsearch.
Action Parameters:
server Host name or IP address of the Elasticsearch server. Defaults to “localhost”
HTTP port to connect to Elasticsearch. Defaults to 9200
searchIndex Elasticsearch index to send your logs to. Defaults to “system”
dynSearchIndex<on/off> Whether the string provided for searchIndex should be taken as a template. Defaults to “off”, which means the index name will be taken literally. Otherwise, it will look for a template with that name, and the resulting string will be the index name. For example, let’s assume you define a template named “date-days” containing “%timereported:1:10:date-rfc3339%”. Then, with dynSearchIndex=”on”, if you say searchIndex=”date-days”, each log will be sent to and index named after the first 10 characters of the timestamp, like “2013-03-22”.
searchType Elasticsearch type to send your index to. Defaults to “events”
dynSearchType <on/off> Like dynSearchIndex, it allows you to specify a template for searchType, instead of a static string.
asyncrepl<on/off> By default, an indexing operation returns after all replica shards have indexed the document. With asyncrepl=”on” it will return after it was indexed on the primary shard only - thus trading some consistency for speed.
usehttps<on/off> Send events over HTTPS instead of HTTP. Good for when you have Elasticsearch behind Apache or something else that can add HTTPS.
timeout How long Elasticsearch will wait for a primary shard to be available for indexing your log before sending back an error. Defaults to “1m”.
template This is the JSON document that will be indexed in Elasticsearch. The resulting string needs to be a valid JSON, otherwise Elasticsearch will return an error. Defaults to:
$template JSONDefault, "{\"message\":\"%msg:::json%\",\"fromhost\":\"%HOSTNAME:::json%\",\"facility\":\"%syslogfacility-text%\",\"priority\":\"%syslogpriority-text%\",\"timereported\":\"%timereported:::date-rfc3339%\",\"timegenerated\":\"%timegenerated:::date-rfc3339%\"}"
Which will produce this sort of documents (pretty-printed here for readability):
{
"message": " this is a test message",
"fromhost": "test-host",
"facility": "user",
"priority": "info",
"timereported": "2013-03-12T18:05:01.344864+02:00",
"timegenerated": "2013-03-12T18:05:01.344864+02:00"
}
Samples:
The following sample does the following:
module(load="omelasticsearch")
*.* action(type="omelasticsearch")
The following sample does the following:
module(load="omelasticsearch")
template(name="testTemplate"
type="list"
option.json="on") {
constant(value="{")
constant(value="\"timestamp\":\"") property(name="timereported" dateFormat="rfc3339")
constant(value="\",\"message\":\"") property(name="msg")
constant(value="\",\"host\":\"") property(name="hostname")
constant(value="\",\"severity\":\"") property(name="syslogseverity-text")
constant(value="\",\"facility\":\"") property(name="syslogfacility-text")
constant(value="\",\"syslogtag\":\"") property(name="syslogtag")
constant(value="\"}")
}
action(type="omelasticsearch"
server="myserver.local"
serverport="9200"
template="testTemplate"
searchIndex="test-index"
searchType="test-type"
bulkmode="on"
queue.type="linkedlist"
queue.size="5000"
queue.dequeuebatchsize="300"
action.resumeretrycount="-1")
This documentation is part of the rsyslog project. Copyright © 2008-2014 by Rainer Gerhards and Adiscon. Released under the ASL 2.0.