The ultimate splunk> “Magic 8” for a dramatic performance boost

The magic 6 8 (increased to 8 recently)

Here a well known (for long time splunkers) but hard to find splunk> best practice about how to easily increase efficiency and performance when getting data into Splunk!

ALWAYS SET ALL THESE KEYS … ALWAYS !

These sourcetype changes should always be added into an own TA (Technology Addon) and need to be deployed on your intermediate HF(s) / Indexer(s). For details where things apply in splunk: wiki.splunk.com

 

The splunk> magic 6 8 need to be configured in your props.conf :

[mySourcetype]
TIME_PREFIX = regex of the text that leads up to the timestamp
MAX_TIMESTAMP_LOOKAHEAD = how many characters for the timestamp
TIME_FORMAT = strftime format of the timestamp
# for multiline events: SHOULD_LINEMERGE should always be set to false as LINE_BREAKER will speed up multiline events
SHOULD_LINEMERGE = false
# Wherever the LINE_BREAKER regex matches, Splunk considers the start
# of the first capturing group to be the end of the previous event
# and considers the end of the first capturing group to be the start of the next event.
# Defaults to ([\r\n]+), meaning data is broken into an event for each line
LINE_BREAKER = regular expression for event breaks
TRUNCATE = 999999 (always a high number / not 0)
# Use the following attributes to handle better load balancing from UF.
# Please note the EVENT_BREAKER properties are applicable for Splunk Universal
# Forwarder instances only. Valid with forwarders > 6.5.0
EVENT_BREAKER_ENABLE = true
EVENT_BREAKER = regular expression for event breaks