Configure Syslog for your SIEM

Decide how to send

Syslog lets you send data using UDP or TCP. UDP is much more commonly used (some syslog servers may not even have TCP listeners), but TCP is naturally more flexible. UDP typically uses port 514, but TCP has no standardised port. UDP provides considerably lower overheads in terms of network traffic, but TCP provides flow control, error recovery, and support for queueing.

UDP and TCP configurations can be easily identified by the number of @ symbols in the forwarding line:

• UDP configuration. First line sends access log events; second line sends access and audit log events:

daemon.info @x.x.x.x:514 
daemon.info;auth.=info @x.x.x.x:514

• TCP configuration. as above, but using TCP and an example port

daemon.info @@x.x.x.x:6514 
daemon.info;auth.=info @@x.x.x.x:6514 

Decide what severity to send

The recommended severity is always 6 (Info)! However, the below table shows the available list for reference. The descriptions are provided as examples; the RFC for syslog only recommends that appropriate severities are chosen.

 For more info see: http://tools.ietf.org/html/rfc5424#page-11

Decide what to send

What kind of data do you want to send to the syslog server? All access logs and audit logs? Only access logs, or only blocked requests?

Decide what format to send

Which format does your syslog server require data to be presented in? You may wish to check with your server admin to see which format should be used by SWG:

Use the GUI file editor

When configuring syslog, it is tempting to jump on the command line and edit /etc/rsyslog.conf directly. This will not end well! The SWG Coordinator will overwrite this file and you will lose changes. Use the File Editor on the Configuration page.

Don't write to disk!

It is extremely important that syslog does not write access or audit log data to disk. The default syslog configuration will happily do this and fill the /var partition on the appliance. To avoid this, look for this line in ryslog.conf:

*.info;mail.none;authpriv.none;cron.none /var/log/messages

If you only plan to send access logs, replace it with this line:

*.info;daemon.!=info;mail.none;authpriv.none;cron.none   -/var/log/messages

•  The addition of “daemon.!=info” excludes the writing of anything from daemon facility with a priority of info to the messages file, thus maintaining disk space. It needs to be placed after “*.info” to allow other info-level messages to be written as normal.
• Adding a dash to the beginning of the file path stops rsyslog from syncing the file after every message, protecting performance. If you plan to send both access and audit logs, use the below line instead:

If you plan to send both access and audit logs, use the below line instead:

*.info;daemon.!=info;auth.!=info;mail.none;authpriv.none;cron.none   -/var/log/messages

• As with the first example, an exclusion is added for info-level messages from the auth facility.

Send to syslog

To send using UDP, add a rule to capture messages matching daemon.info (access logs) and auth.info (audit logs) to the destination server:

daemon.=info;auth.=info @x.x.x.x:514 

 This rule should be placed at the bottom of the file.

For TCP logging, things are a little more complex but a lot more configurable. The section between ### begin forwarding rule ### and ### end of the forwarding rule ### contains a complete explanation and example of a queue and TCP forwarding rule. When the comment symbols are removed, they look like this:

Remote Logging (we use TCP for reliable delivery) 

An on-disk queue is created for this action. If the remote host is 
down, messages are spooled to disk and sent when it is up again. 
$WorkDirectory /var/spppl/rsyslog # where to place spool files 
$ActionQueueFileName fwdRule1 # unique name prefix for spool files 
$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) 
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown 
$ActionQueueType LinkedList # run asynchronously 
$ActionResumeRetryCount -1 # infinite retries if host is down 
#remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional 
*.* @@remote-host:514

As you will see, the settings are well explained, but the defaults are fine here. The only line you should edit is the final one for the remote host. As with UDP, specify the messages to capture and then add the desired server:

daemon.=info;auth.=info @x.x.x.x:6514 

Default format

For the default format, all that is required is to create a rule which applies the conditions in which you want data to be sent to the syslog server. In our case, we're sending ALL access log data to the syslog server. The only change required is to create an additional rule to send the logline to syslog

Name: Send to syslog 
Criteria: Always 
Action: Continue 
Event: Syslog (6, User-Defined.logLine) 

Skyhigh Security SIEM (Nitro)

The Skyhigh Security SIEM format includes additional log fields that the Skyhigh Security SIEM will parse. See the Online Ruleset Library (in the Content Security Portal) for the most up-to-date Skyhigh Security SIEM ruleset. Below is an example log entry:

{{corp}}WG|time_stamp=[01/Jan/2015:02:12:31 +0800]|auth_user=jsmith|src_ip=10.10.69 

Below is screenshots of the rule sets installed in the log handler:

CEF Format

The CEF format is much different from the default Skyhigh Security Web Gateway format. See the Online Ruleset Library (in the Content Security Portal) for the most up-to-date CEF format. The CEF format will include the column metadata (i.e. what the column represents) in the log line. The CEF format is a generic format that a large number of SIEM vendors support including Arcsight and Splunk.

Below are some screenshots of what the rules will look like:

Attachments

Below are links to the rulesets referenced in the screenshots above. They can be imported using the Ruleset library.

• Skyhigh Security SIEM (Nitro) logging ruleset
• CEF syslog format ruleset

Format

Audit logs are generated in CEF format; For example:

Nov 23 19:22:33 localhost CEF: 0|{{corp}}|WebGateway|1|USER_LOGIN|USER_LOGIN|3|Time

Facility and Severity

Audit events are sent by the “auth” facility, and using information severity (6). For this reason, the configuration examples above all match for auth.=info.

Filling /var partition

It is common for the /var partition to be filled if the steps in ‘Don’t write to disk’ are not followed – this will usually not be seen immediately. You can check if the messages file is being written to via the GUI and the CLI:

• GUI: Open the messages file located in Troubleshooting > Log files > system > messages
• CLI: run the command

tail /var/log/messages

In both cases, the presence of either access or audit log lines indicates that they have not been correctly excluded.

Messages not received on syslog server

If messages are not received on the destination server, the first check should be for firewalls that may block the specified ports. To verify that the messages are being sent, run a simple tcpdump on the CLI against the port(s) you configured:

tcpdump port 514 

If you see data, rsyslog is doing its job and you should look into the network side – bear in mind that UDP will not show any responses! If you see nothing, it’s possible that the rules are not being triggered properly. You can check a Rule Trace to see if something fails to match in the Logging cycle

Message size

Web Gateway will, by default, truncate messages to 2,000 characters before sending them to the server. To increase the maximum message size, you can add this line to the first line of rsyslog.conf, with a size value in kilobytes:

$MaxMessageSize 4k