Skip to main content

Check out Interactive Visual Stories to gain hands-on experience with the SSE product features. Click here.

Skyhigh Security

Cloud Connector Config Log Processing - General Properties

Log Processing 

IMPORTANT: You must have the Skyhigh Cloud Connector user role to configure Cloud Connector. For details, see About User Roles and Access Levels

It provides options to configure log filtering and processing. 

For other Cloud Connector Configuration tabs and fields, see About Cloud Connector Configuration

Sub-Configurations 

Previously, when you installed Skyhigh Cloud Connector, you had to send a sample log file to Skyhigh Security Support in order to create your Log Processor configuration, also called the Log Parser. 

In the Log Processing Sub Configuration, the Skyhigh Cloud Connector Log Parser Wizard allows you to avoid that step. Now when you install Cloud Connector, a default Log Processor configuration is included.

For details, see Cloud Connector Config Log Processing - Sub-Configuration.

Log Processing.png

General Properties 

clipboard_ebe9ff7ec870879d6e180f429fac440c1.png

Field Description
Production Mode Select Yes to enable token configuration to allow Cloud Connector to process logs. If you select No, Cloud Connector does not process any log files. 
Default Tag Name for Configuration Enter the default tag name for the entire sub-configuration. 
Default Tokenization for Shadow IT Select the value for the default tokenization setting for the Shadow IT configuration. 
Default Tokenization for Sanctioned IT Select the value for the default tokenization setting for the Sanctioned IT configuration. 
Enable Secure LDAP for AD Select Yes to enable secure LDAP for AD. Select No to disable. (This property also appears on the Custom Attributes tab. You can use it on either tab.)

Advanced Settings

Click Show Advanced Settings to display. 

clipboard_e1cf11bcaa3fe3fdaba3a76f1419c90c6.png

Field Description
TLS Version to be used Select the TLS version(s) to be used by Cloud Connector for all incoming and outgoing connections (the highest version will be given preference). 
Record Content Filter Events Select Yes to enable Skyhigh Cloud Connector to record content filter events from raw logs. Select No to disable.
Bytes Per Line Enter the number of bytes reserved for processing one line required for the memory buffer, both read and write. 
Bytes Reserved For Tokens Enter the number of bytes reserved for tokens. 
Bytes Reserved For File Cache Enter the number of bytes reserved for the file cache. 
Bytes Reserved for Housekeeping Enter the number of bytes reserved for housekeeping.
Maximum Line Length Buffer Enter a number for the buffer for the maximum number of lines read. 
Timestamp Processing Threshold Update Frequency Enter the time frequency in minutes for the timestamp processing threshold.
Timestamp Processing Maximum File Age Enter the timestamp processing threshold maximum file age in minutes.
Record Data Movement Select Yes to enable recording data movement information. Select No to disable. 
Event Filtering Future Date Tolerance Enter the future date in milliseconds (max limit) for the date filed in log lines to process. Cloud Connector does not process logs for a date older than this limit. 
Top-level-domain Data Upload Select Yes to enable uploading Top-level-domain data to Skyhigh CASB. Select No to disable.  
Top-level-domain Tracker Full Hostname Select Yes to enable tracking the full hostname for unmatched services. Select No to disable. 
Include Raw Logline Select Yes to include a raw logline for the in file process Status Report. Select No to disable.
Enable Filtering Select Yes to enable event filtering based on the configuration for file processing. Select No to disable.
Frequency at which timeSeries LevelDB needs to be recycled in hours The Frequency at which timeSeries LevelDB needs to be recycled in hours.
Enter the recycle frequency for DualMode LevelDB The Frequency at which DualMode LevelDB needs to be recycled in hours.
Multi-Level Cache Store Enter the frequency in number of days for the Level DB to be rotated.
Heartbeat Frequency Use this frequency of time in milliseconds to send heartbeats to Skyhigh CASB.
Exclude File Types Enter regex to exclude these file types from processing. 
Processing Log Lines Limit Enterprise Connector spawns threads on each processor for parallel processing. This property allows you to configure the number of lines for a thread. 
Exclude File Names Exclude files with these names. 
Download MIME Types Download files with these MIME types. 
Exclude MIME Types Exclude files with these MIME types. 
Filter by Extensions Select Yes to filter by file extensions. Select No to disable.
Filter by MIME Types Select Yes to filter by MIME types. Select No to disable.
Scan Delay Use this delay in seconds to check if new files are available for processing.
Minimum Log Consumers Enter the number of threads processing the log lines. The value cannot not be less than 3.
Minimum Log Collectors Enter the number of Log Collector threads writing out data to the server. The value cannot not be less than 2.
CPU Overload Factor Enter the CPU throttle factor number (in percentage) used for Log Processor where it is installed. 
Log Consumer Share Enter the total number (in percentage) of CPU resources assigned to processing the log lines. 
Log Collector Share Enter the number of total CPU resources assigned to writing the data to the server, from 0.25% to 25%.
Minimum Log Lines Per Chunk Enter the minimum number of log lines to be read into the chunk.
Filter Malware Enter True to enable the malware filter. Enter False to disable it.
Filter Content Enter True to enable the content filter. Enter False to disable it.
Exclude Files by Extension with Regex Enter a regex to exclude specific MIME types. For example, .json, .exe, or .js. 
Status Report Interval Enter the time period in milliseconds to wait before sending the status reports. 
Status Report Batch Size Enter the number of batches in which the Status Report is sent, in case the report is large. 
Configuration Update Interval Enter the configuration update interval in milliseconds.
Time Zone Enter the time zone where Skyhigh Cloud Connector is installed.
Log Collector Host Enter the host name for the Log Collector endpoint.
Log Collector Port Enter the port number to use for Log Collector.
Log Collector Scheme The communication protocol used to connect Skyhigh Cloud Connector with Log Collector (http/https).
Cloud Config Poll Interval Enter the poll interval in milliseconds to fetch the cloud configuration profile. 
Custom Event Filter Enter the custom event filter. The matching line is not processed by Skyhigh Cloud Connector.
DualMode Store Type Select LevelDB or RocksDB for Store Type.
Configuration refresh interval Configuration refresh interval time.
LDAP Users Store Select LevelDB or RocksDB to store ldapusers.
EnableCase Sensitivity Enable CaseSensitivity for unique keys and usernames. Select Yes to consider the usernames or unique keys as case sensitive. No to consider them as case insensitive.
Subnet Aggregation File Path Enter the source path location for the subnet aggregation file.
Resource Algorithm Enter the resource allocation algorithm. 
Top-level-domain URL for Unmatched Uploads Enter the URL to upload Top-level-domain details for unmatched services. 
Log Collector Endpoint Enter the endpoint for Log Collector to send or receive reports and configurations.
Log Collector Endpoint URL Enter the endpoint URL path for Log Collector to send events details.
Log Collector Login Location Enter the endpoint URL path for Log Collector to log in.
Log Collector Pre-login Location Enter the endpoint URL path for Log Collector to use for pre-login.
Log Collector Host Password Enter the password for Log Collector to use for authentication REST calls. 
Log Collector Host ID Enter the Host ID number for Log Collector. 
Log Collector Tenant ID Enter the Tenant ID number for Log Collector default 2. 
Data Source URL Enter the database file path name. 
Data Source SQL Path Enter the path for the input database initial schema. 
Credentials File Enter the credentials provider file path name. 
Adhoc Config String Enter the Adhoc Configuration String. (Currently not used.)
App URL Prefix Enter the host name and app name to be used as a prefix for the REST endpoints. 
Report Upload File Size Enter the file size allowed for the status report.
Environment Configuration URL Enter the URL of the REST endpoint to get Skyhigh Cloud Connector configuration information including integrations.
Environment Report URL Enter the URL of the REST endpoint to send reports.
Environment Health Report URL Enter the URL of the REST endpoint to send health reports.
Environment Code URL Enter the URL of the REST endpoint to send code.
Environment Login URL Enter the URL of the REST endpoint to authenticate the tenant account details with the Skyhigh CASB API.
Environment Configuration List URL Enter the URL of the REST endpoint to get the configuration list per tenant.
Environment Date Format URL Enter the URL of the REST endpoint to get the CSP's date format.
Known Source IP Address Enter the list of IP addresses to ignore.

 

  • Was this article helpful?