About External Lists
Data can be retrieved from external sources, for example, web servers, and used in rules on the appliance.
This data can be a complete list or a single value. It is generally referred to as external lists or external list data. Different data types can be used in an external list, such as strings, numbers, IP addresses, and others.
An important feature of external lists is that they are processed dynamically on the appliance. All retrieving and conversion of external list data happens at run time when the data is first used in a rule.
When the data has been retrieved, it is stored in an internal cache for a period of time that you can configure, but not on disk, so it will not persist after a restart of the appliance. Also external lists do not appear on the lists tree of the user interface.
External lists properties
Access to data retrieved from external sources is provided through special properties. The name of an external list property is ExtLists.<type>, where <type> is the type of elements in the list that is the value of the property. For example, the value of ExtLists.IntegerList is a list of integers. Possible list element types include String, Number, Wildcard Expression, and others.
Usually the value of an external list property is a list, but there also external list properties for single values. When an external source delivers more than one value as input for the latter type of property, only the last value is retrieved and stored.
External list data can be filtered, depending on the source type, and converted into a different format, depending on the type of the property used in a given rule.
By configuring parameters for an external list property, you can specify placeholders that are substituted with property parameters at run time. Using these placeholders, you can let the content of an external list depend on criteria such as a user name or user group name.
For logging purposes, you can use the ExtLists.LastUsedListName property, which has as its value the name of the settings for the External Lists module that were used last.
External Lists module
To specify which data is to be retrieved from an external source, you need to configure the settings of the External Lists module (also known as the External Lists filter or engine), which retrieves the data.
When external data cannot be retrieved successfully, the External Lists module returns an error code, which you can process using Error Handler rules. A separate range of error IDs is available for this purpose.
The External Lists module consumes memory for caching data that it retrieves from external sources. You should take this into account when setting up rules for external list handling.
Sources of external list data
The sources of the content that external lists are filled with can be the following:
- A web service, which is accessed under the HTTP, HTTPS, or FTP protocol
- A file within your local file system
- An LDAP or LDAPS server
- A database:
- PostgreSQL
- SQLite3
For performing queries on the databases, the SQL query language is used. However, the particular query format can be different for both database types.
As an SQLite3 database operates file-based, we recommend it for testing, rather than for production environments. However, you might still want to use it if you already have data in a database of this type. Otherwise it is easier to use Web Service or File data sources for retrieving external list content.
Recommended use
Working with the external lists feature is recommended in cases like the following.
You need to handle a large number of lists that are mostly stored in external sources, you are running multiple appliances as nodes in a Central Management configuration, and you need to apply frequent changes to the list data.
Synchronizing all list data on all nodes could then no longer be scalable.