Event Stream Analysis Configuration Guide

for Version 10.6.2

Copyright © 2016 EMC Corporation. All Rights Reserved.

Trademarks
RSA, the RSA Logo and EMC are either registered trademarks or trademarks of EMC
Corporation in the United States and/or other countries. All other trademarks used herein are the
property of their respective owners. For a list of EMC trademarks, go to
www.emc.com/legal/emc-corporation-trademarks.htm.

License Agreement
This software and the associated documentation are proprietary and confidential to EMC, are
furnished under license, and may be used and copied only in accordance with the terms of such
license and with the inclusion of the copyright notice below. This software and the
documentation, and any copies thereof, may not be provided or otherwise made available to any
other person.
No title to or ownership of the software or documentation or any intellectual property rights
thereto is hereby transferred. Any unauthorized use or reproduction of this software and the
documentation may be subject to civil and/or criminal liability. This software is subject to
change without notice and should not be construed as a commitment by EMC.

Third-Party Licenses
This product may include software developed by parties other than RSA. The text of the license
agreements applicable to third-party software in this product may be viewed in the
thirdpartylicenses.pdf file.

Note on Encryption Technologies

This product may contain encryption technology. Many countries prohibit or restrict the use,
import, or export of encryption technologies, and current use, import, and export regulations
should be followed when using, importing or exporting this product.

Distribution
Use, copying, and distribution of any EMC software described in this publication requires an
applicable software license. EMC believes the information in this publication is accurate as of
its publication date. The information is subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED "AS IS." EMC
CORPORATION MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND
WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND
SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR
FITNESS FOR A PARTICULAR PURPOSE.
Contents

Event Stream Analysis (ESA) Overview 6

Configure Event Stream Analysis (ESA) 7

Prerequisites 7
Procedure 7
Result 8
Step 1. Add Event Stream Analysis Service 8
Prerequisites 8
Procedure 8
Step 2. Add a Data Source to an ESA Service 10
Prerequisites 10
Procedures 10
Step 3. Configure Advanced Settings for an ESA Service 12
Procedures 12
Step 4. Configure an ESA to Connect to the Context Hub on Another ESA 14
Prerequisites 14
Procedure 14
Result 15

Additional Procedures 16
Change Default Storage Passwords 16
Previous ESA Storage Password 17
Dependencies 17
Database Privileges 17
Change MongoDB Password for admin Account 18
Change ESA Storage Password 19

Change Password for ESA Database Account 19

​Change Password for ESA Service 20

Change Incident Management Storage Password 21

Change Password for Incident Management Database Account 21

Change Password for Incident Management Service 21

 Event Stream Analysis Configuration Guide

Change Data Science Storage Password 23

Change Data Science Password for Database Account 23

Change Data Science Password for Security Analytics 24

Change Memory Threshold for Trial Rules 25

Prerequisites 25
Procedure 26
Configure ESA Storage 27
Configuration Parameters 27
Prerequisites 28
Procedure 28
Example 30
Configure ESA to Use a Memory Pool 30
Procedure 32
Result 35
Configure ESA to Use Capture Time Ordering 35
Capture Time Order Workflow 35
Prerequisites 36
Procedures 36
Troubleshooting Tips 38

Disable Capture Time Ordering 38

Disable Position Tracking 39

Start, Stop, or Restart ESA Service 39

Start ESA Service 39
Stop ESA Service 39
Restart ESA Service 40
Audit Logs and Verify ESA Component Versions and Status 40
Audit Log Rules 40
Verify ESA Server Version 41
Verify MongoDB Version 42
Verify MongoDB Status 42

References 43
Services Config View Advanced Tab 43
Features 44

4
 Event Stream Analysis Configuration Guide

Services Config View Data Sources Tab 45

Features 46

5
 Event Stream Analysis Configuration Guide

Event Stream Analysis (ESA) Overview

This topic provides an overview of the Event Stream Analysis module.
The Security Analytics Event Stream Analysis (ESA) service provides advanced stream
analytics such as correlation and complex event processing at high throughputs and low
latency. It is capable of processing large volumes of disparate event data from Concentrators.
ESA's advanced Event Processing Language allows you to express filtering, aggregation, joins,
pattern recognition and correlation across multiple disparate event streams. Event Stream
Analysis helps to perform powerful incident detection and alerting.
The following diagram shows the workflow:

Event Stream Analysis (ESA) Overview 6

 Event Stream Analysis Configuration Guide

Configure Event Stream Analysis (ESA)

This topic provides high-level tasks to configure the Security Analytics Event Stream Analysis.

Prerequisites
Make sure that you:
l Install the Event Stream Analysis service in your network environment.

l Install and configure one or more Concentrators in your network environment.

Procedure

Note: You can configure ESA using an SSL port (50030) only. There is no option to configure
a Non-SSL port.

To configure Event Stream Analysis:

Tasks Reference

1. You can discover, update or add the host on which Refer to "Step 1 : Add or Update a
the ESA service is installed. (Optional) If ESA is not Host" in the "Host and Services
set up, you need to add Event Stream Analysis as a
Getting Started Guide". Refer
core service and add the Event Stream
to Step 1. Add Event Stream Analysis
Analysis service to the host.
Service.

2. Apply license to the Event Stream Analysis service.  Refer to "View Current Enti-
tlements" in the "Licensing Guide."

3. Add the Concentrator as data source to the Event Refer to Step 2. Add a Data Source
Stream Analysis service.  to an ESA Service

4. Configure notifications for Event Stream Refer to "Notification Methods" in

Analysis service. the "Alerting Using ESA Guide."

5. Download Event Stream Analysis content using Refer to "Live Search View" in the
Live. "Live Resource Managment
Guide".

7 Configure Event Stream Analysis (ESA)

 Event Stream Analysis Configuration Guide

Tasks Reference

6. (Optional) Advanced configuration for Event Stream Refer to Step 3. Configure Advanced

Analysis service. Settings for an ESA Service.

7. (Optional) Enable Context Hub. Refer to "Step 1. Add the Context

Hub Service" in the "Context Hub
Configuration Guide".

8. (Optional) Configure ESA to connect to the Context Refer to  Step 4. Configure an ESA

Hub on another ESA.  to Connect to the Context Hub on
Another ESA.

Result
The Event Stream Analysis service is configured and you can now add ESA Rules for event
processing and alerting. For information on adding ESA Rules, see "Add Rules to the Rule
Library" in the "Alerting Using ESA Guide."

Step 1. Add Event Stream Analysis Service

This topic provides information on how to add the Event Stream Analysis (ESA) service on a
host.

Prerequisites
Ensure that you have installed an ESA service and added the host in Security Analytics. For
more information, see "Step 1: Add or Update a Host" in the "Host and Services Getting Started
Guide."

Procedure
To add the Event Stream Analysis service:
1. In the Security Analytics menu, select Administration > Services.
The services view is displayed.

2. In the Services panel, select  > Event Stream Analysis.

Configure Event Stream Analysis (ESA) 8

 Event Stream Analysis Configuration Guide

The Add Service dialog is displayed.

3. Provide the following details.

Field Description

Host Select the host on which you want to install the ESA service.

Name Type a name for the service.

Port Default port is 50030.

Note: ESA can be configured using the SSL port 50030 only. You cannot
configure a Non-SSL port.

Entitle Select if you want to apply the entitlements currently configured to this
Service service.

4. Click Test Connection to determine if Security Analytics connects to the service.

Note:  While adding the service, Security Analytics sends ICMP packets to the service to
verify if the hostname/ip address entered is valid for successful test connection​.

5. When the result is successful, click Save.​

The added service is now displayed in the Services panel.

9 Configure Event Stream Analysis (ESA)

 Event Stream Analysis Configuration Guide

Note: If the test is unsuccessful, edit the service information and retry.

Step 2. Add a Data Source to an ESA Service

This topic describes how to add a new or existing data source to the Event Stream Analysis
service.
An ESA service ingests data from a Concentrator to detect incidents and alert the user. For ESA
to analyze data, you need to configure the sources from which the ESA will read data. Use the
procedures in this topic to add data sources for your ESA.

Prerequisites
You must have one or more Concentrators configured in Security Analytics.
You must perform the following steps to add a data source:
l Add an Available Data Source

l Specify username and password for the Data Source 

Procedures

Add Existing Services as Data Source

1. In the Security Analytics menu, select Administration > Services.

The Services view is displayed.

2. In Services view, select an ESA service. 

3. In the Actions column, select View > Config.

4. In the Data Sources tab, click .

The available services are displayed as shown in the following figure.

Configure Event Stream Analysis (ESA) 10

 Event Stream Analysis Configuration Guide

5. Select one or more services and click OK.

The service is added to the list of services in the DataSources tab.

6. (Optional) Click Enable to enable the data source.

7. Click Apply to save the configuration.

Specify Username and Password for the Data Source

Note: You can add a Log Decoder as a data source for ESA but RSA recommends you add a
Concentrator to take advantage of undivided aggregation as the Decoder may have other
processes aggregating from it. 

To specify the username and password for the data source:

1. In the Security Analytics menu, select Administration > Services.
The Services view is displayed.

2. In the Services view, select a Concentrator service. 

11 Configure Event Stream Analysis (ESA)

 Event Stream Analysis Configuration Guide

3. Click  .

4. Specify the username and password.

5. Click Save.

Step 3. Configure Advanced Settings for an ESA Service

This topic provides instructions to configure advanced settings for an Event Stream Analysis
service.
In the Advanced view, you can configure advanced settings to improve performance, to preserve
events for rules with multiple events, to buffer events in memory, and the number of events to be
stored on the ESA.

Procedures

Configure Advanced Settings

To access the Advanced view and configure advanced settings for an ESA service:
1. In the Security Analytics menu, select Administration > Services.
The Services view is displayed.

2. In Services view, select an ESA service. 

3. In the Actions column, select View > Config.

4. Select the Advanced tab.
The Advanced view is displayed.

Configure Event Stream Analysis (ESA) 12

 Event Stream Analysis Configuration Guide

Configure Alert Engine Settings

In the Alert Engine section, you specify values to preserve events for rules that choose multiple
events.

Note: After you upgrade to 10.5, the Debug Rules option if enabled previously will be
disabled. You will need to enable this option after upgrade. 

The following figure shows the Alert Engine section.

To configure Alert Engine settings:

1. In the Alert Engine section, specify a value for Max Constituent Events. The default value
is 100.

13 Configure Event Stream Analysis (ESA)

 Event Stream Analysis Configuration Guide

2. If you want alerts to be sent to Message Bus and Incident Management, select the Forward
Alerts On Message Bus option.

3. ​Select Debug Rules? to enable debugging rules.

4. Click Apply to save the changes and put them into effect immediately.

Note: For more information on the parameters in the Alert Engine section, see Alert Engine
Settings in ESA Advanced View.

Configure Event Stream Engine Settings

In the Event Stream Engine section, you specify details to improve performance.
The following figure shows the Event Stream Engine section.

To configure Event Stream Engine settings:

1. In the Event Stream Engine section, specify Max Pattern Subexpressions.

2. Click Apply to save the changes and put them into effect immediately.

Note: For more information on the parameters in the Event Stream Engine section, see Event
Stream Engine Settings in ESA Advanced View.

Step 4. Configure an ESA to Connect to the Context Hub on

Another ESA
This topic tells administrators how to configure an ESA to connect to the Context Hub on
another ESA. Only one Context Hub can be installed per Security Analytics installation. If you
have more than one ESA and you run the Context Hub, you need to enable the ESA without the
Context Hub to communicate with the Context Hub on another ESA. 

Prerequisites
You must be running multiple ESAs and a Context Hub.

Procedure
Configure ESA to connect to the Context Hub on another ESA. 

Configure Event Stream Analysis (ESA) 14

 Event Stream Analysis Configuration Guide

1. Note the IP address of the ESA that is running the context hub service.

2. From Administration > Services, select the ESA service that is not running the Context Hub
and then   > View > Explore. 

3.  In the left hand panel, navigate to Service > ContextHub, then select
contextHubTransport.

4. Edit the Host field to point to the Domain name or IP address of the ESA that is running the
Context Hub service.

Result
The ESA connects to the Context Hub on another ESA service.

15 Configure Event Stream Analysis (ESA)

 Event Stream Analysis Configuration Guide

Additional Procedures
This topic is a collection of individual procedures, which an Administrator may perform at any
time and they are not required to complete the initial setup of ESA. These procedures are
presented in alphabetical order.
Use this section when you are looking for instructions to perform a specific task after the initial
setup of ESA.
l Change Default Storage Passwords

l Change Memory Threshold for Trial Rules

l Configure ESA Storage

l Configure ESA to Use a Memory Pool

l Configure ESA to Use Capture Time Ordering

l Start, Stop, or Restart ESA Service

l Audit Logs and Verify ESA Component Versions and Status

Change Default Storage Passwords

This topic tells administrators how to change default storage passwords for database accounts
that store alerts in ESA, Incident Management and Data Science.
Security Analytics 10.5 uses MongoDB as the database to store alerts in the following modules:
l ESA

l Incident Management

l Data Science

The database in each module has an account to control access and each Security Analytics
service account has a default password.
To strengthen security, RSA recommends that you change default passwords. Some
organizations do not allow default passwords. In those cases, the procedures in this topic would
be required.
This topic explains how to change the default storage password for the database account in each
module.

Additional Procedures 16
 Event Stream Analysis Configuration Guide

Previous ESA Storage Password

ESA was introduced in Security Analytics 10.3 when the database was in PostgreSQL. If you
used ESA in version 10.3 and created a custom password for the PostgreSQL database, it has no
impact on MongoDB. When you install or or upgrade to Security Analytics 10.5, MongoDB is
installed with a default password. 
Incident Management and Data Science were introduced in Security Analytics 10.4 so they have
only used MongoDB.

Dependencies
MongoDB has a master admin account that has privileges over the database accounts for
the ESA, IM and Data Science services.

Note: You must change the admin account password first. You can change passwords for the
services in any sequence.

ESA is a requirement for Incident Management and Data Science. The configuration for each
module points to the host that runs the ESA service. Databases for ESA, Incident Management
and Data Science are located on the host that runs the ESA service. 

Database Privileges
The following figure shows the privileges assigned to each account during the installation or
upgrade process. 

Account Privileges Database

admin readWriteAnyDatabase All

​userAdminAnyDatabase
dbAdminAnyDatabase

Event Stream Analysis readWrite ESA

dbAdmin
clusterAdmin

Incident Management readWrite IM

dbAdmin
clusterAdmin

17 Additional Procedures
 Event Stream Analysis Configuration Guide

Account Privileges Database

Data Science readWrite Data Science

dbAdmin
clusterAdmin

For details on changing each password, see:

l Change MongoDB Password for admin Account

l Change ESA Storage Password

l Change Incident Management Storage Password

l Change Data Science Storage Password

Change MongoDB Password for admin Account

This topic tells administrators how to change the default storage password for the MongoDB
admin account.
In Security Analytics, this procedure is optional. However, it is always a best practice for
administrators to change any default password for added security. Some organizations do not
allow default passwords.

Note: You must change the MondoDB admin account password first. You must enter it before
you can change the passwords for ESA, Incident Management and Data Science.

Prerequisites

You must have Administrator role privileges.

Procedure

1. Log on to the ESA host that runs the ESA service:

a. SSH to the ESA host.

b. Log on as root.

Additional Procedures 18
 Event Stream Analysis Configuration Guide

2. Log on to MongoDB as admin. The default password is netwitness.

mongo admin -u admin -p <current_password>

3. To change the admin account password, type

db.changeUserPassword(‘admin’,’<new_password>’)

Now you can change the password for the ESA, Incident Management and Data Science
services.

Change ESA Storage Password

This topic tells administrators how to change the default storage password for the ESA database.
In Security Analytics, this procedure is optional. However, it is always a best practice for
administrators to change any default password for added security. Some organizations do not
allow default passwords and make this procedure mandatory.

Prerequisites

You must have Administrator role privileges.

Procedures

Change Password for ESA Database Account

1. ​Log on to the host that runs the ESA service:

a. SSH to the ESA host.

b. Log on as root.

2. Log on to the MongoDB as the admin user:

mongo esa -u admin -p <current_admin_password> --
authenticationDatabase admin

3. Type the following command to change the ESA account password. The default pasword is
esa.
db.changeUserPassword(‘esa’,’<new_password>’)

19 Additional Procedures
 Event Stream Analysis Configuration Guide

​C hange Password for ESA Service

1. Log on to Security Analytics as admin.

2. In the Security Analytics menu, select Administration > Services.

3. Select the ESA service, then > View > Explore.

4. In the Explore view on the left, select Alert > Storage > configuration.

5. In the right panel, type the database account password in the ConnectionPassword field.

Note: The password for the database and for the Security Analytics service
configuration must be the same.

6. To validate that the database and Security Analytics passwords match, in the Security

Analytics menu select Alerts > Summary. 
If content appears in the Summary tab, the passwords match and were changed
successfully. 

Additional Procedures 20
 Event Stream Analysis Configuration Guide

If you do not see content in the Summary tab, revise the service password to match the
MongoDB password.

Change Incident Management Storage Password

This topic tells administrators how to change the default storage password for the Incident
Management database.
In Security Analytics, this procedure is optional. However, it is always a best practice to change
any default password for added security. In organizations that do not allow default passwords,
this procedure is mandatory.

Prerequisites

You must have Administrator role privileges.

The default password for the MongoDB admin account must be changed.

Procedures

Change Password for Incident Management Database Account

1. Log on to the host that runs the ESA service:

a. SSH to the ESA host.

b. Log on as root​.

2. Log on to the MongoDB as admin:

mongo im -u admin -p {current_admin_password} --
authenticationDatabase admin

3. Type the following command to change the Incident Management account password. The
default password is im.
db.changeUserPassword(‘im’,’{new_password}’)

Change Password for Incident Management Service

1. Log on to Security Analytics as admin.

2. In the Security Analytics menu, select Administration > Services.

21 Additional Procedures
 Event Stream Analysis Configuration Guide

3. Select the Incident Management service, then > View > Explore.

4. In the Explore view on the left, select Configuration > database.

5. In the right panel, type the database account password in the Password field.

Note: The password for the database and for the Security Analytics service
configuration must be the same.

6. Restart the Incident Management service to accept the password change and force the
session to start using the new password.

a. Select Administration > Services.

b. Select the Incident Management service, and click   > Restart.

7. To validate the new passwords match, select Incidents > Alerts.

If you see content in the Alerts tab, you changed the passwords successfully.

Additional Procedures 22
 Event Stream Analysis Configuration Guide

If you do not see content in the Alerts tab, revise the service password to match the
MongoDB password.

Change Data Science Storage Password

This topic tells administrators how to change the default storage password for the Data Science
database.
In Security Analytics, this procedure is optional. However, it is always a best practice to change
any default password for added security. In organizations that do not allow default passwords,
this procedure is mandatory.

Prerequisites

You must have Administrator role privileges.

The default password for the MongoDB admin account must be changed.

Procedures

Change Data Science Password for Database Account

1. Log on to the host that runs the ESA service.

a. SSH to the ESA host.

b. Log on as root.

2. Log on to MongoDB as admin.

mongo ds -u admin -p {current_admin_password} --
authenticationDatabase admin

3. To change the Data Science account password, type

db.changeUserPassword(‘ds’,’{new_password}’)

23 Additional Procedures
 Event Stream Analysis Configuration Guide

Change Data Science Password for Security Analytics

1. Log on to Security Analytics as admin.

2. In the Security Analytics menu, select Administration > Services.

3. Select the Reporting Engine service, then  > View > Config.

​The Config view is displayed with the General tab open.

4. Select Warehouse Analytics Output Configuration.

5. Type the database account password in the Password field.​

Note: ​The password for the database and for the Security Analytics service
configuration must be the same.

6. To validate the new passwords match, execute a Report in Reporting Engine that
uses Warehouse Analytics.

Additional Procedures 24
 Event Stream Analysis Configuration Guide

Change Memory Threshold for Trial Rules

This topic tells administrators how to set a memory usage threshold for trial rules. When the
threshold is exceeded, all deployed trial rules are disabled.
This procedure is optional. Administrators can increase or decrease the memory threshold for
trial rules. Threshold refers to the ESA memory usage, which includes ESA base memory, trial
rules and non-trial rules. When the threshold is exceeded, all deployed trial rules on an
ESA service are disabled. 
You use trial rules to see if a rule runs efficiently and does not use excessive memory, which
can impact performance or force the service to shut down.
By default, the memory threshold is 85, which is the percentage of Java Virtual Memory (JVM).
l The memory threshold is per ESA, not per rule.

l When the memory threshold is exceeded, all trial rules running on the ESA are automatically
disabled. 

l The ESA configuration has two parameters for trial rules:

l MemoryThresholdforTrialRules

l MemoryCheckPeriod, which has a default value of 300 seconds

For more information, see "Work with Trial Rules" in the "Alerting Using ESA Guide".

Prerequisites
A role with administrative privileges must be assigned to you.

25 Additional Procedures
 Event Stream Analysis Configuration Guide

Procedure
1. Log on to Security Analytics as admin.

2. In the Security Analytics menu, select Administration > Services.

3. Select the ESA service, then   > View > Explore.

4. On the left, select CEP > Module > configuration.

5. In the right panel, in MemoryThresholdForTrialRules type a percentage of JVM that trial

rules on the ESA can not exceed.
The new memory threshold takes effect immediately.

Additional Procedures 26
 Event Stream Analysis Configuration Guide

Configure ESA Storage

This topic explains how to configure the ESA database to maintain a healthy level of alerts. 
This procedure is optional. Administrators can specify a retention period for alerts. Deleting old
alerts is a best practice to maintain the alerts database. Otherwise, the database could continue
to grow and eventually have a negative impact on performance.
By default, the feature to automatically delete alerts is not enabled because each company has
its own policies. This topic teaches you how to perform the following tasks:
l Enable automatic deletion of alerts

l Specify criteria to delete alerts

l By database size

l By alert age

l By a both database size and alert age

Configuration Parameters
The configuration parameters are as follows:​

Parameter Description

Enabled Turns on alert retention feature.

NextMaintenanceScheduledAt (Read only) When the next maintenance is scheduled to

run. 

HaveAlertForDays (Read-only) Current number days that alerts have been

stored in the database. For example, if this number is
checked on June 4th, and there were alerts generated
every day from June 1st, then value would be 4. 

DatabaseDiskUsage (Read-only) Current database size.

Schedule  Schedule for running the alert maintenance. The schedul-

ing uses the UNIX Cron tab and must be specified in the
correct Cron tab format. The default value is displayed
in the procedure below. For more information on Cron
scheduling, see http://www.cronmaker.com.

27 Additional Procedures
 Event Stream Analysis Configuration Guide

Parameter Description

DatabaseDiskUsageLimtInMB Database size threshold; when exceeded, alerts will be

deleted.

Valid Read-only parameter indicating whether the current con-

figuration is valid. 

DaysToDeleteWhenLimitExceeded Number of days to remove when Data-

baseDiskUsageLmitInMB is exceeded.

KeepAlertsForDays Number of days to keep the alerts in the database before

they are removed.

Prerequisites
You must have Administrator permissions.

Procedure
1. Log on to Security Analytics as admin.

2. In the Security Analytics menu, select Administration > Services.

3. Select the ESA service, then  View > Explore.

Additional Procedures 28
 Event Stream Analysis Configuration Guide

4. On the left, select Alert > Storage > maintenance.

5. In the Enabled field, select true to turn on the alert retention feature.

6. Configure how you want to remove old alerts:

l By database size – Type the maximum database size in DatabaseDiskUsageLimitInMB.

Then type how many days of the oldest alerts to delete
in DaysToDeleteWhenLimitExceeded. For example, when disk usage reaches 5120 MB
delete the oldest alerts for 7 days. 

l By alert age – all alerts older than KeepAlertsForDays are deleted.

Note: For Security Analytics 10.4.1 and below, you must use

KeepAlertsForDays. You cannot use DatabaseDiskUsageLimitInMB.

l By database size and alert age. If you configure both of these parameters, whichever rule
deletes the greater number of days is used. 

7. Schedule
Use the schedule parameter to tell the ESA how frequently to run the alert maintenance job
(i.e. how frequently to check the database and apply the deletion rules). Use the syntax for a
Cron schedule job.  For more information on Cron scheduling,

29 Additional Procedures
 Event Stream Analysis Configuration Guide

see http://www.cronmaker.com.

8. Refresh the browser.

l Date and time of next maintenance run is displayed

in the NextMaintenanceScheduledAt field. 

l In the Valid field, true is displayed to indicate the configuration is valid.

If false is displayed, correct the disk size or alert age settings. 

9. (Optional) The maintenance status can also be monitored in the /opt/rsa/esa/logs/esa.log file

on the ESA host, which will display messages similar to the example below.

Example
The maintenance status can also be monitored in the /opt/rsa/esa/logs/esa.log file on the ESA
service, which will display messages similar to the example below.
2015-03-12 09:46:48,197 [Carlos@65dd6c04-56] INFO
com.rsa.netwitness.carlos.config.ConfigurationMXBean -
MongoStorageMaintenance changed by admin
2015-03-12 09:46:51,121 [scheduler_Worker-1] INFO
com.rsa.netwitness.core.alert.dispatch.SQLStorageMaintenance -
Starting the scheduled database maintenance
job with policy {keepAlertForDays=30, maxDiskUsageInMb=5120}
2015-03-12 09:46:51,122 [Carlos@3801f0b3-58] INFO
com.rsa.netwitness.core.alert.dispatch.SQLStorageMaintenance -
Scheduled a database maintenance job with
policy {keepAlertForDays=30, maxDiskUsageInMb=5120} to run at 2/28/15
2:00 AM
2015-03-12 09:46:51,129 [Carlos@3801f0b3-58] INFO
com.rsa.netwitness.carlos.config.ConfigurationMXBean -
MongoStorageMaintenance changed by admin
2015-03-12 09:46:51,133 [scheduler_Worker-1] INFO
com.rsa.netwitness.core.alert.dispatch.SQLStorageMaintenance -
Finished the database maintenance job,
deleted 0 partitions, next run scheduled at 3/14/15 2:00 AM

Configure ESA to Use a Memory Pool

This topic tells administrators how to configure the ESA to use a memory pool.

Additional Procedures 30
 Event Stream Analysis Configuration Guide

 A memory pool is a customized implementation of virtual memory for events held by rules in
ESA. This helps in scaling the capability of rules by an order of magnitude. When you want to
create rules that cover a large time span or which are very complex, you may want to use a
memory pool to handle memory more efficiently.  When you use a memory pool, instead of
holding all of the events in memory, they can be written to disk. This is helpful because when a
rule exists that is complex or extends over a long time frame, a large number of events must be
held in memory.
You can configure memory pool to run in non-batch mode or batch mode:
l Non-batch mode. In non-batch mode, events are written to disk as they enter the memory
pool. To configure non-batch mode, set the MapPoolBatchWriteSize attribute to 1. Non-
batch mode provides a more stable solution because each event is landed and fetched
separately without creating memory spikes.

l Batch mode. In batch mode, events are grouped into batches and then written to disk. To
configure batch mode, set the batch size attribute MapPoolBatchWriteSize to a value
greater than 1. Batch mode gives better performance since the disk activity for landing events
to disk are optimized.

Note: Any changes to these settings will require you to restart the ESA. When ESA restarts,
if any events are currently being held by the memory pool, they will be discarded upon restart. 

Caution: While this feature can be very helpful in managing memory, it can impact the event
processing rate of the ESA. Performance can be affected from 10 to 30 percent, depending on
your rules and configuration settings.

Workflow

The following diagram shows the data flow using the memory pool for batch mode:

31 Additional Procedures
 Event Stream Analysis Configuration Guide

1. Events are added to the memory pool and references to the events are stored in the memory
pool.

2. The events are then batched to be sent to disk (in non-batch mode, this step is skipped).

3. Once the batch has met the threshold, the events are written to disk (in non-batch mode no
threshold is required).

4. When the EPL requires an event that was written to disk, the event is sent to the cache and
used in the EPL rule. 

Procedure
Complete the following steps to configure an ESA memory pool. 
1. From Administration > Services, select your ESA service and then   > View >
Explore. 

2. Select CEP > EsperPool > Configuration.

3. Enter values for the following fields:

Attribute Description Configuration 

Additional Procedures 32
 Event Stream Analysis Configuration Guide

MapPoolPersistenceURI Location to The default value is

/opt/rsa/esa/pool/esperPool. RSA
store
recommends you do not modify the default
the memory value.
pool file.  If you modify this setting to use a different
partition, ensure the partition contains at least
10 times more space than the memory
allocated for ESA.

Caution: If the memory pool is in use while

this path is changed,a n ESA restart is
required. When this occurs, ESA does not
discard the stored events so you must
manually purge them.

MapPoolEnable Enable or dis- The default value is false. Set the value to true
able the to enable the memory pool.  Requires a restart
memory pool. when you enable or disable memory pool.

MapPoolFlushIntervalSecs Time interval to The default value is 15 minutes. A smaller

flush events to value ensures that the ESA is more stable
disk. For when there are EPLs holding a large number of
example, any events in memory. A larger value (greater
event held in than 30 minutes), ensures that only relevant
Esper longer events required over a longer period of time
than 15 are flushed to disk. 
minutes gets
flushed to disk. Note: Due to Java memory management
design, sometimes events not held by EPL
may be sent to disk. To help prevent this
from occurring, you can set a higher value
for MapPoolFlushIntervalSecs.

33 Additional Procedures
 Event Stream Analysis Configuration Guide

MapPoolBatchWriteSize Specify the The default batch size is 100,000 events. At

batch size (and the end of flush interval, if the batch capacity
whether to use is not reached, the batch expires in 30 seconds
batch and all contents of the batch are written to disk
mode). The as memory pool files.
events are A smaller value for the batch size (for
batched into example, 10,000 events) ensures that when
groups and then events are fetched from disk, they do not pose
flushed to disk. a risk of bloating the memory, which
To use non- creates more stability. However, a larger batch
batch mode, set size (100,000 events) minimizes the
this value to 1. input/output activity when writing events to
To use batch disk, which can create better performance. 
mode, set this
value to greater
than 1.

MapPoolMinSize Minimum size The default value is 10,000 events. A higher

of the memory value may increase performance. A lower
pool. This value value ensures that the system is more stable. 
is used for ini-
tialization, so it
does not
typically require
editing. 

MapPool Persist Type This is a view-  The default value is RMSerialize. 

only parameter
that displays the
type of optim-
ization used. 

Note: The effectiveness of this feature depends on your environment. If you write rules that
require frequent access of events over a period of time, this feature may degrade performance
with no or minimal improvement in scalability.

Note: Memory pool files get deleted when all the events held in the pool file are no longer
referenced by an EPL. 

Additional Procedures 34
 Event Stream Analysis Configuration Guide

Result
For a simple EPL rule, ESA typically improves memory approximately 8 to 9 times.

Configure ESA to Use Capture Time Ordering

This topic tells administrators how to configure the ESA to use capture time ordering when using
two or more Concentrators as a source. 
By default, ESA uses the ESA time stamp (time at which events are received by the ESA) to
correlate events. However, ESA also supports session-ordering based on capture time (the time
at which the packet or log event reached the Decoders). This feature is useful if you are
correlating events from two or more Concentrators. When you have two or more Concentrators
as sources, time ordering ensures that their sessions are correlated together by capture
time. This ensures that sessions captured at the same time are correlated together and alerts are
consistent with user’s expectation even with transmission delays. If any of the sources go offline
or are slow to send sessions, the ESA will pause to ensure that sessions with same capture
timestamps are correlated together.
For example, you have two sources with events that occur at 10:00 a.m. Using Capture Time
Ordering, these events are held in the buffer until the ESA detects that all events occurring at
10:00 a.m. have been added to the buffer. Once all the events have arrived, events are then
processed using EPL rules. This ensures that a rule has all events with the same time-stamp
from different sources in order to obtain correct results. If, for example, one Concentrator lags
behind another, the ESA pauses until it has all the events time-stamped at 10:00 a.m. from both
sources before it runs the EPL rules against the events. 

Caution: Although this feature increases accuracy, it impacts performance. The default
configuration of the ESA ensures that data is constantly streaming, but because Capture Time
Ordering uses a buffer, it takes longer to process events. This is especially true if the ESA
must pause for any length of time to wait for the buffer to fill. There are several parameters
you can configure (see below) to handle this situation; however, there may still be
performance impact. 

By default, this feature is disabled.

Capture Time Order Workflow

The following diagram shows the workflow when Capture Time Ordering is enabled. 

35 Additional Procedures
 Event Stream Analysis Configuration Guide

1. Events are time-stamped as they are captured by the Decoder.

2. After Concentrator processing, events are buffered and ordered. The buffer size is
calculated via two parameters MaxEPSExpectedPerSource (the maximum volume of traffic
(EPS) you expect per source for the ESA to receive) times TimeOrderHoldWindow( the
amount of time to allow for events to arrive from all sources).

3. The ordered events are then correctly correlated in EPL rules. 

Prerequisites
Two or more Concentrators must be configured as a data source in ESA.
When the StreamEnabled parameter is set to true, it is important that all the machines running
Core Services should be in NTP Sync. 

Procedures
The following procedures tell you how to enable and configure Capture Time Ordering.

Additional Procedures 36
 Event Stream Analysis Configuration Guide

Enable Buffering and Capture Time Ordering

Note: After an upgrade or in a high EPS environment, you need to re-add datasources to start

seeing the benefits. Or, you must wait until the sessions catch up before you enable Capture
Time Ordering.

1. In the Security Analytics menu, select Administration > Services. Select your ESA service
and then   > View > Explore. 

2.  Go to Workflow > Source > nextgenAggregationSource.

3. Set the StreamEnabled attribute to true. StreamEnabled allows ESA to buffer events

received from Concentrators.

4. Set the TimeOrdered attribute to true. This enables the buffered events to be ordered by
the time stamp from the Concentrator.

Configure Capture Time Ordering

When you work with Capture Time Ordering, you need to configure several other parameters to
ensure performance. The following table shows parameters and their function. Configuring these
parameters requires knowledge of your traffic volume and rate.

Note: If you do not know your traffic volume or latency, consult with your Professional
Services representative before configuring this feature.

MaxEPSExpectedPerSource Specify the maximum volume of traffic (EPS, or

events per second) you expect for the ESA service to
receive from your busiest source (for example, if one
source receives 20K EPS, and another receives 25K
EPS, set the value at 25K EPS).
If you set this rate too low, there is a short-term impact
on performance. However, ESA automatically
increases the value
for ​MaxEPSExpectedPerSource as needed to make
progress in Time Ordered mode.
The default value is 20K.

37 Additional Procedures
 Event Stream Analysis Configuration Guide

TimeOrderHoldWindow Specify in seconds (whole integers) the amount of time

to allow for events to arrive from all sources. 
Configure this value based on the latency between the
sources.
The default value is 2 seconds. Decreasing this
value can increase the chance of dropped events.
Increasing this value can decrease performance
because more memory is consumed. 

IdleSourceAdvanceAfterSeconds Specify the interval (in seconds) after which the ESA

takes an idle source (no events are coming from the
source, but the source is not offline) out of the equation
to allow progress on a capture time ordered stream. The
default value is 0, meaning that the ESA waits
indefinitely for events to arrive.

OfflineSourceAdvanceAfterSeconds Specify the interval (in seconds) after which the

ESA takes an offline source out of the equation to allow
progress on a capture time ordered stream. The default
value is 0, which means the ESA waits indefinitely.
This parameter does not affect the re-connection
retries; those which are performed in all cases.

Troubleshooting Tips
Using this feature, it is possible to encounter a situation where events become backlogged. To
fix this issue, you can perform one of the following options.

Disable Capture Time Ordering

1. In the Security Analytics menu, select Administration > Services. Select your ESA service
and then   > View > Explore. 

2.  Go to Workflow > Source > nextgenAggregationSource.

3. Set the StreamEnabled attribute to false. 

4. Set the TimeOrdered attribute to false. 

If you disable Capture Time Ordering, you will lose the backlogged data, and events will no
longer be ordered by capture time. 

Additional Procedures 38
 Event Stream Analysis Configuration Guide

Disable Position Tracking

Position tracking allows ESA to track where it stopped processing events if the ESA stops or is
shut down. Position tracking is enabled by default with Capture Time Ordering. If you disable
position tracking, this allows ESA to skip the backlogged events. For example, if the ESA goes
down at 7:00 a.m., and you restart it at 11:00 a.m. with position tracking disabled, the ESA will
start processing events that occurred at 10:55 a.m. With position tracking enabled, the ESA will
start processing events at the point at which it stopped.
1. In the Security Analytics menu, select Administration > Services. Select your ESA service
and then   > View > Explore. 

2.  Go to Workflow > Source > nextgenAggregationSource.

3. Set the PositionTrackingEnabled attribute to false. 
If you disable Position Tracking, you will lose the backlogged data, but going forward, events
will be ordered by capture time. 

Start, Stop, or Restart ESA Service

This topic provides instructions to start, stop, or restart Event Stream Analysis service. 

Start ESA Service
Before you start:
l Make sure that MongoDB is running. 

l If the MongoDB service is not running, use the following command to start the MongoDB
service:
service tokumx start

To start ESA service:

1. Use ssh to connect to the ESA service and log in as the root user.

2. Type the following command and press ENTER:

service rsa-esa start

Stop ESA Service

To stop ESA service:
1.  Use ssh to connect to the ESA service and log in as the root user.

2. Type the following command and press ENTER:

service rsa-esa stop

39 Additional Procedures
 Event Stream Analysis Configuration Guide

Restart ESA Service

To restart ESA service:
1. Use ssh to connect to the ESA service and log in as the root user.

2. Type the following command and press ENTER:

service rsa-esa restart

Audit Logs and Verify ESA Component Versions and Status

This topic provides details about audit logging and instructions to verify the versions of the Event
Stream Analysis components installed.

Audit Log Rules

Audit logging allows you to view details about rules that are created and edited in Security
Analytics.
For details on how to access your audit logs, see Local Audit Log Locations in the System
Configuration Guide.
The following sample shows a create, update, and delete log for a given rule.
l Create log example: 2016-03-10 14:19:37,951 deviceVersion:
"10.6.1.0-SNAPSHOT" deviceService: "EVENT_STREAM_ANALYSIS"
category: SYSTEM operation: "CREATE RULE" parameters: "Epl
Module Identifier: 56e1f2adbee8290008241296, Esper Instance:
default, Rule Enabled: true, Trial Rule: false " key: "Epl
Rule: @RSAAlert select * from Event;" identity: "admin"
userRole: "ROLE_ESA_ADMINISTRATOR"

l Update log example:  2016-03-10 14:19:37,951 deviceVersion:

"10.6.1.0-SNAPSHOT" deviceService: "EVENT_STREAM_ANALYSIS"
category: SYSTEM operation: "UPDATE RULE" parameters: "Epl
Module Identifier: 56e1f2adbee8290008241296, Esper Instance:
default, Rule Enabled: true , Trial Rule: false " key: "Epl
Rule: @RSAAlert select * from Event;" identity: "admin"
userRole: "ROLE_ESA_ADMINISTRATOR

l Delete log example: 2016-03-10 14:19:37,951 deviceVersion:

"10.6.1.0-SNAPSHOT" deviceService: "EVENT_STREAM_
ANALYSIS"category: SYSTEM operation: "DELETE RULE" parameters:
"Epl Module Identifier: 56e1f2adbee8290008241296, Esper

Additional Procedures 40
 Event Stream Analysis Configuration Guide

Instance: default, Rule Enabled: true , Trial Rule: false "

key: "Epl Rule: @RSAAlert select * from Event;" identity:
"admin" userRole: "ROLE_ESA_ADMINISTRATOR "

Each log contains the following parameters:

l Time stamp: Time the rule was modified. Example: 2016-03-10 14:19:37,951

l DeviceVersion: Version of your ESA device. Example: "10.6.1.0-SNAPSHOT"

l DeviceService: Example: EVENT_STREAM_ANALYSIS

l Category: Example: SYSTEM

l Operation:Example:DELETE/CREATE/UPDATE RULE

l Parameters:Placeholder for the following keys:

l Epl Module Identifier: unique identifier for the rule. Example: 56e1f2adbee8290008241296

l Esper Instance: Esper instance on which rule is deployed. Example: default

l Rule Enabled: Displays if the rule is enabled or not. Example: Rule Enabled: true

l Trial Rule: Displays if the rule is configured as a trial rule or not. Example: Trial Rule: false

l Epl Rule: Displays the rule syntax. Example:

@RSAAlert select * from Event;" identity: "admin" userRole:
"ROLE_ESA_ADMINISTRATOR+ROLE_ESA_ADMINISTRATOR+ROLE_ESA_ADMIN"

l Identity: Example: “admin"

l userRole: Example: "ROLE_ESA_ADMINISTRATOR"

Note: When a rule is disabled, two logs are generated for the same rule. First a ‘Delete
Rule’ [Rule enabled attribute = true] audit log is created, followed by a ‘Create Rule’ [Rule
enabled attribute =false] audit log.

Verify ESA Server Version

To verify the ESA Server version:
1. Use ssh to connect to the ESA service and log in as the root user.

2. Type the following command and press ENTER:

rpm -qa | grep rsa-esa-server
The ESA server version is displayed.

41 Additional Procedures
 Event Stream Analysis Configuration Guide

Verify MongoDB Version
To verify the MongoDB version:
1. Use ssh to connect to the ESA service and log in as the root user.

2. Type the following command and press ENTER:

mongo --version
The MongoDB version is displayed.

Verify MongoDB Status

To verify the MongoDB status:
1. Use ssh to connect to the ESA service and log in as the root user.

2. Type the following command and press ENTER:

service tokumx status

3. Run the following command if MongoDB is not running.

service tokumx start

Additional Procedures 42
 Event Stream Analysis Configuration Guide

References
This topic is a collection of references, which describe the user interface for ESA in Security
Analytics. These topics are presented in alphabetical order.
Use this section when you are looking for descriptions of the entitlements user interface and
definitions of the features of the user interface.
See the following sections for details:
l Services Config View Advanced Tab

l Services Config View Data Sources Tab

Services Config View Advanced Tab

This topic describes the components of the Services Config view Advanced tab for ESA.
If you want to configure advanced settings for an ESA service, you can do that from the
Services Config view > Advanced tab of the ESA.

43 References
 Event Stream Analysis Configuration Guide

Features
The following are the sections in the Advanced view:
l Alert Engine

l Event Stream Engine

Alert Engine Settings

In the Alert Engine section, you specify values to preserve events for rules that choose multiple
events.
The following figure shows the Alert Engine section.

The following table lists the parameters in the Alert Engine section and their descriptions.

Parameter Description

Max Con- For rules that choose multiple events, this configuration value decides how
stituent many of the associated events are preserved. For example, if a rule fires an
Events alert with 200 associated events and this parameter is set to 100, only the first
100 are preserved by ESA, the rest are dropped. The default value is 100.

Forward To forward ESA alerts for Incident Management, you must select this option.
Alerts On The ESA alerts generated will be sent to the Message Bus and subsequently to
Message Incident Management. This option is selected by default. You may want to
Bus ensure that the Incident Management service is running.

Debug Selecting enables debugging rules.

Rules?

Event Stream Engine Settings

In the Event Stream Engine section, you specify details to improve performance.

References 44
 Event Stream Analysis Configuration Guide

The following figure shows the Event Stream Engine section.

The following table lists the parameter in the Event Stream Engine section and its description.

Parameter Description

Max Pattern Certain rules require ESPER to maintain subexpressions in memory before
Subexpressions deciding to fire them or not. These subexpressions consume memory and if
left unchecked may cause the service to go down with memory exhaustion.
This parameter is a safety measure that keeps such memory hogging rules
under check. If a rule exceeds the specified number of subexpressions, its
processing is delayed. The default value is 0 which means this setting is dis-
abled. You must set a value if there are service stability issues.

Services Config View Data Sources Tab

This topic describes the components of the Services Config view Data Sources tab for ESA.
The Services Config view > Data Sources tab of the ESA is used to configure data sources for
ESA.

45 References
 Event Stream Analysis Configuration Guide

Features
The following are the sections in the Data Source tab:
l Toolbar

l Data Source grid

Toolbar

The following table describes the options in the toolbar.

Parameter Description

Adds a new data source to ESA.

Deletes a data source from ESA.

Edits a data source. You must have the username and password credentials for
the service in order to make changes.

Enables the selected data source.

References 46
 Event Stream Analysis Configuration Guide

Parameter Description

Disables the selected data source.

Data Source Grid

In the Data Source grid, all data sources which have been added to the ESA service are
displayed. The following table describes the parameters in the Data Source grid.

Parameter Description

Name The name of the data source service.

Address The address of the data source service.

Port The port used by the data source.

User The user connected with the data source.

Enabled Indicates if the data source is enabled.

SSL Indicates if SSL communication is enabled.

Compression Indicates if compression is enabled.

47 References