Flow-based Anomaly Detection:
How and Why it Works
Presenter: David Salter
1
2006 Lancope, Inc. Company Confidential All Rights Reserved
NETWORK ANOMALY DETECTION USING FLOWS
The Challenge in securing the network:
Traditional solutions require in-line devices and / or
host based agents.
Signature and pattern matching technologies only
protect the network from known threats.
In-line devices can impact throughput
Host based solutions protect the perimeter but add a
significant overhead in terms of both management
and host resources and are not infallible
2
2006 Lancope, Inc. Company Confidential All Rights Reserved
NETWORK ANOMALY DETECTION USING FLOWS
Based on analysis of flow data (statistics, changes
in behaviour)
sFlow (Extreme, HP Procurve, Foundry)
NetFlow (Cisco, Juniper)
IPFIX (an
Not signature-based (behavior based)
Designed primarily for internal network deployments
(but can exist at the perimeter if necessary)
Mature but evolving technology
Perfect complement to existing security and network
management technologies
3
2006 Lancope, Inc. Company Confidential All Rights Reserved
COLLECTING FLOW DATA FROM ROUTERS AND SWITCHES
Remote
Sites
Remote
Users
Extranet
Qu ickTime and a
TIFF (LZW) d ecompressor
are needed to see th is picture.
Flow Collector
Marketing
Sales
2006 Lancope, Inc. Company Confidential All Rights Reserved
Servers
WHAT IS NETFLOW?
Cisco Router
NetFlow Packet Header
5
2006 Lancope, Inc. Company Confidential All Rights Reserved
WHAT IS SFLOW?
Almost all Foundry products support
sFlow as well as Extreme and HP
sFlow includes payload
1 in N packets are sent from the switch to
the flow collector
Statistical scaling is used to recover the
actual network traffic patterns from the
sFlow samples
The more samples, the more accurate
analysis becomes
Duplicate sFlow PDUs must be handled
and removed
6
2006 Lancope, Inc. Company Confidential All Rights Reserved
CONFIGURING NETFLOW AND SFLOW
Foundry switch (sFlow)
config>inte1/1to1/48
interface>sflowforwarding
config>sflowdestination10.1.1.56343
config>sflowsample128
config>sflowpollinginterval30
config>sflowenable
Cisco router (NetFlow)
router(config)#ipcacheflowtimeoutactive5
router(config)#ipflowexportversion5peeras
router(config)#ipflowexportdestination10.1.1.52055
router(configif)#iproutecacheflow
7
2006 Lancope, Inc. Company Confidential All Rights Reserved
NETFLOW IMPACT ON THE ROUTER (CPU)
Check on
current router
CPU utilization*
* NetFlow v5 adds approximately 10% to overall CPU
2006 Lancope, Inc. Company Confidential All Rights Reserved
NETFLOW IMPACT ON THE NETWORK (BANDWIDTH)
Number of active flows
Flows per second (fps)
9
2006 Lancope, Inc. Company Confidential All Rights Reserved
VIEWING THE ROUTER NETFLOW CACHE DIRECTLY
Worm Infected
Host
Target Hosts
Target Port
(0x87=135)
10
2006 Lancope, Inc. Company Confidential All Rights Reserved
CAPTURING AND VIEWING NETFLOW PACKETS: FLOW-TOOLS
FLOW-TOOLS example of scanning activity
start and end
times
src
interface
src
IP
src
port
dst
interface
dst
IP
proto
dst
port
pkts
bytes
TCP
flags
(2=SYN)
11
2006 Lancope, Inc. Company Confidential All Rights Reserved
DATA REDUCTION: FLOW NORMALIZATION
1. Request webpage from [Link] ([Link])
2. Two NetFlow records are exported from the router
3. StealthWatch associates the two NetFlow records, building one stateful entry
Start Time
3/26/05 9:04
Client Host
[Link]
Server Host
[Link]
Protocol Client Pkts Server Pkts Client Port
TCP
28
42
32806
Server Port
80
12
2006 Lancope, Inc. Company Confidential All Rights Reserved
CHALLENGES WITH FLOW-BASED MONITORING
Duplicate flows are often seen (and must be removed)
Implementations vary from vendor to vendor
No payload data (must rely on statistics; not so easy)
Requires all routers be NTP synced and share similar
settings (for proper security processing)
13
2006 Lancope, Inc. Company Confidential All Rights Reserved
BEHAVIOR-BASED FLOW ANALYSIS FUNCTIONAL OVERVIEW
Build Profile Host
Attributes
Apply
Algorithms to
Flow data
Store Detailed Log
of All Flows
Generate
Alarms, Alerts,
and Reports
Generate
Profile-Enhanced
Alarms, Alerts,
and Reports
Send SYSLOG,
SNMP, and
Emails
Perform
Mitigation Action
Display in UI
Collect,
Deduplicate, and
Process Flow
Statistics
Flow Enabled
Routers
2006 Lancope, Inc. Company Confidential All Rights Reserved
14
IF WE DONT HAVE PAYLOAD, HOW DO DETECT ATTACKS?
Look for patterns of behaviour in flow traffic
One hosts contacting large numbers of other hosts
in short time frame (P2P applications, worms)
Long flow durations (VPNs, covert channels)
Bandwidth anomalies (DoS, warez servers)
Unauthorized ports in use (rogue servers,
applications)
Unauthorized communications (VPN host talking to
accounting server)
15
2006 Lancope, Inc. Company Confidential All Rights Reserved
BENEFIT: ENTERPRISE-WIDE VISIBILITY
16
2006 Lancope, Inc. Company Confidential All Rights Reserved
BENEFIT: ENTERPRISE WIDE VISIBILITY IN ACTION
17
2006 Lancope, Inc. Company Confidential All Rights Reserved
BENEFIT: LIGHT-WEIGHT, EASY TO DEPLOY
12
IDS/IPS
2 IDP/IPS
Sensors
Sensors
Required
Required
18
2006 Lancope, Inc. Company Confidential All Rights Reserved
BENEFIT: LIGHT-WEIGHT, EASY TO DEPLOY
1 NetFlow
Collector
Required
2 IDP/IPS
Sensors
Required
19
2006 Lancope, Inc. Company Confidential All Rights Reserved
BENEFIT: POWERFUL LOGGING AND FORENSICS
NetFlow v5 Details
PIX Firewall Log Details
Flow Duration
Client Host IP
Server Host IP
Start Time
Last Time
Status
Protocol
Server Port
Client Port
Server Packets
TCP Flags
Client Packets
Client payload
Server Payload
Source AS
Destination AS
ToS
Source Interface
Destination Interface
Kbps Rate
Server Header Bytes
Client Header Bytes
Server Payload Bytes
Client Payload Bytes
Fragmentation
Nexthop Router
Source Netmask
Target Netmask
Source IP
Target IP
Protocol
Port
Length
IP Precedence
Status
Interface
2006 Lancope, Inc. Company Confidential All Rights Reserved
20
INFRASTRUCTURE IPS: HOW IT WORKS
Remote
Sites
Remote
Users
disable
port
Extranet
!
Marketing
Sales
2006 Lancope, Inc. Company Confidential All Rights Reserved
StealthWatch
Servers
Qu ickTime and a
TIFF (LZW) d ecompressor
are needed to see th is picture.
21
NETWORK TRAFFIC ANALYSIS AND VISUALIZATION
Flow Records
Traffic Analysis
Visualization
22
2006 Lancope, Inc. Company Confidential All Rights Reserved
SUMMARY
Flow analysis provides more than just traffic
monitoring.
Flow analysis provides powerful forensics, auditing,
and attack detection capability without the need for
additional hardware or software updates.
Both open-source and commercial products are
available for analyzing Flow data.
Flow analysis allows for detection of new worms
without the need for signature updates and in-line
solutions.
23
2006 Lancope, Inc. Company Confidential All Rights Reserved
Thank you
Presenter:
David Salter, Lancope
dsalter@[Link]
24
2006 Lancope, Inc. Company Confidential All Rights Reserved
