yaf deep packet inspection
If yaf
is built with plugin support enabled (using the --enable-plugins option to ./configure when yaf
is built), then yaf
can examine packet payloads, capture useful information for a specific protocol, and export it in a variable length field.
The DPI plugin requires payload capture to be enabled with the --max-payload option. A minimum payload capture length of 384 octets is recommended for best results. --applabel is also required, as the application label determines how the inspection will execute.
In order to enable DPI in yaf
the following should be added to the command line:
--plugin-name=/usr/local/lib/dpacketplugin.la
You can also add the option flag to specify which protocols to perform DPI:
--plugin-opts="53 80 21"
The above will perform DPI for DNS, HTTP, and FTP.
DPI operates differently depending on whether the protocol is plugin-based or regex-based in the yafApplabelRules.conf file. If the protocol uses a regex rule for application labeling, it will have a file denoted in the yafDPIRules.conf file that consists of a number of regular expressions that are compared against the captured payload. Any matches are stored and later exported in an IPFIX information element. If the protocol is based on a plugin rule, it will store important information while it is decoding the payload using the dynamically loaded plugin listed in the yafApplabelRules.conf file. See the source code to the plugins included with yaf
for details on the specific protocol implementations.
DPI in yaf
is directly related to application labeling as it will only perform DPI if a match was found during the application labeling phase, and it will only execute an inspection specific to the protocol denoted by the application label.
The yafDPIRules.conf file should be in the same location as the yafApplabelRules.conf file. The file is a list of file name, application label pairs, separated by a space, such as:
/HTTPRules.conf 80
The files listed in yafDPIRules.conf should be located with the yafApplabelRules.conf file. The DPI plugin will open each of the files listed and compile and store all of the regular expressions with the given label. After the flow closes and is given an application label, it will search for any DPI regular expressions associated with the label, and compare the payload against them. The file format for the files listed in the yafDPIRules.conf should have the following format:
<ID> <Regular Expression>
The regular expression is a PCRE regular expression and will be stored and associated with the ID number preceding it. Each ID number should be unique across all DPI rule files for easy correlation. This ID number is an unsigned 16-bit decimal integer in the range 0-65535. For example the following is in the HTTPRules.conf file:
3 (HTTP/\d/\d\b)
An ID of 3 is associated with the HTTP version number. Also note, there should be parenthesis around the substring you want to capture and store. If there is more than 1 set of parenthesis in the regular expression, the most outer set is the substring captured. (See PCRE documentation for details on regular expressions and substring matching.)
Upon yaf
startup and capture, you will be able to see if the rule files and their regular expressions were accepted using the --verbose flag.
[2010-06-22 20:05:23] DPI Running for All Protocols
[2010-06-22 20:05:23] Reading packets from packets.pcap
[2010-06-22 20:05:23] DPI rule scanner accepted 14 rules from /usr/local/etc/HTTPRules.conf
[2010-06-22 20:05:23] DPI rule scanner accepted 1 rules from /usr/local/etc/SSHRules.conf
[2010-06-22 20:05:23] DPI rule scanner accepted 9 rules from /usr/local/etc/SMTPRules.conf
[2010-06-22 20:05:23] DPI rule scanner accepted 5 rules from /usr/local/etc/FTPRules.conf
[2010-06-22 20:05:23] DPI rule scanner accepted 7 rules from /usr/local/etc/IMAPRules.conf
[2010-06-22 20:05:23] DPI rule scanner accepted 11 rules from /usr/local/etc/RTSPRules.conf
[2010-06-22 20:05:23] DPI rule scanner accepted 8 rules from /usr/local/etc/SIPRules.conf
[2010-06-22 20:05:23] DPI Rule Scanner accepted 7 rule files
Note that the above statements will not print unless yaf
is receiving input. An unacceptable regular expression will be brought to your attention with the above statements. If you choose certain protocols for inspection using the --plugin-opts
flag, only the appropriate files will be loaded into the DPI Rule Scanner.
These options can be given to ./configure when yaf
is built to export DNS authoritative and NXDomain Responses only.
Enable export of DNS Authoritative Responses only. The default is to capture and export all DNS Responses. This flag can be used in conjunction with --enable-exportDNSNXDomain. It is only recognized if --plugin-name is set to the DPI plugin, application labeling is enabled, and --max-payload is set.
Enable export of DNS NXDomain Responses only. The default is to capture and export all DNS Responses. This flag can be used in conjunction with --enable-exportDNSAuth. It is only recognized if --plugin-name is set to the DPI plugin, application labeling is enabled, and --max-payload is set.
Two variable length data fields will be used in the exported IFPIX template to represent all captured strings and their properties. The variable index field consists of a set of unsigned 16-bit identifier and offset pairs. The variable data field will contain all of the captured strings concatenated together. As stated above, each captured string is associated with an ID number. The variable index field will contain the ID number and the length of each string captured. One of the major advantages to this format is that new identifiers can be added without formally adding information elements and changing yaf
source code. See the below list for a full table of current DPI ID numbers.
[<id=45><len=14><id=46><len=16>]
[<len=30>www.google.comwww.l.google.com]
ID 45 correlates to a DNS Query and ID 46 correlates to a DNS Response by table lookup. Therefore, the DNS Query has length of 14 and the DNS Response has a length of 16. Now parsing the varData string, we have www.google.com as the DNS Query and www.l.google.com as the DNS Response.
Currently, yaf
can export a maximum of 256 bytes in the varIndex and varData fields.
51 (REST | RETR| STO)
52 USER
53 PASS
54 TYPE
55 Return code
=over 1
1 GET
2 Connection
3 HTTP Version
4 Referer
5 Accept
6 Accept-Language
7 Host
8 Content-Length
9 Age
10 Server
11 User-Agent
12 Return Code
13 Content-Type
14 Location
61 CAPABILITY
62 LOGIN
63 STARTTLS
64 AUTHENTICATE
65 SELECT | EXAMINE | CREATE | DELETE | RENAME | SUBSCRIBE | UNSUBSCRIBE | LIST | LSUB| STATUS | APPEND
66 EXISTS
67 RECENT
5541 rtsp://
5542 RTSP Version
5543 RTSP Version + Return Code
5544 Content-Length
5545 Return value
5546 Content-type
5547 Transport
5548 CSeq
5549 Packets Received
5540 Location
231 INVITE
232 OPTIONS | ACK | BYE | CANCEL | REGISTER
233 Via
234 Max-Forwards
235 To
236 From
237 Contact
238 Content Length
101 HE|EH LO
102 From:
103 RCPT To|Cc|Bcc
104 Content-type
105 Subject
106 filename
107 Content-disposition
108 Return Code
201 SSH Version
45 Query
46 Response
200 message
201 Join message
202 Operational message
120 Response Message
119 Command Message
110 Request Msg
111 Response Msg
90 Version
91 Function Code
69 Filename
70 mode (netascii|octet|mail)
Emily Ecoff <ecoff@cert.org> and the CERT Network Situational Awareness Group Engineering Team, http://www.cert.org/netsa.
yaf(1), yafscii(1), PCRE Documentation