seabreg / yaf Goto Github PK

YAF
===

YAF is Yet Another Flowmeter. It processes packet data from pcap(3) dumpfiles 
as generated by tcpdump(1) or via live capture from an interface using pcap(3),
an Endace DAG card, or a Napatech adapter into bidirectional flows, then 
exports those flows to IPFIX Collecting Processes or in an IPFIX-based file 
format. YAF's output can be used with the SiLK flow analysis tools and the 
NetSA Aggregated Flow (NAF) toolchain. 

YAF also supports partial payload capture - this feature is intended for use 
in "banner grabbing" for protocol verification and service presence detection, 
and is presently experimental.

Why does the world need another network flow event generator? YAF is 
intended as an experimental implementation tracking developments in the IETF 
IPFIX working group, specifically bidirectional flow representation, 
archival storage formats, and structured data export with Deep Packet Inspection. 
It is designed to perform acceptably as a flow sensor
on any network on which white-box flow collection with commodity hardware is 
appropriate, but tradeoffs between raw performance and clarity of design have 
generally been made in favor of the latter.

The YAF toolchain presently consists of two primary tools, yaf itself, and yafscii, 
which converts yaf output into ASCII format. getFlowKeyHash and yafMeta2Pcap are
also distributed with YAF and are helpful tools if using yaf's pcap export options.

By default YAF generates flows based on the standard 5-tuple and VLAN tag, if available.
The 5-tuple consists of the source IP address,
destination IP address, source port, destination port, and protocol.
If YAF is configured with MPLS support, YAF will use the top
three MPLS labels from the MPLS label stack in addition to the 5-tuple and vlan
to determine the flow.  In MPLS mode, it will also export the top three MPLS
labels in the IPFIX record.  The exported fields will not include the
experimental bits and the bottom of stack bit.  In addition, if YAF is
configured with --enable-nonip, YAF will accept non-IP data and generate
flow data using just the MPLS labels.  The 5-tuple and VLAN fields
will be set to 0, and the exported flow will contain start and end times,
packet counts, byte counts, and MPLS labels.  Since the byte count is typically
taken from the length in the IP header, YAF will use the length provided
by libpcap.  Non-IP data can only be exported if MPLS mode is enabled.


Building
========

YAF requires glib 2.4.7 or later; glib is available at 
http://www.gtk.org. Build and install glib before building YAF. Note that 
glib is also included in many operating environments or ports collections.

YAF requires libfixbuf version 2.3.0 or later; libfixbuf is available at
http://tools.netsa.cert.org/fixbuf. Build and install libfixbuf before
building YAF. 

Spread support requires Spread 4.1 or later.  Build and install Spread before 
building YAF. YAF requires libfixbuf version 0.9.0 or later if YAF is compiled
with Spread support.

YAF requires libpcap from http://www.tcpdump.org. Note that libpcap is included
with many operating environments or ports collections.

YAF requires libairframe which is built and installed with the YAF tools
distribution.

Endace DAG live input support requires libdag. Use the --with-dag option to
./configure to enable DAG support.

Napatech live input support requires libpcapexpress. libpcapexpress library 
is available to Napatech customers for download at www.pcapexpress.com. 
Use the --with-napatech option to ./configure to enable Napatech support.  Before
starting YAF with --live napatech, reload Napatech drivers using
pcapxctl -r (Napatech Driver Load/Unload Script).

The YAF application labeling functionality requires the Perl regular expression
library, PCRE.  This library is available at http://www.pcre.org.  If YAF is 
installed in a nonstandard location, it may be necessary to set the 
LTDL_LIBRARY_PATH environment variable to the location where the application
labeling plugins are installed. By default, YAF installs the application labeling
plugins in /usr/local/lib/yaf.

PF_RING and PF_RING ZC (ZERO COPY) support requires libpfring (v. 6.2.0 or higher), available at
http://www.ntop.org/get-started/download/.  Download and install the library
before installing YAF. Install the PF_RING kernel modules and drivers (if necessary).
To use PF_RING ZC, you are required to run yafzcbalance which will load
balance the traffic on one or more interfaces to one or more yaf applications.


The YAF applications also require the included libyaf library.
libyaf implements YAF file and network I/O, packet decoding, fragment 
assembly, and flow generation. This library is built and installed
with the YAF tools distribution.

YAF uses a reasonably standard autotools-based build system. The customary
build procedure (./configure && make && make install) should work in most
environments. Note that YAF finds libfixbuf and libairframe using the
pkg-config(1) facility, so you may have to set the PKG_CONFIG_PATH variable 
on the configure command line if these libraries are installed in a 
nonstandard location, other than the prefix to which you are installing 
YAF itself.

Common Issues when Installing or Running YAF
============================================

Configure Error: configure: error: Cannot find a suitable libfixbuf (>= 1.0.0) 
(Try setting PKG_CONFIG_PATH): No package 'libfixbuf' found

Solution: export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
if libfixbuf was installed in the default location (Otherwise /$prefix/lib/pkgconfig)

---

Runtime Error: yaf: error while loading libraries: libairframe-2.3.0.so.4: 
cannot open share object file: No such file or directory

Solution: Most likely yaf libraries were installed in a nonstandard location.
Try running `ldconfig` or setting LD_LIBRARY_PATH to the location of libairframe. 

---

Runtime Error: "couldn't open library "dnsplugin": file not found"

Solution: Most likely yaf application labeling libraries were installed
in a nonstandard location (default: /usr/local/lib/yaf).
Set LTDL_LIBRARY_PATH to the location of those libraries ($prefix/lib/yaf).
If you are starting yaf via a startup script, it may be necessary to 
export this environment variable from the startup script.

---

Error: yaf terminating on error: Failed to load certificate file: error:0906D06C:PEM routines:PEM_read_bio:no start line

Solution: When running yaf exporting via TLS, the certificate files given to 
--tls-ca and --tls-cert must be in PEM format.  DER format is not accepted.

---

Error: yaf terminating on error: Failed to load private key file: error:0906A068:PEM routines:PEM_do_header:bad password read

Solution: Most likely the key file given to --tls-key requires a password and
the YAF_TLS_PASS environment variable was not set.  Set the YAF_TLS_PASS 
environment variable to the correct password for the --tls-key or remove the
password from the key file (openssl rsa -in key.key -out key.key).

---

Error: yaf terminating on error: Failed to load private key file: error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt

Solution: Most likely the key file given to --tls-key requires a password and
the YAF_TLS_PASS environment variable was set to the incorrect password.  Set
the YAF_TLS_PASS environment variable to the correct password or remove the
password from the key file (openssl rsa -in key.key -out key.key)


Known Issues
============

YAF BPF Filtering is ignored when using --live dag because libpcap is not
used.

If YAF is intended to export to SiLK, the --silk command line flag should
be used.

YAF will not rotate output files if it is not seeing any flow data.  However,
it will continue to write process statistics messages at the configured
interval time to the most recent output file.

Presently, the destinationTransportPort information element contains
ICMP type and code information for ICMP or ICMP6 flows; this is nonstandard
and may not be interoperable with other IPFIX implementations.

Please send bug reports, feature requests, and questions to 
<[email protected]>.