Meta-data format and associated tools for the analysis of pcap data

This package contains meta-data format for the analysis of pcap file, and associated tools to:

1) annotate analysis results according to the meta-data format,

2) compare analysis results from different algorithms, and

3) slice and merge marked regions from multiple pcap files.

Download from this link

This tool was created in a desire to accurately communicate the result of pcap analysis.  Many engineers and scientists have been working on pcap files, yet we did not have any effective means to communicate what we have found.

Here we consider adopting common meta-data format across different analysis techniques.  If different analysis techniques can produce compatible mark-ups against the same dataset, we can compare their results without translating or converting the mark-ups.  There are lots of potential benefits that we can obtain from common meta-data format.

As a starting point of meta-data format, XML Schema for annotating the result of analysis made available here, along with C and C++ API to annotate pcap dataset according to the XML Schema.  All algorithms and analysis programs may benefit from this XML Schema and API.  Publicly available datasets will also benefit from this XML Schema, since they will be richly annotated with analysis made by multiple algorithms.

The primary focus of this XML Schema is content (annotated results) and reproducibility (algorithm description and parameters).  The nature of each pcap dataset, e.g., date and observation point, should be better described by CAIDA's DatCat tools.  This tool focuses more on individual record or flow in pcap datasets.

Installation

It requires boost, xerces, pthread, pcap libraries for compilation.  It currently runs on Linux and Mac OSX.

It also requires CodeSynthesis XSD, in order to generate XML data bindings from XML schema definition.  XSD can be downloaded from the following website:

    http://www.codesynthesis.com/products/xsd/

Autoconf tools are used to generate Makefiles.  Simple type:

    $ ./bootstrap

    $ ./configure

    $ make

Developer Documentation

This software is currently intended for developers/scientists who have his/her pcap analyzers handy.  C and C++ APIs are available; API documentation can be automatically generated by Doxygen.  There are some rudimentary examples in sample/ subdirectory.

This tool and meta-data requires your input.  Your feedback is welcomed.  Please send your feedbacks to discussion forum or mailing list on admd.sourceforge.net.

Acknowledgments

The development of this particular implementation has been funded by the National Institute of Information and Communications Technology (NICT), Japan.

This work has been motivated by our collaboration with Kensuke Fukuda of National Institute of Informatics (NII), Japan, Patrice Abry and Pierre Borgnat of ENS-Lyon, France, and k claffy and Emile Aben of CAIDA, USA.