struct ndpi_packet_struct from struct ndpi_flow_struct (#1319)
There are no real reasons to embed `struct ndpi_packet_struct` (i.e. "packet") in `struct ndpi_flow_struct` (i.e. "flow"). In other words, we can avoid saving dissection information of "current packet" into the "flow" state, i.e. in the flow management table. The nDPI detection module processes only one packet at the time, so it is safe to save packet dissection information in `struct ndpi_detection_module_struct`, reusing always the same "packet" instance and saving a huge amount of memory. Bottom line: we need only one copy of "packet" (for detection module), not one for each "flow". It is not clear how/why "packet" ended up in "flow" in the first place. It has been there since the beginning of the GIT history, but in the original OpenDPI code `struct ipoque_packet_struct` was embedded in `struct ipoque_detection_module_struct`, i.e. there was the same exact situation this commit wants to achieve. Most of the changes in this PR are some boilerplate to update something like "flow->packet" into something like "module->packet" throughout the code. Some attention has been paid to update `ndpi_init_packet()` since we need to reset some "packet" fields before starting to process another packet. There has been one important change, though, in ndpi_detection_giveup(). Nothing changed for the applications/users, but this function can't access "packet" anymore. The reason is that this function can be called "asynchronously" with respect to the data processing, i.e in context where there is no valid notion of "current packet"; for example ndpiReader calls it after having processed all the traffic, iterating the entire session table. Mining LRU stuff seems a bit odd (even before this patch): probably we need to rethink it, as a follow-up.
nDPI
What is nDPI ?
nDPI® is an open source LGPLv3 library for deep-packet inspection. Based on OpenDPI it includes ntop extensions. We have tried to push them into the OpenDPI source tree but nobody answered emails so we have decided to create our own source tree
How To Compile nDPI
In order to compile this library do
- ./autogen.sh
- ./configure
- make
To run tests do additionally:
- cd tests; ./do.sh
Please note that the pre-requisites for compilation include:
- GNU tools (autogen, automake, autoconf, libtool)
- GNU C compiler (gcc)
On Debian/Ubuntu systems do:
- sudo apt-get install build-essential git bison flex libpcap-dev libtool libtool-bin autoconf pkg-config automake autogen libjson-c-dev libnuma-dev libgcrypt20-dev libpcre2-dev
FreeBSD
- sudo pkg install gcc git autoconf automake libtool devel/pkgconf json-c gmake
Remember to use gmake and not make on FreeBSD
How To Add A New Protocol Dissector
The entire procedure of adding new protocols in detail:
- Add new protocol together with its unique ID to: src/include/ndpi_protocol_ids.h
- Create a new protocol in: src/lib/protocols/
- Variables to be kept for the duration of the entire flow (as state variables) need to be placed in: src/include/ndpi_typedefs.h in ndpi_flow_tcp_struct (for TCP only), ndpi_flow_udp_struct (for UDP only), or ndpi_flow_struct (for both).
- Add a new entry for the search function for the new protocol in: src/include/ndpi_protocols.h
- Choose (do not change anything) a selection bitmask from: src/include/ndpi_define.h
- Add a new entry in ndpi_set_protocol_detection_bitmask2 in: src/lib/ndpi_main.c
- Set protocol default ports in ndpi_init_protocol_defaults in: src/lib/ndpi_main.c
- ./autogen.sh
- make
- make check
How to use nDPI to Block Selected Traffic
You can use nDPI to selectively block selected Internet traffic by embedding it onto an application (remember that nDPI is just a library). Both ntopng and nProbe cento can do this.
nDPI Paper Citation
- Deri, Luca, et al. nDPI: Open-source high-speed deep packet inspection 2014 International Wireless Communications and Mobile Computing Conference (IWCMC). IEEE, 2014.
nDPI-Related Projects
DISCLAIMER
While we do our best to detect network protocols, we cannot guarantee that our software is error free and 100% accurate in protocol detection. Please make sure that you respect the privacy of users and you have proper authorization to listen, capture and inspect network traffic.
nDPI is a registered trademark in the US and EU.