Ivan Nardi acb1de69aa Reduce memory used by ndpiReader (#1371)
`ndpiReader` is only an example, aiming to show nDPI capabilities
and integration, without any claim about performances.

Nonetheless its memory usage per flow is *huge*, limiting the kinds
of traces that we can test on a "normal" hardware (example: scan
attacks).

The key reason of that behaviour is that we preallocate all the memory
needed for *all* the available features.

Try to reduce memory usage simply allocating some structures only
when they are really needed. Most significant example: JOY algorithms.

This way we should use a lot less memory in the two most common
user-cases:
 * `ndpiReader` invoked without any particular flag (i.e `ndpiReader -i
$FILENAME_OR_IFACE`)
 * internal unit tests

Before (on x86_64):
```
struct ndpi_flow_info {
[...]
	/* size: 7320, cachelines: 115, members: 72 */
```
After:
```
struct ndpi_flow_info {
[...]
	/* size: 2128, cachelines: 34, members: 75 */
```
2021-11-11 12:37:25 +01:00
2021-11-02 22:08:15 +01:00
2021-11-11 12:36:55 +01:00
2021-11-11 12:36:55 +01:00
2021-11-11 12:36:55 +01:00
2020-01-08 22:44:30 +01:00
2019-12-16 01:09:09 +01:00
2021-07-24 17:54:32 +02:00
2019-07-23 10:47:25 +01:00
2015-04-19 07:25:59 +02:00
2015-04-19 07:25:59 +02:00
2021-09-22 12:10:52 +02:00
2020-08-22 17:02:11 +02:00

ntop ntop

nDPI

Build Status Code Quality: Cpp Total Alerts Fuzzing Status

What is nDPI ?

nDPI® is an open source LGPLv3 library for deep-packet inspection. Based on OpenDPI it includes ntop extensions. We have tried to push them into the OpenDPI source tree but nobody answered emails so we have decided to create our own source tree

How To Compile nDPI

In order to compile this library do

  • ./autogen.sh
  • ./configure
  • make

To run tests do additionally:

  • cd tests; ./do.sh

Please note that the pre-requisites for compilation include:

  • GNU tools (autogen, automake, autoconf, libtool)
  • GNU C compiler (gcc)

On Debian/Ubuntu systems do:

  • sudo apt-get install build-essential git bison flex libpcap-dev libtool libtool-bin autoconf pkg-config automake autogen libjson-c-dev libnuma-dev libgcrypt20-dev libpcre2-dev

FreeBSD

  • sudo pkg install gcc git autoconf automake libtool devel/pkgconf json-c gmake

Remember to use gmake and not make on FreeBSD

How To Add A New Protocol Dissector

The entire procedure of adding new protocols in detail:

  1. Add new protocol together with its unique ID to: src/include/ndpi_protocol_ids.h
  2. Create a new protocol in: src/lib/protocols/
  3. Variables to be kept for the duration of the entire flow (as state variables) need to be placed in: src/include/ndpi_typedefs.h in ndpi_flow_tcp_struct (for TCP only), ndpi_flow_udp_struct (for UDP only), or ndpi_flow_struct (for both).
  4. Add a new entry for the search function for the new protocol in: src/include/ndpi_protocols.h
  5. Choose (do not change anything) a selection bitmask from: src/include/ndpi_define.h
  6. Add a new entry in ndpi_set_protocol_detection_bitmask2 in: src/lib/ndpi_main.c
  7. Set protocol default ports in ndpi_init_protocol_defaults in: src/lib/ndpi_main.c
  8. ./autogen.sh
  9. make
  10. make check

How to use nDPI to Block Selected Traffic

You can use nDPI to selectively block selected Internet traffic by embedding it onto an application (remember that nDPI is just a library). Both ntopng and nProbe cento can do this.

nDPI Paper Citation

DISCLAIMER

While we do our best to detect network protocols, we cannot guarantee that our software is error free and 100% accurate in protocol detection. Please make sure that you respect the privacy of users and you have proper authorization to listen, capture and inspect network traffic.

nDPI is a registered trademark in the US and EU.

S
Description
No description provided
Readme
226 MiB
Languages
C 91.8%
Lua 2.3%
C++ 2%
Makefile 1.3%
Shell 1.1%
Other 1.5%