Commit Graph
25 Commits
Author SHA1 Message Date
Ivan Nardi 513e386959 Extend protocols support (#1422)
Add detection of AccuWeather site/app and Google Classroom.
Improve detection of Azure, Zattoo, Whatsapp, MQTT and LDAP.

Fix some RX false positives.

Fix some "Uncommon TLS ALPN"-risk false positives.

Fix "confidence" value for some Zoom/Torrent classifications.

Minor fix in Lua script for Wireshark extcap.

Update .gitignore file.

Let GitHub correctly detect the language type of *.inc files.

Zattoo example has been provided by @subhajit-cdot in #1148.
2022-01-29 09:19:26 +01:00
Ivan Nardi 3a087e951d Add a "confidence" field about the reliability of the classification. (#1395)
As a general rule, the higher the confidence value, the higher the
"reliability/precision" of the classification.

In other words, this new field provides an hint about "how" the flow
classification has been obtained.
For example, the application may want to ignore classification "by-port"
(they are not real DPI classifications, after all) or give a second
glance at flows classified via LRU caches (because of false positives).

Setting only one value for the confidence field is a bit tricky: more
work is probably needed in the next future to tweak/fix/improve the logic.
2022-01-11 15:23:39 +01:00
Alfredo Cardigliano 23a4761276 Update copyright 2022-01-03 11:00:45 +01:00
Ivan Nardi 7cba34a1e3 Improve IPv6 support, enabling IPv6 traffic on (almost) all dissectors. (#1380)
There are no valid reasons for a (generic) protocol to ignore IPv6
traffic.
Note that:
* I have not found the specifications of "CheckPoint High Availability
Protocol", so I don't know how/if it supports IPv6
* all LRU caches are still IPv4 only

Even if src_id/dst_id stuff is probably useless (see #1279), the right
way to update the protocol classification is via `ndpi_set_detected_protocol()`
2021-12-04 13:30:10 +01:00
Ivan Nardi 730c2360bd Remove struct ndpi_packet_struct from struct ndpi_flow_struct (#1319)
There are no real reasons to embed `struct ndpi_packet_struct` (i.e. "packet")
in `struct ndpi_flow_struct` (i.e. "flow"). In other words, we can avoid
saving dissection information of "current packet" into the "flow" state,
i.e. in the flow management table.

The nDPI detection module processes only one packet at the time, so it is
safe to save packet dissection information in `struct ndpi_detection_module_struct`,
reusing always the same "packet" instance and saving a huge amount of memory.
Bottom line: we need only one copy of "packet" (for detection module),
not one for each "flow".

It is not clear how/why "packet" ended up in "flow" in the first place.
It has been there since the beginning of the GIT history, but in the original
OpenDPI code `struct ipoque_packet_struct` was embedded in
`struct ipoque_detection_module_struct`, i.e. there was the same exact
situation this commit wants to achieve.

Most of the changes in this PR are some boilerplate to update something
like "flow->packet" into something like "module->packet" throughout the code.
Some attention has been paid to update `ndpi_init_packet()` since we need
to reset some "packet" fields before starting to process another packet.

There has been one important change, though, in ndpi_detection_giveup().
Nothing changed for the applications/users, but this function can't access
"packet" anymore.
The reason is that this function can be called "asynchronously" with respect
to the data processing, i.e in context where there is no valid notion of
"current packet"; for example ndpiReader calls it after having processed all
the traffic, iterating the entire session table.

Mining LRU stuff seems a bit odd (even before this patch): probably we need
to rethink it, as a follow-up.
2021-10-05 15:49:52 +02:00
Ivan Nardi 5cf3fef8f7 Remove detected_protocol_stack field from ndpi_packet_struct (#1317)
This field is an exact copy of `ndpi_flow_struct->detected_protocol_stack[2]`:
* at the very beginning of packet dissection, the value saved in
`flow->detected_protocol_stack` is copied in `packet->detected_protocol_stack`
(via `ndpi_detection_process_packet()` -> `ndpi_init_packet_header()`)

* every time we update `flow->detected_protocol_stack` we update
`packet->detected_protocol_stack` too (via `ndpi_int_change_protocol()`
-> `ndpi_int_change_packet_protocol()`)

These two fields are always in sync: keeping the same value in two
different places is useless.
2021-09-29 13:11:32 +02:00
Ivan Nardi 8fdffbf3a1 Compile everything with "-W -Wall -Wno-unused-parameter" flags (#1276)
Fix all the warnings.

Getting rid of "-Wno-unused-parameter" is quite complex because some
parameters usage depends on compilation variable (i.e.
`--enable-debug-messages`).

The "-Werror" flag has been added only in Travis builds to avoid
breaking the builds to users using uncommon/untested
OS/compiler/enviroment.

Tested on:
* x86_64; Ubuntu 20.04; gcc 7,8,9,10,11; clang 7,8,9,10,11,12
* x86_64; CentOS 7.7; gcc 4.8.5 (with "--disable-gcrypt" flag)
* Raspberry 4; Debian 10.10; gcc 8.3.0
2021-08-20 18:11:13 +02:00
Luca Deri 68b6ac7da8 (C) Update 2021-01-07 11:13:36 +01:00
Nardi Ivan b1a6c6b895 Fix some compilation warnings 2020-04-20 16:53:39 +02:00
Luca Deri 29dd45838d Updated (C) 2020-01-05 18:24:58 +01:00
Luca Deri ab9246f51f Adedd DTLS check in STUN
Uodated (C)
2019-09-21 09:16:56 +02:00
Luca 667e77c609 Major code cleanup
Converted some not popular protocols to NDPI_PROTOCOL_GENERIC with category detection
2018-07-21 16:02:54 +02:00
Luca Deri 78feabfad7 Inital hyperscan work 2018-01-24 22:11:05 +01:00
Vitaly Lavrov 2787c2390c Refactoring the debugging output.
levels of debug output:
 0 - ERROR: Only for errors.
 1 - TRACE: Start of each packets and if found protocol.
 2 - DEBUG: Start of searching each protocol and excluding protocols.
 3 - DEBUG_EXTRA: For all other messages.

Added field ndpi_struct->debug_logging for enable debug output of each protocols.

Simple macros for debugging output are added:
NDPI_LOG_ERR(), NDPI_LOG_INFO(), NDPI_LOG_DBG(), NDPI_LOG_DBG2(),
NDPI_EXCLUDE_PROTO()
2017-10-26 20:41:22 +03:00
Josh Soref 7bac815938 spelling: protocol 2017-03-12 22:00:38 +00:00
Alexis La Goutte 65f61fd5b1 rx: fix unused variable 'found' [-Wunused-variable] 2016-11-13 18:22:03 +01:00
Luca Deri 72efa329db Enhanced fragments support in ndpiReader
Improved RX protocol dissection and removed port guess as it caused false positives.
2016-05-31 12:20:25 +02:00
Luca Deri a79b338208 Compilation fix due t pack/unpack 2016-05-30 09:09:00 +02:00
Campus f8ecdc004f added new fb detection - remove old bittorrent functions - fix rx warning 2016-05-24 23:28:43 +02:00
Alexis La Goutte 3d23ee0e23 RX: fix indent 2016-05-23 19:56:45 +02:00
Campus 1b9aa2ab71 fixed rx protocol for better logic conditions 2016-04-26 18:49:52 +02:00
Campus 1d17ece9ce fix missing conditional and 2016-04-26 15:02:32 +02:00
Campus f66cfca2a9 fixed and improved rx protocol detection - there was error on detection when -d option was enabled 2016-04-26 13:48:43 +02:00
Giovanni Mascellani 4b50045ecd Add file license. 2016-04-15 16:41:23 +02:00
Giovanni Mascellani 18f5362a34 Support RX protocol (used by AFS). 2016-04-15 15:21:02 +02:00