Remove classification "by-ip" from protocol stack (#1743)

Basically:
* "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is
NEVER returned in the protocol stack (i.e.
`flow->detected_protocol_stack[]`);
* if the application is interested into such information, it can access
`ndpi_protocol->protocol_by_ip` itself.

There are mainly 4 points in the code that set the "classification
by-ip" in the protocol stack:  the generic `ndpi_set_detected_protocol()`/
`ndpi_detection_giveup()` functions and the HTTP/STUN  dissectors.

In the unit tests output, a print about `ndpi_protocol->protocol_by_ip`
has been added for each flow: the huge diff of this commit is mainly due
to that.

Strictly speaking, this change is NOT an API/ABI breakage, but there are
important differences in the classification results. For examples:
* TLS flows without the initial handshake (or without a matching
SNI/certificate) are simply classified as `TLS`;
* similar for HTTP or QUIC flows;
* DNS flows without a matching request domain are simply classified as
`DNS`; we don't have `DNS/Google` anymore just because the server is
8.8.8.8 (that was an outrageous behaviour...);
* flows previusoly classified only "by-ip" are now classified as
`NDPI_PROTOCOL_UNKNOWN`.

See #1425 for other examples of why adding the "classification by-ip" in
the protocol stack is a bad idea.

Please, note that IPV6 is not supported :(  (long standing issue in nDPI) i.e.
`ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for
IPv6 flows.

Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed.

Close #1687
This commit is contained in:
Ivan Nardi
2022-09-20 22:24:47 +02:00
committed by GitHub
parent 174cd739db
commit a7c2734b38
379 changed files with 9022 additions and 9061 deletions
+10 -10
View File
@@ -916,14 +916,14 @@ typedef struct {
} ndpi_port_range;
typedef enum {
NDPI_CONFIDENCE_UNKNOWN = 0, /* Unknown classification */
NDPI_CONFIDENCE_MATCH_BY_PORT, /* Classification obtained looking only at the L4 ports */
NDPI_CONFIDENCE_MATCH_BY_IP, /* Classification obtained looking only at the L3 addresses */
NDPI_CONFIDENCE_DPI_PARTIAL, /* Classification results based on partial/incomplete DPI information */
NDPI_CONFIDENCE_DPI_PARTIAL_CACHE, /* Classification results based on some LRU cache with partial/incomplete DPI information */
NDPI_CONFIDENCE_DPI_CACHE, /* Classification results based on some LRU cache (i.e. correlation among sessions) */
NDPI_CONFIDENCE_DPI, /* Deep packet inspection */
NDPI_CONFIDENCE_NBPF, /* PF_RING nBPF (custom protocol) */
/* Try to have "stable" values (across releases/changes) */
NDPI_CONFIDENCE_UNKNOWN = 0, /* Unknown classification */
NDPI_CONFIDENCE_MATCH_BY_PORT = 10, /* Classification obtained looking only at the L4 ports */
NDPI_CONFIDENCE_NBPF = 50, /* PF_RING nBPF (custom protocol) */
NDPI_CONFIDENCE_DPI_PARTIAL = 100, /* Classification results based on partial/incomplete DPI information */
NDPI_CONFIDENCE_DPI_PARTIAL_CACHE = 110, /* Classification results based on some LRU cache with partial/incomplete DPI information */
NDPI_CONFIDENCE_DPI_CACHE = 200, /* Classification results based on some LRU cache (i.e. correlation among sessions) */
NDPI_CONFIDENCE_DPI = 210, /* Deep packet inspection */
/*
IMPORTANT
@@ -1075,12 +1075,12 @@ typedef struct ndpi_proto {
below we do not use ndpi_protocol_id_t as users can define their own
custom protocols and thus the typedef could be too short in size.
*/
u_int16_t master_protocol /* e.g. HTTP */, app_protocol /* e.g. FaceBook */;
u_int16_t master_protocol /* e.g. HTTP */, app_protocol /* e.g. FaceBook */, protocol_by_ip;
ndpi_protocol_category_t category;
void *custom_category_userdata;
} ndpi_protocol;
#define NDPI_PROTOCOL_NULL { NDPI_PROTOCOL_UNKNOWN , NDPI_PROTOCOL_UNKNOWN , NDPI_PROTOCOL_CATEGORY_UNSPECIFIED, NULL }
#define NDPI_PROTOCOL_NULL { NDPI_PROTOCOL_UNKNOWN , NDPI_PROTOCOL_UNKNOWN , NDPI_PROTOCOL_UNKNOWN, NDPI_PROTOCOL_CATEGORY_UNSPECIFIED, NULL }
#define NUM_CUSTOM_CATEGORIES 5
#define CUSTOM_CATEGORY_LABEL_LEN 32