Rework how hostname/SNI info is saved (#1330)

Looking at `struct ndpi_flow_struct` the two bigger fields are
`host_server_name[240]` (mainly for HTTP hostnames and DNS domains) and
`protos.tls_quic.client_requested_server_name[256]`
(for TLS/QUIC SNIs).

This commit aims to reduce `struct ndpi_flow_struct` size, according to
two simple observations:
 1) maximum one of these two fields is used for each flow. So it seems safe
to merge them;
 2) even if hostnames/SNIs might be very long, in practice they are rarely
longer than a fews tens of bytes. So, using a (single) large buffer is a
waste of memory for all kinds of flows. If we need to truncate the name,
we keep the *last* characters, easing domain matching.

Analyzing some real traffic, it seems safe to assume that the vast
majority of hostnames/SNIs is shorter than 80 bytes.

Hostnames/SNIs are always converted to lowercase.

Attention was given so as to be sure that unit-tests outputs are not
affected by this change.

Because of a bug, TLS/QUIC SNI were always truncated to 64 bytes (the
*first* 64 ones): as a consequence, there were some "Suspicious DGA
domain name" and "TLS Certificate Mismatch" false positives.
This commit is contained in:
Ivan Nardi
2021-11-24 10:46:48 +01:00
committed by GitHub
parent fd02e1b304
commit a8ffcd8bb0
22 changed files with 167 additions and 188 deletions
+3 -4
View File
@@ -986,8 +986,8 @@ struct ndpi_flow_struct {
/* Place textual flow info here */
char flow_extra_info[16];
/* HTTP host or DNS query */
uint8_t host_server_name[240];
char host_server_name[80];
uint8_t initial_binary_bytes[8], initial_binary_bytes_len;
uint8_t risk_checked;
ndpi_risk risk; /* Issues found with this flow [bitmask of ndpi_risk] */
@@ -1051,8 +1051,7 @@ struct ndpi_flow_struct {
struct {
char ssl_version_str[12];
uint16_t ssl_version, server_names_len;
char client_requested_server_name[64], *server_names,
*alpn, *tls_supported_versions, *issuerDN, *subjectDN;
char *server_names, *alpn, *tls_supported_versions, *issuerDN, *subjectDN;
uint32_t notBefore, notAfter;
char ja3_client[33], ja3_server[33];
uint16_t server_cipher;
+1 -1
View File
@@ -686,7 +686,7 @@ NDPIFlowStruct._fields_ = [
("num_processed_pkts", c_uint8),
("extra_packets_func", CFUNCTYPE(c_int, POINTER(NDPIDetectionModuleStruct), POINTER(NDPIFlowStruct))),
("l4", L4),
("host_server_name", c_ubyte * 256),
("host_server_name", c_char * 80),
("http", Http),
("stun", Stun),
("ftp_imap_pop_smtp", FtpImapPopSmtp),