Commit Graph
17 Commits
Author SHA1 Message Date
Toni 9b8679a320 Fix some race conditions by using atomic operations. (#1420)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-29 09:18:51 +01:00
Toni 011ee3ecbd Fixed wrong ip tuple comparison. #1386 (#1418)
* Added u32 pads to `union ip_tuple` so btree search should now work as expected.
   The bug caused new flow's when the remote answers, resulting in two Flows per direction. Fail.
 * Fixed a race condition during shutdown phase.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-26 15:47:38 +01:00
Ivan Nardi 91bb77a880 A final(?) effort to reduce memory usage per flow (#1389)
Remove some unused fields and re-organize other ones.
In particular:
* Update the parameters of `ndpi_ssl_version2str()` function
* Zattoo, Thunder: these timestamps aren't really used.
* Ftp/mail: these protocols are dissected only over TCP.
* Attention must be paid to TLS.Bittorrent flows to avoid invalid
read/write to `flow->protos.bittorrent.hash` field.

This is the last(?) commit of a long series (see 22241a1d, 227e586e,
730c2360, a8ffcd8b) aiming to reduce library memory consumption.

Before, at nDPI 4.0 (more precisly, at a6b10cf7, because memory stats
were wrong until that commit):
```
nDPI Memory statistics:
	nDPI Memory (once):      221.15 KB
	Flow Memory (per flow):  2.94 KB
```
Now:
```
nDPI Memory statistics:
	nDPI Memory (once):      231.71 KB
	Flow Memory (per flow):  1008 B       <---------
```
i.e. memory usage per flow has been reduced by 66%, dropping below the
psychological threshold of 1 KB.

To further reduce this value, we probably need to look into #1279:
let's fight this battle another day.
2021-12-22 19:54:06 +01:00
Ivan Nardi a8ffcd8bb0 Rework how hostname/SNI info is saved (#1330)
Looking at `struct ndpi_flow_struct` the two bigger fields are
`host_server_name[240]` (mainly for HTTP hostnames and DNS domains) and
`protos.tls_quic.client_requested_server_name[256]`
(for TLS/QUIC SNIs).

This commit aims to reduce `struct ndpi_flow_struct` size, according to
two simple observations:
 1) maximum one of these two fields is used for each flow. So it seems safe
to merge them;
 2) even if hostnames/SNIs might be very long, in practice they are rarely
longer than a fews tens of bytes. So, using a (single) large buffer is a
waste of memory for all kinds of flows. If we need to truncate the name,
we keep the *last* characters, easing domain matching.

Analyzing some real traffic, it seems safe to assume that the vast
majority of hostnames/SNIs is shorter than 80 bytes.

Hostnames/SNIs are always converted to lowercase.

Attention was given so as to be sure that unit-tests outputs are not
affected by this change.

Because of a bug, TLS/QUIC SNI were always truncated to 64 bytes (the
*first* 64 ones): as a consequence, there were some "Suspicious DGA
domain name" and "TLS Certificate Mismatch" false positives.
2021-11-24 10:46:48 +01:00
Ivan Nardi afc2b641eb Fix writes to flow->protos union fields (#1354)
We can write to `flow->protos` only after a proper classification.

This issue has been found in Kerberos, DHCP, HTTP, STUN, IMO, FTP,
SMTP, IMAP and POP code.
There are two kinds of fixes:
 * write to `flow->protos` only if a final protocol has been detected
 * move protocol state out of `flow->protos`
The hard part is to find, for each protocol, the right tradeoff between
memory usage and code complexity.

Handle Kerberos like DNS: if we find a request, we set the protocol
and an extra callback to further parsing the reply.

For all the other protocols, move the state out of `flow->protos`. This
is an issue only for the FTP/MAIL stuff.

Add DHCP Class Identification value to the output of ndpiReader and to
the Jason serialization.

Extend code coverage of fuzz tests.

Close #1343
Close #1342
2021-11-15 16:20:57 +01:00
Ivan Nardi 9e0f0ce3df Fix access to some TLS fields in flow structure (#1277)
Fields 'tls.hello_processed` and `tls.subprotocol_detected` are used by
QUIC (i.e UDP...), too.
2021-08-20 18:11:37 +02:00
Toni 8d0c7b1fae Fixed Mingw64 build, SonerCloud-CI and more. (#1273)
* Added ARM build and unit test run for SonarCloud-CI.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Fixed Mingw64 build.

 * adapted to SonarCloud-CI workflow
 * removed broken and incomplete Windows example (tested on VS2017/VS2019)
 * removed unnecessary include (e.g. pthread.h for the library which does not make use of it)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-08-18 11:34:16 +02:00
Toni 6ad0d6666c Implemented function to retrieve flow information. #1253 (#1254)
* fixed [h]euristic typo

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-23 10:37:20 +02:00
Luca Deri 87ad2b58dc Compilation fix 2021-04-26 10:50:19 +02:00
Luca Deri fc3db8f169 Implemented TLS Certificate Sibject matching
Improved AnyDesk detection
2021-02-22 22:37:33 +01:00
Luca Deri 288ccd6215 Fixes due to datatype rename 2021-01-22 09:17:34 +01:00
Luca Deri 019a64630b Merge pull request #983 from lnslbrty/fix/libpcap-obsolete-pcap_lookupdev-usage
Replaced obsolete libpcap pcap_lookupdev with pcap_findalldevs.
2020-08-16 10:03:33 +02:00
Toni Uhlig b31fde4bbb Replaced obsolete libpcap pcap_lookupdev with pcap_findalldevs.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-15 12:35:49 +02:00
Toni Uhlig aa856735c0 num_extra_packets_checked check can be 0 for some protocols and therefor requires lesser-or-equal condition for max_extra_packets_to_check
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-13 10:32:31 +02:00
Toni Uhlig 8da5f42fa0 Changed ndpi_ssl_version2str function call in ndpiSimpleIntegration.
Fixes build error introduced with 23c072153.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-08 12:05:11 +02:00
lucaderi ecdf7df454 Compilation fixes for non-Linux (or outdated Linux) platforms 2020-06-25 10:25:24 +02:00
Toni Uhlig 17c26911fb ndpiSimpleIntegration: added another integration example
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-24 22:03:18 +02:00