Commit Graph
3468 Commits
Author SHA1 Message Date
Toni bb59f2e371 Removed Makefile references to legacy code. (#1591)
- feature removed with f5545a80f9

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-08 13:37:44 +02:00
Luca Deri 5f05054c8a Added priv_data
Cherry pick of https://github.com/ntop/nDPI/commit/2560260a41172a07b6b272027f441ccda01622a5
2022-05-20 11:56:22 +02:00
Vitaliy Ivanov 51b4f5502f ndpireader: add json output back. (#1508)
- partial revert of:
    commit 51cfdfb0d8
    Author: Luca Deri <deri@ntop.org>
    Date: Sun Nov 17 17:51:45 2019 +0100
      Removed unused JSON-C code

- Json option is changed from 'j' to 'k' as it's used in the new codebase.
- use HAVE_LIBJSON_C instead of HAVE_JSON_C.
- tabs vs spaces clean ups.

Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com>
2022-04-01 11:46:15 +02:00
Luca Deri 258fedbca0 Fixed invalid ndpi_serialize_risk() prototype 2022-02-10 20:22:17 +01:00
Luca Deri 3b1c2d8793 Fixes skype (mis)detection 2022-02-09 11:41:10 +01:00
Luca 8b5c6af71b Updated release to 4.2 4.2 2022-02-01 09:14:05 +01:00
Alfredo Cardigliano 03cb8efad6 Update changelog 2022-01-31 18:38:02 +01:00
Ivan Nardi 5bb5bec477 Remove struct ndpi_id_struct (#1427)
Remove the last uses of `struct ndpi_id_struct`.
That code is not really used and it has not been updated for a very long
time: see #1279 for details.

Correlation among flows is achieved via LRU caches.

This change allows to further reduce memory consumption (see also
91bb77a8).

At nDPI 4.0 (more precisly, at a6b10cf, because memory stats
were wrong until that commit):
```
nDPI Memory statistics:
	nDPI Memory (once):      221.15 KB
	Flow Memory (per flow):  2.94 KB
```
Now:
```
nDPI Memory statistics:
	nDPI Memory (once):      235.27 KB
	Flow Memory (per flow):  688 B        <--------
```
i.e. memory usage per flow has been reduced by 77%.

Close #1279
2022-01-30 19:18:12 +01:00
Ivan Nardi 1fdcddb33d Remove Playstation VUE protocol (#1426)
PS VUE service has been discontinued on January 30, 2020
https://en.wikipedia.org/wiki/PlayStation_Vue
2022-01-30 19:17:53 +01:00
Luca Deri 5f745ee425 Commented old code (see https://github.com/ntop/nDPI/pull/1425) 2022-01-30 19:17:23 +01:00
Ivan Nardi f455421870 Improve protocol stacks (#1425)
We should have two protocols in classification results only when the
"master" protocol allows some sub-protocols.

Classifications like `AmazonAWS`, `TLS/AmazonAWS`, `DNS/AmazonAWS` are
fine. However classifications like `NTP/Apple`, `BitTorrent/Azure`,
`DNScrypt.AmazonAWS` or `NestLogSink.Google` are misleading.
For example, `ndpiReader`shows  `BitTorrent/Azure` flows under `Azure`
statistics; that seems to be wrong or, at least, very misleading.

This is quite important since we have lots of addresses from CDN
operators.

The only drawback of this solution is that right now ICMP traffic is
classified simply as `ICMP`; if we are really interested in ICMP stuff
we can restore the old behaviour later.
2022-01-30 19:15:42 +01:00
Ivan Nardi 513e386959 Extend protocols support (#1422)
Add detection of AccuWeather site/app and Google Classroom.
Improve detection of Azure, Zattoo, Whatsapp, MQTT and LDAP.

Fix some RX false positives.

Fix some "Uncommon TLS ALPN"-risk false positives.

Fix "confidence" value for some Zoom/Torrent classifications.

Minor fix in Lua script for Wireshark extcap.

Update .gitignore file.

Let GitHub correctly detect the language type of *.inc files.

Zattoo example has been provided by @subhajit-cdot in #1148.
2022-01-29 09:19:26 +01:00
Toni 9b8679a320 Fix some race conditions by using atomic operations. (#1420)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-29 09:18:51 +01:00
Ivan Nardi 0c70411b1b Make some protocols more "big-endian" friendly (#1402)
See #1312
2022-01-29 09:18:32 +01:00
Ivan Nardi 86b97ffb73 Sync unit tests results (#1423)
Fix: 7a3aa41a
2022-01-28 17:17:39 +01:00
Luca 7a3aa41afc Updated alert description case 2022-01-28 15:02:42 +01:00
Toni c53c82d482 Fixed heap overflow in nDPI realloc wrapper if new size < old size. (#1421)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-27 15:29:16 +01:00
Ivan Nardi 9dcc4a2334 Kerberos, TLS, example: fix some memory errors (#1419)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43823
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43921
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43925
2022-01-27 10:59:35 +01:00
Toni 011ee3ecbd Fixed wrong ip tuple comparison. #1386 (#1418)
* Added u32 pads to `union ip_tuple` so btree search should now work as expected.
   The bug caused new flow's when the remote answers, resulting in two Flows per direction. Fail.
 * Fixed a race condition during shutdown phase.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-26 15:47:38 +01:00
Luca Deri 6d185c052e Added support for the .goog Google TLD 2022-01-26 15:22:33 +01:00
Toni 2cd0479204 Serializing empty `risk blocks' pollutes the resulting string. (#1417)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-26 13:36:45 +01:00
Luca Deri 81b33f0d06 Updated test results after the risk NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE has been added 2022-01-26 09:46:46 +01:00
Luca Deri 58a9aff17c Added NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE flow risk
Added ndpi_set_tls_cert_expire_days() API call to modify the number of days for triggering the above alert that by default is set to 30 days
2022-01-26 09:23:23 +01:00
Luca Deri 14e8808f03 Removed pandora.tv from pandora protocol as they are different services that shouldd not be mixed 2022-01-25 19:11:16 +01:00
Luca Deri bbb3b8349f Improved pandora TV detection 2022-01-24 18:28:43 +01:00
Luca Deri b6b4967aa6 Improved Zoom protocol detection 2022-01-23 20:59:36 +01:00
Luca Deri a8fe74e502 Tool for generating automatically the Azure IP list 2022-01-23 18:56:51 +01:00
Alfredo Cardigliano 20b9f01f12 Fix ndpi_serialize_string_int64 2022-01-21 17:35:38 +01:00
Alfredo Cardigliano a0bb0ab0ca Add unit test for ndpi_serialize_string_int64 2022-01-21 13:05:58 +01:00
havsah e16b6a18be Fix Grease values parsing (#1416)
The check for grease was too broad and filtered some valid values.
In particular, the value 257 was skipped because it matched the previous check.
This has been discovered while parsing tests/pcap/443-firefox.pcap
expected ja3:
771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-51-57-47-53-10,0-23-65281-10-11-35-16-5-51-43-13-45-28-21,29-23-24-25-256-257,0
previously generated ja3:
771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-51-57-47-53-10,0-23-65281-10-11-35-16-5-51-43-13-45-28-21,29-23-24-25-256,0

Signed-off-by: Patrick Havelange <patrick.havelange_ext@softathome.com>
2022-01-21 12:40:35 +01:00
Luca Deri 6f336f8307 Added JA3 in risj exceptions 2022-01-20 22:22:56 +01:00
Luca Deri d91477b18d Fixed certificate mismatch check 2022-01-19 23:10:10 +01:00
Luca Deri 771b6606c8 Fixed an issue on CentOS 7 introduced by
https://github.com/ntop/nDPI/commit/c2b7d77784beeae5f86f19e33ba3da5ddce55898
2022-01-19 10:19:58 +01:00
Luca Deri 9eee257be0 Updated confidence type 2022-01-18 22:28:58 +01:00
Ivan Nardi 9f050fa0a6 TLS, H323, examples: fix some memory errors (#1414)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=26880
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=26906
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43782
https://oss-fuzz.com/testcase-detail/6334089358082048
2022-01-18 21:52:37 +01:00
Ivan Nardi bd036f96f9 Netbios, CSGO: fix two memory errors (#1413)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43754
https://oss-fuzz.com/testcase-detail/5329842395021312
2022-01-18 19:11:01 +01:00
Sam JamesandLuca Deri c2b7d77784 build: respect environment options more (#1392)
* build: update m4/ax_pthread.m4 from serial 23 -> serial 31

Update ax_pthread.m4 to the latest version from the autoconf-archive
project.

Signed-off-by: Sam James <sam@gentoo.org>

* build: properly detect AR, CC, RANLIB

It's necessary to be able to override choice of AR/CC/RANLIB and other toolchain
variables/tools for cross-compilation, testing with other toolchains, and
to ensure the compiler chosen by the user is actually used for the build.

Previously, GNU_PREFIX was kind-of used for this but this isn't a standard
variable (at all) and it wasn't applied consistently anyway.

We now use the standard autoconf mechanisms for finding these tools.

(RANLIB is already covered by LT_INIT.)

Signed-off-by: Sam James <sam@gentoo.org>

* build: use $(MAKE)

This ensures that parallel make works correctly, as otherwise, a fresh
make job will be started without the jobserver fd, and hence
not know about its parent, forcing -j1.

* build: respect CPPFLAGS, LDFLAGS

- CPPFLAGS is for the C preprocessor (usually for setting defines)
- LDFLAGS should be placed before objects for certain flags to work
  (e.g. -Wl,--as-needed)

Signed-off-by: Sam James <sam@gentoo.org>

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2022-01-18 14:30:14 +01:00
Ivan Nardi e4a7990a4e H323: fix a use-after-poison error (#1412)
Detected by oss-fuzz
See: https://oss-fuzz.com/testcase-detail/6730505580576768

Fix a function prototype
Update a unit test results
2022-01-17 19:47:27 +01:00
Luca Deri d19bd96555 Improved Badoo detection (missing mobile app domain) 2022-01-17 18:54:27 +01:00
Luca Deri 6877e46273 Added Badoo detection 2022-01-17 15:13:14 +01:00
Luca 6679453d86 Adds some risk exceptions for popular services and domain names
via a new (internal) function named ndpi_add_domain_risk_exceptions()
2022-01-17 09:00:25 +01:00
Ivan Nardi 1ff308af09 TLS: fix a use-of-uninitialized-value error (#1411)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43705
2022-01-16 20:31:48 +01:00
Ivan Nardi 05734f3d31 Zattoo: fix Null-dereference READ with ipv6 traffic (#1410)
Fix: 20b5f6d7
Detected by oss-fux:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43700
2022-01-16 19:28:39 +01:00
Ivan Nardi 611c753da5 XBox, Diameter: fix dissectors initialization (#1405)
These dissectors have *never* been triggered because their registration
functions use the wrong parameter/bitmask.
Diameter code is buggy since the origianl commit (1d108234), while
XBox code since 5266c726.

Fix some false positives in Xbox code.
2022-01-16 16:19:00 +01:00
Luca Deri f3af39ee42 Added performance tests tools 2022-01-16 12:47:56 +01:00
Luca Deri 42d74171b2 Minor cosmetic changes 2022-01-16 12:47:56 +01:00
Luca Deri dd325248f1 Reduced Patricia tree bucket memory footprint 2022-01-16 12:47:56 +01:00
Ivan Nardi e46b508a36 Kerberos: fix use-of-uninitialized-value error (#1409)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43677
2022-01-15 20:53:36 +01:00
Ivan Nardi eb5d7b07af TLS: fix heap-buffer-overflow error (#1408)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43664
2022-01-15 18:47:59 +01:00
Ivan Nardi 63c44a24b6 STUN: fix "confidence" value for some classifications (#1407) 2022-01-15 17:35:06 +01:00