Commit Graph
1164 Commits
Author SHA1 Message Date
Ivan Nardi 6c1accd2bd DTLS: fix access to certificate cache (#1450)
```
protocols/tls.c:650:54: runtime error: member access within null pointer of type 'const struct ndpi_tcphdr'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in
protocols/tls.c:650:54: runtime error: load of null pointer of type 'const u_int16_t' (aka 'const unsigned short')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in
AddressSanitizer:DEADLYSIGNAL
=================================================================
==47401==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x55f7a61b661c bp 0x7f38190f91b0 sp 0x7f38190f70e0 T1)
==47401==The signal is caused by a READ memory access.
==47401==Hint: address points to the zero page.
    #0 0x55f7a61b661c in processCertificateElements /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:650:41
    #1 0x55f7a61ac3cc in processCertificate /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:792:7
    #2 0x55f7a61d34e1 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:846:13
```
2022-02-21 20:30:22 +01:00
Vitaly Lavrov f2411958d7 Added lightweight implementation of libgcrypt. (#1444)
Implementation borrowed from the
https://github.com/ARMmbed/mbedtls.git project (v3.1.0)

Speed testing (Xeon(R) CPU E3-1230 V2 @ 3.30GHz):

gcrypt-gnu         Test md   2897 ms enc   2777 ms dec    942 ms
gcrypt-int         Test md   3668 ms enc   1312 ms dec   2836 ms
gcrypt-int-noaesni Test md   3652 ms enc   1916 ms dec   4458 ms

gcrypt-gnu-nonopt  Test md   3763 ms enc   4978 ms dec   3999 ms

gcrypt-gnu-nonopt - libgcrypt compiled without hardware acceleration
  --disable-padlock-support --disable-aesni-support \
  --disable-shaext-support --disable-pclmul-support \
  --disable-sse41-support --disable-drng-support \
  --disable-avx-support --disable-avx2-support \
  --disable-neon-support --disable-arm-crypto-support \
  --disable-ppc-crypto-support
  --disable-amd64-as-feature-detection
2022-02-20 10:16:46 +01:00
Ivan Nardi f28a3b293e Fix compilation and sync unit tests results (#1445)
'strcasestr' is not defined in all enviroments: quicker fix is to use
'ndpi_strncasestr' instead.
2022-02-19 19:18:02 +01:00
Luca Deri a2878af1ee Added newflow risk NDPI_HTTP_CRAWLER_BOT 2022-02-17 17:20:52 +01:00
Luca Deri c4113ffd7e Added SNMP error code check 2022-02-14 23:17:51 +01:00
Ivan Nardi ada7d32c50 HSRP: fix dissection over IPv6 (#1443)
Handle all message types.
2022-02-10 11:46:32 +01:00
Ivan Nardi b5eea436ae HSRP: add support for IPv6 (#1440) 2022-02-09 11:47:37 +01:00
Dmytrii Vitman 4cf853548c Added VXLAN dissector (#1439)
* RFC 7348
2022-02-09 11:47:09 +01:00
Luca Deri be5a29ba9d Added HSRP protocol detection
Removed attic directory now obsolete
2022-02-08 18:04:57 +01:00
Luca Deri 922a7be897 Added check to ignore multicast packets marking the as Skype 2022-02-08 17:23:25 +01:00
Toni d59f0327a5 Improved MDNS/LLMNR detection. (#1437)
* Checking for port 5353/5355 is not enough.
 * Added additional multicast address and header checks.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-02-07 18:05:23 +01:00
Ivan Nardi ff38d4b980 TLS: fix parsing of certificate elements (#1435)
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44280
```
==263603==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x592478 in ndpi_is_printable_string ndpi/src/lib/ndpi_utils.c:2200:9
    #1 0x5b047c in processCertificateElements ndpi/src/lib/protocols/tls.c:400:7
    #2 0x5ac880 in processCertificate ndpi/src/lib/protocols/tls.c:790:7
    #3 0x5c3a32 in processTLSBlock ndpi/src/lib/protocols/tls.c:844:13
    #4 0x5c2c61 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:973:2
    #5 0x5c117d in ndpi_search_tls_wrapper ndpi/src/lib/protocols/tls.c:2367:5
    #6 0x552a50 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4792:6
```
2022-02-07 18:05:12 +01:00
Luca Deri 7aef27f85e Added NDPI_ERROR_CODE_DETECTED risk 2022-02-03 13:20:54 +01:00
Luca Deri 0dd5ac89f8 Renamed DCERPC to more generic RPC protocol so we can use also for other types of RPCs (not limited to DCE)
Extended HTTP plugin to support RPC
Improved HTTP crear text detection to limit it to Basic and Digest
2022-02-03 13:06:27 +01:00
Ivan Nardi 5bb5bec477 Remove struct ndpi_id_struct (#1427)
Remove the last uses of `struct ndpi_id_struct`.
That code is not really used and it has not been updated for a very long
time: see #1279 for details.

Correlation among flows is achieved via LRU caches.

This change allows to further reduce memory consumption (see also
91bb77a8).

At nDPI 4.0 (more precisly, at a6b10cf, because memory stats
were wrong until that commit):
```
nDPI Memory statistics:
	nDPI Memory (once):      221.15 KB
	Flow Memory (per flow):  2.94 KB
```
Now:
```
nDPI Memory statistics:
	nDPI Memory (once):      235.27 KB
	Flow Memory (per flow):  688 B        <--------
```
i.e. memory usage per flow has been reduced by 77%.

Close #1279
2022-01-30 19:18:12 +01:00
Ivan Nardi 513e386959 Extend protocols support (#1422)
Add detection of AccuWeather site/app and Google Classroom.
Improve detection of Azure, Zattoo, Whatsapp, MQTT and LDAP.

Fix some RX false positives.

Fix some "Uncommon TLS ALPN"-risk false positives.

Fix "confidence" value for some Zoom/Torrent classifications.

Minor fix in Lua script for Wireshark extcap.

Update .gitignore file.

Let GitHub correctly detect the language type of *.inc files.

Zattoo example has been provided by @subhajit-cdot in #1148.
2022-01-29 09:19:26 +01:00
Ivan Nardi 0c70411b1b Make some protocols more "big-endian" friendly (#1402)
See #1312
2022-01-29 09:18:32 +01:00
Ivan Nardi 9dcc4a2334 Kerberos, TLS, example: fix some memory errors (#1419)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43823
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43921
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43925
2022-01-27 10:59:35 +01:00
Luca Deri 58a9aff17c Added NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE flow risk
Added ndpi_set_tls_cert_expire_days() API call to modify the number of days for triggering the above alert that by default is set to 30 days
2022-01-26 09:23:23 +01:00
Luca Deri b6b4967aa6 Improved Zoom protocol detection 2022-01-23 20:59:36 +01:00
havsah e16b6a18be Fix Grease values parsing (#1416)
The check for grease was too broad and filtered some valid values.
In particular, the value 257 was skipped because it matched the previous check.
This has been discovered while parsing tests/pcap/443-firefox.pcap
expected ja3:
771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-51-57-47-53-10,0-23-65281-10-11-35-16-5-51-43-13-45-28-21,29-23-24-25-256-257,0
previously generated ja3:
771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-51-57-47-53-10,0-23-65281-10-11-35-16-5-51-43-13-45-28-21,29-23-24-25-256,0

Signed-off-by: Patrick Havelange <patrick.havelange_ext@softathome.com>
2022-01-21 12:40:35 +01:00
Luca Deri d91477b18d Fixed certificate mismatch check 2022-01-19 23:10:10 +01:00
Ivan Nardi 9f050fa0a6 TLS, H323, examples: fix some memory errors (#1414)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=26880
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=26906
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43782
https://oss-fuzz.com/testcase-detail/6334089358082048
2022-01-18 21:52:37 +01:00
Ivan Nardi bd036f96f9 Netbios, CSGO: fix two memory errors (#1413)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43754
https://oss-fuzz.com/testcase-detail/5329842395021312
2022-01-18 19:11:01 +01:00
Ivan Nardi e4a7990a4e H323: fix a use-after-poison error (#1412)
Detected by oss-fuzz
See: https://oss-fuzz.com/testcase-detail/6730505580576768

Fix a function prototype
Update a unit test results
2022-01-17 19:47:27 +01:00
Ivan Nardi 1ff308af09 TLS: fix a use-of-uninitialized-value error (#1411)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43705
2022-01-16 20:31:48 +01:00
Ivan Nardi 05734f3d31 Zattoo: fix Null-dereference READ with ipv6 traffic (#1410)
Fix: 20b5f6d7
Detected by oss-fux:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43700
2022-01-16 19:28:39 +01:00
Ivan Nardi 611c753da5 XBox, Diameter: fix dissectors initialization (#1405)
These dissectors have *never* been triggered because their registration
functions use the wrong parameter/bitmask.
Diameter code is buggy since the origianl commit (1d108234), while
XBox code since 5266c726.

Fix some false positives in Xbox code.
2022-01-16 16:19:00 +01:00
Ivan Nardi e46b508a36 Kerberos: fix use-of-uninitialized-value error (#1409)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43677
2022-01-15 20:53:36 +01:00
Ivan Nardi eb5d7b07af TLS: fix heap-buffer-overflow error (#1408)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43664
2022-01-15 18:47:59 +01:00
Ivan Nardi 63c44a24b6 STUN: fix "confidence" value for some classifications (#1407) 2022-01-15 17:35:06 +01:00
Ivan Nardi 20b5f6d7cc Improve IPv6 support, enabling IPv6 traffic on (almost) all dissectors. (#1406)
Follow-up of 7cba34a1
2022-01-15 17:34:56 +01:00
Luca Deri 406ac7e8c8 Added the ability to specify trusted issueDN often used in companies to self-signed certificates
This allows to avoid triggering alerts for trusted albeit private certificate issuers.

Extended the example/protos.txt with the new syntax for specifying trusted issueDN.
Example:

trusted_issuer_dn:"CN=813845657003339838, O=Code42, OU=TEST, ST=MN, C=US"
2022-01-13 19:06:21 +01:00
Luca Deri 1e1cfb89d2 Added EthernetIP dissector 2022-01-12 21:48:39 +01:00
Ivan Nardi b080a1c136 Fix two use-of-uninitialized-value errors (#1398)
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=40269
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=41432

Fix fuzz compilation (follow-up of f5545a80)
2022-01-12 20:24:57 +01:00
Ivan Nardi 06f3b07909 QUIC: fix an integer overflow (#1396)
Reported by oss-fuzz:
```
==685288==ERROR: AddressSanitizer: SEGV on unknown address 0x61a100000687 (pc 0x0000005aba64 bp 0x7ffe3f29f510 sp 0x7ffe3f29f400 T0)
==685288==The signal is caused by a READ memory access.
SCARINESS: 20 (wild-addr-read)
 #0 0x5aba64 in quic_len ndpi/src/lib/protocols/quic.c:203:12
 #1 0x5aba64 in decrypt_initial_packet ndpi/src/lib/protocols/quic.c:993:16
 #2 0x5aba64 in get_clear_payload ndpi/src/lib/protocols/quic.c:1302:21
 #3 0x5aba64 in ndpi_search_quic ndpi/src/lib/protocols/quic.c:1658:19
 #4 0x579f00 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4683:6
 #5 0x57abe6 in ndpi_check_flow_func ndpi/src/lib/ndpi_main.c:0
 #6 0x583b2c in ndpi_detection_process_packet ndpi/src/lib/ndpi_main.c:5545:15
 #7 0x55e75e in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_process_packet.c:30:3
[...]
```
2022-01-11 17:57:34 +01:00
Ivan Nardi 3a087e951d Add a "confidence" field about the reliability of the classification. (#1395)
As a general rule, the higher the confidence value, the higher the
"reliability/precision" of the classification.

In other words, this new field provides an hint about "how" the flow
classification has been obtained.
For example, the application may want to ignore classification "by-port"
(they are not real DPI classifications, after all) or give a second
glance at flows classified via LRU caches (because of false positives).

Setting only one value for the confidence field is a bit tricky: more
work is probably needed in the next future to tweak/fix/improve the logic.
2022-01-11 15:23:39 +01:00
Luca Deri 708d4ea33a Improved user agent analysis 2022-01-09 18:47:47 +01:00
Ivan Nardi bf69321a29 GTP: fix some false positives (#1394) 2022-01-08 20:40:24 +01:00
Ivan Nardi 4d27001d1d Remove some unused fields (#1393) 2022-01-08 20:40:10 +01:00
Luca Deri f24f68f59d Invalid check 2022-01-06 07:15:45 +01:00
Alfredo Cardigliano 23a4761276 Update copyright 2022-01-03 11:00:45 +01:00
Luca Deri c4ac53a03f Added support for Log4J/Log4Shell detection in nDPI via a new flow risk named NDPI_POSSIBLE_EXPLOIT 2021-12-23 21:30:16 +01:00
Ivan Nardi 91bb77a880 A final(?) effort to reduce memory usage per flow (#1389)
Remove some unused fields and re-organize other ones.
In particular:
* Update the parameters of `ndpi_ssl_version2str()` function
* Zattoo, Thunder: these timestamps aren't really used.
* Ftp/mail: these protocols are dissected only over TCP.
* Attention must be paid to TLS.Bittorrent flows to avoid invalid
read/write to `flow->protos.bittorrent.hash` field.

This is the last(?) commit of a long series (see 22241a1d, 227e586e,
730c2360, a8ffcd8b) aiming to reduce library memory consumption.

Before, at nDPI 4.0 (more precisly, at a6b10cf7, because memory stats
were wrong until that commit):
```
nDPI Memory statistics:
	nDPI Memory (once):      221.15 KB
	Flow Memory (per flow):  2.94 KB
```
Now:
```
nDPI Memory statistics:
	nDPI Memory (once):      231.71 KB
	Flow Memory (per flow):  1008 B       <---------
```
i.e. memory usage per flow has been reduced by 66%, dropping below the
psychological threshold of 1 KB.

To further reduce this value, we probably need to look into #1279:
let's fight this battle another day.
2021-12-22 19:54:06 +01:00
Luca Deri 4c02c81bf4 Improved bittorrent heuristic 2021-12-21 23:04:27 +01:00
Ivan Nardi 7153b8933c Improve/add several protocols (#1383)
Improve Microsoft, GMail, Likee, Whatsapp, DisneyPlus and Tiktok
detection.
Add Vimeo, Fuze, Alibaba and Firebase Crashlytics detection.

Try to differentiate between Messenger/Signal standard flows (i.e chat)
and their VOIP (video)calls (like we already do for Whatsapp and
Snapchat).

Add a partial list of some ADS/Tracking stuff.

Fix Cassandra, Radius and GTP false positives.
Fix DNS, Syslog and SIP false negatives.

Improve GTP (sub)classification: differentiate among GTP-U, GTP_C and
GTP_PRIME.

Fix 3 LGTM warnings.
2021-12-18 13:24:51 +01:00
Ivan Nardi 58b33dcb2d TLS: add support for IPV6 in Subject Alt Names field (#1385) 2021-12-18 12:03:46 +01:00
Ivan Nardi ec6eea0c23 TLS: fix usage of certificate cache (#1384) 2021-12-18 12:03:32 +01:00
Luca Deri 74519f6e10 Improved bittorrent detection 2021-12-17 21:17:18 +01:00
Ivan Nardi f11b215cfb QUIC: fix old versions of GQUIC on big-endian machines (#1387)
Credits to @viniciussn (see #1312)
2021-12-17 12:10:34 +01:00