Commit Graph
3056 Commits
Author SHA1 Message Date
Toni 39a5d234fd Serialize amount of found TLS blocks and signature algos. (#2384)
* partially fixes #2282

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-11 16:21:41 +02:00
Luca Deri 8b4fdbc0fb Renamed ndpi_fill_ip6_protocol_category in ndpi_fill_ipv6_protocol_category 2024-04-11 12:37:39 +02:00
Vladimir Gavrilov 9ff4bece33 Add The Elder Scrolls Online support (#2376)
* Add The Elder Scrolls Online support

* Use ndpi_memmem instead of memmem from libc

* Add protocol description

* Change selection bitmask to V4_V6

* Update protocols.rst
2024-04-10 18:04:02 +02:00
Vladimir Gavrilov 142c8f5afb Add memmem() implementation (#2378)
* Add memmem() implementation

* Fix build

* Add fix to avoid too many memcmp calls
2024-04-10 15:39:08 +02:00
Toni 1d0be6c4f4 Add Shellscript risk detection. (#2375)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-10 15:36:54 +02:00
Ivan Nardi 54517d8e04 Fix a warning and restore a unit test result (#2379) 2024-04-10 09:56:06 +02:00
Luca Deri bd2968dcd3 Tuned DNS risk values
Modified NDPI_BINARY_TRANSFER_ATTEMPT in NDPI_BINARY_DATA_TRANSFER
2024-04-09 10:22:01 +02:00
Luca Deri 00a360f30a Disabled "known proto on non standard port" for FTP_DATA 2024-04-09 00:01:47 +02:00
Luca Deri 98bf0e243e Fixed false positives on binary application transfer risk 2024-04-08 23:52:27 +02:00
Ivan Nardi 1b3ef7d7b2 STUN: improve extraction of Mapped-Address metadata (#2370)
Enable parsing of Mapped-Address attribute for all STUN flows: that
means that STUN classification might require more packets.

Add a configuration knob to enable/disable this feature.

Note that we can have (any) STUN metadata also for flows *not*
classified as STUN (because of DTLS).

Add support for ipv6.

Restore the correct extra dissection logic for Telegram flows.
2024-04-08 10:24:51 +02:00
Ivan Nardi f5905a62c7 Fix invalid memory access (#2374)
The bug is triggered when `pe_offset == (u_int32_t)-1`

```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==23719==ERROR: AddressSanitizer: SEGV on unknown address 0x5081000002b3 (pc 0x55c69274ac72 bp 0x7ffffffc8e70 sp 0x7ffffffc8cc0 T0)
==23719==The signal is caused by a READ memory access.
    #0 0x55c69274ac72 in ndpi_search_portable_executable /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8191:7
    #1 0x55c69271606b in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8596:5
    #2 0x55c69270f58f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8629:22
    #3 0x55c6926a07e7 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:24:5
    #4 0x55c6925a79b6 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x64e9b6) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
    #5 0x55c692590d48 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x637d48) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
    #6 0x55c69259685a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x63d85a) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
    #7 0x55c6925c0e02 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x667e02) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
    #8 0x7f8e99793082 in __libc_start_main /build/glibc-wuryBv/glibc-2.31/csu/../csu/libc-start.c:308:16
    #9 0x55c69258baed in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x632aed) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
```

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67881
2024-04-06 20:23:06 +02:00
Toni a5d45253c4 Add ELF risk detection (detect transmitted linux executables). (#2373)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-06 19:32:51 +02:00
Toni 727e72d1f1 Calculate packet entropy for unknown protocols. (#2369)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-06 17:01:19 +02:00
Vladimir Gavrilov b535033674 Get rid of Apache Kafka false positives (#2372) 2024-04-06 16:59:38 +02:00
Toni 0f77f49b77 Add PE32/PE32+ risk detection (detect transmitted windows executables). (#2312)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-05 14:43:28 +02:00
Toni 09bb383437 Fixed yet another another "unused function" warning. (#2367)
* added `-Wextra` to the CI

```
In file included from ndpi_bitmap64_fuse.c:31:
./third_party/include/binaryfusefilter.h:31:24: error: unused function 'binary_fuse_rotl64' [-Werror,-Wunused-function]
static inline uint64_t binary_fuse_rotl64(uint64_t n, unsigned int c) {

..snip..
```

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 20:41:45 +02:00
RoboSchmied ead9aa72da Fix: 3 typos (#2366)
Signed-off-by: RoboSchmied <github@roboschmie.de>
2024-04-04 20:06:47 +02:00
Toni df29e12f5e Fixed yet another "unused function" warning. (#2365)
* seems like clang on `ubuntu-latest` warns about unused static inlined functions

```
In file included from ndpi_bitmap.c:41:
./third_party/include/roaring.h:422:19: error: unused function 'roaring_leading_zeroes' [-Werror,-Wunused-function]
static inline int roaring_leading_zeroes(unsigned long long input_num) {

..snip..
```

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 18:18:44 +02:00
Ivan Nardi c0d3f8a92e STUN: rework sub-classification (#2361)
The main goal is to have the "real" application (if any; i.e.
Signal/Whatsapp/Telegram/...) always as "application" protocol and not
as "master" one
2024-04-04 18:16:40 +02:00
Toni 6e61368cd6 Add -Wno-unused-function as CFLAG for third_party/src/gcrypt_light.c. (#2364)
* fixes failing nDPI build from an external project with clang and `-Wextra`

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 13:51:27 +02:00
Luca Deri 9185c2ccc4 Added support for STUN Mapped IP address 2024-04-03 23:03:46 +02:00
Luca 225ff7f8a4 Added binary data transfer risk alert 2024-04-03 15:41:26 +02:00
Ivan Nardi 1acc9ee396 Patrica tree: fix handling of mac addresses (#2362)
Found looking at the fuzzer statistics...
2024-04-03 14:10:37 +02:00
Toni 41eef9246c Disable -Wno-unused-parameter -Wno-unused-function. (#2358)
* unused parameters and functions pollute the code and decrease readability

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-03 14:10:21 +02:00
Vladimir Gavrilov 599cc0f4b8 Improve LoL: Wild Rift detection (#2359) 2024-03-28 16:01:01 +01:00
Vladimir Gavrilov 5b32c98a21 Add LoL: Wild Rift detection (#2356) 2024-03-26 08:11:14 +01:00
Luca Deri 51f5fc7140 Added support for roaring bitmap v3 (#2355)
* Integrated RoaringBitmap v3

* Renamed ndpi_bitmap64 ro ndpi_bitmap64_fuse

* Fixes to ndpi_bitmap for new roaring library

* Fixes for bitmap serialization

* Fixed format

* Warning fix

* Conversion fix

* Warning fix

* Added check for roaring v3 support

* Updated file name

* Updated path

* Uses clang-9 (instead of clang-7) for builds

* Fixed fuzz_ds_bitmap64_fuse

* Fixes nDPI printf handling

* Disabled printf

* Yet another printf fix

* Cleaup

* Fx for compiling on older platforms

* Fixes for old compilers

* Initialization changes

* Added compiler check

* Fixes for old compilers

* Inline function is not static inline

* Added missing include
2024-03-25 08:15:19 +01:00
Vladimir Gavrilov be0fd7444f Add more TencentGames signatures (#2354) 2024-03-21 14:38:47 +01:00
Vladimir Gavrilov 27f9ca9d04 Improve TencentGames detection (#2353)
* Improve TencentGames detection

* Add more signatures
2024-03-20 12:13:32 +01:00
Nardi Ivan 15a80527c6 STUN: remove workaround to identify RTP traffic
We are able to demultiplex RTP packets in STUN flows since 3608ab01b, at
least; no need to explicity call the RTP dissector
2024-03-20 09:39:15 +01:00
Nardi Ivan 5ccf41739b STUN: fix fallback when DTL dissection fails 2024-03-20 09:39:15 +01:00
Nardi Ivan f758f3605a STUN: dissection of TURN Channel Data 2024-03-20 09:39:15 +01:00
Nardi Ivan ee87c4bc66 STUN: add dissection of XOR-PEER-ADDRESS with ipv6 address 2024-03-20 09:39:15 +01:00
Nardi Ivan 58894dd788 STUN: improve heurstic to detect old classic-stun 2024-03-20 09:39:15 +01:00
Ivan Nardi dabf29ea61 STUN: update demultiplexing logic according to RFC9443 (#2349) 2024-03-19 09:11:51 +01:00
Vladimir Gavrilov e6474d835f Add FLUTE protocol dissector (#2351)
* Add FLUTE protocol dissector

* Add flute.c to MSVC project
2024-03-19 09:11:04 +01:00
Ivan Nardi 231748bb0e LRU cache: move to 64 bits long keys (#2346)
Tradeoff between key comparison efficiency (i.e. no `memcmp`) and key
length.
At least in the ipv4 cases, we have no more different entries with the
same key.
2024-03-15 10:12:51 +01:00
Ivan Nardi 97fae6e00a STUN: fix parsing of DATA attribute (#2345) 2024-03-14 14:20:51 +01:00
Ivan Nardi 37ddf02e63 bitmap64: fix memory access error (#2344)
```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==29723==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x562910b70ddb bp 0x7ffcb22c5b70 sp 0x7ffcb22c5a80 T0)
==29723==The signal is caused by a READ memory access.
==29723==Hint: address points to the zero page.
    #0 0x562910b70ddb in binary_fuse16_contain /home/ivan/svnrepos/nDPI/src/lib/./third_party/include/binaryfusefilter.h:492:8
    #1 0x562910b70bbe in ndpi_bitmap64_isset /home/ivan/svnrepos/nDPI/src/lib/ndpi_bitmap64.c:178:10
    #2 0x562910788fd3 in ndpi_domain_classify_longest_prefix /home/ivan/svnrepos/nDPI/src/lib/ndpi_domain_classify.c:261:5
    #3 0x56291078940e in ndpi_domain_classify_contains /home/ivan/svnrepos/nDPI/src/lib/ndpi_domain_classify.c:291:9
    #4 0x56291069a392 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ds_domain_classify.cpp:52:5
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67369
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67372
2024-03-14 14:20:35 +01:00
Vladimir Gavrilov 8fad77991d Add PFCP protocol dissector (#2342) 2024-03-13 20:18:43 +01:00
Luca Deri 1aedbefffa Fixes bitmap memory calculation 2024-03-11 17:59:24 +01:00
Ivan Nardi 6152d595e8 STUN: add a parameter to configure how long the extra dissection lasts (#2336)
Tradeoff: performance (i.e. number of packets) vs sub-classification
2024-03-07 14:39:32 +01:00
Ivan Nardi 56ce228a8b Add a specific protocol id for audio/video calls made using Google apps (#2341)
Same logic already used for Signal/Whatsapp/Line/Facebook/...
2024-03-07 13:48:19 +01:00
Ivan Nardi c09b4dacdf Fix memory leak (#2340)
```
Direct leak of 12 byte(s) in 1 object(s) allocated from:
    #0 0x55779e1a46ff in malloc (/home/ivan/svnrepos/nDPI/example/ndpiReader+0x8706ff) (BuildId: 14c2fc626744710d49d652ea1c5bbb24a8cbab4f)
    #1 0x55779e2120c7 in ndpi_malloc_wrapper /home/ivan/svnrepos/nDPI/example/ndpiReader.c:298:10
    #2 0x55779e5fa215 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
    #3 0x55779e5fa500 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13
    #4 0x55779e42153c in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2554:46
    #5 0x55779e4359a1 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:908:5
    #6 0x55779e432de7 in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1097:2
    #7 0x55779e4133f9 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2913:5
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67250
2024-03-07 13:11:13 +01:00
Vladimir Gavrilov e2949048e0 Add Path of Exile protocol dissector (#2337)
* Add Path of Exile protocol dissector

* Update protocols.rst
2024-03-06 19:59:09 +01:00
Ivan Nardi 21da53d3a0 ahocorasick: improve matching with subdomains (#2331)
The basic idea is to have the following logic:
* pattern "DOMAIN" matches the domain itself (i.e exact match) *and* any
subdomains (i.e. "ANYTHING.DOMAIN")
* pattern "DOMAIN." matches *also* any strings for which is a prefix
[please, note that this kind of match is handy but it is quite
dangerous...]
* pattern "-DOMAIN" matches *also* any strings for which is a postfix

Examples:
* pattern "wikipedia.it":
  * "wikipiedia.it" -> OK
  * "foo.wikipedia.it -> OK
  * "foowikipedia.it -> NO MATCH
  * "wikipedia.it.com -> NO MATCH
* pattern "wikipedia.":
  * "wikipedia.it" -> OK
  * "foo.wikipedia.it -> OK
  * "foowikipedia.it -> NO MATCH
  * "wikipedia.it.com -> OK
* pattern "-wikipedia.it":
  * "wikipedia.it" -> NO MATCH
  * "foo.wikipedia.it -> NO MATCH
  * "0001-wikipedia.it -> OK
  * "foo.0001-wikipedia.it -> OK

Bottom line:
* exact match
* prefix with "." (always, implicit)
* prefix with "-" (only if esplicitly set)
* postfix with "." (only if esplicitly set)

That means that the patterns cannot start with '.' anymore.

Close #2330
2024-03-06 19:25:59 +01:00
Luca Deri 8f63a11735 Extended connectivity checks 2024-03-05 17:45:46 +01:00
Luca Deri 65b9ad88a9 Improved alert on suspicious DNS traffic 2024-03-05 11:14:34 +01:00
Vladimir Gavrilov e7f1946ae7 Add NetEase Games detection support (#2335) 2024-03-05 11:00:52 +01:00
Vladimir Gavrilov 58fdc9fafb Add Naraka Bladepoint detection support (#2334) 2024-03-04 08:30:54 +01:00