Commit Graph
4679 Commits
Author SHA1 Message Date
Toni e9dc035c5c Added optimized memmem/strlcpy version (#2424)
* credits goes to Vladimir Gavrilov

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-08 11:38:53 +02:00
Luca Deri 2b4e2f9c9a Removed trace 2024-05-07 23:42:32 +02:00
Luca Deri 5f8ad0527d Reworked code and added two new API calls
- int ndpi_load_ipv4_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);
- int ndpi_load_ipv6_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);
2024-05-07 23:05:31 +02:00
Luca Deri 3d2b611ea6 ndpi_mod is now optional (albeit better to spcify it) in ndpi_domain_classify_xxx calls 2024-05-07 13:09:00 +02:00
Luca Deri 0d495294bf Win warning fixes 2024-05-07 10:04:11 +02:00
Luca Deri 6195dbad76 Renamed radius source file to avoid name clashes on Windows 2024-05-07 09:05:15 +02:00
Ivan Nardi 1773d7ff2d Add support for Mastodon, Bluesky and (FB-)Threads (#2418) 2024-05-06 13:37:18 +02:00
Ivan Nardi 95fe21015d Remove "zoom" cache (#2420)
This cache was added in b6b4967aa, when there was no real Zoom support.
With 63f349319, a proper identification of multimedia stream has been
added, making this cache quite useless: any improvements on Zoom
classification should be properly done in Zoom dissector.

Tested for some months with a few 10Gbits links of residential traffic: the
cache pretty much never returned a valid hit.
2024-05-06 12:51:45 +02:00
Ivan Nardi b7d77972bc Remove workaround for TCP flows with multiple SYNs (#2421)
Deciding when a session starts and ends is responsability of the
applicationi (via its flow manager)i, not of the library.
BTW, the removed code is incomplete at beast
2024-05-06 10:20:40 +02:00
Ivan Nardi e31ef00715 TLS: avoid setting NDPI_TLS_SELFSIGNED_CERTIFICATE for webrtc traffic (#2417)
See RFC8122: it is quite likely that STUN/DTLS/SRTP flows use
self-signed certificates

Follow-up of b287d6ec8
2024-05-06 10:20:07 +02:00
Ivan Nardi 266af02752 Merge RTP and RTCP logic (#2416)
Avoid code duplication between these two protocols.

We remove support for RTCP over TCP; it is quite rare to find this kind
of traffic and, more important, we have never had support for RTP
over TCP: we should try to add both detecion as follow-up.

Fix a message log in the LINE code
2024-05-06 10:19:46 +02:00
Ivan Nardi a1845da359 TLS: fix Ja4 fingerprint computation (#2419)
The new values has been checked against the ones reported by Wireshark.

Found while fixing a Use-of-uninitialized-value error reported by
oss-fuzz

```
==7582==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x5a6549abc368 in ndpi_compute_ja4 ndpi/src/lib/protocols/tls.c:1762:10
    #1 0x5a6549ab88a0 in processClientServerHello ndpi/src/lib/protocols/tls.c:2863:10
    #2 0x5a6549ac1452 in processTLSBlock ndpi/src/lib/protocols/tls.c:909:5
    #3 0x5a6549abf588 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:1098:2
    #4 0x5a65499c53ec in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:7215:6
```

See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68449&q=ndpi&can=1&sort=-id
2024-05-05 17:27:29 +02:00
Ivan Nardi a6fd981fcf eDonkey: improve/update classification (#2410)
eDonkey is definitely not as used as >10 years ago, but it seems it is
still active.

While having a basic TCP support seems easy, identification over UDP doesn't
work and it is hard to do it rightly (packets might be only 2 bytes long):
remove it.

Credits to V.G <v.gavrilov@securitycode.ru>
2024-05-04 19:11:31 +02:00
Luca Deri 57ecbf38c0 Updated JA4 test results 2024-05-02 17:40:24 +02:00
Luca Deri 66133d1501 Fixes JA4 computation adding a better GREASE detect funzion 2024-05-02 16:58:44 +02:00
Luca Deri af955169e1 typo 2024-05-02 16:03:30 +02:00
Ivan Nardi 9e549ba781 Workaround for fixing GitHub runners on macOS (#2411)
GitHub switched "macos-latest" from "macos-12" to "macos-14", which is only
on ARM64!
https://github.com/actions/runner/issues/3256
https://github.blog/changelog/2024-01-30-github-actions-macos-14-sonoma-is-now-available/

However we are having some issues build nDPI on macos-14 with external
libraries:

```
configure: error: libgpg-error required (because of --with-local-libgcrypt) but not found or too old.
```
See: https://github.com/ntop/nDPI/actions/runs/8869020568/job/24350356867

```
ndpi_utils.c:69:10: fatal error: 'pcre2.h' file not found
         ^~~~~~~~~
1 error generated.
```
See: https://github.com/ntop/nDPI/actions/runs/8869020568/job/24349242251

Everything is still fine with macos-14 and no external dependencies

As workaround, test only macos-12 and macos-13 in our main matrix.
2024-04-28 22:17:39 +02:00
Ivan Nardi d5bda47efe DTLS: add support for Alert message type (similar to TLS) (#2406) 2024-04-25 11:23:05 +02:00
0x41CEA55 7040847eed Add Adobe Connect support (#2407) 2024-04-24 12:41:25 +02:00
0x41CEA55 66036a14f4 Remove PPStream protocol and add iQIYI (#2403)
P2P video player PPStream was discontinued shortly after the purchase of PPS.tv by Baidu (iQIYI) on 2013 (see https://www.techinasia.com/report-baidu-acquires-video-rival-pps)
So we remove the old `NDPI_PROTOCOL_PPSTREAM` logic and add `NDPI_PROTOCOL_IQIYI` id to handle all the iQIYI traffic, which is basically video streaming traffic.

A video hosting service, called PPS.tv, is still offered by the same company: for the time being we classified both services with the same protocol id.
2024-04-23 18:01:36 +02:00
0x41CEA55 fd388845d5 Add BFCP protocol support (#2401) 2024-04-23 15:35:19 +02:00
Ivan Nardi abc7e430e2 STUN: slightly faster sub-classification with DTLS (#2404) 2024-04-23 09:31:48 +02:00
Ivan Nardi a62679952c fuzz: fix shoco fuzzer (#2405)
```
==22779==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f0900701020 at pc 0x555bcd2a6f02 bp 0x7ffe3ba5e790 sp 0x7ffe3ba5e788
READ of size 1 at 0x7f0900701020 thread T0
    #0 0x555bcd2a6f01 in shoco_decompress /home/ivan/svnrepos/nDPI/src/lib/third_party/src/shoco.c:184:26
    #1 0x555bcd2a4018 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco.cpp:18:3
    #2 0x555bcd1aa816 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x4f816) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)
    #3 0x555bcd193be8 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x38be8) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)
    #4 0x555bcd1996fa in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x3e6fa) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)
    #5 0x555bcd1c3c92 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x68c92) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)
    #6 0x7f090257a082 in __libc_start_main /build/glibc-e2p3jK/glibc-2.31/csu/../csu/libc-start.c:308:16
    #7 0x555bcd18e96d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x3396d) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)

Address 0x7f0900701020 is located in stack of thread T0 at offset 4128 in frame
    #0 0x555bcd2a3d97 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco.cpp:5

  This frame has 4 object(s):
    [32, 4128) 'out' (line 9) <== Memory access at offset 4128 overflows this variable
    [4256, 8352) 'orig' (line 9)
```

Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68211
2024-04-22 18:39:23 +02:00
Nardi Ivan f494bdd653 Fix parameters check
```
==17==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000546050 bp 0x7fff113c82a0 sp 0x7fff113c7a58 T0)
==17==The signal is caused by a READ memory access.
==17==Hint: address points to the zero page.
SCARINESS: 10 (null-deref)
    #0 0x546050 in __sanitizer::internal_strlen(char const*) /src/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_libc.cpp:167:10
    #1 0x4c6ba5 in __interceptor_strrchr /src/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:740:5
    #2 0x5fb9b9 in ndpi_get_host_domain_suffix /src/ndpi/src/lib/ndpi_domains.c:105:20
    #3 0x578058 in LLVMFuzzerTestOneInput /src/ndpi/fuzz/fuzz_config.cpp:503:3
```

Found while fuzzing
2024-04-21 12:29:09 +02:00
0x41CEA55 246e972f9c Replace my personal email with my corporate one in all my contributions (#2399) 2024-04-20 21:37:01 +02:00
Nardi Ivan 95f4d0b852 Fix a warning
```
nDPI/PcDebug64/src/include/ndpi_api.h:1970:3: error: function declaration isn’t a prototype [-Werror=strict-prototypes]
```
2024-04-20 19:30:42 +02:00
Ivan Nardi ef89183469 fuzz: improvements (#2400)
Create the zip file with all the traces only once.

Add a new fuzzer to test "shoco" compression algorithm
2024-04-20 18:15:23 +02:00
0x41CEA55 905120588b Remove obsolete protocols: tuenty, tvuplayer and kontiki (#2398) 2024-04-19 21:35:32 +02:00
0x41CEA55 1b2e2cd968 Add strlcpy implementation (#2395) 2024-04-19 17:16:40 +02:00
0x41CEA55 e75d7a620e Add KNXnet/IP protocol support (#2397)
* Add KNXnet/IP protocol support

* Improve KNXnet/IP over TCP detection
2024-04-19 12:54:00 +02:00
Luca Deri ad117bfaab Domain Classification Improvements (#2396)
* Added
size_t ndpi_compress_str(const char * in, size_t len, char * out, size_t bufsize);
size_t ndpi_decompress_str(const char * in, size_t len, char * out, size_t bufsize);

used to compress short strings such as domain names. This code is based on
https://github.com/Ed-von-Schleck/shoco

* Major code rewrite for ndpi_hash and ndpi_domain_classify

* Improvements to make sure custom categories are loaded and enabled

* Fixed string encoding

* Extended SalesForce/Cloudflare domains list
2024-04-18 23:21:40 +02:00
Luca Deri 108b8331d5 Invalid initialization 2024-04-15 18:13:17 +02:00
Luca Deri 95897c6436 Fixed minor glitches 2024-04-15 14:25:26 +02:00
Ivan Nardi d97afd6efd STUN: fix attributes list iteration (#2391)
We need to check all the attributes, to look for any possible metadata
2024-04-13 18:43:41 +02:00
Ivan Nardi 06f4814085 STUN: try to stop extra dissection earlier, if possible (#2390) 2024-04-13 14:45:17 +02:00
Ivan Nardi 8edb2f133c STUN: add support for ipv6 in some metadata (#2389) 2024-04-13 14:12:20 +02:00
Ivan Nardi f51dfc6973 STUN: simplify ip/port parsing (#2388)
Add other 2 configuration options
2024-04-13 12:00:04 +02:00
Ivan Nardi 0535e54484 STUN: fix boundary checks on attribute list parsing (#2387)
Restore all unit tests.
Add some configuration knobs.
Fix the endianess.
2024-04-12 22:55:51 +02:00
Luca Deri bb4fccd2cf Added comment 2024-04-12 20:29:14 +02:00
Luca Deri b83eb7c7a2 Implemented STUN peer_address, relayed_address, response_origin, other_address parsing
Added code to ignore invalid STUN realm
Extended JSON output with STUN information
2024-04-12 19:50:04 +02:00
Toni Uhlig 31de94ba12 fix invalid read
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-12 17:55:48 +02:00
Vladimir Gavrilov c63697205b Add Label Distribution Protocol support (#2385)
* Add Label Distribution Protocol support

* Fix typo

* Update unit test results
2024-04-12 17:44:36 +02:00
Ivan Nardi 93f02ee507 Fix ndpi_reconcile_msteams_udp (#2377)
Microsoft UDP traffic over port ~3478 is voip traffic, using some kind
of proprietary STUN-like protocol: so use the most specific protocol id.

More important, we definitely want `Stun/Skype_TeamsCall` and not
`Stun/Skype_Teams`
2024-04-12 17:05:28 +02:00
Toni Uhlig 65fef09ff2 Updated unit test results
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-12 16:52:00 +02:00
Toni Uhlig 082b00ede7 Updated unit test results
* fixed invalid read

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-12 16:38:44 +02:00
Luca Deri a8507c3308 STUN:
- Fixed issue with XOR-MAPPED-ADDRESS decoding
- Implemented MAPPED_ADDRESS (IPv4 only)
2024-04-12 15:49:42 +02:00
Nardi Ivan 17965c8852 Update unit tests results 2024-04-11 18:36:43 +02:00
Luca Deri fd2838e9a1 Minor code cleanup 2024-04-11 17:44:44 +02:00
Toni 39a5d234fd Serialize amount of found TLS blocks and signature algos. (#2384)
* partially fixes #2282

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-11 16:21:41 +02:00
Luca Deri 8b4fdbc0fb Renamed ndpi_fill_ip6_protocol_category in ndpi_fill_ipv6_protocol_category 2024-04-11 12:37:39 +02:00