ubuntu-latest s390x CI doesn't like snapshot bigger than 262144 bytes.
Fix an error found by fuzz CI
```
=================================================================
==55399==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x561e41cb684d bp 0x7ffd54ce3650 sp 0x7ffd54ce3520 T0)
==55399==The signal is caused by a READ memory access.
==55399==Hint: address points to the zero page.
#0 0x561e41cb684d in ndpi_network_ptree_match /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2321:41
#1 0x561e41d30879 in ndpi_guess_undetected_protocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7767:8
#2 0x561e41ca804d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_config.cpp:104:5
#3 0x561e41bb96a0 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x4726a0) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
#4 0x561e41ba3c2f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x45cc2f) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
#5 0x561e41ba96f6 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x4626f6) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
#6 0x561e41bd22e2 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x48b2e2) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
#7 0x7f94f0e5c082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
#8 0x561e41b9eb0d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x457b0d) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
```
Check the mining cache at the end of the function, like all the others
LRU caches.
Rewrite the STUN checks: same identical semantic but in a cleaner way,
hopefully
Classification "by-port" should be the last possible effort, *after*
having test all the LRU caches.
Remove some dead code from `ndpi_detection_giveup()`:
`flow->guessed_protocol_id` is never set to any od those voip protocols
and at that point in this function we never have both a master *and* a
application protocols.
Coverage reports (both from unit tests and from fuzzing) confirms that
was dead code.
No traces and no flows has been removed; only long sessions
has been reduced, keeping only their first packets.
This is quite important in fuzzing systems, since these pcaps are used
as initial seed. There is no value in fuzzing long sessions, because
only the very first packets are really used/processed by nDPI.
Before:
```
du -h tests/pcap/
200M tests/pcap/
```
After:
```
du -h tests/pcap/
98M tests/pcap/
```
These protocols:
* have been addeded in the OpenDPI era
* have never been updated since then
* we don't have any pcap examples [*]
If (and it is a big if...) some of these protocols are still somehow
used and if someone is still interested in them, we can probably
re-add them starting from scratch (because the current detection
rules are probably outdated)
Protocols removed: DIRECT_DOWNLOAD_LINK, APPLEJUICE, DIRECTCONNECT,
OPENFT, FASTTRACK, SHOUTCAST, THUNDER, AYIYA, STEALTHNET, FIESTA,
FLORENSIA, AIMINI, SOPCAST
PPSTREAM dissector works (...) only on UDP.
[*]: with do have an AIMINI test pcap but it was some trivial http
traffic detected only by hostname matching, on domains no more
available...
Tuya IoTOS Embedded Wi-Fi and BLE SDK for bk7231n. Used by many "smart"
devices such as LED light strips, bulbs, etc.
Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
Keep using the existing function to handle reassembling buffer: rename
it from `ndpi_search_tls_tcp_memory` to
`ndpi_search_tls_memory` and make it "transport" agnostic
Simplest solution, keeping the existing cache data structure
TLS certificate cache is used for DTLS traffic, too.
Note that Ookla cache already works with ipv6 flows.
TODO:
* make the key/hashing more robust (extending the key size?)
* update bittorrent cache too. That task is quite difficult because
ntopng uses a public function (`ndpi_guess_undetected_protocol()`)
intrinsically ipv4 only...
nDPI is able to properly classify QUIC flows only if it elaborates the
very first packets of the flow.
The protocol list in `is_udp_guessable_protocol()` is basically a list
of protocols which can be detected from *any* packets in the flow.
Rename such function to `is_udp_not_guessable_protocol()`: the name is
still quite cryptic, but at least not plainly wrong
We already performed exactly these lookups in the generic code to
populate `flow->guessed_protocol_id_by_ip`: use it!
This code probably needs a deeper review, since it is basicaly a simple
matching on ip + port.
Tell "Advertised" ALPN list from "Negotiated" ALPN; the former is
extracted from the CH, the latter from the SH.
Add some entries to the known ALPN list.
Fix printing of "TLS Supported Versions" field.
In some rare cases, it is possible to sub-classify the flow via ALPN
matching. This is particularly usefull for asymmetric traffic where the
Client Hello doens't have the SNI.
For the time being there is only one rule, about ANYDESK.
TLS classification usually stops after processing *server* certificates
(if any). That means, that *client* certificate, if present, is usually
ignored.
However in some corner cases (i.e. unidirectional traffic) we might end
up processing client certificate and exposing its metadata: the issue is
that the application will think that this metadata are about the server
and not about the client.
So, for the time being, always ignore client certificate processing.
As a future work, we might find an efficient way to process and export both
certificates.