These protocols:
* have been addeded in the OpenDPI era
* have never been updated since then
* we don't have any pcap examples [*]
If (and it is a big if...) some of these protocols are still somehow
used and if someone is still interested in them, we can probably
re-add them starting from scratch (because the current detection
rules are probably outdated)
Protocols removed: DIRECT_DOWNLOAD_LINK, APPLEJUICE, DIRECTCONNECT,
OPENFT, FASTTRACK, SHOUTCAST, THUNDER, AYIYA, STEALTHNET, FIESTA,
FLORENSIA, AIMINI, SOPCAST
PPSTREAM dissector works (...) only on UDP.
[*]: with do have an AIMINI test pcap but it was some trivial http
traffic detected only by hostname matching, on domains no more
available...
Tuya IoTOS Embedded Wi-Fi and BLE SDK for bk7231n. Used by many "smart"
devices such as LED light strips, bulbs, etc.
Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
Keep using the existing function to handle reassembling buffer: rename
it from `ndpi_search_tls_tcp_memory` to
`ndpi_search_tls_memory` and make it "transport" agnostic
Simplest solution, keeping the existing cache data structure
TLS certificate cache is used for DTLS traffic, too.
Note that Ookla cache already works with ipv6 flows.
TODO:
* make the key/hashing more robust (extending the key size?)
* update bittorrent cache too. That task is quite difficult because
ntopng uses a public function (`ndpi_guess_undetected_protocol()`)
intrinsically ipv4 only...
nDPI is able to properly classify QUIC flows only if it elaborates the
very first packets of the flow.
The protocol list in `is_udp_guessable_protocol()` is basically a list
of protocols which can be detected from *any* packets in the flow.
Rename such function to `is_udp_not_guessable_protocol()`: the name is
still quite cryptic, but at least not plainly wrong
We already performed exactly these lookups in the generic code to
populate `flow->guessed_protocol_id_by_ip`: use it!
This code probably needs a deeper review, since it is basicaly a simple
matching on ip + port.
Tell "Advertised" ALPN list from "Negotiated" ALPN; the former is
extracted from the CH, the latter from the SH.
Add some entries to the known ALPN list.
Fix printing of "TLS Supported Versions" field.
In some rare cases, it is possible to sub-classify the flow via ALPN
matching. This is particularly usefull for asymmetric traffic where the
Client Hello doens't have the SNI.
For the time being there is only one rule, about ANYDESK.
TLS classification usually stops after processing *server* certificates
(if any). That means, that *client* certificate, if present, is usually
ignored.
However in some corner cases (i.e. unidirectional traffic) we might end
up processing client certificate and exposing its metadata: the issue is
that the application will think that this metadata are about the server
and not about the client.
So, for the time being, always ignore client certificate processing.
As a future work, we might find an efficient way to process and export both
certificates.
These flows are classifed as `LINE_CALL`; another option was
`RTP/LINE_CALL`. No sure about the best solution...
Extend LINE domains list.
Remove RTP dead code.
* Standard support for LINE app
* Added test pcap for LINE app
* make check result for LINE app
* Make check success as 1kxun has LINE packets
* Added the ASN inc file for LINE
* Removed extra lines as its effecting make check
* Editing the SNI required a new pcap output file for TLS.Line format
* Run Configure with --with-pcre --with-maxminddb to enable the generation of h323-overflow.pcap.out
Co-authored-by: Sharon Enoch <sharone@amzetta.com>
0 as size value disable the cache.
The diffs in unit tests are due to the fact that some lookups are
performed before the first insert: before this change these lookups
weren't counted because the cache was not yet initialized, now they are.
(see exaple/protos.txt)
nbpf:"host 192.168.1.1 and port 80"@HomeRouter
In order to have nBPF support, you need to compile nDPI with it. Just download
https://github.com/ntop/PF_RING in the same directory where you have downloaded
nDPI and compile PF_RING/userland/nbpf
Basically:
* "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is
NEVER returned in the protocol stack (i.e.
`flow->detected_protocol_stack[]`);
* if the application is interested into such information, it can access
`ndpi_protocol->protocol_by_ip` itself.
There are mainly 4 points in the code that set the "classification
by-ip" in the protocol stack: the generic `ndpi_set_detected_protocol()`/
`ndpi_detection_giveup()` functions and the HTTP/STUN dissectors.
In the unit tests output, a print about `ndpi_protocol->protocol_by_ip`
has been added for each flow: the huge diff of this commit is mainly due
to that.
Strictly speaking, this change is NOT an API/ABI breakage, but there are
important differences in the classification results. For examples:
* TLS flows without the initial handshake (or without a matching
SNI/certificate) are simply classified as `TLS`;
* similar for HTTP or QUIC flows;
* DNS flows without a matching request domain are simply classified as
`DNS`; we don't have `DNS/Google` anymore just because the server is
8.8.8.8 (that was an outrageous behaviour...);
* flows previusoly classified only "by-ip" are now classified as
`NDPI_PROTOCOL_UNKNOWN`.
See #1425 for other examples of why adding the "classification by-ip" in
the protocol stack is a bad idea.
Please, note that IPV6 is not supported :( (long standing issue in nDPI) i.e.
`ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for
IPv6 flows.
Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed.
Close#1687