Commit Graph
343 Commits
Author SHA1 Message Date
Ivan Nardi 8c7071e040 DTLS: handle (certificate) fragments (#1811)
Keep using the existing function to handle reassembling buffer: rename
it from `ndpi_search_tls_tcp_memory` to
`ndpi_search_tls_memory` and make it "transport" agnostic
2022-12-10 18:32:25 +01:00
Ivan Nardi 81e1ea545c Make LRU caches ipv6 aware (#1810)
Simplest solution, keeping the existing cache data structure

TLS certificate cache is used for DTLS traffic, too.

Note that Ookla cache already works with ipv6 flows.

TODO:
 * make the key/hashing more robust (extending the key size?)
 * update bittorrent cache too. That task is quite difficult because
   ntopng uses a public function (`ndpi_guess_undetected_protocol()`)
   intrinsically ipv4 only...
2022-12-03 12:07:32 +01:00
Toni b5e8d86def Improved Teamspeak3 License/Weblist detection. (#1802)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-11-15 08:59:45 +01:00
Toni c521595383 Add Elasticsearch protocol dissector. (#1782)
* all credits goes to @verzulli

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-21 20:01:54 +02:00
Toni Uhlig 29242cbcb6 Add Munin protocol dissector.
* all credits goes to @verzulli

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-20 22:10:35 +02:00
Ivan Nardi 35fdbc8148 TLS: explicit ignore client certificate (#1776)
TLS classification usually stops after processing *server* certificates
(if any). That means, that *client* certificate, if present, is usually
ignored.
However in some corner cases (i.e. unidirectional traffic) we might end
up processing client certificate and exposing its metadata: the issue is
that the application will think that this metadata are about the server
and not about the client.
So, for the time being, always ignore client certificate processing.

As a future work, we might find an efficient way to process and export both
certificates.
2022-10-18 16:40:15 +02:00
Ivan Nardi 1796a1d814 LINE_CALL: add detection of LINE voip calls (#1761)
These flows are classifed as `LINE_CALL`; another option was
`RTP/LINE_CALL`. No sure about the best solution...

Extend LINE domains list.

Remove RTP dead code.
2022-10-06 17:09:26 +02:00
sharonenochandSharon Enoch 503aac70bc Line app support (#1759)
* Standard support for LINE app

* Added test pcap for LINE app

* make check result for LINE app

* Make check success as 1kxun has LINE packets

* Added the ASN inc file for LINE

* Removed extra lines as its effecting make check

* Editing the SNI required a new pcap output file for TLS.Line format

* Run Configure with --with-pcre --with-maxminddb to enable the generation of h323-overflow.pcap.out

Co-authored-by: Sharon Enoch <sharone@amzetta.com>
2022-10-01 12:01:41 +02:00
Nardi Ivan c6201284d7 NAT-PMP: fix metadata extraction 2022-09-21 20:02:23 +02:00
Toni Uhlig a966d37a21 Add CryNetwork dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-16 16:48:43 +02:00
Toni Uhlig ec1981c20c Add Syncthing dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-16 15:21:49 +02:00
Toni de020b1747 Add NATPMP dissector. (#1738)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-16 13:32:30 +02:00
Ivan Nardi b4f5243716 Anydesk: improve detection (#1735) 2022-09-13 20:55:11 +02:00
Nardi Ivan 678dd61866 STUN: several improvements
Add detection over TCP and fix detection over IPv6.
Rename some variables since Stun dissector is no more "udp-centric".
Stun dissector should always classified the flow as `STUN` or
`STUN/Something`.
Don't touch `flow->guessed_host_protocol_id` field, which should be
always be related to "ip-classification" only.
2022-09-11 13:33:32 +02:00
Nardi Ivan 1b3de449a1 HTTP: correctly set the classification protocols
Classification should always be set via `ndpi_set_detected_protocol()`
to be sure to set a correct `confidence` value, too.
Having a "known" protocol stack with `NDPI_CONFIDENCE_UNKNOWN` as
confidence, is not valid.

This code in HTTP dissector likely needs some more thoughts (the
classification itself of the attached example doesn't make a lot of
sense), but the goal of this commit is only to always have a valid
`confidence` value.
2022-09-06 14:58:31 +02:00
Nardi Ivan b9cb391756 Add support to opportunistic TLS
A lot of protocols provide the feature to upgrade their plain text
connections to an encrypted one, via some kind of "STARTTLS" command.

Add generic code to support this extension, and allow dissection of the
entire TLS handshake.

As examples, SMTP, POP, IMAP and FTP dissectors have been updated.

Since this feature requires to process more packets per flow, add the
possibility to disable it.

Fix some log messages.

Slight improvement on TCP sequence number tracking.

As a side effect, this commit fix also a memory leak found by
oss-fuzzer
```
==108966==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 22 byte(s) in 1 object(s) allocated from:
    #0 0x55f8b367a0be in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x5480be) (BuildId: 94debacb4a6784c30420ab748c8bf3cc59621063)
    #1 0x55f8b36e1345 in ndpi_malloc_wrapper /home/ivan/svnrepos/nDPI/example/reader_util.c:321:10
    #2 0x55f8b379c7d2 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:212:25
    #3 0x55f8b379cb18 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:279:13
    #4 0x55f8b386ce46 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2153:34
    #5 0x55f8b385ebf7 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:867:5
    #6 0x55f8b39e708c in ndpi_extra_search_mail_smtp_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/mail_smtp.c:422:9
    #7 0x55f8b37e636c in ndpi_process_extra_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5884:9
    #8 0x55f8b37edc05 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6276:5
    #9 0x55f8b3701ffc in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1619:31
    #10 0x55f8b36faf14 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2189:10
    #11 0x55f8b36b6a50 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7

```
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50765
2022-09-04 17:22:19 +02:00
Toni fe28d55801 Improved MGCP dissector. (#1717)
* typ0s fixed
 * dissect endpoint hostnames

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-30 11:21:58 +02:00
Nardi Ivan 8bfb1712d8 QUIC: add support for 0-RTT packets received before the Initial
RFC9001 4.6.1: "A client that wishes to send 0-RTT packets uses the
early_data extension in the ClientHello message of a subsequent handshake;
see Section 4.2.10 of [TLS13]. It then sends application data in 0-RTT
packets."
That means the client sends before the CH (in the Initial) and then the
0-RTT (in the same UDP datagram or not)".

However, because of packet loss or out-of-order delivery, it might
happens that a 0-RTT packet is received before the Initial (the original
one or a retransmission).
For example, Google and Facebook servers save 0-RTT packets for a small
amount of time in hopes of receiving the corresponding Initial.

Update the QUIC dissector to detect 0-RTT packets and keep looking for
the Initial.

Issue found by @utoni in #1706; the trace example has been taken from that
PR.
2022-08-24 15:38:30 +02:00
Toni 0c8bc9f055 Add FastCGI protocol detection. (#1711)
* CQL: fixed byte order conversion (BigEndian not LittleEndian)
 * CQL: increased required successful dissected packets to prevent false-positives

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 13:22:46 +02:00
Toni 30730e95e5 Add Kismet protocol detection. (#1710)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 10:50:11 +02:00
Toni 5915bc8828 Improved Discord pattern. (#1707)
Signed-off-by: lns <matzeton@googlemail.com>

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-10 21:00:11 +02:00
Toni 2e25c36396 Add TiVoConnect dissector. Fixes #1697. (#1699)
* added static assert if supported, to complain if the flow struct changes

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-08 19:04:20 +02:00
Toni ac24b35b1f Add Discord dissector. (#1694)
* fixed RiotGames false positive

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-03 12:03:36 +02:00
Toni 26aafd931c Add Activision dissector. (#1693)
Signed-off-by: lns <matzeton@googlemail.com>
2022-08-02 16:15:07 +02:00
Ivan Nardi 2a596c79e6 HTTP: fix classification (#1692)
If we have a valid HTTP sessions, we should ignore
`flow->guessed_protocol_id` field (i.e. classification "by-port")
altogheter.

The attached trace was classified as "SIP/HTTP" only because the *client*
port was 5060...
As a general rule, having a classification such as "XXXX/HTTP" is
*extremely* suspicious.
2022-07-30 22:57:20 +02:00
Ivan Nardi d54d5083b3 SMTPS, POPS, IMAPS: fix classification and extra dissection (#1685)
The big change in TLS code is to allow "master" protocols other than
TLS/DTLS, like SMTPS, POPS and IMAPS.
This change will allow, in a future, a proper and complete TLS dissection
for all these protocols with "STARTTLS"-like messages.
2022-07-30 12:05:43 +02:00
Toni ed4f106a0d Add Softether dissector. (#1679)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-29 19:29:54 +02:00
Toni ab3a678ad4 Add AVAST dissector. (#1674)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-25 18:07:44 +02:00
Ivan Nardi b190dab6bc Improve handling of HTTP-Proxy and HTTP-Connect (#1673)
Treat HTTP-Proxy and HTTP-Connect flows like the HTTP ones:
print/serialize all the attributes and allow parsing of replies.

The line about "1kxun" has been removed to avoid regressions in 1KXUN
classification in `tests/pcap/1kxun.pcap`. I haven't fully understod
what was happening but the comment at the beginning of `static
ndpi_category_match category_match[]` says that we can't have overlaps
between `host_match` and `category_match` lists and that is no longer true
since 938e89ca.
Bottom line: removing this line seems the right thing to do, anyway.
2022-07-25 12:57:33 +02:00
Toni a25b2a7e37 Added AliCloud server access dissector. (#1672)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-23 11:21:49 +02:00
Ivan Nardi 52005e88ed TLS: improve reassembler (#1669)
* TLS: cosmetic changes

* TLS: improve reassembler

We might need to contemporary re-order messages from both directions:
use one buffer per direction.
2022-07-22 12:19:21 +02:00
Nardi Ivan d66aa49787 DTLS: fix exclusion of DTLS protocol
Add an helper to exclude a generic protocol
2022-07-20 19:16:18 +02:00
Toni ae2bedce3a Improved Jabber/XMPP detection. (#1661)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-13 17:55:33 +02:00
Ivan Nardi df599e5eff HTTP: improve detection of WindowsUpdate (#1658)
WindowsUpdate is also transported over HTTP, using a numeric IP as
hostname (some kinds of CDN?)
2022-07-10 17:08:37 +02:00
Ivan Nardi 997dce0f04 SIP: improve detection (#1654) 2022-07-09 05:45:42 +02:00
Ivan Nardi f8076e3a58 SMB: add (partial) support for messages split into multiple TCP segments (#1644) 2022-07-07 19:24:31 +02:00
Ivan Nardi feaa1df1ed Kerberos: add support for Krb-Error messages (#1647) 2022-07-07 16:45:49 +02:00
Nardi Ivan 2636c07571 MONGODB: avoid false positives 2022-07-07 15:36:05 +02:00
Nardi Ivan a31e79fc3c TLS: ignore invalid Content Type values 2022-07-07 15:36:05 +02:00
Toni 15042870f9 Added Threema Messenger. (#1643)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 19:30:10 +02:00
Toni Uhlig a1c3d05a74 Added another RiotGames signature.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 14:37:26 +02:00
Toni 4ff8aa48b2 Added UltraSurf protocol dissector. (#1618)
* TLSv1.3 UltraSurf flows are not detected by now

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 16:04:53 +02:00
Ivan Nardi 4445989588 Update host content list match (#1633)
Improve classifications of Outlook, Cachefly, Cloudflare, Tiktok and
Cybersecurity.
2022-07-04 13:21:11 +02:00
Toni 75f7da5c26 Added Psiphon detection patterns. See #566 and #1099. (#1631)
* The traces are not up to date, but this is the best we got so far.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 10:34:54 +02:00
Toni a74fc089c4 Added i3D and RiotGames protocol dissectors. (#1609)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 20:43:30 +02:00
Ivan Nardi faaff58620 TargusDataspeed: avoid false positives (#1628)
TargusDataspeed dissector doesn't perform any real DPI checks but it only
looks at the TCP/UDP ports.
Delete it, and use standard logic to classify these flows by port.
2022-07-03 20:28:58 +02:00
Ivan Nardi 422d002542 Skinny: rework and improve classification (#1625) 2022-07-03 19:25:00 +02:00
Ivan Nardi 5fe6087686 TLS: add support for old DTLS versions and for detection of mid-sessions (#1619) 2022-07-03 17:44:17 +02:00
Toni 1a01e8dc68 Improved TFTP. Dissect Read/Write Request filenames. (#1617)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 14:37:05 +02:00
Toni 7c5c811eb0 Added Cloudflare WARP detection patterns. (#1615) (#1616)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-02 14:57:56 +02:00