Commit Graph
1443 Commits
Author SHA1 Message Date
Ivan Nardi 31a9da238c STUN: add dissection of DTLS handshake (#2018) 2023-06-26 12:07:26 +02:00
Ivan Nardi 88425e0199 Simplify the report of streaming multimedia info (#2026)
The two fields `flow->flow_type` and `flow->protos.rtp.stream_type` are
pretty much identical: rename the former in `flow->flow_multimedia_type`
and remove the latter.
2023-06-26 12:05:16 +02:00
Ivan Nardi 3a1600ff26 Thrift: fix heap-buffer-overflow (#2024) 2023-06-26 11:41:01 +02:00
Ivan Nardi 7e64d9f66d RTP: rework code (#2021)
Try avoiding false positives: look for 3 RTP packets before classifing
the flow as such.

Add a generic function `is_rtp_or_rtcp()` to identify RTP/RTCP packets also
in other dissectors (see 3608ab01b commit message for an example)
2023-06-23 08:12:40 +02:00
Ivan KapranovandIvan Kapranov 15f0e1480c minor fixes (#2023)
fixed 'handle leak' in ndpi_load_malicious_sha1_file and removed the redundant comparison ndpi_search_eaq

Co-authored-by: Ivan Kapranov <i.kapranov@securitycode.ru>
2023-06-22 21:36:40 +02:00
Ivan Kapranov aa25c06a18 removed useless call of ndpi_set_risk func (#2022) 2023-06-22 18:26:33 +02:00
Luca Deri cdc0bb8c61 Compilation fix 2023-06-22 16:04:14 +02:00
Toni 1678888284 Add Apache Thrift protocol dissector. (#2007)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-06-22 13:07:32 +02:00
Luca Deri 8dbd591053 Waring fix 2023-06-21 21:26:29 +02:00
Ivan Nardi cc582052b3 Gnutella: improve detection (#2019) 2023-06-21 14:25:34 +02:00
Nardi Ivan 570c75d601 STUN: fix detection over TCP
TCP framing is optional
2023-06-21 10:38:44 +02:00
Nardi Ivan 2ac240ce6a STUN: improve WhatsappCall detection 2023-06-21 10:38:44 +02:00
Ivan Nardi 46aab35a29 Line: fix heap-buffer-overflow error (#2015) 2023-06-21 09:22:07 +02:00
Ivan Nardi 3608ab01b6 STUN: keep monitoring/processing STUN flows (#2012)
Look for RTP packets in the STUN sessions.
TODO: tell RTP from RTCP
2023-06-21 09:16:20 +02:00
Luca Deri 9cc4cbb9d1 Reworked teams handling 2023-06-15 22:31:11 +02:00
Luca Deri d0609ea601 Implemented Zoom/Teams stream type detection 2023-06-14 23:44:57 +02:00
Luca Deri 66bee475ae Improved line protocol dissection with heuristic 2023-06-13 19:20:02 +02:00
Luca Deri b02adcc381 Added check to avoid skype heuristic false positives 2023-06-12 23:29:20 +02:00
Nardi Ivan 7150b40c63 QUIC: fix dissection of packets forcing VN 2023-06-08 17:07:25 +02:00
Nardi Ivan 3e640a13a4 QUIC: fix a memory access error
Found while fuzzing
2023-06-08 17:07:25 +02:00
Nardi Ivan fd4cb10190 QUIC: add support for QUIC version 2
See: https://www.rfc-editor.org/rfc/rfc9369.txt

Old v2-01 version has been removed, since it has never been really used.
2023-06-08 17:07:25 +02:00
headshog 7dcceb4daf Numeric truncation at tls.c:1010 (#2005)
* fixed numtrunc error in protocols/tls.c

* fixed build error for tls.c
2023-06-08 07:52:18 +02:00
Maatuq e17fa1259a Add bitcoing protocol dissector. (#1992)
* Add bitcoing protocol dissector.

* remove bitcoin protcol detection from mining.c
* add a new bitcoin deissector.
* add a new category: Cryptocurrency.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* Remove useless checks and add missing windows and docs file.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* update affected tests.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* add a brief version.

Add notes on the difference between normal bitcoin protocol and the
mining protocol.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* update enable_payload_stat test after dev rebasing.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

---------

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
2023-05-31 07:31:01 +02:00
Ivan Nardi 7ce14da0c8 Remove some useless checks (#1993)
In the main dissector callbacks the flow protocols are (almost) always
unknown. Only two exceptions:
* extra dissection data path
* HTTP sub-protocols
2023-05-28 12:59:18 +02:00
Ivan Nardi b11e6a453b Add support for Epic Games and GeForceNow/Nvidia (#1990) 2023-05-27 12:13:54 +02:00
Ivan Nardi ace32c9dfe Add support for SRTP (#1977)
The goal is to have Zoom flows classified as "Encrypted" and not as
"Cleartext".

Start documenting the list of protocols supported by nDPI;
format, verbosity and content are still a work-in-progress.
2023-05-16 16:02:14 +02:00
Ivan Nardi 4e186f6bfb HTTP: rework state machine (#1966)
The goal if to correlate the right request-response pair, exporting
metadata from only one transaction (for example, the right url & return
state pair)

As a nice side effect, the code should be much cleaner, but that is a
matter of taste.

Two differences respect to the previous code:
* as it happens in the CI, if in the flow there are only one response
(before) and one request (after), only the metadata of the response are
saved/exported
* for performance reasons, we don't call `ndpi_parse_packet_line_info()`
anymore for ALL packets triggering the HTTP dissector, but only for the
packets that we already know belong to an HTTP flow. This is the reason
for the changes in RTSP/SOAP/... code
2023-05-16 16:01:55 +02:00
Ivan Nardi 31d23aee56 All protocols should be excluded sooner or later (#1969)
For a lot of protocols, reduce the number of packets after which the
protocols dissector gives up.
The values are quite arbitary, tring to not impact on classification
2023-05-10 12:50:24 +02:00
Ivan Nardi 684e041998 Improve detection of crawlers/bots (#1968)
Add support for Facebook crawler
2023-05-09 16:42:29 +02:00
Luca Deri 2f9059dc54 Disabled query string validation in MDNS in order to avoid zapping chars that in DNS (instead) are not permitted 2023-05-09 15:04:02 +02:00
Ivan Nardi 0223d3c4f5 HTTP: improve extraction of metadata and of flow risks (#1959) 2023-05-05 13:35:20 +02:00
Ivan Nardi a11ddd2dc4 HTTP: remove useless code about XBOX (#1958)
Old nDPI versions were able to detect XBOX flows over HTTP via
user-agent matching. This feature has been removed from a long time
(89d548f9d, at very least)
2023-05-04 11:28:03 +02:00
Ivan Nardi bb370f5ef0 Make Bittorrent LRU cache IPv6 aware. (#1909)
It was the only remaining LRU cache without IPv6 support.
See 81e1ea545c
2023-04-29 11:09:23 +02:00
Ivan Nardi de693cbbc9 RTP: remove dead-code (#1953)
The checks `isValidMSRTPType(..) == 1` is a subset of
`is_valid_rtp_payload_type()` so this if-branch is never reached.

More importantly, the article describing how to detect Microsoft Lync and
Skype for Business is from 2014. These payload types are static or they
are in the dynamic range: in both cases, these values might be used (and
they are used indeed) pretty much by every application.
Bottom line: we can't use PT alone to identify a specific protocol.

Keep the list, since it is used to tell audio streams from video ones.
2023-04-25 14:16:27 +02:00
Luca Deri c4ba981370 Compilation fix on RH7 2023-04-24 08:49:11 +02:00
nikitamishagin d3e67faf20 Add "Heroes of the Storm" video game signature detection. (#1949) 2023-04-22 18:28:43 +02:00
Toni 85751ba9d7 Added OICQ dissector. (#1950)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-21 11:30:10 +02:00
Ivan Nardi 032e778a6d Simplify ndpi_internal_guess_undetected_protocol() (#1941)
`ndpi_guess_undetected_protocol()/ndpi_internal_guess_undetected_protocol()`
is a strange function:
* it is exported by the library and it is actively used by `ntopng`
* it is intrinsecally ipv4-only
* it returns basically something like "classification_by_ip"/"classification_by_port"
(these information have already been calculated in `ndpi_do_guess()`...)
* it access the bittorrent LRU caches (similarly to
`ndpi_detection_giveup()` but without all the other caches...)

So:
* make the interface IPv4/6 agnostic
* use the classifications already available

This work will allow to make the Bittorrent caches IPV6-aware (see
81e1ea5).

Handle Dropbox classification "by-port" in the "standard" way.
2023-04-12 14:39:10 +02:00
Toni 6660feed35 Added BACnet dissector. (#1940)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 20:26:27 +02:00
Toni bebcce5281 Added Source Engine dissector. (#1937)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 14:31:57 +02:00
Ivan Nardi 06a63dcbaa MySql: improve detection (#1928)
Add support for flows with "caching_sha2_password" authentication plugin.

See #1924
2023-04-06 09:34:39 +02:00
Ivan Nardi 5c28dbbae2 H323: fix false positives (#1916) 2023-04-06 09:32:57 +02:00
Ivan Nardi 4d11941d32 Ookla: rework detection (#1922)
The logic of the LRU cache has been changed: once we know an ip has
connected to an Ookla server, all the following (unknown) flows (for
a short time interval) from the same ip to the port 8080 are treated
as Ookla ones.

Most of the changes in this commit are about introducing the concept of
"aggressive detection". In some cases, to properly detect a
protocol we might use some statistical/behavior logic that, from one
side, let us to identify the protocol more often but, from the other
side, might lead to some false positives.
To allow the user/application to easily detect when such logic has been
triggered, the new confidence value `NDPI_CONFIDENCE_DPI_AGGRESSIVE` has been
added.
It is always possible to disable/configure this kind of logic via the
API.

Detection of Ookla flows using plain TLS over port 8080 is the first
example of aggressive detection in nDPI.

Tested with:
* Android 9.0 with app 4.8.3
* Ubuntu 20.04 with Firefox 110
* Win 10 with app 1.15 and 1.16
* Win 10 with Chrome 108, Edge 108 and Firefox 106
2023-03-30 17:13:51 +02:00
Ivan Nardi 6e0cd16225 TLS: fix another interger overflow in certificate processing (#1915) 2023-03-30 05:12:40 +02:00
Maatuq 9ea8a57839 remove redefinition to vxlanhdr struct in vxlan dissector (#1911) 2023-03-25 19:25:22 +01:00
Ivan Nardi ddb9aa0f95 TLS: fix parsing of certificate elements (#1910)
```
==1228==ERROR: AddressSanitizer: SEGV on unknown address 0x6040000bed05 (pc 0x00000056e148 bp 0x7ffcca534320 sp 0x7ffcca5330c0 T0)
==1228==The signal is caused by a WRITE memory access.
    #0 0x56e148 in processCertificateElements ndpi/src/lib/protocols/tls.c:682:79
    #1 0x56c60f in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_tls_certificate.c:43:3
    #2 0x43de63 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15
    #3 0x4295c2 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:324:6
    #4 0x42ee6c in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:860:9
    #5 0x4583a2 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    #6 0x7f8c021c9082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
    #7 0x41f78d in _start
```

Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=57317
2023-03-22 18:18:36 +01:00
Ivan Nardi 3a75a46212 Add a new protocol id for generic Adult Content traffic (#1906)
The list has been taken from https://www.similarweb.com/top-websites/adult/
Fix a GoTo false positive.
2023-03-20 17:56:02 +01:00
Ivan Nardi 0e80828e14 fuzz: add a new fuzzer to test TLS certificates (#1901) 2023-03-20 17:00:46 +01:00
Ivan Nardi 89cae9ddf2 Add a new flow risk about literal IP addresses used as SNI (#1892)
RFC 6066 3: "Literal IPv4 and IPv6 addresses are not permitted in
"HostName"."

Don't set this risk if we have a valid sub-classification (example:
via certificate)

Since a similar risk already exists for HTTP hostnames, reuse it, with a
more generic name.
2023-03-02 15:27:30 +01:00
Nardi Ivan 9eb9664516 SoftEther: fix invalid memory access
We can't write `flow->protos` union until we are really sure about protocol
classification

```
==28334==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x558db5554512 bp 0x000000000000 sp 0x7ffcb22c2880 T0)
==28334==The signal is caused by a READ memory access.
==28334==Hint: this fault was caused by a dereference of a high value address (see register values below).  Disassemble the provided pc to learn which register was used.
    #0 0x558db5554512 in __asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x48e512) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7)
    #1 0x558db55ea54b in __interceptor_free (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x52454b) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7)
    #2 0x558db56977ca in ndpi_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:274:7
    #3 0x558db56c20e3 in ndpi_free_flow_data /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5175:2
    #4 0x558db569783f in ndpi_free_flow /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8394:5
    #5 0x558db5627936 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:38:3
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56272
2023-03-01 20:34:56 +01:00