Commit Graph
1106 Commits
Author SHA1 Message Date
Toni fe28d55801 Improved MGCP dissector. (#1717)
* typ0s fixed
 * dissect endpoint hostnames

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-30 11:21:58 +02:00
Nardi Ivan 8bfb1712d8 QUIC: add support for 0-RTT packets received before the Initial
RFC9001 4.6.1: "A client that wishes to send 0-RTT packets uses the
early_data extension in the ClientHello message of a subsequent handshake;
see Section 4.2.10 of [TLS13]. It then sends application data in 0-RTT
packets."
That means the client sends before the CH (in the Initial) and then the
0-RTT (in the same UDP datagram or not)".

However, because of packet loss or out-of-order delivery, it might
happens that a 0-RTT packet is received before the Initial (the original
one or a retransmission).
For example, Google and Facebook servers save 0-RTT packets for a small
amount of time in hopes of receiving the corresponding Initial.

Update the QUIC dissector to detect 0-RTT packets and keep looking for
the Initial.

Issue found by @utoni in #1706; the trace example has been taken from that
PR.
2022-08-24 15:38:30 +02:00
Toni 0c8bc9f055 Add FastCGI protocol detection. (#1711)
* CQL: fixed byte order conversion (BigEndian not LittleEndian)
 * CQL: increased required successful dissected packets to prevent false-positives

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 13:22:46 +02:00
Toni 30730e95e5 Add Kismet protocol detection. (#1710)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 10:50:11 +02:00
lns 93d65ed650 Support serialization of double-precision floating-point numbers. Fixes #1702.
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 10:49:27 +02:00
Luca Deri a53f476585 Modified definition of hll_add 2022-08-17 16:25:10 +02:00
Toni 2e25c36396 Add TiVoConnect dissector. Fixes #1697. (#1699)
* added static assert if supported, to complain if the flow struct changes

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-08 19:04:20 +02:00
Ivan Nardi 1d4e27c473 Further simplification of ndpi_process_extra_packet() (#1698)
See 95e16872.

After c0732eda, we can safely remove the protocol list from
`ndpi_process_extra_packet()`.

The field `flow->check_extra_packets` is redundant; remove it.
2022-08-05 15:15:45 +02:00
Ivan Nardi c0732eda45 HTTP: improve sub-classification (#1696)
Content-matched sub-protocols (DASH, IPP, MPEGDASH...) shouldn't
ovewrite the previous master protocol (if any; usually HTTP).

Furthermore. the HTTP dissector shouldn't update the classification
(in the extra-dissection code path) if a content-matched sub-protocols
has already been found.

This commit should address the first part of the changes described in #1687.
2022-08-05 14:14:38 +02:00
Luca Deri 9fb11c909e Win fixes 2022-08-05 11:17:28 +02:00
Toni ac24b35b1f Add Discord dissector. (#1694)
* fixed RiotGames false positive

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-03 12:03:36 +02:00
Luca Deri 154d6a5069 Compilation fix (not all compilers like restrict) 2022-08-03 10:47:31 +02:00
Toni b3e722e5a8 Improved nDPI JSON serialization. (#1689)
* fixed autoconf CFLAGS/LDFLAGS MSAN issue which could lead to build errors
 * introduced portable version of gmtime_r aka ndpi_gmtime_r
 * do as most as possible of the serialization work in ndpi_utils.c
 * use flow2json in ndpiReader

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-02 17:54:44 +02:00
Toni 26aafd931c Add Activision dissector. (#1693)
Signed-off-by: lns <matzeton@googlemail.com>
2022-08-02 16:15:07 +02:00
Ivan Nardi 013212c67c TLS: avoid useless extra dissection (#1690)
If we have seen Application Data blocks in both directions, it means we
are after the handshake. At that point, extra dissection is useless.

Useful feature with TLS mid-sessions, i.e. sessions without initial
packets.
2022-07-31 16:57:17 +02:00
Toni ed4f106a0d Add Softether dissector. (#1679)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-29 19:29:54 +02:00
Ivan Nardi 405a52ed65 Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1683)
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.

Add an option to `ndpiReader` to dump them; enabled by default in the
unit tests.
This new option enables/disables dumping of "num dissectors calls"
values, too (see b4cb14ec).
2022-07-29 15:25:00 +02:00
Ivan Nardi df2e11ef51 Revert "Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1677)" (#1682)
This reverts commit bb83899985.
2022-07-29 12:08:40 +02:00
Ivan Nardi bb83899985 Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1677)
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.

Add an option to `ndpiReader` to dump them; disabled by default to avoid
too much fuss with the unit tests.
2022-07-29 12:07:41 +02:00
Ivan Nardi 172e698bb8 TINC: avoid processing SYN packets (#1676)
Since e6b332aa, we have proper support for detecting client/server
direction. So Tinc dissector is now able to properly initialize the
cache entry only when needed and not anymore at the SYN time; initializing
that entry for **every** SYN packets was a complete waste of resources.

Since 4896dabb, the various `struct ndpi_call_function_struct`
structures are not more separate objects and therefore comparing them
using only their pointers is bogus: this bug was triggered by this
change because `ndpi_str->callback_buffer_size_tcp_no_payload` is now 0.
2022-07-28 12:39:18 +02:00
Ivan Nardi d8d525fff2 Update the protocol bitmask for some protocols (#1675)
Tcp retransmissions should be ignored.

Remove some unused protocol bitmasks.

Update script to download Whatsapp IP list.
2022-07-27 11:46:45 +02:00
Toni ab3a678ad4 Add AVAST dissector. (#1674)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-25 18:07:44 +02:00
Ivan Nardi e6b332aa4a Add support for flow client/server information (#1671)
In a lot of places in ndPI we use *packet* source/dest info
(address/port/direction) when we are interested in *flow* client/server
info, instead.

Add basic logic to autodetect this kind of information.

nDPI doesn't perform any "flow management" itself but this task is
delegated to the external application. It is then likely that the
application might provide more reliable hints about flow
client/server direction and about the TCP handshake presence: in that case,
these information might be (optionally) passed to the library, disabling
the internal "autodetect" logic.

These new fields have been used in some LRU caches and in the "guessing"
algorithm.
It is quite likely that some other code needs to be updated.
2022-07-24 17:46:24 +02:00
Toni a25b2a7e37 Added AliCloud server access dissector. (#1672)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-23 11:21:49 +02:00
Zied Aouini 40d7354dd7 Windows error C2466 fix (array size init with 0). 2022-07-22 21:58:25 +02:00
Zied Aouini 9d040f6317 Windows compilation fix. 2022-07-22 20:14:30 +02:00
Ivan Nardi 52005e88ed TLS: improve reassembler (#1669)
* TLS: cosmetic changes

* TLS: improve reassembler

We might need to contemporary re-order messages from both directions:
use one buffer per direction.
2022-07-22 12:19:21 +02:00
Nardi Ivan d66aa49787 DTLS: fix exclusion of DTLS protocol
Add an helper to exclude a generic protocol
2022-07-20 19:16:18 +02:00
Nardi Ivan e1edb08f06 SKYPE: fix detection over UDP
Commit ba6a48c9 is completely bogus: we can't set extra dissection
without having set a proper classification.

The idea behind that commit seems to be that we need to look for 2
(consecutives?) packets with the same crc/pattern: try to implement this
logic in a saner way.
2022-07-20 16:13:55 +02:00
Nardi Ivan 5702c6fb08 SKYPE: remove detection over TCP
Skype detection over TCP has been completely disable since 659f75138 (3
years ago!).
Since that logic was too weak anyway, remove it.
2022-07-20 16:13:55 +02:00
Ivan Nardi 407155755d ASN1/BER: fix signed integer overflow (#1660)
```
protocols/snmp_proto.c:77:23: runtime error: signed integer overflow: 6 + 2147483647 cannot be represented in type 'int'
 #0 0x52f69e in ndpi_search_snmp ndpi/src/lib/protocols/snmp_proto.c:77:23
 #1 0x4c5347 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:5211:4
 #2 0x4c5591 in ndpi_check_flow_func ndpi/src/lib/ndpi_main.c:0
 #3 0x4c8903 in ndpi_detection_process_packet ndpi/src/lib/ndpi_main.c:6145:15
 #4 0x4b3712 in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_process_packet.c:29:5
[...]
```
Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49057
2022-07-12 18:39:05 +02:00
Luca 9c235796af Introduced risk accountability 2022-07-12 10:36:40 +02:00
Ivan Nardi b4cb14ec19 Keep track of how many dissectors calls we made for each flow (#1657) 2022-07-11 09:47:47 +02:00
Ivan Nardi 510517126a LDAP: rewrite dissection (#1649) 2022-07-08 12:50:46 +02:00
Toni 15042870f9 Added Threema Messenger. (#1643)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 19:30:10 +02:00
Toni 175f863665 Label SMTP w/ STARTTLS as SMTPS *and* dissect TLS clho. (#1639)
* Label SMTP w/ STARTTLS as SMTPS *and* dissect TLS clho.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Revert "SMTP with STARTTLS is now identified as SMTPS"

This reverts commit 52d987b603.

* Revert "Compilation fix"

This reverts commit c019946f60.

* Sync unit tests.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 12:40:25 +02:00
Luca Deri c019946f60 Compilation fix 2022-07-05 17:38:32 +02:00
Ivan Nardi 7645909460 Fix handling of NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1636) 2022-07-05 17:01:00 +02:00
Luca Deri 461589517e Updated ndpi_check_flow_risk_exceptions() signature 2022-07-04 21:38:38 +02:00
Luca Deri e7a5eaecde Cleaned-up issuer DN check code adding
u_int8_t ndpi_check_issuerdn_risk_exception(struct ndpi_detection_module_struct *ndpi_str, char *issuerDN);

Added new API function for checking nDPI-configured exceptions
u_int8_t ndpi_check_flow_risk_exception(struct ndpi_detection_module_struct *ndpi_str,
                                        u_int num_params,
                                        ndpi_risk_params **params);
2022-07-04 18:41:01 +02:00
Toni Uhlig b3d3e3b210 Replaced malicious JA3-md5/SSL-cert-sha1 ac automata with hashmaps.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 16:05:22 +02:00
Toni 4ff8aa48b2 Added UltraSurf protocol dissector. (#1618)
* TLSv1.3 UltraSurf flows are not detected by now

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 16:04:53 +02:00
Ivan Nardi 5aa3d9126f Add two new confidence values: confidence by partial DPI (#1632)
Used for all classifications based on partial/incomplete DPI
information, i.e. all classifications done in `ndpi_detection_giveup()`.
2022-07-04 13:56:51 +02:00
Toni 75f7da5c26 Added Psiphon detection patterns. See #566 and #1099. (#1631)
* The traces are not up to date, but this is the best we got so far.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 10:34:54 +02:00
Toni a74fc089c4 Added i3D and RiotGames protocol dissectors. (#1609)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 20:43:30 +02:00
Ivan Nardi faaff58620 TargusDataspeed: avoid false positives (#1628)
TargusDataspeed dissector doesn't perform any real DPI checks but it only
looks at the TCP/UDP ports.
Delete it, and use standard logic to classify these flows by port.
2022-07-03 20:28:58 +02:00
Ivan Nardi b5fb2066cb bins: add support for 64bit bins (#1626) 2022-07-03 19:25:15 +02:00
Toni 1a01e8dc68 Improved TFTP. Dissect Read/Write Request filenames. (#1617)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 14:37:05 +02:00
Toni 7c5c811eb0 Added Cloudflare WARP detection patterns. (#1615) (#1616)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-02 14:57:56 +02:00
Toni bb72aa4767 Added TunnelBear VPN detection patterns. (#1615)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-01 13:19:17 +02:00