Commit Graph
24 Commits
Author SHA1 Message Date
Ivan Nardi ccc5a20710 Add detection of Tailscale (#1857)
Extend the example of wireguard traffic
2023-01-17 21:55:38 +01:00
Ivan Nardi 470eaa6ff3 Add Meraki Cloud protocol and improve other protocols (#1854)
Avoid some LineCall and Jabber false positives.
Detect Discord mid flows.
Fix Bittorrent detection.
2023-01-17 11:38:39 +01:00
Ivan Nardi a944514dde Reduce the size of some traces used as unit test (#1845)
No traces and no flows has been removed; only long sessions
has been reduced, keeping only their first packets.

This is quite important in fuzzing systems, since these pcaps are used
as initial seed. There is no value in fuzzing long sessions, because
only the very first packets are really used/processed by nDPI.

Before:
```
du -h tests/pcap/
200M	tests/pcap/
```
After:
```
du -h tests/pcap/
98M	tests/pcap/
```
2023-01-05 16:43:35 +01:00
Ivan Nardi 3de76812d9 Remove some old protocols (#1832)
These protocols:
* have been addeded in the OpenDPI era
* have never been updated since then
* we don't have any pcap examples [*]

If (and it is a big if...) some of these protocols are still somehow
used and if someone is still interested in them, we can probably
re-add them starting from scratch (because the current detection
rules are probably outdated)

Protocols removed: DIRECT_DOWNLOAD_LINK, APPLEJUICE, DIRECTCONNECT,
OPENFT, FASTTRACK, SHOUTCAST, THUNDER, AYIYA, STEALTHNET, FIESTA,
FLORENSIA, AIMINI, SOPCAST

PPSTREAM dissector works (...) only on UDP.

[*]: with do have an AIMINI test pcap but it was some trivial http
traffic detected only by hostname matching, on domains no more
available...
2022-12-23 10:04:08 +01:00
Darryl Sokoloski b8972d1b38 Added TP-LINK Smart Home Protocol dissector. (#1841)
Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2022-12-20 20:46:06 +01:00
Darryl Sokoloski 9854aed408 Added TUYA LAN Protocol dissector. (#1838)
Tuya IoTOS Embedded Wi-Fi and BLE SDK for bk7231n.  Used by many "smart"
devices such as LED light strips, bulbs, etc.

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2022-12-19 17:02:19 +01:00
Ivan Nardi 5704e4c142 STUN: add detection of ZOOM peer-to-peer flows (#1825)
See: "Enabling Passive Measurement of Zoom Performance in Production Networks"
https://dl.acm.org/doi/pdf/10.1145/3517745.3561414
2022-12-11 23:07:35 +01:00
Ivan Nardi de16fd35aa Hangout: avoid useless lookups in the protocol tree (#1755)
We already performed exactly these lookups in the generic code to
populate `flow->guessed_protocol_id_by_ip`: use it!

This code probably needs a deeper review, since it is basicaly a simple
matching on ip + port.
2022-10-25 18:24:03 +02:00
Ivan Nardi 1796a1d814 LINE_CALL: add detection of LINE voip calls (#1761)
These flows are classifed as `LINE_CALL`; another option was
`RTP/LINE_CALL`. No sure about the best solution...

Extend LINE domains list.

Remove RTP dead code.
2022-10-06 17:09:26 +02:00
Nardi Ivan 1f345b311f Sizes of LRU caches are now configurable
0 as size value disable the cache.

The diffs in unit tests are due to the fact that some lookups are
performed before the first insert: before this change these lookups
weren't counted because the cache was not yet initialized, now they are.
2022-09-23 18:33:48 +02:00
Ivan Nardi a7c2734b38 Remove classification "by-ip" from protocol stack (#1743)
Basically:
* "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is
NEVER returned in the protocol stack (i.e.
`flow->detected_protocol_stack[]`);
* if the application is interested into such information, it can access
`ndpi_protocol->protocol_by_ip` itself.

There are mainly 4 points in the code that set the "classification
by-ip" in the protocol stack:  the generic `ndpi_set_detected_protocol()`/
`ndpi_detection_giveup()` functions and the HTTP/STUN  dissectors.

In the unit tests output, a print about `ndpi_protocol->protocol_by_ip`
has been added for each flow: the huge diff of this commit is mainly due
to that.

Strictly speaking, this change is NOT an API/ABI breakage, but there are
important differences in the classification results. For examples:
* TLS flows without the initial handshake (or without a matching
SNI/certificate) are simply classified as `TLS`;
* similar for HTTP or QUIC flows;
* DNS flows without a matching request domain are simply classified as
`DNS`; we don't have `DNS/Google` anymore just because the server is
8.8.8.8 (that was an outrageous behaviour...);
* flows previusoly classified only "by-ip" are now classified as
`NDPI_PROTOCOL_UNKNOWN`.

See #1425 for other examples of why adding the "classification by-ip" in
the protocol stack is a bad idea.

Please, note that IPV6 is not supported :(  (long standing issue in nDPI) i.e.
`ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for
IPv6 flows.

Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed.

Close #1687
2022-09-20 22:24:47 +02:00
Toni Uhlig a966d37a21 Add CryNetwork dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-16 16:48:43 +02:00
Toni Uhlig ec1981c20c Add Syncthing dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-16 15:21:49 +02:00
Toni de020b1747 Add NATPMP dissector. (#1738)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-16 13:32:30 +02:00
Ivan Nardi 4f584f78a0 Fix ndpi_do_guess() (#1731)
Avoid a double call of `ndpi_guess_host_protocol_id()`.
Some code paths work for ipv4/6 both
Remove some never used code.
2022-09-12 19:28:41 +02:00
Toni 2e25c36396 Add TiVoConnect dissector. Fixes #1697. (#1699)
* added static assert if supported, to complain if the flow struct changes

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-08 19:04:20 +02:00
Toni ac24b35b1f Add Discord dissector. (#1694)
* fixed RiotGames false positive

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-03 12:03:36 +02:00
Toni 26aafd931c Add Activision dissector. (#1693)
Signed-off-by: lns <matzeton@googlemail.com>
2022-08-02 16:15:07 +02:00
Toni ed4f106a0d Add Softether dissector. (#1679)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-29 19:29:54 +02:00
Ivan Nardi 405a52ed65 Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1683)
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.

Add an option to `ndpiReader` to dump them; enabled by default in the
unit tests.
This new option enables/disables dumping of "num dissectors calls"
values, too (see b4cb14ec).
2022-07-29 15:25:00 +02:00
Nardi Ivan d66aa49787 DTLS: fix exclusion of DTLS protocol
Add an helper to exclude a generic protocol
2022-07-20 19:16:18 +02:00
Ivan Nardi b4cb14ec19 Keep track of how many dissectors calls we made for each flow (#1657) 2022-07-11 09:47:47 +02:00
Luca Deri ab09b8ce2e Added unidirectional traffic flow risk 2022-06-20 00:22:13 +02:00
Ivan Nardi 7aee856aa0 Extend tests coverage (#1476)
Now there is at least one flow under `tests/pcap` for 249 protocols out
of the 284 ones supported by nDPI.

The 35 protocols without any tests are:

* P2P/sharing protocols: DIRECT_DOWNLOAD_LINK, OPENFT, FASTTRACK,
EDONKEY, SOPCAST, THUNDER, APPLEJUICE, DIRECTCONNECT, STEALTHNET

* games: CSGO, HALFLIFE2, ARMAGETRON, CROSSFIRE, DOFUS, FIESTA,
FLORENSIA, GUILDWARS, MAPLESTORY, WORLD_OF_KUNG_FU

* voip/streaming: VHUA, ICECAST, SHOUTCAST, TVUPLAYER, TRUPHONE

* other: AYIYA, SOAP, TARGUS_GETDATA, RPC, ZMQ, REDIS, VMWARE, NOE,
LOTUS_NOTES, EGP, SAP

Most of these protocols (expecially the P2P and games ones) have been
inherited by OpenDPI and have not been updated since then: even if they
are still used, the detection rules might be outdated.

However code coverage (of `lib/protocols`) only increases from 65.6% to
68.9%.

Improve Citrix, Corba, Fix, Aimini, Megaco, PPStream, SNMP and Some/IP
dissection.
Treat IPP as a HTTP sub protocol.
Fix Cassandra false positives.

Remove `NDPI_PROTOCOL_QQLIVE` and `NDPI_PROTOCOL_REMOTE_SCAN`:
these protocol ids are defined but they are never used.

Remove Collectd support: its code has never been called. If someone is
really interested in this protocol, we can re-add it later, updating the
dissector.

Add decoding of PPI (Per-Packet Information) data link type.
2022-03-09 22:37:35 +01:00