Commit Graph
65 Commits
Author SHA1 Message Date
Toni 0c8bc9f055 Add FastCGI protocol detection. (#1711)
* CQL: fixed byte order conversion (BigEndian not LittleEndian)
 * CQL: increased required successful dissected packets to prevent false-positives

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 13:22:46 +02:00
Toni 30730e95e5 Add Kismet protocol detection. (#1710)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 10:50:11 +02:00
Toni 2e25c36396 Add TiVoConnect dissector. Fixes #1697. (#1699)
* added static assert if supported, to complain if the flow struct changes

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-08 19:04:20 +02:00
Ivan Nardi 405a52ed65 Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1683)
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.

Add an option to `ndpiReader` to dump them; enabled by default in the
unit tests.
This new option enables/disables dumping of "num dissectors calls"
values, too (see b4cb14ec).
2022-07-29 15:25:00 +02:00
Ivan Nardi 172e698bb8 TINC: avoid processing SYN packets (#1676)
Since e6b332aa, we have proper support for detecting client/server
direction. So Tinc dissector is now able to properly initialize the
cache entry only when needed and not anymore at the SYN time; initializing
that entry for **every** SYN packets was a complete waste of resources.

Since 4896dabb, the various `struct ndpi_call_function_struct`
structures are not more separate objects and therefore comparing them
using only their pointers is bogus: this bug was triggered by this
change because `ndpi_str->callback_buffer_size_tcp_no_payload` is now 0.
2022-07-28 12:39:18 +02:00
Ivan Nardi d8d525fff2 Update the protocol bitmask for some protocols (#1675)
Tcp retransmissions should be ignored.

Remove some unused protocol bitmasks.

Update script to download Whatsapp IP list.
2022-07-27 11:46:45 +02:00
Toni ab3a678ad4 Add AVAST dissector. (#1674)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-25 18:07:44 +02:00
Toni a25b2a7e37 Added AliCloud server access dissector. (#1672)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-23 11:21:49 +02:00
Toni ae2bedce3a Improved Jabber/XMPP detection. (#1661)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-13 17:55:33 +02:00
Ivan Nardi b4cb14ec19 Keep track of how many dissectors calls we made for each flow (#1657) 2022-07-11 09:47:47 +02:00
Toni 15042870f9 Added Threema Messenger. (#1643)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 19:30:10 +02:00
Ivan Nardi 7645909460 Fix handling of NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1636) 2022-07-05 17:01:00 +02:00
Toni 4ff8aa48b2 Added UltraSurf protocol dissector. (#1618)
* TLSv1.3 UltraSurf flows are not detected by now

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 16:04:53 +02:00
Toni 75f7da5c26 Added Psiphon detection patterns. See #566 and #1099. (#1631)
* The traces are not up to date, but this is the best we got so far.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 10:34:54 +02:00
Toni a74fc089c4 Added i3D and RiotGames protocol dissectors. (#1609)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 20:43:30 +02:00
Ivan Nardi faaff58620 TargusDataspeed: avoid false positives (#1628)
TargusDataspeed dissector doesn't perform any real DPI checks but it only
looks at the TCP/UDP ports.
Delete it, and use standard logic to classify these flows by port.
2022-07-03 20:28:58 +02:00
Toni 7c5c811eb0 Added Cloudflare WARP detection patterns. (#1615) (#1616)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-02 14:57:56 +02:00
Luca Deri 008a1790e4 Fixed SMTP default port 587 2022-07-02 11:49:22 +02:00
Toni bb72aa4767 Added TunnelBear VPN detection patterns. (#1615)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-01 13:19:17 +02:00
Luca Deri 227ab5c105 Enhanced TLS risk info reported to users 2022-06-28 00:01:00 +02:00
Luca Deri ab09b8ce2e Added unidirectional traffic flow risk 2022-06-20 00:22:13 +02:00
Toni 432de5eb57 Added collectd dissector (again). (#1601)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-17 19:56:33 +02:00
Toni 20a29c393f Improved IPSec/ISAKMP detection. (#1600)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-16 09:16:18 +02:00
Ivan Nardi e2cc08bfe5 Add support for PIM (Protocol Indipendent Multicast) protocol (#1599)
Close #1598
2022-06-15 12:25:26 +02:00
Luca Deri 11babc7ea3 Added Pragmatic General Multicast (PGM) protocol detection 2022-06-08 09:11:22 +02:00
Toni 938e89ca33 Reimplemented 1kxun application protocol. (#1585)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-06 18:04:49 +02:00
Toni 7419cfee64 Added RSH dissector. Fixes #202. (#1581)
- added syslog false-positive pcap that was missing in 09fbe0a64a
 - added NDPI_ARRAY_LENGTH() macro, usable on `type var[]` declarations

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-04 19:12:53 +02:00
Ivan Nardi 338427d712 Add support for GoTo products (mainly GoToMeeting) (#1580)
There is some overlap with Citrix protocol.
2022-06-04 14:30:52 +02:00
Luca Deri 1da9f1a36f Updated tests results
Code cleanup
2022-05-30 00:54:17 +02:00
Ivan Nardi 9c1a53f39f Dazn: add support for Dazn streaming service (#1559)
Update .gitignore file
2022-05-29 17:47:16 +02:00
Toni 33f9729ee4 Added MPEG-DASH dissector. Fixes #1223. (#1555)
* Improved HTTP POST detection
 * Refactored subprotocol detection

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-29 13:12:13 +02:00
Toni 915ffebade Added Softether(-VPN) DDNS service detection. (#1544)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-09 08:16:19 +02:00
Toni b2648a45a3 Added Edgecast and Cachefly CDNs. (#1540)
* Improved ASN update script
 * Ran `utils/update_every_lists.sh'
 * `tests/do.sh.in' prints the amount of failed pcap(s)
 * `utils/asn_update.sh' prints the amount of failed download(s)

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-07 09:33:25 +02:00
Ivan Nardi 71636dcafd Sync unit tests results (#1533) 2022-04-27 18:22:11 +02:00
Ivan Nardi 075bce5f3d XIAOMI: add detection of Xiaomi traffic (#1529)
Most of the credits should go to @utoni (see #1521)
2022-04-25 11:00:02 +02:00
Toni ac0f50b561 Added RakNet protocol dissector. (#1527)
* Frame Set PDU's do not get fully dissected for the sake of simplicity

Signed-off-by: lns <matzeton@googlemail.com>
2022-04-24 13:08:57 +02:00
Ivan Nardi 42909673ce Add some scripts to easily update some IPs lists (#1522)
Follow-up of 8b062295

Add a new protocol id for generic Tencent/Wechat flows
2022-04-21 20:43:52 +02:00
Vitaly Lavrov a1451935b8 Errors fixed (#1482)
Fixed errors for bigendian platforms in ndpiReader.
All address and port comparisons and hash calculations are done with
endian in mind.
The get_ndpi_flow_info() function searched for an existing flow for the
forward and reverse direction of the packet.
The ndpi_workflow_node_cmp() function looked for a flow regardless of
the packet's direction. This is what led to an error in determining the
direction of transmission of the packet.

Fixed error in "synscan" test: the number of packets in the forward and
reverse direction is incorrectly defined (verified via tcpdump).

Fixed bug with icmp protocol checksum check for big endian platforms.
2022-03-08 00:20:56 +01:00
Ivan Nardi 2f5f445f72 Add support for Google Cloud (#1447)
Differentiate between Google its own apps/services and Google Cloud.
We already do something similar for Amazon vs AWS and Microsoft vs Azure.
2022-02-20 10:19:52 +01:00
Luca Deri 5ddbbf4dd2 Added cybersecurity protocol and category that groups traffic towards leading cybersecurity companies and CDNs, useful to make destinations that should be marked as trusted in firewalls and security gateways 2022-02-10 09:50:06 +01:00
Luca Deri be5a29ba9d Added HSRP protocol detection
Removed attic directory now obsolete
2022-02-08 18:04:57 +01:00
Ivan Nardi 79968f3245 Sync utests (#1433)
* Sync utest results

* Fix read-heap-buffer-overflow error reported by CI

See: https://github.com/ntop/nDPI/runs/5055876515?check_suite_focus=true
2022-02-04 15:57:17 +01:00
Luca Deri 7aef27f85e Added NDPI_ERROR_CODE_DETECTED risk 2022-02-03 13:20:54 +01:00
Ivan Nardi 513e386959 Extend protocols support (#1422)
Add detection of AccuWeather site/app and Google Classroom.
Improve detection of Azure, Zattoo, Whatsapp, MQTT and LDAP.

Fix some RX false positives.

Fix some "Uncommon TLS ALPN"-risk false positives.

Fix "confidence" value for some Zoom/Torrent classifications.

Minor fix in Lua script for Wireshark extcap.

Update .gitignore file.

Let GitHub correctly detect the language type of *.inc files.

Zattoo example has been provided by @subhajit-cdot in #1148.
2022-01-29 09:19:26 +01:00
Ivan Nardi e4a7990a4e H323: fix a use-after-poison error (#1412)
Detected by oss-fuzz
See: https://oss-fuzz.com/testcase-detail/6730505580576768

Fix a function prototype
Update a unit test results
2022-01-17 19:47:27 +01:00
Luca Deri d59fefd06e Improved MicrosoftAzure detection 2022-01-12 21:49:31 +01:00
Ivan Nardi 3a087e951d Add a "confidence" field about the reliability of the classification. (#1395)
As a general rule, the higher the confidence value, the higher the
"reliability/precision" of the classification.

In other words, this new field provides an hint about "how" the flow
classification has been obtained.
For example, the application may want to ignore classification "by-port"
(they are not real DPI classifications, after all) or give a second
glance at flows classified via LRU caches (because of false positives).

Setting only one value for the confidence field is a bit tricky: more
work is probably needed in the next future to tweak/fix/improve the logic.
2022-01-11 15:23:39 +01:00
Ivan Nardi fdb6481cd6 Add support for ICloud Private Relay (#1390)
See:
https://www.apple.com/privacy/docs/iCloud_Private_Relay_Overview_Dec2021.PDF

TODO: an up-to-date list of egress IP ranges is publicly available. Can
we use it somehow?
2021-12-22 19:55:57 +01:00
Luca Deri 7aad97fffb Added Microsoft Azure support 2021-12-19 18:59:45 +01:00
Ivan Nardi 7153b8933c Improve/add several protocols (#1383)
Improve Microsoft, GMail, Likee, Whatsapp, DisneyPlus and Tiktok
detection.
Add Vimeo, Fuze, Alibaba and Firebase Crashlytics detection.

Try to differentiate between Messenger/Signal standard flows (i.e chat)
and their VOIP (video)calls (like we already do for Whatsapp and
Snapchat).

Add a partial list of some ADS/Tracking stuff.

Fix Cassandra, Radius and GTP false positives.
Fix DNS, Syslog and SIP false negatives.

Improve GTP (sub)classification: differentiate among GTP-U, GTP_C and
GTP_PRIME.

Fix 3 LGTM warnings.
2021-12-18 13:24:51 +01:00