Commit Graph
77 Commits
Author SHA1 Message Date
Toni ed4f106a0d Add Softether dissector. (#1679)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-29 19:29:54 +02:00
Ivan Nardi 405a52ed65 Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1683)
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.

Add an option to `ndpiReader` to dump them; enabled by default in the
unit tests.
This new option enables/disables dumping of "num dissectors calls"
values, too (see b4cb14ec).
2022-07-29 15:25:00 +02:00
Ivan Nardi 172e698bb8 TINC: avoid processing SYN packets (#1676)
Since e6b332aa, we have proper support for detecting client/server
direction. So Tinc dissector is now able to properly initialize the
cache entry only when needed and not anymore at the SYN time; initializing
that entry for **every** SYN packets was a complete waste of resources.

Since 4896dabb, the various `struct ndpi_call_function_struct`
structures are not more separate objects and therefore comparing them
using only their pointers is bogus: this bug was triggered by this
change because `ndpi_str->callback_buffer_size_tcp_no_payload` is now 0.
2022-07-28 12:39:18 +02:00
Ivan Nardi d8d525fff2 Update the protocol bitmask for some protocols (#1675)
Tcp retransmissions should be ignored.

Remove some unused protocol bitmasks.

Update script to download Whatsapp IP list.
2022-07-27 11:46:45 +02:00
Toni ab3a678ad4 Add AVAST dissector. (#1674)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-25 18:07:44 +02:00
Toni a25b2a7e37 Added AliCloud server access dissector. (#1672)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-23 11:21:49 +02:00
Nardi Ivan d66aa49787 DTLS: fix exclusion of DTLS protocol
Add an helper to exclude a generic protocol
2022-07-20 19:16:18 +02:00
Nardi Ivan 5702c6fb08 SKYPE: remove detection over TCP
Skype detection over TCP has been completely disable since 659f75138 (3
years ago!).
Since that logic was too weak anyway, remove it.
2022-07-20 16:13:55 +02:00
Ivan Nardi c72660d7d3 reader_util: stop processing a flow (#1666)
We should stop processing a flow if all protocols have been excluded or
if we have already processed too many packets.
2022-07-20 14:48:09 +02:00
Toni ae2bedce3a Improved Jabber/XMPP detection. (#1661)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-13 17:55:33 +02:00
Ivan Nardi b4cb14ec19 Keep track of how many dissectors calls we made for each flow (#1657) 2022-07-11 09:47:47 +02:00
Ivan Nardi f8076e3a58 SMB: add (partial) support for messages split into multiple TCP segments (#1644) 2022-07-07 19:24:31 +02:00
Ivan Nardi ff4e010501 Avoid spurious calls to extra dissection (#1648)
If the extra callabck is not set, calling the extra dissection is only a
waste of resources...
2022-07-07 17:49:35 +02:00
Ivan Nardi 7645909460 Fix handling of NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1636) 2022-07-05 17:01:00 +02:00
Luca Deri ab09b8ce2e Added unidirectional traffic flow risk 2022-06-20 00:22:13 +02:00
Luca Deri defe7d7f79 Updated DNS alert triggered only with TTL == 0 2022-06-14 00:13:05 +02:00
Luca Deri cf5873ffd7 Improved DNS traffic analysis
Added ability to identify application and network protocols
2022-06-13 23:19:47 +02:00
Toni 054d151373 Improved Viber (TCP) detection. (#1547)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-10 21:37:03 +02:00
Ivan NardiandLuca Deri a6ff0dd0e3 Add few scripts to easily update some IPs lists (#1436)
* Add few scripts to easily update some IPs lists

Some IPs lists should be updated frequently: try to easy the process.
The basic idea is taken from d59fefd0 and a8fe74e5 (for Azure
addresses): one specific .c.inc file and one script for each protocol.

Add the possibility to don't load a specific list.

Rename the old NDPI_PROTOCOL_HOTMAIL id to NDPI_PROTOCOL_MS_OUTLOOK,
to identify Hotmail/Outlook/Exchange flows.

TODO: ipv6

Remove the 9 addresses associated to BitTorrent: they have been added in
e2f21116 but it is not clear why all the traffic to/from these ips
should be classified as BitTorrent.

* Added quotes

* Added quotes

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2022-02-09 11:45:48 +01:00
Ivan Nardi 3a087e951d Add a "confidence" field about the reliability of the classification. (#1395)
As a general rule, the higher the confidence value, the higher the
"reliability/precision" of the classification.

In other words, this new field provides an hint about "how" the flow
classification has been obtained.
For example, the application may want to ignore classification "by-port"
(they are not real DPI classifications, after all) or give a second
glance at flows classified via LRU caches (because of false positives).

Setting only one value for the confidence field is a bit tricky: more
work is probably needed in the next future to tweak/fix/improve the logic.
2022-01-11 15:23:39 +01:00
Ivan Nardi 7153b8933c Improve/add several protocols (#1383)
Improve Microsoft, GMail, Likee, Whatsapp, DisneyPlus and Tiktok
detection.
Add Vimeo, Fuze, Alibaba and Firebase Crashlytics detection.

Try to differentiate between Messenger/Signal standard flows (i.e chat)
and their VOIP (video)calls (like we already do for Whatsapp and
Snapchat).

Add a partial list of some ADS/Tracking stuff.

Fix Cassandra, Radius and GTP false positives.
Fix DNS, Syslog and SIP false negatives.

Improve GTP (sub)classification: differentiate among GTP-U, GTP_C and
GTP_PRIME.

Fix 3 LGTM warnings.
2021-12-18 13:24:51 +01:00
Ivan Nardi b1e9245d94 ndpiReader: slight simplificaton of the output (#1378) 2021-11-27 17:32:23 +01:00
Ivan Nardi 5464bad6db Differentiate between standard Amazon stuff (i.e market) and AWS (#1369) 2021-11-04 00:20:45 +01:00
Ivan Nardi b6d9536533 Fixed cleartext protocol assignment (#1357) 2021-10-25 15:04:04 +02:00
Luca Deri 9e97d20c25 Refreshed results list 2021-10-16 12:03:16 +02:00
Luca Deri e8455236bd Updated output 2021-08-07 17:38:33 +02:00
Ivan Nardi cccf794265 ndpiReader: add statistics about nDPI performance (#1240)
The goal is to have a (roughly) idea about how many packets nDPI needs
to properly classify a flow.

Log this information (and guessed flows number too) during unit tests,
to keep track of improvements/regressions across commits.
2021-07-13 12:28:39 +02:00
Luca Deri 43a8576efb Reworked human readeable string search in flows
Removed fragment manager code
2021-05-17 20:55:06 +02:00
Luca Deri ac1eaca8a6 Added browser TLS heuristic 2021-05-13 20:00:27 +02:00
Luca Deri fba61adf5e Improved DNS dissector 2021-02-26 00:42:15 +01:00
Luca Deri be58d78417 (Temporarely) Disabled fragment manager (#1129) 2021-02-05 23:03:08 +01:00
Luca Deri 155b1aaa68 Updated results due to the new fragment manager handler 2021-02-03 11:52:46 +01:00
Luca Deri 044ed14b4f Various optimizations to reduce not-necessary calls
Optimized various UDP dissectors
Removed dead protocols such as pando and pplive
2020-09-24 23:26:03 +02:00
Luca Deri d81bc1add6 Reworked MDNS dissector that is not based on the DNS dissector 2020-09-17 23:24:02 +02:00
Nardi Ivan 23ec82b59d Major rework of QUIC dissector
Improve support for GQUIC (up to Q046) and add support for Q050 and (IETF-)QUIC
Still no sub-classification for Q050 and QUIC
2020-08-21 22:04:55 +02:00
Luca Deri 2ae4c6675d Fixed partial TLS dissection 2020-07-30 18:30:07 +02:00
Luca Deri e71df49b3e Changed due to bin size extension 2020-07-30 00:06:46 +02:00
Luca Deri 12abcd516b Updated test results due to bin changes 2020-07-09 17:28:02 +02:00
Luca Deri d9af1562f0 Fixes #906
Packet bins are not printed wehn empty
2020-06-22 14:30:26 +02:00
Luca Deri 1a62f4c799 Added ndpi_bin_XXX API
Added packet lenght distribution bins
2020-06-22 01:02:54 +02:00
Luca Deri 4a09b4efa0 Added TLS issuerDN and subjectDN 2020-05-07 18:44:51 +02:00
Luca Deri 748df7972a Results update 2020-04-17 10:36:27 +02:00
emanuele-f fd94270507 Remove decimals in test results for IAT, packet lengths and goodput ratio 2020-02-14 11:42:20 +01:00
Luca Deri 5cad39f0e8 Added export of TLS supported version in TLS header 2020-02-07 23:26:03 +01:00
Luca Deri 3be263aafc Added TLS ALPN support 2020-02-07 21:54:04 +01:00
Luca Deri 0703ab5ac5 Improved DNS response decoding
The first decoded address is now reported by ndpiReader
2020-02-04 22:16:54 +01:00
Luca daae1cc9b1 Reworked TLS dissection 2020-01-01 12:59:19 +01:00
Luca Deri e98b994a39 Updated results 2019-11-21 13:35:04 +01:00
Luca Deri 0974075fa0 Major cleanup
Removed ndpi_pref_http_dont_dissect_response and ndpi_pref_dns_dont_dissect_response as the ndpi_extra_dissection_possible() call will now handle everything
2019-10-24 19:48:55 +02:00
Luca Deri 4fd7e5734a Manual merge of pull #769 2019-10-02 23:01:29 +02:00