Commit Graph
69 Commits
Author SHA1 Message Date
Ivan Nardi 405a52ed65 Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1683)
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.

Add an option to `ndpiReader` to dump them; enabled by default in the
unit tests.
This new option enables/disables dumping of "num dissectors calls"
values, too (see b4cb14ec).
2022-07-29 15:25:00 +02:00
Ivan Nardi 172e698bb8 TINC: avoid processing SYN packets (#1676)
Since e6b332aa, we have proper support for detecting client/server
direction. So Tinc dissector is now able to properly initialize the
cache entry only when needed and not anymore at the SYN time; initializing
that entry for **every** SYN packets was a complete waste of resources.

Since 4896dabb, the various `struct ndpi_call_function_struct`
structures are not more separate objects and therefore comparing them
using only their pointers is bogus: this bug was triggered by this
change because `ndpi_str->callback_buffer_size_tcp_no_payload` is now 0.
2022-07-28 12:39:18 +02:00
Ivan Nardi d8d525fff2 Update the protocol bitmask for some protocols (#1675)
Tcp retransmissions should be ignored.

Remove some unused protocol bitmasks.

Update script to download Whatsapp IP list.
2022-07-27 11:46:45 +02:00
Toni ae2bedce3a Improved Jabber/XMPP detection. (#1661)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-13 17:55:33 +02:00
Ivan Nardi b4cb14ec19 Keep track of how many dissectors calls we made for each flow (#1657) 2022-07-11 09:47:47 +02:00
Ivan Nardi 7645909460 Fix handling of NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1636) 2022-07-05 17:01:00 +02:00
Luca Deri 227ab5c105 Enhanced TLS risk info reported to users 2022-06-28 00:01:00 +02:00
Luca Deri ab09b8ce2e Added unidirectional traffic flow risk 2022-06-20 00:22:13 +02:00
Ivan Nardi 3b825fca6d Certificate timestamps should be printed in UTC (#1563)
This way, the CI results are consistent
2022-05-30 13:43:05 +02:00
Luca Deri 1da9f1a36f Updated tests results
Code cleanup
2022-05-30 00:54:17 +02:00
Luca Deri 5dcb9c1c16 Modified risk labels 2022-05-04 12:10:01 +02:00
Ivan Nardi 71636dcafd Sync unit tests results (#1533) 2022-04-27 18:22:11 +02:00
Ivan Nardi 86b97ffb73 Sync unit tests results (#1423)
Fix: 7a3aa41a
2022-01-28 17:17:39 +01:00
Luca Deri 81b33f0d06 Updated test results after the risk NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE has been added 2022-01-26 09:46:46 +01:00
Ivan Nardi 3a087e951d Add a "confidence" field about the reliability of the classification. (#1395)
As a general rule, the higher the confidence value, the higher the
"reliability/precision" of the classification.

In other words, this new field provides an hint about "how" the flow
classification has been obtained.
For example, the application may want to ignore classification "by-port"
(they are not real DPI classifications, after all) or give a second
glance at flows classified via LRU caches (because of false positives).

Setting only one value for the confidence field is a bit tricky: more
work is probably needed in the next future to tweak/fix/improve the logic.
2022-01-11 15:23:39 +01:00
Luca Deri 7aad97fffb Added Microsoft Azure support 2021-12-19 18:59:45 +01:00
Ivan Nardi b1e9245d94 ndpiReader: slight simplificaton of the output (#1378) 2021-11-27 17:32:23 +01:00
Ivan Nardi b6d9536533 Fixed cleartext protocol assignment (#1357) 2021-10-25 15:04:04 +02:00
Luca Deri 9e97d20c25 Refreshed results list 2021-10-16 12:03:16 +02:00
Luca Deri c2e7fbce94 Updated test results after latest commit 2021-10-16 11:57:32 +02:00
Luca Deri 408d78e628 Improved DGA detection for skipping potential DGAs of known/popular domain names 2021-10-05 16:51:24 +02:00
Ivan Nardi f3fcf1e7c0 Update unit tests results after da8eed5a (#1323) 2021-10-05 15:06:34 +02:00
Luca Deri e8455236bd Updated output 2021-08-07 17:38:33 +02:00
Ivan Nardi cccf794265 ndpiReader: add statistics about nDPI performance (#1240)
The goal is to have a (roughly) idea about how many packets nDPI needs
to properly classify a flow.

Log this information (and guessed flows number too) during unit tests,
to keep track of improvements/regressions across commits.
2021-07-13 12:28:39 +02:00
Luca Deri 732bcecd17 Added flow risk score 2021-05-18 21:05:47 +02:00
Luca Deri 0f8a994841 Improved DGA detection
Before
Accuracy 66%, Precision 86%, Recall 38%

After
Accuracy 71%, Precision 89%, Recall 49%
2021-03-03 19:30:01 +01:00
Luca Deri 56bfb439f8 Improved DGA detection with trigrams. Disadvantage: slower startup time
Reworked Tor dissector embedded in TLS (fixes #1141)
Removed false positive on HTTP User-Agent
2021-03-03 00:41:07 +01:00
Luca Deri 23a15bae5f Fixes #1029 2020-11-27 18:51:56 +01:00
Luca Deri d670850f55 Updated results 2020-11-16 21:25:59 +01:00
Luca Deri 638624869a Added new risk for NDPI_UNSAFE_PROTOCOL that identifies protocols that are not condidered safe/secure 2020-08-30 20:48:58 +02:00
Luca Deri dd75060932 Fixed false positive in suspicous user agent
Optimized stddev calculation
2020-08-30 12:25:15 +02:00
Luca Deri 2ae4c6675d Fixed partial TLS dissection 2020-07-30 18:30:07 +02:00
Luca Deri e71df49b3e Changed due to bin size extension 2020-07-30 00:06:46 +02:00
Luca Deri 12abcd516b Updated test results due to bin changes 2020-07-09 17:28:02 +02:00
Luca Deri d9af1562f0 Fixes #906
Packet bins are not printed wehn empty
2020-06-22 14:30:26 +02:00
Luca Deri 1a62f4c799 Added ndpi_bin_XXX API
Added packet lenght distribution bins
2020-06-22 01:02:54 +02:00
Luca Deri b2c24558c5 DGA detection improvements 2020-06-18 00:17:30 +02:00
Luca Deri ee15c6149d Added TLS weak cipher and obsolete protocol version detection 2020-05-10 21:55:35 +02:00
Luca Deri 4a09b4efa0 Added TLS issuerDN and subjectDN 2020-05-07 18:44:51 +02:00
Luca Deri 942a71c7eb NetBIOS dissection improvements 2020-03-01 11:42:21 +01:00
emanuele-f fd94270507 Remove decimals in test results for IAT, packet lengths and goodput ratio 2020-02-14 11:42:20 +01:00
Luca daae1cc9b1 Reworked TLS dissection 2020-01-01 12:59:19 +01:00
Luca Deri e98b994a39 Updated results 2019-11-21 13:35:04 +01:00
Luca Deri 0974075fa0 Major cleanup
Removed ndpi_pref_http_dont_dissect_response and ndpi_pref_dns_dont_dissect_response as the ndpi_extra_dissection_possible() call will now handle everything
2019-10-24 19:48:55 +02:00
Luca Deri 24a26a2e94 Added NetBIOS metadata export 2019-10-08 23:40:56 +02:00
Luca Deri 4fd7e5734a Manual merge of pull #769 2019-10-02 23:01:29 +02:00
Luca 0ed679e795 Improves IAT calculation 2019-09-24 16:37:42 +02:00
Luca Deri 086c511a11 Parsed TLS certificate validity 2019-09-10 00:34:48 +02:00
Luca 886d575157 Added -C to generate CSV analysis files
Improved IAT and byte distribution
2019-09-03 18:38:54 +02:00
Luca c328452687 Average calculation fix 2019-08-29 16:08:18 +02:00