Commit Graph
413 Commits
Author SHA1 Message Date
Luca ae2470fad4 Initial work towards detection via TLS of browser types 2021-05-06 21:42:06 +02:00
Luca Deri dd65142020 Compilation fix 2021-04-27 08:26:08 +02:00
Luca Deri 70686249c9 Updated code due to https://github.com/ntop/nDPI/pull/1175 2021-04-27 08:12:14 +02:00
Luca Deri 4a09707e48 Added flow risk to wireshark dissection 2021-04-26 10:17:29 +02:00
Ivan Nardi fb74785282 Fix some warnings about unused variables/functions (#1160) 2021-04-05 19:21:30 +02:00
Luca Deri a1dba74346 Trace fix 2021-04-02 12:55:15 +02:00
Luca Deri 4f8ca9485a Fixed incapoatibilities with the latest extcap/wireshark 2021-04-01 23:53:53 +02:00
Luca Deri fcbc16da00 Fixed invalid guess stats 2021-03-30 17:49:48 +02:00
Ivan Nardi a6029d250d ndpiReader: print an error msg if we found an unsupported datalink type (#1157) 2021-03-23 11:47:29 +01:00
Toni b040407683 Refactored nDPI subprotocol handling and aimini protocol detection. (#1156)
* Refactored and merged callback buffer routines for non-udp-tcp / udp / tcp / tcp-wo-payload.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Try to detect one subprotocol if a detected protocol can have one.

 * This adds a performance overhead due to much more protocol detection routine calls.
   See #1148 for more information.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactor subprotocol handling (1/2).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactor subprotocol handling (2/2).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Prevent some code duplication by using macros for ndpi_int_one_line_struct string comparision.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactored aimini HTTP detection parts (somehow related to #1148).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Added aimini client/server test pcap.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Removed master protocol as it was only used for STUN and via also removed API function ndpi_get_protocol_id_master_proto

 * Adjusted Python code to conform to the changes made during the refactoring process.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-03-23 11:46:12 +01:00
Luca Deri 627299e4dd Better DGA detection (slightly decreased accuracy) 2021-03-20 17:56:24 +01:00
Luca Deri b22cb70a68 Added % of flows with risks 2021-03-14 21:59:33 +01:00
Luca Deri 1b6275005e Added in stats the number of flows with risks 2021-03-14 21:54:35 +01:00
Luca Deri 3b02cfab69 Added flows risks report 2021-03-14 21:46:23 +01:00
Luca Deri 565a7bfce3 Reworked extendal dependency across testing tools 2021-03-14 20:48:21 +01:00
Luca Deri e58527536d Help crash fix 2021-03-14 19:55:21 +01:00
Luca Deri 9419015711 Implemented square erro rollup to avoid overflow 2021-03-14 11:01:51 +01:00
Luca 192fad4402 Added double exponential smoothing implementation 2021-03-11 09:39:52 +01:00
Luca Deri 6833ee2bbe Added single exponential smoothing API
int ndpi_ses_init(struct ndpi_ses_struct *ses, double alpha, float significance);
int ndpi_ses_add_value(struct ndpi_ses_struct *ses, const u_int32_t _value, double *forecast, double *confidence_band);
2021-03-11 00:04:33 +01:00
Luca Deri f6ad16d8f8 Added experiemntal JA3+ implementation that can be used with -z i ndpiReader 2021-03-09 23:38:29 +01:00
Ivan Nardi c50a8d4808 Add support for Snapchat voip calls (#1147)
* Add support for Snapchat voip calls

Snapchat multiplexes some of its audio/video real time traffic with QUIC
sessions. The peculiarity of these sessions is that they are Q046 and
don't have any SNI.

* Fix tests with libgcrypt disabled
2021-03-06 05:48:36 +01:00
Luca Deri 0f8a994841 Improved DGA detection
Before
Accuracy 66%, Precision 86%, Recall 38%

After
Accuracy 71%, Precision 89%, Recall 49%
2021-03-03 19:30:01 +01:00
Luca Deri 56bfb439f8 Improved DGA detection with trigrams. Disadvantage: slower startup time
Reworked Tor dissector embedded in TLS (fixes #1141)
Removed false positive on HTTP User-Agent
2021-03-03 00:41:07 +01:00
Luca Deri 4bff595733 Holt-Winters calculation improvement 2021-02-27 11:32:51 +01:00
Toni 16890a6632 Added NDPI_MALICIOUS_SHA1 flow risk. (#1142)
* An external file which contains known malicious SSL certificate SHA-1 hashes
   can be loaded via ndpi_load_malicious_sha1_file(...)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-26 17:00:05 +01:00
Luca Deri f1b22b199f Added NDPI_MALICIOUS_JA3 flow risk
Added ndpi_load_malicious_ja3_file() API call
2021-02-22 23:19:23 +01:00
Luca Deri fc3db8f169 Implemented TLS Certificate Sibject matching
Improved AnyDesk detection
2021-02-22 22:37:33 +01:00
Luca Deri fc16c9368e Added risky domain flow-risk support 2021-02-21 21:45:46 +01:00
Ivan Nardi 421609475e Fix small memory leak (#1133)
Now function definition matches the prototype in ndpi_api.h.in
2021-02-10 15:24:34 +01:00
Luca Deri 1331e0aec9 Extended the API to calculate jitter
- ndpi_jitter_init()
- ndpi_jitter_free()
- ndpi_jitter_add_value()
2021-02-09 15:56:03 +01:00
Luca Deri 4abaf3e279 Removed debug statement 2021-02-09 10:05:29 +01:00
Luca Deri 732579b72b Added timeseries forecasting support implementing Holt-Winters with confidence interval
New API calls added
- ndpi_hw_init()
- ndpi_hw_add_value()
- ndpi_hw_free()
2021-02-08 19:10:25 +01:00
Luca Deri 8dd7716ae5 Implemented more efficient and memory savvy RSI 2021-02-05 12:38:41 +01:00
Luca Deri 60b58dbd67 RSI enhancements 2021-02-05 10:59:09 +01:00
Luca Deri 1eedf734be Implemented API for computing RSI (Relative Strenght Index)
void  ndpi_init_rsi(struct ndpi_rsi_struct *s, u_int16_t num_learning_values);
void  ndpi_free_rsi(struct ndpi_rsi_struct *s);
float ndpi_rsi_add_value(struct ndpi_rsi_struct *s, const u_int32_t value);
2021-02-04 23:52:33 +01:00
Luca Deri 3e5e9569ff Added simple hash implementation to the nDPI API 2021-01-20 21:30:19 +01:00
Luca Deri d964c3e081 Code cleanup: third party uthash is at the right place 2021-01-20 19:11:36 +01:00
Luca Deri 68b6ac7da8 (C) Update 2021-01-07 11:13:36 +01:00
Luca Deri eb37f8f1fb Split HTTP request from response Content-Type. Request Content-Type should be present with POSTs and not with other methods such as GET 2021-01-06 18:28:24 +01:00
Ivan Nardi 3aa16b63e8 Fix some warnings when compiling with "-W -Wall" flags (#1103) 2021-01-04 15:49:39 +01:00
Toni 74a77e7b3d Added --ignore-vlanid / -I to exclude VLAN ids for flow hash calculation. #1073 (#1085)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-12-11 21:01:51 +01:00
Luca Deri eb689b2069 nDPI rules (work in progress) implementation 2020-11-30 22:01:49 +01:00
Luca Deri 48d640583a Moved global in reader_util.c 2020-10-27 08:40:00 +01:00
Luca Deri 948a906037 Added -D flag for detecting DoH in the wild
Removed heuristic from CiscoVPN as it leads to false positives
2020-10-26 21:40:59 +01:00
Luca Deri 9b85669a64 Added -x for checking patterns 2020-10-22 23:28:18 +02:00
Alfredo Cardigliano e6d206fd15 Add unit tests to travis. Move ndpi serializer tests to unit tests. 2020-09-21 17:24:06 +02:00
Luca Deri 753b5dde16 Merge pull request #1012 from IvanNardi/ua
QUIC: extract User Agent information
2020-09-17 21:32:25 +02:00
Luca Deri 7086197047 Added extension to detect nested subdomains as used in Browsertunnel attack tool
https://github.com/veggiedefender/browsertunnel
2020-09-09 23:25:19 +02:00
Nardi Ivan a1014e8895 http: create a common function to parse User Agent field
Prepare the code to handle UA information from flows other than HTTP
2020-09-08 10:34:05 +02:00
Toni Uhlig adab0139c2 Fixed invalid memory access leading to a SIGSEGV in ndpiReader's option parser.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-28 06:53:58 +02:00