Commit Graph
145 Commits
Author SHA1 Message Date
Ivan Nardi 227e586e63 A (tiny) effort to reduce memory usage per flow (#1304)
Follow-up of 22241a1d
Only trivial changes:
* remove completely unused fields
* remove fields only written (but never read)
* CSGO protocol only handles UDP traffic
2021-09-18 09:29:45 +02:00
Luca Deri 00857abf2c Added new risk for clear text credentials 2021-09-10 22:00:04 +02:00
Ivan Nardi 8fdffbf3a1 Compile everything with "-W -Wall -Wno-unused-parameter" flags (#1276)
Fix all the warnings.

Getting rid of "-Wno-unused-parameter" is quite complex because some
parameters usage depends on compilation variable (i.e.
`--enable-debug-messages`).

The "-Werror" flag has been added only in Travis builds to avoid
breaking the builds to users using uncommon/untested
OS/compiler/enviroment.

Tested on:
* x86_64; Ubuntu 20.04; gcc 7,8,9,10,11; clang 7,8,9,10,11,12
* x86_64; CentOS 7.7; gcc 4.8.5 (with "--disable-gcrypt" flag)
* Raspberry 4; Debian 10.10; gcc 8.3.0
2021-08-20 18:11:13 +02:00
Toni 4e856a41d6 Skip whitespaces between HTTP method and URL. (#1271)
* be less case-restrictive, RFC2616 wants it that way

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-08-08 17:00:10 +02:00
Toni ad57af9f79 Improved RTSP detection and fixed HTTP false-positive. Fixes #1229. (#1266)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-31 23:31:49 +02:00
Luca Deri 61fc5be202 Reworked flow risk implementation 2021-07-23 17:27:15 +02:00
Vitaly Lavrov daf1fbfc7b Code cleanup. (#1246)
ndpi_utils.c: use ndpi_malloc,ndpi_calloc,ndpi_free
genshin_impact.c, git.c, hpvirtgrp.c, http.c, z3950.c: removed "#include stdlib.h"
2021-07-16 13:51:54 +02:00
Vitaly Lavrov 398ad8e5ff Fix for #1230 (#1235)
* Revert "Fix return value of ndpi_match_string_subprotocol() (#1230)"

This reverts commit 58665e93a9.

* Checking the return code after calling ndpi_match_string_subprotocol()

ndpi_api.h: Description of the returned error codes for
the ndpi_match_string_subprotocol() function.

If the ndpi_match_string_subprotocol() function returned an error,
then return NDPI_PROTOCOL_UNKNOWN.

http: The "Content-type" header is only checked if it is not empty.
2021-07-07 14:32:57 +02:00
Toni 34d0718bca Improved RTSP detection the second. (#1232) (#1233)
* RTSP is no subprotocol of HTTP (most of the time)
 * detection patterns should stay in rtsp.c
 * set detected HTTP protocol only if at least a valid HTTP method detected

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-07 09:55:36 +02:00
Toni b040407683 Refactored nDPI subprotocol handling and aimini protocol detection. (#1156)
* Refactored and merged callback buffer routines for non-udp-tcp / udp / tcp / tcp-wo-payload.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Try to detect one subprotocol if a detected protocol can have one.

 * This adds a performance overhead due to much more protocol detection routine calls.
   See #1148 for more information.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactor subprotocol handling (1/2).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactor subprotocol handling (2/2).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Prevent some code duplication by using macros for ndpi_int_one_line_struct string comparision.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactored aimini HTTP detection parts (somehow related to #1148).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Added aimini client/server test pcap.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Removed master protocol as it was only used for STUN and via also removed API function ndpi_get_protocol_id_master_proto

 * Adjusted Python code to conform to the changes made during the refactoring process.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-03-23 11:46:12 +01:00
Ivan Nardi 8074bc8201 HTTP: fix memory access in ndpi_http_parse_subprotocol() (#1151) 2021-03-09 19:46:32 +01:00
Luca Deri 3032864ec9 Added Ookla detection over IPv6 2021-03-09 12:55:14 +01:00
Luca Deri db716d0ab0 Ookla fixes 2021-03-09 11:42:23 +01:00
Luca Deri 477e4db650 Improved detection of Ookla speedtest and openspeedtest.com 2021-03-09 11:38:31 +01:00
Luca Deri 56bfb439f8 Improved DGA detection with trigrams. Disadvantage: slower startup time
Reworked Tor dissector embedded in TLS (fixes #1141)
Removed false positive on HTTP User-Agent
2021-03-03 00:41:07 +01:00
Ivan Nardi 8c0ea694f8 HTTP: fix user-agent parsing (#1124)
User-agent information is used to try to detect the user OS; since the
UA is extracted for QUIC traffic too, the "detected_os" field must be
generic and not associated to HTTP flows only.

Otherwise, you might overwrite some "tls_quic_stun" fields (SNI...) with
random data.

Strangely enough, the "detected_os" field is never used: it is never
logged, or printed, or exported...
2021-02-03 11:54:11 +01:00
Ivan Nardi 8cee718e8b HTTP: fix logs when NDPI_ENABLE_DEBUG_MESSAGES is defined (#1123) 2021-02-03 11:53:41 +01:00
Luca Deri 1f3ade5923 Increased number of extra packets that is necessary since the frgament mananger introduction 2021-02-03 11:51:55 +01:00
Luca Deri 68b6ac7da8 (C) Update 2021-01-07 11:13:36 +01:00
Ivan Nardi 1e2b57bed4 HTTP: fix compilation and a memory error when NDPI_ENABLE_DEBUG_MESSAGES is defined (#1109) 2021-01-07 10:58:02 +01:00
Luca Deri eb37f8f1fb Split HTTP request from response Content-Type. Request Content-Type should be present with POSTs and not with other methods such as GET 2021-01-06 18:28:24 +01:00
Luca Deri cd21f0d316 Added check for invalid HTTP content 2021-01-06 12:25:32 +01:00
Luca Deri a91e6179c5 Fixed missing symbol 2021-01-02 21:21:15 +01:00
Luca Deri 05d76525b0 Added HTTP suspicious content securirty risk (useful for tracking trickbot) 2021-01-02 21:11:42 +01:00
Luca Deri 84132c6735 Added known protocol on unknown port for ntop 2020-12-28 16:11:39 +01:00
Luca Deri df1b3367a9 Type change to avoid Windows compilation issues 2020-12-17 12:58:12 +01:00
Luca Deri 2768da0637 Improved HTTP dissection 2020-12-16 14:45:29 +01:00
Luca Deri b502bb08e0 Fix for detecting numeric IPs 2020-11-01 13:31:00 +01:00
Luca Deri 9dac9945c9 Fixes #1033 2020-10-21 20:59:02 +02:00
Luca Deri 5f99433ee7 Minor UA handling improvement to avoid heap-overflow 2020-09-22 08:56:35 +02:00
Luca Deri 753b5dde16 Merge pull request #1012 from IvanNardi/ua
QUIC: extract User Agent information
2020-09-17 21:32:25 +02:00
Luca Deri 7086197047 Added extension to detect nested subdomains as used in Browsertunnel attack tool
https://github.com/veggiedefender/browsertunnel
2020-09-09 23:25:19 +02:00
Nardi Ivan a1014e8895 http: create a common function to parse User Agent field
Prepare the code to handle UA information from flows other than HTTP
2020-09-08 10:34:05 +02:00
Luca Deri 647c7d59bb Improved ntop detection over HTTP
Added cap on number of attempts for CiscoVPN
2020-08-30 19:42:09 +02:00
Luca Deri dd75060932 Fixed false positive in suspicous user agent
Optimized stddev calculation
2020-08-30 12:25:15 +02:00
Simone Mainardi f0325f713a Passes method_len param to ndpi_http_str2method 2020-08-27 10:05:38 +02:00
Luca Deri 2e8aa9f08e Added ndpi_http_method ndpi_http_str2method(const char* method) API call 2020-08-26 23:17:19 +02:00
Luca Deri 00b2763399 Added check on payload lenght during extra packet processing 2020-08-04 21:59:45 +02:00
Luca Deri 879cec94b2 User agent detection improvements 2020-07-21 12:06:34 +02:00
Luca Deri db707e0829 Merge pull request #932 from IvanNardi/log
Log
2020-07-07 14:43:32 +02:00
Toni Uhlig 4b8c8608d1 Improved HTTP line parsing if request splitted into multiple packets.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-05 18:36:57 +02:00
Nardi Ivan c08693fda5 Incorporated some feedback 2020-07-01 20:16:16 +02:00
Luca Deri 8566288e43 Added malformed packet risk support 2020-06-26 22:37:52 +02:00
Nardi Ivan 56d87186f7 Fix compilation with --enable-debug-messages flag
NDPI_LOG* macros dereference ndpi_detection_module_struct object which is
private to ndpi library (via NDPI_LIB_COMPILATION define). So we can't use
them outside the library itself, i.e. in ndpiReader code
Therefore, in files in example/, convert all (rare) uses of NDPI_LOG* macros
to a new very simple macro, private to ndpiReader program. If necessary,
such macro may be improved.

According to a comment in ndpi_define.h, each dissector must define its own
NDPI_CURRENT_PROTO macro before including ndpi_api.h file
2020-06-26 12:04:02 +02:00
Luca Deri 46d96e7f32 Added checks for DGA detection 2020-06-17 19:46:37 +02:00
Luca Deri 55364ef0b4 Added DGA risk for names that look like a DGA 2020-06-11 18:51:53 +02:00
Luca Deri 29d9c2d2c8 Reworked HTTP header processing workflow 2020-06-01 09:54:05 +02:00
Luca Deri 669b7579ff HTTP comparison optimization 2020-05-31 19:31:01 +02:00
Luca Deri a4f8d913ea Minor changes 2020-05-31 19:11:12 +02:00
Luca Deri 374a24a1fb HTTP dissector improvements 2020-05-30 18:40:15 +02:00