Commit Graph
553 Commits
Author SHA1 Message Date
Toni 8d0c7b1fae Fixed Mingw64 build, SonerCloud-CI and more. (#1273)
* Added ARM build and unit test run for SonarCloud-CI.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Fixed Mingw64 build.

 * adapted to SonarCloud-CI workflow
 * removed broken and incomplete Windows example (tested on VS2017/VS2019)
 * removed unnecessary include (e.g. pthread.h for the library which does not make use of it)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-08-18 11:34:16 +02:00
Luca Deri 821f4c9249 Fixed some invalid TLS guesses 2021-08-17 19:04:07 +02:00
Luca Deri 5c33fbf19b Added extraction of hostname in SMTP
Fixed mail incalid subprotocol calculation
2021-08-11 11:52:24 +02:00
Luca Deri 0df8aa19ec Clode cleanup (after last merge) 2021-08-08 17:10:08 +02:00
ToniandLuca Deri 259b60f574 Added entropy calculation to check for suspicious (encrypted) payload. (#1270)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2021-08-08 17:03:23 +02:00
Toni 4e856a41d6 Skip whitespaces between HTTP method and URL. (#1271)
* be less case-restrictive, RFC2616 wants it that way

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-08-08 17:00:10 +02:00
Luca Deri 3706267d68 Updated test output 2021-08-07 19:43:55 +02:00
Luca Deri 0522e562e8 Added testing pcap for TLS fatal alert 2021-08-07 19:42:53 +02:00
Luca Deri e8455236bd Updated output 2021-08-07 17:38:33 +02:00
Luca Deri 6b7e5fa8d2 Added legacy whatsapp rules used to match old pcaps that need to be removed in the future 2021-07-31 23:49:48 +02:00
Toni ad57af9f79 Improved RTSP detection and fixed HTTP false-positive. Fixes #1229. (#1266)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-31 23:31:49 +02:00
Toni 13c5d6801e Improved libm check via Autoconf. (#1263)
* libm will now be part of @ADDITIONAL_LIBS@ (if required)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-28 22:24:03 +02:00
Toni 29ec34f66d Improved TFTP detection. Fixes #1242, #1256 (#1262)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-25 21:54:19 +02:00
Luca Deri 61fc5be202 Reworked flow risk implementation 2021-07-23 17:27:15 +02:00
Ivan Nardi 4ffe1eb3c0 Fix unit tests when "--disable-gcrypt" flag is used (#1255) 2021-07-23 10:37:36 +02:00
Toni 32275543c4 Added risk: TLS_EXTENSION_SUSPICIOUS (#1252)
* validates client/server hello TLS extensions
 * inspects content for some extensions

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-19 16:23:24 +02:00
Luca Deri 0adcd28450 Updated download category name 2021-07-16 14:04:47 +02:00
Luca Deri b95bd0358f Code cleanup 2021-07-14 11:19:24 +02:00
pacantandpacant 19a29e1e22 TLS Risks - Certificate Validity Too Long (#1239)
* Added flow risk: TLS certificate too long

* Added flow risk: TLS certificate too long

* Date for TLS limit added

* TLS certificate check fixed

Co-authored-by: pacant <a.pace97@outlook.com>
2021-07-14 11:13:22 +02:00
Toni c411df523e Added AVAST SecureDNS protocol. (#1244)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-14 11:11:59 +02:00
Toni 62bae30a4a Improved Steam detection (Steam Datagram Relay - SDR). (#1243)
* improved DNSCrypt midstream detection again (sufficient for all tested use-cases)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-14 11:09:58 +02:00
Toni e4453938d5 Improved dnscrypt midstream detection. (#1241)
* fixed skype false-positive detection of dnscrypt traffic

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-13 15:10:18 +02:00
Ivan Nardi cccf794265 ndpiReader: add statistics about nDPI performance (#1240)
The goal is to have a (roughly) idea about how many packets nDPI needs
to properly classify a flow.

Log this information (and guessed flows number too) during unit tests,
to keep track of improvements/regressions across commits.
2021-07-13 12:28:39 +02:00
Luca Deri 96b71def49 Minor change 2021-07-12 17:51:25 +02:00
Vitaly Lavrov c418b7110b ahoсorasick. Code review. Part 2. (#1236)
Simplified the process of adding lines to AC_AUTOMATA_t.
Use the ndpi_string_to_automa() function to add patterns with domain names.
For other cases can use ndpi_add_string_value_to_automa().

ac_automata_feature(ac_automa, AC_FEATURE_LC) allows adding
and compare data in a case insensitive manner. For mandatory pattern comparison
from the end of the line, the "ac_pattern.rep.at_end=1" flag is used.
This eliminated unnecessary conversions to lowercase and adding "$" for
end-of-line matching in domain name patterns.

ac_match_handler() has been renamed ac_domain_match_handler() and has been greatly simplified.
ac_domain_match_handler() looks for the template with the highest domain level.
For special cases it is possible to manually specify the domain level.
Added test for checking ambiguous domain names like:
 - short.weixin.qq.com is QQ, not Wechat
 - instagram.faae1-1.fna.fbcdn.net is Instagram, not Facebook

If you specify a NULL handler when creating the AC_AUTOMATA_t structure,
then a pattern with the maximum length that satisfies the search conditions will be found
(exact match, from the beginning of the string, from the end of the string, or a substring).

Added debugging for ac_automata_search.
To do this, you need to enable debugging globally using ac_automata_enable_debug(1) and
enable debugging in the AC_AUTOMATA_t structure using ac_automata_name("name", AC_FEATURE_DEBUG).
The search will display "name" and a list of matching patterns.
Running "AHO_DEBUG=1 ndpiReader ..." will show the lines that were searched for templates
and which templates were found.

The ac_automata_dump() prototype has been changed. Now it outputs data to a file.
If it is specified as NULL, then the output will be directed to stdout.
If you need to get data as a string, then use open_memstream().

Added the ability to run individual tests via the do.sh script
2021-07-12 17:39:43 +02:00
Luca Deri bec8361638 Updated category labels 2021-07-08 20:05:50 +02:00
Toni 34d0718bca Improved RTSP detection the second. (#1232) (#1233)
* RTSP is no subprotocol of HTTP (most of the time)
 * detection patterns should stay in rtsp.c
 * set detected HTTP protocol only if at least a valid HTTP method detected

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-07 09:55:36 +02:00
Luca Deri c408710c52 Added test pcap 2021-07-06 18:57:52 +02:00
Toni 6f29733262 Improved RTSP via HTTP detection. (#1232)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-06 18:57:04 +02:00
Ivan Nardi 1116d0e4b1 Improve content match lists (#1226)
Add support for Likee app (https://likee.video/) and GitLab
Improve detection of Google domains ("gtv1.com" added in 6dd42d19f was
likely a typo)
Improve Snapchat and Tiktok detection
2021-07-05 09:42:11 +02:00
Toni 21b2cce715 Improved Z39.50 detection. (#1225)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-05 09:41:35 +02:00
Toni 1c2a0c36f1 Added Z39.50 protocol. (#1219)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-06-29 15:32:16 +02:00
Ivan Nardi f8fe3ee520 QUIC: add basic support for fragmented Client Hello (#1216)
Only in-order and non overlapping fragments are handled
See #1195
2021-06-24 18:30:34 +02:00
Luca Deri f19937c8c9 Addesses partial application matching (#1207) 2021-06-21 21:59:15 +02:00
Luca 41ec807d79 New testing pcap with syn scan attack 2021-06-08 10:39:41 +02:00
Luca Deri b241dd5e6b Fixed expected output 2021-06-03 09:06:01 +02:00
Luca Deri 8761ac1887 Renamed Skyp in Skype_Teams as the protocol is now shared across these apps 2021-06-02 21:47:01 +02:00
Luca Deri d485c1cecb Improved TLS browser detection heuristics 2021-05-19 21:28:33 +02:00
Luca Deri 732bcecd17 Added flow risk score 2021-05-18 21:05:47 +02:00
Luca Deri 43a8576efb Reworked human readeable string search in flows
Removed fragment manager code
2021-05-17 20:55:06 +02:00
Luca Deri 9c5ba9c060 Updated protocol category 2021-05-15 11:20:15 +02:00
Luca Deri abd6bce6f9 Added TLS certifiacate caching
Added Fortigate protocol
2021-05-15 10:52:16 +02:00
Luca Deri ca92660d98 Converted some test .pcapng files to pcap format 2021-05-13 20:51:11 +02:00
Luca Deri ac1eaca8a6 Added browser TLS heuristic 2021-05-13 20:00:27 +02:00
Toni 5918a6542d Improved SSL certificate name wildcard handling and risk. #1182 (#1183)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-11 21:38:26 +02:00
Ivan Nardi 12c6ab768d TLS: fix another use-of-uninitialized-value error in ClientHello parsing (#1179)
Error detected with valgrind.

==13127== Conditional jump or move depends on uninitialised value(s)
==13127==    at 0x483EF58: strlen (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==13127==    by 0x1A93B6: ndpi_strdup (ndpi_main.c:159)
==13127==    by 0x1C07CC: processClientServerHello (tls.c:1678)
==13127==    by 0x1C0C4C: processTLSBlock (tls.c:712)
==13127==    by 0x1C0C4C: ndpi_search_tls_tcp.part.0 (tls.c:849)

See also 8c3674e9
2021-05-09 15:10:14 +02:00
Luca Deri d5b395f35a Fixed invalid DNS dissection 2021-04-26 23:08:00 +02:00
Luca Deri 0ef7998348 Removed protocol space in Genshin Impact 2021-04-25 10:06:22 +02:00
Toni b4a8c8c63f Add Genshin Impact protocol. (#1173)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-25 10:02:07 +02:00
Toni 9377991263 Add HP Virtual Machine Group Management (hpvirtgrp) protocol. (#1170)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-20 14:12:16 +02:00