Commit Graph
275 Commits
Author SHA1 Message Date
Ivan Nardi 6325aebda6 TLS: avoid zeroing large structures (#1300)
Zeroing large structures (i.e. size > KB) is quite costly (from a CPU point
of view): we can safely avoid doing that for a couple of big structures.

Standard and Valgrind tests have been diverging quite a lot: it is time
to re-sync them. Use the same script and enable Valgrind via an
enviroment variable:

  NDPI_TESTS_VALGRIND=1 ./tests/do.sh
2021-09-16 14:34:59 +02:00
Ivan Nardi 0eb7a0388c Hangout: fix usage of STUN function (#1299) 2021-09-13 18:25:43 +02:00
Ivan Nardi 9e0f0ce3df Fix access to some TLS fields in flow structure (#1277)
Fields 'tls.hello_processed` and `tls.subprotocol_detected` are used by
QUIC (i.e UDP...), too.
2021-08-20 18:11:37 +02:00
Ivan Nardi 8fdffbf3a1 Compile everything with "-W -Wall -Wno-unused-parameter" flags (#1276)
Fix all the warnings.

Getting rid of "-Wno-unused-parameter" is quite complex because some
parameters usage depends on compilation variable (i.e.
`--enable-debug-messages`).

The "-Werror" flag has been added only in Travis builds to avoid
breaking the builds to users using uncommon/untested
OS/compiler/enviroment.

Tested on:
* x86_64; Ubuntu 20.04; gcc 7,8,9,10,11; clang 7,8,9,10,11,12
* x86_64; CentOS 7.7; gcc 4.8.5 (with "--disable-gcrypt" flag)
* Raspberry 4; Debian 10.10; gcc 8.3.0
2021-08-20 18:11:13 +02:00
Luca Deri 821f4c9249 Fixed some invalid TLS guesses 2021-08-17 19:04:07 +02:00
Luca Deri 4183718952 Added TLS fatal alert flow risk 2021-08-07 19:40:44 +02:00
Luca Deri 51e4888442 Implemented ALPN automa for checking uncommon ALPNs 2021-07-24 17:50:32 +02:00
Luca Deri 61fc5be202 Reworked flow risk implementation 2021-07-23 17:27:15 +02:00
Toni 6ad0d6666c Implemented function to retrieve flow information. #1253 (#1254)
* fixed [h]euristic typo

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-23 10:37:20 +02:00
Toni 32275543c4 Added risk: TLS_EXTENSION_SUSPICIOUS (#1252)
* validates client/server hello TLS extensions
 * inspects content for some extensions

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-19 16:23:24 +02:00
pacantandpacant 488166833e Fixed TLS certificate threshold (#1248)
Co-authored-by: pacant <a.pace97@outlook.com>
2021-07-16 13:52:55 +02:00
Luca Deri b95bd0358f Code cleanup 2021-07-14 11:19:24 +02:00
pacantandpacant 19a29e1e22 TLS Risks - Certificate Validity Too Long (#1239)
* Added flow risk: TLS certificate too long

* Added flow risk: TLS certificate too long

* Date for TLS limit added

* TLS certificate check fixed

Co-authored-by: pacant <a.pace97@outlook.com>
2021-07-14 11:13:22 +02:00
Ivan Nardi 46c32d2433 TLS: fix a memory error in JA3 code (#1227)
protocols/tls.c:1856:5: runtime error: index 256 out of bounds for type 'char [256]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:1856:5
2021-07-05 09:42:52 +02:00
Toni ced6fca184 Fixed off-by-one memory error for TLS-JA3. (#1222)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-06-29 15:33:12 +02:00
Luca Deri 0cbb4b2ffa Restored 32 bit value in ndpi_match_string_value() 2021-06-21 17:06:22 +02:00
Vitaly Lavrov be808c30f3 Code review. (#1205)
The common actions required to call the ac_automata_search() function
have been moved to the ndpi_match_string_common function. This made it
possible to simplify the ndpi_match_string, ndpi_match_string_protocol_id,
ndpi_match_string_value, ndpi_match_custom_category, ndpi_match_string_subprotocol,
ndpi_match_bigram, ndpi_match_trigram functions.

Using u_int16_t type for protocol identifiers when working with the
ahocorasick library (changes src/include/ndpi_api.h.in and src/include/ndpi_typedefs.h).

Reworked "finalization" of all AC_AUTOMATA_t structures.

Changing the order of fields in the ndpi_call_function_struct structure
reduces the size of the ndpi_detection_module_struct structure by 10 kB (for x86_64).
2021-06-15 11:28:09 +02:00
Ivan Nardi f4dbeecba9 TLS: improve check for common ALPNs (#1191)
Facebook is still using its own ALPN for HTTP2 as well
2021-06-01 16:50:28 +02:00
Luca eb0115d4e0 Added TLS check to avoid crash with UDP-based traffic 2021-05-25 09:03:24 +02:00
Luca Deri d485c1cecb Improved TLS browser detection heuristics 2021-05-19 21:28:33 +02:00
Luca Deri ca15e3295e Added risk/score dump (ndpiReader -h)
Added ndpi_dump_risks_score() API score
2021-05-18 19:34:17 +02:00
Luca Deri 1ec621c85b Added further checks 2021-05-15 19:53:46 +02:00
Luca Deri abd6bce6f9 Added TLS certifiacate caching
Added Fortigate protocol
2021-05-15 10:52:16 +02:00
Luca Deri ac1eaca8a6 Added browser TLS heuristic 2021-05-13 20:00:27 +02:00
Luca Deri a62be9b8ec Implemented heuristic to detect Safari and Firefox TLS browsing 2021-05-13 12:37:07 +02:00
Toni 5918a6542d Improved SSL certificate name wildcard handling and risk. #1182 (#1183)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-11 21:38:26 +02:00
Ivan Nardi 2ca37ff8be TLS: fix extraction for TLS signature algorithms (#1180)
```
==69562==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6100009000fb at pc 0x7f41882003a7 bp 0x7f4183cfbfc0 sp 0x7f4183cfb768
READ of size 32 at 0x6100009000fb thread T1
    #0 0x7f41882003a6 in __interceptor_memcpy ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827
    #1 0x560b2d7462a1 in processClientServerHello protocols/tls.c:1647
    #2 0x560b2d73be6a in processTLSBlock protocols/tls.c:712
    #3 0x560b2d73e61f in ndpi_search_tls_udp protocols/tls.c:968
```
2021-05-09 15:10:41 +02:00
Ivan Nardi 12c6ab768d TLS: fix another use-of-uninitialized-value error in ClientHello parsing (#1179)
Error detected with valgrind.

==13127== Conditional jump or move depends on uninitialised value(s)
==13127==    at 0x483EF58: strlen (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==13127==    by 0x1A93B6: ndpi_strdup (ndpi_main.c:159)
==13127==    by 0x1C07CC: processClientServerHello (tls.c:1678)
==13127==    by 0x1C0C4C: processTLSBlock (tls.c:712)
==13127==    by 0x1C0C4C: ndpi_search_tls_tcp.part.0 (tls.c:849)

See also 8c3674e9
2021-05-09 15:10:14 +02:00
Luca ae2470fad4 Initial work towards detection via TLS of browser types 2021-05-06 21:42:06 +02:00
Luca 808050e635 Add extraction for TLS signature algorithms 2021-05-06 08:56:01 +02:00
Toni da3e6bd61b Check for common ALPNs and set a flow risk if not known. (#1175)
* Increased risk bitmask to 64bit (instead of 32bit).
 * Removed annoying "Unknown datalink" error message for fuzzers.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-27 07:22:04 +02:00
Ivan Nardi 8c3674e9a3 TLS: fix some use-of-uninitialized-value errors in ClientHello parsing (#1169)
Error detected with valgrind.

==125883== Conditional jump or move depends on uninitialised value(s)
==125883==    at 0x438F57: processClientServerHello (tls.c:1421)
==125883==    by 0x43B35A: processTLSBlock (tls.c:712)
==125883==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125883==    by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)
==125883==    by 0x42E920: ndpi_detection_process_packet (ndpi_main.c:5301)

==125916== Conditional jump or move depends on uninitialised value(s)
==125916==    at 0x438D7D: processClientServerHello (tls.c:1379)
==125916==    by 0x43B35A: processTLSBlock (tls.c:712)
==125916==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125916==    by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)

==125932== Conditional jump or move depends on uninitialised value(s)
==125932==    at 0x438C1D: processClientServerHello (tls.c:1298)
==125932==    by 0x43B35A: processTLSBlock (tls.c:712)
==125932==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125932==    by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)

==125950== Conditional jump or move depends on uninitialised value(s)
==125950==    at 0x438D4F: processClientServerHello (tls.c:1371)
==125950==    by 0x43B35A: processTLSBlock (tls.c:712)
==125950==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125950==    by 0x42C079: check_ndpi_detection_func (ndpi_main.c:4443)
2021-04-18 21:38:01 +02:00
Luca Deri 18c6c1c2d6 Added NDPI_DESKTOP_OR_FILE_SHARING_SESSION risk to remote protocols for remote assistance sessions 2021-04-12 18:11:14 +02:00
Luca Deri bf318e0b86 Added NDPI_DESKTOP_OR_FILE_SHARING_SESSION flow risk 2021-04-11 14:42:27 +02:00
Luca Deri 0cae9bf4a4 Win compilation fix 2021-03-22 21:41:28 +01:00
Luca Deri 627299e4dd Better DGA detection (slightly decreased accuracy) 2021-03-20 17:56:24 +01:00
Luca Deri 6333bb1702 Removed duplicate extesions len 2021-03-19 22:35:08 +01:00
Luca Deri d96f4ca736 Added ALPN and elliptic curve in JA3S+ 2021-03-19 22:33:04 +01:00
Luca Deri fa238bbe42 Implemented JA3+ also for JA3S 2021-03-19 22:27:36 +01:00
Luca Deri e2c3445a20 Reworked JA3 2021-03-19 22:20:52 +01:00
Luca Deri 61f7ec1696 JA3 debug improvements 2021-03-19 19:10:31 +01:00
Alfredo Cardigliano efc5630378 Fix compilation warning 2021-03-12 09:25:00 +01:00
Luca Deri 5b7fe1360a Fixed JA3+ computation 2021-03-11 00:04:12 +01:00
Luca Deri f6ad16d8f8 Added experiemntal JA3+ implementation that can be used with -z i ndpiReader 2021-03-09 23:38:29 +01:00
Luca Deri 56bfb439f8 Improved DGA detection with trigrams. Disadvantage: slower startup time
Reworked Tor dissector embedded in TLS (fixes #1141)
Removed false positive on HTTP User-Agent
2021-03-03 00:41:07 +01:00
Ivan Nardi 4c00ff89df DTLS: improve support (#1146)
* DTLS: add some pcap tests

* DTLS: fix parsing of Client/Server Helllo message

* DTLS: add parsing of server certificates
2021-03-02 21:15:40 +01:00
Toni 16890a6632 Added NDPI_MALICIOUS_SHA1 flow risk. (#1142)
* An external file which contains known malicious SSL certificate SHA-1 hashes
   can be loaded via ndpi_load_malicious_sha1_file(...)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-26 17:00:05 +01:00
Luca Deri f1b22b199f Added NDPI_MALICIOUS_JA3 flow risk
Added ndpi_load_malicious_ja3_file() API call
2021-02-22 23:19:23 +01:00
Luca Deri fc3db8f169 Implemented TLS Certificate Sibject matching
Improved AnyDesk detection
2021-02-22 22:37:33 +01:00
Luca Deri 54636a3213 Improved (partial) TLS dissection 2021-02-04 22:06:18 +01:00