Commit Graph
1017 Commits
Author SHA1 Message Date
Vitaly Lavrov be808c30f3 Code review. (#1205)
The common actions required to call the ac_automata_search() function
have been moved to the ndpi_match_string_common function. This made it
possible to simplify the ndpi_match_string, ndpi_match_string_protocol_id,
ndpi_match_string_value, ndpi_match_custom_category, ndpi_match_string_subprotocol,
ndpi_match_bigram, ndpi_match_trigram functions.

Using u_int16_t type for protocol identifiers when working with the
ahocorasick library (changes src/include/ndpi_api.h.in and src/include/ndpi_typedefs.h).

Reworked "finalization" of all AC_AUTOMATA_t structures.

Changing the order of fields in the ndpi_call_function_struct structure
reduces the size of the ndpi_detection_module_struct structure by 10 kB (for x86_64).
2021-06-15 11:28:09 +02:00
Luca Deri a79b8ee834 Reimplememnted SNMP dissector 2021-06-11 21:44:51 +02:00
Luca Deri 8761ac1887 Renamed Skyp in Skype_Teams as the protocol is now shared across these apps 2021-06-02 21:47:01 +02:00
Ivan Nardi f4dbeecba9 TLS: improve check for common ALPNs (#1191)
Facebook is still using its own ALPN for HTTP2 as well
2021-06-01 16:50:28 +02:00
Luca eb0115d4e0 Added TLS check to avoid crash with UDP-based traffic 2021-05-25 09:03:24 +02:00
Luca Deri d485c1cecb Improved TLS browser detection heuristics 2021-05-19 21:28:33 +02:00
Luca Deri ca15e3295e Added risk/score dump (ndpiReader -h)
Added ndpi_dump_risks_score() API score
2021-05-18 19:34:17 +02:00
Luca Deri 1ec621c85b Added further checks 2021-05-15 19:53:46 +02:00
Luca Deri abd6bce6f9 Added TLS certifiacate caching
Added Fortigate protocol
2021-05-15 10:52:16 +02:00
Luca Deri ac1eaca8a6 Added browser TLS heuristic 2021-05-13 20:00:27 +02:00
Luca Deri a62be9b8ec Implemented heuristic to detect Safari and Firefox TLS browsing 2021-05-13 12:37:07 +02:00
Toni 5918a6542d Improved SSL certificate name wildcard handling and risk. #1182 (#1183)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-11 21:38:26 +02:00
Luca Deri fff60ec317 Added check to reduce MongoDB false positive detection 2021-05-10 21:14:55 +02:00
Ivan Nardi 2ca37ff8be TLS: fix extraction for TLS signature algorithms (#1180)
```
==69562==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6100009000fb at pc 0x7f41882003a7 bp 0x7f4183cfbfc0 sp 0x7f4183cfb768
READ of size 32 at 0x6100009000fb thread T1
    #0 0x7f41882003a6 in __interceptor_memcpy ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827
    #1 0x560b2d7462a1 in processClientServerHello protocols/tls.c:1647
    #2 0x560b2d73be6a in processTLSBlock protocols/tls.c:712
    #3 0x560b2d73e61f in ndpi_search_tls_udp protocols/tls.c:968
```
2021-05-09 15:10:41 +02:00
Ivan Nardi 12c6ab768d TLS: fix another use-of-uninitialized-value error in ClientHello parsing (#1179)
Error detected with valgrind.

==13127== Conditional jump or move depends on uninitialised value(s)
==13127==    at 0x483EF58: strlen (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==13127==    by 0x1A93B6: ndpi_strdup (ndpi_main.c:159)
==13127==    by 0x1C07CC: processClientServerHello (tls.c:1678)
==13127==    by 0x1C0C4C: processTLSBlock (tls.c:712)
==13127==    by 0x1C0C4C: ndpi_search_tls_tcp.part.0 (tls.c:849)

See also 8c3674e9
2021-05-09 15:10:14 +02:00
Luca ae2470fad4 Initial work towards detection via TLS of browser types 2021-05-06 21:42:06 +02:00
Luca 808050e635 Add extraction for TLS signature algorithms 2021-05-06 08:56:01 +02:00
Luca Deri 153d11c920 Enhanced netbios decoding check 2021-04-28 22:58:22 +02:00
Toni da3e6bd61b Check for common ALPNs and set a flow risk if not known. (#1175)
* Increased risk bitmask to 64bit (instead of 32bit).
 * Removed annoying "Unknown datalink" error message for fuzzers.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-27 07:22:04 +02:00
Luca Deri d5b395f35a Fixed invalid DNS dissection 2021-04-26 23:08:00 +02:00
Toni b4a8c8c63f Add Genshin Impact protocol. (#1173)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-25 10:02:07 +02:00
Toni 9377991263 Add HP Virtual Machine Group Management (hpvirtgrp) protocol. (#1170)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-20 14:12:16 +02:00
Ivan Nardi 8c3674e9a3 TLS: fix some use-of-uninitialized-value errors in ClientHello parsing (#1169)
Error detected with valgrind.

==125883== Conditional jump or move depends on uninitialised value(s)
==125883==    at 0x438F57: processClientServerHello (tls.c:1421)
==125883==    by 0x43B35A: processTLSBlock (tls.c:712)
==125883==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125883==    by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)
==125883==    by 0x42E920: ndpi_detection_process_packet (ndpi_main.c:5301)

==125916== Conditional jump or move depends on uninitialised value(s)
==125916==    at 0x438D7D: processClientServerHello (tls.c:1379)
==125916==    by 0x43B35A: processTLSBlock (tls.c:712)
==125916==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125916==    by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)

==125932== Conditional jump or move depends on uninitialised value(s)
==125932==    at 0x438C1D: processClientServerHello (tls.c:1298)
==125932==    by 0x43B35A: processTLSBlock (tls.c:712)
==125932==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125932==    by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)

==125950== Conditional jump or move depends on uninitialised value(s)
==125950==    at 0x438D4F: processClientServerHello (tls.c:1371)
==125950==    by 0x43B35A: processTLSBlock (tls.c:712)
==125950==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125950==    by 0x42C079: check_ndpi_detection_func (ndpi_main.c:4443)
2021-04-18 21:38:01 +02:00
Luca Deri 18c6c1c2d6 Added NDPI_DESKTOP_OR_FILE_SHARING_SESSION risk to remote protocols for remote assistance sessions 2021-04-12 18:11:14 +02:00
Luca Deri bf318e0b86 Added NDPI_DESKTOP_OR_FILE_SHARING_SESSION flow risk 2021-04-11 14:42:27 +02:00
Ivan Nardi 0bfe444362 GTP: fix parsing of GTP headers (#1161)
Message length checks and basic headers are not uniform across GTP-U,
GTP-C and GTP-PRIME.

Note that, even if the length checks were wrong, the GTP sessions were almost
always correctly classified because of the "guessing" algorithm.

This patch has been tested with GTP-U, GTP-C-V1, GTP-C-V2 and GPT-PRIME-V2
traffic using ndpiReader with "-d" flag (to avoid "guessing" algorithm) and
without "-t" flag (to avoid GTP-U de-tunneling).

See #1148
2021-04-05 19:23:03 +02:00
Ivan Nardi 1290c40968 Mining: lru cache is ipv4 only (for the time being) (#1159)
Fix memory error with ipv6 traffic
2021-03-31 14:13:07 +02:00
Luca Deri c1d6e3f145 Improved mining detection support 2021-03-30 17:50:19 +02:00
Toni b040407683 Refactored nDPI subprotocol handling and aimini protocol detection. (#1156)
* Refactored and merged callback buffer routines for non-udp-tcp / udp / tcp / tcp-wo-payload.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Try to detect one subprotocol if a detected protocol can have one.

 * This adds a performance overhead due to much more protocol detection routine calls.
   See #1148 for more information.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactor subprotocol handling (1/2).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactor subprotocol handling (2/2).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Prevent some code duplication by using macros for ndpi_int_one_line_struct string comparision.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactored aimini HTTP detection parts (somehow related to #1148).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Added aimini client/server test pcap.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Removed master protocol as it was only used for STUN and via also removed API function ndpi_get_protocol_id_master_proto

 * Adjusted Python code to conform to the changes made during the refactoring process.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-03-23 11:46:12 +01:00
Luca Deri 0cae9bf4a4 Win compilation fix 2021-03-22 21:41:28 +01:00
Luca Deri 627299e4dd Better DGA detection (slightly decreased accuracy) 2021-03-20 17:56:24 +01:00
Luca Deri 6333bb1702 Removed duplicate extesions len 2021-03-19 22:35:08 +01:00
Luca Deri d96f4ca736 Added ALPN and elliptic curve in JA3S+ 2021-03-19 22:33:04 +01:00
Luca Deri fa238bbe42 Implemented JA3+ also for JA3S 2021-03-19 22:27:36 +01:00
Luca Deri e2c3445a20 Reworked JA3 2021-03-19 22:20:52 +01:00
Luca Deri 61f7ec1696 JA3 debug improvements 2021-03-19 19:10:31 +01:00
Alfredo Cardigliano efc5630378 Fix compilation warning 2021-03-12 09:25:00 +01:00
Luca Deri 5b7fe1360a Fixed JA3+ computation 2021-03-11 00:04:12 +01:00
Luca Deri f6ad16d8f8 Added experiemntal JA3+ implementation that can be used with -z i ndpiReader 2021-03-09 23:38:29 +01:00
Ivan Nardi 8074bc8201 HTTP: fix memory access in ndpi_http_parse_subprotocol() (#1151) 2021-03-09 19:46:32 +01:00
Luca Deri 574fc4f09e Ookla detection improvement 2021-03-09 17:43:12 +01:00
Luca Deri 3032864ec9 Added Ookla detection over IPv6 2021-03-09 12:55:14 +01:00
Luca Deri db716d0ab0 Ookla fixes 2021-03-09 11:42:23 +01:00
Luca Deri 477e4db650 Improved detection of Ookla speedtest and openspeedtest.com 2021-03-09 11:38:31 +01:00
Ivan Nardi c50a8d4808 Add support for Snapchat voip calls (#1147)
* Add support for Snapchat voip calls

Snapchat multiplexes some of its audio/video real time traffic with QUIC
sessions. The peculiarity of these sessions is that they are Q046 and
don't have any SNI.

* Fix tests with libgcrypt disabled
2021-03-06 05:48:36 +01:00
Luca Deri 56bfb439f8 Improved DGA detection with trigrams. Disadvantage: slower startup time
Reworked Tor dissector embedded in TLS (fixes #1141)
Removed false positive on HTTP User-Agent
2021-03-03 00:41:07 +01:00
Ivan Nardi 4c00ff89df DTLS: improve support (#1146)
* DTLS: add some pcap tests

* DTLS: fix parsing of Client/Server Helllo message

* DTLS: add parsing of server certificates
2021-03-02 21:15:40 +01:00
Toni 16890a6632 Added NDPI_MALICIOUS_SHA1 flow risk. (#1142)
* An external file which contains known malicious SSL certificate SHA-1 hashes
   can be loaded via ndpi_load_malicious_sha1_file(...)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-26 17:00:05 +01:00
Luca Deri fba61adf5e Improved DNS dissector 2021-02-26 00:42:15 +01:00
Luca Deri f1b22b199f Added NDPI_MALICIOUS_JA3 flow risk
Added ndpi_load_malicious_ja3_file() API call
2021-02-22 23:19:23 +01:00