Commit Graph
4471 Commits
Author SHA1 Message Date
Nardi Ivan b22fa558ff ndpiReader: fix memory leak
Change the working directory of `ndpiReader` in the Github Actions so
that it can load the domain suffix list during `domainsUnitTest()`
2024-01-15 19:49:27 +01:00
Luca d29c2f3ad1 Added ndpi tracing 2024-01-15 19:26:28 +01:00
Luca a1634604e1 Removes extraneous parentheses that caused macOS to complain 2024-01-15 19:12:31 +01:00
Luca 162c38f18f Added new API calls
- ndpi_load_domain_suffixes()
- ndpi_get_host_domain_suffix()

whose goal is to find the domain name of a hostname. Example:

www.bbc.co.uk   -> co.uk
mail.apple.com  -> com
2024-01-15 19:03:46 +01:00
Ivan Nardi 61a18623e7 Bittorrent: improve detection of UTPv1 (#2259)
* fix header length check: we must take into account optional extension
header
* it seems the check on UTPv1 header is weak only in the DATA case: wait
for multiple packets only in that case
2024-01-12 18:53:24 +01:00
Ivan Nardi dd8be1fcb1 Fix some warnings reported by CODESonar (#2227)
Remove some unreached/duplicated code.

Add error checking for `atoi()` calls.

About `isdigit()` and similar functions. The warning reported is:
```
Negative Character Value help
isdigit() is invoked here with an argument of signed type char, but only
has defined behavior for int arguments that are either representable
as unsigned char or equal to the value of macro EOF(-1).
Casting the argument to unsigned char will avoid the undefined behavior.
In a number of libc implementations, isdigit() is implemented using lookup
tables (arrays): passing in a negative value can result in a read underrun.
```
Switching to our macros fix that.
Add a check to `check_symbols.sh` to avoid using the original functions
from libc.
2024-01-12 13:30:43 +01:00
Toni 0aea509e23 Add KCP protocol dissector. (#2257)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-12 12:14:59 +01:00
Vladimir Gavrilov 75d3f78f7d Add more tracker/ad hosts (#2256) 2024-01-12 10:44:31 +01:00
Toni 7b2bbb2309 Improved uTorrent via utp (TCP-like streams over UDP). (#2255)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-11 16:49:25 +01:00
Luca Deri 30c2386ee2 Added missing (C) 2024-01-09 22:27:41 +01:00
Luca Deri 4f9a08cf05 Moved ndpi_private.h to include directory 2024-01-09 22:00:40 +01:00
Vladimir Gavrilov 3d57dec6b4 Add PIA (Private Internet Access) support (#2250)
* Fix typo in protocols.rst

* Add PIA (Private Internet Access) support

* Update ndpi_main.c
2024-01-09 18:28:57 +01:00
Vladimir Gavrilov ead5bf2adb Add more adult content hostnames (#2247) 2024-01-09 07:57:01 +01:00
Toni c5b0b05b80 Add Roughtime protocol dissector. (#2248)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-09 07:56:27 +01:00
Toni 6c3d162cd6 Add realtime protocol output to ndpiReader. (#2197)
* support for using a new flow callback invoked before the flow memory is free'd
 * minor fixes

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-09 00:39:59 +01:00
Ivan Nardi 90b999e3f1 Fix default port conflicts between AJP and CiscoVPN (#2245)
We shouldn't have the same default port associated with multiple
protocols. The key reason is that we use only one of them (for
classification by-port, for example) and which one we really use
depends on the internal order of the protocolsi's initialization (which
is unknown to the user).

We have 3 port collisions:
* 8009: AJP and CiscoVPN
* 445: SMBv1 and SMBv23
* 8080: HTTP_Connect and HTTP_Proxy
I think that the two last cases don't really matter...

About the first one: it seems that CiscoVPN is retired from 2011/2014
(see
https://www.cisco.com/c/en/us/obsolete/security/cisco-vpn-client.html)
and superseded by AnyConnect, which uses different ports (see
https://community.cisco.com/t5/network-security/what-protocols-are-used-on-anyconnect-mobility-client/td-p/4713525).
Therefore it should be safe to remove 8009 from the CiscoVPN default
ports.
2024-01-08 17:04:46 +01:00
Ivan Nardi 8321b79539 Make some test traces smaller (#2243)
Having smaller traces help fuzzing: we want the fuzzers to mutate
"interesting" packets analyzed by nDPI, i.e. the first packets of each
flows.
Try hard to keep the same classification and extraction capabilities
2024-01-08 17:04:12 +01:00
Vladimir Gavrilov ce08291ccd Add Google Chat support (#2244) 2024-01-07 18:04:59 +01:00
Ivan Nardi 40797521af ndpiReader: add breed stats on output used for CI (#2236) 2024-01-05 13:02:39 +01:00
Vladimir Gavrilov 3d09b25653 Add Ceph protocol dissector (#2242)
* Add Ceph protocol dissector

* Update protocols.rst
2024-01-04 13:22:23 +01:00
Ivan Nardi f23e9dc7bb Add an implementation of the BSD function strtonum (#2238)
The main difference with the original function is that we allow to
specify the base.
Credit for the original idea and the first implementation to @0xA50C1A1
2024-01-04 13:16:39 +01:00
Vladimir Gavrilov 7f9973bd0c Add HL7 protocol dissector (#2240)
* Add HL7 protocol dissector

* Small fixes

* Small fixes
2024-01-02 20:57:05 +01:00
Ivan Nardi 88d1416b70 STUN: fix detection of Google Meet over IPv6 (#2241) 2024-01-02 19:30:59 +01:00
Vladimir GavrilovandNardi Ivan 0180c1f04a Add IEC62056 (DLMS/COSEM) protocol dissector (#2229)
* Add IEC62056 (DLMS/COSEM) protocol dissector

* Fix detection on big endian architectures

* Update protocols.rst

* Add ndpi_crc16_x25 to fuzz/fuzz_alg_crc32_md5.c

* Update pcap sample

* Remove empty .out file

* iec62056: add some documentation

---------

Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2024-01-02 16:45:54 +01:00
Ivan Nardi 3c7ed34ce9 fuzz: improve fuzzing coverage (#2239) 2024-01-02 15:22:44 +01:00
Vladimir Gavrilov 0f4d9f5054 Remove Google Hangouts/Duo stuff (#2233)
* Remove Google Hangouts/Duo support

* Update protocols.rst
2024-01-02 14:01:33 +01:00
Ivan Nardi 3dbd89eee8 sha256: fix undefined-shift error (#2237)
```
third_party/src/ndpi_sha256.c:51:21: runtime error: left shift of 128 by 24 places cannot be represented in type 'int'
```

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65305
2024-01-02 12:29:20 +01:00
Ivan Nardi 308f71a6e8 TLS: fix heap-buffer-overflow (#2235)
```
==53992==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x511000007e04 at pc 0x555da2165fd0 bp 0x7ffddf7e3990 sp 0x7ffddf7e3988
READ of size 2 at 0x511000007e04 thread T0
    #0 0x555da2165fcf in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2384:50
    #1 0x555da217c31f in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:908:5
    #2 0x555da2176720 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1273:11
    #3 0x555da215a628 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2883:5
    #4 0x555da1e95c30 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6720:6
    #5 0x555da1e969f3 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6756:10
    #6 0x555da1e96394 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6789:12
    #7 0x555da1ea7991 in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7929:15
    #8 0x555da1ea547f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8104:22
    #9 0x555da1de137f in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1721:31
    #10 0x555da1de137f in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2438:1
```

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65362
2024-01-02 11:37:40 +01:00
Ivan Nardi d886a6107f Teamviewer: varius fixes (#2228)
We already have a generic (and up to date) logic to handle ip addresses:
remove that stale list.

Teamviewer uses TCP and UDP, both; we can't access `flow->l4.udp`.

According to a comment, we set the flow risk
`NDPI_DESKTOP_OR_FILE_SHARING_SESSION` only for the UDP flows.
2024-01-02 11:22:43 +01:00
Vladimir Gavrilov 2796bc9b47 Add NoMachine NX protocol dissector (#2234)
* Add NoMachine protocol dissector

* Fix detection on big endian architectures

* Make NoMachine over UDP check more strict

* Small fixes
2024-01-02 10:23:42 +01:00
Luca 2f657cb8f9 Implemented ndpi_is_outlier() for detecting outliers using z-score 2023-12-28 19:59:54 +01:00
Luca Deri 1366518bff Implements ndpi_pearson_correlation for measuring how correlated are two series 2023-12-27 22:42:37 +01:00
Luca Deri 99d4838328 New ndpi_sha256() nDPI API call (#2230)
* Added ndpi_sha256.c to the Windows project

* Added ndpi_sha256() nDPI API call
2023-12-23 10:47:28 +01:00
Luca Deri f2be972224 Removed type redefinition 2023-12-23 09:16:35 +01:00
Luca Deri caa223d2ba Changes to avoid type redefinition on windows 2023-12-23 09:15:42 +01:00
Luca Deri 5c7200f2bb Buffer overflow fix 2023-12-23 08:07:28 +01:00
Luca Deri 7c4be6d077 Updated result 2023-12-22 21:29:45 +01:00
Luca Deri 53b9ae23e8 Warning fix 2023-12-22 21:10:03 +01:00
Luca Deri cbea1b2203 Fixed check (when using ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --with-thread-sanitizer --with-pcre2 --with-maxminddb --enable-tls-sigs) 2023-12-22 21:01:14 +01:00
Luca Deri a95d4744b6 Fixes nDPI datastructs size check 2023-12-22 20:52:56 +01:00
Luca Deri 8285fffdae Implements JA4 Support (#2191) 2023-12-22 20:40:42 +01:00
Luca Deri b90c18e906 Changes to make room for JA4 2023-12-22 20:40:42 +01:00
Vladimir Gavrilov 5eb468d07b Add Apache Kafka protocol dissector (#2226) 2023-12-22 14:42:47 +01:00
Vladimir Gavrilov 6fc8aa4e61 Add WebDAV detection support (#2224)
* Add WebDAV detection support

* Add pcap example

* Update test results

* Remove redundant checks

* Add WebDAV related HTTP methods to fuzz/dictionary.dict

* Add note about WebDAV
2023-12-22 13:23:37 +01:00
Luca 41aa2f96b6 Various MDNS flow risks fixes 2023-12-21 12:24:30 +01:00
Luca 28b1ce41cb MDNS fix 2023-12-21 11:11:28 +01:00
Ivan Nardi a5595d16c0 CI: update list of compilers (#2223)
Try using latest gcc and clang versions.
We still care about RHEL7: since handling a RHEL7 runner on GitHub is
quite complex, let try to use a similar version of gcc, at least
2023-12-20 19:22:22 +01:00
Vladimir Gavrilov 149067b3fc Add JSON-RPC protocol dissector (#2217)
* Add JSON-RPC protocol dissector

* Small fixes

* Improve detection
2023-12-20 12:42:25 +01:00
Vladimir Gavrilov 33f11cb10f Add OpenFlow protocol dissector (#2222) 2023-12-20 10:48:45 +01:00
Ivan Nardi 8aa09f9c99 mining: a better identification logic (#2221)
It is quite simple (and not so efficient) but it should fix all the
false positives reported in #2216. Add support for Ethereum mining.

Merge all the mining traces.

Remove duplicated function.

Close #2216
2023-12-20 10:46:57 +01:00