Compare commits

..
726 Commits
Author SHA1 Message Date
Ivan Nardi ebcea42e2b fuzz: improve fuzzers using pl7m (#2486)
For some unclear reasons, fuzzers using pl7m create huge corpus,
triggering OOM in oss-fuzz runs (where the memory RSS limit is set to
2560Mb). Example:

```
==25340== ERROR: libFuzzer: out-of-memory (used: 2564Mb; limit: 2560Mb)
To change the out-of-memory limit use -rss_limit_mb=<N>

Live Heap Allocations: 2364004039 bytes in 133791 chunks; quarantined: 60662293 bytes in 3664 chunks; 176432 other chunks; total chunks: 313887; showing top 95% (at most 8 unique contexts)
1285841683 byte(s) (54%) in 2956 allocation(s)
    #0 0x56f814ef4bde in __interceptor_malloc /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
    #1 0x56f814e04416 in operator new(unsigned long) cxa_noexception.cpp:0
    #2 0x56f814de6b2d in assign<unsigned char *, 0> /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1443:3
    #3 0x56f814de6b2d in operator= /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1412:9
    #4 0x56f814de6b2d in fuzzer::InputCorpus::AddToCorpus(std::__Fuzzer::vector<unsigned char, std::__Fuzzer::allocator<unsigned char>> const&, unsigned long, bool, bool, bool, std::__Fuzzer::chrono::duration<long long, std::__Fuzzer::ratio<1l, 1000000l>>, std::__Fuzzer::vector<unsigned int, std::__Fuzzer::allocator<unsigned int>> const&, fuzzer::DataFlowTrace const&, fuzzer::InputInfo const*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerCorpus.h:221:10
    #5 0x56f814de60e5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:539:16
    #6 0x56f814de7df2 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:829:7
    #7 0x56f814de8127 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:867:3
    #8 0x56f814dd6736 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:914:6
    #9 0x56f814e02c62 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    #10 0x7fa11e2c3082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16

1031350683 byte(s) (43%) in 2468 allocation(s)
    #0 0x56f814ef4bde in __interceptor_malloc /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
    #1 0x56f814e04416 in operator new(unsigned long) cxa_noexception.cpp:0
    #2 0x56f814de6b2d in assign<unsigned char *, 0> /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1443:3
    #3 0x56f814de6b2d in operator= /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1412:9
    #4 0x56f814de6b2d in fuzzer::InputCorpus::AddToCorpus(std::__Fuzzer::vector<unsigned char, std::__Fuzzer::allocator<unsigned char>> const&, unsigned long, bool, bool, bool, std::__Fuzzer::chrono::duration<long long, std::__Fuzzer::ratio<1l, 1000000l>>, std::__Fuzzer::vector<unsigned int, std::__Fuzzer::allocator<unsigned int>> const&, fuzzer::DataFlowTrace const&, fuzzer::InputInfo const*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerCorpus.h:221:10
    #5 0x56f814de60e5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:539:16
    #6 0x56f814de7635 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:760:19
    #7 0x56f814de8425 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:905:5
    #8 0x56f814dd6736 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:914:6
    #9 0x56f814e02c62 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    #10 0x7fa11e2c3082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
```

See: https://oss-fuzz.com/testcase-detail/4717811415449600
See: https://oss-fuzz.com/testcase-detail/6164130982068224

Let's *try* the following workaround: set the parameter `-max-len` to
512K, to force the engine to not genereate inputs (i.e. pcap files...)
larger than 512K. Right now the value used is 1MB, i.e the default,
because we have file larger than 1MB in the initial seeds (i.e.
`/tests/pcaps/*`).

Let's hope than having smaller files lead to smaller corpus...

Update pl7m code (fix a Use-of-uninitialized-value error)
2024-06-29 18:01:05 +02:00
Ivan Nardi 83e6e753af fuzz: pl7m: add a custom mutator for better fuzzing of pcap files (#2483)
Pl7m is a custom mutator (used for structure aware fuzzing) for network
traffic packet captures (i.e. pcap files).

The output of the mutator is always a valid pcap file, containing the
same flows/sessions of the input file. That's it: the mutator only
changes the packet payload after the TCP/UDP header, keeping all the
original L2/L3 information (IP addresses and L4 ports).

See: https://github.com/IvanNardi/pl7m
2024-06-27 18:07:43 +02:00
Anshul ThakurandAnshul Thakur 8f6f73505d Remove duplicate API for ndpi_serialize_uint32_double in ndpi_api.h (#2480)
Co-authored-by: Anshul Thakur <athakur@cdot.in>
2024-06-26 14:10:57 +02:00
Nardi Ivan 556f892a56 wireshark: lua: export some metadata
Export some metadata (for the moment, SNI and TLS fingerprints) to
Wireshark/tshark via extcap.
Note that:
* metadata are exported only once per flow
* metadata are exported (all together) when nDPI stopped processing
the flow

Still room for a lot of improvements!
In particular:
* we need to add some boundary checks (if we are going to export other
attributes)
* we should try to have a variable length trailer
2024-06-25 16:39:45 +02:00
Nardi Ivan f44832cc51 wireshark: lua: filtering and trailer dissection work with tshark, too
```
ivan@ivan-Latitude-E6540:~/$ tshark -C "nDPI extcap" -i ndpi -o extcap.ndpi.i:/home/ivan/svnrepos/nDPI/tests/pcap/anydesk.pcapng -Y "ndpi.protocol.name contains DNS"
Capturing on 'nDPI interface: ndpi'
   62 22635386.425683 192.168.1.187 DNS.AnyDesk 192.168.1.1  128   Standard query 0xec22 A relay-3185a847.net.anydesk.com
   63 22635386.439540  192.168.1.1 DNS.AnyDesk 192.168.1.187 144   Standard query response 0xec22 A relay-3185a847.net.anydesk.com A 37.61.223.15
   64 22635386.721277 192.168.1.187 DNS.AnyDesk 192.168.1.1  128   Standard query 0xea89 A relay-9b6827f2.net.anydesk.com
   65 22635386.732444  192.168.1.1 DNS.AnyDesk 192.168.1.187 144   Standard query response 0xea89 A relay-9b6827f2.net.anydesk.com A 138.199.36.115
4 packets captured

```
2024-06-25 16:39:45 +02:00
Nardi Ivan 3f0ea18866 wireshark: lua: fix DNS dissection
Not sure when we (or Wireshark, or Lua...) broke it, but we can't call
tonumber() on Bool variables.
2024-06-25 16:39:45 +02:00
Nardi Ivan b5afa165f0 wireshark: extcap: restore filtering mechanism 2024-06-25 16:39:45 +02:00
Nardi Ivan 2daab3f248 wireshark: lua: latest Wireshark versions correctly handle 64 bit mask 2024-06-25 16:39:45 +02:00
Nardi Ivan 46e450456b wireshark: lua: minor improvements
* Use a proper TVB to parse the nDPI trailer
* Fix some flow risks definitions
2024-06-25 16:39:45 +02:00
Luca Deri e92568e4bb Improved logic for checking invalid DNS queries 2024-06-23 22:27:19 +02:00
Mark Jeffery aa1d7247d1 Added default port mappings to ndpiReader help -H (#2477)
Close #2125
2024-06-19 13:47:18 +02:00
Ivan Nardi 9feede9a14 Zoom: fix detection of screen sharing (#2476) 2024-06-17 17:30:24 +02:00
Ivan Nardi 26cc1f131f fuzz: improve fuzzing coverage (#2474)
Remove some code never triggered

AFP: the removed check is included in the following one
MQTT: fix flags extraction
2024-06-17 13:45:47 +02:00
Nardi Ivan a35fae6b75 Sync unit tests results 2024-06-17 11:23:39 +02:00
Toni 8fd649ab1e Add Ripe Atlas probe protocol. (#2473)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-06-17 11:00:38 +02:00
Nardi Ivan bbe52da5cf Zoom: harden RTP/RTCP detection 2024-06-17 10:19:55 +02:00
Nardi Ivan 526cf6f291 Zoom: remove "stun_zoom" LRU cache
Since 070a0908b we are able to detect P2P calls directly from the packet
content, without any correlation among flows
2024-06-17 10:19:55 +02:00
Ivan Nardi 2bedd14aae Update main Readme and remove commented out code (#2471)
Avoid that useless code is copy-and-pasted into new dissector, as in
https://github.com/ntop/nDPI/pull/2470#discussion_r1639384434
2024-06-17 10:11:17 +02:00
Mark Jeffery f796c94375 Added protocol - JRMI - Java Remote Method Invocation (#2470) 2024-06-15 10:52:28 +02:00
Luca 7c7c375b45 Improved detection of Android connectiity checks 2024-06-12 10:55:07 +02:00
Mark Jeffery 312dc424bd Added NDPI_PROTOCOL_NTOP assert and removed percentage comparison (#2460)
Close #2413
2024-06-10 19:45:19 +02:00
Ivan Nardi aee2c81f76 Zoom: fix integer overflow (#2469)
```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==29508==ERROR: AddressSanitizer: SEGV on unknown address 0x50710145d51d (pc 0x55cb788f25fe bp 0x7ffcfefa15f0 sp 0x7ffcfefa1240 T0)
==29508==The signal is caused by a READ memory access.
    #0 0x55cb788f25fe in ndpi_search_zoom /home/ivan/svnrepos/nDPI/src/lib/protocols/zoom.c:210:24
    #1 0x55cb787e9418 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7174:6
    #2 0x55cb7883f753 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7209:10
    #3 0x55cb7883bc9d in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7240:12
```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=69520
2024-06-10 11:06:17 +02:00
Ivan Nardi b90d39c4ac RTP/STUN: look for STUN packets after RTP/RTCP classification (#2465)
After a flow has been classified as RTP or RTCP, nDPI might analyse more
packets to look for STUN/DTLS packets, i.e. to try to tell if this flow
is a "pure" RTP/RTCP flow or if the RTP/RTCP packets are multiplexed with
STUN/DTLS.
Useful for proper (sub)classification when the beginning of the flows
are not captured or if there are lost packets in the the captured traffic.

Disabled by default
2024-06-07 13:12:04 +02:00
Ivan Nardi 070a0908b3 Zoom: faster detection of P2P flows (#2467) 2024-06-07 09:50:41 +02:00
Ivan Nardi 619005c5b2 STUN: add support for Microsoft Multiplexed TURN channels (#2464) 2024-06-05 16:55:58 +02:00
Ivan Nardi b6beeef623 TLS: add support for DTLS (over STUN) over TCP (#2463)
TODO: TCP reassembler on top of UDP reassembler

See: #2414
2024-06-05 13:49:34 +02:00
Ivan Nardi 6a9d4b1ab6 CI: more parallel work (#2459) 2024-06-05 11:56:03 +02:00
Ivan Nardi 7be482f5b1 Update unit tests results (#2466) 2024-06-05 11:55:30 +02:00
Luca Deri 85587a8f98 Fixes accounting of number of processed packets in ndpi_internal_detection_process_packet() 2024-06-04 17:00:45 +02:00
Ivan Nardi 81e42b748e RTP: fix detection over TCP (#2462)
RFC4571 is not the only way to wrap RTP messages in TCP streams.
For example, when RTP is encapsulated over TURN flows (i.e. via DATA
attribute) there is no additional framing.
See also 6127e0490
2024-05-29 18:31:10 +02:00
Maatuq 6127e04900 support rtp/rtcp over tcp (#2422) (#2457)
Support rtp/rtcp over tcp as per rfc4571.

Signed-off-by: mmaatuq <mahmoudmatook.mm@gmail.com>
2024-05-28 22:01:08 +02:00
Toni 80171dbcf3 Add ZUG consensus protocol dissector. (#2458)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-28 20:29:48 +02:00
Ivan Nardi 25f8964a23 CiscoVPN: we detect it only over UDP (#2454)
The original code handled also TCP/TLS, but it was removed in 6fc29b3ae
2024-05-28 14:07:48 +02:00
ToniandNardi Ivan abce6d4023 Improved Kafka dissector. (#2456)
* detect more Kafka request packet's
 * requires less flow memory
 * same detection behavior as before e.g. no asym detection implemented
   (can be done by dissecting responses, requires more effort)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2024-05-27 09:46:32 +02:00
Vladimir Gavrilov d1a59c0308 Rename Messenger to FacebookMessenger (#2453) 2024-05-23 09:23:31 +02:00
Ivan Nardi 41050af5b5 fuzz: fix build of oss-introspector (#2452) 2024-05-22 19:41:59 +02:00
Nardi Ivan 521ce609c8 Sync unit tests results 2024-05-22 19:00:33 +02:00
Luca 44a290286b More NDPI_PROBING_ATTEMPT changes 2024-05-22 18:04:33 +02:00
Luca 74d3843ebe Added NDPI_PROBING_ATTEMPT risk 2024-05-22 17:55:16 +02:00
Vladimir Gavrilov 15643547fe Replace ndpi_strnstr() implementation with an optimal one (#2447) 2024-05-22 12:47:27 +02:00
Luca Deri 5a25f89ab3 Enlarged buffer 2024-05-22 11:00:01 +02:00
Ivan Nardi fd02baa13a DTLS: fix JA4 fingerprint (#2446) 2024-05-21 18:13:25 +02:00
Toni 3639d2045b Remove unused code. (#2450)
* some `#ifdef`ed code dates back to 2019, 2020 and 2021
 * some function signatures were still present in `ndpi_main.h`
   which may cause linker errors for libnDPI dependee's
 * return an error while trying to serialize a double instead
   of `fprintf(stderr, ...)`

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-21 13:08:07 +02:00
Ivan Nardi 83840f1bb9 DTLS: add support for DTLS 1.3 (#2445) 2024-05-21 12:49:27 +02:00
Ivan Nardi 0109014f2c Follow-up of 2093ac5bf (#2451) 2024-05-21 12:47:25 +02:00
Ivan Nardi bc8bc69beb CI: enable parallel tests (for x86_64, at least) (#2444)
TODO: enable parallel tests when using docker with no-x86_64 archs.

When I tried the obviuos solutions:
```
NDPI_FORCE_PARALLEL_UTESTS=1 NDPI_SKIP_PARALLEL_BAR=1 make check VERBOSE=1
```
I got:
```
Run configuration "caches_cfg" [--cfg=lru.ookla.size,0 --cfg=lru.msteams.ttl,1]
ookla.pcap	/bin/sh: 1: run_single_pcap: not found
teams.pcap	/bin/sh: 1: run_single_pcap: not found
Run configuration "caches_global" [--cfg=lru.ookla.scope,1 --cfg=lru.bittorrent.scope,1 --cfg=lru.stun.scope,1 --cfg=lru.tls_cert.scope,1 --cfg=lru.mining.scope,1 --cfg=lru.msteams.scope,1 --cfg=lru.stun_zoom.scope,1]
bittorrent.pcap	/bin/sh: 1: run_single_pcap: not found
lru_ipv6_caches.pcapng	/bin/sh: 1: run_single_pcap: not found
mining.pcapng	/bin/sh: 1: run_single_pcap: not found
...
```
2024-05-20 19:07:16 +02:00
Ivan Nardi 399be12585 Small fixes after API cleanup done in c63446e59 (#2449) 2024-05-20 19:06:24 +02:00
Luca Deri 2093ac5bf6 Minor dissector optimizations 2024-05-20 12:17:04 +02:00
Luca Deri 42dba2e4af Added dpi.compute_entropy configuration parameter 2024-05-18 09:46:15 +02:00
Luca Deri c63446e592 Cleaned up API
Removed
-  int ndpi_load_ipv4_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);
-  int ndpi_load_ipv6_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);

Added (it supports both IPv4 and v6)
+  int ndpi_load_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);
2024-05-17 16:38:02 +02:00
Ivan Nardi 33d3d254ac CI: add support for ubuntu-24.04 runners (#2443) 2024-05-16 17:05:13 +02:00
Vladimir Gavrilov 3d1da00d8d Add Call of Duty Mobile support (#2438) 2024-05-15 12:46:02 +02:00
Ivan Nardi d6b1c24079 Parallel execution of unit tests (#2435)
Running unit tests is quite a bottleneck while developing or while
waiting for GitHub CI results...
Try to run the tests in parallel, using the `parallel` tool.

By default, tests still run one after the other, as usual; to enable
parallel execution you need `NDPI_FORCE_PARALLEL_UTESTS=1 ./tests/do.sh`

Please note that the output is quite different in parallel mode!

A big part of the script has been rewritten to avoid code dupication
between "serial" and "parallel" path

On my notebook:
```
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(parallel)$ time ./tests/do.sh
[...]
real	3m12,684s
[...]
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(parallel)$ time NDPI_FORCE_PARALLEL_UTESTS=1 ./tests/do.sh
[...]
real	0m58,463s
```
2024-05-15 08:17:12 +02:00
Ivan Nardi 0110623b4e H323: improve detection and avoid false positives (#2432) 2024-05-11 23:39:54 +02:00
Ivan Nardi a064261e85 Revert ndpi_strnstr() optimization introduced in a813121e0 (#2439)
New implementation fails tests 11b, 12 and 13.
Revert to the original (BSD) implementation (with also some basic
parameters check)
2024-05-11 23:37:31 +02:00
Vladimir Gavrilov 2d33431948 Add Ethernet Global Data support (#2437) 2024-05-11 16:49:16 +02:00
Ivan Nardi 8c5e6e9761 IRC: simplify detection (#2423)
IRC has its best times well behind, but there are still some servers
using it.

We should try to simplify the detection logic, still based on OpenDPI
logic.

Let's start with some easy changes:
* try to detect TLS connection via standard hostname/SNI matching,
removing an old heuristic (we have never had any trace matching it);
* add some basic server names;
* once we detect that the flow is IRC, we don't have to perform
anything else;
* remove HTTP stuff; real HTTP flows never trigger that data path
* use `ndpi_memmem()` when possible
2024-05-11 14:12:13 +02:00
Vladimir GavrilovandIvan Nardi 70c4920184 Remove Vevo support (#2436)
Co-authored-by: Ivan Nardi <12729895+IvanNardi@users.noreply.github.com>
2024-05-11 11:57:53 +02:00
Ivan Nardi b116456fc5 Viber: add detection of voip calls and avoid false positives (#2434) 2024-05-11 09:21:13 +02:00
Vladimir Gavrilov a813121e0a ndpi_strnstr() optimization (#2433) 2024-05-10 22:43:59 +02:00
Ivan Nardi 4b4b358562 Line: use common helper to detect RTP/RTCP packets (#2429)
Add an explicit upper limit on the number of packets processed before
giving up.
2024-05-10 19:23:50 +02:00
Luca Deri 6bbae9a70a Update README.md 2024-05-10 08:53:19 +02:00
Ivan Nardi d4650f0f81 Raknet/RTP: avoid Raknet false positives and harden RTP heuristic (#2427)
There is some overlap between RTP and Raknet detection: give precedence
to RTP logic.
Consequences:
* Raknet might require a little bit more packets for some flows (not a
big issue)
* some very small (1-2 pkts) Raknet flows are not classified (not sure
what do do about that..)
2024-05-09 21:18:18 +02:00
Luca Deri e4fd6e47a3 Added #ifdef for Windows builds (#2431) 2024-05-09 17:27:25 +02:00
Ivan Nardi 2e7fc82e55 Protobuf: fix false positives (#2428) 2024-05-09 15:38:06 +02:00
Toni 18e03a26ca Add extra entropy checks and more precise(?) analysis. (#2383)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-09 15:24:11 +02:00
Ivan Nardi b65a755e85 Fix a memory access error and some leaks (#2425)
```
SCARINESS: 12 (1-byte-read-heap-buffer-overflow)
    #0 0x557f3a5b5100 in ndpi_get_host_domain /src/ndpi/src/lib/ndpi_domains.c:158:8
    #1 0x557f3a59b561 in ndpi_check_dga_name /src/ndpi/src/lib/ndpi_main.c:10412:17
    #2 0x557f3a51163a in process_chlo /src/ndpi/src/lib/protocols/quic.c:1467:7
    #3 0x557f3a469f4b in LLVMFuzzerTestOneInput /src/ndpi/fuzz/fuzz_quic_get_crypto_data.c:44:7
    #4 0x557f3a46abc8 in NaloFuzzerTestOneInput (/out/fuzz_quic_get_crypto_data+0x4cfbc8)
```

Some notes about the leak: if the insertion into the uthash fails (because of an
allocation failure), we need to free the just allocated entry. But the only
way to check if the `HASH_ADD_*` failed, is to perform a new lookup: a bit
costly, but we don't use that code in the fast-path.
See also efb261a95c

Credits for finding the issues to Philippe Antoine (@catenacyber) and his
`nallocfuzz` fuzzing engine
See: https://github.com/catenacyber/nallocfuzz
See: https://github.com/google/oss-fuzz/pull/9902
2024-05-08 11:46:46 +02:00
Ivan Nardi 7c6910d9e5 Fix/improve fuzzing (#2426) 2024-05-08 11:46:02 +02:00
Toni e9dc035c5c Added optimized memmem/strlcpy version (#2424)
* credits goes to Vladimir Gavrilov

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-08 11:38:53 +02:00
Luca Deri 2b4e2f9c9a Removed trace 2024-05-07 23:42:32 +02:00
Luca Deri 5f8ad0527d Reworked code and added two new API calls
- int ndpi_load_ipv4_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);
- int ndpi_load_ipv6_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);
2024-05-07 23:05:31 +02:00
Luca Deri 3d2b611ea6 ndpi_mod is now optional (albeit better to spcify it) in ndpi_domain_classify_xxx calls 2024-05-07 13:09:00 +02:00
Luca Deri 0d495294bf Win warning fixes 2024-05-07 10:04:11 +02:00
Luca Deri 6195dbad76 Renamed radius source file to avoid name clashes on Windows 2024-05-07 09:05:15 +02:00
Ivan Nardi 1773d7ff2d Add support for Mastodon, Bluesky and (FB-)Threads (#2418) 2024-05-06 13:37:18 +02:00
Ivan Nardi 95fe21015d Remove "zoom" cache (#2420)
This cache was added in b6b4967aa, when there was no real Zoom support.
With 63f349319, a proper identification of multimedia stream has been
added, making this cache quite useless: any improvements on Zoom
classification should be properly done in Zoom dissector.

Tested for some months with a few 10Gbits links of residential traffic: the
cache pretty much never returned a valid hit.
2024-05-06 12:51:45 +02:00
Ivan Nardi b7d77972bc Remove workaround for TCP flows with multiple SYNs (#2421)
Deciding when a session starts and ends is responsability of the
applicationi (via its flow manager)i, not of the library.
BTW, the removed code is incomplete at beast
2024-05-06 10:20:40 +02:00
Ivan Nardi e31ef00715 TLS: avoid setting NDPI_TLS_SELFSIGNED_CERTIFICATE for webrtc traffic (#2417)
See RFC8122: it is quite likely that STUN/DTLS/SRTP flows use
self-signed certificates

Follow-up of b287d6ec8
2024-05-06 10:20:07 +02:00
Ivan Nardi 266af02752 Merge RTP and RTCP logic (#2416)
Avoid code duplication between these two protocols.

We remove support for RTCP over TCP; it is quite rare to find this kind
of traffic and, more important, we have never had support for RTP
over TCP: we should try to add both detecion as follow-up.

Fix a message log in the LINE code
2024-05-06 10:19:46 +02:00
Ivan Nardi a1845da359 TLS: fix Ja4 fingerprint computation (#2419)
The new values has been checked against the ones reported by Wireshark.

Found while fixing a Use-of-uninitialized-value error reported by
oss-fuzz

```
==7582==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x5a6549abc368 in ndpi_compute_ja4 ndpi/src/lib/protocols/tls.c:1762:10
    #1 0x5a6549ab88a0 in processClientServerHello ndpi/src/lib/protocols/tls.c:2863:10
    #2 0x5a6549ac1452 in processTLSBlock ndpi/src/lib/protocols/tls.c:909:5
    #3 0x5a6549abf588 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:1098:2
    #4 0x5a65499c53ec in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:7215:6
```

See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68449&q=ndpi&can=1&sort=-id
2024-05-05 17:27:29 +02:00
Ivan Nardi a6fd981fcf eDonkey: improve/update classification (#2410)
eDonkey is definitely not as used as >10 years ago, but it seems it is
still active.

While having a basic TCP support seems easy, identification over UDP doesn't
work and it is hard to do it rightly (packets might be only 2 bytes long):
remove it.

Credits to V.G <v.gavrilov@securitycode.ru>
2024-05-04 19:11:31 +02:00
Luca Deri 57ecbf38c0 Updated JA4 test results 2024-05-02 17:40:24 +02:00
Luca Deri 66133d1501 Fixes JA4 computation adding a better GREASE detect funzion 2024-05-02 16:58:44 +02:00
Luca Deri af955169e1 typo 2024-05-02 16:03:30 +02:00
Ivan Nardi 9e549ba781 Workaround for fixing GitHub runners on macOS (#2411)
GitHub switched "macos-latest" from "macos-12" to "macos-14", which is only
on ARM64!
https://github.com/actions/runner/issues/3256
https://github.blog/changelog/2024-01-30-github-actions-macos-14-sonoma-is-now-available/

However we are having some issues build nDPI on macos-14 with external
libraries:

```
configure: error: libgpg-error required (because of --with-local-libgcrypt) but not found or too old.
```
See: https://github.com/ntop/nDPI/actions/runs/8869020568/job/24350356867

```
ndpi_utils.c:69:10: fatal error: 'pcre2.h' file not found
         ^~~~~~~~~
1 error generated.
```
See: https://github.com/ntop/nDPI/actions/runs/8869020568/job/24349242251

Everything is still fine with macos-14 and no external dependencies

As workaround, test only macos-12 and macos-13 in our main matrix.
2024-04-28 22:17:39 +02:00
Ivan Nardi d5bda47efe DTLS: add support for Alert message type (similar to TLS) (#2406) 2024-04-25 11:23:05 +02:00
0x41CEA55 7040847eed Add Adobe Connect support (#2407) 2024-04-24 12:41:25 +02:00
0x41CEA55 66036a14f4 Remove PPStream protocol and add iQIYI (#2403)
P2P video player PPStream was discontinued shortly after the purchase of PPS.tv by Baidu (iQIYI) on 2013 (see https://www.techinasia.com/report-baidu-acquires-video-rival-pps)
So we remove the old `NDPI_PROTOCOL_PPSTREAM` logic and add `NDPI_PROTOCOL_IQIYI` id to handle all the iQIYI traffic, which is basically video streaming traffic.

A video hosting service, called PPS.tv, is still offered by the same company: for the time being we classified both services with the same protocol id.
2024-04-23 18:01:36 +02:00
0x41CEA55 fd388845d5 Add BFCP protocol support (#2401) 2024-04-23 15:35:19 +02:00
Ivan Nardi abc7e430e2 STUN: slightly faster sub-classification with DTLS (#2404) 2024-04-23 09:31:48 +02:00
Ivan Nardi a62679952c fuzz: fix shoco fuzzer (#2405)
```
==22779==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f0900701020 at pc 0x555bcd2a6f02 bp 0x7ffe3ba5e790 sp 0x7ffe3ba5e788
READ of size 1 at 0x7f0900701020 thread T0
    #0 0x555bcd2a6f01 in shoco_decompress /home/ivan/svnrepos/nDPI/src/lib/third_party/src/shoco.c:184:26
    #1 0x555bcd2a4018 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco.cpp:18:3
    #2 0x555bcd1aa816 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x4f816) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)
    #3 0x555bcd193be8 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x38be8) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)
    #4 0x555bcd1996fa in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x3e6fa) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)
    #5 0x555bcd1c3c92 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x68c92) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)
    #6 0x7f090257a082 in __libc_start_main /build/glibc-e2p3jK/glibc-2.31/csu/../csu/libc-start.c:308:16
    #7 0x555bcd18e96d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x3396d) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)

Address 0x7f0900701020 is located in stack of thread T0 at offset 4128 in frame
    #0 0x555bcd2a3d97 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco.cpp:5

  This frame has 4 object(s):
    [32, 4128) 'out' (line 9) <== Memory access at offset 4128 overflows this variable
    [4256, 8352) 'orig' (line 9)
```

Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68211
2024-04-22 18:39:23 +02:00
Nardi Ivan f494bdd653 Fix parameters check
```
==17==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000546050 bp 0x7fff113c82a0 sp 0x7fff113c7a58 T0)
==17==The signal is caused by a READ memory access.
==17==Hint: address points to the zero page.
SCARINESS: 10 (null-deref)
    #0 0x546050 in __sanitizer::internal_strlen(char const*) /src/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_libc.cpp:167:10
    #1 0x4c6ba5 in __interceptor_strrchr /src/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:740:5
    #2 0x5fb9b9 in ndpi_get_host_domain_suffix /src/ndpi/src/lib/ndpi_domains.c:105:20
    #3 0x578058 in LLVMFuzzerTestOneInput /src/ndpi/fuzz/fuzz_config.cpp:503:3
```

Found while fuzzing
2024-04-21 12:29:09 +02:00
0x41CEA55 246e972f9c Replace my personal email with my corporate one in all my contributions (#2399) 2024-04-20 21:37:01 +02:00
Nardi Ivan 95f4d0b852 Fix a warning
```
nDPI/PcDebug64/src/include/ndpi_api.h:1970:3: error: function declaration isn’t a prototype [-Werror=strict-prototypes]
```
2024-04-20 19:30:42 +02:00
Ivan Nardi ef89183469 fuzz: improvements (#2400)
Create the zip file with all the traces only once.

Add a new fuzzer to test "shoco" compression algorithm
2024-04-20 18:15:23 +02:00
0x41CEA55 905120588b Remove obsolete protocols: tuenty, tvuplayer and kontiki (#2398) 2024-04-19 21:35:32 +02:00
0x41CEA55 1b2e2cd968 Add strlcpy implementation (#2395) 2024-04-19 17:16:40 +02:00
0x41CEA55 e75d7a620e Add KNXnet/IP protocol support (#2397)
* Add KNXnet/IP protocol support

* Improve KNXnet/IP over TCP detection
2024-04-19 12:54:00 +02:00
Luca Deri ad117bfaab Domain Classification Improvements (#2396)
* Added
size_t ndpi_compress_str(const char * in, size_t len, char * out, size_t bufsize);
size_t ndpi_decompress_str(const char * in, size_t len, char * out, size_t bufsize);

used to compress short strings such as domain names. This code is based on
https://github.com/Ed-von-Schleck/shoco

* Major code rewrite for ndpi_hash and ndpi_domain_classify

* Improvements to make sure custom categories are loaded and enabled

* Fixed string encoding

* Extended SalesForce/Cloudflare domains list
2024-04-18 23:21:40 +02:00
Luca Deri 108b8331d5 Invalid initialization 2024-04-15 18:13:17 +02:00
Luca Deri 95897c6436 Fixed minor glitches 2024-04-15 14:25:26 +02:00
Ivan Nardi d97afd6efd STUN: fix attributes list iteration (#2391)
We need to check all the attributes, to look for any possible metadata
2024-04-13 18:43:41 +02:00
Ivan Nardi 06f4814085 STUN: try to stop extra dissection earlier, if possible (#2390) 2024-04-13 14:45:17 +02:00
Ivan Nardi 8edb2f133c STUN: add support for ipv6 in some metadata (#2389) 2024-04-13 14:12:20 +02:00
Ivan Nardi f51dfc6973 STUN: simplify ip/port parsing (#2388)
Add other 2 configuration options
2024-04-13 12:00:04 +02:00
Ivan Nardi 0535e54484 STUN: fix boundary checks on attribute list parsing (#2387)
Restore all unit tests.
Add some configuration knobs.
Fix the endianess.
2024-04-12 22:55:51 +02:00
Luca Deri bb4fccd2cf Added comment 2024-04-12 20:29:14 +02:00
Luca Deri b83eb7c7a2 Implemented STUN peer_address, relayed_address, response_origin, other_address parsing
Added code to ignore invalid STUN realm
Extended JSON output with STUN information
2024-04-12 19:50:04 +02:00
Toni Uhlig 31de94ba12 fix invalid read
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-12 17:55:48 +02:00
Vladimir Gavrilov c63697205b Add Label Distribution Protocol support (#2385)
* Add Label Distribution Protocol support

* Fix typo

* Update unit test results
2024-04-12 17:44:36 +02:00
Ivan Nardi 93f02ee507 Fix ndpi_reconcile_msteams_udp (#2377)
Microsoft UDP traffic over port ~3478 is voip traffic, using some kind
of proprietary STUN-like protocol: so use the most specific protocol id.

More important, we definitely want `Stun/Skype_TeamsCall` and not
`Stun/Skype_Teams`
2024-04-12 17:05:28 +02:00
Toni Uhlig 65fef09ff2 Updated unit test results
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-12 16:52:00 +02:00
Toni Uhlig 082b00ede7 Updated unit test results
* fixed invalid read

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-12 16:38:44 +02:00
Luca Deri a8507c3308 STUN:
- Fixed issue with XOR-MAPPED-ADDRESS decoding
- Implemented MAPPED_ADDRESS (IPv4 only)
2024-04-12 15:49:42 +02:00
Nardi Ivan 17965c8852 Update unit tests results 2024-04-11 18:36:43 +02:00
Luca Deri fd2838e9a1 Minor code cleanup 2024-04-11 17:44:44 +02:00
Toni 39a5d234fd Serialize amount of found TLS blocks and signature algos. (#2384)
* partially fixes #2282

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-11 16:21:41 +02:00
Luca Deri 8b4fdbc0fb Renamed ndpi_fill_ip6_protocol_category in ndpi_fill_ipv6_protocol_category 2024-04-11 12:37:39 +02:00
Vladimir Gavrilov 9ff4bece33 Add The Elder Scrolls Online support (#2376)
* Add The Elder Scrolls Online support

* Use ndpi_memmem instead of memmem from libc

* Add protocol description

* Change selection bitmask to V4_V6

* Update protocols.rst
2024-04-10 18:04:02 +02:00
Vladimir Gavrilov 142c8f5afb Add memmem() implementation (#2378)
* Add memmem() implementation

* Fix build

* Add fix to avoid too many memcmp calls
2024-04-10 15:39:08 +02:00
Toni 1d0be6c4f4 Add Shellscript risk detection. (#2375)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-10 15:36:54 +02:00
Ivan Nardi 54517d8e04 Fix a warning and restore a unit test result (#2379) 2024-04-10 09:56:06 +02:00
Luca Deri bd2968dcd3 Tuned DNS risk values
Modified NDPI_BINARY_TRANSFER_ATTEMPT in NDPI_BINARY_DATA_TRANSFER
2024-04-09 10:22:01 +02:00
Luca Deri 00a360f30a Disabled "known proto on non standard port" for FTP_DATA 2024-04-09 00:01:47 +02:00
Luca Deri 98bf0e243e Fixed false positives on binary application transfer risk 2024-04-08 23:52:27 +02:00
Luca Deri 48b0f8e2c1 Typo 2024-04-08 18:45:32 +02:00
Ivan Nardi 1b3ef7d7b2 STUN: improve extraction of Mapped-Address metadata (#2370)
Enable parsing of Mapped-Address attribute for all STUN flows: that
means that STUN classification might require more packets.

Add a configuration knob to enable/disable this feature.

Note that we can have (any) STUN metadata also for flows *not*
classified as STUN (because of DTLS).

Add support for ipv6.

Restore the correct extra dissection logic for Telegram flows.
2024-04-08 10:24:51 +02:00
Ivan Nardi f5905a62c7 Fix invalid memory access (#2374)
The bug is triggered when `pe_offset == (u_int32_t)-1`

```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==23719==ERROR: AddressSanitizer: SEGV on unknown address 0x5081000002b3 (pc 0x55c69274ac72 bp 0x7ffffffc8e70 sp 0x7ffffffc8cc0 T0)
==23719==The signal is caused by a READ memory access.
    #0 0x55c69274ac72 in ndpi_search_portable_executable /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8191:7
    #1 0x55c69271606b in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8596:5
    #2 0x55c69270f58f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8629:22
    #3 0x55c6926a07e7 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:24:5
    #4 0x55c6925a79b6 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x64e9b6) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
    #5 0x55c692590d48 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x637d48) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
    #6 0x55c69259685a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x63d85a) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
    #7 0x55c6925c0e02 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x667e02) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
    #8 0x7f8e99793082 in __libc_start_main /build/glibc-wuryBv/glibc-2.31/csu/../csu/libc-start.c:308:16
    #9 0x55c69258baed in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x632aed) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
```

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67881
2024-04-06 20:23:06 +02:00
Toni a5d45253c4 Add ELF risk detection (detect transmitted linux executables). (#2373)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-06 19:32:51 +02:00
Toni 727e72d1f1 Calculate packet entropy for unknown protocols. (#2369)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-06 17:01:19 +02:00
Vladimir Gavrilov b535033674 Get rid of Apache Kafka false positives (#2372) 2024-04-06 16:59:38 +02:00
Ivan Nardi 700637a162 fuzzing: extend fuzzing coverage (#2371) 2024-04-05 21:02:54 +02:00
Toni 0f77f49b77 Add PE32/PE32+ risk detection (detect transmitted windows executables). (#2312)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-05 14:43:28 +02:00
Toni 99e521eaf8 Limit git commit hash to 7 characters. (#2368)
* ensure consistency across all platforms

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-05 11:04:22 +02:00
Toni 09bb383437 Fixed yet another another "unused function" warning. (#2367)
* added `-Wextra` to the CI

```
In file included from ndpi_bitmap64_fuse.c:31:
./third_party/include/binaryfusefilter.h:31:24: error: unused function 'binary_fuse_rotl64' [-Werror,-Wunused-function]
static inline uint64_t binary_fuse_rotl64(uint64_t n, unsigned int c) {

..snip..
```

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 20:41:45 +02:00
RoboSchmied ead9aa72da Fix: 3 typos (#2366)
Signed-off-by: RoboSchmied <github@roboschmie.de>
2024-04-04 20:06:47 +02:00
Toni df29e12f5e Fixed yet another "unused function" warning. (#2365)
* seems like clang on `ubuntu-latest` warns about unused static inlined functions

```
In file included from ndpi_bitmap.c:41:
./third_party/include/roaring.h:422:19: error: unused function 'roaring_leading_zeroes' [-Werror,-Wunused-function]
static inline int roaring_leading_zeroes(unsigned long long input_num) {

..snip..
```

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 18:18:44 +02:00
Ivan Nardi c0d3f8a92e STUN: rework sub-classification (#2361)
The main goal is to have the "real" application (if any; i.e.
Signal/Whatsapp/Telegram/...) always as "application" protocol and not
as "master" one
2024-04-04 18:16:40 +02:00
Toni 6e61368cd6 Add -Wno-unused-function as CFLAG for third_party/src/gcrypt_light.c. (#2364)
* fixes failing nDPI build from an external project with clang and `-Wextra`

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 13:51:27 +02:00
Toni Uhlig dface63157 Fixed flow struct static assertion size check (this time with struct padding in mind).
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 12:40:09 +02:00
Toni Uhlig 6917acffda Fixed flow struct static assertion size check.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 12:36:42 +02:00
Luca Deri 9185c2ccc4 Added support for STUN Mapped IP address 2024-04-03 23:03:46 +02:00
Nardi Ivan aacbc9e918 Update unit tests results 2024-04-03 16:18:24 +02:00
Luca 225ff7f8a4 Added binary data transfer risk alert 2024-04-03 15:41:26 +02:00
Ivan Nardi 1acc9ee396 Patrica tree: fix handling of mac addresses (#2362)
Found looking at the fuzzer statistics...
2024-04-03 14:10:37 +02:00
Toni 41eef9246c Disable -Wno-unused-parameter -Wno-unused-function. (#2358)
* unused parameters and functions pollute the code and decrease readability

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-03 14:10:21 +02:00
Vitaly Lavrov b5e8bc2214 Fixed gcc version checking. (#2360)
Using the "test" utility instead of "[".
2024-04-02 10:58:55 +02:00
Vladimir Gavrilov 599cc0f4b8 Improve LoL: Wild Rift detection (#2359) 2024-03-28 16:01:01 +01:00
Vladimir Gavrilov 5b32c98a21 Add LoL: Wild Rift detection (#2356) 2024-03-26 08:11:14 +01:00
Luca Deri 51f5fc7140 Added support for roaring bitmap v3 (#2355)
* Integrated RoaringBitmap v3

* Renamed ndpi_bitmap64 ro ndpi_bitmap64_fuse

* Fixes to ndpi_bitmap for new roaring library

* Fixes for bitmap serialization

* Fixed format

* Warning fix

* Conversion fix

* Warning fix

* Added check for roaring v3 support

* Updated file name

* Updated path

* Uses clang-9 (instead of clang-7) for builds

* Fixed fuzz_ds_bitmap64_fuse

* Fixes nDPI printf handling

* Disabled printf

* Yet another printf fix

* Cleaup

* Fx for compiling on older platforms

* Fixes for old compilers

* Initialization changes

* Added compiler check

* Fixes for old compilers

* Inline function is not static inline

* Added missing include
2024-03-25 08:15:19 +01:00
Vladimir Gavrilov be0fd7444f Add more TencentGames signatures (#2354) 2024-03-21 14:38:47 +01:00
Vladimir Gavrilov 27f9ca9d04 Improve TencentGames detection (#2353)
* Improve TencentGames detection

* Add more signatures
2024-03-20 12:13:32 +01:00
Nardi Ivan 15a80527c6 STUN: remove workaround to identify RTP traffic
We are able to demultiplex RTP packets in STUN flows since 3608ab01b, at
least; no need to explicity call the RTP dissector
2024-03-20 09:39:15 +01:00
Nardi Ivan 5ccf41739b STUN: fix fallback when DTL dissection fails 2024-03-20 09:39:15 +01:00
Nardi Ivan f758f3605a STUN: dissection of TURN Channel Data 2024-03-20 09:39:15 +01:00
Nardi Ivan ee87c4bc66 STUN: add dissection of XOR-PEER-ADDRESS with ipv6 address 2024-03-20 09:39:15 +01:00
Nardi Ivan 58894dd788 STUN: improve heurstic to detect old classic-stun 2024-03-20 09:39:15 +01:00
Ivan Nardi dabf29ea61 STUN: update demultiplexing logic according to RFC9443 (#2349) 2024-03-19 09:11:51 +01:00
Vladimir Gavrilov e6474d835f Add FLUTE protocol dissector (#2351)
* Add FLUTE protocol dissector

* Add flute.c to MSVC project
2024-03-19 09:11:04 +01:00
Toni 15f61e7abe Enable USE_GLOBAL_CONTEXT via CFLAGS passed to the CC. (#2348)
* `ndpi_typedefs.h`: requires to include `ndpi_config.h` for the `HAVE_STRUCT_TIMESPEC` check
   That will never happen, because `USE_GLOBAL_CONTEXT` is defined inside `ndpi_config.h`.
   It's better to use `CFLAGS` to achieve the same.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-03-15 12:47:13 +01:00
Ivan Nardi 231748bb0e LRU cache: move to 64 bits long keys (#2346)
Tradeoff between key comparison efficiency (i.e. no `memcmp`) and key
length.
At least in the ipv4 cases, we have no more different entries with the
same key.
2024-03-15 10:12:51 +01:00
Ivan Nardi 97fae6e00a STUN: fix parsing of DATA attribute (#2345) 2024-03-14 14:20:51 +01:00
Ivan Nardi 37ddf02e63 bitmap64: fix memory access error (#2344)
```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==29723==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x562910b70ddb bp 0x7ffcb22c5b70 sp 0x7ffcb22c5a80 T0)
==29723==The signal is caused by a READ memory access.
==29723==Hint: address points to the zero page.
    #0 0x562910b70ddb in binary_fuse16_contain /home/ivan/svnrepos/nDPI/src/lib/./third_party/include/binaryfusefilter.h:492:8
    #1 0x562910b70bbe in ndpi_bitmap64_isset /home/ivan/svnrepos/nDPI/src/lib/ndpi_bitmap64.c:178:10
    #2 0x562910788fd3 in ndpi_domain_classify_longest_prefix /home/ivan/svnrepos/nDPI/src/lib/ndpi_domain_classify.c:261:5
    #3 0x56291078940e in ndpi_domain_classify_contains /home/ivan/svnrepos/nDPI/src/lib/ndpi_domain_classify.c:291:9
    #4 0x56291069a392 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ds_domain_classify.cpp:52:5
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67369
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67372
2024-03-14 14:20:35 +01:00
Ivan Nardi 25c145f26d CI actions: fix Ubuntu jobs with sanitizers (#2347)
See: https://github.com/actions/runner-images/issues/9491
2024-03-14 14:20:21 +01:00
Vladimir Gavrilov 8fad77991d Add PFCP protocol dissector (#2342) 2024-03-13 20:18:43 +01:00
Ivan Nardi 4102c8df72 CI: fix build on MacOS-13 runners (#2343)
Workaroud for Homebrew's python link error
See: https://github.com/Homebrew/homebrew-core/issues/165793#issuecomment-1991817938
2024-03-13 19:22:13 +01:00
Luca Deri 1aedbefffa Fixes bitmap memory calculation 2024-03-11 17:59:24 +01:00
Ivan Nardi 6152d595e8 STUN: add a parameter to configure how long the extra dissection lasts (#2336)
Tradeoff: performance (i.e. number of packets) vs sub-classification
2024-03-07 14:39:32 +01:00
Ivan Nardi 56ce228a8b Add a specific protocol id for audio/video calls made using Google apps (#2341)
Same logic already used for Signal/Whatsapp/Line/Facebook/...
2024-03-07 13:48:19 +01:00
Ivan Nardi c09b4dacdf Fix memory leak (#2340)
```
Direct leak of 12 byte(s) in 1 object(s) allocated from:
    #0 0x55779e1a46ff in malloc (/home/ivan/svnrepos/nDPI/example/ndpiReader+0x8706ff) (BuildId: 14c2fc626744710d49d652ea1c5bbb24a8cbab4f)
    #1 0x55779e2120c7 in ndpi_malloc_wrapper /home/ivan/svnrepos/nDPI/example/ndpiReader.c:298:10
    #2 0x55779e5fa215 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
    #3 0x55779e5fa500 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13
    #4 0x55779e42153c in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2554:46
    #5 0x55779e4359a1 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:908:5
    #6 0x55779e432de7 in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1097:2
    #7 0x55779e4133f9 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2913:5
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67250
2024-03-07 13:11:13 +01:00
Toni 1fb4719e59 Disable AX_PTHREAD for MingW/MSYS builds. (#2338)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-03-07 13:09:01 +01:00
Vladimir Gavrilov e2949048e0 Add Path of Exile protocol dissector (#2337)
* Add Path of Exile protocol dissector

* Update protocols.rst
2024-03-06 19:59:09 +01:00
Ivan Nardi 21da53d3a0 ahocorasick: improve matching with subdomains (#2331)
The basic idea is to have the following logic:
* pattern "DOMAIN" matches the domain itself (i.e exact match) *and* any
subdomains (i.e. "ANYTHING.DOMAIN")
* pattern "DOMAIN." matches *also* any strings for which is a prefix
[please, note that this kind of match is handy but it is quite
dangerous...]
* pattern "-DOMAIN" matches *also* any strings for which is a postfix

Examples:
* pattern "wikipedia.it":
  * "wikipiedia.it" -> OK
  * "foo.wikipedia.it -> OK
  * "foowikipedia.it -> NO MATCH
  * "wikipedia.it.com -> NO MATCH
* pattern "wikipedia.":
  * "wikipedia.it" -> OK
  * "foo.wikipedia.it -> OK
  * "foowikipedia.it -> NO MATCH
  * "wikipedia.it.com -> OK
* pattern "-wikipedia.it":
  * "wikipedia.it" -> NO MATCH
  * "foo.wikipedia.it -> NO MATCH
  * "0001-wikipedia.it -> OK
  * "foo.0001-wikipedia.it -> OK

Bottom line:
* exact match
* prefix with "." (always, implicit)
* prefix with "-" (only if esplicitly set)
* postfix with "." (only if esplicitly set)

That means that the patterns cannot start with '.' anymore.

Close #2330
2024-03-06 19:25:59 +01:00
Luca Deri 8f63a11735 Extended connectivity checks 2024-03-05 17:45:46 +01:00
Luca Deri 65b9ad88a9 Improved alert on suspicious DNS traffic 2024-03-05 11:14:34 +01:00
Vladimir Gavrilov e7f1946ae7 Add NetEase Games detection support (#2335) 2024-03-05 11:00:52 +01:00
Vladimir Gavrilov 58fdc9fafb Add Naraka Bladepoint detection support (#2334) 2024-03-04 08:30:54 +01:00
Luca Deri 70aae6e998 Added DGA exception for Dropbox 2024-03-03 22:14:34 +01:00
Luca Deri e0587fe75e indent 2024-03-03 22:12:13 +01:00
Vladimir Gavrilov f2e3c7fb90 Add BFD protocol dissector (#2332) 2024-02-29 08:19:00 +01:00
Ivan Nardi c2b5b48fc8 ndpiReader: restore ndpiReader -x $DOMAIN_NAME functionality (#2329) 2024-02-26 20:50:07 +01:00
Nardi Ivan b287d6ec85 TLS: avoid setting some flow risks for webrtc traffic
Is quite rare to have a SNI or an ALPN on Client Hello of STUN/DTLS/SRTP
traffic
2024-02-26 10:20:05 +01:00
Nardi Ivan 478c31b6c9 utils: update script to download Cloudflare ips 2024-02-26 09:26:21 +01:00
Nardi Ivan ed5ba179f6 Telegram: improve identification
Follow up of 31c706c3db and
75485e177c.

Allow fast classification by ip, but give time to other dissectors to
kick in (for example, the TLS code for the Telegram Web flows).

Even if we don't classify it anymore at the very first packet (i.e. SYN)
we fully classify Telegram traffic at the first packet with payload, as
*any* other protocol.
This way, we always have the proper category, the proper confidence
for the UDP flows and we don't overwrite previous classifications (TLS
or ICMP)

Remove old and stale identification logic for TCP flows
2024-02-26 09:26:21 +01:00
Ivan Nardi c95e8c184e STUN: fix category when sub-classification is set in "extra-dissection" data path (#2320) 2024-02-24 09:44:24 +01:00
Luca Deri 71b7afccf5 Updated telegam out 2024-02-23 16:22:13 +01:00
Luca Deri 75485e177c Improved Telegram detection 2024-02-23 16:15:29 +01:00
Luca Deri bfb21dc09b Fixes exception handling glitch 2024-02-22 23:19:01 +01:00
Luca Deri 31c706c3db Improved telegram detection 2024-02-22 22:23:23 +01:00
Luca Deri 2399abd272 Added missing telegram networks 2024-02-22 22:04:54 +01:00
Vladimir Gavrilov 66b6e2b3f2 Add DLEP protocol dissector (#2326) 2024-02-20 16:05:41 +01:00
Ivan Nardi 4f2e642fbe make install: avoid copying private header (#2323) 2024-02-20 15:39:25 +01:00
Ivan Nardi a1280a2fd1 Move some defines (expecially log related) to the private header (#2324) 2024-02-20 15:35:38 +01:00
Ivan Nardi 12e142565e Add a script to download/update the domain suffix list (#2321) 2024-02-20 11:51:58 +01:00
Ivan Nardi 2f814c526b Add identification of Huawei generic and cloud traffic (#2325) 2024-02-20 11:51:40 +01:00
Ivan Nardi ffd6c3edee TLS: fix disabling of JA3C fingerprint (#2319) 2024-02-19 11:11:57 +01:00
Luca Deri 23536ff77a Improved modbus dissection to discard false positives 2024-02-16 15:48:34 +01:00
Luca Deri d1aed7b391 Indentation 2024-02-16 15:31:01 +01:00
Vladimir Gavrilov e93bcfd619 Add ANSI C12.22 protocol dissector (#2317)
* Add ANSI C12.22 protocol dissector

* Add UDP sample
2024-02-15 09:36:06 +01:00
Ivan Nardi ae36648c6c Skype: remove old detection logic (#1954)
Skype has been using standard protocols (STUN/ICE or TLS) for a long,
long time, now. Long gone are the days of Skype as a distribuited
protocol.

See: #2166
2024-02-12 18:39:24 +01:00
Nardi Ivan 0d36d9c8c2 Remove spurious call to exit() 2024-02-12 17:10:45 +01:00
Luca Deri 4072cb8862 Added stress test 2024-02-11 22:35:33 +01:00
Toni f4d7aa45fe Improved Polish gambling sites fetch script. (#2315)
* fails quite often in the CI, so ignore potential xmllint error

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-10 13:33:56 +01:00
Ivan Nardi ad25affcb7 reader_util: fix GRE detunneling (#2314) 2024-02-10 09:16:27 +01:00
Ivan Nardi 31f0608b9a TLS: add configuration of JA* fingerprints (#2313) 2024-02-10 09:16:13 +01:00
Ivan Nardi 03ecb026ff fuzz: improve fuzzing coverage (#2309) 2024-02-09 19:19:03 +01:00
Vladimir Gavrilov f5cec001f3 Add detection of Gaijin Entertainment games (#2311)
* Add detection of Gaijin Entertainment games

* Short NDPI_PROTOCOL_GAIJINENTERTAINMENT to NDPI_PROTOCOL_GAIJIN

* Add default UDP port for Gaijin Entertainment games

* Remove NDPI_PROTOCOL_CROSSOUT protocol id
2024-02-09 11:07:48 +01:00
Ivan Nardi f352e4f77b Improve normalization of flow->host_server_name (#2310)
Follow-up of 4543385d10

Remove trailing spaces for any HTTP header (we already remove leading
spaces)

We want:
* a "normalized" string in `flow->host_server_name`, but
* to parse the original string for flow risk checking

`ndpi_hostname_sni_set()` is a private function, so there is no need to
export its flags.
2024-02-09 09:20:02 +01:00
Vladimir Gavrilov 2abb68c8ca Add new AppsFlyer domain (#2307) 2024-02-08 08:10:55 +01:00
Vladimir Gavrilov 6207be43fa Add TencentGames protocol dissector (#2306) 2024-02-08 08:10:35 +01:00
Vitaly Lavrov 4543385d10 Normalization of host_server_name (#2299)
* Normalization of host_server_name

The ndpi_hostname_sni_set() function replaces all non-printable
characters with the "?" character and removing whitespace characters
at the end of the line.

* Added conditional hostname normalization.
2024-02-05 19:44:45 +01:00
Luca Deri a4da4467ac Fixed incompatibity with RH7 introduced by https://github.com/ntop/nDPI/commit/02030ac16e5016b00e0da6ff7512d97751bf87d2 2024-02-05 18:12:42 +01:00
Luca Deri b86452f2b1 Removed un-necessary include that caused configure-based applications sitting on top of nDPI to faile at compilation 2024-02-05 17:01:21 +01:00
Ivan Nardi 66371c202f Fix ndpi_get_lru_cache_stats() (#2303)
Found while fuzzing
2024-02-05 12:38:33 +01:00
Toni f2b09014f9 Serialize JA4 to JSON. (#2305)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-05 12:35:38 +01:00
Toni 02030ac16e Build RPM package in the CI. (#2304)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-05 12:13:09 +01:00
Luca Deri b15b877c51 Minor hash improvement 2024-02-04 20:23:49 +01:00
Toni b59994fa20 Completly disable all pthread related code in the library if USE_GLOBAL_CONTEXT macro is not defined. (#2302)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-03 11:53:49 +01:00
Ivan Nardi 9af2b9d815 collectd: fix extraction of hostname (#2301) 2024-02-03 04:11:58 +01:00
Toni ede25cc2b3 Improve ndpi_set_config error printing. (#2300)
* exit `ndpiReader` if a invalid configuration setting detected

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-02 15:15:30 +01:00
Toni f8aa642c3c Simplify and fix JA4 string computation. (#2298)
* additional JA4 string buffer is not needed and may cause a string truncation warning

```
protocols/tls.c: In function ‘ndpi_compute_ja4’:
protocols/tls.c:1738:3: warning: ‘strncpy’ output may be truncated copying 36 bytes from a string of length 1023 [-Wstringop-truncation]
 1738 |   strncpy(flow->protos.tls_quic.ja4_client, ja_str, 36);
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-02 14:25:43 +01:00
Ivan Nardi 13212711ab CIP: fix infinite-loop (#2295)
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=66342
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=66320
2024-02-01 20:30:43 +01:00
Ivan Nardi 8fbef7fb5a GitHub Actions: update to latest macOS runners (#2293)
Add a simple job with macos-14 on M1.
https://github.blog/changelog/2024-01-30-github-actions-introducing-the-new-m1-macos-runner-available-to-open-source/)
There are some issues with external dependencies (they are installed but
autoconf script doens;t find them) so keep it simple.

On macos-13 it seems that:
* there is no `realpath` program (even if coreutils has been
installed...)
* most of the filesystem is read only (we can't write on /usr/lib).
So I change
```
make install DESTDIR=$(realpath _install)
ls -alhHR _install
```
to
```
DESTDIR=/tmp/ndpi make install
ls -alhHR /tmp/ndpi
```
for all the jobs

Fix a warning on GitHub logs:
```
Node.js 16 actions are deprecated. Please update the following actions
to use Node.js 20: actions/checkout@v3. For more information see:
https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
```
2024-02-01 20:30:27 +01:00
Vladimir Gavrilov 4c9446379f Add Gearman protocol dissector (#2297) 2024-02-01 19:46:57 +01:00
Ivan Nardi 6c5ecae607 Fix function parameters check on domain code (#2296)
Found while fuzzing fuzz_config
2024-02-01 17:06:16 +01:00
Nardi Ivan c78ed93ec3 Sync unit tests results 2024-02-01 15:45:36 +01:00
Ivan Nardi 400cd516b5 Allow multiple struct ndpi_detection_module_struct to share some state (#2271)
Add the concept of "global context".

Right now every instance of `struct ndpi_detection_module_struct` (we
will call it "local context" in this description) is completely
independent from each other. This provide optimal performances in
multithreaded environment, where we pin each local context to a thread,
and each thread to a specific CPU core: we don't have any data shared
across the cores.

Each local context has, internally, also some information correlating
**different** flows; something like:
```
if flow1 (PeerA <-> Peer B) is PROTOCOL_X; then
  flow2 (PeerC <-> PeerD) will be PROTOCOL_Y
```
To get optimal classification results, both flow1 and flow2 must be
processed by the same local context. This is not an issue at all in the far
most common scenario where there is only one local context, but it might
be impractical in some more complex scenarios.

Create the concept of "global context": multiple local contexts can use
the same global context and share some data (structures) using it.
This way the data correlating multiple flows can be read/write from
different local contexts.
This is an optional feature, disabled by default.

Obviously data structures shared in a global context must be thread safe.
This PR updates the code of the LRU implementation to be, optionally,
thread safe.

Right now, only the LRU caches can be shared; the other main structures
(trees and automas) are basically read-only: there is little sense in
sharing them. Furthermore, these structures don't have any information
correlating multiple flows.

Every LRU cache can be shared, independently from the others, via
`ndpi_set_config(ndpi_struct, NULL, "lru.$CACHE_NAME.scope", "1")`.

It's up to the user to find the right trade-off between performances
(i.e. without shared data) and classification results (i.e. with some
shared data among the local contexts), depending on the specific traffic
patterns and on the algorithms used to balance the flows across the
threads/cores/local contexts.

Add some basic examples of library initialization in
`doc/library_initialization.md`.

This code needs libpthread as external dependency. It shouldn't be a big
issue; however a configure flag has been added to disable global context
support. A new CI job has been added to test it.

TODO: we should need to find a proper way to add some tests on
multithreaded enviroment... not an easy task...

*** API changes ***

If you are not interested in this feature, simply add a NULL parameter to
any `ndpi_init_detection_module()` calls.
2024-02-01 15:33:11 +01:00
Toni 44c2e59661 Provide a u64 wrapper for ndpi_set_config() (#2292)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-30 17:13:35 +01:00
Toni Uhlig d2f22d1308 Updated results (missing cip_io.pcap.out)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-29 20:49:47 +01:00
Luca Deri 47f72443fa Implemented CIP I/O (UDP version of the CIP protocol), Common Industrial protocol 2024-01-29 13:28:41 +01:00
Luca Deri 6c08601c3d Updated result 2024-01-29 12:27:52 +01:00
Luca Deri 426494b4ba Fixes invalid skype detection on UDP 2024-01-29 12:27:52 +01:00
Luca Deri 14cfa174be Added public_suffix_list.dat in RPM package 2024-01-29 11:48:39 +01:00
Luca Deri fb58f838e5 Added public_suffix_list.dat in packages 2024-01-29 11:43:23 +01:00
Luca Deri ae501e2503 Enhanced exception lists for binary application transfer 2024-01-29 11:37:04 +01:00
Ivan Nardi 92c2ac5a0f fuzz: fuzz_config: try restoring good coverage (#2291)
Last changes reduce fuzzing coverage of this fuzzer :(
2024-01-29 10:53:28 +01:00
Vladimir Gavrilov fb095a339d Add missing NDPI_LOG_DBG in some dissectors (#2290)
* Add missing NDPI_LOG_DBG

* Add missing NDPI_LOG_DBG in yojimbo.c
2024-01-29 09:25:29 +01:00
Vladimir Gavrilov c807d84054 Fix RESP detection (#2289)
* Rename redis_net.c to resp.c

* Fix RESP detection
2024-01-27 21:19:34 +01:00
Luca Deri 8e562c4ecd Removed check fir skipping .arpa domains 2024-01-27 20:54:46 +01:00
Luca Deri ec83a592b8 Check DGA on domain name (as it should be) rather than on the host name 2024-01-27 20:46:00 +01:00
Luca Deri 7a7df2b7d7 Fixed warning 2024-01-27 20:45:52 +01:00
Luca Deri 65b9c68d7d Fixed loading of non-ICANN domains that caused false positives with ndpi_load_domain_suffixes
Minor hash optimization
2024-01-27 20:40:27 +01:00
Toni bcca89b78c Add Raft protocol dissector. (#2286)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-25 20:26:18 +01:00
Vladimir Gavrilov 9a0affc6aa Improve MySQL detection (#2279)
* Improve MySQL detection

* Update copyright
2024-01-25 11:12:08 +01:00
Ivan Nardi b3efa7d3fe fuzz: fuzz_config: we need bigegr inputs (#2285) 2024-01-25 09:58:20 +01:00
Vladimir Gavrilov 4e712e3ab5 Add Radmin protocol dissector (#2283)
* Add Radmin protocol dissector

* Update test results
2024-01-25 08:10:29 +01:00
Ivan Nardi d577508727 fuzz: extend fuzzing coverage (#2281) 2024-01-24 21:16:58 +01:00
Ivan Nardi 7a83a8dc91 QUIC: fix decryption with CH fragments with different Destination CID (#2278)
QUIC decryption fails when the Client Hello is split into multiple UDP
packets and these packets have different Destination Connection IDs
(because the server told the client to switch to a different CID; see
RFC 9000 7.2)

```
The Destination Connection ID field from the first Initial packet sent by
a client is used to determine packet protection keys for Initial packets.
[..]
Upon first receiving an Initial or Retry packet from the server, the
client uses the Source Connection ID supplied by the server as the
Destination Connection ID for subsequent packets

```

From a logical point of view, the ciphers used for decryption should be
initialized only once, with the first Initial pkt sent by the client and
kept for later usage with the following packets (if any).

However it seems that we can safely initialize them at each packet, if
we keep using the DCID of the **first** packet sent by the client.
Keep initializing the ciphers at each packet greatly simplifie this patch.

This issue has been undetected for so long because:
* in the vast majority of the cases we only decrypt one packet per flow;
* the available traces with the Client Hello split into multiple packets
(i.e. cases where we need to decrypt at least two packets per flow) were
created in a simple test environment to simulate Post-Quantum handshake,
and in that scenario the client sent all the packets (with the same
DCID) before any reply from the server.

However, in the last months all major browsers started supporting PQ
key, so it is now common to have split CH in real traffic.

Please note that in the attached example, the CH is split into 2
(in-order) fragments (in different UDP packets) and the second one in
turn is divided into 9 (out-of-order) CRYPTO frames; the reassembler
code works out-of-the-box even in this (new) scenario.
2024-01-24 09:57:28 +01:00
Vladimir Gavrilov f04b4450a1 Add STOMP protocol dissector (#2280) 2024-01-23 21:08:13 +01:00
Ivan Nardi 9b26e74bb7 example: rework code between ndpiReader.c and reader_util.c (#2273) 2024-01-22 18:12:06 +01:00
Ivan Nardi 82e8bf91dd Improve handling of custom rules (#2276)
Avoid collisions between user-ids and internal-ids protocols in the
`example/protos.txt` file.
Add a new value for the classification confidence:
`NDPI_CONFIDENCE_CUSTOM_RULE`

With `./example/ndpiReader -p example/protos.txt -H` we now see also the
custom protocols and their internal/external ids:

```
nDPI supported protocols:
 Id Userd-id Protocol               Layer_4    Nw_Proto Breed        Category
  0        0 Unknown                TCP        X        Unrated      Unspecified

...

387      387 Mumble                 UDP        X        Fun          VoIP
388      388 iSCSI                  TCP                 Acceptable   Unspecified
389      389 Kibana                 TCP                 Acceptable   Unspecified
390      390 TestProto              TCP                 Acceptable   Unspecified
391      391 HomeRouter             TCP                 Acceptable   Unspecified
392      392 CustomProtocol         TCP                 Acceptable   Unspecified
393      393 AmazonPrime            TCP                 Acceptable   Unspecified
394      394 CustomProtocolA        TCP                 Acceptable   Unspecified
395      395 CustomProtocolB        TCP                 Acceptable   Unspecified
396      800 CustomProtocolC        TCP                 Acceptable   Unspecified
397     1024 CustomProtocolD        TCP                 Acceptable   Unspecified
398     2048 CustomProtocolE        TCP                 Acceptable   Unspecified
399     2049 CustomProtocolF        TCP                 Acceptable   Unspecified
400     2050 CustomProtocolG        TCP                 Acceptable   Unspecified
401    65535 CustomProtocolH        TCP                 Acceptable   Unspecified
```

We likely need to take a better look in general at the iteration between
internal and external protocols ids...

This PR fixes the issue observed in
https://github.com/ntop/nDPI/pull/2274#discussion_r1460674874 and in
https://github.com/ntop/nDPI/pull/2275.
2024-01-21 19:53:32 +01:00
Vladimir Gavrilov 5620e10742 Add ElectronicArts detection support (#2274)
* Add ElectronicArts detection support

* Merge electronicarts.pcapng into sites.pcapng
2024-01-21 18:58:12 +01:00
Toni 7d24e1258d Add Yojimbo (netcode) protocol dissector (#2277)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-21 17:47:42 +01:00
Ivan Nardi 49591dda9b Zoom: improve detection (#2270) 2024-01-21 12:17:34 +01:00
Ivan Nardi 42d23cff6a config: follow-up (#2268)
Some changes in the parameters names.
Add a fuzzer to fuzz the configuration file format.
Add the infrastructure to configuratin callbacks.
Add an helper to map LRU cache indexes to names.
2024-01-20 16:14:41 +01:00
Vladimir Gavrilov 8651ce9811 Fix detection of new Cassandra versions (#2272)
* Fix detection of new Cassandra versions

* Add Cassandra Internode Communication protocol support

* Add default port for Cassandra Internode Communication protocol
2024-01-20 15:58:14 +01:00
Ivan Nardi eb129297e9 Add a dedicated dissector for Zoom (#2265)
Move it from the RTP code and extend it
2024-01-19 10:01:38 +01:00
Ivan Nardi c83698c957 STUN: fix flow risks when DTLS packets are found (#2266)
When switching to (D)TLS dissector from the STUN one, we need to clear
any flow risks set from the latter (because we don't have anymore
`NDPI_PROTOCOL_STUN` in the classification results)
2024-01-19 08:57:39 +01:00
Vladimir Gavrilov 6ac2ce84f8 Add Mumble detection support (#2269) 2024-01-19 07:46:51 +01:00
Vladimir Gavrilov 248f3d5588 Rework Steam detection (part 1) (#2264)
* Clean up Steam dissector

* Add Steam Datagram Relay dissector

* Update docs

* Update test results

* Remove csgo.c from MSVC project

* Small fixes

* Add Steam TLS pcap sample

* Merge Steam pcap samples into single one

* Fix typo

* Update test results
2024-01-18 19:23:44 +01:00
Nardi Ivan 0712d496fe config: allow configuration of guessing algorithms 2024-01-18 10:21:24 +01:00
Nardi Ivan 6c85f10cd5 config: move debug/log configuration to the new API 2024-01-18 10:21:24 +01:00
Nardi Ivan c704be1a20 config: DNS: add two configuration options
* Enable/disable sub-classification of DNS flows
* Enable/disable processing of DNS responses
2024-01-18 10:21:24 +01:00
Nardi Ivan 950f209a17 config: HTTP: enable/disable processing of HTTP responses 2024-01-18 10:21:24 +01:00
Nardi Ivan c669044a44 config: configure TLS certificate expiration with the new API 2024-01-18 10:21:24 +01:00
Nardi Ivan 88720331ae config: remove enum ndpi_prefs 2024-01-18 10:21:24 +01:00
Nardi Ivan 1289951b32 config: remove ndpi_set_detection_preferences() 2024-01-18 10:21:24 +01:00
Nardi Ivan 311d8b6dae config: move cfg of aggressiviness and opportunistic TLS to the new API 2024-01-18 10:21:24 +01:00
Nardi Ivan 4cbe2674ab config: move IP lists configurations to the new API 2024-01-18 10:21:24 +01:00
Nardi Ivan f55358973f config: move LRU cache configurations to the new API 2024-01-18 10:21:24 +01:00
Nardi Ivan 3107a95881 Make ndpi_finalize_initialization() returns an error code
We should check if the initialization was fine or not
2024-01-18 10:21:24 +01:00
Nardi Ivan d72a760ac3 New API for library configuration
This is the first step into providing (more) configuration options in nDPI.

The idea is to have a simple way to configure (most of) nDPI: only one
function (`ndpi_set_config()`) to set any configuration parameters
(in the present or on in the future) and we try to keep this function
prototype as agnostic as possible.

You can configure the library:
* via API, using `ndpi_set_config()`
* via a configuration file, in a text format

This way, anytime we need to add a new configuration parameter:
* we don't need to add two public functions (a getter and a setter)
* we don't break API/ABI compatibility of the library; even changing
the parameter type (from integer to a list of integer, for example)
doesn't break the compatibility.

The complete list of configuration options is provided in
`doc/configuration_parameters.md`.

As a first example, two configuration knobs are provided:
* the ability to enable/disable the extraction of the sha1 fingerprint of
the TLS certificates.
* the upper limit on the number of packets per flow that will be subject
to inspection
2024-01-18 10:21:24 +01:00
Luca ca7df1db82 Improved ndpi_get_host_domain 2024-01-16 07:25:03 +01:00
Luca 1637a991a4 Added ndpi_get_host_domain() for returning the host domain
vs ndpi_get_host_domain_prefix() that instead returnd the host TLD
2024-01-16 06:56:51 +01:00
Ivan Nardi 111015b872 ndpiReader: improve the check on max number of pkts processed per flow (#2261)
Allow to disable this check.

I don't know how much sense these limits have in the application
(especially with those default values...) since we have always had a
hard limit on the library itself (`max_packets_to_process` set to 32).
The only value might be that they provide different limits for TCP and
UDP traffic.

Keep them for the time being...
2024-01-15 20:12:57 +01:00
Nardi Ivan 0891ad0e77 Added missing lists file to the distribution tarball 2024-01-15 20:00:36 +01:00
Nardi Ivan b22fa558ff ndpiReader: fix memory leak
Change the working directory of `ndpiReader` in the Github Actions so
that it can load the domain suffix list during `domainsUnitTest()`
2024-01-15 19:49:27 +01:00
Luca d29c2f3ad1 Added ndpi tracing 2024-01-15 19:26:28 +01:00
Luca a1634604e1 Removes extraneous parentheses that caused macOS to complain 2024-01-15 19:12:31 +01:00
Luca 162c38f18f Added new API calls
- ndpi_load_domain_suffixes()
- ndpi_get_host_domain_suffix()

whose goal is to find the domain name of a hostname. Example:

www.bbc.co.uk   -> co.uk
mail.apple.com  -> com
2024-01-15 19:03:46 +01:00
Ivan Nardi 61a18623e7 Bittorrent: improve detection of UTPv1 (#2259)
* fix header length check: we must take into account optional extension
header
* it seems the check on UTPv1 header is weak only in the DATA case: wait
for multiple packets only in that case
2024-01-12 18:53:24 +01:00
Ivan Nardi dd8be1fcb1 Fix some warnings reported by CODESonar (#2227)
Remove some unreached/duplicated code.

Add error checking for `atoi()` calls.

About `isdigit()` and similar functions. The warning reported is:
```
Negative Character Value help
isdigit() is invoked here with an argument of signed type char, but only
has defined behavior for int arguments that are either representable
as unsigned char or equal to the value of macro EOF(-1).
Casting the argument to unsigned char will avoid the undefined behavior.
In a number of libc implementations, isdigit() is implemented using lookup
tables (arrays): passing in a negative value can result in a read underrun.
```
Switching to our macros fix that.
Add a check to `check_symbols.sh` to avoid using the original functions
from libc.
2024-01-12 13:30:43 +01:00
Toni 0aea509e23 Add KCP protocol dissector. (#2257)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-12 12:14:59 +01:00
Vladimir Gavrilov 75d3f78f7d Add more tracker/ad hosts (#2256) 2024-01-12 10:44:31 +01:00
Toni 7b2bbb2309 Improved uTorrent via utp (TCP-like streams over UDP). (#2255)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-11 16:49:25 +01:00
Luca Deri 30c2386ee2 Added missing (C) 2024-01-09 22:27:41 +01:00
Luca Deri 4f9a08cf05 Moved ndpi_private.h to include directory 2024-01-09 22:00:40 +01:00
Vladimir Gavrilov 3d57dec6b4 Add PIA (Private Internet Access) support (#2250)
* Fix typo in protocols.rst

* Add PIA (Private Internet Access) support

* Update ndpi_main.c
2024-01-09 18:28:57 +01:00
Vladimir Gavrilov ead5bf2adb Add more adult content hostnames (#2247) 2024-01-09 07:57:01 +01:00
Toni c5b0b05b80 Add Roughtime protocol dissector. (#2248)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-09 07:56:27 +01:00
Toni 6c3d162cd6 Add realtime protocol output to ndpiReader. (#2197)
* support for using a new flow callback invoked before the flow memory is free'd
 * minor fixes

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-09 00:39:59 +01:00
Ivan Nardi 90b999e3f1 Fix default port conflicts between AJP and CiscoVPN (#2245)
We shouldn't have the same default port associated with multiple
protocols. The key reason is that we use only one of them (for
classification by-port, for example) and which one we really use
depends on the internal order of the protocolsi's initialization (which
is unknown to the user).

We have 3 port collisions:
* 8009: AJP and CiscoVPN
* 445: SMBv1 and SMBv23
* 8080: HTTP_Connect and HTTP_Proxy
I think that the two last cases don't really matter...

About the first one: it seems that CiscoVPN is retired from 2011/2014
(see
https://www.cisco.com/c/en/us/obsolete/security/cisco-vpn-client.html)
and superseded by AnyConnect, which uses different ports (see
https://community.cisco.com/t5/network-security/what-protocols-are-used-on-anyconnect-mobility-client/td-p/4713525).
Therefore it should be safe to remove 8009 from the CiscoVPN default
ports.
2024-01-08 17:04:46 +01:00
Ivan Nardi 8321b79539 Make some test traces smaller (#2243)
Having smaller traces help fuzzing: we want the fuzzers to mutate
"interesting" packets analyzed by nDPI, i.e. the first packets of each
flows.
Try hard to keep the same classification and extraction capabilities
2024-01-08 17:04:12 +01:00
Vladimir Gavrilov ce08291ccd Add Google Chat support (#2244) 2024-01-07 18:04:59 +01:00
Ivan Nardi 40797521af ndpiReader: add breed stats on output used for CI (#2236) 2024-01-05 13:02:39 +01:00
Vladimir Gavrilov 3d09b25653 Add Ceph protocol dissector (#2242)
* Add Ceph protocol dissector

* Update protocols.rst
2024-01-04 13:22:23 +01:00
Ivan Nardi f23e9dc7bb Add an implementation of the BSD function strtonum (#2238)
The main difference with the original function is that we allow to
specify the base.
Credit for the original idea and the first implementation to @0xA50C1A1
2024-01-04 13:16:39 +01:00
Vladimir Gavrilov 7f9973bd0c Add HL7 protocol dissector (#2240)
* Add HL7 protocol dissector

* Small fixes

* Small fixes
2024-01-02 20:57:05 +01:00
Ivan Nardi 88d1416b70 STUN: fix detection of Google Meet over IPv6 (#2241) 2024-01-02 19:30:59 +01:00
Vladimir GavrilovandNardi Ivan 0180c1f04a Add IEC62056 (DLMS/COSEM) protocol dissector (#2229)
* Add IEC62056 (DLMS/COSEM) protocol dissector

* Fix detection on big endian architectures

* Update protocols.rst

* Add ndpi_crc16_x25 to fuzz/fuzz_alg_crc32_md5.c

* Update pcap sample

* Remove empty .out file

* iec62056: add some documentation

---------

Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2024-01-02 16:45:54 +01:00
Ivan Nardi 3c7ed34ce9 fuzz: improve fuzzing coverage (#2239) 2024-01-02 15:22:44 +01:00
Vladimir Gavrilov 0f4d9f5054 Remove Google Hangouts/Duo stuff (#2233)
* Remove Google Hangouts/Duo support

* Update protocols.rst
2024-01-02 14:01:33 +01:00
Ivan Nardi 3dbd89eee8 sha256: fix undefined-shift error (#2237)
```
third_party/src/ndpi_sha256.c:51:21: runtime error: left shift of 128 by 24 places cannot be represented in type 'int'
```

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65305
2024-01-02 12:29:20 +01:00
Ivan Nardi 308f71a6e8 TLS: fix heap-buffer-overflow (#2235)
```
==53992==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x511000007e04 at pc 0x555da2165fd0 bp 0x7ffddf7e3990 sp 0x7ffddf7e3988
READ of size 2 at 0x511000007e04 thread T0
    #0 0x555da2165fcf in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2384:50
    #1 0x555da217c31f in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:908:5
    #2 0x555da2176720 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1273:11
    #3 0x555da215a628 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2883:5
    #4 0x555da1e95c30 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6720:6
    #5 0x555da1e969f3 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6756:10
    #6 0x555da1e96394 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6789:12
    #7 0x555da1ea7991 in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7929:15
    #8 0x555da1ea547f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8104:22
    #9 0x555da1de137f in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1721:31
    #10 0x555da1de137f in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2438:1
```

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65362
2024-01-02 11:37:40 +01:00
Ivan Nardi d886a6107f Teamviewer: varius fixes (#2228)
We already have a generic (and up to date) logic to handle ip addresses:
remove that stale list.

Teamviewer uses TCP and UDP, both; we can't access `flow->l4.udp`.

According to a comment, we set the flow risk
`NDPI_DESKTOP_OR_FILE_SHARING_SESSION` only for the UDP flows.
2024-01-02 11:22:43 +01:00
Vladimir Gavrilov 2796bc9b47 Add NoMachine NX protocol dissector (#2234)
* Add NoMachine protocol dissector

* Fix detection on big endian architectures

* Make NoMachine over UDP check more strict

* Small fixes
2024-01-02 10:23:42 +01:00
Luca 2f657cb8f9 Implemented ndpi_is_outlier() for detecting outliers using z-score 2023-12-28 19:59:54 +01:00
Luca Deri 1366518bff Implements ndpi_pearson_correlation for measuring how correlated are two series 2023-12-27 22:42:37 +01:00
Luca Deri 99d4838328 New ndpi_sha256() nDPI API call (#2230)
* Added ndpi_sha256.c to the Windows project

* Added ndpi_sha256() nDPI API call
2023-12-23 10:47:28 +01:00
Luca Deri f2be972224 Removed type redefinition 2023-12-23 09:16:35 +01:00
Luca Deri caa223d2ba Changes to avoid type redefinition on windows 2023-12-23 09:15:42 +01:00
Luca Deri 5c7200f2bb Buffer overflow fix 2023-12-23 08:07:28 +01:00
Luca Deri 7c4be6d077 Updated result 2023-12-22 21:29:45 +01:00
Luca Deri 53b9ae23e8 Warning fix 2023-12-22 21:10:03 +01:00
Luca Deri cbea1b2203 Fixed check (when using ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --with-thread-sanitizer --with-pcre2 --with-maxminddb --enable-tls-sigs) 2023-12-22 21:01:14 +01:00
Luca Deri a95d4744b6 Fixes nDPI datastructs size check 2023-12-22 20:52:56 +01:00
Luca Deri 8285fffdae Implements JA4 Support (#2191) 2023-12-22 20:40:42 +01:00
Luca Deri b90c18e906 Changes to make room for JA4 2023-12-22 20:40:42 +01:00
Vladimir Gavrilov 5eb468d07b Add Apache Kafka protocol dissector (#2226) 2023-12-22 14:42:47 +01:00
Vladimir Gavrilov 6fc8aa4e61 Add WebDAV detection support (#2224)
* Add WebDAV detection support

* Add pcap example

* Update test results

* Remove redundant checks

* Add WebDAV related HTTP methods to fuzz/dictionary.dict

* Add note about WebDAV
2023-12-22 13:23:37 +01:00
Luca 41aa2f96b6 Various MDNS flow risks fixes 2023-12-21 12:24:30 +01:00
Luca 28b1ce41cb MDNS fix 2023-12-21 11:11:28 +01:00
Ivan Nardi a5595d16c0 CI: update list of compilers (#2223)
Try using latest gcc and clang versions.
We still care about RHEL7: since handling a RHEL7 runner on GitHub is
quite complex, let try to use a similar version of gcc, at least
2023-12-20 19:22:22 +01:00
Vladimir Gavrilov 149067b3fc Add JSON-RPC protocol dissector (#2217)
* Add JSON-RPC protocol dissector

* Small fixes

* Improve detection
2023-12-20 12:42:25 +01:00
Vladimir Gavrilov 33f11cb10f Add OpenFlow protocol dissector (#2222) 2023-12-20 10:48:45 +01:00
Ivan Nardi 8aa09f9c99 mining: a better identification logic (#2221)
It is quite simple (and not so efficient) but it should fix all the
false positives reported in #2216. Add support for Ethereum mining.

Merge all the mining traces.

Remove duplicated function.

Close #2216
2023-12-20 10:46:57 +01:00
Ivan Nardi 308b266333 fuzz: improve fuzzing coverage (#2220) 2023-12-19 20:33:08 +01:00
Vladimir Gavrilov 6af93c09c8 Rename NDPI_PROTOCOL_RPC to NDPI_PROTOCOL_MS_RPCH (#2218)
* Rename NDPI_PROTOCOL_RPC to NDPI_PROTOCOL_MS_RPCH

* Add protocol description

* Improve MS-RPCH detection
2023-12-19 11:56:26 +01:00
Vladimir Gavrilov 59c8eabc0e Add UFTP protocol dissector (#2215)
* Add UFTP protocol dissector

* Update docs

* Merge pcap files
2023-12-18 11:21:07 +01:00
Vladimir Gavrilov d8c7a76611 Add HiSLIP protocol dissector (#2214)
* Add HiSLIP protocol dissector

* Fix error
2023-12-17 11:52:55 +01:00
Vladimir Gavrilov 0f3e6d832b Add PROFINET/IO protocol dissector (#2213)
* Add PROFINET/IO protocol dissector

* Add LE (Little Endian) to the file name

* Rework dissector

* Remove redundant check
2023-12-16 13:30:21 +01:00
Ivan Nardi 8ea16ffd77 TLS: be sure to always set a category value (#2206)
We probably need to check everywhere we set the category in the
"extra dissection" data path: a flow with a proper classification can't
have NDPI_PROTOCOL_CATEGORY_UNSPECIFIED as category.
2023-12-14 08:07:04 +01:00
Toni ef62391dba Add Monero protocol classification. (#2196)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-12-13 19:55:18 +01:00
Ivan Nardi bda037c57f Sync unit tests results after latest merge (#2211) 2023-12-13 17:47:20 +01:00
Ivan Nardi 193f28582b QUIC: add heuristic to detect unidirectional *G*QUIC flows (#2207)
Fix extraction of `flow->protos.tls_quic.quic_version` metadata.
2023-12-13 17:14:04 +01:00
Ivan Nardi 8e14aac5e0 ndpiReader: avoid creating two detection modules when processing traffic/traces (#2209) 2023-12-12 19:44:29 +01:00
Ivan Nardi 241c42ad7e ndpiReader: fix guessed_flow_protocols statistic (#2203)
Increment the counter only if the flow has been guessed
2023-12-12 19:44:03 +01:00
Ivan Nardi 7b5354588b fuzz: extend fuzzing coverage (#2208) 2023-12-11 19:24:17 +01:00
Ivan Nardi b3f2b1bb7f STUN: rework extra dissection (#2202)
Keep looking for RTP packets but remove the monitoring concept.
We will re-introduce a more general concept of "flow in monitoring
state" later.
The function was disabled by default.
Some configuration knobs will be provided when/if #2190 is merged.
2023-12-11 14:53:12 +01:00
Ivan Nardi 673b6e7345 HTTP: faster processing of asymmetric flows (#2198) 2023-12-11 14:52:31 +01:00
Ivan Nardi adf8982d8e fuzz: extend fuzzing coverage (#2205) 2023-12-11 12:48:50 +01:00
rl1987 59d476195c Fix typos (#2204)
* Fix typo in ndpiSimpleIntegration.c

* Fix misspelling in a comment
2023-12-10 19:58:22 +01:00
Vladimir Gavrilov c809e7c069 Replace complicated TPKT header validation with an helper function (#2201)
* Replace complicated TPKT header validation with an helper function

* Move tpkt_verify_hdr function definition to ndpi_utils.c
2023-12-10 12:10:50 +01:00
Ivan Nardi f74cf16c36 OpenVPN: rework detection (#2199)
Close #1873
2023-12-06 10:24:26 +01:00
Vladimir Gavrilov ad20846fad Add Ether-S-Bus protocol dissector (#2200) 2023-12-05 17:20:38 +01:00
Vladimir Gavrilov 72601a69ff Add some fast CRC16 algorithms implementation (#2195)
* Add some fast CRC16 algorithms implementation

* Update ndpi_crc.c

* Move crc16 stuff to ndpi_analyze.c

* IEEE C37.118: use new fast CRC-16/CCITT-FALSE implementation
2023-12-05 15:58:11 +01:00
Vladimir Gavrilov be50493f44 Add IEEE C37.118 protocol dissector (#2193) 2023-12-05 08:06:15 +01:00
Ivan Nardi 7b0c16a70d TLS: remove JA3+ fingerprints. (#2192)
See: #2191
2023-12-05 08:05:44 +01:00
Vladimir Gavrilov c34bded4ef Add ISO 9506-1 MMS protocol dissector (#2189)
* Add ISO 9506-1 MMS protocol dissector
* Fix detection on big-endian architectures
2023-12-01 09:03:07 +01:00
Ivan Nardi 983b8e8eee STUN: parsing of DATA attribute (#2179) 2023-12-01 07:01:49 +01:00
Vladimir Gavrilov 24df1913ac Add Beckhoff ADS protocol dissector (#2181)
* Add Beckhoff ADS protocol dissector

* Remove redundant le32toh

* Fix detection on big-endian architectures
2023-11-30 09:13:45 +01:00
Ivan Nardi 6f046df0dc STUN: fix detection of DTLS (#2187)
Fix a memory leak
```
==97697==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x55a6967cfa7e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x701a7e) (BuildId: c7124999fa1ccc54346fa7bd536d8eab88c3ea01)
    #1 0x55a696972ab5 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
    #2 0x55a696972da0 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13
    #3 0x55a696b7658d in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2394:46
    #4 0x55a696b86e81 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:897:5
    #5 0x55a696b80649 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1262:11
    #6 0x55a696b67a57 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2751:5
    #7 0x55a696b67758 in switch_to_tls /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1408:3
    #8 0x55a696c47810 in stun_search_again /home/ivan/svnrepos/nDPI/src/lib/protocols/stun.c:422:4
    #9 0x55a6968a22af in ndpi_process_extra_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7247:9
    #10 0x55a6968acd6f in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7746:5
    #11 0x55a6968aba3f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8013:22
    #12 0x55a69683d30e in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1723:31
    #13 0x55a69683d30e in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2440:10
    #14 0x55a69680f08f in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:135:7
[...]
SUMMARY: AddressSanitizer: 16 byte(s) leaked in 1 allocation(s).
```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64564
2023-11-30 09:09:40 +01:00
Ivan Nardi d85f8592b8 Rework extraction of HTTP headers (#2183)
Extract only the headers that we really need/use.
Avoid too many `strcmp`; the algorithm might be a little bit
overwhelming right now but it might be useful if we have further headers
in the future.
2023-11-29 19:44:25 +01:00
Ivan Nardi 1100326edb Fix FAQ (#2186) 2023-11-29 18:56:21 +01:00
Ivan Nardi d3d9770b96 Update FAQ.md 2023-11-29 18:52:35 +01:00
Ivan Nardi cc721fd3ce Update FAQ.md 2023-11-29 18:28:26 +01:00
Ivan Nardi 0114992f5f Add a FAQ for the project (#2185) 2023-11-29 18:25:36 +01:00
Ivan Nardi 3b35cb37d9 Keep separating public and private API (#2157)
See: b08c787fe
2023-11-29 17:13:00 +01:00
Ivan Nardi ac90b1f009 Fix detection of NDPI_TCP_ISSUES flow risk (#2177)
We need to take into account retransmissions: they increase
`flow->all_packets_counter` counter but not `flows->packet_counter`
one.
Therefore, the right way to check for 3WH + RST pattern involves checking
for `flows->packet_counter == 0`
2023-11-29 16:55:39 +01:00
Toni 5fb631c8fe Improved belgium gambling sites regex. (#2184)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-29 16:15:23 +01:00
Vladimir Gavrilov c60c03766c Add Schneider Electric’s UMAS detection support (#2180)
* Add Schneider Electric’s UMAS detection support

* Swap proto IDs in ndpi_set_detected_protocol

* Update unit test result
2023-11-28 18:03:00 +01:00
Vladimir Gavrilov 688a197b85 Fix FINS false positives (#2176)
* Fix FINS false positives

* Add rsv (reserved) field check
2023-11-27 21:46:14 +01:00
Vladimir Gavrilov ebb1bc2f34 Add Ether-S-I/O protocol dissector (#2174) 2023-11-27 19:04:05 +01:00
Vladimir Gavrilov 84427b0754 Add Omron FINS protocol dissector (#2172)
* Add Omron FINS protocol dissector

* Add a kludge to avoid invalid FINS over UDP detection as SkypeTeams and RTP

* Update unit test results

* Update protocols.rst

* Remove dummy flows from fins.pcap
2023-11-27 17:09:53 +01:00
3763c702f0 Rework S7Comm dissector; add S7Comm Plus support (#2165)
* Rework S7Comm dissector; add S7Comm Plus support

* Cleanup s7comm.c

* Improve S7Comm Plus detection

* s7comm/s7commplus: faster detection

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2023-11-27 14:37:48 +01:00
Vladimir Gavrilov 0b6e261523 Improve CORBA detection (#2167)
* Improve CORBA detection

* Remove dummy flow from ziop.pcap

* Merge ziop.pcap and miop.pcap into corba.pcap
2023-11-27 13:10:50 +01:00
Vladimir Gavrilov da629709f3 Add OPC UA protocol dissector (#2169) 2023-11-27 12:13:23 +01:00
Ivan Nardi 7ff22a7e3c STUN: improve demultiplexing of DTLS packets (#2153)
Keep demultiplexing STUN/RTP/RTCP packets after DTLS ones.

We might end up processing the session a little longer, because we will
process the STUN/RTP/RTCP packets after the DTLS handshake.
2023-11-27 11:10:38 +01:00
Vladimir Gavrilov 87399b3544 Add RTPS protocol dissector (#2168) 2023-11-27 07:17:39 +01:00
Vladimir Gavrilovand0xA50C1A1 27802b0134 Reduce false positives for H.323 over TCP (#2164)
Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
2023-11-23 17:29:00 +01:00
fbae51ae9d Get rid of RDP false positives (#2161)
* Get rid of false positives in the RDP protocol dissector

* Remove kludge for RDP

* RDP: improve detection

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2023-11-23 09:35:43 +01:00
Vladimir Gavrilovand0xA50C1A1 5c8c5c90c2 Add HART-IP protocol dissector (#2163)
* Add HART-IP protocol dissector

* Update docs

* Update protocols.rst

* Reuse free proto id and re-run tests

* docs: move HART-IP to top of list

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
2023-11-22 22:04:22 +01:00
Toni 21f2574033 Improved TFTP. Fixes #2075. (#2149)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-21 16:56:46 +01:00
Vladimir Gavrilovand0xA50C1A1 35abafec4f Get rid of Apache Cassandra false positives (#2159)
* Rewrite Apache Cassandra dissector

* Replace memcmp with strncmp

* Add payload length check

* Update Cassandra dissector

* Update test results

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
2023-11-21 16:56:01 +01:00
Vladimir Gavrilovand0xA50C1A1 ae6e6d61f0 Add IEEE 1588-2008 (PTPv2) dissector (#2156)
* Add IEEE 1588-2008 (PTPv2) dissector

PTPv2 is a time synchronization protocol in computer networks, similar to NTP.

* Add default protocol ports

* Update default test result for PTPv2

* Update copyright

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
2023-11-21 13:39:54 +01:00
Ivan Nardi 55664392a9 Ip address list: aggregate Mullvad and Tor lists too (#2154)
Missing from bdb73db1a
See #2150
2023-11-21 08:28:32 +01:00
Vladimir Gavrilovand0xA50C1A1 8d71998670 Remove Google+ support (#2155)
* Remove Google+ support

Google+ was discontiued in 2019, so I think that its protocol id can be freed for reuse.

* Fix typo

* Update tests

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
2023-11-21 08:14:20 +01:00
Ivan Nardi bdb73db1a4 IP lists: aggregate addresses wherever possible (#2152)
See #2150
2023-11-17 12:26:23 +01:00
Ivan Nardi 6c9571d9a9 Remove duplicate addreess list (#2151)
We are loading the same AS list as GOTO
See #2150
2023-11-16 20:08:06 +01:00
Maatuq 344ac69468 protocols/ethereum: enhance node discovery protocol detection. (#2145)
*  packet-type == 0x0 shouldn't be allowed.
* allow more packet-type e.g. ENRRequest, ENRResponse.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
2023-11-15 19:45:48 +01:00
Toni 38f9a74713 Added TeslaServices and improved TikTok host names. Fixes #2140. (#2144)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-10 11:25:10 +01:00
Toni 0cb6f4cb75 Fixed hash buffer size in ndpiSimpleIntegration. (#2143)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-10 10:23:37 +01:00
Ivan Nardi b08c787fe2 Have a clear distinction between public and private/internal API (#2137)
1) Public API/headers in `src/include/` [as it has always been]
2) Private API/headers in `src/lib/`

Try to keep the "ndpi_" prefix only for the public functions
2023-11-09 10:50:59 +01:00
Toni 0673da54b5 Fixed implicit u32 cast in ndpi_data_min() / ndpi_data_max(). (#2139)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-09 10:16:57 +01:00
Ivan Nardi b539b0d090 fuzz: improve coverage and remove dead code (#2135)
We are not able to remove custom rules: remove the empty stubs (which
originate from the original OpenDPI code).

`ndpi_guess_protocol_id()` is only called on the first packet of the
flow, so the bitmask `flow->excluded_protocol_bitmask` is always empty,
since we didn't call any dissectors yet.

Move another hash function to the dedicated source file.
2023-11-07 17:46:29 +01:00
Ivan Nardi 58a9e2d967 Remove struct ndpi_packet_struct from the public API (#2129)
Right now, the only instance of `struct ndpi_packet_struct` is embedded
into `struct ndpi_detection_module_struct`. Since the latter is a
private structure (because of `NDPI_LIB_COMPILATION` ) there is no way for
the application to get a pointer to `ndpi_struct->packet`.

Bottom line: the application can't use any API functions having `struct
ndpi_packet_struct *` as parameter. Remove them all (since they are
completly unused and unusable).

There are no public helper functions to initialize/populate/deinit a
`struct ndpi_packet_struct` object, so the application can't neither
create its own instance of this object.

Protect `struct ndpi_packet_struct` via the same define
`NDPI_LIB_COMPILATION`.
2023-11-07 15:30:56 +01:00
Luca Deri 0db12b1390 In case a pattern is redefined (for instance with protos.txt), the custom definition overwrites the previous one (so far it was ignored) 2023-11-02 20:46:14 +01:00
Toni 6dcecd73d3 Added malicious sites from the polish cert. (#2121)
* added handling of parsing errors

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-02 09:04:04 +01:00
Christian Marangi d5c9a16a48 Move from PCRE to PCRE2 (#2134)
Move from PCRE to PCRE2. PCRE is EOL and won't receive any security
updates anymore. Convert to PCRE2 by converting any function PCRE2 new
API.

Also update every entry in github workflows and README to point to the
new configure flag. (--with-pcre2)

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-11-01 14:16:49 +01:00
Ivan KapranovandIvan Kapranov 5a2666ce9e Fix proto_name and proto_id missmatch for Sina and SinaWeibo (#2131)
* minor fixes

fixed 'handle leak' in ndpi_load_malicious_sha1_file and removed the redundant comparison ndpi_search_eaq

* fix Stack overflow caused by invalid write in ndpi_automa_match_string_subprotocol

* fix compile errors

* fix

* Fix name missmatch for Sina and Sina Weibo

* fix

* add Sina Weibo to doc

* fix

* add Sina Weibo to doc

---------

Co-authored-by: Ivan Kapranov <i.kapranov@securitycode.ru>
2023-11-01 14:12:49 +01:00
Ivan Nardi cd59485f7e Custom rules: add missing check (#2132)
Fix an use-of-uninitialized-value error

```
==28646==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x55a9b7a3dea2 in ndpi_patricia_lookup /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ndpi_patricia.c:739:8
    #1 0x55a9b7442cac in add_to_ptree /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2588:10
    #2 0x55a9b7469290 in ndpi_add_ip_risk_mask /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4161:12
    #3 0x55a9b746cf14 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4297:11
    #4 0x55a9b74842ae in ndpi_load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4944:8
    #5 0x55a9b7424706 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:38:3
[...]
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=63754
2023-10-31 20:02:37 +01:00
Ivan Nardi 50006516b3 Custom rules: fixes for Windows (#2130)
Even after adding `WSAStartup()` call, the behavior with IPv6 addresses
was different on Windows; it is the same issue reported on:
https://stackoverflow.com/questions/66755681/getaddrinfo-c-on-windows-not-handling-ipv6-correctly-returning-error-code-1

Fix bitmasks.
2023-10-31 16:04:30 +01:00
Luca Deri 773e9b8170 Added pointer check 2023-10-31 15:48:06 +01:00
Luca Deri f49cedc1e8 Added missing freeaddrinfo() call 2023-10-30 22:58:17 +01:00
Luca Deri 5e9f10f820 Replaced gethostbyname2 with getaddrinfo (portable code) 2023-10-30 22:55:50 +01:00
Luca Deri c6af36990e Warning fixes 2023-10-30 13:10:28 +01:00
Ivan Nardi 42d24f8799 STUN: major code rework (#2116)
Try to have a faster classification, on first packet; use standard extra
dissection data path for sub-classification, metadata extraction and
monitoring.

STUN caches:
* use the proper confidence value
* lookup into the caches only once per flow, after having found a proper
STUN classification

Add identification of Telegram VoIP calls.
2023-10-30 10:28:19 +01:00
Ivan Nardi e399bd7e3d Custom rules: fix a stack overflow (#2128)
```
==19255==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f515bb3bf80 at pc 0x55796e01394a bp 0x7fff4fb5c050 sp 0x7fff4fb5b7e0
WRITE of size 58 at 0x7f515bb3bf80 thread T0
    #0 0x55796e013949 in scanf_common(void*, int, bool, char const*, __va_list_tag*) asan_interceptors.cpp.o
    #1 0x55796e0147df in __isoc99_sscanf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x77f7df) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)
    #2 0x55796e0fc74a in ndpi_add_host_ip_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2771:13
    #3 0x55796e0fb029 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4411:16
    #4 0x55796e103738 in ndpi_load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4901:8
    #5 0x55796e0ca96d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:38:3
    #6 0x55796dfd78e0 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x7428e0) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)
    #7 0x55796dfc0e93 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x72be93) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)
    #8 0x55796dfc6d96 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x731d96) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)
    #9 0x55796dff1672 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x75c672) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)
    #10 0x7f515df19082 in __libc_start_main /build/glibc-BHL3KM/glibc-2.31/csu/../csu/libc-start.c:308:16
    #11 0x55796dfbbb0d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x726b0d) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)

Address 0x7f515bb3bf80 is located in stack of thread T0 at offset 128 in frame
    #0 0x55796e0fb977 in ndpi_add_host_ip_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2703

  This frame has 4 object(s):
    [32, 36) 'pin' (line 2705)
    [48, 64) 'pin6' (line 2706)
    [80, 96) 'd' (line 2769)
    [112, 128) 'tail' (line 2770) <== Memory access at offset 128 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow asan_interceptors.cpp.o in scanf_common(void*, int, bool, char const*, __va_list_tag*)
Shadow bytes around the buggy address:

```
2023-10-30 09:21:57 +01:00
Luca Deri 8769a2282b Disbling AES-NI on FreeBSD as the feature checke does not work well
and it enables it by mistake on IntelB. Coreb" i3-3240 Processor
2023-10-30 00:05:25 +01:00
Luca Deri 9fb7a63576 FreeBSD compilation fix (missingcvg #include <netdb.h>) 2023-10-29 23:18:17 +01:00
Luca Deri 76829b413f Implements support for symbolic host names (#2123) 2023-10-29 22:54:45 +01:00
Luca Deri b8e0602bfd Added check 2023-10-29 22:54:45 +01:00
Ivan Nardi 2c2eaf8e79 Rename some functions with more useful/clear names (#2127) 2023-10-29 20:21:24 +01:00
Ivan Nardi 03fd155ae3 IPv6: add support for custom categories (#2126) 2023-10-29 12:56:44 +01:00
Ivan Nardi 32b50f5aa4 IPv6: add support for IPv6 risk exceptions (#2122) 2023-10-29 12:14:20 +01:00
Ivan Nardi c711251578 IPv6: add support for custom rules (#2120) 2023-10-29 11:26:35 +01:00
Ivan Nardi e8e4b9e8ff IPv6: add support for IPv6 risk tree (#2118)
Fix the script to download crawler addressess
2023-10-27 13:58:15 +02:00
Toni ed17f4d658 Improved Protobuf dissector. (#2119)
* tag extraction/validation was done wrong

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-27 10:19:47 +02:00
Ivan Nardi 8b07be4b9f Jabber: remove support for UDP (#2115)
Jabber/XMPP is only over TCP (even the name `ndpi_search_jabber_tcp`
suggests that...).

Bug introduced in 5266c726f
2023-10-26 20:16:27 +02:00
Ivan Nardi 611c3b66f0 ipv6: add support for ipv6 addresses lists (#2113) 2023-10-26 20:15:44 +02:00
Luca Deri 1832d247b3 Tool for creating bitcoing IP files 2023-10-26 00:01:44 +02:00
Toni 9509cd507a Fixed endian issue while DEBUG_PROTOBUF is enabled. (#2112)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-25 13:11:17 +02:00
Maatuq 4a8e7105b2 add ethereum protocol dissector. (#2111)
as explained here for bitcoin https://www.ntop.org/guides/nDPI/protocols.html#ndpi-protocol-bitcoin
the same is applicable for ethereum.
ethereum detection was removed from mining protocol and is now handled separately.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
2023-10-25 12:44:33 +02:00
Toni e70333de87 Added generic Google Protobuf dissector. (#2109)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-24 12:18:31 +02:00
Luca Deri 71d768d036 Improved detection as non DGA for hostnames belnging to a CDN (#2068) 2023-10-23 17:49:50 +02:00
Toni Uhlig a443bba0dd Add CAN over Ethernet dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-23 13:45:56 +02:00
Toni Uhlig 25c54dd6d7 Improved CryNetwork protocol dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-23 13:44:34 +02:00
Luca Deri 7122599287 Added CLA section 2023-10-23 12:05:56 +02:00
Luca Deri 23759c6d2d Moved to 4.9 2023-10-23 11:46:04 +02:00
Luca Deri 2a99e16728 Added 4.8 changeling 2023-10-23 11:38:46 +02:00
Toni Uhlig f69909d49b Add Remote Management Control Protocol (RMCP).
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-19 19:50:57 +02:00
Alfredo Cardigliano c59fe0b2b5 PR template 2023-10-19 09:29:38 +02:00
Toni 01f384f7ff Improved Steam detection by adding steamdiscover pattern. (#2105)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-17 22:13:02 +02:00
Toni 32b0aeea59 Fixed OpenWRT arm related build issues. (#2104)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-16 19:49:50 +02:00
Toni d75a8d339a Fixed two OpenWRT arm related build issues. (#2103)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-16 18:14:28 +02:00
Luca Deri d9b0ba1d14 Windows warning checks 2023-10-16 16:17:41 +02:00
Nardi Ivan 16b4913be6 fuzz: extend fuzzing coverage 2023-10-15 12:00:26 +02:00
Luca Deri b6179467eb Added NDPI_MALWARE_HOST_CONTACTED flow risk 2023-10-13 00:01:55 +02:00
Luca 30fc89e556 Fix for buffer overflow in serialization 2023-10-11 17:14:17 +02:00
lns 304747f1fa Improved MGCP detection by allowing '\r' as line feed.
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-11 16:44:26 +02:00
Nardi Ivan 4a0eda69ad QUIC: export QUIC version as metadata 2023-10-11 15:15:20 +02:00
Luca Deri a76b9d151f Serialization fix 2023-10-11 00:14:27 +02:00
Nardi Ivan 1366d94156 fuzzing: extend fuzzing coverage
Try fuzzing some functions which write to file/file descriptor; to avoid
slowing the fuzzer, close its stdout
2023-10-09 15:41:46 +02:00
Nardi Ivan 86115a8a65 fuzz: extend fuzzing coverage 2023-10-07 13:34:37 +02:00
Luca Deri 8ca842edec version of dirent.c that is liked by both VC++ and MinGW 2023-10-05 14:35:41 +02:00
Luca Deri e1e6a01d6a Windows code rework 2023-10-05 12:24:39 +02:00
Luca Deri 3881d162f7 Windows compilation fixes 2023-10-05 11:05:14 +02:00
Luca Deri 36112a645b Win include change 2023-10-05 10:47:15 +02:00
Toni a98d7ff433 Added HAProxy protocol. (#2088)
* fixed tests/do.sh.in failure print

Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-02 18:10:47 +02:00
Luca 77e5daf03e Cleaned up mining datastructure 2023-09-27 17:05:12 +02:00
Toni ef3adb9830 Added printf/fprintf replacement for some internal modules. (#1974)
* logging is instead redirected to `ndpi_debug_printf`

Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-09-26 23:10:57 +02:00
Luca Deri 725fcf4852 Renamed HTTP/2 to HTTP2 as the '/' can have side effects with applications sitting on top of nDPI 2023-09-20 11:17:09 +02:00
Ivan Nardi d4e0b79dd6 Fix some prototypes (#2085)
```
error: function declaration isn’t a prototype [-Werror=strict-prototypes]
```
2023-09-18 14:06:47 +02:00
Ivan Nardi 6925890383 Add support for (un-encrypted) HTTP/2 (#2087)
Plaintext HTTP/2 is quite rare on the general "internet" but it is
used in some private networks (example: 5G core network)
2023-09-18 14:06:09 +02:00
Nardi Ivan 70814002a9 fuzz: extend fuzzing coverage 2023-09-16 11:26:11 +02:00
Luca 0828dff3ac Language fix 2023-09-15 12:56:29 +02:00
Ivan Nardi c5cd4df056 Add ndpi_domain_classify_finalize() function (#2084)
The "domain classify" data structure is immutable, since it uses
"bitmap64".
Allow to finalize it before starting to process packets (i.e. before
calling `ndpi_domain_classify_contains()`) to avoid, in the data-path,
all the memory allocations due to compression.

Calling `ndpi_domain_classify_finalize()` is optional.
2023-09-12 19:47:14 +02:00
Thomas Winter 5bcf3c2ddd tftp: update pcap results
The two malformed TFTP packets are no longer considered as risk
and instead match by port only.
This is because the TFTP detection was rather sparse so could
match on several other protocols if the first two opcode bytes
happened to match.
2023-09-12 13:12:14 +02:00
Thomas Winter a5f0cb7e6c tftp: check incrementation for DATA and ACK packets
The 2 bytes following the opcode for DATA and ACK packets are the
block number and this should be incrementing every packet.
We should check to see that this is occurring otherwise false matches
can occur, eg L2TPv3 over UDP matches the DATA opcode but the next two
bytes are always zero.

Remove the DATA max block size assumption since this can be false if
the blksize option is used to increase it.

Fixes #2070
2023-09-12 13:12:14 +02:00
Thomas Winter 7263562514 tftp: check for Option Acknowledgements
TFTP Option Acknowledgement packets were being excluded.
When a read/write request contains options, an Option Acknowledgement
is returned that contains the option strings that the transaction
will use.
The options sent in the request are not compared with what was
acknowledged.
2023-09-12 13:12:14 +02:00
Thomas Winter 1e9517ab74 tftp: rework request checking to account for options
Read and write requests were not handling options.
The existing code inspecting request messages assumed that the string
before the end of the payload was the mode and that the filename
length depended on the mode length.
However the first two strings are the filename and mode which can be
followed by any number of option strings.

Rework the checking of the filename and mode to just search the first
two strings which should always exist. Any options after that are
ignored.

Absence of the filename or mode is now excludes the TFTP protocol.
Absence of the filename no longer is considered malformed packet
because it can match other protocols falsely.
2023-09-12 13:12:14 +02:00
Luca Deri 1bf7e5face Fixes matches with domain name strings that start with a dot 2023-09-11 22:50:03 +02:00
Ivan Nardi 8e6500a0da fuzz: extend fuzzing coverage (#2083) 2023-09-10 21:30:26 +02:00
Ivan Nardi ef6085370f fuzz: add fuzzers to test bitmap64 and domain_classify data structures (#2082) 2023-09-10 18:44:50 +02:00
Ivan Nardi 2c5e22123e Update protocols documentation (#2081) 2023-09-10 15:23:34 +02:00
Ivan Nardi 2a0052f25e fuzz: add fuzzers to test reader_util code (#2080) 2023-09-10 15:07:52 +02:00
Ivan Nardi 0aa1cf7245 Update every ip lists (#2079) 2023-09-10 12:13:20 +02:00
Ivan Nardi 2b883b93be Fix some errors found by fuzzers (#2078)
Fix compilation on Windows.
"dirent.h" file has been taken from https://github.com/tronkko/dirent/

Fix Python bindings

Fix some warnings with x86_64-w64-mingw32-gcc:
```
protocols/dns.c: In function ‘ndpi_search_dns’:
protocols/dns.c:775:41: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
  775 |       unsigned long first_element_len = (unsigned long)dot - (unsigned long)_hostname;
      |                                         ^
protocols/dns.c:775:62: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
  775 |       unsigned long first_element_len = (unsigned long)dot - (unsigned long)_hostname;
      |
```
```
In file included from ndpi_bitmap64.c:31:
third_party/include/binaryfusefilter.h: In function ‘binary_fuse8_hash’:
third_party/include/binaryfusefilter.h:160:32: error: left shift count >= width of type [-Werror=shift-count-overflow]
  160 |     uint64_t hh = hash & ((1UL << 36) - 1);
```
```
In function ‘ndpi_match_custom_category’,
    inlined from ‘ndpi_fill_protocol_category.part.0’ at ndpi_main.c:7056:16:
ndpi_main.c:3419:3: error: ‘strncpy’ specified bound depends on the length of the source argument [-Werror=stringop-overflow=]
 3419 |   strncpy(buf, name, name_len);
```
2023-09-10 11:09:59 +02:00
Luca Deri 805df2e5ce Reworked initialization 2023-09-10 09:36:54 +02:00
Luca Deri 63977459b6 Added OperaVPN detection 2023-09-09 18:17:26 +02:00
Luca Deri 076edeab54 Enhance DNS risk for long hostnames (> 32) 2023-09-09 18:09:57 +02:00
Luca Deri f50a4d7e85 Improved detection of invalid chars in DNS names 2023-09-09 17:46:13 +02:00
Luca Deri 1a797d7b74 Win fix 2023-09-08 12:24:27 +02:00
Luca Deri 770e460ba9 Added NDPI_TLS_ALPN_SNI_MISMATCH flow risk 2023-09-07 23:42:42 +02:00
Luca 47fb5e9f3d Cleanup 2023-09-07 11:12:37 +02:00
Luca Deri 2bd986a00a Disabled debug message 2023-09-05 18:39:26 +02:00
Luca Deri ad9ed4f954 Minor warning fixes 2023-09-05 18:08:16 +02:00
Luca Deri 978df906b3 Improved classification further reducing memory used 2023-09-05 17:03:20 +02:00
Luca Deri 0080d0d092 Compilation fixes for old compilers 2023-09-05 10:43:36 +02:00
Luca Deri feaaf83bed Added sub-domain classification fix 2023-09-05 07:21:48 +02:00
Luca Deri cf67f483f4 Classification fixes 2023-09-05 01:17:02 +02:00
Luca Deri c5ebd84fd8 Added ndpi_bitmap64 support 2023-09-05 00:21:16 +02:00
Luca Deri 4f2ce2d43b Added ndpi_murmur_hash to the nDPI API 2023-09-04 12:53:42 +02:00
Luca Deri f0dc3347ec Merged new and old version of ndpi_domain_classify.c code 2023-09-02 19:27:19 +02:00
Luca Deri 1d480c18e3 Reworked domain classification based on binary filters 2023-09-02 19:16:40 +02:00
Luca Deri 854c2d80f1 Improvement for reducing false positives 2023-09-01 10:26:07 +02:00
Luca Deri 74fdac7d77 Code cleanup 2023-09-01 08:50:55 +02:00
Luca Deri 133f45f3f3 Added ndpi_binary_bitmap datastruture
It is similar to ndpi_filter but based on binary search and with
the ability to store a category per value (as ndpi_domain_classify)
2023-08-31 23:51:50 +02:00
Luca Deri 19381f330a Fixed heap-buffer-overflow issue 2023-08-31 18:35:49 +02:00
Luca Deri 16b0ce3710 Code cleanup 2023-08-31 09:14:17 +02:00
Luca Deri f824939662 Added comment 2023-08-31 09:00:36 +02:00
Luca Deri 3c7b2a714e Added missing lists files 2023-08-29 19:21:57 +02:00
Luca Deri c5c610c941 Fixes dist 2023-08-29 18:49:43 +02:00
Luca Deri 65142b2a53 Added -G flag 2023-08-29 18:32:57 +02:00
Luca Deri 1de7fd9bf0 Warning fixes 2023-08-29 18:08:56 +02:00
Luca Deri 3dbc4f65fc Added README file 2023-08-29 17:45:36 +02:00
Luca Deri 36abf06c6f Swap from Aho-Corasick to an experimental/home-grown algorithm that uses a probabilistic
approach for handling Internet domain names.

For switching back to Aho-Corasick it is necessary to edit
ndpi-typedefs.h and uncomment the line
// #define USE_LEGACY_AHO_CORASICK

[1] With Aho-Corasick
$ ./example/ndpiReader -G ./lists/ -i tests/pcap/ookla.pcap | grep Memory
nDPI Memory statistics:
nDPI Memory (once):      37.34 KB
Flow Memory (per flow):  960 B
Actual Memory:           33.09 MB
Peak Memory:             33.09 MB

[2] With the new algorithm
$ ./example/ndpiReader -G ./lists/ -i tests/pcap/ookla.pcap | grep Memory
nDPI Memory statistics:
nDPI Memory (once):      37.31 KB
Flow Memory (per flow):  960 B
Actual Memory:           7.42 MB
Peak Memory:             7.42 MB

In essence from ~33 MB to ~7 MB

This new algorithm will enable larger lists to be loaded (e.g. top 1M domans
https://s3-us-west-1.amazonaws.com/umbrella-static/index.html)

In ./lists there are file names that are named as <category>_<string>.list
With -G ndpiReader can load all of them at startup
2023-08-29 17:34:04 +02:00
Toni Uhlig 1f693c3f5a Added lists/gambling.list to extra dist.
* make dist in `./fuzz`: fixed inconsistent `*.dict` file pattern

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-27 15:53:31 +02:00
Toni Uhlig afd250f5d9 fix compilation and symbol check
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-27 15:32:42 +02:00
Toni Uhlig 25f3a7cd1d fix compilation
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-27 15:17:56 +02:00
Luca Deri 34986de297 Search fixes 2023-08-26 19:47:50 +02:00
Luca Deri eeeee46b1e Changes for supporinng more efficient sub-string matching 2023-08-26 17:55:50 +02:00
Luca Deri 4ca94369e1 Improved domain search tet unit 2023-08-26 10:43:19 +02:00
Luca Deri ac00542df2 LEak fix 2023-08-26 10:10:49 +02:00
Luca Deri 2c565c77c9 Added ndpi_domain_classify_XXX(0 API 2023-08-26 00:24:33 +02:00
Luca Deri 42fcb011ad Warning fix 2023-08-25 19:29:43 +02:00
Luca Deri eb9932164f added bimap and/or with allocation 2023-08-24 07:45:38 +02:00
Luca Deri ff4f291afe Minor improvements 2023-08-23 11:04:31 +02:00
Luca 95fcd08687 Added ndpi_bitmap_is_empty() and ndpi_bitmap_optimize() API calls 2023-08-23 11:03:59 +02:00
Luca 9933b5910d Boundary check 2023-08-21 23:19:57 +02:00
Luca bd25df3334 Improved Wireguard detection 2023-08-21 13:40:42 +02:00
Luca 9e4bcc7771 Removed empty line 2023-08-21 13:40:30 +02:00
Luca 6e5466f9e5 Added ndpi_bitmap_andnot API call 2023-08-21 13:26:38 +02:00
Ivan Nardi cc4461f424 fuzz: extend coverage (#2073) 2023-08-20 15:18:19 +02:00
Ivan Nardi 5867f43fae Fix compilation on Windows (#2072) 2023-08-14 22:51:09 +02:00
Luca Deri cfbb7a423e Replaces free() with ndpi_free() 2023-08-14 18:51:11 +02:00
Luca Deri 049fbf819c Reworked ndpi_filter_xxx implementation using compressed bitmaps 2023-08-14 12:50:54 +02:00
Luca Deri 3b34941be1 Typo 2023-08-12 11:39:19 +02:00
Luca Deri 9a44954365 ARM compilation fix 2023-08-12 07:39:35 +02:00
Luca Deri fe634b8d55 Fixed to address issue
Run ./utils/check_symbols.sh || { FAILED=$?; echo "::error file=${NDPI_LIB}::Unwanted libc symbols found: ${FAILED}. Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of malloc/calloc/realloc/free."; false; }
[ndpi_filter.o]: calloc
Unwanted symbols found: 1
Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of malloc/calloc/realloc/free
Error: Unwanted libc symbols found: 1. Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of
2023-08-12 00:27:37 +02:00
Luca Deri 8e17f50853 Added ndpi_filter_add_multi() API call 2023-08-11 23:55:44 +02:00
Luca Deri 890dcf91d9 Minor initialization fixes 2023-08-11 23:51:16 +02:00
Luca Deri ec7adc212e Added new API calls for implementing Bloom-filter like data structures
ndpi_filter* ndpi_filter_alloc(uint32_t elements_number);
bool         ndpi_filter_add(ndpi_filter *f, uint64_t value);
bool         ndpi_filter_contains(ndpi_filter *f, uint64_t value);
void         ndpi_filter_free(ndpi_filter *f);
2023-08-11 22:49:31 +02:00
Luca 5fad72ebd9 Typo fix 2023-08-07 10:20:21 +02:00
Luca 33e9dbc24d Added ndpi_bitmap_copy() API call 2023-08-06 09:35:56 +02:00
snicket2100 1fbe8a2385 Mullvad VPN service added (based on entry node IP addresses) (#2062) 2023-08-02 19:44:16 +02:00
Luca Deri 196395398e Compilation fixes for older C compilers 2023-08-01 15:35:33 +02:00
Toni e4d3d619bc Add Service Location Protocol dissector. (#2036)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-01 08:50:46 +02:00
Toni 083275f617 Added slackb.com SNI. (#2067)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-01 08:39:10 +02:00
Ivan Nardi 5019022e13 DNS: extract geolocation information, if available (#2065)
The option NSID (RFC5001) is used by Google DNS to report the
airport code of the metro where the DNS query is handled.

This option is quite rare, but the added overhead in DNS code is pretty
much zero for "normal" DNS traffic
2023-07-31 07:44:43 +02:00
snicket2100 d477da997f Typo in a comment fixed (#2063) 2023-07-28 17:58:25 +02:00
Ivan Nardi bc91192aca ProtonVPN: split the ip list (#2060)
Use two separate lists:
* one for the ingress nodes, which triggers a ProtonVPN classification
* one for the egress nodes, which triggers the
`NDPI_ANONYMOUS_SUBSCRIBER` risk

Add a command line option (to `ndpiReader`) to easily test IP/port
matching.

Add another example of custom rule.
2023-07-27 09:05:22 +02:00
Ivan Nardi 3326fa258e Add an heuristic to detect fully encrypted flows (#2058)
A fully encrypted session is a flow where every bytes of the
payload is encrypted in an attempt to “look like nothing”.
The heuristic needs only the very first packet of the flow.
See: https://www.usenix.org/system/files/sec23fall-prepub-234-wu-mingshi.pdf

A basic, but generic, inplementation of the popcpunt alg has been added
2023-07-26 09:09:12 +02:00
Toni 2b230e28e0 Allow init of app protocols w/o any hostnames set. (#2057) 2023-07-22 14:51:33 +02:00
Toni 88a757c4cd Fix string truncation. (#2056)
ndpi_main.c: In function ‘ndpi_load_ip_category’:
ndpi_main.c:6598:3: warning: ‘strncpy’ specified bound 64 equals destination size [-Wstringop-truncation]
 6598 |   strncpy(ipbuf, ip_address_and_mask, sizeof(ipbuf));
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-07-22 12:45:45 +02:00
Luca Deri 6831187d62 Debian 12 fixes 2023-07-21 12:24:34 -04:00
Ivan Nardi 7a1509bf60 zabbix: improve detection (#2055) 2023-07-21 03:42:36 +02:00
Ivan Nardi c85f2fb0f4 TLS: add basic, basic, detection of Encrypted ClientHello (#2053) 2023-07-21 03:41:43 +02:00
Ivan Nardi fa0bd515b5 Add detection of Roblox games (#2054) 2023-07-21 03:39:40 +02:00
Ivan Nardi 3edfad01a1 fuzz: extend fuzzing coverage (#2052)
Added/merged some traces.
Improved Socks identification
2023-07-18 07:41:56 +02:00
Ivan Nardi 09548bb7cf tests: restore some old paths as symbolic links (#2050) 2023-07-16 13:47:35 +02:00
Ivan Nardi 5267858b3d HTTP: fix another memory access error (#2049)
```
=================================================================
==199079==ERROR: AddressSanitizer: negative-size-param: (size=-1)
    #0 0x559a2a6efd4f in strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x94ad4f) (BuildId: 34aaabba403c6bc5482553ef355360fd2762a157)
    #1 0x559a2a9890f0 in ndpi_http_check_content /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:300:8
    #2 0x559a2a9812c0 in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:910:46
    #3 0x559a2a978fee in process_response /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1289:3
    #4 0x559a2a97622f in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1382:9
    #5 0x559a2a975d95 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1468:3
    #6 0x559a2a864970 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5948:4
    #7 0x559a2a8660df in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6013:12
    #8 0x559a2a865d7f in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6032:12
    #9 0x559a2a876fd6 in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7038:15
    #10 0x559a2a87311f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7205:22
    #11 0x559a2a77381e in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1710:31
    #12 0x559a2a77381e in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2427:10
[...]
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60605
2023-07-15 22:23:52 +02:00
Ivan Nardi 2bbde5bad3 Fix compilation in CI jobs (#2048) 2023-07-15 14:26:51 +02:00
Ivan Nardi 890f17788b ndpireader: fix detection of DoH traffic based on packet distributions (#2045) 2023-07-14 23:20:06 +02:00
Ivan Nardi 5811a5613b Add a configuration knob to enable/disable loading of gambling list (#2047) 2023-07-14 23:19:30 +02:00
Luca Deri 8e960f033d Adds new pcap for testing "funny" HTTP servers 2023-07-14 21:12:11 +02:00
Luca Deri a499edd2f2 Restored files 2023-07-14 21:12:04 +02:00
Luca Deri fea09e825b Fixes risk mask exception handling while improving the overall performance 2023-07-14 19:52:34 +02:00
snicket2100 abee1a2a6f Included Gambling website data from the Polish hazard.mf.gov.pl list (#2041)
* Refreshed the Belgium Gambling Site list data

Unfortunately some hostnames have been removed from that list,
which means they are disappearing from the `ndpi_gambling_match.c.inc`
file as well.

* build: added `libxml2-utils` (for `xmllint`)

* Included Gambling website data from the Polish `hazard.mf.gov.pl` list

The list contains over 30k gambling website hostnames as of today.
2023-07-14 09:55:46 +02:00
Ivan Nardi 0dd1ee5656 HTTP: fix extraction of filename (#2046) 2023-07-14 01:33:00 +02:00
Luca Deri bfbbd18757 Removed -fno-color-diagnostics not available on all platforms 2023-07-13 23:30:05 +02:00
Luca Deri 33c42f9830 Cosmetic fixes 2023-07-13 23:26:35 +02:00
Luca Deri 1f55dc511f Implemented Count-Min Sketch [count how many times a value has been observed]
- ndpi_cm_sketch_init()
- ndpi_cm_sketch_add()
- ndpi_cm_sketch_count()
- ndpi_cm_sketch_destroy()
2023-07-13 21:54:51 +02:00
Ivan Nardi bd0fcb2e62 HTTP: fix heap-buffer-overflow (#2044)
```
=================================================================
==673246==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x5080000002fd at pc 0x555c6a22eca2 bp 0x7ffe1db9b290 sp 0x7ffe1db9aa48
READ of size 1 at 0x5080000002fd thread T0
    #0 0x555c6a22eca1 in strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x53aca1) (BuildId: 464e7474cde3de91c01588c7ea0161a05ddd6b63)
    #1 0x555c6a462a9d in ndpi_http_check_content /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:309:8
    #2 0x555c6a45a9a0 in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:901:46
    #3 0x555c6a4526ce in process_response /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1280:3
    #4 0x555c6a44f90f in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1373:9
    #5 0x555c6a44f475 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1459:3
    #6 0x555c6a348520 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5949:6
    #7 0x555c6a348f2f in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5997:12
    #8 0x555c6a348bcf in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6016:12
    #9 0x555c6a359e26 in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7017:15
    #10 0x555c6a355f6f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7184:22
    #11 0x555c6a284958 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:24:5
```

Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60557
Fix: 0b0f255cc
2023-07-13 10:50:59 +02:00
Ivan Nardi 0c5a17accb RDP: improve detection over UDP (#2043) 2023-07-13 10:50:13 +02:00
Ivan Nardi 5e6a8c0c1f fuzz: add fuzzer for DGA detection code (#2042) 2023-07-13 10:47:34 +02:00
Alfredo Cardigliano 669a02cd76 Add support for multiline json 2023-07-12 10:47:49 +02:00
Luca Deri b1d25193d2 Minor code cleanup 2023-07-11 22:49:06 +02:00
Chiara Maggi 0b0f255cc2 added feature to extract filename from http attachment (#2037)
* added feature to extract filename from http attachment

* fixed some issues

* added check for filename format

* added check for filename format

* remove an unnecessary print

* changed the size from 952 to 960

* modified some test result files

* small changes string size

* comment removed and mallocs checked
2023-07-11 22:45:19 +02:00
Ivan Nardi 950f5cc4e3 fuzz: extend fuzzing coverage (#2040)
Some notes:
* libinjection: according to https://github.com/libinjection/libinjection/issues/44,
it seems NULL characters are valid in the input string;
* RTP: `rtp_get_stream_type()` is called only for RTP packets; if you
want to tell RTP from RTCP you should use `is_rtp_or_rtcp()`;
* TLS: unnecessary check; we already make the same check just above, at
the beginning of the `while` loop
2023-07-11 10:12:08 +02:00
Toni 859d9ea3c3 Fixed heap-overflow if compiled with --enable-tls-sigs. (#2038)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-07-07 18:52:06 +02:00
Toni 72814bed33 Added coverage targets to Makefile.am for convenience. (#2039)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-07-07 17:40:50 +02:00
Ivan KapranovandIvan Kapranov bdd295bc2c fix Stack overflow caused by invalid write in ndpi_automa_match_strin… (#2035)
* minor fixes

fixed 'handle leak' in ndpi_load_malicious_sha1_file and removed the redundant comparison ndpi_search_eaq

* fix Stack overflow caused by invalid write in ndpi_automa_match_string_subprotocol

* fix compile errors

* fix

---------

Co-authored-by: Ivan Kapranov <i.kapranov@securitycode.ru>
2023-07-06 11:13:34 +02:00
Ivan Nardi bb978907e6 STUN: fix detection of Google Voip apps (#2031)
Fix: 2c7fb9179
2023-07-05 14:37:38 +02:00
headshog 076ba967a8 fixed numeric truncation error in diameter.c (#2034) 2023-07-05 14:29:55 +02:00
headshog d51f8929e0 fixed numeric truncation error in rtcp.c (#2033) 2023-07-05 14:29:44 +02:00
headshog 2e5004ddb2 fixed numeric truncation error in kerberos.c (#2032) 2023-07-05 14:29:32 +02:00
Ivan Nardi 7ffd31ebc3 STUN: avoid FacebookVoip false positives (#2029)
Attribute 0xC057 is defined in the Google public implementation of
webrtc (which is used by Google products but also by other applications)
2023-07-03 17:21:46 +02:00
Ivan Nardi dff1f25193 STUN: fix Skype/MsTeams detection and monitoring logic (#2028) 2023-07-03 17:21:32 +02:00
Ivan Nardi 86e89b4e23 STUN: tell RTP from RTCP while in monitoring state (#2027) 2023-06-27 10:33:48 +02:00
Ivan Nardi 2c7fb91794 Hangout: detect Hangout/Duo/GoogleMeet/... in the STUN code (#2025)
Regardless of the name, the removed trace doesn't contain meaningful
Hangout traffic.

Remove last piece of sub-classifiction based only on ip addresses.
2023-06-27 10:33:28 +02:00
Ivan Nardi 31a9da238c STUN: add dissection of DTLS handshake (#2018) 2023-06-26 12:07:26 +02:00
Ivan Nardi 88425e0199 Simplify the report of streaming multimedia info (#2026)
The two fields `flow->flow_type` and `flow->protos.rtp.stream_type` are
pretty much identical: rename the former in `flow->flow_multimedia_type`
and remove the latter.
2023-06-26 12:05:16 +02:00
Ivan Nardi 3a1600ff26 Thrift: fix heap-buffer-overflow (#2024) 2023-06-26 11:41:01 +02:00
Luca Deri 7ef80f6b78 Optimizes and fixes possible out0of0boundary write in ndpi_fill_prefix_v4() 2023-06-23 21:40:55 +02:00
Ivan Nardi 7e64d9f66d RTP: rework code (#2021)
Try avoiding false positives: look for 3 RTP packets before classifing
the flow as such.

Add a generic function `is_rtp_or_rtcp()` to identify RTP/RTCP packets also
in other dissectors (see 3608ab01b commit message for an example)
2023-06-23 08:12:40 +02:00
Ivan KapranovandIvan Kapranov 15f0e1480c minor fixes (#2023)
fixed 'handle leak' in ndpi_load_malicious_sha1_file and removed the redundant comparison ndpi_search_eaq

Co-authored-by: Ivan Kapranov <i.kapranov@securitycode.ru>
2023-06-22 21:36:40 +02:00
Ivan Kapranov aa25c06a18 removed useless call of ndpi_set_risk func (#2022) 2023-06-22 18:26:33 +02:00
Luca Deri cdc0bb8c61 Compilation fix 2023-06-22 16:04:14 +02:00
Toni 1678888284 Add Apache Thrift protocol dissector. (#2007)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-06-22 13:07:32 +02:00
Luca Deri 8dbd591053 Waring fix 2023-06-21 21:26:29 +02:00
Ivan Nardi cc582052b3 Gnutella: improve detection (#2019) 2023-06-21 14:25:34 +02:00
Nardi Ivan 570c75d601 STUN: fix detection over TCP
TCP framing is optional
2023-06-21 10:38:44 +02:00
Nardi Ivan 2ac240ce6a STUN: improve WhatsappCall detection 2023-06-21 10:38:44 +02:00
Ivan Nardi 46aab35a29 Line: fix heap-buffer-overflow error (#2015) 2023-06-21 09:22:07 +02:00
Ivan Nardi 3608ab01b6 STUN: keep monitoring/processing STUN flows (#2012)
Look for RTP packets in the STUN sessions.
TODO: tell RTP from RTCP
2023-06-21 09:16:20 +02:00
Ivan Nardi 04be308092 Fix access to packet/flow information (#2013)
We can't access to `ndpi_str->packet` from `ndpi_detection_giveup()`
2023-06-17 15:46:06 +02:00
Luca Deri ac06ba8320 Added hyperlink 2023-06-16 21:48:21 +02:00
Luca Deri 9cc4cbb9d1 Reworked teams handling 2023-06-15 22:31:11 +02:00
Ivan Nardi 2321c11eb5 Fix compilation (#2011) 2023-06-15 12:01:11 +02:00
Luca Deri d0609ea601 Implemented Zoom/Teams stream type detection 2023-06-14 23:44:57 +02:00
Luca Deri 66bee475ae Improved line protocol dissection with heuristic 2023-06-13 19:20:02 +02:00
Luca Deri de3cf260f8 Updated line test result 2023-06-13 19:07:02 +02:00
Luca Deri be8178fc8d Refreshed ASN lists
Enhanced the Line IP list with https://ipinfo.io/AS23576/125.209.252.0/24 used by line
2023-06-13 19:04:08 +02:00
Luca Deri b02adcc381 Added check to avoid skype heuristic false positives 2023-06-12 23:29:20 +02:00
Toni 4e284b5e40 Set _DEFAULT_SOURCE and _GNU_SOURCE globally. (#2010)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-06-12 19:53:57 +02:00
Toni 8ea0eaa0d0 Fix missing u_char, u_short and u_int typedefs for some platforms e.g.: (#2009)
In file included from ../include/ndpi_main.h:34,
                 from ../include/ndpi_api.h:28,
                 from protocols/activision.c:26:
../include/ndpi_typedefs.h:294:3: error: unknown type name 'u_char'
  294 |   u_char h_dest[6];       /* destination eth addr */
      |   ^~~~~~

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-06-12 17:52:52 +02:00
Nardi Ivan 96340ccea2 oss-fuzz: sync build script with upstream
File copied from https://github.com/google/oss-fuzz/blob/master/projects/ndpi/build.sh

The general idea is to keep the build script in our repository and use
it from oss-fuzz builder: updating it from our side is easier and faster
then passing via an oss-fuzz PR.

The original idea is from @utoni in 3068306b60.

Once this change has been merged, we can update the code in oss-fuzz.
2023-06-12 15:17:56 +02:00
Luca Deri 63a2b359b9 Added vlan_id in ndpi_flow2json() prototype 2023-06-09 19:50:47 +02:00
Nardi Ivan 7150b40c63 QUIC: fix dissection of packets forcing VN 2023-06-08 17:07:25 +02:00
Nardi Ivan 3e640a13a4 QUIC: fix a memory access error
Found while fuzzing
2023-06-08 17:07:25 +02:00
Nardi Ivan fd4cb10190 QUIC: add support for QUIC version 2
See: https://www.rfc-editor.org/rfc/rfc9369.txt

Old v2-01 version has been removed, since it has never been really used.
2023-06-08 17:07:25 +02:00
Ivan Nardi 9987e5b482 ndpiReader: allow to configure LRU caches TTL and size (#2004) 2023-06-08 17:06:32 +02:00
Ivan Nardi 3e673e91a9 ProtonVPN: add basic detection (#2006) 2023-06-08 16:52:55 +02:00
headshog 7dcceb4daf Numeric truncation at tls.c:1010 (#2005)
* fixed numtrunc error in protocols/tls.c

* fixed build error for tls.c
2023-06-08 07:52:18 +02:00
zehraInandNardi Ivan 436ab54f5a added new domain names (#2002)
* added new domain names

* Sync unit tests results

---------

Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2023-06-01 13:13:49 +02:00
Ivan Nardi e14d0acbd4 tests: add an option to force the overwrite of the unit tests results (#2001)
Usage: `FORCE_UPDATING_UTESTS_RESULTS=1 ./tests/do.sh`
2023-05-31 09:05:00 +02:00
Maatuq e17fa1259a Add bitcoing protocol dissector. (#1992)
* Add bitcoing protocol dissector.

* remove bitcoin protcol detection from mining.c
* add a new bitcoin deissector.
* add a new category: Cryptocurrency.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* Remove useless checks and add missing windows and docs file.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* update affected tests.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* add a brief version.

Add notes on the difference between normal bitcoin protocol and the
mining protocol.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* update enable_payload_stat test after dev rebasing.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

---------

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
2023-05-31 07:31:01 +02:00
headshog a8d2eeddd2 Numeric truncation at ndpi_analyze.c at lines 101, 104, 107, 110 (#1999)
* fixed numeric truncation error in ndpi_analyze.c

* fixed numeric truncation error in ndpi_analyze.c x2

* fixed numeric truncation error in ndpi_analyze.c x3

* fixed numeric truncation error in ndpi_analyze.c and printf format

* fixed tests
2023-05-30 12:27:47 +02:00
Toni 04f5c5196e Changed logging callback function sig. (#2000)
* make user data available for any build config

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-30 08:27:37 +02:00
headshog 552c9b3cc6 fixed numeric truncation error in ndpi_main.c:6837 (#1998) 2023-05-29 19:34:26 +02:00
Ivan Nardi efb261a95c Fix some memory errors triggered by allocation failures (#1995)
Some low hanging fruits found using nallocfuzz.
See: https://github.com/catenacyber/nallocfuzz
See: https://github.com/google/oss-fuzz/pull/9902

Most of these errors are quite trivial to fix; the only exception is the
stuff in the uthash.
If the insertion fails (because of an allocation failure), we need to
avoid some memory leaks. But the only way to check if the `HASH_ADD_*`
failed, is to perform a new lookup: a bit costly, but we don't use that
code in any critical data-path.
2023-05-29 19:24:00 +02:00
Ivan Nardi 346bb268e2 Avoid calling ndpi_reconcile_protocols() twice in ndpi_detection_giveup() (#1996)
`ndpi_reconcile_protocols()` is already called by
`ndpi_set_detected_protocol()`
2023-05-29 16:55:53 +02:00
Ivan Nardi 82fa3a0986 Fix an heap-buffer-overflow (#1994)
```
==32439==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x604000000080 at pc 0x564a9a1639ac bp 0x7ffc77d42e20 sp 0x7ffc77d42e18
READ of size 1 at 0x604000000080 thread T0
    #0 0x564a9a1639ab in ndpi_check_punycode_string /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:98:12
    #1 0x564a9a26025a in ndpi_match_host_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8652:6
    #2 0x564a9a33e45c in ndpi_http_parse_subprotocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:446:9
    #3 0x564a9a3354b8 in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:921:3
    #4 0x564a9a32dc8b in process_request /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1237:3
    #5 0x564a9a329a9c in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1351:5
    #6 0x564a9a3293ed in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1425:3
    #7 0x564a9a21c64a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5906:6
    #8 0x564a9a21d057 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5954:12
```

Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=59393
2023-05-29 16:53:39 +02:00
Ivan Nardi 46ff069117 ndpiReader: improve printing of payload statistics (#1989)
Add a basic unit test

Fix an endianess issue
2023-05-29 16:53:11 +02:00
Ivan Nardi 7ce14da0c8 Remove some useless checks (#1993)
In the main dissector callbacks the flow protocols are (almost) always
unknown. Only two exceptions:
* extra dissection data path
* HTTP sub-protocols
2023-05-28 12:59:18 +02:00
Toni 6da3474203 Improved helper scripts. (#1986)
* added additional (more restrictive) checks

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-28 12:45:44 +02:00
Ivan Nardi b11e6a453b Add support for Epic Games and GeForceNow/Nvidia (#1990) 2023-05-27 12:13:54 +02:00
Ivan Nardi 63ac50e4f4 Improve detection of Alibaba flows (#1991) 2023-05-27 10:19:58 +02:00
Toni 334b43579e Fixed invalid use of ndpi_free(). Sorry, my fault. (#1988)
* Fixed invalid use of ndpi_free(). Sorry, my fault.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Fine tuned symbol check script.

 * added check for expected syms in modules

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

---------

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-05-24 13:19:06 +02:00
Luca Deri f692342f3c Cosmetic changes 2023-05-23 22:16:34 +02:00
Ivan Nardi 86b56646b5 ndpiReader: fix export of DNS/BitTorrent attributes (#1985)
There is no BitTorrent hash in the DNS flows
2023-05-20 17:23:48 +02:00
Toni 5e8f93c2d1 Improved missing usage of nDPIs malloc wrapper. Fixes #1978. (#1979)
* added CI check

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-20 16:18:52 +02:00
Luca Deri 8f718c9051 Made VK protocol detection more strict 2023-05-20 15:46:47 +02:00
Toni c97e2d67ff Added scripts to auto generate hostname/SNI *.inc files. (#1984)
* add illegal gambling sites (Belgium)

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-20 15:41:15 +02:00
Ivan Nardi 9004d5c2ca ndpiReader: fix export of HTTP attributes (#1982) 2023-05-20 15:12:14 +02:00
Nardi Ivan 1ab5318f9c Sync unit tests results 2023-05-20 15:09:51 +02:00
headshog dd7198fcaf fixed numeric truncation error 2023-05-20 15:09:51 +02:00
Luca Deri 5ca6f0ac62 Implemented ndpi_predict_linear() for predicting a timeseries value overtime 2023-05-19 11:46:03 +02:00
Yuriy Rymarchuk a8b3baf2b8 Add support for roaring_bitmap_xor_inplace (#1983) 2023-05-18 13:28:32 +02:00
Ivan Nardi b1bcf1ff60 Fix classification-by-ip in ndpi_detection_giveup (#1981)
Return the "classification-by-ip" as protocol results only if no other
results are available.
In particular, never return something like
"protocol_by_port/protocol_by_ip" (i.e. `NTP/Apple`,
BitTorrent/GoogleCloud`, `Zoom/AWS`) because this kind of classification
is quite confusing, if not plainly wrong.

Notes:
* the information about "classification-by-ip" is always available, so
no information is lost with this change;
* in the unit tests, the previous classifications with confidence
`NDPI_CONFIDENCE_DPI_PARTIAL` were wrong, as noted in #1957
2023-05-17 11:26:25 +02:00
Ivan Nardi ace32c9dfe Add support for SRTP (#1977)
The goal is to have Zoom flows classified as "Encrypted" and not as
"Cleartext".

Start documenting the list of protocols supported by nDPI;
format, verbosity and content are still a work-in-progress.
2023-05-16 16:02:14 +02:00
Ivan Nardi 4e186f6bfb HTTP: rework state machine (#1966)
The goal if to correlate the right request-response pair, exporting
metadata from only one transaction (for example, the right url & return
state pair)

As a nice side effect, the code should be much cleaner, but that is a
matter of taste.

Two differences respect to the previous code:
* as it happens in the CI, if in the flow there are only one response
(before) and one request (after), only the metadata of the response are
saved/exported
* for performance reasons, we don't call `ndpi_parse_packet_line_info()`
anymore for ALL packets triggering the HTTP dissector, but only for the
packets that we already know belong to an HTTP flow. This is the reason
for the changes in RTSP/SOAP/... code
2023-05-16 16:01:55 +02:00
Ivan Nardi 8c224b464f Fix MS Teams detection with heuristic (#1972)
When we reconcile a TLS session to Teams, we need to keep TLs as master.
This way:
* we keep exporting all the TLS metadata
* we avoid some memory leaks (of these metadata themeselves)
2023-05-10 21:00:27 +02:00
Luca Deri 596a5fd5dc Updated results 2023-05-10 17:11:14 +02:00
Luca 64853dff02 Improved MS Teams detection with heuristic 2023-05-10 16:59:28 +02:00
Luca 98f029a091 Enhanced MS teams STUN/Azure detection 2023-05-10 16:28:34 +02:00
Luca 7d5e4d0f7e MS Teams enhancement 2023-05-10 15:41:26 +02:00
Luca 8de7c49298 MS Teams improvements 2023-05-10 15:16:39 +02:00
Ivan Nardi 31d23aee56 All protocols should be excluded sooner or later (#1969)
For a lot of protocols, reduce the number of packets after which the
protocols dissector gives up.
The values are quite arbitary, tring to not impact on classification
2023-05-10 12:50:24 +02:00
Ivan Nardi 99d7066ea0 Remove special handling of some TCP flows without SYN (#1965)
This piece of code has multiple problems:
* nDPI is able to detect some TCP protocols even with mid-flows (i.e.
without the initial packets of the session); TLS is the most
significative example
* since e6b332aa4a it is perfectly valid
to not pass the TCP Handshake packets to nDPI
* in any case, we shouldn't call `ndpi_detection_giveup()`. That
function is usually called by the application and we end up calling it
twice in some cases.

The simple solution is to completely remove that code: process these
kinds of flows like everyone else.

Note that the application can always avoid to pass to nDPI any TCP flows
without the initial handshake; the flow managemnt is always up to the
application.

Looking at the CI results, some rare flows are now processed significantly
longer. As a follow-up we could look into that.
2023-05-09 19:36:02 +02:00
Ivan Nardi 684e041998 Improve detection of crawlers/bots (#1968)
Add support for Facebook crawler
2023-05-09 16:42:29 +02:00
Ivan Nardi 79c1dbe0b2 Extend content-match list (#1967) 2023-05-09 16:41:12 +02:00
Luca Deri 2f9059dc54 Disabled query string validation in MDNS in order to avoid zapping chars that in DNS (instead) are not permitted 2023-05-09 15:04:02 +02:00
Luca Deri bfe79243bc Refreshed lists 2023-05-08 16:51:38 +02:00
Clément BRUGUERA 998bedbbb7 add 2 ns from fdn.fr to DoH section (#1964) 2023-05-05 18:08:13 +02:00
Ivan Nardi 0223d3c4f5 HTTP: improve extraction of metadata and of flow risks (#1959) 2023-05-05 13:35:20 +02:00
Ivan Nardi 02a2c80453 ndpiReader: fix print of flow payload (#1960) 2023-05-04 11:52:48 +02:00
Ivan Nardi a11ddd2dc4 HTTP: remove useless code about XBOX (#1958)
Old nDPI versions were able to detect XBOX flows over HTTP via
user-agent matching. This feature has been removed from a long time
(89d548f9d, at very least)
2023-05-04 11:28:03 +02:00
Ivan Nardi 6b94c9675a Improve detection of crawler/bot traffic (#1956) 2023-05-04 11:27:34 +02:00
Ivan Nardi 9a5d60bff6 Sync unit tests results (#1962) 2023-05-04 09:47:52 +02:00
Luca Deri 3681de4795 Updated pcap detection results after Facebook Reel/Stories support 2023-05-03 22:52:58 +02:00
Luca Deri de13c6c0bd Added detection of Facebook Reels and Stories
https://www.switcherstudio.com/blog/the-ultimate-guide-to-reels-and-stories#:~:text=Audience%3A%20Reels%20are%20served%20to,things%20like%20photos%20and%20boomerangs.
2023-05-03 22:50:02 +02:00
Ivan Nardi bb370f5ef0 Make Bittorrent LRU cache IPv6 aware. (#1909)
It was the only remaining LRU cache without IPv6 support.
See 81e1ea545c
2023-04-29 11:09:23 +02:00
Luca Deri 6b803f48dc Added missing check 2023-04-28 23:08:54 +02:00
Luca Deri 6e9dce357b Added ndpi_guess_undetected_protocol_v4() to restore logic that
was removed in https://github.com/ntop/nDPI/commit/032e778a6dc1ee2173c4cdb2c671180c5b5df245
2023-04-28 23:00:33 +02:00
Luca Deri f7138f07a6 Missing module termination 2023-04-28 23:00:33 +02:00
Luca Deri b39ece4428 Updated (C) 2023-04-28 23:00:33 +02:00
Luca Deri 791d3e6ad3 Added missing check 2023-04-28 23:00:33 +02:00
Ivan Nardi 8934f7b45f Add an heuristic to detect/ignore some anomalous TCP ACK packets (#1948)
In some networks, there are some anomalous TCP flows where the smallest
ACK packets have some kind of zero padding.
It looks like the IP and TCP headers in those frames wrongly consider the
0x00 Ethernet padding bytes as part of the TCP payload.
While this kind of packets is perfectly valid per-se, in some conditions
they might be treated by the TCP reassembler logic as (partial) overlaps,
deceiving the classification engine.
Add an heuristic to detect these packets and to ignore them, allowing
correct detection/classification.

This heuristic is configurable. Default value:
* in the library, it is disabled
* in `ndpiReader` and in the fuzzers, it is enabled (to ease testing)

Credit to @vel21ripn for the initial patch.

Close #1946
2023-04-25 19:25:07 +02:00
Ivan Nardi 40b6d5a2e1 fuzz: extend fuzzers coverage (#1952) 2023-04-25 16:37:28 +02:00
Ivan Nardi de693cbbc9 RTP: remove dead-code (#1953)
The checks `isValidMSRTPType(..) == 1` is a subset of
`is_valid_rtp_payload_type()` so this if-branch is never reached.

More importantly, the article describing how to detect Microsoft Lync and
Skype for Business is from 2014. These payload types are static or they
are in the dynamic range: in both cases, these values might be used (and
they are used indeed) pretty much by every application.
Bottom line: we can't use PT alone to identify a specific protocol.

Keep the list, since it is used to tell audio streams from video ones.
2023-04-25 14:16:27 +02:00
Luca Deri c4ba981370 Compilation fix on RH7 2023-04-24 08:49:11 +02:00
nikitamishagin d3e67faf20 Add "Heroes of the Storm" video game signature detection. (#1949) 2023-04-22 18:28:43 +02:00
Toni b6629ba2be Improved debug output. (#1951)
* try to get rid of some `printf(..)`s as they do not belong to a shared library
 * replaced all `exit(..)`s with `abort()`s to indicate an abnormal process termination

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-21 12:40:26 +02:00
Toni 85751ba9d7 Added OICQ dissector. (#1950)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-21 11:30:10 +02:00
Ivan Nardi ba99330031 ndpiReader: fix flow stats (#1943) 2023-04-14 11:17:37 +02:00
Ivan Nardi 032e778a6d Simplify ndpi_internal_guess_undetected_protocol() (#1941)
`ndpi_guess_undetected_protocol()/ndpi_internal_guess_undetected_protocol()`
is a strange function:
* it is exported by the library and it is actively used by `ntopng`
* it is intrinsecally ipv4-only
* it returns basically something like "classification_by_ip"/"classification_by_port"
(these information have already been calculated in `ndpi_do_guess()`...)
* it access the bittorrent LRU caches (similarly to
`ndpi_detection_giveup()` but without all the other caches...)

So:
* make the interface IPv4/6 agnostic
* use the classifications already available

This work will allow to make the Bittorrent caches IPV6-aware (see
81e1ea5).

Handle Dropbox classification "by-port" in the "standard" way.
2023-04-12 14:39:10 +02:00
Toni 6660feed35 Added BACnet dissector. (#1940)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 20:26:27 +02:00
lns 77ea80862f Improved debug logging.
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 15:39:09 +02:00
lns 6da958655c Improved invalid logging via printf().
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 15:39:09 +02:00
Toni bebcce5281 Added Source Engine dissector. (#1937)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 14:31:57 +02:00
Luca Deri 9ba16a4837 Test files for riit games 2023-04-11 10:14:18 +02:00
Jacopo a469009842 RiotGames: add detection of flows (#1935)
* RiotGames: add detection of flows

* remove of akamai domains
2023-04-11 10:04:09 +02:00
Ivan Nardi c7ef3608d9 CI: fix Performance job (#1936)
Fix: 7714507f
2023-04-09 10:16:28 +02:00
Luca Deri c47e9d201d Implemented ndpi_XXX_reset() API calls whre XXX is ses, des, hw 2023-04-08 09:52:14 +02:00
Ivan Nardi 7714507f81 Test multiple ndpiReader configurations (#1931)
Extend internal unit tests to handle multiple configurations.
As some examples, add tests about:
* disabling some protocols
* disabling Ookla aggressiveness

Every configurations data is stored in a dedicated directory under
`tests\cfgs`
2023-04-06 11:30:36 +02:00
Ivan Nardi 7a627296f0 Fix LRU/Patricia/Automa stats in ndpiReader with multiple threads (#1934) 2023-04-06 09:36:11 +02:00
Ivan Nardi b639744c3a Extend content list of Microsoft protocols (#1930) 2023-04-06 09:35:41 +02:00
Ivan Nardi 22cf8b5d8f configure: add an option to enable debug build, i.e -g (#1929)
Add this new flag to one CI job, to test it

Close #1925
2023-04-06 09:35:27 +02:00
Ivan Nardi 06a63dcbaa MySql: improve detection (#1928)
Add support for flows with "caching_sha2_password" authentication plugin.

See #1924
2023-04-06 09:34:39 +02:00
Ivan Nardi 5c28dbbae2 H323: fix false positives (#1916) 2023-04-06 09:32:57 +02:00
Ivan Nardi 25c1111911 fuzz: add a new fuzzer triggering the payload analyzer function(s) (#1926) 2023-04-04 14:39:29 +02:00
Maatuq f1193d5e6f add support for gre decapsulation (#1442) (#1921) 2023-04-04 14:20:11 +02:00
Ivan Nardi cc5aec5f0a fuzz: add fuzzer to test internal gcrypt code (#1920) 2023-04-04 14:19:41 +02:00
Ivan Nardi d7662379b4 Update libinjection code (#1918)
Update libinjection code to the current master https://github.com/libinjection/libinjection/commit/7e4b74e824dc3f0623ba4894df2a0d817351ec39

The goal is to finally fix #1820
See: https://github.com/libinjection/libinjection/issues/33

Update the corpus of the libinjection fuzzers

Close #1820
2023-04-04 14:18:57 +02:00
Ivan Nardi 4d11941d32 Ookla: rework detection (#1922)
The logic of the LRU cache has been changed: once we know an ip has
connected to an Ookla server, all the following (unknown) flows (for
a short time interval) from the same ip to the port 8080 are treated
as Ookla ones.

Most of the changes in this commit are about introducing the concept of
"aggressive detection". In some cases, to properly detect a
protocol we might use some statistical/behavior logic that, from one
side, let us to identify the protocol more often but, from the other
side, might lead to some false positives.
To allow the user/application to easily detect when such logic has been
triggered, the new confidence value `NDPI_CONFIDENCE_DPI_AGGRESSIVE` has been
added.
It is always possible to disable/configure this kind of logic via the
API.

Detection of Ookla flows using plain TLS over port 8080 is the first
example of aggressive detection in nDPI.

Tested with:
* Android 9.0 with app 4.8.3
* Ubuntu 20.04 with Firefox 110
* Win 10 with app 1.15 and 1.16
* Win 10 with Chrome 108, Edge 108 and Firefox 106
2023-03-30 17:13:51 +02:00
Ivan Nardi 3e06bcce8d Add another example of custom rules (#1923)
Add an example where traffic matching the same IP, but different ports
is classified to different protocols.

Close #189
2023-03-30 08:45:17 +02:00
Ivan Nardi 6e0cd16225 TLS: fix another interger overflow in certificate processing (#1915) 2023-03-30 05:12:40 +02:00
Maatuq 9ea8a57839 remove redefinition to vxlanhdr struct in vxlan dissector (#1911) 2023-03-25 19:25:22 +01:00
Ivan Nardi 04a426feef ndpiReader: fix VXLAN de-tunneling (#1913)
```
==20665==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6040000aec81 at pc 0x0000004f5c6f bp 0x7fff07e9e1f0 sp 0x7fff07e9e1e8
READ of size 1 at 0x6040000aec81 thread T0
SCARINESS: 12 (1-byte-read-heap-buffer-overflow)
    #0 0x4f5c6e in ndpi_is_valid_vxlan ndpi/example/reader_util.c:1784:6
    #1 0x4f5c6e in ndpi_workflow_process_packet ndpi/example/reader_util.c:2292:16
    #2 0x4dd821 in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_ndpi_reader.c:135:7
    #3 0x4f91ba in ExecuteFilesOnyByOne /src/aflplusplus/utils/aflpp_driver/aflpp_driver.c:234:7
    #4 0x4f8f8c in main /src/aflplusplus/utils/aflpp_driver/aflpp_driver.c:318:12
    #5 0x7f2289324082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
    #6 0x41e6cd in _start
```
Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=57369
2023-03-25 19:19:51 +01:00
Ivan Nardi ddb9aa0f95 TLS: fix parsing of certificate elements (#1910)
```
==1228==ERROR: AddressSanitizer: SEGV on unknown address 0x6040000bed05 (pc 0x00000056e148 bp 0x7ffcca534320 sp 0x7ffcca5330c0 T0)
==1228==The signal is caused by a WRITE memory access.
    #0 0x56e148 in processCertificateElements ndpi/src/lib/protocols/tls.c:682:79
    #1 0x56c60f in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_tls_certificate.c:43:3
    #2 0x43de63 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15
    #3 0x4295c2 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:324:6
    #4 0x42ee6c in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:860:9
    #5 0x4583a2 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    #6 0x7f8c021c9082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
    #7 0x41f78d in _start
```

Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=57317
2023-03-22 18:18:36 +01:00
Maatuq 530d0de438 Add support for vxlan decapsulation (#1441) (#1900)
Close #1441
2023-03-22 18:18:12 +01:00
Ivan Nardi 04e017a854 Rework CI jobs to try reducing CI duration (#1903)
CI duration is quite long: the longest jobs is the "Performance" one.
Try to reduce the overall duration: that job (and some others) will not
be triggered for each PR/commit anymore, but asynchronously, once a day
(this scheduling seems right since the frequency of the PR/commits in
the project).
It should be possibly to trigger them manually, via GUI, anyway.

Remove two identical jobs; we already tests ASAN with 4 different
compilers.

After 9eff0754 it is safe to reduce fuzzing time.

Bottom line: try to have as upper-time of CI tests the duration of the
fuzzing jobs
2023-03-22 18:17:25 +01:00
Luca Deri 64ebf73b29 Added the ability to define custom protocols with arbitrary Ids in proto.txt
Example
- ip:213.75.170.11/32:443@CustomProtocol
nDPI assigns an is that can change based on protos.txt content

- ip:213.75.170.11/32:443@CustomProtocol=9999
nDPI assigns 9999 as protocolId to CustomProtocol and won't change when
protos.txt content will chaneg
2023-03-22 00:15:56 +01:00
Luca Deri b3457c644f Removed overlapping port 2023-03-21 19:27:56 +01:00
Luca Deri c22d30786c Improved checks for duplicated entries in protocols file 2023-03-21 19:25:03 +01:00
Ivan Nardi 2a3ade397b DisneyPlus/Hulu ip lists should be auto-generated (#1905)
Remove two stale ip lists:
1) these 3 ips are in the Amazon ranges (now)...
2) the Instagram list originated from AS32934, which is now a Facebook
AS; see https://github.com/ntop/nDPI/pull/1264/commits/8dabd06301a802dd38616ba8684a1d995783e023
2023-03-20 19:26:40 +01:00
Ivan Nardi 3a75a46212 Add a new protocol id for generic Adult Content traffic (#1906)
The list has been taken from https://www.similarweb.com/top-websites/adult/
Fix a GoTo false positive.
2023-03-20 17:56:02 +01:00
Ivan Nardi 12867962b0 Add a new protocol id for generic advertisement/analytics/tracking stuff (#1904) 2023-03-20 17:01:18 +01:00
Ivan Nardi 0e80828e14 fuzz: add a new fuzzer to test TLS certificates (#1901) 2023-03-20 17:00:46 +01:00
Luca Deri 9f46d41eca FreeBSD compilation fix
(C) update
2023-03-14 22:16:57 +01:00
Ivan Nardi 9eff075419 fuzz: simplify fuzzers dependencies in CIFuzz (#1896)
CIFuzz (based on oss-fuzz) is the GitHub action/CI job that runs fuzz
targets on pull requests. It only runs the fuzzers affected by a pull
request/commit. Otherwise it will divide up the allotted fuzzing time
among all fuzzers in the project.
Since:
* we have more than 20 fuzzers and most of them use the custom memory
allocation functions (to force allocation failures) even if they are not
strictly about DPI stuff;
* we need to keep fuzzing time relatively small (to avoid waiting the CI
results for a long time)

it is important that fuzzers dependencies (which are based on *files*
changed by the single commit/PR) are as small as possible.

Bottom line: move all the low-level allocation callbacks to a dedicated
file; this way most of the fuzzers don't depend anymore on `ndpi_main.c`
file (which is touched by ever commit/PR).

The goal is to have only the "most important" fuzzers running during (most
of) the CI.
2023-03-14 19:34:01 +01:00
Luca Deri 3585e2d201 Added ability to define an unlimited number of custom rules IP:port for the same IP (it used tobe limited to 2) 2023-03-13 21:57:14 +01:00
Luca Deri ebfc1fe827 Minor cleanup 2023-03-13 20:52:08 +01:00
Luca Deri d398f232be Added fix for better handling exceptions rollback in case of later match 2023-03-06 17:33:08 +01:00
Ivan Nardi 89cae9ddf2 Add a new flow risk about literal IP addresses used as SNI (#1892)
RFC 6066 3: "Literal IPv4 and IPv6 addresses are not permitted in
"HostName"."

Don't set this risk if we have a valid sub-classification (example:
via certificate)

Since a similar risk already exists for HTTP hostnames, reuse it, with a
more generic name.
2023-03-02 15:27:30 +01:00
Ivan Nardi 3047e286c0 Fix insert of ip addresses into patricia tree(s) (#1895)
Fix: d0b46dcf
2023-03-02 14:30:20 +01:00
Ivan Nardi 22fb8349b9 ndpiReader: print how many packets (per flow) were needed to perform full DPI (#1891)
Average values are already printed, but this change should ease to
identify regressions/improvements.
2023-03-01 21:50:47 +01:00
Toni 2768366969 Fixed missing AS_HELP_STRING in configure.ac. (#1893)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-03-01 20:46:40 +01:00
Nardi Ivan e0ad5b49c0 Sync unit tests results 2023-03-01 20:34:56 +01:00
Nardi Ivan d65f8cf15f Keep master protocol in ndpi_reconcile_protocols
As a side effect of this change, the following memory leak has been
fixed.
```
==55523==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 2 byte(s) in 1 object(s) allocated from:
    #0 0x55657d01b7be in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_alloc_fail+0x5b27be) (BuildId: 9dfa7bd9081421417b8b1c2e8b44633a9bfcfe52)
    #1 0x55657d169182 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:243:25
    #2 0x55657d1694c8 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:310:13
    #3 0x55657d25ddc7 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2308:46
    #4 0x55657d2701b9 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:895:5
    #5 0x55657d26a408 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1223:11
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56466
2023-03-01 20:34:56 +01:00
Nardi Ivan 9eb9664516 SoftEther: fix invalid memory access
We can't write `flow->protos` union until we are really sure about protocol
classification

```
==28334==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x558db5554512 bp 0x000000000000 sp 0x7ffcb22c2880 T0)
==28334==The signal is caused by a READ memory access.
==28334==Hint: this fault was caused by a dereference of a high value address (see register values below).  Disassemble the provided pc to learn which register was used.
    #0 0x558db5554512 in __asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x48e512) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7)
    #1 0x558db55ea54b in __interceptor_free (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x52454b) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7)
    #2 0x558db56977ca in ndpi_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:274:7
    #3 0x558db56c20e3 in ndpi_free_flow_data /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5175:2
    #4 0x558db569783f in ndpi_free_flow /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8394:5
    #5 0x558db5627936 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:38:3
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56272
2023-03-01 20:34:56 +01:00
Luca Deri 127d8d0d35 Fixed uninitialized variable 2023-02-27 21:26:41 +01:00
Luca Deri 21c7550157 Moving to bash 2023-02-27 16:21:12 +01:00
Luca Deri a303756edb Added missing break 2023-02-27 14:38:40 +01:00
Luca Deri 25420e266e Updated results after the latest changes 2023-02-27 12:20:39 +01:00
Luca Deri 807c97344f in case of failure, failing result files are not listed 2023-02-27 12:20:15 +01:00
Luca Deri 96f0f85e56 Indent fix 2023-02-27 12:20:06 +01:00
Luca Deri 6a2e8c9fb1 Improved protocol detection exploiting IP-based guess
Reworked ndpi_reconcile_protocols() that is now called only in front of a match (less overhead)
2023-02-27 12:18:39 +01:00
Luca Deri d0b46dcf88 Enhanced custom port definition and improved error reporting in case of duplications 2023-02-24 09:35:26 +01:00
Ivan Nardi 4075324e2b fuzz: extend fuzz coverage (#1888) 2023-02-16 18:04:34 +01:00
Ivan Nardi 06d4f8c7e5 Update GitHub runners versions (#1889)
See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
2023-02-14 21:30:54 +01:00
Ivan Nardi f2f8f9d0ee Fix export/serialization of flow->risk (#1885)
Any risks bigger than 31 was never exported/serialized; found looking
a the fuzz coverage report.
2023-02-14 21:30:36 +01:00
Ivan Nardi 090ba5735f Sync utests results (#1887) 2023-02-13 13:17:01 +01:00
Ivan Nardi f10178f8d2 Fix packet counters (#1884)
We need to keep separete counters to keep tracks of packet numbers with
and without any payload.

Regression introduced in 5849863ef
2023-02-13 11:32:17 +01:00
Ivan Nardi 200d1d4cfc Fix detection of packet direction and NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1883) 2023-02-13 11:31:52 +01:00
Ivan Nardi a3a9a72f31 Fix compilation with GCC-7 and latest RoaringBitmap code (#1886)
Latest RoaringBitmap version (introduced with bf413afb) triggers a new
warning with GCC-7:

```
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ CC=gcc-7 CXX=g++-7 ./autogen.sh && make -s
autoreconf: Entering directory `.'
[...]
third_party/src/roaring.c:1815:1: warning: ‘no_sanitize’ attribute directive ignored [-Wattributes]
 static inline int array_container_cardinality(const array_container_t *array) {
 ^~~~~~
third_party/src/roaring.c:1964:5: warning: ‘no_sanitize’ attribute directive ignored [-Wattributes]
     const array_container_t *container2) {
[..]
```

The core issue is that `no_sanitize` attribute is defined only for GCC
>= 8.
That breaks the CI since we still use GCC-7 and `-Werror`: add a simple
workaround.

Fix compilation on Windows
2023-02-13 11:19:11 +01:00
Luca Deri 85bab9e660 Centos7 fixes 2023-02-13 01:02:27 +01:00
Luca Deri bf413afba1 Update roaring bitmap code 2023-02-12 12:50:24 +01:00
0xA50C1A1 ba4e145aad Add Yandex services detection (#1882)
Add Yandex services detection

Add VK and Yandex to the TLS certificate match list
2023-02-09 20:02:43 +01:00
Ivan Nardi b51a2ac72a fuzz: some improvements and add two new fuzzers (#1881)
Remove `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` define from
`fuzz/Makefile.am`; it is already included by the main configure script
(when fuzzing).

Add a knob to force disabling of AESNI optimizations: this way we can
fuzz also no-aesni crypto code.

Move CRC32 algorithm into the library.

Add some fake traces to extend fuzzing coverage. Note that these traces
are hand-made (via scapy/curl) and must not be used as "proof" that the
dissectors are really able to identify this kind of traffic.

Some small updates to some dissectors:

CSGO: remove a wrong rule (never triggered, BTW). Any UDP packet starting
with "VS01" will be classified as STEAM (see steam.c around line 111).
Googling it, it seems right so.

XBOX: XBOX only analyses UDP flows while HTTP only TCP ones; therefore
that condition is false.

RTP, STUN: removed useless "break"s

Zattoo: `flow->zattoo_stage` is never set to any values greater or equal
to 5, so these checks are never true.

PPStream: `flow->l4.udp.ppstream_stage` is never read. Delete it.

TeamSpeak: we check for `flow->packet_counter == 3` just above, so the
following check `flow->packet_counter >= 3` is always false.
2023-02-09 20:02:12 +01:00
0xA50C1A1 4bb851384e Add VK detection (#1880) 2023-02-02 15:27:59 +01:00
Luca Deri 64b8ae38db Typos 2023-02-01 21:15:30 +00:00
Luca Deri 563a24bf6b Minor update 2023-02-01 18:31:59 +01:00
Luca Deri c711f480e3 Moved to 4.7 2023-02-01 18:30:17 +01:00
3958 changed files with 337204 additions and 109867 deletions
+2
View File
@@ -1,3 +1,5 @@
mpile
# Codige Code Review ignore
# Prevent Codiga complaining about Sphinx config.py
+1 -1
View File
@@ -21,7 +21,7 @@ A clear and concise description of what happening.
* OS version: [e.g. 18.04]
* Architecture: [e.g. arm64]
* nDPI version or commit hash: [e.g. 4.0-stable or 937357e4bc55610f116f66d15a8e0fc1e260c02c].
* nDPI compilation flags used: if you are building from source [e.g. --with-pcre --with-local-libgcrypt].
* nDPI compilation flags used: if you are building from source [e.g. --with-pcre2 --with-local-libgcrypt].
* Attach the `config.log` file generated after `./configure` ran (if you are building from source).
## How to reproduce the reported bug
+12
View File
@@ -0,0 +1,12 @@
Please sign (check) the below before submitting the Pull Request:
- [ ] I have signed the ntop Contributor License Agreement at https://github.com/ntop/legal/blob/main/individual-contributor-licence-agreement.md
- [ ] I have read the contributing guide lines at https://github.com/ntop/nDPI/blob/dev/CONTRIBUTING.md
- [ ] I have updated the documentation (in doc/) to reflect the changes made (if applicable)
Link to the related [issue](https://github.com/ntop/nDPI/issues):
Describe changes:
+3 -3
View File
@@ -15,10 +15,10 @@ jobs:
runs-on: windows-latest
steps:
- name: Checkout
uses: actions/checkout@v3
uses: actions/checkout@v4
- name: Add msbuild to PATH
uses: microsoft/setup-msbuild@v1.1
uses: microsoft/setup-msbuild@v2
- name: Download NuGet packages
run: msbuild windows/nDPI.sln -t:restore -p:RestorePackagesConfig=true
@@ -36,7 +36,7 @@ jobs:
run: msbuild windows/nDPI.sln -t:rebuild -property:Configuration=Debug-ndpiReader
- name: Upload artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: build-artifacts
path: windows/bin
+31
View File
@@ -0,0 +1,31 @@
name: RPM Build
on:
workflow_dispatch:
schedule:
# At the end of every day
- cron: '0 0 * * *'
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
- name: Configure nDPI
run: |
# fake CentOS 7 env
echo 'CentOS Linux release 7.9.2009 (Core)' | sudo tee /etc/centos-release
# symlink source directory for ndpi.spec
ln -sr . $HOME/nDPI
./autogen.sh
cd packages/rpm
./configure
- name: Build RPM package
run: |
make -C packages/rpm package
+107 -193
View File
@@ -10,51 +10,17 @@ on:
release:
types: [created]
jobs:
coverage:
name: Coverage (ubuntu-latest)
runs-on: ubuntu-latest
env:
CFLAGS: -Werror
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
sudo apt-get install rrdtool librrd-dev
sudo apt-get install libpcre3-dev libmaxminddb-dev lcov
sudo apt-get install wdiff colordiff
- name: Configure
run: ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-code-coverage --with-pcre --with-maxminddb --enable-tls-sigs
- name: Build
run: make all
- name: Test
run: |
make check VERBOSE=1
- name: Generate Coverage Report
run: |
mkdir -vp coverage_report/ndpi_coverage_report
lcov --directory . --capture --output-file lcov.info
genhtml -o coverage_report/ndpi_coverage_report lcov.info
- uses: actions/upload-artifact@v3
with:
name: ndpi-coverage-report
path: coverage_report
retention-days: 7
python-bindings:
name: Python Bindings (ubuntu-latest)
runs-on: ubuntu-latest
env:
CFLAGS: -Werror
CFLAGS: -Wextra -Werror
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python 3.9
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: "3.9"
- name: Install Ubuntu Prerequisites
@@ -64,7 +30,7 @@ jobs:
- name: Build nDPI library
run: |
./autogen.sh
make
make -j
sudo make install
- name: Generate Python bindings
run: |
@@ -77,87 +43,17 @@ jobs:
run: |
cd python
python tests.py
documentation:
name: Documentation (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Python 3.9
uses: actions/setup-python@v4
with:
python-version: "3.9"
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison doxygen
- name: Configure nDPI library
run: |
./autogen.sh --with-only-libndpi --enable-option-checking=fatal
- name: Generate Documentation
run: |
pip install --upgrade pip
pip install -r doc/requirements.txt
make doc
mkdir -vp doc/_build/ndpi-documentation-upload/ndpi-documentation
mv -v doc/_build/html doc/_build/ndpi-documentation-upload/ndpi-documentation/html
- uses: actions/upload-artifact@v3
with:
name: ndpi-documentation
path: doc/_build/ndpi-documentation-upload
retention-days: 7
performance:
name: Performance (ubuntu-latest)
runs-on: ubuntu-latest
env:
GO111MODULE: on
CFLAGS: -Werror
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo add-apt-repository ppa:longsleep/golang-backports
sudo apt-get update
sudo apt-get install libunwind-dev
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison libpcap-dev
sudo apt-get install golang-1.16-go libgoogle-perftools-dev graphviz
go install github.com/google/pprof@latest
sudo ln -s ${HOME}/go/bin/pprof /usr/bin/pprof
pprof -h
- name: Configure nDPI library
run: |
./autogen.sh --enable-gprof --enable-option-checking=fatal --with-pcre --with-maxminddb --enable-tls-sigs
- name: Build nDPI library
run: |
make
- name: Performance Profiling
run: |
./tests/do.sh
mkdir ndpi-performance-upload
mv -v tests/result/cpu_profile.png ndpi-performance-upload/cpu_profile.png
mv -v tests/result/heap_profile.png ndpi-performance-upload/heap_profile.png
- uses: actions/upload-artifact@v3
with:
name: ndpi-performance
path: ndpi-performance-upload
retention-days: 7
test-scripts:
name: Test Utils (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install python3-netaddr git
sudo apt-get install python3-netaddr git whois libxml2-utils
- name: Run Scripts
run: |
echo 'Running ./utils/bitcoinnodes.sh'
@@ -171,15 +67,17 @@ jobs:
git diff-index --quiet HEAD -- || true
test:
name: ${{ matrix.os }} ${{ matrix.arch }} ${{ matrix.gcrypt }} ${{ matrix.compiler }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} ${{ matrix.msan }} ${{ matrix.nBPF }} ${{matrix.lto_gold_linker}}
name: ${{ matrix.os }} ${{ matrix.arch }} ${{ matrix.gcrypt }} ${{ matrix.compiler }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} ${{ matrix.msan }} ${{ matrix.nBPF }} ${{matrix.lto_gold_linker}} ${{matrix.global_context}}
runs-on: ${{ matrix.os }}
env:
CC: ${{ matrix.compiler }}
CFLAGS: -Werror
CFLAGS: -Wextra -Werror -DNDPI_EXTENDED_SANITY_CHECKS
strategy:
fail-fast: true
matrix:
os: ["ubuntu-latest", "ubuntu-22.04", "macOS-latest", "macos-12", "windows-latest"]
# macOS-latest == macos-14 on **ARM64**
# There are some issues with external dependencies on macOS-latest. Disable it for the time being
os: ["ubuntu-20.04", "ubuntu-22.04", "ubuntu-24.04", "macOS-12", "macOS-13", "windows-latest"]
arch: ["x86_64"]
gcrypt: ["--with-local-libgcrypt", ""]
compiler: ["cc"]
@@ -190,56 +88,64 @@ jobs:
msan: [""]
nBPF: [""]
lto_gold_linker: [""]
global_context: [""] #Enable by default
include:
- compiler: "gcc-7" # "Oldest" gcc easily available
- compiler: "gcc-4.9" # "Oldest" gcc easily available. To simulate RHEL7
os: ubuntu-20.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
- compiler: "gcc-12" # "Newest" gcc easily available
os: ubuntu-22.04
- compiler: "gcc-14" # "Newest" gcc easily available
os: ubuntu-24.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
lto_gold_linker: "--with-lto-and-gold-linker"
- compiler: "clang-7" # "Oldest" clang easily available
- compiler: "clang-9" # "Oldest" clang easily available
os: ubuntu-20.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
- compiler: "clang-14" # "Newest" clang easily available
ar: "llvm-ar-14"
ranlib: "llvm-ranlib-14"
os: ubuntu-22.04
- compiler: "clang-17" # "Newest" clang easily available. See also below...
ar: "llvm-ar-17"
ranlib: "llvm-ranlib-17"
os: ubuntu-24.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
lto_gold_linker: "--with-lto-and-gold-linker"
- compiler: "cc"
os: ubuntu-latest
- compiler: "clang-18" # "The latest clang version easily available should be 18,
# but it is buggy on ubuntu-24.04 with LTO and Gold linker:
# https://github.com/llvm/llvm-project/issues/87553
# https://bugs.launchpad.net/ubuntu/+source/llvm-toolchain-18/+bug/2064187
# Waiting for a fix...
ar: "llvm-ar-18"
ranlib: "llvm-ranlib-18"
os: ubuntu-24.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
lto_gold_linker: ""
- compiler: "cc"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-thread-sanitizer"
nBPF: ""
@@ -247,47 +153,48 @@ jobs:
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: "nBPF"
- compiler: "clang-14"
os: ubuntu-22.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-memory-sanitizer"
nBPF: ""
- compiler: "cc"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
- compiler: "cc"
os: macOS-latest
global_context: "--disable-global-context-support"
- compiler: "clang" #TODO: some issues with masan/clang/ubuntu-24.04
os: ubuntu-22.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-memory-sanitizer"
nBPF: ""
- compiler: "cc"
os: macOS-13
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "" # Disable sanitizer on macos
nBPF: ""
- compiler: "cc"
os: macos-12
os: macos-14
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
pcre: ""
maxminddb: ""
msan: "" # Disable sanitizer on macos
nBPF: ""
- compiler: "cc"
os: ubuntu-latest
arch: "arm64"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "" # Disable sanitizer on arm64
nBPF: ""
@@ -295,7 +202,7 @@ jobs:
os: ubuntu-latest
arch: "armhf"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
@@ -303,7 +210,7 @@ jobs:
os: ubuntu-latest
arch: "s390x"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: ""
nBPF: ""
@@ -312,9 +219,15 @@ jobs:
if: startsWith(matrix.os, 'ubuntu') && !startsWith(matrix.arch, 'x86_64')
run: |
docker run --rm --privileged multiarch/qemu-user-static:register --reset
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis
- name: Fix kernel mmap rnd bits on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
run: |
# Workaround for compatinility between latest kernel and sanitizer
# See https://github.com/actions/runner-images/issues/9491
sudo sysctl vm.mmap_rnd_bits=28
- name: Install Ubuntu Prerequisites
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
run: |
@@ -322,13 +235,13 @@ jobs:
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
sudo apt-get install gcc-mingw-w64 libc6-dev
sudo apt-get install doxygen python3-sphinx python3-sphinx-rtd-theme python3-breathe python3-pip
sudo apt-get install rrdtool librrd-dev
sudo apt-get install rrdtool librrd-dev parallel
- name: Install Ubuntu Prerequisites (libgcrypt)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.gcrypt, '--with-local-libgcrypt')
run: |
sudo apt-get install libgcrypt20-dev
- name: Install Ubuntu Prerequisites (libpcre)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.pcre, '--with-pcre')
- name: Install Ubuntu Prerequisites (libpcre2)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.pcre, '--with-pcre2')
run: |
sudo apt-get install libpcre3-dev
- name: Install Ubuntu Prerequisites (maxminddb)
@@ -344,8 +257,13 @@ jobs:
make
cd -
- name: Setup Ubuntu specified compiler
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && ! startsWith(matrix.compiler, 'cc')
if: startsWith(matrix.os, 'ubuntu-20.04') && startsWith(matrix.arch, 'x86_64') && ! startsWith(matrix.compiler, 'cc')
run: |
#For gcc-4.9 (on ubuntu-20.04)
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial main" | sudo tee -a /etc/apt/sources.list
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial universe" | sudo tee -a /etc/apt/sources.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 40976EAF437D05B5
sudo apt-get update
sudo apt-get install ${{ matrix.compiler }}
- name: Install Windows msys2 prerequisites
if: startsWith(matrix.os, 'windows')
@@ -353,39 +271,22 @@ jobs:
with:
msystem: MINGW64
update: true
install: git mingw-w64-x86_64-toolchain automake1.16 automake-wrapper autoconf libtool make mingw-w64-x86_64-json-c mingw-w64-x86_64-crt-git mingw-w64-x86_64-pcre mingw-w64-x86_64-libpcap
- name: Setup Windows msys2 prerequisites (libgcrypt)
if: startsWith(matrix.os, 'windows') && startsWith(matrix.gcrypt, '--with-local-libgcrypt')
run: |
msys2 -c 'git clone --branch libgpg-error-1.42 https://github.com/gpg/libgpg-error'
cd libgpg-error
msys2 -c './autogen.sh'
msys2 -c './configure -enable-maintainer-mode --enable-static --enable-shared --with-pic --disable-doc --disable-nls'
msys2 -c 'make'
msys2 -c 'make install'
msys2 -c 'cd ..'
msys2 -c 'rm -rf libgpg-error/'
msys2 -c 'git clone --branch libgcrypt-1.8.8 https://github.com/gpg/libgcrypt'
cd libgcrypt
msys2 -c './autogen.sh'
msys2 -c './configure -enable-maintainer-mode --enable-static --enable-shared --with-pic --disable-doc'
msys2 -c 'make'
msys2 -c 'make install'
msys2 -c 'cd ..'
msys2 -c 'rm -rf libgcrypt/'
env:
CC:
CFLAGS:
install: git mingw-w64-x86_64-toolchain automake1.16 automake-wrapper autoconf libtool make mingw-w64-x86_64-json-c mingw-w64-x86_64-crt-git mingw-w64-x86_64-pcre mingw-w64-x86_64-libpcap mingw-w64-x86_64-libgcrypt parallel
- name: Installing MacOS prerequisites
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64')
run: |
brew install coreutils wdiff colordiff autoconf automake libtool pkg-config gettext json-c rrdtool
# A workaround for "The `brew link` step did not complete successfully" error.
# See https://github.com/Homebrew/homebrew-core/issues/165793#issuecomment-1991817938
find /usr/local/bin -lname '*/Library/Frameworks/Python.framework/*' -delete
sudo rm -rf /Library/Frameworks/Python.framework/
brew install --force python3 && brew unlink python3 && brew unlink python3 && brew link --overwrite python3
brew install coreutils wdiff colordiff autoconf automake libtool pkg-config gettext json-c rrdtool parallel
- name: Install MacOS Prerequisites (libgcrypt)
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.gcrypt, '--with-local-libgcrypt')
run: |
brew install libgcrypt
- name: Install MacOS Prerequisites (libpcre)
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.pcre, '--with-pcre')
- name: Install MacOS Prerequisites (libpcre2)
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.pcre, '--with-pcre2')
run: |
brew install pcre
- name: Install MacOS Prerequisites (maxminddb)
@@ -395,7 +296,7 @@ jobs:
- name: Configure nDPI on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
run: |
AR=${{ matrix.ar }} RANLIB=${{ matrix.ranlib }} ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs ${{matrix.lto_gold_linker}}
AR=${{ matrix.ar }} RANLIB=${{ matrix.ranlib }} ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs ${{matrix.lto_gold_linker}} ${{matrix.global_context}}
- name: Configure nDPI on MacOS
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'cc')
run: |
@@ -407,30 +308,43 @@ jobs:
- name: Build nDPI on Windows msys2
if: startsWith(matrix.os, 'windows') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'cc')
run: |
msys2 -c 'make all'
msys2 -c 'make -j all'
msys2 -c 'ldd ./example/ndpiReader.exe'
- name: Build nDPI
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows') && !startsWith(matrix.os, 'macos-14')
run: |
make all
make -j all
make -C example ndpiSimpleIntegration
make -C rrdtool
- name: Build nDPI (MacOS M1)
if: startsWith(matrix.os, 'macos-14')
run: |
make -j all
make -C example ndpiSimpleIntegration
#There are somes issues with librrd
#make -C rrdtool
- name: Print nDPI long help
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
./example/ndpiReader -H
cd ./example && ./ndpiReader -H
- name: Install nDPI
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
make install DESTDIR=$(realpath _install)
ls -alhHR _install
DESTDIR=/tmp/ndpi make install
ls -alhHR /tmp/ndpi
- name: Test nDPI [SYMBOLS]
if: (startsWith(matrix.os, 'ubuntu') || startsWith(matrix.os, 'mac')) && startsWith(matrix.arch, 'x86_64')
run: |
./utils/check_symbols.sh || { FAILED=$?; echo "::error file=${NDPI_LIB}::Unwanted libc symbols found: ${FAILED}. Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of malloc/calloc/realloc/free."; false; }
env:
NDPI_LIB: src/lib/libndpi.a
- name: Test nDPI [DIFF]
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
./tests/do.sh
NDPI_FORCE_PARALLEL_UTESTS=1 NDPI_SKIP_PARALLEL_BAR=1 ./tests/do.sh
- name: Test nDPI [UNIT]
#Some issues with masan + json-c. Disable the test as workaround
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows') && !startsWith(matrix.msan, '--with-memory-sanitizer')
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows') && !startsWith(matrix.msan, '--with-memory-sanitizer') && !startsWith(matrix.os, 'macos-14')
run: |
./tests/do-unit.sh
- name: Test nDPI [DGA]
@@ -440,7 +354,7 @@ jobs:
- name: Test nDPI [DIFF] (runs only on windows jobs)
if: startsWith(matrix.arch, 'x86_64') && startsWith(matrix.os, 'windows')
run: |
msys2 -c './tests/do.sh'
msys2 -c 'NDPI_FORCE_PARALLEL_UTESTS=1 NDPI_SKIP_PARALLEL_BAR=1 ./tests/do.sh'
- name: Test nDPI [UNIT] (runs only on windows jobs)
if: startsWith(matrix.arch, 'x86_64') && startsWith(matrix.os, 'windows')
run: |
@@ -459,7 +373,7 @@ jobs:
run: |
make distclean
./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --host=x86_64-w64-mingw32
make all
make -j $(nproc) all
env:
CC:
- name: Display qemu specified architecture (arm64 - little endian)
@@ -481,7 +395,7 @@ jobs:
apt-get -y install git wdiff colordiff autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
git config --global --add safe.directory $(realpath .) &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs &&
make all &&
make -j all &&
make -C example ndpiSimpleIntegration &&
make -C rrdtool &&
make check VERBOSE=1
@@ -505,7 +419,7 @@ jobs:
apt-get -y install git wdiff colordiff autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
git config --global --add safe.directory $(realpath .) &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs &&
make all &&
make -j all &&
make -C example ndpiSimpleIntegration &&
make -C rrdtool &&
make check VERBOSE=1
@@ -529,7 +443,7 @@ jobs:
apt-get -y install git wdiff colordiff autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
git config --global --add safe.directory $(realpath .) &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs &&
make all &&
make -j all &&
make -C example ndpiSimpleIntegration &&
make -C rrdtool &&
make check VERBOSE=1
+111
View File
@@ -0,0 +1,111 @@
name: Scheduled builds
on:
workflow_dispatch:
schedule:
#At the end of every day
- cron: '0 0 * * *'
jobs:
coverage:
name: Coverage (ubuntu-latest)
runs-on: ubuntu-latest
env:
CFLAGS: -Werror
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
sudo apt-get install rrdtool librrd-dev
sudo apt-get install libpcre3-dev libmaxminddb-dev lcov
sudo apt-get install wdiff colordiff
- name: Configure
run: ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-code-coverage --with-pcre2 --with-maxminddb --enable-tls-sigs
- name: Build
run: make all
- name: Test
run: |
make check VERBOSE=1
- name: Generate Coverage Report
run: |
make coverage
- uses: actions/upload-artifact@v4
with:
name: ndpi-coverage-report
path: coverage_report
retention-days: 7
documentation:
name: Documentation (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python 3.9
uses: actions/setup-python@v4
with:
python-version: "3.9"
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison doxygen
- name: Configure nDPI library
run: |
./autogen.sh --with-only-libndpi --enable-option-checking=fatal --enable-debug-build
- name: Generate Documentation
run: |
pip install --upgrade pip
pip install -r doc/requirements.txt
make doc
mkdir -vp doc/_build/ndpi-documentation-upload/ndpi-documentation
mv -v doc/_build/html doc/_build/ndpi-documentation-upload/ndpi-documentation/html
- uses: actions/upload-artifact@v4
with:
name: ndpi-documentation
path: doc/_build/ndpi-documentation-upload
retention-days: 7
performance:
name: Performance (ubuntu-latest)
runs-on: ubuntu-latest
env:
GO111MODULE: on
CFLAGS: -Werror
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo add-apt-repository ppa:longsleep/golang-backports
sudo apt-get update
sudo apt-get install libunwind-dev
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison libpcap-dev parallel
sudo apt-get install golang-1.16-go libgoogle-perftools-dev graphviz
go install github.com/google/pprof@latest
sudo ln -s ${HOME}/go/bin/pprof /usr/bin/pprof
pprof -h
- name: Configure nDPI library
run: |
./autogen.sh --enable-gprof --enable-option-checking=fatal --with-pcre2 --with-maxminddb --enable-tls-sigs
- name: Build nDPI library
run: |
make -j
- name: Performance Profiling
run: |
NDPI_FORCE_PARALLEL_UTESTS=1 NDPI_SKIP_PARALLEL_BAR=1 ./tests/do.sh
mkdir ndpi-performance-upload
for d in $(find ./tests/cfgs/* -type d -maxdepth 0 2>/dev/null) ; do
PROFILE="$(basename $d)"
mv -v tests/cfgs/${PROFILE}/result/cpu_profile.png ndpi-performance-upload/${PROFILE}_cpu_profile.png
mv -v tests/cfgs/${PROFILE}/result/heap_profile.png ndpi-performance-upload/${PROFILE}_heap_profile.png
done
- uses: actions/upload-artifact@v4
with:
name: ndpi-performance
path: ndpi-performance-upload
retention-days: 7
+2 -2
View File
@@ -18,7 +18,7 @@ jobs:
uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@master
with:
oss-fuzz-project-name: 'ndpi'
fuzz-seconds: 1600
fuzz-seconds: 1320
dry-run: false
sanitizer: ${{ matrix.sanitizer }}
- name: Check Crash (fails when a crash is detected)
@@ -27,7 +27,7 @@ jobs:
run: |
exit $(ls out/artifacts |wc -l)
- name: Upload Crash (upload detected crash as artifacts)
uses: actions/upload-artifact@v1
uses: actions/upload-artifact@v4
if: failure()
with:
name: ${{ matrix.sanitizer }}-artifacts
+6 -6
View File
@@ -24,10 +24,10 @@ jobs:
strategy:
fail-fast: false
matrix:
language: [ 'cpp', 'python', 'ruby' ]
language: [ 'cpp', 'python' ]
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
@@ -42,18 +42,18 @@ jobs:
pprof -h
- name: Configure nDPI library
run: |
./autogen.sh --enable-gprof --enable-option-checking=fatal --with-pcre --with-maxminddb --enable-tls-sigs --enable-debug-messages
./autogen.sh --enable-gprof --enable-option-checking=fatal --with-pcre2 --with-maxminddb --enable-tls-sigs --enable-debug-messages
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
- name: Build nDPI library
run: |
make
make -j
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
uses: github/codeql-action/analyze@v3
with:
category: "/language:${{matrix.language}}"
+60 -5
View File
@@ -58,30 +58,84 @@
/fuzz/fuzz_alg_hll
/fuzz/fuzz_alg_hw_rsi_outliers_da
/fuzz/fuzz_alg_jitter
/fuzz/fuzz_alg_crc32_md5
/fuzz/fuzz_alg_ses_des
/fuzz/fuzz_alg_bytestream
/fuzz/fuzz_alg_shoco
/fuzz/fuzz_config
/fuzz/fuzz_community_id
/fuzz/fuzz_serialization
/fuzz/fuzz_ds_patricia
/fuzz/fuzz_ds_libcache
/fuzz/fuzz_ds_tree
/fuzz/fuzz_ds_ptree
/fuzz/fuzz_ds_hash
/fuzz/fuzz_ds_ahocorasick
/fuzz/fuzz_ds_bitmap64_fuse
/fuzz/fuzz_ds_domain_classify
/fuzz/fuzz_libinjection
/fuzz/fuzz_binaryfusefilter
/fuzz/fuzz_tls_certificate
/fuzz/fuzz_dga
/fuzz/fuzz_is_stun_udp
/fuzz/fuzz_is_stun_tcp
/fuzz/fuzz_ds_cmsketch
/fuzz/fuzz_gcrypt_light
/fuzz/fuzz_gcrypt_aes
/fuzz/fuzz_gcrypt_gcm
/fuzz/fuzz_gcrypt_cipher
/fuzz/fuzz_ndpi_reader_payload_analyzer
/fuzz/fuzz_ndpi_reader_pl7m_simplest
/fuzz/fuzz_ndpi_reader_pl7m_simplest_internal
/fuzz/fuzz_ndpi_reader_pl7m
/fuzz/fuzz_ndpi_reader_pl7m_64k
/fuzz/fuzz_ndpi_reader_pl7m_internal
/fuzz/fuzz_filecfg_protocols
/fuzz/fuzz_filecfg_categories
/fuzz/fuzz_filecfg_category
/fuzz/fuzz_filecfg_malicious_sha1
/fuzz/fuzz_filecfg_malicious_ja3
/fuzz/fuzz_filecfg_risk_domains
/fuzz/fuzz_filecfg_config
/fuzz/fuzz_readerutils_workflow
/fuzz/fuzz_readerutils_parseprotolist
/fuzz/fuzz_ndpi_reader_alloc_fail_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_pl7m_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_pl7m_64k_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_pl7m_internal_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_pl7m_simplest_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_pl7m_simplest_internal_seed_corpus.zip
/fuzz/fuzz_quic_get_crypto_data_seed_corpus.zip
/fuzz/fuzz_config_seed_corpus.zip
/fuzz/fuzz_community_id_seed_corpus.zip
/fuzz/fuzz_is_stun_udp_seed_corpus.zip
/fuzz/fuzz_is_stun_tcp_seed_corpus.zip
/fuzz/fuzz_serialization_seed_corpus.zip
/fuzz/fuzz_alg_ses_des_seed_corpus.zip
/fuzz/fuzz_alg_bins_seed_corpus.zip
/fuzz/fuzz_alg_hll_seed_corpus.zip
/fuzz/fuzz_alg_jitter_seed_corpus.zip
/fuzz/fuzz_alg_hw_rsi_outliers_da_seed_corpus.zip
/fuzz/fuzz_ds_patricia_seed_corpus.zip
/fuzz/fuzz_alg_crc32_md5_seed_corpus.zip
/fuzz/fuzz_alg_bytestream_seed_corpus.zip
/fuzz/fuzz_ds_libcache_seed_corpus.zip
/fuzz/fuzz_ds_tree_seed_corpus.zip
/fuzz/fuzz_ds_ahocorasick_seed_corpus.zip
/fuzz/fuzz_ds_ptree_seed_corpus.zip
/fuzz/fuzz_ds_bitmap64_fuse_seed_corpus.zip
/fuzz/fuzz_ds_domain_classify_seed_corpus.zip
/fuzz/fuzz_libinjection_seed_corpus.zip
/fuzz/fuzz_tls_certificate_seed_corpus.zip
/fuzz/fuzz_filecfg_protocols_seed_corpus.zip
/fuzz/fuzz_filecfg_categories_seed_corpus.zip
/fuzz/fuzz_filecfg_category_seed_corpus.zip
/fuzz/fuzz_filecfg_malicious_sha1_seed_corpus.zip
/fuzz/fuzz_filecfg_malicious_ja3_seed_corpus.zip
/fuzz/fuzz_filecfg_risk_domains_seed_corpus.zip
/fuzz/fuzz_filecfg_config_seed_corpus.zip
/fuzz/fuzz_dga_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_payload_analyzer_seed_corpus.zip
/fuzz/fuzz_readerutils_workflow_seed_corpus.zip
/fuzz/fuzz_readerutils_parseprotolist_seed_corpus.zip
/fuzz/fuzz_*.dict
/fuzz/testpcaps.zip
/influxdb/Makefile
/install-sh
/libndpi.pc
@@ -108,6 +162,7 @@
/src/lib/libndpi.so*
/src/lib/protocols/.deps/
/src/lib/third_party/src/.deps/
/src/lib/third_party/src/fuzz/.deps/
/tags
/test-driver
/tests/Makefile
+278 -2
View File
@@ -1,10 +1,286 @@
# CHANGELOG
#### nDPI 4.8 (Oct 2023)
## Major Changes
- Reworked lists implementation that decreased memory usage of orders of magnitude
- Improved code robustness via extensive code fuzzing
- Various improvements to overall library performance
- Extended IPv6 support
## New Supported Protocols and Services
- Add "Heroes of the Storm" video game signature detection. (#1949)
- Add Apache Thrift protocol dissector. (#2007)
- Add Remote Management Control Protocol (RMCP).
- Add Service Location Protocol dissector. (#2036)
- Add VK detection (#1880)
- Add Yandex services detection (#1882)
- Add a new protocol id for generic Adult Content traffic (#1906)
- Add a new protocol id for generic advertisement/analytics/tracking stuff (#1904)
- Add bitcoing protocol dissector. (#1992)
- Add detection of Roblox games (#2054)
- Add support for (un-encrypted) HTTP/2 (#2087)
- Add support for Epic Games and GeForceNow/Nvidia (#1990)
- Add support for SRTP (#1977)
- Added BACnet dissector. (#1940)
- Added HAProxy protocol. (#2088)
- Added OICQ dissector. (#1950)
- Added OperaVPN detection
- ProtonVPN: add basic detection (#2006)
- Added detection of Facebook Reels and Stories
- Add an heuristic to detect fully encrypted flows (#2058)
- Added NDPI_MALWARE_HOST_CONTACTED flow risk
- Added NDPI_TLS_ALPN_SNI_MISMATCH flow risk
## Improvements
- Improve protocol detection for:
- FreeBSD compilation fix (C) update
- Gnutella: improve detection (#2019)
- H323: fix false positives (#1916)
- HTTP: fix another memory access error (#2049)
- HTTP: fix extraction of filename (#2046)
- HTTP: fix heap-buffer-overflow (#2044)
- HTTP: improve extraction of metadata and of flow risks (#1959)
- HTTP: remove useless code about XBOX (#1958)
- HTTP: rework state machine (#1966)
- Hangout: detect Hangout/Duo/GoogleMeet/... in the STUN code (#2025)
- Enhance DNS risk for long hostnames (> 32)
- Enhanced MS teams STUN/Azure detection
- Enhanced custom port definition and improved error reporting in case of duplications
- Improve detection of Alibaba flows (#1991)
- Improve detection of crawler/bot traffic (#1956)
- Improve detection of crawlers/bots (#1968)
- Improved MGCP detection by allowing '\r' as line feed.
- Improved MS Teams detection with heuristic
- Improved Steam detection by adding steamdiscover pattern. (#2105)
- Improved Wireguard detection
- Improved checks for duplicated entries in protocols file
- Improved classification further reducing memory used
- Improved detection of invalid chars in DNS names
- Improved domain search tet unit
- Improved helper scripts. (#1986)
- MS Teams enhancement
- MySql: improve detection (#1928)
- zabbix: improve detection (#2055)
## Tools
- ndpiReader: allow to configure LRU caches TTL and size (#2004)
- ndpiReader: fix VXLAN de-tunneling (#1913)
- ndpiReader: fix export of DNS/BitTorrent attributes (#1985)
- ndpiReader: fix export of HTTP attributes (#1982)
- ndpiReader: fix flow stats (#1943)
- ndpiReader: fix print of flow payload (#1960)
- ndpiReader: improve printing of payload statistics (#1989)
- ndpiReader: print how many packets (per flow) were needed to perform full DPI (#1891)
- ndpireader: fix detection of DoH traffic based on packet distributions (#2045)
## Misc
- ARM compilation fix
- Add `ndpi_domain_classify_finalize()` function (#2084)
- Add a configuration knob to enable/disable loading of gambling list (#2047)
- Add a new flow risk about literal IP addresses used as SNI (#1892)
- Add an heuristic to detect/ignore some anomalous TCP ACK packets (#1948)
- Add another example of custom rules (#1923)
- Add support for multiline json
- Add support for roaring_bitmap_xor_inplace (#1983)
- Add support for vxlan decapsulation (#1441) (#1900)
- Added Source Engine dissector. (#1937)
- Added `lists/gambling.list` to extra dist.
- Added `slackb.com` SNI. (#2067)
- Added ability to define an unlimited number of custom rules IP:port for the same IP (it used tobe limited to 2)
- Added check to avoid skype heuristic false positives
- Added comment
- Added coverage targets to `Makefile.am` for convenience. (#2039)
- Added fix for better handling exceptions rollback in case of later match
- Added hyperlink
- Added ndpi_binary_bitmap data structure
- Added ndpi_bitmap64 support
- Added ndpi_bitmap_andnot API call
- Added ndpi_bitmap_copy() API call
- Added ndpi_bitmap_is_empty() and ndpi_bitmap_optimize() API calls
- Added ndpi_domain_classify_XXX(0 API
- Added ndpi_filter_add_multi() API call
- Added ndpi_murmur_hash to the nDPI API
- Added new API calls for implementing Bloom-filter like data structures
- Added printf/fprintf replacement for some internal modules. (#1974)
- Added scripts to auto generate hostname/SNI *.inc files. (#1984)
- Added sub-domain classification fix
- Added the ability to define custom protocols with arbitrary Ids in proto.txt
- Added vlan_id in ndpi_flow2json() prototype
- Adds new pcap for testing "funny" HTTP servers
- All protocols should be excluded sooner or later (#1969)
- Allow init of app protocols w/o any hostnames set. (#2057)
- Avoid calling `ndpi_reconcile_protocols()` twice in `ndpi_detection_giveup()` (#1996)
- Boundary check
- CI: fix `Performance` job (#1936)
- Centos7 fixes
- Changed logging callback function sig. (#2000)
- Changes for supporting more efficient sub-string matching
- Classification fixes
- DNS: extract geolocation information, if available (#2065)
- Debian 12 fixes
- Disabled query string validation in MDNS in order to avoid zapping chars that in DNS (instead) are not permitted
- DisneyPlus/Hulu ip lists should be auto-generated (#1905)
- Extend content list of Microsoft protocols (#1930)
- Extend content-match list (#1967)
- Fix LRU/Patricia/Automa stats in `ndpiReader` with multiple threads (#1934)
- Fix MS Teams detection with heuristic (#1972)
- Fix access to packet/flow information (#2013)
- Fix an heap-buffer-overflow (#1994)
- Fix classification-by-ip in `ndpi_detection_giveup` (#1981)
- Fix compilation (#2011)
- Fix compilation in CI jobs (#2048)
- Fix compilation on Windows (#2072)
- Fix compilation with GCC-7 and latest RoaringBitmap code (#1886)
- Fix detection of packet direction and NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1883)
- Fix export/serialization of `flow->risk` (#1885)
- Fix for buffer overflow in serialization
- Fix insert of ip addresses into patricia tree(s) (#1895)
- Fix missing u_char, u_short and u_int typedefs for some platforms e.g.: (#2009)
- Fix packet counters (#1884)
- Fix some errors found by fuzzers (#2078)
- Fix some memory errors triggered by allocation failures (#1995)
- Fix some prototypes (#2085)
- Fix string truncation. (#2056)
- Fixed OpenWRT arm related build issues. (#2104)
- Fixed heap-buffer-overflow issue
- Fixed heap-overflow if compiled with `--enable-tls-sigs`. (#2038)
- Fixed invalid use of ndpi_free(). Sorry, my fault. (#1988)
- Fixed missing AS_HELP_STRING in configure.ac. (#1893)
- Fixed two OpenWRT arm related build issues. (#2103)
- Fixes matches with domain name strings that start with a dot
- Fixes risk mask exception handling while improving the overall performance
- Implemented Count-Min Sketch [count how many times a value has been observed]
- Implemented Zoom/Teams stream type detection
- Implemented ndpi_XXX_reset() API calls whre XXX is ses, des, hw
- Implemented ndpi_predict_linear() for predicting a timeseries value overtime
- Improved debug output. (#1951)
- Improved invalid logging via printf().
- Improved line protocol dissection with heuristic
- Improved missing usage of nDPIs malloc wrapper. Fixes #1978. (#1979)
- Improved protocol detection exploiting IP-based guess Reworked ndpi_reconcile_protocols() that is now called only in front of a match (less overhead)
- Improvement for reducing false positives
- Included Gambling website data from the Polish `hazard.mf.gov.pl` list (#2041)
- Keep master protocol in `ndpi_reconcile_protocols`
- Leak fix
- Language fix
- Line: fix heap-buffer-overflow error (#2015)
- Made VK protocol detection more strict
- Make Bittorrent LRU cache IPv6 aware. (#1909)
- Merged new and old version of ndpi_domain_classify.c code
- Mullvad VPN service added (based on entry node IP addresses) (#2062)
- Numeric truncation at `ndpi_analyze.c` at lines 101, 104, 107, 110 (#1999)
- Numeric truncation at `tls.c:1010` (#2005)
- Ookla: rework detection (#1922)
- Optimizes and fixes possible out0of0boundary write in ndpi_fill_prefix_v4()
- ProtonVPN: split the ip list (#2060)
- QUIC: add support for QUIC version 2
- QUIC: export QUIC version as metadata
- QUIC: fix a memory access error
- QUIC: fix dissection of packets forcing VN
- RDP: improve detection over UDP (#2043)
- RTP: remove dead-code (#1953)
- RTP: rework code (#2021)
- Refreshed ASN lists Enhanced the Line IP list with https://ipinfo.io/AS23576/125.209.252.0/24 used by line
- Remove some useless checks (#1993)
- Remove special handling of some TCP flows without SYN (#1965)
- Removed overlapping port
- Renamed HTTP/2 to HTTP2 as the '/' can have side effects with applications sitting on top of nDPI
- Replaces free() with ndpi_free()
- Rework CI jobs to try reducing CI duration (#1903)
- Reworked domain classification based on binary filters
- Reworked initialization
- Reworked ndpi_filter_xxx implementation using compressed bitmaps
- Reworked teams handling
- RiotGames: add detection of flows (#1935)
- STUN: add dissection of DTLS handshake (#2018)
- STUN: avoid FacebookVoip false positives (#2029)
- STUN: fix Skype/MsTeams detection and monitoring logic (#2028)
- STUN: fix detection of Google Voip apps (#2031)
- STUN: fix detection over TCP
- STUN: improve WhatsappCall detection
- STUN: keep monitoring/processing STUN flows (#2012)
- STUN: tell RTP from RTCP while in monitoring state (#2027)
- Serialization fix
- Set _DEFAULT_SOURCE and _GNU_SOURCE globally. (#2010)
- Simplify `ndpi_internal_guess_undetected_protocol()` (#1941)
- Simplify the report of streaming multimedia info (#2026)
- SoftEther: fix invalid memory access
- Swap from Aho-Corasick to an experimental/home-grown algorithm that uses a probabilistic approach for handling Internet domain names.
- Sync unit tests results
- Sync unit tests results
- Sync unit tests results (#1962)
- Sync utests results (#1887)
- TLS: add basic, basic, detection of Encrypted ClientHello (#2053)
- TLS: fix another interger overflow in certificate processing (#1915)
- TLS: fix parsing of certificate elements (#1910)
- Test files for riit games
- Test multiple `ndpiReader` configurations (#1931)
- Thrift: fix heap-buffer-overflow (#2024)
- Update GitHub runners versions (#1889)
- Update every ip lists (#2079)
- Update libinjection code (#1918)
- Update protocols documentation (#2081)
- Update roaring bitmap code
- Updated line test result
- Updated pcap detection results after Facebook Reel/Stories support
- Updated results
- Updated results after the latest changes
- Win include change
- Windows code rework
- Windows compilation fixes
- Windows warning checks
- add 2 ns from fdn.fr to DoH section (#1964)
- add support for gre decapsulation (#1442) (#1921)
- added bimap and/or with allocation
- added feature to extract filename from http attachment (#2037)
- added new domain names (#2002)
- configure: add an option to enable debug build, i.e `-g` (#1929)
- fix Stack overflow caused by invalid write in ndpi_automa_match_strin… (#2035)
- fixed numeric truncation error
- fixed numeric truncation error in diameter.c (#2034)
- fixed numeric truncation error in kerberos.c (#2032)
- fixed numeric truncation error in ndpi_main.c:6837 (#1998)
- fixed numeric truncation error in rtcp.c (#2033)
- fuzz: add a new fuzzer to test TLS certificates (#1901)
- fuzz: add a new fuzzer triggering the payload analyzer function(s) (#1926)
- fuzz: add fuzzer for DGA detection code (#2042)
- fuzz: add fuzzer to test internal gcrypt code (#1920)
- fuzz: add fuzzers to test bitmap64 and domain_classify data structures (#2082)
- fuzz: add fuzzers to test reader_util code (#2080)
- fuzz: extend coverage (#2073)
- fuzz: extend fuzz coverage (#1888)
- fuzz: extend fuzzers coverage (#1952)
- fuzz: extend fuzzing coverage (#2040)
- fuzz: extend fuzzing coverage (#2052)
- fuzz: extend fuzzing coverage (#2083)
- fuzz: simplify fuzzers dependencies in CIFuzz (#1896)
- fuzz: some improvements and add two new fuzzers (#1881)
- fuzzing: extend fuzzing coverage
- in case of failure, failing result files are not listed
- minor fixes (#2023)
- oss-fuzz: sync build script with upstream
- remove redefinition to vxlanhdr struct in vxlan dissector (#1911)
- removed useless call of ndpi_set_risk func (#2022)
- tests: add an option to force the overwrite of the unit tests results (#2001)
- tests: restore some old paths as symbolic links (#2050)
- tftp: check for Option Acknowledgements
- tftp: check incrementation for DATA and ACK packets
- tftp: rework request checking to account for options
- tftp: update pcap results
- version of dirent.c that is liked by both VC++ and MinGW
#### nDPI 4.6 (Feb 2023)
## New Features
- New support for custom BPF protocol definition using nBPF (see example/protos.txt)
- Improved dissection performance
- Added fuzzing all over
## New Supported Protocols and Services
@@ -18,7 +294,7 @@
- Elasticsearch
- FastCGI
- Kismet
- Line App and Line Voip valls
- Line App and Line VoIP calls
- Meraki Cloud
- Munin
- NATPMP
@@ -108,7 +384,7 @@
- Fix stop of extra dissection in HTTP, Bittorrent, Kerberos
- Fix signed integer overflow in ASN1/BER dissector
- Fix char/uchar bug in ahocorasick
- Fix endianess in IP-Port lookup
- Fix endianness in IP-Port lookup
- Fix FastCGI memory allocation issue
- Fix metadata extraction in NAT-PMP
- Fix invalid unidirectional traffic alert for unidirectional protocols (e.g. sFlow)
+6
View File
@@ -24,4 +24,10 @@ Please keep all code, comments, issues and pull requests in English, otherwise t
- If there is an open issue for the bug, ensure the description of the pull request contains `Fixes #issue_number`.
- Ensure the pull request passes CI testing. Any pull request which fails CI will be **ignored**.
## Before you submit your code
As we want to make sure that we can incorporate your changes in nDPI, we ask all contributors to read and sign the
[Contributor License Agreement](https://github.com/ntop/legal/blob/main/individual-contributor-licence-agreement.md)
that guarantees that your code is original and royalty-free to keep nDPI free and available for everybody,
Finally, thank you for your contributions.
+20 -4
View File
@@ -12,21 +12,37 @@ EXTRA_DIST = README.md README.fuzzer.md CHANGELOG.md CONTRIBUTING.md \
autogen.sh wireshark windows utils packages \
influxdb/Makefile.in influxdb/metric_anomaly.c influxdb/README.txt \
rrdtool/Makefile.in rrdtool/README.txt rrdtool/rrd_anomaly.c rrdtool/rrd_similarity.c \
doc/requirements.txt doc/conf.py doc/flow_risks.rst doc/guide/nDPI_QuickStartGuide.pages \
doc/requirements.txt doc/conf.py doc/flow_risks.rst doc/protocols.rst doc/guide/nDPI_QuickStartGuide.pages \
doc/guide/nDPI_QuickStartGuide.pdf doc/img/logo.png doc/index.rst \
doc/Makefile doc/what_is_ndpi.rst \
doc/Makefile doc/what_is_ndpi.rst doc/FAQ.md doc/configuration_parameters.md \
doc/library_initialization.md \
python/DEV_GUIDE.md python/dev_requirements.txt python/ndpi_example.py python/ndpi/__init__.py \
python/ndpi/ndpi_build.py python/ndpi/ndpi.py python/README.md \
python/requirements.txt python/setup.py python/tests.py \
lists/100_malware.list \
lists/107_gambling.list \
lists/107_gambling_custom.list \
lists/public_suffix_list.dat \
lists/README.md \
sonar-project.properties .github .ci-ignore
.PHONY: doc doc-view
doc:
make -C doc html
doc-view:
python3 -m http.server --bind 127.0.0.1 --directory doc/_build/html 8080
coverage: all
mkdir -vp coverage_report/ndpi_coverage_report
lcov --directory . --capture --output-file lcov.info
genhtml -o coverage_report/ndpi_coverage_report lcov.info
clean-coverage:
rm -f lcov.info
find . -type f -name "*.gcda" -exec rm "{}" +
find . -type f -name "*.gcno" -exec rm "{}" +
changelog:
git log --since={`curl -s https://github.com/ntop/ndpi/releases | grep datetime | head -n1 | egrep -o "[0-9]+\-[0-9]+\-[0-9]+"`} --name-only --pretty=format:" - %s" | grep "^ " > CHANGELOG.latest
.PHONY: doc doc-view coverage clean-coverage changelog
+7 -4
View File
@@ -1,4 +1,3 @@
![ntop][ntopng_logo] ![ntop][ntop_logo]
# nDPI
[![Build Status](https://img.shields.io/github/actions/workflow/status/ntop/nDPI/build.yml?branch=dev&logo=github)](https://github.com/ntop/nDPI/actions?query=workflow%3ABuild)
@@ -8,6 +7,8 @@
nDPI® is an open source LGPLv3 library for deep-packet inspection. Based on OpenDPI it includes ntop extensions. We have tried to push them into the OpenDPI source tree but nobody answered emails so we have decided to create our own source tree
A generic FAQ about nDPI® is available [here](https://github.com/ntop/nDPI/blob/dev/doc/FAQ.md)
### How To Compile nDPI
In order to compile this project do
@@ -51,7 +52,7 @@ On Windows:
There are three supported ways to build nDPI:
1. MSYS2 (assuming [MSYS2](https://www.msys2.org/) already installed):
- msys2 -c "pacman --noconfirm -S --needed --overwrite '\*' git mingw-w64-x86\_64-toolchain automake1.16 automake-wrapper autoconf libtool make mingw-w64-x86\_64-json-c mingw-w64-x86\_64-crt-git mingw-w64-x86\_64-pcre mingw-w64-x86\_64-libpcap"
- msys2 -c "pacman --noconfirm -S --needed --overwrite '\*' git mingw-w64-x86\_64-toolchain automake1.16 automake-wrapper autoconf libtool make mingw-w64-x86\_64-json-c mingw-w64-x86\_64-crt-git mingw-w64-x86\_64-pcre2 mingw-w64-x86\_64-libpcap"
2. Mingw-w64
@@ -78,10 +79,12 @@ The entire procedure of adding new protocols in detail:
4. Add a new entry for the search function for the new protocol in: `src/include/ndpi_protocols.h`
5. Choose (do not change anything) a selection bitmask from: `src/include/ndpi_define.h`
6. Set protocol default ports in `ndpi_init_protocol_defaults` in: `src/lib/ndpi_main.c`
7. Be sure to have nBPF support, cloning `PF_RING` in the same directory where you cloned `nDPI`: `git clone https://github.com/ntop/PF_RING/ && cd PF_RING/userland/nbpf && ./configure && make`
8. From the `nDPI` root directory, `./autogen.sh --with-pcre` (nBPF and PCRE are usually optional, but they are needed to run/update *all* the unit tests)
7. Be sure to have nBPF support, cloning `PF_RING` in the same directory where you cloned `nDPI`: `git clone https://github.com/ntop/PF_RING/ && cd PF_RING/userland/nbpf && ./configure && make`. You can ignore the `/bin/sh: 1: ../lib/pfring_config: not found` error
8. From the `nDPI` root directory, `./autogen.sh --with-pcre2` (nBPF and PCRE2 are usually optional, but they are needed to run/update *all* the unit tests)
9. `make`
10. `make check`
11. Update the documentation, adding this new protocol to `doc/protocols.rst`
12. Update the Windows Visual Studio configuration, adding the new c file in `windows/nDPI.vcxproj`
### How to use nDPI to Block Selected Traffic
+89 -26
View File
@@ -1,4 +1,4 @@
AC_INIT([libndpi],[4.6.0])
AC_INIT([libndpi],[4.9.0])
AC_CONFIG_AUX_DIR([.])
AC_CONFIG_MACRO_DIR([m4])
@@ -8,7 +8,7 @@ AM_INIT_AUTOMAKE([foreign subdir-objects])
AC_PREFIX_DEFAULT(/usr)
EXTRA_TARGETS="example tests tests/dga"
AC_ARG_WITH(only-libndpi, AS_HELP_STRING([--with-only-libndpi], [Build only libndpi (no examples, tests etc)]))
AC_ARG_WITH(only-libndpi, AS_HELP_STRING([--with-only-libndpi], [Build only libndpi (no examples, tests etc)]))
AS_IF([test "${with_only_libndpi+set}" = set],[
EXTRA_TARGETS=""
])
@@ -16,17 +16,22 @@ AS_IF([test "${with_only_libndpi+set}" = set],[
PWD=`pwd`
AC_ARG_WITH(sanitizer, AS_HELP_STRING([--with-sanitizer], [Build with support for address, undefined and leak sanitizer]))
AC_ARG_WITH(sanitizer, AS_HELP_STRING([--with-sanitizer], [Build with support for address, undefined and leak sanitizer]))
AC_ARG_WITH(thread-sanitizer, AS_HELP_STRING([--with-thread-sanitizer], [Build with support for thread sanitizer]))
AC_ARG_WITH(memory-sanitizer, AS_HELP_STRING([--with-memory-sanitizer], [Build with support for memory sanitizer]))
AC_ARG_WITH(macos-memory-sanitizer, AS_HELP_STRING([--with-macos-memory-sanitizer], [Build with support for memory sanitizer macOS]))
AC_ARG_ENABLE(fuzztargets, AS_HELP_STRING([--enable-fuzztargets], [Enable fuzz targets]),[enable_fuzztargets=$enableval],[enable_fuzztargets=no])
AC_ARG_ENABLE(gprof, AS_HELP_STRING([--enable-gprof], [Enable CPU/HEAP profiling with gperftools]),[enable_gprof=$enableval],[enable_gprof=no])
AC_ARG_ENABLE(code-coverage, AS_HELP_STRING([--enable-code-coverage], [Generate Code Coverage report]))
AC_ARG_WITH(local-libgcrypt, AS_HELP_STRING([--with-local-libgcrypt], [Build with libgcrypt (if present) instead of the enclosed gcrypt light]))
AC_ARG_WITH(local-libgcrypt, AS_HELP_STRING([--with-local-libgcrypt], [Build with libgcrypt (if present) instead of the enclosed gcrypt light]))
AC_ARG_ENABLE(tls-sigs, AS_HELP_STRING([--enable-tls-sigs], [Enable TLS Client signature algorithm dissection. Rarely used, but requires significantly more memory.]))
AC_ARG_ENABLE(npcap, AS_HELP_STRING([--disable-npcap], [msys2 only: Disable linkage against the wpcap/npcap import library in windows/WpdPack/Lib.]))
AC_ARG_WITH(nbpf-path, AS_HELP_STRING([--with-nbpf-path], [nBPF library custom path; default: ${PWD}/../PF_RING/userland/nbpf]),[NBPF_HOME=$withval],[NBPF_HOME=${PWD}/../PF_RING/userland/nbpf])
AC_ARG_WITH(lto-and-gold-linker, AS_HELP_STRING([--with-lto-and-gold-linker], [Build with LTO and Gold linker])])
AC_ARG_WITH(lto-and-gold-linker, AS_HELP_STRING([--with-lto-and-gold-linker], [Build with LTO and Gold linker]))
AC_ARG_ENABLE(debug-build, AS_HELP_STRING([--enable-debug-build], [Enable debug build (`-g` flag)]),[enable_debugbuild=$enableval],[enable_debugbuild=no])
AC_ARG_ENABLE(global-context-support, AS_HELP_STRING([--disable-global-context-support], [Disable support for global context. No external dependency on libpthread]))
NDPI_CFLAGS="${NDPI_CFLAGS} -D_DEFAULT_SOURCE=1 -D_GNU_SOURCE=1"
AS_IF([test "x$enable_fuzztargets" = "xyes"], [
BUILD_FUZZTARGETS=1
@@ -34,6 +39,10 @@ AS_IF([test "x$enable_fuzztargets" = "xyes"], [
], [BUILD_FUZZTARGETS=0])
AM_CONDITIONAL([BUILD_FUZZTARGETS], [test "x$enable_fuzztargets" = "xyes"])
AS_IF([test "x$enable_debugbuild" = "xyes"], [
NDPI_CFLAGS="${NDPI_CFLAGS} -g"
])
AS_IF([test "${with_sanitizer+set}" = set -a "${with_thread_sanitizer+set}" = set],[
AC_MSG_ERROR([Configure options `--with-sanitizer' and `--with_thread_sanitizer' can not used at the same time.])
])
@@ -56,8 +65,14 @@ AS_IF([test "${with_sanitizer+set}" = set -o "${with_thread_sanitizer+set}" = se
])
AS_IF([test "${with_sanitizer+set}" = set],[
NDPI_CFLAGS="${NDPI_CFLAGS} -fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fsanitize=leak -fno-omit-frame-pointer"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fsanitize=leak"
NDPI_CFLAGS="${NDPI_CFLAGS} -fsanitize=address -fsanitize=undefined -fsanitize=leak -fno-omit-frame-pointer"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fsanitize=address -fsanitize=undefined -fsanitize=leak"
#Sanitizers should work on any compilers that we support (or that we test on CI, at least)
#Exception: "-fsanitize=alignment" is not supported in gcc 4.9
AX_CHECK_COMPILE_FLAG([-fno-sanitize=alignment], [
NDPI_CFLAGS="${NDPI_CFLAGS} -fno-sanitize=alignment"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fno-sanitize=alignment"
])
])
AS_IF([test "${with_thread_sanitizer+set}" = set],[
@@ -70,6 +85,11 @@ AS_IF([test "${with_memory_sanitizer+set}" = set],[
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fsanitize=memory"
])
AS_IF([test "${with_macos_memory_sanitizer+set}" = set],[
NDPI_CFLAGS="${NDPI_CFLAGS} -fsanitize=address -fno-omit-frame-pointer -fsanitize=signed-integer-overflow -fno-sanitize-recover=address"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fsanitize=address"
])
AS_IF([test "x${enable_code_coverage}" = "xyes"],[
NDPI_CFLAGS="${NDPI_CFLAGS} -fprofile-arcs -ftest-coverage"
NDPI_LDFLAGS="${NDPI_LDFLAGS} --coverage"
@@ -82,6 +102,8 @@ LT_LIB_M
PKG_PROG_PKG_CONFIG
AC_PROG_CC
AC_PROG_CPP_WERROR
AC_C_INLINE
SYSTEM=`uname -s`
if test $SYSTEM = "Darwin"; then
@@ -98,6 +120,29 @@ AC_CHECK_TOOL(AR, ar, [false])
AC_LANG_WERROR
AC_MSG_CHECKING([whether roaring v3 works])
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#if __has_include (<stdio.h>)
#endif
]])],
[
AC_MSG_RESULT([yes])
],
[
AC_MSG_RESULT([no])
AC_DEFINE_UNQUOTED(USE_ROARING_V2, "1", [Use CRoaring 2.1.x])
])
GCC_VERSION=`gcc --version | cut -d ' ' -f 3 | head -1|cut -d '.' -f 1`
if test "${GCC_VERSION}" == "version" ; then
GCC_VERSION=`gcc --version | cut -d ' ' -f 4 | head -1|cut -d '.' -f 1`
fi
if test "${GCC_VERSION}" -lt "7" ; then
AC_DEFINE_UNQUOTED(USE_ROARING_V2, "1", [Use CRoaring 2.1.x])
fi
NDPI_MAJOR=`echo "${PACKAGE_VERSION}" | cut -d . -f 1`
NDPI_MINOR=`echo "${PACKAGE_VERSION}" | cut -d . -f 2`
NDPI_PATCH=`echo "${PACKAGE_VERSION}" | cut -d . -f 3`
@@ -110,8 +155,8 @@ AC_DEFINE_UNQUOTED(NDPI_PATCH_LEVEL, "${NDPI_PATCH}", [nDPI patch level])
# .git as directory in a cloned repo
# .git as file in submodule based integration
if test -d ".git" || test -r ".git"; then
GIT_TAG=`git log -1 --format=%h`
GIT_DATE=`git log -1 --format=%cd`
GIT_TAG=`git log -1 --abbrev=7 --format=%h`
GIT_DATE=`git log -1 --abbrev=7 --format=%cd`
#
# On CentOS 6 `git rev-list HEAD --count` does not work
#
@@ -156,12 +201,19 @@ if ! test "${with_only_libndpi+set}" = set; then :
JSONC_LIBS=""
JSONC_CFLAGS=""
])
AX_PTHREAD([],[
AC_MSG_WARN([POSIX Threads not available. Building library only.])
JSONC_LIBS=""
JSONC_CFLAGS=""
EXTRA_TARGETS=""
])
dnl> AX_PTHREAD test does not work along with MingW/MSYS anymore
case "$host" in
*-*-mingw32*|*-*-msys)
;;
*)
AX_PTHREAD([],[
AC_MSG_WARN([POSIX Threads not available. Building library only.])
JSONC_LIBS=""
JSONC_CFLAGS=""
EXTRA_TARGETS=""
])
;;
esac
fi
AM_CONDITIONAL([BUILD_UNITTESTS], [test "x$build_unittests" = "xyes"])
@@ -170,7 +222,7 @@ AS_IF([test "x${enable_tls_sigs}" = "xyes"],[
HANDLE_TLS_SIGS=""
])
NDPI_CFLAGS="-W -Wall -Wno-unused-parameter -Wno-unused-function -Wno-address-of-packed-member ${NDPI_CFLAGS}"
NDPI_CFLAGS="-W -Wall -Wno-address-of-packed-member ${NDPI_CFLAGS}"
AS_IF([test "${with_lto_and_gold_linker+set}" = set], [
NDPI_CFLAGS="${NDPI_CFLAGS} -flto -fuse-ld=gold -Wno-unused-command-line-argument"
@@ -186,6 +238,13 @@ if test "x$ac_cv_lib_rrd_rrd_fetch_r" = xyes; then :
EXTRA_TARGETS="$EXTRA_TARGETS rrdtool"
fi
GLOBAL_CONTEXT_ENABLED=0
AS_IF([test "x$enable_global_context_support" != "xno"], [
AC_CHECK_LIB([pthread], [pthread_mutex_init])
NDPI_CFLAGS="${NDPI_CFLAGS} -DUSE_GLOBAL_CONTEXT"
GLOBAL_CONTEXT_ENABLED=1
])
ADDITIONAL_INCS=
ADDITIONAL_LIBS="$LIBM"
PCAP_HOME=$HOME/PF_RING/userland
@@ -342,19 +401,19 @@ AS_IF([test "${with_local_libgcrypt+set}" = set],[
AC_DEFINE_UNQUOTED(USE_HOST_LIBGCRYPT, 1, [Use locally installed libgcrypt instead of builtin gcrypt-light])
])
dnl> PCRE
PCRE_ENABLED=0
AC_ARG_WITH(pcre, [ --with-pcre Enable nDPI build with libpcre])
if test "${with_pcre+set}" = set; then :
AC_CHECK_LIB(pcre, pcre_compile, AC_DEFINE_UNQUOTED(HAVE_PCRE, 1, [libpcre(-dev) is present]))
if test "x$ac_cv_lib_pcre_pcre_compile" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lpcre"
PCRE_ENABLED=1
dnl> PCRE2
PCRE2_ENABLED=0
AC_ARG_WITH(pcre2, AS_HELP_STRING([--with-pcre2], [Enable nDPI build with libpcre2]))
if test "${with_pcre2+set}" = set; then :
AC_CHECK_LIB(pcre2-8, pcre2_compile_8, AC_DEFINE_UNQUOTED(HAVE_PCRE2, 1, [libpcre2(-dev) is present]))
if test "x$ac_cv_lib_pcre2_8_pcre2_compile_8" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lpcre2-8"
PCRE2_ENABLED=1
fi
fi
dnl> GeoIP
AC_ARG_WITH(maxminddb, [ --with-maxminddb Enable nDPI build with libmaxminddb])
AC_ARG_WITH(maxminddb, AS_HELP_STRING([--with-maxminddb], [Enable nDPI build with libmaxminddb]))
if test "${with_maxminddb+set}" = set; then :
AC_CHECK_LIB([maxminddb], [MMDB_lookup_sockaddr])
AC_CHECK_HEADERS([maxminddb.h])
@@ -364,6 +423,8 @@ if test "${with_maxminddb+set}" = set; then :
fi
fi
AC_DEFINE_UNQUOTED(NDPI_BASE_DIR, "${PWD}", [nDPI base directory])
dnl> Curl
dnl> AC_CHECK_LIB([curl], [curl_easy_perform])
dnl> if test ${ac_cv_lib_curl_curl_easy_perform} = "no"; then
@@ -401,11 +462,13 @@ AC_SUBST(GPROF_CFLAGS)
AC_SUBST(GPROF_LIBS)
AC_SUBST(GPROF_ENABLED)
AC_SUBST(USE_HOST_LIBGCRYPT)
AC_SUBST(PCRE_ENABLED)
AC_SUBST(PCRE2_ENABLED)
AC_SUBST(NBPF_ENABLED)
AC_SUBST(GLOBAL_CONTEXT_ENABLED)
AC_SUBST(HANDLE_TLS_SIGS)
AC_SUBST(DISABLE_NPCAP)
AC_SUBST(EXE_SUFFIX)
AC_SUBST(NDPI_CFLAGS)
AC_SUBST(NDPI_LDFLAGS)
AC_SUBST(NDPI_BASE_DIR)
AC_OUTPUT
+36
View File
@@ -0,0 +1,36 @@
# FAQ
From https://www.ntop.org/ndpi/ndpi-internals-and-frequent-questions/
**Q**: How nDPI implements protocol detection?\
**A**: nDPI includes a list of protocol dissectors (364 as of today) that are able to dissect protocols such as WhatsApp or TLS. As soon as a new flow is submitted to nDPI, the library applies in sequence dissectors that can potentially match the protocols (i.e. telnet is a TCP-based protocol and it will not be considered for UDP flows). We start from the dissector that can most probably match using the port number. This means for traffic on TCP/22 nDPI will start with the SSH dissectors and if not matching continue with the others. Dissection completes as soon as a protocol matches or when none of them matched and in this case the flow will be labelled as Unknown.
**Q**: What is the nDPI release cycle?\
**A**: We cut release a approximately every 6-8 months, fixes and improvements are on a daily basis (check the nDPI code on GitHub).
**Q**: Is nDPI running on all popular platforms?\
**A**: Yes it runs on Linux, macOS, Windows… and also on not-so-popular ones such as IBM mainframes. We support ARM, Intel, RISC… architectures.
**Q**: How many packets does nDPI need in order to implement detection?\
**A**: It depends on the protocol. For UDP-based protocols such as DNS one packet is enough, for more complex protocols such as TLS about 10 packets. For sure if after 15-20 packets nDPI has not detected the application protocol, then the protocol is labelled as Unknown.
**Q**: Is nDPI detection only based on protocol dissectors?\
**A**: No, payload inspection is the main technique, but nDPI can also use IP address, ports, TLS certificates etc as signatures for protocols. In this case, after detection is complete, nDPI will report if the match was performed on payload inspection or other means (e.g. IP address).
**Q**: Does nDPI contain list of known IP addresses?\
**A**: Yes it includes lists of well known IPs such as those provided by Microsoft of Meta for identifying known service.
**Q**: Can I extend nDPI by defining new protocols with a configuration file?\
**A**: Yes you can. See this [file](https://github.com/ntop/nDPI/blob/dev/example/protos.txt) as an example for defining new protocols..
**Q**: Is nDPI able to detect VPNs?\
**A**: Yes it can detect VPNS such as Tailscale, WireGuard, OpenVPN, FortiClient.. and also in-app VPNs such as UltraSurf or OperaVPN.
**Q**: Is nDPI able to detect malware and viruses?\
**A**: It can detect anomalous behaviour that can be caused by a malware, but nDPI is not a signature-based tool so it does not include signatures for malware A or B. This is because [signature-based tools](https://en.wikipedia.org/wiki/Intrusion_detection_system) have various limitations and resource intensive, whereas nDPI has been designed to be used also in high-speed (100 Gbit+) networks.
**Q**: Is nDPI able to detect security issues?\
**A**: Yes it can by means of a technique called [flow risk](https://github.com/ntop/nDPI/blob/dev/doc/flow_risks.rst). It can identify 50+ threats (e.g. a host that is talking with a malware host).
**Q**: Is nDPI able to block traffic?\
**A**: No, nDPI is a passive traffic analysis library that does not manipulate packets. You can create applications on top of it for policing (i.e. blocking or shaping) traffic. Examples of such applications are ntopng Edge, nProbe IPS and nProbe Cento.
+49
View File
@@ -0,0 +1,49 @@
# Configuration knobs
TODO
| Protocol | Parameter | Default value | Min value | Max value | Description | Notes |
| ------ | ------ | ------ | ------ | ------ | ------ | ------ |
| NULL | "packets_limit_per_flow" | 32 | 0 | 255 | The upper limit on the number of packets per flow that will be subject to DPI, after which classification will be considered complete (0 = no limit) |
| NULL | "flow.direction_detection" | enable | NULL | NULL | Enable/disable internal detection of packet direction (client to server or server to client) |
| NULL | "flow.track_payload" | disable | NULL | NULL | Enable/disable tracking/export of flow payload (i.e. L5/7 data): if enabled, the library exports the first 1024 bytes of payload for each flow |
| NULL | "tcp_ack_payload_heuristic" | disable | NULL | NULL | In some networks, there are some anomalous TCP flows where the smallest ACK packets have some kind of zero padding. It looks like the IP and TCP headers in those frames wrongly consider the 0x00 Ethernet padding bytes as part of the TCP payload. While this kind of packets is perfectly valid per-se, in some conditions they might be treated by the TCP reassembler logic as (partial) overlaps, deceiving the classification engine. This parameter enable/disable an heuristic to detect these packets and to ignore them, allowing correct detection/classification. See #1946 for other details |
| NULL | "fully_encrypted_heuristic" | enable | NULL | NULL | Enable/disable an heuristic to detect fully encrypted sessions, i.e. flows where every bytes of the payload is encrypted in an attempt to “look like nothing”. This heuristic only analyzes the first packet of the flow. See: https://www.usenix.org/system/files/sec23fall-prepub-234-wu-mingshi.pdf |
| NULL | "libgcrypt.init" | 1 | NULL | NULL | Enable/disable initialization of libgcrypt. When using the external libgcrypt (instead of the internal crypto code) the libgcrypt runtime must be initialized. If, for whatever reasons, the application alread does it, nDPI must be told to skip it. Note that, by default, nDPI uses the crypto code and not libgcrypt: in that case this parameter is ignored |
| NULL | "dpi.compute_entropy" | 1 | NULL | NULL | Enable/disable computation of flow entropy |
| NULL | "dpi.guess_on_giveup" | 0x03 | 0x00 | 0x03 | Tell the library to guess flow classification, if any DPI algorithms/logics fail. The value is a bitmask. Values: 0x0 = disabled; 0x01 = enable guessing by port; 0x02 = enable guessing by ip |
| NULL | "flow_risk_lists.load" | 1 | NULL | NULL | Enable/disable loading of every IP addresses lists used to check any flow risks |
| NULL | "flow_risk.anonymous_subscriber.list.icloudprivaterelay.load" | 1 | NULL | NULL | Enable/disable loading of internal iCouldPrivateRealy IP address list used to check `NDPI_ANONYMOUS_SUBSCRIBER` flow risk |
| NULL | "flow_risk.anonymous_subscriber.list.protonvpn.load" | 1 | NULL | NULL | Enable/disable loading of internal IP address list of ProtonVPN exit nodes used to check `NDPI_ANONYMOUS_SUBSCRIBER` flow risk |
| NULL | "flow_risk.crawler_bot.list.load" | 1 | NULL | NULL | Enable/disable loading of internal IP address list used to check `NDPI_HTTP_CRAWLER_BOT` flow risk |
| NULL | "filename.config" | NULL | NULL | NULL | Name of the file containing a list of configuration knobs itself (one per line)!. Useful to configure nDPI via text file instead of via API |
| NULL | "log.level" | 0 | 0 | 3 | Configure the log/debug level. Possible values: 0 = error, 1 = trace, 2 = debug, 3 = extra debug |
| NULL | "lru.$CACHE_NAME.size" | See description | 0 | 16777215 | Set the size (in number of elements) of the specified LRU cache (0 = the cache is disabled). The keyword "$CACHE_NAME" is a placeholder for the cache name and the possible values are: ookla, bittorrent, stun, tls_cert, mining, msteams. The default value is "32768" for the bittorrent cache and "1024" for all the other caches |
| NULL | "lru.$CACHE_NAME.ttl" | See description | 0 | 16777215 | Set the TTL (in seconds) for the elements of the specified LRU cache (0 = the elements never explicitly expire). The keyword "$CACHE_NAME" is a placeholder for the cache name and the possible values are: ookla, bittorrent, stun, tls_cert, mining, msteams. The default value is "120" for the ookla cache, "60" for the msteams cache and "0" for all the other caches |
| NULL | "lru.$CACHE_NAME.scope" | 0 | 0 | 1 | Set the scope of the specified LRU cache (0 = the cache is local, 1 = the cache is global). The keyword "$CACHE_NAME" is a placeholder for the cache name and the possible values are: ookla, bittorrent, stun, tls_cert, mining, msteams. The global scope con be set only if a global context has been initialized |
| "tls" | "certificate_expiration_threshold" | 30 | 0 | 365 | The threshold (in days) used to trigger the `NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE` flow risk |
| "tls" | "application_blocks_tracking" | disable | NULL | NULL | Enable/disable processing of TLS Application Blocks (post handshake) to extract statistical information about the flow |
| "tls" | "metadata.sha1_fingerprint" | enable | NULL | NULL | Enable/disable computation and export of SHA1 fingerprint for TLS flows. Note that if it is disable, the flow risk `NDPI_MALICIOUS_SHA1_CERTIFICATE` is not checked |
| "tls" | "metadata.ja3c_fingerprint" | enable | NULL | NULL | Enable/disable computation and export of JA3C fingerprint for TLS flows. Note that if it is disable, the flow risk `NDPI_MALICIOUS_JA3` is not checked |
| "tls" | "metadata.ja3s_fingerprint" | enable | NULL | NULL | Enable/disable computation and export of JA3S fingerprint for TLS flows |
| "tls" | "metadata.ja4c_fingerprint" | enable | NULL | NULL | Enable/disable computation and export of JA4C fingerprint for TLS flows |
| "smtp" | "tls_dissection" | enable | NULL | NULL | Enable/disable dissection of TLS packets in cleartext SMTP flows (because of opportunistic TLS, via STARTTLS msg) |
| "imap" | "tls_dissection" | enable | NULL | NULL | Enable/disable dissection of TLS packets in cleartext IMAP flows (because of opportunistic TLS, via STARTTLS msg) |
| "pop" | "tls_dissection" | enable | NULL | NULL | Enable/disable dissection of TLS packets in cleartext POP flows (because of opportunistic TLS, via STARTTLS msg) |
| "ftp" | "tls_dissection" | enable | NULL | NULL | Enable/disable dissection of TLS packets in cleartext FTP flows (because of opportunistic TLS, via AUTH TLS msg) |
| "stun" | "max_packets_extra_dissection" | 4 | 0 | 255 | After a flow has been classified has STUN, nDPI might analyse more packets to look for a sub-classification or for metadata. This parameter set the upper limit on the number of these packets |
| "stun" | "tls_dissection" | enable | NULL | NULL | Enable/disable dissection of TLS packets multiplexed into STUN flows |
| "stun" | "metadata.attribute.mapped_address" | enable | NULL | NULL | Enable/disable extraction of (xor)-mapped-address attribute for STUN flows. If it is disabled, STUN classification might be significant faster |
| "stun" | "metadata.attribute.response_origin" | enable | NULL | NULL | Enable/disable extraction of response-origin attribute for STUN flows. If it is disabled, STUN classification might be significant faster |
| "stun" | "metadata.attribute.other_address" | enable | NULL | NULL | Enable/disable extraction of other-address attribute for STUN flows. If it is disabled, STUN classification might be significant faster |
| "stun" | "metadata.attribute.relayed_address" | enable | NULL | NULL | Enable/disable extraction of (xor-)relayed-address attribute for STUN flows. If it is disabled, STUN classification might be significant faster |
| "stun" | "metadata.attribute.peer_address" | enable | NULL | NULL | Enable/disable extraction of (xor-)peer-address attribute for STUN flows. If it is disabled, STUN classification might be significant faster; however sub-classification capability might be negatively impacted |
| "dns" | "subclassification" | enable | NULL | NULL | Enable/disable sub-classification of DNS flows (via query/response domain name). If disabled, some flow risks are not checked |
| "dns" | "process_response" | enable | NULL | NULL | Enable/disable processing of DNS responses. By default, DNS flows are fully classified after the first request/response pair (or after the first response, if the request is missing). If this parameter is disabled, the flows are fully classified after the first packet, i.e. usually after the first request; in that case, some flow risks are not checked and some metadata are not exported |
| "http" | "process_response" | enable | NULL | NULL | Enable/disable processing of HTTP responses. By default, HTTP flows are usually fully classified after the first request/response pair. If this parameter is disabled, the flows are fully classified after the first request (or after the first response, if the request is missing); in that case, some flow risks are not checked and some metadata are not exported |
| "ookla" | "dpi.aggressiveness", | 0x01 | 0x00 | 0x01 | Detection aggressiveness for Ookla. The value is a bitmask. Values: 0x0 = disabled; 0x01 = enable heuristic for detection over TLS (via Ookla LRU cache) |
| "zoom" | "max_packets_extra_dissection" | 4 | 0 | 255 | After a flow has been classified has Zoom, nDPI might analyse more packets to look for a sub-classification or for metadata. This parameter set the upper limit on the number of these packets |
| "rtp" | "search_for_stun" | disable | NULL | NULL | After a flow has been classified as RTP or RTCP, nDPI might analyse more packets to look for STUN/DTLS packets, i.e. to try to tell if this flow is a "pure" RTP/RTCP flow or if the RTP/RTCP packets are multiplexed with STUN/DTLS. Useful for proper (sub)classification when the beginning of the flows are not captured or if there are lost packets in the the captured traffic. If enabled, nDPI requires more packets to process for each RTP/RTCP flow. |
| $PROTO_NAME | "log" | disable | NULL | NULL | Enable/disable logging/debug for specific protocol. Use "any" as protocol name if you want to easily enable/disable logging/debug for all protocols |
| $PROTO_NAME | "ip_list.load" | 1 | NULL | NULL | Enable/disable loading of internal list of IP addresses (used for (sub)classification) specific to that protocol. Use "any" as protocol name if you want to easily enable/disable all lists. This knob is valid only for the following protocols: Alibaba, Amazon AWS, Apple, Avast, Bloomberg, Cachefly, Cloudflare, Discord, Disney+, Dropbox, Edgecast, EpicGames, Ethereum, Facebook, Github, Google, Google Cloud, GoTo, Hotspot Shield, Hulu, Line, Microsoft 365, Microsoft Azure, Microsoft One Drive, Microsoft Outlook, Mullvad, Netflix, Nvidia, OpenDNS, ProtonVPN, RiotGames, Roblox, Skype/Teams, Starcraft, Steam, Teamviewer, Telegram, Tencent, Threema, TOR, Twitch, Twitter, UbuntuONE, VK, Yandex, Yandex Cloud, Webex, Whatsapp, Zoom |
+38 -7
View File
@@ -73,9 +73,9 @@ HTTP only: this risk is triggered whenever the user agent contains suspicious ch
.. _Risk 012:
NDPI_HTTP_NUMERIC_IP_HOST
NDPI_NUMERIC_IP_HOST
=========================
HTTP only: this risk is triggered whenever we're accessing a host using its IP rather than its symbolic name. Example http://1.2.3.4.
This risk is triggered whenever a HTTP/TLS/QUIC connection is using a literal IPv4 or IPv6 address as ServerName (TLS/QUIC; example: SNI=1.2.3.4) or as Hostname (HTTP; example: http://1.2.3.4.).
.. _Risk 013:
@@ -105,7 +105,7 @@ A `DGA <https://en.wikipedia.org/wiki/Domain_generation_algorithm>`_ is used to
NDPI_MALFORMED_PACKET
=====================
This risk is generated when a packet (e.g. a DNS packet) has an unexpected formt. This can indicate a protocol error or more often an attempt to jeopardize a valid protocol to carry other type of data.
This risk is generated when a packet (e.g. a DNS packet) has an unexpected format. This can indicate a protocol error or more often an attempt to jeopardize a valid protocol to carry other type of data.
.. _Risk 018:
@@ -141,7 +141,8 @@ This risk indicates that the protocol used is insecure and that a secure protoco
NDPI_DNS_SUSPICIOUS_TRAFFIC
===========================
This risk is returned when DNS traffic returns an unexpected/obsolete `record type <https://en.wikipedia.org/wiki/List_of_DNS_record_types>`_.
This risk is returned when DNS traffic returns an unexpected/obsolete `record type <https://en.wikipedia.org/wiki/List_of_DNS_record_types>`_
or when a suspicious query with a very long host name is detected.
.. _Risk 024:
@@ -219,7 +220,7 @@ This risk is used to detect suspicious data carried in ICMP packets whose entrop
NDPI_CLEAR_TEXT_CREDENTIALS
===========================
Clear text protocols are not bad per-se, but they should be avoided when they carry credentials as they can be intercepted by malicious users. This risk is triggered whenever clear text protocols (e.g. FTP, HTTP, IMAP...) contain credentials in clear text (read it as nDPI does not trigger this risk for HTTP connections that do not carry credentials).
Clear text protocols are not intrinsically bad, but they should be avoided when they carry credentials as they can be intercepted by malicious users. This risk is triggered whenever clear text protocols (e.g. FTP, HTTP, IMAP...) contain credentials in clear text (read it as nDPI does not trigger this risk for HTTP connections that do not carry credentials).
.. _Risk 037:
@@ -246,7 +247,7 @@ Additionally, some TLS protocol fields are checked for printable characters as w
NDPI_POSSIBLE_EXPLOIT
=====================
The risk is set whenever a possible exploit (e.g. `Log4J/Log4Shell <https://en.wikipedia.org/wiki/Log4Shell>`_) is detected.
The risk is set whenever a possible exploit attempt (e.g. `Log4J/Log4Shell <https://en.wikipedia.org/wiki/Log4Shell>`_) is detected.
.. _Risk 041:
@@ -276,7 +277,7 @@ The risk is set whenever a crawler/bot/robot has been detected
NDPI_ANONYMOUS_SUBSCRIBER
===================================
The risk is set whenever the (source) ip address has been anonymized and it can't be used to identify the subscriber.
The risk is set whenever the (source) IP address has been anonymized and it can't be used to identify the subscriber.
Example: the flow is generated by an iCloud-private-relay exit node.
.. _Risk 046:
@@ -309,3 +310,33 @@ Minor packet/flow issues (e.g. DNS traffic with zero TTL) have been detected.
NDPI_TCP_ISSUES
===============
Relevant TCP connection issues such as connection refused, scan, or probe attempt.
.. _Risk 051
NDPI_FULLY_ENCRYPTED
====================
Flow with Unknown protocol containing encrypted traffic.
.. _Risk 052
NDPI_TLS_ALPN_SNI_MISMATCH
=========================
Invalid TLS ALPN/SNI mismatch. For instance ALPN advertises the flow as h2 (HTTP/2.0) and no SNI is reported.
.. _Risk 053
NDPI_MALWARE_CONTACTED
======================
Client contacted a server host labelled as malware.
.. _Risk 054:
NDPI_BINARY_DATA_TRANSFER
============================
HTTP only: this risk indicates that a binary file/data application transfer (attempt).
.. _Risk 055:
NDPI_PROBING_ATTEMPT
====================
Connection with no data exchagef that looks like a probing attempt
+1
View File
@@ -14,6 +14,7 @@ nDPI is an open source DPI (Deep Packet Inspection) toolkit for traffic analysis
:caption: Developer's Guide
api/library_root
protocols
flow_risks
.. toctree::
+84
View File
@@ -0,0 +1,84 @@
A simple, common example
```
struct ndpi_detection_module_struct *ndpi_struct;
ndpi_cfg_error rc;
int ret;
ndpi_struct = ndpi_init_detection_module(NULL);
if(!ndpi_struct) {
ERROR;
}
/* Configuration */
rc = ndpi_set_config(ndpi_struct, "tls", "certificate_expiration_threshold", "10");
if(rc != NDPI_CFG_OK) {
ERROR;
}
/* Finalization */
ret = ndpi_finalize_initialization(ndpi_struct);
if(ret != 0) {
ERROR;
}
/* Initialization done, now you can feed packets to the library */
/* Cleanup */
ndpi_exit_detection_module(ndpi_struct);
```
A more complex example, with global context and a shared Oookla LRU cache (all the others caches are local)
```
struct ndpi_global_context *g_ctx;
struct ndpi_detection_module_struct *ndpi_structs[num_local_contexts];
ndpi_cfg_error rc;
int i, ret;
g_ctx = ndpi_global_init();
if(!g_ctx) {
ERROR;
}
for(i = 0; i < num_local_contexts; i++) {
ndpi_structs[i] = ndpi_init_detection_module(g_ctx);
if(!ndpi_struct[i]) {
ERROR;
}
rc = ndpi_set_config(ndpi_structs[i], NULL, "lru.ookla.scope", "1");
if(rc != NDPI_CFG_OK) {
ERROR;
}
ret = ndpi_finalize_initialization(ndpi_structs[i]);
if(ret != 0) {
ERROR;
}
}
/* Initialization done */
/* Cleanup */
for(i = 0; i < num_local_contexts; i++) {
ndpi_exit_detection_module(ndpi_structs[i]);
}
ndpi_global_deinit(g_ctx);
```
+858
View File
@@ -0,0 +1,858 @@
nDPI Protocols List
###################
This page provides the list of the protocols/applications supported by nDPI. For each protocol there is a brief description, some links to further, more detailed information and, optionally, some notes that might be useful when handling such a protocol (from the application/integrator point of view)
Work in progress!
.. _Proto 32:
`NDPI_PROTOCOL_BFCP`
=======================
BFCP (Binary Floor Control Protocol) is used for controlling and coordinating real-time data sharing and collaboration during video conferencing sessions.
References: `RFC: <https://datatracker.ietf.org/doc/html/rfc8855>`_
.. _Proto 54:
`NDPI_PROTOCOL_IQIYI`
===========================
iQIYI is a Chinese online video platform that offers a wide range of original and licensed content including movies, dramas, variety shows, and anime.
References: `Main site: <https://www.iqiyi.com/>`_
.. _Proto 59:
`NDPI_PROTOCOL_ADOBE_CONNECT`
===========================
Adobe Connect is a web conferencing platform that allows users to conduct online meetings, webinars, and virtual classrooms.
References: `Main site: <https://www.adobe.com/products/adobeconnect.html>`_
.. _Proto 65:
`NDPI_PROTOCOL_IRC`
==================
IRC (Internet Relay Chat) is a text-based chat system for instant messaging.
References: `Wikipiedia: <https://en.wikipedia.org/wiki/IRC>`_ and `Some statistics: <https://netsplit.de/networks/top10.php>`
.. _Proto 72:
`NDPI_PROTOCOL_HART_IP`
=======================
Highway Addressable Remote Transducer over IP
References: `Protocol Specs: <https://library.fieldcommgroup.org/20085/TS20085>`_
.. _Proto 96:
`NDPI_PROTOCOL_TFTP`
====================
TFTP is a very simple protocol used to transfer files. It is from this that its name comes, Trivial File Transfer Protocol or TFTP.
References: `RFC1350 <https://datatracker.ietf.org/doc/html/rfc1350>`_ and `RFC2347 <https://datatracker.ietf.org/doc/html/rfc2347>`_ and `RFC2349 <https://datatracker.ietf.org/doc/html/rfc2349>`_
.. _Proto 127:
`NDPI_PROTOCOL_MS_RPCH`
=======================
A Remote Procedure Call protocol over HTTP from Microsoft.
References: `Protocol Specs: <https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-rpch/c0f4c9c5-1a61-4d10-b8e2-005378d1d212>`_
.. _Proto 149:
`NDPI_PROTOCOL_EGD`
=========================
Ethernet Global Data (EGD) is a communication protocol developed by GE Fanuc Automation for real-time data exchange between automation devices and control systems using standard Ethernet technology. It is widely adopted in industrial environments for its simplicity and reliability in data communication.
References: `Wikipedia <https://en.wikipedia.org/wiki/Ethernet_Global_Data_Protocol>`_
Notes:
- This dissector only works for data packets, not configuration commands.
- IPv6 is not supported
.. _Proto 182:
`NDPI_PROTOCOL_RESP`
=======================
Redis Serialization Protocol
References: `Protocol Specs: <https://redis.io/docs/reference/protocol-spec/>`_
.. _Proto 186:
`NDPI_PROTOCOL_COD_MOBILE`
=======================
Call of Duty: Mobile is a free-to-play shooter game for iOS and Android devices. It has amassed over 650 million downloads worldwide, making it one of the most popular mobile games.
References: `Main site: <https://www.callofduty.com/mobile/>`_
.. _Proto 201:
`NDPI_PROTOCOL_GOOGLE_MEET`
===========================
Google Meet is a video conferencing service from Google.
References: `Main site: <https://meet.google.com/>`_
.. _Proto 235:
`NDPI_PROTOCOL_VALVE_SDR`
===========================
Steam Datagram Relay (SDR) is Valve's virtual private gaming network. Used in all modern games from Valve, but can also be used by developers via the Steamworks SDK for their games.
References: `Main site: <https://partner.steamgames.com/doc/features/multiplayer/steamdatagramrelay>`_
.. _Proto 338:
`NDPI_PROTOCOL_SRTP`
====================
The Secure Real-time Transport Protocol (SRTP) is a profile for Real-time Transport Protocol (RTP) intended to provide encryption, message authentication, integrity, and replay attack protection to the RTP data.
References: `RFC3711 <https://datatracker.ietf.org/doc/html/rfc3711>`_
Notes:
- You can think of SRTP simply as the "encrypted" version of RTP, something like HTTPS vs HTTP;
- It is not usually possible to tell RTP from SRTP. nDPI generally uses the former and it uses the latter only when it is really sure that the media stream has been encrypted.
.. _Proto 339:
`NDPI_PROTOCOL_OPERA_VPN`
=========================
Opera VPN is a free VPN included with Opera Browser.
References: `Main site <https://www.opera.com/it/features/free-vpn>`_
.. _Proto 340:
`NDPI_PROTOCOL_EPICGAMES`
=========================
Epic Games is a video game company developing the Unreal Engine and some successful games as Fortnite and Gears of War.
References: `Main site <https://store.epicgames.com/en-US/>`_ and `Fortnite <https://www.fortnite.com/>`_
.. _Proto 341:
`NDPI_PROTOCOL_GEFORCENOW`
==========================
GeForce Now is the brand used by Nvidia for its cloud gaming service.
References: `Main site <https://www.nvidia.com/en-us/geforce-now/>`_
.. _Proto 342:
`NDPI_PROTOCOL_NVIDIA`
======================
Generic web traffic from Nvidia sites.
References: `Main site <https://www.nvidia.com>`_
.. _Proto 343:
`NDPI_PROTOCOL_BITCOIN`
=======================
Bitcoin is one of the most common crypto currencies.
References: `Main site <https://en.bitcoin.it/wiki/Protocol_documentation>`_
Notes:
- Not each crypto exchange is a mining, it could be a normal transaction, sending or receving.
- Bitcoin network protocol covers the broader set of rules that govern how all nodes in the network communicate and sync with each others blocks and transactions.
- On the other hand mining protocols are more specific and deal directly with how miners interact with the network and participate in the mining process.
.. _Proto 344:
`NDPI_PROTOCOL_PROTONVPN`
=========================
Proton VPN is a VPN service operated by the Swiss company Proton AG, the company behind the email service Proton Mail
References: `Main site <https://protonvpn.com/>`_
.. _Proto 345:
`NDPI_PROTOCOL_THRIFT`
======================
Apache Thrift is a generic data interchange framework that supports a bunch of different languages and platforms.
References: `Official site <https://thrift.apache.org>`_ and `Github <https://github.com/apache/thrift>`_
.. _Proto 346:
`NDPI_PROTOCOL_ROBLOX`
======================
Roblox is an online game platform and game creation system.
References: `Main site <https://www.roblox.com/>`_
Notes:
- Since Roblox games use a custom version of the RakNet protocol, some Roblox flows might be classified as RakNet.
.. _Proto 347:
`NDPI_PROTOCOL_SERVICE_LOCATION`
================================
The Service Location Protocol is a service discovery protocol that allows computers and other devices to find services in a local area network without prior configuration.
References: `SLPv1 <https://datatracker.ietf.org/doc/html/rfc2165>`_ and `SLPv2 <https://datatracker.ietf.org/doc/html/rfc2608>`_
.. _Proto 348:
`NDPI_PROTOCOL_MULLVAD`
=======================
Mullvad is a VPN service operated by Mullvad VPN AB, based in Sweden
References: `Main site <https://mullvad.net/>`_
.. _Proto 349:
`NDPI_PROTOCOL_HTTP2`
=====================
HTTP/2 (originally named HTTP/2.0) is a major revision of the HTTP network protocol used by the World Wide Web.
References: `RFC <https://datatracker.ietf.org/doc/html/rfc9113>`_
Notes:
- HTTP/2 is almost always encrypted, i.e. transported over TLS.
.. _Proto 350:
`NDPI_PROTOCOL_HAPROXY`
=======================
HAProxy is a free and open source software that provides a high availability load balancer and reverse proxy for TCP and HTTP-based applications that spreads requests across multiple servers.
References: `Main site: <https://www.haproxy.org>`_
.. _Proto 351:
`NDPI_PROTOCOL_RMCP`
====================
The Intelligent Platform Management Interface (IPMI) is a set of computer interface specifications for an autonomous computer subsystem that provides management and monitoring capabilities independently of the host system's CPU, firmware (BIOS or UEFI) and operating system.
References: `Protocol Specs: <https://www.dmtf.org/sites/default/files/standards/documents/DSP0114.pdf>`_
.. _Proto 352:
`NDPI_PROTOCOL_CAN`
===================
Controller Area Network (CAN) is used extensively in automotive applications, with in excess of 400 million CAN enabled microcontrollers manufactured each year.
References: `Protocol Specs: <https://www.iso.org/standard/63648.html>`_
.. _Proto 353:
`NDPI_PROTOCOL_PROTOBUF`
========================
Protocol Buffers (Protobuf) is a free and open-source cross-platform data format used to serialize structured data.
References: `Encoding: <https://protobuf.dev/programming-guides/encoding>`_
.. _Proto 354:
`NDPI_PROTOCOL_ETHEREUM`
=======================
Ethereum is a decentralized, open-source blockchain with smart contract functionality.
References: `Main site <https://ethereum.org/en/developers/docs/intro-to-ethereum/>`_
Notes:
- same as Bitcoin, not each crypto exchange is a mining, it could be a normal transaction, sending or receving or even blockchain exploration.
.. _Proto 355:
`NDPI_PROTOCOL_TELEGRAM_VOIP`
============================
Audio/video calls made using the telegram app.
References: `Wikipedia <https://en.wikipedia.org/wiki/telegram_(software)/>`_
.. _Proto 356:
`NDPI_PROTOCOL_SINA_WEIBO`
============================
Chinese microblogging (weibo) website.
References: `Wikipedia <https://en.wikipedia.org/wiki/Sina_Weibo>`_
.. _Proto 358:
`NDPI_PROTOCOL_PTPV2`
============================
IEEE 1588-2008 Precision Time Protocol (PTP) Version 2.
References: `Protocol Specs: <https://standards.ieee.org/ieee/1588/4355/>`_
.. _Proto 359:
`NDPI_PROTOCOL_RTPS`
============================
Real-Time Publish Subscribe Protocol
References: `Protocol Specs: <https://www.omg.org/spec/DDSI-RTPS/>`_
.. _Proto 360:
`NDPI_PROTOCOL_OPC_UA`
============================
IEC62541 OPC Unified Architecture
References: `Protocol Specs: <https://reference.opcfoundation.org/>`_
.. _Proto 361:
`NDPI_PROTOCOL_S7COMM_PLUS`
============================
A proprietary protocol from Siemens used for data exchange between PLCs and access PLC data via SCADA systems.
Completely different from classic S7Comm, but also uses TPKT/COTP as a transport.
References: `Unofficial description: <https://plc4x.apache.org/protocols/s7/s7comm-plus.html>`_
.. _Proto 362:
`NDPI_PROTOCOL_FINS`
============================
Factory Interface Network Service (FINS) is a network protocol used by Omron PLCs.
References: `Protocol Specs: <https://assets.omron.eu/downloads/manual/en/v4/w421_cj1w-etn21_cs1w-etn21_ethernet_units_-_construction_of_applications_operation_manual_en.pdf>`_
.. _Proto 363:
`NDPI_PROTOCOL_ETHERSIO`
============================
Ether-S-I/O is a proprietary protocol used by Saia-Burgess's PLCs.
References: `Wireshark wiki: <https://wiki.wireshark.org/EtherSIO.md>`_
.. _Proto 364:
`NDPI_PROTOCOL_UMAS`
============================
UMAS is a proprietary Schneider Electric protocol based on Modbus. It's used in Modicon M580 and Modicon M340 CPU-based PLCs.
References: `Unofficial article: <https://ics-cert.kaspersky.com/publications/reports/2022/09/29/the-secrets-of-schneider-electrics-umas-protocol/>`_
.. _Proto 365:
`NDPI_PROTOCOL_BECKHOFF_ADS`
============================
Automation Device Specification is the protocol used for interfacing with Beckhoff PLCs via TwinCAT.
References: `Protocol Specs: <https://infosys.beckhoff.com/english.php?content=../content/1033/tc3_ads_intro/115847307.html>`_
.. _Proto 366:
`NDPI_PROTOCOL_ISO9506_1_MMS`
============================
The international standard MMS (Manufacturing Message Specification) is an OSI application layer messaging protocol origionally designed for the remote control and monitoring of devices such as Remote Terminal Units (RTU), Programmable Logic Controllers (PLC), Numerical Controllers (NC), or Robot Controllers (RC).
References: `Paid Specs: <https://www.iso.org/ru/standard/37079.html>`_
.. _Proto 367:
`NDPI_PROTOCOL_IEEE_C37118`
============================
IEEE Standard for Synchrophasor Data Transfer for Power Systems
References: `Paid Specs: <https://standards.ieee.org/ieee/C37.118.1/4902/>`_
.. _Proto 368:
`NDPI_PROTOCOL_ETHERSBUS`
============================
Ether-S-Bus is a proprietary protocol used for the communication with and between PLCs manufactured by Saia-Burgess Controls Ltd.
References: `Wireshark wiki: <https://wiki.wireshark.org/EtherSBus>`_
.. _Proto 369:
`NDPI_PROTOCOL_MONERO`
======================
Monero is a private and decentralized cryptocurrency with focus on confidentiality and security.
.. _Proto 370:
`NDPI_PROTOCOL_DCERPC`
======================
DCE/RPC is a specification for a remote procedure call mechanism that defines both APIs and an over-the-network protocol.
References: `Wireshark wiki: <https://wiki.wireshark.org/DCE/RPC>`_
.. _Proto 371:
`NDPI_PROTOCOL_PROFINET_IO`
===========================
PROFINET/IO is a field bus protocol based on connectionless DCE/RPC.
References: `Protocol Specs: <https://www.profibus.com/download/profinet-specification>`_
.. _Proto 372:
`NDPI_PROTOCOL_HISLIP`
======================
High-Speed LAN Instrument Protocol (HiSLIP) is a protocol for remote instrument control of LAN-based test and measurement instruments.
References: `Protocol Specs: <https://www.ivifoundation.org/downloads/Protocol%20Specifications/IVI-6.1_HiSLIP-2.0-2020-04-23.pdf>`_
.. _Proto 373:
`NDPI_PROTOCOL_UFTP`
====================
Encrypted UDP based FTP with multicast.
References: `Protocol Specs: <https://uftp-multicast.sourceforge.net/protocol.txt>`_.
.. _Proto 374:
`NDPI_PROTOCOL_OPENFLOW`
========================
OpenFlow protocol is a network protocol closely associated with Software-Defined Networking (SDN).
References: `Protocol Specs: <https://opennetworking.org/wp-content/uploads/2014/10/openflow-switch-v1.5.1.pdf>`_
.. _Proto 375:
`NDPI_PROTOCOL_JSON_RPC`
========================
JSON-RPC is a remote procedure call protocol encoded in JSON.
References: `Protocol Specs: <https://www.jsonrpc.org/specification>`_
.. _Proto 376:
`NDPI_PROTOCOL_WEBDAV`
======================
WebDAV is a set of extensions to the HTTP protocol that allows WebDAV clients to collaboratively edit and manage files on remote Web servers.
References: `RFC4918: <https://datatracker.ietf.org/doc/html/rfc4918>`_
Notes:
- WebDAV is almost always encrypted, i.e. transported over TLS.
.. _Proto 377:
`NDPI_PROTOCOL_APACHE_KAFKA`
============================
Apache Kafka is a distributed event store and stream-processing platform.
References: `Official site <https://kafka.apache.org>`_ and `Github <https://github.com/apache/kafka>`_
.. _Proto 378:
`NDPI_PROTOCOL_NOMACHINE`
=========================
NoMachine is a popular proprietary remote desktop software.
References: `Main site <https://www.nomachine.com/>`_
.. _Proto 379:
`NDPI_PROTOCOL_IEC62056`
============================
IEC 62056-4-7 DLMS/COSEM is a transport layer for IP networks.
References: `Paid Specs: <https://webstore.iec.ch/publication/22487>`_
Notes:
- Wireshark is not able to recognize this protocol. Some old plugins/code (with some documentation) are available `here <https://github.com/bearxiong99/wireshark-dlms>` and `here <https://github.com/matousp/dlms-analysis/tree/master>`.
.. _Proto 380:
`NDPI_PROTOCOL_HL7`
=========================
HL7 is a range of global standards for the transfer of clinical and administrative health data between applications.
References: `Main site <https://www.hl7.org/>`_
.. _Proto 381:
`NDPI_PROTOCOL_CEPH`
=========================
Ceph is a scalable distributed storage system.
References: `Main site <https://ceph.io/en/>`_
.. _Proto 382:
`NDPI_PROTOCOL_GOOGLE_CHAT`
=========================
Google Chat is an instant messaging service from Google, which replaced Hangouts.
References: `Main site <https://chat.google.com/>`_
.. _Proto 383:
`NDPI_PROTOCOL_ROUGHTIME`
=========================
A protocol that aims to achieve rough time synchronization while detecting servers that provide inaccurate time and providing cryptographic proof of their malfeasance.
References: `IETF Draft <https://www.ietf.org/archive/id/draft-ietf-ntp-roughtime-08.html>`_
.. _Proto 384:
`NDPI_PROTOCOL_PIA`
=========================
Private Internet Access (PIA) is a popular VPN service from Kape Technologies.
References: `Main site <https://www.privateinternetaccess.com/>`_
.. _Proto 385:
`NDPI_PROTOCOL_KCP`
===================
KCP - A Fast and Reliable ARQ Protocol. It provides TCP-like stream support with low latency at the cost of bandwidth usage - used by lot's of Open Source / Third Party applications.
References: `Protocol Specs: <https://github.com/skywind3000/kcp/blob/master/protocol.txt>`_
.. _Proto 386:
`NDPI_PROTOCOL_DOTA2`
=========================
Dota 2 is an extremely popular multiplayer MOBA game from Valve.
References: `Main site <https://www.dota2.com/>`_
.. _Proto 387:
`NDPI_PROTOCOL_MUMBLE`
=========================
Mumble is a free, open source, low latency, high quality voice chat application.
References: `Main site <https://www.mumble.info/>`_
.. _Proto 388:
`NDPI_PROTOCOL_YOJIMBO`
=======================
Yojimbo (netcode) is a secure connection-based client/server protocol built on top of UDP.
References: `Protocol Specs: <https://github.com/mas-bandwidth/netcode/blob/main/STANDARD.md>`_
.. _Proto 389:
`NDPI_PROTOCOL_ELECTRONICARTS`
=========================
Electronic Arts is a leading publisher of games on Console, PC and Mobile.
References: `Main site <https://www.ea.com/>`_
Notes:
- Almost all of that traffic is related to their EA Origin game store.
.. _Proto 390:
`NDPI_PROTOCOL_STOMP`
========================
STOMP is a simple interoperable protocol designed for asynchronous message passing between clients via mediating servers. Supported in ActiveMQ and RabbitMQ.
References: `Protocol Specs: <https://stomp.github.io/stomp-specification-1.2.html>`_
.. _Proto 391:
`NDPI_PROTOCOL_RADMIN`
=========================
Radmin is remote access software for the Microsoft Windows platform.
References: `Main site <https://www.radmin.com/>`_
.. _Proto 392:
`NDPI_PROTOCOL_RAFT`
====================
Raft is a consensus algorithm and protocol for managing a replicated log.
References: `C implementation <https://github.com/canonical/raft>`_ and `Paper <https://raft.github.io/raft.pdf>`_
.. _Proto 394:
`NDPI_PROTOCOL_GEARMAN`
====================
Gearman is a network-based job-queuing system that was initially developed by Danga Interactive in order to process large volumes of jobs.
References: `Main site <http://gearman.org/>`_
.. _Proto 395:
`NDPI_PROTOCOL_TENCENTGAMES`
====================
A protocol used by various games from Tencent (mostly mobile games).
References: `Main site <https://www.tencentgames.com/>`_
.. _Proto 396:
`NDPI_PROTOCOL_GAIJIN`
====================
Protocols used in various games from Gaijin Entertainment.
References: `Main site <https://gaijin.net/>`_
.. _Proto 397:
`NDPI_PROTOCOL_C1222`
====================
ANSI C12.22 (IEEE Std 1703) describe a protocol for transporting ANSI C12.19 table data over networks. It's mostly used to communicate with electric meters.
References: `Paid specs <https://www.nema.org/Standards/view/American-National-Standard-for-Protocol-Specification-for-Interfacing-to-Data-Communication-Networks/>`_
.. _Proto 398:
`NDPI_PROTOCOL_HUAWEI`
======================
Generic Huawei traffic.
References: `Main site <https://www.huawei.com/>`_
.. _Proto 399:
`NDPI_PROTOCOL_HUAWEI_CLOUD`
============================
Huawei Mobile Cloud.
References: `Main site <https://cloud.huawei.com/>`_
.. _Proto 400:
`NDPI_PROTOCOL_DLEP`
=====================
The Dynamic Link ExchangeProtocol (DLEP) is a radio aware routing (RAR) protocol.
References: `RFC <https://datatracker.ietf.org/doc/html/rfc8175>`_
.. _Proto 401:
`NDPI_PROTOCOL_BFD`
=====================
Bidirectional Forwarding Detection is a network protocol that is used to detect faults between two routers or switches.
References: `RFC <https://datatracker.ietf.org/doc/html/rfc5880>`_
.. _Proto 402:
`NDPI_PROTOCOL_NETEASE_GAMES`
============================
Traffic of various NetEase games.
References: `Main site <https://www.neteasegames.com/>`_
.. _Proto 403:
`NDPI_PROTOCOL_PATHOFEXILE`
============================
Path of Exile is a free-to-play online Action RPG.
References: `Main site <https://pathofexile.com/>`_
.. _Proto 404:
`NDPI_PROTOCOL_GOOGLE_CALL`
===========================
Audio/video calls made by (any) Google applications (i.e Google Meet).
References: `Main site: <https://meet.google.com/>`_
Notes:
- nDPI usually uses different protocol ids for the generic application traffic and for its "realtime" traffic (examples: NDPI_PROTOCOL_MEET/NDPI_PROTOCOL_GOOGLE_CALL, NDPI_PROTOCOL_WHATSAPP/NDPI_PROTOCOL_WHATSAPP_CALL, ...)
.. _Proto 405:
`NDPI_PROTOCOL_PFCP`
=====================
PFCP is a protocol used for communicating between control plane (CP) and user plane (UP) functions in 4G and 5G networks.
References: `Protocol Specs: <https://www.etsi.org/deliver/etsi_ts/129200_129299/129244/16.05.00_60/ts_129244v160500p.pdf>`_
.. _Proto 406:
`NDPI_PROTOCOL_FLUTE`
=====================
File Delivery over Unidirectional Transport.
References: `RFC <https://datatracker.ietf.org/doc/html/rfc6726>`_
.. _Proto 407:
`NDPI_PROTOCOL_LOLWILDRIFT`
============================
League of Legends: Wild Rift is a mobile MOBA game.
References: `Main site <https://wildrift.leagueoflegends.com/>`_
.. _Proto 408:
`NDPI_PROTOCOL_TESO`
============================
The Elder Scrolls Online is a MMORPG set in the fantasy world of Tamriel.
References: `Main site <https://www.elderscrollsonline.com/>`_
.. _Proto 409:
`NDPI_PROTOCOL_LDP`
=====================
The Label Distribution Protocol (LDP) is a routing protocol used to establish and maintain label-switched paths in a Multiprotocol Label Switching (MPLS) network.
References: `RFC <https://datatracker.ietf.org/doc/html/rfc5036>`_
.. _Proto 410:
`NDPI_PROTOCOL_KNXNET_IP`
=====================
KNXnet/IP is a building automation protocol that enables the exchange of data and control information over IP networks, extending the KNX standard for home and building automation.
References: `Paid Specs: <https://webstore.ansi.org/standards/ds/dsiso225102019>`_
.. _Proto 411:
`NDPI_PROTOCOL_BLUESKY`
======================
Bluesky, also known as Bluesky Social, is a decentralized microblogging social platform.
References: `Main site: <https://bsky.app/>`_
.. _Proto 412:
`NDPI_PROTOCOL_MASTODON`
=======================
Mastodon is free and open-source software for running self-hosted social networking services. It has microblogging features similar to Twitter.
References: `Main site: <https://joinmastodon.org/>`_
.. _Proto 413:
`NDPI_PROTOCOL_THREADS`
======================
Threads is an online social media and social networking service operated by Meta Platforms.
References: `Main site: <https://www.threads.net>`_
.. _Proto 414:
`NDPI_PROTOCOL_VIBER_VOIP`
=========================
Audio/video calls made using the Viber app.
References: `Wikipedia <https://en.wikipedia.org/wiki/Viber>`_
.. _Proto 415:
`NDPI_PROTOCOL_ZUG`
=========================
The ZUG protocol is part of the Casper 2.0 consensus model.
References: `Main Site <https://casperlabs.io>`_ and `Blog Post <https://casperlabs.io/blog/beyond-eth-30-theres-casper-20>`_
.. _Proto 416:
`NDPI_PROTOCOL_JRMI`
=========================
The JRMI protocol is the Java Remote Method Invocation protocol.
References: `Oracle site <https://docs.oracle.com/en/java/javase/21/docs/specs/rmi/protocol.html>`_
.. _Proto 417:
`NDPI_PROTOCOL_RIPE_ATLAS`
==========================
The RIPE Atlas probe protocol is used for the world's largest active Internet measurement network.
References: `Main Site <https://atlas.ripe.net/>`_ and `Documentation <https://ripe-atlas-tools.readthedocs.io/en/latest/index.html>`_
+6
View File
@@ -1,6 +1,12 @@
AR=@AR@
CC=@CC@
CXX=@CXX@
ifneq ($(OS),Windows_NT)
OS := $(shell uname)
endif
ifeq ($(OS),Darwin)
CC=clang -fno-color-diagnostics
endif
BUILD_MINGW=@BUILD_MINGW@
BUILD_MINGW_X64=@BUILD_MINGW_X64@
DISABLE_NPCAP=@DISABLE_NPCAP@
+3
View File
@@ -1,3 +1,6 @@
# Format: name\tcategory_id
internetbadguys.com 100
144.139.247.220 100
172.30.69.0/24 100
[2001:1670:8:40a6:a08e:332b:aa69:18dc] 100
[2001:db8:1::1]/127 100
+13
View File
@@ -0,0 +1,13 @@
#Format: <proto, param, value>
#proto might be empty
#Basic example of how to set nDPI configuration parameters via file
#We set only some default values
#See doc/configuration_parameters.md for a complete list and description of all the accepted knobs
packets_limit_per_flow,32
ookla,dpi.aggressiveness,0x1
tls,metadata.sha1_fingerprint,1
lru.bittorrent.ttl,0
+1641 -636
View File
File diff suppressed because it is too large Load Diff
+46 -28
View File
@@ -188,8 +188,7 @@ static struct nDPI_workflow * init_workflow(char const * const file_or_device)
printf("pcap_setfilter error: '%s'\n", pcap_geterr(workflow->pcap_handle));
}
ndpi_init_prefs init_prefs = ndpi_no_prefs;
workflow->ndpi_struct = ndpi_init_detection_module(init_prefs);
workflow->ndpi_struct = ndpi_init_detection_module(NULL);
if (workflow->ndpi_struct == NULL) {
free_workflow(&workflow);
return NULL;
@@ -230,6 +229,7 @@ static void ndpi_flow_info_freer(void * const node)
static void free_workflow(struct nDPI_workflow ** const workflow)
{
struct nDPI_workflow * const w = *workflow;
size_t i;
if (w == NULL) {
return;
@@ -243,7 +243,7 @@ static void free_workflow(struct nDPI_workflow ** const workflow)
if (w->ndpi_struct != NULL) {
ndpi_exit_detection_module(w->ndpi_struct);
}
for(size_t i = 0; i < w->max_active_flows; i++) {
for(i = 0; i < w->max_active_flows; i++) {
ndpi_tdestroy(w->ndpi_flows_active[i], ndpi_flow_info_freer);
}
ndpi_free(w->ndpi_flows_active);
@@ -272,6 +272,7 @@ static int setup_reader_threads(char const * const file_or_device)
{
char * file_or_default_device;
char pcap_error_buffer[PCAP_ERRBUF_SIZE];
int i;
if (reader_thread_count > MAX_READER_THREADS) {
return 1;
@@ -290,7 +291,7 @@ static int setup_reader_threads(char const * const file_or_device)
}
}
for (int i = 0; i < reader_thread_count; ++i) {
for (i = 0; i < reader_thread_count; ++i) {
reader_threads[i].workflow = init_workflow(file_or_default_device);
if (reader_threads[i].workflow == NULL)
{
@@ -500,8 +501,10 @@ static int ndpi_workflow_node_cmp(void const * const A, void const * const B) {
static void check_for_idle_flows(struct nDPI_workflow * const workflow)
{
size_t idle_scan_index;
if (workflow->last_idle_scan_time + IDLE_SCAN_PERIOD < workflow->last_time) {
for (size_t idle_scan_index = 0; idle_scan_index < workflow->max_active_flows; ++idle_scan_index) {
for (idle_scan_index = 0; idle_scan_index < workflow->max_active_flows; ++idle_scan_index) {
ndpi_twalk(workflow->ndpi_flows_active[idle_scan_index], ndpi_idle_scan_walker, workflow);
while (workflow->cur_idle_flows > 0) {
@@ -530,7 +533,7 @@ static void ndpi_process_packet(uint8_t * const args,
struct nDPI_reader_thread * const reader_thread =
(struct nDPI_reader_thread *)args;
struct nDPI_workflow * workflow;
struct nDPI_flow_info flow = {};
struct nDPI_flow_info flow;
size_t hashed_index;
void * tree_result;
@@ -551,6 +554,8 @@ static void ndpi_process_packet(uint8_t * const args,
uint16_t type;
uint32_t thread_index = INITIAL_THREAD_HASH; // generated with `dd if=/dev/random bs=1024 count=1 |& hd'
memset(&flow, '\0', sizeof(flow));
if (reader_thread == NULL) {
return;
}
@@ -736,24 +741,33 @@ static void ndpi_process_packet(uint8_t * const args,
print_packet_info(reader_thread, header, l4_len, &flow);
#endif
/* calculate flow hash for btree find, search(insert) */
if (flow.l3_type == L3_IP) {
if (ndpi_flowv4_flow_hash(flow.l4_protocol, flow.ip_tuple.v4.src, flow.ip_tuple.v4.dst,
flow.src_port, flow.dst_port, 0, 0,
(uint8_t *)&flow.hashval, sizeof(flow.hashval)) != 0)
{
uint64_t tmp[4] = {};
/* calculate flow hash for btree find, search(insert) */
if (flow.l3_type == L3_IP) {
if (ndpi_flowv4_flow_hash(flow.l4_protocol, flow.ip_tuple.v4.src, flow.ip_tuple.v4.dst,
flow.src_port, flow.dst_port, 0, 0,
(uint8_t *)&tmp[0], sizeof(tmp)) != 0)
{
flow.hashval = flow.ip_tuple.v4.src + flow.ip_tuple.v4.dst; // fallback
flow.hashval = flow.ip_tuple.v4.src + flow.ip_tuple.v4.dst; // fallback
} else {
flow.hashval = tmp[0] + tmp[1] + tmp[2] + tmp[3];
}
} else if (flow.l3_type == L3_IP6) {
if (ndpi_flowv6_flow_hash(flow.l4_protocol, &ip6->ip6_src, &ip6->ip6_dst,
flow.src_port, flow.dst_port, 0, 0,
(uint8_t *)&flow.hashval, sizeof(flow.hashval)) != 0)
} else if (flow.l3_type == L3_IP6) {
if (ndpi_flowv6_flow_hash(flow.l4_protocol, &ip6->ip6_src, &ip6->ip6_dst,
flow.src_port, flow.dst_port, 0, 0,
(uint8_t *)&tmp[0], sizeof(tmp)) != 0)
{
flow.hashval = flow.ip_tuple.v6.src[0] + flow.ip_tuple.v6.src[1];
flow.hashval += flow.ip_tuple.v6.dst[0] + flow.ip_tuple.v6.dst[1];
flow.hashval = flow.ip_tuple.v6.src[0] + flow.ip_tuple.v6.src[1];
flow.hashval += flow.ip_tuple.v6.dst[0] + flow.ip_tuple.v6.dst[1];
} else {
flow.hashval = tmp[0] + tmp[1] + tmp[2] + tmp[3];
}
}
flow.hashval += flow.l4_protocol + flow.src_port + flow.dst_port;
}
flow.hashval += flow.l4_protocol + flow.src_port + flow.dst_port;
hashed_index = flow.hashval % workflow->max_active_flows;
tree_result = ndpi_tfind(&flow, &workflow->ndpi_flows_active[hashed_index], ndpi_workflow_node_cmp);
@@ -866,7 +880,7 @@ static void ndpi_process_packet(uint8_t * const args,
flow_to_process->guessed_protocol =
ndpi_detection_giveup(workflow->ndpi_struct,
flow_to_process->ndpi_flow,
1, &protocol_was_guessed);
&protocol_was_guessed);
if (protocol_was_guessed != 0) {
printf("[%8llu, %d, %4d][GUESSED] protocol: %s | app protocol: %s | category: %s\n",
workflow->packets_captured,
@@ -886,8 +900,7 @@ static void ndpi_process_packet(uint8_t * const args,
ip != NULL ? (uint8_t *)ip : (uint8_t *)ip6,
ip_size, time_ms, NULL);
if (ndpi_is_protocol_detected(workflow->ndpi_struct,
flow_to_process->detected_l7_protocol) != 0 &&
if (ndpi_is_protocol_detected(flow_to_process->detected_l7_protocol) != 0 &&
flow_to_process->detection_completed == 0)
{
if (flow_to_process->detected_l7_protocol.master_protocol != NDPI_PROTOCOL_UNKNOWN ||
@@ -1019,7 +1032,9 @@ static void * processing_thread(void * const ndpi_thread_arg)
static int processing_threads_error_or_eof(void)
{
for (int i = 0; i < reader_thread_count; ++i) {
int i;
for (i = 0; i < reader_thread_count; ++i) {
if (__sync_fetch_and_add(&reader_threads[i].workflow->error_or_eof, 0) == 0) {
return 0;
}
@@ -1029,6 +1044,8 @@ static int processing_threads_error_or_eof(void)
static int start_reader_threads(void)
{
int i;
#ifndef WIN32
sigset_t thread_signal_set, old_signal_set;
@@ -1041,7 +1058,7 @@ static int start_reader_threads(void)
}
#endif
for (int i = 0; i < reader_thread_count; ++i) {
for (i = 0; i < reader_thread_count; ++i) {
reader_threads[i].array_index = i;
if (reader_threads[i].workflow == NULL) {
@@ -1067,6 +1084,7 @@ static int start_reader_threads(void)
static int stop_reader_threads(void)
{
int i;
unsigned long long int total_packets_captured = 0;
unsigned long long int total_packets_processed = 0;
unsigned long long int total_l4_data_len = 0;
@@ -1074,13 +1092,13 @@ static int stop_reader_threads(void)
unsigned long long int total_flows_idle = 0;
unsigned long long int total_flows_detected = 0;
for (int i = 0; i < reader_thread_count; ++i) {
for (i = 0; i < reader_thread_count; ++i) {
break_pcap_loop(&reader_threads[i]);
}
printf("------------------------------------ Stopping reader threads\n");
for (int i = 0; i < reader_thread_count; ++i) {
for (i = 0; i < reader_thread_count; ++i) {
if (reader_threads[i].workflow == NULL) {
continue;
}
@@ -1105,7 +1123,7 @@ static int stop_reader_threads(void)
/* total packets captured: same value for all threads as packet2thread distribution happens later */
total_packets_captured = reader_threads[0].workflow->packets_captured;
for (int i = 0; i < reader_thread_count; ++i) {
for (i = 0; i < reader_thread_count; ++i) {
if (reader_threads[i].workflow == NULL) {
continue;
}
@@ -1137,7 +1155,7 @@ static void sighandler(int signum)
int main(int argc, char ** argv)
{
if (argc == 0) {
printf("usage: ndpiSimpleIntegration Mdevice name>\n");
printf("usage: ndpiSimpleIntegration <device name>\n");
return 1;
}
+24 -2
View File
@@ -2,7 +2,7 @@
# <tcp|udp>:<port>,<tcp|udp>:<port>,.....@<proto>
tcp:81,tcp:8181@HTTP
udp:5061-5062@SIP
udp:5062@SIP
tcp:860,udp:860,tcp:3260,udp:3260@iSCSI
tcp:3000@ntop
tcp:9002@Elasticsearch
@@ -27,6 +27,7 @@ host:"api-global.netflix.com"@Netflix
# IP based Subprotocols
# Format:
# ip:<value>,ip:<value>,.....@<subproto>
# ipv6:[<value>],ipv6:[<value>],.....@<subproto>
#
# NOTES
@@ -45,11 +46,30 @@ host:"api-global.netflix.com"@Netflix
# detected as <L7 proto>.Google but only
# as <L7 proto>
#
ip:213.75.170.11/32:443@CustomProtocol
ip:8.248.73.247:443@AmazonPrime
ip:54.80.47.130@AmazonPrime
#You can specify a protocol Id. In that case you probably want to avoid conflict with internal ids.
#You can use any number up to 65535
ip:3.3.3.3:443@CustomProtocolA
ip:3.3.3.3:444@CustomProtocolB
ip:3.3.3.3:446@CustomProtocolC=800
ipv6:[3ffe:507:0:1:200:86ff:fe05:80da]@CustomProtocolD=1024
ipv6:[247f:855b:5e16:3caf::]/64:100@CustomProtocolE=2048
ipv6:[247f:855b:5e16:3caf::]/64@CustomProtocolF=2049
ipv6:[fe80::76ac:b9ff:fe6c:c124]:12717@CustomProtocolG=2050
ipv6:[fe80::76ac:b9ff:fe6c:c124]:12718@CustomProtocolH=65535
ipv6:[fe80::76ac:b9ff:fe6c:c124]:12719@CustomProtocolI=65534
#
# You can use symbolic IP addreses if you want
#
ip:www.ntop.org@ntop
ipv6:www.ntop.org@ntop
#
# Risk Exceptions
#
@@ -63,6 +83,8 @@ ip:54.80.47.130@AmazonPrime
#ip_risk_mask:192.168.1.0/24=0
ip_risk_mask:10.10.120.0/24=0
ip_risk_mask:10.196.157.228=0
ipv6_risk_mask:[fe80::356b:e047:3695:0]/112=0
ipv6_risk_mask:[fe80::7c0:e74e:87c3:5d93]=0
host_risk_mask:".home"=0
+323 -242
View File
File diff suppressed because it is too large Load Diff
+57 -15
View File
@@ -93,6 +93,10 @@ extern int dpdk_port_deinit(int port);
#define SERIALIZATION_BUFSIZ (8192 * 2)
#ifdef __cplusplus
extern "C" {
#endif
// inner hash table (ja3 -> security state)
typedef struct ndpi_ja3_info {
char * ja3;
@@ -166,7 +170,6 @@ enum info_type {
INFO_TIVOCONNECT,
INFO_FTP_IMAP_POP_SMTP,
INFO_NATPMP,
INFO_RTP
};
// flow tracking
@@ -206,6 +209,7 @@ typedef struct ndpi_flow_info {
ndpi_protocol detected_protocol;
ndpi_confidence_t confidence;
u_int16_t num_dissector_calls;
u_int16_t dpi_packets;
// Flow data analysis
pkt_timeval src2dst_last_pkt_time, dst2src_last_pkt_time, flow_last_pkt_time;
@@ -213,30 +217,36 @@ typedef struct ndpi_flow_info {
*pktlen_c_to_s, *pktlen_s_to_c;
enum info_type info_type;
union {
char info[256];
struct {
unsigned char auth_failed;
char username[127];
char password[128];
} ftp_imap_pop_smtp;
struct {
char domain[85];
char hostname[85];
char username[86];
} kerberos;
struct {
char ip[16];
char port[6];
char hostname[48];
char fqdn[48];
} softether;
struct {
char identity_uuid[36];
char machine[48];
char platform[32];
char services[48];
} tivoconnect;
struct {
uint16_t result_code;
uint16_t internal_port;
@@ -247,12 +257,15 @@ typedef struct ndpi_flow_info {
ndpi_serializer ndpi_flow_serializer;
char flow_extra_info[16];
char host_server_name[80]; /* Hostname/SNI */
char *bittorent_hash;
char *dhcp_fingerprint;
char *dhcp_class_ident;
ndpi_risk risk;
struct {
char currency[16];
} mining;
struct {
u_int16_t ssl_version;
@@ -260,7 +273,7 @@ typedef struct ndpi_flow_info {
client_hassh[33], server_hassh[33], *server_names,
*advertised_alpns, *negotiated_alpn, *tls_supported_versions,
*tls_issuerDN, *tls_subjectDN,
ja3_client[33], ja3_server[33],
ja3_client[33], ja3_server[33], ja4_client[37],
sha1_cert_fingerprint[20];
u_int8_t sha1_cert_fingerprint_set;
struct tls_heuristics browser_heuristics;
@@ -270,24 +283,38 @@ typedef struct ndpi_flow_info {
char *esni;
} encrypted_sni;
struct {
u_int16_t version;
} encrypted_ch;
time_t notBefore, notAfter;
u_int16_t server_cipher;
ndpi_cipher_weakness client_unsafe_cipher, server_unsafe_cipher;
u_int32_t quic_version;
} ssh_tls;
struct {
enum ndpi_rtp_stream_type stream_type;
} rtp;
struct {
char url[256], request_content_type[64], content_type[64], user_agent[256], server[128];
char url[256], request_content_type[64], content_type[64],
user_agent[256], server[128], nat_ip[32], filename[256];
u_int response_status_code;
} http;
struct {
ndpi_address_port mapped_address, peer_address,
relayed_address, response_origin, other_address;
} stun;
struct {
char *username, *password;
} telnet;
struct {
char geolocation_iata_code[4];
} dns;
ndpi_multimedia_flow_type multimedia_flow_type;
void *src_id, *dst_id;
struct ndpi_entropy *entropy;
@@ -353,12 +380,16 @@ typedef struct ndpi_workflow {
struct ndpi_workflow_prefs prefs;
struct ndpi_stats stats;
ndpi_workflow_callback_ptr flow_callback;
void * flow_callback_userdata;
/* outside referencies */
pcap_t *pcap_handle;
/* allocated by prefs */
void **ndpi_flows_root;
struct ndpi_detection_module_struct *ndpi_struct;
struct ndpi_global_context *g_ctx;
u_int32_t num_allocated_flows;
/* CSV,TLV,JSON serialization interface */
@@ -367,7 +398,7 @@ typedef struct ndpi_workflow {
/* TODO: remove wrappers parameters and use ndpi global, when their initialization will be fixed... */
struct ndpi_workflow * ndpi_workflow_init(const struct ndpi_workflow_prefs * prefs, pcap_t * pcap_handle, int do_init_flows_root, ndpi_serialization_format serialization_format);
struct ndpi_workflow * ndpi_workflow_init(const struct ndpi_workflow_prefs * prefs, pcap_t * pcap_handle, int do_init_flows_root, ndpi_serialization_format serialization_format, struct ndpi_global_context *g_ctx);
/* workflow main free function */
@@ -385,25 +416,32 @@ void ndpi_free_flow_info_half(struct ndpi_flow_info *flow);
struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
const struct pcap_pkthdr *header,
const u_char *packet,
ndpi_risk *flow_risk);
ndpi_risk *flow_risk,
struct ndpi_flow_info **flow);
/* Flow callback for completed flows, before the flow memory will be freed. */
static inline void ndpi_workflow_set_flow_callback(struct ndpi_workflow * workflow, ndpi_workflow_callback_ptr callback, void * userdata) {
workflow->flow_callback = callback;
workflow->flow_callback_userdata = userdata;
}
int ndpi_is_datalink_supported(int datalink_type);
/* compare two nodes in workflow */
int ndpi_workflow_node_cmp(const void *a, const void *b);
void process_ndpi_collected_info(struct ndpi_workflow * workflow, struct ndpi_flow_info *flow);
u_int32_t ethernet_crc32(const void* data, size_t n_bytes);
void ndpi_flow_info_free_data(struct ndpi_flow_info *flow);
void ndpi_flow_info_freer(void *node);
const char* print_cipher_id(u_int32_t cipher);
float ndpi_flow_get_byte_count_entropy(const uint32_t byte_count[256], unsigned int num_bytes);
int parse_proto_name_list(char *str, NDPI_PROTOCOL_BITMASK *bitmask, int inverted_logic);
extern int nDPI_LogLevel;
extern int reader_log_level;
#ifdef NDPI_ENABLE_DEBUG_MESSAGES
#if defined(NDPI_ENABLE_DEBUG_MESSAGES) && !defined(FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION)
#define LOG(log_level, args...) \
{ \
if(log_level <= nDPI_LogLevel) \
if(log_level <= reader_log_level) \
printf(args); \
}
#else
@@ -414,4 +452,8 @@ extern int nDPI_LogLevel;
#define LINKTYPE_LINUX_SLL2 276
#endif
#ifdef __cplusplus
}
#endif
#endif
+3 -1
View File
@@ -7,6 +7,8 @@
################################################################
#
# Listingdate,SHA1,Listingreason
#The first entry (and only the first one) is fake. It is used to trigger the risk in the tests
1970-01-01 00:00:00,0DDB34F875632C7E1EC09D75827F82D2336DFEB6, Fake
2022-07-02 07:02:11,3784422d67a6ae0c82d60eeaf3850960a9896cb2,AsyncRAT C&C
2022-07-01 18:28:10,a0d78b3ddb27cd4f110a0292cab800beadaf9bd2,AsyncRAT C&C
2022-07-01 09:44:14,4c319ec8b5a66a256a0137d70cd0f9a76bb23fd7,AsyncRAT C&C
@@ -4878,4 +4880,4 @@
2014-05-04 08:10:31,86d6aade4ba1414a91b1e7fb3cdd7d503692f410,Shylock C&C
2014-05-04 08:10:26,5afc236d1dd00c9c45457b75226b501b815a59c7,Shylock C&C
2014-05-04 08:09:56,b08a4939fb88f375a2757eaddc47b1fb8b554439,Shylock C&C
# END (4871) entries
# END (4871) entries
1 ################################################################
7 ################################################################
8 #
9 # Listingdate,SHA1,Listingreason
10 #The first entry (and only the first one) is fake. It is used to trigger the risk in the tests
11 1970-01-01 00:00:00,0DDB34F875632C7E1EC09D75827F82D2336DFEB6, Fake
12 2022-07-02 07:02:11,3784422d67a6ae0c82d60eeaf3850960a9896cb2,AsyncRAT C&C
13 2022-07-01 18:28:10,a0d78b3ddb27cd4f110a0292cab800beadaf9bd2,AsyncRAT C&C
14 2022-07-01 09:44:14,4c319ec8b5a66a256a0137d70cd0f9a76bb23fd7,AsyncRAT C&C
4880 2014-05-04 08:10:31,86d6aade4ba1414a91b1e7fb3cdd7d503692f410,Shylock C&C
4881 2014-05-04 08:10:26,5afc236d1dd00c9c45457b75226b501b815a59c7,Shylock C&C
4882 2014-05-04 08:09:56,b08a4939fb88f375a2757eaddc47b1fb8b554439,Shylock C&C
4883 # END (4871) entries
+618 -71
View File
@@ -1,8 +1,18 @@
bin_PROGRAMS = fuzz_process_packet fuzz_ndpi_reader fuzz_ndpi_reader_alloc_fail fuzz_quic_get_crypto_data fuzz_config fuzz_community_id fuzz_serialization
bin_PROGRAMS = fuzz_process_packet fuzz_ndpi_reader fuzz_ndpi_reader_alloc_fail fuzz_ndpi_reader_payload_analyzer fuzz_quic_get_crypto_data fuzz_config fuzz_community_id fuzz_serialization fuzz_tls_certificate fuzz_dga fuzz_is_stun_udp fuzz_is_stun_tcp
#Alghoritms
bin_PROGRAMS += fuzz_alg_bins fuzz_alg_hll fuzz_alg_hw_rsi_outliers_da fuzz_alg_jitter fuzz_alg_ses_des
bin_PROGRAMS += fuzz_alg_bins fuzz_alg_hll fuzz_alg_hw_rsi_outliers_da fuzz_alg_jitter fuzz_alg_ses_des fuzz_alg_crc32_md5 fuzz_alg_bytestream fuzz_alg_shoco
#Data structures
bin_PROGRAMS += fuzz_ds_patricia fuzz_ds_ahocorasick fuzz_ds_libcache fuzz_ds_tree
bin_PROGRAMS += fuzz_ds_patricia fuzz_ds_ahocorasick fuzz_ds_libcache fuzz_ds_tree fuzz_ds_ptree fuzz_ds_hash fuzz_ds_cmsketch fuzz_ds_bitmap64_fuse fuzz_ds_domain_classify
#Third party
bin_PROGRAMS += fuzz_libinjection fuzz_binaryfusefilter
#Internal crypto
bin_PROGRAMS += fuzz_gcrypt_light fuzz_gcrypt_aes fuzz_gcrypt_gcm fuzz_gcrypt_cipher
#Configuration files
bin_PROGRAMS += fuzz_filecfg_protocols fuzz_filecfg_categories fuzz_filecfg_malicious_sha1 fuzz_filecfg_malicious_ja3 fuzz_filecfg_risk_domains fuzz_filecfg_config fuzz_filecfg_category
#Reader utils
bin_PROGRAMS += fuzz_readerutils_workflow fuzz_readerutils_parseprotolist
#Mutators
bin_PROGRAMS += fuzz_ndpi_reader_pl7m fuzz_ndpi_reader_pl7m_64k fuzz_ndpi_reader_pl7m_simplest fuzz_ndpi_reader_pl7m_internal fuzz_ndpi_reader_pl7m_simplest_internal
fuzz_process_packet_SOURCES = fuzz_process_packet.c fuzz_common_code.c
fuzz_process_packet_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
@@ -17,7 +27,7 @@ fuzz_process_packet_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_process_packet_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_SOURCES = fuzz_ndpi_reader.c ../example/reader_util.c
fuzz_ndpi_reader_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c
fuzz_ndpi_reader_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ndpi_reader_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_LDFLAGS = $(PCAP_LIB) $(LIBS)
@@ -31,7 +41,7 @@ fuzz_ndpi_reader_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(fuzz_ndpi_reader_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_alloc_fail_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c
fuzz_ndpi_reader_alloc_fail_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -DENABLE_MEM_ALLOC_FAILURES
fuzz_ndpi_reader_alloc_fail_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -DENABLE_MEM_ALLOC_FAILURES -DCRYPT_FORCE_NO_AESNI
fuzz_ndpi_reader_alloc_fail_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_alloc_fail_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
@@ -43,8 +53,21 @@ fuzz_ndpi_reader_alloc_fail_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAG
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_alloc_fail_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_payload_analyzer_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c
fuzz_ndpi_reader_payload_analyzer_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -DENABLE_MEM_ALLOC_FAILURES -DENABLE_PAYLOAD_ANALYZER
fuzz_ndpi_reader_payload_analyzer_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_payload_analyzer_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_payload_analyzer_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_payload_analyzer_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_payload_analyzer_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_payload_analyzer_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_quic_get_crypto_data_SOURCES = fuzz_quic_get_crypto_data.c fuzz_common_code.c
fuzz_quic_get_crypto_data_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_quic_get_crypto_data_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_quic_get_crypto_data_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_quic_get_crypto_data_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
@@ -57,8 +80,8 @@ fuzz_quic_get_crypto_data_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS)
$(fuzz_quic_get_crypto_data_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_config_SOURCES = fuzz_config.cpp fuzz_common_code.c
fuzz_config_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_config_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_config_CXXFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_config_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_config_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_config_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
@@ -72,8 +95,8 @@ fuzz_config_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(fuzz_config_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_community_id_SOURCES = fuzz_community_id.cpp fuzz_common_code.c
fuzz_community_id_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_community_id_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_community_id_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_community_id_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_community_id_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_community_id_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
@@ -87,8 +110,8 @@ fuzz_community_id_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(fuzz_community_id_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_serialization_SOURCES = fuzz_serialization.cpp fuzz_common_code.c
fuzz_serialization_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_serialization_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_serialization_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_serialization_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_serialization_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_serialization_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
@@ -102,8 +125,8 @@ fuzz_serialization_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(fuzz_serialization_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_bins_SOURCES = fuzz_alg_bins.cpp fuzz_common_code.c
fuzz_alg_bins_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_alg_bins_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_alg_bins_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_bins_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_bins_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_bins_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
@@ -117,8 +140,8 @@ fuzz_alg_bins_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(fuzz_alg_bins_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_hll_SOURCES = fuzz_alg_hll.cpp fuzz_common_code.c
fuzz_alg_hll_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_alg_hll_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_alg_hll_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_hll_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_hll_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_hll_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
@@ -132,8 +155,8 @@ fuzz_alg_hll_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(fuzz_alg_hll_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_hw_rsi_outliers_da_SOURCES = fuzz_alg_hw_rsi_outliers_da.cpp fuzz_common_code.c
fuzz_alg_hw_rsi_outliers_da_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_alg_hw_rsi_outliers_da_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_alg_hw_rsi_outliers_da_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_hw_rsi_outliers_da_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_hw_rsi_outliers_da_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_hw_rsi_outliers_da_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
@@ -147,8 +170,8 @@ fuzz_alg_hw_rsi_outliers_da_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAG
$(fuzz_alg_hw_rsi_outliers_da_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_jitter_SOURCES = fuzz_alg_jitter.cpp fuzz_common_code.c
fuzz_alg_jitter_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_alg_jitter_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_alg_jitter_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_jitter_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_jitter_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_jitter_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
@@ -161,9 +184,48 @@ fuzz_alg_jitter_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_jitter_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_crc32_md5_SOURCES = fuzz_alg_crc32_md5.c
fuzz_alg_crc32_md5_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_alg_crc32_md5_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_crc32_md5_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_crc32_md5_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_crc32_md5_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_crc32_md5_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_crc32_md5_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_bytestream_SOURCES = fuzz_alg_bytestream.c
fuzz_alg_bytestream_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_bytestream_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_bytestream_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_bytestream_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_bytestream_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_bytestream_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_bytestream_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_shoco_SOURCES = fuzz_alg_shoco.cpp
fuzz_alg_shoco_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_alg_shoco_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_shoco_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_shoco_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_shoco_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_shoco_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_shoco_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_ses_des_SOURCES = fuzz_alg_ses_des.cpp fuzz_common_code.c
fuzz_alg_ses_des_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_alg_ses_des_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_alg_ses_des_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_ses_des_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_ses_des_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_ses_des_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
@@ -177,8 +239,8 @@ fuzz_alg_ses_des_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(fuzz_alg_ses_des_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_patricia_SOURCES = fuzz_ds_patricia.cpp fuzz_common_code.c
fuzz_ds_patricia_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_ds_patricia_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_ds_patricia_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_patricia_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_patricia_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_patricia_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
@@ -192,8 +254,8 @@ fuzz_ds_patricia_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(fuzz_ds_patricia_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_ahocorasick_SOURCES = fuzz_ds_ahocorasick.cpp fuzz_common_code.c
fuzz_ds_ahocorasick_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_ds_ahocorasick_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_ds_ahocorasick_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_ahocorasick_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_ahocorasick_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_ahocorasick_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
@@ -207,8 +269,8 @@ fuzz_ds_ahocorasick_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(fuzz_ds_ahocorasick_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_libcache_SOURCES = fuzz_ds_libcache.cpp fuzz_common_code.c
fuzz_ds_libcache_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_ds_libcache_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_ds_libcache_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_libcache_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_libcache_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_libcache_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
@@ -222,8 +284,8 @@ fuzz_ds_libcache_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(fuzz_ds_libcache_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_tree_SOURCES = fuzz_ds_tree.cpp fuzz_common_code.c
fuzz_ds_tree_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_ds_tree_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION
fuzz_ds_tree_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_tree_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_tree_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_tree_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
@@ -236,83 +298,545 @@ fuzz_ds_tree_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_tree_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_ptree_SOURCES = fuzz_ds_ptree.cpp fuzz_common_code.c
fuzz_ds_ptree_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_ptree_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_ptree_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_ptree_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_ptree_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_ptree_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_ptree_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_ptree_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_ptree_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_hash_SOURCES = fuzz_ds_hash.cpp fuzz_common_code.c
fuzz_ds_hash_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_hash_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_hash_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_hash_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_hash_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_hash_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_hash_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_hash_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_hash_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_cmsketch_SOURCES = fuzz_ds_cmsketch.cpp fuzz_common_code.c
fuzz_ds_cmsketch_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_cmsketch_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_cmsketch_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_cmsketch_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_cmsketch_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_cmsketch_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_cmsketch_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_cmsketch_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_cmsketch_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_bitmap64_fuse_SOURCES = fuzz_ds_bitmap64_fuse.cpp fuzz_common_code.c
fuzz_ds_bitmap64_fuse_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_bitmap64_fuse_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_bitmap64_fuse_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_bitmap64_fuse_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_bitmap64_fuse_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_bitmap64_fuse_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_bitmap64_fuse_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_bitmap64_fuse_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_bitmap64_fuse_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_domain_classify_SOURCES = fuzz_ds_domain_classify.cpp fuzz_common_code.c
fuzz_ds_domain_classify_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_domain_classify_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_domain_classify_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_domain_classify_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_domain_classify_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_domain_classify_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_domain_classify_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_domain_classify_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_domain_classify_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_libinjection_SOURCES = fuzz_libinjection.c
fuzz_libinjection_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_libinjection_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_libinjection_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_libinjection_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_libinjection_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_libinjection_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_libinjection_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_binaryfusefilter_SOURCES = fuzz_binaryfusefilter.cpp fuzz_common_code.c
fuzz_binaryfusefilter_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_binaryfusefilter_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_binaryfusefilter_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_binaryfusefilter_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_binaryfusefilter_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_binaryfusefilter_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_binaryfusefilter_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_binaryfusefilter_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_binaryfusefilter_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_tls_certificate_SOURCES = fuzz_tls_certificate.c fuzz_common_code.c
fuzz_tls_certificate_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_tls_certificate_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_tls_certificate_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_tls_certificate_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_tls_certificate_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_tls_certificate_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_tls_certificate_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_dga_SOURCES = fuzz_dga.c fuzz_common_code.c
fuzz_dga_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_dga_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_dga_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_dga_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_dga_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_dga_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_dga_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_is_stun_udp_SOURCES = fuzz_is_stun.c fuzz_common_code.c
fuzz_is_stun_udp_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_is_stun_udp_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_is_stun_udp_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_is_stun_udp_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_is_stun_udp_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_is_stun_udp_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_is_stun_udp_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_is_stun_tcp_SOURCES = fuzz_is_stun.c fuzz_common_code.c
fuzz_is_stun_tcp_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION -DSTUN_TCP
fuzz_is_stun_tcp_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_is_stun_tcp_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_is_stun_tcp_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_is_stun_tcp_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_is_stun_tcp_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_is_stun_tcp_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_gcrypt_light_SOURCES = fuzz_gcrypt_light.cpp fuzz_common_code.c
fuzz_gcrypt_light_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_light_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_light_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_gcrypt_light_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_gcrypt_light_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_light_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_light_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_gcrypt_light_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_gcrypt_light_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_gcrypt_aes_SOURCES = fuzz_gcrypt_aes.cpp
fuzz_gcrypt_aes_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_aes_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_aes_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_gcrypt_aes_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_gcrypt_aes_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_aes_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_aes_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_gcrypt_aes_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_gcrypt_aes_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_gcrypt_gcm_SOURCES = fuzz_gcrypt_gcm.cpp
fuzz_gcrypt_gcm_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_gcm_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_gcm_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_gcrypt_gcm_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_gcrypt_gcm_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_gcm_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_gcm_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_gcrypt_gcm_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_gcrypt_gcm_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_gcrypt_cipher_SOURCES = fuzz_gcrypt_cipher.cpp
fuzz_gcrypt_cipher_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_cipher_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_cipher_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_gcrypt_cipher_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_gcrypt_cipher_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_cipher_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_cipher_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_gcrypt_cipher_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_gcrypt_cipher_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_protocols_SOURCES = fuzz_filecfg_protocols.c fuzz_common_code.c
fuzz_filecfg_protocols_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_protocols_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_protocols_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_protocols_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_protocols_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_protocols_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_protocols_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_categories_SOURCES = fuzz_filecfg_categories.c fuzz_common_code.c
fuzz_filecfg_categories_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_categories_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_categories_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_categories_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_categories_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_categories_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_categories_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_malicious_sha1_SOURCES = fuzz_filecfg_malicious_sha1.c fuzz_common_code.c
fuzz_filecfg_malicious_sha1_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_malicious_sha1_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_malicious_sha1_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_malicious_sha1_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_malicious_sha1_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_malicious_sha1_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_malicious_sha1_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_malicious_ja3_SOURCES = fuzz_filecfg_malicious_ja3.c fuzz_common_code.c
fuzz_filecfg_malicious_ja3_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_malicious_ja3_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_malicious_ja3_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_malicious_ja3_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_malicious_ja3_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_malicious_ja3_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_malicious_ja3_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_risk_domains_SOURCES = fuzz_filecfg_risk_domains.c fuzz_common_code.c
fuzz_filecfg_risk_domains_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_risk_domains_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_risk_domains_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_risk_domains_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_risk_domains_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_risk_domains_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_risk_domains_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_config_SOURCES = fuzz_filecfg_config.c fuzz_common_code.c
fuzz_filecfg_config_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_config_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_config_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_config_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_config_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_config_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_config_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_category_SOURCES = fuzz_filecfg_category.c fuzz_common_code.c
fuzz_filecfg_category_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_category_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_category_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_category_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_category_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_category_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_category_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_readerutils_workflow_SOURCES = fuzz_readerutils_workflow.cpp fuzz_common_code.c ../example/reader_util.c
fuzz_readerutils_workflow_CXXFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_readerutils_workflow_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_readerutils_workflow_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_readerutils_workflow_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_readerutils_workflow_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_readerutils_workflow_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_readerutils_workflow_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_readerutils_workflow_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_readerutils_workflow_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_readerutils_parseprotolist_SOURCES = fuzz_readerutils_parseprotolist.cpp fuzz_common_code.c ../example/reader_util.c
fuzz_readerutils_parseprotolist_CXXFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_readerutils_parseprotolist_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_readerutils_parseprotolist_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_readerutils_parseprotolist_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_readerutils_parseprotolist_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_readerutils_parseprotolist_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_readerutils_parseprotolist_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_readerutils_parseprotolist_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_readerutils_parseprotolist_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_pl7m_simplest_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c ../src/lib/third_party/src/fuzz/pl7m.c
fuzz_ndpi_reader_pl7m_simplest_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -I../src/lib/third_party/include/ -DENABLE_PCAP_L7_MUTATOR -DPL7M_USE_SIMPLEST_MUTATOR
fuzz_ndpi_reader_pl7m_simplest_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_pl7m_simplest_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_pl7m_simplest_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_pl7m_simplest_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_pl7m_simplest_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_pl7m_simplest_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_pl7m_simplest_internal_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c ../src/lib/third_party/src/fuzz/pl7m.c
fuzz_ndpi_reader_pl7m_simplest_internal_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -I../src/lib/third_party/include/ -DENABLE_PCAP_L7_MUTATOR -DPL7M_USE_SIMPLEST_MUTATOR -DPL7M_USE_INTERNAL_FUZZER_MUTATE
fuzz_ndpi_reader_pl7m_simplest_internal_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_pl7m_simplest_internal_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_pl7m_simplest_internal_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_pl7m_simplest_internal_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_pl7m_simplest_internal_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_pl7m_simplest_internal_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_pl7m_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c ../src/lib/third_party/src/fuzz/pl7m.c
fuzz_ndpi_reader_pl7m_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -I../src/lib/third_party/include/ -DENABLE_PCAP_L7_MUTATOR
fuzz_ndpi_reader_pl7m_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_pl7m_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_pl7m_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_pl7m_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_pl7m_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_pl7m_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_pl7m_64k_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c ../src/lib/third_party/src/fuzz/pl7m.c
fuzz_ndpi_reader_pl7m_64k_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -I../src/lib/third_party/include/ -DENABLE_PCAP_L7_MUTATOR -DPL7M_USE_64K_PACKETS
fuzz_ndpi_reader_pl7m_64k_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_pl7m_64k_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_pl7m_64k_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_pl7m_64k_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_pl7m_64k_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_pl7m_64k_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_pl7m_internal_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c ../src/lib/third_party/src/fuzz/pl7m.c
fuzz_ndpi_reader_pl7m_internal_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -I../src/lib/third_party/include/ -DENABLE_PCAP_L7_MUTATOR -DPL7M_USE_INTERNAL_FUZZER_MUTATE
fuzz_ndpi_reader_pl7m_internal_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_pl7m_internal_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_pl7m_internal_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_pl7m_internal_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_pl7m_internal_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_pl7m_internal_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
# required for Google oss-fuzz
# see https://github.com/google/oss-fuzz/tree/master/projects/ndpi
testpcaps := $(wildcard ../tests/pcap/*.pcap*)
testpcaps := $(wildcard ../tests/cfgs/default/pcap/*.pcap*)
fuzz_ndpi_reader_seed_corpus.zip: $(testpcaps)
zip -j fuzz_ndpi_reader_seed_corpus.zip $(testpcaps)
testpcaps.zip: $(testpcaps)
zip -j testpcaps.zip $(testpcaps)
fuzz_ndpi_reader_alloc_fail_seed_corpus.zip: $(testpcaps)
zip -j fuzz_ndpi_reader_alloc_fail_seed_corpus.zip $(testpcaps)
fuzz_ndpi_reader_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_seed_corpus.zip
fuzz_ndpi_reader_alloc_fail_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_alloc_fail_seed_corpus.zip
fuzz_ndpi_reader_payload_analyzer_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_payload_analyzer_seed_corpus.zip
fuzz_ndpi_reader_pl7m_simplest_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_pl7m_simplest_seed_corpus.zip
fuzz_ndpi_reader_pl7m_simplest_internal_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_pl7m_simplest_internal_seed_corpus.zip
fuzz_ndpi_reader_pl7m_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_pl7m_seed_corpus.zip
fuzz_ndpi_reader_pl7m_64k_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_pl7m_64k_seed_corpus.zip
fuzz_ndpi_reader_pl7m_internal_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_pl7m_internal_seed_corpus.zip
files_corpus_fuzz_quic_get_crypto_data := $(wildcard corpus/fuzz_quic_get_crypto_data/*)
fuzz_quic_get_crypto_data_seed_corpus.zip: $(files_corpus_fuzz_quic_get_crypto_data)
zip -j fuzz_quic_get_crypto_data_seed_corpus.zip $(files_corpus_fuzz_quic_get_crypto_data)
files_corpus_fuzz_config := $(wildcard corpus/fuzz_config/*)
fuzz_config_seed_corpus.zip: $(files_corpus_fuzz_config)
zip -j fuzz_config_seed_corpus.zip $(files_corpus_fuzz_config)
files_corpus_fuzz_serialization := $(wildcard corpus/fuzz_serialization/*)
fuzz_serialization_seed_corpus.zip: $(files_corpus_fuzz_serialization)
zip -j fuzz_serialization_seed_corpus.zip $(files_corpus_fuzz_serialization)
files_corpus_fuzz_community_id := $(wildcard corpus/fuzz_community_id/*)
fuzz_community_id_seed_corpus.zip: $(files_corpus_fuzz_community_id)
zip -j fuzz_community_id_seed_corpus.zip $(files_corpus_fuzz_community_id)
files_corpus_fuzz_ds_patricia := $(wildcard corpus/fuzz_ds_patricia/*)
files_corpus_fuzz_is_stun_udp := $(wildcard corpus/fuzz_is_stun_udp/*)
fuzz_is_stun_udp_seed_corpus.zip: $(files_corpus_fuzz_is_stun_udp)
zip -j fuzz_is_stun_udp_seed_corpus.zip $(files_corpus_fuzz_is_stun_udp)
fuzz_ds_patricia_seed_corpus.zip: $(files_corpus_fuzz_ds_patricia)
zip -j fuzz_ds_patricia_seed_corpus.zip $(files_corpus_fuzz_ds_patricia)
files_corpus_fuzz_ds_ahocorasick := $(wildcard corpus/fuzz_ds_ahocorasick/*)
fuzz_ds_ahocorasick_seed_corpus.zip: $(files_corpus_fuzz_ds_ahocorasick)
zip -j fuzz_ds_ahocorasick_seed_corpus.zip $(files_corpus_fuzz_ds_ahocorasick)
files_corpus_fuzz_is_stun_tcp := $(wildcard corpus/fuzz_is_stun_tcp/*)
fuzz_is_stun_tcp_seed_corpus.zip: $(files_corpus_fuzz_is_stun_tcp)
zip -j fuzz_is_stun_tcp_seed_corpus.zip $(files_corpus_fuzz_is_stun_tcp)
files_corpus_fuzz_alg_ses_des := $(wildcard corpus/fuzz_alg_ses_des/*)
fuzz_alg_ses_des_seed_corpus.zip: $(files_corpus_fuzz_alg_ses_des)
zip -j fuzz_alg_ses_des_seed_corpus.zip $(files_corpus_fuzz_alg_ses_des)
files_corpus_fuzz_alg_hw_rsi_outliers_da := $(wildcard corpus/fuzz_alg_hw_rsi_outliers_da/*)
fuzz_alg_hw_rsi_outliers_da_seed_corpus.zip: $(files_corpus_fuzz_alg_hw_rsi_outliers_da)
zip -j fuzz_alg_hw_rsi_outliers_da_seed_corpus.zip $(files_corpus_fuzz_alg_hw_rsi_outliers_da)
files_corpus_fuzz_alg_bins := $(wildcard corpus/fuzz_alg_bins/*)
fuzz_alg_bins_seed_corpus.zip: $(files_corpus_fuzz_alg_bins)
zip -j fuzz_alg_bins_seed_corpus.zip $(files_corpus_fuzz_alg_bins)
files_corpus_fuzz_alg_hll := $(wildcard corpus/fuzz_alg_hll/*)
fuzz_alg_hll_seed_corpus.zip: $(files_corpus_fuzz_alg_hll)
zip -j fuzz_alg_hll_seed_corpus.zip $(files_corpus_fuzz_alg_hll)
files_corpus_fuzz_alg_jitter := $(wildcard corpus/fuzz_alg_jitter/*)
fuzz_alg_jitter_seed_corpus.zip: $(files_corpus_fuzz_alg_jitter)
zip -j fuzz_alg_jitter_seed_corpus.zip $(files_corpus_fuzz_alg_jitter)
files_corpus_fuzz_ds_libcache := $(wildcard corpus/fuzz_ds_libcache/*)
files_corpus_fuzz_alg_crc32_md5 := $(wildcard corpus/fuzz_alg_crc32_md5/*)
fuzz_alg_crc32_md5_seed_corpus.zip: $(files_corpus_fuzz_alg_crc32_md5)
zip -j fuzz_alg_crc32_md5_seed_corpus.zip $(files_corpus_fuzz_alg_crc32_md5)
files_corpus_fuzz_ds_libcache := $(wildcard corpus/fuzz_ds_libcache/*)
fuzz_ds_libcache_seed_corpus.zip: $(files_corpus_fuzz_ds_libcache)
zip -j fuzz_ds_libcache_seed_corpus.zip $(files_corpus_fuzz_ds_libcache)
files_corpus_fuzz_ds_tree := $(wildcard corpus/fuzz_ds_tree/*)
files_corpus_fuzz_ds_ptree := $(wildcard corpus/fuzz_ds_ptree/*)
fuzz_ds_ptree_seed_corpus.zip: $(files_corpus_fuzz_ds_ptree)
zip -j fuzz_ds_ptree_seed_corpus.zip $(files_corpus_fuzz_ds_ptree)
fuzz_ds_tree_seed_corpus.zip: $(files_corpus_fuzz_ds_tree)
zip -j fuzz_ds_tree_seed_corpus.zip $(files_corpus_fuzz_ds_tree)
files_corpus_fuzz_alg_bytestream := $(wildcard corpus/fuzz_alg_bytestream/*)
fuzz_alg_bytestream_seed_corpus.zip: $(files_corpus_fuzz_alg_bytestream)
zip -j fuzz_alg_bytestream_seed_corpus.zip $(files_corpus_fuzz_alg_bytestream)
corpus: fuzz_ndpi_reader_seed_corpus.zip fuzz_ndpi_reader_alloc_fail_seed_corpus.zip fuzz_quic_get_crypto_data_seed_corpus.zip fuzz_config_seed_corpus.zip fuzz_ds_patricia_seed_corpus.zip fuzz_ds_ahocorasick_seed_corpus.zip fuzz_alg_ses_des_seed_corpus.zip fuzz_alg_hw_rsi_outliers_da_seed_corpus.zip fuzz_alg_bins_seed_corpus.zip fuzz_alg_hll_seed_corpus.zip fuzz_alg_jitter_seed_corpus.zip fuzz_ds_libcache_seed_corpus.zip fuzz_community_id_seed_corpus.zip fuzz_ds_tree_seed_corpus.zip fuzz_serialization_seed_corpus.zip
files_corpus_fuzz_libinjection := $(wildcard corpus/fuzz_libinjection/*)
fuzz_libinjection_seed_corpus.zip: $(files_corpus_fuzz_libinjection)
zip -j fuzz_libinjection_seed_corpus.zip $(files_corpus_fuzz_libinjection)
files_corpus_fuzz_tls_certificate := $(wildcard corpus/fuzz_tls_certificate/*)
fuzz_tls_certificate_seed_corpus.zip: $(files_corpus_fuzz_tls_certificate)
zip -j fuzz_tls_certificate_seed_corpus.zip $(files_corpus_fuzz_tls_certificate)
files_corpus_fuzz_filecfg_protocols := $(wildcard corpus/fuzz_filecfg_protocols/*)
fuzz_filecfg_protocols_seed_corpus.zip: $(files_corpus_fuzz_filecfg_protocols)
zip -j fuzz_filecfg_protocols_seed_corpus.zip $(files_corpus_fuzz_filecfg_protocols)
files_corpus_fuzz_filecfg_categories := $(wildcard corpus/fuzz_filecfg_categories/*)
fuzz_filecfg_categories_seed_corpus.zip: $(files_corpus_fuzz_filecfg_categories)
zip -j fuzz_filecfg_categories_seed_corpus.zip $(files_corpus_fuzz_filecfg_categories)
files_corpus_fuzz_filecfg_malicious_sha1 := $(wildcard corpus/fuzz_filecfg_malicious_sha1/*)
fuzz_filecfg_malicious_sha1_seed_corpus.zip: $(files_corpus_fuzz_filecfg_malicious_sha1)
zip -j fuzz_filecfg_malicious_sha1_seed_corpus.zip $(files_corpus_fuzz_filecfg_malicious_sha1)
files_corpus_fuzz_filecfg_malicious_ja3 := $(wildcard corpus/fuzz_filecfg_malicious_ja3/*)
fuzz_filecfg_malicious_ja3_seed_corpus.zip: $(files_corpus_fuzz_filecfg_malicious_ja3)
zip -j fuzz_filecfg_malicious_ja3_seed_corpus.zip $(files_corpus_fuzz_filecfg_malicious_ja3)
files_corpus_fuzz_filecfg_risk_domains := $(wildcard corpus/fuzz_filecfg_risk_domains/*)
fuzz_filecfg_risk_domains_seed_corpus.zip: $(files_corpus_fuzz_filecfg_risk_domains)
zip -j fuzz_filecfg_risk_domains_seed_corpus.zip $(files_corpus_fuzz_filecfg_risk_domains)
files_corpus_fuzz_filecfg_config := $(wildcard corpus/fuzz_filecfg_config/*)
fuzz_filecfg_config_seed_corpus.zip: $(files_corpus_fuzz_filecfg_config)
zip -j fuzz_filecfg_config_seed_corpus.zip $(files_corpus_fuzz_filecfg_config)
files_corpus_fuzz_filecfg_category := $(wildcard corpus/fuzz_filecfg_category/*)
fuzz_filecfg_category_seed_corpus.zip: $(files_corpus_fuzz_filecfg_category)
zip -j fuzz_filecfg_category_seed_corpus.zip $(files_corpus_fuzz_filecfg_category)
files_corpus_fuzz_readerutils_workflow := $(wildcard corpus/fuzz_readerutils_workflow/*)
fuzz_readerutils_workflow_seed_corpus.zip: $(files_corpus_fuzz_readerutils_workflow)
zip -j fuzz_readerutils_workflow_seed_corpus.zip $(files_corpus_fuzz_readerutils_workflow)
files_corpus_fuzz_readerutils_parseprotolist := $(wildcard corpus/fuzz_readerutils_parseprotolist/*)
fuzz_readerutils_parseprotolist_seed_corpus.zip: $(files_corpus_fuzz_readerutils_parseprotolist)
zip -j fuzz_readerutils_parseprotolist_seed_corpus.zip $(files_corpus_fuzz_readerutils_parseprotolist)
files_corpus_fuzz_ds_bitmap64_fuse := $(wildcard corpus/fuzz_ds_bitmap64_fuse/*)
fuzz_ds_bitmap64_fuse_seed_corpus.zip: $(files_corpus_fuzz_ds_bitmap64_fuse)
zip -j fuzz_ds_bitmap64_fuse_seed_corpus.zip $(files_corpus_fuzz_ds_bitmap64_fuse)
files_corpus_fuzz_ds_domain_classify := $(wildcard corpus/fuzz_ds_domain_classify/*)
fuzz_ds_domain_classify_seed_corpus.zip: $(files_corpus_fuzz_ds_domain_classify)
zip -j fuzz_ds_domain_classify_seed_corpus.zip $(files_corpus_fuzz_ds_domain_classify)
corpus: fuzz_ndpi_reader_seed_corpus.zip fuzz_ndpi_reader_alloc_fail_seed_corpus.zip fuzz_ndpi_reader_payload_analyzer_seed_corpus.zip fuzz_quic_get_crypto_data_seed_corpus.zip fuzz_alg_ses_des_seed_corpus.zip fuzz_alg_bins_seed_corpus.zip fuzz_alg_hll_seed_corpus.zip fuzz_alg_jitter_seed_corpus.zip fuzz_ds_libcache_seed_corpus.zip fuzz_community_id_seed_corpus.zip fuzz_serialization_seed_corpus.zip fuzz_ds_ptree_seed_corpus.zip fuzz_alg_crc32_md5_seed_corpus.zip fuzz_alg_bytestream_seed_corpus.zip fuzz_libinjection_seed_corpus.zip fuzz_tls_certificate_seed_corpus.zip fuzz_filecfg_protocols_seed_corpus.zip fuzz_readerutils_workflow_seed_corpus.zip fuzz_readerutils_parseprotolist_seed_corpus.zip fuzz_ds_bitmap64_fuse_seed_corpus.zip fuzz_ds_domain_classify_seed_corpus.zip fuzz_filecfg_protocols_seed_corpus.zip fuzz_is_stun_udp_seed_corpus.zip fuzz_is_stun_tcp_seed_corpus.zip fuzz_ndpi_reader_pl7m_simplest_seed_corpus.zip fuzz_ndpi_reader_pl7m_seed_corpus.zip fuzz_ndpi_reader_pl7m_64k_seed_corpus.zip fuzz_ndpi_reader_pl7m_simplest_internal_seed_corpus.zip fuzz_ndpi_reader_pl7m_internal_seed_corpus.zip
cp corpus/fuzz_*seed_corpus.zip .
#Create dictionaries exactly as expected by oss-fuzz.
#This way, if we need to change/update/add something,
@@ -320,7 +844,9 @@ corpus: fuzz_ndpi_reader_seed_corpus.zip fuzz_ndpi_reader_alloc_fail_seed_corpus
dictionaries:
cp dictionary.dict fuzz_ndpi_reader.dict
cp dictionary.dict fuzz_ndpi_reader_alloc_fail.dict
cp dictionary.dict fuzz_ndpi_reader_payload_analyzer.dict
cp dictionary.dict fuzz_process_packet.dict
cp dictionary_tls_certificate.dict fuzz_tls_certificate.dict
distdir:
find . -type d | xargs -I'{}' mkdir -p '$(distdir)/{}'
@@ -328,19 +854,40 @@ distdir:
-o -name '*.am' \
-o -name '*.h' \
-o -name '*.cpp' \
-o -name '*.dict' \
-o -name '*.options' \
-o -name 'ipv4_addresses.txt' \
-o -name 'ipv6_addresses.txt' \
-o -name 'bd_param.txt' \
-o -name 'splt_param.txt' \
-o -name 'random_list.list' \
-o -path './dictionary.dict' \
-o -path './dictionary_tls_certificate.dict' \
-o -path './corpus/fuzz_*.zip' \
-o -path './corpus/fuzz_quic_get_crypto_data/*' \
-o -path './corpus/fuzz_config/*' \
-o -path './corpus/fuzz_filecfg_protocols/*' \
-o -path './corpus/fuzz_filecfg_categories/*' \
-o -path './corpus/fuzz_filecfg_malicious_sha1/*' \
-o -path './corpus/fuzz_filecfg_malicious_ja3/*' \
-o -path './corpus/fuzz_filecfg_risk_domains/*' \
-o -path './corpus/fuzz_filecfg_config/*' \
-o -path './corpus/fuzz_filecfg_category/*' \
-o -path './corpus/fuzz_readerutils_workflow/*' \
-o -path './corpus/fuzz_readerutils_parseprotolist/*' \
-o -path './corpus/fuzz_is_stun_udp/*' \
-o -path './corpus/fuzz_is_stun_tcp/*' \
-o -path './corpus/fuzz_serialization/*' \
-o -path './corpus/fuzz_community_id/*' \
-o -path './corpus/fuzz_libinjection/*' \
-o -path './corpus/fuzz_tls_certificate/*' \
-o -path './corpus/fuzz_alg_ses_des/*' \
-o -path './corpus/fuzz_alg_bins/*' \
-o -path './corpus/fuzz_alg_hll/*' \
-o -path './corpus/fuzz_alg_jitter/*' \
-o -path './corpus/fuzz_alg_hw_rsi_outliers_da/*' \
-o -path './corpus/fuzz_ds_ahocorasick/*' \
-o -path './corpus/fuzz_alg_crc32_md5/*' \
-o -path './corpus/fuzz_alg_bytestream/*' \
-o -path './corpus/fuzz_ds_libcache/*' \
-o -path './corpus/fuzz_ds_tree/*' \
-o -path './corpus/fuzz_ds_patricia/*' | xargs -I'{}' cp -r '{}' '$(distdir)/{}'
-o -path './corpus/fuzz_ds_bitmap64_fuse/*' \
-o -path './corpus/fuzz_ds_domain_classify/*' \
-o -path './corpus/fuzz_ds_ptree/*' | xargs -I'{}' cp -r '{}' '$(distdir)/{}'
all: corpus dictionaries
+627
View File
@@ -0,0 +1,627 @@
-2.088057846500587456e+00
7.763936238952200239e-05
4.404309737393306595e-05
-9.467385027293546973e-02
4.348947142638090457e-01
-2.091409170053043390e-04
-5.788902107267982974e-04
4.481443450852441001e-10
-3.136135459023654537e+00
-1.507730262127600751e+00
-1.204663669965535977e+00
-1.171839254318371104e+00
4.329302247232582057e-01
8.310653628092458334e+00
3.299246725156660176e+00
0.000000000000000000e+00
1.847454931582027254e-02
-1.498024139966201096e+00
-7.660670007653060942e-01
-2.908130300830076731e+00
-1.252564844610269734e+00
-1.910955328742287573e+00
9.471710980110392697e-01
2.352302758516665371e+00
2.982269972214651954e+00
4.280736383314343918e+00
4.633629909719495288e+00
-2.198052637823726840e+00
-1.150759637168392580e+00
3.420433363184381292e+00
1.857878113059351077e-02
-1.559806674919653746e+00
4.197498183183401288e+00
6.262186949633183453e+00
1.100694844370524095e+01
2.778688785515088000e+01
3.679948298336883195e+00
-2.432801394376875592e+00
5.133442052706843617e-01
2.181172654073517680e+00
-8.577551729671881731e-01
7.013844214023315926e-01
3.138233436228588857e+00
7.319940508466630247e-01
0.000000000000000000e+00
3.529209394581482861e+00
1.464585117707144413e+01
8.506550226820598359e-01
-9.060397326548508268e-01
6.787474954688997641e+00
8.125411068867387954e+00
4.515740684104064151e+00
5.372135582950940069e+00
9.210951196799497254e-01
4.802177410869620466e+00
2.945445016176073594e+01
1.575032253128311632e+00
-1.355276854364796946e-01
-3.322474764169629502e-01
3.018397817188666732e+00
1.186503569922195744e+00
0.000000000000000000e+00
8.883242370198487503e-01
7.248276146728496627e+00
0.000000000000000000e+00
0.000000000000000000e+00
-4.831246718433664711e+00
6.124136970173365002e-01
4.145693892559814686e-01
2.683998941637626867e+00
2.063906603639539039e+00
2.989801217386735210e+00
2.262965767379551962e-01
2.240332214649647380e+00
5.984550782416063086e+00
4.587011255338186544e+00
1.233118485315272039e+01
1.115223490909697857e+00
-3.682686422016995476e+00
6.096498453291562258e-01
1.119275528656461516e+00
1.377886278915177731e-01
3.828176805973048324e+00
0.000000000000000000e+00
0.000000000000000000e+00
1.442927634029647344e+01
0.000000000000000000e+00
5.719118583309401593e-01
1.993632609731877392e-01
3.047472271520709430e+00
5.736784864911910198e+00
6.677826247219391220e+00
6.307175478564531090e+00
3.150295169417364249e+01
3.738597740702392258e+00
1.129754590514236234e+01
6.108506268573830056e+00
1.605489516792866667e+00
2.929631990348545489e+00
-2.832543082245212937e-02
1.358286530670594461e+00
1.655932469853677924e+00
6.701964773769768513e-01
2.131182050917533211e+00
2.998351165769753468e+00
7.772095996358327596e+00
1.285014785269981141e+00
4.407334784589962418e+00
1.719858214230612026e+00
-1.012765674651314063e+00
-5.749271123172469133e-01
-3.559614093795681278e+00
-3.073088477387719397e+00
-4.492469521371540431e+00
-3.753286990415885427e+00
-3.219255423324282273e+00
-2.806436518181075090e+00
-2.697305948568419875e+00
-7.879608430851776646e-01
4.625507221739111330e+00
4.809280703883450414e+00
-3.435194026629848629e+00
-3.218943068168937049e+00
3.335535704890596698e+00
2.071359212435486263e+00
4.538992059175040339e+00
-2.770772323566738038e+01
2.903047708571506735e+00
-4.436143805989154032e+00
-2.647991280011542381e-01
1.737252348126810064e+00
-4.121989655995259128e+00
3.209709099445720581e-01
1.012758514896711759e+01
3.313255624721038295e+00
4.631467619785444967e+00
7.668642402146534032e+00
6.780938812710099128e+00
-3.256164342602652972e+00
6.749565128319576779e-01
0.000000000000000000e+00
-4.407265954524525853e+00
0.000000000000000000e+00
-3.666522115024547901e+01
-7.886029397826226273e+01
0.000000000000000000e+00
0.000000000000000000e+00
-2.261283814517791058e+01
-4.024317426178160240e+00
3.213063737030031342e-01
5.079805145796887800e+00
1.326813226475260343e+00
1.233684078112145643e+00
8.671852503871454232e+00
-2.041800256066371944e+00
0.000000000000000000e+00
0.000000000000000000e+00
-1.607347800380474823e+01
-4.430790279223246309e+00
1.177552465851384511e+00
6.342921220500139512e+00
-2.466913734548706327e-02
3.451642566010713065e-01
-6.012767168531006234e+00
7.328146570137336724e+00
7.500088131707050465e+00
0.000000000000000000e+00
-3.547913249211809017e+01
-3.130964814607208879e+00
8.247326544297072237e-01
3.757262485775580418e-01
-2.136528302027558723e+00
-2.631627236037529793e-01
-2.016718799388414141e+01
0.000000000000000000e+00
0.000000000000000000e+00
0.000000000000000000e+00
-7.708602132869285528e-01
-2.602868328868111814e+00
1.435184800833797958e+00
0.000000000000000000e+00
-2.080420864280113413e+00
1.169498351211070819e+00
-1.798334115637199560e+01
-1.193885252696202670e+01
0.000000000000000000e+00
0.000000000000000000e+00
4.304089297965300709e+00
-3.020893216686394656e+00
-1.234427481614708721e+00
0.000000000000000000e+00
1.853340741926325697e+00
-2.686000064995862147e+01
-1.672275139058893600e+01
-2.826268691607605987e+01
0.000000000000000000e+00
0.000000000000000000e+00
-1.547397429377200817e+00
-4.018181657009961327e+00
-7.289186736637049968e+00
-7.458655219230571731e+00
-9.625538282761622710e+00
-1.103039457077456298e+01
-6.262675161142102809e+01
-9.265912629799268885e+00
-8.961543476816615339e+00
-9.622764435629340696e+00
-1.097978292092879826e+01
-1.097978292092879826e+01
-2.088057846500587456e+00
7.763936238952200239e-05
4.404309737393306595e-05
-9.467385027293546973e-02
4.348947142638090457e-01
-2.091409170053043390e-04
-5.788902107267982974e-04
4.481443450852441001e-10
-3.136135459023654537e+00
-1.507730262127600751e+00
-1.204663669965535977e+00
-1.171839254318371104e+00
4.329302247232582057e-01
8.310653628092458334e+00
3.299246725156660176e+00
0.000000000000000000e+00
1.847454931582027254e-02
-1.498024139966201096e+00
-7.660670007653060942e-01
-2.908130300830076731e+00
-1.252564844610269734e+00
-1.910955328742287573e+00
9.471710980110392697e-01
2.352302758516665371e+00
2.982269972214651954e+00
4.280736383314343918e+00
4.633629909719495288e+00
-2.198052637823726840e+00
-1.150759637168392580e+00
3.420433363184381292e+00
1.857878113059351077e-02
-1.559806674919653746e+00
4.197498183183401288e+00
6.262186949633183453e+00
1.100694844370524095e+01
2.778688785515088000e+01
3.679948298336883195e+00
-2.432801394376875592e+00
5.133442052706843617e-01
2.181172654073517680e+00
-8.577551729671881731e-01
7.013844214023315926e-01
3.138233436228588857e+00
7.319940508466630247e-01
0.000000000000000000e+00
3.529209394581482861e+00
1.464585117707144413e+01
8.506550226820598359e-01
-9.060397326548508268e-01
6.787474954688997641e+00
8.125411068867387954e+00
4.515740684104064151e+00
5.372135582950940069e+00
9.210951196799497254e-01
4.802177410869620466e+00
2.945445016176073594e+01
1.575032253128311632e+00
-1.355276854364796946e-01
-3.322474764169629502e-01
3.018397817188666732e+00
1.186503569922195744e+00
0.000000000000000000e+00
8.883242370198487503e-01
7.248276146728496627e+00
0.000000000000000000e+00
0.000000000000000000e+00
-4.831246718433664711e+00
6.124136970173365002e-01
4.145693892559814686e-01
2.683998941637626867e+00
2.063906603639539039e+00
2.989801217386735210e+00
2.262965767379551962e-01
2.240332214649647380e+00
5.984550782416063086e+00
4.587011255338186544e+00
1.233118485315272039e+01
1.115223490909697857e+00
-3.682686422016995476e+00
6.096498453291562258e-01
1.119275528656461516e+00
1.377886278915177731e-01
3.828176805973048324e+00
0.000000000000000000e+00
0.000000000000000000e+00
1.442927634029647344e+01
0.000000000000000000e+00
5.719118583309401593e-01
1.993632609731877392e-01
3.047472271520709430e+00
5.736784864911910198e+00
6.677826247219391220e+00
6.307175478564531090e+00
3.150295169417364249e+01
3.738597740702392258e+00
1.129754590514236234e+01
6.108506268573830056e+00
1.605489516792866667e+00
2.929631990348545489e+00
-2.832543082245212937e-02
1.358286530670594461e+00
1.655932469853677924e+00
6.701964773769768513e-01
2.131182050917533211e+00
2.998351165769753468e+00
7.772095996358327596e+00
1.285014785269981141e+00
4.407334784589962418e+00
1.719858214230612026e+00
-1.012765674651314063e+00
-5.749271123172469133e-01
-3.559614093795681278e+00
-3.073088477387719397e+00
-4.492469521371540431e+00
-3.753286990415885427e+00
-3.219255423324282273e+00
-2.806436518181075090e+00
-2.697305948568419875e+00
-7.879608430851776646e-01
4.625507221739111330e+00
4.809280703883450414e+00
-3.435194026629848629e+00
-3.218943068168937049e+00
3.335535704890596698e+00
2.071359212435486263e+00
4.538992059175040339e+00
-2.770772323566738038e+01
2.903047708571506735e+00
-4.436143805989154032e+00
-2.647991280011542381e-01
1.737252348126810064e+00
-4.121989655995259128e+00
3.209709099445720581e-01
1.012758514896711759e+01
3.313255624721038295e+00
4.631467619785444967e+00
7.668642402146534032e+00
6.780938812710099128e+00
-3.256164342602652972e+00
6.749565128319576779e-01
0.000000000000000000e+00
-4.407265954524525853e+00
0.000000000000000000e+00
-3.666522115024547901e+01
-7.886029397826226273e+01
0.000000000000000000e+00
0.000000000000000000e+00
-2.261283814517791058e+01
-4.024317426178160240e+00
3.213063737030031342e-01
5.079805145796887800e+00
1.326813226475260343e+00
1.233684078112145643e+00
8.671852503871454232e+00
-2.041800256066371944e+00
0.000000000000000000e+00
0.000000000000000000e+00
-1.607347800380474823e+01
-4.430790279223246309e+00
1.177552465851384511e+00
6.342921220500139512e+00
-2.466913734548706327e-02
3.451642566010713065e-01
-6.012767168531006234e+00
7.328146570137336724e+00
7.500088131707050465e+00
0.000000000000000000e+00
-3.547913249211809017e+01
-3.130964814607208879e+00
8.247326544297072237e-01
3.757262485775580418e-01
-2.136528302027558723e+00
-2.631627236037529793e-01
-2.016718799388414141e+01
0.000000000000000000e+00
0.000000000000000000e+00
0.000000000000000000e+00
-7.708602132869285528e-01
-2.602868328868111814e+00
1.435184800833797958e+00
0.000000000000000000e+00
-2.080420864280113413e+00
1.169498351211070819e+00
-1.798334115637199560e+01
-1.193885252696202670e+01
0.000000000000000000e+00
0.000000000000000000e+00
4.304089297965300709e+00
-3.020893216686394656e+00
-1.234427481614708721e+00
0.000000000000000000e+00
1.853340741926325697e+00
-2.686000064995862147e+01
-1.672275139058893600e+01
-2.826268691607605987e+01
0.000000000000000000e+00
0.000000000000000000e+00
-1.547397429377200817e+00
-4.018181657009961327e+00
-7.289186736637049968e+00
-7.458655219230571731e+00
-9.625538282761622710e+00
-1.103039457077456298e+01
-6.262675161142102809e+01
-9.265912629799268885e+00
-8.961543476816615339e+00
-9.622764435629340696e+00
-1.097978292092879826e+01
-1.097978292092879826e+01
-2.088057846500587456e+00
7.763936238952200239e-05
4.404309737393306595e-05
-9.467385027293546973e-02
4.348947142638090457e-01
-2.091409170053043390e-04
-5.788902107267982974e-04
4.481443450852441001e-10
-3.136135459023654537e+00
-1.507730262127600751e+00
-1.204663669965535977e+00
-1.171839254318371104e+00
4.329302247232582057e-01
8.310653628092458334e+00
3.299246725156660176e+00
0.000000000000000000e+00
1.847454931582027254e-02
-1.498024139966201096e+00
-7.660670007653060942e-01
-2.908130300830076731e+00
-1.252564844610269734e+00
-1.910955328742287573e+00
9.471710980110392697e-01
2.352302758516665371e+00
2.982269972214651954e+00
4.280736383314343918e+00
4.633629909719495288e+00
-2.198052637823726840e+00
-1.150759637168392580e+00
3.420433363184381292e+00
1.857878113059351077e-02
-1.559806674919653746e+00
4.197498183183401288e+00
6.262186949633183453e+00
1.100694844370524095e+01
2.778688785515088000e+01
3.679948298336883195e+00
-2.432801394376875592e+00
5.133442052706843617e-01
2.181172654073517680e+00
-8.577551729671881731e-01
7.013844214023315926e-01
3.138233436228588857e+00
7.319940508466630247e-01
0.000000000000000000e+00
3.529209394581482861e+00
1.464585117707144413e+01
8.506550226820598359e-01
-9.060397326548508268e-01
6.787474954688997641e+00
8.125411068867387954e+00
4.515740684104064151e+00
5.372135582950940069e+00
9.210951196799497254e-01
4.802177410869620466e+00
2.945445016176073594e+01
1.575032253128311632e+00
-1.355276854364796946e-01
-3.322474764169629502e-01
3.018397817188666732e+00
1.186503569922195744e+00
0.000000000000000000e+00
8.883242370198487503e-01
7.248276146728496627e+00
0.000000000000000000e+00
0.000000000000000000e+00
-4.831246718433664711e+00
6.124136970173365002e-01
4.145693892559814686e-01
2.683998941637626867e+00
2.063906603639539039e+00
2.989801217386735210e+00
2.262965767379551962e-01
2.240332214649647380e+00
5.984550782416063086e+00
4.587011255338186544e+00
1.233118485315272039e+01
1.115223490909697857e+00
-3.682686422016995476e+00
6.096498453291562258e-01
1.119275528656461516e+00
1.377886278915177731e-01
3.828176805973048324e+00
0.000000000000000000e+00
0.000000000000000000e+00
1.442927634029647344e+01
0.000000000000000000e+00
5.719118583309401593e-01
1.993632609731877392e-01
3.047472271520709430e+00
5.736784864911910198e+00
6.677826247219391220e+00
6.307175478564531090e+00
3.150295169417364249e+01
3.738597740702392258e+00
1.129754590514236234e+01
6.108506268573830056e+00
1.605489516792866667e+00
2.929631990348545489e+00
-2.832543082245212937e-02
1.358286530670594461e+00
1.655932469853677924e+00
6.701964773769768513e-01
2.131182050917533211e+00
2.998351165769753468e+00
7.772095996358327596e+00
1.285014785269981141e+00
4.407334784589962418e+00
1.719858214230612026e+00
-1.012765674651314063e+00
-5.749271123172469133e-01
-3.559614093795681278e+00
-3.073088477387719397e+00
-4.492469521371540431e+00
-3.753286990415885427e+00
-3.219255423324282273e+00
-2.806436518181075090e+00
-2.697305948568419875e+00
-7.879608430851776646e-01
4.625507221739111330e+00
4.809280703883450414e+00
-3.435194026629848629e+00
-3.218943068168937049e+00
3.335535704890596698e+00
2.071359212435486263e+00
4.538992059175040339e+00
-2.770772323566738038e+01
2.903047708571506735e+00
-4.436143805989154032e+00
-2.647991280011542381e-01
1.737252348126810064e+00
-4.121989655995259128e+00
3.209709099445720581e-01
1.012758514896711759e+01
3.313255624721038295e+00
4.631467619785444967e+00
7.668642402146534032e+00
6.780938812710099128e+00
-3.256164342602652972e+00
6.749565128319576779e-01
0.000000000000000000e+00
-4.407265954524525853e+00
0.000000000000000000e+00
-3.666522115024547901e+01
-7.886029397826226273e+01
0.000000000000000000e+00
0.000000000000000000e+00
-2.261283814517791058e+01
-4.024317426178160240e+00
3.213063737030031342e-01
5.079805145796887800e+00
1.326813226475260343e+00
1.233684078112145643e+00
8.671852503871454232e+00
-2.041800256066371944e+00
0.000000000000000000e+00
0.000000000000000000e+00
-1.607347800380474823e+01
-4.430790279223246309e+00
1.177552465851384511e+00
6.342921220500139512e+00
-2.466913734548706327e-02
3.451642566010713065e-01
-6.012767168531006234e+00
7.328146570137336724e+00
7.500088131707050465e+00
0.000000000000000000e+00
-3.547913249211809017e+01
-3.130964814607208879e+00
8.247326544297072237e-01
3.757262485775580418e-01
-2.136528302027558723e+00
-2.631627236037529793e-01
-2.016718799388414141e+01
0.000000000000000000e+00
0.000000000000000000e+00
0.000000000000000000e+00
-7.708602132869285528e-01
-2.602868328868111814e+00
1.435184800833797958e+00
0.000000000000000000e+00
-2.080420864280113413e+00
1.169498351211070819e+00
-1.798334115637199560e+01
-1.193885252696202670e+01
0.000000000000000000e+00
0.000000000000000000e+00
4.304089297965300709e+00
-3.020893216686394656e+00
-1.234427481614708721e+00
0.000000000000000000e+00
1.853340741926325697e+00
-2.686000064995862147e+01
-1.672275139058893600e+01
-2.826268691607605987e+01
0.000000000000000000e+00
0.000000000000000000e+00
-1.547397429377200817e+00
-4.018181657009961327e+00
-7.289186736637049968e+00
-7.458655219230571731e+00
-9.625538282761622710e+00
-1.103039457077456298e+01
-6.262675161142102809e+01
-9.265912629799268885e+00
-8.961543476816615339e+00
-9.622764435629340696e+00
-1.097978292092879826e+01
-1.097978292092879826e+01
+1
View File
@@ -0,0 +1 @@
59708719594136008472212676481906
@@ -0,0 +1 @@
5コ705コ(770(777
+1
View File
@@ -0,0 +1 @@
2Bba5a7FEEcfced4fbbBfFEe16bfcaA4
@@ -0,0 +1 @@
5コ70(77247・906:
@@ -0,0 +1 @@
5コ700772212676481906:
Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More