Compare commits

...
1285 Commits
Author SHA1 Message Date
Ivan Nardi ebcea42e2b fuzz: improve fuzzers using pl7m (#2486)
For some unclear reasons, fuzzers using pl7m create huge corpus,
triggering OOM in oss-fuzz runs (where the memory RSS limit is set to
2560Mb). Example:

```
==25340== ERROR: libFuzzer: out-of-memory (used: 2564Mb; limit: 2560Mb)
To change the out-of-memory limit use -rss_limit_mb=<N>

Live Heap Allocations: 2364004039 bytes in 133791 chunks; quarantined: 60662293 bytes in 3664 chunks; 176432 other chunks; total chunks: 313887; showing top 95% (at most 8 unique contexts)
1285841683 byte(s) (54%) in 2956 allocation(s)
    #0 0x56f814ef4bde in __interceptor_malloc /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
    #1 0x56f814e04416 in operator new(unsigned long) cxa_noexception.cpp:0
    #2 0x56f814de6b2d in assign<unsigned char *, 0> /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1443:3
    #3 0x56f814de6b2d in operator= /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1412:9
    #4 0x56f814de6b2d in fuzzer::InputCorpus::AddToCorpus(std::__Fuzzer::vector<unsigned char, std::__Fuzzer::allocator<unsigned char>> const&, unsigned long, bool, bool, bool, std::__Fuzzer::chrono::duration<long long, std::__Fuzzer::ratio<1l, 1000000l>>, std::__Fuzzer::vector<unsigned int, std::__Fuzzer::allocator<unsigned int>> const&, fuzzer::DataFlowTrace const&, fuzzer::InputInfo const*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerCorpus.h:221:10
    #5 0x56f814de60e5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:539:16
    #6 0x56f814de7df2 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:829:7
    #7 0x56f814de8127 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:867:3
    #8 0x56f814dd6736 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:914:6
    #9 0x56f814e02c62 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    #10 0x7fa11e2c3082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16

1031350683 byte(s) (43%) in 2468 allocation(s)
    #0 0x56f814ef4bde in __interceptor_malloc /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
    #1 0x56f814e04416 in operator new(unsigned long) cxa_noexception.cpp:0
    #2 0x56f814de6b2d in assign<unsigned char *, 0> /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1443:3
    #3 0x56f814de6b2d in operator= /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1412:9
    #4 0x56f814de6b2d in fuzzer::InputCorpus::AddToCorpus(std::__Fuzzer::vector<unsigned char, std::__Fuzzer::allocator<unsigned char>> const&, unsigned long, bool, bool, bool, std::__Fuzzer::chrono::duration<long long, std::__Fuzzer::ratio<1l, 1000000l>>, std::__Fuzzer::vector<unsigned int, std::__Fuzzer::allocator<unsigned int>> const&, fuzzer::DataFlowTrace const&, fuzzer::InputInfo const*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerCorpus.h:221:10
    #5 0x56f814de60e5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:539:16
    #6 0x56f814de7635 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:760:19
    #7 0x56f814de8425 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:905:5
    #8 0x56f814dd6736 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:914:6
    #9 0x56f814e02c62 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    #10 0x7fa11e2c3082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
```

See: https://oss-fuzz.com/testcase-detail/4717811415449600
See: https://oss-fuzz.com/testcase-detail/6164130982068224

Let's *try* the following workaround: set the parameter `-max-len` to
512K, to force the engine to not genereate inputs (i.e. pcap files...)
larger than 512K. Right now the value used is 1MB, i.e the default,
because we have file larger than 1MB in the initial seeds (i.e.
`/tests/pcaps/*`).

Let's hope than having smaller files lead to smaller corpus...

Update pl7m code (fix a Use-of-uninitialized-value error)
2024-06-29 18:01:05 +02:00
Ivan Nardi 83e6e753af fuzz: pl7m: add a custom mutator for better fuzzing of pcap files (#2483)
Pl7m is a custom mutator (used for structure aware fuzzing) for network
traffic packet captures (i.e. pcap files).

The output of the mutator is always a valid pcap file, containing the
same flows/sessions of the input file. That's it: the mutator only
changes the packet payload after the TCP/UDP header, keeping all the
original L2/L3 information (IP addresses and L4 ports).

See: https://github.com/IvanNardi/pl7m
2024-06-27 18:07:43 +02:00
Anshul ThakurandAnshul Thakur 8f6f73505d Remove duplicate API for ndpi_serialize_uint32_double in ndpi_api.h (#2480)
Co-authored-by: Anshul Thakur <athakur@cdot.in>
2024-06-26 14:10:57 +02:00
Nardi Ivan 556f892a56 wireshark: lua: export some metadata
Export some metadata (for the moment, SNI and TLS fingerprints) to
Wireshark/tshark via extcap.
Note that:
* metadata are exported only once per flow
* metadata are exported (all together) when nDPI stopped processing
the flow

Still room for a lot of improvements!
In particular:
* we need to add some boundary checks (if we are going to export other
attributes)
* we should try to have a variable length trailer
2024-06-25 16:39:45 +02:00
Nardi Ivan f44832cc51 wireshark: lua: filtering and trailer dissection work with tshark, too
```
ivan@ivan-Latitude-E6540:~/$ tshark -C "nDPI extcap" -i ndpi -o extcap.ndpi.i:/home/ivan/svnrepos/nDPI/tests/pcap/anydesk.pcapng -Y "ndpi.protocol.name contains DNS"
Capturing on 'nDPI interface: ndpi'
   62 22635386.425683 192.168.1.187 DNS.AnyDesk 192.168.1.1  128   Standard query 0xec22 A relay-3185a847.net.anydesk.com
   63 22635386.439540  192.168.1.1 DNS.AnyDesk 192.168.1.187 144   Standard query response 0xec22 A relay-3185a847.net.anydesk.com A 37.61.223.15
   64 22635386.721277 192.168.1.187 DNS.AnyDesk 192.168.1.1  128   Standard query 0xea89 A relay-9b6827f2.net.anydesk.com
   65 22635386.732444  192.168.1.1 DNS.AnyDesk 192.168.1.187 144   Standard query response 0xea89 A relay-9b6827f2.net.anydesk.com A 138.199.36.115
4 packets captured

```
2024-06-25 16:39:45 +02:00
Nardi Ivan 3f0ea18866 wireshark: lua: fix DNS dissection
Not sure when we (or Wireshark, or Lua...) broke it, but we can't call
tonumber() on Bool variables.
2024-06-25 16:39:45 +02:00
Nardi Ivan b5afa165f0 wireshark: extcap: restore filtering mechanism 2024-06-25 16:39:45 +02:00
Nardi Ivan 2daab3f248 wireshark: lua: latest Wireshark versions correctly handle 64 bit mask 2024-06-25 16:39:45 +02:00
Nardi Ivan 46e450456b wireshark: lua: minor improvements
* Use a proper TVB to parse the nDPI trailer
* Fix some flow risks definitions
2024-06-25 16:39:45 +02:00
Luca Deri e92568e4bb Improved logic for checking invalid DNS queries 2024-06-23 22:27:19 +02:00
Mark Jeffery aa1d7247d1 Added default port mappings to ndpiReader help -H (#2477)
Close #2125
2024-06-19 13:47:18 +02:00
Ivan Nardi 9feede9a14 Zoom: fix detection of screen sharing (#2476) 2024-06-17 17:30:24 +02:00
Ivan Nardi 26cc1f131f fuzz: improve fuzzing coverage (#2474)
Remove some code never triggered

AFP: the removed check is included in the following one
MQTT: fix flags extraction
2024-06-17 13:45:47 +02:00
Nardi Ivan a35fae6b75 Sync unit tests results 2024-06-17 11:23:39 +02:00
Toni 8fd649ab1e Add Ripe Atlas probe protocol. (#2473)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-06-17 11:00:38 +02:00
Nardi Ivan bbe52da5cf Zoom: harden RTP/RTCP detection 2024-06-17 10:19:55 +02:00
Nardi Ivan 526cf6f291 Zoom: remove "stun_zoom" LRU cache
Since 070a0908b we are able to detect P2P calls directly from the packet
content, without any correlation among flows
2024-06-17 10:19:55 +02:00
Ivan Nardi 2bedd14aae Update main Readme and remove commented out code (#2471)
Avoid that useless code is copy-and-pasted into new dissector, as in
https://github.com/ntop/nDPI/pull/2470#discussion_r1639384434
2024-06-17 10:11:17 +02:00
Mark Jeffery f796c94375 Added protocol - JRMI - Java Remote Method Invocation (#2470) 2024-06-15 10:52:28 +02:00
Luca 7c7c375b45 Improved detection of Android connectiity checks 2024-06-12 10:55:07 +02:00
Mark Jeffery 312dc424bd Added NDPI_PROTOCOL_NTOP assert and removed percentage comparison (#2460)
Close #2413
2024-06-10 19:45:19 +02:00
Ivan Nardi aee2c81f76 Zoom: fix integer overflow (#2469)
```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==29508==ERROR: AddressSanitizer: SEGV on unknown address 0x50710145d51d (pc 0x55cb788f25fe bp 0x7ffcfefa15f0 sp 0x7ffcfefa1240 T0)
==29508==The signal is caused by a READ memory access.
    #0 0x55cb788f25fe in ndpi_search_zoom /home/ivan/svnrepos/nDPI/src/lib/protocols/zoom.c:210:24
    #1 0x55cb787e9418 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7174:6
    #2 0x55cb7883f753 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7209:10
    #3 0x55cb7883bc9d in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7240:12
```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=69520
2024-06-10 11:06:17 +02:00
Ivan Nardi b90d39c4ac RTP/STUN: look for STUN packets after RTP/RTCP classification (#2465)
After a flow has been classified as RTP or RTCP, nDPI might analyse more
packets to look for STUN/DTLS packets, i.e. to try to tell if this flow
is a "pure" RTP/RTCP flow or if the RTP/RTCP packets are multiplexed with
STUN/DTLS.
Useful for proper (sub)classification when the beginning of the flows
are not captured or if there are lost packets in the the captured traffic.

Disabled by default
2024-06-07 13:12:04 +02:00
Ivan Nardi 070a0908b3 Zoom: faster detection of P2P flows (#2467) 2024-06-07 09:50:41 +02:00
Ivan Nardi 619005c5b2 STUN: add support for Microsoft Multiplexed TURN channels (#2464) 2024-06-05 16:55:58 +02:00
Ivan Nardi b6beeef623 TLS: add support for DTLS (over STUN) over TCP (#2463)
TODO: TCP reassembler on top of UDP reassembler

See: #2414
2024-06-05 13:49:34 +02:00
Ivan Nardi 6a9d4b1ab6 CI: more parallel work (#2459) 2024-06-05 11:56:03 +02:00
Ivan Nardi 7be482f5b1 Update unit tests results (#2466) 2024-06-05 11:55:30 +02:00
Luca Deri 85587a8f98 Fixes accounting of number of processed packets in ndpi_internal_detection_process_packet() 2024-06-04 17:00:45 +02:00
Ivan Nardi 81e42b748e RTP: fix detection over TCP (#2462)
RFC4571 is not the only way to wrap RTP messages in TCP streams.
For example, when RTP is encapsulated over TURN flows (i.e. via DATA
attribute) there is no additional framing.
See also 6127e0490
2024-05-29 18:31:10 +02:00
Maatuq 6127e04900 support rtp/rtcp over tcp (#2422) (#2457)
Support rtp/rtcp over tcp as per rfc4571.

Signed-off-by: mmaatuq <mahmoudmatook.mm@gmail.com>
2024-05-28 22:01:08 +02:00
Toni 80171dbcf3 Add ZUG consensus protocol dissector. (#2458)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-28 20:29:48 +02:00
Ivan Nardi 25f8964a23 CiscoVPN: we detect it only over UDP (#2454)
The original code handled also TCP/TLS, but it was removed in 6fc29b3ae
2024-05-28 14:07:48 +02:00
ToniandNardi Ivan abce6d4023 Improved Kafka dissector. (#2456)
* detect more Kafka request packet's
 * requires less flow memory
 * same detection behavior as before e.g. no asym detection implemented
   (can be done by dissecting responses, requires more effort)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2024-05-27 09:46:32 +02:00
Vladimir Gavrilov d1a59c0308 Rename Messenger to FacebookMessenger (#2453) 2024-05-23 09:23:31 +02:00
Ivan Nardi 41050af5b5 fuzz: fix build of oss-introspector (#2452) 2024-05-22 19:41:59 +02:00
Nardi Ivan 521ce609c8 Sync unit tests results 2024-05-22 19:00:33 +02:00
Luca 44a290286b More NDPI_PROBING_ATTEMPT changes 2024-05-22 18:04:33 +02:00
Luca 74d3843ebe Added NDPI_PROBING_ATTEMPT risk 2024-05-22 17:55:16 +02:00
Vladimir Gavrilov 15643547fe Replace ndpi_strnstr() implementation with an optimal one (#2447) 2024-05-22 12:47:27 +02:00
Luca Deri 5a25f89ab3 Enlarged buffer 2024-05-22 11:00:01 +02:00
Ivan Nardi fd02baa13a DTLS: fix JA4 fingerprint (#2446) 2024-05-21 18:13:25 +02:00
Toni 3639d2045b Remove unused code. (#2450)
* some `#ifdef`ed code dates back to 2019, 2020 and 2021
 * some function signatures were still present in `ndpi_main.h`
   which may cause linker errors for libnDPI dependee's
 * return an error while trying to serialize a double instead
   of `fprintf(stderr, ...)`

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-21 13:08:07 +02:00
Ivan Nardi 83840f1bb9 DTLS: add support for DTLS 1.3 (#2445) 2024-05-21 12:49:27 +02:00
Ivan Nardi 0109014f2c Follow-up of 2093ac5bf (#2451) 2024-05-21 12:47:25 +02:00
Ivan Nardi bc8bc69beb CI: enable parallel tests (for x86_64, at least) (#2444)
TODO: enable parallel tests when using docker with no-x86_64 archs.

When I tried the obviuos solutions:
```
NDPI_FORCE_PARALLEL_UTESTS=1 NDPI_SKIP_PARALLEL_BAR=1 make check VERBOSE=1
```
I got:
```
Run configuration "caches_cfg" [--cfg=lru.ookla.size,0 --cfg=lru.msteams.ttl,1]
ookla.pcap	/bin/sh: 1: run_single_pcap: not found
teams.pcap	/bin/sh: 1: run_single_pcap: not found
Run configuration "caches_global" [--cfg=lru.ookla.scope,1 --cfg=lru.bittorrent.scope,1 --cfg=lru.stun.scope,1 --cfg=lru.tls_cert.scope,1 --cfg=lru.mining.scope,1 --cfg=lru.msteams.scope,1 --cfg=lru.stun_zoom.scope,1]
bittorrent.pcap	/bin/sh: 1: run_single_pcap: not found
lru_ipv6_caches.pcapng	/bin/sh: 1: run_single_pcap: not found
mining.pcapng	/bin/sh: 1: run_single_pcap: not found
...
```
2024-05-20 19:07:16 +02:00
Ivan Nardi 399be12585 Small fixes after API cleanup done in c63446e59 (#2449) 2024-05-20 19:06:24 +02:00
Luca Deri 2093ac5bf6 Minor dissector optimizations 2024-05-20 12:17:04 +02:00
Luca Deri 42dba2e4af Added dpi.compute_entropy configuration parameter 2024-05-18 09:46:15 +02:00
Luca Deri c63446e592 Cleaned up API
Removed
-  int ndpi_load_ipv4_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);
-  int ndpi_load_ipv6_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);

Added (it supports both IPv4 and v6)
+  int ndpi_load_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);
2024-05-17 16:38:02 +02:00
Ivan Nardi 33d3d254ac CI: add support for ubuntu-24.04 runners (#2443) 2024-05-16 17:05:13 +02:00
Vladimir Gavrilov 3d1da00d8d Add Call of Duty Mobile support (#2438) 2024-05-15 12:46:02 +02:00
Ivan Nardi d6b1c24079 Parallel execution of unit tests (#2435)
Running unit tests is quite a bottleneck while developing or while
waiting for GitHub CI results...
Try to run the tests in parallel, using the `parallel` tool.

By default, tests still run one after the other, as usual; to enable
parallel execution you need `NDPI_FORCE_PARALLEL_UTESTS=1 ./tests/do.sh`

Please note that the output is quite different in parallel mode!

A big part of the script has been rewritten to avoid code dupication
between "serial" and "parallel" path

On my notebook:
```
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(parallel)$ time ./tests/do.sh
[...]
real	3m12,684s
[...]
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(parallel)$ time NDPI_FORCE_PARALLEL_UTESTS=1 ./tests/do.sh
[...]
real	0m58,463s
```
2024-05-15 08:17:12 +02:00
Ivan Nardi 0110623b4e H323: improve detection and avoid false positives (#2432) 2024-05-11 23:39:54 +02:00
Ivan Nardi a064261e85 Revert ndpi_strnstr() optimization introduced in a813121e0 (#2439)
New implementation fails tests 11b, 12 and 13.
Revert to the original (BSD) implementation (with also some basic
parameters check)
2024-05-11 23:37:31 +02:00
Vladimir Gavrilov 2d33431948 Add Ethernet Global Data support (#2437) 2024-05-11 16:49:16 +02:00
Ivan Nardi 8c5e6e9761 IRC: simplify detection (#2423)
IRC has its best times well behind, but there are still some servers
using it.

We should try to simplify the detection logic, still based on OpenDPI
logic.

Let's start with some easy changes:
* try to detect TLS connection via standard hostname/SNI matching,
removing an old heuristic (we have never had any trace matching it);
* add some basic server names;
* once we detect that the flow is IRC, we don't have to perform
anything else;
* remove HTTP stuff; real HTTP flows never trigger that data path
* use `ndpi_memmem()` when possible
2024-05-11 14:12:13 +02:00
Vladimir GavrilovandIvan Nardi 70c4920184 Remove Vevo support (#2436)
Co-authored-by: Ivan Nardi <12729895+IvanNardi@users.noreply.github.com>
2024-05-11 11:57:53 +02:00
Ivan Nardi b116456fc5 Viber: add detection of voip calls and avoid false positives (#2434) 2024-05-11 09:21:13 +02:00
Vladimir Gavrilov a813121e0a ndpi_strnstr() optimization (#2433) 2024-05-10 22:43:59 +02:00
Ivan Nardi 4b4b358562 Line: use common helper to detect RTP/RTCP packets (#2429)
Add an explicit upper limit on the number of packets processed before
giving up.
2024-05-10 19:23:50 +02:00
Luca Deri 6bbae9a70a Update README.md 2024-05-10 08:53:19 +02:00
Ivan Nardi d4650f0f81 Raknet/RTP: avoid Raknet false positives and harden RTP heuristic (#2427)
There is some overlap between RTP and Raknet detection: give precedence
to RTP logic.
Consequences:
* Raknet might require a little bit more packets for some flows (not a
big issue)
* some very small (1-2 pkts) Raknet flows are not classified (not sure
what do do about that..)
2024-05-09 21:18:18 +02:00
Luca Deri e4fd6e47a3 Added #ifdef for Windows builds (#2431) 2024-05-09 17:27:25 +02:00
Ivan Nardi 2e7fc82e55 Protobuf: fix false positives (#2428) 2024-05-09 15:38:06 +02:00
Toni 18e03a26ca Add extra entropy checks and more precise(?) analysis. (#2383)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-09 15:24:11 +02:00
Ivan Nardi b65a755e85 Fix a memory access error and some leaks (#2425)
```
SCARINESS: 12 (1-byte-read-heap-buffer-overflow)
    #0 0x557f3a5b5100 in ndpi_get_host_domain /src/ndpi/src/lib/ndpi_domains.c:158:8
    #1 0x557f3a59b561 in ndpi_check_dga_name /src/ndpi/src/lib/ndpi_main.c:10412:17
    #2 0x557f3a51163a in process_chlo /src/ndpi/src/lib/protocols/quic.c:1467:7
    #3 0x557f3a469f4b in LLVMFuzzerTestOneInput /src/ndpi/fuzz/fuzz_quic_get_crypto_data.c:44:7
    #4 0x557f3a46abc8 in NaloFuzzerTestOneInput (/out/fuzz_quic_get_crypto_data+0x4cfbc8)
```

Some notes about the leak: if the insertion into the uthash fails (because of an
allocation failure), we need to free the just allocated entry. But the only
way to check if the `HASH_ADD_*` failed, is to perform a new lookup: a bit
costly, but we don't use that code in the fast-path.
See also efb261a95c

Credits for finding the issues to Philippe Antoine (@catenacyber) and his
`nallocfuzz` fuzzing engine
See: https://github.com/catenacyber/nallocfuzz
See: https://github.com/google/oss-fuzz/pull/9902
2024-05-08 11:46:46 +02:00
Ivan Nardi 7c6910d9e5 Fix/improve fuzzing (#2426) 2024-05-08 11:46:02 +02:00
Toni e9dc035c5c Added optimized memmem/strlcpy version (#2424)
* credits goes to Vladimir Gavrilov

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-08 11:38:53 +02:00
Luca Deri 2b4e2f9c9a Removed trace 2024-05-07 23:42:32 +02:00
Luca Deri 5f8ad0527d Reworked code and added two new API calls
- int ndpi_load_ipv4_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);
- int ndpi_load_ipv6_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);
2024-05-07 23:05:31 +02:00
Luca Deri 3d2b611ea6 ndpi_mod is now optional (albeit better to spcify it) in ndpi_domain_classify_xxx calls 2024-05-07 13:09:00 +02:00
Luca Deri 0d495294bf Win warning fixes 2024-05-07 10:04:11 +02:00
Luca Deri 6195dbad76 Renamed radius source file to avoid name clashes on Windows 2024-05-07 09:05:15 +02:00
Ivan Nardi 1773d7ff2d Add support for Mastodon, Bluesky and (FB-)Threads (#2418) 2024-05-06 13:37:18 +02:00
Ivan Nardi 95fe21015d Remove "zoom" cache (#2420)
This cache was added in b6b4967aa, when there was no real Zoom support.
With 63f349319, a proper identification of multimedia stream has been
added, making this cache quite useless: any improvements on Zoom
classification should be properly done in Zoom dissector.

Tested for some months with a few 10Gbits links of residential traffic: the
cache pretty much never returned a valid hit.
2024-05-06 12:51:45 +02:00
Ivan Nardi b7d77972bc Remove workaround for TCP flows with multiple SYNs (#2421)
Deciding when a session starts and ends is responsability of the
applicationi (via its flow manager)i, not of the library.
BTW, the removed code is incomplete at beast
2024-05-06 10:20:40 +02:00
Ivan Nardi e31ef00715 TLS: avoid setting NDPI_TLS_SELFSIGNED_CERTIFICATE for webrtc traffic (#2417)
See RFC8122: it is quite likely that STUN/DTLS/SRTP flows use
self-signed certificates

Follow-up of b287d6ec8
2024-05-06 10:20:07 +02:00
Ivan Nardi 266af02752 Merge RTP and RTCP logic (#2416)
Avoid code duplication between these two protocols.

We remove support for RTCP over TCP; it is quite rare to find this kind
of traffic and, more important, we have never had support for RTP
over TCP: we should try to add both detecion as follow-up.

Fix a message log in the LINE code
2024-05-06 10:19:46 +02:00
Ivan Nardi a1845da359 TLS: fix Ja4 fingerprint computation (#2419)
The new values has been checked against the ones reported by Wireshark.

Found while fixing a Use-of-uninitialized-value error reported by
oss-fuzz

```
==7582==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x5a6549abc368 in ndpi_compute_ja4 ndpi/src/lib/protocols/tls.c:1762:10
    #1 0x5a6549ab88a0 in processClientServerHello ndpi/src/lib/protocols/tls.c:2863:10
    #2 0x5a6549ac1452 in processTLSBlock ndpi/src/lib/protocols/tls.c:909:5
    #3 0x5a6549abf588 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:1098:2
    #4 0x5a65499c53ec in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:7215:6
```

See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68449&q=ndpi&can=1&sort=-id
2024-05-05 17:27:29 +02:00
Ivan Nardi a6fd981fcf eDonkey: improve/update classification (#2410)
eDonkey is definitely not as used as >10 years ago, but it seems it is
still active.

While having a basic TCP support seems easy, identification over UDP doesn't
work and it is hard to do it rightly (packets might be only 2 bytes long):
remove it.

Credits to V.G <v.gavrilov@securitycode.ru>
2024-05-04 19:11:31 +02:00
Luca Deri 57ecbf38c0 Updated JA4 test results 2024-05-02 17:40:24 +02:00
Luca Deri 66133d1501 Fixes JA4 computation adding a better GREASE detect funzion 2024-05-02 16:58:44 +02:00
Luca Deri af955169e1 typo 2024-05-02 16:03:30 +02:00
Ivan Nardi 9e549ba781 Workaround for fixing GitHub runners on macOS (#2411)
GitHub switched "macos-latest" from "macos-12" to "macos-14", which is only
on ARM64!
https://github.com/actions/runner/issues/3256
https://github.blog/changelog/2024-01-30-github-actions-macos-14-sonoma-is-now-available/

However we are having some issues build nDPI on macos-14 with external
libraries:

```
configure: error: libgpg-error required (because of --with-local-libgcrypt) but not found or too old.
```
See: https://github.com/ntop/nDPI/actions/runs/8869020568/job/24350356867

```
ndpi_utils.c:69:10: fatal error: 'pcre2.h' file not found
         ^~~~~~~~~
1 error generated.
```
See: https://github.com/ntop/nDPI/actions/runs/8869020568/job/24349242251

Everything is still fine with macos-14 and no external dependencies

As workaround, test only macos-12 and macos-13 in our main matrix.
2024-04-28 22:17:39 +02:00
Ivan Nardi d5bda47efe DTLS: add support for Alert message type (similar to TLS) (#2406) 2024-04-25 11:23:05 +02:00
0x41CEA55 7040847eed Add Adobe Connect support (#2407) 2024-04-24 12:41:25 +02:00
0x41CEA55 66036a14f4 Remove PPStream protocol and add iQIYI (#2403)
P2P video player PPStream was discontinued shortly after the purchase of PPS.tv by Baidu (iQIYI) on 2013 (see https://www.techinasia.com/report-baidu-acquires-video-rival-pps)
So we remove the old `NDPI_PROTOCOL_PPSTREAM` logic and add `NDPI_PROTOCOL_IQIYI` id to handle all the iQIYI traffic, which is basically video streaming traffic.

A video hosting service, called PPS.tv, is still offered by the same company: for the time being we classified both services with the same protocol id.
2024-04-23 18:01:36 +02:00
0x41CEA55 fd388845d5 Add BFCP protocol support (#2401) 2024-04-23 15:35:19 +02:00
Ivan Nardi abc7e430e2 STUN: slightly faster sub-classification with DTLS (#2404) 2024-04-23 09:31:48 +02:00
Ivan Nardi a62679952c fuzz: fix shoco fuzzer (#2405)
```
==22779==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f0900701020 at pc 0x555bcd2a6f02 bp 0x7ffe3ba5e790 sp 0x7ffe3ba5e788
READ of size 1 at 0x7f0900701020 thread T0
    #0 0x555bcd2a6f01 in shoco_decompress /home/ivan/svnrepos/nDPI/src/lib/third_party/src/shoco.c:184:26
    #1 0x555bcd2a4018 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco.cpp:18:3
    #2 0x555bcd1aa816 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x4f816) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)
    #3 0x555bcd193be8 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x38be8) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)
    #4 0x555bcd1996fa in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x3e6fa) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)
    #5 0x555bcd1c3c92 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x68c92) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)
    #6 0x7f090257a082 in __libc_start_main /build/glibc-e2p3jK/glibc-2.31/csu/../csu/libc-start.c:308:16
    #7 0x555bcd18e96d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x3396d) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)

Address 0x7f0900701020 is located in stack of thread T0 at offset 4128 in frame
    #0 0x555bcd2a3d97 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco.cpp:5

  This frame has 4 object(s):
    [32, 4128) 'out' (line 9) <== Memory access at offset 4128 overflows this variable
    [4256, 8352) 'orig' (line 9)
```

Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68211
2024-04-22 18:39:23 +02:00
Nardi Ivan f494bdd653 Fix parameters check
```
==17==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000546050 bp 0x7fff113c82a0 sp 0x7fff113c7a58 T0)
==17==The signal is caused by a READ memory access.
==17==Hint: address points to the zero page.
SCARINESS: 10 (null-deref)
    #0 0x546050 in __sanitizer::internal_strlen(char const*) /src/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_libc.cpp:167:10
    #1 0x4c6ba5 in __interceptor_strrchr /src/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:740:5
    #2 0x5fb9b9 in ndpi_get_host_domain_suffix /src/ndpi/src/lib/ndpi_domains.c:105:20
    #3 0x578058 in LLVMFuzzerTestOneInput /src/ndpi/fuzz/fuzz_config.cpp:503:3
```

Found while fuzzing
2024-04-21 12:29:09 +02:00
0x41CEA55 246e972f9c Replace my personal email with my corporate one in all my contributions (#2399) 2024-04-20 21:37:01 +02:00
Nardi Ivan 95f4d0b852 Fix a warning
```
nDPI/PcDebug64/src/include/ndpi_api.h:1970:3: error: function declaration isn’t a prototype [-Werror=strict-prototypes]
```
2024-04-20 19:30:42 +02:00
Ivan Nardi ef89183469 fuzz: improvements (#2400)
Create the zip file with all the traces only once.

Add a new fuzzer to test "shoco" compression algorithm
2024-04-20 18:15:23 +02:00
0x41CEA55 905120588b Remove obsolete protocols: tuenty, tvuplayer and kontiki (#2398) 2024-04-19 21:35:32 +02:00
0x41CEA55 1b2e2cd968 Add strlcpy implementation (#2395) 2024-04-19 17:16:40 +02:00
0x41CEA55 e75d7a620e Add KNXnet/IP protocol support (#2397)
* Add KNXnet/IP protocol support

* Improve KNXnet/IP over TCP detection
2024-04-19 12:54:00 +02:00
Luca Deri ad117bfaab Domain Classification Improvements (#2396)
* Added
size_t ndpi_compress_str(const char * in, size_t len, char * out, size_t bufsize);
size_t ndpi_decompress_str(const char * in, size_t len, char * out, size_t bufsize);

used to compress short strings such as domain names. This code is based on
https://github.com/Ed-von-Schleck/shoco

* Major code rewrite for ndpi_hash and ndpi_domain_classify

* Improvements to make sure custom categories are loaded and enabled

* Fixed string encoding

* Extended SalesForce/Cloudflare domains list
2024-04-18 23:21:40 +02:00
Luca Deri 108b8331d5 Invalid initialization 2024-04-15 18:13:17 +02:00
Luca Deri 95897c6436 Fixed minor glitches 2024-04-15 14:25:26 +02:00
Ivan Nardi d97afd6efd STUN: fix attributes list iteration (#2391)
We need to check all the attributes, to look for any possible metadata
2024-04-13 18:43:41 +02:00
Ivan Nardi 06f4814085 STUN: try to stop extra dissection earlier, if possible (#2390) 2024-04-13 14:45:17 +02:00
Ivan Nardi 8edb2f133c STUN: add support for ipv6 in some metadata (#2389) 2024-04-13 14:12:20 +02:00
Ivan Nardi f51dfc6973 STUN: simplify ip/port parsing (#2388)
Add other 2 configuration options
2024-04-13 12:00:04 +02:00
Ivan Nardi 0535e54484 STUN: fix boundary checks on attribute list parsing (#2387)
Restore all unit tests.
Add some configuration knobs.
Fix the endianess.
2024-04-12 22:55:51 +02:00
Luca Deri bb4fccd2cf Added comment 2024-04-12 20:29:14 +02:00
Luca Deri b83eb7c7a2 Implemented STUN peer_address, relayed_address, response_origin, other_address parsing
Added code to ignore invalid STUN realm
Extended JSON output with STUN information
2024-04-12 19:50:04 +02:00
Toni Uhlig 31de94ba12 fix invalid read
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-12 17:55:48 +02:00
Vladimir Gavrilov c63697205b Add Label Distribution Protocol support (#2385)
* Add Label Distribution Protocol support

* Fix typo

* Update unit test results
2024-04-12 17:44:36 +02:00
Ivan Nardi 93f02ee507 Fix ndpi_reconcile_msteams_udp (#2377)
Microsoft UDP traffic over port ~3478 is voip traffic, using some kind
of proprietary STUN-like protocol: so use the most specific protocol id.

More important, we definitely want `Stun/Skype_TeamsCall` and not
`Stun/Skype_Teams`
2024-04-12 17:05:28 +02:00
Toni Uhlig 65fef09ff2 Updated unit test results
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-12 16:52:00 +02:00
Toni Uhlig 082b00ede7 Updated unit test results
* fixed invalid read

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-12 16:38:44 +02:00
Luca Deri a8507c3308 STUN:
- Fixed issue with XOR-MAPPED-ADDRESS decoding
- Implemented MAPPED_ADDRESS (IPv4 only)
2024-04-12 15:49:42 +02:00
Nardi Ivan 17965c8852 Update unit tests results 2024-04-11 18:36:43 +02:00
Luca Deri fd2838e9a1 Minor code cleanup 2024-04-11 17:44:44 +02:00
Toni 39a5d234fd Serialize amount of found TLS blocks and signature algos. (#2384)
* partially fixes #2282

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-11 16:21:41 +02:00
Luca Deri 8b4fdbc0fb Renamed ndpi_fill_ip6_protocol_category in ndpi_fill_ipv6_protocol_category 2024-04-11 12:37:39 +02:00
Vladimir Gavrilov 9ff4bece33 Add The Elder Scrolls Online support (#2376)
* Add The Elder Scrolls Online support

* Use ndpi_memmem instead of memmem from libc

* Add protocol description

* Change selection bitmask to V4_V6

* Update protocols.rst
2024-04-10 18:04:02 +02:00
Vladimir Gavrilov 142c8f5afb Add memmem() implementation (#2378)
* Add memmem() implementation

* Fix build

* Add fix to avoid too many memcmp calls
2024-04-10 15:39:08 +02:00
Toni 1d0be6c4f4 Add Shellscript risk detection. (#2375)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-10 15:36:54 +02:00
Ivan Nardi 54517d8e04 Fix a warning and restore a unit test result (#2379) 2024-04-10 09:56:06 +02:00
Luca Deri bd2968dcd3 Tuned DNS risk values
Modified NDPI_BINARY_TRANSFER_ATTEMPT in NDPI_BINARY_DATA_TRANSFER
2024-04-09 10:22:01 +02:00
Luca Deri 00a360f30a Disabled "known proto on non standard port" for FTP_DATA 2024-04-09 00:01:47 +02:00
Luca Deri 98bf0e243e Fixed false positives on binary application transfer risk 2024-04-08 23:52:27 +02:00
Luca Deri 48b0f8e2c1 Typo 2024-04-08 18:45:32 +02:00
Ivan Nardi 1b3ef7d7b2 STUN: improve extraction of Mapped-Address metadata (#2370)
Enable parsing of Mapped-Address attribute for all STUN flows: that
means that STUN classification might require more packets.

Add a configuration knob to enable/disable this feature.

Note that we can have (any) STUN metadata also for flows *not*
classified as STUN (because of DTLS).

Add support for ipv6.

Restore the correct extra dissection logic for Telegram flows.
2024-04-08 10:24:51 +02:00
Ivan Nardi f5905a62c7 Fix invalid memory access (#2374)
The bug is triggered when `pe_offset == (u_int32_t)-1`

```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==23719==ERROR: AddressSanitizer: SEGV on unknown address 0x5081000002b3 (pc 0x55c69274ac72 bp 0x7ffffffc8e70 sp 0x7ffffffc8cc0 T0)
==23719==The signal is caused by a READ memory access.
    #0 0x55c69274ac72 in ndpi_search_portable_executable /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8191:7
    #1 0x55c69271606b in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8596:5
    #2 0x55c69270f58f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8629:22
    #3 0x55c6926a07e7 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:24:5
    #4 0x55c6925a79b6 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x64e9b6) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
    #5 0x55c692590d48 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x637d48) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
    #6 0x55c69259685a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x63d85a) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
    #7 0x55c6925c0e02 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x667e02) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
    #8 0x7f8e99793082 in __libc_start_main /build/glibc-wuryBv/glibc-2.31/csu/../csu/libc-start.c:308:16
    #9 0x55c69258baed in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x632aed) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
```

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67881
2024-04-06 20:23:06 +02:00
Toni a5d45253c4 Add ELF risk detection (detect transmitted linux executables). (#2373)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-06 19:32:51 +02:00
Toni 727e72d1f1 Calculate packet entropy for unknown protocols. (#2369)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-06 17:01:19 +02:00
Vladimir Gavrilov b535033674 Get rid of Apache Kafka false positives (#2372) 2024-04-06 16:59:38 +02:00
Ivan Nardi 700637a162 fuzzing: extend fuzzing coverage (#2371) 2024-04-05 21:02:54 +02:00
Toni 0f77f49b77 Add PE32/PE32+ risk detection (detect transmitted windows executables). (#2312)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-05 14:43:28 +02:00
Toni 99e521eaf8 Limit git commit hash to 7 characters. (#2368)
* ensure consistency across all platforms

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-05 11:04:22 +02:00
Toni 09bb383437 Fixed yet another another "unused function" warning. (#2367)
* added `-Wextra` to the CI

```
In file included from ndpi_bitmap64_fuse.c:31:
./third_party/include/binaryfusefilter.h:31:24: error: unused function 'binary_fuse_rotl64' [-Werror,-Wunused-function]
static inline uint64_t binary_fuse_rotl64(uint64_t n, unsigned int c) {

..snip..
```

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 20:41:45 +02:00
RoboSchmied ead9aa72da Fix: 3 typos (#2366)
Signed-off-by: RoboSchmied <github@roboschmie.de>
2024-04-04 20:06:47 +02:00
Toni df29e12f5e Fixed yet another "unused function" warning. (#2365)
* seems like clang on `ubuntu-latest` warns about unused static inlined functions

```
In file included from ndpi_bitmap.c:41:
./third_party/include/roaring.h:422:19: error: unused function 'roaring_leading_zeroes' [-Werror,-Wunused-function]
static inline int roaring_leading_zeroes(unsigned long long input_num) {

..snip..
```

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 18:18:44 +02:00
Ivan Nardi c0d3f8a92e STUN: rework sub-classification (#2361)
The main goal is to have the "real" application (if any; i.e.
Signal/Whatsapp/Telegram/...) always as "application" protocol and not
as "master" one
2024-04-04 18:16:40 +02:00
Toni 6e61368cd6 Add -Wno-unused-function as CFLAG for third_party/src/gcrypt_light.c. (#2364)
* fixes failing nDPI build from an external project with clang and `-Wextra`

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 13:51:27 +02:00
Toni Uhlig dface63157 Fixed flow struct static assertion size check (this time with struct padding in mind).
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 12:40:09 +02:00
Toni Uhlig 6917acffda Fixed flow struct static assertion size check.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 12:36:42 +02:00
Luca Deri 9185c2ccc4 Added support for STUN Mapped IP address 2024-04-03 23:03:46 +02:00
Nardi Ivan aacbc9e918 Update unit tests results 2024-04-03 16:18:24 +02:00
Luca 225ff7f8a4 Added binary data transfer risk alert 2024-04-03 15:41:26 +02:00
Ivan Nardi 1acc9ee396 Patrica tree: fix handling of mac addresses (#2362)
Found looking at the fuzzer statistics...
2024-04-03 14:10:37 +02:00
Toni 41eef9246c Disable -Wno-unused-parameter -Wno-unused-function. (#2358)
* unused parameters and functions pollute the code and decrease readability

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-03 14:10:21 +02:00
Vitaly Lavrov b5e8bc2214 Fixed gcc version checking. (#2360)
Using the "test" utility instead of "[".
2024-04-02 10:58:55 +02:00
Vladimir Gavrilov 599cc0f4b8 Improve LoL: Wild Rift detection (#2359) 2024-03-28 16:01:01 +01:00
Vladimir Gavrilov 5b32c98a21 Add LoL: Wild Rift detection (#2356) 2024-03-26 08:11:14 +01:00
Luca Deri 51f5fc7140 Added support for roaring bitmap v3 (#2355)
* Integrated RoaringBitmap v3

* Renamed ndpi_bitmap64 ro ndpi_bitmap64_fuse

* Fixes to ndpi_bitmap for new roaring library

* Fixes for bitmap serialization

* Fixed format

* Warning fix

* Conversion fix

* Warning fix

* Added check for roaring v3 support

* Updated file name

* Updated path

* Uses clang-9 (instead of clang-7) for builds

* Fixed fuzz_ds_bitmap64_fuse

* Fixes nDPI printf handling

* Disabled printf

* Yet another printf fix

* Cleaup

* Fx for compiling on older platforms

* Fixes for old compilers

* Initialization changes

* Added compiler check

* Fixes for old compilers

* Inline function is not static inline

* Added missing include
2024-03-25 08:15:19 +01:00
Vladimir Gavrilov be0fd7444f Add more TencentGames signatures (#2354) 2024-03-21 14:38:47 +01:00
Vladimir Gavrilov 27f9ca9d04 Improve TencentGames detection (#2353)
* Improve TencentGames detection

* Add more signatures
2024-03-20 12:13:32 +01:00
Nardi Ivan 15a80527c6 STUN: remove workaround to identify RTP traffic
We are able to demultiplex RTP packets in STUN flows since 3608ab01b, at
least; no need to explicity call the RTP dissector
2024-03-20 09:39:15 +01:00
Nardi Ivan 5ccf41739b STUN: fix fallback when DTL dissection fails 2024-03-20 09:39:15 +01:00
Nardi Ivan f758f3605a STUN: dissection of TURN Channel Data 2024-03-20 09:39:15 +01:00
Nardi Ivan ee87c4bc66 STUN: add dissection of XOR-PEER-ADDRESS with ipv6 address 2024-03-20 09:39:15 +01:00
Nardi Ivan 58894dd788 STUN: improve heurstic to detect old classic-stun 2024-03-20 09:39:15 +01:00
Ivan Nardi dabf29ea61 STUN: update demultiplexing logic according to RFC9443 (#2349) 2024-03-19 09:11:51 +01:00
Vladimir Gavrilov e6474d835f Add FLUTE protocol dissector (#2351)
* Add FLUTE protocol dissector

* Add flute.c to MSVC project
2024-03-19 09:11:04 +01:00
Toni 15f61e7abe Enable USE_GLOBAL_CONTEXT via CFLAGS passed to the CC. (#2348)
* `ndpi_typedefs.h`: requires to include `ndpi_config.h` for the `HAVE_STRUCT_TIMESPEC` check
   That will never happen, because `USE_GLOBAL_CONTEXT` is defined inside `ndpi_config.h`.
   It's better to use `CFLAGS` to achieve the same.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-03-15 12:47:13 +01:00
Ivan Nardi 231748bb0e LRU cache: move to 64 bits long keys (#2346)
Tradeoff between key comparison efficiency (i.e. no `memcmp`) and key
length.
At least in the ipv4 cases, we have no more different entries with the
same key.
2024-03-15 10:12:51 +01:00
Ivan Nardi 97fae6e00a STUN: fix parsing of DATA attribute (#2345) 2024-03-14 14:20:51 +01:00
Ivan Nardi 37ddf02e63 bitmap64: fix memory access error (#2344)
```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==29723==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x562910b70ddb bp 0x7ffcb22c5b70 sp 0x7ffcb22c5a80 T0)
==29723==The signal is caused by a READ memory access.
==29723==Hint: address points to the zero page.
    #0 0x562910b70ddb in binary_fuse16_contain /home/ivan/svnrepos/nDPI/src/lib/./third_party/include/binaryfusefilter.h:492:8
    #1 0x562910b70bbe in ndpi_bitmap64_isset /home/ivan/svnrepos/nDPI/src/lib/ndpi_bitmap64.c:178:10
    #2 0x562910788fd3 in ndpi_domain_classify_longest_prefix /home/ivan/svnrepos/nDPI/src/lib/ndpi_domain_classify.c:261:5
    #3 0x56291078940e in ndpi_domain_classify_contains /home/ivan/svnrepos/nDPI/src/lib/ndpi_domain_classify.c:291:9
    #4 0x56291069a392 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ds_domain_classify.cpp:52:5
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67369
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67372
2024-03-14 14:20:35 +01:00
Ivan Nardi 25c145f26d CI actions: fix Ubuntu jobs with sanitizers (#2347)
See: https://github.com/actions/runner-images/issues/9491
2024-03-14 14:20:21 +01:00
Vladimir Gavrilov 8fad77991d Add PFCP protocol dissector (#2342) 2024-03-13 20:18:43 +01:00
Ivan Nardi 4102c8df72 CI: fix build on MacOS-13 runners (#2343)
Workaroud for Homebrew's python link error
See: https://github.com/Homebrew/homebrew-core/issues/165793#issuecomment-1991817938
2024-03-13 19:22:13 +01:00
Luca Deri 1aedbefffa Fixes bitmap memory calculation 2024-03-11 17:59:24 +01:00
Ivan Nardi 6152d595e8 STUN: add a parameter to configure how long the extra dissection lasts (#2336)
Tradeoff: performance (i.e. number of packets) vs sub-classification
2024-03-07 14:39:32 +01:00
Ivan Nardi 56ce228a8b Add a specific protocol id for audio/video calls made using Google apps (#2341)
Same logic already used for Signal/Whatsapp/Line/Facebook/...
2024-03-07 13:48:19 +01:00
Ivan Nardi c09b4dacdf Fix memory leak (#2340)
```
Direct leak of 12 byte(s) in 1 object(s) allocated from:
    #0 0x55779e1a46ff in malloc (/home/ivan/svnrepos/nDPI/example/ndpiReader+0x8706ff) (BuildId: 14c2fc626744710d49d652ea1c5bbb24a8cbab4f)
    #1 0x55779e2120c7 in ndpi_malloc_wrapper /home/ivan/svnrepos/nDPI/example/ndpiReader.c:298:10
    #2 0x55779e5fa215 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
    #3 0x55779e5fa500 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13
    #4 0x55779e42153c in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2554:46
    #5 0x55779e4359a1 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:908:5
    #6 0x55779e432de7 in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1097:2
    #7 0x55779e4133f9 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2913:5
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67250
2024-03-07 13:11:13 +01:00
Toni 1fb4719e59 Disable AX_PTHREAD for MingW/MSYS builds. (#2338)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-03-07 13:09:01 +01:00
Vladimir Gavrilov e2949048e0 Add Path of Exile protocol dissector (#2337)
* Add Path of Exile protocol dissector

* Update protocols.rst
2024-03-06 19:59:09 +01:00
Ivan Nardi 21da53d3a0 ahocorasick: improve matching with subdomains (#2331)
The basic idea is to have the following logic:
* pattern "DOMAIN" matches the domain itself (i.e exact match) *and* any
subdomains (i.e. "ANYTHING.DOMAIN")
* pattern "DOMAIN." matches *also* any strings for which is a prefix
[please, note that this kind of match is handy but it is quite
dangerous...]
* pattern "-DOMAIN" matches *also* any strings for which is a postfix

Examples:
* pattern "wikipedia.it":
  * "wikipiedia.it" -> OK
  * "foo.wikipedia.it -> OK
  * "foowikipedia.it -> NO MATCH
  * "wikipedia.it.com -> NO MATCH
* pattern "wikipedia.":
  * "wikipedia.it" -> OK
  * "foo.wikipedia.it -> OK
  * "foowikipedia.it -> NO MATCH
  * "wikipedia.it.com -> OK
* pattern "-wikipedia.it":
  * "wikipedia.it" -> NO MATCH
  * "foo.wikipedia.it -> NO MATCH
  * "0001-wikipedia.it -> OK
  * "foo.0001-wikipedia.it -> OK

Bottom line:
* exact match
* prefix with "." (always, implicit)
* prefix with "-" (only if esplicitly set)
* postfix with "." (only if esplicitly set)

That means that the patterns cannot start with '.' anymore.

Close #2330
2024-03-06 19:25:59 +01:00
Luca Deri 8f63a11735 Extended connectivity checks 2024-03-05 17:45:46 +01:00
Luca Deri 65b9ad88a9 Improved alert on suspicious DNS traffic 2024-03-05 11:14:34 +01:00
Vladimir Gavrilov e7f1946ae7 Add NetEase Games detection support (#2335) 2024-03-05 11:00:52 +01:00
Vladimir Gavrilov 58fdc9fafb Add Naraka Bladepoint detection support (#2334) 2024-03-04 08:30:54 +01:00
Luca Deri 70aae6e998 Added DGA exception for Dropbox 2024-03-03 22:14:34 +01:00
Luca Deri e0587fe75e indent 2024-03-03 22:12:13 +01:00
Vladimir Gavrilov f2e3c7fb90 Add BFD protocol dissector (#2332) 2024-02-29 08:19:00 +01:00
Ivan Nardi c2b5b48fc8 ndpiReader: restore ndpiReader -x $DOMAIN_NAME functionality (#2329) 2024-02-26 20:50:07 +01:00
Nardi Ivan b287d6ec85 TLS: avoid setting some flow risks for webrtc traffic
Is quite rare to have a SNI or an ALPN on Client Hello of STUN/DTLS/SRTP
traffic
2024-02-26 10:20:05 +01:00
Nardi Ivan 478c31b6c9 utils: update script to download Cloudflare ips 2024-02-26 09:26:21 +01:00
Nardi Ivan ed5ba179f6 Telegram: improve identification
Follow up of 31c706c3db and
75485e177c.

Allow fast classification by ip, but give time to other dissectors to
kick in (for example, the TLS code for the Telegram Web flows).

Even if we don't classify it anymore at the very first packet (i.e. SYN)
we fully classify Telegram traffic at the first packet with payload, as
*any* other protocol.
This way, we always have the proper category, the proper confidence
for the UDP flows and we don't overwrite previous classifications (TLS
or ICMP)

Remove old and stale identification logic for TCP flows
2024-02-26 09:26:21 +01:00
Ivan Nardi c95e8c184e STUN: fix category when sub-classification is set in "extra-dissection" data path (#2320) 2024-02-24 09:44:24 +01:00
Luca Deri 71b7afccf5 Updated telegam out 2024-02-23 16:22:13 +01:00
Luca Deri 75485e177c Improved Telegram detection 2024-02-23 16:15:29 +01:00
Luca Deri bfb21dc09b Fixes exception handling glitch 2024-02-22 23:19:01 +01:00
Luca Deri 31c706c3db Improved telegram detection 2024-02-22 22:23:23 +01:00
Luca Deri 2399abd272 Added missing telegram networks 2024-02-22 22:04:54 +01:00
Vladimir Gavrilov 66b6e2b3f2 Add DLEP protocol dissector (#2326) 2024-02-20 16:05:41 +01:00
Ivan Nardi 4f2e642fbe make install: avoid copying private header (#2323) 2024-02-20 15:39:25 +01:00
Ivan Nardi a1280a2fd1 Move some defines (expecially log related) to the private header (#2324) 2024-02-20 15:35:38 +01:00
Ivan Nardi 12e142565e Add a script to download/update the domain suffix list (#2321) 2024-02-20 11:51:58 +01:00
Ivan Nardi 2f814c526b Add identification of Huawei generic and cloud traffic (#2325) 2024-02-20 11:51:40 +01:00
Ivan Nardi ffd6c3edee TLS: fix disabling of JA3C fingerprint (#2319) 2024-02-19 11:11:57 +01:00
Luca Deri 23536ff77a Improved modbus dissection to discard false positives 2024-02-16 15:48:34 +01:00
Luca Deri d1aed7b391 Indentation 2024-02-16 15:31:01 +01:00
Vladimir Gavrilov e93bcfd619 Add ANSI C12.22 protocol dissector (#2317)
* Add ANSI C12.22 protocol dissector

* Add UDP sample
2024-02-15 09:36:06 +01:00
Ivan Nardi ae36648c6c Skype: remove old detection logic (#1954)
Skype has been using standard protocols (STUN/ICE or TLS) for a long,
long time, now. Long gone are the days of Skype as a distribuited
protocol.

See: #2166
2024-02-12 18:39:24 +01:00
Nardi Ivan 0d36d9c8c2 Remove spurious call to exit() 2024-02-12 17:10:45 +01:00
Luca Deri 4072cb8862 Added stress test 2024-02-11 22:35:33 +01:00
Toni f4d7aa45fe Improved Polish gambling sites fetch script. (#2315)
* fails quite often in the CI, so ignore potential xmllint error

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-10 13:33:56 +01:00
Ivan Nardi ad25affcb7 reader_util: fix GRE detunneling (#2314) 2024-02-10 09:16:27 +01:00
Ivan Nardi 31f0608b9a TLS: add configuration of JA* fingerprints (#2313) 2024-02-10 09:16:13 +01:00
Ivan Nardi 03ecb026ff fuzz: improve fuzzing coverage (#2309) 2024-02-09 19:19:03 +01:00
Vladimir Gavrilov f5cec001f3 Add detection of Gaijin Entertainment games (#2311)
* Add detection of Gaijin Entertainment games

* Short NDPI_PROTOCOL_GAIJINENTERTAINMENT to NDPI_PROTOCOL_GAIJIN

* Add default UDP port for Gaijin Entertainment games

* Remove NDPI_PROTOCOL_CROSSOUT protocol id
2024-02-09 11:07:48 +01:00
Ivan Nardi f352e4f77b Improve normalization of flow->host_server_name (#2310)
Follow-up of 4543385d10

Remove trailing spaces for any HTTP header (we already remove leading
spaces)

We want:
* a "normalized" string in `flow->host_server_name`, but
* to parse the original string for flow risk checking

`ndpi_hostname_sni_set()` is a private function, so there is no need to
export its flags.
2024-02-09 09:20:02 +01:00
Vladimir Gavrilov 2abb68c8ca Add new AppsFlyer domain (#2307) 2024-02-08 08:10:55 +01:00
Vladimir Gavrilov 6207be43fa Add TencentGames protocol dissector (#2306) 2024-02-08 08:10:35 +01:00
Vitaly Lavrov 4543385d10 Normalization of host_server_name (#2299)
* Normalization of host_server_name

The ndpi_hostname_sni_set() function replaces all non-printable
characters with the "?" character and removing whitespace characters
at the end of the line.

* Added conditional hostname normalization.
2024-02-05 19:44:45 +01:00
Luca Deri a4da4467ac Fixed incompatibity with RH7 introduced by https://github.com/ntop/nDPI/commit/02030ac16e5016b00e0da6ff7512d97751bf87d2 2024-02-05 18:12:42 +01:00
Luca Deri b86452f2b1 Removed un-necessary include that caused configure-based applications sitting on top of nDPI to faile at compilation 2024-02-05 17:01:21 +01:00
Ivan Nardi 66371c202f Fix ndpi_get_lru_cache_stats() (#2303)
Found while fuzzing
2024-02-05 12:38:33 +01:00
Toni f2b09014f9 Serialize JA4 to JSON. (#2305)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-05 12:35:38 +01:00
Toni 02030ac16e Build RPM package in the CI. (#2304)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-05 12:13:09 +01:00
Luca Deri b15b877c51 Minor hash improvement 2024-02-04 20:23:49 +01:00
Toni b59994fa20 Completly disable all pthread related code in the library if USE_GLOBAL_CONTEXT macro is not defined. (#2302)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-03 11:53:49 +01:00
Ivan Nardi 9af2b9d815 collectd: fix extraction of hostname (#2301) 2024-02-03 04:11:58 +01:00
Toni ede25cc2b3 Improve ndpi_set_config error printing. (#2300)
* exit `ndpiReader` if a invalid configuration setting detected

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-02 15:15:30 +01:00
Toni f8aa642c3c Simplify and fix JA4 string computation. (#2298)
* additional JA4 string buffer is not needed and may cause a string truncation warning

```
protocols/tls.c: In function ‘ndpi_compute_ja4’:
protocols/tls.c:1738:3: warning: ‘strncpy’ output may be truncated copying 36 bytes from a string of length 1023 [-Wstringop-truncation]
 1738 |   strncpy(flow->protos.tls_quic.ja4_client, ja_str, 36);
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-02 14:25:43 +01:00
Ivan Nardi 13212711ab CIP: fix infinite-loop (#2295)
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=66342
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=66320
2024-02-01 20:30:43 +01:00
Ivan Nardi 8fbef7fb5a GitHub Actions: update to latest macOS runners (#2293)
Add a simple job with macos-14 on M1.
https://github.blog/changelog/2024-01-30-github-actions-introducing-the-new-m1-macos-runner-available-to-open-source/)
There are some issues with external dependencies (they are installed but
autoconf script doens;t find them) so keep it simple.

On macos-13 it seems that:
* there is no `realpath` program (even if coreutils has been
installed...)
* most of the filesystem is read only (we can't write on /usr/lib).
So I change
```
make install DESTDIR=$(realpath _install)
ls -alhHR _install
```
to
```
DESTDIR=/tmp/ndpi make install
ls -alhHR /tmp/ndpi
```
for all the jobs

Fix a warning on GitHub logs:
```
Node.js 16 actions are deprecated. Please update the following actions
to use Node.js 20: actions/checkout@v3. For more information see:
https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
```
2024-02-01 20:30:27 +01:00
Vladimir Gavrilov 4c9446379f Add Gearman protocol dissector (#2297) 2024-02-01 19:46:57 +01:00
Ivan Nardi 6c5ecae607 Fix function parameters check on domain code (#2296)
Found while fuzzing fuzz_config
2024-02-01 17:06:16 +01:00
Nardi Ivan c78ed93ec3 Sync unit tests results 2024-02-01 15:45:36 +01:00
Ivan Nardi 400cd516b5 Allow multiple struct ndpi_detection_module_struct to share some state (#2271)
Add the concept of "global context".

Right now every instance of `struct ndpi_detection_module_struct` (we
will call it "local context" in this description) is completely
independent from each other. This provide optimal performances in
multithreaded environment, where we pin each local context to a thread,
and each thread to a specific CPU core: we don't have any data shared
across the cores.

Each local context has, internally, also some information correlating
**different** flows; something like:
```
if flow1 (PeerA <-> Peer B) is PROTOCOL_X; then
  flow2 (PeerC <-> PeerD) will be PROTOCOL_Y
```
To get optimal classification results, both flow1 and flow2 must be
processed by the same local context. This is not an issue at all in the far
most common scenario where there is only one local context, but it might
be impractical in some more complex scenarios.

Create the concept of "global context": multiple local contexts can use
the same global context and share some data (structures) using it.
This way the data correlating multiple flows can be read/write from
different local contexts.
This is an optional feature, disabled by default.

Obviously data structures shared in a global context must be thread safe.
This PR updates the code of the LRU implementation to be, optionally,
thread safe.

Right now, only the LRU caches can be shared; the other main structures
(trees and automas) are basically read-only: there is little sense in
sharing them. Furthermore, these structures don't have any information
correlating multiple flows.

Every LRU cache can be shared, independently from the others, via
`ndpi_set_config(ndpi_struct, NULL, "lru.$CACHE_NAME.scope", "1")`.

It's up to the user to find the right trade-off between performances
(i.e. without shared data) and classification results (i.e. with some
shared data among the local contexts), depending on the specific traffic
patterns and on the algorithms used to balance the flows across the
threads/cores/local contexts.

Add some basic examples of library initialization in
`doc/library_initialization.md`.

This code needs libpthread as external dependency. It shouldn't be a big
issue; however a configure flag has been added to disable global context
support. A new CI job has been added to test it.

TODO: we should need to find a proper way to add some tests on
multithreaded enviroment... not an easy task...

*** API changes ***

If you are not interested in this feature, simply add a NULL parameter to
any `ndpi_init_detection_module()` calls.
2024-02-01 15:33:11 +01:00
Toni 44c2e59661 Provide a u64 wrapper for ndpi_set_config() (#2292)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-30 17:13:35 +01:00
Toni Uhlig d2f22d1308 Updated results (missing cip_io.pcap.out)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-29 20:49:47 +01:00
Luca Deri 47f72443fa Implemented CIP I/O (UDP version of the CIP protocol), Common Industrial protocol 2024-01-29 13:28:41 +01:00
Luca Deri 6c08601c3d Updated result 2024-01-29 12:27:52 +01:00
Luca Deri 426494b4ba Fixes invalid skype detection on UDP 2024-01-29 12:27:52 +01:00
Luca Deri 14cfa174be Added public_suffix_list.dat in RPM package 2024-01-29 11:48:39 +01:00
Luca Deri fb58f838e5 Added public_suffix_list.dat in packages 2024-01-29 11:43:23 +01:00
Luca Deri ae501e2503 Enhanced exception lists for binary application transfer 2024-01-29 11:37:04 +01:00
Ivan Nardi 92c2ac5a0f fuzz: fuzz_config: try restoring good coverage (#2291)
Last changes reduce fuzzing coverage of this fuzzer :(
2024-01-29 10:53:28 +01:00
Vladimir Gavrilov fb095a339d Add missing NDPI_LOG_DBG in some dissectors (#2290)
* Add missing NDPI_LOG_DBG

* Add missing NDPI_LOG_DBG in yojimbo.c
2024-01-29 09:25:29 +01:00
Vladimir Gavrilov c807d84054 Fix RESP detection (#2289)
* Rename redis_net.c to resp.c

* Fix RESP detection
2024-01-27 21:19:34 +01:00
Luca Deri 8e562c4ecd Removed check fir skipping .arpa domains 2024-01-27 20:54:46 +01:00
Luca Deri ec83a592b8 Check DGA on domain name (as it should be) rather than on the host name 2024-01-27 20:46:00 +01:00
Luca Deri 7a7df2b7d7 Fixed warning 2024-01-27 20:45:52 +01:00
Luca Deri 65b9c68d7d Fixed loading of non-ICANN domains that caused false positives with ndpi_load_domain_suffixes
Minor hash optimization
2024-01-27 20:40:27 +01:00
Toni bcca89b78c Add Raft protocol dissector. (#2286)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-25 20:26:18 +01:00
Vladimir Gavrilov 9a0affc6aa Improve MySQL detection (#2279)
* Improve MySQL detection

* Update copyright
2024-01-25 11:12:08 +01:00
Ivan Nardi b3efa7d3fe fuzz: fuzz_config: we need bigegr inputs (#2285) 2024-01-25 09:58:20 +01:00
Vladimir Gavrilov 4e712e3ab5 Add Radmin protocol dissector (#2283)
* Add Radmin protocol dissector

* Update test results
2024-01-25 08:10:29 +01:00
Ivan Nardi d577508727 fuzz: extend fuzzing coverage (#2281) 2024-01-24 21:16:58 +01:00
Ivan Nardi 7a83a8dc91 QUIC: fix decryption with CH fragments with different Destination CID (#2278)
QUIC decryption fails when the Client Hello is split into multiple UDP
packets and these packets have different Destination Connection IDs
(because the server told the client to switch to a different CID; see
RFC 9000 7.2)

```
The Destination Connection ID field from the first Initial packet sent by
a client is used to determine packet protection keys for Initial packets.
[..]
Upon first receiving an Initial or Retry packet from the server, the
client uses the Source Connection ID supplied by the server as the
Destination Connection ID for subsequent packets

```

From a logical point of view, the ciphers used for decryption should be
initialized only once, with the first Initial pkt sent by the client and
kept for later usage with the following packets (if any).

However it seems that we can safely initialize them at each packet, if
we keep using the DCID of the **first** packet sent by the client.
Keep initializing the ciphers at each packet greatly simplifie this patch.

This issue has been undetected for so long because:
* in the vast majority of the cases we only decrypt one packet per flow;
* the available traces with the Client Hello split into multiple packets
(i.e. cases where we need to decrypt at least two packets per flow) were
created in a simple test environment to simulate Post-Quantum handshake,
and in that scenario the client sent all the packets (with the same
DCID) before any reply from the server.

However, in the last months all major browsers started supporting PQ
key, so it is now common to have split CH in real traffic.

Please note that in the attached example, the CH is split into 2
(in-order) fragments (in different UDP packets) and the second one in
turn is divided into 9 (out-of-order) CRYPTO frames; the reassembler
code works out-of-the-box even in this (new) scenario.
2024-01-24 09:57:28 +01:00
Vladimir Gavrilov f04b4450a1 Add STOMP protocol dissector (#2280) 2024-01-23 21:08:13 +01:00
Ivan Nardi 9b26e74bb7 example: rework code between ndpiReader.c and reader_util.c (#2273) 2024-01-22 18:12:06 +01:00
Ivan Nardi 82e8bf91dd Improve handling of custom rules (#2276)
Avoid collisions between user-ids and internal-ids protocols in the
`example/protos.txt` file.
Add a new value for the classification confidence:
`NDPI_CONFIDENCE_CUSTOM_RULE`

With `./example/ndpiReader -p example/protos.txt -H` we now see also the
custom protocols and their internal/external ids:

```
nDPI supported protocols:
 Id Userd-id Protocol               Layer_4    Nw_Proto Breed        Category
  0        0 Unknown                TCP        X        Unrated      Unspecified

...

387      387 Mumble                 UDP        X        Fun          VoIP
388      388 iSCSI                  TCP                 Acceptable   Unspecified
389      389 Kibana                 TCP                 Acceptable   Unspecified
390      390 TestProto              TCP                 Acceptable   Unspecified
391      391 HomeRouter             TCP                 Acceptable   Unspecified
392      392 CustomProtocol         TCP                 Acceptable   Unspecified
393      393 AmazonPrime            TCP                 Acceptable   Unspecified
394      394 CustomProtocolA        TCP                 Acceptable   Unspecified
395      395 CustomProtocolB        TCP                 Acceptable   Unspecified
396      800 CustomProtocolC        TCP                 Acceptable   Unspecified
397     1024 CustomProtocolD        TCP                 Acceptable   Unspecified
398     2048 CustomProtocolE        TCP                 Acceptable   Unspecified
399     2049 CustomProtocolF        TCP                 Acceptable   Unspecified
400     2050 CustomProtocolG        TCP                 Acceptable   Unspecified
401    65535 CustomProtocolH        TCP                 Acceptable   Unspecified
```

We likely need to take a better look in general at the iteration between
internal and external protocols ids...

This PR fixes the issue observed in
https://github.com/ntop/nDPI/pull/2274#discussion_r1460674874 and in
https://github.com/ntop/nDPI/pull/2275.
2024-01-21 19:53:32 +01:00
Vladimir Gavrilov 5620e10742 Add ElectronicArts detection support (#2274)
* Add ElectronicArts detection support

* Merge electronicarts.pcapng into sites.pcapng
2024-01-21 18:58:12 +01:00
Toni 7d24e1258d Add Yojimbo (netcode) protocol dissector (#2277)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-21 17:47:42 +01:00
Ivan Nardi 49591dda9b Zoom: improve detection (#2270) 2024-01-21 12:17:34 +01:00
Ivan Nardi 42d23cff6a config: follow-up (#2268)
Some changes in the parameters names.
Add a fuzzer to fuzz the configuration file format.
Add the infrastructure to configuratin callbacks.
Add an helper to map LRU cache indexes to names.
2024-01-20 16:14:41 +01:00
Vladimir Gavrilov 8651ce9811 Fix detection of new Cassandra versions (#2272)
* Fix detection of new Cassandra versions

* Add Cassandra Internode Communication protocol support

* Add default port for Cassandra Internode Communication protocol
2024-01-20 15:58:14 +01:00
Ivan Nardi eb129297e9 Add a dedicated dissector for Zoom (#2265)
Move it from the RTP code and extend it
2024-01-19 10:01:38 +01:00
Ivan Nardi c83698c957 STUN: fix flow risks when DTLS packets are found (#2266)
When switching to (D)TLS dissector from the STUN one, we need to clear
any flow risks set from the latter (because we don't have anymore
`NDPI_PROTOCOL_STUN` in the classification results)
2024-01-19 08:57:39 +01:00
Vladimir Gavrilov 6ac2ce84f8 Add Mumble detection support (#2269) 2024-01-19 07:46:51 +01:00
Vladimir Gavrilov 248f3d5588 Rework Steam detection (part 1) (#2264)
* Clean up Steam dissector

* Add Steam Datagram Relay dissector

* Update docs

* Update test results

* Remove csgo.c from MSVC project

* Small fixes

* Add Steam TLS pcap sample

* Merge Steam pcap samples into single one

* Fix typo

* Update test results
2024-01-18 19:23:44 +01:00
Nardi Ivan 0712d496fe config: allow configuration of guessing algorithms 2024-01-18 10:21:24 +01:00
Nardi Ivan 6c85f10cd5 config: move debug/log configuration to the new API 2024-01-18 10:21:24 +01:00
Nardi Ivan c704be1a20 config: DNS: add two configuration options
* Enable/disable sub-classification of DNS flows
* Enable/disable processing of DNS responses
2024-01-18 10:21:24 +01:00
Nardi Ivan 950f209a17 config: HTTP: enable/disable processing of HTTP responses 2024-01-18 10:21:24 +01:00
Nardi Ivan c669044a44 config: configure TLS certificate expiration with the new API 2024-01-18 10:21:24 +01:00
Nardi Ivan 88720331ae config: remove enum ndpi_prefs 2024-01-18 10:21:24 +01:00
Nardi Ivan 1289951b32 config: remove ndpi_set_detection_preferences() 2024-01-18 10:21:24 +01:00
Nardi Ivan 311d8b6dae config: move cfg of aggressiviness and opportunistic TLS to the new API 2024-01-18 10:21:24 +01:00
Nardi Ivan 4cbe2674ab config: move IP lists configurations to the new API 2024-01-18 10:21:24 +01:00
Nardi Ivan f55358973f config: move LRU cache configurations to the new API 2024-01-18 10:21:24 +01:00
Nardi Ivan 3107a95881 Make ndpi_finalize_initialization() returns an error code
We should check if the initialization was fine or not
2024-01-18 10:21:24 +01:00
Nardi Ivan d72a760ac3 New API for library configuration
This is the first step into providing (more) configuration options in nDPI.

The idea is to have a simple way to configure (most of) nDPI: only one
function (`ndpi_set_config()`) to set any configuration parameters
(in the present or on in the future) and we try to keep this function
prototype as agnostic as possible.

You can configure the library:
* via API, using `ndpi_set_config()`
* via a configuration file, in a text format

This way, anytime we need to add a new configuration parameter:
* we don't need to add two public functions (a getter and a setter)
* we don't break API/ABI compatibility of the library; even changing
the parameter type (from integer to a list of integer, for example)
doesn't break the compatibility.

The complete list of configuration options is provided in
`doc/configuration_parameters.md`.

As a first example, two configuration knobs are provided:
* the ability to enable/disable the extraction of the sha1 fingerprint of
the TLS certificates.
* the upper limit on the number of packets per flow that will be subject
to inspection
2024-01-18 10:21:24 +01:00
Luca ca7df1db82 Improved ndpi_get_host_domain 2024-01-16 07:25:03 +01:00
Luca 1637a991a4 Added ndpi_get_host_domain() for returning the host domain
vs ndpi_get_host_domain_prefix() that instead returnd the host TLD
2024-01-16 06:56:51 +01:00
Ivan Nardi 111015b872 ndpiReader: improve the check on max number of pkts processed per flow (#2261)
Allow to disable this check.

I don't know how much sense these limits have in the application
(especially with those default values...) since we have always had a
hard limit on the library itself (`max_packets_to_process` set to 32).
The only value might be that they provide different limits for TCP and
UDP traffic.

Keep them for the time being...
2024-01-15 20:12:57 +01:00
Nardi Ivan 0891ad0e77 Added missing lists file to the distribution tarball 2024-01-15 20:00:36 +01:00
Nardi Ivan b22fa558ff ndpiReader: fix memory leak
Change the working directory of `ndpiReader` in the Github Actions so
that it can load the domain suffix list during `domainsUnitTest()`
2024-01-15 19:49:27 +01:00
Luca d29c2f3ad1 Added ndpi tracing 2024-01-15 19:26:28 +01:00
Luca a1634604e1 Removes extraneous parentheses that caused macOS to complain 2024-01-15 19:12:31 +01:00
Luca 162c38f18f Added new API calls
- ndpi_load_domain_suffixes()
- ndpi_get_host_domain_suffix()

whose goal is to find the domain name of a hostname. Example:

www.bbc.co.uk   -> co.uk
mail.apple.com  -> com
2024-01-15 19:03:46 +01:00
Ivan Nardi 61a18623e7 Bittorrent: improve detection of UTPv1 (#2259)
* fix header length check: we must take into account optional extension
header
* it seems the check on UTPv1 header is weak only in the DATA case: wait
for multiple packets only in that case
2024-01-12 18:53:24 +01:00
Ivan Nardi dd8be1fcb1 Fix some warnings reported by CODESonar (#2227)
Remove some unreached/duplicated code.

Add error checking for `atoi()` calls.

About `isdigit()` and similar functions. The warning reported is:
```
Negative Character Value help
isdigit() is invoked here with an argument of signed type char, but only
has defined behavior for int arguments that are either representable
as unsigned char or equal to the value of macro EOF(-1).
Casting the argument to unsigned char will avoid the undefined behavior.
In a number of libc implementations, isdigit() is implemented using lookup
tables (arrays): passing in a negative value can result in a read underrun.
```
Switching to our macros fix that.
Add a check to `check_symbols.sh` to avoid using the original functions
from libc.
2024-01-12 13:30:43 +01:00
Toni 0aea509e23 Add KCP protocol dissector. (#2257)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-12 12:14:59 +01:00
Vladimir Gavrilov 75d3f78f7d Add more tracker/ad hosts (#2256) 2024-01-12 10:44:31 +01:00
Toni 7b2bbb2309 Improved uTorrent via utp (TCP-like streams over UDP). (#2255)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-11 16:49:25 +01:00
Luca Deri 30c2386ee2 Added missing (C) 2024-01-09 22:27:41 +01:00
Luca Deri 4f9a08cf05 Moved ndpi_private.h to include directory 2024-01-09 22:00:40 +01:00
Vladimir Gavrilov 3d57dec6b4 Add PIA (Private Internet Access) support (#2250)
* Fix typo in protocols.rst

* Add PIA (Private Internet Access) support

* Update ndpi_main.c
2024-01-09 18:28:57 +01:00
Vladimir Gavrilov ead5bf2adb Add more adult content hostnames (#2247) 2024-01-09 07:57:01 +01:00
Toni c5b0b05b80 Add Roughtime protocol dissector. (#2248)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-09 07:56:27 +01:00
Toni 6c3d162cd6 Add realtime protocol output to ndpiReader. (#2197)
* support for using a new flow callback invoked before the flow memory is free'd
 * minor fixes

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-09 00:39:59 +01:00
Ivan Nardi 90b999e3f1 Fix default port conflicts between AJP and CiscoVPN (#2245)
We shouldn't have the same default port associated with multiple
protocols. The key reason is that we use only one of them (for
classification by-port, for example) and which one we really use
depends on the internal order of the protocolsi's initialization (which
is unknown to the user).

We have 3 port collisions:
* 8009: AJP and CiscoVPN
* 445: SMBv1 and SMBv23
* 8080: HTTP_Connect and HTTP_Proxy
I think that the two last cases don't really matter...

About the first one: it seems that CiscoVPN is retired from 2011/2014
(see
https://www.cisco.com/c/en/us/obsolete/security/cisco-vpn-client.html)
and superseded by AnyConnect, which uses different ports (see
https://community.cisco.com/t5/network-security/what-protocols-are-used-on-anyconnect-mobility-client/td-p/4713525).
Therefore it should be safe to remove 8009 from the CiscoVPN default
ports.
2024-01-08 17:04:46 +01:00
Ivan Nardi 8321b79539 Make some test traces smaller (#2243)
Having smaller traces help fuzzing: we want the fuzzers to mutate
"interesting" packets analyzed by nDPI, i.e. the first packets of each
flows.
Try hard to keep the same classification and extraction capabilities
2024-01-08 17:04:12 +01:00
Vladimir Gavrilov ce08291ccd Add Google Chat support (#2244) 2024-01-07 18:04:59 +01:00
Ivan Nardi 40797521af ndpiReader: add breed stats on output used for CI (#2236) 2024-01-05 13:02:39 +01:00
Vladimir Gavrilov 3d09b25653 Add Ceph protocol dissector (#2242)
* Add Ceph protocol dissector

* Update protocols.rst
2024-01-04 13:22:23 +01:00
Ivan Nardi f23e9dc7bb Add an implementation of the BSD function strtonum (#2238)
The main difference with the original function is that we allow to
specify the base.
Credit for the original idea and the first implementation to @0xA50C1A1
2024-01-04 13:16:39 +01:00
Vladimir Gavrilov 7f9973bd0c Add HL7 protocol dissector (#2240)
* Add HL7 protocol dissector

* Small fixes

* Small fixes
2024-01-02 20:57:05 +01:00
Ivan Nardi 88d1416b70 STUN: fix detection of Google Meet over IPv6 (#2241) 2024-01-02 19:30:59 +01:00
Vladimir GavrilovandNardi Ivan 0180c1f04a Add IEC62056 (DLMS/COSEM) protocol dissector (#2229)
* Add IEC62056 (DLMS/COSEM) protocol dissector

* Fix detection on big endian architectures

* Update protocols.rst

* Add ndpi_crc16_x25 to fuzz/fuzz_alg_crc32_md5.c

* Update pcap sample

* Remove empty .out file

* iec62056: add some documentation

---------

Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2024-01-02 16:45:54 +01:00
Ivan Nardi 3c7ed34ce9 fuzz: improve fuzzing coverage (#2239) 2024-01-02 15:22:44 +01:00
Vladimir Gavrilov 0f4d9f5054 Remove Google Hangouts/Duo stuff (#2233)
* Remove Google Hangouts/Duo support

* Update protocols.rst
2024-01-02 14:01:33 +01:00
Ivan Nardi 3dbd89eee8 sha256: fix undefined-shift error (#2237)
```
third_party/src/ndpi_sha256.c:51:21: runtime error: left shift of 128 by 24 places cannot be represented in type 'int'
```

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65305
2024-01-02 12:29:20 +01:00
Ivan Nardi 308f71a6e8 TLS: fix heap-buffer-overflow (#2235)
```
==53992==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x511000007e04 at pc 0x555da2165fd0 bp 0x7ffddf7e3990 sp 0x7ffddf7e3988
READ of size 2 at 0x511000007e04 thread T0
    #0 0x555da2165fcf in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2384:50
    #1 0x555da217c31f in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:908:5
    #2 0x555da2176720 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1273:11
    #3 0x555da215a628 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2883:5
    #4 0x555da1e95c30 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6720:6
    #5 0x555da1e969f3 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6756:10
    #6 0x555da1e96394 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6789:12
    #7 0x555da1ea7991 in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7929:15
    #8 0x555da1ea547f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8104:22
    #9 0x555da1de137f in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1721:31
    #10 0x555da1de137f in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2438:1
```

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65362
2024-01-02 11:37:40 +01:00
Ivan Nardi d886a6107f Teamviewer: varius fixes (#2228)
We already have a generic (and up to date) logic to handle ip addresses:
remove that stale list.

Teamviewer uses TCP and UDP, both; we can't access `flow->l4.udp`.

According to a comment, we set the flow risk
`NDPI_DESKTOP_OR_FILE_SHARING_SESSION` only for the UDP flows.
2024-01-02 11:22:43 +01:00
Vladimir Gavrilov 2796bc9b47 Add NoMachine NX protocol dissector (#2234)
* Add NoMachine protocol dissector

* Fix detection on big endian architectures

* Make NoMachine over UDP check more strict

* Small fixes
2024-01-02 10:23:42 +01:00
Luca 2f657cb8f9 Implemented ndpi_is_outlier() for detecting outliers using z-score 2023-12-28 19:59:54 +01:00
Luca Deri 1366518bff Implements ndpi_pearson_correlation for measuring how correlated are two series 2023-12-27 22:42:37 +01:00
Luca Deri 99d4838328 New ndpi_sha256() nDPI API call (#2230)
* Added ndpi_sha256.c to the Windows project

* Added ndpi_sha256() nDPI API call
2023-12-23 10:47:28 +01:00
Luca Deri f2be972224 Removed type redefinition 2023-12-23 09:16:35 +01:00
Luca Deri caa223d2ba Changes to avoid type redefinition on windows 2023-12-23 09:15:42 +01:00
Luca Deri 5c7200f2bb Buffer overflow fix 2023-12-23 08:07:28 +01:00
Luca Deri 7c4be6d077 Updated result 2023-12-22 21:29:45 +01:00
Luca Deri 53b9ae23e8 Warning fix 2023-12-22 21:10:03 +01:00
Luca Deri cbea1b2203 Fixed check (when using ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --with-thread-sanitizer --with-pcre2 --with-maxminddb --enable-tls-sigs) 2023-12-22 21:01:14 +01:00
Luca Deri a95d4744b6 Fixes nDPI datastructs size check 2023-12-22 20:52:56 +01:00
Luca Deri 8285fffdae Implements JA4 Support (#2191) 2023-12-22 20:40:42 +01:00
Luca Deri b90c18e906 Changes to make room for JA4 2023-12-22 20:40:42 +01:00
Vladimir Gavrilov 5eb468d07b Add Apache Kafka protocol dissector (#2226) 2023-12-22 14:42:47 +01:00
Vladimir Gavrilov 6fc8aa4e61 Add WebDAV detection support (#2224)
* Add WebDAV detection support

* Add pcap example

* Update test results

* Remove redundant checks

* Add WebDAV related HTTP methods to fuzz/dictionary.dict

* Add note about WebDAV
2023-12-22 13:23:37 +01:00
Luca 41aa2f96b6 Various MDNS flow risks fixes 2023-12-21 12:24:30 +01:00
Luca 28b1ce41cb MDNS fix 2023-12-21 11:11:28 +01:00
Ivan Nardi a5595d16c0 CI: update list of compilers (#2223)
Try using latest gcc and clang versions.
We still care about RHEL7: since handling a RHEL7 runner on GitHub is
quite complex, let try to use a similar version of gcc, at least
2023-12-20 19:22:22 +01:00
Vladimir Gavrilov 149067b3fc Add JSON-RPC protocol dissector (#2217)
* Add JSON-RPC protocol dissector

* Small fixes

* Improve detection
2023-12-20 12:42:25 +01:00
Vladimir Gavrilov 33f11cb10f Add OpenFlow protocol dissector (#2222) 2023-12-20 10:48:45 +01:00
Ivan Nardi 8aa09f9c99 mining: a better identification logic (#2221)
It is quite simple (and not so efficient) but it should fix all the
false positives reported in #2216. Add support for Ethereum mining.

Merge all the mining traces.

Remove duplicated function.

Close #2216
2023-12-20 10:46:57 +01:00
Ivan Nardi 308b266333 fuzz: improve fuzzing coverage (#2220) 2023-12-19 20:33:08 +01:00
Vladimir Gavrilov 6af93c09c8 Rename NDPI_PROTOCOL_RPC to NDPI_PROTOCOL_MS_RPCH (#2218)
* Rename NDPI_PROTOCOL_RPC to NDPI_PROTOCOL_MS_RPCH

* Add protocol description

* Improve MS-RPCH detection
2023-12-19 11:56:26 +01:00
Vladimir Gavrilov 59c8eabc0e Add UFTP protocol dissector (#2215)
* Add UFTP protocol dissector

* Update docs

* Merge pcap files
2023-12-18 11:21:07 +01:00
Vladimir Gavrilov d8c7a76611 Add HiSLIP protocol dissector (#2214)
* Add HiSLIP protocol dissector

* Fix error
2023-12-17 11:52:55 +01:00
Vladimir Gavrilov 0f3e6d832b Add PROFINET/IO protocol dissector (#2213)
* Add PROFINET/IO protocol dissector

* Add LE (Little Endian) to the file name

* Rework dissector

* Remove redundant check
2023-12-16 13:30:21 +01:00
Ivan Nardi 8ea16ffd77 TLS: be sure to always set a category value (#2206)
We probably need to check everywhere we set the category in the
"extra dissection" data path: a flow with a proper classification can't
have NDPI_PROTOCOL_CATEGORY_UNSPECIFIED as category.
2023-12-14 08:07:04 +01:00
Toni ef62391dba Add Monero protocol classification. (#2196)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-12-13 19:55:18 +01:00
Ivan Nardi bda037c57f Sync unit tests results after latest merge (#2211) 2023-12-13 17:47:20 +01:00
Ivan Nardi 193f28582b QUIC: add heuristic to detect unidirectional *G*QUIC flows (#2207)
Fix extraction of `flow->protos.tls_quic.quic_version` metadata.
2023-12-13 17:14:04 +01:00
Ivan Nardi 8e14aac5e0 ndpiReader: avoid creating two detection modules when processing traffic/traces (#2209) 2023-12-12 19:44:29 +01:00
Ivan Nardi 241c42ad7e ndpiReader: fix guessed_flow_protocols statistic (#2203)
Increment the counter only if the flow has been guessed
2023-12-12 19:44:03 +01:00
Ivan Nardi 7b5354588b fuzz: extend fuzzing coverage (#2208) 2023-12-11 19:24:17 +01:00
Ivan Nardi b3f2b1bb7f STUN: rework extra dissection (#2202)
Keep looking for RTP packets but remove the monitoring concept.
We will re-introduce a more general concept of "flow in monitoring
state" later.
The function was disabled by default.
Some configuration knobs will be provided when/if #2190 is merged.
2023-12-11 14:53:12 +01:00
Ivan Nardi 673b6e7345 HTTP: faster processing of asymmetric flows (#2198) 2023-12-11 14:52:31 +01:00
Ivan Nardi adf8982d8e fuzz: extend fuzzing coverage (#2205) 2023-12-11 12:48:50 +01:00
rl1987 59d476195c Fix typos (#2204)
* Fix typo in ndpiSimpleIntegration.c

* Fix misspelling in a comment
2023-12-10 19:58:22 +01:00
Vladimir Gavrilov c809e7c069 Replace complicated TPKT header validation with an helper function (#2201)
* Replace complicated TPKT header validation with an helper function

* Move tpkt_verify_hdr function definition to ndpi_utils.c
2023-12-10 12:10:50 +01:00
Ivan Nardi f74cf16c36 OpenVPN: rework detection (#2199)
Close #1873
2023-12-06 10:24:26 +01:00
Vladimir Gavrilov ad20846fad Add Ether-S-Bus protocol dissector (#2200) 2023-12-05 17:20:38 +01:00
Vladimir Gavrilov 72601a69ff Add some fast CRC16 algorithms implementation (#2195)
* Add some fast CRC16 algorithms implementation

* Update ndpi_crc.c

* Move crc16 stuff to ndpi_analyze.c

* IEEE C37.118: use new fast CRC-16/CCITT-FALSE implementation
2023-12-05 15:58:11 +01:00
Vladimir Gavrilov be50493f44 Add IEEE C37.118 protocol dissector (#2193) 2023-12-05 08:06:15 +01:00
Ivan Nardi 7b0c16a70d TLS: remove JA3+ fingerprints. (#2192)
See: #2191
2023-12-05 08:05:44 +01:00
Vladimir Gavrilov c34bded4ef Add ISO 9506-1 MMS protocol dissector (#2189)
* Add ISO 9506-1 MMS protocol dissector
* Fix detection on big-endian architectures
2023-12-01 09:03:07 +01:00
Ivan Nardi 983b8e8eee STUN: parsing of DATA attribute (#2179) 2023-12-01 07:01:49 +01:00
Vladimir Gavrilov 24df1913ac Add Beckhoff ADS protocol dissector (#2181)
* Add Beckhoff ADS protocol dissector

* Remove redundant le32toh

* Fix detection on big-endian architectures
2023-11-30 09:13:45 +01:00
Ivan Nardi 6f046df0dc STUN: fix detection of DTLS (#2187)
Fix a memory leak
```
==97697==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x55a6967cfa7e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x701a7e) (BuildId: c7124999fa1ccc54346fa7bd536d8eab88c3ea01)
    #1 0x55a696972ab5 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
    #2 0x55a696972da0 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13
    #3 0x55a696b7658d in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2394:46
    #4 0x55a696b86e81 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:897:5
    #5 0x55a696b80649 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1262:11
    #6 0x55a696b67a57 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2751:5
    #7 0x55a696b67758 in switch_to_tls /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1408:3
    #8 0x55a696c47810 in stun_search_again /home/ivan/svnrepos/nDPI/src/lib/protocols/stun.c:422:4
    #9 0x55a6968a22af in ndpi_process_extra_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7247:9
    #10 0x55a6968acd6f in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7746:5
    #11 0x55a6968aba3f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8013:22
    #12 0x55a69683d30e in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1723:31
    #13 0x55a69683d30e in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2440:10
    #14 0x55a69680f08f in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:135:7
[...]
SUMMARY: AddressSanitizer: 16 byte(s) leaked in 1 allocation(s).
```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64564
2023-11-30 09:09:40 +01:00
Ivan Nardi d85f8592b8 Rework extraction of HTTP headers (#2183)
Extract only the headers that we really need/use.
Avoid too many `strcmp`; the algorithm might be a little bit
overwhelming right now but it might be useful if we have further headers
in the future.
2023-11-29 19:44:25 +01:00
Ivan Nardi 1100326edb Fix FAQ (#2186) 2023-11-29 18:56:21 +01:00
Ivan Nardi d3d9770b96 Update FAQ.md 2023-11-29 18:52:35 +01:00
Ivan Nardi cc721fd3ce Update FAQ.md 2023-11-29 18:28:26 +01:00
Ivan Nardi 0114992f5f Add a FAQ for the project (#2185) 2023-11-29 18:25:36 +01:00
Ivan Nardi 3b35cb37d9 Keep separating public and private API (#2157)
See: b08c787fe
2023-11-29 17:13:00 +01:00
Ivan Nardi ac90b1f009 Fix detection of NDPI_TCP_ISSUES flow risk (#2177)
We need to take into account retransmissions: they increase
`flow->all_packets_counter` counter but not `flows->packet_counter`
one.
Therefore, the right way to check for 3WH + RST pattern involves checking
for `flows->packet_counter == 0`
2023-11-29 16:55:39 +01:00
Toni 5fb631c8fe Improved belgium gambling sites regex. (#2184)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-29 16:15:23 +01:00
Vladimir Gavrilov c60c03766c Add Schneider Electric’s UMAS detection support (#2180)
* Add Schneider Electric’s UMAS detection support

* Swap proto IDs in ndpi_set_detected_protocol

* Update unit test result
2023-11-28 18:03:00 +01:00
Vladimir Gavrilov 688a197b85 Fix FINS false positives (#2176)
* Fix FINS false positives

* Add rsv (reserved) field check
2023-11-27 21:46:14 +01:00
Vladimir Gavrilov ebb1bc2f34 Add Ether-S-I/O protocol dissector (#2174) 2023-11-27 19:04:05 +01:00
Vladimir Gavrilov 84427b0754 Add Omron FINS protocol dissector (#2172)
* Add Omron FINS protocol dissector

* Add a kludge to avoid invalid FINS over UDP detection as SkypeTeams and RTP

* Update unit test results

* Update protocols.rst

* Remove dummy flows from fins.pcap
2023-11-27 17:09:53 +01:00
3763c702f0 Rework S7Comm dissector; add S7Comm Plus support (#2165)
* Rework S7Comm dissector; add S7Comm Plus support

* Cleanup s7comm.c

* Improve S7Comm Plus detection

* s7comm/s7commplus: faster detection

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2023-11-27 14:37:48 +01:00
Vladimir Gavrilov 0b6e261523 Improve CORBA detection (#2167)
* Improve CORBA detection

* Remove dummy flow from ziop.pcap

* Merge ziop.pcap and miop.pcap into corba.pcap
2023-11-27 13:10:50 +01:00
Vladimir Gavrilov da629709f3 Add OPC UA protocol dissector (#2169) 2023-11-27 12:13:23 +01:00
Ivan Nardi 7ff22a7e3c STUN: improve demultiplexing of DTLS packets (#2153)
Keep demultiplexing STUN/RTP/RTCP packets after DTLS ones.

We might end up processing the session a little longer, because we will
process the STUN/RTP/RTCP packets after the DTLS handshake.
2023-11-27 11:10:38 +01:00
Vladimir Gavrilov 87399b3544 Add RTPS protocol dissector (#2168) 2023-11-27 07:17:39 +01:00
Vladimir Gavrilovand0xA50C1A1 27802b0134 Reduce false positives for H.323 over TCP (#2164)
Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
2023-11-23 17:29:00 +01:00
fbae51ae9d Get rid of RDP false positives (#2161)
* Get rid of false positives in the RDP protocol dissector

* Remove kludge for RDP

* RDP: improve detection

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2023-11-23 09:35:43 +01:00
Vladimir Gavrilovand0xA50C1A1 5c8c5c90c2 Add HART-IP protocol dissector (#2163)
* Add HART-IP protocol dissector

* Update docs

* Update protocols.rst

* Reuse free proto id and re-run tests

* docs: move HART-IP to top of list

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
2023-11-22 22:04:22 +01:00
Toni 21f2574033 Improved TFTP. Fixes #2075. (#2149)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-21 16:56:46 +01:00
Vladimir Gavrilovand0xA50C1A1 35abafec4f Get rid of Apache Cassandra false positives (#2159)
* Rewrite Apache Cassandra dissector

* Replace memcmp with strncmp

* Add payload length check

* Update Cassandra dissector

* Update test results

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
2023-11-21 16:56:01 +01:00
Vladimir Gavrilovand0xA50C1A1 ae6e6d61f0 Add IEEE 1588-2008 (PTPv2) dissector (#2156)
* Add IEEE 1588-2008 (PTPv2) dissector

PTPv2 is a time synchronization protocol in computer networks, similar to NTP.

* Add default protocol ports

* Update default test result for PTPv2

* Update copyright

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
2023-11-21 13:39:54 +01:00
Ivan Nardi 55664392a9 Ip address list: aggregate Mullvad and Tor lists too (#2154)
Missing from bdb73db1a
See #2150
2023-11-21 08:28:32 +01:00
Vladimir Gavrilovand0xA50C1A1 8d71998670 Remove Google+ support (#2155)
* Remove Google+ support

Google+ was discontiued in 2019, so I think that its protocol id can be freed for reuse.

* Fix typo

* Update tests

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
2023-11-21 08:14:20 +01:00
Ivan Nardi bdb73db1a4 IP lists: aggregate addresses wherever possible (#2152)
See #2150
2023-11-17 12:26:23 +01:00
Ivan Nardi 6c9571d9a9 Remove duplicate addreess list (#2151)
We are loading the same AS list as GOTO
See #2150
2023-11-16 20:08:06 +01:00
Maatuq 344ac69468 protocols/ethereum: enhance node discovery protocol detection. (#2145)
*  packet-type == 0x0 shouldn't be allowed.
* allow more packet-type e.g. ENRRequest, ENRResponse.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
2023-11-15 19:45:48 +01:00
Toni 38f9a74713 Added TeslaServices and improved TikTok host names. Fixes #2140. (#2144)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-10 11:25:10 +01:00
Toni 0cb6f4cb75 Fixed hash buffer size in ndpiSimpleIntegration. (#2143)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-10 10:23:37 +01:00
Ivan Nardi b08c787fe2 Have a clear distinction between public and private/internal API (#2137)
1) Public API/headers in `src/include/` [as it has always been]
2) Private API/headers in `src/lib/`

Try to keep the "ndpi_" prefix only for the public functions
2023-11-09 10:50:59 +01:00
Toni 0673da54b5 Fixed implicit u32 cast in ndpi_data_min() / ndpi_data_max(). (#2139)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-09 10:16:57 +01:00
Ivan Nardi b539b0d090 fuzz: improve coverage and remove dead code (#2135)
We are not able to remove custom rules: remove the empty stubs (which
originate from the original OpenDPI code).

`ndpi_guess_protocol_id()` is only called on the first packet of the
flow, so the bitmask `flow->excluded_protocol_bitmask` is always empty,
since we didn't call any dissectors yet.

Move another hash function to the dedicated source file.
2023-11-07 17:46:29 +01:00
Ivan Nardi 58a9e2d967 Remove struct ndpi_packet_struct from the public API (#2129)
Right now, the only instance of `struct ndpi_packet_struct` is embedded
into `struct ndpi_detection_module_struct`. Since the latter is a
private structure (because of `NDPI_LIB_COMPILATION` ) there is no way for
the application to get a pointer to `ndpi_struct->packet`.

Bottom line: the application can't use any API functions having `struct
ndpi_packet_struct *` as parameter. Remove them all (since they are
completly unused and unusable).

There are no public helper functions to initialize/populate/deinit a
`struct ndpi_packet_struct` object, so the application can't neither
create its own instance of this object.

Protect `struct ndpi_packet_struct` via the same define
`NDPI_LIB_COMPILATION`.
2023-11-07 15:30:56 +01:00
Luca Deri 0db12b1390 In case a pattern is redefined (for instance with protos.txt), the custom definition overwrites the previous one (so far it was ignored) 2023-11-02 20:46:14 +01:00
Toni 6dcecd73d3 Added malicious sites from the polish cert. (#2121)
* added handling of parsing errors

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-02 09:04:04 +01:00
Christian Marangi d5c9a16a48 Move from PCRE to PCRE2 (#2134)
Move from PCRE to PCRE2. PCRE is EOL and won't receive any security
updates anymore. Convert to PCRE2 by converting any function PCRE2 new
API.

Also update every entry in github workflows and README to point to the
new configure flag. (--with-pcre2)

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-11-01 14:16:49 +01:00
Ivan KapranovandIvan Kapranov 5a2666ce9e Fix proto_name and proto_id missmatch for Sina and SinaWeibo (#2131)
* minor fixes

fixed 'handle leak' in ndpi_load_malicious_sha1_file and removed the redundant comparison ndpi_search_eaq

* fix Stack overflow caused by invalid write in ndpi_automa_match_string_subprotocol

* fix compile errors

* fix

* Fix name missmatch for Sina and Sina Weibo

* fix

* add Sina Weibo to doc

* fix

* add Sina Weibo to doc

---------

Co-authored-by: Ivan Kapranov <i.kapranov@securitycode.ru>
2023-11-01 14:12:49 +01:00
Ivan Nardi cd59485f7e Custom rules: add missing check (#2132)
Fix an use-of-uninitialized-value error

```
==28646==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x55a9b7a3dea2 in ndpi_patricia_lookup /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ndpi_patricia.c:739:8
    #1 0x55a9b7442cac in add_to_ptree /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2588:10
    #2 0x55a9b7469290 in ndpi_add_ip_risk_mask /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4161:12
    #3 0x55a9b746cf14 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4297:11
    #4 0x55a9b74842ae in ndpi_load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4944:8
    #5 0x55a9b7424706 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:38:3
[...]
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=63754
2023-10-31 20:02:37 +01:00
Ivan Nardi 50006516b3 Custom rules: fixes for Windows (#2130)
Even after adding `WSAStartup()` call, the behavior with IPv6 addresses
was different on Windows; it is the same issue reported on:
https://stackoverflow.com/questions/66755681/getaddrinfo-c-on-windows-not-handling-ipv6-correctly-returning-error-code-1

Fix bitmasks.
2023-10-31 16:04:30 +01:00
Luca Deri 773e9b8170 Added pointer check 2023-10-31 15:48:06 +01:00
Luca Deri f49cedc1e8 Added missing freeaddrinfo() call 2023-10-30 22:58:17 +01:00
Luca Deri 5e9f10f820 Replaced gethostbyname2 with getaddrinfo (portable code) 2023-10-30 22:55:50 +01:00
Luca Deri c6af36990e Warning fixes 2023-10-30 13:10:28 +01:00
Ivan Nardi 42d24f8799 STUN: major code rework (#2116)
Try to have a faster classification, on first packet; use standard extra
dissection data path for sub-classification, metadata extraction and
monitoring.

STUN caches:
* use the proper confidence value
* lookup into the caches only once per flow, after having found a proper
STUN classification

Add identification of Telegram VoIP calls.
2023-10-30 10:28:19 +01:00
Ivan Nardi e399bd7e3d Custom rules: fix a stack overflow (#2128)
```
==19255==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f515bb3bf80 at pc 0x55796e01394a bp 0x7fff4fb5c050 sp 0x7fff4fb5b7e0
WRITE of size 58 at 0x7f515bb3bf80 thread T0
    #0 0x55796e013949 in scanf_common(void*, int, bool, char const*, __va_list_tag*) asan_interceptors.cpp.o
    #1 0x55796e0147df in __isoc99_sscanf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x77f7df) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)
    #2 0x55796e0fc74a in ndpi_add_host_ip_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2771:13
    #3 0x55796e0fb029 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4411:16
    #4 0x55796e103738 in ndpi_load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4901:8
    #5 0x55796e0ca96d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:38:3
    #6 0x55796dfd78e0 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x7428e0) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)
    #7 0x55796dfc0e93 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x72be93) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)
    #8 0x55796dfc6d96 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x731d96) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)
    #9 0x55796dff1672 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x75c672) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)
    #10 0x7f515df19082 in __libc_start_main /build/glibc-BHL3KM/glibc-2.31/csu/../csu/libc-start.c:308:16
    #11 0x55796dfbbb0d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x726b0d) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)

Address 0x7f515bb3bf80 is located in stack of thread T0 at offset 128 in frame
    #0 0x55796e0fb977 in ndpi_add_host_ip_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2703

  This frame has 4 object(s):
    [32, 36) 'pin' (line 2705)
    [48, 64) 'pin6' (line 2706)
    [80, 96) 'd' (line 2769)
    [112, 128) 'tail' (line 2770) <== Memory access at offset 128 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow asan_interceptors.cpp.o in scanf_common(void*, int, bool, char const*, __va_list_tag*)
Shadow bytes around the buggy address:

```
2023-10-30 09:21:57 +01:00
Luca Deri 8769a2282b Disbling AES-NI on FreeBSD as the feature checke does not work well
and it enables it by mistake on IntelB. Coreb" i3-3240 Processor
2023-10-30 00:05:25 +01:00
Luca Deri 9fb7a63576 FreeBSD compilation fix (missingcvg #include <netdb.h>) 2023-10-29 23:18:17 +01:00
Luca Deri 76829b413f Implements support for symbolic host names (#2123) 2023-10-29 22:54:45 +01:00
Luca Deri b8e0602bfd Added check 2023-10-29 22:54:45 +01:00
Ivan Nardi 2c2eaf8e79 Rename some functions with more useful/clear names (#2127) 2023-10-29 20:21:24 +01:00
Ivan Nardi 03fd155ae3 IPv6: add support for custom categories (#2126) 2023-10-29 12:56:44 +01:00
Ivan Nardi 32b50f5aa4 IPv6: add support for IPv6 risk exceptions (#2122) 2023-10-29 12:14:20 +01:00
Ivan Nardi c711251578 IPv6: add support for custom rules (#2120) 2023-10-29 11:26:35 +01:00
Ivan Nardi e8e4b9e8ff IPv6: add support for IPv6 risk tree (#2118)
Fix the script to download crawler addressess
2023-10-27 13:58:15 +02:00
Toni ed17f4d658 Improved Protobuf dissector. (#2119)
* tag extraction/validation was done wrong

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-27 10:19:47 +02:00
Ivan Nardi 8b07be4b9f Jabber: remove support for UDP (#2115)
Jabber/XMPP is only over TCP (even the name `ndpi_search_jabber_tcp`
suggests that...).

Bug introduced in 5266c726f
2023-10-26 20:16:27 +02:00
Ivan Nardi 611c3b66f0 ipv6: add support for ipv6 addresses lists (#2113) 2023-10-26 20:15:44 +02:00
Luca Deri 1832d247b3 Tool for creating bitcoing IP files 2023-10-26 00:01:44 +02:00
Toni 9509cd507a Fixed endian issue while DEBUG_PROTOBUF is enabled. (#2112)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-25 13:11:17 +02:00
Maatuq 4a8e7105b2 add ethereum protocol dissector. (#2111)
as explained here for bitcoin https://www.ntop.org/guides/nDPI/protocols.html#ndpi-protocol-bitcoin
the same is applicable for ethereum.
ethereum detection was removed from mining protocol and is now handled separately.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
2023-10-25 12:44:33 +02:00
Toni e70333de87 Added generic Google Protobuf dissector. (#2109)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-24 12:18:31 +02:00
Luca Deri 71d768d036 Improved detection as non DGA for hostnames belnging to a CDN (#2068) 2023-10-23 17:49:50 +02:00
Toni Uhlig a443bba0dd Add CAN over Ethernet dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-23 13:45:56 +02:00
Toni Uhlig 25c54dd6d7 Improved CryNetwork protocol dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-23 13:44:34 +02:00
Luca Deri 7122599287 Added CLA section 2023-10-23 12:05:56 +02:00
Luca Deri 23759c6d2d Moved to 4.9 2023-10-23 11:46:04 +02:00
Luca Deri 2a99e16728 Added 4.8 changeling 2023-10-23 11:38:46 +02:00
Toni Uhlig f69909d49b Add Remote Management Control Protocol (RMCP).
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-19 19:50:57 +02:00
Alfredo Cardigliano c59fe0b2b5 PR template 2023-10-19 09:29:38 +02:00
Toni 01f384f7ff Improved Steam detection by adding steamdiscover pattern. (#2105)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-17 22:13:02 +02:00
Toni 32b0aeea59 Fixed OpenWRT arm related build issues. (#2104)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-16 19:49:50 +02:00
Toni d75a8d339a Fixed two OpenWRT arm related build issues. (#2103)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-16 18:14:28 +02:00
Luca Deri d9b0ba1d14 Windows warning checks 2023-10-16 16:17:41 +02:00
Nardi Ivan 16b4913be6 fuzz: extend fuzzing coverage 2023-10-15 12:00:26 +02:00
Luca Deri b6179467eb Added NDPI_MALWARE_HOST_CONTACTED flow risk 2023-10-13 00:01:55 +02:00
Luca 30fc89e556 Fix for buffer overflow in serialization 2023-10-11 17:14:17 +02:00
lns 304747f1fa Improved MGCP detection by allowing '\r' as line feed.
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-11 16:44:26 +02:00
Nardi Ivan 4a0eda69ad QUIC: export QUIC version as metadata 2023-10-11 15:15:20 +02:00
Luca Deri a76b9d151f Serialization fix 2023-10-11 00:14:27 +02:00
Nardi Ivan 1366d94156 fuzzing: extend fuzzing coverage
Try fuzzing some functions which write to file/file descriptor; to avoid
slowing the fuzzer, close its stdout
2023-10-09 15:41:46 +02:00
Nardi Ivan 86115a8a65 fuzz: extend fuzzing coverage 2023-10-07 13:34:37 +02:00
Luca Deri 8ca842edec version of dirent.c that is liked by both VC++ and MinGW 2023-10-05 14:35:41 +02:00
Luca Deri e1e6a01d6a Windows code rework 2023-10-05 12:24:39 +02:00
Luca Deri 3881d162f7 Windows compilation fixes 2023-10-05 11:05:14 +02:00
Luca Deri 36112a645b Win include change 2023-10-05 10:47:15 +02:00
Toni a98d7ff433 Added HAProxy protocol. (#2088)
* fixed tests/do.sh.in failure print

Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-02 18:10:47 +02:00
Luca 77e5daf03e Cleaned up mining datastructure 2023-09-27 17:05:12 +02:00
Toni ef3adb9830 Added printf/fprintf replacement for some internal modules. (#1974)
* logging is instead redirected to `ndpi_debug_printf`

Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-09-26 23:10:57 +02:00
Luca Deri 725fcf4852 Renamed HTTP/2 to HTTP2 as the '/' can have side effects with applications sitting on top of nDPI 2023-09-20 11:17:09 +02:00
Ivan Nardi d4e0b79dd6 Fix some prototypes (#2085)
```
error: function declaration isn’t a prototype [-Werror=strict-prototypes]
```
2023-09-18 14:06:47 +02:00
Ivan Nardi 6925890383 Add support for (un-encrypted) HTTP/2 (#2087)
Plaintext HTTP/2 is quite rare on the general "internet" but it is
used in some private networks (example: 5G core network)
2023-09-18 14:06:09 +02:00
Nardi Ivan 70814002a9 fuzz: extend fuzzing coverage 2023-09-16 11:26:11 +02:00
Luca 0828dff3ac Language fix 2023-09-15 12:56:29 +02:00
Ivan Nardi c5cd4df056 Add ndpi_domain_classify_finalize() function (#2084)
The "domain classify" data structure is immutable, since it uses
"bitmap64".
Allow to finalize it before starting to process packets (i.e. before
calling `ndpi_domain_classify_contains()`) to avoid, in the data-path,
all the memory allocations due to compression.

Calling `ndpi_domain_classify_finalize()` is optional.
2023-09-12 19:47:14 +02:00
Thomas Winter 5bcf3c2ddd tftp: update pcap results
The two malformed TFTP packets are no longer considered as risk
and instead match by port only.
This is because the TFTP detection was rather sparse so could
match on several other protocols if the first two opcode bytes
happened to match.
2023-09-12 13:12:14 +02:00
Thomas Winter a5f0cb7e6c tftp: check incrementation for DATA and ACK packets
The 2 bytes following the opcode for DATA and ACK packets are the
block number and this should be incrementing every packet.
We should check to see that this is occurring otherwise false matches
can occur, eg L2TPv3 over UDP matches the DATA opcode but the next two
bytes are always zero.

Remove the DATA max block size assumption since this can be false if
the blksize option is used to increase it.

Fixes #2070
2023-09-12 13:12:14 +02:00
Thomas Winter 7263562514 tftp: check for Option Acknowledgements
TFTP Option Acknowledgement packets were being excluded.
When a read/write request contains options, an Option Acknowledgement
is returned that contains the option strings that the transaction
will use.
The options sent in the request are not compared with what was
acknowledged.
2023-09-12 13:12:14 +02:00
Thomas Winter 1e9517ab74 tftp: rework request checking to account for options
Read and write requests were not handling options.
The existing code inspecting request messages assumed that the string
before the end of the payload was the mode and that the filename
length depended on the mode length.
However the first two strings are the filename and mode which can be
followed by any number of option strings.

Rework the checking of the filename and mode to just search the first
two strings which should always exist. Any options after that are
ignored.

Absence of the filename or mode is now excludes the TFTP protocol.
Absence of the filename no longer is considered malformed packet
because it can match other protocols falsely.
2023-09-12 13:12:14 +02:00
Luca Deri 1bf7e5face Fixes matches with domain name strings that start with a dot 2023-09-11 22:50:03 +02:00
Ivan Nardi 8e6500a0da fuzz: extend fuzzing coverage (#2083) 2023-09-10 21:30:26 +02:00
Ivan Nardi ef6085370f fuzz: add fuzzers to test bitmap64 and domain_classify data structures (#2082) 2023-09-10 18:44:50 +02:00
Ivan Nardi 2c5e22123e Update protocols documentation (#2081) 2023-09-10 15:23:34 +02:00
Ivan Nardi 2a0052f25e fuzz: add fuzzers to test reader_util code (#2080) 2023-09-10 15:07:52 +02:00
Ivan Nardi 0aa1cf7245 Update every ip lists (#2079) 2023-09-10 12:13:20 +02:00
Ivan Nardi 2b883b93be Fix some errors found by fuzzers (#2078)
Fix compilation on Windows.
"dirent.h" file has been taken from https://github.com/tronkko/dirent/

Fix Python bindings

Fix some warnings with x86_64-w64-mingw32-gcc:
```
protocols/dns.c: In function ‘ndpi_search_dns’:
protocols/dns.c:775:41: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
  775 |       unsigned long first_element_len = (unsigned long)dot - (unsigned long)_hostname;
      |                                         ^
protocols/dns.c:775:62: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
  775 |       unsigned long first_element_len = (unsigned long)dot - (unsigned long)_hostname;
      |
```
```
In file included from ndpi_bitmap64.c:31:
third_party/include/binaryfusefilter.h: In function ‘binary_fuse8_hash’:
third_party/include/binaryfusefilter.h:160:32: error: left shift count >= width of type [-Werror=shift-count-overflow]
  160 |     uint64_t hh = hash & ((1UL << 36) - 1);
```
```
In function ‘ndpi_match_custom_category’,
    inlined from ‘ndpi_fill_protocol_category.part.0’ at ndpi_main.c:7056:16:
ndpi_main.c:3419:3: error: ‘strncpy’ specified bound depends on the length of the source argument [-Werror=stringop-overflow=]
 3419 |   strncpy(buf, name, name_len);
```
2023-09-10 11:09:59 +02:00
Luca Deri 805df2e5ce Reworked initialization 2023-09-10 09:36:54 +02:00
Luca Deri 63977459b6 Added OperaVPN detection 2023-09-09 18:17:26 +02:00
Luca Deri 076edeab54 Enhance DNS risk for long hostnames (> 32) 2023-09-09 18:09:57 +02:00
Luca Deri f50a4d7e85 Improved detection of invalid chars in DNS names 2023-09-09 17:46:13 +02:00
Luca Deri 1a797d7b74 Win fix 2023-09-08 12:24:27 +02:00
Luca Deri 770e460ba9 Added NDPI_TLS_ALPN_SNI_MISMATCH flow risk 2023-09-07 23:42:42 +02:00
Luca 47fb5e9f3d Cleanup 2023-09-07 11:12:37 +02:00
Luca Deri 2bd986a00a Disabled debug message 2023-09-05 18:39:26 +02:00
Luca Deri ad9ed4f954 Minor warning fixes 2023-09-05 18:08:16 +02:00
Luca Deri 978df906b3 Improved classification further reducing memory used 2023-09-05 17:03:20 +02:00
Luca Deri 0080d0d092 Compilation fixes for old compilers 2023-09-05 10:43:36 +02:00
Luca Deri feaaf83bed Added sub-domain classification fix 2023-09-05 07:21:48 +02:00
Luca Deri cf67f483f4 Classification fixes 2023-09-05 01:17:02 +02:00
Luca Deri c5ebd84fd8 Added ndpi_bitmap64 support 2023-09-05 00:21:16 +02:00
Luca Deri 4f2ce2d43b Added ndpi_murmur_hash to the nDPI API 2023-09-04 12:53:42 +02:00
Luca Deri f0dc3347ec Merged new and old version of ndpi_domain_classify.c code 2023-09-02 19:27:19 +02:00
Luca Deri 1d480c18e3 Reworked domain classification based on binary filters 2023-09-02 19:16:40 +02:00
Luca Deri 854c2d80f1 Improvement for reducing false positives 2023-09-01 10:26:07 +02:00
Luca Deri 74fdac7d77 Code cleanup 2023-09-01 08:50:55 +02:00
Luca Deri 133f45f3f3 Added ndpi_binary_bitmap datastruture
It is similar to ndpi_filter but based on binary search and with
the ability to store a category per value (as ndpi_domain_classify)
2023-08-31 23:51:50 +02:00
Luca Deri 19381f330a Fixed heap-buffer-overflow issue 2023-08-31 18:35:49 +02:00
Luca Deri 16b0ce3710 Code cleanup 2023-08-31 09:14:17 +02:00
Luca Deri f824939662 Added comment 2023-08-31 09:00:36 +02:00
Luca Deri 3c7b2a714e Added missing lists files 2023-08-29 19:21:57 +02:00
Luca Deri c5c610c941 Fixes dist 2023-08-29 18:49:43 +02:00
Luca Deri 65142b2a53 Added -G flag 2023-08-29 18:32:57 +02:00
Luca Deri 1de7fd9bf0 Warning fixes 2023-08-29 18:08:56 +02:00
Luca Deri 3dbc4f65fc Added README file 2023-08-29 17:45:36 +02:00
Luca Deri 36abf06c6f Swap from Aho-Corasick to an experimental/home-grown algorithm that uses a probabilistic
approach for handling Internet domain names.

For switching back to Aho-Corasick it is necessary to edit
ndpi-typedefs.h and uncomment the line
// #define USE_LEGACY_AHO_CORASICK

[1] With Aho-Corasick
$ ./example/ndpiReader -G ./lists/ -i tests/pcap/ookla.pcap | grep Memory
nDPI Memory statistics:
nDPI Memory (once):      37.34 KB
Flow Memory (per flow):  960 B
Actual Memory:           33.09 MB
Peak Memory:             33.09 MB

[2] With the new algorithm
$ ./example/ndpiReader -G ./lists/ -i tests/pcap/ookla.pcap | grep Memory
nDPI Memory statistics:
nDPI Memory (once):      37.31 KB
Flow Memory (per flow):  960 B
Actual Memory:           7.42 MB
Peak Memory:             7.42 MB

In essence from ~33 MB to ~7 MB

This new algorithm will enable larger lists to be loaded (e.g. top 1M domans
https://s3-us-west-1.amazonaws.com/umbrella-static/index.html)

In ./lists there are file names that are named as <category>_<string>.list
With -G ndpiReader can load all of them at startup
2023-08-29 17:34:04 +02:00
Toni Uhlig 1f693c3f5a Added lists/gambling.list to extra dist.
* make dist in `./fuzz`: fixed inconsistent `*.dict` file pattern

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-27 15:53:31 +02:00
Toni Uhlig afd250f5d9 fix compilation and symbol check
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-27 15:32:42 +02:00
Toni Uhlig 25f3a7cd1d fix compilation
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-27 15:17:56 +02:00
Luca Deri 34986de297 Search fixes 2023-08-26 19:47:50 +02:00
Luca Deri eeeee46b1e Changes for supporinng more efficient sub-string matching 2023-08-26 17:55:50 +02:00
Luca Deri 4ca94369e1 Improved domain search tet unit 2023-08-26 10:43:19 +02:00
Luca Deri ac00542df2 LEak fix 2023-08-26 10:10:49 +02:00
Luca Deri 2c565c77c9 Added ndpi_domain_classify_XXX(0 API 2023-08-26 00:24:33 +02:00
Luca Deri 42fcb011ad Warning fix 2023-08-25 19:29:43 +02:00
Luca Deri eb9932164f added bimap and/or with allocation 2023-08-24 07:45:38 +02:00
Luca Deri ff4f291afe Minor improvements 2023-08-23 11:04:31 +02:00
Luca 95fcd08687 Added ndpi_bitmap_is_empty() and ndpi_bitmap_optimize() API calls 2023-08-23 11:03:59 +02:00
Luca 9933b5910d Boundary check 2023-08-21 23:19:57 +02:00
Luca bd25df3334 Improved Wireguard detection 2023-08-21 13:40:42 +02:00
Luca 9e4bcc7771 Removed empty line 2023-08-21 13:40:30 +02:00
Luca 6e5466f9e5 Added ndpi_bitmap_andnot API call 2023-08-21 13:26:38 +02:00
Ivan Nardi cc4461f424 fuzz: extend coverage (#2073) 2023-08-20 15:18:19 +02:00
Ivan Nardi 5867f43fae Fix compilation on Windows (#2072) 2023-08-14 22:51:09 +02:00
Luca Deri cfbb7a423e Replaces free() with ndpi_free() 2023-08-14 18:51:11 +02:00
Luca Deri 049fbf819c Reworked ndpi_filter_xxx implementation using compressed bitmaps 2023-08-14 12:50:54 +02:00
Luca Deri 3b34941be1 Typo 2023-08-12 11:39:19 +02:00
Luca Deri 9a44954365 ARM compilation fix 2023-08-12 07:39:35 +02:00
Luca Deri fe634b8d55 Fixed to address issue
Run ./utils/check_symbols.sh || { FAILED=$?; echo "::error file=${NDPI_LIB}::Unwanted libc symbols found: ${FAILED}. Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of malloc/calloc/realloc/free."; false; }
[ndpi_filter.o]: calloc
Unwanted symbols found: 1
Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of malloc/calloc/realloc/free
Error: Unwanted libc symbols found: 1. Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of
2023-08-12 00:27:37 +02:00
Luca Deri 8e17f50853 Added ndpi_filter_add_multi() API call 2023-08-11 23:55:44 +02:00
Luca Deri 890dcf91d9 Minor initialization fixes 2023-08-11 23:51:16 +02:00
Luca Deri ec7adc212e Added new API calls for implementing Bloom-filter like data structures
ndpi_filter* ndpi_filter_alloc(uint32_t elements_number);
bool         ndpi_filter_add(ndpi_filter *f, uint64_t value);
bool         ndpi_filter_contains(ndpi_filter *f, uint64_t value);
void         ndpi_filter_free(ndpi_filter *f);
2023-08-11 22:49:31 +02:00
Luca 5fad72ebd9 Typo fix 2023-08-07 10:20:21 +02:00
Luca 33e9dbc24d Added ndpi_bitmap_copy() API call 2023-08-06 09:35:56 +02:00
snicket2100 1fbe8a2385 Mullvad VPN service added (based on entry node IP addresses) (#2062) 2023-08-02 19:44:16 +02:00
Luca Deri 196395398e Compilation fixes for older C compilers 2023-08-01 15:35:33 +02:00
Toni e4d3d619bc Add Service Location Protocol dissector. (#2036)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-01 08:50:46 +02:00
Toni 083275f617 Added slackb.com SNI. (#2067)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-01 08:39:10 +02:00
Ivan Nardi 5019022e13 DNS: extract geolocation information, if available (#2065)
The option NSID (RFC5001) is used by Google DNS to report the
airport code of the metro where the DNS query is handled.

This option is quite rare, but the added overhead in DNS code is pretty
much zero for "normal" DNS traffic
2023-07-31 07:44:43 +02:00
snicket2100 d477da997f Typo in a comment fixed (#2063) 2023-07-28 17:58:25 +02:00
Ivan Nardi bc91192aca ProtonVPN: split the ip list (#2060)
Use two separate lists:
* one for the ingress nodes, which triggers a ProtonVPN classification
* one for the egress nodes, which triggers the
`NDPI_ANONYMOUS_SUBSCRIBER` risk

Add a command line option (to `ndpiReader`) to easily test IP/port
matching.

Add another example of custom rule.
2023-07-27 09:05:22 +02:00
Ivan Nardi 3326fa258e Add an heuristic to detect fully encrypted flows (#2058)
A fully encrypted session is a flow where every bytes of the
payload is encrypted in an attempt to “look like nothing”.
The heuristic needs only the very first packet of the flow.
See: https://www.usenix.org/system/files/sec23fall-prepub-234-wu-mingshi.pdf

A basic, but generic, inplementation of the popcpunt alg has been added
2023-07-26 09:09:12 +02:00
Toni 2b230e28e0 Allow init of app protocols w/o any hostnames set. (#2057) 2023-07-22 14:51:33 +02:00
Toni 88a757c4cd Fix string truncation. (#2056)
ndpi_main.c: In function ‘ndpi_load_ip_category’:
ndpi_main.c:6598:3: warning: ‘strncpy’ specified bound 64 equals destination size [-Wstringop-truncation]
 6598 |   strncpy(ipbuf, ip_address_and_mask, sizeof(ipbuf));
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-07-22 12:45:45 +02:00
Luca Deri 6831187d62 Debian 12 fixes 2023-07-21 12:24:34 -04:00
Ivan Nardi 7a1509bf60 zabbix: improve detection (#2055) 2023-07-21 03:42:36 +02:00
Ivan Nardi c85f2fb0f4 TLS: add basic, basic, detection of Encrypted ClientHello (#2053) 2023-07-21 03:41:43 +02:00
Ivan Nardi fa0bd515b5 Add detection of Roblox games (#2054) 2023-07-21 03:39:40 +02:00
Ivan Nardi 3edfad01a1 fuzz: extend fuzzing coverage (#2052)
Added/merged some traces.
Improved Socks identification
2023-07-18 07:41:56 +02:00
Ivan Nardi 09548bb7cf tests: restore some old paths as symbolic links (#2050) 2023-07-16 13:47:35 +02:00
Ivan Nardi 5267858b3d HTTP: fix another memory access error (#2049)
```
=================================================================
==199079==ERROR: AddressSanitizer: negative-size-param: (size=-1)
    #0 0x559a2a6efd4f in strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x94ad4f) (BuildId: 34aaabba403c6bc5482553ef355360fd2762a157)
    #1 0x559a2a9890f0 in ndpi_http_check_content /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:300:8
    #2 0x559a2a9812c0 in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:910:46
    #3 0x559a2a978fee in process_response /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1289:3
    #4 0x559a2a97622f in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1382:9
    #5 0x559a2a975d95 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1468:3
    #6 0x559a2a864970 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5948:4
    #7 0x559a2a8660df in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6013:12
    #8 0x559a2a865d7f in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6032:12
    #9 0x559a2a876fd6 in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7038:15
    #10 0x559a2a87311f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7205:22
    #11 0x559a2a77381e in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1710:31
    #12 0x559a2a77381e in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2427:10
[...]
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60605
2023-07-15 22:23:52 +02:00
Ivan Nardi 2bbde5bad3 Fix compilation in CI jobs (#2048) 2023-07-15 14:26:51 +02:00
Ivan Nardi 890f17788b ndpireader: fix detection of DoH traffic based on packet distributions (#2045) 2023-07-14 23:20:06 +02:00
Ivan Nardi 5811a5613b Add a configuration knob to enable/disable loading of gambling list (#2047) 2023-07-14 23:19:30 +02:00
Luca Deri 8e960f033d Adds new pcap for testing "funny" HTTP servers 2023-07-14 21:12:11 +02:00
Luca Deri a499edd2f2 Restored files 2023-07-14 21:12:04 +02:00
Luca Deri fea09e825b Fixes risk mask exception handling while improving the overall performance 2023-07-14 19:52:34 +02:00
snicket2100 abee1a2a6f Included Gambling website data from the Polish hazard.mf.gov.pl list (#2041)
* Refreshed the Belgium Gambling Site list data

Unfortunately some hostnames have been removed from that list,
which means they are disappearing from the `ndpi_gambling_match.c.inc`
file as well.

* build: added `libxml2-utils` (for `xmllint`)

* Included Gambling website data from the Polish `hazard.mf.gov.pl` list

The list contains over 30k gambling website hostnames as of today.
2023-07-14 09:55:46 +02:00
Ivan Nardi 0dd1ee5656 HTTP: fix extraction of filename (#2046) 2023-07-14 01:33:00 +02:00
Luca Deri bfbbd18757 Removed -fno-color-diagnostics not available on all platforms 2023-07-13 23:30:05 +02:00
Luca Deri 33c42f9830 Cosmetic fixes 2023-07-13 23:26:35 +02:00
Luca Deri 1f55dc511f Implemented Count-Min Sketch [count how many times a value has been observed]
- ndpi_cm_sketch_init()
- ndpi_cm_sketch_add()
- ndpi_cm_sketch_count()
- ndpi_cm_sketch_destroy()
2023-07-13 21:54:51 +02:00
Ivan Nardi bd0fcb2e62 HTTP: fix heap-buffer-overflow (#2044)
```
=================================================================
==673246==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x5080000002fd at pc 0x555c6a22eca2 bp 0x7ffe1db9b290 sp 0x7ffe1db9aa48
READ of size 1 at 0x5080000002fd thread T0
    #0 0x555c6a22eca1 in strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x53aca1) (BuildId: 464e7474cde3de91c01588c7ea0161a05ddd6b63)
    #1 0x555c6a462a9d in ndpi_http_check_content /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:309:8
    #2 0x555c6a45a9a0 in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:901:46
    #3 0x555c6a4526ce in process_response /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1280:3
    #4 0x555c6a44f90f in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1373:9
    #5 0x555c6a44f475 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1459:3
    #6 0x555c6a348520 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5949:6
    #7 0x555c6a348f2f in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5997:12
    #8 0x555c6a348bcf in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6016:12
    #9 0x555c6a359e26 in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7017:15
    #10 0x555c6a355f6f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7184:22
    #11 0x555c6a284958 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:24:5
```

Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60557
Fix: 0b0f255cc
2023-07-13 10:50:59 +02:00
Ivan Nardi 0c5a17accb RDP: improve detection over UDP (#2043) 2023-07-13 10:50:13 +02:00
Ivan Nardi 5e6a8c0c1f fuzz: add fuzzer for DGA detection code (#2042) 2023-07-13 10:47:34 +02:00
Alfredo Cardigliano 669a02cd76 Add support for multiline json 2023-07-12 10:47:49 +02:00
Luca Deri b1d25193d2 Minor code cleanup 2023-07-11 22:49:06 +02:00
Chiara Maggi 0b0f255cc2 added feature to extract filename from http attachment (#2037)
* added feature to extract filename from http attachment

* fixed some issues

* added check for filename format

* added check for filename format

* remove an unnecessary print

* changed the size from 952 to 960

* modified some test result files

* small changes string size

* comment removed and mallocs checked
2023-07-11 22:45:19 +02:00
Ivan Nardi 950f5cc4e3 fuzz: extend fuzzing coverage (#2040)
Some notes:
* libinjection: according to https://github.com/libinjection/libinjection/issues/44,
it seems NULL characters are valid in the input string;
* RTP: `rtp_get_stream_type()` is called only for RTP packets; if you
want to tell RTP from RTCP you should use `is_rtp_or_rtcp()`;
* TLS: unnecessary check; we already make the same check just above, at
the beginning of the `while` loop
2023-07-11 10:12:08 +02:00
Toni 859d9ea3c3 Fixed heap-overflow if compiled with --enable-tls-sigs. (#2038)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-07-07 18:52:06 +02:00
Toni 72814bed33 Added coverage targets to Makefile.am for convenience. (#2039)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-07-07 17:40:50 +02:00
Ivan KapranovandIvan Kapranov bdd295bc2c fix Stack overflow caused by invalid write in ndpi_automa_match_strin… (#2035)
* minor fixes

fixed 'handle leak' in ndpi_load_malicious_sha1_file and removed the redundant comparison ndpi_search_eaq

* fix Stack overflow caused by invalid write in ndpi_automa_match_string_subprotocol

* fix compile errors

* fix

---------

Co-authored-by: Ivan Kapranov <i.kapranov@securitycode.ru>
2023-07-06 11:13:34 +02:00
Ivan Nardi bb978907e6 STUN: fix detection of Google Voip apps (#2031)
Fix: 2c7fb9179
2023-07-05 14:37:38 +02:00
headshog 076ba967a8 fixed numeric truncation error in diameter.c (#2034) 2023-07-05 14:29:55 +02:00
headshog d51f8929e0 fixed numeric truncation error in rtcp.c (#2033) 2023-07-05 14:29:44 +02:00
headshog 2e5004ddb2 fixed numeric truncation error in kerberos.c (#2032) 2023-07-05 14:29:32 +02:00
Ivan Nardi 7ffd31ebc3 STUN: avoid FacebookVoip false positives (#2029)
Attribute 0xC057 is defined in the Google public implementation of
webrtc (which is used by Google products but also by other applications)
2023-07-03 17:21:46 +02:00
Ivan Nardi dff1f25193 STUN: fix Skype/MsTeams detection and monitoring logic (#2028) 2023-07-03 17:21:32 +02:00
Ivan Nardi 86e89b4e23 STUN: tell RTP from RTCP while in monitoring state (#2027) 2023-06-27 10:33:48 +02:00
Ivan Nardi 2c7fb91794 Hangout: detect Hangout/Duo/GoogleMeet/... in the STUN code (#2025)
Regardless of the name, the removed trace doesn't contain meaningful
Hangout traffic.

Remove last piece of sub-classifiction based only on ip addresses.
2023-06-27 10:33:28 +02:00
Ivan Nardi 31a9da238c STUN: add dissection of DTLS handshake (#2018) 2023-06-26 12:07:26 +02:00
Ivan Nardi 88425e0199 Simplify the report of streaming multimedia info (#2026)
The two fields `flow->flow_type` and `flow->protos.rtp.stream_type` are
pretty much identical: rename the former in `flow->flow_multimedia_type`
and remove the latter.
2023-06-26 12:05:16 +02:00
Ivan Nardi 3a1600ff26 Thrift: fix heap-buffer-overflow (#2024) 2023-06-26 11:41:01 +02:00
Luca Deri 7ef80f6b78 Optimizes and fixes possible out0of0boundary write in ndpi_fill_prefix_v4() 2023-06-23 21:40:55 +02:00
Ivan Nardi 7e64d9f66d RTP: rework code (#2021)
Try avoiding false positives: look for 3 RTP packets before classifing
the flow as such.

Add a generic function `is_rtp_or_rtcp()` to identify RTP/RTCP packets also
in other dissectors (see 3608ab01b commit message for an example)
2023-06-23 08:12:40 +02:00
Ivan KapranovandIvan Kapranov 15f0e1480c minor fixes (#2023)
fixed 'handle leak' in ndpi_load_malicious_sha1_file and removed the redundant comparison ndpi_search_eaq

Co-authored-by: Ivan Kapranov <i.kapranov@securitycode.ru>
2023-06-22 21:36:40 +02:00
Ivan Kapranov aa25c06a18 removed useless call of ndpi_set_risk func (#2022) 2023-06-22 18:26:33 +02:00
Luca Deri cdc0bb8c61 Compilation fix 2023-06-22 16:04:14 +02:00
Toni 1678888284 Add Apache Thrift protocol dissector. (#2007)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-06-22 13:07:32 +02:00
Luca Deri 8dbd591053 Waring fix 2023-06-21 21:26:29 +02:00
Ivan Nardi cc582052b3 Gnutella: improve detection (#2019) 2023-06-21 14:25:34 +02:00
Nardi Ivan 570c75d601 STUN: fix detection over TCP
TCP framing is optional
2023-06-21 10:38:44 +02:00
Nardi Ivan 2ac240ce6a STUN: improve WhatsappCall detection 2023-06-21 10:38:44 +02:00
Ivan Nardi 46aab35a29 Line: fix heap-buffer-overflow error (#2015) 2023-06-21 09:22:07 +02:00
Ivan Nardi 3608ab01b6 STUN: keep monitoring/processing STUN flows (#2012)
Look for RTP packets in the STUN sessions.
TODO: tell RTP from RTCP
2023-06-21 09:16:20 +02:00
Ivan Nardi 04be308092 Fix access to packet/flow information (#2013)
We can't access to `ndpi_str->packet` from `ndpi_detection_giveup()`
2023-06-17 15:46:06 +02:00
Luca Deri ac06ba8320 Added hyperlink 2023-06-16 21:48:21 +02:00
Luca Deri 9cc4cbb9d1 Reworked teams handling 2023-06-15 22:31:11 +02:00
Ivan Nardi 2321c11eb5 Fix compilation (#2011) 2023-06-15 12:01:11 +02:00
Luca Deri d0609ea601 Implemented Zoom/Teams stream type detection 2023-06-14 23:44:57 +02:00
Luca Deri 66bee475ae Improved line protocol dissection with heuristic 2023-06-13 19:20:02 +02:00
Luca Deri de3cf260f8 Updated line test result 2023-06-13 19:07:02 +02:00
Luca Deri be8178fc8d Refreshed ASN lists
Enhanced the Line IP list with https://ipinfo.io/AS23576/125.209.252.0/24 used by line
2023-06-13 19:04:08 +02:00
Luca Deri b02adcc381 Added check to avoid skype heuristic false positives 2023-06-12 23:29:20 +02:00
Toni 4e284b5e40 Set _DEFAULT_SOURCE and _GNU_SOURCE globally. (#2010)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-06-12 19:53:57 +02:00
Toni 8ea0eaa0d0 Fix missing u_char, u_short and u_int typedefs for some platforms e.g.: (#2009)
In file included from ../include/ndpi_main.h:34,
                 from ../include/ndpi_api.h:28,
                 from protocols/activision.c:26:
../include/ndpi_typedefs.h:294:3: error: unknown type name 'u_char'
  294 |   u_char h_dest[6];       /* destination eth addr */
      |   ^~~~~~

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-06-12 17:52:52 +02:00
Nardi Ivan 96340ccea2 oss-fuzz: sync build script with upstream
File copied from https://github.com/google/oss-fuzz/blob/master/projects/ndpi/build.sh

The general idea is to keep the build script in our repository and use
it from oss-fuzz builder: updating it from our side is easier and faster
then passing via an oss-fuzz PR.

The original idea is from @utoni in 3068306b60.

Once this change has been merged, we can update the code in oss-fuzz.
2023-06-12 15:17:56 +02:00
Luca Deri 63a2b359b9 Added vlan_id in ndpi_flow2json() prototype 2023-06-09 19:50:47 +02:00
Nardi Ivan 7150b40c63 QUIC: fix dissection of packets forcing VN 2023-06-08 17:07:25 +02:00
Nardi Ivan 3e640a13a4 QUIC: fix a memory access error
Found while fuzzing
2023-06-08 17:07:25 +02:00
Nardi Ivan fd4cb10190 QUIC: add support for QUIC version 2
See: https://www.rfc-editor.org/rfc/rfc9369.txt

Old v2-01 version has been removed, since it has never been really used.
2023-06-08 17:07:25 +02:00
Ivan Nardi 9987e5b482 ndpiReader: allow to configure LRU caches TTL and size (#2004) 2023-06-08 17:06:32 +02:00
Ivan Nardi 3e673e91a9 ProtonVPN: add basic detection (#2006) 2023-06-08 16:52:55 +02:00
headshog 7dcceb4daf Numeric truncation at tls.c:1010 (#2005)
* fixed numtrunc error in protocols/tls.c

* fixed build error for tls.c
2023-06-08 07:52:18 +02:00
zehraInandNardi Ivan 436ab54f5a added new domain names (#2002)
* added new domain names

* Sync unit tests results

---------

Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2023-06-01 13:13:49 +02:00
Ivan Nardi e14d0acbd4 tests: add an option to force the overwrite of the unit tests results (#2001)
Usage: `FORCE_UPDATING_UTESTS_RESULTS=1 ./tests/do.sh`
2023-05-31 09:05:00 +02:00
Maatuq e17fa1259a Add bitcoing protocol dissector. (#1992)
* Add bitcoing protocol dissector.

* remove bitcoin protcol detection from mining.c
* add a new bitcoin deissector.
* add a new category: Cryptocurrency.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* Remove useless checks and add missing windows and docs file.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* update affected tests.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* add a brief version.

Add notes on the difference between normal bitcoin protocol and the
mining protocol.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* update enable_payload_stat test after dev rebasing.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

---------

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
2023-05-31 07:31:01 +02:00
headshog a8d2eeddd2 Numeric truncation at ndpi_analyze.c at lines 101, 104, 107, 110 (#1999)
* fixed numeric truncation error in ndpi_analyze.c

* fixed numeric truncation error in ndpi_analyze.c x2

* fixed numeric truncation error in ndpi_analyze.c x3

* fixed numeric truncation error in ndpi_analyze.c and printf format

* fixed tests
2023-05-30 12:27:47 +02:00
Toni 04f5c5196e Changed logging callback function sig. (#2000)
* make user data available for any build config

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-30 08:27:37 +02:00
headshog 552c9b3cc6 fixed numeric truncation error in ndpi_main.c:6837 (#1998) 2023-05-29 19:34:26 +02:00
Ivan Nardi efb261a95c Fix some memory errors triggered by allocation failures (#1995)
Some low hanging fruits found using nallocfuzz.
See: https://github.com/catenacyber/nallocfuzz
See: https://github.com/google/oss-fuzz/pull/9902

Most of these errors are quite trivial to fix; the only exception is the
stuff in the uthash.
If the insertion fails (because of an allocation failure), we need to
avoid some memory leaks. But the only way to check if the `HASH_ADD_*`
failed, is to perform a new lookup: a bit costly, but we don't use that
code in any critical data-path.
2023-05-29 19:24:00 +02:00
Ivan Nardi 346bb268e2 Avoid calling ndpi_reconcile_protocols() twice in ndpi_detection_giveup() (#1996)
`ndpi_reconcile_protocols()` is already called by
`ndpi_set_detected_protocol()`
2023-05-29 16:55:53 +02:00
Ivan Nardi 82fa3a0986 Fix an heap-buffer-overflow (#1994)
```
==32439==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x604000000080 at pc 0x564a9a1639ac bp 0x7ffc77d42e20 sp 0x7ffc77d42e18
READ of size 1 at 0x604000000080 thread T0
    #0 0x564a9a1639ab in ndpi_check_punycode_string /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:98:12
    #1 0x564a9a26025a in ndpi_match_host_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8652:6
    #2 0x564a9a33e45c in ndpi_http_parse_subprotocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:446:9
    #3 0x564a9a3354b8 in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:921:3
    #4 0x564a9a32dc8b in process_request /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1237:3
    #5 0x564a9a329a9c in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1351:5
    #6 0x564a9a3293ed in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1425:3
    #7 0x564a9a21c64a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5906:6
    #8 0x564a9a21d057 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5954:12
```

Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=59393
2023-05-29 16:53:39 +02:00
Ivan Nardi 46ff069117 ndpiReader: improve printing of payload statistics (#1989)
Add a basic unit test

Fix an endianess issue
2023-05-29 16:53:11 +02:00
Ivan Nardi 7ce14da0c8 Remove some useless checks (#1993)
In the main dissector callbacks the flow protocols are (almost) always
unknown. Only two exceptions:
* extra dissection data path
* HTTP sub-protocols
2023-05-28 12:59:18 +02:00
Toni 6da3474203 Improved helper scripts. (#1986)
* added additional (more restrictive) checks

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-28 12:45:44 +02:00
Ivan Nardi b11e6a453b Add support for Epic Games and GeForceNow/Nvidia (#1990) 2023-05-27 12:13:54 +02:00
Ivan Nardi 63ac50e4f4 Improve detection of Alibaba flows (#1991) 2023-05-27 10:19:58 +02:00
Toni 334b43579e Fixed invalid use of ndpi_free(). Sorry, my fault. (#1988)
* Fixed invalid use of ndpi_free(). Sorry, my fault.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Fine tuned symbol check script.

 * added check for expected syms in modules

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

---------

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-05-24 13:19:06 +02:00
Luca Deri f692342f3c Cosmetic changes 2023-05-23 22:16:34 +02:00
Ivan Nardi 86b56646b5 ndpiReader: fix export of DNS/BitTorrent attributes (#1985)
There is no BitTorrent hash in the DNS flows
2023-05-20 17:23:48 +02:00
Toni 5e8f93c2d1 Improved missing usage of nDPIs malloc wrapper. Fixes #1978. (#1979)
* added CI check

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-20 16:18:52 +02:00
Luca Deri 8f718c9051 Made VK protocol detection more strict 2023-05-20 15:46:47 +02:00
Toni c97e2d67ff Added scripts to auto generate hostname/SNI *.inc files. (#1984)
* add illegal gambling sites (Belgium)

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-20 15:41:15 +02:00
Ivan Nardi 9004d5c2ca ndpiReader: fix export of HTTP attributes (#1982) 2023-05-20 15:12:14 +02:00
Nardi Ivan 1ab5318f9c Sync unit tests results 2023-05-20 15:09:51 +02:00
headshog dd7198fcaf fixed numeric truncation error 2023-05-20 15:09:51 +02:00
Luca Deri 5ca6f0ac62 Implemented ndpi_predict_linear() for predicting a timeseries value overtime 2023-05-19 11:46:03 +02:00
Yuriy Rymarchuk a8b3baf2b8 Add support for roaring_bitmap_xor_inplace (#1983) 2023-05-18 13:28:32 +02:00
Ivan Nardi b1bcf1ff60 Fix classification-by-ip in ndpi_detection_giveup (#1981)
Return the "classification-by-ip" as protocol results only if no other
results are available.
In particular, never return something like
"protocol_by_port/protocol_by_ip" (i.e. `NTP/Apple`,
BitTorrent/GoogleCloud`, `Zoom/AWS`) because this kind of classification
is quite confusing, if not plainly wrong.

Notes:
* the information about "classification-by-ip" is always available, so
no information is lost with this change;
* in the unit tests, the previous classifications with confidence
`NDPI_CONFIDENCE_DPI_PARTIAL` were wrong, as noted in #1957
2023-05-17 11:26:25 +02:00
Ivan Nardi ace32c9dfe Add support for SRTP (#1977)
The goal is to have Zoom flows classified as "Encrypted" and not as
"Cleartext".

Start documenting the list of protocols supported by nDPI;
format, verbosity and content are still a work-in-progress.
2023-05-16 16:02:14 +02:00
Ivan Nardi 4e186f6bfb HTTP: rework state machine (#1966)
The goal if to correlate the right request-response pair, exporting
metadata from only one transaction (for example, the right url & return
state pair)

As a nice side effect, the code should be much cleaner, but that is a
matter of taste.

Two differences respect to the previous code:
* as it happens in the CI, if in the flow there are only one response
(before) and one request (after), only the metadata of the response are
saved/exported
* for performance reasons, we don't call `ndpi_parse_packet_line_info()`
anymore for ALL packets triggering the HTTP dissector, but only for the
packets that we already know belong to an HTTP flow. This is the reason
for the changes in RTSP/SOAP/... code
2023-05-16 16:01:55 +02:00
Ivan Nardi 8c224b464f Fix MS Teams detection with heuristic (#1972)
When we reconcile a TLS session to Teams, we need to keep TLs as master.
This way:
* we keep exporting all the TLS metadata
* we avoid some memory leaks (of these metadata themeselves)
2023-05-10 21:00:27 +02:00
Luca Deri 596a5fd5dc Updated results 2023-05-10 17:11:14 +02:00
Luca 64853dff02 Improved MS Teams detection with heuristic 2023-05-10 16:59:28 +02:00
Luca 98f029a091 Enhanced MS teams STUN/Azure detection 2023-05-10 16:28:34 +02:00
Luca 7d5e4d0f7e MS Teams enhancement 2023-05-10 15:41:26 +02:00
Luca 8de7c49298 MS Teams improvements 2023-05-10 15:16:39 +02:00
Ivan Nardi 31d23aee56 All protocols should be excluded sooner or later (#1969)
For a lot of protocols, reduce the number of packets after which the
protocols dissector gives up.
The values are quite arbitary, tring to not impact on classification
2023-05-10 12:50:24 +02:00
Ivan Nardi 99d7066ea0 Remove special handling of some TCP flows without SYN (#1965)
This piece of code has multiple problems:
* nDPI is able to detect some TCP protocols even with mid-flows (i.e.
without the initial packets of the session); TLS is the most
significative example
* since e6b332aa4a it is perfectly valid
to not pass the TCP Handshake packets to nDPI
* in any case, we shouldn't call `ndpi_detection_giveup()`. That
function is usually called by the application and we end up calling it
twice in some cases.

The simple solution is to completely remove that code: process these
kinds of flows like everyone else.

Note that the application can always avoid to pass to nDPI any TCP flows
without the initial handshake; the flow managemnt is always up to the
application.

Looking at the CI results, some rare flows are now processed significantly
longer. As a follow-up we could look into that.
2023-05-09 19:36:02 +02:00
Ivan Nardi 684e041998 Improve detection of crawlers/bots (#1968)
Add support for Facebook crawler
2023-05-09 16:42:29 +02:00
Ivan Nardi 79c1dbe0b2 Extend content-match list (#1967) 2023-05-09 16:41:12 +02:00
Luca Deri 2f9059dc54 Disabled query string validation in MDNS in order to avoid zapping chars that in DNS (instead) are not permitted 2023-05-09 15:04:02 +02:00
Luca Deri bfe79243bc Refreshed lists 2023-05-08 16:51:38 +02:00
Clément BRUGUERA 998bedbbb7 add 2 ns from fdn.fr to DoH section (#1964) 2023-05-05 18:08:13 +02:00
Ivan Nardi 0223d3c4f5 HTTP: improve extraction of metadata and of flow risks (#1959) 2023-05-05 13:35:20 +02:00
Ivan Nardi 02a2c80453 ndpiReader: fix print of flow payload (#1960) 2023-05-04 11:52:48 +02:00
Ivan Nardi a11ddd2dc4 HTTP: remove useless code about XBOX (#1958)
Old nDPI versions were able to detect XBOX flows over HTTP via
user-agent matching. This feature has been removed from a long time
(89d548f9d, at very least)
2023-05-04 11:28:03 +02:00
Ivan Nardi 6b94c9675a Improve detection of crawler/bot traffic (#1956) 2023-05-04 11:27:34 +02:00
Ivan Nardi 9a5d60bff6 Sync unit tests results (#1962) 2023-05-04 09:47:52 +02:00
Luca Deri 3681de4795 Updated pcap detection results after Facebook Reel/Stories support 2023-05-03 22:52:58 +02:00
Luca Deri de13c6c0bd Added detection of Facebook Reels and Stories
https://www.switcherstudio.com/blog/the-ultimate-guide-to-reels-and-stories#:~:text=Audience%3A%20Reels%20are%20served%20to,things%20like%20photos%20and%20boomerangs.
2023-05-03 22:50:02 +02:00
Ivan Nardi bb370f5ef0 Make Bittorrent LRU cache IPv6 aware. (#1909)
It was the only remaining LRU cache without IPv6 support.
See 81e1ea545c
2023-04-29 11:09:23 +02:00
Luca Deri 6b803f48dc Added missing check 2023-04-28 23:08:54 +02:00
Luca Deri 6e9dce357b Added ndpi_guess_undetected_protocol_v4() to restore logic that
was removed in https://github.com/ntop/nDPI/commit/032e778a6dc1ee2173c4cdb2c671180c5b5df245
2023-04-28 23:00:33 +02:00
Luca Deri f7138f07a6 Missing module termination 2023-04-28 23:00:33 +02:00
Luca Deri b39ece4428 Updated (C) 2023-04-28 23:00:33 +02:00
Luca Deri 791d3e6ad3 Added missing check 2023-04-28 23:00:33 +02:00
Ivan Nardi 8934f7b45f Add an heuristic to detect/ignore some anomalous TCP ACK packets (#1948)
In some networks, there are some anomalous TCP flows where the smallest
ACK packets have some kind of zero padding.
It looks like the IP and TCP headers in those frames wrongly consider the
0x00 Ethernet padding bytes as part of the TCP payload.
While this kind of packets is perfectly valid per-se, in some conditions
they might be treated by the TCP reassembler logic as (partial) overlaps,
deceiving the classification engine.
Add an heuristic to detect these packets and to ignore them, allowing
correct detection/classification.

This heuristic is configurable. Default value:
* in the library, it is disabled
* in `ndpiReader` and in the fuzzers, it is enabled (to ease testing)

Credit to @vel21ripn for the initial patch.

Close #1946
2023-04-25 19:25:07 +02:00
Ivan Nardi 40b6d5a2e1 fuzz: extend fuzzers coverage (#1952) 2023-04-25 16:37:28 +02:00
Ivan Nardi de693cbbc9 RTP: remove dead-code (#1953)
The checks `isValidMSRTPType(..) == 1` is a subset of
`is_valid_rtp_payload_type()` so this if-branch is never reached.

More importantly, the article describing how to detect Microsoft Lync and
Skype for Business is from 2014. These payload types are static or they
are in the dynamic range: in both cases, these values might be used (and
they are used indeed) pretty much by every application.
Bottom line: we can't use PT alone to identify a specific protocol.

Keep the list, since it is used to tell audio streams from video ones.
2023-04-25 14:16:27 +02:00
Luca Deri c4ba981370 Compilation fix on RH7 2023-04-24 08:49:11 +02:00
nikitamishagin d3e67faf20 Add "Heroes of the Storm" video game signature detection. (#1949) 2023-04-22 18:28:43 +02:00
Toni b6629ba2be Improved debug output. (#1951)
* try to get rid of some `printf(..)`s as they do not belong to a shared library
 * replaced all `exit(..)`s with `abort()`s to indicate an abnormal process termination

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-21 12:40:26 +02:00
Toni 85751ba9d7 Added OICQ dissector. (#1950)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-21 11:30:10 +02:00
Ivan Nardi ba99330031 ndpiReader: fix flow stats (#1943) 2023-04-14 11:17:37 +02:00
Ivan Nardi 032e778a6d Simplify ndpi_internal_guess_undetected_protocol() (#1941)
`ndpi_guess_undetected_protocol()/ndpi_internal_guess_undetected_protocol()`
is a strange function:
* it is exported by the library and it is actively used by `ntopng`
* it is intrinsecally ipv4-only
* it returns basically something like "classification_by_ip"/"classification_by_port"
(these information have already been calculated in `ndpi_do_guess()`...)
* it access the bittorrent LRU caches (similarly to
`ndpi_detection_giveup()` but without all the other caches...)

So:
* make the interface IPv4/6 agnostic
* use the classifications already available

This work will allow to make the Bittorrent caches IPV6-aware (see
81e1ea5).

Handle Dropbox classification "by-port" in the "standard" way.
2023-04-12 14:39:10 +02:00
Toni 6660feed35 Added BACnet dissector. (#1940)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 20:26:27 +02:00
lns 77ea80862f Improved debug logging.
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 15:39:09 +02:00
lns 6da958655c Improved invalid logging via printf().
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 15:39:09 +02:00
Toni bebcce5281 Added Source Engine dissector. (#1937)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 14:31:57 +02:00
Luca Deri 9ba16a4837 Test files for riit games 2023-04-11 10:14:18 +02:00
Jacopo a469009842 RiotGames: add detection of flows (#1935)
* RiotGames: add detection of flows

* remove of akamai domains
2023-04-11 10:04:09 +02:00
Ivan Nardi c7ef3608d9 CI: fix Performance job (#1936)
Fix: 7714507f
2023-04-09 10:16:28 +02:00
Luca Deri c47e9d201d Implemented ndpi_XXX_reset() API calls whre XXX is ses, des, hw 2023-04-08 09:52:14 +02:00
Ivan Nardi 7714507f81 Test multiple ndpiReader configurations (#1931)
Extend internal unit tests to handle multiple configurations.
As some examples, add tests about:
* disabling some protocols
* disabling Ookla aggressiveness

Every configurations data is stored in a dedicated directory under
`tests\cfgs`
2023-04-06 11:30:36 +02:00
Ivan Nardi 7a627296f0 Fix LRU/Patricia/Automa stats in ndpiReader with multiple threads (#1934) 2023-04-06 09:36:11 +02:00
Ivan Nardi b639744c3a Extend content list of Microsoft protocols (#1930) 2023-04-06 09:35:41 +02:00
Ivan Nardi 22cf8b5d8f configure: add an option to enable debug build, i.e -g (#1929)
Add this new flag to one CI job, to test it

Close #1925
2023-04-06 09:35:27 +02:00
Ivan Nardi 06a63dcbaa MySql: improve detection (#1928)
Add support for flows with "caching_sha2_password" authentication plugin.

See #1924
2023-04-06 09:34:39 +02:00
Ivan Nardi 5c28dbbae2 H323: fix false positives (#1916) 2023-04-06 09:32:57 +02:00
Ivan Nardi 25c1111911 fuzz: add a new fuzzer triggering the payload analyzer function(s) (#1926) 2023-04-04 14:39:29 +02:00
Maatuq f1193d5e6f add support for gre decapsulation (#1442) (#1921) 2023-04-04 14:20:11 +02:00
Ivan Nardi cc5aec5f0a fuzz: add fuzzer to test internal gcrypt code (#1920) 2023-04-04 14:19:41 +02:00
Ivan Nardi d7662379b4 Update libinjection code (#1918)
Update libinjection code to the current master https://github.com/libinjection/libinjection/commit/7e4b74e824dc3f0623ba4894df2a0d817351ec39

The goal is to finally fix #1820
See: https://github.com/libinjection/libinjection/issues/33

Update the corpus of the libinjection fuzzers

Close #1820
2023-04-04 14:18:57 +02:00
Ivan Nardi 4d11941d32 Ookla: rework detection (#1922)
The logic of the LRU cache has been changed: once we know an ip has
connected to an Ookla server, all the following (unknown) flows (for
a short time interval) from the same ip to the port 8080 are treated
as Ookla ones.

Most of the changes in this commit are about introducing the concept of
"aggressive detection". In some cases, to properly detect a
protocol we might use some statistical/behavior logic that, from one
side, let us to identify the protocol more often but, from the other
side, might lead to some false positives.
To allow the user/application to easily detect when such logic has been
triggered, the new confidence value `NDPI_CONFIDENCE_DPI_AGGRESSIVE` has been
added.
It is always possible to disable/configure this kind of logic via the
API.

Detection of Ookla flows using plain TLS over port 8080 is the first
example of aggressive detection in nDPI.

Tested with:
* Android 9.0 with app 4.8.3
* Ubuntu 20.04 with Firefox 110
* Win 10 with app 1.15 and 1.16
* Win 10 with Chrome 108, Edge 108 and Firefox 106
2023-03-30 17:13:51 +02:00
Ivan Nardi 3e06bcce8d Add another example of custom rules (#1923)
Add an example where traffic matching the same IP, but different ports
is classified to different protocols.

Close #189
2023-03-30 08:45:17 +02:00
Ivan Nardi 6e0cd16225 TLS: fix another interger overflow in certificate processing (#1915) 2023-03-30 05:12:40 +02:00
Maatuq 9ea8a57839 remove redefinition to vxlanhdr struct in vxlan dissector (#1911) 2023-03-25 19:25:22 +01:00
Ivan Nardi 04a426feef ndpiReader: fix VXLAN de-tunneling (#1913)
```
==20665==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6040000aec81 at pc 0x0000004f5c6f bp 0x7fff07e9e1f0 sp 0x7fff07e9e1e8
READ of size 1 at 0x6040000aec81 thread T0
SCARINESS: 12 (1-byte-read-heap-buffer-overflow)
    #0 0x4f5c6e in ndpi_is_valid_vxlan ndpi/example/reader_util.c:1784:6
    #1 0x4f5c6e in ndpi_workflow_process_packet ndpi/example/reader_util.c:2292:16
    #2 0x4dd821 in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_ndpi_reader.c:135:7
    #3 0x4f91ba in ExecuteFilesOnyByOne /src/aflplusplus/utils/aflpp_driver/aflpp_driver.c:234:7
    #4 0x4f8f8c in main /src/aflplusplus/utils/aflpp_driver/aflpp_driver.c:318:12
    #5 0x7f2289324082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
    #6 0x41e6cd in _start
```
Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=57369
2023-03-25 19:19:51 +01:00
Ivan Nardi ddb9aa0f95 TLS: fix parsing of certificate elements (#1910)
```
==1228==ERROR: AddressSanitizer: SEGV on unknown address 0x6040000bed05 (pc 0x00000056e148 bp 0x7ffcca534320 sp 0x7ffcca5330c0 T0)
==1228==The signal is caused by a WRITE memory access.
    #0 0x56e148 in processCertificateElements ndpi/src/lib/protocols/tls.c:682:79
    #1 0x56c60f in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_tls_certificate.c:43:3
    #2 0x43de63 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15
    #3 0x4295c2 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:324:6
    #4 0x42ee6c in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:860:9
    #5 0x4583a2 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    #6 0x7f8c021c9082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
    #7 0x41f78d in _start
```

Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=57317
2023-03-22 18:18:36 +01:00
Maatuq 530d0de438 Add support for vxlan decapsulation (#1441) (#1900)
Close #1441
2023-03-22 18:18:12 +01:00
Ivan Nardi 04e017a854 Rework CI jobs to try reducing CI duration (#1903)
CI duration is quite long: the longest jobs is the "Performance" one.
Try to reduce the overall duration: that job (and some others) will not
be triggered for each PR/commit anymore, but asynchronously, once a day
(this scheduling seems right since the frequency of the PR/commits in
the project).
It should be possibly to trigger them manually, via GUI, anyway.

Remove two identical jobs; we already tests ASAN with 4 different
compilers.

After 9eff0754 it is safe to reduce fuzzing time.

Bottom line: try to have as upper-time of CI tests the duration of the
fuzzing jobs
2023-03-22 18:17:25 +01:00
Luca Deri 64ebf73b29 Added the ability to define custom protocols with arbitrary Ids in proto.txt
Example
- ip:213.75.170.11/32:443@CustomProtocol
nDPI assigns an is that can change based on protos.txt content

- ip:213.75.170.11/32:443@CustomProtocol=9999
nDPI assigns 9999 as protocolId to CustomProtocol and won't change when
protos.txt content will chaneg
2023-03-22 00:15:56 +01:00
Luca Deri b3457c644f Removed overlapping port 2023-03-21 19:27:56 +01:00
Luca Deri c22d30786c Improved checks for duplicated entries in protocols file 2023-03-21 19:25:03 +01:00
Ivan Nardi 2a3ade397b DisneyPlus/Hulu ip lists should be auto-generated (#1905)
Remove two stale ip lists:
1) these 3 ips are in the Amazon ranges (now)...
2) the Instagram list originated from AS32934, which is now a Facebook
AS; see https://github.com/ntop/nDPI/pull/1264/commits/8dabd06301a802dd38616ba8684a1d995783e023
2023-03-20 19:26:40 +01:00
Ivan Nardi 3a75a46212 Add a new protocol id for generic Adult Content traffic (#1906)
The list has been taken from https://www.similarweb.com/top-websites/adult/
Fix a GoTo false positive.
2023-03-20 17:56:02 +01:00
Ivan Nardi 12867962b0 Add a new protocol id for generic advertisement/analytics/tracking stuff (#1904) 2023-03-20 17:01:18 +01:00
Ivan Nardi 0e80828e14 fuzz: add a new fuzzer to test TLS certificates (#1901) 2023-03-20 17:00:46 +01:00
Luca Deri 9f46d41eca FreeBSD compilation fix
(C) update
2023-03-14 22:16:57 +01:00
Ivan Nardi 9eff075419 fuzz: simplify fuzzers dependencies in CIFuzz (#1896)
CIFuzz (based on oss-fuzz) is the GitHub action/CI job that runs fuzz
targets on pull requests. It only runs the fuzzers affected by a pull
request/commit. Otherwise it will divide up the allotted fuzzing time
among all fuzzers in the project.
Since:
* we have more than 20 fuzzers and most of them use the custom memory
allocation functions (to force allocation failures) even if they are not
strictly about DPI stuff;
* we need to keep fuzzing time relatively small (to avoid waiting the CI
results for a long time)

it is important that fuzzers dependencies (which are based on *files*
changed by the single commit/PR) are as small as possible.

Bottom line: move all the low-level allocation callbacks to a dedicated
file; this way most of the fuzzers don't depend anymore on `ndpi_main.c`
file (which is touched by ever commit/PR).

The goal is to have only the "most important" fuzzers running during (most
of) the CI.
2023-03-14 19:34:01 +01:00
Luca Deri 3585e2d201 Added ability to define an unlimited number of custom rules IP:port for the same IP (it used tobe limited to 2) 2023-03-13 21:57:14 +01:00
Luca Deri ebfc1fe827 Minor cleanup 2023-03-13 20:52:08 +01:00
Luca Deri d398f232be Added fix for better handling exceptions rollback in case of later match 2023-03-06 17:33:08 +01:00
Ivan Nardi 89cae9ddf2 Add a new flow risk about literal IP addresses used as SNI (#1892)
RFC 6066 3: "Literal IPv4 and IPv6 addresses are not permitted in
"HostName"."

Don't set this risk if we have a valid sub-classification (example:
via certificate)

Since a similar risk already exists for HTTP hostnames, reuse it, with a
more generic name.
2023-03-02 15:27:30 +01:00
Ivan Nardi 3047e286c0 Fix insert of ip addresses into patricia tree(s) (#1895)
Fix: d0b46dcf
2023-03-02 14:30:20 +01:00
Ivan Nardi 22fb8349b9 ndpiReader: print how many packets (per flow) were needed to perform full DPI (#1891)
Average values are already printed, but this change should ease to
identify regressions/improvements.
2023-03-01 21:50:47 +01:00
Toni 2768366969 Fixed missing AS_HELP_STRING in configure.ac. (#1893)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-03-01 20:46:40 +01:00
Nardi Ivan e0ad5b49c0 Sync unit tests results 2023-03-01 20:34:56 +01:00
Nardi Ivan d65f8cf15f Keep master protocol in ndpi_reconcile_protocols
As a side effect of this change, the following memory leak has been
fixed.
```
==55523==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 2 byte(s) in 1 object(s) allocated from:
    #0 0x55657d01b7be in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_alloc_fail+0x5b27be) (BuildId: 9dfa7bd9081421417b8b1c2e8b44633a9bfcfe52)
    #1 0x55657d169182 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:243:25
    #2 0x55657d1694c8 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:310:13
    #3 0x55657d25ddc7 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2308:46
    #4 0x55657d2701b9 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:895:5
    #5 0x55657d26a408 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1223:11
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56466
2023-03-01 20:34:56 +01:00
Nardi Ivan 9eb9664516 SoftEther: fix invalid memory access
We can't write `flow->protos` union until we are really sure about protocol
classification

```
==28334==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x558db5554512 bp 0x000000000000 sp 0x7ffcb22c2880 T0)
==28334==The signal is caused by a READ memory access.
==28334==Hint: this fault was caused by a dereference of a high value address (see register values below).  Disassemble the provided pc to learn which register was used.
    #0 0x558db5554512 in __asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x48e512) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7)
    #1 0x558db55ea54b in __interceptor_free (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x52454b) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7)
    #2 0x558db56977ca in ndpi_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:274:7
    #3 0x558db56c20e3 in ndpi_free_flow_data /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5175:2
    #4 0x558db569783f in ndpi_free_flow /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8394:5
    #5 0x558db5627936 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:38:3
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56272
2023-03-01 20:34:56 +01:00
Luca Deri 127d8d0d35 Fixed uninitialized variable 2023-02-27 21:26:41 +01:00
Luca Deri 21c7550157 Moving to bash 2023-02-27 16:21:12 +01:00
Luca Deri a303756edb Added missing break 2023-02-27 14:38:40 +01:00
Luca Deri 25420e266e Updated results after the latest changes 2023-02-27 12:20:39 +01:00
Luca Deri 807c97344f in case of failure, failing result files are not listed 2023-02-27 12:20:15 +01:00
Luca Deri 96f0f85e56 Indent fix 2023-02-27 12:20:06 +01:00
Luca Deri 6a2e8c9fb1 Improved protocol detection exploiting IP-based guess
Reworked ndpi_reconcile_protocols() that is now called only in front of a match (less overhead)
2023-02-27 12:18:39 +01:00
Luca Deri d0b46dcf88 Enhanced custom port definition and improved error reporting in case of duplications 2023-02-24 09:35:26 +01:00
Ivan Nardi 4075324e2b fuzz: extend fuzz coverage (#1888) 2023-02-16 18:04:34 +01:00
Ivan Nardi 06d4f8c7e5 Update GitHub runners versions (#1889)
See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
2023-02-14 21:30:54 +01:00
Ivan Nardi f2f8f9d0ee Fix export/serialization of flow->risk (#1885)
Any risks bigger than 31 was never exported/serialized; found looking
a the fuzz coverage report.
2023-02-14 21:30:36 +01:00
Ivan Nardi 090ba5735f Sync utests results (#1887) 2023-02-13 13:17:01 +01:00
Ivan Nardi f10178f8d2 Fix packet counters (#1884)
We need to keep separete counters to keep tracks of packet numbers with
and without any payload.

Regression introduced in 5849863ef
2023-02-13 11:32:17 +01:00
Ivan Nardi 200d1d4cfc Fix detection of packet direction and NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1883) 2023-02-13 11:31:52 +01:00
Ivan Nardi a3a9a72f31 Fix compilation with GCC-7 and latest RoaringBitmap code (#1886)
Latest RoaringBitmap version (introduced with bf413afb) triggers a new
warning with GCC-7:

```
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ CC=gcc-7 CXX=g++-7 ./autogen.sh && make -s
autoreconf: Entering directory `.'
[...]
third_party/src/roaring.c:1815:1: warning: ‘no_sanitize’ attribute directive ignored [-Wattributes]
 static inline int array_container_cardinality(const array_container_t *array) {
 ^~~~~~
third_party/src/roaring.c:1964:5: warning: ‘no_sanitize’ attribute directive ignored [-Wattributes]
     const array_container_t *container2) {
[..]
```

The core issue is that `no_sanitize` attribute is defined only for GCC
>= 8.
That breaks the CI since we still use GCC-7 and `-Werror`: add a simple
workaround.

Fix compilation on Windows
2023-02-13 11:19:11 +01:00
Luca Deri 85bab9e660 Centos7 fixes 2023-02-13 01:02:27 +01:00
Luca Deri bf413afba1 Update roaring bitmap code 2023-02-12 12:50:24 +01:00
0xA50C1A1 ba4e145aad Add Yandex services detection (#1882)
Add Yandex services detection

Add VK and Yandex to the TLS certificate match list
2023-02-09 20:02:43 +01:00
Ivan Nardi b51a2ac72a fuzz: some improvements and add two new fuzzers (#1881)
Remove `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` define from
`fuzz/Makefile.am`; it is already included by the main configure script
(when fuzzing).

Add a knob to force disabling of AESNI optimizations: this way we can
fuzz also no-aesni crypto code.

Move CRC32 algorithm into the library.

Add some fake traces to extend fuzzing coverage. Note that these traces
are hand-made (via scapy/curl) and must not be used as "proof" that the
dissectors are really able to identify this kind of traffic.

Some small updates to some dissectors:

CSGO: remove a wrong rule (never triggered, BTW). Any UDP packet starting
with "VS01" will be classified as STEAM (see steam.c around line 111).
Googling it, it seems right so.

XBOX: XBOX only analyses UDP flows while HTTP only TCP ones; therefore
that condition is false.

RTP, STUN: removed useless "break"s

Zattoo: `flow->zattoo_stage` is never set to any values greater or equal
to 5, so these checks are never true.

PPStream: `flow->l4.udp.ppstream_stage` is never read. Delete it.

TeamSpeak: we check for `flow->packet_counter == 3` just above, so the
following check `flow->packet_counter >= 3` is always false.
2023-02-09 20:02:12 +01:00
0xA50C1A1 4bb851384e Add VK detection (#1880) 2023-02-02 15:27:59 +01:00
Luca Deri 64b8ae38db Typos 2023-02-01 21:15:30 +00:00
Luca Deri 563a24bf6b Minor update 2023-02-01 18:31:59 +01:00
Luca Deri c711f480e3 Moved to 4.7 2023-02-01 18:30:17 +01:00
Alfredo Cardigliano b0a340cbc6 Update CHANGELOG.md 2023-02-01 10:23:41 +01:00
Luca Deri c42b046d87 Updated elastic_search file name in project 2023-01-31 21:02:41 +01:00
Luca Deri 80368772bd File rename to avoid name clashes (e.g. on Windows) 2023-01-31 20:54:00 +01:00
Luca Deri 2086058d00 Updated (C) 2023-01-31 20:08:30 +01:00
Alfredo Cardigliano a151276db8 Update changelog 2023-01-31 17:25:55 +01:00
Ivan Nardi fe043d3af5 Further reduction of the size of some traces used as unit test (#1879)
See a944514d. No flow/classification/metadata have been removed.
2023-01-30 11:10:54 +01:00
Ivan Nardi 9f27cd56b0 ndpiReader: fix packet dissection (CAPWAP and TSO) (#1878)
Fix decapsulation of CAPWAP; we are interested only in "real" user data
tunneled via CAPWAP.
When Tcp Segmentation Offload is enabled in the NIC, the received packet
might have 0 as "ip length" in the IPv4 header
(see
https://osqa-ask.wireshark.org/questions/16279/why-are-the-bytes-00-00-but-wireshark-shows-an-ip-total-length-of-2016/)

The effect of these two bugs was that some packets were discarded.

Be sure that flows order is deterministic
2023-01-30 10:59:18 +01:00
Ivan Nardi 3e6cadbb76 ndpireader: fix "Discarded bytes" statistics (#1877) 2023-01-27 07:09:34 +01:00
Ivan Nardi 6d00a9e0bd fuzz: add a new fuzzer to test serialization/deserialization code (#1876)
Autodetecting the needed buffer size is quite complex (especially with
float/double values) so it is mandatory to properly check for
`ndpi_snprintf` truncation.
These issues have been undetected so far probably because the default
buffer is big enough for all common cases.

Add an example of usage of `ndpi_deserialize_clone_all()` (taken from
`ntopng`)
2023-01-27 07:09:18 +01:00
Luca Deri eea9956430 Improved connection refused detection 2023-01-25 22:18:50 +01:00
Ivan Nardi b2a8b48386 Fix compilation and CI (#1875)
ubuntu-latest s390x CI doesn't like snapshot bigger than 262144 bytes.

Fix an error found by fuzz CI

```
=================================================================
==55399==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x561e41cb684d bp 0x7ffd54ce3650 sp 0x7ffd54ce3520 T0)
==55399==The signal is caused by a READ memory access.
==55399==Hint: address points to the zero page.
    #0 0x561e41cb684d in ndpi_network_ptree_match /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2321:41
    #1 0x561e41d30879 in ndpi_guess_undetected_protocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7767:8
    #2 0x561e41ca804d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_config.cpp:104:5
    #3 0x561e41bb96a0 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x4726a0) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
    #4 0x561e41ba3c2f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x45cc2f) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
    #5 0x561e41ba96f6 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x4626f6) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
    #6 0x561e41bd22e2 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x48b2e2) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
    #7 0x7f94f0e5c082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #8 0x561e41b9eb0d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x457b0d) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
```
2023-01-25 14:49:45 +01:00
Ivan Nardi 3477560639 Update every ip lists (#1872) 2023-01-25 11:53:18 +01:00
Luca Deri 464269e5ed CenOS compilation fix 2023-01-25 11:52:46 +01:00
Luca Deri a3d3856c80 Compilation fixes 2023-01-25 11:45:35 +01:00
Ivan Nardi 9fc724de5a Add some fuzzers to test other data structures. (#1870)
Start using a dictionary for fuzzing (see:
https://llvm.org/docs/LibFuzzer.html#dictionaries).
Remove some dead code.
Fuzzing with debug enabled is not usually a great idea (from performance
POV). Keep the code since it might be useful while debugging.
2023-01-25 11:44:59 +01:00
Ivan Nardi 29c5cc39fb Some small changes (#1869)
All dissector callbacks should not be exported by the library; make static
some other local functions.
The callback logic in `ndpiReader` has never been used.
With internal libgcrypt, `gcry_control()` should always return no
errors.
We can check `categories` length at compilation time.
2023-01-25 11:44:09 +01:00
Luca Deri 5849863ef9 Added new risk NDPI_TCP_ISSUES 2023-01-24 22:58:17 +01:00
Ivan Nardi 5e8c1ebbb7 fuzz: fix memory allocation failure logic (#1867)
We *do* want to have some allocation errors.
Fix some related bugs
Fix: 29be01ef
2023-01-20 14:27:33 +01:00
Ivan Nardi 496b284c98 Bittorrent: fix detection over TCP (#1868)
Close #1866
2023-01-19 22:02:23 +01:00
Ivan Nardi c4274a4aec Minor changes in ndpi_detection_giveup() (#1861)
Check the mining cache at the end of the function, like all the others
LRU caches.
Rewrite the STUN checks: same identical semantic but in a cleaner way,
hopefully
2023-01-19 21:45:35 +01:00
Ivan Nardi f5c2f9280c Fix undefined-behaviour in ahocorasick callback (#1864)
```
ndpi_main.c:2119:28: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
```
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=55226
2023-01-19 14:13:17 +01:00
Ivan Nardi f95bdaf625 Bittorrent: fix heap-buffer-overflow (#1863)
```
==258287==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600068ff9d at pc 0x5653a6e35def bp 0x7ffeef5aa620 sp 0x7ffeef5a9dc8
READ of size 22 at 0x60600068ff9d thread T0
    #0 0x5653a6e35dee in strncmp (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x4d2dee) (BuildId: 133b8c3c8ff99408109fcb9be2538bb8341f07f7)
    #1 0x5653a70d6624 in ndpi_search_bittorrent /home/ivan/svnrepos/nDPI/src/lib/protocols/bittorrent.c:500:71
    #2 0x5653a6ff255a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5686:6
    #3 0x5653a6ff331b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5722:10
    #4 0x5653a6ff2cbc in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5755:12
    #5 0x5653a70016bf in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6578:15
    #6 0x5653a6f1836d in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1678:31
    #7 0x5653a6f140a1 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2256:10
```
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=55218
Fix: 470eaa6f
2023-01-19 14:12:51 +01:00
Ivan Nardi edb8165ab9 Sync unit tests results (#1865) 2023-01-19 11:03:45 +01:00
Luca Deri bc44334599 Shrinked symbolic flow risks labels 2023-01-19 10:14:32 +01:00
Ivan Nardi 02f52207be Fuzz: fix compilation with nBPF (#1860)
Close #1859
2023-01-18 18:26:54 +01:00
Ivan Nardi 1b98bec0ab LRU caches: add a generic (optional and configurable) expiration logic (#1855)
Two caches already implemented a similar mechanism: make it generic.
2023-01-18 18:18:36 +01:00
Ivan Nardi de24206adc POP3: improve detection (#1856) 2023-01-18 07:19:44 +01:00
Ivan Nardi 97014c53f3 Improve support for Snapchat voip calls (#1858)
Latest Snapchat versions use QUICv1 for their audio/video real time
sessions. See c50a8d480
2023-01-17 22:02:23 +01:00
Ivan Nardi ccc5a20710 Add detection of Tailscale (#1857)
Extend the example of wireguard traffic
2023-01-17 21:55:38 +01:00
Ivan Nardi 470eaa6ff3 Add Meraki Cloud protocol and improve other protocols (#1854)
Avoid some LineCall and Jabber false positives.
Detect Discord mid flows.
Fix Bittorrent detection.
2023-01-17 11:38:39 +01:00
Ivan Nardi 29be01ef3a Add some fuzzers to test algorithms and data structures (#1852)
Fix some issues found with these new fuzzers
2023-01-17 08:31:59 +01:00
Ivan Nardi ebb9ebd2a0 Fix classification "by-port" (#1851)
Classification "by-port" should be the last possible effort, *after*
having test all the LRU caches.

Remove some dead code from `ndpi_detection_giveup()`:
`flow->guessed_protocol_id` is never set to any od those voip protocols
and at that point in this function we never have both a master *and* a
application protocols.
Coverage reports (both from unit tests and from fuzzing) confirms that
was dead code.
2023-01-17 08:26:42 +01:00
Luca Deri 1f7c57deff Improved DGA detection 2023-01-12 10:06:31 +01:00
Ivan Nardi a944514dde Reduce the size of some traces used as unit test (#1845)
No traces and no flows has been removed; only long sessions
has been reduced, keeping only their first packets.

This is quite important in fuzzing systems, since these pcaps are used
as initial seed. There is no value in fuzzing long sessions, because
only the very first packets are really used/processed by nDPI.

Before:
```
du -h tests/pcap/
200M	tests/pcap/
```
After:
```
du -h tests/pcap/
98M	tests/pcap/
```
2023-01-05 16:43:35 +01:00
Ivan Nardi 5a5e01a9e2 PPSTREAM: fix some heap-buffer overflows (#1846)
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=54802
2023-01-05 16:41:53 +01:00
Ege ÇetinandNardi Ivan 663df7e475 Fix missing bracket at ppstream (#1843)
* add missing bracket

* Sync unit test results

Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2023-01-03 20:11:24 +01:00
Ivan Nardi 9c83f0f3c5 Fix some warnings and two errors found while fuzzing (#1844)
Fix CI
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=54614
2023-01-03 13:21:54 +01:00
Luca Deri cb15d2b328 Added missing prototype 2023-01-03 12:58:52 +01:00
Luca Deri 2c551afbd7 Added NDPI_MINOR_ISSUES risk used for storing generic/relevant information about issues found on traffic. 2022-12-31 16:55:55 +01:00
Luca Deri 1735931f67 Added NDPI_PERIODIC_FLOW flow risk to be used by apps based on nDPI 2022-12-30 19:20:07 +01:00
Luca Deri 8f91b8ba72 Implemented EDNS(0) support in DNS dissector
Improved DNS dissection
2022-12-29 19:38:25 +01:00
Ivan Nardi 560280e6f0 fuzz: add fuzzer testing nDPI (initial) configurations (#1830)
The goal of this fuzzer is to test init and deinit of the library, with
different configurations. In details:
* random memory allocation failures, even during init phase
* random `ndpi_init_prefs` parameter of `ndpi_init_detection_module()`
* random LRU caches sizes
* random bitmask of enabled protocols
* random parameters of `ndpi_set_detection_preferences()`
* random initialization of opportunistic TLS
* random load/don't load of configuration files

This new fuzzer is a C++ file, because it uses `FuzzedDataProvider`
class (see
https://github.com/google/fuzzing/blob/master/docs/split-inputs.md).
Note that the (existing) fuzzers need to be linked with C++ compiler
anyway, so this new fuzzer doesn't add any new requirements.
2022-12-23 19:07:13 +01:00
Ivan Nardi 3de76812d9 Remove some old protocols (#1832)
These protocols:
* have been addeded in the OpenDPI era
* have never been updated since then
* we don't have any pcap examples [*]

If (and it is a big if...) some of these protocols are still somehow
used and if someone is still interested in them, we can probably
re-add them starting from scratch (because the current detection
rules are probably outdated)

Protocols removed: DIRECT_DOWNLOAD_LINK, APPLEJUICE, DIRECTCONNECT,
OPENFT, FASTTRACK, SHOUTCAST, THUNDER, AYIYA, STEALTHNET, FIESTA,
FLORENSIA, AIMINI, SOPCAST

PPSTREAM dissector works (...) only on UDP.

[*]: with do have an AIMINI test pcap but it was some trivial http
traffic detected only by hostname matching, on domains no more
available...
2022-12-23 10:04:08 +01:00
Ivan Nardi 5fafe8374a postgres: improve detection (#1831)
Remove some dead code (found via coverage report)
2022-12-22 21:41:32 +01:00
Ivan Nardi e9d5e72fb5 Fix infinite loop when a custom rule has port 65535 (#1833)
Close #1829
2022-12-21 10:55:14 +01:00
Ivan Nardi 22821241e2 Fix some errors found by oss-fuzz (#1834) 2022-12-20 22:50:18 +01:00
Ivan Nardi fa67166fcc Remove any references to LGTM (#1837) 2022-12-20 22:49:34 +01:00
Darryl Sokoloski b8972d1b38 Added TP-LINK Smart Home Protocol dissector. (#1841)
Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2022-12-20 20:46:06 +01:00
Ivan Nardi d8de8eba36 Sync unit tests results (#1840)
Update the documentation, hinting how to test/update *all* the unit
tests.
2022-12-19 17:57:44 +01:00
Darryl Sokoloski 9854aed408 Added TUYA LAN Protocol dissector. (#1838)
Tuya IoTOS Embedded Wi-Fi and BLE SDK for bk7231n.  Used by many "smart"
devices such as LED light strips, bulbs, etc.

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2022-12-19 17:02:19 +01:00
Ivan Nardi ad6bfbad4d Add protocol disabling feature (#1808)
The application may enable only some protocols.
Disabling a protocol means:
*) don't register/use the protocol dissector code (if any)
*) disable classification by-port for such a protocol
*) disable string matchings for domains/certificates involving this protocol
*) disable subprotocol registration (if any)

This feature can be tested with `ndpiReader -B list_of_protocols_to_disable`.

Custom protocols are always enabled.

Technically speaking, this commit doesn't introduce any API/ABI
incompatibility. However, calling `ndpi_set_protocol_detection_bitmask2()`
is now mandatory, just after having called `ndpi_init_detection_module()`.

Most of the diffs (and all the diffs in `/src/lib/protocols/`) are due to
the removing of some function parameters.

Fix the low level macro `NDPI_LOG`. This issue hasn't been detected
sooner simply because almost all the code uses only the helpers `NDPI_LOG_*`
2022-12-18 08:10:57 +00:00
Luca Deri fb0a73c0c7 Updated decription 2022-12-17 19:36:00 +01:00
Ivan Nardi 3c4b12e061 README: another try to fix build badge (#1836) 2022-12-16 12:22:07 +01:00
Ivan Nardi 390dc1d155 README: fix build badge link (#1835)
See: https://github.com/badges/shields/issues/8671
2022-12-16 12:04:16 +01:00
Luca Deri de289f1305 Added ability to decode DNS PTR records used for reverse address resolution 2022-12-15 21:22:53 +01:00
Luca Deri dbff2fbadd Minor RTP fix 2022-12-15 19:17:05 +01:00
Ivan Nardi 5704e4c142 STUN: add detection of ZOOM peer-to-peer flows (#1825)
See: "Enabling Passive Measurement of Zoom Performance in Production Networks"
https://dl.acm.org/doi/pdf/10.1145/3517745.3561414
2022-12-11 23:07:35 +01:00
Luca Deri 2edbe3c11f Centos7 fix 2022-12-11 10:24:42 +01:00
Luca Deri 88d4a25cb3 Rocky9 fix 2022-12-11 10:21:44 +01:00
Ivan Nardi 48a7f6d487 fuzz: some enhancements (#1827)
Load some custom configuration (like in the unit tests) and factorize some
(fuzzing) common code.

There is no way to pass file paths to the fuzzers as parameters. The safe
solution seems to be to load them from the process working dir. Anyway,
missing file is not a blocking error.

Remove some dead code (found looking at the coverage report)
2022-12-10 19:49:11 +01:00
Ivan Nardi 8c7071e040 DTLS: handle (certificate) fragments (#1811)
Keep using the existing function to handle reassembling buffer: rename
it from `ndpi_search_tls_tcp_memory` to
`ndpi_search_tls_memory` and make it "transport" agnostic
2022-12-10 18:32:25 +01:00
Ivan Nardi 79931b2ec7 Fix undefined-behavior when setting empty user-agent (#1821)
```
ndpi_main.c:9111:35: runtime error: null pointer passed as argument 2, which is declared to never be null
/usr/include/string.h:44:28: note: nonnull attribute specified here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ndpi_main.c:9111:35 in
```
2022-12-10 18:31:51 +01:00
Luca Deri 9dc433f709 Packaing changes 2022-12-10 13:14:33 +01:00
Ivan Nardi 9f7ef72332 fuzz: fix signed-integer-overflow (#1822)
```
fuzz_ndpi_reader.c:33:29: runtime error: signed integer overflow: 214013 * 24360337 cannot be represented in type 'int'
    #0 0x4c1cf7 in fastrand ndpi/fuzz/fuzz_ndpi_reader.c:33:29
    #1 0x4c1cf7 in malloc_wrapper ndpi/fuzz/fuzz_ndpi_reader.c:38:11
    #2 0x523057 in ndpi_malloc ndpi/src/lib/ndpi_main.c:220:25
```
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=54112
2022-12-10 09:18:36 +01:00
Ivan Nardi 6004e55d7b Build: fix make install when --prefix= is used in configure script (#1824)
Use the same logic already used in `example/Makefile.in`

Close #1823
2022-12-10 09:13:20 +01:00
Luca Deri eacc2b8e32 Added Zoom screen share detection 2022-12-09 21:32:45 +01:00
Luca Deri fc7b070030 Added RTP stream type in flow metadata 2022-12-09 14:26:53 +01:00
Luca Deri 63f349319f Improved Zoom protocol detection 2022-12-08 22:00:24 +01:00
Ivan Nardi ada4fe4aa8 fuzz: add a new fuzzer testing memory allocation failures (#1818)
Try to fuzz error paths triggered by allocation errors.
Fix some errors already found by this new fuzzer.
Basic idea taken from: https://github.com/harfbuzz/harfbuzz/pull/2566/files

`FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` is a standard define used to
(not)compile specific code in fuzzing builds.
See: https://llvm.org/docs/LibFuzzer.html
2022-12-06 17:41:58 +01:00
Ivan Nardi 946c3dba0f Fix compilation (#1819) 2022-12-05 22:33:29 +01:00
Luca Deri 57562b3dfe Updated test results 2022-12-05 21:32:45 +01:00
Luca Deri e0afc16aa2 Exported HTTP server in metadata 2022-12-05 21:27:30 +01:00
Gowa2017 c882120afd thread_index may by negative. (#1814)
* thread_index may by negative.

Like:
192.168.8.155:55848 --> 183.3.224.139

* reader thread index also need to uint32
2022-12-05 10:22:05 +01:00
Ivan Nardi 3e4ab39b52 Add support for LTO and Gold linker (#1812)
This commit add (optional) support for Link-Time-Optimization and Gold
linker.
This is the first, mandatory step needed to make nDPI compliant with
"introspector" sanitizer requirements in OSS-Fuzz: see
https://github.com/google/oss-fuzz/issues/8939

Gold linker is not supported by Windows and by macOS, so this feature is
disabled by default. It has been enable in CI in two linux targets
("latest" gcc and clang).

Fix some warnings triggered by LTO.

The changes in `src/lib/ndpi_serializer.c` seams reasonable.
However, the change in `tests/unit/unit.c` is due to the following
warning, which seems to be a false positive.

```
unit.c: In function ‘serializerUnitTest’:
ndpi_serializer.c:2258:13: error: ‘MEM[(struct ndpi_private_serializer *)&deserializer].buffer.size’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
unit.c:67:31: note: ‘MEM[(struct ndpi_private_serializer *)&deserializer].buffer.size’ was declared here
   67 |   ndpi_serializer serializer, deserializer;
      |                               ^
ndpi_serializer.c:2605:10: error: ‘MEM[(struct ndpi_private_serializer *)&deserializer].status.buffer.size_used’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
unit.c:67:31: note: ‘MEM[(struct ndpi_private_serializer *)&deserializer].status.buffer.size_used’ was declared here
   67 |   ndpi_serializer serializer, deserializer;
```
Since this warning is triggered only with an old version of gcc and
`tests/unit/unit.c` is used only during the tests, the easiest fix has
been applied.

Some (unknown to me) combinations of OS and compiler trigger the
following warnings at linker time (with sanitizer and gold linker)
```
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load1_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load2_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load4_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load8_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load16_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store1_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store2_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store4_asm'
[..]
```
I have not found any references to this kind of message, with the only
exception of https://sourceware.org/bugzilla/show_bug.cgi?id=25975
which seems to suggest that these messages can be safely ignored.
In any case, the compilation results are sound.

Fix `clean` target in the Makefile in the `example` directory.

In OSS-Fuzz enviroments, `fuzz_ndpi_reader` reports a strange link error
(as always, when the gold linker is involved...).
It's come out that the culprit was the `tempnam` function: the code has
been changed to use `tmpfile` instead. No sure why... :(

Fuzzing target `fuzz_ndpi_reader.c` doesn't use `libndpiReader.a`
anymore: this way we can use `--with-only-libndpi` flag on Oss-Fuzz builds
as workaround for the "missing dependencies errors" described in
https://github.com/google/oss-fuzz/issues/8939
2022-12-05 10:21:42 +01:00
Ivan Nardi b9f63458e6 CI: GitHub is updating Ubuntu runners (#1817)
GitHub is moving `ubuntu-latest` to `ubuntu-22.04`: update our
dependencies.
See: https://github.blog/changelog/2022-11-09-github-actions-ubuntu-latest-workflows-will-use-ubuntu-22-04/

This is the reason of the recent random failures in CI.

Update "newest" tested gcc to gcc-12.

Fix a memory error introduced in 557bbcfc5a
2022-12-03 18:29:44 +01:00
Ivan Nardi 81e1ea545c Make LRU caches ipv6 aware (#1810)
Simplest solution, keeping the existing cache data structure

TLS certificate cache is used for DTLS traffic, too.

Note that Ookla cache already works with ipv6 flows.

TODO:
 * make the key/hashing more robust (extending the key size?)
 * update bittorrent cache too. That task is quite difficult because
   ntopng uses a public function (`ndpi_guess_undetected_protocol()`)
   intrinsically ipv4 only...
2022-12-03 12:07:32 +01:00
Luca Deri a387072872 Updated tests results 2022-12-03 11:33:04 +01:00
Luca Deri 2a8d0d1fe4 Added check for empty/missing user-agent in HTTP 2022-12-03 11:32:39 +01:00
Luca Deri 557bbcfc5a Improved HTTP header parsing skipping lines that won't match 2022-12-03 11:32:29 +01:00
Ivan Nardi 197a9a6cfc Extend CodeQL configuration and fix two warnings (#1816)
```
Call to `memset` may be deleted
Comparison of narrow type with wide type in loop condition
```

See https://www.usenix.org/system/files/conference/usenixsecurity17/sec17-yang.pdf
for the new portable function `secure_memzero`.
2022-12-02 23:26:22 +01:00
Alvaro Muñoz e49fe32f77 Add CodeQL workflow (#1815) 2022-12-02 13:06:54 +01:00
Ivan Nardi b2a1bf413b Remove some useless fuzzing target (#1813)
We don't need specific targets to reproduce fuzzing issues.
After all, calling `./fuzz/fuzz_process_packet_with_main $ARTIFACT_FILE`
is equivalento to `./fuzz/fuzz_process_packet $ARTIFACT_FILE`
2022-12-01 14:39:44 +01:00
Ivan Nardi 02e7e3c23b Add a new fuzzer for QUIC (#1800)
QUIC packets are encrypted/obfuscated; that means that we need to
decrypt them before parsing the real (TLS) message.
Fuzzing is not effective here, since a random buffer is hardly a valid
encrypted QUIC packet.

Add a new fuzzer, testing *decrypted* QUIC packets.

Add a basic corpus.

Fix a few bugs already found by this fuzzer.
2022-12-01 12:21:04 +01:00
Ivan Nardi afb5c3e6ba Fix classification "by-port" of QUIC flows (#1785)
nDPI is able to properly classify QUIC flows only if it elaborates the
very first packets of the flow.

The protocol list in `is_udp_guessable_protocol()` is basically a list
of protocols which can be detected from *any* packets in the flow.
Rename such function to `is_udp_not_guessable_protocol()`: the name is
still quite cryptic, but at least not plainly wrong
2022-11-22 18:00:02 +01:00
Ivan Nardi 6e6f1e4489 GitHub Actions: fix a warning (#1807)
```
The `set-output` command is deprecated and will be disabled soon.
Please upgrade to using Environment Files. For more information see:
https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
```
2022-11-22 16:44:08 +01:00
Ivan Nardi 2b65a4e05e TLS: be sure to always set ssl_version field (#1806)
Useful with asymmetric traffic with (D)TLS <= 1.2
2022-11-22 16:13:28 +01:00
Luca Deri 4231f48059 Added support for Linux Cooked Capture v2 2022-11-16 17:48:28 +01:00
Ivan Nardi 388d01a345 DNS: fix an infinite loop, due to an integer overflow (#1804)
Found by sydr-fuzz
Close #1803
2022-11-15 16:29:10 +01:00
Toni b5e8d86def Improved Teamspeak3 License/Weblist detection. (#1802)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-11-15 08:59:45 +01:00
Ivan Nardi cd41ab7c8f Improve export/print of L4 protocol information (#1799)
Close #1797
2022-11-13 22:35:46 +01:00
Ivan Nardi 07872b3527 GitHub Actions: update to Node.js 16 actions (#1801)
Fix warnings on recent CI results; example:
https://github.com/ntop/nDPI/actions/runs/3455588082

See: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/
2022-11-13 19:42:13 +01:00
Toni 6d3446b114 Merge pull request #1796 from IvanNardi/issue-1792
RTP: remove a superfluous check
2022-11-07 20:38:40 +01:00
Nardi Ivan 52b562c328 Fix json export of ipv6 addresses
The "string" buffer was to short; better start using `INET6_ADDRSTRLEN`
as reported in the man page of `inet_ntop`.

Close: #1794
2022-11-07 20:36:55 +01:00
Nardi Ivan aa5dd25392 RTP: remove a superfluous check
Static analyzer complains about dereferencing `packet->udp` before
checking.
Since this function is called only with UDP flows, remove the check.

Close: #1792
2022-11-07 18:47:19 +01:00
Zied Aouini 9e73d61fb0 Fix supported versions. 2022-10-31 13:53:23 +01:00
Zied Aouini 41a5ba1b65 Add a note about required Python version. 2022-10-31 13:50:59 +01:00
Ivan Nardi db9f6ec1b4 Add basic profiling of memory allocations on data-path (#1789)
The goal is to have an idea of the memory allocation sizes performed in
the **library data-path**, i.e. excluding init/deinit phases and all
the allocations made by the application itself.
In other words, how much memory is needed per-flow, by nDPI, other than
`struct ndpi_flow_struct`?

It works only on single-thread configurations.

It is not enabled by default (in the unit tests) since different
canfiguration options (example: `--enable-pcre`) lead to diffferent
results.

See: #1781
2022-10-28 20:41:37 +02:00
Luca Deri 175981a422 Removed unused variable 2022-10-26 16:14:32 +02:00
Alfredo Cardigliano f30479b4a4 Fix typo 2022-10-26 16:05:32 +02:00
Luca Deri a16cd9634f Cleanup 2022-10-26 13:59:07 +02:00
Luca Deri 15d58db2f9 Typo 2022-10-26 13:56:24 +02:00
Luca Deri 55f885f3ed Improved AESNI check 2022-10-26 13:51:00 +02:00
Ivan Nardi de16fd35aa Hangout: avoid useless lookups in the protocol tree (#1755)
We already performed exactly these lookups in the generic code to
populate `flow->guessed_protocol_id_by_ip`: use it!

This code probably needs a deeper review, since it is basicaly a simple
matching on ip + port.
2022-10-25 18:24:03 +02:00
Ivan Nardi ca5ffc4988 TLS: improve handling of ALPN(s) (#1784)
Tell "Advertised" ALPN list from "Negotiated" ALPN; the former is
extracted from the CH, the latter from the SH.

Add some entries to the known ALPN list.

Fix printing of "TLS Supported Versions" field.
2022-10-25 17:06:29 +02:00
Ivan Nardi 2ed2e5dc7e Fix memory corruption on TCP flows (#1786) 2022-10-25 16:22:40 +02:00
Ivan Nardi 550f3e5b9d IRC: fix memory access (#1787) 2022-10-25 16:16:24 +02:00
Ivan Nardi e674a8ae37 Sync unit tests results (#1783) 2022-10-22 12:57:06 +02:00
Luca Deri 24cc949f14 Enhanced HTTP numeric IP check 2022-10-22 10:06:09 +02:00
Toni c521595383 Add Elasticsearch protocol dissector. (#1782)
* all credits goes to @verzulli

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-21 20:01:54 +02:00
Toni Uhlig 29242cbcb6 Add Munin protocol dissector.
* all credits goes to @verzulli

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-20 22:10:35 +02:00
Nardi Ivan 9c0caa5362 TLS: allow sub-classification via ALPN
In some rare cases, it is possible to sub-classify the flow via ALPN
matching. This is particularly usefull for asymmetric traffic where the
Client Hello doens't have the SNI.

For the time being there is only one rule, about ANYDESK.
2022-10-20 22:00:12 +02:00
Nardi Ivan c1b6aa9483 Sync unit tests results 2022-10-19 13:01:34 +02:00
Luca Deri 9a34d870e9 Updated FortiClient default ports 2022-10-19 11:25:19 +02:00
Luca Deri df0dd76ed2 Reworked severities adding critical and emergency, thus shifting values down 2022-10-19 10:53:11 +02:00
Ivan Nardi 35fdbc8148 TLS: explicit ignore client certificate (#1776)
TLS classification usually stops after processing *server* certificates
(if any). That means, that *client* certificate, if present, is usually
ignored.
However in some corner cases (i.e. unidirectional traffic) we might end
up processing client certificate and exposing its metadata: the issue is
that the application will think that this metadata are about the server
and not about the client.
So, for the time being, always ignore client certificate processing.

As a future work, we might find an efficient way to process and export both
certificates.
2022-10-18 16:40:15 +02:00
Nardi Ivan 223a6fb9f7 Fix a use-of-uninitialized-value error on PCRE code
This is likely a false positive, triggered by the fact that libpcre is
usually compiled without MASAN support.
It it was a real error, ASAN would complain loudly with a invalid-free
error at the end of the same function.

```
==83793==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x562296111174 in ndpi_compile_rce_regex /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1631:3
    #1 0x5622960e3e4a in ndpi_is_rce_injection /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1636:5
    #2 0x5622960de7cd in ndpi_validate_url /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1741:12
    #3 0x5622960dae45 in ndpi_dpi2json /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1362:29
    #4 0x5622960e2751 in ndpi_flow2json /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1512:10
    #5 0x562296033b0f in process_ndpi_collected_info /home/ivan/svnrepos/nDPI/example/reader_util.c:1310:9
    #6 0x5622960501f9 in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1659:2
    #7 0x562296045aef in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2202:10
    #8 0x562295e85374 in ndpi_process_packet /home/ivan/svnrepos/nDPI/example/ndpiReader.c:3937:7
    #9 0x7f1235053466  (/lib/x86_64-linux-gnu/libpcap.so.0.8+0x23466) (BuildId: b84c893ea2516d6fb2c1c6726b1fe93b3be78f61)
    #10 0x7f1235041f67 in pcap_loop (/lib/x86_64-linux-gnu/libpcap.so.0.8+0x11f67) (BuildId: b84c893ea2516d6fb2c1c6726b1fe93b3be78f61)
    #11 0x562295e53139 in runPcapLoop /home/ivan/svnrepos/nDPI/example/ndpiReader.c:4060:15
    #12 0x562295e51e7f in processing_thread /home/ivan/svnrepos/nDPI/example/ndpiReader.c:4130:3
    #13 0x7f1234e53608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
    #14 0x7f1234d2f132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

  Uninitialized value was created by an allocation of 'pcreErrorStr' in the stack frame
    #0 0x5622961108a6 in ndpi_compile_rce_regex /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1603:3
```
2022-10-17 23:15:07 +02:00
Nardi Ivan 3d0c36cf22 Add some popular streaming applications
A simply back-porting from https://github.com/vel21ripn/nDPI/pull/142;
full credits to @ChrisNelson-CyberReef.

Close #1716
2022-10-17 23:08:25 +02:00
Toni 01622c4c63 Updated protocol dissector serialisation code. (#1767)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-15 12:58:44 +02:00
Ivan Nardi 2d153fba45 HTTP: fix stack-buffer-overflow (#1768)
```
==24879==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fa085b31e60 at pc 0x55cc63f203e2 bp 0x7ffc9ec91b10 sp 0x7ffc9ec91298
READ of size 17 at 0x7fa085b31e60 thread T0
    #0 0x55cc63f203e1 in printf_common(void*, char const*, __va_list_tag*) asan_interceptors.cpp.o
    #1 0x55cc63f20769 in vsnprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x50e769) (BuildId: cce2b6b1344bfd0bdc9626fef604c2b3caad485b)
    #2 0x55cc63f22210 in __interceptor_snprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x510210) (BuildId: cce2b6b1344bfd0bdc9626fef604c2b3caad485b)
    #3 0x55cc6420fc76 in ndpi_check_http_server /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:668:4
    #4 0x55cc6420344b in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:742:5
    #5 0x55cc642031ce in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:737:7
    #6 0x55cc641fac9f in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1352:4
    #7 0x55cc641f2fd5 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1461:3
    #8 0x55cc64085275 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5580:6
    #9 0x55cc64085c87 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5628:12
    #10 0x55cc64085927 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5647:12
    #11 0x55cc64095fcb in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6458:15
    #12 0x55cc63fd08b4 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #13 0x55cc63fd09f7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #14 0x7fa0880fb082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #15 0x55cc63efb45d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4e945d) (BuildId: cce2b6b1344bfd0bdc9626fef604c2b3caad485b)

Address 0x7fa085b31e60 is located in stack of thread T0 at offset 96 in frame
    #0 0x55cc6420f1bf in ndpi_check_http_server /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:644

  This frame has 5 object(s):
    [32, 36) 'a' (line 653)
    [48, 52) 'b' (line 653)
    [64, 68) 'c' (line 653)
    [80, 96) 'buf' (line 654)
    [112, 176) 'msg' (line 662) <== Memory access at offset 96 partially underflows this variable

```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=52229
2022-10-14 18:59:33 +00:00
Ivan Nardi 6c84ce85e4 ndpiReader: fix help message. There isn't a 'J' option (#1770) 2022-10-14 20:16:47 +02:00
Ivan Nardi 8c4ece941e Sync utests results (#1774) 2022-10-14 20:16:29 +02:00
Luca Deri 3bb6bf628d Removed gettimeofday as win files has located in windows/src/ 2022-10-14 19:01:41 +02:00
Luca Deri 1fecf69359 Added DNS error code mapping number -> string 2022-10-14 09:39:15 +02:00
Luca Deri cd76bacd4a Added detection of URLs starting with a dot (.) 2022-10-10 22:34:26 +02:00
Ivan Nardi 1796a1d814 LINE_CALL: add detection of LINE voip calls (#1761)
These flows are classifed as `LINE_CALL`; another option was
`RTP/LINE_CALL`. No sure about the best solution...

Extend LINE domains list.

Remove RTP dead code.
2022-10-06 17:09:26 +02:00
Luca Deri e4dcec560e Windows changes 2022-10-06 11:58:16 +02:00
Ivan Nardi c4fb2a590e irc: fix heap-buffer-overflow (#1765) 2022-10-06 09:25:21 +00:00
Luca Deri 3c0021d603 Fixes Stack-buffer-overflow READ {*} · check_content_type_and_change_protocol 2022-10-06 09:16:02 +02:00
Luca Deri fba926d6a4 Updated IRC result 2022-10-05 23:02:02 +02:00
Luca Deri 199c86e3df Improved credentials check in IRC 2022-10-05 22:51:20 +02:00
Luca 37c88b129f Added new flow risk NDPI_HTTP_OBSOLETE_SERVER. Currently Apache and nginx are supported 2022-10-04 22:32:45 +02:00
Nardi Ivan cca585053e Fix compilation and sync utests results 2022-10-04 22:17:05 +02:00
Luca 131d53896f Improved HTTP payload handling 2022-10-04 14:03:39 +02:00
Luca 3fa9416981 Improved detection of WebShell and PHP code in HTTP URLs that is reported via flow risk 2022-10-04 11:26:44 +02:00
Luca de59eb8237 Added the ability to track the payload via -E and via the new option 'ndpi_track_flow_payload' 2022-10-04 11:26:44 +02:00
Toni 66a8010ef7 Fix ndpi_timeval_to_milliseconds/microseconds for platforms with tv_usec is an unsigned int. (#1762)
* implicit conversion to an 32 bit unsigned int leads to an overflow

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-02 12:48:24 +02:00
Toni c83e0b3c8d Restore Confidence enum constants to ascending values. (#1760)
* NDPI_CONFIDENCE_MAX is useless otherwise since it is
   supposed the be the highest value w/o any gaps
 * for client applications, it makes sense to use that
   the numbers directly w/o any additional mapping

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-02 12:47:56 +02:00
sharonenochandSharon Enoch 503aac70bc Line app support (#1759)
* Standard support for LINE app

* Added test pcap for LINE app

* make check result for LINE app

* Make check success as 1kxun has LINE packets

* Added the ASN inc file for LINE

* Removed extra lines as its effecting make check

* Editing the SNI required a new pcap output file for TLS.Line format

* Run Configure with --with-pcre --with-maxminddb to enable the generation of h323-overflow.pcap.out

Co-authored-by: Sharon Enoch <sharone@amzetta.com>
2022-10-01 12:01:41 +02:00
Nardi Ivan f3a74d97d8 TLS/DTLS: we process certificate for UDP flows, too
Note that current code access `certificate_processed` state even before
setting the protocol classification, so this piece of information can't
be saved in `flow->protos` union.
2022-09-30 06:53:29 +02:00
Luca fbf30d1aad Fixed invalid unidirectional traffic alert for unidirectional protocols (e.g. sFlow) 2022-09-29 18:15:37 +02:00
Nardi Ivan a27538b764 Remove unused code
LRU callbacks have been added in 460ff3c7a, but they have never been
used and they have never been extended to the other LRU caches.

`ndpi_search_tcp_or_udp()` basically returns the classification by port/ip
of the flow; calling it from the dissector is useless.
The same for TOR detection: ips are checked in the generic code
2022-09-28 21:09:27 +02:00
Nardi Ivan ac7a3669da Fix value of ndpi_protocol->protocol_by_ip
Fix: a7c2734b
2022-09-25 13:31:57 +02:00
Nardi Ivan 39f9b8cadf DNS: change category of DNS flows
DNS flows should have `NDPI_PROTOCOL_CATEGORY_NETWORK` as category,
regardless of the subprotocol (if any).
Follow-up of 83de3e47
2022-09-25 13:31:04 +02:00
Toni Uhlig b7b9577254 Changed some protocol categories to align with others.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-23 20:35:55 +02:00
Nardi Ivan 1f345b311f Sizes of LRU caches are now configurable
0 as size value disable the cache.

The diffs in unit tests are due to the fact that some lookups are
performed before the first insert: before this change these lookups
weren't counted because the cache was not yet initialized, now they are.
2022-09-23 18:33:48 +02:00
Nardi Ivan 83de3e4716 DNS: change category of DNS flows
DNS flows should have `NDPI_PROTOCOL_CATEGORY_NETWORK` as category,
regardless of the subprotocol (if any).
2022-09-22 13:23:55 +02:00
Nardi Ivan b250f9d86c Extend content match lists 2022-09-22 09:30:09 +02:00
Nardi Ivan c6201284d7 NAT-PMP: fix metadata extraction 2022-09-21 20:02:23 +02:00
Toni 644ad34962 Improved NATPMP dissection. (#1745)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:24:04 +02:00
Toni Uhlig d6701e8979 Build ndpiReader and run regression tests.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:03:22 +02:00
segfault 3db6413ab0 Fixed VS2019 build.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:03:22 +02:00
Toni Uhlig 50ee6768b7 Provide extra ndpi_define.h for Visual Studio.
* Caution: file needs to keep in sync with `src/include/ndpi_define.h.in` manually

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:03:22 +02:00
Toni Uhlig d27fb1e73e Moved NDPI_API_VERSION to src/include/ndpi_define.h.in where it belongs.
That way, we can make `src/include/ndpi_api.h` independent of autoconf.
MSBuild will like that.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:03:22 +02:00
segfault 817a7c616d Retargeted to Windows 10.
* disable Spectre Mitigation (may change)
 * remove autoconf generated header files
2022-09-21 18:03:22 +02:00
mark andrews 5d5b46e514 Add proj and sln for compile of dynamic x64 lib under Visual Studio 2019.
* add CI support via MSBuild

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:03:22 +02:00
Ivan Nardi 03d217eae6 Fix CI after nBPF integration (#1746)
Add one CI job testing nBPF
2022-09-21 16:49:51 +02:00
Luca Deri c960bb50f0 Copilation fix 2022-09-21 10:27:46 +02:00
Luca Deri aef8e53b7d Added missing library 2022-09-21 08:24:37 +02:00
Luca Deri 1d0bc7c350 Finalized nBPF support. You can now define custom protocols such as
(see exaple/protos.txt)

nbpf:"host 192.168.1.1 and port 80"@HomeRouter

In order to have nBPF support, you need to compile nDPI with it. Just download
https://github.com/ntop/PF_RING in the same directory where you have downloaded
nDPI and compile PF_RING/userland/nbpf
2022-09-21 00:03:12 +02:00
Ivan Nardi a7c2734b38 Remove classification "by-ip" from protocol stack (#1743)
Basically:
* "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is
NEVER returned in the protocol stack (i.e.
`flow->detected_protocol_stack[]`);
* if the application is interested into such information, it can access
`ndpi_protocol->protocol_by_ip` itself.

There are mainly 4 points in the code that set the "classification
by-ip" in the protocol stack:  the generic `ndpi_set_detected_protocol()`/
`ndpi_detection_giveup()` functions and the HTTP/STUN  dissectors.

In the unit tests output, a print about `ndpi_protocol->protocol_by_ip`
has been added for each flow: the huge diff of this commit is mainly due
to that.

Strictly speaking, this change is NOT an API/ABI breakage, but there are
important differences in the classification results. For examples:
* TLS flows without the initial handshake (or without a matching
SNI/certificate) are simply classified as `TLS`;
* similar for HTTP or QUIC flows;
* DNS flows without a matching request domain are simply classified as
`DNS`; we don't have `DNS/Google` anymore just because the server is
8.8.8.8 (that was an outrageous behaviour...);
* flows previusoly classified only "by-ip" are now classified as
`NDPI_PROTOCOL_UNKNOWN`.

See #1425 for other examples of why adding the "classification by-ip" in
the protocol stack is a bad idea.

Please, note that IPV6 is not supported :(  (long standing issue in nDPI) i.e.
`ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for
IPv6 flows.

Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed.

Close #1687
2022-09-20 22:24:47 +02:00
Toni 174cd739db Fix warnings on OpenWrt CCs, caused by the memcpy builtin. (#1741)
In file included from ../include/ndpi_includes.h:31,
                 from ../include/ndpi_main.h:27,
                 from ../include/ndpi_api.h:28,
                 from protocols/quic.c:27:
In function 'memcpy',
    inlined from 'tls13_hkdf_expand_label_context' at protocols/quic.c:473:5,
    inlined from 'tls13_hkdf_expand_label' at protocols/quic.c:498:10,
    inlined from 'quic_hkdf_expand_label.constprop' at protocols/quic.c:512:6:
/home/build/openwrt/staging_dir/toolchain-mips_24kc_gcc-11.3.0_musl/include/fortify/string.h:53:16: error: argument 2 null where non-null expected [-Werror=nonnull]
   53 |         return __builtin_memcpy(__od, __os, __n);
      |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
protocols/quic.c: In function 'quic_hkdf_expand_label.constprop':
/home/build/openwrt/staging_dir/toolchain-mips_24kc_gcc-11.3.0_musl/include/fortify/string.h:53:16: note: in a call to built-in function '__builtin_memcpy'

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-18 12:40:49 +02:00
Luca Deri 48d00eae35 Added MAX_NBPF_CUSTOM_PROTO define 2022-09-17 11:04:40 +02:00
Luca 0f07ad35be Initial work towards custom BPF protocol definition 2022-09-16 19:48:12 +02:00
Toni Uhlig a966d37a21 Add CryNetwork dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-16 16:48:43 +02:00
Toni Uhlig ec1981c20c Add Syncthing dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-16 15:21:49 +02:00
Toni de020b1747 Add NATPMP dissector. (#1738)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-16 13:32:30 +02:00
Nardi Ivan 9ce4d40d14 Remove a case of guessed sub-classification
This code is triggered only for "unknown" flows with a valid
sni/hostname.
Why in that case the guessed classification should be
something like `DNS/Subprotocol_depending_on_hostname`? Why DNS as
master and not HTTP or TLS or QUIC?

Furthermore, I have not been able to trigger a positive match from that
lookup. I strongly think that if we had a valid subprotocol, we would
have a valid master in the first place.
In doubt, remove it completely.

As a follow up, we should investigate why some dissectors (the HTTP one,
at least) set the sni/hostname field without setting a valid protocol,
in the first place.
This behaviour seems quite suspicious, if not plainly buggy.
2022-09-14 17:52:01 +02:00
Nardi Ivan 7571f48392 Dissectors shouldn't update flow->guessed_host_protocol_id
The field `flow->guessed_host_protocol_id` is set at the beginning of
the flow analysis and it represents the "classification by ip" of the flow
itself.

This field should never be changed. Dissectors which want to provide an
"hint" about the classification, should update `flow->guessed_protocol_id`
instead. Such "hint" is useless if the dissector set the "extra-dissection"
data-path.

Rename such field to `guessed_protocol_id_by_ip` to better describe its
role.

Preliminary work necessary for #1687
2022-09-14 17:51:39 +02:00
Ivan Nardi b4f5243716 Anydesk: improve detection (#1735) 2022-09-13 20:55:11 +02:00
Nardi Ivan eea3c448de HTTP: remove some wrong code
This code seems wrong or in the wrong place, at least:
* "classification by port" and "classification by ip" protocols (i.e
"guessed" protocols) should be used to set the protocol stack only after
trying all the dissectors, and only by the generic code
* there are no reason (for a dissector) to update the "guessed"
information using the protocol stack values: it is usually the other way
around (see previous point)
2022-09-13 20:48:20 +02:00
Alfredo Cardigliano 973950d881 Replace obsolete linux macro 2022-09-13 10:41:44 +02:00
Ivan Nardi 4f584f78a0 Fix ndpi_do_guess() (#1731)
Avoid a double call of `ndpi_guess_host_protocol_id()`.
Some code paths work for ipv4/6 both
Remove some never used code.
2022-09-12 19:28:41 +02:00
Ivan Nardi 4bb81f7609 Update Github CI OS and compilers (#1733)
ubuntu-18.04 is deprecated (ubuntu-latest points to 20.04).
macos-latest points to macos-11, so it makes sense to test macos-12,
too.

About the compilers, the general idea it to test the oldest and the
newest versions easily available: switch to gcc-11 and clang-14.

See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
2022-09-12 19:11:43 +02:00
Nardi Ivan 678dd61866 STUN: several improvements
Add detection over TCP and fix detection over IPv6.
Rename some variables since Stun dissector is no more "udp-centric".
Stun dissector should always classified the flow as `STUN` or
`STUN/Something`.
Don't touch `flow->guessed_host_protocol_id` field, which should be
always be related to "ip-classification" only.
2022-09-11 13:33:32 +02:00
Nardi Ivan f44413e039 Update Changelog file (version 4.4)
Notes shameless taken from https://github.com/ntop/nDPI/releases
2022-09-10 22:51:40 +02:00
Nardi Ivan 1b3de449a1 HTTP: correctly set the classification protocols
Classification should always be set via `ndpi_set_detected_protocol()`
to be sure to set a correct `confidence` value, too.
Having a "known" protocol stack with `NDPI_CONFIDENCE_UNKNOWN` as
confidence, is not valid.

This code in HTTP dissector likely needs some more thoughts (the
classification itself of the attached example doesn't make a lot of
sense), but the goal of this commit is only to always have a valid
`confidence` value.
2022-09-06 14:58:31 +02:00
Toni 37f918322c Serialize nDPI detection/dissection data as object. (#1725)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-06 11:38:31 +02:00
Nardi Ivan e5063f9883 Fix some warnings
Ubuntu-20.04, clang-16 (nightly build)

```
Making all in src/lib
protocols/smpp.c:70:17: warning: variable 'pdu_c' set but not used [-Wunused-but-set-variable]
      u_int16_t pdu_c = 1;
                ^
1 warning generated.
third_party/src/ahocorasick.c:173:20: warning: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Wsingle-bit-bitfield-constant-conversion]
  thiz->root->root = 1;
                   ^ ~
third_party/src/ahocorasick.c:336:15: warning: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Wsingle-bit-bitfield-constant-conversion]
        n->ff = 1;
              ^ ~
third_party/src/ahocorasick.c:716:21: warning: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Wsingle-bit-bitfield-constant-conversion]
        node->final = 1;
[...]
```
2022-09-05 23:44:51 +02:00
Ivan Nardi 0a47f745cc Avoid useless host automa lookup (#1724)
The host automa is used for two tasks:
* protocol sub-classification (obviously);
* DGA evaluation: the idea is that if a domain is present in this
automa, it can't be a DGA, regardless of its format/name.

In most dissectors both checks are executed, i.e. the code is something
like:

```
ndpi_match_host_subprotocol(..., flow->host_server_name, ...);
ndpi_check_dga_name(..., flow->host_server_name,...);

```

In that common case, we can perform only one automa lookup: if we check the
sub-classification before the DGA, we can avoid the second lookup in
the DGA function itself.
2022-09-05 13:59:51 +02:00
Toni 01028ee779 Fix string copy truncation warning. (#1723)
protocols/ubntac2.c: In function ‘ndpi_search_ubntac2’:
protocols/ubntac2.c:69:4: warning: ‘strncpy’ output may be truncated copying between 0 and 31 bytes from a string of length 255 [-Wstringop-truncation]
   69 |    strncpy(flow->protos.ubntac2.version, (const char *)version, len);
      |    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-05 11:16:06 +02:00
Toni 378b0c5953 Fixed FastCGI memory issue (was not using nDPI's malloc wrapper). (#1722)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-05 10:46:02 +02:00
Nardi Ivan 95c1daa851 Fix endianess in IP-Port lookup
Port parameter in `ndpi_network_port_ptree_match()` must be in network
order.
2022-09-05 10:17:58 +02:00
Toni b7c1030804 Fix broken ndpi_timeval_to_(milli|micro)seconds (>UINT_MAX). (#1720)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-04 19:19:05 +02:00
Nardi Ivan b9cb391756 Add support to opportunistic TLS
A lot of protocols provide the feature to upgrade their plain text
connections to an encrypted one, via some kind of "STARTTLS" command.

Add generic code to support this extension, and allow dissection of the
entire TLS handshake.

As examples, SMTP, POP, IMAP and FTP dissectors have been updated.

Since this feature requires to process more packets per flow, add the
possibility to disable it.

Fix some log messages.

Slight improvement on TCP sequence number tracking.

As a side effect, this commit fix also a memory leak found by
oss-fuzzer
```
==108966==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 22 byte(s) in 1 object(s) allocated from:
    #0 0x55f8b367a0be in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x5480be) (BuildId: 94debacb4a6784c30420ab748c8bf3cc59621063)
    #1 0x55f8b36e1345 in ndpi_malloc_wrapper /home/ivan/svnrepos/nDPI/example/reader_util.c:321:10
    #2 0x55f8b379c7d2 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:212:25
    #3 0x55f8b379cb18 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:279:13
    #4 0x55f8b386ce46 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2153:34
    #5 0x55f8b385ebf7 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:867:5
    #6 0x55f8b39e708c in ndpi_extra_search_mail_smtp_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/mail_smtp.c:422:9
    #7 0x55f8b37e636c in ndpi_process_extra_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5884:9
    #8 0x55f8b37edc05 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6276:5
    #9 0x55f8b3701ffc in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1619:31
    #10 0x55f8b36faf14 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2189:10
    #11 0x55f8b36b6a50 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7

```
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50765
2022-09-04 17:22:19 +02:00
Luca Deri 7578d02de9 Fixes on hw calculation 2022-09-01 22:06:57 +02:00
Alfredo Cardigliano bd29fa46ba Serializer supports double for JSON and CSV only for the time being (TLV needs to be extended) 2022-09-01 14:43:52 +02:00
Alfredo Cardigliano 2372063d25 Temporarily restore ndpi_serialization_unknown = 0 2022-09-01 12:18:04 +02:00
Alfredo Cardigliano 8f410b751f Restore ndpi_serialization_type item values. Use 0 for the new ndpi_serialization_double as there is no more key left (4 bit). Use 16 for ndpi_serialization_unknown which is not used in TLV encoded data. 2022-09-01 10:37:39 +02:00
Luca Deri f684d44aa7 Removed maxmind check 2022-08-31 11:48:18 +02:00
Alfredo Cardigliano 981a72cd80 Improve AES-NI check 2022-08-30 16:37:30 +02:00
Toni f7907976c9 FastCGI: dissect PARAMS (#1715)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
2022-08-30 11:38:36 +02:00
Toni fe28d55801 Improved MGCP dissector. (#1717)
* typ0s fixed
 * dissect endpoint hostnames

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-30 11:21:58 +02:00
Nardi Ivan 79653f99e0 Internal gcrypt: fix memory leak
Fix: ef99eb67
2022-08-30 11:12:00 +02:00
Luca Deri ef99eb674e Improved AES-NI presence check on Linux 2022-08-29 17:42:28 +02:00
Nardi Ivan a329730d24 QUIC: fix heap-buffer-overflow
```
==12318==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x604000000032 at pc 0x55a59ec97959 bp 0x7fffee67fdd0 sp 0x7fffee67fdc8
READ of size 1 at 0x604000000032 thread T0
    #0 0x55a59ec97958 in may_be_0rtt /home/ivan/svnrepos/nDPI/src/lib/protocols/quic.c:1483:24
    #1 0x55a59ec9515f in ndpi_search_quic /home/ivan/svnrepos/nDPI/src/lib/protocols/quic.c:1708:13
    #2 0x55a59ec32e95 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5428:6
    #3 0x55a59ec33c5b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5464:10
    #4 0x55a59ec335fc in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5497:12
    #5 0x55a59ec44615 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6322:15
    #6 0x55a59eb8884e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #7 0x55a59eb889c7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #8 0x7fb5b3ba2082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #9 0x55a59eac742d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x47d42d) (BuildId: 712c87b21cf5c05f64174745909c693d3ba0b62e)

0x604000000032 is located 0 bytes to the right of 34-byte region [0x604000000010,0x604000000032)
allocated by thread T0 here:
    #0 0x55a59eb4bfee in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x501fee) (BuildId: 712c87b21cf5c05f64174745909c693d3ba0b62e)
    #1 0x55a59eb8899c in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:87:17

```

Found by CI tests.
See: https://github.com/ntop/nDPI/runs/7996151458?check_suite_focus=true
2022-08-25 06:44:38 +02:00
Nardi Ivan 8bfb1712d8 QUIC: add support for 0-RTT packets received before the Initial
RFC9001 4.6.1: "A client that wishes to send 0-RTT packets uses the
early_data extension in the ClientHello message of a subsequent handshake;
see Section 4.2.10 of [TLS13]. It then sends application data in 0-RTT
packets."
That means the client sends before the CH (in the Initial) and then the
0-RTT (in the same UDP datagram or not)".

However, because of packet loss or out-of-order delivery, it might
happens that a 0-RTT packet is received before the Initial (the original
one or a retransmission).
For example, Google and Facebook servers save 0-RTT packets for a small
amount of time in hopes of receiving the corresponding Initial.

Update the QUIC dissector to detect 0-RTT packets and keep looking for
the Initial.

Issue found by @utoni in #1706; the trace example has been taken from that
PR.
2022-08-24 15:38:30 +02:00
Toni 0c8bc9f055 Add FastCGI protocol detection. (#1711)
* CQL: fixed byte order conversion (BigEndian not LittleEndian)
 * CQL: increased required successful dissected packets to prevent false-positives

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 13:22:46 +02:00
Toni e135c1c5e3 Merge pull request #1712 from IvanNardi/oss-fuzzer
HTTP, SoftEther, Florensia: fix some memory corruptions
2022-08-24 11:24:25 +02:00
Toni 30730e95e5 Add Kismet protocol detection. (#1710)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 10:50:11 +02:00
Toni Uhlig 93a80a1c4b Fixed serializing bug caused by an unitialized serializer.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 10:49:27 +02:00
lns 93d65ed650 Support serialization of double-precision floating-point numbers. Fixes #1702.
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 10:49:27 +02:00
Nardi Ivan ac0d7ccb7e HTTP, SoftEther, Florensia: fix some memory corruptions
```
==12308==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600068f4b4 at pc 0x5603d2b56098 bp 0x7ffc967bd3b0 sp 0x7ffc967bd3a8
READ of size 2 at 0x60600068f4b4 thread T0
    #0 0x5603d2b56097 in ndpi_search_florensia /home/ivan/svnrepos/nDPI/src/lib/protocols/florensia.c:93:40
    #1 0x5603d2a2e835 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6
    #2 0x5603d2a2f247 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5462:12
    #3 0x5603d2a2eee7 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5481:12
    #4 0x5603d2a3ffb5 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15
    #5 0x5603d2955f4c in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1612:31
    #6 0x5603d294ee64 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2182:10
    #7 0x5603d290aa50 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #8 0x5603d290ab8b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17
    #9 0x7f972cdc9082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #10 0x5603d28494fd in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x4be4fd) (BuildId: e81373a75c70e248ba9ae985fcb4134e7113121a)
```
```
==14164==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60d0000000c1 at pc 0x561608bada2c bp 0x7ffc40d38ed0 sp 0x7ffc40d38690
READ of size 10 at 0x60d0000000c1 thread T0
    #0 0x561608bada2b in __interceptor_strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4e9a2b) (BuildId: 2e4fd67f1aeb93c491a04ae424331669c993ee9b)
    #1 0x561608e66f2a in dissect_softether_host_fqdn /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:226:2
    #2 0x561608e64e88 in ndpi_search_softether /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:316:8
    #3 0x561608ca7475 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6
    #4 0x561608ca823b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5450:10
    #5 0x561608ca7bdc in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5483:12
    #6 0x561608cb8bf5 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15
    #7 0x561608bff84e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #8 0x561608bff9c7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #9 0x7fc50662d082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #10 0x561608b3e42d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x47a42d) (BuildId: 2e4fd67f1aeb93c491a04ae424331669c993ee9b)
```

Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50105
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50110
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50268
2022-08-22 17:53:52 +02:00
Luca Deri a53f476585 Modified definition of hll_add 2022-08-17 16:25:10 +02:00
Luca Deri 82c83ef75e Updated RPM package 2022-08-15 16:29:08 +02:00
Luca Deri 4a843709fa Packages changes 2022-08-15 11:32:58 +02:00
alcir 780cbf5f76 Update Makefile.in (#1708)
Use `@libdir@` instead of *hardcoded* path, and also `@includedir@`
2022-08-15 08:20:17 +00:00
Luca Deri 743d046f6f Added nTap reference 2022-08-12 10:38:51 +02:00
Toni 5915bc8828 Improved Discord pattern. (#1707)
Signed-off-by: lns <matzeton@googlemail.com>

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-10 21:00:11 +02:00
Ivan Nardi 540848c254 HTTP: fix classification in some corner cases (#1704)
Initializing `low->detected_protocol_stack[1]` but not
`low->detected_protocol_stack[0]` lead to *very* strange errors.
Oss-fuzzer, exploiting this bug, has been able to crash the application
or to leak some memory

```
==19775==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x5624c0dd1802 bp 0x5624c197e848 sp 0x7fff4b4db690 T0)
==19775==The signal is caused by a READ memory access.
==19775==Hint: this fault was caused by a dereference of a high value address (see register values below).  Disassemble the provided pc to learn which register was used.
    #0 0x5624c0dd1802 in __asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x522802) (BuildId: 9a1ed9efcaf47e345767c86520372d28e31ca1aa)
    #1 0x5624c0e53ea6 in __interceptor_free (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x5a4ea6) (BuildId: 9a1ed9efcaf47e345767c86520372d28e31ca1aa)
    #2 0x5624c0ebb474 in free_wrapper /home/ivan/svnrepos/nDPI/example/reader_util.c:330:3
    #3 0x5624c0f75561 in ndpi_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:240:7
    #4 0x5624c0faa921 in ndpi_free_flow_data /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4915:2
    #5 0x5624c0f755ef in ndpi_free_flow /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8076:5
    #6 0x5624c0f755c4 in ndpi_flow_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:253:5
    #7 0x5624c0eba981 in ndpi_free_flow_info_half /home/ivan/svnrepos/nDPI/example/reader_util.c:305:25
    #8 0x5624c0ecb4bc in process_ndpi_collected_info /home/ivan/svnrepos/nDPI/example/reader_util.c:1317:5
    #9 0x5624c0edca36 in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1639:2
    #10 0x5624c0ed5172 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2182:10
    #11 0x5624c0e90a80 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #12 0x5624c0e90bbb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17
    #13 0x7f580731f082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #14 0x5624c0dcf52d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x52052d) (BuildId: 9a1ed9efcaf47e345767c86520372d28e31ca1aa)
```

```
==17492==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 5 byte(s) in 1 object(s) allocated from:
    #0 0x55b799b0f01e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x56101e) (BuildId: d2adbfb29a6eda6dc59fdfb8930d7e6496ac7b8b)
    #1 0x55b799bb3bf4 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:212:46
    #2 0x55b799bb3f28 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:279:13
    #3 0x55b799c8b9bf in processCertificateElements /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:683:39
    #4 0x55b799c81e89 in processCertificate /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:835:7
    #5 0x55b799c8d500 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:889:13
    #6 0x55b799ca80f1 in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1034:2
    #7 0x55b799ca3be2 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2475:5
    #8 0x55b799bf9345 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6
    #9 0x55b799bf9d57 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5462:12
    #10 0x55b799bf99f7 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5481:12
    #11 0x55b799c0b838 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15
    #12 0x55b799b4b87e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
```

```
==17511==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1 byte(s) in 1 object(s) allocated from:
    #0 0x55aed645e01e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x56101e) (BuildId: d2adbfb29a6eda6dc59fdfb8930d7e6496ac7b8b)
    #1 0x55aed6502bf4 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:212:46
    #2 0x55aed6502f28 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:279:13
    #3 0x55aed65ea0c6 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2153:34
    #4 0x55aed65dbe77 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:867:5
    #5 0x55aed65f70f1 in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1034:2
    #6 0x55aed65f2be2 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2475:5
    #7 0x55aed6548345 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6
    #8 0x55aed6548d57 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5462:12
    #9 0x55aed65489f7 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5481:12
    #10 0x55aed655a838 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15
    #11 0x55aed649a87e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
```

See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49844
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49842
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49906
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49915
2022-08-09 18:41:10 +02:00
Ivan Nardi 59a9bdeb55 SoftEtehr: fix a heap-buffer overflow (#1705)
```
==26717==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60c0000000bf at pc 0x55e503f8959b bp 0x7fff7f6876b0 sp 0x7fff7f686e58
READ of size 1 at 0x60c0000000bf thread T0
    #0 0x55e503f8959a in strncmp (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4f459a) (BuildId: 44038d7ca9391c877fae9b2735828da9905a6daa)
    #1 0x55e5042ee98f in dissect_softether_host_fqdn /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:236:8
    #2 0x55e5042ec41b in ndpi_search_softether /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:318:8
    #3 0x55e5040e0345 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6
    #4 0x55e5040e110b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5450:10
    #5 0x55e5040e0aac in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5483:12
    #6 0x55e5040f2838 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15
    #7 0x55e50403287e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #8 0x55e5040329f7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #9 0x7fd09fdbb082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #10 0x55e503f7145d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4dc45d) (BuildId: 44038d7ca9391c877fae9b2735828da9905a6daa)
```

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49901
2022-08-09 18:09:16 +02:00
Toni 2e25c36396 Add TiVoConnect dissector. Fixes #1697. (#1699)
* added static assert if supported, to complain if the flow struct changes

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-08 19:04:20 +02:00
Ivan Nardi 523360001e HTTP, Bittorrent, Kerberos: fix stopping of extra dissection (#1700)
The return value of the extra-dissection callback indicates if the extra
dissection needs to be called again.

In the HTTP cose, this setting to NULL of the callabck is wrong since we
stop extra dissection only if we have a hostname *and* a return code.
2022-08-06 20:06:56 +02:00
Ivan Nardi 120e01566a Bittorrent: don't use absolute time reference (#1701) 2022-08-06 20:01:06 +02:00
Ivan Nardi 1d4e27c473 Further simplification of ndpi_process_extra_packet() (#1698)
See 95e16872.

After c0732eda, we can safely remove the protocol list from
`ndpi_process_extra_packet()`.

The field `flow->check_extra_packets` is redundant; remove it.
2022-08-05 15:15:45 +02:00
Ivan Nardi c0732eda45 HTTP: improve sub-classification (#1696)
Content-matched sub-protocols (DASH, IPP, MPEGDASH...) shouldn't
ovewrite the previous master protocol (if any; usually HTTP).

Furthermore. the HTTP dissector shouldn't update the classification
(in the extra-dissection code path) if a content-matched sub-protocols
has already been found.

This commit should address the first part of the changes described in #1687.
2022-08-05 14:14:38 +02:00
Ivan Nardi 346e274a1b SoftEther: fix two heap-buffer-overflows (#1695)
The first change is a proper (hopefully) fix for the bug reported in
8b6a00f8.
The second one is related to:
```
==15096==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60f0000d7a00 at pc 0x55a2c593bd0b bp 0x7ffc92021cd0 sp 0x7ffc92021478
READ of size 3 at 0x60f0000d7a00 thread T0
    #0 0x55a2c593bd0a in strncmp (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x56fd0a) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #1 0x55a2c5d1d9f9 in dissect_softether_host_fqdn /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:249:9
    #2 0x55a2c5d1b55b in ndpi_search_softether /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:348:9
    #3 0x55a2c5b0e9c5 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5407:6
    #4 0x55a2c5b0f78b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5443:10
    #5 0x55a2c5b0f12c in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5476:12
    #6 0x55a2c5b20f39 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6303:15
    #7 0x55a2c5a3014c in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1600:31
    #8 0x55a2c5a29062 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2170:10
    #9 0x55a2c59e51a2 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #10 0x55a2c590acb2 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x53ecb2) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #11 0x55a2c590a3c5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x53e3c5) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #12 0x55a2c590c0f6 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::vector<fuzzer::SizedFile, std::allocator<fuzzer::SizedFile>>&) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x5400f6) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #13 0x55a2c590c663 in fuzzer::Fuzzer::Loop(std::vector<fuzzer::SizedFile, std::allocator<fuzzer::SizedFile>>&) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x540663) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #14 0x55a2c58faff2 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x52eff2) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #15 0x55a2c5923c82 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x557c82) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #16 0x7f504ab98082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #17 0x55a2c58efb1d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x523b1d) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)

0x60f0000d7a00 is located 0 bytes to the right of 176-byte region [0x60f0000d7950,0x60f0000d7a00)
```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49736
2022-08-05 13:43:09 +02:00
Luca Deri 10f283b467 Fix for systems with no realpath (MacOS) 2022-08-05 11:17:59 +02:00
Luca Deri 9fb11c909e Win fixes 2022-08-05 11:17:28 +02:00
Luca Deri bcf7f1713c Win and indentation fixes 2022-08-05 11:17:09 +02:00
Toni ac24b35b1f Add Discord dissector. (#1694)
* fixed RiotGames false positive

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-03 12:03:36 +02:00
Luca Deri c3c73e7bff Cosmetic changes 2022-08-03 11:31:32 +02:00
Luca Deri 63fe4b4306 Compilation fix (not all compilers like restrict) (part 2) 2022-08-03 11:12:15 +02:00
Luca Deri 154d6a5069 Compilation fix (not all compilers like restrict) 2022-08-03 10:47:31 +02:00
Toni b3e722e5a8 Improved nDPI JSON serialization. (#1689)
* fixed autoconf CFLAGS/LDFLAGS MSAN issue which could lead to build errors
 * introduced portable version of gmtime_r aka ndpi_gmtime_r
 * do as most as possible of the serialization work in ndpi_utils.c
 * use flow2json in ndpiReader

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-02 17:54:44 +02:00
Toni 26aafd931c Add Activision dissector. (#1693)
Signed-off-by: lns <matzeton@googlemail.com>
2022-08-02 16:15:07 +02:00
Ivan Nardi 013212c67c TLS: avoid useless extra dissection (#1690)
If we have seen Application Data blocks in both directions, it means we
are after the handshake. At that point, extra dissection is useless.

Useful feature with TLS mid-sessions, i.e. sessions without initial
packets.
2022-07-31 16:57:17 +02:00
Ivan Nardi 2a596c79e6 HTTP: fix classification (#1692)
If we have a valid HTTP sessions, we should ignore
`flow->guessed_protocol_id` field (i.e. classification "by-port")
altogheter.

The attached trace was classified as "SIP/HTTP" only because the *client*
port was 5060...
As a general rule, having a classification such as "XXXX/HTTP" is
*extremely* suspicious.
2022-07-30 22:57:20 +02:00
Ivan Nardi 8b6a00f84b SoftEther: fix heap-buffer-overflow (#1691)
```
==160665==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x604000000038 at pc 0x55f7250d9a5c bp 0x7fff02c82b90 sp 0x7fff02c82350
READ of size 4 at 0x604000000038 thread T0
    #0 0x55f7250d9a5b in __interceptor_strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x548a5b) (BuildId: 12fd06e7a171f035d3a25f016184ac357088379c)
    #1 0x55f7253e6495 in dissect_softether_ip_port /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:303:3
    #2 0x55f7253e5703 in ndpi_search_softether /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:330:9
    #3 0x55f7251d87c5 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5397:6
    #4 0x55f7251d958b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5433:10
    #5 0x55f7251d8f2c in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5466:12
    #6 0x55f7251ead39 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6293:15
    #7 0x55f72512b87e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #8 0x55f72512b9f7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #9 0x7fdef837b082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #10 0x55f72506a45d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4d945d) (BuildId: 12fd06e7a171f035d3a25f016184ac357088379c)
```

Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49638
2022-07-30 22:55:46 +02:00
Ivan Nardi d54d5083b3 SMTPS, POPS, IMAPS: fix classification and extra dissection (#1685)
The big change in TLS code is to allow "master" protocols other than
TLS/DTLS, like SMTPS, POPS and IMAPS.
This change will allow, in a future, a proper and complete TLS dissection
for all these protocols with "STARTTLS"-like messages.
2022-07-30 12:05:43 +02:00
Toni 8402bd68ad Tiny Softether dissector improvement. (#1688)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-30 10:03:42 +02:00
Toni ed4f106a0d Add Softether dissector. (#1679)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-29 19:29:54 +02:00
Ivan Nardi 405a52ed65 Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1683)
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.

Add an option to `ndpiReader` to dump them; enabled by default in the
unit tests.
This new option enables/disables dumping of "num dissectors calls"
values, too (see b4cb14ec).
2022-07-29 15:25:00 +02:00
Ivan Nardi df2e11ef51 Revert "Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1677)" (#1682)
This reverts commit bb83899985.
2022-07-29 12:08:40 +02:00
Ivan Nardi bb83899985 Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1677)
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.

Add an option to `ndpiReader` to dump them; disabled by default to avoid
too much fuss with the unit tests.
2022-07-29 12:07:41 +02:00
Ivan Nardi 95e16872fa First step in simplify ndpi_process_extra_packet() (#1680)
Move the prottocol specific logic into the proper dissector code, where
it belongs.

Next step: remove that list of protocols. Long goal: remove this
function altogether...
2022-07-29 12:07:14 +02:00
Ivan Nardi 172e698bb8 TINC: avoid processing SYN packets (#1676)
Since e6b332aa, we have proper support for detecting client/server
direction. So Tinc dissector is now able to properly initialize the
cache entry only when needed and not anymore at the SYN time; initializing
that entry for **every** SYN packets was a complete waste of resources.

Since 4896dabb, the various `struct ndpi_call_function_struct`
structures are not more separate objects and therefore comparing them
using only their pointers is bogus: this bug was triggered by this
change because `ndpi_str->callback_buffer_size_tcp_no_payload` is now 0.
2022-07-28 12:39:18 +02:00
Ivan Nardi d8d525fff2 Update the protocol bitmask for some protocols (#1675)
Tcp retransmissions should be ignored.

Remove some unused protocol bitmasks.

Update script to download Whatsapp IP list.
2022-07-27 11:46:45 +02:00
Toni ab3a678ad4 Add AVAST dissector. (#1674)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-25 18:07:44 +02:00
Ivan Nardi b190dab6bc Improve handling of HTTP-Proxy and HTTP-Connect (#1673)
Treat HTTP-Proxy and HTTP-Connect flows like the HTTP ones:
print/serialize all the attributes and allow parsing of replies.

The line about "1kxun" has been removed to avoid regressions in 1KXUN
classification in `tests/pcap/1kxun.pcap`. I haven't fully understod
what was happening but the comment at the beginning of `static
ndpi_category_match category_match[]` says that we can't have overlaps
between `host_match` and `category_match` lists and that is no longer true
since 938e89ca.
Bottom line: removing this line seems the right thing to do, anyway.
2022-07-25 12:57:33 +02:00
Ivan Nardi 86a3e4c8c3 Add support for Memory sanitizer (#1652) 2022-07-24 18:12:25 +02:00
Ivan Nardi e6b332aa4a Add support for flow client/server information (#1671)
In a lot of places in ndPI we use *packet* source/dest info
(address/port/direction) when we are interested in *flow* client/server
info, instead.

Add basic logic to autodetect this kind of information.

nDPI doesn't perform any "flow management" itself but this task is
delegated to the external application. It is then likely that the
application might provide more reliable hints about flow
client/server direction and about the TCP handshake presence: in that case,
these information might be (optionally) passed to the library, disabling
the internal "autodetect" logic.

These new fields have been used in some LRU caches and in the "guessing"
algorithm.
It is quite likely that some other code needs to be updated.
2022-07-24 17:46:24 +02:00
Luca Deri 523f22b942 Added vides/presentations section 2022-07-23 17:05:31 +02:00
Toni a25b2a7e37 Added AliCloud server access dissector. (#1672)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-23 11:21:49 +02:00
Zied Aouini 40d7354dd7 Windows error C2466 fix (array size init with 0). 2022-07-22 21:58:25 +02:00
Zied Aouini 9d040f6317 Windows compilation fix. 2022-07-22 20:14:30 +02:00
Ivan Nardi 52005e88ed TLS: improve reassembler (#1669)
* TLS: cosmetic changes

* TLS: improve reassembler

We might need to contemporary re-order messages from both directions:
use one buffer per direction.
2022-07-22 12:19:21 +02:00
Ivan Nardi ce6f11840d Bittorrent: fix keys in cache code (#1670) 2022-07-22 12:10:43 +02:00
Luca Deri e85d6d7db7 RPM fix 2022-07-20 21:26:07 +02:00
Nardi Ivan 00eb2393f4 STUN: improve detection of Hangout/Duo and FB voip calls 2022-07-20 20:36:31 +02:00
Nardi Ivan d66aa49787 DTLS: fix exclusion of DTLS protocol
Add an helper to exclude a generic protocol
2022-07-20 19:16:18 +02:00
Luca Deri a9204eb936 Rocky fix 2022-07-20 17:23:17 +02:00
Nardi Ivan e1edb08f06 SKYPE: fix detection over UDP
Commit ba6a48c9 is completely bogus: we can't set extra dissection
without having set a proper classification.

The idea behind that commit seems to be that we need to look for 2
(consecutives?) packets with the same crc/pattern: try to implement this
logic in a saner way.
2022-07-20 16:13:55 +02:00
Nardi Ivan 5702c6fb08 SKYPE: remove detection over TCP
Skype detection over TCP has been completely disable since 659f75138 (3
years ago!).
Since that logic was too weak anyway, remove it.
2022-07-20 16:13:55 +02:00
Ivan Nardi c72660d7d3 reader_util: stop processing a flow (#1666)
We should stop processing a flow if all protocols have been excluded or
if we have already processed too many packets.
2022-07-20 14:48:09 +02:00
Ivan Nardi b472a49271 BITTORRENT: fix confidence (#1664)
Remove two unused parameters.
2022-07-20 13:59:51 +02:00
Ivan Nardi ac1113b9ac ahocorasick: fix char/uchar bug (#1597)
Porting of https://github.com/vel21ripn/nDPI/commit/06e2967d0c26ab214683a2f5565f4012fd523537
Close #1506
2022-07-20 10:59:17 +02:00
Toni be1ac1b430 Fix unitialized use of W[16]. (#1662)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-15 14:15:37 +02:00
Luca b80218a0e4 Added null pointer checks 2022-07-14 18:04:51 +02:00
Luca 842e0a3648 Restored -O2 in default builds 2022-07-14 17:24:18 +02:00
Toni 7c19de4904 Do not interfere with CFLAGS/LDFLAGS env anymore. (#1659)
* CI fixes
 * some build systems do not like that (e.g. OpenWrt)
 * fixed some rrdtool related build warnings/errors

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-13 19:44:18 +02:00
Toni ae2bedce3a Improved Jabber/XMPP detection. (#1661)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-13 17:55:33 +02:00
Ivan Nardi 407155755d ASN1/BER: fix signed integer overflow (#1660)
```
protocols/snmp_proto.c:77:23: runtime error: signed integer overflow: 6 + 2147483647 cannot be represented in type 'int'
 #0 0x52f69e in ndpi_search_snmp ndpi/src/lib/protocols/snmp_proto.c:77:23
 #1 0x4c5347 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:5211:4
 #2 0x4c5591 in ndpi_check_flow_func ndpi/src/lib/ndpi_main.c:0
 #3 0x4c8903 in ndpi_detection_process_packet ndpi/src/lib/ndpi_main.c:6145:15
 #4 0x4b3712 in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_process_packet.c:29:5
[...]
```
Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49057
2022-07-12 18:39:05 +02:00
Luca 9c235796af Introduced risk accountability 2022-07-12 10:36:40 +02:00
Luca cba2d12051 Cosmetic change 2022-07-12 10:36:32 +02:00
Ivan Nardi b4cb14ec19 Keep track of how many dissectors calls we made for each flow (#1657) 2022-07-11 09:47:47 +02:00
Ivan Nardi df599e5eff HTTP: improve detection of WindowsUpdate (#1658)
WindowsUpdate is also transported over HTTP, using a numeric IP as
hostname (some kinds of CDN?)
2022-07-10 17:08:37 +02:00
Ivan Nardi 1fcd03a6b6 Remove unsafe access to flow->protos union (#1656)
We can access `flow->protos` only if we already have set a valid
classification.

It is quite likely that this code is never trigger, anyway.
2022-07-10 17:08:22 +02:00
Ivan Nardi 2edfaeba4a SNMP: fix detection (#1655)
We can write to `flow->protos` only when we are sure about SNMP
classification.

Use the generic wrapper to decode ASN1 BER integer
2022-07-10 17:07:52 +02:00
Ivan Nardi 997dce0f04 SIP: improve detection (#1654) 2022-07-09 05:45:42 +02:00
Ivan Nardi dfe6557e18 TFTP: fix memory access (#1653) 2022-07-08 18:39:05 +02:00
Ivan Nardi 510517126a LDAP: rewrite dissection (#1649) 2022-07-08 12:50:46 +02:00
Toni 9b95876973 Enhances gprof usage. (#1651)
* gprof results were incorrectly displayed

Signed-off-by: lns <matzeton@googlemail.com>
2022-07-08 12:05:55 +02:00
Ivan Nardi f8076e3a58 SMB: add (partial) support for messages split into multiple TCP segments (#1644) 2022-07-07 19:24:31 +02:00
Ivan Nardi ff4e010501 Avoid spurious calls to extra dissection (#1648)
If the extra callabck is not set, calling the extra dissection is only a
waste of resources...
2022-07-07 17:49:35 +02:00
Ivan Nardi d254ae54f3 SMTP: add support for X-ANONYMOUSTLS comamnd (#1650) 2022-07-07 16:46:18 +02:00
Ivan Nardi feaa1df1ed Kerberos: add support for Krb-Error messages (#1647) 2022-07-07 16:45:49 +02:00
Ivan Nardi 056e742304 Spotify: remove some useless ip ranges (#1646)
These AS numbers are no more related to Spotify (or, if they are, they
don't have any prefixes anyway).
Even if we find some valid Spotify AS, we should handle them via the
generic "autogenerated logic" used for every AS, and not in the
dissector code.
2022-07-07 15:39:19 +02:00
Nardi Ivan 2636c07571 MONGODB: avoid false positives 2022-07-07 15:36:05 +02:00
Nardi Ivan a31e79fc3c TLS: ignore invalid Content Type values 2022-07-07 15:36:05 +02:00
Toni 15042870f9 Added Threema Messenger. (#1643)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 19:30:10 +02:00
Toni Uhlig 105f661e46 Added RiotGames ASN update.
* updated asn lists

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 14:37:26 +02:00
Toni Uhlig a1c3d05a74 Added another RiotGames signature.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 14:37:26 +02:00
Luca Deri 814f9cd21e Moved to 4.5 2022-07-06 12:43:56 +02:00
Toni 175f863665 Label SMTP w/ STARTTLS as SMTPS *and* dissect TLS clho. (#1639)
* Label SMTP w/ STARTTLS as SMTPS *and* dissect TLS clho.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Revert "SMTP with STARTTLS is now identified as SMTPS"

This reverts commit 52d987b603.

* Revert "Compilation fix"

This reverts commit c019946f60.

* Sync unit tests.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 12:40:25 +02:00
Luca Deri c019946f60 Compilation fix 2022-07-05 17:38:32 +02:00
Ivan Nardi 7645909460 Fix handling of NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1636) 2022-07-05 17:01:00 +02:00
Luca Deri 52d987b603 SMTP with STARTTLS is now identified as SMTPS 2022-07-05 17:00:21 +02:00
Toni f4a1739f9c Detect SMTPs w/ STARTTLS as TLS and dissect client/server hello. Fixes #1630. (#1637)
* FTP needs to get updated as well as it has similiar STARTTLS semantics -> follow-up

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-05 16:35:23 +02:00
Toni 388dfb8e13 Run regression tests from different locations at the same time w/o side effects on the results. (#1638)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-05 15:08:59 +02:00
Luca Deri 7fa8d882d8 Exported username in flow information 2022-07-04 22:52:54 +02:00
Luca Deri 461589517e Updated ndpi_check_flow_risk_exceptions() signature 2022-07-04 21:38:38 +02:00
Luca Deri e7a5eaecde Cleaned-up issuer DN check code adding
u_int8_t ndpi_check_issuerdn_risk_exception(struct ndpi_detection_module_struct *ndpi_str, char *issuerDN);

Added new API function for checking nDPI-configured exceptions
u_int8_t ndpi_check_flow_risk_exception(struct ndpi_detection_module_struct *ndpi_str,
                                        u_int num_params,
                                        ndpi_risk_params **params);
2022-07-04 18:41:01 +02:00
Luca Deri 8ff2860601 Set CiscoVPN as a network protocol 2022-07-04 18:41:01 +02:00
Toni Uhlig b3ab66020f Updated JA3/SSL fingerprints.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 16:05:22 +02:00
Toni Uhlig b3d3e3b210 Replaced malicious JA3-md5/SSL-cert-sha1 ac automata with hashmaps.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 16:05:22 +02:00
Toni 4ff8aa48b2 Added UltraSurf protocol dissector. (#1618)
* TLSv1.3 UltraSurf flows are not detected by now

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 16:04:53 +02:00
Ivan Nardi 5aa3d9126f Add two new confidence values: confidence by partial DPI (#1632)
Used for all classifications based on partial/incomplete DPI
information, i.e. all classifications done in `ndpi_detection_giveup()`.
2022-07-04 13:56:51 +02:00
Ivan Nardi 4445989588 Update host content list match (#1633)
Improve classifications of Outlook, Cachefly, Cloudflare, Tiktok and
Cybersecurity.
2022-07-04 13:21:11 +02:00
Toni a1de0285eb Sync Psiphon unit test. (#1634)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 13:20:44 +02:00
Toni 75f7da5c26 Added Psiphon detection patterns. See #566 and #1099. (#1631)
* The traces are not up to date, but this is the best we got so far.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 10:34:54 +02:00
Ivan Nardi 192a32207c OCSP: improve detection (#1629) 2022-07-04 07:22:56 +02:00
Toni a74fc089c4 Added i3D and RiotGames protocol dissectors. (#1609)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 20:43:30 +02:00
Ivan Nardi faaff58620 TargusDataspeed: avoid false positives (#1628)
TargusDataspeed dissector doesn't perform any real DPI checks but it only
looks at the TCP/UDP ports.
Delete it, and use standard logic to classify these flows by port.
2022-07-03 20:28:58 +02:00
Ivan Nardi 50c0212df1 Update ASN/IPs lists (#1627) 2022-07-03 19:50:47 +02:00
Ivan Nardi b5fb2066cb bins: add support for 64bit bins (#1626) 2022-07-03 19:25:15 +02:00
Ivan Nardi 422d002542 Skinny: rework and improve classification (#1625) 2022-07-03 19:25:00 +02:00
Ivan Nardi eed47acfc8 Skype_Teams, Mining, SnapchatCall: fix flow category (#1624) 2022-07-03 18:51:16 +02:00
Ivan Nardi 77ac58e553 Minor changes in how classification results are set (#1623)
Protocol classification should always be set via
`ndpi_set_detected_protocol()`: this way, the values in
`flow->detected_protocol_stack[]` are always coherent.
2022-07-03 18:45:24 +02:00
Ivan Nardi 060e894d5b Usenet: improve dissection (#1622) 2022-07-03 18:08:04 +02:00
Ivan Nardi fdb1649a49 Fix category for mail sessions (#1621)
Close #629
2022-07-03 17:47:58 +02:00
Ivan Nardi 5fe6087686 TLS: add support for old DTLS versions and for detection of mid-sessions (#1619) 2022-07-03 17:44:17 +02:00
Ivan Nardi 5f6fa6d164 Fix a compilation warning (#1620)
With clang-15 (nightly build)
```
In file included from ndpi_bitmap.c:39:
./third_party/src/roaring.cc:14233:13: warning: variable 'run_count' set but not used [-Wunused-but-set-variable]
        int run_count = 0;
```
2022-07-03 17:43:38 +02:00
Toni Uhlig 69ccb39741 Generate profiling results as PNG.
* use -ltcmalloc_and_profiler and try to get rid of LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libprofiler.so

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 17:38:43 +02:00
lns f2d1edbedf gprof test/CI integration
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 17:38:43 +02:00
Toni 1a01e8dc68 Improved TFTP. Dissect Read/Write Request filenames. (#1617)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 14:37:05 +02:00
Toni 5ca82ad84b Added TSAN support. (#1613)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 14:35:21 +02:00
Toni 59b00b00a7 Fix byte-order issue during ndpiReader tcp/udp src/dst port serialization. Fixes #1608. (#1614)
* fixed possible memory leak caused by an invalid call to `node_proto_guess_walker()` during serialization
 * execute serialization code while running regression tests

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 11:16:52 +02:00
Toni 7c5c811eb0 Added Cloudflare WARP detection patterns. (#1615) (#1616)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-02 14:57:56 +02:00
Luca Deri 008a1790e4 Fixed SMTP default port 587 2022-07-02 11:49:22 +02:00
Toni bb72aa4767 Added TunnelBear VPN detection patterns. (#1615)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-01 13:19:17 +02:00
Luca Deri 8f6a006e36 Updated (C) 2022-06-30 14:53:47 +02:00
Luca Deri 6cf7838295 Removed space from "Genshin Impact" 2022-06-30 14:50:05 +02:00
Toni c96f4512fa sync unit tests (#1612)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-06-30 09:37:30 +02:00
Luca Deri 5f1caeb54e Fix after the protocol name update 2022-06-28 17:26:11 +02:00
Luca d393fb969d Renamed Z39.50 -> Z3950 as the '.' breaks the naming convention
QUIC is a network protocol
2022-06-28 17:20:58 +02:00
Luca Deri 227ab5c105 Enhanced TLS risk info reported to users 2022-06-28 00:01:00 +02:00
Luca Deri 4296ec969a Added default port for syslog TCP 2022-06-27 18:36:56 +02:00
Ivan Nardi b2b61011ae Fix compilation and sync unit tests results (#1606) 2022-06-20 14:27:13 +02:00
Luca Deri ab09b8ce2e Added unidirectional traffic flow risk 2022-06-20 00:22:13 +02:00
Toni c287eb835b Improved SOAP via HTTP. (#1605)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-18 17:19:16 +02:00
Toni 6cd8f8cc6d Improved GenshinImpact protocol dissector. (#1604)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-18 15:11:59 +02:00
Toni 432de5eb57 Added collectd dissector (again). (#1601)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-17 19:56:33 +02:00
Toni 9c8b2d63da Replaced nDPI's internal hashmap with uthash. (#1602)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-17 19:50:31 +02:00
Toni 20a29c393f Improved IPSec/ISAKMP detection. (#1600)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-16 09:16:18 +02:00
Luca c43360f8e6 Added new test pcaps 2022-06-15 18:57:46 +02:00
claudio_burrafatoandToni c73a0988cd Add some statistics to ndpiReader (#1587)
* Add some statistics to ndpiReader

The purpose of this version of ndpiReader is too adding some other statistics printed by ndpiReader. In this simple version the domain names(in the flows) that are collected are:
flow-> ssh_tls.server_info
flow-> host_server_name
and are placed in a UT_hash_table, ordering them by number of occurrences.

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update ndpiReader.c

* Update ndpiReader.c

* Update ndpiReader.c

* Update ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>
2022-06-15 16:36:25 +02:00
Ivan Nardi e2cc08bfe5 Add support for PIM (Protocol Indipendent Multicast) protocol (#1599)
Close #1598
2022-06-15 12:25:26 +02:00
Toni d1773cc8e3 Improved WhatsApp detection. (#1595)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-14 20:06:48 +02:00
Ivan Nardi bdf54d725b Fix invalid memory access (#1596)
We can access `flow->protos` union only after checking the protocol.

Checking `flow->detected_protocol.master_protocol` is redundant because
we already check it in `is_ndpi_proto`

```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==29739==ERROR: AddressSanitizer: SEGV on unknown address 0x000000353820 (pc 0x7f9b64dd2717 bp 0x7fff161a52f0 sp 0x7fff161a4aa8 T0)
==29739==The signal is caused by a READ memory access.
    #0 0x7f9b64dd2717  /build/glibc-SzIz7B/glibc-2.31/string/../sysdeps/x86_64/multiarch/strlen-avx2.S:96
    #1 0x555c65e597d8 in __interceptor_strlen (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x6407d8) (BuildId: 11ac8ec30f1d49fb0276c9b03368e491505d2bba)
    #2 0x555c65fd85fa in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:269:13
    #3 0x555c65f3e8c6 in process_ndpi_collected_info /home/ivan/svnrepos/nDPI/example/reader_util.c:1188:36
    #4 0x555c65f52cab in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1567:2
    #5 0x555c65f4b632 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2110:10
    #6 0x555c65f04d29 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:109:7
    #7 0x555c65f054bb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:181:17
    #8 0x7f9b64c6e082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #9 0x555c65e4253d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x62953d) (BuildId: 11ac8ec30f1d49fb0276c9b03368e491505d2bba)

```

Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=48020
2022-06-14 11:44:32 +02:00
Ivan Nardi 8dcaa5c0e1 DNS: fix TTL check and sync unit test results (#1594) 2022-06-14 09:29:16 +02:00
Luca Deri defe7d7f79 Updated DNS alert triggered only with TTL == 0 2022-06-14 00:13:05 +02:00
Luca Deri 831562c141 Restored ndpi_set_proto_defaults() prototype
Updated test results
2022-06-13 23:53:32 +02:00
Luca Deri 7a172ce11e Added check for DGA names that resolve to a valid record 2022-06-13 23:42:07 +02:00
Luca Deri cf5873ffd7 Improved DNS traffic analysis
Added ability to identify application and network protocols
2022-06-13 23:19:47 +02:00
Luca Deri 32428026d6 Added DNS record TTL check 2022-06-13 17:05:56 +02:00
Toni df0ff9bcbd Added gprof CPU/HEAP profiling support. (#1592)
* Some small auto{conf,make} improvements

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-12 21:00:41 +02:00
Toni 341f58fd80 Removed Makefile references to legacy code. (#1589)
- feature removed with f5545a80f9

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-08 13:37:11 +02:00
Luca Deri 11babc7ea3 Added Pragmatic General Multicast (PGM) protocol detection 2022-06-08 09:11:22 +02:00
Toni bb3c2e82de Dissect host line if SSDP contains such. (#1586)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-07 11:15:47 +02:00
Toni 938e89ca33 Reimplemented 1kxun application protocol. (#1585)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-06 18:04:49 +02:00
Toni df78e67518 Prevent compilation failure if, for whatever reason, NDPI_API_VERSION is empty. (#1584)
- This is not a fix for #1494, just an improvment.

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-06 15:53:48 +02:00
Toni 0b3f8ed849 Fixed syslog false negatives. (#1582)
- RSH vs Syslog may still happen for midstream traffic

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-05 23:01:56 +02:00
Ivan Nardi 1d492c9ed8 Fix some debug messages (#1583)
Increase max number of flows handled during fuzzing
2022-06-05 14:45:21 +02:00
Luca Deri 59c084b166 Updated test results 2022-06-05 11:34:10 +02:00
Luca Deri 2202365b95 Fixed invalid DHCP dissection 2022-06-05 11:19:28 +02:00
Luca Deri b6ae9642d3 Fixed DHCP dissection bug 2022-06-05 11:14:23 +02:00
Toni 7419cfee64 Added RSH dissector. Fixes #202. (#1581)
- added syslog false-positive pcap that was missing in 09fbe0a64a
 - added NDPI_ARRAY_LENGTH() macro, usable on `type var[]` declarations

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-04 19:12:53 +02:00
Ivan Nardi 338427d712 Add support for GoTo products (mainly GoToMeeting) (#1580)
There is some overlap with Citrix protocol.
2022-06-04 14:30:52 +02:00
Toni ff8e1e197e Fix syslog heap overflow introduced in 09fbe0a64a. (#1579)
- fixes #1578

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-04 08:47:47 +02:00
Toni 09fbe0a64a Fixed syslog false positives. (#1577)
* syslog: removed unnecessary/unreliable printable string check
 * added `ndpi_isalnum()`
 * splitted `ndpi_is_printable_string()` into `ndpi_is_printable_buffer()` and `ndpi_normalize_printable_string()`

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-03 18:21:29 +02:00
Toni 6149c0f880 Fix heap buffer overflow mentioned in #1574. (#1576)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-02 13:07:09 +02:00
Ivan Nardi 4ff1bf29df TLS: fix use-of-uninitialized-value error (#1573)
Proper fix for the error already reported in 9040bc74

```
Uninitialized bytes in __interceptor_strlen at offset 3 inside [0x7ffc7a147390, 4)
==111876==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x55e3e4f32e5b in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:268:13
    #1 0x55e3e4ef7391 in ndpi_set_risk /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:2254:12
    #2 0x55e3e5022fdf in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1523:8
    #3 0x55e3e503af44 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:865:5
    #4 0x55e3e50397cd in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1024:2
    #5 0x55e3e503570c in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2453:5
    #6 0x55e3e4f84a6a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5150:6
    #7 0x55e3e4f85778 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5198:12
    #8 0x55e3e4f851e1 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5217:12
    #9 0x55e3e4f96c7a in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6076:15
    #10 0x55e3e4ed91ef in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #11 0x55e3e4eda27e in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #12 0x7f5cb3146082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #13 0x55e3e4e5140d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0xa340d) (BuildId: 0c02c433e039970dd13a60382b94dd5a8e19f625)
```
2022-06-02 00:15:52 +02:00
Toni eac11030ac Removed README.nDPI as it does not provide any new information not covered by README.md (#1572)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-31 22:11:48 +02:00
Toni 00c62a7c3c Removed LGTM ql query for packet payload integer arithmetic. (#1570)
* Detection is unreliable and results in too much false-positives (and false-negatives!)

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-31 16:46:44 +02:00
Toni d8d806c79f Force roaring bitmap to use ndpi memory wrappers. (#1569)
GCC analyzer won't complain about possible use-after-free (false positive).

 * tests/do.sh prints word diff's only once and not the same over and over again
 * sync unit tests

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-31 12:06:01 +02:00
Ivan Nardi 9040bc74c9 TLS: fix stack-buffer-overflow error (#1567)
```
==300852==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f108951f060 at pc 0x5641db0ee78c bp 0x7fff3b10b910 sp 0x7fff3b10b0d0
WRITE of size 116 at 0x7f108951f060 thread T0
    #0 0x5641db0ee78b in __interceptor_strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x64e78b) (BuildId: 23cb34bbaf8ac11eb97563bbdc12e29ead9fb0fa)
    #1 0x5641db28efad in tlsCheckUncommonALPN /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1224:7
    #2 0x5641db27ec76 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1533:6
    #3 0x5641db295677 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:865:5
    #4 0x5641db2935bb in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1024:2
    #5 0x5641db28f692 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2446:5
    #6 0x5641db1d87ce in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5150:6
    #7 0x5641db1d91e4 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5198:12
    #8 0x5641db1d8e87 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5217:12
    #9 0x5641db1eb4a7 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6076:15
    #10 0x5641db140b45 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #11 0x5641db14130b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #12 0x7f108bcab082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #13 0x5641db07f46d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x5df46d) (BuildId: 23cb34bbaf8ac11eb97563bbdc12e29ead9fb0fa)

Address 0x7f108951f060 is located in stack of thread T0 at offset 96 in frame
    #0 0x5641db28ec4f in tlsCheckUncommonALPN /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1204

  This frame has 1 object(s):
    [32, 96) 'str' (line 1218) <== Memory access at offset 96 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x64e78b) (BuildId: 23cb34bbaf8ac11eb97563bbdc12e29ead9fb0fa) in __interceptor_strncpy
Shadow bytes around the buggy address:
```

Avoid zeroing the entire string.

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47730
2022-05-31 10:33:42 +02:00
Luca Deri 354addd693 Updated risk results 2022-05-30 23:28:59 +02:00
Luca Deri b306a4b8ed Improved message for known proto on non std port 2022-05-30 23:23:18 +02:00
Luca Deri b3bf695723 Added check 2022-05-30 22:39:23 +02:00
Toni 00ee364a7e Updated README.md (#1562)
* make check great again (not so much)
 * make doc/doc-view
 * CI updates

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-30 19:48:05 +02:00
Ivan Nardi ca115774c9 TLS: fix use-of-uninitialized-value error (#1566)
```
Uninitialized bytes in __interceptor_strlen at offset 0 inside [0x7ffc987443a0, 7)
==282918==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x5651277e001b in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:268:13
    #1 0x5651277a45c1 in ndpi_set_risk /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:2254:12
    #2 0x5651278cfaa6 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1515:8
    #3 0x5651278e7834 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:865:5
    #4 0x5651278e60bd in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1024:2
    #5 0x5651278e1ffc in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2445:5
    #6 0x565127831c2a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5150:6
    #7 0x565127832938 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5198:12
    #8 0x5651278323a1 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5217:12
    #9 0x565127843e3a in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6076:15
    #10 0x56512778641f in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #11 0x5651277874ae in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #12 0x7f4c46794082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #13 0x5651276fe40d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0xa340d)
```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47728
2022-05-30 19:40:47 +02:00
Luca Deri 912b69af60 Redefined type name to avoid conflicts 2022-05-30 18:29:08 +02:00
Luca 34ede63c15 Added ability to return risk info in JSON format in ndpi_get_flow_risk_info() 2022-05-30 17:54:30 +02:00
Toni 48065d80e9 Support word diff for tests/do.sh for better readability. (#1565)
* Sync unit tests

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-30 17:33:03 +02:00
Toni 32750271c3 Prohibit MPEG-DASH to set HTTP as application protocol. (#1560)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-30 14:54:27 +02:00
Ivan Nardi c4f50b2cda HTTP: fix heap-buffer-overflow error (#1564)
```
==222479==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60b000014739 at pc 0x55af06f2364f bp 0x7ffd7b6f4bf0 sp 0x7ffd7b6f4378
READ of size 12 at 0x60b000014739 thread T0
    #0 0x55af06f2364e in printf_common(void*, char const*, __va_list_tag*) asan_interceptors.cpp.o
    #1 0x55af06f24f70 in __interceptor_snprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x613f70) (BuildId: f6545ec2bd7663bc3f16aeeb87bddc64d173a2a8)
    #2 0x55af0720927f in ndpi_check_http_header /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:982:2
    #3 0x55af071f4797 in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1219:5
    #4 0x55af071f05c5 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1402:3
    #5 0x55af07080d1e in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5150:6
    #6 0x55af07081734 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5198:12
    #7 0x55af070813d7 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5217:12
    #8 0x55af070939f7 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6076:15
    #9 0x55af06fc7e1f in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1541
    #10 0x55af06fc7e1f in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2110
    #11 0x55af06fc2859 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:109:7
    #12 0x55af06fc2feb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:181:17
    #13 0x7efe5eaac082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #14 0x55af06f0055d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x5ef55d) (BuildId: f6545ec2bd7663bc3f16aeeb87bddc64d173a2a8)

0x60b000014739 is located 0 bytes to the right of 105-byte region [0x60b0000146d0,0x60b000014739)
allocated by thread T0 here:
    #0 0x55af06f84bae in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x673bae) (BuildId: f6545ec2bd7663bc3f16aeeb87bddc64d173a2a8)
    #1 0x55af06fc2673 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:103:31
    #2 0x55af06fc2feb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:181:17
    #3 0x7efe5eaac082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16

```
Fiund by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47724
2022-05-30 14:39:01 +02:00
Ivan Nardi 3b825fca6d Certificate timestamps should be printed in UTC (#1563)
This way, the CI results are consistent
2022-05-30 13:43:05 +02:00
Luca Deri 897f8314b2 Fixed dispay bug for risk_info 2022-05-30 01:12:38 +02:00
Luca Deri 1da9f1a36f Updated tests results
Code cleanup
2022-05-30 00:54:17 +02:00
Luca Deri f25deeccb1 Added RiskInfo string 2022-05-30 00:32:32 +02:00
Ivan Nardi a9d7cc4841 Fix dissection of IPv4 header (#1561)
See: https://github.com/ntop/nDPI/runs/6643914510?check_suite_focus=true

Convert al the `MIN(a,b)` calls to `ndpi_min(a,b)`
2022-05-29 22:39:48 +02:00
Ivan Nardi 9c1a53f39f Dazn: add support for Dazn streaming service (#1559)
Update .gitignore file
2022-05-29 17:47:16 +02:00
Luca Deri 406ac81a37 Compilation fixes for old ggc's 2022-05-29 16:40:16 +02:00
Luca Deri dfbe863f4e Comment 2022-05-29 16:35:08 +02:00
Luca Deri 1093aafa5f Added detection for WordPress exploits
Fixed ndpi_iph_is_valid_and_not_fragmented() that was bugged with non UDP traffic
2022-05-29 16:31:26 +02:00
Toni 6b7b23b01d Use Doxygen to generate the API documentation. (#1558)
* Integrated Doxygen documentation into Sphinx

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-29 13:44:52 +02:00
Toni 33f9729ee4 Added MPEG-DASH dissector. Fixes #1223. (#1555)
* Improved HTTP POST detection
 * Refactored subprotocol detection

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-29 13:12:13 +02:00
Toni cfd23e8819 Fixed HTTP lower/upper protocol mess for Aimini/IPP. (#1557)
* If HTTP was already detected, there is no need to overwrite the detection again

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-28 16:31:35 +02:00
Luca Deri 77dc884a4c Compilation fixes for old gcc compiler 2022-05-28 16:06:34 +02:00
Luca Deri 7e3d061428 Compilation fixes 2022-05-27 17:35:30 +02:00
Luca Deri 6d543808ad Version cut fix 2022-05-27 16:58:39 +02:00
Luca Deri 02ba12b0de Fixes compilation issues on RedHat systems 2022-05-26 23:34:34 +02:00
Ivan Nardi f81f0c5e69 Sync unit test results (#1554) 2022-05-26 20:15:30 +02:00
Luca Deri cedbd32030 Updated SkypeCall -> Skype_TeamsCall 2022-05-26 19:27:12 +02:00
Luca Deri 1645fea85c Fixed false positives with NATS 2022-05-26 19:01:13 +02:00
Toni 78dfe959cb Added script to compare and verify the output of `make dist'. (#1551)
This fixes some build/test issues resulting when using tarballs.

 * nDPI uses autotools (especially autoconf) in a wrong way, see #1163

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-26 15:07:22 +02:00
Toni 9614b66b44 Replaced obsolete autoconf macros. (#1553)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-26 11:31:09 +02:00
Toni 7010d17ccc Fixed windows-latest build error. (#1552)
* The warning itself looks like a bug

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-26 10:19:57 +02:00
Luca Deri 83e1ea2506 Improved invalid host detection 2022-05-24 21:17:33 +02:00
Luca Deri 0c703ef8eb Added invalid SNI check in QUIC 2022-05-24 19:51:49 +02:00
Luca Deri 4f9dee164e Improved detection of invalid SNI and hostnames in TLS, HTTP 2022-05-24 19:47:54 +02:00
Luca Deri 2560260a41 Added room for storing information used by custom third-party dissectors 2022-05-19 23:24:42 +02:00
lns 8b750686c9 Moved RTSP http patterns to the protocol source file.
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-19 19:58:11 +02:00
Toni Uhlig dae7986a37 Yet another approach to fix #1499 (basically a copy&pasta from @socketpair).
* Related to #1545, #1494 and #1189 as well

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-05-16 16:54:18 +02:00
Toni Uhlig 3c08a3c875 Removed MacOS XCode integration.
* It is outdated (from 2018) and will most likely not work anymore
   due to changed protocol files and outdated API usage in the example integration
 * Removal is the only option besides fixing those issues and
   provide a CI integration for future automatic checks

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-05-16 16:54:18 +02:00
Toni Uhlig 7162bf6abf Moved mgcp.pcapng to tests/pcap/ instead of tests/
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-05-16 16:54:18 +02:00
Ivan Nardi f822aad8b0 DNS-over-QUIC: update default port (#1548)
Final RFC (https://www.rfc-editor.org/rfc/rfc9250.txt) changed the
default port to 853 (from the 784 used on the early drafts).
Since there is at least one deployed implementation still stuck at
draft-02, keep also the old value, for the time being.
2022-05-12 14:31:26 +02:00
Toni 054d151373 Improved Viber (TCP) detection. (#1547)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-10 21:37:03 +02:00
Toni 704920414a Improved Xiaomi HTTP detection. (#1546)
* Merged Xiaomi pcap files

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-10 11:10:04 +02:00
Toni 6fc29b3ae8 Removed TLS patterns in the CiscoVPN aka Anyconnect dissector as mentioned in PR #1534. (#1543)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-09 10:29:52 +02:00
Toni 915ffebade Added Softether(-VPN) DDNS service detection. (#1544)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-09 08:16:19 +02:00
Toni 4319d760e0 Improved TLS alert detection. (#1542)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-08 20:33:59 +02:00
Toni 34882d9cf0 Improved TLS application data detection. (#1541)
* #1532 did fx TLS appdata detection only partially
 * use flow->l4.tcp.tls.message.buffer_used instead of packet->payload

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-08 19:56:08 +02:00
Toni b2648a45a3 Added Edgecast and Cachefly CDNs. (#1540)
* Improved ASN update script
 * Ran `utils/update_every_lists.sh'
 * `tests/do.sh.in' prints the amount of failed pcap(s)
 * `utils/asn_update.sh' prints the amount of failed download(s)

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-07 09:33:25 +02:00
Toni 87f93ea4fd Replaced ndpiReader's libjson-c support with libnDPI's internal serialization interface. (#1535)
* Fixes #1528
 * Serialization Interface should also fuzzed
 * libjson-c may only be used in the unit test to verify the internal serialization interface
 * Serialization Interface supports tlv(broken), csv and json
 * Unit test does work again and requires libjson-c

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-07 09:26:09 +02:00
Ivan Nardi 2e0dedbaae Fix compilation (if --enable-debug-messages is used) (#1539) 2022-05-04 18:43:37 +02:00
Luca Deri 434c38c74d Added extra check to make sure that the guessed protocol is the one we expect and not another one 2022-05-04 16:27:36 +02:00
Luca Deri 5996cf1725 Fixes bug that prevents triggering alerts for traffic on non-standard ports that have been defined in the custom protocols file
Fixes https://github.com/ntop/ntopng/issues/6458
2022-05-04 16:07:05 +02:00
Luca Deri 81a367f1ae Fixes outdated description 2022-05-04 16:06:50 +02:00
Luca Deri 5dcb9c1c16 Modified risk labels 2022-05-04 12:10:01 +02:00
Toni d5b922cfe4 Added some Pluralsight Hostnames/SNIs. May fix #1501. (#1538)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-03 13:20:44 +02:00
Luca Deri 65dd052a61 Updated RRD dependencies 2022-05-03 12:37:22 +02:00
Toni 47d6a65522 Improved suspicious http user agent detection. (#1537)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-02 19:17:32 +02:00
Luca Deri c8f72ef764 Added ndpi_get_flow_error_code() API call
Fixed typo
2022-05-02 10:51:08 +02:00
Toni 02d0b5fe13 Improved AES-NI check. (#1536)
* A library should not open a subshell

Signed-off-by: lns <matzeton@googlemail.com>
2022-04-30 20:46:20 +02:00
Luca Deri e4318ffc2d Improved AES-NI check on Linux to avoid crashes on CPUs that do
not support it (e.g. Intel Celeron N2930)
2022-04-29 14:50:21 +02:00
Ivan Nardi 71636dcafd Sync unit tests results (#1533) 2022-04-27 18:22:11 +02:00
Toni 10161448bc Improved TLS application data detection. (#1532)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-27 17:22:53 +02:00
Luca Deri 3ad989f6a8 Added BPF filtering for discarding non-IP packets 2022-04-27 17:05:33 +02:00
Luca Deri 3d755fce73 String messages have been shrinked 2022-04-27 10:28:06 +02:00
Luca Deri a7c1152397 Added ability to store custom category file in patricia tree 2022-04-26 14:42:31 +02:00
Alfredo Cardigliano e8d81123cc Add ndpi_json_string_escape to the API 2022-04-26 12:25:34 +02:00
Ivan Nardi d5005f584c Raknet: fix heap-buffer-overflow (#1531)
```
==120637==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x606000000488 at pc 0x55a0598d97ec bp 0x7ffcfe13f2e0 sp 0x7ffcfe13f2d8
READ of size 2 at 0x606000000488 thread T0
    #0 0x55a0598d97eb in ndpi_search_raknet /home/ivan/svnrepos/nDPI/src/lib/protocols/raknet.c:152:38
    #1 0x55a05966c48e in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5060:6
    #2 0x55a05966e428 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5119:10
    #3 0x55a05966dddc in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5152:12
    #4 0x55a05967fa7a in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5971:15
    #5 0x55a05962b860 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:24:3
    #6 0x55a05962bd9b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:84:17
    #7 0x7f4aad7d80b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
    #8 0x55a05956b46d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x5d746d) (BuildId: 9429d3d08edc3836e5536f93c07f140716d8b82e)

0x606000000488 is located 9 bytes to the right of 63-byte region [0x606000000440,0x60600000047f)
allocated by thread T0 here:
    #0 0x55a0595ee80e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x65a80e) (BuildId: 9429d3d08edc3836e5536f93c07f140716d8b82e)
    #1 0x55a05962bd06 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:70:17
    #2 0x7f4aad7d80b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47000
2022-04-25 19:01:11 +02:00
Toni bc2ad3407a Added generic user agent setter. (#1530)
* ndpiReader: Print user agent if one was set and not just for certain protocols.

Signed-off-by: lns <matzeton@googlemail.com>
2022-04-25 13:00:50 +02:00
Ivan Nardi 075bce5f3d XIAOMI: add detection of Xiaomi traffic (#1529)
Most of the credits should go to @utoni (see #1521)
2022-04-25 11:00:02 +02:00
Toni ac0f50b561 Added RakNet protocol dissector. (#1527)
* Frame Set PDU's do not get fully dissected for the sake of simplicity

Signed-off-by: lns <matzeton@googlemail.com>
2022-04-24 13:08:57 +02:00
Luca Deri 9d31b7450b Code cleanup (removed redundancy) 2022-04-22 19:03:50 +02:00
Toni d5ae9170ba Tiny gnutella improvement if gtk-gnutella used. (#1525)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-22 18:33:27 +02:00
Toni 824c343602 Updated `utils/whatsapp_ip_addresses_download.sh' to scrape the required IP addresses/ranges. (#1524)
* Replaced return statements in `utils/*.sh' with exit's (such scripts should never source'd)
 * Ran `utils/update_every_lists.sh'

Signed-off-by: lns <matzeton@googlemail.com>
2022-04-22 13:27:41 +02:00
Ivan Nardi 42909673ce Add some scripts to easily update some IPs lists (#1522)
Follow-up of 8b062295

Add a new protocol id for generic Tencent/Wechat flows
2022-04-21 20:43:52 +02:00
Toni ddc256b09e Reduce ndpiReader's -h' spam. -H' does this job now. (#1523)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-21 20:43:36 +02:00
Toni c1d46ebc4e Added proprietary Agora Software Defined Real-time Network (SD-RTN) protocol dissector. (#1520)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-20 18:31:28 +02:00
Toni dbf2fb8c9f Added Toca Boca protocol dissector. (#1517)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-19 19:32:49 +02:00
Toni a83ca555fb Removed superfluous ifdef'd includes. (#1519)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-19 17:51:48 +02:00
Toni fa79f07d15 Improved sflow protocol detection false-positives. (#1518)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-19 17:46:40 +02:00
Ivan Nardi 739dfc54b0 Kerberos: fix Undefined-shift error (#1516)
Similar to the error fixed in 4775be3d
Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=46713
2022-04-18 20:54:13 +02:00
Luca Deri 8b2c9860be DGA improvements 2022-04-16 09:56:08 +02:00
Zied Aouini 9c8a3f270a Minor fix. 2022-04-15 18:01:34 +02:00
Zied Aouini fba75a3cf5 Merge pull request #1491 from utoni/fix/windows-msys2
Fixed msys2 build and re-activated CI Mingw-w64 build.
2022-04-15 13:22:25 +02:00
Toni Uhlig c3df3a12aa Fixed msys2 build warnings and re-activated CI Mingw64 build.
* Removed Visual Studio leftovers. Maintaining an autotools project with VS integration requires some additional overhead.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-14 19:17:48 +02:00
Ivan Nardi 4775be3d85 Kerberos: fix some memory access errors (#1514)
```
==19724==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60e00000045e at pc 0x5620b8b3d3cc bp 0x7ffe0fda6b50 sp 0x7ffe0fda6310
READ of size 2 at 0x60e00000045e thread T0
    #0 0x5620b8b3d3cb in __interceptor_strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x63f3cb) (BuildId: ee53ff920c8cd4c226d8520a0d4846d8864726b6)
    #1 0x5620b8d9b69c in strncpy_lower /home/ivan/svnrepos/nDPI/src/lib/protocols/kerberos.c:208:4
    #2 0x5620b8d995a0 in krb_parse /home/ivan/svnrepos/nDPI/src/lib/protocols/kerberos.c:316:5
    #3 0x5620b8d97a90 in ndpi_search_kerberos /home/ivan/svnrepos/nDPI/src/lib/protocols/kerberos.c:687:12
    #4 0x5620b8bcef35 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4996:4
    #5 0x5620b8bd1be8 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5072:10
    #6 0x5620b8bd159c in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5105:12
    #7 0x5620b8be323a in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5924:15
    #8 0x5620b8b8f7e0 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:24:3
    #9 0x5620b8b8fd1b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:84:17
    #10 0x7f45b32b90b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
    #11 0x5620b8acf47d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x5d147d) (BuildId: ee53ff920c8cd4c226d8520a0d4846d8864726b6)

0x60e00000045e is located 0 bytes to the right of 158-byte region [0x60e0000003c0,0x60e00000045e)
allocated by thread T0 here:
    #0 0x5620b8b5283e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x65483e) (BuildId: ee53ff920c8cd4c226d8520a0d4846d8864726b6)
    #1 0x5620b8b8fc86 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:70:17
    #2 0x7f45b32b90b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
```

```
protocols/kerberos.c:79:52: runtime error: left shift of 255 by 24 places cannot be represented in type 'int'
```

Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=46670
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=46636
2022-04-13 18:10:56 +02:00
Luca Deri e00997ea7c Extended list of cybersecurity domains 2022-04-13 18:05:29 +02:00
Wayneand林文烽 939572c61e fix(ndpi_main):Fix memory leak about ndpi_str; (#1513)
Co-authored-by: 林文烽 <wenfeng.lin@baishan.com>
2022-04-12 15:34:37 +02:00
Ivan Nardi 06a0abb6fd TINC: fix invalid memory read (#1512)
```
=================================================================
==19324==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600061be96 at pc 0x55b4a4cb4460 bp 0x7ffc7b461a70 sp 0x7ffc7b461a68
READ of size 1 at 0x60600061be96 thread T0
    #0 0x55b4a4cb445f in ndpi_check_tinc /home/ivan/svnrepos/nDPI/src/lib/protocols/tinc.c:105:9
    #1 0x55b4a4cb1888 in ndpi_search_tinc /home/ivan/svnrepos/nDPI/src/lib/protocols/tinc.c:135:5
    #2 0x55b4a4b4a6e1 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5013:6
    #3 0x55b4a4b4c2d4 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5084:12
    #4 0x55b4a4b4bf77 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5103:12
    #5 0x55b4a4b5dcca in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5924:15
    #6 0x55b4a4a87734 in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1519:31
    #7 0x55b4a4a80761 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2093:10
    #8 0x55b4a4a39c8d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #9 0x55b4a4a3a46b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17
    #10 0x7f69c63760b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
    #11 0x55b4a497954d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x61654d) (BuildId: 705ebc5c412d267294a65cb01f03a1f012aeaf20)

0x60600061be96 is located 0 bytes to the right of 54-byte region [0x60600061be60,0x60600061be96)
allocated by thread T0 here:
[...]
```
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=46499
2022-04-10 16:59:34 +02:00
Toni e3faecf9de Improved ASN.1 parsing for Keberos. Fixes #1492. (#1497)
* This is a quick fix, the Kerberos protocol dissector requires some refactoring effort.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-10 10:36:10 +02:00
Vinicius Silva Nogueira 713cdd0b13 QUIC: handle retransmissions and overlapping fragments in reassembler (#1195) (#1498)
* QUIC: handle retransmissions and overlapping fragments in reassembler

* Trigger CI

* minor fix: parentheses

* Changing ndpi_malloc to ndpi_calloc

* fix memory leak

* quic_reasm_buf calloc to malloc

* change order of is_ch_complete && is_reasm_buf_complete call

* is_reasm_buf_complete: added handling for case where frame size is not multiple of 8

* add extra check
2022-04-07 21:18:58 +02:00
aouinizied 9537788ade Fix JSON-C. 2022-04-07 17:59:11 +02:00
aouinizied ff993e2f57 Python bindings fix. 2022-04-07 17:54:41 +02:00
Luca Deri a6e2f4a15a Added ndpi_find_outliers() API call using Z-Score 2022-04-04 10:02:45 +02:00
Luca Deri 19019383f0 Added -z flag 2022-04-03 11:45:23 +02:00
Ivan Nardi e0e231756c ndpiReader: fix compilation (#1510)
Not sure why Windows started complaining... anyway, the fixes has been
taken from https://github.com/ntop/nDPI/pull/1491: credits to @lnslbrty
2022-04-01 22:01:39 +02:00
Luca Deri 8080b3d14f Removed un-necessary guess in mining 2022-04-01 19:23:13 +02:00
Luca Deri 9f3a6f131b update 2022-04-01 19:07:46 +02:00
Luca Deri 32a7d42469 Fixed incompatibilities due to https://github.com/ntop/nDPI/pull/1509 2022-04-01 19:07:00 +02:00
Luca Deri 6eb5bd88a7 DGA improvements 2022-04-01 17:39:29 +02:00
Luca Deri a75d77b536 Waring fixes 2022-04-01 17:35:25 +02:00
Vitaliy Ivanov 61bc9815d5 ndpireader: add json output back. (#1509)
- partial revert of:
    commit 51cfdfb0d8
    Author: Luca Deri <deri@ntop.org>
    Date: Sun Nov 17 17:51:45 2019 +0100
      Removed unused JSON-C code

- Json option is changed from 'j' to 'k' as it's used in the new codebase.
- use HAVE_LIBJSON_C instead of HAVE_JSON_C.
- tabs vs spaces clean ups.

Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com>

 Conflicts:
	example/ndpiReader.c
2022-04-01 11:47:01 +02:00
Luca Deri 8238e68275 Improvements for CUSTOM_NDPI_PROTOCOLS 2022-04-01 09:02:05 +02:00
Luca Deri 1e8cdd893c Moved geneated file to a separate folder 2022-04-01 08:58:53 +02:00
Luca Deri 0ff3e77c42 Improved twitter detection 2022-04-01 08:50:56 +02:00
Luca Deri 6677d692c4 Removed SRV record from suspicious DNS traffic 2022-03-31 11:32:30 +02:00
Luca Deri 98a548c642 Improved DGA detection 2022-03-31 11:29:30 +02:00
Darryl Sokoloski 46dc8b5bda [autoconf] Fixed .git submodule detection test. (#1507)
Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2022-03-31 03:21:51 +02:00
Luca Deri 4e199abd39 Added code for identifiying anomalies with metrics stored in InfluxDB 2022-03-30 22:12:39 +02:00
Ivan Nardi 6bcba118d4 reader_util: add support for userAgent in SSDP (#1502)
Update unit tests results
Follow-up of d668ab4b
2022-03-28 04:25:34 +02:00
Ivan Nardi 6acf7a6abe Add support for Pluralsight site (#1503) 2022-03-27 15:13:12 +02:00
Ivan Nardi ec80f1fb3b Fix CI tests results (#1504)
CI integration is failing since 856d7d2.
2022-03-27 15:12:39 +02:00
Vitaly Lavrov 4896dabb6b Reducing the size of the ndpi_detection_module_struct structure. (#1490)
The ndpi_detection_module_struct structure contains 5 arrays "struct
ndpi_call_function_struct" size 286*144=41 kB size,
which are occupied by a small number of elements.

At the moment we have callback_buffer_size = 172, tcp_with_payload=114,
tcp_no_payload=8, udp=93, other 8.
NDPI_MAX_SUPPORTED_PROTOCOLS = 285.

Size of struct ndpi_detection_module_struct is 253136 bytes.
Size of all structs ndpi_call_function_struct 5*286*144=205920 bytes.
Real use memory size for struct ndpi_call_function_struct is
(173+224)*144=57168 bytes.
2022-03-27 12:00:21 +02:00
Darryl Sokoloski d668ab4bec [SSDP] Extract HTTP user-agent when available. (#1500)
[SSDP] Added capture file with UA header.
[SSDP] Added pcap test output log file.

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2022-03-27 11:10:44 +02:00
Luca Deri 856d7d2916 Improved DGA detection skipping names containign at least 3 consecutive digits in the first word 2022-03-26 09:59:55 +01:00
Ivan Nardi aeb5f1f947 QUIC: add support for version 2 draft 01 (#1493)
Support for v2-00 has been removed (it has never been used in real
networks and it is incompatible with v2-01).

Chrome already supports v2-01 in latest versions in Chrome Beta channel.
2022-03-25 10:16:30 +01:00
Ivan Nardi af1d20bca1 Mining: cleanup registration (#1496)
Use the same pattern of all the other dissectors: one registration and
one callback.
Spotted by @dsokoloski
2022-03-25 10:07:47 +01:00
Vinicius Silva Nogueira 26df1403e6 Trying to improve QUIC reassembler (#1195) (#1489)
* handling QUIC out-of-order fragments

* minor fix

* updated quic_frags_ch_out_of_order_same_packet_craziness.pcapng.out

* quic test: buf_len + last_pos

* QUIC: comment update in __reassemble function and minor change is_ch_complete function
2022-03-24 10:51:31 +01:00
aouinizied bb12837ca7 Update Python bindings guide. 2022-03-22 15:01:55 +01:00
aouinizied c7059db84a Fix typo. 2022-03-22 14:58:22 +01:00
aouinizied 1207ea118a Add HOWTO Python. 2022-03-22 14:56:44 +01:00
aouinizied c47d710d8e Fix python bindings CI. 2022-03-22 13:40:31 +01:00
aouinizied beef4f997b Complete rework of nDPI Python bindings (cffi API, automatic generation, packaging and CI integration) 2022-03-22 13:19:27 +01:00
Luca Deri 93f723d50f Extended the list of cybersecurity protocol 2022-03-21 15:12:50 +01:00
Vitaly Lavrov 8dd70b7023 Bug fixing. (#1487)
Using the protocol_id instead of its index.
2022-03-15 19:40:18 +01:00
Ivan Nardi 9e22c88e0a QUIC: convert logs to standard mechanism (#1485) 2022-03-15 16:55:47 +01:00
Ivan Nardi b83e9f20e7 QUIC: fix dissection of draft-34 (#1484)
QUIC-34 is probably not used in production, but fixing it is trivial and
it doesn't add any noise to the already complex QUIC code.
2022-03-09 22:37:56 +01:00
Ivan Nardi 7aee856aa0 Extend tests coverage (#1476)
Now there is at least one flow under `tests/pcap` for 249 protocols out
of the 284 ones supported by nDPI.

The 35 protocols without any tests are:

* P2P/sharing protocols: DIRECT_DOWNLOAD_LINK, OPENFT, FASTTRACK,
EDONKEY, SOPCAST, THUNDER, APPLEJUICE, DIRECTCONNECT, STEALTHNET

* games: CSGO, HALFLIFE2, ARMAGETRON, CROSSFIRE, DOFUS, FIESTA,
FLORENSIA, GUILDWARS, MAPLESTORY, WORLD_OF_KUNG_FU

* voip/streaming: VHUA, ICECAST, SHOUTCAST, TVUPLAYER, TRUPHONE

* other: AYIYA, SOAP, TARGUS_GETDATA, RPC, ZMQ, REDIS, VMWARE, NOE,
LOTUS_NOTES, EGP, SAP

Most of these protocols (expecially the P2P and games ones) have been
inherited by OpenDPI and have not been updated since then: even if they
are still used, the detection rules might be outdated.

However code coverage (of `lib/protocols`) only increases from 65.6% to
68.9%.

Improve Citrix, Corba, Fix, Aimini, Megaco, PPStream, SNMP and Some/IP
dissection.
Treat IPP as a HTTP sub protocol.
Fix Cassandra false positives.

Remove `NDPI_PROTOCOL_QQLIVE` and `NDPI_PROTOCOL_REMOTE_SCAN`:
these protocol ids are defined but they are never used.

Remove Collectd support: its code has never been called. If someone is
really interested in this protocol, we can re-add it later, updating the
dissector.

Add decoding of PPI (Per-Packet Information) data link type.
2022-03-09 22:37:35 +01:00
Toni f646a4bce0 Improved ASN/IP update scripts and CI integration. (#1474)
* CI will print a warning if ASN/IP addresses changed.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-09 13:53:04 +01:00
Zied Aouini 74ae315e36 Implement CI on Windows. (#1483)
* Switch fail fast to True.
* Windows CI.
2022-03-09 10:57:21 +01:00
Ivan Nardi 269be6c7ef Some small fixes (#1481)
FTP: if the authentication fails, stop analyzing the flow
WSD: call the initialization routine; the dissector code has never been
triggered
MINING: fix dissection
2022-03-08 00:21:51 +01:00
sharonenochandSharon Enoch f91218360b Extracting the Azure Origin url from the download link (#1480)
* Extracting the Azure Origin url from the download link

* Response code check to address double quotes

Co-authored-by: Sharon Enoch <sharone@amzetta.com>
2022-03-08 00:21:32 +01:00
Vitaly Lavrov a1451935b8 Errors fixed (#1482)
Fixed errors for bigendian platforms in ndpiReader.
All address and port comparisons and hash calculations are done with
endian in mind.
The get_ndpi_flow_info() function searched for an existing flow for the
forward and reverse direction of the packet.
The ndpi_workflow_node_cmp() function looked for a flow regardless of
the packet's direction. This is what led to an error in determining the
direction of transmission of the packet.

Fixed error in "synscan" test: the number of packets in the forward and
reverse direction is incorrectly defined (verified via tcpdump).

Fixed bug with icmp protocol checksum check for big endian platforms.
2022-03-08 00:20:56 +01:00
Ivan Nardi c345b3c7af EthernetIP: fix integer conversion on big-endian archs (#1477) 2022-03-05 15:05:05 +01:00
Vitaly Lavrov 438f04066b Fixed a bug for BE architectures (#1478)
Fixed a bug in the internal implementation of libgcrypt for bigendian architectures
2022-03-05 11:44:21 +01:00
Ivan Nardi a173d3e98f configure: fix usage of libgpg-error with --with-local-libgcrypt (#1472)
Right now, using external libgcrypt, nDPI is not linked to libgpg-error
because configure script never checks for it.

```
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ CC=gcc-11 CXX=g++-11 CFLAGS="-O3 -g -Werror" ./autogen.sh --enable-debug-messages --with-pcre  --with-local-libgcrypt && make -s -j
[...]
checking for numa_available in -lnuma... yes
checking for pcap_open_live in -lpcap... yes
checking for pthread_setaffinity_np in -lpthread... yes
checking for gcry_cipher_checktag in -lgcrypt... yes             <------- missing check for libgpg-error
checking for pcre_compile in -lpcre... yes
checking that generated files are newer than configure... done
[...]
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ grep HAVE_LIBGPG_ERROR src/include/ndpi_config.h
/* #undef HAVE_LIBGPG_ERROR */

```

Make both libgcrypt and libgpg-error mandatory if
`--with-local-libgcrypt` is used.

Technically speaking, libgpg-error might be optional, because it is used
only for debug messages. However having both libraries mandatory
slightly simplified the logic.
In most environments, libgpg-error is a dependency of libgcrypt anyway,
so having both libraries should be the standard case.
2022-03-05 11:41:37 +01:00
Toni 95a3d4fffe Added autoconf option `--enable-tls-sigs'. (#1471)
* Testing more code in CI environments.
 * Added strict option checking for `./configure' in CI environments.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-03 10:16:08 +01:00
Toni cb62dfd249 Drop support for non-gcrypt builds. (#1469)
* As there is now a builtin, lightweight libgcrypt
   there is no need to disable tls-clho decryption.
 * It is still possible to use a host libgcrypt
   with `--with-local-libgcrypt'.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-02 19:48:46 +01:00
Ivan Nardi e23507d3e9 Internal crypto: increase size of authentication buffer (#1468)
Some QUIC flows are not properly decoded while using internal crypto
code: the authentication buffer is too small.
The new value (like the old one) is arbitrary.

Close #1463
2022-03-02 15:14:57 +01:00
Ivan Nardi 6f431057c5 reader_util: fix parsing of MPLS packets (#1467)
```
==20492==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60300000578c at pc 0x55c47455e3ea bp 0x7ffc62ca1eb0 sp 0x7ffc62ca1ea8
READ of size 4 at 0x60300000578c thread T0
    #0 0x55c47455e3e9 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:1840:16
    #1 0x55c47451b9cd in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #2 0x55c47451c1ab in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17
    #3 0x7f661b50e0b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
    #4 0x55c47445b54d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x61054d) (BuildId: eba4bb4cd43b7101e4f0028ec0fb79087bae0e37)

0x60300000578d is located 0 bytes to the right of 29-byte region [0x603000005770,0x60300000578d)

```
2022-03-02 15:14:41 +01:00
Toni e8559a4127 Add ICMP checksum check and set risk if mismatch detected. (#1464)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-02 13:12:01 +01:00
Luca Deri 6c4df21238 Typo 2022-03-02 12:47:04 +01:00
Toni 4da1453d90 Added configureable ndpi packet processing limit. (#1466)
* The current behaviour ignores any user preferences
   and was also incorrectly implemented, because the
   flow->num_processed_pkts wraps every 65535 and nDPI
   will process packets again until
   NDPI_MAX_NUM_PKTS_PER_FLOW_TO_DISSECT reached.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-02 12:02:09 +01:00
Toni 736a80f9ce Fix libgcrypt(-light/-internal) compile error. (#1465)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-02 11:03:47 +01:00
Ivan Nardi 7a7e4ee69f Add a new flow risk NDPI_ANONYMOUS_SUBSCRIBER (#1462)
The main goal of a DPI engine is usually to determine "what", i.e. which
types of traffic flow on the network.
However the applications using DPI are often interested also in "who",
i.e. which "user/subscriber" generated that traffic.

The association between a flow and a subscriber is usually done via some
kind of DHCP/GTP/RADIUS/NAT mappings. In all these cases the key element
of the flow used to identify the user is the source ip address.

That usually happens for the vast majority of the traffic.

However, depending on the protocols involved and on the position on the net
where the traffic is captured, the source ip address might have been
changed/anonymized. In that case, that address is useless for any
flow-username association.

Example: iCloud Private Relay traffic captured between the exit relay and
the server.
See the picture at page 5 on:
https://www.apple.com/privacy/docs/iCloud_Private_Relay_Overview_Dec2021.PDF

This commit adds new generic flow risk `NDPI_ANONYMOUS_SUBSCRIBER` hinting
that the ip addresses shouldn't be used to identify the user associated
with the flow.
As a first example of this new feature, the entire list of the relay ip
addresses used by Private Relay is added.

A key point to note is that list is NOT used for flow classification
(unlike all the other ip lists present in nDPI) but only for setting this
new flow risk.

TODO: IPv6
2022-02-28 15:25:58 +01:00
Luca Deri 1a8a6aa161 Unless --with-libgcrypt is used, nDPI now uses its internal gcrypt implementation 2022-02-28 15:18:10 +01:00
Ivan Nardi e193f3713d Removed some unused fields (#1461) 2022-02-28 15:03:49 +01:00
Vitaly Lavrov c390085f91 Bug fixing. (#1459)
The '--enable-debug-messages' option works again.
Fixed warning in ahocorasick.c
Fixed integer overflow in ndpiReader.c for 32bit systems.
2022-02-28 15:01:00 +01:00
Toni 00d9a16b6a Added `--enable-code-coverage' build using lcov for coverage generation. (#1430)
* CI job generates a coverage report

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-02-28 14:58:45 +01:00
Ivan Nardi 61a3c2eb5b reader_util: fix TZSP decapsulation (#1460)
```
==38674==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60400008957f at pc 0x5653fcee6434 bp 0x7ffe9b554b50 sp 0x7ffe9b554b48
READ of size 1 at 0x60400008957f thread T0
    #0 0x5653fcee6433 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2050:18
    #1 0x5653fce9fc0d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #2 0x5653fcea03eb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17
    #3 0x7fe71dc3d0b2 in __libc_start_main /build/glibc-YbNSs7/glibc-2.31/csu/../csu/libc-start.c:308:16
    #4 0x5653fcddf67d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x58a67d) (BuildId: 525418a27e8c37d6c492cc3220e0e97809c40f98)

0x60400008957f is located 0 bytes to the right of 47-byte region [0x604000089550,0x60400008957f)
allocated by thread T0 here:
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=45036
2022-02-27 14:18:23 +01:00
Ivan Nardi 8b062295cc Add some scripts to easily update some IPs lists (#1449)
While the lists in a6ff0dd0 and 2f5f445f are somehow provided by the
companies themselves (or by some interested parties), these new lists
are directly extracted from BGP information, via AS prefixes.

*Usually*, these new lists are far more stable than the previous ones.

TODO:
* add some other ASNs (see `src/lib/ndpi_content_match.c.inc`)
* IPv6, as usual :-(
2022-02-25 14:26:26 +01:00
Toni f14dcb8344 Provide some API functions for convenience. (#1456)
* Extended JSON serializsation: risk, risk score, confidence

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-02-25 14:26:04 +01:00
Luca Deri 34e020ac35 Win fixes 2022-02-23 21:00:53 +01:00
Luca Deri 16ed0fd3f8 Replaced strdup with ndpi_strup 2022-02-23 21:37:52 +01:00
Ivan Nardi d506a0e12f Directly drop malformed packets (#1455)
TCP/UDP/ICMP/ICMPv6 packets with invalid L4 header length should be
ignored.
2022-02-22 22:15:46 +01:00
Ivan Nardi 96f8942f75 reader_util: fix parsing of IPv6 extension headers (#1453)
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44942
2022-02-22 19:58:17 +01:00
Ivan Nardi 279779062c reader_util: fix infinite loop in packet dissection (#1454)
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44944
2022-02-22 19:56:41 +01:00
Ivan Nardi fbb9700086 fuzz: purge old sessions (#1451)
At every fuzz iteration (i.e for every trace file):
* keep the same ndpi context (`ndpi_init_detection_module` is very
slow);
* reset the flow table, otherwise it grows indefinitely.

This change should fix the "out-of-memory" errors reported by oss-fuzz.
2022-02-21 20:32:50 +01:00
Ivan Nardi 6c1accd2bd DTLS: fix access to certificate cache (#1450)
```
protocols/tls.c:650:54: runtime error: member access within null pointer of type 'const struct ndpi_tcphdr'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in
protocols/tls.c:650:54: runtime error: load of null pointer of type 'const u_int16_t' (aka 'const unsigned short')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in
AddressSanitizer:DEADLYSIGNAL
=================================================================
==47401==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x55f7a61b661c bp 0x7f38190f91b0 sp 0x7f38190f70e0 T1)
==47401==The signal is caused by a READ memory access.
==47401==Hint: address points to the zero page.
    #0 0x55f7a61b661c in processCertificateElements /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:650:41
    #1 0x55f7a61ac3cc in processCertificate /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:792:7
    #2 0x55f7a61d34e1 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:846:13
```
2022-02-21 20:30:22 +01:00
Ivan Nardi 59993e294e EthernetIP: add missing initialization (#1448)
Fix:1e1cfb89
2022-02-20 10:20:46 +01:00
Ivan Nardi 2f5f445f72 Add support for Google Cloud (#1447)
Differentiate between Google its own apps/services and Google Cloud.
We already do something similar for Amazon vs AWS and Microsoft vs Azure.
2022-02-20 10:19:52 +01:00
Ivan Nardi 75a53ad064 fuzz: make fuzz_ndpi_reader faster (#1446)
Initialize ndpi_workflow_init context only once.

On a quite old notebook, before:
```
$ ./fuzz/fuzz_ndpi_reader -max_total_time="${MAX_TOTAL_TIME:-360}" -print_pcs=1 -workers="${FUZZY_WORKERS:-0}" -jobs="${FUZZY_JOBS:-0}" ./tests/pcap/
[...]
Done 3256 runs in 361 second(s)
```
after:
```
Done 5032771 runs in 361 second(s)   <----------- ~1400X
```
oss-fuzz will be happy!
2022-02-20 10:19:22 +01:00
Vitaly Lavrov f2411958d7 Added lightweight implementation of libgcrypt. (#1444)
Implementation borrowed from the
https://github.com/ARMmbed/mbedtls.git project (v3.1.0)

Speed testing (Xeon(R) CPU E3-1230 V2 @ 3.30GHz):

gcrypt-gnu         Test md   2897 ms enc   2777 ms dec    942 ms
gcrypt-int         Test md   3668 ms enc   1312 ms dec   2836 ms
gcrypt-int-noaesni Test md   3652 ms enc   1916 ms dec   4458 ms

gcrypt-gnu-nonopt  Test md   3763 ms enc   4978 ms dec   3999 ms

gcrypt-gnu-nonopt - libgcrypt compiled without hardware acceleration
  --disable-padlock-support --disable-aesni-support \
  --disable-shaext-support --disable-pclmul-support \
  --disable-sse41-support --disable-drng-support \
  --disable-avx-support --disable-avx2-support \
  --disable-neon-support --disable-arm-crypto-support \
  --disable-ppc-crypto-support
  --disable-amd64-as-feature-detection
2022-02-20 10:16:46 +01:00
Ivan Nardi f28a3b293e Fix compilation and sync unit tests results (#1445)
'strcasestr' is not defined in all enviroments: quicker fix is to use
'ndpi_strncasestr' instead.
2022-02-19 19:18:02 +01:00
Luca Deri a2878af1ee Added newflow risk NDPI_HTTP_CRAWLER_BOT 2022-02-17 17:20:52 +01:00
Luca Deri 8a2a47e62a Silenced
NDPI_SUSPICIOUS_DGA_DOMAIN,
   NDPI_BINARY_APPLICATION_TRANSFER,
   NDPI_HTTP_NUMERIC_IP_HOST,
   NDPI_MALICIOUS_JA3,

for predefined connectivity check and cybersec categories
2022-02-14 23:38:21 +01:00
Luca Deri 92da30f017 Extended cybersecurity protocol dissection 2022-02-14 23:19:21 +01:00
Luca Deri c4113ffd7e Added SNMP error code check 2022-02-14 23:17:51 +01:00
Luca Deri de6905f41b Exteended cybersecurity list 2022-02-10 20:16:48 +01:00
Luca Deri 023618c13a Invalid prototupe fix 2022-02-10 19:38:11 +01:00
Ivan Nardi ada7d32c50 HSRP: fix dissection over IPv6 (#1443)
Handle all message types.
2022-02-10 11:46:32 +01:00
Luca Deri 1431aa3d1c Added cybersecurity category mapping to string 2022-02-10 10:35:34 +01:00
Luca Deri 5ddbbf4dd2 Added cybersecurity protocol and category that groups traffic towards leading cybersecurity companies and CDNs, useful to make destinations that should be marked as trusted in firewalls and security gateways 2022-02-10 09:50:06 +01:00
Ivan Nardi b5eea436ae HSRP: add support for IPv6 (#1440) 2022-02-09 11:47:37 +01:00
Dmytrii Vitman 4cf853548c Added VXLAN dissector (#1439)
* RFC 7348
2022-02-09 11:47:09 +01:00
Ivan Nardi f229068fd3 Fix memory access in ndpi_strnstr() (#1438)
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44482

It should be the same error reported (and only partially fixed) in
79968f32
2022-02-09 11:46:39 +01:00
Ivan NardiandLuca Deri a6ff0dd0e3 Add few scripts to easily update some IPs lists (#1436)
* Add few scripts to easily update some IPs lists

Some IPs lists should be updated frequently: try to easy the process.
The basic idea is taken from d59fefd0 and a8fe74e5 (for Azure
addresses): one specific .c.inc file and one script for each protocol.

Add the possibility to don't load a specific list.

Rename the old NDPI_PROTOCOL_HOTMAIL id to NDPI_PROTOCOL_MS_OUTLOOK,
to identify Hotmail/Outlook/Exchange flows.

TODO: ipv6

Remove the 9 addresses associated to BitTorrent: they have been added in
e2f21116 but it is not clear why all the traffic to/from these ips
should be classified as BitTorrent.

* Added quotes

* Added quotes

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2022-02-09 11:45:48 +01:00
ol-andreyizrailevandAndrey Izrailev 8cc5cb9f76 Increment current/total number of active flows on successful flow insertion (#1434)
Memory allocation or ndpi_tsearch might fail, so the two values should be
incremented only when insertion actually happened.

Co-authored-by: Andrey Izrailev <Andrey.Izrailev@oktetlabs.ru>
2022-02-09 11:43:10 +01:00
Luca Deri 9838ceb7cd Added ndpi_serialize_string_string_len() APi call
Fixed CSV string serialization
2022-02-08 22:17:26 +01:00
Luca Deri be5a29ba9d Added HSRP protocol detection
Removed attic directory now obsolete
2022-02-08 18:04:57 +01:00
Luca Deri 922a7be897 Added check to ignore multicast packets marking the as Skype 2022-02-08 17:23:25 +01:00
Toni d59f0327a5 Improved MDNS/LLMNR detection. (#1437)
* Checking for port 5353/5355 is not enough.
 * Added additional multicast address and header checks.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-02-07 18:05:23 +01:00
Ivan Nardi ff38d4b980 TLS: fix parsing of certificate elements (#1435)
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44280
```
==263603==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x592478 in ndpi_is_printable_string ndpi/src/lib/ndpi_utils.c:2200:9
    #1 0x5b047c in processCertificateElements ndpi/src/lib/protocols/tls.c:400:7
    #2 0x5ac880 in processCertificate ndpi/src/lib/protocols/tls.c:790:7
    #3 0x5c3a32 in processTLSBlock ndpi/src/lib/protocols/tls.c:844:13
    #4 0x5c2c61 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:973:2
    #5 0x5c117d in ndpi_search_tls_wrapper ndpi/src/lib/protocols/tls.c:2367:5
    #6 0x552a50 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4792:6
```
2022-02-07 18:05:12 +01:00
Ivan Nardi 79968f3245 Sync utests (#1433)
* Sync utest results

* Fix read-heap-buffer-overflow error reported by CI

See: https://github.com/ntop/nDPI/runs/5055876515?check_suite_focus=true
2022-02-04 15:57:17 +01:00
Alfredo Cardigliano f5035521e8 Add comment 2022-02-03 18:43:31 +01:00
Luca Deri c40496eaac Updated test results 2022-02-03 13:54:38 +01:00
Luca Deri 7aef27f85e Added NDPI_ERROR_CODE_DETECTED risk 2022-02-03 13:20:54 +01:00
Luca Deri 0dd5ac89f8 Renamed DCERPC to more generic RPC protocol so we can use also for other types of RPCs (not limited to DCE)
Extended HTTP plugin to support RPC
Improved HTTP crear text detection to limit it to Basic and Digest
2022-02-03 13:06:27 +01:00
Luca Deri 6975f265ec Typo 2022-02-03 12:21:51 +01:00
Luca Deri ae09e7fb14 Improved risks description 2022-02-03 12:19:01 +01:00
Luca Deri a96d7eb180 Updated risk documentation 2022-02-03 12:10:18 +01:00
Luca 37ff626e78 Added new IDN/Punycode risk for spotting internationalized domain names 2022-02-03 09:17:54 +01:00
Luca Deri cd3d720ae3 Added missing __sync_fetch_and_add() definition in Windows 2022-02-02 10:36:50 +01:00
Luca 46bd093934 Moved to 4.3 2022-02-01 09:13:55 +01:00
3599 changed files with 373701 additions and 104895 deletions
+6
View File
@@ -0,0 +1,6 @@
mpile
# Codige Code Review ignore
# Prevent Codiga complaining about Sphinx config.py
doc/conf.py
+1 -1
View File
@@ -21,7 +21,7 @@ A clear and concise description of what happening.
* OS version: [e.g. 18.04]
* Architecture: [e.g. arm64]
* nDPI version or commit hash: [e.g. 4.0-stable or 937357e4bc55610f116f66d15a8e0fc1e260c02c].
* nDPI compilation flags used: if you are building from source [e.g. --with-pcre --disable-gcrypt].
* nDPI compilation flags used: if you are building from source [e.g. --with-pcre2 --with-local-libgcrypt].
* Attach the `config.log` file generated after `./configure` ran (if you are building from source).
## How to reproduce the reported bug
+12
View File
@@ -0,0 +1,12 @@
Please sign (check) the below before submitting the Pull Request:
- [ ] I have signed the ntop Contributor License Agreement at https://github.com/ntop/legal/blob/main/individual-contributor-licence-agreement.md
- [ ] I have read the contributing guide lines at https://github.com/ntop/nDPI/blob/dev/CONTRIBUTING.md
- [ ] I have updated the documentation (in doc/) to reflect the changes made (if applicable)
Link to the related [issue](https://github.com/ntop/nDPI/issues):
Describe changes:
+43
View File
@@ -0,0 +1,43 @@
name: MSBuild
on:
push:
branches:
- dev
pull_request:
branches:
- dev
types: [opened, synchronize, reopened]
release:
types: [created]
jobs:
test:
name: MSBuild
runs-on: windows-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Add msbuild to PATH
uses: microsoft/setup-msbuild@v2
- name: Download NuGet packages
run: msbuild windows/nDPI.sln -t:restore -p:RestorePackagesConfig=true
- name: Build libnDPI only (debug)
run: msbuild windows/nDPI.sln -t:rebuild -property:Configuration=Debug
- name: Download Npcap
run: Invoke-WebRequest -Uri 'https://npcap.com/dist/npcap-1.71.exe' -OutFile 'C:\npcap-1.71.exe'
- name: Install Npcap
run: C:\npcap-1.71.exe /S /winpcap_mode=yes
- name: Build ndpiReader (debug)
run: msbuild windows/nDPI.sln -t:rebuild -property:Configuration=Debug-ndpiReader
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: build-artifacts
path: windows/bin
retention-days: 14
+31
View File
@@ -0,0 +1,31 @@
name: RPM Build
on:
workflow_dispatch:
schedule:
# At the end of every day
- cron: '0 0 * * *'
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
- name: Configure nDPI
run: |
# fake CentOS 7 env
echo 'CentOS Linux release 7.9.2009 (Core)' | sudo tee /etc/centos-release
# symlink source directory for ndpi.spec
ln -sr . $HOME/nDPI
./autogen.sh
cd packages/rpm
./configure
- name: Build RPM package
run: |
make -C packages/rpm package
+272 -111
View File
@@ -10,105 +10,224 @@ on:
release:
types: [created]
jobs:
python-bindings:
name: Python Bindings (ubuntu-latest)
runs-on: ubuntu-latest
env:
CFLAGS: -Wextra -Werror
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python 3.9
uses: actions/setup-python@v5
with:
python-version: "3.9"
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
- name: Build nDPI library
run: |
./autogen.sh
make -j
sudo make install
- name: Generate Python bindings
run: |
pip install --upgrade pip
pip install -r python/requirements.txt
cd python
python setup.py install
cd ..
- name: Test Python Bindings
run: |
cd python
python tests.py
test-scripts:
name: Test Utils (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install python3-netaddr git whois libxml2-utils
- name: Run Scripts
run: |
echo 'Running ./utils/bitcoinnodes.sh'
./utils/bitcoinnodes.sh >/dev/null
echo 'Running ./utils/get_routes_by_asn.sh AS714'
./utils/get_routes_by_asn.sh AS714 >/dev/null
echo 'Running ./utils/update_every_lists.sh'
./utils/update_every_lists.sh
echo 'Checking for changes in the git tree..'
git update-index --refresh || echo "::warning file=utils/update_every_lists.sh::Please re-run utils/update_every_lists.sh and commit any changes."
git diff-index --quiet HEAD -- || true
test:
name: ${{ matrix.os }} ${{ matrix.arch }} ${{ matrix.gcrypt }} ${{ matrix.compiler }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} ${{ matrix.msan }}
name: ${{ matrix.os }} ${{ matrix.arch }} ${{ matrix.gcrypt }} ${{ matrix.compiler }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} ${{ matrix.msan }} ${{ matrix.nBPF }} ${{matrix.lto_gold_linker}} ${{matrix.global_context}}
runs-on: ${{ matrix.os }}
env:
CC: ${{ matrix.compiler }}
CFLAGS: -Wextra -Werror -DNDPI_EXTENDED_SANITY_CHECKS
strategy:
fail-fast: false # Debugging purposes, switch to true when all issues on big endian is solved
fail-fast: true
matrix:
os: ["ubuntu-latest", "ubuntu-18.04", "macOS-latest", "macos-11"]
# macOS-latest == macos-14 on **ARM64**
# There are some issues with external dependencies on macOS-latest. Disable it for the time being
os: ["ubuntu-20.04", "ubuntu-22.04", "ubuntu-24.04", "macOS-12", "macOS-13", "windows-latest"]
arch: ["x86_64"]
gcrypt: ["--disable-gcrypt", ""]
compiler: ["default-cc"]
gcrypt: ["--with-local-libgcrypt", ""]
compiler: ["cc"]
ar: ["ar"]
ranlib: ["ranlib"]
pcre: [""]
maxminddb: [""]
msan: [""]
nBPF: [""]
lto_gold_linker: [""]
global_context: [""] #Enable by default
include:
- compiler: "gcc-7"
- compiler: "gcc-4.9" # "Oldest" gcc easily available. To simulate RHEL7
os: ubuntu-20.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
- compiler: "gcc-14" # "Newest" gcc easily available
os: ubuntu-24.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
lto_gold_linker: "--with-lto-and-gold-linker"
- compiler: "clang-9" # "Oldest" clang easily available
os: ubuntu-20.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
- compiler: "clang-17" # "Newest" clang easily available. See also below...
ar: "llvm-ar-17"
ranlib: "llvm-ranlib-17"
os: ubuntu-24.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
lto_gold_linker: "--with-lto-and-gold-linker"
- compiler: "clang-18" # "The latest clang version easily available should be 18,
# but it is buggy on ubuntu-24.04 with LTO and Gold linker:
# https://github.com/llvm/llvm-project/issues/87553
# https://bugs.launchpad.net/ubuntu/+source/llvm-toolchain-18/+bug/2064187
# Waiting for a fix...
ar: "llvm-ar-18"
ranlib: "llvm-ranlib-18"
os: ubuntu-24.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
lto_gold_linker: ""
- compiler: "cc"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "gcc-10"
msan: "--with-thread-sanitizer"
nBPF: ""
- compiler: "cc"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "clang-7"
nBPF: "nBPF"
- compiler: "cc"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "clang-12"
os: ubuntu-latest
nBPF: ""
global_context: "--disable-global-context-support"
- compiler: "clang" #TODO: some issues with masan/clang/ubuntu-24.04
os: ubuntu-22.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "default-cc"
os: ubuntu-latest
msan: "--with-memory-sanitizer"
nBPF: ""
- compiler: "cc"
os: macOS-13
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "default-cc"
os: ubuntu-18.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "default-cc"
os: macOS-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "" # Disable sanitizer on macos
- compiler: "default-cc"
os: macos-11
nBPF: ""
- compiler: "cc"
os: macos-14
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
pcre: ""
maxminddb: ""
msan: "" # Disable sanitizer on macos
- compiler: "default-cc"
nBPF: ""
- compiler: "cc"
os: ubuntu-latest
arch: "arm64"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "" # Disable sanitizer on arm64
- compiler: "default-cc"
nBPF: ""
- compiler: "cc"
os: ubuntu-latest
arch: "armhf"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "default-cc"
nBPF: ""
- compiler: "cc"
os: ubuntu-latest
arch: "s390x"
gcrypt: ""
pcre: "--with-pcre"
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: ""
nBPF: ""
steps:
- name: Setup multiarch/qemu-user-static
if: startsWith(matrix.os, 'ubuntu') && !startsWith(matrix.arch, 'x86_64')
run: |
docker run --rm --privileged multiarch/qemu-user-static:register --reset
- uses: actions/checkout@v2
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis
- name: Fix kernel mmap rnd bits on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
run: |
# Workaround for compatinility between latest kernel and sanitizer
# See https://github.com/actions/runner-images/issues/9491
sudo sysctl vm.mmap_rnd_bits=28
- name: Install Ubuntu Prerequisites
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
run: |
@@ -116,39 +235,58 @@ jobs:
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
sudo apt-get install gcc-mingw-w64 libc6-dev
sudo apt-get install doxygen python3-sphinx python3-sphinx-rtd-theme python3-breathe python3-pip
sudo apt-get install rrdtool librrd-dev
sudo apt-get install rrdtool librrd-dev parallel
- name: Install Ubuntu Prerequisites (libgcrypt)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.gcrypt, '--disable-gcrypt')
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.gcrypt, '--with-local-libgcrypt')
run: |
sudo apt-get install libgcrypt20-dev
- name: Install Ubuntu Prerequisites (libpcre)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.pcre, '--with-pcre')
- name: Install Ubuntu Prerequisites (libpcre2)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.pcre, '--with-pcre2')
run: |
sudo apt-get install libpcre3-dev
- name: Install Ubuntu Prerequisites (maxminddb)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.maxminddb, '--with-maxminddb')
run: |
sudo apt-get install libmaxminddb-dev
- name: Setup Ubuntu specified compiler (gcc)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'gcc')
- name: Install Ubuntu Prerequisites (nBPF)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.nBPF, 'nBPF')
run: |
sudo apt-get install ${{ matrix.compiler }}
- name: Setup Ubuntu specified compiler (clang)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'clang')
git clone https://github.com/ntop/PF_RING.git ../PF_RING
cd ../PF_RING/userland/nbpf
./configure
make
cd -
- name: Setup Ubuntu specified compiler
if: startsWith(matrix.os, 'ubuntu-20.04') && startsWith(matrix.arch, 'x86_64') && ! startsWith(matrix.compiler, 'cc')
run: |
#For gcc-4.9 (on ubuntu-20.04)
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial main" | sudo tee -a /etc/apt/sources.list
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial universe" | sudo tee -a /etc/apt/sources.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 40976EAF437D05B5
sudo apt-get update
sudo apt-get install ${{ matrix.compiler }}
- name: Install Windows msys2 prerequisites
if: startsWith(matrix.os, 'windows')
uses: msys2/setup-msys2@v2
with:
msystem: MINGW64
update: true
install: git mingw-w64-x86_64-toolchain automake1.16 automake-wrapper autoconf libtool make mingw-w64-x86_64-json-c mingw-w64-x86_64-crt-git mingw-w64-x86_64-pcre mingw-w64-x86_64-libpcap mingw-w64-x86_64-libgcrypt parallel
- name: Installing MacOS prerequisites
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64')
run: |
brew install autoconf automake libtool pkg-config gettext json-c
brew install coreutils
brew install rrdtool
# A workaround for "The `brew link` step did not complete successfully" error.
# See https://github.com/Homebrew/homebrew-core/issues/165793#issuecomment-1991817938
find /usr/local/bin -lname '*/Library/Frameworks/Python.framework/*' -delete
sudo rm -rf /Library/Frameworks/Python.framework/
brew install --force python3 && brew unlink python3 && brew unlink python3 && brew link --overwrite python3
brew install coreutils wdiff colordiff autoconf automake libtool pkg-config gettext json-c rrdtool parallel
- name: Install MacOS Prerequisites (libgcrypt)
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.gcrypt, '--disable-gcrypt')
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.gcrypt, '--with-local-libgcrypt')
run: |
brew install libgcrypt
- name: Install MacOS Prerequisites (libpcre)
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.pcre, '--with-pcre')
- name: Install MacOS Prerequisites (libpcre2)
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.pcre, '--with-pcre2')
run: |
brew install pcre
- name: Install MacOS Prerequisites (maxminddb)
@@ -156,63 +294,88 @@ jobs:
run: |
brew install libmaxminddb
- name: Configure nDPI on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'default-cc')
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
run: |
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }}
AR=${{ matrix.ar }} RANLIB=${{ matrix.ranlib }} ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs ${{matrix.lto_gold_linker}} ${{matrix.global_context}}
- name: Configure nDPI on MacOS
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'default-cc')
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'cc')
run: |
env CC=clang CFLAGS='-Werror' ./autogen.sh --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }}
- name: Configure nDPI with specified GCC version on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'gcc')
./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs
- name: Configure nDPI on Windows msys2
if: startsWith(matrix.os, 'windows') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'cc')
run: |
env CC=${{ matrix.compiler }} CFLAGS='-Werror' ./autogen.sh --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }}
- name: Configure nDPI with specified CLANG on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'clang')
msys2 -c './autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --disable-npcap ${{ matrix.gcrypt }}'
- name: Build nDPI on Windows msys2
if: startsWith(matrix.os, 'windows') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'cc')
run: |
env CC=${{ matrix.compiler }} CFLAGS='-Werror' ./autogen.sh --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }}
msys2 -c 'make -j all'
msys2 -c 'ldd ./example/ndpiReader.exe'
- name: Build nDPI
if: startsWith(matrix.arch, 'x86_64')
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows') && !startsWith(matrix.os, 'macos-14')
run: |
make all
make -j all
make -C example ndpiSimpleIntegration
make -C rrdtool
make -C python
- name: Build nDPI (MacOS M1)
if: startsWith(matrix.os, 'macos-14')
run: |
make -j all
make -C example ndpiSimpleIntegration
#There are somes issues with librrd
#make -C rrdtool
- name: Print nDPI long help
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
cd ./example && ./ndpiReader -H
- name: Install nDPI
if: startsWith(matrix.arch, 'x86_64')
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
make install DESTDIR=$(realpath _install)
ls -alhHR _install
DESTDIR=/tmp/ndpi make install
ls -alhHR /tmp/ndpi
- name: Test nDPI [SYMBOLS]
if: (startsWith(matrix.os, 'ubuntu') || startsWith(matrix.os, 'mac')) && startsWith(matrix.arch, 'x86_64')
run: |
./utils/check_symbols.sh || { FAILED=$?; echo "::error file=${NDPI_LIB}::Unwanted libc symbols found: ${FAILED}. Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of malloc/calloc/realloc/free."; false; }
env:
NDPI_LIB: src/lib/libndpi.a
- name: Test nDPI [DIFF]
if: startsWith(matrix.arch, 'x86_64')
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
./tests/do.sh
NDPI_FORCE_PARALLEL_UTESTS=1 NDPI_SKIP_PARALLEL_BAR=1 ./tests/do.sh
- name: Test nDPI [UNIT]
if: startsWith(matrix.arch, 'x86_64')
#Some issues with masan + json-c. Disable the test as workaround
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows') && !startsWith(matrix.msan, '--with-memory-sanitizer') && !startsWith(matrix.os, 'macos-14')
run: |
./tests/do-unit.sh
- name: Test nDPI [DGA]
if: startsWith(matrix.arch, 'x86_64')
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
./tests/do-dga.sh
- name: Generate tarball
- name: Test nDPI [DIFF] (runs only on windows jobs)
if: startsWith(matrix.arch, 'x86_64') && startsWith(matrix.os, 'windows')
run: |
msys2 -c 'NDPI_FORCE_PARALLEL_UTESTS=1 NDPI_SKIP_PARALLEL_BAR=1 ./tests/do.sh'
- name: Test nDPI [UNIT] (runs only on windows jobs)
if: startsWith(matrix.arch, 'x86_64') && startsWith(matrix.os, 'windows')
run: |
msys2 -c './tests/do-unit.sh'
- name: Test nDPI [DGA] (runs only on windows jobs)
if: startsWith(matrix.arch, 'x86_64') && startsWith(matrix.os, 'windows')
run: |
msys2 -c './tests/do-dga.sh'
- name: Generate/Verify tarball
if: startsWith(matrix.os, 'ubuntu-latest') && startsWith(matrix.arch, 'x86_64')
run: |
make dist
- name: Generate documentation
if: startsWith(matrix.os, 'ubuntu-latest') && startsWith(matrix.arch, 'x86_64')
run: |
pip3 install --user sphinxcontrib.swaggerdoc
make -C doc text
make -C doc html
- name: Configure nDPI [Mingw-w64] (runs only on ubuntu jobs)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
run: |
make distclean && ./autogen.sh --host=x86_64-w64-mingw32 ${{ matrix.gcrypt }} ${{ matrix.pcre }} ${{ matrix.maxminddb }}
./utils/verify_dist_tarball.sh
- name: Build nDPI [Mingw-w64] (runs only on ubuntu jobs)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.nBPF, 'nBPF')
run: |
make all
make distclean
./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --host=x86_64-w64-mingw32
make -j $(nproc) all
env:
CC:
- name: Display qemu specified architecture (arm64 - little endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'arm64')
uses: docker://multiarch/ubuntu-core:arm64-bionic
@@ -229,14 +392,13 @@ jobs:
args: >
bash -c
"apt-get -y update &&
apt-get -y install git autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} &&
make all &&
apt-get -y install git wdiff colordiff autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
git config --global --add safe.directory $(realpath .) &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs &&
make -j all &&
make -C example ndpiSimpleIntegration &&
make -C rrdtool &&
make -C python &&
./tests/do.sh &&
./tests/do-unit.sh
make check VERBOSE=1
"
- name: Display qemu specified architecture (armhf - little endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'armhf')
@@ -254,14 +416,13 @@ jobs:
args: >
bash -c
"apt-get -y update &&
apt-get -y install git autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} &&
make all &&
apt-get -y install git wdiff colordiff autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
git config --global --add safe.directory $(realpath .) &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs &&
make -j all &&
make -C example ndpiSimpleIntegration &&
make -C rrdtool &&
make -C python &&
./tests/do.sh &&
./tests/do-unit.sh
make check VERBOSE=1
"
- name: Display qemu specified architecture (s390x - big endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 's390x')
@@ -272,18 +433,18 @@ jobs:
"uname -a &&
lscpu | grep Endian
"
- name: Configure and compile (no tests) using qemu for the specified architecture (s390x - big endian)
- name: Configure and compile using qemu for the specified architecture (s390x - big endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 's390x')
uses: docker://multiarch/ubuntu-core:s390x-bionic
with: #./tests/do.sh disabled because we know we have some problems with big-endian machines
with:
args: >
bash -c
"apt-get -y update &&
apt-get -y install git autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} &&
make all &&
apt-get -y install git wdiff colordiff autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
git config --global --add safe.directory $(realpath .) &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs &&
make -j all &&
make -C example ndpiSimpleIntegration &&
make -C rrdtool &&
make -C python &&
./tests/do-unit.sh
make check VERBOSE=1
"
+111
View File
@@ -0,0 +1,111 @@
name: Scheduled builds
on:
workflow_dispatch:
schedule:
#At the end of every day
- cron: '0 0 * * *'
jobs:
coverage:
name: Coverage (ubuntu-latest)
runs-on: ubuntu-latest
env:
CFLAGS: -Werror
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
sudo apt-get install rrdtool librrd-dev
sudo apt-get install libpcre3-dev libmaxminddb-dev lcov
sudo apt-get install wdiff colordiff
- name: Configure
run: ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-code-coverage --with-pcre2 --with-maxminddb --enable-tls-sigs
- name: Build
run: make all
- name: Test
run: |
make check VERBOSE=1
- name: Generate Coverage Report
run: |
make coverage
- uses: actions/upload-artifact@v4
with:
name: ndpi-coverage-report
path: coverage_report
retention-days: 7
documentation:
name: Documentation (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python 3.9
uses: actions/setup-python@v4
with:
python-version: "3.9"
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison doxygen
- name: Configure nDPI library
run: |
./autogen.sh --with-only-libndpi --enable-option-checking=fatal --enable-debug-build
- name: Generate Documentation
run: |
pip install --upgrade pip
pip install -r doc/requirements.txt
make doc
mkdir -vp doc/_build/ndpi-documentation-upload/ndpi-documentation
mv -v doc/_build/html doc/_build/ndpi-documentation-upload/ndpi-documentation/html
- uses: actions/upload-artifact@v4
with:
name: ndpi-documentation
path: doc/_build/ndpi-documentation-upload
retention-days: 7
performance:
name: Performance (ubuntu-latest)
runs-on: ubuntu-latest
env:
GO111MODULE: on
CFLAGS: -Werror
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo add-apt-repository ppa:longsleep/golang-backports
sudo apt-get update
sudo apt-get install libunwind-dev
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison libpcap-dev parallel
sudo apt-get install golang-1.16-go libgoogle-perftools-dev graphviz
go install github.com/google/pprof@latest
sudo ln -s ${HOME}/go/bin/pprof /usr/bin/pprof
pprof -h
- name: Configure nDPI library
run: |
./autogen.sh --enable-gprof --enable-option-checking=fatal --with-pcre2 --with-maxminddb --enable-tls-sigs
- name: Build nDPI library
run: |
make -j
- name: Performance Profiling
run: |
NDPI_FORCE_PARALLEL_UTESTS=1 NDPI_SKIP_PARALLEL_BAR=1 ./tests/do.sh
mkdir ndpi-performance-upload
for d in $(find ./tests/cfgs/* -type d -maxdepth 0 2>/dev/null) ; do
PROFILE="$(basename $d)"
mv -v tests/cfgs/${PROFILE}/result/cpu_profile.png ndpi-performance-upload/${PROFILE}_cpu_profile.png
mv -v tests/cfgs/${PROFILE}/result/heap_profile.png ndpi-performance-upload/${PROFILE}_heap_profile.png
done
- uses: actions/upload-artifact@v4
with:
name: ndpi-performance
path: ndpi-performance-upload
retention-days: 7
+2 -2
View File
@@ -18,7 +18,7 @@ jobs:
uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@master
with:
oss-fuzz-project-name: 'ndpi'
fuzz-seconds: 600
fuzz-seconds: 1320
dry-run: false
sanitizer: ${{ matrix.sanitizer }}
- name: Check Crash (fails when a crash is detected)
@@ -27,7 +27,7 @@ jobs:
run: |
exit $(ls out/artifacts |wc -l)
- name: Upload Crash (upload detected crash as artifacts)
uses: actions/upload-artifact@v1
uses: actions/upload-artifact@v4
if: failure()
with:
name: ${{ matrix.sanitizer }}-artifacts
+59
View File
@@ -0,0 +1,59 @@
name: "CodeQL"
on:
push:
branches: [ 'dev' ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ 'dev' ]
schedule:
- cron: '25 19 * * 2'
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
env:
GO111MODULE: on
CFLAGS: -Werror
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: [ 'cpp', 'python' ]
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo add-apt-repository ppa:longsleep/golang-backports
sudo apt-get update
sudo apt-get install -y libunwind-dev
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison libpcap-dev
sudo apt-get install golang-1.16-go libgoogle-perftools-dev graphviz
go install github.com/google/pprof@latest
sudo ln -s ${HOME}/go/bin/pprof /usr/bin/pprof
pprof -h
- name: Configure nDPI library
run: |
./autogen.sh --enable-gprof --enable-option-checking=fatal --with-pcre2 --with-maxminddb --enable-tls-sigs --enable-debug-messages
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
- name: Build nDPI library
run: |
make -j
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
category: "/language:${{matrix.language}}"
+155 -55
View File
@@ -1,8 +1,34 @@
*.[o]
*.db
*.db-*
*.dll
*.ilk
*.in
*.ipch
*.ko
*.la
*.lo
*.log
*.obj
*.opendb
*.pdb
*.recipe
*.suo
*.swp
*.tlog
*.user
*o.cmd
*o.cmd
*~
.autotools
.cproject
.dirstamp
.libs
.project
.settings
/Makefile
/aclocal.m4
/autom4te.cache/
.autotools
compile
/compile
/config.guess
/config.h
@@ -12,70 +38,144 @@ compile
/config.sub
/configure
/configure.tmp
.cproject
/depcomp
.dirstamp
/doc/Doxyfile.cfg
/example/.deps/
/example/Makefile
/example/ndpiReader
/example/Makefile.dpdk
/example/build
/example/libndpiReader.a
/example/ndpiReader
/example/ndpiReader.exe
/example/ndpiSimpleIntegration
*.in
/install-sh
*.ko
*.la
/libndpi.pc
.libs
/libtool
*.lo
/ltmain.sh
/m4/libtool.m4
/m4/lt~obsolete.m4
/m4/ltoptions.m4
/m4/ltsugar.m4
/m4/ltversion.m4
/Makefile
/missing
*.[o]
*.obj
*o.cmd
*o.cmd
.project
.settings
/src/include/Makefile
/src/include/ndpi_config.h
/src/include/ndpi_api.h
/src/include/ndpi_define.h
/src/lib/.deps/
/src/lib/Makefile
/src/lib/protocols/.deps/
/src/lib/third_party/src/.deps/
stamp-h1
*.swp
/test-driver
/tests/Makefile
/fuzz/.deps/
/fuzz/Makefile
/fuzz/fuzz_ndpi_reader
/fuzz/fuzz_ndpi_reader_with_main
/fuzz/fuzz_process_packet
/fuzz/fuzz_process_packet_with_main
/python/Makefile
/tests/dga/Makefile
/tests/do.sh
/tests/unit/Makefile
/example/libndpiReader.a
/example/ndpiReader.exe
/src/lib/libndpi.a
/src/lib/libndpi.so*
/src/lib/libndpi
/src/lib/libndpi-4.1.0.dll
/src/lib/libndpi.4
/tests/dga/dga_evaluate
/tests/unit/unit
/tags
/fuzz/fuzz_ndpi_reader_alloc_fail
/fuzz/fuzz_quic_get_crypto_data
/fuzz/fuzz_alg_bins
/fuzz/fuzz_alg_hll
/fuzz/fuzz_alg_hw_rsi_outliers_da
/fuzz/fuzz_alg_jitter
/fuzz/fuzz_alg_crc32_md5
/fuzz/fuzz_alg_ses_des
/fuzz/fuzz_alg_bytestream
/fuzz/fuzz_alg_shoco
/fuzz/fuzz_config
/fuzz/fuzz_community_id
/fuzz/fuzz_serialization
/fuzz/fuzz_ds_patricia
/fuzz/fuzz_ds_libcache
/fuzz/fuzz_ds_tree
/fuzz/fuzz_ds_ptree
/fuzz/fuzz_ds_hash
/fuzz/fuzz_ds_ahocorasick
/fuzz/fuzz_ds_bitmap64_fuse
/fuzz/fuzz_ds_domain_classify
/fuzz/fuzz_libinjection
/fuzz/fuzz_binaryfusefilter
/fuzz/fuzz_tls_certificate
/fuzz/fuzz_dga
/fuzz/fuzz_is_stun_udp
/fuzz/fuzz_is_stun_tcp
/fuzz/fuzz_ds_cmsketch
/fuzz/fuzz_gcrypt_light
/fuzz/fuzz_gcrypt_aes
/fuzz/fuzz_gcrypt_gcm
/fuzz/fuzz_gcrypt_cipher
/fuzz/fuzz_ndpi_reader_payload_analyzer
/fuzz/fuzz_ndpi_reader_pl7m_simplest
/fuzz/fuzz_ndpi_reader_pl7m_simplest_internal
/fuzz/fuzz_ndpi_reader_pl7m
/fuzz/fuzz_ndpi_reader_pl7m_64k
/fuzz/fuzz_ndpi_reader_pl7m_internal
/fuzz/fuzz_filecfg_protocols
/fuzz/fuzz_filecfg_categories
/fuzz/fuzz_filecfg_category
/fuzz/fuzz_filecfg_malicious_sha1
/fuzz/fuzz_filecfg_malicious_ja3
/fuzz/fuzz_filecfg_risk_domains
/fuzz/fuzz_filecfg_config
/fuzz/fuzz_readerutils_workflow
/fuzz/fuzz_readerutils_parseprotolist
/fuzz/fuzz_ndpi_reader_alloc_fail_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_pl7m_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_pl7m_64k_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_pl7m_internal_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_pl7m_simplest_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_pl7m_simplest_internal_seed_corpus.zip
/fuzz/fuzz_quic_get_crypto_data_seed_corpus.zip
/fuzz/fuzz_community_id_seed_corpus.zip
/fuzz/fuzz_is_stun_udp_seed_corpus.zip
/fuzz/fuzz_is_stun_tcp_seed_corpus.zip
/fuzz/fuzz_serialization_seed_corpus.zip
/fuzz/fuzz_alg_ses_des_seed_corpus.zip
/fuzz/fuzz_alg_bins_seed_corpus.zip
/fuzz/fuzz_alg_hll_seed_corpus.zip
/fuzz/fuzz_alg_jitter_seed_corpus.zip
/fuzz/fuzz_alg_crc32_md5_seed_corpus.zip
/fuzz/fuzz_alg_bytestream_seed_corpus.zip
/fuzz/fuzz_ds_libcache_seed_corpus.zip
/fuzz/fuzz_ds_ptree_seed_corpus.zip
/fuzz/fuzz_ds_bitmap64_fuse_seed_corpus.zip
/fuzz/fuzz_ds_domain_classify_seed_corpus.zip
/fuzz/fuzz_libinjection_seed_corpus.zip
/fuzz/fuzz_tls_certificate_seed_corpus.zip
/fuzz/fuzz_filecfg_protocols_seed_corpus.zip
/fuzz/fuzz_filecfg_categories_seed_corpus.zip
/fuzz/fuzz_filecfg_category_seed_corpus.zip
/fuzz/fuzz_filecfg_malicious_sha1_seed_corpus.zip
/fuzz/fuzz_filecfg_malicious_ja3_seed_corpus.zip
/fuzz/fuzz_filecfg_risk_domains_seed_corpus.zip
/fuzz/fuzz_filecfg_config_seed_corpus.zip
/fuzz/fuzz_dga_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_payload_analyzer_seed_corpus.zip
/fuzz/fuzz_readerutils_workflow_seed_corpus.zip
/fuzz/fuzz_readerutils_parseprotolist_seed_corpus.zip
/fuzz/fuzz_*.dict
/fuzz/testpcaps.zip
/influxdb/Makefile
/install-sh
/libndpi.pc
/libtool
/ltmain.sh
/m4/libtool.m4
/m4/ltoptions.m4
/m4/ltsugar.m4
/m4/ltversion.m4
/m4/lt~obsolete.m4
/missing
/rrdtool/Makefile
/rrdtool/rrd_anomaly
/rrdtool/rrd_similarity
/src/include/Makefile
/src/include/ndpi_config.h
/src/include/ndpi_define.h
/src/lib/.deps/
/src/lib/Makefile
/src/lib/libndpi
/src/lib/libndpi-*.dll
/src/lib/libndpi.4
/src/lib/libndpi.a
/src/lib/libndpi.so*
/src/lib/protocols/.deps/
/src/lib/third_party/src/.deps/
/src/lib/third_party/src/fuzz/.deps/
/tags
/test-driver
/tests/Makefile
/tests/dga/Makefile
/tests/dga/dga_evaluate
/tests/dga/dga_evaluate.exe
/tests/do.sh
/tests/performance/Makefile
/tests/unit/Makefile
/tests/unit/unit
compile
stamp-h1
/windows/packages/pthreads.2.9.1.4
/windows/packages/pthreads.redist.2.9.1.4
/windows/bin/x64_Debug-ndpiReader/ndpiReader.exe
/windows/obj/nDPI_Debug-ndpiReader_x64/nDPI.vcxproj.FileListAbsolute.txt
@@ -1,83 +0,0 @@
/**
* @name Suspicious packet->payload based integer arithmetic
* @description An arithmetic operation influenced array access is suspicious
* if it uses an integer value that is likely to be network-controlled, and
* may require a closer manual audit.
* @kind problem
* @problem.severity warning
* @id cpp/packet-payload-integer-arithmetic
* @tags audit security
*/
import cpp
import semmle.code.cpp.dataflow.TaintTracking
import semmle.code.cpp.rangeanalysis.SimpleRangeAnalysis
/** A source of an integer value that is likely to come from the network.
* This is produced by an invocation of a macro of the form `ntoh*` or `get_u_int*_t`,
* called with `packet->payload` as an argument.
*/
class NetworkMacro extends Macro {
NetworkMacro() { this.getName().regexpMatch("^ntoh(ll|l|s)") }
}
class NetworkIntegerSource extends Expr {
NetworkIntegerSource() {
exists(MacroInvocation mi |
this = mi.getExpr() and
mi.getUnexpandedArgument(0).regexpMatch(".*packet->payload.*") |
// catch all get_u_int*_t(x)
mi.getMacroName().regexpMatch("^get_u_int(64|32|16|8)_t") and
// dedup ntoh*(get_u_int*_t(x)) since we'll catch those in the next case
not mi.getOutermostMacroAccess().getMacro() instanceof NetworkMacro
or
// catch all ntoh*(x) ... this will also catch the nested cases
mi.getMacro() instanceof NetworkMacro
)
}
}
class ArithmeticOperation extends Operation {
ArithmeticOperation() {
this instanceof UnaryArithmeticOperation or this instanceof BinaryArithmeticOperation
}
}
class NetworkToArrayAccess extends TaintTracking::Configuration {
NetworkToArrayAccess() { this = "NetworkToArrayAccess" }
override predicate isSource(DataFlow::Node source) {
source.asExpr() instanceof NetworkIntegerSource
}
override predicate isSink(DataFlow::Node sink) {
exists(ArrayExpr ae | sink.asExpr() = ae.getArrayOffset())
}
}
class NetworkToArithmetic extends TaintTracking::Configuration {
NetworkToArithmetic() { this = "NetworkToArithmetic" }
override predicate isSource(DataFlow::Node source) {
source.asExpr() instanceof NetworkIntegerSource
}
override predicate isSink(DataFlow::Node sink) {
exists (Assignment assign |
sink.asExpr() = assign.getRValue().(ArithmeticOperation) or
sink.asExpr() = assign.(AssignArithmeticOperation)
) or
exists(LocalVariable var |
sink.asExpr() = var.getInitializer().getExpr().(ArithmeticOperation)
)
}
}
// find audit candidates based on suspicious network integer use
from NetworkIntegerSource source, Expr sink1, Expr sink2, NetworkToArithmetic config1, NetworkToArrayAccess config2
where config1.hasFlow(DataFlow::exprNode(source), DataFlow::exprNode(sink1))
// or this if you want integer arithmeric _OR_ array accesses
and config2.hasFlow(DataFlow::exprNode(source), DataFlow::exprNode(sink2))
select source, "Suspicious use of network integer arithmetic."
+479
View File
@@ -1,5 +1,484 @@
# CHANGELOG
#### nDPI 4.8 (Oct 2023)
## Major Changes
- Reworked lists implementation that decreased memory usage of orders of magnitude
- Improved code robustness via extensive code fuzzing
- Various improvements to overall library performance
- Extended IPv6 support
## New Supported Protocols and Services
- Add "Heroes of the Storm" video game signature detection. (#1949)
- Add Apache Thrift protocol dissector. (#2007)
- Add Remote Management Control Protocol (RMCP).
- Add Service Location Protocol dissector. (#2036)
- Add VK detection (#1880)
- Add Yandex services detection (#1882)
- Add a new protocol id for generic Adult Content traffic (#1906)
- Add a new protocol id for generic advertisement/analytics/tracking stuff (#1904)
- Add bitcoing protocol dissector. (#1992)
- Add detection of Roblox games (#2054)
- Add support for (un-encrypted) HTTP/2 (#2087)
- Add support for Epic Games and GeForceNow/Nvidia (#1990)
- Add support for SRTP (#1977)
- Added BACnet dissector. (#1940)
- Added HAProxy protocol. (#2088)
- Added OICQ dissector. (#1950)
- Added OperaVPN detection
- ProtonVPN: add basic detection (#2006)
- Added detection of Facebook Reels and Stories
- Add an heuristic to detect fully encrypted flows (#2058)
- Added NDPI_MALWARE_HOST_CONTACTED flow risk
- Added NDPI_TLS_ALPN_SNI_MISMATCH flow risk
## Improvements
- Improve protocol detection for:
- FreeBSD compilation fix (C) update
- Gnutella: improve detection (#2019)
- H323: fix false positives (#1916)
- HTTP: fix another memory access error (#2049)
- HTTP: fix extraction of filename (#2046)
- HTTP: fix heap-buffer-overflow (#2044)
- HTTP: improve extraction of metadata and of flow risks (#1959)
- HTTP: remove useless code about XBOX (#1958)
- HTTP: rework state machine (#1966)
- Hangout: detect Hangout/Duo/GoogleMeet/... in the STUN code (#2025)
- Enhance DNS risk for long hostnames (> 32)
- Enhanced MS teams STUN/Azure detection
- Enhanced custom port definition and improved error reporting in case of duplications
- Improve detection of Alibaba flows (#1991)
- Improve detection of crawler/bot traffic (#1956)
- Improve detection of crawlers/bots (#1968)
- Improved MGCP detection by allowing '\r' as line feed.
- Improved MS Teams detection with heuristic
- Improved Steam detection by adding steamdiscover pattern. (#2105)
- Improved Wireguard detection
- Improved checks for duplicated entries in protocols file
- Improved classification further reducing memory used
- Improved detection of invalid chars in DNS names
- Improved domain search tet unit
- Improved helper scripts. (#1986)
- MS Teams enhancement
- MySql: improve detection (#1928)
- zabbix: improve detection (#2055)
## Tools
- ndpiReader: allow to configure LRU caches TTL and size (#2004)
- ndpiReader: fix VXLAN de-tunneling (#1913)
- ndpiReader: fix export of DNS/BitTorrent attributes (#1985)
- ndpiReader: fix export of HTTP attributes (#1982)
- ndpiReader: fix flow stats (#1943)
- ndpiReader: fix print of flow payload (#1960)
- ndpiReader: improve printing of payload statistics (#1989)
- ndpiReader: print how many packets (per flow) were needed to perform full DPI (#1891)
- ndpireader: fix detection of DoH traffic based on packet distributions (#2045)
## Misc
- ARM compilation fix
- Add `ndpi_domain_classify_finalize()` function (#2084)
- Add a configuration knob to enable/disable loading of gambling list (#2047)
- Add a new flow risk about literal IP addresses used as SNI (#1892)
- Add an heuristic to detect/ignore some anomalous TCP ACK packets (#1948)
- Add another example of custom rules (#1923)
- Add support for multiline json
- Add support for roaring_bitmap_xor_inplace (#1983)
- Add support for vxlan decapsulation (#1441) (#1900)
- Added Source Engine dissector. (#1937)
- Added `lists/gambling.list` to extra dist.
- Added `slackb.com` SNI. (#2067)
- Added ability to define an unlimited number of custom rules IP:port for the same IP (it used tobe limited to 2)
- Added check to avoid skype heuristic false positives
- Added comment
- Added coverage targets to `Makefile.am` for convenience. (#2039)
- Added fix for better handling exceptions rollback in case of later match
- Added hyperlink
- Added ndpi_binary_bitmap data structure
- Added ndpi_bitmap64 support
- Added ndpi_bitmap_andnot API call
- Added ndpi_bitmap_copy() API call
- Added ndpi_bitmap_is_empty() and ndpi_bitmap_optimize() API calls
- Added ndpi_domain_classify_XXX(0 API
- Added ndpi_filter_add_multi() API call
- Added ndpi_murmur_hash to the nDPI API
- Added new API calls for implementing Bloom-filter like data structures
- Added printf/fprintf replacement for some internal modules. (#1974)
- Added scripts to auto generate hostname/SNI *.inc files. (#1984)
- Added sub-domain classification fix
- Added the ability to define custom protocols with arbitrary Ids in proto.txt
- Added vlan_id in ndpi_flow2json() prototype
- Adds new pcap for testing "funny" HTTP servers
- All protocols should be excluded sooner or later (#1969)
- Allow init of app protocols w/o any hostnames set. (#2057)
- Avoid calling `ndpi_reconcile_protocols()` twice in `ndpi_detection_giveup()` (#1996)
- Boundary check
- CI: fix `Performance` job (#1936)
- Centos7 fixes
- Changed logging callback function sig. (#2000)
- Changes for supporting more efficient sub-string matching
- Classification fixes
- DNS: extract geolocation information, if available (#2065)
- Debian 12 fixes
- Disabled query string validation in MDNS in order to avoid zapping chars that in DNS (instead) are not permitted
- DisneyPlus/Hulu ip lists should be auto-generated (#1905)
- Extend content list of Microsoft protocols (#1930)
- Extend content-match list (#1967)
- Fix LRU/Patricia/Automa stats in `ndpiReader` with multiple threads (#1934)
- Fix MS Teams detection with heuristic (#1972)
- Fix access to packet/flow information (#2013)
- Fix an heap-buffer-overflow (#1994)
- Fix classification-by-ip in `ndpi_detection_giveup` (#1981)
- Fix compilation (#2011)
- Fix compilation in CI jobs (#2048)
- Fix compilation on Windows (#2072)
- Fix compilation with GCC-7 and latest RoaringBitmap code (#1886)
- Fix detection of packet direction and NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1883)
- Fix export/serialization of `flow->risk` (#1885)
- Fix for buffer overflow in serialization
- Fix insert of ip addresses into patricia tree(s) (#1895)
- Fix missing u_char, u_short and u_int typedefs for some platforms e.g.: (#2009)
- Fix packet counters (#1884)
- Fix some errors found by fuzzers (#2078)
- Fix some memory errors triggered by allocation failures (#1995)
- Fix some prototypes (#2085)
- Fix string truncation. (#2056)
- Fixed OpenWRT arm related build issues. (#2104)
- Fixed heap-buffer-overflow issue
- Fixed heap-overflow if compiled with `--enable-tls-sigs`. (#2038)
- Fixed invalid use of ndpi_free(). Sorry, my fault. (#1988)
- Fixed missing AS_HELP_STRING in configure.ac. (#1893)
- Fixed two OpenWRT arm related build issues. (#2103)
- Fixes matches with domain name strings that start with a dot
- Fixes risk mask exception handling while improving the overall performance
- Implemented Count-Min Sketch [count how many times a value has been observed]
- Implemented Zoom/Teams stream type detection
- Implemented ndpi_XXX_reset() API calls whre XXX is ses, des, hw
- Implemented ndpi_predict_linear() for predicting a timeseries value overtime
- Improved debug output. (#1951)
- Improved invalid logging via printf().
- Improved line protocol dissection with heuristic
- Improved missing usage of nDPIs malloc wrapper. Fixes #1978. (#1979)
- Improved protocol detection exploiting IP-based guess Reworked ndpi_reconcile_protocols() that is now called only in front of a match (less overhead)
- Improvement for reducing false positives
- Included Gambling website data from the Polish `hazard.mf.gov.pl` list (#2041)
- Keep master protocol in `ndpi_reconcile_protocols`
- Leak fix
- Language fix
- Line: fix heap-buffer-overflow error (#2015)
- Made VK protocol detection more strict
- Make Bittorrent LRU cache IPv6 aware. (#1909)
- Merged new and old version of ndpi_domain_classify.c code
- Mullvad VPN service added (based on entry node IP addresses) (#2062)
- Numeric truncation at `ndpi_analyze.c` at lines 101, 104, 107, 110 (#1999)
- Numeric truncation at `tls.c:1010` (#2005)
- Ookla: rework detection (#1922)
- Optimizes and fixes possible out0of0boundary write in ndpi_fill_prefix_v4()
- ProtonVPN: split the ip list (#2060)
- QUIC: add support for QUIC version 2
- QUIC: export QUIC version as metadata
- QUIC: fix a memory access error
- QUIC: fix dissection of packets forcing VN
- RDP: improve detection over UDP (#2043)
- RTP: remove dead-code (#1953)
- RTP: rework code (#2021)
- Refreshed ASN lists Enhanced the Line IP list with https://ipinfo.io/AS23576/125.209.252.0/24 used by line
- Remove some useless checks (#1993)
- Remove special handling of some TCP flows without SYN (#1965)
- Removed overlapping port
- Renamed HTTP/2 to HTTP2 as the '/' can have side effects with applications sitting on top of nDPI
- Replaces free() with ndpi_free()
- Rework CI jobs to try reducing CI duration (#1903)
- Reworked domain classification based on binary filters
- Reworked initialization
- Reworked ndpi_filter_xxx implementation using compressed bitmaps
- Reworked teams handling
- RiotGames: add detection of flows (#1935)
- STUN: add dissection of DTLS handshake (#2018)
- STUN: avoid FacebookVoip false positives (#2029)
- STUN: fix Skype/MsTeams detection and monitoring logic (#2028)
- STUN: fix detection of Google Voip apps (#2031)
- STUN: fix detection over TCP
- STUN: improve WhatsappCall detection
- STUN: keep monitoring/processing STUN flows (#2012)
- STUN: tell RTP from RTCP while in monitoring state (#2027)
- Serialization fix
- Set _DEFAULT_SOURCE and _GNU_SOURCE globally. (#2010)
- Simplify `ndpi_internal_guess_undetected_protocol()` (#1941)
- Simplify the report of streaming multimedia info (#2026)
- SoftEther: fix invalid memory access
- Swap from Aho-Corasick to an experimental/home-grown algorithm that uses a probabilistic approach for handling Internet domain names.
- Sync unit tests results
- Sync unit tests results
- Sync unit tests results (#1962)
- Sync utests results (#1887)
- TLS: add basic, basic, detection of Encrypted ClientHello (#2053)
- TLS: fix another interger overflow in certificate processing (#1915)
- TLS: fix parsing of certificate elements (#1910)
- Test files for riit games
- Test multiple `ndpiReader` configurations (#1931)
- Thrift: fix heap-buffer-overflow (#2024)
- Update GitHub runners versions (#1889)
- Update every ip lists (#2079)
- Update libinjection code (#1918)
- Update protocols documentation (#2081)
- Update roaring bitmap code
- Updated line test result
- Updated pcap detection results after Facebook Reel/Stories support
- Updated results
- Updated results after the latest changes
- Win include change
- Windows code rework
- Windows compilation fixes
- Windows warning checks
- add 2 ns from fdn.fr to DoH section (#1964)
- add support for gre decapsulation (#1442) (#1921)
- added bimap and/or with allocation
- added feature to extract filename from http attachment (#2037)
- added new domain names (#2002)
- configure: add an option to enable debug build, i.e `-g` (#1929)
- fix Stack overflow caused by invalid write in ndpi_automa_match_strin… (#2035)
- fixed numeric truncation error
- fixed numeric truncation error in diameter.c (#2034)
- fixed numeric truncation error in kerberos.c (#2032)
- fixed numeric truncation error in ndpi_main.c:6837 (#1998)
- fixed numeric truncation error in rtcp.c (#2033)
- fuzz: add a new fuzzer to test TLS certificates (#1901)
- fuzz: add a new fuzzer triggering the payload analyzer function(s) (#1926)
- fuzz: add fuzzer for DGA detection code (#2042)
- fuzz: add fuzzer to test internal gcrypt code (#1920)
- fuzz: add fuzzers to test bitmap64 and domain_classify data structures (#2082)
- fuzz: add fuzzers to test reader_util code (#2080)
- fuzz: extend coverage (#2073)
- fuzz: extend fuzz coverage (#1888)
- fuzz: extend fuzzers coverage (#1952)
- fuzz: extend fuzzing coverage (#2040)
- fuzz: extend fuzzing coverage (#2052)
- fuzz: extend fuzzing coverage (#2083)
- fuzz: simplify fuzzers dependencies in CIFuzz (#1896)
- fuzz: some improvements and add two new fuzzers (#1881)
- fuzzing: extend fuzzing coverage
- in case of failure, failing result files are not listed
- minor fixes (#2023)
- oss-fuzz: sync build script with upstream
- remove redefinition to vxlanhdr struct in vxlan dissector (#1911)
- removed useless call of ndpi_set_risk func (#2022)
- tests: add an option to force the overwrite of the unit tests results (#2001)
- tests: restore some old paths as symbolic links (#2050)
- tftp: check for Option Acknowledgements
- tftp: check incrementation for DATA and ACK packets
- tftp: rework request checking to account for options
- tftp: update pcap results
- version of dirent.c that is liked by both VC++ and MinGW
#### nDPI 4.6 (Feb 2023)
## New Features
- New support for custom BPF protocol definition using nBPF (see example/protos.txt)
- Improved dissection performance
- Added fuzzing all over
## New Supported Protocols and Services
- Add protocol detection for:
- Activision
- AliCloud server access
- AVAST
- CryNetwork
- Discord
- EDNS
- Elasticsearch
- FastCGI
- Kismet
- Line App and Line VoIP calls
- Meraki Cloud
- Munin
- NATPMP
- Syncthing
- TP-LINK Smart Home
- TUYA LAN
- SoftEther VPN
- Tailscale
- TiVoConnect
## Improvements
- Improve protocol detection for:
- Anydesk
- Bittorrent (fix confidence, detection over TCP)
- DNS, add ability to decode DNS PTR records used for reverse address resolution
- DTLS (handle certificate fragments)
- Facebook Voip calls
- FastCGI (dissect PARAMS)
- FortiClient (update default ports)
- Zoom
- Add Zoom screen share detection
- Add detection of Zoom peer-to-peer flows in STUN
- Hangout/Duo Voip calls detection, optimize lookups in the protocol tree
- HTTP
- Handling of HTTP-Proxy and HTTP-Connect
- HTTP subclassification
- Check for empty/missing user-agent in HTTP
- IRC (credentials check)
- Jabber/XMPP
- Kerberos (support for Krb-Error messages)
- LDAP
- MGCP
- MONGODB (avoid false positives)
- Postgres
- POP3
- QUIC (support for 0-RTT packets received before the initial)
- Snapchat Voip calls
- SIP
- SNMP
- SMB (support for messages split into multiple TCP segments)
- SMTP (support for X-ANONYMOUSTLS command)
- STUN
- SKYPE (improve detection over UDP, remove detection over TCP)
- Teamspeak3 (License/Weblist detection)
- Threema Messenger
- TINC (avoid processing SYN packets)
- TLS
- improve reassembler
- handling of ALPN(s) and subclassification
- ignore invalid Content Type values
- WindowsUpdate
- Add flow risk:
- NDPI_HTTP_OBSOLETE_SERVER
- NDPI_MINOR_ISSUES (generic/relevant information about issues found on traffic)
- NDPI_HTTP_OBSOLETE_SERVER (Apache and nginx are supported)
- NDPI_PERIODIC_FLOW (reserved bit to be used by apps based on nDPI)
- NDPI_TCP_ISSUES
- Improve detection of WebShell and PHP code in HTTP URLs that is reported via flow risk
- Improve DGA detection
- Improve AES-NI check
- Improve nDPI JSON serialization
- Improve export/print of L4 protocol information
- Improve connection refused detection
- Add statistics for Patricia tree, Ahocarasick automa, LRU cache
- Add a generic (optional and configurable) expiration logic in LRU caches
- Add RTP stream type in flow metadata
- LRU cache is now IPv6 aware
## Tools
- ndpiReader
- Add support for Linux Cooked Capture v2
- Fix packet dissection (CAPWAP and TSO)
- Fix Discarded bytes statistics
## Fixes
- Fix classification by-port
- Fix exclusion of DTLS protocol
- Fix undefined-behaviour in ahocorasick callback
- Fix infinite loop when a custom rule has port 65535
- Fix undefined-behavior when setting empty user-agent
- Fix infinite loop in DNS dissector (due to an integer overflow)
- Fix JSON export of IPv6 addresses
- Fix memory corruptions in Bittorrent, HTTP, SoftEther, Florensia, QUIC, IRC, TFTP dissectors
- Fix stop of extra dissection in HTTP, Bittorrent, Kerberos
- Fix signed integer overflow in ASN1/BER dissector
- Fix char/uchar bug in ahocorasick
- Fix endianness in IP-Port lookup
- Fix FastCGI memory allocation issue
- Fix metadata extraction in NAT-PMP
- Fix invalid unidirectional traffic alert for unidirectional protocols (e.g. sFlow)
## Misc
- Support for Rocky Linux 9
- Enhance fuzzers to test nDPI configurations, memory allocation failures, serialization/deserialization, algorithms and data structures
- GitHub Actions: update to Node.js 16
- Size of LRU caches is now configurable
#### nDPI 4.4 (July 2022)
## New Features
- Add risk information that describes why a specific risk was triggered also providing metadata
- Added API call ndpi_check_flow_risk_exceptions() for handling risk exceptions
- Split protocols in: network (e.g. TLS) and application protocols (e.g. Google)
- Extended confidence level with two new values (NDPI_CONFIDENCE_DPI_PARTIAL and NDPI_CONFIDENCE_DPI_PARTIAL_CACHE)
- Added ndpi_get_flow_error_code() API call
## New Supported Protocols and Services
- Add protocol detection for:
- UltraSurf
- i3D
- RiotGames
- TSAN
- TunnelBear VPN
- collectd
- PIM (Protocol Indipendent Multicast)
- Pragmatic General Multicast (PGM)
- RSH
- GoTo products (mainly GoToMeeting)
- Dazn
- MPEG-DASH
- Agora Software Defined Real-time Network (SD-RTN)
- Toca Boca
- VXLAN
- MDNS/LLMNR
## Improvements
- Improve protocol detection for:
- SMTP/SMTPS now supports STARTTLS
- OCSP
- TargusDataspeed
- Usenet
- DTLS (added support for old versions)
- TFTP
- SOAP via HTTP
- GenshinImpact
- IPSec/ISAKMP
- DNS
- syslog
- DHCP (various bug fixes and improvements)
- NATS
- Viber
- Xiaomi
- Raknet
- gnutella
- Kerberos
- QUIC (Added support for v2drft 01)
- SSDP
- SNMP
- Improved DGA detection
- Improved AES-NI check
- Add flow risk:
- NDPI_PUNYCODE_IDN
- NDPI_ERROR_CODE_DETECTED
- NDPI_HTTP_CRAWLER_BOT
- NDPI_ANONYMOUS_SUBSCRIBER
- NDPI_UNIDIRECTIONAL_TRAFFIC
## Changes
- Added support for 64 bit bins
- Added Cloudflare WARP detection patterns
- Renamed Z39.50 -> Z3950
- Replaced nDPI's internal hashmap with uthash
- Reimplemented 1kxun application protoco
- Renamed SkypeCall to Skype_TeamsCall
- Updated Python Bindings
- Unless --with-libgcrypt is used, nDPI now uses its internal gcrypt implementation
## Fixes
- Fixes for some protocol classification families
- Fixed default protocol ports for email protocols
- Various memory and overflow fixes
- Disabled various risks for specific protocols (e.g. disable missing ALPN for CiscoVPN)
- Fix TZSP decapsulation
## Misc
- Update ASN/IPs lists
- Improved code profiling
- Use Doxygen to generate the API documentation
- Added Edgecast and Cachefly CDNs.
#### nDPI 4.2 (Feb 2022)
## New Features
+6
View File
@@ -24,4 +24,10 @@ Please keep all code, comments, issues and pull requests in English, otherwise t
- If there is an open issue for the bug, ensure the description of the pull request contains `Fixes #issue_number`.
- Ensure the pull request passes CI testing. Any pull request which fails CI will be **ignored**.
## Before you submit your code
As we want to make sure that we can incorporate your changes in nDPI, we ask all contributors to read and sign the
[Contributor License Agreement](https://github.com/ntop/legal/blob/main/individual-contributor-licence-agreement.md)
that guarantees that your code is original and royalty-free to keep nDPI free and available for everybody,
Finally, thank you for your contributions.
+35 -1
View File
@@ -8,7 +8,41 @@ endif
pkgconfigdir = $(libdir)/pkgconfig
pkgconfig_DATA = libndpi.pc
EXTRA_DIST = README.md CHANGELOG.md CONTRIBUTING.md autogen.sh wireshark python windows utils packages example/MacOS doc/api.rst doc/conf.py doc/flow_risks.rst doc/guide/nDPI_QuickStartGuide.pages doc/guide/nDPI_QuickStartGuide.pdf doc/img/logo.png doc/index.rst doc/Makefile doc/what_is_ndpi.rst
EXTRA_DIST = README.md README.fuzzer.md CHANGELOG.md CONTRIBUTING.md \
autogen.sh wireshark windows utils packages \
influxdb/Makefile.in influxdb/metric_anomaly.c influxdb/README.txt \
rrdtool/Makefile.in rrdtool/README.txt rrdtool/rrd_anomaly.c rrdtool/rrd_similarity.c \
doc/requirements.txt doc/conf.py doc/flow_risks.rst doc/protocols.rst doc/guide/nDPI_QuickStartGuide.pages \
doc/guide/nDPI_QuickStartGuide.pdf doc/img/logo.png doc/index.rst \
doc/Makefile doc/what_is_ndpi.rst doc/FAQ.md doc/configuration_parameters.md \
doc/library_initialization.md \
python/DEV_GUIDE.md python/dev_requirements.txt python/ndpi_example.py python/ndpi/__init__.py \
python/ndpi/ndpi_build.py python/ndpi/ndpi.py python/README.md \
python/requirements.txt python/setup.py python/tests.py \
lists/100_malware.list \
lists/107_gambling.list \
lists/107_gambling_custom.list \
lists/public_suffix_list.dat \
lists/README.md \
sonar-project.properties .github .ci-ignore
doc:
make -C doc html
doc-view:
python3 -m http.server --bind 127.0.0.1 --directory doc/_build/html 8080
coverage: all
mkdir -vp coverage_report/ndpi_coverage_report
lcov --directory . --capture --output-file lcov.info
genhtml -o coverage_report/ndpi_coverage_report lcov.info
clean-coverage:
rm -f lcov.info
find . -type f -name "*.gcda" -exec rm "{}" +
find . -type f -name "*.gcno" -exec rm "{}" +
changelog:
git log --since={`curl -s https://github.com/ntop/ndpi/releases | grep datetime | head -n1 | egrep -o "[0-9]+\-[0-9]+\-[0-9]+"`} --name-only --pretty=format:" - %s" | grep "^ " > CHANGELOG.latest
.PHONY: doc doc-view coverage clean-coverage changelog
+3 -3
View File
@@ -13,11 +13,11 @@
## Testing nDPI with ClusterFuzz Artifacts
- Download the artifact (example clusterfuzz-testcase-fuzz_process_packet-4992218834796544)
- Run nDPI against the artifact
- Example: ./fuzz/fuzz_process_packet_with_main clusterfuzz-testcase-fuzz_process_packet-4992218834796544
- Example: ./fuzz/fuzz_process_packet clusterfuzz-testcase-fuzz_process_packet-4992218834796544
The output is the error report
```
./fuzz/fuzz_process_packet_with_main /tmp/clusterfuzz-testcase-fuzz_process_packet-4992218834796544
./fuzz/fuzz_process_packet /tmp/clusterfuzz-testcase-fuzz_process_packet-4992218834796544
AddressSanitizer:DEADLYSIGNAL
=================================================================
==11590==ERROR: AddressSanitizer: SEGV on unknown address 0x61a100000087 (pc 0x00000056e6a4 bp 0x7ffd624fa170 sp 0x7ffd624fa090 T0)
@@ -33,7 +33,7 @@ The output is the error report
#8 0x4c709b in LLVMFuzzerTestOneInput /home/deri/nDPI/fuzz/fuzz_process_packet.c:30:3
#9 0x4c7640 in main /home/deri/nDPI/fuzz/fuzz_process_packet.c:90:17
#10 0x7f888e5dabf6 in __libc_start_main /build/glibc-S9d2JN/glibc-2.27/csu/../csu/libc-start.c:310
#11 0x41c399 in _start (/home/deri/nDPI/fuzz/fuzz_process_packet_with_main+0x41c399)
#11 0x41c399 in _start (/home/deri/nDPI/fuzz/fuzz_process_packet+0x41c399)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/deri/nDPI/src/lib/protocols/quic.c:203:12 in quic_len
+70 -24
View File
@@ -1,65 +1,111 @@
![ntop][ntopng_logo] ![ntop][ntop_logo]
# nDPI
[![Build Status](https://img.shields.io/github/workflow/status/ntop/nDPI/Build/dev?logo=github)](https://github.com/ntop/nDPI/actions?query=workflow%3ABuild)
[![Code Quality: Cpp](https://img.shields.io/lgtm/grade/cpp/g/ntop/nDPI.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/ntop/nDPI/context:cpp)
[![Total Alerts](https://img.shields.io/lgtm/alerts/g/ntop/nDPI.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/ntop/nDPI/alerts)
[![Build Status](https://img.shields.io/github/actions/workflow/status/ntop/nDPI/build.yml?branch=dev&logo=github)](https://github.com/ntop/nDPI/actions?query=workflow%3ABuild)
[![Fuzzing Status](https://oss-fuzz-build-logs.storage.googleapis.com/badges/ndpi.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened&can=1&q=proj:ndpi)
## What is nDPI ?
nDPI® is an open source LGPLv3 library for deep-packet inspection. Based on OpenDPI it includes ntop extensions. We have tried to push them into the OpenDPI source tree but nobody answered emails so we have decided to create our own source tree
A generic FAQ about nDPI® is available [here](https://github.com/ntop/nDPI/blob/dev/doc/FAQ.md)
### How To Compile nDPI
In order to compile this library do
In order to compile this project do
- ./autogen.sh
- ./configure
- make
To compile the library w/o any tools or tests:
- ./autogen.sh --with-only-libndpi
- make
To run tests do additionally:
- cd tests; ./do.sh
- ./tests/do.sh # Generate and check for diff's in PCAP files
- ./tests/do-unit.sh # Run unit tests
- ./tests/do-dga.sh # Run DGA detection test
Please note that the pre-requisites for compilation include:
- GNU tools (autogen, automake, autoconf, libtool)
- GNU C compiler (gcc)
or run all with: `make check`
Please note that the (minimal) pre-requisites for compilation include:
- GNU tools (autoconf automake libtool pkg-config gettext flex bison)
- GNU C compiler (gcc) or Clang
On Debian/Ubuntu systems do:
- sudo apt-get install build-essential git bison flex libpcap-dev libtool libtool-bin autoconf pkg-config automake autogen libjson-c-dev libnuma-dev libgcrypt20-dev libpcre2-dev
- sudo apt-get install build-essential git gettext flex bison libtool autoconf automake pkg-config libpcap-dev libjson-c-dev libnuma-dev libpcre2-dev libmaxminddb-dev librrd-dev
FreeBSD
- sudo pkg install gcc git autoconf automake libtool devel/pkgconf json-c gmake
On Arch Linux:
- sudo pacman -S gcc git gettext flex bison libtool autoconf automake pkg-config libpcap json-c numactl pcre2 libmaxminddb rrdtool
Remember to use gmake and not make on FreeBSD
On FreeBSD:
- sudo pkg install gcc git gettext flex bison libtool autoconf automake devel/pkgconf gmake libpcap json-c pcre2 libmaxminddb rrdtool
Remember to use `gmake` and not `make` on FreeBSD
On MacOS:
- brew install coreutils gcc git gettext flex bison libtool autoconf automake pkg-config libpcap json-c pcre2 libmaxminddb rrdtool
On Windows:
There are three supported ways to build nDPI:
1. MSYS2 (assuming [MSYS2](https://www.msys2.org/) already installed):
- msys2 -c "pacman --noconfirm -S --needed --overwrite '\*' git mingw-w64-x86\_64-toolchain automake1.16 automake-wrapper autoconf libtool make mingw-w64-x86\_64-json-c mingw-w64-x86\_64-crt-git mingw-w64-x86\_64-pcre2 mingw-w64-x86\_64-libpcap"
2. Mingw-w64
3. Visual Studio (see `windows/nDPI.sln`)
Note: All Windows versions require [npcap](https://npcap.com/#download) with WinPcap compatibility mode enabled.
### How To Build The Documentation
- pip install --upgrade pip
- pip install -r doc/requirements.txt
- make doc
Use the builtin python3 webserver to view documentation:
- make doc-view
### How To Add A New Protocol Dissector
The entire procedure of adding new protocols in detail:
1. Add new protocol together with its unique ID to: src/include/ndpi_protocol_ids.h
2. Create a new protocol in: src/lib/protocols/
3. Variables to be kept for the duration of the entire flow (as state variables) need to be placed in: src/include/ndpi_typedefs.h in ndpi_flow_tcp_struct (for TCP only), ndpi_flow_udp_struct (for UDP only), or ndpi_flow_struct (for both).
4. Add a new entry for the search function for the new protocol in: src/include/ndpi_protocols.h
5. Choose (do not change anything) a selection bitmask from: src/include/ndpi_define.h
6. Add a new entry in ndpi_set_protocol_detection_bitmask2 in: src/lib/ndpi_main.c
7. Set protocol default ports in ndpi_init_protocol_defaults in: src/lib/ndpi_main.c
8. ./autogen.sh
9. make
10. make check
1. Add new protocol together with its unique ID to: `src/include/ndpi_protocol_ids.h`
2. Create a new protocol in: `src/lib/protocols/`
3. Variables to be kept for the duration of the entire flow (as state variables) need to be placed in: `src/include/ndpi_typedefs.h` in `ndpi_flow_tcp_struct` (for TCP only), `ndpi_flow_udp_struct` (for UDP only), or `ndpi_flow_struct` (for both).
4. Add a new entry for the search function for the new protocol in: `src/include/ndpi_protocols.h`
5. Choose (do not change anything) a selection bitmask from: `src/include/ndpi_define.h`
6. Set protocol default ports in `ndpi_init_protocol_defaults` in: `src/lib/ndpi_main.c`
7. Be sure to have nBPF support, cloning `PF_RING` in the same directory where you cloned `nDPI`: `git clone https://github.com/ntop/PF_RING/ && cd PF_RING/userland/nbpf && ./configure && make`. You can ignore the `/bin/sh: 1: ../lib/pfring_config: not found` error
8. From the `nDPI` root directory, `./autogen.sh --with-pcre2` (nBPF and PCRE2 are usually optional, but they are needed to run/update *all* the unit tests)
9. `make`
10. `make check`
11. Update the documentation, adding this new protocol to `doc/protocols.rst`
12. Update the Windows Visual Studio configuration, adding the new c file in `windows/nDPI.vcxproj`
### How to use nDPI to Block Selected Traffic
You can use nDPI to selectively block selected Internet traffic by embedding it onto an application (remember that nDPI is just a library). Both [ntopng](https://github.com/ntop/ntopng) and [nProbe cento](http://www.ntop.org/products/netflow/nprobe-cento/) can do this.
### nDPI Paper Citation
- Deri, Luca, et al. [nDPI: Open-source high-speed deep packet inspection](http://luca.ntop.org/nDPI.pdf) 2014 International Wireless Communications and Mobile Computing Conference (IWCMC). IEEE, 2014.
### Videos and Presentations
- [Using nDPI for Monitoring and Security](https://archive.fosdem.org/2021/schedule/event/nemondpi/)
- [Network Traffic Classification for Cybersecurity and Monitoring](https://fosdem.org/2022/schedule/event/using_ndpi_to_efficiently_classify_network_traffic/)
### nDPI-Related Projects
- [nfstream](https://github.com/aouinizied/nfstream)
- [nDPId](https://github.com/utoni/nDPId)
### DISCLAIMER
While we do our best to detect network protocols, we cannot guarantee that our software is error free and 100% accurate in protocol detection. Please make sure that you respect the privacy of users and you have proper authorization to listen, capture and inspect network traffic.
nDPI is a registered trademark in the US and EU.
-22
View File
@@ -1,22 +0,0 @@
Prerequisites for Compilation
-----------------------------
Prerequisites
- GNU autotools/libtool
- libpcap or PF_RING (optional but recommended)
- apt-get install libjson-c-dev (for JSON support)
- libgcrypt20-dev (for QUIC support)
On Ubuntu/Debian
- apt-get install build-essential
- apt-get install git autoconf automake autogen libpcap-dev libtool libgcrypt20-dev
On Fedora/CentOS
- yum groupinstall "Development tools"
- yum install git autoconf automake autogen libpcap-devel libtool libgcrypt-devel
On MacOSX (using http://brew.sh)
brew install autoconf automake libtool git libgcrypt
On FreeBSD
- pkg install autoconf automake libtool gmake git libgcrypt
+11 -5
View File
@@ -8,32 +8,38 @@ LIBTOOL=$(command -v libtool)
LIBTOOLIZE=$(command -v libtoolize)
AUTORECONF=$(command -v autoreconf)
PKG_CONFIG=$(command -v pkg-config)
MAKE=$(command -v make)
if test -z $AUTOCONF; then
if test -z "${AUTOCONF}"; then
echo "autoconf is missing: please install it and try again"
exit
fi
if test -z $AUTOMAKE; then
if test -z "${AUTOMAKE}"; then
echo "automake is missing: please install it and try again"
exit
fi
if test -z $LIBTOOL && test -z $LIBTOOLIZE ; then
if test -z "${LIBTOOL}" && test -z "${LIBTOOLIZE}"; then
echo "libtool and libtoolize is missing: please install it and try again"
exit
fi
if test -z $AUTORECONF; then
if test -z "${AUTORECONF}"; then
echo "autoreconf is missing: please install it and try again"
exit
fi
if test -z $PKG_CONFIG; then
if test -z "${PKG_CONFIG}"; then
echo "pkg-config is missing: please install it (apt-get install pkg-config) and try again"
exit
fi
if test -z "${MAKE}"; then
echo "make is missing; please install it (apt-get install make) and try again"
exit
fi
autoreconf -ivf
echo "./configure $@"
+235 -70
View File
@@ -1,4 +1,4 @@
AC_INIT([libndpi], [4.1.0])
AC_INIT([libndpi],[4.9.0])
AC_CONFIG_AUX_DIR([.])
AC_CONFIG_MACRO_DIR([m4])
@@ -7,36 +7,112 @@ AM_INIT_AUTOMAKE([foreign subdir-objects])
AC_PREFIX_DEFAULT(/usr)
EXTRA_TARGETS="example tests tests/unit tests/dga"
AC_ARG_WITH(only-libndpi, AS_HELP_STRING([--with-only-libndpi], [Build only libndpi (no examples, tests etc)]))
EXTRA_TARGETS="example tests tests/dga"
AC_ARG_WITH(only-libndpi, AS_HELP_STRING([--with-only-libndpi], [Build only libndpi (no examples, tests etc)]))
AS_IF([test "${with_only_libndpi+set}" = set],[
EXTRA_TARGETS=""
])
AC_ARG_WITH(sanitizer, AS_HELP_STRING([--with-sanitizer], [Build with support for address, undefined and leak sanitizer]))
PWD=`pwd`
AC_ARG_WITH(sanitizer, AS_HELP_STRING([--with-sanitizer], [Build with support for address, undefined and leak sanitizer]))
AC_ARG_WITH(thread-sanitizer, AS_HELP_STRING([--with-thread-sanitizer], [Build with support for thread sanitizer]))
AC_ARG_WITH(memory-sanitizer, AS_HELP_STRING([--with-memory-sanitizer], [Build with support for memory sanitizer]))
AC_ARG_WITH(macos-memory-sanitizer, AS_HELP_STRING([--with-macos-memory-sanitizer], [Build with support for memory sanitizer macOS]))
AC_ARG_ENABLE(fuzztargets, AS_HELP_STRING([--enable-fuzztargets], [Enable fuzz targets]),[enable_fuzztargets=$enableval],[enable_fuzztargets=no])
AS_IF([test "x$enable_fuzztargets" = "xyes"], [BUILD_FUZZTARGETS=1], [BUILD_FUZZTARGETS=0])
AC_ARG_ENABLE(gprof, AS_HELP_STRING([--enable-gprof], [Enable CPU/HEAP profiling with gperftools]),[enable_gprof=$enableval],[enable_gprof=no])
AC_ARG_ENABLE(code-coverage, AS_HELP_STRING([--enable-code-coverage], [Generate Code Coverage report]))
AC_ARG_WITH(local-libgcrypt, AS_HELP_STRING([--with-local-libgcrypt], [Build with libgcrypt (if present) instead of the enclosed gcrypt light]))
AC_ARG_ENABLE(tls-sigs, AS_HELP_STRING([--enable-tls-sigs], [Enable TLS Client signature algorithm dissection. Rarely used, but requires significantly more memory.]))
AC_ARG_ENABLE(npcap, AS_HELP_STRING([--disable-npcap], [msys2 only: Disable linkage against the wpcap/npcap import library in windows/WpdPack/Lib.]))
AC_ARG_WITH(nbpf-path, AS_HELP_STRING([--with-nbpf-path], [nBPF library custom path; default: ${PWD}/../PF_RING/userland/nbpf]),[NBPF_HOME=$withval],[NBPF_HOME=${PWD}/../PF_RING/userland/nbpf])
AC_ARG_WITH(lto-and-gold-linker, AS_HELP_STRING([--with-lto-and-gold-linker], [Build with LTO and Gold linker]))
AC_ARG_ENABLE(debug-build, AS_HELP_STRING([--enable-debug-build], [Enable debug build (`-g` flag)]),[enable_debugbuild=$enableval],[enable_debugbuild=no])
AC_ARG_ENABLE(global-context-support, AS_HELP_STRING([--disable-global-context-support], [Disable support for global context. No external dependency on libpthread]))
NDPI_CFLAGS="${NDPI_CFLAGS} -D_DEFAULT_SOURCE=1 -D_GNU_SOURCE=1"
AS_IF([test "x$enable_fuzztargets" = "xyes"], [
BUILD_FUZZTARGETS=1
NDPI_CFLAGS="${NDPI_CFLAGS} -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION"
], [BUILD_FUZZTARGETS=0])
AM_CONDITIONAL([BUILD_FUZZTARGETS], [test "x$enable_fuzztargets" = "xyes"])
AS_IF([test "${with_sanitizer+set}" = set],[
CFLAGS="${CFLAGS} -g3 -O0 -fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fsanitize=leak -fno-omit-frame-pointer"
LDFLAGS="${LDFLAGS} -fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fsanitize=leak"
AS_IF([test "x$enable_debugbuild" = "xyes"], [
NDPI_CFLAGS="${NDPI_CFLAGS} -g"
])
AS_IF([test "${with_sanitizer+set}" = set -a "${with_thread_sanitizer+set}" = set],[
AC_MSG_ERROR([Configure options `--with-sanitizer' and `--with_thread_sanitizer' can not used at the same time.])
])
AS_IF([test "${with_sanitizer+set}" = set -a "${with_memory_sanitizer+set}" = set],[
AC_MSG_ERROR([Configure options `--with-sanitizer' and `--with_memory_sanitizer' can not used at the same time.])
])
AS_IF([test "${with_thread_sanitizer+set}" = set -a "${with_memory_sanitizer+set}" = set],[
AC_MSG_ERROR([Configure options `--with_thread_sanitizer' and `--with_memory_sanitizer' can not used at the same time.])
])
AS_IF([test "${with_sanitizer+set}" = set -o "${with_thread_sanitizer+set}" = set -o "${with_memory_sanitizer+set}" = set],[
NDPI_CFLAGS="${NDPI_CFLAGS} -O0 -g3"
],[
dnl> Oss-fuzz doesn't really like any optimizaton flags don't set by itself
AS_IF([test "x$enable_fuzztargets" != "xyes"], [NDPI_CFLAGS="${NDPI_CFLAGS} -O2"])
])
AS_IF([test "${with_sanitizer+set}" = set -o "${with_thread_sanitizer+set}" = set -o "${with_memory_sanitizer+set}" = set],[
AS_IF([test "x$enable_gprof" = "xyes"], [
AC_MSG_ERROR([Configure options `--with-sanitizer' / `--with_thread_sanitizer' / `--with_memory_sanitizer' can not used together with `--enable-gprof'.])
])
])
AS_IF([test "${with_sanitizer+set}" = set],[
NDPI_CFLAGS="${NDPI_CFLAGS} -fsanitize=address -fsanitize=undefined -fsanitize=leak -fno-omit-frame-pointer"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fsanitize=address -fsanitize=undefined -fsanitize=leak"
#Sanitizers should work on any compilers that we support (or that we test on CI, at least)
#Exception: "-fsanitize=alignment" is not supported in gcc 4.9
AX_CHECK_COMPILE_FLAG([-fno-sanitize=alignment], [
NDPI_CFLAGS="${NDPI_CFLAGS} -fno-sanitize=alignment"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fno-sanitize=alignment"
])
])
AS_IF([test "${with_thread_sanitizer+set}" = set],[
NDPI_CFLAGS="${NDPI_CFLAGS} -fsanitize=thread -fno-omit-frame-pointer"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fsanitize=thread"
])
AS_IF([test "${with_memory_sanitizer+set}" = set],[
NDPI_CFLAGS="${NDPI_CFLAGS} -fsanitize=memory -fsanitize-memory-track-origins -fno-omit-frame-pointer"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fsanitize=memory"
])
AS_IF([test "${with_macos_memory_sanitizer+set}" = set],[
NDPI_CFLAGS="${NDPI_CFLAGS} -fsanitize=address -fno-omit-frame-pointer -fsanitize=signed-integer-overflow -fno-sanitize-recover=address"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fsanitize=address"
])
AS_IF([test "x${enable_code_coverage}" = "xyes"],[
NDPI_CFLAGS="${NDPI_CFLAGS} -fprofile-arcs -ftest-coverage"
NDPI_LDFLAGS="${NDPI_LDFLAGS} --coverage"
])
AS_IF([test "${enable_npcap+set}" = set],[DISABLE_NPCAP=1],[DISABLE_NPCAP=0])
LT_INIT
LT_LIB_M
PKG_PROG_PKG_CONFIG
AC_PROG_CC
AC_PROG_CPP_WERROR
AC_C_INLINE
SYSTEM=`uname -s`
if test $SYSTEM = "Darwin"; then
dnl> AC_PROG_CC(clang gcc)
AM_PROG_CC_C_O(clang gcc)
AC_PROG_CXX(clang++ g++)
AC_PROG_CC_STDC(clang gcc)
else
AC_PROG_CC_C_O
AC_PROG_CXX
AC_PROG_CC_STDC
fi
dnl> Can't iuse AM_PROG_AR because not all of our Makefiles are automake (yet?)
@@ -44,6 +120,29 @@ AC_CHECK_TOOL(AR, ar, [false])
AC_LANG_WERROR
AC_MSG_CHECKING([whether roaring v3 works])
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#if __has_include (<stdio.h>)
#endif
]])],
[
AC_MSG_RESULT([yes])
],
[
AC_MSG_RESULT([no])
AC_DEFINE_UNQUOTED(USE_ROARING_V2, "1", [Use CRoaring 2.1.x])
])
GCC_VERSION=`gcc --version | cut -d ' ' -f 3 | head -1|cut -d '.' -f 1`
if test "${GCC_VERSION}" == "version" ; then
GCC_VERSION=`gcc --version | cut -d ' ' -f 4 | head -1|cut -d '.' -f 1`
fi
if test "${GCC_VERSION}" -lt "7" ; then
AC_DEFINE_UNQUOTED(USE_ROARING_V2, "1", [Use CRoaring 2.1.x])
fi
NDPI_MAJOR=`echo "${PACKAGE_VERSION}" | cut -d . -f 1`
NDPI_MINOR=`echo "${PACKAGE_VERSION}" | cut -d . -f 2`
NDPI_PATCH=`echo "${PACKAGE_VERSION}" | cut -d . -f 3`
@@ -55,9 +154,9 @@ AC_DEFINE_UNQUOTED(NDPI_PATCH_LEVEL, "${NDPI_PATCH}", [nDPI patch level])
# .git as directory in a cloned repo
# .git as file in submodule based integration
if test -d ".git" || test -f ".git" ; then :
GIT_TAG=`git log -1 --format=%h`
GIT_DATE=`git log -1 --format=%cd`
if test -d ".git" || test -r ".git"; then
GIT_TAG=`git log -1 --abbrev=7 --format=%h`
GIT_DATE=`git log -1 --abbrev=7 --format=%cd`
#
# On CentOS 6 `git rev-list HEAD --count` does not work
#
@@ -67,7 +166,7 @@ if test -d ".git" || test -f ".git" ; then :
A=`git log src/include/ndpi_typedefs.h|wc -l`
B=`git log src/include/ndpi_protocol_ids.h|wc -l`
C=`git log src/include/ndpi_api.h.in|wc -l`
C=`git log src/include/ndpi_api.h|wc -l`
NDPI_API_VERSION=$((A+B+C))
else
GIT_RELEASE="${PACKAGE_VERSION}"
@@ -75,35 +174,60 @@ else
if test -z "$SOURCE_DATE_EPOCH" ; then :
SOURCE_DATE_EPOCH=`date +%s`
fi
NDPI_API_VERSION=`echo $SOURCE_DATE_EPOCH | cut -c7-10`
NDPI_API_VERSION=$((SOURCE_DATE_EPOCH % 65536))
fi
NDPI_API_VERSION=`echo $NDPI_API_VERSION | sed 's/^0*//'`
echo "Setting API version to $NDPI_API_VERSION"
AS_IF([test "x${NDPI_API_VERSION}" = "x"],[
AC_MSG_WARN([Could not get NDPI_API_VERSION, defaulting to 0.])
NDPI_API_VERSION=0
])
echo "Setting API version to ${NDPI_API_VERSION}"
AC_DEFINE_UNQUOTED(NDPI_GIT_RELEASE, "${GIT_RELEASE}", [GIT Release])
AC_DEFINE_UNQUOTED(NDPI_GIT_DATE, "${GIT_DATE}", [Last GIT change])
if ! test "${with_only_libndpi+set}" = set; then :
dnl> used by json-c for unit tests
PKG_CHECK_MODULES([JSONC], [json-c], [JSONC_LIBS="${pkg_cv_JSONC_LIBS}" JSONC_CFLAGS="${pkg_cv_JSONC_CFLAGS}"])
AC_CHECK_LIB([json-c], [json_object_put], [], [
PKG_CHECK_MODULES([JSONC], [json-c], [
AC_DEFINE(HAVE_LIBJSON_C, 1, [libjson-c is present])
JSONC_LIBS="${pkg_cv_JSONC_LIBS}"
JSONC_CFLAGS="${pkg_cv_JSONC_CFLAGS}"
], [AC_MSG_WARN([JSON-C not available. Disabled unit test.])])
AC_CHECK_LIB([json-c], [json_object_put], [
EXTRA_TARGETS="$EXTRA_TARGETS tests/unit"
build_unittests=yes
],[
AC_MSG_WARN([JSON-C not available. Disabled unit test.])
JSONC_LIBS=""
JSONC_CFLAGS=""
EXTRA_TARGETS="example tests tests/dga"
])
AX_PTHREAD([],[
AC_MSG_WARN([POSIX Threads not available. Building library only.])
JSONC_LIBS=""
JSONC_CFLAGS=""
EXTRA_TARGETS=""
])
dnl> AX_PTHREAD test does not work along with MingW/MSYS anymore
case "$host" in
*-*-mingw32*|*-*-msys)
;;
*)
AX_PTHREAD([],[
AC_MSG_WARN([POSIX Threads not available. Building library only.])
JSONC_LIBS=""
JSONC_CFLAGS=""
EXTRA_TARGETS=""
])
;;
esac
fi
AM_CONDITIONAL([BUILD_UNITTESTS], [test "x$build_unittests" = "xyes"])
LIBS="$LIBS $JSONC_LIBS"
CFLAGS="-W -Wall -Wno-unused-parameter -Wno-unused-function $CFLAGS $JSONC_CFLAGS"
HANDLE_TLS_SIGS="//"
AS_IF([test "x${enable_tls_sigs}" = "xyes"],[
HANDLE_TLS_SIGS=""
])
NDPI_CFLAGS="-W -Wall -Wno-address-of-packed-member ${NDPI_CFLAGS}"
AS_IF([test "${with_lto_and_gold_linker+set}" = set], [
NDPI_CFLAGS="${NDPI_CFLAGS} -flto -fuse-ld=gold -Wno-unused-command-line-argument"
NDPI_LDFLAGS="${NDPI_LDFLAGS} ${NDPI_CFLAGS}"
])
AC_CHECK_HEADERS([netinet/in.h stdint.h stdlib.h string.h unistd.h math.h float.h])
AC_CHECK_LIB([m], [sqrt], [], [LIBM="-lm"])
@@ -114,7 +238,15 @@ if test "x$ac_cv_lib_rrd_rrd_fetch_r" = xyes; then :
EXTRA_TARGETS="$EXTRA_TARGETS rrdtool"
fi
ADDITIONAL_LIBS="$LIBRRD $LIBM"
GLOBAL_CONTEXT_ENABLED=0
AS_IF([test "x$enable_global_context_support" != "xno"], [
AC_CHECK_LIB([pthread], [pthread_mutex_init])
NDPI_CFLAGS="${NDPI_CFLAGS} -DUSE_GLOBAL_CONTEXT"
GLOBAL_CONTEXT_ENABLED=1
])
ADDITIONAL_INCS=
ADDITIONAL_LIBS="$LIBM"
PCAP_HOME=$HOME/PF_RING/userland
DPDK_TARGET=
@@ -138,17 +270,25 @@ fi
MACHINE=`uname -m`
CUSTOM_NDPI=
NBPF_ENABLED=0
AC_MSG_CHECKING([PF_RING nBPF ($NBPF_HOME)])
LIBNBPF=${NBPF_HOME}/libnbpf.a
if test -f ${LIBNBPF}; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} ${LIBNBPF}"
ADDITIONAL_INCS="${ADDITIONAL_INCS} -I${NBPF_HOME}"
AC_DEFINE_UNQUOTED(HAVE_NBPF, "1", [PF_RING nBPF is present])
AC_MSG_RESULT(yes)
NBPF_ENABLED=1
else
AC_MSG_RESULT(no)
fi
CUSTOM_NDPI=
if test -d ../nDPI-custom; then :
CUSTOM_NDPI="-DCUSTOM_NDPI_PROTOCOLS"
AC_MSG_RESULT([Compiling with custom nDPI protocols])
fi
if test -d ../ndpi-pro; then :
with_maxminddb=1
fi
case "$host" in
*-*-mingw32*|*-*-msys)
PCAP_INC=""
@@ -156,16 +296,22 @@ case "$host" in
case "$host" in
x86_64-*)
BUILD_MINGW_X64=1
NDPI_LDFLAGS="${NDPI_LDFLAGS} -L/mingw64/lib"
;;
i686-*)
NDPI_LDFLAGS="${NDPI_LDFLAGS} -L/mingw32/lib"
;;
*)
AC_MSG_ERROR([Unsupported MingW cross compiler.])
;;
esac
CFLAGS="${CFLAGS}"
NDPI_CFLAGS="-D__USE_MINGW_ANSI_STDIO -D__STDC_FORMAT_MACROS ${NDPI_CFLAGS}"
LIBS="${LIBS} -lws2_32"
BUILD_MINGW=1
EXE_SUFFIX=".exe"
AS_IF([test "${enable_npcap+set}" != set && test "${with_only_libndpi+set}" != set],,
[PKG_CHECK_MODULES([PCAP], [libpcap], [PCAP_LIB="" PCAP_INC="${pkg_cv_PCAP_CFLAGS}"])
AC_CHECK_LIB([pcap], [pcap_open_live],, [AC_MSG_ERROR([Missing msys2/mingw libpcap library. Install it with `pacman -S mingw-w64-x86_64-libpcap' (msys2).])])])
;;
*)
if test -f $PCAP_HOME/libpcap/libpcap.a; then :
@@ -190,6 +336,8 @@ case "$host" in
AC_MSG_ERROR([Missing libpcap(-dev) library required to compile the example application.])
fi
fi
EXE_SUFFIX=""
AS_IF([test "${enable_npcap+set}" = set],AC_MSG_ERROR([--disable-npcap does only work with msys2/mingw CCs]))
;;
esac
@@ -228,51 +376,55 @@ return 0;
])
AM_CONDITIONAL([HAS_FUZZLDFLAGS], [test "x$has_sanitizefuzzer" = "xyes"])
AC_CHECK_LIB(pthread, pthread_setaffinity_np, AC_DEFINE_UNQUOTED(HAVE_PTHREAD_SETAFFINITY_NP, 1, [libc has pthread_setaffinity_np]))
dnl> GCRYPT
GCRYPT_ENABLED=1
AC_ARG_ENABLE([gcrypt],
[AS_HELP_STRING([--disable-gcrypt], [Avoid compiling with libgcrypt/libgpg-error, even if they are present. QUIC sub-classification may be missing])],
[GCRYPT_ENABLED=0],
[AC_CHECK_LIB(gcrypt, gcry_cipher_checktag)])
AS_IF([test ${GCRYPT_ENABLED} -eq 1], [
if test "x$ac_cv_lib_gcrypt_gcry_cipher_checktag" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lgcrypt"
else
$as_unset ac_cv_lib_gcrypt_gcry_cipher_checktag
AC_CHECK_LIB(gpg-error, gpg_strerror_r)
AC_CHECK_LIB(gcrypt, gcry_cipher_checktag)
if test "x$ac_cv_lib_gcrypt_gcry_cipher_checktag" = xyes -a "x$ac_cv_lib_gpg_error_gpg_strerror_r" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lgcrypt -lgpg-error"
else
GCRYPT_ENABLED=0
fi
fi
GPROF_ENABLED=0
AS_IF([test "x$enable_gprof" = "xyes"], [
PKG_CHECK_MODULES([PROFILER], [libprofiler],,[
AC_MSG_ERROR([libprofiler not available. Required for profiling support.])
])
PKG_CHECK_MODULES([TCMALLOC], [libtcmalloc],,[
AC_MSG_ERROR([libtcmalloc not available. Required for profiling support.])
])
GPROF_ENABLED=1
GPROF_LIBS="-ltcmalloc_and_profiler"
GPROF_CFLAGS="-g3 -fvisibility=default -fno-omit-frame-pointer ${pkg_cv_PROFILER_CFLAGS} ${pkg_cv_TCMALLOC_CFLAGS}"
])
dnl> PCRE
PCRE_ENABLED=0
AC_ARG_WITH(pcre, [ --with-pcre Enable nDPI build with libpcre])
if test "${with_pcre+set}" = set; then :
AC_CHECK_LIB(pcre, pcre_compile, AC_DEFINE_UNQUOTED(HAVE_PCRE, 1, [libpcre(-dev) is present]))
if test "x$ac_cv_lib_pcre_pcre_compile" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lpcre"
PCRE_ENABLED=1
AC_CHECK_LIB(pthread, pthread_setaffinity_np, AC_DEFINE_UNQUOTED(HAVE_PTHREAD_SETAFFINITY_NP, 1, [libc has pthread_setaffinity_np]))
dnl> libgcrypt (external)
USE_HOST_LIBGCRYPT=0
AS_IF([test "${with_local_libgcrypt+set}" = set],[
USE_HOST_LIBGCRYPT=1
AC_CHECK_LIB(gpg-error, gpg_strerror_r, [], AC_MSG_ERROR([libgpg-error required (because of --with-local-libgcrypt) but not found or too old.]))
AC_CHECK_LIB(gcrypt, gcry_cipher_checktag, [], AC_MSG_ERROR([libgcrypt required (because of --with-local-libgcrypt) but not found or too old.]))
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lgcrypt -lgpg-error"
AC_DEFINE_UNQUOTED(USE_HOST_LIBGCRYPT, 1, [Use locally installed libgcrypt instead of builtin gcrypt-light])
])
dnl> PCRE2
PCRE2_ENABLED=0
AC_ARG_WITH(pcre2, AS_HELP_STRING([--with-pcre2], [Enable nDPI build with libpcre2]))
if test "${with_pcre2+set}" = set; then :
AC_CHECK_LIB(pcre2-8, pcre2_compile_8, AC_DEFINE_UNQUOTED(HAVE_PCRE2, 1, [libpcre2(-dev) is present]))
if test "x$ac_cv_lib_pcre2_8_pcre2_compile_8" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lpcre2-8"
PCRE2_ENABLED=1
fi
fi
dnl> GeoIP
AC_ARG_WITH(maxminddb, [ --with-maxminddb Enable nDPI build with libmaxminddb])
AC_ARG_WITH(maxminddb, AS_HELP_STRING([--with-maxminddb], [Enable nDPI build with libmaxminddb]))
if test "${with_maxminddb+set}" = set; then :
AC_CHECK_LIB([maxminddb], [MMDB_lookup_sockaddr])
AC_HAVE_HEADERS(maxminddb.h)
AC_CHECK_HEADERS([maxminddb.h])
if test ".${ac_cv_lib_maxminddb_MMDB_lookup_sockaddr}" = ".yes" && test ".${ac_cv_header_maxminddb_h}" = ".yes"; then
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lmaxminddb"
AC_DEFINE_UNQUOTED(HAVE_MAXMINDDB, 1, [MaxMind DB support])
fi
fi
AC_DEFINE_UNQUOTED(NDPI_BASE_DIR, "${PWD}", [nDPI base directory])
dnl> Curl
dnl> AC_CHECK_LIB([curl], [curl_easy_perform])
dnl> if test ${ac_cv_lib_curl_curl_easy_perform} = "no"; then
@@ -282,7 +434,7 @@ dnl> ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lcurl"
dnl> AC_DEFINE_UNQUOTED(HAVE_CURL, 1, [curl is present])
dnl> fi
AC_CONFIG_FILES([Makefile example/Makefile example/Makefile.dpdk tests/Makefile tests/unit/Makefile tests/performance/Makefile tests/dga/Makefile rrdtool/Makefile libndpi.pc src/include/ndpi_define.h src/lib/Makefile python/Makefile fuzz/Makefile src/include/ndpi_api.h])
AC_CONFIG_FILES([Makefile example/Makefile example/Makefile.dpdk tests/Makefile tests/unit/Makefile tests/performance/Makefile tests/dga/Makefile rrdtool/Makefile influxdb/Makefile libndpi.pc src/include/ndpi_define.h src/lib/Makefile fuzz/Makefile doc/Doxyfile.cfg])
AC_CONFIG_FILES([tests/do.sh], [chmod +x tests/do.sh])
AC_CONFIG_HEADERS(src/include/ndpi_config.h)
AC_SUBST(GIT_RELEASE)
@@ -292,6 +444,8 @@ AC_SUBST(NDPI_PATCH)
AC_SUBST(NDPI_VERSION_SHORT)
AC_SUBST(PCAP_INC)
AC_SUBST(PCAP_LIB)
AC_SUBST(LIBRRD)
AC_SUBST(ADDITIONAL_INCS)
AC_SUBST(ADDITIONAL_LIBS)
AC_SUBST(DL_LIB)
AC_SUBST(DPDK_TARGET)
@@ -304,6 +458,17 @@ AC_SUBST(BUILD_MINGW_X64)
AC_SUBST(BUILD_FUZZTARGETS)
AC_SUBST(JSONC_CFLAGS)
AC_SUBST(JSONC_LIBS)
AC_SUBST(GCRYPT_ENABLED)
AC_SUBST(PCRE_ENABLED)
AC_SUBST(GPROF_CFLAGS)
AC_SUBST(GPROF_LIBS)
AC_SUBST(GPROF_ENABLED)
AC_SUBST(USE_HOST_LIBGCRYPT)
AC_SUBST(PCRE2_ENABLED)
AC_SUBST(NBPF_ENABLED)
AC_SUBST(GLOBAL_CONTEXT_ENABLED)
AC_SUBST(HANDLE_TLS_SIGS)
AC_SUBST(DISABLE_NPCAP)
AC_SUBST(EXE_SUFFIX)
AC_SUBST(NDPI_CFLAGS)
AC_SUBST(NDPI_LDFLAGS)
AC_SUBST(NDPI_BASE_DIR)
AC_OUTPUT
+2658
View File
File diff suppressed because it is too large Load Diff
+36
View File
@@ -0,0 +1,36 @@
# FAQ
From https://www.ntop.org/ndpi/ndpi-internals-and-frequent-questions/
**Q**: How nDPI implements protocol detection?\
**A**: nDPI includes a list of protocol dissectors (364 as of today) that are able to dissect protocols such as WhatsApp or TLS. As soon as a new flow is submitted to nDPI, the library applies in sequence dissectors that can potentially match the protocols (i.e. telnet is a TCP-based protocol and it will not be considered for UDP flows). We start from the dissector that can most probably match using the port number. This means for traffic on TCP/22 nDPI will start with the SSH dissectors and if not matching continue with the others. Dissection completes as soon as a protocol matches or when none of them matched and in this case the flow will be labelled as Unknown.
**Q**: What is the nDPI release cycle?\
**A**: We cut release a approximately every 6-8 months, fixes and improvements are on a daily basis (check the nDPI code on GitHub).
**Q**: Is nDPI running on all popular platforms?\
**A**: Yes it runs on Linux, macOS, Windows… and also on not-so-popular ones such as IBM mainframes. We support ARM, Intel, RISC… architectures.
**Q**: How many packets does nDPI need in order to implement detection?\
**A**: It depends on the protocol. For UDP-based protocols such as DNS one packet is enough, for more complex protocols such as TLS about 10 packets. For sure if after 15-20 packets nDPI has not detected the application protocol, then the protocol is labelled as Unknown.
**Q**: Is nDPI detection only based on protocol dissectors?\
**A**: No, payload inspection is the main technique, but nDPI can also use IP address, ports, TLS certificates etc as signatures for protocols. In this case, after detection is complete, nDPI will report if the match was performed on payload inspection or other means (e.g. IP address).
**Q**: Does nDPI contain list of known IP addresses?\
**A**: Yes it includes lists of well known IPs such as those provided by Microsoft of Meta for identifying known service.
**Q**: Can I extend nDPI by defining new protocols with a configuration file?\
**A**: Yes you can. See this [file](https://github.com/ntop/nDPI/blob/dev/example/protos.txt) as an example for defining new protocols..
**Q**: Is nDPI able to detect VPNs?\
**A**: Yes it can detect VPNS such as Tailscale, WireGuard, OpenVPN, FortiClient.. and also in-app VPNs such as UltraSurf or OperaVPN.
**Q**: Is nDPI able to detect malware and viruses?\
**A**: It can detect anomalous behaviour that can be caused by a malware, but nDPI is not a signature-based tool so it does not include signatures for malware A or B. This is because [signature-based tools](https://en.wikipedia.org/wiki/Intrusion_detection_system) have various limitations and resource intensive, whereas nDPI has been designed to be used also in high-speed (100 Gbit+) networks.
**Q**: Is nDPI able to detect security issues?\
**A**: Yes it can by means of a technique called [flow risk](https://github.com/ntop/nDPI/blob/dev/doc/flow_risks.rst). It can identify 50+ threats (e.g. a host that is talking with a malware host).
**Q**: Is nDPI able to block traffic?\
**A**: No, nDPI is a passive traffic analysis library that does not manipulate packets. You can create applications on top of it for policing (i.e. blocking or shaping) traffic. Examples of such applications are ntopng Edge, nProbe IPS and nProbe Cento.
+8 -5
View File
@@ -22,8 +22,8 @@ I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " release to make standalone HTML files for ntop documentation release"
@echo " html to make standalone HTML files"
@echo " release to make standalone Doxygen/HTML files for ntop documentation release"
@echo " html to make standalone Doxygen/HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " singlehtml to make a single large HTML file"
@echo " pickle to make pickle files"
@@ -50,12 +50,15 @@ help:
.PHONY: clean
clean:
rm -rf $(BUILDDIR)/*
cd doxygen; make clean
rm -rf $(BUILDDIR)/* api doxygen
.PHONY: html
html:
cd doxygen; make
@test -r Doxyfile.cfg || { \
echo 'Doxyfile.cfg does not exist; Please run $(realpath ..)/autogen.sh or $(realpath ..)/configure first.'; \
false; \
}
doxygen Doxyfile.cfg
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
-4
View File
@@ -1,4 +0,0 @@
nDPI Programming API
####################
Placeholder
+16 -3
View File
@@ -21,7 +21,8 @@ sys.path.append( "breathe/" )
# ones.
extensions = [
'sphinx.ext.intersphinx',
'breathe'
'breathe',
'exhale'
]
# Workaround for platforms where swaggerdoc is not available
@@ -31,6 +32,18 @@ if not os.environ.get("PYTHON_SKIP_SWAGGERDOC"):
breathe_projects = { "apidoc" : "doxygen/xml/" }
breathe_default_project = "apidoc"
exhale_args = {
"containmentFolder": "./api",
"rootFileName": "library_root.rst",
"rootFileTitle": "Library API",
"doxygenStripFromPath": ".",
"createTreeView": True,
# "treeViewIsBootstrap": True,
}
primary_domain = 'c'
highlight_language = 'c'
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
@@ -139,7 +152,7 @@ html_logo = "img/logo.png"
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
#html_static_path = ['_static']
# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
@@ -288,4 +301,4 @@ texinfo_documents = [
intersphinx_mapping = {'https://docs.python.org/': None}
def setup(app):
app.add_stylesheet('css/ntop.css')
app.add_css_file('css/ntop.css')
+49
View File
@@ -0,0 +1,49 @@
# Configuration knobs
TODO
| Protocol | Parameter | Default value | Min value | Max value | Description | Notes |
| ------ | ------ | ------ | ------ | ------ | ------ | ------ |
| NULL | "packets_limit_per_flow" | 32 | 0 | 255 | The upper limit on the number of packets per flow that will be subject to DPI, after which classification will be considered complete (0 = no limit) |
| NULL | "flow.direction_detection" | enable | NULL | NULL | Enable/disable internal detection of packet direction (client to server or server to client) |
| NULL | "flow.track_payload" | disable | NULL | NULL | Enable/disable tracking/export of flow payload (i.e. L5/7 data): if enabled, the library exports the first 1024 bytes of payload for each flow |
| NULL | "tcp_ack_payload_heuristic" | disable | NULL | NULL | In some networks, there are some anomalous TCP flows where the smallest ACK packets have some kind of zero padding. It looks like the IP and TCP headers in those frames wrongly consider the 0x00 Ethernet padding bytes as part of the TCP payload. While this kind of packets is perfectly valid per-se, in some conditions they might be treated by the TCP reassembler logic as (partial) overlaps, deceiving the classification engine. This parameter enable/disable an heuristic to detect these packets and to ignore them, allowing correct detection/classification. See #1946 for other details |
| NULL | "fully_encrypted_heuristic" | enable | NULL | NULL | Enable/disable an heuristic to detect fully encrypted sessions, i.e. flows where every bytes of the payload is encrypted in an attempt to “look like nothing”. This heuristic only analyzes the first packet of the flow. See: https://www.usenix.org/system/files/sec23fall-prepub-234-wu-mingshi.pdf |
| NULL | "libgcrypt.init" | 1 | NULL | NULL | Enable/disable initialization of libgcrypt. When using the external libgcrypt (instead of the internal crypto code) the libgcrypt runtime must be initialized. If, for whatever reasons, the application alread does it, nDPI must be told to skip it. Note that, by default, nDPI uses the crypto code and not libgcrypt: in that case this parameter is ignored |
| NULL | "dpi.compute_entropy" | 1 | NULL | NULL | Enable/disable computation of flow entropy |
| NULL | "dpi.guess_on_giveup" | 0x03 | 0x00 | 0x03 | Tell the library to guess flow classification, if any DPI algorithms/logics fail. The value is a bitmask. Values: 0x0 = disabled; 0x01 = enable guessing by port; 0x02 = enable guessing by ip |
| NULL | "flow_risk_lists.load" | 1 | NULL | NULL | Enable/disable loading of every IP addresses lists used to check any flow risks |
| NULL | "flow_risk.anonymous_subscriber.list.icloudprivaterelay.load" | 1 | NULL | NULL | Enable/disable loading of internal iCouldPrivateRealy IP address list used to check `NDPI_ANONYMOUS_SUBSCRIBER` flow risk |
| NULL | "flow_risk.anonymous_subscriber.list.protonvpn.load" | 1 | NULL | NULL | Enable/disable loading of internal IP address list of ProtonVPN exit nodes used to check `NDPI_ANONYMOUS_SUBSCRIBER` flow risk |
| NULL | "flow_risk.crawler_bot.list.load" | 1 | NULL | NULL | Enable/disable loading of internal IP address list used to check `NDPI_HTTP_CRAWLER_BOT` flow risk |
| NULL | "filename.config" | NULL | NULL | NULL | Name of the file containing a list of configuration knobs itself (one per line)!. Useful to configure nDPI via text file instead of via API |
| NULL | "log.level" | 0 | 0 | 3 | Configure the log/debug level. Possible values: 0 = error, 1 = trace, 2 = debug, 3 = extra debug |
| NULL | "lru.$CACHE_NAME.size" | See description | 0 | 16777215 | Set the size (in number of elements) of the specified LRU cache (0 = the cache is disabled). The keyword "$CACHE_NAME" is a placeholder for the cache name and the possible values are: ookla, bittorrent, stun, tls_cert, mining, msteams. The default value is "32768" for the bittorrent cache and "1024" for all the other caches |
| NULL | "lru.$CACHE_NAME.ttl" | See description | 0 | 16777215 | Set the TTL (in seconds) for the elements of the specified LRU cache (0 = the elements never explicitly expire). The keyword "$CACHE_NAME" is a placeholder for the cache name and the possible values are: ookla, bittorrent, stun, tls_cert, mining, msteams. The default value is "120" for the ookla cache, "60" for the msteams cache and "0" for all the other caches |
| NULL | "lru.$CACHE_NAME.scope" | 0 | 0 | 1 | Set the scope of the specified LRU cache (0 = the cache is local, 1 = the cache is global). The keyword "$CACHE_NAME" is a placeholder for the cache name and the possible values are: ookla, bittorrent, stun, tls_cert, mining, msteams. The global scope con be set only if a global context has been initialized |
| "tls" | "certificate_expiration_threshold" | 30 | 0 | 365 | The threshold (in days) used to trigger the `NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE` flow risk |
| "tls" | "application_blocks_tracking" | disable | NULL | NULL | Enable/disable processing of TLS Application Blocks (post handshake) to extract statistical information about the flow |
| "tls" | "metadata.sha1_fingerprint" | enable | NULL | NULL | Enable/disable computation and export of SHA1 fingerprint for TLS flows. Note that if it is disable, the flow risk `NDPI_MALICIOUS_SHA1_CERTIFICATE` is not checked |
| "tls" | "metadata.ja3c_fingerprint" | enable | NULL | NULL | Enable/disable computation and export of JA3C fingerprint for TLS flows. Note that if it is disable, the flow risk `NDPI_MALICIOUS_JA3` is not checked |
| "tls" | "metadata.ja3s_fingerprint" | enable | NULL | NULL | Enable/disable computation and export of JA3S fingerprint for TLS flows |
| "tls" | "metadata.ja4c_fingerprint" | enable | NULL | NULL | Enable/disable computation and export of JA4C fingerprint for TLS flows |
| "smtp" | "tls_dissection" | enable | NULL | NULL | Enable/disable dissection of TLS packets in cleartext SMTP flows (because of opportunistic TLS, via STARTTLS msg) |
| "imap" | "tls_dissection" | enable | NULL | NULL | Enable/disable dissection of TLS packets in cleartext IMAP flows (because of opportunistic TLS, via STARTTLS msg) |
| "pop" | "tls_dissection" | enable | NULL | NULL | Enable/disable dissection of TLS packets in cleartext POP flows (because of opportunistic TLS, via STARTTLS msg) |
| "ftp" | "tls_dissection" | enable | NULL | NULL | Enable/disable dissection of TLS packets in cleartext FTP flows (because of opportunistic TLS, via AUTH TLS msg) |
| "stun" | "max_packets_extra_dissection" | 4 | 0 | 255 | After a flow has been classified has STUN, nDPI might analyse more packets to look for a sub-classification or for metadata. This parameter set the upper limit on the number of these packets |
| "stun" | "tls_dissection" | enable | NULL | NULL | Enable/disable dissection of TLS packets multiplexed into STUN flows |
| "stun" | "metadata.attribute.mapped_address" | enable | NULL | NULL | Enable/disable extraction of (xor)-mapped-address attribute for STUN flows. If it is disabled, STUN classification might be significant faster |
| "stun" | "metadata.attribute.response_origin" | enable | NULL | NULL | Enable/disable extraction of response-origin attribute for STUN flows. If it is disabled, STUN classification might be significant faster |
| "stun" | "metadata.attribute.other_address" | enable | NULL | NULL | Enable/disable extraction of other-address attribute for STUN flows. If it is disabled, STUN classification might be significant faster |
| "stun" | "metadata.attribute.relayed_address" | enable | NULL | NULL | Enable/disable extraction of (xor-)relayed-address attribute for STUN flows. If it is disabled, STUN classification might be significant faster |
| "stun" | "metadata.attribute.peer_address" | enable | NULL | NULL | Enable/disable extraction of (xor-)peer-address attribute for STUN flows. If it is disabled, STUN classification might be significant faster; however sub-classification capability might be negatively impacted |
| "dns" | "subclassification" | enable | NULL | NULL | Enable/disable sub-classification of DNS flows (via query/response domain name). If disabled, some flow risks are not checked |
| "dns" | "process_response" | enable | NULL | NULL | Enable/disable processing of DNS responses. By default, DNS flows are fully classified after the first request/response pair (or after the first response, if the request is missing). If this parameter is disabled, the flows are fully classified after the first packet, i.e. usually after the first request; in that case, some flow risks are not checked and some metadata are not exported |
| "http" | "process_response" | enable | NULL | NULL | Enable/disable processing of HTTP responses. By default, HTTP flows are usually fully classified after the first request/response pair. If this parameter is disabled, the flows are fully classified after the first request (or after the first response, if the request is missing); in that case, some flow risks are not checked and some metadata are not exported |
| "ookla" | "dpi.aggressiveness", | 0x01 | 0x00 | 0x01 | Detection aggressiveness for Ookla. The value is a bitmask. Values: 0x0 = disabled; 0x01 = enable heuristic for detection over TLS (via Ookla LRU cache) |
| "zoom" | "max_packets_extra_dissection" | 4 | 0 | 255 | After a flow has been classified has Zoom, nDPI might analyse more packets to look for a sub-classification or for metadata. This parameter set the upper limit on the number of these packets |
| "rtp" | "search_for_stun" | disable | NULL | NULL | After a flow has been classified as RTP or RTCP, nDPI might analyse more packets to look for STUN/DTLS packets, i.e. to try to tell if this flow is a "pure" RTP/RTCP flow or if the RTP/RTCP packets are multiplexed with STUN/DTLS. Useful for proper (sub)classification when the beginning of the flows are not captured or if there are lost packets in the the captured traffic. If enabled, nDPI requires more packets to process for each RTP/RTCP flow. |
| $PROTO_NAME | "log" | disable | NULL | NULL | Enable/disable logging/debug for specific protocol. Use "any" as protocol name if you want to easily enable/disable logging/debug for all protocols |
| $PROTO_NAME | "ip_list.load" | 1 | NULL | NULL | Enable/disable loading of internal list of IP addresses (used for (sub)classification) specific to that protocol. Use "any" as protocol name if you want to easily enable/disable all lists. This knob is valid only for the following protocols: Alibaba, Amazon AWS, Apple, Avast, Bloomberg, Cachefly, Cloudflare, Discord, Disney+, Dropbox, Edgecast, EpicGames, Ethereum, Facebook, Github, Google, Google Cloud, GoTo, Hotspot Shield, Hulu, Line, Microsoft 365, Microsoft Azure, Microsoft One Drive, Microsoft Outlook, Mullvad, Netflix, Nvidia, OpenDNS, ProtonVPN, RiotGames, Roblox, Skype/Teams, Starcraft, Steam, Teamviewer, Telegram, Tencent, Threema, TOR, Twitch, Twitter, UbuntuONE, VK, Yandex, Yandex Cloud, Webex, Whatsapp, Zoom |
+100 -13
View File
@@ -45,25 +45,25 @@ TLS/QUIC only: this risk is triggered when a `self-signed certificate <https://e
NDPI_TLS_OBSOLETE_VERSION
=========================
Risk 0triggered when TLS version is older than 1.1.
Risk triggered when TLS version is older than 1.1.
.. _Risk 008:
NDPI_TLS_WEAK_CIPHER
====================
Risk 0triggered when an unsafe TLS cipher is used. See `this page <https://community.qualys.com/thread/18212-how-does-qualys-determine-the-server-cipher-suites>`_ for a list of insecure ciphers.
Risk triggered when an unsafe TLS cipher is used. See `this page <https://community.qualys.com/thread/18212-how-does-qualys-determine-the-server-cipher-suites>`_ for a list of insecure ciphers.
.. _Risk 009:
NDPI_TLS_CERTIFICATE_EXPIRED
============================
Risk 0triggered when a TLS certificate is expired, i.e. the current date falls outside of the certificate validity dates.
Risk triggered when a TLS certificate is expired, i.e. the current date falls outside of the certificate validity dates.
.. _Risk 010:
NDPI_TLS_CERTIFICATE_MISMATCH
=============================
Risk 0triggered when a TLS certificate does not match the hostname we're accessing. Example you do http://www.aaa.com and the TLS certificate returned is for www.bbb.com.
Risk triggered when a TLS certificate does not match the hostname we're accessing. Example you do http://www.aaa.com and the TLS certificate returned is for www.bbb.com.
.. _Risk 011:
@@ -73,9 +73,9 @@ HTTP only: this risk is triggered whenever the user agent contains suspicious ch
.. _Risk 012:
NDPI_HTTP_NUMERIC_IP_HOST
NDPI_NUMERIC_IP_HOST
=========================
HTTP only: this risk is triggered whenever we're accessing a host using its IP rather than its symbolic name. Example http://1.2.3.4.
This risk is triggered whenever a HTTP/TLS/QUIC connection is using a literal IPv4 or IPv6 address as ServerName (TLS/QUIC; example: SNI=1.2.3.4) or as Hostname (HTTP; example: http://1.2.3.4.).
.. _Risk 013:
@@ -105,7 +105,7 @@ A `DGA <https://en.wikipedia.org/wiki/Domain_generation_algorithm>`_ is used to
NDPI_MALFORMED_PACKET
=====================
This risk is generated when a packet (e.g. a DNS packet) has an unexpected formt. This can indicate a protocol error or more often an attempt to jeopardize a valid protocol to carry other type of data.
This risk is generated when a packet (e.g. a DNS packet) has an unexpected format. This can indicate a protocol error or more often an attempt to jeopardize a valid protocol to carry other type of data.
.. _Risk 018:
@@ -141,7 +141,8 @@ This risk indicates that the protocol used is insecure and that a secure protoco
NDPI_DNS_SUSPICIOUS_TRAFFIC
===========================
This risk is returned when DNS traffic returns an unexpected/obsolete `record type <https://en.wikipedia.org/wiki/List_of_DNS_record_types>`_.
This risk is returned when DNS traffic returns an unexpected/obsolete `record type <https://en.wikipedia.org/wiki/List_of_DNS_record_types>`_
or when a suspicious query with a very long host name is detected.
.. _Risk 024:
@@ -195,13 +196,13 @@ This risk is set when the `ALPN <https://en.wikipedia.org/wiki/Application-Layer
NDPI_TLS_CERT_VALIDITY_TOO_LONG
===============================
From 01/09/2020 TLS certificates lifespan is limited to 13 months. This risk is triggered for certificates not respecting this directive.
From 01/09/2020 TLS certificates lifespan is limited to `13 months <https://www.appviewx.com/blogs/tls-certificate-lifespans-now-capped-at-13-months/>`_. This risk is triggered for certificates not respecting this directive.
.. _Risk 033:
NDPI_TLS_SUSPICIOUS_EXTENSION
=============================
This risk is triggered when the domain name (SNI extension) is not printable and thus it is a problem.
This risk is triggered when the domain name (SNI extension) is not printable and thus it is a problem. In TLS extensions can be dynamically specified by the client in the hello packet.
.. _Risk 034:
@@ -213,13 +214,13 @@ This risk is triggered when a TLS fatal alert is detected in the TLS flow. See `
NDPI_SUSPICIOUS_ENTROPY
=======================
This risk is used to detect suspicious data carried in ICMP packets whose entropy (used to measure how data is distributed, hence to indirectly guess the type of data carried on) is suspicious and thus that it can indicate a data leak.
This risk is used to detect suspicious data carried in ICMP packets whose entropy (used to measure how data is distributed, hence to indirectly guess the type of data carried on) is suspicious and thus that it can indicate a data leak. Suspicious values indicate random entropy or entropy that is similar to encrypted traffic. In the latter case, this can be a suspicious data exfiltration symptom.
.. _Risk 036:
NDPI_CLEAR_TEXT_CREDENTIALS
===========================
Clear text protocols are not bad per-se, but they should be avoided when they carry credentials as they can be intercepted by malicious users. This risk is triggered whenever clear text protocols (e.g. FTP, HTTP, IMAP...) contain credentials in clear text (read it as nDPI does not trigger this risk for HTTP connections that do not carry credentials).
Clear text protocols are not intrinsically bad, but they should be avoided when they carry credentials as they can be intercepted by malicious users. This risk is triggered whenever clear text protocols (e.g. FTP, HTTP, IMAP...) contain credentials in clear text (read it as nDPI does not trigger this risk for HTTP connections that do not carry credentials).
.. _Risk 037:
@@ -246,10 +247,96 @@ Additionally, some TLS protocol fields are checked for printable characters as w
NDPI_POSSIBLE_EXPLOIT
=====================
The risk is set whenever a possible exploit (e.g. Log4J/Log4Shell) is detected.
The risk is set whenever a possible exploit attempt (e.g. `Log4J/Log4Shell <https://en.wikipedia.org/wiki/Log4Shell>`_) is detected.
.. _Risk 041:
NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE
===================================
The risk is set whenever a TLS certificate is close to the expiration date.
.. _Risk 042:
NDPI_PUNYCODE_IDN
===================================
The risk is set whenever a domain name is specified in IDN format as they are sometimes used in `IDN homograph attacks <https://en.wikipedia.org/wiki/IDN_homograph_attack>`_.
.. _Risk 043:
NDPI_ERROR_CODE_DETECTED
===================================
The risk is set whenever an error code is detected in the underlying protocol (e.g. HTTP and DNS).
.. _Risk 044:
NDPI_HTTP_CRAWLER_BOT
===================================
The risk is set whenever a crawler/bot/robot has been detected
.. _Risk 045:
NDPI_ANONYMOUS_SUBSCRIBER
===================================
The risk is set whenever the (source) IP address has been anonymized and it can't be used to identify the subscriber.
Example: the flow is generated by an iCloud-private-relay exit node.
.. _Risk 046:
NDPI_UNIDIRECTIONAL_TRAFFIC
===================================
The risk is set whenever the flow has unidirectional traffic (typically no traffic on the server to client direction). THis
risk is not triggered for multicast/broadcast destinations.
.. _Risk 047:
NDPI_HTTP_OBSOLETE_SERVER
===================================
This risk is generated whenever a HTTP server uses an obsolete HTTP server version.
.. _Risk 048:
NDPI_PERIODIC_FLOW
==================
This risk is generated whenever a flow is observed at a specific periodic pace (e.g. every 10 seconds).
.. _Risk 049:
NDPI_MINOR_ISSUES
=================
Minor packet/flow issues (e.g. DNS traffic with zero TTL) have been detected.
.. _Risk 050:
NDPI_TCP_ISSUES
===============
Relevant TCP connection issues such as connection refused, scan, or probe attempt.
.. _Risk 051
NDPI_FULLY_ENCRYPTED
====================
Flow with Unknown protocol containing encrypted traffic.
.. _Risk 052
NDPI_TLS_ALPN_SNI_MISMATCH
=========================
Invalid TLS ALPN/SNI mismatch. For instance ALPN advertises the flow as h2 (HTTP/2.0) and no SNI is reported.
.. _Risk 053
NDPI_MALWARE_CONTACTED
======================
Client contacted a server host labelled as malware.
.. _Risk 054:
NDPI_BINARY_DATA_TRANSFER
============================
HTTP only: this risk indicates that a binary file/data application transfer (attempt).
.. _Risk 055:
NDPI_PROBING_ATTEMPT
====================
Connection with no data exchagef that looks like a probing attempt
+4 -2
View File
@@ -13,7 +13,8 @@ nDPI is an open source DPI (Deep Packet Inspection) toolkit for traffic analysis
:maxdepth: 2
:caption: Developer's Guide
api
api/library_root
protocols
flow_risks
.. toctree::
@@ -27,7 +28,8 @@ nDPI is an open source DPI (Deep Packet Inspection) toolkit for traffic analysis
nEdge <https://www.ntop.org/guides/nedge/>
nScrub <https://www.ntop.org/guides/nscrub/>
nBox <https://www.ntop.org/guides/nbox/>
nTap <https://www.ntop.org/guides/ntap/>
.. Indices and tables
.. ==================
..
+84
View File
@@ -0,0 +1,84 @@
A simple, common example
```
struct ndpi_detection_module_struct *ndpi_struct;
ndpi_cfg_error rc;
int ret;
ndpi_struct = ndpi_init_detection_module(NULL);
if(!ndpi_struct) {
ERROR;
}
/* Configuration */
rc = ndpi_set_config(ndpi_struct, "tls", "certificate_expiration_threshold", "10");
if(rc != NDPI_CFG_OK) {
ERROR;
}
/* Finalization */
ret = ndpi_finalize_initialization(ndpi_struct);
if(ret != 0) {
ERROR;
}
/* Initialization done, now you can feed packets to the library */
/* Cleanup */
ndpi_exit_detection_module(ndpi_struct);
```
A more complex example, with global context and a shared Oookla LRU cache (all the others caches are local)
```
struct ndpi_global_context *g_ctx;
struct ndpi_detection_module_struct *ndpi_structs[num_local_contexts];
ndpi_cfg_error rc;
int i, ret;
g_ctx = ndpi_global_init();
if(!g_ctx) {
ERROR;
}
for(i = 0; i < num_local_contexts; i++) {
ndpi_structs[i] = ndpi_init_detection_module(g_ctx);
if(!ndpi_struct[i]) {
ERROR;
}
rc = ndpi_set_config(ndpi_structs[i], NULL, "lru.ookla.scope", "1");
if(rc != NDPI_CFG_OK) {
ERROR;
}
ret = ndpi_finalize_initialization(ndpi_structs[i]);
if(ret != 0) {
ERROR;
}
}
/* Initialization done */
/* Cleanup */
for(i = 0; i < num_local_contexts; i++) {
ndpi_exit_detection_module(ndpi_structs[i]);
}
ndpi_global_deinit(g_ctx);
```
+858
View File
@@ -0,0 +1,858 @@
nDPI Protocols List
###################
This page provides the list of the protocols/applications supported by nDPI. For each protocol there is a brief description, some links to further, more detailed information and, optionally, some notes that might be useful when handling such a protocol (from the application/integrator point of view)
Work in progress!
.. _Proto 32:
`NDPI_PROTOCOL_BFCP`
=======================
BFCP (Binary Floor Control Protocol) is used for controlling and coordinating real-time data sharing and collaboration during video conferencing sessions.
References: `RFC: <https://datatracker.ietf.org/doc/html/rfc8855>`_
.. _Proto 54:
`NDPI_PROTOCOL_IQIYI`
===========================
iQIYI is a Chinese online video platform that offers a wide range of original and licensed content including movies, dramas, variety shows, and anime.
References: `Main site: <https://www.iqiyi.com/>`_
.. _Proto 59:
`NDPI_PROTOCOL_ADOBE_CONNECT`
===========================
Adobe Connect is a web conferencing platform that allows users to conduct online meetings, webinars, and virtual classrooms.
References: `Main site: <https://www.adobe.com/products/adobeconnect.html>`_
.. _Proto 65:
`NDPI_PROTOCOL_IRC`
==================
IRC (Internet Relay Chat) is a text-based chat system for instant messaging.
References: `Wikipiedia: <https://en.wikipedia.org/wiki/IRC>`_ and `Some statistics: <https://netsplit.de/networks/top10.php>`
.. _Proto 72:
`NDPI_PROTOCOL_HART_IP`
=======================
Highway Addressable Remote Transducer over IP
References: `Protocol Specs: <https://library.fieldcommgroup.org/20085/TS20085>`_
.. _Proto 96:
`NDPI_PROTOCOL_TFTP`
====================
TFTP is a very simple protocol used to transfer files. It is from this that its name comes, Trivial File Transfer Protocol or TFTP.
References: `RFC1350 <https://datatracker.ietf.org/doc/html/rfc1350>`_ and `RFC2347 <https://datatracker.ietf.org/doc/html/rfc2347>`_ and `RFC2349 <https://datatracker.ietf.org/doc/html/rfc2349>`_
.. _Proto 127:
`NDPI_PROTOCOL_MS_RPCH`
=======================
A Remote Procedure Call protocol over HTTP from Microsoft.
References: `Protocol Specs: <https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-rpch/c0f4c9c5-1a61-4d10-b8e2-005378d1d212>`_
.. _Proto 149:
`NDPI_PROTOCOL_EGD`
=========================
Ethernet Global Data (EGD) is a communication protocol developed by GE Fanuc Automation for real-time data exchange between automation devices and control systems using standard Ethernet technology. It is widely adopted in industrial environments for its simplicity and reliability in data communication.
References: `Wikipedia <https://en.wikipedia.org/wiki/Ethernet_Global_Data_Protocol>`_
Notes:
- This dissector only works for data packets, not configuration commands.
- IPv6 is not supported
.. _Proto 182:
`NDPI_PROTOCOL_RESP`
=======================
Redis Serialization Protocol
References: `Protocol Specs: <https://redis.io/docs/reference/protocol-spec/>`_
.. _Proto 186:
`NDPI_PROTOCOL_COD_MOBILE`
=======================
Call of Duty: Mobile is a free-to-play shooter game for iOS and Android devices. It has amassed over 650 million downloads worldwide, making it one of the most popular mobile games.
References: `Main site: <https://www.callofduty.com/mobile/>`_
.. _Proto 201:
`NDPI_PROTOCOL_GOOGLE_MEET`
===========================
Google Meet is a video conferencing service from Google.
References: `Main site: <https://meet.google.com/>`_
.. _Proto 235:
`NDPI_PROTOCOL_VALVE_SDR`
===========================
Steam Datagram Relay (SDR) is Valve's virtual private gaming network. Used in all modern games from Valve, but can also be used by developers via the Steamworks SDK for their games.
References: `Main site: <https://partner.steamgames.com/doc/features/multiplayer/steamdatagramrelay>`_
.. _Proto 338:
`NDPI_PROTOCOL_SRTP`
====================
The Secure Real-time Transport Protocol (SRTP) is a profile for Real-time Transport Protocol (RTP) intended to provide encryption, message authentication, integrity, and replay attack protection to the RTP data.
References: `RFC3711 <https://datatracker.ietf.org/doc/html/rfc3711>`_
Notes:
- You can think of SRTP simply as the "encrypted" version of RTP, something like HTTPS vs HTTP;
- It is not usually possible to tell RTP from SRTP. nDPI generally uses the former and it uses the latter only when it is really sure that the media stream has been encrypted.
.. _Proto 339:
`NDPI_PROTOCOL_OPERA_VPN`
=========================
Opera VPN is a free VPN included with Opera Browser.
References: `Main site <https://www.opera.com/it/features/free-vpn>`_
.. _Proto 340:
`NDPI_PROTOCOL_EPICGAMES`
=========================
Epic Games is a video game company developing the Unreal Engine and some successful games as Fortnite and Gears of War.
References: `Main site <https://store.epicgames.com/en-US/>`_ and `Fortnite <https://www.fortnite.com/>`_
.. _Proto 341:
`NDPI_PROTOCOL_GEFORCENOW`
==========================
GeForce Now is the brand used by Nvidia for its cloud gaming service.
References: `Main site <https://www.nvidia.com/en-us/geforce-now/>`_
.. _Proto 342:
`NDPI_PROTOCOL_NVIDIA`
======================
Generic web traffic from Nvidia sites.
References: `Main site <https://www.nvidia.com>`_
.. _Proto 343:
`NDPI_PROTOCOL_BITCOIN`
=======================
Bitcoin is one of the most common crypto currencies.
References: `Main site <https://en.bitcoin.it/wiki/Protocol_documentation>`_
Notes:
- Not each crypto exchange is a mining, it could be a normal transaction, sending or receving.
- Bitcoin network protocol covers the broader set of rules that govern how all nodes in the network communicate and sync with each others blocks and transactions.
- On the other hand mining protocols are more specific and deal directly with how miners interact with the network and participate in the mining process.
.. _Proto 344:
`NDPI_PROTOCOL_PROTONVPN`
=========================
Proton VPN is a VPN service operated by the Swiss company Proton AG, the company behind the email service Proton Mail
References: `Main site <https://protonvpn.com/>`_
.. _Proto 345:
`NDPI_PROTOCOL_THRIFT`
======================
Apache Thrift is a generic data interchange framework that supports a bunch of different languages and platforms.
References: `Official site <https://thrift.apache.org>`_ and `Github <https://github.com/apache/thrift>`_
.. _Proto 346:
`NDPI_PROTOCOL_ROBLOX`
======================
Roblox is an online game platform and game creation system.
References: `Main site <https://www.roblox.com/>`_
Notes:
- Since Roblox games use a custom version of the RakNet protocol, some Roblox flows might be classified as RakNet.
.. _Proto 347:
`NDPI_PROTOCOL_SERVICE_LOCATION`
================================
The Service Location Protocol is a service discovery protocol that allows computers and other devices to find services in a local area network without prior configuration.
References: `SLPv1 <https://datatracker.ietf.org/doc/html/rfc2165>`_ and `SLPv2 <https://datatracker.ietf.org/doc/html/rfc2608>`_
.. _Proto 348:
`NDPI_PROTOCOL_MULLVAD`
=======================
Mullvad is a VPN service operated by Mullvad VPN AB, based in Sweden
References: `Main site <https://mullvad.net/>`_
.. _Proto 349:
`NDPI_PROTOCOL_HTTP2`
=====================
HTTP/2 (originally named HTTP/2.0) is a major revision of the HTTP network protocol used by the World Wide Web.
References: `RFC <https://datatracker.ietf.org/doc/html/rfc9113>`_
Notes:
- HTTP/2 is almost always encrypted, i.e. transported over TLS.
.. _Proto 350:
`NDPI_PROTOCOL_HAPROXY`
=======================
HAProxy is a free and open source software that provides a high availability load balancer and reverse proxy for TCP and HTTP-based applications that spreads requests across multiple servers.
References: `Main site: <https://www.haproxy.org>`_
.. _Proto 351:
`NDPI_PROTOCOL_RMCP`
====================
The Intelligent Platform Management Interface (IPMI) is a set of computer interface specifications for an autonomous computer subsystem that provides management and monitoring capabilities independently of the host system's CPU, firmware (BIOS or UEFI) and operating system.
References: `Protocol Specs: <https://www.dmtf.org/sites/default/files/standards/documents/DSP0114.pdf>`_
.. _Proto 352:
`NDPI_PROTOCOL_CAN`
===================
Controller Area Network (CAN) is used extensively in automotive applications, with in excess of 400 million CAN enabled microcontrollers manufactured each year.
References: `Protocol Specs: <https://www.iso.org/standard/63648.html>`_
.. _Proto 353:
`NDPI_PROTOCOL_PROTOBUF`
========================
Protocol Buffers (Protobuf) is a free and open-source cross-platform data format used to serialize structured data.
References: `Encoding: <https://protobuf.dev/programming-guides/encoding>`_
.. _Proto 354:
`NDPI_PROTOCOL_ETHEREUM`
=======================
Ethereum is a decentralized, open-source blockchain with smart contract functionality.
References: `Main site <https://ethereum.org/en/developers/docs/intro-to-ethereum/>`_
Notes:
- same as Bitcoin, not each crypto exchange is a mining, it could be a normal transaction, sending or receving or even blockchain exploration.
.. _Proto 355:
`NDPI_PROTOCOL_TELEGRAM_VOIP`
============================
Audio/video calls made using the telegram app.
References: `Wikipedia <https://en.wikipedia.org/wiki/telegram_(software)/>`_
.. _Proto 356:
`NDPI_PROTOCOL_SINA_WEIBO`
============================
Chinese microblogging (weibo) website.
References: `Wikipedia <https://en.wikipedia.org/wiki/Sina_Weibo>`_
.. _Proto 358:
`NDPI_PROTOCOL_PTPV2`
============================
IEEE 1588-2008 Precision Time Protocol (PTP) Version 2.
References: `Protocol Specs: <https://standards.ieee.org/ieee/1588/4355/>`_
.. _Proto 359:
`NDPI_PROTOCOL_RTPS`
============================
Real-Time Publish Subscribe Protocol
References: `Protocol Specs: <https://www.omg.org/spec/DDSI-RTPS/>`_
.. _Proto 360:
`NDPI_PROTOCOL_OPC_UA`
============================
IEC62541 OPC Unified Architecture
References: `Protocol Specs: <https://reference.opcfoundation.org/>`_
.. _Proto 361:
`NDPI_PROTOCOL_S7COMM_PLUS`
============================
A proprietary protocol from Siemens used for data exchange between PLCs and access PLC data via SCADA systems.
Completely different from classic S7Comm, but also uses TPKT/COTP as a transport.
References: `Unofficial description: <https://plc4x.apache.org/protocols/s7/s7comm-plus.html>`_
.. _Proto 362:
`NDPI_PROTOCOL_FINS`
============================
Factory Interface Network Service (FINS) is a network protocol used by Omron PLCs.
References: `Protocol Specs: <https://assets.omron.eu/downloads/manual/en/v4/w421_cj1w-etn21_cs1w-etn21_ethernet_units_-_construction_of_applications_operation_manual_en.pdf>`_
.. _Proto 363:
`NDPI_PROTOCOL_ETHERSIO`
============================
Ether-S-I/O is a proprietary protocol used by Saia-Burgess's PLCs.
References: `Wireshark wiki: <https://wiki.wireshark.org/EtherSIO.md>`_
.. _Proto 364:
`NDPI_PROTOCOL_UMAS`
============================
UMAS is a proprietary Schneider Electric protocol based on Modbus. It's used in Modicon M580 and Modicon M340 CPU-based PLCs.
References: `Unofficial article: <https://ics-cert.kaspersky.com/publications/reports/2022/09/29/the-secrets-of-schneider-electrics-umas-protocol/>`_
.. _Proto 365:
`NDPI_PROTOCOL_BECKHOFF_ADS`
============================
Automation Device Specification is the protocol used for interfacing with Beckhoff PLCs via TwinCAT.
References: `Protocol Specs: <https://infosys.beckhoff.com/english.php?content=../content/1033/tc3_ads_intro/115847307.html>`_
.. _Proto 366:
`NDPI_PROTOCOL_ISO9506_1_MMS`
============================
The international standard MMS (Manufacturing Message Specification) is an OSI application layer messaging protocol origionally designed for the remote control and monitoring of devices such as Remote Terminal Units (RTU), Programmable Logic Controllers (PLC), Numerical Controllers (NC), or Robot Controllers (RC).
References: `Paid Specs: <https://www.iso.org/ru/standard/37079.html>`_
.. _Proto 367:
`NDPI_PROTOCOL_IEEE_C37118`
============================
IEEE Standard for Synchrophasor Data Transfer for Power Systems
References: `Paid Specs: <https://standards.ieee.org/ieee/C37.118.1/4902/>`_
.. _Proto 368:
`NDPI_PROTOCOL_ETHERSBUS`
============================
Ether-S-Bus is a proprietary protocol used for the communication with and between PLCs manufactured by Saia-Burgess Controls Ltd.
References: `Wireshark wiki: <https://wiki.wireshark.org/EtherSBus>`_
.. _Proto 369:
`NDPI_PROTOCOL_MONERO`
======================
Monero is a private and decentralized cryptocurrency with focus on confidentiality and security.
.. _Proto 370:
`NDPI_PROTOCOL_DCERPC`
======================
DCE/RPC is a specification for a remote procedure call mechanism that defines both APIs and an over-the-network protocol.
References: `Wireshark wiki: <https://wiki.wireshark.org/DCE/RPC>`_
.. _Proto 371:
`NDPI_PROTOCOL_PROFINET_IO`
===========================
PROFINET/IO is a field bus protocol based on connectionless DCE/RPC.
References: `Protocol Specs: <https://www.profibus.com/download/profinet-specification>`_
.. _Proto 372:
`NDPI_PROTOCOL_HISLIP`
======================
High-Speed LAN Instrument Protocol (HiSLIP) is a protocol for remote instrument control of LAN-based test and measurement instruments.
References: `Protocol Specs: <https://www.ivifoundation.org/downloads/Protocol%20Specifications/IVI-6.1_HiSLIP-2.0-2020-04-23.pdf>`_
.. _Proto 373:
`NDPI_PROTOCOL_UFTP`
====================
Encrypted UDP based FTP with multicast.
References: `Protocol Specs: <https://uftp-multicast.sourceforge.net/protocol.txt>`_.
.. _Proto 374:
`NDPI_PROTOCOL_OPENFLOW`
========================
OpenFlow protocol is a network protocol closely associated with Software-Defined Networking (SDN).
References: `Protocol Specs: <https://opennetworking.org/wp-content/uploads/2014/10/openflow-switch-v1.5.1.pdf>`_
.. _Proto 375:
`NDPI_PROTOCOL_JSON_RPC`
========================
JSON-RPC is a remote procedure call protocol encoded in JSON.
References: `Protocol Specs: <https://www.jsonrpc.org/specification>`_
.. _Proto 376:
`NDPI_PROTOCOL_WEBDAV`
======================
WebDAV is a set of extensions to the HTTP protocol that allows WebDAV clients to collaboratively edit and manage files on remote Web servers.
References: `RFC4918: <https://datatracker.ietf.org/doc/html/rfc4918>`_
Notes:
- WebDAV is almost always encrypted, i.e. transported over TLS.
.. _Proto 377:
`NDPI_PROTOCOL_APACHE_KAFKA`
============================
Apache Kafka is a distributed event store and stream-processing platform.
References: `Official site <https://kafka.apache.org>`_ and `Github <https://github.com/apache/kafka>`_
.. _Proto 378:
`NDPI_PROTOCOL_NOMACHINE`
=========================
NoMachine is a popular proprietary remote desktop software.
References: `Main site <https://www.nomachine.com/>`_
.. _Proto 379:
`NDPI_PROTOCOL_IEC62056`
============================
IEC 62056-4-7 DLMS/COSEM is a transport layer for IP networks.
References: `Paid Specs: <https://webstore.iec.ch/publication/22487>`_
Notes:
- Wireshark is not able to recognize this protocol. Some old plugins/code (with some documentation) are available `here <https://github.com/bearxiong99/wireshark-dlms>` and `here <https://github.com/matousp/dlms-analysis/tree/master>`.
.. _Proto 380:
`NDPI_PROTOCOL_HL7`
=========================
HL7 is a range of global standards for the transfer of clinical and administrative health data between applications.
References: `Main site <https://www.hl7.org/>`_
.. _Proto 381:
`NDPI_PROTOCOL_CEPH`
=========================
Ceph is a scalable distributed storage system.
References: `Main site <https://ceph.io/en/>`_
.. _Proto 382:
`NDPI_PROTOCOL_GOOGLE_CHAT`
=========================
Google Chat is an instant messaging service from Google, which replaced Hangouts.
References: `Main site <https://chat.google.com/>`_
.. _Proto 383:
`NDPI_PROTOCOL_ROUGHTIME`
=========================
A protocol that aims to achieve rough time synchronization while detecting servers that provide inaccurate time and providing cryptographic proof of their malfeasance.
References: `IETF Draft <https://www.ietf.org/archive/id/draft-ietf-ntp-roughtime-08.html>`_
.. _Proto 384:
`NDPI_PROTOCOL_PIA`
=========================
Private Internet Access (PIA) is a popular VPN service from Kape Technologies.
References: `Main site <https://www.privateinternetaccess.com/>`_
.. _Proto 385:
`NDPI_PROTOCOL_KCP`
===================
KCP - A Fast and Reliable ARQ Protocol. It provides TCP-like stream support with low latency at the cost of bandwidth usage - used by lot's of Open Source / Third Party applications.
References: `Protocol Specs: <https://github.com/skywind3000/kcp/blob/master/protocol.txt>`_
.. _Proto 386:
`NDPI_PROTOCOL_DOTA2`
=========================
Dota 2 is an extremely popular multiplayer MOBA game from Valve.
References: `Main site <https://www.dota2.com/>`_
.. _Proto 387:
`NDPI_PROTOCOL_MUMBLE`
=========================
Mumble is a free, open source, low latency, high quality voice chat application.
References: `Main site <https://www.mumble.info/>`_
.. _Proto 388:
`NDPI_PROTOCOL_YOJIMBO`
=======================
Yojimbo (netcode) is a secure connection-based client/server protocol built on top of UDP.
References: `Protocol Specs: <https://github.com/mas-bandwidth/netcode/blob/main/STANDARD.md>`_
.. _Proto 389:
`NDPI_PROTOCOL_ELECTRONICARTS`
=========================
Electronic Arts is a leading publisher of games on Console, PC and Mobile.
References: `Main site <https://www.ea.com/>`_
Notes:
- Almost all of that traffic is related to their EA Origin game store.
.. _Proto 390:
`NDPI_PROTOCOL_STOMP`
========================
STOMP is a simple interoperable protocol designed for asynchronous message passing between clients via mediating servers. Supported in ActiveMQ and RabbitMQ.
References: `Protocol Specs: <https://stomp.github.io/stomp-specification-1.2.html>`_
.. _Proto 391:
`NDPI_PROTOCOL_RADMIN`
=========================
Radmin is remote access software for the Microsoft Windows platform.
References: `Main site <https://www.radmin.com/>`_
.. _Proto 392:
`NDPI_PROTOCOL_RAFT`
====================
Raft is a consensus algorithm and protocol for managing a replicated log.
References: `C implementation <https://github.com/canonical/raft>`_ and `Paper <https://raft.github.io/raft.pdf>`_
.. _Proto 394:
`NDPI_PROTOCOL_GEARMAN`
====================
Gearman is a network-based job-queuing system that was initially developed by Danga Interactive in order to process large volumes of jobs.
References: `Main site <http://gearman.org/>`_
.. _Proto 395:
`NDPI_PROTOCOL_TENCENTGAMES`
====================
A protocol used by various games from Tencent (mostly mobile games).
References: `Main site <https://www.tencentgames.com/>`_
.. _Proto 396:
`NDPI_PROTOCOL_GAIJIN`
====================
Protocols used in various games from Gaijin Entertainment.
References: `Main site <https://gaijin.net/>`_
.. _Proto 397:
`NDPI_PROTOCOL_C1222`
====================
ANSI C12.22 (IEEE Std 1703) describe a protocol for transporting ANSI C12.19 table data over networks. It's mostly used to communicate with electric meters.
References: `Paid specs <https://www.nema.org/Standards/view/American-National-Standard-for-Protocol-Specification-for-Interfacing-to-Data-Communication-Networks/>`_
.. _Proto 398:
`NDPI_PROTOCOL_HUAWEI`
======================
Generic Huawei traffic.
References: `Main site <https://www.huawei.com/>`_
.. _Proto 399:
`NDPI_PROTOCOL_HUAWEI_CLOUD`
============================
Huawei Mobile Cloud.
References: `Main site <https://cloud.huawei.com/>`_
.. _Proto 400:
`NDPI_PROTOCOL_DLEP`
=====================
The Dynamic Link ExchangeProtocol (DLEP) is a radio aware routing (RAR) protocol.
References: `RFC <https://datatracker.ietf.org/doc/html/rfc8175>`_
.. _Proto 401:
`NDPI_PROTOCOL_BFD`
=====================
Bidirectional Forwarding Detection is a network protocol that is used to detect faults between two routers or switches.
References: `RFC <https://datatracker.ietf.org/doc/html/rfc5880>`_
.. _Proto 402:
`NDPI_PROTOCOL_NETEASE_GAMES`
============================
Traffic of various NetEase games.
References: `Main site <https://www.neteasegames.com/>`_
.. _Proto 403:
`NDPI_PROTOCOL_PATHOFEXILE`
============================
Path of Exile is a free-to-play online Action RPG.
References: `Main site <https://pathofexile.com/>`_
.. _Proto 404:
`NDPI_PROTOCOL_GOOGLE_CALL`
===========================
Audio/video calls made by (any) Google applications (i.e Google Meet).
References: `Main site: <https://meet.google.com/>`_
Notes:
- nDPI usually uses different protocol ids for the generic application traffic and for its "realtime" traffic (examples: NDPI_PROTOCOL_MEET/NDPI_PROTOCOL_GOOGLE_CALL, NDPI_PROTOCOL_WHATSAPP/NDPI_PROTOCOL_WHATSAPP_CALL, ...)
.. _Proto 405:
`NDPI_PROTOCOL_PFCP`
=====================
PFCP is a protocol used for communicating between control plane (CP) and user plane (UP) functions in 4G and 5G networks.
References: `Protocol Specs: <https://www.etsi.org/deliver/etsi_ts/129200_129299/129244/16.05.00_60/ts_129244v160500p.pdf>`_
.. _Proto 406:
`NDPI_PROTOCOL_FLUTE`
=====================
File Delivery over Unidirectional Transport.
References: `RFC <https://datatracker.ietf.org/doc/html/rfc6726>`_
.. _Proto 407:
`NDPI_PROTOCOL_LOLWILDRIFT`
============================
League of Legends: Wild Rift is a mobile MOBA game.
References: `Main site <https://wildrift.leagueoflegends.com/>`_
.. _Proto 408:
`NDPI_PROTOCOL_TESO`
============================
The Elder Scrolls Online is a MMORPG set in the fantasy world of Tamriel.
References: `Main site <https://www.elderscrollsonline.com/>`_
.. _Proto 409:
`NDPI_PROTOCOL_LDP`
=====================
The Label Distribution Protocol (LDP) is a routing protocol used to establish and maintain label-switched paths in a Multiprotocol Label Switching (MPLS) network.
References: `RFC <https://datatracker.ietf.org/doc/html/rfc5036>`_
.. _Proto 410:
`NDPI_PROTOCOL_KNXNET_IP`
=====================
KNXnet/IP is a building automation protocol that enables the exchange of data and control information over IP networks, extending the KNX standard for home and building automation.
References: `Paid Specs: <https://webstore.ansi.org/standards/ds/dsiso225102019>`_
.. _Proto 411:
`NDPI_PROTOCOL_BLUESKY`
======================
Bluesky, also known as Bluesky Social, is a decentralized microblogging social platform.
References: `Main site: <https://bsky.app/>`_
.. _Proto 412:
`NDPI_PROTOCOL_MASTODON`
=======================
Mastodon is free and open-source software for running self-hosted social networking services. It has microblogging features similar to Twitter.
References: `Main site: <https://joinmastodon.org/>`_
.. _Proto 413:
`NDPI_PROTOCOL_THREADS`
======================
Threads is an online social media and social networking service operated by Meta Platforms.
References: `Main site: <https://www.threads.net>`_
.. _Proto 414:
`NDPI_PROTOCOL_VIBER_VOIP`
=========================
Audio/video calls made using the Viber app.
References: `Wikipedia <https://en.wikipedia.org/wiki/Viber>`_
.. _Proto 415:
`NDPI_PROTOCOL_ZUG`
=========================
The ZUG protocol is part of the Casper 2.0 consensus model.
References: `Main Site <https://casperlabs.io>`_ and `Blog Post <https://casperlabs.io/blog/beyond-eth-30-theres-casper-20>`_
.. _Proto 416:
`NDPI_PROTOCOL_JRMI`
=========================
The JRMI protocol is the Java Remote Method Invocation protocol.
References: `Oracle site <https://docs.oracle.com/en/java/javase/21/docs/specs/rmi/protocol.html>`_
.. _Proto 417:
`NDPI_PROTOCOL_RIPE_ATLAS`
==========================
The RIPE Atlas probe protocol is used for the world's largest active Internet measurement network.
References: `Main Site <https://atlas.ripe.net/>`_ and `Documentation <https://ripe-atlas-tools.readthedocs.io/en/latest/index.html>`_
+4
View File
@@ -0,0 +1,4 @@
breathe
exhale
sphinx-rtd-theme
sphinxcontrib-swaggerdoc
-18
View File
@@ -1,18 +0,0 @@
# How to use?
You need to first compile the nDPI library as usual:
- ./autogen.sh
- ./configure
- make
Then open the Xcode project and you are ready to go. The default behavior is to analyze an embeded pcap file `capture.pcap`. You can change the behavior by changing command line input in `ViewController.m` file.
# What does the XCode project do?
It's a dummy Mac App project with a **Run** button. It doesn't modify any nDPI code except that it renamed the `main` function to `orginal_main` in `ndpiReader.c` (because the Mac App has it's own main function) and call the `orginal_main` with synthetic command line input from `ViewController.m` file when the **Run** button is clicked.
It also fixes some problems when compiling with Xcode. Some are listed below:
- Add missed `NDPI_LOG_DEBUG2` macro definition implementation (defined as `NDPI_LOG_DEBUG2_XCODE_PROJ` in `ViewController.m`)
- Add an empty ndpi_utils.h file to make `protocols/attic/ftp.c` and `protocols/attic/secondlife.c` can compile
- Specially treat `ndpi_patricia.c` by not adding it into compilation source, since it's directly included in `ndpi_main.c`
File diff suppressed because it is too large Load Diff
@@ -1,7 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<Workspace
version = "1.0">
<FileRef
location = "self:ndpiExample.xcodeproj">
</FileRef>
</Workspace>
@@ -1,29 +0,0 @@
/*
*
* Copyright (C) 2011-18 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#import <Cocoa/Cocoa.h>
@interface AppDelegate : NSObject <NSApplicationDelegate>
@end
@@ -1,41 +0,0 @@
/*
*
* Copyright (C) 2011-18 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#import "AppDelegate.h"
@interface AppDelegate ()
@end
@implementation AppDelegate
- (void)applicationDidFinishLaunching:(NSNotification *)aNotification {
// Insert code here to initialize your application
}
- (void)applicationWillTerminate:(NSNotification *)aNotification {
// Insert code here to tear down your application
}
@end
@@ -1,58 +0,0 @@
{
"images" : [
{
"idiom" : "mac",
"size" : "16x16",
"scale" : "1x"
},
{
"idiom" : "mac",
"size" : "16x16",
"scale" : "2x"
},
{
"idiom" : "mac",
"size" : "32x32",
"scale" : "1x"
},
{
"idiom" : "mac",
"size" : "32x32",
"scale" : "2x"
},
{
"idiom" : "mac",
"size" : "128x128",
"scale" : "1x"
},
{
"idiom" : "mac",
"size" : "128x128",
"scale" : "2x"
},
{
"idiom" : "mac",
"size" : "256x256",
"scale" : "1x"
},
{
"idiom" : "mac",
"size" : "256x256",
"scale" : "2x"
},
{
"idiom" : "mac",
"size" : "512x512",
"scale" : "1x"
},
{
"idiom" : "mac",
"size" : "512x512",
"scale" : "2x"
}
],
"info" : {
"version" : 1,
"author" : "xcode"
}
}
@@ -1,732 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<document type="com.apple.InterfaceBuilder3.Cocoa.Storyboard.XIB" version="3.0" toolsVersion="13771" targetRuntime="MacOSX.Cocoa" propertyAccessControl="none" useAutolayout="YES" initialViewController="B8D-0N-5wS">
<dependencies>
<deployment identifier="macosx"/>
<plugIn identifier="com.apple.InterfaceBuilder.CocoaPlugin" version="13771"/>
<capability name="documents saved in the Xcode 8 format" minToolsVersion="8.0"/>
</dependencies>
<scenes>
<!--Application-->
<scene sceneID="JPo-4y-FX3">
<objects>
<application id="hnw-xV-0zn" sceneMemberID="viewController">
<menu key="mainMenu" title="Main Menu" systemMenu="main" id="AYu-sK-qS6">
<items>
<menuItem title="ndpiExample" id="1Xt-HY-uBw">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="ndpiExample" systemMenu="apple" id="uQy-DD-JDr">
<items>
<menuItem title="About ndpiExample" id="5kV-Vb-QxS">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="orderFrontStandardAboutPanel:" target="Ady-hI-5gd" id="Exp-CZ-Vem"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="VOq-y0-SEH"/>
<menuItem title="Preferences…" keyEquivalent="," id="BOF-NM-1cW"/>
<menuItem isSeparatorItem="YES" id="wFC-TO-SCJ"/>
<menuItem title="Services" id="NMo-om-nkz">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Services" systemMenu="services" id="hz9-B4-Xy5"/>
</menuItem>
<menuItem isSeparatorItem="YES" id="4je-JR-u6R"/>
<menuItem title="Hide ndpiExample" keyEquivalent="h" id="Olw-nP-bQN">
<connections>
<action selector="hide:" target="Ady-hI-5gd" id="PnN-Uc-m68"/>
</connections>
</menuItem>
<menuItem title="Hide Others" keyEquivalent="h" id="Vdr-fp-XzO">
<modifierMask key="keyEquivalentModifierMask" option="YES" command="YES"/>
<connections>
<action selector="hideOtherApplications:" target="Ady-hI-5gd" id="VT4-aY-XCT"/>
</connections>
</menuItem>
<menuItem title="Show All" id="Kd2-mp-pUS">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="unhideAllApplications:" target="Ady-hI-5gd" id="Dhg-Le-xox"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="kCx-OE-vgT"/>
<menuItem title="Quit ndpiExample" keyEquivalent="q" id="4sb-4s-VLi">
<connections>
<action selector="terminate:" target="Ady-hI-5gd" id="Te7-pn-YzF"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="File" id="dMs-cI-mzQ">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="File" id="bib-Uj-vzu">
<items>
<menuItem title="New" keyEquivalent="n" id="Was-JA-tGl">
<connections>
<action selector="newDocument:" target="Ady-hI-5gd" id="4Si-XN-c54"/>
</connections>
</menuItem>
<menuItem title="Open…" keyEquivalent="o" id="IAo-SY-fd9">
<connections>
<action selector="openDocument:" target="Ady-hI-5gd" id="bVn-NM-KNZ"/>
</connections>
</menuItem>
<menuItem title="Open Recent" id="tXI-mr-wws">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Open Recent" systemMenu="recentDocuments" id="oas-Oc-fiZ">
<items>
<menuItem title="Clear Menu" id="vNY-rz-j42">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="clearRecentDocuments:" target="Ady-hI-5gd" id="Daa-9d-B3U"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem isSeparatorItem="YES" id="m54-Is-iLE"/>
<menuItem title="Close" keyEquivalent="w" id="DVo-aG-piG">
<connections>
<action selector="performClose:" target="Ady-hI-5gd" id="HmO-Ls-i7Q"/>
</connections>
</menuItem>
<menuItem title="Save…" keyEquivalent="s" id="pxx-59-PXV">
<connections>
<action selector="saveDocument:" target="Ady-hI-5gd" id="teZ-XB-qJY"/>
</connections>
</menuItem>
<menuItem title="Save As…" keyEquivalent="S" id="Bw7-FT-i3A">
<connections>
<action selector="saveDocumentAs:" target="Ady-hI-5gd" id="mDf-zr-I0C"/>
</connections>
</menuItem>
<menuItem title="Revert to Saved" keyEquivalent="r" id="KaW-ft-85H">
<connections>
<action selector="revertDocumentToSaved:" target="Ady-hI-5gd" id="iJ3-Pv-kwq"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="aJh-i4-bef"/>
<menuItem title="Page Setup…" keyEquivalent="P" id="qIS-W8-SiK">
<modifierMask key="keyEquivalentModifierMask" shift="YES" command="YES"/>
<connections>
<action selector="runPageLayout:" target="Ady-hI-5gd" id="Din-rz-gC5"/>
</connections>
</menuItem>
<menuItem title="Print…" keyEquivalent="p" id="aTl-1u-JFS">
<connections>
<action selector="print:" target="Ady-hI-5gd" id="qaZ-4w-aoO"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Edit" id="5QF-Oa-p0T">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Edit" id="W48-6f-4Dl">
<items>
<menuItem title="Undo" keyEquivalent="z" id="dRJ-4n-Yzg">
<connections>
<action selector="undo:" target="Ady-hI-5gd" id="M6e-cu-g7V"/>
</connections>
</menuItem>
<menuItem title="Redo" keyEquivalent="Z" id="6dh-zS-Vam">
<connections>
<action selector="redo:" target="Ady-hI-5gd" id="oIA-Rs-6OD"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="WRV-NI-Exz"/>
<menuItem title="Cut" keyEquivalent="x" id="uRl-iY-unG">
<connections>
<action selector="cut:" target="Ady-hI-5gd" id="YJe-68-I9s"/>
</connections>
</menuItem>
<menuItem title="Copy" keyEquivalent="c" id="x3v-GG-iWU">
<connections>
<action selector="copy:" target="Ady-hI-5gd" id="G1f-GL-Joy"/>
</connections>
</menuItem>
<menuItem title="Paste" keyEquivalent="v" id="gVA-U4-sdL">
<connections>
<action selector="paste:" target="Ady-hI-5gd" id="UvS-8e-Qdg"/>
</connections>
</menuItem>
<menuItem title="Paste and Match Style" keyEquivalent="V" id="WeT-3V-zwk">
<modifierMask key="keyEquivalentModifierMask" option="YES" command="YES"/>
<connections>
<action selector="pasteAsPlainText:" target="Ady-hI-5gd" id="cEh-KX-wJQ"/>
</connections>
</menuItem>
<menuItem title="Delete" id="pa3-QI-u2k">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="delete:" target="Ady-hI-5gd" id="0Mk-Ml-PaM"/>
</connections>
</menuItem>
<menuItem title="Select All" keyEquivalent="a" id="Ruw-6m-B2m">
<connections>
<action selector="selectAll:" target="Ady-hI-5gd" id="VNm-Mi-diN"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="uyl-h8-XO2"/>
<menuItem title="Find" id="4EN-yA-p0u">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Find" id="1b7-l0-nxx">
<items>
<menuItem title="Find…" tag="1" keyEquivalent="f" id="Xz5-n4-O0W">
<connections>
<action selector="performFindPanelAction:" target="Ady-hI-5gd" id="cD7-Qs-BN4"/>
</connections>
</menuItem>
<menuItem title="Find and Replace…" tag="12" keyEquivalent="f" id="YEy-JH-Tfz">
<modifierMask key="keyEquivalentModifierMask" option="YES" command="YES"/>
<connections>
<action selector="performFindPanelAction:" target="Ady-hI-5gd" id="WD3-Gg-5AJ"/>
</connections>
</menuItem>
<menuItem title="Find Next" tag="2" keyEquivalent="g" id="q09-fT-Sye">
<connections>
<action selector="performFindPanelAction:" target="Ady-hI-5gd" id="NDo-RZ-v9R"/>
</connections>
</menuItem>
<menuItem title="Find Previous" tag="3" keyEquivalent="G" id="OwM-mh-QMV">
<connections>
<action selector="performFindPanelAction:" target="Ady-hI-5gd" id="HOh-sY-3ay"/>
</connections>
</menuItem>
<menuItem title="Use Selection for Find" tag="7" keyEquivalent="e" id="buJ-ug-pKt">
<connections>
<action selector="performFindPanelAction:" target="Ady-hI-5gd" id="U76-nv-p5D"/>
</connections>
</menuItem>
<menuItem title="Jump to Selection" keyEquivalent="j" id="S0p-oC-mLd">
<connections>
<action selector="centerSelectionInVisibleArea:" target="Ady-hI-5gd" id="IOG-6D-g5B"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Spelling and Grammar" id="Dv1-io-Yv7">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Spelling" id="3IN-sU-3Bg">
<items>
<menuItem title="Show Spelling and Grammar" keyEquivalent=":" id="HFo-cy-zxI">
<connections>
<action selector="showGuessPanel:" target="Ady-hI-5gd" id="vFj-Ks-hy3"/>
</connections>
</menuItem>
<menuItem title="Check Document Now" keyEquivalent=";" id="hz2-CU-CR7">
<connections>
<action selector="checkSpelling:" target="Ady-hI-5gd" id="fz7-VC-reM"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="bNw-od-mp5"/>
<menuItem title="Check Spelling While Typing" id="rbD-Rh-wIN">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleContinuousSpellChecking:" target="Ady-hI-5gd" id="7w6-Qz-0kB"/>
</connections>
</menuItem>
<menuItem title="Check Grammar With Spelling" id="mK6-2p-4JG">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleGrammarChecking:" target="Ady-hI-5gd" id="muD-Qn-j4w"/>
</connections>
</menuItem>
<menuItem title="Correct Spelling Automatically" id="78Y-hA-62v">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleAutomaticSpellingCorrection:" target="Ady-hI-5gd" id="2lM-Qi-WAP"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Substitutions" id="9ic-FL-obx">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Substitutions" id="FeM-D8-WVr">
<items>
<menuItem title="Show Substitutions" id="z6F-FW-3nz">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="orderFrontSubstitutionsPanel:" target="Ady-hI-5gd" id="oku-mr-iSq"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="gPx-C9-uUO"/>
<menuItem title="Smart Copy/Paste" id="9yt-4B-nSM">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleSmartInsertDelete:" target="Ady-hI-5gd" id="3IJ-Se-DZD"/>
</connections>
</menuItem>
<menuItem title="Smart Quotes" id="hQb-2v-fYv">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleAutomaticQuoteSubstitution:" target="Ady-hI-5gd" id="ptq-xd-QOA"/>
</connections>
</menuItem>
<menuItem title="Smart Dashes" id="rgM-f4-ycn">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleAutomaticDashSubstitution:" target="Ady-hI-5gd" id="oCt-pO-9gS"/>
</connections>
</menuItem>
<menuItem title="Smart Links" id="cwL-P1-jid">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleAutomaticLinkDetection:" target="Ady-hI-5gd" id="Gip-E3-Fov"/>
</connections>
</menuItem>
<menuItem title="Data Detectors" id="tRr-pd-1PS">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleAutomaticDataDetection:" target="Ady-hI-5gd" id="R1I-Nq-Kbl"/>
</connections>
</menuItem>
<menuItem title="Text Replacement" id="HFQ-gK-NFA">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleAutomaticTextReplacement:" target="Ady-hI-5gd" id="DvP-Fe-Py6"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Transformations" id="2oI-Rn-ZJC">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Transformations" id="c8a-y6-VQd">
<items>
<menuItem title="Make Upper Case" id="vmV-6d-7jI">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="uppercaseWord:" target="Ady-hI-5gd" id="sPh-Tk-edu"/>
</connections>
</menuItem>
<menuItem title="Make Lower Case" id="d9M-CD-aMd">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="lowercaseWord:" target="Ady-hI-5gd" id="iUZ-b5-hil"/>
</connections>
</menuItem>
<menuItem title="Capitalize" id="UEZ-Bs-lqG">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="capitalizeWord:" target="Ady-hI-5gd" id="26H-TL-nsh"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Speech" id="xrE-MZ-jX0">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Speech" id="3rS-ZA-NoH">
<items>
<menuItem title="Start Speaking" id="Ynk-f8-cLZ">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="startSpeaking:" target="Ady-hI-5gd" id="654-Ng-kyl"/>
</connections>
</menuItem>
<menuItem title="Stop Speaking" id="Oyz-dy-DGm">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="stopSpeaking:" target="Ady-hI-5gd" id="dX8-6p-jy9"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Format" id="jxT-CU-nIS">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Format" id="GEO-Iw-cKr">
<items>
<menuItem title="Font" id="Gi5-1S-RQB">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Font" systemMenu="font" id="aXa-aM-Jaq">
<items>
<menuItem title="Show Fonts" keyEquivalent="t" id="Q5e-8K-NDq">
<connections>
<action selector="orderFrontFontPanel:" target="YLy-65-1bz" id="WHr-nq-2xA"/>
</connections>
</menuItem>
<menuItem title="Bold" tag="2" keyEquivalent="b" id="GB9-OM-e27">
<connections>
<action selector="addFontTrait:" target="YLy-65-1bz" id="hqk-hr-sYV"/>
</connections>
</menuItem>
<menuItem title="Italic" tag="1" keyEquivalent="i" id="Vjx-xi-njq">
<connections>
<action selector="addFontTrait:" target="YLy-65-1bz" id="IHV-OB-c03"/>
</connections>
</menuItem>
<menuItem title="Underline" keyEquivalent="u" id="WRG-CD-K1S">
<connections>
<action selector="underline:" target="Ady-hI-5gd" id="FYS-2b-JAY"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="5gT-KC-WSO"/>
<menuItem title="Bigger" tag="3" keyEquivalent="+" id="Ptp-SP-VEL">
<connections>
<action selector="modifyFont:" target="YLy-65-1bz" id="Uc7-di-UnL"/>
</connections>
</menuItem>
<menuItem title="Smaller" tag="4" keyEquivalent="-" id="i1d-Er-qST">
<connections>
<action selector="modifyFont:" target="YLy-65-1bz" id="HcX-Lf-eNd"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="kx3-Dk-x3B"/>
<menuItem title="Kern" id="jBQ-r6-VK2">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Kern" id="tlD-Oa-oAM">
<items>
<menuItem title="Use Default" id="GUa-eO-cwY">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="useStandardKerning:" target="Ady-hI-5gd" id="6dk-9l-Ckg"/>
</connections>
</menuItem>
<menuItem title="Use None" id="cDB-IK-hbR">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="turnOffKerning:" target="Ady-hI-5gd" id="U8a-gz-Maa"/>
</connections>
</menuItem>
<menuItem title="Tighten" id="46P-cB-AYj">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="tightenKerning:" target="Ady-hI-5gd" id="hr7-Nz-8ro"/>
</connections>
</menuItem>
<menuItem title="Loosen" id="ogc-rX-tC1">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="loosenKerning:" target="Ady-hI-5gd" id="8i4-f9-FKE"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Ligatures" id="o6e-r0-MWq">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Ligatures" id="w0m-vy-SC9">
<items>
<menuItem title="Use Default" id="agt-UL-0e3">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="useStandardLigatures:" target="Ady-hI-5gd" id="7uR-wd-Dx6"/>
</connections>
</menuItem>
<menuItem title="Use None" id="J7y-lM-qPV">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="turnOffLigatures:" target="Ady-hI-5gd" id="iX2-gA-Ilz"/>
</connections>
</menuItem>
<menuItem title="Use All" id="xQD-1f-W4t">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="useAllLigatures:" target="Ady-hI-5gd" id="KcB-kA-TuK"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Baseline" id="OaQ-X3-Vso">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Baseline" id="ijk-EB-dga">
<items>
<menuItem title="Use Default" id="3Om-Ey-2VK">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="unscript:" target="Ady-hI-5gd" id="0vZ-95-Ywn"/>
</connections>
</menuItem>
<menuItem title="Superscript" id="Rqc-34-cIF">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="superscript:" target="Ady-hI-5gd" id="3qV-fo-wpU"/>
</connections>
</menuItem>
<menuItem title="Subscript" id="I0S-gh-46l">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="subscript:" target="Ady-hI-5gd" id="Q6W-4W-IGz"/>
</connections>
</menuItem>
<menuItem title="Raise" id="2h7-ER-AoG">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="raiseBaseline:" target="Ady-hI-5gd" id="4sk-31-7Q9"/>
</connections>
</menuItem>
<menuItem title="Lower" id="1tx-W0-xDw">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="lowerBaseline:" target="Ady-hI-5gd" id="OF1-bc-KW4"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem isSeparatorItem="YES" id="Ndw-q3-faq"/>
<menuItem title="Show Colors" keyEquivalent="C" id="bgn-CT-cEk">
<connections>
<action selector="orderFrontColorPanel:" target="Ady-hI-5gd" id="mSX-Xz-DV3"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="iMs-zA-UFJ"/>
<menuItem title="Copy Style" keyEquivalent="c" id="5Vv-lz-BsD">
<modifierMask key="keyEquivalentModifierMask" option="YES" command="YES"/>
<connections>
<action selector="copyFont:" target="Ady-hI-5gd" id="GJO-xA-L4q"/>
</connections>
</menuItem>
<menuItem title="Paste Style" keyEquivalent="v" id="vKC-jM-MkH">
<modifierMask key="keyEquivalentModifierMask" option="YES" command="YES"/>
<connections>
<action selector="pasteFont:" target="Ady-hI-5gd" id="JfD-CL-leO"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Text" id="Fal-I4-PZk">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Text" id="d9c-me-L2H">
<items>
<menuItem title="Align Left" keyEquivalent="{" id="ZM1-6Q-yy1">
<connections>
<action selector="alignLeft:" target="Ady-hI-5gd" id="zUv-R1-uAa"/>
</connections>
</menuItem>
<menuItem title="Center" keyEquivalent="|" id="VIY-Ag-zcb">
<connections>
<action selector="alignCenter:" target="Ady-hI-5gd" id="spX-mk-kcS"/>
</connections>
</menuItem>
<menuItem title="Justify" id="J5U-5w-g23">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="alignJustified:" target="Ady-hI-5gd" id="ljL-7U-jND"/>
</connections>
</menuItem>
<menuItem title="Align Right" keyEquivalent="}" id="wb2-vD-lq4">
<connections>
<action selector="alignRight:" target="Ady-hI-5gd" id="r48-bG-YeY"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="4s2-GY-VfK"/>
<menuItem title="Writing Direction" id="H1b-Si-o9J">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Writing Direction" id="8mr-sm-Yjd">
<items>
<menuItem title="Paragraph" enabled="NO" id="ZvO-Gk-QUH">
<modifierMask key="keyEquivalentModifierMask"/>
</menuItem>
<menuItem id="YGs-j5-SAR">
<string key="title"> Default</string>
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="makeBaseWritingDirectionNatural:" target="Ady-hI-5gd" id="qtV-5e-UBP"/>
</connections>
</menuItem>
<menuItem id="Lbh-J2-qVU">
<string key="title"> Left to Right</string>
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="makeBaseWritingDirectionLeftToRight:" target="Ady-hI-5gd" id="S0X-9S-QSf"/>
</connections>
</menuItem>
<menuItem id="jFq-tB-4Kx">
<string key="title"> Right to Left</string>
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="makeBaseWritingDirectionRightToLeft:" target="Ady-hI-5gd" id="5fk-qB-AqJ"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="swp-gr-a21"/>
<menuItem title="Selection" enabled="NO" id="cqv-fj-IhA">
<modifierMask key="keyEquivalentModifierMask"/>
</menuItem>
<menuItem id="Nop-cj-93Q">
<string key="title"> Default</string>
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="makeTextWritingDirectionNatural:" target="Ady-hI-5gd" id="lPI-Se-ZHp"/>
</connections>
</menuItem>
<menuItem id="BgM-ve-c93">
<string key="title"> Left to Right</string>
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="makeTextWritingDirectionLeftToRight:" target="Ady-hI-5gd" id="caW-Bv-w94"/>
</connections>
</menuItem>
<menuItem id="RB4-Sm-HuC">
<string key="title"> Right to Left</string>
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="makeTextWritingDirectionRightToLeft:" target="Ady-hI-5gd" id="EXD-6r-ZUu"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem isSeparatorItem="YES" id="fKy-g9-1gm"/>
<menuItem title="Show Ruler" id="vLm-3I-IUL">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleRuler:" target="Ady-hI-5gd" id="FOx-HJ-KwY"/>
</connections>
</menuItem>
<menuItem title="Copy Ruler" keyEquivalent="c" id="MkV-Pr-PK5">
<modifierMask key="keyEquivalentModifierMask" control="YES" command="YES"/>
<connections>
<action selector="copyRuler:" target="Ady-hI-5gd" id="71i-fW-3W2"/>
</connections>
</menuItem>
<menuItem title="Paste Ruler" keyEquivalent="v" id="LVM-kO-fVI">
<modifierMask key="keyEquivalentModifierMask" control="YES" command="YES"/>
<connections>
<action selector="pasteRuler:" target="Ady-hI-5gd" id="cSh-wd-qM2"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="View" id="H8h-7b-M4v">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="View" id="HyV-fh-RgO">
<items>
<menuItem title="Show Toolbar" keyEquivalent="t" id="snW-S8-Cw5">
<modifierMask key="keyEquivalentModifierMask" option="YES" command="YES"/>
<connections>
<action selector="toggleToolbarShown:" target="Ady-hI-5gd" id="BXY-wc-z0C"/>
</connections>
</menuItem>
<menuItem title="Customize Toolbar…" id="1UK-8n-QPP">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="runToolbarCustomizationPalette:" target="Ady-hI-5gd" id="pQI-g3-MTW"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="hB3-LF-h0Y"/>
<menuItem title="Show Sidebar" keyEquivalent="s" id="kIP-vf-haE">
<modifierMask key="keyEquivalentModifierMask" control="YES" command="YES"/>
<connections>
<action selector="toggleSourceList:" target="Ady-hI-5gd" id="iwa-gc-5KM"/>
</connections>
</menuItem>
<menuItem title="Enter Full Screen" keyEquivalent="f" id="4J7-dP-txa">
<modifierMask key="keyEquivalentModifierMask" control="YES" command="YES"/>
<connections>
<action selector="toggleFullScreen:" target="Ady-hI-5gd" id="dU3-MA-1Rq"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Window" id="aUF-d1-5bR">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Window" systemMenu="window" id="Td7-aD-5lo">
<items>
<menuItem title="Minimize" keyEquivalent="m" id="OY7-WF-poV">
<connections>
<action selector="performMiniaturize:" target="Ady-hI-5gd" id="VwT-WD-YPe"/>
</connections>
</menuItem>
<menuItem title="Zoom" id="R4o-n2-Eq4">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="performZoom:" target="Ady-hI-5gd" id="DIl-cC-cCs"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="eu3-7i-yIM"/>
<menuItem title="Bring All to Front" id="LE2-aR-0XJ">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="arrangeInFront:" target="Ady-hI-5gd" id="DRN-fu-gQh"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Help" id="wpr-3q-Mcd">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Help" systemMenu="help" id="F2S-fz-NVQ">
<items>
<menuItem title="ndpiExample Help" keyEquivalent="?" id="FKE-Sm-Kum">
<connections>
<action selector="showHelp:" target="Ady-hI-5gd" id="y7X-2Q-9no"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
</items>
</menu>
<connections>
<outlet property="delegate" destination="Voe-Tx-rLC" id="PrD-fu-P6m"/>
</connections>
</application>
<customObject id="Voe-Tx-rLC" customClass="AppDelegate"/>
<customObject id="YLy-65-1bz" customClass="NSFontManager"/>
<customObject id="Ady-hI-5gd" userLabel="First Responder" customClass="NSResponder" sceneMemberID="firstResponder"/>
</objects>
<point key="canvasLocation" x="75" y="0.0"/>
</scene>
<!--Window Controller-->
<scene sceneID="R2V-B0-nI4">
<objects>
<windowController id="B8D-0N-5wS" sceneMemberID="viewController">
<window key="window" title="Window" allowsToolTipsWhenApplicationIsInactive="NO" autorecalculatesKeyViewLoop="NO" oneShot="NO" releasedWhenClosed="NO" showsToolbarButton="NO" visibleAtLaunch="NO" animationBehavior="default" id="IQv-IB-iLA">
<windowStyleMask key="styleMask" titled="YES" closable="YES" miniaturizable="YES" resizable="YES"/>
<windowPositionMask key="initialPositionMask" leftStrut="YES" rightStrut="YES" topStrut="YES" bottomStrut="YES"/>
<rect key="contentRect" x="196" y="240" width="480" height="270"/>
<rect key="screenRect" x="0.0" y="0.0" width="1680" height="1027"/>
<connections>
<outlet property="delegate" destination="B8D-0N-5wS" id="98r-iN-zZc"/>
</connections>
</window>
<connections>
<segue destination="XfG-lQ-9wD" kind="relationship" relationship="window.shadowedContentViewController" id="cq2-FE-JQM"/>
</connections>
</windowController>
<customObject id="Oky-zY-oP4" userLabel="First Responder" customClass="NSResponder" sceneMemberID="firstResponder"/>
</objects>
<point key="canvasLocation" x="75" y="250"/>
</scene>
<!--View Controller-->
<scene sceneID="hIz-AP-VOD">
<objects>
<viewController id="XfG-lQ-9wD" customClass="ViewController" sceneMemberID="viewController">
<view key="view" wantsLayer="YES" id="m2S-Jp-Qdl">
<rect key="frame" x="0.0" y="0.0" width="480" height="270"/>
<autoresizingMask key="autoresizingMask"/>
<subviews>
<button verticalHuggingPriority="750" fixedFrame="YES" translatesAutoresizingMaskIntoConstraints="NO" id="Jdf-YK-4JB">
<rect key="frame" x="23" y="173" width="64" height="32"/>
<autoresizingMask key="autoresizingMask" flexibleMaxX="YES" flexibleMinY="YES"/>
<buttonCell key="cell" type="push" title="Run" bezelStyle="rounded" alignment="center" borderStyle="border" imageScaling="proportionallyDown" inset="2" id="1eG-Fb-Ybk">
<behavior key="behavior" pushIn="YES" lightByBackground="YES" lightByGray="YES"/>
<font key="font" metaFont="system"/>
</buttonCell>
<connections>
<action selector="onRunButtonClicked:" target="XfG-lQ-9wD" id="zyK-t7-vwt"/>
</connections>
</button>
</subviews>
</view>
</viewController>
<customObject id="rPt-NT-nkU" userLabel="First Responder" customClass="NSResponder" sceneMemberID="firstResponder"/>
</objects>
<point key="canvasLocation" x="75" y="655"/>
</scene>
</scenes>
</document>
@@ -1,32 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CFBundleDevelopmentRegion</key>
<string>$(DEVELOPMENT_LANGUAGE)</string>
<key>CFBundleExecutable</key>
<string>$(EXECUTABLE_NAME)</string>
<key>CFBundleIconFile</key>
<string></string>
<key>CFBundleIdentifier</key>
<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
<key>CFBundleInfoDictionaryVersion</key>
<string>6.0</string>
<key>CFBundleName</key>
<string>$(PRODUCT_NAME)</string>
<key>CFBundlePackageType</key>
<string>APPL</string>
<key>CFBundleShortVersionString</key>
<string>1.0</string>
<key>CFBundleVersion</key>
<string>1</string>
<key>LSMinimumSystemVersion</key>
<string>$(MACOSX_DEPLOYMENT_TARGET)</string>
<key>NSHumanReadableCopyright</key>
<string>Copyright © 2018年 ZengYingpei. All rights reserved.</string>
<key>NSMainStoryboardFile</key>
<string>Main</string>
<key>NSPrincipalClass</key>
<string>NSApplication</string>
</dict>
</plist>
@@ -1,29 +0,0 @@
/*
*
* Copyright (C) 2011-18 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#import <Cocoa/Cocoa.h>
@interface ViewController : NSViewController
@end
@@ -1,106 +0,0 @@
/*
*
* Copyright (C) 2011-18 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#import "ViewController.h"
#include "ndpi_api.h"
// Declare the orginal_main defined in ndpiReader.c here
extern int orginal_main(int argc, char **argv);
@implementation ViewController
- (void)viewDidLoad {
[super viewDidLoad];
// Do any additional setup after loading the view.
}
- (void)setRepresentedObject:(id)representedObject {
[super setRepresentedObject:representedObject];
// Update the view, if already loaded.
}
- (IBAction)onRunButtonClicked:(id)sender
{
char* args[10];
extern int optind;
optind = 1; // reset the parse of getopt_long
// Check the "nDPI_QuickStartGuide.pdf" for comand option explanation.
/* Following code it to execute below command (remember to change args[2] to
* absolute path):
* ./ndpiReader -i capture.pcap
*/
args[0] = (char*)"ndpiReader";
args[1] = (char*)"-i";
NSString* pcap_file = [[NSBundle mainBundle]pathForResource:@"capture" ofType:@"pcap"];
args[2] = (char*)[pcap_file cStringUsingEncoding:NSUTF8StringEncoding];
// Change to you pcap file path if you want.
//args[2] = (char*)"/Users/zengyingpei/Documents/code/nDPI/example/MacOS/ndpiExample/ndpiExample/capture.pcap";
// Remember to change below number of args when you change to other command inputs.
orginal_main(3, args);
/* Following code it to execute below command:
* ./ndpiReader -i en1 -s 10 -p protos.txt
* The process seems to be not support re-entering. You may have to re-run the App.
*/
/*
args[0] = (char*)"ndpiReader";
args[1] = (char*)"-i";
args[2] = (char*)"en0";
args[3] = (char*)"-s";
args[4] = (char*)"10";
args[5] = (char*)"-p";
NSString* proto_file = [[NSBundle mainBundle]pathForResource:@"protos" ofType:@"txt"];
args[6] = (char*)[proto_file cStringUsingEncoding:NSUTF8StringEncoding];
//args[6] = (char*)"/Users/zengyingpei/Documents/code/nDPI/example/protos.txt";
orginal_main(7, args);
*/
}
// In order to fix the missing of NDPI_LOG_DEBUG2 (used in ndpi_main.c), we define
// NDPI_LOG_DEBUG2 as NDPI_LOG_DEBUG2_XCODE_PROJ.
void vNDPI_LOG_DEBUG2_XCODE_PROJ(struct ndpi_detection_module_struct * ndpi_struct,
const char *format, va_list ap)
{
vprintf(format, ap);
}
void NDPI_LOG_DEBUG2_XCODE_PROJ(struct ndpi_detection_module_struct * ndpi_struct,
const char *format, ...)
{
va_list ap;
va_start (ap, format);
vNDPI_LOG_DEBUG2_XCODE_PROJ(ndpi_struct, format, ap);
va_end (ap);
}
@end
Binary file not shown.
@@ -1,27 +0,0 @@
/*
*
* Copyright (C) 2011-18 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#import <Cocoa/Cocoa.h>
int main(int argc, const char * argv[]) {
return NSApplicationMain(argc, argv);
}
@@ -1,5 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict/>
</plist>
@@ -1,29 +0,0 @@
/*
*
* Copyright (C) 2011-18 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#ifndef ndpi_utils_h
#define ndpi_utils_h
// Empty placeholder head file. Just to make the compilations of protocols/attic/ftp.c and
// protocols/attic/secondlife.c don't break.
#endif /* ndpi_utils_h */
+1 -1
View File
@@ -17,7 +17,7 @@ include $(RTE_SDK)/mk/rte.vars.mk
APP = ndpiReader.dpdk
LIBNDPI = $(PWD)/../src/lib/libndpi.a
SRCS-y := reader_util.c intrusion_detection.c ndpiReader.c
SRCS-y := reader_util.c ndpiReader.c
CFLAGS += -g
CFLAGS += -Wno-strict-prototypes -Wno-missing-prototypes -Wno-missing-declarations -Wno-unused-parameter -I $(PWD)/../src/include @CFLAGS@ -DUSE_DPDK
+45 -20
View File
@@ -1,22 +1,37 @@
AR=@AR@
CC=@CC@
CXX=@CXX@
ifneq ($(OS),Windows_NT)
OS := $(shell uname)
endif
ifeq ($(OS),Darwin)
CC=clang -fno-color-diagnostics
endif
BUILD_MINGW=@BUILD_MINGW@
BUILD_MINGW_X64=@BUILD_MINGW_X64@
DISABLE_NPCAP=@DISABLE_NPCAP@
EXE_SUFFIX=@EXE_SUFFIX@
SRCHOME=../src
CFLAGS=-g -fPIC -DPIC -I$(SRCHOME)/include @PCAP_INC@ @CFLAGS@
LDFLAGS=@LDFLAGS@
ifneq ($(OS),Windows_NT)
CFLAGS+=-fPIC -DPIC
endif
CFLAGS+=-I$(SRCHOME)/include @NDPI_CFLAGS@ @PCAP_INC@ @GPROF_CFLAGS@
LDFLAGS+=@NDPI_LDFLAGS@
LIBNDPI=$(SRCHOME)/lib/libndpi.a
LIBS=$(LIBNDPI) @PCAP_LIB@ @LIBS@ @ADDITIONAL_LIBS@
HEADERS=intrusion_detection.h reader_util.h $(SRCHOME)/include/ndpi_api.h \
LIBS=$(LIBNDPI) @PCAP_LIB@ @ADDITIONAL_LIBS@ @LIBS@ @GPROF_LIBS@
HEADERS=reader_util.h $(SRCHOME)/include/ndpi_api.h \
$(SRCHOME)/include/ndpi_typedefs.h $(SRCHOME)/include/ndpi_protocol_ids.h
OBJS=ndpiReader.o reader_util.o intrusion_detection.o
PREFIX?=@prefix@
ifneq ($(BUILD_MINGW),)
SUFFIX:=.exe
ifeq ($(DISABLE_NPCAP),0)
CFLAGS+=-I@srcdir@/../windows/WpdPack/Include -I@srcdir@/../windows/WpdPack/Include/pcap
LIBS+=-Wl,-Bstatic -lpthread -Wl,-Bdynamic
else
CFLAGS+=-DDISABLE_NPCAP
endif
ifeq ($(DISABLE_NPCAP),0)
ifneq ($(BUILD_MINGW_X64),)
LIBS+=@srcdir@/../windows/WpdPack/Lib/x64/wpcap.lib
@@ -24,13 +39,16 @@ else
LIBS+=@srcdir@/../windows/WpdPack/Lib/wpcap.lib
endif
else
LIBS+=-lpthread
endif
all: ndpiReader$(SUFFIX) @DPDK_TARGET@
LIBS+=-Wl,-Bstatic -lpthread -Wl,-Bdynamic
else
LIBS+=-pthread
endif
CFLAGS+=-pthread
all: ndpiReader$(EXE_SUFFIX) @DPDK_TARGET@
EXECUTABLE_SOURCES := ndpiReader.c ndpiSimpleIntegration.c
COMMON_SOURCES := $(filter-out $(EXECUTABLE_SOURCES),$(wildcard *.c ))
@@ -38,22 +56,22 @@ COMMON_SOURCES := $(filter-out $(EXECUTABLE_SOURCES),$(wildcard *.c ))
libndpiReader.a: $(COMMON_SOURCES:%.c=%.o) $(LIBNDPI)
$(AR) rsv libndpiReader.a $(COMMON_SOURCES:%.c=%.o)
ndpiReader$(SUFFIX): libndpiReader.a $(LIBNDPI) ndpiReader.o
ndpiReader$(EXE_SUFFIX): libndpiReader.a $(LIBNDPI) ndpiReader.o
$(CC) $(CFLAGS) $(LDFLAGS) ndpiReader.o libndpiReader.a $(LIBS) -o $@
ndpiSimpleIntegration$(SUFFIX): ndpiSimpleIntegration.o
ndpiSimpleIntegration$(EXE_SUFFIX): ndpiSimpleIntegration.o
$(CC) $(CFLAGS) $(LDFLAGS) $< $(LIBS) -o $@
%.o: %.c $(HEADERS) Makefile
$(CC) $(CFLAGS) $(CPPFLAGS) -c $< -o $@
install: ndpiReader$(SUFFIX)
install: ndpiReader$(EXE_SUFFIX)
mkdir -p $(DESTDIR)$(PREFIX)/bin/
mkdir -p $(DESTDIR)$(PREFIX)/share/ndpi
cp ndpiReader$(SUFFIX) $(DESTDIR)$(PREFIX)/bin/
cp ndpiReader$(EXE_SUFFIX) $(DESTDIR)$(PREFIX)/bin/
cp protos.txt $(DESTDIR)$(PREFIX)/share/ndpi/ndpiProtos.txt
cp mining_hosts.txt $(DESTDIR)$(PREFIX)/share/ndpi/ndpiCustomCategory.txt
[ -f ndpiSimpleIntegration$(SUFFIX) ] && cp ndpiSimpleIntegration$(SUFFIX) $(DESTDIR)$(PREFIX)/bin/ndpiSimpleIntegration$(SUFFIX) || true
[ -f ndpiSimpleIntegration$(EXE_SUFFIX) ] && cp ndpiSimpleIntegration$(EXE_SUFFIX) $(DESTDIR)$(PREFIX)/bin/ndpiSimpleIntegration$(EXE_SUFFIX) || true
[ -f build/app/ndpiReader.dpdk ] && cp build/app/ndpiReader.dpdk $(DESTDIR)$(PREFIX)/bin/ || true
[ -f ndpiReader.dpdk ] && cp ndpiReader.dpdk $(DESTDIR)$(PREFIX)/bin/ || true
@@ -61,17 +79,24 @@ dpdk:
$(MAKE) -f Makefile.dpdk
check:
true # nothing to do here
cppcheck:
cppcheck --template='{file}:{line}:{severity}:{message}' --quiet --enable=all --force -I$(SRCHOME)/include *.c
clean:
/bin/rm -f *.o ndpiReader ndpiSimpleIntegration ndpiReader$(SUFFIX) ndpiSimpleIntegration$(SUFFIX) ndpiReader.dpdk
/bin/rm -f *.o ndpiReader ndpiSimpleIntegration ndpiReader$(EXE_SUFFIX) ndpiSimpleIntegration$(EXE_SUFFIX) ndpiReader.dpdk libndpiReader.a
/bin/rm -f .*.dpdk.cmd .*.o.cmd *.dpdk.map .*.o.d
/bin/rm -f _install _postbuild _postinstall _preinstall
/bin/rm -rf build
distdir:
cp categories.txt mining_hosts.txt protos.txt README.DPDK '$(distdir)/'
find . -maxdepth 1 -type f -name '*.c' -o -name '*.h' -o -name '*.py' | xargs -I'{}' cp '{}' '$(distdir)/{}'
cp README.DPDK '$(distdir)/'
find . -maxdepth 1 -type f -name '*.c' \
-o -name '*.h' \
-o -name '*.py' \
-o -name '*.txt' \
-o -name '*.csv' | xargs -I'{}' cp '{}' '$(distdir)/{}'
distclean: clean
/bin/rm -f Makefile.dpdk
+3
View File
@@ -1,3 +1,6 @@
# Format: name\tcategory_id
internetbadguys.com 100
144.139.247.220 100
172.30.69.0/24 100
[2001:1670:8:40a6:a08e:332b:aa69:18dc] 100
[2001:db8:1::1]/127 100
+13
View File
@@ -0,0 +1,13 @@
#Format: <proto, param, value>
#proto might be empty
#Basic example of how to set nDPI configuration parameters via file
#We set only some default values
#See doc/configuration_parameters.md for a complete list and description of all the accepted knobs
packets_limit_per_flow,32
ookla,dpi.aggressiveness,0x1
tls,metadata.sha1_fingerprint,1
lru.bittorrent.ttl,0
+108 -109
View File
@@ -1,109 +1,108 @@
################################################################
# abuse.ch Suricata JA3 Fingerprint Blacklist (CSV) #
# For Suricata 4.1.0 or newer #
# Last updated: 2020-04-09 06:48:14 UTC #
# #
# Terms Of Use: https://sslbl.abuse.ch/blacklist/ #
# For questions please contact sslbl [at] abuse.ch #
################################################################
#
# ja3_md5,Firstseen,Lastseen,Listingreason
b386946a5a44d1ddcc843bc75336dfce,2017-07-14 18:08:15,2019-07-27 20:42:54,Dridex
8991a387e4cc841740f25d6f5139f92d,2017-07-14 19:02:03,2019-07-28 00:34:38,Adware
cb98a24ee4b9134448ffb5714fd870ac,2017-07-14 19:48:28,2019-05-22 03:22:38,Dridex
1aa7bf8b97e540ca5edd75f7b8384bfa,2017-07-14 20:23:38,2019-07-28 01:38:22,TrickBot
3d89c0dfb1fa44911b8fa7523ef8dedb,2017-07-15 04:23:45,2021-02-01 18:23:25,Adware
bc6c386f480ee97b9d9e52d472b772d8,2017-07-15 10:57:38,2021-02-21 13:13:30,Adware
8f52d1ce303fb4a6515836aec3cc16b1,2017-07-15 19:05:11,2019-07-27 20:00:57,TrickBot
d6f04b5a910115f4b50ecec09d40a1df,2017-07-15 19:42:24,2018-10-14 08:12:51,Dridex
35c0a31c481927f022a3b530255ac080,2017-07-15 19:43:19,2021-02-06 03:21:55,Tofsee
d551fafc4f40f1dec2bb45980bfa9492,2017-07-15 19:59:29,2020-11-16 13:06:20,Adware
e330bca99c8a5256ae126a55c4c725c5,2017-07-15 19:59:29,2021-01-13 00:29:37,Adware
b8f81673c0e1d29908346f3bab892b9b,2017-07-16 01:32:03,2018-12-17 06:08:03,Adware
83e04bc58d402f9633983cbf22724b02,2017-07-16 01:32:03,2021-01-14 14:23:42,Adware
70722097d1fe1d78d8c2164640ab6df4,2017-07-16 02:39:08,2021-02-22 04:56:57,Tofsee
9c2589e1c0e9f533a022c6205f9719e1,2017-07-16 08:37:17,2021-02-19 08:46:32,Adware
849b04bdbd1d2b983f6e8a457e0632a8,2017-07-16 08:37:17,2021-02-19 08:46:32,Adware
16efcf0e00504ddfedde13bfea997952,2017-07-16 19:45:45,2020-12-23 15:10:32,Adware
4d7a28d6f2263ed61de88ca66eb011e3,2017-07-16 21:20:29,2020-12-08 18:10:55,Tofsee
550dce18de1bb143e69d6dd9413b8355,2017-07-16 22:17:20,2018-12-21 07:04:50,Adware
c50f6a8b9173676b47ba6085bd0c6cee,2017-07-16 22:38:41,2019-05-21 09:42:17,TrickBot
20dd18bdd3209ea718989030a6f93364,2017-07-18 10:22:58,2019-04-28 09:23:31,Adware
8498fe4268764dbf926a38283e9d3d8f,2017-07-18 10:22:58,2020-12-31 02:06:27,Adware
590a232d04d56409fab72e752a8a2634,2017-07-18 18:53:24,2020-10-11 20:48:33,Tofsee
51a7ad14509fd614c7bb3a50c4982b8c,2017-07-19 07:28:19,2019-07-14 11:58:32,JBifrost
96eba628dcb2b47607192ba74a3b55ba,2017-07-19 18:53:48,2021-01-04 12:45:08,Tofsee
df5c30e670dba99f9270ed36060cf054,2017-07-20 17:44:07,2018-04-11 15:57:59,Tofsee
098f55e27d8c4b0a590102cbdb3a5f3a,2017-07-21 09:52:01,2019-04-08 01:09:54,Adware
46efd49abcca8ea9baa932da68fdb529,2017-07-22 14:07:36,2021-02-20 20:45:32,Adware
29085f03f8e8a03f0b399c5c7cf0b0b8,2017-07-22 14:07:36,2021-02-21 20:56:01,Adware
d7150af4514b868defb854db0f62a441,2017-07-23 09:39:24,2018-07-24 01:04:58,Tofsee
03e186a7f83285e93341de478334006e,2017-07-24 18:17:14,2021-02-13 23:55:58,Tofsee
3cda52da4ade09f1f781ad2e82dcfa20,2017-07-30 18:41:36,2019-05-21 17:34:18,Quakbot
b13d01846ad7a14a70bf030a16775c78,2017-08-08 07:12:49,2021-02-21 01:32:12,Adware
1543a7c46633acf71e8401baccbd0568,2017-08-08 21:32:28,2020-11-10 05:30:17,Tofsee
1d095e68489d3c535297cd8dffb06cb9,2017-08-12 19:56:28,2020-10-28 11:06:23,Tofsee
93d056782d649deb51cda44ecb714bb0,2017-08-28 12:20:47,2019-04-15 23:47:27,Adware
698e36219f3979420fa2581b21dac7ec,2017-08-28 12:20:47,2020-12-31 02:06:31,Adware
1712287800ac91b34cadd5884ce85568,2017-08-28 16:01:59,2021-01-23 09:36:32,TorrentLocker
5e573c9c9f8ba720ef9b18e9fce2e2f7,2017-08-30 13:44:56,2021-02-17 17:11:34,Adware
f6fd83a21f9f3c5f9ff7b5c63bbc179d,2017-10-20 08:03:21,2018-11-06 06:42:12,Adware
92579701f145605e9edc0b01a901c6d5,2017-10-23 00:10:48,2021-02-18 22:56:49,Adware
a61299f9b501adcf680b9275d79d4ac6,2017-11-04 18:03:59,2020-04-21 17:08:24,Tofsee
b2b61db7b9490a60d270ccb20b462826,2017-11-14 20:12:03,2021-02-18 22:56:49,Adware
7dcce5b76c8b17472d024758970a406b,2017-11-22 12:42:46,2020-12-22 15:15:49,Tofsee
534ce2dbc413c68e908363b5df0ae5e0,2017-12-22 09:36:21,2019-07-27 15:22:33,TrickBot
fb00055a1196aeea8d1bc609885ba953,2018-01-01 22:49:25,2019-04-09 06:58:58,TrickBot
a50a861119aceb0ccc74902e8fddb618,2018-01-02 08:16:23,2018-07-05 02:33:08,Tofsee
e7643725fcff971e3051fe0e47fc2c71,2018-01-31 08:06:13,2020-03-25 16:19:48,Tofsee
7c410ce832e848a3321432c9a82e972b,2018-01-31 20:04:25,2021-02-21 22:51:31,Tofsee
da949afd9bd6df820730f8f171584a71,2018-02-03 05:19:37,2021-02-21 05:20:43,Tofsee
906004246f3ba5e755b043c057254a29,2018-03-11 08:25:38,2018-04-14 00:59:16,Tofsee
fd80fa9c6120cdeea8520510f3c644ac,2018-03-11 09:34:30,2021-02-21 22:16:39,Tofsee
b90bdbe961a648f0427db21aaa6ccb59,2018-03-11 10:37:43,2020-05-29 23:39:01,Tofsee
1fe4c7a3544eb27afec2adfb3a3dbf60,2018-03-11 19:23:08,2021-02-20 11:19:52,Tofsee
c201b92f8b483fa388be174d6689f534,2018-03-12 13:43:52,2021-01-28 06:17:06,Gozi
9f62c4f26b90d3d757bea609e82f2eaf,2018-03-13 06:23:41,2020-12-18 16:07:31,Tofsee
1be3ecebe5aa9d3654e6e703d81f6928,2018-03-13 11:50:02,2021-02-14 23:03:13,Ransomware.Troldesh
e3b2ab1f9a56f2fb4c9248f2f41631fa,2018-03-15 01:06:34,2021-02-22 10:36:48,Tofsee
dff8a0aa1c904aaea76c5bf624e88333,2018-03-18 09:41:15,2020-10-27 09:50:24,Tofsee
17fd49722f8d11f3d76dce84f8e099a7,2018-03-19 23:02:27,2020-12-09 11:32:21,Tofsee
911479ac8a0813ed1241b3686ccdade9,2018-03-19 23:24:59,2020-03-30 04:09:18,Tofsee
c5deb9465d47232dd48772f9c4d14679,2018-03-22 15:42:48,2021-02-17 07:39:32,Tofsee
f22bdd57e3a52de86cda40da2d84e83b,2018-03-27 13:40:19,2019-01-20 14:31:39,Tofsee
d18a4da84af59e1108862a39bae7c9d4,2018-04-03 00:40:51,2021-02-06 01:53:12,Tofsee
2d8794cb7b52b777bee2695e79c15760,2018-04-04 06:56:37,2021-02-10 08:22:31,Ransomware
40adfd923eb82b89d8836ba37a19bca1,2018-04-15 15:49:08,2021-02-21 22:22:56,CoinMiner
1aee0238942d453d679fc1e37a303387,2018-05-13 01:59:49,2021-02-10 13:19:24,Tofsee
2092e1fffb45d7e4a19a57f9bc5e203a,2018-05-16 21:59:36,2018-09-05 01:58:33,Adware
bffa4501966196d3d6e90cee1f88fc89,2018-06-07 15:08:04,2020-03-16 00:03:44,Tofsee
807fca46d9d0cf63adf4e5e80e414bbe,2018-06-07 16:51:03,2021-02-20 03:15:38,Tofsee
fb58831f892190644fe44e25bc830b45,2018-06-08 12:07:59,2019-05-31 21:02:37,Adware
0cc1e84568e471aa1d62ad4158ade6b5,2018-06-24 10:50:47,2021-02-10 21:48:57,Tofsee
d2935c58fe676744fecc8614ee5356c7,2018-08-14 21:48:41,2021-02-21 21:44:55,Adwind
8916410db85077a5460817142dcbc8de,2018-08-21 12:32:28,2021-02-21 20:02:19,TrickBot
c5235d3a8b9934b7fbbd204d50bc058d,2018-08-23 17:36:08,2019-10-13 05:11:09,Gootkit
57f3642b4e37e28f5cbe3020c9331b4c,2018-08-28 15:54:53,2021-02-22 11:09:43,Gozi
e62a5f4d538cbf169c2af71bec2399b4,2018-08-30 15:45:40,2021-02-21 20:36:55,TrickBot
51c64c77e60f3980eea90869b68c58a8,2018-08-30 21:04:57,2021-02-21 22:09:16,Dridex
7691297bcb20a41233fd0a0baa0a3628,2018-09-17 02:50:05,2021-02-21 17:05:57,Adware
7dd50e112cd23734a310b90f6f44a7cd,2018-09-17 17:54:58,2021-02-22 11:14:04,Quakbot
52c7396a501e4fecbdfa99c5408334ac,2018-09-18 00:29:04,2019-12-03 17:24:02,Tofsee
f735bbc6b69723b9df7b0e7ef27872af,2018-10-02 18:04:16,2021-02-19 06:55:39,TrickBot
49ed2ef3f1321e5f044f1e71b0e6fdd5,2018-10-02 18:04:17,2021-02-19 06:55:39,TrickBot
d76ee64fb7273733cbe455ac81c292e6,2018-11-16 13:26:39,2018-11-18 19:19:36,Tofsee
8f6c918dcb585ebbea05e2cc94530e3d,2018-11-16 13:26:41,2020-05-06 15:45:21,Tofsee
34f14a69ad7009ca5863379218af17f3,2018-11-17 05:17:22,2021-01-28 08:19:18,Tofsee
c2b4710c6888a5d47befe865c8e6fb19,2018-11-29 20:46:04,2020-12-16 15:08:09,Tofsee
decfb48a53789ebe081b88aabb58ee34,2018-12-21 09:06:16,2021-02-16 12:28:53,Adwind
08a8a4e85b25ac42e1490bc85cfdb5ce,2019-01-30 02:48:34,2020-10-27 09:50:19,Tofsee
c0220cd64849a629397a9cb68f78a0ea,2019-03-24 00:12:32,2021-02-13 05:13:33,Tofsee
7a29c223fb122ec64d10f0a159e07996,2019-06-09 22:55:29,2020-10-27 09:50:26,Tofsee
44dab16d680ef93487bc16ad23b3ffb1,2019-06-09 22:55:29,2020-10-27 09:50:25,Tofsee
70a04365be5bbd4653698bebeb43ce68,2019-07-02 06:26:56,2020-05-30 04:19:00,Tofsee
d81d654effb94714a4086734fa0adad9,2019-07-16 23:29:02,2020-10-27 09:50:21,Tofsee
25d74b7b4b779eb1efd4b31d26d651c6,2019-08-03 20:15:33,2020-07-14 21:43:25,Tofsee
fc2299d5b2964cd242c5a2c8c531a5f0,2019-08-09 23:56:32,2021-02-22 08:49:18,Tofsee
32926ca3e59f0413d0b98725454594f5,2019-09-12 06:56:10,2020-10-27 21:49:31,Tofsee
ffefafdb86336d057eda5fdf02b3d5ce,2019-10-26 07:31:49,2020-07-25 00:14:09,Tofsee
# END (98) entries
################################################################
# abuse.ch Suricata JA3 Fingerprint Blacklist (CSV) #
# For Suricata 4.1.0 or newer #
# Last updated: 2021-08-03 14:33:44 UTC #
# #
# Terms Of Use: https://sslbl.abuse.ch/blacklist/ #
# For questions please contact sslbl [at] abuse.ch #
################################################################
#
# ja3_md5,Firstseen,Lastseen,Listingreason
b386946a5a44d1ddcc843bc75336dfce,2017-07-14 18:08:15,2019-07-27 20:42:54,Dridex
8991a387e4cc841740f25d6f5139f92d,2017-07-14 19:02:03,2019-07-28 00:34:38,Adware
cb98a24ee4b9134448ffb5714fd870ac,2017-07-14 19:48:28,2019-05-22 03:22:38,Dridex
1aa7bf8b97e540ca5edd75f7b8384bfa,2017-07-14 20:23:38,2019-07-28 01:38:22,TrickBot
3d89c0dfb1fa44911b8fa7523ef8dedb,2017-07-15 04:23:45,2021-02-01 18:23:25,Adware
bc6c386f480ee97b9d9e52d472b772d8,2017-07-15 10:57:38,2021-03-13 07:33:39,Adware
8f52d1ce303fb4a6515836aec3cc16b1,2017-07-15 19:05:11,2019-07-27 20:00:57,TrickBot
d6f04b5a910115f4b50ecec09d40a1df,2017-07-15 19:42:24,2018-10-14 08:12:51,Dridex
35c0a31c481927f022a3b530255ac080,2017-07-15 19:43:19,2021-04-10 12:54:04,Tofsee
e330bca99c8a5256ae126a55c4c725c5,2017-07-15 19:59:29,2021-01-13 00:29:37,Adware
d551fafc4f40f1dec2bb45980bfa9492,2017-07-15 19:59:29,2020-11-16 13:06:20,Adware
83e04bc58d402f9633983cbf22724b02,2017-07-16 01:32:03,2021-03-02 04:07:43,Adware
b8f81673c0e1d29908346f3bab892b9b,2017-07-16 01:32:03,2021-03-02 04:07:36,Adware
70722097d1fe1d78d8c2164640ab6df4,2017-07-16 02:39:08,2021-05-04 09:52:20,Tofsee
9c2589e1c0e9f533a022c6205f9719e1,2017-07-16 08:37:17,2021-07-25 08:33:18,Adware
849b04bdbd1d2b983f6e8a457e0632a8,2017-07-16 08:37:17,2021-07-25 08:33:18,Adware
16efcf0e00504ddfedde13bfea997952,2017-07-16 19:45:45,2020-12-23 15:10:32,Adware
4d7a28d6f2263ed61de88ca66eb011e3,2017-07-16 21:20:29,2020-12-08 18:10:55,Tofsee
550dce18de1bb143e69d6dd9413b8355,2017-07-16 22:17:20,2018-12-21 07:04:50,Adware
c50f6a8b9173676b47ba6085bd0c6cee,2017-07-16 22:38:41,2019-05-21 09:42:17,TrickBot
590a232d04d56409fab72e752a8a2634,2017-07-18 18:53:24,2020-10-11 20:48:33,Tofsee
51a7ad14509fd614c7bb3a50c4982b8c,2017-07-19 07:28:19,2019-07-14 11:58:32,JBifrost
96eba628dcb2b47607192ba74a3b55ba,2017-07-19 18:53:48,2021-07-31 01:48:32,Tofsee
df5c30e670dba99f9270ed36060cf054,2017-07-20 17:44:07,2018-04-11 15:57:59,Tofsee
098f55e27d8c4b0a590102cbdb3a5f3a,2017-07-21 09:52:01,2019-04-08 01:09:54,Adware
29085f03f8e8a03f0b399c5c7cf0b0b8,2017-07-22 14:07:36,2021-04-11 06:42:45,Adware
46efd49abcca8ea9baa932da68fdb529,2017-07-22 14:07:36,2021-04-11 05:54:57,Adware
d7150af4514b868defb854db0f62a441,2017-07-23 09:39:24,2018-07-24 01:04:58,Tofsee
03e186a7f83285e93341de478334006e,2017-07-24 18:17:14,2021-03-20 07:45:40,Tofsee
3cda52da4ade09f1f781ad2e82dcfa20,2017-07-30 18:41:36,2019-05-21 17:34:18,Quakbot
b13d01846ad7a14a70bf030a16775c78,2017-08-08 07:12:49,2021-04-10 00:03:33,Adware
1543a7c46633acf71e8401baccbd0568,2017-08-08 21:32:28,2021-04-08 19:50:46,Tofsee
1d095e68489d3c535297cd8dffb06cb9,2017-08-12 19:56:28,2020-10-28 11:06:23,Tofsee
698e36219f3979420fa2581b21dac7ec,2017-08-28 12:20:47,2020-12-31 02:06:31,Adware
93d056782d649deb51cda44ecb714bb0,2017-08-28 12:20:47,2019-04-15 23:47:27,Adware
1712287800ac91b34cadd5884ce85568,2017-08-28 16:01:59,2021-07-28 14:16:00,TorrentLocker
5e573c9c9f8ba720ef9b18e9fce2e2f7,2017-08-30 13:44:56,2021-03-13 07:33:38,Adware
f6fd83a21f9f3c5f9ff7b5c63bbc179d,2017-10-20 08:03:21,2018-11-06 06:42:12,Adware
92579701f145605e9edc0b01a901c6d5,2017-10-23 00:10:48,2021-07-25 17:07:34,Adware
a61299f9b501adcf680b9275d79d4ac6,2017-11-04 18:03:59,2020-04-21 17:08:24,Tofsee
b2b61db7b9490a60d270ccb20b462826,2017-11-14 20:12:03,2021-06-06 20:27:10,Adware
7dcce5b76c8b17472d024758970a406b,2017-11-22 12:42:46,2021-03-16 12:53:35,Tofsee
534ce2dbc413c68e908363b5df0ae5e0,2017-12-22 09:36:21,2019-07-27 15:22:33,TrickBot
fb00055a1196aeea8d1bc609885ba953,2018-01-01 22:49:25,2019-04-09 06:58:58,TrickBot
a50a861119aceb0ccc74902e8fddb618,2018-01-02 08:16:23,2018-07-05 02:33:08,Tofsee
e7643725fcff971e3051fe0e47fc2c71,2018-01-31 08:06:13,2020-03-25 16:19:48,Tofsee
7c410ce832e848a3321432c9a82e972b,2018-01-31 20:04:25,2021-08-01 06:13:14,Tofsee
da949afd9bd6df820730f8f171584a71,2018-02-03 05:19:37,2021-03-08 22:10:10,Tofsee
906004246f3ba5e755b043c057254a29,2018-03-11 08:25:38,2018-04-14 00:59:16,Tofsee
fd80fa9c6120cdeea8520510f3c644ac,2018-03-11 09:34:30,2021-08-11 12:34:00,Tofsee
b90bdbe961a648f0427db21aaa6ccb59,2018-03-11 10:37:43,2020-05-29 23:39:01,Tofsee
1fe4c7a3544eb27afec2adfb3a3dbf60,2018-03-11 19:23:08,2021-08-09 11:42:58,Tofsee
c201b92f8b483fa388be174d6689f534,2018-03-12 13:43:52,2021-01-28 06:17:06,Gozi
9f62c4f26b90d3d757bea609e82f2eaf,2018-03-13 06:23:41,2021-05-20 23:01:06,Tofsee
1be3ecebe5aa9d3654e6e703d81f6928,2018-03-13 11:50:02,2021-08-11 13:02:35,Ransomware.Troldesh
e3b2ab1f9a56f2fb4c9248f2f41631fa,2018-03-15 01:06:34,2021-07-02 21:51:49,Tofsee
dff8a0aa1c904aaea76c5bf624e88333,2018-03-18 09:41:15,2020-10-27 09:50:24,Tofsee
17fd49722f8d11f3d76dce84f8e099a7,2018-03-19 23:02:27,2021-08-01 00:40:52,Tofsee
911479ac8a0813ed1241b3686ccdade9,2018-03-19 23:24:59,2020-03-30 04:09:18,Tofsee
c5deb9465d47232dd48772f9c4d14679,2018-03-22 15:42:48,2021-03-23 00:34:25,Tofsee
f22bdd57e3a52de86cda40da2d84e83b,2018-03-27 13:40:19,2019-01-20 14:31:39,Tofsee
d18a4da84af59e1108862a39bae7c9d4,2018-04-03 00:40:51,2021-02-06 01:53:12,Tofsee
2d8794cb7b52b777bee2695e79c15760,2018-04-04 06:56:37,2021-07-26 08:07:00,Ransomware
40adfd923eb82b89d8836ba37a19bca1,2018-04-15 15:49:08,2021-04-11 04:42:47,CoinMiner
1aee0238942d453d679fc1e37a303387,2018-05-13 01:59:49,2021-07-30 12:27:07,Tofsee
2092e1fffb45d7e4a19a57f9bc5e203a,2018-05-16 21:59:36,2018-09-05 01:58:33,Adware
bffa4501966196d3d6e90cee1f88fc89,2018-06-07 15:08:04,2020-03-16 00:03:44,Tofsee
807fca46d9d0cf63adf4e5e80e414bbe,2018-06-07 16:51:03,2021-08-07 03:15:42,Tofsee
fb58831f892190644fe44e25bc830b45,2018-06-08 12:07:59,2021-07-20 01:39:05,Adware
0cc1e84568e471aa1d62ad4158ade6b5,2018-06-24 10:50:47,2021-06-21 02:35:57,Tofsee
d2935c58fe676744fecc8614ee5356c7,2018-08-14 21:48:41,2021-08-11 11:54:42,Adwind
8916410db85077a5460817142dcbc8de,2018-08-21 12:32:28,2021-08-11 15:00:50,TrickBot
c5235d3a8b9934b7fbbd204d50bc058d,2018-08-23 17:36:08,2019-10-13 05:11:09,Gootkit
57f3642b4e37e28f5cbe3020c9331b4c,2018-08-28 15:54:53,2021-08-11 13:05:18,Gozi
e62a5f4d538cbf169c2af71bec2399b4,2018-08-30 15:45:40,2021-08-11 09:48:52,TrickBot
51c64c77e60f3980eea90869b68c58a8,2018-08-30 21:04:57,2021-08-11 08:13:08,Dridex
7691297bcb20a41233fd0a0baa0a3628,2018-09-17 02:50:05,2021-08-11 12:20:33,Adware
7dd50e112cd23734a310b90f6f44a7cd,2018-09-17 17:54:58,2021-08-01 11:28:46,Quakbot
52c7396a501e4fecbdfa99c5408334ac,2018-09-18 00:29:04,2019-12-03 17:24:02,Tofsee
fc54e0d16d9764783542f0146a98b300,2018-09-24 12:33:44,2021-08-11 12:51:10,AsyncRAT
f735bbc6b69723b9df7b0e7ef27872af,2018-10-02 18:04:16,2021-08-11 07:25:14,TrickBot
49ed2ef3f1321e5f044f1e71b0e6fdd5,2018-10-02 18:04:17,2021-08-08 22:08:01,TrickBot
d76ee64fb7273733cbe455ac81c292e6,2018-11-16 13:26:39,2018-11-18 19:19:36,Tofsee
8f6c918dcb585ebbea05e2cc94530e3d,2018-11-16 13:26:41,2020-05-06 15:45:21,Tofsee
34f14a69ad7009ca5863379218af17f3,2018-11-17 05:17:22,2021-01-28 08:19:18,Tofsee
c2b4710c6888a5d47befe865c8e6fb19,2018-11-29 20:46:04,2021-08-03 23:37:22,Tofsee
decfb48a53789ebe081b88aabb58ee34,2018-12-21 09:06:16,2021-06-14 05:27:16,Adwind
08a8a4e85b25ac42e1490bc85cfdb5ce,2019-01-30 02:48:34,2020-10-27 09:50:19,Tofsee
c0220cd64849a629397a9cb68f78a0ea,2019-03-24 00:12:32,2021-07-31 00:26:06,Tofsee
7a29c223fb122ec64d10f0a159e07996,2019-06-09 22:55:29,2020-10-27 09:50:26,None
70a04365be5bbd4653698bebeb43ce68,2019-07-02 06:26:56,2020-05-30 04:19:00,Tofsee
d81d654effb94714a4086734fa0adad9,2019-07-16 23:29:02,2020-10-27 09:50:21,Tofsee
25d74b7b4b779eb1efd4b31d26d651c6,2019-08-03 20:15:33,2020-07-14 21:43:25,Tofsee
fc2299d5b2964cd242c5a2c8c531a5f0,2019-08-09 23:56:32,2021-08-11 02:20:22,Tofsee
32926ca3e59f0413d0b98725454594f5,2019-09-12 06:56:10,2021-04-08 19:50:43,Tofsee
ffefafdb86336d057eda5fdf02b3d5ce,2019-10-26 07:31:49,2020-07-25 00:14:09,Tofsee
8515076cbbca9dce33151b798f782456,2020-12-27 16:53:04,2021-08-11 15:06:36,BitRAT
# END (97) entries
1 ################################################################
2 # abuse.ch Suricata JA3 Fingerprint Blacklist (CSV) #
3 # For Suricata 4.1.0 or newer #
4 # Last updated: 2020-04-09 06:48:14 UTC # # Last updated: 2021-08-03 14:33:44 UTC #
5 # #
6 # Terms Of Use: https://sslbl.abuse.ch/blacklist/ #
7 # For questions please contact sslbl [at] abuse.ch #
8 ################################################################
9 #
10 # ja3_md5,Firstseen,Lastseen,Listingreason
11 b386946a5a44d1ddcc843bc75336dfce,2017-07-14 18:08:15,2019-07-27 20:42:54,Dridex
12 8991a387e4cc841740f25d6f5139f92d,2017-07-14 19:02:03,2019-07-28 00:34:38,Adware
13 cb98a24ee4b9134448ffb5714fd870ac,2017-07-14 19:48:28,2019-05-22 03:22:38,Dridex
14 1aa7bf8b97e540ca5edd75f7b8384bfa,2017-07-14 20:23:38,2019-07-28 01:38:22,TrickBot
15 3d89c0dfb1fa44911b8fa7523ef8dedb,2017-07-15 04:23:45,2021-02-01 18:23:25,Adware
16 bc6c386f480ee97b9d9e52d472b772d8,2017-07-15 10:57:38,2021-02-21 13:13:30,Adware bc6c386f480ee97b9d9e52d472b772d8,2017-07-15 10:57:38,2021-03-13 07:33:39,Adware
17 8f52d1ce303fb4a6515836aec3cc16b1,2017-07-15 19:05:11,2019-07-27 20:00:57,TrickBot
18 d6f04b5a910115f4b50ecec09d40a1df,2017-07-15 19:42:24,2018-10-14 08:12:51,Dridex
19 35c0a31c481927f022a3b530255ac080,2017-07-15 19:43:19,2021-02-06 03:21:55,Tofsee 35c0a31c481927f022a3b530255ac080,2017-07-15 19:43:19,2021-04-10 12:54:04,Tofsee
20 d551fafc4f40f1dec2bb45980bfa9492,2017-07-15 19:59:29,2020-11-16 13:06:20,Adware e330bca99c8a5256ae126a55c4c725c5,2017-07-15 19:59:29,2021-01-13 00:29:37,Adware
21 e330bca99c8a5256ae126a55c4c725c5,2017-07-15 19:59:29,2021-01-13 00:29:37,Adware d551fafc4f40f1dec2bb45980bfa9492,2017-07-15 19:59:29,2020-11-16 13:06:20,Adware
22 b8f81673c0e1d29908346f3bab892b9b,2017-07-16 01:32:03,2018-12-17 06:08:03,Adware 83e04bc58d402f9633983cbf22724b02,2017-07-16 01:32:03,2021-03-02 04:07:43,Adware
23 83e04bc58d402f9633983cbf22724b02,2017-07-16 01:32:03,2021-01-14 14:23:42,Adware b8f81673c0e1d29908346f3bab892b9b,2017-07-16 01:32:03,2021-03-02 04:07:36,Adware
24 70722097d1fe1d78d8c2164640ab6df4,2017-07-16 02:39:08,2021-02-22 04:56:57,Tofsee 70722097d1fe1d78d8c2164640ab6df4,2017-07-16 02:39:08,2021-05-04 09:52:20,Tofsee
25 9c2589e1c0e9f533a022c6205f9719e1,2017-07-16 08:37:17,2021-02-19 08:46:32,Adware 9c2589e1c0e9f533a022c6205f9719e1,2017-07-16 08:37:17,2021-07-25 08:33:18,Adware
26 849b04bdbd1d2b983f6e8a457e0632a8,2017-07-16 08:37:17,2021-02-19 08:46:32,Adware 849b04bdbd1d2b983f6e8a457e0632a8,2017-07-16 08:37:17,2021-07-25 08:33:18,Adware
27 16efcf0e00504ddfedde13bfea997952,2017-07-16 19:45:45,2020-12-23 15:10:32,Adware
28 4d7a28d6f2263ed61de88ca66eb011e3,2017-07-16 21:20:29,2020-12-08 18:10:55,Tofsee
29 550dce18de1bb143e69d6dd9413b8355,2017-07-16 22:17:20,2018-12-21 07:04:50,Adware
30 c50f6a8b9173676b47ba6085bd0c6cee,2017-07-16 22:38:41,2019-05-21 09:42:17,TrickBot
31 20dd18bdd3209ea718989030a6f93364,2017-07-18 10:22:58,2019-04-28 09:23:31,Adware 590a232d04d56409fab72e752a8a2634,2017-07-18 18:53:24,2020-10-11 20:48:33,Tofsee
32 8498fe4268764dbf926a38283e9d3d8f,2017-07-18 10:22:58,2020-12-31 02:06:27,Adware 51a7ad14509fd614c7bb3a50c4982b8c,2017-07-19 07:28:19,2019-07-14 11:58:32,JBifrost
33 590a232d04d56409fab72e752a8a2634,2017-07-18 18:53:24,2020-10-11 20:48:33,Tofsee 96eba628dcb2b47607192ba74a3b55ba,2017-07-19 18:53:48,2021-07-31 01:48:32,Tofsee
34 51a7ad14509fd614c7bb3a50c4982b8c,2017-07-19 07:28:19,2019-07-14 11:58:32,JBifrost df5c30e670dba99f9270ed36060cf054,2017-07-20 17:44:07,2018-04-11 15:57:59,Tofsee
35 96eba628dcb2b47607192ba74a3b55ba,2017-07-19 18:53:48,2021-01-04 12:45:08,Tofsee 098f55e27d8c4b0a590102cbdb3a5f3a,2017-07-21 09:52:01,2019-04-08 01:09:54,Adware
36 df5c30e670dba99f9270ed36060cf054,2017-07-20 17:44:07,2018-04-11 15:57:59,Tofsee 29085f03f8e8a03f0b399c5c7cf0b0b8,2017-07-22 14:07:36,2021-04-11 06:42:45,Adware
37 098f55e27d8c4b0a590102cbdb3a5f3a,2017-07-21 09:52:01,2019-04-08 01:09:54,Adware 46efd49abcca8ea9baa932da68fdb529,2017-07-22 14:07:36,2021-04-11 05:54:57,Adware
38 46efd49abcca8ea9baa932da68fdb529,2017-07-22 14:07:36,2021-02-20 20:45:32,Adware d7150af4514b868defb854db0f62a441,2017-07-23 09:39:24,2018-07-24 01:04:58,Tofsee
39 29085f03f8e8a03f0b399c5c7cf0b0b8,2017-07-22 14:07:36,2021-02-21 20:56:01,Adware 03e186a7f83285e93341de478334006e,2017-07-24 18:17:14,2021-03-20 07:45:40,Tofsee
40 d7150af4514b868defb854db0f62a441,2017-07-23 09:39:24,2018-07-24 01:04:58,Tofsee 3cda52da4ade09f1f781ad2e82dcfa20,2017-07-30 18:41:36,2019-05-21 17:34:18,Quakbot
41 03e186a7f83285e93341de478334006e,2017-07-24 18:17:14,2021-02-13 23:55:58,Tofsee b13d01846ad7a14a70bf030a16775c78,2017-08-08 07:12:49,2021-04-10 00:03:33,Adware
42 3cda52da4ade09f1f781ad2e82dcfa20,2017-07-30 18:41:36,2019-05-21 17:34:18,Quakbot 1543a7c46633acf71e8401baccbd0568,2017-08-08 21:32:28,2021-04-08 19:50:46,Tofsee
43 b13d01846ad7a14a70bf030a16775c78,2017-08-08 07:12:49,2021-02-21 01:32:12,Adware 1d095e68489d3c535297cd8dffb06cb9,2017-08-12 19:56:28,2020-10-28 11:06:23,Tofsee
44 1543a7c46633acf71e8401baccbd0568,2017-08-08 21:32:28,2020-11-10 05:30:17,Tofsee 698e36219f3979420fa2581b21dac7ec,2017-08-28 12:20:47,2020-12-31 02:06:31,Adware
45 1d095e68489d3c535297cd8dffb06cb9,2017-08-12 19:56:28,2020-10-28 11:06:23,Tofsee 93d056782d649deb51cda44ecb714bb0,2017-08-28 12:20:47,2019-04-15 23:47:27,Adware
46 93d056782d649deb51cda44ecb714bb0,2017-08-28 12:20:47,2019-04-15 23:47:27,Adware 1712287800ac91b34cadd5884ce85568,2017-08-28 16:01:59,2021-07-28 14:16:00,TorrentLocker
47 698e36219f3979420fa2581b21dac7ec,2017-08-28 12:20:47,2020-12-31 02:06:31,Adware 5e573c9c9f8ba720ef9b18e9fce2e2f7,2017-08-30 13:44:56,2021-03-13 07:33:38,Adware
48 1712287800ac91b34cadd5884ce85568,2017-08-28 16:01:59,2021-01-23 09:36:32,TorrentLocker f6fd83a21f9f3c5f9ff7b5c63bbc179d,2017-10-20 08:03:21,2018-11-06 06:42:12,Adware
49 5e573c9c9f8ba720ef9b18e9fce2e2f7,2017-08-30 13:44:56,2021-02-17 17:11:34,Adware 92579701f145605e9edc0b01a901c6d5,2017-10-23 00:10:48,2021-07-25 17:07:34,Adware
50 f6fd83a21f9f3c5f9ff7b5c63bbc179d,2017-10-20 08:03:21,2018-11-06 06:42:12,Adware a61299f9b501adcf680b9275d79d4ac6,2017-11-04 18:03:59,2020-04-21 17:08:24,Tofsee
51 92579701f145605e9edc0b01a901c6d5,2017-10-23 00:10:48,2021-02-18 22:56:49,Adware b2b61db7b9490a60d270ccb20b462826,2017-11-14 20:12:03,2021-06-06 20:27:10,Adware
52 a61299f9b501adcf680b9275d79d4ac6,2017-11-04 18:03:59,2020-04-21 17:08:24,Tofsee 7dcce5b76c8b17472d024758970a406b,2017-11-22 12:42:46,2021-03-16 12:53:35,Tofsee
53 b2b61db7b9490a60d270ccb20b462826,2017-11-14 20:12:03,2021-02-18 22:56:49,Adware 534ce2dbc413c68e908363b5df0ae5e0,2017-12-22 09:36:21,2019-07-27 15:22:33,TrickBot
54 7dcce5b76c8b17472d024758970a406b,2017-11-22 12:42:46,2020-12-22 15:15:49,Tofsee fb00055a1196aeea8d1bc609885ba953,2018-01-01 22:49:25,2019-04-09 06:58:58,TrickBot
55 534ce2dbc413c68e908363b5df0ae5e0,2017-12-22 09:36:21,2019-07-27 15:22:33,TrickBot a50a861119aceb0ccc74902e8fddb618,2018-01-02 08:16:23,2018-07-05 02:33:08,Tofsee
56 fb00055a1196aeea8d1bc609885ba953,2018-01-01 22:49:25,2019-04-09 06:58:58,TrickBot e7643725fcff971e3051fe0e47fc2c71,2018-01-31 08:06:13,2020-03-25 16:19:48,Tofsee
57 a50a861119aceb0ccc74902e8fddb618,2018-01-02 08:16:23,2018-07-05 02:33:08,Tofsee 7c410ce832e848a3321432c9a82e972b,2018-01-31 20:04:25,2021-08-01 06:13:14,Tofsee
58 e7643725fcff971e3051fe0e47fc2c71,2018-01-31 08:06:13,2020-03-25 16:19:48,Tofsee da949afd9bd6df820730f8f171584a71,2018-02-03 05:19:37,2021-03-08 22:10:10,Tofsee
59 7c410ce832e848a3321432c9a82e972b,2018-01-31 20:04:25,2021-02-21 22:51:31,Tofsee 906004246f3ba5e755b043c057254a29,2018-03-11 08:25:38,2018-04-14 00:59:16,Tofsee
60 da949afd9bd6df820730f8f171584a71,2018-02-03 05:19:37,2021-02-21 05:20:43,Tofsee fd80fa9c6120cdeea8520510f3c644ac,2018-03-11 09:34:30,2021-08-11 12:34:00,Tofsee
61 906004246f3ba5e755b043c057254a29,2018-03-11 08:25:38,2018-04-14 00:59:16,Tofsee b90bdbe961a648f0427db21aaa6ccb59,2018-03-11 10:37:43,2020-05-29 23:39:01,Tofsee
62 fd80fa9c6120cdeea8520510f3c644ac,2018-03-11 09:34:30,2021-02-21 22:16:39,Tofsee 1fe4c7a3544eb27afec2adfb3a3dbf60,2018-03-11 19:23:08,2021-08-09 11:42:58,Tofsee
63 b90bdbe961a648f0427db21aaa6ccb59,2018-03-11 10:37:43,2020-05-29 23:39:01,Tofsee c201b92f8b483fa388be174d6689f534,2018-03-12 13:43:52,2021-01-28 06:17:06,Gozi
64 1fe4c7a3544eb27afec2adfb3a3dbf60,2018-03-11 19:23:08,2021-02-20 11:19:52,Tofsee 9f62c4f26b90d3d757bea609e82f2eaf,2018-03-13 06:23:41,2021-05-20 23:01:06,Tofsee
65 c201b92f8b483fa388be174d6689f534,2018-03-12 13:43:52,2021-01-28 06:17:06,Gozi 1be3ecebe5aa9d3654e6e703d81f6928,2018-03-13 11:50:02,2021-08-11 13:02:35,Ransomware.Troldesh
66 9f62c4f26b90d3d757bea609e82f2eaf,2018-03-13 06:23:41,2020-12-18 16:07:31,Tofsee e3b2ab1f9a56f2fb4c9248f2f41631fa,2018-03-15 01:06:34,2021-07-02 21:51:49,Tofsee
67 1be3ecebe5aa9d3654e6e703d81f6928,2018-03-13 11:50:02,2021-02-14 23:03:13,Ransomware.Troldesh dff8a0aa1c904aaea76c5bf624e88333,2018-03-18 09:41:15,2020-10-27 09:50:24,Tofsee
68 e3b2ab1f9a56f2fb4c9248f2f41631fa,2018-03-15 01:06:34,2021-02-22 10:36:48,Tofsee 17fd49722f8d11f3d76dce84f8e099a7,2018-03-19 23:02:27,2021-08-01 00:40:52,Tofsee
69 dff8a0aa1c904aaea76c5bf624e88333,2018-03-18 09:41:15,2020-10-27 09:50:24,Tofsee 911479ac8a0813ed1241b3686ccdade9,2018-03-19 23:24:59,2020-03-30 04:09:18,Tofsee
70 17fd49722f8d11f3d76dce84f8e099a7,2018-03-19 23:02:27,2020-12-09 11:32:21,Tofsee c5deb9465d47232dd48772f9c4d14679,2018-03-22 15:42:48,2021-03-23 00:34:25,Tofsee
71 911479ac8a0813ed1241b3686ccdade9,2018-03-19 23:24:59,2020-03-30 04:09:18,Tofsee f22bdd57e3a52de86cda40da2d84e83b,2018-03-27 13:40:19,2019-01-20 14:31:39,Tofsee
72 c5deb9465d47232dd48772f9c4d14679,2018-03-22 15:42:48,2021-02-17 07:39:32,Tofsee d18a4da84af59e1108862a39bae7c9d4,2018-04-03 00:40:51,2021-02-06 01:53:12,Tofsee
73 f22bdd57e3a52de86cda40da2d84e83b,2018-03-27 13:40:19,2019-01-20 14:31:39,Tofsee 2d8794cb7b52b777bee2695e79c15760,2018-04-04 06:56:37,2021-07-26 08:07:00,Ransomware
74 d18a4da84af59e1108862a39bae7c9d4,2018-04-03 00:40:51,2021-02-06 01:53:12,Tofsee 40adfd923eb82b89d8836ba37a19bca1,2018-04-15 15:49:08,2021-04-11 04:42:47,CoinMiner
75 2d8794cb7b52b777bee2695e79c15760,2018-04-04 06:56:37,2021-02-10 08:22:31,Ransomware 1aee0238942d453d679fc1e37a303387,2018-05-13 01:59:49,2021-07-30 12:27:07,Tofsee
76 40adfd923eb82b89d8836ba37a19bca1,2018-04-15 15:49:08,2021-02-21 22:22:56,CoinMiner 2092e1fffb45d7e4a19a57f9bc5e203a,2018-05-16 21:59:36,2018-09-05 01:58:33,Adware
77 1aee0238942d453d679fc1e37a303387,2018-05-13 01:59:49,2021-02-10 13:19:24,Tofsee bffa4501966196d3d6e90cee1f88fc89,2018-06-07 15:08:04,2020-03-16 00:03:44,Tofsee
78 2092e1fffb45d7e4a19a57f9bc5e203a,2018-05-16 21:59:36,2018-09-05 01:58:33,Adware 807fca46d9d0cf63adf4e5e80e414bbe,2018-06-07 16:51:03,2021-08-07 03:15:42,Tofsee
79 bffa4501966196d3d6e90cee1f88fc89,2018-06-07 15:08:04,2020-03-16 00:03:44,Tofsee fb58831f892190644fe44e25bc830b45,2018-06-08 12:07:59,2021-07-20 01:39:05,Adware
80 807fca46d9d0cf63adf4e5e80e414bbe,2018-06-07 16:51:03,2021-02-20 03:15:38,Tofsee 0cc1e84568e471aa1d62ad4158ade6b5,2018-06-24 10:50:47,2021-06-21 02:35:57,Tofsee
81 fb58831f892190644fe44e25bc830b45,2018-06-08 12:07:59,2019-05-31 21:02:37,Adware d2935c58fe676744fecc8614ee5356c7,2018-08-14 21:48:41,2021-08-11 11:54:42,Adwind
82 0cc1e84568e471aa1d62ad4158ade6b5,2018-06-24 10:50:47,2021-02-10 21:48:57,Tofsee 8916410db85077a5460817142dcbc8de,2018-08-21 12:32:28,2021-08-11 15:00:50,TrickBot
83 d2935c58fe676744fecc8614ee5356c7,2018-08-14 21:48:41,2021-02-21 21:44:55,Adwind c5235d3a8b9934b7fbbd204d50bc058d,2018-08-23 17:36:08,2019-10-13 05:11:09,Gootkit
84 8916410db85077a5460817142dcbc8de,2018-08-21 12:32:28,2021-02-21 20:02:19,TrickBot 57f3642b4e37e28f5cbe3020c9331b4c,2018-08-28 15:54:53,2021-08-11 13:05:18,Gozi
85 c5235d3a8b9934b7fbbd204d50bc058d,2018-08-23 17:36:08,2019-10-13 05:11:09,Gootkit e62a5f4d538cbf169c2af71bec2399b4,2018-08-30 15:45:40,2021-08-11 09:48:52,TrickBot
86 57f3642b4e37e28f5cbe3020c9331b4c,2018-08-28 15:54:53,2021-02-22 11:09:43,Gozi 51c64c77e60f3980eea90869b68c58a8,2018-08-30 21:04:57,2021-08-11 08:13:08,Dridex
87 e62a5f4d538cbf169c2af71bec2399b4,2018-08-30 15:45:40,2021-02-21 20:36:55,TrickBot 7691297bcb20a41233fd0a0baa0a3628,2018-09-17 02:50:05,2021-08-11 12:20:33,Adware
88 51c64c77e60f3980eea90869b68c58a8,2018-08-30 21:04:57,2021-02-21 22:09:16,Dridex 7dd50e112cd23734a310b90f6f44a7cd,2018-09-17 17:54:58,2021-08-01 11:28:46,Quakbot
89 7691297bcb20a41233fd0a0baa0a3628,2018-09-17 02:50:05,2021-02-21 17:05:57,Adware 52c7396a501e4fecbdfa99c5408334ac,2018-09-18 00:29:04,2019-12-03 17:24:02,Tofsee
90 7dd50e112cd23734a310b90f6f44a7cd,2018-09-17 17:54:58,2021-02-22 11:14:04,Quakbot fc54e0d16d9764783542f0146a98b300,2018-09-24 12:33:44,2021-08-11 12:51:10,AsyncRAT
91 52c7396a501e4fecbdfa99c5408334ac,2018-09-18 00:29:04,2019-12-03 17:24:02,Tofsee f735bbc6b69723b9df7b0e7ef27872af,2018-10-02 18:04:16,2021-08-11 07:25:14,TrickBot
92 f735bbc6b69723b9df7b0e7ef27872af,2018-10-02 18:04:16,2021-02-19 06:55:39,TrickBot 49ed2ef3f1321e5f044f1e71b0e6fdd5,2018-10-02 18:04:17,2021-08-08 22:08:01,TrickBot
93 49ed2ef3f1321e5f044f1e71b0e6fdd5,2018-10-02 18:04:17,2021-02-19 06:55:39,TrickBot d76ee64fb7273733cbe455ac81c292e6,2018-11-16 13:26:39,2018-11-18 19:19:36,Tofsee
94 d76ee64fb7273733cbe455ac81c292e6,2018-11-16 13:26:39,2018-11-18 19:19:36,Tofsee 8f6c918dcb585ebbea05e2cc94530e3d,2018-11-16 13:26:41,2020-05-06 15:45:21,Tofsee
95 8f6c918dcb585ebbea05e2cc94530e3d,2018-11-16 13:26:41,2020-05-06 15:45:21,Tofsee 34f14a69ad7009ca5863379218af17f3,2018-11-17 05:17:22,2021-01-28 08:19:18,Tofsee
96 34f14a69ad7009ca5863379218af17f3,2018-11-17 05:17:22,2021-01-28 08:19:18,Tofsee c2b4710c6888a5d47befe865c8e6fb19,2018-11-29 20:46:04,2021-08-03 23:37:22,Tofsee
97 c2b4710c6888a5d47befe865c8e6fb19,2018-11-29 20:46:04,2020-12-16 15:08:09,Tofsee decfb48a53789ebe081b88aabb58ee34,2018-12-21 09:06:16,2021-06-14 05:27:16,Adwind
98 decfb48a53789ebe081b88aabb58ee34,2018-12-21 09:06:16,2021-02-16 12:28:53,Adwind 08a8a4e85b25ac42e1490bc85cfdb5ce,2019-01-30 02:48:34,2020-10-27 09:50:19,Tofsee
99 08a8a4e85b25ac42e1490bc85cfdb5ce,2019-01-30 02:48:34,2020-10-27 09:50:19,Tofsee c0220cd64849a629397a9cb68f78a0ea,2019-03-24 00:12:32,2021-07-31 00:26:06,Tofsee
100 c0220cd64849a629397a9cb68f78a0ea,2019-03-24 00:12:32,2021-02-13 05:13:33,Tofsee 7a29c223fb122ec64d10f0a159e07996,2019-06-09 22:55:29,2020-10-27 09:50:26,None
101 7a29c223fb122ec64d10f0a159e07996,2019-06-09 22:55:29,2020-10-27 09:50:26,Tofsee 70a04365be5bbd4653698bebeb43ce68,2019-07-02 06:26:56,2020-05-30 04:19:00,Tofsee
102 44dab16d680ef93487bc16ad23b3ffb1,2019-06-09 22:55:29,2020-10-27 09:50:25,Tofsee d81d654effb94714a4086734fa0adad9,2019-07-16 23:29:02,2020-10-27 09:50:21,Tofsee
103 70a04365be5bbd4653698bebeb43ce68,2019-07-02 06:26:56,2020-05-30 04:19:00,Tofsee 25d74b7b4b779eb1efd4b31d26d651c6,2019-08-03 20:15:33,2020-07-14 21:43:25,Tofsee
104 d81d654effb94714a4086734fa0adad9,2019-07-16 23:29:02,2020-10-27 09:50:21,Tofsee fc2299d5b2964cd242c5a2c8c531a5f0,2019-08-09 23:56:32,2021-08-11 02:20:22,Tofsee
105 25d74b7b4b779eb1efd4b31d26d651c6,2019-08-03 20:15:33,2020-07-14 21:43:25,Tofsee 32926ca3e59f0413d0b98725454594f5,2019-09-12 06:56:10,2021-04-08 19:50:43,Tofsee
106 fc2299d5b2964cd242c5a2c8c531a5f0,2019-08-09 23:56:32,2021-02-22 08:49:18,Tofsee ffefafdb86336d057eda5fdf02b3d5ce,2019-10-26 07:31:49,2020-07-25 00:14:09,Tofsee
107 32926ca3e59f0413d0b98725454594f5,2019-09-12 06:56:10,2020-10-27 21:49:31,Tofsee 8515076cbbca9dce33151b798f782456,2020-12-27 16:53:04,2021-08-11 15:06:36,BitRAT
108 ffefafdb86336d057eda5fdf02b3d5ce,2019-10-26 07:31:49,2020-07-25 00:14:09,Tofsee # END (97) entries
# END (98) entries
+2995 -1253
View File
File diff suppressed because it is too large Load Diff
+107 -52
View File
@@ -1,8 +1,25 @@
/*
*
* Copyright (C) 2011-22 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#ifndef WIN32
#include <arpa/inet.h>
#include <netinet/in.h>
#else
#include <windows.h>
#endif
#include <errno.h>
#include <ndpi_api.h>
@@ -16,6 +33,10 @@
#include <string.h>
#include <unistd.h>
#ifdef WIN32
#include <windows.h>
#endif
//#define VERBOSE 1
#define MAX_FLOW_ROOTS_PER_THREAD 2048
#define MAX_IDLE_FLOWS_PER_THREAD 64
@@ -90,7 +111,7 @@ struct nDPI_flow_info {
struct nDPI_workflow {
pcap_t * pcap_handle;
uint8_t error_or_eof;
volatile long int error_or_eof;
unsigned long long int packets_captured;
unsigned long long int packets_processed;
@@ -116,13 +137,13 @@ struct nDPI_workflow {
struct nDPI_reader_thread {
struct nDPI_workflow * workflow;
pthread_t thread_id;
int array_index;
uint32_t array_index;
};
static struct nDPI_reader_thread reader_threads[MAX_READER_THREADS] = {};
static int reader_thread_count = MAX_READER_THREADS;
static int main_thread_shutdown = 0;
static uint32_t flow_id = 0;
static volatile long int main_thread_shutdown = 0;
static volatile long int flow_id = 0;
static void free_workflow(struct nDPI_workflow ** const workflow);
@@ -130,7 +151,10 @@ static struct nDPI_workflow * init_workflow(char const * const file_or_device)
{
char pcap_error_buffer[PCAP_ERRBUF_SIZE];
struct nDPI_workflow * workflow = (struct nDPI_workflow *)ndpi_calloc(1, sizeof(*workflow));
const char *bpfFilter = "ip or ip6";
static struct bpf_program bpf_code;
static struct bpf_program *bpf_cfilter = NULL;
if (workflow == NULL) {
return NULL;
}
@@ -153,8 +177,18 @@ static struct nDPI_workflow * init_workflow(char const * const file_or_device)
return NULL;
}
ndpi_init_prefs init_prefs = ndpi_no_prefs;
workflow->ndpi_struct = ndpi_init_detection_module(init_prefs);
if(pcap_compile(workflow->pcap_handle, &bpf_code, bpfFilter, 1, 0xFFFFFF00) < 0) {
printf("pcap_compile error: '%s'\n", pcap_geterr(workflow->pcap_handle));
exit(-1);
}
bpf_cfilter = &bpf_code;
if(pcap_setfilter(workflow->pcap_handle, bpf_cfilter) < 0) {
printf("pcap_setfilter error: '%s'\n", pcap_geterr(workflow->pcap_handle));
}
workflow->ndpi_struct = ndpi_init_detection_module(NULL);
if (workflow->ndpi_struct == NULL) {
free_workflow(&workflow);
return NULL;
@@ -195,6 +229,7 @@ static void ndpi_flow_info_freer(void * const node)
static void free_workflow(struct nDPI_workflow ** const workflow)
{
struct nDPI_workflow * const w = *workflow;
size_t i;
if (w == NULL) {
return;
@@ -208,7 +243,7 @@ static void free_workflow(struct nDPI_workflow ** const workflow)
if (w->ndpi_struct != NULL) {
ndpi_exit_detection_module(w->ndpi_struct);
}
for(size_t i = 0; i < w->max_active_flows; i++) {
for(i = 0; i < w->max_active_flows; i++) {
ndpi_tdestroy(w->ndpi_flows_active[i], ndpi_flow_info_freer);
}
ndpi_free(w->ndpi_flows_active);
@@ -237,6 +272,7 @@ static int setup_reader_threads(char const * const file_or_device)
{
char * file_or_default_device;
char pcap_error_buffer[PCAP_ERRBUF_SIZE];
int i;
if (reader_thread_count > MAX_READER_THREADS) {
return 1;
@@ -255,7 +291,7 @@ static int setup_reader_threads(char const * const file_or_device)
}
}
for (int i = 0; i < reader_thread_count; ++i) {
for (i = 0; i < reader_thread_count; ++i) {
reader_threads[i].workflow = init_workflow(file_or_default_device);
if (reader_threads[i].workflow == NULL)
{
@@ -300,7 +336,7 @@ static void print_packet_info(struct nDPI_reader_thread const * const reader_thr
char buf[256];
int used = 0, ret;
ret = snprintf(buf, sizeof(buf), "[%8llu, %d, %4u] %4u bytes: ",
ret = ndpi_snprintf(buf, sizeof(buf), "[%8llu, %d, %4u] %4u bytes: ",
workflow->packets_captured, reader_thread->array_index,
flow->flow_id, header->caplen);
if (ret > 0) {
@@ -308,9 +344,9 @@ static void print_packet_info(struct nDPI_reader_thread const * const reader_thr
}
if (ip_tuple_to_string(flow, src_addr_str, sizeof(src_addr_str), dst_addr_str, sizeof(dst_addr_str)) != 0) {
ret = snprintf(buf + used, sizeof(buf) - used, "IP[%s -> %s]", src_addr_str, dst_addr_str);
ret = ndpi_snprintf(buf + used, sizeof(buf) - used, "IP[%s -> %s]", src_addr_str, dst_addr_str);
} else {
ret = snprintf(buf + used, sizeof(buf) - used, "IP[ERROR]");
ret = ndpi_snprintf(buf + used, sizeof(buf) - used, "IP[ERROR]");
}
if (ret > 0) {
used += ret;
@@ -318,24 +354,24 @@ static void print_packet_info(struct nDPI_reader_thread const * const reader_thr
switch (flow->l4_protocol) {
case IPPROTO_UDP:
ret = snprintf(buf + used, sizeof(buf) - used, " -> UDP[%u -> %u, %u bytes]",
ret = ndpi_snprintf(buf + used, sizeof(buf) - used, " -> UDP[%u -> %u, %u bytes]",
flow->src_port, flow->dst_port, l4_data_len);
break;
case IPPROTO_TCP:
ret = snprintf(buf + used, sizeof(buf) - used, " -> TCP[%u -> %u, %u bytes]",
ret = ndpi_snprintf(buf + used, sizeof(buf) - used, " -> TCP[%u -> %u, %u bytes]",
flow->src_port, flow->dst_port, l4_data_len);
break;
case IPPROTO_ICMP:
ret = snprintf(buf + used, sizeof(buf) - used, " -> ICMP");
ret = ndpi_snprintf(buf + used, sizeof(buf) - used, " -> ICMP");
break;
case IPPROTO_ICMPV6:
ret = snprintf(buf + used, sizeof(buf) - used, " -> ICMP6");
ret = ndpi_snprintf(buf + used, sizeof(buf) - used, " -> ICMP6");
break;
case IPPROTO_HOPOPTS:
ret = snprintf(buf + used, sizeof(buf) - used, " -> ICMP6 Hop-By-Hop");
ret = ndpi_snprintf(buf + used, sizeof(buf) - used, " -> ICMP6 Hop-By-Hop");
break;
default:
ret = snprintf(buf + used, sizeof(buf) - used, " -> Unknown[0x%X]", flow->l4_protocol);
ret = ndpi_snprintf(buf + used, sizeof(buf) - used, " -> Unknown[0x%X]", flow->l4_protocol);
break;
}
if (ret > 0) {
@@ -465,8 +501,10 @@ static int ndpi_workflow_node_cmp(void const * const A, void const * const B) {
static void check_for_idle_flows(struct nDPI_workflow * const workflow)
{
size_t idle_scan_index;
if (workflow->last_idle_scan_time + IDLE_SCAN_PERIOD < workflow->last_time) {
for (size_t idle_scan_index = 0; idle_scan_index < workflow->max_active_flows; ++idle_scan_index) {
for (idle_scan_index = 0; idle_scan_index < workflow->max_active_flows; ++idle_scan_index) {
ndpi_twalk(workflow->ndpi_flows_active[idle_scan_index], ndpi_idle_scan_walker, workflow);
while (workflow->cur_idle_flows > 0) {
@@ -495,7 +533,7 @@ static void ndpi_process_packet(uint8_t * const args,
struct nDPI_reader_thread * const reader_thread =
(struct nDPI_reader_thread *)args;
struct nDPI_workflow * workflow;
struct nDPI_flow_info flow = {};
struct nDPI_flow_info flow;
size_t hashed_index;
void * tree_result;
@@ -514,7 +552,9 @@ static void ndpi_process_packet(uint8_t * const args,
uint16_t l4_len = 0;
uint16_t type;
int thread_index = INITIAL_THREAD_HASH; // generated with `dd if=/dev/random bs=1024 count=1 |& hd'
uint32_t thread_index = INITIAL_THREAD_HASH; // generated with `dd if=/dev/random bs=1024 count=1 |& hd'
memset(&flow, '\0', sizeof(flow));
if (reader_thread == NULL) {
return;
@@ -701,24 +741,33 @@ static void ndpi_process_packet(uint8_t * const args,
print_packet_info(reader_thread, header, l4_len, &flow);
#endif
/* calculate flow hash for btree find, search(insert) */
if (flow.l3_type == L3_IP) {
if (ndpi_flowv4_flow_hash(flow.l4_protocol, flow.ip_tuple.v4.src, flow.ip_tuple.v4.dst,
flow.src_port, flow.dst_port, 0, 0,
(uint8_t *)&flow.hashval, sizeof(flow.hashval)) != 0)
{
uint64_t tmp[4] = {};
/* calculate flow hash for btree find, search(insert) */
if (flow.l3_type == L3_IP) {
if (ndpi_flowv4_flow_hash(flow.l4_protocol, flow.ip_tuple.v4.src, flow.ip_tuple.v4.dst,
flow.src_port, flow.dst_port, 0, 0,
(uint8_t *)&tmp[0], sizeof(tmp)) != 0)
{
flow.hashval = flow.ip_tuple.v4.src + flow.ip_tuple.v4.dst; // fallback
flow.hashval = flow.ip_tuple.v4.src + flow.ip_tuple.v4.dst; // fallback
} else {
flow.hashval = tmp[0] + tmp[1] + tmp[2] + tmp[3];
}
} else if (flow.l3_type == L3_IP6) {
if (ndpi_flowv6_flow_hash(flow.l4_protocol, &ip6->ip6_src, &ip6->ip6_dst,
flow.src_port, flow.dst_port, 0, 0,
(uint8_t *)&flow.hashval, sizeof(flow.hashval)) != 0)
} else if (flow.l3_type == L3_IP6) {
if (ndpi_flowv6_flow_hash(flow.l4_protocol, &ip6->ip6_src, &ip6->ip6_dst,
flow.src_port, flow.dst_port, 0, 0,
(uint8_t *)&tmp[0], sizeof(tmp)) != 0)
{
flow.hashval = flow.ip_tuple.v6.src[0] + flow.ip_tuple.v6.src[1];
flow.hashval += flow.ip_tuple.v6.dst[0] + flow.ip_tuple.v6.dst[1];
flow.hashval = flow.ip_tuple.v6.src[0] + flow.ip_tuple.v6.src[1];
flow.hashval += flow.ip_tuple.v6.dst[0] + flow.ip_tuple.v6.dst[1];
} else {
flow.hashval = tmp[0] + tmp[1] + tmp[2] + tmp[3];
}
}
flow.hashval += flow.l4_protocol + flow.src_port + flow.dst_port;
}
flow.hashval += flow.l4_protocol + flow.src_port + flow.dst_port;
hashed_index = flow.hashval % workflow->max_active_flows;
tree_result = ndpi_tfind(&flow, &workflow->ndpi_flows_active[hashed_index], ndpi_workflow_node_cmp);
@@ -776,8 +825,6 @@ static void ndpi_process_packet(uint8_t * const args,
return;
}
workflow->cur_active_flows++;
workflow->total_active_flows++;
memcpy(flow_to_process, &flow, sizeof(*flow_to_process));
flow_to_process->flow_id = __sync_fetch_and_add(&flow_id, 1);
@@ -796,6 +843,9 @@ static void ndpi_process_packet(uint8_t * const args,
/* Possible Leak, but should not happen as we'd abort earlier. */
return;
}
workflow->cur_active_flows++;
workflow->total_active_flows++;
} else {
flow_to_process = *(struct nDPI_flow_info **)tree_result;
}
@@ -830,7 +880,7 @@ static void ndpi_process_packet(uint8_t * const args,
flow_to_process->guessed_protocol =
ndpi_detection_giveup(workflow->ndpi_struct,
flow_to_process->ndpi_flow,
1, &protocol_was_guessed);
&protocol_was_guessed);
if (protocol_was_guessed != 0) {
printf("[%8llu, %d, %4d][GUESSED] protocol: %s | app protocol: %s | category: %s\n",
workflow->packets_captured,
@@ -848,10 +898,9 @@ static void ndpi_process_packet(uint8_t * const args,
flow_to_process->detected_l7_protocol =
ndpi_detection_process_packet(workflow->ndpi_struct, flow_to_process->ndpi_flow,
ip != NULL ? (uint8_t *)ip : (uint8_t *)ip6,
ip_size, time_ms);
ip_size, time_ms, NULL);
if (ndpi_is_protocol_detected(workflow->ndpi_struct,
flow_to_process->detected_l7_protocol) != 0 &&
if (ndpi_is_protocol_detected(flow_to_process->detected_l7_protocol) != 0 &&
flow_to_process->detection_completed == 0)
{
if (flow_to_process->detected_l7_protocol.master_protocol != NDPI_PROTOCOL_UNKNOWN ||
@@ -907,7 +956,7 @@ static void ndpi_process_packet(uint8_t * const args,
{
uint8_t unknown_tls_version = 0;
char buf_ver[16];
printf("[%8llu, %d, %4d][TLS-CLIENT-HELLO] version: %s | sni: %s | alpn: %s\n",
printf("[%8llu, %d, %4d][TLS-CLIENT-HELLO] version: %s | sni: %s | (advertised) ALPNs: %s\n",
workflow->packets_captured,
reader_thread->array_index,
flow_to_process->flow_id,
@@ -915,12 +964,12 @@ static void ndpi_process_packet(uint8_t * const args,
flow_to_process->ndpi_flow->protos.tls_quic.ssl_version,
&unknown_tls_version),
flow_to_process->ndpi_flow->host_server_name,
(flow_to_process->ndpi_flow->protos.tls_quic.alpn != NULL ?
flow_to_process->ndpi_flow->protos.tls_quic.alpn : "-"));
(flow_to_process->ndpi_flow->protos.tls_quic.advertised_alpns != NULL ?
flow_to_process->ndpi_flow->protos.tls_quic.advertised_alpns : "-"));
flow_to_process->tls_client_hello_seen = 1;
}
if (flow_to_process->tls_server_hello_seen == 0 &&
flow_to_process->ndpi_flow->l4.tcp.tls.certificate_processed != 0)
flow_to_process->ndpi_flow->tls_quic.certificate_processed != 0)
{
uint8_t unknown_tls_version = 0;
char buf_ver[16];
@@ -983,7 +1032,9 @@ static void * processing_thread(void * const ndpi_thread_arg)
static int processing_threads_error_or_eof(void)
{
for (int i = 0; i < reader_thread_count; ++i) {
int i;
for (i = 0; i < reader_thread_count; ++i) {
if (__sync_fetch_and_add(&reader_threads[i].workflow->error_or_eof, 0) == 0) {
return 0;
}
@@ -993,6 +1044,8 @@ static int processing_threads_error_or_eof(void)
static int start_reader_threads(void)
{
int i;
#ifndef WIN32
sigset_t thread_signal_set, old_signal_set;
@@ -1005,7 +1058,7 @@ static int start_reader_threads(void)
}
#endif
for (int i = 0; i < reader_thread_count; ++i) {
for (i = 0; i < reader_thread_count; ++i) {
reader_threads[i].array_index = i;
if (reader_threads[i].workflow == NULL) {
@@ -1031,6 +1084,7 @@ static int start_reader_threads(void)
static int stop_reader_threads(void)
{
int i;
unsigned long long int total_packets_captured = 0;
unsigned long long int total_packets_processed = 0;
unsigned long long int total_l4_data_len = 0;
@@ -1038,13 +1092,13 @@ static int stop_reader_threads(void)
unsigned long long int total_flows_idle = 0;
unsigned long long int total_flows_detected = 0;
for (int i = 0; i < reader_thread_count; ++i) {
for (i = 0; i < reader_thread_count; ++i) {
break_pcap_loop(&reader_threads[i]);
}
printf("------------------------------------ Stopping reader threads\n");
for (int i = 0; i < reader_thread_count; ++i) {
for (i = 0; i < reader_thread_count; ++i) {
if (reader_threads[i].workflow == NULL) {
continue;
}
@@ -1069,7 +1123,7 @@ static int stop_reader_threads(void)
/* total packets captured: same value for all threads as packet2thread distribution happens later */
total_packets_captured = reader_threads[0].workflow->packets_captured;
for (int i = 0; i < reader_thread_count; ++i) {
for (i = 0; i < reader_thread_count; ++i) {
if (reader_threads[i].workflow == NULL) {
continue;
}
@@ -1101,9 +1155,10 @@ static void sighandler(int signum)
int main(int argc, char ** argv)
{
if (argc == 0) {
printf("usage: ndpiSimpleIntegration <device name>\n");
return 1;
}
printf("usage: %s [PCAP-FILE-OR-INTERFACE]\n"
"----------------------------------\n"
"nDPI version: %s\n"
+37 -3
View File
@@ -2,9 +2,20 @@
# <tcp|udp>:<port>,<tcp|udp>:<port>,.....@<proto>
tcp:81,tcp:8181@HTTP
udp:5061-5062@SIP
udp:5062@SIP
tcp:860,udp:860,tcp:3260,udp:3260@iSCSI
tcp:3000@ntop
tcp:9002@Elasticsearch
tcp:5601@Kibana
tcp:65535@TestProto
#
# nBPF filters (https://github.com/ntop/PF_RING/tree/dev/userland/nbpf))
#
# NOTE: they are evaluated in the same order they are defined !
#
nbpf:"host 192.168.1.1 and port 80"@HomeRouter
# Subprotocols
# Format:
@@ -16,6 +27,7 @@ host:"api-global.netflix.com"@Netflix
# IP based Subprotocols
# Format:
# ip:<value>,ip:<value>,.....@<subproto>
# ipv6:[<value>],ipv6:[<value>],.....@<subproto>
#
# NOTES
@@ -34,11 +46,30 @@ host:"api-global.netflix.com"@Netflix
# detected as <L7 proto>.Google but only
# as <L7 proto>
#
ip:213.75.170.11/32:443@CustomProtocol
ip:8.248.73.247:443@AmazonPrime
ip:54.80.47.130@AmazonPrime
#You can specify a protocol Id. In that case you probably want to avoid conflict with internal ids.
#You can use any number up to 65535
ip:3.3.3.3:443@CustomProtocolA
ip:3.3.3.3:444@CustomProtocolB
ip:3.3.3.3:446@CustomProtocolC=800
ipv6:[3ffe:507:0:1:200:86ff:fe05:80da]@CustomProtocolD=1024
ipv6:[247f:855b:5e16:3caf::]/64:100@CustomProtocolE=2048
ipv6:[247f:855b:5e16:3caf::]/64@CustomProtocolF=2049
ipv6:[fe80::76ac:b9ff:fe6c:c124]:12717@CustomProtocolG=2050
ipv6:[fe80::76ac:b9ff:fe6c:c124]:12718@CustomProtocolH=65535
ipv6:[fe80::76ac:b9ff:fe6c:c124]:12719@CustomProtocolI=65534
#
# You can use symbolic IP addreses if you want
#
ip:www.ntop.org@ntop
ipv6:www.ntop.org@ntop
#
# Risk Exceptions
#
@@ -50,7 +81,10 @@ ip:54.80.47.130@AmazonPrime
# For IPs, the flow risk is put in AND (source IP mask OR destination IP mask)
# For Flows with a hostname (e.g. TLS) the risk is also put in AND with the host_risk_mask
#ip_risk_mask:192.168.1.0/24=0
#ip_risk_mask:10.196.157.228=0
ip_risk_mask:10.10.120.0/24=0
ip_risk_mask:10.196.157.228=0
ipv6_risk_mask:[fe80::356b:e047:3695:0]/112=0
ipv6_risk_mask:[fe80::7c0:e74e:87c3:5d93]=0
host_risk_mask:".home"=0
+602 -362
View File
File diff suppressed because it is too large Load Diff
+124 -32
View File
@@ -90,8 +90,13 @@ extern int dpdk_port_deinit(int port);
#define MAX_TABLE_SIZE_1 4096
#define MAX_TABLE_SIZE_2 8192
#define INIT_VAL -1
#define SERIALIZATION_BUFSIZ (8192 * 2)
#ifdef __cplusplus
extern "C" {
#endif
// inner hash table (ja3 -> security state)
typedef struct ndpi_ja3_info {
char * ja3;
@@ -157,19 +162,31 @@ struct ndpi_entropy {
float score;
};
enum info_type {
INFO_INVALID = 0,
INFO_GENERIC,
INFO_KERBEROS,
INFO_SOFTETHER,
INFO_TIVOCONNECT,
INFO_FTP_IMAP_POP_SMTP,
INFO_NATPMP,
};
// flow tracking
typedef struct ndpi_flow_info {
u_int32_t flow_id;
u_int32_t hashval;
u_int32_t src_ip; /* network order */
u_int32_t dst_ip; /* network order */
struct ndpi_in6_addr src_ip6; /* network order */
struct ndpi_in6_addr dst_ip6; /* network order */
u_int16_t src_port; /* network order */
u_int16_t dst_port; /* network order */
u_int8_t detection_completed, protocol, bidirectional, check_extra_packets;
u_int16_t vlan_id;
ndpi_packet_tunnel tunnel_type;
struct ndpi_flow_struct *ndpi_flow;
char src_name[48], dst_name[48];
char src_name[INET6_ADDRSTRLEN], dst_name[INET6_ADDRSTRLEN];
u_int8_t ip_version;
u_int32_t cwr_count, src2dst_cwr_count, dst2src_cwr_count;
u_int32_t ece_count, src2dst_ece_count, dst2src_ece_count;
@@ -186,31 +203,77 @@ typedef struct ndpi_flow_info {
u_int32_t src2dst_packets, dst2src_packets;
u_int32_t has_human_readeable_strings;
char human_readeable_string_buffer[32];
char *risk_str;
// result only, not used for flow identification
ndpi_protocol detected_protocol;
ndpi_confidence_t confidence;
u_int16_t num_dissector_calls;
u_int16_t dpi_packets;
// Flow data analysis
pkt_timeval src2dst_last_pkt_time, dst2src_last_pkt_time, flow_last_pkt_time;
struct ndpi_analyze_struct *iat_c_to_s, *iat_s_to_c, *iat_flow,
*pktlen_c_to_s, *pktlen_s_to_c;
char info[255];
char flow_extra_info[16];
enum info_type info_type;
union {
char info[256];
struct {
unsigned char auth_failed;
char username[127];
char password[128];
} ftp_imap_pop_smtp;
struct {
char domain[85];
char hostname[85];
char username[86];
} kerberos;
struct {
char ip[16];
char port[6];
char hostname[48];
char fqdn[48];
} softether;
struct {
char identity_uuid[36];
char machine[48];
char platform[32];
char services[48];
} tivoconnect;
struct {
uint16_t result_code;
uint16_t internal_port;
uint16_t external_port;
char ip[16];
} natpmp;
};
ndpi_serializer ndpi_flow_serializer;
char host_server_name[80]; /* Hostname/SNI */
char *bittorent_hash;
char *dhcp_fingerprint;
char *dhcp_class_ident;
ndpi_risk risk;
struct {
char currency[16];
} mining;
struct {
u_int16_t ssl_version;
char server_info[64],
client_hassh[33], server_hassh[33], *server_names,
*tls_alpn, *tls_supported_versions,
*advertised_alpns, *negotiated_alpn, *tls_supported_versions,
*tls_issuerDN, *tls_subjectDN,
ja3_client[33], ja3_server[33],
ja3_client[33], ja3_server[33], ja4_client[37],
sha1_cert_fingerprint[20];
u_int8_t sha1_cert_fingerprint_set;
struct tls_heuristics browser_heuristics;
@@ -220,20 +283,38 @@ typedef struct ndpi_flow_info {
char *esni;
} encrypted_sni;
struct {
u_int16_t version;
} encrypted_ch;
time_t notBefore, notAfter;
u_int16_t server_cipher;
ndpi_cipher_weakness client_unsafe_cipher, server_unsafe_cipher;
u_int32_t quic_version;
} ssh_tls;
struct {
char url[256], request_content_type[64], content_type[64], user_agent[256];
char url[256], request_content_type[64], content_type[64],
user_agent[256], server[128], nat_ip[32], filename[256];
u_int response_status_code;
} http;
struct {
ndpi_address_port mapped_address, peer_address,
relayed_address, response_origin, other_address;
} stun;
struct {
char *username, *password;
} telnet;
struct {
char geolocation_iata_code[4];
} dns;
ndpi_multimedia_flow_type multimedia_flow_type;
void *src_id, *dst_id;
struct ndpi_entropy *entropy;
@@ -245,6 +326,10 @@ typedef struct ndpi_flow_info {
#else
struct ndpi_bin payload_len_bin;
#endif
/* Flow payload */
u_int16_t flow_payload_len;
char *flow_payload;
} ndpi_flow_info_t;
@@ -265,6 +350,11 @@ typedef struct ndpi_stats {
u_int16_t max_packet_len;
u_int64_t dpi_packet_count[3];
u_int64_t flow_confidence[NDPI_CONFIDENCE_MAX];
u_int64_t num_dissector_calls;
struct ndpi_lru_cache_stats lru_stats[NDPI_LRUCACHE_MAX];
struct ndpi_automa_stats automa_stats[NDPI_AUTOMA_MAX];
struct ndpi_patricia_tree_stats patricia_stats[NDPI_PTREE_MAX];
} ndpi_stats_t;
@@ -290,10 +380,8 @@ typedef struct ndpi_workflow {
struct ndpi_workflow_prefs prefs;
struct ndpi_stats stats;
ndpi_workflow_callback_ptr __flow_detected_callback;
void * __flow_detected_udata;
ndpi_workflow_callback_ptr __flow_giveup_callback;
void * __flow_giveup_udata;
ndpi_workflow_callback_ptr flow_callback;
void * flow_callback_userdata;
/* outside referencies */
pcap_t *pcap_handle;
@@ -301,12 +389,16 @@ typedef struct ndpi_workflow {
/* allocated by prefs */
void **ndpi_flows_root;
struct ndpi_detection_module_struct *ndpi_struct;
struct ndpi_global_context *g_ctx;
u_int32_t num_allocated_flows;
/* CSV,TLV,JSON serialization interface */
ndpi_serialization_format ndpi_serialization_format;
} ndpi_workflow_t;
/* TODO: remove wrappers parameters and use ndpi global, when their initialization will be fixed... */
struct ndpi_workflow * ndpi_workflow_init(const struct ndpi_workflow_prefs * prefs, pcap_t * pcap_handle);
struct ndpi_workflow * ndpi_workflow_init(const struct ndpi_workflow_prefs * prefs, pcap_t * pcap_handle, int do_init_flows_root, ndpi_serialization_format serialization_format, struct ndpi_global_context *g_ctx);
/* workflow main free function */
@@ -325,43 +417,43 @@ struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
const struct pcap_pkthdr *header,
const u_char *packet,
ndpi_risk *flow_risk,
FILE * csv_fp);
struct ndpi_flow_info **flow);
/* Flow callback for completed flows, before the flow memory will be freed. */
static inline void ndpi_workflow_set_flow_callback(struct ndpi_workflow * workflow, ndpi_workflow_callback_ptr callback, void * userdata) {
workflow->flow_callback = callback;
workflow->flow_callback_userdata = userdata;
}
int ndpi_is_datalink_supported(int datalink_type);
/* flow callbacks for complete detected flow
(ndpi_flow_info will be freed right after) */
static inline void ndpi_workflow_set_flow_detected_callback(struct ndpi_workflow * workflow, ndpi_workflow_callback_ptr callback, void * udata) {
workflow->__flow_detected_callback = callback;
workflow->__flow_detected_udata = udata;
}
/* flow callbacks for sufficient detected flow
(ndpi_flow_info will be freed right after) */
static inline void ndpi_workflow_set_flow_giveup_callback(struct ndpi_workflow * workflow, ndpi_workflow_callback_ptr callback, void * udata) {
workflow->__flow_giveup_callback = callback;
workflow->__flow_giveup_udata = udata;
}
/* compare two nodes in workflow */
int ndpi_workflow_node_cmp(const void *a, const void *b);
void process_ndpi_collected_info(struct ndpi_workflow * workflow, struct ndpi_flow_info *flow, FILE * csv_fp);
u_int32_t ethernet_crc32(const void* data, size_t n_bytes);
void process_ndpi_collected_info(struct ndpi_workflow * workflow, struct ndpi_flow_info *flow);
void ndpi_flow_info_free_data(struct ndpi_flow_info *flow);
void ndpi_flow_info_freer(void *node);
const char* print_cipher_id(u_int32_t cipher);
float ndpi_flow_get_byte_count_entropy(const uint32_t byte_count[256], unsigned int num_bytes);
int parse_proto_name_list(char *str, NDPI_PROTOCOL_BITMASK *bitmask, int inverted_logic);
extern int nDPI_LogLevel;
extern int reader_log_level;
#ifdef NDPI_ENABLE_DEBUG_MESSAGES
#if defined(NDPI_ENABLE_DEBUG_MESSAGES) && !defined(FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION)
#define LOG(log_level, args...) \
{ \
if(log_level <= nDPI_LogLevel) \
if(log_level <= reader_log_level) \
printf(args); \
}
#else
#define LOG(...) {}
#endif
#ifndef LINKTYPE_LINUX_SLL2
#define LINKTYPE_LINUX_SLL2 276
#endif
#ifdef __cplusplus
}
#endif
#endif
File diff suppressed because it is too large Load Diff
+873 -32
View File
@@ -1,52 +1,893 @@
bin_PROGRAMS = fuzz_process_packet fuzz_process_packet_with_main fuzz_ndpi_reader fuzz_ndpi_reader_with_main
bin_PROGRAMS = fuzz_process_packet fuzz_ndpi_reader fuzz_ndpi_reader_alloc_fail fuzz_ndpi_reader_payload_analyzer fuzz_quic_get_crypto_data fuzz_config fuzz_community_id fuzz_serialization fuzz_tls_certificate fuzz_dga fuzz_is_stun_udp fuzz_is_stun_tcp
#Alghoritms
bin_PROGRAMS += fuzz_alg_bins fuzz_alg_hll fuzz_alg_hw_rsi_outliers_da fuzz_alg_jitter fuzz_alg_ses_des fuzz_alg_crc32_md5 fuzz_alg_bytestream fuzz_alg_shoco
#Data structures
bin_PROGRAMS += fuzz_ds_patricia fuzz_ds_ahocorasick fuzz_ds_libcache fuzz_ds_tree fuzz_ds_ptree fuzz_ds_hash fuzz_ds_cmsketch fuzz_ds_bitmap64_fuse fuzz_ds_domain_classify
#Third party
bin_PROGRAMS += fuzz_libinjection fuzz_binaryfusefilter
#Internal crypto
bin_PROGRAMS += fuzz_gcrypt_light fuzz_gcrypt_aes fuzz_gcrypt_gcm fuzz_gcrypt_cipher
#Configuration files
bin_PROGRAMS += fuzz_filecfg_protocols fuzz_filecfg_categories fuzz_filecfg_malicious_sha1 fuzz_filecfg_malicious_ja3 fuzz_filecfg_risk_domains fuzz_filecfg_config fuzz_filecfg_category
#Reader utils
bin_PROGRAMS += fuzz_readerutils_workflow fuzz_readerutils_parseprotolist
#Mutators
bin_PROGRAMS += fuzz_ndpi_reader_pl7m fuzz_ndpi_reader_pl7m_64k fuzz_ndpi_reader_pl7m_simplest fuzz_ndpi_reader_pl7m_internal fuzz_ndpi_reader_pl7m_simplest_internal
fuzz_process_packet_SOURCES = fuzz_process_packet.c
fuzz_process_packet_CFLAGS =
fuzz_process_packet_LDADD = ../src/lib/libndpi.a
fuzz_process_packet_LDFLAGS = $(ADDITIONAL_LIBS) $(LIBS)
fuzz_process_packet_SOURCES = fuzz_process_packet.c fuzz_common_code.c
fuzz_process_packet_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_process_packet_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_process_packet_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_process_packet_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_process_packet_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_process_packet_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_process_packet_LDFLAGS) $(LDFLAGS) -o $@
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_process_packet_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_process_packet_with_main_SOURCES = fuzz_process_packet.c
fuzz_process_packet_with_main_CFLAGS = -DBUILD_MAIN
fuzz_process_packet_with_main_LDADD = ../src/lib/libndpi.a
fuzz_process_packet_with_main_LDFLAGS = $(ADDITIONAL_LIBS) $(LIBS)
# force usage of CXX for linker
fuzz_process_packet_with_main_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_process_packet_with_main_LDFLAGS) $(LDFLAGS) -o $@
fuzz_ndpi_reader_SOURCES = fuzz_ndpi_reader.c
fuzz_ndpi_reader_CFLAGS = -I../example/
fuzz_ndpi_reader_LDADD = ../example/libndpiReader.a ../src/lib/libndpi.a
fuzz_ndpi_reader_LDFLAGS = $(PCAP_LIB) $(ADDITIONAL_LIBS) $(LIBS)
fuzz_ndpi_reader_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c
fuzz_ndpi_reader_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ndpi_reader_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_LDFLAGS) $(LDFLAGS) -o $@
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_with_main_SOURCES = fuzz_ndpi_reader.c
fuzz_ndpi_reader_with_main_CFLAGS = -I../example/ -DBUILD_MAIN
fuzz_ndpi_reader_with_main_LDADD = ../src/lib/libndpi.a
fuzz_ndpi_reader_with_main_LDFLAGS = ../example/libndpiReader.a $(PCAP_LIB) $(ADDITIONAL_LIBS) $(LIBS)
fuzz_ndpi_reader_alloc_fail_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c
fuzz_ndpi_reader_alloc_fail_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -DENABLE_MEM_ALLOC_FAILURES -DCRYPT_FORCE_NO_AESNI
fuzz_ndpi_reader_alloc_fail_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_alloc_fail_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_alloc_fail_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_alloc_fail_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_with_main_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_with_main_LDFLAGS) $(LDFLAGS) -o $@
fuzz_ndpi_reader_alloc_fail_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_alloc_fail_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_payload_analyzer_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c
fuzz_ndpi_reader_payload_analyzer_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -DENABLE_MEM_ALLOC_FAILURES -DENABLE_PAYLOAD_ANALYZER
fuzz_ndpi_reader_payload_analyzer_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_payload_analyzer_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_payload_analyzer_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_payload_analyzer_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_payload_analyzer_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_payload_analyzer_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_quic_get_crypto_data_SOURCES = fuzz_quic_get_crypto_data.c fuzz_common_code.c
fuzz_quic_get_crypto_data_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_quic_get_crypto_data_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_quic_get_crypto_data_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_quic_get_crypto_data_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_quic_get_crypto_data_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_quic_get_crypto_data_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_quic_get_crypto_data_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_config_SOURCES = fuzz_config.cpp fuzz_common_code.c
fuzz_config_CXXFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_config_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_config_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_config_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_config_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_config_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_config_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_config_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_config_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_community_id_SOURCES = fuzz_community_id.cpp fuzz_common_code.c
fuzz_community_id_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_community_id_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_community_id_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_community_id_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_community_id_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_community_id_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_community_id_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_community_id_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_community_id_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_serialization_SOURCES = fuzz_serialization.cpp fuzz_common_code.c
fuzz_serialization_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_serialization_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_serialization_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_serialization_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_serialization_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_serialization_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_serialization_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_serialization_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_serialization_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_bins_SOURCES = fuzz_alg_bins.cpp fuzz_common_code.c
fuzz_alg_bins_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_bins_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_bins_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_bins_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_bins_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_bins_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_bins_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_bins_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_bins_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_hll_SOURCES = fuzz_alg_hll.cpp fuzz_common_code.c
fuzz_alg_hll_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_hll_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_hll_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_hll_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_hll_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_hll_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_hll_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_hll_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_hll_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_hw_rsi_outliers_da_SOURCES = fuzz_alg_hw_rsi_outliers_da.cpp fuzz_common_code.c
fuzz_alg_hw_rsi_outliers_da_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_hw_rsi_outliers_da_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_hw_rsi_outliers_da_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_hw_rsi_outliers_da_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_hw_rsi_outliers_da_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_hw_rsi_outliers_da_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_hw_rsi_outliers_da_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_hw_rsi_outliers_da_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_hw_rsi_outliers_da_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_jitter_SOURCES = fuzz_alg_jitter.cpp fuzz_common_code.c
fuzz_alg_jitter_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_jitter_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_jitter_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_jitter_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_jitter_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_jitter_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_jitter_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_jitter_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_jitter_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_crc32_md5_SOURCES = fuzz_alg_crc32_md5.c
fuzz_alg_crc32_md5_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_alg_crc32_md5_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_crc32_md5_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_crc32_md5_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_crc32_md5_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_crc32_md5_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_crc32_md5_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_bytestream_SOURCES = fuzz_alg_bytestream.c
fuzz_alg_bytestream_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_bytestream_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_bytestream_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_bytestream_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_bytestream_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_bytestream_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_bytestream_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_shoco_SOURCES = fuzz_alg_shoco.cpp
fuzz_alg_shoco_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_alg_shoco_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_shoco_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_shoco_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_shoco_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_shoco_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_shoco_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_ses_des_SOURCES = fuzz_alg_ses_des.cpp fuzz_common_code.c
fuzz_alg_ses_des_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_ses_des_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_ses_des_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_ses_des_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_ses_des_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_ses_des_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_ses_des_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_ses_des_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_ses_des_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_patricia_SOURCES = fuzz_ds_patricia.cpp fuzz_common_code.c
fuzz_ds_patricia_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_patricia_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_patricia_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_patricia_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_patricia_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_patricia_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_patricia_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_patricia_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_patricia_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_ahocorasick_SOURCES = fuzz_ds_ahocorasick.cpp fuzz_common_code.c
fuzz_ds_ahocorasick_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_ahocorasick_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_ahocorasick_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_ahocorasick_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_ahocorasick_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_ahocorasick_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_ahocorasick_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_ahocorasick_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_ahocorasick_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_libcache_SOURCES = fuzz_ds_libcache.cpp fuzz_common_code.c
fuzz_ds_libcache_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_libcache_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_libcache_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_libcache_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_libcache_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_libcache_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_libcache_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_libcache_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_libcache_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_tree_SOURCES = fuzz_ds_tree.cpp fuzz_common_code.c
fuzz_ds_tree_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_tree_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_tree_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_tree_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_tree_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_tree_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_tree_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_tree_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_tree_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_ptree_SOURCES = fuzz_ds_ptree.cpp fuzz_common_code.c
fuzz_ds_ptree_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_ptree_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_ptree_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_ptree_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_ptree_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_ptree_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_ptree_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_ptree_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_ptree_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_hash_SOURCES = fuzz_ds_hash.cpp fuzz_common_code.c
fuzz_ds_hash_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_hash_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_hash_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_hash_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_hash_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_hash_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_hash_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_hash_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_hash_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_cmsketch_SOURCES = fuzz_ds_cmsketch.cpp fuzz_common_code.c
fuzz_ds_cmsketch_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_cmsketch_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_cmsketch_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_cmsketch_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_cmsketch_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_cmsketch_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_cmsketch_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_cmsketch_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_cmsketch_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_bitmap64_fuse_SOURCES = fuzz_ds_bitmap64_fuse.cpp fuzz_common_code.c
fuzz_ds_bitmap64_fuse_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_bitmap64_fuse_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_bitmap64_fuse_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_bitmap64_fuse_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_bitmap64_fuse_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_bitmap64_fuse_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_bitmap64_fuse_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_bitmap64_fuse_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_bitmap64_fuse_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_domain_classify_SOURCES = fuzz_ds_domain_classify.cpp fuzz_common_code.c
fuzz_ds_domain_classify_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_domain_classify_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_domain_classify_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_domain_classify_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_domain_classify_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_domain_classify_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_domain_classify_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_domain_classify_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_domain_classify_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_libinjection_SOURCES = fuzz_libinjection.c
fuzz_libinjection_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_libinjection_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_libinjection_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_libinjection_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_libinjection_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_libinjection_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_libinjection_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_binaryfusefilter_SOURCES = fuzz_binaryfusefilter.cpp fuzz_common_code.c
fuzz_binaryfusefilter_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_binaryfusefilter_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_binaryfusefilter_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_binaryfusefilter_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_binaryfusefilter_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_binaryfusefilter_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_binaryfusefilter_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_binaryfusefilter_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_binaryfusefilter_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_tls_certificate_SOURCES = fuzz_tls_certificate.c fuzz_common_code.c
fuzz_tls_certificate_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_tls_certificate_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_tls_certificate_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_tls_certificate_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_tls_certificate_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_tls_certificate_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_tls_certificate_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_dga_SOURCES = fuzz_dga.c fuzz_common_code.c
fuzz_dga_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_dga_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_dga_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_dga_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_dga_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_dga_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_dga_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_is_stun_udp_SOURCES = fuzz_is_stun.c fuzz_common_code.c
fuzz_is_stun_udp_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_is_stun_udp_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_is_stun_udp_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_is_stun_udp_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_is_stun_udp_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_is_stun_udp_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_is_stun_udp_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_is_stun_tcp_SOURCES = fuzz_is_stun.c fuzz_common_code.c
fuzz_is_stun_tcp_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION -DSTUN_TCP
fuzz_is_stun_tcp_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_is_stun_tcp_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_is_stun_tcp_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_is_stun_tcp_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_is_stun_tcp_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_is_stun_tcp_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_gcrypt_light_SOURCES = fuzz_gcrypt_light.cpp fuzz_common_code.c
fuzz_gcrypt_light_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_light_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_light_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_gcrypt_light_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_gcrypt_light_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_light_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_light_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_gcrypt_light_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_gcrypt_light_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_gcrypt_aes_SOURCES = fuzz_gcrypt_aes.cpp
fuzz_gcrypt_aes_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_aes_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_aes_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_gcrypt_aes_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_gcrypt_aes_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_aes_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_aes_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_gcrypt_aes_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_gcrypt_aes_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_gcrypt_gcm_SOURCES = fuzz_gcrypt_gcm.cpp
fuzz_gcrypt_gcm_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_gcm_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_gcm_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_gcrypt_gcm_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_gcrypt_gcm_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_gcm_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_gcm_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_gcrypt_gcm_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_gcrypt_gcm_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_gcrypt_cipher_SOURCES = fuzz_gcrypt_cipher.cpp
fuzz_gcrypt_cipher_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_cipher_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_cipher_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_gcrypt_cipher_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_gcrypt_cipher_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_cipher_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_cipher_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_gcrypt_cipher_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_gcrypt_cipher_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_protocols_SOURCES = fuzz_filecfg_protocols.c fuzz_common_code.c
fuzz_filecfg_protocols_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_protocols_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_protocols_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_protocols_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_protocols_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_protocols_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_protocols_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_categories_SOURCES = fuzz_filecfg_categories.c fuzz_common_code.c
fuzz_filecfg_categories_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_categories_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_categories_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_categories_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_categories_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_categories_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_categories_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_malicious_sha1_SOURCES = fuzz_filecfg_malicious_sha1.c fuzz_common_code.c
fuzz_filecfg_malicious_sha1_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_malicious_sha1_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_malicious_sha1_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_malicious_sha1_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_malicious_sha1_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_malicious_sha1_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_malicious_sha1_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_malicious_ja3_SOURCES = fuzz_filecfg_malicious_ja3.c fuzz_common_code.c
fuzz_filecfg_malicious_ja3_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_malicious_ja3_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_malicious_ja3_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_malicious_ja3_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_malicious_ja3_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_malicious_ja3_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_malicious_ja3_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_risk_domains_SOURCES = fuzz_filecfg_risk_domains.c fuzz_common_code.c
fuzz_filecfg_risk_domains_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_risk_domains_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_risk_domains_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_risk_domains_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_risk_domains_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_risk_domains_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_risk_domains_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_config_SOURCES = fuzz_filecfg_config.c fuzz_common_code.c
fuzz_filecfg_config_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_config_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_config_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_config_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_config_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_config_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_config_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_category_SOURCES = fuzz_filecfg_category.c fuzz_common_code.c
fuzz_filecfg_category_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_category_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_category_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_category_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_category_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_category_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_category_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_readerutils_workflow_SOURCES = fuzz_readerutils_workflow.cpp fuzz_common_code.c ../example/reader_util.c
fuzz_readerutils_workflow_CXXFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_readerutils_workflow_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_readerutils_workflow_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_readerutils_workflow_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_readerutils_workflow_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_readerutils_workflow_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_readerutils_workflow_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_readerutils_workflow_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_readerutils_workflow_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_readerutils_parseprotolist_SOURCES = fuzz_readerutils_parseprotolist.cpp fuzz_common_code.c ../example/reader_util.c
fuzz_readerutils_parseprotolist_CXXFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_readerutils_parseprotolist_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_readerutils_parseprotolist_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_readerutils_parseprotolist_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_readerutils_parseprotolist_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_readerutils_parseprotolist_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_readerutils_parseprotolist_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_readerutils_parseprotolist_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_readerutils_parseprotolist_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_pl7m_simplest_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c ../src/lib/third_party/src/fuzz/pl7m.c
fuzz_ndpi_reader_pl7m_simplest_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -I../src/lib/third_party/include/ -DENABLE_PCAP_L7_MUTATOR -DPL7M_USE_SIMPLEST_MUTATOR
fuzz_ndpi_reader_pl7m_simplest_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_pl7m_simplest_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_pl7m_simplest_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_pl7m_simplest_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_pl7m_simplest_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_pl7m_simplest_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_pl7m_simplest_internal_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c ../src/lib/third_party/src/fuzz/pl7m.c
fuzz_ndpi_reader_pl7m_simplest_internal_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -I../src/lib/third_party/include/ -DENABLE_PCAP_L7_MUTATOR -DPL7M_USE_SIMPLEST_MUTATOR -DPL7M_USE_INTERNAL_FUZZER_MUTATE
fuzz_ndpi_reader_pl7m_simplest_internal_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_pl7m_simplest_internal_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_pl7m_simplest_internal_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_pl7m_simplest_internal_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_pl7m_simplest_internal_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_pl7m_simplest_internal_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_pl7m_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c ../src/lib/third_party/src/fuzz/pl7m.c
fuzz_ndpi_reader_pl7m_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -I../src/lib/third_party/include/ -DENABLE_PCAP_L7_MUTATOR
fuzz_ndpi_reader_pl7m_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_pl7m_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_pl7m_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_pl7m_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_pl7m_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_pl7m_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_pl7m_64k_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c ../src/lib/third_party/src/fuzz/pl7m.c
fuzz_ndpi_reader_pl7m_64k_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -I../src/lib/third_party/include/ -DENABLE_PCAP_L7_MUTATOR -DPL7M_USE_64K_PACKETS
fuzz_ndpi_reader_pl7m_64k_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_pl7m_64k_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_pl7m_64k_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_pl7m_64k_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_pl7m_64k_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_pl7m_64k_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_pl7m_internal_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c ../src/lib/third_party/src/fuzz/pl7m.c
fuzz_ndpi_reader_pl7m_internal_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -I../src/lib/third_party/include/ -DENABLE_PCAP_L7_MUTATOR -DPL7M_USE_INTERNAL_FUZZER_MUTATE
fuzz_ndpi_reader_pl7m_internal_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_pl7m_internal_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_pl7m_internal_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_pl7m_internal_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_pl7m_internal_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_pl7m_internal_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
# required for Google oss-fuzz
# see https://github.com/google/oss-fuzz/tree/master/projects/ndpi
testpcaps := $(wildcard ../tests/pcap/*.pcap*)
testpcaps := $(wildcard ../tests/cfgs/default/pcap/*.pcap*)
fuzz_ndpi_reader_seed_corpus.zip: $(testpcaps)
zip -r fuzz_ndpi_reader_seed_corpus.zip $(testpcaps)
testpcaps.zip: $(testpcaps)
zip -j testpcaps.zip $(testpcaps)
fuzz_ndpi_reader_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_seed_corpus.zip
fuzz_ndpi_reader_alloc_fail_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_alloc_fail_seed_corpus.zip
fuzz_ndpi_reader_payload_analyzer_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_payload_analyzer_seed_corpus.zip
fuzz_ndpi_reader_pl7m_simplest_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_pl7m_simplest_seed_corpus.zip
fuzz_ndpi_reader_pl7m_simplest_internal_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_pl7m_simplest_internal_seed_corpus.zip
fuzz_ndpi_reader_pl7m_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_pl7m_seed_corpus.zip
fuzz_ndpi_reader_pl7m_64k_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_pl7m_64k_seed_corpus.zip
fuzz_ndpi_reader_pl7m_internal_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_pl7m_internal_seed_corpus.zip
files_corpus_fuzz_quic_get_crypto_data := $(wildcard corpus/fuzz_quic_get_crypto_data/*)
fuzz_quic_get_crypto_data_seed_corpus.zip: $(files_corpus_fuzz_quic_get_crypto_data)
zip -j fuzz_quic_get_crypto_data_seed_corpus.zip $(files_corpus_fuzz_quic_get_crypto_data)
files_corpus_fuzz_serialization := $(wildcard corpus/fuzz_serialization/*)
fuzz_serialization_seed_corpus.zip: $(files_corpus_fuzz_serialization)
zip -j fuzz_serialization_seed_corpus.zip $(files_corpus_fuzz_serialization)
files_corpus_fuzz_community_id := $(wildcard corpus/fuzz_community_id/*)
fuzz_community_id_seed_corpus.zip: $(files_corpus_fuzz_community_id)
zip -j fuzz_community_id_seed_corpus.zip $(files_corpus_fuzz_community_id)
files_corpus_fuzz_is_stun_udp := $(wildcard corpus/fuzz_is_stun_udp/*)
fuzz_is_stun_udp_seed_corpus.zip: $(files_corpus_fuzz_is_stun_udp)
zip -j fuzz_is_stun_udp_seed_corpus.zip $(files_corpus_fuzz_is_stun_udp)
files_corpus_fuzz_is_stun_tcp := $(wildcard corpus/fuzz_is_stun_tcp/*)
fuzz_is_stun_tcp_seed_corpus.zip: $(files_corpus_fuzz_is_stun_tcp)
zip -j fuzz_is_stun_tcp_seed_corpus.zip $(files_corpus_fuzz_is_stun_tcp)
files_corpus_fuzz_alg_ses_des := $(wildcard corpus/fuzz_alg_ses_des/*)
fuzz_alg_ses_des_seed_corpus.zip: $(files_corpus_fuzz_alg_ses_des)
zip -j fuzz_alg_ses_des_seed_corpus.zip $(files_corpus_fuzz_alg_ses_des)
files_corpus_fuzz_alg_bins := $(wildcard corpus/fuzz_alg_bins/*)
fuzz_alg_bins_seed_corpus.zip: $(files_corpus_fuzz_alg_bins)
zip -j fuzz_alg_bins_seed_corpus.zip $(files_corpus_fuzz_alg_bins)
files_corpus_fuzz_alg_hll := $(wildcard corpus/fuzz_alg_hll/*)
fuzz_alg_hll_seed_corpus.zip: $(files_corpus_fuzz_alg_hll)
zip -j fuzz_alg_hll_seed_corpus.zip $(files_corpus_fuzz_alg_hll)
files_corpus_fuzz_alg_jitter := $(wildcard corpus/fuzz_alg_jitter/*)
fuzz_alg_jitter_seed_corpus.zip: $(files_corpus_fuzz_alg_jitter)
zip -j fuzz_alg_jitter_seed_corpus.zip $(files_corpus_fuzz_alg_jitter)
files_corpus_fuzz_alg_crc32_md5 := $(wildcard corpus/fuzz_alg_crc32_md5/*)
fuzz_alg_crc32_md5_seed_corpus.zip: $(files_corpus_fuzz_alg_crc32_md5)
zip -j fuzz_alg_crc32_md5_seed_corpus.zip $(files_corpus_fuzz_alg_crc32_md5)
files_corpus_fuzz_ds_libcache := $(wildcard corpus/fuzz_ds_libcache/*)
fuzz_ds_libcache_seed_corpus.zip: $(files_corpus_fuzz_ds_libcache)
zip -j fuzz_ds_libcache_seed_corpus.zip $(files_corpus_fuzz_ds_libcache)
files_corpus_fuzz_ds_ptree := $(wildcard corpus/fuzz_ds_ptree/*)
fuzz_ds_ptree_seed_corpus.zip: $(files_corpus_fuzz_ds_ptree)
zip -j fuzz_ds_ptree_seed_corpus.zip $(files_corpus_fuzz_ds_ptree)
files_corpus_fuzz_alg_bytestream := $(wildcard corpus/fuzz_alg_bytestream/*)
fuzz_alg_bytestream_seed_corpus.zip: $(files_corpus_fuzz_alg_bytestream)
zip -j fuzz_alg_bytestream_seed_corpus.zip $(files_corpus_fuzz_alg_bytestream)
files_corpus_fuzz_libinjection := $(wildcard corpus/fuzz_libinjection/*)
fuzz_libinjection_seed_corpus.zip: $(files_corpus_fuzz_libinjection)
zip -j fuzz_libinjection_seed_corpus.zip $(files_corpus_fuzz_libinjection)
files_corpus_fuzz_tls_certificate := $(wildcard corpus/fuzz_tls_certificate/*)
fuzz_tls_certificate_seed_corpus.zip: $(files_corpus_fuzz_tls_certificate)
zip -j fuzz_tls_certificate_seed_corpus.zip $(files_corpus_fuzz_tls_certificate)
files_corpus_fuzz_filecfg_protocols := $(wildcard corpus/fuzz_filecfg_protocols/*)
fuzz_filecfg_protocols_seed_corpus.zip: $(files_corpus_fuzz_filecfg_protocols)
zip -j fuzz_filecfg_protocols_seed_corpus.zip $(files_corpus_fuzz_filecfg_protocols)
files_corpus_fuzz_filecfg_categories := $(wildcard corpus/fuzz_filecfg_categories/*)
fuzz_filecfg_categories_seed_corpus.zip: $(files_corpus_fuzz_filecfg_categories)
zip -j fuzz_filecfg_categories_seed_corpus.zip $(files_corpus_fuzz_filecfg_categories)
files_corpus_fuzz_filecfg_malicious_sha1 := $(wildcard corpus/fuzz_filecfg_malicious_sha1/*)
fuzz_filecfg_malicious_sha1_seed_corpus.zip: $(files_corpus_fuzz_filecfg_malicious_sha1)
zip -j fuzz_filecfg_malicious_sha1_seed_corpus.zip $(files_corpus_fuzz_filecfg_malicious_sha1)
files_corpus_fuzz_filecfg_malicious_ja3 := $(wildcard corpus/fuzz_filecfg_malicious_ja3/*)
fuzz_filecfg_malicious_ja3_seed_corpus.zip: $(files_corpus_fuzz_filecfg_malicious_ja3)
zip -j fuzz_filecfg_malicious_ja3_seed_corpus.zip $(files_corpus_fuzz_filecfg_malicious_ja3)
files_corpus_fuzz_filecfg_risk_domains := $(wildcard corpus/fuzz_filecfg_risk_domains/*)
fuzz_filecfg_risk_domains_seed_corpus.zip: $(files_corpus_fuzz_filecfg_risk_domains)
zip -j fuzz_filecfg_risk_domains_seed_corpus.zip $(files_corpus_fuzz_filecfg_risk_domains)
files_corpus_fuzz_filecfg_config := $(wildcard corpus/fuzz_filecfg_config/*)
fuzz_filecfg_config_seed_corpus.zip: $(files_corpus_fuzz_filecfg_config)
zip -j fuzz_filecfg_config_seed_corpus.zip $(files_corpus_fuzz_filecfg_config)
files_corpus_fuzz_filecfg_category := $(wildcard corpus/fuzz_filecfg_category/*)
fuzz_filecfg_category_seed_corpus.zip: $(files_corpus_fuzz_filecfg_category)
zip -j fuzz_filecfg_category_seed_corpus.zip $(files_corpus_fuzz_filecfg_category)
files_corpus_fuzz_readerutils_workflow := $(wildcard corpus/fuzz_readerutils_workflow/*)
fuzz_readerutils_workflow_seed_corpus.zip: $(files_corpus_fuzz_readerutils_workflow)
zip -j fuzz_readerutils_workflow_seed_corpus.zip $(files_corpus_fuzz_readerutils_workflow)
files_corpus_fuzz_readerutils_parseprotolist := $(wildcard corpus/fuzz_readerutils_parseprotolist/*)
fuzz_readerutils_parseprotolist_seed_corpus.zip: $(files_corpus_fuzz_readerutils_parseprotolist)
zip -j fuzz_readerutils_parseprotolist_seed_corpus.zip $(files_corpus_fuzz_readerutils_parseprotolist)
files_corpus_fuzz_ds_bitmap64_fuse := $(wildcard corpus/fuzz_ds_bitmap64_fuse/*)
fuzz_ds_bitmap64_fuse_seed_corpus.zip: $(files_corpus_fuzz_ds_bitmap64_fuse)
zip -j fuzz_ds_bitmap64_fuse_seed_corpus.zip $(files_corpus_fuzz_ds_bitmap64_fuse)
files_corpus_fuzz_ds_domain_classify := $(wildcard corpus/fuzz_ds_domain_classify/*)
fuzz_ds_domain_classify_seed_corpus.zip: $(files_corpus_fuzz_ds_domain_classify)
zip -j fuzz_ds_domain_classify_seed_corpus.zip $(files_corpus_fuzz_ds_domain_classify)
corpus: fuzz_ndpi_reader_seed_corpus.zip fuzz_ndpi_reader_alloc_fail_seed_corpus.zip fuzz_ndpi_reader_payload_analyzer_seed_corpus.zip fuzz_quic_get_crypto_data_seed_corpus.zip fuzz_alg_ses_des_seed_corpus.zip fuzz_alg_bins_seed_corpus.zip fuzz_alg_hll_seed_corpus.zip fuzz_alg_jitter_seed_corpus.zip fuzz_ds_libcache_seed_corpus.zip fuzz_community_id_seed_corpus.zip fuzz_serialization_seed_corpus.zip fuzz_ds_ptree_seed_corpus.zip fuzz_alg_crc32_md5_seed_corpus.zip fuzz_alg_bytestream_seed_corpus.zip fuzz_libinjection_seed_corpus.zip fuzz_tls_certificate_seed_corpus.zip fuzz_filecfg_protocols_seed_corpus.zip fuzz_readerutils_workflow_seed_corpus.zip fuzz_readerutils_parseprotolist_seed_corpus.zip fuzz_ds_bitmap64_fuse_seed_corpus.zip fuzz_ds_domain_classify_seed_corpus.zip fuzz_filecfg_protocols_seed_corpus.zip fuzz_is_stun_udp_seed_corpus.zip fuzz_is_stun_tcp_seed_corpus.zip fuzz_ndpi_reader_pl7m_simplest_seed_corpus.zip fuzz_ndpi_reader_pl7m_seed_corpus.zip fuzz_ndpi_reader_pl7m_64k_seed_corpus.zip fuzz_ndpi_reader_pl7m_simplest_internal_seed_corpus.zip fuzz_ndpi_reader_pl7m_internal_seed_corpus.zip
cp corpus/fuzz_*seed_corpus.zip .
#Create dictionaries exactly as expected by oss-fuzz.
#This way, if we need to change/update/add something,
#we don't need to update scripts in oss-fuzz repository
dictionaries:
cp dictionary.dict fuzz_ndpi_reader.dict
cp dictionary.dict fuzz_ndpi_reader_alloc_fail.dict
cp dictionary.dict fuzz_ndpi_reader_payload_analyzer.dict
cp dictionary.dict fuzz_process_packet.dict
cp dictionary_tls_certificate.dict fuzz_tls_certificate.dict
distdir:
find . -type d | xargs -I'{}' mkdir -p '$(distdir)/{}'
find . -type f -name '*.c' \
-o -name '*.am' \
-o -name '*.h' \
-o -name '*.cpp' \
-o -name '*.options' \
-o -name 'ipv4_addresses.txt' \
-o -name 'ipv6_addresses.txt' \
-o -name 'bd_param.txt' \
-o -name 'splt_param.txt' \
-o -name 'random_list.list' \
-o -path './dictionary.dict' \
-o -path './dictionary_tls_certificate.dict' \
-o -path './corpus/fuzz_*.zip' \
-o -path './corpus/fuzz_quic_get_crypto_data/*' \
-o -path './corpus/fuzz_filecfg_protocols/*' \
-o -path './corpus/fuzz_filecfg_categories/*' \
-o -path './corpus/fuzz_filecfg_malicious_sha1/*' \
-o -path './corpus/fuzz_filecfg_malicious_ja3/*' \
-o -path './corpus/fuzz_filecfg_risk_domains/*' \
-o -path './corpus/fuzz_filecfg_config/*' \
-o -path './corpus/fuzz_filecfg_category/*' \
-o -path './corpus/fuzz_readerutils_workflow/*' \
-o -path './corpus/fuzz_readerutils_parseprotolist/*' \
-o -path './corpus/fuzz_is_stun_udp/*' \
-o -path './corpus/fuzz_is_stun_tcp/*' \
-o -path './corpus/fuzz_serialization/*' \
-o -path './corpus/fuzz_community_id/*' \
-o -path './corpus/fuzz_libinjection/*' \
-o -path './corpus/fuzz_tls_certificate/*' \
-o -path './corpus/fuzz_alg_ses_des/*' \
-o -path './corpus/fuzz_alg_bins/*' \
-o -path './corpus/fuzz_alg_hll/*' \
-o -path './corpus/fuzz_alg_jitter/*' \
-o -path './corpus/fuzz_alg_crc32_md5/*' \
-o -path './corpus/fuzz_alg_bytestream/*' \
-o -path './corpus/fuzz_ds_libcache/*' \
-o -path './corpus/fuzz_ds_bitmap64_fuse/*' \
-o -path './corpus/fuzz_ds_domain_classify/*' \
-o -path './corpus/fuzz_ds_ptree/*' | xargs -I'{}' cp -r '{}' '$(distdir)/{}'
all: corpus dictionaries
+627
View File
@@ -0,0 +1,627 @@
-2.088057846500587456e+00
7.763936238952200239e-05
4.404309737393306595e-05
-9.467385027293546973e-02
4.348947142638090457e-01
-2.091409170053043390e-04
-5.788902107267982974e-04
4.481443450852441001e-10
-3.136135459023654537e+00
-1.507730262127600751e+00
-1.204663669965535977e+00
-1.171839254318371104e+00
4.329302247232582057e-01
8.310653628092458334e+00
3.299246725156660176e+00
0.000000000000000000e+00
1.847454931582027254e-02
-1.498024139966201096e+00
-7.660670007653060942e-01
-2.908130300830076731e+00
-1.252564844610269734e+00
-1.910955328742287573e+00
9.471710980110392697e-01
2.352302758516665371e+00
2.982269972214651954e+00
4.280736383314343918e+00
4.633629909719495288e+00
-2.198052637823726840e+00
-1.150759637168392580e+00
3.420433363184381292e+00
1.857878113059351077e-02
-1.559806674919653746e+00
4.197498183183401288e+00
6.262186949633183453e+00
1.100694844370524095e+01
2.778688785515088000e+01
3.679948298336883195e+00
-2.432801394376875592e+00
5.133442052706843617e-01
2.181172654073517680e+00
-8.577551729671881731e-01
7.013844214023315926e-01
3.138233436228588857e+00
7.319940508466630247e-01
0.000000000000000000e+00
3.529209394581482861e+00
1.464585117707144413e+01
8.506550226820598359e-01
-9.060397326548508268e-01
6.787474954688997641e+00
8.125411068867387954e+00
4.515740684104064151e+00
5.372135582950940069e+00
9.210951196799497254e-01
4.802177410869620466e+00
2.945445016176073594e+01
1.575032253128311632e+00
-1.355276854364796946e-01
-3.322474764169629502e-01
3.018397817188666732e+00
1.186503569922195744e+00
0.000000000000000000e+00
8.883242370198487503e-01
7.248276146728496627e+00
0.000000000000000000e+00
0.000000000000000000e+00
-4.831246718433664711e+00
6.124136970173365002e-01
4.145693892559814686e-01
2.683998941637626867e+00
2.063906603639539039e+00
2.989801217386735210e+00
2.262965767379551962e-01
2.240332214649647380e+00
5.984550782416063086e+00
4.587011255338186544e+00
1.233118485315272039e+01
1.115223490909697857e+00
-3.682686422016995476e+00
6.096498453291562258e-01
1.119275528656461516e+00
1.377886278915177731e-01
3.828176805973048324e+00
0.000000000000000000e+00
0.000000000000000000e+00
1.442927634029647344e+01
0.000000000000000000e+00
5.719118583309401593e-01
1.993632609731877392e-01
3.047472271520709430e+00
5.736784864911910198e+00
6.677826247219391220e+00
6.307175478564531090e+00
3.150295169417364249e+01
3.738597740702392258e+00
1.129754590514236234e+01
6.108506268573830056e+00
1.605489516792866667e+00
2.929631990348545489e+00
-2.832543082245212937e-02
1.358286530670594461e+00
1.655932469853677924e+00
6.701964773769768513e-01
2.131182050917533211e+00
2.998351165769753468e+00
7.772095996358327596e+00
1.285014785269981141e+00
4.407334784589962418e+00
1.719858214230612026e+00
-1.012765674651314063e+00
-5.749271123172469133e-01
-3.559614093795681278e+00
-3.073088477387719397e+00
-4.492469521371540431e+00
-3.753286990415885427e+00
-3.219255423324282273e+00
-2.806436518181075090e+00
-2.697305948568419875e+00
-7.879608430851776646e-01
4.625507221739111330e+00
4.809280703883450414e+00
-3.435194026629848629e+00
-3.218943068168937049e+00
3.335535704890596698e+00
2.071359212435486263e+00
4.538992059175040339e+00
-2.770772323566738038e+01
2.903047708571506735e+00
-4.436143805989154032e+00
-2.647991280011542381e-01
1.737252348126810064e+00
-4.121989655995259128e+00
3.209709099445720581e-01
1.012758514896711759e+01
3.313255624721038295e+00
4.631467619785444967e+00
7.668642402146534032e+00
6.780938812710099128e+00
-3.256164342602652972e+00
6.749565128319576779e-01
0.000000000000000000e+00
-4.407265954524525853e+00
0.000000000000000000e+00
-3.666522115024547901e+01
-7.886029397826226273e+01
0.000000000000000000e+00
0.000000000000000000e+00
-2.261283814517791058e+01
-4.024317426178160240e+00
3.213063737030031342e-01
5.079805145796887800e+00
1.326813226475260343e+00
1.233684078112145643e+00
8.671852503871454232e+00
-2.041800256066371944e+00
0.000000000000000000e+00
0.000000000000000000e+00
-1.607347800380474823e+01
-4.430790279223246309e+00
1.177552465851384511e+00
6.342921220500139512e+00
-2.466913734548706327e-02
3.451642566010713065e-01
-6.012767168531006234e+00
7.328146570137336724e+00
7.500088131707050465e+00
0.000000000000000000e+00
-3.547913249211809017e+01
-3.130964814607208879e+00
8.247326544297072237e-01
3.757262485775580418e-01
-2.136528302027558723e+00
-2.631627236037529793e-01
-2.016718799388414141e+01
0.000000000000000000e+00
0.000000000000000000e+00
0.000000000000000000e+00
-7.708602132869285528e-01
-2.602868328868111814e+00
1.435184800833797958e+00
0.000000000000000000e+00
-2.080420864280113413e+00
1.169498351211070819e+00
-1.798334115637199560e+01
-1.193885252696202670e+01
0.000000000000000000e+00
0.000000000000000000e+00
4.304089297965300709e+00
-3.020893216686394656e+00
-1.234427481614708721e+00
0.000000000000000000e+00
1.853340741926325697e+00
-2.686000064995862147e+01
-1.672275139058893600e+01
-2.826268691607605987e+01
0.000000000000000000e+00
0.000000000000000000e+00
-1.547397429377200817e+00
-4.018181657009961327e+00
-7.289186736637049968e+00
-7.458655219230571731e+00
-9.625538282761622710e+00
-1.103039457077456298e+01
-6.262675161142102809e+01
-9.265912629799268885e+00
-8.961543476816615339e+00
-9.622764435629340696e+00
-1.097978292092879826e+01
-1.097978292092879826e+01
-2.088057846500587456e+00
7.763936238952200239e-05
4.404309737393306595e-05
-9.467385027293546973e-02
4.348947142638090457e-01
-2.091409170053043390e-04
-5.788902107267982974e-04
4.481443450852441001e-10
-3.136135459023654537e+00
-1.507730262127600751e+00
-1.204663669965535977e+00
-1.171839254318371104e+00
4.329302247232582057e-01
8.310653628092458334e+00
3.299246725156660176e+00
0.000000000000000000e+00
1.847454931582027254e-02
-1.498024139966201096e+00
-7.660670007653060942e-01
-2.908130300830076731e+00
-1.252564844610269734e+00
-1.910955328742287573e+00
9.471710980110392697e-01
2.352302758516665371e+00
2.982269972214651954e+00
4.280736383314343918e+00
4.633629909719495288e+00
-2.198052637823726840e+00
-1.150759637168392580e+00
3.420433363184381292e+00
1.857878113059351077e-02
-1.559806674919653746e+00
4.197498183183401288e+00
6.262186949633183453e+00
1.100694844370524095e+01
2.778688785515088000e+01
3.679948298336883195e+00
-2.432801394376875592e+00
5.133442052706843617e-01
2.181172654073517680e+00
-8.577551729671881731e-01
7.013844214023315926e-01
3.138233436228588857e+00
7.319940508466630247e-01
0.000000000000000000e+00
3.529209394581482861e+00
1.464585117707144413e+01
8.506550226820598359e-01
-9.060397326548508268e-01
6.787474954688997641e+00
8.125411068867387954e+00
4.515740684104064151e+00
5.372135582950940069e+00
9.210951196799497254e-01
4.802177410869620466e+00
2.945445016176073594e+01
1.575032253128311632e+00
-1.355276854364796946e-01
-3.322474764169629502e-01
3.018397817188666732e+00
1.186503569922195744e+00
0.000000000000000000e+00
8.883242370198487503e-01
7.248276146728496627e+00
0.000000000000000000e+00
0.000000000000000000e+00
-4.831246718433664711e+00
6.124136970173365002e-01
4.145693892559814686e-01
2.683998941637626867e+00
2.063906603639539039e+00
2.989801217386735210e+00
2.262965767379551962e-01
2.240332214649647380e+00
5.984550782416063086e+00
4.587011255338186544e+00
1.233118485315272039e+01
1.115223490909697857e+00
-3.682686422016995476e+00
6.096498453291562258e-01
1.119275528656461516e+00
1.377886278915177731e-01
3.828176805973048324e+00
0.000000000000000000e+00
0.000000000000000000e+00
1.442927634029647344e+01
0.000000000000000000e+00
5.719118583309401593e-01
1.993632609731877392e-01
3.047472271520709430e+00
5.736784864911910198e+00
6.677826247219391220e+00
6.307175478564531090e+00
3.150295169417364249e+01
3.738597740702392258e+00
1.129754590514236234e+01
6.108506268573830056e+00
1.605489516792866667e+00
2.929631990348545489e+00
-2.832543082245212937e-02
1.358286530670594461e+00
1.655932469853677924e+00
6.701964773769768513e-01
2.131182050917533211e+00
2.998351165769753468e+00
7.772095996358327596e+00
1.285014785269981141e+00
4.407334784589962418e+00
1.719858214230612026e+00
-1.012765674651314063e+00
-5.749271123172469133e-01
-3.559614093795681278e+00
-3.073088477387719397e+00
-4.492469521371540431e+00
-3.753286990415885427e+00
-3.219255423324282273e+00
-2.806436518181075090e+00
-2.697305948568419875e+00
-7.879608430851776646e-01
4.625507221739111330e+00
4.809280703883450414e+00
-3.435194026629848629e+00
-3.218943068168937049e+00
3.335535704890596698e+00
2.071359212435486263e+00
4.538992059175040339e+00
-2.770772323566738038e+01
2.903047708571506735e+00
-4.436143805989154032e+00
-2.647991280011542381e-01
1.737252348126810064e+00
-4.121989655995259128e+00
3.209709099445720581e-01
1.012758514896711759e+01
3.313255624721038295e+00
4.631467619785444967e+00
7.668642402146534032e+00
6.780938812710099128e+00
-3.256164342602652972e+00
6.749565128319576779e-01
0.000000000000000000e+00
-4.407265954524525853e+00
0.000000000000000000e+00
-3.666522115024547901e+01
-7.886029397826226273e+01
0.000000000000000000e+00
0.000000000000000000e+00
-2.261283814517791058e+01
-4.024317426178160240e+00
3.213063737030031342e-01
5.079805145796887800e+00
1.326813226475260343e+00
1.233684078112145643e+00
8.671852503871454232e+00
-2.041800256066371944e+00
0.000000000000000000e+00
0.000000000000000000e+00
-1.607347800380474823e+01
-4.430790279223246309e+00
1.177552465851384511e+00
6.342921220500139512e+00
-2.466913734548706327e-02
3.451642566010713065e-01
-6.012767168531006234e+00
7.328146570137336724e+00
7.500088131707050465e+00
0.000000000000000000e+00
-3.547913249211809017e+01
-3.130964814607208879e+00
8.247326544297072237e-01
3.757262485775580418e-01
-2.136528302027558723e+00
-2.631627236037529793e-01
-2.016718799388414141e+01
0.000000000000000000e+00
0.000000000000000000e+00
0.000000000000000000e+00
-7.708602132869285528e-01
-2.602868328868111814e+00
1.435184800833797958e+00
0.000000000000000000e+00
-2.080420864280113413e+00
1.169498351211070819e+00
-1.798334115637199560e+01
-1.193885252696202670e+01
0.000000000000000000e+00
0.000000000000000000e+00
4.304089297965300709e+00
-3.020893216686394656e+00
-1.234427481614708721e+00
0.000000000000000000e+00
1.853340741926325697e+00
-2.686000064995862147e+01
-1.672275139058893600e+01
-2.826268691607605987e+01
0.000000000000000000e+00
0.000000000000000000e+00
-1.547397429377200817e+00
-4.018181657009961327e+00
-7.289186736637049968e+00
-7.458655219230571731e+00
-9.625538282761622710e+00
-1.103039457077456298e+01
-6.262675161142102809e+01
-9.265912629799268885e+00
-8.961543476816615339e+00
-9.622764435629340696e+00
-1.097978292092879826e+01
-1.097978292092879826e+01
-2.088057846500587456e+00
7.763936238952200239e-05
4.404309737393306595e-05
-9.467385027293546973e-02
4.348947142638090457e-01
-2.091409170053043390e-04
-5.788902107267982974e-04
4.481443450852441001e-10
-3.136135459023654537e+00
-1.507730262127600751e+00
-1.204663669965535977e+00
-1.171839254318371104e+00
4.329302247232582057e-01
8.310653628092458334e+00
3.299246725156660176e+00
0.000000000000000000e+00
1.847454931582027254e-02
-1.498024139966201096e+00
-7.660670007653060942e-01
-2.908130300830076731e+00
-1.252564844610269734e+00
-1.910955328742287573e+00
9.471710980110392697e-01
2.352302758516665371e+00
2.982269972214651954e+00
4.280736383314343918e+00
4.633629909719495288e+00
-2.198052637823726840e+00
-1.150759637168392580e+00
3.420433363184381292e+00
1.857878113059351077e-02
-1.559806674919653746e+00
4.197498183183401288e+00
6.262186949633183453e+00
1.100694844370524095e+01
2.778688785515088000e+01
3.679948298336883195e+00
-2.432801394376875592e+00
5.133442052706843617e-01
2.181172654073517680e+00
-8.577551729671881731e-01
7.013844214023315926e-01
3.138233436228588857e+00
7.319940508466630247e-01
0.000000000000000000e+00
3.529209394581482861e+00
1.464585117707144413e+01
8.506550226820598359e-01
-9.060397326548508268e-01
6.787474954688997641e+00
8.125411068867387954e+00
4.515740684104064151e+00
5.372135582950940069e+00
9.210951196799497254e-01
4.802177410869620466e+00
2.945445016176073594e+01
1.575032253128311632e+00
-1.355276854364796946e-01
-3.322474764169629502e-01
3.018397817188666732e+00
1.186503569922195744e+00
0.000000000000000000e+00
8.883242370198487503e-01
7.248276146728496627e+00
0.000000000000000000e+00
0.000000000000000000e+00
-4.831246718433664711e+00
6.124136970173365002e-01
4.145693892559814686e-01
2.683998941637626867e+00
2.063906603639539039e+00
2.989801217386735210e+00
2.262965767379551962e-01
2.240332214649647380e+00
5.984550782416063086e+00
4.587011255338186544e+00
1.233118485315272039e+01
1.115223490909697857e+00
-3.682686422016995476e+00
6.096498453291562258e-01
1.119275528656461516e+00
1.377886278915177731e-01
3.828176805973048324e+00
0.000000000000000000e+00
0.000000000000000000e+00
1.442927634029647344e+01
0.000000000000000000e+00
5.719118583309401593e-01
1.993632609731877392e-01
3.047472271520709430e+00
5.736784864911910198e+00
6.677826247219391220e+00
6.307175478564531090e+00
3.150295169417364249e+01
3.738597740702392258e+00
1.129754590514236234e+01
6.108506268573830056e+00
1.605489516792866667e+00
2.929631990348545489e+00
-2.832543082245212937e-02
1.358286530670594461e+00
1.655932469853677924e+00
6.701964773769768513e-01
2.131182050917533211e+00
2.998351165769753468e+00
7.772095996358327596e+00
1.285014785269981141e+00
4.407334784589962418e+00
1.719858214230612026e+00
-1.012765674651314063e+00
-5.749271123172469133e-01
-3.559614093795681278e+00
-3.073088477387719397e+00
-4.492469521371540431e+00
-3.753286990415885427e+00
-3.219255423324282273e+00
-2.806436518181075090e+00
-2.697305948568419875e+00
-7.879608430851776646e-01
4.625507221739111330e+00
4.809280703883450414e+00
-3.435194026629848629e+00
-3.218943068168937049e+00
3.335535704890596698e+00
2.071359212435486263e+00
4.538992059175040339e+00
-2.770772323566738038e+01
2.903047708571506735e+00
-4.436143805989154032e+00
-2.647991280011542381e-01
1.737252348126810064e+00
-4.121989655995259128e+00
3.209709099445720581e-01
1.012758514896711759e+01
3.313255624721038295e+00
4.631467619785444967e+00
7.668642402146534032e+00
6.780938812710099128e+00
-3.256164342602652972e+00
6.749565128319576779e-01
0.000000000000000000e+00
-4.407265954524525853e+00
0.000000000000000000e+00
-3.666522115024547901e+01
-7.886029397826226273e+01
0.000000000000000000e+00
0.000000000000000000e+00
-2.261283814517791058e+01
-4.024317426178160240e+00
3.213063737030031342e-01
5.079805145796887800e+00
1.326813226475260343e+00
1.233684078112145643e+00
8.671852503871454232e+00
-2.041800256066371944e+00
0.000000000000000000e+00
0.000000000000000000e+00
-1.607347800380474823e+01
-4.430790279223246309e+00
1.177552465851384511e+00
6.342921220500139512e+00
-2.466913734548706327e-02
3.451642566010713065e-01
-6.012767168531006234e+00
7.328146570137336724e+00
7.500088131707050465e+00
0.000000000000000000e+00
-3.547913249211809017e+01
-3.130964814607208879e+00
8.247326544297072237e-01
3.757262485775580418e-01
-2.136528302027558723e+00
-2.631627236037529793e-01
-2.016718799388414141e+01
0.000000000000000000e+00
0.000000000000000000e+00
0.000000000000000000e+00
-7.708602132869285528e-01
-2.602868328868111814e+00
1.435184800833797958e+00
0.000000000000000000e+00
-2.080420864280113413e+00
1.169498351211070819e+00
-1.798334115637199560e+01
-1.193885252696202670e+01
0.000000000000000000e+00
0.000000000000000000e+00
4.304089297965300709e+00
-3.020893216686394656e+00
-1.234427481614708721e+00
0.000000000000000000e+00
1.853340741926325697e+00
-2.686000064995862147e+01
-1.672275139058893600e+01
-2.826268691607605987e+01
0.000000000000000000e+00
0.000000000000000000e+00
-1.547397429377200817e+00
-4.018181657009961327e+00
-7.289186736637049968e+00
-7.458655219230571731e+00
-9.625538282761622710e+00
-1.103039457077456298e+01
-6.262675161142102809e+01
-9.265912629799268885e+00
-8.961543476816615339e+00
-9.622764435629340696e+00
-1.097978292092879826e+01
-1.097978292092879826e+01
+1
View File
@@ -0,0 +1 @@
59708719594136008472212676481906
@@ -0,0 +1 @@
5コ705コ(770(777
+1
View File
@@ -0,0 +1 @@
2Bba5a7FEEcfced4fbbBfFEe16bfcaA4
@@ -0,0 +1 @@
5コ70(77247・906:
@@ -0,0 +1 @@
5コ700772212676481906:
Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More