Compare commits

..
4685 Commits
Author SHA1 Message Date
Ivan Nardi ebcea42e2b fuzz: improve fuzzers using pl7m (#2486)
For some unclear reasons, fuzzers using pl7m create huge corpus,
triggering OOM in oss-fuzz runs (where the memory RSS limit is set to
2560Mb). Example:

```
==25340== ERROR: libFuzzer: out-of-memory (used: 2564Mb; limit: 2560Mb)
To change the out-of-memory limit use -rss_limit_mb=<N>

Live Heap Allocations: 2364004039 bytes in 133791 chunks; quarantined: 60662293 bytes in 3664 chunks; 176432 other chunks; total chunks: 313887; showing top 95% (at most 8 unique contexts)
1285841683 byte(s) (54%) in 2956 allocation(s)
    #0 0x56f814ef4bde in __interceptor_malloc /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
    #1 0x56f814e04416 in operator new(unsigned long) cxa_noexception.cpp:0
    #2 0x56f814de6b2d in assign<unsigned char *, 0> /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1443:3
    #3 0x56f814de6b2d in operator= /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1412:9
    #4 0x56f814de6b2d in fuzzer::InputCorpus::AddToCorpus(std::__Fuzzer::vector<unsigned char, std::__Fuzzer::allocator<unsigned char>> const&, unsigned long, bool, bool, bool, std::__Fuzzer::chrono::duration<long long, std::__Fuzzer::ratio<1l, 1000000l>>, std::__Fuzzer::vector<unsigned int, std::__Fuzzer::allocator<unsigned int>> const&, fuzzer::DataFlowTrace const&, fuzzer::InputInfo const*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerCorpus.h:221:10
    #5 0x56f814de60e5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:539:16
    #6 0x56f814de7df2 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:829:7
    #7 0x56f814de8127 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:867:3
    #8 0x56f814dd6736 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:914:6
    #9 0x56f814e02c62 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    #10 0x7fa11e2c3082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16

1031350683 byte(s) (43%) in 2468 allocation(s)
    #0 0x56f814ef4bde in __interceptor_malloc /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
    #1 0x56f814e04416 in operator new(unsigned long) cxa_noexception.cpp:0
    #2 0x56f814de6b2d in assign<unsigned char *, 0> /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1443:3
    #3 0x56f814de6b2d in operator= /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1412:9
    #4 0x56f814de6b2d in fuzzer::InputCorpus::AddToCorpus(std::__Fuzzer::vector<unsigned char, std::__Fuzzer::allocator<unsigned char>> const&, unsigned long, bool, bool, bool, std::__Fuzzer::chrono::duration<long long, std::__Fuzzer::ratio<1l, 1000000l>>, std::__Fuzzer::vector<unsigned int, std::__Fuzzer::allocator<unsigned int>> const&, fuzzer::DataFlowTrace const&, fuzzer::InputInfo const*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerCorpus.h:221:10
    #5 0x56f814de60e5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:539:16
    #6 0x56f814de7635 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:760:19
    #7 0x56f814de8425 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:905:5
    #8 0x56f814dd6736 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:914:6
    #9 0x56f814e02c62 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    #10 0x7fa11e2c3082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
```

See: https://oss-fuzz.com/testcase-detail/4717811415449600
See: https://oss-fuzz.com/testcase-detail/6164130982068224

Let's *try* the following workaround: set the parameter `-max-len` to
512K, to force the engine to not genereate inputs (i.e. pcap files...)
larger than 512K. Right now the value used is 1MB, i.e the default,
because we have file larger than 1MB in the initial seeds (i.e.
`/tests/pcaps/*`).

Let's hope than having smaller files lead to smaller corpus...

Update pl7m code (fix a Use-of-uninitialized-value error)
2024-06-29 18:01:05 +02:00
Ivan Nardi 83e6e753af fuzz: pl7m: add a custom mutator for better fuzzing of pcap files (#2483)
Pl7m is a custom mutator (used for structure aware fuzzing) for network
traffic packet captures (i.e. pcap files).

The output of the mutator is always a valid pcap file, containing the
same flows/sessions of the input file. That's it: the mutator only
changes the packet payload after the TCP/UDP header, keeping all the
original L2/L3 information (IP addresses and L4 ports).

See: https://github.com/IvanNardi/pl7m
2024-06-27 18:07:43 +02:00
Anshul ThakurandAnshul Thakur 8f6f73505d Remove duplicate API for ndpi_serialize_uint32_double in ndpi_api.h (#2480)
Co-authored-by: Anshul Thakur <athakur@cdot.in>
2024-06-26 14:10:57 +02:00
Nardi Ivan 556f892a56 wireshark: lua: export some metadata
Export some metadata (for the moment, SNI and TLS fingerprints) to
Wireshark/tshark via extcap.
Note that:
* metadata are exported only once per flow
* metadata are exported (all together) when nDPI stopped processing
the flow

Still room for a lot of improvements!
In particular:
* we need to add some boundary checks (if we are going to export other
attributes)
* we should try to have a variable length trailer
2024-06-25 16:39:45 +02:00
Nardi Ivan f44832cc51 wireshark: lua: filtering and trailer dissection work with tshark, too
```
ivan@ivan-Latitude-E6540:~/$ tshark -C "nDPI extcap" -i ndpi -o extcap.ndpi.i:/home/ivan/svnrepos/nDPI/tests/pcap/anydesk.pcapng -Y "ndpi.protocol.name contains DNS"
Capturing on 'nDPI interface: ndpi'
   62 22635386.425683 192.168.1.187 DNS.AnyDesk 192.168.1.1  128   Standard query 0xec22 A relay-3185a847.net.anydesk.com
   63 22635386.439540  192.168.1.1 DNS.AnyDesk 192.168.1.187 144   Standard query response 0xec22 A relay-3185a847.net.anydesk.com A 37.61.223.15
   64 22635386.721277 192.168.1.187 DNS.AnyDesk 192.168.1.1  128   Standard query 0xea89 A relay-9b6827f2.net.anydesk.com
   65 22635386.732444  192.168.1.1 DNS.AnyDesk 192.168.1.187 144   Standard query response 0xea89 A relay-9b6827f2.net.anydesk.com A 138.199.36.115
4 packets captured

```
2024-06-25 16:39:45 +02:00
Nardi Ivan 3f0ea18866 wireshark: lua: fix DNS dissection
Not sure when we (or Wireshark, or Lua...) broke it, but we can't call
tonumber() on Bool variables.
2024-06-25 16:39:45 +02:00
Nardi Ivan b5afa165f0 wireshark: extcap: restore filtering mechanism 2024-06-25 16:39:45 +02:00
Nardi Ivan 2daab3f248 wireshark: lua: latest Wireshark versions correctly handle 64 bit mask 2024-06-25 16:39:45 +02:00
Nardi Ivan 46e450456b wireshark: lua: minor improvements
* Use a proper TVB to parse the nDPI trailer
* Fix some flow risks definitions
2024-06-25 16:39:45 +02:00
Luca Deri e92568e4bb Improved logic for checking invalid DNS queries 2024-06-23 22:27:19 +02:00
Mark Jeffery aa1d7247d1 Added default port mappings to ndpiReader help -H (#2477)
Close #2125
2024-06-19 13:47:18 +02:00
Ivan Nardi 9feede9a14 Zoom: fix detection of screen sharing (#2476) 2024-06-17 17:30:24 +02:00
Ivan Nardi 26cc1f131f fuzz: improve fuzzing coverage (#2474)
Remove some code never triggered

AFP: the removed check is included in the following one
MQTT: fix flags extraction
2024-06-17 13:45:47 +02:00
Nardi Ivan a35fae6b75 Sync unit tests results 2024-06-17 11:23:39 +02:00
Toni 8fd649ab1e Add Ripe Atlas probe protocol. (#2473)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-06-17 11:00:38 +02:00
Nardi Ivan bbe52da5cf Zoom: harden RTP/RTCP detection 2024-06-17 10:19:55 +02:00
Nardi Ivan 526cf6f291 Zoom: remove "stun_zoom" LRU cache
Since 070a0908b we are able to detect P2P calls directly from the packet
content, without any correlation among flows
2024-06-17 10:19:55 +02:00
Ivan Nardi 2bedd14aae Update main Readme and remove commented out code (#2471)
Avoid that useless code is copy-and-pasted into new dissector, as in
https://github.com/ntop/nDPI/pull/2470#discussion_r1639384434
2024-06-17 10:11:17 +02:00
Mark Jeffery f796c94375 Added protocol - JRMI - Java Remote Method Invocation (#2470) 2024-06-15 10:52:28 +02:00
Luca 7c7c375b45 Improved detection of Android connectiity checks 2024-06-12 10:55:07 +02:00
Mark Jeffery 312dc424bd Added NDPI_PROTOCOL_NTOP assert and removed percentage comparison (#2460)
Close #2413
2024-06-10 19:45:19 +02:00
Ivan Nardi aee2c81f76 Zoom: fix integer overflow (#2469)
```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==29508==ERROR: AddressSanitizer: SEGV on unknown address 0x50710145d51d (pc 0x55cb788f25fe bp 0x7ffcfefa15f0 sp 0x7ffcfefa1240 T0)
==29508==The signal is caused by a READ memory access.
    #0 0x55cb788f25fe in ndpi_search_zoom /home/ivan/svnrepos/nDPI/src/lib/protocols/zoom.c:210:24
    #1 0x55cb787e9418 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7174:6
    #2 0x55cb7883f753 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7209:10
    #3 0x55cb7883bc9d in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7240:12
```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=69520
2024-06-10 11:06:17 +02:00
Ivan Nardi b90d39c4ac RTP/STUN: look for STUN packets after RTP/RTCP classification (#2465)
After a flow has been classified as RTP or RTCP, nDPI might analyse more
packets to look for STUN/DTLS packets, i.e. to try to tell if this flow
is a "pure" RTP/RTCP flow or if the RTP/RTCP packets are multiplexed with
STUN/DTLS.
Useful for proper (sub)classification when the beginning of the flows
are not captured or if there are lost packets in the the captured traffic.

Disabled by default
2024-06-07 13:12:04 +02:00
Ivan Nardi 070a0908b3 Zoom: faster detection of P2P flows (#2467) 2024-06-07 09:50:41 +02:00
Ivan Nardi 619005c5b2 STUN: add support for Microsoft Multiplexed TURN channels (#2464) 2024-06-05 16:55:58 +02:00
Ivan Nardi b6beeef623 TLS: add support for DTLS (over STUN) over TCP (#2463)
TODO: TCP reassembler on top of UDP reassembler

See: #2414
2024-06-05 13:49:34 +02:00
Ivan Nardi 6a9d4b1ab6 CI: more parallel work (#2459) 2024-06-05 11:56:03 +02:00
Ivan Nardi 7be482f5b1 Update unit tests results (#2466) 2024-06-05 11:55:30 +02:00
Luca Deri 85587a8f98 Fixes accounting of number of processed packets in ndpi_internal_detection_process_packet() 2024-06-04 17:00:45 +02:00
Ivan Nardi 81e42b748e RTP: fix detection over TCP (#2462)
RFC4571 is not the only way to wrap RTP messages in TCP streams.
For example, when RTP is encapsulated over TURN flows (i.e. via DATA
attribute) there is no additional framing.
See also 6127e0490
2024-05-29 18:31:10 +02:00
Maatuq 6127e04900 support rtp/rtcp over tcp (#2422) (#2457)
Support rtp/rtcp over tcp as per rfc4571.

Signed-off-by: mmaatuq <mahmoudmatook.mm@gmail.com>
2024-05-28 22:01:08 +02:00
Toni 80171dbcf3 Add ZUG consensus protocol dissector. (#2458)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-28 20:29:48 +02:00
Ivan Nardi 25f8964a23 CiscoVPN: we detect it only over UDP (#2454)
The original code handled also TCP/TLS, but it was removed in 6fc29b3ae
2024-05-28 14:07:48 +02:00
ToniandNardi Ivan abce6d4023 Improved Kafka dissector. (#2456)
* detect more Kafka request packet's
 * requires less flow memory
 * same detection behavior as before e.g. no asym detection implemented
   (can be done by dissecting responses, requires more effort)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2024-05-27 09:46:32 +02:00
Vladimir Gavrilov d1a59c0308 Rename Messenger to FacebookMessenger (#2453) 2024-05-23 09:23:31 +02:00
Ivan Nardi 41050af5b5 fuzz: fix build of oss-introspector (#2452) 2024-05-22 19:41:59 +02:00
Nardi Ivan 521ce609c8 Sync unit tests results 2024-05-22 19:00:33 +02:00
Luca 44a290286b More NDPI_PROBING_ATTEMPT changes 2024-05-22 18:04:33 +02:00
Luca 74d3843ebe Added NDPI_PROBING_ATTEMPT risk 2024-05-22 17:55:16 +02:00
Vladimir Gavrilov 15643547fe Replace ndpi_strnstr() implementation with an optimal one (#2447) 2024-05-22 12:47:27 +02:00
Luca Deri 5a25f89ab3 Enlarged buffer 2024-05-22 11:00:01 +02:00
Ivan Nardi fd02baa13a DTLS: fix JA4 fingerprint (#2446) 2024-05-21 18:13:25 +02:00
Toni 3639d2045b Remove unused code. (#2450)
* some `#ifdef`ed code dates back to 2019, 2020 and 2021
 * some function signatures were still present in `ndpi_main.h`
   which may cause linker errors for libnDPI dependee's
 * return an error while trying to serialize a double instead
   of `fprintf(stderr, ...)`

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-21 13:08:07 +02:00
Ivan Nardi 83840f1bb9 DTLS: add support for DTLS 1.3 (#2445) 2024-05-21 12:49:27 +02:00
Ivan Nardi 0109014f2c Follow-up of 2093ac5bf (#2451) 2024-05-21 12:47:25 +02:00
Ivan Nardi bc8bc69beb CI: enable parallel tests (for x86_64, at least) (#2444)
TODO: enable parallel tests when using docker with no-x86_64 archs.

When I tried the obviuos solutions:
```
NDPI_FORCE_PARALLEL_UTESTS=1 NDPI_SKIP_PARALLEL_BAR=1 make check VERBOSE=1
```
I got:
```
Run configuration "caches_cfg" [--cfg=lru.ookla.size,0 --cfg=lru.msteams.ttl,1]
ookla.pcap	/bin/sh: 1: run_single_pcap: not found
teams.pcap	/bin/sh: 1: run_single_pcap: not found
Run configuration "caches_global" [--cfg=lru.ookla.scope,1 --cfg=lru.bittorrent.scope,1 --cfg=lru.stun.scope,1 --cfg=lru.tls_cert.scope,1 --cfg=lru.mining.scope,1 --cfg=lru.msteams.scope,1 --cfg=lru.stun_zoom.scope,1]
bittorrent.pcap	/bin/sh: 1: run_single_pcap: not found
lru_ipv6_caches.pcapng	/bin/sh: 1: run_single_pcap: not found
mining.pcapng	/bin/sh: 1: run_single_pcap: not found
...
```
2024-05-20 19:07:16 +02:00
Ivan Nardi 399be12585 Small fixes after API cleanup done in c63446e59 (#2449) 2024-05-20 19:06:24 +02:00
Luca Deri 2093ac5bf6 Minor dissector optimizations 2024-05-20 12:17:04 +02:00
Luca Deri 42dba2e4af Added dpi.compute_entropy configuration parameter 2024-05-18 09:46:15 +02:00
Luca Deri c63446e592 Cleaned up API
Removed
-  int ndpi_load_ipv4_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);
-  int ndpi_load_ipv6_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);

Added (it supports both IPv4 and v6)
+  int ndpi_load_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);
2024-05-17 16:38:02 +02:00
Ivan Nardi 33d3d254ac CI: add support for ubuntu-24.04 runners (#2443) 2024-05-16 17:05:13 +02:00
Vladimir Gavrilov 3d1da00d8d Add Call of Duty Mobile support (#2438) 2024-05-15 12:46:02 +02:00
Ivan Nardi d6b1c24079 Parallel execution of unit tests (#2435)
Running unit tests is quite a bottleneck while developing or while
waiting for GitHub CI results...
Try to run the tests in parallel, using the `parallel` tool.

By default, tests still run one after the other, as usual; to enable
parallel execution you need `NDPI_FORCE_PARALLEL_UTESTS=1 ./tests/do.sh`

Please note that the output is quite different in parallel mode!

A big part of the script has been rewritten to avoid code dupication
between "serial" and "parallel" path

On my notebook:
```
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(parallel)$ time ./tests/do.sh
[...]
real	3m12,684s
[...]
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(parallel)$ time NDPI_FORCE_PARALLEL_UTESTS=1 ./tests/do.sh
[...]
real	0m58,463s
```
2024-05-15 08:17:12 +02:00
Ivan Nardi 0110623b4e H323: improve detection and avoid false positives (#2432) 2024-05-11 23:39:54 +02:00
Ivan Nardi a064261e85 Revert ndpi_strnstr() optimization introduced in a813121e0 (#2439)
New implementation fails tests 11b, 12 and 13.
Revert to the original (BSD) implementation (with also some basic
parameters check)
2024-05-11 23:37:31 +02:00
Vladimir Gavrilov 2d33431948 Add Ethernet Global Data support (#2437) 2024-05-11 16:49:16 +02:00
Ivan Nardi 8c5e6e9761 IRC: simplify detection (#2423)
IRC has its best times well behind, but there are still some servers
using it.

We should try to simplify the detection logic, still based on OpenDPI
logic.

Let's start with some easy changes:
* try to detect TLS connection via standard hostname/SNI matching,
removing an old heuristic (we have never had any trace matching it);
* add some basic server names;
* once we detect that the flow is IRC, we don't have to perform
anything else;
* remove HTTP stuff; real HTTP flows never trigger that data path
* use `ndpi_memmem()` when possible
2024-05-11 14:12:13 +02:00
Vladimir GavrilovandIvan Nardi 70c4920184 Remove Vevo support (#2436)
Co-authored-by: Ivan Nardi <12729895+IvanNardi@users.noreply.github.com>
2024-05-11 11:57:53 +02:00
Ivan Nardi b116456fc5 Viber: add detection of voip calls and avoid false positives (#2434) 2024-05-11 09:21:13 +02:00
Vladimir Gavrilov a813121e0a ndpi_strnstr() optimization (#2433) 2024-05-10 22:43:59 +02:00
Ivan Nardi 4b4b358562 Line: use common helper to detect RTP/RTCP packets (#2429)
Add an explicit upper limit on the number of packets processed before
giving up.
2024-05-10 19:23:50 +02:00
Luca Deri 6bbae9a70a Update README.md 2024-05-10 08:53:19 +02:00
Ivan Nardi d4650f0f81 Raknet/RTP: avoid Raknet false positives and harden RTP heuristic (#2427)
There is some overlap between RTP and Raknet detection: give precedence
to RTP logic.
Consequences:
* Raknet might require a little bit more packets for some flows (not a
big issue)
* some very small (1-2 pkts) Raknet flows are not classified (not sure
what do do about that..)
2024-05-09 21:18:18 +02:00
Luca Deri e4fd6e47a3 Added #ifdef for Windows builds (#2431) 2024-05-09 17:27:25 +02:00
Ivan Nardi 2e7fc82e55 Protobuf: fix false positives (#2428) 2024-05-09 15:38:06 +02:00
Toni 18e03a26ca Add extra entropy checks and more precise(?) analysis. (#2383)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-09 15:24:11 +02:00
Ivan Nardi b65a755e85 Fix a memory access error and some leaks (#2425)
```
SCARINESS: 12 (1-byte-read-heap-buffer-overflow)
    #0 0x557f3a5b5100 in ndpi_get_host_domain /src/ndpi/src/lib/ndpi_domains.c:158:8
    #1 0x557f3a59b561 in ndpi_check_dga_name /src/ndpi/src/lib/ndpi_main.c:10412:17
    #2 0x557f3a51163a in process_chlo /src/ndpi/src/lib/protocols/quic.c:1467:7
    #3 0x557f3a469f4b in LLVMFuzzerTestOneInput /src/ndpi/fuzz/fuzz_quic_get_crypto_data.c:44:7
    #4 0x557f3a46abc8 in NaloFuzzerTestOneInput (/out/fuzz_quic_get_crypto_data+0x4cfbc8)
```

Some notes about the leak: if the insertion into the uthash fails (because of an
allocation failure), we need to free the just allocated entry. But the only
way to check if the `HASH_ADD_*` failed, is to perform a new lookup: a bit
costly, but we don't use that code in the fast-path.
See also efb261a95c

Credits for finding the issues to Philippe Antoine (@catenacyber) and his
`nallocfuzz` fuzzing engine
See: https://github.com/catenacyber/nallocfuzz
See: https://github.com/google/oss-fuzz/pull/9902
2024-05-08 11:46:46 +02:00
Ivan Nardi 7c6910d9e5 Fix/improve fuzzing (#2426) 2024-05-08 11:46:02 +02:00
Toni e9dc035c5c Added optimized memmem/strlcpy version (#2424)
* credits goes to Vladimir Gavrilov

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-05-08 11:38:53 +02:00
Luca Deri 2b4e2f9c9a Removed trace 2024-05-07 23:42:32 +02:00
Luca Deri 5f8ad0527d Reworked code and added two new API calls
- int ndpi_load_ipv4_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);
- int ndpi_load_ipv6_ptree_file(ndpi_ptree_t *tree, const char *path, u_int16_t protocol_id);
2024-05-07 23:05:31 +02:00
Luca Deri 3d2b611ea6 ndpi_mod is now optional (albeit better to spcify it) in ndpi_domain_classify_xxx calls 2024-05-07 13:09:00 +02:00
Luca Deri 0d495294bf Win warning fixes 2024-05-07 10:04:11 +02:00
Luca Deri 6195dbad76 Renamed radius source file to avoid name clashes on Windows 2024-05-07 09:05:15 +02:00
Ivan Nardi 1773d7ff2d Add support for Mastodon, Bluesky and (FB-)Threads (#2418) 2024-05-06 13:37:18 +02:00
Ivan Nardi 95fe21015d Remove "zoom" cache (#2420)
This cache was added in b6b4967aa, when there was no real Zoom support.
With 63f349319, a proper identification of multimedia stream has been
added, making this cache quite useless: any improvements on Zoom
classification should be properly done in Zoom dissector.

Tested for some months with a few 10Gbits links of residential traffic: the
cache pretty much never returned a valid hit.
2024-05-06 12:51:45 +02:00
Ivan Nardi b7d77972bc Remove workaround for TCP flows with multiple SYNs (#2421)
Deciding when a session starts and ends is responsability of the
applicationi (via its flow manager)i, not of the library.
BTW, the removed code is incomplete at beast
2024-05-06 10:20:40 +02:00
Ivan Nardi e31ef00715 TLS: avoid setting NDPI_TLS_SELFSIGNED_CERTIFICATE for webrtc traffic (#2417)
See RFC8122: it is quite likely that STUN/DTLS/SRTP flows use
self-signed certificates

Follow-up of b287d6ec8
2024-05-06 10:20:07 +02:00
Ivan Nardi 266af02752 Merge RTP and RTCP logic (#2416)
Avoid code duplication between these two protocols.

We remove support for RTCP over TCP; it is quite rare to find this kind
of traffic and, more important, we have never had support for RTP
over TCP: we should try to add both detecion as follow-up.

Fix a message log in the LINE code
2024-05-06 10:19:46 +02:00
Ivan Nardi a1845da359 TLS: fix Ja4 fingerprint computation (#2419)
The new values has been checked against the ones reported by Wireshark.

Found while fixing a Use-of-uninitialized-value error reported by
oss-fuzz

```
==7582==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x5a6549abc368 in ndpi_compute_ja4 ndpi/src/lib/protocols/tls.c:1762:10
    #1 0x5a6549ab88a0 in processClientServerHello ndpi/src/lib/protocols/tls.c:2863:10
    #2 0x5a6549ac1452 in processTLSBlock ndpi/src/lib/protocols/tls.c:909:5
    #3 0x5a6549abf588 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:1098:2
    #4 0x5a65499c53ec in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:7215:6
```

See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68449&q=ndpi&can=1&sort=-id
2024-05-05 17:27:29 +02:00
Ivan Nardi a6fd981fcf eDonkey: improve/update classification (#2410)
eDonkey is definitely not as used as >10 years ago, but it seems it is
still active.

While having a basic TCP support seems easy, identification over UDP doesn't
work and it is hard to do it rightly (packets might be only 2 bytes long):
remove it.

Credits to V.G <v.gavrilov@securitycode.ru>
2024-05-04 19:11:31 +02:00
Luca Deri 57ecbf38c0 Updated JA4 test results 2024-05-02 17:40:24 +02:00
Luca Deri 66133d1501 Fixes JA4 computation adding a better GREASE detect funzion 2024-05-02 16:58:44 +02:00
Luca Deri af955169e1 typo 2024-05-02 16:03:30 +02:00
Ivan Nardi 9e549ba781 Workaround for fixing GitHub runners on macOS (#2411)
GitHub switched "macos-latest" from "macos-12" to "macos-14", which is only
on ARM64!
https://github.com/actions/runner/issues/3256
https://github.blog/changelog/2024-01-30-github-actions-macos-14-sonoma-is-now-available/

However we are having some issues build nDPI on macos-14 with external
libraries:

```
configure: error: libgpg-error required (because of --with-local-libgcrypt) but not found or too old.
```
See: https://github.com/ntop/nDPI/actions/runs/8869020568/job/24350356867

```
ndpi_utils.c:69:10: fatal error: 'pcre2.h' file not found
         ^~~~~~~~~
1 error generated.
```
See: https://github.com/ntop/nDPI/actions/runs/8869020568/job/24349242251

Everything is still fine with macos-14 and no external dependencies

As workaround, test only macos-12 and macos-13 in our main matrix.
2024-04-28 22:17:39 +02:00
Ivan Nardi d5bda47efe DTLS: add support for Alert message type (similar to TLS) (#2406) 2024-04-25 11:23:05 +02:00
0x41CEA55 7040847eed Add Adobe Connect support (#2407) 2024-04-24 12:41:25 +02:00
0x41CEA55 66036a14f4 Remove PPStream protocol and add iQIYI (#2403)
P2P video player PPStream was discontinued shortly after the purchase of PPS.tv by Baidu (iQIYI) on 2013 (see https://www.techinasia.com/report-baidu-acquires-video-rival-pps)
So we remove the old `NDPI_PROTOCOL_PPSTREAM` logic and add `NDPI_PROTOCOL_IQIYI` id to handle all the iQIYI traffic, which is basically video streaming traffic.

A video hosting service, called PPS.tv, is still offered by the same company: for the time being we classified both services with the same protocol id.
2024-04-23 18:01:36 +02:00
0x41CEA55 fd388845d5 Add BFCP protocol support (#2401) 2024-04-23 15:35:19 +02:00
Ivan Nardi abc7e430e2 STUN: slightly faster sub-classification with DTLS (#2404) 2024-04-23 09:31:48 +02:00
Ivan Nardi a62679952c fuzz: fix shoco fuzzer (#2405)
```
==22779==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f0900701020 at pc 0x555bcd2a6f02 bp 0x7ffe3ba5e790 sp 0x7ffe3ba5e788
READ of size 1 at 0x7f0900701020 thread T0
    #0 0x555bcd2a6f01 in shoco_decompress /home/ivan/svnrepos/nDPI/src/lib/third_party/src/shoco.c:184:26
    #1 0x555bcd2a4018 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco.cpp:18:3
    #2 0x555bcd1aa816 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x4f816) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)
    #3 0x555bcd193be8 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x38be8) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)
    #4 0x555bcd1996fa in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x3e6fa) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)
    #5 0x555bcd1c3c92 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x68c92) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)
    #6 0x7f090257a082 in __libc_start_main /build/glibc-e2p3jK/glibc-2.31/csu/../csu/libc-start.c:308:16
    #7 0x555bcd18e96d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x3396d) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309)

Address 0x7f0900701020 is located in stack of thread T0 at offset 4128 in frame
    #0 0x555bcd2a3d97 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco.cpp:5

  This frame has 4 object(s):
    [32, 4128) 'out' (line 9) <== Memory access at offset 4128 overflows this variable
    [4256, 8352) 'orig' (line 9)
```

Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68211
2024-04-22 18:39:23 +02:00
Nardi Ivan f494bdd653 Fix parameters check
```
==17==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000546050 bp 0x7fff113c82a0 sp 0x7fff113c7a58 T0)
==17==The signal is caused by a READ memory access.
==17==Hint: address points to the zero page.
SCARINESS: 10 (null-deref)
    #0 0x546050 in __sanitizer::internal_strlen(char const*) /src/llvm-project/compiler-rt/lib/sanitizer_common/sanitizer_libc.cpp:167:10
    #1 0x4c6ba5 in __interceptor_strrchr /src/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:740:5
    #2 0x5fb9b9 in ndpi_get_host_domain_suffix /src/ndpi/src/lib/ndpi_domains.c:105:20
    #3 0x578058 in LLVMFuzzerTestOneInput /src/ndpi/fuzz/fuzz_config.cpp:503:3
```

Found while fuzzing
2024-04-21 12:29:09 +02:00
0x41CEA55 246e972f9c Replace my personal email with my corporate one in all my contributions (#2399) 2024-04-20 21:37:01 +02:00
Nardi Ivan 95f4d0b852 Fix a warning
```
nDPI/PcDebug64/src/include/ndpi_api.h:1970:3: error: function declaration isn’t a prototype [-Werror=strict-prototypes]
```
2024-04-20 19:30:42 +02:00
Ivan Nardi ef89183469 fuzz: improvements (#2400)
Create the zip file with all the traces only once.

Add a new fuzzer to test "shoco" compression algorithm
2024-04-20 18:15:23 +02:00
0x41CEA55 905120588b Remove obsolete protocols: tuenty, tvuplayer and kontiki (#2398) 2024-04-19 21:35:32 +02:00
0x41CEA55 1b2e2cd968 Add strlcpy implementation (#2395) 2024-04-19 17:16:40 +02:00
0x41CEA55 e75d7a620e Add KNXnet/IP protocol support (#2397)
* Add KNXnet/IP protocol support

* Improve KNXnet/IP over TCP detection
2024-04-19 12:54:00 +02:00
Luca Deri ad117bfaab Domain Classification Improvements (#2396)
* Added
size_t ndpi_compress_str(const char * in, size_t len, char * out, size_t bufsize);
size_t ndpi_decompress_str(const char * in, size_t len, char * out, size_t bufsize);

used to compress short strings such as domain names. This code is based on
https://github.com/Ed-von-Schleck/shoco

* Major code rewrite for ndpi_hash and ndpi_domain_classify

* Improvements to make sure custom categories are loaded and enabled

* Fixed string encoding

* Extended SalesForce/Cloudflare domains list
2024-04-18 23:21:40 +02:00
Luca Deri 108b8331d5 Invalid initialization 2024-04-15 18:13:17 +02:00
Luca Deri 95897c6436 Fixed minor glitches 2024-04-15 14:25:26 +02:00
Ivan Nardi d97afd6efd STUN: fix attributes list iteration (#2391)
We need to check all the attributes, to look for any possible metadata
2024-04-13 18:43:41 +02:00
Ivan Nardi 06f4814085 STUN: try to stop extra dissection earlier, if possible (#2390) 2024-04-13 14:45:17 +02:00
Ivan Nardi 8edb2f133c STUN: add support for ipv6 in some metadata (#2389) 2024-04-13 14:12:20 +02:00
Ivan Nardi f51dfc6973 STUN: simplify ip/port parsing (#2388)
Add other 2 configuration options
2024-04-13 12:00:04 +02:00
Ivan Nardi 0535e54484 STUN: fix boundary checks on attribute list parsing (#2387)
Restore all unit tests.
Add some configuration knobs.
Fix the endianess.
2024-04-12 22:55:51 +02:00
Luca Deri bb4fccd2cf Added comment 2024-04-12 20:29:14 +02:00
Luca Deri b83eb7c7a2 Implemented STUN peer_address, relayed_address, response_origin, other_address parsing
Added code to ignore invalid STUN realm
Extended JSON output with STUN information
2024-04-12 19:50:04 +02:00
Toni Uhlig 31de94ba12 fix invalid read
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-12 17:55:48 +02:00
Vladimir Gavrilov c63697205b Add Label Distribution Protocol support (#2385)
* Add Label Distribution Protocol support

* Fix typo

* Update unit test results
2024-04-12 17:44:36 +02:00
Ivan Nardi 93f02ee507 Fix ndpi_reconcile_msteams_udp (#2377)
Microsoft UDP traffic over port ~3478 is voip traffic, using some kind
of proprietary STUN-like protocol: so use the most specific protocol id.

More important, we definitely want `Stun/Skype_TeamsCall` and not
`Stun/Skype_Teams`
2024-04-12 17:05:28 +02:00
Toni Uhlig 65fef09ff2 Updated unit test results
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-12 16:52:00 +02:00
Toni Uhlig 082b00ede7 Updated unit test results
* fixed invalid read

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-12 16:38:44 +02:00
Luca Deri a8507c3308 STUN:
- Fixed issue with XOR-MAPPED-ADDRESS decoding
- Implemented MAPPED_ADDRESS (IPv4 only)
2024-04-12 15:49:42 +02:00
Nardi Ivan 17965c8852 Update unit tests results 2024-04-11 18:36:43 +02:00
Luca Deri fd2838e9a1 Minor code cleanup 2024-04-11 17:44:44 +02:00
Toni 39a5d234fd Serialize amount of found TLS blocks and signature algos. (#2384)
* partially fixes #2282

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-11 16:21:41 +02:00
Luca Deri 8b4fdbc0fb Renamed ndpi_fill_ip6_protocol_category in ndpi_fill_ipv6_protocol_category 2024-04-11 12:37:39 +02:00
Vladimir Gavrilov 9ff4bece33 Add The Elder Scrolls Online support (#2376)
* Add The Elder Scrolls Online support

* Use ndpi_memmem instead of memmem from libc

* Add protocol description

* Change selection bitmask to V4_V6

* Update protocols.rst
2024-04-10 18:04:02 +02:00
Vladimir Gavrilov 142c8f5afb Add memmem() implementation (#2378)
* Add memmem() implementation

* Fix build

* Add fix to avoid too many memcmp calls
2024-04-10 15:39:08 +02:00
Toni 1d0be6c4f4 Add Shellscript risk detection. (#2375)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-10 15:36:54 +02:00
Ivan Nardi 54517d8e04 Fix a warning and restore a unit test result (#2379) 2024-04-10 09:56:06 +02:00
Luca Deri bd2968dcd3 Tuned DNS risk values
Modified NDPI_BINARY_TRANSFER_ATTEMPT in NDPI_BINARY_DATA_TRANSFER
2024-04-09 10:22:01 +02:00
Luca Deri 00a360f30a Disabled "known proto on non standard port" for FTP_DATA 2024-04-09 00:01:47 +02:00
Luca Deri 98bf0e243e Fixed false positives on binary application transfer risk 2024-04-08 23:52:27 +02:00
Luca Deri 48b0f8e2c1 Typo 2024-04-08 18:45:32 +02:00
Ivan Nardi 1b3ef7d7b2 STUN: improve extraction of Mapped-Address metadata (#2370)
Enable parsing of Mapped-Address attribute for all STUN flows: that
means that STUN classification might require more packets.

Add a configuration knob to enable/disable this feature.

Note that we can have (any) STUN metadata also for flows *not*
classified as STUN (because of DTLS).

Add support for ipv6.

Restore the correct extra dissection logic for Telegram flows.
2024-04-08 10:24:51 +02:00
Ivan Nardi f5905a62c7 Fix invalid memory access (#2374)
The bug is triggered when `pe_offset == (u_int32_t)-1`

```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==23719==ERROR: AddressSanitizer: SEGV on unknown address 0x5081000002b3 (pc 0x55c69274ac72 bp 0x7ffffffc8e70 sp 0x7ffffffc8cc0 T0)
==23719==The signal is caused by a READ memory access.
    #0 0x55c69274ac72 in ndpi_search_portable_executable /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8191:7
    #1 0x55c69271606b in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8596:5
    #2 0x55c69270f58f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8629:22
    #3 0x55c6926a07e7 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:24:5
    #4 0x55c6925a79b6 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x64e9b6) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
    #5 0x55c692590d48 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x637d48) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
    #6 0x55c69259685a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x63d85a) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
    #7 0x55c6925c0e02 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x667e02) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
    #8 0x7f8e99793082 in __libc_start_main /build/glibc-wuryBv/glibc-2.31/csu/../csu/libc-start.c:308:16
    #9 0x55c69258baed in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x632aed) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b)
```

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67881
2024-04-06 20:23:06 +02:00
Toni a5d45253c4 Add ELF risk detection (detect transmitted linux executables). (#2373)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-06 19:32:51 +02:00
Toni 727e72d1f1 Calculate packet entropy for unknown protocols. (#2369)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-06 17:01:19 +02:00
Vladimir Gavrilov b535033674 Get rid of Apache Kafka false positives (#2372) 2024-04-06 16:59:38 +02:00
Ivan Nardi 700637a162 fuzzing: extend fuzzing coverage (#2371) 2024-04-05 21:02:54 +02:00
Toni 0f77f49b77 Add PE32/PE32+ risk detection (detect transmitted windows executables). (#2312)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-05 14:43:28 +02:00
Toni 99e521eaf8 Limit git commit hash to 7 characters. (#2368)
* ensure consistency across all platforms

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-05 11:04:22 +02:00
Toni 09bb383437 Fixed yet another another "unused function" warning. (#2367)
* added `-Wextra` to the CI

```
In file included from ndpi_bitmap64_fuse.c:31:
./third_party/include/binaryfusefilter.h:31:24: error: unused function 'binary_fuse_rotl64' [-Werror,-Wunused-function]
static inline uint64_t binary_fuse_rotl64(uint64_t n, unsigned int c) {

..snip..
```

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 20:41:45 +02:00
RoboSchmied ead9aa72da Fix: 3 typos (#2366)
Signed-off-by: RoboSchmied <github@roboschmie.de>
2024-04-04 20:06:47 +02:00
Toni df29e12f5e Fixed yet another "unused function" warning. (#2365)
* seems like clang on `ubuntu-latest` warns about unused static inlined functions

```
In file included from ndpi_bitmap.c:41:
./third_party/include/roaring.h:422:19: error: unused function 'roaring_leading_zeroes' [-Werror,-Wunused-function]
static inline int roaring_leading_zeroes(unsigned long long input_num) {

..snip..
```

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 18:18:44 +02:00
Ivan Nardi c0d3f8a92e STUN: rework sub-classification (#2361)
The main goal is to have the "real" application (if any; i.e.
Signal/Whatsapp/Telegram/...) always as "application" protocol and not
as "master" one
2024-04-04 18:16:40 +02:00
Toni 6e61368cd6 Add -Wno-unused-function as CFLAG for third_party/src/gcrypt_light.c. (#2364)
* fixes failing nDPI build from an external project with clang and `-Wextra`

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 13:51:27 +02:00
Toni Uhlig dface63157 Fixed flow struct static assertion size check (this time with struct padding in mind).
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 12:40:09 +02:00
Toni Uhlig 6917acffda Fixed flow struct static assertion size check.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-04 12:36:42 +02:00
Luca Deri 9185c2ccc4 Added support for STUN Mapped IP address 2024-04-03 23:03:46 +02:00
Nardi Ivan aacbc9e918 Update unit tests results 2024-04-03 16:18:24 +02:00
Luca 225ff7f8a4 Added binary data transfer risk alert 2024-04-03 15:41:26 +02:00
Ivan Nardi 1acc9ee396 Patrica tree: fix handling of mac addresses (#2362)
Found looking at the fuzzer statistics...
2024-04-03 14:10:37 +02:00
Toni 41eef9246c Disable -Wno-unused-parameter -Wno-unused-function. (#2358)
* unused parameters and functions pollute the code and decrease readability

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-04-03 14:10:21 +02:00
Vitaly Lavrov b5e8bc2214 Fixed gcc version checking. (#2360)
Using the "test" utility instead of "[".
2024-04-02 10:58:55 +02:00
Vladimir Gavrilov 599cc0f4b8 Improve LoL: Wild Rift detection (#2359) 2024-03-28 16:01:01 +01:00
Vladimir Gavrilov 5b32c98a21 Add LoL: Wild Rift detection (#2356) 2024-03-26 08:11:14 +01:00
Luca Deri 51f5fc7140 Added support for roaring bitmap v3 (#2355)
* Integrated RoaringBitmap v3

* Renamed ndpi_bitmap64 ro ndpi_bitmap64_fuse

* Fixes to ndpi_bitmap for new roaring library

* Fixes for bitmap serialization

* Fixed format

* Warning fix

* Conversion fix

* Warning fix

* Added check for roaring v3 support

* Updated file name

* Updated path

* Uses clang-9 (instead of clang-7) for builds

* Fixed fuzz_ds_bitmap64_fuse

* Fixes nDPI printf handling

* Disabled printf

* Yet another printf fix

* Cleaup

* Fx for compiling on older platforms

* Fixes for old compilers

* Initialization changes

* Added compiler check

* Fixes for old compilers

* Inline function is not static inline

* Added missing include
2024-03-25 08:15:19 +01:00
Vladimir Gavrilov be0fd7444f Add more TencentGames signatures (#2354) 2024-03-21 14:38:47 +01:00
Vladimir Gavrilov 27f9ca9d04 Improve TencentGames detection (#2353)
* Improve TencentGames detection

* Add more signatures
2024-03-20 12:13:32 +01:00
Nardi Ivan 15a80527c6 STUN: remove workaround to identify RTP traffic
We are able to demultiplex RTP packets in STUN flows since 3608ab01b, at
least; no need to explicity call the RTP dissector
2024-03-20 09:39:15 +01:00
Nardi Ivan 5ccf41739b STUN: fix fallback when DTL dissection fails 2024-03-20 09:39:15 +01:00
Nardi Ivan f758f3605a STUN: dissection of TURN Channel Data 2024-03-20 09:39:15 +01:00
Nardi Ivan ee87c4bc66 STUN: add dissection of XOR-PEER-ADDRESS with ipv6 address 2024-03-20 09:39:15 +01:00
Nardi Ivan 58894dd788 STUN: improve heurstic to detect old classic-stun 2024-03-20 09:39:15 +01:00
Ivan Nardi dabf29ea61 STUN: update demultiplexing logic according to RFC9443 (#2349) 2024-03-19 09:11:51 +01:00
Vladimir Gavrilov e6474d835f Add FLUTE protocol dissector (#2351)
* Add FLUTE protocol dissector

* Add flute.c to MSVC project
2024-03-19 09:11:04 +01:00
Toni 15f61e7abe Enable USE_GLOBAL_CONTEXT via CFLAGS passed to the CC. (#2348)
* `ndpi_typedefs.h`: requires to include `ndpi_config.h` for the `HAVE_STRUCT_TIMESPEC` check
   That will never happen, because `USE_GLOBAL_CONTEXT` is defined inside `ndpi_config.h`.
   It's better to use `CFLAGS` to achieve the same.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-03-15 12:47:13 +01:00
Ivan Nardi 231748bb0e LRU cache: move to 64 bits long keys (#2346)
Tradeoff between key comparison efficiency (i.e. no `memcmp`) and key
length.
At least in the ipv4 cases, we have no more different entries with the
same key.
2024-03-15 10:12:51 +01:00
Ivan Nardi 97fae6e00a STUN: fix parsing of DATA attribute (#2345) 2024-03-14 14:20:51 +01:00
Ivan Nardi 37ddf02e63 bitmap64: fix memory access error (#2344)
```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==29723==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x562910b70ddb bp 0x7ffcb22c5b70 sp 0x7ffcb22c5a80 T0)
==29723==The signal is caused by a READ memory access.
==29723==Hint: address points to the zero page.
    #0 0x562910b70ddb in binary_fuse16_contain /home/ivan/svnrepos/nDPI/src/lib/./third_party/include/binaryfusefilter.h:492:8
    #1 0x562910b70bbe in ndpi_bitmap64_isset /home/ivan/svnrepos/nDPI/src/lib/ndpi_bitmap64.c:178:10
    #2 0x562910788fd3 in ndpi_domain_classify_longest_prefix /home/ivan/svnrepos/nDPI/src/lib/ndpi_domain_classify.c:261:5
    #3 0x56291078940e in ndpi_domain_classify_contains /home/ivan/svnrepos/nDPI/src/lib/ndpi_domain_classify.c:291:9
    #4 0x56291069a392 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ds_domain_classify.cpp:52:5
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67369
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67372
2024-03-14 14:20:35 +01:00
Ivan Nardi 25c145f26d CI actions: fix Ubuntu jobs with sanitizers (#2347)
See: https://github.com/actions/runner-images/issues/9491
2024-03-14 14:20:21 +01:00
Vladimir Gavrilov 8fad77991d Add PFCP protocol dissector (#2342) 2024-03-13 20:18:43 +01:00
Ivan Nardi 4102c8df72 CI: fix build on MacOS-13 runners (#2343)
Workaroud for Homebrew's python link error
See: https://github.com/Homebrew/homebrew-core/issues/165793#issuecomment-1991817938
2024-03-13 19:22:13 +01:00
Luca Deri 1aedbefffa Fixes bitmap memory calculation 2024-03-11 17:59:24 +01:00
Ivan Nardi 6152d595e8 STUN: add a parameter to configure how long the extra dissection lasts (#2336)
Tradeoff: performance (i.e. number of packets) vs sub-classification
2024-03-07 14:39:32 +01:00
Ivan Nardi 56ce228a8b Add a specific protocol id for audio/video calls made using Google apps (#2341)
Same logic already used for Signal/Whatsapp/Line/Facebook/...
2024-03-07 13:48:19 +01:00
Ivan Nardi c09b4dacdf Fix memory leak (#2340)
```
Direct leak of 12 byte(s) in 1 object(s) allocated from:
    #0 0x55779e1a46ff in malloc (/home/ivan/svnrepos/nDPI/example/ndpiReader+0x8706ff) (BuildId: 14c2fc626744710d49d652ea1c5bbb24a8cbab4f)
    #1 0x55779e2120c7 in ndpi_malloc_wrapper /home/ivan/svnrepos/nDPI/example/ndpiReader.c:298:10
    #2 0x55779e5fa215 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
    #3 0x55779e5fa500 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13
    #4 0x55779e42153c in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2554:46
    #5 0x55779e4359a1 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:908:5
    #6 0x55779e432de7 in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1097:2
    #7 0x55779e4133f9 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2913:5
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67250
2024-03-07 13:11:13 +01:00
Toni 1fb4719e59 Disable AX_PTHREAD for MingW/MSYS builds. (#2338)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-03-07 13:09:01 +01:00
Vladimir Gavrilov e2949048e0 Add Path of Exile protocol dissector (#2337)
* Add Path of Exile protocol dissector

* Update protocols.rst
2024-03-06 19:59:09 +01:00
Ivan Nardi 21da53d3a0 ahocorasick: improve matching with subdomains (#2331)
The basic idea is to have the following logic:
* pattern "DOMAIN" matches the domain itself (i.e exact match) *and* any
subdomains (i.e. "ANYTHING.DOMAIN")
* pattern "DOMAIN." matches *also* any strings for which is a prefix
[please, note that this kind of match is handy but it is quite
dangerous...]
* pattern "-DOMAIN" matches *also* any strings for which is a postfix

Examples:
* pattern "wikipedia.it":
  * "wikipiedia.it" -> OK
  * "foo.wikipedia.it -> OK
  * "foowikipedia.it -> NO MATCH
  * "wikipedia.it.com -> NO MATCH
* pattern "wikipedia.":
  * "wikipedia.it" -> OK
  * "foo.wikipedia.it -> OK
  * "foowikipedia.it -> NO MATCH
  * "wikipedia.it.com -> OK
* pattern "-wikipedia.it":
  * "wikipedia.it" -> NO MATCH
  * "foo.wikipedia.it -> NO MATCH
  * "0001-wikipedia.it -> OK
  * "foo.0001-wikipedia.it -> OK

Bottom line:
* exact match
* prefix with "." (always, implicit)
* prefix with "-" (only if esplicitly set)
* postfix with "." (only if esplicitly set)

That means that the patterns cannot start with '.' anymore.

Close #2330
2024-03-06 19:25:59 +01:00
Luca Deri 8f63a11735 Extended connectivity checks 2024-03-05 17:45:46 +01:00
Luca Deri 65b9ad88a9 Improved alert on suspicious DNS traffic 2024-03-05 11:14:34 +01:00
Vladimir Gavrilov e7f1946ae7 Add NetEase Games detection support (#2335) 2024-03-05 11:00:52 +01:00
Vladimir Gavrilov 58fdc9fafb Add Naraka Bladepoint detection support (#2334) 2024-03-04 08:30:54 +01:00
Luca Deri 70aae6e998 Added DGA exception for Dropbox 2024-03-03 22:14:34 +01:00
Luca Deri e0587fe75e indent 2024-03-03 22:12:13 +01:00
Vladimir Gavrilov f2e3c7fb90 Add BFD protocol dissector (#2332) 2024-02-29 08:19:00 +01:00
Ivan Nardi c2b5b48fc8 ndpiReader: restore ndpiReader -x $DOMAIN_NAME functionality (#2329) 2024-02-26 20:50:07 +01:00
Nardi Ivan b287d6ec85 TLS: avoid setting some flow risks for webrtc traffic
Is quite rare to have a SNI or an ALPN on Client Hello of STUN/DTLS/SRTP
traffic
2024-02-26 10:20:05 +01:00
Nardi Ivan 478c31b6c9 utils: update script to download Cloudflare ips 2024-02-26 09:26:21 +01:00
Nardi Ivan ed5ba179f6 Telegram: improve identification
Follow up of 31c706c3db and
75485e177c.

Allow fast classification by ip, but give time to other dissectors to
kick in (for example, the TLS code for the Telegram Web flows).

Even if we don't classify it anymore at the very first packet (i.e. SYN)
we fully classify Telegram traffic at the first packet with payload, as
*any* other protocol.
This way, we always have the proper category, the proper confidence
for the UDP flows and we don't overwrite previous classifications (TLS
or ICMP)

Remove old and stale identification logic for TCP flows
2024-02-26 09:26:21 +01:00
Ivan Nardi c95e8c184e STUN: fix category when sub-classification is set in "extra-dissection" data path (#2320) 2024-02-24 09:44:24 +01:00
Luca Deri 71b7afccf5 Updated telegam out 2024-02-23 16:22:13 +01:00
Luca Deri 75485e177c Improved Telegram detection 2024-02-23 16:15:29 +01:00
Luca Deri bfb21dc09b Fixes exception handling glitch 2024-02-22 23:19:01 +01:00
Luca Deri 31c706c3db Improved telegram detection 2024-02-22 22:23:23 +01:00
Luca Deri 2399abd272 Added missing telegram networks 2024-02-22 22:04:54 +01:00
Vladimir Gavrilov 66b6e2b3f2 Add DLEP protocol dissector (#2326) 2024-02-20 16:05:41 +01:00
Ivan Nardi 4f2e642fbe make install: avoid copying private header (#2323) 2024-02-20 15:39:25 +01:00
Ivan Nardi a1280a2fd1 Move some defines (expecially log related) to the private header (#2324) 2024-02-20 15:35:38 +01:00
Ivan Nardi 12e142565e Add a script to download/update the domain suffix list (#2321) 2024-02-20 11:51:58 +01:00
Ivan Nardi 2f814c526b Add identification of Huawei generic and cloud traffic (#2325) 2024-02-20 11:51:40 +01:00
Ivan Nardi ffd6c3edee TLS: fix disabling of JA3C fingerprint (#2319) 2024-02-19 11:11:57 +01:00
Luca Deri 23536ff77a Improved modbus dissection to discard false positives 2024-02-16 15:48:34 +01:00
Luca Deri d1aed7b391 Indentation 2024-02-16 15:31:01 +01:00
Vladimir Gavrilov e93bcfd619 Add ANSI C12.22 protocol dissector (#2317)
* Add ANSI C12.22 protocol dissector

* Add UDP sample
2024-02-15 09:36:06 +01:00
Ivan Nardi ae36648c6c Skype: remove old detection logic (#1954)
Skype has been using standard protocols (STUN/ICE or TLS) for a long,
long time, now. Long gone are the days of Skype as a distribuited
protocol.

See: #2166
2024-02-12 18:39:24 +01:00
Nardi Ivan 0d36d9c8c2 Remove spurious call to exit() 2024-02-12 17:10:45 +01:00
Luca Deri 4072cb8862 Added stress test 2024-02-11 22:35:33 +01:00
Toni f4d7aa45fe Improved Polish gambling sites fetch script. (#2315)
* fails quite often in the CI, so ignore potential xmllint error

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-10 13:33:56 +01:00
Ivan Nardi ad25affcb7 reader_util: fix GRE detunneling (#2314) 2024-02-10 09:16:27 +01:00
Ivan Nardi 31f0608b9a TLS: add configuration of JA* fingerprints (#2313) 2024-02-10 09:16:13 +01:00
Ivan Nardi 03ecb026ff fuzz: improve fuzzing coverage (#2309) 2024-02-09 19:19:03 +01:00
Vladimir Gavrilov f5cec001f3 Add detection of Gaijin Entertainment games (#2311)
* Add detection of Gaijin Entertainment games

* Short NDPI_PROTOCOL_GAIJINENTERTAINMENT to NDPI_PROTOCOL_GAIJIN

* Add default UDP port for Gaijin Entertainment games

* Remove NDPI_PROTOCOL_CROSSOUT protocol id
2024-02-09 11:07:48 +01:00
Ivan Nardi f352e4f77b Improve normalization of flow->host_server_name (#2310)
Follow-up of 4543385d10

Remove trailing spaces for any HTTP header (we already remove leading
spaces)

We want:
* a "normalized" string in `flow->host_server_name`, but
* to parse the original string for flow risk checking

`ndpi_hostname_sni_set()` is a private function, so there is no need to
export its flags.
2024-02-09 09:20:02 +01:00
Vladimir Gavrilov 2abb68c8ca Add new AppsFlyer domain (#2307) 2024-02-08 08:10:55 +01:00
Vladimir Gavrilov 6207be43fa Add TencentGames protocol dissector (#2306) 2024-02-08 08:10:35 +01:00
Vitaly Lavrov 4543385d10 Normalization of host_server_name (#2299)
* Normalization of host_server_name

The ndpi_hostname_sni_set() function replaces all non-printable
characters with the "?" character and removing whitespace characters
at the end of the line.

* Added conditional hostname normalization.
2024-02-05 19:44:45 +01:00
Luca Deri a4da4467ac Fixed incompatibity with RH7 introduced by https://github.com/ntop/nDPI/commit/02030ac16e5016b00e0da6ff7512d97751bf87d2 2024-02-05 18:12:42 +01:00
Luca Deri b86452f2b1 Removed un-necessary include that caused configure-based applications sitting on top of nDPI to faile at compilation 2024-02-05 17:01:21 +01:00
Ivan Nardi 66371c202f Fix ndpi_get_lru_cache_stats() (#2303)
Found while fuzzing
2024-02-05 12:38:33 +01:00
Toni f2b09014f9 Serialize JA4 to JSON. (#2305)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-05 12:35:38 +01:00
Toni 02030ac16e Build RPM package in the CI. (#2304)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-05 12:13:09 +01:00
Luca Deri b15b877c51 Minor hash improvement 2024-02-04 20:23:49 +01:00
Toni b59994fa20 Completly disable all pthread related code in the library if USE_GLOBAL_CONTEXT macro is not defined. (#2302)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-03 11:53:49 +01:00
Ivan Nardi 9af2b9d815 collectd: fix extraction of hostname (#2301) 2024-02-03 04:11:58 +01:00
Toni ede25cc2b3 Improve ndpi_set_config error printing. (#2300)
* exit `ndpiReader` if a invalid configuration setting detected

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-02 15:15:30 +01:00
Toni f8aa642c3c Simplify and fix JA4 string computation. (#2298)
* additional JA4 string buffer is not needed and may cause a string truncation warning

```
protocols/tls.c: In function ‘ndpi_compute_ja4’:
protocols/tls.c:1738:3: warning: ‘strncpy’ output may be truncated copying 36 bytes from a string of length 1023 [-Wstringop-truncation]
 1738 |   strncpy(flow->protos.tls_quic.ja4_client, ja_str, 36);
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-02-02 14:25:43 +01:00
Ivan Nardi 13212711ab CIP: fix infinite-loop (#2295)
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=66342
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=66320
2024-02-01 20:30:43 +01:00
Ivan Nardi 8fbef7fb5a GitHub Actions: update to latest macOS runners (#2293)
Add a simple job with macos-14 on M1.
https://github.blog/changelog/2024-01-30-github-actions-introducing-the-new-m1-macos-runner-available-to-open-source/)
There are some issues with external dependencies (they are installed but
autoconf script doens;t find them) so keep it simple.

On macos-13 it seems that:
* there is no `realpath` program (even if coreutils has been
installed...)
* most of the filesystem is read only (we can't write on /usr/lib).
So I change
```
make install DESTDIR=$(realpath _install)
ls -alhHR _install
```
to
```
DESTDIR=/tmp/ndpi make install
ls -alhHR /tmp/ndpi
```
for all the jobs

Fix a warning on GitHub logs:
```
Node.js 16 actions are deprecated. Please update the following actions
to use Node.js 20: actions/checkout@v3. For more information see:
https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
```
2024-02-01 20:30:27 +01:00
Vladimir Gavrilov 4c9446379f Add Gearman protocol dissector (#2297) 2024-02-01 19:46:57 +01:00
Ivan Nardi 6c5ecae607 Fix function parameters check on domain code (#2296)
Found while fuzzing fuzz_config
2024-02-01 17:06:16 +01:00
Nardi Ivan c78ed93ec3 Sync unit tests results 2024-02-01 15:45:36 +01:00
Ivan Nardi 400cd516b5 Allow multiple struct ndpi_detection_module_struct to share some state (#2271)
Add the concept of "global context".

Right now every instance of `struct ndpi_detection_module_struct` (we
will call it "local context" in this description) is completely
independent from each other. This provide optimal performances in
multithreaded environment, where we pin each local context to a thread,
and each thread to a specific CPU core: we don't have any data shared
across the cores.

Each local context has, internally, also some information correlating
**different** flows; something like:
```
if flow1 (PeerA <-> Peer B) is PROTOCOL_X; then
  flow2 (PeerC <-> PeerD) will be PROTOCOL_Y
```
To get optimal classification results, both flow1 and flow2 must be
processed by the same local context. This is not an issue at all in the far
most common scenario where there is only one local context, but it might
be impractical in some more complex scenarios.

Create the concept of "global context": multiple local contexts can use
the same global context and share some data (structures) using it.
This way the data correlating multiple flows can be read/write from
different local contexts.
This is an optional feature, disabled by default.

Obviously data structures shared in a global context must be thread safe.
This PR updates the code of the LRU implementation to be, optionally,
thread safe.

Right now, only the LRU caches can be shared; the other main structures
(trees and automas) are basically read-only: there is little sense in
sharing them. Furthermore, these structures don't have any information
correlating multiple flows.

Every LRU cache can be shared, independently from the others, via
`ndpi_set_config(ndpi_struct, NULL, "lru.$CACHE_NAME.scope", "1")`.

It's up to the user to find the right trade-off between performances
(i.e. without shared data) and classification results (i.e. with some
shared data among the local contexts), depending on the specific traffic
patterns and on the algorithms used to balance the flows across the
threads/cores/local contexts.

Add some basic examples of library initialization in
`doc/library_initialization.md`.

This code needs libpthread as external dependency. It shouldn't be a big
issue; however a configure flag has been added to disable global context
support. A new CI job has been added to test it.

TODO: we should need to find a proper way to add some tests on
multithreaded enviroment... not an easy task...

*** API changes ***

If you are not interested in this feature, simply add a NULL parameter to
any `ndpi_init_detection_module()` calls.
2024-02-01 15:33:11 +01:00
Toni 44c2e59661 Provide a u64 wrapper for ndpi_set_config() (#2292)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-30 17:13:35 +01:00
Toni Uhlig d2f22d1308 Updated results (missing cip_io.pcap.out)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-29 20:49:47 +01:00
Luca Deri 47f72443fa Implemented CIP I/O (UDP version of the CIP protocol), Common Industrial protocol 2024-01-29 13:28:41 +01:00
Luca Deri 6c08601c3d Updated result 2024-01-29 12:27:52 +01:00
Luca Deri 426494b4ba Fixes invalid skype detection on UDP 2024-01-29 12:27:52 +01:00
Luca Deri 14cfa174be Added public_suffix_list.dat in RPM package 2024-01-29 11:48:39 +01:00
Luca Deri fb58f838e5 Added public_suffix_list.dat in packages 2024-01-29 11:43:23 +01:00
Luca Deri ae501e2503 Enhanced exception lists for binary application transfer 2024-01-29 11:37:04 +01:00
Ivan Nardi 92c2ac5a0f fuzz: fuzz_config: try restoring good coverage (#2291)
Last changes reduce fuzzing coverage of this fuzzer :(
2024-01-29 10:53:28 +01:00
Vladimir Gavrilov fb095a339d Add missing NDPI_LOG_DBG in some dissectors (#2290)
* Add missing NDPI_LOG_DBG

* Add missing NDPI_LOG_DBG in yojimbo.c
2024-01-29 09:25:29 +01:00
Vladimir Gavrilov c807d84054 Fix RESP detection (#2289)
* Rename redis_net.c to resp.c

* Fix RESP detection
2024-01-27 21:19:34 +01:00
Luca Deri 8e562c4ecd Removed check fir skipping .arpa domains 2024-01-27 20:54:46 +01:00
Luca Deri ec83a592b8 Check DGA on domain name (as it should be) rather than on the host name 2024-01-27 20:46:00 +01:00
Luca Deri 7a7df2b7d7 Fixed warning 2024-01-27 20:45:52 +01:00
Luca Deri 65b9c68d7d Fixed loading of non-ICANN domains that caused false positives with ndpi_load_domain_suffixes
Minor hash optimization
2024-01-27 20:40:27 +01:00
Toni bcca89b78c Add Raft protocol dissector. (#2286)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-25 20:26:18 +01:00
Vladimir Gavrilov 9a0affc6aa Improve MySQL detection (#2279)
* Improve MySQL detection

* Update copyright
2024-01-25 11:12:08 +01:00
Ivan Nardi b3efa7d3fe fuzz: fuzz_config: we need bigegr inputs (#2285) 2024-01-25 09:58:20 +01:00
Vladimir Gavrilov 4e712e3ab5 Add Radmin protocol dissector (#2283)
* Add Radmin protocol dissector

* Update test results
2024-01-25 08:10:29 +01:00
Ivan Nardi d577508727 fuzz: extend fuzzing coverage (#2281) 2024-01-24 21:16:58 +01:00
Ivan Nardi 7a83a8dc91 QUIC: fix decryption with CH fragments with different Destination CID (#2278)
QUIC decryption fails when the Client Hello is split into multiple UDP
packets and these packets have different Destination Connection IDs
(because the server told the client to switch to a different CID; see
RFC 9000 7.2)

```
The Destination Connection ID field from the first Initial packet sent by
a client is used to determine packet protection keys for Initial packets.
[..]
Upon first receiving an Initial or Retry packet from the server, the
client uses the Source Connection ID supplied by the server as the
Destination Connection ID for subsequent packets

```

From a logical point of view, the ciphers used for decryption should be
initialized only once, with the first Initial pkt sent by the client and
kept for later usage with the following packets (if any).

However it seems that we can safely initialize them at each packet, if
we keep using the DCID of the **first** packet sent by the client.
Keep initializing the ciphers at each packet greatly simplifie this patch.

This issue has been undetected for so long because:
* in the vast majority of the cases we only decrypt one packet per flow;
* the available traces with the Client Hello split into multiple packets
(i.e. cases where we need to decrypt at least two packets per flow) were
created in a simple test environment to simulate Post-Quantum handshake,
and in that scenario the client sent all the packets (with the same
DCID) before any reply from the server.

However, in the last months all major browsers started supporting PQ
key, so it is now common to have split CH in real traffic.

Please note that in the attached example, the CH is split into 2
(in-order) fragments (in different UDP packets) and the second one in
turn is divided into 9 (out-of-order) CRYPTO frames; the reassembler
code works out-of-the-box even in this (new) scenario.
2024-01-24 09:57:28 +01:00
Vladimir Gavrilov f04b4450a1 Add STOMP protocol dissector (#2280) 2024-01-23 21:08:13 +01:00
Ivan Nardi 9b26e74bb7 example: rework code between ndpiReader.c and reader_util.c (#2273) 2024-01-22 18:12:06 +01:00
Ivan Nardi 82e8bf91dd Improve handling of custom rules (#2276)
Avoid collisions between user-ids and internal-ids protocols in the
`example/protos.txt` file.
Add a new value for the classification confidence:
`NDPI_CONFIDENCE_CUSTOM_RULE`

With `./example/ndpiReader -p example/protos.txt -H` we now see also the
custom protocols and their internal/external ids:

```
nDPI supported protocols:
 Id Userd-id Protocol               Layer_4    Nw_Proto Breed        Category
  0        0 Unknown                TCP        X        Unrated      Unspecified

...

387      387 Mumble                 UDP        X        Fun          VoIP
388      388 iSCSI                  TCP                 Acceptable   Unspecified
389      389 Kibana                 TCP                 Acceptable   Unspecified
390      390 TestProto              TCP                 Acceptable   Unspecified
391      391 HomeRouter             TCP                 Acceptable   Unspecified
392      392 CustomProtocol         TCP                 Acceptable   Unspecified
393      393 AmazonPrime            TCP                 Acceptable   Unspecified
394      394 CustomProtocolA        TCP                 Acceptable   Unspecified
395      395 CustomProtocolB        TCP                 Acceptable   Unspecified
396      800 CustomProtocolC        TCP                 Acceptable   Unspecified
397     1024 CustomProtocolD        TCP                 Acceptable   Unspecified
398     2048 CustomProtocolE        TCP                 Acceptable   Unspecified
399     2049 CustomProtocolF        TCP                 Acceptable   Unspecified
400     2050 CustomProtocolG        TCP                 Acceptable   Unspecified
401    65535 CustomProtocolH        TCP                 Acceptable   Unspecified
```

We likely need to take a better look in general at the iteration between
internal and external protocols ids...

This PR fixes the issue observed in
https://github.com/ntop/nDPI/pull/2274#discussion_r1460674874 and in
https://github.com/ntop/nDPI/pull/2275.
2024-01-21 19:53:32 +01:00
Vladimir Gavrilov 5620e10742 Add ElectronicArts detection support (#2274)
* Add ElectronicArts detection support

* Merge electronicarts.pcapng into sites.pcapng
2024-01-21 18:58:12 +01:00
Toni 7d24e1258d Add Yojimbo (netcode) protocol dissector (#2277)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-21 17:47:42 +01:00
Ivan Nardi 49591dda9b Zoom: improve detection (#2270) 2024-01-21 12:17:34 +01:00
Ivan Nardi 42d23cff6a config: follow-up (#2268)
Some changes in the parameters names.
Add a fuzzer to fuzz the configuration file format.
Add the infrastructure to configuratin callbacks.
Add an helper to map LRU cache indexes to names.
2024-01-20 16:14:41 +01:00
Vladimir Gavrilov 8651ce9811 Fix detection of new Cassandra versions (#2272)
* Fix detection of new Cassandra versions

* Add Cassandra Internode Communication protocol support

* Add default port for Cassandra Internode Communication protocol
2024-01-20 15:58:14 +01:00
Ivan Nardi eb129297e9 Add a dedicated dissector for Zoom (#2265)
Move it from the RTP code and extend it
2024-01-19 10:01:38 +01:00
Ivan Nardi c83698c957 STUN: fix flow risks when DTLS packets are found (#2266)
When switching to (D)TLS dissector from the STUN one, we need to clear
any flow risks set from the latter (because we don't have anymore
`NDPI_PROTOCOL_STUN` in the classification results)
2024-01-19 08:57:39 +01:00
Vladimir Gavrilov 6ac2ce84f8 Add Mumble detection support (#2269) 2024-01-19 07:46:51 +01:00
Vladimir Gavrilov 248f3d5588 Rework Steam detection (part 1) (#2264)
* Clean up Steam dissector

* Add Steam Datagram Relay dissector

* Update docs

* Update test results

* Remove csgo.c from MSVC project

* Small fixes

* Add Steam TLS pcap sample

* Merge Steam pcap samples into single one

* Fix typo

* Update test results
2024-01-18 19:23:44 +01:00
Nardi Ivan 0712d496fe config: allow configuration of guessing algorithms 2024-01-18 10:21:24 +01:00
Nardi Ivan 6c85f10cd5 config: move debug/log configuration to the new API 2024-01-18 10:21:24 +01:00
Nardi Ivan c704be1a20 config: DNS: add two configuration options
* Enable/disable sub-classification of DNS flows
* Enable/disable processing of DNS responses
2024-01-18 10:21:24 +01:00
Nardi Ivan 950f209a17 config: HTTP: enable/disable processing of HTTP responses 2024-01-18 10:21:24 +01:00
Nardi Ivan c669044a44 config: configure TLS certificate expiration with the new API 2024-01-18 10:21:24 +01:00
Nardi Ivan 88720331ae config: remove enum ndpi_prefs 2024-01-18 10:21:24 +01:00
Nardi Ivan 1289951b32 config: remove ndpi_set_detection_preferences() 2024-01-18 10:21:24 +01:00
Nardi Ivan 311d8b6dae config: move cfg of aggressiviness and opportunistic TLS to the new API 2024-01-18 10:21:24 +01:00
Nardi Ivan 4cbe2674ab config: move IP lists configurations to the new API 2024-01-18 10:21:24 +01:00
Nardi Ivan f55358973f config: move LRU cache configurations to the new API 2024-01-18 10:21:24 +01:00
Nardi Ivan 3107a95881 Make ndpi_finalize_initialization() returns an error code
We should check if the initialization was fine or not
2024-01-18 10:21:24 +01:00
Nardi Ivan d72a760ac3 New API for library configuration
This is the first step into providing (more) configuration options in nDPI.

The idea is to have a simple way to configure (most of) nDPI: only one
function (`ndpi_set_config()`) to set any configuration parameters
(in the present or on in the future) and we try to keep this function
prototype as agnostic as possible.

You can configure the library:
* via API, using `ndpi_set_config()`
* via a configuration file, in a text format

This way, anytime we need to add a new configuration parameter:
* we don't need to add two public functions (a getter and a setter)
* we don't break API/ABI compatibility of the library; even changing
the parameter type (from integer to a list of integer, for example)
doesn't break the compatibility.

The complete list of configuration options is provided in
`doc/configuration_parameters.md`.

As a first example, two configuration knobs are provided:
* the ability to enable/disable the extraction of the sha1 fingerprint of
the TLS certificates.
* the upper limit on the number of packets per flow that will be subject
to inspection
2024-01-18 10:21:24 +01:00
Luca ca7df1db82 Improved ndpi_get_host_domain 2024-01-16 07:25:03 +01:00
Luca 1637a991a4 Added ndpi_get_host_domain() for returning the host domain
vs ndpi_get_host_domain_prefix() that instead returnd the host TLD
2024-01-16 06:56:51 +01:00
Ivan Nardi 111015b872 ndpiReader: improve the check on max number of pkts processed per flow (#2261)
Allow to disable this check.

I don't know how much sense these limits have in the application
(especially with those default values...) since we have always had a
hard limit on the library itself (`max_packets_to_process` set to 32).
The only value might be that they provide different limits for TCP and
UDP traffic.

Keep them for the time being...
2024-01-15 20:12:57 +01:00
Nardi Ivan 0891ad0e77 Added missing lists file to the distribution tarball 2024-01-15 20:00:36 +01:00
Nardi Ivan b22fa558ff ndpiReader: fix memory leak
Change the working directory of `ndpiReader` in the Github Actions so
that it can load the domain suffix list during `domainsUnitTest()`
2024-01-15 19:49:27 +01:00
Luca d29c2f3ad1 Added ndpi tracing 2024-01-15 19:26:28 +01:00
Luca a1634604e1 Removes extraneous parentheses that caused macOS to complain 2024-01-15 19:12:31 +01:00
Luca 162c38f18f Added new API calls
- ndpi_load_domain_suffixes()
- ndpi_get_host_domain_suffix()

whose goal is to find the domain name of a hostname. Example:

www.bbc.co.uk   -> co.uk
mail.apple.com  -> com
2024-01-15 19:03:46 +01:00
Ivan Nardi 61a18623e7 Bittorrent: improve detection of UTPv1 (#2259)
* fix header length check: we must take into account optional extension
header
* it seems the check on UTPv1 header is weak only in the DATA case: wait
for multiple packets only in that case
2024-01-12 18:53:24 +01:00
Ivan Nardi dd8be1fcb1 Fix some warnings reported by CODESonar (#2227)
Remove some unreached/duplicated code.

Add error checking for `atoi()` calls.

About `isdigit()` and similar functions. The warning reported is:
```
Negative Character Value help
isdigit() is invoked here with an argument of signed type char, but only
has defined behavior for int arguments that are either representable
as unsigned char or equal to the value of macro EOF(-1).
Casting the argument to unsigned char will avoid the undefined behavior.
In a number of libc implementations, isdigit() is implemented using lookup
tables (arrays): passing in a negative value can result in a read underrun.
```
Switching to our macros fix that.
Add a check to `check_symbols.sh` to avoid using the original functions
from libc.
2024-01-12 13:30:43 +01:00
Toni 0aea509e23 Add KCP protocol dissector. (#2257)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-12 12:14:59 +01:00
Vladimir Gavrilov 75d3f78f7d Add more tracker/ad hosts (#2256) 2024-01-12 10:44:31 +01:00
Toni 7b2bbb2309 Improved uTorrent via utp (TCP-like streams over UDP). (#2255)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-11 16:49:25 +01:00
Luca Deri 30c2386ee2 Added missing (C) 2024-01-09 22:27:41 +01:00
Luca Deri 4f9a08cf05 Moved ndpi_private.h to include directory 2024-01-09 22:00:40 +01:00
Vladimir Gavrilov 3d57dec6b4 Add PIA (Private Internet Access) support (#2250)
* Fix typo in protocols.rst

* Add PIA (Private Internet Access) support

* Update ndpi_main.c
2024-01-09 18:28:57 +01:00
Vladimir Gavrilov ead5bf2adb Add more adult content hostnames (#2247) 2024-01-09 07:57:01 +01:00
Toni c5b0b05b80 Add Roughtime protocol dissector. (#2248)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-09 07:56:27 +01:00
Toni 6c3d162cd6 Add realtime protocol output to ndpiReader. (#2197)
* support for using a new flow callback invoked before the flow memory is free'd
 * minor fixes

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-09 00:39:59 +01:00
Ivan Nardi 90b999e3f1 Fix default port conflicts between AJP and CiscoVPN (#2245)
We shouldn't have the same default port associated with multiple
protocols. The key reason is that we use only one of them (for
classification by-port, for example) and which one we really use
depends on the internal order of the protocolsi's initialization (which
is unknown to the user).

We have 3 port collisions:
* 8009: AJP and CiscoVPN
* 445: SMBv1 and SMBv23
* 8080: HTTP_Connect and HTTP_Proxy
I think that the two last cases don't really matter...

About the first one: it seems that CiscoVPN is retired from 2011/2014
(see
https://www.cisco.com/c/en/us/obsolete/security/cisco-vpn-client.html)
and superseded by AnyConnect, which uses different ports (see
https://community.cisco.com/t5/network-security/what-protocols-are-used-on-anyconnect-mobility-client/td-p/4713525).
Therefore it should be safe to remove 8009 from the CiscoVPN default
ports.
2024-01-08 17:04:46 +01:00
Ivan Nardi 8321b79539 Make some test traces smaller (#2243)
Having smaller traces help fuzzing: we want the fuzzers to mutate
"interesting" packets analyzed by nDPI, i.e. the first packets of each
flows.
Try hard to keep the same classification and extraction capabilities
2024-01-08 17:04:12 +01:00
Vladimir Gavrilov ce08291ccd Add Google Chat support (#2244) 2024-01-07 18:04:59 +01:00
Ivan Nardi 40797521af ndpiReader: add breed stats on output used for CI (#2236) 2024-01-05 13:02:39 +01:00
Vladimir Gavrilov 3d09b25653 Add Ceph protocol dissector (#2242)
* Add Ceph protocol dissector

* Update protocols.rst
2024-01-04 13:22:23 +01:00
Ivan Nardi f23e9dc7bb Add an implementation of the BSD function strtonum (#2238)
The main difference with the original function is that we allow to
specify the base.
Credit for the original idea and the first implementation to @0xA50C1A1
2024-01-04 13:16:39 +01:00
Vladimir Gavrilov 7f9973bd0c Add HL7 protocol dissector (#2240)
* Add HL7 protocol dissector

* Small fixes

* Small fixes
2024-01-02 20:57:05 +01:00
Ivan Nardi 88d1416b70 STUN: fix detection of Google Meet over IPv6 (#2241) 2024-01-02 19:30:59 +01:00
Vladimir GavrilovandNardi Ivan 0180c1f04a Add IEC62056 (DLMS/COSEM) protocol dissector (#2229)
* Add IEC62056 (DLMS/COSEM) protocol dissector

* Fix detection on big endian architectures

* Update protocols.rst

* Add ndpi_crc16_x25 to fuzz/fuzz_alg_crc32_md5.c

* Update pcap sample

* Remove empty .out file

* iec62056: add some documentation

---------

Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2024-01-02 16:45:54 +01:00
Ivan Nardi 3c7ed34ce9 fuzz: improve fuzzing coverage (#2239) 2024-01-02 15:22:44 +01:00
Vladimir Gavrilov 0f4d9f5054 Remove Google Hangouts/Duo stuff (#2233)
* Remove Google Hangouts/Duo support

* Update protocols.rst
2024-01-02 14:01:33 +01:00
Ivan Nardi 3dbd89eee8 sha256: fix undefined-shift error (#2237)
```
third_party/src/ndpi_sha256.c:51:21: runtime error: left shift of 128 by 24 places cannot be represented in type 'int'
```

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65305
2024-01-02 12:29:20 +01:00
Ivan Nardi 308f71a6e8 TLS: fix heap-buffer-overflow (#2235)
```
==53992==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x511000007e04 at pc 0x555da2165fd0 bp 0x7ffddf7e3990 sp 0x7ffddf7e3988
READ of size 2 at 0x511000007e04 thread T0
    #0 0x555da2165fcf in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2384:50
    #1 0x555da217c31f in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:908:5
    #2 0x555da2176720 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1273:11
    #3 0x555da215a628 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2883:5
    #4 0x555da1e95c30 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6720:6
    #5 0x555da1e969f3 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6756:10
    #6 0x555da1e96394 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6789:12
    #7 0x555da1ea7991 in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7929:15
    #8 0x555da1ea547f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8104:22
    #9 0x555da1de137f in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1721:31
    #10 0x555da1de137f in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2438:1
```

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65362
2024-01-02 11:37:40 +01:00
Ivan Nardi d886a6107f Teamviewer: varius fixes (#2228)
We already have a generic (and up to date) logic to handle ip addresses:
remove that stale list.

Teamviewer uses TCP and UDP, both; we can't access `flow->l4.udp`.

According to a comment, we set the flow risk
`NDPI_DESKTOP_OR_FILE_SHARING_SESSION` only for the UDP flows.
2024-01-02 11:22:43 +01:00
Vladimir Gavrilov 2796bc9b47 Add NoMachine NX protocol dissector (#2234)
* Add NoMachine protocol dissector

* Fix detection on big endian architectures

* Make NoMachine over UDP check more strict

* Small fixes
2024-01-02 10:23:42 +01:00
Luca 2f657cb8f9 Implemented ndpi_is_outlier() for detecting outliers using z-score 2023-12-28 19:59:54 +01:00
Luca Deri 1366518bff Implements ndpi_pearson_correlation for measuring how correlated are two series 2023-12-27 22:42:37 +01:00
Luca Deri 99d4838328 New ndpi_sha256() nDPI API call (#2230)
* Added ndpi_sha256.c to the Windows project

* Added ndpi_sha256() nDPI API call
2023-12-23 10:47:28 +01:00
Luca Deri f2be972224 Removed type redefinition 2023-12-23 09:16:35 +01:00
Luca Deri caa223d2ba Changes to avoid type redefinition on windows 2023-12-23 09:15:42 +01:00
Luca Deri 5c7200f2bb Buffer overflow fix 2023-12-23 08:07:28 +01:00
Luca Deri 7c4be6d077 Updated result 2023-12-22 21:29:45 +01:00
Luca Deri 53b9ae23e8 Warning fix 2023-12-22 21:10:03 +01:00
Luca Deri cbea1b2203 Fixed check (when using ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --with-thread-sanitizer --with-pcre2 --with-maxminddb --enable-tls-sigs) 2023-12-22 21:01:14 +01:00
Luca Deri a95d4744b6 Fixes nDPI datastructs size check 2023-12-22 20:52:56 +01:00
Luca Deri 8285fffdae Implements JA4 Support (#2191) 2023-12-22 20:40:42 +01:00
Luca Deri b90c18e906 Changes to make room for JA4 2023-12-22 20:40:42 +01:00
Vladimir Gavrilov 5eb468d07b Add Apache Kafka protocol dissector (#2226) 2023-12-22 14:42:47 +01:00
Vladimir Gavrilov 6fc8aa4e61 Add WebDAV detection support (#2224)
* Add WebDAV detection support

* Add pcap example

* Update test results

* Remove redundant checks

* Add WebDAV related HTTP methods to fuzz/dictionary.dict

* Add note about WebDAV
2023-12-22 13:23:37 +01:00
Luca 41aa2f96b6 Various MDNS flow risks fixes 2023-12-21 12:24:30 +01:00
Luca 28b1ce41cb MDNS fix 2023-12-21 11:11:28 +01:00
Ivan Nardi a5595d16c0 CI: update list of compilers (#2223)
Try using latest gcc and clang versions.
We still care about RHEL7: since handling a RHEL7 runner on GitHub is
quite complex, let try to use a similar version of gcc, at least
2023-12-20 19:22:22 +01:00
Vladimir Gavrilov 149067b3fc Add JSON-RPC protocol dissector (#2217)
* Add JSON-RPC protocol dissector

* Small fixes

* Improve detection
2023-12-20 12:42:25 +01:00
Vladimir Gavrilov 33f11cb10f Add OpenFlow protocol dissector (#2222) 2023-12-20 10:48:45 +01:00
Ivan Nardi 8aa09f9c99 mining: a better identification logic (#2221)
It is quite simple (and not so efficient) but it should fix all the
false positives reported in #2216. Add support for Ethereum mining.

Merge all the mining traces.

Remove duplicated function.

Close #2216
2023-12-20 10:46:57 +01:00
Ivan Nardi 308b266333 fuzz: improve fuzzing coverage (#2220) 2023-12-19 20:33:08 +01:00
Vladimir Gavrilov 6af93c09c8 Rename NDPI_PROTOCOL_RPC to NDPI_PROTOCOL_MS_RPCH (#2218)
* Rename NDPI_PROTOCOL_RPC to NDPI_PROTOCOL_MS_RPCH

* Add protocol description

* Improve MS-RPCH detection
2023-12-19 11:56:26 +01:00
Vladimir Gavrilov 59c8eabc0e Add UFTP protocol dissector (#2215)
* Add UFTP protocol dissector

* Update docs

* Merge pcap files
2023-12-18 11:21:07 +01:00
Vladimir Gavrilov d8c7a76611 Add HiSLIP protocol dissector (#2214)
* Add HiSLIP protocol dissector

* Fix error
2023-12-17 11:52:55 +01:00
Vladimir Gavrilov 0f3e6d832b Add PROFINET/IO protocol dissector (#2213)
* Add PROFINET/IO protocol dissector

* Add LE (Little Endian) to the file name

* Rework dissector

* Remove redundant check
2023-12-16 13:30:21 +01:00
Ivan Nardi 8ea16ffd77 TLS: be sure to always set a category value (#2206)
We probably need to check everywhere we set the category in the
"extra dissection" data path: a flow with a proper classification can't
have NDPI_PROTOCOL_CATEGORY_UNSPECIFIED as category.
2023-12-14 08:07:04 +01:00
Toni ef62391dba Add Monero protocol classification. (#2196)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-12-13 19:55:18 +01:00
Ivan Nardi bda037c57f Sync unit tests results after latest merge (#2211) 2023-12-13 17:47:20 +01:00
Ivan Nardi 193f28582b QUIC: add heuristic to detect unidirectional *G*QUIC flows (#2207)
Fix extraction of `flow->protos.tls_quic.quic_version` metadata.
2023-12-13 17:14:04 +01:00
Ivan Nardi 8e14aac5e0 ndpiReader: avoid creating two detection modules when processing traffic/traces (#2209) 2023-12-12 19:44:29 +01:00
Ivan Nardi 241c42ad7e ndpiReader: fix guessed_flow_protocols statistic (#2203)
Increment the counter only if the flow has been guessed
2023-12-12 19:44:03 +01:00
Ivan Nardi 7b5354588b fuzz: extend fuzzing coverage (#2208) 2023-12-11 19:24:17 +01:00
Ivan Nardi b3f2b1bb7f STUN: rework extra dissection (#2202)
Keep looking for RTP packets but remove the monitoring concept.
We will re-introduce a more general concept of "flow in monitoring
state" later.
The function was disabled by default.
Some configuration knobs will be provided when/if #2190 is merged.
2023-12-11 14:53:12 +01:00
Ivan Nardi 673b6e7345 HTTP: faster processing of asymmetric flows (#2198) 2023-12-11 14:52:31 +01:00
Ivan Nardi adf8982d8e fuzz: extend fuzzing coverage (#2205) 2023-12-11 12:48:50 +01:00
rl1987 59d476195c Fix typos (#2204)
* Fix typo in ndpiSimpleIntegration.c

* Fix misspelling in a comment
2023-12-10 19:58:22 +01:00
Vladimir Gavrilov c809e7c069 Replace complicated TPKT header validation with an helper function (#2201)
* Replace complicated TPKT header validation with an helper function

* Move tpkt_verify_hdr function definition to ndpi_utils.c
2023-12-10 12:10:50 +01:00
Ivan Nardi f74cf16c36 OpenVPN: rework detection (#2199)
Close #1873
2023-12-06 10:24:26 +01:00
Vladimir Gavrilov ad20846fad Add Ether-S-Bus protocol dissector (#2200) 2023-12-05 17:20:38 +01:00
Vladimir Gavrilov 72601a69ff Add some fast CRC16 algorithms implementation (#2195)
* Add some fast CRC16 algorithms implementation

* Update ndpi_crc.c

* Move crc16 stuff to ndpi_analyze.c

* IEEE C37.118: use new fast CRC-16/CCITT-FALSE implementation
2023-12-05 15:58:11 +01:00
Vladimir Gavrilov be50493f44 Add IEEE C37.118 protocol dissector (#2193) 2023-12-05 08:06:15 +01:00
Ivan Nardi 7b0c16a70d TLS: remove JA3+ fingerprints. (#2192)
See: #2191
2023-12-05 08:05:44 +01:00
Vladimir Gavrilov c34bded4ef Add ISO 9506-1 MMS protocol dissector (#2189)
* Add ISO 9506-1 MMS protocol dissector
* Fix detection on big-endian architectures
2023-12-01 09:03:07 +01:00
Ivan Nardi 983b8e8eee STUN: parsing of DATA attribute (#2179) 2023-12-01 07:01:49 +01:00
Vladimir Gavrilov 24df1913ac Add Beckhoff ADS protocol dissector (#2181)
* Add Beckhoff ADS protocol dissector

* Remove redundant le32toh

* Fix detection on big-endian architectures
2023-11-30 09:13:45 +01:00
Ivan Nardi 6f046df0dc STUN: fix detection of DTLS (#2187)
Fix a memory leak
```
==97697==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x55a6967cfa7e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x701a7e) (BuildId: c7124999fa1ccc54346fa7bd536d8eab88c3ea01)
    #1 0x55a696972ab5 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
    #2 0x55a696972da0 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13
    #3 0x55a696b7658d in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2394:46
    #4 0x55a696b86e81 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:897:5
    #5 0x55a696b80649 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1262:11
    #6 0x55a696b67a57 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2751:5
    #7 0x55a696b67758 in switch_to_tls /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1408:3
    #8 0x55a696c47810 in stun_search_again /home/ivan/svnrepos/nDPI/src/lib/protocols/stun.c:422:4
    #9 0x55a6968a22af in ndpi_process_extra_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7247:9
    #10 0x55a6968acd6f in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7746:5
    #11 0x55a6968aba3f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8013:22
    #12 0x55a69683d30e in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1723:31
    #13 0x55a69683d30e in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2440:10
    #14 0x55a69680f08f in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:135:7
[...]
SUMMARY: AddressSanitizer: 16 byte(s) leaked in 1 allocation(s).
```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64564
2023-11-30 09:09:40 +01:00
Ivan Nardi d85f8592b8 Rework extraction of HTTP headers (#2183)
Extract only the headers that we really need/use.
Avoid too many `strcmp`; the algorithm might be a little bit
overwhelming right now but it might be useful if we have further headers
in the future.
2023-11-29 19:44:25 +01:00
Ivan Nardi 1100326edb Fix FAQ (#2186) 2023-11-29 18:56:21 +01:00
Ivan Nardi d3d9770b96 Update FAQ.md 2023-11-29 18:52:35 +01:00
Ivan Nardi cc721fd3ce Update FAQ.md 2023-11-29 18:28:26 +01:00
Ivan Nardi 0114992f5f Add a FAQ for the project (#2185) 2023-11-29 18:25:36 +01:00
Ivan Nardi 3b35cb37d9 Keep separating public and private API (#2157)
See: b08c787fe
2023-11-29 17:13:00 +01:00
Ivan Nardi ac90b1f009 Fix detection of NDPI_TCP_ISSUES flow risk (#2177)
We need to take into account retransmissions: they increase
`flow->all_packets_counter` counter but not `flows->packet_counter`
one.
Therefore, the right way to check for 3WH + RST pattern involves checking
for `flows->packet_counter == 0`
2023-11-29 16:55:39 +01:00
Toni 5fb631c8fe Improved belgium gambling sites regex. (#2184)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-29 16:15:23 +01:00
Vladimir Gavrilov c60c03766c Add Schneider Electric’s UMAS detection support (#2180)
* Add Schneider Electric’s UMAS detection support

* Swap proto IDs in ndpi_set_detected_protocol

* Update unit test result
2023-11-28 18:03:00 +01:00
Vladimir Gavrilov 688a197b85 Fix FINS false positives (#2176)
* Fix FINS false positives

* Add rsv (reserved) field check
2023-11-27 21:46:14 +01:00
Vladimir Gavrilov ebb1bc2f34 Add Ether-S-I/O protocol dissector (#2174) 2023-11-27 19:04:05 +01:00
Vladimir Gavrilov 84427b0754 Add Omron FINS protocol dissector (#2172)
* Add Omron FINS protocol dissector

* Add a kludge to avoid invalid FINS over UDP detection as SkypeTeams and RTP

* Update unit test results

* Update protocols.rst

* Remove dummy flows from fins.pcap
2023-11-27 17:09:53 +01:00
3763c702f0 Rework S7Comm dissector; add S7Comm Plus support (#2165)
* Rework S7Comm dissector; add S7Comm Plus support

* Cleanup s7comm.c

* Improve S7Comm Plus detection

* s7comm/s7commplus: faster detection

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2023-11-27 14:37:48 +01:00
Vladimir Gavrilov 0b6e261523 Improve CORBA detection (#2167)
* Improve CORBA detection

* Remove dummy flow from ziop.pcap

* Merge ziop.pcap and miop.pcap into corba.pcap
2023-11-27 13:10:50 +01:00
Vladimir Gavrilov da629709f3 Add OPC UA protocol dissector (#2169) 2023-11-27 12:13:23 +01:00
Ivan Nardi 7ff22a7e3c STUN: improve demultiplexing of DTLS packets (#2153)
Keep demultiplexing STUN/RTP/RTCP packets after DTLS ones.

We might end up processing the session a little longer, because we will
process the STUN/RTP/RTCP packets after the DTLS handshake.
2023-11-27 11:10:38 +01:00
Vladimir Gavrilov 87399b3544 Add RTPS protocol dissector (#2168) 2023-11-27 07:17:39 +01:00
Vladimir Gavrilovand0xA50C1A1 27802b0134 Reduce false positives for H.323 over TCP (#2164)
Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
2023-11-23 17:29:00 +01:00
fbae51ae9d Get rid of RDP false positives (#2161)
* Get rid of false positives in the RDP protocol dissector

* Remove kludge for RDP

* RDP: improve detection

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2023-11-23 09:35:43 +01:00
Vladimir Gavrilovand0xA50C1A1 5c8c5c90c2 Add HART-IP protocol dissector (#2163)
* Add HART-IP protocol dissector

* Update docs

* Update protocols.rst

* Reuse free proto id and re-run tests

* docs: move HART-IP to top of list

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
2023-11-22 22:04:22 +01:00
Toni 21f2574033 Improved TFTP. Fixes #2075. (#2149)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-21 16:56:46 +01:00
Vladimir Gavrilovand0xA50C1A1 35abafec4f Get rid of Apache Cassandra false positives (#2159)
* Rewrite Apache Cassandra dissector

* Replace memcmp with strncmp

* Add payload length check

* Update Cassandra dissector

* Update test results

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
2023-11-21 16:56:01 +01:00
Vladimir Gavrilovand0xA50C1A1 ae6e6d61f0 Add IEEE 1588-2008 (PTPv2) dissector (#2156)
* Add IEEE 1588-2008 (PTPv2) dissector

PTPv2 is a time synchronization protocol in computer networks, similar to NTP.

* Add default protocol ports

* Update default test result for PTPv2

* Update copyright

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
2023-11-21 13:39:54 +01:00
Ivan Nardi 55664392a9 Ip address list: aggregate Mullvad and Tor lists too (#2154)
Missing from bdb73db1a
See #2150
2023-11-21 08:28:32 +01:00
Vladimir Gavrilovand0xA50C1A1 8d71998670 Remove Google+ support (#2155)
* Remove Google+ support

Google+ was discontiued in 2019, so I think that its protocol id can be freed for reuse.

* Fix typo

* Update tests

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
2023-11-21 08:14:20 +01:00
Ivan Nardi bdb73db1a4 IP lists: aggregate addresses wherever possible (#2152)
See #2150
2023-11-17 12:26:23 +01:00
Ivan Nardi 6c9571d9a9 Remove duplicate addreess list (#2151)
We are loading the same AS list as GOTO
See #2150
2023-11-16 20:08:06 +01:00
Maatuq 344ac69468 protocols/ethereum: enhance node discovery protocol detection. (#2145)
*  packet-type == 0x0 shouldn't be allowed.
* allow more packet-type e.g. ENRRequest, ENRResponse.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
2023-11-15 19:45:48 +01:00
Toni 38f9a74713 Added TeslaServices and improved TikTok host names. Fixes #2140. (#2144)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-10 11:25:10 +01:00
Toni 0cb6f4cb75 Fixed hash buffer size in ndpiSimpleIntegration. (#2143)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-10 10:23:37 +01:00
Ivan Nardi b08c787fe2 Have a clear distinction between public and private/internal API (#2137)
1) Public API/headers in `src/include/` [as it has always been]
2) Private API/headers in `src/lib/`

Try to keep the "ndpi_" prefix only for the public functions
2023-11-09 10:50:59 +01:00
Toni 0673da54b5 Fixed implicit u32 cast in ndpi_data_min() / ndpi_data_max(). (#2139)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-09 10:16:57 +01:00
Ivan Nardi b539b0d090 fuzz: improve coverage and remove dead code (#2135)
We are not able to remove custom rules: remove the empty stubs (which
originate from the original OpenDPI code).

`ndpi_guess_protocol_id()` is only called on the first packet of the
flow, so the bitmask `flow->excluded_protocol_bitmask` is always empty,
since we didn't call any dissectors yet.

Move another hash function to the dedicated source file.
2023-11-07 17:46:29 +01:00
Ivan Nardi 58a9e2d967 Remove struct ndpi_packet_struct from the public API (#2129)
Right now, the only instance of `struct ndpi_packet_struct` is embedded
into `struct ndpi_detection_module_struct`. Since the latter is a
private structure (because of `NDPI_LIB_COMPILATION` ) there is no way for
the application to get a pointer to `ndpi_struct->packet`.

Bottom line: the application can't use any API functions having `struct
ndpi_packet_struct *` as parameter. Remove them all (since they are
completly unused and unusable).

There are no public helper functions to initialize/populate/deinit a
`struct ndpi_packet_struct` object, so the application can't neither
create its own instance of this object.

Protect `struct ndpi_packet_struct` via the same define
`NDPI_LIB_COMPILATION`.
2023-11-07 15:30:56 +01:00
Luca Deri 0db12b1390 In case a pattern is redefined (for instance with protos.txt), the custom definition overwrites the previous one (so far it was ignored) 2023-11-02 20:46:14 +01:00
Toni 6dcecd73d3 Added malicious sites from the polish cert. (#2121)
* added handling of parsing errors

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-11-02 09:04:04 +01:00
Christian Marangi d5c9a16a48 Move from PCRE to PCRE2 (#2134)
Move from PCRE to PCRE2. PCRE is EOL and won't receive any security
updates anymore. Convert to PCRE2 by converting any function PCRE2 new
API.

Also update every entry in github workflows and README to point to the
new configure flag. (--with-pcre2)

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2023-11-01 14:16:49 +01:00
Ivan KapranovandIvan Kapranov 5a2666ce9e Fix proto_name and proto_id missmatch for Sina and SinaWeibo (#2131)
* minor fixes

fixed 'handle leak' in ndpi_load_malicious_sha1_file and removed the redundant comparison ndpi_search_eaq

* fix Stack overflow caused by invalid write in ndpi_automa_match_string_subprotocol

* fix compile errors

* fix

* Fix name missmatch for Sina and Sina Weibo

* fix

* add Sina Weibo to doc

* fix

* add Sina Weibo to doc

---------

Co-authored-by: Ivan Kapranov <i.kapranov@securitycode.ru>
2023-11-01 14:12:49 +01:00
Ivan Nardi cd59485f7e Custom rules: add missing check (#2132)
Fix an use-of-uninitialized-value error

```
==28646==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x55a9b7a3dea2 in ndpi_patricia_lookup /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ndpi_patricia.c:739:8
    #1 0x55a9b7442cac in add_to_ptree /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2588:10
    #2 0x55a9b7469290 in ndpi_add_ip_risk_mask /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4161:12
    #3 0x55a9b746cf14 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4297:11
    #4 0x55a9b74842ae in ndpi_load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4944:8
    #5 0x55a9b7424706 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:38:3
[...]
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=63754
2023-10-31 20:02:37 +01:00
Ivan Nardi 50006516b3 Custom rules: fixes for Windows (#2130)
Even after adding `WSAStartup()` call, the behavior with IPv6 addresses
was different on Windows; it is the same issue reported on:
https://stackoverflow.com/questions/66755681/getaddrinfo-c-on-windows-not-handling-ipv6-correctly-returning-error-code-1

Fix bitmasks.
2023-10-31 16:04:30 +01:00
Luca Deri 773e9b8170 Added pointer check 2023-10-31 15:48:06 +01:00
Luca Deri f49cedc1e8 Added missing freeaddrinfo() call 2023-10-30 22:58:17 +01:00
Luca Deri 5e9f10f820 Replaced gethostbyname2 with getaddrinfo (portable code) 2023-10-30 22:55:50 +01:00
Luca Deri c6af36990e Warning fixes 2023-10-30 13:10:28 +01:00
Ivan Nardi 42d24f8799 STUN: major code rework (#2116)
Try to have a faster classification, on first packet; use standard extra
dissection data path for sub-classification, metadata extraction and
monitoring.

STUN caches:
* use the proper confidence value
* lookup into the caches only once per flow, after having found a proper
STUN classification

Add identification of Telegram VoIP calls.
2023-10-30 10:28:19 +01:00
Ivan Nardi e399bd7e3d Custom rules: fix a stack overflow (#2128)
```
==19255==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f515bb3bf80 at pc 0x55796e01394a bp 0x7fff4fb5c050 sp 0x7fff4fb5b7e0
WRITE of size 58 at 0x7f515bb3bf80 thread T0
    #0 0x55796e013949 in scanf_common(void*, int, bool, char const*, __va_list_tag*) asan_interceptors.cpp.o
    #1 0x55796e0147df in __isoc99_sscanf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x77f7df) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)
    #2 0x55796e0fc74a in ndpi_add_host_ip_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2771:13
    #3 0x55796e0fb029 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4411:16
    #4 0x55796e103738 in ndpi_load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4901:8
    #5 0x55796e0ca96d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:38:3
    #6 0x55796dfd78e0 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x7428e0) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)
    #7 0x55796dfc0e93 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x72be93) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)
    #8 0x55796dfc6d96 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x731d96) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)
    #9 0x55796dff1672 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x75c672) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)
    #10 0x7f515df19082 in __libc_start_main /build/glibc-BHL3KM/glibc-2.31/csu/../csu/libc-start.c:308:16
    #11 0x55796dfbbb0d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x726b0d) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c)

Address 0x7f515bb3bf80 is located in stack of thread T0 at offset 128 in frame
    #0 0x55796e0fb977 in ndpi_add_host_ip_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2703

  This frame has 4 object(s):
    [32, 36) 'pin' (line 2705)
    [48, 64) 'pin6' (line 2706)
    [80, 96) 'd' (line 2769)
    [112, 128) 'tail' (line 2770) <== Memory access at offset 128 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow asan_interceptors.cpp.o in scanf_common(void*, int, bool, char const*, __va_list_tag*)
Shadow bytes around the buggy address:

```
2023-10-30 09:21:57 +01:00
Luca Deri 8769a2282b Disbling AES-NI on FreeBSD as the feature checke does not work well
and it enables it by mistake on IntelB. Coreb" i3-3240 Processor
2023-10-30 00:05:25 +01:00
Luca Deri 9fb7a63576 FreeBSD compilation fix (missingcvg #include <netdb.h>) 2023-10-29 23:18:17 +01:00
Luca Deri 76829b413f Implements support for symbolic host names (#2123) 2023-10-29 22:54:45 +01:00
Luca Deri b8e0602bfd Added check 2023-10-29 22:54:45 +01:00
Ivan Nardi 2c2eaf8e79 Rename some functions with more useful/clear names (#2127) 2023-10-29 20:21:24 +01:00
Ivan Nardi 03fd155ae3 IPv6: add support for custom categories (#2126) 2023-10-29 12:56:44 +01:00
Ivan Nardi 32b50f5aa4 IPv6: add support for IPv6 risk exceptions (#2122) 2023-10-29 12:14:20 +01:00
Ivan Nardi c711251578 IPv6: add support for custom rules (#2120) 2023-10-29 11:26:35 +01:00
Ivan Nardi e8e4b9e8ff IPv6: add support for IPv6 risk tree (#2118)
Fix the script to download crawler addressess
2023-10-27 13:58:15 +02:00
Toni ed17f4d658 Improved Protobuf dissector. (#2119)
* tag extraction/validation was done wrong

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-27 10:19:47 +02:00
Ivan Nardi 8b07be4b9f Jabber: remove support for UDP (#2115)
Jabber/XMPP is only over TCP (even the name `ndpi_search_jabber_tcp`
suggests that...).

Bug introduced in 5266c726f
2023-10-26 20:16:27 +02:00
Ivan Nardi 611c3b66f0 ipv6: add support for ipv6 addresses lists (#2113) 2023-10-26 20:15:44 +02:00
Luca Deri 1832d247b3 Tool for creating bitcoing IP files 2023-10-26 00:01:44 +02:00
Toni 9509cd507a Fixed endian issue while DEBUG_PROTOBUF is enabled. (#2112)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-25 13:11:17 +02:00
Maatuq 4a8e7105b2 add ethereum protocol dissector. (#2111)
as explained here for bitcoin https://www.ntop.org/guides/nDPI/protocols.html#ndpi-protocol-bitcoin
the same is applicable for ethereum.
ethereum detection was removed from mining protocol and is now handled separately.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
2023-10-25 12:44:33 +02:00
Toni e70333de87 Added generic Google Protobuf dissector. (#2109)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-24 12:18:31 +02:00
Luca Deri 71d768d036 Improved detection as non DGA for hostnames belnging to a CDN (#2068) 2023-10-23 17:49:50 +02:00
Toni Uhlig a443bba0dd Add CAN over Ethernet dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-23 13:45:56 +02:00
Toni Uhlig 25c54dd6d7 Improved CryNetwork protocol dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-23 13:44:34 +02:00
Luca Deri 7122599287 Added CLA section 2023-10-23 12:05:56 +02:00
Luca Deri 23759c6d2d Moved to 4.9 2023-10-23 11:46:04 +02:00
Luca Deri 2a99e16728 Added 4.8 changeling 2023-10-23 11:38:46 +02:00
Toni Uhlig f69909d49b Add Remote Management Control Protocol (RMCP).
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-19 19:50:57 +02:00
Alfredo Cardigliano c59fe0b2b5 PR template 2023-10-19 09:29:38 +02:00
Toni 01f384f7ff Improved Steam detection by adding steamdiscover pattern. (#2105)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-17 22:13:02 +02:00
Toni 32b0aeea59 Fixed OpenWRT arm related build issues. (#2104)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-16 19:49:50 +02:00
Toni d75a8d339a Fixed two OpenWRT arm related build issues. (#2103)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-16 18:14:28 +02:00
Luca Deri d9b0ba1d14 Windows warning checks 2023-10-16 16:17:41 +02:00
Nardi Ivan 16b4913be6 fuzz: extend fuzzing coverage 2023-10-15 12:00:26 +02:00
Luca Deri b6179467eb Added NDPI_MALWARE_HOST_CONTACTED flow risk 2023-10-13 00:01:55 +02:00
Luca 30fc89e556 Fix for buffer overflow in serialization 2023-10-11 17:14:17 +02:00
lns 304747f1fa Improved MGCP detection by allowing '\r' as line feed.
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-11 16:44:26 +02:00
Nardi Ivan 4a0eda69ad QUIC: export QUIC version as metadata 2023-10-11 15:15:20 +02:00
Luca Deri a76b9d151f Serialization fix 2023-10-11 00:14:27 +02:00
Nardi Ivan 1366d94156 fuzzing: extend fuzzing coverage
Try fuzzing some functions which write to file/file descriptor; to avoid
slowing the fuzzer, close its stdout
2023-10-09 15:41:46 +02:00
Nardi Ivan 86115a8a65 fuzz: extend fuzzing coverage 2023-10-07 13:34:37 +02:00
Luca Deri 8ca842edec version of dirent.c that is liked by both VC++ and MinGW 2023-10-05 14:35:41 +02:00
Luca Deri e1e6a01d6a Windows code rework 2023-10-05 12:24:39 +02:00
Luca Deri 3881d162f7 Windows compilation fixes 2023-10-05 11:05:14 +02:00
Luca Deri 36112a645b Win include change 2023-10-05 10:47:15 +02:00
Toni a98d7ff433 Added HAProxy protocol. (#2088)
* fixed tests/do.sh.in failure print

Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-02 18:10:47 +02:00
Luca 77e5daf03e Cleaned up mining datastructure 2023-09-27 17:05:12 +02:00
Toni ef3adb9830 Added printf/fprintf replacement for some internal modules. (#1974)
* logging is instead redirected to `ndpi_debug_printf`

Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-09-26 23:10:57 +02:00
Luca Deri 725fcf4852 Renamed HTTP/2 to HTTP2 as the '/' can have side effects with applications sitting on top of nDPI 2023-09-20 11:17:09 +02:00
Ivan Nardi d4e0b79dd6 Fix some prototypes (#2085)
```
error: function declaration isn’t a prototype [-Werror=strict-prototypes]
```
2023-09-18 14:06:47 +02:00
Ivan Nardi 6925890383 Add support for (un-encrypted) HTTP/2 (#2087)
Plaintext HTTP/2 is quite rare on the general "internet" but it is
used in some private networks (example: 5G core network)
2023-09-18 14:06:09 +02:00
Nardi Ivan 70814002a9 fuzz: extend fuzzing coverage 2023-09-16 11:26:11 +02:00
Luca 0828dff3ac Language fix 2023-09-15 12:56:29 +02:00
Ivan Nardi c5cd4df056 Add ndpi_domain_classify_finalize() function (#2084)
The "domain classify" data structure is immutable, since it uses
"bitmap64".
Allow to finalize it before starting to process packets (i.e. before
calling `ndpi_domain_classify_contains()`) to avoid, in the data-path,
all the memory allocations due to compression.

Calling `ndpi_domain_classify_finalize()` is optional.
2023-09-12 19:47:14 +02:00
Thomas Winter 5bcf3c2ddd tftp: update pcap results
The two malformed TFTP packets are no longer considered as risk
and instead match by port only.
This is because the TFTP detection was rather sparse so could
match on several other protocols if the first two opcode bytes
happened to match.
2023-09-12 13:12:14 +02:00
Thomas Winter a5f0cb7e6c tftp: check incrementation for DATA and ACK packets
The 2 bytes following the opcode for DATA and ACK packets are the
block number and this should be incrementing every packet.
We should check to see that this is occurring otherwise false matches
can occur, eg L2TPv3 over UDP matches the DATA opcode but the next two
bytes are always zero.

Remove the DATA max block size assumption since this can be false if
the blksize option is used to increase it.

Fixes #2070
2023-09-12 13:12:14 +02:00
Thomas Winter 7263562514 tftp: check for Option Acknowledgements
TFTP Option Acknowledgement packets were being excluded.
When a read/write request contains options, an Option Acknowledgement
is returned that contains the option strings that the transaction
will use.
The options sent in the request are not compared with what was
acknowledged.
2023-09-12 13:12:14 +02:00
Thomas Winter 1e9517ab74 tftp: rework request checking to account for options
Read and write requests were not handling options.
The existing code inspecting request messages assumed that the string
before the end of the payload was the mode and that the filename
length depended on the mode length.
However the first two strings are the filename and mode which can be
followed by any number of option strings.

Rework the checking of the filename and mode to just search the first
two strings which should always exist. Any options after that are
ignored.

Absence of the filename or mode is now excludes the TFTP protocol.
Absence of the filename no longer is considered malformed packet
because it can match other protocols falsely.
2023-09-12 13:12:14 +02:00
Luca Deri 1bf7e5face Fixes matches with domain name strings that start with a dot 2023-09-11 22:50:03 +02:00
Ivan Nardi 8e6500a0da fuzz: extend fuzzing coverage (#2083) 2023-09-10 21:30:26 +02:00
Ivan Nardi ef6085370f fuzz: add fuzzers to test bitmap64 and domain_classify data structures (#2082) 2023-09-10 18:44:50 +02:00
Ivan Nardi 2c5e22123e Update protocols documentation (#2081) 2023-09-10 15:23:34 +02:00
Ivan Nardi 2a0052f25e fuzz: add fuzzers to test reader_util code (#2080) 2023-09-10 15:07:52 +02:00
Ivan Nardi 0aa1cf7245 Update every ip lists (#2079) 2023-09-10 12:13:20 +02:00
Ivan Nardi 2b883b93be Fix some errors found by fuzzers (#2078)
Fix compilation on Windows.
"dirent.h" file has been taken from https://github.com/tronkko/dirent/

Fix Python bindings

Fix some warnings with x86_64-w64-mingw32-gcc:
```
protocols/dns.c: In function ‘ndpi_search_dns’:
protocols/dns.c:775:41: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
  775 |       unsigned long first_element_len = (unsigned long)dot - (unsigned long)_hostname;
      |                                         ^
protocols/dns.c:775:62: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
  775 |       unsigned long first_element_len = (unsigned long)dot - (unsigned long)_hostname;
      |
```
```
In file included from ndpi_bitmap64.c:31:
third_party/include/binaryfusefilter.h: In function ‘binary_fuse8_hash’:
third_party/include/binaryfusefilter.h:160:32: error: left shift count >= width of type [-Werror=shift-count-overflow]
  160 |     uint64_t hh = hash & ((1UL << 36) - 1);
```
```
In function ‘ndpi_match_custom_category’,
    inlined from ‘ndpi_fill_protocol_category.part.0’ at ndpi_main.c:7056:16:
ndpi_main.c:3419:3: error: ‘strncpy’ specified bound depends on the length of the source argument [-Werror=stringop-overflow=]
 3419 |   strncpy(buf, name, name_len);
```
2023-09-10 11:09:59 +02:00
Luca Deri 805df2e5ce Reworked initialization 2023-09-10 09:36:54 +02:00
Luca Deri 63977459b6 Added OperaVPN detection 2023-09-09 18:17:26 +02:00
Luca Deri 076edeab54 Enhance DNS risk for long hostnames (> 32) 2023-09-09 18:09:57 +02:00
Luca Deri f50a4d7e85 Improved detection of invalid chars in DNS names 2023-09-09 17:46:13 +02:00
Luca Deri 1a797d7b74 Win fix 2023-09-08 12:24:27 +02:00
Luca Deri 770e460ba9 Added NDPI_TLS_ALPN_SNI_MISMATCH flow risk 2023-09-07 23:42:42 +02:00
Luca 47fb5e9f3d Cleanup 2023-09-07 11:12:37 +02:00
Luca Deri 2bd986a00a Disabled debug message 2023-09-05 18:39:26 +02:00
Luca Deri ad9ed4f954 Minor warning fixes 2023-09-05 18:08:16 +02:00
Luca Deri 978df906b3 Improved classification further reducing memory used 2023-09-05 17:03:20 +02:00
Luca Deri 0080d0d092 Compilation fixes for old compilers 2023-09-05 10:43:36 +02:00
Luca Deri feaaf83bed Added sub-domain classification fix 2023-09-05 07:21:48 +02:00
Luca Deri cf67f483f4 Classification fixes 2023-09-05 01:17:02 +02:00
Luca Deri c5ebd84fd8 Added ndpi_bitmap64 support 2023-09-05 00:21:16 +02:00
Luca Deri 4f2ce2d43b Added ndpi_murmur_hash to the nDPI API 2023-09-04 12:53:42 +02:00
Luca Deri f0dc3347ec Merged new and old version of ndpi_domain_classify.c code 2023-09-02 19:27:19 +02:00
Luca Deri 1d480c18e3 Reworked domain classification based on binary filters 2023-09-02 19:16:40 +02:00
Luca Deri 854c2d80f1 Improvement for reducing false positives 2023-09-01 10:26:07 +02:00
Luca Deri 74fdac7d77 Code cleanup 2023-09-01 08:50:55 +02:00
Luca Deri 133f45f3f3 Added ndpi_binary_bitmap datastruture
It is similar to ndpi_filter but based on binary search and with
the ability to store a category per value (as ndpi_domain_classify)
2023-08-31 23:51:50 +02:00
Luca Deri 19381f330a Fixed heap-buffer-overflow issue 2023-08-31 18:35:49 +02:00
Luca Deri 16b0ce3710 Code cleanup 2023-08-31 09:14:17 +02:00
Luca Deri f824939662 Added comment 2023-08-31 09:00:36 +02:00
Luca Deri 3c7b2a714e Added missing lists files 2023-08-29 19:21:57 +02:00
Luca Deri c5c610c941 Fixes dist 2023-08-29 18:49:43 +02:00
Luca Deri 65142b2a53 Added -G flag 2023-08-29 18:32:57 +02:00
Luca Deri 1de7fd9bf0 Warning fixes 2023-08-29 18:08:56 +02:00
Luca Deri 3dbc4f65fc Added README file 2023-08-29 17:45:36 +02:00
Luca Deri 36abf06c6f Swap from Aho-Corasick to an experimental/home-grown algorithm that uses a probabilistic
approach for handling Internet domain names.

For switching back to Aho-Corasick it is necessary to edit
ndpi-typedefs.h and uncomment the line
// #define USE_LEGACY_AHO_CORASICK

[1] With Aho-Corasick
$ ./example/ndpiReader -G ./lists/ -i tests/pcap/ookla.pcap | grep Memory
nDPI Memory statistics:
nDPI Memory (once):      37.34 KB
Flow Memory (per flow):  960 B
Actual Memory:           33.09 MB
Peak Memory:             33.09 MB

[2] With the new algorithm
$ ./example/ndpiReader -G ./lists/ -i tests/pcap/ookla.pcap | grep Memory
nDPI Memory statistics:
nDPI Memory (once):      37.31 KB
Flow Memory (per flow):  960 B
Actual Memory:           7.42 MB
Peak Memory:             7.42 MB

In essence from ~33 MB to ~7 MB

This new algorithm will enable larger lists to be loaded (e.g. top 1M domans
https://s3-us-west-1.amazonaws.com/umbrella-static/index.html)

In ./lists there are file names that are named as <category>_<string>.list
With -G ndpiReader can load all of them at startup
2023-08-29 17:34:04 +02:00
Toni Uhlig 1f693c3f5a Added lists/gambling.list to extra dist.
* make dist in `./fuzz`: fixed inconsistent `*.dict` file pattern

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-27 15:53:31 +02:00
Toni Uhlig afd250f5d9 fix compilation and symbol check
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-27 15:32:42 +02:00
Toni Uhlig 25f3a7cd1d fix compilation
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-27 15:17:56 +02:00
Luca Deri 34986de297 Search fixes 2023-08-26 19:47:50 +02:00
Luca Deri eeeee46b1e Changes for supporinng more efficient sub-string matching 2023-08-26 17:55:50 +02:00
Luca Deri 4ca94369e1 Improved domain search tet unit 2023-08-26 10:43:19 +02:00
Luca Deri ac00542df2 LEak fix 2023-08-26 10:10:49 +02:00
Luca Deri 2c565c77c9 Added ndpi_domain_classify_XXX(0 API 2023-08-26 00:24:33 +02:00
Luca Deri 42fcb011ad Warning fix 2023-08-25 19:29:43 +02:00
Luca Deri eb9932164f added bimap and/or with allocation 2023-08-24 07:45:38 +02:00
Luca Deri ff4f291afe Minor improvements 2023-08-23 11:04:31 +02:00
Luca 95fcd08687 Added ndpi_bitmap_is_empty() and ndpi_bitmap_optimize() API calls 2023-08-23 11:03:59 +02:00
Luca 9933b5910d Boundary check 2023-08-21 23:19:57 +02:00
Luca bd25df3334 Improved Wireguard detection 2023-08-21 13:40:42 +02:00
Luca 9e4bcc7771 Removed empty line 2023-08-21 13:40:30 +02:00
Luca 6e5466f9e5 Added ndpi_bitmap_andnot API call 2023-08-21 13:26:38 +02:00
Ivan Nardi cc4461f424 fuzz: extend coverage (#2073) 2023-08-20 15:18:19 +02:00
Ivan Nardi 5867f43fae Fix compilation on Windows (#2072) 2023-08-14 22:51:09 +02:00
Luca Deri cfbb7a423e Replaces free() with ndpi_free() 2023-08-14 18:51:11 +02:00
Luca Deri 049fbf819c Reworked ndpi_filter_xxx implementation using compressed bitmaps 2023-08-14 12:50:54 +02:00
Luca Deri 3b34941be1 Typo 2023-08-12 11:39:19 +02:00
Luca Deri 9a44954365 ARM compilation fix 2023-08-12 07:39:35 +02:00
Luca Deri fe634b8d55 Fixed to address issue
Run ./utils/check_symbols.sh || { FAILED=$?; echo "::error file=${NDPI_LIB}::Unwanted libc symbols found: ${FAILED}. Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of malloc/calloc/realloc/free."; false; }
[ndpi_filter.o]: calloc
Unwanted symbols found: 1
Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of malloc/calloc/realloc/free
Error: Unwanted libc symbols found: 1. Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of
2023-08-12 00:27:37 +02:00
Luca Deri 8e17f50853 Added ndpi_filter_add_multi() API call 2023-08-11 23:55:44 +02:00
Luca Deri 890dcf91d9 Minor initialization fixes 2023-08-11 23:51:16 +02:00
Luca Deri ec7adc212e Added new API calls for implementing Bloom-filter like data structures
ndpi_filter* ndpi_filter_alloc(uint32_t elements_number);
bool         ndpi_filter_add(ndpi_filter *f, uint64_t value);
bool         ndpi_filter_contains(ndpi_filter *f, uint64_t value);
void         ndpi_filter_free(ndpi_filter *f);
2023-08-11 22:49:31 +02:00
Luca 5fad72ebd9 Typo fix 2023-08-07 10:20:21 +02:00
Luca 33e9dbc24d Added ndpi_bitmap_copy() API call 2023-08-06 09:35:56 +02:00
snicket2100 1fbe8a2385 Mullvad VPN service added (based on entry node IP addresses) (#2062) 2023-08-02 19:44:16 +02:00
Luca Deri 196395398e Compilation fixes for older C compilers 2023-08-01 15:35:33 +02:00
Toni e4d3d619bc Add Service Location Protocol dissector. (#2036)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-01 08:50:46 +02:00
Toni 083275f617 Added slackb.com SNI. (#2067)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-01 08:39:10 +02:00
Ivan Nardi 5019022e13 DNS: extract geolocation information, if available (#2065)
The option NSID (RFC5001) is used by Google DNS to report the
airport code of the metro where the DNS query is handled.

This option is quite rare, but the added overhead in DNS code is pretty
much zero for "normal" DNS traffic
2023-07-31 07:44:43 +02:00
snicket2100 d477da997f Typo in a comment fixed (#2063) 2023-07-28 17:58:25 +02:00
Ivan Nardi bc91192aca ProtonVPN: split the ip list (#2060)
Use two separate lists:
* one for the ingress nodes, which triggers a ProtonVPN classification
* one for the egress nodes, which triggers the
`NDPI_ANONYMOUS_SUBSCRIBER` risk

Add a command line option (to `ndpiReader`) to easily test IP/port
matching.

Add another example of custom rule.
2023-07-27 09:05:22 +02:00
Ivan Nardi 3326fa258e Add an heuristic to detect fully encrypted flows (#2058)
A fully encrypted session is a flow where every bytes of the
payload is encrypted in an attempt to “look like nothing”.
The heuristic needs only the very first packet of the flow.
See: https://www.usenix.org/system/files/sec23fall-prepub-234-wu-mingshi.pdf

A basic, but generic, inplementation of the popcpunt alg has been added
2023-07-26 09:09:12 +02:00
Toni 2b230e28e0 Allow init of app protocols w/o any hostnames set. (#2057) 2023-07-22 14:51:33 +02:00
Toni 88a757c4cd Fix string truncation. (#2056)
ndpi_main.c: In function ‘ndpi_load_ip_category’:
ndpi_main.c:6598:3: warning: ‘strncpy’ specified bound 64 equals destination size [-Wstringop-truncation]
 6598 |   strncpy(ipbuf, ip_address_and_mask, sizeof(ipbuf));
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-07-22 12:45:45 +02:00
Luca Deri 6831187d62 Debian 12 fixes 2023-07-21 12:24:34 -04:00
Ivan Nardi 7a1509bf60 zabbix: improve detection (#2055) 2023-07-21 03:42:36 +02:00
Ivan Nardi c85f2fb0f4 TLS: add basic, basic, detection of Encrypted ClientHello (#2053) 2023-07-21 03:41:43 +02:00
Ivan Nardi fa0bd515b5 Add detection of Roblox games (#2054) 2023-07-21 03:39:40 +02:00
Ivan Nardi 3edfad01a1 fuzz: extend fuzzing coverage (#2052)
Added/merged some traces.
Improved Socks identification
2023-07-18 07:41:56 +02:00
Ivan Nardi 09548bb7cf tests: restore some old paths as symbolic links (#2050) 2023-07-16 13:47:35 +02:00
Ivan Nardi 5267858b3d HTTP: fix another memory access error (#2049)
```
=================================================================
==199079==ERROR: AddressSanitizer: negative-size-param: (size=-1)
    #0 0x559a2a6efd4f in strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x94ad4f) (BuildId: 34aaabba403c6bc5482553ef355360fd2762a157)
    #1 0x559a2a9890f0 in ndpi_http_check_content /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:300:8
    #2 0x559a2a9812c0 in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:910:46
    #3 0x559a2a978fee in process_response /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1289:3
    #4 0x559a2a97622f in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1382:9
    #5 0x559a2a975d95 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1468:3
    #6 0x559a2a864970 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5948:4
    #7 0x559a2a8660df in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6013:12
    #8 0x559a2a865d7f in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6032:12
    #9 0x559a2a876fd6 in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7038:15
    #10 0x559a2a87311f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7205:22
    #11 0x559a2a77381e in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1710:31
    #12 0x559a2a77381e in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2427:10
[...]
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60605
2023-07-15 22:23:52 +02:00
Ivan Nardi 2bbde5bad3 Fix compilation in CI jobs (#2048) 2023-07-15 14:26:51 +02:00
Ivan Nardi 890f17788b ndpireader: fix detection of DoH traffic based on packet distributions (#2045) 2023-07-14 23:20:06 +02:00
Ivan Nardi 5811a5613b Add a configuration knob to enable/disable loading of gambling list (#2047) 2023-07-14 23:19:30 +02:00
Luca Deri 8e960f033d Adds new pcap for testing "funny" HTTP servers 2023-07-14 21:12:11 +02:00
Luca Deri a499edd2f2 Restored files 2023-07-14 21:12:04 +02:00
Luca Deri fea09e825b Fixes risk mask exception handling while improving the overall performance 2023-07-14 19:52:34 +02:00
snicket2100 abee1a2a6f Included Gambling website data from the Polish hazard.mf.gov.pl list (#2041)
* Refreshed the Belgium Gambling Site list data

Unfortunately some hostnames have been removed from that list,
which means they are disappearing from the `ndpi_gambling_match.c.inc`
file as well.

* build: added `libxml2-utils` (for `xmllint`)

* Included Gambling website data from the Polish `hazard.mf.gov.pl` list

The list contains over 30k gambling website hostnames as of today.
2023-07-14 09:55:46 +02:00
Ivan Nardi 0dd1ee5656 HTTP: fix extraction of filename (#2046) 2023-07-14 01:33:00 +02:00
Luca Deri bfbbd18757 Removed -fno-color-diagnostics not available on all platforms 2023-07-13 23:30:05 +02:00
Luca Deri 33c42f9830 Cosmetic fixes 2023-07-13 23:26:35 +02:00
Luca Deri 1f55dc511f Implemented Count-Min Sketch [count how many times a value has been observed]
- ndpi_cm_sketch_init()
- ndpi_cm_sketch_add()
- ndpi_cm_sketch_count()
- ndpi_cm_sketch_destroy()
2023-07-13 21:54:51 +02:00
Ivan Nardi bd0fcb2e62 HTTP: fix heap-buffer-overflow (#2044)
```
=================================================================
==673246==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x5080000002fd at pc 0x555c6a22eca2 bp 0x7ffe1db9b290 sp 0x7ffe1db9aa48
READ of size 1 at 0x5080000002fd thread T0
    #0 0x555c6a22eca1 in strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x53aca1) (BuildId: 464e7474cde3de91c01588c7ea0161a05ddd6b63)
    #1 0x555c6a462a9d in ndpi_http_check_content /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:309:8
    #2 0x555c6a45a9a0 in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:901:46
    #3 0x555c6a4526ce in process_response /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1280:3
    #4 0x555c6a44f90f in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1373:9
    #5 0x555c6a44f475 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1459:3
    #6 0x555c6a348520 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5949:6
    #7 0x555c6a348f2f in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5997:12
    #8 0x555c6a348bcf in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6016:12
    #9 0x555c6a359e26 in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7017:15
    #10 0x555c6a355f6f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7184:22
    #11 0x555c6a284958 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:24:5
```

Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60557
Fix: 0b0f255cc
2023-07-13 10:50:59 +02:00
Ivan Nardi 0c5a17accb RDP: improve detection over UDP (#2043) 2023-07-13 10:50:13 +02:00
Ivan Nardi 5e6a8c0c1f fuzz: add fuzzer for DGA detection code (#2042) 2023-07-13 10:47:34 +02:00
Alfredo Cardigliano 669a02cd76 Add support for multiline json 2023-07-12 10:47:49 +02:00
Luca Deri b1d25193d2 Minor code cleanup 2023-07-11 22:49:06 +02:00
Chiara Maggi 0b0f255cc2 added feature to extract filename from http attachment (#2037)
* added feature to extract filename from http attachment

* fixed some issues

* added check for filename format

* added check for filename format

* remove an unnecessary print

* changed the size from 952 to 960

* modified some test result files

* small changes string size

* comment removed and mallocs checked
2023-07-11 22:45:19 +02:00
Ivan Nardi 950f5cc4e3 fuzz: extend fuzzing coverage (#2040)
Some notes:
* libinjection: according to https://github.com/libinjection/libinjection/issues/44,
it seems NULL characters are valid in the input string;
* RTP: `rtp_get_stream_type()` is called only for RTP packets; if you
want to tell RTP from RTCP you should use `is_rtp_or_rtcp()`;
* TLS: unnecessary check; we already make the same check just above, at
the beginning of the `while` loop
2023-07-11 10:12:08 +02:00
Toni 859d9ea3c3 Fixed heap-overflow if compiled with --enable-tls-sigs. (#2038)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-07-07 18:52:06 +02:00
Toni 72814bed33 Added coverage targets to Makefile.am for convenience. (#2039)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-07-07 17:40:50 +02:00
Ivan KapranovandIvan Kapranov bdd295bc2c fix Stack overflow caused by invalid write in ndpi_automa_match_strin… (#2035)
* minor fixes

fixed 'handle leak' in ndpi_load_malicious_sha1_file and removed the redundant comparison ndpi_search_eaq

* fix Stack overflow caused by invalid write in ndpi_automa_match_string_subprotocol

* fix compile errors

* fix

---------

Co-authored-by: Ivan Kapranov <i.kapranov@securitycode.ru>
2023-07-06 11:13:34 +02:00
Ivan Nardi bb978907e6 STUN: fix detection of Google Voip apps (#2031)
Fix: 2c7fb9179
2023-07-05 14:37:38 +02:00
headshog 076ba967a8 fixed numeric truncation error in diameter.c (#2034) 2023-07-05 14:29:55 +02:00
headshog d51f8929e0 fixed numeric truncation error in rtcp.c (#2033) 2023-07-05 14:29:44 +02:00
headshog 2e5004ddb2 fixed numeric truncation error in kerberos.c (#2032) 2023-07-05 14:29:32 +02:00
Ivan Nardi 7ffd31ebc3 STUN: avoid FacebookVoip false positives (#2029)
Attribute 0xC057 is defined in the Google public implementation of
webrtc (which is used by Google products but also by other applications)
2023-07-03 17:21:46 +02:00
Ivan Nardi dff1f25193 STUN: fix Skype/MsTeams detection and monitoring logic (#2028) 2023-07-03 17:21:32 +02:00
Ivan Nardi 86e89b4e23 STUN: tell RTP from RTCP while in monitoring state (#2027) 2023-06-27 10:33:48 +02:00
Ivan Nardi 2c7fb91794 Hangout: detect Hangout/Duo/GoogleMeet/... in the STUN code (#2025)
Regardless of the name, the removed trace doesn't contain meaningful
Hangout traffic.

Remove last piece of sub-classifiction based only on ip addresses.
2023-06-27 10:33:28 +02:00
Ivan Nardi 31a9da238c STUN: add dissection of DTLS handshake (#2018) 2023-06-26 12:07:26 +02:00
Ivan Nardi 88425e0199 Simplify the report of streaming multimedia info (#2026)
The two fields `flow->flow_type` and `flow->protos.rtp.stream_type` are
pretty much identical: rename the former in `flow->flow_multimedia_type`
and remove the latter.
2023-06-26 12:05:16 +02:00
Ivan Nardi 3a1600ff26 Thrift: fix heap-buffer-overflow (#2024) 2023-06-26 11:41:01 +02:00
Luca Deri 7ef80f6b78 Optimizes and fixes possible out0of0boundary write in ndpi_fill_prefix_v4() 2023-06-23 21:40:55 +02:00
Ivan Nardi 7e64d9f66d RTP: rework code (#2021)
Try avoiding false positives: look for 3 RTP packets before classifing
the flow as such.

Add a generic function `is_rtp_or_rtcp()` to identify RTP/RTCP packets also
in other dissectors (see 3608ab01b commit message for an example)
2023-06-23 08:12:40 +02:00
Ivan KapranovandIvan Kapranov 15f0e1480c minor fixes (#2023)
fixed 'handle leak' in ndpi_load_malicious_sha1_file and removed the redundant comparison ndpi_search_eaq

Co-authored-by: Ivan Kapranov <i.kapranov@securitycode.ru>
2023-06-22 21:36:40 +02:00
Ivan Kapranov aa25c06a18 removed useless call of ndpi_set_risk func (#2022) 2023-06-22 18:26:33 +02:00
Luca Deri cdc0bb8c61 Compilation fix 2023-06-22 16:04:14 +02:00
Toni 1678888284 Add Apache Thrift protocol dissector. (#2007)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-06-22 13:07:32 +02:00
Luca Deri 8dbd591053 Waring fix 2023-06-21 21:26:29 +02:00
Ivan Nardi cc582052b3 Gnutella: improve detection (#2019) 2023-06-21 14:25:34 +02:00
Nardi Ivan 570c75d601 STUN: fix detection over TCP
TCP framing is optional
2023-06-21 10:38:44 +02:00
Nardi Ivan 2ac240ce6a STUN: improve WhatsappCall detection 2023-06-21 10:38:44 +02:00
Ivan Nardi 46aab35a29 Line: fix heap-buffer-overflow error (#2015) 2023-06-21 09:22:07 +02:00
Ivan Nardi 3608ab01b6 STUN: keep monitoring/processing STUN flows (#2012)
Look for RTP packets in the STUN sessions.
TODO: tell RTP from RTCP
2023-06-21 09:16:20 +02:00
Ivan Nardi 04be308092 Fix access to packet/flow information (#2013)
We can't access to `ndpi_str->packet` from `ndpi_detection_giveup()`
2023-06-17 15:46:06 +02:00
Luca Deri ac06ba8320 Added hyperlink 2023-06-16 21:48:21 +02:00
Luca Deri 9cc4cbb9d1 Reworked teams handling 2023-06-15 22:31:11 +02:00
Ivan Nardi 2321c11eb5 Fix compilation (#2011) 2023-06-15 12:01:11 +02:00
Luca Deri d0609ea601 Implemented Zoom/Teams stream type detection 2023-06-14 23:44:57 +02:00
Luca Deri 66bee475ae Improved line protocol dissection with heuristic 2023-06-13 19:20:02 +02:00
Luca Deri de3cf260f8 Updated line test result 2023-06-13 19:07:02 +02:00
Luca Deri be8178fc8d Refreshed ASN lists
Enhanced the Line IP list with https://ipinfo.io/AS23576/125.209.252.0/24 used by line
2023-06-13 19:04:08 +02:00
Luca Deri b02adcc381 Added check to avoid skype heuristic false positives 2023-06-12 23:29:20 +02:00
Toni 4e284b5e40 Set _DEFAULT_SOURCE and _GNU_SOURCE globally. (#2010)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-06-12 19:53:57 +02:00
Toni 8ea0eaa0d0 Fix missing u_char, u_short and u_int typedefs for some platforms e.g.: (#2009)
In file included from ../include/ndpi_main.h:34,
                 from ../include/ndpi_api.h:28,
                 from protocols/activision.c:26:
../include/ndpi_typedefs.h:294:3: error: unknown type name 'u_char'
  294 |   u_char h_dest[6];       /* destination eth addr */
      |   ^~~~~~

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-06-12 17:52:52 +02:00
Nardi Ivan 96340ccea2 oss-fuzz: sync build script with upstream
File copied from https://github.com/google/oss-fuzz/blob/master/projects/ndpi/build.sh

The general idea is to keep the build script in our repository and use
it from oss-fuzz builder: updating it from our side is easier and faster
then passing via an oss-fuzz PR.

The original idea is from @utoni in 3068306b60.

Once this change has been merged, we can update the code in oss-fuzz.
2023-06-12 15:17:56 +02:00
Luca Deri 63a2b359b9 Added vlan_id in ndpi_flow2json() prototype 2023-06-09 19:50:47 +02:00
Nardi Ivan 7150b40c63 QUIC: fix dissection of packets forcing VN 2023-06-08 17:07:25 +02:00
Nardi Ivan 3e640a13a4 QUIC: fix a memory access error
Found while fuzzing
2023-06-08 17:07:25 +02:00
Nardi Ivan fd4cb10190 QUIC: add support for QUIC version 2
See: https://www.rfc-editor.org/rfc/rfc9369.txt

Old v2-01 version has been removed, since it has never been really used.
2023-06-08 17:07:25 +02:00
Ivan Nardi 9987e5b482 ndpiReader: allow to configure LRU caches TTL and size (#2004) 2023-06-08 17:06:32 +02:00
Ivan Nardi 3e673e91a9 ProtonVPN: add basic detection (#2006) 2023-06-08 16:52:55 +02:00
headshog 7dcceb4daf Numeric truncation at tls.c:1010 (#2005)
* fixed numtrunc error in protocols/tls.c

* fixed build error for tls.c
2023-06-08 07:52:18 +02:00
zehraInandNardi Ivan 436ab54f5a added new domain names (#2002)
* added new domain names

* Sync unit tests results

---------

Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2023-06-01 13:13:49 +02:00
Ivan Nardi e14d0acbd4 tests: add an option to force the overwrite of the unit tests results (#2001)
Usage: `FORCE_UPDATING_UTESTS_RESULTS=1 ./tests/do.sh`
2023-05-31 09:05:00 +02:00
Maatuq e17fa1259a Add bitcoing protocol dissector. (#1992)
* Add bitcoing protocol dissector.

* remove bitcoin protcol detection from mining.c
* add a new bitcoin deissector.
* add a new category: Cryptocurrency.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* Remove useless checks and add missing windows and docs file.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* update affected tests.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* add a brief version.

Add notes on the difference between normal bitcoin protocol and the
mining protocol.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* update enable_payload_stat test after dev rebasing.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

---------

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
2023-05-31 07:31:01 +02:00
headshog a8d2eeddd2 Numeric truncation at ndpi_analyze.c at lines 101, 104, 107, 110 (#1999)
* fixed numeric truncation error in ndpi_analyze.c

* fixed numeric truncation error in ndpi_analyze.c x2

* fixed numeric truncation error in ndpi_analyze.c x3

* fixed numeric truncation error in ndpi_analyze.c and printf format

* fixed tests
2023-05-30 12:27:47 +02:00
Toni 04f5c5196e Changed logging callback function sig. (#2000)
* make user data available for any build config

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-30 08:27:37 +02:00
headshog 552c9b3cc6 fixed numeric truncation error in ndpi_main.c:6837 (#1998) 2023-05-29 19:34:26 +02:00
Ivan Nardi efb261a95c Fix some memory errors triggered by allocation failures (#1995)
Some low hanging fruits found using nallocfuzz.
See: https://github.com/catenacyber/nallocfuzz
See: https://github.com/google/oss-fuzz/pull/9902

Most of these errors are quite trivial to fix; the only exception is the
stuff in the uthash.
If the insertion fails (because of an allocation failure), we need to
avoid some memory leaks. But the only way to check if the `HASH_ADD_*`
failed, is to perform a new lookup: a bit costly, but we don't use that
code in any critical data-path.
2023-05-29 19:24:00 +02:00
Ivan Nardi 346bb268e2 Avoid calling ndpi_reconcile_protocols() twice in ndpi_detection_giveup() (#1996)
`ndpi_reconcile_protocols()` is already called by
`ndpi_set_detected_protocol()`
2023-05-29 16:55:53 +02:00
Ivan Nardi 82fa3a0986 Fix an heap-buffer-overflow (#1994)
```
==32439==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x604000000080 at pc 0x564a9a1639ac bp 0x7ffc77d42e20 sp 0x7ffc77d42e18
READ of size 1 at 0x604000000080 thread T0
    #0 0x564a9a1639ab in ndpi_check_punycode_string /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:98:12
    #1 0x564a9a26025a in ndpi_match_host_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8652:6
    #2 0x564a9a33e45c in ndpi_http_parse_subprotocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:446:9
    #3 0x564a9a3354b8 in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:921:3
    #4 0x564a9a32dc8b in process_request /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1237:3
    #5 0x564a9a329a9c in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1351:5
    #6 0x564a9a3293ed in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1425:3
    #7 0x564a9a21c64a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5906:6
    #8 0x564a9a21d057 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5954:12
```

Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=59393
2023-05-29 16:53:39 +02:00
Ivan Nardi 46ff069117 ndpiReader: improve printing of payload statistics (#1989)
Add a basic unit test

Fix an endianess issue
2023-05-29 16:53:11 +02:00
Ivan Nardi 7ce14da0c8 Remove some useless checks (#1993)
In the main dissector callbacks the flow protocols are (almost) always
unknown. Only two exceptions:
* extra dissection data path
* HTTP sub-protocols
2023-05-28 12:59:18 +02:00
Toni 6da3474203 Improved helper scripts. (#1986)
* added additional (more restrictive) checks

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-28 12:45:44 +02:00
Ivan Nardi b11e6a453b Add support for Epic Games and GeForceNow/Nvidia (#1990) 2023-05-27 12:13:54 +02:00
Ivan Nardi 63ac50e4f4 Improve detection of Alibaba flows (#1991) 2023-05-27 10:19:58 +02:00
Toni 334b43579e Fixed invalid use of ndpi_free(). Sorry, my fault. (#1988)
* Fixed invalid use of ndpi_free(). Sorry, my fault.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Fine tuned symbol check script.

 * added check for expected syms in modules

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

---------

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-05-24 13:19:06 +02:00
Luca Deri f692342f3c Cosmetic changes 2023-05-23 22:16:34 +02:00
Ivan Nardi 86b56646b5 ndpiReader: fix export of DNS/BitTorrent attributes (#1985)
There is no BitTorrent hash in the DNS flows
2023-05-20 17:23:48 +02:00
Toni 5e8f93c2d1 Improved missing usage of nDPIs malloc wrapper. Fixes #1978. (#1979)
* added CI check

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-20 16:18:52 +02:00
Luca Deri 8f718c9051 Made VK protocol detection more strict 2023-05-20 15:46:47 +02:00
Toni c97e2d67ff Added scripts to auto generate hostname/SNI *.inc files. (#1984)
* add illegal gambling sites (Belgium)

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-20 15:41:15 +02:00
Ivan Nardi 9004d5c2ca ndpiReader: fix export of HTTP attributes (#1982) 2023-05-20 15:12:14 +02:00
Nardi Ivan 1ab5318f9c Sync unit tests results 2023-05-20 15:09:51 +02:00
headshog dd7198fcaf fixed numeric truncation error 2023-05-20 15:09:51 +02:00
Luca Deri 5ca6f0ac62 Implemented ndpi_predict_linear() for predicting a timeseries value overtime 2023-05-19 11:46:03 +02:00
Yuriy Rymarchuk a8b3baf2b8 Add support for roaring_bitmap_xor_inplace (#1983) 2023-05-18 13:28:32 +02:00
Ivan Nardi b1bcf1ff60 Fix classification-by-ip in ndpi_detection_giveup (#1981)
Return the "classification-by-ip" as protocol results only if no other
results are available.
In particular, never return something like
"protocol_by_port/protocol_by_ip" (i.e. `NTP/Apple`,
BitTorrent/GoogleCloud`, `Zoom/AWS`) because this kind of classification
is quite confusing, if not plainly wrong.

Notes:
* the information about "classification-by-ip" is always available, so
no information is lost with this change;
* in the unit tests, the previous classifications with confidence
`NDPI_CONFIDENCE_DPI_PARTIAL` were wrong, as noted in #1957
2023-05-17 11:26:25 +02:00
Ivan Nardi ace32c9dfe Add support for SRTP (#1977)
The goal is to have Zoom flows classified as "Encrypted" and not as
"Cleartext".

Start documenting the list of protocols supported by nDPI;
format, verbosity and content are still a work-in-progress.
2023-05-16 16:02:14 +02:00
Ivan Nardi 4e186f6bfb HTTP: rework state machine (#1966)
The goal if to correlate the right request-response pair, exporting
metadata from only one transaction (for example, the right url & return
state pair)

As a nice side effect, the code should be much cleaner, but that is a
matter of taste.

Two differences respect to the previous code:
* as it happens in the CI, if in the flow there are only one response
(before) and one request (after), only the metadata of the response are
saved/exported
* for performance reasons, we don't call `ndpi_parse_packet_line_info()`
anymore for ALL packets triggering the HTTP dissector, but only for the
packets that we already know belong to an HTTP flow. This is the reason
for the changes in RTSP/SOAP/... code
2023-05-16 16:01:55 +02:00
Ivan Nardi 8c224b464f Fix MS Teams detection with heuristic (#1972)
When we reconcile a TLS session to Teams, we need to keep TLs as master.
This way:
* we keep exporting all the TLS metadata
* we avoid some memory leaks (of these metadata themeselves)
2023-05-10 21:00:27 +02:00
Luca Deri 596a5fd5dc Updated results 2023-05-10 17:11:14 +02:00
Luca 64853dff02 Improved MS Teams detection with heuristic 2023-05-10 16:59:28 +02:00
Luca 98f029a091 Enhanced MS teams STUN/Azure detection 2023-05-10 16:28:34 +02:00
Luca 7d5e4d0f7e MS Teams enhancement 2023-05-10 15:41:26 +02:00
Luca 8de7c49298 MS Teams improvements 2023-05-10 15:16:39 +02:00
Ivan Nardi 31d23aee56 All protocols should be excluded sooner or later (#1969)
For a lot of protocols, reduce the number of packets after which the
protocols dissector gives up.
The values are quite arbitary, tring to not impact on classification
2023-05-10 12:50:24 +02:00
Ivan Nardi 99d7066ea0 Remove special handling of some TCP flows without SYN (#1965)
This piece of code has multiple problems:
* nDPI is able to detect some TCP protocols even with mid-flows (i.e.
without the initial packets of the session); TLS is the most
significative example
* since e6b332aa4a it is perfectly valid
to not pass the TCP Handshake packets to nDPI
* in any case, we shouldn't call `ndpi_detection_giveup()`. That
function is usually called by the application and we end up calling it
twice in some cases.

The simple solution is to completely remove that code: process these
kinds of flows like everyone else.

Note that the application can always avoid to pass to nDPI any TCP flows
without the initial handshake; the flow managemnt is always up to the
application.

Looking at the CI results, some rare flows are now processed significantly
longer. As a follow-up we could look into that.
2023-05-09 19:36:02 +02:00
Ivan Nardi 684e041998 Improve detection of crawlers/bots (#1968)
Add support for Facebook crawler
2023-05-09 16:42:29 +02:00
Ivan Nardi 79c1dbe0b2 Extend content-match list (#1967) 2023-05-09 16:41:12 +02:00
Luca Deri 2f9059dc54 Disabled query string validation in MDNS in order to avoid zapping chars that in DNS (instead) are not permitted 2023-05-09 15:04:02 +02:00
Luca Deri bfe79243bc Refreshed lists 2023-05-08 16:51:38 +02:00
Clément BRUGUERA 998bedbbb7 add 2 ns from fdn.fr to DoH section (#1964) 2023-05-05 18:08:13 +02:00
Ivan Nardi 0223d3c4f5 HTTP: improve extraction of metadata and of flow risks (#1959) 2023-05-05 13:35:20 +02:00
Ivan Nardi 02a2c80453 ndpiReader: fix print of flow payload (#1960) 2023-05-04 11:52:48 +02:00
Ivan Nardi a11ddd2dc4 HTTP: remove useless code about XBOX (#1958)
Old nDPI versions were able to detect XBOX flows over HTTP via
user-agent matching. This feature has been removed from a long time
(89d548f9d, at very least)
2023-05-04 11:28:03 +02:00
Ivan Nardi 6b94c9675a Improve detection of crawler/bot traffic (#1956) 2023-05-04 11:27:34 +02:00
Ivan Nardi 9a5d60bff6 Sync unit tests results (#1962) 2023-05-04 09:47:52 +02:00
Luca Deri 3681de4795 Updated pcap detection results after Facebook Reel/Stories support 2023-05-03 22:52:58 +02:00
Luca Deri de13c6c0bd Added detection of Facebook Reels and Stories
https://www.switcherstudio.com/blog/the-ultimate-guide-to-reels-and-stories#:~:text=Audience%3A%20Reels%20are%20served%20to,things%20like%20photos%20and%20boomerangs.
2023-05-03 22:50:02 +02:00
Ivan Nardi bb370f5ef0 Make Bittorrent LRU cache IPv6 aware. (#1909)
It was the only remaining LRU cache without IPv6 support.
See 81e1ea545c
2023-04-29 11:09:23 +02:00
Luca Deri 6b803f48dc Added missing check 2023-04-28 23:08:54 +02:00
Luca Deri 6e9dce357b Added ndpi_guess_undetected_protocol_v4() to restore logic that
was removed in https://github.com/ntop/nDPI/commit/032e778a6dc1ee2173c4cdb2c671180c5b5df245
2023-04-28 23:00:33 +02:00
Luca Deri f7138f07a6 Missing module termination 2023-04-28 23:00:33 +02:00
Luca Deri b39ece4428 Updated (C) 2023-04-28 23:00:33 +02:00
Luca Deri 791d3e6ad3 Added missing check 2023-04-28 23:00:33 +02:00
Ivan Nardi 8934f7b45f Add an heuristic to detect/ignore some anomalous TCP ACK packets (#1948)
In some networks, there are some anomalous TCP flows where the smallest
ACK packets have some kind of zero padding.
It looks like the IP and TCP headers in those frames wrongly consider the
0x00 Ethernet padding bytes as part of the TCP payload.
While this kind of packets is perfectly valid per-se, in some conditions
they might be treated by the TCP reassembler logic as (partial) overlaps,
deceiving the classification engine.
Add an heuristic to detect these packets and to ignore them, allowing
correct detection/classification.

This heuristic is configurable. Default value:
* in the library, it is disabled
* in `ndpiReader` and in the fuzzers, it is enabled (to ease testing)

Credit to @vel21ripn for the initial patch.

Close #1946
2023-04-25 19:25:07 +02:00
Ivan Nardi 40b6d5a2e1 fuzz: extend fuzzers coverage (#1952) 2023-04-25 16:37:28 +02:00
Ivan Nardi de693cbbc9 RTP: remove dead-code (#1953)
The checks `isValidMSRTPType(..) == 1` is a subset of
`is_valid_rtp_payload_type()` so this if-branch is never reached.

More importantly, the article describing how to detect Microsoft Lync and
Skype for Business is from 2014. These payload types are static or they
are in the dynamic range: in both cases, these values might be used (and
they are used indeed) pretty much by every application.
Bottom line: we can't use PT alone to identify a specific protocol.

Keep the list, since it is used to tell audio streams from video ones.
2023-04-25 14:16:27 +02:00
Luca Deri c4ba981370 Compilation fix on RH7 2023-04-24 08:49:11 +02:00
nikitamishagin d3e67faf20 Add "Heroes of the Storm" video game signature detection. (#1949) 2023-04-22 18:28:43 +02:00
Toni b6629ba2be Improved debug output. (#1951)
* try to get rid of some `printf(..)`s as they do not belong to a shared library
 * replaced all `exit(..)`s with `abort()`s to indicate an abnormal process termination

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-21 12:40:26 +02:00
Toni 85751ba9d7 Added OICQ dissector. (#1950)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-21 11:30:10 +02:00
Ivan Nardi ba99330031 ndpiReader: fix flow stats (#1943) 2023-04-14 11:17:37 +02:00
Ivan Nardi 032e778a6d Simplify ndpi_internal_guess_undetected_protocol() (#1941)
`ndpi_guess_undetected_protocol()/ndpi_internal_guess_undetected_protocol()`
is a strange function:
* it is exported by the library and it is actively used by `ntopng`
* it is intrinsecally ipv4-only
* it returns basically something like "classification_by_ip"/"classification_by_port"
(these information have already been calculated in `ndpi_do_guess()`...)
* it access the bittorrent LRU caches (similarly to
`ndpi_detection_giveup()` but without all the other caches...)

So:
* make the interface IPv4/6 agnostic
* use the classifications already available

This work will allow to make the Bittorrent caches IPV6-aware (see
81e1ea5).

Handle Dropbox classification "by-port" in the "standard" way.
2023-04-12 14:39:10 +02:00
Toni 6660feed35 Added BACnet dissector. (#1940)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 20:26:27 +02:00
lns 77ea80862f Improved debug logging.
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 15:39:09 +02:00
lns 6da958655c Improved invalid logging via printf().
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 15:39:09 +02:00
Toni bebcce5281 Added Source Engine dissector. (#1937)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 14:31:57 +02:00
Luca Deri 9ba16a4837 Test files for riit games 2023-04-11 10:14:18 +02:00
Jacopo a469009842 RiotGames: add detection of flows (#1935)
* RiotGames: add detection of flows

* remove of akamai domains
2023-04-11 10:04:09 +02:00
Ivan Nardi c7ef3608d9 CI: fix Performance job (#1936)
Fix: 7714507f
2023-04-09 10:16:28 +02:00
Luca Deri c47e9d201d Implemented ndpi_XXX_reset() API calls whre XXX is ses, des, hw 2023-04-08 09:52:14 +02:00
Ivan Nardi 7714507f81 Test multiple ndpiReader configurations (#1931)
Extend internal unit tests to handle multiple configurations.
As some examples, add tests about:
* disabling some protocols
* disabling Ookla aggressiveness

Every configurations data is stored in a dedicated directory under
`tests\cfgs`
2023-04-06 11:30:36 +02:00
Ivan Nardi 7a627296f0 Fix LRU/Patricia/Automa stats in ndpiReader with multiple threads (#1934) 2023-04-06 09:36:11 +02:00
Ivan Nardi b639744c3a Extend content list of Microsoft protocols (#1930) 2023-04-06 09:35:41 +02:00
Ivan Nardi 22cf8b5d8f configure: add an option to enable debug build, i.e -g (#1929)
Add this new flag to one CI job, to test it

Close #1925
2023-04-06 09:35:27 +02:00
Ivan Nardi 06a63dcbaa MySql: improve detection (#1928)
Add support for flows with "caching_sha2_password" authentication plugin.

See #1924
2023-04-06 09:34:39 +02:00
Ivan Nardi 5c28dbbae2 H323: fix false positives (#1916) 2023-04-06 09:32:57 +02:00
Ivan Nardi 25c1111911 fuzz: add a new fuzzer triggering the payload analyzer function(s) (#1926) 2023-04-04 14:39:29 +02:00
Maatuq f1193d5e6f add support for gre decapsulation (#1442) (#1921) 2023-04-04 14:20:11 +02:00
Ivan Nardi cc5aec5f0a fuzz: add fuzzer to test internal gcrypt code (#1920) 2023-04-04 14:19:41 +02:00
Ivan Nardi d7662379b4 Update libinjection code (#1918)
Update libinjection code to the current master https://github.com/libinjection/libinjection/commit/7e4b74e824dc3f0623ba4894df2a0d817351ec39

The goal is to finally fix #1820
See: https://github.com/libinjection/libinjection/issues/33

Update the corpus of the libinjection fuzzers

Close #1820
2023-04-04 14:18:57 +02:00
Ivan Nardi 4d11941d32 Ookla: rework detection (#1922)
The logic of the LRU cache has been changed: once we know an ip has
connected to an Ookla server, all the following (unknown) flows (for
a short time interval) from the same ip to the port 8080 are treated
as Ookla ones.

Most of the changes in this commit are about introducing the concept of
"aggressive detection". In some cases, to properly detect a
protocol we might use some statistical/behavior logic that, from one
side, let us to identify the protocol more often but, from the other
side, might lead to some false positives.
To allow the user/application to easily detect when such logic has been
triggered, the new confidence value `NDPI_CONFIDENCE_DPI_AGGRESSIVE` has been
added.
It is always possible to disable/configure this kind of logic via the
API.

Detection of Ookla flows using plain TLS over port 8080 is the first
example of aggressive detection in nDPI.

Tested with:
* Android 9.0 with app 4.8.3
* Ubuntu 20.04 with Firefox 110
* Win 10 with app 1.15 and 1.16
* Win 10 with Chrome 108, Edge 108 and Firefox 106
2023-03-30 17:13:51 +02:00
Ivan Nardi 3e06bcce8d Add another example of custom rules (#1923)
Add an example where traffic matching the same IP, but different ports
is classified to different protocols.

Close #189
2023-03-30 08:45:17 +02:00
Ivan Nardi 6e0cd16225 TLS: fix another interger overflow in certificate processing (#1915) 2023-03-30 05:12:40 +02:00
Maatuq 9ea8a57839 remove redefinition to vxlanhdr struct in vxlan dissector (#1911) 2023-03-25 19:25:22 +01:00
Ivan Nardi 04a426feef ndpiReader: fix VXLAN de-tunneling (#1913)
```
==20665==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6040000aec81 at pc 0x0000004f5c6f bp 0x7fff07e9e1f0 sp 0x7fff07e9e1e8
READ of size 1 at 0x6040000aec81 thread T0
SCARINESS: 12 (1-byte-read-heap-buffer-overflow)
    #0 0x4f5c6e in ndpi_is_valid_vxlan ndpi/example/reader_util.c:1784:6
    #1 0x4f5c6e in ndpi_workflow_process_packet ndpi/example/reader_util.c:2292:16
    #2 0x4dd821 in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_ndpi_reader.c:135:7
    #3 0x4f91ba in ExecuteFilesOnyByOne /src/aflplusplus/utils/aflpp_driver/aflpp_driver.c:234:7
    #4 0x4f8f8c in main /src/aflplusplus/utils/aflpp_driver/aflpp_driver.c:318:12
    #5 0x7f2289324082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
    #6 0x41e6cd in _start
```
Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=57369
2023-03-25 19:19:51 +01:00
Ivan Nardi ddb9aa0f95 TLS: fix parsing of certificate elements (#1910)
```
==1228==ERROR: AddressSanitizer: SEGV on unknown address 0x6040000bed05 (pc 0x00000056e148 bp 0x7ffcca534320 sp 0x7ffcca5330c0 T0)
==1228==The signal is caused by a WRITE memory access.
    #0 0x56e148 in processCertificateElements ndpi/src/lib/protocols/tls.c:682:79
    #1 0x56c60f in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_tls_certificate.c:43:3
    #2 0x43de63 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15
    #3 0x4295c2 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:324:6
    #4 0x42ee6c in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:860:9
    #5 0x4583a2 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    #6 0x7f8c021c9082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
    #7 0x41f78d in _start
```

Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=57317
2023-03-22 18:18:36 +01:00
Maatuq 530d0de438 Add support for vxlan decapsulation (#1441) (#1900)
Close #1441
2023-03-22 18:18:12 +01:00
Ivan Nardi 04e017a854 Rework CI jobs to try reducing CI duration (#1903)
CI duration is quite long: the longest jobs is the "Performance" one.
Try to reduce the overall duration: that job (and some others) will not
be triggered for each PR/commit anymore, but asynchronously, once a day
(this scheduling seems right since the frequency of the PR/commits in
the project).
It should be possibly to trigger them manually, via GUI, anyway.

Remove two identical jobs; we already tests ASAN with 4 different
compilers.

After 9eff0754 it is safe to reduce fuzzing time.

Bottom line: try to have as upper-time of CI tests the duration of the
fuzzing jobs
2023-03-22 18:17:25 +01:00
Luca Deri 64ebf73b29 Added the ability to define custom protocols with arbitrary Ids in proto.txt
Example
- ip:213.75.170.11/32:443@CustomProtocol
nDPI assigns an is that can change based on protos.txt content

- ip:213.75.170.11/32:443@CustomProtocol=9999
nDPI assigns 9999 as protocolId to CustomProtocol and won't change when
protos.txt content will chaneg
2023-03-22 00:15:56 +01:00
Luca Deri b3457c644f Removed overlapping port 2023-03-21 19:27:56 +01:00
Luca Deri c22d30786c Improved checks for duplicated entries in protocols file 2023-03-21 19:25:03 +01:00
Ivan Nardi 2a3ade397b DisneyPlus/Hulu ip lists should be auto-generated (#1905)
Remove two stale ip lists:
1) these 3 ips are in the Amazon ranges (now)...
2) the Instagram list originated from AS32934, which is now a Facebook
AS; see https://github.com/ntop/nDPI/pull/1264/commits/8dabd06301a802dd38616ba8684a1d995783e023
2023-03-20 19:26:40 +01:00
Ivan Nardi 3a75a46212 Add a new protocol id for generic Adult Content traffic (#1906)
The list has been taken from https://www.similarweb.com/top-websites/adult/
Fix a GoTo false positive.
2023-03-20 17:56:02 +01:00
Ivan Nardi 12867962b0 Add a new protocol id for generic advertisement/analytics/tracking stuff (#1904) 2023-03-20 17:01:18 +01:00
Ivan Nardi 0e80828e14 fuzz: add a new fuzzer to test TLS certificates (#1901) 2023-03-20 17:00:46 +01:00
Luca Deri 9f46d41eca FreeBSD compilation fix
(C) update
2023-03-14 22:16:57 +01:00
Ivan Nardi 9eff075419 fuzz: simplify fuzzers dependencies in CIFuzz (#1896)
CIFuzz (based on oss-fuzz) is the GitHub action/CI job that runs fuzz
targets on pull requests. It only runs the fuzzers affected by a pull
request/commit. Otherwise it will divide up the allotted fuzzing time
among all fuzzers in the project.
Since:
* we have more than 20 fuzzers and most of them use the custom memory
allocation functions (to force allocation failures) even if they are not
strictly about DPI stuff;
* we need to keep fuzzing time relatively small (to avoid waiting the CI
results for a long time)

it is important that fuzzers dependencies (which are based on *files*
changed by the single commit/PR) are as small as possible.

Bottom line: move all the low-level allocation callbacks to a dedicated
file; this way most of the fuzzers don't depend anymore on `ndpi_main.c`
file (which is touched by ever commit/PR).

The goal is to have only the "most important" fuzzers running during (most
of) the CI.
2023-03-14 19:34:01 +01:00
Luca Deri 3585e2d201 Added ability to define an unlimited number of custom rules IP:port for the same IP (it used tobe limited to 2) 2023-03-13 21:57:14 +01:00
Luca Deri ebfc1fe827 Minor cleanup 2023-03-13 20:52:08 +01:00
Luca Deri d398f232be Added fix for better handling exceptions rollback in case of later match 2023-03-06 17:33:08 +01:00
Ivan Nardi 89cae9ddf2 Add a new flow risk about literal IP addresses used as SNI (#1892)
RFC 6066 3: "Literal IPv4 and IPv6 addresses are not permitted in
"HostName"."

Don't set this risk if we have a valid sub-classification (example:
via certificate)

Since a similar risk already exists for HTTP hostnames, reuse it, with a
more generic name.
2023-03-02 15:27:30 +01:00
Ivan Nardi 3047e286c0 Fix insert of ip addresses into patricia tree(s) (#1895)
Fix: d0b46dcf
2023-03-02 14:30:20 +01:00
Ivan Nardi 22fb8349b9 ndpiReader: print how many packets (per flow) were needed to perform full DPI (#1891)
Average values are already printed, but this change should ease to
identify regressions/improvements.
2023-03-01 21:50:47 +01:00
Toni 2768366969 Fixed missing AS_HELP_STRING in configure.ac. (#1893)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-03-01 20:46:40 +01:00
Nardi Ivan e0ad5b49c0 Sync unit tests results 2023-03-01 20:34:56 +01:00
Nardi Ivan d65f8cf15f Keep master protocol in ndpi_reconcile_protocols
As a side effect of this change, the following memory leak has been
fixed.
```
==55523==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 2 byte(s) in 1 object(s) allocated from:
    #0 0x55657d01b7be in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_alloc_fail+0x5b27be) (BuildId: 9dfa7bd9081421417b8b1c2e8b44633a9bfcfe52)
    #1 0x55657d169182 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:243:25
    #2 0x55657d1694c8 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:310:13
    #3 0x55657d25ddc7 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2308:46
    #4 0x55657d2701b9 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:895:5
    #5 0x55657d26a408 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1223:11
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56466
2023-03-01 20:34:56 +01:00
Nardi Ivan 9eb9664516 SoftEther: fix invalid memory access
We can't write `flow->protos` union until we are really sure about protocol
classification

```
==28334==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x558db5554512 bp 0x000000000000 sp 0x7ffcb22c2880 T0)
==28334==The signal is caused by a READ memory access.
==28334==Hint: this fault was caused by a dereference of a high value address (see register values below).  Disassemble the provided pc to learn which register was used.
    #0 0x558db5554512 in __asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x48e512) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7)
    #1 0x558db55ea54b in __interceptor_free (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x52454b) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7)
    #2 0x558db56977ca in ndpi_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:274:7
    #3 0x558db56c20e3 in ndpi_free_flow_data /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5175:2
    #4 0x558db569783f in ndpi_free_flow /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8394:5
    #5 0x558db5627936 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:38:3
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56272
2023-03-01 20:34:56 +01:00
Luca Deri 127d8d0d35 Fixed uninitialized variable 2023-02-27 21:26:41 +01:00
Luca Deri 21c7550157 Moving to bash 2023-02-27 16:21:12 +01:00
Luca Deri a303756edb Added missing break 2023-02-27 14:38:40 +01:00
Luca Deri 25420e266e Updated results after the latest changes 2023-02-27 12:20:39 +01:00
Luca Deri 807c97344f in case of failure, failing result files are not listed 2023-02-27 12:20:15 +01:00
Luca Deri 96f0f85e56 Indent fix 2023-02-27 12:20:06 +01:00
Luca Deri 6a2e8c9fb1 Improved protocol detection exploiting IP-based guess
Reworked ndpi_reconcile_protocols() that is now called only in front of a match (less overhead)
2023-02-27 12:18:39 +01:00
Luca Deri d0b46dcf88 Enhanced custom port definition and improved error reporting in case of duplications 2023-02-24 09:35:26 +01:00
Ivan Nardi 4075324e2b fuzz: extend fuzz coverage (#1888) 2023-02-16 18:04:34 +01:00
Ivan Nardi 06d4f8c7e5 Update GitHub runners versions (#1889)
See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
2023-02-14 21:30:54 +01:00
Ivan Nardi f2f8f9d0ee Fix export/serialization of flow->risk (#1885)
Any risks bigger than 31 was never exported/serialized; found looking
a the fuzz coverage report.
2023-02-14 21:30:36 +01:00
Ivan Nardi 090ba5735f Sync utests results (#1887) 2023-02-13 13:17:01 +01:00
Ivan Nardi f10178f8d2 Fix packet counters (#1884)
We need to keep separete counters to keep tracks of packet numbers with
and without any payload.

Regression introduced in 5849863ef
2023-02-13 11:32:17 +01:00
Ivan Nardi 200d1d4cfc Fix detection of packet direction and NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1883) 2023-02-13 11:31:52 +01:00
Ivan Nardi a3a9a72f31 Fix compilation with GCC-7 and latest RoaringBitmap code (#1886)
Latest RoaringBitmap version (introduced with bf413afb) triggers a new
warning with GCC-7:

```
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ CC=gcc-7 CXX=g++-7 ./autogen.sh && make -s
autoreconf: Entering directory `.'
[...]
third_party/src/roaring.c:1815:1: warning: ‘no_sanitize’ attribute directive ignored [-Wattributes]
 static inline int array_container_cardinality(const array_container_t *array) {
 ^~~~~~
third_party/src/roaring.c:1964:5: warning: ‘no_sanitize’ attribute directive ignored [-Wattributes]
     const array_container_t *container2) {
[..]
```

The core issue is that `no_sanitize` attribute is defined only for GCC
>= 8.
That breaks the CI since we still use GCC-7 and `-Werror`: add a simple
workaround.

Fix compilation on Windows
2023-02-13 11:19:11 +01:00
Luca Deri 85bab9e660 Centos7 fixes 2023-02-13 01:02:27 +01:00
Luca Deri bf413afba1 Update roaring bitmap code 2023-02-12 12:50:24 +01:00
0xA50C1A1 ba4e145aad Add Yandex services detection (#1882)
Add Yandex services detection

Add VK and Yandex to the TLS certificate match list
2023-02-09 20:02:43 +01:00
Ivan Nardi b51a2ac72a fuzz: some improvements and add two new fuzzers (#1881)
Remove `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` define from
`fuzz/Makefile.am`; it is already included by the main configure script
(when fuzzing).

Add a knob to force disabling of AESNI optimizations: this way we can
fuzz also no-aesni crypto code.

Move CRC32 algorithm into the library.

Add some fake traces to extend fuzzing coverage. Note that these traces
are hand-made (via scapy/curl) and must not be used as "proof" that the
dissectors are really able to identify this kind of traffic.

Some small updates to some dissectors:

CSGO: remove a wrong rule (never triggered, BTW). Any UDP packet starting
with "VS01" will be classified as STEAM (see steam.c around line 111).
Googling it, it seems right so.

XBOX: XBOX only analyses UDP flows while HTTP only TCP ones; therefore
that condition is false.

RTP, STUN: removed useless "break"s

Zattoo: `flow->zattoo_stage` is never set to any values greater or equal
to 5, so these checks are never true.

PPStream: `flow->l4.udp.ppstream_stage` is never read. Delete it.

TeamSpeak: we check for `flow->packet_counter == 3` just above, so the
following check `flow->packet_counter >= 3` is always false.
2023-02-09 20:02:12 +01:00
0xA50C1A1 4bb851384e Add VK detection (#1880) 2023-02-02 15:27:59 +01:00
Luca Deri 64b8ae38db Typos 2023-02-01 21:15:30 +00:00
Luca Deri 563a24bf6b Minor update 2023-02-01 18:31:59 +01:00
Luca Deri c711f480e3 Moved to 4.7 2023-02-01 18:30:17 +01:00
Alfredo Cardigliano b0a340cbc6 Update CHANGELOG.md 2023-02-01 10:23:41 +01:00
Luca Deri c42b046d87 Updated elastic_search file name in project 2023-01-31 21:02:41 +01:00
Luca Deri 80368772bd File rename to avoid name clashes (e.g. on Windows) 2023-01-31 20:54:00 +01:00
Luca Deri 2086058d00 Updated (C) 2023-01-31 20:08:30 +01:00
Alfredo Cardigliano a151276db8 Update changelog 2023-01-31 17:25:55 +01:00
Ivan Nardi fe043d3af5 Further reduction of the size of some traces used as unit test (#1879)
See a944514d. No flow/classification/metadata have been removed.
2023-01-30 11:10:54 +01:00
Ivan Nardi 9f27cd56b0 ndpiReader: fix packet dissection (CAPWAP and TSO) (#1878)
Fix decapsulation of CAPWAP; we are interested only in "real" user data
tunneled via CAPWAP.
When Tcp Segmentation Offload is enabled in the NIC, the received packet
might have 0 as "ip length" in the IPv4 header
(see
https://osqa-ask.wireshark.org/questions/16279/why-are-the-bytes-00-00-but-wireshark-shows-an-ip-total-length-of-2016/)

The effect of these two bugs was that some packets were discarded.

Be sure that flows order is deterministic
2023-01-30 10:59:18 +01:00
Ivan Nardi 3e6cadbb76 ndpireader: fix "Discarded bytes" statistics (#1877) 2023-01-27 07:09:34 +01:00
Ivan Nardi 6d00a9e0bd fuzz: add a new fuzzer to test serialization/deserialization code (#1876)
Autodetecting the needed buffer size is quite complex (especially with
float/double values) so it is mandatory to properly check for
`ndpi_snprintf` truncation.
These issues have been undetected so far probably because the default
buffer is big enough for all common cases.

Add an example of usage of `ndpi_deserialize_clone_all()` (taken from
`ntopng`)
2023-01-27 07:09:18 +01:00
Luca Deri eea9956430 Improved connection refused detection 2023-01-25 22:18:50 +01:00
Ivan Nardi b2a8b48386 Fix compilation and CI (#1875)
ubuntu-latest s390x CI doesn't like snapshot bigger than 262144 bytes.

Fix an error found by fuzz CI

```
=================================================================
==55399==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x561e41cb684d bp 0x7ffd54ce3650 sp 0x7ffd54ce3520 T0)
==55399==The signal is caused by a READ memory access.
==55399==Hint: address points to the zero page.
    #0 0x561e41cb684d in ndpi_network_ptree_match /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2321:41
    #1 0x561e41d30879 in ndpi_guess_undetected_protocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7767:8
    #2 0x561e41ca804d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_config.cpp:104:5
    #3 0x561e41bb96a0 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x4726a0) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
    #4 0x561e41ba3c2f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x45cc2f) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
    #5 0x561e41ba96f6 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x4626f6) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
    #6 0x561e41bd22e2 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x48b2e2) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
    #7 0x7f94f0e5c082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #8 0x561e41b9eb0d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x457b0d) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
```
2023-01-25 14:49:45 +01:00
Ivan Nardi 3477560639 Update every ip lists (#1872) 2023-01-25 11:53:18 +01:00
Luca Deri 464269e5ed CenOS compilation fix 2023-01-25 11:52:46 +01:00
Luca Deri a3d3856c80 Compilation fixes 2023-01-25 11:45:35 +01:00
Ivan Nardi 9fc724de5a Add some fuzzers to test other data structures. (#1870)
Start using a dictionary for fuzzing (see:
https://llvm.org/docs/LibFuzzer.html#dictionaries).
Remove some dead code.
Fuzzing with debug enabled is not usually a great idea (from performance
POV). Keep the code since it might be useful while debugging.
2023-01-25 11:44:59 +01:00
Ivan Nardi 29c5cc39fb Some small changes (#1869)
All dissector callbacks should not be exported by the library; make static
some other local functions.
The callback logic in `ndpiReader` has never been used.
With internal libgcrypt, `gcry_control()` should always return no
errors.
We can check `categories` length at compilation time.
2023-01-25 11:44:09 +01:00
Luca Deri 5849863ef9 Added new risk NDPI_TCP_ISSUES 2023-01-24 22:58:17 +01:00
Ivan Nardi 5e8c1ebbb7 fuzz: fix memory allocation failure logic (#1867)
We *do* want to have some allocation errors.
Fix some related bugs
Fix: 29be01ef
2023-01-20 14:27:33 +01:00
Ivan Nardi 496b284c98 Bittorrent: fix detection over TCP (#1868)
Close #1866
2023-01-19 22:02:23 +01:00
Ivan Nardi c4274a4aec Minor changes in ndpi_detection_giveup() (#1861)
Check the mining cache at the end of the function, like all the others
LRU caches.
Rewrite the STUN checks: same identical semantic but in a cleaner way,
hopefully
2023-01-19 21:45:35 +01:00
Ivan Nardi f5c2f9280c Fix undefined-behaviour in ahocorasick callback (#1864)
```
ndpi_main.c:2119:28: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
```
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=55226
2023-01-19 14:13:17 +01:00
Ivan Nardi f95bdaf625 Bittorrent: fix heap-buffer-overflow (#1863)
```
==258287==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600068ff9d at pc 0x5653a6e35def bp 0x7ffeef5aa620 sp 0x7ffeef5a9dc8
READ of size 22 at 0x60600068ff9d thread T0
    #0 0x5653a6e35dee in strncmp (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x4d2dee) (BuildId: 133b8c3c8ff99408109fcb9be2538bb8341f07f7)
    #1 0x5653a70d6624 in ndpi_search_bittorrent /home/ivan/svnrepos/nDPI/src/lib/protocols/bittorrent.c:500:71
    #2 0x5653a6ff255a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5686:6
    #3 0x5653a6ff331b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5722:10
    #4 0x5653a6ff2cbc in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5755:12
    #5 0x5653a70016bf in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6578:15
    #6 0x5653a6f1836d in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1678:31
    #7 0x5653a6f140a1 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2256:10
```
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=55218
Fix: 470eaa6f
2023-01-19 14:12:51 +01:00
Ivan Nardi edb8165ab9 Sync unit tests results (#1865) 2023-01-19 11:03:45 +01:00
Luca Deri bc44334599 Shrinked symbolic flow risks labels 2023-01-19 10:14:32 +01:00
Ivan Nardi 02f52207be Fuzz: fix compilation with nBPF (#1860)
Close #1859
2023-01-18 18:26:54 +01:00
Ivan Nardi 1b98bec0ab LRU caches: add a generic (optional and configurable) expiration logic (#1855)
Two caches already implemented a similar mechanism: make it generic.
2023-01-18 18:18:36 +01:00
Ivan Nardi de24206adc POP3: improve detection (#1856) 2023-01-18 07:19:44 +01:00
Ivan Nardi 97014c53f3 Improve support for Snapchat voip calls (#1858)
Latest Snapchat versions use QUICv1 for their audio/video real time
sessions. See c50a8d480
2023-01-17 22:02:23 +01:00
Ivan Nardi ccc5a20710 Add detection of Tailscale (#1857)
Extend the example of wireguard traffic
2023-01-17 21:55:38 +01:00
Ivan Nardi 470eaa6ff3 Add Meraki Cloud protocol and improve other protocols (#1854)
Avoid some LineCall and Jabber false positives.
Detect Discord mid flows.
Fix Bittorrent detection.
2023-01-17 11:38:39 +01:00
Ivan Nardi 29be01ef3a Add some fuzzers to test algorithms and data structures (#1852)
Fix some issues found with these new fuzzers
2023-01-17 08:31:59 +01:00
Ivan Nardi ebb9ebd2a0 Fix classification "by-port" (#1851)
Classification "by-port" should be the last possible effort, *after*
having test all the LRU caches.

Remove some dead code from `ndpi_detection_giveup()`:
`flow->guessed_protocol_id` is never set to any od those voip protocols
and at that point in this function we never have both a master *and* a
application protocols.
Coverage reports (both from unit tests and from fuzzing) confirms that
was dead code.
2023-01-17 08:26:42 +01:00
Luca Deri 1f7c57deff Improved DGA detection 2023-01-12 10:06:31 +01:00
Ivan Nardi a944514dde Reduce the size of some traces used as unit test (#1845)
No traces and no flows has been removed; only long sessions
has been reduced, keeping only their first packets.

This is quite important in fuzzing systems, since these pcaps are used
as initial seed. There is no value in fuzzing long sessions, because
only the very first packets are really used/processed by nDPI.

Before:
```
du -h tests/pcap/
200M	tests/pcap/
```
After:
```
du -h tests/pcap/
98M	tests/pcap/
```
2023-01-05 16:43:35 +01:00
Ivan Nardi 5a5e01a9e2 PPSTREAM: fix some heap-buffer overflows (#1846)
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=54802
2023-01-05 16:41:53 +01:00
Ege ÇetinandNardi Ivan 663df7e475 Fix missing bracket at ppstream (#1843)
* add missing bracket

* Sync unit test results

Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2023-01-03 20:11:24 +01:00
Ivan Nardi 9c83f0f3c5 Fix some warnings and two errors found while fuzzing (#1844)
Fix CI
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=54614
2023-01-03 13:21:54 +01:00
Luca Deri cb15d2b328 Added missing prototype 2023-01-03 12:58:52 +01:00
Luca Deri 2c551afbd7 Added NDPI_MINOR_ISSUES risk used for storing generic/relevant information about issues found on traffic. 2022-12-31 16:55:55 +01:00
Luca Deri 1735931f67 Added NDPI_PERIODIC_FLOW flow risk to be used by apps based on nDPI 2022-12-30 19:20:07 +01:00
Luca Deri 8f91b8ba72 Implemented EDNS(0) support in DNS dissector
Improved DNS dissection
2022-12-29 19:38:25 +01:00
Ivan Nardi 560280e6f0 fuzz: add fuzzer testing nDPI (initial) configurations (#1830)
The goal of this fuzzer is to test init and deinit of the library, with
different configurations. In details:
* random memory allocation failures, even during init phase
* random `ndpi_init_prefs` parameter of `ndpi_init_detection_module()`
* random LRU caches sizes
* random bitmask of enabled protocols
* random parameters of `ndpi_set_detection_preferences()`
* random initialization of opportunistic TLS
* random load/don't load of configuration files

This new fuzzer is a C++ file, because it uses `FuzzedDataProvider`
class (see
https://github.com/google/fuzzing/blob/master/docs/split-inputs.md).
Note that the (existing) fuzzers need to be linked with C++ compiler
anyway, so this new fuzzer doesn't add any new requirements.
2022-12-23 19:07:13 +01:00
Ivan Nardi 3de76812d9 Remove some old protocols (#1832)
These protocols:
* have been addeded in the OpenDPI era
* have never been updated since then
* we don't have any pcap examples [*]

If (and it is a big if...) some of these protocols are still somehow
used and if someone is still interested in them, we can probably
re-add them starting from scratch (because the current detection
rules are probably outdated)

Protocols removed: DIRECT_DOWNLOAD_LINK, APPLEJUICE, DIRECTCONNECT,
OPENFT, FASTTRACK, SHOUTCAST, THUNDER, AYIYA, STEALTHNET, FIESTA,
FLORENSIA, AIMINI, SOPCAST

PPSTREAM dissector works (...) only on UDP.

[*]: with do have an AIMINI test pcap but it was some trivial http
traffic detected only by hostname matching, on domains no more
available...
2022-12-23 10:04:08 +01:00
Ivan Nardi 5fafe8374a postgres: improve detection (#1831)
Remove some dead code (found via coverage report)
2022-12-22 21:41:32 +01:00
Ivan Nardi e9d5e72fb5 Fix infinite loop when a custom rule has port 65535 (#1833)
Close #1829
2022-12-21 10:55:14 +01:00
Ivan Nardi 22821241e2 Fix some errors found by oss-fuzz (#1834) 2022-12-20 22:50:18 +01:00
Ivan Nardi fa67166fcc Remove any references to LGTM (#1837) 2022-12-20 22:49:34 +01:00
Darryl Sokoloski b8972d1b38 Added TP-LINK Smart Home Protocol dissector. (#1841)
Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2022-12-20 20:46:06 +01:00
Ivan Nardi d8de8eba36 Sync unit tests results (#1840)
Update the documentation, hinting how to test/update *all* the unit
tests.
2022-12-19 17:57:44 +01:00
Darryl Sokoloski 9854aed408 Added TUYA LAN Protocol dissector. (#1838)
Tuya IoTOS Embedded Wi-Fi and BLE SDK for bk7231n.  Used by many "smart"
devices such as LED light strips, bulbs, etc.

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2022-12-19 17:02:19 +01:00
Ivan Nardi ad6bfbad4d Add protocol disabling feature (#1808)
The application may enable only some protocols.
Disabling a protocol means:
*) don't register/use the protocol dissector code (if any)
*) disable classification by-port for such a protocol
*) disable string matchings for domains/certificates involving this protocol
*) disable subprotocol registration (if any)

This feature can be tested with `ndpiReader -B list_of_protocols_to_disable`.

Custom protocols are always enabled.

Technically speaking, this commit doesn't introduce any API/ABI
incompatibility. However, calling `ndpi_set_protocol_detection_bitmask2()`
is now mandatory, just after having called `ndpi_init_detection_module()`.

Most of the diffs (and all the diffs in `/src/lib/protocols/`) are due to
the removing of some function parameters.

Fix the low level macro `NDPI_LOG`. This issue hasn't been detected
sooner simply because almost all the code uses only the helpers `NDPI_LOG_*`
2022-12-18 08:10:57 +00:00
Luca Deri fb0a73c0c7 Updated decription 2022-12-17 19:36:00 +01:00
Ivan Nardi 3c4b12e061 README: another try to fix build badge (#1836) 2022-12-16 12:22:07 +01:00
Ivan Nardi 390dc1d155 README: fix build badge link (#1835)
See: https://github.com/badges/shields/issues/8671
2022-12-16 12:04:16 +01:00
Luca Deri de289f1305 Added ability to decode DNS PTR records used for reverse address resolution 2022-12-15 21:22:53 +01:00
Luca Deri dbff2fbadd Minor RTP fix 2022-12-15 19:17:05 +01:00
Ivan Nardi 5704e4c142 STUN: add detection of ZOOM peer-to-peer flows (#1825)
See: "Enabling Passive Measurement of Zoom Performance in Production Networks"
https://dl.acm.org/doi/pdf/10.1145/3517745.3561414
2022-12-11 23:07:35 +01:00
Luca Deri 2edbe3c11f Centos7 fix 2022-12-11 10:24:42 +01:00
Luca Deri 88d4a25cb3 Rocky9 fix 2022-12-11 10:21:44 +01:00
Ivan Nardi 48a7f6d487 fuzz: some enhancements (#1827)
Load some custom configuration (like in the unit tests) and factorize some
(fuzzing) common code.

There is no way to pass file paths to the fuzzers as parameters. The safe
solution seems to be to load them from the process working dir. Anyway,
missing file is not a blocking error.

Remove some dead code (found looking at the coverage report)
2022-12-10 19:49:11 +01:00
Ivan Nardi 8c7071e040 DTLS: handle (certificate) fragments (#1811)
Keep using the existing function to handle reassembling buffer: rename
it from `ndpi_search_tls_tcp_memory` to
`ndpi_search_tls_memory` and make it "transport" agnostic
2022-12-10 18:32:25 +01:00
Ivan Nardi 79931b2ec7 Fix undefined-behavior when setting empty user-agent (#1821)
```
ndpi_main.c:9111:35: runtime error: null pointer passed as argument 2, which is declared to never be null
/usr/include/string.h:44:28: note: nonnull attribute specified here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ndpi_main.c:9111:35 in
```
2022-12-10 18:31:51 +01:00
Luca Deri 9dc433f709 Packaing changes 2022-12-10 13:14:33 +01:00
Ivan Nardi 9f7ef72332 fuzz: fix signed-integer-overflow (#1822)
```
fuzz_ndpi_reader.c:33:29: runtime error: signed integer overflow: 214013 * 24360337 cannot be represented in type 'int'
    #0 0x4c1cf7 in fastrand ndpi/fuzz/fuzz_ndpi_reader.c:33:29
    #1 0x4c1cf7 in malloc_wrapper ndpi/fuzz/fuzz_ndpi_reader.c:38:11
    #2 0x523057 in ndpi_malloc ndpi/src/lib/ndpi_main.c:220:25
```
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=54112
2022-12-10 09:18:36 +01:00
Ivan Nardi 6004e55d7b Build: fix make install when --prefix= is used in configure script (#1824)
Use the same logic already used in `example/Makefile.in`

Close #1823
2022-12-10 09:13:20 +01:00
Luca Deri eacc2b8e32 Added Zoom screen share detection 2022-12-09 21:32:45 +01:00
Luca Deri fc7b070030 Added RTP stream type in flow metadata 2022-12-09 14:26:53 +01:00
Luca Deri 63f349319f Improved Zoom protocol detection 2022-12-08 22:00:24 +01:00
Ivan Nardi ada4fe4aa8 fuzz: add a new fuzzer testing memory allocation failures (#1818)
Try to fuzz error paths triggered by allocation errors.
Fix some errors already found by this new fuzzer.
Basic idea taken from: https://github.com/harfbuzz/harfbuzz/pull/2566/files

`FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` is a standard define used to
(not)compile specific code in fuzzing builds.
See: https://llvm.org/docs/LibFuzzer.html
2022-12-06 17:41:58 +01:00
Ivan Nardi 946c3dba0f Fix compilation (#1819) 2022-12-05 22:33:29 +01:00
Luca Deri 57562b3dfe Updated test results 2022-12-05 21:32:45 +01:00
Luca Deri e0afc16aa2 Exported HTTP server in metadata 2022-12-05 21:27:30 +01:00
Gowa2017 c882120afd thread_index may by negative. (#1814)
* thread_index may by negative.

Like:
192.168.8.155:55848 --> 183.3.224.139

* reader thread index also need to uint32
2022-12-05 10:22:05 +01:00
Ivan Nardi 3e4ab39b52 Add support for LTO and Gold linker (#1812)
This commit add (optional) support for Link-Time-Optimization and Gold
linker.
This is the first, mandatory step needed to make nDPI compliant with
"introspector" sanitizer requirements in OSS-Fuzz: see
https://github.com/google/oss-fuzz/issues/8939

Gold linker is not supported by Windows and by macOS, so this feature is
disabled by default. It has been enable in CI in two linux targets
("latest" gcc and clang).

Fix some warnings triggered by LTO.

The changes in `src/lib/ndpi_serializer.c` seams reasonable.
However, the change in `tests/unit/unit.c` is due to the following
warning, which seems to be a false positive.

```
unit.c: In function ‘serializerUnitTest’:
ndpi_serializer.c:2258:13: error: ‘MEM[(struct ndpi_private_serializer *)&deserializer].buffer.size’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
unit.c:67:31: note: ‘MEM[(struct ndpi_private_serializer *)&deserializer].buffer.size’ was declared here
   67 |   ndpi_serializer serializer, deserializer;
      |                               ^
ndpi_serializer.c:2605:10: error: ‘MEM[(struct ndpi_private_serializer *)&deserializer].status.buffer.size_used’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
unit.c:67:31: note: ‘MEM[(struct ndpi_private_serializer *)&deserializer].status.buffer.size_used’ was declared here
   67 |   ndpi_serializer serializer, deserializer;
```
Since this warning is triggered only with an old version of gcc and
`tests/unit/unit.c` is used only during the tests, the easiest fix has
been applied.

Some (unknown to me) combinations of OS and compiler trigger the
following warnings at linker time (with sanitizer and gold linker)
```
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load1_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load2_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load4_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load8_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load16_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store1_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store2_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store4_asm'
[..]
```
I have not found any references to this kind of message, with the only
exception of https://sourceware.org/bugzilla/show_bug.cgi?id=25975
which seems to suggest that these messages can be safely ignored.
In any case, the compilation results are sound.

Fix `clean` target in the Makefile in the `example` directory.

In OSS-Fuzz enviroments, `fuzz_ndpi_reader` reports a strange link error
(as always, when the gold linker is involved...).
It's come out that the culprit was the `tempnam` function: the code has
been changed to use `tmpfile` instead. No sure why... :(

Fuzzing target `fuzz_ndpi_reader.c` doesn't use `libndpiReader.a`
anymore: this way we can use `--with-only-libndpi` flag on Oss-Fuzz builds
as workaround for the "missing dependencies errors" described in
https://github.com/google/oss-fuzz/issues/8939
2022-12-05 10:21:42 +01:00
Ivan Nardi b9f63458e6 CI: GitHub is updating Ubuntu runners (#1817)
GitHub is moving `ubuntu-latest` to `ubuntu-22.04`: update our
dependencies.
See: https://github.blog/changelog/2022-11-09-github-actions-ubuntu-latest-workflows-will-use-ubuntu-22-04/

This is the reason of the recent random failures in CI.

Update "newest" tested gcc to gcc-12.

Fix a memory error introduced in 557bbcfc5a
2022-12-03 18:29:44 +01:00
Ivan Nardi 81e1ea545c Make LRU caches ipv6 aware (#1810)
Simplest solution, keeping the existing cache data structure

TLS certificate cache is used for DTLS traffic, too.

Note that Ookla cache already works with ipv6 flows.

TODO:
 * make the key/hashing more robust (extending the key size?)
 * update bittorrent cache too. That task is quite difficult because
   ntopng uses a public function (`ndpi_guess_undetected_protocol()`)
   intrinsically ipv4 only...
2022-12-03 12:07:32 +01:00
Luca Deri a387072872 Updated tests results 2022-12-03 11:33:04 +01:00
Luca Deri 2a8d0d1fe4 Added check for empty/missing user-agent in HTTP 2022-12-03 11:32:39 +01:00
Luca Deri 557bbcfc5a Improved HTTP header parsing skipping lines that won't match 2022-12-03 11:32:29 +01:00
Ivan Nardi 197a9a6cfc Extend CodeQL configuration and fix two warnings (#1816)
```
Call to `memset` may be deleted
Comparison of narrow type with wide type in loop condition
```

See https://www.usenix.org/system/files/conference/usenixsecurity17/sec17-yang.pdf
for the new portable function `secure_memzero`.
2022-12-02 23:26:22 +01:00
Alvaro Muñoz e49fe32f77 Add CodeQL workflow (#1815) 2022-12-02 13:06:54 +01:00
Ivan Nardi b2a1bf413b Remove some useless fuzzing target (#1813)
We don't need specific targets to reproduce fuzzing issues.
After all, calling `./fuzz/fuzz_process_packet_with_main $ARTIFACT_FILE`
is equivalento to `./fuzz/fuzz_process_packet $ARTIFACT_FILE`
2022-12-01 14:39:44 +01:00
Ivan Nardi 02e7e3c23b Add a new fuzzer for QUIC (#1800)
QUIC packets are encrypted/obfuscated; that means that we need to
decrypt them before parsing the real (TLS) message.
Fuzzing is not effective here, since a random buffer is hardly a valid
encrypted QUIC packet.

Add a new fuzzer, testing *decrypted* QUIC packets.

Add a basic corpus.

Fix a few bugs already found by this fuzzer.
2022-12-01 12:21:04 +01:00
Ivan Nardi afb5c3e6ba Fix classification "by-port" of QUIC flows (#1785)
nDPI is able to properly classify QUIC flows only if it elaborates the
very first packets of the flow.

The protocol list in `is_udp_guessable_protocol()` is basically a list
of protocols which can be detected from *any* packets in the flow.
Rename such function to `is_udp_not_guessable_protocol()`: the name is
still quite cryptic, but at least not plainly wrong
2022-11-22 18:00:02 +01:00
Ivan Nardi 6e6f1e4489 GitHub Actions: fix a warning (#1807)
```
The `set-output` command is deprecated and will be disabled soon.
Please upgrade to using Environment Files. For more information see:
https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
```
2022-11-22 16:44:08 +01:00
Ivan Nardi 2b65a4e05e TLS: be sure to always set ssl_version field (#1806)
Useful with asymmetric traffic with (D)TLS <= 1.2
2022-11-22 16:13:28 +01:00
Luca Deri 4231f48059 Added support for Linux Cooked Capture v2 2022-11-16 17:48:28 +01:00
Ivan Nardi 388d01a345 DNS: fix an infinite loop, due to an integer overflow (#1804)
Found by sydr-fuzz
Close #1803
2022-11-15 16:29:10 +01:00
Toni b5e8d86def Improved Teamspeak3 License/Weblist detection. (#1802)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-11-15 08:59:45 +01:00
Ivan Nardi cd41ab7c8f Improve export/print of L4 protocol information (#1799)
Close #1797
2022-11-13 22:35:46 +01:00
Ivan Nardi 07872b3527 GitHub Actions: update to Node.js 16 actions (#1801)
Fix warnings on recent CI results; example:
https://github.com/ntop/nDPI/actions/runs/3455588082

See: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/
2022-11-13 19:42:13 +01:00
Toni 6d3446b114 Merge pull request #1796 from IvanNardi/issue-1792
RTP: remove a superfluous check
2022-11-07 20:38:40 +01:00
Nardi Ivan 52b562c328 Fix json export of ipv6 addresses
The "string" buffer was to short; better start using `INET6_ADDRSTRLEN`
as reported in the man page of `inet_ntop`.

Close: #1794
2022-11-07 20:36:55 +01:00
Nardi Ivan aa5dd25392 RTP: remove a superfluous check
Static analyzer complains about dereferencing `packet->udp` before
checking.
Since this function is called only with UDP flows, remove the check.

Close: #1792
2022-11-07 18:47:19 +01:00
Zied Aouini 9e73d61fb0 Fix supported versions. 2022-10-31 13:53:23 +01:00
Zied Aouini 41a5ba1b65 Add a note about required Python version. 2022-10-31 13:50:59 +01:00
Ivan Nardi db9f6ec1b4 Add basic profiling of memory allocations on data-path (#1789)
The goal is to have an idea of the memory allocation sizes performed in
the **library data-path**, i.e. excluding init/deinit phases and all
the allocations made by the application itself.
In other words, how much memory is needed per-flow, by nDPI, other than
`struct ndpi_flow_struct`?

It works only on single-thread configurations.

It is not enabled by default (in the unit tests) since different
canfiguration options (example: `--enable-pcre`) lead to diffferent
results.

See: #1781
2022-10-28 20:41:37 +02:00
Luca Deri 175981a422 Removed unused variable 2022-10-26 16:14:32 +02:00
Alfredo Cardigliano f30479b4a4 Fix typo 2022-10-26 16:05:32 +02:00
Luca Deri a16cd9634f Cleanup 2022-10-26 13:59:07 +02:00
Luca Deri 15d58db2f9 Typo 2022-10-26 13:56:24 +02:00
Luca Deri 55f885f3ed Improved AESNI check 2022-10-26 13:51:00 +02:00
Ivan Nardi de16fd35aa Hangout: avoid useless lookups in the protocol tree (#1755)
We already performed exactly these lookups in the generic code to
populate `flow->guessed_protocol_id_by_ip`: use it!

This code probably needs a deeper review, since it is basicaly a simple
matching on ip + port.
2022-10-25 18:24:03 +02:00
Ivan Nardi ca5ffc4988 TLS: improve handling of ALPN(s) (#1784)
Tell "Advertised" ALPN list from "Negotiated" ALPN; the former is
extracted from the CH, the latter from the SH.

Add some entries to the known ALPN list.

Fix printing of "TLS Supported Versions" field.
2022-10-25 17:06:29 +02:00
Ivan Nardi 2ed2e5dc7e Fix memory corruption on TCP flows (#1786) 2022-10-25 16:22:40 +02:00
Ivan Nardi 550f3e5b9d IRC: fix memory access (#1787) 2022-10-25 16:16:24 +02:00
Ivan Nardi e674a8ae37 Sync unit tests results (#1783) 2022-10-22 12:57:06 +02:00
Luca Deri 24cc949f14 Enhanced HTTP numeric IP check 2022-10-22 10:06:09 +02:00
Toni c521595383 Add Elasticsearch protocol dissector. (#1782)
* all credits goes to @verzulli

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-21 20:01:54 +02:00
Toni Uhlig 29242cbcb6 Add Munin protocol dissector.
* all credits goes to @verzulli

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-20 22:10:35 +02:00
Nardi Ivan 9c0caa5362 TLS: allow sub-classification via ALPN
In some rare cases, it is possible to sub-classify the flow via ALPN
matching. This is particularly usefull for asymmetric traffic where the
Client Hello doens't have the SNI.

For the time being there is only one rule, about ANYDESK.
2022-10-20 22:00:12 +02:00
Nardi Ivan c1b6aa9483 Sync unit tests results 2022-10-19 13:01:34 +02:00
Luca Deri 9a34d870e9 Updated FortiClient default ports 2022-10-19 11:25:19 +02:00
Luca Deri df0dd76ed2 Reworked severities adding critical and emergency, thus shifting values down 2022-10-19 10:53:11 +02:00
Ivan Nardi 35fdbc8148 TLS: explicit ignore client certificate (#1776)
TLS classification usually stops after processing *server* certificates
(if any). That means, that *client* certificate, if present, is usually
ignored.
However in some corner cases (i.e. unidirectional traffic) we might end
up processing client certificate and exposing its metadata: the issue is
that the application will think that this metadata are about the server
and not about the client.
So, for the time being, always ignore client certificate processing.

As a future work, we might find an efficient way to process and export both
certificates.
2022-10-18 16:40:15 +02:00
Nardi Ivan 223a6fb9f7 Fix a use-of-uninitialized-value error on PCRE code
This is likely a false positive, triggered by the fact that libpcre is
usually compiled without MASAN support.
It it was a real error, ASAN would complain loudly with a invalid-free
error at the end of the same function.

```
==83793==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x562296111174 in ndpi_compile_rce_regex /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1631:3
    #1 0x5622960e3e4a in ndpi_is_rce_injection /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1636:5
    #2 0x5622960de7cd in ndpi_validate_url /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1741:12
    #3 0x5622960dae45 in ndpi_dpi2json /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1362:29
    #4 0x5622960e2751 in ndpi_flow2json /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1512:10
    #5 0x562296033b0f in process_ndpi_collected_info /home/ivan/svnrepos/nDPI/example/reader_util.c:1310:9
    #6 0x5622960501f9 in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1659:2
    #7 0x562296045aef in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2202:10
    #8 0x562295e85374 in ndpi_process_packet /home/ivan/svnrepos/nDPI/example/ndpiReader.c:3937:7
    #9 0x7f1235053466  (/lib/x86_64-linux-gnu/libpcap.so.0.8+0x23466) (BuildId: b84c893ea2516d6fb2c1c6726b1fe93b3be78f61)
    #10 0x7f1235041f67 in pcap_loop (/lib/x86_64-linux-gnu/libpcap.so.0.8+0x11f67) (BuildId: b84c893ea2516d6fb2c1c6726b1fe93b3be78f61)
    #11 0x562295e53139 in runPcapLoop /home/ivan/svnrepos/nDPI/example/ndpiReader.c:4060:15
    #12 0x562295e51e7f in processing_thread /home/ivan/svnrepos/nDPI/example/ndpiReader.c:4130:3
    #13 0x7f1234e53608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
    #14 0x7f1234d2f132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

  Uninitialized value was created by an allocation of 'pcreErrorStr' in the stack frame
    #0 0x5622961108a6 in ndpi_compile_rce_regex /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1603:3
```
2022-10-17 23:15:07 +02:00
Nardi Ivan 3d0c36cf22 Add some popular streaming applications
A simply back-porting from https://github.com/vel21ripn/nDPI/pull/142;
full credits to @ChrisNelson-CyberReef.

Close #1716
2022-10-17 23:08:25 +02:00
Toni 01622c4c63 Updated protocol dissector serialisation code. (#1767)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-15 12:58:44 +02:00
Ivan Nardi 2d153fba45 HTTP: fix stack-buffer-overflow (#1768)
```
==24879==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fa085b31e60 at pc 0x55cc63f203e2 bp 0x7ffc9ec91b10 sp 0x7ffc9ec91298
READ of size 17 at 0x7fa085b31e60 thread T0
    #0 0x55cc63f203e1 in printf_common(void*, char const*, __va_list_tag*) asan_interceptors.cpp.o
    #1 0x55cc63f20769 in vsnprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x50e769) (BuildId: cce2b6b1344bfd0bdc9626fef604c2b3caad485b)
    #2 0x55cc63f22210 in __interceptor_snprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x510210) (BuildId: cce2b6b1344bfd0bdc9626fef604c2b3caad485b)
    #3 0x55cc6420fc76 in ndpi_check_http_server /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:668:4
    #4 0x55cc6420344b in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:742:5
    #5 0x55cc642031ce in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:737:7
    #6 0x55cc641fac9f in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1352:4
    #7 0x55cc641f2fd5 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1461:3
    #8 0x55cc64085275 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5580:6
    #9 0x55cc64085c87 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5628:12
    #10 0x55cc64085927 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5647:12
    #11 0x55cc64095fcb in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6458:15
    #12 0x55cc63fd08b4 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #13 0x55cc63fd09f7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #14 0x7fa0880fb082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #15 0x55cc63efb45d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4e945d) (BuildId: cce2b6b1344bfd0bdc9626fef604c2b3caad485b)

Address 0x7fa085b31e60 is located in stack of thread T0 at offset 96 in frame
    #0 0x55cc6420f1bf in ndpi_check_http_server /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:644

  This frame has 5 object(s):
    [32, 36) 'a' (line 653)
    [48, 52) 'b' (line 653)
    [64, 68) 'c' (line 653)
    [80, 96) 'buf' (line 654)
    [112, 176) 'msg' (line 662) <== Memory access at offset 96 partially underflows this variable

```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=52229
2022-10-14 18:59:33 +00:00
Ivan Nardi 6c84ce85e4 ndpiReader: fix help message. There isn't a 'J' option (#1770) 2022-10-14 20:16:47 +02:00
Ivan Nardi 8c4ece941e Sync utests results (#1774) 2022-10-14 20:16:29 +02:00
Luca Deri 3bb6bf628d Removed gettimeofday as win files has located in windows/src/ 2022-10-14 19:01:41 +02:00
Luca Deri 1fecf69359 Added DNS error code mapping number -> string 2022-10-14 09:39:15 +02:00
Luca Deri cd76bacd4a Added detection of URLs starting with a dot (.) 2022-10-10 22:34:26 +02:00
Ivan Nardi 1796a1d814 LINE_CALL: add detection of LINE voip calls (#1761)
These flows are classifed as `LINE_CALL`; another option was
`RTP/LINE_CALL`. No sure about the best solution...

Extend LINE domains list.

Remove RTP dead code.
2022-10-06 17:09:26 +02:00
Luca Deri e4dcec560e Windows changes 2022-10-06 11:58:16 +02:00
Ivan Nardi c4fb2a590e irc: fix heap-buffer-overflow (#1765) 2022-10-06 09:25:21 +00:00
Luca Deri 3c0021d603 Fixes Stack-buffer-overflow READ {*} · check_content_type_and_change_protocol 2022-10-06 09:16:02 +02:00
Luca Deri fba926d6a4 Updated IRC result 2022-10-05 23:02:02 +02:00
Luca Deri 199c86e3df Improved credentials check in IRC 2022-10-05 22:51:20 +02:00
Luca 37c88b129f Added new flow risk NDPI_HTTP_OBSOLETE_SERVER. Currently Apache and nginx are supported 2022-10-04 22:32:45 +02:00
Nardi Ivan cca585053e Fix compilation and sync utests results 2022-10-04 22:17:05 +02:00
Luca 131d53896f Improved HTTP payload handling 2022-10-04 14:03:39 +02:00
Luca 3fa9416981 Improved detection of WebShell and PHP code in HTTP URLs that is reported via flow risk 2022-10-04 11:26:44 +02:00
Luca de59eb8237 Added the ability to track the payload via -E and via the new option 'ndpi_track_flow_payload' 2022-10-04 11:26:44 +02:00
Toni 66a8010ef7 Fix ndpi_timeval_to_milliseconds/microseconds for platforms with tv_usec is an unsigned int. (#1762)
* implicit conversion to an 32 bit unsigned int leads to an overflow

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-02 12:48:24 +02:00
Toni c83e0b3c8d Restore Confidence enum constants to ascending values. (#1760)
* NDPI_CONFIDENCE_MAX is useless otherwise since it is
   supposed the be the highest value w/o any gaps
 * for client applications, it makes sense to use that
   the numbers directly w/o any additional mapping

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-02 12:47:56 +02:00
sharonenochandSharon Enoch 503aac70bc Line app support (#1759)
* Standard support for LINE app

* Added test pcap for LINE app

* make check result for LINE app

* Make check success as 1kxun has LINE packets

* Added the ASN inc file for LINE

* Removed extra lines as its effecting make check

* Editing the SNI required a new pcap output file for TLS.Line format

* Run Configure with --with-pcre --with-maxminddb to enable the generation of h323-overflow.pcap.out

Co-authored-by: Sharon Enoch <sharone@amzetta.com>
2022-10-01 12:01:41 +02:00
Nardi Ivan f3a74d97d8 TLS/DTLS: we process certificate for UDP flows, too
Note that current code access `certificate_processed` state even before
setting the protocol classification, so this piece of information can't
be saved in `flow->protos` union.
2022-09-30 06:53:29 +02:00
Luca fbf30d1aad Fixed invalid unidirectional traffic alert for unidirectional protocols (e.g. sFlow) 2022-09-29 18:15:37 +02:00
Nardi Ivan a27538b764 Remove unused code
LRU callbacks have been added in 460ff3c7a, but they have never been
used and they have never been extended to the other LRU caches.

`ndpi_search_tcp_or_udp()` basically returns the classification by port/ip
of the flow; calling it from the dissector is useless.
The same for TOR detection: ips are checked in the generic code
2022-09-28 21:09:27 +02:00
Nardi Ivan ac7a3669da Fix value of ndpi_protocol->protocol_by_ip
Fix: a7c2734b
2022-09-25 13:31:57 +02:00
Nardi Ivan 39f9b8cadf DNS: change category of DNS flows
DNS flows should have `NDPI_PROTOCOL_CATEGORY_NETWORK` as category,
regardless of the subprotocol (if any).
Follow-up of 83de3e47
2022-09-25 13:31:04 +02:00
Toni Uhlig b7b9577254 Changed some protocol categories to align with others.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-23 20:35:55 +02:00
Nardi Ivan 1f345b311f Sizes of LRU caches are now configurable
0 as size value disable the cache.

The diffs in unit tests are due to the fact that some lookups are
performed before the first insert: before this change these lookups
weren't counted because the cache was not yet initialized, now they are.
2022-09-23 18:33:48 +02:00
Nardi Ivan 83de3e4716 DNS: change category of DNS flows
DNS flows should have `NDPI_PROTOCOL_CATEGORY_NETWORK` as category,
regardless of the subprotocol (if any).
2022-09-22 13:23:55 +02:00
Nardi Ivan b250f9d86c Extend content match lists 2022-09-22 09:30:09 +02:00
Nardi Ivan c6201284d7 NAT-PMP: fix metadata extraction 2022-09-21 20:02:23 +02:00
Toni 644ad34962 Improved NATPMP dissection. (#1745)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:24:04 +02:00
Toni Uhlig d6701e8979 Build ndpiReader and run regression tests.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:03:22 +02:00
segfault 3db6413ab0 Fixed VS2019 build.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:03:22 +02:00
Toni Uhlig 50ee6768b7 Provide extra ndpi_define.h for Visual Studio.
* Caution: file needs to keep in sync with `src/include/ndpi_define.h.in` manually

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:03:22 +02:00
Toni Uhlig d27fb1e73e Moved NDPI_API_VERSION to src/include/ndpi_define.h.in where it belongs.
That way, we can make `src/include/ndpi_api.h` independent of autoconf.
MSBuild will like that.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:03:22 +02:00
segfault 817a7c616d Retargeted to Windows 10.
* disable Spectre Mitigation (may change)
 * remove autoconf generated header files
2022-09-21 18:03:22 +02:00
mark andrews 5d5b46e514 Add proj and sln for compile of dynamic x64 lib under Visual Studio 2019.
* add CI support via MSBuild

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:03:22 +02:00
Ivan Nardi 03d217eae6 Fix CI after nBPF integration (#1746)
Add one CI job testing nBPF
2022-09-21 16:49:51 +02:00
Luca Deri c960bb50f0 Copilation fix 2022-09-21 10:27:46 +02:00
Luca Deri aef8e53b7d Added missing library 2022-09-21 08:24:37 +02:00
Luca Deri 1d0bc7c350 Finalized nBPF support. You can now define custom protocols such as
(see exaple/protos.txt)

nbpf:"host 192.168.1.1 and port 80"@HomeRouter

In order to have nBPF support, you need to compile nDPI with it. Just download
https://github.com/ntop/PF_RING in the same directory where you have downloaded
nDPI and compile PF_RING/userland/nbpf
2022-09-21 00:03:12 +02:00
Ivan Nardi a7c2734b38 Remove classification "by-ip" from protocol stack (#1743)
Basically:
* "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is
NEVER returned in the protocol stack (i.e.
`flow->detected_protocol_stack[]`);
* if the application is interested into such information, it can access
`ndpi_protocol->protocol_by_ip` itself.

There are mainly 4 points in the code that set the "classification
by-ip" in the protocol stack:  the generic `ndpi_set_detected_protocol()`/
`ndpi_detection_giveup()` functions and the HTTP/STUN  dissectors.

In the unit tests output, a print about `ndpi_protocol->protocol_by_ip`
has been added for each flow: the huge diff of this commit is mainly due
to that.

Strictly speaking, this change is NOT an API/ABI breakage, but there are
important differences in the classification results. For examples:
* TLS flows without the initial handshake (or without a matching
SNI/certificate) are simply classified as `TLS`;
* similar for HTTP or QUIC flows;
* DNS flows without a matching request domain are simply classified as
`DNS`; we don't have `DNS/Google` anymore just because the server is
8.8.8.8 (that was an outrageous behaviour...);
* flows previusoly classified only "by-ip" are now classified as
`NDPI_PROTOCOL_UNKNOWN`.

See #1425 for other examples of why adding the "classification by-ip" in
the protocol stack is a bad idea.

Please, note that IPV6 is not supported :(  (long standing issue in nDPI) i.e.
`ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for
IPv6 flows.

Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed.

Close #1687
2022-09-20 22:24:47 +02:00
Toni 174cd739db Fix warnings on OpenWrt CCs, caused by the memcpy builtin. (#1741)
In file included from ../include/ndpi_includes.h:31,
                 from ../include/ndpi_main.h:27,
                 from ../include/ndpi_api.h:28,
                 from protocols/quic.c:27:
In function 'memcpy',
    inlined from 'tls13_hkdf_expand_label_context' at protocols/quic.c:473:5,
    inlined from 'tls13_hkdf_expand_label' at protocols/quic.c:498:10,
    inlined from 'quic_hkdf_expand_label.constprop' at protocols/quic.c:512:6:
/home/build/openwrt/staging_dir/toolchain-mips_24kc_gcc-11.3.0_musl/include/fortify/string.h:53:16: error: argument 2 null where non-null expected [-Werror=nonnull]
   53 |         return __builtin_memcpy(__od, __os, __n);
      |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
protocols/quic.c: In function 'quic_hkdf_expand_label.constprop':
/home/build/openwrt/staging_dir/toolchain-mips_24kc_gcc-11.3.0_musl/include/fortify/string.h:53:16: note: in a call to built-in function '__builtin_memcpy'

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-18 12:40:49 +02:00
Luca Deri 48d00eae35 Added MAX_NBPF_CUSTOM_PROTO define 2022-09-17 11:04:40 +02:00
Luca 0f07ad35be Initial work towards custom BPF protocol definition 2022-09-16 19:48:12 +02:00
Toni Uhlig a966d37a21 Add CryNetwork dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-16 16:48:43 +02:00
Toni Uhlig ec1981c20c Add Syncthing dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-16 15:21:49 +02:00
Toni de020b1747 Add NATPMP dissector. (#1738)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-16 13:32:30 +02:00
Nardi Ivan 9ce4d40d14 Remove a case of guessed sub-classification
This code is triggered only for "unknown" flows with a valid
sni/hostname.
Why in that case the guessed classification should be
something like `DNS/Subprotocol_depending_on_hostname`? Why DNS as
master and not HTTP or TLS or QUIC?

Furthermore, I have not been able to trigger a positive match from that
lookup. I strongly think that if we had a valid subprotocol, we would
have a valid master in the first place.
In doubt, remove it completely.

As a follow up, we should investigate why some dissectors (the HTTP one,
at least) set the sni/hostname field without setting a valid protocol,
in the first place.
This behaviour seems quite suspicious, if not plainly buggy.
2022-09-14 17:52:01 +02:00
Nardi Ivan 7571f48392 Dissectors shouldn't update flow->guessed_host_protocol_id
The field `flow->guessed_host_protocol_id` is set at the beginning of
the flow analysis and it represents the "classification by ip" of the flow
itself.

This field should never be changed. Dissectors which want to provide an
"hint" about the classification, should update `flow->guessed_protocol_id`
instead. Such "hint" is useless if the dissector set the "extra-dissection"
data-path.

Rename such field to `guessed_protocol_id_by_ip` to better describe its
role.

Preliminary work necessary for #1687
2022-09-14 17:51:39 +02:00
Ivan Nardi b4f5243716 Anydesk: improve detection (#1735) 2022-09-13 20:55:11 +02:00
Nardi Ivan eea3c448de HTTP: remove some wrong code
This code seems wrong or in the wrong place, at least:
* "classification by port" and "classification by ip" protocols (i.e
"guessed" protocols) should be used to set the protocol stack only after
trying all the dissectors, and only by the generic code
* there are no reason (for a dissector) to update the "guessed"
information using the protocol stack values: it is usually the other way
around (see previous point)
2022-09-13 20:48:20 +02:00
Alfredo Cardigliano 973950d881 Replace obsolete linux macro 2022-09-13 10:41:44 +02:00
Ivan Nardi 4f584f78a0 Fix ndpi_do_guess() (#1731)
Avoid a double call of `ndpi_guess_host_protocol_id()`.
Some code paths work for ipv4/6 both
Remove some never used code.
2022-09-12 19:28:41 +02:00
Ivan Nardi 4bb81f7609 Update Github CI OS and compilers (#1733)
ubuntu-18.04 is deprecated (ubuntu-latest points to 20.04).
macos-latest points to macos-11, so it makes sense to test macos-12,
too.

About the compilers, the general idea it to test the oldest and the
newest versions easily available: switch to gcc-11 and clang-14.

See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
2022-09-12 19:11:43 +02:00
Nardi Ivan 678dd61866 STUN: several improvements
Add detection over TCP and fix detection over IPv6.
Rename some variables since Stun dissector is no more "udp-centric".
Stun dissector should always classified the flow as `STUN` or
`STUN/Something`.
Don't touch `flow->guessed_host_protocol_id` field, which should be
always be related to "ip-classification" only.
2022-09-11 13:33:32 +02:00
Nardi Ivan f44413e039 Update Changelog file (version 4.4)
Notes shameless taken from https://github.com/ntop/nDPI/releases
2022-09-10 22:51:40 +02:00
Nardi Ivan 1b3de449a1 HTTP: correctly set the classification protocols
Classification should always be set via `ndpi_set_detected_protocol()`
to be sure to set a correct `confidence` value, too.
Having a "known" protocol stack with `NDPI_CONFIDENCE_UNKNOWN` as
confidence, is not valid.

This code in HTTP dissector likely needs some more thoughts (the
classification itself of the attached example doesn't make a lot of
sense), but the goal of this commit is only to always have a valid
`confidence` value.
2022-09-06 14:58:31 +02:00
Toni 37f918322c Serialize nDPI detection/dissection data as object. (#1725)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-06 11:38:31 +02:00
Nardi Ivan e5063f9883 Fix some warnings
Ubuntu-20.04, clang-16 (nightly build)

```
Making all in src/lib
protocols/smpp.c:70:17: warning: variable 'pdu_c' set but not used [-Wunused-but-set-variable]
      u_int16_t pdu_c = 1;
                ^
1 warning generated.
third_party/src/ahocorasick.c:173:20: warning: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Wsingle-bit-bitfield-constant-conversion]
  thiz->root->root = 1;
                   ^ ~
third_party/src/ahocorasick.c:336:15: warning: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Wsingle-bit-bitfield-constant-conversion]
        n->ff = 1;
              ^ ~
third_party/src/ahocorasick.c:716:21: warning: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Wsingle-bit-bitfield-constant-conversion]
        node->final = 1;
[...]
```
2022-09-05 23:44:51 +02:00
Ivan Nardi 0a47f745cc Avoid useless host automa lookup (#1724)
The host automa is used for two tasks:
* protocol sub-classification (obviously);
* DGA evaluation: the idea is that if a domain is present in this
automa, it can't be a DGA, regardless of its format/name.

In most dissectors both checks are executed, i.e. the code is something
like:

```
ndpi_match_host_subprotocol(..., flow->host_server_name, ...);
ndpi_check_dga_name(..., flow->host_server_name,...);

```

In that common case, we can perform only one automa lookup: if we check the
sub-classification before the DGA, we can avoid the second lookup in
the DGA function itself.
2022-09-05 13:59:51 +02:00
Toni 01028ee779 Fix string copy truncation warning. (#1723)
protocols/ubntac2.c: In function ‘ndpi_search_ubntac2’:
protocols/ubntac2.c:69:4: warning: ‘strncpy’ output may be truncated copying between 0 and 31 bytes from a string of length 255 [-Wstringop-truncation]
   69 |    strncpy(flow->protos.ubntac2.version, (const char *)version, len);
      |    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-05 11:16:06 +02:00
Toni 378b0c5953 Fixed FastCGI memory issue (was not using nDPI's malloc wrapper). (#1722)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-05 10:46:02 +02:00
Nardi Ivan 95c1daa851 Fix endianess in IP-Port lookup
Port parameter in `ndpi_network_port_ptree_match()` must be in network
order.
2022-09-05 10:17:58 +02:00
Toni b7c1030804 Fix broken ndpi_timeval_to_(milli|micro)seconds (>UINT_MAX). (#1720)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-04 19:19:05 +02:00
Nardi Ivan b9cb391756 Add support to opportunistic TLS
A lot of protocols provide the feature to upgrade their plain text
connections to an encrypted one, via some kind of "STARTTLS" command.

Add generic code to support this extension, and allow dissection of the
entire TLS handshake.

As examples, SMTP, POP, IMAP and FTP dissectors have been updated.

Since this feature requires to process more packets per flow, add the
possibility to disable it.

Fix some log messages.

Slight improvement on TCP sequence number tracking.

As a side effect, this commit fix also a memory leak found by
oss-fuzzer
```
==108966==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 22 byte(s) in 1 object(s) allocated from:
    #0 0x55f8b367a0be in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x5480be) (BuildId: 94debacb4a6784c30420ab748c8bf3cc59621063)
    #1 0x55f8b36e1345 in ndpi_malloc_wrapper /home/ivan/svnrepos/nDPI/example/reader_util.c:321:10
    #2 0x55f8b379c7d2 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:212:25
    #3 0x55f8b379cb18 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:279:13
    #4 0x55f8b386ce46 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2153:34
    #5 0x55f8b385ebf7 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:867:5
    #6 0x55f8b39e708c in ndpi_extra_search_mail_smtp_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/mail_smtp.c:422:9
    #7 0x55f8b37e636c in ndpi_process_extra_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5884:9
    #8 0x55f8b37edc05 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6276:5
    #9 0x55f8b3701ffc in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1619:31
    #10 0x55f8b36faf14 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2189:10
    #11 0x55f8b36b6a50 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7

```
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50765
2022-09-04 17:22:19 +02:00
Luca Deri 7578d02de9 Fixes on hw calculation 2022-09-01 22:06:57 +02:00
Alfredo Cardigliano bd29fa46ba Serializer supports double for JSON and CSV only for the time being (TLV needs to be extended) 2022-09-01 14:43:52 +02:00
Alfredo Cardigliano 2372063d25 Temporarily restore ndpi_serialization_unknown = 0 2022-09-01 12:18:04 +02:00
Alfredo Cardigliano 8f410b751f Restore ndpi_serialization_type item values. Use 0 for the new ndpi_serialization_double as there is no more key left (4 bit). Use 16 for ndpi_serialization_unknown which is not used in TLV encoded data. 2022-09-01 10:37:39 +02:00
Luca Deri f684d44aa7 Removed maxmind check 2022-08-31 11:48:18 +02:00
Alfredo Cardigliano 981a72cd80 Improve AES-NI check 2022-08-30 16:37:30 +02:00
Toni f7907976c9 FastCGI: dissect PARAMS (#1715)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
2022-08-30 11:38:36 +02:00
Toni fe28d55801 Improved MGCP dissector. (#1717)
* typ0s fixed
 * dissect endpoint hostnames

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-30 11:21:58 +02:00
Nardi Ivan 79653f99e0 Internal gcrypt: fix memory leak
Fix: ef99eb67
2022-08-30 11:12:00 +02:00
Luca Deri ef99eb674e Improved AES-NI presence check on Linux 2022-08-29 17:42:28 +02:00
Nardi Ivan a329730d24 QUIC: fix heap-buffer-overflow
```
==12318==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x604000000032 at pc 0x55a59ec97959 bp 0x7fffee67fdd0 sp 0x7fffee67fdc8
READ of size 1 at 0x604000000032 thread T0
    #0 0x55a59ec97958 in may_be_0rtt /home/ivan/svnrepos/nDPI/src/lib/protocols/quic.c:1483:24
    #1 0x55a59ec9515f in ndpi_search_quic /home/ivan/svnrepos/nDPI/src/lib/protocols/quic.c:1708:13
    #2 0x55a59ec32e95 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5428:6
    #3 0x55a59ec33c5b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5464:10
    #4 0x55a59ec335fc in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5497:12
    #5 0x55a59ec44615 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6322:15
    #6 0x55a59eb8884e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #7 0x55a59eb889c7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #8 0x7fb5b3ba2082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #9 0x55a59eac742d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x47d42d) (BuildId: 712c87b21cf5c05f64174745909c693d3ba0b62e)

0x604000000032 is located 0 bytes to the right of 34-byte region [0x604000000010,0x604000000032)
allocated by thread T0 here:
    #0 0x55a59eb4bfee in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x501fee) (BuildId: 712c87b21cf5c05f64174745909c693d3ba0b62e)
    #1 0x55a59eb8899c in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:87:17

```

Found by CI tests.
See: https://github.com/ntop/nDPI/runs/7996151458?check_suite_focus=true
2022-08-25 06:44:38 +02:00
Nardi Ivan 8bfb1712d8 QUIC: add support for 0-RTT packets received before the Initial
RFC9001 4.6.1: "A client that wishes to send 0-RTT packets uses the
early_data extension in the ClientHello message of a subsequent handshake;
see Section 4.2.10 of [TLS13]. It then sends application data in 0-RTT
packets."
That means the client sends before the CH (in the Initial) and then the
0-RTT (in the same UDP datagram or not)".

However, because of packet loss or out-of-order delivery, it might
happens that a 0-RTT packet is received before the Initial (the original
one or a retransmission).
For example, Google and Facebook servers save 0-RTT packets for a small
amount of time in hopes of receiving the corresponding Initial.

Update the QUIC dissector to detect 0-RTT packets and keep looking for
the Initial.

Issue found by @utoni in #1706; the trace example has been taken from that
PR.
2022-08-24 15:38:30 +02:00
Toni 0c8bc9f055 Add FastCGI protocol detection. (#1711)
* CQL: fixed byte order conversion (BigEndian not LittleEndian)
 * CQL: increased required successful dissected packets to prevent false-positives

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 13:22:46 +02:00
Toni e135c1c5e3 Merge pull request #1712 from IvanNardi/oss-fuzzer
HTTP, SoftEther, Florensia: fix some memory corruptions
2022-08-24 11:24:25 +02:00
Toni 30730e95e5 Add Kismet protocol detection. (#1710)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 10:50:11 +02:00
Toni Uhlig 93a80a1c4b Fixed serializing bug caused by an unitialized serializer.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 10:49:27 +02:00
lns 93d65ed650 Support serialization of double-precision floating-point numbers. Fixes #1702.
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 10:49:27 +02:00
Nardi Ivan ac0d7ccb7e HTTP, SoftEther, Florensia: fix some memory corruptions
```
==12308==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600068f4b4 at pc 0x5603d2b56098 bp 0x7ffc967bd3b0 sp 0x7ffc967bd3a8
READ of size 2 at 0x60600068f4b4 thread T0
    #0 0x5603d2b56097 in ndpi_search_florensia /home/ivan/svnrepos/nDPI/src/lib/protocols/florensia.c:93:40
    #1 0x5603d2a2e835 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6
    #2 0x5603d2a2f247 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5462:12
    #3 0x5603d2a2eee7 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5481:12
    #4 0x5603d2a3ffb5 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15
    #5 0x5603d2955f4c in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1612:31
    #6 0x5603d294ee64 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2182:10
    #7 0x5603d290aa50 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #8 0x5603d290ab8b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17
    #9 0x7f972cdc9082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #10 0x5603d28494fd in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x4be4fd) (BuildId: e81373a75c70e248ba9ae985fcb4134e7113121a)
```
```
==14164==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60d0000000c1 at pc 0x561608bada2c bp 0x7ffc40d38ed0 sp 0x7ffc40d38690
READ of size 10 at 0x60d0000000c1 thread T0
    #0 0x561608bada2b in __interceptor_strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4e9a2b) (BuildId: 2e4fd67f1aeb93c491a04ae424331669c993ee9b)
    #1 0x561608e66f2a in dissect_softether_host_fqdn /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:226:2
    #2 0x561608e64e88 in ndpi_search_softether /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:316:8
    #3 0x561608ca7475 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6
    #4 0x561608ca823b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5450:10
    #5 0x561608ca7bdc in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5483:12
    #6 0x561608cb8bf5 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15
    #7 0x561608bff84e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #8 0x561608bff9c7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #9 0x7fc50662d082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #10 0x561608b3e42d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x47a42d) (BuildId: 2e4fd67f1aeb93c491a04ae424331669c993ee9b)
```

Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50105
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50110
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50268
2022-08-22 17:53:52 +02:00
Luca Deri a53f476585 Modified definition of hll_add 2022-08-17 16:25:10 +02:00
Luca Deri 82c83ef75e Updated RPM package 2022-08-15 16:29:08 +02:00
Luca Deri 4a843709fa Packages changes 2022-08-15 11:32:58 +02:00
alcir 780cbf5f76 Update Makefile.in (#1708)
Use `@libdir@` instead of *hardcoded* path, and also `@includedir@`
2022-08-15 08:20:17 +00:00
Luca Deri 743d046f6f Added nTap reference 2022-08-12 10:38:51 +02:00
Toni 5915bc8828 Improved Discord pattern. (#1707)
Signed-off-by: lns <matzeton@googlemail.com>

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-10 21:00:11 +02:00
Ivan Nardi 540848c254 HTTP: fix classification in some corner cases (#1704)
Initializing `low->detected_protocol_stack[1]` but not
`low->detected_protocol_stack[0]` lead to *very* strange errors.
Oss-fuzzer, exploiting this bug, has been able to crash the application
or to leak some memory

```
==19775==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x5624c0dd1802 bp 0x5624c197e848 sp 0x7fff4b4db690 T0)
==19775==The signal is caused by a READ memory access.
==19775==Hint: this fault was caused by a dereference of a high value address (see register values below).  Disassemble the provided pc to learn which register was used.
    #0 0x5624c0dd1802 in __asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x522802) (BuildId: 9a1ed9efcaf47e345767c86520372d28e31ca1aa)
    #1 0x5624c0e53ea6 in __interceptor_free (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x5a4ea6) (BuildId: 9a1ed9efcaf47e345767c86520372d28e31ca1aa)
    #2 0x5624c0ebb474 in free_wrapper /home/ivan/svnrepos/nDPI/example/reader_util.c:330:3
    #3 0x5624c0f75561 in ndpi_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:240:7
    #4 0x5624c0faa921 in ndpi_free_flow_data /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4915:2
    #5 0x5624c0f755ef in ndpi_free_flow /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8076:5
    #6 0x5624c0f755c4 in ndpi_flow_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:253:5
    #7 0x5624c0eba981 in ndpi_free_flow_info_half /home/ivan/svnrepos/nDPI/example/reader_util.c:305:25
    #8 0x5624c0ecb4bc in process_ndpi_collected_info /home/ivan/svnrepos/nDPI/example/reader_util.c:1317:5
    #9 0x5624c0edca36 in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1639:2
    #10 0x5624c0ed5172 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2182:10
    #11 0x5624c0e90a80 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #12 0x5624c0e90bbb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17
    #13 0x7f580731f082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #14 0x5624c0dcf52d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x52052d) (BuildId: 9a1ed9efcaf47e345767c86520372d28e31ca1aa)
```

```
==17492==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 5 byte(s) in 1 object(s) allocated from:
    #0 0x55b799b0f01e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x56101e) (BuildId: d2adbfb29a6eda6dc59fdfb8930d7e6496ac7b8b)
    #1 0x55b799bb3bf4 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:212:46
    #2 0x55b799bb3f28 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:279:13
    #3 0x55b799c8b9bf in processCertificateElements /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:683:39
    #4 0x55b799c81e89 in processCertificate /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:835:7
    #5 0x55b799c8d500 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:889:13
    #6 0x55b799ca80f1 in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1034:2
    #7 0x55b799ca3be2 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2475:5
    #8 0x55b799bf9345 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6
    #9 0x55b799bf9d57 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5462:12
    #10 0x55b799bf99f7 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5481:12
    #11 0x55b799c0b838 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15
    #12 0x55b799b4b87e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
```

```
==17511==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1 byte(s) in 1 object(s) allocated from:
    #0 0x55aed645e01e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x56101e) (BuildId: d2adbfb29a6eda6dc59fdfb8930d7e6496ac7b8b)
    #1 0x55aed6502bf4 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:212:46
    #2 0x55aed6502f28 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:279:13
    #3 0x55aed65ea0c6 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2153:34
    #4 0x55aed65dbe77 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:867:5
    #5 0x55aed65f70f1 in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1034:2
    #6 0x55aed65f2be2 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2475:5
    #7 0x55aed6548345 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6
    #8 0x55aed6548d57 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5462:12
    #9 0x55aed65489f7 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5481:12
    #10 0x55aed655a838 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15
    #11 0x55aed649a87e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
```

See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49844
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49842
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49906
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49915
2022-08-09 18:41:10 +02:00
Ivan Nardi 59a9bdeb55 SoftEtehr: fix a heap-buffer overflow (#1705)
```
==26717==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60c0000000bf at pc 0x55e503f8959b bp 0x7fff7f6876b0 sp 0x7fff7f686e58
READ of size 1 at 0x60c0000000bf thread T0
    #0 0x55e503f8959a in strncmp (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4f459a) (BuildId: 44038d7ca9391c877fae9b2735828da9905a6daa)
    #1 0x55e5042ee98f in dissect_softether_host_fqdn /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:236:8
    #2 0x55e5042ec41b in ndpi_search_softether /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:318:8
    #3 0x55e5040e0345 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6
    #4 0x55e5040e110b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5450:10
    #5 0x55e5040e0aac in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5483:12
    #6 0x55e5040f2838 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15
    #7 0x55e50403287e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #8 0x55e5040329f7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #9 0x7fd09fdbb082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #10 0x55e503f7145d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4dc45d) (BuildId: 44038d7ca9391c877fae9b2735828da9905a6daa)
```

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49901
2022-08-09 18:09:16 +02:00
Toni 2e25c36396 Add TiVoConnect dissector. Fixes #1697. (#1699)
* added static assert if supported, to complain if the flow struct changes

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-08 19:04:20 +02:00
Ivan Nardi 523360001e HTTP, Bittorrent, Kerberos: fix stopping of extra dissection (#1700)
The return value of the extra-dissection callback indicates if the extra
dissection needs to be called again.

In the HTTP cose, this setting to NULL of the callabck is wrong since we
stop extra dissection only if we have a hostname *and* a return code.
2022-08-06 20:06:56 +02:00
Ivan Nardi 120e01566a Bittorrent: don't use absolute time reference (#1701) 2022-08-06 20:01:06 +02:00
Ivan Nardi 1d4e27c473 Further simplification of ndpi_process_extra_packet() (#1698)
See 95e16872.

After c0732eda, we can safely remove the protocol list from
`ndpi_process_extra_packet()`.

The field `flow->check_extra_packets` is redundant; remove it.
2022-08-05 15:15:45 +02:00
Ivan Nardi c0732eda45 HTTP: improve sub-classification (#1696)
Content-matched sub-protocols (DASH, IPP, MPEGDASH...) shouldn't
ovewrite the previous master protocol (if any; usually HTTP).

Furthermore. the HTTP dissector shouldn't update the classification
(in the extra-dissection code path) if a content-matched sub-protocols
has already been found.

This commit should address the first part of the changes described in #1687.
2022-08-05 14:14:38 +02:00
Ivan Nardi 346e274a1b SoftEther: fix two heap-buffer-overflows (#1695)
The first change is a proper (hopefully) fix for the bug reported in
8b6a00f8.
The second one is related to:
```
==15096==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60f0000d7a00 at pc 0x55a2c593bd0b bp 0x7ffc92021cd0 sp 0x7ffc92021478
READ of size 3 at 0x60f0000d7a00 thread T0
    #0 0x55a2c593bd0a in strncmp (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x56fd0a) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #1 0x55a2c5d1d9f9 in dissect_softether_host_fqdn /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:249:9
    #2 0x55a2c5d1b55b in ndpi_search_softether /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:348:9
    #3 0x55a2c5b0e9c5 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5407:6
    #4 0x55a2c5b0f78b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5443:10
    #5 0x55a2c5b0f12c in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5476:12
    #6 0x55a2c5b20f39 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6303:15
    #7 0x55a2c5a3014c in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1600:31
    #8 0x55a2c5a29062 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2170:10
    #9 0x55a2c59e51a2 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #10 0x55a2c590acb2 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x53ecb2) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #11 0x55a2c590a3c5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x53e3c5) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #12 0x55a2c590c0f6 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::vector<fuzzer::SizedFile, std::allocator<fuzzer::SizedFile>>&) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x5400f6) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #13 0x55a2c590c663 in fuzzer::Fuzzer::Loop(std::vector<fuzzer::SizedFile, std::allocator<fuzzer::SizedFile>>&) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x540663) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #14 0x55a2c58faff2 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x52eff2) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #15 0x55a2c5923c82 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x557c82) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #16 0x7f504ab98082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #17 0x55a2c58efb1d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x523b1d) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)

0x60f0000d7a00 is located 0 bytes to the right of 176-byte region [0x60f0000d7950,0x60f0000d7a00)
```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49736
2022-08-05 13:43:09 +02:00
Luca Deri 10f283b467 Fix for systems with no realpath (MacOS) 2022-08-05 11:17:59 +02:00
Luca Deri 9fb11c909e Win fixes 2022-08-05 11:17:28 +02:00
Luca Deri bcf7f1713c Win and indentation fixes 2022-08-05 11:17:09 +02:00
Toni ac24b35b1f Add Discord dissector. (#1694)
* fixed RiotGames false positive

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-03 12:03:36 +02:00
Luca Deri c3c73e7bff Cosmetic changes 2022-08-03 11:31:32 +02:00
Luca Deri 63fe4b4306 Compilation fix (not all compilers like restrict) (part 2) 2022-08-03 11:12:15 +02:00
Luca Deri 154d6a5069 Compilation fix (not all compilers like restrict) 2022-08-03 10:47:31 +02:00
Toni b3e722e5a8 Improved nDPI JSON serialization. (#1689)
* fixed autoconf CFLAGS/LDFLAGS MSAN issue which could lead to build errors
 * introduced portable version of gmtime_r aka ndpi_gmtime_r
 * do as most as possible of the serialization work in ndpi_utils.c
 * use flow2json in ndpiReader

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-02 17:54:44 +02:00
Toni 26aafd931c Add Activision dissector. (#1693)
Signed-off-by: lns <matzeton@googlemail.com>
2022-08-02 16:15:07 +02:00
Ivan Nardi 013212c67c TLS: avoid useless extra dissection (#1690)
If we have seen Application Data blocks in both directions, it means we
are after the handshake. At that point, extra dissection is useless.

Useful feature with TLS mid-sessions, i.e. sessions without initial
packets.
2022-07-31 16:57:17 +02:00
Ivan Nardi 2a596c79e6 HTTP: fix classification (#1692)
If we have a valid HTTP sessions, we should ignore
`flow->guessed_protocol_id` field (i.e. classification "by-port")
altogheter.

The attached trace was classified as "SIP/HTTP" only because the *client*
port was 5060...
As a general rule, having a classification such as "XXXX/HTTP" is
*extremely* suspicious.
2022-07-30 22:57:20 +02:00
Ivan Nardi 8b6a00f84b SoftEther: fix heap-buffer-overflow (#1691)
```
==160665==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x604000000038 at pc 0x55f7250d9a5c bp 0x7fff02c82b90 sp 0x7fff02c82350
READ of size 4 at 0x604000000038 thread T0
    #0 0x55f7250d9a5b in __interceptor_strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x548a5b) (BuildId: 12fd06e7a171f035d3a25f016184ac357088379c)
    #1 0x55f7253e6495 in dissect_softether_ip_port /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:303:3
    #2 0x55f7253e5703 in ndpi_search_softether /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:330:9
    #3 0x55f7251d87c5 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5397:6
    #4 0x55f7251d958b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5433:10
    #5 0x55f7251d8f2c in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5466:12
    #6 0x55f7251ead39 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6293:15
    #7 0x55f72512b87e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #8 0x55f72512b9f7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #9 0x7fdef837b082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #10 0x55f72506a45d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4d945d) (BuildId: 12fd06e7a171f035d3a25f016184ac357088379c)
```

Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49638
2022-07-30 22:55:46 +02:00
Ivan Nardi d54d5083b3 SMTPS, POPS, IMAPS: fix classification and extra dissection (#1685)
The big change in TLS code is to allow "master" protocols other than
TLS/DTLS, like SMTPS, POPS and IMAPS.
This change will allow, in a future, a proper and complete TLS dissection
for all these protocols with "STARTTLS"-like messages.
2022-07-30 12:05:43 +02:00
Toni 8402bd68ad Tiny Softether dissector improvement. (#1688)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-30 10:03:42 +02:00
Toni ed4f106a0d Add Softether dissector. (#1679)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-29 19:29:54 +02:00
Ivan Nardi 405a52ed65 Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1683)
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.

Add an option to `ndpiReader` to dump them; enabled by default in the
unit tests.
This new option enables/disables dumping of "num dissectors calls"
values, too (see b4cb14ec).
2022-07-29 15:25:00 +02:00
Ivan Nardi df2e11ef51 Revert "Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1677)" (#1682)
This reverts commit bb83899985.
2022-07-29 12:08:40 +02:00
Ivan Nardi bb83899985 Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1677)
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.

Add an option to `ndpiReader` to dump them; disabled by default to avoid
too much fuss with the unit tests.
2022-07-29 12:07:41 +02:00
Ivan Nardi 95e16872fa First step in simplify ndpi_process_extra_packet() (#1680)
Move the prottocol specific logic into the proper dissector code, where
it belongs.

Next step: remove that list of protocols. Long goal: remove this
function altogether...
2022-07-29 12:07:14 +02:00
Ivan Nardi 172e698bb8 TINC: avoid processing SYN packets (#1676)
Since e6b332aa, we have proper support for detecting client/server
direction. So Tinc dissector is now able to properly initialize the
cache entry only when needed and not anymore at the SYN time; initializing
that entry for **every** SYN packets was a complete waste of resources.

Since 4896dabb, the various `struct ndpi_call_function_struct`
structures are not more separate objects and therefore comparing them
using only their pointers is bogus: this bug was triggered by this
change because `ndpi_str->callback_buffer_size_tcp_no_payload` is now 0.
2022-07-28 12:39:18 +02:00
Ivan Nardi d8d525fff2 Update the protocol bitmask for some protocols (#1675)
Tcp retransmissions should be ignored.

Remove some unused protocol bitmasks.

Update script to download Whatsapp IP list.
2022-07-27 11:46:45 +02:00
Toni ab3a678ad4 Add AVAST dissector. (#1674)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-25 18:07:44 +02:00
Ivan Nardi b190dab6bc Improve handling of HTTP-Proxy and HTTP-Connect (#1673)
Treat HTTP-Proxy and HTTP-Connect flows like the HTTP ones:
print/serialize all the attributes and allow parsing of replies.

The line about "1kxun" has been removed to avoid regressions in 1KXUN
classification in `tests/pcap/1kxun.pcap`. I haven't fully understod
what was happening but the comment at the beginning of `static
ndpi_category_match category_match[]` says that we can't have overlaps
between `host_match` and `category_match` lists and that is no longer true
since 938e89ca.
Bottom line: removing this line seems the right thing to do, anyway.
2022-07-25 12:57:33 +02:00
Ivan Nardi 86a3e4c8c3 Add support for Memory sanitizer (#1652) 2022-07-24 18:12:25 +02:00
Ivan Nardi e6b332aa4a Add support for flow client/server information (#1671)
In a lot of places in ndPI we use *packet* source/dest info
(address/port/direction) when we are interested in *flow* client/server
info, instead.

Add basic logic to autodetect this kind of information.

nDPI doesn't perform any "flow management" itself but this task is
delegated to the external application. It is then likely that the
application might provide more reliable hints about flow
client/server direction and about the TCP handshake presence: in that case,
these information might be (optionally) passed to the library, disabling
the internal "autodetect" logic.

These new fields have been used in some LRU caches and in the "guessing"
algorithm.
It is quite likely that some other code needs to be updated.
2022-07-24 17:46:24 +02:00
Luca Deri 523f22b942 Added vides/presentations section 2022-07-23 17:05:31 +02:00
Toni a25b2a7e37 Added AliCloud server access dissector. (#1672)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-23 11:21:49 +02:00
Zied Aouini 40d7354dd7 Windows error C2466 fix (array size init with 0). 2022-07-22 21:58:25 +02:00
Zied Aouini 9d040f6317 Windows compilation fix. 2022-07-22 20:14:30 +02:00
Ivan Nardi 52005e88ed TLS: improve reassembler (#1669)
* TLS: cosmetic changes

* TLS: improve reassembler

We might need to contemporary re-order messages from both directions:
use one buffer per direction.
2022-07-22 12:19:21 +02:00
Ivan Nardi ce6f11840d Bittorrent: fix keys in cache code (#1670) 2022-07-22 12:10:43 +02:00
Luca Deri e85d6d7db7 RPM fix 2022-07-20 21:26:07 +02:00
Nardi Ivan 00eb2393f4 STUN: improve detection of Hangout/Duo and FB voip calls 2022-07-20 20:36:31 +02:00
Nardi Ivan d66aa49787 DTLS: fix exclusion of DTLS protocol
Add an helper to exclude a generic protocol
2022-07-20 19:16:18 +02:00
Luca Deri a9204eb936 Rocky fix 2022-07-20 17:23:17 +02:00
Nardi Ivan e1edb08f06 SKYPE: fix detection over UDP
Commit ba6a48c9 is completely bogus: we can't set extra dissection
without having set a proper classification.

The idea behind that commit seems to be that we need to look for 2
(consecutives?) packets with the same crc/pattern: try to implement this
logic in a saner way.
2022-07-20 16:13:55 +02:00
Nardi Ivan 5702c6fb08 SKYPE: remove detection over TCP
Skype detection over TCP has been completely disable since 659f75138 (3
years ago!).
Since that logic was too weak anyway, remove it.
2022-07-20 16:13:55 +02:00
Ivan Nardi c72660d7d3 reader_util: stop processing a flow (#1666)
We should stop processing a flow if all protocols have been excluded or
if we have already processed too many packets.
2022-07-20 14:48:09 +02:00
Ivan Nardi b472a49271 BITTORRENT: fix confidence (#1664)
Remove two unused parameters.
2022-07-20 13:59:51 +02:00
Ivan Nardi ac1113b9ac ahocorasick: fix char/uchar bug (#1597)
Porting of https://github.com/vel21ripn/nDPI/commit/06e2967d0c26ab214683a2f5565f4012fd523537
Close #1506
2022-07-20 10:59:17 +02:00
Toni be1ac1b430 Fix unitialized use of W[16]. (#1662)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-15 14:15:37 +02:00
Luca b80218a0e4 Added null pointer checks 2022-07-14 18:04:51 +02:00
Luca 842e0a3648 Restored -O2 in default builds 2022-07-14 17:24:18 +02:00
Toni 7c19de4904 Do not interfere with CFLAGS/LDFLAGS env anymore. (#1659)
* CI fixes
 * some build systems do not like that (e.g. OpenWrt)
 * fixed some rrdtool related build warnings/errors

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-13 19:44:18 +02:00
Toni ae2bedce3a Improved Jabber/XMPP detection. (#1661)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-13 17:55:33 +02:00
Ivan Nardi 407155755d ASN1/BER: fix signed integer overflow (#1660)
```
protocols/snmp_proto.c:77:23: runtime error: signed integer overflow: 6 + 2147483647 cannot be represented in type 'int'
 #0 0x52f69e in ndpi_search_snmp ndpi/src/lib/protocols/snmp_proto.c:77:23
 #1 0x4c5347 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:5211:4
 #2 0x4c5591 in ndpi_check_flow_func ndpi/src/lib/ndpi_main.c:0
 #3 0x4c8903 in ndpi_detection_process_packet ndpi/src/lib/ndpi_main.c:6145:15
 #4 0x4b3712 in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_process_packet.c:29:5
[...]
```
Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49057
2022-07-12 18:39:05 +02:00
Luca 9c235796af Introduced risk accountability 2022-07-12 10:36:40 +02:00
Luca cba2d12051 Cosmetic change 2022-07-12 10:36:32 +02:00
Ivan Nardi b4cb14ec19 Keep track of how many dissectors calls we made for each flow (#1657) 2022-07-11 09:47:47 +02:00
Ivan Nardi df599e5eff HTTP: improve detection of WindowsUpdate (#1658)
WindowsUpdate is also transported over HTTP, using a numeric IP as
hostname (some kinds of CDN?)
2022-07-10 17:08:37 +02:00
Ivan Nardi 1fcd03a6b6 Remove unsafe access to flow->protos union (#1656)
We can access `flow->protos` only if we already have set a valid
classification.

It is quite likely that this code is never trigger, anyway.
2022-07-10 17:08:22 +02:00
Ivan Nardi 2edfaeba4a SNMP: fix detection (#1655)
We can write to `flow->protos` only when we are sure about SNMP
classification.

Use the generic wrapper to decode ASN1 BER integer
2022-07-10 17:07:52 +02:00
Ivan Nardi 997dce0f04 SIP: improve detection (#1654) 2022-07-09 05:45:42 +02:00
Ivan Nardi dfe6557e18 TFTP: fix memory access (#1653) 2022-07-08 18:39:05 +02:00
Ivan Nardi 510517126a LDAP: rewrite dissection (#1649) 2022-07-08 12:50:46 +02:00
Toni 9b95876973 Enhances gprof usage. (#1651)
* gprof results were incorrectly displayed

Signed-off-by: lns <matzeton@googlemail.com>
2022-07-08 12:05:55 +02:00
Ivan Nardi f8076e3a58 SMB: add (partial) support for messages split into multiple TCP segments (#1644) 2022-07-07 19:24:31 +02:00
Ivan Nardi ff4e010501 Avoid spurious calls to extra dissection (#1648)
If the extra callabck is not set, calling the extra dissection is only a
waste of resources...
2022-07-07 17:49:35 +02:00
Ivan Nardi d254ae54f3 SMTP: add support for X-ANONYMOUSTLS comamnd (#1650) 2022-07-07 16:46:18 +02:00
Ivan Nardi feaa1df1ed Kerberos: add support for Krb-Error messages (#1647) 2022-07-07 16:45:49 +02:00
Ivan Nardi 056e742304 Spotify: remove some useless ip ranges (#1646)
These AS numbers are no more related to Spotify (or, if they are, they
don't have any prefixes anyway).
Even if we find some valid Spotify AS, we should handle them via the
generic "autogenerated logic" used for every AS, and not in the
dissector code.
2022-07-07 15:39:19 +02:00
Nardi Ivan 2636c07571 MONGODB: avoid false positives 2022-07-07 15:36:05 +02:00
Nardi Ivan a31e79fc3c TLS: ignore invalid Content Type values 2022-07-07 15:36:05 +02:00
Toni 15042870f9 Added Threema Messenger. (#1643)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 19:30:10 +02:00
Toni Uhlig 105f661e46 Added RiotGames ASN update.
* updated asn lists

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 14:37:26 +02:00
Toni Uhlig a1c3d05a74 Added another RiotGames signature.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 14:37:26 +02:00
Luca Deri 814f9cd21e Moved to 4.5 2022-07-06 12:43:56 +02:00
Toni 175f863665 Label SMTP w/ STARTTLS as SMTPS *and* dissect TLS clho. (#1639)
* Label SMTP w/ STARTTLS as SMTPS *and* dissect TLS clho.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Revert "SMTP with STARTTLS is now identified as SMTPS"

This reverts commit 52d987b603.

* Revert "Compilation fix"

This reverts commit c019946f60.

* Sync unit tests.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 12:40:25 +02:00
Luca Deri c019946f60 Compilation fix 2022-07-05 17:38:32 +02:00
Ivan Nardi 7645909460 Fix handling of NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1636) 2022-07-05 17:01:00 +02:00
Luca Deri 52d987b603 SMTP with STARTTLS is now identified as SMTPS 2022-07-05 17:00:21 +02:00
Toni f4a1739f9c Detect SMTPs w/ STARTTLS as TLS and dissect client/server hello. Fixes #1630. (#1637)
* FTP needs to get updated as well as it has similiar STARTTLS semantics -> follow-up

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-05 16:35:23 +02:00
Toni 388dfb8e13 Run regression tests from different locations at the same time w/o side effects on the results. (#1638)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-05 15:08:59 +02:00
Luca Deri 7fa8d882d8 Exported username in flow information 2022-07-04 22:52:54 +02:00
Luca Deri 461589517e Updated ndpi_check_flow_risk_exceptions() signature 2022-07-04 21:38:38 +02:00
Luca Deri e7a5eaecde Cleaned-up issuer DN check code adding
u_int8_t ndpi_check_issuerdn_risk_exception(struct ndpi_detection_module_struct *ndpi_str, char *issuerDN);

Added new API function for checking nDPI-configured exceptions
u_int8_t ndpi_check_flow_risk_exception(struct ndpi_detection_module_struct *ndpi_str,
                                        u_int num_params,
                                        ndpi_risk_params **params);
2022-07-04 18:41:01 +02:00
Luca Deri 8ff2860601 Set CiscoVPN as a network protocol 2022-07-04 18:41:01 +02:00
Toni Uhlig b3ab66020f Updated JA3/SSL fingerprints.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 16:05:22 +02:00
Toni Uhlig b3d3e3b210 Replaced malicious JA3-md5/SSL-cert-sha1 ac automata with hashmaps.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 16:05:22 +02:00
Toni 4ff8aa48b2 Added UltraSurf protocol dissector. (#1618)
* TLSv1.3 UltraSurf flows are not detected by now

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 16:04:53 +02:00
Ivan Nardi 5aa3d9126f Add two new confidence values: confidence by partial DPI (#1632)
Used for all classifications based on partial/incomplete DPI
information, i.e. all classifications done in `ndpi_detection_giveup()`.
2022-07-04 13:56:51 +02:00
Ivan Nardi 4445989588 Update host content list match (#1633)
Improve classifications of Outlook, Cachefly, Cloudflare, Tiktok and
Cybersecurity.
2022-07-04 13:21:11 +02:00
Toni a1de0285eb Sync Psiphon unit test. (#1634)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 13:20:44 +02:00
Toni 75f7da5c26 Added Psiphon detection patterns. See #566 and #1099. (#1631)
* The traces are not up to date, but this is the best we got so far.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 10:34:54 +02:00
Ivan Nardi 192a32207c OCSP: improve detection (#1629) 2022-07-04 07:22:56 +02:00
Toni a74fc089c4 Added i3D and RiotGames protocol dissectors. (#1609)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 20:43:30 +02:00
Ivan Nardi faaff58620 TargusDataspeed: avoid false positives (#1628)
TargusDataspeed dissector doesn't perform any real DPI checks but it only
looks at the TCP/UDP ports.
Delete it, and use standard logic to classify these flows by port.
2022-07-03 20:28:58 +02:00
Ivan Nardi 50c0212df1 Update ASN/IPs lists (#1627) 2022-07-03 19:50:47 +02:00
Ivan Nardi b5fb2066cb bins: add support for 64bit bins (#1626) 2022-07-03 19:25:15 +02:00
Ivan Nardi 422d002542 Skinny: rework and improve classification (#1625) 2022-07-03 19:25:00 +02:00
Ivan Nardi eed47acfc8 Skype_Teams, Mining, SnapchatCall: fix flow category (#1624) 2022-07-03 18:51:16 +02:00
Ivan Nardi 77ac58e553 Minor changes in how classification results are set (#1623)
Protocol classification should always be set via
`ndpi_set_detected_protocol()`: this way, the values in
`flow->detected_protocol_stack[]` are always coherent.
2022-07-03 18:45:24 +02:00
Ivan Nardi 060e894d5b Usenet: improve dissection (#1622) 2022-07-03 18:08:04 +02:00
Ivan Nardi fdb1649a49 Fix category for mail sessions (#1621)
Close #629
2022-07-03 17:47:58 +02:00
Ivan Nardi 5fe6087686 TLS: add support for old DTLS versions and for detection of mid-sessions (#1619) 2022-07-03 17:44:17 +02:00
Ivan Nardi 5f6fa6d164 Fix a compilation warning (#1620)
With clang-15 (nightly build)
```
In file included from ndpi_bitmap.c:39:
./third_party/src/roaring.cc:14233:13: warning: variable 'run_count' set but not used [-Wunused-but-set-variable]
        int run_count = 0;
```
2022-07-03 17:43:38 +02:00
Toni Uhlig 69ccb39741 Generate profiling results as PNG.
* use -ltcmalloc_and_profiler and try to get rid of LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libprofiler.so

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 17:38:43 +02:00
lns f2d1edbedf gprof test/CI integration
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 17:38:43 +02:00
Toni 1a01e8dc68 Improved TFTP. Dissect Read/Write Request filenames. (#1617)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 14:37:05 +02:00
Toni 5ca82ad84b Added TSAN support. (#1613)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 14:35:21 +02:00
Toni 59b00b00a7 Fix byte-order issue during ndpiReader tcp/udp src/dst port serialization. Fixes #1608. (#1614)
* fixed possible memory leak caused by an invalid call to `node_proto_guess_walker()` during serialization
 * execute serialization code while running regression tests

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 11:16:52 +02:00
Toni 7c5c811eb0 Added Cloudflare WARP detection patterns. (#1615) (#1616)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-02 14:57:56 +02:00
Luca Deri 008a1790e4 Fixed SMTP default port 587 2022-07-02 11:49:22 +02:00
Toni bb72aa4767 Added TunnelBear VPN detection patterns. (#1615)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-01 13:19:17 +02:00
Luca Deri 8f6a006e36 Updated (C) 2022-06-30 14:53:47 +02:00
Luca Deri 6cf7838295 Removed space from "Genshin Impact" 2022-06-30 14:50:05 +02:00
Toni c96f4512fa sync unit tests (#1612)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-06-30 09:37:30 +02:00
Luca Deri 5f1caeb54e Fix after the protocol name update 2022-06-28 17:26:11 +02:00
Luca d393fb969d Renamed Z39.50 -> Z3950 as the '.' breaks the naming convention
QUIC is a network protocol
2022-06-28 17:20:58 +02:00
Luca Deri 227ab5c105 Enhanced TLS risk info reported to users 2022-06-28 00:01:00 +02:00
Luca Deri 4296ec969a Added default port for syslog TCP 2022-06-27 18:36:56 +02:00
Ivan Nardi b2b61011ae Fix compilation and sync unit tests results (#1606) 2022-06-20 14:27:13 +02:00
Luca Deri ab09b8ce2e Added unidirectional traffic flow risk 2022-06-20 00:22:13 +02:00
Toni c287eb835b Improved SOAP via HTTP. (#1605)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-18 17:19:16 +02:00
Toni 6cd8f8cc6d Improved GenshinImpact protocol dissector. (#1604)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-18 15:11:59 +02:00
Toni 432de5eb57 Added collectd dissector (again). (#1601)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-17 19:56:33 +02:00
Toni 9c8b2d63da Replaced nDPI's internal hashmap with uthash. (#1602)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-17 19:50:31 +02:00
Toni 20a29c393f Improved IPSec/ISAKMP detection. (#1600)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-16 09:16:18 +02:00
Luca c43360f8e6 Added new test pcaps 2022-06-15 18:57:46 +02:00
claudio_burrafatoandToni c73a0988cd Add some statistics to ndpiReader (#1587)
* Add some statistics to ndpiReader

The purpose of this version of ndpiReader is too adding some other statistics printed by ndpiReader. In this simple version the domain names(in the flows) that are collected are:
flow-> ssh_tls.server_info
flow-> host_server_name
and are placed in a UT_hash_table, ordering them by number of occurrences.

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update ndpiReader.c

* Update ndpiReader.c

* Update ndpiReader.c

* Update ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>
2022-06-15 16:36:25 +02:00
Ivan Nardi e2cc08bfe5 Add support for PIM (Protocol Indipendent Multicast) protocol (#1599)
Close #1598
2022-06-15 12:25:26 +02:00
Toni d1773cc8e3 Improved WhatsApp detection. (#1595)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-14 20:06:48 +02:00
Ivan Nardi bdf54d725b Fix invalid memory access (#1596)
We can access `flow->protos` union only after checking the protocol.

Checking `flow->detected_protocol.master_protocol` is redundant because
we already check it in `is_ndpi_proto`

```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==29739==ERROR: AddressSanitizer: SEGV on unknown address 0x000000353820 (pc 0x7f9b64dd2717 bp 0x7fff161a52f0 sp 0x7fff161a4aa8 T0)
==29739==The signal is caused by a READ memory access.
    #0 0x7f9b64dd2717  /build/glibc-SzIz7B/glibc-2.31/string/../sysdeps/x86_64/multiarch/strlen-avx2.S:96
    #1 0x555c65e597d8 in __interceptor_strlen (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x6407d8) (BuildId: 11ac8ec30f1d49fb0276c9b03368e491505d2bba)
    #2 0x555c65fd85fa in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:269:13
    #3 0x555c65f3e8c6 in process_ndpi_collected_info /home/ivan/svnrepos/nDPI/example/reader_util.c:1188:36
    #4 0x555c65f52cab in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1567:2
    #5 0x555c65f4b632 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2110:10
    #6 0x555c65f04d29 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:109:7
    #7 0x555c65f054bb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:181:17
    #8 0x7f9b64c6e082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #9 0x555c65e4253d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x62953d) (BuildId: 11ac8ec30f1d49fb0276c9b03368e491505d2bba)

```

Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=48020
2022-06-14 11:44:32 +02:00
Ivan Nardi 8dcaa5c0e1 DNS: fix TTL check and sync unit test results (#1594) 2022-06-14 09:29:16 +02:00
Luca Deri defe7d7f79 Updated DNS alert triggered only with TTL == 0 2022-06-14 00:13:05 +02:00
Luca Deri 831562c141 Restored ndpi_set_proto_defaults() prototype
Updated test results
2022-06-13 23:53:32 +02:00
Luca Deri 7a172ce11e Added check for DGA names that resolve to a valid record 2022-06-13 23:42:07 +02:00
Luca Deri cf5873ffd7 Improved DNS traffic analysis
Added ability to identify application and network protocols
2022-06-13 23:19:47 +02:00
Luca Deri 32428026d6 Added DNS record TTL check 2022-06-13 17:05:56 +02:00
Toni df0ff9bcbd Added gprof CPU/HEAP profiling support. (#1592)
* Some small auto{conf,make} improvements

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-12 21:00:41 +02:00
Toni 341f58fd80 Removed Makefile references to legacy code. (#1589)
- feature removed with f5545a80f9

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-08 13:37:11 +02:00
Luca Deri 11babc7ea3 Added Pragmatic General Multicast (PGM) protocol detection 2022-06-08 09:11:22 +02:00
Toni bb3c2e82de Dissect host line if SSDP contains such. (#1586)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-07 11:15:47 +02:00
Toni 938e89ca33 Reimplemented 1kxun application protocol. (#1585)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-06 18:04:49 +02:00
Toni df78e67518 Prevent compilation failure if, for whatever reason, NDPI_API_VERSION is empty. (#1584)
- This is not a fix for #1494, just an improvment.

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-06 15:53:48 +02:00
Toni 0b3f8ed849 Fixed syslog false negatives. (#1582)
- RSH vs Syslog may still happen for midstream traffic

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-05 23:01:56 +02:00
Ivan Nardi 1d492c9ed8 Fix some debug messages (#1583)
Increase max number of flows handled during fuzzing
2022-06-05 14:45:21 +02:00
Luca Deri 59c084b166 Updated test results 2022-06-05 11:34:10 +02:00
Luca Deri 2202365b95 Fixed invalid DHCP dissection 2022-06-05 11:19:28 +02:00
Luca Deri b6ae9642d3 Fixed DHCP dissection bug 2022-06-05 11:14:23 +02:00
Toni 7419cfee64 Added RSH dissector. Fixes #202. (#1581)
- added syslog false-positive pcap that was missing in 09fbe0a64a
 - added NDPI_ARRAY_LENGTH() macro, usable on `type var[]` declarations

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-04 19:12:53 +02:00
Ivan Nardi 338427d712 Add support for GoTo products (mainly GoToMeeting) (#1580)
There is some overlap with Citrix protocol.
2022-06-04 14:30:52 +02:00
Toni ff8e1e197e Fix syslog heap overflow introduced in 09fbe0a64a. (#1579)
- fixes #1578

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-04 08:47:47 +02:00
Toni 09fbe0a64a Fixed syslog false positives. (#1577)
* syslog: removed unnecessary/unreliable printable string check
 * added `ndpi_isalnum()`
 * splitted `ndpi_is_printable_string()` into `ndpi_is_printable_buffer()` and `ndpi_normalize_printable_string()`

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-03 18:21:29 +02:00
Toni 6149c0f880 Fix heap buffer overflow mentioned in #1574. (#1576)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-02 13:07:09 +02:00
Ivan Nardi 4ff1bf29df TLS: fix use-of-uninitialized-value error (#1573)
Proper fix for the error already reported in 9040bc74

```
Uninitialized bytes in __interceptor_strlen at offset 3 inside [0x7ffc7a147390, 4)
==111876==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x55e3e4f32e5b in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:268:13
    #1 0x55e3e4ef7391 in ndpi_set_risk /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:2254:12
    #2 0x55e3e5022fdf in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1523:8
    #3 0x55e3e503af44 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:865:5
    #4 0x55e3e50397cd in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1024:2
    #5 0x55e3e503570c in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2453:5
    #6 0x55e3e4f84a6a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5150:6
    #7 0x55e3e4f85778 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5198:12
    #8 0x55e3e4f851e1 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5217:12
    #9 0x55e3e4f96c7a in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6076:15
    #10 0x55e3e4ed91ef in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #11 0x55e3e4eda27e in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #12 0x7f5cb3146082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #13 0x55e3e4e5140d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0xa340d) (BuildId: 0c02c433e039970dd13a60382b94dd5a8e19f625)
```
2022-06-02 00:15:52 +02:00
Toni eac11030ac Removed README.nDPI as it does not provide any new information not covered by README.md (#1572)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-31 22:11:48 +02:00
Toni 00c62a7c3c Removed LGTM ql query for packet payload integer arithmetic. (#1570)
* Detection is unreliable and results in too much false-positives (and false-negatives!)

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-31 16:46:44 +02:00
Toni d8d806c79f Force roaring bitmap to use ndpi memory wrappers. (#1569)
GCC analyzer won't complain about possible use-after-free (false positive).

 * tests/do.sh prints word diff's only once and not the same over and over again
 * sync unit tests

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-31 12:06:01 +02:00
Ivan Nardi 9040bc74c9 TLS: fix stack-buffer-overflow error (#1567)
```
==300852==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f108951f060 at pc 0x5641db0ee78c bp 0x7fff3b10b910 sp 0x7fff3b10b0d0
WRITE of size 116 at 0x7f108951f060 thread T0
    #0 0x5641db0ee78b in __interceptor_strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x64e78b) (BuildId: 23cb34bbaf8ac11eb97563bbdc12e29ead9fb0fa)
    #1 0x5641db28efad in tlsCheckUncommonALPN /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1224:7
    #2 0x5641db27ec76 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1533:6
    #3 0x5641db295677 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:865:5
    #4 0x5641db2935bb in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1024:2
    #5 0x5641db28f692 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2446:5
    #6 0x5641db1d87ce in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5150:6
    #7 0x5641db1d91e4 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5198:12
    #8 0x5641db1d8e87 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5217:12
    #9 0x5641db1eb4a7 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6076:15
    #10 0x5641db140b45 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #11 0x5641db14130b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #12 0x7f108bcab082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #13 0x5641db07f46d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x5df46d) (BuildId: 23cb34bbaf8ac11eb97563bbdc12e29ead9fb0fa)

Address 0x7f108951f060 is located in stack of thread T0 at offset 96 in frame
    #0 0x5641db28ec4f in tlsCheckUncommonALPN /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1204

  This frame has 1 object(s):
    [32, 96) 'str' (line 1218) <== Memory access at offset 96 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x64e78b) (BuildId: 23cb34bbaf8ac11eb97563bbdc12e29ead9fb0fa) in __interceptor_strncpy
Shadow bytes around the buggy address:
```

Avoid zeroing the entire string.

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47730
2022-05-31 10:33:42 +02:00
Luca Deri 354addd693 Updated risk results 2022-05-30 23:28:59 +02:00
Luca Deri b306a4b8ed Improved message for known proto on non std port 2022-05-30 23:23:18 +02:00
Luca Deri b3bf695723 Added check 2022-05-30 22:39:23 +02:00
Toni 00ee364a7e Updated README.md (#1562)
* make check great again (not so much)
 * make doc/doc-view
 * CI updates

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-30 19:48:05 +02:00
Ivan Nardi ca115774c9 TLS: fix use-of-uninitialized-value error (#1566)
```
Uninitialized bytes in __interceptor_strlen at offset 0 inside [0x7ffc987443a0, 7)
==282918==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x5651277e001b in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:268:13
    #1 0x5651277a45c1 in ndpi_set_risk /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:2254:12
    #2 0x5651278cfaa6 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1515:8
    #3 0x5651278e7834 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:865:5
    #4 0x5651278e60bd in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1024:2
    #5 0x5651278e1ffc in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2445:5
    #6 0x565127831c2a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5150:6
    #7 0x565127832938 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5198:12
    #8 0x5651278323a1 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5217:12
    #9 0x565127843e3a in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6076:15
    #10 0x56512778641f in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #11 0x5651277874ae in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #12 0x7f4c46794082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #13 0x5651276fe40d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0xa340d)
```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47728
2022-05-30 19:40:47 +02:00
Luca Deri 912b69af60 Redefined type name to avoid conflicts 2022-05-30 18:29:08 +02:00
Luca 34ede63c15 Added ability to return risk info in JSON format in ndpi_get_flow_risk_info() 2022-05-30 17:54:30 +02:00
Toni 48065d80e9 Support word diff for tests/do.sh for better readability. (#1565)
* Sync unit tests

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-30 17:33:03 +02:00
Toni 32750271c3 Prohibit MPEG-DASH to set HTTP as application protocol. (#1560)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-30 14:54:27 +02:00
Ivan Nardi c4f50b2cda HTTP: fix heap-buffer-overflow error (#1564)
```
==222479==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60b000014739 at pc 0x55af06f2364f bp 0x7ffd7b6f4bf0 sp 0x7ffd7b6f4378
READ of size 12 at 0x60b000014739 thread T0
    #0 0x55af06f2364e in printf_common(void*, char const*, __va_list_tag*) asan_interceptors.cpp.o
    #1 0x55af06f24f70 in __interceptor_snprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x613f70) (BuildId: f6545ec2bd7663bc3f16aeeb87bddc64d173a2a8)
    #2 0x55af0720927f in ndpi_check_http_header /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:982:2
    #3 0x55af071f4797 in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1219:5
    #4 0x55af071f05c5 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1402:3
    #5 0x55af07080d1e in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5150:6
    #6 0x55af07081734 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5198:12
    #7 0x55af070813d7 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5217:12
    #8 0x55af070939f7 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6076:15
    #9 0x55af06fc7e1f in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1541
    #10 0x55af06fc7e1f in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2110
    #11 0x55af06fc2859 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:109:7
    #12 0x55af06fc2feb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:181:17
    #13 0x7efe5eaac082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #14 0x55af06f0055d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x5ef55d) (BuildId: f6545ec2bd7663bc3f16aeeb87bddc64d173a2a8)

0x60b000014739 is located 0 bytes to the right of 105-byte region [0x60b0000146d0,0x60b000014739)
allocated by thread T0 here:
    #0 0x55af06f84bae in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x673bae) (BuildId: f6545ec2bd7663bc3f16aeeb87bddc64d173a2a8)
    #1 0x55af06fc2673 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:103:31
    #2 0x55af06fc2feb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:181:17
    #3 0x7efe5eaac082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16

```
Fiund by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47724
2022-05-30 14:39:01 +02:00
Ivan Nardi 3b825fca6d Certificate timestamps should be printed in UTC (#1563)
This way, the CI results are consistent
2022-05-30 13:43:05 +02:00
Luca Deri 897f8314b2 Fixed dispay bug for risk_info 2022-05-30 01:12:38 +02:00
Luca Deri 1da9f1a36f Updated tests results
Code cleanup
2022-05-30 00:54:17 +02:00
Luca Deri f25deeccb1 Added RiskInfo string 2022-05-30 00:32:32 +02:00
Ivan Nardi a9d7cc4841 Fix dissection of IPv4 header (#1561)
See: https://github.com/ntop/nDPI/runs/6643914510?check_suite_focus=true

Convert al the `MIN(a,b)` calls to `ndpi_min(a,b)`
2022-05-29 22:39:48 +02:00
Ivan Nardi 9c1a53f39f Dazn: add support for Dazn streaming service (#1559)
Update .gitignore file
2022-05-29 17:47:16 +02:00
Luca Deri 406ac81a37 Compilation fixes for old ggc's 2022-05-29 16:40:16 +02:00
Luca Deri dfbe863f4e Comment 2022-05-29 16:35:08 +02:00
Luca Deri 1093aafa5f Added detection for WordPress exploits
Fixed ndpi_iph_is_valid_and_not_fragmented() that was bugged with non UDP traffic
2022-05-29 16:31:26 +02:00
Toni 6b7b23b01d Use Doxygen to generate the API documentation. (#1558)
* Integrated Doxygen documentation into Sphinx

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-29 13:44:52 +02:00
Toni 33f9729ee4 Added MPEG-DASH dissector. Fixes #1223. (#1555)
* Improved HTTP POST detection
 * Refactored subprotocol detection

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-29 13:12:13 +02:00
Toni cfd23e8819 Fixed HTTP lower/upper protocol mess for Aimini/IPP. (#1557)
* If HTTP was already detected, there is no need to overwrite the detection again

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-28 16:31:35 +02:00
Luca Deri 77dc884a4c Compilation fixes for old gcc compiler 2022-05-28 16:06:34 +02:00
Luca Deri 7e3d061428 Compilation fixes 2022-05-27 17:35:30 +02:00
Luca Deri 6d543808ad Version cut fix 2022-05-27 16:58:39 +02:00
Luca Deri 02ba12b0de Fixes compilation issues on RedHat systems 2022-05-26 23:34:34 +02:00
Ivan Nardi f81f0c5e69 Sync unit test results (#1554) 2022-05-26 20:15:30 +02:00
Luca Deri cedbd32030 Updated SkypeCall -> Skype_TeamsCall 2022-05-26 19:27:12 +02:00
Luca Deri 1645fea85c Fixed false positives with NATS 2022-05-26 19:01:13 +02:00
Toni 78dfe959cb Added script to compare and verify the output of `make dist'. (#1551)
This fixes some build/test issues resulting when using tarballs.

 * nDPI uses autotools (especially autoconf) in a wrong way, see #1163

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-26 15:07:22 +02:00
Toni 9614b66b44 Replaced obsolete autoconf macros. (#1553)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-26 11:31:09 +02:00
Toni 7010d17ccc Fixed windows-latest build error. (#1552)
* The warning itself looks like a bug

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-26 10:19:57 +02:00
Luca Deri 83e1ea2506 Improved invalid host detection 2022-05-24 21:17:33 +02:00
Luca Deri 0c703ef8eb Added invalid SNI check in QUIC 2022-05-24 19:51:49 +02:00
Luca Deri 4f9dee164e Improved detection of invalid SNI and hostnames in TLS, HTTP 2022-05-24 19:47:54 +02:00
Luca Deri 2560260a41 Added room for storing information used by custom third-party dissectors 2022-05-19 23:24:42 +02:00
lns 8b750686c9 Moved RTSP http patterns to the protocol source file.
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-19 19:58:11 +02:00
Toni Uhlig dae7986a37 Yet another approach to fix #1499 (basically a copy&pasta from @socketpair).
* Related to #1545, #1494 and #1189 as well

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-05-16 16:54:18 +02:00
Toni Uhlig 3c08a3c875 Removed MacOS XCode integration.
* It is outdated (from 2018) and will most likely not work anymore
   due to changed protocol files and outdated API usage in the example integration
 * Removal is the only option besides fixing those issues and
   provide a CI integration for future automatic checks

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-05-16 16:54:18 +02:00
Toni Uhlig 7162bf6abf Moved mgcp.pcapng to tests/pcap/ instead of tests/
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-05-16 16:54:18 +02:00
Ivan Nardi f822aad8b0 DNS-over-QUIC: update default port (#1548)
Final RFC (https://www.rfc-editor.org/rfc/rfc9250.txt) changed the
default port to 853 (from the 784 used on the early drafts).
Since there is at least one deployed implementation still stuck at
draft-02, keep also the old value, for the time being.
2022-05-12 14:31:26 +02:00
Toni 054d151373 Improved Viber (TCP) detection. (#1547)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-10 21:37:03 +02:00
Toni 704920414a Improved Xiaomi HTTP detection. (#1546)
* Merged Xiaomi pcap files

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-10 11:10:04 +02:00
Toni 6fc29b3ae8 Removed TLS patterns in the CiscoVPN aka Anyconnect dissector as mentioned in PR #1534. (#1543)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-09 10:29:52 +02:00
Toni 915ffebade Added Softether(-VPN) DDNS service detection. (#1544)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-09 08:16:19 +02:00
Toni 4319d760e0 Improved TLS alert detection. (#1542)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-08 20:33:59 +02:00
Toni 34882d9cf0 Improved TLS application data detection. (#1541)
* #1532 did fx TLS appdata detection only partially
 * use flow->l4.tcp.tls.message.buffer_used instead of packet->payload

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-08 19:56:08 +02:00
Toni b2648a45a3 Added Edgecast and Cachefly CDNs. (#1540)
* Improved ASN update script
 * Ran `utils/update_every_lists.sh'
 * `tests/do.sh.in' prints the amount of failed pcap(s)
 * `utils/asn_update.sh' prints the amount of failed download(s)

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-07 09:33:25 +02:00
Toni 87f93ea4fd Replaced ndpiReader's libjson-c support with libnDPI's internal serialization interface. (#1535)
* Fixes #1528
 * Serialization Interface should also fuzzed
 * libjson-c may only be used in the unit test to verify the internal serialization interface
 * Serialization Interface supports tlv(broken), csv and json
 * Unit test does work again and requires libjson-c

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-07 09:26:09 +02:00
Ivan Nardi 2e0dedbaae Fix compilation (if --enable-debug-messages is used) (#1539) 2022-05-04 18:43:37 +02:00
Luca Deri 434c38c74d Added extra check to make sure that the guessed protocol is the one we expect and not another one 2022-05-04 16:27:36 +02:00
Luca Deri 5996cf1725 Fixes bug that prevents triggering alerts for traffic on non-standard ports that have been defined in the custom protocols file
Fixes https://github.com/ntop/ntopng/issues/6458
2022-05-04 16:07:05 +02:00
Luca Deri 81a367f1ae Fixes outdated description 2022-05-04 16:06:50 +02:00
Luca Deri 5dcb9c1c16 Modified risk labels 2022-05-04 12:10:01 +02:00
Toni d5b922cfe4 Added some Pluralsight Hostnames/SNIs. May fix #1501. (#1538)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-03 13:20:44 +02:00
Luca Deri 65dd052a61 Updated RRD dependencies 2022-05-03 12:37:22 +02:00
Toni 47d6a65522 Improved suspicious http user agent detection. (#1537)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-02 19:17:32 +02:00
Luca Deri c8f72ef764 Added ndpi_get_flow_error_code() API call
Fixed typo
2022-05-02 10:51:08 +02:00
Toni 02d0b5fe13 Improved AES-NI check. (#1536)
* A library should not open a subshell

Signed-off-by: lns <matzeton@googlemail.com>
2022-04-30 20:46:20 +02:00
Luca Deri e4318ffc2d Improved AES-NI check on Linux to avoid crashes on CPUs that do
not support it (e.g. Intel Celeron N2930)
2022-04-29 14:50:21 +02:00
Ivan Nardi 71636dcafd Sync unit tests results (#1533) 2022-04-27 18:22:11 +02:00
Toni 10161448bc Improved TLS application data detection. (#1532)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-27 17:22:53 +02:00
Luca Deri 3ad989f6a8 Added BPF filtering for discarding non-IP packets 2022-04-27 17:05:33 +02:00
Luca Deri 3d755fce73 String messages have been shrinked 2022-04-27 10:28:06 +02:00
Luca Deri a7c1152397 Added ability to store custom category file in patricia tree 2022-04-26 14:42:31 +02:00
Alfredo Cardigliano e8d81123cc Add ndpi_json_string_escape to the API 2022-04-26 12:25:34 +02:00
Ivan Nardi d5005f584c Raknet: fix heap-buffer-overflow (#1531)
```
==120637==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x606000000488 at pc 0x55a0598d97ec bp 0x7ffcfe13f2e0 sp 0x7ffcfe13f2d8
READ of size 2 at 0x606000000488 thread T0
    #0 0x55a0598d97eb in ndpi_search_raknet /home/ivan/svnrepos/nDPI/src/lib/protocols/raknet.c:152:38
    #1 0x55a05966c48e in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5060:6
    #2 0x55a05966e428 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5119:10
    #3 0x55a05966dddc in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5152:12
    #4 0x55a05967fa7a in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5971:15
    #5 0x55a05962b860 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:24:3
    #6 0x55a05962bd9b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:84:17
    #7 0x7f4aad7d80b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
    #8 0x55a05956b46d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x5d746d) (BuildId: 9429d3d08edc3836e5536f93c07f140716d8b82e)

0x606000000488 is located 9 bytes to the right of 63-byte region [0x606000000440,0x60600000047f)
allocated by thread T0 here:
    #0 0x55a0595ee80e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x65a80e) (BuildId: 9429d3d08edc3836e5536f93c07f140716d8b82e)
    #1 0x55a05962bd06 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:70:17
    #2 0x7f4aad7d80b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47000
2022-04-25 19:01:11 +02:00
Toni bc2ad3407a Added generic user agent setter. (#1530)
* ndpiReader: Print user agent if one was set and not just for certain protocols.

Signed-off-by: lns <matzeton@googlemail.com>
2022-04-25 13:00:50 +02:00
Ivan Nardi 075bce5f3d XIAOMI: add detection of Xiaomi traffic (#1529)
Most of the credits should go to @utoni (see #1521)
2022-04-25 11:00:02 +02:00
Toni ac0f50b561 Added RakNet protocol dissector. (#1527)
* Frame Set PDU's do not get fully dissected for the sake of simplicity

Signed-off-by: lns <matzeton@googlemail.com>
2022-04-24 13:08:57 +02:00
Luca Deri 9d31b7450b Code cleanup (removed redundancy) 2022-04-22 19:03:50 +02:00
Toni d5ae9170ba Tiny gnutella improvement if gtk-gnutella used. (#1525)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-22 18:33:27 +02:00
Toni 824c343602 Updated `utils/whatsapp_ip_addresses_download.sh' to scrape the required IP addresses/ranges. (#1524)
* Replaced return statements in `utils/*.sh' with exit's (such scripts should never source'd)
 * Ran `utils/update_every_lists.sh'

Signed-off-by: lns <matzeton@googlemail.com>
2022-04-22 13:27:41 +02:00
Ivan Nardi 42909673ce Add some scripts to easily update some IPs lists (#1522)
Follow-up of 8b062295

Add a new protocol id for generic Tencent/Wechat flows
2022-04-21 20:43:52 +02:00
Toni ddc256b09e Reduce ndpiReader's -h' spam. -H' does this job now. (#1523)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-21 20:43:36 +02:00
Toni c1d46ebc4e Added proprietary Agora Software Defined Real-time Network (SD-RTN) protocol dissector. (#1520)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-20 18:31:28 +02:00
Toni dbf2fb8c9f Added Toca Boca protocol dissector. (#1517)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-19 19:32:49 +02:00
Toni a83ca555fb Removed superfluous ifdef'd includes. (#1519)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-19 17:51:48 +02:00
Toni fa79f07d15 Improved sflow protocol detection false-positives. (#1518)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-19 17:46:40 +02:00
Ivan Nardi 739dfc54b0 Kerberos: fix Undefined-shift error (#1516)
Similar to the error fixed in 4775be3d
Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=46713
2022-04-18 20:54:13 +02:00
Luca Deri 8b2c9860be DGA improvements 2022-04-16 09:56:08 +02:00
Zied Aouini 9c8a3f270a Minor fix. 2022-04-15 18:01:34 +02:00
Zied Aouini fba75a3cf5 Merge pull request #1491 from utoni/fix/windows-msys2
Fixed msys2 build and re-activated CI Mingw-w64 build.
2022-04-15 13:22:25 +02:00
Toni Uhlig c3df3a12aa Fixed msys2 build warnings and re-activated CI Mingw64 build.
* Removed Visual Studio leftovers. Maintaining an autotools project with VS integration requires some additional overhead.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-14 19:17:48 +02:00
Ivan Nardi 4775be3d85 Kerberos: fix some memory access errors (#1514)
```
==19724==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60e00000045e at pc 0x5620b8b3d3cc bp 0x7ffe0fda6b50 sp 0x7ffe0fda6310
READ of size 2 at 0x60e00000045e thread T0
    #0 0x5620b8b3d3cb in __interceptor_strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x63f3cb) (BuildId: ee53ff920c8cd4c226d8520a0d4846d8864726b6)
    #1 0x5620b8d9b69c in strncpy_lower /home/ivan/svnrepos/nDPI/src/lib/protocols/kerberos.c:208:4
    #2 0x5620b8d995a0 in krb_parse /home/ivan/svnrepos/nDPI/src/lib/protocols/kerberos.c:316:5
    #3 0x5620b8d97a90 in ndpi_search_kerberos /home/ivan/svnrepos/nDPI/src/lib/protocols/kerberos.c:687:12
    #4 0x5620b8bcef35 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4996:4
    #5 0x5620b8bd1be8 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5072:10
    #6 0x5620b8bd159c in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5105:12
    #7 0x5620b8be323a in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5924:15
    #8 0x5620b8b8f7e0 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:24:3
    #9 0x5620b8b8fd1b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:84:17
    #10 0x7f45b32b90b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
    #11 0x5620b8acf47d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x5d147d) (BuildId: ee53ff920c8cd4c226d8520a0d4846d8864726b6)

0x60e00000045e is located 0 bytes to the right of 158-byte region [0x60e0000003c0,0x60e00000045e)
allocated by thread T0 here:
    #0 0x5620b8b5283e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x65483e) (BuildId: ee53ff920c8cd4c226d8520a0d4846d8864726b6)
    #1 0x5620b8b8fc86 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:70:17
    #2 0x7f45b32b90b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
```

```
protocols/kerberos.c:79:52: runtime error: left shift of 255 by 24 places cannot be represented in type 'int'
```

Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=46670
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=46636
2022-04-13 18:10:56 +02:00
Luca Deri e00997ea7c Extended list of cybersecurity domains 2022-04-13 18:05:29 +02:00
Wayneand林文烽 939572c61e fix(ndpi_main):Fix memory leak about ndpi_str; (#1513)
Co-authored-by: 林文烽 <wenfeng.lin@baishan.com>
2022-04-12 15:34:37 +02:00
Ivan Nardi 06a0abb6fd TINC: fix invalid memory read (#1512)
```
=================================================================
==19324==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600061be96 at pc 0x55b4a4cb4460 bp 0x7ffc7b461a70 sp 0x7ffc7b461a68
READ of size 1 at 0x60600061be96 thread T0
    #0 0x55b4a4cb445f in ndpi_check_tinc /home/ivan/svnrepos/nDPI/src/lib/protocols/tinc.c:105:9
    #1 0x55b4a4cb1888 in ndpi_search_tinc /home/ivan/svnrepos/nDPI/src/lib/protocols/tinc.c:135:5
    #2 0x55b4a4b4a6e1 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5013:6
    #3 0x55b4a4b4c2d4 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5084:12
    #4 0x55b4a4b4bf77 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5103:12
    #5 0x55b4a4b5dcca in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5924:15
    #6 0x55b4a4a87734 in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1519:31
    #7 0x55b4a4a80761 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2093:10
    #8 0x55b4a4a39c8d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #9 0x55b4a4a3a46b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17
    #10 0x7f69c63760b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
    #11 0x55b4a497954d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x61654d) (BuildId: 705ebc5c412d267294a65cb01f03a1f012aeaf20)

0x60600061be96 is located 0 bytes to the right of 54-byte region [0x60600061be60,0x60600061be96)
allocated by thread T0 here:
[...]
```
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=46499
2022-04-10 16:59:34 +02:00
Toni e3faecf9de Improved ASN.1 parsing for Keberos. Fixes #1492. (#1497)
* This is a quick fix, the Kerberos protocol dissector requires some refactoring effort.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-10 10:36:10 +02:00
Vinicius Silva Nogueira 713cdd0b13 QUIC: handle retransmissions and overlapping fragments in reassembler (#1195) (#1498)
* QUIC: handle retransmissions and overlapping fragments in reassembler

* Trigger CI

* minor fix: parentheses

* Changing ndpi_malloc to ndpi_calloc

* fix memory leak

* quic_reasm_buf calloc to malloc

* change order of is_ch_complete && is_reasm_buf_complete call

* is_reasm_buf_complete: added handling for case where frame size is not multiple of 8

* add extra check
2022-04-07 21:18:58 +02:00
aouinizied 9537788ade Fix JSON-C. 2022-04-07 17:59:11 +02:00
aouinizied ff993e2f57 Python bindings fix. 2022-04-07 17:54:41 +02:00
Luca Deri a6e2f4a15a Added ndpi_find_outliers() API call using Z-Score 2022-04-04 10:02:45 +02:00
Luca Deri 19019383f0 Added -z flag 2022-04-03 11:45:23 +02:00
Ivan Nardi e0e231756c ndpiReader: fix compilation (#1510)
Not sure why Windows started complaining... anyway, the fixes has been
taken from https://github.com/ntop/nDPI/pull/1491: credits to @lnslbrty
2022-04-01 22:01:39 +02:00
Luca Deri 8080b3d14f Removed un-necessary guess in mining 2022-04-01 19:23:13 +02:00
Luca Deri 9f3a6f131b update 2022-04-01 19:07:46 +02:00
Luca Deri 32a7d42469 Fixed incompatibilities due to https://github.com/ntop/nDPI/pull/1509 2022-04-01 19:07:00 +02:00
Luca Deri 6eb5bd88a7 DGA improvements 2022-04-01 17:39:29 +02:00
Luca Deri a75d77b536 Waring fixes 2022-04-01 17:35:25 +02:00
Vitaliy Ivanov 61bc9815d5 ndpireader: add json output back. (#1509)
- partial revert of:
    commit 51cfdfb0d8
    Author: Luca Deri <deri@ntop.org>
    Date: Sun Nov 17 17:51:45 2019 +0100
      Removed unused JSON-C code

- Json option is changed from 'j' to 'k' as it's used in the new codebase.
- use HAVE_LIBJSON_C instead of HAVE_JSON_C.
- tabs vs spaces clean ups.

Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com>

 Conflicts:
	example/ndpiReader.c
2022-04-01 11:47:01 +02:00
Luca Deri 8238e68275 Improvements for CUSTOM_NDPI_PROTOCOLS 2022-04-01 09:02:05 +02:00
Luca Deri 1e8cdd893c Moved geneated file to a separate folder 2022-04-01 08:58:53 +02:00
Luca Deri 0ff3e77c42 Improved twitter detection 2022-04-01 08:50:56 +02:00
Luca Deri 6677d692c4 Removed SRV record from suspicious DNS traffic 2022-03-31 11:32:30 +02:00
Luca Deri 98a548c642 Improved DGA detection 2022-03-31 11:29:30 +02:00
Darryl Sokoloski 46dc8b5bda [autoconf] Fixed .git submodule detection test. (#1507)
Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2022-03-31 03:21:51 +02:00
Luca Deri 4e199abd39 Added code for identifiying anomalies with metrics stored in InfluxDB 2022-03-30 22:12:39 +02:00
Ivan Nardi 6bcba118d4 reader_util: add support for userAgent in SSDP (#1502)
Update unit tests results
Follow-up of d668ab4b
2022-03-28 04:25:34 +02:00
Ivan Nardi 6acf7a6abe Add support for Pluralsight site (#1503) 2022-03-27 15:13:12 +02:00
Ivan Nardi ec80f1fb3b Fix CI tests results (#1504)
CI integration is failing since 856d7d2.
2022-03-27 15:12:39 +02:00
Vitaly Lavrov 4896dabb6b Reducing the size of the ndpi_detection_module_struct structure. (#1490)
The ndpi_detection_module_struct structure contains 5 arrays "struct
ndpi_call_function_struct" size 286*144=41 kB size,
which are occupied by a small number of elements.

At the moment we have callback_buffer_size = 172, tcp_with_payload=114,
tcp_no_payload=8, udp=93, other 8.
NDPI_MAX_SUPPORTED_PROTOCOLS = 285.

Size of struct ndpi_detection_module_struct is 253136 bytes.
Size of all structs ndpi_call_function_struct 5*286*144=205920 bytes.
Real use memory size for struct ndpi_call_function_struct is
(173+224)*144=57168 bytes.
2022-03-27 12:00:21 +02:00
Darryl Sokoloski d668ab4bec [SSDP] Extract HTTP user-agent when available. (#1500)
[SSDP] Added capture file with UA header.
[SSDP] Added pcap test output log file.

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2022-03-27 11:10:44 +02:00
Luca Deri 856d7d2916 Improved DGA detection skipping names containign at least 3 consecutive digits in the first word 2022-03-26 09:59:55 +01:00
Ivan Nardi aeb5f1f947 QUIC: add support for version 2 draft 01 (#1493)
Support for v2-00 has been removed (it has never been used in real
networks and it is incompatible with v2-01).

Chrome already supports v2-01 in latest versions in Chrome Beta channel.
2022-03-25 10:16:30 +01:00
Ivan Nardi af1d20bca1 Mining: cleanup registration (#1496)
Use the same pattern of all the other dissectors: one registration and
one callback.
Spotted by @dsokoloski
2022-03-25 10:07:47 +01:00
Vinicius Silva Nogueira 26df1403e6 Trying to improve QUIC reassembler (#1195) (#1489)
* handling QUIC out-of-order fragments

* minor fix

* updated quic_frags_ch_out_of_order_same_packet_craziness.pcapng.out

* quic test: buf_len + last_pos

* QUIC: comment update in __reassemble function and minor change is_ch_complete function
2022-03-24 10:51:31 +01:00
aouinizied bb12837ca7 Update Python bindings guide. 2022-03-22 15:01:55 +01:00
aouinizied c7059db84a Fix typo. 2022-03-22 14:58:22 +01:00
aouinizied 1207ea118a Add HOWTO Python. 2022-03-22 14:56:44 +01:00
aouinizied c47d710d8e Fix python bindings CI. 2022-03-22 13:40:31 +01:00
aouinizied beef4f997b Complete rework of nDPI Python bindings (cffi API, automatic generation, packaging and CI integration) 2022-03-22 13:19:27 +01:00
Luca Deri 93f723d50f Extended the list of cybersecurity protocol 2022-03-21 15:12:50 +01:00
Vitaly Lavrov 8dd70b7023 Bug fixing. (#1487)
Using the protocol_id instead of its index.
2022-03-15 19:40:18 +01:00
Ivan Nardi 9e22c88e0a QUIC: convert logs to standard mechanism (#1485) 2022-03-15 16:55:47 +01:00
Ivan Nardi b83e9f20e7 QUIC: fix dissection of draft-34 (#1484)
QUIC-34 is probably not used in production, but fixing it is trivial and
it doesn't add any noise to the already complex QUIC code.
2022-03-09 22:37:56 +01:00
Ivan Nardi 7aee856aa0 Extend tests coverage (#1476)
Now there is at least one flow under `tests/pcap` for 249 protocols out
of the 284 ones supported by nDPI.

The 35 protocols without any tests are:

* P2P/sharing protocols: DIRECT_DOWNLOAD_LINK, OPENFT, FASTTRACK,
EDONKEY, SOPCAST, THUNDER, APPLEJUICE, DIRECTCONNECT, STEALTHNET

* games: CSGO, HALFLIFE2, ARMAGETRON, CROSSFIRE, DOFUS, FIESTA,
FLORENSIA, GUILDWARS, MAPLESTORY, WORLD_OF_KUNG_FU

* voip/streaming: VHUA, ICECAST, SHOUTCAST, TVUPLAYER, TRUPHONE

* other: AYIYA, SOAP, TARGUS_GETDATA, RPC, ZMQ, REDIS, VMWARE, NOE,
LOTUS_NOTES, EGP, SAP

Most of these protocols (expecially the P2P and games ones) have been
inherited by OpenDPI and have not been updated since then: even if they
are still used, the detection rules might be outdated.

However code coverage (of `lib/protocols`) only increases from 65.6% to
68.9%.

Improve Citrix, Corba, Fix, Aimini, Megaco, PPStream, SNMP and Some/IP
dissection.
Treat IPP as a HTTP sub protocol.
Fix Cassandra false positives.

Remove `NDPI_PROTOCOL_QQLIVE` and `NDPI_PROTOCOL_REMOTE_SCAN`:
these protocol ids are defined but they are never used.

Remove Collectd support: its code has never been called. If someone is
really interested in this protocol, we can re-add it later, updating the
dissector.

Add decoding of PPI (Per-Packet Information) data link type.
2022-03-09 22:37:35 +01:00
Toni f646a4bce0 Improved ASN/IP update scripts and CI integration. (#1474)
* CI will print a warning if ASN/IP addresses changed.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-09 13:53:04 +01:00
Zied Aouini 74ae315e36 Implement CI on Windows. (#1483)
* Switch fail fast to True.
* Windows CI.
2022-03-09 10:57:21 +01:00
Ivan Nardi 269be6c7ef Some small fixes (#1481)
FTP: if the authentication fails, stop analyzing the flow
WSD: call the initialization routine; the dissector code has never been
triggered
MINING: fix dissection
2022-03-08 00:21:51 +01:00
sharonenochandSharon Enoch f91218360b Extracting the Azure Origin url from the download link (#1480)
* Extracting the Azure Origin url from the download link

* Response code check to address double quotes

Co-authored-by: Sharon Enoch <sharone@amzetta.com>
2022-03-08 00:21:32 +01:00
Vitaly Lavrov a1451935b8 Errors fixed (#1482)
Fixed errors for bigendian platforms in ndpiReader.
All address and port comparisons and hash calculations are done with
endian in mind.
The get_ndpi_flow_info() function searched for an existing flow for the
forward and reverse direction of the packet.
The ndpi_workflow_node_cmp() function looked for a flow regardless of
the packet's direction. This is what led to an error in determining the
direction of transmission of the packet.

Fixed error in "synscan" test: the number of packets in the forward and
reverse direction is incorrectly defined (verified via tcpdump).

Fixed bug with icmp protocol checksum check for big endian platforms.
2022-03-08 00:20:56 +01:00
Ivan Nardi c345b3c7af EthernetIP: fix integer conversion on big-endian archs (#1477) 2022-03-05 15:05:05 +01:00
Vitaly Lavrov 438f04066b Fixed a bug for BE architectures (#1478)
Fixed a bug in the internal implementation of libgcrypt for bigendian architectures
2022-03-05 11:44:21 +01:00
Ivan Nardi a173d3e98f configure: fix usage of libgpg-error with --with-local-libgcrypt (#1472)
Right now, using external libgcrypt, nDPI is not linked to libgpg-error
because configure script never checks for it.

```
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ CC=gcc-11 CXX=g++-11 CFLAGS="-O3 -g -Werror" ./autogen.sh --enable-debug-messages --with-pcre  --with-local-libgcrypt && make -s -j
[...]
checking for numa_available in -lnuma... yes
checking for pcap_open_live in -lpcap... yes
checking for pthread_setaffinity_np in -lpthread... yes
checking for gcry_cipher_checktag in -lgcrypt... yes             <------- missing check for libgpg-error
checking for pcre_compile in -lpcre... yes
checking that generated files are newer than configure... done
[...]
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ grep HAVE_LIBGPG_ERROR src/include/ndpi_config.h
/* #undef HAVE_LIBGPG_ERROR */

```

Make both libgcrypt and libgpg-error mandatory if
`--with-local-libgcrypt` is used.

Technically speaking, libgpg-error might be optional, because it is used
only for debug messages. However having both libraries mandatory
slightly simplified the logic.
In most environments, libgpg-error is a dependency of libgcrypt anyway,
so having both libraries should be the standard case.
2022-03-05 11:41:37 +01:00
Toni 95a3d4fffe Added autoconf option `--enable-tls-sigs'. (#1471)
* Testing more code in CI environments.
 * Added strict option checking for `./configure' in CI environments.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-03 10:16:08 +01:00
Toni cb62dfd249 Drop support for non-gcrypt builds. (#1469)
* As there is now a builtin, lightweight libgcrypt
   there is no need to disable tls-clho decryption.
 * It is still possible to use a host libgcrypt
   with `--with-local-libgcrypt'.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-02 19:48:46 +01:00
Ivan Nardi e23507d3e9 Internal crypto: increase size of authentication buffer (#1468)
Some QUIC flows are not properly decoded while using internal crypto
code: the authentication buffer is too small.
The new value (like the old one) is arbitrary.

Close #1463
2022-03-02 15:14:57 +01:00
Ivan Nardi 6f431057c5 reader_util: fix parsing of MPLS packets (#1467)
```
==20492==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60300000578c at pc 0x55c47455e3ea bp 0x7ffc62ca1eb0 sp 0x7ffc62ca1ea8
READ of size 4 at 0x60300000578c thread T0
    #0 0x55c47455e3e9 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:1840:16
    #1 0x55c47451b9cd in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #2 0x55c47451c1ab in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17
    #3 0x7f661b50e0b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
    #4 0x55c47445b54d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x61054d) (BuildId: eba4bb4cd43b7101e4f0028ec0fb79087bae0e37)

0x60300000578d is located 0 bytes to the right of 29-byte region [0x603000005770,0x60300000578d)

```
2022-03-02 15:14:41 +01:00
Toni e8559a4127 Add ICMP checksum check and set risk if mismatch detected. (#1464)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-02 13:12:01 +01:00
Luca Deri 6c4df21238 Typo 2022-03-02 12:47:04 +01:00
Toni 4da1453d90 Added configureable ndpi packet processing limit. (#1466)
* The current behaviour ignores any user preferences
   and was also incorrectly implemented, because the
   flow->num_processed_pkts wraps every 65535 and nDPI
   will process packets again until
   NDPI_MAX_NUM_PKTS_PER_FLOW_TO_DISSECT reached.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-02 12:02:09 +01:00
Toni 736a80f9ce Fix libgcrypt(-light/-internal) compile error. (#1465)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-02 11:03:47 +01:00
Ivan Nardi 7a7e4ee69f Add a new flow risk NDPI_ANONYMOUS_SUBSCRIBER (#1462)
The main goal of a DPI engine is usually to determine "what", i.e. which
types of traffic flow on the network.
However the applications using DPI are often interested also in "who",
i.e. which "user/subscriber" generated that traffic.

The association between a flow and a subscriber is usually done via some
kind of DHCP/GTP/RADIUS/NAT mappings. In all these cases the key element
of the flow used to identify the user is the source ip address.

That usually happens for the vast majority of the traffic.

However, depending on the protocols involved and on the position on the net
where the traffic is captured, the source ip address might have been
changed/anonymized. In that case, that address is useless for any
flow-username association.

Example: iCloud Private Relay traffic captured between the exit relay and
the server.
See the picture at page 5 on:
https://www.apple.com/privacy/docs/iCloud_Private_Relay_Overview_Dec2021.PDF

This commit adds new generic flow risk `NDPI_ANONYMOUS_SUBSCRIBER` hinting
that the ip addresses shouldn't be used to identify the user associated
with the flow.
As a first example of this new feature, the entire list of the relay ip
addresses used by Private Relay is added.

A key point to note is that list is NOT used for flow classification
(unlike all the other ip lists present in nDPI) but only for setting this
new flow risk.

TODO: IPv6
2022-02-28 15:25:58 +01:00
Luca Deri 1a8a6aa161 Unless --with-libgcrypt is used, nDPI now uses its internal gcrypt implementation 2022-02-28 15:18:10 +01:00
Ivan Nardi e193f3713d Removed some unused fields (#1461) 2022-02-28 15:03:49 +01:00
Vitaly Lavrov c390085f91 Bug fixing. (#1459)
The '--enable-debug-messages' option works again.
Fixed warning in ahocorasick.c
Fixed integer overflow in ndpiReader.c for 32bit systems.
2022-02-28 15:01:00 +01:00
Toni 00d9a16b6a Added `--enable-code-coverage' build using lcov for coverage generation. (#1430)
* CI job generates a coverage report

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-02-28 14:58:45 +01:00
Ivan Nardi 61a3c2eb5b reader_util: fix TZSP decapsulation (#1460)
```
==38674==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60400008957f at pc 0x5653fcee6434 bp 0x7ffe9b554b50 sp 0x7ffe9b554b48
READ of size 1 at 0x60400008957f thread T0
    #0 0x5653fcee6433 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2050:18
    #1 0x5653fce9fc0d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #2 0x5653fcea03eb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17
    #3 0x7fe71dc3d0b2 in __libc_start_main /build/glibc-YbNSs7/glibc-2.31/csu/../csu/libc-start.c:308:16
    #4 0x5653fcddf67d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x58a67d) (BuildId: 525418a27e8c37d6c492cc3220e0e97809c40f98)

0x60400008957f is located 0 bytes to the right of 47-byte region [0x604000089550,0x60400008957f)
allocated by thread T0 here:
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=45036
2022-02-27 14:18:23 +01:00
Ivan Nardi 8b062295cc Add some scripts to easily update some IPs lists (#1449)
While the lists in a6ff0dd0 and 2f5f445f are somehow provided by the
companies themselves (or by some interested parties), these new lists
are directly extracted from BGP information, via AS prefixes.

*Usually*, these new lists are far more stable than the previous ones.

TODO:
* add some other ASNs (see `src/lib/ndpi_content_match.c.inc`)
* IPv6, as usual :-(
2022-02-25 14:26:26 +01:00
Toni f14dcb8344 Provide some API functions for convenience. (#1456)
* Extended JSON serializsation: risk, risk score, confidence

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-02-25 14:26:04 +01:00
Luca Deri 34e020ac35 Win fixes 2022-02-23 21:00:53 +01:00
Luca Deri 16ed0fd3f8 Replaced strdup with ndpi_strup 2022-02-23 21:37:52 +01:00
Ivan Nardi d506a0e12f Directly drop malformed packets (#1455)
TCP/UDP/ICMP/ICMPv6 packets with invalid L4 header length should be
ignored.
2022-02-22 22:15:46 +01:00
Ivan Nardi 96f8942f75 reader_util: fix parsing of IPv6 extension headers (#1453)
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44942
2022-02-22 19:58:17 +01:00
Ivan Nardi 279779062c reader_util: fix infinite loop in packet dissection (#1454)
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44944
2022-02-22 19:56:41 +01:00
Ivan Nardi fbb9700086 fuzz: purge old sessions (#1451)
At every fuzz iteration (i.e for every trace file):
* keep the same ndpi context (`ndpi_init_detection_module` is very
slow);
* reset the flow table, otherwise it grows indefinitely.

This change should fix the "out-of-memory" errors reported by oss-fuzz.
2022-02-21 20:32:50 +01:00
Ivan Nardi 6c1accd2bd DTLS: fix access to certificate cache (#1450)
```
protocols/tls.c:650:54: runtime error: member access within null pointer of type 'const struct ndpi_tcphdr'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in
protocols/tls.c:650:54: runtime error: load of null pointer of type 'const u_int16_t' (aka 'const unsigned short')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in
AddressSanitizer:DEADLYSIGNAL
=================================================================
==47401==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x55f7a61b661c bp 0x7f38190f91b0 sp 0x7f38190f70e0 T1)
==47401==The signal is caused by a READ memory access.
==47401==Hint: address points to the zero page.
    #0 0x55f7a61b661c in processCertificateElements /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:650:41
    #1 0x55f7a61ac3cc in processCertificate /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:792:7
    #2 0x55f7a61d34e1 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:846:13
```
2022-02-21 20:30:22 +01:00
Ivan Nardi 59993e294e EthernetIP: add missing initialization (#1448)
Fix:1e1cfb89
2022-02-20 10:20:46 +01:00
Ivan Nardi 2f5f445f72 Add support for Google Cloud (#1447)
Differentiate between Google its own apps/services and Google Cloud.
We already do something similar for Amazon vs AWS and Microsoft vs Azure.
2022-02-20 10:19:52 +01:00
Ivan Nardi 75a53ad064 fuzz: make fuzz_ndpi_reader faster (#1446)
Initialize ndpi_workflow_init context only once.

On a quite old notebook, before:
```
$ ./fuzz/fuzz_ndpi_reader -max_total_time="${MAX_TOTAL_TIME:-360}" -print_pcs=1 -workers="${FUZZY_WORKERS:-0}" -jobs="${FUZZY_JOBS:-0}" ./tests/pcap/
[...]
Done 3256 runs in 361 second(s)
```
after:
```
Done 5032771 runs in 361 second(s)   <----------- ~1400X
```
oss-fuzz will be happy!
2022-02-20 10:19:22 +01:00
Vitaly Lavrov f2411958d7 Added lightweight implementation of libgcrypt. (#1444)
Implementation borrowed from the
https://github.com/ARMmbed/mbedtls.git project (v3.1.0)

Speed testing (Xeon(R) CPU E3-1230 V2 @ 3.30GHz):

gcrypt-gnu         Test md   2897 ms enc   2777 ms dec    942 ms
gcrypt-int         Test md   3668 ms enc   1312 ms dec   2836 ms
gcrypt-int-noaesni Test md   3652 ms enc   1916 ms dec   4458 ms

gcrypt-gnu-nonopt  Test md   3763 ms enc   4978 ms dec   3999 ms

gcrypt-gnu-nonopt - libgcrypt compiled without hardware acceleration
  --disable-padlock-support --disable-aesni-support \
  --disable-shaext-support --disable-pclmul-support \
  --disable-sse41-support --disable-drng-support \
  --disable-avx-support --disable-avx2-support \
  --disable-neon-support --disable-arm-crypto-support \
  --disable-ppc-crypto-support
  --disable-amd64-as-feature-detection
2022-02-20 10:16:46 +01:00
Ivan Nardi f28a3b293e Fix compilation and sync unit tests results (#1445)
'strcasestr' is not defined in all enviroments: quicker fix is to use
'ndpi_strncasestr' instead.
2022-02-19 19:18:02 +01:00
Luca Deri a2878af1ee Added newflow risk NDPI_HTTP_CRAWLER_BOT 2022-02-17 17:20:52 +01:00
Luca Deri 8a2a47e62a Silenced
NDPI_SUSPICIOUS_DGA_DOMAIN,
   NDPI_BINARY_APPLICATION_TRANSFER,
   NDPI_HTTP_NUMERIC_IP_HOST,
   NDPI_MALICIOUS_JA3,

for predefined connectivity check and cybersec categories
2022-02-14 23:38:21 +01:00
Luca Deri 92da30f017 Extended cybersecurity protocol dissection 2022-02-14 23:19:21 +01:00
Luca Deri c4113ffd7e Added SNMP error code check 2022-02-14 23:17:51 +01:00
Luca Deri de6905f41b Exteended cybersecurity list 2022-02-10 20:16:48 +01:00
Luca Deri 023618c13a Invalid prototupe fix 2022-02-10 19:38:11 +01:00
Ivan Nardi ada7d32c50 HSRP: fix dissection over IPv6 (#1443)
Handle all message types.
2022-02-10 11:46:32 +01:00
Luca Deri 1431aa3d1c Added cybersecurity category mapping to string 2022-02-10 10:35:34 +01:00
Luca Deri 5ddbbf4dd2 Added cybersecurity protocol and category that groups traffic towards leading cybersecurity companies and CDNs, useful to make destinations that should be marked as trusted in firewalls and security gateways 2022-02-10 09:50:06 +01:00
Ivan Nardi b5eea436ae HSRP: add support for IPv6 (#1440) 2022-02-09 11:47:37 +01:00
Dmytrii Vitman 4cf853548c Added VXLAN dissector (#1439)
* RFC 7348
2022-02-09 11:47:09 +01:00
Ivan Nardi f229068fd3 Fix memory access in ndpi_strnstr() (#1438)
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44482

It should be the same error reported (and only partially fixed) in
79968f32
2022-02-09 11:46:39 +01:00
Ivan NardiandLuca Deri a6ff0dd0e3 Add few scripts to easily update some IPs lists (#1436)
* Add few scripts to easily update some IPs lists

Some IPs lists should be updated frequently: try to easy the process.
The basic idea is taken from d59fefd0 and a8fe74e5 (for Azure
addresses): one specific .c.inc file and one script for each protocol.

Add the possibility to don't load a specific list.

Rename the old NDPI_PROTOCOL_HOTMAIL id to NDPI_PROTOCOL_MS_OUTLOOK,
to identify Hotmail/Outlook/Exchange flows.

TODO: ipv6

Remove the 9 addresses associated to BitTorrent: they have been added in
e2f21116 but it is not clear why all the traffic to/from these ips
should be classified as BitTorrent.

* Added quotes

* Added quotes

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2022-02-09 11:45:48 +01:00
ol-andreyizrailevandAndrey Izrailev 8cc5cb9f76 Increment current/total number of active flows on successful flow insertion (#1434)
Memory allocation or ndpi_tsearch might fail, so the two values should be
incremented only when insertion actually happened.

Co-authored-by: Andrey Izrailev <Andrey.Izrailev@oktetlabs.ru>
2022-02-09 11:43:10 +01:00
Luca Deri 9838ceb7cd Added ndpi_serialize_string_string_len() APi call
Fixed CSV string serialization
2022-02-08 22:17:26 +01:00
Luca Deri be5a29ba9d Added HSRP protocol detection
Removed attic directory now obsolete
2022-02-08 18:04:57 +01:00
Luca Deri 922a7be897 Added check to ignore multicast packets marking the as Skype 2022-02-08 17:23:25 +01:00
Toni d59f0327a5 Improved MDNS/LLMNR detection. (#1437)
* Checking for port 5353/5355 is not enough.
 * Added additional multicast address and header checks.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-02-07 18:05:23 +01:00
Ivan Nardi ff38d4b980 TLS: fix parsing of certificate elements (#1435)
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44280
```
==263603==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x592478 in ndpi_is_printable_string ndpi/src/lib/ndpi_utils.c:2200:9
    #1 0x5b047c in processCertificateElements ndpi/src/lib/protocols/tls.c:400:7
    #2 0x5ac880 in processCertificate ndpi/src/lib/protocols/tls.c:790:7
    #3 0x5c3a32 in processTLSBlock ndpi/src/lib/protocols/tls.c:844:13
    #4 0x5c2c61 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:973:2
    #5 0x5c117d in ndpi_search_tls_wrapper ndpi/src/lib/protocols/tls.c:2367:5
    #6 0x552a50 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4792:6
```
2022-02-07 18:05:12 +01:00
Ivan Nardi 79968f3245 Sync utests (#1433)
* Sync utest results

* Fix read-heap-buffer-overflow error reported by CI

See: https://github.com/ntop/nDPI/runs/5055876515?check_suite_focus=true
2022-02-04 15:57:17 +01:00
Alfredo Cardigliano f5035521e8 Add comment 2022-02-03 18:43:31 +01:00
Luca Deri c40496eaac Updated test results 2022-02-03 13:54:38 +01:00
Luca Deri 7aef27f85e Added NDPI_ERROR_CODE_DETECTED risk 2022-02-03 13:20:54 +01:00
Luca Deri 0dd5ac89f8 Renamed DCERPC to more generic RPC protocol so we can use also for other types of RPCs (not limited to DCE)
Extended HTTP plugin to support RPC
Improved HTTP crear text detection to limit it to Basic and Digest
2022-02-03 13:06:27 +01:00
Luca Deri 6975f265ec Typo 2022-02-03 12:21:51 +01:00
Luca Deri ae09e7fb14 Improved risks description 2022-02-03 12:19:01 +01:00
Luca Deri a96d7eb180 Updated risk documentation 2022-02-03 12:10:18 +01:00
Luca 37ff626e78 Added new IDN/Punycode risk for spotting internationalized domain names 2022-02-03 09:17:54 +01:00
Luca Deri cd3d720ae3 Added missing __sync_fetch_and_add() definition in Windows 2022-02-02 10:36:50 +01:00
Luca 46bd093934 Moved to 4.3 2022-02-01 09:13:55 +01:00
Alfredo Cardigliano 03cb8efad6 Update changelog 2022-01-31 18:38:02 +01:00
Ivan Nardi 5bb5bec477 Remove struct ndpi_id_struct (#1427)
Remove the last uses of `struct ndpi_id_struct`.
That code is not really used and it has not been updated for a very long
time: see #1279 for details.

Correlation among flows is achieved via LRU caches.

This change allows to further reduce memory consumption (see also
91bb77a8).

At nDPI 4.0 (more precisly, at a6b10cf, because memory stats
were wrong until that commit):
```
nDPI Memory statistics:
	nDPI Memory (once):      221.15 KB
	Flow Memory (per flow):  2.94 KB
```
Now:
```
nDPI Memory statistics:
	nDPI Memory (once):      235.27 KB
	Flow Memory (per flow):  688 B        <--------
```
i.e. memory usage per flow has been reduced by 77%.

Close #1279
2022-01-30 19:18:12 +01:00
Ivan Nardi 1fdcddb33d Remove Playstation VUE protocol (#1426)
PS VUE service has been discontinued on January 30, 2020
https://en.wikipedia.org/wiki/PlayStation_Vue
2022-01-30 19:17:53 +01:00
Luca Deri 5f745ee425 Commented old code (see https://github.com/ntop/nDPI/pull/1425) 2022-01-30 19:17:23 +01:00
Ivan Nardi f455421870 Improve protocol stacks (#1425)
We should have two protocols in classification results only when the
"master" protocol allows some sub-protocols.

Classifications like `AmazonAWS`, `TLS/AmazonAWS`, `DNS/AmazonAWS` are
fine. However classifications like `NTP/Apple`, `BitTorrent/Azure`,
`DNScrypt.AmazonAWS` or `NestLogSink.Google` are misleading.
For example, `ndpiReader`shows  `BitTorrent/Azure` flows under `Azure`
statistics; that seems to be wrong or, at least, very misleading.

This is quite important since we have lots of addresses from CDN
operators.

The only drawback of this solution is that right now ICMP traffic is
classified simply as `ICMP`; if we are really interested in ICMP stuff
we can restore the old behaviour later.
2022-01-30 19:15:42 +01:00
Ivan Nardi 513e386959 Extend protocols support (#1422)
Add detection of AccuWeather site/app and Google Classroom.
Improve detection of Azure, Zattoo, Whatsapp, MQTT and LDAP.

Fix some RX false positives.

Fix some "Uncommon TLS ALPN"-risk false positives.

Fix "confidence" value for some Zoom/Torrent classifications.

Minor fix in Lua script for Wireshark extcap.

Update .gitignore file.

Let GitHub correctly detect the language type of *.inc files.

Zattoo example has been provided by @subhajit-cdot in #1148.
2022-01-29 09:19:26 +01:00
Toni 9b8679a320 Fix some race conditions by using atomic operations. (#1420)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-29 09:18:51 +01:00
Ivan Nardi 0c70411b1b Make some protocols more "big-endian" friendly (#1402)
See #1312
2022-01-29 09:18:32 +01:00
Ivan Nardi 86b97ffb73 Sync unit tests results (#1423)
Fix: 7a3aa41a
2022-01-28 17:17:39 +01:00
Luca 7a3aa41afc Updated alert description case 2022-01-28 15:02:42 +01:00
Toni c53c82d482 Fixed heap overflow in nDPI realloc wrapper if new size < old size. (#1421)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-27 15:29:16 +01:00
Ivan Nardi 9dcc4a2334 Kerberos, TLS, example: fix some memory errors (#1419)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43823
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43921
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43925
2022-01-27 10:59:35 +01:00
Toni 011ee3ecbd Fixed wrong ip tuple comparison. #1386 (#1418)
* Added u32 pads to `union ip_tuple` so btree search should now work as expected.
   The bug caused new flow's when the remote answers, resulting in two Flows per direction. Fail.
 * Fixed a race condition during shutdown phase.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-26 15:47:38 +01:00
Luca Deri 6d185c052e Added support for the .goog Google TLD 2022-01-26 15:22:33 +01:00
Toni 2cd0479204 Serializing empty `risk blocks' pollutes the resulting string. (#1417)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-26 13:36:45 +01:00
Luca Deri 81b33f0d06 Updated test results after the risk NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE has been added 2022-01-26 09:46:46 +01:00
Luca Deri 58a9aff17c Added NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE flow risk
Added ndpi_set_tls_cert_expire_days() API call to modify the number of days for triggering the above alert that by default is set to 30 days
2022-01-26 09:23:23 +01:00
Luca Deri 14e8808f03 Removed pandora.tv from pandora protocol as they are different services that shouldd not be mixed 2022-01-25 19:11:16 +01:00
Luca Deri bbb3b8349f Improved pandora TV detection 2022-01-24 18:28:43 +01:00
Luca Deri b6b4967aa6 Improved Zoom protocol detection 2022-01-23 20:59:36 +01:00
Luca Deri a8fe74e502 Tool for generating automatically the Azure IP list 2022-01-23 18:56:51 +01:00
Alfredo Cardigliano 20b9f01f12 Fix ndpi_serialize_string_int64 2022-01-21 17:35:38 +01:00
Alfredo Cardigliano a0bb0ab0ca Add unit test for ndpi_serialize_string_int64 2022-01-21 13:05:58 +01:00
havsah e16b6a18be Fix Grease values parsing (#1416)
The check for grease was too broad and filtered some valid values.
In particular, the value 257 was skipped because it matched the previous check.
This has been discovered while parsing tests/pcap/443-firefox.pcap
expected ja3:
771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-51-57-47-53-10,0-23-65281-10-11-35-16-5-51-43-13-45-28-21,29-23-24-25-256-257,0
previously generated ja3:
771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-51-57-47-53-10,0-23-65281-10-11-35-16-5-51-43-13-45-28-21,29-23-24-25-256,0

Signed-off-by: Patrick Havelange <patrick.havelange_ext@softathome.com>
2022-01-21 12:40:35 +01:00
Luca Deri 6f336f8307 Added JA3 in risj exceptions 2022-01-20 22:22:56 +01:00
Luca Deri d91477b18d Fixed certificate mismatch check 2022-01-19 23:10:10 +01:00
Luca Deri 771b6606c8 Fixed an issue on CentOS 7 introduced by
https://github.com/ntop/nDPI/commit/c2b7d77784beeae5f86f19e33ba3da5ddce55898
2022-01-19 10:19:58 +01:00
Luca Deri 9eee257be0 Updated confidence type 2022-01-18 22:28:58 +01:00
Ivan Nardi 9f050fa0a6 TLS, H323, examples: fix some memory errors (#1414)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=26880
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=26906
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43782
https://oss-fuzz.com/testcase-detail/6334089358082048
2022-01-18 21:52:37 +01:00
Ivan Nardi bd036f96f9 Netbios, CSGO: fix two memory errors (#1413)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43754
https://oss-fuzz.com/testcase-detail/5329842395021312
2022-01-18 19:11:01 +01:00
Sam JamesandLuca Deri c2b7d77784 build: respect environment options more (#1392)
* build: update m4/ax_pthread.m4 from serial 23 -> serial 31

Update ax_pthread.m4 to the latest version from the autoconf-archive
project.

Signed-off-by: Sam James <sam@gentoo.org>

* build: properly detect AR, CC, RANLIB

It's necessary to be able to override choice of AR/CC/RANLIB and other toolchain
variables/tools for cross-compilation, testing with other toolchains, and
to ensure the compiler chosen by the user is actually used for the build.

Previously, GNU_PREFIX was kind-of used for this but this isn't a standard
variable (at all) and it wasn't applied consistently anyway.

We now use the standard autoconf mechanisms for finding these tools.

(RANLIB is already covered by LT_INIT.)

Signed-off-by: Sam James <sam@gentoo.org>

* build: use $(MAKE)

This ensures that parallel make works correctly, as otherwise, a fresh
make job will be started without the jobserver fd, and hence
not know about its parent, forcing -j1.

* build: respect CPPFLAGS, LDFLAGS

- CPPFLAGS is for the C preprocessor (usually for setting defines)
- LDFLAGS should be placed before objects for certain flags to work
  (e.g. -Wl,--as-needed)

Signed-off-by: Sam James <sam@gentoo.org>

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2022-01-18 14:30:14 +01:00
Ivan Nardi e4a7990a4e H323: fix a use-after-poison error (#1412)
Detected by oss-fuzz
See: https://oss-fuzz.com/testcase-detail/6730505580576768

Fix a function prototype
Update a unit test results
2022-01-17 19:47:27 +01:00
Luca Deri d19bd96555 Improved Badoo detection (missing mobile app domain) 2022-01-17 18:54:27 +01:00
Luca Deri 6877e46273 Added Badoo detection 2022-01-17 15:13:14 +01:00
Luca 6679453d86 Adds some risk exceptions for popular services and domain names
via a new (internal) function named ndpi_add_domain_risk_exceptions()
2022-01-17 09:00:25 +01:00
Ivan Nardi 1ff308af09 TLS: fix a use-of-uninitialized-value error (#1411)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43705
2022-01-16 20:31:48 +01:00
Ivan Nardi 05734f3d31 Zattoo: fix Null-dereference READ with ipv6 traffic (#1410)
Fix: 20b5f6d7
Detected by oss-fux:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43700
2022-01-16 19:28:39 +01:00
Ivan Nardi 611c753da5 XBox, Diameter: fix dissectors initialization (#1405)
These dissectors have *never* been triggered because their registration
functions use the wrong parameter/bitmask.
Diameter code is buggy since the origianl commit (1d108234), while
XBox code since 5266c726.

Fix some false positives in Xbox code.
2022-01-16 16:19:00 +01:00
Luca Deri f3af39ee42 Added performance tests tools 2022-01-16 12:47:56 +01:00
Luca Deri 42d74171b2 Minor cosmetic changes 2022-01-16 12:47:56 +01:00
Luca Deri dd325248f1 Reduced Patricia tree bucket memory footprint 2022-01-16 12:47:56 +01:00
Ivan Nardi e46b508a36 Kerberos: fix use-of-uninitialized-value error (#1409)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43677
2022-01-15 20:53:36 +01:00
Ivan Nardi eb5d7b07af TLS: fix heap-buffer-overflow error (#1408)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43664
2022-01-15 18:47:59 +01:00
Ivan Nardi 63c44a24b6 STUN: fix "confidence" value for some classifications (#1407) 2022-01-15 17:35:06 +01:00
Ivan Nardi 20b5f6d7cc Improve IPv6 support, enabling IPv6 traffic on (almost) all dissectors. (#1406)
Follow-up of 7cba34a1
2022-01-15 17:34:56 +01:00
Ivan Nardi 7f69de0b51 Restore a unit test result (#1403)
Deleted, probably by mistake, in 406ac7e8

Fix Makefile and add compilation of `rrdtool` in CI tests
2022-01-13 23:14:28 +01:00
Luca Deri 533b3ab0e3 FreeBSD fixes 2022-01-13 21:15:21 +01:00
Luca Deri 406ac7e8c8 Added the ability to specify trusted issueDN often used in companies to self-signed certificates
This allows to avoid triggering alerts for trusted albeit private certificate issuers.

Extended the example/protos.txt with the new syntax for specifying trusted issueDN.
Example:

trusted_issuer_dn:"CN=813845657003339838, O=Code42, OU=TEST, ST=MN, C=US"
2022-01-13 19:06:21 +01:00
Vinicius Silva Nogueira dc60cd09c4 fix ahocorasick on big-endian machines (#1401) 2022-01-13 17:35:06 +01:00
Luca Deri d59fefd06e Improved MicrosoftAzure detection 2022-01-12 21:49:31 +01:00
Luca Deri 1e1cfb89d2 Added EthernetIP dissector 2022-01-12 21:48:39 +01:00
Ivan Nardi d4da3b6516 Run fuzzing tests also on push event (like the CI tests) (#1400) 2022-01-12 21:04:01 +01:00
Ivan Nardi f30fbc3212 Fix DGA false positive (#1399)
Close #1397
2022-01-12 20:51:29 +01:00
Ivan Nardi b080a1c136 Fix two use-of-uninitialized-value errors (#1398)
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=40269
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=41432

Fix fuzz compilation (follow-up of f5545a80)
2022-01-12 20:24:57 +01:00
Luca Deri 552d199d2e Removed outdated comment 2022-01-11 21:46:30 +01:00
Luca Deri f5545a80f9 Removed legacy code 2022-01-11 21:45:27 +01:00
Ivan Nardi 06f3b07909 QUIC: fix an integer overflow (#1396)
Reported by oss-fuzz:
```
==685288==ERROR: AddressSanitizer: SEGV on unknown address 0x61a100000687 (pc 0x0000005aba64 bp 0x7ffe3f29f510 sp 0x7ffe3f29f400 T0)
==685288==The signal is caused by a READ memory access.
SCARINESS: 20 (wild-addr-read)
 #0 0x5aba64 in quic_len ndpi/src/lib/protocols/quic.c:203:12
 #1 0x5aba64 in decrypt_initial_packet ndpi/src/lib/protocols/quic.c:993:16
 #2 0x5aba64 in get_clear_payload ndpi/src/lib/protocols/quic.c:1302:21
 #3 0x5aba64 in ndpi_search_quic ndpi/src/lib/protocols/quic.c:1658:19
 #4 0x579f00 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4683:6
 #5 0x57abe6 in ndpi_check_flow_func ndpi/src/lib/ndpi_main.c:0
 #6 0x583b2c in ndpi_detection_process_packet ndpi/src/lib/ndpi_main.c:5545:15
 #7 0x55e75e in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_process_packet.c:30:3
[...]
```
2022-01-11 17:57:34 +01:00
Ivan Nardi 3a087e951d Add a "confidence" field about the reliability of the classification. (#1395)
As a general rule, the higher the confidence value, the higher the
"reliability/precision" of the classification.

In other words, this new field provides an hint about "how" the flow
classification has been obtained.
For example, the application may want to ignore classification "by-port"
(they are not real DPI classifications, after all) or give a second
glance at flows classified via LRU caches (because of false positives).

Setting only one value for the confidence field is a bit tricky: more
work is probably needed in the next future to tweak/fix/improve the logic.
2022-01-11 15:23:39 +01:00
Luca Deri a2916d2e4c Readme for using fuzzer with nDPI 2022-01-11 15:12:45 +01:00
Luca Deri c02574807e Updated result 2022-01-09 18:52:30 +01:00
Luca Deri 708d4ea33a Improved user agent analysis 2022-01-09 18:47:47 +01:00
Ivan Nardi bf69321a29 GTP: fix some false positives (#1394) 2022-01-08 20:40:24 +01:00
Ivan Nardi 4d27001d1d Remove some unused fields (#1393) 2022-01-08 20:40:10 +01:00
Luca Deri eac3ae9ded Improved bittorrent detection 2022-01-06 19:50:31 +01:00
Luca Deri f24f68f59d Invalid check 2022-01-06 07:15:45 +01:00
Luca Deri 8f805a8163 Added missing dependency 2022-01-05 11:11:58 +01:00
Alfredo Cardigliano 948c0367f9 Update issue tempalte 2022-01-03 17:26:23 +01:00
Alfredo Cardigliano 23a4761276 Update copyright 2022-01-03 11:00:45 +01:00
Luca Deri d72a40a0fc Improved BitTorrent guess 2021-12-26 19:41:35 +01:00
Luca Deri c4ac53a03f Added support for Log4J/Log4Shell detection in nDPI via a new flow risk named NDPI_POSSIBLE_EXPLOIT 2021-12-23 21:30:16 +01:00
Ivan Nardi fdb6481cd6 Add support for ICloud Private Relay (#1390)
See:
https://www.apple.com/privacy/docs/iCloud_Private_Relay_Overview_Dec2021.PDF

TODO: an up-to-date list of egress IP ranges is publicly available. Can
we use it somehow?
2021-12-22 19:55:57 +01:00
Ivan Nardi 91bb77a880 A final(?) effort to reduce memory usage per flow (#1389)
Remove some unused fields and re-organize other ones.
In particular:
* Update the parameters of `ndpi_ssl_version2str()` function
* Zattoo, Thunder: these timestamps aren't really used.
* Ftp/mail: these protocols are dissected only over TCP.
* Attention must be paid to TLS.Bittorrent flows to avoid invalid
read/write to `flow->protos.bittorrent.hash` field.

This is the last(?) commit of a long series (see 22241a1d, 227e586e,
730c2360, a8ffcd8b) aiming to reduce library memory consumption.

Before, at nDPI 4.0 (more precisly, at a6b10cf7, because memory stats
were wrong until that commit):
```
nDPI Memory statistics:
	nDPI Memory (once):      221.15 KB
	Flow Memory (per flow):  2.94 KB
```
Now:
```
nDPI Memory statistics:
	nDPI Memory (once):      231.71 KB
	Flow Memory (per flow):  1008 B       <---------
```
i.e. memory usage per flow has been reduced by 66%, dropping below the
psychological threshold of 1 KB.

To further reduce this value, we probably need to look into #1279:
let's fight this battle another day.
2021-12-22 19:54:06 +01:00
Alfredo Cardigliano 02da143e45 Add links to other user guides 2021-12-22 10:23:14 +01:00
Luca Deri 4c02c81bf4 Improved bittorrent heuristic 2021-12-21 23:04:27 +01:00
Ivan Nardi a16b4aa564 Fix a global-buffer-overflow when loading Azure list (#1388) 2021-12-19 22:29:25 +01:00
Luca Deri 7aad97fffb Added Microsoft Azure support 2021-12-19 18:59:45 +01:00
Ivan Nardi 7153b8933c Improve/add several protocols (#1383)
Improve Microsoft, GMail, Likee, Whatsapp, DisneyPlus and Tiktok
detection.
Add Vimeo, Fuze, Alibaba and Firebase Crashlytics detection.

Try to differentiate between Messenger/Signal standard flows (i.e chat)
and their VOIP (video)calls (like we already do for Whatsapp and
Snapchat).

Add a partial list of some ADS/Tracking stuff.

Fix Cassandra, Radius and GTP false positives.
Fix DNS, Syslog and SIP false negatives.

Improve GTP (sub)classification: differentiate among GTP-U, GTP_C and
GTP_PRIME.

Fix 3 LGTM warnings.
2021-12-18 13:24:51 +01:00
Ivan Nardi 58b33dcb2d TLS: add support for IPV6 in Subject Alt Names field (#1385) 2021-12-18 12:03:46 +01:00
Ivan Nardi ec6eea0c23 TLS: fix usage of certificate cache (#1384) 2021-12-18 12:03:32 +01:00
Luca Deri 74519f6e10 Improved bittorrent detection 2021-12-17 21:17:18 +01:00
Ivan Nardi f11b215cfb QUIC: fix old versions of GQUIC on big-endian machines (#1387)
Credits to @viniciussn (see #1312)
2021-12-17 12:10:34 +01:00
Luca Deri 8e512ff503 Improved BitTorrent classification 2021-12-07 21:49:00 +01:00
Luca Deri 519d70fa04 Optimized ndpi_serialize_string_int32 2021-12-07 15:54:23 +01:00
Luca Deri 85b396ca43 Fixed issue that prevented alt certificate names to be fully detected when ipAddress and rfc822Name were specified in certificates 2021-12-07 14:45:42 +01:00
Luca Deri cefcc25b13 Added missing install target 2021-12-07 09:00:58 +01:00
Ivan Nardi 947896ad7d Fix configure script (after fb85dac9) (#1381)
Fix/disable some LGTM warnings
2021-12-04 20:22:05 +01:00
Ivan Nardi 7cba34a1e3 Improve IPv6 support, enabling IPv6 traffic on (almost) all dissectors. (#1380)
There are no valid reasons for a (generic) protocol to ignore IPv6
traffic.
Note that:
* I have not found the specifications of "CheckPoint High Availability
Protocol", so I don't know how/if it supports IPv6
* all LRU caches are still IPv4 only

Even if src_id/dst_id stuff is probably useless (see #1279), the right
way to update the protocol classification is via `ndpi_set_detected_protocol()`
2021-12-04 13:30:10 +01:00
Ivan Nardi 6e86e6d924 QUIC: add support for QUICv2 (draft 00) (#1379)
It is already time to start looking at the new QUIC version.
See: https://datatracker.ietf.org/doc/html/draft-ietf-quic-v2-00
2021-12-04 13:29:30 +01:00
Toni 4d60c70c8b Make serialize risk and proto not dependant on any flow. (#1377)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-12-04 13:28:55 +01:00
Luca Deri fb85dac905 Configure improvements
Fixed invalid anomaly detection in rrd_anomaly
2021-12-04 13:21:57 +01:00
Luca Deri fe2822c6a8 Added example for finding similarities in RRDs using nDPI statistical APIs 2021-12-04 10:09:01 +01:00
Luca Deri 6ab1367846 Added quick mode (-q) 2021-12-03 21:18:03 +01:00
Luca Deri 8c4ae7527b Updated rrdtool makefile 2021-12-03 21:10:43 +01:00
Luca Deri 1b077ba65f Demo tool for finding anomalies in RRD files 2021-12-03 17:36:49 +01:00
Alfredo Cardigliano 3dd9b5176e Remove printf when not required 2021-12-01 10:24:12 +01:00
Alfredo Cardigliano 1d3428d538 Improve JSON serialization 2021-12-01 10:04:17 +01:00
Luca Deri 33557c5fbc Improved CSV serialization 2021-11-30 22:15:26 +01:00
Luca Deri 5e0befd56c Fixed seriealizer unit 2021-11-30 17:24:42 +01:00
Luca Deri 4f81cdc103 Removed spaces from some protocol names as they create problems with some applications 2021-11-30 10:12:24 +01:00
Ivan Nardi b1e9245d94 ndpiReader: slight simplificaton of the output (#1378) 2021-11-27 17:32:23 +01:00
Luca Deri 320bcfafa2 Added ndpi_serializer_skip_header() serialization API 2021-11-26 22:18:59 +01:00
Luca Deri a7b5e09195 Added Salesforce detection 2021-11-26 19:07:45 +01:00
Luca Deri 9d59b8b1a9 HTTP proxy improvement 2021-11-25 23:17:27 +01:00
Luca Deri ea435c46f5 Reworked HTTP protocol dissection including HTTP proxy and HTTP connect 2021-11-25 22:53:46 +01:00
Luca Deri 772957349b Aded default port to HTTP_CONNECT 2021-11-25 21:46:00 +01:00
Luca Deri 40ff2c54c1 Added googletagmanager.com to the host_match list 2021-11-25 21:45:38 +01:00
Ivan Nardi a8ffcd8bb0 Rework how hostname/SNI info is saved (#1330)
Looking at `struct ndpi_flow_struct` the two bigger fields are
`host_server_name[240]` (mainly for HTTP hostnames and DNS domains) and
`protos.tls_quic.client_requested_server_name[256]`
(for TLS/QUIC SNIs).

This commit aims to reduce `struct ndpi_flow_struct` size, according to
two simple observations:
 1) maximum one of these two fields is used for each flow. So it seems safe
to merge them;
 2) even if hostnames/SNIs might be very long, in practice they are rarely
longer than a fews tens of bytes. So, using a (single) large buffer is a
waste of memory for all kinds of flows. If we need to truncate the name,
we keep the *last* characters, easing domain matching.

Analyzing some real traffic, it seems safe to assume that the vast
majority of hostnames/SNIs is shorter than 80 bytes.

Hostnames/SNIs are always converted to lowercase.

Attention was given so as to be sure that unit-tests outputs are not
affected by this change.

Because of a bug, TLS/QUIC SNI were always truncated to 64 bytes (the
*first* 64 ones): as a consequence, there were some "Suspicious DGA
domain name" and "TLS Certificate Mismatch" false positives.
2021-11-24 10:46:48 +01:00
Ivan Nardi fd02e1b304 QUIC: fix extra dissection (#1376)
When we have fully reassembled the Client Hello, we need to stop extra
dissection.
2021-11-23 10:57:13 +01:00
Luca Deri 4c39ed293e BitTorrent dissector code cleanup and merge of https://github.com/ntop/nDPI/pull/1374 2021-11-17 23:18:22 +01:00
Luca Deri 2e2d4c911a BitTorrent detection improvements 2021-11-17 22:41:35 +01:00
Luca Deri 2a6e5c6341 Improved BT hashing to minimize false positives 2021-11-17 10:47:21 +01:00
Luca Deri a6545ea095 bittorrent old code cleanup. Enlarged BT cache 2021-11-16 10:05:18 +01:00
Luca Deri f2a279e441 Improved BitTorrent detection 2021-11-15 17:25:13 +01:00
Ivan Nardi afc2b641eb Fix writes to flow->protos union fields (#1354)
We can write to `flow->protos` only after a proper classification.

This issue has been found in Kerberos, DHCP, HTTP, STUN, IMO, FTP,
SMTP, IMAP and POP code.
There are two kinds of fixes:
 * write to `flow->protos` only if a final protocol has been detected
 * move protocol state out of `flow->protos`
The hard part is to find, for each protocol, the right tradeoff between
memory usage and code complexity.

Handle Kerberos like DNS: if we find a request, we set the protocol
and an extra callback to further parsing the reply.

For all the other protocols, move the state out of `flow->protos`. This
is an issue only for the FTP/MAIL stuff.

Add DHCP Class Identification value to the output of ndpiReader and to
the Jason serialization.

Extend code coverage of fuzz tests.

Close #1343
Close #1342
2021-11-15 16:20:57 +01:00
ragostino da47357762 Update ndpi_main.c for HTTP line parser optimization and cleaning (#1372)
* Update ndpi_main.c

HTTP line parser: optimization and cleaning

* Update ndpi_main.c
2021-11-11 12:38:31 +01:00
Ivan Nardi acb1de69aa Reduce memory used by ndpiReader (#1371)
`ndpiReader` is only an example, aiming to show nDPI capabilities
and integration, without any claim about performances.

Nonetheless its memory usage per flow is *huge*, limiting the kinds
of traces that we can test on a "normal" hardware (example: scan
attacks).

The key reason of that behaviour is that we preallocate all the memory
needed for *all* the available features.

Try to reduce memory usage simply allocating some structures only
when they are really needed. Most significant example: JOY algorithms.

This way we should use a lot less memory in the two most common
user-cases:
 * `ndpiReader` invoked without any particular flag (i.e `ndpiReader -i
$FILENAME_OR_IFACE`)
 * internal unit tests

Before (on x86_64):
```
struct ndpi_flow_info {
[...]
	/* size: 7320, cachelines: 115, members: 72 */
```
After:
```
struct ndpi_flow_info {
[...]
	/* size: 2128, cachelines: 34, members: 75 */
```
2021-11-11 12:37:25 +01:00
Ivan Nardi 3e5491fa10 Add detection of OCSP (#1370)
This protocol is detected via HTTP Content-Type header.

Until 89d548f9, nDPI had a dedicated automa (`content_automa`) to
classify a HTTP flow according to this header. Since then, this automa has
been useless because it is always empty.
Re-enable it to match only a string seems overkilling.

Remove all `content_automa` leftovers.
2021-11-11 12:36:55 +01:00
Ivan Nardi 0f168d9150 IMAP, POP3, SMTP: improve dissection (#1368)
Avoid NATS false positives
2021-11-11 11:55:56 +01:00
Luca Deri 9abf528c8e BitTorrent cache debug 2021-11-08 12:42:15 +01:00
Ivan Nardi 5464bad6db Differentiate between standard Amazon stuff (i.e market) and AWS (#1369) 2021-11-04 00:20:45 +01:00
Luca Deri 4173775b60 Improved BitTorrent detection 2021-11-04 00:20:26 +01:00
Ivan Nardi a8a3c6f7c2 Fix use-of-uninitialized-value (#1366)
Close #1346
2021-11-03 15:40:07 +01:00
Ivan Nardi 55880e4ae4 TLS: fix two warnings (#1365)
Disable unit tests on CI for big-endian target. We know we have multiple
issues on big-endian architectures (see #1312) and so the unit tests
always fail there. Ignore this error for the time being and let the CI
pass if we don't have other issues.

Remove an unused automa definition
2021-11-02 22:08:15 +01:00
Luca Deri 1f6c94e663 Updated test results 2021-11-02 12:08:20 +01:00
Luca Deri b11065b2e3 Fixed certificate mismatch for TLS flows with no client hello observed 2021-11-02 12:06:18 +01:00
Luca Deri 9f8c1e8630 BitTorrent detection improvements 2021-10-31 18:45:13 +01:00
Luca Deri a9aa46287d Little change in TLS debug traces 2021-10-28 10:16:33 +02:00
Luca Deri a25d9d871e Updated results 2021-10-27 23:04:29 +02:00
Luca Deri 5283755afa Fixed TLS certificate dissection of subjectAltName 2021-10-27 22:51:59 +02:00
Luca Deri aaa474a37d Minor code cleanup 2021-10-27 09:32:11 +02:00
Ivan Nardi 6edb7bedd7 Avoid overwriting valid protocol in ndpi_detection_giveup (#1360)
We should avoid updating any valid protocol in `ndpi_detection_giveup`; we
should try to find a proper classification only if the flow is still
completely unclassified.

For example in the attached pcap there is a valid TLS session, recognized
as such by TLS dissector. However, the `ndpi_detection_giveup`function
updates it to "HTTP/TLS" (!?) simply because the server port is 80.

Note that the real issue is not the wrong classification, but the
wrong access to `flow->protos` union. If we already set some fields of
`flow->protos` and we change the protocol in `ndpi_detection_giveup`, we
might end up freeing some invalid pointers in `ndpi_free_flow_data`
(no wonder this issue has been found while fuzzing #1354)

Fix GIT and TLS dissectors (issues found by CI fuzzer)
2021-10-27 09:23:07 +02:00
Toni c4eebaaf5e Serialize additional information stored in the flow struct. (#1362)
* Changed function signature of ndpi_flow2json (removed unused vlan_id; API break)
 * Serialize NTP information.
 * Improved QUIC serialization.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-27 09:22:19 +02:00
Luca Deri f41d8ac106 Improved STUN and RTP detection 2021-10-27 00:06:05 +02:00
Toni 41765efcf8 Detect invalid characters in text and set a risk. Fixes #1347. (#1363)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-26 21:34:01 +02:00
Luca Deri 5ccc61d1cb Fixed Git protocol dissection (#1355) 2021-10-25 18:52:27 +02:00
Ivan Nardi b6d9536533 Fixed cleartext protocol assignment (#1357) 2021-10-25 15:04:04 +02:00
Ivan Nardi 333a6d60e8 TLS: fix a heap-buffer-overflow (#1356)
Revert of c3d1c697
Error reproducible with the attached pcap and valgrind
2021-10-22 14:57:49 +02:00
Luca Deri c3d1c69722 Fixed heap-buffer-overflow in TLS dissector 2021-10-20 22:07:47 +02:00
Ivan Nardi 8db8c8ae34 Fix QUIC log and remove SoulSeek leftovers after b97dc6ba (#1351)
Update .gitignore file
Fix a function prototype
Close #1349
2021-10-19 21:42:55 +02:00
Ivan Nardi fea3991ad1 Fix some invalid memory reads (#1350)
`ndpi_detection_giveup()` (and any functions called by it) can't access
`ndpi_detection_module_struct->packet` anymore since 730c236.

Sync unit tests results

Close #1348
2021-10-19 12:07:47 +02:00
Luca Deri 06f13fd2de Added clertext alert with telnet 2021-10-19 08:44:29 +02:00
Toni ed51987e3a Fix broken fuzz_process_packet fuzzer by adding a call to ndpi_finalize_initialization(). (#1334)
* fixed several memory errors (heap-overflow, unitialized memory, etc)
 * ability to build fuzz_process_packet with a main()
   allowing to replay crash data generated with fuzz_process_packet
   by LLVMs libfuzzer
 * temporarily disable fuzzing if `tests/do.sh`
   executed with env FUZZY_TESTING_ENABLED=1

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-18 23:16:32 +02:00
Luca Deri 7d3c3b23f8 Implemented RDP over UDP dissection 2021-10-18 22:12:28 +02:00
Luca Deri 9e97d20c25 Refreshed results list 2021-10-16 12:03:16 +02:00
Luca Deri b08b6a6a28 Fixed cleartext protocol assignment 2021-10-16 12:02:04 +02:00
Luca Deri c2e7fbce94 Updated test results after latest commit 2021-10-16 11:57:32 +02:00
Luca Deri 1b3c41a752 Reworked flow risks asignment
Added esceptions for windows update and binary application transfer risk
2021-10-16 11:55:08 +02:00
Luca Deri b97dc6baa4 Removed outdated (and broken) soulseek dissector 2021-10-15 17:27:27 +02:00
Luca Deri e696f31487 Updated test results 2021-10-14 22:18:34 +02:00
Luca Deri b1a43cbcff Added missing call to ndpi_reconcile_protocols() with protocol guess 2021-10-14 22:08:20 +02:00
Luca Deri 6e649e0cb6 Fixed issue on DGA numeric IP detection 2021-10-13 19:21:27 +02:00
Luca Deri 09668496c3 Indemt 2021-10-13 19:18:05 +02:00
Zied Aouini a2e9730b23 Add issue templates. (#1340)
* Add issue template.
* Add config log instruction.
2021-10-13 18:39:10 +02:00
Luca Deri 937357e4bc Implemented ndpi_ses_fitting() and ndpi_des_fitting()
for comuting the best alpha/beta values for exponential smoothing
2021-10-12 13:08:58 +02:00
Alfredo Cardigliano d947437233 Return 0 on ndpi_init_serializer_ll success for consistency 2021-10-12 10:47:44 +00:00
Luca Deri d13bc0e01a Fixed DES initialisation 2021-10-12 12:38:40 +02:00
Ivan Nardi 550e6fe6fc QUIC: fix an integer overflow (#1337)
Long standing bug: credits to @lnslbrty for digging into it and to
@aouinizied for the CI improvements
2021-10-11 23:08:10 +02:00
Toni a8f938f7ac Removed README.protocols because: (#1333)
* Tor via TLS should be detectable via DGA as a risk
 * protocol limitations should be part of the official documentation in `doc/`

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-11 23:07:36 +02:00
Luca Deri 960e8ab22a Win fix 2021-10-11 20:59:28 +02:00
Zied Aouini ca827ae548 Fix FuzzCI. (#1338) 2021-10-11 20:17:12 +02:00
Zied Aouini 7e0724d576 Enable sanitizers on CIFuzz. (#1336) 2021-10-11 19:01:50 +02:00
Luca Deri 246d115d7b Cleaned up code moving specific includes in files their are using it. Thi prevents ndpi_config.h to be included everywhere in apps using nDPI that might leade to #define redefinitions after the latest changes 2021-10-11 11:45:45 +02:00
Toni 60eca79fdf Additional fix related to cf931fda6b. (#1332)
* configure.seed references removed

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-10 14:14:02 +02:00
Toni 549134ef10 Fix make dist and add it to the CI along with make -C doc text. Fixes #1324 (#1327)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-10 13:15:38 +02:00
Toni cf931fda6b Get rid of configure.seed as it comes with some disadvantages. (#1328)
* using Autotools best-practices to achieve (hopefully) the same result

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-10 13:13:52 +02:00
Toni 6ff67b5043 Add more WindowsUpdate URLs as requested in #698. (#1329)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-08 18:29:49 +02:00
Luca Deri 3ffb65a81c Added extra check 2021-10-07 08:21:40 +02:00
Ivan Nardi 2dfc478ad4 Fix compilation with clang-13 or if some debug macros are enabled (#1326) 2021-10-06 18:32:15 +02:00
Luca Deri 944e8ed105 Added check 2021-10-06 18:28:09 +02:00
Luca Deri 2b4821a97b Compilation fix for old Linux distributions 2021-10-06 09:28:04 +02:00
Luca Deri d9c72999ee Test update 2021-10-06 00:06:34 +02:00
Luca Deri 181a03c5ad Compilation fix 2021-10-05 17:17:12 +02:00
Luca Deri 8957c52a48 Numeric IPs are not considered for DGA checks 2021-10-05 17:14:31 +02:00
Luca Deri 408d78e628 Improved DGA detection for skipping potential DGAs of known/popular domain names 2021-10-05 16:51:24 +02:00
Luca Deri 10576eab0a TLS obsolete protocol is set when TLS < 1.2 (used to be 1.1) 2021-10-05 16:44:56 +02:00
Ivan Nardi 730c2360bd Remove struct ndpi_packet_struct from struct ndpi_flow_struct (#1319)
There are no real reasons to embed `struct ndpi_packet_struct` (i.e. "packet")
in `struct ndpi_flow_struct` (i.e. "flow"). In other words, we can avoid
saving dissection information of "current packet" into the "flow" state,
i.e. in the flow management table.

The nDPI detection module processes only one packet at the time, so it is
safe to save packet dissection information in `struct ndpi_detection_module_struct`,
reusing always the same "packet" instance and saving a huge amount of memory.
Bottom line: we need only one copy of "packet" (for detection module),
not one for each "flow".

It is not clear how/why "packet" ended up in "flow" in the first place.
It has been there since the beginning of the GIT history, but in the original
OpenDPI code `struct ipoque_packet_struct` was embedded in
`struct ipoque_detection_module_struct`, i.e. there was the same exact
situation this commit wants to achieve.

Most of the changes in this PR are some boilerplate to update something
like "flow->packet" into something like "module->packet" throughout the code.
Some attention has been paid to update `ndpi_init_packet()` since we need
to reset some "packet" fields before starting to process another packet.

There has been one important change, though, in ndpi_detection_giveup().
Nothing changed for the applications/users, but this function can't access
"packet" anymore.
The reason is that this function can be called "asynchronously" with respect
to the data processing, i.e in context where there is no valid notion of
"current packet"; for example ndpiReader calls it after having processed all
the traffic, iterating the entire session table.

Mining LRU stuff seems a bit odd (even before this patch): probably we need
to rethink it, as a follow-up.
2021-10-05 15:49:52 +02:00
Ivan Nardi f3fcf1e7c0 Update unit tests results after da8eed5a (#1323) 2021-10-05 15:06:34 +02:00
Luca Deri da8eed5a32 Updated description 2021-10-05 11:08:16 +02:00
Ivan Nardi c1e794366f WHOIS: enhance detection, avoiding false positives (#1320)
We are interested only in the domain name required, not in the long reply.
2021-10-05 09:35:04 +02:00
Luca Deri bb7aff6526 Added -a <num> to ndpiReader for generating OPNsense configuration
See https://github.com/ntop/opnsense
2021-10-04 22:34:49 +02:00
Luca Deri fd0e65cb57 Removed trace 2021-10-03 23:41:31 +02:00
Ivan Nardi bc5be1e76c Fix how some protocols handle tcp retransmissions (#1321)
Most (all?) protocols don't care about (tcp) retransmissions.

If a protocol registers itself with a
NDPI_SELECTION_BITMASK_PROTOCOL_*_WITHOUT_RETRANSMISSION value, its
callback is never triggered with a retransmitted packet.
2021-10-03 21:38:49 +02:00
Toni e84da386c7 Added nDPI pkg-config file to Debian / Ubuntu ndpi-dev packaging. (#1318)
* Added nDPI pkg-config file to Debian / Ubuntu ndpi-dev packaging.

 * fixed missing gcrypt library dependency in libndpi.pc

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-03 21:29:56 +02:00
Simone Mainardi 06f763e6c3 Adds sections labels with risk id to the docs 2021-10-01 15:02:14 +02:00
Luca Deri a9d4f5d8d0 Warning fix 2021-10-01 11:36:35 +02:00
Luca Deri 8e6de00ce3 Initial attempt to write nDPI documentation. Starting with flow risks. Please contribute 2021-10-01 11:32:27 +02:00
Ivan Nardi 5cf3fef8f7 Remove detected_protocol_stack field from ndpi_packet_struct (#1317)
This field is an exact copy of `ndpi_flow_struct->detected_protocol_stack[2]`:
* at the very beginning of packet dissection, the value saved in
`flow->detected_protocol_stack` is copied in `packet->detected_protocol_stack`
(via `ndpi_detection_process_packet()` -> `ndpi_init_packet_header()`)

* every time we update `flow->detected_protocol_stack` we update
`packet->detected_protocol_stack` too (via `ndpi_int_change_protocol()`
-> `ndpi_int_change_packet_protocol()`)

These two fields are always in sync: keeping the same value in two
different places is useless.
2021-09-29 13:11:32 +02:00
Alfredo Cardigliano 721031210b Fix warning 2021-09-28 09:44:31 +02:00
Luca Deri af35d6b963 Added unit test for bitmap iteration 2021-09-27 12:47:05 +02:00
Luca Deri b9ccff0761 Compilation fixed on CentOS 7
Bitmap APi changes
2021-09-27 12:41:40 +02:00
Luca Deri 241ee32cbc Bitmap code 2021-09-27 12:06:46 +02:00
Luca Deri 519c69c5cc Reworked bitmap code 2021-09-27 10:29:03 +02:00
Luca Deri 72df138a7d Warnign fix 2021-09-27 09:08:04 +02:00
eac7024489 Fix armhf (#1315)
* Fix unaligned memory accesses with get_u_int64_t at armhf

see: https://bugs.debian.org/993627

* Use get_u_int64_t to avoid unaligned memory access at armhf

see: https://bugs.debian.org/993627

* Update src/include/ndpi_define.h.in

Drop const type from get_u_int64_t, from lnslbrty

Co-authored-by: Bernhard Übelacker <bernhardu@mailbox.org>
Co-authored-by: Toni <matzeton@googlemail.com>
2021-09-26 23:53:00 +02:00
Luca Deri 5ed1930a93 Added include for defining bools (not present on all platforms) 2021-09-26 23:00:13 +02:00
Luca Deri 1efabef4cf Added API for handling compressed bitmaps
ndpi_bitmap* ndpi_bitmap_alloc();
void ndpi_bitmap_free(ndpi_bitmap* b);
u_int64_t ndpi_bitmap_cardinality(ndpi_bitmap* b);
void ndpi_bitmap_set(ndpi_bitmap* b, u_int32_t value);
void ndpi_bitmap_unset(ndpi_bitmap* b, u_int32_t value);
bool ndpi_bitmap_isset(ndpi_bitmap* b, u_int32_t value);
void ndpi_bitmap_clear(ndpi_bitmap* b);
size_t ndpi_bitmap_serialize(ndpi_bitmap* b, char **buf);
ndpi_bitmap* ndpi_bitmap_deserialize(char *buf);

based on https://github.com/RoaringBitmap/CRoaring
2021-09-26 22:55:15 +02:00
Zied Aouini c40cf4e0ec Add arm64, armhf and s390x as part of CI. (#1314)
* Add arm64, armhf and s390x as part on CI.

* Minor fix.

* Fix code inspector complaints.
2021-09-23 14:41:56 +02:00
Ivan Nardi 7eb73bc92a QUIC: fix old GQUIC versions on big-endian machines (#1313) 2021-09-23 13:24:49 +02:00
Zied Aouini 0994771974 Improve CI (#1303)
* Improve CI pipeline

* Fix branch name.

* Fix branch name.

* Fix libgcrypt configuration.

* Update build.yml

* Move to Github Actions instead of Travis CI.

* Fix mingw on ubuntu bionic.

* Reactivate cross compile on Ubuntu Bionic.

* Switch to single line steps.

* Add several compilers versions

* Minor fix.

* Fix build all and delete cxx

* Fix RCE detection.

* Fix PCRE configuration.

* Add condition on PCRE test pcap.

* Update WebattackRCE.pcap.out

* Add missing SUBST.

* Delete WebattackRCE.pcap.out

* Update WebAttackRCE result.

* Fix typo.

* Extend jobs with pcre+msan+maxminddb.

* Fix code inpector warnings.

* Delete .appveyor.yml
2021-09-22 12:10:52 +02:00
Toni 9717afb2da Merge pull request #1311 from IvanNardi/ftp-start-tls
FTP: fix support for START-TLS sessions
2021-09-22 12:08:04 +02:00
Nardi Ivan 017c1a4239 FTP: fix support for START-TLS sessions
When TLS-over-FTP is used, the credentials are encrypted. So we must not
wait for the username and the password commands, otherwise we elaborate a
lot of packets for nothing.
2021-09-21 20:53:04 +02:00
Toni bb52f1362c Merge pull request #1309 from IvanNardi/ipv6-headers
Fix parsing of ipv6 packets with extension headers
2021-09-20 12:54:01 +02:00
Toni d74c2739d5 Merge pull request #1310 from IvanNardi/stun-realm
STUN: fix extraction of Realm attribute
2021-09-20 11:53:40 +02:00
Nardi Ivan 95ac8fd41a STUN: fix extraction of Realm attribute
While at it, improve detection of Facebook Messenger
2021-09-20 11:04:54 +02:00
Nardi Ivan 03d3e1bafc Fix parsing of ipv6 packets with extension headers
Decoding of ipv6 traffic with extension headers was completely broken,
since the beginning of the L4 header was always set to a wrong value.

Handle the ipv6 fragments in the same way as the ipv4 ones: keep the first
one and drop the others.
2021-09-19 17:29:22 +02:00
Ivan Nardi 994bd0696b Small fixes after latest commits (#1308) 2021-09-18 09:31:00 +02:00
Ivan Nardi 227e586e63 A (tiny) effort to reduce memory usage per flow (#1304)
Follow-up of 22241a1d
Only trivial changes:
* remove completely unused fields
* remove fields only written (but never read)
* CSGO protocol only handles UDP traffic
2021-09-18 09:29:45 +02:00
Luca Deri 4539e96ce2 Added DNS fragmented test pcap 2021-09-17 15:56:18 +02:00
Luca Deri 3303dac7cf Improved fragmented DNS detection 2021-09-17 15:17:39 +02:00
Luca Deri 119a14c6e3 DNS dissection fixes 2021-09-17 12:20:19 +02:00
deboracerretiniandLuca Deri 8f113c1d37 Progetto esame Gestione di Reti - Debora Cerretini (#1290)
* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2021-09-17 11:01:50 +02:00
Ivan Nardi 6325aebda6 TLS: avoid zeroing large structures (#1300)
Zeroing large structures (i.e. size > KB) is quite costly (from a CPU point
of view): we can safely avoid doing that for a couple of big structures.

Standard and Valgrind tests have been diverging quite a lot: it is time
to re-sync them. Use the same script and enable Valgrind via an
enviroment variable:

  NDPI_TESTS_VALGRIND=1 ./tests/do.sh
2021-09-16 14:34:59 +02:00
Simone Mainardi 978c9cfda3 Adds instructions to update flow risks in ntopng 2021-09-15 17:49:47 +02:00
Ivan Nardi 0eb7a0388c Hangout: fix usage of STUN function (#1299) 2021-09-13 18:25:43 +02:00
Ivan Nardi 3e06e6f5b8 Fix some ndpi_malloc/ndpi_free pairs (#1298) 2021-09-13 18:25:22 +02:00
Ivan Nardi 31404f4865 Update the list of default ports for QUIC protocol (#1297)
There are no reasons to register UDP/80 as a default port for QUIC
2021-09-13 18:25:04 +02:00
Ivan Nardi 17774bf09c Bittorrent: fix a potential use-of-uninitialized-value error (#1296)
Not sure if this is the right fix (from a logical point-of-view): this
code hasn't changed since OpenDPI era (!) and I have't found a trace
triggering this code path.
Anyway, the use-of-uninitialized-value error itself should be fixed.
2021-09-13 18:24:42 +02:00
Vitaly Lavrov 22241a1d36 Removing unused fields from the id_struct structure. (#1293)
Reduced by 40 bytes.
2021-09-13 18:19:09 +02:00
Ivan Nardi 951bcb0c39 Cassandra: fix compilation when "--enable-debug-messages" option is used (#1294)
Let's try adding a dedicated compilation in GitHub Actions to easily
detect this kind of errors in the future
2021-09-11 11:10:46 +02:00
Ivan Nardi 083e464929 Fix unit test results after 00857abf (#1295) 2021-09-11 11:09:55 +02:00
Luca Deri 00857abf2c Added new risk for clear text credentials 2021-09-10 22:00:04 +02:00
lucasbaileandLucas Santos 1fadf4754a Add Cassandra protocol dissector (#1285)
Co-authored-by: Lucas Santos <lucas.santos@zerum.com>
2021-09-09 22:47:58 +02:00
Luca Deri 36f08b8d2e Win fix 2021-09-07 22:37:22 +02:00
Alfredo Cardigliano 57fce4e350 Fix unused var 2021-09-03 17:46:04 +02:00
Alfredo Cardigliano 89af9b7e3c Print API version 2021-09-01 09:48:37 +02:00
Luca Deri a6b10cf73f Fixed memory stats 2021-08-26 12:12:22 +02:00
Luca Deri 90c6fd99a7 ARM fixes 2021-08-25 09:46:29 +02:00
Luca Deri 88040f0449 Compilation fix 2021-08-20 22:47:46 +02:00
Ivan Nardi 9e0f0ce3df Fix access to some TLS fields in flow structure (#1277)
Fields 'tls.hello_processed` and `tls.subprotocol_detected` are used by
QUIC (i.e UDP...), too.
2021-08-20 18:11:37 +02:00
Ivan Nardi 8fdffbf3a1 Compile everything with "-W -Wall -Wno-unused-parameter" flags (#1276)
Fix all the warnings.

Getting rid of "-Wno-unused-parameter" is quite complex because some
parameters usage depends on compilation variable (i.e.
`--enable-debug-messages`).

The "-Werror" flag has been added only in Travis builds to avoid
breaking the builds to users using uncommon/untested
OS/compiler/enviroment.

Tested on:
* x86_64; Ubuntu 20.04; gcc 7,8,9,10,11; clang 7,8,9,10,11,12
* x86_64; CentOS 7.7; gcc 4.8.5 (with "--disable-gcrypt" flag)
* Raspberry 4; Debian 10.10; gcc 8.3.0
2021-08-20 18:11:13 +02:00
Ivan Nardi 55eec29c08 wireshark/lua: restore full flow risks dissection (#1275)
It was partially disabled in 3eba8cc5.
Wireshark and Wireshark-Lua bindings don't handle 64 bit integer very well.
(see https://www.wireshark.org/docs/wsdg_html_chunked/lua_module_Int64.html).

As workaround, only for visualization purpose, split the (64 bit) risk mask
into two 32 bit integer values.
2021-08-18 11:34:46 +02:00
Toni 8d0c7b1fae Fixed Mingw64 build, SonerCloud-CI and more. (#1273)
* Added ARM build and unit test run for SonarCloud-CI.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Fixed Mingw64 build.

 * adapted to SonarCloud-CI workflow
 * removed broken and incomplete Windows example (tested on VS2017/VS2019)
 * removed unnecessary include (e.g. pthread.h for the library which does not make use of it)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-08-18 11:34:16 +02:00
Luca Deri 677b513f70 Fixes a crash on ARM (Raspberry Pi 4 Model B Rev 1.1) 2021-08-18 11:12:06 +02:00
Luca Deri 821f4c9249 Fixed some invalid TLS guesses 2021-08-17 19:04:07 +02:00
Luca Deri a2ea67152c Added check to avoid clases on similar protocols (FTP adn SMTP) on setting hostname 2021-08-11 12:11:58 +02:00
Luca Deri 5c33fbf19b Added extraction of hostname in SMTP
Fixed mail incalid subprotocol calculation
2021-08-11 11:52:24 +02:00
Alfredo Cardigliano 846b546dbc Fix typo 2021-08-10 11:43:32 +02:00
Luca Deri c1acb6d801 Typo 2021-08-09 18:36:36 +02:00
Luca Deri 0fbe895103 Addef ETH_ARP definition 2021-08-09 18:33:33 +02:00
Luca Deri 05492ec8e8 Defined more standard ethernet protocol types 2021-08-09 17:01:10 +02:00
Luca Deri 58ca7b5a7f Added protocol defines 2021-08-09 16:45:03 +02:00
Luca Deri 0df8aa19ec Clode cleanup (after last merge) 2021-08-08 17:10:08 +02:00
ToniandLuca Deri 259b60f574 Added entropy calculation to check for suspicious (encrypted) payload. (#1270)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2021-08-08 17:03:23 +02:00
Toni 4e856a41d6 Skip whitespaces between HTTP method and URL. (#1271)
* be less case-restrictive, RFC2616 wants it that way

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-08-08 17:00:10 +02:00
Toni 8e996f7f7c Fixed broken SonarCloud-CI. (#1272)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-08-08 16:59:36 +02:00
Luca Deri 3706267d68 Updated test output 2021-08-07 19:43:55 +02:00
Luca Deri 0522e562e8 Added testing pcap for TLS fatal alert 2021-08-07 19:42:53 +02:00
Luca Deri 4183718952 Added TLS fatal alert flow risk 2021-08-07 19:40:44 +02:00
Luca Deri e8455236bd Updated output 2021-08-07 17:38:33 +02:00
Luca Deri a13f1fe52f Report whether a protocol is encrypted 2021-08-07 17:35:34 +02:00
Luca Deri 46ebd7128f Added ability to report whether a protocol is encrypted
Added new API calls
- u_int8_t ndpi_is_valid_protoId(u_int16_t protoId);
Cleaned up code and organized in a function.

- u_int8_t ndpi_is_encrypted_proto(struct ndpi_detection_module_struct *ndpi_str, ndpi_protocol proto);
Check whether a protocol is encrypted or not based on master and app protocols
2021-08-07 17:10:18 +02:00
Luca Deri dc841574af Win fix 2021-08-03 09:44:50 +02:00
Luca Deri 6b7e5fa8d2 Added legacy whatsapp rules used to match old pcaps that need to be removed in the future 2021-07-31 23:49:48 +02:00
Toni ad57af9f79 Improved RTSP detection and fixed HTTP false-positive. Fixes #1229. (#1266)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-31 23:31:49 +02:00
Fábio Depin ce597b4806 Updated WhatsAPP and Instagram addresses (#1264)
* Updated WhatsAPP addresses

* Added Instagram addresses
2021-07-31 23:30:34 +02:00
Toni 13c5d6801e Improved libm check via Autoconf. (#1263)
* libm will now be part of @ADDITIONAL_LIBS@ (if required)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-28 22:24:03 +02:00
Luca Deri cb26a8c82d Moved to 4.1.0 2021-07-26 09:40:50 +02:00
Luca Deri 4cafa7cb1e Improved risk detection mask algorithm 2021-07-26 09:38:53 +02:00
Toni 29ec34f66d Improved TFTP detection. Fixes #1242, #1256 (#1262)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-25 21:54:19 +02:00
Luca Deri a482e1c0be Typo (Thanks Ivan) 2021-07-24 18:12:09 +02:00
Luca Deri 1d9a830b2f Update changelog 2021-07-24 17:54:32 +02:00
Luca Deri 51e4888442 Implemented ALPN automa for checking uncommon ALPNs 2021-07-24 17:50:32 +02:00
Luca Deri 526568fcd5 Risk check improvement 2021-07-24 16:18:18 +02:00
Alfredo Cardigliano c73501f859 Typo 2021-07-23 18:14:11 +02:00
Alfredo Cardigliano d5a60880d0 Fix prototype 2021-07-23 18:12:54 +02:00
Luca Deri 61fc5be202 Reworked flow risk implementation 2021-07-23 17:27:15 +02:00
Ivan Nardi 4ffe1eb3c0 Fix unit tests when "--disable-gcrypt" flag is used (#1255) 2021-07-23 10:37:36 +02:00
Toni 6ad0d6666c Implemented function to retrieve flow information. #1253 (#1254)
* fixed [h]euristic typo

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-23 10:37:20 +02:00
Luca Deri 8ea8ba8e9b Fixed risk mask implementation 2021-07-23 08:29:36 +02:00
Luca Deri b01b60a2b5 Implementation of flow risk eception (work in progress) 2021-07-22 01:35:57 +02:00
Alfredo Cardigliano fc9901292d Update changelog 2021-07-20 17:04:35 +00:00
Toni 32275543c4 Added risk: TLS_EXTENSION_SUSPICIOUS (#1252)
* validates client/server hello TLS extensions
 * inspects content for some extensions

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-19 16:23:24 +02:00
Ivan Nardi 57b8969a3d Fix setting of flow risks on 32 bit machines (#1251)
Since 19a29e1e (NDPI_TLS_CERT_VALIDITY_TOO_LONG is 32), unit tests are
failing on 32 bit machines (i.e Raspberry 4)
2021-07-19 16:22:39 +02:00
Luca Deri 0adcd28450 Updated download category name 2021-07-16 14:04:47 +02:00
Vitaly Lavrov 295cb3ee6a Bug fix. (#1250)
Restore part of the fields flow->packet after cleaning flow structure.
2021-07-16 13:53:36 +02:00
pacantandpacant 488166833e Fixed TLS certificate threshold (#1248)
Co-authored-by: pacant <a.pace97@outlook.com>
2021-07-16 13:52:55 +02:00
Vitaly Lavrov daf1fbfc7b Code cleanup. (#1246)
ndpi_utils.c: use ndpi_malloc,ndpi_calloc,ndpi_free
genshin_impact.c, git.c, hpvirtgrp.c, http.c, z3950.c: removed "#include stdlib.h"
2021-07-16 13:51:54 +02:00
Luca Deri a6de94a930 hulu and netflix dissection improvements 2021-07-14 12:53:03 +02:00
Luca Deri 3eba8cc5b5 Temporary fix for avoiding wireshark errors 2021-07-14 12:51:42 +02:00
Luca Deri b95bd0358f Code cleanup 2021-07-14 11:19:24 +02:00
pacantandpacant 19a29e1e22 TLS Risks - Certificate Validity Too Long (#1239)
* Added flow risk: TLS certificate too long

* Added flow risk: TLS certificate too long

* Date for TLS limit added

* TLS certificate check fixed

Co-authored-by: pacant <a.pace97@outlook.com>
2021-07-14 11:13:22 +02:00
Toni c411df523e Added AVAST SecureDNS protocol. (#1244)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-14 11:11:59 +02:00
Toni 62bae30a4a Improved Steam detection (Steam Datagram Relay - SDR). (#1243)
* improved DNSCrypt midstream detection again (sufficient for all tested use-cases)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-14 11:09:58 +02:00
Toni e4453938d5 Improved dnscrypt midstream detection. (#1241)
* fixed skype false-positive detection of dnscrypt traffic

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-13 15:10:18 +02:00
Ivan Nardi cccf794265 ndpiReader: add statistics about nDPI performance (#1240)
The goal is to have a (roughly) idea about how many packets nDPI needs
to properly classify a flow.

Log this information (and guessed flows number too) during unit tests,
to keep track of improvements/regressions across commits.
2021-07-13 12:28:39 +02:00
Luca Deri 96b71def49 Minor change 2021-07-12 17:51:25 +02:00
Vitaly Lavrov c418b7110b ahoсorasick. Code review. Part 2. (#1236)
Simplified the process of adding lines to AC_AUTOMATA_t.
Use the ndpi_string_to_automa() function to add patterns with domain names.
For other cases can use ndpi_add_string_value_to_automa().

ac_automata_feature(ac_automa, AC_FEATURE_LC) allows adding
and compare data in a case insensitive manner. For mandatory pattern comparison
from the end of the line, the "ac_pattern.rep.at_end=1" flag is used.
This eliminated unnecessary conversions to lowercase and adding "$" for
end-of-line matching in domain name patterns.

ac_match_handler() has been renamed ac_domain_match_handler() and has been greatly simplified.
ac_domain_match_handler() looks for the template with the highest domain level.
For special cases it is possible to manually specify the domain level.
Added test for checking ambiguous domain names like:
 - short.weixin.qq.com is QQ, not Wechat
 - instagram.faae1-1.fna.fbcdn.net is Instagram, not Facebook

If you specify a NULL handler when creating the AC_AUTOMATA_t structure,
then a pattern with the maximum length that satisfies the search conditions will be found
(exact match, from the beginning of the string, from the end of the string, or a substring).

Added debugging for ac_automata_search.
To do this, you need to enable debugging globally using ac_automata_enable_debug(1) and
enable debugging in the AC_AUTOMATA_t structure using ac_automata_name("name", AC_FEATURE_DEBUG).
The search will display "name" and a list of matching patterns.
Running "AHO_DEBUG=1 ndpiReader ..." will show the lines that were searched for templates
and which templates were found.

The ac_automata_dump() prototype has been changed. Now it outputs data to a file.
If it is specified as NULL, then the output will be directed to stdout.
If you need to get data as a string, then use open_memstream().

Added the ability to run individual tests via the do.sh script
2021-07-12 17:39:43 +02:00
Luca Deri 78b1295dc1 Fixed false positives in Z39.50 2021-07-09 00:16:48 +02:00
Luca Deri bec8361638 Updated category labels 2021-07-08 20:05:50 +02:00
Vitaly Lavrov 398ad8e5ff Fix for #1230 (#1235)
* Revert "Fix return value of ndpi_match_string_subprotocol() (#1230)"

This reverts commit 58665e93a9.

* Checking the return code after calling ndpi_match_string_subprotocol()

ndpi_api.h: Description of the returned error codes for
the ndpi_match_string_subprotocol() function.

If the ndpi_match_string_subprotocol() function returned an error,
then return NDPI_PROTOCOL_UNKNOWN.

http: The "Content-type" header is only checked if it is not empty.
2021-07-07 14:32:57 +02:00
Toni 34d0718bca Improved RTSP detection the second. (#1232) (#1233)
* RTSP is no subprotocol of HTTP (most of the time)
 * detection patterns should stay in rtsp.c
 * set detected HTTP protocol only if at least a valid HTTP method detected

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-07 09:55:36 +02:00
Luca Deri c408710c52 Added test pcap 2021-07-06 18:57:52 +02:00
Toni 6f29733262 Improved RTSP via HTTP detection. (#1232)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-06 18:57:04 +02:00
Ivan Nardi 58665e93a9 Fix return value of ndpi_match_string_subprotocol() (#1230) 2021-07-06 18:55:14 +02:00
Luca Deri 1ee7355b91 Updated prototype 2021-07-05 22:35:11 +02:00
Luca Deri e5dacc28e7 Minor fix 2021-07-05 22:02:46 +02:00
Luca Deri 48db44a567 Code cleanup 2021-07-05 21:45:30 +02:00
Alfredo Cardigliano 8e7b063804 Fix possible segfault/leak 2021-07-05 15:09:48 +02:00
Toni 6f7acfcab5 Added protocols to HTTP as subprotocols. (#1228)
* if HTTP was detected,
   all of those defined subprotocol detection routines should called

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-05 09:43:15 +02:00
Ivan Nardi 46c32d2433 TLS: fix a memory error in JA3 code (#1227)
protocols/tls.c:1856:5: runtime error: index 256 out of bounds for type 'char [256]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:1856:5
2021-07-05 09:42:52 +02:00
Ivan Nardi 1116d0e4b1 Improve content match lists (#1226)
Add support for Likee app (https://likee.video/) and GitLab
Improve detection of Google domains ("gtv1.com" added in 6dd42d19f was
likely a typo)
Improve Snapchat and Tiktok detection
2021-07-05 09:42:11 +02:00
Toni 21b2cce715 Improved Z39.50 detection. (#1225)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-05 09:41:35 +02:00
Toni ced6fca184 Fixed off-by-one memory error for TLS-JA3. (#1222)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-06-29 15:33:12 +02:00
Toni 8222ac5bdc Removed ht_hash as it is not used anymore. (#1220)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-06-29 15:32:48 +02:00
Toni 1c2a0c36f1 Added Z39.50 protocol. (#1219)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-06-29 15:32:16 +02:00
vpiserchiaandvpiserchia e5fd86393e reconcile PPSTREAM protocol category with content_match declarations (#1218)
* reconcile PPSTREAM protocol category with content_match declarations

* remove duplicate declaration for PPSTREAM protocol

Co-authored-by: vpiserchia <vito.piserchia@dreamlab.net>
2021-06-28 14:58:42 +02:00
Ivan Nardi f8fe3ee520 QUIC: add basic support for fragmented Client Hello (#1216)
Only in-order and non overlapping fragments are handled
See #1195
2021-06-24 18:30:34 +02:00
Luca Deri 6a1fd9ad97 Added missing check to prevent crashes 2021-06-23 12:17:21 +02:00
Vitaly Lavrov 2234b97149 ndpiReader: memory leak (#1215)
Non-critical bugs.
If a file list is used, then all files except the last are not closed.
Opening the next file loses the memory allocated via pcap_open_offline() for the previous file.
If a bpf filter is used, then no memory is freed after pcap_compile.
2021-06-23 12:04:03 +02:00
Vitaly Lavrov 35fc6a6de5 Speed and memory size optimisation (#1214)
Removed bigram_automata, impossible_bigram_automata, trigram_automata.
The ahocorasick structure is replaced with a bitmap.
The bitmap size for ndpi_en_bigram is 176 bytes.
The bitmap size for ndpi_en_trigram is 2201 bytes.
On the test machine, the test execution time was reduced from 27.3 seconds to 24.7 (9%).
2021-06-23 12:03:52 +02:00
Luca Deri f19937c8c9 Addesses partial application matching (#1207) 2021-06-21 21:59:15 +02:00
Luca Deri 0cbb4b2ffa Restored 32 bit value in ndpi_match_string_value() 2021-06-21 17:06:22 +02:00
Luca Deri 613e21002a Upgraded exponential smoothing to 64 bit values 2021-06-18 22:56:56 +02:00
Luca Deri 0afc8ace3d Added nDPI Score report 2021-06-17 23:52:01 +02:00
martinscheuandtinu 0b9b6a683d fixed lua errors in non-iec104 packets (#1209)
* Update iec.lua

fixed lua errors in non iec 104 packets

* Update iec.lua

Co-authored-by: tinu <martin.scheu@switch.ch>
2021-06-17 15:35:31 +02:00
Luca Deri b0b3e1bc6c Added bitmap boundary checks 2021-06-17 10:49:11 +02:00
Luca Deri e89c06256e Extende nDPI API with ndpi_netbios_name_interpret() 2021-06-17 09:35:19 +02:00
Luca Deri 2eba77e3b0 NetBIOS decoding changes 2021-06-16 19:57:20 +02:00
Vitaly Lavrov be808c30f3 Code review. (#1205)
The common actions required to call the ac_automata_search() function
have been moved to the ndpi_match_string_common function. This made it
possible to simplify the ndpi_match_string, ndpi_match_string_protocol_id,
ndpi_match_string_value, ndpi_match_custom_category, ndpi_match_string_subprotocol,
ndpi_match_bigram, ndpi_match_trigram functions.

Using u_int16_t type for protocol identifiers when working with the
ahocorasick library (changes src/include/ndpi_api.h.in and src/include/ndpi_typedefs.h).

Reworked "finalization" of all AC_AUTOMATA_t structures.

Changing the order of fields in the ndpi_call_function_struct structure
reduces the size of the ndpi_detection_module_struct structure by 10 kB (for x86_64).
2021-06-15 11:28:09 +02:00
Luca Deri 09293fabd9 Win fix 2021-06-15 11:26:36 +02:00
Luca Deri 43ccaef2f5 IEC analysis wireshark plugin 2021-06-14 21:18:37 +02:00
Alfredo Cardigliano 4aefbe0c7a Call ac_automata_release with free_pattern = 1 (malloc'ed patterns expected in ndpi_add_string_to_automa) 2021-06-14 14:41:14 +02:00
Luca Deri 380286c069 Fixes https://github.com/ntop/ntopng/issues/5482 2021-06-11 22:21:03 +02:00
Luca Deri a79b8ee834 Reimplememnted SNMP dissector 2021-06-11 21:44:51 +02:00
Luca Deri dee6f90d6c Syntax error fixes 2021-06-11 21:01:27 +02:00
Luca Deri c9f582805b Fixed flow score label 2021-06-11 19:53:07 +02:00
Matteo Biscosi a01a2b7443 Changed output for the sharkfest lua scripts 2021-06-11 11:00:06 +02:00
Vitaly Lavrov 6704649a75 Bug in ndpi_lru_find_cache (#1204)
Key value check is missing.
2021-06-10 11:06:19 +02:00
Luca 1a8ae710cf Minor code cleanup 2021-06-09 08:58:18 +02:00
Toni 65e67a2f88 autoconf: Do not check for json-c if --with-only-libndpi is set. (#1202)
* json-c is used by a unit test
 * required to fix some libnDPI cross compilation issues

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-06-09 08:01:10 +02:00
Luca Deri 77710ed8b4 OpenWRT document fix 2021-06-08 21:39:28 +00:00
Luca Deri 177f9456bd Win changes 2021-06-08 18:15:46 +02:00
Luca Deri d3aa3df975 Updated scripts 2021-06-08 17:52:03 +02:00
Luca 41ec807d79 New testing pcap with syn scan attack 2021-06-08 10:39:41 +02:00
Luca 2f8d3ac0b0 Companion scripts written for the Sharkfest conference 2021-06-08 10:39:19 +02:00
Luca Deri 2af7b33de0 Added report on score cli/srv breakdown 2021-06-07 14:59:18 +02:00
Vitaly Lavrov 65678dbeea New version of the ahocorasick library (#1200)
The new version is about 25% faster with -O2 and 45% faster with -O3.
No recursion is used (smaller stack size required).
Uses less memory (by valgrind info)
 bigram:
  - original 1796 allocs,   247864 bytes allocated
  - new      1232 allocs,   158880 bytes allocated
 host_match:
  - original 18038 allocs, 3004576 bytes allocated
  - new       6861 allocs,  396624 bytes allocated

The function ac_automata_search() is thread safe.
Optional case-insensitive comparison.
Matching at the beginning and at the end of the string is supported.
One code file and one header file.
2021-06-07 14:19:40 +02:00
Vitaly Lavrov 2fcf641e87 Fixed uninitialized memory use. (#1198)
The pointer "header" must be initialized before first call pcap_next_ex().
2021-06-07 10:08:39 +02:00
Luca Deri b241dd5e6b Fixed expected output 2021-06-03 09:06:01 +02:00
Luca Deri 03ae090d39 Warnign fix 2021-06-03 09:05:53 +02:00
Luca Deri 8761ac1887 Renamed Skyp in Skype_Teams as the protocol is now shared across these apps 2021-06-02 21:47:01 +02:00
Ivan Nardi 654e84ff02 wireshark/lua: improve flow risk visualization (#1194)
Create a separated proto field entry for each possible flow risk.
This way, filtering will be more natural: you can use something like
"ndpi.flow_risk.desktop_file_sharing_session"
2021-06-02 21:28:50 +02:00
Luca Deri 06c6941f03 Removed xbox and playstation invalid port guesses 2021-06-02 00:24:41 +02:00
Ivan Nardi 9d427faafe ndpiReader: fix collecting of risks statistics (#1192) 2021-06-01 16:50:46 +02:00
Ivan Nardi f4dbeecba9 TLS: improve check for common ALPNs (#1191)
Facebook is still using its own ALPN for HTTP2 as well
2021-06-01 16:50:28 +02:00
Luca c620858671 Reworked ndpi flow risk score adding client and server score 2021-06-01 09:17:26 +02:00
Alfredo Cardigliano c4084ca3c7 Fix warnings 2021-05-25 15:17:48 +02:00
Luca eb0115d4e0 Added TLS check to avoid crash with UDP-based traffic 2021-05-25 09:03:24 +02:00
Luca Deri d485c1cecb Improved TLS browser detection heuristics 2021-05-19 21:28:33 +02:00
Ivan Nardi 71b81d3799 wireshark/lua: fix offsets (#1187) 2021-05-18 21:36:46 +02:00
Luca Deri 732bcecd17 Added flow risk score 2021-05-18 21:05:47 +02:00
Luca Deri 86f3c29d03 Typo 2021-05-18 19:52:33 +02:00
Luca Deri ca15e3295e Added risk/score dump (ndpiReader -h)
Added ndpi_dump_risks_score() API score
2021-05-18 19:34:17 +02:00
Luca Deri 0b3d627185 Updated API for ndpi_risk2score()
Added ndpi_get_upper_proto() API call
2021-05-17 22:39:07 +02:00
Luca Deri 43a8576efb Reworked human readeable string search in flows
Removed fragment manager code
2021-05-17 20:55:06 +02:00
Luca Deri 1ec621c85b Added further checks 2021-05-15 19:53:46 +02:00
Luca Deri 9c5ba9c060 Updated protocol category 2021-05-15 11:20:15 +02:00
Luca Deri 5646ffa725 Added ndpi_check_subprotocol_risk() API call definition 2021-05-15 11:16:27 +02:00
Luca Deri abd6bce6f9 Added TLS certifiacate caching
Added Fortigate protocol
2021-05-15 10:52:16 +02:00
Luca Deri ca92660d98 Converted some test .pcapng files to pcap format 2021-05-13 20:51:11 +02:00
Luca Deri ac1eaca8a6 Added browser TLS heuristic 2021-05-13 20:00:27 +02:00
Luca Deri a62be9b8ec Implemented heuristic to detect Safari and Firefox TLS browsing 2021-05-13 12:37:07 +02:00
Toni 87076dcd5b Fixed obsolete error printing if CTRL-C is pressed. #1165 (#1184)
* This fix was proposed by @robertsong2019

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-11 21:38:56 +02:00
Toni 5918a6542d Improved SSL certificate name wildcard handling and risk. #1182 (#1183)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-11 21:38:26 +02:00
Luca Deri a5ecdf9df8 Changes for sonarcloud integration 2021-05-11 21:36:25 +02:00
Luca Deri 5079bb1106 Fixed typo 2021-05-11 21:33:48 +02:00
Luca Deri 4297a65ce8 Implemented flow score in Wireshark integration 2021-05-10 22:43:05 +02:00
Luca Deri fff60ec317 Added check to reduce MongoDB false positive detection 2021-05-10 21:14:55 +02:00
Luca Deri 66ebe444ae Added (partial) Activision protocol support (based on tencent cloud) 2021-05-10 15:41:52 +02:00
Ivan Nardi 1deae1132b Fix some warnings (#1181)
```
In file included from protocols/fasttrack.c:29:
../include/ndpi_api.h:1504:3: warning: type qualifiers ignored on function return type [-Wignored-qualifiers]
 1504 |   const ndpi_risk_severity ndpi_risk2severity(ndpi_risk_enum risk);
      |   ^~~~~
In file included from protocols/amazon_video.c:28:
../include/ndpi_api.h:1504:3: warning: type qualifiers ignored on function return type [-Wignored-qualifiers]
 1504 |   const ndpi_risk_severity ndpi_risk2severity(ndpi_risk_enum risk);
      |   ^~~~~

...

ndpi_utils.c: In function ‘ndpi_risk2severity’:
ndpi_utils.c:1834:1: warning: control reaches end of non-void function [-Wreturn-type]
 1834 | }
      | ^
```
2021-05-09 15:11:05 +02:00
Ivan Nardi 2ca37ff8be TLS: fix extraction for TLS signature algorithms (#1180)
```
==69562==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6100009000fb at pc 0x7f41882003a7 bp 0x7f4183cfbfc0 sp 0x7f4183cfb768
READ of size 32 at 0x6100009000fb thread T1
    #0 0x7f41882003a6 in __interceptor_memcpy ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827
    #1 0x560b2d7462a1 in processClientServerHello protocols/tls.c:1647
    #2 0x560b2d73be6a in processTLSBlock protocols/tls.c:712
    #3 0x560b2d73e61f in ndpi_search_tls_udp protocols/tls.c:968
```
2021-05-09 15:10:41 +02:00
Ivan Nardi 12c6ab768d TLS: fix another use-of-uninitialized-value error in ClientHello parsing (#1179)
Error detected with valgrind.

==13127== Conditional jump or move depends on uninitialised value(s)
==13127==    at 0x483EF58: strlen (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==13127==    by 0x1A93B6: ndpi_strdup (ndpi_main.c:159)
==13127==    by 0x1C07CC: processClientServerHello (tls.c:1678)
==13127==    by 0x1C0C4C: processTLSBlock (tls.c:712)
==13127==    by 0x1C0C4C: ndpi_search_tls_tcp.part.0 (tls.c:849)

See also 8c3674e9
2021-05-09 15:10:14 +02:00
Toni 8c28613eb2 Check datalink during fuzzing to prevent console / logfile spam. See #1175 for more information. (#1177)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-09 15:09:43 +02:00
Bernhard M. Wiedemann 0663544be8 Allow to override build date with SOURCE_DATE_EPOCH (#1176)
in order to make builds reproducible.
See https://reproducible-builds.org/ for why this is good
and https://reproducible-builds.org/specs/source-date-epoch/
for the definition of this variable.
2021-05-09 15:09:15 +02:00
Luca ae2470fad4 Initial work towards detection via TLS of browser types 2021-05-06 21:42:06 +02:00
Luca 808050e635 Add extraction for TLS signature algorithms 2021-05-06 08:56:01 +02:00
Luca ce54a6ab57 Added ndpi_risk2severity() API call 2021-05-02 19:35:01 +02:00
Luca Deri 153d11c920 Enhanced netbios decoding check 2021-04-28 22:58:22 +02:00
Luca Deri dd65142020 Compilation fix 2021-04-27 08:26:08 +02:00
Luca Deri 70686249c9 Updated code due to https://github.com/ntop/nDPI/pull/1175 2021-04-27 08:12:14 +02:00
Luca Deri 7b62db81c3 Removed DGA check for ipv6 reverse IPs 2021-04-27 07:23:11 +02:00
Toni da3e6bd61b Check for common ALPNs and set a flow risk if not known. (#1175)
* Increased risk bitmask to 64bit (instead of 32bit).
 * Removed annoying "Unknown datalink" error message for fuzzers.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-27 07:22:04 +02:00
Luca Deri d5b395f35a Fixed invalid DNS dissection 2021-04-26 23:08:00 +02:00
Luca Deri 87ad2b58dc Compilation fix 2021-04-26 10:50:19 +02:00
Luca Deri 4a09707e48 Added flow risk to wireshark dissection 2021-04-26 10:17:29 +02:00
Luca Deri a04efb7ce7 Added tshark description 2021-04-25 11:47:19 +02:00
Luca Deri fad246289f README for the tshark class 2021-04-25 11:45:35 +02:00
Luca Deri b586c223dd Lua tshark class and examples 2021-04-25 11:40:53 +02:00
Luca Deri 0ef7998348 Removed protocol space in Genshin Impact 2021-04-25 10:06:22 +02:00
Toni b4a8c8c63f Add Genshin Impact protocol. (#1173)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-25 10:02:07 +02:00
Alfredo Cardigliano d95cd9912c Add vxlan enum to ndpi_packet_tunnel 2021-04-21 12:43:12 +02:00
Toni 9377991263 Add HP Virtual Machine Group Management (hpvirtgrp) protocol. (#1170)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-20 14:12:16 +02:00
Luca Deri c6a835746e Reworked GeoIP API
int ndpi_get_geoip_asn(struct ndpi_detection_module_struct *ndpi_str,
                        char *ip, u_int32_t *asn);
int ndpi_get_geoip_country_continent(struct ndpi_detection_module_struct *ndpi_str, char *ip,
                        char *country_code, u_int8_t country_code_len,
                        char *continent, u_int8_t continent_len);
2021-04-19 21:39:55 +02:00
Ivan Nardi 8c3674e9a3 TLS: fix some use-of-uninitialized-value errors in ClientHello parsing (#1169)
Error detected with valgrind.

==125883== Conditional jump or move depends on uninitialised value(s)
==125883==    at 0x438F57: processClientServerHello (tls.c:1421)
==125883==    by 0x43B35A: processTLSBlock (tls.c:712)
==125883==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125883==    by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)
==125883==    by 0x42E920: ndpi_detection_process_packet (ndpi_main.c:5301)

==125916== Conditional jump or move depends on uninitialised value(s)
==125916==    at 0x438D7D: processClientServerHello (tls.c:1379)
==125916==    by 0x43B35A: processTLSBlock (tls.c:712)
==125916==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125916==    by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)

==125932== Conditional jump or move depends on uninitialised value(s)
==125932==    at 0x438C1D: processClientServerHello (tls.c:1298)
==125932==    by 0x43B35A: processTLSBlock (tls.c:712)
==125932==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125932==    by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)

==125950== Conditional jump or move depends on uninitialised value(s)
==125950==    at 0x438D4F: processClientServerHello (tls.c:1371)
==125950==    by 0x43B35A: processTLSBlock (tls.c:712)
==125950==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125950==    by 0x42C079: check_ndpi_detection_func (ndpi_main.c:4443)
2021-04-18 21:38:01 +02:00
Ivan Nardi 9ca62ed7ac Fix detunneling of GTP-U traffic (#1168)
Fuzzing #1161 exposed some (completely unrelated) issues on GTP-U
detunneling code.
(see https://github.com/ntop/nDPI/actions/runs/719882047)
2021-04-18 21:37:51 +02:00
Ivan Nardi d9278d55c3 Improve classification of Outlook/MicrosoftMail traffic (#1167)
See #1148
2021-04-18 21:37:26 +02:00
Luca Deri 9fcf98c067 GeoIP handlign fixes 2021-04-18 21:36:03 +02:00
Luca Deri 18c6c1c2d6 Added NDPI_DESKTOP_OR_FILE_SHARING_SESSION risk to remote protocols for remote assistance sessions 2021-04-12 18:11:14 +02:00
Luca Deri bf318e0b86 Added NDPI_DESKTOP_OR_FILE_SHARING_SESSION flow risk 2021-04-11 14:42:27 +02:00
Ivan Nardi ceb98a3f9c wireshark/lua: fix handling of VLAN traffic (#1162) 2021-04-05 19:23:56 +02:00
Ivan Nardi 0bfe444362 GTP: fix parsing of GTP headers (#1161)
Message length checks and basic headers are not uniform across GTP-U,
GTP-C and GTP-PRIME.

Note that, even if the length checks were wrong, the GTP sessions were almost
always correctly classified because of the "guessing" algorithm.

This patch has been tested with GTP-U, GTP-C-V1, GTP-C-V2 and GPT-PRIME-V2
traffic using ndpiReader with "-d" flag (to avoid "guessing" algorithm) and
without "-t" flag (to avoid GTP-U de-tunneling).

See #1148
2021-04-05 19:23:03 +02:00
Ivan Nardi fb74785282 Fix some warnings about unused variables/functions (#1160) 2021-04-05 19:21:30 +02:00
Luca Deri a1dba74346 Trace fix 2021-04-02 12:55:15 +02:00
Luca Deri 4f8ca9485a Fixed incapoatibilities with the latest extcap/wireshark 2021-04-01 23:53:53 +02:00
Luca Deri db5aeb0cb1 Readme update 2021-04-01 10:24:44 +02:00
Luca Deri 28879c570e Reworked ndpi patricia includes to avoid compilation issues on some platforms 2021-03-31 19:43:55 +02:00
Ivan Nardi 1290c40968 Mining: lru cache is ipv4 only (for the time being) (#1159)
Fix memory error with ipv6 traffic
2021-03-31 14:13:07 +02:00
Luca Deri 48726301e1 Fixed mispelled word 2021-03-31 12:52:54 +02:00
Luca Deri 2b84f28df5 Improved mining results 2021-03-30 17:54:44 +02:00
Luca Deri c1d6e3f145 Improved mining detection support 2021-03-30 17:50:19 +02:00
Luca Deri 637b2063ed Added missing tracker/Ads breed 2021-03-30 17:50:03 +02:00
Luca Deri fcbc16da00 Fixed invalid guess stats 2021-03-30 17:49:48 +02:00
Luca Deri ffbce931b9 Ignore TLD .local .lan and .home in DGA domain check 2021-03-26 15:53:04 +01:00
Ivan Nardi a6029d250d ndpiReader: print an error msg if we found an unsupported datalink type (#1157) 2021-03-23 11:47:29 +01:00
Toni b040407683 Refactored nDPI subprotocol handling and aimini protocol detection. (#1156)
* Refactored and merged callback buffer routines for non-udp-tcp / udp / tcp / tcp-wo-payload.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Try to detect one subprotocol if a detected protocol can have one.

 * This adds a performance overhead due to much more protocol detection routine calls.
   See #1148 for more information.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactor subprotocol handling (1/2).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactor subprotocol handling (2/2).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Prevent some code duplication by using macros for ndpi_int_one_line_struct string comparision.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactored aimini HTTP detection parts (somehow related to #1148).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Added aimini client/server test pcap.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Removed master protocol as it was only used for STUN and via also removed API function ndpi_get_protocol_id_master_proto

 * Adjusted Python code to conform to the changes made during the refactoring process.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-03-23 11:46:12 +01:00
Luca Deri 0cae9bf4a4 Win compilation fix 2021-03-22 21:41:28 +01:00
Luca Deri 627299e4dd Better DGA detection (slightly decreased accuracy) 2021-03-20 17:56:24 +01:00
Luca Deri 6333bb1702 Removed duplicate extesions len 2021-03-19 22:35:08 +01:00
Luca Deri d96f4ca736 Added ALPN and elliptic curve in JA3S+ 2021-03-19 22:33:04 +01:00
Luca Deri fa238bbe42 Implemented JA3+ also for JA3S 2021-03-19 22:27:36 +01:00
Luca Deri e2c3445a20 Reworked JA3 2021-03-19 22:20:52 +01:00
Luca Deri 61f7ec1696 JA3 debug improvements 2021-03-19 19:10:31 +01:00
Luca Deri 9eed0a60d3 Fixed support for custom DGA detection libraryr 2021-03-18 11:32:56 +01:00
Luca Deri b22cb70a68 Added % of flows with risks 2021-03-14 21:59:33 +01:00
Luca Deri 1b6275005e Added in stats the number of flows with risks 2021-03-14 21:54:35 +01:00
Luca Deri 3b02cfab69 Added flows risks report 2021-03-14 21:46:23 +01:00
Luca Deri 565a7bfce3 Reworked extendal dependency across testing tools 2021-03-14 20:48:21 +01:00
Luca Deri e58527536d Help crash fix 2021-03-14 19:55:21 +01:00
Luca Deri 9419015711 Implemented square erro rollup to avoid overflow 2021-03-14 11:01:51 +01:00
Zied Aouini 7f9a18fd05 Make libmaxminddb build configurable. (#1154) 2021-03-14 10:47:28 +01:00
Alfredo Cardigliano efc5630378 Fix compilation warning 2021-03-12 09:25:00 +01:00
Luca 192fad4402 Added double exponential smoothing implementation 2021-03-11 09:39:52 +01:00
Luca Deri 6833ee2bbe Added single exponential smoothing API
int ndpi_ses_init(struct ndpi_ses_struct *ses, double alpha, float significance);
int ndpi_ses_add_value(struct ndpi_ses_struct *ses, const u_int32_t _value, double *forecast, double *confidence_band);
2021-03-11 00:04:33 +01:00
Luca Deri 5b7fe1360a Fixed JA3+ computation 2021-03-11 00:04:12 +01:00
Luca Deri 5c6f070097 Fixed nDPI prefs definition 2021-03-09 23:50:31 +01:00
Luca Deri f6ad16d8f8 Added experiemntal JA3+ implementation that can be used with -z i ndpiReader 2021-03-09 23:38:29 +01:00
Ivan Nardi 8074bc8201 HTTP: fix memory access in ndpi_http_parse_subprotocol() (#1151) 2021-03-09 19:46:32 +01:00
Luca Deri 574fc4f09e Ookla detection improvement 2021-03-09 17:43:12 +01:00
Luca Deri 3032864ec9 Added Ookla detection over IPv6 2021-03-09 12:55:14 +01:00
Luca Deri db716d0ab0 Ookla fixes 2021-03-09 11:42:23 +01:00
Luca Deri 477e4db650 Improved detection of Ookla speedtest and openspeedtest.com 2021-03-09 11:38:31 +01:00
Luca Deri bb6423a79f Added the ability to define a custom DGA detection function by overwriting
the value of the function pointer ndpi_dga_function curently set to NULL
(that means the nDPI internal DGA function will be used)
2021-03-08 22:57:30 +01:00
Ivan Nardi c3490e80a7 Fix some stack-use-after-return errors in automa code (#1150) 2021-03-08 21:14:01 +01:00
Ivan Nardi c50a8d4808 Add support for Snapchat voip calls (#1147)
* Add support for Snapchat voip calls

Snapchat multiplexes some of its audio/video real time traffic with QUIC
sessions. The peculiarity of these sessions is that they are Q046 and
don't have any SNI.

* Fix tests with libgcrypt disabled
2021-03-06 05:48:36 +01:00
Alfredo Cardigliano 73e4348570 Add ndpi_serialize_binary_boolean for consistency. Fix comments. 2021-03-04 10:31:03 +01:00
Luca Deri 0f8a994841 Improved DGA detection
Before
Accuracy 66%, Precision 86%, Recall 38%

After
Accuracy 71%, Precision 89%, Recall 49%
2021-03-03 19:30:01 +01:00
Luca 49843509e5 Added verbose option 2021-03-03 08:12:16 +01:00
Luca Deri 1a37595de0 Removed check for knowns protocols (major and app protocols) 2021-03-03 00:57:56 +01:00
Luca Deri 56bfb439f8 Improved DGA detection with trigrams. Disadvantage: slower startup time
Reworked Tor dissector embedded in TLS (fixes #1141)
Removed false positive on HTTP User-Agent
2021-03-03 00:41:07 +01:00
Ivan Nardi 4c00ff89df DTLS: improve support (#1146)
* DTLS: add some pcap tests

* DTLS: fix parsing of Client/Server Helllo message

* DTLS: add parsing of server certificates
2021-03-02 21:15:40 +01:00
Ivan Nardi 2b71a329c1 QUIC: fix mvfst-27 test (#1145)
Regardless of its name, quic-mvfst-27 trace doesn't contain mvfst-27
traffic
2021-03-02 21:15:02 +01:00
Ivan Nardi c12a697f2d TOR: update node list (#1144)
https://panwdbl.appspot.com/lists/ettor.txt list is no more available

Close #1141
2021-03-02 21:14:36 +01:00
Luca Deri 2d558afb54 Added HW checks 2021-03-01 19:59:14 +01:00
Luca Deri 2724a14bf7 Added further HW checks 2021-03-01 19:34:20 +01:00
Luca Deri 4bff595733 Holt-Winters calculation improvement 2021-02-27 11:32:51 +01:00
Toni 16890a6632 Added NDPI_MALICIOUS_SHA1 flow risk. (#1142)
* An external file which contains known malicious SSL certificate SHA-1 hashes
   can be loaded via ndpi_load_malicious_sha1_file(...)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-26 17:00:05 +01:00
Luca Deri fba61adf5e Improved DNS dissector 2021-02-26 00:42:15 +01:00
pengtian 28330edb7a [Fix] replace free to ndpi_free (#1140)
same as https://github.com/ntop/nDPI/issues/1096
2021-02-25 08:46:55 +01:00
Toni 70c35addcb Added protocol breed to JSON serializer. (#1137)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-25 08:46:38 +01:00
Alfredo Cardigliano d898fddea0 Fix ndpi_fill_prefix_v6 2021-02-24 14:43:12 +00:00
Luca Deri 545f270132 Windows code cleanup 2021-02-24 11:04:04 +01:00
Luca Deri 4bd175b07e Modified JA3 fingerprint message 2021-02-24 10:42:26 +01:00
Alfredo Cardigliano 566d5db402 Add more utility functions to work with patricia trees 2021-02-23 11:53:15 +01:00
Alfredo Cardigliano 888ddfcd17 Update ndpi_patricia_walk_inorder API 2021-02-23 10:23:52 +01:00
Alfredo Cardigliano f8e83f7e35 Add support for MAC to Patricia tree. Expose full API to applications. Add utility functions. 2021-02-23 10:01:56 +01:00
Luca Deri b757cf606b Fixed mispelled API call 2021-02-22 23:59:08 +01:00
Luca Deri f1b22b199f Added NDPI_MALICIOUS_JA3 flow risk
Added ndpi_load_malicious_ja3_file() API call
2021-02-22 23:19:23 +01:00
Luca Deri fc3db8f169 Implemented TLS Certificate Sibject matching
Improved AnyDesk detection
2021-02-22 22:37:33 +01:00
Luca Deri a21ad2fbe4 Added new data for risky domains test 2021-02-21 22:08:01 +01:00
Luca Deri 5ad056e99b Removed unused NDPI_RISKY_COUNTRY 2021-02-21 21:45:46 +01:00
Luca Deri fc16c9368e Added risky domain flow-risk support 2021-02-21 21:45:46 +01:00
Luca Deri 6db20b3ba9 Updated dependency 2021-02-19 17:59:40 +01:00
Luca Deri 09559611cc Fixes #1136 2021-02-19 15:15:02 +01:00
Luca Deri dee24f7acf Removed old unused code 2021-02-19 15:12:03 +01:00
Alfredo Cardigliano e9350d8fc7 Fix leak 2021-02-19 09:27:24 +01:00
Luca Deri 92e8d95f38 Added ndpi_get_geoip() APi call 2021-02-18 23:15:27 +01:00
Luca Deri bdf95a5c22 Package fix 2021-02-18 22:08:15 +00:00
Luca Deri 393a844c9a GeoIP support 2021-02-18 22:40:57 +01:00
Luca Deri bce54079d8 Initial geoip support 2021-02-18 22:35:25 +01:00
Luca Deri a2c5adc374 Improved nDPI string matching algorithm 2021-02-18 21:47:09 +01:00
Luca Deri fb4cdecb68 Fixes due to the fragment mananegr code 2021-02-18 13:31:02 +01:00
Luca Deri 413dc287b4 Added paper citation 2021-02-17 08:22:38 +01:00
Luca Deri 4609e2084b Added new risks (future use)
-  NDPI_RISKY_ASN
-  NDPI_RISKY_DOMAIN
-  NDPI_RISKY_COUNTRY
2021-02-16 18:23:19 +01:00
Alfredo Cardigliano 177f1785da Fix warning 2021-02-12 09:11:14 +01:00
Luca Deri 813df77d07 Performance optimization to avoid un-necessary calls and thus increase the overall performance 2021-02-11 17:54:55 +01:00
Luca Deri d99d636696 Some optimizations during flow guess 2021-02-10 19:51:26 +01:00
Luca Deri fffff3760b Code refactory 2021-02-10 19:28:35 +01:00
Luca Deri ea0309d21b Removed now obsolete NDPI_DETECTION_SUPPORT_IPV6: code is more readeable now 2021-02-10 19:09:11 +01:00
Luca Deri 76ac551ed3 Improved FTP_CONTROL detection 2021-02-10 17:55:07 +01:00
Luca Deri 79102ef4f2 Added check for avoiding long dissections 2021-02-10 17:36:09 +01:00
Ivan Nardi 421609475e Fix small memory leak (#1133)
Now function definition matches the prototype in ndpi_api.h.in
2021-02-10 15:24:34 +01:00
Toni 1e12c90c66 Fixed memory leaks caused by conditional free'ing for some TLS connec… (#1132)
* Fixed memory leaks caused by conditional free'ing for some TLS connections.

 * Members of tls_quic struct should also free'd if the detected master protocol is IMAPS / POPS / SMTPS / etc.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Prevent reader_util.c from exit()'ing if maximum flow count reached.
This confuses the fuzzer.

 * Improved fuzz/Makefile.am to use LDADD for ../example/libndpiReader.a instead of LDFLAGS.
   That way, fuzz_ndpi_reader re-links to ../example/libndpiReader.a if something changed there.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-10 15:24:11 +01:00
Luca Deri c408df1b0e STUN improvements 2021-02-10 15:22:20 +01:00
Luca Deri e2f6569adb Fixed CPHA missing protocol initialization
Improved IEC104 and IRC detection
2021-02-10 15:22:20 +01:00
Luca Deri 0de3d4c37a Dissection inprovements 2021-02-09 21:48:46 +01:00
Luca Deri 512f73f381 Added checks for giving up faster on IRC and SMTP 2021-02-09 21:27:09 +01:00
Luca Deri 1a557d0918 IRC test files 2021-02-09 21:25:48 +01:00
Luca Deri 1331e0aec9 Extended the API to calculate jitter
- ndpi_jitter_init()
- ndpi_jitter_free()
- ndpi_jitter_add_value()
2021-02-09 15:56:03 +01:00
Luca Deri b960809d3e Minor code improvements 2021-02-09 12:55:11 +01:00
Luca Deri 4abaf3e279 Removed debug statement 2021-02-09 10:05:29 +01:00
Luca Deri 732579b72b Added timeseries forecasting support implementing Holt-Winters with confidence interval
New API calls added
- ndpi_hw_init()
- ndpi_hw_add_value()
- ndpi_hw_free()
2021-02-08 19:10:25 +01:00
Luca Deri d9da752aa8 Updated skype addresses 2021-02-07 19:05:56 +01:00
Luca Deri 3ecda45bc4 IP address matching update 2021-02-07 17:42:31 +01:00
Luca Deri 8297dacfe5 Added Makefile for building the development version of libndpi for OpenWRT 2021-02-07 09:29:01 +00:00
Luca Deri be58d78417 (Temporarely) Disabled fragment manager (#1129) 2021-02-05 23:03:08 +01:00
Luca Deri 45e9c3c438 Partial fix for #1129 2021-02-05 22:22:33 +01:00
Alfredo Cardigliano 95a6a2072b Code cleanup and safety checks in the fragment manager (#1129) 2021-02-05 17:16:11 +01:00
Luca Deri 8dd7716ae5 Implemented more efficient and memory savvy RSI 2021-02-05 12:38:41 +01:00
Luca Deri 60b58dbd67 RSI enhancements 2021-02-05 10:59:09 +01:00
Luca Deri 1eedf734be Implemented API for computing RSI (Relative Strenght Index)
void  ndpi_init_rsi(struct ndpi_rsi_struct *s, u_int16_t num_learning_values);
void  ndpi_free_rsi(struct ndpi_rsi_struct *s);
float ndpi_rsi_add_value(struct ndpi_rsi_struct *s, const u_int32_t value);
2021-02-04 23:52:33 +01:00
Luca Deri 54636a3213 Improved (partial) TLS dissection 2021-02-04 22:06:18 +01:00
Ivan Nardi 679c317c57 Fix utests when "--disable-gcrypt" flag is used (#1128)
Fix: d6684f4b
2021-02-04 19:29:48 +01:00
Ivan Nardi 2a65321884 Fix some memory leakes in reassembler code (#1127) 2021-02-04 19:29:27 +01:00
ToniandLuca Deri 7255dbe4b7 Added fuzzy targets conditional in tests/do.sh.in which prevents the fuzzer from running if nDPI was configured previously --enable-fuzztargets but not for the current config (may produce invalid results). (#1126)
* fixed possible NULL pointer dereference for memcpy(), src pointer should never be NULL

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2021-02-04 19:28:52 +01:00
Luca Deri c68c8c79c9 Added missing check 2021-02-04 18:17:45 +01:00
Luca Deri 6c366d73e2 Fixed leak with DTLS 2021-02-03 23:36:11 +01:00
Luca Deri 6289413fea Updated results 2021-02-03 11:58:17 +01:00
Ivan Nardi a772e18977 Fix a warning (#1125)
Introduced in 5f7b9d802

reader_util.c: In function ‘process_ndpi_collected_info’:
reader_util.c:1148:60: warning: ‘%s’ directive output may be truncated writing up to 255 bytes into a region of size 64 [-Wformat-truncation=]
 1148 |       sizeof(flow->ssh_tls.client_requested_server_name), "%s",
      |                                                            ^~
reader_util.c:1147:5: note: ‘snprintf’ output between 1 and 256 bytes into a destination of size 64
 1147 |     snprintf(flow->ssh_tls.client_requested_server_name,
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 1148 |       sizeof(flow->ssh_tls.client_requested_server_name), "%s",
      |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 1149 |       flow->ndpi_flow->protos.tls_quic_stun.tls_quic.client_requested_server_name);
      |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-02-03 11:56:37 +01:00
Luca Deri d6684f4b4c Added test pcap 2021-02-03 11:56:14 +01:00
Ivan Nardi 8c0ea694f8 HTTP: fix user-agent parsing (#1124)
User-agent information is used to try to detect the user OS; since the
UA is extracted for QUIC traffic too, the "detected_os" field must be
generic and not associated to HTTP flows only.

Otherwise, you might overwrite some "tls_quic_stun" fields (SNI...) with
random data.

Strangely enough, the "detected_os" field is never used: it is never
logged, or printed, or exported...
2021-02-03 11:54:11 +01:00
Ivan Nardi 8cee718e8b HTTP: fix logs when NDPI_ENABLE_DEBUG_MESSAGES is defined (#1123) 2021-02-03 11:53:41 +01:00
Luca Deri 155b1aaa68 Updated results due to the new fragment manager handler 2021-02-03 11:52:46 +01:00
Luca Deri 6e87daab69 Cosmetic fixes 2021-02-03 11:52:32 +01:00
Luca Deri 1f3ade5923 Increased number of extra packets that is necessary since the frgament mananger introduction 2021-02-03 11:51:55 +01:00
Luca Deri 4b181be58e Improved debug message 2021-02-03 11:49:14 +01:00
Luca Deri d19bad1581 Added pcap for testing fragments reassembly 2021-02-03 11:48:53 +01:00
Luca Deri 01f4a57118 Fixes an issue with https://github.com/ntop/nDPI/pull/1122 that misprocsssed packets belonging to flows whose initial part (e.g. the 3WH) was not observed by nDPI (e.g. capture started in the middle of the flow) 2021-02-03 11:47:21 +01:00
b70ad0e2f1 fragments management added (#1122)
Management of tcp segments managements.

Co-authored-by: ragostino <ragostino73@gmail.com>
Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2021-02-03 10:28:51 +01:00
ragostino ee94534906 debug message bugfix (#1108)
you can not look for memory enlargement if you print debug message after updating the variables
2021-02-03 09:31:17 +01:00
Luca Deri 50f4049b58 Improved wireguard dissection 2021-01-29 16:38:43 +01:00
Luca Deri bb74b903d0 DCE/RPC improvement to avoid false positives 2021-01-29 16:23:18 +01:00
Luca Deri aeeccee106 DGA name improvement 2021-01-27 11:43:35 +01:00
Luca Deri aa5486f5f8 Added missing library 2021-01-22 09:31:37 +01:00
Luca Deri 288ccd6215 Fixes due to datatype rename 2021-01-22 09:17:34 +01:00
Luca Deri a31bd5ac3c Cleaned up tls/quic datatypes 2021-01-21 19:17:33 +01:00
Luca Deri 15295ef4c5 Reworked TLS fingerprint calcolation
Modified TLS memory free
2021-01-21 19:06:05 +01:00
Toni 399755607d Disable tests that require libgcrypt if --disable-gcrypt set. (#1121)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-01-21 08:59:06 +01:00
morefigs 53415c8855 Added missing comma (#1116)
I presume there is a comma missing in this comma separated list.
2021-01-21 08:58:42 +01:00
Luca Deri 3e5e9569ff Added simple hash implementation to the nDPI API 2021-01-20 21:30:19 +01:00
Luca Deri d964c3e081 Code cleanup: third party uthash is at the right place 2021-01-20 19:11:36 +01:00
Luca 0809956e5f Rewored UPnP protocol that in essence was WSD hence it has been renamed
Cleaned up TLS code for DTLS detection by defining a new DTLS protocol
2021-01-20 09:53:30 +01:00
Luca Deri e9f43516ec Improves STUN dissection removing an invalid termination condition that prevented Skype calls to be properly identified 2021-01-13 16:46:51 +01:00
Luca Deri 68b6ac7da8 (C) Update 2021-01-07 11:13:36 +01:00
Luca Deri 367184628e Warning fix 2021-01-07 11:07:17 +01:00
Darryl Sokoloski 5f7b9d8024 Increase SNI hostname buffer length to 256. (#1111)
According to RFC 4366, SNI host names can be up to 255 bytes.
Previous size of 64 resulted in failed application matches due to truncation.

For example:
 0976e041e65b1aece3e720df36ac6bd7.safeframe.googlesyndication.co|m

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2021-01-07 10:58:48 +01:00
Ivan Nardi ec2735694b STUN: avoid false positives (#1110)
STUN traffic doesn't use multicast addresses
2021-01-07 10:58:22 +01:00
Ivan Nardi 1e2b57bed4 HTTP: fix compilation and a memory error when NDPI_ENABLE_DEBUG_MESSAGES is defined (#1109) 2021-01-07 10:58:02 +01:00
Ivan Nardi 2080cc7365 QUIC: add suppport for DNS-over-QUIC (#1107)
Even if it is only an early internet draft, DoQ has already (at least)
one deployed implementation.
See: https://www.zdnet.com/article/ad-blocker-adguard-deploys-worlds-first-dns-over-quic-resolver/
Draft: https://tools.ietf.org/html/draft-huitema-dprive-dnsoquic-00

In the future, if this protocol will be really used, it might be worth to
rename NDPI_PROTOCOL_DOH_DOT in NDPI_PROTOCOL_DOH_DOT_DOQ
2021-01-07 10:56:39 +01:00
Ivan Nardi 00dabce65e Quic fixes (#1106)
* QUIC: fix heap-buffer-overflow

* TLS: fix parsing of QUIC Transport Parameters
2021-01-07 10:55:55 +01:00
Ivan Nardi b8a5358e80 QUIC: improve handling of SNI (#1105)
* QUIC: SNI should be always saved in flow->protos.stun_ssl.ssl.client_requested_server_name

Close #1077

* QUIC: fix matching of custom categories

* QUIC: add NDPI_TLS_MISSING_SNI support for older GQUIC versions

* QUIC: fix serialization

* QUIC: add DGA check for older GQUIC versions
2021-01-07 10:55:23 +01:00
Luca Deri eb37f8f1fb Split HTTP request from response Content-Type. Request Content-Type should be present with POSTs and not with other methods such as GET 2021-01-06 18:28:24 +01:00
Luca Deri cd21f0d316 Added check for invalid HTTP content 2021-01-06 12:25:32 +01:00
Ivan Nardi 1b524f5538 QUIC: update to draft-33 (#1104)
QUIC (final!?) constants for v1 are defined in draft-33
2021-01-04 15:50:14 +01:00
Ivan Nardi 3aa16b63e8 Fix some warnings when compiling with "-W -Wall" flags (#1103) 2021-01-04 15:49:39 +01:00
Ivan Nardi 23b84cd3ee Remove FB_ZERO protocol (#1102)
FB_ZERO was an experimental protocol run by Facebook.
They switched to QUIC/TLS1.3 more than 2 years ago; no one ever used it but
them so it is definitely dead.
See: https://engineering.fb.com/2018/08/06/security/fizz/
2021-01-04 15:49:19 +01:00
Toni 53ee6db795 Added a new API function `ndpi_free_flow_data' which free's all members of ndpi_flow_struct but not the struct itself. (#1101)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-01-04 12:47:05 +01:00
Ivan Nardi 3bb12fde63 Fix memory leak introduced in b7376cc6 (#1100) 2021-01-04 12:46:30 +01:00
Luca Deri 4ddb5f4245 Added TLS test with long certificate 2021-01-04 11:31:25 +01:00
Luca Deri a91e6179c5 Fixed missing symbol 2021-01-02 21:21:15 +01:00
Luca Deri 05d76525b0 Added HTTP suspicious content securirty risk (useful for tracking trickbot) 2021-01-02 21:11:42 +01:00
Luca Deri 32f0446c9c Updated ndpi_ptree_match_addr() prototype 2020-12-30 18:23:40 +01:00
Luca Deri b7376cc690 Restored QUIC stats 2020-12-30 12:12:33 +01:00
Luca Deri dc401f8a74 Split ptree user data in 32 and 64 bit entries 2020-12-30 12:12:33 +01:00
pengtian ab037e2c82 Bugfix for host check (#1097)
this bug is from commit `427002d14` `2020-05-06 00:31:40`
2020-12-29 14:41:41 +01:00
Luca Deri 84132c6735 Added known protocol on unknown port for ntop 2020-12-28 16:11:39 +01:00
Luca Deri 7f944cc43b Initialization fix 2020-12-28 11:57:20 +01:00
Luca Deri 0fceb6576a Free flow fix 2020-12-28 09:32:06 +01:00
Luca Deri 9c1827a77b Fixed output when tLS (nad not QUIC) is used 2020-12-28 09:19:39 +01:00
Luca Deri cf00ffed32 Removed test code 2020-12-26 18:13:10 +01:00
Luca Deri e1be363037 Removed space from protocol name 2020-12-23 20:38:02 +00:00
Luca Deri 34fc9d5d50 Introduced fix on TLS for discarding traffic out of sequence that might invalidate dissection 2020-12-22 09:47:39 +01:00
Luca Deri 6462f4671c Fixed invalid TLS check for extra processing detection 2020-12-21 22:42:05 +01:00
Luca Deri b231982286 Win fixes 2020-12-17 15:34:17 +01:00
Luca Deri df1b3367a9 Type change to avoid Windows compilation issues 2020-12-17 12:58:12 +01:00
Luca Deri 2768da0637 Improved HTTP dissection 2020-12-16 14:45:29 +01:00
Luca Deri a89642ad04 Fixes bug introduced by https://github.com/ntop/nDPI/pull/1085 2020-12-12 12:32:44 +01:00
Zied Aouini 5bd5461f96 Fix minimum packet length condition (#1087) 2020-12-12 11:12:59 +01:00
Toni 74a77e7b3d Added --ignore-vlanid / -I to exclude VLAN ids for flow hash calculation. #1073 (#1085)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-12-11 21:01:51 +01:00
Toni edf3a57a6a Added `fast.com' host pattern to Netflix detection. #1080 (#1084)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-12-11 21:01:28 +01:00
Ivan Nardi 5587010cfb soulseek: fix heap buffer overflow (#1083)
Close #1082
2020-12-11 21:01:07 +01:00
Luca Deri 5cb6ddfd22 Rule changes work in progress 2020-12-11 17:25:57 +01:00
Luca Deri 21ad3a1775 Added initialization 2020-12-11 17:25:57 +01:00
Alfredo Cardigliano 499897219e Add NDPI_PROTOCOL_ANY_CATEGORY to ndpi_protocol_category_t enum 2020-12-11 15:56:57 +01:00
Luca Deri 9c5fbb5208 Added FreeBSD ibuils instructions 2020-12-10 14:19:59 +00:00
Luca Deri 4cb8712a35 Added --with-mipsel for building nDPI on mipsel devices 2020-12-09 11:01:40 +00:00
Toni 62cd852c6f Rename Jabber detection name as we are not sure if it is unencrypted e.g. if START_TLS used. (#1079)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-12-08 15:49:17 +01:00
rafaliuszandrafal 1ecc6d323e Add a connectionless DCE/RPC detection (#1078)
* Add connectionless DCE/RPC detection

* Add DCE/RPC pcap file as well as its test result

Co-authored-by: rafal <rafal.burzynski@cryptomage.com>
2020-12-08 15:48:53 +01:00
Ivan Nardi 860ef58ace QUIC: sync with Wireshark latest changes (#1074)
Most of the QUIC crypto code has been "copied-and-pasted" from Wireshark;
try to stay in sync with the original sources to ease backporting of fixes.

Only cosmetic changes and code refactoring; no behaviour changes or bugfixes.
See:
https://gitlab.com/wireshark/wireshark/-/commit/5e45f770fd79ca979c41ed397fee72d2e8fb5f1e
https://gitlab.com/wireshark/wireshark/-/commit/5798b91c1526747bf688b6746b33562c1b24a9e0
2020-12-08 15:47:58 +01:00
Luca Deri 1fb1848586 Added chec for avoiding doung ntop_config.h include 2020-12-01 08:57:09 +00:00
Luca Deri eb689b2069 nDPI rules (work in progress) implementation 2020-11-30 22:01:49 +01:00
Luca Deri b77a4276f9 Fixed JSON-C check 2020-11-30 16:41:58 +01:00
Luca Deri d4a42ed733 Improved JSON-C detection 2020-11-30 16:30:38 +01:00
Luca Deri 6b9155589a Removed unnecessary field 2020-11-29 19:04:26 +01:00
Luca Deri 23a15bae5f Fixes #1029 2020-11-27 18:51:56 +01:00
Luca Deri 6c119c0b06 Fixes #1032 2020-11-27 18:48:25 +01:00
Toni daa1c8ef8c Fix/overflow and libgerror check (#1068)
* Fixed stack overflow caused by missing buffer space for the trailing \0 added by sprintf()

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Remove the autoconf cache value from the previous and failed check before checking again.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-26 19:46:31 +01:00
Alfredo Cardigliano 7437ecaab0 Warning fix 2020-11-23 10:19:35 +01:00
Luca Deri 38984af03e Updated disney definition 2020-11-22 18:49:28 +01:00
Luca Deri 78f762bd36 Typo 2020-11-22 16:51:13 +01:00
Luca Deri 21a6112b91 Added 129.205.94.0/23 to Facebook network list 2020-11-22 11:33:31 +01:00
Ivan Nardi 53a5c354d8 Quic fixes (#1067)
* QUIC: fix return value on error path on quic_cipher_init()

* QUIC: allow dissection of sessions forcing version negotiation

Enhance heuristic to avoid false positives.
2020-11-22 11:04:10 +01:00
Ivan Nardi fb2027cc8e iec60870-5-104: fix heap-buffer-overflow error (#1066) 2020-11-22 11:03:26 +01:00
Luca Deri fe3d926741 Added prerequisites for Ubuntu/Debian systems 2020-11-18 16:40:45 +01:00
Alfredo Cardigliano e45f0a4b64 Install ndpi under /usr 2020-11-17 09:19:07 +01:00
Luca Deri d670850f55 Updated results 2020-11-16 21:25:59 +01:00
Luca Deri acffd7bd1e Compilation fix 2020-11-16 21:22:54 +01:00
Zied AouiniandLuca Deri bfabb0ddf4 Add Virtual Asssitant (Alexa, Siri) support. (#1057)
* Add AmazonAlexa protocol.

* Add AmazonAlexa test file and result.

* Include pcapng as file format.

* Rename Category to VirtualAssistant.

* Add AppleSiri virtual assistant.

* Fix pcapng test files format support.

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2020-11-16 21:19:38 +01:00
Oleksandr Natalenko 5c8e125f06 lib/ndpi_content_match: update TOR IPs (#1063)
Signed-off-by: Oleksandr Natalenko <oleksandr@natalenko.name>
2020-11-16 21:18:17 +01:00
Zied Aouini 3d8fd42307 Implement DGA detection performances tracking workflow. (#1064)
* Implement dga evaluation helper.

* Add test set for DGA classification.

* Add DGA classification performances tracking as part of Travis.

* Add DGA evaluation doc.

* Fix CI on OSX.

* Add missing backquote.
2020-11-16 21:17:16 +01:00
Zied Aouini 76bb83085b Improve subprotocols detection. (#1062)
* Improve Spotify detection.

* Improve Skype detection.

* Improve Microsoft detection.

* Fix Microsoft detection categories.

* Improve Waze detection.

* Improve Apple detection.

* Improve WindowsUpdate detection.

* Improve TikTok detection.

* Improve Teams detection.

* Improve Youtube detection.

* Improve Messenger detection.

* Improve Twitch detection.

* Improve Hulu detection.

* Improve Facebook detection.

* Improve AmazonVideo detection.
2020-11-16 21:16:38 +01:00
Zied AouiniandLuca Deri 3529268df8 Add Tumblr support. (#1061)
* Add Tumblr protocol.

* Add Tumblr test file and result.

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2020-11-16 21:14:06 +01:00
Zied AouiniandLuca Deri 22780da8d5 Add Reddit support. (#1060)
* Add Reddit protocol.

* Add Reddit test file and result.

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2020-11-16 21:13:01 +01:00
Zied AouiniandLuca Deri 13dab51cc7 Add Pinterest support. (#1059)
* Add Pinterest protocol.

* Add Pinterest test file and result.

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2020-11-16 21:11:43 +01:00
Zied Aouini f35fad13c3 Add DisneyPlus protocol. (#1058) 2020-11-16 21:10:11 +01:00
Luca Deri 419483f128 File nor necessary and generated by configure 2020-11-12 23:22:15 +01:00
Luca Deri 328ff24657 Renumbered AmongUs protocol 2020-11-09 16:23:01 +01:00
Toni 6b5bdf773d Added support for AmongUs. (#1054)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-09 16:19:00 +01:00
Toni af02ffb60f Support raw IPv4 / IPv6 pcap packet processing. (#1053)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-09 16:18:05 +01:00
Toni ac18d5a084 Improved SSH protocol detection. (#1052)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-09 16:17:42 +01:00
Don J. Rude bbf42c3610 fixes issue #1050 Syntax error caused buffer pointer to equal 0x1 (#1051)
* Syntax error caused buffer pointer to equal 0x1

Possible copy-paste from lines 141-142?

* Another comma operator

* whitespace matching

* another comma operator

* another comma operator

* another comma operator

* Check for non-zero payload
2020-11-09 16:17:10 +01:00
Luca Deri 89a363aff6 Updated ESNI/SNI alarm generation prolicy 2020-11-08 10:07:35 +01:00
Luca Deri f6cb869c63 Reworked IEC60870 dissector 2020-11-04 14:29:11 +01:00
Luca Deri 044a11faca IEC60870 dissection improvements 2020-11-04 12:40:31 +01:00
Luca Deri 017e395ed1 Cosmetic changes 2020-11-03 16:46:30 +01:00
Leonn 0576dc2a49 💡 Add mongodb protocol dissector (#1048) 2020-11-03 16:16:02 +01:00
Luca Deri fc58e564f0 Restored Ubuntu/Debian packaging 2020-11-03 16:08:57 +01:00
Toni 8e84abfa0e Fix/packaging (#1047)
* Fix OpenWrt build.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Fixed Debian/Ubuntu packaging.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Added DPKG package build to a CI job.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-03 11:36:40 +01:00
Ivan Nardi a9547da138 QUIC: fix dissection of Initial packets coalesced with 0-RTT one (#1044)
* QUIC: fix dissection of Initial packets coalesced with 0-RTT one

* QUIC: fix a memory leak
2020-11-03 11:35:52 +01:00
Luca Deri 92b80e2468 Updated results with numeric IP detection 2020-11-01 13:31:00 +01:00
Luca Deri b502bb08e0 Fix for detecting numeric IPs 2020-11-01 13:31:00 +01:00
Luca Deri 50ab34bfe6 Centos8 package fix 2020-10-29 18:36:56 +01:00
Alfredo Cardigliano 53f133f669 Fix libdir on centos8 2020-10-29 18:20:51 +01:00
Alfredo Cardigliano 29dbf78c4e Removed generation of /usr/lib/.build-id 2020-10-29 17:50:05 +01:00
Luca Deri efbfb7cbd6 Set /usr as installation prefix 2020-10-29 17:45:55 +01:00
Alfredo Cardigliano 1c4ce8b0f4 Move libraries from /usr/local to /usr 2020-10-29 12:43:33 +01:00
Vitaliy Ivanov 10506b5173 Generate NDPI_API_VERSION based on git commands.
https://github.com/ntop/PF_RING/issues/668

Check for .git directory works in case of usual clone of nDPI repo.
But if you integrate is as submodule then .git is a file.

Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com>
2020-10-29 12:35:05 +01:00
Luca Deri 94471193c2 Added boundary check 2020-10-27 08:48:12 +01:00
Igor Duarte ba6a48c9fe Improve skype detection (#1039)
* Add new skype pcap

PCAP extracted from SkypeIRC.cap (available in https://wiki.wireshark.org/SampleCaptures?action=AttachFile&do=get&target=SkypeIRC.cap)

* Improve skype detection
2020-10-27 08:45:09 +01:00
Luca Deri 48d640583a Moved global in reader_util.c 2020-10-27 08:40:00 +01:00
Luca Deri 948a906037 Added -D flag for detecting DoH in the wild
Removed heuristic from CiscoVPN as it leads to false positives
2020-10-26 21:40:59 +01:00
Luca Deri 9873972acb Various improvemement when using ndpi_pref_enable_tls_block_dissection:
application data TLS blocks are now ignored when exchanged before
- the end of certificate negotiation (up to TLS 1.2)
- change cipher
2020-10-24 19:22:56 +02:00
Luca Deri 9b85669a64 Added -x for checking patterns 2020-10-22 23:28:18 +02:00
Luca Deri 833d0eee53 Added CPHA - CheckPoint High Availability Protocol protocl support 2020-10-22 18:39:13 +02:00
Ivan Nardi 6027a7c799 Fix parsing of DLT_PPP datalink type (#1042) 2020-10-21 22:27:42 +02:00
Luca Deri 9dac9945c9 Fixes #1033 2020-10-21 20:59:02 +02:00
DouglasandDouglas Wordell efb712c2c0 Added new IPs for TeamViewer (#1040)
Co-authored-by: Douglas Wordell <douglas.wordell@seti.com.br>
2020-10-21 15:09:14 +02:00
Zied Aouini 43c1f6a3fd CAPWAP tunnel decoding fix (#1038)
* Fix CAPWAP processing.

* Update result.
2020-10-21 15:07:20 +02:00
Toni 822c8e56f7 Improved/autoconf (#1037)
* Switched to PKG_CHECK_MODULES to keep pkg-config checks more portable.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Improved GCrypt autoconf check to detect a possible gpg-error inter-library dependency.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-10-21 15:05:33 +02:00
Hosein Ghahremanzadeh bb33d57971 Fix a memory leak on fuzzing code (#1036)
After allocation of pcap_buffer it is necessary to free it
2020-10-21 15:05:11 +02:00
Igor Duarte 13d99a9390 Remove possible invalid read (#1035) 2020-10-21 15:04:55 +02:00
Luca Deri 6c0a809de8 Initial 3.5 commit 2020-10-19 16:23:03 +02:00
Luca Deri 64929a75e0 3.4 nDPI Release 2020-10-19 16:18:25 +02:00
Luca Deri 02de8e10ca Added makefile target for creating changelog
Updated Changelog
2020-10-19 15:39:48 +02:00
Toni c2d8955c18 Fixed missing PCAP include directories in Makefiles. (#1034)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-10-19 11:25:22 +02:00
Luca Deri e9721eb4ce Improved processing of IPv6 header
Improved QUIC serialization
2020-10-15 21:57:59 +02:00
Alfredo Cardigliano 88f3519ab3 Handle list items in ndpi_deserialize_get_single_size 2020-10-15 12:02:43 +02:00
Alfredo Cardigliano 031cd04b7c Add serialization of values list in TLV 2020-10-15 10:45:43 +02:00
aouinizied d5d2a7e3f3 Fix CAPWAP handling. 2020-10-13 19:13:07 +02:00
Alfredo Cardigliano 2051e51380 Decoupled fuzzy and unit tests 2020-10-12 12:11:35 +02:00
Alfredo Cardigliano c92e0d13fd Serialized doxygen doc 2020-10-05 11:25:48 +02:00
Luca 05d93790e4 Added ndpi_quick_16_byte_hash
Warning fix
2020-10-05 08:26:24 +02:00
Luca Deri 32e4922c5a Warning fix 2020-10-03 17:14:40 +02:00
Luca Deri d4d82c27a5 Tests update 2020-10-02 21:35:15 +02:00
Luca Deri addc775811 Added fix for invalid SNI check when SNI is missing 2020-10-02 21:26:43 +02:00
Luca Deri b68a3707f6 Updated serialization test unit 2020-10-02 12:39:08 +02:00
b0c4d01734 don't pass arguments to ./configure as 1 string (#1027)
this causes the following error if there are arguments after --host :
'checking host system type... config.sub: too many arguments'

Co-authored-by: sahnalys12 <boris.vanhoof_ext@softathome.com>
Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2020-09-29 18:02:08 +02:00
Ivan Nardi 2c1b7cf1bd QUIC: fix dissection of "offset" field (#1025)
The "offset" field is a variable-length integer.
This bug hasn't any practical effects right now, since we are ignoring any
packet with "offset" != 0 (and the value 0 is always encoded in only one byte).
But extracting a correct "offset" is important if we are ever going to handle
fragmented Client Hello messages.
2020-09-29 17:59:03 +02:00
Toni 656323c334 Added missing files to `make dist' target which are not required to build nDPI but still somehow essential. (#1024)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-29 17:58:33 +02:00
Luca Deri 8e1a3f5cdc Added JSON-C check for unit tests 2020-09-26 12:50:30 +02:00
Luca Deri 58538b7d19 Added extra boundary checks 2020-09-26 12:33:57 +02:00
Luca Deri d0dbcf6dd6 Added back ndpi_check_flow_func (correct) prototype 2020-09-25 10:39:33 +02:00
Luca Deri 8413faf2ef Boundary fix 2020-09-25 09:56:24 +02:00
Luca Deri 204f3ddff5 Compilation fix 2020-09-25 09:36:10 +02:00
Luca Deri 044ed14b4f Various optimizations to reduce not-necessary calls
Optimized various UDP dissectors
Removed dead protocols such as pando and pplive
2020-09-24 23:26:03 +02:00
Luca Deri 51e86346d3 Added -fno-sanitize=alignment in clang-7 2020-09-24 16:46:28 +02:00
Luca Deri 8e7a707be1 -O0 -> -Og 2020-09-23 18:49:44 +02:00
Luca Deri 837ce7306b Attempt to avoid errors due to misaligned packet headers 2020-09-23 18:17:51 +02:00
Luca Deri 05dfae6430 Improved boundary check to prevent overflow 2020-09-23 17:14:40 +02:00
Luca Deri d29a0241ee Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2020-09-22 12:45:44 +02:00
Luca Deri cbf1be5789 Macro redefinition 2020-09-22 12:23:44 +02:00
Luca Deri 5f99433ee7 Minor UA handling improvement to avoid heap-overflow 2020-09-22 08:56:35 +02:00
lucaderi 27bad14738 Added missing install target in newly added unit tsts 2020-09-22 08:42:50 +02:00
Luca Deri 362baa6321 Minor change for alignment issue 2020-09-21 21:15:52 +02:00
Luca Deri 60a9f6610d Added risks for checking
- invalid DNS traffic (probably carrying exfiltrated data)
- TLS traffic with no SNI extension
2020-09-21 19:57:23 +02:00
Alfredo Cardigliano 0259ff58e1 Exclude sanitizer on unit tests involving json-c due to a bug in the lib 2020-09-21 17:59:06 +02:00
Alfredo Cardigliano d700ab3994 Add distdir directive 2020-09-21 17:32:57 +02:00
Alfredo Cardigliano e6d206fd15 Add unit tests to travis. Move ndpi serializer tests to unit tests. 2020-09-21 17:24:06 +02:00
Luca Deri a263ac9024 Merge pull request #1022 from lnslbrty/fix/ndpi-mingw-fn-redef
Do not re-define libc functions for mingw builds.
2020-09-21 14:42:06 +02:00
Toni Uhlig 87c43450f3 Do not re-define libc functions for mingw builds.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-20 23:29:23 +02:00
Luca Deri 8ba1baace1 Added sample Windows MSVC project 2020-09-20 18:44:57 +02:00
Luca Deri ff1cc1fa41 Merge pull request #1019 from IvanNardi/quic_fb
QUIC: add support for MVFST EXPERIMENTAL version
2020-09-20 18:34:27 +02:00
Luca Deri ea9177cc93 Merge pull request #1018 from lnslbrty/fix/make-dist
Fixed broken `make dist' and added CI check.
2020-09-20 18:33:57 +02:00
Luca Deri 1385f05713 Merge pull request #1017 from lnslbrty/fix/mingw-xcompile
Added support for mingw xcompile.
2020-09-20 18:33:40 +02:00
Nardi Ivan dcac633878 QUIC: add support for MVFST EXPERIMENTAL version 2020-09-20 16:38:28 +02:00
Toni Uhlig fdef6f3545 Fixed broken `make dist' and added CI check.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-19 02:23:53 +02:00
Toni Uhlig d0052fd5b2 Using NDPI_I64_FORMAT, NDPI_U64_FORMAT format string to differentiate between Windows and non-Windows.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-18 19:46:20 +02:00
Toni Uhlig bcce122665 Fixed mingw build w/o examples/tests/fuzzer.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-18 19:46:07 +02:00
Luca Deri 6a7139bb24 Updated results 2020-09-18 00:17:43 +02:00
Luca Deri b5b4512022 Added MDNS in extra packet dissection function 2020-09-18 00:08:27 +02:00
Luca Deri d81bc1add6 Reworked MDNS dissector that is not based on the DNS dissector 2020-09-17 23:24:02 +02:00
Luca Deri 753b5dde16 Merge pull request #1012 from IvanNardi/ua
QUIC: extract User Agent information
2020-09-17 21:32:25 +02:00
Luca Deri 9189aee83b Disabled QUIC tracing that pollutes the output 2020-09-17 14:37:56 +02:00
aouinizied 4cc4efa228 Update example. 2020-09-14 16:42:33 +02:00
Luca ad8318e247 Commented unused function 2020-09-13 09:48:29 +02:00
Alfredo Cardigliano 91e3292e6b Temporarily keep using snprintf 2020-09-11 19:17:10 +02:00
Luca Deri 98a81cb163 Reworked DGA checks 2020-09-11 10:31:19 +02:00
Alfredo Cardigliano 3a6ba7fa12 Replace snprintf with ndpi_snappend 2020-09-11 10:04:43 +02:00
Luca Deri 37abe0daea Added boundary check 2020-09-10 20:06:25 +02:00
Luca Deri 1c0e617d80 Added buffer check in DGA analysis 2020-09-10 19:58:31 +02:00
Nardi Ivan 8db084ab06 TLS: fix memory accesses in QUIC transport parameters extension 2020-09-10 18:19:18 +02:00
lucaderi ea4240055d Complation fix as not on all platforms isnumber() is available 2020-09-10 00:05:02 +02:00
Luca Deri 5ac870074b Merge pull request #1014 from lnslbrty/improved/teamspeak
Improved Teamspeak(3) protocol detection.
2020-09-09 23:28:21 +02:00
Luca Deri 7086197047 Added extension to detect nested subdomains as used in Browsertunnel attack tool
https://github.com/veggiedefender/browsertunnel
2020-09-09 23:25:19 +02:00
Toni Uhlig 8ca13bc46a Improved Teamspeak(3) protocol detection.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-09 21:57:31 +02:00
Alfredo Cardigliano 842c00e681 Add missing low-level serializer calls to the API 2020-09-09 17:09:01 +02:00
Toni Uhlig 4c776501a6 Fixed shlib xcompile for x86_64-w64-mingw32
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-08 19:52:46 +02:00
Nardi Ivan 7d5a0e1f04 QUIC: extract User Agent information 2020-09-08 11:03:22 +02:00
Nardi Ivan a1014e8895 http: create a common function to parse User Agent field
Prepare the code to handle UA information from flows other than HTTP
2020-09-08 10:34:05 +02:00
Luca Deri 601928cc8e Merge pull request #1008 from lnslbrty/add/dnscrypt
Improved dnscrypt v1/v2 protocol detection (and DoH provider).
2020-09-08 09:53:22 +02:00
Luca Deri 5a3956fe27 Added command for specifying the aux files dir 2020-09-08 09:51:02 +02:00
Toni Uhlig df14d225f6 Added pcap file which contains dnscrypt-v1 data and resolver update requests/responses (v1/v2).
* Renamed dnscrypt.pcap to simple-dnscrypt.pcap

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-07 21:04:23 +02:00
Toni Uhlig fe5aa7ebca Added dnscrypt-v2-doh resolver test pcaps.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-07 20:22:52 +02:00
Toni Uhlig 41d35fc7d0 Updated DoT/DoH provider.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-07 20:22:45 +02:00
Toni Uhlig 799a6abfe6 Improved dnscrypt v1/v2 protocol detection.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-06 23:24:55 +02:00
Alfredo Cardigliano 8ed6409b35 Add start_of_block/end_of_block support to TLV 2020-09-04 17:23:04 +02:00
Luca Deri 3f49062db9 Added some additional TLS mappings 2020-09-02 23:35:22 +02:00
Luca Deri 3cca5af8c3 Merge pull request #1003 from lnslbrty/fix/fals-positive-cisco-hsrp-radius-detection
Fixed false positive detection for Skype.SkypeCall (affects at least Cisco HSRP and RADIUS).
2020-09-02 12:32:48 +02:00
Toni Uhlig 7051acc0b5 Fixed off-by-one error in Kerberos protocol.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-02 07:37:26 +02:00
Toni Uhlig 580859a47d Fixed false positive detection for Skype.SkypeCall (affects at least Cisco HSRP and RADIUS).
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-02 07:29:33 +02:00
Luca Deri 5d2551ff3e Added boundary check 2020-09-01 22:31:34 +02:00
Luca Deri cefb667d8b Added trademark information 2020-09-01 18:03:01 +02:00
Luca Deri 8575a07113 Added check for ndpi_ssl_version2str() 2020-08-31 17:23:00 +02:00
Luca Deri 460ff3c7ab Added (optional) notifier for LRU add 2020-08-31 08:47:16 +02:00
Luca Deri 029448759b Merge pull request #999 from IvanNardi/quic
QUIC: add support for GQUIC T050 and T051
2020-08-30 20:56:15 +02:00
Luca Deri 448265a151 Merge pull request #998 from lnslbrty/fix/ndpireader-opt-sigsegv
Fixed invalid memory access leading to a SIGSEGV in ndpiReader's option parser.
2020-08-30 20:55:44 +02:00
Nardi Ivan 7da4abe6ad QUIC: add support for GQUIC T050 and T051
QUIC versioning wasn't complex enough without T05X family...
These versions are very similar to Q050, but use TLS as their handshake
protocol.
2020-08-30 20:51:33 +02:00
Luca Deri 638624869a Added new risk for NDPI_UNSAFE_PROTOCOL that identifies protocols that are not condidered safe/secure 2020-08-30 20:48:58 +02:00
Luca Deri 647c7d59bb Improved ntop detection over HTTP
Added cap on number of attempts for CiscoVPN
2020-08-30 19:42:09 +02:00
Luca Deri f597086386 Stddev calculation changes 2020-08-30 12:48:32 +02:00
Luca Deri dd75060932 Fixed false positive in suspicous user agent
Optimized stddev calculation
2020-08-30 12:25:15 +02:00
Luca Deri 8bf95b6198 Merge pull request #996 from lnslbrty/fix/travis-ci
Fix travis-ci related errors.
2020-08-28 13:04:26 +02:00
Toni Uhlig adab0139c2 Fixed invalid memory access leading to a SIGSEGV in ndpiReader's option parser.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-28 06:53:58 +02:00
Toni Uhlig bf9dbd6ccf Fixed travis-ci fuzzm job. Might be a false positive related to clang-7's MSAN.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-27 16:42:24 +02:00
Toni Uhlig e998fc28d1 Fixed use-of-uninitialized-value in QUIC clho decryption probably caused by a BUG in libgcrypt (not verified).
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-27 16:30:05 +02:00
Simone Mainardi 1810281d7f Fixes control reaches end of non-void function 2020-08-27 10:39:36 +02:00
Simone Mainardi f0325f713a Passes method_len param to ndpi_http_str2method 2020-08-27 10:05:38 +02:00
Toni Uhlig 5b27dfdd0c Moved NDPI_CURRENT_PROTO define before ndpi_api.h include to prevent a redefinition warning.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-27 00:29:44 +02:00
Toni Uhlig c17463539f Fixed broken travis-ci YAML.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-27 00:27:31 +02:00
Luca Deri 2e8aa9f08e Added ndpi_http_method ndpi_http_str2method(const char* method) API call 2020-08-26 23:17:19 +02:00
Luca Deri 487d915262 Added ndpi_http_method2str() API call 2020-08-26 12:25:28 +02:00
Luca Deri 5e1bb0dbc1 Merge pull request #992 from lnslbrty/fix/pkg-config
Fixed broken pkg-config file which did not care about gcrypt/pcre.
2020-08-26 10:29:39 +02:00
Toni Uhlig 363ab2259c Fixed broken pkg-config file which did not care about gcrypt/pcre.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-24 18:12:49 +02:00
Luca Deri b94e13da43 Merge pull request #991 from IvanNardi/quic2
QUIC: minor fixes
2020-08-24 14:45:04 +02:00
Nardi Ivan 97b80a8838 QUIC: minor fixes
LGTM found a real issue on a boundary check
Fix unit tests: a pcap ha been uploaded twice (with different names)
Fix compilation when using DPDK (see #990)
2020-08-24 13:53:36 +02:00
Luca Deri b8307eb855 Creared IoT-Scada category
Minor dnp3 changes
2020-08-23 13:32:36 +02:00
Luca Deri 46ba657562 Cosmetic fix 2020-08-22 23:48:00 +02:00
Luca Deri 4570dbfece Added QUIC dependency 2020-08-22 17:02:11 +02:00
Luca Deri 35fd4ddb27 Added libgcrypt20-dev dependency to handle QUIC 2020-08-22 16:53:44 +02:00
Luca Deri fe1e2c241f Added som GQUIC and IETF QUIC test pcaps 2020-08-22 16:47:05 +02:00
Luca Deri af3ce74e02 Fixes compilation issues introduced by https://github.com/ntop/nDPI/pull/989 2020-08-22 16:41:51 +02:00
Luca Deri 93003d0523 Warning fix 2020-08-22 16:41:42 +02:00
Luca Deri 519ba7a9d5 Merge pull request #989 from IvanNardi/quic
Improve QUIC detection
2020-08-22 16:30:29 +02:00
Nardi Ivan b23cfd6b84 Add sub-classification for GQUIC >= Q050 and (IETF-)QUIC
Add QUIC payload and header decryption: most of the crypto code has been
"copied-and-incolled" from Wireshark. That code has been clearly marked
as such. All credits for that code should go to the original authors.

I tried to keep the Wireshark code as similar as possible to the original,
comments included, to ease future backporting of fixes.
Inevitably, glibc data types and data structures, tvbuff abstraction and
allocation functions have been converted.
2020-08-21 22:04:55 +02:00
Nardi Ivan 7a1147d733 Update TLS dissector to handle QUIC flows
Latest QUIC versions use TLS for the encryption layer: reuse existing code
to allow Client Hello parsing and sub-classification based on SNI value.
Side effect: we might have J3AC, TLS negotiated version, SNI value and
supported cipher list for QUIC, too.
2020-08-21 22:04:55 +02:00
Nardi Ivan d62ae567d1 Add (optional) dependency on external libraries: libgcrypt and libgpg-error
To support QUIC payload and header decryption, it is necessary to choose an
external crypto library to handle the low-level crypto stuff. Since we will
use some Wireshark code, it is quite natural to choose the same library used
by Wireshark itself: libgcrypt.

More precisely, we will use libgcrypt and libgpg-error.
Both libraries have LGPL license, so there should be no issue from this point
of view.

These libraries are not required to build nDPI, and their usage is optional:
nDPI will keep working (and compiling) even if they are not available.
However, without them, QUIC sub-classification is next to impossible.
The configure flag "--disable-gcrypt" forces the build system to ignore these
libraries.

libgpg-error is only used for debug to have meaningful error messages and its
usage is trivial.
The same cannot be said for libgcrypt because its initialization is a significant
issue.

The rest of this commit message try explaining how libgcrypt is
initialized.

According to the documentation
   https://gnupg.org/documentation/manuals/gcrypt/Initializing-the-library.html
   https://gnupg.org/documentation/manuals/gcrypt/Multi_002dThreading.html#Multi_002dThreading
libgcrypt must be initialized before using it, but such initialization should
be performed by the actual application and not by any library.

Forcing the users to proper initialize libgcrypt in their own code seems
unreasonable: most people using nDPI might be complete unaware of any crypto
stuff and update each and every one application linking to nDPI with specific
libgcrypt code should be out of question, anyway.

Fortunately, it seems a workaround exists to initialize libgcrypt in a library
    https://lists.gnupg.org/pipermail/gcrypt-devel/2003-August/000458.html

Therefore, we could provide a wrapper to this initialization stuff in a nDPI
function. Unfortunately nDPI API lacks a global init function that must be
called only once, before any other functions. We could add it, but that would
be a major API break.
AFAIK, ndpi_init_detection_module() might be called multiple times, for example
to create multiple independent dpi engines in the same program.

The proposed solution is to (optionally) initialize libgcrypt in
ndpi_init_detection_module() anyway:
* if the actual application doesn't directly use libgcrypt and only calls
  ndpi_init_detection_module() once, everything is formally correct and it
  should work out of the box [by far the most common user case];
* if the actual application already uses libgcrypt directly, it already
  performs the required initialization. In this case the ndpi_prefs.ndpi_dont_init_libgcrypt
  flag should be passed to ndpi_init_detection_module() to avoid further
  initializations.

The only scenario not supported by this solution is when the application is
unaware of libgcrypt and calls ndpi_init_detection_module() multiple times
concurrently. But this scenario should be uncommon.

A completely different option should be to switch to another crypto library,
with a huge impact on the QUIC dissector code.

Bottom line: crypto is hard, using libgcrypt is complex and the proposed
initialization, even if not perfect, should cover the most frequent user
cases and should work, for the time being.
If anyone has some suggestions...
2020-08-21 22:04:55 +02:00
Nardi Ivan 23ec82b59d Major rework of QUIC dissector
Improve support for GQUIC (up to Q046) and add support for Q050 and (IETF-)QUIC
Still no sub-classification for Q050 and QUIC
2020-08-21 22:04:55 +02:00
Luca Deri fef199ad45 Added new check for detecting suspicious (too long) names 2020-08-21 19:35:27 +02:00
Luca Deri b23781e807 Added the ability do identigy as DGA those host/domain names with too many consucutive repeated characters
such as ckaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa used fr netbios reflection attacks
https://www.akamai.com/uk/en/multimedia/documents/state-of-the-internet/ddos-reflection-netbios-name-server-rpc-portmap-sentinel-udp-threat-advisory.pdf
2020-08-21 18:41:35 +02:00
Luca Deri da2684dbe1 MySQL8 update 2020-08-21 07:17:34 +02:00
Luca Deri 7e9354ef23 Merge pull request #988 from lnslbrty/add/mysql-8-manipulated-pkt
Added (manipulated) MySQL 8 test pcap.
2020-08-21 06:49:18 +02:00
Toni Uhlig f4421314b0 Added (manipulated) MySQL 8 test pcap.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-20 23:46:47 +02:00
Luca Deri 59ac73b37f Configure code cleanup 2020-08-19 22:01:03 +02:00
Luca Deri 115888ef7a Compilation fix 2020-08-19 21:56:34 +02:00
Luca Deri 248d756e99 Merge pull request #987 from lnslbrty/update/mysql-protocol-detection
Updated MySQL protocol detection to support server version 8.
2020-08-19 21:49:53 +02:00
Luca Deri 9f431f9218 Merge pull request #985 from lnslbrty/add/SOAP
Added support for SOAP.
2020-08-19 21:49:12 +02:00
Luca Deri d87bdf9302 Merge pull request #977 from adek05/dev
Enable building on OpenBSD 6.7
2020-08-19 21:48:38 +02:00
Toni Uhlig 3a76786121 Updated MySQL protocol detection to support server version 8.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-19 18:33:06 +02:00
Toni Uhlig 8e93f48c43 Added support for SOAP.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-18 16:21:26 +02:00
Luca Deri 673d8b7fac Updated OpenWRT instructions 2020-08-17 18:17:43 +02:00
Luca Deri 1f921562d1 Added fix for API versions starting with 0 2020-08-17 18:11:18 +02:00
Luca Deri 34a98abcc0 Added --with-only-libndpi configure option to build just lindpi 2020-08-17 18:00:38 +02:00
Luca Deri b501239d3f Added README for building libndpi under OpenWRT 2020-08-17 17:17:09 +02:00
Luca Deri ebf89f46e3 Merge pull request #984 from lnslbrty/fix/invalid-fncall-dpdk
Fixed invalid dpdk fn call.
2020-08-16 10:03:55 +02:00
Luca Deri 019a64630b Merge pull request #983 from lnslbrty/fix/libpcap-obsolete-pcap_lookupdev-usage
Replaced obsolete libpcap pcap_lookupdev with pcap_findalldevs.
2020-08-16 10:03:33 +02:00
Luca Deri 98a9afc40c Added support for discord 2020-08-16 10:01:40 +02:00
Toni Uhlig 13cbd1e4ae Fixed invalid dpdk fn call.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-15 12:38:47 +02:00
Toni Uhlig b31fde4bbb Replaced obsolete libpcap pcap_lookupdev with pcap_findalldevs.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-15 12:35:49 +02:00
Luca Deri 46ec91ca4b Merge pull request #976 from lnslbrty/fix/ndpi_simpleint_builderr
Fix/ndpi simpleint builderr
2020-08-13 10:41:25 +02:00
Luca Deri 8090765a64 Merge pull request #974 from IvanNardi/esni4
Suspicious ESNI usage: add a comment and a pcap example
2020-08-13 10:40:51 +02:00
Toni Uhlig aa856735c0 num_extra_packets_checked check can be 0 for some protocols and therefor requires lesser-or-equal condition for max_extra_packets_to_check
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-13 10:32:31 +02:00
Luca Deri 9edddee0b7 Fixes invalid detection on traffic on non standard ports 2020-08-12 11:08:28 +02:00
Luca Deri d5cac570d6 Improved DGA detection algoritm 2020-08-11 17:13:40 +02:00
Luca Deri 0e363d0ca6 Added HLL notes 2020-08-11 16:23:35 +02:00
aouinizied 8c2c388d54 Add Connectivity check category and blacklisted host risk. 2020-08-10 21:19:17 +02:00
aouinizied 8fcb49af25 Fix typo. 2020-08-10 21:17:36 +02:00
Luca Deri dfa9dd66c0 Added case-insensitive substring matching 2020-08-10 19:36:43 +02:00
Adrian Zgorzałek 56a3a33bc0 OpenBSD: Do not redefine __LITTLE_ENDIAN__
Will silence omnipresent compiler warnings when building ntopng.
2020-08-09 14:30:12 +01:00
Adrian Zgorzałek 8f74d5733d OpenBSD: Introduce pkt_timeval to deal with (bpf_)_timeval
Some BSD APIs called in example/ return `struct bpf_timeval`, where nDPI
APIs expect `struct timeval`. These two structs, besides having
a different name, share the exact same set of fields.
2020-08-09 14:30:12 +01:00
Toni Uhlig 4ead6d6594 travis-ci: build ndpiSimpleIntegration as well
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-08 12:17:54 +02:00
Toni Uhlig 8da5f42fa0 Changed ndpi_ssl_version2str function call in ndpiSimpleIntegration.
Fixes build error introduced with 23c072153.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-08 12:05:11 +02:00
Nardi Ivan 2722861d6e Suspicious ESNI usage: add a comment and a pcap example
See: 79b89d2866
2020-08-06 10:29:35 +02:00
Luca Deri 95dfbdc64a Merge pull request #973 from IvanNardi/esni3
Add risk flag about suspicious ESNI usage
2020-08-06 10:18:27 +02:00
Luca Deri c2156a5161 Added note on memory management 2020-08-06 09:19:04 +02:00
Luca Deri 5b6ffad278 Added new ndpi_string_sha1_hash API call 2020-08-05 21:45:38 +02:00
Luca Deri d3fb1fb25a Fixed possible memory leak in TLS certificate handling 2020-08-05 17:57:00 +02:00
Nardi Ivan 79b89d2866 Add risk flag about suspicious ESNI usage
In a Client Hello, the presence of both SNI and ESNI may obfuscate the real
domain of an HTTPS connection, fooling DPI engines and firewalls, similarly
to Domain Fronting.

Such technique is reported in a presentation at DEF CON 28:
"Domain Fronting is Dead, Long Live Domain Fronting: Using TLS 1.3 to evade
censors, bypass network defenses, and blend in with the noise"
Full credit for the idea must go the original author

At the moment, the only way to get the pdf presention and related video is via
https://forum.defcon.org/node/234492
Hopefully a direct link (and an example pcap) will be available soon
2020-08-05 17:13:23 +02:00
Luca Deri 07d9fa7f96 Win #define fix 2020-08-05 11:13:27 +02:00
Luca Deri 00b2763399 Added check on payload lenght during extra packet processing 2020-08-04 21:59:45 +02:00
Luca Deri e16675b700 Added new traffic category for connectivity check detection 2020-08-04 18:09:13 +02:00
Luca Deri ea10b8e757 Added memory checks 2020-08-02 13:00:31 +02:00
aouinizied 57e28e03ee Sync TLS definitions. 2020-08-01 01:27:29 +02:00
Luca Deri 2ae4c6675d Fixed partial TLS dissection 2020-07-30 18:30:07 +02:00
Luca Deri e71df49b3e Changed due to bin size extension 2020-07-30 00:06:46 +02:00
Luca Deri 6904935934 Restored TLS dissection 2020-07-30 00:06:35 +02:00
Luca Deri a828ac0191 Tiny changes for TLS block lenght dissection 2020-07-29 22:36:27 +02:00
aouinizied c4b19fd6c5 Update CFFI definitions. 2020-07-29 12:31:21 +02:00
Luca Deri 02dc8d5feb Merge pull request #970 from IvanNardi/fix
Minor fixes
2020-07-28 19:38:12 +02:00
Luca Deri 32bd3d7a59 TLS dissection improvements 2020-07-28 01:06:38 +02:00
Luca Deri da87cc3157 Added NDPI_SMB_INSECURE_VERSION for detecting insecure SMB versions (e.g. v1) 2020-07-27 13:05:06 +02:00
Luca Deri 69f140878c Boundary check on QUIC 2020-07-27 07:40:50 +02:00
Luca Deri 4b1cae2c2b Boundary check 2020-07-27 06:52:35 +02:00
Nardi Ivan 7d843f4d17 Minor fixes
Fix a memory leak and an issue (re)-introduced in configure script
2020-07-26 21:11:34 +02:00
Luca Deri 95263fc32e Added parentheses to avoid issues with macro expansion 2020-07-25 18:59:48 +02:00
Luca Deri 1c405e382a SSH code cleanup 2020-07-25 16:43:54 +02:00
Luca Deri b26539d65a Merge pull request #967 from MrRadix/dev
Ssh signature checking
2020-07-25 16:17:24 +02:00
MrRadix 6719fa30f9 added other ssh implementations to check 2020-07-24 20:53:02 +02:00
Luca Deri 21e2e57614 Fixed bin similarity 2020-07-24 01:30:58 +02:00
Luca Deri 439558f6a3 Improved bin clustering 2020-07-22 23:56:50 +02:00
Luca Deri 0956badd11 Merge pull request #968 from catenacyber/fuzzci
Reenables CI with sanitizers
2020-07-22 19:38:05 +02:00
MrRadix a9ca47fcdb added cipher check 2020-07-22 17:17:12 +02:00
MrRadix 1c1be5a0d1 Resolved conflicts on fetch 2020-07-22 16:25:01 +02:00
Philippe Antoine 2723ac70ce msan cxx 2020-07-22 16:09:13 +02:00
Luca Deri 3cd1ec5c9a Added changes for handlign SSSH cipher detection 2020-07-22 15:58:46 +02:00
Philippe Antoine a279423bff Reenables CI with sanitizers 2020-07-22 15:49:03 +02:00
MrRadix a3ba9253ef fixed bug inside set bit macro call 2020-07-22 15:39:44 +02:00
MrRadix c450caae94 modified new last two risks 2020-07-22 15:38:31 +02:00
Luca Deri 36af97a14c Minor HLL fixes 2020-07-22 14:40:15 +02:00
MrRadix af5d792c08 added sscanf error handling 2020-07-22 13:06:08 +02:00
MrRadix 8e2cd9ff43 improved performance and legibility 2020-07-22 12:42:26 +02:00
MrRadix 88dd3ebd62 added modified risks 2020-07-22 12:41:59 +02:00
MrRadix d9fc4d52e9 merged with remote 2020-07-22 11:56:51 +02:00
MrRadix a688e36b51 improved ndpi_risk2str output for new risks 2020-07-22 11:44:56 +02:00
Luca Deri f83d0b18c6 Introduced SSH rick checks 2020-07-22 11:40:57 +02:00
Luca Deri 66178e37cc Merge pull request #966 from catenacyber/ossfuzzbuild
Enables fuzz targets even without sanitizer
2020-07-22 11:36:48 +02:00
Luca Deri 1740042cbb Merge branch 'dev' into ossfuzzbuild 2020-07-22 11:36:31 +02:00
Luca Deri f6242f0e46 HLL memory usage notes 2020-07-22 11:28:14 +02:00
MrRadix 0a182c6d18 Merge remote-tracking branch 'ntop_origin/dev' into dev 2020-07-22 11:15:02 +02:00
MrRadix 53b2b08aeb added new risks to ndpi_risk_enum 2020-07-22 10:56:59 +02:00
MrRadix 9c521c5ddd added new risks iside ndpi_risk2str function 2020-07-22 10:55:53 +02:00
MrRadix 847eb7b180 improved performance by removing linear scan 2020-07-22 10:54:55 +02:00
Luca Deri d90b333363 Merge pull request #965 from lnslbrty/fix/autoconf-ubuntu
fixes autoconf issue on ubuntu 18 (#953)
2020-07-21 21:41:02 +02:00
MrRadix f66cd5aabc added ssh_analyse_signature_version and ssh_has_old_signature for check old signature version of ssh 2020-07-21 19:31:02 +02:00
MrRadix 4b4dcabeda added ssh obsolete version risks 2020-07-21 19:29:59 +02:00
Luca Deri 879cec94b2 User agent detection improvements 2020-07-21 12:06:34 +02:00
Luca Deri 9f4f228832 Added skeleton for checking SSH signature 2020-07-20 17:25:17 +02:00
Philippe Antoine b568c26989 Enables fuzz targets even without sanitizer
Should fix ossfuzz build
2020-07-20 10:07:54 +02:00
Toni Uhlig 29315cee6d fixes autoconf issue on ubuntu 18 (#953)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-19 20:30:41 +02:00
Luca Deri f452dabd92 Fix for invalid boundary check 2020-07-17 23:26:28 +02:00
Luca Deri 64fb40d21e Added ndpi_reset_data_analysis() API call 2020-07-17 13:16:01 +02:00
Luca Deri 6e1dcba9dd Indendentation fix 2020-07-16 17:56:37 +02:00
Luca Deri ebf18a0c61 Merge pull request #964 from yskcg/dev
1:add free pointer NULL check;2:fix xbox and teredo protocol detected…
2020-07-16 17:05:40 +02:00
ysk a268952068 remove the unused code 2020-07-16 17:26:46 +08:00
Luca Deri acd05b171e Added ndpi_data_last() API call 2020-07-16 11:22:06 +02:00
ysk 69ca98ca35 1:add free pointer NULL check;2:fix xbox and teredo protocol detected error when use the commm udp port 3544 2020-07-16 16:07:04 +08:00
Luca Deri 46e97d5da3 Added ndpi_data_window_variance() and ndpi_data_window_stddev() API calls 2020-07-15 18:13:06 +02:00
Luca Deri 9f3e3e8456 Add ndpi_hll_reset() API call
Fixes bug in ndpi_data_window_average() with zero points
2020-07-15 10:30:45 +02:00
Luca Deri 91d132ea3d Check for avoiding buffer overflow in netbios dissector 2020-07-14 22:16:43 +02:00
Luca Deri 3fd3153089 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2020-07-13 15:48:53 +02:00
Luca Deri a8ad99aca5 Fixed makefile error message
Code hardedning fix
2020-07-13 15:46:19 +02:00
aouinizied 7744c225f9 Add note for sync with cffi bindings. 2020-07-13 12:51:32 +02:00
aouinizied e2e0303099 Add ssl_version_str to flow structure. 2020-07-13 12:48:35 +02:00
Luca Deri 802254327a wrapper cleanup 2020-07-13 10:53:25 +02:00
Luca Deri 9d8717a0c7 Merge pull request #963 from yskcg/dev
add improved boundary check and check malloc return is NULL
2020-07-13 10:24:23 +02:00
Luca Deri 9d35364ef1 Merge pull request #961 from lnslbrty/fix/ssl-version2str-race-condition
Fixed race condition in ndpi_ssl_version2str() caused by static quali…
2020-07-13 10:23:47 +02:00
Luca Deri 051bdb1957 Merge pull request #960 from lnslbrty/fix/heap-overflow-in-llc-readear-util
Fixed heap overflow caused by missing lengthcheck in 802.11 LLC heade…
2020-07-13 10:23:06 +02:00
Toni Uhlig 96c193cf79 Fixed heap overflow caused by missing lengthcheck in 802.11 LLC header parsing.
* triggered by fuzz traces from wireshark

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-11 13:10:03 +02:00
Toni Uhlig 23c0721538 Fixed race condition in ndpi_ssl_version2str() caused by static qualifier in the version string buffer.
* added also GREASE supported tls versions as specified in
   https://tools.ietf.org/html/draft-davidben-tls-grease-01#page-4

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-11 01:05:39 +02:00
ysk 35f1c362b9 add improved boundary check and check malloc return is NULL 2020-07-10 17:49:35 +08:00
Luca Deri 12abcd516b Updated test results due to bin changes 2020-07-09 17:28:02 +02:00
Luca Deri 014fdd9a02 Various fixes in bins implementation
Added -b flag in ndpiReader to test bins
2020-07-09 17:28:02 +02:00
Luca Deri 9d8967236e Merge pull request #957 from lnslbrty/fix/build
Fix/build
2020-07-09 17:27:53 +02:00
Alfredo Cardigliano 6511db8941 Guide update 2020-07-09 15:37:02 +02:00
aouinizied b8d406b1f8 Update note. 2020-07-09 00:58:48 +02:00
aouinizied ce02359953 Add NDPI_MALFORMED_PACKET risk. 2020-07-09 00:56:44 +02:00
Toni Uhlig 24fea66a26 travis-ci cleanups
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-09 00:33:20 +02:00
Luca Deri d658ac9e61 Added -b to ndpiReader to evaluate bins 2020-07-09 00:15:21 +02:00
Toni Uhlig f75984633b ndpiSimpleIntegration should not be included in a static library
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-08 23:47:51 +02:00
Toni Uhlig 20fed83e0f Removed csv_fp as external symbol. Instead passing csv_fp through as argument.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-08 23:21:35 +02:00
Luca Deri f72b402aa2 Merge pull request #956 from IvanNardi/leak
Fix memory leak reported in #955
2020-07-08 17:15:13 +02:00
Luca Deri 4f794aec22 Merge pull request #955 from lnslbrty/fix/heap-overflow-thunder
Fixed thunder protocol detection heap overflow caused by missing leng…
2020-07-08 17:14:45 +02:00
Nardi Ivan fd6a5bc1cd Fix memory leak reported in #955
This leak is clearly happening because of missing/disabled code in "bins"
handling.
This is a trivial workaround useful until such code is completed.
In this way tavis-ci will stop complaining
2020-07-08 11:30:39 +02:00
Alfredo Cardigliano 5be9370cd2 Fix warnings 2020-07-08 09:15:48 +02:00
Toni Uhlig ccb30a04ad Fixed thunder protocol detection heap overflow caused by missing lengthcheck.
* triggered by fuzz traces from wireshark

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-08 00:21:00 +02:00
Luca Deri fb04dbbc47 Added testing bin code 2020-07-07 19:18:43 +02:00
Luca Deri f1dd8bc8c5 Memory allocation fixes 2020-07-07 17:32:26 +02:00
Luca Deri 0c80e6ee3b Merge pull request #954 from lnslbrty/fix/memleak-cluster-bins
fixed memory leak in ndpi_cluster_bins / binUnitTest
2020-07-07 17:18:03 +02:00
Toni Uhlig e308e59002 fixed memory leak in ndpi_cluster_bins / binUnitTest
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-07 17:07:20 +02:00
Luca Deri 1c60c22893 Added ndpi_cluster_bins() for clustering bins and ancillary functions for bins manipulation 2020-07-07 15:10:51 +02:00
Luca Deri db707e0829 Merge pull request #932 from IvanNardi/log
Log
2020-07-07 14:43:32 +02:00
Luca Deri 540326f3ff Merge pull request #953 from lnslbrty/fix/broken-autoconf-fsanitizer
fix broken sanitizer CFLAGS, --enable-fuzztargets should also set -fs…
2020-07-07 14:42:42 +02:00
Luca Deri 94bf7b0130 Merge pull request #952 from lnslbrty/fix/missing-lengthcheck-in-reader-uutil
Fixed heap overflow caused by missing lengthcheck in reader uutil.
2020-07-07 14:42:20 +02:00
Nardi Ivan 030f3f3d48 Fix a memory leak 2020-07-07 10:27:57 +02:00
Toni Uhlig de25ba7d0b Fixed heap overflow caused by missing lengthcheck in reader uutil.
* triggered by fuzz traces from wireshark

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-06 23:30:40 +02:00
Toni Uhlig 9bbcc2548e fix broken sanitizer CFLAGS, --enable-fuzztargets should also set -fsanitize
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-06 22:09:11 +02:00
lucaderi 055422503c configure fixes 2020-07-06 10:38:29 +02:00
Luca Deri c0b34555c2 Merge pull request #951 from lnslbrty/fix/ossfuzzer
ossfuzz.sh: do not use wildcards for fuzzer e.g. fuzz/fuzz*
2020-07-06 10:23:50 +02:00
Luca Deri b4edb75824 Merge pull request #950 from lnslbrty/improved/http-line-parsing
Improved HTTP line parsing if request splitted into multiple packets.
2020-07-06 10:23:29 +02:00
Luca Deri dfb9e8ec1f Merge pull request #940 from lnslbrty/fix/small-and-optimistic-improvments
Fixed CodeInspector issues.
2020-07-06 10:23:04 +02:00
Toni Uhlig c17a21359b Fixed CodeInspector issues.
* Added compiler search list for AC_PROG_CC, AM_PROG_CC_C_O, AC_PROG_CXX, AC_PROG_CC_STDC
   for Mac OS X only
   The list rendered the AX_CHECK_COMPILE_FLAG([-fsanitize=fuzzer]) useless as it did use
   clang for AX_CHECK_COMPILE_FLAG but gcc during the compile process. Seems broken somehow.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-05 20:41:02 +02:00
Toni Uhlig 6e8f268873 ossfuzz.sh: do not use wildcards for fuzzer e.g. fuzz/fuzz*
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-05 20:18:16 +02:00
Toni Uhlig 4b8c8608d1 Improved HTTP line parsing if request splitted into multiple packets.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-05 18:36:57 +02:00
Luca Deri 503e3bb495 Added comment on stddev 2020-07-03 14:45:58 +02:00
Luca Deri 513c857631 Added unit test 2020-07-03 14:43:14 +02:00
Luca Deri f7b91b46fa Merge pull request #947 from lnslbrty/add/download-script-wireshark-traces
added shell script to download wireshark traces, can be used in combi…
2020-07-03 00:45:45 +02:00
Luca Deri 143fcf0b0e Merge pull request #946 from lnslbrty/fix/teamviewer-iprange
Improved TeamViewer IP range (fixes #944)
2020-07-03 00:45:11 +02:00
Toni Uhlig ae89831055 added shell script to download wireshark fuzzing traces, can be used in combination with ./tests/do.sh
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-02 20:22:58 +02:00
Toni Uhlig 8f53c231df Improved TeamViewer IP range (fixes #944)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-02 18:11:39 +02:00
Nardi Ivan 974c1cc681 Improve help message of --dbg-proto option
Make it clear that such option is general, not about extcap
functionality
2020-07-02 09:42:53 +02:00
Nardi Ivan c08693fda5 Incorporated some feedback 2020-07-01 20:16:16 +02:00
Luca Deri 08698c65e5 Merge pull request #943 from lnslbrty/fix/missing-lengthcheck-in-tls-esni
Fixed heap overflow in tls esni extraction triggered by manipulated p…
2020-07-01 12:37:29 +02:00
Luca Deri 2d68f8f635 Merge pull request #942 from lnslbrty/improved/fuzzer-automake-dependency-tracking
Added fuzz_*_LDADD to give Automake the opportunity to do some depend…
2020-07-01 12:37:06 +02:00
Luca Deri 2860e3af15 Merge pull request #941 from lnslbrty/add/ossfuzz-script
Copy&Paste ./tests/ossfuzz.sh from https://github.com/google/oss-fuzz…
2020-07-01 12:36:42 +02:00
Luca Deri be35d423b4 Make clean enhancement 2020-07-01 12:34:28 +02:00
Luca Deri cf9c0b96b8 Indent 2020-06-30 16:25:53 +02:00
Alfredo Cardigliano ea6332e004 Fix segfault on ndpi_guess_protocol_id with flow = null 2020-06-30 15:54:52 +02:00
Luca Deri 392ce44573 Added ndpi_print_bin() API call 2020-06-29 22:36:46 +02:00
Toni Uhlig 05d7400563 Fixed heap overflow in tls esni extraction triggered by manipulated packets.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-29 21:51:46 +02:00
Toni Uhlig ea30fcb5de Added fuzz_*_LDADD to give Automake the opportunity to do some dependency tracking.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-29 21:27:03 +02:00
Toni Uhlig 3068306b60 Copy&Paste ./tests/ossfuzz.sh from https://github.com/google/oss-fuzz/pull/4041
* Changing the OSS-Fuzz script from our side is much more easier then
   opening a PR to google/oss-fuzz every time we have to change a single line.
 * https://github.com/google/oss-fuzz/pull/4041 will be updated once this PR is merged

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-29 20:50:09 +02:00
Luca Deri 0d2d44f1b6 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2020-06-29 19:51:36 +02:00
Luca Deri 07615903e2 Merge pull request #939 from lnslbrty/improved/fuzzy-reader-smp
Improved fuzz_ndpi_reader which supports now SMP/MT w/o race-coniditi…
2020-06-29 19:15:46 +02:00
Luca Deri 2c263bc726 Added ndpi_bin_similarity() for computing bin similarity 2020-06-29 19:09:38 +02:00
Toni Uhlig 4a6c525db8 Improved fuzz_ndpi_reader which supports now SMP/MT w/o race-coniditions.
./tests/do.sh can supports SMP/MT via environment variables.
Removed -fno-sanitize=shift as well, was fixed by 317d3ffd.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-29 19:07:59 +02:00
Luca Deri 93a4a4387a Merge pull request #938 from IvanNardi/errors
Errors
2020-06-28 21:32:08 +02:00
Nardi Ivan f39e3c98f0 Update test results 2020-06-28 12:47:27 +02:00
Nardi Ivan 317d3ffd3e Fix undefined behaviour in internal tests
Error messages:
ndpiReader.c:3211:2: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
ndpiReader.c:3207:5: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'

The errors started popping up since 3d9285f1
2020-06-28 12:13:21 +02:00
Nardi Ivan b24f5c4c0a Fix memory leak about purged/expired flows
Create an helper to avoid similar errors in the future
Fixes: 1a62f4c7
2020-06-28 12:05:12 +02:00
Nardi Ivan d6a97219ea Fix use-after-free in http content parsing 2020-06-28 12:05:12 +02:00
Nardi Ivan 3669c14afd DNP3: add missing initialization 2020-06-28 12:05:12 +02:00
Nardi Ivan ece5d3e199 Fix (harmless) memory leaks when DPDK is enabled 2020-06-28 12:05:12 +02:00
Nardi Ivan 2cdf7ce806 Fix startup when DPDK is enabled
Fixes:a58c838c4
2020-06-28 12:05:12 +02:00
Nardi Ivan 2effa57d8a ndpiReader: fix ports statistics
They should take idle/expired flows into account, too
2020-06-28 12:05:12 +02:00
Nardi Ivan b68b45f3bb TLS: extract JA3 signatures in some corner cases
In some (rare) cases, Client Hello message contains lots of cipher
suits.
2020-06-28 12:05:12 +02:00
Luca Deri b2e89e6736 Merge pull request #937 from lnslbrty/re-add-corpus-zipfile-for-google-oss-fuzz
Re-Added corpus zipfile generation required for google-oss fuzzer
2020-06-28 07:29:45 +02:00
Luca Deri 1010520cc4 Merge pull request #936 from lnslbrty/fix/h323-off-by-one
Fixed off-by-one error in h323.
2020-06-28 07:28:55 +02:00
Toni Uhlig f028cce209 Re-Added corpus zipfile generation required for google-oss fuzzer
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-27 23:04:20 +02:00
Toni Uhlig fbfa54eee6 Fixed off-by-one error in h323.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-27 22:58:05 +02:00
Luca Deri ff67e09101 Restored compiler check for fuzzy targets 2020-06-27 19:34:57 +02:00
Luca Deri 76e6c38459 Use clang where available and fallback to gcc when missing 2020-06-27 19:15:52 +02:00
Luca Deri 243104a8c2 Merge pull request #935 from lnslbrty/fix/fuzzing-tests
fixed fuzzing tests in a way that ./tests/do.sh is now able to use co…
2020-06-27 18:57:56 +02:00
Toni Uhlig 1f4523e331 fixed fuzzing tests in a way that ./tests/do.sh is now able to use corpus *.pcap files from ./tests/pcap
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-27 18:41:03 +02:00
Luca Deri b5f3facf7c Added notes whenever a new flow risk is added 2020-06-26 23:39:48 +02:00
Luca Deri 8566288e43 Added malformed packet risk support 2020-06-26 22:37:52 +02:00
Luca Deri 8de62c6d34 Values stored in patricia tree are now 32 bit (they used to be 16 bit) long 2020-06-26 17:49:45 +02:00
Nardi Ivan 0cd0905400 mqtt: add boundary check in log message 2020-06-26 12:04:02 +02:00
Nardi Ivan 56d87186f7 Fix compilation with --enable-debug-messages flag
NDPI_LOG* macros dereference ndpi_detection_module_struct object which is
private to ndpi library (via NDPI_LIB_COMPILATION define). So we can't use
them outside the library itself, i.e. in ndpiReader code
Therefore, in files in example/, convert all (rare) uses of NDPI_LOG* macros
to a new very simple macro, private to ndpiReader program. If necessary,
such macro may be improved.

According to a comment in ndpi_define.h, each dissector must define its own
NDPI_CURRENT_PROTO macro before including ndpi_api.h file
2020-06-26 12:04:02 +02:00
lucaderi ebfb5e7f01 Check for CentOS 6 2020-06-26 09:01:58 +02:00
Luca Deri d805611fcb Merge pull request #931 from lnslbrty/fix/autogen.sh
fixed autoconf version check
2020-06-25 23:31:10 +02:00
Luca Deri 9742dcc45c Merge pull request #930 from IvanNardi/extcap
Extcap
2020-06-25 23:30:20 +02:00
Toni Uhlig f38a8bd7a5 fixed autoconf version check
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-25 20:02:23 +02:00
Nardi Ivan 3c66ca236b Make lua script more robust 2020-06-25 18:02:03 +02:00
Nardi Ivan 70a926088f Fixed harmless memory leak in extcap initialization 2020-06-25 18:02:03 +02:00
Nardi Ivan 6b69e7ebbc Restore extcap functionality, i.e. integration with wireshark 2020-06-25 18:02:03 +02:00
Luca Deri 0b53bc2aab Disabled unused code 2020-06-25 10:50:39 +02:00
lucaderi ecdf7df454 Compilation fixes for non-Linux (or outdated Linux) platforms 2020-06-25 10:25:24 +02:00
lucaderi 3034e53d00 CentOS6 fix (santize won't work as too old system)
Fixes warning
2020-06-25 10:22:16 +02:00
Luca Deri 2350daa2dc Merge pull request #928 from lnslbrty/added/yet-another-ndpi-integration-example
PROPOSAL: ndpiSimpleIntegration: added another integration example
2020-06-25 10:05:49 +02:00
Luca Deri d710b8291d Merge pull request #927 from lnslbrty/fix/fbzero-missing-length-check
Fixed missing length check in fbzero.
2020-06-24 22:17:35 +02:00
Luca Deri 485e56d855 Merge pull request #926 from lnslbrty/added/hulustream-sni-pattern
Added hulustream SNI pattern. Fixes #904.
2020-06-24 22:17:09 +02:00
Luca Deri 38c0d927a9 Merge pull request #925 from lnslbrty/fix/ndpi-rader-unitialized-values
Fixed unitialized values in ndpiReader protocol detection bitmask dur…
2020-06-24 22:16:48 +02:00
Luca Deri 94a68451df Merge pull request #924 from lnslbrty/improved/autoconf-sanitizer-compile-flag-checks
Allow -fsanitize for LLVM as well as for GCC.
2020-06-24 22:15:11 +02:00
Toni Uhlig 17c26911fb ndpiSimpleIntegration: added another integration example
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-24 22:03:18 +02:00
Toni Uhlig ca68beda85 Fixed missing length check in fbzero.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-23 18:35:50 +02:00
Toni Uhlig 2f5766d2f5 Added hulustream SNI pattern. Fixes #904.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-23 17:30:37 +02:00
Toni Uhlig 39800c88fa Fixed unitialized values in ndpiReader protocol detection bitmask during dga selftest.
* make ./tests/vagrind_test.sh directory agnostic

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-23 17:23:56 +02:00
Toni Uhlig 678fbcdfd6 Allow -fsanitize for LLVM as well as for GCC.
* renamed --with-llvm-sanitizer to --with-sanitizer
 * disable all optimisations (-O0) if --with-sanitizer set,
   no functions/paramaters/variables will be optimised, improves debugging
 * enable undefined behaviour sanitizer (ubsan)
 * enable leak sanitizer (lsan)
 * check if -fsanitize=fuzzer is available and --enable-fuzztargets set,
   fail if not (only supported by clang)
 * include level 3 debugging information (-g3), improves macro debugging
 * disabled ubsan misaligned pointer access and lshift overflow

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-23 00:22:33 +02:00
Luca Deri d9af1562f0 Fixes #906
Packet bins are not printed wehn empty
2020-06-22 14:30:26 +02:00
Luca Deri e29e14b7aa Merge pull request #921 from lnslbrty/improved/test-sh
make tests/do.sh directory agnostic
2020-06-22 12:51:17 +02:00
Luca Deri c9d8efc2a4 Warning fix 2020-06-22 10:20:05 +02:00
Luca Deri 1a62f4c799 Added ndpi_bin_XXX API
Added packet lenght distribution bins
2020-06-22 01:02:54 +02:00
Luca Deri 2a137100da Merge pull request #922 from lnslbrty/fix/http-content-disposition-crash
Fixed use after free caused by dangling pointer
2020-06-22 00:57:15 +02:00
Toni Uhlig 6a9f5e4f7c Fixed use after free caused by dangling pointer
* This fix also improved RCE Injection detection

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-21 20:05:38 +02:00
Luca Deri fd0591b4fc Merge pull request #920 from lnslbrty/fix/tls-rdn-crash
Fixed stack overflow caused by missing length check
2020-06-19 11:44:37 +02:00
Luca Deri 48758d28ea Added GoogleDNS DoH on Android 10 2020-06-19 09:55:58 +02:00
Toni Uhlig 14f514134d make tests/do.sh directory agnostic
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-19 07:15:20 +02:00
Luca Deri 63670927e7 Fixed API documentation: packet tiestamp is expressed in milliseconds 2020-06-18 14:15:18 +02:00
Toni Uhlig 23594f0365 Fixed stack overflow caused by missing length check
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-18 00:52:04 +02:00
Luca Deri b2c24558c5 DGA detection improvements 2020-06-18 00:17:30 +02:00
Luca Deri 46d96e7f32 Added checks for DGA detection 2020-06-17 19:46:37 +02:00
Luca Deri ffd20ebdf9 Merge pull request #919 from lnslbrty/new/protocol/anydesk
Implemented proprietary AnyDesk protocol
2020-06-17 06:47:35 +02:00
Toni Uhlig da37f2444f Implemented proprietary AnyDesk protocol
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-06-17 01:23:03 +02:00
Alfredo Cardigliano 073e3a027b Add API ndpi_serializer_get_format 2020-06-16 18:40:17 +02:00
Luca Deri 0ddc3a0052 Fixed invalid assignment (typo) 2020-06-14 09:37:56 +02:00
Luca Deri 38e4910636 Added improved boundary check 2020-06-14 09:29:52 +02:00
aouinizied 4247dd8322 Synchronize type definitions. 2020-06-12 22:58:58 +02:00
Luca Deri 55364ef0b4 Added DGA risk for names that look like a DGA 2020-06-11 18:51:53 +02:00
Luca Deri 60aaa80570 Added HyperLogLog cardinality estimator API calls
/* Memory lifecycle */
int ndpi_hll_init(struct ndpi_hll *hll, u_int8_t bits);
void ndpi_hll_destroy(struct ndpi_hll *hll);

/* Add values */
void ndpi_hll_add(struct ndpi_hll *hll, const char *data, size_t data_len);
void ndpi_hll_add_number(struct ndpi_hll *hll, u_int32_t value) ;

/* Get cardinality estimation */
double ndpi_hll_count(struct ndpi_hll *hll);
2020-06-10 23:43:35 +02:00
Luca Deri 0b9e46e65a Updated (C) 2020-06-08 23:11:20 +02:00
Luca Deri 64fab4a478 Win fixes 2020-06-08 23:08:12 +02:00
Luca Deri 3506a07864 Added check in TLS 1.2+ for reporting a risk when TLS is not used to carry HTTPS 2020-06-08 14:20:10 +02:00
Luca Deri d318285cae Added HTTP check 2020-06-07 09:46:27 +02:00
Luca Deri 1479c0a278 Added TLS bounadry check 2020-06-07 09:25:19 +02:00
Luca Deri 43ddbfdfba Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2020-06-06 11:30:47 +02:00
Luca Deri 801c9481cb Removed some obsolete protocols (battlefield, oscar, pcanywhere, tvants) 2020-06-06 11:29:03 +02:00
Alfredo Cardigliano 2beecdc8af Handle EOR in TLV test 2020-06-06 11:18:17 +02:00
Luca Deri 605d548d4a removed obsolete yahoo plugin 2020-06-06 09:38:19 +02:00
Alfredo Cardigliano 0da76fdac6 Support for multiple records in CSV serialization 2020-06-05 17:28:27 +02:00
Luca Deri 597d6e5d60 Merge pull request #913 from yskcg/fix_segment_fault_dev
Fix segment fault dev
2020-06-05 16:57:46 +02:00
Alfredo Cardigliano 67c72a8cb5 Add ndpi_serializer_get_header API (CSV only) 2020-06-05 16:52:26 +02:00
Luca Deri 07c54f9992 Added ndpi_dpi2json() API call 2020-06-05 16:08:23 +02:00
Luca Deri 16ca69733a Improved teamviewer detection 2020-06-03 10:14:27 +02:00
ysk 40550073ef fix segment fault cause by the ssl.server_names when it may NULL 2020-06-03 11:19:37 +08:00
ysk 52893d99f6 fix segment fault cause by the ssl.server_names when it may NULL 2020-06-03 10:44:35 +08:00
ysk 86a3dc5bea Merge branch 'dev' of github.com:yskcg/nDPI into dev 2020-06-03 10:41:40 +08:00
Luca Deri 5afa3ad818 Improved spotify detection 2020-06-02 15:00:14 +02:00
Luca Deri af2a44ae5f Added boundary check in kerberos protocol 2020-06-01 18:22:10 +02:00
Luca Deri 241af016e9 Added check for spotify payload lenght 2020-06-01 16:36:12 +02:00
Luca Deri 16dc0ec29e Check to avoid allocating twice the esni memory 2020-06-01 16:34:24 +02:00
Luca Deri 29d9c2d2c8 Reworked HTTP header processing workflow 2020-06-01 09:54:05 +02:00
Luca Deri 669b7579ff HTTP comparison optimization 2020-05-31 19:31:01 +02:00
Luca Deri a4f8d913ea Minor changes 2020-05-31 19:11:12 +02:00
Luca Deri dc5b4cbde0 Reworked ndpi_strncasestr 2020-05-31 18:50:25 +02:00
Luca Deri b0bf66c436 Added offset check in kerberos dissector 2020-05-31 08:30:55 +02:00
Luca Deri c793e16050 Added extra TLS memory boundary checks 2020-05-31 08:28:02 +02:00
Luca Deri b6eef17e54 Added check to avoid producing alerts for known protocol on unknown port when using TLS 2020-05-30 19:33:13 +02:00
Luca Deri 374a24a1fb HTTP dissector improvements 2020-05-30 18:40:15 +02:00
Luca Deri cbb0b864a2 Minor cleanup 2020-05-29 21:51:51 +02:00
Luca Deri 61066fb106 Added check for heap buffer overflow read 2020-05-29 21:43:06 +02:00
Luca Deri 4ceff1dc77 Fixes for https://github.com/ntop/nDPI/pull/911
Added code for dumping invalid HTTP header
2020-05-29 21:23:46 +02:00
Luca Deri 0271e29097 Added references to ESNI 2020-05-29 21:23:34 +02:00
Luca Deri fcb50dec70 Merge pull request #911 from tmacchioni/dev
Added check for suspicious HTTP headers
2020-05-29 21:14:39 +02:00
Tommaso Macchioni db5cd92fe1 Added check for suspicious HTTP headers 2020-05-29 15:55:52 +02:00
Luca Deri 3085d8e4ff Refreshed test pcap 2020-05-28 21:23:02 +02:00
Luca Deri 9c3bfeca80 Added support for Encrypted TLS SNI dissection
https://datatracker.ietf.org/doc/draft-ietf-tls-sni-encryption/
2020-05-28 17:44:18 +02:00
Luca Deri f6038c358a Added ndpi_serialize_risk() API function 2020-05-27 17:35:11 +02:00
Luca Deri 3108c75059 Result update 2020-05-27 15:26:30 +02:00
Luca Deri 811d7a39b5 Added pcap with encrypted SNI
- https://blog.cloudflare.com/encrypted-sni/
- https://www.inmotionhosting.com/support/website/security/dns-over-https-encrypted-sni-in-firefox/
2020-05-27 15:00:55 +02:00
Luca Deri 030e9dddb7 Extended the cache for services that need to be reconciled such as Microsoft Teams
Added JSON-formatted Microsoft list of IP/services
2020-05-27 12:40:35 +02:00
Alfredo Cardigliano 2dce6cd525 Add ndpi_serialize_start_of_list/ndpi_serialize_end_of_list to serialize simple lists in JSON 2020-05-25 16:37:00 +02:00
Luca Deri 8fa5c31996 Added ndpi_serialize_risk() to the nDPI API 2020-05-24 08:54:38 +02:00
Luca Deri ee35834be6 Added flow risk serilization 2020-05-24 08:46:10 +02:00
aouinizied e5c2c400ef Update APIs and structures. 2020-05-22 23:48:54 +02:00
Luca Deri bbbc5fdbae Added memory boundary checks 2020-05-22 07:24:02 +02:00
Luca Deri 3874f0e0e0 Added stub for checking HTTP header
Updated Teams result
2020-05-21 15:19:55 +02:00
ysk 83ae5b7256 nDpi#remove build warnings 2020-05-21 14:06:52 +08:00
Luca Deri c02b00e0ce MS Teams uses as underlying protocol for voice/video. This commit adds the ability
to mark as MS Teams all Skype traffic made by a host with active MS Teams flows
2020-05-21 00:06:22 +02:00
Luca Deri 07d9e4f9ba Fixed valse positive whatsapp detection
Cleaned Microsoft IP addresses list
2020-05-20 23:28:21 +02:00
Luca Deri e9519c9aca Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2020-05-20 23:14:46 +02:00
Luca Deri 329eff069e Removed bittorrent false positive detection 2020-05-20 23:14:09 +02:00
Luca Deri 246ec53bcf Merge pull request #905 from lnslbrty/dev
Fixed docstring typos for ndpi_finalize_initalization
2020-05-20 08:48:49 +02:00
ysk cd65da69d3 Merge branch 'dev' of github.com:yskcg/nDPI into dev 2020-05-20 13:21:36 +08:00
ysk 068c1b1f22 someip#fix when run across architectures such ARM ,may be occour sigbus error(),Unhandled fault: alignment fault (0x92000021) at 0x00000000f67004aa 2020-05-20 13:15:42 +08:00
Luca Deri b7e666e465 Added fix to avoid potential heap buffer overflow in H.323 dissector
Modified HTTP report information to make it closer to the HTTP field names
2020-05-19 08:31:05 +02:00
Toni Uhlig c5e16b4ff7 Fixed docstring typos for ndpi_finalize_initalization
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-05-17 16:59:57 +02:00
Luca Deri 3d9285f1be Added check for invalid HTTP URLs 2020-05-16 00:10:35 +02:00
Luca Deri c375782b96 Added check for binary scripts
Added NDPI_HTTP_NUMERIC_IP_HOST risk
ndpi_risk moved to 32 bit
2020-05-15 22:49:55 +02:00
Luca Deri 8e7b1ea7a1 Fix for potential heap-buffer-overflow in ndpi_search_openvpn 2020-05-15 21:10:37 +02:00
Luca Deri e90c5c7c32 Added NDPI_HTTP_SUSPICIOUS_USER_AGENT ndpi_risk 2020-05-15 19:19:17 +02:00
Luca Deri 7dfbfff743 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2020-05-15 18:58:10 +02:00
Luca Deri da22aa5fc7 Added NDPI_TLS_CERTIFICATE_EXPIRED, NDPI_TLS_CERTIFICATE_MISMATCH, to ndpi_risk 2020-05-15 18:57:49 +02:00
Luca Deri a183b2ab9b Added link to nfstream 2020-05-15 18:11:59 +02:00
Luca Deri adfe6b763c Gotomeeting address range fix 2020-05-15 11:20:21 +02:00
Luca Deri 9ed94a722c Improvements on GotoMeeting
Added pcap for testing malware
2020-05-15 10:52:23 +02:00
Luca Deri 7037e79604 Merge pull request #903 from Loures/dev
Extend packet struct with Content-Disposition HTTP header field
2020-05-14 20:56:07 +02:00
loures 1edf5c49d6 Extend filetype matching for Content-Disposition header 2020-05-14 14:30:34 +02:00
loures baddfbb6c3 Extend packet struct with Content-Disposition HTTP header field
and improve HTTP binary transfer mime type check
2020-05-14 12:47:22 +02:00
Luca Deri fb64346e28 Code clenup for PR #902 2020-05-14 09:49:47 +02:00
Luca Deri 50be0b0049 Merge pull request #902 from Loures/dev
Add check for HTTP transfer of executable files
2020-05-14 09:12:17 +02:00
loures 08f32f2e0e Set risk field instead of changing protocol when checking for dangerous
HTTP traffic
2020-05-13 19:56:49 +02:00
loures 0a4fbb8cfb Add check for HTTP transfer of executable files 2020-05-13 18:32:29 +02:00
Luca Deri 9dfd0d0071 Code cleanup 2020-05-13 17:49:02 +02:00
Luca Deri 17235d234e Merge pull request #897 from catenacyber/fuzzoracle
Adds bound check in oracle protocol
2020-05-12 13:53:55 +02:00
Luca Deri db16a987c5 Merge pull request #896 from IvanNardi/nats
Fix NATS dissector
2020-05-12 13:52:42 +02:00
Luca Deri f7f705406f Invalid TLS check 2020-05-12 12:25:07 +02:00
Luca Deri cd765c08d6 Added NDPI_XXX_BIT_16 2020-05-11 09:07:30 +02:00
Luca Deri ee15c6149d Added TLS weak cipher and obsolete protocol version detection 2020-05-10 21:55:35 +02:00
Luca Deri ae803c8b51 Added detection of self-signed TLS certificates 2020-05-10 21:40:35 +02:00
Luca Deri e5e69d0f7a Added the ability to detect when a known protocol is using a non-standard port
Added check to spot executables exchanged via HTTP
2020-05-10 21:25:38 +02:00
Philippe Antoine b69177be2f Adds bound check in oracle protocol
Found by oss-fuzz
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=21780
2020-05-10 15:04:23 +02:00
Nardi Ivan f8503da9e9 Fix NATS dissector 2020-05-09 15:41:20 +02:00
Luca Deri 39ae57e6a3 Cleaned hyperscan leftover
Added further hyperscan hooks
2020-05-08 18:24:07 +02:00
Luca Deri c9b37b92f5 Added self signed certificate test pcap 2020-05-08 09:09:58 +02:00
Luca Deri 4a09b4efa0 Added TLS issuerDN and subjectDN 2020-05-07 18:44:51 +02:00
Luca Deri fd646bd05f Reworked TLS dissector with a certificate RDN sequence reader 2020-05-07 17:22:01 +02:00
Luca Deri 6a1b8baa00 Fixed category matching 2020-05-06 23:50:35 +02:00
Luca Deri 2ccd2c204b API cleanup for indetifying explicitly in automa's what we're searching (protocol or category)
Removed hyperscan support that is apperently unused
2020-05-06 23:19:59 +02:00
Luca Deri 263547e77d Updated automa API to use 32 bit values splits from protocol/categpry 2020-05-06 21:57:32 +02:00
Luca Deri 86e34fbf6d Added support for Telegram v6 2020-05-06 18:06:30 +02:00
Luca Deri 84f66b4d6b Introduced custom protocols with IP and (optional) port support
Example

- Single IP address
  ip:213.75.170.11@CustomProtocol

- IP address with CIDR
  ip:213.75.170.11/32@CustomProtocol

- IP address with CIDR and port
  ip:213.75.170.11/32:443@CustomProtocol

Please note that there are some restrictions on the port
usage. They have been listed in example/protos.txt
2020-05-06 12:51:44 +02:00
Luca Deri 7855e0318d Various fixes to patricia tree handling 2020-05-06 11:13:57 +02:00
Luca Deri 48282369e2 False positive fixes 2020-05-06 01:34:55 +02:00
Luca Deri 7d63149ced Updated API 2020-05-06 00:41:07 +02:00
Luca Deri 427002d14f Reworked protocol handling chnging it is u_int16_t 2020-05-06 00:31:40 +02:00
Luca Deri 4148c5e065 Removed now obsolete MSN protocol
Added nats.io protocol dissector
2020-05-03 18:20:21 +02:00
Luca Deri c120cca662 SSH boundary check rework 2020-04-30 20:08:03 +02:00
Luca Deri 7344bf65bc Minor cleanup 2020-04-30 19:58:07 +02:00
Luca Deri 6f79b78f71 Merge pull request #891 from catenacyber/ghsl2
Better fix for integer overflow in SSH
2020-04-30 19:53:16 +02:00
Philippe Antoine a70fd6ed3b Better fix for integer overflow in SSH
Credits to GHSL
2020-04-30 18:44:40 +02:00
Luca Deri bd0fd6cf8d Merge pull request #883 from leonn/websocket
💡 implement WebSocket protocol dissector
2020-04-27 23:28:23 +02:00
Luca Deri 0bf809f8e5 Merge pull request #884 from lnslbrty/dev
Added missing ndpiReader dependency for the install target
2020-04-27 23:27:26 +02:00
Luca Deri 44259ab2b9 Merge pull request #887 from IvanNardi/tunnel
Tunnel
2020-04-27 23:27:03 +02:00
Alfredo Cardigliano 328016895f Handle empty blocks 2020-04-27 13:34:54 +00:00
Leonn Paiva 780dc8d1e7 💡 implement websocket protocol dissector 2020-04-26 02:53:12 -03:00
Alfredo Cardigliano 4d746dabe9 Enable IEC 60870-5-104 dissector 2020-04-24 15:31:44 +00:00
Nardi Ivan 097127c31d Fix heap-overflow error in CAPWAP detunneling code 2020-04-24 10:42:52 +02:00
Nardi Ivan c2ebbb15ad Fix "division by zero" runtime error 2020-04-23 14:24:49 +02:00
Nardi Ivan f965983c23 Add basic support for some ip-in-ip tunnels
Add support for 4in4, 6in6 and 4in6 encapsulations
Add support for ipv6 traffic in gtp tunnels, too

To allow gtp unit test, gtp detunneling flag has been globally enabled
in the test suite
2020-04-23 10:55:33 +02:00
Toni Uhlig 8d8e61b256 Added missing ndpiReader dependency for the install target
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-04-22 21:51:57 +02:00
Luca Deri 0b702c20d3 Warning fix 2020-04-22 18:52:36 +02:00
Luca Deri 473a5d1d0b Merge pull request #881 from leonn/bug_protocol_was_guessed
Bug protocol was guessed
2020-04-22 18:49:55 +02:00
Luca Deri 65b428899d Fix for nDPI code not downlooaded from git 2020-04-22 16:15:15 +02:00
Leonn Paiva 6769168d89 🐛 use protocol_was_guessed in ndpi_detection_giveup 2020-04-22 03:48:12 -03:00
Leonn Paiva 4d5ca2c473 🆒 code ident & style 2020-04-22 03:40:55 -03:00
Luca Deri e8cae3cff9 Updated results 2020-04-21 19:26:49 +02:00
Luca Deri 019b51bb17 Merge pull request #879 from IvanNardi/warnings
Fix some compilation warnings
2020-04-21 19:23:57 +02:00
Luca Deri e2be7daed6 Merge pull request #880 from catenacyber/fuzz8fix
Gets right protocol after IPv6 header
2020-04-21 19:22:50 +02:00
Philippe Antoine 1b73f7372e Gets right protocol after IPv6 header 2020-04-21 15:34:53 +02:00
Nardi Ivan b1a6c6b895 Fix some compilation warnings 2020-04-20 16:53:39 +02:00
Luca Deri e603549967 Office365 renamed to Microsoft365 (by Microsoft) 2020-04-20 15:44:27 +02:00
Luca Deri 3bdec5ae5f Updated node list (fixes #766) 2020-04-20 15:16:31 +02:00
Luca Deri 25cd2a23a6 Compilation fixes 2020-04-20 15:08:51 +02:00
Luca Deri 97badc574b Merge pull request #877 from catenacyber/fuzz7fix
Adds bound check before calling ndpi_handle_ipv6_extension_headers
2020-04-20 13:56:53 +02:00
Luca Deri 493bffb3e0 Added check for pkg-config presence 2020-04-19 21:48:24 +02:00
Philippe Antoine c2b2692e65 Seeting right flow protocol after IP6 extensions
Finally fixing https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20727
2020-04-18 14:39:57 +02:00
Philippe Antoine db5f3b38b7 Adds CIFuzz to check PRs
cf https://google.github.io/oss-fuzz/getting-started/continuous-integration/
2020-04-18 10:50:56 +02:00
Philippe Antoine da0889d3ba Adds bound check before calling ndpi_handle_ipv6_extension_headers 2020-04-17 22:21:03 +02:00
Luca Deri 94eee66a7b CiscoVPN dissection improvements 2020-04-17 16:56:55 +02:00
Luca Deri 748df7972a Results update 2020-04-17 10:36:27 +02:00
Luca Deri a766de7b94 Merge pull request #876 from a-czyrny/dev
additional csv semicolon fix
2020-04-17 10:34:41 +02:00
Luca Deri 711ba99eaa Added detection of Microsoft Teams 2020-04-16 15:23:07 +02:00
Alexander Czyrny 32d25bfdaf additional csv semicolon fix
Created function correct_csv_data_field to pevent duplicated code. Additionally used for _flow->ndpi_flow->protos.stun_ssl.ssl.alpn_ and _flow->ndpi_flow->protos.stun_ssl.ssl.tls_supported_versions_ to guarantee a valid csv output (commas replaced by semicolon) .
2020-04-16 14:12:48 +02:00
Luca Deri 287015ffdc Fixed invalid check 2020-04-15 22:42:09 +02:00
Luca Deri f4c24663fc Merge pull request #874 from catenacyber/fuzz6fix
Fuzz6fix
2020-04-15 18:05:16 +02:00
Luca Deri 9f2dabbda4 Merge pull request #873 from a-czyrny/a-czyrny-ndpireader-csv-patch
Fix for missing line breaks in CSV file
2020-04-15 18:04:26 +02:00
Philippe Antoine 4f370fe7c4 Adds netbios bound check 2020-04-15 16:40:00 +02:00
Philippe Antoine 9483c842b5 TLS initializes version_str 2020-04-15 16:40:00 +02:00
Philippe Antoine c1f9f05d33 Adds tls check before reading memory 2020-04-15 16:22:16 +02:00
Philippe Antoine cf47ba234a Use ndpi_handle_ipv6_extension_headers in reader_util 2020-04-15 16:19:57 +02:00
Philippe Antoine 98f2d3a73c Adds bound check for IRC 2020-04-15 15:56:58 +02:00
Philippe Antoine c1baf1516d Adds bound check for TZSP 2020-04-15 15:50:58 +02:00
Philippe Antoine 369dc65c1a Fix integer overflow in quic 2020-04-15 15:45:32 +02:00
Philippe Antoine 4ec3e6c064 Adds bound check in TLS 2020-04-15 15:42:54 +02:00
Philippe Antoine 81e97d7515 Fixes OOB reads in postgres
Reported by GHSL
2020-04-15 15:35:34 +02:00
Alexander Czyrny 32ab88a26e Fix for missing line breaks in CSV file
If "-v" is used as an argument, the line breaks are missing in the csv file ("-C"), because the argument "-J" is assumed. Moving the condition for the CSV file handler out of the scope of the "enable_joy_stats" condition removes this dependency and the CSV file is formatted correctly.
2020-04-15 10:12:37 +02:00
Luca Deri afdae85b30 Restored ndpi_get_api_version() prototype 2020-04-15 09:06:02 +02:00
Luca Deri 63c4eaae9b Invalid function definition fix 2020-04-13 22:40:20 +02:00
Luca Deri 4e876a4543 Merge pull request #866 from anticomputer/packet_payload_integer_ql
ql query to identify suspicious use of network sourced integers
2020-04-15 08:38:23 +02:00
Luca Deri 7b54d028a3 Merge pull request #872 from aouinizied/dev
Update python example.
2020-04-15 08:37:28 +02:00
Luca Deri 9cba06a080 Merge pull request #869 from disconnect3d/patch-2
Fix incorrect "<iq from=\"' parsing
2020-04-15 08:36:27 +02:00
Luca Deri 794f7fa124 Merge pull request #868 from disconnect3d/patch-1
Fix off by one when checking for "GET / HTTP" string
2020-04-15 08:35:59 +02:00
Luca Deri d3b4b81d6f Merge pull request #867 from IvanNardi/memory-leaks
Memory leaks
2020-04-15 08:35:28 +02:00
Zied Aouini 1bee9d85a4 Fix flow_printer example. 2020-04-15 08:05:03 +02:00
Disconnect3d fa15e170b4 Fix incorrect "<iq from=\"' parsing
This PR fixes a wrong size arg passed to memcmp when checking for "<iq from=\"" and "<iq from \'" strings.
2020-04-13 16:35:03 +02:00
Disconnect3d b75838b771 Fix off by one when checking for "GET / HTTP" string 2020-04-13 16:33:27 +02:00
Luca Deri d665df64e2 Added ndpi_serialize_raw_record() API call 2020-04-10 21:39:23 +02:00
Nardi Ivan e84563f971 ndpiReader: fix memory leak in idle sessions purging 2020-04-08 15:15:34 +02:00
Nardi Ivan a60854bae6 Fix some debug messages 2020-04-08 15:15:34 +02:00
Nardi Ivan 7b690d8b2d kerberos: fix heap-overflow error 2020-04-08 15:15:34 +02:00
Nardi Ivan 0c1322a718 s7comm: fix heap-overflow error 2020-04-08 15:15:34 +02:00
Nardi Ivan b40732d8a5 Fix an harmless memory leak
Leak introduced in 90e08b35, while fixing #845
2020-04-08 15:15:34 +02:00
Luca Deri db8497778d Fixes #853 and adds a self check for duplicates 2020-04-07 19:17:37 +02:00
Luca Deri dea883a25d Minor refresh 2020-04-07 19:01:38 +02:00
Luca Deri 81a2f54dd7 Merge pull request #865 from catenacyber/ghsl-2020-05x
Ghsl 2020 05x
2020-04-05 08:29:43 +02:00
Luca Deri c2c01b618b Fix for string serialization with initial empty string 2020-04-03 15:26:24 +02:00
Bas Alberts fb863c6bfb ql query to identify suspicious use of network sourced integers 2020-04-02 17:14:49 -04:00
Philippe Antoine 7ce478a58b ssh: fixing unsigned overflow leading to heap overflow
cf GHSL-2020-051
2020-04-02 16:48:35 +02:00
Philippe Antoine 3bbb0cd329 ssh: adds systematic bounds checks in concat_hash_string
cf GHSL-2020-052
2020-04-02 16:35:10 +02:00
Luca Deri 56ca71bda9 Merge pull request #863 from IvanNardi/memory-errors
Memory errors
2020-03-31 17:51:20 +02:00
Nardi Ivan 498571354d openvpn: fix heap-overflow error 2020-03-27 11:09:53 +01:00
Nardi Ivan 3924257f32 telnet: fix another heap-overflow error 2020-03-27 11:09:53 +01:00
Nardi Ivan 817aa54214 telnet: fix heap-overflow error
There is some boilerplate since I removed an if branch and I had to reindent
the code
2020-03-27 11:09:53 +01:00
Nardi Ivan 97fc94c7e8 kerberos: fix use-after-free error
After leaving kerberos code, the original packet may be processed from
other dissector (i.e. TLS)
2020-03-27 11:09:53 +01:00
Nardi Ivan 9ec711aa90 tls: fix heap-overflow error 2020-03-27 11:09:53 +01:00
Nardi Ivan 9c2c04d8ce ssh: fix heap-overflow error 2020-03-27 11:09:53 +01:00
Nardi Ivan 4f399356a6 quic: fix heap-overflow error 2020-03-27 11:09:53 +01:00
Nardi Ivan 45ff2a3f5a h323: fix heap-overflow error 2020-03-27 11:09:53 +01:00
Nardi Ivan 58f0f29113 soulseek: fix heap-overflow error 2020-03-27 11:09:53 +01:00
Nardi Ivan da5e5a3e2a ciscovpn: fix heap-overflow error 2020-03-27 11:09:53 +01:00
Luca Deri 17d531e3db Added s7comm test pcap 2020-03-27 09:35:59 +01:00
Luca Deri 0d223ada3f Compilation fix 2020-03-27 08:46:03 +01:00
Luca Deri 890e8644ac Merge pull request #861 from havup/dev
pull request with s7comm
2020-03-27 08:38:44 +01:00
havup 1bab8a1c22 Update s7comm.c 2020-03-27 14:02:10 +08:00
havup 24bb0b7f53 Update s7comm.c 2020-03-27 13:09:46 +08:00
havup 803b13a7a8 Update s7comm.c 2020-03-27 12:58:56 +08:00
havup 7841145d9c Update tls_long_cert.pcap.out 2020-03-27 12:28:15 +08:00
havup efdac78ddc Update s7comm.c 2020-03-27 11:19:50 +08:00
havup 74825ffc6e Update s7comm.c 2020-03-26 13:06:02 +08:00
emanuele-f 1b8f7ed101 Fix API version generation 2020-03-25 14:07:19 +01:00
emanuele-f 9cf016c5f1 Fix ndpi_get_api_version version truncation 2020-03-25 14:00:47 +01:00
Luca Deri 47a4507d9e NDPI_API_VERSION is now computed automatically 2020-03-25 12:51:06 +01:00
havup 779d9a6e51 Add files via upload 2020-03-25 19:09:14 +08:00
havup ac17e0bea7 Update ndpi_main.c 2020-03-25 19:06:36 +08:00
havup 4d95dae2ae Update ndpi_main.c 2020-03-25 19:05:49 +08:00
havup 6c3cbe50e2 Update ndpi_protocols.h 2020-03-25 19:04:35 +08:00
havup a2b163beca Update ndpi_protocol_ids.h 2020-03-25 19:04:00 +08:00
Luca Deri c8a1767465 Fixed invalid allocation 2020-03-24 13:36:03 +01:00
Luca Deri 40be74c629 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2020-03-23 14:45:08 +01:00
Luca Deri df5cf59787 Format update 2020-03-23 14:44:33 +01:00
Luca Deri d26349e082 Format update 2020-03-23 14:37:14 +01:00
Luca Deri 7add3acc53 Added fuzz-2020-02-16-11 fuzzy pcap 2020-03-23 10:27:32 +01:00
Luca Deri 1c1a18d12f Added fuzz-2006-09-29-2858 fuzzy pcap 2020-03-23 10:25:28 +01:00
Luca Deri 92bd223c34 Added fuzz-2006-06-26-2594.pcap fuzzy pcap 2020-03-23 10:24:11 +01:00
Luca Deri 9d53c3c3b8 Added wa_video.pcap 2020-03-23 10:23:05 +01:00
Luca Deri 906d7fe0c1 Added wa_voice.pcap 2020-03-23 10:22:46 +01:00
Luca Deri cd78740caf Added netflow-fritz.pcap 2020-03-23 10:11:51 +01:00
Luca Deri 626ee52049 Added smb_deletefile.pcap 2020-03-23 10:09:47 +01:00
Luca Deri 7a93994802 Added iphone.pcap 2020-03-23 10:09:14 +01:00
Luca Deri 1bb10863e4 Added android.pcap 2020-03-23 10:08:57 +01:00
Luca Deri 924bc9698b Added badpackets.pcap 2020-03-23 10:06:16 +01:00
Luca Deri 9aba94713c Added dns_exfiltration.pcap 2020-03-23 10:06:00 +01:00
Luca Deri 00beac6b1a Added netbios.pcap 2020-03-23 10:05:41 +01:00
Luca Deri 8c2ffc5b37 Added dos_win98_smb_netbeui.pcap 2020-03-23 10:05:24 +01:00
Luca Deri a845e99720 Merge pull request #859 from catenacyber/fuzzudpfix
Checks enough data for UDP header
2020-03-20 18:02:27 +01:00
Luca Deri 8cda02bb14 Fixed loop in serialization 2020-03-20 18:01:37 +01:00
Philippe Antoine 3e259aac98 tls_supported_versions only if version_str is initialized
With version_len == (extension_len-1)
2020-03-19 17:19:28 +01:00
Philippe Antoine 2aec630202 Fix use of uninitialized value in TLS 2020-03-19 17:16:41 +01:00
Philippe Antoine c6acf97bfb Adds different checks against overflows 2020-03-19 16:54:31 +01:00
Philippe Antoine e9195589d2 Checks enough data for UDP header 2020-03-19 16:44:53 +01:00
Luca Deri b2cbe76e71 Added QUIC v046 test pcap 2020-03-17 16:51:25 +01:00
Luca Deri 4227be7989 Update for QUIC 046 2020-03-17 16:48:34 +01:00
Simone Mainardi e282e43279 Fixes netbios overread
This commit inherits from https://github.com/ntop/nDPI/commit/7a2bcd9c395f9fe554109e04add33e9e65564d82
but leaves ndpi_netbios_name_interpret as part of the API as it
is used by ntopng
2020-03-12 22:21:07 +01:00
Simone Mainardi 2f15ce706f Fixed API change which removed ndpi_netbios_name_interpret
This reverts commit 7a2bcd9c39.
2020-03-12 22:08:59 +01:00
Luca Deri 70ee793ff3 Merge pull request #856 from catenacyber/fuzzfix5
Fuzzfix5
2020-03-12 18:13:30 +01:00
Philippe Antoine 7806eb5f5b Fix infinite loop in tls 2020-03-12 14:31:47 +01:00
Philippe Antoine 9d1340a8e7 Fix buffer overread in yahoo 2020-03-12 14:26:08 +01:00
Philippe Antoine 7a2bcd9c39 Fix buffer overread in netbios 2020-03-12 14:22:46 +01:00
Philippe Antoine fe513014a8 Fix buffer over read in dns 2020-03-12 14:13:04 +01:00
Philippe Antoine 3c05d4551d Fix snprintf return value check for tls 2020-03-12 14:05:48 +01:00
Philippe Antoine 4976d93d4e Fix buffer overread in ndpi_search_setup_capwap 2020-03-12 14:03:31 +01:00
Luca Deri 1e933e8b02 Win fixes 2020-03-12 11:26:38 +01:00
Luca Deri 942a71c7eb NetBIOS dissection improvements 2020-03-01 11:42:21 +01:00
Luca Deri 7fd135a8c9 Added SMBv1 over NetBIOS detection 2020-03-01 11:32:09 +01:00
Luca Deri 34ad06fef5 Compilation fix 2020-02-28 16:03:27 +01:00
Luca Deri 8f45f51e6c Compilation fix 2020-02-28 15:28:41 +01:00
Luca Deri 9f72b15fff Merge pull request #850 from catenacyber/fuzz4fix
Fuzz4fix
2020-02-28 14:39:17 +01:00
Philippe Antoine 6fe9535881 Adds check before memcpy for bittorrent 2020-02-26 17:12:46 +01:00
Philippe Antoine 5fc9d41eb0 Run ndpi_detection_process_packet only with payload
Real problem is decapsulation of all ipv6 headers
2020-02-26 17:05:08 +01:00
Philippe Antoine 933351840d Fix off by one in imap 2020-02-26 16:59:04 +01:00
Philippe Antoine 83fdfe24d0 Fix read overflow before UDP header 2020-02-26 16:16:29 +01:00
Alfredo Cardigliano d4c093e828 Check NULL strings in ndpi_serialize_string_string 2020-02-26 09:55:24 +01:00
Luca Deri c0ec78885d Merge pull request #849 from annp1987/fix_ndpi_flow2json_segfault
ndpi_flow2json should check http.url before serializer
2020-02-26 08:06:11 +01:00
Luca Deri a55c410202 Merge pull request #848 from aouinizied/dev
nDPI 3.2 support in python cffi bindings
2020-02-26 08:05:45 +01:00
Nguyen Phuong An e675be1a83 ndpi_flow2json should check http.url before serializer
This patch will fix issue: https://github.com/ntop/PF_RING/issues/557
2020-02-26 13:32:13 +07:00
Luca Deri f07a329447 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2020-02-23 19:10:14 +01:00
Luca Deri 2ceff3e214 Fixed swapped protocol 2020-02-23 19:09:58 +01:00
Zied Aouini dcb6fde0c9 Update Python cffi bindings. 2020-02-21 17:28:44 +01:00
Zied Aouini e67604a583 Remove trailing alignment directives. 2020-02-21 16:37:36 +01:00
Zied Aouini 40896d3d44 Fix docstrings. 2020-02-21 15:51:17 +01:00
Zied Aouini 5a0f609086 Add ndpi_flow_tcp_struct and ndpi_flow_udp_struct api calls. 2020-02-21 15:49:46 +01:00
Alfredo Cardigliano 2972e29097 Moved to 3.3 2020-02-20 10:15:11 +01:00
Luca Deri dd80ec869a Fixes #837 2020-02-19 23:44:22 +01:00
Luca Deri 1714d7aa52 Fixes #842 2020-02-19 23:40:53 +01:00
Luca Deri 90e08b3568 Fixes #845 2020-02-19 23:38:33 +01:00
Luca Deri 080e23e30e Merge pull request #846 from catenacyber/fuzzofix
Fix various buffer over reads
2020-02-19 22:55:18 +01:00
Luca Deri edce5a8c1f Fix for disabling guess when not requested 2020-02-19 12:24:26 +01:00
Alfredo Cardigliano c8ccf33f07 nDPI 3.2 changelog 2020-02-19 12:10:49 +01:00
Philippe Antoine ee979ac14a Fix kerberos leak 2020-02-18 13:32:20 +01:00
Philippe Antoine 3eb9907dd7 Fix various buffer over reads 2020-02-18 11:50:22 +01:00
Luca Deri fdf8dd724f Minor fix 2020-02-17 22:15:36 +01:00
Luca Deri 37121a9b58 Added protocol range check 2020-02-17 19:34:47 +01:00
Luca Deri bbaf9147b6 Fixed TLS client SNI dissection 2020-02-17 18:48:21 +01:00
Luca Deri 9796fdc0e9 Temporarely disabled x390 s390x as we have no way to debug the code on such platforms 2020-02-15 17:31:30 +01:00
Luca Deri e0a058af67 Merge pull request #844 from catenacyber/fuzz2fix
Fuzz2fix
2020-02-15 17:21:52 +01:00
Alfredo Cardigliano 80e9826040 Add ndpi_serialize_string_raw to add raw json values 2020-02-14 16:32:00 +01:00
Philippe Antoine bf7dcd63c3 Checks length for next ip header 2020-02-14 14:18:31 +01:00
Philippe Antoine b287dccecf Checks l4 size against l3 size 2020-02-14 14:02:19 +01:00
emanuele-f cb34c6b233 Add fpic in the reader 2020-02-14 12:06:55 +01:00
emanuele-f fd94270507 Remove decimals in test results for IAT, packet lengths and goodput ratio 2020-02-14 11:42:20 +01:00
Emanuele Faranda 03cf975c07 Merge pull request #840 from xnox/more-tests
travis: bump to bionic, add arm64 & s390x.
2020-02-14 10:32:31 +00:00
emanuele-f 428661c00b Add debug symbols when using --with-llvm-sanitizer 2020-02-14 11:09:00 +01:00
emanuele-f 5c52c00ea4 Add the --with-llvm-sanitizer flag
To enable the leaks detector, set the ASAN_OPTIONS=detect_leaks=1
environment variable, e.g:

	ASAN_OPTIONS=detect_leaks=1 example/ndpiReader -i test.pcap
2020-02-13 18:42:13 +01:00
Luca Deri f89f25c118 Added ndpi_is_protocol_detected() API call 2020-02-13 15:20:47 +01:00
emanuele-f d4ae8a5f16 Fix memory invalid access in SMTP and leaks in TLS 2020-02-13 10:46:30 +01:00
emanuele-f 7234f36949 Fix possible leak 2020-02-13 10:14:57 +01:00
emanuele-f 2efc09a20e Fix invalid memory read 2020-02-12 19:36:47 +01:00
emanuele-f 3fa61a394b Fix little leak on the proto_defaults strdup 2020-02-12 17:15:01 +01:00
Dimitri John Ledkov 582a01303f travis: bump to bionic, add arm64 & s390x. 2020-02-10 18:26:47 +00:00
Luca 97fb50e837 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2020-02-10 15:26:21 +01:00
Luca Deri a3da5604a3 Allocation fix in kerberos dissector 2020-02-10 15:25:40 +01:00
Luca af6c86de28 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2020-02-10 15:04:37 +01:00
Luca Deri 34984f1aec Removed outdated AppVeyor link 2020-02-08 11:25:39 +01:00
Luca Deri 83bbb4cb69 Compliation warning fixes 2020-02-08 11:24:40 +01:00
Luca Deri ff4f32ff48 Merge pull request #839 from catenacyber/fuzzbadge
Adds oss-fuzz badge to REAMDE
2020-02-08 11:21:23 +01:00
Luca Deri 33d761a55a Merge pull request #838 from catenacyber/fix2
Adds missing checks
2020-02-08 11:20:54 +01:00
Luca Deri 2bd89be3e2 Added new TLS test files 2020-02-08 10:38:22 +01:00
Luca Deri 5cad39f0e8 Added export of TLS supported version in TLS header 2020-02-07 23:26:03 +01:00
Luca Deri 3be263aafc Added TLS ALPN support 2020-02-07 21:54:04 +01:00
Luca 452ba1f339 Improvements for DNS continuous flow dissection 2020-02-06 19:19:36 +01:00
emanuele-f ec15089208 Little enhancements 2020-02-06 18:22:00 +01:00
Philippe Antoine af661bd652 Adds oss-fuzz badge to REAMDE 2020-02-06 10:56:33 +01:00
Luca Deri 4613515635 Category matching fix 2020-02-05 08:37:24 +01:00
Luca Deri 2009361b89 Fix for IPv6 address format across the various platforms/distributions 2020-02-04 22:51:07 +01:00
Luca Deri f027ab1ae7 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2020-02-04 22:50:37 +01:00
Luca Deri fa26c62b0c Fix for IPv6 address format across the various platforms/distributions 2020-02-04 22:50:08 +01:00
Luca Deri 0faba29351 Result update 2020-02-04 22:38:56 +01:00
Luca Deri 2701cc9491 Warnign fix 2020-02-04 22:34:08 +01:00
Luca Deri 88fc9232ff Code improvements 2020-02-04 22:31:02 +01:00
Luca Deri 0703ab5ac5 Improved DNS response decoding
The first decoded address is now reported by ndpiReader
2020-02-04 22:16:54 +01:00
Philippe Antoine 418ea234ae Adds missing checks
Found by fuzzing
2020-02-03 22:38:31 +01:00
Luca 3e99b73635 Updte WebattackRCE.pcap.out with libpcre disabled 2020-02-02 22:15:55 +01:00
Luca 6dbe81588f Added --with-pcre to enable libpcre usage 2020-02-02 22:07:21 +01:00
Luca 2b6a639192 Libfuzz fixes 2020-02-02 21:54:36 +01:00
Luca 9e87645022 PCRE fixes 2020-02-02 19:54:31 +01:00
Luca 626fcf1418 Added missing additiona libs 2020-02-02 17:01:07 +01:00
Luca Deri 5b11c2ffc5 Merge pull request #836 from MrTiz9/dev
nDPI now detects RCE injections in HTTP requests
2020-02-02 16:19:26 +01:00
Luca Deri 8486eb08ea Merge pull request #835 from catenacyber/fuzzfix
Fuzz fixes
2020-02-02 16:17:03 +01:00
MrTiz9 ce4f4fbe2f Added missing pcap.out for RCE detections 2020-02-01 17:23:22 +01:00
MrTiz9 3422574c17 FIXED - nDPI now detect RCE injections via PCRE instead Intel Hyperscan 2020-02-01 17:18:35 +01:00
MrTiz9 b3138d5e3e Merge branch 'dev' of https://github.com/ntop/nDPI into dev-unstable 2020-02-01 13:47:42 +01:00
Philippe Antoine 0bf2ca56a8 Uses ip_size in get_ndpi_flow_info6 2020-01-31 15:37:36 +01:00
Philippe Antoine b97360d931 Fix infinite loop in ndpi_workflow_process_packet 2020-01-31 15:26:34 +01:00
Philippe Antoine 579dd7dff9 Fix memcached read overflow 2020-01-31 15:21:03 +01:00
Philippe Antoine ff89a14b49 Modifies fuzz target to fix leaks 2020-01-31 15:13:35 +01:00
Philippe Antoine b5b2260f2e Fix undefined shift in dns 2020-01-31 14:50:18 +01:00
lucaderi 39b1ad3087 PCRE fix 2020-01-31 07:09:54 +01:00
Luca Deri c68ea3067b Added libpcre lib if present 2020-01-30 22:24:24 +01:00
Alfredo Cardigliano 4e538b1e4d Disabeld community id debug 2020-01-30 15:20:52 +01:00
MrTiz9 5c8c2d843a nDPI now detect RCE injections via PCRE instead Intel Hyperscan - BUGGY, DOES NOT COMPILE 2020-01-30 15:08:26 +01:00
MrTiz9 ea957687e1 Merge branch 'dev' of https://github.com/ntop/nDPI into dev-unstable 2020-01-30 14:57:58 +01:00
Luca Deri c84ed5a712 Added check for libpcre presence 2020-01-29 00:41:35 +01:00
Alfredo Cardigliano ab2a011cf9 Community ID hash fixes 2020-01-25 16:04:00 +01:00
Luca Deri d86d632fe6 Added memory boundary check in HTTP dissector 2020-01-24 21:04:27 +01:00
Luca Deri 87d228f78e Fixed heap-buffer-overflow in citrix decoder 2020-01-24 19:50:38 +01:00
Luca Deri a7a4bf32d3 Merge pull request #833 from catenacyber/travisfuzz2
Fix sanitizers build on Travis
2020-01-24 19:38:18 +01:00
Alfredo Cardigliano e20c4acbde Implement ndpi_flowv6_flow_hash ndpi_flowv4_flow_hash. Add ndpi_base64_encode. 2020-01-24 18:36:38 +01:00
MrTiz9 daf15e70bf Added missing pcap.out for RCE detections 2020-01-24 17:16:18 +01:00
MrTiz9 daa1171593 nDPI now detect RCE in HTTP GET requests 2020-01-24 17:16:18 +01:00
Philippe Antoine dc5fecfc92 Use MSAN_SYMBOLIZER_PATH now that it works 2020-01-24 10:15:16 +01:00
Philippe Antoine 79e1ec9f50 Complete absolute path for symbolizer 2020-01-24 10:03:23 +01:00
Philippe Antoine 902a1c7b4c Trying with ASAN_SYMBOLIZER_PATH 2020-01-24 09:53:06 +01:00
Philippe Antoine 5b88867084 Looking for symbolizer 2020-01-24 09:46:53 +01:00
Philippe Antoine 4749e61bdb Debug symbols in LDFLAGS 2020-01-24 09:23:56 +01:00
Philippe Antoine 4f230943cf Debug compilation flag for fuzz targets
And adding include file to fuzz_ndpi_reader
2020-01-24 09:10:52 +01:00
Philippe Antoine 6186e1ab01 Only one env 2020-01-24 08:40:05 +01:00
Philippe Antoine 9fe8bf018a Use clang++7 for fuzz targets in Travis 2020-01-24 08:24:18 +01:00
Philippe Antoine e082134a0d Fix sanitizers build on Travis 2020-01-24 08:14:44 +01:00
Luca Deri cedbf3b636 Merge pull request #832 from catenacyber/travisfuzz
Adds fuzz targets build and run to travis
2020-01-23 18:36:40 +01:00
Luca Deri 5f6fd8d2a5 Merge pull request #831 from catenacyber/fuzzcorpus
Generates automatically corpus for fuzz target
2020-01-23 18:35:48 +01:00
Philippe Antoine cbb755666e Typo fix 2020-01-23 09:09:21 +01:00
Philippe Antoine 1d024ece7a Adds fuzz targets build and run to travis
Build comes with the different sanitizers
2020-01-22 16:36:12 +01:00
Philippe Antoine 77db14fd7a Generates automatically corpus for fuzz target 2020-01-22 16:12:50 +01:00
Luca Deri 5a40f9b7fd Minor fixes 2020-01-21 20:52:42 +01:00
Luca Deri 931f51bd77 ndpi_load_categories_file() now returns the number of loaded hosts or -1 in case of error 2020-01-20 23:37:09 +01:00
Luca Deri 283588a37b Crash fix when not using CSV output 2020-01-19 20:50:23 +01:00
Luca 812505b56b Added sanity check 2020-01-15 20:49:21 -07:00
Luca Deri 98d9f524f9 Merge pull request #830 from catenacyber/fuzzpcap
Adds another fuzz target
2020-01-14 16:37:55 -08:00
Luca Deri 0023a96f80 Merge pull request #828 from vitalivanov/pull_764
example: fix compilation of ndpireader on latest dpdk.
2020-01-14 16:37:28 -08:00
Luca Deri 9f7b1c84f2 Merge pull request #829 from vitalivanov/pull_warning_vit
example: fix build warning.
2020-01-14 16:36:51 -08:00
Alfredo Cardigliano e1047b324b Fix misleading error message 2020-01-14 10:57:05 +01:00
Philippe Antoine 10738a0ca3 New fuzz target : fuzz_ndpi_reader 2020-01-14 08:20:15 +01:00
Philippe Antoine fe312ffbe1 Compile libndpireader 2020-01-14 08:20:15 +01:00
Vitaliy Ivanov 24f2cd6fc1 example: fix build warning.
ndpiReader.c:24:0: warning: "_GNU_SOURCE" redefined
 #define _GNU_SOURCE
<command-line>:0:0: note: this is the location of the previous definition

Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com>
2020-01-13 16:15:58 +02:00
Vitaliy Ivanov cbabbd5865 example: fix compilation of ndpireader on latest dpdk.
reader_util.c:1708:43: error: 'ETHER_MAX_LEN' undeclared here (not in a function)
             .rxmode = { .max_rx_pkt_len = ETHER_MAX_LEN }
                                           ^~~~~~~~~~~~~

DPDK before 19.08 had a macro ETHER_MAX_LEN, in later versions it was
changed to RTE_ETHER_MAX_LEN.

Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com>
2020-01-13 16:10:09 +02:00
Luca Deri da5208ee68 Updated results with the new output 2020-01-10 23:21:11 +01:00
Luca Deri bccf1c433a Added missing flow extra info export in flows 2020-01-10 23:15:54 +01:00
Luca Deri cc7593fe5a Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2020-01-10 22:21:42 +01:00
Luca Deri 5571ce114d Added flow extra info field
Updated tests/results
2020-01-10 22:21:16 +01:00
Luca 4dd887e516 Added further TLS checks 2020-01-10 18:18:07 +01:00
Luca Deri a3e06da2a8 Minor cleanup 2020-01-10 12:46:53 +01:00
Luca Deri dc21079851 Fix for invalid TLS leght 2020-01-09 18:51:03 +01:00
Luca Deri 5987ec6014 Added check for preventing TLS misbehavior 2020-01-09 17:14:37 +01:00
Luca 757b0c602f Fix for avoiding misdetection in TLS 2020-01-09 10:30:14 +01:00
Luca Deri 498b346219 Added bootstrap ethereum nodes 2020-01-08 22:44:30 +01:00
Luca Deri 3e66bbc5bc Tool for listing the current bitcoin IPv4 nodes 2020-01-08 22:14:34 +01:00
Luca Deri 3ba7667887 Various ethereum improvements 2020-01-08 22:01:45 +01:00
Luca Deri fd943d6000 Code hardening 2020-01-08 17:13:24 +01:00
Luca Deri 691778df33 Cosmetic fixes 2020-01-06 22:25:20 +01:00
Luca Deri 81c3f5813c Added check to avoid double allocations with retransmitted packets 2020-01-06 21:27:51 +01:00
Luca Deri fdf402d027 Added missing RPM files 2020-01-06 18:20:52 +01:00
Luca Deri f0014a5a24 Removed TLS debug code that could have caused crashes 2020-01-05 21:28:35 +01:00
Luca Deri 53a5be3399 Added packet boundary checks 2020-01-05 20:58:16 +01:00
Luca Deri f389969c0b Use share instead of sbin for sample files
Fixes #826
2020-01-05 18:50:56 +01:00
Luca Deri dc02afa3ea Merge pull request #825 from aouinizied/fix_python_bindings
Update Python cffi bindings.
2020-01-05 18:47:43 +01:00
Luca Deri d1fb41a161 Minor cleanup 2020-01-05 18:42:36 +01:00
Luca Deri ef16591f3f Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2020-01-05 18:25:44 +01:00
Luca Deri 29dd45838d Updated (C) 2020-01-05 18:24:58 +01:00
emanuele-f 798bb6e2e1 Fix leaks and sha1 certificate detection 2020-01-02 14:39:51 +01:00
emanuele-f 2332cbfefe Fix invalid free on non-tls flows 2020-01-02 14:10:21 +01:00
emanuele-f a58a135758 Add fingerprint_set flag 2020-01-02 12:56:25 +01:00
Luca 634457615c Updated TLS support 2020-01-02 10:47:23 +01:00
Luca cb825766ea Minor changes 2020-01-02 10:37:53 +01:00
Luca 8b01056b21 Renamed TLS requested server name 2020-01-02 07:37:03 +01:00
Luca f6c7a33177 Added TLS test 2020-01-01 21:27:18 +01:00
Luca daae1cc9b1 Reworked TLS dissection 2020-01-01 12:59:19 +01:00
Luca Deri 499c80535b Minor fix 2019-12-29 22:27:48 +01:00
Luca 9fb3a57a71 Kerberos fixes
Minor TLS cleanup
2019-12-29 10:45:42 +01:00
Luca 257ec7cc5f Removed disable_metadata_export preference that is no longer useful
since ndpi_process_extra_packet() can drive limited or full metadata export
2019-12-29 08:07:35 +01:00
emanuele-f 73c7ccdb65 Fix crash in ndpi_fill_ip_protocol_category when both saddr and daddr are 0
In this corner case, the "prefix" variable was not initialized leading to a failed assertion and crash:

ndpi_patricia_search_best2: Assertion `prefix->bitlen <= patricia->maxbits' failed.
2019-12-23 15:20:09 +01:00
Zied Aouini cffb11fbc9 Update ndpi_flow_struct definition. 2019-12-23 11:22:23 +01:00
Luca Deri 23421b47d9 Added snprintf() return code check 2019-12-18 19:49:46 +01:00
Luca Deri 1cab9fe838 Fix for custom protocol detection 2019-12-18 19:35:40 +01:00
Luca Deri 27ea04c08e Merge pull request #824 from catenacyber/stunfix
Fix read buffer overflow in stun
2019-12-18 19:29:23 +01:00
Luca Deri 5c225b8a2e Merge pull request #821 from aouinizied/win32-lib-appveyor
Win32 lib appveyor
2019-12-18 19:22:12 +01:00
Luca Deri 4961241c62 Merge pull request #823 from haroflow/dev
Changed 13.64.0.0/11 range from Skype to Microsoft
2019-12-18 19:18:28 +01:00
Philippe Antoine 843204299c Fix read buffer overflow in stun 2019-12-18 13:48:33 +01:00
Simone Mainardi 09cd95d54a Fixes informative subprotocols
Wrongly marked subprotocols were causing issues on ntopng.
This fixes https://github.com/ntop/ntopng/issues/3129
2019-12-17 18:12:15 +01:00
Renan de Souza 1d3bef41a4 Added *whatsapp-*.fbcdn.net IPs as WhatsApp 2019-12-16 15:32:46 -03:00
Luca Deri e21fe5d594 Added intrusion_detection.c in DPDK makefile 2019-12-16 17:56:08 +01:00
Renan de Souza 9133955219 Changed 13.64.0.0/11 range from Skype to Microsoft, blocking this range also blocks some azure-hosted websites 2019-12-16 10:50:19 -03:00
Zied Aouini acb74905a5 Switch to dev branch. 2019-12-16 01:36:02 +01:00
Zied Aouini ed3fff554a Add Appveyor build status. 2019-12-16 01:34:42 +01:00
Zied Aouini 7ebe6a0c82 Add LDFLAGS for Win32. 2019-12-16 01:24:42 +01:00
Zied Aouini 87333aa68c Add branch. 2019-12-16 01:19:37 +01:00
Zied Aouini e593319ba0 Fix Win32. 2019-12-16 01:14:10 +01:00
Zied Aouini 812a3e7099 Add mingw host target 2019-12-16 01:11:55 +01:00
Zied Aouini 1097f5488d Add .appveyor.yml 2019-12-16 01:10:26 +01:00
Zied Aouini b90410e335 Add .gitattributes 2019-12-16 01:09:09 +01:00
Luca Deri deed27cbef Implemented nDPI timeline visualizer 2019-12-15 23:35:43 +01:00
Luca Deri ea001b439a TLS decoding improvements 2019-12-14 16:17:34 +01:00
Luca Deri 14e5adc7c2 Win fixes 2019-12-14 15:05:47 +01:00
Luca Deri 81012b3201 CSV output enhancements 2019-12-13 12:50:52 +01:00
Luca Deri 406aa12468 Fixed CSV output 2019-12-13 09:45:34 +01:00
Luca 2c2bb11c9b Fixed output format 2019-12-11 11:27:00 +01:00
Luca Deri 1bd63d68a5 CSV export improvements 2019-12-11 09:45:14 +01:00
Luca Deri bf5e9b7b0b Fixed bug that prevented SMTP email to be dissected 2019-12-09 19:04:13 +01:00
Luca Deri 7fe9dddf65 Fixed invalid password report on telnet 2019-12-09 18:43:05 +01:00
Luca Deri 6d7f3976f3 Code cleanup 2019-12-09 18:02:48 +01:00
Luca Deri 65d526d8f6 Code cleanup 2019-12-09 15:28:44 +01:00
Luca Deri 56eeb965bf Code cleanup 2019-12-09 13:10:21 +01:00
Luca Deri c4d476cc58 Code improvements 2019-12-09 00:29:02 +01:00
Luca Deri 23b0b8625d Minor changes to the IDS score calculation 2019-12-09 00:11:32 +01:00
Luca Deri 558983c99c Merge pull request #813 from SimoneRicci97/dev
Fixed intrusion detection
2019-12-09 00:01:09 +01:00
Luca Deri c8d6f2845f Improved memory management in the kerberos dissector 2019-12-08 23:59:53 +01:00
Luca Deri 1a115cac5a Merge pull request #817 from MrTiz9/dev
Integration of the libinjection library
2019-12-08 23:49:23 +01:00
Luca Deri 11401edfe7 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-12-08 23:47:25 +01:00
Luca Deri 239842b821 Major kerberos rework 2019-12-08 23:47:04 +01:00
MrTiz9 606ff10ecf Integration of the libinjection library to detect SQL injections and XSS type attacks in HTTP requests 2019-12-05 18:09:43 +01:00
emanuele-f d37b69ce9c Fix corner case causing access to already freed memory 2019-12-04 18:36:54 +01:00
emanuele-f 226a9abf22 Additional memory bounds checks 2019-12-04 18:34:08 +01:00
emanuele-f f26096ee4c Fix reversed offset check 2019-12-04 15:20:49 +01:00
emanuele-f 2b3ef7e762 Fix invalid memory access 2019-12-04 14:21:04 +01:00
emanuele-f d62526f9ed Fix invalid reads and add valgrind test 2019-12-04 13:06:05 +01:00
Luca Deri e66d7216b8 Removed header space with JSON serialization 2019-12-04 10:52:51 +01:00
Luca Deri fbc59c5f81 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-12-01 22:27:26 +01:00
Luca Deri 32dc9e3225 Refreshed Whatsapp server list 2019-12-01 22:26:57 +01:00
Luca Deri b4468239b9 Merge pull request #814 from aouinizied/python_cffi_bindings
Implement cffi bindings.
2019-12-01 21:56:10 +01:00
Luca Deri 95c1282ef4 Improved categories with streaming 2019-12-01 21:52:28 +01:00
Luca Deri a7b0457753 Improved category detection with HTTP 2019-12-01 21:40:04 +01:00
Luca Deri 7da7ae622b Improvements to stop dissection when the first protocol is detected
Used IP-based detection to compute the application protocol
Improved application detection
2019-11-28 23:20:40 +01:00
aouinizied a95cfcc989 Implement cffi bindings. 2019-11-28 03:09:13 +01:00
Simone Ricci a65c959940 Fixed intrusion detection 2019-11-26 21:20:22 +01:00
Luca Deri d328c3bccc Overflow fix 2019-11-26 17:58:49 +01:00
Luca Deri 8368568443 Added buffer overflow check in smpp dissector 2019-11-26 17:45:08 +01:00
Luca Deri c223bb1f10 Fixed buffer overflow in DNS dissection 2019-11-26 17:42:59 +01:00
Luca Deri c7efd0892f Fixed buffer overflow in amazon video dissector 2019-11-26 17:30:36 +01:00
Luca ae9429408b Added webattacj results 2019-11-25 18:40:14 +01:00
Luca Deri 0ce56d5714 Updated results 2019-11-23 19:55:06 +01:00
Luca Deri 509cfcb556 Fixed invalid category matching in HTTP
Fixed overlapping categories
2019-11-23 19:54:06 +01:00
Luca Deri 3b203d57e9 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-11-23 13:28:15 +01:00
Luca Deri 68d66b780c Added new test pcaps
Renamed protocol 104 to IEC60870 (more meaningful)
2019-11-23 13:27:34 +01:00
Luca Deri f94d6cca7d Memory fixes 2019-11-23 12:14:52 +01:00
Luca Deri 382217887b Nested encapsulation dissection 2019-11-23 11:53:00 +01:00
Luca Deri 537c11a985 Fixes for uninitialized variables 2019-11-23 11:39:23 +01:00
Luca Deri bdc0719e76 Added auth failed support with FTP 2019-11-21 23:31:52 +01:00
Luca Deri fc82cdfa4a Implemented telnet password export 2019-11-21 19:36:01 +01:00
Luca Deri e98b994a39 Updated results 2019-11-21 13:35:04 +01:00
Luca Deri a58c838c4c Added flow duration and goodput in output (-v and -c) 2019-11-21 13:31:56 +01:00
Luca Deri f91f2c57ef Fixed skype/soom false positives 2019-11-20 23:28:24 +01:00
emanuele-f 68b7986d9b Expose API to calculate the MD5 hash 2019-11-20 11:57:21 +01:00
Alfredo Cardigliano ead0325deb Rounding up serializer buffer to be divisible by 4 (required by zmq encryption) 2019-11-19 19:03:06 +01:00
emanuele-f 0b819badd2 Possible leak fix 2019-11-19 11:42:04 +01:00
Simone Mainardi 3754e6dd12 Fixes crash when searching kerberos usernames
Fixes

(gdb) f 0
101                         if(cname_str[cname_len-1] == '$') {
(gdb) p cname_len
$3 = 0
2019-11-19 11:34:34 +01:00
Luca Deri da029607a0 Removed (unreliable) line protocol detection 2019-11-18 21:21:52 +01:00
Luca Deri 803dc6ff77 Minor fix 2019-11-17 22:42:32 +01:00
Luca Deri 65a629304c Kerberos dissection improvements 2019-11-17 22:35:10 +01:00
Luca Deri 51cfdfb0d8 Removed unused JSON-C code 2019-11-17 17:51:45 +01:00
Luca Deri 53108c9ff1 Merge pull request #811 from aouinizied/python_bindings_fix
Python bindings fix and improve
2019-11-15 19:51:15 +01:00
aouinizied 82a186e096 Polish. 2019-11-15 19:30:50 +01:00
aouinizied e467fe9aa6 Improve python bindings. 2019-11-15 19:18:30 +01:00
Luca Deri 7a1940e1f7 Modified Diameter category to network
Updated test results
2019-11-14 15:13:13 +01:00
Alfredo Cardigliano 6b7c56c909 New ndpi_serialize_string_boolean/ndpi_serialize_uint32_boolean 2019-11-13 18:16:44 +01:00
Luca 470d30ac23 Fixed SQL Injection detection 2019-11-13 17:37:48 +01:00
Luca 7e87bfd039 Added support of default /32 in ndpi_load_ipv4_ptree() 2019-11-12 16:37:18 +01:00
Luca Deri ab9ccf9eb9 Fixed new nDPi API 2019-11-12 12:28:34 +01:00
Luca Deri 15ec013af8 Added ndpi_load_ipv4_ptree() API call 2019-11-11 23:52:24 +01:00
emanuele-f 748f10c0d6 Fix possible crash due to unsigned int number wrapping 2019-11-11 10:18:55 +01:00
Luca Deri 8181d63a95 Added ndpi_init_detection_module() API preferences 2019-11-10 22:57:21 +01:00
Luca d0e7e69552 Renamed DNSoverHTTPS to handle bot DoH and DoT 2019-11-08 09:23:52 +00:00
Luca 0558d641f2 Added ndpi_finalize_initalization() initialization function 2019-11-07 19:28:16 +00:00
Alfredo Cardigliano ac46a4dd58 Added L2TP tunnel type. Warning fix. 2019-11-06 09:55:22 +01:00
Luca fb56b042c0 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-11-05 12:15:27 +00:00
Luca 6298ecc271 Added tunnelling decapsulation 2019-11-05 08:21:31 +00:00
Luca Deri a26fe08aa6 Updated results 2019-11-04 22:11:33 +01:00
Luca Deri 5b2c0f68fe Telnet fix 2019-11-04 22:06:02 +01:00
Luca Deri e695dd6ead Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-11-03 17:15:48 +01:00
Luca Deri 953571e3d8 Added comment 2019-11-03 17:15:40 +01:00
Luca Deri 01455c3188 Merge pull request #808 from oleg-umnik/dev
Change type
2019-11-03 17:15:06 +01:00
Luca Deri a5210af823 Merge pull request #807 from oleg-umnik/fix_3
Align CSV header with actual row data
2019-11-03 17:14:38 +01:00
Luca Deri 9e42b525ec Merge pull request #806 from oleg-umnik/fix_1
Don't leak memory in live capture mode
2019-11-03 17:13:49 +01:00
Luca Deri 02d69b72f5 Merge pull request #803 from haroflow/dev
Updated Breed's descriptions in ndpi_get_proto_breed_name
2019-11-03 17:13:18 +01:00
Luca Deri c998e2e9ef Merge pull request #802 from AndrewKarelin/dev
Added some public DNSoverHTTPS servers
2019-11-03 17:07:17 +01:00
Luca Deri a3a85106a1 Implemented SQL Injection and XSS attack detection 2019-11-01 23:05:11 +01:00
Luca 75302a10bd Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-10-31 09:31:42 +01:00
Luca 2ab0433aca Updated content-type parsing 2019-10-31 09:31:26 +01:00
Luca e272db0f7f content type parsing fix 2019-10-31 09:15:57 +01:00
Luca Deri 3580765ba3 Updated result 2019-10-31 09:04:52 +01:00
Luca cb79d89f83 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-10-31 08:58:03 +01:00
Luca 2e178d0f05 Improved libary initialization for custom protocols 2019-10-31 08:57:33 +01:00
Luca Deri edf0b171e2 Merge pull request #810 from catenacyber/fuzztarget
Adds fuzz target
2019-10-31 08:54:44 +01:00
Philippe Antoine 148506ede8 Adds fuzz target
And configur option enable-fuzztargets
2019-10-31 08:37:42 +01:00
Luca c0a08d9ca2 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-10-31 00:14:40 +01:00
Luca 4802987178 Initial work towards HTTP content-type export 2019-10-31 00:14:20 +01:00
Luca Deri 1cd7cf9990 Merge pull request #809 from catenacyber/fuzz1fix
Fix buffer over read in ndpi_handle_ipv6_extension_headers
2019-10-30 17:31:28 +01:00
Philippe Antoine 4830755fac Fix buffer over read in ndpi_handle_ipv6_extension_headers 2019-10-30 17:18:21 +01:00
Renan de Souza 124545e1ed Added twitch.tv website 2019-10-29 16:26:17 -03:00
Luca af01e61c89 Fixed invalid guess with the top defined protocol 2019-10-29 19:45:15 +01:00
Renan de Souza 9400a3b9ef Added ranges 20.180.0.0/14, 20.184.0.0/13 to Skype 2019-10-29 15:28:08 -03:00
Luca 6e314ea6d7 Added zabbix dissector 2019-10-29 19:25:46 +01:00
Oleg A. Arkhangelsky dfd962a8bc Change type
Use double type instead of float when printing time for CSV file.
Float type is too small for 64 bit int time so cast corrupts value.
2019-10-29 21:23:42 +03:00
Oleg A. Arkhangelsky cfcbfe1e8f Align CSV header with actual row data 2019-10-29 21:20:56 +03:00
Luca 0e54f87b18 Added telnet dissector
Improved data report
2019-10-29 19:12:42 +01:00
Oleg A. Arkhangelsky 9f5e5b90e5 Don't leak memory in live capture mode 2019-10-29 21:11:31 +03:00
Renan de Souza d619ba0857 Merge remote-tracking branch 'upstream/dev' into dev 2019-10-29 14:59:18 -03:00
Renan de Souza fbea243e56 Added brasilbandalarga.com.br and .eaqbr.com.br as EAQ on host_match[] 2019-10-29 11:48:23 -03:00
Renan de Souza d062386656 Added ^pastebin.com to host_match[] 2019-10-29 09:54:42 -03:00
Luca Deri d2f04f5bdf Fixed prototype 2019-10-29 10:32:13 +01:00
Luca Deri 42c8d3ac27 Added ability to defien custom protocols 2019-10-29 10:29:12 +01:00
Luca Deri d53563ab46 Implemented Microsoft range 2019-10-28 00:15:51 +01:00
Luca Deri 3d5f0ee315 Implemented stacked DPI decoding 2019-10-27 23:20:56 +01:00
Luca Deri ecdb7cdc55 Various improvements in particular for CapWAP and Bloomberg 2019-10-27 23:05:24 +01:00
Luca Deri fd38b752c4 Added capwap support 2019-10-27 19:03:23 +01:00
Luca Deri 64804a2a24 Fixed some false positivies with skype and stun-based protocols 2019-10-27 12:23:57 +01:00
Luca Deri b603cb4530 nDPI TLS improvements using the server certificate 2019-10-26 10:54:32 +02:00
Luca Deri 8f2c7c0638 Code cleanup 2019-10-26 08:28:14 +02:00
Luca Deri 044ba7697a Improved guess 2019-10-25 16:02:44 +02:00
Luca Deri e6bd64b3ea Improved HTTP reporting in ndpiReader 2019-10-25 15:56:47 +02:00
Renan de Souza 56db916833 Merge remote-tracking branch 'upstream/dev' into dev 2019-10-25 08:57:49 -03:00
Luca Deri 12cf0c4763 Further SMB improvements 2019-10-25 07:22:29 +02:00
Luca Deri 0a92e0f532 Ficed invalid SMBv1 detection 2019-10-25 07:00:57 +02:00
Luca Deri e6f9177fdb SSH dissection improvements 2019-10-25 00:05:18 +02:00
Luca Deri c3f6a4a57a SMTP and SSH dissection fixes 2019-10-24 23:28:32 +02:00
Renan de Souza 653c209253 Added 52.84.0.0/14 range as Amazon 2019-10-24 14:50:54 -03:00
Luca Deri 0974075fa0 Major cleanup
Removed ndpi_pref_http_dont_dissect_response and ndpi_pref_dns_dont_dissect_response as the ndpi_extra_dissection_possible() call will now handle everything
2019-10-24 19:48:55 +02:00
Renan de Souza 53e1ee3475 Changed Breeds descriptions in ndpi_get_proto_breed_name: Dangerous to Potentially Dangerous, added Dangerous (SMBv1) 2019-10-24 14:43:10 -03:00
Andrew Karelin 46affea055 Added some public DNSoverHTTPS servers
List of servers taken from https://github.com/curl/curl/wiki/DNS-over-HTTPS
2019-10-23 18:47:51 +05:00
Luca Deri 0ffe5cf1ff Added extra processing for POP and SMTP 2019-10-22 00:57:35 +02:00
Luca Deri d023f9bf2d Silenced TLS warning 2019-10-21 15:33:20 +02:00
Alfredo Cardigliano b2832cee8e rpm-sign fix for centos8 2019-10-21 09:26:40 +02:00
Luca Deri 960aa77b6d Reworked email metaata extraction 2019-10-21 01:27:07 +02:00
Luca Deri 58daa5ed7d Added POP3 metadata extraction 2019-10-20 23:06:05 +02:00
Luca Deri 437e8bd706 Implemented IMAP data extraction 2019-10-20 22:46:08 +02:00
Luca Deri 1a5c7daaf6 Implemented FTP user/pwd extraction 2019-10-20 21:49:45 +02:00
Luca Deri c1ba4764b5 Merge pull request #799 from ntop/ndpi_memleak_fix
Fix memory leak in ndpiReader.
2019-10-20 11:47:13 +02:00
Ravi Kerur 7cb3cb7353 Fix memory leak in ndpiReader.
Changes in library for strict function prototype.

Signed-off-by: Ravi Kerur <ravi.kerur@viasat.com>
2019-10-19 08:59:48 -07:00
Alfredo Cardigliano 604abaf81c Disabled debug_package 2019-10-18 12:30:42 +02:00
Luca Deri 8b157bb13f Ports are not JSON printed when 0 (e.g. with ICMP) 2019-10-17 00:38:42 +02:00
Luca Deri a8c71dcd20 Improved ndpi_flow2json 2019-10-17 00:29:30 +02:00
Alfredo Cardigliano 363a8f6282 Checking gmtime_r return value 2019-10-16 07:54:33 +02:00
Luca Deri 5b314cf1c3 Updated ndpi_flow2json 2019-10-16 12:31:41 +02:00
Luca Deri 02783bd7c6 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-10-15 23:06:41 +02:00
Luca Deri d5c918d258 Initial implementation of ndpi_flow2json APi call 2019-10-15 23:06:30 +02:00
emanuele-f 6b044335cd Implement nDPI patricia tree API 2019-10-15 01:22:17 +02:00
Luca Deri cbe20d3740 Fixes for API changes 2019-10-09 22:25:40 +02:00
Luca Deri 21bc84e837 Modified API signatures for
- ndpi_ssl_version2str()
- ndpi_detection_giveup()
2019-10-09 22:09:58 +02:00
Luca Deri 203359ed7f Fixed ndpi_ssl_version2str() 2019-10-08 23:57:56 +02:00
Luca Deri 24a26a2e94 Added NetBIOS metadata export 2019-10-08 23:40:56 +02:00
Luca Deri 0a7521060a Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-10-08 13:32:43 +02:00
Luca Deri 6b5a9aa992 Implemented Kerberos metadata extraction 2019-10-08 13:32:21 +02:00
emanuele-f 2dfdaa4d70 Replace strtok with reentrant version 2019-10-08 11:59:20 +02:00
emanuele-f f2504494fe New API functions: ndpi_load_categories_file and ndpi_load_category 2019-10-08 11:55:47 +02:00
Luca Deri 256858d2e5 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-10-08 11:36:11 +02:00
Luca Deri 10873bfe35 Added ndpi_get_l4_proto_name() and ndpi_get_l4_proto_info() API calls to print L4 protocol 2019-10-08 11:35:33 +02:00
emanuele-f e939f1c1d1 Fix leaks in ndpi_load_hostname_category when ac_automata_add fails 2019-10-08 11:07:59 +02:00
emanuele-f b2c8cb655c ndpi_load_hostname_category now performs strdup on the name argument
This simplifies the API as an application is not required to keep references to the strings to free
2019-10-08 10:23:00 +02:00
emanuele-f 4bdbf02c2d Fix invalid memory accesses 2019-10-04 14:35:51 +02:00
Luca Deri f83b2a0940 Fixed ndpi_extra_dissection_possible() 2019-10-04 14:13:53 +02:00
emanuele-f 76ec74cbf1 Fix out of bounds read in ndpi_match_custom_category 2019-10-04 11:54:05 +02:00
Luca Deri 8ae5da1472 Moved to 3.1 release 2019-10-04 07:45:57 +00:00
Luca Deri d8067cdc1d Extended changelog for v3 2019-10-04 09:42:43 +02:00
Luca Deri 67dbcb7375 Fixed invalid network searchthat caused categories to be set randomly 2019-10-04 07:19:20 +00:00
Luca Deri 3180f279c8 Merge pull request #792 from ntop/ndpi_streaming
Move Hulu and Disney+ to category match.
2019-10-03 22:35:31 +02:00
Simone Mainardi 1982e6cef9 Adds nDPI 3.0 changelog 2019-10-03 20:04:26 +02:00
emanuele-f d2f0a8d493 Add missing results 2019-10-03 16:26:14 +02:00
emanuele-f 37cb113c45 Handle TCP DNS replies and add is_reply flag 2019-10-03 16:19:15 +02:00
emanuele-f 04a97fa72b Fix DNS reply dissection issues 2019-10-03 15:52:55 +02:00
Ravi Kerur 8897aa6acb Move Hulu and Disney+ to category match.
Signed-off-by: Ravi Kerur <ravi.kerur@viasat.com>
2019-10-02 20:03:18 -07:00
Luca Deri c247a9d191 Code cleanup 2019-10-02 23:25:05 +02:00
Luca Deri 4fd7e5734a Manual merge of pull #769 2019-10-02 23:01:29 +02:00
Luca Deri 3dbf7bbabb Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-10-02 20:58:02 +02:00
Luca Deri 8f20e482ce Improved Viber protocol dissection
Added Hulu dissection
2019-10-02 20:57:17 +02:00
Luca Deri 329a33ed59 Merge pull request #791 from ntop/ndpi_hs
Fix crash due to recent code changes when using hyperscan and mining_…
2019-10-02 20:50:19 +02:00
Ravi Kerur 77edf1f680 Fix crash due to recent code changes when using hyperscan and mining_hosts.
Signed-off-by: Ravi Kerur <ravi.kerur@viasat.com>
2019-10-02 09:32:39 -07:00
emanuele-f 7f510c10fe Fix double free after b19bfa1e20 2019-10-02 16:26:52 +02:00
Luca Deri a559ba20b1 Merge pull request #790 from emanuele-f/managed_strings
Fixes leaks in ndpi_add_host_url_subprotocol
2019-10-02 15:30:33 +02:00
emanuele-f b19bfa1e20 Fixes leaks in ndpi_add_host_url_subprotocol
It is now possible to deallocate strings in ac_automata_release via
an additional parameter
2019-10-02 14:02:19 +02:00
Luca Deri 6a22bee2ca Added URL in results 2019-10-01 12:26:15 +02:00
Luca Deri e45237a93f Removed http:// from HTTP url
Reported URL in ndpiReader
2019-10-01 12:25:39 +02:00
Luca Deri a62cf10233 Added SSH handling in ndpi_extra_dissection_possible() 2019-09-30 23:23:52 +02:00
Luca Deri 1361c8fbdf Updated the logic of ndpi_is_subprotocol_informative() 2019-09-30 17:28:41 +02:00
Luca Deri f2a5bbef17 Reworked categories handling
Removed GenericProtocol and replaced with categories
Removed ndpi_pref_enable_category_substring_match option: substring matching is now default
2019-09-29 21:46:41 +02:00
Luca Deri 016d568c2b Merge pull request #787 from bohmax/explanation-file
Adding explanation file
2019-09-29 18:32:46 +02:00
bohmax 02609114d3 Add files via upload 2019-09-29 11:10:46 +02:00
Luca Deri 53ea3481ea Fixed ICMP parsing bug introduced by pull #770 2019-09-29 08:10:12 +02:00
Luca Deri 59b8399923 Merge pull request #778 from choyos/dev
Added support for dissection of DNP3 and IEC104 protocols
2019-09-29 08:02:49 +02:00
Luca Deri 3b06d2191c Merge pull request #770 from ntop/ndpi_icmp
Add icmp and icmp6 to calculate splt and bd values.
2019-09-29 07:59:02 +02:00
Luca Deri e9df2b187c Merge pull request #786 from ntop/ndpi_ac_vpn
Add Cisco anyconnect VPN signature.
2019-09-29 07:58:27 +02:00
Ravi Kerur bc1b266174 Add Cisco anyconnect VPN signature.
Signed-off-by: Ravi Kerur <ravi.kerur@viasat.com>
2019-09-28 19:52:53 -07:00
Luca Deri b0c68dc29b Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-09-27 17:35:08 +02:00
Luca Deri c839dcb74c Improved category handlign in subprotocols
Further DNS dissection fixes
Fixed WeChat invalid category
2019-09-27 17:34:22 +02:00
emanuele-f 05aa27e477 Add test for custom categories match on HTTP and SSL flows 2019-09-27 14:01:12 +02:00
Luca Deri 7c788f0779 Fixed partial string matches 2019-09-27 13:42:41 +02:00
emanuele-f 5099646938 Fix missing strdup on ndpi_load_hostname_category 2019-09-27 12:09:51 +02:00
emanuele-f 277b4a060a Add categories test PCAP 2019-09-27 10:15:20 +02:00
Luca Deri 17f560a1cd Added ndpi_extra_dissection_possible() API call 2019-09-26 23:50:02 +02:00
Luca Deri 194cdcf581 Fixed false positive with STUN detection 2019-09-26 22:18:56 +02:00
Luca Deri 7de9fd7170 Updated instagram check 2019-09-26 22:08:21 +02:00
Luca Deri 0e9918464b Added Zoom protocol support removing invalid STUN/Skype detections 2019-09-26 21:52:42 +02:00
Luca Deri ce40155377 Merge pull request #783 from ntop/ndpi_ps_vue
Ndpi ps vue
2019-09-26 19:37:49 +02:00
emanuele-f 4dc60f151d Restructure DNS search code 2019-09-26 13:20:13 +02:00
emanuele-f 937dcb38ea Fix bad protocols returned in ndpi_detection_giveup when detection is already complete 2019-09-26 12:15:49 +02:00
Ravi Kerur 4170eb3a1d Modified logic and test pcap file included.
Signed-off-by: Ravi Kerur <ravi.kerur@viasat.com>
2019-09-25 16:21:22 -07:00
Ravi Kerur c3a5650071 Initial PS_VUE support based on public IP.
Signed-off-by: Ravi Kerur <ravi.kerur@viasat.com>
2019-09-25 15:20:19 -07:00
Luca Deri 62fa0908a3 Improved packet datastructure cleanup after packet processing
Cleaned up code of steam dissector
Added steam test pcap
2019-09-25 23:59:22 +02:00
Ravi Kerur d2c0b9872d Initial Zoom detection based on IP and ports.
Following IP and ports are reported by Zoom support
to open up in firewall.

https://support.zoom.us/hc/en-us/articles/201362683-Network-Firewall-or-Proxy-Server-Settings-for-Zoom

Signed-off-by: Ravi Kerur <ravi.kerur@viasat.com>
2019-09-25 12:59:34 -07:00
Alfredo Cardigliano 3709e673d9 New serializer APIs ndpi_serialize_start_of_block / ndpi_serialize_end_of_block (JSON only) 2019-09-25 17:19:33 +02:00
Luca Deri 0b62c0ad45 Merge pull request #781 from marco-testa/dev
Update dissector stun.c
2019-09-24 16:39:46 +02:00
Luca 0ed679e795 Improves IAT calculation 2019-09-24 16:37:42 +02:00
marco-testa b706730e10 Loaded in the wrong place 2019-09-24 09:45:34 +02:00
marco-testa bd83847027 Changed the packets handling with STUN msg_type > 0x000C and other fixes.
Eliminated double call to the ndpi_int_stun_add_connection function.
2019-09-24 09:43:47 +02:00
marco-testa d3348ff730 Changed the packets handling with STUN msg_type > 0x000C and other fixes.
Eliminated double call to the ndpi_int_stun_add_connection function.
2019-09-24 09:41:58 +02:00
Ravi Kerur e29032085d Merge branch 'dev' of https://github.com/ntop/nDPI into ndpi_icmp 2019-09-23 17:06:43 -07:00
Luca Deri 19dbcaa3a9 Fixes #777 2019-09-23 18:04:55 +02:00
Luca Deri 3f56b2476a Merge pull request #780 from bohmax/fix_wrap
Fix wrap
2019-09-23 17:48:02 +02:00
bohmax 8246467022 Add files via upload 2019-09-23 17:40:34 +02:00
bohmax 90c28e9d63 Delete ndpi_typestruct.py 2019-09-23 17:40:16 +02:00
Luca Deri 64513b1c0e Rewritten qq dissector 2019-09-23 17:39:58 +02:00
Ravi Kerur e8d501158e Restore protocol extraction to original method.
Add microseconds to time series if milliseconds returns 0.

Signed-off-by: Ravi Kerur <ravi.kerur@viasat.com>
2019-09-21 10:38:04 -07:00
Ravi Kerur d92015a092 Merge branch 'dev' of https://github.com/ntop/nDPI into ndpi_icmp 2019-09-21 10:16:10 -07:00
Luca Deri 5c2e495792 ndpiReader now report protocol category and breed
Modified TikTok category to Fun
2019-09-21 09:52:44 +02:00
Luca Deri 421f819b8a Added signal test pcap 2019-09-21 09:40:20 +02:00
Luca Deri ab9246f51f Adedd DTLS check in STUN
Uodated (C)
2019-09-21 09:16:56 +02:00
Ravi Kerur e2e05cd499 Merge branch 'dev' of https://github.com/ntop/nDPI into ndpi_icmp 2019-09-20 10:54:58 -07:00
Luca bdcc6dc173 STUN protocol dissector code cleanup 2019-09-20 18:47:39 +02:00
Luca f9716c5139 Unified WhatsApp Video and Audio under WhatsAppCall 2019-09-20 17:37:41 +02:00
choyos bae36d2e25 added dnp3 and iec104 pcaps for tests 2019-09-20 13:35:46 +02:00
choyos cbda4c2af1 Added support for industrial protocols DNP3, IEC 60780-5-104 2019-09-20 13:29:49 +02:00
Luca f678486393 Improved STUN-based protocol heuristic both in terms of accuracy and packets necessary for the detection 2019-09-20 07:29:02 +02:00
emanuele-f 518df786ed Add ndpi_parse_ip_string and ndpi_get_ip_string API 2019-09-19 16:08:47 +02:00
Luca Deri 16da3f58cc Fixed JA3C compuration wtih SSL (no TLS) traffic 2019-09-19 00:01:44 +02:00
Luca Deri cf0ca7db51 Improved STUN caching 2019-09-18 23:31:25 +02:00
Luca Deri 0d8a161d15 Further line update 2019-09-18 23:24:41 +02:00
Luca Deri 487110693c Line protocol update 2019-09-18 23:23:13 +02:00
Luca Deri 3287dfbff4 Added dissection of TLS version in supported_versions of Hello packets 2019-09-18 14:20:01 +02:00
Luca Deri ac1f7d71dd New instagram testing set 2019-09-18 12:23:55 +02:00
Luca Deri a716d5a317 Improved Line protocol heuristic 2019-09-18 12:14:23 +02:00
Ravi Kerur 7e3fd16b0d Merge branch 'dev' of https://github.com/ntop/nDPI into ndpi_icmp 2019-09-17 17:01:17 -07:00
emanuele-f e152e687a5 Fix notBefore/notAfter broken in previous commit 2019-09-17 18:43:11 +02:00
emanuele-f 10e560a508 Fix notBefore/notAfter issues 2019-09-17 18:09:17 +02:00
Luca Deri 2fc68a441d Improved -C output 2019-09-17 12:07:00 +02:00
Luca Deri 64ed0ac6b3 Various STUN improvements 2019-09-17 00:11:17 +02:00
Ravi Kerur ce79626679 Merge branch 'dev' of https://github.com/ntop/nDPI into ndpi_icmp 2019-09-16 13:56:17 -07:00
Luca Deri 8f4e8d8254 Code cleanup 2019-09-15 22:56:55 +02:00
Luca Deri 62c9766cc1 Warnign fix 2019-09-15 22:43:43 +02:00
Luca Deri f0013e826e Added python bindings for nDPI 2019-09-15 22:32:18 +02:00
Luca Deri 5fd79567f0 Merge pull request #767 from LocutusOfBorg/fix-cross-ftbfs
Fix cross-build from source, because of direct pkg-config check calls
2019-09-15 20:57:13 +02:00
Luca Deri 11e9fd8cef As TLS certificate fingerprint is computed, TLS without certificate protocol has been removed
Various improvemenets in detection quality
2019-09-15 12:18:24 +02:00
Luca Deri 2b0945b88d TLS disection improvements 2019-09-15 10:56:08 +02:00
Ravi Kerur 23780a655d Fix protocol for v4 and v6.
Signed-off-by: Ravi Kerur <ravi.kerur@viasat.com>
2019-09-14 18:38:18 -07:00
Ravi Kerur 9f83bf4624 Add icmp and icmp6 to calculate splt and bd values.
Signed-off-by: Ravi Kerur <ravi.kerur@viasat.com>
2019-09-14 10:06:37 -07:00
Luca Deri 00e639d513 TLS certificate hash is not reported 2019-09-14 15:00:52 +02:00
Luca Deri 659f75138c TLS cerficate hash calculation 2019-09-14 12:38:58 +02:00
Alfredo Cardigliano d7c2ced475 Fix ndpi_serializer_rollback_snapshot with single records 2019-09-11 20:04:32 +02:00
Helmut Grohne 2e01a1fe53 Fix cross-build from source, because of direct pkg-config check calls
ndpi fails to cross build from source, because configure.seed hard codes
the build architecture pkg-config. It is best to use PKG_CHECK_MODULES
instead. Please consider applying the attached patch.

See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=939286
2019-09-11 19:11:42 +02:00
Luca Deri e687a73cb2 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-09-11 17:14:52 +02:00
Luca Deri 6966e0d19b Added STUN check to avoid false positives
Added fingerprint comments in SSH/TLS
Added netflow test pcap
2019-09-11 17:13:49 +02:00
Alfredo Cardigliano 7d9ed162c7 Optimized memory utlization resizing serialization buffer buffer 2019-09-11 16:32:40 +02:00
Alfredo Cardigliano 848c9db4fb Added ndpi_init_serializer_ll() to specify low-level settings for optimizing memory allocations 2019-09-11 16:10:18 +02:00
Alfredo Cardigliano 16fe2574b1 Fix ndpi_serializer_rollback_snapshot with JSON 2019-09-11 11:32:11 +02:00
Luca Deri c3c975a51a Renamed snapshot API call 2019-09-11 09:45:29 +02:00
Alfredo Cardigliano a265b50a66 Support for nDPI serializer snapshot set/reset 2019-09-10 20:24:36 +02:00
Luca Deri 67a1455619 ndpi_serializer_set_buffer_len() fix 2019-09-10 23:55:00 +02:00
Luca Deri 844e2bab3b warnng fix 2019-09-10 22:46:37 +02:00
Luca Deri b1583cc29e Added ndpi_serializer_set_buffer_len() API call 2019-09-10 22:45:52 +02:00
Luca Deri 0f9107bef3 Fixed timezone conversion 2019-09-10 22:15:23 +02:00
Luca Deri 068611e8ba Changes in validity format 2019-09-10 21:44:42 +02:00
Luca Deri 1f417fe67d Validity format fix 2019-09-10 21:44:04 +02:00
Alfredo Cardigliano 8186b72c11 Added ndpi_deserialize_clone_all to the public API 2019-09-10 19:30:52 +02:00
Alfredo Cardigliano 4c0cc80771 New API ndpi_deserialize_clone_all to clone an record from TLV to a different format (e.g. JSON) 2019-09-10 18:57:28 +02:00
Alfredo Cardigliano 9c8c165a2c Added ndpi_deserialize_clone to the public API 2019-09-10 17:32:13 +02:00
Alfredo Cardigliano 84ce1861a2 New API ndpi_deserialize_clone to clone an element from a deserialized TLV to another TLV 2019-09-10 17:30:15 +02:00
Alfredo Cardigliano 990d06d10c TLC EOR/Unknown items size 2019-09-10 16:40:29 +02:00
Alfredo Cardigliano 3e1c811780 Added ndpi_deserialize_get_format() to read the serialization format 2019-09-10 15:02:02 +02:00
Alfredo Cardigliano b96b618443 nDPI serialization: uint key compression, new deserialization API 2019-09-10 14:47:47 +02:00
Luca Deri 65bdd0de9a Uddated results 2019-09-10 00:43:03 +02:00
Luca Deri 086c511a11 Parsed TLS certificate validity 2019-09-10 00:34:48 +02:00
Luca Deri c5fffce96c Added IMO dissector
Fixed TLS/UDP detection
Reduced UBNT buffer for saving space
2019-09-09 21:17:53 +02:00
Luca Deri d457b07f9f Refreshed Tor node list 2019-09-08 21:42:33 +02:00
Luca a0a8149833 Skype STUN enhancements 2019-09-06 10:41:53 +02:00
Luca 241cfb319b Enhanced Signal detection 2019-09-05 10:45:08 +02:00
Luca 9b80c6314d Enhanced STUN cache 2019-09-05 10:23:07 +02:00
Luca Deri 3d21c33409 Compilation fix when JSON-C is not present 2019-09-03 18:45:51 +02:00
Luca b3c88a7e24 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-09-03 18:39:36 +02:00
Luca 886d575157 Added -C to generate CSV analysis files
Improved IAT and byte distribution
2019-09-03 18:38:54 +02:00
Alfredo Cardigliano 2a8f9c3a81 Return value fix in ndpi_deserialize_* 2019-09-03 11:44:32 +02:00
Alfredo Cardigliano 2cbad47d7b Added vxlan header 2019-09-03 02:26:40 +02:00
Luca Deri eb154bac69 Added ndpi_alloc_data_analysis APi call 2019-09-02 22:12:46 +02:00
Alfredo Cardigliano ab178e62f5 Transparently handle int8/16 deserialization 2019-09-02 17:29:08 +02:00
Luca Deri bc4c48fa0b Added packetIDs with -P 2019-09-02 14:47:37 +02:00
Luca Deri 5d5d056849 Added payload memory free 2019-09-02 14:14:13 +02:00
Luca Deri f1ad951bf6 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-09-02 14:00:45 +02:00
Luca Deri d0436e7b29 Added new parameter to -P 2019-09-02 14:00:18 +02:00
Alfredo Cardigliano 2616dcb424 TLV signed int compression 2019-09-02 12:33:16 +02:00
Luca Deri eb6afa17c0 Added ndpi_serialize_string_binary() call 2019-09-02 10:18:54 +02:00
Alfredo Cardigliano 77409a3833 TLV: encoding uint 32/64 as 16/8 bit whenever possible 2019-08-30 11:00:04 +02:00
Alfredo Cardigliano 49a02c9b96 TLV deserialization of uint16/uint8/int16/int8 when deserializing uint32/uint64/int32/int64 2019-08-30 10:31:54 +02:00
emanuele-f 1bf23dc2a8 Fix mangled stack by invalid sscanf 2019-08-29 22:40:29 +02:00
emanuele-f fce2cd9817 Fix memory corruptions in ndpi_node handler
Crash occurred with clang version 6.0.0-1ubuntu2 on ubuntu 18
2019-08-29 20:13:19 +02:00
emanuele-f 2c4495e483 Add missing lib 2019-08-29 17:55:51 +02:00
emanuele-f 6781485c42 Warning fix 2019-08-29 16:49:44 +02:00
emanuele-f d27e7e2786 Add missing pack_on on ajp struct 2019-08-29 16:47:33 +02:00
Luca Deri 68f6310222 AJP crash fix 2019-08-29 16:17:43 +02:00
Luca c328452687 Average calculation fix 2019-08-29 16:08:18 +02:00
Luca Deri 536b520750 Added stub for ndpi_free_memory_at_termination() (TODO) 2019-08-29 15:47:34 +02:00
Luca Deri 8b8fd25626 Compilation fix on systems without JSON-C 2019-08-29 15:35:29 +02:00
emanuele-f e98eb294c7 Add missing json cleanup 2019-08-29 15:25:52 +02:00
Luca b1270fc7bb Uodated results 2019-08-29 15:23:01 +02:00
Luca 9eb1676b4b Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-08-29 15:22:46 +02:00
emanuele-f aea91781aa Fix alignment issue on IPv6 structs 2019-08-29 14:50:17 +02:00
Luca 9a6f6d9fe4 Implemented IAT (Inter Arrival Time) stats 2019-08-29 13:40:44 +02:00
Luca e4e40e3c70 Added entropy, average, stddev, variance, bytes ratio calculation 2019-08-28 14:02:39 +02:00
Luca Deri 84aeee49bd Disable debug code 2019-08-27 10:45:33 +02:00
Luca Deri 6011790d79 Entropy calculation example 2019-08-27 10:44:41 +02:00
Luca 2acffb4163 Initial work towards traffic classification 2019-08-27 12:56:15 +02:00
Luca Deri b837f3753d SSH hash fix 2019-08-26 19:13:45 +02:00
Alfredo Cardigliano d043cd2ac7 Serialization/deserialization of int32/int64/float with uint key 2019-08-26 12:55:23 +02:00
Alfredo Cardigliano dbf8c24ecb Fix ndpi_serialize_end_of_record in JSON mode 2019-08-24 03:08:07 +02:00
Alfredo Cardigliano c50709a8d0 Fix ndpi_reset_serializer 2019-08-24 02:56:17 +02:00
emanuele-f 1231b81839 Add more length checks in HASSH
This to prevent possible crashes on invalid packets
2019-08-23 11:36:35 +02:00
Luca Deri d2fe21ddff Enhanced -P in ndpiReader so that it can now specify additional parameters 2019-08-22 23:42:07 +02:00
Luca Deri 2134f83948 Warning fixes 2019-08-22 20:04:15 +02:00
Luca Deri 3d3cae1f20 SSH test file 2019-08-22 19:36:36 +02:00
Luca Deri 1e8a43d51e Merge pull request #758 from tansly/wireguard
Add support for WireGuard VPN
2019-08-22 19:31:51 +02:00
Luca Deri e7c0ac37d8 Implemented HASSH (https://github.com/salesforce/hassh) 2019-08-22 19:25:58 +02:00
Luca Deri 385e848a0f Disabled weak (port-based) teamspeak detection 2019-08-22 15:37:31 +02:00
Luca Deri 9f42a4482a Initial work workards payload analysis 2019-08-21 19:01:56 +02:00
Luca fdeecf5d41 Implemented STUN cache to enhance matching of STUN-based protocols 2019-08-12 14:16:28 +02:00
Luca 51ac583d83 Improved google duo detection 2019-08-12 11:59:57 +02:00
emanuele-f e226f99e4e Fix possible intoaV4 1 byte overflow 2019-08-09 17:02:15 +02:00
emanuele-f f73dc61ff8 Add NDPI_PROTOCOL_DANGEROUS classification 2019-08-09 10:04:56 +02:00
Luca 90547c4fd2 Added SMBv1 test file 2019-08-08 23:33:40 +02:00
Luca c6e832be18 Various TLS/STUN improvememnts 2019-08-08 18:09:12 +02:00
Luca cce8a6026f Reworked SSL/TLS field naming 2019-08-08 15:20:05 +02:00
Luca 419160f351 Implemented DTLS support
Renamed ssl to tls
2019-08-08 12:45:27 +02:00
Luca Deri 3c4a959c1e Updated results when adding client hello 2019-08-07 20:06:31 +02:00
Luca 800deb3f54 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-08-07 20:00:59 +02:00
Luca 49f5beab84 Added client SSL hello 2019-08-07 19:35:33 +02:00
emanuele-f a6fe581d48 Change SMBv1 breed to potentially dangerous
https://techcommunity.microsoft.com/t5/Storage-at-Microsoft/Stop-using-SMB1/ba-p/425858
2019-08-07 12:39:07 +02:00
Luca Deri ff1234cce3 Changes in QUIC dissector to avoid miing QUIC with STUN 2019-08-06 23:13:47 +02:00
Luca Deri 32665e6417 Removed non-ASCII chars from JSON header 2019-07-31 00:11:40 +02:00
Luca Deri b2f7a52f98 Serialization fix 2019-07-30 23:46:20 +02:00
Luca Deri 4ebc8d54f4 Compilation fix 2019-07-30 22:51:54 +02:00
Luca Deri 9c3e623bb4 Split serialization code from ndpi_utils 2019-07-30 22:48:55 +02:00
Luca Deri e1b7005c2b Minro change 2019-07-30 10:28:30 +02:00
Luca Deri 398e9718ea JSON serialization fixes 2019-07-30 08:04:39 +02:00
Luca a0be9ea639 Fixed wrong detection of RTP as WhatsApp voice 2019-07-29 16:28:56 +02:00
Luca Deri fbe87ed640 Fixed CSV initialization 2019-07-29 00:41:29 +02:00
Luca Deri 23af4d8211 Initial CSV implementation 2019-07-29 00:33:48 +02:00
Luca Deri bfc3a00e6c Fix for ndpi_serialize_string_uint32_format with JSON 2019-07-28 23:02:55 +02:00
Luca Deri 40f55e2d06 Added string/float support in (de)serialization 2019-07-27 17:48:23 +02:00
Luca Deri 3e044a759b Code cleanup 2019-07-27 17:24:32 +02:00
Luca Deri 5c38e7b126 Better messenger traffic guess 2019-07-25 17:23:32 +02:00
Alfredo Cardigliano 32c61d3ea1 RPM spec cleanup 2019-07-25 14:23:11 +02:00
Alfredo Cardigliano ea80c4dacc Removed conflict with nprobe-dev 2019-07-25 11:59:01 +02:00
Luca Deri 7f66f1053f Updated google docs detection in wechat.pcap 2019-07-25 09:13:46 +02:00
Luca Deri 734305e111 Updated Google docs category 2019-07-25 08:58:44 +02:00
Luca Deri 3369716fc3 Added SPLT (sequence of packet length and time) and BD (byte distribution) with -J 2019-07-24 23:55:07 +02:00
Yağmur Oymak cdf61de129 Add test for WireGuard 2019-07-24 19:18:20 +03:00
Yağmur Oymak 786ea3c0f9 Merge branch 'dev' into wireguard
Conflicts:
	src/include/ndpi_protocols.h
	src/lib/ndpi_main.c

Get upstream developments.
2019-07-24 19:16:44 +03:00
Yağmur Oymak 9a899c54c9 Harden WireGuard detection
Exploit the fixed size handshake messages and sender/receiver indices.
2019-07-24 18:51:29 +03:00
Luca Deri 2148568347 Added int32/int64 (with sign) serialization/deserialization 2019-07-24 15:20:31 +02:00
Luca Deri 5c6c64ed1d Warnign fix 2019-07-24 14:16:38 +02:00
Luca Deri b99ef159ed Added missing result for nest 2019-07-24 11:32:15 +02:00
lucaderi a9ce7ccb83 Packages fixes 2019-07-24 08:47:36 +02:00
Luca Deri b715467da6 Added -e option to ndpiReader for searchign human readeable strings lenght
Default human readeable strings lenght is not 5 chars (used to be 8)
2019-07-24 00:25:00 +02:00
Luca Deri 63173e7360 Updated results with new dissection 2019-07-24 00:13:07 +02:00
Luca Deri d398b0aba6 Simplified makefile
Fixed warning
2019-07-23 22:45:31 +02:00
Luca Deri 8ff68a58f2 Merge pull request #759 from themadprofessor/patch-2
Create CONTRIBUTING.md
2019-07-23 22:33:29 +02:00
Luca Deri f55391f99d Merge pull request #760 from ntop/nDPI-SPLT-BD
Ndpi splt bd
2019-07-23 22:32:59 +02:00
Ravi Kerur a262f8dd29 Add SPLT and BD classification to the lib.
Signed-off-by: Ravi Kerur <ravi.kerur@viasat.com>
2019-07-23 10:00:34 -07:00
Ravi Kerur c6ca6ec712 Add new datastructures.
Signed-off-by: Ravi Kerur <ravi.kerur@viasat.com>
2019-07-23 09:56:32 -07:00
Ravi Kerur 1ec160d4c9 nDPI ovs changes.
Signed-off-by: Ravi Kerur <ravi.kerur@viasat.com>
2019-07-23 09:56:24 -07:00
Ravi Kerur 6b8234d938 Xbox and PS4 static port classification.
Signed-off-by: Ravi Kerur <ravi.kerur@viasat.com>
2019-07-23 09:56:15 -07:00
Ravi Kerur b6db991710 Amazon video static classification.
Signed-off-by: Ravi Kerur <ravi.kerur@viasat.com>
2019-07-23 09:56:05 -07:00
Ravi Kerur 6ff9b8b90e Add default port based iperf classification.
Signed-off-by: Ravi Kerur <ravi.kerur@viasat.com>
2019-07-23 09:55:32 -07:00
Stuart Reilly e781ddfa43 Create CONTRIBUTING.md
Add simple contribution guidelines.
2019-07-23 10:47:25 +01:00
Yağmur Oymak a0d8ce4159 Add support for WireGuard VPN 2019-07-23 12:00:23 +03:00
Luca Deri 974abaa275 Merged Google Hangout and Duo as they are pretty similar from the network standpoint and from the features they implement 2019-07-22 23:24:40 +02:00
Luca Deri ccea4669de Improved WhatsApp detection 2019-07-22 23:06:25 +02:00
Luca Deri db36294dcb Warnign fix 2019-07-22 21:42:55 +02:00
Alfredo Cardigliano b13780c6e0 Serialization of arrays in JSON format 2019-07-22 17:24:04 +02:00
Luca Deri 6d00b59bb8 Compilarion fix
Updated SIP result
2019-07-22 16:08:44 +02:00
Luca Deri 72b05fed75 STUN, Hangout, Duo dissection improvements 2019-07-21 08:17:28 +02:00
Alfredo Cardigliano e2d2ca5fe0 Added braces in JSON object serialization 2019-07-19 19:55:21 +02:00
Alfredo Cardigliano d43cfe1ead JSON serialization fixes in buffer offset and room computarion, string escape 2019-07-19 17:33:17 +00:00
Luca Deri 11004de056 Restored serializerUnitTest() after fixes 2019-07-19 16:54:48 +02:00
Alfredo Cardigliano f3faaec9d8 serializer char* parameters are now const 2019-07-19 14:36:41 +00:00
Alfredo Cardigliano 8350b7269f Fix Makefile.dpdk 2019-07-19 10:00:12 +00:00
Luca Deri 6bce0c4a9c Added support for Google Duo 2019-07-19 00:48:07 +02:00
Luca Deri 4bda18a079 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-07-18 15:14:06 +02:00
Luca Deri 46499a53aa Improved Facebook messnger mobile detection 2019-07-18 15:13:48 +02:00
Luca Deri 57ef7c1276 Added QUIC 046 support 2019-07-18 15:00:24 +02:00
Luca Deri b8867642fc Refresh after data leak detection 2019-07-18 11:49:53 +02:00
Luca Deri e902478bd3 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-07-18 11:38:43 +02:00
Luca Deri b50635cb89 Renamed ndpi_util.c -> reader_util.c
Added crash fix with IPv6
2019-07-18 11:38:16 +02:00
Luca Deri cbb31fa204 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-07-18 11:26:17 +02:00
Luca Deri 20bac32e1f Improvements for dta dissection 2019-07-18 11:24:44 +02:00
Luca Deri f394102b80 Merge pull request #752 from marcopiangatello/dev
ndpi_has_human_readeable_string Test
2019-07-18 07:32:38 +02:00
Alfredo Cardigliano dc375aed81 Corruptioon fix 2019-07-18 00:01:52 +02:00
Alfredo Cardigliano c85f7de2c1 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-07-17 22:06:15 +02:00
Alfredo Cardigliano ac3a618458 Improved deserialization safety checks 2019-07-17 22:06:00 +02:00
Alfredo Cardigliano 6081e46ad9 Fix memory corruption (reworked space computation and safety checks) 2019-07-17 21:20:51 +02:00
Alfredo Cardigliano c990d6bbd9 Deserialization boundary check fix, new end_of_record type 2019-07-17 17:58:14 +02:00
marcopiangatello 4ad12dcd19 Update ndpiReader.c 2019-07-17 17:09:33 +02:00
marcopiangatello dea4f784e2 Update ndpiReader.c 2019-07-17 16:55:49 +02:00
marcopiangatello 20c3d90163 Update ndpiReader.c 2019-07-17 16:53:24 +02:00
marcopiangatello 7053721aae Add files via upload 2019-07-17 16:50:38 +02:00
marcopiangatello adcb05d213 Delete ndpiReader.c 2019-07-17 16:50:25 +02:00
marcopiangatello 11639dcc8b Update ndpiReader.c 2019-07-17 16:34:13 +02:00
marcopiangatello 44dd09b438 Add files via upload 2019-07-17 16:08:22 +02:00
marcopiangatello 7280fc7d31 Delete ndpi_util.h 2019-07-17 16:08:00 +02:00
marcopiangatello e93ccab279 Delete ndpi_util.c 2019-07-17 16:07:53 +02:00
marcopiangatello 5294945554 Delete ndpiReader.c 2019-07-17 16:07:45 +02:00
Luca Deri 9ca87a620a Added extar check
Warning fix
2019-07-17 15:31:48 +02:00
Luca Deri b54863edc6 Merge pull request #747 from themadprofessor/travis-gcc
Add more compilers to travis
2019-07-17 15:26:47 +02:00
Stuart Reilly 065d0c7e9f Add ubuntu toolchain to clang builds 2019-07-17 12:14:45 +01:00
Stuart Reilly e2fcb942e8 Add clang-7 2019-07-17 11:44:44 +01:00
Stuart Reilly cfdf832d3a Remove brew update 2019-07-17 11:44:44 +01:00
Stuart Reilly 098c4228fb Make travis more explicit 2019-07-17 11:44:44 +01:00
Stuart Reilly 79a0cfbcdb Remove gcc7, add clang 8 2019-07-17 11:44:44 +01:00
Stuart Reilly e923ca4f17 Add gcc-8 and gcc-9 2019-07-17 11:44:44 +01:00
Stuart Reilly 95ba3fa142 Add gcc-7 to travis.yml 2019-07-17 11:44:38 +01:00
Alfredo Cardigliano 2fcc4f3e73 Added ndpi_serialization_uint32_uint64 and ndpi_serialization_string_uint64 2019-07-17 10:10:41 +00:00
Alfredo Cardigliano 1657686eb4 Moved json2tlv to ntopng 2019-07-17 08:55:46 +00:00
Alfredo Cardigliano c0835968af ZMQ export in json2tlv 2019-07-16 16:04:04 +00:00
Alfredo Cardigliano 4b2fbdb277 json2tlv tool to convert from JSON to TLV and run performance tests 2019-07-16 13:31:26 +00:00
Alfredo Cardigliano 2ce4adf09d Added ndpi_serialization_string_uint32 2019-07-16 12:31:41 +02:00
Luca Deri 1777841ec1 Line dissection improvements 2019-07-16 00:02:34 +02:00
Luca Deri f570eccf81 Fixes #741 2019-07-15 23:53:28 +02:00
Luca 67dfddd065 Line fix 2019-07-15 16:25:53 +02:00
Luca 9a4d01cb26 Lenght fix 2019-07-15 16:17:23 +02:00
Luca Deri 5c758d4564 Reworked output
-v 1 now prints only unknown flows
-v 2 now prints all flows
2019-07-15 14:45:25 +02:00
emanuele-f fc531334ae Remove Werror 2019-07-15 09:31:15 +02:00
Luca Deri 5a018e0bc0 Added Line protocol dissection
Add fix for discarding STUN over TCP flows
2019-07-15 00:19:45 +02:00
Luca Deri f8fb1f838c Fixes #740 2019-07-14 12:25:08 +02:00
Luca Deri 7b16c68e92 JA3 format cleanup 2019-07-13 19:32:51 +02:00
Luca Deri af2dc0860c Merge pull request #734 from glpaparelli/dev
fixed the printing layout, even with IPv6
2019-07-13 18:44:03 +02:00
Luca Deri acd3db13f8 Fixes #735 2019-07-13 18:39:18 +02:00
Luca Deri c8a867bf29 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-07-13 18:38:20 +02:00
Luca Deri 17c49b2e6d Updated test resultss after export changes 2019-07-13 18:37:57 +02:00
Luca Deri 9bdc021f98 Merge pull request #736 from themadprofessor/patch-1
Fix invalid function prototype
2019-07-13 18:36:09 +02:00
Luca Deri c65025f7b0 Merge pull request #737 from themadprofessor/dev
Add compiler warnings and fix them
2019-07-13 18:34:40 +02:00
Luca Deri ea32d535eb initial work towards simple serilizer/deserialier 2019-07-13 18:32:22 +02:00
Stuart Reilly 92c769f206 Solve remaining warnings 2019-07-12 16:41:39 +01:00
Stuart Reilly 343ee0c9b6 Merge branch 'dev' of github.com:themadprofessor/nDPI into github 2019-07-12 16:28:38 +01:00
Stuart Reilly 8b1c68aed3 Add compiler warnings and fix them 2019-07-12 16:01:22 +01:00
Stuart Reilly 929cbbb7fc Add compiler warnings and fix them 2019-07-12 15:59:11 +01:00
Stuart Reilly 22e323528e Fix invalid function prototype
Solves a warning when compiling with strict-prototypes.
2019-07-12 15:02:20 +01:00
Giulio Paparelli 52533c3b66 all fixed 2019-07-12 12:06:10 +02:00
Luca Deri 563215a05e Compilation fix 2019-07-12 07:24:07 +02:00
Luca Deri 4ac60d88d7 Improved whatsapp dissection 2019-07-11 22:49:24 +02:00
Luca Deri 3e2e616507 Dissection improvents
Fixed QUIC dissection
Disabled debug message
2019-07-11 22:35:50 +02:00
Luca Deri da7e595fcf Fixed uninitialized variable that caused weird errors 2019-07-10 19:27:27 +02:00
Luca Deri 829a284d48 Fix that caused nDPI to provide false positive results with pattern matching 2019-07-10 16:08:12 +02:00
Luca Deri c0d9dc9969 Typo 2019-07-03 18:03:44 +02:00
Luca Deri cc15389729 Improved protocol guess 2019-07-02 23:25:55 +02:00
Luca edd4f2e37d Remvoed debug info 2019-07-02 15:28:40 +02:00
Luca Deri 8205f2c6f1 Various TLS improvements 2019-07-02 00:05:17 +02:00
Luca Deri dd9db5d0b7 SSL ciphers improvements 2019-07-01 23:25:48 +02:00
Luca Deri 9590b18fdd Fixed invalid changes in ndpi_get_proto_category() and ndpi_get_proto_name() dur to https://github.com/ntop/nDPI/pull/731/commits/4b717276614e20feada22e6fe29ea2099ba9fd37 2019-07-01 23:16:43 +02:00
Luca Deri 7cb7a12f96 Merge pull request #731 from themadprofessor/dev
Stability Fixes
2019-07-01 20:02:49 +02:00
Stuart Reilly 5ce5bae99d Merge remote-tracking branch 'upstream' into dev 2019-07-01 17:49:14 +01:00
Luca Deri 8fdef484f6 Minor change 2019-07-01 12:51:55 +02:00
Luca Deri 65e842effa Merge pull request #730 from hoermto/hoermto-patch-alpine
Alpine linux: "ln -F" not supported in alpine
2019-06-30 15:34:00 +02:00
Luca Deri 2c43b61798 Added DNSoverHTTPS detection 2019-06-30 12:36:58 +02:00
Luca Deri a1ac31062d Added ndpi_has_human_readeable_string for detecting human-readeable strings in buffers 2019-06-30 11:59:51 +02:00
Luca Deri 06b05e21c6 Improvemnts in SSL certificate detection 2019-06-30 08:19:51 +02:00
Luca Deri 3f460ad5a9 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-06-29 17:34:45 +02:00
Luca Deri e9e647f043 ntohl Portability fixes 2019-06-29 17:34:08 +02:00
Luca Deri e8c84f3c5d Initial implementation of datasaver protocol support 2019-06-29 13:04:06 +02:00
Stuart Reilly f3161309ff Fix missing braces 2019-06-28 16:21:49 +01:00
Stuart Reilly ef1b8e36a0 Fix potential NULL dref in btlib.c 2019-06-28 16:12:42 +01:00
Stuart Reilly 79634a6789 Fix two resource leaks 2019-06-28 16:11:33 +01:00
Stuart Reilly d8ec0a9229 Move NULL checks to start of functions 2019-06-28 16:08:36 +01:00
Stuart Reilly bc1b4885de Fix potential NULL deref in libcache 2019-06-28 16:00:32 +01:00
Stuart Reilly 4b71727661 Fix two potential OOB reads in ndpi_main.c 2019-06-28 15:55:00 +01:00
Stuart Reilly cc39ef0946 Remove duplicate defintions and fix function prototypes 2019-06-28 15:53:11 +01:00
hoermto e47c0c3e49 Makefile: ln -F not supported in alpine
removed -F option from makefile for alpine support. Actually not sure the relevance of -F flag for other OS
2019-06-28 15:54:59 +02:00
Luca Deri 81d3fc8a24 Dissection improvements 2019-06-27 16:38:36 +02:00
Luca Deri cc51253007 Removed debug message 2019-06-25 16:30:51 +02:00
Luca Deri a4d797f832 WhatsApp improvements for video/audio detection 2019-06-25 15:18:49 +02:00
Luca Deri 6ed8ccfa46 Merge pull request #723 from lance-wyvern/dev
OICQ protocol version:0x371b
2019-06-22 15:42:15 +02:00
Luca Deri c4f513f6e7 Merge pull request #717 from weiyanhua100/dev
add null process for malloc.
2019-06-22 15:40:25 +02:00
Luca Deri 4ea8eaf103 Merge pull request #724 from themadprofessor/fix_dhcp_tests
Fix tests broken by new DHCP fingerprinting
2019-06-22 15:40:08 +02:00
Luca Deri 00ed2653fe Further WhatsApp improvements 2019-06-21 19:06:41 +02:00
Luca Deri 179b93d625 Better UDP flow handling 2019-06-21 12:14:57 +02:00
Luca Deri 3e43e77b1c Cleanup 2019-06-21 12:14:51 +02:00
Luca Deri b66606bc16 Improved OpenVPN support for UDP-based VPNs 2019-06-21 12:01:07 +02:00
Luca Deri b6ac750e21 Added WhatsApp Video 2019-06-20 23:16:36 +02:00
Luca Deri 936a95e10e WhatsApp improvements 2019-06-20 23:11:16 +02:00
Luca Deri 13e8354f37 JSON-C fix 2019-06-20 22:16:53 +02:00
Luca Deri 0fe7f16ecf Fixed invalid false positive 2019-06-20 21:11:21 +02:00
Stuart Reilly 784f4e4bba Add dhcp fingerprints to expected results 2019-06-19 13:20:55 +01:00
zhenxian.wang 1e616b227b new QQ version:0x3823 2019-06-19 15:58:30 +08:00
Luca 4830aa9a14 DHCP fingerprint is now in human readeable format 2019-06-18 17:44:05 +02:00
Luca 5f26149938 Added DHCP fingerpring handling 2019-06-18 17:36:39 +02:00
zhenxian.wang 1891f687b6 new QQ version 2019-06-18 18:04:46 +08:00
Luca Deri 41b75485eb SSL cipher improvements 2019-06-16 21:42:12 +02:00
Luca Deri 1a3f2b3f0d Reworked SSL cipher handling 2019-06-16 16:05:13 +02:00
Luca Deri 5b7a08bac3 Added SSL cipher print 2019-06-16 10:08:21 +02:00
weiyanhua e814cd0b9e add null process for malloc. 2019-06-13 15:05:22 +08:00
Luca Deri 709a87c8d4 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-06-05 23:15:42 +02:00
Luca Deri 4e8055742e Added fix for discarding core protocols as subprotocols 2019-06-05 23:15:06 +02:00
lucaderi 3979c28cac Added missing antimalware category 2019-06-03 19:50:18 +02:00
Luca Deri e6813846b8 Adds CUSTOM_CATEGORY_ANTIMALWARE category referenced by #710. 2019-06-03 08:18:24 +02:00
Luca Deri 165decdf6c Merge pull request #691 from pallas/fix-double-strdup
Fix double-strdup memory leak in ndpi_handle_rule
2019-06-03 08:10:01 +02:00
Luca Deri 4441bc7033 Merge pull request #711 from dfranusic/dev
Added PUBLISH/SUBSCRIBE/MESSAGE methods to SIP
2019-06-03 08:05:26 +02:00
Damir Franusic a015cb7f48 Merge remote-tracking branch 'upstream/dev' into dev 2019-05-31 15:21:04 +02:00
Damir Franusic 5c7e0a0319 SIP Message extension
* RFC 3248
2019-05-31 15:19:07 +02:00
Luca Deri c43bec826d Updated results with the new SSL dissection
Added RDP dissector
2019-05-30 11:15:50 +02:00
Luca a3b564eb92 Warning fix 2019-05-29 18:54:17 +02:00
Damir Franusic 1b9955e9ef Added PUBLISH/SUBSCRIBE methods to SIP 2019-05-29 17:12:32 +02:00
Luca Deri 5da363cbde Fixed MySQL dissector
SSL dissector now reports the protocol version
Fixed bug in ndpiReader that ivalidates in some cases the protocl dissection and used to slow down the dissection
2019-05-28 23:35:49 +02:00
Luca Deri 53119e71f4 Added ndpi_is_safe_ssl_cipher(0 call 2019-05-23 00:16:05 +02:00
Luca Deri fcc9c76249 Disabled debug 2019-05-22 21:19:22 +02:00
Luca Deri f29f9bbb38 Update SSL server ciphers 2019-05-22 12:37:27 +02:00
Luca Deri 04b543caa9 SSL certificate analysis improvement 2019-05-20 22:30:53 +02:00
Luca Deri 3f9383c343 Added missing definitions 2019-05-20 22:13:21 +02:00
Luca Deri 0d2f8f4498 Implements #705 2019-05-20 22:02:21 +02:00
Luca Deri 53e64df8e2 Removed debug code 2019-05-20 15:40:09 +02:00
Luca Deri f97be61e8b Fixed JA3 decoding
Made ndpi_process_extra_packet() optional
2019-05-20 15:32:50 +02:00
Luca Deri 9604386698 Merge pull request #704 from TelecomsSansFrontieres/dev
Adding 2 missing Whatsapp IPs
2019-05-18 11:18:07 +02:00
Luca Deri 7b2d9adbdf Merge pull request #707 from zyingp/xcode_fix
Fix the Xcode project
2019-05-18 11:17:47 +02:00
Luca d592902553 JA3 hardeding code 2019-05-15 17:55:01 +02:00
Yingpei Zeng 08ca25c9c8 Fix the Xcode project 2019-05-15 09:56:10 +08:00
Luca Deri e5a95cf92e Skept GREASE in SSL JA3 fingerprint 2019-05-13 23:22:19 +02:00
Clément BRUGUERA 1a2b8a98a6 Adding 2 missing Whatsapp IPs 2019-05-13 17:26:19 +02:00
Luca Deri 0d0d6893fc JA3 fixes 2019-05-12 15:25:14 +02:00
Luca Deri 3b185a00fa Disabled debug traces 2019-05-11 00:53:02 +02:00
Luca Deri a00ad9fc92 Initial JA3 support (https://github.com/salesforce/ja3) for SSL 2019-05-11 00:50:56 +02:00
Luca Deri b96e6a9e2d Merge pull request #696 from pallas/avoid-hyperscan-leak
Avoid leaking hyperscan struct
2019-05-08 12:13:23 +02:00
emanuele-f ea1ea99ce1 Add allowed site category 2019-05-06 12:01:53 +02:00
Luca Deri e99a0abfba Hardening fixes for packer parsing 2019-05-03 18:08:12 +02:00
cardigliano 361ab030bc ndpi_handle_rule accepted formats 2019-04-29 09:44:39 +02:00
Derrick Lyndon Pallas 33d17dbf1f Avoid leaking hyperscan struct 2019-04-22 17:02:11 +00:00
Alfredo Cardigliano 8a09ca2180 Fixed RPM pkgconfig path 2019-04-19 11:44:10 +02:00
Alfredo Cardigliano 1bbb992ca6 Fixed pkgconfig path (installing libndpi.pc in the default location) 2019-04-19 11:38:34 +02:00
emanuele-f 4dd51a165c Fix DNS rsp_addr missing in some tiny responses 2019-04-15 19:54:17 +02:00
Luca Deri 8365afee8d Merge pull request #688 from Dark-Vex/whatsapp
Added new Whatsapp addresses ntop/nDPI issue #683
2019-04-12 19:39:20 +02:00
emanuele-f 0961caec1b Add missing fclose in ndpiReader 2019-04-12 19:39:01 +02:00
Luca Deri 1bf9351a67 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-04-12 19:33:17 +02:00
Luca Deri 3cb0b3fc7b Fixes #692 2019-04-12 19:32:44 +02:00
Luca Deri ffb1b9c14f Merge pull request #690 from pallas/make-path-const
Make path const in ndpi_load_protocols_file
2019-04-12 19:25:27 +02:00
Luca Deri 82817c1cac Merge pull request #687 from Dark-Vex/viber
Added Viber IPs issue #684
2019-04-12 19:25:00 +02:00
Luca Deri 173a04007f Merge pull request #682 from chiehmin/dev
Parse HTTP PATCH method
2019-04-12 19:24:14 +02:00
Luca Deri 83d5627c4d Merge pull request #694 from pallas/avoid-csgo-short-read
csgo: fix invalid read on short packet
2019-04-12 19:20:14 +02:00
Luca Deri 6f8a01ca18 Merge pull request #695 from pallas/avoid-memcached-short-read
memcached: fix invalid read on short packet
2019-04-12 19:19:46 +02:00
emanuele-f 5b6526cc5a Add missing newlines 2019-04-12 17:25:38 +02:00
emanuele-f 5e24b32467 Fix category detection 2019-04-12 17:08:32 +02:00
Luca Deri 2f14b8abbf Added test file for FTP 2019-04-12 12:19:11 +02:00
Derrick Lyndon Pallas 74715b12b0 memcached: fix invalid read on short packet 2019-04-06 01:00:44 +00:00
Derrick Lyndon Pallas 7415af5086 csgo: fix invalid read on short packet 2019-04-06 00:31:41 +00:00
Derrick Lyndon Pallas 85776a532d Fix double-strdup memory leak in ndpi_handle_rule
proto is being strdup'd both in the call to ndpi_set_proto_defaults and
inside of that function as well, leaking the memory.
2019-04-05 19:00:39 +00:00
Derrick Lyndon Pallas bb2f35fe79 Make path const in ndpi_load_protocols_file 2019-04-05 18:48:42 +00:00
Luca 1290706fad Tests result fix
Merge branch 'dev' of https://github.com/ntop/nDPI into dev
2019-04-05 12:51:59 +02:00
Luca 4e7fa82aff Fixed false positive with FTP_DATA protocol
Added coherency in application protol guess by exploiting the host server name
Various protocol optimizations
2019-04-05 12:47:58 +02:00
emanuele-f 796472cf84 Fix category detection due to missing initialization 2019-04-05 10:11:17 +02:00
Luca Deri a455e0d8d4 Improved flow category guess 2019-04-02 23:01:03 +02:00
Dark-Vex 55c4108d8b [FIX] Added new whatsapp addresses ntop/nDPI issue #683 2019-04-02 16:47:49 +02:00
Daniele De Lorenzi 2ef19c7e49 [FIX] Added IPs reported in ntop/nDPI issue #684 2019-04-02 16:04:01 +02:00
Daniele De Lorenzi e27022b698 Merge pull request #11 from ntop/dev
Add all dev branch modifications
2019-04-02 15:49:54 +02:00
Luca Deri 153c77c2cd Improvements with category detection 2019-04-01 18:40:14 +02:00
Luca Deri 6693151052 Added custom category support to ndpi_get_partial_detection() 2019-04-01 16:12:44 +02:00
Luca Deri 1915a63cf2 Implemented ndpi_process_partial_detection() API call to handle partial matches due to the nDPI specified configuration 2019-04-01 12:30:51 +02:00
emanuele-f c496c7975f Add DNS response address 2019-03-29 19:58:10 +01:00
Luca Deri 8dbe7818eb Improvments for AmazonVideo 2019-03-27 07:00:11 +01:00
chiehminw 1df5b453cb Parse HTTP PATCH method
HTTP PATCH is introduced in RFC 5789 (https://tools.ietf.org/html/rfc5789)

Signed-off-by: chiehminw <chiehminw@synology.com>
2019-03-26 11:46:57 +08:00
Luca Deri 0b5e5a5f82 SSL cleanup 2019-03-25 22:08:39 +01:00
Luca Deri f47be6ef60 Added SSL organization support 2019-03-23 10:32:18 +01:00
Luca Deri f592b9fc31 Merge pull request #681 from chiehmin/certificate-parsing
Parse server certificate organization
2019-03-23 09:58:54 +01:00
Luca Deri 6b1f158322 Merge pull request #675 from chiehmin/dev
Respect CXX variable during configure
2019-03-23 09:54:17 +01:00
Luca Deri 99748cddcc Merge pull request #679 from chiehmin/link-use-relative-path
Use relative path for library link
2019-03-23 09:53:35 +01:00
Luca Deri e2cd89e354 Merge pull request #678 from madpilot78/FreeBSD_endian.h
The le32toh() function used in some places on BSD OSes requires sys/en…
2019-03-23 09:53:05 +01:00
Luca Deri e5f6f68cd6 Merge pull request #677 from chiehmin/add-http-method-prefix
Add NDPI prefix for HTTP_METHOD enum to avoid name collisions
2019-03-23 09:52:34 +01:00
Luca Deri e2ad6bf79f Merge pull request #673 from vel21ripn/dns_fix_name
More accurate hostname verification in a DNS query.
2019-03-23 09:51:53 +01:00
Luca Deri ef9f395cb6 Merge pull request #665 from vel21ripn/whatsapp
Fix whatsapp signature detecting.
2019-03-23 09:49:35 +01:00
Luca Deri e6f8a66197 Merge pull request #662 from bmwiedemann/date
Use ChangeLog date instead of build date
2019-03-23 09:49:06 +01:00
Luca 5656a41f69 Improved snapchat detection 2019-03-22 12:36:42 +01:00
chiehminw 96ed31874d we only need one dot to confirm it is a valid domain name
github.com is valid but can not be retreive by server certificate
parsing

Signed-off-by: chiehminw <chiehminw@synology.com>
2019-03-20 17:47:51 +08:00
chiehminw db91837647 retreive ssl certificate origanization
Signed-off-by: chiehminw <chiehminw@synology.com>
2019-03-20 17:47:48 +08:00
emanuele-f 6c9fbc2ddb Fix invalid categories labels array length and add check 2019-03-18 13:29:59 +01:00
Luca Deri 7237f7081f Improved FTP dissection 2019-03-14 22:07:21 +01:00
Luca Deri 54f90c7556 Added fix to avoid FTP false positives 2019-03-14 21:50:09 +01:00
chiehminw 43890b31fc Use relative path for library link
relative link path can preserve link relation after packaging and
doplying to new machine

ex:
$ make DESTDIR=/tmp/ndpi install
$ tar czf ndpi.tar.gz /tmp/ndpi
$ tar xf ndpi.tar.gz -C /lib

Orignal install script will caused the /lib/libndpi.so.2 links to
/tmp/ndpi/libndpi.so.2.9.0 which is not intended

Signed-off-by: chiehminw <chiehminw@synology.com>
2019-03-13 15:04:10 +08:00
Guido Falsi b1282da542 The le32toh() function used in some places on BSD OSes require sys/endian.h to be included. 2019-03-12 18:12:18 +01:00
Luca 22e431ca67 Sporify fix 2019-03-12 18:01:14 +01:00
chiehminw b1bd7d6f8c Add NDPI prefix for HTTP_METHOD enum to avoid name collisions
ref: https://stackoverflow.com/questions/35380279/avoid-name-collisions-with-enum-in-c-c99

Signed-off-by: chiehminw <chiehminw@synology.com>
2019-03-12 16:11:33 +08:00
Luca Deri 6c1d236a03 Category cleanup 2019-03-10 21:44:11 +01:00
Luca Deri 89d548f9d3 Removed categories based on file content 2019-03-10 21:18:19 +01:00
Luca fde4380422 Proto cleanup 2019-03-08 20:17:11 +01:00
Luca Deri aaa686b9fe Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-03-08 15:59:26 +00:00
Luca Deri 03b0aa7185 Added new categories
- NDPI_PROTOCOL_CATEGORY_MUSIC
- NDPI_PROTOCOL_CATEGORY_VIDEO
- NDPI_PROTOCOL_CATEGORY_SHOPPING
- NDPI_PROTOCOL_CATEGORY_PRODUCTIVITY
- NDPI_PROTOCOL_CATEGORY_FILE_SHARING

Protocol vs category rework
2019-03-08 15:58:21 +00:00
chiehminw 518dbbc96b Respect CXX variable during configure
commit 6ac83ce7eb use CXX to link ndpiReader however CXX is not
properly set in configure step

Fixes: 6ac83ce7eb ("Fixes broken hyperscan compilation")
Signed-off-by: chiehminw <chiehminw@synology.com>
2019-03-06 15:27:50 +08:00
Simone Mainardi 59f683858b Fixes possibly broken dissection of invalid DNS responses 2019-03-04 18:48:38 +01:00
Luca Deri c339211ac7 Re-Added TikTok/Musical.ly protocol 2019-03-03 21:07:54 +01:00
Simone Mainardi 93582a4204 Bump to version 2.9 2019-03-01 18:06:28 +01:00
Simone Mainardi 5bb53cf995 Changelog update 2019-03-01 18:02:24 +01:00
Luca 05ef62781f Fixes for wireshark 3 2019-03-01 15:19:05 +01:00
Luca d283538146 Fixed false positive mining detection 2019-03-01 12:56:33 +01:00
Vitaly Lavrov a99754d88b More accurate hostname verification in a DNS query. 2019-02-27 15:26:11 +03:00
Luca Deri de16b01bb2 Improved MDNS dissection 2019-02-24 11:03:06 +01:00
Luca Deri 47669abd44 Added extra check to avoid nDPI wrong matches 2019-02-21 23:29:06 +01:00
emanuele-f ab3833a708 Fix invalid TCP DNS dissection 2019-02-19 17:17:02 +01:00
Luca Deri dc8d582b18 Fixes #671 2019-02-14 10:31:42 +01:00
emanuele-f a035763afd Change ndpi_fill_ip_protocol_category to pass IPs explicitly 2019-02-12 17:28:30 +01:00
Luca Deri 0c4fada265 Fix for avoid wrong substring match to happen 2019-02-10 23:52:24 +01:00
Luca Deri ddf0066c11 Aligned DNS to HTTP dissection 2019-02-07 22:44:48 +01:00
Vitaly Lavrov 476fbbfe4e Fix whatsapp signature detecting. 2019-01-31 11:27:40 +03:00
cardigliano 2ec41c1732 Releasing old buffer if realloc fails 2019-01-31 09:12:04 +01:00
Alfredo Cardigliano 6a98aec2ca Support for protocol file files of arbitrary length 2019-01-30 08:44:32 +00:00
Bernhard M. Wiedemann e91123e17a Use ChangeLog date instead of build date
in order to make builds reproducible.
See https://reproducible-builds.org/ for why this is good.

This date call works with GNU date and BSD date.
Also use UTC/gmtime to be independent of timezone.
2019-01-24 14:27:10 +01:00
Luca Deri e2ffcd7229 Improved HTTP response code handling 2019-01-23 00:53:56 +01:00
Luca Deri efa96da3e0 Fixes #657 2019-01-20 23:47:24 +01:00
Luca Deri b9ae650146 Merge pull request #659 from cavedon/direction-endianness
Compute packet_direction consistently independently from endianness
2019-01-20 23:18:46 +01:00
Luca Deri 72edc508eb Merge pull request #661 from pcein/propagate-prefix
Propagate prefix settings to Makefiles
2019-01-20 23:17:46 +01:00
Pramode 81649d9979 Propagate prefix settings to Makefiles
A "make install" was failing because the --prefix flag
setting was not being propagated to the Makefiles.
2019-01-20 19:09:07 +05:30
Luca Deri 7efe1f06ee Removed debug code 2019-01-17 22:03:39 +01:00
Luca Deri 85155d5899 Removed debug code 2019-01-17 00:42:45 +01:00
Luca Deri d088a968a9 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2019-01-17 00:41:05 +01:00
Luca Deri efef99cbad Removed this party LRU and replaced with home grown 2019-01-17 00:40:28 +01:00
Luca Deri f3b0878a37 Warnign fixes
Updated test results
2019-01-17 00:39:45 +01:00
Luca Deri c856e0a563 Disabled ookla statistics: needs to be improved 2019-01-16 22:30:55 +01:00
Ludovico Cavedon 2c672a4ea2 Compute packet_direction consistently independently from endianness
Ensure packet_direction is computed in the same way on little endian and
big endian architectures.
This change will convert IP addresses and port from little endian to host
endian (instead of converting from network endian to host endian) so
that it does not change the behavior on little endian architecture where
ndpi is usually developed.
A better (but more invasive) change would be to use the ntoh*()
functions and then adjust all affected tests.
2019-01-16 01:15:43 -08:00
Ludovico Cavedon 0cb4bae275 QUIC: convert little endian offsets to host endianness
Offset in the QUIC protocol are little endian. Reading them as uint32_t
works on little endian architectures, but breaks on big endian ones.
This change applies the proper conversion and fixes running the
http_ipv6 and quic tests on big endian architectures.
2019-01-15 23:49:26 -08:00
Simone Mainardi 2ade0be365 Fixes ookla lru cache use with IPv6 2019-01-15 09:56:33 +01:00
Luca Deri f2f2dbe381 Fix for IPv6 HTTP traffic 2019-01-14 20:03:06 +01:00
Luca Deri 60eee5f9e9 Merge pull request #649 from Dark-Vex/dev
Bank website detected as TOR from SSL Dissector (bigram check)
2019-01-13 11:12:12 +01:00
Luca Deri 4a6b0d7de5 Merge pull request #654 from u-devel/patch-1
Update radius.c to RFC2865
2019-01-13 11:11:10 +01:00
Luca Deri f41ffe136a Merge pull request #655 from u-devel/dev
FIX H.323 broken detection
2019-01-13 11:10:36 +01:00
u-devel 1d155ab2eb FIX H.323 broken detection
TPKT header length field can have value more then 255, and in fact in all of the cases I've met it is more then 255.
Thus checking real H.323 packet like this:
(packet->payload[2] == 0x00)
stop detecting H.323 conversation at all.
2019-01-13 00:07:45 +06:00
u-devel 396f794bf4 Update radius.c to RFC2865
According to RFC2865 code can have value up to 13, also payload length ranges should be applied.
2019-01-12 23:49:24 +06:00
Luca Deri 97bdfe295d nDPi now finally honours dissection of HTTP responses 2019-01-07 00:28:29 +01:00
Luca Deri 5241c9f3cf Added Modbus over TCP dissector 2018-12-21 18:25:44 +01:00
Luca Deri 91da00f470 Updated results 2018-12-21 17:31:51 +01:00
Alfredo Cardigliano c7260b6436 Moved to 2.7 2018-12-21 10:50:23 +01:00
Daniele De Lorenzi c51405e99b Bank website detected as TOR from SSL Dissector (bigram check)
Commented "vp" for vpbank.com
2018-12-20 11:11:32 +01:00
Daniele De Lorenzi 2aea4da9ad Merge pull request #10 from ntop/dev
Repo sync
2018-12-20 11:04:03 +01:00
Alfredo Cardigliano d3be349fa0 Changelog update 2018-12-20 09:16:59 +01:00
Luca Deri 92ad057332 Added Ookla cache 2018-12-17 22:33:58 +01:00
emanuele-f d2b2aba6e8 Expose ndpi_fill_ip_protocol_category api to handle ICMP flows category 2018-12-12 11:31:07 +01:00
Luca Deri c1d31bb409 Fixed eDonkey false positives 2018-12-12 08:58:34 +01:00
Alfredo Cardigliano 181ec4992b Deb fix (shared lib links) 2018-12-10 16:54:01 +01:00
Alfredo Cardigliano 60337330bf Creating shared lib with major version 2018-12-10 16:33:09 +01:00
Alfredo Cardigliano af0fd3b96d RPM fix 2018-12-10 16:18:53 +01:00
Luca Deri e7cf989c7a Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2018-12-10 12:19:53 +01:00
Luca Deri 2a1313ffd5 Improvements in SpeedTest (Ookla) detection 2018-12-10 12:19:34 +01:00
Luca Deri 140ed9887b Merge pull request #639 from madpilot78/Create_libndpi.so.MAJOR_link
Also create the libndpi.so.2 link.
2018-12-10 02:30:05 -08:00
Alfredo Cardigliano b2c00b3b24 Added script to get the package version 2018-12-07 15:50:23 +01:00
Luca Deri 70d0566480 Build fix 2018-12-05 06:58:29 +01:00
Luca Deri 067c3dad44 Merge pull request #645 from vitalivanov/pull_example_dpdk_cleanups
build: cleanup of dpdk examples.
2018-12-04 15:43:18 -08:00
Luca be2934f9ad Added the concept of protocols that can have subprotocols. Example Spotify cannot have subprotocols, DNS can (DNS.Spotify)
Merged Skype call in/out in Skype Call
2018-11-30 18:41:16 +01:00
Luca 2993d9d37e Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2018-11-30 08:40:37 +01:00
Luca bae75c8f9b Fix for custom protocol load 2018-11-30 08:39:54 +01:00
lucaderi b6be0eebf6 Fix for shared library symbolic link 2018-11-30 06:53:41 +01:00
Luca Deri d63e000a45 Fixed package signing 2018-11-30 00:57:34 +01:00
Vitaliy Ivanov fd963538f3 build: cleanup of dpdk examples.
- clean install dependencies on example binary
- add more rules to clean command

Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com>
2018-11-29 13:40:37 +02:00
Luca c09284ba27 Improved heuristics
Used the host guessed protocol if upper protocol has not been detected
2018-11-29 08:44:41 +01:00
Luca 622519e90e Warning fix 2018-11-29 07:53:41 +01:00
Luca Deri f1e5ed4766 Merge pull request #642 from vitalivanov/pull_buildsys_improvements
Pull buildsys improvements
2018-11-28 23:20:34 +01:00
Vitaliy Ivanov a3abbc3ebb build: do not fail if example app is not present.
Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com>
2018-11-28 19:23:27 +02:00
Vitaliy Ivanov 8df6b5c1bf build: install example binaries to dst dir.
make install DESTDIR=/build/nDPI/

Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com>
2018-11-28 17:40:01 +02:00
Vitaliy Ivanov 9cbf25a76f build: cleanup .so.X.X.X if present.
Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com>
2018-11-28 16:56:22 +02:00
Vitaliy Ivanov 923ba84f65 gitignore: ignore dpdk artifacts.
Also similarly to shared libraries do not hide static lib.

Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com>
2018-11-28 16:43:17 +02:00
Vitaliy Ivanov ea7e7c753b build: correct example dpdk linking with pcap.
- take libpcap from configure information.
- clean up distclean.

Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com>
2018-11-28 16:40:34 +02:00
Luca 5429063546 Improved WhatsApp detection 2018-11-28 00:24:41 +01:00
Luca 54deb1302d Added Signal.org dissector 2018-11-28 00:09:44 +01:00
Luca Deri f15aed6c46 Renamed internal function 2018-11-21 23:25:27 +01:00
Luca Deri e9dd5985b3 Made configure more friendly
Added time measurement in ndpiReader (processing and setup time)
2018-11-21 17:22:15 +01:00
Luca Deri 669fdf6b5f Improved skype, teredo, netbios heuristics
Changed ndpi_detection_giveup() API: guess is now part of the call
2018-11-21 00:32:32 +01:00
Alfredo Cardigliano 21504c8492 rpm spec file 2018-11-19 19:01:53 +01:00
Luca Deri 12adb25665 Improved Skype detection
Update test results
Added initialization for some free protocols
2018-11-19 18:28:18 +01:00
Luca Deri 5259ba2bcb Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2018-11-19 18:28:10 +01:00
Alfredo Cardigliano babb1c864c removed -r not supported on all systems 2018-11-19 18:25:12 +01:00
Luca Deri 582da8d4bb Improved WeChat support
Removed Musical.ly protocol as the service has been merged with another one and thus it is no longer used
Improved guess for UDP protocols
2018-11-19 18:24:33 +01:00
Alfredo Cardigliano 1537669178 rpm package 2018-11-19 18:02:06 +01:00
Alfredo Cardigliano b0cfe0b9cc settins arch for -dev package as it contains static libs 2018-11-19 15:27:24 +01:00
Alfredo Cardigliano 30305cf4fa deb dependency fix 2018-11-19 15:13:38 +01:00
Alfredo Cardigliano 219951760f ubuntu ndpi/ndpi-dev package (testing) 2018-11-19 11:16:28 +01:00
emanuele-f 0ee451040d Warning fix 2018-11-18 12:50:53 +01:00
Alfredo Cardigliano 22c2568c27 commented out debug print 2018-11-16 19:21:57 +01:00
Luca Deri 6e6b825e92 Improved skype heuristic 2018-11-15 16:18:45 +01:00
Guido Falsi 8c63f25478 Also create the libndpi.so.2 link.
This is created bydefault by all build tools for libraries and required in certain cases. FreeBSD linker requires it when using SONAME.
2018-11-15 09:09:52 +01:00
Luca Deri 977751efdb Added ndpi_flowv4_flow_hash and ndpi_flowv6_flow_hash skeletons for
implementing https://github.com/corelight/community-id-spec
2018-11-14 15:24:15 +01:00
Luca Deri 771a6e9aab Merge pull request #636 from 2miksyn/dev
Improve SSL detection, prevent false positive SSL detection
2018-11-14 00:36:29 +01:00
Luca 2b4554ad44 Updated test results 2018-11-13 15:35:17 +01:00
emanuele-f 9e72884c7b Add OpenWrt Makefile 2018-11-13 09:58:31 +01:00
Luca Deri 71b2c19cf2 Added DPDK support to ndpiReader 2018-11-10 16:10:22 +01:00
kyrol 6d929bf4cc Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2018-11-09 14:53:33 +01:00
kyrol de9b2b202e FIX missing Application Data pkt for TLS 2018-11-09 14:52:32 +01:00
Luca Deri b31337db4a Improved Skype detection 2018-11-09 11:31:08 +01:00
Mikhail Smirnov 8f8ca40f63 Improve SSL detection, prevent false positive SSL detection 2018-11-09 00:46:25 +03:00
Luca Deri b98373f390 Improved Messenger/Hangout detection 2018-11-08 22:31:58 +01:00
Luca Deri 8e6640605a Removed skype false positives
better STUP detection
2018-11-08 22:13:05 +01:00
Luca Deri 9c5a4a6d1f Added UPnP test pcap
updated Nintendo test ourcome
2018-11-07 22:45:29 +01:00
Luca Deri e240bbd4a4 Implemented UPnP dissector 2018-11-07 22:43:33 +01:00
Luca Deri f7e7ef2f30 Improved SSL decoding 2018-11-03 10:05:13 +01:00
Luca 875b1e0dd5 Better clang check 2018-11-01 09:36:11 +01:00
Luca Deri 6a4324f451 Merge pull request #628 from madpilot78/Respect_DESTDIR_and_PREFIX
Make the examples Makefile respect DESTDIR and PREFIX env vars.
2018-11-01 09:20:38 +01:00
Luca Deri de37d08c24 Merge pull request #632 from madpilot78/Add_SONAME
Add SONAME property to shared library
2018-11-01 09:20:05 +01:00
Guido Falsi 00827903ef The OS variable was not being populated by default. 2018-10-29 23:15:28 +01:00
Guido Falsi 023830f5ab Try to fix build on Mac. 2018-10-29 21:24:08 +01:00
Guido Falsi c2169b3dca Add command line options to correctly populate the shared object SONAME property. 2018-10-29 17:06:58 +01:00
Luca 01444f79e1 Added NDPI_MIN_NUM_STUN_DETECTION constant 2018-10-29 14:47:53 +01:00
Luca ecdece83fe Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2018-10-29 14:44:53 +01:00
Michele Campus d7478c6ed3 Merge pull request #630 from pchaigno/fix-typo
Fix typo in README
2018-10-16 11:01:41 +02:00
Paul Chaignon b467005fac Fix typo in README 2018-10-16 10:14:09 +02:00
Guido Falsi b7b12d5b4e Make the examples Makefile respect DESTDIR and PREFIX env vars. 2018-10-12 17:22:13 +02:00
Luca 952215c0ac Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2018-10-11 17:47:10 +02:00
Luca 3e5db956a8 Added further ubuntu servers 2018-10-11 17:47:00 +02:00
kyrol a62f90073a fix domain name checking for https://github.com/ntop/nDPI/issues/617 2018-10-11 16:16:55 +02:00
emanuele-f 6789d41993 Fix category match not using client certificate 2018-10-11 12:40:11 +02:00
emanuele-f 4b4f4be3a5 Add -g flag 2018-10-11 12:27:13 +02:00
emanuele-f 40294f3aed Custom categories have now priority over protocol related categories 2018-10-10 15:56:46 +02:00
emanuele-f 67a1a07ac1 Fix custom categories match by IP not working 2018-10-10 13:33:02 +02:00
emanuele-f cdf197ccfe Add function to search host in custom categories 2018-10-10 13:29:49 +02:00
emanuele-f 6ff6c2e5af Fix category field not accounted in ndpi_get_proto_category
Fixes #627
2018-10-10 10:43:51 +02:00
Luca Deri cf0c8be63d Merge pull request #616 from wang-xuefei/dev
remake clean or remake install failed.
2018-10-09 17:45:23 +03:00
root 1d56c9aa1b fix remake 2018-10-02 23:54:01 +08:00
Luca Deri ab2f3cefc8 Merge pull request #612 from xcorail/dev
Add LGTM code quality badges
2018-10-02 10:07:04 +02:00
Luca Deri 3cbdbc1db5 Merge pull request #610 from eglooca/pr-nest-log-sink
New dissector: Nest Log Sink
2018-10-02 10:06:07 +02:00
Luca Deri 90ca91caa8 Renamed files to avoid name clashes during compilation with other libs 2018-09-29 08:38:00 +02:00
Luca Deri 1f24adc9cd Renamed source files to avoid clashes 2018-09-29 08:28:26 +02:00
Xavier RENE-CORAIL a1f7ab776e Add LGTM.com code quality badges 2018-09-27 13:00:00 +02:00
Luca Deri f639c237a1 Added check for not going beyond the string lenght 2018-09-24 09:57:51 +02:00
Darryl Sokoloski 46ebf541e1 Removed unused varaibles.
Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2018-09-19 22:05:39 -04:00
Darryl Sokoloski 53910f0737 Added missing declarations for nest_log_sink dissector.
Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2018-09-19 22:05:03 -04:00
Darryl Sokoloski 31f62c5cf5 New dissector: Nest Log Sink
This is a dissector for the Nest Log Sink protocol which is used by the Nest Protect (https://nest.com/ca/smoke-co-alarm/overview/).

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2018-09-19 21:25:16 -04:00
Luca 7117f0532c Mapped the mining category in ndpi_category_get_name() 2018-09-18 18:32:08 +02:00
Luca c6b427c252 Implementation of Bitcoin, Ethereum, ZCash, Monero dissectors all identified as mining
Implements #503
2018-09-18 18:13:04 +02:00
Luca Deri b0f9f76b79 Addes support for SMBv1 traffic that has been split from SMBv23 2018-09-17 23:15:44 +02:00
Luca 8689d3293a Added ndpi_protocol2id() API call 2018-09-17 17:57:47 +02:00
Luca Deri 51fdd12d23 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2018-09-13 00:11:09 +02:00
Luca Deri 24110acafc Merge pull request #606 from eglooca/pr-fix-parse-packet-line-info
Prevent missing text lines and invalid reads past end-of-buffer.
2018-09-13 01:11:01 +03:00
Michele Campus f6f242b899 Merge pull request #604 from DanieleDeSensi/dev
Bug fix and performance improvement for spotify dissector.
2018-09-13 00:09:56 +02:00
Luca Deri e8c738282d Improved Facebook Messenger support 2018-09-13 00:04:19 +02:00
Darryl Sokoloski c0ed50d5b0 Prevent invalid reads past end-of-buffer.
The recent revert commit applied to ndpi_parse_packet_line_info resurrects an old bug where the last lines in packets that end with a CR+NL will not be parsed.  This revert commit is an attempt to prevent invalid reads past the end of the packet buffer.  This PR moves the end-of-bounds test to before the 16-bit read and returns if true.  This fixes the case where a text line ends aligned to the buffer-end boundary, and it fixes the invalid read issue.

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2018-09-11 17:16:12 -04:00
Daniele De Sensi f48c7ab8e6 Fixed bug in spotify dissector (subnet /24 was matched as /22).
Slight performance improvements in spotify dissector (avoid doing ntohl and masking in each and all if conditions).
2018-09-11 12:41:05 +02:00
诟屍 adcb82d2b5 parse protocol's string_to_match into hex which could be match by hyperscan (#589)
* if one protocol do not have it's regex (pattern_to_match)
then parse it's string (string_to_match) into hex
so we can match every protocol with hyperscan

* fix string2hex()

* fix init_hyperscan():
check return value of string2hex()
2018-09-11 10:46:13 +02:00
Simone Mainardi 6ac83ce7eb Fixes broken hyperscan compilation 2018-09-11 10:43:04 +02:00
Luca b9ae0f8888 Fixes #578 2018-09-11 10:07:39 +03:00
Luca Deri 7b4d335944 Merge pull request #603 from pavlosantoniou/dev
Avoid possible NULL pointer dereference in ndpi_detection_process_packet
2018-09-11 10:04:15 +03:00
Luca 9bf05c3534 Indentation fix 2018-09-11 09:33:09 +03:00
Luca 1075283d8e Fixes #601 2018-09-11 09:26:01 +03:00
Pavlos Antoniou 77ea1c25bd Avoid possible NULL pointer dereference in ndpi_detection_process_packet 2018-09-09 08:20:54 +00:00
Luca Deri 6373c355cf Added fix for out of boundary memory access
Fixes warning
2018-09-05 23:23:54 +02:00
Campus 225826ce83 FIX: dropbox dissector. UPD: updated pcap file with new dropbox pkts 2018-09-05 16:10:07 +02:00
Luca Deri 0c3989b40f Merge pull request #596 from madpilot78/Add_back_pkgconfig_file
Add back pkgconfig file. It is required by ntopng to find the install…
2018-08-30 11:29:12 +02:00
Luca Deri 4a2525241f Merge pull request #598 from madpilot78/include_hyperscan_fix
Move the configure include file inclusion and code depending on it in…
2018-08-30 11:28:41 +02:00
Luca ed5b04791e Added missing -lpthread 2018-08-30 11:26:49 +02:00
Luca 71b28e0774 Cleanup 2018-08-30 11:19:09 +02:00
Luca 33d6f2d52f Added missing file 2018-08-30 11:17:19 +02:00
Luca e4f01976a6 Added missing categorization when giveup/guess is called
Added optimization for TCP flows that do not start with a SYN packet: early giveup is performed
Code cleanup
2018-08-30 11:10:30 +02:00
Luca 0a33e439bd Better clang handling 2018-08-30 10:05:53 +02:00
Luca f948f3525f Cleaned up makefile
Used clang instead of gcc on MacOS
2018-08-30 09:59:39 +02:00
Guido Falsi 396e72f4fe Move the configure include file inclusion and code depending on it in code protected by the NDPI_LIB_COMPILATION define, this should avoid it polluting the environment when including this file from ntopng. 2018-08-27 17:45:38 +02:00
Guido Falsi d31a29bf6a Adapt pkgconfig file include path to new Makefile install location. 2018-08-26 00:35:01 +02:00
Guido Falsi 405e2f8dd0 Add back pkgconfig file. It is required by ntopng to find the installed shared library. 2018-08-25 23:42:20 +02:00
Luca a98c0a81be Added target for shared liubrary creation
Added target for install
Fixes #593
2018-08-25 10:50:16 +02:00
Campus 2f83375eec Merge branch 'eglooca-pr-remove-autoconf-include' into dev 2018-08-23 12:09:51 +02:00
Campus 2b9b996148 Merge branch 'pr-remove-autoconf-include' of https://github.com/eglooca/ndpi-pr into eglooca-pr-remove-autoconf-include 2018-08-23 12:01:14 +02:00
Michele Campus 48f3104774 Merge pull request #591 from alm-A/patch-2
README add protocol : removed Makefile.am step
2018-08-23 12:00:02 +02:00
Michele Campus b8571475c9 Merge pull request #592 from alm-A/patch-3
ndpi_typedefs.h : missing include
2018-08-23 11:59:31 +02:00
Campus ff83bfbad0 ADD: add prototype of function ht_free in hash.h - DEL: delete include file .c in ndpi_main.c 2018-08-23 11:22:21 +02:00
alm-A dd307af215 ndpi_typedefs.h : missing include
include to ndpi_protocol_ids.h needed for:
- NDPI_PROTOCOL_SIZE
- NDPI_MAX_SUPPORTED_PROTOCOLS
- NDPI_MAX_NUM_CUSTOM_PROTOCOLS
2018-08-22 16:37:59 +02:00
alm-A 58a784e342 README add protocol : removed Makefile.am step 2018-08-22 16:18:44 +02:00
Luca Deri 89b3b085af Moved to 2.5.0 2018-08-22 08:45:13 +02:00
Simone Mainardi e6c3c0139a Updates changelog release date 2018-08-21 18:13:54 +02:00
Simone Mainardi aad896c2a8 Updates ChangeLog 2018-08-21 18:12:36 +02:00
Luca Deri 1a6a9593fb Changed type to ease compilation accross platforms 2018-08-20 14:49:59 +02:00
Luca Deri 6cb40ac75e Compilation fix 2018-08-20 12:39:55 +02:00
Luca Deri 50591238e5 Merge pull request #586 from eglooca/pr-sys-types-include-for-actypes-h
Include <sys/types.h> for u_int16_t.
2018-08-17 14:18:27 +02:00
Simone Mainardi 291acb9459 Hyperscan regex for domains such as .com.cn or .co.uk 2018-08-17 13:13:26 +02:00
Simone Mainardi 47c2ad1b5f Webex category fix 2018-08-17 12:59:19 +02:00
Simone Mainardi 6491568e15 Fixes hyperscan WeChat detection 2018-08-17 12:48:53 +02:00
Simone Mainardi 28ac30d827 Fixes hyperscan skype detection 2018-08-17 12:41:35 +02:00
Simone Mainardi db74459ca2 Fixes weibo detection 2018-08-17 12:29:08 +02:00
Simone Mainardi d2bc3ea209 Initial hyperscan support for all protocols 2018-08-17 12:00:28 +02:00
Darryl Sokoloski cd9d456994 Include sys/types.h for u_int16_t.
On various embedded environments (OpenWrt, EdgeOS), <sys/types.h> must be included for u_intX_t types.  The Hyperscan changes made recently introducted a u_int16_t member in actypes.h which is undeclared in certainly environments without this include.

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2018-08-16 22:31:23 -04:00
Darryl Sokoloski 06befec642 Remove autoconf include from typedefs headers.
Including this file in any "public" API header breaks all projects that also use autotools because macros such as PACKAGE_VERSION will be redefined.

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2018-08-16 22:24:15 -04:00
Simone Mainardi 579f1b05db Adds hyperscan support for Apple and Dropbox 2018-08-16 19:42:04 +02:00
Simone Mainardi 7e41309fa9 Hyperscan compilation fix 2018-08-16 15:43:35 +02:00
Luca Deri 3091dc3c48 Fix for #400 2018-08-16 15:23:30 +02:00
Luca Deri 01bf295a19 Completed custom category implementation 2018-08-16 14:57:44 +02:00
Luca Deri f88648fbc8 Tests update 2018-08-16 12:05:07 +02:00
Luca Deri 18b9589e50 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2018-08-16 12:03:23 +02:00
Luca Deri 9847c2ccbb Generic protocol cleanup (work in progress) 2018-08-16 12:03:09 +02:00
Simone Mainardi 335b336968 Hyperscan compilation fixes 2018-08-16 11:47:10 +02:00
Luca Deri 70eb619b86 Merge pull request #584 from eglooca/pr-memcached
Added new dissector for Memcached.
2018-08-16 09:36:35 +02:00
Darryl Sokoloski b3b8feabf1 Added new dissector for Memcached.
Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2018-08-15 16:47:21 -04:00
Luca Deri d9cf798753 Updated YouTube support 2018-08-06 10:34:45 +02:00
Luca Deri cbb2eb8c8f Merge pull request #576 from eglooca/pr-ssdp-min-length
Adjust the SSDP dissector's minimum packet length
2018-07-25 08:18:17 +02:00
Luca Deri 6474125c93 Merge pull request #577 from eglooca/pr-parse-packet-line-info-length
Fix end-of-line bounds handling.
2018-07-25 08:17:32 +02:00
Luca Deri 5100cc0eae Added checks for missing protocol initialization
Fixed missing LOG_ERR initialization
Fixed logging warnings
2018-07-23 14:58:16 +02:00
Luca Deri 49ed41142c Fixed SEGV with undepcified protos 2018-07-22 21:06:10 +02:00
Luca Deri 659649d2e9 Fixed warning 2018-07-21 18:23:48 +02:00
Luca 5246656ae0 Fixed required library prefix 2018-07-21 16:27:05 +02:00
Luca f828dfc202 Added comment about typedefs 2018-07-21 16:17:12 +02:00
Luca 591158bb92 Defined type ndpi_protocol_id_t for protocol identifiers 2018-07-21 16:09:32 +02:00
Luca 667e77c609 Major code cleanup
Converted some not popular protocols to NDPI_PROTOCOL_GENERIC with category detection
2018-07-21 16:02:54 +02:00
Luca a499f369a5 Updated results based on the new output format 2018-07-21 15:20:11 +02:00
Darryl Sokoloski fbf6bd2be8 Fix end-of-line bounds handling.
The existing implementation misses ending lines and as a result, fails
to match certain protocols (SMTP for example, which needs to see at
least 3 commain/response matches).  It appears from the commit history
that an attempt was made (end-1) to prevent reads past payload length.

This can be ensured by simply not reading any payload that is < 3 bytes.

The updated logic for this loop is:
- Payload length is >= 3 bytes, or return.
- Loop over payload, compare for EOL (CR + NL) sequence.
- If found, process string.
- If index 'a' plus two is less than payload length, increment 'a' by
  one and continue.
- Loop return always increments index 'a' by one.
2018-07-20 13:51:17 -04:00
Darryl Sokoloski 212598b45e Added example SSDP M-SEARCH capture file. 2018-07-20 13:28:38 -04:00
Darryl Sokoloski 8b0e735f67 Decreased minimum packet payload length down to 19 from 100 for SSDP.
The minimum packet payload length is: "NOTIFY * HTTP/1.1" + 0x0d + 0x0a (19 bytes).
2018-07-20 13:26:44 -04:00
Simone Mainardi d9c963061b Fixes invalid NDPI_MAX_SUPPORTED_PROTOCOLS 2018-07-20 18:52:00 +02:00
Luca Deri 30b72eae64 Changed RTP from VoIP to Media category 2018-07-18 21:29:26 +02:00
Luca Deri 475f54e9d1 Fixed name clash 2018-07-18 17:33:28 +02:00
Luca Deri 34dfe7d375 Improved tor detection to limit false positives 2018-07-17 18:21:06 +02:00
Luca Deri 06789406ad Pattern match improvements 2018-07-17 17:49:59 +02:00
Luca Deri 1ebc510bcb Fied substring matching 2018-07-17 15:48:24 +02:00
Luca Deri 90ffa1fc2c Updated twitter invalid address range
Using content substring match in category search
2018-07-16 09:21:34 +02:00
Michele Campus f942cef2f1 Merge pull request #573 from protohuf/dev
Added showmax.com detection
2018-07-02 23:04:31 +02:00
Marcus Hufvudsson 7fce8f02d2 Added showmax.com detection 2018-07-02 19:54:37 +02:00
Luca 3986a951d8 Suppressed invalid messages with -q
-j now accepts - as stdout
2018-06-21 16:13:18 +02:00
Luca Deri 22356fa69b Added site unavailable category 2018-06-11 17:01:20 +02:00
Luca Deri e00b06926e Fix for handling IP address based custom categories 2018-06-11 16:22:02 +02:00
Luca Deri dc9409ae12 Added missing include 2018-06-05 23:02:24 +02:00
Daniele De Lorenzi 3b1047b0c8 Merge pull request #9 from ntop/dev
updates from dev
2018-05-31 16:29:04 +02:00
Simone Mainardi 2b4321b5c3 Fixes invalid NDPI_MAX_SUPPORTED_PROTOCOLS
Fixes https://github.com/ntop/nDPI/commit/c77ec63e
2018-05-31 16:19:48 +02:00
Luca Deri 6b16ad709e Fix for #560 2018-05-29 22:50:18 +02:00
Campus 3407db11cd added new viber pcap and updated results 2018-05-29 20:37:02 +02:00
Luca Deri 326a2fb7ed Fix for https://github.com/ntop/nDPI/issues/572 2018-05-28 18:18:19 +02:00
Luca Deri eef175e579 Merge pull request #571 from Dark-Vex/rapidvideo
Added Rapidvideo
2018-05-28 17:27:53 +02:00
Daniele De Lorenzi 7989b8d969 Merge pull request #8 from Dark-Vex/rapidvideo
Added RapidVideo
2018-05-25 22:03:14 +02:00
Daniele De Lorenzi 9af00fdb4c Merge branch 'dev' into rapidvideo 2018-05-25 21:57:38 +02:00
Daniele De Lorenzi 6c2cfb34f7 Merge pull request #7 from Dark-Vex/vidto
Added VidTO streaming service
2018-05-25 21:51:15 +02:00
Daniele 5171309c43 Added RapidVideo 2018-05-25 21:41:14 +02:00
Daniele c77ec63ed8 Added VidTO streaming service 2018-05-25 20:34:37 +02:00
Daniele De Lorenzi 30afa082e4 Merge pull request #6 from ntop/dev
Merge branch 'Dark-Vex-viber' into dev
2018-05-25 18:54:26 +02:00
Daniele De Lorenzi ea64f8b0da Merge pull request #5 from Dark-Vex/viber
Viber
2018-05-25 18:52:06 +02:00
Daniele De Lorenzi e2310751f1 Updated Viber ports
New ports taken from
https://support.viber.com/customer/portal/articles/1506350-opening-ports-for-viber-desktop
2018-05-24 12:14:09 +02:00
Daniele De Lorenzi 47b5e45f13 Updated Viber subnet and domain
Added the new subnet, added viber.it and removed the old subnet 54.169.X since are no more part of viber network

daniele@bart:~$ whois 54.169.63.160
[Querying whois.arin.net]
[whois.arin.net]

#
# ARIN WHOIS data and services are subject to the Terms of Use
# available at: https://www.arin.net/whois_tou.html
#
# If you see inaccuracies in the results, please report at
# https://www.arin.net/resources/whois_reporting/index.html
#


NetRange:       54.160.0.0 - 54.175.255.255
CIDR:           54.160.0.0/12
NetName:        AMAZON-2011L
NetHandle:      NET-54-160-0-0-1
Parent:         NET54 (NET-54-0-0-0-0)
NetType:        Direct Allocation
OriginAS:       
Organization:   Amazon Technologies Inc. (AT-88-Z)
RegDate:        2014-06-20
Updated:        2014-06-20
Ref:            https://whois.arin.net/rest/net/NET-54-160-0-0-1
2018-05-24 12:14:09 +02:00
Campus 31275398aa Merge branch 'Dark-Vex-viber' into dev 2018-05-24 11:40:51 +02:00
Daniele De Lorenzi c1166c37d1 Updated Viber ports
New ports taken from
https://support.viber.com/customer/portal/articles/1506350-opening-ports-for-viber-desktop
2018-05-24 11:16:32 +02:00
Daniele De Lorenzi 5420b6359b Updated Viber subnet and domain
Added the new subnet, added viber.it and removed the old subnet 54.169.X since are no more part of viber network

daniele@bart:~$ whois 54.169.63.160
[Querying whois.arin.net]
[whois.arin.net]

#
# ARIN WHOIS data and services are subject to the Terms of Use
# available at: https://www.arin.net/whois_tou.html
#
# If you see inaccuracies in the results, please report at
# https://www.arin.net/resources/whois_reporting/index.html
#


NetRange:       54.160.0.0 - 54.175.255.255
CIDR:           54.160.0.0/12
NetName:        AMAZON-2011L
NetHandle:      NET-54-160-0-0-1
Parent:         NET54 (NET-54-0-0-0-0)
NetType:        Direct Allocation
OriginAS:       
Organization:   Amazon Technologies Inc. (AT-88-Z)
RegDate:        2014-06-20
Updated:        2014-06-20
Ref:            https://whois.arin.net/rest/net/NET-54-160-0-0-1
2018-05-24 11:12:15 +02:00
Daniele De Lorenzi 5e47871256 Merge pull request #1 from ntop/dev
fetch new commit
2018-05-24 10:57:28 +02:00
Luca Deri 1585503809 Added Musical.ly support 2018-05-23 22:53:03 +02:00
Campus 60636a9a88 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2018-05-23 18:32:04 +02:00
Campus fcf4b25bc3 update google hangout detection from string 2018-05-23 18:31:27 +02:00
Luca 0efc4a706d Added custom categories
CUSTOM_CATEGORY_MINING = 99
 CUSTOM_CATEGORY_MALWARE = 100
 CUSTOM_CATEGORY_ADVERTISEMENT = 101
 CUSTOM_CATEGORY_BANNED_SITE = 102

Updated test results
2018-05-23 17:49:07 +02:00
Luca Deri fb7416cb9d Merge pull request #565 from Dark-Vex/dev
SMTPS on port 587 is not detected and Updated AmazonVideo
2018-05-23 16:53:26 +02:00
Daniele De Lorenzi 32c62faa78 Updated AmazonVideo
Updated CDN list of PrimeVideo
2018-05-22 11:28:17 +02:00
Daniele De Lorenzi 363deeb23d Added support for SMTPS on port 587
Added support for SMTPS on port 587
2018-05-21 17:26:00 +02:00
Daniele De Lorenzi 131f68117b Update ssl.c
Added support for SMTPS on port 587
2018-05-21 17:23:32 +02:00
Michele Campus 05cef1d69a Merge pull request #564 from Dark-Vex/dev
Cloudflare detection
2018-05-21 16:31:11 +02:00
Daniele De Lorenzi 71ef54f561 Update ndpi_content_match.c.inc
Removed 64.68.192.0 since is no more part of Cloudflare according to the whois data

NetRange:       64.68.192.0 - 64.68.207.255
CIDR:           64.68.192.0/20
NetName:        EDNS
NetHandle:      NET-64-68-192-0-1
Parent:         NET64 (NET-64-0-0-0-0)
NetType:        Direct Assignment
OriginAS:       AS16686
Organization:   easyDNS Technologies, Inc. (EASYD-2)
RegDate:        2008-06-12
Updated:        2012-03-02
Comment:        ARINA51-ARIN
Ref:            https://whois.arin.net/rest/net/NET-64-68-192-0-1
2018-05-21 08:14:01 +02:00
Daniele De Lorenzi f9c217e42a Update ndpi_content_match.c.inc
Updated Cloudflare subnet
2018-05-19 20:21:26 +02:00
Luca Deri 36c1b72118 Updated test resuls 2018-05-18 23:22:14 +02:00
Luca Deri 6dd42d19f7 Implemented Facebook Messenger (mobile) support
Code cleanup
2018-05-18 23:10:00 +02:00
Luca Deri 5d3ac78250 Fixed memory leak
Optimized memory usage
2018-05-18 10:15:05 +02:00
Luca Deri f56b083bc3 Implemented FacebookZero protocol support
https://code.facebook.com/posts/608854979307125/building-zero-protocol-for-fast-secure-mobile-connections/

Discontinued filetopia protocol
2018-05-18 00:55:21 +02:00
Campus 79be44ddea fix spotify detection 2018-05-17 22:54:03 +02:00
Luca Deri dcdd7562f4 Implemented hash-based categories 2018-05-16 00:09:12 +02:00
Luca Deri 66b759ca69 Hidden ndpi_detection_module_struct definition 2018-05-14 22:58:14 +02:00
Luca Deri 529492e81f Updated test results 2018-05-14 21:42:19 +02:00
Luca Deri 1076455c01 Added ndpi_set_detection_preferences() APi call 2018-05-14 21:40:27 +02:00
Luca Deri 2d486aeed8 Skype CallIn/CallOut are not set as
Skype.SkypeCallOut
Skype.SkypeCallIn
2018-05-14 19:23:15 +02:00
Luca Deri 00c35e48a2 Added
- ndpi_load_hostname_category()
- ndpi_enable_loaded_categories()

in the list of exported API calls
2018-05-11 16:04:47 +02:00
Luca Deri 8380b505f4 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2018-05-11 13:49:11 +02:00
Luca Deri 564d622cc9 Added ndpi_fill_protocol_category call
Introduced intitial Skype in/out support
2018-05-11 13:48:14 +02:00
Luca Deri a067350098 Hyperscan fixes 2018-05-11 09:06:07 +02:00
Luca Deri 50216991bd Improved hyperscan support and category definition 2018-05-10 23:46:35 +02:00
Alfredo Cardigliano 3707553906 host_server_name size fix 2018-05-05 10:01:02 +00:00
Michele Campus b8fec1e3ce Merge pull request #557 from dsokoloski/pr-gitignore-updates
Clean-up/sort: .gitignore
2018-05-04 18:50:44 +02:00
Michele Campus 979330392e Merge pull request #558 from gozzy/dev
SSH: increase client's maximum payload length from 100 to 500
2018-05-04 18:49:56 +02:00
Alexander Gozman 11898f0268 SSH: increase client's maximum payload length from 100 to 500
According to RFC4253, a client may send additional data right after
its identification string (before receiving the server's identification
string). For instance, PuTTY sends supported ciphers. This exceeds
100 bytes and nDPI fails to detect such SSH sessions.
2018-05-04 07:17:17 +00:00
Luca Deri 78176d2b7b Added timeseries dump (disabled by default) 2018-05-02 00:11:55 +02:00
Luca Deri 64633939e0 Added NetFlix block for Rogers 2018-04-28 10:07:54 +02:00
emanuele-f f717be19cb Fix custom categories matching 2018-04-27 15:02:51 +02:00
Darryl Sokoloski 9c6c15cd6b Clean-up/sort: .gitignore 2018-04-26 16:33:58 -04:00
Luca 3adccc98f7 Implemented custom category loading.
Modified ndpiReader with -c <file> (use mining_hosts.txt as example) for loading name-based categories
2018-04-26 21:10:59 +02:00
Michele Campus 79f3199339 Merge pull request #553 from dsokoloski/pr-debug-call-back
Added missing file, func, and line parameters to call-back typedef.
2018-04-26 11:45:36 +02:00
Campus 23042b3c9a Merge branch 'dsokoloski-pr-autoconf-config-include-fix' into dev 2018-04-26 11:43:59 +02:00
Campus e819899221 Merge branch 'pr-autoconf-config-include-fix' of https://github.com/dsokoloski/nDPI into dsokoloski-pr-autoconf-config-include-fix 2018-04-26 11:43:00 +02:00
Luca Deri b92541a08e Added flow and timing support 2018-04-26 00:04:26 +02:00
Darryl Sokoloski c54a8f8cfa Moved ndpi_config.h from ndpi_main.h to ndpi_main.c, and example source files. 2018-04-24 10:21:47 -04:00
Darryl Sokoloski 3b5f32f261 Added missing file, func, and line parameters to debug call-back typedef. 2018-04-24 10:20:06 -04:00
Luca Deri 7798b964fe Replaces outdates socrates protocol with ntop 2018-04-24 15:48:46 +02:00
Campus 44e2e0beea Merge branch 'leonn-AJP_protocol' into dev 2018-04-23 15:50:06 +02:00
Campus 32a9b8fd14 Merge branch 'AJP_protocol' of https://github.com/leonn/nDPI into leonn-AJP_protocol 2018-04-23 15:48:46 +02:00
Leonn 5a7f58d23f 💡 Add Apache JServ Protocol Dissector 2018-04-22 01:54:28 -03:00
Luca 6aaca82d0e Added fbsbx.com to the list of FaceBook sites 2018-04-18 18:25:11 +02:00
Luca Deri 07a81f5e15 Updated results 2018-04-16 14:21:59 +02:00
Luca Deri c9a2eb54c1 Merge pull request #539 from felix-jia/fix-mem-leak
Fixed memory leak when flow is zeroed out.
2018-04-16 14:19:00 +02:00
Luca Deri 8a59581668 Merge pull request #547 from zyingp/free-after-detect
Fix several bugs found by Address Sanitizer (ASan)
2018-04-16 14:16:09 +02:00
Yingpei Zeng 5950ad7ef8 Add length check before several memcmps in msn.c [ASan detected]. 2018-04-16 18:47:49 +08:00
Yingpei Zeng 4b720c4f3c Add length check before do memcmp to "x-steam-sid" in http.c 2018-04-16 16:07:01 +08:00
Yingpei Zeng 480791a057 Add length check before checking a Tor certificate name 2018-04-16 16:01:31 +08:00
Yingpei Zeng 076bd3d1f4 Add a new file to Xcode project 2018-04-16 10:48:30 +08:00
NETAGO d8e4111a9d Improved netflix support 2018-04-13 16:36:33 -04:00
Michele Campus 9c1029ff6c Merge pull request #543 from vel21ripn/fix_packet_time
Repaired wrong timestamp calculation.
2018-04-09 16:44:38 +02:00
Michele Campus 56f14ebee4 Merge pull request #545 from vpiserchia/dev
added ndpi_process_extra_packet to exported symbols
2018-04-09 16:40:05 +02:00
Vito Piserchia aa5d06bb1a added ndpi_process_extra_packet to exported symbols 2018-04-09 00:37:34 +02:00
Vitaly Lavrov 025ca46981 Repaired wrong timestamp calculation.
Fixed incorrect type conversion.
  Hardcoded constant replaced by structure field.
2018-04-06 22:56:43 +03:00
Simone Mainardi 521afe7e1f Adds version 2.2.2 ChangeLog 2018-04-06 17:12:47 +02:00
Luca Deri c7e53724dc Added protocol placeholders for undefined protocols 2018-04-06 11:34:26 +02:00
Luca Deri 85f25fad6d Removed NDPI_PROTOCOL_MOVE protocol no longer used
Merged NDPI_PROTOCOL_CITRIX_ONLINE with NDPI_PROTOCOL_CITRIX
2018-04-05 23:08:10 +02:00
Luca Deri b63279af8d Updated WhatsApp address range 2018-04-03 18:13:42 +02:00
Felix Jia 5281fcf3cd Fixed memory leak when flow is zeroed out. 2018-03-26 10:20:03 +13:00
Luca Deri 462da90a21 Added API calls
ndpi_network_ptree_match
ndpi_check_flow_func

ndpi_config.h is now installed with "make install"
2018-03-23 19:05:32 +01:00
Luca a2acd09263 Added ndpi_get_api_version() API call to be used in applications that
are dynamically linking with nDPI, to make sure the APIs are in sync
2018-03-21 18:33:08 +01:00
Michele Campus 957b01aa71 Merge pull request #531 from zyingp/dev-autogen
Add chmod +x to the generated configure file in autogen.sh
2018-03-13 19:44:43 +01:00
Yingpei Zeng ac31c1ba98 Add chmod +x to the generated configure file
Fix an error when running autogen.sh on Ubuntu (16.04 LTS):
./autogen.sh: 49: ./autogen.sh: ./configure: Permission denied
2018-03-06 15:14:38 +08:00
Michele Campus 3aaae46c66 Merge pull request #529 from sparlane/office365_sni
Improve SNI matching when using office365
2018-03-05 15:01:40 +01:00
Scott Parlane 53882ef736 Improve SNI matching when using office365
Adds additional SNIs that are used when accessing
office365 applications, these didn't have any existing
match.
2018-03-05 13:16:30 +13:00
Luca Deri 21da70fd75 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2018-03-01 20:54:17 +01:00
Luca Deri d4a20ba6a8 Increased number of protocols to 512
Implemented WhatsApp file exchage support (added test pcap)
2018-03-01 20:52:06 +01:00
Campus 4204e2f614 Merge branch 'dillinger79-dev' into dev 2018-03-01 16:28:32 +01:00
dillinger79 38199a589f Update ndpi_main.c 2018-03-01 15:45:26 +02:00
dillinger79 b19cd086b4 Merge pull request #3 from ntop/dev
update to latest
2018-03-01 14:13:50 +02:00
Luca Deri e935ee77bf Updated results after the latest RTP fixes 2018-02-21 08:37:03 +01:00
Luca Deri 6a9fc935eb Added fix for invalid RTP/Skype detection
Added change in config.h generated by nDPI to avoid name clashes (the GNU autotools are really outdated)
2018-02-21 08:18:37 +01:00
Luca Deri 06bb9a9c57 Remove too strict check for RTP 2018-02-21 07:43:59 +01:00
Luca Deri 6e7a1022b2 Added PACKAGE_XXX rename to avoid name clashes when nDPI is used inside another project that use the autotools 2018-02-19 18:50:23 +01:00
Campus 76d058360e Merge branch 'bjornfor-for-upstream' into dev 2018-02-18 19:49:27 +01:00
Bjørn Forsman 9fa6119f14 autogen.sh: break overlong sed line 2018-02-18 18:47:42 +01:00
Bjørn Forsman be38c3d90e autogen.sh: which -> command -v
command -v is a POSIX standard, available in all POSIX sh. which is not.
Prefer command -v for portability.
2018-02-18 18:46:04 +01:00
Bjørn Forsman 6ba79afc23 autogen.sh: /bin/rm -> rm
For portability, don't assume that /bin/rm exists, rely on $PATH lookup
instead.

This fixes building in sandboxed Nix(pkgs) builds.
2018-02-18 18:45:09 +01:00
Campus 12a1b0d7a3 Merge branch 'vel21ripn-config_gebug' into dev 2018-02-16 22:20:07 +01:00
Campus 029ca0797a Merge branch 'config_gebug' of https://github.com/vel21ripn/nDPI into vel21ripn-config_gebug 2018-02-16 22:18:43 +01:00
Michele Campus a5dd00eedb Merge pull request #523 from zyingp/dev
[For Xcode project] Fix some typos and add Macro definitions for Release build as well
2018-02-16 22:14:52 +01:00
Campus b98f79eaa6 Merge branch 'vel21ripn-libcache' into dev 2018-02-16 22:10:20 +01:00
Campus 16e57f64cd Merge branch 'libcache' of https://github.com/vel21ripn/nDPI into vel21ripn-libcache 2018-02-16 22:08:21 +01:00
Campus 9e47e49ade fix wrong increment for MPLS double header 2018-02-15 21:26:38 +01:00
Luca Deri df54e8b926 Improved numeric to protocol conversion 2018-02-12 21:44:48 +01:00
Luca 01949c47da Fixed typo 2018-02-12 11:08:37 +01:00
Luca Deri 52a2c0589e Updadate last protocol id
Updated tests with new protocols
2018-02-11 21:11:36 +01:00
Luca Deri 12368ba2d5 Implemented supporto for GoogleDocs and AmazonVideo
Fixed various bugs in detecting Google Drive
Improved Spotify support
2018-02-11 19:36:13 +01:00
Yingpei Zeng 053dbd73a6 [For Xcode project] Change some typos in README.md and add macro definition for release build as well 2018-02-11 13:58:09 +08:00
Yingpei Zeng 8f631fa19d Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2018-02-11 11:09:51 +08:00
Luca Deri 344392868b Added 179 to the default BGP port list 2018-02-09 18:24:35 +01:00
Vitaly Lavrov 0624afd422 Added option --enable-debug-messages.
The configurator option "--enable-debug-messages" is added to enable
debug information output (define NDPI_ENABLE_DEBUG_MESSAGES=1).
Mandatory inclusion of the file ndpi_config.h in all the compiled files.
2018-02-07 16:31:35 +03:00
Vitaly Lavrov c3f5307a3d Update libcache
Using ndpi_calloc(), ndpi_free() in libcache.
Fixing warnings about mixing declaration and code.
2018-02-06 14:06:38 +03:00
zyingp e6412949f1 Merge pull request #1 from ntop/dev
Merge the latest main branch
2018-02-05 09:32:28 +08:00
Luca Deri ea74d87fd8 Merge pull request #519 from zyingp/dev
Add an Xcode project for easier debugging on MacOS
2018-02-04 10:01:39 -08:00
ZengYingpei 5e0880dac6 Reset optind of getopt_long for synthesizing command line input stably in the Xcode project 2018-02-04 15:34:04 +08:00
ZengYingpei 26e36a19bf Unify Copyright declaration for all added files of the Xcode project 2018-02-04 14:35:56 +08:00
ZengYingpei aea81e4a1a Add an Xcode project for debugging under MacOS (add all missed files) 2018-02-04 14:21:52 +08:00
ZengYingpei 39b9f6752c Add an Xcode project for debugging under MacOS 2018-02-04 14:19:38 +08:00
Campus 73b4019561 update results for test pcap 2018-02-03 19:56:14 +01:00
Campus 3b5d8177f7 fix wrong conversion dec to hex 2018-02-03 17:06:28 +01:00
Simone Mainardi f8c6e9d714 Fixes possible out-of-bounds due to malformed DHCP packets 2018-01-31 17:15:55 +01:00
Luca Deri da588f6a30 Improved hyperscan support 2018-01-26 23:40:23 +01:00
Luca ee7a30385b Added minor hardening code in STUN 2018-01-26 19:45:41 +01:00
Michele Campus 809535e0a2 Merge pull request #510 from vel21ripn/dev
Fix buffer overflow in function ndpi_debug_printf()
2018-01-26 15:04:38 +01:00
Vitaly Lavrov 57fb117571 Fix buffer overflow in function ndpi_debug_printf() 2018-01-26 13:03:17 +03:00
Luca Deri 78feabfad7 Inital hyperscan work 2018-01-24 22:11:05 +01:00
Luca Deri 54306411a4 Merge pull request #508 from emanuele-f/hyperscan
Initial support for Intel hyperscan pattern matching library
2018-01-24 18:09:12 +01:00
Luca Deri f2280c90a4 Merge pull request #482 from hvanderheide/dev
Revert the revert of 2155dd9 after fixing ntopng Travis
2018-01-24 18:08:44 +01:00
emanuele-f e09ebab0c2 Initial support for Intel hyperscan pattern matching library. 2018-01-24 16:47:23 +01:00
Luca Deri d18824a73f Cleaned up Amazon IP range
Improved HostShield VPN
2018-01-19 14:33:43 +01:00
Luca Deri d2d20a8aa4 Warnign fix 2018-01-16 16:39:30 +01:00
Luca Deri fbf7fe2424 Implemented Google services detection
Refreshed google-based services support
2018-01-16 15:57:45 +01:00
Luca Deri ced9658c63 Added ApplePush support 2018-01-15 20:13:34 +01:00
Hidde van der Heide d896ce7469 Merge remote-tracking branch 'upstream/dev' into dev 2018-01-10 10:40:54 +01:00
Michele Campus 77cb8ccaea Merge pull request #506 from cavedon/dev
Fix MPLS header parsing in ndpiReader.
2018-01-09 22:13:57 +01:00
Ludovico Cavedon 35dc6e27ae Fix MPLS header parsing in ndpiReader.
Reported at https://bugs.debian.org/886133.
The current parsing for the MPLS header in examples/ndpi_util.c has
multiple issues:
- the bitfield order is incorrect for little endian architectures
- ntohl() is applied to a 20 bit label, which has unclear purpose
- if multiple labels are detected, the while loop parsing labels will
never exit due to a missing re-read of the mpls label
- the last label is identified by looking inside the label field, while
it should be done by looking at the S bit

This change fixes the above issues.
Notice that bitfield ordering is implementation-dependent, so C
bitfields should not be used in the first place to parse network
packets.
2018-01-09 10:26:10 -08:00
Campus 61e297b851 add missing check for free TCP/UDP root in tree 2018-01-05 00:34:33 +01:00
Campus 1d1082342f added diameter protocol dissector 2018-01-02 13:47:46 +01:00
Luca Deri f5e366d8c4 Added third party libs 2017-12-25 19:57:03 +01:00
Luca Deri 8d52491508 Added simplified makefile
Fixed small warnings
Compilation fixes
2017-12-25 19:26:17 +01:00
Luca Deri caf602c4a9 Merged categories network and network_tool (way too similar)
Replaced former network_tool category with streaming category
2017-12-23 19:40:39 +01:00
Hidde van der Heide bf8c9c2625 Merge remote-tracking branch 'upstream/dev' into dev 2017-12-20 13:19:24 +01:00
Campus f024e72eff Merge branch 'qianguozheng-dev' into dev 2017-12-19 19:56:06 +01:00
qianguozheng 9561f37998 Fix ndpiReader long run crash due to dupilicate nodes exist, cause
double free failed.
2017-12-19 17:37:46 +08:00
Campus 13e592275a Merge branch 'madpilot78-fix_symbols' into dev 2017-12-15 23:51:33 +01:00
Campus 3f9434b81d Merge branch 'fix_symbols' of https://github.com/madpilot78/nDPI into madpilot78-fix_symbols 2017-12-15 23:51:15 +01:00
Luca Deri 00e6ee3dc3 Implemented GooglePlus support that replaced ancient Quake protocol now obsolete 2017-12-15 10:33:01 +01:00
Luca Deri a06c399530 Compilation fixes 2017-12-13 08:04:51 +01:00
Simone Mainardi cd12a86088 Implements ndpi_get_category_id 2017-12-11 20:15:00 +01:00
Luca Deri 61bc528159 Simplified IPv6 datatypes 2017-12-08 18:13:31 +01:00
Campus 811717e7c1 Merge branch 'vel21ripn-dev' into dev 2017-12-07 22:54:49 +01:00
Michele Campus cb8f4f87bf Merge branch 'dev' into dev 2017-12-07 22:45:28 +01:00
Luca dfd8cbc4e1 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2017-12-07 20:31:15 +01:00
Luca d88a73397d Fix for #491 (SpeedTest/Ookla) 2017-12-07 18:29:46 +01:00
Luca Deri 90ec8040e6 Improved Apple update pattern 2017-12-06 21:39:56 +01:00
Guido Falsi f1f8afdddf Add ndpi_set_proto_category, which is actually uses by ntopng. 2017-12-05 23:10:32 +01:00
Luca Deri 7fd8406916 Fixed invalid symbolic categories 2017-12-05 11:30:29 +01:00
Simone Mainardi b5a8c6bae8 Bump to version 2.3 2017-12-03 19:09:11 +01:00
Simone Mainardi a6c56b5682 2.2 changelog 2017-12-03 16:23:35 +01:00
Campus e5546eb988 Merge branch 'dsokoloski-pull-request' into dev 2017-11-30 21:59:55 +01:00
Darryl Sokoloski 3577072b12 Return before calling ndpi_match_host_subprotocol when dns_dissect_response enabled.
Otherwise responses will never be seen if a host sub-protocol matched.
2017-11-30 14:54:23 -05:00
Luca 5b03443693 Added test pcap for check_mk protocol 2017-11-30 10:15:19 +01:00
Luca Deri 8af18763cc Compilation fixes 2017-11-28 22:05:34 +01:00
Luca Deri 840dab894f Merge pull request #496 from cymaniac/check_mk
Added dissector for protocol check_mk
2017-11-28 21:53:16 +01:00
cymaniac df7015f3a5 Added dissector for protocol check_mk 2017-11-28 21:29:38 +01:00
Luca Deri 8547b50be7 Merge pull request #493 from evengx/dev
Fixed ndpi_win32.h so that it compiles on MinGW+GCC
2017-11-28 17:51:53 +01:00
Luca Deri 7090ef838c Minor optimizations 2017-11-28 00:44:08 +01:00
Jacob Masen-Smith 950f5db3dd Changed MinGW __forceinline strategy to something with less side effects 2017-11-23 20:40:43 -08:00
Luca 8eaf0cefa1 Warning fix 2017-11-23 15:39:20 +01:00
Luca Deri 8b1f8636d1 Further check if a STUN flow will eventually become RTP 2017-11-23 15:25:02 +01:00
Luca Deri 1b5c90cfbe STUN improvements 2017-11-23 14:16:27 +01:00
Jacob Masen-Smith 33e0be73f7 Fixed ndpi_win32.h so that it compiles on MinGW+GCC 2017-11-22 01:11:49 -08:00
Campus 6e62367b56 add ICQ (instant messaging client) detection - deleted unused skyfile protocol 2017-11-20 17:05:15 +01:00
Luca Deri d6176ff668 Improved ndpiReader output
Updated tests
2017-11-19 18:35:26 +01:00
Luca Deri 2831d64f6a Added YouTube Upload protocol (used the identified unused by NDPI_PROTOCOL_SKYFILE_PREPAID) 2017-11-19 18:11:37 +01:00
Campus a4532a4999 update Telegram match 2017-11-11 16:23:14 +01:00
Campus 3caac691d5 removed bigrams for src/lib/ndpi_content_match.c.inc 2017-11-11 15:56:16 +01:00
Hidde van der Heide ee957e0831 Merge remote-tracking branch 'upstream/dev' into dev 2017-11-09 10:14:02 +01:00
Luca c9f32ee76d Fixed begin/end type when capturing live traffic 2017-11-03 17:15:04 +01:00
Luca Deri a488c9b7dc Fixes #485 2017-11-03 09:32:43 +01:00
Vitaly Lavrov 50645e11d8 Merge branch 'dev' into dev 2017-11-02 10:59:21 +03:00
Michele Campus 3e53577936 Merge pull request #486 from vel21ripn/dev-sip
Bug fix. Protocol SIP: command 'CANCEL sip:' is not recognized.
2017-11-01 11:07:37 +01:00
Vitaly Lavrov a0980228ef Bug fix. Protocol SIP: command 'CANCEL sip:' is not recognized.
Add sip.pcap for tests ( Source: https://wiki.wireshark.org/SIP ).
2017-10-31 22:04:14 +03:00
Alfredo Cardigliano 6e6b623057 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2017-10-30 11:30:14 +01:00
Alfredo Cardigliano 159e6228fe --ndpi-proto-filter now accepts also protocol name 2017-10-30 11:30:09 +01:00
Luca Deri cf07c92538 Dropbox and apple improvements
Fixed warning
2017-10-29 18:31:31 +01:00
Campus 23e08c93b6 Merge branch 'dsokoloski-upstream-dhcp-client-ident' into dev 2017-10-27 09:52:47 +02:00
Darryl Sokoloski 169aa11adc Added DHCP class identifier support. 2017-10-26 21:07:00 -04:00
Alfredo Cardigliano 7e38a4f790 Update README.md 2017-10-26 23:23:29 +02:00
Vitaly Lavrov 2787c2390c Refactoring the debugging output.
levels of debug output:
 0 - ERROR: Only for errors.
 1 - TRACE: Start of each packets and if found protocol.
 2 - DEBUG: Start of searching each protocol and excluding protocols.
 3 - DEBUG_EXTRA: For all other messages.

Added field ndpi_struct->debug_logging for enable debug output of each protocols.

Simple macros for debugging output are added:
NDPI_LOG_ERR(), NDPI_LOG_INFO(), NDPI_LOG_DBG(), NDPI_LOG_DBG2(),
NDPI_EXCLUDE_PROTO()
2017-10-26 20:41:22 +03:00
Hidde van der Heide 441e43c167 Revert the revert of 2155dd9 after fixing ntopng Travis
This reverts commit 4f72b954da.
2017-10-24 17:05:48 +02:00
Simone Mainardi 4f72b954da Undoes 2155dd9 that breaks ntopng Travis 2017-10-23 15:20:30 +02:00
Campus 5dbe705dd3 Merge branch 'hvanderheide-dev' into dev 2017-10-23 11:05:18 +02:00
Hidde van der Heide 2155dd9153 Add libcache to installed headers and removed relative import 2017-10-19 16:10:56 +02:00
Luca Deri f7e95b4ee8 Fixed DHCP layout 2017-10-14 14:23:49 +02:00
Luca Deri 36adc23507 Fixed warning generated by some compilers 2017-10-14 13:15:35 +02:00
Luca Deri 2ee871f5e6 Added Android fingerprint 2017-10-14 12:59:28 +02:00
Luca Deri 39d7738042 Implemented DHCP fingerprinting 2017-10-14 12:29:25 +02:00
Luca Deri 75993242b0 Implemented HTTP User Agent discovery 2017-10-14 11:21:01 +02:00
Luca Deri 83d5a46ec4 Added ndpi_icmphdr datatype 2017-10-13 19:41:34 +02:00
Michele Campus 7f197d12dd Merge pull request #473 from beratx/dev
fixed bug
2017-10-12 11:33:47 +02:00
berat d273b67fd7 fixed bug 2017-10-12 11:18:05 +02:00
Michele Campus aefd16647e Merge pull request #472 from beratx/dev
extended (-b flag) statistics with top receiver stats
2017-10-10 09:15:34 +02:00
berat 98d1c762cd extended (-b flag) statistics with top receiver stats 2017-10-09 22:21:40 +02:00
Luca Deri 2476c3c1b6 Fixed bug in someip dissector (access out of boundary)
Fixed memory leak in ndpiReader
2017-10-08 18:55:40 +02:00
Campus ed830d2474 update README for json install lib 2017-10-06 20:04:26 +02:00
Luca Deri 8bcafff569 Fixes for iQIYI and 1kxun protocols 2017-10-06 19:08:08 +02:00
Luca Deri 9b91623d57 Added ndpi_is_subprotocol_informative() API call 2017-10-06 15:20:36 +02:00
Luca Deri 27d66f6845 Typo 2017-10-06 09:51:42 +02:00
Luca Deri b10e526364 Leak fix 2017-10-05 16:21:32 +02:00
Michele Campus cf26e8429e Merge pull request #469 from vel21ripn/dev
Fix undefined behavior (detected by gcc):
2017-10-03 16:08:14 +02:00
Michele Campus 094ddc2048 Merge pull request #468 from danielengbert/dev
Fixed issue #459 by adding missing symbols.
2017-10-03 15:58:07 +02:00
Vitaly Lavrov 25a4569e67 Fix undefined behavior (detected by gcc):
ndpi_define.h.in:218: Macros NDPI_SET(), NDPI_CLR(), NDPI_ISSET()
  runtime error: left shift of 1 by 31 places cannot be represented in type 'int'

http.c:364: strncpy((char*)flow->protos.http.nat_ip,(char*)packet->forwarded_line.ptr,len);
 runtime error: null pointer passed as argument 2, which is declared to never be null

ndpi_patricia.c:ndpi_comp_with_mask(): int m = ((-1) << (8 - (mask % 8)));
  runtime error: left shift of negative value -1

Original function ndpi_comp_with_mask() is unreadable and slow.
We have only 2 type of address: 32 bit and 128 bit. The optimized version works 6 times faster (IPv4).
2017-10-03 16:43:23 +03:00
dangbert 47f922fa7b Fixed issue #459 by adding missing symbols. 2017-10-02 15:35:22 -04:00
Michele Campus 82c244b04f Merge pull request #466 from vel21ripn/dev-fixerr
Fix error: Use of uninitialized structures in tcp_udp.c
2017-10-02 00:45:17 +02:00
Vitaly Lavrov f80f0eedc0 Fix error: Use of uninitialized structures in tcp_udp.c
Now the test results do not depend on the compiler optimization level and architecture.
2017-10-01 22:43:20 +03:00
Campus 1b162234f0 Merge branch 'vel21ripn-dev-tests' into dev 2017-09-29 12:18:00 +02:00
Campus 3ac47a5104 Merge branch 'dev-tests' of https://github.com/vel21ripn/nDPI into vel21ripn-dev-tests 2017-09-29 12:15:13 +02:00
Vitaly Lavrov b2f0558080 A more predictable sorting of the flows list.
It is necessary for correct testing.
2017-09-29 12:46:27 +03:00
Michele Campus 50624b390e Merge pull request #463 from vel21ripn/dev
Fixed a typo in lisp.c
2017-09-29 10:15:19 +02:00
Vitaly Lavrov 1dd93590e4 Fixed a typo in lisp.c 2017-09-29 10:04:03 +03:00
Campus 662ef2f65b update results after many commits 2017-09-27 16:17:28 +02:00
Michele Campus 83104eb8b6 Merge pull request #460 from beratx/dev
memory improvement again for (-b flag) statistics
2017-09-26 22:36:35 +02:00
emanuele-f cd2a328cc5 Add configure.ac and ndpi_define.h to gitignore 2017-09-26 20:25:40 +02:00
berat f255b10524 memory improvement again for (-b flag) statistics 2017-09-26 17:20:51 +02:00
berat 4ba6ba70c5 memory usage is improved (-b flag) 2017-09-26 17:01:10 +02:00
Michele Campus 0827a6bf8b Merge pull request #457 from la10736/fix_ndpiReader_dependencies
Fix #456: ndpiReader depend from uthash.h heder
2017-09-25 15:21:51 +02:00
Michele Campus 4348109931 Merge pull request #455 from beratx/dev
improved memory usage by (-b flag) statistics
2017-09-25 15:21:23 +02:00
Michele d'Amico 59097a2d30 Fix #456: ndpiReader depend from uthash.h heder 2017-09-25 15:08:55 +02:00
Luca Deri b129487636 Custom protocols are not set as application and no longer master protocols 2017-09-25 14:51:46 +02:00
berat 2c014c5693 improved memory usage by (-b flag) statistics 2017-09-25 12:45:30 +02:00
Michele Campus 80fa0b5f98 Merge pull request #454 from beratx/dev
improved (-x flag) bpf filter condition
2017-09-25 11:25:52 +02:00
berat 688d7c33e4 improved (-x flag) bpf filter condition 2017-09-25 11:14:57 +02:00
berat b0f3852fed improved (-x flag) bpf filter condition 2017-09-25 11:06:47 +02:00
Michele Campus cd595caf8d Merge pull request #452 from beratx/dev
fixed another memory leak
2017-09-25 08:29:04 +02:00
Luca Deri c115903fbb Patch for giving prority in case of match to custom-defined protocols 2017-09-24 22:49:07 +02:00
berat 807db80afc fixed another memory leak 2017-09-24 18:43:31 +02:00
Michele Campus 8939a72ea8 Merge pull request #450 from beratx/dev
fixed memory leak
2017-09-24 15:16:23 +02:00
berat baae16ec2f memory leak fixed 2017-09-23 19:55:47 +02:00
berat 41b40ddf18 memory leak fixed 2017-09-23 19:50:52 +02:00
Luca Deri 8cc2b04f3c Fixed category for email services 2017-09-23 11:34:12 +02:00
Michele Campus 98b3268bfb Merge pull request #449 from beratx/dev
typos and renames
2017-09-22 20:55:02 +02:00
berat f575959cd2 typos and renames 2017-09-22 19:00:27 +02:00
Luca Deri c2002cd48f Removed / from category names that can cause issues in applications 2017-09-21 19:35:03 +02:00
Luca Deri 9fb5981128 Added ndpi_category_set_name() API call 2017-09-19 22:39:00 +02:00
Luca Deri 1a4e57e736 Code rework 2017-09-17 23:34:46 +02:00
Luca Deri d6f7dd9c08 Implemented DHCP field 55 decode 2017-09-17 21:25:55 +02:00
Luca Deri 4176fd5068 Reworked categories 2017-09-15 11:56:40 +02:00
Luca Deri 1808519b5b Reorganized protocol families
Added new API calls ndpi_set_proto_breed() and ndpi_set_proto_category() for overwriting nDPI protocol specifications
2017-09-15 11:23:34 +02:00
Luca Deri 83b1b34c18 Added custom 5 protocol categories to be used by applications. They are currently empty
Added ndpi_is_custom_category() to check if a defined category is custom
2017-09-15 11:13:25 +02:00
Campus b4d535e184 Merge branch 'huniyaarif1-dev' into dev 2017-09-14 11:45:11 +02:00
huniya arif, xflowResearch f7e9264c36 LISP PROTOCOL by Huniya Arif (xFlowResearch) 2017-09-14 12:52:18 +05:00
huniya arif, xflowResearch d4d4f2a17c Added LISP protocol. By Huniya Arif (xFlowResearch) 2017-09-14 11:38:31 +05:00
huniya arif, xflowResearch d3d6a95da7 Added protocol LISP. By Huniya Arif xFlow Research 2017-09-14 11:05:20 +05:00
Michele Campus 8a6a8ea07e Merge pull request #445 from beratx/dev
Extended filter produced by -b flag
2017-09-13 11:22:04 +02:00
berat ecb048a7bc Extended filter produced by -b flag 2017-09-08 18:00:32 +02:00
Campus 8e8445e02f fix telegram detection 2017-09-03 16:40:41 +02:00
Campus 0e3248b0f7 Merge branch 'k0ste-newfeature' into dev 2017-08-31 11:32:30 +02:00
Campus 0e8756ab46 Merge branch 'newfeature' of https://github.com/k0ste/nDPI into k0ste-newfeature 2017-08-31 11:28:58 +02:00
Luca b2bc6039da Partial fix for #410: added case insensitive HTTP header checks.
If the user will provide a pcap for reproducing further potential issues reported we will consider them
2017-08-29 21:32:52 +02:00
Luca 22beb7714d Implements #376 2017-08-29 20:56:08 +02:00
Luca 299b9435d0 Fixed memory out-of-bound issue
Added extra check for fixing #366
2017-08-29 19:56:32 +02:00
Konstantin Shalygin 31478703c2 Backported CSGO/DOTA2 protocol. 2017-08-26 15:41:04 +07:00
Michele Campus a1367a9d39 Merge pull request #438 from pauloangelo/dev
HTTP respose status code after validation check.
2017-08-21 14:08:45 +02:00
Paulo Angelo 7dc20134c0 Merge remote-tracking branch 'upstream/dev' into dev 2017-08-19 14:21:06 -03:00
Paulo Angelo 63c97753a1 Respose status code after validation check. 2017-08-19 14:19:30 -03:00
Campus 5515c3a109 fix for https://github.com/ntop/nDPI/issues/437 2017-08-18 10:11:41 +02:00
Luca Deri 5547ef06b3 Collapsed email send and receive categories in a new email category 2017-08-17 17:55:52 +02:00
Campus 65b39240aa Merge branch 'pauloangelo-i434' into dev 2017-08-14 10:44:43 +02:00
Paulo Angelo 449e06df77 Code formating and some improvements on HTTP (issue 434) 2017-08-11 17:52:32 -03:00
Paulo Angelo c4885fe260 Merge remote-tracking branch 'upstream/dev' into dev 2017-08-11 13:39:12 -03:00
Paulo Angelo 5a8aa3d72d Windows 10 detection from UA and indentation 2017-08-11 13:38:46 -03:00
Campus 7ad04a2184 Merge branch 'pauloangelo-dev' into dev 2017-08-09 19:41:52 +02:00
Paulo Angelo d8ed5b1005 final revision. 2017-08-09 14:17:07 -03:00
Paulo Angelo 93ceb14302 few classification adjusts 2017-08-09 14:08:28 -03:00
Campus 0326442faf add SoundCloud detection for https://github.com/ntop/nDPI/issues/396 2017-08-09 18:14:41 +02:00
Campus 89f1e663fa update results 2017-08-09 15:57:15 +02:00
Campus 5e2f75f5f0 Merge branch 'pauloangelo-dev' into dev 2017-08-09 15:51:48 +02:00
Paulo Angelo 7cab6efcd5 Issue 118 implementation (risk classification), indentation, and PasteBin&LinkedIn added. 2017-08-08 18:06:33 -03:00
Paulo Angelo d79c5ffc24 Identation and comments only. 2017-08-08 15:48:27 -03:00
Luca 3ba3a08141 Implemented flow sort based on total bytes so that we can (with -v X) immediately spot elephants and mice 2017-08-02 20:15:21 +02:00
Luca Deri 1d4eeaa328 Removed invalid networks from google list 2017-07-30 13:57:12 +02:00
Campus b1f6ac1a84 updated results 2017-07-29 23:48:25 +02:00
Campus a90b97bc10 fix wrong datalink type for Cisco PPP 2017-07-29 23:44:47 +02:00
Michele Campus f336f1a340 Merge pull request #428 from k0ste/newfeature
Added Sony Playstation protocol.
2017-07-29 22:14:52 +02:00
Konstantin Shalygin fbfe3e004c Added Sony Playstation protocol. 2017-07-29 16:52:03 +07:00
Luca 9a809bf673 Improved TeamViewer support 2017-07-29 06:55:50 +02:00
Michele Campus baa761a2e2 Merge pull request #427 from alagoutte/fix_warning
Fix warning
2017-07-28 18:59:47 +02:00
Alexis La Goutte ad401c4192 ndpi_main.c: fix unused variable 'rc' [-Werror,-Wunused-variable] 2017-07-28 13:46:02 +00:00
Alexis La Goutte 5ebc85e705 ndpiReader.c: fix variable ‘fileName’ set but not used [-Werror=unused-but-set-variable] 2017-07-28 13:45:59 +00:00
Alexis La Goutte b021f5bce1 ndpiReader.c fix ‘getTopStats/deleteTopStats/top_stats_sort/scanners_port_sort/scanners_sort/deleteScanners’ defined but not used [-Werror=unused-function] 2017-07-28 13:45:56 +00:00
Alexis La Goutte 5f1131cae2 ndpiReader: fix file_first_time/_statsFilePath defined but not used [-Werror=unused-variable] 2017-07-28 13:45:49 +00:00
Campus f664aee00b results updated 2 2017-07-27 13:37:09 +02:00
Campus 01649f097c results updated 2017-07-27 13:15:37 +02:00
Campus abf5bea425 Merge branch 'MicahLyle-ssl-certificate-fix' into dev 2017-07-27 13:12:12 +02:00
Michele Campus 6ca6d96821 Merge branch 'dev' into ssl-certificate-fix 2017-07-27 13:10:08 +02:00
Luca Deri e6b594a626 Fixed TINC bug (cache usage)
Merged MS Lync with Skype (Microsoft renamed MS Lync in Skype for Business)
Renumbered Nintendo protocols in former MS Lync that was no longer used
Fix for #425
2017-07-27 00:18:15 +02:00
Luca Deri c15f2bda97 Implemented support for Nintendo (switch) gaming protocol 2017-07-26 19:59:46 +02:00
Luca Deri 41c720d951 Added test file for skype conf call 2017-07-26 11:57:51 +02:00
Luca Deri be52d4e9ad Removed unused variable 2017-07-25 16:56:15 +02:00
Luca 7ec3af2156 Removed reference to ndpi_init_placeholder_proto() that is no longer necessary 2017-07-25 16:45:22 +02:00
emanuele-f bab373f992 Add nintendo protocol 2017-07-24 12:06:13 +02:00
Micah Lyle 55d1362924 Updated tests to match new SSL server certificate fix/addition 2017-07-18 12:53:02 -07:00
Micah Lyle 6e85b93442 Implementation for extra packet processing if desired 2017-07-17 18:06:07 -07:00
Micah Lyle 2e13e44b9c Added testing scripts and commited files that only had the server diff on the right 2017-07-14 18:19:49 -07:00
Micah Lyle 0b72c2fce3 Fixed tcp ssl case where client cert got set but server did not 2017-07-14 14:55:03 -07:00
Campus 6d7f5552de changed name from _bpf_filter to bpfFilter to avoid compilation error in OpenBSD caused by redefinition by https://github.com/openbsd/src/blob/master/sys/net/bpf.h#L296 2017-07-13 16:22:34 +02:00
Campus c5bc91145f removed warning compilation for - warning: passing argument 3 of 'pcap_compile' from incompatible pointer type 2017-07-09 18:10:10 +02:00
Campus cfa9a79f67 fix for OpenBSD compile error https://github.com/ntop/nDPI/issues/417 2017-07-07 12:50:06 +02:00
Luca Deri d66e473a2c Code cleanup
Updated test results
Renamed SOMEIP -> someip
2017-07-06 09:28:15 +02:00
Luca Deri 9bfab472e4 Win fix 2017-07-06 08:29:21 +02:00
Campus 1a605e2b77 Merge branch 'beratx-dev' into dev 2017-07-01 11:45:19 +02:00
Campus 5c39d65f8f Merge branch 'dev' of https://github.com/beratx/nDPI into beratx-dev 2017-07-01 11:41:22 +02:00
Luca Deri 5bf6be7eb8 Improved checks on libnuma 2017-06-30 11:32:23 -04:00
berat e857c5fb41 extended bpf filter produced by -x flag to include hosts 2017-06-30 09:52:26 +02:00
Simone Mainardi 4a1ced3709 Removes invalid char in SOME/IP proto name 2017-06-27 12:10:50 +02:00
Campus dbb9724c4e added fix protocol https://github.com/ntop/nDPI/issues/372 2017-06-27 11:38:44 +02:00
Campus 100062a156 Merge branch 'YumingMa-dev' into dev 2017-06-26 12:14:50 +02:00
MaYuming a40507b8b2 fix compiler error about json 2017-06-25 16:58:50 +08:00
MaYuming 592b769f67 fix lync over ICE detection 2017-06-25 11:15:59 +08:00
Campus 5b8d566a79 Merge branch 'beratx-dev' into dev 2017-06-23 12:41:04 +02:00
berat 7235c9a73a Added -x flag. Produces a bpf filter based on diagnose information generated by -b flag 2017-06-23 12:14:31 +02:00
Luca 8395bdac97 Implemented RPC latency 2017-06-20 23:16:59 -04:00
Luca e2119896db Added protocl info in wireshark info columm 2017-06-20 14:19:55 -04:00
Campus 68f1653dec fix lync detection 2017-06-19 20:39:36 +02:00
Campus 30d29f7ac3 Merge branch 'YumingMa-dev' into dev 2017-06-19 20:37:52 +02:00
Campus 117e61c34b Merge branch 'dev' of https://github.com/YumingMa/nDPI into YumingMa-dev 2017-06-19 15:53:36 +02:00
Luca 412cb2e6e8 Label fix 2017-06-17 07:15:50 -04:00
Luca Deri b11390429c Merge pull request #409 from beratx/dev
Added scanner host stats to (-b) diagnose statistics
2017-06-17 07:08:55 -04:00
berat 5a08303c43 Added scanner host stats to (-b) diagnose statistics 2017-06-16 14:10:54 +02:00
Michele Campus 5fef520d0e Merge pull request #408 from beratx/dev
Fixed bug causing wrong results in (-b) diagnose statistics
2017-06-16 11:34:45 +02:00
berat 6edece628e Fixed bug causing wrong results in (-b) diagnose statistics 2017-06-16 11:24:16 +02:00
Luca be749e75a5 Removed tracing 2017-06-15 07:13:29 -04:00
Luca ac5cb53066 mDNS improvemennts to recognize reponses senst to non multicast addresses 2017-06-15 05:25:27 -04:00
Luca 684c924587 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2017-06-14 21:37:08 -04:00
Luca 090dc752bd Added OOO, restransmission, segment losts 2017-06-14 21:35:46 -04:00
Campus 348159dc3a Merge branch 'alexei-argus-SOMEIP' into dev 2017-06-14 16:04:15 +02:00
alexei-argus d4ca57884b CR fixes #2 2017-06-14 16:50:01 +03:00
alexei-argus 8ec2867433 CR fixes 2017-06-14 16:49:58 +03:00
Yoni.Linder 0aa5ebde5d CR comments and fixes 2017-06-14 16:49:56 +03:00
alexei-argus 6029510ee7 Add CR comments 2017-06-14 16:49:53 +03:00
Yoni.Linder 7d39ba2124 removed debug prints, changed line of documentation 2017-06-14 16:49:50 +03:00
Yoni.Linder ae959f2d78 SOMEIP debug + bug fixes 2017-06-14 16:49:47 +03:00
Yoni.Linder 968085c965 last updated protocol updated to SOMEIP(228) 2017-06-14 16:49:45 +03:00
alexei-argus 62a65aa4cc Fix case-sensitivity issue 2017-06-14 16:47:43 +03:00
Yoni.Linder bc7f0b3686 endianity for ports 2017-06-14 16:46:44 +03:00
alexei-argus 87b49c5249 Fix build errors 2017-06-14 16:46:42 +03:00
Yoni.Linder ef1a84fb39 ndpi someip initial version with hardcoded port enforcing 2017-06-14 16:43:47 +03:00
Luca Deri c46af1291a Improved AppleStore detection
Modified string search matching to match the longest match (and not the first one)
2017-06-13 15:06:27 +02:00
Luca Deri f3f83f2eee Adding AppleStore and PlayStore (work in progress) 2017-06-13 14:48:59 +02:00
MaYuming d9bef8fa46 fix ms-lync detection missing, More detail at https://github.com/ntop/nDPI/issues/401 2017-06-13 17:17:19 +08:00
Campus af3086fb82 add check for whatsapp in ssl 2017-06-09 13:22:58 +02:00
Campus e0366a9965 improved whatsapp and wechat detection by new ip-ranges 2017-06-09 13:15:09 +02:00
Campus 88d47399b5 updated results 2017-06-09 01:18:41 +02:00
Campus 05912a49ae fixed 1kxun application protocol to avoid false positive in udp detection 2017-06-09 01:15:11 +02:00
Michele Campus b462a0845f Merge pull request #398 from kYroL01/dev
added GoogleDrive detection
2017-06-07 18:20:46 +02:00
Campus 39dc99443c added GoogleDrive detection 2017-06-07 18:18:12 +02:00
Michele Campus c2f51cf7ab Merge pull request #395 from pavlosantoniou/dev
Use flag %d to print signed integers in ndpiReader.
2017-06-07 15:03:14 +02:00
Pavlos Antoniou 9c4a446c76 Use flag %d to print signed integers in ndpiReader. 2017-06-07 13:14:37 +02:00
Campus e49c9f36e8 Merge branch 'pavlosantoniou-dev' into dev 2017-06-07 11:27:37 +02:00
Pavlos Antoniou 1467ad68d4 Remove unused variables for ndpiReader.c and tinc.c 2017-06-07 11:18:38 +02:00
Luca Deri 9a4634adbb Fixes for latency calculation 2017-06-06 21:22:46 +02:00
Luca Deri 984eedc5ea Code cleanup 2017-06-06 11:40:30 +02:00
Luca Deri 7bf508df0a Improved ARP/VLAN stats 2017-06-05 18:05:02 +02:00
Luca Deri 5570e6cbee Fixed bug that was preventiv packets to be analyzed properly
Fixed ARP stats calculation
2017-06-03 18:39:43 +02:00
Luca Deri 3e529dd042 Cleanup 2017-06-03 12:38:00 +02:00
Luca Deri 2049e3d655 Added SSL dissection 2017-06-03 11:35:51 +02:00
Luca Deri 2d72e2a2ae Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2017-06-02 21:48:37 +02:00
Luca Deri 67aa9f2350 Implemented Application/Network latency
Implemented DNS stats
Moved all menu entries unser Tools -> ntop menu
2017-06-02 21:47:55 +02:00
Campus a5268b1137 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2017-06-02 20:10:58 +02:00
Campus 6b5f640849 improved spotify detection 2017-06-02 20:10:05 +02:00
Michele Campus 1b5b38a3ba Merge pull request #391 from deselmo/dev
Updated libcache's license
2017-06-02 12:13:55 +02:00
William Guglielmo e1b5b54c4c Update license libcache 2017-06-02 00:29:04 +02:00
cardigliano 22b7b407c3 compilation fix 2017-06-01 18:13:31 +02:00
Campus 284bb21d61 Merge branch 'deselmo-dev' into dev 2017-06-01 16:45:24 +02:00
Campus f6eebb9c81 Merge branch 'dev' of https://github.com/deselmo/nDPI into deselmo-dev 2017-05-31 09:57:00 +02:00
William Guglielmo b08fbe46c0 Delete test.c 2017-05-30 21:04:57 +02:00
William Guglielmo 4a751f9d05 Updated libcache 2017-05-30 19:17:41 +02:00
Luca c5c417e526 Snapchat is now detected as sociaal network 2017-05-30 07:03:55 +02:00
William Guglielmo 46284e1537 Update ndpi_typedefs.h 2017-05-29 21:30:30 +02:00
William Guglielmo 694bc039e8 Added tinc protocol detection 2017-05-29 19:09:32 +02:00
Luca d21dbee102 TYpo 2017-05-29 18:58:02 +02:00
Luca 2b9f8f6a89 Fixed nDPI report formatting 2017-05-29 16:46:28 +02:00
Luca Deri 61f8f56719 Implemented network latency dissector 2017-05-26 12:46:19 +02:00
Luca Deri d67d3eed3c Compilation fix for systems that do not have JSON-C installed 2017-05-26 12:11:37 +02:00
Campus 4829222e73 update results after modifications 2017-05-26 11:26:22 +02:00
Campus 554278e678 Merge branch 'beratx-dev' into dev 2017-05-26 11:24:01 +02:00
berat b1ecc45e4f Added timestamp, flow count and app. level protocol to (-b) diagnose statistics 2017-05-26 02:47:46 +02:00
Campus d17ea73273 update 2017-05-25 17:13:43 +02:00
Luca Deri 07b1a93026 Latency calculation (work in progress) 2017-05-25 16:44:48 +02:00
Luca Deri 98dcfd8393 Reworked MAC stats 2017-05-25 10:55:41 +02:00
Luca Deri e1efee8594 Added check for searching JSON-C in home directory if not installed
Fixed type
2017-05-24 10:11:04 -07:00
Luca 993605d4f3 Fixed bug that was forcing the reader to pass several times the same flow
renamed some variables
2017-05-24 18:53:51 +02:00
Luca Deri 904210e87c Split src2dst / dst2src traffic 2017-05-24 16:06:23 +02:00
Luca Deri 50e26ca400 Implemented MAC stats 2017-05-24 00:51:21 +02:00
Luca Deri c723f7e668 Added VLAN / ARP stats 2017-05-23 23:31:43 +02:00
Alfredo Cardigliano 1a20029c82 compilation fixes 2017-05-23 18:56:43 +02:00
Alfredo Cardigliano 26bd42a71c crc32 fix 2017-05-23 18:02:53 +02:00
cardigliano 601f7f59ff forcing quiet mode in extcap mode 2017-05-22 16:55:48 +02:00
Luca Deri b39eb67836 Moved to 2.1.0
Updated test results
2017-05-22 14:44:31 +02:00
Luca Deri cb6aabbf19 Fixed JSON format when -v 3 and -b is used 2017-05-22 14:14:45 +02:00
Luca Deri b89a2dce6f Removed warning 2017-05-22 01:16:38 +02:00
Campus 97a57ac68d Merge branch 'beratx-dev' into dev 2017-05-20 10:20:16 +02:00
Campus a487a58884 Merge branch 'dev' of https://github.com/beratx/nDPI into beratx-dev 2017-05-20 10:19:44 +02:00
Michele Campus ccfe37870c Merge pull request #386 from Manouchehri/brew
Added HEAD for homebrew.
2017-05-20 10:08:34 +02:00
David Manouchehri 867acf717a Added HEAD for homebrew. 2017-05-19 19:34:52 -04:00
berat 6f27a48721 Added -b for saving port based diagnose statistics in json 2017-05-19 23:26:02 +02:00
Michele Campus 185cd4fb30 Merge pull request #383 from pavlosantoniou/dev
Remove unused variable and clarify operator precedence in ndpi_util.c
2017-05-18 16:52:57 +02:00
Pavlos Antoniou 73b3ee51d1 Remove unused variable and clarify operator precedence in ndpi_util.c 2017-05-18 14:08:01 +03:00
Michele Campus b579408711 Merge pull request #382 from pavlosantoniou/dev
Remove redundant check from btlib.c
2017-05-18 12:16:30 +02:00
Pavlos Antoniou ec0e6f98d4 Remove redundant check from btlib.c 2017-05-18 13:08:45 +03:00
Campus b4b682a4bf Merge branch 'pavlosantoniou-dev' into dev 2017-05-18 10:47:17 +02:00
Pavlos Antoniou 167732cae4 Check pointer for NULL before dereferencing 2017-05-18 11:09:56 +03:00
Michele Campus 2b128a2f1b Merge pull request #380 from pavlosantoniou/dev
Use correct subscripts when checking IP array in ndpi_main.c
2017-05-18 10:06:12 +02:00
Pavlos Antoniou f4bba23ba7 use correct subscripts when checking IP array in ndpi_main.c 2017-05-18 10:42:23 +03:00
Luca Deri 724d182939 Restored file selection as the wireshark bug seems to be fixed in 2.3.x wireshark series 2017-05-17 22:33:44 +02:00
Campus ebb6d4499a update Snapchat detection 2017-05-17 17:33:16 +02:00
Campus b2282a49b5 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2017-05-17 17:20:47 +02:00
Campus a044178b47 Merge branch 'pavlosantoniou-dev' into dev 2017-05-17 17:17:38 +02:00
Luca a874e7869c Typo 2017-05-17 17:12:22 +02:00
Luca cef42b4050 Updated tests instruction 2017-05-17 17:11:54 +02:00
Pavlos Antoniou 0a2cbe4926 Fix buffer overflows in http.c 2017-05-17 18:11:25 +03:00
Luca Deri 99c9f1d36b Updated results are ndpiReader output has been modified 2017-05-17 15:53:52 +02:00
Luca Deri 0091f35e02 Reworked flow structure handler 2017-05-17 15:50:59 +02:00
Simone Mainardi f2730f06a0 Changlogs for the forthcoming release 2017-05-12 14:43:28 +02:00
Campus 5271233774 Merge branch 'alexei-argus-example_caplen_fix' into dev 2017-05-11 15:59:17 +02:00
Campus 5da83360aa Merge branch 'example_caplen_fix' of https://github.com/alexei-argus/nDPI into alexei-argus-example_caplen_fix 2017-05-11 15:50:09 +02:00
Campus cd26942edf delete check for old protocol 2017-05-11 15:48:12 +02:00
alexei-argus 8a7dc10f51 Fix out-of-bounds heap read caused by using header->len instead of header->caplen (as provided in pcap_loop) 2017-05-10 14:16:46 +03:00
Campus c2bbe79a3e Results update 2017-05-09 19:57:47 +02:00
Campus e6dc092b73 Merge branch 'beratx-dev' into dev 2017-05-09 19:52:46 +02:00
berat 5c1954012b Added IPv6 support to Port Stats 2017-05-09 19:32:40 +02:00
Luca 21567e8686 Added missing direction update with non TCP-UDP protocols 2017-05-08 08:29:18 +02:00
Campus 1af3d571d9 fix for https://github.com/ntop/nDPI/issues/371 2017-05-02 16:50:25 +02:00
Luca Deri 442adce51d Fixed invalid ndpi_match_string_id() return code 2017-05-01 23:05:42 +02:00
Luca Deri 3f5e4cb7ee Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2017-05-01 21:40:02 +02:00
Luca Deri 4c6f9d5c25 Result update 2017-05-01 21:39:46 +02:00
Luca Deri a43244e493 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2017-05-01 21:36:59 +02:00
Luca Deri 5c2fa2d2e7 Updated results 2017-05-01 21:36:45 +02:00
Luca Deri aa6167bb42 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2017-05-01 21:21:03 +02:00
Luca Deri 205b82f6ba Added new API calls
- ndpi_add_string_value_to_automa
- ndpi_match_string_id
2017-05-01 21:20:43 +02:00
Luca Deri a03a343723 Minor code cleanup 2017-05-01 19:44:46 +02:00
Luca Deri 92d19ae2ed Merge pull request #370 from beratx/dev
Added IP Statistics to Port Stats
2017-05-01 19:36:49 +02:00
berat 32d02836f0 fixed syntax for gcc compiler 2017-05-01 14:37:12 +02:00
berat 4030b52ecb Added IP Statistics to Port Stats 2017-05-01 14:07:41 +02:00
Luca Deri 064b50df81 Added -m for splitting analysis in sub-analysis steps 2017-04-30 10:12:28 +02:00
Luca Deri af5021d571 Fixed warning 2017-04-29 08:32:28 +00:00
Luca e1b905124f Improved SSL dissection 2017-04-27 14:57:04 +02:00
Luca Deri 6cae3a7dce Removed Napatech dependency 2017-04-27 12:17:32 +02:00
Luca Deri d4a16d9e55 Improced extcap configuration window with sorted protocol list
Reported flow stats in Statistics -> nDPI menu
2017-04-25 11:21:40 +02:00
Luca Deri 6c2c885176 Extcap crash fix 2017-04-25 10:42:03 +02:00
Luca Deri 43f18ddb0f Updated Wireshark lua dissector
Added -v to ndpiReader to print port stats
Recomputed protocol test results
2017-04-24 00:45:43 +02:00
Luca Deri b9a2511ea8 Initial Wireshark nDPI integration 2017-04-23 19:58:41 +02:00
Campus c96507b7a4 updated tests pcap 2017-04-21 02:01:09 +02:00
Campus e2f2111637 update bittorrent detection - fix https://github.com/ntop/nDPI/issues/362 2017-04-21 01:42:54 +02:00
Campus 9a94c34b91 fix for https://github.com/ntop/nDPI/issues/364 2017-04-20 23:00:11 +02:00
Campus 6581579d37 update flow thanks to commit a9c01ded17 2017-04-20 17:57:40 +02:00
Luca Deri 4cc54cceb5 Added port stats when verbose mode (-v) is used 2017-04-19 23:31:45 +02:00
Luca Deri a9c01ded17 ndpiReader now prints (-v) the flows with the correct direction 2017-04-19 21:55:49 +02:00
Luca cbc08f7e50 Added DNScrypt test files 2017-04-19 09:58:35 +02:00
Luca 45582194cf Added DNScrypt detection 2017-04-19 09:56:51 +02:00
Luca 56b6c7ed93 Fixed thread crash caused by commit 29cd6ef994 2017-04-19 09:50:54 +02:00
Luca c1db9a830d Compilation fix for old compilers 2017-04-18 17:20:43 +02:00
Luca 4c5de9ef8e Added ethernet checksum reforging and nDPI protocol export in nDPI-extcap reader 2017-04-17 09:38:30 +02:00
Campus 3a21152f4c improved wechat detection 2017-04-15 21:18:50 +02:00
Campus b97e3ee51a add initial detection for WeChat 2017-04-15 11:23:03 +02:00
Campus 2b0809f3dc name correction for llc_snap header and minor fix 2017-04-15 00:40:01 +02:00
Campus 29cd6ef994 fix segmentation fault caused by missing spanning tree check - add control for threads return values 2017-04-15 00:29:53 +02:00
Luca Deri d8b2189cc3 Initial wireshark integration via extcap interface 2017-04-14 00:07:46 +02:00
Michele Campus 3a8c2d0b6f Merge pull request #360 from jaspervandenberg/dev
unified HTTPDownload name with other HTTP protocols (as HTTP_Download)
2017-04-07 17:01:39 +02:00
Jasper van den Berg eed3ba5d1e unified HTTPDownload name with other HTTP protocols (as HTTP_Download) for consistency 2017-04-07 11:47:44 +02:00
Luca bc68949c76 Added AMQP (Advanced Message Queueing Protocol) 2017-04-07 08:05:39 +02:00
Luca e4ef2d8140 Improved Ookla protocol 2017-04-04 14:42:45 +02:00
emanuele-f 0659f9b592 Replace BSD dependent strnstr with ndpi portable one 2017-04-02 11:17:57 +02:00
Luca Deri 0390c8ccfb Added Ookla test pcap
Updated results
2017-04-01 21:39:47 +02:00
Luca Deri 281ba768ec Added Ookla (SpeedTest) protocol detection 2017-04-01 21:33:15 +02:00
Campus 806fae6596 Merge branch 'vpiserchia-set_memory_functions' into dev 2017-03-29 10:59:41 +02:00
Campus 65359e53d8 Merge branch 'set_memory_functions' of https://github.com/vpiserchia/nDPI into vpiserchia-set_memory_functions 2017-03-29 10:46:38 +02:00
Luca Deri 92b1b46fa6 SSL server certificate is now detected 2017-03-21 00:48:16 +01:00
Luca Deri 760d468aca Improved netbios name decoding 2017-03-19 23:42:36 +01:00
Luca Deri 6a5b380bc7 Exported ndpi_netbios_name_interpret() to all nDPI apps 2017-03-19 22:51:13 +01:00
Luca Deri 689d3e7602 Improved netbios dissection 2017-03-19 22:38:13 +01:00
Vito Piserchia 4300208642 Added ndpi_flow_ free/malloc and set_ functions:
Ideally these two new functions will provide more flexibility
in regards of application that want to use specific allocation
approaches, for example storage queues for this structures
from where the application can pop and release the data it
needs.
Semantically they can be set up as the other ndpi_malloc and ndpi_free
functions, that is via the set_ndpi_flow_malloc and set_ndpi_flow_free.
In case a NULL parameter is passed to the above functions the old
approach will be used, that means that the ndpi_malloc will be used as
the ndpi_flow_malloc function and ndpi_free_flow as the ndpi_flow_free one.

Application that use the old functions will be not affected by this change,
and the ndpi_free_flow can be deprecated in the future and made private.
2017-03-16 17:35:10 +01:00
Campus 149818671d minor fix for http 2017-03-16 10:31:36 +01:00
Luca Deri dc1f005823 Tests update 2017-03-15 20:25:08 +01:00
Luca Deri eeb6f9334a Warning fix 2017-03-15 20:21:18 +01:00
Campus 3add72c686 deleted useless function - change macro name 2017-03-15 18:42:44 +01:00
Campus 5de6a96af5 change behaviour for https://github.com/ntop/nDPI/issues/344 - fix skype detection deleting parts that detect host name and ip addr (duplicate behaviour) - improved any detection after fix - added skype host name 2017-03-15 17:28:31 +01:00
Michele Campus 1e014aa95b Merge pull request #348 from jsoref/spelling
Spelling fixes
2017-03-13 14:46:22 +01:00
Luca Deri 9962eb1f08 Fixed invalid QUIC header parsing 2017-03-13 14:33:03 +01:00
Luca Deri 6bc35b92d3 Result file for youtube_quic.pcap 2017-03-13 01:19:35 +01:00
Luca Deri 40aaa05456 Minor changes for tracking Youtube content
Added test pcap for working on quic/youtube dissector
2017-03-13 01:16:58 +01:00
Josh Soref e17f62d261 spelling: values 2017-03-12 22:21:51 +00:00
Josh Soref 65cde78f6d spelling: unknown 2017-03-12 22:11:52 +00:00
Josh Soref 9a5402518c spelling: unencrypted 2017-03-12 22:13:00 +00:00
Josh Soref 11fc826e52 spelling: switch 2017-03-12 22:10:54 +00:00
Josh Soref 249b5d6783 spelling: structure 2017-03-12 22:10:38 +00:00
Josh Soref 7a2dcf0f9f spelling: soulseek 2017-03-12 22:10:05 +00:00
Josh Soref 710d528215 spelling: second 2017-03-12 22:07:00 +00:00
Josh Soref ce0a3e858a spelling: scrupulous 2017-03-12 22:06:44 +00:00
Josh Soref d178bec60e spelling: reset 2017-03-12 22:02:56 +00:00
Josh Soref fba88e225f spelling: remove 2017-03-12 22:06:05 +00:00
Josh Soref 7bac815938 spelling: protocol 2017-03-12 22:00:38 +00:00
Josh Soref 18916e682f spelling: prerequisites 2017-03-12 22:01:58 +00:00
Josh Soref fbe1fb285d spelling: platform 2017-03-12 22:00:21 +00:00
Josh Soref 04014d73d9 spelling: pedantic 2017-03-12 22:00:07 +00:00
Josh Soref 1aa10f5211 spelling: optimization 2017-03-12 21:58:56 +00:00
Josh Soref 35fe91116a spelling: ninety 2017-03-12 21:48:05 +00:00
Josh Soref fa34be1a00 spelling: modified 2017-03-12 21:52:59 +00:00
Josh Soref 7bb615a262 spelling: length 2017-03-12 21:45:36 +00:00
Josh Soref c51f88e62a spelling: interface 2017-03-12 21:44:26 +00:00
Josh Soref 120e3473f8 spelling: initialize 2017-03-12 21:43:57 +00:00
Josh Soref 4a5407feb7 spelling: incomplete 2017-03-12 21:43:45 +00:00
Josh Soref d332370298 spelling: googlesyndication 2017-03-12 21:37:41 +00:00
Josh Soref d5b15c498b spelling: five 2017-03-12 21:34:28 +00:00
Josh Soref 66e93da6f5 spelling: excluding 2017-03-12 21:33:20 +00:00
Josh Soref f8fed9be7e spelling: exclude 2017-03-12 21:33:32 +00:00
Josh Soref 3dbb97413c spelling: encryption 2017-03-12 21:32:57 +00:00
Josh Soref 47f9f54b89 spelling: discarded 2017-03-12 21:32:19 +00:00
Josh Soref e89ce717ba spelling: delivered 2017-03-12 21:31:56 +00:00
Josh Soref 0645625585 spelling: datastructure 2017-03-12 21:31:28 +00:00
Josh Soref e115e0b89e spelling: creating 2017-03-12 21:26:33 +00:00
Josh Soref 6a7ad10ae0 spelling: conservative 2017-03-12 21:22:34 +00:00
Josh Soref 5a2f556ff7 spelling: compatibility 2017-03-12 21:20:18 +00:00
Josh Soref b0e0b29201 spelling: associated 2017-03-09 04:56:31 +00:00
Josh Soref 616b6ce4d5 spelling: address 2017-03-09 04:54:42 +00:00
Luca Deri 6a27b6262d Added disclaimer 2017-03-07 19:25:34 +01:00
Luca Deri 940c185c5d Improved BJNP protocol 2017-03-07 10:40:42 +01:00
Luca Deri f831f30556 Fixed invalid SNMP detection 2017-03-07 10:19:29 +01:00
Campus fd93036b79 better improvement of netflix traffic detection - added netflix pcap and output - change little bit http detection behaviour 2017-03-05 12:38:15 +01:00
Luca Deri 889cdbff3c Fixed bug that can lkead to a crash in iqiyi dissector 2017-02-25 06:31:07 +01:00
Campus a787287bfa fix and improved netflix detection by user-agent 2017-02-22 17:14:52 +01:00
Campus c2c92b2e9b added double tagging 802.1Q in dissection of vlan pkt 2017-02-21 13:04:17 +01:00
Campus e68aeb69eb Merge branch 'k0ste-newfeature' into dev 2017-02-20 17:26:09 +01:00
Campus 846f457db4 update detection and tests 2017-02-20 17:25:19 +01:00
Campus 35ab1b5d1c Merge branch 'newfeature' of https://github.com/k0ste/nDPI into k0ste-newfeature 2017-02-20 16:47:46 +01:00
Campus 76e1bed466 update test dir to fix wrong pcap 2017-02-20 16:45:26 +01:00
Campus 100acbbe7b added missing .out 2017-02-20 16:42:54 +01:00
Konstantin Shalygin 66044578c2 ndpi_content_match: Add origin routes for many services. 2017-02-20 18:26:22 +07:00
Campus eb92c6b676 improved skype detection 2017-02-16 20:11:21 +01:00
Campus af04f78bd7 update Netflix ip-range list 2017-02-16 12:43:52 +01:00
Campus b35c1ab696 minor fix mgcp dissector 2017-02-14 01:23:47 +01:00
Campus 5c1c8b80e2 Merge branch 'numeronove-dev-numeronove-openbsd' into dev 2017-02-13 13:16:14 +01:00
Campus 787ca096e6 Merge branch 'dev-numeronove-openbsd' of https://github.com/numeronove/nDPI into numeronove-dev-numeronove-openbsd 2017-02-13 13:11:54 +01:00
Campus 363b38ceb0 fix dhcp buffer overflow 2017-02-13 13:03:27 +01:00
Luca Deri e2cfc96577 Improvements for exporting info in MDNS and UBNTAC2 protocols
Added test pcap file for UBNTAC2
2017-02-13 01:29:25 +01:00
Luca Deri b16769ddbe Refined guess that does not happen for protocols that have been excluded 2017-02-12 20:14:11 +01:00
Numeronove ffbe5d1e22 Easing OpenBSD build
Addedd macro AM_PROG_CC_C_O in configure.ac, needed to have autogen.sh
manage the error:

    C objects in subdir but `AM_PROG_CC_C_O' not in `configure.ac'

Modified case instance in example/ndpi_util.c, since DLT_LINUX_SLL
isn't defined in OpenBSD pcap.h.
2017-02-11 11:04:59 +01:00
Luca Deri 1d2e67d38c Dissected SSH client/server versions and reported in data structures 2017-02-11 10:48:08 +01:00
Alfredo Cardigliano b6eecd7d0c Update .travis.yml 2017-02-09 22:57:49 +01:00
Campus b229b89c78 fix for https://github.com/ntop/nDPI/issues/339 2017-02-09 17:01:12 +01:00
Simone Mainardi b161a6d7f7 Fix for FreeBSD build 2017-02-04 17:01:40 +01:00
Michele Campus 799e39227c Merge pull request #334 from alagoutte/fix
fix hardcoded value of version in Makefile
2017-02-04 12:22:54 +01:00
Michele Campus 38820ec916 Merge pull request #335 from alagoutte/warning
ndpi_main(.c): fix unused function 'tor_ptree_match' [-Werror,-Wunuse…
2017-02-04 12:22:22 +01:00
Alexis La Goutte be1ee30fa8 ndpi_main(.c): fix unused function 'tor_ptree_match' [-Werror,-Wunused-function] 2017-02-04 10:11:28 +01:00
Alexis La Goutte beb069df64 fix hardcoded value of version in Makefile
Issue report by @clauderobi (issue #333)
2017-02-04 09:01:46 +01:00
Luca Deri 9941045468 Renumbered ids that were changed by mistake
MQTT (NOW 208, latest stable version 222)
RX ( NOW 209, latest stable version 223)
OPENDNS ( NOW 197, latest stable version 225 ).
2017-02-01 18:10:02 +01:00
Luca Deri 8fa2d8f0d6 Improved whatsapp detection 2017-01-31 16:38:31 +01:00
Luca Deri a3d32d9d80 Improved HTTP subprotocol matching 2017-01-31 16:01:58 +01:00
Luca Deri bf35300817 Updated readme 2017-01-31 15:43:14 +01:00
Luca Deri ff9af3166f Updated results for HTTP-subprotocols 2017-01-31 15:06:21 +01:00
Luca Deri 8805d8523b Optimized code to avoid multiple calls to ndpi_network_ptree_match()
HTTP-based subprotocols have not HTTP set as master_protocols
2017-01-31 15:04:24 +01:00
Luca Deri 0610ae2d21 Fixed issues caused by (invalid) protocol renumbering. The problem has been solved and now protocols are in sync with its previous values in the stable version 2017-01-31 13:16:36 +01:00
Campus 5972fca190 Merge branch 'dillinger79-dev' into dev 2017-01-20 16:20:44 +01:00
dillinger79 9f8fedb3b1 Merge pull request #1 from dillinger79/dillinger79-patch-1
Update smpp.c
2017-01-20 11:31:36 +02:00
dillinger79 048ab4b5d3 Merge pull request #2 from dillinger79/dillinger79-patch-2
Add files via upload
2017-01-20 11:31:12 +02:00
dillinger79 8aa1d770f0 Add files via upload
this pcap file causes a busy loop
2017-01-18 18:27:56 +01:00
dillinger79 4e6dda6361 Update smpp.c 2017-01-18 18:21:47 +01:00
Luca e44834e0c5 Fixed warning 2017-01-02 22:05:40 +01:00
Luca Deri 1a3ee31865 Renamed enum constant 2017-01-02 21:52:53 +01:00
Michele Campus 721af38aed Merge pull request #322 from emanuele-f/dev
Add enum items number
2016-12-30 09:51:20 +01:00
emanuele-f 2ac04dba4d Add enum items number 2016-12-29 23:41:09 +01:00
Michele Campus e7fbcc690b Merge pull request #321 from alagoutte/fix_warning
SMPP: fix comparison of unsigned expression >= 0 is always true [-Wer…
2016-12-29 11:26:11 +01:00
Alexis La Goutte f4ce6a8023 SMPP: fix comparison of unsigned expression >= 0 is always true [-Werror,-Wtautological-compare] 2016-12-29 10:14:17 +00:00
Luca Deri 773dbf4ef2 Added BJNP default ports 2016-12-19 17:33:28 +01:00
Luca Deri 5486dcd1f9 Added default AFP port 2016-12-19 14:29:51 +01:00
Campus 61f0eb062e fix indent code and int type to keep conformity of the rest of the code (u_intxx instead uintxx) 2016-12-15 12:15:48 +01:00
Campus b68f197fea Merge branch 'dfranusic-dev' into dev 2016-12-15 12:12:15 +01:00
Campus 39e475b4ea Merge branch 'dev' of https://github.com/dfranusic/nDPI into dfranusic-dev 2016-12-15 12:00:08 +01:00
Damir Franusic f5b8855fed Implemented Short Message Peer-to-Peer (SMPP) dissector 2016-12-14 21:57:33 +01:00
Campus 03e8111e1b Merge branch 'k0ste-newfeature' into dev 2016-12-14 18:21:13 +01:00
Konstantin Shalygin e26b64a0fc Add Valve AS routes and domain names for better Steam detection. 2016-12-14 23:20:42 +07:00
Campus 8b1aa4a7bd improved steam detection 2016-12-13 19:00:34 +01:00
valentina b2b36be6cf Removed "yp" element from "ndpi_en_impossible_bigrams" structure because tor was detected instead of ssl (paypal). 2016-12-11 22:12:26 +01:00
Luca Deri 7d82948bd1 Improvement on ndpi_int_change_protocol() accuracy 2016-12-10 19:30:42 +01:00
Michele Campus f7515514bd Merge pull request #314 from sologub/dev
Fix 'IPVERSION undeclared' error on WIN32
2016-12-07 10:45:43 +01:00
Ruslan Sologub 12a3d80f71 Fix 'IPVERSION undeclared' error on WIN32 2016-12-07 11:36:39 +02:00
Campus ef69a1610a Merge branch 'denisvm-magicnumbers' into dev 2016-12-06 22:23:36 +01:00
Denis Volpato Martins 483eb63408 Removed some magic numbers 2016-12-06 18:48:26 -02:00
Luca Deri b019dcd11a Added missing -ldl library 2016-12-05 19:15:45 +01:00
Luca Deri 163945f4b4 Added check to avoid loops with invalid DHCP packets 2016-12-05 18:25:39 +01:00
Luca Deri 22ee439215 Added memory check in patricia tree
Improved AFP dissection
Updated DHCP test results
2016-12-05 13:58:43 +01:00
Luca Deri 582f5e0f72 Implemented DHCP host name extraction 2016-12-04 21:44:48 +01:00
Campus 5f672dfb42 added comment to function process_packet 2016-12-02 02:32:46 +01:00
Campus 6f31ae722e added prototype of function packet_processing 2016-12-02 02:25:43 +01:00
Campus cc5167c09a Merge branch 'alagoutte-osx' into dev 2016-11-28 21:45:44 +01:00
Alexis La Goutte c6eb41b7bf Remove gcc when build with osx
osx buildbot use always clang for the build
2016-11-28 19:16:09 +01:00
Alexis La Goutte eeef26ae59 Add macOS build on Travis 2016-11-28 10:55:29 +01:00
Alexis La Goutte d53b655c32 Update ax_pthread
for fix issue when build with clang on macos

clang: error: argument unused during compilation: '-pthread'
No need -pthread when linking
2016-11-28 10:45:54 +01:00
Luca Deri be81d33495 Disabled coverage (work in progress) 2016-11-27 12:21:02 +01:00
Luca Deri 035a290ab5 Coverage (work in progress) 2016-11-27 11:52:50 +01:00
Luca Deri c041a723c1 Added coverage 2016-11-27 11:34:52 +01:00
Luca Deri 45ca2ace0b Code cleanup. Added coveralls badge 2016-11-27 10:57:42 +01:00
Luca 424fbb710c Added fixes to properly honour subprotocols 2016-11-26 18:36:08 +02:00
Luca 169a8fb97f Test update 2016-11-25 17:52:19 +02:00
Campus 098812d3cc Merge branch 'mgigante-dev_libdl' into dev 2016-11-24 10:07:10 +01:00
Campus 4eb5297a79 Merge branch 'dev_libdl' of https://github.com/mgigante/nDPI into mgigante-dev_libdl 2016-11-24 10:05:56 +01:00
Marco Gigante c52c738408 Checking for libdl vs. libc during configure 2016-11-23 08:44:07 +01:00
Campus 797eb2c3ca fix initialization of struct ndpi_flow_info *flow 2016-11-22 23:11:24 +01:00
Campus d9a2d9a6bd updated facebook detection (continue) by ip server ranges 2016-11-22 22:57:31 +01:00
Michele Campus 4da4db1e73 Merge pull request #303 from alagoutte/fix_warning
Fix warning
2016-11-22 22:18:11 +01:00
Alexis La Goutte 4c23c3fec9 ndiReader: fix ‘jObj_main/jArray_detProto’ may be used uninitialized in this function [-Werror=maybe-uninitialized] 2016-11-22 20:56:14 +01:00
Alexis La Goutte 2ab1faf05a http: unused variable 'no_pps' [-Werror=unused-variable] 2016-11-22 20:56:12 +01:00
Campus f670eefd84 update facebook detection (2) for https://github.com/ntop/nDPI/issues/300#issuecomment-261977387 2016-11-21 17:28:53 +01:00
Campus 4e1c301b2a update facebook detection 2016-11-21 16:14:28 +01:00
Luca Deri b25e3e0917 Compilation fix 2016-11-20 20:24:39 +01:00
Campus f571d7ed3a fix ssl for https://github.com/ntop/nDPI/issues/301 2016-11-20 13:19:20 +01:00
Campus 9abbef7e05 added iqiyi media service and updated ppsetream protocol - added 1kxun media service 2016-11-20 13:07:00 +01:00
Campus 1fbe25f91e fix facebook detection in http dissector 2016-11-18 20:21:08 +01:00
Michele Campus 19ad081bb1 Merge pull request #298 from denisvm/twitter-nets
Update and merge Twitter networks
2016-11-18 10:23:12 +01:00
Denis Volpato Martins 911be714e5 Update and merge Twitter networks 2016-11-17 16:42:16 -02:00
Michele Campus f51fef6027 Merge pull request #296 from sologub/dev
Fix byte packing on Windows
2016-11-15 15:14:29 +01:00
Ruslan Sologub 8761e3a869 Fix byte packing on Windows 2016-11-15 15:47:08 +02:00
Campus b9a1eba152 updated host matching services 2016-11-15 12:44:39 +01:00
Campus 1e19b689c4 minor fix for https://github.com/ntop/nDPI/commit/0f94a46f868e6818790b0c278121c081a63c9eeb 2016-11-15 12:02:02 +01:00
Campus 0f94a46f86 Merge branch 'mkfsn-bjnp-protocol' into dev 2016-11-15 11:50:45 +01:00
mkfsn d9659617e9 Add BJNP Protocol 2016-11-15 17:59:02 +08:00
Luca Deri edf0b2fbfe Updated Twitter networks 2016-11-15 10:19:27 +01:00
Campus 61c8dca469 fixed wrong function call 2016-11-14 13:35:16 +01:00
Campus d86e504779 Merge branch 'alagoutte-warning' into dev 2016-11-13 18:45:50 +01:00
Alexis La Goutte 3d6b4d4dff ndpi_util: fix ‘type/ip_offset’ may be used uninitialized in this function [-Werror=maybe-uninitialized] 2016-11-13 18:22:03 +01:00
Alexis La Goutte 45bcb9927d bittorrent: fix variable ‘peer_id’ set but not used [-Werror=unused-but-set-variable] 2016-11-13 18:22:03 +01:00
Alexis La Goutte 612a07f420 ndpi_main: fix variable 'daddr/saddr' set but not used [-Wunused-but-set-variable] 2016-11-13 18:22:03 +01:00
Alexis La Goutte d129abcde0 ndpi_util: unused variable 'nDPI_traceLevel' [-Wunused-const-variable] 2016-11-13 18:22:03 +01:00
Alexis La Goutte 6fad364daa ndpi_util: fix unused variable 'ip6_offset/ethernet_2/llc_off/cdp/slarp' [-Wunused-variable] 2016-11-13 18:22:03 +01:00
Alexis La Goutte dc7a85ee76 ndpiReader: fix unused function 'debug_printf' [-Wunused-function]
Comment the function
2016-11-13 18:22:03 +01:00
Alexis La Goutte 26aec8714e ndpi_main: fix enumeration value 'NDPI_PROTOCOL_CATEGORY_UNSPECIFIED' not handled in switch [-Wswitch] 2016-11-13 18:22:03 +01:00
Alexis La Goutte 65f61fd5b1 rx: fix unused variable 'found' [-Wunused-variable] 2016-11-13 18:22:03 +01:00
Alexis La Goutte 57ad066e66 QUIC: fix unused variable 'begin' [-Wunused-variable] 2016-11-13 18:22:03 +01:00
Alexis La Goutte 2a2a0d4aeb git: fix unused variable 'git_len' [-Wunused-variable] 2016-11-13 18:22:03 +01:00
Alexis La Goutte 1082a234fe bittorrent: fix unused variable 'no_bittorrent' [-Wunused-variable] 2016-11-13 18:22:03 +01:00
Alexis La Goutte 141bb48d76 BGP: fix unused function 'ndpi_int_bgp_add_connection' [-Wunused-function] 2016-11-13 18:22:03 +01:00
Alexis La Goutte 85bb227b45 ndpi_main: fix unused variable 'node/ret' [-Wunused-variable] 2016-11-13 18:22:03 +01:00
Campus 2005979f46 update test results after new services added 2016-11-12 16:38:37 +01:00
Campus 48a206a434 Merge branch 'alagoutte-scan-build' into dev 2016-11-12 16:35:25 +01:00
Alexis La Goutte 3a6d2c7c5f ndpi_main: fix Dereference of null pointer 2016-11-12 16:30:15 +01:00
Alexis La Goutte 800ad1737c COAP: fix typo about checking twice source port
Found via Clang Analyzer (Used d_port variable)
2016-11-12 16:02:32 +01:00
Alexis La Goutte c7eb8d1082 HTTP: Fix Dead Store (Dead assignement/Dead increment) Warning found by Clang Analyzer 2016-11-12 16:00:10 +01:00
Michele Campus 4564b973df Merge pull request #289 from mkfsn/service-github
Add Github service
2016-11-11 18:46:22 +01:00
mkfsn 2586618c1b Add Github service 2016-11-12 00:15:31 +08:00
Campus 7758bc821e Merge branch 'vpiserchia-dev' into dev 2016-11-03 23:10:23 +01:00
Campus 78f7f19359 Merge branch 'dev' of https://github.com/vpiserchia/nDPI into vpiserchia-dev 2016-11-03 23:08:44 +01:00
Vito Piserchia 04e45f503b setup the debug function 2016-11-03 17:41:31 +01:00
Campus 58e97c966c Merge branch 'vpiserchia-dev' into dev 2016-11-03 15:39:28 +01:00
Vito Piserchia e3d15ef4e7 Prevent Segfault when building with -DNDPI_ENABLE_DEBUG_MESSAGES 2016-11-03 15:23:06 +01:00
Vito Piserchia 61f1ace43f Fix NDPI_PROTOCOL_MSSQL_TDS macro name 2016-11-03 15:22:04 +01:00
Vito Piserchia 087ca9d0d5 Merge remote-tracking branch 'upstream/dev' into dev 2016-11-03 15:08:56 +01:00
Luca Deri 002b8b04e9 Win fixes 2016-11-03 00:22:51 +01:00
Luca Deri 6f0232763c Added -ldl needed by PF_RING 2016-11-02 20:48:22 +01:00
Vito Piserchia 8169b35fa6 Merge remote-tracking branch 'upstream/dev' into dev 2016-11-02 10:16:16 +01:00
Campus 097ba22a59 minor fix to keep same coherence of type used 2016-10-30 01:14:49 +02:00
Vito Piserchia 78ebb3445a Merge remote-tracking branch 'upstream/dev' into dev 2016-10-25 14:37:31 +02:00
Campus b8fb5fb403 added iflix service 2016-10-24 23:25:29 +02:00
Luca Deri 93f01e614f Added new functions defined in #279 for exporiting them in shared libs
Disabked some search dissectors from Office365
Fixed warning
2016-10-23 13:00:41 +02:00
Luca Deri e9f00a4e8a Merge pull request #279 from emanuele-f/category
Add protocol category ids
2016-10-23 12:54:30 +02:00
emanuele-f 7d0397a8fb Add protocol category ids 2016-10-22 22:22:15 +02:00
Campus 2682ab6791 added skype ip-address server name range 2016-10-20 23:54:15 +02:00
Michele Campus b60e82c702 Merge pull request #278 from mgigante/dev_funcproto
Avoid c99 complains about function prototypes
2016-10-20 17:08:04 +02:00
Marco Gigante bac82eeb8e Avoid c99 complains about function prototypes 2016-10-20 16:01:53 +02:00
Campus 94c134318b fix msn dissector - add pcap for test 2016-10-17 18:43:47 +02:00
Luca Deri 6fd334dcd7 Updated quickstart guide
The new master file for the guide is keynote (deleted docx)
2016-10-15 11:41:40 +02:00
Campus df6b71c5ea fix debug function 2016-10-15 00:47:13 +02:00
Vito Piserchia 4c467d7cad Merge remote-tracking branch 'upstream/dev' into dev 2016-10-13 13:50:29 +02:00
Campus db794beb84 added vnc pcap to test folder 2016-10-12 00:14:46 +02:00
Campus 3b95cd9a11 fix vnc dissector 2016-10-12 00:10:06 +02:00
Campus bbe4717075 removed imesh protocol (iMesh is no longer availlable) 2016-10-09 19:12:15 +02:00
Campus 2c47dca086 fixed some dissector for logic or syntax 2016-10-07 11:37:42 +02:00
Vito Piserchia a17553cbf1 more ignores 2016-10-04 15:42:20 +02:00
Campus 7e8e243e68 fix check for issue https://github.com/ntop/nDPI/issues/272 2016-10-03 14:36:51 +02:00
Luca cf470ec038 Fix for #218 2016-09-28 18:15:51 +02:00
Luca 016623decb Added protos.txt to the list of tests 2016-09-28 17:50:17 +02:00
Luca Deri a0a8c0294d Fix for #249 2016-09-27 19:24:57 +02:00
Campus 9fe92f6145 fixed drda_db2 dissector 2016-09-27 11:17:45 +02:00
Campus 9f3804444e fixed drda_db2 dissector 2016-09-27 11:17:06 +02:00
lucaderi 67ac0b431c Fix for #249 (not sure is enough) 2016-09-27 08:01:15 +02:00
Campus 1f807a1348 fix snaplen for pcap_open_live function - optimal value is described here http://pcap.man.potaroo.net/ 2016-09-26 10:43:44 +02:00
Campus bf778d6df1 fix typo 2016-09-24 16:13:41 +02:00
Campus f8a9faf2fa added adobe flash host name - fix twitter and netflix detection - removed twitter dissector (useless have a dissector that looking for ip range; moved matching in ndpi_content_match.c.inc) 2016-09-24 13:45:28 +02:00
Campus 054f8376e4 deleted meebo dissector (service not availlable anymore) - change Weibo subservice name to Sina service (more correct) 2016-09-24 11:32:47 +02:00
Campus d601b0ce20 fix ftp on tcp 2016-09-22 18:59:56 +02:00
Campus 730c8d6879 fix wrong comment rsync - deleted old tds dissector 2016-09-16 02:19:20 +02:00
Campus dbb3d8280e fix and merge mssql and tds in unique dissector mssql_tds due to latest release + minor fixes 2016-09-16 02:05:14 +02:00
Michele Campus 36956ebf68 Merge pull request #267 from tianzhishuizhu/dev
check udp
2016-09-15 09:50:38 +02:00
tianzhishuizhu 3c8d5bb788 check udp 2016-09-15 10:49:03 +08:00
Michele Campus 9e9b64a7e7 Merge pull request #266 from tianzhishuizhu/dev
check udp
2016-09-14 10:35:07 +02:00
tianzhishuizhu 0a73a9bd9c Merge branch 'dev' of github.com:tianzhishuizhu/nDPI into dev 2016-09-14 16:16:37 +08:00
tianzhishuizhu d9addc9245 check packet udp != NULL 2016-09-14 16:16:24 +08:00
Michele Campus 6b4b7a9083 Merge pull request #265 from tianzhishuizhu/dev
improve readability
2016-09-13 10:51:35 +02:00
tianzhishuizhu afcf567538 Update syslog.c 2016-09-13 15:55:05 +08:00
tianzhishuizhu 2b67b3bc81 improve readability 2016-09-13 15:51:17 +08:00
Michele Campus 3cd6456349 Merge pull request #263 from tianzhishuizhu/dev
payload min length
2016-09-08 15:01:28 +02:00
tianzhishuizhu c10052764b payload min length 2016-09-08 20:55:26 +08:00
Michele Campus 2ae4db34b0 Merge pull request #261 from tianzhishuizhu/dev
optimization for code duplication
2016-09-08 14:48:54 +02:00
tianzhishuizhu 9ddaccbae0 text error 2016-09-08 19:53:03 +08:00
tianzhishuizhu f7ca670ce0 delete variables unused 2016-09-08 19:49:13 +08:00
tianzhishuizhu ce3493952b logical error 2016-09-08 17:02:50 +08:00
tianzhishuizhu ac8aaf0e08 optimization for code duplicationdup 2016-09-08 16:29:39 +08:00
Campus 3ceb19a9c6 fixed dhcp dissector 2016-09-07 10:31:11 +02:00
Luca Deri 95974b0d24 Merge pull request #258 from emanuele-f/facebook
Fix facebook certificate recognition
2016-09-06 08:10:35 +03:00
emanuele-f ba3bf6c3d9 Fix facebook certificate recognition 2016-09-06 00:44:17 +02:00
Campus 7300e88cdb fixed h323 dissector 2016-09-05 00:49:48 +02:00
Michele Campus 7364fcc7df Merge pull request #255 from emanuele-f/openvpn
Fix openvpn with multiple hard_reset_client not being recognized
2016-08-28 01:05:08 +02:00
emanuele-f 562b90d93c Fix openvpn with multiple hard_reset_client not being recognized 2016-08-28 00:41:39 +02:00
Campus 210602476f update Skype ip address range 2016-08-27 22:56:01 +02:00
Campus 58e37f8130 update Skype and MS_OneDrive range IP address 2016-08-27 22:25:28 +02:00
Michele Campus b6802767aa Merge pull request #253 from theirix/fix-drda2
Fix drda in addition to #242
2016-08-22 14:54:56 +02:00
theirix 76f7454049 Fixed drda loop logic 2016-08-22 15:09:06 +03:00
theirix fb687b7217 Fix another overflow in DRDA 2016-08-22 15:09:06 +03:00
Campus b34d418b57 fix indentation ssl.c 2016-08-22 12:03:56 +02:00
Michele Campus 45f8f72bae Merge pull request #252 from emanuele-f/gmail
Add hotmail service and enable full master protocol detection for sub-protocols
2016-08-22 11:56:06 +02:00
emanuele-f 11ca0a01d3 Enable full master protocol detection for sub-protocols 2016-08-22 11:32:57 +02:00
emanuele-f 5f4bbe311e Add Hotmail service 2016-08-22 11:08:22 +02:00
Luca Deri 305e310f0a Merge pull request #251 from Lzz/dev
Fix README.md
2016-08-20 08:28:47 +02:00
yimao ed35c466ae Fix README.md 2016-08-20 11:05:10 +08:00
Michele Campus 48341094a3 Merge pull request #248 from theirix/build-check2
Added instructions how to run tests
2016-08-13 16:22:40 +02:00
theirix fb73ab95ea Added instructions on running tests 2016-08-13 17:17:35 +03:00
theirix 9c4f80449b Fixed dist build 2016-08-13 17:14:01 +03:00
Luca Deri dc88cc4e64 Merge pull request #247 from theirix/build-check
Allowed to run tests by 'make check' in addition to plain do.sh
2016-08-13 13:32:20 +02:00
theirix bee9090a92 Allowed to run tests by 'make check' 2016-08-13 13:35:14 +03:00
Michele Campus d1bf5d2bb6 Merge pull request #245 from denisvm/service-slack
added Slack service
2016-08-13 10:54:27 +02:00
Denis Volpato Martins 9c2eba2009 fix Slack service domains without A DNS RRs and fix slack-imgs.com access 2016-08-12 15:18:11 -03:00
Denis Volpato Martins 44271e7bb8 added Slack service 2016-08-12 15:08:32 -03:00
Campus 5e399498c3 minor fixes 2016-08-12 15:51:41 +02:00
Campus 3bfa021760 fix quic result detection for YouTube 2016-08-12 15:33:34 +02:00
Michele Campus 7853c4a90b Merge pull request #244 from emanuele-f/starttls
Add STARTTLS detection
2016-08-12 15:27:11 +02:00
emanuele-f e7613a3851 Add IMAPS protocol on stack for use on detection giveup 2016-08-12 14:28:16 +02:00
emanuele-f b34e9bf42a Keep previous code 2016-08-12 13:28:40 +02:00
emanuele-f 6e1341c148 Add STARTTLS detection 2016-08-12 13:19:42 +02:00
Campus 92cac451c8 added new Youtube server name to improve detection 2016-08-11 20:06:08 +02:00
Michele Campus 0e04068952 Merge pull request #242 from theirix/fix-drda
Fixed DRDA detection
2016-08-11 09:16:42 +02:00
theirix 0a06b3b208 Fixed DRDA detection 2016-08-11 01:55:24 +03:00
Campus 8bccfa5b10 fixed invalid termination with ipv6 address 2016-08-09 23:00:33 +02:00
Michele Campus 5e1e6a48a4 Merge pull request #241 from theirix/fix-quic
Fixed buffer overflow in QUIC
2016-08-08 09:52:38 +02:00
theirix 4981005572 Fixed QUIC crash 2016-08-07 22:37:36 +03:00
Luca Deri 9e207f7f24 Fix for systems without beXXtoh 2016-08-05 17:29:33 +02:00
Michele Campus 5ec1b685d8 Merge pull request #240 from emanuele-f/openvpn
Add openvpn test pcap
2016-08-05 09:16:49 +02:00
emanuele-f c41571879b Add openvpn test pcap 2016-08-05 01:20:05 +02:00
Michele Campus 62a5f2350b Merge pull request #239 from emanuele-f/openvpn
OpenVPN detection by opcode, packet ID and session ID
2016-08-05 00:21:31 +02:00
emanuele-f 82f10ddc6a OpenVPN detection by opcode, packet ID and session ID 2016-08-04 23:08:09 +02:00
Michele Campus 4fa76bee78 Merge pull request #235 from theirix/fix-overflow-oscar
Fixed overflow in Oscar
2016-07-26 10:37:36 +02:00
theirix 384dee1f95 Fail when Oscar family or type is 0 2016-07-25 18:24:45 +03:00
theirix 3e5033ff13 Fail when Oscar family is 0 2016-07-25 17:44:45 +03:00
theirix 01ac65eee4 Fail fast when Oscar family cannot be detected 2016-07-25 17:02:39 +03:00
Luca Deri da80bd8bb4 Added ndpi_get_proto_by_name() for mapping protocolIDs to protocol names
Added a new parameter to ndpi_guess_protocol_id() to indicate if the guessed protocol has been set by the user through a custom protocols file
2016-07-25 14:52:44 +02:00
theirix 470901dc58 Fixed overflow in Oscar 2016-07-25 15:23:44 +03:00
Michele Campus 32e6a79e9c Merge pull request #234 from theirix/fix-overflows
Fixed payload overflow in MQTT, DNS
2016-07-25 12:12:47 +02:00
theirix d1f0b56adc Fixed payload overflow in MQTT, DNS 2016-07-25 12:58:27 +03:00
Michele Campus 77fc4be458 Merge pull request #233 from theirix/fix-bgp
Fixed BGP payload overflow
2016-07-25 10:22:47 +02:00
theirix 3dec8d1880 Fixed BGP payload overflow 2016-07-25 11:19:00 +03:00
Michele Campus b102caf482 Merge pull request #232 from theirix/dns-int-overflow
Fixed integer overflow in DNS extensions
2016-07-25 10:05:47 +02:00
theirix 7ec9e78e4d Fixed integer overflow in DNS extensions 2016-07-25 10:04:39 +03:00
Campus b6237bc39d minor fixes hangout 2016-07-20 13:31:29 +02:00
Luca Deri 54935e4aa3 Removed un-necessary files 2016-07-20 01:41:26 +02:00
Luca Deri 18b9addc85 Implemented #228 2016-07-20 01:40:16 +02:00
Luca Deri abdccaa8eb Harvested some unused protocols 2016-07-20 00:28:32 +02:00
Luca Deri a93c83fe6b Added check on payload len (#227) 2016-07-12 16:05:49 +02:00
Luca 4fe8d350a2 Added fix to avoid IPv6 formatting differences across platforms 2016-07-12 00:38:07 +02:00
Simone Mainardi 72e3930e08 Fixes nDPI protocol initialization for Git
[NDPI] ndpi_init_protocol_defaults(missing protoId=226) INTERNAL ERROR: not all protocols have been initialized

This is actually a merge of branch 'dev' into 1.8-stable.
2016-07-08 15:23:45 +02:00
Simone Mainardi 31140c8da7 Fixes test cases for QUIC.Google 2016-07-08 15:03:40 +02:00
Luca Deri 47bb46176b Fix for #225 2016-07-05 19:07:16 +02:00
Campus 0e49eb1d17 added drda protocol - fix quic output after commit 87717dd77f 2016-07-02 18:29:02 +02:00
Luca Deri 87717dd77f Added QUICK subprotocol support 2016-06-30 17:03:36 +02:00
Luca Deri 6fb81f146e Refreshed DNS implementation 2016-06-27 11:19:47 +02:00
Campus ff6b193821 minor fixes - deleted useless part on git and quic protos 2016-06-24 13:45:43 +02:00
Campus 96623c0e4f Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2016-06-24 13:19:56 +02:00
Campus 0f089bd49a added git protocol dissector and pcap for test 2016-06-24 13:19:14 +02:00
Luca Deri b318d191a5 Removed variables not used 2016-06-23 23:53:03 +02:00
Luca Deri d0665bb301 Recomputed results due to the previous commit 2016-06-19 21:29:19 +02:00
Luca Deri adbba69988 Removed false positives from CoAP protocol
Improved DNS detection
Added misisng default DropBox port
Fix for #154
Added sample pcap of Tor traffic for regression testing
2016-06-19 21:25:58 +02:00
Luca Deri 539ab241cd Reverted https://github.com/ntop/nDPI/commit/392a14241fa68512099ea6096a640275461af7d0 that prevented Tor dissector to work 2016-06-19 20:48:05 +02:00
Luca Deri f18b7c5e14 Compilation fix 2016-06-15 06:04:07 +02:00
Luca Deri 19324f2904 Enhanced string boundary check to avoid crashes with malformed packets 2016-06-15 06:12:41 +02:00
Luca Deri 6c83cd627f Enhanced string boundary check to avoid crashes with malformed packets 2016-06-14 20:25:23 +02:00
Luca Deri ed09d78ed9 Added 1e100.net to Google 2016-06-12 21:01:59 +02:00
Luca Deri 0b7a392b30 Restored code harvested in
https://github.com/ntop/nDPI/commit/60e7e19dab72a37d57bc7854e702264e7d25886d#diff-0856ce19e6ed4e80946656d41f7edd65L135

DNS reply dissection can now be controlled by a preference as demonstrated in ndpiReader.c
2016-06-07 17:14:16 +02:00
Luca Deri 2379443ef5 Fix on multicast address check on teredo 2016-05-31 12:58:44 +02:00
Luca Deri 8c50b9692e Improved bigram detection (used by Tor) 2016-05-31 12:47:51 +02:00
Luca Deri 787be6ec08 Improved teredo detection by discarding packets sent towards multicast addresses 2016-05-31 12:32:47 +02:00
Luca Deri 72efa329db Enhanced fragments support in ndpiReader
Improved RX protocol dissection and removed port guess as it caused false positives.
2016-05-31 12:20:25 +02:00
Luca Deri a79b338208 Compilation fix due t pack/unpack 2016-05-30 09:09:00 +02:00
Luca Deri 08b2899676 Merge pull request #210 from mscherer/patch-1
Fix typo
2016-05-29 10:10:22 +02:00
Michael Scherer 074e489fe3 Fix typo 2016-05-29 10:09:07 +02:00
Luca Deri f8badefa90 Update homebrew 1.8 formula 2016-05-28 11:00:32 +02:00
Luca Deri d113f8db5a Moved to 1.8 release 2016-05-28 09:24:57 +02:00
Luca Deri db36040fe1 Merge pull request #207 from alagoutte/scan-build
ndpi_main: fix Value stored to 'saddr'/'daddr' is never read
2016-05-28 09:06:34 +02:00
Alexis La Goutte f0c1688b1b ndpi_main: fix Value stored to 'saddr'/'daddr' is never read
ndpi_main.c:3286:7: warning: Value stored to 'saddr' is never read
ndpi_main.c:3290:7: warning: Value stored to 'saddr' is never read
ndpi_main.c:3291:7: warning: Value stored to 'daddr' is never read
2016-05-26 19:52:25 +02:00
Campus 67d3c9e9c8 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2016-05-24 23:33:13 +02:00
Luca Deri ac664940da Merge pull request #206 from theirix/l4-api
Updated ndpi_l4_detection_process_packet
2016-05-24 23:32:53 +02:00
Luca Deri 8eb0829267 Merge pull request #205 from theirix/dns-overflow
Fixed dns buffer overflow
2016-05-24 23:31:54 +02:00
Campus f8ecdc004f added new fb detection - remove old bittorrent functions - fix rx warning 2016-05-24 23:28:43 +02:00
theirix c088672632 Synchronized ndpi_l4_detection_process_packet impl 2016-05-24 23:39:10 +03:00
theirix 552076a36c Fixed dns buffer overflow 2016-05-24 23:34:15 +03:00
Michele Campus a0fe6665b2 Merge pull request #203 from alagoutte/scan-build
fix warning found by Clang Analyzer (scan-build)
2016-05-24 22:04:33 +02:00
Michele Campus b05907f592 Merge pull request #204 from alagoutte/fix_warning
fix warning when use -Wdocumentation CFLAG (clang)
2016-05-24 16:54:13 +02:00
Alexis La Goutte bae6a09a7c ndpi_main(.c): fix Null pointer passed as an argument to a 'nonnull' parameter 2016-05-24 13:46:17 +02:00
Alexis La Goutte 8bc1ed7d86 ndpi_api(.h): fix '@return' command used in a comment that is attached to a function returning void [-Wdocumentation] 2016-05-24 13:19:53 +02:00
Alexis La Goutte 9935cb0e3b ndpi_util.c: fix The right operand of '+' is a garbage value 2016-05-24 08:12:36 +02:00
Alexis La Goutte 594f7074f7 ndpi_util: Value stored to 'fcs' is never read 2016-05-23 23:04:50 +02:00
Alexis La Goutte a6c9d33c09 ndpi_util: replace if / else by switch/case
Fix also
The left operand of '==' is a garbage value
Value stored to 'slarp' is never read
Value stored to 'cdp' is never read
2016-05-23 23:03:26 +02:00
Alexis La Goutte e0e1ed6936 rtcp: fix Value stored to 'sport'/'dport' is never read 2016-05-23 22:32:39 +02:00
Alexis La Goutte 7fb4b42092 bittorrent: fix Value stored to 'a' is never read 2016-05-23 22:30:47 +02:00
Alexis La Goutte 1b3055faef ndpi_main: fix Value stored to 'saddr'/'daddr' is never read 2016-05-23 22:21:50 +02:00
Alexis La Goutte cf905d0c7f FTP: fix Value stored to 'packet' during its initialization is never read 2016-05-23 22:19:05 +02:00
Luca Deri f084bbb6d0 Merge pull request #201 from alagoutte/scan-build
Fix missing break
2016-05-23 21:39:42 +02:00
Luca Deri a2843084d6 Merge pull request #200 from alagoutte/typo
Fix some typo
2016-05-23 21:38:36 +02:00
Alexis La Goutte 3d23ee0e23 RX: fix indent 2016-05-23 19:56:45 +02:00
Alexis La Goutte 92718c4def QUIC: fix typo 2016-05-23 19:56:45 +02:00
Alexis La Goutte e1d3a8ea7d QUIC: remove trailing whitespace 2016-05-23 19:56:45 +02:00
Alexis La Goutte 2f7a9bb789 Fix missing break 2016-05-23 08:01:51 +02:00
Luca Deri ef2cb860a3 Recodede QUIC from scratch to make it more robust and export SNI (server name) as with SSL/HTTP 2016-05-21 15:34:10 +02:00
Campus 1e37916156 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2016-05-17 23:44:05 +02:00
Campus f3fb445166 upgrade quic test pcap with version 33 2016-05-17 23:43:05 +02:00
Michele Campus ac55d8dd8b Merge pull request #193 from backer-and/dev
Fix QUIC
2016-05-17 23:40:01 +02:00
Andrea Buscarinu f517a6a373 Fix QUIC 2016-05-17 23:15:38 +02:00
Campus ccdcf9d82b added OpenDNS service 2016-05-17 18:28:54 +02:00
Campus b7863e2dc2 added Weibo service and pcap test 2016-05-13 01:05:13 +02:00
Campus 6edb280cc7 update test pcap due to modification of quic protocol - improved readability of code 2016-05-12 23:17:41 +02:00
Michele Campus 32847fcfd0 Merge pull request #190 from backer-and/dev
enhance/refactor quic.c
2016-05-12 22:53:41 +02:00
Luca 69bdc44441 Fix for invalid DNS reponse detection 2016-05-12 13:10:23 +02:00
Andrea Buscarinu 4d58e55781 enhance/refactor quic.c 2016-05-11 12:42:30 +02:00
Luca Deri 90843a4228 Merge pull request #188 from alagoutte/QUIC
Enhance QUIC support
2016-05-10 23:50:02 +02:00
Alexis La Goutte d233459195 QUIC: Remove a wrong heuritics
There is always the QUIC version on first frame
2016-05-10 22:21:50 +02:00
Alexis La Goutte 6dab2b8c62 QUIC: Fix trailing whitespace 2016-05-09 19:12:09 +02:00
Michele Campus 86893a2984 Merge pull request #187 from madpilot78/add_needed_symbols
Add symbols needed by ntopng to the shared library export list.
2016-05-09 13:22:23 +02:00
Guido Falsi 3618f066bb Add symbols needed by ntopng to the shared library export list. 2016-05-09 12:36:30 +02:00
Luca Deri 5c149e0363 Simplified ndpi_exit_detection_module() 2016-05-08 11:04:20 +02:00
Luca Deri 22e03ee98a Simplified nDPI initialization function 2016-05-08 10:55:46 +02:00
Luca Deri 18901ca4d6 Added wrappers for substring-search
ndpi_init_automa()
ndpi_free_automa()
ndpi_add_string_to_automa()
ndpi_finalize_automa()
ndpi_match_string()
set_ndpi_malloc()
set_ndpi_free()
2016-05-07 18:45:18 +02:00
Luca Deri d5f77e92f7 Fixed the travis link from master -> dev 2016-05-03 08:59:52 +02:00
Luca Deri d139c31972 Merge pull request #185 from alagoutte/QUIC
Fix test suite
2016-05-03 08:55:47 +02:00
Alexis La Goutte 53550d8bb4 Fix test suite
there is some change with IPv6 Address...
2016-05-03 08:00:57 +02:00
Alexis La Goutte 356f9914cc QUIC: fix test suite after typo change 2016-05-03 07:52:39 +02:00
Luca Deri 6e57c4618e Merge pull request #184 from alagoutte/QUIC
Enhance QUIC support
2016-04-29 11:34:32 +02:00
Alexis La Goutte 9b0bef50de QUIC: fix typo the protocol name is QUIC (and not Quic) 2016-04-29 09:25:28 +02:00
Alexis La Goutte c957a12742 QUIC: last version of QUIC is Q033
Look Recent Changes by Version from QUIC Wire Layout Specification
2016-04-29 09:21:42 +02:00
Campus 1b9aa2ab71 fixed rx protocol for better logic conditions 2016-04-26 18:49:52 +02:00
Campus 1d17ece9ce fix missing conditional and 2016-04-26 15:02:32 +02:00
Campus 34ce35dcfa minor fix makefile 2016-04-26 14:20:23 +02:00
Campus f66cfca2a9 fixed and improved rx protocol detection - there was error on detection when -d option was enabled 2016-04-26 13:48:43 +02:00
Campus 7c1e8ba2bf rework some stuffs - added missing comments on functions 2016-04-24 23:43:22 +02:00
Luca Deri e23446389c Reworked https://github.com/ntop/nDPI/commit/8a4c15ecf5c5c1430e8047324a5e336db2f503f1 to avoid adding (un-needed) pcap dependency in nDPI core 2016-04-24 23:17:18 +02:00
Luca Deri 8cb6172e4b Added #include <sys/types.h>
https://github.com/voidlinux/void-packages/blob/master/srcpkgs/ndpi/patches/types.patch
2016-04-24 22:59:05 +02:00
Luca Deri d427647ce7 Firxed warnings 2016-04-24 10:35:54 +02:00
Luca Deri 8a4c15ecf5 Merge pull request #180 from emanuele-f/dev
High level API and ndpiReader simplification
2016-04-24 10:29:26 +02:00
Luca Deri 1be9194734 Code hardening for #181 2016-04-21 12:04:33 +02:00
emanuele-f 792d2fc79a Remove floating stack pointer causing segmentation fault 2016-04-20 16:20:50 +02:00
emanuele-f 5386aede36 Integrate root flows free and little changes 2016-04-20 15:36:44 +02:00
emanuele-f 4e1317cd0f Fix inline travis complaining 2016-04-20 02:23:37 +02:00
emanuele-f 9ee6442004 Other util API stuff 2016-04-20 01:32:01 +02:00
emanuele-f 6f84b85358 Use calloc to force 0 initialization 2016-04-20 00:03:21 +02:00
emanuele-f d7548e72b2 Fix some issues 2016-04-19 23:35:35 +02:00
emanuele-f 7964e11c7f Add header stuff 2016-04-19 22:39:19 +02:00
emanuele-f a4d0af1f96 Move relevant functions to ndpi_util API module 2016-04-19 22:35:18 +02:00
emanuele-f 282bc16243 add malloc wrapper TODO 2016-04-19 22:19:06 +02:00
emanuele-f 4ba106a525 Change confusing "ndpi_flow" structure name to "ndpi_flow_info" 2016-04-19 19:25:42 +02:00
Michele Campus 1d54092c7d Merge pull request #177 from giomasce/dev
Support RX protocol (used by AFS).
2016-04-16 18:03:26 +02:00
Giovanni Mascellani 4b50045ecd Add file license. 2016-04-15 16:41:23 +02:00
Giovanni Mascellani a6e715f574 Add RX testcase. 2016-04-15 15:47:39 +02:00
Giovanni Mascellani 18f5362a34 Support RX protocol (used by AFS). 2016-04-15 15:21:02 +02:00
Michele Campus a71f525a2b Merge pull request #175 from theirix/reader-asan-2
Combined ASAN overflow detection and buffer modify
2016-04-13 11:06:32 +02:00
theirix efae85de84 Combined ASAN overflow detection and buffer modify
Combined approach from pull request #172 and commit 6a63ea4.
2016-04-13 11:42:59 +03:00
Michele Campus da811fbdd0 Merge pull request #173 from theirix/buffer-overflows
Fixed multiple buffer overflows
2016-04-12 23:41:02 +02:00
theirix d7a2515093 Fixed oscar buffer overflow 2016-04-12 22:26:10 +03:00
Luca Deri 885cc3864e Merge pull request #172 from theirix/reader-asan
Allocate exact-size buffer with address sanitizer
2016-04-12 21:22:38 +02:00
theirix c22d3d3cae Fixed more buffer overflows with small packets 2016-04-12 22:14:13 +03:00
theirix fb3fc0c6de Fixed buffer overflows with safe str search
1. Detected a lot of memory errors using address sanitizer
and ndpi-scapy tool.

2. Added ndpi_match_prefix function that compares strings
with taking care of payload packet len. Almost drop-in
replacement for match_first_bytes function.

3. Replaced unsafe match_first_bytes usage with a ndpi_match_prefix
and additional length checks.
2016-04-12 22:08:30 +03:00
theirix 624f61193c Allocate exact-size buffer with address sanitizer
Allows address sanitizer to detect buffer overflow by
using special packet buffer when compiled with address sanitizer
support (autodetected clang and gcc).
2016-04-12 22:04:47 +03:00
Luca Deri 5a37ee9976 Removed some Tor false positives (more work necessary) 2016-04-06 09:45:38 +02:00
Luca Deri 6a63ea46a4 Added check for alerting when nDPI modified packet payload (should not happen) 2016-04-04 21:40:23 +02:00
Campus f437fac412 fix for test pcap and output 2016-04-04 00:15:57 +02:00
Michele Campus afea67e303 Merge pull request #170 from srzamfir/dev
Mqtt dissector fix
2016-04-04 00:12:13 +02:00
Campus 3d46f3df61 fixed wrong dropbox detection 2016-04-04 00:10:43 +02:00
Michele Campus 697617622c Merge pull request #169 from theirix/build-json-c
Improved autogen.sh, json-c search
2016-04-03 23:58:51 +02:00
Sorin Zamfir 7e9e3903d8 Included test result for CoAP and Mqtt. 2016-04-03 15:05:38 +03:00
Sorin Zamfir 7b66fcff37 Merge remote-tracking branch 'upstream/dev' into dev 2016-04-03 14:45:39 +03:00
Sorin Zamfir 40b219c397 Fixed false positives on skype and KakaoTalk. Tests run smooth now. 2016-04-03 14:16:18 +03:00
Sorin Zamfir b5e74ee3f1 Solved invalid detection on tests. Needs further work to detect one of the flows.
Please enter the commit message for your changes. Lines starting
2016-04-03 02:00:36 +03:00
theirix 6f95182751 Improved detection of json-c package.
- Improved detection code for json-c package to use pkg-config
  with custom search path that includes /usr/local
- Added optional option --disable-json-c to disable json-c package for
  reducing runtime dependencies.
2016-04-01 17:18:33 +03:00
theirix 41d7430442 Call configure with args from autogen 2016-04-01 17:18:23 +03:00
Campus aa86387ba9 improved Microsoft detection (minor fix) 2016-04-01 02:17:58 +02:00
Campus 2a1f44610f improved Microsoft detection 2016-04-01 02:11:51 +02:00
Campus 4704f48daa minox fix dns 2016-03-29 15:07:16 +02:00
Campus 8dacc91d51 fixed dns check host name server 2016-03-29 15:04:15 +02:00
Campus a594273989 fixed coap.c and dropbox.c after detecting wrong detection - added test for dropbox 2016-03-29 00:05:56 +02:00
Campus d9264e415c fixed and improved bittorrent detection for issue https://github.com/ntop/nDPI/issues/164 2016-03-27 01:39:38 +01:00
Luca Deri b3df389bd6 Removed WinMX reference from Makefile 2016-03-23 08:46:24 +01:00
Campus 2998b78797 deleted winmx protocol due to abandoned development since July 2, 2004 2016-03-23 01:27:36 +01:00
Campus 46a526ce26 deleted mqtt.c 2016-03-23 00:35:22 +01:00
Campus 95fffec172 fix for pull request https://github.com/ntop/nDPI/pull/161 2016-03-20 16:10:26 +01:00
Michele Campus 323912371e Merge pull request #161 from srzamfir/dev
Add CoAP and Mqtt (IoT specific protocols) dissectors
2016-03-20 15:51:11 +01:00
Sorin Zamfir 3213d2205a Removed unused type definitions.
Disabled tracing in the example application.
2016-03-20 13:38:29 +02:00
Sorin Zamfir 3dde80ed0f Included test file for coap and mqtt dissectors. 2016-03-20 12:12:42 +02:00
Sorin Zamfir 6f428c03e2 Merge remote-tracking branch 'remotes/iot/dev' into dev
Conflicts:
	src/include/ndpi_protocol_ids.h
	src/lib/ndpi_main.c
2016-03-20 11:53:27 +02:00
Sorin Zamfir 2d5d39db7b Updated with mqtt callbacks. 2016-03-20 11:22:04 +02:00
Sorin Zamfir 28806f3187 Minor changes to a comment 2016-03-19 20:50:38 +02:00
Luca Deri 5349d33abf Fixed unitialized variable 2016-03-19 14:10:26 +01:00
Campus a499da00ba added tcp port 51413 for TCP BT detection 2016-03-19 01:18:48 +01:00
Luca Deri a4d9c8cc3b Fix for https://github.com/ntop/nDPI/issues/159 2016-03-18 11:00:11 +01:00
Luca Deri 27d09c33ce Fix for invalid DNS dissection that caused packet corruption 2016-03-17 08:28:24 +01:00
Campus 5b86186e85 deleted old and not yet good skype ip addr - added new Netflix and CloudFlare IP addr 2016-03-16 10:47:35 +01:00
Campus 3b399f62f4 improved Skype detection - redized Viber pcap for test 2016-03-15 15:03:06 +01:00
Campus 7c511fb2f2 add new detection of Viber (service) 2016-03-15 00:56:09 +01:00
Campus 4370781b49 rework API (part 1) - minor fixes functions 2016-03-14 01:04:50 +01:00
Luca Deri 1cf4926b4e Enhanced Google detection 2016-03-13 21:30:21 +01:00
Campus 7ae06a114d added check_punycode_string for issue https://github.com/ntop/nDPI/issues/157 2016-03-10 11:44:24 +01:00
Campus 8321b2fa66 added support for MS_OneDrive service 2016-03-08 01:49:48 +01:00
Luca Deri ba966634f5 Source code rename to avoid conflicts on Windows 2016-03-06 18:52:43 +01:00
Luca Deri dd12757493 Added check to harden code 2016-03-05 08:22:04 +01:00
Luca Deri 7ea37edfbf Improved protocol detection 2016-03-04 22:13:37 +01:00
Luca Deri 462368f959 Added new ndpi_detection_giveup() API call to call before giving up for a given flow
Code cleanup
2016-03-04 19:15:13 +01:00
Michele Campus 73833e0ef0 Merge pull request #153 from Timothyyy/dev
protocols: sip CANCEL packet detection fix
2016-03-02 11:54:05 +01:00
Timothy 7d699ade0c protocols: sip CANCEL packet detection fix 2016-03-02 11:04:36 +03:00
Luca bc52b8f310 Changed label case 2016-02-29 23:22:34 +01:00
Luca ecb8d117ac Reworked sockets support
Fixed TFTP support
2016-02-29 17:23:05 +01:00
Luca Deri ead8c4933e Removed VEOHTV protocol and replaced with HTTPDownload
Fixed bug that prevented content type to be properly detected
2016-02-28 23:19:13 +01:00
Luca Deri 20374b542d Fixed warning
Reworked protocol initialization
2016-02-27 09:00:02 +01:00
Campus 6df31099c2 typo fixed 2016-02-26 20:48:14 +01:00
Luca Deri b7e2d8f019 Minor BT optimization 2016-02-25 10:25:03 +01:00
Luca Deri 80a35226c5 Improve BT-uTP protocol dissection 2016-02-25 09:58:23 +01:00
Luca Deri e025d696f6 Added trace for BitTorrrent u-TP 2016-02-25 08:31:59 +01:00
Sorin Zamfir 245ca20eea Fixed the bug. Damn comparison. 2016-02-20 00:15:09 +02:00
Sorin Zamfir 8f60ad8874 We have a bug :(. 2016-02-19 21:18:28 +02:00
Sorin Zamfir d18a1edb67 Functional mqtt 2016-02-19 20:22:54 +02:00
Sorin Zamfir a8050635fa Improved Mqtt detection. 3 messages remain for identification 2016-02-19 19:55:03 +02:00
Campus 33fa1989bc improved netflix and amazon cloud detection 2016-02-19 02:00:23 +01:00
Sorin Zamfir 7d3ab6d0b4 Some filtering applied. Not yet functional. 2016-02-18 21:36:06 +02:00
Sorin Zamfir 237102b5f2 First draft of the Mqtt dissector. 2016-02-18 20:29:41 +02:00
Campus 407b4fd1d5 improved facebook detection adding host name match 2016-02-18 00:43:43 +01:00
Campus 882cbd8b4e Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2016-02-18 00:30:02 +01:00
Campus 4c3d9788dd added missing free for idle flow in ndpiReader.c 2016-02-18 00:29:06 +01:00
Luca Deri 59c65118e1 Updated readme 2016-02-17 10:58:52 +01:00
Campus 84976ed5dc fix for https://github.com/ntop/nDPI/issues/148 2016-02-16 12:23:56 +01:00
Luca Deri 15ccb70ae6 Improved WhatsAppVoice detection 2016-02-15 22:16:39 +01:00
Luca Deri e1481f3a90 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2016-02-15 12:00:56 +01:00
Luca Deri 01413c8d9f Disabled list of skype hosts: it looks outdated 2016-02-15 11:55:21 +01:00
Campus 392a14241f fixed ssl/tor bug for https://github.com/ntop/nDPI/issues/147 2016-02-15 11:02:40 +01:00
Luca Deri 16a5c6995c Added ability to extract BitTorrent hash (and eventually peerId) 2016-02-15 09:57:26 +01:00
Campus c7334e7c56 added cloudflare service - minor fixes for test pcap 2016-02-14 19:38:30 +01:00
Luca Deri cc4ceb88a0 Fix for #143 2016-02-12 22:07:37 +01:00
Sorin Zamfir 7ee464709b Functional dissector. Guestimative. 2016-02-11 23:55:21 +02:00
Sorin Zamfir ee5e7449ac First running example. No actual low-level detection.
* logging is now running
* included example capture
* included example log
* skeleton for coap detection
2016-02-08 23:01:22 +02:00
Sorin Zamfir 278a067d43 First skeleton for Coap dissector. 2016-02-07 19:29:30 +02:00
Luca e9f7df081f Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2016-01-21 16:33:20 +01:00
Luca d0b83a7196 Minor change to ndpi_protocol2name() 2016-01-21 13:52:52 +01:00
Luca bc48b770de Added ndpi_automa_match_string_subprotocol call
Fixed DNS dissector decoding bug
2016-01-21 13:25:55 +01:00
Campus e7cb3d324a fixed wrong dns detection for host name server that invalidated test pcaps 2016-01-12 17:21:04 +01:00
Luca Deri 0f3222e489 Fixed compilation errors due to previous commit 2016-01-06 18:11:57 +01:00
Luca Deri a08a0077e6 Hardened DNS dissector 2016-01-06 17:38:41 +01:00
Campus 5336aecde7 added ip range feature for file loaded with -p option to support custom protocols specified 2015-12-31 17:07:22 +01:00
Michele Campus a2c51a586e Merge pull request #132 from pavel-odintsov/dev
Made ndpi memory allocator type compatible with C++ on x686 platforms
2015-12-31 10:27:54 +01:00
Campus 3aa506eb1b added Office 365 URLs and IP address ranges (part 1) 2015-12-29 19:41:15 +01:00
Campus 60e7e19dab wrote new dns dissector - deleted useless part of dns to slim data structures used in nDPI and ntopng 2015-12-29 18:17:52 +01:00
Pavel Odintsov 1b2702df24 Made ndpi memory allocator type compatible with C++ on x686 platforms. Switch from unsigned long to size_t as allocation functions parameter. 2015-12-28 14:08:57 +03:00
Campus 6d37ee2970 added Google server address range 2015-12-24 00:26:31 +01:00
Campus 2aa007fbad Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2015-12-24 00:17:26 +01:00
Campus bfded90875 added OCS service and related pcap for testing 2015-12-24 00:16:33 +01:00
Luca e0c6d80c09 Fix for #131 2015-12-18 11:53:01 +01:00
Campus 237de95285 fixed makefile include header ndpi_includes.h 2015-12-13 17:21:47 +01:00
Luca Deri 357d8c98a3 Merge pull request #128 from lmangani/dev
minor fix for tests after d2c355bdb0
2015-12-10 10:24:25 +01:00
Lorenzo Mangani 5f40394501 minor fix for tests 2015-12-10 00:07:37 +01:00
Luca 41c6d2d031 Improvement for Dropbox detection
Fix for #124
2015-12-08 00:32:30 +08:00
Luca d2c355bdb0 Enhanced QQ support 2015-12-08 00:19:13 +08:00
Luca Deri 6f3d5a7b33 Win fixes 2015-11-30 14:38:16 +01:00
Luca Deri 34491f4be1 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2015-11-30 14:22:01 +01:00
Luca Deri 34e0a74c07 Win fixes 2015-11-30 14:21:50 +01:00
Campus 587f156557 reorganized header files - added ndpi_includes.h 2015-11-30 12:37:15 +01:00
Luca Deri f04c3c6acb Minor fix 2015-11-27 00:29:19 +01:00
Luca Deri 80171b7c54 Removed warning 2015-11-26 23:53:21 +01:00
Luca Deri cb68d4d00a Win32 compilation fixes 2015-11-26 23:47:55 +01:00
Luca Deri 98ef945d2e fixes for packing datastructures 2015-11-26 19:46:51 +01:00
Campus 12b47fece0 fixed missed library netinet/in.h under FreeBSD 2015-11-26 12:32:15 +01:00
Luca Deri bfb44934b7 Fixed IPv6 HTTPs support 2015-11-23 10:08:44 +01:00
Luca Deri 08ca34f5d0 Fix for #119 2015-11-23 10:07:19 +01:00
Luca Deri a46c12c014 Fixed string offset 2015-11-22 23:28:50 +01:00
Luca Deri d24093c4a6 Improvements on netbios name detection 2015-11-22 23:13:50 +01:00
Campus ec034da5c9 fixed potential buff overflow with sizeof control 2015-11-19 20:18:37 +01:00
Campus d5b9e4ccd5 fixed wrong output 6in4tunnel after updates in nDPI 2015-11-19 20:10:46 +01:00
Luca Deri 169c837613 Fix for https://github.com/ntop/nProbe/issues/7 for wrong RTP identification 2015-11-18 10:46:58 +01:00
Luca 33f38dd437 Added ndpi_l4_detection_process_packet function 2015-11-17 14:16:09 +01:00
Simone Mainardi bd7004ecd7 Fixes #116, adds extra #ifndef to prevent redefinitions
Extra #ifndef directives are added to prevent possible redefinitions
of __LITTLE_ENDIAN__ and __BIG_ENDIAN__
2015-11-15 22:44:06 +01:00
lucaderi 8babee0efb Fix for CentOS6 2015-11-12 16:25:12 +01:00
Luca a407da79a9 Update ndpi_ethernet definition 2015-11-12 15:32:44 +01:00
Luca 8399afa607 Removed legacy linux_compat.h file
Updated IPv6 address dump format
2015-11-12 15:23:00 +01:00
Luca 68091c7cdb Added missing prototype 2015-11-12 15:11:28 +01:00
Luca e62be406ce Added missing ndpi_detection_get_sizeof_ndpi_flow_struct(), ndpi_detection_get_sizeof_ndpi_id_struct() functions 2015-11-12 15:09:49 +01:00
Michele Campus dd3a79dac2 Merge pull request #114 from kYroL01/dev
Refactoring nDPI
2015-11-12 14:57:02 +01:00
Campus a4f170604c removed kernel support and #if 0s 2015-11-12 12:08:41 +01:00
Campus 349e0b6fd2 other changes related to refactoring, including Makefile.am modifications 2015-11-12 10:43:32 +01:00
Campus 78eceb43a1 deleted useless or not used functions and #ifdef 2015-11-10 01:12:00 +01:00
Campus 0cd8d939f1 deleted useless header 2015-11-09 14:36:49 +01:00
Campus 9fa2c7404f fixed mdns and tcp/udp protocols after changes 2015-11-09 13:44:23 +01:00
Campus 8cd0a8838a fixing ndpiReader.c after previous changes 2015-11-09 13:39:33 +01:00
Campus d92ffa6771 initial refactoring nDPI - typedef defined only in ndpi_typedef.h 2015-11-09 13:36:35 +01:00
Michele Campus 63cab68a96 Merge pull request #111 from xumjbn/dev
fix dissector QQ unkown bug
2015-10-27 10:33:05 +01:00
xumjbn f6eb2724e2 fix dissector QQ unkown bug 2015-10-28 15:07:26 +08:00
Luca Deri fc6980e0c9 Fix for #86 2015-10-26 13:43:38 +01:00
Luca Deri 7a95a21e09 Fix for #110 2015-10-26 13:21:39 +01:00
Luca Deri 8edd628973 Minor core reorganization 2015-10-26 01:34:39 +01:00
Luca Deri 50e7e959fe Updated test result 2015-10-25 23:26:03 +01:00
Luca Deri b5f70d02b2 Fixed IPv6-in-IPv4 implementation 2015-10-25 23:23:51 +01:00
Luca Deri 76c2fb3de7 Removed linux kernel support 2015-10-24 12:47:23 +02:00
Campus 27ea5c0026 added 6in4 tunneling pcap for test 2015-10-23 17:15:28 +02:00
Campus 59d0af33b8 added support for 6in4 tunneling in ndpiReader.c 2015-10-23 17:14:03 +02:00
Luca Deri 25631a25f7 Implemented MS Lync support 2015-10-22 00:32:48 +02:00
Luca ac9b4f8135 Removed non IP traffic to shrink pcaps 2015-10-20 16:20:59 +02:00
Campus 46694ece47 added new BGP pcap for detection of Cisco datalink layer 2015-10-19 01:58:03 +02:00
Campus 74b016c8b6 fixed BGP detection - added BGP pcap with Cisco hDLC and Cisco SLARP 2015-10-19 01:54:41 +02:00
Campus 7242b08609 deleted useless header 2015-10-16 19:28:53 +02:00
Campus ee0954c8cb Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2015-10-16 17:49:29 +02:00
Campus fd9f4753bb added cisco hdlc datalink type - fix MPLS header - added BGP pcap with cisco hdlc & MPLS header 2015-10-16 17:48:38 +02:00
Luca Deri dc9ad66fe2 Windows changes 2015-10-15 12:44:51 +02:00
Luca Deri 247acce949 Simplified IPv6 typedefs 2015-10-12 09:15:02 +02:00
Luca Deri 91b2e0bff2 Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2015-10-12 01:17:19 +02:00
Luca Deri 17ad351386 Changed to make sure IPv6 defines are properly defined 2015-10-12 01:16:28 +02:00
Luca Deri 72eafff843 Improvements for NetFlix detection 2015-10-11 22:27:03 +02:00
Luca Deri 4ec03b7e87 Added TZSP encapsulation support 2015-10-11 18:35:51 +02:00
Luca Deri 88d466e58e Added fix for Webex protol detection 2015-10-11 17:55:56 +02:00
Luca Deri cf2c59b1d7 Fixed travis link 2015-10-11 17:44:41 +02:00
Luca 93f02c08dc Fixed invalid indexes used in SMTP dissector.
See https://github.com/ntop/nDPI/commit/8c4fada42f8e6dcba42281fdc129e0a32b6050dc#commitcomment-13685139
2015-10-09 15:03:10 +02:00
Michele Campus efac255821 Merge pull request #101 from Tomasu/dev
NDPI_PROTOCOL_KAKAOTALK_VOICE was used instead of NDPI_SERVICE_KAKAOT…
2015-10-08 13:27:57 +02:00
Thomas Fjellstrom 06515b43d8 NDPI_PROTOCOL_KAKAOTALK_VOICE was used instead of NDPI_SERVICE_KAKAOTALK_VOICE 2015-10-08 05:04:16 -06:00
Luca 0e436e66de Reworked copyright to give credit to the developer 2015-10-06 16:24:03 +02:00
Luca Deri 3f2eec75c7 Merge pull request #98 from Tomasu/dev
Add Ubiquity AirControl 2 protocol detector
2015-10-06 16:09:59 +02:00
Thomas Fjellstrom 6519a1e326 add ndpi_malloc, ndpi_calloc and ndpi_set_detected_protocol to list of exported symbols 2015-10-04 20:04:39 -06:00
Thomas Fjellstrom a2ff20990c remove traces of DHCP detector from ubntac2.c 2015-10-04 19:53:25 -06:00
Thomas Fjellstrom 0a6d43c5f6 Add Ubiquity AirControl 2 protocol detection 2015-10-04 19:50:47 -06:00
Thomas Fjellstrom 64a368dd8e add Ubiquity AirControl 2 protocol detection 2015-10-04 19:48:26 -06:00
Campus 1dc03d5741 added radio tap header identification for ieee802.11 packets 2015-10-04 14:00:47 +02:00
Luca Deri e5c2fee022 Travis logo changes 2015-10-03 08:34:18 +02:00
Lorenzo Mangani 00bfba1d77 Update README.md 2015-09-29 01:04:46 +02:00
Luca Deri 76c963a70c Added support for pfring_config 2015-09-23 15:53:01 +02:00
valentina f78f6b59cc Added some tags in order to recognize ebay 2015-09-23 00:53:25 +02:00
Luca Deri faf3c81be3 Merge pull request #96 from mgigante/dbg_mgigante_freebsd81
Remove duplicated function declarations
2015-09-22 16:11:14 +02:00
Marco Gigante a994722bb1 Remove duplicated function declarations 2015-09-22 15:25:09 +02:00
valentina b383475282 Fixed DCE_RPC protocol when the port is different to default port (135) 2015-09-20 23:22:48 +02:00
Luca Deri ce7866a6a6 Added missing symbols
ndpi_get_lower_proto
ndpi_is_proto
2015-09-04 09:00:00 +02:00
Lorenzo Mangani 0dfd6d3986 Merge pull request #92 from lmangani/dev
Added HEP protocol detection support (sipcapture)
2015-09-01 18:11:52 +02:00
Lorenzo Mangani 50757e7afe Remove HEP2 match 2015-09-01 10:02:27 +02:00
Lorenzo Mangani df64a1069e Added HEP protocol detection support (sipcapture) 2015-08-30 23:07:31 +02:00
Luca Deri 9450226675 Merge pull request #91 from rs-dev/steam_NDPI_PARSE_PACKET_LINE_INFO
Make sure packet->user_agent_line.ptr is up to date before using it.
2015-08-30 22:42:33 +02:00
Martin Natano 8f8052735c Make sure packet->user_agent_line.ptr is up to date before using it.
Otherwise we might dereference a stale pointer.
2015-08-28 14:49:51 +02:00
Luca Deri aa6640e1c0 Merge pull request #84 from bcronje/dev
Ensure usage of struct in_addr is in network byte order.
2015-08-27 08:45:05 +02:00
Beyers Cronje 5f1b82d696 Ensure usage of struct in_addr is in network byte order.
Closes issue #81
2015-08-27 01:18:32 +02:00
Luca Deri 19e96f7dd2 Fixes #83. Critical fix: due to an invalid endianess conversion some protocol were not properly indetified 2015-08-26 16:09:24 +02:00
Luca Deri 591d6df489 Merge pull request #80 from bcronje/dev
Do not write to packet.iph, which should be readonly.
2015-08-26 01:18:45 +02:00
Beyers Cronje 8fe318495f Do not write to packet.iph, which should be readonly.
Resolves issue #77
2015-08-26 00:49:01 +02:00
Luca Deri 569c055389 Fir for #79 2015-08-26 00:29:18 +02:00
Luca Deri 26ce076f61 Merge pull request #76 from alagoutte/wdoc
Fix warning when use -Wdocumentation (Clang CFLAG)
2015-08-18 15:20:24 +02:00
Alexis La Goutte 07c7d63d44 Fix warning when use -Wdocumentation
src/include/ndpi_api.h:73:13: error: parameter 'ndpi_malloc' not found in the function declaration [-Werror,-Wdocumentation]
src/include/ndpi_api.h:91:13: error: parameter 'cache' not found in the function declaration [-Werror,-Wdocumentation]
src/include/ndpi_api.h:92:13: error: parameter 'cache' not found in the function declaration [-Werror,-Wdocumentation]
ndpiReader.c:152:22: error: not a Doxygen trailing comment [-Werror,-Wdocumentation]
ndpiReader.c:153:36: error: not a Doxygen trailing comment [-Werror,-Wdocumentation]
ndpiReader.c:156:39: error: not a Doxygen trailing comment [-Werror,-Wdocumentation]
2015-08-18 14:44:05 +02:00
Luca Deri 0a4c8e2959 Moved to 1.7.1 development branch 2015-08-18 11:58:57 +02:00
Luca Deri 5352c34ba5 Updated homebrew formula for 1.7 release 2015-08-18 11:58:23 +02:00
Luca Deri 41fcb7b1c1 Warning fix 2015-08-18 11:02:29 +02:00
Luca Deri a0bef995b6 Initial work for supporting HotspotShield VPN detection 2015-08-13 12:44:47 +02:00
Luca Deri d6b5d8e2c2 Added teredo protocol support. Fixed #74 2015-08-07 15:56:03 +02:00
Luca Deri 449742e69d Fixed a bug in the STUN dissector courtesy of Asher Yermiyahu
Fixed a bug in the NetFlow dissector that caused flows to be wrongly decoded as Netflow
Updated test results for starcraft and whatsapp
2015-08-03 14:35:31 +02:00
Luca Deri bc786b48fa Merge pull request #73 from c4stan/dev
battle.net removed
2015-07-30 14:33:52 +02:00
m.bracci 506b4e5f81 battle.net removed
battle.net removed, fixed a typo
2015-07-30 14:04:25 +02:00
Luca Deri 8365b127f2 Merge pull request #72 from pavel-odintsov/dev
Enhance NTP support, add protocol version identification; Add pcap ex…
2015-07-29 16:18:19 +02:00
Pavel Odintsov 471ea81594 Enhance NTP support, add protocol version identification; Add pcap examples for NTPv2, NTPv3, NTPv4; Fix bug with identification of NTP monlist packets as QUIC 2015-07-29 14:19:32 +03:00
Luca Deri b0ac39fe28 Various Instagram enhancements 2015-07-27 23:25:47 +02:00
Luca Deri aced1e9673 Improvements to WhatsApp voice 2015-07-25 10:13:47 +02:00
Luca e37be3b43d Fix to limit STUN to UDP only 2015-07-22 16:51:39 +02:00
Luca 6648d19239 Minor starcraft/WoW code cleanup 2015-07-21 17:30:47 +02:00
Luca Deri 8d8a204519 Cleaned up starcraft protocol code
Fixed false-positive in Skype dissector
2015-07-21 14:10:50 +02:00
Luca Deri 2ab10ec9b7 Merge pull request #66 from c4stan/dev
Starcraft 2/Battle.net support
2015-07-21 13:25:46 +02:00
Matteo Bracci 0ce34f341e Update starcraft2.c
Removed debug leftovers
2015-07-20 12:55:14 +02:00
Michele Campus 0d68f8ab5d added free for flow in get_ndpi_flow - deleted dead part in ndpi_typedef.c 2015-07-19 01:21:37 +02:00
Matteo Bracci 3ea5f2f0a6 Update ndpi_protocol_ids.h 2015-07-17 19:00:36 +02:00
Matteo Bracci 912c160ad3 Update ndpi_typedefs.h 2015-07-17 18:50:21 +02:00
Matteo Bracci d9745a2e2c Update ndpi_protocol_ids.h 2015-07-17 18:48:05 +02:00
Matteo Bracci deb96d83d3 Update ndpi_protocol_ids.h 2015-07-17 18:47:50 +02:00
Matteo Bracci e5f3d21f81 Update ndpi_protocol_ids.h 2015-07-17 18:47:31 +02:00
Matteo Bracci eb49f15453 Update ndpi_protocol_ids.h 2015-07-17 18:47:00 +02:00
Matteo Bracci 5165102f81 Update world_of_warcraft.c
Commented the first part of the dissector, as it steals traffic from other Battle.net based protocols.
2015-07-17 18:43:32 +02:00
Matteo Bracci d215d6d861 Update ndpi_protocols.h 2015-07-17 18:42:42 +02:00
Matteo Bracci 0739b577d4 Update starcraft2.c
starcraft2
2015-07-17 15:30:34 +02:00
Matteo Bracci c62e8dadb6 Update ndpi_main.c
starcraft2
2015-07-17 13:53:31 +02:00
Matteo Bracci e29bd2270f Update ndpi_content_match.c.inc
starcraft2
2015-07-17 13:44:50 +02:00
Matteo Bracci aa4bdebf40 Update starcraft2.c 2015-07-17 13:40:35 +02:00
Matteo Bracci 5f460c95cc Create starcraft2.c 2015-07-17 13:38:41 +02:00
Luca Deri 73358d5ec2 Added check for removing characters from nDPI custom-defined protocols that might cause apps to misbehave. 2015-07-16 21:01:30 +02:00
Luca Deri e98108a8be Updated guide courtesy of Sudeepta Bhuyan (#62) 2015-07-14 00:18:39 +02:00
Luca 5af906ad83 Enhanced FTP_DATA recognition (issue #28) 2015-07-13 16:41:51 +02:00
Luca 1d711d849d Added fix for #31 2015-07-13 16:16:58 +02:00
Luca 0b7af2dbbc Fixes for #27 2015-07-13 15:34:47 +02:00
Michele Campus 9d1e99ad0a added Oscar test 2015-07-13 15:01:51 +02:00
Luca Deri f948e76c45 Merge pull request #65 from kYroL01/dev
fixed Oscar family_type
2015-07-13 15:01:11 +02:00
Michele Campus e3e8b5c6e4 fixed Oscar protocol dissector (4) 2015-07-13 01:05:57 +02:00
Michele Campus e4f4cfb1de fixed Oscar protocol dissector (4) 2015-07-13 01:02:23 +02:00
Luca Deri 085d7f0ed4 Removed warnings
Added instagram to the list of tests
2015-07-13 00:05:04 +02:00
Luca Deri f3105f6c81 Merge pull request #64 from backer-and/dev
Added Instagram detection service
2015-07-12 23:54:59 +02:00
backer-and 5c55b970c6 Added Instagram pcap file 2015-07-12 23:38:55 +02:00
backer-and cad7e541b6 Added Instagram detection
Detected "instagram.c10r.facebook.com" host.  Omitted "*amazonaws.com",
"*akamai.com" and "*facebook.com" CDNs e.g. "a1408.dspw43.akamai.net"
2015-07-12 23:15:43 +02:00
Luca Deri 312c7ce1af Merge pull request #63 from kYroL01/dev
Fixed Oscar protocol dissector
2015-07-12 23:05:41 +02:00
backer-and c354ff85a6 Added Instagram as last implemented protocol 2015-07-12 22:49:24 +02:00
Luca fc12541e8a Added word version of the manual 2015-07-12 20:38:41 +02:00
Michele Campus e178653f48 fixed Oscar protocol dissector (3) 2015-07-12 18:30:08 +02:00
Michele Campus 05f2eb6270 fixed Oscar protocol dissector (2) 2015-07-12 18:03:44 +02:00
Michele Campus 0307c85bd3 fixed Oscar protocol dissector 2015-07-12 17:51:21 +02:00
Luca 71ed576e18 Fixed #46 (Deezer support) 2015-07-11 18:54:55 +02:00
Luca 2f347a33d8 Fixes for issues #40 -> #52
Implemented sub-protocol detection for SSL/HTTP/HTTPS
2015-07-11 16:04:58 +02:00
Luca Deri 436af982fd Merge pull request #60 from kYroL01/dev
fixed upper/lower name and init_non_tcp_udp_dissector
2015-07-09 00:29:03 -07:00
Michele Campus 1dd0c912c7 fixed upper/lower name and init_non_tcp_udp_dissector 2015-07-09 00:00:27 +02:00
Michele Campus 824c5f82d3 Merge remote-tracking branch 'upstream/dev' into dev
Conflicts:
	src/include/ndpi_protocols.h
	src/lib/ndpi_main.c
	src/lib/protocols/afp.c
	src/lib/protocols/aimini.c
	src/lib/protocols/applejuice.c
	src/lib/protocols/armagetron.c
	src/lib/protocols/ayiya.c
	src/lib/protocols/battlefield.c
	src/lib/protocols/bgp.c
	src/lib/protocols/bittorrent.c
	src/lib/protocols/ciscovpn.c
	src/lib/protocols/citrix.c
	src/lib/protocols/corba.c
	src/lib/protocols/crossfire.c
	src/lib/protocols/dcerpc.c
	src/lib/protocols/dhcp.c
	src/lib/protocols/dhcpv6.c
	src/lib/protocols/directconnect.c
	src/lib/protocols/directdownloadlink.c
	src/lib/protocols/dns.c
	src/lib/protocols/dofus.c
	src/lib/protocols/dropbox.c
	src/lib/protocols/eaq.c
	src/lib/protocols/edonkey.c
	src/lib/protocols/fasttrack.c
	src/lib/protocols/fiesta.c
	src/lib/protocols/filetopia.c
	src/lib/protocols/florensia.c
	src/lib/protocols/ftp_control.c
	src/lib/protocols/ftp_data.c
	src/lib/protocols/gnutella.c
	src/lib/protocols/gtp.c
	src/lib/protocols/guildwars.c
	src/lib/protocols/h323.c
	src/lib/protocols/halflife2_and_mods.c
	src/lib/protocols/http.c
	src/lib/protocols/http_activesync.c
	src/lib/protocols/iax.c
	src/lib/protocols/icecast.c
	src/lib/protocols/imesh.c
	src/lib/protocols/ipp.c
	src/lib/protocols/irc.c
	src/lib/protocols/jabber.c
	src/lib/protocols/kakaotalk_voice.c
	src/lib/protocols/kerberos.c
	src/lib/protocols/kontiki.c
	src/lib/protocols/ldap.c
	src/lib/protocols/lotus_notes.c
	src/lib/protocols/mail_imap.c
	src/lib/protocols/mail_pop.c
	src/lib/protocols/mail_smtp.c
	src/lib/protocols/maplestory.c
	src/lib/protocols/mdns.c
	src/lib/protocols/meebo.c
	src/lib/protocols/megaco.c
	src/lib/protocols/mgcp.c
	src/lib/protocols/mms.c
	src/lib/protocols/mpegts.c
	src/lib/protocols/msn.c
	src/lib/protocols/mssql.c
	src/lib/protocols/mysql.c
	src/lib/protocols/netbios.c
	src/lib/protocols/netflow.c
	src/lib/protocols/nfs.c
	src/lib/protocols/noe.c
	src/lib/protocols/ntp.c
	src/lib/protocols/openft.c
	src/lib/protocols/openvpn.c
	src/lib/protocols/oracle.c
	src/lib/protocols/oscar.c
	src/lib/protocols/pando.c
	src/lib/protocols/pcanywhere.c
	src/lib/protocols/postgres.c
	src/lib/protocols/pplive.c
	src/lib/protocols/ppstream.c
	src/lib/protocols/pptp.c
	src/lib/protocols/qq.c
	src/lib/protocols/quake.c
	src/lib/protocols/quic.c
	src/lib/protocols/radius.c
	src/lib/protocols/rdp.c
	src/lib/protocols/redis_net.c
	src/lib/protocols/rsync.c
	src/lib/protocols/rtcp.c
	src/lib/protocols/rtmp.c
	src/lib/protocols/rtp.c
	src/lib/protocols/rtsp.c
	src/lib/protocols/sflow.c
	src/lib/protocols/shoutcast.c
	src/lib/protocols/sip.c
	src/lib/protocols/skinny.c
	src/lib/protocols/skype.c
	src/lib/protocols/smb.c
	src/lib/protocols/snmp.c
	src/lib/protocols/socrates.c
	src/lib/protocols/sopcast.c
	src/lib/protocols/soulseek.c
	src/lib/protocols/spotify.c
	src/lib/protocols/ssdp.c
	src/lib/protocols/ssh.c
	src/lib/protocols/ssl.c
	src/lib/protocols/stealthnet.c
	src/lib/protocols/steam.c
	src/lib/protocols/stun.c
	src/lib/protocols/syslog.c
	src/lib/protocols/tds.c
	src/lib/protocols/teamspeak.c
	src/lib/protocols/teamviewer.c
	src/lib/protocols/telegram.c
	src/lib/protocols/telnet.c
	src/lib/protocols/tftp.c
	src/lib/protocols/thunder.c
	src/lib/protocols/tor.c
	src/lib/protocols/tvants.c
	src/lib/protocols/tvuplayer.c
	src/lib/protocols/twitter.c
	src/lib/protocols/usenet.c
	src/lib/protocols/veohtv.c
	src/lib/protocols/vhua.c
	src/lib/protocols/viber.c
	src/lib/protocols/vmware.c
	src/lib/protocols/vnc.c
	src/lib/protocols/warcraft3.c
	src/lib/protocols/whoisdas.c
	src/lib/protocols/winmx.c
	src/lib/protocols/world_of_kung_fu.c
	src/lib/protocols/world_of_warcraft.c
	src/lib/protocols/xbox.c
	src/lib/protocols/xdmcp.c
	src/lib/protocols/yahoo.c
	src/lib/protocols/zattoo.c
	src/lib/protocols/zeromq.c
2015-07-08 22:21:10 +02:00
Luca 5266c726f2 Reworked protocol initialization. Work in progress (more cleanup is needed) 2015-07-08 11:58:55 -07:00
Michele Campus a4b22a612e adding init protocol (2) - removed host_match proto defaults 2015-07-08 12:50:46 +02:00
Michele Campus e222345c85 Merge remote-tracking branch 'upstream/dev' into dev 2015-07-08 10:21:00 +02:00
Michele Campus 2b30cc8f2c adding init protocol (1) 2015-07-07 21:12:49 +02:00
Luca dc1d7bf47f Introduced ndpi_protocol datatype to handle master+subprotocol
Reworked protocol/master_protocol
Added new API call to print protocol/master_protocol
      - ndpi_protocol2name()
2015-07-07 10:50:49 -04:00
Luca f5d6cd511e Added support for Snapchat
Reworked code to better handle host-based protocols
2015-07-05 19:21:12 +02:00
Luca 09c137b27a Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2015-07-01 18:36:54 +02:00
Luca 151f74d596 Reworked skype code 2015-07-01 18:36:33 +02:00
Luca Deri 2ac44376cb Moved to version 1.7.0 2015-07-01 18:14:37 +02:00
Luca 7f807b81fe Added support for subprotocols
Modified ndpiReaded to print subprotocols
2015-07-01 18:12:47 +02:00
Luca 60884f9047 Split former protocol into upper and lower protocol 2015-07-01 17:40:14 +02:00
Luca 49ea23530f Code cleanup for improving readability 2015-07-01 16:57:32 +02:00
Luca 07ebed7fae Removed commented code
Renamed ndpi_int_add_connection() with ndpi_set_detected_protocol()
2015-07-01 16:45:29 +02:00
Luca 857020c475 Added missing type that defaulted to int 2015-07-01 16:40:43 +02:00
Luca Deri 68af5c221d Merge pull request #53 from kYroL01/dev
deleted protocol_type (real-correlated)
2015-07-01 16:32:34 +02:00
Michele Campus 4089227863 no_real for mpegts.c 2015-06-29 17:09:05 +02:00
Michele Campus a968718b52 no_real for mpegts.c 2015-06-29 17:06:04 +02:00
Michele Campus 83d46d02d4 Merge pull request #1 from kYroL01/no_real
No real
2015-06-29 16:56:54 +02:00
Michele Campus 32cdefb7d7 deleted protocol type (real-correlated)_1 2015-06-29 16:33:29 +02:00
Michele Campus 8da7af7b7b deleted protocol type (real-correlated) 2015-06-29 16:24:51 +02:00
Luca 276a35d36d Added test results for mpeg pcap 2015-06-25 03:59:17 -07:00
Luca 7e28cc0981 Added MPEG TS protocol
Fixed possible decoding loop in RTCP dissector
Added test pcap for mpegts
2015-06-25 03:57:50 -07:00
Luca Deri 40292a737a Merge pull request #38 from jnicholls/dns-fix
Account for the skipping of the DNS RR CLASS field by progressing the…
2015-06-24 17:42:00 -07:00
Luca Deri 8d5dda9d33 Merge pull request #36 from kYroL01/dev
fix quic.c
2015-06-24 17:40:21 -07:00
Jarred Nicholls 1d9d0b89ab Account for the skipping of the DNS RR CLASS field by progressing the payload index forward by an extra 2 octets. Remove commented code. 2015-06-24 20:38:01 -04:00
Michele Campus dd834c0eeb fix quic.c 2015-06-24 21:39:25 +02:00
Michele Campus 1d698899f6 fix quic.c 2015-06-24 19:36:35 +02:00
Luca 9bb99ed01d Added acknowldgement section 2015-06-24 08:03:53 -07:00
Luca ce94b7e545 Added support for TIM services generated by tim.com.br and tim.it 2015-06-24 07:57:09 -07:00
Luca 53c0b81dad Reverted fix in quic.c as apparently it invalidates protocol detection
Added testing files for quickplay
2015-06-24 07:49:02 -07:00
Luca 2be84d4d64 Added support for QuickPlay streaming service used by companies like hooq.tv 2015-06-24 07:34:57 -07:00
Luca 806c1a48d5 Added fix for fixing a crash when some CLI combination options are used 2015-06-23 07:06:09 -07:00
Luca Deri 4f691de0b5 Merge pull request #32 from kYroL01/dev
fix quic.c and add libtoolize option to autogen.sh
2015-06-20 18:51:27 +02:00
Michele Campus 37912b6045 fix quic.c and add libtoolize option to autogen.sh 2015-06-20 18:35:04 +02:00
Luca Deri d04979402c Added further check 2015-06-19 15:59:46 +02:00
Luca Deri 8a2493c26a Fixed crash caused by IPv6 traffic and KakaoTalk 2015-06-19 11:39:38 +02:00
Luca b2816d72da Disabled debug messages 2015-06-18 16:31:58 +02:00
Luca 9d361ec772 HTTP response is now dissected so we can match mime types on responses 2015-06-18 16:24:03 +02:00
Luca Deri 5b62fa3464 Merge pull request #29 from commsquare/dev
Allow Host IP based subprotocols to be added in protos.txt
2015-06-18 15:52:24 +02:00
Luca Deri 92e813cca4 Merge pull request #30 from sparkon/dev
Adding more Twitch.tv support
2015-06-18 15:51:59 +02:00
Edoardo Dominici a7b7254ac0 Added Twitch ingestion servers and JustinTV CDN
[ Fixed newline ]
2015-06-17 15:17:45 +02:00
sparkon 23048378d8 Added Twitch ingestion servers and JustinTV CDN 2015-06-17 14:55:07 +02:00
george.vakras@gmail.com b16ff0a3dc Fix ip based protocol example in protos.txt 2015-06-17 14:14:19 +02:00
Luca Deri f97fe9a7b5 Added pcap for SSL regression 2015-06-16 18:34:43 +02:00
Luca Deri 85db524a35 Added fix for detecting SSL certificate decoding. The old code did not discard non-SSL packets thus incrementing a counter when not necessary and thus terminating SSL decoding too early. This causes some SSL certificates not to be properly dissected and thus handled. 2015-06-16 18:31:01 +02:00
Luca 031f1709f2 Test file for mpeg detection 2015-06-15 16:48:50 +02:00
george.vakras@gmail.com 7391fe0c5e Allow loading host IP based protocols from protos.txt 2015-06-15 09:50:13 +02:00
Luca Deri 984eca8a27 Added support for Twitch detection
Minor cosmetic fix in test tool
2015-06-15 00:47:39 +02:00
Luca Deri 8b321fda41 Added testing files for EAQ, KakaoTalk, Torcedor, Meu 2015-06-14 12:28:59 +02:00
Luca Deri 702a9d0867 Improved RTCP decoder
Added detection of KakaoTalk Voice and Chat
2015-06-14 12:21:29 +02:00
Luca Deri 20740e1b68 Added Meu service detection 2015-06-14 11:04:31 +02:00
Luca Deri c4adefe75b Added EAQ protocol support 2015-06-14 10:37:34 +02:00
Luca Deri 11fb7e1796 Added missing L in license 2015-06-12 11:02:44 +02:00
Lorenzo Mangani 12f89fff49 Typo fix in README 2015-06-07 16:25:39 +02:00
Luca Deri f3bb7fb9c2 Added misisng symbols
ndpi_get_proto_by_id
ndpi_get_protocol_id_master_proto
ndpi_guess_protocol_id
2015-06-07 08:36:32 +02:00
Luca 72b205f5c1 Fixed bug that caused configure to generate GIT version information containing spaces
Modified ndpiReader to include flow details in dumps (used for testing)
Rebuilt all test results to include flow details
2015-06-04 19:07:23 +02:00
Luca 4ce9dec54e Reworked the STUN plugin
Added support for WhatsApp Voice vs WhatsApp (chat, no voice)
2015-06-04 18:44:09 +02:00
3386 changed files with 515723 additions and 35267 deletions
+6
View File
@@ -0,0 +1,6 @@
mpile
# Codige Code Review ignore
# Prevent Codiga complaining about Sphinx config.py
doc/conf.py
+6
View File
@@ -0,0 +1,6 @@
*.sh -crlf
*.ac -crlf
*.am -crlf
#Content files are C file
*.inc linguist-language=C
+50
View File
@@ -0,0 +1,50 @@
---
name: "🐞 Report a Bug"
about: Create a report to help us improve nDPI
title: ''
labels: 'bug'
assignees: ''
---
## Describe the bug
A clear and concise description of what the bug is.
### Expected behavior
A clear and concise description of what you expected to happen.
### Obtained behavior
A clear and concise description of what happening.
## nDPI Environment (please complete the following information):
* OS name: [e.g. Ubuntu].
* OS version: [e.g. 18.04]
* Architecture: [e.g. arm64]
* nDPI version or commit hash: [e.g. 4.0-stable or 937357e4bc55610f116f66d15a8e0fc1e260c02c].
* nDPI compilation flags used: if you are building from source [e.g. --with-pcre2 --with-local-libgcrypt].
* Attach the `config.log` file generated after `./configure` ran (if you are building from source).
## How to reproduce the reported bug
### Reproducible using ndpiReader?
- [x] The reported bug is reproducible using ndpiReader.
- [x] The reported bug is not reproducible using ndpiReader.
### If applicable, the used ndpiReader options:
``` shell
ndpiReader -q -t -i wlo1 -T 20 -U 20
```
### If your bug is reproducible using a pcap, please attach a pcap file (or a valid link to download it)
Example: test.pcap
### Steps to reproduce the behavior:
1. Run '...'
2. Set '....'
3. Do '....'
4. See error
## Additional context
Add any other context about the problem here.
@@ -0,0 +1,20 @@
---
name: "💡 Suggest an Improvement or Feature"
about: Suggest an idea or a feature for nDPI
title: ''
labels: 'enhancement'
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I always need to [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
@@ -0,0 +1,10 @@
---
name: General question
about: Ask a question about nDPI
title: ''
labels: 'question'
assignees: ''
---
Type your question in a clear and concise way to help the community answering it.
+12
View File
@@ -0,0 +1,12 @@
Please sign (check) the below before submitting the Pull Request:
- [ ] I have signed the ntop Contributor License Agreement at https://github.com/ntop/legal/blob/main/individual-contributor-licence-agreement.md
- [ ] I have read the contributing guide lines at https://github.com/ntop/nDPI/blob/dev/CONTRIBUTING.md
- [ ] I have updated the documentation (in doc/) to reflect the changes made (if applicable)
Link to the related [issue](https://github.com/ntop/nDPI/issues):
Describe changes:
+43
View File
@@ -0,0 +1,43 @@
name: MSBuild
on:
push:
branches:
- dev
pull_request:
branches:
- dev
types: [opened, synchronize, reopened]
release:
types: [created]
jobs:
test:
name: MSBuild
runs-on: windows-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Add msbuild to PATH
uses: microsoft/setup-msbuild@v2
- name: Download NuGet packages
run: msbuild windows/nDPI.sln -t:restore -p:RestorePackagesConfig=true
- name: Build libnDPI only (debug)
run: msbuild windows/nDPI.sln -t:rebuild -property:Configuration=Debug
- name: Download Npcap
run: Invoke-WebRequest -Uri 'https://npcap.com/dist/npcap-1.71.exe' -OutFile 'C:\npcap-1.71.exe'
- name: Install Npcap
run: C:\npcap-1.71.exe /S /winpcap_mode=yes
- name: Build ndpiReader (debug)
run: msbuild windows/nDPI.sln -t:rebuild -property:Configuration=Debug-ndpiReader
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: build-artifacts
path: windows/bin
retention-days: 14
+31
View File
@@ -0,0 +1,31 @@
name: RPM Build
on:
workflow_dispatch:
schedule:
# At the end of every day
- cron: '0 0 * * *'
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
- name: Configure nDPI
run: |
# fake CentOS 7 env
echo 'CentOS Linux release 7.9.2009 (Core)' | sudo tee /etc/centos-release
# symlink source directory for ndpi.spec
ln -sr . $HOME/nDPI
./autogen.sh
cd packages/rpm
./configure
- name: Build RPM package
run: |
make -C packages/rpm package
+450
View File
@@ -0,0 +1,450 @@
name: Build
on:
push:
branches:
- dev
pull_request:
branches:
- dev
types: [opened, synchronize, reopened]
release:
types: [created]
jobs:
python-bindings:
name: Python Bindings (ubuntu-latest)
runs-on: ubuntu-latest
env:
CFLAGS: -Wextra -Werror
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python 3.9
uses: actions/setup-python@v5
with:
python-version: "3.9"
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
- name: Build nDPI library
run: |
./autogen.sh
make -j
sudo make install
- name: Generate Python bindings
run: |
pip install --upgrade pip
pip install -r python/requirements.txt
cd python
python setup.py install
cd ..
- name: Test Python Bindings
run: |
cd python
python tests.py
test-scripts:
name: Test Utils (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install python3-netaddr git whois libxml2-utils
- name: Run Scripts
run: |
echo 'Running ./utils/bitcoinnodes.sh'
./utils/bitcoinnodes.sh >/dev/null
echo 'Running ./utils/get_routes_by_asn.sh AS714'
./utils/get_routes_by_asn.sh AS714 >/dev/null
echo 'Running ./utils/update_every_lists.sh'
./utils/update_every_lists.sh
echo 'Checking for changes in the git tree..'
git update-index --refresh || echo "::warning file=utils/update_every_lists.sh::Please re-run utils/update_every_lists.sh and commit any changes."
git diff-index --quiet HEAD -- || true
test:
name: ${{ matrix.os }} ${{ matrix.arch }} ${{ matrix.gcrypt }} ${{ matrix.compiler }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} ${{ matrix.msan }} ${{ matrix.nBPF }} ${{matrix.lto_gold_linker}} ${{matrix.global_context}}
runs-on: ${{ matrix.os }}
env:
CC: ${{ matrix.compiler }}
CFLAGS: -Wextra -Werror -DNDPI_EXTENDED_SANITY_CHECKS
strategy:
fail-fast: true
matrix:
# macOS-latest == macos-14 on **ARM64**
# There are some issues with external dependencies on macOS-latest. Disable it for the time being
os: ["ubuntu-20.04", "ubuntu-22.04", "ubuntu-24.04", "macOS-12", "macOS-13", "windows-latest"]
arch: ["x86_64"]
gcrypt: ["--with-local-libgcrypt", ""]
compiler: ["cc"]
ar: ["ar"]
ranlib: ["ranlib"]
pcre: [""]
maxminddb: [""]
msan: [""]
nBPF: [""]
lto_gold_linker: [""]
global_context: [""] #Enable by default
include:
- compiler: "gcc-4.9" # "Oldest" gcc easily available. To simulate RHEL7
os: ubuntu-20.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
- compiler: "gcc-14" # "Newest" gcc easily available
os: ubuntu-24.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
lto_gold_linker: "--with-lto-and-gold-linker"
- compiler: "clang-9" # "Oldest" clang easily available
os: ubuntu-20.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
- compiler: "clang-17" # "Newest" clang easily available. See also below...
ar: "llvm-ar-17"
ranlib: "llvm-ranlib-17"
os: ubuntu-24.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
lto_gold_linker: "--with-lto-and-gold-linker"
- compiler: "clang-18" # "The latest clang version easily available should be 18,
# but it is buggy on ubuntu-24.04 with LTO and Gold linker:
# https://github.com/llvm/llvm-project/issues/87553
# https://bugs.launchpad.net/ubuntu/+source/llvm-toolchain-18/+bug/2064187
# Waiting for a fix...
ar: "llvm-ar-18"
ranlib: "llvm-ranlib-18"
os: ubuntu-24.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
lto_gold_linker: ""
- compiler: "cc"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-thread-sanitizer"
nBPF: ""
- compiler: "cc"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: "nBPF"
- compiler: "cc"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
global_context: "--disable-global-context-support"
- compiler: "clang" #TODO: some issues with masan/clang/ubuntu-24.04
os: ubuntu-22.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-memory-sanitizer"
nBPF: ""
- compiler: "cc"
os: macOS-13
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "" # Disable sanitizer on macos
nBPF: ""
- compiler: "cc"
os: macos-14
arch: "x86_64"
gcrypt: ""
pcre: ""
maxminddb: ""
msan: "" # Disable sanitizer on macos
nBPF: ""
- compiler: "cc"
os: ubuntu-latest
arch: "arm64"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "" # Disable sanitizer on arm64
nBPF: ""
- compiler: "cc"
os: ubuntu-latest
arch: "armhf"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
nBPF: ""
- compiler: "cc"
os: ubuntu-latest
arch: "s390x"
gcrypt: ""
pcre: "--with-pcre2"
maxminddb: "--with-maxminddb"
msan: ""
nBPF: ""
steps:
- name: Setup multiarch/qemu-user-static
if: startsWith(matrix.os, 'ubuntu') && !startsWith(matrix.arch, 'x86_64')
run: |
docker run --rm --privileged multiarch/qemu-user-static:register --reset
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis
- name: Fix kernel mmap rnd bits on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
run: |
# Workaround for compatinility between latest kernel and sanitizer
# See https://github.com/actions/runner-images/issues/9491
sudo sysctl vm.mmap_rnd_bits=28
- name: Install Ubuntu Prerequisites
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
sudo apt-get install gcc-mingw-w64 libc6-dev
sudo apt-get install doxygen python3-sphinx python3-sphinx-rtd-theme python3-breathe python3-pip
sudo apt-get install rrdtool librrd-dev parallel
- name: Install Ubuntu Prerequisites (libgcrypt)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.gcrypt, '--with-local-libgcrypt')
run: |
sudo apt-get install libgcrypt20-dev
- name: Install Ubuntu Prerequisites (libpcre2)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.pcre, '--with-pcre2')
run: |
sudo apt-get install libpcre3-dev
- name: Install Ubuntu Prerequisites (maxminddb)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.maxminddb, '--with-maxminddb')
run: |
sudo apt-get install libmaxminddb-dev
- name: Install Ubuntu Prerequisites (nBPF)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.nBPF, 'nBPF')
run: |
git clone https://github.com/ntop/PF_RING.git ../PF_RING
cd ../PF_RING/userland/nbpf
./configure
make
cd -
- name: Setup Ubuntu specified compiler
if: startsWith(matrix.os, 'ubuntu-20.04') && startsWith(matrix.arch, 'x86_64') && ! startsWith(matrix.compiler, 'cc')
run: |
#For gcc-4.9 (on ubuntu-20.04)
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial main" | sudo tee -a /etc/apt/sources.list
echo "deb http://dk.archive.ubuntu.com/ubuntu/ xenial universe" | sudo tee -a /etc/apt/sources.list
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 40976EAF437D05B5
sudo apt-get update
sudo apt-get install ${{ matrix.compiler }}
- name: Install Windows msys2 prerequisites
if: startsWith(matrix.os, 'windows')
uses: msys2/setup-msys2@v2
with:
msystem: MINGW64
update: true
install: git mingw-w64-x86_64-toolchain automake1.16 automake-wrapper autoconf libtool make mingw-w64-x86_64-json-c mingw-w64-x86_64-crt-git mingw-w64-x86_64-pcre mingw-w64-x86_64-libpcap mingw-w64-x86_64-libgcrypt parallel
- name: Installing MacOS prerequisites
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64')
run: |
# A workaround for "The `brew link` step did not complete successfully" error.
# See https://github.com/Homebrew/homebrew-core/issues/165793#issuecomment-1991817938
find /usr/local/bin -lname '*/Library/Frameworks/Python.framework/*' -delete
sudo rm -rf /Library/Frameworks/Python.framework/
brew install --force python3 && brew unlink python3 && brew unlink python3 && brew link --overwrite python3
brew install coreutils wdiff colordiff autoconf automake libtool pkg-config gettext json-c rrdtool parallel
- name: Install MacOS Prerequisites (libgcrypt)
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.gcrypt, '--with-local-libgcrypt')
run: |
brew install libgcrypt
- name: Install MacOS Prerequisites (libpcre2)
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.pcre, '--with-pcre2')
run: |
brew install pcre
- name: Install MacOS Prerequisites (maxminddb)
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.maxminddb, '--with-maxminddb')
run: |
brew install libmaxminddb
- name: Configure nDPI on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
run: |
AR=${{ matrix.ar }} RANLIB=${{ matrix.ranlib }} ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs ${{matrix.lto_gold_linker}} ${{matrix.global_context}}
- name: Configure nDPI on MacOS
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'cc')
run: |
./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs
- name: Configure nDPI on Windows msys2
if: startsWith(matrix.os, 'windows') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'cc')
run: |
msys2 -c './autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --disable-npcap ${{ matrix.gcrypt }}'
- name: Build nDPI on Windows msys2
if: startsWith(matrix.os, 'windows') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'cc')
run: |
msys2 -c 'make -j all'
msys2 -c 'ldd ./example/ndpiReader.exe'
- name: Build nDPI
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows') && !startsWith(matrix.os, 'macos-14')
run: |
make -j all
make -C example ndpiSimpleIntegration
make -C rrdtool
- name: Build nDPI (MacOS M1)
if: startsWith(matrix.os, 'macos-14')
run: |
make -j all
make -C example ndpiSimpleIntegration
#There are somes issues with librrd
#make -C rrdtool
- name: Print nDPI long help
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
cd ./example && ./ndpiReader -H
- name: Install nDPI
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
DESTDIR=/tmp/ndpi make install
ls -alhHR /tmp/ndpi
- name: Test nDPI [SYMBOLS]
if: (startsWith(matrix.os, 'ubuntu') || startsWith(matrix.os, 'mac')) && startsWith(matrix.arch, 'x86_64')
run: |
./utils/check_symbols.sh || { FAILED=$?; echo "::error file=${NDPI_LIB}::Unwanted libc symbols found: ${FAILED}. Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of malloc/calloc/realloc/free."; false; }
env:
NDPI_LIB: src/lib/libndpi.a
- name: Test nDPI [DIFF]
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
NDPI_FORCE_PARALLEL_UTESTS=1 NDPI_SKIP_PARALLEL_BAR=1 ./tests/do.sh
- name: Test nDPI [UNIT]
#Some issues with masan + json-c. Disable the test as workaround
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows') && !startsWith(matrix.msan, '--with-memory-sanitizer') && !startsWith(matrix.os, 'macos-14')
run: |
./tests/do-unit.sh
- name: Test nDPI [DGA]
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
./tests/do-dga.sh
- name: Test nDPI [DIFF] (runs only on windows jobs)
if: startsWith(matrix.arch, 'x86_64') && startsWith(matrix.os, 'windows')
run: |
msys2 -c 'NDPI_FORCE_PARALLEL_UTESTS=1 NDPI_SKIP_PARALLEL_BAR=1 ./tests/do.sh'
- name: Test nDPI [UNIT] (runs only on windows jobs)
if: startsWith(matrix.arch, 'x86_64') && startsWith(matrix.os, 'windows')
run: |
msys2 -c './tests/do-unit.sh'
- name: Test nDPI [DGA] (runs only on windows jobs)
if: startsWith(matrix.arch, 'x86_64') && startsWith(matrix.os, 'windows')
run: |
msys2 -c './tests/do-dga.sh'
- name: Generate/Verify tarball
if: startsWith(matrix.os, 'ubuntu-latest') && startsWith(matrix.arch, 'x86_64')
run: |
make dist
./utils/verify_dist_tarball.sh
- name: Build nDPI [Mingw-w64] (runs only on ubuntu jobs)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.nBPF, 'nBPF')
run: |
make distclean
./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --host=x86_64-w64-mingw32
make -j $(nproc) all
env:
CC:
- name: Display qemu specified architecture (arm64 - little endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'arm64')
uses: docker://multiarch/ubuntu-core:arm64-bionic
with:
args: >
bash -c
"uname -a &&
lscpu | grep Endian
"
- name: Configure, compile and test using qemu for the specified architecture (arm64 - little endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'arm64')
uses: docker://multiarch/ubuntu-core:arm64-bionic
with:
args: >
bash -c
"apt-get -y update &&
apt-get -y install git wdiff colordiff autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
git config --global --add safe.directory $(realpath .) &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs &&
make -j all &&
make -C example ndpiSimpleIntegration &&
make -C rrdtool &&
make check VERBOSE=1
"
- name: Display qemu specified architecture (armhf - little endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'armhf')
uses: docker://multiarch/ubuntu-core:armhf-bionic
with:
args: >
bash -c
"uname -a &&
lscpu | grep Endian
"
- name: Configure, compile and test using qemu for the specified architecture (armhf - little endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'armhf')
uses: docker://multiarch/ubuntu-core:armhf-bionic
with:
args: >
bash -c
"apt-get -y update &&
apt-get -y install git wdiff colordiff autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
git config --global --add safe.directory $(realpath .) &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs &&
make -j all &&
make -C example ndpiSimpleIntegration &&
make -C rrdtool &&
make check VERBOSE=1
"
- name: Display qemu specified architecture (s390x - big endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 's390x')
uses: docker://multiarch/ubuntu-core:s390x-bionic
with:
args: >
bash -c
"uname -a &&
lscpu | grep Endian
"
- name: Configure and compile using qemu for the specified architecture (s390x - big endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 's390x')
uses: docker://multiarch/ubuntu-core:s390x-bionic
with:
args: >
bash -c
"apt-get -y update &&
apt-get -y install git wdiff colordiff autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
git config --global --add safe.directory $(realpath .) &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs &&
make -j all &&
make -C example ndpiSimpleIntegration &&
make -C rrdtool &&
make check VERBOSE=1
"
+111
View File
@@ -0,0 +1,111 @@
name: Scheduled builds
on:
workflow_dispatch:
schedule:
#At the end of every day
- cron: '0 0 * * *'
jobs:
coverage:
name: Coverage (ubuntu-latest)
runs-on: ubuntu-latest
env:
CFLAGS: -Werror
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
sudo apt-get install rrdtool librrd-dev
sudo apt-get install libpcre3-dev libmaxminddb-dev lcov
sudo apt-get install wdiff colordiff
- name: Configure
run: ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-code-coverage --with-pcre2 --with-maxminddb --enable-tls-sigs
- name: Build
run: make all
- name: Test
run: |
make check VERBOSE=1
- name: Generate Coverage Report
run: |
make coverage
- uses: actions/upload-artifact@v4
with:
name: ndpi-coverage-report
path: coverage_report
retention-days: 7
documentation:
name: Documentation (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python 3.9
uses: actions/setup-python@v4
with:
python-version: "3.9"
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison doxygen
- name: Configure nDPI library
run: |
./autogen.sh --with-only-libndpi --enable-option-checking=fatal --enable-debug-build
- name: Generate Documentation
run: |
pip install --upgrade pip
pip install -r doc/requirements.txt
make doc
mkdir -vp doc/_build/ndpi-documentation-upload/ndpi-documentation
mv -v doc/_build/html doc/_build/ndpi-documentation-upload/ndpi-documentation/html
- uses: actions/upload-artifact@v4
with:
name: ndpi-documentation
path: doc/_build/ndpi-documentation-upload
retention-days: 7
performance:
name: Performance (ubuntu-latest)
runs-on: ubuntu-latest
env:
GO111MODULE: on
CFLAGS: -Werror
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo add-apt-repository ppa:longsleep/golang-backports
sudo apt-get update
sudo apt-get install libunwind-dev
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison libpcap-dev parallel
sudo apt-get install golang-1.16-go libgoogle-perftools-dev graphviz
go install github.com/google/pprof@latest
sudo ln -s ${HOME}/go/bin/pprof /usr/bin/pprof
pprof -h
- name: Configure nDPI library
run: |
./autogen.sh --enable-gprof --enable-option-checking=fatal --with-pcre2 --with-maxminddb --enable-tls-sigs
- name: Build nDPI library
run: |
make -j
- name: Performance Profiling
run: |
NDPI_FORCE_PARALLEL_UTESTS=1 NDPI_SKIP_PARALLEL_BAR=1 ./tests/do.sh
mkdir ndpi-performance-upload
for d in $(find ./tests/cfgs/* -type d -maxdepth 0 2>/dev/null) ; do
PROFILE="$(basename $d)"
mv -v tests/cfgs/${PROFILE}/result/cpu_profile.png ndpi-performance-upload/${PROFILE}_cpu_profile.png
mv -v tests/cfgs/${PROFILE}/result/heap_profile.png ndpi-performance-upload/${PROFILE}_heap_profile.png
done
- uses: actions/upload-artifact@v4
with:
name: ndpi-performance
path: ndpi-performance-upload
retention-days: 7
+34
View File
@@ -0,0 +1,34 @@
name: CIFuzz
on: [push, pull_request]
jobs:
Fuzzing:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
sanitizer: [address, undefined, memory]
steps:
- name: Build Fuzzers - ${{ matrix.sanitizer }}
uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@master
with:
oss-fuzz-project-name: 'ndpi'
dry-run: false
sanitizer: ${{ matrix.sanitizer }}
- name: Run Fuzzers - ${{ matrix.sanitizer }}
uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@master
with:
oss-fuzz-project-name: 'ndpi'
fuzz-seconds: 1320
dry-run: false
sanitizer: ${{ matrix.sanitizer }}
- name: Check Crash (fails when a crash is detected)
# Run Fuzzers return success even when setting dry-run to false.
# A temporal workaround is to trigger failure manually if we fing crash files.
run: |
exit $(ls out/artifacts |wc -l)
- name: Upload Crash (upload detected crash as artifacts)
uses: actions/upload-artifact@v4
if: failure()
with:
name: ${{ matrix.sanitizer }}-artifacts
path: ./out/artifacts
+59
View File
@@ -0,0 +1,59 @@
name: "CodeQL"
on:
push:
branches: [ 'dev' ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ 'dev' ]
schedule:
- cron: '25 19 * * 2'
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
env:
GO111MODULE: on
CFLAGS: -Werror
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: [ 'cpp', 'python' ]
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo add-apt-repository ppa:longsleep/golang-backports
sudo apt-get update
sudo apt-get install -y libunwind-dev
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison libpcap-dev
sudo apt-get install golang-1.16-go libgoogle-perftools-dev graphviz
go install github.com/google/pprof@latest
sudo ln -s ${HOME}/go/bin/pprof /usr/bin/pprof
pprof -h
- name: Configure nDPI library
run: |
./autogen.sh --enable-gprof --enable-option-checking=fatal --with-pcre2 --with-maxminddb --enable-tls-sigs --enable-debug-messages
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
- name: Build nDPI library
run: |
make -j
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
category: "/language:${{matrix.language}}"
+164 -20
View File
@@ -1,37 +1,181 @@
*.[oa]
*~
*o.cmd
*.lo
*.obj
*o.cmd
*.ko
*.swp
*.[o]
*.db
*.db-*
*.dll
*.ilk
*.in
*.ipch
*.ko
*.la
.libs
*.lo
*.log
*.obj
*.opendb
*.pdb
*.recipe
*.suo
*.swp
*.tlog
*.user
*o.cmd
*o.cmd
*~
.autotools
.cproject
.dirstamp
stamp-h1
/configure
.libs
.project
.settings
/Makefile
/aclocal.m4
/autom4te.cache/
/compile
/config.guess
/config.h
/config.h.in
/config.sub
/config.log
/config.status
/config.sub
/configure
/configure.tmp
/depcomp
/doc/Doxyfile.cfg
/example/.deps/
/example/Makefile
/example/Makefile.dpdk
/example/build
/example/libndpiReader.a
/example/ndpiReader
/example/ndpiReader.exe
/example/ndpiSimpleIntegration
/fuzz/.deps/
/fuzz/Makefile
/fuzz/fuzz_ndpi_reader
/fuzz/fuzz_process_packet
/fuzz/fuzz_ndpi_reader_alloc_fail
/fuzz/fuzz_quic_get_crypto_data
/fuzz/fuzz_alg_bins
/fuzz/fuzz_alg_hll
/fuzz/fuzz_alg_hw_rsi_outliers_da
/fuzz/fuzz_alg_jitter
/fuzz/fuzz_alg_crc32_md5
/fuzz/fuzz_alg_ses_des
/fuzz/fuzz_alg_bytestream
/fuzz/fuzz_alg_shoco
/fuzz/fuzz_config
/fuzz/fuzz_community_id
/fuzz/fuzz_serialization
/fuzz/fuzz_ds_patricia
/fuzz/fuzz_ds_libcache
/fuzz/fuzz_ds_tree
/fuzz/fuzz_ds_ptree
/fuzz/fuzz_ds_hash
/fuzz/fuzz_ds_ahocorasick
/fuzz/fuzz_ds_bitmap64_fuse
/fuzz/fuzz_ds_domain_classify
/fuzz/fuzz_libinjection
/fuzz/fuzz_binaryfusefilter
/fuzz/fuzz_tls_certificate
/fuzz/fuzz_dga
/fuzz/fuzz_is_stun_udp
/fuzz/fuzz_is_stun_tcp
/fuzz/fuzz_ds_cmsketch
/fuzz/fuzz_gcrypt_light
/fuzz/fuzz_gcrypt_aes
/fuzz/fuzz_gcrypt_gcm
/fuzz/fuzz_gcrypt_cipher
/fuzz/fuzz_ndpi_reader_payload_analyzer
/fuzz/fuzz_ndpi_reader_pl7m_simplest
/fuzz/fuzz_ndpi_reader_pl7m_simplest_internal
/fuzz/fuzz_ndpi_reader_pl7m
/fuzz/fuzz_ndpi_reader_pl7m_64k
/fuzz/fuzz_ndpi_reader_pl7m_internal
/fuzz/fuzz_filecfg_protocols
/fuzz/fuzz_filecfg_categories
/fuzz/fuzz_filecfg_category
/fuzz/fuzz_filecfg_malicious_sha1
/fuzz/fuzz_filecfg_malicious_ja3
/fuzz/fuzz_filecfg_risk_domains
/fuzz/fuzz_filecfg_config
/fuzz/fuzz_readerutils_workflow
/fuzz/fuzz_readerutils_parseprotolist
/fuzz/fuzz_ndpi_reader_alloc_fail_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_pl7m_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_pl7m_64k_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_pl7m_internal_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_pl7m_simplest_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_pl7m_simplest_internal_seed_corpus.zip
/fuzz/fuzz_quic_get_crypto_data_seed_corpus.zip
/fuzz/fuzz_community_id_seed_corpus.zip
/fuzz/fuzz_is_stun_udp_seed_corpus.zip
/fuzz/fuzz_is_stun_tcp_seed_corpus.zip
/fuzz/fuzz_serialization_seed_corpus.zip
/fuzz/fuzz_alg_ses_des_seed_corpus.zip
/fuzz/fuzz_alg_bins_seed_corpus.zip
/fuzz/fuzz_alg_hll_seed_corpus.zip
/fuzz/fuzz_alg_jitter_seed_corpus.zip
/fuzz/fuzz_alg_crc32_md5_seed_corpus.zip
/fuzz/fuzz_alg_bytestream_seed_corpus.zip
/fuzz/fuzz_ds_libcache_seed_corpus.zip
/fuzz/fuzz_ds_ptree_seed_corpus.zip
/fuzz/fuzz_ds_bitmap64_fuse_seed_corpus.zip
/fuzz/fuzz_ds_domain_classify_seed_corpus.zip
/fuzz/fuzz_libinjection_seed_corpus.zip
/fuzz/fuzz_tls_certificate_seed_corpus.zip
/fuzz/fuzz_filecfg_protocols_seed_corpus.zip
/fuzz/fuzz_filecfg_categories_seed_corpus.zip
/fuzz/fuzz_filecfg_category_seed_corpus.zip
/fuzz/fuzz_filecfg_malicious_sha1_seed_corpus.zip
/fuzz/fuzz_filecfg_malicious_ja3_seed_corpus.zip
/fuzz/fuzz_filecfg_risk_domains_seed_corpus.zip
/fuzz/fuzz_filecfg_config_seed_corpus.zip
/fuzz/fuzz_dga_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_payload_analyzer_seed_corpus.zip
/fuzz/fuzz_readerutils_workflow_seed_corpus.zip
/fuzz/fuzz_readerutils_parseprotolist_seed_corpus.zip
/fuzz/fuzz_*.dict
/fuzz/testpcaps.zip
/influxdb/Makefile
/install-sh
/ltmain.sh
/missing
/Makefile
/libndpi.pc
/libtool
/src/lib/Makefile
/src/include/Makefile
/example/ndpiReader
/example/Makefile
/aclocal.m4
/ltmain.sh
/m4/libtool.m4
/m4/ltoptions.m4
/m4/ltsugar.m4
/m4/ltversion.m4
/m4/lt~obsolete.m4
/missing
/rrdtool/Makefile
/rrdtool/rrd_anomaly
/rrdtool/rrd_similarity
/src/include/Makefile
/src/include/ndpi_config.h
/src/include/ndpi_define.h
/src/lib/.deps/
/src/lib/Makefile
/src/lib/libndpi
/src/lib/libndpi-*.dll
/src/lib/libndpi.4
/src/lib/libndpi.a
/src/lib/libndpi.so*
/src/lib/protocols/.deps/
/src/lib/third_party/src/.deps/
/src/lib/third_party/src/fuzz/.deps/
/tags
/test-driver
/tests/Makefile
/tests/dga/Makefile
/tests/dga/dga_evaluate
/tests/dga/dga_evaluate.exe
/tests/do.sh
/tests/performance/Makefile
/tests/unit/Makefile
/tests/unit/unit
compile
stamp-h1
/windows/packages/pthreads.2.9.1.4
/windows/packages/pthreads.redist.2.9.1.4
/windows/bin/x64_Debug-ndpiReader/ndpiReader.exe
/windows/obj/nDPI_Debug-ndpiReader_x64/nDPI.vcxproj.FileListAbsolute.txt
-22
View File
@@ -1,22 +0,0 @@
language: c
compiler:
- clang
- gcc
install:
- sudo apt-get update || true
- sudo apt-get install build-essential
- sudo apt-get install libpcap-dev libtool autoconf automake autogen
before_script:
- ./autogen.sh
script:
- ./configure
- make
after_script:
- cd tests
- ./do.sh
+1212
View File
File diff suppressed because it is too large Load Diff
+33
View File
@@ -0,0 +1,33 @@
# How to contribute
Please keep all code, comments, issues and pull requests in English, otherwise they will be **removed**.
## Found a bug?
- First, check there are no already open issues for the bug. See [here](https://github.com/ntop/nDPI/issues).
- If there is not open issue addressing the bug, [open a new one](https://github.com/ntop/nDPI/issues/new).
- Make sure to include a **descriptive and clear title and description**, including the **version of nDPI** being used.
- Try to include steps to reproduce the bug.
- If applicable, try to provide a stacktrace.
- Providing a simple test case which shows the bug would help resolve the issue faster.
## Written a new protocol dissector?
- Ensure you have followed the guide in the [README](https://github.com/ntop/nDPI#how-to-add-a-new-protocol-dissector).
- Open a new pull request with a **clear, descriptive** title.
- Ensure the pull request passes CI testing. Any pull request which fails CI will be **ignored**.
- If possible, add a new test for the new protocol. See [existing tests](https://github.com/ntop/nDPI/tree/dev/tests) for examples.
## Written a fix for a bug?
- Open a new pull request with a **clear, descriptive** title.
- If there is an open issue for the bug, ensure the description of the pull request contains `Fixes #issue_number`.
- Ensure the pull request passes CI testing. Any pull request which fails CI will be **ignored**.
## Before you submit your code
As we want to make sure that we can incorporate your changes in nDPI, we ask all contributors to read and sign the
[Contributor License Agreement](https://github.com/ntop/legal/blob/main/individual-contributor-licence-agreement.md)
that guarantees that your code is original and royalty-free to keep nDPI free and available for everybody,
Finally, thank you for your contributions.
-16
View File
@@ -1,16 +0,0 @@
2014-03-21:
- improved support for eDonkey/eMule/Kademlia
- improved support for PPLive
2014-03-20:
- code optimizations
- consistency improvements
- added support for new applications: Pando Media Booster
- improved support for Steam
- added support for new web services: Wikipedia, MSN, Amazon, eBay, CNN
2014-03-19:
- added new protocols: FTP, code improvements
2014-03-17:
- added new protocols: SOCKSv4, SOCKSv5, RTMP
+42 -2
View File
@@ -1,8 +1,48 @@
ACLOCAL_AMFLAGS = -I m4
SUBDIRS = src/lib @EXTRA_TARGETS@
SUBDIRS = src/lib example
if BUILD_FUZZTARGETS
SUBDIRS += fuzz
endif
pkgconfigdir = $(libdir)/pkgconfig
pkgconfig_DATA = libndpi.pc
EXTRA_DIST = libndpi.sym autogen.sh
EXTRA_DIST = README.md README.fuzzer.md CHANGELOG.md CONTRIBUTING.md \
autogen.sh wireshark windows utils packages \
influxdb/Makefile.in influxdb/metric_anomaly.c influxdb/README.txt \
rrdtool/Makefile.in rrdtool/README.txt rrdtool/rrd_anomaly.c rrdtool/rrd_similarity.c \
doc/requirements.txt doc/conf.py doc/flow_risks.rst doc/protocols.rst doc/guide/nDPI_QuickStartGuide.pages \
doc/guide/nDPI_QuickStartGuide.pdf doc/img/logo.png doc/index.rst \
doc/Makefile doc/what_is_ndpi.rst doc/FAQ.md doc/configuration_parameters.md \
doc/library_initialization.md \
python/DEV_GUIDE.md python/dev_requirements.txt python/ndpi_example.py python/ndpi/__init__.py \
python/ndpi/ndpi_build.py python/ndpi/ndpi.py python/README.md \
python/requirements.txt python/setup.py python/tests.py \
lists/100_malware.list \
lists/107_gambling.list \
lists/107_gambling_custom.list \
lists/public_suffix_list.dat \
lists/README.md \
sonar-project.properties .github .ci-ignore
doc:
make -C doc html
doc-view:
python3 -m http.server --bind 127.0.0.1 --directory doc/_build/html 8080
coverage: all
mkdir -vp coverage_report/ndpi_coverage_report
lcov --directory . --capture --output-file lcov.info
genhtml -o coverage_report/ndpi_coverage_report lcov.info
clean-coverage:
rm -f lcov.info
find . -type f -name "*.gcda" -exec rm "{}" +
find . -type f -name "*.gcno" -exec rm "{}" +
changelog:
git log --since={`curl -s https://github.com/ntop/ndpi/releases | grep datetime | head -n1 | egrep -o "[0-9]+\-[0-9]+\-[0-9]+"`} --name-only --pretty=format:" - %s" | grep "^ " > CHANGELOG.latest
.PHONY: doc doc-view coverage clean-coverage changelog
+41
View File
@@ -0,0 +1,41 @@
## Compiling nDPI with Fuzzer
- Install the latest clang (sudo apt-get install clang-10)
- Export environamental variables to prefer clang over g++ (of installed)
- export CC=/usr/bin/clang-10
- export CPP=/usr/bin/clang-cpp-10
- export CXX=/usr/bin/clang++-10
- export LD=/usr/bin/ld.lld-10
- Run autogen.sh
- ./autogen.sh --with-sanitizer --enable-fuzztargets
## Testing nDPI with ClusterFuzz Artifacts
- Download the artifact (example clusterfuzz-testcase-fuzz_process_packet-4992218834796544)
- Run nDPI against the artifact
- Example: ./fuzz/fuzz_process_packet clusterfuzz-testcase-fuzz_process_packet-4992218834796544
The output is the error report
```
./fuzz/fuzz_process_packet /tmp/clusterfuzz-testcase-fuzz_process_packet-4992218834796544
AddressSanitizer:DEADLYSIGNAL
=================================================================
==11590==ERROR: AddressSanitizer: SEGV on unknown address 0x61a100000087 (pc 0x00000056e6a4 bp 0x7ffd624fa170 sp 0x7ffd624fa090 T0)
==11590==The signal is caused by a READ memory access.
#0 0x56e6a4 in quic_len /home/deri/nDPI/src/lib/protocols/quic.c:203:12
#1 0x575d6b in decrypt_initial_packet /home/deri/nDPI/src/lib/protocols/quic.c:993:16
#2 0x571776 in get_clear_payload /home/deri/nDPI/src/lib/protocols/quic.c:1302:21
#3 0x56f149 in ndpi_search_quic /home/deri/nDPI/src/lib/protocols/quic.c:1658:19
#4 0x503935 in check_ndpi_detection_func /home/deri/nDPI/src/lib/ndpi_main.c:4683:6
#5 0x5056fb in check_ndpi_udp_flow_func /home/deri/nDPI/src/lib/ndpi_main.c:4742:10
#6 0x505152 in ndpi_check_flow_func /home/deri/nDPI/src/lib/ndpi_main.c:4775:12
#7 0x5174cf in ndpi_detection_process_packet /home/deri/nDPI/src/lib/ndpi_main.c:5545:15
#8 0x4c709b in LLVMFuzzerTestOneInput /home/deri/nDPI/fuzz/fuzz_process_packet.c:30:3
#9 0x4c7640 in main /home/deri/nDPI/fuzz/fuzz_process_packet.c:90:17
#10 0x7f888e5dabf6 in __libc_start_main /build/glibc-S9d2JN/glibc-2.27/csu/../csu/libc-start.c:310
#11 0x41c399 in _start (/home/deri/nDPI/fuzz/fuzz_process_packet+0x41c399)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/deri/nDPI/src/lib/protocols/quic.c:203:12 in quic_len
==11590==ABORTING
```
+101 -23
View File
@@ -1,37 +1,115 @@
# What is nDPI ?
# nDPI
nDPI is an open source GPLv3 library for deep-packet inspection. Based on OpenDPI it includes ntop extensions. We have tried to push them into the OpenDPI source tree but nobody answered emails so we have decided to create mour own source tree
[![Build Status](https://img.shields.io/github/actions/workflow/status/ntop/nDPI/build.yml?branch=dev&logo=github)](https://github.com/ntop/nDPI/actions?query=workflow%3ABuild)
[![Fuzzing Status](https://oss-fuzz-build-logs.storage.googleapis.com/badges/ndpi.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened&can=1&q=proj:ndpi)
# How To Compile nDPI
## What is nDPI ?
In order to compile this library do
nDPI® is an open source LGPLv3 library for deep-packet inspection. Based on OpenDPI it includes ntop extensions. We have tried to push them into the OpenDPI source tree but nobody answered emails so we have decided to create our own source tree
A generic FAQ about nDPI® is available [here](https://github.com/ntop/nDPI/blob/dev/doc/FAQ.md)
### How To Compile nDPI
In order to compile this project do
- ./autogen.sh
- ./configure
- make
Please note that the pre-requisites for compilation include:
- GNU tools (autogen, automake, autoconf, libtool)
- GNU C compiler (gcc)
To compile the library w/o any tools or tests:
# How To Add A New Protocol Dissector
- ./autogen.sh --with-only-libndpi
- make
To run tests do additionally:
- ./tests/do.sh # Generate and check for diff's in PCAP files
- ./tests/do-unit.sh # Run unit tests
- ./tests/do-dga.sh # Run DGA detection test
or run all with: `make check`
Please note that the (minimal) pre-requisites for compilation include:
- GNU tools (autoconf automake libtool pkg-config gettext flex bison)
- GNU C compiler (gcc) or Clang
On Debian/Ubuntu systems do:
- sudo apt-get install build-essential git gettext flex bison libtool autoconf automake pkg-config libpcap-dev libjson-c-dev libnuma-dev libpcre2-dev libmaxminddb-dev librrd-dev
On Arch Linux:
- sudo pacman -S gcc git gettext flex bison libtool autoconf automake pkg-config libpcap json-c numactl pcre2 libmaxminddb rrdtool
On FreeBSD:
- sudo pkg install gcc git gettext flex bison libtool autoconf automake devel/pkgconf gmake libpcap json-c pcre2 libmaxminddb rrdtool
Remember to use `gmake` and not `make` on FreeBSD
On MacOS:
- brew install coreutils gcc git gettext flex bison libtool autoconf automake pkg-config libpcap json-c pcre2 libmaxminddb rrdtool
On Windows:
There are three supported ways to build nDPI:
1. MSYS2 (assuming [MSYS2](https://www.msys2.org/) already installed):
- msys2 -c "pacman --noconfirm -S --needed --overwrite '\*' git mingw-w64-x86\_64-toolchain automake1.16 automake-wrapper autoconf libtool make mingw-w64-x86\_64-json-c mingw-w64-x86\_64-crt-git mingw-w64-x86\_64-pcre2 mingw-w64-x86\_64-libpcap"
2. Mingw-w64
3. Visual Studio (see `windows/nDPI.sln`)
Note: All Windows versions require [npcap](https://npcap.com/#download) with WinPcap compatibility mode enabled.
### How To Build The Documentation
- pip install --upgrade pip
- pip install -r doc/requirements.txt
- make doc
Use the builtin python3 webserver to view documentation:
- make doc-view
### How To Add A New Protocol Dissector
The entire procedure of adding new protocols in detail:
1. Add new protocol together with its unique ID to: src/include/ndpi_protocols_osdpi.h
2. Create a new protocol in: src/lib/protocols/
3. Variables to be kept for the duration of the entire flow (as state variables) needs to be placed in: /include/ndpi_structs.h in ndpi_flow_tcp_struct (for TCP only), ndpi_flow_udp_struct (for UDP only), or ndpi_flow_struct (for both).
4. Add a new entry for the search function for the new protocol in: src/include/ndpi_protocols.h
5. Choose (do not change anything) a selection bitmask from: src/include/ndpi_define.h
6. Add a new entry in ndpi_set_protocol_detection_bitmask2 in: src/lib/ndpi_main.c
7. Set protocol default ports in ndpi_init_protocol_defaults in: src/lib/ndpi_main.c
8. Add the new protocol file to: src/lib/Makefile.am
9. ./autogen.sh
10. ./configure
11. make
1. Add new protocol together with its unique ID to: `src/include/ndpi_protocol_ids.h`
2. Create a new protocol in: `src/lib/protocols/`
3. Variables to be kept for the duration of the entire flow (as state variables) need to be placed in: `src/include/ndpi_typedefs.h` in `ndpi_flow_tcp_struct` (for TCP only), `ndpi_flow_udp_struct` (for UDP only), or `ndpi_flow_struct` (for both).
4. Add a new entry for the search function for the new protocol in: `src/include/ndpi_protocols.h`
5. Choose (do not change anything) a selection bitmask from: `src/include/ndpi_define.h`
6. Set protocol default ports in `ndpi_init_protocol_defaults` in: `src/lib/ndpi_main.c`
7. Be sure to have nBPF support, cloning `PF_RING` in the same directory where you cloned `nDPI`: `git clone https://github.com/ntop/PF_RING/ && cd PF_RING/userland/nbpf && ./configure && make`. You can ignore the `/bin/sh: 1: ../lib/pfring_config: not found` error
8. From the `nDPI` root directory, `./autogen.sh --with-pcre2` (nBPF and PCRE2 are usually optional, but they are needed to run/update *all* the unit tests)
9. `make`
10. `make check`
11. Update the documentation, adding this new protocol to `doc/protocols.rst`
12. Update the Windows Visual Studio configuration, adding the new c file in `windows/nDPI.vcxproj`
# Creating A Source File Tar Ball
### How to use nDPI to Block Selected Traffic
If you want to distribute a source tar file of nDPI do:
You can use nDPI to selectively block selected Internet traffic by embedding it onto an application (remember that nDPI is just a library). Both [ntopng](https://github.com/ntop/ntopng) and [nProbe cento](http://www.ntop.org/products/netflow/nprobe-cento/) can do this.
- make dist
### nDPI Paper Citation
- Deri, Luca, et al. [nDPI: Open-source high-speed deep packet inspection](http://luca.ntop.org/nDPI.pdf) 2014 International Wireless Communications and Mobile Computing Conference (IWCMC). IEEE, 2014.
### Videos and Presentations
- [Using nDPI for Monitoring and Security](https://archive.fosdem.org/2021/schedule/event/nemondpi/)
- [Network Traffic Classification for Cybersecurity and Monitoring](https://fosdem.org/2022/schedule/event/using_ndpi_to_efficiently_classify_network_traffic/)
### nDPI-Related Projects
- [nfstream](https://github.com/aouinizied/nfstream)
- [nDPId](https://github.com/utoni/nDPId)
### DISCLAIMER
While we do our best to detect network protocols, we cannot guarantee that our software is error free and 100% accurate in protocol detection. Please make sure that you respect the privacy of users and you have proper authorization to listen, capture and inspect network traffic.
nDPI is a registered trademark in the US and EU.
[ntopng_logo]: https://camo.githubusercontent.com/0f789abcef232035c05e0d2e82afa3cc3be46485/687474703a2f2f7777772e6e746f702e6f72672f77702d636f6e74656e742f75706c6f6164732f323031312f30382f6e746f706e672d69636f6e2d313530783135302e706e67
[ntop_logo]: https://camo.githubusercontent.com/58e2a1ecfff62d8ecc9d74633bd1013f26e06cba/687474703a2f2f7777772e6e746f702e6f72672f77702d636f6e74656e742f75706c6f6164732f323031352f30352f6e746f702e706e67
-20
View File
@@ -1,20 +0,0 @@
rerequisites for Compilation
-----------------------------
Prerequisites
- GNU autotools/libtool
- libpcap or PF_RING (optional but recommended)
On Ubuntu/Debian
- apt-get install build-essential
- apt-get install git autoconf automake autogen libpcap-dev libtool
On Fedora/CentOS
- yum groupinstall "Development tools"
- yum install git autoconf automake autogen libpcap-devel libtool
On MacOSX (using http://brew.sh)
brew install autoconf automake libtool git
On FreeBSD
- pkg install autoconf automake libtool gmake git
-18
View File
@@ -1,18 +0,0 @@
Tor
---
Tor protocol can use SSL to hide itself. These are examples:
TCP 37.128.208.46:9001 <-> 172.16.253.130:2078 [VLAN: 0][proto: 91/SSL][132 pkts/93834 bytes][SSL client: www.jwrpsthzrih.com]
TCP 172.16.253.130:2021 <-> 75.147.140.249:443 [VLAN: 0][proto: 91/SSL][28 pkts/8053 bytes][SSL client: www.5akw23dx.com]
TCP 172.16.253.130:2077 <-> 77.247.181.163:443 [VLAN: 0][proto: 91/SSL][136 pkts/94329 bytes][SSL client: www.fk4pprq42hsvl2wey.com]
It can be detected by analyzing the SSL client certificate and checking the name that does not match to a real host in
addition of begin a bit weird. As doing DNS resolution is not a task for nDPI we let applications do and then recognize
SSL-tunnelled connections.
See http://www.netresec.com/?page=Blog&month=2013-04&post=Detecting-TOR-Communication-in-Network-Traffic
For this reason nDPI uses a heuristic, non-DNS based, approach to detect tor communications. If possible, apps
should validate the certificate using the DNS. This is not something nDPI can afford to do for performance
reasons
+28 -12
View File
@@ -1,32 +1,48 @@
#!/bin/sh
rm -f configure config.h config.h.in
/bin/rm -f configure config.h config.h.in src/lib/Makefile.in
AUTOCONF=$(command -v autoconf)
AUTOMAKE=$(command -v automake)
LIBTOOL=$(command -v libtool)
LIBTOOLIZE=$(command -v libtoolize)
AUTORECONF=$(command -v autoreconf)
PKG_CONFIG=$(command -v pkg-config)
MAKE=$(command -v make)
AUTOCONF=$(which autoconf)
AUTOMAKE=$(which automake)
LIBTOOL=$(which libtool)
AUTORECONF=$(which autoreconf)
if test -z $AUTOCONF; then
if test -z "${AUTOCONF}"; then
echo "autoconf is missing: please install it and try again"
exit
fi
if test -z $AUTOMAKE; then
if test -z "${AUTOMAKE}"; then
echo "automake is missing: please install it and try again"
exit
fi
if test -z $LIBTOOL; then
echo "libtool is missing: please install it and try again"
if test -z "${LIBTOOL}" && test -z "${LIBTOOLIZE}"; then
echo "libtool and libtoolize is missing: please install it and try again"
exit
fi
if test -z $AUTORECONF; then
if test -z "${AUTORECONF}"; then
echo "autoreconf is missing: please install it and try again"
exit
fi
if test -z "${PKG_CONFIG}"; then
echo "pkg-config is missing: please install it (apt-get install pkg-config) and try again"
exit
fi
if test -z "${MAKE}"; then
echo "make is missing; please install it (apt-get install make) and try again"
exit
fi
autoreconf -ivf
./configure
echo "./configure $@"
chmod +x configure
./configure $@
+428 -51
View File
@@ -1,97 +1,474 @@
AC_INIT([libndpi], [1.6.0])
AC_INIT([libndpi],[4.9.0])
AC_CONFIG_AUX_DIR([.])
AC_CONFIG_MACRO_DIR([m4])
AM_INIT_AUTOMAKE([foreign subdir-objects])
AC_PREFIX_DEFAULT(/usr)
EXTRA_TARGETS="example tests tests/dga"
AC_ARG_WITH(only-libndpi, AS_HELP_STRING([--with-only-libndpi], [Build only libndpi (no examples, tests etc)]))
AS_IF([test "${with_only_libndpi+set}" = set],[
EXTRA_TARGETS=""
])
PWD=`pwd`
AC_ARG_WITH(sanitizer, AS_HELP_STRING([--with-sanitizer], [Build with support for address, undefined and leak sanitizer]))
AC_ARG_WITH(thread-sanitizer, AS_HELP_STRING([--with-thread-sanitizer], [Build with support for thread sanitizer]))
AC_ARG_WITH(memory-sanitizer, AS_HELP_STRING([--with-memory-sanitizer], [Build with support for memory sanitizer]))
AC_ARG_WITH(macos-memory-sanitizer, AS_HELP_STRING([--with-macos-memory-sanitizer], [Build with support for memory sanitizer macOS]))
AC_ARG_ENABLE(fuzztargets, AS_HELP_STRING([--enable-fuzztargets], [Enable fuzz targets]),[enable_fuzztargets=$enableval],[enable_fuzztargets=no])
AC_ARG_ENABLE(gprof, AS_HELP_STRING([--enable-gprof], [Enable CPU/HEAP profiling with gperftools]),[enable_gprof=$enableval],[enable_gprof=no])
AC_ARG_ENABLE(code-coverage, AS_HELP_STRING([--enable-code-coverage], [Generate Code Coverage report]))
AC_ARG_WITH(local-libgcrypt, AS_HELP_STRING([--with-local-libgcrypt], [Build with libgcrypt (if present) instead of the enclosed gcrypt light]))
AC_ARG_ENABLE(tls-sigs, AS_HELP_STRING([--enable-tls-sigs], [Enable TLS Client signature algorithm dissection. Rarely used, but requires significantly more memory.]))
AC_ARG_ENABLE(npcap, AS_HELP_STRING([--disable-npcap], [msys2 only: Disable linkage against the wpcap/npcap import library in windows/WpdPack/Lib.]))
AC_ARG_WITH(nbpf-path, AS_HELP_STRING([--with-nbpf-path], [nBPF library custom path; default: ${PWD}/../PF_RING/userland/nbpf]),[NBPF_HOME=$withval],[NBPF_HOME=${PWD}/../PF_RING/userland/nbpf])
AC_ARG_WITH(lto-and-gold-linker, AS_HELP_STRING([--with-lto-and-gold-linker], [Build with LTO and Gold linker]))
AC_ARG_ENABLE(debug-build, AS_HELP_STRING([--enable-debug-build], [Enable debug build (`-g` flag)]),[enable_debugbuild=$enableval],[enable_debugbuild=no])
AC_ARG_ENABLE(global-context-support, AS_HELP_STRING([--disable-global-context-support], [Disable support for global context. No external dependency on libpthread]))
NDPI_CFLAGS="${NDPI_CFLAGS} -D_DEFAULT_SOURCE=1 -D_GNU_SOURCE=1"
AS_IF([test "x$enable_fuzztargets" = "xyes"], [
BUILD_FUZZTARGETS=1
NDPI_CFLAGS="${NDPI_CFLAGS} -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION"
], [BUILD_FUZZTARGETS=0])
AM_CONDITIONAL([BUILD_FUZZTARGETS], [test "x$enable_fuzztargets" = "xyes"])
AS_IF([test "x$enable_debugbuild" = "xyes"], [
NDPI_CFLAGS="${NDPI_CFLAGS} -g"
])
AS_IF([test "${with_sanitizer+set}" = set -a "${with_thread_sanitizer+set}" = set],[
AC_MSG_ERROR([Configure options `--with-sanitizer' and `--with_thread_sanitizer' can not used at the same time.])
])
AS_IF([test "${with_sanitizer+set}" = set -a "${with_memory_sanitizer+set}" = set],[
AC_MSG_ERROR([Configure options `--with-sanitizer' and `--with_memory_sanitizer' can not used at the same time.])
])
AS_IF([test "${with_thread_sanitizer+set}" = set -a "${with_memory_sanitizer+set}" = set],[
AC_MSG_ERROR([Configure options `--with_thread_sanitizer' and `--with_memory_sanitizer' can not used at the same time.])
])
AS_IF([test "${with_sanitizer+set}" = set -o "${with_thread_sanitizer+set}" = set -o "${with_memory_sanitizer+set}" = set],[
NDPI_CFLAGS="${NDPI_CFLAGS} -O0 -g3"
],[
dnl> Oss-fuzz doesn't really like any optimizaton flags don't set by itself
AS_IF([test "x$enable_fuzztargets" != "xyes"], [NDPI_CFLAGS="${NDPI_CFLAGS} -O2"])
])
AS_IF([test "${with_sanitizer+set}" = set -o "${with_thread_sanitizer+set}" = set -o "${with_memory_sanitizer+set}" = set],[
AS_IF([test "x$enable_gprof" = "xyes"], [
AC_MSG_ERROR([Configure options `--with-sanitizer' / `--with_thread_sanitizer' / `--with_memory_sanitizer' can not used together with `--enable-gprof'.])
])
])
AS_IF([test "${with_sanitizer+set}" = set],[
NDPI_CFLAGS="${NDPI_CFLAGS} -fsanitize=address -fsanitize=undefined -fsanitize=leak -fno-omit-frame-pointer"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fsanitize=address -fsanitize=undefined -fsanitize=leak"
#Sanitizers should work on any compilers that we support (or that we test on CI, at least)
#Exception: "-fsanitize=alignment" is not supported in gcc 4.9
AX_CHECK_COMPILE_FLAG([-fno-sanitize=alignment], [
NDPI_CFLAGS="${NDPI_CFLAGS} -fno-sanitize=alignment"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fno-sanitize=alignment"
])
])
AS_IF([test "${with_thread_sanitizer+set}" = set],[
NDPI_CFLAGS="${NDPI_CFLAGS} -fsanitize=thread -fno-omit-frame-pointer"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fsanitize=thread"
])
AS_IF([test "${with_memory_sanitizer+set}" = set],[
NDPI_CFLAGS="${NDPI_CFLAGS} -fsanitize=memory -fsanitize-memory-track-origins -fno-omit-frame-pointer"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fsanitize=memory"
])
AS_IF([test "${with_macos_memory_sanitizer+set}" = set],[
NDPI_CFLAGS="${NDPI_CFLAGS} -fsanitize=address -fno-omit-frame-pointer -fsanitize=signed-integer-overflow -fno-sanitize-recover=address"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fsanitize=address"
])
AS_IF([test "x${enable_code_coverage}" = "xyes"],[
NDPI_CFLAGS="${NDPI_CFLAGS} -fprofile-arcs -ftest-coverage"
NDPI_LDFLAGS="${NDPI_LDFLAGS} --coverage"
])
AS_IF([test "${enable_npcap+set}" = set],[DISABLE_NPCAP=1],[DISABLE_NPCAP=0])
LT_INIT
LT_LIB_M
PKG_PROG_PKG_CONFIG
AC_PROG_CC
AX_PTHREAD
AC_PROG_CPP_WERROR
AC_C_INLINE
if test -d ".git"; then :
GIT_TAG=`git log -1 --format=%h`
GIT_DATE=`git log -1 --format=%cd`
SYSTEM=`uname -s`
if test $SYSTEM = "Darwin"; then
dnl> AC_PROG_CC(clang gcc)
AM_PROG_CC_C_O(clang gcc)
AC_PROG_CXX(clang++ g++)
else
AC_PROG_CC_C_O
AC_PROG_CXX
fi
dnl> Can't iuse AM_PROG_AR because not all of our Makefiles are automake (yet?)
AC_CHECK_TOOL(AR, ar, [false])
AC_LANG_WERROR
AC_MSG_CHECKING([whether roaring v3 works])
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#if __has_include (<stdio.h>)
#endif
]])],
[
AC_MSG_RESULT([yes])
],
[
AC_MSG_RESULT([no])
AC_DEFINE_UNQUOTED(USE_ROARING_V2, "1", [Use CRoaring 2.1.x])
])
GCC_VERSION=`gcc --version | cut -d ' ' -f 3 | head -1|cut -d '.' -f 1`
if test "${GCC_VERSION}" == "version" ; then
GCC_VERSION=`gcc --version | cut -d ' ' -f 4 | head -1|cut -d '.' -f 1`
fi
if test "${GCC_VERSION}" -lt "7" ; then
AC_DEFINE_UNQUOTED(USE_ROARING_V2, "1", [Use CRoaring 2.1.x])
fi
NDPI_MAJOR=`echo "${PACKAGE_VERSION}" | cut -d . -f 1`
NDPI_MINOR=`echo "${PACKAGE_VERSION}" | cut -d . -f 2`
NDPI_PATCH=`echo "${PACKAGE_VERSION}" | cut -d . -f 3`
NDPI_VERSION_SHORT="$NDPI_MAJOR.$NDPI_MINOR.$NDPI_PATCH"
AC_DEFINE_UNQUOTED(NDPI_MAJOR_RELEASE, "${NDPI_MAJOR}", [nDPI major release])
AC_DEFINE_UNQUOTED(NDPI_MINOR_RELEASE, "${NDPI_MINOR}", [nDPI minor release])
AC_DEFINE_UNQUOTED(NDPI_PATCH_LEVEL, "${NDPI_PATCH}", [nDPI patch level])
# .git as directory in a cloned repo
# .git as file in submodule based integration
if test -d ".git" || test -r ".git"; then
GIT_TAG=`git log -1 --abbrev=7 --format=%h`
GIT_DATE=`git log -1 --abbrev=7 --format=%cd`
#
# On CentOS 6 `git rev-list HEAD --count` does not work
#
#
GIT_NUM=`git log --pretty=oneline | wc -l`
GIT_BRANCH=`git rev-parse --abbrev-ref HEAD`
GIT_RELEASE="${PACKAGE_VERSION}-${GIT_BRANCH}-${GIT_NUM}-${GIT_TAG}"
#
GIT_NUM=`git log --pretty=oneline | wc -l | tr -d '[[:space:]]'`
GIT_RELEASE="${PACKAGE_VERSION}-${GIT_NUM}-${GIT_TAG}"
A=`git log src/include/ndpi_typedefs.h|wc -l`
B=`git log src/include/ndpi_protocol_ids.h|wc -l`
C=`git log src/include/ndpi_api.h|wc -l`
NDPI_API_VERSION=$((A+B+C))
else
GIT_RELEASE="${PACKAGE_VERSION}"
GIT_DATE=`date`
GIT_DATE=`date -u -r CHANGELOG.md`
if test -z "$SOURCE_DATE_EPOCH" ; then :
SOURCE_DATE_EPOCH=`date +%s`
fi
NDPI_API_VERSION=$((SOURCE_DATE_EPOCH % 65536))
fi
AS_IF([test "x${NDPI_API_VERSION}" = "x"],[
AC_MSG_WARN([Could not get NDPI_API_VERSION, defaulting to 0.])
NDPI_API_VERSION=0
])
echo "Setting API version to ${NDPI_API_VERSION}"
AC_DEFINE_UNQUOTED(NDPI_GIT_RELEASE, "${GIT_RELEASE}", [GIT Release])
AC_DEFINE_UNQUOTED(NDPI_GIT_DATE, "${GIT_DATE}", [Last GIT change])
AC_CHECK_HEADERS([netinet/in.h stdint.h stdlib.h string.h unistd.h])
if ! test "${with_only_libndpi+set}" = set; then :
dnl> used by json-c for unit tests
PKG_CHECK_MODULES([JSONC], [json-c], [
AC_DEFINE(HAVE_LIBJSON_C, 1, [libjson-c is present])
JSONC_LIBS="${pkg_cv_JSONC_LIBS}"
JSONC_CFLAGS="${pkg_cv_JSONC_CFLAGS}"
], [AC_MSG_WARN([JSON-C not available. Disabled unit test.])])
AC_CHECK_LIB([json-c], [json_object_put], [
EXTRA_TARGETS="$EXTRA_TARGETS tests/unit"
build_unittests=yes
],[
AC_MSG_WARN([JSON-C not available. Disabled unit test.])
JSONC_LIBS=""
JSONC_CFLAGS=""
])
dnl> AX_PTHREAD test does not work along with MingW/MSYS anymore
case "$host" in
*-*-mingw32*|*-*-msys)
;;
*)
AX_PTHREAD([],[
AC_MSG_WARN([POSIX Threads not available. Building library only.])
JSONC_LIBS=""
JSONC_CFLAGS=""
EXTRA_TARGETS=""
])
;;
esac
fi
AM_CONDITIONAL([BUILD_UNITTESTS], [test "x$build_unittests" = "xyes"])
HANDLE_TLS_SIGS="//"
AS_IF([test "x${enable_tls_sigs}" = "xyes"],[
HANDLE_TLS_SIGS=""
])
NDPI_CFLAGS="-W -Wall -Wno-address-of-packed-member ${NDPI_CFLAGS}"
AS_IF([test "${with_lto_and_gold_linker+set}" = set], [
NDPI_CFLAGS="${NDPI_CFLAGS} -flto -fuse-ld=gold -Wno-unused-command-line-argument"
NDPI_LDFLAGS="${NDPI_LDFLAGS} ${NDPI_CFLAGS}"
])
AC_CHECK_HEADERS([netinet/in.h stdint.h stdlib.h string.h unistd.h math.h float.h])
AC_CHECK_LIB([m], [sqrt], [], [LIBM="-lm"])
AC_CHECK_LIB([rrd], [rrd_fetch_r], [LIBRRD=-lrrd])
if test "x$ac_cv_lib_rrd_rrd_fetch_r" = xyes; then :
AC_DEFINE_UNQUOTED(HAVE_RRDTOOL, "1", [rrdtool is present])
EXTRA_TARGETS="$EXTRA_TARGETS rrdtool"
fi
GLOBAL_CONTEXT_ENABLED=0
AS_IF([test "x$enable_global_context_support" != "xno"], [
AC_CHECK_LIB([pthread], [pthread_mutex_init])
NDPI_CFLAGS="${NDPI_CFLAGS} -DUSE_GLOBAL_CONTEXT"
GLOBAL_CONTEXT_ENABLED=1
])
ADDITIONAL_INCS=
ADDITIONAL_LIBS="$LIBM"
PCAP_HOME=$HOME/PF_RING/userland
if test -d $PCAP_HOME; then :
echo -n ""
DPDK_TARGET=
AC_MSG_CHECKING([DPDK (used by ndpiReader)])
if test -d $HOME/DPDK; then :
AC_MSG_RESULT(yes)
DPDK_TARGET=dpdk
else
AC_MSG_RESULT([no (missing $HOME/DPDK)])
fi
if ! test -d $PCAP_HOME; then :
PCAP_HOME=`pwd`/../../PF_RING/userland
fi
SHORT_MACHINE=`uname -m | cut -b1-3`
if test $SHORT_MACHINE = "arm"; then
LIBNUMA=""
else
LIBNUMA="-lnuma"
AC_CHECK_LIB([numa], [numa_available], [LIBNUMA="-lnuma"])
fi
if test -f $PCAP_HOME/libpcap/libpcap.a; then :
echo "Using libpcap from $PCAP_HOME"
PCAP_INC="-I $PCAP_HOME/libpcap"
PCAP_LIB="$PCAP_HOME/libpcap/libpcap.a $PCAP_HOME/lib/libpfring.a $LIBNUMA"
MACHINE=`uname -m`
AC_CHECK_LIB([rt], [clock_gettime], [PCAP_LIB="$PCAP_LIB -lrt"])
AC_CHECK_LIB([nl], [nl_handle_alloc], [PCAP_LIB="$PCAP_LIB -lnl"])
NBPF_ENABLED=0
AC_MSG_CHECKING([PF_RING nBPF ($NBPF_HOME)])
LIBNBPF=${NBPF_HOME}/libnbpf.a
if test -f ${LIBNBPF}; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} ${LIBNBPF}"
ADDITIONAL_INCS="${ADDITIONAL_INCS} -I${NBPF_HOME}"
AC_DEFINE_UNQUOTED(HAVE_NBPF, "1", [PF_RING nBPF is present])
AC_MSG_RESULT(yes)
NBPF_ENABLED=1
else
AC_CHECK_LIB([pcap], [pcap_open_live], [PCAP_LIB="-lpcap"])
if test $ac_cv_lib_pcap_pcap_open_live = "no"; then :
echo ""
echo "ERROR: Missing libpcap(-dev) library required to compile the example application"
echo "ERROR: Please install it and try again"
exit
fi
AC_MSG_RESULT(no)
fi
PKG_CONFIG=$(which pkg-config)
if test -d /usr/local/include/json-c/; then :
CFLAGS="$CFLAGS -I/usr/local/include/json-c/"
LDFLAGS="$LDFLAGS -L/usr/local/lib -ljson-c"
else
if ! test -z "$PKG_CONFIG"; then :
CFLAGS="$CFLAGS $(pkg-config --cflags json-c)"
LDFLAGS="$LDFLAGS $(pkg-config --libs json-c)"
fi
CUSTOM_NDPI=
if test -d ../nDPI-custom; then :
CUSTOM_NDPI="-DCUSTOM_NDPI_PROTOCOLS"
AC_MSG_RESULT([Compiling with custom nDPI protocols])
fi
OLD_LIBS=$LIBS
LIBS="-L/opt/napatech3/lib $LIBS"
AC_CHECK_LIB([ntapi],
[NT_Init],
[PCAP_LIB="$PCAP_LIB -L/opt/napatech3/lib -lntapi"],
[], [] )
LIBS=$OLD_LIBS
case "$host" in
*-*-mingw32*|*-*-msys)
PCAP_INC=""
PCAP_LIB=""
case "$host" in
x86_64-*)
BUILD_MINGW_X64=1
NDPI_LDFLAGS="${NDPI_LDFLAGS} -L/mingw64/lib"
;;
i686-*)
NDPI_LDFLAGS="${NDPI_LDFLAGS} -L/mingw32/lib"
;;
*)
AC_MSG_ERROR([Unsupported MingW cross compiler.])
;;
esac
NDPI_CFLAGS="-D__USE_MINGW_ANSI_STDIO -D__STDC_FORMAT_MACROS ${NDPI_CFLAGS}"
LIBS="${LIBS} -lws2_32"
BUILD_MINGW=1
EXE_SUFFIX=".exe"
AS_IF([test "${enable_npcap+set}" != set && test "${with_only_libndpi+set}" != set],,
[PKG_CHECK_MODULES([PCAP], [libpcap], [PCAP_LIB="" PCAP_INC="${pkg_cv_PCAP_CFLAGS}"])
AC_CHECK_LIB([pcap], [pcap_open_live],, [AC_MSG_ERROR([Missing msys2/mingw libpcap library. Install it with `pacman -S mingw-w64-x86_64-libpcap' (msys2).])])])
;;
*)
if test -f $PCAP_HOME/libpcap/libpcap.a; then :
echo "Using libpcap from $PCAP_HOME"
PCAP_INC="-I $PCAP_HOME/libpcap"
PFRING_LIB=
if test -f $PCAP_HOME/lib/libpfring.a; then :
PFRING_LIB=$PCAP_HOME/lib/libpfring.a
fi
PCAP_LIB="$PCAP_HOME/libpcap/libpcap.a $PFRING_LIB $LIBNUMA `$PCAP_HOME/lib/pfring_config --libs`"
AC_CHECK_LIB([rt], [clock_gettime], [PCAP_LIB="$PCAP_LIB -lrt"])
AC_CHECK_LIB([nl], [nl_handle_alloc], [PCAP_LIB="$PCAP_LIB -lnl"])
# The dlopen() function is in libdl on GLIBC-based systems
# and in the C library for *BSD systems
AC_CHECK_LIB([dl], [dlopen], [DL_LIB="-ldl"],[AC_CHECK_LIB([c],
[dlopen], [DL_LIB="-lc"],
[AC_MSG_ERROR([unable to find the dlopen(), dlsym() functions]) ]) ])
elif test "${with_only_libndpi+set}" != set; then :
AC_CHECK_LIB([pcap], [pcap_open_live], [PCAP_LIB="-lpcap"])
if test $ac_cv_lib_pcap_pcap_open_live = "no"; then :
AC_MSG_ERROR([Missing libpcap(-dev) library required to compile the example application.])
fi
fi
EXE_SUFFIX=""
AS_IF([test "${enable_npcap+set}" = set],AC_MSG_ERROR([--disable-npcap does only work with msys2/mingw CCs]))
;;
esac
AC_CHECK_LIB(json-c, json_object_new_object, AC_DEFINE_UNQUOTED(HAVE_JSON_C, 1, [The JSON-C library is present]))
AC_ARG_ENABLE([debug-messages],
AS_HELP_STRING([--enable-debug-messages], [Define NDPI_ENABLE_DEBUG_MESSAGES=1]), [
AC_DEFINE(NDPI_ENABLE_DEBUG_MESSAGES, 1, [Enable ndpi_debug_messages]) ])
AS_IF([test "x$enable_fuzztargets" = "xyes"], [
AC_PROG_CXX
AC_LANG_PUSH(C++)
tmp_saved_flags=$[]_AC_LANG_PREFIX[]FLAGS
AX_CHECK_COMPILE_FLAG([-fsanitize=fuzzer],,
[AC_MSG_ERROR([--enable-fuzztargets requires -fsanitize=fuzzer which is only supported by LLVM])],
[-Werror])
AS_IF([test "x$LIB_FUZZING_ENGINE" = "x"], [
LIB_FUZZING_ENGINE=-fsanitize=fuzzer
AC_SUBST(LIB_FUZZING_ENGINE)
])
_AC_LANG_PREFIX[]FLAGS="$[]_AC_LANG_PREFIX[]FLAGS $LIB_FUZZING_ENGINE"
AC_MSG_CHECKING([whether $CXX accepts $LIB_FUZZING_ENGINE])
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#include <sys/types.h>
extern "C" int LLVMFuzzerTestOneInput(const unsigned char *Data, size_t Size);
extern "C" int LLVMFuzzerTestOneInput(const unsigned char *Data, size_t Size) {
(void)Data;
(void)Size;
return 0;
}
]])],
[ AC_MSG_RESULT(yes)
has_sanitizefuzzer=yes],
[ AC_MSG_RESULT(no) ]
)
_AC_LANG_PREFIX[]FLAGS=$tmp_saved_flags
AC_LANG_POP()
])
AM_CONDITIONAL([HAS_FUZZLDFLAGS], [test "x$has_sanitizefuzzer" = "xyes"])
GPROF_ENABLED=0
AS_IF([test "x$enable_gprof" = "xyes"], [
PKG_CHECK_MODULES([PROFILER], [libprofiler],,[
AC_MSG_ERROR([libprofiler not available. Required for profiling support.])
])
PKG_CHECK_MODULES([TCMALLOC], [libtcmalloc],,[
AC_MSG_ERROR([libtcmalloc not available. Required for profiling support.])
])
GPROF_ENABLED=1
GPROF_LIBS="-ltcmalloc_and_profiler"
GPROF_CFLAGS="-g3 -fvisibility=default -fno-omit-frame-pointer ${pkg_cv_PROFILER_CFLAGS} ${pkg_cv_TCMALLOC_CFLAGS}"
])
AC_CHECK_LIB(pthread, pthread_setaffinity_np, AC_DEFINE_UNQUOTED(HAVE_PTHREAD_SETAFFINITY_NP, 1, [libc has pthread_setaffinity_np]))
AC_CONFIG_FILES([Makefile src/lib/Makefile example/Makefile libndpi.pc])
AC_CONFIG_HEADERS(config.h)
dnl> libgcrypt (external)
USE_HOST_LIBGCRYPT=0
AS_IF([test "${with_local_libgcrypt+set}" = set],[
USE_HOST_LIBGCRYPT=1
AC_CHECK_LIB(gpg-error, gpg_strerror_r, [], AC_MSG_ERROR([libgpg-error required (because of --with-local-libgcrypt) but not found or too old.]))
AC_CHECK_LIB(gcrypt, gcry_cipher_checktag, [], AC_MSG_ERROR([libgcrypt required (because of --with-local-libgcrypt) but not found or too old.]))
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lgcrypt -lgpg-error"
AC_DEFINE_UNQUOTED(USE_HOST_LIBGCRYPT, 1, [Use locally installed libgcrypt instead of builtin gcrypt-light])
])
dnl> PCRE2
PCRE2_ENABLED=0
AC_ARG_WITH(pcre2, AS_HELP_STRING([--with-pcre2], [Enable nDPI build with libpcre2]))
if test "${with_pcre2+set}" = set; then :
AC_CHECK_LIB(pcre2-8, pcre2_compile_8, AC_DEFINE_UNQUOTED(HAVE_PCRE2, 1, [libpcre2(-dev) is present]))
if test "x$ac_cv_lib_pcre2_8_pcre2_compile_8" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lpcre2-8"
PCRE2_ENABLED=1
fi
fi
dnl> GeoIP
AC_ARG_WITH(maxminddb, AS_HELP_STRING([--with-maxminddb], [Enable nDPI build with libmaxminddb]))
if test "${with_maxminddb+set}" = set; then :
AC_CHECK_LIB([maxminddb], [MMDB_lookup_sockaddr])
AC_CHECK_HEADERS([maxminddb.h])
if test ".${ac_cv_lib_maxminddb_MMDB_lookup_sockaddr}" = ".yes" && test ".${ac_cv_header_maxminddb_h}" = ".yes"; then
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lmaxminddb"
AC_DEFINE_UNQUOTED(HAVE_MAXMINDDB, 1, [MaxMind DB support])
fi
fi
AC_DEFINE_UNQUOTED(NDPI_BASE_DIR, "${PWD}", [nDPI base directory])
dnl> Curl
dnl> AC_CHECK_LIB([curl], [curl_easy_perform])
dnl> if test ${ac_cv_lib_curl_curl_easy_perform} = "no"; then
dnl> echo "Please install libcurl4-openssl-dev (http://curl.haxx.se/) in order to enable curl support"
dnl> else
dnl> ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lcurl"
dnl> AC_DEFINE_UNQUOTED(HAVE_CURL, 1, [curl is present])
dnl> fi
AC_CONFIG_FILES([Makefile example/Makefile example/Makefile.dpdk tests/Makefile tests/unit/Makefile tests/performance/Makefile tests/dga/Makefile rrdtool/Makefile influxdb/Makefile libndpi.pc src/include/ndpi_define.h src/lib/Makefile fuzz/Makefile doc/Doxyfile.cfg])
AC_CONFIG_FILES([tests/do.sh], [chmod +x tests/do.sh])
AC_CONFIG_HEADERS(src/include/ndpi_config.h)
AC_SUBST(GIT_RELEASE)
AC_SUBST(SVN_DATE)
AC_SUBST(JSON_C_LIB)
AC_SUBST(NDPI_MAJOR)
AC_SUBST(NDPI_MINOR)
AC_SUBST(NDPI_PATCH)
AC_SUBST(NDPI_VERSION_SHORT)
AC_SUBST(PCAP_INC)
AC_SUBST(PCAP_LIB)
AC_SUBST(LIBRRD)
AC_SUBST(ADDITIONAL_INCS)
AC_SUBST(ADDITIONAL_LIBS)
AC_SUBST(DL_LIB)
AC_SUBST(DPDK_TARGET)
AC_SUBST(HAVE_PTHREAD_SETAFFINITY_NP)
AC_SUBST(CUSTOM_NDPI)
AC_SUBST(NDPI_API_VERSION)
AC_SUBST(EXTRA_TARGETS)
AC_SUBST(BUILD_MINGW)
AC_SUBST(BUILD_MINGW_X64)
AC_SUBST(BUILD_FUZZTARGETS)
AC_SUBST(JSONC_CFLAGS)
AC_SUBST(JSONC_LIBS)
AC_SUBST(GPROF_CFLAGS)
AC_SUBST(GPROF_LIBS)
AC_SUBST(GPROF_ENABLED)
AC_SUBST(USE_HOST_LIBGCRYPT)
AC_SUBST(PCRE2_ENABLED)
AC_SUBST(NBPF_ENABLED)
AC_SUBST(GLOBAL_CONTEXT_ENABLED)
AC_SUBST(HANDLE_TLS_SIGS)
AC_SUBST(DISABLE_NPCAP)
AC_SUBST(EXE_SUFFIX)
AC_SUBST(NDPI_CFLAGS)
AC_SUBST(NDPI_LDFLAGS)
AC_SUBST(NDPI_BASE_DIR)
AC_OUTPUT
+2658
View File
File diff suppressed because it is too large Load Diff
+36
View File
@@ -0,0 +1,36 @@
# FAQ
From https://www.ntop.org/ndpi/ndpi-internals-and-frequent-questions/
**Q**: How nDPI implements protocol detection?\
**A**: nDPI includes a list of protocol dissectors (364 as of today) that are able to dissect protocols such as WhatsApp or TLS. As soon as a new flow is submitted to nDPI, the library applies in sequence dissectors that can potentially match the protocols (i.e. telnet is a TCP-based protocol and it will not be considered for UDP flows). We start from the dissector that can most probably match using the port number. This means for traffic on TCP/22 nDPI will start with the SSH dissectors and if not matching continue with the others. Dissection completes as soon as a protocol matches or when none of them matched and in this case the flow will be labelled as Unknown.
**Q**: What is the nDPI release cycle?\
**A**: We cut release a approximately every 6-8 months, fixes and improvements are on a daily basis (check the nDPI code on GitHub).
**Q**: Is nDPI running on all popular platforms?\
**A**: Yes it runs on Linux, macOS, Windows… and also on not-so-popular ones such as IBM mainframes. We support ARM, Intel, RISC… architectures.
**Q**: How many packets does nDPI need in order to implement detection?\
**A**: It depends on the protocol. For UDP-based protocols such as DNS one packet is enough, for more complex protocols such as TLS about 10 packets. For sure if after 15-20 packets nDPI has not detected the application protocol, then the protocol is labelled as Unknown.
**Q**: Is nDPI detection only based on protocol dissectors?\
**A**: No, payload inspection is the main technique, but nDPI can also use IP address, ports, TLS certificates etc as signatures for protocols. In this case, after detection is complete, nDPI will report if the match was performed on payload inspection or other means (e.g. IP address).
**Q**: Does nDPI contain list of known IP addresses?\
**A**: Yes it includes lists of well known IPs such as those provided by Microsoft of Meta for identifying known service.
**Q**: Can I extend nDPI by defining new protocols with a configuration file?\
**A**: Yes you can. See this [file](https://github.com/ntop/nDPI/blob/dev/example/protos.txt) as an example for defining new protocols..
**Q**: Is nDPI able to detect VPNs?\
**A**: Yes it can detect VPNS such as Tailscale, WireGuard, OpenVPN, FortiClient.. and also in-app VPNs such as UltraSurf or OperaVPN.
**Q**: Is nDPI able to detect malware and viruses?\
**A**: It can detect anomalous behaviour that can be caused by a malware, but nDPI is not a signature-based tool so it does not include signatures for malware A or B. This is because [signature-based tools](https://en.wikipedia.org/wiki/Intrusion_detection_system) have various limitations and resource intensive, whereas nDPI has been designed to be used also in high-speed (100 Gbit+) networks.
**Q**: Is nDPI able to detect security issues?\
**A**: Yes it can by means of a technique called [flow risk](https://github.com/ntop/nDPI/blob/dev/doc/flow_risks.rst). It can identify 50+ threats (e.g. a host that is talking with a malware host).
**Q**: Is nDPI able to block traffic?\
**A**: No, nDPI is a passive traffic analysis library that does not manipulate packets. You can create applications on top of it for policing (i.e. blocking or shaping) traffic. Examples of such applications are ntopng Edge, nProbe IPS and nProbe Cento.
+234
View File
@@ -0,0 +1,234 @@
# Makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
# User-friendly check for sphinx-build
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
endif
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " release to make standalone Doxygen/HTML files for ntop documentation release"
@echo " html to make standalone Doxygen/HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " singlehtml to make a single large HTML file"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " qthelp to make HTML files and a qthelp project"
@echo " applehelp to make an Apple Help Book"
@echo " devhelp to make HTML files and a Devhelp project"
@echo " epub to make an epub"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " latexpdf to make LaTeX files and run them through pdflatex"
@echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
@echo " text to make text files"
@echo " man to make manual pages"
@echo " texinfo to make Texinfo files"
@echo " info to make Texinfo files and run them through makeinfo"
@echo " gettext to make PO message catalogs"
@echo " changes to make an overview of all changed/added/deprecated items"
@echo " xml to make Docutils-native XML files"
@echo " pseudoxml to make pseudoxml-XML files for display purposes"
@echo " linkcheck to check all external links for integrity"
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
@echo " coverage to run coverage check of the documentation (if enabled)"
.PHONY: clean
clean:
rm -rf $(BUILDDIR)/* api doxygen
.PHONY: html
html:
@test -r Doxyfile.cfg || { \
echo 'Doxyfile.cfg does not exist; Please run $(realpath ..)/autogen.sh or $(realpath ..)/configure first.'; \
false; \
}
doxygen Doxyfile.cfg
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
.PHONY: release
release:
make clean
@{ \
git_root=$$(git rev-parse --show-toplevel);\
version=$$(grep -Po '^NTOPNG_SHORT_VERSION="\K.+(?=")' $${git_root}/configure.ac);\
echo Release version $${version};\
sed -i "s/version = u'.*'/version = u'$${version}'/" conf.py;\
sed -i "s/release = u'.*'/release = u'$${version}'/" conf.py;\
}
make html
.PHONY: dirhtml
dirhtml:
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
.PHONY: singlehtml
singlehtml:
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
@echo
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
.PHONY: pickle
pickle:
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
@echo
@echo "Build finished; now you can process the pickle files."
.PHONY: json
json:
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
@echo
@echo "Build finished; now you can process the JSON files."
.PHONY: htmlhelp
htmlhelp:
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
@echo
@echo "Build finished; now you can run HTML Help Workshop with the" \
".hhp project file in $(BUILDDIR)/htmlhelp."
.PHONY: qthelp
qthelp:
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
@echo
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/ntop.qhcp"
@echo "To view the help file:"
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/ntop.qhc"
.PHONY: applehelp
applehelp:
$(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp
@echo
@echo "Build finished. The help book is in $(BUILDDIR)/applehelp."
@echo "N.B. You won't be able to view it unless you put it in" \
"~/Library/Documentation/Help or install it in your application" \
"bundle."
.PHONY: devhelp
devhelp:
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
@echo
@echo "Build finished."
@echo "To view the help file:"
@echo "# mkdir -p $$HOME/.local/share/devhelp/ntop"
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/ntop"
@echo "# devhelp"
.PHONY: epub
epub:
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
@echo
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
.PHONY: latex
latex:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
@echo "Run \`make' in that directory to run these through (pdf)latex" \
"(use \`make latexpdf' here to do that automatically)."
.PHONY: latexpdf
latexpdf:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through pdflatex..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
.PHONY: latexpdfja
latexpdfja:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through platex and dvipdfmx..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
.PHONY: text
text:
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
@echo
@echo "Build finished. The text files are in $(BUILDDIR)/text."
.PHONY: man
man:
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
@echo
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
.PHONY: texinfo
texinfo:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
@echo "Run \`make' in that directory to run these through makeinfo" \
"(use \`make info' here to do that automatically)."
.PHONY: info
info:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo "Running Texinfo files through makeinfo..."
make -C $(BUILDDIR)/texinfo info
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
.PHONY: gettext
gettext:
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
@echo
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
.PHONY: changes
changes:
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
@echo
@echo "The overview file is in $(BUILDDIR)/changes."
.PHONY: linkcheck
linkcheck:
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or in $(BUILDDIR)/linkcheck/output.txt."
.PHONY: doctest
doctest:
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
@echo "Testing of doctests in the sources finished, look at the " \
"results in $(BUILDDIR)/doctest/output.txt."
.PHONY: coverage
coverage:
$(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage
@echo "Testing of coverage in the sources finished, look at the " \
"results in $(BUILDDIR)/coverage/python.txt."
.PHONY: xml
xml:
$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
@echo
@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
.PHONY: pseudoxml
pseudoxml:
$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
@echo
@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."
+304
View File
@@ -0,0 +1,304 @@
# -*- coding: utf-8 -*-
import sys
import os
import sphinx_rtd_theme
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#sys.path.insert(0, os.path.abspath('.'))
sys.path.append( "breathe/" )
# -- General configuration ------------------------------------------------
# If documentation needs a minimal Sphinx version, state it here.
#needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.intersphinx',
'breathe',
'exhale'
]
# Workaround for platforms where swaggerdoc is not available
if not os.environ.get("PYTHON_SKIP_SWAGGERDOC"):
extensions.append('sphinxcontrib.swaggerdoc')
breathe_projects = { "apidoc" : "doxygen/xml/" }
breathe_default_project = "apidoc"
exhale_args = {
"containmentFolder": "./api",
"rootFileName": "library_root.rst",
"rootFileTitle": "Library API",
"doxygenStripFromPath": ".",
"createTreeView": True,
# "treeViewIsBootstrap": True,
}
primary_domain = 'c'
highlight_language = 'c'
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
source_suffix = ['.rst', '.md']
#source_suffix = '.rst'
# The encoding of source files.
#source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'nDPI'
copyright = u'2021, ntop and contributors'
author = u'ntop.org'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = u'4.1'
# The full version, including alpha/beta/rc tags.
release = u'4.1'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ['_build', 'README.md', 'example.rst']
# The reST default role (used for this markup: `text`) to use for all
# documents.
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
#add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
#show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
#modindex_common_prefix = []
# If true, keep warnings as "system message" paragraphs in the built documents.
#keep_warnings = False
# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = False
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#html_theme = 'alabaster'
html_theme = "sphinx_rtd_theme"
#html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#html_theme_options = {}
# Add any paths that contain custom themes here, relative to this directory.
#html_theme_path = []
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
#html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
html_logo = "img/logo.png"
# The name of an image file (relative to this directory) to use as a favicon of
# the docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
#html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
#html_static_path = ['_static']
# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
# directly to the root of the documentation.
#html_extra_path = []
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
#html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}
# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# If false, no module index is generated.
#html_domain_indices = True
# If false, no index is generated.
#html_use_index = True
# If true, the index is split into individual pages for each letter.
#html_split_index = False
# If true, links to the reST sources are added to the pages.
#html_show_sourcelink = True
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
#html_show_sphinx = True
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
#html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = None
# Language to be used for generating the HTML full-text search index.
# Sphinx supports the following languages:
# 'da', 'de', 'en', 'es', 'fi', 'fr', 'hu', 'it', 'ja'
# 'nl', 'no', 'pt', 'ro', 'ru', 'sv', 'tr'
#html_search_language = 'en'
# A dictionary with options for the search language support, empty by default.
# Now only 'ja' uses this config value
#html_search_options = {'type': 'default'}
# The name of a javascript file (relative to the configuration directory) that
# implements a search results scorer. If empty, the default will be used.
#html_search_scorer = 'scorer.js'
# Output file base name for HTML help builder.
htmlhelp_basename = 'ntopdoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
# Latex figure (float) alignment
#'figure_align': 'htbp',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'nDPI.tex', u'nDPI Documentation',
u'ntop', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
#latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# If true, show page references after internal links.
#latex_show_pagerefs = False
# If true, show URL addresses after external links.
#latex_show_urls = False
# Documents to append as an appendix to all manuals.
#latex_appendices = []
# If false, no module index is generated.
#latex_domain_indices = True
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'nDPI', u'nDPI Documentation',
[author], 1)
]
# If true, show URL addresses after external links.
#man_show_urls = False
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'nDPI', u'nDPI Documentation',
author, 'ntop and contributors', 'Deep Packet Inspection toolkit',
'Miscellaneous'),
]
# Documents to append as an appendix to all manuals.
#texinfo_appendices = []
# If false, no module index is generated.
#texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
#texinfo_show_urls = 'footnote'
# If true, do not generate a @detailmenu in the "Top" node's menu.
#texinfo_no_detailmenu = False
intersphinx_mapping = {'https://docs.python.org/': None}
def setup(app):
app.add_css_file('css/ntop.css')
+49
View File
@@ -0,0 +1,49 @@
# Configuration knobs
TODO
| Protocol | Parameter | Default value | Min value | Max value | Description | Notes |
| ------ | ------ | ------ | ------ | ------ | ------ | ------ |
| NULL | "packets_limit_per_flow" | 32 | 0 | 255 | The upper limit on the number of packets per flow that will be subject to DPI, after which classification will be considered complete (0 = no limit) |
| NULL | "flow.direction_detection" | enable | NULL | NULL | Enable/disable internal detection of packet direction (client to server or server to client) |
| NULL | "flow.track_payload" | disable | NULL | NULL | Enable/disable tracking/export of flow payload (i.e. L5/7 data): if enabled, the library exports the first 1024 bytes of payload for each flow |
| NULL | "tcp_ack_payload_heuristic" | disable | NULL | NULL | In some networks, there are some anomalous TCP flows where the smallest ACK packets have some kind of zero padding. It looks like the IP and TCP headers in those frames wrongly consider the 0x00 Ethernet padding bytes as part of the TCP payload. While this kind of packets is perfectly valid per-se, in some conditions they might be treated by the TCP reassembler logic as (partial) overlaps, deceiving the classification engine. This parameter enable/disable an heuristic to detect these packets and to ignore them, allowing correct detection/classification. See #1946 for other details |
| NULL | "fully_encrypted_heuristic" | enable | NULL | NULL | Enable/disable an heuristic to detect fully encrypted sessions, i.e. flows where every bytes of the payload is encrypted in an attempt to “look like nothing”. This heuristic only analyzes the first packet of the flow. See: https://www.usenix.org/system/files/sec23fall-prepub-234-wu-mingshi.pdf |
| NULL | "libgcrypt.init" | 1 | NULL | NULL | Enable/disable initialization of libgcrypt. When using the external libgcrypt (instead of the internal crypto code) the libgcrypt runtime must be initialized. If, for whatever reasons, the application alread does it, nDPI must be told to skip it. Note that, by default, nDPI uses the crypto code and not libgcrypt: in that case this parameter is ignored |
| NULL | "dpi.compute_entropy" | 1 | NULL | NULL | Enable/disable computation of flow entropy |
| NULL | "dpi.guess_on_giveup" | 0x03 | 0x00 | 0x03 | Tell the library to guess flow classification, if any DPI algorithms/logics fail. The value is a bitmask. Values: 0x0 = disabled; 0x01 = enable guessing by port; 0x02 = enable guessing by ip |
| NULL | "flow_risk_lists.load" | 1 | NULL | NULL | Enable/disable loading of every IP addresses lists used to check any flow risks |
| NULL | "flow_risk.anonymous_subscriber.list.icloudprivaterelay.load" | 1 | NULL | NULL | Enable/disable loading of internal iCouldPrivateRealy IP address list used to check `NDPI_ANONYMOUS_SUBSCRIBER` flow risk |
| NULL | "flow_risk.anonymous_subscriber.list.protonvpn.load" | 1 | NULL | NULL | Enable/disable loading of internal IP address list of ProtonVPN exit nodes used to check `NDPI_ANONYMOUS_SUBSCRIBER` flow risk |
| NULL | "flow_risk.crawler_bot.list.load" | 1 | NULL | NULL | Enable/disable loading of internal IP address list used to check `NDPI_HTTP_CRAWLER_BOT` flow risk |
| NULL | "filename.config" | NULL | NULL | NULL | Name of the file containing a list of configuration knobs itself (one per line)!. Useful to configure nDPI via text file instead of via API |
| NULL | "log.level" | 0 | 0 | 3 | Configure the log/debug level. Possible values: 0 = error, 1 = trace, 2 = debug, 3 = extra debug |
| NULL | "lru.$CACHE_NAME.size" | See description | 0 | 16777215 | Set the size (in number of elements) of the specified LRU cache (0 = the cache is disabled). The keyword "$CACHE_NAME" is a placeholder for the cache name and the possible values are: ookla, bittorrent, stun, tls_cert, mining, msteams. The default value is "32768" for the bittorrent cache and "1024" for all the other caches |
| NULL | "lru.$CACHE_NAME.ttl" | See description | 0 | 16777215 | Set the TTL (in seconds) for the elements of the specified LRU cache (0 = the elements never explicitly expire). The keyword "$CACHE_NAME" is a placeholder for the cache name and the possible values are: ookla, bittorrent, stun, tls_cert, mining, msteams. The default value is "120" for the ookla cache, "60" for the msteams cache and "0" for all the other caches |
| NULL | "lru.$CACHE_NAME.scope" | 0 | 0 | 1 | Set the scope of the specified LRU cache (0 = the cache is local, 1 = the cache is global). The keyword "$CACHE_NAME" is a placeholder for the cache name and the possible values are: ookla, bittorrent, stun, tls_cert, mining, msteams. The global scope con be set only if a global context has been initialized |
| "tls" | "certificate_expiration_threshold" | 30 | 0 | 365 | The threshold (in days) used to trigger the `NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE` flow risk |
| "tls" | "application_blocks_tracking" | disable | NULL | NULL | Enable/disable processing of TLS Application Blocks (post handshake) to extract statistical information about the flow |
| "tls" | "metadata.sha1_fingerprint" | enable | NULL | NULL | Enable/disable computation and export of SHA1 fingerprint for TLS flows. Note that if it is disable, the flow risk `NDPI_MALICIOUS_SHA1_CERTIFICATE` is not checked |
| "tls" | "metadata.ja3c_fingerprint" | enable | NULL | NULL | Enable/disable computation and export of JA3C fingerprint for TLS flows. Note that if it is disable, the flow risk `NDPI_MALICIOUS_JA3` is not checked |
| "tls" | "metadata.ja3s_fingerprint" | enable | NULL | NULL | Enable/disable computation and export of JA3S fingerprint for TLS flows |
| "tls" | "metadata.ja4c_fingerprint" | enable | NULL | NULL | Enable/disable computation and export of JA4C fingerprint for TLS flows |
| "smtp" | "tls_dissection" | enable | NULL | NULL | Enable/disable dissection of TLS packets in cleartext SMTP flows (because of opportunistic TLS, via STARTTLS msg) |
| "imap" | "tls_dissection" | enable | NULL | NULL | Enable/disable dissection of TLS packets in cleartext IMAP flows (because of opportunistic TLS, via STARTTLS msg) |
| "pop" | "tls_dissection" | enable | NULL | NULL | Enable/disable dissection of TLS packets in cleartext POP flows (because of opportunistic TLS, via STARTTLS msg) |
| "ftp" | "tls_dissection" | enable | NULL | NULL | Enable/disable dissection of TLS packets in cleartext FTP flows (because of opportunistic TLS, via AUTH TLS msg) |
| "stun" | "max_packets_extra_dissection" | 4 | 0 | 255 | After a flow has been classified has STUN, nDPI might analyse more packets to look for a sub-classification or for metadata. This parameter set the upper limit on the number of these packets |
| "stun" | "tls_dissection" | enable | NULL | NULL | Enable/disable dissection of TLS packets multiplexed into STUN flows |
| "stun" | "metadata.attribute.mapped_address" | enable | NULL | NULL | Enable/disable extraction of (xor)-mapped-address attribute for STUN flows. If it is disabled, STUN classification might be significant faster |
| "stun" | "metadata.attribute.response_origin" | enable | NULL | NULL | Enable/disable extraction of response-origin attribute for STUN flows. If it is disabled, STUN classification might be significant faster |
| "stun" | "metadata.attribute.other_address" | enable | NULL | NULL | Enable/disable extraction of other-address attribute for STUN flows. If it is disabled, STUN classification might be significant faster |
| "stun" | "metadata.attribute.relayed_address" | enable | NULL | NULL | Enable/disable extraction of (xor-)relayed-address attribute for STUN flows. If it is disabled, STUN classification might be significant faster |
| "stun" | "metadata.attribute.peer_address" | enable | NULL | NULL | Enable/disable extraction of (xor-)peer-address attribute for STUN flows. If it is disabled, STUN classification might be significant faster; however sub-classification capability might be negatively impacted |
| "dns" | "subclassification" | enable | NULL | NULL | Enable/disable sub-classification of DNS flows (via query/response domain name). If disabled, some flow risks are not checked |
| "dns" | "process_response" | enable | NULL | NULL | Enable/disable processing of DNS responses. By default, DNS flows are fully classified after the first request/response pair (or after the first response, if the request is missing). If this parameter is disabled, the flows are fully classified after the first packet, i.e. usually after the first request; in that case, some flow risks are not checked and some metadata are not exported |
| "http" | "process_response" | enable | NULL | NULL | Enable/disable processing of HTTP responses. By default, HTTP flows are usually fully classified after the first request/response pair. If this parameter is disabled, the flows are fully classified after the first request (or after the first response, if the request is missing); in that case, some flow risks are not checked and some metadata are not exported |
| "ookla" | "dpi.aggressiveness", | 0x01 | 0x00 | 0x01 | Detection aggressiveness for Ookla. The value is a bitmask. Values: 0x0 = disabled; 0x01 = enable heuristic for detection over TLS (via Ookla LRU cache) |
| "zoom" | "max_packets_extra_dissection" | 4 | 0 | 255 | After a flow has been classified has Zoom, nDPI might analyse more packets to look for a sub-classification or for metadata. This parameter set the upper limit on the number of these packets |
| "rtp" | "search_for_stun" | disable | NULL | NULL | After a flow has been classified as RTP or RTCP, nDPI might analyse more packets to look for STUN/DTLS packets, i.e. to try to tell if this flow is a "pure" RTP/RTCP flow or if the RTP/RTCP packets are multiplexed with STUN/DTLS. Useful for proper (sub)classification when the beginning of the flows are not captured or if there are lost packets in the the captured traffic. If enabled, nDPI requires more packets to process for each RTP/RTCP flow. |
| $PROTO_NAME | "log" | disable | NULL | NULL | Enable/disable logging/debug for specific protocol. Use "any" as protocol name if you want to easily enable/disable logging/debug for all protocols |
| $PROTO_NAME | "ip_list.load" | 1 | NULL | NULL | Enable/disable loading of internal list of IP addresses (used for (sub)classification) specific to that protocol. Use "any" as protocol name if you want to easily enable/disable all lists. This knob is valid only for the following protocols: Alibaba, Amazon AWS, Apple, Avast, Bloomberg, Cachefly, Cloudflare, Discord, Disney+, Dropbox, Edgecast, EpicGames, Ethereum, Facebook, Github, Google, Google Cloud, GoTo, Hotspot Shield, Hulu, Line, Microsoft 365, Microsoft Azure, Microsoft One Drive, Microsoft Outlook, Mullvad, Netflix, Nvidia, OpenDNS, ProtonVPN, RiotGames, Roblox, Skype/Teams, Starcraft, Steam, Teamviewer, Telegram, Tencent, Threema, TOR, Twitch, Twitter, UbuntuONE, VK, Yandex, Yandex Cloud, Webex, Whatsapp, Zoom |
+342
View File
@@ -0,0 +1,342 @@
nDPI Flow Risks
###############
nDPI is designed not just to detect application protocols in traffic flows but also to evaluate potential security risks associated with the traffic. In nDPI parlance this is called a "flow risk". Flows can have multiple risks detected hence nDPI reports them with a bitmap. Each risk detected corresponds to a bit in the flow risk bitmap. You can read more about `ndpi_risk_enum <https://github.com/ntop/nDPI/blob/dev/src/include/ndpi_typedefs.h>`_ for the list of all numeric risks currently supported.
Below you can find a description of each flow risk so that you can easily understand when a risk is triggered and its meaning. The flow risks are listed in numerical order as they are defined in ndpi_risk_enum.
.. _Risk 001:
NDPI_URL_POSSIBLE_XSS
=====================
HTTP only: this risk indicates a possible `XSS (Cross Side Scripting) <https://en.wikipedia.org/wiki/Cross-site_scripting>`_ attack.
.. _Risk 002:
NDPI_URL_POSSIBLE_SQL_INJECTION
===============================
HTTP only: this risk indicates a possible `SQL Injection attack <https://en.wikipedia.org/wiki/SQL_injection>`_.
.. _Risk 003:
NDPI_URL_POSSIBLE_RCE_INJECTION
===============================
HTTP only: this risk indicates a possible `RCE (Remote Code Execution) attack <https://en.wikipedia.org/wiki/Arbitrary_code_execution>`_.
.. _Risk 004:
NDPI_BINARY_APPLICATION_TRANSFER
================================
HTTP only: this risk indicates that a binary application is downloaded/uploaded. Detected applications include Windows binaries, Linux executables, Unix scripts and Android apps.
.. _Risk 005:
NDPI_KNOWN_PROTOCOL_ON_NON_STANDARD_PORT
========================================
This risk indicates a known protocol used on a non standard port. Example HTTP is supposed to use TCP/80, and in case it is detected on TCP/1234 this risk is detected.
.. _Risk 006:
NDPI_TLS_SELFSIGNED_CERTIFICATE
===============================
TLS/QUIC only: this risk is triggered when a `self-signed certificate <https://en.wikipedia.org/wiki/Self-signed_certificate>`_ is used.
.. _Risk 007:
NDPI_TLS_OBSOLETE_VERSION
=========================
Risk triggered when TLS version is older than 1.1.
.. _Risk 008:
NDPI_TLS_WEAK_CIPHER
====================
Risk triggered when an unsafe TLS cipher is used. See `this page <https://community.qualys.com/thread/18212-how-does-qualys-determine-the-server-cipher-suites>`_ for a list of insecure ciphers.
.. _Risk 009:
NDPI_TLS_CERTIFICATE_EXPIRED
============================
Risk triggered when a TLS certificate is expired, i.e. the current date falls outside of the certificate validity dates.
.. _Risk 010:
NDPI_TLS_CERTIFICATE_MISMATCH
=============================
Risk triggered when a TLS certificate does not match the hostname we're accessing. Example you do http://www.aaa.com and the TLS certificate returned is for www.bbb.com.
.. _Risk 011:
NDPI_HTTP_SUSPICIOUS_USER_AGENT
===============================
HTTP only: this risk is triggered whenever the user agent contains suspicious characters or its format is suspicious. Example: <?php something ?> is a typical suspicious user agent.
.. _Risk 012:
NDPI_NUMERIC_IP_HOST
=========================
This risk is triggered whenever a HTTP/TLS/QUIC connection is using a literal IPv4 or IPv6 address as ServerName (TLS/QUIC; example: SNI=1.2.3.4) or as Hostname (HTTP; example: http://1.2.3.4.).
.. _Risk 013:
NDPI_HTTP_SUSPICIOUS_URL
========================
HTTP only: this risk is triggered whenever the accessed URL is suspicious. Example: http://127.0.0.1/msadc/..%255c../..%255c../..%255c../winnt/system32/cmd.exe.
.. _Risk 014:
NDPI_HTTP_SUSPICIOUS_HEADER
===========================
HTTP only: this risk is triggered whenever the HTTP peader contains suspicious entries such as Uuid, TLS_version, Osname that are unexpected on the HTTP header.
.. _Risk 015:
NDPI_TLS_NOT_CARRYING_HTTPS
===========================
TLS only: this risk indicates that this TLS flow will not be used to transport HTTP content. Example VPNs use TLS to encrypt data rather to carry HTTP. This is useful to spot this type of cases.
.. _Risk 016:
NDPI_SUSPICIOUS_DGA_DOMAIN
==========================
A `DGA <https://en.wikipedia.org/wiki/Domain_generation_algorithm>`_ is used to generate domain names often used by malwares. This risk indicates that this domain name can (but it's not 100% sure) a DGA as its name is suspicious.
.. _Risk 017:
NDPI_MALFORMED_PACKET
=====================
This risk is generated when a packet (e.g. a DNS packet) has an unexpected format. This can indicate a protocol error or more often an attempt to jeopardize a valid protocol to carry other type of data.
.. _Risk 018:
NDPI_SSH_OBSOLETE_CLIENT_VERSION_OR_CIPHER
==========================================
This risk is generated whenever a SSH client uses an obsolete SSH protocol version or insecure ciphers.
.. _Risk 019:
NDPI_SSH_OBSOLETE_SERVER_VERSION_OR_CIPHER
==========================================
This risk is generated whenever a SSH server uses an obsolete SSH protocol version or insecure ciphers.
.. _Risk 020:
NDPI_SMB_INSECURE_VERSION
=========================
This risk indicates that the `SMB <https://en.wikipedia.org/wiki/Server_Message_Block>`_ version used is insecure (i.e. v1).
.. _Risk 021:
NDPI_TLS_SUSPICIOUS_ESNI_USAGE
==============================
`SNI <https://en.wikipedia.org/wiki/Server_Name_Indication>`_ is a way to carry in TLS the host/domain name we're accessing. ESNI means encrypted SNI and it is a way to mask SNI (carried in clear text in the TLS header) with encryption. While this practice is legal, it could be used for hiding data or for attacks such as a suspicious `domain fronting <https://github.com/SixGenInc/Noctilucent/blob/master/docs/>`_.
.. _Risk 022:
NDPI_UNSAFE_PROTOCOL
====================
This risk indicates that the protocol used is insecure and that a secure protocol should be used (e.g. Telnet vs SSH).
.. _Risk 023:
NDPI_DNS_SUSPICIOUS_TRAFFIC
===========================
This risk is returned when DNS traffic returns an unexpected/obsolete `record type <https://en.wikipedia.org/wiki/List_of_DNS_record_types>`_
or when a suspicious query with a very long host name is detected.
.. _Risk 024:
NDPI_TLS_MISSING_SNI
====================
TLS needs to carry the the `SNI <https://en.wikipedia.org/wiki/Server_Name_Indication>`_ of the remote server we're accessing. Unfortunately SNI is optional in TLS so it can be omitted. In this case this risk is triggered as this is a non-standard situation that indicates a potential security problem or a protocol using TLS for other purposes (or a protocol bug).
.. _Risk 025:
NDPI_HTTP_SUSPICIOUS_CONTENT
============================
HTTP only: risk reported when HTTP carries content in expected format. Example the HTTP header indicates that the context is text/html but the real content is not readeable (i.e. it can transport binary data). In general this is an attempt to use a valid MIME type to carry data that does not match the type.
.. _Risk 026:
NDPI_RISKY_ASN
==============
This is a placeholder for traffic exchanged with `ASN <https://en.wikipedia.org/wiki/Autonomous_system_(Internet)>`_ that are considered risky. nDPI does not fill this risk that instead should be filled by aplications sitting on top of nDPI (e.g. ntopng).
.. _Risk 027:
NDPI_RISKY_DOMAIN
=================
This is a placeholder for traffic exchanged with domain names that are considered risky. nDPI does not fill this risk that instead should be filled by aplications sitting on top of nDPI (e.g. ntopng).
.. _Risk 028:
NDPI_MALICIOUS_JA3
==================
`JA3 <https://engineering.salesforce.com/tls-fingerprinting-with-ja3-and-ja3s-247362855967>`_ is a method to fingerprint TLS traffic. This risk indicates that the JA3 of the TLS connection is considered suspicious (i.e. it has been found in known malware JA3 blacklists). nDPI does not fill this risk that instead should be filled by aplications sitting on top of nDPI (e.g. ntopng).
.. _Risk 029:
NDPI_MALICIOUS_SHA1_CERTIFICATE
===============================
TLS certificates are uniquely identified with a `SHA1 <https://en.wikipedia.org/wiki/SHA-1>`_ hash value. If such hash is found on a blacklist, this risk can be used. As for other risks, this is a placeholder as nDPI does not fill this risk that instead should be filled by aplications sitting on top of nDPI (e.g. ntopng).
.. _Risk 030:
NDPI_DESKTOP_OR_FILE_SHARING_SESSION
====================================
This risk is set when the flow carries desktop or file sharing sessions (e.g. TeamViewer or AnyDesk just to mention two).
.. _Risk 031:
NDPI_TLS_UNCOMMON_ALPN
======================
This risk is set when the `ALPN <https://en.wikipedia.org/wiki/Application-Layer_Protocol_Negotiation>`_ (it indicates the protocol carried into this TLS flow, for instance HTTP/1.1) is uncommon with respect to the list of expected values.
.. _Risk 032:
NDPI_TLS_CERT_VALIDITY_TOO_LONG
===============================
From 01/09/2020 TLS certificates lifespan is limited to `13 months <https://www.appviewx.com/blogs/tls-certificate-lifespans-now-capped-at-13-months/>`_. This risk is triggered for certificates not respecting this directive.
.. _Risk 033:
NDPI_TLS_SUSPICIOUS_EXTENSION
=============================
This risk is triggered when the domain name (SNI extension) is not printable and thus it is a problem. In TLS extensions can be dynamically specified by the client in the hello packet.
.. _Risk 034:
NDPI_TLS_FATAL_ALERT
====================
This risk is triggered when a TLS fatal alert is detected in the TLS flow. See `this page <https://techcommunity.microsoft.com/t5/iis-support-blog/ssl-tls-alert-protocol-and-the-alert-codes/ba-p/377132>`_ for details.
.. _Risk 035:
NDPI_SUSPICIOUS_ENTROPY
=======================
This risk is used to detect suspicious data carried in ICMP packets whose entropy (used to measure how data is distributed, hence to indirectly guess the type of data carried on) is suspicious and thus that it can indicate a data leak. Suspicious values indicate random entropy or entropy that is similar to encrypted traffic. In the latter case, this can be a suspicious data exfiltration symptom.
.. _Risk 036:
NDPI_CLEAR_TEXT_CREDENTIALS
===========================
Clear text protocols are not intrinsically bad, but they should be avoided when they carry credentials as they can be intercepted by malicious users. This risk is triggered whenever clear text protocols (e.g. FTP, HTTP, IMAP...) contain credentials in clear text (read it as nDPI does not trigger this risk for HTTP connections that do not carry credentials).
.. _Risk 037:
NDPI_DNS_LARGE_PACKET
=====================
`DNS <https://en.wikipedia.org/wiki/Domain_Name_System>`_ packets over UDP should be limited to 512 bytes. DNS packets over this threshold indicate a potential security risk (e.g. use DNS to carry data) or a misconfiguration.
.. _Risk 038:
NDPI_DNS_FRAGMENTED
===================
UDP `DNS <https://en.wikipedia.org/wiki/Domain_Name_System>`_ packets cannot be fragmented. If so, this indicates a potential security risk (e.g. use DNS to carry data) or a misconfiguration.
.. _Risk 039:
NDPI_INVALID_CHARACTERS
=======================
The risk is set whenever a dissected protocol contains characters not allowed in that protocol field.
For example a DNS hostname must only contain a subset of all printable characters or else this risk is set.
Additionally, some TLS protocol fields are checked for printable characters as well.
.. _Risk 040:
NDPI_POSSIBLE_EXPLOIT
=====================
The risk is set whenever a possible exploit attempt (e.g. `Log4J/Log4Shell <https://en.wikipedia.org/wiki/Log4Shell>`_) is detected.
.. _Risk 041:
NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE
===================================
The risk is set whenever a TLS certificate is close to the expiration date.
.. _Risk 042:
NDPI_PUNYCODE_IDN
===================================
The risk is set whenever a domain name is specified in IDN format as they are sometimes used in `IDN homograph attacks <https://en.wikipedia.org/wiki/IDN_homograph_attack>`_.
.. _Risk 043:
NDPI_ERROR_CODE_DETECTED
===================================
The risk is set whenever an error code is detected in the underlying protocol (e.g. HTTP and DNS).
.. _Risk 044:
NDPI_HTTP_CRAWLER_BOT
===================================
The risk is set whenever a crawler/bot/robot has been detected
.. _Risk 045:
NDPI_ANONYMOUS_SUBSCRIBER
===================================
The risk is set whenever the (source) IP address has been anonymized and it can't be used to identify the subscriber.
Example: the flow is generated by an iCloud-private-relay exit node.
.. _Risk 046:
NDPI_UNIDIRECTIONAL_TRAFFIC
===================================
The risk is set whenever the flow has unidirectional traffic (typically no traffic on the server to client direction). THis
risk is not triggered for multicast/broadcast destinations.
.. _Risk 047:
NDPI_HTTP_OBSOLETE_SERVER
===================================
This risk is generated whenever a HTTP server uses an obsolete HTTP server version.
.. _Risk 048:
NDPI_PERIODIC_FLOW
==================
This risk is generated whenever a flow is observed at a specific periodic pace (e.g. every 10 seconds).
.. _Risk 049:
NDPI_MINOR_ISSUES
=================
Minor packet/flow issues (e.g. DNS traffic with zero TTL) have been detected.
.. _Risk 050:
NDPI_TCP_ISSUES
===============
Relevant TCP connection issues such as connection refused, scan, or probe attempt.
.. _Risk 051
NDPI_FULLY_ENCRYPTED
====================
Flow with Unknown protocol containing encrypted traffic.
.. _Risk 052
NDPI_TLS_ALPN_SNI_MISMATCH
=========================
Invalid TLS ALPN/SNI mismatch. For instance ALPN advertises the flow as h2 (HTTP/2.0) and no SNI is reported.
.. _Risk 053
NDPI_MALWARE_CONTACTED
======================
Client contacted a server host labelled as malware.
.. _Risk 054:
NDPI_BINARY_DATA_TRANSFER
============================
HTTP only: this risk indicates that a binary file/data application transfer (attempt).
.. _Risk 055:
NDPI_PROBING_ATTEMPT
====================
Connection with no data exchagef that looks like a probing attempt
Binary file not shown.
Binary file not shown.
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 47 KiB

+39
View File
@@ -0,0 +1,39 @@
nDPI Documentation
====================
nDPI is an open source DPI (Deep Packet Inspection) toolkit for traffic analysis.
.. toctree::
:maxdepth: 2
:caption: User's Guide
what_is_ndpi
.. toctree::
:maxdepth: 2
:caption: Developer's Guide
api/library_root
protocols
flow_risks
.. toctree::
:caption: Other Products
ntopng <https://www.ntop.org/guides/ntopng/>
nProbe <https://www.ntop.org/guides/nprobe/>
nProbe Cento <https://www.ntop.org/guides/cento/>
n2disk <https://www.ntop.org/guides/n2disk/>
PF_RING <https://www.ntop.org/guides/pf_ring/>
nEdge <https://www.ntop.org/guides/nedge/>
nScrub <https://www.ntop.org/guides/nscrub/>
nBox <https://www.ntop.org/guides/nbox/>
nTap <https://www.ntop.org/guides/ntap/>
.. Indices and tables
.. ==================
..
.. * :ref:`genindex`
.. * :ref:`modindex`
.. * :ref:`search`
+84
View File
@@ -0,0 +1,84 @@
A simple, common example
```
struct ndpi_detection_module_struct *ndpi_struct;
ndpi_cfg_error rc;
int ret;
ndpi_struct = ndpi_init_detection_module(NULL);
if(!ndpi_struct) {
ERROR;
}
/* Configuration */
rc = ndpi_set_config(ndpi_struct, "tls", "certificate_expiration_threshold", "10");
if(rc != NDPI_CFG_OK) {
ERROR;
}
/* Finalization */
ret = ndpi_finalize_initialization(ndpi_struct);
if(ret != 0) {
ERROR;
}
/* Initialization done, now you can feed packets to the library */
/* Cleanup */
ndpi_exit_detection_module(ndpi_struct);
```
A more complex example, with global context and a shared Oookla LRU cache (all the others caches are local)
```
struct ndpi_global_context *g_ctx;
struct ndpi_detection_module_struct *ndpi_structs[num_local_contexts];
ndpi_cfg_error rc;
int i, ret;
g_ctx = ndpi_global_init();
if(!g_ctx) {
ERROR;
}
for(i = 0; i < num_local_contexts; i++) {
ndpi_structs[i] = ndpi_init_detection_module(g_ctx);
if(!ndpi_struct[i]) {
ERROR;
}
rc = ndpi_set_config(ndpi_structs[i], NULL, "lru.ookla.scope", "1");
if(rc != NDPI_CFG_OK) {
ERROR;
}
ret = ndpi_finalize_initialization(ndpi_structs[i]);
if(ret != 0) {
ERROR;
}
}
/* Initialization done */
/* Cleanup */
for(i = 0; i < num_local_contexts; i++) {
ndpi_exit_detection_module(ndpi_structs[i]);
}
ndpi_global_deinit(g_ctx);
```
Binary file not shown.
Binary file not shown.
+858
View File
@@ -0,0 +1,858 @@
nDPI Protocols List
###################
This page provides the list of the protocols/applications supported by nDPI. For each protocol there is a brief description, some links to further, more detailed information and, optionally, some notes that might be useful when handling such a protocol (from the application/integrator point of view)
Work in progress!
.. _Proto 32:
`NDPI_PROTOCOL_BFCP`
=======================
BFCP (Binary Floor Control Protocol) is used for controlling and coordinating real-time data sharing and collaboration during video conferencing sessions.
References: `RFC: <https://datatracker.ietf.org/doc/html/rfc8855>`_
.. _Proto 54:
`NDPI_PROTOCOL_IQIYI`
===========================
iQIYI is a Chinese online video platform that offers a wide range of original and licensed content including movies, dramas, variety shows, and anime.
References: `Main site: <https://www.iqiyi.com/>`_
.. _Proto 59:
`NDPI_PROTOCOL_ADOBE_CONNECT`
===========================
Adobe Connect is a web conferencing platform that allows users to conduct online meetings, webinars, and virtual classrooms.
References: `Main site: <https://www.adobe.com/products/adobeconnect.html>`_
.. _Proto 65:
`NDPI_PROTOCOL_IRC`
==================
IRC (Internet Relay Chat) is a text-based chat system for instant messaging.
References: `Wikipiedia: <https://en.wikipedia.org/wiki/IRC>`_ and `Some statistics: <https://netsplit.de/networks/top10.php>`
.. _Proto 72:
`NDPI_PROTOCOL_HART_IP`
=======================
Highway Addressable Remote Transducer over IP
References: `Protocol Specs: <https://library.fieldcommgroup.org/20085/TS20085>`_
.. _Proto 96:
`NDPI_PROTOCOL_TFTP`
====================
TFTP is a very simple protocol used to transfer files. It is from this that its name comes, Trivial File Transfer Protocol or TFTP.
References: `RFC1350 <https://datatracker.ietf.org/doc/html/rfc1350>`_ and `RFC2347 <https://datatracker.ietf.org/doc/html/rfc2347>`_ and `RFC2349 <https://datatracker.ietf.org/doc/html/rfc2349>`_
.. _Proto 127:
`NDPI_PROTOCOL_MS_RPCH`
=======================
A Remote Procedure Call protocol over HTTP from Microsoft.
References: `Protocol Specs: <https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-rpch/c0f4c9c5-1a61-4d10-b8e2-005378d1d212>`_
.. _Proto 149:
`NDPI_PROTOCOL_EGD`
=========================
Ethernet Global Data (EGD) is a communication protocol developed by GE Fanuc Automation for real-time data exchange between automation devices and control systems using standard Ethernet technology. It is widely adopted in industrial environments for its simplicity and reliability in data communication.
References: `Wikipedia <https://en.wikipedia.org/wiki/Ethernet_Global_Data_Protocol>`_
Notes:
- This dissector only works for data packets, not configuration commands.
- IPv6 is not supported
.. _Proto 182:
`NDPI_PROTOCOL_RESP`
=======================
Redis Serialization Protocol
References: `Protocol Specs: <https://redis.io/docs/reference/protocol-spec/>`_
.. _Proto 186:
`NDPI_PROTOCOL_COD_MOBILE`
=======================
Call of Duty: Mobile is a free-to-play shooter game for iOS and Android devices. It has amassed over 650 million downloads worldwide, making it one of the most popular mobile games.
References: `Main site: <https://www.callofduty.com/mobile/>`_
.. _Proto 201:
`NDPI_PROTOCOL_GOOGLE_MEET`
===========================
Google Meet is a video conferencing service from Google.
References: `Main site: <https://meet.google.com/>`_
.. _Proto 235:
`NDPI_PROTOCOL_VALVE_SDR`
===========================
Steam Datagram Relay (SDR) is Valve's virtual private gaming network. Used in all modern games from Valve, but can also be used by developers via the Steamworks SDK for their games.
References: `Main site: <https://partner.steamgames.com/doc/features/multiplayer/steamdatagramrelay>`_
.. _Proto 338:
`NDPI_PROTOCOL_SRTP`
====================
The Secure Real-time Transport Protocol (SRTP) is a profile for Real-time Transport Protocol (RTP) intended to provide encryption, message authentication, integrity, and replay attack protection to the RTP data.
References: `RFC3711 <https://datatracker.ietf.org/doc/html/rfc3711>`_
Notes:
- You can think of SRTP simply as the "encrypted" version of RTP, something like HTTPS vs HTTP;
- It is not usually possible to tell RTP from SRTP. nDPI generally uses the former and it uses the latter only when it is really sure that the media stream has been encrypted.
.. _Proto 339:
`NDPI_PROTOCOL_OPERA_VPN`
=========================
Opera VPN is a free VPN included with Opera Browser.
References: `Main site <https://www.opera.com/it/features/free-vpn>`_
.. _Proto 340:
`NDPI_PROTOCOL_EPICGAMES`
=========================
Epic Games is a video game company developing the Unreal Engine and some successful games as Fortnite and Gears of War.
References: `Main site <https://store.epicgames.com/en-US/>`_ and `Fortnite <https://www.fortnite.com/>`_
.. _Proto 341:
`NDPI_PROTOCOL_GEFORCENOW`
==========================
GeForce Now is the brand used by Nvidia for its cloud gaming service.
References: `Main site <https://www.nvidia.com/en-us/geforce-now/>`_
.. _Proto 342:
`NDPI_PROTOCOL_NVIDIA`
======================
Generic web traffic from Nvidia sites.
References: `Main site <https://www.nvidia.com>`_
.. _Proto 343:
`NDPI_PROTOCOL_BITCOIN`
=======================
Bitcoin is one of the most common crypto currencies.
References: `Main site <https://en.bitcoin.it/wiki/Protocol_documentation>`_
Notes:
- Not each crypto exchange is a mining, it could be a normal transaction, sending or receving.
- Bitcoin network protocol covers the broader set of rules that govern how all nodes in the network communicate and sync with each others blocks and transactions.
- On the other hand mining protocols are more specific and deal directly with how miners interact with the network and participate in the mining process.
.. _Proto 344:
`NDPI_PROTOCOL_PROTONVPN`
=========================
Proton VPN is a VPN service operated by the Swiss company Proton AG, the company behind the email service Proton Mail
References: `Main site <https://protonvpn.com/>`_
.. _Proto 345:
`NDPI_PROTOCOL_THRIFT`
======================
Apache Thrift is a generic data interchange framework that supports a bunch of different languages and platforms.
References: `Official site <https://thrift.apache.org>`_ and `Github <https://github.com/apache/thrift>`_
.. _Proto 346:
`NDPI_PROTOCOL_ROBLOX`
======================
Roblox is an online game platform and game creation system.
References: `Main site <https://www.roblox.com/>`_
Notes:
- Since Roblox games use a custom version of the RakNet protocol, some Roblox flows might be classified as RakNet.
.. _Proto 347:
`NDPI_PROTOCOL_SERVICE_LOCATION`
================================
The Service Location Protocol is a service discovery protocol that allows computers and other devices to find services in a local area network without prior configuration.
References: `SLPv1 <https://datatracker.ietf.org/doc/html/rfc2165>`_ and `SLPv2 <https://datatracker.ietf.org/doc/html/rfc2608>`_
.. _Proto 348:
`NDPI_PROTOCOL_MULLVAD`
=======================
Mullvad is a VPN service operated by Mullvad VPN AB, based in Sweden
References: `Main site <https://mullvad.net/>`_
.. _Proto 349:
`NDPI_PROTOCOL_HTTP2`
=====================
HTTP/2 (originally named HTTP/2.0) is a major revision of the HTTP network protocol used by the World Wide Web.
References: `RFC <https://datatracker.ietf.org/doc/html/rfc9113>`_
Notes:
- HTTP/2 is almost always encrypted, i.e. transported over TLS.
.. _Proto 350:
`NDPI_PROTOCOL_HAPROXY`
=======================
HAProxy is a free and open source software that provides a high availability load balancer and reverse proxy for TCP and HTTP-based applications that spreads requests across multiple servers.
References: `Main site: <https://www.haproxy.org>`_
.. _Proto 351:
`NDPI_PROTOCOL_RMCP`
====================
The Intelligent Platform Management Interface (IPMI) is a set of computer interface specifications for an autonomous computer subsystem that provides management and monitoring capabilities independently of the host system's CPU, firmware (BIOS or UEFI) and operating system.
References: `Protocol Specs: <https://www.dmtf.org/sites/default/files/standards/documents/DSP0114.pdf>`_
.. _Proto 352:
`NDPI_PROTOCOL_CAN`
===================
Controller Area Network (CAN) is used extensively in automotive applications, with in excess of 400 million CAN enabled microcontrollers manufactured each year.
References: `Protocol Specs: <https://www.iso.org/standard/63648.html>`_
.. _Proto 353:
`NDPI_PROTOCOL_PROTOBUF`
========================
Protocol Buffers (Protobuf) is a free and open-source cross-platform data format used to serialize structured data.
References: `Encoding: <https://protobuf.dev/programming-guides/encoding>`_
.. _Proto 354:
`NDPI_PROTOCOL_ETHEREUM`
=======================
Ethereum is a decentralized, open-source blockchain with smart contract functionality.
References: `Main site <https://ethereum.org/en/developers/docs/intro-to-ethereum/>`_
Notes:
- same as Bitcoin, not each crypto exchange is a mining, it could be a normal transaction, sending or receving or even blockchain exploration.
.. _Proto 355:
`NDPI_PROTOCOL_TELEGRAM_VOIP`
============================
Audio/video calls made using the telegram app.
References: `Wikipedia <https://en.wikipedia.org/wiki/telegram_(software)/>`_
.. _Proto 356:
`NDPI_PROTOCOL_SINA_WEIBO`
============================
Chinese microblogging (weibo) website.
References: `Wikipedia <https://en.wikipedia.org/wiki/Sina_Weibo>`_
.. _Proto 358:
`NDPI_PROTOCOL_PTPV2`
============================
IEEE 1588-2008 Precision Time Protocol (PTP) Version 2.
References: `Protocol Specs: <https://standards.ieee.org/ieee/1588/4355/>`_
.. _Proto 359:
`NDPI_PROTOCOL_RTPS`
============================
Real-Time Publish Subscribe Protocol
References: `Protocol Specs: <https://www.omg.org/spec/DDSI-RTPS/>`_
.. _Proto 360:
`NDPI_PROTOCOL_OPC_UA`
============================
IEC62541 OPC Unified Architecture
References: `Protocol Specs: <https://reference.opcfoundation.org/>`_
.. _Proto 361:
`NDPI_PROTOCOL_S7COMM_PLUS`
============================
A proprietary protocol from Siemens used for data exchange between PLCs and access PLC data via SCADA systems.
Completely different from classic S7Comm, but also uses TPKT/COTP as a transport.
References: `Unofficial description: <https://plc4x.apache.org/protocols/s7/s7comm-plus.html>`_
.. _Proto 362:
`NDPI_PROTOCOL_FINS`
============================
Factory Interface Network Service (FINS) is a network protocol used by Omron PLCs.
References: `Protocol Specs: <https://assets.omron.eu/downloads/manual/en/v4/w421_cj1w-etn21_cs1w-etn21_ethernet_units_-_construction_of_applications_operation_manual_en.pdf>`_
.. _Proto 363:
`NDPI_PROTOCOL_ETHERSIO`
============================
Ether-S-I/O is a proprietary protocol used by Saia-Burgess's PLCs.
References: `Wireshark wiki: <https://wiki.wireshark.org/EtherSIO.md>`_
.. _Proto 364:
`NDPI_PROTOCOL_UMAS`
============================
UMAS is a proprietary Schneider Electric protocol based on Modbus. It's used in Modicon M580 and Modicon M340 CPU-based PLCs.
References: `Unofficial article: <https://ics-cert.kaspersky.com/publications/reports/2022/09/29/the-secrets-of-schneider-electrics-umas-protocol/>`_
.. _Proto 365:
`NDPI_PROTOCOL_BECKHOFF_ADS`
============================
Automation Device Specification is the protocol used for interfacing with Beckhoff PLCs via TwinCAT.
References: `Protocol Specs: <https://infosys.beckhoff.com/english.php?content=../content/1033/tc3_ads_intro/115847307.html>`_
.. _Proto 366:
`NDPI_PROTOCOL_ISO9506_1_MMS`
============================
The international standard MMS (Manufacturing Message Specification) is an OSI application layer messaging protocol origionally designed for the remote control and monitoring of devices such as Remote Terminal Units (RTU), Programmable Logic Controllers (PLC), Numerical Controllers (NC), or Robot Controllers (RC).
References: `Paid Specs: <https://www.iso.org/ru/standard/37079.html>`_
.. _Proto 367:
`NDPI_PROTOCOL_IEEE_C37118`
============================
IEEE Standard for Synchrophasor Data Transfer for Power Systems
References: `Paid Specs: <https://standards.ieee.org/ieee/C37.118.1/4902/>`_
.. _Proto 368:
`NDPI_PROTOCOL_ETHERSBUS`
============================
Ether-S-Bus is a proprietary protocol used for the communication with and between PLCs manufactured by Saia-Burgess Controls Ltd.
References: `Wireshark wiki: <https://wiki.wireshark.org/EtherSBus>`_
.. _Proto 369:
`NDPI_PROTOCOL_MONERO`
======================
Monero is a private and decentralized cryptocurrency with focus on confidentiality and security.
.. _Proto 370:
`NDPI_PROTOCOL_DCERPC`
======================
DCE/RPC is a specification for a remote procedure call mechanism that defines both APIs and an over-the-network protocol.
References: `Wireshark wiki: <https://wiki.wireshark.org/DCE/RPC>`_
.. _Proto 371:
`NDPI_PROTOCOL_PROFINET_IO`
===========================
PROFINET/IO is a field bus protocol based on connectionless DCE/RPC.
References: `Protocol Specs: <https://www.profibus.com/download/profinet-specification>`_
.. _Proto 372:
`NDPI_PROTOCOL_HISLIP`
======================
High-Speed LAN Instrument Protocol (HiSLIP) is a protocol for remote instrument control of LAN-based test and measurement instruments.
References: `Protocol Specs: <https://www.ivifoundation.org/downloads/Protocol%20Specifications/IVI-6.1_HiSLIP-2.0-2020-04-23.pdf>`_
.. _Proto 373:
`NDPI_PROTOCOL_UFTP`
====================
Encrypted UDP based FTP with multicast.
References: `Protocol Specs: <https://uftp-multicast.sourceforge.net/protocol.txt>`_.
.. _Proto 374:
`NDPI_PROTOCOL_OPENFLOW`
========================
OpenFlow protocol is a network protocol closely associated with Software-Defined Networking (SDN).
References: `Protocol Specs: <https://opennetworking.org/wp-content/uploads/2014/10/openflow-switch-v1.5.1.pdf>`_
.. _Proto 375:
`NDPI_PROTOCOL_JSON_RPC`
========================
JSON-RPC is a remote procedure call protocol encoded in JSON.
References: `Protocol Specs: <https://www.jsonrpc.org/specification>`_
.. _Proto 376:
`NDPI_PROTOCOL_WEBDAV`
======================
WebDAV is a set of extensions to the HTTP protocol that allows WebDAV clients to collaboratively edit and manage files on remote Web servers.
References: `RFC4918: <https://datatracker.ietf.org/doc/html/rfc4918>`_
Notes:
- WebDAV is almost always encrypted, i.e. transported over TLS.
.. _Proto 377:
`NDPI_PROTOCOL_APACHE_KAFKA`
============================
Apache Kafka is a distributed event store and stream-processing platform.
References: `Official site <https://kafka.apache.org>`_ and `Github <https://github.com/apache/kafka>`_
.. _Proto 378:
`NDPI_PROTOCOL_NOMACHINE`
=========================
NoMachine is a popular proprietary remote desktop software.
References: `Main site <https://www.nomachine.com/>`_
.. _Proto 379:
`NDPI_PROTOCOL_IEC62056`
============================
IEC 62056-4-7 DLMS/COSEM is a transport layer for IP networks.
References: `Paid Specs: <https://webstore.iec.ch/publication/22487>`_
Notes:
- Wireshark is not able to recognize this protocol. Some old plugins/code (with some documentation) are available `here <https://github.com/bearxiong99/wireshark-dlms>` and `here <https://github.com/matousp/dlms-analysis/tree/master>`.
.. _Proto 380:
`NDPI_PROTOCOL_HL7`
=========================
HL7 is a range of global standards for the transfer of clinical and administrative health data between applications.
References: `Main site <https://www.hl7.org/>`_
.. _Proto 381:
`NDPI_PROTOCOL_CEPH`
=========================
Ceph is a scalable distributed storage system.
References: `Main site <https://ceph.io/en/>`_
.. _Proto 382:
`NDPI_PROTOCOL_GOOGLE_CHAT`
=========================
Google Chat is an instant messaging service from Google, which replaced Hangouts.
References: `Main site <https://chat.google.com/>`_
.. _Proto 383:
`NDPI_PROTOCOL_ROUGHTIME`
=========================
A protocol that aims to achieve rough time synchronization while detecting servers that provide inaccurate time and providing cryptographic proof of their malfeasance.
References: `IETF Draft <https://www.ietf.org/archive/id/draft-ietf-ntp-roughtime-08.html>`_
.. _Proto 384:
`NDPI_PROTOCOL_PIA`
=========================
Private Internet Access (PIA) is a popular VPN service from Kape Technologies.
References: `Main site <https://www.privateinternetaccess.com/>`_
.. _Proto 385:
`NDPI_PROTOCOL_KCP`
===================
KCP - A Fast and Reliable ARQ Protocol. It provides TCP-like stream support with low latency at the cost of bandwidth usage - used by lot's of Open Source / Third Party applications.
References: `Protocol Specs: <https://github.com/skywind3000/kcp/blob/master/protocol.txt>`_
.. _Proto 386:
`NDPI_PROTOCOL_DOTA2`
=========================
Dota 2 is an extremely popular multiplayer MOBA game from Valve.
References: `Main site <https://www.dota2.com/>`_
.. _Proto 387:
`NDPI_PROTOCOL_MUMBLE`
=========================
Mumble is a free, open source, low latency, high quality voice chat application.
References: `Main site <https://www.mumble.info/>`_
.. _Proto 388:
`NDPI_PROTOCOL_YOJIMBO`
=======================
Yojimbo (netcode) is a secure connection-based client/server protocol built on top of UDP.
References: `Protocol Specs: <https://github.com/mas-bandwidth/netcode/blob/main/STANDARD.md>`_
.. _Proto 389:
`NDPI_PROTOCOL_ELECTRONICARTS`
=========================
Electronic Arts is a leading publisher of games on Console, PC and Mobile.
References: `Main site <https://www.ea.com/>`_
Notes:
- Almost all of that traffic is related to their EA Origin game store.
.. _Proto 390:
`NDPI_PROTOCOL_STOMP`
========================
STOMP is a simple interoperable protocol designed for asynchronous message passing between clients via mediating servers. Supported in ActiveMQ and RabbitMQ.
References: `Protocol Specs: <https://stomp.github.io/stomp-specification-1.2.html>`_
.. _Proto 391:
`NDPI_PROTOCOL_RADMIN`
=========================
Radmin is remote access software for the Microsoft Windows platform.
References: `Main site <https://www.radmin.com/>`_
.. _Proto 392:
`NDPI_PROTOCOL_RAFT`
====================
Raft is a consensus algorithm and protocol for managing a replicated log.
References: `C implementation <https://github.com/canonical/raft>`_ and `Paper <https://raft.github.io/raft.pdf>`_
.. _Proto 394:
`NDPI_PROTOCOL_GEARMAN`
====================
Gearman is a network-based job-queuing system that was initially developed by Danga Interactive in order to process large volumes of jobs.
References: `Main site <http://gearman.org/>`_
.. _Proto 395:
`NDPI_PROTOCOL_TENCENTGAMES`
====================
A protocol used by various games from Tencent (mostly mobile games).
References: `Main site <https://www.tencentgames.com/>`_
.. _Proto 396:
`NDPI_PROTOCOL_GAIJIN`
====================
Protocols used in various games from Gaijin Entertainment.
References: `Main site <https://gaijin.net/>`_
.. _Proto 397:
`NDPI_PROTOCOL_C1222`
====================
ANSI C12.22 (IEEE Std 1703) describe a protocol for transporting ANSI C12.19 table data over networks. It's mostly used to communicate with electric meters.
References: `Paid specs <https://www.nema.org/Standards/view/American-National-Standard-for-Protocol-Specification-for-Interfacing-to-Data-Communication-Networks/>`_
.. _Proto 398:
`NDPI_PROTOCOL_HUAWEI`
======================
Generic Huawei traffic.
References: `Main site <https://www.huawei.com/>`_
.. _Proto 399:
`NDPI_PROTOCOL_HUAWEI_CLOUD`
============================
Huawei Mobile Cloud.
References: `Main site <https://cloud.huawei.com/>`_
.. _Proto 400:
`NDPI_PROTOCOL_DLEP`
=====================
The Dynamic Link ExchangeProtocol (DLEP) is a radio aware routing (RAR) protocol.
References: `RFC <https://datatracker.ietf.org/doc/html/rfc8175>`_
.. _Proto 401:
`NDPI_PROTOCOL_BFD`
=====================
Bidirectional Forwarding Detection is a network protocol that is used to detect faults between two routers or switches.
References: `RFC <https://datatracker.ietf.org/doc/html/rfc5880>`_
.. _Proto 402:
`NDPI_PROTOCOL_NETEASE_GAMES`
============================
Traffic of various NetEase games.
References: `Main site <https://www.neteasegames.com/>`_
.. _Proto 403:
`NDPI_PROTOCOL_PATHOFEXILE`
============================
Path of Exile is a free-to-play online Action RPG.
References: `Main site <https://pathofexile.com/>`_
.. _Proto 404:
`NDPI_PROTOCOL_GOOGLE_CALL`
===========================
Audio/video calls made by (any) Google applications (i.e Google Meet).
References: `Main site: <https://meet.google.com/>`_
Notes:
- nDPI usually uses different protocol ids for the generic application traffic and for its "realtime" traffic (examples: NDPI_PROTOCOL_MEET/NDPI_PROTOCOL_GOOGLE_CALL, NDPI_PROTOCOL_WHATSAPP/NDPI_PROTOCOL_WHATSAPP_CALL, ...)
.. _Proto 405:
`NDPI_PROTOCOL_PFCP`
=====================
PFCP is a protocol used for communicating between control plane (CP) and user plane (UP) functions in 4G and 5G networks.
References: `Protocol Specs: <https://www.etsi.org/deliver/etsi_ts/129200_129299/129244/16.05.00_60/ts_129244v160500p.pdf>`_
.. _Proto 406:
`NDPI_PROTOCOL_FLUTE`
=====================
File Delivery over Unidirectional Transport.
References: `RFC <https://datatracker.ietf.org/doc/html/rfc6726>`_
.. _Proto 407:
`NDPI_PROTOCOL_LOLWILDRIFT`
============================
League of Legends: Wild Rift is a mobile MOBA game.
References: `Main site <https://wildrift.leagueoflegends.com/>`_
.. _Proto 408:
`NDPI_PROTOCOL_TESO`
============================
The Elder Scrolls Online is a MMORPG set in the fantasy world of Tamriel.
References: `Main site <https://www.elderscrollsonline.com/>`_
.. _Proto 409:
`NDPI_PROTOCOL_LDP`
=====================
The Label Distribution Protocol (LDP) is a routing protocol used to establish and maintain label-switched paths in a Multiprotocol Label Switching (MPLS) network.
References: `RFC <https://datatracker.ietf.org/doc/html/rfc5036>`_
.. _Proto 410:
`NDPI_PROTOCOL_KNXNET_IP`
=====================
KNXnet/IP is a building automation protocol that enables the exchange of data and control information over IP networks, extending the KNX standard for home and building automation.
References: `Paid Specs: <https://webstore.ansi.org/standards/ds/dsiso225102019>`_
.. _Proto 411:
`NDPI_PROTOCOL_BLUESKY`
======================
Bluesky, also known as Bluesky Social, is a decentralized microblogging social platform.
References: `Main site: <https://bsky.app/>`_
.. _Proto 412:
`NDPI_PROTOCOL_MASTODON`
=======================
Mastodon is free and open-source software for running self-hosted social networking services. It has microblogging features similar to Twitter.
References: `Main site: <https://joinmastodon.org/>`_
.. _Proto 413:
`NDPI_PROTOCOL_THREADS`
======================
Threads is an online social media and social networking service operated by Meta Platforms.
References: `Main site: <https://www.threads.net>`_
.. _Proto 414:
`NDPI_PROTOCOL_VIBER_VOIP`
=========================
Audio/video calls made using the Viber app.
References: `Wikipedia <https://en.wikipedia.org/wiki/Viber>`_
.. _Proto 415:
`NDPI_PROTOCOL_ZUG`
=========================
The ZUG protocol is part of the Casper 2.0 consensus model.
References: `Main Site <https://casperlabs.io>`_ and `Blog Post <https://casperlabs.io/blog/beyond-eth-30-theres-casper-20>`_
.. _Proto 416:
`NDPI_PROTOCOL_JRMI`
=========================
The JRMI protocol is the Java Remote Method Invocation protocol.
References: `Oracle site <https://docs.oracle.com/en/java/javase/21/docs/specs/rmi/protocol.html>`_
.. _Proto 417:
`NDPI_PROTOCOL_RIPE_ATLAS`
==========================
The RIPE Atlas probe protocol is used for the world's largest active Internet measurement network.
References: `Main Site <https://atlas.ripe.net/>`_ and `Documentation <https://ripe-atlas-tools.readthedocs.io/en/latest/index.html>`_
+4
View File
@@ -0,0 +1,4 @@
breathe
exhale
sphinx-rtd-theme
sphinxcontrib-swaggerdoc
+56
View File
@@ -0,0 +1,56 @@
What is nDPI
##############
nDPI is a DPI (Deep Packet Inspection) toolkit able to:
- Detect application protocol on traffic flows
- Analize encrypted traffic flows via ET (Encrypted Traffic Analysis)
- Extract selected protocol metadata from traffic
- Implement APIs for analysing traffic
Releases and Features
---------------------
nDPI development lifecycle is typically 6 to 9 months. The history of changes and features implemented by every release, is available on its `Changelog <https://github.com/ntop/nDPI/blob/dev/CHANGELOG.md>`_.
Installation
============
nDPI is open source and available on `GitHub
<https://github.com/ntop/nDPI>`_. In addition, pre-compiled, binary
nDPI packages are available both for Linux and other platforms. Installation
instructions for binary packages are available below.
Installing on Linux
-------------------
Installation instructions can be found at
http://packages.ntop.org/. Development and stable builds are
available. Stable builds are intended for production environments whereas
development builds are intended for testing or early feature access.
Software Updates
================
General instructions for updating the software can be found at
http://packages.ntop.org/ together with the installation instructions.
Depending on the Operating System, nDPI supports also automatic updates
through the GUI as described in the below sections.
Updating the Software on Linux
------------------------------
Instructions for updating the software via command line can be found
at http://packages.ntop.org/. For example on Ubuntu/Debian systems the
below commands will update the repository, check for updates and install
the latest software update if any:
.. code:: bash
apt-get update
apt-get upgrade
.. _AvailableVersions:
-12
View File
@@ -1,12 +0,0 @@
bin_PROGRAMS = ndpiReader
AM_CPPFLAGS = -I$(top_srcdir)/src/include @PCAP_INC@
AM_CFLAGS = @PTHREAD_CFLAGS@
LDADD = $(top_builddir)/src/lib/libndpi.la @JSON_C_LIB@ @PTHREAD_LIBS@ @PCAP_LIB@
AM_LDFLAGS = -static
ndpiReader_SOURCES = ndpiReader.c
ndpiReader.o: ndpiReader.c
+27
View File
@@ -0,0 +1,27 @@
#
# Run 'make -f Makefile.dpdk' to compile the DPDK examples
#
# See http://core.dpdk.org/doc/quick-start/ for DPDK installation and setup
#
ifeq ($(RTE_SDK),)
#$(error "Please define RTE_SDK environment variable")
RTE_SDK = $(HOME)/DPDK
RTE_TARGET = build
endif
# Default target, can be overridden by command line or environment
RTE_TARGET ?= x86_64-native-linuxapp-gcc
include $(RTE_SDK)/mk/rte.vars.mk
APP = ndpiReader.dpdk
LIBNDPI = $(PWD)/../src/lib/libndpi.a
SRCS-y := reader_util.c ndpiReader.c
CFLAGS += -g
CFLAGS += -Wno-strict-prototypes -Wno-missing-prototypes -Wno-missing-declarations -Wno-unused-parameter -I $(PWD)/../src/include @CFLAGS@ -DUSE_DPDK
LDLIBS = $(LIBNDPI) @PCAP_LIB@ @LIBS@ @ADDITIONAL_LIBS@ -lpthread @LDFLAGS@
include $(RTE_SDK)/mk/rte.extapp.mk
+103
View File
@@ -0,0 +1,103 @@
AR=@AR@
CC=@CC@
CXX=@CXX@
ifneq ($(OS),Windows_NT)
OS := $(shell uname)
endif
ifeq ($(OS),Darwin)
CC=clang -fno-color-diagnostics
endif
BUILD_MINGW=@BUILD_MINGW@
BUILD_MINGW_X64=@BUILD_MINGW_X64@
DISABLE_NPCAP=@DISABLE_NPCAP@
EXE_SUFFIX=@EXE_SUFFIX@
SRCHOME=../src
ifneq ($(OS),Windows_NT)
CFLAGS+=-fPIC -DPIC
endif
CFLAGS+=-I$(SRCHOME)/include @NDPI_CFLAGS@ @PCAP_INC@ @GPROF_CFLAGS@
LDFLAGS+=@NDPI_LDFLAGS@
LIBNDPI=$(SRCHOME)/lib/libndpi.a
LIBS=$(LIBNDPI) @PCAP_LIB@ @ADDITIONAL_LIBS@ @LIBS@ @GPROF_LIBS@
HEADERS=reader_util.h $(SRCHOME)/include/ndpi_api.h \
$(SRCHOME)/include/ndpi_typedefs.h $(SRCHOME)/include/ndpi_protocol_ids.h
PREFIX?=@prefix@
ifneq ($(BUILD_MINGW),)
ifeq ($(DISABLE_NPCAP),0)
CFLAGS+=-I@srcdir@/../windows/WpdPack/Include -I@srcdir@/../windows/WpdPack/Include/pcap
else
CFLAGS+=-DDISABLE_NPCAP
endif
ifeq ($(DISABLE_NPCAP),0)
ifneq ($(BUILD_MINGW_X64),)
LIBS+=@srcdir@/../windows/WpdPack/Lib/x64/wpcap.lib
else
LIBS+=@srcdir@/../windows/WpdPack/Lib/wpcap.lib
endif
endif
LIBS+=-Wl,-Bstatic -lpthread -Wl,-Bdynamic
else
LIBS+=-pthread
endif
CFLAGS+=-pthread
all: ndpiReader$(EXE_SUFFIX) @DPDK_TARGET@
EXECUTABLE_SOURCES := ndpiReader.c ndpiSimpleIntegration.c
COMMON_SOURCES := $(filter-out $(EXECUTABLE_SOURCES),$(wildcard *.c ))
libndpiReader.a: $(COMMON_SOURCES:%.c=%.o) $(LIBNDPI)
$(AR) rsv libndpiReader.a $(COMMON_SOURCES:%.c=%.o)
ndpiReader$(EXE_SUFFIX): libndpiReader.a $(LIBNDPI) ndpiReader.o
$(CC) $(CFLAGS) $(LDFLAGS) ndpiReader.o libndpiReader.a $(LIBS) -o $@
ndpiSimpleIntegration$(EXE_SUFFIX): ndpiSimpleIntegration.o
$(CC) $(CFLAGS) $(LDFLAGS) $< $(LIBS) -o $@
%.o: %.c $(HEADERS) Makefile
$(CC) $(CFLAGS) $(CPPFLAGS) -c $< -o $@
install: ndpiReader$(EXE_SUFFIX)
mkdir -p $(DESTDIR)$(PREFIX)/bin/
mkdir -p $(DESTDIR)$(PREFIX)/share/ndpi
cp ndpiReader$(EXE_SUFFIX) $(DESTDIR)$(PREFIX)/bin/
cp protos.txt $(DESTDIR)$(PREFIX)/share/ndpi/ndpiProtos.txt
cp mining_hosts.txt $(DESTDIR)$(PREFIX)/share/ndpi/ndpiCustomCategory.txt
[ -f ndpiSimpleIntegration$(EXE_SUFFIX) ] && cp ndpiSimpleIntegration$(EXE_SUFFIX) $(DESTDIR)$(PREFIX)/bin/ndpiSimpleIntegration$(EXE_SUFFIX) || true
[ -f build/app/ndpiReader.dpdk ] && cp build/app/ndpiReader.dpdk $(DESTDIR)$(PREFIX)/bin/ || true
[ -f ndpiReader.dpdk ] && cp ndpiReader.dpdk $(DESTDIR)$(PREFIX)/bin/ || true
dpdk:
$(MAKE) -f Makefile.dpdk
check:
true # nothing to do here
cppcheck:
cppcheck --template='{file}:{line}:{severity}:{message}' --quiet --enable=all --force -I$(SRCHOME)/include *.c
clean:
/bin/rm -f *.o ndpiReader ndpiSimpleIntegration ndpiReader$(EXE_SUFFIX) ndpiSimpleIntegration$(EXE_SUFFIX) ndpiReader.dpdk libndpiReader.a
/bin/rm -f .*.dpdk.cmd .*.o.cmd *.dpdk.map .*.o.d
/bin/rm -f _install _postbuild _postinstall _preinstall
/bin/rm -rf build
distdir:
cp README.DPDK '$(distdir)/'
find . -maxdepth 1 -type f -name '*.c' \
-o -name '*.h' \
-o -name '*.py' \
-o -name '*.txt' \
-o -name '*.csv' | xargs -I'{}' cp '{}' '$(distdir)/{}'
distclean: clean
/bin/rm -f Makefile.dpdk
/bin/rm -f Makefile
+31
View File
@@ -0,0 +1,31 @@
Prerequisites
-------------
You need to install and compile DPDK in your HOME directory as explained in
See http://core.dpdk.org/doc/quick-start/ for DPDK installation and setup
Once DPDK is built make sure to create a symbolic link
$ cd
$ ln -s dpdk-18.08 DPDK
so the build process will use the DPDK directory letting you have multiple
DPDK versions available on your system
Build
-----
Everything will happen automagically but if you want to do it by hand
do: make -f Makefile.dpdk
Run Application
---------------
Supposing to capture packets from device eno1 you can start the
application as follows:
sudo ./build/ndpiReader -c 1 --vdev=net_pcap0,iface=eno1 -- -v 1
NOTE:
- ndpiReader without DPDK support sits in this directory
- ndpiReader with DPDK support can be found inside the ./build directory
-20
View File
@@ -1,20 +0,0 @@
Microsoft Visual Studio Solution File, Format Version 11.00
# Visual C++ Express 2010
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pcapExample", "pcapExample\pcapExample.vcxproj", "{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Win32 = Debug|Win32
Release|Win32 = Release|Win32
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}.Debug|Win32.ActiveCfg = Debug|Win32
{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}.Debug|Win32.Build.0 = Debug|Win32
{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}.Release|Win32.ActiveCfg = Release|Win32
{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}.Release|Win32.Build.0 = Release|Win32
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
EndGlobal
Binary file not shown.
-11
View File
@@ -1,11 +0,0 @@
// pcapExample.cpp : Defines the entry point for the console application.
//
#include "stdafx.h"
int _tmain(int argc, _TCHAR* argv[])
{
return 0;
}
@@ -1,231 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|Win32">
<Configuration>Release</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
</ItemGroup>
<PropertyGroup Label="Globals">
<ProjectGuid>{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}</ProjectGuid>
<Keyword>Win32Proj</Keyword>
<RootNamespace>pcapExample</RootNamespace>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<WholeProgramOptimization>true</WholeProgramOptimization>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<PropertyGroup Label="UserMacros" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<LinkIncremental>true</LinkIncremental>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<LinkIncremental>false</LinkIncremental>
</PropertyGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<ClCompile>
<PrecompiledHeader>NotUsing</PrecompiledHeader>
<WarningLevel>Level3</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>C:\ntop\nprobe\nprobe-win-32;C:\ntop\winpcap\Include;C:\ntop\dependencies\nDPI\src\lib\third_party\include;C:\ntop\dependencies\nDPI\src\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<AdditionalDependencies>Wsock32.lib;Ws2_32.lib;C:\ntop\winpcap\Lib\Packet.lib;C:\ntop\winpcap\Lib\wpcap.lib;%(AdditionalDependencies)</AdditionalDependencies>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<ClCompile>
<WarningLevel>Level3</WarningLevel>
<PrecompiledHeader>Use</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
</Link>
</ItemDefinitionGroup>
<ItemGroup>
<ClCompile Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt.c" />
<ClCompile Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt1.c" />
<ClCompile Include="..\..\..\src\lib\ndpi_main.c" />
<ClCompile Include="..\..\..\src\lib\protocols\afp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\aimini.c" />
<ClCompile Include="..\..\..\src\lib\protocols\applejuice.c" />
<ClCompile Include="..\..\..\src\lib\protocols\armagetron.c" />
<ClCompile Include="..\..\..\src\lib\protocols\battlefield.c" />
<ClCompile Include="..\..\..\src\lib\protocols\bgp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\bittorrent.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ciscovpn.c" />
<ClCompile Include="..\..\..\src\lib\protocols\citrix.c" />
<ClCompile Include="..\..\..\src\lib\protocols\corba.c" />
<ClCompile Include="..\..\..\src\lib\protocols\crossfire.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dcerpc.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dhcp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dhcpv6.c" />
<ClCompile Include="..\..\..\src\lib\protocols\directconnect.c" />
<ClCompile Include="..\..\..\src\lib\protocols\directdownloadlink.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dns.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dofus.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dropbox.c" />
<ClCompile Include="..\..\..\src\lib\protocols\edonkey.c" />
<ClCompile Include="..\..\..\src\lib\protocols\fasttrack.c" />
<ClCompile Include="..\..\..\src\lib\protocols\feidian.c" />
<ClCompile Include="..\..\..\src\lib\protocols\fiesta.c" />
<ClCompile Include="..\..\..\src\lib\protocols\filetopia.c" />
<ClCompile Include="..\..\..\src\lib\protocols\flash.c" />
<ClCompile Include="..\..\..\src\lib\protocols\florensia.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ftp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\gadu_gadu.c" />
<ClCompile Include="..\..\..\src\lib\protocols\gnutella.c" />
<ClCompile Include="..\..\..\src\lib\protocols\gtp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\guildwars.c" />
<ClCompile Include="..\..\..\src\lib\protocols\h323.c" />
<ClCompile Include="..\..\..\src\lib\protocols\halflife2_and_mods.c" />
<ClCompile Include="..\..\..\src\lib\protocols\http.c" />
<ClCompile Include="..\..\..\src\lib\protocols\http_activesync.c" />
<ClCompile Include="..\..\..\src\lib\protocols\i23v5.c" />
<ClCompile Include="..\..\..\src\lib\protocols\iax.c" />
<ClCompile Include="..\..\..\src\lib\protocols\icecast.c" />
<ClCompile Include="..\..\..\src\lib\protocols\imesh.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ipp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\irc.c" />
<ClCompile Include="..\..\..\src\lib\protocols\jabber.c" />
<ClCompile Include="..\..\..\src\lib\protocols\kerberos.c" />
<ClCompile Include="..\..\..\src\lib\protocols\kontiki.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ldap.c" />
<ClCompile Include="..\..\..\src\lib\protocols\lotus_notes.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mail_imap.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mail_pop.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mail_smtp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\maplestory.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mdns.c" />
<ClCompile Include="..\..\..\src\lib\protocols\meebo.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mgcp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mms.c" />
<ClCompile Include="..\..\..\src\lib\protocols\msn.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mssql.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mysql.c" />
<ClCompile Include="..\..\..\src\lib\protocols\netbios.c" />
<ClCompile Include="..\..\..\src\lib\protocols\netflow.c" />
<ClCompile Include="..\..\..\src\lib\protocols\nfs.c" />
<ClCompile Include="..\..\..\src\lib\protocols\noe.c" />
<ClCompile Include="..\..\..\src\lib\protocols\non_tcp_udp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ntp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\openft.c" />
<ClCompile Include="..\..\..\src\lib\protocols\openvpn.c" />
<ClCompile Include="..\..\..\src\lib\protocols\oracle.c" />
<ClCompile Include="..\..\..\src\lib\protocols\oscar.c" />
<ClCompile Include="..\..\..\src\lib\protocols\pando.c" />
<ClCompile Include="..\..\..\src\lib\protocols\pcanywhere.c" />
<ClCompile Include="..\..\..\src\lib\protocols\popo.c" />
<ClCompile Include="..\..\..\src\lib\protocols\postgres.c" />
<ClCompile Include="..\..\..\src\lib\protocols\pplive.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ppstream.c" />
<ClCompile Include="..\..\..\src\lib\protocols\pptp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\qq.c" />
<ClCompile Include="..\..\..\src\lib\protocols\quake.c" />
<ClCompile Include="..\..\..\src\lib\protocols\quic.c" />
<ClCompile Include="..\..\..\src\lib\protocols\radius.c" />
<ClCompile Include="..\..\..\src\lib\protocols\rdp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\rsync.c" />
<ClCompile Include="..\..\..\src\lib\protocols\rtcp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\rtp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\rtsp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\sflow.c" />
<ClCompile Include="..\..\..\src\lib\protocols\shoutcast.c" />
<ClCompile Include="..\..\..\src\lib\protocols\sip.c" />
<ClCompile Include="..\..\..\src\lib\protocols\skinny.c" />
<ClCompile Include="..\..\..\src\lib\protocols\skype.c" />
<ClCompile Include="..\..\..\src\lib\protocols\smb.c" />
<ClCompile Include="..\..\..\src\lib\protocols\snmp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\socrates.c" />
<ClCompile Include="..\..\..\src\lib\protocols\sopcast.c" />
<ClCompile Include="..\..\..\src\lib\protocols\soulseek.c" />
<ClCompile Include="..\..\..\src\lib\protocols\spotify.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ssdp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ssh.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ssl.c" />
<ClCompile Include="..\..\..\src\lib\protocols\stealthnet.c" />
<ClCompile Include="..\..\..\src\lib\protocols\steam.c" />
<ClCompile Include="..\..\..\src\lib\protocols\stun.c" />
<ClCompile Include="..\..\..\src\lib\protocols\syslog.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tcp_udp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tds.c" />
<ClCompile Include="..\..\..\src\lib\protocols\teamspeak.c" />
<ClCompile Include="..\..\..\src\lib\protocols\teamviewer.c" />
<ClCompile Include="..\..\..\src\lib\protocols\telnet.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tftp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\thunder.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tor.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tvants.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tvuplayer.c" />
<ClCompile Include="..\..\..\src\lib\protocols\usenet.c" />
<ClCompile Include="..\..\..\src\lib\protocols\veohtv.c" />
<ClCompile Include="..\..\..\src\lib\protocols\viber.c" />
<ClCompile Include="..\..\..\src\lib\protocols\vmware.c" />
<ClCompile Include="..\..\..\src\lib\protocols\vnc.c" />
<ClCompile Include="..\..\..\src\lib\protocols\warcraft3.c" />
<ClCompile Include="..\..\..\src\lib\protocols\winmx.c" />
<ClCompile Include="..\..\..\src\lib\protocols\world_of_kung_fu.c" />
<ClCompile Include="..\..\..\src\lib\protocols\world_of_warcraft.c" />
<ClCompile Include="..\..\..\src\lib\protocols\xbox.c" />
<ClCompile Include="..\..\..\src\lib\protocols\xdmcp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\yahoo.c" />
<ClCompile Include="..\..\..\src\lib\protocols\zattoo.c" />
<ClCompile Include="..\..\..\src\lib\third_party\src\ahocorasick.c" />
<ClCompile Include="..\..\..\src\lib\third_party\src\node.c" />
<ClCompile Include="..\..\..\src\lib\third_party\src\sort.c" />
<ClCompile Include="..\..\pcapReader.c" />
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt.h" />
<ClInclude Include="..\..\..\src\include\linux_compat.h" />
<ClInclude Include="..\..\..\src\include\ndpi_api.h" />
<ClInclude Include="..\..\..\src\include\ndpi_debug_functions.h" />
<ClInclude Include="..\..\..\src\include\ndpi_define.h" />
<ClInclude Include="..\..\..\src\include\ndpi_macros.h" />
<ClInclude Include="..\..\..\src\include\ndpi_main.h" />
<ClInclude Include="..\..\..\src\include\ndpi_protocols.h" />
<ClInclude Include="..\..\..\src\include\ndpi_protocols_osdpi.h" />
<ClInclude Include="..\..\..\src\include\ndpi_protocol_history.h" />
<ClInclude Include="..\..\..\src\include\ndpi_public_functions.h" />
<ClInclude Include="..\..\..\src\include\ndpi_structs.h" />
<ClInclude Include="..\..\..\src\include\ndpi_utils.h" />
<ClInclude Include="..\..\..\src\lib\third_party\include\actypes.h" />
<ClInclude Include="..\..\..\src\lib\third_party\include\ahocorasick.h" />
<ClInclude Include="..\..\..\src\lib\third_party\include\node.h" />
<ClInclude Include="..\..\..\src\lib\third_party\include\sort.h" />
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
</Project>
@@ -1,465 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Filter Include="Source Files">
<UniqueIdentifier>{4FC737F1-C7A5-4376-A066-2A32D752A2FF}</UniqueIdentifier>
<Extensions>cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx</Extensions>
</Filter>
<Filter Include="Header Files">
<UniqueIdentifier>{93995380-89BD-4b04-88EB-625FBE52EBFB}</UniqueIdentifier>
<Extensions>h;hpp;hxx;hm;inl;inc;xsd</Extensions>
</Filter>
<Filter Include="Resource Files">
<UniqueIdentifier>{67DA6AB6-F800-4c08-8B7A-83BB121AAD01}</UniqueIdentifier>
<Extensions>rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav;mfcribbon-ms</Extensions>
</Filter>
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\pcapReader.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\ndpi_main.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\zattoo.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\afp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\aimini.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\applejuice.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\armagetron.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\battlefield.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\bgp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\bittorrent.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ciscovpn.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\citrix.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\corba.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\crossfire.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dcerpc.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dhcp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dhcpv6.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\directconnect.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\directdownloadlink.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dns.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dofus.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dropbox.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\edonkey.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\fasttrack.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\feidian.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\fiesta.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\filetopia.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\flash.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\florensia.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ftp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\gadu_gadu.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\gnutella.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\gtp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\guildwars.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\h323.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\halflife2_and_mods.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\http.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\http_activesync.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\i23v5.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\iax.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\icecast.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\imesh.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ipp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\irc.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\jabber.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\kerberos.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\kontiki.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ldap.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\lotus_notes.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mail_imap.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mail_pop.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mail_smtp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\maplestory.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mdns.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\meebo.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mgcp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mms.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\msn.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mssql.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mysql.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\netbios.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\netflow.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\nfs.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\noe.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\non_tcp_udp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ntp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\openft.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\openvpn.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\oracle.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\oscar.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\pando.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\pcanywhere.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\popo.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\postgres.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\pplive.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ppstream.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\pptp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\qq.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\quake.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\quic.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\radius.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\rdp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\rsync.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\rtcp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\rtp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\rtsp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\sflow.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\shoutcast.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\sip.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\skinny.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\skype.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\smb.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\snmp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\socrates.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\sopcast.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\soulseek.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\spotify.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ssdp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ssh.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ssl.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\stealthnet.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\steam.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\stun.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\syslog.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tcp_udp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tds.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\teamspeak.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\teamviewer.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\telnet.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tftp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\thunder.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tor.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tvants.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tvuplayer.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\usenet.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\veohtv.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\viber.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\vmware.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\vnc.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\warcraft3.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\winmx.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\world_of_kung_fu.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\world_of_warcraft.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\xbox.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\xdmcp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\yahoo.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\third_party\src\ahocorasick.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\third_party\src\node.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\third_party\src\sort.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt1.c">
<Filter>Source Files</Filter>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\src\lib\third_party\include\actypes.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\lib\third_party\include\ahocorasick.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\lib\third_party\include\node.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\lib\third_party\include\sort.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_api.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_debug_functions.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_define.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_macros.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_main.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_protocol_history.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_protocols.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_protocols_osdpi.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_public_functions.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_structs.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_utils.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\linux_compat.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt.h">
<Filter>Header Files</Filter>
</ClInclude>
</ItemGroup>
</Project>
@@ -1,3 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
</Project>
+6
View File
@@ -0,0 +1,6 @@
# Format: name\tcategory_id
internetbadguys.com 100
144.139.247.220 100
172.30.69.0/24 100
[2001:1670:8:40a6:a08e:332b:aa69:18dc] 100
[2001:db8:1::1]/127 100
+13
View File
@@ -0,0 +1,13 @@
#Format: <proto, param, value>
#proto might be empty
#Basic example of how to set nDPI configuration parameters via file
#We set only some default values
#See doc/configuration_parameters.md for a complete list and description of all the accepted knobs
packets_limit_per_flow,32
ookla,dpi.aggressiveness,0x1
tls,metadata.sha1_fingerprint,1
lru.bittorrent.ttl,0
+108
View File
@@ -0,0 +1,108 @@
################################################################
# abuse.ch Suricata JA3 Fingerprint Blacklist (CSV) #
# For Suricata 4.1.0 or newer #
# Last updated: 2021-08-03 14:33:44 UTC #
# #
# Terms Of Use: https://sslbl.abuse.ch/blacklist/ #
# For questions please contact sslbl [at] abuse.ch #
################################################################
#
# ja3_md5,Firstseen,Lastseen,Listingreason
b386946a5a44d1ddcc843bc75336dfce,2017-07-14 18:08:15,2019-07-27 20:42:54,Dridex
8991a387e4cc841740f25d6f5139f92d,2017-07-14 19:02:03,2019-07-28 00:34:38,Adware
cb98a24ee4b9134448ffb5714fd870ac,2017-07-14 19:48:28,2019-05-22 03:22:38,Dridex
1aa7bf8b97e540ca5edd75f7b8384bfa,2017-07-14 20:23:38,2019-07-28 01:38:22,TrickBot
3d89c0dfb1fa44911b8fa7523ef8dedb,2017-07-15 04:23:45,2021-02-01 18:23:25,Adware
bc6c386f480ee97b9d9e52d472b772d8,2017-07-15 10:57:38,2021-03-13 07:33:39,Adware
8f52d1ce303fb4a6515836aec3cc16b1,2017-07-15 19:05:11,2019-07-27 20:00:57,TrickBot
d6f04b5a910115f4b50ecec09d40a1df,2017-07-15 19:42:24,2018-10-14 08:12:51,Dridex
35c0a31c481927f022a3b530255ac080,2017-07-15 19:43:19,2021-04-10 12:54:04,Tofsee
e330bca99c8a5256ae126a55c4c725c5,2017-07-15 19:59:29,2021-01-13 00:29:37,Adware
d551fafc4f40f1dec2bb45980bfa9492,2017-07-15 19:59:29,2020-11-16 13:06:20,Adware
83e04bc58d402f9633983cbf22724b02,2017-07-16 01:32:03,2021-03-02 04:07:43,Adware
b8f81673c0e1d29908346f3bab892b9b,2017-07-16 01:32:03,2021-03-02 04:07:36,Adware
70722097d1fe1d78d8c2164640ab6df4,2017-07-16 02:39:08,2021-05-04 09:52:20,Tofsee
9c2589e1c0e9f533a022c6205f9719e1,2017-07-16 08:37:17,2021-07-25 08:33:18,Adware
849b04bdbd1d2b983f6e8a457e0632a8,2017-07-16 08:37:17,2021-07-25 08:33:18,Adware
16efcf0e00504ddfedde13bfea997952,2017-07-16 19:45:45,2020-12-23 15:10:32,Adware
4d7a28d6f2263ed61de88ca66eb011e3,2017-07-16 21:20:29,2020-12-08 18:10:55,Tofsee
550dce18de1bb143e69d6dd9413b8355,2017-07-16 22:17:20,2018-12-21 07:04:50,Adware
c50f6a8b9173676b47ba6085bd0c6cee,2017-07-16 22:38:41,2019-05-21 09:42:17,TrickBot
590a232d04d56409fab72e752a8a2634,2017-07-18 18:53:24,2020-10-11 20:48:33,Tofsee
51a7ad14509fd614c7bb3a50c4982b8c,2017-07-19 07:28:19,2019-07-14 11:58:32,JBifrost
96eba628dcb2b47607192ba74a3b55ba,2017-07-19 18:53:48,2021-07-31 01:48:32,Tofsee
df5c30e670dba99f9270ed36060cf054,2017-07-20 17:44:07,2018-04-11 15:57:59,Tofsee
098f55e27d8c4b0a590102cbdb3a5f3a,2017-07-21 09:52:01,2019-04-08 01:09:54,Adware
29085f03f8e8a03f0b399c5c7cf0b0b8,2017-07-22 14:07:36,2021-04-11 06:42:45,Adware
46efd49abcca8ea9baa932da68fdb529,2017-07-22 14:07:36,2021-04-11 05:54:57,Adware
d7150af4514b868defb854db0f62a441,2017-07-23 09:39:24,2018-07-24 01:04:58,Tofsee
03e186a7f83285e93341de478334006e,2017-07-24 18:17:14,2021-03-20 07:45:40,Tofsee
3cda52da4ade09f1f781ad2e82dcfa20,2017-07-30 18:41:36,2019-05-21 17:34:18,Quakbot
b13d01846ad7a14a70bf030a16775c78,2017-08-08 07:12:49,2021-04-10 00:03:33,Adware
1543a7c46633acf71e8401baccbd0568,2017-08-08 21:32:28,2021-04-08 19:50:46,Tofsee
1d095e68489d3c535297cd8dffb06cb9,2017-08-12 19:56:28,2020-10-28 11:06:23,Tofsee
698e36219f3979420fa2581b21dac7ec,2017-08-28 12:20:47,2020-12-31 02:06:31,Adware
93d056782d649deb51cda44ecb714bb0,2017-08-28 12:20:47,2019-04-15 23:47:27,Adware
1712287800ac91b34cadd5884ce85568,2017-08-28 16:01:59,2021-07-28 14:16:00,TorrentLocker
5e573c9c9f8ba720ef9b18e9fce2e2f7,2017-08-30 13:44:56,2021-03-13 07:33:38,Adware
f6fd83a21f9f3c5f9ff7b5c63bbc179d,2017-10-20 08:03:21,2018-11-06 06:42:12,Adware
92579701f145605e9edc0b01a901c6d5,2017-10-23 00:10:48,2021-07-25 17:07:34,Adware
a61299f9b501adcf680b9275d79d4ac6,2017-11-04 18:03:59,2020-04-21 17:08:24,Tofsee
b2b61db7b9490a60d270ccb20b462826,2017-11-14 20:12:03,2021-06-06 20:27:10,Adware
7dcce5b76c8b17472d024758970a406b,2017-11-22 12:42:46,2021-03-16 12:53:35,Tofsee
534ce2dbc413c68e908363b5df0ae5e0,2017-12-22 09:36:21,2019-07-27 15:22:33,TrickBot
fb00055a1196aeea8d1bc609885ba953,2018-01-01 22:49:25,2019-04-09 06:58:58,TrickBot
a50a861119aceb0ccc74902e8fddb618,2018-01-02 08:16:23,2018-07-05 02:33:08,Tofsee
e7643725fcff971e3051fe0e47fc2c71,2018-01-31 08:06:13,2020-03-25 16:19:48,Tofsee
7c410ce832e848a3321432c9a82e972b,2018-01-31 20:04:25,2021-08-01 06:13:14,Tofsee
da949afd9bd6df820730f8f171584a71,2018-02-03 05:19:37,2021-03-08 22:10:10,Tofsee
906004246f3ba5e755b043c057254a29,2018-03-11 08:25:38,2018-04-14 00:59:16,Tofsee
fd80fa9c6120cdeea8520510f3c644ac,2018-03-11 09:34:30,2021-08-11 12:34:00,Tofsee
b90bdbe961a648f0427db21aaa6ccb59,2018-03-11 10:37:43,2020-05-29 23:39:01,Tofsee
1fe4c7a3544eb27afec2adfb3a3dbf60,2018-03-11 19:23:08,2021-08-09 11:42:58,Tofsee
c201b92f8b483fa388be174d6689f534,2018-03-12 13:43:52,2021-01-28 06:17:06,Gozi
9f62c4f26b90d3d757bea609e82f2eaf,2018-03-13 06:23:41,2021-05-20 23:01:06,Tofsee
1be3ecebe5aa9d3654e6e703d81f6928,2018-03-13 11:50:02,2021-08-11 13:02:35,Ransomware.Troldesh
e3b2ab1f9a56f2fb4c9248f2f41631fa,2018-03-15 01:06:34,2021-07-02 21:51:49,Tofsee
dff8a0aa1c904aaea76c5bf624e88333,2018-03-18 09:41:15,2020-10-27 09:50:24,Tofsee
17fd49722f8d11f3d76dce84f8e099a7,2018-03-19 23:02:27,2021-08-01 00:40:52,Tofsee
911479ac8a0813ed1241b3686ccdade9,2018-03-19 23:24:59,2020-03-30 04:09:18,Tofsee
c5deb9465d47232dd48772f9c4d14679,2018-03-22 15:42:48,2021-03-23 00:34:25,Tofsee
f22bdd57e3a52de86cda40da2d84e83b,2018-03-27 13:40:19,2019-01-20 14:31:39,Tofsee
d18a4da84af59e1108862a39bae7c9d4,2018-04-03 00:40:51,2021-02-06 01:53:12,Tofsee
2d8794cb7b52b777bee2695e79c15760,2018-04-04 06:56:37,2021-07-26 08:07:00,Ransomware
40adfd923eb82b89d8836ba37a19bca1,2018-04-15 15:49:08,2021-04-11 04:42:47,CoinMiner
1aee0238942d453d679fc1e37a303387,2018-05-13 01:59:49,2021-07-30 12:27:07,Tofsee
2092e1fffb45d7e4a19a57f9bc5e203a,2018-05-16 21:59:36,2018-09-05 01:58:33,Adware
bffa4501966196d3d6e90cee1f88fc89,2018-06-07 15:08:04,2020-03-16 00:03:44,Tofsee
807fca46d9d0cf63adf4e5e80e414bbe,2018-06-07 16:51:03,2021-08-07 03:15:42,Tofsee
fb58831f892190644fe44e25bc830b45,2018-06-08 12:07:59,2021-07-20 01:39:05,Adware
0cc1e84568e471aa1d62ad4158ade6b5,2018-06-24 10:50:47,2021-06-21 02:35:57,Tofsee
d2935c58fe676744fecc8614ee5356c7,2018-08-14 21:48:41,2021-08-11 11:54:42,Adwind
8916410db85077a5460817142dcbc8de,2018-08-21 12:32:28,2021-08-11 15:00:50,TrickBot
c5235d3a8b9934b7fbbd204d50bc058d,2018-08-23 17:36:08,2019-10-13 05:11:09,Gootkit
57f3642b4e37e28f5cbe3020c9331b4c,2018-08-28 15:54:53,2021-08-11 13:05:18,Gozi
e62a5f4d538cbf169c2af71bec2399b4,2018-08-30 15:45:40,2021-08-11 09:48:52,TrickBot
51c64c77e60f3980eea90869b68c58a8,2018-08-30 21:04:57,2021-08-11 08:13:08,Dridex
7691297bcb20a41233fd0a0baa0a3628,2018-09-17 02:50:05,2021-08-11 12:20:33,Adware
7dd50e112cd23734a310b90f6f44a7cd,2018-09-17 17:54:58,2021-08-01 11:28:46,Quakbot
52c7396a501e4fecbdfa99c5408334ac,2018-09-18 00:29:04,2019-12-03 17:24:02,Tofsee
fc54e0d16d9764783542f0146a98b300,2018-09-24 12:33:44,2021-08-11 12:51:10,AsyncRAT
f735bbc6b69723b9df7b0e7ef27872af,2018-10-02 18:04:16,2021-08-11 07:25:14,TrickBot
49ed2ef3f1321e5f044f1e71b0e6fdd5,2018-10-02 18:04:17,2021-08-08 22:08:01,TrickBot
d76ee64fb7273733cbe455ac81c292e6,2018-11-16 13:26:39,2018-11-18 19:19:36,Tofsee
8f6c918dcb585ebbea05e2cc94530e3d,2018-11-16 13:26:41,2020-05-06 15:45:21,Tofsee
34f14a69ad7009ca5863379218af17f3,2018-11-17 05:17:22,2021-01-28 08:19:18,Tofsee
c2b4710c6888a5d47befe865c8e6fb19,2018-11-29 20:46:04,2021-08-03 23:37:22,Tofsee
decfb48a53789ebe081b88aabb58ee34,2018-12-21 09:06:16,2021-06-14 05:27:16,Adwind
08a8a4e85b25ac42e1490bc85cfdb5ce,2019-01-30 02:48:34,2020-10-27 09:50:19,Tofsee
c0220cd64849a629397a9cb68f78a0ea,2019-03-24 00:12:32,2021-07-31 00:26:06,Tofsee
7a29c223fb122ec64d10f0a159e07996,2019-06-09 22:55:29,2020-10-27 09:50:26,None
70a04365be5bbd4653698bebeb43ce68,2019-07-02 06:26:56,2020-05-30 04:19:00,Tofsee
d81d654effb94714a4086734fa0adad9,2019-07-16 23:29:02,2020-10-27 09:50:21,Tofsee
25d74b7b4b779eb1efd4b31d26d651c6,2019-08-03 20:15:33,2020-07-14 21:43:25,Tofsee
fc2299d5b2964cd242c5a2c8c531a5f0,2019-08-09 23:56:32,2021-08-11 02:20:22,Tofsee
32926ca3e59f0413d0b98725454594f5,2019-09-12 06:56:10,2021-04-08 19:50:43,Tofsee
ffefafdb86336d057eda5fdf02b3d5ce,2019-10-26 07:31:49,2020-07-25 00:14:09,Tofsee
8515076cbbca9dce33151b798f782456,2020-12-27 16:53:04,2021-08-11 15:06:36,BitRAT
# END (97) entries
1 ################################################################
2 # abuse.ch Suricata JA3 Fingerprint Blacklist (CSV) #
3 # For Suricata 4.1.0 or newer #
4 # Last updated: 2021-08-03 14:33:44 UTC #
5 # #
6 # Terms Of Use: https://sslbl.abuse.ch/blacklist/ #
7 # For questions please contact sslbl [at] abuse.ch #
8 ################################################################
9 #
10 # ja3_md5,Firstseen,Lastseen,Listingreason
11 b386946a5a44d1ddcc843bc75336dfce,2017-07-14 18:08:15,2019-07-27 20:42:54,Dridex
12 8991a387e4cc841740f25d6f5139f92d,2017-07-14 19:02:03,2019-07-28 00:34:38,Adware
13 cb98a24ee4b9134448ffb5714fd870ac,2017-07-14 19:48:28,2019-05-22 03:22:38,Dridex
14 1aa7bf8b97e540ca5edd75f7b8384bfa,2017-07-14 20:23:38,2019-07-28 01:38:22,TrickBot
15 3d89c0dfb1fa44911b8fa7523ef8dedb,2017-07-15 04:23:45,2021-02-01 18:23:25,Adware
16 bc6c386f480ee97b9d9e52d472b772d8,2017-07-15 10:57:38,2021-03-13 07:33:39,Adware
17 8f52d1ce303fb4a6515836aec3cc16b1,2017-07-15 19:05:11,2019-07-27 20:00:57,TrickBot
18 d6f04b5a910115f4b50ecec09d40a1df,2017-07-15 19:42:24,2018-10-14 08:12:51,Dridex
19 35c0a31c481927f022a3b530255ac080,2017-07-15 19:43:19,2021-04-10 12:54:04,Tofsee
20 e330bca99c8a5256ae126a55c4c725c5,2017-07-15 19:59:29,2021-01-13 00:29:37,Adware
21 d551fafc4f40f1dec2bb45980bfa9492,2017-07-15 19:59:29,2020-11-16 13:06:20,Adware
22 83e04bc58d402f9633983cbf22724b02,2017-07-16 01:32:03,2021-03-02 04:07:43,Adware
23 b8f81673c0e1d29908346f3bab892b9b,2017-07-16 01:32:03,2021-03-02 04:07:36,Adware
24 70722097d1fe1d78d8c2164640ab6df4,2017-07-16 02:39:08,2021-05-04 09:52:20,Tofsee
25 9c2589e1c0e9f533a022c6205f9719e1,2017-07-16 08:37:17,2021-07-25 08:33:18,Adware
26 849b04bdbd1d2b983f6e8a457e0632a8,2017-07-16 08:37:17,2021-07-25 08:33:18,Adware
27 16efcf0e00504ddfedde13bfea997952,2017-07-16 19:45:45,2020-12-23 15:10:32,Adware
28 4d7a28d6f2263ed61de88ca66eb011e3,2017-07-16 21:20:29,2020-12-08 18:10:55,Tofsee
29 550dce18de1bb143e69d6dd9413b8355,2017-07-16 22:17:20,2018-12-21 07:04:50,Adware
30 c50f6a8b9173676b47ba6085bd0c6cee,2017-07-16 22:38:41,2019-05-21 09:42:17,TrickBot
31 590a232d04d56409fab72e752a8a2634,2017-07-18 18:53:24,2020-10-11 20:48:33,Tofsee
32 51a7ad14509fd614c7bb3a50c4982b8c,2017-07-19 07:28:19,2019-07-14 11:58:32,JBifrost
33 96eba628dcb2b47607192ba74a3b55ba,2017-07-19 18:53:48,2021-07-31 01:48:32,Tofsee
34 df5c30e670dba99f9270ed36060cf054,2017-07-20 17:44:07,2018-04-11 15:57:59,Tofsee
35 098f55e27d8c4b0a590102cbdb3a5f3a,2017-07-21 09:52:01,2019-04-08 01:09:54,Adware
36 29085f03f8e8a03f0b399c5c7cf0b0b8,2017-07-22 14:07:36,2021-04-11 06:42:45,Adware
37 46efd49abcca8ea9baa932da68fdb529,2017-07-22 14:07:36,2021-04-11 05:54:57,Adware
38 d7150af4514b868defb854db0f62a441,2017-07-23 09:39:24,2018-07-24 01:04:58,Tofsee
39 03e186a7f83285e93341de478334006e,2017-07-24 18:17:14,2021-03-20 07:45:40,Tofsee
40 3cda52da4ade09f1f781ad2e82dcfa20,2017-07-30 18:41:36,2019-05-21 17:34:18,Quakbot
41 b13d01846ad7a14a70bf030a16775c78,2017-08-08 07:12:49,2021-04-10 00:03:33,Adware
42 1543a7c46633acf71e8401baccbd0568,2017-08-08 21:32:28,2021-04-08 19:50:46,Tofsee
43 1d095e68489d3c535297cd8dffb06cb9,2017-08-12 19:56:28,2020-10-28 11:06:23,Tofsee
44 698e36219f3979420fa2581b21dac7ec,2017-08-28 12:20:47,2020-12-31 02:06:31,Adware
45 93d056782d649deb51cda44ecb714bb0,2017-08-28 12:20:47,2019-04-15 23:47:27,Adware
46 1712287800ac91b34cadd5884ce85568,2017-08-28 16:01:59,2021-07-28 14:16:00,TorrentLocker
47 5e573c9c9f8ba720ef9b18e9fce2e2f7,2017-08-30 13:44:56,2021-03-13 07:33:38,Adware
48 f6fd83a21f9f3c5f9ff7b5c63bbc179d,2017-10-20 08:03:21,2018-11-06 06:42:12,Adware
49 92579701f145605e9edc0b01a901c6d5,2017-10-23 00:10:48,2021-07-25 17:07:34,Adware
50 a61299f9b501adcf680b9275d79d4ac6,2017-11-04 18:03:59,2020-04-21 17:08:24,Tofsee
51 b2b61db7b9490a60d270ccb20b462826,2017-11-14 20:12:03,2021-06-06 20:27:10,Adware
52 7dcce5b76c8b17472d024758970a406b,2017-11-22 12:42:46,2021-03-16 12:53:35,Tofsee
53 534ce2dbc413c68e908363b5df0ae5e0,2017-12-22 09:36:21,2019-07-27 15:22:33,TrickBot
54 fb00055a1196aeea8d1bc609885ba953,2018-01-01 22:49:25,2019-04-09 06:58:58,TrickBot
55 a50a861119aceb0ccc74902e8fddb618,2018-01-02 08:16:23,2018-07-05 02:33:08,Tofsee
56 e7643725fcff971e3051fe0e47fc2c71,2018-01-31 08:06:13,2020-03-25 16:19:48,Tofsee
57 7c410ce832e848a3321432c9a82e972b,2018-01-31 20:04:25,2021-08-01 06:13:14,Tofsee
58 da949afd9bd6df820730f8f171584a71,2018-02-03 05:19:37,2021-03-08 22:10:10,Tofsee
59 906004246f3ba5e755b043c057254a29,2018-03-11 08:25:38,2018-04-14 00:59:16,Tofsee
60 fd80fa9c6120cdeea8520510f3c644ac,2018-03-11 09:34:30,2021-08-11 12:34:00,Tofsee
61 b90bdbe961a648f0427db21aaa6ccb59,2018-03-11 10:37:43,2020-05-29 23:39:01,Tofsee
62 1fe4c7a3544eb27afec2adfb3a3dbf60,2018-03-11 19:23:08,2021-08-09 11:42:58,Tofsee
63 c201b92f8b483fa388be174d6689f534,2018-03-12 13:43:52,2021-01-28 06:17:06,Gozi
64 9f62c4f26b90d3d757bea609e82f2eaf,2018-03-13 06:23:41,2021-05-20 23:01:06,Tofsee
65 1be3ecebe5aa9d3654e6e703d81f6928,2018-03-13 11:50:02,2021-08-11 13:02:35,Ransomware.Troldesh
66 e3b2ab1f9a56f2fb4c9248f2f41631fa,2018-03-15 01:06:34,2021-07-02 21:51:49,Tofsee
67 dff8a0aa1c904aaea76c5bf624e88333,2018-03-18 09:41:15,2020-10-27 09:50:24,Tofsee
68 17fd49722f8d11f3d76dce84f8e099a7,2018-03-19 23:02:27,2021-08-01 00:40:52,Tofsee
69 911479ac8a0813ed1241b3686ccdade9,2018-03-19 23:24:59,2020-03-30 04:09:18,Tofsee
70 c5deb9465d47232dd48772f9c4d14679,2018-03-22 15:42:48,2021-03-23 00:34:25,Tofsee
71 f22bdd57e3a52de86cda40da2d84e83b,2018-03-27 13:40:19,2019-01-20 14:31:39,Tofsee
72 d18a4da84af59e1108862a39bae7c9d4,2018-04-03 00:40:51,2021-02-06 01:53:12,Tofsee
73 2d8794cb7b52b777bee2695e79c15760,2018-04-04 06:56:37,2021-07-26 08:07:00,Ransomware
74 40adfd923eb82b89d8836ba37a19bca1,2018-04-15 15:49:08,2021-04-11 04:42:47,CoinMiner
75 1aee0238942d453d679fc1e37a303387,2018-05-13 01:59:49,2021-07-30 12:27:07,Tofsee
76 2092e1fffb45d7e4a19a57f9bc5e203a,2018-05-16 21:59:36,2018-09-05 01:58:33,Adware
77 bffa4501966196d3d6e90cee1f88fc89,2018-06-07 15:08:04,2020-03-16 00:03:44,Tofsee
78 807fca46d9d0cf63adf4e5e80e414bbe,2018-06-07 16:51:03,2021-08-07 03:15:42,Tofsee
79 fb58831f892190644fe44e25bc830b45,2018-06-08 12:07:59,2021-07-20 01:39:05,Adware
80 0cc1e84568e471aa1d62ad4158ade6b5,2018-06-24 10:50:47,2021-06-21 02:35:57,Tofsee
81 d2935c58fe676744fecc8614ee5356c7,2018-08-14 21:48:41,2021-08-11 11:54:42,Adwind
82 8916410db85077a5460817142dcbc8de,2018-08-21 12:32:28,2021-08-11 15:00:50,TrickBot
83 c5235d3a8b9934b7fbbd204d50bc058d,2018-08-23 17:36:08,2019-10-13 05:11:09,Gootkit
84 57f3642b4e37e28f5cbe3020c9331b4c,2018-08-28 15:54:53,2021-08-11 13:05:18,Gozi
85 e62a5f4d538cbf169c2af71bec2399b4,2018-08-30 15:45:40,2021-08-11 09:48:52,TrickBot
86 51c64c77e60f3980eea90869b68c58a8,2018-08-30 21:04:57,2021-08-11 08:13:08,Dridex
87 7691297bcb20a41233fd0a0baa0a3628,2018-09-17 02:50:05,2021-08-11 12:20:33,Adware
88 7dd50e112cd23734a310b90f6f44a7cd,2018-09-17 17:54:58,2021-08-01 11:28:46,Quakbot
89 52c7396a501e4fecbdfa99c5408334ac,2018-09-18 00:29:04,2019-12-03 17:24:02,Tofsee
90 fc54e0d16d9764783542f0146a98b300,2018-09-24 12:33:44,2021-08-11 12:51:10,AsyncRAT
91 f735bbc6b69723b9df7b0e7ef27872af,2018-10-02 18:04:16,2021-08-11 07:25:14,TrickBot
92 49ed2ef3f1321e5f044f1e71b0e6fdd5,2018-10-02 18:04:17,2021-08-08 22:08:01,TrickBot
93 d76ee64fb7273733cbe455ac81c292e6,2018-11-16 13:26:39,2018-11-18 19:19:36,Tofsee
94 8f6c918dcb585ebbea05e2cc94530e3d,2018-11-16 13:26:41,2020-05-06 15:45:21,Tofsee
95 34f14a69ad7009ca5863379218af17f3,2018-11-17 05:17:22,2021-01-28 08:19:18,Tofsee
96 c2b4710c6888a5d47befe865c8e6fb19,2018-11-29 20:46:04,2021-08-03 23:37:22,Tofsee
97 decfb48a53789ebe081b88aabb58ee34,2018-12-21 09:06:16,2021-06-14 05:27:16,Adwind
98 08a8a4e85b25ac42e1490bc85cfdb5ce,2019-01-30 02:48:34,2020-10-27 09:50:19,Tofsee
99 c0220cd64849a629397a9cb68f78a0ea,2019-03-24 00:12:32,2021-07-31 00:26:06,Tofsee
100 7a29c223fb122ec64d10f0a159e07996,2019-06-09 22:55:29,2020-10-27 09:50:26,None
101 70a04365be5bbd4653698bebeb43ce68,2019-07-02 06:26:56,2020-05-30 04:19:00,Tofsee
102 d81d654effb94714a4086734fa0adad9,2019-07-16 23:29:02,2020-10-27 09:50:21,Tofsee
103 25d74b7b4b779eb1efd4b31d26d651c6,2019-08-03 20:15:33,2020-07-14 21:43:25,Tofsee
104 fc2299d5b2964cd242c5a2c8c531a5f0,2019-08-09 23:56:32,2021-08-11 02:20:22,Tofsee
105 32926ca3e59f0413d0b98725454594f5,2019-09-12 06:56:10,2021-04-08 19:50:43,Tofsee
106 ffefafdb86336d057eda5fdf02b3d5ce,2019-10-26 07:31:49,2020-07-25 00:14:09,Tofsee
107 8515076cbbca9dce33151b798f782456,2020-12-27 16:53:04,2021-08-11 15:06:36,BitRAT
108 # END (97) entries
+262
View File
@@ -0,0 +1,262 @@
# NoCoin Filter List
# Blocking Web Browser Bitcoin Mining
#
# Homepage: https://github.com/hoshsadiq/adblock-nocoin-list/
# Contribute: https://github.com/hoshsadiq/adblock-nocoin-list/issues
# Changelog: https://github.com/hoshsadiq/adblock-nocoin-list/commits/master/hosts.txt
# License: https://mit-license.org/
#
# Last modified: 24 April 2018
#
cnhv.co 99
coin-hive.com 99
coinhive.com 99
authedmine.com 99
gus.host 99
load.jsecoin.com 99
miner.pr0gramm.com 99
minemytraffic.com 99
ppoi.org 99
projectpoi.com 99
crypto-loot.com 99
coinerra.com 99
coin-have.com 99
minero.pw 99
minero-proxy-01.now.sh 99
minero-proxy-02.now.sh 99
minero-proxy-03.now.sh 99
api.inwemo.com 99
rocks.io 99
adminer.com 99
ad-miner.com 99
jsccnn.com 99
jscdndel.com 99
coinhiveproxy.com 99
coinblind.com 99
coinnebula.com 99
monerominer.rocks 99
cdn.cloudcoins.co 99
coinlab.biz 99
go.megabanners.cf 99
baiduccdn1.com 99
wsp.marketgid.com 99
papoto.com 99
flare-analytics.com 99
www.sparechange.io 99
static.sparechange.io 99
miner.nablabee.com 99
m.anyfiles.ovh 99
www.freecontent.bid 99
www.freecontent.loan 99
www.freecontent.win 99
www.cryptonoter.com 99
www.mutuza.win 99
cryweb.github.io 99
crywebber.github.io 99
crypto-webminer.com 99
cdn.adless.io 99
hegrinhar.com 99
verresof.com 99
hemnes.win 99
tidafors.xyz 99
moneone.ga 99
plexcoin.info 99
www.monkeyminer.net 99
go2.mercy.ga 99
coinpirate.cf 99
d.cpufan.club 99
krb.devphp.org.ua 99
nfwebminer.com 99
cfcdist.gdn 99
node.cfcdist.gdn 99
webxmr.com 99
xmr.mining.best 99
webminepool.com 99
webminepool.tk 99
hive.tubetitties.com 99
playerassets.info 99
tokyodrift.ga 99
webassembly.stream 99
okeyletsgo.ml 99
candid.zone 99
webmine.pro 99
andlache.com 99
bablace.com 99
bewaslac.com 99
biberukalap.com 99
bowithow.com 99
butcalve.com 99
evengparme.com 99
gridiogrid.com 99
hatcalter.com 99
kedtise.com 99
ledinund.com 99
nathetsof.com 99
renhertfo.com 99
rintindown.com 99
sparnove.com 99
witthethim.com 99
1q2w3.fun 99
1q2w3.me 99
cryptoloot.pro 99
bjorksta.men 99
crypto.csgocpu.com 99
noblock.pro 99
miner.cryptobara.com 99
digger.cryptobara.com 99
dev.cryptobara.com 99
reservedoffers.club 99
mine.torrent.pw 99
host.d-ns.ga 99
abc.pema.cl 99
mine.nahnoji.cz 99
webmine.cz 99
intactoffers.club 99
analytics.blue 99
smectapop12.pl 99
berserkpl.net.pl 99
hodlers.party 99
hodling.faith 99
chainblock.science 99
minescripts.info 99
cdn.minescripts.info 99
miner.nablabee.com 99
wss.nablabee.com 99
clickwith.bid 99
dronml.ml 99
niematego.tk 99
tulip18.com 99
p.estream.to 99
didnkinrab.com 99
ledhenone.com 99
losital.ru 99
mebablo.com 99
moonsade.com 99
nebabrop.com 99
pearno.com 99
rintinwa.com 99
willacrit.com 99
www2.adfreetv.ch 99
minr.pw 99
new.minr.pw 99
test.minr.pw 99
staticsfs.host 99
cdn-code.host 99
g-content.bid 99
ad.g-content.bid 99
cdn.static-cnt.bid 99
cnt.statistic.date 99
cdn.jquery-uim.download 99
cdn-jquery.host 99
p1.interestingz.pw 99
kippbeak.cf 99
pasoherb.gq 99
axoncoho.tk 99
depttake.ga 99
flophous.cf 99
pr0gram.org 99
authedmine.eu 99
www.monero-miner.com 99
www.datasecu.download 99
www.jquery-cdn.download 99
www.etzbnfuigipwvs.ru 99
www.terethat.ru 99
freshrefresher.com 99
api.pzoifaum.info 99
ws.pzoifaum.info 99
api.bhzejltg.info 99
ws.bhzejltg.info 99
d.cfcnet.top 99
vip.cfcnet.top 99
eu.cfcnet.top 99
as.cfcnet.top 99
us.cfcnet.top 99
eu.cfcdist.loan 99
as.cfcdist.loan 99
us.cfcdist.loan 99
gustaver.ddns.net 99
worker.salon.com 99
s2.appelamule.com 99
mepirtedic.com 99
cdn.streambeam.io 99
adzjzewsma.cf 99
ffinwwfpqi.gq 99
ininmacerad.pro 99
mhiobjnirs.gq 99
open-hive-server-1.pp.ua 99
pool.hws.ru 99
pool.etn.spacepools.org 99
api.aalbbh84.info 99
www.aymcsx.ru 99
aeros01.tk 99
aeros02.tk 99
aeros03.tk 99
aeros04.tk 99
aeros05.tk 99
aeros06.tk 99
aeros07.tk 99
aeros08.tk 99
aeros09.tk 99
aeros10.tk 99
aeros11.tk 99
aeros12.tk 99
npcdn1.now.sh 99
mxcdn2.now.sh 99
sxcdn6.now.sh 99
mxcdn1.now.sh 99
sxcdn02.now.sh 99
sxcdn4.now.sh 99
jqcdn2.herokuapp.com 99
sxcdn1.herokuapp.com 99
sxcdn5.herokuapp.com 99
wpcdn1.herokuapp.com 99
jqcdn01.herokuapp.com 99
jqcdn03.herokuapp.com 99
1q2w3.website 99
video.videos.vidto.me 99
video.streaming.estream.to 99
eth-pocket.de 99
xvideosharing.site 99
bestcoinsignals.com 99
eucsoft.com 99
traviilo.com 99
wasm24.ru 99
xmr.cool 99
api.netflare.info 99
cdnjs.cloudflane.com 99
www.cloudflane.com 99
hide.ovh 99
graftpool.ovh 99
encoding.ovh 99
altavista.ovh 99
scaleway.ovh 99
nexttime.ovh 99
never.ovh 99
2giga.download 99
support.2giga.link 99
webminerpool.com 99
minercry.pt 99
adplusplus.fr 99
# Does not seem to work anymore, but keeping it here just in case it gets revived
azvjudwr.info 99
jroqvbvw.info 99
jyhfuqoh.info 99
kdowqlpt.info 99
xbasfbno.info 99
# These were Obfuscated
1beb2a44.space 99
300ca0d0.space 99
310ca263.space 99
320ca3f6.space 99
330ca589.space 99
340ca71c.space 99
360caa42.space 99
370cabd5.space 99
3c0cb3b4.space 99
3d0cb547.space 99
imageshack.us 99
img265.imageshack.us 99
+136
View File
@@ -0,0 +1,136 @@
#!/usr/bin/env python
#
# Copyright (C) 2019 - ntop.org
#
# nDPI is free software: you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# nDPI is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public License
# along with nDPI. If not, see <http://www.gnu.org/licenses/>.
#
#
# Usage
#
# 1) Generate a CSV file using the ndpiReader tool. Example:
# ./ndpiReader -C sample.json -i sample.pcap
#
# 2) Open Google Chrome and type in the URL 'chrome://tracing/'
#
# 3) Inside Chrome click on 'Load' or drop sample.json in the
# Chrome window to visualize the output
#
import sys
import json
protos = {}
lastId = 1
def get_timestamp(seen):
tok = seen.split(".")
return int(tok[0]) * 1000 + int(tok[1])
def get_record(toks, csv_fields):
global protos
global lastId
if len(toks) < 11:
return None
record = dict()
ndpiProtocol = toks[10]
ndpi_protos = ndpiProtocol.split(".")
if(len(ndpi_protos) == 1):
app_proto = ndpi_protos[0]
else:
app_proto = ndpi_protos[1]
id = protos.get(ndpiProtocol)
if(id == None):
lastId = lastId + 1
protos[ndpiProtocol] = lastId
id = lastId
#print(ndpiProtocol+"="+str(id))
ip_address = toks[5]
server_name = toks[11]
record["cat"] = "flow"
record["pid"] = ip_address
record["tid"] = ndpiProtocol # id
record["ts"] = get_timestamp(toks[2])
record["ph"] = "X"
record["name"] = app_proto
if(server_name == ""):
args = {}
else:
args = { "name": server_name }
record["args"] = args
record["dur"] = get_timestamp(toks[3]) - record["ts"]
# if we do not have the legend we just return
if csv_fields is None:
return record
# Otherwise we just add everything we find as a string
if(0):
idx = 0
for tok in toks:
name = csv_fields[idx]
idx += 1
record["args"][name] = str(tok)
return record
def get_record_dict(filename):
csv_fields = None
records = []
fin = open(filename, "r");
for line in fin:
line = line.replace("\n","")
# Get the legend if present
if line[0] == '#':
csv_fields = []
line = line.replace("#", "")
toks = line.split(",")
for tok in toks:
csv_fields.append(tok)
continue
toks = line.split(",")
flow_id = int(toks[0])
record = get_record(toks, csv_fields)
if record is None:
print("Error while parsing " + line)
continue
records.append(record)
json_dict = dict()
json_dict["traceEvents"] = records
return json_dict
if __name__ == "__main__":
if len(sys.argv) != 3:
print("ndpi2json <csv_file> <json_file>")
sys.exit(0)
record_dict = get_record_dict(sys.argv[1])
#print(record_dict)
#json_string = json.dumps(json_dict)
#print(json_string)
with open(sys.argv[2], 'w') as fp:
json.dump(record_dict, fp)
print("Written " + str(len(record_dict["traceEvents"])) + " records")
+5752 -1305
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+78 -4
View File
@@ -2,17 +2,91 @@
# <tcp|udp>:<port>,<tcp|udp>:<port>,.....@<proto>
tcp:81,tcp:8181@HTTP
udp:5061-5062@SIP
udp:5062@SIP
tcp:860,udp:860,tcp:3260,udp:3260@iSCSI
tcp:3000@ntop
tcp:9002@Elasticsearch
tcp:5601@Kibana
tcp:65535@TestProto
#
# nBPF filters (https://github.com/ntop/PF_RING/tree/dev/userland/nbpf))
#
# NOTE: they are evaluated in the same order they are defined !
#
nbpf:"host 192.168.1.1 and port 80"@HomeRouter
# Subprotocols
# Format:
# host:"<value>",host:"<value>",.....@<subproto>
host:"googlesyndacation.com"@Google
host:"venere.com"@Venere
host:"kataweb.it",host:"repubblica.it"@Repubblica
host:"disneyplus.com",host:"cdn.registerdisney.go.com",host:"disney-portal.my.onetrust.com",host:"disneyplus.bn5x.net",host:"disney-plus.net"@DisneyPlus
host:"*.lvlt.dash.us.aiv-cdn.net.c.footprint.net"@AmazonVideo
host:"api-global.netflix.com"@Netflix
# IP based Subprotocols
# Format:
# ip:<value>,ip:<value>,.....@<subproto>
# ipv6:[<value>],ipv6:[<value>],.....@<subproto>
#
# NOTES
# 1) the port of a custom protocol is optional but if
# specified it must match the port.
# 2) you can specify up to 1 port per IP address
# 3) if you specify a custom ip:<IP>:<PORT> rule,
# even if the <PORT> doesn't match the <IP>
# (if best match during the search) will
# have priority as best match. Example if
# you specify a <Google IP>:<port 9999> and
# in your traffic have match for such IP but
# with a port other than 9999, the IP address
# begin a best match will hve preference over
# <Google IP> so this protocol will not be
# detected as <L7 proto>.Google but only
# as <L7 proto>
#
ip:213.75.170.11/32:443@CustomProtocol
ip:8.248.73.247:443@AmazonPrime
ip:54.80.47.130@AmazonPrime
#You can specify a protocol Id. In that case you probably want to avoid conflict with internal ids.
#You can use any number up to 65535
ip:3.3.3.3:443@CustomProtocolA
ip:3.3.3.3:444@CustomProtocolB
ip:3.3.3.3:446@CustomProtocolC=800
ipv6:[3ffe:507:0:1:200:86ff:fe05:80da]@CustomProtocolD=1024
ipv6:[247f:855b:5e16:3caf::]/64:100@CustomProtocolE=2048
ipv6:[247f:855b:5e16:3caf::]/64@CustomProtocolF=2049
ipv6:[fe80::76ac:b9ff:fe6c:c124]:12717@CustomProtocolG=2050
ipv6:[fe80::76ac:b9ff:fe6c:c124]:12718@CustomProtocolH=65535
ipv6:[fe80::76ac:b9ff:fe6c:c124]:12719@CustomProtocolI=65534
#
# You can use symbolic IP addreses if you want
#
ip:www.ntop.org@ntop
ipv6:www.ntop.org@ntop
#
# Risk Exceptions
#
# ip_risk_mask: used to mask flow risks for IP addresses
# host_risk_mask: used to mask exceptions for domain names and hosts
#
# Syntax: <name>=<64 bit mask to be put in AND with the risk
#
# For IPs, the flow risk is put in AND (source IP mask OR destination IP mask)
# For Flows with a hostname (e.g. TLS) the risk is also put in AND with the host_risk_mask
#ip_risk_mask:192.168.1.0/24=0
ip_risk_mask:10.10.120.0/24=0
ip_risk_mask:10.196.157.228=0
ipv6_risk_mask:[fe80::356b:e047:3695:0]/112=0
ipv6_risk_mask:[fe80::7c0:e74e:87c3:5d93]=0
host_risk_mask:".home"=0
# Custom certification autorities we trust
trusted_issuer_dn:"CN=813845657003339838, O=Code42, OU=TEST, ST=MN, C=US"
File diff suppressed because it is too large Load Diff
+459
View File
@@ -0,0 +1,459 @@
/*
* ndpi_util.h
*
* Copyright (C) 2011-22 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
/**
* This module contains routines to help setup a simple nDPI program.
*
* If you concern about performance or have to integrate nDPI in your
* application, you could need to reimplement them yourself.
*
* WARNING: this API is just a demo od nDPI usage: Use it at your own risk!
*/
#ifndef __NDPI_UTIL_H__
#define __NDPI_UTIL_H__
#include "../src/lib/third_party/include/uthash.h"
#include <pcap.h>
#include "ndpi_includes.h"
#include "ndpi_classify.h"
#include "ndpi_typedefs.h"
#ifdef USE_DPDK
#include <rte_eal.h>
#include <rte_ether.h>
#include <rte_ethdev.h>
#include <rte_cycles.h>
#include <rte_lcore.h>
#include <rte_mbuf.h>
#define RX_RING_SIZE 128
#define TX_RING_SIZE 512
#define NUM_MBUFS 8191
#define MBUF_CACHE_SIZE 250
#define BURST_SIZE 32
#define PREFETCH_OFFSET 3
extern int dpdk_port_init(int port, struct rte_mempool *mbuf_pool);
extern int dpdk_port_deinit(int port);
#endif
#define PLEN_MAX 1504
#define PLEN_BIN_LEN 32
#define PLEN_NUM_BINS 48 /* 47*32 = 1504 */
#define MAX_NUM_BIN_PKTS 256
/* ETTA Spec defiintions for feature readiness */
#define ETTA_MIN_PACKETS 10
#define ETTA_MIN_OCTETS 4000
/** maximum line length */
#define LINEMAX 512
#define MAX_BYTE_COUNT_ARRAY_LENGTH 256
#define MAX_NUM_PKTS 10
#define MAX_NUM_READER_THREADS 16
#define IDLE_SCAN_PERIOD 10 /* msec (use TICK_RESOLUTION = 1000) */
#define MAX_IDLE_TIME 30000
#define IDLE_SCAN_BUDGET 1024
#define NUM_ROOTS 512
#define MAX_EXTRA_PACKETS_TO_CHECK 7
#define MAX_NDPI_FLOWS 200000000
#define TICK_RESOLUTION 1000
#define MAX_NUM_IP_ADDRESS 5 /* len of ip address array */
#define UPDATED_TREE 1
#define AGGRESSIVE_PERCENT 95.00
#define DIR_SRC 10
#define DIR_DST 20
#define PORT_ARRAY_SIZE 20
#define HOST_ARRAY_SIZE 20
#define FLOWS_PACKETS_THRESHOLD 0.9
#define FLOWS_PERCENT_THRESHOLD 1.0
#define FLOWS_PERCENT_THRESHOLD_2 0.2
#define FLOWS_THRESHOLD 1000
#define PKTS_PERCENT_THRESHOLD 0.1
#define MAX_TABLE_SIZE_1 4096
#define MAX_TABLE_SIZE_2 8192
#define INIT_VAL -1
#define SERIALIZATION_BUFSIZ (8192 * 2)
#ifdef __cplusplus
extern "C" {
#endif
// inner hash table (ja3 -> security state)
typedef struct ndpi_ja3_info {
char * ja3;
ndpi_cipher_weakness unsafe_cipher;
UT_hash_handle hh;
} ndpi_ja3_info;
// external hash table (host ip -> <ip string, hash table ja3c, hash table ja3s>)
// used to aggregate ja3 fingerprints by hosts
typedef struct ndpi_host_ja3_fingerprints {
u_int32_t ip;
char *ip_string;
char *dns_name;
ndpi_ja3_info *host_client_info_hasht;
ndpi_ja3_info *host_server_info_hasht;
UT_hash_handle hh;
} ndpi_host_ja3_fingerprints;
//inner hash table
typedef struct ndpi_ip_dns{
u_int32_t ip;
char *ip_string;
char *dns_name; //server name if any;
UT_hash_handle hh;
} ndpi_ip_dns;
//hash table ja3 -> <host, ip, security>, used to aggregate host by ja3 fingerprints
typedef struct ndpi_ja3_fingerprints_host{
char *ja3; //key
ndpi_cipher_weakness unsafe_cipher;
ndpi_ip_dns *ipToDNS_ht;
UT_hash_handle hh;
} ndpi_ja3_fingerprints_host;
struct flow_metrics {
float entropy, average, stddev;
};
struct ndpi_entropy {
// Entropy fields
u_int16_t src2dst_pkt_len[MAX_NUM_PKTS]; /*!< array of packet appdata lengths */
pkt_timeval src2dst_pkt_time[MAX_NUM_PKTS]; /*!< array of arrival times */
u_int16_t dst2src_pkt_len[MAX_NUM_PKTS]; /*!< array of packet appdata lengths */
pkt_timeval dst2src_pkt_time[MAX_NUM_PKTS]; /*!< array of arrival times */
pkt_timeval src2dst_start; /*!< first packet arrival time */
pkt_timeval dst2src_start; /*!< first packet arrival time */
u_int32_t src2dst_opackets; /*!< non-zero packet counts */
u_int32_t dst2src_opackets; /*!< non-zero packet counts */
u_int16_t src2dst_pkt_count; /*!< packet counts */
u_int16_t dst2src_pkt_count; /*!< packet counts */
u_int32_t src2dst_l4_bytes; /*!< packet counts */
u_int32_t dst2src_l4_bytes; /*!< packet counts */
u_int32_t src2dst_byte_count[MAX_BYTE_COUNT_ARRAY_LENGTH]; /*!< number of occurences of each byte */
u_int32_t dst2src_byte_count[MAX_BYTE_COUNT_ARRAY_LENGTH]; /*!< number of occurences of each byte */
u_int32_t src2dst_num_bytes;
u_int32_t dst2src_num_bytes;
double src2dst_bd_mean;
double src2dst_bd_variance;
double dst2src_bd_mean;
double dst2src_bd_variance;
float score;
};
enum info_type {
INFO_INVALID = 0,
INFO_GENERIC,
INFO_KERBEROS,
INFO_SOFTETHER,
INFO_TIVOCONNECT,
INFO_FTP_IMAP_POP_SMTP,
INFO_NATPMP,
};
// flow tracking
typedef struct ndpi_flow_info {
u_int32_t flow_id;
u_int32_t hashval;
u_int32_t src_ip; /* network order */
u_int32_t dst_ip; /* network order */
struct ndpi_in6_addr src_ip6; /* network order */
struct ndpi_in6_addr dst_ip6; /* network order */
u_int16_t src_port; /* network order */
u_int16_t dst_port; /* network order */
u_int8_t detection_completed, protocol, bidirectional, check_extra_packets;
u_int16_t vlan_id;
ndpi_packet_tunnel tunnel_type;
struct ndpi_flow_struct *ndpi_flow;
char src_name[INET6_ADDRSTRLEN], dst_name[INET6_ADDRSTRLEN];
u_int8_t ip_version;
u_int32_t cwr_count, src2dst_cwr_count, dst2src_cwr_count;
u_int32_t ece_count, src2dst_ece_count, dst2src_ece_count;
u_int32_t urg_count, src2dst_urg_count, dst2src_urg_count;
u_int32_t ack_count, src2dst_ack_count, dst2src_ack_count;
u_int32_t psh_count, src2dst_psh_count, dst2src_psh_count;
u_int32_t syn_count, src2dst_syn_count, dst2src_syn_count;
u_int32_t fin_count, src2dst_fin_count, dst2src_fin_count;
u_int32_t rst_count, src2dst_rst_count, dst2src_rst_count;
u_int32_t c_to_s_init_win, s_to_c_init_win;
u_int64_t first_seen_ms, last_seen_ms;
u_int64_t src2dst_bytes, dst2src_bytes;
u_int64_t src2dst_goodput_bytes, dst2src_goodput_bytes;
u_int32_t src2dst_packets, dst2src_packets;
u_int32_t has_human_readeable_strings;
char human_readeable_string_buffer[32];
char *risk_str;
// result only, not used for flow identification
ndpi_protocol detected_protocol;
ndpi_confidence_t confidence;
u_int16_t num_dissector_calls;
u_int16_t dpi_packets;
// Flow data analysis
pkt_timeval src2dst_last_pkt_time, dst2src_last_pkt_time, flow_last_pkt_time;
struct ndpi_analyze_struct *iat_c_to_s, *iat_s_to_c, *iat_flow,
*pktlen_c_to_s, *pktlen_s_to_c;
enum info_type info_type;
union {
char info[256];
struct {
unsigned char auth_failed;
char username[127];
char password[128];
} ftp_imap_pop_smtp;
struct {
char domain[85];
char hostname[85];
char username[86];
} kerberos;
struct {
char ip[16];
char port[6];
char hostname[48];
char fqdn[48];
} softether;
struct {
char identity_uuid[36];
char machine[48];
char platform[32];
char services[48];
} tivoconnect;
struct {
uint16_t result_code;
uint16_t internal_port;
uint16_t external_port;
char ip[16];
} natpmp;
};
ndpi_serializer ndpi_flow_serializer;
char host_server_name[80]; /* Hostname/SNI */
char *bittorent_hash;
char *dhcp_fingerprint;
char *dhcp_class_ident;
ndpi_risk risk;
struct {
char currency[16];
} mining;
struct {
u_int16_t ssl_version;
char server_info[64],
client_hassh[33], server_hassh[33], *server_names,
*advertised_alpns, *negotiated_alpn, *tls_supported_versions,
*tls_issuerDN, *tls_subjectDN,
ja3_client[33], ja3_server[33], ja4_client[37],
sha1_cert_fingerprint[20];
u_int8_t sha1_cert_fingerprint_set;
struct tls_heuristics browser_heuristics;
struct {
u_int16_t cipher_suite;
char *esni;
} encrypted_sni;
struct {
u_int16_t version;
} encrypted_ch;
time_t notBefore, notAfter;
u_int16_t server_cipher;
ndpi_cipher_weakness client_unsafe_cipher, server_unsafe_cipher;
u_int32_t quic_version;
} ssh_tls;
struct {
char url[256], request_content_type[64], content_type[64],
user_agent[256], server[128], nat_ip[32], filename[256];
u_int response_status_code;
} http;
struct {
ndpi_address_port mapped_address, peer_address,
relayed_address, response_origin, other_address;
} stun;
struct {
char *username, *password;
} telnet;
struct {
char geolocation_iata_code[4];
} dns;
ndpi_multimedia_flow_type multimedia_flow_type;
void *src_id, *dst_id;
struct ndpi_entropy *entropy;
struct ndpi_entropy *last_entropy;
/* Payload lenght bins */
#ifdef DIRECTION_BINS
struct ndpi_bin payload_len_bin_src2dst, payload_len_bin_dst2src;
#else
struct ndpi_bin payload_len_bin;
#endif
/* Flow payload */
u_int16_t flow_payload_len;
char *flow_payload;
} ndpi_flow_info_t;
// flow statistics info
typedef struct ndpi_stats {
u_int32_t guessed_flow_protocols;
u_int64_t raw_packet_count;
u_int64_t ip_packet_count;
u_int64_t total_wire_bytes, total_ip_bytes, total_discarded_bytes;
u_int64_t protocol_counter[NDPI_MAX_SUPPORTED_PROTOCOLS + NDPI_MAX_NUM_CUSTOM_PROTOCOLS + 1];
u_int64_t protocol_counter_bytes[NDPI_MAX_SUPPORTED_PROTOCOLS + NDPI_MAX_NUM_CUSTOM_PROTOCOLS + 1];
u_int32_t protocol_flows[NDPI_MAX_SUPPORTED_PROTOCOLS + NDPI_MAX_NUM_CUSTOM_PROTOCOLS + 1];
u_int32_t ndpi_flow_count;
u_int32_t flow_count[3];
u_int64_t tcp_count, udp_count;
u_int64_t mpls_count, pppoe_count, vlan_count, fragmented_count;
u_int64_t packet_len[6];
u_int16_t max_packet_len;
u_int64_t dpi_packet_count[3];
u_int64_t flow_confidence[NDPI_CONFIDENCE_MAX];
u_int64_t num_dissector_calls;
struct ndpi_lru_cache_stats lru_stats[NDPI_LRUCACHE_MAX];
struct ndpi_automa_stats automa_stats[NDPI_AUTOMA_MAX];
struct ndpi_patricia_tree_stats patricia_stats[NDPI_PTREE_MAX];
} ndpi_stats_t;
// flow preferences
typedef struct ndpi_workflow_prefs {
u_int8_t decode_tunnels;
u_int8_t quiet_mode;
u_int8_t ignore_vlanid;
u_int32_t num_roots;
u_int32_t max_ndpi_flows;
} ndpi_workflow_prefs_t;
struct ndpi_workflow;
/** workflow, flow, user data */
typedef void (*ndpi_workflow_callback_ptr) (struct ndpi_workflow *, struct ndpi_flow_info *, void *);
// workflow main structure
typedef struct ndpi_workflow {
u_int64_t last_time;
struct ndpi_workflow_prefs prefs;
struct ndpi_stats stats;
ndpi_workflow_callback_ptr flow_callback;
void * flow_callback_userdata;
/* outside referencies */
pcap_t *pcap_handle;
/* allocated by prefs */
void **ndpi_flows_root;
struct ndpi_detection_module_struct *ndpi_struct;
struct ndpi_global_context *g_ctx;
u_int32_t num_allocated_flows;
/* CSV,TLV,JSON serialization interface */
ndpi_serialization_format ndpi_serialization_format;
} ndpi_workflow_t;
/* TODO: remove wrappers parameters and use ndpi global, when their initialization will be fixed... */
struct ndpi_workflow * ndpi_workflow_init(const struct ndpi_workflow_prefs * prefs, pcap_t * pcap_handle, int do_init_flows_root, ndpi_serialization_format serialization_format, struct ndpi_global_context *g_ctx);
/* workflow main free function */
void ndpi_workflow_free(struct ndpi_workflow * workflow);
/** Free flow_info ndpi support structures but not the flow_info itself
*
* TODO remove! Half freeing things is bad!
*/
void ndpi_free_flow_info_half(struct ndpi_flow_info *flow);
/* Process a packet and update the workflow */
struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
const struct pcap_pkthdr *header,
const u_char *packet,
ndpi_risk *flow_risk,
struct ndpi_flow_info **flow);
/* Flow callback for completed flows, before the flow memory will be freed. */
static inline void ndpi_workflow_set_flow_callback(struct ndpi_workflow * workflow, ndpi_workflow_callback_ptr callback, void * userdata) {
workflow->flow_callback = callback;
workflow->flow_callback_userdata = userdata;
}
int ndpi_is_datalink_supported(int datalink_type);
/* compare two nodes in workflow */
int ndpi_workflow_node_cmp(const void *a, const void *b);
void process_ndpi_collected_info(struct ndpi_workflow * workflow, struct ndpi_flow_info *flow);
void ndpi_flow_info_free_data(struct ndpi_flow_info *flow);
void ndpi_flow_info_freer(void *node);
const char* print_cipher_id(u_int32_t cipher);
int parse_proto_name_list(char *str, NDPI_PROTOCOL_BITMASK *bitmask, int inverted_logic);
extern int reader_log_level;
#if defined(NDPI_ENABLE_DEBUG_MESSAGES) && !defined(FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION)
#define LOG(log_level, args...) \
{ \
if(log_level <= reader_log_level) \
printf(args); \
}
#else
#define LOG(...) {}
#endif
#ifndef LINKTYPE_LINUX_SLL2
#define LINKTYPE_LINUX_SLL2 276
#endif
#ifdef __cplusplus
}
#endif
#endif
+11
View File
@@ -0,0 +1,11 @@
# https://www.spamhaus.org/statistics/tlds/
.fail
.viajes
.exposed
.london
.work
.gq
.ml
.surk
.tk
.xyz
File diff suppressed because it is too large Load Diff
+893
View File
@@ -0,0 +1,893 @@
bin_PROGRAMS = fuzz_process_packet fuzz_ndpi_reader fuzz_ndpi_reader_alloc_fail fuzz_ndpi_reader_payload_analyzer fuzz_quic_get_crypto_data fuzz_config fuzz_community_id fuzz_serialization fuzz_tls_certificate fuzz_dga fuzz_is_stun_udp fuzz_is_stun_tcp
#Alghoritms
bin_PROGRAMS += fuzz_alg_bins fuzz_alg_hll fuzz_alg_hw_rsi_outliers_da fuzz_alg_jitter fuzz_alg_ses_des fuzz_alg_crc32_md5 fuzz_alg_bytestream fuzz_alg_shoco
#Data structures
bin_PROGRAMS += fuzz_ds_patricia fuzz_ds_ahocorasick fuzz_ds_libcache fuzz_ds_tree fuzz_ds_ptree fuzz_ds_hash fuzz_ds_cmsketch fuzz_ds_bitmap64_fuse fuzz_ds_domain_classify
#Third party
bin_PROGRAMS += fuzz_libinjection fuzz_binaryfusefilter
#Internal crypto
bin_PROGRAMS += fuzz_gcrypt_light fuzz_gcrypt_aes fuzz_gcrypt_gcm fuzz_gcrypt_cipher
#Configuration files
bin_PROGRAMS += fuzz_filecfg_protocols fuzz_filecfg_categories fuzz_filecfg_malicious_sha1 fuzz_filecfg_malicious_ja3 fuzz_filecfg_risk_domains fuzz_filecfg_config fuzz_filecfg_category
#Reader utils
bin_PROGRAMS += fuzz_readerutils_workflow fuzz_readerutils_parseprotolist
#Mutators
bin_PROGRAMS += fuzz_ndpi_reader_pl7m fuzz_ndpi_reader_pl7m_64k fuzz_ndpi_reader_pl7m_simplest fuzz_ndpi_reader_pl7m_internal fuzz_ndpi_reader_pl7m_simplest_internal
fuzz_process_packet_SOURCES = fuzz_process_packet.c fuzz_common_code.c
fuzz_process_packet_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_process_packet_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_process_packet_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_process_packet_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_process_packet_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_process_packet_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_process_packet_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c
fuzz_ndpi_reader_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ndpi_reader_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_alloc_fail_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c
fuzz_ndpi_reader_alloc_fail_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -DENABLE_MEM_ALLOC_FAILURES -DCRYPT_FORCE_NO_AESNI
fuzz_ndpi_reader_alloc_fail_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_alloc_fail_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_alloc_fail_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_alloc_fail_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_alloc_fail_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_alloc_fail_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_payload_analyzer_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c
fuzz_ndpi_reader_payload_analyzer_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -DENABLE_MEM_ALLOC_FAILURES -DENABLE_PAYLOAD_ANALYZER
fuzz_ndpi_reader_payload_analyzer_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_payload_analyzer_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_payload_analyzer_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_payload_analyzer_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_payload_analyzer_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_payload_analyzer_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_quic_get_crypto_data_SOURCES = fuzz_quic_get_crypto_data.c fuzz_common_code.c
fuzz_quic_get_crypto_data_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_quic_get_crypto_data_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_quic_get_crypto_data_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_quic_get_crypto_data_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_quic_get_crypto_data_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_quic_get_crypto_data_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_quic_get_crypto_data_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_config_SOURCES = fuzz_config.cpp fuzz_common_code.c
fuzz_config_CXXFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_config_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_config_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_config_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_config_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_config_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_config_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_config_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_config_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_community_id_SOURCES = fuzz_community_id.cpp fuzz_common_code.c
fuzz_community_id_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_community_id_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_community_id_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_community_id_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_community_id_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_community_id_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_community_id_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_community_id_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_community_id_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_serialization_SOURCES = fuzz_serialization.cpp fuzz_common_code.c
fuzz_serialization_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_serialization_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_serialization_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_serialization_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_serialization_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_serialization_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_serialization_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_serialization_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_serialization_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_bins_SOURCES = fuzz_alg_bins.cpp fuzz_common_code.c
fuzz_alg_bins_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_bins_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_bins_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_bins_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_bins_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_bins_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_bins_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_bins_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_bins_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_hll_SOURCES = fuzz_alg_hll.cpp fuzz_common_code.c
fuzz_alg_hll_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_hll_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_hll_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_hll_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_hll_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_hll_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_hll_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_hll_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_hll_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_hw_rsi_outliers_da_SOURCES = fuzz_alg_hw_rsi_outliers_da.cpp fuzz_common_code.c
fuzz_alg_hw_rsi_outliers_da_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_hw_rsi_outliers_da_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_hw_rsi_outliers_da_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_hw_rsi_outliers_da_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_hw_rsi_outliers_da_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_hw_rsi_outliers_da_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_hw_rsi_outliers_da_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_hw_rsi_outliers_da_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_hw_rsi_outliers_da_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_jitter_SOURCES = fuzz_alg_jitter.cpp fuzz_common_code.c
fuzz_alg_jitter_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_jitter_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_jitter_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_jitter_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_jitter_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_jitter_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_jitter_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_jitter_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_jitter_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_crc32_md5_SOURCES = fuzz_alg_crc32_md5.c
fuzz_alg_crc32_md5_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_alg_crc32_md5_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_crc32_md5_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_crc32_md5_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_crc32_md5_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_crc32_md5_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_crc32_md5_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_bytestream_SOURCES = fuzz_alg_bytestream.c
fuzz_alg_bytestream_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_bytestream_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_bytestream_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_bytestream_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_bytestream_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_bytestream_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_bytestream_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_shoco_SOURCES = fuzz_alg_shoco.cpp
fuzz_alg_shoco_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_alg_shoco_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_shoco_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_shoco_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_shoco_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_shoco_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_shoco_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_ses_des_SOURCES = fuzz_alg_ses_des.cpp fuzz_common_code.c
fuzz_alg_ses_des_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_ses_des_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_ses_des_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_ses_des_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_ses_des_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_ses_des_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_ses_des_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_ses_des_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_ses_des_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_patricia_SOURCES = fuzz_ds_patricia.cpp fuzz_common_code.c
fuzz_ds_patricia_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_patricia_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_patricia_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_patricia_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_patricia_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_patricia_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_patricia_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_patricia_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_patricia_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_ahocorasick_SOURCES = fuzz_ds_ahocorasick.cpp fuzz_common_code.c
fuzz_ds_ahocorasick_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_ahocorasick_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_ahocorasick_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_ahocorasick_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_ahocorasick_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_ahocorasick_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_ahocorasick_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_ahocorasick_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_ahocorasick_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_libcache_SOURCES = fuzz_ds_libcache.cpp fuzz_common_code.c
fuzz_ds_libcache_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_libcache_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_libcache_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_libcache_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_libcache_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_libcache_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_libcache_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_libcache_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_libcache_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_tree_SOURCES = fuzz_ds_tree.cpp fuzz_common_code.c
fuzz_ds_tree_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_tree_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_tree_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_tree_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_tree_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_tree_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_tree_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_tree_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_tree_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_ptree_SOURCES = fuzz_ds_ptree.cpp fuzz_common_code.c
fuzz_ds_ptree_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_ptree_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_ptree_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_ptree_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_ptree_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_ptree_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_ptree_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_ptree_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_ptree_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_hash_SOURCES = fuzz_ds_hash.cpp fuzz_common_code.c
fuzz_ds_hash_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_hash_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_hash_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_hash_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_hash_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_hash_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_hash_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_hash_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_hash_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_cmsketch_SOURCES = fuzz_ds_cmsketch.cpp fuzz_common_code.c
fuzz_ds_cmsketch_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_cmsketch_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_cmsketch_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_cmsketch_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_cmsketch_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_cmsketch_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_cmsketch_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_cmsketch_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_cmsketch_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_bitmap64_fuse_SOURCES = fuzz_ds_bitmap64_fuse.cpp fuzz_common_code.c
fuzz_ds_bitmap64_fuse_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_bitmap64_fuse_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_bitmap64_fuse_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_bitmap64_fuse_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_bitmap64_fuse_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_bitmap64_fuse_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_bitmap64_fuse_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_bitmap64_fuse_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_bitmap64_fuse_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_domain_classify_SOURCES = fuzz_ds_domain_classify.cpp fuzz_common_code.c
fuzz_ds_domain_classify_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_domain_classify_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_domain_classify_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_domain_classify_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_domain_classify_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_domain_classify_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_domain_classify_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_domain_classify_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_domain_classify_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_libinjection_SOURCES = fuzz_libinjection.c
fuzz_libinjection_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_libinjection_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_libinjection_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_libinjection_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_libinjection_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_libinjection_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_libinjection_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_binaryfusefilter_SOURCES = fuzz_binaryfusefilter.cpp fuzz_common_code.c
fuzz_binaryfusefilter_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_binaryfusefilter_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_binaryfusefilter_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_binaryfusefilter_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_binaryfusefilter_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_binaryfusefilter_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_binaryfusefilter_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_binaryfusefilter_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_binaryfusefilter_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_tls_certificate_SOURCES = fuzz_tls_certificate.c fuzz_common_code.c
fuzz_tls_certificate_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_tls_certificate_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_tls_certificate_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_tls_certificate_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_tls_certificate_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_tls_certificate_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_tls_certificate_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_dga_SOURCES = fuzz_dga.c fuzz_common_code.c
fuzz_dga_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_dga_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_dga_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_dga_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_dga_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_dga_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_dga_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_is_stun_udp_SOURCES = fuzz_is_stun.c fuzz_common_code.c
fuzz_is_stun_udp_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_is_stun_udp_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_is_stun_udp_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_is_stun_udp_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_is_stun_udp_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_is_stun_udp_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_is_stun_udp_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_is_stun_tcp_SOURCES = fuzz_is_stun.c fuzz_common_code.c
fuzz_is_stun_tcp_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION -DSTUN_TCP
fuzz_is_stun_tcp_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_is_stun_tcp_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_is_stun_tcp_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_is_stun_tcp_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_is_stun_tcp_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_is_stun_tcp_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_gcrypt_light_SOURCES = fuzz_gcrypt_light.cpp fuzz_common_code.c
fuzz_gcrypt_light_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_light_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_light_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_gcrypt_light_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_gcrypt_light_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_light_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_light_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_gcrypt_light_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_gcrypt_light_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_gcrypt_aes_SOURCES = fuzz_gcrypt_aes.cpp
fuzz_gcrypt_aes_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_aes_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_aes_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_gcrypt_aes_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_gcrypt_aes_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_aes_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_aes_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_gcrypt_aes_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_gcrypt_aes_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_gcrypt_gcm_SOURCES = fuzz_gcrypt_gcm.cpp
fuzz_gcrypt_gcm_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_gcm_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_gcm_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_gcrypt_gcm_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_gcrypt_gcm_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_gcm_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_gcm_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_gcrypt_gcm_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_gcrypt_gcm_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_gcrypt_cipher_SOURCES = fuzz_gcrypt_cipher.cpp
fuzz_gcrypt_cipher_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_cipher_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_cipher_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_gcrypt_cipher_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_gcrypt_cipher_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_cipher_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_cipher_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_gcrypt_cipher_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_gcrypt_cipher_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_protocols_SOURCES = fuzz_filecfg_protocols.c fuzz_common_code.c
fuzz_filecfg_protocols_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_protocols_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_protocols_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_protocols_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_protocols_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_protocols_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_protocols_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_categories_SOURCES = fuzz_filecfg_categories.c fuzz_common_code.c
fuzz_filecfg_categories_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_categories_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_categories_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_categories_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_categories_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_categories_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_categories_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_malicious_sha1_SOURCES = fuzz_filecfg_malicious_sha1.c fuzz_common_code.c
fuzz_filecfg_malicious_sha1_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_malicious_sha1_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_malicious_sha1_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_malicious_sha1_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_malicious_sha1_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_malicious_sha1_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_malicious_sha1_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_malicious_ja3_SOURCES = fuzz_filecfg_malicious_ja3.c fuzz_common_code.c
fuzz_filecfg_malicious_ja3_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_malicious_ja3_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_malicious_ja3_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_malicious_ja3_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_malicious_ja3_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_malicious_ja3_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_malicious_ja3_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_risk_domains_SOURCES = fuzz_filecfg_risk_domains.c fuzz_common_code.c
fuzz_filecfg_risk_domains_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_risk_domains_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_risk_domains_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_risk_domains_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_risk_domains_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_risk_domains_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_risk_domains_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_config_SOURCES = fuzz_filecfg_config.c fuzz_common_code.c
fuzz_filecfg_config_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_config_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_config_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_config_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_config_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_config_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_config_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_category_SOURCES = fuzz_filecfg_category.c fuzz_common_code.c
fuzz_filecfg_category_CFLAGS = -I../src/lib/ @NDPI_CFLAGS@ $(CXXFLAGS) -DNDPI_LIB_COMPILATION
fuzz_filecfg_category_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_category_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_category_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_category_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_category_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_category_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_readerutils_workflow_SOURCES = fuzz_readerutils_workflow.cpp fuzz_common_code.c ../example/reader_util.c
fuzz_readerutils_workflow_CXXFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_readerutils_workflow_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_readerutils_workflow_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_readerutils_workflow_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_readerutils_workflow_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_readerutils_workflow_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_readerutils_workflow_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_readerutils_workflow_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_readerutils_workflow_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_readerutils_parseprotolist_SOURCES = fuzz_readerutils_parseprotolist.cpp fuzz_common_code.c ../example/reader_util.c
fuzz_readerutils_parseprotolist_CXXFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_readerutils_parseprotolist_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_readerutils_parseprotolist_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_readerutils_parseprotolist_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_readerutils_parseprotolist_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_readerutils_parseprotolist_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_readerutils_parseprotolist_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_readerutils_parseprotolist_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_readerutils_parseprotolist_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_pl7m_simplest_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c ../src/lib/third_party/src/fuzz/pl7m.c
fuzz_ndpi_reader_pl7m_simplest_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -I../src/lib/third_party/include/ -DENABLE_PCAP_L7_MUTATOR -DPL7M_USE_SIMPLEST_MUTATOR
fuzz_ndpi_reader_pl7m_simplest_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_pl7m_simplest_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_pl7m_simplest_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_pl7m_simplest_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_pl7m_simplest_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_pl7m_simplest_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_pl7m_simplest_internal_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c ../src/lib/third_party/src/fuzz/pl7m.c
fuzz_ndpi_reader_pl7m_simplest_internal_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -I../src/lib/third_party/include/ -DENABLE_PCAP_L7_MUTATOR -DPL7M_USE_SIMPLEST_MUTATOR -DPL7M_USE_INTERNAL_FUZZER_MUTATE
fuzz_ndpi_reader_pl7m_simplest_internal_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_pl7m_simplest_internal_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_pl7m_simplest_internal_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_pl7m_simplest_internal_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_pl7m_simplest_internal_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_pl7m_simplest_internal_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_pl7m_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c ../src/lib/third_party/src/fuzz/pl7m.c
fuzz_ndpi_reader_pl7m_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -I../src/lib/third_party/include/ -DENABLE_PCAP_L7_MUTATOR
fuzz_ndpi_reader_pl7m_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_pl7m_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_pl7m_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_pl7m_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_pl7m_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_pl7m_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_pl7m_64k_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c ../src/lib/third_party/src/fuzz/pl7m.c
fuzz_ndpi_reader_pl7m_64k_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -I../src/lib/third_party/include/ -DENABLE_PCAP_L7_MUTATOR -DPL7M_USE_64K_PACKETS
fuzz_ndpi_reader_pl7m_64k_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_pl7m_64k_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_pl7m_64k_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_pl7m_64k_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_pl7m_64k_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_pl7m_64k_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_pl7m_internal_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c ../src/lib/third_party/src/fuzz/pl7m.c
fuzz_ndpi_reader_pl7m_internal_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -I../src/lib/third_party/include/ -DENABLE_PCAP_L7_MUTATOR -DPL7M_USE_INTERNAL_FUZZER_MUTATE
fuzz_ndpi_reader_pl7m_internal_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_pl7m_internal_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_pl7m_internal_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_pl7m_internal_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_pl7m_internal_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_pl7m_internal_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
# required for Google oss-fuzz
# see https://github.com/google/oss-fuzz/tree/master/projects/ndpi
testpcaps := $(wildcard ../tests/cfgs/default/pcap/*.pcap*)
testpcaps.zip: $(testpcaps)
zip -j testpcaps.zip $(testpcaps)
fuzz_ndpi_reader_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_seed_corpus.zip
fuzz_ndpi_reader_alloc_fail_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_alloc_fail_seed_corpus.zip
fuzz_ndpi_reader_payload_analyzer_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_payload_analyzer_seed_corpus.zip
fuzz_ndpi_reader_pl7m_simplest_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_pl7m_simplest_seed_corpus.zip
fuzz_ndpi_reader_pl7m_simplest_internal_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_pl7m_simplest_internal_seed_corpus.zip
fuzz_ndpi_reader_pl7m_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_pl7m_seed_corpus.zip
fuzz_ndpi_reader_pl7m_64k_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_pl7m_64k_seed_corpus.zip
fuzz_ndpi_reader_pl7m_internal_seed_corpus.zip: testpcaps.zip
cp testpcaps.zip fuzz_ndpi_reader_pl7m_internal_seed_corpus.zip
files_corpus_fuzz_quic_get_crypto_data := $(wildcard corpus/fuzz_quic_get_crypto_data/*)
fuzz_quic_get_crypto_data_seed_corpus.zip: $(files_corpus_fuzz_quic_get_crypto_data)
zip -j fuzz_quic_get_crypto_data_seed_corpus.zip $(files_corpus_fuzz_quic_get_crypto_data)
files_corpus_fuzz_serialization := $(wildcard corpus/fuzz_serialization/*)
fuzz_serialization_seed_corpus.zip: $(files_corpus_fuzz_serialization)
zip -j fuzz_serialization_seed_corpus.zip $(files_corpus_fuzz_serialization)
files_corpus_fuzz_community_id := $(wildcard corpus/fuzz_community_id/*)
fuzz_community_id_seed_corpus.zip: $(files_corpus_fuzz_community_id)
zip -j fuzz_community_id_seed_corpus.zip $(files_corpus_fuzz_community_id)
files_corpus_fuzz_is_stun_udp := $(wildcard corpus/fuzz_is_stun_udp/*)
fuzz_is_stun_udp_seed_corpus.zip: $(files_corpus_fuzz_is_stun_udp)
zip -j fuzz_is_stun_udp_seed_corpus.zip $(files_corpus_fuzz_is_stun_udp)
files_corpus_fuzz_is_stun_tcp := $(wildcard corpus/fuzz_is_stun_tcp/*)
fuzz_is_stun_tcp_seed_corpus.zip: $(files_corpus_fuzz_is_stun_tcp)
zip -j fuzz_is_stun_tcp_seed_corpus.zip $(files_corpus_fuzz_is_stun_tcp)
files_corpus_fuzz_alg_ses_des := $(wildcard corpus/fuzz_alg_ses_des/*)
fuzz_alg_ses_des_seed_corpus.zip: $(files_corpus_fuzz_alg_ses_des)
zip -j fuzz_alg_ses_des_seed_corpus.zip $(files_corpus_fuzz_alg_ses_des)
files_corpus_fuzz_alg_bins := $(wildcard corpus/fuzz_alg_bins/*)
fuzz_alg_bins_seed_corpus.zip: $(files_corpus_fuzz_alg_bins)
zip -j fuzz_alg_bins_seed_corpus.zip $(files_corpus_fuzz_alg_bins)
files_corpus_fuzz_alg_hll := $(wildcard corpus/fuzz_alg_hll/*)
fuzz_alg_hll_seed_corpus.zip: $(files_corpus_fuzz_alg_hll)
zip -j fuzz_alg_hll_seed_corpus.zip $(files_corpus_fuzz_alg_hll)
files_corpus_fuzz_alg_jitter := $(wildcard corpus/fuzz_alg_jitter/*)
fuzz_alg_jitter_seed_corpus.zip: $(files_corpus_fuzz_alg_jitter)
zip -j fuzz_alg_jitter_seed_corpus.zip $(files_corpus_fuzz_alg_jitter)
files_corpus_fuzz_alg_crc32_md5 := $(wildcard corpus/fuzz_alg_crc32_md5/*)
fuzz_alg_crc32_md5_seed_corpus.zip: $(files_corpus_fuzz_alg_crc32_md5)
zip -j fuzz_alg_crc32_md5_seed_corpus.zip $(files_corpus_fuzz_alg_crc32_md5)
files_corpus_fuzz_ds_libcache := $(wildcard corpus/fuzz_ds_libcache/*)
fuzz_ds_libcache_seed_corpus.zip: $(files_corpus_fuzz_ds_libcache)
zip -j fuzz_ds_libcache_seed_corpus.zip $(files_corpus_fuzz_ds_libcache)
files_corpus_fuzz_ds_ptree := $(wildcard corpus/fuzz_ds_ptree/*)
fuzz_ds_ptree_seed_corpus.zip: $(files_corpus_fuzz_ds_ptree)
zip -j fuzz_ds_ptree_seed_corpus.zip $(files_corpus_fuzz_ds_ptree)
files_corpus_fuzz_alg_bytestream := $(wildcard corpus/fuzz_alg_bytestream/*)
fuzz_alg_bytestream_seed_corpus.zip: $(files_corpus_fuzz_alg_bytestream)
zip -j fuzz_alg_bytestream_seed_corpus.zip $(files_corpus_fuzz_alg_bytestream)
files_corpus_fuzz_libinjection := $(wildcard corpus/fuzz_libinjection/*)
fuzz_libinjection_seed_corpus.zip: $(files_corpus_fuzz_libinjection)
zip -j fuzz_libinjection_seed_corpus.zip $(files_corpus_fuzz_libinjection)
files_corpus_fuzz_tls_certificate := $(wildcard corpus/fuzz_tls_certificate/*)
fuzz_tls_certificate_seed_corpus.zip: $(files_corpus_fuzz_tls_certificate)
zip -j fuzz_tls_certificate_seed_corpus.zip $(files_corpus_fuzz_tls_certificate)
files_corpus_fuzz_filecfg_protocols := $(wildcard corpus/fuzz_filecfg_protocols/*)
fuzz_filecfg_protocols_seed_corpus.zip: $(files_corpus_fuzz_filecfg_protocols)
zip -j fuzz_filecfg_protocols_seed_corpus.zip $(files_corpus_fuzz_filecfg_protocols)
files_corpus_fuzz_filecfg_categories := $(wildcard corpus/fuzz_filecfg_categories/*)
fuzz_filecfg_categories_seed_corpus.zip: $(files_corpus_fuzz_filecfg_categories)
zip -j fuzz_filecfg_categories_seed_corpus.zip $(files_corpus_fuzz_filecfg_categories)
files_corpus_fuzz_filecfg_malicious_sha1 := $(wildcard corpus/fuzz_filecfg_malicious_sha1/*)
fuzz_filecfg_malicious_sha1_seed_corpus.zip: $(files_corpus_fuzz_filecfg_malicious_sha1)
zip -j fuzz_filecfg_malicious_sha1_seed_corpus.zip $(files_corpus_fuzz_filecfg_malicious_sha1)
files_corpus_fuzz_filecfg_malicious_ja3 := $(wildcard corpus/fuzz_filecfg_malicious_ja3/*)
fuzz_filecfg_malicious_ja3_seed_corpus.zip: $(files_corpus_fuzz_filecfg_malicious_ja3)
zip -j fuzz_filecfg_malicious_ja3_seed_corpus.zip $(files_corpus_fuzz_filecfg_malicious_ja3)
files_corpus_fuzz_filecfg_risk_domains := $(wildcard corpus/fuzz_filecfg_risk_domains/*)
fuzz_filecfg_risk_domains_seed_corpus.zip: $(files_corpus_fuzz_filecfg_risk_domains)
zip -j fuzz_filecfg_risk_domains_seed_corpus.zip $(files_corpus_fuzz_filecfg_risk_domains)
files_corpus_fuzz_filecfg_config := $(wildcard corpus/fuzz_filecfg_config/*)
fuzz_filecfg_config_seed_corpus.zip: $(files_corpus_fuzz_filecfg_config)
zip -j fuzz_filecfg_config_seed_corpus.zip $(files_corpus_fuzz_filecfg_config)
files_corpus_fuzz_filecfg_category := $(wildcard corpus/fuzz_filecfg_category/*)
fuzz_filecfg_category_seed_corpus.zip: $(files_corpus_fuzz_filecfg_category)
zip -j fuzz_filecfg_category_seed_corpus.zip $(files_corpus_fuzz_filecfg_category)
files_corpus_fuzz_readerutils_workflow := $(wildcard corpus/fuzz_readerutils_workflow/*)
fuzz_readerutils_workflow_seed_corpus.zip: $(files_corpus_fuzz_readerutils_workflow)
zip -j fuzz_readerutils_workflow_seed_corpus.zip $(files_corpus_fuzz_readerutils_workflow)
files_corpus_fuzz_readerutils_parseprotolist := $(wildcard corpus/fuzz_readerutils_parseprotolist/*)
fuzz_readerutils_parseprotolist_seed_corpus.zip: $(files_corpus_fuzz_readerutils_parseprotolist)
zip -j fuzz_readerutils_parseprotolist_seed_corpus.zip $(files_corpus_fuzz_readerutils_parseprotolist)
files_corpus_fuzz_ds_bitmap64_fuse := $(wildcard corpus/fuzz_ds_bitmap64_fuse/*)
fuzz_ds_bitmap64_fuse_seed_corpus.zip: $(files_corpus_fuzz_ds_bitmap64_fuse)
zip -j fuzz_ds_bitmap64_fuse_seed_corpus.zip $(files_corpus_fuzz_ds_bitmap64_fuse)
files_corpus_fuzz_ds_domain_classify := $(wildcard corpus/fuzz_ds_domain_classify/*)
fuzz_ds_domain_classify_seed_corpus.zip: $(files_corpus_fuzz_ds_domain_classify)
zip -j fuzz_ds_domain_classify_seed_corpus.zip $(files_corpus_fuzz_ds_domain_classify)
corpus: fuzz_ndpi_reader_seed_corpus.zip fuzz_ndpi_reader_alloc_fail_seed_corpus.zip fuzz_ndpi_reader_payload_analyzer_seed_corpus.zip fuzz_quic_get_crypto_data_seed_corpus.zip fuzz_alg_ses_des_seed_corpus.zip fuzz_alg_bins_seed_corpus.zip fuzz_alg_hll_seed_corpus.zip fuzz_alg_jitter_seed_corpus.zip fuzz_ds_libcache_seed_corpus.zip fuzz_community_id_seed_corpus.zip fuzz_serialization_seed_corpus.zip fuzz_ds_ptree_seed_corpus.zip fuzz_alg_crc32_md5_seed_corpus.zip fuzz_alg_bytestream_seed_corpus.zip fuzz_libinjection_seed_corpus.zip fuzz_tls_certificate_seed_corpus.zip fuzz_filecfg_protocols_seed_corpus.zip fuzz_readerutils_workflow_seed_corpus.zip fuzz_readerutils_parseprotolist_seed_corpus.zip fuzz_ds_bitmap64_fuse_seed_corpus.zip fuzz_ds_domain_classify_seed_corpus.zip fuzz_filecfg_protocols_seed_corpus.zip fuzz_is_stun_udp_seed_corpus.zip fuzz_is_stun_tcp_seed_corpus.zip fuzz_ndpi_reader_pl7m_simplest_seed_corpus.zip fuzz_ndpi_reader_pl7m_seed_corpus.zip fuzz_ndpi_reader_pl7m_64k_seed_corpus.zip fuzz_ndpi_reader_pl7m_simplest_internal_seed_corpus.zip fuzz_ndpi_reader_pl7m_internal_seed_corpus.zip
cp corpus/fuzz_*seed_corpus.zip .
#Create dictionaries exactly as expected by oss-fuzz.
#This way, if we need to change/update/add something,
#we don't need to update scripts in oss-fuzz repository
dictionaries:
cp dictionary.dict fuzz_ndpi_reader.dict
cp dictionary.dict fuzz_ndpi_reader_alloc_fail.dict
cp dictionary.dict fuzz_ndpi_reader_payload_analyzer.dict
cp dictionary.dict fuzz_process_packet.dict
cp dictionary_tls_certificate.dict fuzz_tls_certificate.dict
distdir:
find . -type d | xargs -I'{}' mkdir -p '$(distdir)/{}'
find . -type f -name '*.c' \
-o -name '*.am' \
-o -name '*.h' \
-o -name '*.cpp' \
-o -name '*.options' \
-o -name 'ipv4_addresses.txt' \
-o -name 'ipv6_addresses.txt' \
-o -name 'bd_param.txt' \
-o -name 'splt_param.txt' \
-o -name 'random_list.list' \
-o -path './dictionary.dict' \
-o -path './dictionary_tls_certificate.dict' \
-o -path './corpus/fuzz_*.zip' \
-o -path './corpus/fuzz_quic_get_crypto_data/*' \
-o -path './corpus/fuzz_filecfg_protocols/*' \
-o -path './corpus/fuzz_filecfg_categories/*' \
-o -path './corpus/fuzz_filecfg_malicious_sha1/*' \
-o -path './corpus/fuzz_filecfg_malicious_ja3/*' \
-o -path './corpus/fuzz_filecfg_risk_domains/*' \
-o -path './corpus/fuzz_filecfg_config/*' \
-o -path './corpus/fuzz_filecfg_category/*' \
-o -path './corpus/fuzz_readerutils_workflow/*' \
-o -path './corpus/fuzz_readerutils_parseprotolist/*' \
-o -path './corpus/fuzz_is_stun_udp/*' \
-o -path './corpus/fuzz_is_stun_tcp/*' \
-o -path './corpus/fuzz_serialization/*' \
-o -path './corpus/fuzz_community_id/*' \
-o -path './corpus/fuzz_libinjection/*' \
-o -path './corpus/fuzz_tls_certificate/*' \
-o -path './corpus/fuzz_alg_ses_des/*' \
-o -path './corpus/fuzz_alg_bins/*' \
-o -path './corpus/fuzz_alg_hll/*' \
-o -path './corpus/fuzz_alg_jitter/*' \
-o -path './corpus/fuzz_alg_crc32_md5/*' \
-o -path './corpus/fuzz_alg_bytestream/*' \
-o -path './corpus/fuzz_ds_libcache/*' \
-o -path './corpus/fuzz_ds_bitmap64_fuse/*' \
-o -path './corpus/fuzz_ds_domain_classify/*' \
-o -path './corpus/fuzz_ds_ptree/*' | xargs -I'{}' cp -r '{}' '$(distdir)/{}'
all: corpus dictionaries
+627
View File
@@ -0,0 +1,627 @@
-2.088057846500587456e+00
7.763936238952200239e-05
4.404309737393306595e-05
-9.467385027293546973e-02
4.348947142638090457e-01
-2.091409170053043390e-04
-5.788902107267982974e-04
4.481443450852441001e-10
-3.136135459023654537e+00
-1.507730262127600751e+00
-1.204663669965535977e+00
-1.171839254318371104e+00
4.329302247232582057e-01
8.310653628092458334e+00
3.299246725156660176e+00
0.000000000000000000e+00
1.847454931582027254e-02
-1.498024139966201096e+00
-7.660670007653060942e-01
-2.908130300830076731e+00
-1.252564844610269734e+00
-1.910955328742287573e+00
9.471710980110392697e-01
2.352302758516665371e+00
2.982269972214651954e+00
4.280736383314343918e+00
4.633629909719495288e+00
-2.198052637823726840e+00
-1.150759637168392580e+00
3.420433363184381292e+00
1.857878113059351077e-02
-1.559806674919653746e+00
4.197498183183401288e+00
6.262186949633183453e+00
1.100694844370524095e+01
2.778688785515088000e+01
3.679948298336883195e+00
-2.432801394376875592e+00
5.133442052706843617e-01
2.181172654073517680e+00
-8.577551729671881731e-01
7.013844214023315926e-01
3.138233436228588857e+00
7.319940508466630247e-01
0.000000000000000000e+00
3.529209394581482861e+00
1.464585117707144413e+01
8.506550226820598359e-01
-9.060397326548508268e-01
6.787474954688997641e+00
8.125411068867387954e+00
4.515740684104064151e+00
5.372135582950940069e+00
9.210951196799497254e-01
4.802177410869620466e+00
2.945445016176073594e+01
1.575032253128311632e+00
-1.355276854364796946e-01
-3.322474764169629502e-01
3.018397817188666732e+00
1.186503569922195744e+00
0.000000000000000000e+00
8.883242370198487503e-01
7.248276146728496627e+00
0.000000000000000000e+00
0.000000000000000000e+00
-4.831246718433664711e+00
6.124136970173365002e-01
4.145693892559814686e-01
2.683998941637626867e+00
2.063906603639539039e+00
2.989801217386735210e+00
2.262965767379551962e-01
2.240332214649647380e+00
5.984550782416063086e+00
4.587011255338186544e+00
1.233118485315272039e+01
1.115223490909697857e+00
-3.682686422016995476e+00
6.096498453291562258e-01
1.119275528656461516e+00
1.377886278915177731e-01
3.828176805973048324e+00
0.000000000000000000e+00
0.000000000000000000e+00
1.442927634029647344e+01
0.000000000000000000e+00
5.719118583309401593e-01
1.993632609731877392e-01
3.047472271520709430e+00
5.736784864911910198e+00
6.677826247219391220e+00
6.307175478564531090e+00
3.150295169417364249e+01
3.738597740702392258e+00
1.129754590514236234e+01
6.108506268573830056e+00
1.605489516792866667e+00
2.929631990348545489e+00
-2.832543082245212937e-02
1.358286530670594461e+00
1.655932469853677924e+00
6.701964773769768513e-01
2.131182050917533211e+00
2.998351165769753468e+00
7.772095996358327596e+00
1.285014785269981141e+00
4.407334784589962418e+00
1.719858214230612026e+00
-1.012765674651314063e+00
-5.749271123172469133e-01
-3.559614093795681278e+00
-3.073088477387719397e+00
-4.492469521371540431e+00
-3.753286990415885427e+00
-3.219255423324282273e+00
-2.806436518181075090e+00
-2.697305948568419875e+00
-7.879608430851776646e-01
4.625507221739111330e+00
4.809280703883450414e+00
-3.435194026629848629e+00
-3.218943068168937049e+00
3.335535704890596698e+00
2.071359212435486263e+00
4.538992059175040339e+00
-2.770772323566738038e+01
2.903047708571506735e+00
-4.436143805989154032e+00
-2.647991280011542381e-01
1.737252348126810064e+00
-4.121989655995259128e+00
3.209709099445720581e-01
1.012758514896711759e+01
3.313255624721038295e+00
4.631467619785444967e+00
7.668642402146534032e+00
6.780938812710099128e+00
-3.256164342602652972e+00
6.749565128319576779e-01
0.000000000000000000e+00
-4.407265954524525853e+00
0.000000000000000000e+00
-3.666522115024547901e+01
-7.886029397826226273e+01
0.000000000000000000e+00
0.000000000000000000e+00
-2.261283814517791058e+01
-4.024317426178160240e+00
3.213063737030031342e-01
5.079805145796887800e+00
1.326813226475260343e+00
1.233684078112145643e+00
8.671852503871454232e+00
-2.041800256066371944e+00
0.000000000000000000e+00
0.000000000000000000e+00
-1.607347800380474823e+01
-4.430790279223246309e+00
1.177552465851384511e+00
6.342921220500139512e+00
-2.466913734548706327e-02
3.451642566010713065e-01
-6.012767168531006234e+00
7.328146570137336724e+00
7.500088131707050465e+00
0.000000000000000000e+00
-3.547913249211809017e+01
-3.130964814607208879e+00
8.247326544297072237e-01
3.757262485775580418e-01
-2.136528302027558723e+00
-2.631627236037529793e-01
-2.016718799388414141e+01
0.000000000000000000e+00
0.000000000000000000e+00
0.000000000000000000e+00
-7.708602132869285528e-01
-2.602868328868111814e+00
1.435184800833797958e+00
0.000000000000000000e+00
-2.080420864280113413e+00
1.169498351211070819e+00
-1.798334115637199560e+01
-1.193885252696202670e+01
0.000000000000000000e+00
0.000000000000000000e+00
4.304089297965300709e+00
-3.020893216686394656e+00
-1.234427481614708721e+00
0.000000000000000000e+00
1.853340741926325697e+00
-2.686000064995862147e+01
-1.672275139058893600e+01
-2.826268691607605987e+01
0.000000000000000000e+00
0.000000000000000000e+00
-1.547397429377200817e+00
-4.018181657009961327e+00
-7.289186736637049968e+00
-7.458655219230571731e+00
-9.625538282761622710e+00
-1.103039457077456298e+01
-6.262675161142102809e+01
-9.265912629799268885e+00
-8.961543476816615339e+00
-9.622764435629340696e+00
-1.097978292092879826e+01
-1.097978292092879826e+01
-2.088057846500587456e+00
7.763936238952200239e-05
4.404309737393306595e-05
-9.467385027293546973e-02
4.348947142638090457e-01
-2.091409170053043390e-04
-5.788902107267982974e-04
4.481443450852441001e-10
-3.136135459023654537e+00
-1.507730262127600751e+00
-1.204663669965535977e+00
-1.171839254318371104e+00
4.329302247232582057e-01
8.310653628092458334e+00
3.299246725156660176e+00
0.000000000000000000e+00
1.847454931582027254e-02
-1.498024139966201096e+00
-7.660670007653060942e-01
-2.908130300830076731e+00
-1.252564844610269734e+00
-1.910955328742287573e+00
9.471710980110392697e-01
2.352302758516665371e+00
2.982269972214651954e+00
4.280736383314343918e+00
4.633629909719495288e+00
-2.198052637823726840e+00
-1.150759637168392580e+00
3.420433363184381292e+00
1.857878113059351077e-02
-1.559806674919653746e+00
4.197498183183401288e+00
6.262186949633183453e+00
1.100694844370524095e+01
2.778688785515088000e+01
3.679948298336883195e+00
-2.432801394376875592e+00
5.133442052706843617e-01
2.181172654073517680e+00
-8.577551729671881731e-01
7.013844214023315926e-01
3.138233436228588857e+00
7.319940508466630247e-01
0.000000000000000000e+00
3.529209394581482861e+00
1.464585117707144413e+01
8.506550226820598359e-01
-9.060397326548508268e-01
6.787474954688997641e+00
8.125411068867387954e+00
4.515740684104064151e+00
5.372135582950940069e+00
9.210951196799497254e-01
4.802177410869620466e+00
2.945445016176073594e+01
1.575032253128311632e+00
-1.355276854364796946e-01
-3.322474764169629502e-01
3.018397817188666732e+00
1.186503569922195744e+00
0.000000000000000000e+00
8.883242370198487503e-01
7.248276146728496627e+00
0.000000000000000000e+00
0.000000000000000000e+00
-4.831246718433664711e+00
6.124136970173365002e-01
4.145693892559814686e-01
2.683998941637626867e+00
2.063906603639539039e+00
2.989801217386735210e+00
2.262965767379551962e-01
2.240332214649647380e+00
5.984550782416063086e+00
4.587011255338186544e+00
1.233118485315272039e+01
1.115223490909697857e+00
-3.682686422016995476e+00
6.096498453291562258e-01
1.119275528656461516e+00
1.377886278915177731e-01
3.828176805973048324e+00
0.000000000000000000e+00
0.000000000000000000e+00
1.442927634029647344e+01
0.000000000000000000e+00
5.719118583309401593e-01
1.993632609731877392e-01
3.047472271520709430e+00
5.736784864911910198e+00
6.677826247219391220e+00
6.307175478564531090e+00
3.150295169417364249e+01
3.738597740702392258e+00
1.129754590514236234e+01
6.108506268573830056e+00
1.605489516792866667e+00
2.929631990348545489e+00
-2.832543082245212937e-02
1.358286530670594461e+00
1.655932469853677924e+00
6.701964773769768513e-01
2.131182050917533211e+00
2.998351165769753468e+00
7.772095996358327596e+00
1.285014785269981141e+00
4.407334784589962418e+00
1.719858214230612026e+00
-1.012765674651314063e+00
-5.749271123172469133e-01
-3.559614093795681278e+00
-3.073088477387719397e+00
-4.492469521371540431e+00
-3.753286990415885427e+00
-3.219255423324282273e+00
-2.806436518181075090e+00
-2.697305948568419875e+00
-7.879608430851776646e-01
4.625507221739111330e+00
4.809280703883450414e+00
-3.435194026629848629e+00
-3.218943068168937049e+00
3.335535704890596698e+00
2.071359212435486263e+00
4.538992059175040339e+00
-2.770772323566738038e+01
2.903047708571506735e+00
-4.436143805989154032e+00
-2.647991280011542381e-01
1.737252348126810064e+00
-4.121989655995259128e+00
3.209709099445720581e-01
1.012758514896711759e+01
3.313255624721038295e+00
4.631467619785444967e+00
7.668642402146534032e+00
6.780938812710099128e+00
-3.256164342602652972e+00
6.749565128319576779e-01
0.000000000000000000e+00
-4.407265954524525853e+00
0.000000000000000000e+00
-3.666522115024547901e+01
-7.886029397826226273e+01
0.000000000000000000e+00
0.000000000000000000e+00
-2.261283814517791058e+01
-4.024317426178160240e+00
3.213063737030031342e-01
5.079805145796887800e+00
1.326813226475260343e+00
1.233684078112145643e+00
8.671852503871454232e+00
-2.041800256066371944e+00
0.000000000000000000e+00
0.000000000000000000e+00
-1.607347800380474823e+01
-4.430790279223246309e+00
1.177552465851384511e+00
6.342921220500139512e+00
-2.466913734548706327e-02
3.451642566010713065e-01
-6.012767168531006234e+00
7.328146570137336724e+00
7.500088131707050465e+00
0.000000000000000000e+00
-3.547913249211809017e+01
-3.130964814607208879e+00
8.247326544297072237e-01
3.757262485775580418e-01
-2.136528302027558723e+00
-2.631627236037529793e-01
-2.016718799388414141e+01
0.000000000000000000e+00
0.000000000000000000e+00
0.000000000000000000e+00
-7.708602132869285528e-01
-2.602868328868111814e+00
1.435184800833797958e+00
0.000000000000000000e+00
-2.080420864280113413e+00
1.169498351211070819e+00
-1.798334115637199560e+01
-1.193885252696202670e+01
0.000000000000000000e+00
0.000000000000000000e+00
4.304089297965300709e+00
-3.020893216686394656e+00
-1.234427481614708721e+00
0.000000000000000000e+00
1.853340741926325697e+00
-2.686000064995862147e+01
-1.672275139058893600e+01
-2.826268691607605987e+01
0.000000000000000000e+00
0.000000000000000000e+00
-1.547397429377200817e+00
-4.018181657009961327e+00
-7.289186736637049968e+00
-7.458655219230571731e+00
-9.625538282761622710e+00
-1.103039457077456298e+01
-6.262675161142102809e+01
-9.265912629799268885e+00
-8.961543476816615339e+00
-9.622764435629340696e+00
-1.097978292092879826e+01
-1.097978292092879826e+01
-2.088057846500587456e+00
7.763936238952200239e-05
4.404309737393306595e-05
-9.467385027293546973e-02
4.348947142638090457e-01
-2.091409170053043390e-04
-5.788902107267982974e-04
4.481443450852441001e-10
-3.136135459023654537e+00
-1.507730262127600751e+00
-1.204663669965535977e+00
-1.171839254318371104e+00
4.329302247232582057e-01
8.310653628092458334e+00
3.299246725156660176e+00
0.000000000000000000e+00
1.847454931582027254e-02
-1.498024139966201096e+00
-7.660670007653060942e-01
-2.908130300830076731e+00
-1.252564844610269734e+00
-1.910955328742287573e+00
9.471710980110392697e-01
2.352302758516665371e+00
2.982269972214651954e+00
4.280736383314343918e+00
4.633629909719495288e+00
-2.198052637823726840e+00
-1.150759637168392580e+00
3.420433363184381292e+00
1.857878113059351077e-02
-1.559806674919653746e+00
4.197498183183401288e+00
6.262186949633183453e+00
1.100694844370524095e+01
2.778688785515088000e+01
3.679948298336883195e+00
-2.432801394376875592e+00
5.133442052706843617e-01
2.181172654073517680e+00
-8.577551729671881731e-01
7.013844214023315926e-01
3.138233436228588857e+00
7.319940508466630247e-01
0.000000000000000000e+00
3.529209394581482861e+00
1.464585117707144413e+01
8.506550226820598359e-01
-9.060397326548508268e-01
6.787474954688997641e+00
8.125411068867387954e+00
4.515740684104064151e+00
5.372135582950940069e+00
9.210951196799497254e-01
4.802177410869620466e+00
2.945445016176073594e+01
1.575032253128311632e+00
-1.355276854364796946e-01
-3.322474764169629502e-01
3.018397817188666732e+00
1.186503569922195744e+00
0.000000000000000000e+00
8.883242370198487503e-01
7.248276146728496627e+00
0.000000000000000000e+00
0.000000000000000000e+00
-4.831246718433664711e+00
6.124136970173365002e-01
4.145693892559814686e-01
2.683998941637626867e+00
2.063906603639539039e+00
2.989801217386735210e+00
2.262965767379551962e-01
2.240332214649647380e+00
5.984550782416063086e+00
4.587011255338186544e+00
1.233118485315272039e+01
1.115223490909697857e+00
-3.682686422016995476e+00
6.096498453291562258e-01
1.119275528656461516e+00
1.377886278915177731e-01
3.828176805973048324e+00
0.000000000000000000e+00
0.000000000000000000e+00
1.442927634029647344e+01
0.000000000000000000e+00
5.719118583309401593e-01
1.993632609731877392e-01
3.047472271520709430e+00
5.736784864911910198e+00
6.677826247219391220e+00
6.307175478564531090e+00
3.150295169417364249e+01
3.738597740702392258e+00
1.129754590514236234e+01
6.108506268573830056e+00
1.605489516792866667e+00
2.929631990348545489e+00
-2.832543082245212937e-02
1.358286530670594461e+00
1.655932469853677924e+00
6.701964773769768513e-01
2.131182050917533211e+00
2.998351165769753468e+00
7.772095996358327596e+00
1.285014785269981141e+00
4.407334784589962418e+00
1.719858214230612026e+00
-1.012765674651314063e+00
-5.749271123172469133e-01
-3.559614093795681278e+00
-3.073088477387719397e+00
-4.492469521371540431e+00
-3.753286990415885427e+00
-3.219255423324282273e+00
-2.806436518181075090e+00
-2.697305948568419875e+00
-7.879608430851776646e-01
4.625507221739111330e+00
4.809280703883450414e+00
-3.435194026629848629e+00
-3.218943068168937049e+00
3.335535704890596698e+00
2.071359212435486263e+00
4.538992059175040339e+00
-2.770772323566738038e+01
2.903047708571506735e+00
-4.436143805989154032e+00
-2.647991280011542381e-01
1.737252348126810064e+00
-4.121989655995259128e+00
3.209709099445720581e-01
1.012758514896711759e+01
3.313255624721038295e+00
4.631467619785444967e+00
7.668642402146534032e+00
6.780938812710099128e+00
-3.256164342602652972e+00
6.749565128319576779e-01
0.000000000000000000e+00
-4.407265954524525853e+00
0.000000000000000000e+00
-3.666522115024547901e+01
-7.886029397826226273e+01
0.000000000000000000e+00
0.000000000000000000e+00
-2.261283814517791058e+01
-4.024317426178160240e+00
3.213063737030031342e-01
5.079805145796887800e+00
1.326813226475260343e+00
1.233684078112145643e+00
8.671852503871454232e+00
-2.041800256066371944e+00
0.000000000000000000e+00
0.000000000000000000e+00
-1.607347800380474823e+01
-4.430790279223246309e+00
1.177552465851384511e+00
6.342921220500139512e+00
-2.466913734548706327e-02
3.451642566010713065e-01
-6.012767168531006234e+00
7.328146570137336724e+00
7.500088131707050465e+00
0.000000000000000000e+00
-3.547913249211809017e+01
-3.130964814607208879e+00
8.247326544297072237e-01
3.757262485775580418e-01
-2.136528302027558723e+00
-2.631627236037529793e-01
-2.016718799388414141e+01
0.000000000000000000e+00
0.000000000000000000e+00
0.000000000000000000e+00
-7.708602132869285528e-01
-2.602868328868111814e+00
1.435184800833797958e+00
0.000000000000000000e+00
-2.080420864280113413e+00
1.169498351211070819e+00
-1.798334115637199560e+01
-1.193885252696202670e+01
0.000000000000000000e+00
0.000000000000000000e+00
4.304089297965300709e+00
-3.020893216686394656e+00
-1.234427481614708721e+00
0.000000000000000000e+00
1.853340741926325697e+00
-2.686000064995862147e+01
-1.672275139058893600e+01
-2.826268691607605987e+01
0.000000000000000000e+00
0.000000000000000000e+00
-1.547397429377200817e+00
-4.018181657009961327e+00
-7.289186736637049968e+00
-7.458655219230571731e+00
-9.625538282761622710e+00
-1.103039457077456298e+01
-6.262675161142102809e+01
-9.265912629799268885e+00
-8.961543476816615339e+00
-9.622764435629340696e+00
-1.097978292092879826e+01
-1.097978292092879826e+01
+1
View File
@@ -0,0 +1 @@
59708719594136008472212676481906
@@ -0,0 +1 @@
5コ705コ(770(777
+1
View File
@@ -0,0 +1 @@
2Bba5a7FEEcfced4fbbBfFEe16bfcaA4
@@ -0,0 +1 @@
5コ70(77247・906:

Some files were not shown because too many files have changed in this diff Show More