Compare commits

...
595 Commits
Author SHA1 Message Date
Luca Deri 467c27ce8e Result update 2024-02-23 19:55:35 +01:00
Luca Deri 12255e2fe0 Fixed risk evaluation 2024-02-23 19:52:08 +01:00
Luca Deri 816b00190a In case a pattern is redefined (for instance with protos.txt), the custom definition overwrites the previous one (so far it was ignored) 2023-11-02 20:51:08 +01:00
Luca Deri d92536bd97 Disbling AES-NI on FreeBSD as the feature checke does not work well
and it enables it by mistake on IntelB. Coreb" i3-3240 Processor
2023-10-30 00:05:56 +01:00
Luca Deri b328916966 Moved to 4.8 2023-10-23 11:45:30 +02:00
Luca Deri 2a99e16728 Added 4.8 changeling 2023-10-23 11:38:46 +02:00
Toni Uhlig f69909d49b Add Remote Management Control Protocol (RMCP).
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-19 19:50:57 +02:00
Alfredo Cardigliano c59fe0b2b5 PR template 2023-10-19 09:29:38 +02:00
Toni 01f384f7ff Improved Steam detection by adding steamdiscover pattern. (#2105)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-17 22:13:02 +02:00
Toni 32b0aeea59 Fixed OpenWRT arm related build issues. (#2104)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-16 19:49:50 +02:00
Toni d75a8d339a Fixed two OpenWRT arm related build issues. (#2103)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-16 18:14:28 +02:00
Luca Deri d9b0ba1d14 Windows warning checks 2023-10-16 16:17:41 +02:00
Nardi Ivan 16b4913be6 fuzz: extend fuzzing coverage 2023-10-15 12:00:26 +02:00
Luca Deri b6179467eb Added NDPI_MALWARE_HOST_CONTACTED flow risk 2023-10-13 00:01:55 +02:00
Luca 30fc89e556 Fix for buffer overflow in serialization 2023-10-11 17:14:17 +02:00
lns 304747f1fa Improved MGCP detection by allowing '\r' as line feed.
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-11 16:44:26 +02:00
Nardi Ivan 4a0eda69ad QUIC: export QUIC version as metadata 2023-10-11 15:15:20 +02:00
Luca Deri a76b9d151f Serialization fix 2023-10-11 00:14:27 +02:00
Nardi Ivan 1366d94156 fuzzing: extend fuzzing coverage
Try fuzzing some functions which write to file/file descriptor; to avoid
slowing the fuzzer, close its stdout
2023-10-09 15:41:46 +02:00
Nardi Ivan 86115a8a65 fuzz: extend fuzzing coverage 2023-10-07 13:34:37 +02:00
Luca Deri 8ca842edec version of dirent.c that is liked by both VC++ and MinGW 2023-10-05 14:35:41 +02:00
Luca Deri e1e6a01d6a Windows code rework 2023-10-05 12:24:39 +02:00
Luca Deri 3881d162f7 Windows compilation fixes 2023-10-05 11:05:14 +02:00
Luca Deri 36112a645b Win include change 2023-10-05 10:47:15 +02:00
Toni a98d7ff433 Added HAProxy protocol. (#2088)
* fixed tests/do.sh.in failure print

Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-10-02 18:10:47 +02:00
Luca 77e5daf03e Cleaned up mining datastructure 2023-09-27 17:05:12 +02:00
Toni ef3adb9830 Added printf/fprintf replacement for some internal modules. (#1974)
* logging is instead redirected to `ndpi_debug_printf`

Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-09-26 23:10:57 +02:00
Luca Deri 725fcf4852 Renamed HTTP/2 to HTTP2 as the '/' can have side effects with applications sitting on top of nDPI 2023-09-20 11:17:09 +02:00
Ivan Nardi d4e0b79dd6 Fix some prototypes (#2085)
```
error: function declaration isn’t a prototype [-Werror=strict-prototypes]
```
2023-09-18 14:06:47 +02:00
Ivan Nardi 6925890383 Add support for (un-encrypted) HTTP/2 (#2087)
Plaintext HTTP/2 is quite rare on the general "internet" but it is
used in some private networks (example: 5G core network)
2023-09-18 14:06:09 +02:00
Nardi Ivan 70814002a9 fuzz: extend fuzzing coverage 2023-09-16 11:26:11 +02:00
Luca 0828dff3ac Language fix 2023-09-15 12:56:29 +02:00
Ivan Nardi c5cd4df056 Add ndpi_domain_classify_finalize() function (#2084)
The "domain classify" data structure is immutable, since it uses
"bitmap64".
Allow to finalize it before starting to process packets (i.e. before
calling `ndpi_domain_classify_contains()`) to avoid, in the data-path,
all the memory allocations due to compression.

Calling `ndpi_domain_classify_finalize()` is optional.
2023-09-12 19:47:14 +02:00
Thomas Winter 5bcf3c2ddd tftp: update pcap results
The two malformed TFTP packets are no longer considered as risk
and instead match by port only.
This is because the TFTP detection was rather sparse so could
match on several other protocols if the first two opcode bytes
happened to match.
2023-09-12 13:12:14 +02:00
Thomas Winter a5f0cb7e6c tftp: check incrementation for DATA and ACK packets
The 2 bytes following the opcode for DATA and ACK packets are the
block number and this should be incrementing every packet.
We should check to see that this is occurring otherwise false matches
can occur, eg L2TPv3 over UDP matches the DATA opcode but the next two
bytes are always zero.

Remove the DATA max block size assumption since this can be false if
the blksize option is used to increase it.

Fixes #2070
2023-09-12 13:12:14 +02:00
Thomas Winter 7263562514 tftp: check for Option Acknowledgements
TFTP Option Acknowledgement packets were being excluded.
When a read/write request contains options, an Option Acknowledgement
is returned that contains the option strings that the transaction
will use.
The options sent in the request are not compared with what was
acknowledged.
2023-09-12 13:12:14 +02:00
Thomas Winter 1e9517ab74 tftp: rework request checking to account for options
Read and write requests were not handling options.
The existing code inspecting request messages assumed that the string
before the end of the payload was the mode and that the filename
length depended on the mode length.
However the first two strings are the filename and mode which can be
followed by any number of option strings.

Rework the checking of the filename and mode to just search the first
two strings which should always exist. Any options after that are
ignored.

Absence of the filename or mode is now excludes the TFTP protocol.
Absence of the filename no longer is considered malformed packet
because it can match other protocols falsely.
2023-09-12 13:12:14 +02:00
Luca Deri 1bf7e5face Fixes matches with domain name strings that start with a dot 2023-09-11 22:50:03 +02:00
Ivan Nardi 8e6500a0da fuzz: extend fuzzing coverage (#2083) 2023-09-10 21:30:26 +02:00
Ivan Nardi ef6085370f fuzz: add fuzzers to test bitmap64 and domain_classify data structures (#2082) 2023-09-10 18:44:50 +02:00
Ivan Nardi 2c5e22123e Update protocols documentation (#2081) 2023-09-10 15:23:34 +02:00
Ivan Nardi 2a0052f25e fuzz: add fuzzers to test reader_util code (#2080) 2023-09-10 15:07:52 +02:00
Ivan Nardi 0aa1cf7245 Update every ip lists (#2079) 2023-09-10 12:13:20 +02:00
Ivan Nardi 2b883b93be Fix some errors found by fuzzers (#2078)
Fix compilation on Windows.
"dirent.h" file has been taken from https://github.com/tronkko/dirent/

Fix Python bindings

Fix some warnings with x86_64-w64-mingw32-gcc:
```
protocols/dns.c: In function ‘ndpi_search_dns’:
protocols/dns.c:775:41: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
  775 |       unsigned long first_element_len = (unsigned long)dot - (unsigned long)_hostname;
      |                                         ^
protocols/dns.c:775:62: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
  775 |       unsigned long first_element_len = (unsigned long)dot - (unsigned long)_hostname;
      |
```
```
In file included from ndpi_bitmap64.c:31:
third_party/include/binaryfusefilter.h: In function ‘binary_fuse8_hash’:
third_party/include/binaryfusefilter.h:160:32: error: left shift count >= width of type [-Werror=shift-count-overflow]
  160 |     uint64_t hh = hash & ((1UL << 36) - 1);
```
```
In function ‘ndpi_match_custom_category’,
    inlined from ‘ndpi_fill_protocol_category.part.0’ at ndpi_main.c:7056:16:
ndpi_main.c:3419:3: error: ‘strncpy’ specified bound depends on the length of the source argument [-Werror=stringop-overflow=]
 3419 |   strncpy(buf, name, name_len);
```
2023-09-10 11:09:59 +02:00
Luca Deri 805df2e5ce Reworked initialization 2023-09-10 09:36:54 +02:00
Luca Deri 63977459b6 Added OperaVPN detection 2023-09-09 18:17:26 +02:00
Luca Deri 076edeab54 Enhance DNS risk for long hostnames (> 32) 2023-09-09 18:09:57 +02:00
Luca Deri f50a4d7e85 Improved detection of invalid chars in DNS names 2023-09-09 17:46:13 +02:00
Luca Deri 1a797d7b74 Win fix 2023-09-08 12:24:27 +02:00
Luca Deri 770e460ba9 Added NDPI_TLS_ALPN_SNI_MISMATCH flow risk 2023-09-07 23:42:42 +02:00
Luca 47fb5e9f3d Cleanup 2023-09-07 11:12:37 +02:00
Luca Deri 2bd986a00a Disabled debug message 2023-09-05 18:39:26 +02:00
Luca Deri ad9ed4f954 Minor warning fixes 2023-09-05 18:08:16 +02:00
Luca Deri 978df906b3 Improved classification further reducing memory used 2023-09-05 17:03:20 +02:00
Luca Deri 0080d0d092 Compilation fixes for old compilers 2023-09-05 10:43:36 +02:00
Luca Deri feaaf83bed Added sub-domain classification fix 2023-09-05 07:21:48 +02:00
Luca Deri cf67f483f4 Classification fixes 2023-09-05 01:17:02 +02:00
Luca Deri c5ebd84fd8 Added ndpi_bitmap64 support 2023-09-05 00:21:16 +02:00
Luca Deri 4f2ce2d43b Added ndpi_murmur_hash to the nDPI API 2023-09-04 12:53:42 +02:00
Luca Deri f0dc3347ec Merged new and old version of ndpi_domain_classify.c code 2023-09-02 19:27:19 +02:00
Luca Deri 1d480c18e3 Reworked domain classification based on binary filters 2023-09-02 19:16:40 +02:00
Luca Deri 854c2d80f1 Improvement for reducing false positives 2023-09-01 10:26:07 +02:00
Luca Deri 74fdac7d77 Code cleanup 2023-09-01 08:50:55 +02:00
Luca Deri 133f45f3f3 Added ndpi_binary_bitmap datastruture
It is similar to ndpi_filter but based on binary search and with
the ability to store a category per value (as ndpi_domain_classify)
2023-08-31 23:51:50 +02:00
Luca Deri 19381f330a Fixed heap-buffer-overflow issue 2023-08-31 18:35:49 +02:00
Luca Deri 16b0ce3710 Code cleanup 2023-08-31 09:14:17 +02:00
Luca Deri f824939662 Added comment 2023-08-31 09:00:36 +02:00
Luca Deri 3c7b2a714e Added missing lists files 2023-08-29 19:21:57 +02:00
Luca Deri c5c610c941 Fixes dist 2023-08-29 18:49:43 +02:00
Luca Deri 65142b2a53 Added -G flag 2023-08-29 18:32:57 +02:00
Luca Deri 1de7fd9bf0 Warning fixes 2023-08-29 18:08:56 +02:00
Luca Deri 3dbc4f65fc Added README file 2023-08-29 17:45:36 +02:00
Luca Deri 36abf06c6f Swap from Aho-Corasick to an experimental/home-grown algorithm that uses a probabilistic
approach for handling Internet domain names.

For switching back to Aho-Corasick it is necessary to edit
ndpi-typedefs.h and uncomment the line
// #define USE_LEGACY_AHO_CORASICK

[1] With Aho-Corasick
$ ./example/ndpiReader -G ./lists/ -i tests/pcap/ookla.pcap | grep Memory
nDPI Memory statistics:
nDPI Memory (once):      37.34 KB
Flow Memory (per flow):  960 B
Actual Memory:           33.09 MB
Peak Memory:             33.09 MB

[2] With the new algorithm
$ ./example/ndpiReader -G ./lists/ -i tests/pcap/ookla.pcap | grep Memory
nDPI Memory statistics:
nDPI Memory (once):      37.31 KB
Flow Memory (per flow):  960 B
Actual Memory:           7.42 MB
Peak Memory:             7.42 MB

In essence from ~33 MB to ~7 MB

This new algorithm will enable larger lists to be loaded (e.g. top 1M domans
https://s3-us-west-1.amazonaws.com/umbrella-static/index.html)

In ./lists there are file names that are named as <category>_<string>.list
With -G ndpiReader can load all of them at startup
2023-08-29 17:34:04 +02:00
Toni Uhlig 1f693c3f5a Added lists/gambling.list to extra dist.
* make dist in `./fuzz`: fixed inconsistent `*.dict` file pattern

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-27 15:53:31 +02:00
Toni Uhlig afd250f5d9 fix compilation and symbol check
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-27 15:32:42 +02:00
Toni Uhlig 25f3a7cd1d fix compilation
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-27 15:17:56 +02:00
Luca Deri 34986de297 Search fixes 2023-08-26 19:47:50 +02:00
Luca Deri eeeee46b1e Changes for supporinng more efficient sub-string matching 2023-08-26 17:55:50 +02:00
Luca Deri 4ca94369e1 Improved domain search tet unit 2023-08-26 10:43:19 +02:00
Luca Deri ac00542df2 LEak fix 2023-08-26 10:10:49 +02:00
Luca Deri 2c565c77c9 Added ndpi_domain_classify_XXX(0 API 2023-08-26 00:24:33 +02:00
Luca Deri 42fcb011ad Warning fix 2023-08-25 19:29:43 +02:00
Luca Deri eb9932164f added bimap and/or with allocation 2023-08-24 07:45:38 +02:00
Luca Deri ff4f291afe Minor improvements 2023-08-23 11:04:31 +02:00
Luca 95fcd08687 Added ndpi_bitmap_is_empty() and ndpi_bitmap_optimize() API calls 2023-08-23 11:03:59 +02:00
Luca 9933b5910d Boundary check 2023-08-21 23:19:57 +02:00
Luca bd25df3334 Improved Wireguard detection 2023-08-21 13:40:42 +02:00
Luca 9e4bcc7771 Removed empty line 2023-08-21 13:40:30 +02:00
Luca 6e5466f9e5 Added ndpi_bitmap_andnot API call 2023-08-21 13:26:38 +02:00
Ivan Nardi cc4461f424 fuzz: extend coverage (#2073) 2023-08-20 15:18:19 +02:00
Ivan Nardi 5867f43fae Fix compilation on Windows (#2072) 2023-08-14 22:51:09 +02:00
Luca Deri cfbb7a423e Replaces free() with ndpi_free() 2023-08-14 18:51:11 +02:00
Luca Deri 049fbf819c Reworked ndpi_filter_xxx implementation using compressed bitmaps 2023-08-14 12:50:54 +02:00
Luca Deri 3b34941be1 Typo 2023-08-12 11:39:19 +02:00
Luca Deri 9a44954365 ARM compilation fix 2023-08-12 07:39:35 +02:00
Luca Deri fe634b8d55 Fixed to address issue
Run ./utils/check_symbols.sh || { FAILED=$?; echo "::error file=${NDPI_LIB}::Unwanted libc symbols found: ${FAILED}. Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of malloc/calloc/realloc/free."; false; }
[ndpi_filter.o]: calloc
Unwanted symbols found: 1
Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of malloc/calloc/realloc/free
Error: Unwanted libc symbols found: 1. Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of
2023-08-12 00:27:37 +02:00
Luca Deri 8e17f50853 Added ndpi_filter_add_multi() API call 2023-08-11 23:55:44 +02:00
Luca Deri 890dcf91d9 Minor initialization fixes 2023-08-11 23:51:16 +02:00
Luca Deri ec7adc212e Added new API calls for implementing Bloom-filter like data structures
ndpi_filter* ndpi_filter_alloc(uint32_t elements_number);
bool         ndpi_filter_add(ndpi_filter *f, uint64_t value);
bool         ndpi_filter_contains(ndpi_filter *f, uint64_t value);
void         ndpi_filter_free(ndpi_filter *f);
2023-08-11 22:49:31 +02:00
Luca 5fad72ebd9 Typo fix 2023-08-07 10:20:21 +02:00
Luca 33e9dbc24d Added ndpi_bitmap_copy() API call 2023-08-06 09:35:56 +02:00
snicket2100 1fbe8a2385 Mullvad VPN service added (based on entry node IP addresses) (#2062) 2023-08-02 19:44:16 +02:00
Luca Deri 196395398e Compilation fixes for older C compilers 2023-08-01 15:35:33 +02:00
Toni e4d3d619bc Add Service Location Protocol dissector. (#2036)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-01 08:50:46 +02:00
Toni 083275f617 Added slackb.com SNI. (#2067)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-08-01 08:39:10 +02:00
Ivan Nardi 5019022e13 DNS: extract geolocation information, if available (#2065)
The option NSID (RFC5001) is used by Google DNS to report the
airport code of the metro where the DNS query is handled.

This option is quite rare, but the added overhead in DNS code is pretty
much zero for "normal" DNS traffic
2023-07-31 07:44:43 +02:00
snicket2100 d477da997f Typo in a comment fixed (#2063) 2023-07-28 17:58:25 +02:00
Ivan Nardi bc91192aca ProtonVPN: split the ip list (#2060)
Use two separate lists:
* one for the ingress nodes, which triggers a ProtonVPN classification
* one for the egress nodes, which triggers the
`NDPI_ANONYMOUS_SUBSCRIBER` risk

Add a command line option (to `ndpiReader`) to easily test IP/port
matching.

Add another example of custom rule.
2023-07-27 09:05:22 +02:00
Ivan Nardi 3326fa258e Add an heuristic to detect fully encrypted flows (#2058)
A fully encrypted session is a flow where every bytes of the
payload is encrypted in an attempt to “look like nothing”.
The heuristic needs only the very first packet of the flow.
See: https://www.usenix.org/system/files/sec23fall-prepub-234-wu-mingshi.pdf

A basic, but generic, inplementation of the popcpunt alg has been added
2023-07-26 09:09:12 +02:00
Toni 2b230e28e0 Allow init of app protocols w/o any hostnames set. (#2057) 2023-07-22 14:51:33 +02:00
Toni 88a757c4cd Fix string truncation. (#2056)
ndpi_main.c: In function ‘ndpi_load_ip_category’:
ndpi_main.c:6598:3: warning: ‘strncpy’ specified bound 64 equals destination size [-Wstringop-truncation]
 6598 |   strncpy(ipbuf, ip_address_and_mask, sizeof(ipbuf));
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-07-22 12:45:45 +02:00
Luca Deri 6831187d62 Debian 12 fixes 2023-07-21 12:24:34 -04:00
Ivan Nardi 7a1509bf60 zabbix: improve detection (#2055) 2023-07-21 03:42:36 +02:00
Ivan Nardi c85f2fb0f4 TLS: add basic, basic, detection of Encrypted ClientHello (#2053) 2023-07-21 03:41:43 +02:00
Ivan Nardi fa0bd515b5 Add detection of Roblox games (#2054) 2023-07-21 03:39:40 +02:00
Ivan Nardi 3edfad01a1 fuzz: extend fuzzing coverage (#2052)
Added/merged some traces.
Improved Socks identification
2023-07-18 07:41:56 +02:00
Ivan Nardi 09548bb7cf tests: restore some old paths as symbolic links (#2050) 2023-07-16 13:47:35 +02:00
Ivan Nardi 5267858b3d HTTP: fix another memory access error (#2049)
```
=================================================================
==199079==ERROR: AddressSanitizer: negative-size-param: (size=-1)
    #0 0x559a2a6efd4f in strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x94ad4f) (BuildId: 34aaabba403c6bc5482553ef355360fd2762a157)
    #1 0x559a2a9890f0 in ndpi_http_check_content /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:300:8
    #2 0x559a2a9812c0 in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:910:46
    #3 0x559a2a978fee in process_response /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1289:3
    #4 0x559a2a97622f in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1382:9
    #5 0x559a2a975d95 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1468:3
    #6 0x559a2a864970 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5948:4
    #7 0x559a2a8660df in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6013:12
    #8 0x559a2a865d7f in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6032:12
    #9 0x559a2a876fd6 in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7038:15
    #10 0x559a2a87311f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7205:22
    #11 0x559a2a77381e in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1710:31
    #12 0x559a2a77381e in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2427:10
[...]
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60605
2023-07-15 22:23:52 +02:00
Ivan Nardi 2bbde5bad3 Fix compilation in CI jobs (#2048) 2023-07-15 14:26:51 +02:00
Ivan Nardi 890f17788b ndpireader: fix detection of DoH traffic based on packet distributions (#2045) 2023-07-14 23:20:06 +02:00
Ivan Nardi 5811a5613b Add a configuration knob to enable/disable loading of gambling list (#2047) 2023-07-14 23:19:30 +02:00
Luca Deri 8e960f033d Adds new pcap for testing "funny" HTTP servers 2023-07-14 21:12:11 +02:00
Luca Deri a499edd2f2 Restored files 2023-07-14 21:12:04 +02:00
Luca Deri fea09e825b Fixes risk mask exception handling while improving the overall performance 2023-07-14 19:52:34 +02:00
snicket2100 abee1a2a6f Included Gambling website data from the Polish hazard.mf.gov.pl list (#2041)
* Refreshed the Belgium Gambling Site list data

Unfortunately some hostnames have been removed from that list,
which means they are disappearing from the `ndpi_gambling_match.c.inc`
file as well.

* build: added `libxml2-utils` (for `xmllint`)

* Included Gambling website data from the Polish `hazard.mf.gov.pl` list

The list contains over 30k gambling website hostnames as of today.
2023-07-14 09:55:46 +02:00
Ivan Nardi 0dd1ee5656 HTTP: fix extraction of filename (#2046) 2023-07-14 01:33:00 +02:00
Luca Deri bfbbd18757 Removed -fno-color-diagnostics not available on all platforms 2023-07-13 23:30:05 +02:00
Luca Deri 33c42f9830 Cosmetic fixes 2023-07-13 23:26:35 +02:00
Luca Deri 1f55dc511f Implemented Count-Min Sketch [count how many times a value has been observed]
- ndpi_cm_sketch_init()
- ndpi_cm_sketch_add()
- ndpi_cm_sketch_count()
- ndpi_cm_sketch_destroy()
2023-07-13 21:54:51 +02:00
Ivan Nardi bd0fcb2e62 HTTP: fix heap-buffer-overflow (#2044)
```
=================================================================
==673246==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x5080000002fd at pc 0x555c6a22eca2 bp 0x7ffe1db9b290 sp 0x7ffe1db9aa48
READ of size 1 at 0x5080000002fd thread T0
    #0 0x555c6a22eca1 in strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x53aca1) (BuildId: 464e7474cde3de91c01588c7ea0161a05ddd6b63)
    #1 0x555c6a462a9d in ndpi_http_check_content /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:309:8
    #2 0x555c6a45a9a0 in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:901:46
    #3 0x555c6a4526ce in process_response /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1280:3
    #4 0x555c6a44f90f in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1373:9
    #5 0x555c6a44f475 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1459:3
    #6 0x555c6a348520 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5949:6
    #7 0x555c6a348f2f in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5997:12
    #8 0x555c6a348bcf in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6016:12
    #9 0x555c6a359e26 in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7017:15
    #10 0x555c6a355f6f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7184:22
    #11 0x555c6a284958 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:24:5
```

Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60557
Fix: 0b0f255cc
2023-07-13 10:50:59 +02:00
Ivan Nardi 0c5a17accb RDP: improve detection over UDP (#2043) 2023-07-13 10:50:13 +02:00
Ivan Nardi 5e6a8c0c1f fuzz: add fuzzer for DGA detection code (#2042) 2023-07-13 10:47:34 +02:00
Alfredo Cardigliano 669a02cd76 Add support for multiline json 2023-07-12 10:47:49 +02:00
Luca Deri b1d25193d2 Minor code cleanup 2023-07-11 22:49:06 +02:00
Chiara Maggi 0b0f255cc2 added feature to extract filename from http attachment (#2037)
* added feature to extract filename from http attachment

* fixed some issues

* added check for filename format

* added check for filename format

* remove an unnecessary print

* changed the size from 952 to 960

* modified some test result files

* small changes string size

* comment removed and mallocs checked
2023-07-11 22:45:19 +02:00
Ivan Nardi 950f5cc4e3 fuzz: extend fuzzing coverage (#2040)
Some notes:
* libinjection: according to https://github.com/libinjection/libinjection/issues/44,
it seems NULL characters are valid in the input string;
* RTP: `rtp_get_stream_type()` is called only for RTP packets; if you
want to tell RTP from RTCP you should use `is_rtp_or_rtcp()`;
* TLS: unnecessary check; we already make the same check just above, at
the beginning of the `while` loop
2023-07-11 10:12:08 +02:00
Toni 859d9ea3c3 Fixed heap-overflow if compiled with --enable-tls-sigs. (#2038)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-07-07 18:52:06 +02:00
Toni 72814bed33 Added coverage targets to Makefile.am for convenience. (#2039)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-07-07 17:40:50 +02:00
Ivan KapranovandIvan Kapranov bdd295bc2c fix Stack overflow caused by invalid write in ndpi_automa_match_strin… (#2035)
* minor fixes

fixed 'handle leak' in ndpi_load_malicious_sha1_file and removed the redundant comparison ndpi_search_eaq

* fix Stack overflow caused by invalid write in ndpi_automa_match_string_subprotocol

* fix compile errors

* fix

---------

Co-authored-by: Ivan Kapranov <i.kapranov@securitycode.ru>
2023-07-06 11:13:34 +02:00
Ivan Nardi bb978907e6 STUN: fix detection of Google Voip apps (#2031)
Fix: 2c7fb9179
2023-07-05 14:37:38 +02:00
headshog 076ba967a8 fixed numeric truncation error in diameter.c (#2034) 2023-07-05 14:29:55 +02:00
headshog d51f8929e0 fixed numeric truncation error in rtcp.c (#2033) 2023-07-05 14:29:44 +02:00
headshog 2e5004ddb2 fixed numeric truncation error in kerberos.c (#2032) 2023-07-05 14:29:32 +02:00
Ivan Nardi 7ffd31ebc3 STUN: avoid FacebookVoip false positives (#2029)
Attribute 0xC057 is defined in the Google public implementation of
webrtc (which is used by Google products but also by other applications)
2023-07-03 17:21:46 +02:00
Ivan Nardi dff1f25193 STUN: fix Skype/MsTeams detection and monitoring logic (#2028) 2023-07-03 17:21:32 +02:00
Ivan Nardi 86e89b4e23 STUN: tell RTP from RTCP while in monitoring state (#2027) 2023-06-27 10:33:48 +02:00
Ivan Nardi 2c7fb91794 Hangout: detect Hangout/Duo/GoogleMeet/... in the STUN code (#2025)
Regardless of the name, the removed trace doesn't contain meaningful
Hangout traffic.

Remove last piece of sub-classifiction based only on ip addresses.
2023-06-27 10:33:28 +02:00
Ivan Nardi 31a9da238c STUN: add dissection of DTLS handshake (#2018) 2023-06-26 12:07:26 +02:00
Ivan Nardi 88425e0199 Simplify the report of streaming multimedia info (#2026)
The two fields `flow->flow_type` and `flow->protos.rtp.stream_type` are
pretty much identical: rename the former in `flow->flow_multimedia_type`
and remove the latter.
2023-06-26 12:05:16 +02:00
Ivan Nardi 3a1600ff26 Thrift: fix heap-buffer-overflow (#2024) 2023-06-26 11:41:01 +02:00
Luca Deri 7ef80f6b78 Optimizes and fixes possible out0of0boundary write in ndpi_fill_prefix_v4() 2023-06-23 21:40:55 +02:00
Ivan Nardi 7e64d9f66d RTP: rework code (#2021)
Try avoiding false positives: look for 3 RTP packets before classifing
the flow as such.

Add a generic function `is_rtp_or_rtcp()` to identify RTP/RTCP packets also
in other dissectors (see 3608ab01b commit message for an example)
2023-06-23 08:12:40 +02:00
Ivan KapranovandIvan Kapranov 15f0e1480c minor fixes (#2023)
fixed 'handle leak' in ndpi_load_malicious_sha1_file and removed the redundant comparison ndpi_search_eaq

Co-authored-by: Ivan Kapranov <i.kapranov@securitycode.ru>
2023-06-22 21:36:40 +02:00
Ivan Kapranov aa25c06a18 removed useless call of ndpi_set_risk func (#2022) 2023-06-22 18:26:33 +02:00
Luca Deri cdc0bb8c61 Compilation fix 2023-06-22 16:04:14 +02:00
Toni 1678888284 Add Apache Thrift protocol dissector. (#2007)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-06-22 13:07:32 +02:00
Luca Deri 8dbd591053 Waring fix 2023-06-21 21:26:29 +02:00
Ivan Nardi cc582052b3 Gnutella: improve detection (#2019) 2023-06-21 14:25:34 +02:00
Nardi Ivan 570c75d601 STUN: fix detection over TCP
TCP framing is optional
2023-06-21 10:38:44 +02:00
Nardi Ivan 2ac240ce6a STUN: improve WhatsappCall detection 2023-06-21 10:38:44 +02:00
Ivan Nardi 46aab35a29 Line: fix heap-buffer-overflow error (#2015) 2023-06-21 09:22:07 +02:00
Ivan Nardi 3608ab01b6 STUN: keep monitoring/processing STUN flows (#2012)
Look for RTP packets in the STUN sessions.
TODO: tell RTP from RTCP
2023-06-21 09:16:20 +02:00
Ivan Nardi 04be308092 Fix access to packet/flow information (#2013)
We can't access to `ndpi_str->packet` from `ndpi_detection_giveup()`
2023-06-17 15:46:06 +02:00
Luca Deri ac06ba8320 Added hyperlink 2023-06-16 21:48:21 +02:00
Luca Deri 9cc4cbb9d1 Reworked teams handling 2023-06-15 22:31:11 +02:00
Ivan Nardi 2321c11eb5 Fix compilation (#2011) 2023-06-15 12:01:11 +02:00
Luca Deri d0609ea601 Implemented Zoom/Teams stream type detection 2023-06-14 23:44:57 +02:00
Luca Deri 66bee475ae Improved line protocol dissection with heuristic 2023-06-13 19:20:02 +02:00
Luca Deri de3cf260f8 Updated line test result 2023-06-13 19:07:02 +02:00
Luca Deri be8178fc8d Refreshed ASN lists
Enhanced the Line IP list with https://ipinfo.io/AS23576/125.209.252.0/24 used by line
2023-06-13 19:04:08 +02:00
Luca Deri b02adcc381 Added check to avoid skype heuristic false positives 2023-06-12 23:29:20 +02:00
Toni 4e284b5e40 Set _DEFAULT_SOURCE and _GNU_SOURCE globally. (#2010)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-06-12 19:53:57 +02:00
Toni 8ea0eaa0d0 Fix missing u_char, u_short and u_int typedefs for some platforms e.g.: (#2009)
In file included from ../include/ndpi_main.h:34,
                 from ../include/ndpi_api.h:28,
                 from protocols/activision.c:26:
../include/ndpi_typedefs.h:294:3: error: unknown type name 'u_char'
  294 |   u_char h_dest[6];       /* destination eth addr */
      |   ^~~~~~

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-06-12 17:52:52 +02:00
Nardi Ivan 96340ccea2 oss-fuzz: sync build script with upstream
File copied from https://github.com/google/oss-fuzz/blob/master/projects/ndpi/build.sh

The general idea is to keep the build script in our repository and use
it from oss-fuzz builder: updating it from our side is easier and faster
then passing via an oss-fuzz PR.

The original idea is from @utoni in 3068306b60.

Once this change has been merged, we can update the code in oss-fuzz.
2023-06-12 15:17:56 +02:00
Luca Deri 63a2b359b9 Added vlan_id in ndpi_flow2json() prototype 2023-06-09 19:50:47 +02:00
Nardi Ivan 7150b40c63 QUIC: fix dissection of packets forcing VN 2023-06-08 17:07:25 +02:00
Nardi Ivan 3e640a13a4 QUIC: fix a memory access error
Found while fuzzing
2023-06-08 17:07:25 +02:00
Nardi Ivan fd4cb10190 QUIC: add support for QUIC version 2
See: https://www.rfc-editor.org/rfc/rfc9369.txt

Old v2-01 version has been removed, since it has never been really used.
2023-06-08 17:07:25 +02:00
Ivan Nardi 9987e5b482 ndpiReader: allow to configure LRU caches TTL and size (#2004) 2023-06-08 17:06:32 +02:00
Ivan Nardi 3e673e91a9 ProtonVPN: add basic detection (#2006) 2023-06-08 16:52:55 +02:00
headshog 7dcceb4daf Numeric truncation at tls.c:1010 (#2005)
* fixed numtrunc error in protocols/tls.c

* fixed build error for tls.c
2023-06-08 07:52:18 +02:00
zehraInandNardi Ivan 436ab54f5a added new domain names (#2002)
* added new domain names

* Sync unit tests results

---------

Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2023-06-01 13:13:49 +02:00
Ivan Nardi e14d0acbd4 tests: add an option to force the overwrite of the unit tests results (#2001)
Usage: `FORCE_UPDATING_UTESTS_RESULTS=1 ./tests/do.sh`
2023-05-31 09:05:00 +02:00
Maatuq e17fa1259a Add bitcoing protocol dissector. (#1992)
* Add bitcoing protocol dissector.

* remove bitcoin protcol detection from mining.c
* add a new bitcoin deissector.
* add a new category: Cryptocurrency.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* Remove useless checks and add missing windows and docs file.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* update affected tests.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* add a brief version.

Add notes on the difference between normal bitcoin protocol and the
mining protocol.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

* update enable_payload_stat test after dev rebasing.

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>

---------

Signed-off-by: Mahmoud Maatuq <mahmoudmatook.mm@gmail.com>
2023-05-31 07:31:01 +02:00
headshog a8d2eeddd2 Numeric truncation at ndpi_analyze.c at lines 101, 104, 107, 110 (#1999)
* fixed numeric truncation error in ndpi_analyze.c

* fixed numeric truncation error in ndpi_analyze.c x2

* fixed numeric truncation error in ndpi_analyze.c x3

* fixed numeric truncation error in ndpi_analyze.c and printf format

* fixed tests
2023-05-30 12:27:47 +02:00
Toni 04f5c5196e Changed logging callback function sig. (#2000)
* make user data available for any build config

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-30 08:27:37 +02:00
headshog 552c9b3cc6 fixed numeric truncation error in ndpi_main.c:6837 (#1998) 2023-05-29 19:34:26 +02:00
Ivan Nardi efb261a95c Fix some memory errors triggered by allocation failures (#1995)
Some low hanging fruits found using nallocfuzz.
See: https://github.com/catenacyber/nallocfuzz
See: https://github.com/google/oss-fuzz/pull/9902

Most of these errors are quite trivial to fix; the only exception is the
stuff in the uthash.
If the insertion fails (because of an allocation failure), we need to
avoid some memory leaks. But the only way to check if the `HASH_ADD_*`
failed, is to perform a new lookup: a bit costly, but we don't use that
code in any critical data-path.
2023-05-29 19:24:00 +02:00
Ivan Nardi 346bb268e2 Avoid calling ndpi_reconcile_protocols() twice in ndpi_detection_giveup() (#1996)
`ndpi_reconcile_protocols()` is already called by
`ndpi_set_detected_protocol()`
2023-05-29 16:55:53 +02:00
Ivan Nardi 82fa3a0986 Fix an heap-buffer-overflow (#1994)
```
==32439==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x604000000080 at pc 0x564a9a1639ac bp 0x7ffc77d42e20 sp 0x7ffc77d42e18
READ of size 1 at 0x604000000080 thread T0
    #0 0x564a9a1639ab in ndpi_check_punycode_string /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:98:12
    #1 0x564a9a26025a in ndpi_match_host_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8652:6
    #2 0x564a9a33e45c in ndpi_http_parse_subprotocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:446:9
    #3 0x564a9a3354b8 in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:921:3
    #4 0x564a9a32dc8b in process_request /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1237:3
    #5 0x564a9a329a9c in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1351:5
    #6 0x564a9a3293ed in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1425:3
    #7 0x564a9a21c64a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5906:6
    #8 0x564a9a21d057 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5954:12
```

Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=59393
2023-05-29 16:53:39 +02:00
Ivan Nardi 46ff069117 ndpiReader: improve printing of payload statistics (#1989)
Add a basic unit test

Fix an endianess issue
2023-05-29 16:53:11 +02:00
Ivan Nardi 7ce14da0c8 Remove some useless checks (#1993)
In the main dissector callbacks the flow protocols are (almost) always
unknown. Only two exceptions:
* extra dissection data path
* HTTP sub-protocols
2023-05-28 12:59:18 +02:00
Toni 6da3474203 Improved helper scripts. (#1986)
* added additional (more restrictive) checks

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-28 12:45:44 +02:00
Ivan Nardi b11e6a453b Add support for Epic Games and GeForceNow/Nvidia (#1990) 2023-05-27 12:13:54 +02:00
Ivan Nardi 63ac50e4f4 Improve detection of Alibaba flows (#1991) 2023-05-27 10:19:58 +02:00
Toni 334b43579e Fixed invalid use of ndpi_free(). Sorry, my fault. (#1988)
* Fixed invalid use of ndpi_free(). Sorry, my fault.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Fine tuned symbol check script.

 * added check for expected syms in modules

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

---------

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-05-24 13:19:06 +02:00
Luca Deri f692342f3c Cosmetic changes 2023-05-23 22:16:34 +02:00
Ivan Nardi 86b56646b5 ndpiReader: fix export of DNS/BitTorrent attributes (#1985)
There is no BitTorrent hash in the DNS flows
2023-05-20 17:23:48 +02:00
Toni 5e8f93c2d1 Improved missing usage of nDPIs malloc wrapper. Fixes #1978. (#1979)
* added CI check

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-20 16:18:52 +02:00
Luca Deri 8f718c9051 Made VK protocol detection more strict 2023-05-20 15:46:47 +02:00
Toni c97e2d67ff Added scripts to auto generate hostname/SNI *.inc files. (#1984)
* add illegal gambling sites (Belgium)

Signed-off-by: lns <matzeton@googlemail.com>
2023-05-20 15:41:15 +02:00
Ivan Nardi 9004d5c2ca ndpiReader: fix export of HTTP attributes (#1982) 2023-05-20 15:12:14 +02:00
Nardi Ivan 1ab5318f9c Sync unit tests results 2023-05-20 15:09:51 +02:00
headshog dd7198fcaf fixed numeric truncation error 2023-05-20 15:09:51 +02:00
Luca Deri 5ca6f0ac62 Implemented ndpi_predict_linear() for predicting a timeseries value overtime 2023-05-19 11:46:03 +02:00
Yuriy Rymarchuk a8b3baf2b8 Add support for roaring_bitmap_xor_inplace (#1983) 2023-05-18 13:28:32 +02:00
Ivan Nardi b1bcf1ff60 Fix classification-by-ip in ndpi_detection_giveup (#1981)
Return the "classification-by-ip" as protocol results only if no other
results are available.
In particular, never return something like
"protocol_by_port/protocol_by_ip" (i.e. `NTP/Apple`,
BitTorrent/GoogleCloud`, `Zoom/AWS`) because this kind of classification
is quite confusing, if not plainly wrong.

Notes:
* the information about "classification-by-ip" is always available, so
no information is lost with this change;
* in the unit tests, the previous classifications with confidence
`NDPI_CONFIDENCE_DPI_PARTIAL` were wrong, as noted in #1957
2023-05-17 11:26:25 +02:00
Ivan Nardi ace32c9dfe Add support for SRTP (#1977)
The goal is to have Zoom flows classified as "Encrypted" and not as
"Cleartext".

Start documenting the list of protocols supported by nDPI;
format, verbosity and content are still a work-in-progress.
2023-05-16 16:02:14 +02:00
Ivan Nardi 4e186f6bfb HTTP: rework state machine (#1966)
The goal if to correlate the right request-response pair, exporting
metadata from only one transaction (for example, the right url & return
state pair)

As a nice side effect, the code should be much cleaner, but that is a
matter of taste.

Two differences respect to the previous code:
* as it happens in the CI, if in the flow there are only one response
(before) and one request (after), only the metadata of the response are
saved/exported
* for performance reasons, we don't call `ndpi_parse_packet_line_info()`
anymore for ALL packets triggering the HTTP dissector, but only for the
packets that we already know belong to an HTTP flow. This is the reason
for the changes in RTSP/SOAP/... code
2023-05-16 16:01:55 +02:00
Ivan Nardi 8c224b464f Fix MS Teams detection with heuristic (#1972)
When we reconcile a TLS session to Teams, we need to keep TLs as master.
This way:
* we keep exporting all the TLS metadata
* we avoid some memory leaks (of these metadata themeselves)
2023-05-10 21:00:27 +02:00
Luca Deri 596a5fd5dc Updated results 2023-05-10 17:11:14 +02:00
Luca 64853dff02 Improved MS Teams detection with heuristic 2023-05-10 16:59:28 +02:00
Luca 98f029a091 Enhanced MS teams STUN/Azure detection 2023-05-10 16:28:34 +02:00
Luca 7d5e4d0f7e MS Teams enhancement 2023-05-10 15:41:26 +02:00
Luca 8de7c49298 MS Teams improvements 2023-05-10 15:16:39 +02:00
Ivan Nardi 31d23aee56 All protocols should be excluded sooner or later (#1969)
For a lot of protocols, reduce the number of packets after which the
protocols dissector gives up.
The values are quite arbitary, tring to not impact on classification
2023-05-10 12:50:24 +02:00
Ivan Nardi 99d7066ea0 Remove special handling of some TCP flows without SYN (#1965)
This piece of code has multiple problems:
* nDPI is able to detect some TCP protocols even with mid-flows (i.e.
without the initial packets of the session); TLS is the most
significative example
* since e6b332aa4a it is perfectly valid
to not pass the TCP Handshake packets to nDPI
* in any case, we shouldn't call `ndpi_detection_giveup()`. That
function is usually called by the application and we end up calling it
twice in some cases.

The simple solution is to completely remove that code: process these
kinds of flows like everyone else.

Note that the application can always avoid to pass to nDPI any TCP flows
without the initial handshake; the flow managemnt is always up to the
application.

Looking at the CI results, some rare flows are now processed significantly
longer. As a follow-up we could look into that.
2023-05-09 19:36:02 +02:00
Ivan Nardi 684e041998 Improve detection of crawlers/bots (#1968)
Add support for Facebook crawler
2023-05-09 16:42:29 +02:00
Ivan Nardi 79c1dbe0b2 Extend content-match list (#1967) 2023-05-09 16:41:12 +02:00
Luca Deri 2f9059dc54 Disabled query string validation in MDNS in order to avoid zapping chars that in DNS (instead) are not permitted 2023-05-09 15:04:02 +02:00
Luca Deri bfe79243bc Refreshed lists 2023-05-08 16:51:38 +02:00
Clément BRUGUERA 998bedbbb7 add 2 ns from fdn.fr to DoH section (#1964) 2023-05-05 18:08:13 +02:00
Ivan Nardi 0223d3c4f5 HTTP: improve extraction of metadata and of flow risks (#1959) 2023-05-05 13:35:20 +02:00
Ivan Nardi 02a2c80453 ndpiReader: fix print of flow payload (#1960) 2023-05-04 11:52:48 +02:00
Ivan Nardi a11ddd2dc4 HTTP: remove useless code about XBOX (#1958)
Old nDPI versions were able to detect XBOX flows over HTTP via
user-agent matching. This feature has been removed from a long time
(89d548f9d, at very least)
2023-05-04 11:28:03 +02:00
Ivan Nardi 6b94c9675a Improve detection of crawler/bot traffic (#1956) 2023-05-04 11:27:34 +02:00
Ivan Nardi 9a5d60bff6 Sync unit tests results (#1962) 2023-05-04 09:47:52 +02:00
Luca Deri 3681de4795 Updated pcap detection results after Facebook Reel/Stories support 2023-05-03 22:52:58 +02:00
Luca Deri de13c6c0bd Added detection of Facebook Reels and Stories
https://www.switcherstudio.com/blog/the-ultimate-guide-to-reels-and-stories#:~:text=Audience%3A%20Reels%20are%20served%20to,things%20like%20photos%20and%20boomerangs.
2023-05-03 22:50:02 +02:00
Ivan Nardi bb370f5ef0 Make Bittorrent LRU cache IPv6 aware. (#1909)
It was the only remaining LRU cache without IPv6 support.
See 81e1ea545c
2023-04-29 11:09:23 +02:00
Luca Deri 6b803f48dc Added missing check 2023-04-28 23:08:54 +02:00
Luca Deri 6e9dce357b Added ndpi_guess_undetected_protocol_v4() to restore logic that
was removed in https://github.com/ntop/nDPI/commit/032e778a6dc1ee2173c4cdb2c671180c5b5df245
2023-04-28 23:00:33 +02:00
Luca Deri f7138f07a6 Missing module termination 2023-04-28 23:00:33 +02:00
Luca Deri b39ece4428 Updated (C) 2023-04-28 23:00:33 +02:00
Luca Deri 791d3e6ad3 Added missing check 2023-04-28 23:00:33 +02:00
Ivan Nardi 8934f7b45f Add an heuristic to detect/ignore some anomalous TCP ACK packets (#1948)
In some networks, there are some anomalous TCP flows where the smallest
ACK packets have some kind of zero padding.
It looks like the IP and TCP headers in those frames wrongly consider the
0x00 Ethernet padding bytes as part of the TCP payload.
While this kind of packets is perfectly valid per-se, in some conditions
they might be treated by the TCP reassembler logic as (partial) overlaps,
deceiving the classification engine.
Add an heuristic to detect these packets and to ignore them, allowing
correct detection/classification.

This heuristic is configurable. Default value:
* in the library, it is disabled
* in `ndpiReader` and in the fuzzers, it is enabled (to ease testing)

Credit to @vel21ripn for the initial patch.

Close #1946
2023-04-25 19:25:07 +02:00
Ivan Nardi 40b6d5a2e1 fuzz: extend fuzzers coverage (#1952) 2023-04-25 16:37:28 +02:00
Ivan Nardi de693cbbc9 RTP: remove dead-code (#1953)
The checks `isValidMSRTPType(..) == 1` is a subset of
`is_valid_rtp_payload_type()` so this if-branch is never reached.

More importantly, the article describing how to detect Microsoft Lync and
Skype for Business is from 2014. These payload types are static or they
are in the dynamic range: in both cases, these values might be used (and
they are used indeed) pretty much by every application.
Bottom line: we can't use PT alone to identify a specific protocol.

Keep the list, since it is used to tell audio streams from video ones.
2023-04-25 14:16:27 +02:00
Luca Deri c4ba981370 Compilation fix on RH7 2023-04-24 08:49:11 +02:00
nikitamishagin d3e67faf20 Add "Heroes of the Storm" video game signature detection. (#1949) 2023-04-22 18:28:43 +02:00
Toni b6629ba2be Improved debug output. (#1951)
* try to get rid of some `printf(..)`s as they do not belong to a shared library
 * replaced all `exit(..)`s with `abort()`s to indicate an abnormal process termination

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-21 12:40:26 +02:00
Toni 85751ba9d7 Added OICQ dissector. (#1950)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-21 11:30:10 +02:00
Ivan Nardi ba99330031 ndpiReader: fix flow stats (#1943) 2023-04-14 11:17:37 +02:00
Ivan Nardi 032e778a6d Simplify ndpi_internal_guess_undetected_protocol() (#1941)
`ndpi_guess_undetected_protocol()/ndpi_internal_guess_undetected_protocol()`
is a strange function:
* it is exported by the library and it is actively used by `ntopng`
* it is intrinsecally ipv4-only
* it returns basically something like "classification_by_ip"/"classification_by_port"
(these information have already been calculated in `ndpi_do_guess()`...)
* it access the bittorrent LRU caches (similarly to
`ndpi_detection_giveup()` but without all the other caches...)

So:
* make the interface IPv4/6 agnostic
* use the classifications already available

This work will allow to make the Bittorrent caches IPV6-aware (see
81e1ea5).

Handle Dropbox classification "by-port" in the "standard" way.
2023-04-12 14:39:10 +02:00
Toni 6660feed35 Added BACnet dissector. (#1940)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 20:26:27 +02:00
lns 77ea80862f Improved debug logging.
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 15:39:09 +02:00
lns 6da958655c Improved invalid logging via printf().
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 15:39:09 +02:00
Toni bebcce5281 Added Source Engine dissector. (#1937)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-04-11 14:31:57 +02:00
Luca Deri 9ba16a4837 Test files for riit games 2023-04-11 10:14:18 +02:00
Jacopo a469009842 RiotGames: add detection of flows (#1935)
* RiotGames: add detection of flows

* remove of akamai domains
2023-04-11 10:04:09 +02:00
Ivan Nardi c7ef3608d9 CI: fix Performance job (#1936)
Fix: 7714507f
2023-04-09 10:16:28 +02:00
Luca Deri c47e9d201d Implemented ndpi_XXX_reset() API calls whre XXX is ses, des, hw 2023-04-08 09:52:14 +02:00
Ivan Nardi 7714507f81 Test multiple ndpiReader configurations (#1931)
Extend internal unit tests to handle multiple configurations.
As some examples, add tests about:
* disabling some protocols
* disabling Ookla aggressiveness

Every configurations data is stored in a dedicated directory under
`tests\cfgs`
2023-04-06 11:30:36 +02:00
Ivan Nardi 7a627296f0 Fix LRU/Patricia/Automa stats in ndpiReader with multiple threads (#1934) 2023-04-06 09:36:11 +02:00
Ivan Nardi b639744c3a Extend content list of Microsoft protocols (#1930) 2023-04-06 09:35:41 +02:00
Ivan Nardi 22cf8b5d8f configure: add an option to enable debug build, i.e -g (#1929)
Add this new flag to one CI job, to test it

Close #1925
2023-04-06 09:35:27 +02:00
Ivan Nardi 06a63dcbaa MySql: improve detection (#1928)
Add support for flows with "caching_sha2_password" authentication plugin.

See #1924
2023-04-06 09:34:39 +02:00
Ivan Nardi 5c28dbbae2 H323: fix false positives (#1916) 2023-04-06 09:32:57 +02:00
Ivan Nardi 25c1111911 fuzz: add a new fuzzer triggering the payload analyzer function(s) (#1926) 2023-04-04 14:39:29 +02:00
Maatuq f1193d5e6f add support for gre decapsulation (#1442) (#1921) 2023-04-04 14:20:11 +02:00
Ivan Nardi cc5aec5f0a fuzz: add fuzzer to test internal gcrypt code (#1920) 2023-04-04 14:19:41 +02:00
Ivan Nardi d7662379b4 Update libinjection code (#1918)
Update libinjection code to the current master https://github.com/libinjection/libinjection/commit/7e4b74e824dc3f0623ba4894df2a0d817351ec39

The goal is to finally fix #1820
See: https://github.com/libinjection/libinjection/issues/33

Update the corpus of the libinjection fuzzers

Close #1820
2023-04-04 14:18:57 +02:00
Ivan Nardi 4d11941d32 Ookla: rework detection (#1922)
The logic of the LRU cache has been changed: once we know an ip has
connected to an Ookla server, all the following (unknown) flows (for
a short time interval) from the same ip to the port 8080 are treated
as Ookla ones.

Most of the changes in this commit are about introducing the concept of
"aggressive detection". In some cases, to properly detect a
protocol we might use some statistical/behavior logic that, from one
side, let us to identify the protocol more often but, from the other
side, might lead to some false positives.
To allow the user/application to easily detect when such logic has been
triggered, the new confidence value `NDPI_CONFIDENCE_DPI_AGGRESSIVE` has been
added.
It is always possible to disable/configure this kind of logic via the
API.

Detection of Ookla flows using plain TLS over port 8080 is the first
example of aggressive detection in nDPI.

Tested with:
* Android 9.0 with app 4.8.3
* Ubuntu 20.04 with Firefox 110
* Win 10 with app 1.15 and 1.16
* Win 10 with Chrome 108, Edge 108 and Firefox 106
2023-03-30 17:13:51 +02:00
Ivan Nardi 3e06bcce8d Add another example of custom rules (#1923)
Add an example where traffic matching the same IP, but different ports
is classified to different protocols.

Close #189
2023-03-30 08:45:17 +02:00
Ivan Nardi 6e0cd16225 TLS: fix another interger overflow in certificate processing (#1915) 2023-03-30 05:12:40 +02:00
Maatuq 9ea8a57839 remove redefinition to vxlanhdr struct in vxlan dissector (#1911) 2023-03-25 19:25:22 +01:00
Ivan Nardi 04a426feef ndpiReader: fix VXLAN de-tunneling (#1913)
```
==20665==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6040000aec81 at pc 0x0000004f5c6f bp 0x7fff07e9e1f0 sp 0x7fff07e9e1e8
READ of size 1 at 0x6040000aec81 thread T0
SCARINESS: 12 (1-byte-read-heap-buffer-overflow)
    #0 0x4f5c6e in ndpi_is_valid_vxlan ndpi/example/reader_util.c:1784:6
    #1 0x4f5c6e in ndpi_workflow_process_packet ndpi/example/reader_util.c:2292:16
    #2 0x4dd821 in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_ndpi_reader.c:135:7
    #3 0x4f91ba in ExecuteFilesOnyByOne /src/aflplusplus/utils/aflpp_driver/aflpp_driver.c:234:7
    #4 0x4f8f8c in main /src/aflplusplus/utils/aflpp_driver/aflpp_driver.c:318:12
    #5 0x7f2289324082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
    #6 0x41e6cd in _start
```
Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=57369
2023-03-25 19:19:51 +01:00
Ivan Nardi ddb9aa0f95 TLS: fix parsing of certificate elements (#1910)
```
==1228==ERROR: AddressSanitizer: SEGV on unknown address 0x6040000bed05 (pc 0x00000056e148 bp 0x7ffcca534320 sp 0x7ffcca5330c0 T0)
==1228==The signal is caused by a WRITE memory access.
    #0 0x56e148 in processCertificateElements ndpi/src/lib/protocols/tls.c:682:79
    #1 0x56c60f in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_tls_certificate.c:43:3
    #2 0x43de63 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15
    #3 0x4295c2 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:324:6
    #4 0x42ee6c in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:860:9
    #5 0x4583a2 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
    #6 0x7f8c021c9082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
    #7 0x41f78d in _start
```

Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=57317
2023-03-22 18:18:36 +01:00
Maatuq 530d0de438 Add support for vxlan decapsulation (#1441) (#1900)
Close #1441
2023-03-22 18:18:12 +01:00
Ivan Nardi 04e017a854 Rework CI jobs to try reducing CI duration (#1903)
CI duration is quite long: the longest jobs is the "Performance" one.
Try to reduce the overall duration: that job (and some others) will not
be triggered for each PR/commit anymore, but asynchronously, once a day
(this scheduling seems right since the frequency of the PR/commits in
the project).
It should be possibly to trigger them manually, via GUI, anyway.

Remove two identical jobs; we already tests ASAN with 4 different
compilers.

After 9eff0754 it is safe to reduce fuzzing time.

Bottom line: try to have as upper-time of CI tests the duration of the
fuzzing jobs
2023-03-22 18:17:25 +01:00
Luca Deri 64ebf73b29 Added the ability to define custom protocols with arbitrary Ids in proto.txt
Example
- ip:213.75.170.11/32:443@CustomProtocol
nDPI assigns an is that can change based on protos.txt content

- ip:213.75.170.11/32:443@CustomProtocol=9999
nDPI assigns 9999 as protocolId to CustomProtocol and won't change when
protos.txt content will chaneg
2023-03-22 00:15:56 +01:00
Luca Deri b3457c644f Removed overlapping port 2023-03-21 19:27:56 +01:00
Luca Deri c22d30786c Improved checks for duplicated entries in protocols file 2023-03-21 19:25:03 +01:00
Ivan Nardi 2a3ade397b DisneyPlus/Hulu ip lists should be auto-generated (#1905)
Remove two stale ip lists:
1) these 3 ips are in the Amazon ranges (now)...
2) the Instagram list originated from AS32934, which is now a Facebook
AS; see https://github.com/ntop/nDPI/pull/1264/commits/8dabd06301a802dd38616ba8684a1d995783e023
2023-03-20 19:26:40 +01:00
Ivan Nardi 3a75a46212 Add a new protocol id for generic Adult Content traffic (#1906)
The list has been taken from https://www.similarweb.com/top-websites/adult/
Fix a GoTo false positive.
2023-03-20 17:56:02 +01:00
Ivan Nardi 12867962b0 Add a new protocol id for generic advertisement/analytics/tracking stuff (#1904) 2023-03-20 17:01:18 +01:00
Ivan Nardi 0e80828e14 fuzz: add a new fuzzer to test TLS certificates (#1901) 2023-03-20 17:00:46 +01:00
Luca Deri 9f46d41eca FreeBSD compilation fix
(C) update
2023-03-14 22:16:57 +01:00
Ivan Nardi 9eff075419 fuzz: simplify fuzzers dependencies in CIFuzz (#1896)
CIFuzz (based on oss-fuzz) is the GitHub action/CI job that runs fuzz
targets on pull requests. It only runs the fuzzers affected by a pull
request/commit. Otherwise it will divide up the allotted fuzzing time
among all fuzzers in the project.
Since:
* we have more than 20 fuzzers and most of them use the custom memory
allocation functions (to force allocation failures) even if they are not
strictly about DPI stuff;
* we need to keep fuzzing time relatively small (to avoid waiting the CI
results for a long time)

it is important that fuzzers dependencies (which are based on *files*
changed by the single commit/PR) are as small as possible.

Bottom line: move all the low-level allocation callbacks to a dedicated
file; this way most of the fuzzers don't depend anymore on `ndpi_main.c`
file (which is touched by ever commit/PR).

The goal is to have only the "most important" fuzzers running during (most
of) the CI.
2023-03-14 19:34:01 +01:00
Luca Deri 3585e2d201 Added ability to define an unlimited number of custom rules IP:port for the same IP (it used tobe limited to 2) 2023-03-13 21:57:14 +01:00
Luca Deri ebfc1fe827 Minor cleanup 2023-03-13 20:52:08 +01:00
Luca Deri d398f232be Added fix for better handling exceptions rollback in case of later match 2023-03-06 17:33:08 +01:00
Ivan Nardi 89cae9ddf2 Add a new flow risk about literal IP addresses used as SNI (#1892)
RFC 6066 3: "Literal IPv4 and IPv6 addresses are not permitted in
"HostName"."

Don't set this risk if we have a valid sub-classification (example:
via certificate)

Since a similar risk already exists for HTTP hostnames, reuse it, with a
more generic name.
2023-03-02 15:27:30 +01:00
Ivan Nardi 3047e286c0 Fix insert of ip addresses into patricia tree(s) (#1895)
Fix: d0b46dcf
2023-03-02 14:30:20 +01:00
Ivan Nardi 22fb8349b9 ndpiReader: print how many packets (per flow) were needed to perform full DPI (#1891)
Average values are already printed, but this change should ease to
identify regressions/improvements.
2023-03-01 21:50:47 +01:00
Toni 2768366969 Fixed missing AS_HELP_STRING in configure.ac. (#1893)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-03-01 20:46:40 +01:00
Nardi Ivan e0ad5b49c0 Sync unit tests results 2023-03-01 20:34:56 +01:00
Nardi Ivan d65f8cf15f Keep master protocol in ndpi_reconcile_protocols
As a side effect of this change, the following memory leak has been
fixed.
```
==55523==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 2 byte(s) in 1 object(s) allocated from:
    #0 0x55657d01b7be in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_alloc_fail+0x5b27be) (BuildId: 9dfa7bd9081421417b8b1c2e8b44633a9bfcfe52)
    #1 0x55657d169182 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:243:25
    #2 0x55657d1694c8 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:310:13
    #3 0x55657d25ddc7 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2308:46
    #4 0x55657d2701b9 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:895:5
    #5 0x55657d26a408 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1223:11
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56466
2023-03-01 20:34:56 +01:00
Nardi Ivan 9eb9664516 SoftEther: fix invalid memory access
We can't write `flow->protos` union until we are really sure about protocol
classification

```
==28334==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x558db5554512 bp 0x000000000000 sp 0x7ffcb22c2880 T0)
==28334==The signal is caused by a READ memory access.
==28334==Hint: this fault was caused by a dereference of a high value address (see register values below).  Disassemble the provided pc to learn which register was used.
    #0 0x558db5554512 in __asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x48e512) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7)
    #1 0x558db55ea54b in __interceptor_free (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x52454b) (BuildId: 2f71e395637a7b748f36d5a04c7281f18b1128d7)
    #2 0x558db56977ca in ndpi_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:274:7
    #3 0x558db56c20e3 in ndpi_free_flow_data /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5175:2
    #4 0x558db569783f in ndpi_free_flow /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8394:5
    #5 0x558db5627936 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:38:3
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=56272
2023-03-01 20:34:56 +01:00
Luca Deri 127d8d0d35 Fixed uninitialized variable 2023-02-27 21:26:41 +01:00
Luca Deri 21c7550157 Moving to bash 2023-02-27 16:21:12 +01:00
Luca Deri a303756edb Added missing break 2023-02-27 14:38:40 +01:00
Luca Deri 25420e266e Updated results after the latest changes 2023-02-27 12:20:39 +01:00
Luca Deri 807c97344f in case of failure, failing result files are not listed 2023-02-27 12:20:15 +01:00
Luca Deri 96f0f85e56 Indent fix 2023-02-27 12:20:06 +01:00
Luca Deri 6a2e8c9fb1 Improved protocol detection exploiting IP-based guess
Reworked ndpi_reconcile_protocols() that is now called only in front of a match (less overhead)
2023-02-27 12:18:39 +01:00
Luca Deri d0b46dcf88 Enhanced custom port definition and improved error reporting in case of duplications 2023-02-24 09:35:26 +01:00
Ivan Nardi 4075324e2b fuzz: extend fuzz coverage (#1888) 2023-02-16 18:04:34 +01:00
Ivan Nardi 06d4f8c7e5 Update GitHub runners versions (#1889)
See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
2023-02-14 21:30:54 +01:00
Ivan Nardi f2f8f9d0ee Fix export/serialization of flow->risk (#1885)
Any risks bigger than 31 was never exported/serialized; found looking
a the fuzz coverage report.
2023-02-14 21:30:36 +01:00
Ivan Nardi 090ba5735f Sync utests results (#1887) 2023-02-13 13:17:01 +01:00
Ivan Nardi f10178f8d2 Fix packet counters (#1884)
We need to keep separete counters to keep tracks of packet numbers with
and without any payload.

Regression introduced in 5849863ef
2023-02-13 11:32:17 +01:00
Ivan Nardi 200d1d4cfc Fix detection of packet direction and NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1883) 2023-02-13 11:31:52 +01:00
Ivan Nardi a3a9a72f31 Fix compilation with GCC-7 and latest RoaringBitmap code (#1886)
Latest RoaringBitmap version (introduced with bf413afb) triggers a new
warning with GCC-7:

```
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ CC=gcc-7 CXX=g++-7 ./autogen.sh && make -s
autoreconf: Entering directory `.'
[...]
third_party/src/roaring.c:1815:1: warning: ‘no_sanitize’ attribute directive ignored [-Wattributes]
 static inline int array_container_cardinality(const array_container_t *array) {
 ^~~~~~
third_party/src/roaring.c:1964:5: warning: ‘no_sanitize’ attribute directive ignored [-Wattributes]
     const array_container_t *container2) {
[..]
```

The core issue is that `no_sanitize` attribute is defined only for GCC
>= 8.
That breaks the CI since we still use GCC-7 and `-Werror`: add a simple
workaround.

Fix compilation on Windows
2023-02-13 11:19:11 +01:00
Luca Deri 85bab9e660 Centos7 fixes 2023-02-13 01:02:27 +01:00
Luca Deri bf413afba1 Update roaring bitmap code 2023-02-12 12:50:24 +01:00
0xA50C1A1 ba4e145aad Add Yandex services detection (#1882)
Add Yandex services detection

Add VK and Yandex to the TLS certificate match list
2023-02-09 20:02:43 +01:00
Ivan Nardi b51a2ac72a fuzz: some improvements and add two new fuzzers (#1881)
Remove `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` define from
`fuzz/Makefile.am`; it is already included by the main configure script
(when fuzzing).

Add a knob to force disabling of AESNI optimizations: this way we can
fuzz also no-aesni crypto code.

Move CRC32 algorithm into the library.

Add some fake traces to extend fuzzing coverage. Note that these traces
are hand-made (via scapy/curl) and must not be used as "proof" that the
dissectors are really able to identify this kind of traffic.

Some small updates to some dissectors:

CSGO: remove a wrong rule (never triggered, BTW). Any UDP packet starting
with "VS01" will be classified as STEAM (see steam.c around line 111).
Googling it, it seems right so.

XBOX: XBOX only analyses UDP flows while HTTP only TCP ones; therefore
that condition is false.

RTP, STUN: removed useless "break"s

Zattoo: `flow->zattoo_stage` is never set to any values greater or equal
to 5, so these checks are never true.

PPStream: `flow->l4.udp.ppstream_stage` is never read. Delete it.

TeamSpeak: we check for `flow->packet_counter == 3` just above, so the
following check `flow->packet_counter >= 3` is always false.
2023-02-09 20:02:12 +01:00
0xA50C1A1 4bb851384e Add VK detection (#1880) 2023-02-02 15:27:59 +01:00
Luca Deri 64b8ae38db Typos 2023-02-01 21:15:30 +00:00
Luca Deri 563a24bf6b Minor update 2023-02-01 18:31:59 +01:00
Luca Deri c711f480e3 Moved to 4.7 2023-02-01 18:30:17 +01:00
Alfredo Cardigliano b0a340cbc6 Update CHANGELOG.md 2023-02-01 10:23:41 +01:00
Luca Deri c42b046d87 Updated elastic_search file name in project 2023-01-31 21:02:41 +01:00
Luca Deri 80368772bd File rename to avoid name clashes (e.g. on Windows) 2023-01-31 20:54:00 +01:00
Luca Deri 2086058d00 Updated (C) 2023-01-31 20:08:30 +01:00
Alfredo Cardigliano a151276db8 Update changelog 2023-01-31 17:25:55 +01:00
Ivan Nardi fe043d3af5 Further reduction of the size of some traces used as unit test (#1879)
See a944514d. No flow/classification/metadata have been removed.
2023-01-30 11:10:54 +01:00
Ivan Nardi 9f27cd56b0 ndpiReader: fix packet dissection (CAPWAP and TSO) (#1878)
Fix decapsulation of CAPWAP; we are interested only in "real" user data
tunneled via CAPWAP.
When Tcp Segmentation Offload is enabled in the NIC, the received packet
might have 0 as "ip length" in the IPv4 header
(see
https://osqa-ask.wireshark.org/questions/16279/why-are-the-bytes-00-00-but-wireshark-shows-an-ip-total-length-of-2016/)

The effect of these two bugs was that some packets were discarded.

Be sure that flows order is deterministic
2023-01-30 10:59:18 +01:00
Ivan Nardi 3e6cadbb76 ndpireader: fix "Discarded bytes" statistics (#1877) 2023-01-27 07:09:34 +01:00
Ivan Nardi 6d00a9e0bd fuzz: add a new fuzzer to test serialization/deserialization code (#1876)
Autodetecting the needed buffer size is quite complex (especially with
float/double values) so it is mandatory to properly check for
`ndpi_snprintf` truncation.
These issues have been undetected so far probably because the default
buffer is big enough for all common cases.

Add an example of usage of `ndpi_deserialize_clone_all()` (taken from
`ntopng`)
2023-01-27 07:09:18 +01:00
Luca Deri eea9956430 Improved connection refused detection 2023-01-25 22:18:50 +01:00
Ivan Nardi b2a8b48386 Fix compilation and CI (#1875)
ubuntu-latest s390x CI doesn't like snapshot bigger than 262144 bytes.

Fix an error found by fuzz CI

```
=================================================================
==55399==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000008 (pc 0x561e41cb684d bp 0x7ffd54ce3650 sp 0x7ffd54ce3520 T0)
==55399==The signal is caused by a READ memory access.
==55399==Hint: address points to the zero page.
    #0 0x561e41cb684d in ndpi_network_ptree_match /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2321:41
    #1 0x561e41d30879 in ndpi_guess_undetected_protocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7767:8
    #2 0x561e41ca804d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_config.cpp:104:5
    #3 0x561e41bb96a0 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x4726a0) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
    #4 0x561e41ba3c2f in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x45cc2f) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
    #5 0x561e41ba96f6 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x4626f6) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
    #6 0x561e41bd22e2 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x48b2e2) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
    #7 0x7f94f0e5c082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #8 0x561e41b9eb0d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_config+0x457b0d) (BuildId: d4741c753aafe7c0df2681a592b7df16b38240e9)
```
2023-01-25 14:49:45 +01:00
Ivan Nardi 3477560639 Update every ip lists (#1872) 2023-01-25 11:53:18 +01:00
Luca Deri 464269e5ed CenOS compilation fix 2023-01-25 11:52:46 +01:00
Luca Deri a3d3856c80 Compilation fixes 2023-01-25 11:45:35 +01:00
Ivan Nardi 9fc724de5a Add some fuzzers to test other data structures. (#1870)
Start using a dictionary for fuzzing (see:
https://llvm.org/docs/LibFuzzer.html#dictionaries).
Remove some dead code.
Fuzzing with debug enabled is not usually a great idea (from performance
POV). Keep the code since it might be useful while debugging.
2023-01-25 11:44:59 +01:00
Ivan Nardi 29c5cc39fb Some small changes (#1869)
All dissector callbacks should not be exported by the library; make static
some other local functions.
The callback logic in `ndpiReader` has never been used.
With internal libgcrypt, `gcry_control()` should always return no
errors.
We can check `categories` length at compilation time.
2023-01-25 11:44:09 +01:00
Luca Deri 5849863ef9 Added new risk NDPI_TCP_ISSUES 2023-01-24 22:58:17 +01:00
Ivan Nardi 5e8c1ebbb7 fuzz: fix memory allocation failure logic (#1867)
We *do* want to have some allocation errors.
Fix some related bugs
Fix: 29be01ef
2023-01-20 14:27:33 +01:00
Ivan Nardi 496b284c98 Bittorrent: fix detection over TCP (#1868)
Close #1866
2023-01-19 22:02:23 +01:00
Ivan Nardi c4274a4aec Minor changes in ndpi_detection_giveup() (#1861)
Check the mining cache at the end of the function, like all the others
LRU caches.
Rewrite the STUN checks: same identical semantic but in a cleaner way,
hopefully
2023-01-19 21:45:35 +01:00
Ivan Nardi f5c2f9280c Fix undefined-behaviour in ahocorasick callback (#1864)
```
ndpi_main.c:2119:28: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
```
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=55226
2023-01-19 14:13:17 +01:00
Ivan Nardi f95bdaf625 Bittorrent: fix heap-buffer-overflow (#1863)
```
==258287==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600068ff9d at pc 0x5653a6e35def bp 0x7ffeef5aa620 sp 0x7ffeef5a9dc8
READ of size 22 at 0x60600068ff9d thread T0
    #0 0x5653a6e35dee in strncmp (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x4d2dee) (BuildId: 133b8c3c8ff99408109fcb9be2538bb8341f07f7)
    #1 0x5653a70d6624 in ndpi_search_bittorrent /home/ivan/svnrepos/nDPI/src/lib/protocols/bittorrent.c:500:71
    #2 0x5653a6ff255a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5686:6
    #3 0x5653a6ff331b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5722:10
    #4 0x5653a6ff2cbc in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5755:12
    #5 0x5653a70016bf in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6578:15
    #6 0x5653a6f1836d in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1678:31
    #7 0x5653a6f140a1 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2256:10
```
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=55218
Fix: 470eaa6f
2023-01-19 14:12:51 +01:00
Ivan Nardi edb8165ab9 Sync unit tests results (#1865) 2023-01-19 11:03:45 +01:00
Luca Deri bc44334599 Shrinked symbolic flow risks labels 2023-01-19 10:14:32 +01:00
Ivan Nardi 02f52207be Fuzz: fix compilation with nBPF (#1860)
Close #1859
2023-01-18 18:26:54 +01:00
Ivan Nardi 1b98bec0ab LRU caches: add a generic (optional and configurable) expiration logic (#1855)
Two caches already implemented a similar mechanism: make it generic.
2023-01-18 18:18:36 +01:00
Ivan Nardi de24206adc POP3: improve detection (#1856) 2023-01-18 07:19:44 +01:00
Ivan Nardi 97014c53f3 Improve support for Snapchat voip calls (#1858)
Latest Snapchat versions use QUICv1 for their audio/video real time
sessions. See c50a8d480
2023-01-17 22:02:23 +01:00
Ivan Nardi ccc5a20710 Add detection of Tailscale (#1857)
Extend the example of wireguard traffic
2023-01-17 21:55:38 +01:00
Ivan Nardi 470eaa6ff3 Add Meraki Cloud protocol and improve other protocols (#1854)
Avoid some LineCall and Jabber false positives.
Detect Discord mid flows.
Fix Bittorrent detection.
2023-01-17 11:38:39 +01:00
Ivan Nardi 29be01ef3a Add some fuzzers to test algorithms and data structures (#1852)
Fix some issues found with these new fuzzers
2023-01-17 08:31:59 +01:00
Ivan Nardi ebb9ebd2a0 Fix classification "by-port" (#1851)
Classification "by-port" should be the last possible effort, *after*
having test all the LRU caches.

Remove some dead code from `ndpi_detection_giveup()`:
`flow->guessed_protocol_id` is never set to any od those voip protocols
and at that point in this function we never have both a master *and* a
application protocols.
Coverage reports (both from unit tests and from fuzzing) confirms that
was dead code.
2023-01-17 08:26:42 +01:00
Luca Deri 1f7c57deff Improved DGA detection 2023-01-12 10:06:31 +01:00
Ivan Nardi a944514dde Reduce the size of some traces used as unit test (#1845)
No traces and no flows has been removed; only long sessions
has been reduced, keeping only their first packets.

This is quite important in fuzzing systems, since these pcaps are used
as initial seed. There is no value in fuzzing long sessions, because
only the very first packets are really used/processed by nDPI.

Before:
```
du -h tests/pcap/
200M	tests/pcap/
```
After:
```
du -h tests/pcap/
98M	tests/pcap/
```
2023-01-05 16:43:35 +01:00
Ivan Nardi 5a5e01a9e2 PPSTREAM: fix some heap-buffer overflows (#1846)
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=54802
2023-01-05 16:41:53 +01:00
Ege ÇetinandNardi Ivan 663df7e475 Fix missing bracket at ppstream (#1843)
* add missing bracket

* Sync unit test results

Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2023-01-03 20:11:24 +01:00
Ivan Nardi 9c83f0f3c5 Fix some warnings and two errors found while fuzzing (#1844)
Fix CI
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=54614
2023-01-03 13:21:54 +01:00
Luca Deri cb15d2b328 Added missing prototype 2023-01-03 12:58:52 +01:00
Luca Deri 2c551afbd7 Added NDPI_MINOR_ISSUES risk used for storing generic/relevant information about issues found on traffic. 2022-12-31 16:55:55 +01:00
Luca Deri 1735931f67 Added NDPI_PERIODIC_FLOW flow risk to be used by apps based on nDPI 2022-12-30 19:20:07 +01:00
Luca Deri 8f91b8ba72 Implemented EDNS(0) support in DNS dissector
Improved DNS dissection
2022-12-29 19:38:25 +01:00
Ivan Nardi 560280e6f0 fuzz: add fuzzer testing nDPI (initial) configurations (#1830)
The goal of this fuzzer is to test init and deinit of the library, with
different configurations. In details:
* random memory allocation failures, even during init phase
* random `ndpi_init_prefs` parameter of `ndpi_init_detection_module()`
* random LRU caches sizes
* random bitmask of enabled protocols
* random parameters of `ndpi_set_detection_preferences()`
* random initialization of opportunistic TLS
* random load/don't load of configuration files

This new fuzzer is a C++ file, because it uses `FuzzedDataProvider`
class (see
https://github.com/google/fuzzing/blob/master/docs/split-inputs.md).
Note that the (existing) fuzzers need to be linked with C++ compiler
anyway, so this new fuzzer doesn't add any new requirements.
2022-12-23 19:07:13 +01:00
Ivan Nardi 3de76812d9 Remove some old protocols (#1832)
These protocols:
* have been addeded in the OpenDPI era
* have never been updated since then
* we don't have any pcap examples [*]

If (and it is a big if...) some of these protocols are still somehow
used and if someone is still interested in them, we can probably
re-add them starting from scratch (because the current detection
rules are probably outdated)

Protocols removed: DIRECT_DOWNLOAD_LINK, APPLEJUICE, DIRECTCONNECT,
OPENFT, FASTTRACK, SHOUTCAST, THUNDER, AYIYA, STEALTHNET, FIESTA,
FLORENSIA, AIMINI, SOPCAST

PPSTREAM dissector works (...) only on UDP.

[*]: with do have an AIMINI test pcap but it was some trivial http
traffic detected only by hostname matching, on domains no more
available...
2022-12-23 10:04:08 +01:00
Ivan Nardi 5fafe8374a postgres: improve detection (#1831)
Remove some dead code (found via coverage report)
2022-12-22 21:41:32 +01:00
Ivan Nardi e9d5e72fb5 Fix infinite loop when a custom rule has port 65535 (#1833)
Close #1829
2022-12-21 10:55:14 +01:00
Ivan Nardi 22821241e2 Fix some errors found by oss-fuzz (#1834) 2022-12-20 22:50:18 +01:00
Ivan Nardi fa67166fcc Remove any references to LGTM (#1837) 2022-12-20 22:49:34 +01:00
Darryl Sokoloski b8972d1b38 Added TP-LINK Smart Home Protocol dissector. (#1841)
Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2022-12-20 20:46:06 +01:00
Ivan Nardi d8de8eba36 Sync unit tests results (#1840)
Update the documentation, hinting how to test/update *all* the unit
tests.
2022-12-19 17:57:44 +01:00
Darryl Sokoloski 9854aed408 Added TUYA LAN Protocol dissector. (#1838)
Tuya IoTOS Embedded Wi-Fi and BLE SDK for bk7231n.  Used by many "smart"
devices such as LED light strips, bulbs, etc.

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2022-12-19 17:02:19 +01:00
Ivan Nardi ad6bfbad4d Add protocol disabling feature (#1808)
The application may enable only some protocols.
Disabling a protocol means:
*) don't register/use the protocol dissector code (if any)
*) disable classification by-port for such a protocol
*) disable string matchings for domains/certificates involving this protocol
*) disable subprotocol registration (if any)

This feature can be tested with `ndpiReader -B list_of_protocols_to_disable`.

Custom protocols are always enabled.

Technically speaking, this commit doesn't introduce any API/ABI
incompatibility. However, calling `ndpi_set_protocol_detection_bitmask2()`
is now mandatory, just after having called `ndpi_init_detection_module()`.

Most of the diffs (and all the diffs in `/src/lib/protocols/`) are due to
the removing of some function parameters.

Fix the low level macro `NDPI_LOG`. This issue hasn't been detected
sooner simply because almost all the code uses only the helpers `NDPI_LOG_*`
2022-12-18 08:10:57 +00:00
Luca Deri fb0a73c0c7 Updated decription 2022-12-17 19:36:00 +01:00
Ivan Nardi 3c4b12e061 README: another try to fix build badge (#1836) 2022-12-16 12:22:07 +01:00
Ivan Nardi 390dc1d155 README: fix build badge link (#1835)
See: https://github.com/badges/shields/issues/8671
2022-12-16 12:04:16 +01:00
Luca Deri de289f1305 Added ability to decode DNS PTR records used for reverse address resolution 2022-12-15 21:22:53 +01:00
Luca Deri dbff2fbadd Minor RTP fix 2022-12-15 19:17:05 +01:00
Ivan Nardi 5704e4c142 STUN: add detection of ZOOM peer-to-peer flows (#1825)
See: "Enabling Passive Measurement of Zoom Performance in Production Networks"
https://dl.acm.org/doi/pdf/10.1145/3517745.3561414
2022-12-11 23:07:35 +01:00
Luca Deri 2edbe3c11f Centos7 fix 2022-12-11 10:24:42 +01:00
Luca Deri 88d4a25cb3 Rocky9 fix 2022-12-11 10:21:44 +01:00
Ivan Nardi 48a7f6d487 fuzz: some enhancements (#1827)
Load some custom configuration (like in the unit tests) and factorize some
(fuzzing) common code.

There is no way to pass file paths to the fuzzers as parameters. The safe
solution seems to be to load them from the process working dir. Anyway,
missing file is not a blocking error.

Remove some dead code (found looking at the coverage report)
2022-12-10 19:49:11 +01:00
Ivan Nardi 8c7071e040 DTLS: handle (certificate) fragments (#1811)
Keep using the existing function to handle reassembling buffer: rename
it from `ndpi_search_tls_tcp_memory` to
`ndpi_search_tls_memory` and make it "transport" agnostic
2022-12-10 18:32:25 +01:00
Ivan Nardi 79931b2ec7 Fix undefined-behavior when setting empty user-agent (#1821)
```
ndpi_main.c:9111:35: runtime error: null pointer passed as argument 2, which is declared to never be null
/usr/include/string.h:44:28: note: nonnull attribute specified here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior ndpi_main.c:9111:35 in
```
2022-12-10 18:31:51 +01:00
Luca Deri 9dc433f709 Packaing changes 2022-12-10 13:14:33 +01:00
Ivan Nardi 9f7ef72332 fuzz: fix signed-integer-overflow (#1822)
```
fuzz_ndpi_reader.c:33:29: runtime error: signed integer overflow: 214013 * 24360337 cannot be represented in type 'int'
    #0 0x4c1cf7 in fastrand ndpi/fuzz/fuzz_ndpi_reader.c:33:29
    #1 0x4c1cf7 in malloc_wrapper ndpi/fuzz/fuzz_ndpi_reader.c:38:11
    #2 0x523057 in ndpi_malloc ndpi/src/lib/ndpi_main.c:220:25
```
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=54112
2022-12-10 09:18:36 +01:00
Ivan Nardi 6004e55d7b Build: fix make install when --prefix= is used in configure script (#1824)
Use the same logic already used in `example/Makefile.in`

Close #1823
2022-12-10 09:13:20 +01:00
Luca Deri eacc2b8e32 Added Zoom screen share detection 2022-12-09 21:32:45 +01:00
Luca Deri fc7b070030 Added RTP stream type in flow metadata 2022-12-09 14:26:53 +01:00
Luca Deri 63f349319f Improved Zoom protocol detection 2022-12-08 22:00:24 +01:00
Ivan Nardi ada4fe4aa8 fuzz: add a new fuzzer testing memory allocation failures (#1818)
Try to fuzz error paths triggered by allocation errors.
Fix some errors already found by this new fuzzer.
Basic idea taken from: https://github.com/harfbuzz/harfbuzz/pull/2566/files

`FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` is a standard define used to
(not)compile specific code in fuzzing builds.
See: https://llvm.org/docs/LibFuzzer.html
2022-12-06 17:41:58 +01:00
Ivan Nardi 946c3dba0f Fix compilation (#1819) 2022-12-05 22:33:29 +01:00
Luca Deri 57562b3dfe Updated test results 2022-12-05 21:32:45 +01:00
Luca Deri e0afc16aa2 Exported HTTP server in metadata 2022-12-05 21:27:30 +01:00
Gowa2017 c882120afd thread_index may by negative. (#1814)
* thread_index may by negative.

Like:
192.168.8.155:55848 --> 183.3.224.139

* reader thread index also need to uint32
2022-12-05 10:22:05 +01:00
Ivan Nardi 3e4ab39b52 Add support for LTO and Gold linker (#1812)
This commit add (optional) support for Link-Time-Optimization and Gold
linker.
This is the first, mandatory step needed to make nDPI compliant with
"introspector" sanitizer requirements in OSS-Fuzz: see
https://github.com/google/oss-fuzz/issues/8939

Gold linker is not supported by Windows and by macOS, so this feature is
disabled by default. It has been enable in CI in two linux targets
("latest" gcc and clang).

Fix some warnings triggered by LTO.

The changes in `src/lib/ndpi_serializer.c` seams reasonable.
However, the change in `tests/unit/unit.c` is due to the following
warning, which seems to be a false positive.

```
unit.c: In function ‘serializerUnitTest’:
ndpi_serializer.c:2258:13: error: ‘MEM[(struct ndpi_private_serializer *)&deserializer].buffer.size’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
unit.c:67:31: note: ‘MEM[(struct ndpi_private_serializer *)&deserializer].buffer.size’ was declared here
   67 |   ndpi_serializer serializer, deserializer;
      |                               ^
ndpi_serializer.c:2605:10: error: ‘MEM[(struct ndpi_private_serializer *)&deserializer].status.buffer.size_used’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
unit.c:67:31: note: ‘MEM[(struct ndpi_private_serializer *)&deserializer].status.buffer.size_used’ was declared here
   67 |   ndpi_serializer serializer, deserializer;
```
Since this warning is triggered only with an old version of gcc and
`tests/unit/unit.c` is used only during the tests, the easiest fix has
been applied.

Some (unknown to me) combinations of OS and compiler trigger the
following warnings at linker time (with sanitizer and gold linker)
```
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load1_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load2_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load4_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load8_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_load16_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store1_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store2_asm'
/usr/bin/ld.gold: warning: Cannot export local symbol '__asan_report_store4_asm'
[..]
```
I have not found any references to this kind of message, with the only
exception of https://sourceware.org/bugzilla/show_bug.cgi?id=25975
which seems to suggest that these messages can be safely ignored.
In any case, the compilation results are sound.

Fix `clean` target in the Makefile in the `example` directory.

In OSS-Fuzz enviroments, `fuzz_ndpi_reader` reports a strange link error
(as always, when the gold linker is involved...).
It's come out that the culprit was the `tempnam` function: the code has
been changed to use `tmpfile` instead. No sure why... :(

Fuzzing target `fuzz_ndpi_reader.c` doesn't use `libndpiReader.a`
anymore: this way we can use `--with-only-libndpi` flag on Oss-Fuzz builds
as workaround for the "missing dependencies errors" described in
https://github.com/google/oss-fuzz/issues/8939
2022-12-05 10:21:42 +01:00
Ivan Nardi b9f63458e6 CI: GitHub is updating Ubuntu runners (#1817)
GitHub is moving `ubuntu-latest` to `ubuntu-22.04`: update our
dependencies.
See: https://github.blog/changelog/2022-11-09-github-actions-ubuntu-latest-workflows-will-use-ubuntu-22-04/

This is the reason of the recent random failures in CI.

Update "newest" tested gcc to gcc-12.

Fix a memory error introduced in 557bbcfc5a
2022-12-03 18:29:44 +01:00
Ivan Nardi 81e1ea545c Make LRU caches ipv6 aware (#1810)
Simplest solution, keeping the existing cache data structure

TLS certificate cache is used for DTLS traffic, too.

Note that Ookla cache already works with ipv6 flows.

TODO:
 * make the key/hashing more robust (extending the key size?)
 * update bittorrent cache too. That task is quite difficult because
   ntopng uses a public function (`ndpi_guess_undetected_protocol()`)
   intrinsically ipv4 only...
2022-12-03 12:07:32 +01:00
Luca Deri a387072872 Updated tests results 2022-12-03 11:33:04 +01:00
Luca Deri 2a8d0d1fe4 Added check for empty/missing user-agent in HTTP 2022-12-03 11:32:39 +01:00
Luca Deri 557bbcfc5a Improved HTTP header parsing skipping lines that won't match 2022-12-03 11:32:29 +01:00
Ivan Nardi 197a9a6cfc Extend CodeQL configuration and fix two warnings (#1816)
```
Call to `memset` may be deleted
Comparison of narrow type with wide type in loop condition
```

See https://www.usenix.org/system/files/conference/usenixsecurity17/sec17-yang.pdf
for the new portable function `secure_memzero`.
2022-12-02 23:26:22 +01:00
Alvaro Muñoz e49fe32f77 Add CodeQL workflow (#1815) 2022-12-02 13:06:54 +01:00
Ivan Nardi b2a1bf413b Remove some useless fuzzing target (#1813)
We don't need specific targets to reproduce fuzzing issues.
After all, calling `./fuzz/fuzz_process_packet_with_main $ARTIFACT_FILE`
is equivalento to `./fuzz/fuzz_process_packet $ARTIFACT_FILE`
2022-12-01 14:39:44 +01:00
Ivan Nardi 02e7e3c23b Add a new fuzzer for QUIC (#1800)
QUIC packets are encrypted/obfuscated; that means that we need to
decrypt them before parsing the real (TLS) message.
Fuzzing is not effective here, since a random buffer is hardly a valid
encrypted QUIC packet.

Add a new fuzzer, testing *decrypted* QUIC packets.

Add a basic corpus.

Fix a few bugs already found by this fuzzer.
2022-12-01 12:21:04 +01:00
Ivan Nardi afb5c3e6ba Fix classification "by-port" of QUIC flows (#1785)
nDPI is able to properly classify QUIC flows only if it elaborates the
very first packets of the flow.

The protocol list in `is_udp_guessable_protocol()` is basically a list
of protocols which can be detected from *any* packets in the flow.
Rename such function to `is_udp_not_guessable_protocol()`: the name is
still quite cryptic, but at least not plainly wrong
2022-11-22 18:00:02 +01:00
Ivan Nardi 6e6f1e4489 GitHub Actions: fix a warning (#1807)
```
The `set-output` command is deprecated and will be disabled soon.
Please upgrade to using Environment Files. For more information see:
https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
```
2022-11-22 16:44:08 +01:00
Ivan Nardi 2b65a4e05e TLS: be sure to always set ssl_version field (#1806)
Useful with asymmetric traffic with (D)TLS <= 1.2
2022-11-22 16:13:28 +01:00
Luca Deri 4231f48059 Added support for Linux Cooked Capture v2 2022-11-16 17:48:28 +01:00
Ivan Nardi 388d01a345 DNS: fix an infinite loop, due to an integer overflow (#1804)
Found by sydr-fuzz
Close #1803
2022-11-15 16:29:10 +01:00
Toni b5e8d86def Improved Teamspeak3 License/Weblist detection. (#1802)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-11-15 08:59:45 +01:00
Ivan Nardi cd41ab7c8f Improve export/print of L4 protocol information (#1799)
Close #1797
2022-11-13 22:35:46 +01:00
Ivan Nardi 07872b3527 GitHub Actions: update to Node.js 16 actions (#1801)
Fix warnings on recent CI results; example:
https://github.com/ntop/nDPI/actions/runs/3455588082

See: https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/
2022-11-13 19:42:13 +01:00
Toni 6d3446b114 Merge pull request #1796 from IvanNardi/issue-1792
RTP: remove a superfluous check
2022-11-07 20:38:40 +01:00
Nardi Ivan 52b562c328 Fix json export of ipv6 addresses
The "string" buffer was to short; better start using `INET6_ADDRSTRLEN`
as reported in the man page of `inet_ntop`.

Close: #1794
2022-11-07 20:36:55 +01:00
Nardi Ivan aa5dd25392 RTP: remove a superfluous check
Static analyzer complains about dereferencing `packet->udp` before
checking.
Since this function is called only with UDP flows, remove the check.

Close: #1792
2022-11-07 18:47:19 +01:00
Zied Aouini 9e73d61fb0 Fix supported versions. 2022-10-31 13:53:23 +01:00
Zied Aouini 41a5ba1b65 Add a note about required Python version. 2022-10-31 13:50:59 +01:00
Ivan Nardi db9f6ec1b4 Add basic profiling of memory allocations on data-path (#1789)
The goal is to have an idea of the memory allocation sizes performed in
the **library data-path**, i.e. excluding init/deinit phases and all
the allocations made by the application itself.
In other words, how much memory is needed per-flow, by nDPI, other than
`struct ndpi_flow_struct`?

It works only on single-thread configurations.

It is not enabled by default (in the unit tests) since different
canfiguration options (example: `--enable-pcre`) lead to diffferent
results.

See: #1781
2022-10-28 20:41:37 +02:00
Luca Deri 175981a422 Removed unused variable 2022-10-26 16:14:32 +02:00
Alfredo Cardigliano f30479b4a4 Fix typo 2022-10-26 16:05:32 +02:00
Luca Deri a16cd9634f Cleanup 2022-10-26 13:59:07 +02:00
Luca Deri 15d58db2f9 Typo 2022-10-26 13:56:24 +02:00
Luca Deri 55f885f3ed Improved AESNI check 2022-10-26 13:51:00 +02:00
Ivan Nardi de16fd35aa Hangout: avoid useless lookups in the protocol tree (#1755)
We already performed exactly these lookups in the generic code to
populate `flow->guessed_protocol_id_by_ip`: use it!

This code probably needs a deeper review, since it is basicaly a simple
matching on ip + port.
2022-10-25 18:24:03 +02:00
Ivan Nardi ca5ffc4988 TLS: improve handling of ALPN(s) (#1784)
Tell "Advertised" ALPN list from "Negotiated" ALPN; the former is
extracted from the CH, the latter from the SH.

Add some entries to the known ALPN list.

Fix printing of "TLS Supported Versions" field.
2022-10-25 17:06:29 +02:00
Ivan Nardi 2ed2e5dc7e Fix memory corruption on TCP flows (#1786) 2022-10-25 16:22:40 +02:00
Ivan Nardi 550f3e5b9d IRC: fix memory access (#1787) 2022-10-25 16:16:24 +02:00
Ivan Nardi e674a8ae37 Sync unit tests results (#1783) 2022-10-22 12:57:06 +02:00
Luca Deri 24cc949f14 Enhanced HTTP numeric IP check 2022-10-22 10:06:09 +02:00
Toni c521595383 Add Elasticsearch protocol dissector. (#1782)
* all credits goes to @verzulli

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-21 20:01:54 +02:00
Toni Uhlig 29242cbcb6 Add Munin protocol dissector.
* all credits goes to @verzulli

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-20 22:10:35 +02:00
Nardi Ivan 9c0caa5362 TLS: allow sub-classification via ALPN
In some rare cases, it is possible to sub-classify the flow via ALPN
matching. This is particularly usefull for asymmetric traffic where the
Client Hello doens't have the SNI.

For the time being there is only one rule, about ANYDESK.
2022-10-20 22:00:12 +02:00
Nardi Ivan c1b6aa9483 Sync unit tests results 2022-10-19 13:01:34 +02:00
Luca Deri 9a34d870e9 Updated FortiClient default ports 2022-10-19 11:25:19 +02:00
Luca Deri df0dd76ed2 Reworked severities adding critical and emergency, thus shifting values down 2022-10-19 10:53:11 +02:00
Ivan Nardi 35fdbc8148 TLS: explicit ignore client certificate (#1776)
TLS classification usually stops after processing *server* certificates
(if any). That means, that *client* certificate, if present, is usually
ignored.
However in some corner cases (i.e. unidirectional traffic) we might end
up processing client certificate and exposing its metadata: the issue is
that the application will think that this metadata are about the server
and not about the client.
So, for the time being, always ignore client certificate processing.

As a future work, we might find an efficient way to process and export both
certificates.
2022-10-18 16:40:15 +02:00
Nardi Ivan 223a6fb9f7 Fix a use-of-uninitialized-value error on PCRE code
This is likely a false positive, triggered by the fact that libpcre is
usually compiled without MASAN support.
It it was a real error, ASAN would complain loudly with a invalid-free
error at the end of the same function.

```
==83793==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x562296111174 in ndpi_compile_rce_regex /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1631:3
    #1 0x5622960e3e4a in ndpi_is_rce_injection /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1636:5
    #2 0x5622960de7cd in ndpi_validate_url /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1741:12
    #3 0x5622960dae45 in ndpi_dpi2json /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1362:29
    #4 0x5622960e2751 in ndpi_flow2json /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1512:10
    #5 0x562296033b0f in process_ndpi_collected_info /home/ivan/svnrepos/nDPI/example/reader_util.c:1310:9
    #6 0x5622960501f9 in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1659:2
    #7 0x562296045aef in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2202:10
    #8 0x562295e85374 in ndpi_process_packet /home/ivan/svnrepos/nDPI/example/ndpiReader.c:3937:7
    #9 0x7f1235053466  (/lib/x86_64-linux-gnu/libpcap.so.0.8+0x23466) (BuildId: b84c893ea2516d6fb2c1c6726b1fe93b3be78f61)
    #10 0x7f1235041f67 in pcap_loop (/lib/x86_64-linux-gnu/libpcap.so.0.8+0x11f67) (BuildId: b84c893ea2516d6fb2c1c6726b1fe93b3be78f61)
    #11 0x562295e53139 in runPcapLoop /home/ivan/svnrepos/nDPI/example/ndpiReader.c:4060:15
    #12 0x562295e51e7f in processing_thread /home/ivan/svnrepos/nDPI/example/ndpiReader.c:4130:3
    #13 0x7f1234e53608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8
    #14 0x7f1234d2f132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95

  Uninitialized value was created by an allocation of 'pcreErrorStr' in the stack frame
    #0 0x5622961108a6 in ndpi_compile_rce_regex /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:1603:3
```
2022-10-17 23:15:07 +02:00
Nardi Ivan 3d0c36cf22 Add some popular streaming applications
A simply back-porting from https://github.com/vel21ripn/nDPI/pull/142;
full credits to @ChrisNelson-CyberReef.

Close #1716
2022-10-17 23:08:25 +02:00
Toni 01622c4c63 Updated protocol dissector serialisation code. (#1767)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-15 12:58:44 +02:00
Ivan Nardi 2d153fba45 HTTP: fix stack-buffer-overflow (#1768)
```
==24879==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fa085b31e60 at pc 0x55cc63f203e2 bp 0x7ffc9ec91b10 sp 0x7ffc9ec91298
READ of size 17 at 0x7fa085b31e60 thread T0
    #0 0x55cc63f203e1 in printf_common(void*, char const*, __va_list_tag*) asan_interceptors.cpp.o
    #1 0x55cc63f20769 in vsnprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x50e769) (BuildId: cce2b6b1344bfd0bdc9626fef604c2b3caad485b)
    #2 0x55cc63f22210 in __interceptor_snprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x510210) (BuildId: cce2b6b1344bfd0bdc9626fef604c2b3caad485b)
    #3 0x55cc6420fc76 in ndpi_check_http_server /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:668:4
    #4 0x55cc6420344b in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:742:5
    #5 0x55cc642031ce in check_content_type_and_change_protocol /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:737:7
    #6 0x55cc641fac9f in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1352:4
    #7 0x55cc641f2fd5 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1461:3
    #8 0x55cc64085275 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5580:6
    #9 0x55cc64085c87 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5628:12
    #10 0x55cc64085927 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5647:12
    #11 0x55cc64095fcb in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6458:15
    #12 0x55cc63fd08b4 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #13 0x55cc63fd09f7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #14 0x7fa0880fb082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #15 0x55cc63efb45d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4e945d) (BuildId: cce2b6b1344bfd0bdc9626fef604c2b3caad485b)

Address 0x7fa085b31e60 is located in stack of thread T0 at offset 96 in frame
    #0 0x55cc6420f1bf in ndpi_check_http_server /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:644

  This frame has 5 object(s):
    [32, 36) 'a' (line 653)
    [48, 52) 'b' (line 653)
    [64, 68) 'c' (line 653)
    [80, 96) 'buf' (line 654)
    [112, 176) 'msg' (line 662) <== Memory access at offset 96 partially underflows this variable

```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=52229
2022-10-14 18:59:33 +00:00
Ivan Nardi 6c84ce85e4 ndpiReader: fix help message. There isn't a 'J' option (#1770) 2022-10-14 20:16:47 +02:00
Ivan Nardi 8c4ece941e Sync utests results (#1774) 2022-10-14 20:16:29 +02:00
Luca Deri 3bb6bf628d Removed gettimeofday as win files has located in windows/src/ 2022-10-14 19:01:41 +02:00
Luca Deri 1fecf69359 Added DNS error code mapping number -> string 2022-10-14 09:39:15 +02:00
Luca Deri cd76bacd4a Added detection of URLs starting with a dot (.) 2022-10-10 22:34:26 +02:00
Ivan Nardi 1796a1d814 LINE_CALL: add detection of LINE voip calls (#1761)
These flows are classifed as `LINE_CALL`; another option was
`RTP/LINE_CALL`. No sure about the best solution...

Extend LINE domains list.

Remove RTP dead code.
2022-10-06 17:09:26 +02:00
Luca Deri e4dcec560e Windows changes 2022-10-06 11:58:16 +02:00
Ivan Nardi c4fb2a590e irc: fix heap-buffer-overflow (#1765) 2022-10-06 09:25:21 +00:00
Luca Deri 3c0021d603 Fixes Stack-buffer-overflow READ {*} · check_content_type_and_change_protocol 2022-10-06 09:16:02 +02:00
Luca Deri fba926d6a4 Updated IRC result 2022-10-05 23:02:02 +02:00
Luca Deri 199c86e3df Improved credentials check in IRC 2022-10-05 22:51:20 +02:00
Luca 37c88b129f Added new flow risk NDPI_HTTP_OBSOLETE_SERVER. Currently Apache and nginx are supported 2022-10-04 22:32:45 +02:00
Nardi Ivan cca585053e Fix compilation and sync utests results 2022-10-04 22:17:05 +02:00
Luca 131d53896f Improved HTTP payload handling 2022-10-04 14:03:39 +02:00
Luca 3fa9416981 Improved detection of WebShell and PHP code in HTTP URLs that is reported via flow risk 2022-10-04 11:26:44 +02:00
Luca de59eb8237 Added the ability to track the payload via -E and via the new option 'ndpi_track_flow_payload' 2022-10-04 11:26:44 +02:00
Toni 66a8010ef7 Fix ndpi_timeval_to_milliseconds/microseconds for platforms with tv_usec is an unsigned int. (#1762)
* implicit conversion to an 32 bit unsigned int leads to an overflow

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-02 12:48:24 +02:00
Toni c83e0b3c8d Restore Confidence enum constants to ascending values. (#1760)
* NDPI_CONFIDENCE_MAX is useless otherwise since it is
   supposed the be the highest value w/o any gaps
 * for client applications, it makes sense to use that
   the numbers directly w/o any additional mapping

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-10-02 12:47:56 +02:00
sharonenochandSharon Enoch 503aac70bc Line app support (#1759)
* Standard support for LINE app

* Added test pcap for LINE app

* make check result for LINE app

* Make check success as 1kxun has LINE packets

* Added the ASN inc file for LINE

* Removed extra lines as its effecting make check

* Editing the SNI required a new pcap output file for TLS.Line format

* Run Configure with --with-pcre --with-maxminddb to enable the generation of h323-overflow.pcap.out

Co-authored-by: Sharon Enoch <sharone@amzetta.com>
2022-10-01 12:01:41 +02:00
Nardi Ivan f3a74d97d8 TLS/DTLS: we process certificate for UDP flows, too
Note that current code access `certificate_processed` state even before
setting the protocol classification, so this piece of information can't
be saved in `flow->protos` union.
2022-09-30 06:53:29 +02:00
Luca fbf30d1aad Fixed invalid unidirectional traffic alert for unidirectional protocols (e.g. sFlow) 2022-09-29 18:15:37 +02:00
Nardi Ivan a27538b764 Remove unused code
LRU callbacks have been added in 460ff3c7a, but they have never been
used and they have never been extended to the other LRU caches.

`ndpi_search_tcp_or_udp()` basically returns the classification by port/ip
of the flow; calling it from the dissector is useless.
The same for TOR detection: ips are checked in the generic code
2022-09-28 21:09:27 +02:00
Nardi Ivan ac7a3669da Fix value of ndpi_protocol->protocol_by_ip
Fix: a7c2734b
2022-09-25 13:31:57 +02:00
Nardi Ivan 39f9b8cadf DNS: change category of DNS flows
DNS flows should have `NDPI_PROTOCOL_CATEGORY_NETWORK` as category,
regardless of the subprotocol (if any).
Follow-up of 83de3e47
2022-09-25 13:31:04 +02:00
Toni Uhlig b7b9577254 Changed some protocol categories to align with others.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-23 20:35:55 +02:00
Nardi Ivan 1f345b311f Sizes of LRU caches are now configurable
0 as size value disable the cache.

The diffs in unit tests are due to the fact that some lookups are
performed before the first insert: before this change these lookups
weren't counted because the cache was not yet initialized, now they are.
2022-09-23 18:33:48 +02:00
Nardi Ivan 83de3e4716 DNS: change category of DNS flows
DNS flows should have `NDPI_PROTOCOL_CATEGORY_NETWORK` as category,
regardless of the subprotocol (if any).
2022-09-22 13:23:55 +02:00
Nardi Ivan b250f9d86c Extend content match lists 2022-09-22 09:30:09 +02:00
Nardi Ivan c6201284d7 NAT-PMP: fix metadata extraction 2022-09-21 20:02:23 +02:00
Toni 644ad34962 Improved NATPMP dissection. (#1745)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:24:04 +02:00
Toni Uhlig d6701e8979 Build ndpiReader and run regression tests.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:03:22 +02:00
segfault 3db6413ab0 Fixed VS2019 build.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:03:22 +02:00
Toni Uhlig 50ee6768b7 Provide extra ndpi_define.h for Visual Studio.
* Caution: file needs to keep in sync with `src/include/ndpi_define.h.in` manually

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:03:22 +02:00
Toni Uhlig d27fb1e73e Moved NDPI_API_VERSION to src/include/ndpi_define.h.in where it belongs.
That way, we can make `src/include/ndpi_api.h` independent of autoconf.
MSBuild will like that.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:03:22 +02:00
segfault 817a7c616d Retargeted to Windows 10.
* disable Spectre Mitigation (may change)
 * remove autoconf generated header files
2022-09-21 18:03:22 +02:00
mark andrews 5d5b46e514 Add proj and sln for compile of dynamic x64 lib under Visual Studio 2019.
* add CI support via MSBuild

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-21 18:03:22 +02:00
Ivan Nardi 03d217eae6 Fix CI after nBPF integration (#1746)
Add one CI job testing nBPF
2022-09-21 16:49:51 +02:00
Luca Deri c960bb50f0 Copilation fix 2022-09-21 10:27:46 +02:00
Luca Deri aef8e53b7d Added missing library 2022-09-21 08:24:37 +02:00
Luca Deri 1d0bc7c350 Finalized nBPF support. You can now define custom protocols such as
(see exaple/protos.txt)

nbpf:"host 192.168.1.1 and port 80"@HomeRouter

In order to have nBPF support, you need to compile nDPI with it. Just download
https://github.com/ntop/PF_RING in the same directory where you have downloaded
nDPI and compile PF_RING/userland/nbpf
2022-09-21 00:03:12 +02:00
Ivan Nardi a7c2734b38 Remove classification "by-ip" from protocol stack (#1743)
Basically:
* "classification by-ip" (i.e. `flow->guessed_protocol_id_by_ip` is
NEVER returned in the protocol stack (i.e.
`flow->detected_protocol_stack[]`);
* if the application is interested into such information, it can access
`ndpi_protocol->protocol_by_ip` itself.

There are mainly 4 points in the code that set the "classification
by-ip" in the protocol stack:  the generic `ndpi_set_detected_protocol()`/
`ndpi_detection_giveup()` functions and the HTTP/STUN  dissectors.

In the unit tests output, a print about `ndpi_protocol->protocol_by_ip`
has been added for each flow: the huge diff of this commit is mainly due
to that.

Strictly speaking, this change is NOT an API/ABI breakage, but there are
important differences in the classification results. For examples:
* TLS flows without the initial handshake (or without a matching
SNI/certificate) are simply classified as `TLS`;
* similar for HTTP or QUIC flows;
* DNS flows without a matching request domain are simply classified as
`DNS`; we don't have `DNS/Google` anymore just because the server is
8.8.8.8 (that was an outrageous behaviour...);
* flows previusoly classified only "by-ip" are now classified as
`NDPI_PROTOCOL_UNKNOWN`.

See #1425 for other examples of why adding the "classification by-ip" in
the protocol stack is a bad idea.

Please, note that IPV6 is not supported :(  (long standing issue in nDPI) i.e.
`ndpi_protocol->protocol_by_ip` wil be always `NDPI_PROTOCOL_UNKNOWN` for
IPv6 flows.

Define `NDPI_CONFIDENCE_MATCH_BY_IP` has been removed.

Close #1687
2022-09-20 22:24:47 +02:00
Toni 174cd739db Fix warnings on OpenWrt CCs, caused by the memcpy builtin. (#1741)
In file included from ../include/ndpi_includes.h:31,
                 from ../include/ndpi_main.h:27,
                 from ../include/ndpi_api.h:28,
                 from protocols/quic.c:27:
In function 'memcpy',
    inlined from 'tls13_hkdf_expand_label_context' at protocols/quic.c:473:5,
    inlined from 'tls13_hkdf_expand_label' at protocols/quic.c:498:10,
    inlined from 'quic_hkdf_expand_label.constprop' at protocols/quic.c:512:6:
/home/build/openwrt/staging_dir/toolchain-mips_24kc_gcc-11.3.0_musl/include/fortify/string.h:53:16: error: argument 2 null where non-null expected [-Werror=nonnull]
   53 |         return __builtin_memcpy(__od, __os, __n);
      |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
protocols/quic.c: In function 'quic_hkdf_expand_label.constprop':
/home/build/openwrt/staging_dir/toolchain-mips_24kc_gcc-11.3.0_musl/include/fortify/string.h:53:16: note: in a call to built-in function '__builtin_memcpy'

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-18 12:40:49 +02:00
Luca Deri 48d00eae35 Added MAX_NBPF_CUSTOM_PROTO define 2022-09-17 11:04:40 +02:00
Luca 0f07ad35be Initial work towards custom BPF protocol definition 2022-09-16 19:48:12 +02:00
Toni Uhlig a966d37a21 Add CryNetwork dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-16 16:48:43 +02:00
Toni Uhlig ec1981c20c Add Syncthing dissector.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-16 15:21:49 +02:00
Toni de020b1747 Add NATPMP dissector. (#1738)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-16 13:32:30 +02:00
Nardi Ivan 9ce4d40d14 Remove a case of guessed sub-classification
This code is triggered only for "unknown" flows with a valid
sni/hostname.
Why in that case the guessed classification should be
something like `DNS/Subprotocol_depending_on_hostname`? Why DNS as
master and not HTTP or TLS or QUIC?

Furthermore, I have not been able to trigger a positive match from that
lookup. I strongly think that if we had a valid subprotocol, we would
have a valid master in the first place.
In doubt, remove it completely.

As a follow up, we should investigate why some dissectors (the HTTP one,
at least) set the sni/hostname field without setting a valid protocol,
in the first place.
This behaviour seems quite suspicious, if not plainly buggy.
2022-09-14 17:52:01 +02:00
Nardi Ivan 7571f48392 Dissectors shouldn't update flow->guessed_host_protocol_id
The field `flow->guessed_host_protocol_id` is set at the beginning of
the flow analysis and it represents the "classification by ip" of the flow
itself.

This field should never be changed. Dissectors which want to provide an
"hint" about the classification, should update `flow->guessed_protocol_id`
instead. Such "hint" is useless if the dissector set the "extra-dissection"
data-path.

Rename such field to `guessed_protocol_id_by_ip` to better describe its
role.

Preliminary work necessary for #1687
2022-09-14 17:51:39 +02:00
Ivan Nardi b4f5243716 Anydesk: improve detection (#1735) 2022-09-13 20:55:11 +02:00
Nardi Ivan eea3c448de HTTP: remove some wrong code
This code seems wrong or in the wrong place, at least:
* "classification by port" and "classification by ip" protocols (i.e
"guessed" protocols) should be used to set the protocol stack only after
trying all the dissectors, and only by the generic code
* there are no reason (for a dissector) to update the "guessed"
information using the protocol stack values: it is usually the other way
around (see previous point)
2022-09-13 20:48:20 +02:00
Alfredo Cardigliano 973950d881 Replace obsolete linux macro 2022-09-13 10:41:44 +02:00
Ivan Nardi 4f584f78a0 Fix ndpi_do_guess() (#1731)
Avoid a double call of `ndpi_guess_host_protocol_id()`.
Some code paths work for ipv4/6 both
Remove some never used code.
2022-09-12 19:28:41 +02:00
Ivan Nardi 4bb81f7609 Update Github CI OS and compilers (#1733)
ubuntu-18.04 is deprecated (ubuntu-latest points to 20.04).
macos-latest points to macos-11, so it makes sense to test macos-12,
too.

About the compilers, the general idea it to test the oldest and the
newest versions easily available: switch to gcc-11 and clang-14.

See: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
2022-09-12 19:11:43 +02:00
Nardi Ivan 678dd61866 STUN: several improvements
Add detection over TCP and fix detection over IPv6.
Rename some variables since Stun dissector is no more "udp-centric".
Stun dissector should always classified the flow as `STUN` or
`STUN/Something`.
Don't touch `flow->guessed_host_protocol_id` field, which should be
always be related to "ip-classification" only.
2022-09-11 13:33:32 +02:00
Nardi Ivan f44413e039 Update Changelog file (version 4.4)
Notes shameless taken from https://github.com/ntop/nDPI/releases
2022-09-10 22:51:40 +02:00
Nardi Ivan 1b3de449a1 HTTP: correctly set the classification protocols
Classification should always be set via `ndpi_set_detected_protocol()`
to be sure to set a correct `confidence` value, too.
Having a "known" protocol stack with `NDPI_CONFIDENCE_UNKNOWN` as
confidence, is not valid.

This code in HTTP dissector likely needs some more thoughts (the
classification itself of the attached example doesn't make a lot of
sense), but the goal of this commit is only to always have a valid
`confidence` value.
2022-09-06 14:58:31 +02:00
Toni 37f918322c Serialize nDPI detection/dissection data as object. (#1725)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-06 11:38:31 +02:00
Nardi Ivan e5063f9883 Fix some warnings
Ubuntu-20.04, clang-16 (nightly build)

```
Making all in src/lib
protocols/smpp.c:70:17: warning: variable 'pdu_c' set but not used [-Wunused-but-set-variable]
      u_int16_t pdu_c = 1;
                ^
1 warning generated.
third_party/src/ahocorasick.c:173:20: warning: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Wsingle-bit-bitfield-constant-conversion]
  thiz->root->root = 1;
                   ^ ~
third_party/src/ahocorasick.c:336:15: warning: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Wsingle-bit-bitfield-constant-conversion]
        n->ff = 1;
              ^ ~
third_party/src/ahocorasick.c:716:21: warning: implicit truncation from 'int' to a one-bit wide bit-field changes value from 1 to -1 [-Wsingle-bit-bitfield-constant-conversion]
        node->final = 1;
[...]
```
2022-09-05 23:44:51 +02:00
Ivan Nardi 0a47f745cc Avoid useless host automa lookup (#1724)
The host automa is used for two tasks:
* protocol sub-classification (obviously);
* DGA evaluation: the idea is that if a domain is present in this
automa, it can't be a DGA, regardless of its format/name.

In most dissectors both checks are executed, i.e. the code is something
like:

```
ndpi_match_host_subprotocol(..., flow->host_server_name, ...);
ndpi_check_dga_name(..., flow->host_server_name,...);

```

In that common case, we can perform only one automa lookup: if we check the
sub-classification before the DGA, we can avoid the second lookup in
the DGA function itself.
2022-09-05 13:59:51 +02:00
Toni 01028ee779 Fix string copy truncation warning. (#1723)
protocols/ubntac2.c: In function ‘ndpi_search_ubntac2’:
protocols/ubntac2.c:69:4: warning: ‘strncpy’ output may be truncated copying between 0 and 31 bytes from a string of length 255 [-Wstringop-truncation]
   69 |    strncpy(flow->protos.ubntac2.version, (const char *)version, len);
      |    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-05 11:16:06 +02:00
Toni 378b0c5953 Fixed FastCGI memory issue (was not using nDPI's malloc wrapper). (#1722)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-05 10:46:02 +02:00
Nardi Ivan 95c1daa851 Fix endianess in IP-Port lookup
Port parameter in `ndpi_network_port_ptree_match()` must be in network
order.
2022-09-05 10:17:58 +02:00
Toni b7c1030804 Fix broken ndpi_timeval_to_(milli|micro)seconds (>UINT_MAX). (#1720)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-09-04 19:19:05 +02:00
Nardi Ivan b9cb391756 Add support to opportunistic TLS
A lot of protocols provide the feature to upgrade their plain text
connections to an encrypted one, via some kind of "STARTTLS" command.

Add generic code to support this extension, and allow dissection of the
entire TLS handshake.

As examples, SMTP, POP, IMAP and FTP dissectors have been updated.

Since this feature requires to process more packets per flow, add the
possibility to disable it.

Fix some log messages.

Slight improvement on TCP sequence number tracking.

As a side effect, this commit fix also a memory leak found by
oss-fuzzer
```
==108966==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 22 byte(s) in 1 object(s) allocated from:
    #0 0x55f8b367a0be in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x5480be) (BuildId: 94debacb4a6784c30420ab748c8bf3cc59621063)
    #1 0x55f8b36e1345 in ndpi_malloc_wrapper /home/ivan/svnrepos/nDPI/example/reader_util.c:321:10
    #2 0x55f8b379c7d2 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:212:25
    #3 0x55f8b379cb18 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:279:13
    #4 0x55f8b386ce46 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2153:34
    #5 0x55f8b385ebf7 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:867:5
    #6 0x55f8b39e708c in ndpi_extra_search_mail_smtp_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/mail_smtp.c:422:9
    #7 0x55f8b37e636c in ndpi_process_extra_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5884:9
    #8 0x55f8b37edc05 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6276:5
    #9 0x55f8b3701ffc in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1619:31
    #10 0x55f8b36faf14 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2189:10
    #11 0x55f8b36b6a50 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7

```
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50765
2022-09-04 17:22:19 +02:00
Luca Deri 7578d02de9 Fixes on hw calculation 2022-09-01 22:06:57 +02:00
Alfredo Cardigliano bd29fa46ba Serializer supports double for JSON and CSV only for the time being (TLV needs to be extended) 2022-09-01 14:43:52 +02:00
Alfredo Cardigliano 2372063d25 Temporarily restore ndpi_serialization_unknown = 0 2022-09-01 12:18:04 +02:00
Alfredo Cardigliano 8f410b751f Restore ndpi_serialization_type item values. Use 0 for the new ndpi_serialization_double as there is no more key left (4 bit). Use 16 for ndpi_serialization_unknown which is not used in TLV encoded data. 2022-09-01 10:37:39 +02:00
Luca Deri f684d44aa7 Removed maxmind check 2022-08-31 11:48:18 +02:00
Alfredo Cardigliano 981a72cd80 Improve AES-NI check 2022-08-30 16:37:30 +02:00
Toni f7907976c9 FastCGI: dissect PARAMS (#1715)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
2022-08-30 11:38:36 +02:00
Toni fe28d55801 Improved MGCP dissector. (#1717)
* typ0s fixed
 * dissect endpoint hostnames

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-30 11:21:58 +02:00
Nardi Ivan 79653f99e0 Internal gcrypt: fix memory leak
Fix: ef99eb67
2022-08-30 11:12:00 +02:00
Luca Deri ef99eb674e Improved AES-NI presence check on Linux 2022-08-29 17:42:28 +02:00
Nardi Ivan a329730d24 QUIC: fix heap-buffer-overflow
```
==12318==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x604000000032 at pc 0x55a59ec97959 bp 0x7fffee67fdd0 sp 0x7fffee67fdc8
READ of size 1 at 0x604000000032 thread T0
    #0 0x55a59ec97958 in may_be_0rtt /home/ivan/svnrepos/nDPI/src/lib/protocols/quic.c:1483:24
    #1 0x55a59ec9515f in ndpi_search_quic /home/ivan/svnrepos/nDPI/src/lib/protocols/quic.c:1708:13
    #2 0x55a59ec32e95 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5428:6
    #3 0x55a59ec33c5b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5464:10
    #4 0x55a59ec335fc in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5497:12
    #5 0x55a59ec44615 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6322:15
    #6 0x55a59eb8884e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #7 0x55a59eb889c7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #8 0x7fb5b3ba2082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #9 0x55a59eac742d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x47d42d) (BuildId: 712c87b21cf5c05f64174745909c693d3ba0b62e)

0x604000000032 is located 0 bytes to the right of 34-byte region [0x604000000010,0x604000000032)
allocated by thread T0 here:
    #0 0x55a59eb4bfee in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x501fee) (BuildId: 712c87b21cf5c05f64174745909c693d3ba0b62e)
    #1 0x55a59eb8899c in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:87:17

```

Found by CI tests.
See: https://github.com/ntop/nDPI/runs/7996151458?check_suite_focus=true
2022-08-25 06:44:38 +02:00
Nardi Ivan 8bfb1712d8 QUIC: add support for 0-RTT packets received before the Initial
RFC9001 4.6.1: "A client that wishes to send 0-RTT packets uses the
early_data extension in the ClientHello message of a subsequent handshake;
see Section 4.2.10 of [TLS13]. It then sends application data in 0-RTT
packets."
That means the client sends before the CH (in the Initial) and then the
0-RTT (in the same UDP datagram or not)".

However, because of packet loss or out-of-order delivery, it might
happens that a 0-RTT packet is received before the Initial (the original
one or a retransmission).
For example, Google and Facebook servers save 0-RTT packets for a small
amount of time in hopes of receiving the corresponding Initial.

Update the QUIC dissector to detect 0-RTT packets and keep looking for
the Initial.

Issue found by @utoni in #1706; the trace example has been taken from that
PR.
2022-08-24 15:38:30 +02:00
Toni 0c8bc9f055 Add FastCGI protocol detection. (#1711)
* CQL: fixed byte order conversion (BigEndian not LittleEndian)
 * CQL: increased required successful dissected packets to prevent false-positives

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 13:22:46 +02:00
Toni e135c1c5e3 Merge pull request #1712 from IvanNardi/oss-fuzzer
HTTP, SoftEther, Florensia: fix some memory corruptions
2022-08-24 11:24:25 +02:00
Toni 30730e95e5 Add Kismet protocol detection. (#1710)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 10:50:11 +02:00
Toni Uhlig 93a80a1c4b Fixed serializing bug caused by an unitialized serializer.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 10:49:27 +02:00
lns 93d65ed650 Support serialization of double-precision floating-point numbers. Fixes #1702.
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-08-24 10:49:27 +02:00
Nardi Ivan ac0d7ccb7e HTTP, SoftEther, Florensia: fix some memory corruptions
```
==12308==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600068f4b4 at pc 0x5603d2b56098 bp 0x7ffc967bd3b0 sp 0x7ffc967bd3a8
READ of size 2 at 0x60600068f4b4 thread T0
    #0 0x5603d2b56097 in ndpi_search_florensia /home/ivan/svnrepos/nDPI/src/lib/protocols/florensia.c:93:40
    #1 0x5603d2a2e835 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6
    #2 0x5603d2a2f247 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5462:12
    #3 0x5603d2a2eee7 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5481:12
    #4 0x5603d2a3ffb5 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15
    #5 0x5603d2955f4c in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1612:31
    #6 0x5603d294ee64 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2182:10
    #7 0x5603d290aa50 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #8 0x5603d290ab8b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17
    #9 0x7f972cdc9082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #10 0x5603d28494fd in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x4be4fd) (BuildId: e81373a75c70e248ba9ae985fcb4134e7113121a)
```
```
==14164==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60d0000000c1 at pc 0x561608bada2c bp 0x7ffc40d38ed0 sp 0x7ffc40d38690
READ of size 10 at 0x60d0000000c1 thread T0
    #0 0x561608bada2b in __interceptor_strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4e9a2b) (BuildId: 2e4fd67f1aeb93c491a04ae424331669c993ee9b)
    #1 0x561608e66f2a in dissect_softether_host_fqdn /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:226:2
    #2 0x561608e64e88 in ndpi_search_softether /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:316:8
    #3 0x561608ca7475 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6
    #4 0x561608ca823b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5450:10
    #5 0x561608ca7bdc in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5483:12
    #6 0x561608cb8bf5 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15
    #7 0x561608bff84e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #8 0x561608bff9c7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #9 0x7fc50662d082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #10 0x561608b3e42d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x47a42d) (BuildId: 2e4fd67f1aeb93c491a04ae424331669c993ee9b)
```

Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50105
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50110
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50268
2022-08-22 17:53:52 +02:00
Luca Deri a53f476585 Modified definition of hll_add 2022-08-17 16:25:10 +02:00
Luca Deri 82c83ef75e Updated RPM package 2022-08-15 16:29:08 +02:00
Luca Deri 4a843709fa Packages changes 2022-08-15 11:32:58 +02:00
alcir 780cbf5f76 Update Makefile.in (#1708)
Use `@libdir@` instead of *hardcoded* path, and also `@includedir@`
2022-08-15 08:20:17 +00:00
Luca Deri 743d046f6f Added nTap reference 2022-08-12 10:38:51 +02:00
Toni 5915bc8828 Improved Discord pattern. (#1707)
Signed-off-by: lns <matzeton@googlemail.com>

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-10 21:00:11 +02:00
Ivan Nardi 540848c254 HTTP: fix classification in some corner cases (#1704)
Initializing `low->detected_protocol_stack[1]` but not
`low->detected_protocol_stack[0]` lead to *very* strange errors.
Oss-fuzzer, exploiting this bug, has been able to crash the application
or to leak some memory

```
==19775==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x5624c0dd1802 bp 0x5624c197e848 sp 0x7fff4b4db690 T0)
==19775==The signal is caused by a READ memory access.
==19775==Hint: this fault was caused by a dereference of a high value address (see register values below).  Disassemble the provided pc to learn which register was used.
    #0 0x5624c0dd1802 in __asan::Allocator::Deallocate(void*, unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x522802) (BuildId: 9a1ed9efcaf47e345767c86520372d28e31ca1aa)
    #1 0x5624c0e53ea6 in __interceptor_free (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x5a4ea6) (BuildId: 9a1ed9efcaf47e345767c86520372d28e31ca1aa)
    #2 0x5624c0ebb474 in free_wrapper /home/ivan/svnrepos/nDPI/example/reader_util.c:330:3
    #3 0x5624c0f75561 in ndpi_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:240:7
    #4 0x5624c0faa921 in ndpi_free_flow_data /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4915:2
    #5 0x5624c0f755ef in ndpi_free_flow /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8076:5
    #6 0x5624c0f755c4 in ndpi_flow_free /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:253:5
    #7 0x5624c0eba981 in ndpi_free_flow_info_half /home/ivan/svnrepos/nDPI/example/reader_util.c:305:25
    #8 0x5624c0ecb4bc in process_ndpi_collected_info /home/ivan/svnrepos/nDPI/example/reader_util.c:1317:5
    #9 0x5624c0edca36 in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1639:2
    #10 0x5624c0ed5172 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2182:10
    #11 0x5624c0e90a80 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #12 0x5624c0e90bbb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17
    #13 0x7f580731f082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #14 0x5624c0dcf52d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x52052d) (BuildId: 9a1ed9efcaf47e345767c86520372d28e31ca1aa)
```

```
==17492==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 5 byte(s) in 1 object(s) allocated from:
    #0 0x55b799b0f01e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x56101e) (BuildId: d2adbfb29a6eda6dc59fdfb8930d7e6496ac7b8b)
    #1 0x55b799bb3bf4 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:212:46
    #2 0x55b799bb3f28 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:279:13
    #3 0x55b799c8b9bf in processCertificateElements /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:683:39
    #4 0x55b799c81e89 in processCertificate /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:835:7
    #5 0x55b799c8d500 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:889:13
    #6 0x55b799ca80f1 in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1034:2
    #7 0x55b799ca3be2 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2475:5
    #8 0x55b799bf9345 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6
    #9 0x55b799bf9d57 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5462:12
    #10 0x55b799bf99f7 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5481:12
    #11 0x55b799c0b838 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15
    #12 0x55b799b4b87e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
```

```
==17511==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 1 byte(s) in 1 object(s) allocated from:
    #0 0x55aed645e01e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x56101e) (BuildId: d2adbfb29a6eda6dc59fdfb8930d7e6496ac7b8b)
    #1 0x55aed6502bf4 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:212:46
    #2 0x55aed6502f28 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:279:13
    #3 0x55aed65ea0c6 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2153:34
    #4 0x55aed65dbe77 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:867:5
    #5 0x55aed65f70f1 in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1034:2
    #6 0x55aed65f2be2 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2475:5
    #7 0x55aed6548345 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6
    #8 0x55aed6548d57 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5462:12
    #9 0x55aed65489f7 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5481:12
    #10 0x55aed655a838 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15
    #11 0x55aed649a87e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
```

See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49844
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49842
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49906
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49915
2022-08-09 18:41:10 +02:00
Ivan Nardi 59a9bdeb55 SoftEtehr: fix a heap-buffer overflow (#1705)
```
==26717==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60c0000000bf at pc 0x55e503f8959b bp 0x7fff7f6876b0 sp 0x7fff7f686e58
READ of size 1 at 0x60c0000000bf thread T0
    #0 0x55e503f8959a in strncmp (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4f459a) (BuildId: 44038d7ca9391c877fae9b2735828da9905a6daa)
    #1 0x55e5042ee98f in dissect_softether_host_fqdn /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:236:8
    #2 0x55e5042ec41b in ndpi_search_softether /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:318:8
    #3 0x55e5040e0345 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5414:6
    #4 0x55e5040e110b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5450:10
    #5 0x55e5040e0aac in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5483:12
    #6 0x55e5040f2838 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6308:15
    #7 0x55e50403287e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #8 0x55e5040329f7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #9 0x7fd09fdbb082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #10 0x55e503f7145d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4dc45d) (BuildId: 44038d7ca9391c877fae9b2735828da9905a6daa)
```

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49901
2022-08-09 18:09:16 +02:00
Toni 2e25c36396 Add TiVoConnect dissector. Fixes #1697. (#1699)
* added static assert if supported, to complain if the flow struct changes

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-08 19:04:20 +02:00
Ivan Nardi 523360001e HTTP, Bittorrent, Kerberos: fix stopping of extra dissection (#1700)
The return value of the extra-dissection callback indicates if the extra
dissection needs to be called again.

In the HTTP cose, this setting to NULL of the callabck is wrong since we
stop extra dissection only if we have a hostname *and* a return code.
2022-08-06 20:06:56 +02:00
Ivan Nardi 120e01566a Bittorrent: don't use absolute time reference (#1701) 2022-08-06 20:01:06 +02:00
Ivan Nardi 1d4e27c473 Further simplification of ndpi_process_extra_packet() (#1698)
See 95e16872.

After c0732eda, we can safely remove the protocol list from
`ndpi_process_extra_packet()`.

The field `flow->check_extra_packets` is redundant; remove it.
2022-08-05 15:15:45 +02:00
Ivan Nardi c0732eda45 HTTP: improve sub-classification (#1696)
Content-matched sub-protocols (DASH, IPP, MPEGDASH...) shouldn't
ovewrite the previous master protocol (if any; usually HTTP).

Furthermore. the HTTP dissector shouldn't update the classification
(in the extra-dissection code path) if a content-matched sub-protocols
has already been found.

This commit should address the first part of the changes described in #1687.
2022-08-05 14:14:38 +02:00
Ivan Nardi 346e274a1b SoftEther: fix two heap-buffer-overflows (#1695)
The first change is a proper (hopefully) fix for the bug reported in
8b6a00f8.
The second one is related to:
```
==15096==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60f0000d7a00 at pc 0x55a2c593bd0b bp 0x7ffc92021cd0 sp 0x7ffc92021478
READ of size 3 at 0x60f0000d7a00 thread T0
    #0 0x55a2c593bd0a in strncmp (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x56fd0a) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #1 0x55a2c5d1d9f9 in dissect_softether_host_fqdn /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:249:9
    #2 0x55a2c5d1b55b in ndpi_search_softether /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:348:9
    #3 0x55a2c5b0e9c5 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5407:6
    #4 0x55a2c5b0f78b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5443:10
    #5 0x55a2c5b0f12c in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5476:12
    #6 0x55a2c5b20f39 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6303:15
    #7 0x55a2c5a3014c in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1600:31
    #8 0x55a2c5a29062 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2170:10
    #9 0x55a2c59e51a2 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #10 0x55a2c590acb2 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x53ecb2) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #11 0x55a2c590a3c5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x53e3c5) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #12 0x55a2c590c0f6 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::vector<fuzzer::SizedFile, std::allocator<fuzzer::SizedFile>>&) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x5400f6) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #13 0x55a2c590c663 in fuzzer::Fuzzer::Loop(std::vector<fuzzer::SizedFile, std::allocator<fuzzer::SizedFile>>&) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x540663) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #14 0x55a2c58faff2 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x52eff2) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #15 0x55a2c5923c82 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x557c82) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)
    #16 0x7f504ab98082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #17 0x55a2c58efb1d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x523b1d) (BuildId: ee8631c0950a8cded5ba60c17f09709bbebbe5d8)

0x60f0000d7a00 is located 0 bytes to the right of 176-byte region [0x60f0000d7950,0x60f0000d7a00)
```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49736
2022-08-05 13:43:09 +02:00
Luca Deri 10f283b467 Fix for systems with no realpath (MacOS) 2022-08-05 11:17:59 +02:00
Luca Deri 9fb11c909e Win fixes 2022-08-05 11:17:28 +02:00
Luca Deri bcf7f1713c Win and indentation fixes 2022-08-05 11:17:09 +02:00
Toni ac24b35b1f Add Discord dissector. (#1694)
* fixed RiotGames false positive

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-03 12:03:36 +02:00
Luca Deri c3c73e7bff Cosmetic changes 2022-08-03 11:31:32 +02:00
Luca Deri 63fe4b4306 Compilation fix (not all compilers like restrict) (part 2) 2022-08-03 11:12:15 +02:00
Luca Deri 154d6a5069 Compilation fix (not all compilers like restrict) 2022-08-03 10:47:31 +02:00
Toni b3e722e5a8 Improved nDPI JSON serialization. (#1689)
* fixed autoconf CFLAGS/LDFLAGS MSAN issue which could lead to build errors
 * introduced portable version of gmtime_r aka ndpi_gmtime_r
 * do as most as possible of the serialization work in ndpi_utils.c
 * use flow2json in ndpiReader

Signed-off-by: lns <matzeton@googlemail.com>
2022-08-02 17:54:44 +02:00
Toni 26aafd931c Add Activision dissector. (#1693)
Signed-off-by: lns <matzeton@googlemail.com>
2022-08-02 16:15:07 +02:00
Ivan Nardi 013212c67c TLS: avoid useless extra dissection (#1690)
If we have seen Application Data blocks in both directions, it means we
are after the handshake. At that point, extra dissection is useless.

Useful feature with TLS mid-sessions, i.e. sessions without initial
packets.
2022-07-31 16:57:17 +02:00
Ivan Nardi 2a596c79e6 HTTP: fix classification (#1692)
If we have a valid HTTP sessions, we should ignore
`flow->guessed_protocol_id` field (i.e. classification "by-port")
altogheter.

The attached trace was classified as "SIP/HTTP" only because the *client*
port was 5060...
As a general rule, having a classification such as "XXXX/HTTP" is
*extremely* suspicious.
2022-07-30 22:57:20 +02:00
Ivan Nardi 8b6a00f84b SoftEther: fix heap-buffer-overflow (#1691)
```
==160665==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x604000000038 at pc 0x55f7250d9a5c bp 0x7fff02c82b90 sp 0x7fff02c82350
READ of size 4 at 0x604000000038 thread T0
    #0 0x55f7250d9a5b in __interceptor_strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x548a5b) (BuildId: 12fd06e7a171f035d3a25f016184ac357088379c)
    #1 0x55f7253e6495 in dissect_softether_ip_port /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:303:3
    #2 0x55f7253e5703 in ndpi_search_softether /home/ivan/svnrepos/nDPI/src/lib/protocols/softether.c:330:9
    #3 0x55f7251d87c5 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5397:6
    #4 0x55f7251d958b in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5433:10
    #5 0x55f7251d8f2c in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5466:12
    #6 0x55f7251ead39 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6293:15
    #7 0x55f72512b87e in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #8 0x55f72512b9f7 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #9 0x7fdef837b082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #10 0x55f72506a45d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x4d945d) (BuildId: 12fd06e7a171f035d3a25f016184ac357088379c)
```

Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49638
2022-07-30 22:55:46 +02:00
Ivan Nardi d54d5083b3 SMTPS, POPS, IMAPS: fix classification and extra dissection (#1685)
The big change in TLS code is to allow "master" protocols other than
TLS/DTLS, like SMTPS, POPS and IMAPS.
This change will allow, in a future, a proper and complete TLS dissection
for all these protocols with "STARTTLS"-like messages.
2022-07-30 12:05:43 +02:00
Toni 8402bd68ad Tiny Softether dissector improvement. (#1688)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-30 10:03:42 +02:00
Toni ed4f106a0d Add Softether dissector. (#1679)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-29 19:29:54 +02:00
Ivan Nardi 405a52ed65 Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1683)
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.

Add an option to `ndpiReader` to dump them; enabled by default in the
unit tests.
This new option enables/disables dumping of "num dissectors calls"
values, too (see b4cb14ec).
2022-07-29 15:25:00 +02:00
Ivan Nardi df2e11ef51 Revert "Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1677)" (#1682)
This reverts commit bb83899985.
2022-07-29 12:08:40 +02:00
Ivan Nardi bb83899985 Patricia tree, Ahocarasick automa, LRU cache: add statistics (#1677)
Add (basic) internal stats to the main data structures used by the
library; they might be usefull to check how effective these structures
are.

Add an option to `ndpiReader` to dump them; disabled by default to avoid
too much fuss with the unit tests.
2022-07-29 12:07:41 +02:00
Ivan Nardi 95e16872fa First step in simplify ndpi_process_extra_packet() (#1680)
Move the prottocol specific logic into the proper dissector code, where
it belongs.

Next step: remove that list of protocols. Long goal: remove this
function altogether...
2022-07-29 12:07:14 +02:00
Ivan Nardi 172e698bb8 TINC: avoid processing SYN packets (#1676)
Since e6b332aa, we have proper support for detecting client/server
direction. So Tinc dissector is now able to properly initialize the
cache entry only when needed and not anymore at the SYN time; initializing
that entry for **every** SYN packets was a complete waste of resources.

Since 4896dabb, the various `struct ndpi_call_function_struct`
structures are not more separate objects and therefore comparing them
using only their pointers is bogus: this bug was triggered by this
change because `ndpi_str->callback_buffer_size_tcp_no_payload` is now 0.
2022-07-28 12:39:18 +02:00
Ivan Nardi d8d525fff2 Update the protocol bitmask for some protocols (#1675)
Tcp retransmissions should be ignored.

Remove some unused protocol bitmasks.

Update script to download Whatsapp IP list.
2022-07-27 11:46:45 +02:00
Toni ab3a678ad4 Add AVAST dissector. (#1674)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-25 18:07:44 +02:00
Ivan Nardi b190dab6bc Improve handling of HTTP-Proxy and HTTP-Connect (#1673)
Treat HTTP-Proxy and HTTP-Connect flows like the HTTP ones:
print/serialize all the attributes and allow parsing of replies.

The line about "1kxun" has been removed to avoid regressions in 1KXUN
classification in `tests/pcap/1kxun.pcap`. I haven't fully understod
what was happening but the comment at the beginning of `static
ndpi_category_match category_match[]` says that we can't have overlaps
between `host_match` and `category_match` lists and that is no longer true
since 938e89ca.
Bottom line: removing this line seems the right thing to do, anyway.
2022-07-25 12:57:33 +02:00
Ivan Nardi 86a3e4c8c3 Add support for Memory sanitizer (#1652) 2022-07-24 18:12:25 +02:00
Ivan Nardi e6b332aa4a Add support for flow client/server information (#1671)
In a lot of places in ndPI we use *packet* source/dest info
(address/port/direction) when we are interested in *flow* client/server
info, instead.

Add basic logic to autodetect this kind of information.

nDPI doesn't perform any "flow management" itself but this task is
delegated to the external application. It is then likely that the
application might provide more reliable hints about flow
client/server direction and about the TCP handshake presence: in that case,
these information might be (optionally) passed to the library, disabling
the internal "autodetect" logic.

These new fields have been used in some LRU caches and in the "guessing"
algorithm.
It is quite likely that some other code needs to be updated.
2022-07-24 17:46:24 +02:00
Luca Deri 523f22b942 Added vides/presentations section 2022-07-23 17:05:31 +02:00
Toni a25b2a7e37 Added AliCloud server access dissector. (#1672)
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-23 11:21:49 +02:00
Zied Aouini 40d7354dd7 Windows error C2466 fix (array size init with 0). 2022-07-22 21:58:25 +02:00
Zied Aouini 9d040f6317 Windows compilation fix. 2022-07-22 20:14:30 +02:00
Ivan Nardi 52005e88ed TLS: improve reassembler (#1669)
* TLS: cosmetic changes

* TLS: improve reassembler

We might need to contemporary re-order messages from both directions:
use one buffer per direction.
2022-07-22 12:19:21 +02:00
Ivan Nardi ce6f11840d Bittorrent: fix keys in cache code (#1670) 2022-07-22 12:10:43 +02:00
Luca Deri e85d6d7db7 RPM fix 2022-07-20 21:26:07 +02:00
Nardi Ivan 00eb2393f4 STUN: improve detection of Hangout/Duo and FB voip calls 2022-07-20 20:36:31 +02:00
Nardi Ivan d66aa49787 DTLS: fix exclusion of DTLS protocol
Add an helper to exclude a generic protocol
2022-07-20 19:16:18 +02:00
Luca Deri a9204eb936 Rocky fix 2022-07-20 17:23:17 +02:00
Nardi Ivan e1edb08f06 SKYPE: fix detection over UDP
Commit ba6a48c9 is completely bogus: we can't set extra dissection
without having set a proper classification.

The idea behind that commit seems to be that we need to look for 2
(consecutives?) packets with the same crc/pattern: try to implement this
logic in a saner way.
2022-07-20 16:13:55 +02:00
Nardi Ivan 5702c6fb08 SKYPE: remove detection over TCP
Skype detection over TCP has been completely disable since 659f75138 (3
years ago!).
Since that logic was too weak anyway, remove it.
2022-07-20 16:13:55 +02:00
Ivan Nardi c72660d7d3 reader_util: stop processing a flow (#1666)
We should stop processing a flow if all protocols have been excluded or
if we have already processed too many packets.
2022-07-20 14:48:09 +02:00
Ivan Nardi b472a49271 BITTORRENT: fix confidence (#1664)
Remove two unused parameters.
2022-07-20 13:59:51 +02:00
Ivan Nardi ac1113b9ac ahocorasick: fix char/uchar bug (#1597)
Porting of https://github.com/vel21ripn/nDPI/commit/06e2967d0c26ab214683a2f5565f4012fd523537
Close #1506
2022-07-20 10:59:17 +02:00
Toni be1ac1b430 Fix unitialized use of W[16]. (#1662)
Signed-off-by: lns <matzeton@googlemail.com>
2022-07-15 14:15:37 +02:00
Luca b80218a0e4 Added null pointer checks 2022-07-14 18:04:51 +02:00
Luca 842e0a3648 Restored -O2 in default builds 2022-07-14 17:24:18 +02:00
Toni 7c19de4904 Do not interfere with CFLAGS/LDFLAGS env anymore. (#1659)
* CI fixes
 * some build systems do not like that (e.g. OpenWrt)
 * fixed some rrdtool related build warnings/errors

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-13 19:44:18 +02:00
Toni ae2bedce3a Improved Jabber/XMPP detection. (#1661)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-13 17:55:33 +02:00
Ivan Nardi 407155755d ASN1/BER: fix signed integer overflow (#1660)
```
protocols/snmp_proto.c:77:23: runtime error: signed integer overflow: 6 + 2147483647 cannot be represented in type 'int'
 #0 0x52f69e in ndpi_search_snmp ndpi/src/lib/protocols/snmp_proto.c:77:23
 #1 0x4c5347 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:5211:4
 #2 0x4c5591 in ndpi_check_flow_func ndpi/src/lib/ndpi_main.c:0
 #3 0x4c8903 in ndpi_detection_process_packet ndpi/src/lib/ndpi_main.c:6145:15
 #4 0x4b3712 in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_process_packet.c:29:5
[...]
```
Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=49057
2022-07-12 18:39:05 +02:00
Luca 9c235796af Introduced risk accountability 2022-07-12 10:36:40 +02:00
Luca cba2d12051 Cosmetic change 2022-07-12 10:36:32 +02:00
Ivan Nardi b4cb14ec19 Keep track of how many dissectors calls we made for each flow (#1657) 2022-07-11 09:47:47 +02:00
Ivan Nardi df599e5eff HTTP: improve detection of WindowsUpdate (#1658)
WindowsUpdate is also transported over HTTP, using a numeric IP as
hostname (some kinds of CDN?)
2022-07-10 17:08:37 +02:00
Ivan Nardi 1fcd03a6b6 Remove unsafe access to flow->protos union (#1656)
We can access `flow->protos` only if we already have set a valid
classification.

It is quite likely that this code is never trigger, anyway.
2022-07-10 17:08:22 +02:00
Ivan Nardi 2edfaeba4a SNMP: fix detection (#1655)
We can write to `flow->protos` only when we are sure about SNMP
classification.

Use the generic wrapper to decode ASN1 BER integer
2022-07-10 17:07:52 +02:00
Ivan Nardi 997dce0f04 SIP: improve detection (#1654) 2022-07-09 05:45:42 +02:00
Ivan Nardi dfe6557e18 TFTP: fix memory access (#1653) 2022-07-08 18:39:05 +02:00
Ivan Nardi 510517126a LDAP: rewrite dissection (#1649) 2022-07-08 12:50:46 +02:00
Toni 9b95876973 Enhances gprof usage. (#1651)
* gprof results were incorrectly displayed

Signed-off-by: lns <matzeton@googlemail.com>
2022-07-08 12:05:55 +02:00
Ivan Nardi f8076e3a58 SMB: add (partial) support for messages split into multiple TCP segments (#1644) 2022-07-07 19:24:31 +02:00
Ivan Nardi ff4e010501 Avoid spurious calls to extra dissection (#1648)
If the extra callabck is not set, calling the extra dissection is only a
waste of resources...
2022-07-07 17:49:35 +02:00
Ivan Nardi d254ae54f3 SMTP: add support for X-ANONYMOUSTLS comamnd (#1650) 2022-07-07 16:46:18 +02:00
Ivan Nardi feaa1df1ed Kerberos: add support for Krb-Error messages (#1647) 2022-07-07 16:45:49 +02:00
Ivan Nardi 056e742304 Spotify: remove some useless ip ranges (#1646)
These AS numbers are no more related to Spotify (or, if they are, they
don't have any prefixes anyway).
Even if we find some valid Spotify AS, we should handle them via the
generic "autogenerated logic" used for every AS, and not in the
dissector code.
2022-07-07 15:39:19 +02:00
Nardi Ivan 2636c07571 MONGODB: avoid false positives 2022-07-07 15:36:05 +02:00
Nardi Ivan a31e79fc3c TLS: ignore invalid Content Type values 2022-07-07 15:36:05 +02:00
Toni 15042870f9 Added Threema Messenger. (#1643)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 19:30:10 +02:00
Toni Uhlig 105f661e46 Added RiotGames ASN update.
* updated asn lists

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 14:37:26 +02:00
Toni Uhlig a1c3d05a74 Added another RiotGames signature.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 14:37:26 +02:00
Luca Deri 814f9cd21e Moved to 4.5 2022-07-06 12:43:56 +02:00
3509 changed files with 127873 additions and 49324 deletions
+12
View File
@@ -0,0 +1,12 @@
Please sign (check) the below before submitting the Pull Request:
- [ ] I have signed the ntop Contributor License Agreement at https://github.com/ntop/legal/blob/main/individual-contributor-licence-agreement.md
- [ ] I have read the contributing guide lines at https://github.com/ntop/nDPI/blob/dev/CONTRIBUTING.md
- [ ] I have updated the documentation (in doc/) to reflect the changes made (if applicable)
Link to the related [issue](https://github.com/ntop/nDPI/issues):
Describe changes:
+43
View File
@@ -0,0 +1,43 @@
name: MSBuild
on:
push:
branches:
- dev
pull_request:
branches:
- dev
types: [opened, synchronize, reopened]
release:
types: [created]
jobs:
test:
name: MSBuild
runs-on: windows-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Add msbuild to PATH
uses: microsoft/setup-msbuild@v1.1
- name: Download NuGet packages
run: msbuild windows/nDPI.sln -t:restore -p:RestorePackagesConfig=true
- name: Build libnDPI only (debug)
run: msbuild windows/nDPI.sln -t:rebuild -property:Configuration=Debug
- name: Download Npcap
run: Invoke-WebRequest -Uri 'https://npcap.com/dist/npcap-1.71.exe' -OutFile 'C:\npcap-1.71.exe'
- name: Install Npcap
run: C:\npcap-1.71.exe /S /winpcap_mode=yes
- name: Build ndpiReader (debug)
run: msbuild windows/nDPI.sln -t:rebuild -property:Configuration=Debug-ndpiReader
- name: Upload artifacts
uses: actions/upload-artifact@v3
with:
name: build-artifacts
path: windows/bin
retention-days: 14
+100 -158
View File
@@ -10,47 +10,17 @@ on:
release:
types: [created]
jobs:
coverage:
name: Coverage (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
sudo apt-get install rrdtool librrd-dev
sudo apt-get install libpcre3-dev libmaxminddb-dev lcov
sudo apt-get install wdiff colordiff
- name: Configure
run: env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-code-coverage --with-pcre --with-maxminddb --enable-tls-sigs
- name: Build
run: make all
- name: Test
run: |
make check VERBOSE=1
- name: Generate Coverage Report
run: |
mkdir -vp coverage_report/ndpi_coverage_report
lcov --directory . --capture --output-file lcov.info
genhtml -o coverage_report/ndpi_coverage_report lcov.info
- uses: actions/upload-artifact@v2
with:
name: ndpi-coverage-report
path: coverage_report
retention-days: 7
python-bindings:
name: Python Bindings (ubuntu-latest)
runs-on: ubuntu-latest
env:
CFLAGS: -Werror
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Python 3.9
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: "3.9"
- name: Install Ubuntu Prerequisites
@@ -59,7 +29,7 @@ jobs:
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
- name: Build nDPI library
run: |
env CC=gcc CFLAGS='-Werror' ./autogen.sh
./autogen.sh
make
sudo make install
- name: Generate Python bindings
@@ -73,82 +43,17 @@ jobs:
run: |
cd python
python tests.py
documentation:
name: Documentation (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: "3.9"
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison doxygen
- name: Configure nDPI library
run: |
./autogen.sh --with-only-libndpi --enable-option-checking=fatal
- name: Generate Documentation
run: |
pip install --upgrade pip
pip install -r doc/requirements.txt
make doc
mkdir -vp doc/_build/ndpi-documentation-upload/ndpi-documentation
mv -v doc/_build/html doc/_build/ndpi-documentation-upload/ndpi-documentation/html
- uses: actions/upload-artifact@v2
with:
name: ndpi-documentation
path: doc/_build/ndpi-documentation-upload
retention-days: 7
performance:
name: Performance (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison libpcap-dev
sudo apt-get install golang-1.16-go libgoogle-perftools-dev graphviz
env GO111MODULE=on go install github.com/google/pprof@latest
sudo ln -s ${HOME}/go/bin/pprof /usr/bin/pprof
pprof -h
- name: Configure nDPI library
run: |
./autogen.sh --enable-gprof --enable-option-checking=fatal --with-pcre --with-maxminddb --enable-tls-sigs
- name: Build nDPI library
run: |
make
- name: Performance Profiling
run: |
./tests/do.sh
mkdir ndpi-performance-upload
mv -v tests/result/cpu_profile.png ndpi-performance-upload/cpu_profile.png
mv -v tests/result/heap_profile.png ndpi-performance-upload/heap_profile.png
- uses: actions/upload-artifact@v2
with:
name: ndpi-performance
path: ndpi-performance-upload
retention-days: 7
test-scripts:
name: Test Utils (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install python3-netaddr git
sudo apt-get install python3-netaddr git whois libxml2-utils
- name: Run Scripts
run: |
echo 'Running ./utils/bitcoinnodes.sh'
@@ -162,109 +67,134 @@ jobs:
git diff-index --quiet HEAD -- || true
test:
name: ${{ matrix.os }} ${{ matrix.arch }} ${{ matrix.gcrypt }} ${{ matrix.compiler }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} ${{ matrix.msan }}
name: ${{ matrix.os }} ${{ matrix.arch }} ${{ matrix.gcrypt }} ${{ matrix.compiler }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} ${{ matrix.msan }} ${{ matrix.nBPF }} ${{matrix.lto_gold_linker}}
runs-on: ${{ matrix.os }}
env:
CC: ${{ matrix.compiler }}
CFLAGS: -Werror
strategy:
fail-fast: true
matrix:
os: ["ubuntu-latest", "ubuntu-18.04", "macOS-latest", "macos-11", "windows-latest"]
# ubuntu-latest == ubuntu-22.04
# macOS-latest == macos-12
os: ["ubuntu-latest", "ubuntu-20.04", "macOS-latest", "macos-11", "windows-latest"]
arch: ["x86_64"]
gcrypt: ["--with-local-libgcrypt", ""]
compiler: ["default-cc"]
compiler: ["cc"]
ar: ["ar"]
ranlib: ["ranlib"]
pcre: [""]
maxminddb: [""]
msan: [""]
nBPF: [""]
lto_gold_linker: [""]
include:
- compiler: "gcc-7"
os: ubuntu-latest
- compiler: "gcc-7" # "Oldest" gcc easily available
os: ubuntu-20.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "gcc-10"
os: ubuntu-latest
nBPF: ""
- compiler: "gcc-12" # "Newest" gcc easily available
os: ubuntu-22.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "clang-7"
os: ubuntu-latest
nBPF: ""
lto_gold_linker: "--with-lto-and-gold-linker"
- compiler: "clang-7" # "Oldest" clang easily available
os: ubuntu-20.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "clang-12"
os: ubuntu-latest
nBPF: ""
- compiler: "clang-14" # "Newest" clang easily available
ar: "llvm-ar-14"
ranlib: "llvm-ranlib-14"
os: ubuntu-22.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "default-cc"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "default-cc"
nBPF: ""
lto_gold_linker: "--with-lto-and-gold-linker"
- compiler: "cc"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-thread-sanitizer"
- compiler: "default-cc"
os: ubuntu-18.04
nBPF: ""
- compiler: "cc"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "default-cc"
nBPF: "nBPF"
- compiler: "clang-14"
os: ubuntu-22.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-memory-sanitizer"
nBPF: ""
- compiler: "cc"
os: macOS-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "" # Disable sanitizer on macos
- compiler: "default-cc"
os: macos-11
nBPF: ""
- compiler: "cc"
os: macos-12
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "" # Disable sanitizer on macos
- compiler: "default-cc"
nBPF: ""
- compiler: "cc"
os: ubuntu-latest
arch: "arm64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "" # Disable sanitizer on arm64
- compiler: "default-cc"
nBPF: ""
- compiler: "cc"
os: ubuntu-latest
arch: "armhf"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "default-cc"
nBPF: ""
- compiler: "cc"
os: ubuntu-latest
arch: "s390x"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: ""
nBPF: ""
steps:
- name: Setup multiarch/qemu-user-static
if: startsWith(matrix.os, 'ubuntu') && !startsWith(matrix.arch, 'x86_64')
run: |
docker run --rm --privileged multiarch/qemu-user-static:register --reset
- uses: actions/checkout@v2
- uses: actions/checkout@v3
with:
fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis
- name: Install Ubuntu Prerequisites
@@ -287,22 +217,26 @@ jobs:
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.maxminddb, '--with-maxminddb')
run: |
sudo apt-get install libmaxminddb-dev
- name: Setup Ubuntu specified compiler (gcc)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'gcc')
- name: Install Ubuntu Prerequisites (nBPF)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.nBPF, 'nBPF')
run: |
git clone https://github.com/ntop/PF_RING.git ../PF_RING
cd ../PF_RING/userland/nbpf
./configure
make
cd -
- name: Setup Ubuntu specified compiler
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && ! startsWith(matrix.compiler, 'cc')
run: |
sudo apt-get install ${{ matrix.compiler }}
- name: Setup Ubuntu specified compiler (clang)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'clang')
run: |
sudo apt-get install ${{ matrix.compiler }}
- name: Install Windows prerequisites
- name: Install Windows msys2 prerequisites
if: startsWith(matrix.os, 'windows')
uses: msys2/setup-msys2@v2
with:
msystem: MINGW64
update: true
install: git mingw-w64-x86_64-toolchain automake1.16 automake-wrapper autoconf libtool make mingw-w64-x86_64-json-c mingw-w64-x86_64-crt-git mingw-w64-x86_64-pcre mingw-w64-x86_64-libpcap
- name: Setup Windows prerequisites (libgcrypt)
- name: Setup Windows msys2 prerequisites (libgcrypt)
if: startsWith(matrix.os, 'windows') && startsWith(matrix.gcrypt, '--with-local-libgcrypt')
run: |
msys2 -c 'git clone --branch libgpg-error-1.42 https://github.com/gpg/libgpg-error'
@@ -321,6 +255,9 @@ jobs:
msys2 -c 'make install'
msys2 -c 'cd ..'
msys2 -c 'rm -rf libgcrypt/'
env:
CC:
CFLAGS:
- name: Installing MacOS prerequisites
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64')
run: |
@@ -338,47 +275,50 @@ jobs:
run: |
brew install libmaxminddb
- name: Configure nDPI on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'default-cc')
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
run: |
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs
AR=${{ matrix.ar }} RANLIB=${{ matrix.ranlib }} ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs ${{matrix.lto_gold_linker}}
- name: Configure nDPI on MacOS
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'default-cc')
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'cc')
run: |
env CC=clang CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs
- name: Configure nDPI on Windows
if: startsWith(matrix.os, 'windows') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'default-cc')
./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs
- name: Configure nDPI on Windows msys2
if: startsWith(matrix.os, 'windows') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'cc')
run: |
msys2 -c 'env CFLAGS=\"-Werror\" ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --disable-npcap ${{ matrix.gcrypt }}'
- name: Build nDPI on Windows
if: startsWith(matrix.os, 'windows') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'default-cc')
msys2 -c './autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --disable-npcap ${{ matrix.gcrypt }}'
- name: Build nDPI on Windows msys2
if: startsWith(matrix.os, 'windows') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'cc')
run: |
msys2 -c 'make all'
msys2 -c 'ldd ./example/ndpiReader.exe'
- name: Configure nDPI with specified GCC version on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'gcc')
run: |
env CC=${{ matrix.compiler }} CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs
- name: Configure nDPI with specified CLANG on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'clang')
run: |
env CC=${{ matrix.compiler }} CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs
- name: Build nDPI
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
make all
make -C example ndpiSimpleIntegration
make -C rrdtool
- name: Print nDPI long help
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
./example/ndpiReader -H
- name: Install nDPI
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
make install DESTDIR=$(realpath _install)
ls -alhHR _install
- name: Test nDPI [SYMBOLS]
if: (startsWith(matrix.os, 'ubuntu') || startsWith(matrix.os, 'mac')) && startsWith(matrix.arch, 'x86_64')
run: |
./utils/check_symbols.sh || { FAILED=$?; echo "::error file=${NDPI_LIB}::Unwanted libc symbols found: ${FAILED}. Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of malloc/calloc/realloc/free."; false; }
env:
NDPI_LIB: src/lib/libndpi.a
- name: Test nDPI [DIFF]
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
./tests/do.sh
- name: Test nDPI [UNIT]
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
#Some issues with masan + json-c. Disable the test as workaround
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows') && !startsWith(matrix.msan, '--with-memory-sanitizer')
run: |
./tests/do-unit.sh
- name: Test nDPI [DGA]
@@ -403,11 +343,13 @@ jobs:
make dist
./utils/verify_dist_tarball.sh
- name: Build nDPI [Mingw-w64] (runs only on ubuntu jobs)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.nBPF, 'nBPF')
run: |
make distclean
env CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --host=x86_64-w64-mingw32
./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --host=x86_64-w64-mingw32
make all
env:
CC:
- name: Display qemu specified architecture (arm64 - little endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'arm64')
uses: docker://multiarch/ubuntu-core:arm64-bionic
+111
View File
@@ -0,0 +1,111 @@
name: Scheduled builds
on:
workflow_dispatch:
schedule:
#At the end of every day
- cron: '0 0 * * *'
jobs:
coverage:
name: Coverage (ubuntu-latest)
runs-on: ubuntu-latest
env:
CFLAGS: -Werror
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
sudo apt-get install rrdtool librrd-dev
sudo apt-get install libpcre3-dev libmaxminddb-dev lcov
sudo apt-get install wdiff colordiff
- name: Configure
run: ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-code-coverage --with-pcre --with-maxminddb --enable-tls-sigs
- name: Build
run: make all
- name: Test
run: |
make check VERBOSE=1
- name: Generate Coverage Report
run: |
make coverage
- uses: actions/upload-artifact@v3
with:
name: ndpi-coverage-report
path: coverage_report
retention-days: 7
documentation:
name: Documentation (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set up Python 3.9
uses: actions/setup-python@v4
with:
python-version: "3.9"
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison doxygen
- name: Configure nDPI library
run: |
./autogen.sh --with-only-libndpi --enable-option-checking=fatal --enable-debug-build
- name: Generate Documentation
run: |
pip install --upgrade pip
pip install -r doc/requirements.txt
make doc
mkdir -vp doc/_build/ndpi-documentation-upload/ndpi-documentation
mv -v doc/_build/html doc/_build/ndpi-documentation-upload/ndpi-documentation/html
- uses: actions/upload-artifact@v3
with:
name: ndpi-documentation
path: doc/_build/ndpi-documentation-upload
retention-days: 7
performance:
name: Performance (ubuntu-latest)
runs-on: ubuntu-latest
env:
GO111MODULE: on
CFLAGS: -Werror
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo add-apt-repository ppa:longsleep/golang-backports
sudo apt-get update
sudo apt-get install libunwind-dev
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison libpcap-dev
sudo apt-get install golang-1.16-go libgoogle-perftools-dev graphviz
go install github.com/google/pprof@latest
sudo ln -s ${HOME}/go/bin/pprof /usr/bin/pprof
pprof -h
- name: Configure nDPI library
run: |
./autogen.sh --enable-gprof --enable-option-checking=fatal --with-pcre --with-maxminddb --enable-tls-sigs
- name: Build nDPI library
run: |
make
- name: Performance Profiling
run: |
./tests/do.sh
mkdir ndpi-performance-upload
for d in $(find ./tests/cfgs/* -type d -maxdepth 0 2>/dev/null) ; do
PROFILE="$(basename $d)"
mv -v tests/cfgs/${PROFILE}/result/cpu_profile.png ndpi-performance-upload/${PROFILE}_cpu_profile.png
mv -v tests/cfgs/${PROFILE}/result/heap_profile.png ndpi-performance-upload/${PROFILE}_heap_profile.png
done
- uses: actions/upload-artifact@v3
with:
name: ndpi-performance
path: ndpi-performance-upload
retention-days: 7
+1 -1
View File
@@ -18,7 +18,7 @@ jobs:
uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@master
with:
oss-fuzz-project-name: 'ndpi'
fuzz-seconds: 600
fuzz-seconds: 1200
dry-run: false
sanitizer: ${{ matrix.sanitizer }}
- name: Check Crash (fails when a crash is detected)
+59
View File
@@ -0,0 +1,59 @@
name: "CodeQL"
on:
push:
branches: [ 'dev' ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ 'dev' ]
schedule:
- cron: '25 19 * * 2'
jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
env:
GO111MODULE: on
CFLAGS: -Werror
permissions:
actions: read
contents: read
security-events: write
strategy:
fail-fast: false
matrix:
language: [ 'cpp', 'python', 'ruby' ]
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo add-apt-repository ppa:longsleep/golang-backports
sudo apt-get update
sudo apt-get install -y libunwind-dev
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison libpcap-dev
sudo apt-get install golang-1.16-go libgoogle-perftools-dev graphviz
go install github.com/google/pprof@latest
sudo ln -s ${HOME}/go/bin/pprof /usr/bin/pprof
pprof -h
- name: Configure nDPI library
run: |
./autogen.sh --enable-gprof --enable-option-checking=fatal --with-pcre --with-maxminddb --enable-tls-sigs --enable-debug-messages
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
- name: Build nDPI library
run: |
make
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
with:
category: "/language:${{matrix.language}}"
+131 -56
View File
@@ -1,8 +1,34 @@
*.[o]
*.db
*.db-*
*.dll
*.ilk
*.in
*.ipch
*.ko
*.la
*.lo
*.log
*.obj
*.opendb
*.pdb
*.recipe
*.suo
*.swp
*.tlog
*.user
*o.cmd
*o.cmd
*~
.autotools
.cproject
.dirstamp
.libs
.project
.settings
/Makefile
/aclocal.m4
/autom4te.cache/
.autotools
compile
/compile
/config.guess
/config.h
@@ -12,71 +38,120 @@ compile
/config.sub
/configure
/configure.tmp
.cproject
/depcomp
.dirstamp
/doc/Doxyfile.cfg
/example/.deps/
/example/Makefile
/example/ndpiReader
/example/Makefile.dpdk
/example/build
/example/libndpiReader.a
/example/ndpiReader
/example/ndpiReader.exe
/example/ndpiSimpleIntegration
*.in
/install-sh
*.ko
*.la
/libndpi.pc
.libs
/libtool
*.lo
/ltmain.sh
/m4/libtool.m4
/m4/lt~obsolete.m4
/m4/ltoptions.m4
/m4/ltsugar.m4
/m4/ltversion.m4
/Makefile
/missing
*.[o]
*.obj
*o.cmd
*o.cmd
.project
.settings
/src/include/Makefile
/src/include/ndpi_config.h
/src/include/ndpi_api.h
/src/include/ndpi_define.h
/src/lib/.deps/
/src/lib/Makefile
/src/lib/protocols/.deps/
/src/lib/third_party/src/.deps/
stamp-h1
*.swp
/test-driver
/tests/Makefile
/fuzz/.deps/
/fuzz/Makefile
/fuzz/fuzz_ndpi_reader
/fuzz/fuzz_ndpi_reader_with_main
/fuzz/fuzz_process_packet
/fuzz/fuzz_process_packet_with_main
/tests/dga/Makefile
/tests/do.sh
/tests/unit/Makefile
/example/libndpiReader.a
/example/ndpiReader.exe
/src/lib/libndpi.a
/src/lib/libndpi.so*
/src/lib/libndpi
/src/lib/libndpi-4.1.0.dll
/src/lib/libndpi.4
/tests/dga/dga_evaluate
/tests/unit/unit
/tags
/fuzz/fuzz_ndpi_reader_alloc_fail
/fuzz/fuzz_quic_get_crypto_data
/fuzz/fuzz_alg_bins
/fuzz/fuzz_alg_hll
/fuzz/fuzz_alg_hw_rsi_outliers_da
/fuzz/fuzz_alg_jitter
/fuzz/fuzz_alg_crc32_md5
/fuzz/fuzz_alg_ses_des
/fuzz/fuzz_alg_bytestream
/fuzz/fuzz_config
/fuzz/fuzz_community_id
/fuzz/fuzz_serialization
/fuzz/fuzz_ds_patricia
/fuzz/fuzz_ds_libcache
/fuzz/fuzz_ds_tree
/fuzz/fuzz_ds_ptree
/fuzz/fuzz_ds_hash
/fuzz/fuzz_ds_ahocorasick
/fuzz/fuzz_ds_bitmap64
/fuzz/fuzz_ds_domain_classify
/fuzz/fuzz_libinjection
/fuzz/fuzz_binaryfusefilter
/fuzz/fuzz_tls_certificate
/fuzz/fuzz_dga
/fuzz/fuzz_ds_cmsketch
/fuzz/fuzz_gcrypt_light
/fuzz/fuzz_gcrypt_aes
/fuzz/fuzz_gcrypt_gcm
/fuzz/fuzz_gcrypt_cipher
/fuzz/fuzz_ndpi_reader_payload_analyzer
/fuzz/fuzz_filecfg_protocols
/fuzz/fuzz_readerutils_workflow
/fuzz/fuzz_readerutils_parseprotolist
/fuzz/fuzz_ndpi_reader_alloc_fail_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_seed_corpus.zip
/fuzz/fuzz_quic_get_crypto_data_seed_corpus.zip
/fuzz/fuzz_config_seed_corpus.zip
/fuzz/fuzz_community_id_seed_corpus.zip
/fuzz/fuzz_serialization_seed_corpus.zip
/fuzz/fuzz_alg_ses_des_seed_corpus.zip
/fuzz/fuzz_alg_bins_seed_corpus.zip
/fuzz/fuzz_alg_hll_seed_corpus.zip
/fuzz/fuzz_alg_jitter_seed_corpus.zip
/fuzz/fuzz_alg_crc32_md5_seed_corpus.zip
/fuzz/fuzz_alg_hw_rsi_outliers_da_seed_corpus.zip
/fuzz/fuzz_alg_bytestream_seed_corpus.zip
/fuzz/fuzz_ds_patricia_seed_corpus.zip
/fuzz/fuzz_ds_libcache_seed_corpus.zip
/fuzz/fuzz_ds_tree_seed_corpus.zip
/fuzz/fuzz_ds_ptree_seed_corpus.zip
/fuzz/fuzz_ds_ahocorasick_seed_corpus.zip
/fuzz/fuzz_ds_bitmap64_seed_corpus.zip
/fuzz/fuzz_ds_domain_classify_seed_corpus.zip
/fuzz/fuzz_libinjection_seed_corpus.zip
/fuzz/fuzz_tls_certificate_seed_corpus.zip
/fuzz/fuzz_filecfg_protocols_seed_corpus.zip
/fuzz/fuzz_dga_seed_corpus.zip
/fuzz/fuzz_ndpi_reader_payload_analyzer_seed_corpus.zip
/fuzz/fuzz_readerutils_workflow_seed_corpus.zip
/fuzz/fuzz_readerutils_parseprotolist_seed_corpus.zip
/fuzz/fuzz_*.dict
/influxdb/Makefile
/install-sh
/libndpi.pc
/libtool
/ltmain.sh
/m4/libtool.m4
/m4/ltoptions.m4
/m4/ltsugar.m4
/m4/ltversion.m4
/m4/lt~obsolete.m4
/missing
/rrdtool/Makefile
/rrdtool/rrd_anomaly
/rrdtool/rrd_similarity
/src/include/Makefile
/src/include/ndpi_config.h
/src/include/ndpi_define.h
/src/lib/.deps/
/src/lib/Makefile
/src/lib/libndpi
/src/lib/libndpi-*.dll
/src/lib/libndpi.4
/src/lib/libndpi.a
/src/lib/libndpi.so*
/src/lib/protocols/.deps/
/src/lib/third_party/src/.deps/
/tags
/test-driver
/tests/Makefile
/tests/dga/Makefile
/tests/dga/dga_evaluate
/tests/dga/dga_evaluate.exe
/tests/do.sh
/tests/performance/Makefile
/influxdb/Makefile
/doc/Doxyfile.cfg
/tests/unit/Makefile
/tests/unit/unit
compile
stamp-h1
/windows/packages/pthreads.2.9.1.4
/windows/packages/pthreads.redist.2.9.1.4
/windows/bin/x64_Debug-ndpiReader/ndpiReader.exe
/windows/obj/nDPI_Debug-ndpiReader_x64/nDPI.vcxproj.FileListAbsolute.txt
+479
View File
@@ -1,5 +1,484 @@
# CHANGELOG
#### nDPI 4.8 (Oct 2023)
## Major Changes
- Reworked lists implementation that decreased memory usage of orders of magnitude
- Improved code robustness via extensive code fuzzing
- Various improvements to overall library performance
- Extended IPv6 support
## New Supported Protocols and Services
- Add "Heroes of the Storm" video game signature detection. (#1949)
- Add Apache Thrift protocol dissector. (#2007)
- Add Remote Management Control Protocol (RMCP).
- Add Service Location Protocol dissector. (#2036)
- Add VK detection (#1880)
- Add Yandex services detection (#1882)
- Add a new protocol id for generic Adult Content traffic (#1906)
- Add a new protocol id for generic advertisement/analytics/tracking stuff (#1904)
- Add bitcoing protocol dissector. (#1992)
- Add detection of Roblox games (#2054)
- Add support for (un-encrypted) HTTP/2 (#2087)
- Add support for Epic Games and GeForceNow/Nvidia (#1990)
- Add support for SRTP (#1977)
- Added BACnet dissector. (#1940)
- Added HAProxy protocol. (#2088)
- Added OICQ dissector. (#1950)
- Added OperaVPN detection
- ProtonVPN: add basic detection (#2006)
- Added detection of Facebook Reels and Stories
- Add an heuristic to detect fully encrypted flows (#2058)
- Added NDPI_MALWARE_HOST_CONTACTED flow risk
- Added NDPI_TLS_ALPN_SNI_MISMATCH flow risk
## Improvements
- Improve protocol detection for:
- FreeBSD compilation fix (C) update
- Gnutella: improve detection (#2019)
- H323: fix false positives (#1916)
- HTTP: fix another memory access error (#2049)
- HTTP: fix extraction of filename (#2046)
- HTTP: fix heap-buffer-overflow (#2044)
- HTTP: improve extraction of metadata and of flow risks (#1959)
- HTTP: remove useless code about XBOX (#1958)
- HTTP: rework state machine (#1966)
- Hangout: detect Hangout/Duo/GoogleMeet/... in the STUN code (#2025)
- Enhance DNS risk for long hostnames (> 32)
- Enhanced MS teams STUN/Azure detection
- Enhanced custom port definition and improved error reporting in case of duplications
- Improve detection of Alibaba flows (#1991)
- Improve detection of crawler/bot traffic (#1956)
- Improve detection of crawlers/bots (#1968)
- Improved MGCP detection by allowing '\r' as line feed.
- Improved MS Teams detection with heuristic
- Improved Steam detection by adding steamdiscover pattern. (#2105)
- Improved Wireguard detection
- Improved checks for duplicated entries in protocols file
- Improved classification further reducing memory used
- Improved detection of invalid chars in DNS names
- Improved domain search tet unit
- Improved helper scripts. (#1986)
- MS Teams enhancement
- MySql: improve detection (#1928)
- zabbix: improve detection (#2055)
## Tools
- ndpiReader: allow to configure LRU caches TTL and size (#2004)
- ndpiReader: fix VXLAN de-tunneling (#1913)
- ndpiReader: fix export of DNS/BitTorrent attributes (#1985)
- ndpiReader: fix export of HTTP attributes (#1982)
- ndpiReader: fix flow stats (#1943)
- ndpiReader: fix print of flow payload (#1960)
- ndpiReader: improve printing of payload statistics (#1989)
- ndpiReader: print how many packets (per flow) were needed to perform full DPI (#1891)
- ndpireader: fix detection of DoH traffic based on packet distributions (#2045)
## Misc
- ARM compilation fix
- Add `ndpi_domain_classify_finalize()` function (#2084)
- Add a configuration knob to enable/disable loading of gambling list (#2047)
- Add a new flow risk about literal IP addresses used as SNI (#1892)
- Add an heuristic to detect/ignore some anomalous TCP ACK packets (#1948)
- Add another example of custom rules (#1923)
- Add support for multiline json
- Add support for roaring_bitmap_xor_inplace (#1983)
- Add support for vxlan decapsulation (#1441) (#1900)
- Added Source Engine dissector. (#1937)
- Added `lists/gambling.list` to extra dist.
- Added `slackb.com` SNI. (#2067)
- Added ability to define an unlimited number of custom rules IP:port for the same IP (it used tobe limited to 2)
- Added check to avoid skype heuristic false positives
- Added comment
- Added coverage targets to `Makefile.am` for convenience. (#2039)
- Added fix for better handling exceptions rollback in case of later match
- Added hyperlink
- Added ndpi_binary_bitmap data structure
- Added ndpi_bitmap64 support
- Added ndpi_bitmap_andnot API call
- Added ndpi_bitmap_copy() API call
- Added ndpi_bitmap_is_empty() and ndpi_bitmap_optimize() API calls
- Added ndpi_domain_classify_XXX(0 API
- Added ndpi_filter_add_multi() API call
- Added ndpi_murmur_hash to the nDPI API
- Added new API calls for implementing Bloom-filter like data structures
- Added printf/fprintf replacement for some internal modules. (#1974)
- Added scripts to auto generate hostname/SNI *.inc files. (#1984)
- Added sub-domain classification fix
- Added the ability to define custom protocols with arbitrary Ids in proto.txt
- Added vlan_id in ndpi_flow2json() prototype
- Adds new pcap for testing "funny" HTTP servers
- All protocols should be excluded sooner or later (#1969)
- Allow init of app protocols w/o any hostnames set. (#2057)
- Avoid calling `ndpi_reconcile_protocols()` twice in `ndpi_detection_giveup()` (#1996)
- Boundary check
- CI: fix `Performance` job (#1936)
- Centos7 fixes
- Changed logging callback function sig. (#2000)
- Changes for supporting more efficient sub-string matching
- Classification fixes
- DNS: extract geolocation information, if available (#2065)
- Debian 12 fixes
- Disabled query string validation in MDNS in order to avoid zapping chars that in DNS (instead) are not permitted
- DisneyPlus/Hulu ip lists should be auto-generated (#1905)
- Extend content list of Microsoft protocols (#1930)
- Extend content-match list (#1967)
- Fix LRU/Patricia/Automa stats in `ndpiReader` with multiple threads (#1934)
- Fix MS Teams detection with heuristic (#1972)
- Fix access to packet/flow information (#2013)
- Fix an heap-buffer-overflow (#1994)
- Fix classification-by-ip in `ndpi_detection_giveup` (#1981)
- Fix compilation (#2011)
- Fix compilation in CI jobs (#2048)
- Fix compilation on Windows (#2072)
- Fix compilation with GCC-7 and latest RoaringBitmap code (#1886)
- Fix detection of packet direction and NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1883)
- Fix export/serialization of `flow->risk` (#1885)
- Fix for buffer overflow in serialization
- Fix insert of ip addresses into patricia tree(s) (#1895)
- Fix missing u_char, u_short and u_int typedefs for some platforms e.g.: (#2009)
- Fix packet counters (#1884)
- Fix some errors found by fuzzers (#2078)
- Fix some memory errors triggered by allocation failures (#1995)
- Fix some prototypes (#2085)
- Fix string truncation. (#2056)
- Fixed OpenWRT arm related build issues. (#2104)
- Fixed heap-buffer-overflow issue
- Fixed heap-overflow if compiled with `--enable-tls-sigs`. (#2038)
- Fixed invalid use of ndpi_free(). Sorry, my fault. (#1988)
- Fixed missing AS_HELP_STRING in configure.ac. (#1893)
- Fixed two OpenWRT arm related build issues. (#2103)
- Fixes matches with domain name strings that start with a dot
- Fixes risk mask exception handling while improving the overall performance
- Implemented Count-Min Sketch [count how many times a value has been observed]
- Implemented Zoom/Teams stream type detection
- Implemented ndpi_XXX_reset() API calls whre XXX is ses, des, hw
- Implemented ndpi_predict_linear() for predicting a timeseries value overtime
- Improved debug output. (#1951)
- Improved invalid logging via printf().
- Improved line protocol dissection with heuristic
- Improved missing usage of nDPIs malloc wrapper. Fixes #1978. (#1979)
- Improved protocol detection exploiting IP-based guess Reworked ndpi_reconcile_protocols() that is now called only in front of a match (less overhead)
- Improvement for reducing false positives
- Included Gambling website data from the Polish `hazard.mf.gov.pl` list (#2041)
- Keep master protocol in `ndpi_reconcile_protocols`
- Leak fix
- Language fix
- Line: fix heap-buffer-overflow error (#2015)
- Made VK protocol detection more strict
- Make Bittorrent LRU cache IPv6 aware. (#1909)
- Merged new and old version of ndpi_domain_classify.c code
- Mullvad VPN service added (based on entry node IP addresses) (#2062)
- Numeric truncation at `ndpi_analyze.c` at lines 101, 104, 107, 110 (#1999)
- Numeric truncation at `tls.c:1010` (#2005)
- Ookla: rework detection (#1922)
- Optimizes and fixes possible out0of0boundary write in ndpi_fill_prefix_v4()
- ProtonVPN: split the ip list (#2060)
- QUIC: add support for QUIC version 2
- QUIC: export QUIC version as metadata
- QUIC: fix a memory access error
- QUIC: fix dissection of packets forcing VN
- RDP: improve detection over UDP (#2043)
- RTP: remove dead-code (#1953)
- RTP: rework code (#2021)
- Refreshed ASN lists Enhanced the Line IP list with https://ipinfo.io/AS23576/125.209.252.0/24 used by line
- Remove some useless checks (#1993)
- Remove special handling of some TCP flows without SYN (#1965)
- Removed overlapping port
- Renamed HTTP/2 to HTTP2 as the '/' can have side effects with applications sitting on top of nDPI
- Replaces free() with ndpi_free()
- Rework CI jobs to try reducing CI duration (#1903)
- Reworked domain classification based on binary filters
- Reworked initialization
- Reworked ndpi_filter_xxx implementation using compressed bitmaps
- Reworked teams handling
- RiotGames: add detection of flows (#1935)
- STUN: add dissection of DTLS handshake (#2018)
- STUN: avoid FacebookVoip false positives (#2029)
- STUN: fix Skype/MsTeams detection and monitoring logic (#2028)
- STUN: fix detection of Google Voip apps (#2031)
- STUN: fix detection over TCP
- STUN: improve WhatsappCall detection
- STUN: keep monitoring/processing STUN flows (#2012)
- STUN: tell RTP from RTCP while in monitoring state (#2027)
- Serialization fix
- Set _DEFAULT_SOURCE and _GNU_SOURCE globally. (#2010)
- Simplify `ndpi_internal_guess_undetected_protocol()` (#1941)
- Simplify the report of streaming multimedia info (#2026)
- SoftEther: fix invalid memory access
- Swap from Aho-Corasick to an experimental/home-grown algorithm that uses a probabilistic approach for handling Internet domain names.
- Sync unit tests results
- Sync unit tests results
- Sync unit tests results (#1962)
- Sync utests results (#1887)
- TLS: add basic, basic, detection of Encrypted ClientHello (#2053)
- TLS: fix another interger overflow in certificate processing (#1915)
- TLS: fix parsing of certificate elements (#1910)
- Test files for riit games
- Test multiple `ndpiReader` configurations (#1931)
- Thrift: fix heap-buffer-overflow (#2024)
- Update GitHub runners versions (#1889)
- Update every ip lists (#2079)
- Update libinjection code (#1918)
- Update protocols documentation (#2081)
- Update roaring bitmap code
- Updated line test result
- Updated pcap detection results after Facebook Reel/Stories support
- Updated results
- Updated results after the latest changes
- Win include change
- Windows code rework
- Windows compilation fixes
- Windows warning checks
- add 2 ns from fdn.fr to DoH section (#1964)
- add support for gre decapsulation (#1442) (#1921)
- added bimap and/or with allocation
- added feature to extract filename from http attachment (#2037)
- added new domain names (#2002)
- configure: add an option to enable debug build, i.e `-g` (#1929)
- fix Stack overflow caused by invalid write in ndpi_automa_match_strin… (#2035)
- fixed numeric truncation error
- fixed numeric truncation error in diameter.c (#2034)
- fixed numeric truncation error in kerberos.c (#2032)
- fixed numeric truncation error in ndpi_main.c:6837 (#1998)
- fixed numeric truncation error in rtcp.c (#2033)
- fuzz: add a new fuzzer to test TLS certificates (#1901)
- fuzz: add a new fuzzer triggering the payload analyzer function(s) (#1926)
- fuzz: add fuzzer for DGA detection code (#2042)
- fuzz: add fuzzer to test internal gcrypt code (#1920)
- fuzz: add fuzzers to test bitmap64 and domain_classify data structures (#2082)
- fuzz: add fuzzers to test reader_util code (#2080)
- fuzz: extend coverage (#2073)
- fuzz: extend fuzz coverage (#1888)
- fuzz: extend fuzzers coverage (#1952)
- fuzz: extend fuzzing coverage (#2040)
- fuzz: extend fuzzing coverage (#2052)
- fuzz: extend fuzzing coverage (#2083)
- fuzz: simplify fuzzers dependencies in CIFuzz (#1896)
- fuzz: some improvements and add two new fuzzers (#1881)
- fuzzing: extend fuzzing coverage
- in case of failure, failing result files are not listed
- minor fixes (#2023)
- oss-fuzz: sync build script with upstream
- remove redefinition to vxlanhdr struct in vxlan dissector (#1911)
- removed useless call of ndpi_set_risk func (#2022)
- tests: add an option to force the overwrite of the unit tests results (#2001)
- tests: restore some old paths as symbolic links (#2050)
- tftp: check for Option Acknowledgements
- tftp: check incrementation for DATA and ACK packets
- tftp: rework request checking to account for options
- tftp: update pcap results
- version of dirent.c that is liked by both VC++ and MinGW
#### nDPI 4.6 (Feb 2023)
## New Features
- New support for custom BPF protocol definition using nBPF (see example/protos.txt)
- Improved dissection performance
- Added fuzzing all over
## New Supported Protocols and Services
- Add protocol detection for:
- Activision
- AliCloud server access
- AVAST
- CryNetwork
- Discord
- EDNS
- Elasticsearch
- FastCGI
- Kismet
- Line App and Line VoIP calls
- Meraki Cloud
- Munin
- NATPMP
- Syncthing
- TP-LINK Smart Home
- TUYA LAN
- SoftEther VPN
- Tailscale
- TiVoConnect
## Improvements
- Improve protocol detection for:
- Anydesk
- Bittorrent (fix confidence, detection over TCP)
- DNS, add ability to decode DNS PTR records used for reverse address resolution
- DTLS (handle certificate fragments)
- Facebook Voip calls
- FastCGI (dissect PARAMS)
- FortiClient (update default ports)
- Zoom
- Add Zoom screen share detection
- Add detection of Zoom peer-to-peer flows in STUN
- Hangout/Duo Voip calls detection, optimize lookups in the protocol tree
- HTTP
- Handling of HTTP-Proxy and HTTP-Connect
- HTTP subclassification
- Check for empty/missing user-agent in HTTP
- IRC (credentials check)
- Jabber/XMPP
- Kerberos (support for Krb-Error messages)
- LDAP
- MGCP
- MONGODB (avoid false positives)
- Postgres
- POP3
- QUIC (support for 0-RTT packets received before the initial)
- Snapchat Voip calls
- SIP
- SNMP
- SMB (support for messages split into multiple TCP segments)
- SMTP (support for X-ANONYMOUSTLS command)
- STUN
- SKYPE (improve detection over UDP, remove detection over TCP)
- Teamspeak3 (License/Weblist detection)
- Threema Messenger
- TINC (avoid processing SYN packets)
- TLS
- improve reassembler
- handling of ALPN(s) and subclassification
- ignore invalid Content Type values
- WindowsUpdate
- Add flow risk:
- NDPI_HTTP_OBSOLETE_SERVER
- NDPI_MINOR_ISSUES (generic/relevant information about issues found on traffic)
- NDPI_HTTP_OBSOLETE_SERVER (Apache and nginx are supported)
- NDPI_PERIODIC_FLOW (reserved bit to be used by apps based on nDPI)
- NDPI_TCP_ISSUES
- Improve detection of WebShell and PHP code in HTTP URLs that is reported via flow risk
- Improve DGA detection
- Improve AES-NI check
- Improve nDPI JSON serialization
- Improve export/print of L4 protocol information
- Improve connection refused detection
- Add statistics for Patricia tree, Ahocarasick automa, LRU cache
- Add a generic (optional and configurable) expiration logic in LRU caches
- Add RTP stream type in flow metadata
- LRU cache is now IPv6 aware
## Tools
- ndpiReader
- Add support for Linux Cooked Capture v2
- Fix packet dissection (CAPWAP and TSO)
- Fix Discarded bytes statistics
## Fixes
- Fix classification by-port
- Fix exclusion of DTLS protocol
- Fix undefined-behaviour in ahocorasick callback
- Fix infinite loop when a custom rule has port 65535
- Fix undefined-behavior when setting empty user-agent
- Fix infinite loop in DNS dissector (due to an integer overflow)
- Fix JSON export of IPv6 addresses
- Fix memory corruptions in Bittorrent, HTTP, SoftEther, Florensia, QUIC, IRC, TFTP dissectors
- Fix stop of extra dissection in HTTP, Bittorrent, Kerberos
- Fix signed integer overflow in ASN1/BER dissector
- Fix char/uchar bug in ahocorasick
- Fix endianess in IP-Port lookup
- Fix FastCGI memory allocation issue
- Fix metadata extraction in NAT-PMP
- Fix invalid unidirectional traffic alert for unidirectional protocols (e.g. sFlow)
## Misc
- Support for Rocky Linux 9
- Enhance fuzzers to test nDPI configurations, memory allocation failures, serialization/deserialization, algorithms and data structures
- GitHub Actions: update to Node.js 16
- Size of LRU caches is now configurable
#### nDPI 4.4 (July 2022)
## New Features
- Add risk information that describes why a specific risk was triggered also providing metadata
- Added API call ndpi_check_flow_risk_exceptions() for handling risk exceptions
- Split protocols in: network (e.g. TLS) and application protocols (e.g. Google)
- Extended confidence level with two new values (NDPI_CONFIDENCE_DPI_PARTIAL and NDPI_CONFIDENCE_DPI_PARTIAL_CACHE)
- Added ndpi_get_flow_error_code() API call
## New Supported Protocols and Services
- Add protocol detection for:
- UltraSurf
- i3D
- RiotGames
- TSAN
- TunnelBear VPN
- collectd
- PIM (Protocol Indipendent Multicast)
- Pragmatic General Multicast (PGM)
- RSH
- GoTo products (mainly GoToMeeting)
- Dazn
- MPEG-DASH
- Agora Software Defined Real-time Network (SD-RTN)
- Toca Boca
- VXLAN
- MDNS/LLMNR
## Improvements
- Improve protocol detection for:
- SMTP/SMTPS now supports STARTTLS
- OCSP
- TargusDataspeed
- Usenet
- DTLS (added support for old versions)
- TFTP
- SOAP via HTTP
- GenshinImpact
- IPSec/ISAKMP
- DNS
- syslog
- DHCP (various bug fixes and improvements)
- NATS
- Viber
- Xiaomi
- Raknet
- gnutella
- Kerberos
- QUIC (Added support for v2drft 01)
- SSDP
- SNMP
- Improved DGA detection
- Improved AES-NI check
- Add flow risk:
- NDPI_PUNYCODE_IDN
- NDPI_ERROR_CODE_DETECTED
- NDPI_HTTP_CRAWLER_BOT
- NDPI_ANONYMOUS_SUBSCRIBER
- NDPI_UNIDIRECTIONAL_TRAFFIC
## Changes
- Added support for 64 bit bins
- Added Cloudflare WARP detection patterns
- Renamed Z39.50 -> Z3950
- Replaced nDPI's internal hashmap with uthash
- Reimplemented 1kxun application protoco
- Renamed SkypeCall to Skype_TeamsCall
- Updated Python Bindings
- Unless --with-libgcrypt is used, nDPI now uses its internal gcrypt implementation
## Fixes
- Fixes for some protocol classification families
- Fixed default protocol ports for email protocols
- Various memory and overflow fixes
- Disabled various risks for specific protocols (e.g. disable missing ALPN for CiscoVPN)
- Fix TZSP decapsulation
## Misc
- Update ASN/IPs lists
- Improved code profiling
- Use Doxygen to generate the API documentation
- Added Edgecast and Cachefly CDNs.
#### nDPI 4.2 (Feb 2022)
## New Features
+17 -4
View File
@@ -12,21 +12,34 @@ EXTRA_DIST = README.md README.fuzzer.md CHANGELOG.md CONTRIBUTING.md \
autogen.sh wireshark windows utils packages \
influxdb/Makefile.in influxdb/metric_anomaly.c influxdb/README.txt \
rrdtool/Makefile.in rrdtool/README.txt rrdtool/rrd_anomaly.c rrdtool/rrd_similarity.c \
doc/requirements.txt doc/conf.py doc/flow_risks.rst doc/guide/nDPI_QuickStartGuide.pages \
doc/requirements.txt doc/conf.py doc/flow_risks.rst doc/protocols.rst doc/guide/nDPI_QuickStartGuide.pages \
doc/guide/nDPI_QuickStartGuide.pdf doc/img/logo.png doc/index.rst \
doc/Makefile doc/what_is_ndpi.rst \
python/DEV_GUIDE.md python/dev_requirements.txt python/ndpi_example.py python/ndpi/__init__.py \
python/ndpi/ndpi_build.py python/ndpi/ndpi.py python/README.md \
python/requirements.txt python/setup.py python/tests.py \
sonar-project.properties .github .ci-ignore lgtm.yml
lists/107_gambling.list \
lists/107_gambling_custom.list \
lists/README.md \
sonar-project.properties .github .ci-ignore
.PHONY: doc doc-view
doc:
make -C doc html
doc-view:
python3 -m http.server --bind 127.0.0.1 --directory doc/_build/html 8080
coverage: all
mkdir -vp coverage_report/ndpi_coverage_report
lcov --directory . --capture --output-file lcov.info
genhtml -o coverage_report/ndpi_coverage_report lcov.info
clean-coverage:
rm -f lcov.info
find . -type f -name "*.gcda" -exec rm "{}" +
find . -type f -name "*.gcno" -exec rm "{}" +
changelog:
git log --since={`curl -s https://github.com/ntop/ndpi/releases | grep datetime | head -n1 | egrep -o "[0-9]+\-[0-9]+\-[0-9]+"`} --name-only --pretty=format:" - %s" | grep "^ " > CHANGELOG.latest
.PHONY: doc doc-view coverage clean-coverage changelog
+3 -3
View File
@@ -13,11 +13,11 @@
## Testing nDPI with ClusterFuzz Artifacts
- Download the artifact (example clusterfuzz-testcase-fuzz_process_packet-4992218834796544)
- Run nDPI against the artifact
- Example: ./fuzz/fuzz_process_packet_with_main clusterfuzz-testcase-fuzz_process_packet-4992218834796544
- Example: ./fuzz/fuzz_process_packet clusterfuzz-testcase-fuzz_process_packet-4992218834796544
The output is the error report
```
./fuzz/fuzz_process_packet_with_main /tmp/clusterfuzz-testcase-fuzz_process_packet-4992218834796544
./fuzz/fuzz_process_packet /tmp/clusterfuzz-testcase-fuzz_process_packet-4992218834796544
AddressSanitizer:DEADLYSIGNAL
=================================================================
==11590==ERROR: AddressSanitizer: SEGV on unknown address 0x61a100000087 (pc 0x00000056e6a4 bp 0x7ffd624fa170 sp 0x7ffd624fa090 T0)
@@ -33,7 +33,7 @@ The output is the error report
#8 0x4c709b in LLVMFuzzerTestOneInput /home/deri/nDPI/fuzz/fuzz_process_packet.c:30:3
#9 0x4c7640 in main /home/deri/nDPI/fuzz/fuzz_process_packet.c:90:17
#10 0x7f888e5dabf6 in __libc_start_main /build/glibc-S9d2JN/glibc-2.27/csu/../csu/libc-start.c:310
#11 0x41c399 in _start (/home/deri/nDPI/fuzz/fuzz_process_packet_with_main+0x41c399)
#11 0x41c399 in _start (/home/deri/nDPI/fuzz/fuzz_process_packet+0x41c399)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/deri/nDPI/src/lib/protocols/quic.c:203:12 in quic_len
+23 -8
View File
@@ -1,9 +1,7 @@
![ntop][ntopng_logo] ![ntop][ntop_logo]
# nDPI
[![Build Status](https://img.shields.io/github/workflow/status/ntop/nDPI/Build/dev?logo=github)](https://github.com/ntop/nDPI/actions?query=workflow%3ABuild)
[![Code Quality: Cpp](https://img.shields.io/lgtm/grade/cpp/g/ntop/nDPI.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/ntop/nDPI/context:cpp)
[![Total Alerts](https://img.shields.io/lgtm/alerts/g/ntop/nDPI.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/ntop/nDPI/alerts)
[![Build Status](https://img.shields.io/github/actions/workflow/status/ntop/nDPI/build.yml?branch=dev&logo=github)](https://github.com/ntop/nDPI/actions?query=workflow%3ABuild)
[![Fuzzing Status](https://oss-fuzz-build-logs.storage.googleapis.com/badges/ndpi.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened&can=1&q=proj:ndpi)
## What is nDPI ?
@@ -48,8 +46,18 @@ Remember to use `gmake` and not `make` on FreeBSD
On MacOS:
- brew install coreutils gcc git gettext flex bison libtool autoconf automake pkg-config libpcap json-c pcre2 libmaxminddb rrdtool
On Windows (assuming [MSYS2](https://www.msys2.org/) already installed):
- msys2 -c "pacman --noconfirm -S --needed --overwrite '\*' git mingw-w64-x86\_64-toolchain automake1.16 automake-wrapper autoconf libtool make mingw-w64-x86\_64-json-c mingw-w64-x86\_64-crt-git mingw-w64-x86\_64-pcre mingw-w64-x86\_64-libpcap"
On Windows:
There are three supported ways to build nDPI:
1. MSYS2 (assuming [MSYS2](https://www.msys2.org/) already installed):
- msys2 -c "pacman --noconfirm -S --needed --overwrite '\*' git mingw-w64-x86\_64-toolchain automake1.16 automake-wrapper autoconf libtool make mingw-w64-x86\_64-json-c mingw-w64-x86\_64-crt-git mingw-w64-x86\_64-pcre mingw-w64-x86\_64-libpcap"
2. Mingw-w64
3. Visual Studio (see `windows/nDPI.sln`)
Note: All Windows versions require [npcap](https://npcap.com/#download) with WinPcap compatibility mode enabled.
### How To Build The Documentation
@@ -70,9 +78,11 @@ The entire procedure of adding new protocols in detail:
4. Add a new entry for the search function for the new protocol in: `src/include/ndpi_protocols.h`
5. Choose (do not change anything) a selection bitmask from: `src/include/ndpi_define.h`
6. Set protocol default ports in `ndpi_init_protocol_defaults` in: `src/lib/ndpi_main.c`
7. `./autogen.sh`
8. `make`
9. `make check`
7. Be sure to have nBPF support, cloning `PF_RING` in the same directory where you cloned `nDPI`: `git clone https://github.com/ntop/PF_RING/ && cd PF_RING/userland/nbpf && ./configure && make`
8. From the `nDPI` root directory, `./autogen.sh --with-pcre` (nBPF and PCRE are usually optional, but they are needed to run/update *all* the unit tests)
9. `make`
10. `make check`
11. Update the documentation, adding this new protocol to `doc/protocols.rst`
### How to use nDPI to Block Selected Traffic
@@ -82,6 +92,11 @@ You can use nDPI to selectively block selected Internet traffic by embedding it
- Deri, Luca, et al. [nDPI: Open-source high-speed deep packet inspection](http://luca.ntop.org/nDPI.pdf) 2014 International Wireless Communications and Mobile Computing Conference (IWCMC). IEEE, 2014.
### Videos and Presentations
- [Using nDPI for Monitoring and Security](https://archive.fosdem.org/2021/schedule/event/nemondpi/)
- [Network Traffic Classification for Cybersecurity and Monitoring](https://fosdem.org/2022/schedule/event/using_ndpi_to_efficiently_classify_network_traffic/)
### nDPI-Related Projects
- [nfstream](https://github.com/aouinizied/nfstream)
+92 -28
View File
@@ -1,4 +1,4 @@
AC_INIT([libndpi],[4.3.0])
AC_INIT([libndpi],[4.8.0])
AC_CONFIG_AUX_DIR([.])
AC_CONFIG_MACRO_DIR([m4])
@@ -8,45 +8,84 @@ AM_INIT_AUTOMAKE([foreign subdir-objects])
AC_PREFIX_DEFAULT(/usr)
EXTRA_TARGETS="example tests tests/dga"
AC_ARG_WITH(only-libndpi, AS_HELP_STRING([--with-only-libndpi], [Build only libndpi (no examples, tests etc)]))
AC_ARG_WITH(only-libndpi, AS_HELP_STRING([--with-only-libndpi], [Build only libndpi (no examples, tests etc)]))
AS_IF([test "${with_only_libndpi+set}" = set],[
EXTRA_TARGETS=""
])
AC_ARG_WITH(sanitizer, AS_HELP_STRING([--with-sanitizer], [Build with support for address, undefined and leak sanitizer]))
PWD=`pwd`
AC_ARG_WITH(sanitizer, AS_HELP_STRING([--with-sanitizer], [Build with support for address, undefined and leak sanitizer]))
AC_ARG_WITH(thread-sanitizer, AS_HELP_STRING([--with-thread-sanitizer], [Build with support for thread sanitizer]))
AC_ARG_WITH(memory-sanitizer, AS_HELP_STRING([--with-memory-sanitizer], [Build with support for memory sanitizer]))
AC_ARG_WITH(macos-memory-sanitizer, AS_HELP_STRING([--with-macos-memory-sanitizer], [Build with support for memory sanitizer macOS]))
AC_ARG_ENABLE(fuzztargets, AS_HELP_STRING([--enable-fuzztargets], [Enable fuzz targets]),[enable_fuzztargets=$enableval],[enable_fuzztargets=no])
AC_ARG_ENABLE(gprof, AS_HELP_STRING([--enable-gprof], [Enable CPU/HEAP profiling with gperftools]),[enable_gprof=$enableval],[enable_gprof=no])
AC_ARG_ENABLE(code-coverage, AS_HELP_STRING([--enable-code-coverage], [Generate Code Coverage report]))
AC_ARG_WITH(local-libgcrypt, AS_HELP_STRING([--with-local-libgcrypt], [Build with libgcrypt (if present) instead of the enclosed gcrypt light]))
AC_ARG_WITH(local-libgcrypt, AS_HELP_STRING([--with-local-libgcrypt], [Build with libgcrypt (if present) instead of the enclosed gcrypt light]))
AC_ARG_ENABLE(tls-sigs, AS_HELP_STRING([--enable-tls-sigs], [Enable TLS Client signature algorithm dissection. Rarely used, but requires significantly more memory.]))
AC_ARG_ENABLE(npcap, AS_HELP_STRING([--disable-npcap], [msys2 only: Disable linkage against the wpcap/npcap import library in windows/WpdPack/Lib.]))
AC_ARG_WITH(nbpf-path, AS_HELP_STRING([--with-nbpf-path], [nBPF library custom path; default: ${PWD}/../PF_RING/userland/nbpf]),[NBPF_HOME=$withval],[NBPF_HOME=${PWD}/../PF_RING/userland/nbpf])
AC_ARG_WITH(lto-and-gold-linker, AS_HELP_STRING([--with-lto-and-gold-linker], [Build with LTO and Gold linker]))
AC_ARG_ENABLE(debug-build, AS_HELP_STRING([--enable-debug-build], [Enable debug build (`-g` flag)]),[enable_debugbuild=$enableval],[enable_debugbuild=no])
AS_IF([test "x$enable_fuzztargets" = "xyes"], [BUILD_FUZZTARGETS=1], [BUILD_FUZZTARGETS=0])
NDPI_CFLAGS="${NDPI_CFLAGS} -D_DEFAULT_SOURCE=1 -D_GNU_SOURCE=1"
AS_IF([test "x$enable_fuzztargets" = "xyes"], [
BUILD_FUZZTARGETS=1
NDPI_CFLAGS="${NDPI_CFLAGS} -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION"
], [BUILD_FUZZTARGETS=0])
AM_CONDITIONAL([BUILD_FUZZTARGETS], [test "x$enable_fuzztargets" = "xyes"])
AS_IF([test "x$enable_debugbuild" = "xyes"], [
NDPI_CFLAGS="${NDPI_CFLAGS} -g"
])
AS_IF([test "${with_sanitizer+set}" = set -a "${with_thread_sanitizer+set}" = set],[
AC_MSG_ERROR([Configure options `--with-sanitizer' and `--with_thread_sanitizer' can not used at the same time.])
])
AS_IF([test "${with_sanitizer+set}" = set -o "${with_thread_sanitizer+set}" = set],[
CFLAGS="${CFLAGS} -O0 -g3"
AS_IF([test "${with_sanitizer+set}" = set -a "${with_memory_sanitizer+set}" = set],[
AC_MSG_ERROR([Configure options `--with-sanitizer' and `--with_memory_sanitizer' can not used at the same time.])
])
AS_IF([test "${with_thread_sanitizer+set}" = set -a "${with_memory_sanitizer+set}" = set],[
AC_MSG_ERROR([Configure options `--with_thread_sanitizer' and `--with_memory_sanitizer' can not used at the same time.])
])
AS_IF([test "${with_sanitizer+set}" = set -o "${with_thread_sanitizer+set}" = set -o "${with_memory_sanitizer+set}" = set],[
NDPI_CFLAGS="${NDPI_CFLAGS} -O0 -g3"
],[
CFLAGS="${CFLAGS} -g"
dnl> Oss-fuzz doesn't really like any optimizaton flags don't set by itself
AS_IF([test "x$enable_fuzztargets" != "xyes"], [NDPI_CFLAGS="${NDPI_CFLAGS} -O2"])
])
AS_IF([test "${with_sanitizer+set}" = set -o "${with_thread_sanitizer+set}" = set -o "${with_memory_sanitizer+set}" = set],[
AS_IF([test "x$enable_gprof" = "xyes"], [
AC_MSG_ERROR([Configure options `--with-sanitizer' / `--with_thread_sanitizer' / `--with_memory_sanitizer' can not used together with `--enable-gprof'.])
])
])
AS_IF([test "${with_sanitizer+set}" = set],[
CFLAGS="${CFLAGS} -fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fsanitize=leak -fno-omit-frame-pointer"
LDFLAGS="${LDFLAGS} -fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fsanitize=leak"
NDPI_CFLAGS="${NDPI_CFLAGS} -fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fsanitize=leak -fno-omit-frame-pointer"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fsanitize=leak"
])
AS_IF([test "${with_thread_sanitizer+set}" = set],[
CFLAGS="${CFLAGS} -fsanitize=thread -fno-omit-frame-pointer"
LDFLAGS="${LDFLAGS} -fsanitize=thread"
NDPI_CFLAGS="${NDPI_CFLAGS} -fsanitize=thread -fno-omit-frame-pointer"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fsanitize=thread"
])
AS_IF([test "${with_memory_sanitizer+set}" = set],[
NDPI_CFLAGS="${NDPI_CFLAGS} -fsanitize=memory -fsanitize-memory-track-origins -fno-omit-frame-pointer"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fsanitize=memory"
])
AS_IF([test "${with_macos_memory_sanitizer+set}" = set],[
NDPI_CFLAGS="${NDPI_CFLAGS} -fsanitize=address -fno-omit-frame-pointer -fsanitize=signed-integer-overflow -fno-sanitize-recover=address"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -fsanitize=address"
])
AS_IF([test "x${enable_code_coverage}" = "xyes"],[
CFLAGS="${CFLAGS} -fprofile-arcs -ftest-coverage"
LDFLAGS="${LDFLAGS} --coverage"
NDPI_CFLAGS="${NDPI_CFLAGS} -fprofile-arcs -ftest-coverage"
NDPI_LDFLAGS="${NDPI_LDFLAGS} --coverage"
])
AS_IF([test "${enable_npcap+set}" = set],[DISABLE_NPCAP=1],[DISABLE_NPCAP=0])
@@ -56,6 +95,8 @@ LT_LIB_M
PKG_PROG_PKG_CONFIG
AC_PROG_CC
AC_PROG_CPP_WERROR
AC_C_INLINE
SYSTEM=`uname -s`
if test $SYSTEM = "Darwin"; then
@@ -95,7 +136,7 @@ if test -d ".git" || test -r ".git"; then
A=`git log src/include/ndpi_typedefs.h|wc -l`
B=`git log src/include/ndpi_protocol_ids.h|wc -l`
C=`git log src/include/ndpi_api.h.in|wc -l`
C=`git log src/include/ndpi_api.h|wc -l`
NDPI_API_VERSION=$((A+B+C))
else
GIT_RELEASE="${PACKAGE_VERSION}"
@@ -144,7 +185,14 @@ AS_IF([test "x${enable_tls_sigs}" = "xyes"],[
HANDLE_TLS_SIGS=""
])
CFLAGS="-W -Wall -Wno-unused-parameter -Wno-unused-function -Wno-address-of-packed-member ${CFLAGS}"
NDPI_CFLAGS="-W -Wall -Wno-unused-parameter -Wno-unused-function -Wno-address-of-packed-member ${NDPI_CFLAGS} -Wno-attributes"
#Workaround for RoaringBitmap with gcc-7
NDPI_CFLAGS="-Wno-attributes ${NDPI_CFLAGS}"
AS_IF([test "${with_lto_and_gold_linker+set}" = set], [
NDPI_CFLAGS="${NDPI_CFLAGS} -flto -fuse-ld=gold -Wno-unused-command-line-argument"
NDPI_LDFLAGS="${NDPI_LDFLAGS} ${NDPI_CFLAGS}"
])
AC_CHECK_HEADERS([netinet/in.h stdint.h stdlib.h string.h unistd.h math.h float.h])
AC_CHECK_LIB([m], [sqrt], [], [LIBM="-lm"])
@@ -155,6 +203,7 @@ if test "x$ac_cv_lib_rrd_rrd_fetch_r" = xyes; then :
EXTRA_TARGETS="$EXTRA_TARGETS rrdtool"
fi
ADDITIONAL_INCS=
ADDITIONAL_LIBS="$LIBM"
PCAP_HOME=$HOME/PF_RING/userland
@@ -179,17 +228,25 @@ fi
MACHINE=`uname -m`
CUSTOM_NDPI=
NBPF_ENABLED=0
AC_MSG_CHECKING([PF_RING nBPF ($NBPF_HOME)])
LIBNBPF=${NBPF_HOME}/libnbpf.a
if test -f ${LIBNBPF}; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} ${LIBNBPF}"
ADDITIONAL_INCS="${ADDITIONAL_INCS} -I${NBPF_HOME}"
AC_DEFINE_UNQUOTED(HAVE_NBPF, "1", [PF_RING nBPF is present])
AC_MSG_RESULT(yes)
NBPF_ENABLED=1
else
AC_MSG_RESULT(no)
fi
CUSTOM_NDPI=
if test -d ../nDPI-custom; then :
CUSTOM_NDPI="-DCUSTOM_NDPI_PROTOCOLS"
AC_MSG_RESULT([Compiling with custom nDPI protocols])
fi
if test -d ../ndpi-pro; then :
with_maxminddb=1
fi
case "$host" in
*-*-mingw32*|*-*-msys)
PCAP_INC=""
@@ -197,16 +254,16 @@ case "$host" in
case "$host" in
x86_64-*)
BUILD_MINGW_X64=1
LDFLAGS="${LDFLAGS} -L/mingw64/lib"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -L/mingw64/lib"
;;
i686-*)
LDFLAGS="${LDFLAGS} -L/mingw32/lib"
NDPI_LDFLAGS="${NDPI_LDFLAGS} -L/mingw32/lib"
;;
*)
AC_MSG_ERROR([Unsupported MingW cross compiler.])
;;
esac
CFLAGS="-D__USE_MINGW_ANSI_STDIO -D__STDC_FORMAT_MACROS ${CFLAGS}"
NDPI_CFLAGS="-D__USE_MINGW_ANSI_STDIO -D__STDC_FORMAT_MACROS ${NDPI_CFLAGS}"
LIBS="${LIBS} -lws2_32"
BUILD_MINGW=1
EXE_SUFFIX=".exe"
@@ -287,7 +344,7 @@ AS_IF([test "x$enable_gprof" = "xyes"], [
])
GPROF_ENABLED=1
GPROF_LIBS="-ltcmalloc_and_profiler"
GPROF_CFLAGS="${pkg_cv_PROFILER_CFLAGS} ${pkg_cv_TCMALLOC_CFLAGS}"
GPROF_CFLAGS="-g3 -fvisibility=default -fno-omit-frame-pointer ${pkg_cv_PROFILER_CFLAGS} ${pkg_cv_TCMALLOC_CFLAGS}"
])
AC_CHECK_LIB(pthread, pthread_setaffinity_np, AC_DEFINE_UNQUOTED(HAVE_PTHREAD_SETAFFINITY_NP, 1, [libc has pthread_setaffinity_np]))
@@ -304,7 +361,7 @@ AS_IF([test "${with_local_libgcrypt+set}" = set],[
dnl> PCRE
PCRE_ENABLED=0
AC_ARG_WITH(pcre, [ --with-pcre Enable nDPI build with libpcre])
AC_ARG_WITH(pcre, AS_HELP_STRING([--with-pcre], [Enable nDPI build with libpcre]))
if test "${with_pcre+set}" = set; then :
AC_CHECK_LIB(pcre, pcre_compile, AC_DEFINE_UNQUOTED(HAVE_PCRE, 1, [libpcre(-dev) is present]))
if test "x$ac_cv_lib_pcre_pcre_compile" = xyes; then :
@@ -314,7 +371,7 @@ if test "${with_pcre+set}" = set; then :
fi
dnl> GeoIP
AC_ARG_WITH(maxminddb, [ --with-maxminddb Enable nDPI build with libmaxminddb])
AC_ARG_WITH(maxminddb, AS_HELP_STRING([--with-maxminddb], [Enable nDPI build with libmaxminddb]))
if test "${with_maxminddb+set}" = set; then :
AC_CHECK_LIB([maxminddb], [MMDB_lookup_sockaddr])
AC_CHECK_HEADERS([maxminddb.h])
@@ -324,6 +381,8 @@ if test "${with_maxminddb+set}" = set; then :
fi
fi
AC_DEFINE_UNQUOTED(NDPI_BASE_DIR, "${PWD}", [nDPI base directory])
dnl> Curl
dnl> AC_CHECK_LIB([curl], [curl_easy_perform])
dnl> if test ${ac_cv_lib_curl_curl_easy_perform} = "no"; then
@@ -333,7 +392,7 @@ dnl> ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lcurl"
dnl> AC_DEFINE_UNQUOTED(HAVE_CURL, 1, [curl is present])
dnl> fi
AC_CONFIG_FILES([Makefile example/Makefile example/Makefile.dpdk tests/Makefile tests/unit/Makefile tests/performance/Makefile tests/dga/Makefile rrdtool/Makefile influxdb/Makefile libndpi.pc src/include/ndpi_define.h src/lib/Makefile fuzz/Makefile src/include/ndpi_api.h doc/Doxyfile.cfg])
AC_CONFIG_FILES([Makefile example/Makefile example/Makefile.dpdk tests/Makefile tests/unit/Makefile tests/performance/Makefile tests/dga/Makefile rrdtool/Makefile influxdb/Makefile libndpi.pc src/include/ndpi_define.h src/lib/Makefile fuzz/Makefile doc/Doxyfile.cfg])
AC_CONFIG_FILES([tests/do.sh], [chmod +x tests/do.sh])
AC_CONFIG_HEADERS(src/include/ndpi_config.h)
AC_SUBST(GIT_RELEASE)
@@ -344,6 +403,7 @@ AC_SUBST(NDPI_VERSION_SHORT)
AC_SUBST(PCAP_INC)
AC_SUBST(PCAP_LIB)
AC_SUBST(LIBRRD)
AC_SUBST(ADDITIONAL_INCS)
AC_SUBST(ADDITIONAL_LIBS)
AC_SUBST(DL_LIB)
AC_SUBST(DPDK_TARGET)
@@ -361,7 +421,11 @@ AC_SUBST(GPROF_LIBS)
AC_SUBST(GPROF_ENABLED)
AC_SUBST(USE_HOST_LIBGCRYPT)
AC_SUBST(PCRE_ENABLED)
AC_SUBST(NBPF_ENABLED)
AC_SUBST(HANDLE_TLS_SIGS)
AC_SUBST(DISABLE_NPCAP)
AC_SUBST(EXE_SUFFIX)
AC_SUBST(NDPI_CFLAGS)
AC_SUBST(NDPI_LDFLAGS)
AC_SUBST(NDPI_BASE_DIR)
AC_OUTPUT
+49 -6
View File
@@ -73,9 +73,9 @@ HTTP only: this risk is triggered whenever the user agent contains suspicious ch
.. _Risk 012:
NDPI_HTTP_NUMERIC_IP_HOST
NDPI_NUMERIC_IP_HOST
=========================
HTTP only: this risk is triggered whenever we're accessing a host using its IP rather than its symbolic name. Example http://1.2.3.4.
This risk is triggered whenever a HTTP/TLS/QUIC connection is using a literal IPv4 or IPv6 address as ServerName (TLS/QUIC; example: SNI=1.2.3.4) or as Hostname (HTTP; example: http://1.2.3.4.).
.. _Risk 013:
@@ -141,7 +141,8 @@ This risk indicates that the protocol used is insecure and that a secure protoco
NDPI_DNS_SUSPICIOUS_TRAFFIC
===========================
This risk is returned when DNS traffic returns an unexpected/obsolete `record type <https://en.wikipedia.org/wiki/List_of_DNS_record_types>`_.
This risk is returned when DNS traffic returns an unexpected/obsolete `record type <https://en.wikipedia.org/wiki/List_of_DNS_record_types>`_
or when a suspicious query with a very long host name is detected.
.. _Risk 024:
@@ -213,13 +214,13 @@ This risk is triggered when a TLS fatal alert is detected in the TLS flow. See `
NDPI_SUSPICIOUS_ENTROPY
=======================
This risk is used to detect suspicious data carried in ICMP packets whose entropy (used to measure how data is distributed, hence to indirectly guess the type of data carried on) is suspicious and thus that it can indicate a data leak.
This risk is used to detect suspicious data carried in ICMP packets whose entropy (used to measure how data is distributed, hence to indirectly guess the type of data carried on) is suspicious and thus that it can indicate a data leak. Suspicious values indicate random entropy or entropy that is similar to encrypted traffic. In the latter case, this can be a suspicious data exfiltration symptom.
.. _Risk 036:
NDPI_CLEAR_TEXT_CREDENTIALS
===========================
Clear text protocols are not bad per-se, but they should be avoided when they carry credentials as they can be intercepted by malicious users. This risk is triggered whenever clear text protocols (e.g. FTP, HTTP, IMAP...) contain credentials in clear text (read it as nDPI does not trigger this risk for HTTP connections that do not carry credentials).
Clear text protocols are not intrinsically bad, but they should be avoided when they carry credentials as they can be intercepted by malicious users. This risk is triggered whenever clear text protocols (e.g. FTP, HTTP, IMAP...) contain credentials in clear text (read it as nDPI does not trigger this risk for HTTP connections that do not carry credentials).
.. _Risk 037:
@@ -281,7 +282,49 @@ Example: the flow is generated by an iCloud-private-relay exit node.
.. _Risk 046:
NDPI_UNIDIRECTIONAL_TRAFFIC_SUBSCRIBER
NDPI_UNIDIRECTIONAL_TRAFFIC
===================================
The risk is set whenever the flow has unidirectional traffic (typically no traffic on the server to client direction). THis
risk is not triggered for multicast/broadcast destinations.
.. _Risk 047:
NDPI_HTTP_OBSOLETE_SERVER
===================================
This risk is generated whenever a HTTP server uses an obsolete HTTP server version.
.. _Risk 048:
NDPI_PERIODIC_FLOW
==================
This risk is generated whenever a flow is observed at a specific periodic pace (e.g. every 10 seconds).
.. _Risk 049:
NDPI_MINOR_ISSUES
=================
Minor packet/flow issues (e.g. DNS traffic with zero TTL) have been detected.
.. _Risk 050:
NDPI_TCP_ISSUES
===============
Relevant TCP connection issues such as connection refused, scan, or probe attempt.
.. _Risk 051
NDPI_FULLY_ENCRYPTED
====================
Flow with Unknown protocol containing encrypted traffic.
.. _Risk 052
NDPI_TLS_ALPN_SNI_MISMATCH
=========================
Invalid TLS ALPN/SNI mismatch. For instance ALPN advertises the flow as h2 (HTTP/2.0) and no SNI is reported.
.. _Risk 053
NDPI_MALWARE_CONTACTED
======================
Client contacted a server host labelled as malware.
+3 -1
View File
@@ -14,6 +14,7 @@ nDPI is an open source DPI (Deep Packet Inspection) toolkit for traffic analysis
:caption: Developer's Guide
api/library_root
protocols
flow_risks
.. toctree::
@@ -27,7 +28,8 @@ nDPI is an open source DPI (Deep Packet Inspection) toolkit for traffic analysis
nEdge <https://www.ntop.org/guides/nedge/>
nScrub <https://www.ntop.org/guides/nscrub/>
nBox <https://www.ntop.org/guides/nbox/>
nTap <https://www.ntop.org/guides/ntap/>
.. Indices and tables
.. ==================
..
+150
View File
@@ -0,0 +1,150 @@
nDPI Protocols List
###################
This page provides the list of the protocols/applications supported by nDPI. For each protocol there is a brief description, some links to further, more detailed information and, optionally, some notes that might be useful when handling such a protocol (from the application/integrator point of view)
Work in progress!
.. _Proto 338:
`NDPI_PROTOCOL_SRTP`
====================
The Secure Real-time Transport Protocol (SRTP) is a profile for Real-time Transport Protocol (RTP) intended to provide encryption, message authentication, integrity, and replay attack protection to the RTP data.
References: `RFC3711 <https://datatracker.ietf.org/doc/html/rfc3711>`_.
Notes:
- You can think of SRTP simply as the "encrypted" version of RTP, something like HTTPS vs HTTP;
- It is not usually possible to tell RTP from SRTP. nDPI generally uses the former and it uses the latter only when it is really sure that the media stream has been encrypted.
.. _Proto 339:
`NDPI_PROTOCOL_OPERA_VPN`
=========================
Opera VPN is a free VPN included with Opera Browser.
References: `Main site <https://www.opera.com/it/features/free-vpn>`_.
.. _Proto 340:
`NDPI_PROTOCOL_EPICGAMES`
=========================
Epic Games is a video game company developing the Unreal Engine and some successful games as Fortnite and Gears of War.
References: `Main site <https://store.epicgames.com/en-US/>`_ and `Fortnite <https://www.fortnite.com/>`_.
.. _Proto 341:
`NDPI_PROTOCOL_GEFORCENOW`
==========================
GeForce Now is the brand used by Nvidia for its cloud gaming service.
References: `Main site <https://www.nvidia.com/en-us/geforce-now/>`_.
.. _Proto 342:
`NDPI_PROTOCOL_NVIDIA`
======================
Generic web traffic from Nvidia sites.
References: `Main site <https://www.nvidia.com>`_.
.. _Proto 343:
`NDPI_PROTOCOL_BITCOIN`
=======================
Bitcoin is one of the most common crypto currencies.
References: `Main site <https://en.bitcoin.it/wiki/Protocol_documentation>`_.
Notes:
- Not each crypto exchange is a mining, it could be a normal transaction, sending or receving.
- Bitcoin network protocol covers the broader set of rules that govern how all nodes in the network communicate and sync with each others blocks and transactions.
- On the other hand mining protocols are more specific and deal directly with how miners interact with the network and participate in the mining process.
.. _Proto 344:
`NDPI_PROTOCOL_PROTONVPN`
=========================
Proton VPN is a VPN service operated by the Swiss company Proton AG, the company behind the email service Proton Mail
References: `Main site <https://protonvpn.com/>`_.
.. _Proto 345:
`NDPI_PROTOCOL_THRIFT`
======================
Apache Thrift is a generic data interchange framework that supports a bunch of different languages and platforms.
References: `Official site <https://thrift.apache.org>`_ and `Github <https://github.com/apache/thrift>`_.
.. _Proto 346:
`NDPI_PROTOCOL_ROBLOX`
======================
Roblox is an online game platform and game creation system.
References: `Main site <https://www.roblox.com/>`_.
Notes:
- Since Roblox games use a custom version of the RakNet protocol, some Roblox flows might be classified as RakNet.
.. _Proto 347:
`NDPI_PROTOCOL_SERVICE_LOCATION`
================================
The Service Location Protocol is a service discovery protocol that allows computers and other devices to find services in a local area network without prior configuration.
References: `SLPv1 <https://datatracker.ietf.org/doc/html/rfc2165>`_ and `SLPv2 <https://datatracker.ietf.org/doc/html/rfc2608>`_.
.. _Proto 348:
`NDPI_PROTOCOL_MULLVAD`
=======================
Mullvad is a VPN service operated by Mullvad VPN AB, based in Sweden
References: `Main site <https://mullvad.net/>`_.
.. _Proto 349:
`NDPI_PROTOCOL_HTTP2`
=====================
HTTP/2 (originally named HTTP/2.0) is a major revision of the HTTP network protocol used by the World Wide Web.
References: `RFC <https://datatracker.ietf.org/doc/html/rfc9113>`_.
Notes:
- HTTP/2 is almost always encrypted, i.e. transported over TLS.
.. _Proto 350:
`NDPI_PROTOCOL_HAPROXY`
=======================
HAProxy is a free and open source software that provides a high availability load balancer and reverse proxy for TCP and HTTP-based applications that spreads requests across multiple servers.
References: `Main site: <https://www.haproxy.org>`_.
.. _Proto 351:
`NDPI_PROTOCOL_RMCP`
====================
The Intelligent Platform Management Interface (IPMI) is a set of computer interface specifications for an autonomous computer subsystem that provides management and monitoring capabilities independently of the host system's CPU, firmware (BIOS or UEFI) and operating system.
References: `Protocol Specs: <https://www.dmtf.org/sites/default/files/standards/documents/DSP0114.pdf>`_.
+6 -3
View File
@@ -1,6 +1,9 @@
AR=@AR@
CC=@CC@
CXX=@CXX@
ifeq ($(OS),Darwin)
CC=clang -fno-color-diagnostics
endif
BUILD_MINGW=@BUILD_MINGW@
BUILD_MINGW_X64=@BUILD_MINGW_X64@
DISABLE_NPCAP=@DISABLE_NPCAP@
@@ -9,8 +12,8 @@ SRCHOME=../src
ifneq ($(OS),Windows_NT)
CFLAGS+=-fPIC -DPIC
endif
CFLAGS+=-I$(SRCHOME)/include @PCAP_INC@ @CFLAGS@ @GPROF_CFLAGS@
LDFLAGS=@LDFLAGS@
CFLAGS+=-I$(SRCHOME)/include @NDPI_CFLAGS@ @PCAP_INC@ @GPROF_CFLAGS@
LDFLAGS+=@NDPI_LDFLAGS@
LIBNDPI=$(SRCHOME)/lib/libndpi.a
LIBS=$(LIBNDPI) @PCAP_LIB@ @ADDITIONAL_LIBS@ @LIBS@ @GPROF_LIBS@
HEADERS=reader_util.h $(SRCHOME)/include/ndpi_api.h \
@@ -79,7 +82,7 @@ cppcheck:
cppcheck --template='{file}:{line}:{severity}:{message}' --quiet --enable=all --force -I$(SRCHOME)/include *.c
clean:
/bin/rm -f *.o ndpiReader ndpiSimpleIntegration ndpiReader$(EXE_SUFFIX) ndpiSimpleIntegration$(EXE_SUFFIX) ndpiReader.dpdk
/bin/rm -f *.o ndpiReader ndpiSimpleIntegration ndpiReader$(EXE_SUFFIX) ndpiSimpleIntegration$(EXE_SUFFIX) ndpiReader.dpdk libndpiReader.a
/bin/rm -f .*.dpdk.cmd .*.o.cmd *.dpdk.map .*.o.d
/bin/rm -f _install _postbuild _postinstall _preinstall
/bin/rm -rf build
+1192 -615
View File
File diff suppressed because it is too large Load Diff
+7 -7
View File
@@ -137,7 +137,7 @@ struct nDPI_workflow {
struct nDPI_reader_thread {
struct nDPI_workflow * workflow;
pthread_t thread_id;
int array_index;
uint32_t array_index;
};
static struct nDPI_reader_thread reader_threads[MAX_READER_THREADS] = {};
@@ -549,7 +549,7 @@ static void ndpi_process_packet(uint8_t * const args,
uint16_t l4_len = 0;
uint16_t type;
int thread_index = INITIAL_THREAD_HASH; // generated with `dd if=/dev/random bs=1024 count=1 |& hd'
uint32_t thread_index = INITIAL_THREAD_HASH; // generated with `dd if=/dev/random bs=1024 count=1 |& hd'
if (reader_thread == NULL) {
return;
@@ -884,7 +884,7 @@ static void ndpi_process_packet(uint8_t * const args,
flow_to_process->detected_l7_protocol =
ndpi_detection_process_packet(workflow->ndpi_struct, flow_to_process->ndpi_flow,
ip != NULL ? (uint8_t *)ip : (uint8_t *)ip6,
ip_size, time_ms);
ip_size, time_ms, NULL);
if (ndpi_is_protocol_detected(workflow->ndpi_struct,
flow_to_process->detected_l7_protocol) != 0 &&
@@ -943,7 +943,7 @@ static void ndpi_process_packet(uint8_t * const args,
{
uint8_t unknown_tls_version = 0;
char buf_ver[16];
printf("[%8llu, %d, %4d][TLS-CLIENT-HELLO] version: %s | sni: %s | alpn: %s\n",
printf("[%8llu, %d, %4d][TLS-CLIENT-HELLO] version: %s | sni: %s | (advertised) ALPNs: %s\n",
workflow->packets_captured,
reader_thread->array_index,
flow_to_process->flow_id,
@@ -951,12 +951,12 @@ static void ndpi_process_packet(uint8_t * const args,
flow_to_process->ndpi_flow->protos.tls_quic.ssl_version,
&unknown_tls_version),
flow_to_process->ndpi_flow->host_server_name,
(flow_to_process->ndpi_flow->protos.tls_quic.alpn != NULL ?
flow_to_process->ndpi_flow->protos.tls_quic.alpn : "-"));
(flow_to_process->ndpi_flow->protos.tls_quic.advertised_alpns != NULL ?
flow_to_process->ndpi_flow->protos.tls_quic.advertised_alpns : "-"));
flow_to_process->tls_client_hello_seen = 1;
}
if (flow_to_process->tls_server_hello_seen == 0 &&
flow_to_process->ndpi_flow->l4.tcp.tls.certificate_processed != 0)
flow_to_process->ndpi_flow->tls_quic.certificate_processed != 0)
{
uint8_t unknown_tls_version = 0;
char buf_ver[16];
+18 -2
View File
@@ -2,9 +2,20 @@
# <tcp|udp>:<port>,<tcp|udp>:<port>,.....@<proto>
tcp:81,tcp:8181@HTTP
udp:5061-5062@SIP
udp:5062@SIP
tcp:860,udp:860,tcp:3260,udp:3260@iSCSI
tcp:3000@ntop
tcp:9002@Elasticsearch
tcp:5601@Kibana
tcp:65535@TestProto
#
# nBPF filters (https://github.com/ntop/PF_RING/tree/dev/userland/nbpf))
#
# NOTE: they are evaluated in the same order they are defined !
#
nbpf:"host 192.168.1.1 and port 80"@HomeRouter
# Subprotocols
# Format:
@@ -39,6 +50,10 @@ ip:213.75.170.11/32:443@CustomProtocol
ip:8.248.73.247:443@AmazonPrime
ip:54.80.47.130@AmazonPrime
ip:3.3.3.3:443@CustomProtocolA
ip:3.3.3.3:444@CustomProtocolB
ip:3.3.3.3:446@CustomProtocolC=400
#
# Risk Exceptions
#
@@ -50,7 +65,8 @@ ip:54.80.47.130@AmazonPrime
# For IPs, the flow risk is put in AND (source IP mask OR destination IP mask)
# For Flows with a hostname (e.g. TLS) the risk is also put in AND with the host_risk_mask
#ip_risk_mask:192.168.1.0/24=0
#ip_risk_mask:10.196.157.228=0
ip_risk_mask:10.10.120.0/24=0
ip_risk_mask:10.196.157.228=0
host_risk_mask:".home"=0
+466 -204
View File
File diff suppressed because it is too large Load Diff
+74 -33
View File
@@ -93,6 +93,10 @@ extern int dpdk_port_deinit(int port);
#define SERIALIZATION_BUFSIZ (8192 * 2)
#ifdef __cplusplus
extern "C" {
#endif
// inner hash table (ja3 -> security state)
typedef struct ndpi_ja3_info {
char * ja3;
@@ -162,9 +166,10 @@ enum info_type {
INFO_INVALID = 0,
INFO_GENERIC,
INFO_KERBEROS,
INFO_SOFTETHER,
INFO_TIVOCONNECT,
INFO_FTP_IMAP_POP_SMTP,
INFO_TLS_QUIC_ALPN_VERSION,
INFO_TLS_QUIC_ALPN_ONLY,
INFO_NATPMP,
};
// flow tracking
@@ -181,7 +186,7 @@ typedef struct ndpi_flow_info {
u_int16_t vlan_id;
ndpi_packet_tunnel tunnel_type;
struct ndpi_flow_struct *ndpi_flow;
char src_name[48], dst_name[48];
char src_name[INET6_ADDRSTRLEN], dst_name[INET6_ADDRSTRLEN];
u_int8_t ip_version;
u_int32_t cwr_count, src2dst_cwr_count, dst2src_cwr_count;
u_int32_t ece_count, src2dst_ece_count, dst2src_ece_count;
@@ -203,6 +208,8 @@ typedef struct ndpi_flow_info {
// result only, not used for flow identification
ndpi_protocol detected_protocol;
ndpi_confidence_t confidence;
u_int16_t num_dissector_calls;
u_int16_t dpi_packets;
// Flow data analysis
pkt_timeval src2dst_last_pkt_time, dst2src_last_pkt_time, flow_last_pkt_time;
@@ -210,36 +217,61 @@ typedef struct ndpi_flow_info {
*pktlen_c_to_s, *pktlen_s_to_c;
enum info_type info_type;
union {
char info[256];
struct {
char alpn[128];
char tls_supported_versions[128];
} tls_quic;
struct {
unsigned char auth_failed;
char username[127];
char password[128];
} ftp_imap_pop_smtp;
struct {
char domain[85];
char hostname[85];
char username[86];
} kerberos;
struct {
char ip[16];
char port[6];
char hostname[48];
char fqdn[48];
} softether;
struct {
char identity_uuid[36];
char machine[48];
char platform[32];
char services[48];
} tivoconnect;
struct {
uint16_t result_code;
uint16_t internal_port;
uint16_t external_port;
char ip[16];
} natpmp;
};
char flow_extra_info[16];
ndpi_serializer ndpi_flow_serializer;
char host_server_name[80]; /* Hostname/SNI */
char *bittorent_hash;
char *dhcp_fingerprint;
char *dhcp_class_ident;
ndpi_risk risk;
struct {
char currency[16];
} mining;
struct {
u_int16_t ssl_version;
char server_info[64],
client_hassh[33], server_hassh[33], *server_names,
*tls_alpn, *tls_supported_versions,
*advertised_alpns, *negotiated_alpn, *tls_supported_versions,
*tls_issuerDN, *tls_subjectDN,
ja3_client[33], ja3_server[33],
sha1_cert_fingerprint[20];
@@ -251,13 +283,19 @@ typedef struct ndpi_flow_info {
char *esni;
} encrypted_sni;
struct {
u_int16_t version;
} encrypted_ch;
time_t notBefore, notAfter;
u_int16_t server_cipher;
ndpi_cipher_weakness client_unsafe_cipher, server_unsafe_cipher;
u_int32_t quic_version;
} ssh_tls;
struct {
char url[256], request_content_type[64], content_type[64], user_agent[256];
char url[256], request_content_type[64], content_type[64], user_agent[256], server[128], nat_ip[32], filename[256];
u_int response_status_code;
} http;
@@ -265,6 +303,12 @@ typedef struct ndpi_flow_info {
char *username, *password;
} telnet;
struct {
char geolocation_iata_code[4];
} dns;
ndpi_multimedia_flow_type multimedia_flow_type;
void *src_id, *dst_id;
struct ndpi_entropy *entropy;
@@ -276,6 +320,10 @@ typedef struct ndpi_flow_info {
#else
struct ndpi_bin payload_len_bin;
#endif
/* Flow payload */
u_int16_t flow_payload_len;
char *flow_payload;
} ndpi_flow_info_t;
@@ -296,6 +344,11 @@ typedef struct ndpi_stats {
u_int16_t max_packet_len;
u_int64_t dpi_packet_count[3];
u_int64_t flow_confidence[NDPI_CONFIDENCE_MAX];
u_int64_t num_dissector_calls;
struct ndpi_lru_cache_stats lru_stats[NDPI_LRUCACHE_MAX];
struct ndpi_automa_stats automa_stats[NDPI_AUTOMA_MAX];
struct ndpi_patricia_tree_stats patricia_stats[NDPI_PTREE_MAX];
} ndpi_stats_t;
@@ -321,11 +374,6 @@ typedef struct ndpi_workflow {
struct ndpi_workflow_prefs prefs;
struct ndpi_stats stats;
ndpi_workflow_callback_ptr __flow_detected_callback;
void * __flow_detected_udata;
ndpi_workflow_callback_ptr __flow_giveup_callback;
void * __flow_giveup_udata;
/* outside referencies */
pcap_t *pcap_handle;
@@ -335,7 +383,7 @@ typedef struct ndpi_workflow {
u_int32_t num_allocated_flows;
/* CSV,TLV,JSON serialization interface */
ndpi_serializer ndpi_serializer;
ndpi_serialization_format ndpi_serialization_format;
} ndpi_workflow_t;
@@ -362,32 +410,17 @@ struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
int ndpi_is_datalink_supported(int datalink_type);
/* flow callbacks for complete detected flow
(ndpi_flow_info will be freed right after) */
static inline void ndpi_workflow_set_flow_detected_callback(struct ndpi_workflow * workflow, ndpi_workflow_callback_ptr callback, void * udata) {
workflow->__flow_detected_callback = callback;
workflow->__flow_detected_udata = udata;
}
/* flow callbacks for sufficient detected flow
(ndpi_flow_info will be freed right after) */
static inline void ndpi_workflow_set_flow_giveup_callback(struct ndpi_workflow * workflow, ndpi_workflow_callback_ptr callback, void * udata) {
workflow->__flow_giveup_callback = callback;
workflow->__flow_giveup_udata = udata;
}
/* compare two nodes in workflow */
int ndpi_workflow_node_cmp(const void *a, const void *b);
void process_ndpi_collected_info(struct ndpi_workflow * workflow, struct ndpi_flow_info *flow);
u_int32_t ethernet_crc32(const void* data, size_t n_bytes);
void ndpi_flow_info_free_data(struct ndpi_flow_info *flow);
void ndpi_flow_info_freer(void *node);
const char* print_cipher_id(u_int32_t cipher);
float ndpi_flow_get_byte_count_entropy(const uint32_t byte_count[256], unsigned int num_bytes);
int parse_proto_name_list(char *str, NDPI_PROTOCOL_BITMASK *bitmask, int inverted_logic);
extern int nDPI_LogLevel;
#ifdef NDPI_ENABLE_DEBUG_MESSAGES
#if defined(NDPI_ENABLE_DEBUG_MESSAGES) && !defined(FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION)
#define LOG(log_level, args...) \
{ \
if(log_level <= nDPI_LogLevel) \
@@ -397,4 +430,12 @@ extern int nDPI_LogLevel;
#define LOG(...) {}
#endif
#ifndef LINKTYPE_LINUX_SLL2
#define LINKTYPE_LINUX_SLL2 276
#endif
#ifdef __cplusplus
}
#endif
#endif
+655 -31
View File
@@ -1,52 +1,676 @@
bin_PROGRAMS = fuzz_process_packet fuzz_process_packet_with_main fuzz_ndpi_reader fuzz_ndpi_reader_with_main
bin_PROGRAMS = fuzz_process_packet fuzz_ndpi_reader fuzz_ndpi_reader_alloc_fail fuzz_ndpi_reader_payload_analyzer fuzz_quic_get_crypto_data fuzz_config fuzz_community_id fuzz_serialization fuzz_tls_certificate fuzz_dga
#Alghoritms
bin_PROGRAMS += fuzz_alg_bins fuzz_alg_hll fuzz_alg_hw_rsi_outliers_da fuzz_alg_jitter fuzz_alg_ses_des fuzz_alg_crc32_md5 fuzz_alg_bytestream
#Data structures
bin_PROGRAMS += fuzz_ds_patricia fuzz_ds_ahocorasick fuzz_ds_libcache fuzz_ds_tree fuzz_ds_ptree fuzz_ds_hash fuzz_ds_cmsketch fuzz_ds_bitmap64 fuzz_ds_domain_classify
#Third party
bin_PROGRAMS += fuzz_libinjection fuzz_binaryfusefilter
#Internal crypto
bin_PROGRAMS += fuzz_gcrypt_light fuzz_gcrypt_aes fuzz_gcrypt_gcm fuzz_gcrypt_cipher
#Configuration files
bin_PROGRAMS += fuzz_filecfg_protocols
#Reader utils
bin_PROGRAMS += fuzz_readerutils_workflow fuzz_readerutils_parseprotolist
fuzz_process_packet_SOURCES = fuzz_process_packet.c
fuzz_process_packet_CFLAGS =
fuzz_process_packet_LDADD = ../src/lib/libndpi.a
fuzz_process_packet_LDFLAGS = $(ADDITIONAL_LIBS) $(LIBS)
fuzz_process_packet_SOURCES = fuzz_process_packet.c fuzz_common_code.c
fuzz_process_packet_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_process_packet_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_process_packet_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_process_packet_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_process_packet_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_process_packet_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_process_packet_LDFLAGS) $(LDFLAGS) -o $@
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_process_packet_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_process_packet_with_main_SOURCES = fuzz_process_packet.c
fuzz_process_packet_with_main_CFLAGS = -DBUILD_MAIN
fuzz_process_packet_with_main_LDADD = ../src/lib/libndpi.a
fuzz_process_packet_with_main_LDFLAGS = $(ADDITIONAL_LIBS) $(LIBS)
# force usage of CXX for linker
fuzz_process_packet_with_main_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_process_packet_with_main_LDFLAGS) $(LDFLAGS) -o $@
fuzz_ndpi_reader_SOURCES = fuzz_ndpi_reader.c
fuzz_ndpi_reader_CFLAGS = -I../example/
fuzz_ndpi_reader_LDADD = ../example/libndpiReader.a ../src/lib/libndpi.a
fuzz_ndpi_reader_LDFLAGS = $(PCAP_LIB) $(ADDITIONAL_LIBS) $(LIBS)
fuzz_ndpi_reader_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c
fuzz_ndpi_reader_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ndpi_reader_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_LDFLAGS) $(LDFLAGS) -o $@
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_with_main_SOURCES = fuzz_ndpi_reader.c
fuzz_ndpi_reader_with_main_CFLAGS = -I../example/ -DBUILD_MAIN
fuzz_ndpi_reader_with_main_LDADD = ../src/lib/libndpi.a
fuzz_ndpi_reader_with_main_LDFLAGS = ../example/libndpiReader.a $(PCAP_LIB) $(ADDITIONAL_LIBS) $(LIBS)
fuzz_ndpi_reader_alloc_fail_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c
fuzz_ndpi_reader_alloc_fail_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -DENABLE_MEM_ALLOC_FAILURES -DCRYPT_FORCE_NO_AESNI
fuzz_ndpi_reader_alloc_fail_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_alloc_fail_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_alloc_fail_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_alloc_fail_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_with_main_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_with_main_LDFLAGS) $(LDFLAGS) -o $@
fuzz_ndpi_reader_alloc_fail_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_alloc_fail_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ndpi_reader_payload_analyzer_SOURCES = fuzz_ndpi_reader.c fuzz_common_code.c ../example/reader_util.c
fuzz_ndpi_reader_payload_analyzer_CFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -DENABLE_MEM_ALLOC_FAILURES -DENABLE_PAYLOAD_ANALYZER
fuzz_ndpi_reader_payload_analyzer_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ndpi_reader_payload_analyzer_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_payload_analyzer_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_payload_analyzer_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ndpi_reader_payload_analyzer_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ndpi_reader_payload_analyzer_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_quic_get_crypto_data_SOURCES = fuzz_quic_get_crypto_data.c fuzz_common_code.c
fuzz_quic_get_crypto_data_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_quic_get_crypto_data_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_quic_get_crypto_data_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_quic_get_crypto_data_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_quic_get_crypto_data_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_quic_get_crypto_data_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_quic_get_crypto_data_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_config_SOURCES = fuzz_config.cpp fuzz_common_code.c
fuzz_config_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_config_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_config_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_config_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_config_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_config_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_config_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_config_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_config_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_community_id_SOURCES = fuzz_community_id.cpp fuzz_common_code.c
fuzz_community_id_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_community_id_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_community_id_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_community_id_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_community_id_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_community_id_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_community_id_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_community_id_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_community_id_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_serialization_SOURCES = fuzz_serialization.cpp fuzz_common_code.c
fuzz_serialization_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_serialization_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_serialization_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_serialization_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_serialization_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_serialization_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_serialization_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_serialization_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_serialization_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_bins_SOURCES = fuzz_alg_bins.cpp fuzz_common_code.c
fuzz_alg_bins_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_bins_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_bins_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_bins_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_bins_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_bins_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_bins_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_bins_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_bins_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_hll_SOURCES = fuzz_alg_hll.cpp fuzz_common_code.c
fuzz_alg_hll_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_hll_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_hll_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_hll_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_hll_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_hll_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_hll_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_hll_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_hll_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_hw_rsi_outliers_da_SOURCES = fuzz_alg_hw_rsi_outliers_da.cpp fuzz_common_code.c
fuzz_alg_hw_rsi_outliers_da_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_hw_rsi_outliers_da_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_hw_rsi_outliers_da_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_hw_rsi_outliers_da_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_hw_rsi_outliers_da_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_hw_rsi_outliers_da_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_hw_rsi_outliers_da_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_hw_rsi_outliers_da_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_hw_rsi_outliers_da_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_jitter_SOURCES = fuzz_alg_jitter.cpp fuzz_common_code.c
fuzz_alg_jitter_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_jitter_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_jitter_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_jitter_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_jitter_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_jitter_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_jitter_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_jitter_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_jitter_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_crc32_md5_SOURCES = fuzz_alg_crc32_md5.c
fuzz_alg_crc32_md5_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_crc32_md5_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_crc32_md5_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_crc32_md5_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_crc32_md5_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_crc32_md5_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_crc32_md5_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_bytestream_SOURCES = fuzz_alg_bytestream.c
fuzz_alg_bytestream_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_bytestream_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_bytestream_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_bytestream_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_bytestream_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_bytestream_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_bytestream_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_alg_ses_des_SOURCES = fuzz_alg_ses_des.cpp fuzz_common_code.c
fuzz_alg_ses_des_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_ses_des_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_alg_ses_des_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_alg_ses_des_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_alg_ses_des_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_ses_des_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_alg_ses_des_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_alg_ses_des_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_alg_ses_des_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_patricia_SOURCES = fuzz_ds_patricia.cpp fuzz_common_code.c
fuzz_ds_patricia_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_patricia_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_patricia_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_patricia_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_patricia_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_patricia_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_patricia_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_patricia_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_patricia_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_ahocorasick_SOURCES = fuzz_ds_ahocorasick.cpp fuzz_common_code.c
fuzz_ds_ahocorasick_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_ahocorasick_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_ahocorasick_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_ahocorasick_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_ahocorasick_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_ahocorasick_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_ahocorasick_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_ahocorasick_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_ahocorasick_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_libcache_SOURCES = fuzz_ds_libcache.cpp fuzz_common_code.c
fuzz_ds_libcache_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_libcache_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_libcache_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_libcache_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_libcache_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_libcache_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_libcache_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_libcache_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_libcache_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_tree_SOURCES = fuzz_ds_tree.cpp fuzz_common_code.c
fuzz_ds_tree_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_tree_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_tree_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_tree_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_tree_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_tree_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_tree_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_tree_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_tree_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_ptree_SOURCES = fuzz_ds_ptree.cpp fuzz_common_code.c
fuzz_ds_ptree_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_ptree_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_ptree_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_ptree_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_ptree_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_ptree_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_ptree_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_ptree_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_ptree_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_hash_SOURCES = fuzz_ds_hash.cpp fuzz_common_code.c
fuzz_ds_hash_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_hash_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_hash_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_hash_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_hash_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_hash_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_hash_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_hash_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_hash_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_cmsketch_SOURCES = fuzz_ds_cmsketch.cpp fuzz_common_code.c
fuzz_ds_cmsketch_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_cmsketch_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_cmsketch_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_cmsketch_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_cmsketch_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_cmsketch_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_cmsketch_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_cmsketch_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_cmsketch_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_bitmap64_SOURCES = fuzz_ds_bitmap64.cpp fuzz_common_code.c
fuzz_ds_bitmap64_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_bitmap64_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_bitmap64_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_bitmap64_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_bitmap64_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_bitmap64_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_bitmap64_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_bitmap64_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_bitmap64_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_ds_domain_classify_SOURCES = fuzz_ds_domain_classify.cpp fuzz_common_code.c
fuzz_ds_domain_classify_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_domain_classify_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_ds_domain_classify_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_ds_domain_classify_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ds_domain_classify_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_domain_classify_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ds_domain_classify_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_ds_domain_classify_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_ds_domain_classify_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_libinjection_SOURCES = fuzz_libinjection.c
fuzz_libinjection_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_libinjection_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_libinjection_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_libinjection_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_libinjection_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_libinjection_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_libinjection_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_binaryfusefilter_SOURCES = fuzz_binaryfusefilter.cpp fuzz_common_code.c
fuzz_binaryfusefilter_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_binaryfusefilter_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_binaryfusefilter_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_binaryfusefilter_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_binaryfusefilter_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_binaryfusefilter_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_binaryfusefilter_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_binaryfusefilter_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_binaryfusefilter_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_tls_certificate_SOURCES = fuzz_tls_certificate.c fuzz_common_code.c
fuzz_tls_certificate_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_tls_certificate_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_tls_certificate_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_tls_certificate_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_tls_certificate_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_tls_certificate_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_tls_certificate_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_dga_SOURCES = fuzz_dga.c fuzz_common_code.c
fuzz_dga_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_dga_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_dga_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_dga_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_dga_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_dga_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_dga_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_gcrypt_light_SOURCES = fuzz_gcrypt_light.cpp fuzz_common_code.c
fuzz_gcrypt_light_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_light_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_light_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_gcrypt_light_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_gcrypt_light_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_light_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_light_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_gcrypt_light_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_gcrypt_light_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_gcrypt_aes_SOURCES = fuzz_gcrypt_aes.cpp
fuzz_gcrypt_aes_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_aes_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_aes_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_gcrypt_aes_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_gcrypt_aes_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_aes_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_aes_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_gcrypt_aes_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_gcrypt_aes_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_gcrypt_gcm_SOURCES = fuzz_gcrypt_gcm.cpp
fuzz_gcrypt_gcm_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_gcm_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_gcm_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_gcrypt_gcm_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_gcrypt_gcm_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_gcm_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_gcm_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_gcrypt_gcm_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_gcrypt_gcm_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_gcrypt_cipher_SOURCES = fuzz_gcrypt_cipher.cpp
fuzz_gcrypt_cipher_CXXFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_cipher_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -I ../src/lib/third_party/include/
fuzz_gcrypt_cipher_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_gcrypt_cipher_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_gcrypt_cipher_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_cipher_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_gcrypt_cipher_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_gcrypt_cipher_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_gcrypt_cipher_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_filecfg_protocols_SOURCES = fuzz_filecfg_protocols.c fuzz_common_code.c
fuzz_filecfg_protocols_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_filecfg_protocols_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_filecfg_protocols_LDFLAGS = $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_filecfg_protocols_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_filecfg_protocols_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_filecfg_protocols_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_filecfg_protocols_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_readerutils_workflow_SOURCES = fuzz_readerutils_workflow.cpp fuzz_common_code.c ../example/reader_util.c
fuzz_readerutils_workflow_CXXFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS) -DDISABLE_CUSTOM_ALLOCATOR_ON_READERUTILS
fuzz_readerutils_workflow_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS) -DDISABLE_CUSTOM_ALLOCATOR_ON_READERUTILS
fuzz_readerutils_workflow_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_readerutils_workflow_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_readerutils_workflow_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_readerutils_workflow_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_readerutils_workflow_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_readerutils_workflow_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_readerutils_workflow_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
fuzz_readerutils_parseprotolist_SOURCES = fuzz_readerutils_parseprotolist.cpp fuzz_common_code.c ../example/reader_util.c
fuzz_readerutils_parseprotolist_CXXFLAGS = -I../example/ @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_readerutils_parseprotolist_CFLAGS = @NDPI_CFLAGS@ $(CXXFLAGS)
fuzz_readerutils_parseprotolist_LDADD = ../src/lib/libndpi.a $(ADDITIONAL_LIBS)
fuzz_readerutils_parseprotolist_LDFLAGS = $(PCAP_LIB) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_readerutils_parseprotolist_CXXFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_readerutils_parseprotolist_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_readerutils_parseprotolist_LDFLAGS += $(LIB_FUZZING_ENGINE)
endif
# force usage of CXX for linker
fuzz_readerutils_parseprotolist_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) @NDPI_CFLAGS@ $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_readerutils_parseprotolist_LDFLAGS) @NDPI_LDFLAGS@ $(LDFLAGS) -o $@
# required for Google oss-fuzz
# see https://github.com/google/oss-fuzz/tree/master/projects/ndpi
testpcaps := $(wildcard ../tests/pcap/*.pcap*)
testpcaps := $(wildcard ../tests/cfgs/default/pcap/*.pcap*)
fuzz_ndpi_reader_seed_corpus.zip: $(testpcaps)
zip -r fuzz_ndpi_reader_seed_corpus.zip $(testpcaps)
zip -j fuzz_ndpi_reader_seed_corpus.zip $(testpcaps)
fuzz_ndpi_reader_alloc_fail_seed_corpus.zip: $(testpcaps)
zip -j fuzz_ndpi_reader_alloc_fail_seed_corpus.zip $(testpcaps)
fuzz_ndpi_reader_payload_analyzer_seed_corpus.zip: $(testpcaps)
zip -j fuzz_ndpi_reader_payload_analyzer_seed_corpus.zip $(testpcaps)
files_corpus_fuzz_quic_get_crypto_data := $(wildcard corpus/fuzz_quic_get_crypto_data/*)
fuzz_quic_get_crypto_data_seed_corpus.zip: $(files_corpus_fuzz_quic_get_crypto_data)
zip -j fuzz_quic_get_crypto_data_seed_corpus.zip $(files_corpus_fuzz_quic_get_crypto_data)
files_corpus_fuzz_config := $(wildcard corpus/fuzz_config/*)
fuzz_config_seed_corpus.zip: $(files_corpus_fuzz_config)
zip -j fuzz_config_seed_corpus.zip $(files_corpus_fuzz_config)
files_corpus_fuzz_serialization := $(wildcard corpus/fuzz_serialization/*)
fuzz_serialization_seed_corpus.zip: $(files_corpus_fuzz_serialization)
zip -j fuzz_serialization_seed_corpus.zip $(files_corpus_fuzz_serialization)
files_corpus_fuzz_community_id := $(wildcard corpus/fuzz_community_id/*)
fuzz_community_id_seed_corpus.zip: $(files_corpus_fuzz_community_id)
zip -j fuzz_community_id_seed_corpus.zip $(files_corpus_fuzz_community_id)
files_corpus_fuzz_ds_patricia := $(wildcard corpus/fuzz_ds_patricia/*)
fuzz_ds_patricia_seed_corpus.zip: $(files_corpus_fuzz_ds_patricia)
zip -j fuzz_ds_patricia_seed_corpus.zip $(files_corpus_fuzz_ds_patricia)
files_corpus_fuzz_ds_ahocorasick := $(wildcard corpus/fuzz_ds_ahocorasick/*)
fuzz_ds_ahocorasick_seed_corpus.zip: $(files_corpus_fuzz_ds_ahocorasick)
zip -j fuzz_ds_ahocorasick_seed_corpus.zip $(files_corpus_fuzz_ds_ahocorasick)
files_corpus_fuzz_alg_ses_des := $(wildcard corpus/fuzz_alg_ses_des/*)
fuzz_alg_ses_des_seed_corpus.zip: $(files_corpus_fuzz_alg_ses_des)
zip -j fuzz_alg_ses_des_seed_corpus.zip $(files_corpus_fuzz_alg_ses_des)
files_corpus_fuzz_alg_hw_rsi_outliers_da := $(wildcard corpus/fuzz_alg_hw_rsi_outliers_da/*)
fuzz_alg_hw_rsi_outliers_da_seed_corpus.zip: $(files_corpus_fuzz_alg_hw_rsi_outliers_da)
zip -j fuzz_alg_hw_rsi_outliers_da_seed_corpus.zip $(files_corpus_fuzz_alg_hw_rsi_outliers_da)
files_corpus_fuzz_alg_bins := $(wildcard corpus/fuzz_alg_bins/*)
fuzz_alg_bins_seed_corpus.zip: $(files_corpus_fuzz_alg_bins)
zip -j fuzz_alg_bins_seed_corpus.zip $(files_corpus_fuzz_alg_bins)
files_corpus_fuzz_alg_hll := $(wildcard corpus/fuzz_alg_hll/*)
fuzz_alg_hll_seed_corpus.zip: $(files_corpus_fuzz_alg_hll)
zip -j fuzz_alg_hll_seed_corpus.zip $(files_corpus_fuzz_alg_hll)
files_corpus_fuzz_alg_jitter := $(wildcard corpus/fuzz_alg_jitter/*)
fuzz_alg_jitter_seed_corpus.zip: $(files_corpus_fuzz_alg_jitter)
zip -j fuzz_alg_jitter_seed_corpus.zip $(files_corpus_fuzz_alg_jitter)
files_corpus_fuzz_alg_crc32_md5 := $(wildcard corpus/fuzz_alg_crc32_md5/*)
fuzz_alg_crc32_md5_seed_corpus.zip: $(files_corpus_fuzz_alg_crc32_md5)
zip -j fuzz_alg_crc32_md5_seed_corpus.zip $(files_corpus_fuzz_alg_crc32_md5)
files_corpus_fuzz_ds_libcache := $(wildcard corpus/fuzz_ds_libcache/*)
fuzz_ds_libcache_seed_corpus.zip: $(files_corpus_fuzz_ds_libcache)
zip -j fuzz_ds_libcache_seed_corpus.zip $(files_corpus_fuzz_ds_libcache)
files_corpus_fuzz_ds_tree := $(wildcard corpus/fuzz_ds_tree/*)
fuzz_ds_tree_seed_corpus.zip: $(files_corpus_fuzz_ds_tree)
zip -j fuzz_ds_tree_seed_corpus.zip $(files_corpus_fuzz_ds_tree)
files_corpus_fuzz_ds_ptree := $(wildcard corpus/fuzz_ds_ptree/*)
fuzz_ds_ptree_seed_corpus.zip: $(files_corpus_fuzz_ds_ptree)
zip -j fuzz_ds_ptree_seed_corpus.zip $(files_corpus_fuzz_ds_ptree)
files_corpus_fuzz_alg_bytestream := $(wildcard corpus/fuzz_alg_bytestream/*)
fuzz_alg_bytestream_seed_corpus.zip: $(files_corpus_fuzz_alg_bytestream)
zip -j fuzz_alg_bytestream_seed_corpus.zip $(files_corpus_fuzz_alg_bytestream)
files_corpus_fuzz_libinjection := $(wildcard corpus/fuzz_libinjection/*)
fuzz_libinjection_seed_corpus.zip: $(files_corpus_fuzz_libinjection)
zip -j fuzz_libinjection_seed_corpus.zip $(files_corpus_fuzz_libinjection)
files_corpus_fuzz_tls_certificate := $(wildcard corpus/fuzz_tls_certificate/*)
fuzz_tls_certificate_seed_corpus.zip: $(files_corpus_fuzz_tls_certificate)
zip -j fuzz_tls_certificate_seed_corpus.zip $(files_corpus_fuzz_tls_certificate)
files_corpus_fuzz_filecfg_protocols := $(wildcard corpus/fuzz_filecfg_protocols/*)
fuzz_filecfg_protocols_seed_corpus.zip: $(files_corpus_fuzz_filecfg_protocols)
zip -j fuzz_filecfg_protocols_seed_corpus.zip $(files_corpus_fuzz_filecfg_protocols)
files_corpus_fuzz_readerutils_workflow := $(wildcard corpus/fuzz_readerutils_workflow/*)
fuzz_readerutils_workflow_seed_corpus.zip: $(files_corpus_fuzz_readerutils_workflow)
zip -j fuzz_readerutils_workflow_seed_corpus.zip $(files_corpus_fuzz_readerutils_workflow)
files_corpus_fuzz_readerutils_parseprotolist := $(wildcard corpus/fuzz_readerutils_parseprotolist/*)
fuzz_readerutils_parseprotolist_seed_corpus.zip: $(files_corpus_fuzz_readerutils_parseprotolist)
zip -j fuzz_readerutils_parseprotolist_seed_corpus.zip $(files_corpus_fuzz_readerutils_parseprotolist)
files_corpus_fuzz_ds_bitmap64 := $(wildcard corpus/fuzz_ds_bitmap64/*)
fuzz_ds_bitmap64_seed_corpus.zip: $(files_corpus_fuzz_ds_bitmap64)
zip -j fuzz_ds_bitmap64_seed_corpus.zip $(files_corpus_fuzz_ds_bitmap64)
files_corpus_fuzz_ds_domain_classify := $(wildcard corpus/fuzz_ds_domain_classify/*)
fuzz_ds_domain_classify_seed_corpus.zip: $(files_corpus_fuzz_ds_domain_classify)
zip -j fuzz_ds_domain_classify_seed_corpus.zip $(files_corpus_fuzz_ds_domain_classify)
corpus: fuzz_ndpi_reader_seed_corpus.zip fuzz_ndpi_reader_alloc_fail_seed_corpus.zip fuzz_ndpi_reader_payload_analyzer_seed_corpus.zip fuzz_quic_get_crypto_data_seed_corpus.zip fuzz_config_seed_corpus.zip fuzz_ds_patricia_seed_corpus.zip fuzz_ds_ahocorasick_seed_corpus.zip fuzz_alg_ses_des_seed_corpus.zip fuzz_alg_hw_rsi_outliers_da_seed_corpus.zip fuzz_alg_bins_seed_corpus.zip fuzz_alg_hll_seed_corpus.zip fuzz_alg_jitter_seed_corpus.zip fuzz_ds_libcache_seed_corpus.zip fuzz_community_id_seed_corpus.zip fuzz_ds_tree_seed_corpus.zip fuzz_serialization_seed_corpus.zip fuzz_ds_ptree_seed_corpus.zip fuzz_alg_crc32_md5_seed_corpus.zip fuzz_alg_bytestream_seed_corpus.zip fuzz_libinjection_seed_corpus.zip fuzz_tls_certificate_seed_corpus.zip fuzz_filecfg_protocols_seed_corpus.zip fuzz_readerutils_workflow_seed_corpus.zip fuzz_readerutils_parseprotolist_seed_corpus.zip fuzz_ds_bitmap64_seed_corpus.zip fuzz_ds_domain_classify_seed_corpus.zip
cp corpus/fuzz_*seed_corpus.zip .
#Create dictionaries exactly as expected by oss-fuzz.
#This way, if we need to change/update/add something,
#we don't need to update scripts in oss-fuzz repository
dictionaries:
cp dictionary.dict fuzz_ndpi_reader.dict
cp dictionary.dict fuzz_ndpi_reader_alloc_fail.dict
cp dictionary.dict fuzz_ndpi_reader_payload_analyzer.dict
cp dictionary.dict fuzz_process_packet.dict
cp dictionary_tls_certificate.dict fuzz_tls_certificate.dict
distdir:
find . -type d | xargs -I'{}' mkdir -p '$(distdir)/{}'
find . -type f -name '*.c' \
-o -name '*.am' \
-o -name '*.h' \
-o -name '*.cpp' \
-o -name '*.options' \
-o -name 'ipv4_addresses.txt' \
-o -name 'bd_param.txt' \
-o -name 'splt_param.txt' \
-o -name 'random_list.list' \
-o -path './dictionary.dict' \
-o -path './dictionary_tls_certificate.dict' \
-o -path './corpus/fuzz_*.zip' \
-o -path './corpus/fuzz_quic_get_crypto_data/*' \
-o -path './corpus/fuzz_filecfg_protocols/*' \
-o -path './corpus/fuzz_readerutils_workflow/*' \
-o -path './corpus/fuzz_readerutils_parseprotolist/*' \
-o -path './corpus/fuzz_config/*' \
-o -path './corpus/fuzz_serialization/*' \
-o -path './corpus/fuzz_community_id/*' \
-o -path './corpus/fuzz_libinjection/*' \
-o -path './corpus/fuzz_tls_certificate/*' \
-o -path './corpus/fuzz_alg_ses_des/*' \
-o -path './corpus/fuzz_alg_bins/*' \
-o -path './corpus/fuzz_alg_hll/*' \
-o -path './corpus/fuzz_alg_jitter/*' \
-o -path './corpus/fuzz_alg_crc32_md5/*' \
-o -path './corpus/fuzz_alg_hw_rsi_outliers_da/*' \
-o -path './corpus/fuzz_alg_bytestream/*' \
-o -path './corpus/fuzz_ds_ahocorasick/*' \
-o -path './corpus/fuzz_ds_libcache/*' \
-o -path './corpus/fuzz_ds_tree/*' \
-o -path './corpus/fuzz_ds_bitmap64/*' \
-o -path './corpus/fuzz_ds_domain_classify/*' \
-o -path './corpus/fuzz_ds_ptree/*' \
-o -path './corpus/fuzz_ds_patricia/*' | xargs -I'{}' cp -r '{}' '$(distdir)/{}'
all: corpus dictionaries
+627
View File
@@ -0,0 +1,627 @@
-2.088057846500587456e+00
7.763936238952200239e-05
4.404309737393306595e-05
-9.467385027293546973e-02
4.348947142638090457e-01
-2.091409170053043390e-04
-5.788902107267982974e-04
4.481443450852441001e-10
-3.136135459023654537e+00
-1.507730262127600751e+00
-1.204663669965535977e+00
-1.171839254318371104e+00
4.329302247232582057e-01
8.310653628092458334e+00
3.299246725156660176e+00
0.000000000000000000e+00
1.847454931582027254e-02
-1.498024139966201096e+00
-7.660670007653060942e-01
-2.908130300830076731e+00
-1.252564844610269734e+00
-1.910955328742287573e+00
9.471710980110392697e-01
2.352302758516665371e+00
2.982269972214651954e+00
4.280736383314343918e+00
4.633629909719495288e+00
-2.198052637823726840e+00
-1.150759637168392580e+00
3.420433363184381292e+00
1.857878113059351077e-02
-1.559806674919653746e+00
4.197498183183401288e+00
6.262186949633183453e+00
1.100694844370524095e+01
2.778688785515088000e+01
3.679948298336883195e+00
-2.432801394376875592e+00
5.133442052706843617e-01
2.181172654073517680e+00
-8.577551729671881731e-01
7.013844214023315926e-01
3.138233436228588857e+00
7.319940508466630247e-01
0.000000000000000000e+00
3.529209394581482861e+00
1.464585117707144413e+01
8.506550226820598359e-01
-9.060397326548508268e-01
6.787474954688997641e+00
8.125411068867387954e+00
4.515740684104064151e+00
5.372135582950940069e+00
9.210951196799497254e-01
4.802177410869620466e+00
2.945445016176073594e+01
1.575032253128311632e+00
-1.355276854364796946e-01
-3.322474764169629502e-01
3.018397817188666732e+00
1.186503569922195744e+00
0.000000000000000000e+00
8.883242370198487503e-01
7.248276146728496627e+00
0.000000000000000000e+00
0.000000000000000000e+00
-4.831246718433664711e+00
6.124136970173365002e-01
4.145693892559814686e-01
2.683998941637626867e+00
2.063906603639539039e+00
2.989801217386735210e+00
2.262965767379551962e-01
2.240332214649647380e+00
5.984550782416063086e+00
4.587011255338186544e+00
1.233118485315272039e+01
1.115223490909697857e+00
-3.682686422016995476e+00
6.096498453291562258e-01
1.119275528656461516e+00
1.377886278915177731e-01
3.828176805973048324e+00
0.000000000000000000e+00
0.000000000000000000e+00
1.442927634029647344e+01
0.000000000000000000e+00
5.719118583309401593e-01
1.993632609731877392e-01
3.047472271520709430e+00
5.736784864911910198e+00
6.677826247219391220e+00
6.307175478564531090e+00
3.150295169417364249e+01
3.738597740702392258e+00
1.129754590514236234e+01
6.108506268573830056e+00
1.605489516792866667e+00
2.929631990348545489e+00
-2.832543082245212937e-02
1.358286530670594461e+00
1.655932469853677924e+00
6.701964773769768513e-01
2.131182050917533211e+00
2.998351165769753468e+00
7.772095996358327596e+00
1.285014785269981141e+00
4.407334784589962418e+00
1.719858214230612026e+00
-1.012765674651314063e+00
-5.749271123172469133e-01
-3.559614093795681278e+00
-3.073088477387719397e+00
-4.492469521371540431e+00
-3.753286990415885427e+00
-3.219255423324282273e+00
-2.806436518181075090e+00
-2.697305948568419875e+00
-7.879608430851776646e-01
4.625507221739111330e+00
4.809280703883450414e+00
-3.435194026629848629e+00
-3.218943068168937049e+00
3.335535704890596698e+00
2.071359212435486263e+00
4.538992059175040339e+00
-2.770772323566738038e+01
2.903047708571506735e+00
-4.436143805989154032e+00
-2.647991280011542381e-01
1.737252348126810064e+00
-4.121989655995259128e+00
3.209709099445720581e-01
1.012758514896711759e+01
3.313255624721038295e+00
4.631467619785444967e+00
7.668642402146534032e+00
6.780938812710099128e+00
-3.256164342602652972e+00
6.749565128319576779e-01
0.000000000000000000e+00
-4.407265954524525853e+00
0.000000000000000000e+00
-3.666522115024547901e+01
-7.886029397826226273e+01
0.000000000000000000e+00
0.000000000000000000e+00
-2.261283814517791058e+01
-4.024317426178160240e+00
3.213063737030031342e-01
5.079805145796887800e+00
1.326813226475260343e+00
1.233684078112145643e+00
8.671852503871454232e+00
-2.041800256066371944e+00
0.000000000000000000e+00
0.000000000000000000e+00
-1.607347800380474823e+01
-4.430790279223246309e+00
1.177552465851384511e+00
6.342921220500139512e+00
-2.466913734548706327e-02
3.451642566010713065e-01
-6.012767168531006234e+00
7.328146570137336724e+00
7.500088131707050465e+00
0.000000000000000000e+00
-3.547913249211809017e+01
-3.130964814607208879e+00
8.247326544297072237e-01
3.757262485775580418e-01
-2.136528302027558723e+00
-2.631627236037529793e-01
-2.016718799388414141e+01
0.000000000000000000e+00
0.000000000000000000e+00
0.000000000000000000e+00
-7.708602132869285528e-01
-2.602868328868111814e+00
1.435184800833797958e+00
0.000000000000000000e+00
-2.080420864280113413e+00
1.169498351211070819e+00
-1.798334115637199560e+01
-1.193885252696202670e+01
0.000000000000000000e+00
0.000000000000000000e+00
4.304089297965300709e+00
-3.020893216686394656e+00
-1.234427481614708721e+00
0.000000000000000000e+00
1.853340741926325697e+00
-2.686000064995862147e+01
-1.672275139058893600e+01
-2.826268691607605987e+01
0.000000000000000000e+00
0.000000000000000000e+00
-1.547397429377200817e+00
-4.018181657009961327e+00
-7.289186736637049968e+00
-7.458655219230571731e+00
-9.625538282761622710e+00
-1.103039457077456298e+01
-6.262675161142102809e+01
-9.265912629799268885e+00
-8.961543476816615339e+00
-9.622764435629340696e+00
-1.097978292092879826e+01
-1.097978292092879826e+01
-2.088057846500587456e+00
7.763936238952200239e-05
4.404309737393306595e-05
-9.467385027293546973e-02
4.348947142638090457e-01
-2.091409170053043390e-04
-5.788902107267982974e-04
4.481443450852441001e-10
-3.136135459023654537e+00
-1.507730262127600751e+00
-1.204663669965535977e+00
-1.171839254318371104e+00
4.329302247232582057e-01
8.310653628092458334e+00
3.299246725156660176e+00
0.000000000000000000e+00
1.847454931582027254e-02
-1.498024139966201096e+00
-7.660670007653060942e-01
-2.908130300830076731e+00
-1.252564844610269734e+00
-1.910955328742287573e+00
9.471710980110392697e-01
2.352302758516665371e+00
2.982269972214651954e+00
4.280736383314343918e+00
4.633629909719495288e+00
-2.198052637823726840e+00
-1.150759637168392580e+00
3.420433363184381292e+00
1.857878113059351077e-02
-1.559806674919653746e+00
4.197498183183401288e+00
6.262186949633183453e+00
1.100694844370524095e+01
2.778688785515088000e+01
3.679948298336883195e+00
-2.432801394376875592e+00
5.133442052706843617e-01
2.181172654073517680e+00
-8.577551729671881731e-01
7.013844214023315926e-01
3.138233436228588857e+00
7.319940508466630247e-01
0.000000000000000000e+00
3.529209394581482861e+00
1.464585117707144413e+01
8.506550226820598359e-01
-9.060397326548508268e-01
6.787474954688997641e+00
8.125411068867387954e+00
4.515740684104064151e+00
5.372135582950940069e+00
9.210951196799497254e-01
4.802177410869620466e+00
2.945445016176073594e+01
1.575032253128311632e+00
-1.355276854364796946e-01
-3.322474764169629502e-01
3.018397817188666732e+00
1.186503569922195744e+00
0.000000000000000000e+00
8.883242370198487503e-01
7.248276146728496627e+00
0.000000000000000000e+00
0.000000000000000000e+00
-4.831246718433664711e+00
6.124136970173365002e-01
4.145693892559814686e-01
2.683998941637626867e+00
2.063906603639539039e+00
2.989801217386735210e+00
2.262965767379551962e-01
2.240332214649647380e+00
5.984550782416063086e+00
4.587011255338186544e+00
1.233118485315272039e+01
1.115223490909697857e+00
-3.682686422016995476e+00
6.096498453291562258e-01
1.119275528656461516e+00
1.377886278915177731e-01
3.828176805973048324e+00
0.000000000000000000e+00
0.000000000000000000e+00
1.442927634029647344e+01
0.000000000000000000e+00
5.719118583309401593e-01
1.993632609731877392e-01
3.047472271520709430e+00
5.736784864911910198e+00
6.677826247219391220e+00
6.307175478564531090e+00
3.150295169417364249e+01
3.738597740702392258e+00
1.129754590514236234e+01
6.108506268573830056e+00
1.605489516792866667e+00
2.929631990348545489e+00
-2.832543082245212937e-02
1.358286530670594461e+00
1.655932469853677924e+00
6.701964773769768513e-01
2.131182050917533211e+00
2.998351165769753468e+00
7.772095996358327596e+00
1.285014785269981141e+00
4.407334784589962418e+00
1.719858214230612026e+00
-1.012765674651314063e+00
-5.749271123172469133e-01
-3.559614093795681278e+00
-3.073088477387719397e+00
-4.492469521371540431e+00
-3.753286990415885427e+00
-3.219255423324282273e+00
-2.806436518181075090e+00
-2.697305948568419875e+00
-7.879608430851776646e-01
4.625507221739111330e+00
4.809280703883450414e+00
-3.435194026629848629e+00
-3.218943068168937049e+00
3.335535704890596698e+00
2.071359212435486263e+00
4.538992059175040339e+00
-2.770772323566738038e+01
2.903047708571506735e+00
-4.436143805989154032e+00
-2.647991280011542381e-01
1.737252348126810064e+00
-4.121989655995259128e+00
3.209709099445720581e-01
1.012758514896711759e+01
3.313255624721038295e+00
4.631467619785444967e+00
7.668642402146534032e+00
6.780938812710099128e+00
-3.256164342602652972e+00
6.749565128319576779e-01
0.000000000000000000e+00
-4.407265954524525853e+00
0.000000000000000000e+00
-3.666522115024547901e+01
-7.886029397826226273e+01
0.000000000000000000e+00
0.000000000000000000e+00
-2.261283814517791058e+01
-4.024317426178160240e+00
3.213063737030031342e-01
5.079805145796887800e+00
1.326813226475260343e+00
1.233684078112145643e+00
8.671852503871454232e+00
-2.041800256066371944e+00
0.000000000000000000e+00
0.000000000000000000e+00
-1.607347800380474823e+01
-4.430790279223246309e+00
1.177552465851384511e+00
6.342921220500139512e+00
-2.466913734548706327e-02
3.451642566010713065e-01
-6.012767168531006234e+00
7.328146570137336724e+00
7.500088131707050465e+00
0.000000000000000000e+00
-3.547913249211809017e+01
-3.130964814607208879e+00
8.247326544297072237e-01
3.757262485775580418e-01
-2.136528302027558723e+00
-2.631627236037529793e-01
-2.016718799388414141e+01
0.000000000000000000e+00
0.000000000000000000e+00
0.000000000000000000e+00
-7.708602132869285528e-01
-2.602868328868111814e+00
1.435184800833797958e+00
0.000000000000000000e+00
-2.080420864280113413e+00
1.169498351211070819e+00
-1.798334115637199560e+01
-1.193885252696202670e+01
0.000000000000000000e+00
0.000000000000000000e+00
4.304089297965300709e+00
-3.020893216686394656e+00
-1.234427481614708721e+00
0.000000000000000000e+00
1.853340741926325697e+00
-2.686000064995862147e+01
-1.672275139058893600e+01
-2.826268691607605987e+01
0.000000000000000000e+00
0.000000000000000000e+00
-1.547397429377200817e+00
-4.018181657009961327e+00
-7.289186736637049968e+00
-7.458655219230571731e+00
-9.625538282761622710e+00
-1.103039457077456298e+01
-6.262675161142102809e+01
-9.265912629799268885e+00
-8.961543476816615339e+00
-9.622764435629340696e+00
-1.097978292092879826e+01
-1.097978292092879826e+01
-2.088057846500587456e+00
7.763936238952200239e-05
4.404309737393306595e-05
-9.467385027293546973e-02
4.348947142638090457e-01
-2.091409170053043390e-04
-5.788902107267982974e-04
4.481443450852441001e-10
-3.136135459023654537e+00
-1.507730262127600751e+00
-1.204663669965535977e+00
-1.171839254318371104e+00
4.329302247232582057e-01
8.310653628092458334e+00
3.299246725156660176e+00
0.000000000000000000e+00
1.847454931582027254e-02
-1.498024139966201096e+00
-7.660670007653060942e-01
-2.908130300830076731e+00
-1.252564844610269734e+00
-1.910955328742287573e+00
9.471710980110392697e-01
2.352302758516665371e+00
2.982269972214651954e+00
4.280736383314343918e+00
4.633629909719495288e+00
-2.198052637823726840e+00
-1.150759637168392580e+00
3.420433363184381292e+00
1.857878113059351077e-02
-1.559806674919653746e+00
4.197498183183401288e+00
6.262186949633183453e+00
1.100694844370524095e+01
2.778688785515088000e+01
3.679948298336883195e+00
-2.432801394376875592e+00
5.133442052706843617e-01
2.181172654073517680e+00
-8.577551729671881731e-01
7.013844214023315926e-01
3.138233436228588857e+00
7.319940508466630247e-01
0.000000000000000000e+00
3.529209394581482861e+00
1.464585117707144413e+01
8.506550226820598359e-01
-9.060397326548508268e-01
6.787474954688997641e+00
8.125411068867387954e+00
4.515740684104064151e+00
5.372135582950940069e+00
9.210951196799497254e-01
4.802177410869620466e+00
2.945445016176073594e+01
1.575032253128311632e+00
-1.355276854364796946e-01
-3.322474764169629502e-01
3.018397817188666732e+00
1.186503569922195744e+00
0.000000000000000000e+00
8.883242370198487503e-01
7.248276146728496627e+00
0.000000000000000000e+00
0.000000000000000000e+00
-4.831246718433664711e+00
6.124136970173365002e-01
4.145693892559814686e-01
2.683998941637626867e+00
2.063906603639539039e+00
2.989801217386735210e+00
2.262965767379551962e-01
2.240332214649647380e+00
5.984550782416063086e+00
4.587011255338186544e+00
1.233118485315272039e+01
1.115223490909697857e+00
-3.682686422016995476e+00
6.096498453291562258e-01
1.119275528656461516e+00
1.377886278915177731e-01
3.828176805973048324e+00
0.000000000000000000e+00
0.000000000000000000e+00
1.442927634029647344e+01
0.000000000000000000e+00
5.719118583309401593e-01
1.993632609731877392e-01
3.047472271520709430e+00
5.736784864911910198e+00
6.677826247219391220e+00
6.307175478564531090e+00
3.150295169417364249e+01
3.738597740702392258e+00
1.129754590514236234e+01
6.108506268573830056e+00
1.605489516792866667e+00
2.929631990348545489e+00
-2.832543082245212937e-02
1.358286530670594461e+00
1.655932469853677924e+00
6.701964773769768513e-01
2.131182050917533211e+00
2.998351165769753468e+00
7.772095996358327596e+00
1.285014785269981141e+00
4.407334784589962418e+00
1.719858214230612026e+00
-1.012765674651314063e+00
-5.749271123172469133e-01
-3.559614093795681278e+00
-3.073088477387719397e+00
-4.492469521371540431e+00
-3.753286990415885427e+00
-3.219255423324282273e+00
-2.806436518181075090e+00
-2.697305948568419875e+00
-7.879608430851776646e-01
4.625507221739111330e+00
4.809280703883450414e+00
-3.435194026629848629e+00
-3.218943068168937049e+00
3.335535704890596698e+00
2.071359212435486263e+00
4.538992059175040339e+00
-2.770772323566738038e+01
2.903047708571506735e+00
-4.436143805989154032e+00
-2.647991280011542381e-01
1.737252348126810064e+00
-4.121989655995259128e+00
3.209709099445720581e-01
1.012758514896711759e+01
3.313255624721038295e+00
4.631467619785444967e+00
7.668642402146534032e+00
6.780938812710099128e+00
-3.256164342602652972e+00
6.749565128319576779e-01
0.000000000000000000e+00
-4.407265954524525853e+00
0.000000000000000000e+00
-3.666522115024547901e+01
-7.886029397826226273e+01
0.000000000000000000e+00
0.000000000000000000e+00
-2.261283814517791058e+01
-4.024317426178160240e+00
3.213063737030031342e-01
5.079805145796887800e+00
1.326813226475260343e+00
1.233684078112145643e+00
8.671852503871454232e+00
-2.041800256066371944e+00
0.000000000000000000e+00
0.000000000000000000e+00
-1.607347800380474823e+01
-4.430790279223246309e+00
1.177552465851384511e+00
6.342921220500139512e+00
-2.466913734548706327e-02
3.451642566010713065e-01
-6.012767168531006234e+00
7.328146570137336724e+00
7.500088131707050465e+00
0.000000000000000000e+00
-3.547913249211809017e+01
-3.130964814607208879e+00
8.247326544297072237e-01
3.757262485775580418e-01
-2.136528302027558723e+00
-2.631627236037529793e-01
-2.016718799388414141e+01
0.000000000000000000e+00
0.000000000000000000e+00
0.000000000000000000e+00
-7.708602132869285528e-01
-2.602868328868111814e+00
1.435184800833797958e+00
0.000000000000000000e+00
-2.080420864280113413e+00
1.169498351211070819e+00
-1.798334115637199560e+01
-1.193885252696202670e+01
0.000000000000000000e+00
0.000000000000000000e+00
4.304089297965300709e+00
-3.020893216686394656e+00
-1.234427481614708721e+00
0.000000000000000000e+00
1.853340741926325697e+00
-2.686000064995862147e+01
-1.672275139058893600e+01
-2.826268691607605987e+01
0.000000000000000000e+00
0.000000000000000000e+00
-1.547397429377200817e+00
-4.018181657009961327e+00
-7.289186736637049968e+00
-7.458655219230571731e+00
-9.625538282761622710e+00
-1.103039457077456298e+01
-6.262675161142102809e+01
-9.265912629799268885e+00
-8.961543476816615339e+00
-9.622764435629340696e+00
-1.097978292092879826e+01
-1.097978292092879826e+01
+1
View File
@@ -0,0 +1 @@
59708719594136008472212676481906
@@ -0,0 +1 @@
5コ705コ(770(777
+1
View File
@@ -0,0 +1 @@
2Bba5a7FEEcfced4fbbBfFEe16bfcaA4
@@ -0,0 +1 @@
5コ70(77247・906:
@@ -0,0 +1 @@
5コ700772212676481906:
Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More