Commit Graph
167 Commits
Author SHA1 Message Date
Toni ed51987e3a Fix broken fuzz_process_packet fuzzer by adding a call to ndpi_finalize_initialization(). (#1334)
* fixed several memory errors (heap-overflow, unitialized memory, etc)
 * ability to build fuzz_process_packet with a main()
   allowing to replay crash data generated with fuzz_process_packet
   by LLVMs libfuzzer
 * temporarily disable fuzzing if `tests/do.sh`
   executed with env FUZZY_TESTING_ENABLED=1

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-18 23:16:32 +02:00
Nardi Ivan 03d3e1bafc Fix parsing of ipv6 packets with extension headers
Decoding of ipv6 traffic with extension headers was completely broken,
since the beginning of the L4 header was always set to a wrong value.

Handle the ipv6 fragments in the same way as the ipv4 ones: keep the first
one and drop the others.
2021-09-19 17:29:22 +02:00
Ivan Nardi 8fdffbf3a1 Compile everything with "-W -Wall -Wno-unused-parameter" flags (#1276)
Fix all the warnings.

Getting rid of "-Wno-unused-parameter" is quite complex because some
parameters usage depends on compilation variable (i.e.
`--enable-debug-messages`).

The "-Werror" flag has been added only in Travis builds to avoid
breaking the builds to users using uncommon/untested
OS/compiler/enviroment.

Tested on:
* x86_64; Ubuntu 20.04; gcc 7,8,9,10,11; clang 7,8,9,10,11,12
* x86_64; CentOS 7.7; gcc 4.8.5 (with "--disable-gcrypt" flag)
* Raspberry 4; Debian 10.10; gcc 8.3.0
2021-08-20 18:11:13 +02:00
Toni 8d0c7b1fae Fixed Mingw64 build, SonerCloud-CI and more. (#1273)
* Added ARM build and unit test run for SonarCloud-CI.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Fixed Mingw64 build.

 * adapted to SonarCloud-CI workflow
 * removed broken and incomplete Windows example (tested on VS2017/VS2019)
 * removed unnecessary include (e.g. pthread.h for the library which does not make use of it)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-08-18 11:34:16 +02:00
Luca Deri 05492ec8e8 Defined more standard ethernet protocol types 2021-08-09 17:01:10 +02:00
Luca Deri 58ca7b5a7f Added protocol defines 2021-08-09 16:45:03 +02:00
Luca Deri 51e4888442 Implemented ALPN automa for checking uncommon ALPNs 2021-07-24 17:50:32 +02:00
Luca Deri 61fc5be202 Reworked flow risk implementation 2021-07-23 17:27:15 +02:00
Toni 6ad0d6666c Implemented function to retrieve flow information. #1253 (#1254)
* fixed [h]euristic typo

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-23 10:37:20 +02:00
Luca Deri b01b60a2b5 Implementation of flow risk eception (work in progress) 2021-07-22 01:35:57 +02:00
Ivan Nardi cccf794265 ndpiReader: add statistics about nDPI performance (#1240)
The goal is to have a (roughly) idea about how many packets nDPI needs
to properly classify a flow.

Log this information (and guessed flows number too) during unit tests,
to keep track of improvements/regressions across commits.
2021-07-13 12:28:39 +02:00
Luca Deri 43a8576efb Reworked human readeable string search in flows
Removed fragment manager code
2021-05-17 20:55:06 +02:00
Toni 8c28613eb2 Check datalink during fuzzing to prevent console / logfile spam. See #1175 for more information. (#1177)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-09 15:09:43 +02:00
Luca ae2470fad4 Initial work towards detection via TLS of browser types 2021-05-06 21:42:06 +02:00
Toni da3e6bd61b Check for common ALPNs and set a flow risk if not known. (#1175)
* Increased risk bitmask to 64bit (instead of 32bit).
 * Removed annoying "Unknown datalink" error message for fuzzers.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-27 07:22:04 +02:00
Luca Deri 4a09707e48 Added flow risk to wireshark dissection 2021-04-26 10:17:29 +02:00
Ivan Nardi 9ca62ed7ac Fix detunneling of GTP-U traffic (#1168)
Fuzzing #1161 exposed some (completely unrelated) issues on GTP-U
detunneling code.
(see https://github.com/ntop/nDPI/actions/runs/719882047)
2021-04-18 21:37:51 +02:00
Luca Deri fcbc16da00 Fixed invalid guess stats 2021-03-30 17:49:48 +02:00
Ivan Nardi a6029d250d ndpiReader: print an error msg if we found an unsupported datalink type (#1157) 2021-03-23 11:47:29 +01:00
Luca Deri 565a7bfce3 Reworked extendal dependency across testing tools 2021-03-14 20:48:21 +01:00
Luca Deri f6ad16d8f8 Added experiemntal JA3+ implementation that can be used with -z i ndpiReader 2021-03-09 23:38:29 +01:00
Ivan Nardi c50a8d4808 Add support for Snapchat voip calls (#1147)
* Add support for Snapchat voip calls

Snapchat multiplexes some of its audio/video real time traffic with QUIC
sessions. The peculiarity of these sessions is that they are Q046 and
don't have any SNI.

* Fix tests with libgcrypt disabled
2021-03-06 05:48:36 +01:00
Toni 1e12c90c66 Fixed memory leaks caused by conditional free'ing for some TLS connec… (#1132)
* Fixed memory leaks caused by conditional free'ing for some TLS connections.

 * Members of tls_quic struct should also free'd if the detected master protocol is IMAPS / POPS / SMTPS / etc.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Prevent reader_util.c from exit()'ing if maximum flow count reached.
This confuses the fuzzer.

 * Improved fuzz/Makefile.am to use LDADD for ../example/libndpiReader.a instead of LDFLAGS.
   That way, fuzz_ndpi_reader re-links to ../example/libndpiReader.a if something changed there.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-10 15:24:11 +01:00
Luca Deri 60b58dbd67 RSI enhancements 2021-02-05 10:59:09 +01:00
Luca Deri 4b181be58e Improved debug message 2021-02-03 11:49:14 +01:00
Luca Deri a31bd5ac3c Cleaned up tls/quic datatypes 2021-01-21 19:17:33 +01:00
Luca Deri 15295ef4c5 Reworked TLS fingerprint calcolation
Modified TLS memory free
2021-01-21 19:06:05 +01:00
Luca Deri 68b6ac7da8 (C) Update 2021-01-07 11:13:36 +01:00
Luca Deri eb37f8f1fb Split HTTP request from response Content-Type. Request Content-Type should be present with POSTs and not with other methods such as GET 2021-01-06 18:28:24 +01:00
Luca Deri b7376cc690 Restored QUIC stats 2020-12-30 12:12:33 +01:00
Luca Deri 9c1827a77b Fixed output when tLS (nad not QUIC) is used 2020-12-28 09:19:39 +01:00
Luca Deri a89642ad04 Fixes bug introduced by https://github.com/ntop/nDPI/pull/1085 2020-12-12 12:32:44 +01:00
Zied Aouini 5bd5461f96 Fix minimum packet length condition (#1087) 2020-12-12 11:12:59 +01:00
Toni 74a77e7b3d Added --ignore-vlanid / -I to exclude VLAN ids for flow hash calculation. #1073 (#1085)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-12-11 21:01:51 +01:00
Toni af02ffb60f Support raw IPv4 / IPv6 pcap packet processing. (#1053)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-09 16:18:05 +01:00
Luca Deri 017e395ed1 Cosmetic changes 2020-11-03 16:46:30 +01:00
Luca Deri 48d640583a Moved global in reader_util.c 2020-10-27 08:40:00 +01:00
Luca Deri 948a906037 Added -D flag for detecting DoH in the wild
Removed heuristic from CiscoVPN as it leads to false positives
2020-10-26 21:40:59 +01:00
Luca Deri 9873972acb Various improvemement when using ndpi_pref_enable_tls_block_dissection:
application data TLS blocks are now ignored when exchanged before
- the end of certificate negotiation (up to TLS 1.2)
- change cipher
2020-10-24 19:22:56 +02:00
Ivan Nardi 6027a7c799 Fix parsing of DLT_PPP datalink type (#1042) 2020-10-21 22:27:42 +02:00
Zied Aouini 43c1f6a3fd CAPWAP tunnel decoding fix (#1038)
* Fix CAPWAP processing.

* Update result.
2020-10-21 15:07:20 +02:00
aouinizied d5d2a7e3f3 Fix CAPWAP handling. 2020-10-13 19:13:07 +02:00
Luca 05d93790e4 Added ndpi_quick_16_byte_hash
Warning fix
2020-10-05 08:26:24 +02:00
Luca Deri 32e4922c5a Warning fix 2020-10-03 17:14:40 +02:00
Luca Deri d81bc1add6 Reworked MDNS dissector that is not based on the DNS dissector 2020-09-17 23:24:02 +02:00
Nardi Ivan a1014e8895 http: create a common function to parse User Agent field
Prepare the code to handle UA information from flows other than HTTP
2020-09-08 10:34:05 +02:00
Adrian Zgorzałek 8f74d5733d OpenBSD: Introduce pkt_timeval to deal with (bpf_)_timeval
Some BSD APIs called in example/ return `struct bpf_timeval`, where nDPI
APIs expect `struct timeval`. These two structs, besides having
a different name, share the exact same set of fields.
2020-08-09 14:30:12 +01:00
Luca Deri a828ac0191 Tiny changes for TLS block lenght dissection 2020-07-29 22:36:27 +02:00
Luca Deri 802254327a wrapper cleanup 2020-07-13 10:53:25 +02:00
Toni Uhlig 96c193cf79 Fixed heap overflow caused by missing lengthcheck in 802.11 LLC header parsing.
* triggered by fuzz traces from wireshark

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-11 13:10:03 +02:00