Commit Graph
604 Commits
Author SHA1 Message Date
Luca Deri 60b58dbd67 RSI enhancements 2021-02-05 10:59:09 +01:00
Luca Deri 1eedf734be Implemented API for computing RSI (Relative Strenght Index)
void  ndpi_init_rsi(struct ndpi_rsi_struct *s, u_int16_t num_learning_values);
void  ndpi_free_rsi(struct ndpi_rsi_struct *s);
float ndpi_rsi_add_value(struct ndpi_rsi_struct *s, const u_int32_t value);
2021-02-04 23:52:33 +01:00
Ivan Nardi a772e18977 Fix a warning (#1125)
Introduced in 5f7b9d802

reader_util.c: In function ‘process_ndpi_collected_info’:
reader_util.c:1148:60: warning: ‘%s’ directive output may be truncated writing up to 255 bytes into a region of size 64 [-Wformat-truncation=]
 1148 |       sizeof(flow->ssh_tls.client_requested_server_name), "%s",
      |                                                            ^~
reader_util.c:1147:5: note: ‘snprintf’ output between 1 and 256 bytes into a destination of size 64
 1147 |     snprintf(flow->ssh_tls.client_requested_server_name,
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 1148 |       sizeof(flow->ssh_tls.client_requested_server_name), "%s",
      |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 1149 |       flow->ndpi_flow->protos.tls_quic_stun.tls_quic.client_requested_server_name);
      |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-02-03 11:56:37 +01:00
Luca Deri 4b181be58e Improved debug message 2021-02-03 11:49:14 +01:00
Luca Deri 288ccd6215 Fixes due to datatype rename 2021-01-22 09:17:34 +01:00
Luca Deri a31bd5ac3c Cleaned up tls/quic datatypes 2021-01-21 19:17:33 +01:00
Luca Deri 15295ef4c5 Reworked TLS fingerprint calcolation
Modified TLS memory free
2021-01-21 19:06:05 +01:00
morefigs 53415c8855 Added missing comma (#1116)
I presume there is a comma missing in this comma separated list.
2021-01-21 08:58:42 +01:00
Luca Deri 3e5e9569ff Added simple hash implementation to the nDPI API 2021-01-20 21:30:19 +01:00
Luca Deri d964c3e081 Code cleanup: third party uthash is at the right place 2021-01-20 19:11:36 +01:00
Luca Deri 68b6ac7da8 (C) Update 2021-01-07 11:13:36 +01:00
Luca Deri eb37f8f1fb Split HTTP request from response Content-Type. Request Content-Type should be present with POSTs and not with other methods such as GET 2021-01-06 18:28:24 +01:00
Ivan Nardi 3aa16b63e8 Fix some warnings when compiling with "-W -Wall" flags (#1103) 2021-01-04 15:49:39 +01:00
Luca Deri b7376cc690 Restored QUIC stats 2020-12-30 12:12:33 +01:00
Luca Deri 9c1827a77b Fixed output when tLS (nad not QUIC) is used 2020-12-28 09:19:39 +01:00
Luca Deri a89642ad04 Fixes bug introduced by https://github.com/ntop/nDPI/pull/1085 2020-12-12 12:32:44 +01:00
Zied Aouini 5bd5461f96 Fix minimum packet length condition (#1087) 2020-12-12 11:12:59 +01:00
Toni 74a77e7b3d Added --ignore-vlanid / -I to exclude VLAN ids for flow hash calculation. #1073 (#1085)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-12-11 21:01:51 +01:00
Luca Deri eb689b2069 nDPI rules (work in progress) implementation 2020-11-30 22:01:49 +01:00
Toni af02ffb60f Support raw IPv4 / IPv6 pcap packet processing. (#1053)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-09 16:18:05 +01:00
Luca Deri 017e395ed1 Cosmetic changes 2020-11-03 16:46:30 +01:00
Luca Deri 48d640583a Moved global in reader_util.c 2020-10-27 08:40:00 +01:00
Luca Deri 948a906037 Added -D flag for detecting DoH in the wild
Removed heuristic from CiscoVPN as it leads to false positives
2020-10-26 21:40:59 +01:00
Luca Deri 9873972acb Various improvemement when using ndpi_pref_enable_tls_block_dissection:
application data TLS blocks are now ignored when exchanged before
- the end of certificate negotiation (up to TLS 1.2)
- change cipher
2020-10-24 19:22:56 +02:00
Luca Deri 9b85669a64 Added -x for checking patterns 2020-10-22 23:28:18 +02:00
Ivan Nardi 6027a7c799 Fix parsing of DLT_PPP datalink type (#1042) 2020-10-21 22:27:42 +02:00
Zied Aouini 43c1f6a3fd CAPWAP tunnel decoding fix (#1038)
* Fix CAPWAP processing.

* Update result.
2020-10-21 15:07:20 +02:00
Toni c2d8955c18 Fixed missing PCAP include directories in Makefiles. (#1034)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-10-19 11:25:22 +02:00
aouinizied d5d2a7e3f3 Fix CAPWAP handling. 2020-10-13 19:13:07 +02:00
Luca 05d93790e4 Added ndpi_quick_16_byte_hash
Warning fix
2020-10-05 08:26:24 +02:00
Luca Deri 32e4922c5a Warning fix 2020-10-03 17:14:40 +02:00
Toni 656323c334 Added missing files to `make dist' target which are not required to build nDPI but still somehow essential. (#1024)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-29 17:58:33 +02:00
Alfredo Cardigliano e6d206fd15 Add unit tests to travis. Move ndpi serializer tests to unit tests. 2020-09-21 17:24:06 +02:00
Luca Deri ea9177cc93 Merge pull request #1018 from lnslbrty/fix/make-dist
Fixed broken `make dist' and added CI check.
2020-09-20 18:33:57 +02:00
Luca Deri 1385f05713 Merge pull request #1017 from lnslbrty/fix/mingw-xcompile
Added support for mingw xcompile.
2020-09-20 18:33:40 +02:00
Toni Uhlig fdef6f3545 Fixed broken `make dist' and added CI check.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-19 02:23:53 +02:00
Toni Uhlig bcce122665 Fixed mingw build w/o examples/tests/fuzzer.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-09-18 19:46:07 +02:00
Luca Deri d81bc1add6 Reworked MDNS dissector that is not based on the DNS dissector 2020-09-17 23:24:02 +02:00
Luca Deri 753b5dde16 Merge pull request #1012 from IvanNardi/ua
QUIC: extract User Agent information
2020-09-17 21:32:25 +02:00
Luca Deri 7086197047 Added extension to detect nested subdomains as used in Browsertunnel attack tool
https://github.com/veggiedefender/browsertunnel
2020-09-09 23:25:19 +02:00
Nardi Ivan a1014e8895 http: create a common function to parse User Agent field
Prepare the code to handle UA information from flows other than HTTP
2020-09-08 10:34:05 +02:00
Toni Uhlig adab0139c2 Fixed invalid memory access leading to a SIGSEGV in ndpiReader's option parser.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-28 06:53:58 +02:00
Nardi Ivan 97b80a8838 QUIC: minor fixes
LGTM found a real issue on a boundary check
Fix unit tests: a pcap ha been uploaded twice (with different names)
Fix compilation when using DPDK (see #990)
2020-08-24 13:53:36 +02:00
Nardi Ivan d62ae567d1 Add (optional) dependency on external libraries: libgcrypt and libgpg-error
To support QUIC payload and header decryption, it is necessary to choose an
external crypto library to handle the low-level crypto stuff. Since we will
use some Wireshark code, it is quite natural to choose the same library used
by Wireshark itself: libgcrypt.

More precisely, we will use libgcrypt and libgpg-error.
Both libraries have LGPL license, so there should be no issue from this point
of view.

These libraries are not required to build nDPI, and their usage is optional:
nDPI will keep working (and compiling) even if they are not available.
However, without them, QUIC sub-classification is next to impossible.
The configure flag "--disable-gcrypt" forces the build system to ignore these
libraries.

libgpg-error is only used for debug to have meaningful error messages and its
usage is trivial.
The same cannot be said for libgcrypt because its initialization is a significant
issue.

The rest of this commit message try explaining how libgcrypt is
initialized.

According to the documentation
   https://gnupg.org/documentation/manuals/gcrypt/Initializing-the-library.html
   https://gnupg.org/documentation/manuals/gcrypt/Multi_002dThreading.html#Multi_002dThreading
libgcrypt must be initialized before using it, but such initialization should
be performed by the actual application and not by any library.

Forcing the users to proper initialize libgcrypt in their own code seems
unreasonable: most people using nDPI might be complete unaware of any crypto
stuff and update each and every one application linking to nDPI with specific
libgcrypt code should be out of question, anyway.

Fortunately, it seems a workaround exists to initialize libgcrypt in a library
    https://lists.gnupg.org/pipermail/gcrypt-devel/2003-August/000458.html

Therefore, we could provide a wrapper to this initialization stuff in a nDPI
function. Unfortunately nDPI API lacks a global init function that must be
called only once, before any other functions. We could add it, but that would
be a major API break.
AFAIK, ndpi_init_detection_module() might be called multiple times, for example
to create multiple independent dpi engines in the same program.

The proposed solution is to (optionally) initialize libgcrypt in
ndpi_init_detection_module() anyway:
* if the actual application doesn't directly use libgcrypt and only calls
  ndpi_init_detection_module() once, everything is formally correct and it
  should work out of the box [by far the most common user case];
* if the actual application already uses libgcrypt directly, it already
  performs the required initialization. In this case the ndpi_prefs.ndpi_dont_init_libgcrypt
  flag should be passed to ndpi_init_detection_module() to avoid further
  initializations.

The only scenario not supported by this solution is when the application is
unaware of libgcrypt and calls ndpi_init_detection_module() multiple times
concurrently. But this scenario should be uncommon.

A completely different option should be to switch to another crypto library,
with a huge impact on the QUIC dissector code.

Bottom line: crypto is hard, using libgcrypt is complex and the proposed
initialization, even if not perfect, should cover the most frequent user
cases and should work, for the time being.
If anyone has some suggestions...
2020-08-21 22:04:55 +02:00
Luca Deri d87bdf9302 Merge pull request #977 from adek05/dev
Enable building on OpenBSD 6.7
2020-08-19 21:48:38 +02:00
Luca Deri ebf89f46e3 Merge pull request #984 from lnslbrty/fix/invalid-fncall-dpdk
Fixed invalid dpdk fn call.
2020-08-16 10:03:55 +02:00
Luca Deri 019a64630b Merge pull request #983 from lnslbrty/fix/libpcap-obsolete-pcap_lookupdev-usage
Replaced obsolete libpcap pcap_lookupdev with pcap_findalldevs.
2020-08-16 10:03:33 +02:00
Toni Uhlig 13cbd1e4ae Fixed invalid dpdk fn call.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-15 12:38:47 +02:00
Toni Uhlig b31fde4bbb Replaced obsolete libpcap pcap_lookupdev with pcap_findalldevs.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-15 12:35:49 +02:00
Toni Uhlig aa856735c0 num_extra_packets_checked check can be 0 for some protocols and therefor requires lesser-or-equal condition for max_extra_packets_to_check
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-08-13 10:32:31 +02:00