Commit Graph
51 Commits
Author SHA1 Message Date
Nardi Ivan dcac633878 QUIC: add support for MVFST EXPERIMENTAL version 2020-09-20 16:38:28 +02:00
Nardi Ivan 8db084ab06 TLS: fix memory accesses in QUIC transport parameters extension 2020-09-10 18:19:18 +02:00
Nardi Ivan 7d5a0e1f04 QUIC: extract User Agent information 2020-09-08 11:03:22 +02:00
Nardi Ivan a1014e8895 http: create a common function to parse User Agent field
Prepare the code to handle UA information from flows other than HTTP
2020-09-08 10:34:05 +02:00
Nardi Ivan 7da4abe6ad QUIC: add support for GQUIC T050 and T051
QUIC versioning wasn't complex enough without T05X family...
These versions are very similar to Q050, but use TLS as their handshake
protocol.
2020-08-30 20:51:33 +02:00
Nardi Ivan 97b80a8838 QUIC: minor fixes
LGTM found a real issue on a boundary check
Fix unit tests: a pcap ha been uploaded twice (with different names)
Fix compilation when using DPDK (see #990)
2020-08-24 13:53:36 +02:00
Nardi Ivan b23cfd6b84 Add sub-classification for GQUIC >= Q050 and (IETF-)QUIC
Add QUIC payload and header decryption: most of the crypto code has been
"copied-and-incolled" from Wireshark. That code has been clearly marked
as such. All credits for that code should go to the original authors.

I tried to keep the Wireshark code as similar as possible to the original,
comments included, to ease future backporting of fixes.
Inevitably, glibc data types and data structures, tvbuff abstraction and
allocation functions have been converted.
2020-08-21 22:04:55 +02:00
Nardi Ivan 7a1147d733 Update TLS dissector to handle QUIC flows
Latest QUIC versions use TLS for the encryption layer: reuse existing code
to allow Client Hello parsing and sub-classification based on SNI value.
Side effect: we might have J3AC, TLS negotiated version, SNI value and
supported cipher list for QUIC, too.
2020-08-21 22:04:55 +02:00
Nardi Ivan d62ae567d1 Add (optional) dependency on external libraries: libgcrypt and libgpg-error
To support QUIC payload and header decryption, it is necessary to choose an
external crypto library to handle the low-level crypto stuff. Since we will
use some Wireshark code, it is quite natural to choose the same library used
by Wireshark itself: libgcrypt.

More precisely, we will use libgcrypt and libgpg-error.
Both libraries have LGPL license, so there should be no issue from this point
of view.

These libraries are not required to build nDPI, and their usage is optional:
nDPI will keep working (and compiling) even if they are not available.
However, without them, QUIC sub-classification is next to impossible.
The configure flag "--disable-gcrypt" forces the build system to ignore these
libraries.

libgpg-error is only used for debug to have meaningful error messages and its
usage is trivial.
The same cannot be said for libgcrypt because its initialization is a significant
issue.

The rest of this commit message try explaining how libgcrypt is
initialized.

According to the documentation
   https://gnupg.org/documentation/manuals/gcrypt/Initializing-the-library.html
   https://gnupg.org/documentation/manuals/gcrypt/Multi_002dThreading.html#Multi_002dThreading
libgcrypt must be initialized before using it, but such initialization should
be performed by the actual application and not by any library.

Forcing the users to proper initialize libgcrypt in their own code seems
unreasonable: most people using nDPI might be complete unaware of any crypto
stuff and update each and every one application linking to nDPI with specific
libgcrypt code should be out of question, anyway.

Fortunately, it seems a workaround exists to initialize libgcrypt in a library
    https://lists.gnupg.org/pipermail/gcrypt-devel/2003-August/000458.html

Therefore, we could provide a wrapper to this initialization stuff in a nDPI
function. Unfortunately nDPI API lacks a global init function that must be
called only once, before any other functions. We could add it, but that would
be a major API break.
AFAIK, ndpi_init_detection_module() might be called multiple times, for example
to create multiple independent dpi engines in the same program.

The proposed solution is to (optionally) initialize libgcrypt in
ndpi_init_detection_module() anyway:
* if the actual application doesn't directly use libgcrypt and only calls
  ndpi_init_detection_module() once, everything is formally correct and it
  should work out of the box [by far the most common user case];
* if the actual application already uses libgcrypt directly, it already
  performs the required initialization. In this case the ndpi_prefs.ndpi_dont_init_libgcrypt
  flag should be passed to ndpi_init_detection_module() to avoid further
  initializations.

The only scenario not supported by this solution is when the application is
unaware of libgcrypt and calls ndpi_init_detection_module() multiple times
concurrently. But this scenario should be uncommon.

A completely different option should be to switch to another crypto library,
with a huge impact on the QUIC dissector code.

Bottom line: crypto is hard, using libgcrypt is complex and the proposed
initialization, even if not perfect, should cover the most frequent user
cases and should work, for the time being.
If anyone has some suggestions...
2020-08-21 22:04:55 +02:00
Nardi Ivan 23ec82b59d Major rework of QUIC dissector
Improve support for GQUIC (up to Q046) and add support for Q050 and (IETF-)QUIC
Still no sub-classification for Q050 and QUIC
2020-08-21 22:04:55 +02:00
Nardi Ivan 2722861d6e Suspicious ESNI usage: add a comment and a pcap example
See: 79b89d2866
2020-08-06 10:29:35 +02:00
Nardi Ivan 79b89d2866 Add risk flag about suspicious ESNI usage
In a Client Hello, the presence of both SNI and ESNI may obfuscate the real
domain of an HTTPS connection, fooling DPI engines and firewalls, similarly
to Domain Fronting.

Such technique is reported in a presentation at DEF CON 28:
"Domain Fronting is Dead, Long Live Domain Fronting: Using TLS 1.3 to evade
censors, bypass network defenses, and blend in with the noise"
Full credit for the idea must go the original author

At the moment, the only way to get the pdf presention and related video is via
https://forum.defcon.org/node/234492
Hopefully a direct link (and an example pcap) will be available soon
2020-08-05 17:13:23 +02:00
Nardi Ivan 7d843f4d17 Minor fixes
Fix a memory leak and an issue (re)-introduced in configure script
2020-07-26 21:11:34 +02:00
Nardi Ivan fd6a5bc1cd Fix memory leak reported in #955
This leak is clearly happening because of missing/disabled code in "bins"
handling.
This is a trivial workaround useful until such code is completed.
In this way tavis-ci will stop complaining
2020-07-08 11:30:39 +02:00
Nardi Ivan 030f3f3d48 Fix a memory leak 2020-07-07 10:27:57 +02:00
Nardi Ivan 974c1cc681 Improve help message of --dbg-proto option
Make it clear that such option is general, not about extcap
functionality
2020-07-02 09:42:53 +02:00
Nardi Ivan c08693fda5 Incorporated some feedback 2020-07-01 20:16:16 +02:00
Nardi Ivan f39e3c98f0 Update test results 2020-06-28 12:47:27 +02:00
Nardi Ivan 317d3ffd3e Fix undefined behaviour in internal tests
Error messages:
ndpiReader.c:3211:2: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
ndpiReader.c:3207:5: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'

The errors started popping up since 3d9285f1
2020-06-28 12:13:21 +02:00
Nardi Ivan b24f5c4c0a Fix memory leak about purged/expired flows
Create an helper to avoid similar errors in the future
Fixes: 1a62f4c7
2020-06-28 12:05:12 +02:00
Nardi Ivan d6a97219ea Fix use-after-free in http content parsing 2020-06-28 12:05:12 +02:00
Nardi Ivan 3669c14afd DNP3: add missing initialization 2020-06-28 12:05:12 +02:00
Nardi Ivan ece5d3e199 Fix (harmless) memory leaks when DPDK is enabled 2020-06-28 12:05:12 +02:00
Nardi Ivan 2cdf7ce806 Fix startup when DPDK is enabled
Fixes:a58c838c4
2020-06-28 12:05:12 +02:00
Nardi Ivan 2effa57d8a ndpiReader: fix ports statistics
They should take idle/expired flows into account, too
2020-06-28 12:05:12 +02:00
Nardi Ivan b68b45f3bb TLS: extract JA3 signatures in some corner cases
In some (rare) cases, Client Hello message contains lots of cipher
suits.
2020-06-28 12:05:12 +02:00
Nardi Ivan 0cd0905400 mqtt: add boundary check in log message 2020-06-26 12:04:02 +02:00
Nardi Ivan 56d87186f7 Fix compilation with --enable-debug-messages flag
NDPI_LOG* macros dereference ndpi_detection_module_struct object which is
private to ndpi library (via NDPI_LIB_COMPILATION define). So we can't use
them outside the library itself, i.e. in ndpiReader code
Therefore, in files in example/, convert all (rare) uses of NDPI_LOG* macros
to a new very simple macro, private to ndpiReader program. If necessary,
such macro may be improved.

According to a comment in ndpi_define.h, each dissector must define its own
NDPI_CURRENT_PROTO macro before including ndpi_api.h file
2020-06-26 12:04:02 +02:00
Nardi Ivan 3c66ca236b Make lua script more robust 2020-06-25 18:02:03 +02:00
Nardi Ivan 70a926088f Fixed harmless memory leak in extcap initialization 2020-06-25 18:02:03 +02:00
Nardi Ivan 6b69e7ebbc Restore extcap functionality, i.e. integration with wireshark 2020-06-25 18:02:03 +02:00
Nardi Ivan f8503da9e9 Fix NATS dissector 2020-05-09 15:41:20 +02:00
Nardi Ivan 097127c31d Fix heap-overflow error in CAPWAP detunneling code 2020-04-24 10:42:52 +02:00
Nardi Ivan c2ebbb15ad Fix "division by zero" runtime error 2020-04-23 14:24:49 +02:00
Nardi Ivan f965983c23 Add basic support for some ip-in-ip tunnels
Add support for 4in4, 6in6 and 4in6 encapsulations
Add support for ipv6 traffic in gtp tunnels, too

To allow gtp unit test, gtp detunneling flag has been globally enabled
in the test suite
2020-04-23 10:55:33 +02:00
Nardi Ivan b1a6c6b895 Fix some compilation warnings 2020-04-20 16:53:39 +02:00
Nardi Ivan e84563f971 ndpiReader: fix memory leak in idle sessions purging 2020-04-08 15:15:34 +02:00
Nardi Ivan a60854bae6 Fix some debug messages 2020-04-08 15:15:34 +02:00
Nardi Ivan 7b690d8b2d kerberos: fix heap-overflow error 2020-04-08 15:15:34 +02:00
Nardi Ivan 0c1322a718 s7comm: fix heap-overflow error 2020-04-08 15:15:34 +02:00
Nardi Ivan b40732d8a5 Fix an harmless memory leak
Leak introduced in 90e08b35, while fixing #845
2020-04-08 15:15:34 +02:00
Nardi Ivan 498571354d openvpn: fix heap-overflow error 2020-03-27 11:09:53 +01:00
Nardi Ivan 3924257f32 telnet: fix another heap-overflow error 2020-03-27 11:09:53 +01:00
Nardi Ivan 817aa54214 telnet: fix heap-overflow error
There is some boilerplate since I removed an if branch and I had to reindent
the code
2020-03-27 11:09:53 +01:00
Nardi Ivan 97fc94c7e8 kerberos: fix use-after-free error
After leaving kerberos code, the original packet may be processed from
other dissector (i.e. TLS)
2020-03-27 11:09:53 +01:00
Nardi Ivan 9ec711aa90 tls: fix heap-overflow error 2020-03-27 11:09:53 +01:00
Nardi Ivan 9c2c04d8ce ssh: fix heap-overflow error 2020-03-27 11:09:53 +01:00
Nardi Ivan 4f399356a6 quic: fix heap-overflow error 2020-03-27 11:09:53 +01:00
Nardi Ivan 45ff2a3f5a h323: fix heap-overflow error 2020-03-27 11:09:53 +01:00
Nardi Ivan 58f0f29113 soulseek: fix heap-overflow error 2020-03-27 11:09:53 +01:00
Nardi Ivan da5e5a3e2a ciscovpn: fix heap-overflow error 2020-03-27 11:09:53 +01:00