Commit Graph
51 Commits
Author SHA1 Message Date
Luca Deri 9cc4cbb9d1 Reworked teams handling 2023-06-15 22:31:11 +02:00
Luca Deri d0609ea601 Implemented Zoom/Teams stream type detection 2023-06-14 23:44:57 +02:00
Ivan Nardi ace32c9dfe Add support for SRTP (#1977)
The goal is to have Zoom flows classified as "Encrypted" and not as
"Cleartext".

Start documenting the list of protocols supported by nDPI;
format, verbosity and content are still a work-in-progress.
2023-05-16 16:02:14 +02:00
Ivan Nardi de693cbbc9 RTP: remove dead-code (#1953)
The checks `isValidMSRTPType(..) == 1` is a subset of
`is_valid_rtp_payload_type()` so this if-branch is never reached.

More importantly, the article describing how to detect Microsoft Lync and
Skype for Business is from 2014. These payload types are static or they
are in the dynamic range: in both cases, these values might be used (and
they are used indeed) pretty much by every application.
Bottom line: we can't use PT alone to identify a specific protocol.

Keep the list, since it is used to tell audio streams from video ones.
2023-04-25 14:16:27 +02:00
Ivan Nardi b51a2ac72a fuzz: some improvements and add two new fuzzers (#1881)
Remove `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` define from
`fuzz/Makefile.am`; it is already included by the main configure script
(when fuzzing).

Add a knob to force disabling of AESNI optimizations: this way we can
fuzz also no-aesni crypto code.

Move CRC32 algorithm into the library.

Add some fake traces to extend fuzzing coverage. Note that these traces
are hand-made (via scapy/curl) and must not be used as "proof" that the
dissectors are really able to identify this kind of traffic.

Some small updates to some dissectors:

CSGO: remove a wrong rule (never triggered, BTW). Any UDP packet starting
with "VS01" will be classified as STEAM (see steam.c around line 111).
Googling it, it seems right so.

XBOX: XBOX only analyses UDP flows while HTTP only TCP ones; therefore
that condition is false.

RTP, STUN: removed useless "break"s

Zattoo: `flow->zattoo_stage` is never set to any values greater or equal
to 5, so these checks are never true.

PPStream: `flow->l4.udp.ppstream_stage` is never read. Delete it.

TeamSpeak: we check for `flow->packet_counter == 3` just above, so the
following check `flow->packet_counter >= 3` is always false.
2023-02-09 20:02:12 +01:00
Ivan Nardi ad6bfbad4d Add protocol disabling feature (#1808)
The application may enable only some protocols.
Disabling a protocol means:
*) don't register/use the protocol dissector code (if any)
*) disable classification by-port for such a protocol
*) disable string matchings for domains/certificates involving this protocol
*) disable subprotocol registration (if any)

This feature can be tested with `ndpiReader -B list_of_protocols_to_disable`.

Custom protocols are always enabled.

Technically speaking, this commit doesn't introduce any API/ABI
incompatibility. However, calling `ndpi_set_protocol_detection_bitmask2()`
is now mandatory, just after having called `ndpi_init_detection_module()`.

Most of the diffs (and all the diffs in `/src/lib/protocols/`) are due to
the removing of some function parameters.

Fix the low level macro `NDPI_LOG`. This issue hasn't been detected
sooner simply because almost all the code uses only the helpers `NDPI_LOG_*`
2022-12-18 08:10:57 +00:00
Luca Deri dbff2fbadd Minor RTP fix 2022-12-15 19:17:05 +01:00
Ivan Nardi 48a7f6d487 fuzz: some enhancements (#1827)
Load some custom configuration (like in the unit tests) and factorize some
(fuzzing) common code.

There is no way to pass file paths to the fuzzers as parameters. The safe
solution seems to be to load them from the process working dir. Anyway,
missing file is not a blocking error.

Remove some dead code (found looking at the coverage report)
2022-12-10 19:49:11 +01:00
Luca Deri eacc2b8e32 Added Zoom screen share detection 2022-12-09 21:32:45 +01:00
Luca Deri fc7b070030 Added RTP stream type in flow metadata 2022-12-09 14:26:53 +01:00
Luca Deri 63f349319f Improved Zoom protocol detection 2022-12-08 22:00:24 +01:00
Nardi Ivan aa5dd25392 RTP: remove a superfluous check
Static analyzer complains about dereferencing `packet->udp` before
checking.
Since this function is called only with UDP flows, remove the check.

Close: #1792
2022-11-07 18:47:19 +01:00
Ivan Nardi 1796a1d814 LINE_CALL: add detection of LINE voip calls (#1761)
These flows are classifed as `LINE_CALL`; another option was
`RTP/LINE_CALL`. No sure about the best solution...

Extend LINE domains list.

Remove RTP dead code.
2022-10-06 17:09:26 +02:00
Ivan Nardi 3a087e951d Add a "confidence" field about the reliability of the classification. (#1395)
As a general rule, the higher the confidence value, the higher the
"reliability/precision" of the classification.

In other words, this new field provides an hint about "how" the flow
classification has been obtained.
For example, the application may want to ignore classification "by-port"
(they are not real DPI classifications, after all) or give a second
glance at flows classified via LRU caches (because of false positives).

Setting only one value for the confidence field is a bit tricky: more
work is probably needed in the next future to tweak/fix/improve the logic.
2022-01-11 15:23:39 +01:00
Alfredo Cardigliano 23a4761276 Update copyright 2022-01-03 11:00:45 +01:00
Ivan Nardi 7cba34a1e3 Improve IPv6 support, enabling IPv6 traffic on (almost) all dissectors. (#1380)
There are no valid reasons for a (generic) protocol to ignore IPv6
traffic.
Note that:
* I have not found the specifications of "CheckPoint High Availability
Protocol", so I don't know how/if it supports IPv6
* all LRU caches are still IPv4 only

Even if src_id/dst_id stuff is probably useless (see #1279), the right
way to update the protocol classification is via `ndpi_set_detected_protocol()`
2021-12-04 13:30:10 +01:00
Ivan Nardi afc2b641eb Fix writes to flow->protos union fields (#1354)
We can write to `flow->protos` only after a proper classification.

This issue has been found in Kerberos, DHCP, HTTP, STUN, IMO, FTP,
SMTP, IMAP and POP code.
There are two kinds of fixes:
 * write to `flow->protos` only if a final protocol has been detected
 * move protocol state out of `flow->protos`
The hard part is to find, for each protocol, the right tradeoff between
memory usage and code complexity.

Handle Kerberos like DNS: if we find a request, we set the protocol
and an extra callback to further parsing the reply.

For all the other protocols, move the state out of `flow->protos`. This
is an issue only for the FTP/MAIL stuff.

Add DHCP Class Identification value to the output of ndpiReader and to
the Jason serialization.

Extend code coverage of fuzz tests.

Close #1343
Close #1342
2021-11-15 16:20:57 +01:00
Ivan Nardi 730c2360bd Remove struct ndpi_packet_struct from struct ndpi_flow_struct (#1319)
There are no real reasons to embed `struct ndpi_packet_struct` (i.e. "packet")
in `struct ndpi_flow_struct` (i.e. "flow"). In other words, we can avoid
saving dissection information of "current packet" into the "flow" state,
i.e. in the flow management table.

The nDPI detection module processes only one packet at the time, so it is
safe to save packet dissection information in `struct ndpi_detection_module_struct`,
reusing always the same "packet" instance and saving a huge amount of memory.
Bottom line: we need only one copy of "packet" (for detection module),
not one for each "flow".

It is not clear how/why "packet" ended up in "flow" in the first place.
It has been there since the beginning of the GIT history, but in the original
OpenDPI code `struct ipoque_packet_struct` was embedded in
`struct ipoque_detection_module_struct`, i.e. there was the same exact
situation this commit wants to achieve.

Most of the changes in this PR are some boilerplate to update something
like "flow->packet" into something like "module->packet" throughout the code.
Some attention has been paid to update `ndpi_init_packet()` since we need
to reset some "packet" fields before starting to process another packet.

There has been one important change, though, in ndpi_detection_giveup().
Nothing changed for the applications/users, but this function can't access
"packet" anymore.
The reason is that this function can be called "asynchronously" with respect
to the data processing, i.e in context where there is no valid notion of
"current packet"; for example ndpiReader calls it after having processed all
the traffic, iterating the entire session table.

Mining LRU stuff seems a bit odd (even before this patch): probably we need
to rethink it, as a follow-up.
2021-10-05 15:49:52 +02:00
Luca Deri 61fc5be202 Reworked flow risk implementation 2021-07-23 17:27:15 +02:00
Luca Deri 8761ac1887 Renamed Skyp in Skype_Teams as the protocol is now shared across these apps 2021-06-02 21:47:01 +02:00
Luca Deri a31bd5ac3c Cleaned up tls/quic datatypes 2021-01-21 19:17:33 +01:00
Luca Deri 68b6ac7da8 (C) Update 2021-01-07 11:13:36 +01:00
Luca Deri 8413faf2ef Boundary fix 2020-09-25 09:56:24 +02:00
Luca Deri 044ed14b4f Various optimizations to reduce not-necessary calls
Optimized various UDP dissectors
Removed dead protocols such as pando and pplive
2020-09-24 23:26:03 +02:00
Luca Deri 3ba7667887 Various ethereum improvements 2020-01-08 22:01:45 +01:00
Luca Deri 29dd45838d Updated (C) 2020-01-05 18:24:58 +01:00
Luca Deri da029607a0 Removed (unreliable) line protocol detection 2019-11-18 21:21:52 +01:00
Luca Deri ab9246f51f Adedd DTLS check in STUN
Uodated (C)
2019-09-21 09:16:56 +02:00
Luca cce8a6026f Reworked SSL/TLS field naming 2019-08-08 15:20:05 +02:00
Luca 419160f351 Implemented DTLS support
Renamed ssl to tls
2019-08-08 12:45:27 +02:00
Luca a0be9ea639 Fixed wrong detection of RTP as WhatsApp voice 2019-07-29 16:28:56 +02:00
Luca Deri 1777841ec1 Line dissection improvements 2019-07-16 00:02:34 +02:00
Luca Deri 5a018e0bc0 Added Line protocol dissection
Add fix for discarding STUN over TCP flows
2019-07-15 00:19:45 +02:00
Luca 667e77c609 Major code cleanup
Converted some not popular protocols to NDPI_PROTOCOL_GENERIC with category detection
2018-07-21 16:02:54 +02:00
Luca Deri 564d622cc9 Added ndpi_fill_protocol_category call
Introduced intitial Skype in/out support
2018-05-11 13:48:14 +02:00
Luca Deri 6a9fc935eb Added fix for invalid RTP/Skype detection
Added change in config.h generated by nDPI to avoid name clashes (the GNU autotools are really outdated)
2018-02-21 08:18:37 +01:00
Luca Deri 06bb9a9c57 Remove too strict check for RTP 2018-02-21 07:43:59 +01:00
Michele Campus cb8f4f87bf Merge branch 'dev' into dev 2017-12-07 22:45:28 +01:00
Luca Deri 8547b50be7 Merge pull request #493 from evengx/dev
Fixed ndpi_win32.h so that it compiles on MinGW+GCC
2017-11-28 17:51:53 +01:00
Jacob Masen-Smith 950f5db3dd Changed MinGW __forceinline strategy to something with less side effects 2017-11-23 20:40:43 -08:00
Luca 8eaf0cefa1 Warning fix 2017-11-23 15:39:20 +01:00
Vitaly Lavrov 2787c2390c Refactoring the debugging output.
levels of debug output:
 0 - ERROR: Only for errors.
 1 - TRACE: Start of each packets and if found protocol.
 2 - DEBUG: Start of searching each protocol and excluding protocols.
 3 - DEBUG_EXTRA: For all other messages.

Added field ndpi_struct->debug_logging for enable debug output of each protocols.

Simple macros for debugging output are added:
NDPI_LOG_ERR(), NDPI_LOG_INFO(), NDPI_LOG_DBG(), NDPI_LOG_DBG2(),
NDPI_EXCLUDE_PROTO()
2017-10-26 20:41:22 +03:00
Luca Deri e6b594a626 Fixed TINC bug (cache usage)
Merged MS Lync with Skype (Microsoft renamed MS Lync in Skype for Business)
Renumbered Nintendo protocols in former MS Lync that was no longer used
Fix for #425
2017-07-27 00:18:15 +02:00
theirix fb3fc0c6de Fixed buffer overflows with safe str search
1. Detected a lot of memory errors using address sanitizer
and ndpi-scapy tool.

2. Added ndpi_match_prefix function that compares strings
with taking care of payload packet len. Almost drop-in
replacement for match_first_bytes function.

3. Replaced unsafe match_first_bytes usage with a ndpi_match_prefix
and additional length checks.
2016-04-12 22:08:30 +03:00
Luca Deri 169c837613 Fix for https://github.com/ntop/nProbe/issues/7 for wrong RTP identification 2015-11-18 10:46:58 +01:00
Luca Deri 25631a25f7 Implemented MS Lync support 2015-10-22 00:32:48 +02:00
Luca 5266c726f2 Reworked protocol initialization. Work in progress (more cleanup is needed) 2015-07-08 11:58:55 -07:00
Luca 60884f9047 Split former protocol into upper and lower protocol 2015-07-01 17:40:14 +02:00
Luca 07ebed7fae Removed commented code
Renamed ndpi_int_add_connection() with ndpi_set_detected_protocol()
2015-07-01 16:45:29 +02:00
Michele Campus 8da7af7b7b deleted protocol type (real-correlated) 2015-06-29 16:24:51 +02:00