Commit Graph
45 Commits
Author SHA1 Message Date
Ivan Nardi 29c5cc39fb Some small changes (#1869)
All dissector callbacks should not be exported by the library; make static
some other local functions.
The callback logic in `ndpiReader` has never been used.
With internal libgcrypt, `gcry_control()` should always return no
errors.
We can check `categories` length at compilation time.
2023-01-25 11:44:09 +01:00
Ivan Nardi ad6bfbad4d Add protocol disabling feature (#1808)
The application may enable only some protocols.
Disabling a protocol means:
*) don't register/use the protocol dissector code (if any)
*) disable classification by-port for such a protocol
*) disable string matchings for domains/certificates involving this protocol
*) disable subprotocol registration (if any)

This feature can be tested with `ndpiReader -B list_of_protocols_to_disable`.

Custom protocols are always enabled.

Technically speaking, this commit doesn't introduce any API/ABI
incompatibility. However, calling `ndpi_set_protocol_detection_bitmask2()`
is now mandatory, just after having called `ndpi_init_detection_module()`.

Most of the diffs (and all the diffs in `/src/lib/protocols/`) are due to
the removing of some function parameters.

Fix the low level macro `NDPI_LOG`. This issue hasn't been detected
sooner simply because almost all the code uses only the helpers `NDPI_LOG_*`
2022-12-18 08:10:57 +00:00
Ivan Nardi 0a47f745cc Avoid useless host automa lookup (#1724)
The host automa is used for two tasks:
* protocol sub-classification (obviously);
* DGA evaluation: the idea is that if a domain is present in this
automa, it can't be a DGA, regardless of its format/name.

In most dissectors both checks are executed, i.e. the code is something
like:

```
ndpi_match_host_subprotocol(..., flow->host_server_name, ...);
ndpi_check_dga_name(..., flow->host_server_name,...);

```

In that common case, we can perform only one automa lookup: if we check the
sub-classification before the DGA, we can avoid the second lookup in
the DGA function itself.
2022-09-05 13:59:51 +02:00
Ivan Nardi bd036f96f9 Netbios, CSGO: fix two memory errors (#1413)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43754
https://oss-fuzz.com/testcase-detail/5329842395021312
2022-01-18 19:11:01 +01:00
Ivan Nardi 20b5f6d7cc Improve IPv6 support, enabling IPv6 traffic on (almost) all dissectors. (#1406)
Follow-up of 7cba34a1
2022-01-15 17:34:56 +01:00
Ivan Nardi 3a087e951d Add a "confidence" field about the reliability of the classification. (#1395)
As a general rule, the higher the confidence value, the higher the
"reliability/precision" of the classification.

In other words, this new field provides an hint about "how" the flow
classification has been obtained.
For example, the application may want to ignore classification "by-port"
(they are not real DPI classifications, after all) or give a second
glance at flows classified via LRU caches (because of false positives).

Setting only one value for the confidence field is a bit tricky: more
work is probably needed in the next future to tweak/fix/improve the logic.
2022-01-11 15:23:39 +01:00
Alfredo Cardigliano 23a4761276 Update copyright 2022-01-03 11:00:45 +01:00
Ivan Nardi a8ffcd8bb0 Rework how hostname/SNI info is saved (#1330)
Looking at `struct ndpi_flow_struct` the two bigger fields are
`host_server_name[240]` (mainly for HTTP hostnames and DNS domains) and
`protos.tls_quic.client_requested_server_name[256]`
(for TLS/QUIC SNIs).

This commit aims to reduce `struct ndpi_flow_struct` size, according to
two simple observations:
 1) maximum one of these two fields is used for each flow. So it seems safe
to merge them;
 2) even if hostnames/SNIs might be very long, in practice they are rarely
longer than a fews tens of bytes. So, using a (single) large buffer is a
waste of memory for all kinds of flows. If we need to truncate the name,
we keep the *last* characters, easing domain matching.

Analyzing some real traffic, it seems safe to assume that the vast
majority of hostnames/SNIs is shorter than 80 bytes.

Hostnames/SNIs are always converted to lowercase.

Attention was given so as to be sure that unit-tests outputs are not
affected by this change.

Because of a bug, TLS/QUIC SNI were always truncated to 64 bytes (the
*first* 64 ones): as a consequence, there were some "Suspicious DGA
domain name" and "TLS Certificate Mismatch" false positives.
2021-11-24 10:46:48 +01:00
Ivan Nardi 730c2360bd Remove struct ndpi_packet_struct from struct ndpi_flow_struct (#1319)
There are no real reasons to embed `struct ndpi_packet_struct` (i.e. "packet")
in `struct ndpi_flow_struct` (i.e. "flow"). In other words, we can avoid
saving dissection information of "current packet" into the "flow" state,
i.e. in the flow management table.

The nDPI detection module processes only one packet at the time, so it is
safe to save packet dissection information in `struct ndpi_detection_module_struct`,
reusing always the same "packet" instance and saving a huge amount of memory.
Bottom line: we need only one copy of "packet" (for detection module),
not one for each "flow".

It is not clear how/why "packet" ended up in "flow" in the first place.
It has been there since the beginning of the GIT history, but in the original
OpenDPI code `struct ipoque_packet_struct` was embedded in
`struct ipoque_detection_module_struct`, i.e. there was the same exact
situation this commit wants to achieve.

Most of the changes in this PR are some boilerplate to update something
like "flow->packet" into something like "module->packet" throughout the code.
Some attention has been paid to update `ndpi_init_packet()` since we need
to reset some "packet" fields before starting to process another packet.

There has been one important change, though, in ndpi_detection_giveup().
Nothing changed for the applications/users, but this function can't access
"packet" anymore.
The reason is that this function can be called "asynchronously" with respect
to the data processing, i.e in context where there is no valid notion of
"current packet"; for example ndpiReader calls it after having processed all
the traffic, iterating the entire session table.

Mining LRU stuff seems a bit odd (even before this patch): probably we need
to rethink it, as a follow-up.
2021-10-05 15:49:52 +02:00
Luca Deri e5dacc28e7 Minor fix 2021-07-05 22:02:46 +02:00
Luca Deri 48db44a567 Code cleanup 2021-07-05 21:45:30 +02:00
Luca Deri 2eba77e3b0 NetBIOS decoding changes 2021-06-16 19:57:20 +02:00
Luca Deri 1ec621c85b Added further checks 2021-05-15 19:53:46 +02:00
Luca Deri 153d11c920 Enhanced netbios decoding check 2021-04-28 22:58:22 +02:00
Luca Deri 68b6ac7da8 (C) Update 2021-01-07 11:13:36 +01:00
Luca Deri 7086197047 Added extension to detect nested subdomains as used in Browsertunnel attack tool
https://github.com/veggiedefender/browsertunnel
2020-09-09 23:25:19 +02:00
Luca Deri 91d132ea3d Check for avoiding buffer overflow in netbios dissector 2020-07-14 22:16:43 +02:00
Luca Deri 46d96e7f32 Added checks for DGA detection 2020-06-17 19:46:37 +02:00
Luca Deri 55364ef0b4 Added DGA risk for names that look like a DGA 2020-06-11 18:51:53 +02:00
Luca Deri 287015ffdc Fixed invalid check 2020-04-15 22:42:09 +02:00
Philippe Antoine 4f370fe7c4 Adds netbios bound check 2020-04-15 16:40:00 +02:00
Simone Mainardi e282e43279 Fixes netbios overread
This commit inherits from https://github.com/ntop/nDPI/commit/7a2bcd9c395f9fe554109e04add33e9e65564d82
but leaves ndpi_netbios_name_interpret as part of the API as it
is used by ntopng
2020-03-12 22:21:07 +01:00
Simone Mainardi 2f15ce706f Fixed API change which removed ndpi_netbios_name_interpret
This reverts commit 7a2bcd9c39.
2020-03-12 22:08:59 +01:00
Philippe Antoine 7a2bcd9c39 Fix buffer overread in netbios 2020-03-12 14:22:46 +01:00
Luca Deri 7fd135a8c9 Added SMBv1 over NetBIOS detection 2020-03-01 11:32:09 +01:00
Luca Deri ef16591f3f Merge branch 'dev' of https://github.com/ntop/nDPI into dev 2020-01-05 18:25:44 +01:00
Luca Deri 29dd45838d Updated (C) 2020-01-05 18:24:58 +01:00
Luca 257ec7cc5f Removed disable_metadata_export preference that is no longer useful
since ndpi_process_extra_packet() can drive limited or full metadata export
2019-12-29 08:07:35 +01:00
Luca Deri 24a26a2e94 Added NetBIOS metadata export 2019-10-08 23:40:56 +02:00
Luca Deri ab9246f51f Adedd DTLS check in STUN
Uodated (C)
2019-09-21 09:16:56 +02:00
Luca Deri 669fdf6b5f Improved skype, teredo, netbios heuristics
Changed ndpi_detection_giveup() API: guess is now part of the call
2018-11-21 00:32:32 +01:00
Luca 667e77c609 Major code cleanup
Converted some not popular protocols to NDPI_PROTOCOL_GENERIC with category detection
2018-07-21 16:02:54 +02:00
Luca Deri 1076455c01 Added ndpi_set_detection_preferences() APi call 2018-05-14 21:40:27 +02:00
Vitaly Lavrov 2787c2390c Refactoring the debugging output.
levels of debug output:
 0 - ERROR: Only for errors.
 1 - TRACE: Start of each packets and if found protocol.
 2 - DEBUG: Start of searching each protocol and excluding protocols.
 3 - DEBUG_EXTRA: For all other messages.

Added field ndpi_struct->debug_logging for enable debug output of each protocols.

Simple macros for debugging output are added:
NDPI_LOG_ERR(), NDPI_LOG_INFO(), NDPI_LOG_DBG(), NDPI_LOG_DBG2(),
NDPI_EXCLUDE_PROTO()
2017-10-26 20:41:22 +03:00
Luca Deri 760d468aca Improved netbios name decoding 2017-03-19 23:42:36 +01:00
Luca Deri 6a5b380bc7 Exported ndpi_netbios_name_interpret() to all nDPI apps 2017-03-19 22:51:13 +01:00
Luca Deri 689d3e7602 Improved netbios dissection 2017-03-19 22:38:13 +01:00
Luca Deri 19324f2904 Enhanced string boundary check to avoid crashes with malformed packets 2016-06-15 06:12:41 +02:00
Luca Deri a46c12c014 Fixed string offset 2015-11-22 23:28:50 +01:00
Luca Deri d24093c4a6 Improvements on netbios name detection 2015-11-22 23:13:50 +01:00
Luca 5266c726f2 Reworked protocol initialization. Work in progress (more cleanup is needed) 2015-07-08 11:58:55 -07:00
Luca 60884f9047 Split former protocol into upper and lower protocol 2015-07-01 17:40:14 +02:00
Luca 07ebed7fae Removed commented code
Renamed ndpi_int_add_connection() with ndpi_set_detected_protocol()
2015-07-01 16:45:29 +02:00
Michele Campus 8da7af7b7b deleted protocol type (real-correlated) 2015-06-29 16:24:51 +02:00
Luca Deri 2e5ceac844 Initial import from SVN 2015-04-19 07:25:59 +02:00