Commit Graph
1096 Commits
Author SHA1 Message Date
Nardi Ivan d72a760ac3 New API for library configuration
This is the first step into providing (more) configuration options in nDPI.

The idea is to have a simple way to configure (most of) nDPI: only one
function (`ndpi_set_config()`) to set any configuration parameters
(in the present or on in the future) and we try to keep this function
prototype as agnostic as possible.

You can configure the library:
* via API, using `ndpi_set_config()`
* via a configuration file, in a text format

This way, anytime we need to add a new configuration parameter:
* we don't need to add two public functions (a getter and a setter)
* we don't break API/ABI compatibility of the library; even changing
the parameter type (from integer to a list of integer, for example)
doesn't break the compatibility.

The complete list of configuration options is provided in
`doc/configuration_parameters.md`.

As a first example, two configuration knobs are provided:
* the ability to enable/disable the extraction of the sha1 fingerprint of
the TLS certificates.
* the upper limit on the number of packets per flow that will be subject
to inspection
2024-01-18 10:21:24 +01:00
Ivan Nardi 61a18623e7 Bittorrent: improve detection of UTPv1 (#2259)
* fix header length check: we must take into account optional extension
header
* it seems the check on UTPv1 header is weak only in the DATA case: wait
for multiple packets only in that case
2024-01-12 18:53:24 +01:00
Toni 0aea509e23 Add KCP protocol dissector. (#2257)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-12 12:14:59 +01:00
Toni 7b2bbb2309 Improved uTorrent via utp (TCP-like streams over UDP). (#2255)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-11 16:49:25 +01:00
Vladimir Gavrilov 3d57dec6b4 Add PIA (Private Internet Access) support (#2250)
* Fix typo in protocols.rst

* Add PIA (Private Internet Access) support

* Update ndpi_main.c
2024-01-09 18:28:57 +01:00
Toni c5b0b05b80 Add Roughtime protocol dissector. (#2248)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2024-01-09 07:56:27 +01:00
Ivan Nardi 8321b79539 Make some test traces smaller (#2243)
Having smaller traces help fuzzing: we want the fuzzers to mutate
"interesting" packets analyzed by nDPI, i.e. the first packets of each
flows.
Try hard to keep the same classification and extraction capabilities
2024-01-08 17:04:12 +01:00
Vladimir Gavrilov ce08291ccd Add Google Chat support (#2244) 2024-01-07 18:04:59 +01:00
Ivan Nardi 40797521af ndpiReader: add breed stats on output used for CI (#2236) 2024-01-05 13:02:39 +01:00
Vladimir Gavrilov 3d09b25653 Add Ceph protocol dissector (#2242)
* Add Ceph protocol dissector

* Update protocols.rst
2024-01-04 13:22:23 +01:00
Vladimir Gavrilov 7f9973bd0c Add HL7 protocol dissector (#2240)
* Add HL7 protocol dissector

* Small fixes

* Small fixes
2024-01-02 20:57:05 +01:00
Ivan Nardi 88d1416b70 STUN: fix detection of Google Meet over IPv6 (#2241) 2024-01-02 19:30:59 +01:00
Vladimir GavrilovandNardi Ivan 0180c1f04a Add IEC62056 (DLMS/COSEM) protocol dissector (#2229)
* Add IEC62056 (DLMS/COSEM) protocol dissector

* Fix detection on big endian architectures

* Update protocols.rst

* Add ndpi_crc16_x25 to fuzz/fuzz_alg_crc32_md5.c

* Update pcap sample

* Remove empty .out file

* iec62056: add some documentation

---------

Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2024-01-02 16:45:54 +01:00
Vladimir Gavrilov 0f4d9f5054 Remove Google Hangouts/Duo stuff (#2233)
* Remove Google Hangouts/Duo support

* Update protocols.rst
2024-01-02 14:01:33 +01:00
Ivan Nardi d886a6107f Teamviewer: varius fixes (#2228)
We already have a generic (and up to date) logic to handle ip addresses:
remove that stale list.

Teamviewer uses TCP and UDP, both; we can't access `flow->l4.udp`.

According to a comment, we set the flow risk
`NDPI_DESKTOP_OR_FILE_SHARING_SESSION` only for the UDP flows.
2024-01-02 11:22:43 +01:00
Vladimir Gavrilov 2796bc9b47 Add NoMachine NX protocol dissector (#2234)
* Add NoMachine protocol dissector

* Fix detection on big endian architectures

* Make NoMachine over UDP check more strict

* Small fixes
2024-01-02 10:23:42 +01:00
Luca Deri 7c4be6d077 Updated result 2023-12-22 21:29:45 +01:00
Luca Deri 8285fffdae Implements JA4 Support (#2191) 2023-12-22 20:40:42 +01:00
Vladimir Gavrilov 5eb468d07b Add Apache Kafka protocol dissector (#2226) 2023-12-22 14:42:47 +01:00
Vladimir Gavrilov 6fc8aa4e61 Add WebDAV detection support (#2224)
* Add WebDAV detection support

* Add pcap example

* Update test results

* Remove redundant checks

* Add WebDAV related HTTP methods to fuzz/dictionary.dict

* Add note about WebDAV
2023-12-22 13:23:37 +01:00
Vladimir Gavrilov 149067b3fc Add JSON-RPC protocol dissector (#2217)
* Add JSON-RPC protocol dissector

* Small fixes

* Improve detection
2023-12-20 12:42:25 +01:00
Vladimir Gavrilov 33f11cb10f Add OpenFlow protocol dissector (#2222) 2023-12-20 10:48:45 +01:00
Ivan Nardi 8aa09f9c99 mining: a better identification logic (#2221)
It is quite simple (and not so efficient) but it should fix all the
false positives reported in #2216. Add support for Ethereum mining.

Merge all the mining traces.

Remove duplicated function.

Close #2216
2023-12-20 10:46:57 +01:00
Ivan Nardi 308b266333 fuzz: improve fuzzing coverage (#2220) 2023-12-19 20:33:08 +01:00
Vladimir Gavrilov 59c8eabc0e Add UFTP protocol dissector (#2215)
* Add UFTP protocol dissector

* Update docs

* Merge pcap files
2023-12-18 11:21:07 +01:00
Vladimir Gavrilov d8c7a76611 Add HiSLIP protocol dissector (#2214)
* Add HiSLIP protocol dissector

* Fix error
2023-12-17 11:52:55 +01:00
Vladimir Gavrilov 0f3e6d832b Add PROFINET/IO protocol dissector (#2213)
* Add PROFINET/IO protocol dissector

* Add LE (Little Endian) to the file name

* Rework dissector

* Remove redundant check
2023-12-16 13:30:21 +01:00
Toni ef62391dba Add Monero protocol classification. (#2196)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2023-12-13 19:55:18 +01:00
Ivan Nardi bda037c57f Sync unit tests results after latest merge (#2211) 2023-12-13 17:47:20 +01:00
Ivan Nardi 193f28582b QUIC: add heuristic to detect unidirectional *G*QUIC flows (#2207)
Fix extraction of `flow->protos.tls_quic.quic_version` metadata.
2023-12-13 17:14:04 +01:00
Ivan Nardi 241c42ad7e ndpiReader: fix guessed_flow_protocols statistic (#2203)
Increment the counter only if the flow has been guessed
2023-12-12 19:44:03 +01:00
Ivan Nardi b3f2b1bb7f STUN: rework extra dissection (#2202)
Keep looking for RTP packets but remove the monitoring concept.
We will re-introduce a more general concept of "flow in monitoring
state" later.
The function was disabled by default.
Some configuration knobs will be provided when/if #2190 is merged.
2023-12-11 14:53:12 +01:00
Ivan Nardi 673b6e7345 HTTP: faster processing of asymmetric flows (#2198) 2023-12-11 14:52:31 +01:00
Ivan Nardi adf8982d8e fuzz: extend fuzzing coverage (#2205) 2023-12-11 12:48:50 +01:00
Ivan Nardi f74cf16c36 OpenVPN: rework detection (#2199)
Close #1873
2023-12-06 10:24:26 +01:00
Vladimir Gavrilov ad20846fad Add Ether-S-Bus protocol dissector (#2200) 2023-12-05 17:20:38 +01:00
Vladimir Gavrilov be50493f44 Add IEEE C37.118 protocol dissector (#2193) 2023-12-05 08:06:15 +01:00
Vladimir Gavrilov c34bded4ef Add ISO 9506-1 MMS protocol dissector (#2189)
* Add ISO 9506-1 MMS protocol dissector
* Fix detection on big-endian architectures
2023-12-01 09:03:07 +01:00
Ivan Nardi 983b8e8eee STUN: parsing of DATA attribute (#2179) 2023-12-01 07:01:49 +01:00
Vladimir Gavrilov 24df1913ac Add Beckhoff ADS protocol dissector (#2181)
* Add Beckhoff ADS protocol dissector

* Remove redundant le32toh

* Fix detection on big-endian architectures
2023-11-30 09:13:45 +01:00
Ivan Nardi 6f046df0dc STUN: fix detection of DTLS (#2187)
Fix a memory leak
```
==97697==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 16 byte(s) in 1 object(s) allocated from:
    #0 0x55a6967cfa7e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x701a7e) (BuildId: c7124999fa1ccc54346fa7bd536d8eab88c3ea01)
    #1 0x55a696972ab5 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25
    #2 0x55a696972da0 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13
    #3 0x55a696b7658d in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2394:46
    #4 0x55a696b86e81 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:897:5
    #5 0x55a696b80649 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1262:11
    #6 0x55a696b67a57 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2751:5
    #7 0x55a696b67758 in switch_to_tls /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1408:3
    #8 0x55a696c47810 in stun_search_again /home/ivan/svnrepos/nDPI/src/lib/protocols/stun.c:422:4
    #9 0x55a6968a22af in ndpi_process_extra_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7247:9
    #10 0x55a6968acd6f in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7746:5
    #11 0x55a6968aba3f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8013:22
    #12 0x55a69683d30e in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1723:31
    #13 0x55a69683d30e in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2440:10
    #14 0x55a69680f08f in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:135:7
[...]
SUMMARY: AddressSanitizer: 16 byte(s) leaked in 1 allocation(s).
```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64564
2023-11-30 09:09:40 +01:00
Ivan Nardi ac90b1f009 Fix detection of NDPI_TCP_ISSUES flow risk (#2177)
We need to take into account retransmissions: they increase
`flow->all_packets_counter` counter but not `flows->packet_counter`
one.
Therefore, the right way to check for 3WH + RST pattern involves checking
for `flows->packet_counter == 0`
2023-11-29 16:55:39 +01:00
Vladimir Gavrilov c60c03766c Add Schneider Electric’s UMAS detection support (#2180)
* Add Schneider Electric’s UMAS detection support

* Swap proto IDs in ndpi_set_detected_protocol

* Update unit test result
2023-11-28 18:03:00 +01:00
Vladimir Gavrilov ebb1bc2f34 Add Ether-S-I/O protocol dissector (#2174) 2023-11-27 19:04:05 +01:00
Vladimir Gavrilov 84427b0754 Add Omron FINS protocol dissector (#2172)
* Add Omron FINS protocol dissector

* Add a kludge to avoid invalid FINS over UDP detection as SkypeTeams and RTP

* Update unit test results

* Update protocols.rst

* Remove dummy flows from fins.pcap
2023-11-27 17:09:53 +01:00
3763c702f0 Rework S7Comm dissector; add S7Comm Plus support (#2165)
* Rework S7Comm dissector; add S7Comm Plus support

* Cleanup s7comm.c

* Improve S7Comm Plus detection

* s7comm/s7commplus: faster detection

---------

Co-authored-by: 0xA50C1A1 <mage.wizard88@gmail.com>
Co-authored-by: Nardi Ivan <nardi.ivan@gmail.com>
2023-11-27 14:37:48 +01:00
Vladimir Gavrilov 0b6e261523 Improve CORBA detection (#2167)
* Improve CORBA detection

* Remove dummy flow from ziop.pcap

* Merge ziop.pcap and miop.pcap into corba.pcap
2023-11-27 13:10:50 +01:00
Vladimir Gavrilov da629709f3 Add OPC UA protocol dissector (#2169) 2023-11-27 12:13:23 +01:00
Ivan Nardi 7ff22a7e3c STUN: improve demultiplexing of DTLS packets (#2153)
Keep demultiplexing STUN/RTP/RTCP packets after DTLS ones.

We might end up processing the session a little longer, because we will
process the STUN/RTP/RTCP packets after the DTLS handshake.
2023-11-27 11:10:38 +01:00
Vladimir Gavrilov 87399b3544 Add RTPS protocol dissector (#2168) 2023-11-27 07:17:39 +01:00