Commit Graph
709 Commits
Author SHA1 Message Date
Toni d8d806c79f Force roaring bitmap to use ndpi memory wrappers. (#1569)
GCC analyzer won't complain about possible use-after-free (false positive).

 * tests/do.sh prints word diff's only once and not the same over and over again
 * sync unit tests

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-31 12:06:01 +02:00
Luca Deri 354addd693 Updated risk results 2022-05-30 23:28:59 +02:00
Toni 00ee364a7e Updated README.md (#1562)
* make check great again (not so much)
 * make doc/doc-view
 * CI updates

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-30 19:48:05 +02:00
Toni 48065d80e9 Support word diff for tests/do.sh for better readability. (#1565)
* Sync unit tests

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-30 17:33:03 +02:00
Toni 32750271c3 Prohibit MPEG-DASH to set HTTP as application protocol. (#1560)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-30 14:54:27 +02:00
Ivan Nardi 3b825fca6d Certificate timestamps should be printed in UTC (#1563)
This way, the CI results are consistent
2022-05-30 13:43:05 +02:00
Luca Deri 897f8314b2 Fixed dispay bug for risk_info 2022-05-30 01:12:38 +02:00
Luca Deri 1da9f1a36f Updated tests results
Code cleanup
2022-05-30 00:54:17 +02:00
Ivan Nardi 9c1a53f39f Dazn: add support for Dazn streaming service (#1559)
Update .gitignore file
2022-05-29 17:47:16 +02:00
Toni 33f9729ee4 Added MPEG-DASH dissector. Fixes #1223. (#1555)
* Improved HTTP POST detection
 * Refactored subprotocol detection

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-29 13:12:13 +02:00
Toni cfd23e8819 Fixed HTTP lower/upper protocol mess for Aimini/IPP. (#1557)
* If HTTP was already detected, there is no need to overwrite the detection again

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-28 16:31:35 +02:00
Ivan Nardi f81f0c5e69 Sync unit test results (#1554) 2022-05-26 20:15:30 +02:00
Luca Deri cedbd32030 Updated SkypeCall -> Skype_TeamsCall 2022-05-26 19:27:12 +02:00
Toni 78dfe959cb Added script to compare and verify the output of `make dist'. (#1551)
This fixes some build/test issues resulting when using tarballs.

 * nDPI uses autotools (especially autoconf) in a wrong way, see #1163

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-26 15:07:22 +02:00
Luca Deri 4f9dee164e Improved detection of invalid SNI and hostnames in TLS, HTTP 2022-05-24 19:47:54 +02:00
Toni Uhlig 7162bf6abf Moved mgcp.pcapng to tests/pcap/ instead of tests/
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-05-16 16:54:18 +02:00
Toni 054d151373 Improved Viber (TCP) detection. (#1547)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-10 21:37:03 +02:00
Toni 704920414a Improved Xiaomi HTTP detection. (#1546)
* Merged Xiaomi pcap files

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-10 11:10:04 +02:00
Toni 6fc29b3ae8 Removed TLS patterns in the CiscoVPN aka Anyconnect dissector as mentioned in PR #1534. (#1543)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-09 10:29:52 +02:00
Toni 915ffebade Added Softether(-VPN) DDNS service detection. (#1544)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-09 08:16:19 +02:00
Toni 4319d760e0 Improved TLS alert detection. (#1542)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-08 20:33:59 +02:00
Toni 34882d9cf0 Improved TLS application data detection. (#1541)
* #1532 did fx TLS appdata detection only partially
 * use flow->l4.tcp.tls.message.buffer_used instead of packet->payload

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-08 19:56:08 +02:00
Toni b2648a45a3 Added Edgecast and Cachefly CDNs. (#1540)
* Improved ASN update script
 * Ran `utils/update_every_lists.sh'
 * `tests/do.sh.in' prints the amount of failed pcap(s)
 * `utils/asn_update.sh' prints the amount of failed download(s)

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-07 09:33:25 +02:00
Toni 87f93ea4fd Replaced ndpiReader's libjson-c support with libnDPI's internal serialization interface. (#1535)
* Fixes #1528
 * Serialization Interface should also fuzzed
 * libjson-c may only be used in the unit test to verify the internal serialization interface
 * Serialization Interface supports tlv(broken), csv and json
 * Unit test does work again and requires libjson-c

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-07 09:26:09 +02:00
Ivan Nardi 2e0dedbaae Fix compilation (if --enable-debug-messages is used) (#1539) 2022-05-04 18:43:37 +02:00
Luca Deri 5dcb9c1c16 Modified risk labels 2022-05-04 12:10:01 +02:00
Toni 47d6a65522 Improved suspicious http user agent detection. (#1537)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-02 19:17:32 +02:00
Luca Deri c8f72ef764 Added ndpi_get_flow_error_code() API call
Fixed typo
2022-05-02 10:51:08 +02:00
Ivan Nardi 71636dcafd Sync unit tests results (#1533) 2022-04-27 18:22:11 +02:00
Toni 10161448bc Improved TLS application data detection. (#1532)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-27 17:22:53 +02:00
Toni bc2ad3407a Added generic user agent setter. (#1530)
* ndpiReader: Print user agent if one was set and not just for certain protocols.

Signed-off-by: lns <matzeton@googlemail.com>
2022-04-25 13:00:50 +02:00
Ivan Nardi 075bce5f3d XIAOMI: add detection of Xiaomi traffic (#1529)
Most of the credits should go to @utoni (see #1521)
2022-04-25 11:00:02 +02:00
Toni ac0f50b561 Added RakNet protocol dissector. (#1527)
* Frame Set PDU's do not get fully dissected for the sake of simplicity

Signed-off-by: lns <matzeton@googlemail.com>
2022-04-24 13:08:57 +02:00
Toni d5ae9170ba Tiny gnutella improvement if gtk-gnutella used. (#1525)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-22 18:33:27 +02:00
Toni 824c343602 Updated `utils/whatsapp_ip_addresses_download.sh' to scrape the required IP addresses/ranges. (#1524)
* Replaced return statements in `utils/*.sh' with exit's (such scripts should never source'd)
 * Ran `utils/update_every_lists.sh'

Signed-off-by: lns <matzeton@googlemail.com>
2022-04-22 13:27:41 +02:00
Ivan Nardi 42909673ce Add some scripts to easily update some IPs lists (#1522)
Follow-up of 8b062295

Add a new protocol id for generic Tencent/Wechat flows
2022-04-21 20:43:52 +02:00
Toni c1d46ebc4e Added proprietary Agora Software Defined Real-time Network (SD-RTN) protocol dissector. (#1520)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-20 18:31:28 +02:00
Toni dbf2fb8c9f Added Toca Boca protocol dissector. (#1517)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-19 19:32:49 +02:00
Toni fa79f07d15 Improved sflow protocol detection false-positives. (#1518)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-19 17:46:40 +02:00
Toni Uhlig c3df3a12aa Fixed msys2 build warnings and re-activated CI Mingw64 build.
* Removed Visual Studio leftovers. Maintaining an autotools project with VS integration requires some additional overhead.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-14 19:17:48 +02:00
Toni e3faecf9de Improved ASN.1 parsing for Keberos. Fixes #1492. (#1497)
* This is a quick fix, the Kerberos protocol dissector requires some refactoring effort.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-10 10:36:10 +02:00
Luca Deri 9f3a6f131b update 2022-04-01 19:07:46 +02:00
Ivan Nardi 6bcba118d4 reader_util: add support for userAgent in SSDP (#1502)
Update unit tests results
Follow-up of d668ab4b
2022-03-28 04:25:34 +02:00
Ivan Nardi 6acf7a6abe Add support for Pluralsight site (#1503) 2022-03-27 15:13:12 +02:00
Ivan Nardi ec80f1fb3b Fix CI tests results (#1504)
CI integration is failing since 856d7d2.
2022-03-27 15:12:39 +02:00
Darryl Sokoloski d668ab4bec [SSDP] Extract HTTP user-agent when available. (#1500)
[SSDP] Added capture file with UA header.
[SSDP] Added pcap test output log file.

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2022-03-27 11:10:44 +02:00
Ivan Nardi aeb5f1f947 QUIC: add support for version 2 draft 01 (#1493)
Support for v2-00 has been removed (it has never been used in real
networks and it is incompatible with v2-01).

Chrome already supports v2-01 in latest versions in Chrome Beta channel.
2022-03-25 10:16:30 +01:00
Vinicius Silva Nogueira 26df1403e6 Trying to improve QUIC reassembler (#1195) (#1489)
* handling QUIC out-of-order fragments

* minor fix

* updated quic_frags_ch_out_of_order_same_packet_craziness.pcapng.out

* quic test: buf_len + last_pos

* QUIC: comment update in __reassemble function and minor change is_ch_complete function
2022-03-24 10:51:31 +01:00
Ivan Nardi b83e9f20e7 QUIC: fix dissection of draft-34 (#1484)
QUIC-34 is probably not used in production, but fixing it is trivial and
it doesn't add any noise to the already complex QUIC code.
2022-03-09 22:37:56 +01:00
Ivan Nardi 7aee856aa0 Extend tests coverage (#1476)
Now there is at least one flow under `tests/pcap` for 249 protocols out
of the 284 ones supported by nDPI.

The 35 protocols without any tests are:

* P2P/sharing protocols: DIRECT_DOWNLOAD_LINK, OPENFT, FASTTRACK,
EDONKEY, SOPCAST, THUNDER, APPLEJUICE, DIRECTCONNECT, STEALTHNET

* games: CSGO, HALFLIFE2, ARMAGETRON, CROSSFIRE, DOFUS, FIESTA,
FLORENSIA, GUILDWARS, MAPLESTORY, WORLD_OF_KUNG_FU

* voip/streaming: VHUA, ICECAST, SHOUTCAST, TVUPLAYER, TRUPHONE

* other: AYIYA, SOAP, TARGUS_GETDATA, RPC, ZMQ, REDIS, VMWARE, NOE,
LOTUS_NOTES, EGP, SAP

Most of these protocols (expecially the P2P and games ones) have been
inherited by OpenDPI and have not been updated since then: even if they
are still used, the detection rules might be outdated.

However code coverage (of `lib/protocols`) only increases from 65.6% to
68.9%.

Improve Citrix, Corba, Fix, Aimini, Megaco, PPStream, SNMP and Some/IP
dissection.
Treat IPP as a HTTP sub protocol.
Fix Cassandra false positives.

Remove `NDPI_PROTOCOL_QQLIVE` and `NDPI_PROTOCOL_REMOTE_SCAN`:
these protocol ids are defined but they are never used.

Remove Collectd support: its code has never been called. If someone is
really interested in this protocol, we can re-add it later, updating the
dissector.

Add decoding of PPI (Per-Packet Information) data link type.
2022-03-09 22:37:35 +01:00