Compare commits

..
395 Commits
Author SHA1 Message Date
Luca Deri b8806bd2de Moved to 4.0.0 2021-07-26 09:42:48 +02:00
Luca Deri 4cafa7cb1e Improved risk detection mask algorithm 2021-07-26 09:38:53 +02:00
Toni 29ec34f66d Improved TFTP detection. Fixes #1242, #1256 (#1262)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-25 21:54:19 +02:00
Luca Deri a482e1c0be Typo (Thanks Ivan) 2021-07-24 18:12:09 +02:00
Luca Deri 1d9a830b2f Update changelog 2021-07-24 17:54:32 +02:00
Luca Deri 51e4888442 Implemented ALPN automa for checking uncommon ALPNs 2021-07-24 17:50:32 +02:00
Luca Deri 526568fcd5 Risk check improvement 2021-07-24 16:18:18 +02:00
Alfredo Cardigliano c73501f859 Typo 2021-07-23 18:14:11 +02:00
Alfredo Cardigliano d5a60880d0 Fix prototype 2021-07-23 18:12:54 +02:00
Luca Deri 61fc5be202 Reworked flow risk implementation 2021-07-23 17:27:15 +02:00
Ivan Nardi 4ffe1eb3c0 Fix unit tests when "--disable-gcrypt" flag is used (#1255) 2021-07-23 10:37:36 +02:00
Toni 6ad0d6666c Implemented function to retrieve flow information. #1253 (#1254)
* fixed [h]euristic typo

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-23 10:37:20 +02:00
Luca Deri 8ea8ba8e9b Fixed risk mask implementation 2021-07-23 08:29:36 +02:00
Luca Deri b01b60a2b5 Implementation of flow risk eception (work in progress) 2021-07-22 01:35:57 +02:00
Alfredo Cardigliano fc9901292d Update changelog 2021-07-20 17:04:35 +00:00
Toni 32275543c4 Added risk: TLS_EXTENSION_SUSPICIOUS (#1252)
* validates client/server hello TLS extensions
 * inspects content for some extensions

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-19 16:23:24 +02:00
Ivan Nardi 57b8969a3d Fix setting of flow risks on 32 bit machines (#1251)
Since 19a29e1e (NDPI_TLS_CERT_VALIDITY_TOO_LONG is 32), unit tests are
failing on 32 bit machines (i.e Raspberry 4)
2021-07-19 16:22:39 +02:00
Luca Deri 0adcd28450 Updated download category name 2021-07-16 14:04:47 +02:00
Vitaly Lavrov 295cb3ee6a Bug fix. (#1250)
Restore part of the fields flow->packet after cleaning flow structure.
2021-07-16 13:53:36 +02:00
pacantandpacant 488166833e Fixed TLS certificate threshold (#1248)
Co-authored-by: pacant <a.pace97@outlook.com>
2021-07-16 13:52:55 +02:00
Vitaly Lavrov daf1fbfc7b Code cleanup. (#1246)
ndpi_utils.c: use ndpi_malloc,ndpi_calloc,ndpi_free
genshin_impact.c, git.c, hpvirtgrp.c, http.c, z3950.c: removed "#include stdlib.h"
2021-07-16 13:51:54 +02:00
Luca Deri a6de94a930 hulu and netflix dissection improvements 2021-07-14 12:53:03 +02:00
Luca Deri 3eba8cc5b5 Temporary fix for avoiding wireshark errors 2021-07-14 12:51:42 +02:00
Luca Deri b95bd0358f Code cleanup 2021-07-14 11:19:24 +02:00
pacantandpacant 19a29e1e22 TLS Risks - Certificate Validity Too Long (#1239)
* Added flow risk: TLS certificate too long

* Added flow risk: TLS certificate too long

* Date for TLS limit added

* TLS certificate check fixed

Co-authored-by: pacant <a.pace97@outlook.com>
2021-07-14 11:13:22 +02:00
Toni c411df523e Added AVAST SecureDNS protocol. (#1244)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-14 11:11:59 +02:00
Toni 62bae30a4a Improved Steam detection (Steam Datagram Relay - SDR). (#1243)
* improved DNSCrypt midstream detection again (sufficient for all tested use-cases)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-14 11:09:58 +02:00
Toni e4453938d5 Improved dnscrypt midstream detection. (#1241)
* fixed skype false-positive detection of dnscrypt traffic

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-13 15:10:18 +02:00
Ivan Nardi cccf794265 ndpiReader: add statistics about nDPI performance (#1240)
The goal is to have a (roughly) idea about how many packets nDPI needs
to properly classify a flow.

Log this information (and guessed flows number too) during unit tests,
to keep track of improvements/regressions across commits.
2021-07-13 12:28:39 +02:00
Luca Deri 96b71def49 Minor change 2021-07-12 17:51:25 +02:00
Vitaly Lavrov c418b7110b ahoсorasick. Code review. Part 2. (#1236)
Simplified the process of adding lines to AC_AUTOMATA_t.
Use the ndpi_string_to_automa() function to add patterns with domain names.
For other cases can use ndpi_add_string_value_to_automa().

ac_automata_feature(ac_automa, AC_FEATURE_LC) allows adding
and compare data in a case insensitive manner. For mandatory pattern comparison
from the end of the line, the "ac_pattern.rep.at_end=1" flag is used.
This eliminated unnecessary conversions to lowercase and adding "$" for
end-of-line matching in domain name patterns.

ac_match_handler() has been renamed ac_domain_match_handler() and has been greatly simplified.
ac_domain_match_handler() looks for the template with the highest domain level.
For special cases it is possible to manually specify the domain level.
Added test for checking ambiguous domain names like:
 - short.weixin.qq.com is QQ, not Wechat
 - instagram.faae1-1.fna.fbcdn.net is Instagram, not Facebook

If you specify a NULL handler when creating the AC_AUTOMATA_t structure,
then a pattern with the maximum length that satisfies the search conditions will be found
(exact match, from the beginning of the string, from the end of the string, or a substring).

Added debugging for ac_automata_search.
To do this, you need to enable debugging globally using ac_automata_enable_debug(1) and
enable debugging in the AC_AUTOMATA_t structure using ac_automata_name("name", AC_FEATURE_DEBUG).
The search will display "name" and a list of matching patterns.
Running "AHO_DEBUG=1 ndpiReader ..." will show the lines that were searched for templates
and which templates were found.

The ac_automata_dump() prototype has been changed. Now it outputs data to a file.
If it is specified as NULL, then the output will be directed to stdout.
If you need to get data as a string, then use open_memstream().

Added the ability to run individual tests via the do.sh script
2021-07-12 17:39:43 +02:00
Luca Deri 78b1295dc1 Fixed false positives in Z39.50 2021-07-09 00:16:48 +02:00
Luca Deri bec8361638 Updated category labels 2021-07-08 20:05:50 +02:00
Vitaly Lavrov 398ad8e5ff Fix for #1230 (#1235)
* Revert "Fix return value of ndpi_match_string_subprotocol() (#1230)"

This reverts commit 58665e93a9.

* Checking the return code after calling ndpi_match_string_subprotocol()

ndpi_api.h: Description of the returned error codes for
the ndpi_match_string_subprotocol() function.

If the ndpi_match_string_subprotocol() function returned an error,
then return NDPI_PROTOCOL_UNKNOWN.

http: The "Content-type" header is only checked if it is not empty.
2021-07-07 14:32:57 +02:00
Toni 34d0718bca Improved RTSP detection the second. (#1232) (#1233)
* RTSP is no subprotocol of HTTP (most of the time)
 * detection patterns should stay in rtsp.c
 * set detected HTTP protocol only if at least a valid HTTP method detected

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-07 09:55:36 +02:00
Luca Deri c408710c52 Added test pcap 2021-07-06 18:57:52 +02:00
Toni 6f29733262 Improved RTSP via HTTP detection. (#1232)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-06 18:57:04 +02:00
Ivan Nardi 58665e93a9 Fix return value of ndpi_match_string_subprotocol() (#1230) 2021-07-06 18:55:14 +02:00
Luca Deri 1ee7355b91 Updated prototype 2021-07-05 22:35:11 +02:00
Luca Deri e5dacc28e7 Minor fix 2021-07-05 22:02:46 +02:00
Luca Deri 48db44a567 Code cleanup 2021-07-05 21:45:30 +02:00
Alfredo Cardigliano 8e7b063804 Fix possible segfault/leak 2021-07-05 15:09:48 +02:00
Toni 6f7acfcab5 Added protocols to HTTP as subprotocols. (#1228)
* if HTTP was detected,
   all of those defined subprotocol detection routines should called

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-05 09:43:15 +02:00
Ivan Nardi 46c32d2433 TLS: fix a memory error in JA3 code (#1227)
protocols/tls.c:1856:5: runtime error: index 256 out of bounds for type 'char [256]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:1856:5
2021-07-05 09:42:52 +02:00
Ivan Nardi 1116d0e4b1 Improve content match lists (#1226)
Add support for Likee app (https://likee.video/) and GitLab
Improve detection of Google domains ("gtv1.com" added in 6dd42d19f was
likely a typo)
Improve Snapchat and Tiktok detection
2021-07-05 09:42:11 +02:00
Toni 21b2cce715 Improved Z39.50 detection. (#1225)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-05 09:41:35 +02:00
Toni ced6fca184 Fixed off-by-one memory error for TLS-JA3. (#1222)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-06-29 15:33:12 +02:00
Toni 8222ac5bdc Removed ht_hash as it is not used anymore. (#1220)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-06-29 15:32:48 +02:00
Toni 1c2a0c36f1 Added Z39.50 protocol. (#1219)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-06-29 15:32:16 +02:00
vpiserchiaandvpiserchia e5fd86393e reconcile PPSTREAM protocol category with content_match declarations (#1218)
* reconcile PPSTREAM protocol category with content_match declarations

* remove duplicate declaration for PPSTREAM protocol

Co-authored-by: vpiserchia <vito.piserchia@dreamlab.net>
2021-06-28 14:58:42 +02:00
Ivan Nardi f8fe3ee520 QUIC: add basic support for fragmented Client Hello (#1216)
Only in-order and non overlapping fragments are handled
See #1195
2021-06-24 18:30:34 +02:00
Luca Deri 6a1fd9ad97 Added missing check to prevent crashes 2021-06-23 12:17:21 +02:00
Vitaly Lavrov 2234b97149 ndpiReader: memory leak (#1215)
Non-critical bugs.
If a file list is used, then all files except the last are not closed.
Opening the next file loses the memory allocated via pcap_open_offline() for the previous file.
If a bpf filter is used, then no memory is freed after pcap_compile.
2021-06-23 12:04:03 +02:00
Vitaly Lavrov 35fc6a6de5 Speed and memory size optimisation (#1214)
Removed bigram_automata, impossible_bigram_automata, trigram_automata.
The ahocorasick structure is replaced with a bitmap.
The bitmap size for ndpi_en_bigram is 176 bytes.
The bitmap size for ndpi_en_trigram is 2201 bytes.
On the test machine, the test execution time was reduced from 27.3 seconds to 24.7 (9%).
2021-06-23 12:03:52 +02:00
Luca Deri f19937c8c9 Addesses partial application matching (#1207) 2021-06-21 21:59:15 +02:00
Luca Deri 0cbb4b2ffa Restored 32 bit value in ndpi_match_string_value() 2021-06-21 17:06:22 +02:00
Luca Deri 613e21002a Upgraded exponential smoothing to 64 bit values 2021-06-18 22:56:56 +02:00
Luca Deri 0afc8ace3d Added nDPI Score report 2021-06-17 23:52:01 +02:00
martinscheuandtinu 0b9b6a683d fixed lua errors in non-iec104 packets (#1209)
* Update iec.lua

fixed lua errors in non iec 104 packets

* Update iec.lua

Co-authored-by: tinu <martin.scheu@switch.ch>
2021-06-17 15:35:31 +02:00
Luca Deri b0b3e1bc6c Added bitmap boundary checks 2021-06-17 10:49:11 +02:00
Luca Deri e89c06256e Extende nDPI API with ndpi_netbios_name_interpret() 2021-06-17 09:35:19 +02:00
Luca Deri 2eba77e3b0 NetBIOS decoding changes 2021-06-16 19:57:20 +02:00
Vitaly Lavrov be808c30f3 Code review. (#1205)
The common actions required to call the ac_automata_search() function
have been moved to the ndpi_match_string_common function. This made it
possible to simplify the ndpi_match_string, ndpi_match_string_protocol_id,
ndpi_match_string_value, ndpi_match_custom_category, ndpi_match_string_subprotocol,
ndpi_match_bigram, ndpi_match_trigram functions.

Using u_int16_t type for protocol identifiers when working with the
ahocorasick library (changes src/include/ndpi_api.h.in and src/include/ndpi_typedefs.h).

Reworked "finalization" of all AC_AUTOMATA_t structures.

Changing the order of fields in the ndpi_call_function_struct structure
reduces the size of the ndpi_detection_module_struct structure by 10 kB (for x86_64).
2021-06-15 11:28:09 +02:00
Luca Deri 09293fabd9 Win fix 2021-06-15 11:26:36 +02:00
Luca Deri 43ccaef2f5 IEC analysis wireshark plugin 2021-06-14 21:18:37 +02:00
Alfredo Cardigliano 4aefbe0c7a Call ac_automata_release with free_pattern = 1 (malloc'ed patterns expected in ndpi_add_string_to_automa) 2021-06-14 14:41:14 +02:00
Luca Deri 380286c069 Fixes https://github.com/ntop/ntopng/issues/5482 2021-06-11 22:21:03 +02:00
Luca Deri a79b8ee834 Reimplememnted SNMP dissector 2021-06-11 21:44:51 +02:00
Luca Deri dee6f90d6c Syntax error fixes 2021-06-11 21:01:27 +02:00
Luca Deri c9f582805b Fixed flow score label 2021-06-11 19:53:07 +02:00
Matteo Biscosi a01a2b7443 Changed output for the sharkfest lua scripts 2021-06-11 11:00:06 +02:00
Vitaly Lavrov 6704649a75 Bug in ndpi_lru_find_cache (#1204)
Key value check is missing.
2021-06-10 11:06:19 +02:00
Luca 1a8ae710cf Minor code cleanup 2021-06-09 08:58:18 +02:00
Toni 65e67a2f88 autoconf: Do not check for json-c if --with-only-libndpi is set. (#1202)
* json-c is used by a unit test
 * required to fix some libnDPI cross compilation issues

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-06-09 08:01:10 +02:00
Luca Deri 77710ed8b4 OpenWRT document fix 2021-06-08 21:39:28 +00:00
Luca Deri 177f9456bd Win changes 2021-06-08 18:15:46 +02:00
Luca Deri d3aa3df975 Updated scripts 2021-06-08 17:52:03 +02:00
Luca 41ec807d79 New testing pcap with syn scan attack 2021-06-08 10:39:41 +02:00
Luca 2f8d3ac0b0 Companion scripts written for the Sharkfest conference 2021-06-08 10:39:19 +02:00
Luca Deri 2af7b33de0 Added report on score cli/srv breakdown 2021-06-07 14:59:18 +02:00
Vitaly Lavrov 65678dbeea New version of the ahocorasick library (#1200)
The new version is about 25% faster with -O2 and 45% faster with -O3.
No recursion is used (smaller stack size required).
Uses less memory (by valgrind info)
 bigram:
  - original 1796 allocs,   247864 bytes allocated
  - new      1232 allocs,   158880 bytes allocated
 host_match:
  - original 18038 allocs, 3004576 bytes allocated
  - new       6861 allocs,  396624 bytes allocated

The function ac_automata_search() is thread safe.
Optional case-insensitive comparison.
Matching at the beginning and at the end of the string is supported.
One code file and one header file.
2021-06-07 14:19:40 +02:00
Vitaly Lavrov 2fcf641e87 Fixed uninitialized memory use. (#1198)
The pointer "header" must be initialized before first call pcap_next_ex().
2021-06-07 10:08:39 +02:00
Luca Deri b241dd5e6b Fixed expected output 2021-06-03 09:06:01 +02:00
Luca Deri 03ae090d39 Warnign fix 2021-06-03 09:05:53 +02:00
Luca Deri 8761ac1887 Renamed Skyp in Skype_Teams as the protocol is now shared across these apps 2021-06-02 21:47:01 +02:00
Ivan Nardi 654e84ff02 wireshark/lua: improve flow risk visualization (#1194)
Create a separated proto field entry for each possible flow risk.
This way, filtering will be more natural: you can use something like
"ndpi.flow_risk.desktop_file_sharing_session"
2021-06-02 21:28:50 +02:00
Luca Deri 06c6941f03 Removed xbox and playstation invalid port guesses 2021-06-02 00:24:41 +02:00
Ivan Nardi 9d427faafe ndpiReader: fix collecting of risks statistics (#1192) 2021-06-01 16:50:46 +02:00
Ivan Nardi f4dbeecba9 TLS: improve check for common ALPNs (#1191)
Facebook is still using its own ALPN for HTTP2 as well
2021-06-01 16:50:28 +02:00
Luca c620858671 Reworked ndpi flow risk score adding client and server score 2021-06-01 09:17:26 +02:00
Alfredo Cardigliano c4084ca3c7 Fix warnings 2021-05-25 15:17:48 +02:00
Luca eb0115d4e0 Added TLS check to avoid crash with UDP-based traffic 2021-05-25 09:03:24 +02:00
Luca Deri d485c1cecb Improved TLS browser detection heuristics 2021-05-19 21:28:33 +02:00
Ivan Nardi 71b81d3799 wireshark/lua: fix offsets (#1187) 2021-05-18 21:36:46 +02:00
Luca Deri 732bcecd17 Added flow risk score 2021-05-18 21:05:47 +02:00
Luca Deri 86f3c29d03 Typo 2021-05-18 19:52:33 +02:00
Luca Deri ca15e3295e Added risk/score dump (ndpiReader -h)
Added ndpi_dump_risks_score() API score
2021-05-18 19:34:17 +02:00
Luca Deri 0b3d627185 Updated API for ndpi_risk2score()
Added ndpi_get_upper_proto() API call
2021-05-17 22:39:07 +02:00
Luca Deri 43a8576efb Reworked human readeable string search in flows
Removed fragment manager code
2021-05-17 20:55:06 +02:00
Luca Deri 1ec621c85b Added further checks 2021-05-15 19:53:46 +02:00
Luca Deri 9c5ba9c060 Updated protocol category 2021-05-15 11:20:15 +02:00
Luca Deri 5646ffa725 Added ndpi_check_subprotocol_risk() API call definition 2021-05-15 11:16:27 +02:00
Luca Deri abd6bce6f9 Added TLS certifiacate caching
Added Fortigate protocol
2021-05-15 10:52:16 +02:00
Luca Deri ca92660d98 Converted some test .pcapng files to pcap format 2021-05-13 20:51:11 +02:00
Luca Deri ac1eaca8a6 Added browser TLS heuristic 2021-05-13 20:00:27 +02:00
Luca Deri a62be9b8ec Implemented heuristic to detect Safari and Firefox TLS browsing 2021-05-13 12:37:07 +02:00
Toni 87076dcd5b Fixed obsolete error printing if CTRL-C is pressed. #1165 (#1184)
* This fix was proposed by @robertsong2019

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-11 21:38:56 +02:00
Toni 5918a6542d Improved SSL certificate name wildcard handling and risk. #1182 (#1183)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-11 21:38:26 +02:00
Luca Deri a5ecdf9df8 Changes for sonarcloud integration 2021-05-11 21:36:25 +02:00
Luca Deri 5079bb1106 Fixed typo 2021-05-11 21:33:48 +02:00
Luca Deri 4297a65ce8 Implemented flow score in Wireshark integration 2021-05-10 22:43:05 +02:00
Luca Deri fff60ec317 Added check to reduce MongoDB false positive detection 2021-05-10 21:14:55 +02:00
Luca Deri 66ebe444ae Added (partial) Activision protocol support (based on tencent cloud) 2021-05-10 15:41:52 +02:00
Ivan Nardi 1deae1132b Fix some warnings (#1181)
```
In file included from protocols/fasttrack.c:29:
../include/ndpi_api.h:1504:3: warning: type qualifiers ignored on function return type [-Wignored-qualifiers]
 1504 |   const ndpi_risk_severity ndpi_risk2severity(ndpi_risk_enum risk);
      |   ^~~~~
In file included from protocols/amazon_video.c:28:
../include/ndpi_api.h:1504:3: warning: type qualifiers ignored on function return type [-Wignored-qualifiers]
 1504 |   const ndpi_risk_severity ndpi_risk2severity(ndpi_risk_enum risk);
      |   ^~~~~

...

ndpi_utils.c: In function ‘ndpi_risk2severity’:
ndpi_utils.c:1834:1: warning: control reaches end of non-void function [-Wreturn-type]
 1834 | }
      | ^
```
2021-05-09 15:11:05 +02:00
Ivan Nardi 2ca37ff8be TLS: fix extraction for TLS signature algorithms (#1180)
```
==69562==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6100009000fb at pc 0x7f41882003a7 bp 0x7f4183cfbfc0 sp 0x7f4183cfb768
READ of size 32 at 0x6100009000fb thread T1
    #0 0x7f41882003a6 in __interceptor_memcpy ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827
    #1 0x560b2d7462a1 in processClientServerHello protocols/tls.c:1647
    #2 0x560b2d73be6a in processTLSBlock protocols/tls.c:712
    #3 0x560b2d73e61f in ndpi_search_tls_udp protocols/tls.c:968
```
2021-05-09 15:10:41 +02:00
Ivan Nardi 12c6ab768d TLS: fix another use-of-uninitialized-value error in ClientHello parsing (#1179)
Error detected with valgrind.

==13127== Conditional jump or move depends on uninitialised value(s)
==13127==    at 0x483EF58: strlen (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==13127==    by 0x1A93B6: ndpi_strdup (ndpi_main.c:159)
==13127==    by 0x1C07CC: processClientServerHello (tls.c:1678)
==13127==    by 0x1C0C4C: processTLSBlock (tls.c:712)
==13127==    by 0x1C0C4C: ndpi_search_tls_tcp.part.0 (tls.c:849)

See also 8c3674e9
2021-05-09 15:10:14 +02:00
Toni 8c28613eb2 Check datalink during fuzzing to prevent console / logfile spam. See #1175 for more information. (#1177)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-09 15:09:43 +02:00
Bernhard M. Wiedemann 0663544be8 Allow to override build date with SOURCE_DATE_EPOCH (#1176)
in order to make builds reproducible.
See https://reproducible-builds.org/ for why this is good
and https://reproducible-builds.org/specs/source-date-epoch/
for the definition of this variable.
2021-05-09 15:09:15 +02:00
Luca ae2470fad4 Initial work towards detection via TLS of browser types 2021-05-06 21:42:06 +02:00
Luca 808050e635 Add extraction for TLS signature algorithms 2021-05-06 08:56:01 +02:00
Luca ce54a6ab57 Added ndpi_risk2severity() API call 2021-05-02 19:35:01 +02:00
Luca Deri 153d11c920 Enhanced netbios decoding check 2021-04-28 22:58:22 +02:00
Luca Deri dd65142020 Compilation fix 2021-04-27 08:26:08 +02:00
Luca Deri 70686249c9 Updated code due to https://github.com/ntop/nDPI/pull/1175 2021-04-27 08:12:14 +02:00
Luca Deri 7b62db81c3 Removed DGA check for ipv6 reverse IPs 2021-04-27 07:23:11 +02:00
Toni da3e6bd61b Check for common ALPNs and set a flow risk if not known. (#1175)
* Increased risk bitmask to 64bit (instead of 32bit).
 * Removed annoying "Unknown datalink" error message for fuzzers.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-27 07:22:04 +02:00
Luca Deri d5b395f35a Fixed invalid DNS dissection 2021-04-26 23:08:00 +02:00
Luca Deri 87ad2b58dc Compilation fix 2021-04-26 10:50:19 +02:00
Luca Deri 4a09707e48 Added flow risk to wireshark dissection 2021-04-26 10:17:29 +02:00
Luca Deri a04efb7ce7 Added tshark description 2021-04-25 11:47:19 +02:00
Luca Deri fad246289f README for the tshark class 2021-04-25 11:45:35 +02:00
Luca Deri b586c223dd Lua tshark class and examples 2021-04-25 11:40:53 +02:00
Luca Deri 0ef7998348 Removed protocol space in Genshin Impact 2021-04-25 10:06:22 +02:00
Toni b4a8c8c63f Add Genshin Impact protocol. (#1173)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-25 10:02:07 +02:00
Alfredo Cardigliano d95cd9912c Add vxlan enum to ndpi_packet_tunnel 2021-04-21 12:43:12 +02:00
Toni 9377991263 Add HP Virtual Machine Group Management (hpvirtgrp) protocol. (#1170)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-20 14:12:16 +02:00
Luca Deri c6a835746e Reworked GeoIP API
int ndpi_get_geoip_asn(struct ndpi_detection_module_struct *ndpi_str,
                        char *ip, u_int32_t *asn);
int ndpi_get_geoip_country_continent(struct ndpi_detection_module_struct *ndpi_str, char *ip,
                        char *country_code, u_int8_t country_code_len,
                        char *continent, u_int8_t continent_len);
2021-04-19 21:39:55 +02:00
Ivan Nardi 8c3674e9a3 TLS: fix some use-of-uninitialized-value errors in ClientHello parsing (#1169)
Error detected with valgrind.

==125883== Conditional jump or move depends on uninitialised value(s)
==125883==    at 0x438F57: processClientServerHello (tls.c:1421)
==125883==    by 0x43B35A: processTLSBlock (tls.c:712)
==125883==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125883==    by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)
==125883==    by 0x42E920: ndpi_detection_process_packet (ndpi_main.c:5301)

==125916== Conditional jump or move depends on uninitialised value(s)
==125916==    at 0x438D7D: processClientServerHello (tls.c:1379)
==125916==    by 0x43B35A: processTLSBlock (tls.c:712)
==125916==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125916==    by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)

==125932== Conditional jump or move depends on uninitialised value(s)
==125932==    at 0x438C1D: processClientServerHello (tls.c:1298)
==125932==    by 0x43B35A: processTLSBlock (tls.c:712)
==125932==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125932==    by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)

==125950== Conditional jump or move depends on uninitialised value(s)
==125950==    at 0x438D4F: processClientServerHello (tls.c:1371)
==125950==    by 0x43B35A: processTLSBlock (tls.c:712)
==125950==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125950==    by 0x42C079: check_ndpi_detection_func (ndpi_main.c:4443)
2021-04-18 21:38:01 +02:00
Ivan Nardi 9ca62ed7ac Fix detunneling of GTP-U traffic (#1168)
Fuzzing #1161 exposed some (completely unrelated) issues on GTP-U
detunneling code.
(see https://github.com/ntop/nDPI/actions/runs/719882047)
2021-04-18 21:37:51 +02:00
Ivan Nardi d9278d55c3 Improve classification of Outlook/MicrosoftMail traffic (#1167)
See #1148
2021-04-18 21:37:26 +02:00
Luca Deri 9fcf98c067 GeoIP handlign fixes 2021-04-18 21:36:03 +02:00
Luca Deri 18c6c1c2d6 Added NDPI_DESKTOP_OR_FILE_SHARING_SESSION risk to remote protocols for remote assistance sessions 2021-04-12 18:11:14 +02:00
Luca Deri bf318e0b86 Added NDPI_DESKTOP_OR_FILE_SHARING_SESSION flow risk 2021-04-11 14:42:27 +02:00
Ivan Nardi ceb98a3f9c wireshark/lua: fix handling of VLAN traffic (#1162) 2021-04-05 19:23:56 +02:00
Ivan Nardi 0bfe444362 GTP: fix parsing of GTP headers (#1161)
Message length checks and basic headers are not uniform across GTP-U,
GTP-C and GTP-PRIME.

Note that, even if the length checks were wrong, the GTP sessions were almost
always correctly classified because of the "guessing" algorithm.

This patch has been tested with GTP-U, GTP-C-V1, GTP-C-V2 and GPT-PRIME-V2
traffic using ndpiReader with "-d" flag (to avoid "guessing" algorithm) and
without "-t" flag (to avoid GTP-U de-tunneling).

See #1148
2021-04-05 19:23:03 +02:00
Ivan Nardi fb74785282 Fix some warnings about unused variables/functions (#1160) 2021-04-05 19:21:30 +02:00
Luca Deri a1dba74346 Trace fix 2021-04-02 12:55:15 +02:00
Luca Deri 4f8ca9485a Fixed incapoatibilities with the latest extcap/wireshark 2021-04-01 23:53:53 +02:00
Luca Deri db5aeb0cb1 Readme update 2021-04-01 10:24:44 +02:00
Luca Deri 28879c570e Reworked ndpi patricia includes to avoid compilation issues on some platforms 2021-03-31 19:43:55 +02:00
Ivan Nardi 1290c40968 Mining: lru cache is ipv4 only (for the time being) (#1159)
Fix memory error with ipv6 traffic
2021-03-31 14:13:07 +02:00
Luca Deri 48726301e1 Fixed mispelled word 2021-03-31 12:52:54 +02:00
Luca Deri 2b84f28df5 Improved mining results 2021-03-30 17:54:44 +02:00
Luca Deri c1d6e3f145 Improved mining detection support 2021-03-30 17:50:19 +02:00
Luca Deri 637b2063ed Added missing tracker/Ads breed 2021-03-30 17:50:03 +02:00
Luca Deri fcbc16da00 Fixed invalid guess stats 2021-03-30 17:49:48 +02:00
Luca Deri ffbce931b9 Ignore TLD .local .lan and .home in DGA domain check 2021-03-26 15:53:04 +01:00
Ivan Nardi a6029d250d ndpiReader: print an error msg if we found an unsupported datalink type (#1157) 2021-03-23 11:47:29 +01:00
Toni b040407683 Refactored nDPI subprotocol handling and aimini protocol detection. (#1156)
* Refactored and merged callback buffer routines for non-udp-tcp / udp / tcp / tcp-wo-payload.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Try to detect one subprotocol if a detected protocol can have one.

 * This adds a performance overhead due to much more protocol detection routine calls.
   See #1148 for more information.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactor subprotocol handling (1/2).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactor subprotocol handling (2/2).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Prevent some code duplication by using macros for ndpi_int_one_line_struct string comparision.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactored aimini HTTP detection parts (somehow related to #1148).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Added aimini client/server test pcap.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Removed master protocol as it was only used for STUN and via also removed API function ndpi_get_protocol_id_master_proto

 * Adjusted Python code to conform to the changes made during the refactoring process.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-03-23 11:46:12 +01:00
Luca Deri 0cae9bf4a4 Win compilation fix 2021-03-22 21:41:28 +01:00
Luca Deri 627299e4dd Better DGA detection (slightly decreased accuracy) 2021-03-20 17:56:24 +01:00
Luca Deri 6333bb1702 Removed duplicate extesions len 2021-03-19 22:35:08 +01:00
Luca Deri d96f4ca736 Added ALPN and elliptic curve in JA3S+ 2021-03-19 22:33:04 +01:00
Luca Deri fa238bbe42 Implemented JA3+ also for JA3S 2021-03-19 22:27:36 +01:00
Luca Deri e2c3445a20 Reworked JA3 2021-03-19 22:20:52 +01:00
Luca Deri 61f7ec1696 JA3 debug improvements 2021-03-19 19:10:31 +01:00
Luca Deri 9eed0a60d3 Fixed support for custom DGA detection libraryr 2021-03-18 11:32:56 +01:00
Luca Deri b22cb70a68 Added % of flows with risks 2021-03-14 21:59:33 +01:00
Luca Deri 1b6275005e Added in stats the number of flows with risks 2021-03-14 21:54:35 +01:00
Luca Deri 3b02cfab69 Added flows risks report 2021-03-14 21:46:23 +01:00
Luca Deri 565a7bfce3 Reworked extendal dependency across testing tools 2021-03-14 20:48:21 +01:00
Luca Deri e58527536d Help crash fix 2021-03-14 19:55:21 +01:00
Luca Deri 9419015711 Implemented square erro rollup to avoid overflow 2021-03-14 11:01:51 +01:00
Zied Aouini 7f9a18fd05 Make libmaxminddb build configurable. (#1154) 2021-03-14 10:47:28 +01:00
Alfredo Cardigliano efc5630378 Fix compilation warning 2021-03-12 09:25:00 +01:00
Luca 192fad4402 Added double exponential smoothing implementation 2021-03-11 09:39:52 +01:00
Luca Deri 6833ee2bbe Added single exponential smoothing API
int ndpi_ses_init(struct ndpi_ses_struct *ses, double alpha, float significance);
int ndpi_ses_add_value(struct ndpi_ses_struct *ses, const u_int32_t _value, double *forecast, double *confidence_band);
2021-03-11 00:04:33 +01:00
Luca Deri 5b7fe1360a Fixed JA3+ computation 2021-03-11 00:04:12 +01:00
Luca Deri 5c6f070097 Fixed nDPI prefs definition 2021-03-09 23:50:31 +01:00
Luca Deri f6ad16d8f8 Added experiemntal JA3+ implementation that can be used with -z i ndpiReader 2021-03-09 23:38:29 +01:00
Ivan Nardi 8074bc8201 HTTP: fix memory access in ndpi_http_parse_subprotocol() (#1151) 2021-03-09 19:46:32 +01:00
Luca Deri 574fc4f09e Ookla detection improvement 2021-03-09 17:43:12 +01:00
Luca Deri 3032864ec9 Added Ookla detection over IPv6 2021-03-09 12:55:14 +01:00
Luca Deri db716d0ab0 Ookla fixes 2021-03-09 11:42:23 +01:00
Luca Deri 477e4db650 Improved detection of Ookla speedtest and openspeedtest.com 2021-03-09 11:38:31 +01:00
Luca Deri bb6423a79f Added the ability to define a custom DGA detection function by overwriting
the value of the function pointer ndpi_dga_function curently set to NULL
(that means the nDPI internal DGA function will be used)
2021-03-08 22:57:30 +01:00
Ivan Nardi c3490e80a7 Fix some stack-use-after-return errors in automa code (#1150) 2021-03-08 21:14:01 +01:00
Ivan Nardi c50a8d4808 Add support for Snapchat voip calls (#1147)
* Add support for Snapchat voip calls

Snapchat multiplexes some of its audio/video real time traffic with QUIC
sessions. The peculiarity of these sessions is that they are Q046 and
don't have any SNI.

* Fix tests with libgcrypt disabled
2021-03-06 05:48:36 +01:00
Alfredo Cardigliano 73e4348570 Add ndpi_serialize_binary_boolean for consistency. Fix comments. 2021-03-04 10:31:03 +01:00
Luca Deri 0f8a994841 Improved DGA detection
Before
Accuracy 66%, Precision 86%, Recall 38%

After
Accuracy 71%, Precision 89%, Recall 49%
2021-03-03 19:30:01 +01:00
Luca 49843509e5 Added verbose option 2021-03-03 08:12:16 +01:00
Luca Deri 1a37595de0 Removed check for knowns protocols (major and app protocols) 2021-03-03 00:57:56 +01:00
Luca Deri 56bfb439f8 Improved DGA detection with trigrams. Disadvantage: slower startup time
Reworked Tor dissector embedded in TLS (fixes #1141)
Removed false positive on HTTP User-Agent
2021-03-03 00:41:07 +01:00
Ivan Nardi 4c00ff89df DTLS: improve support (#1146)
* DTLS: add some pcap tests

* DTLS: fix parsing of Client/Server Helllo message

* DTLS: add parsing of server certificates
2021-03-02 21:15:40 +01:00
Ivan Nardi 2b71a329c1 QUIC: fix mvfst-27 test (#1145)
Regardless of its name, quic-mvfst-27 trace doesn't contain mvfst-27
traffic
2021-03-02 21:15:02 +01:00
Ivan Nardi c12a697f2d TOR: update node list (#1144)
https://panwdbl.appspot.com/lists/ettor.txt list is no more available

Close #1141
2021-03-02 21:14:36 +01:00
Luca Deri 2d558afb54 Added HW checks 2021-03-01 19:59:14 +01:00
Luca Deri 2724a14bf7 Added further HW checks 2021-03-01 19:34:20 +01:00
Luca Deri 4bff595733 Holt-Winters calculation improvement 2021-02-27 11:32:51 +01:00
Toni 16890a6632 Added NDPI_MALICIOUS_SHA1 flow risk. (#1142)
* An external file which contains known malicious SSL certificate SHA-1 hashes
   can be loaded via ndpi_load_malicious_sha1_file(...)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-26 17:00:05 +01:00
Luca Deri fba61adf5e Improved DNS dissector 2021-02-26 00:42:15 +01:00
pengtian 28330edb7a [Fix] replace free to ndpi_free (#1140)
same as https://github.com/ntop/nDPI/issues/1096
2021-02-25 08:46:55 +01:00
Toni 70c35addcb Added protocol breed to JSON serializer. (#1137)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-25 08:46:38 +01:00
Alfredo Cardigliano d898fddea0 Fix ndpi_fill_prefix_v6 2021-02-24 14:43:12 +00:00
Luca Deri 545f270132 Windows code cleanup 2021-02-24 11:04:04 +01:00
Luca Deri 4bd175b07e Modified JA3 fingerprint message 2021-02-24 10:42:26 +01:00
Alfredo Cardigliano 566d5db402 Add more utility functions to work with patricia trees 2021-02-23 11:53:15 +01:00
Alfredo Cardigliano 888ddfcd17 Update ndpi_patricia_walk_inorder API 2021-02-23 10:23:52 +01:00
Alfredo Cardigliano f8e83f7e35 Add support for MAC to Patricia tree. Expose full API to applications. Add utility functions. 2021-02-23 10:01:56 +01:00
Luca Deri b757cf606b Fixed mispelled API call 2021-02-22 23:59:08 +01:00
Luca Deri f1b22b199f Added NDPI_MALICIOUS_JA3 flow risk
Added ndpi_load_malicious_ja3_file() API call
2021-02-22 23:19:23 +01:00
Luca Deri fc3db8f169 Implemented TLS Certificate Sibject matching
Improved AnyDesk detection
2021-02-22 22:37:33 +01:00
Luca Deri a21ad2fbe4 Added new data for risky domains test 2021-02-21 22:08:01 +01:00
Luca Deri 5ad056e99b Removed unused NDPI_RISKY_COUNTRY 2021-02-21 21:45:46 +01:00
Luca Deri fc16c9368e Added risky domain flow-risk support 2021-02-21 21:45:46 +01:00
Luca Deri 6db20b3ba9 Updated dependency 2021-02-19 17:59:40 +01:00
Luca Deri 09559611cc Fixes #1136 2021-02-19 15:15:02 +01:00
Luca Deri dee24f7acf Removed old unused code 2021-02-19 15:12:03 +01:00
Alfredo Cardigliano e9350d8fc7 Fix leak 2021-02-19 09:27:24 +01:00
Luca Deri 92e8d95f38 Added ndpi_get_geoip() APi call 2021-02-18 23:15:27 +01:00
Luca Deri bdf95a5c22 Package fix 2021-02-18 22:08:15 +00:00
Luca Deri 393a844c9a GeoIP support 2021-02-18 22:40:57 +01:00
Luca Deri bce54079d8 Initial geoip support 2021-02-18 22:35:25 +01:00
Luca Deri a2c5adc374 Improved nDPI string matching algorithm 2021-02-18 21:47:09 +01:00
Luca Deri fb4cdecb68 Fixes due to the fragment mananegr code 2021-02-18 13:31:02 +01:00
Luca Deri 413dc287b4 Added paper citation 2021-02-17 08:22:38 +01:00
Luca Deri 4609e2084b Added new risks (future use)
-  NDPI_RISKY_ASN
-  NDPI_RISKY_DOMAIN
-  NDPI_RISKY_COUNTRY
2021-02-16 18:23:19 +01:00
Alfredo Cardigliano 177f1785da Fix warning 2021-02-12 09:11:14 +01:00
Luca Deri 813df77d07 Performance optimization to avoid un-necessary calls and thus increase the overall performance 2021-02-11 17:54:55 +01:00
Luca Deri d99d636696 Some optimizations during flow guess 2021-02-10 19:51:26 +01:00
Luca Deri fffff3760b Code refactory 2021-02-10 19:28:35 +01:00
Luca Deri ea0309d21b Removed now obsolete NDPI_DETECTION_SUPPORT_IPV6: code is more readeable now 2021-02-10 19:09:11 +01:00
Luca Deri 76ac551ed3 Improved FTP_CONTROL detection 2021-02-10 17:55:07 +01:00
Luca Deri 79102ef4f2 Added check for avoiding long dissections 2021-02-10 17:36:09 +01:00
Ivan Nardi 421609475e Fix small memory leak (#1133)
Now function definition matches the prototype in ndpi_api.h.in
2021-02-10 15:24:34 +01:00
Toni 1e12c90c66 Fixed memory leaks caused by conditional free'ing for some TLS connec… (#1132)
* Fixed memory leaks caused by conditional free'ing for some TLS connections.

 * Members of tls_quic struct should also free'd if the detected master protocol is IMAPS / POPS / SMTPS / etc.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Prevent reader_util.c from exit()'ing if maximum flow count reached.
This confuses the fuzzer.

 * Improved fuzz/Makefile.am to use LDADD for ../example/libndpiReader.a instead of LDFLAGS.
   That way, fuzz_ndpi_reader re-links to ../example/libndpiReader.a if something changed there.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-10 15:24:11 +01:00
Luca Deri c408df1b0e STUN improvements 2021-02-10 15:22:20 +01:00
Luca Deri e2f6569adb Fixed CPHA missing protocol initialization
Improved IEC104 and IRC detection
2021-02-10 15:22:20 +01:00
Luca Deri 0de3d4c37a Dissection inprovements 2021-02-09 21:48:46 +01:00
Luca Deri 512f73f381 Added checks for giving up faster on IRC and SMTP 2021-02-09 21:27:09 +01:00
Luca Deri 1a557d0918 IRC test files 2021-02-09 21:25:48 +01:00
Luca Deri 1331e0aec9 Extended the API to calculate jitter
- ndpi_jitter_init()
- ndpi_jitter_free()
- ndpi_jitter_add_value()
2021-02-09 15:56:03 +01:00
Luca Deri b960809d3e Minor code improvements 2021-02-09 12:55:11 +01:00
Luca Deri 4abaf3e279 Removed debug statement 2021-02-09 10:05:29 +01:00
Luca Deri 732579b72b Added timeseries forecasting support implementing Holt-Winters with confidence interval
New API calls added
- ndpi_hw_init()
- ndpi_hw_add_value()
- ndpi_hw_free()
2021-02-08 19:10:25 +01:00
Luca Deri d9da752aa8 Updated skype addresses 2021-02-07 19:05:56 +01:00
Luca Deri 3ecda45bc4 IP address matching update 2021-02-07 17:42:31 +01:00
Luca Deri 8297dacfe5 Added Makefile for building the development version of libndpi for OpenWRT 2021-02-07 09:29:01 +00:00
Luca Deri be58d78417 (Temporarely) Disabled fragment manager (#1129) 2021-02-05 23:03:08 +01:00
Luca Deri 45e9c3c438 Partial fix for #1129 2021-02-05 22:22:33 +01:00
Alfredo Cardigliano 95a6a2072b Code cleanup and safety checks in the fragment manager (#1129) 2021-02-05 17:16:11 +01:00
Luca Deri 8dd7716ae5 Implemented more efficient and memory savvy RSI 2021-02-05 12:38:41 +01:00
Luca Deri 60b58dbd67 RSI enhancements 2021-02-05 10:59:09 +01:00
Luca Deri 1eedf734be Implemented API for computing RSI (Relative Strenght Index)
void  ndpi_init_rsi(struct ndpi_rsi_struct *s, u_int16_t num_learning_values);
void  ndpi_free_rsi(struct ndpi_rsi_struct *s);
float ndpi_rsi_add_value(struct ndpi_rsi_struct *s, const u_int32_t value);
2021-02-04 23:52:33 +01:00
Luca Deri 54636a3213 Improved (partial) TLS dissection 2021-02-04 22:06:18 +01:00
Ivan Nardi 679c317c57 Fix utests when "--disable-gcrypt" flag is used (#1128)
Fix: d6684f4b
2021-02-04 19:29:48 +01:00
Ivan Nardi 2a65321884 Fix some memory leakes in reassembler code (#1127) 2021-02-04 19:29:27 +01:00
ToniandLuca Deri 7255dbe4b7 Added fuzzy targets conditional in tests/do.sh.in which prevents the fuzzer from running if nDPI was configured previously --enable-fuzztargets but not for the current config (may produce invalid results). (#1126)
* fixed possible NULL pointer dereference for memcpy(), src pointer should never be NULL

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2021-02-04 19:28:52 +01:00
Luca Deri c68c8c79c9 Added missing check 2021-02-04 18:17:45 +01:00
Luca Deri 6c366d73e2 Fixed leak with DTLS 2021-02-03 23:36:11 +01:00
Luca Deri 6289413fea Updated results 2021-02-03 11:58:17 +01:00
Ivan Nardi a772e18977 Fix a warning (#1125)
Introduced in 5f7b9d802

reader_util.c: In function ‘process_ndpi_collected_info’:
reader_util.c:1148:60: warning: ‘%s’ directive output may be truncated writing up to 255 bytes into a region of size 64 [-Wformat-truncation=]
 1148 |       sizeof(flow->ssh_tls.client_requested_server_name), "%s",
      |                                                            ^~
reader_util.c:1147:5: note: ‘snprintf’ output between 1 and 256 bytes into a destination of size 64
 1147 |     snprintf(flow->ssh_tls.client_requested_server_name,
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 1148 |       sizeof(flow->ssh_tls.client_requested_server_name), "%s",
      |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 1149 |       flow->ndpi_flow->protos.tls_quic_stun.tls_quic.client_requested_server_name);
      |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-02-03 11:56:37 +01:00
Luca Deri d6684f4b4c Added test pcap 2021-02-03 11:56:14 +01:00
Ivan Nardi 8c0ea694f8 HTTP: fix user-agent parsing (#1124)
User-agent information is used to try to detect the user OS; since the
UA is extracted for QUIC traffic too, the "detected_os" field must be
generic and not associated to HTTP flows only.

Otherwise, you might overwrite some "tls_quic_stun" fields (SNI...) with
random data.

Strangely enough, the "detected_os" field is never used: it is never
logged, or printed, or exported...
2021-02-03 11:54:11 +01:00
Ivan Nardi 8cee718e8b HTTP: fix logs when NDPI_ENABLE_DEBUG_MESSAGES is defined (#1123) 2021-02-03 11:53:41 +01:00
Luca Deri 155b1aaa68 Updated results due to the new fragment manager handler 2021-02-03 11:52:46 +01:00
Luca Deri 6e87daab69 Cosmetic fixes 2021-02-03 11:52:32 +01:00
Luca Deri 1f3ade5923 Increased number of extra packets that is necessary since the frgament mananger introduction 2021-02-03 11:51:55 +01:00
Luca Deri 4b181be58e Improved debug message 2021-02-03 11:49:14 +01:00
Luca Deri d19bad1581 Added pcap for testing fragments reassembly 2021-02-03 11:48:53 +01:00
Luca Deri 01f4a57118 Fixes an issue with https://github.com/ntop/nDPI/pull/1122 that misprocsssed packets belonging to flows whose initial part (e.g. the 3WH) was not observed by nDPI (e.g. capture started in the middle of the flow) 2021-02-03 11:47:21 +01:00
b70ad0e2f1 fragments management added (#1122)
Management of tcp segments managements.

Co-authored-by: ragostino <ragostino73@gmail.com>
Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2021-02-03 10:28:51 +01:00
ragostino ee94534906 debug message bugfix (#1108)
you can not look for memory enlargement if you print debug message after updating the variables
2021-02-03 09:31:17 +01:00
Luca Deri 50f4049b58 Improved wireguard dissection 2021-01-29 16:38:43 +01:00
Luca Deri bb74b903d0 DCE/RPC improvement to avoid false positives 2021-01-29 16:23:18 +01:00
Luca Deri aeeccee106 DGA name improvement 2021-01-27 11:43:35 +01:00
Luca Deri aa5486f5f8 Added missing library 2021-01-22 09:31:37 +01:00
Luca Deri 288ccd6215 Fixes due to datatype rename 2021-01-22 09:17:34 +01:00
Luca Deri a31bd5ac3c Cleaned up tls/quic datatypes 2021-01-21 19:17:33 +01:00
Luca Deri 15295ef4c5 Reworked TLS fingerprint calcolation
Modified TLS memory free
2021-01-21 19:06:05 +01:00
Toni 399755607d Disable tests that require libgcrypt if --disable-gcrypt set. (#1121)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-01-21 08:59:06 +01:00
morefigs 53415c8855 Added missing comma (#1116)
I presume there is a comma missing in this comma separated list.
2021-01-21 08:58:42 +01:00
Luca Deri 3e5e9569ff Added simple hash implementation to the nDPI API 2021-01-20 21:30:19 +01:00
Luca Deri d964c3e081 Code cleanup: third party uthash is at the right place 2021-01-20 19:11:36 +01:00
Luca 0809956e5f Rewored UPnP protocol that in essence was WSD hence it has been renamed
Cleaned up TLS code for DTLS detection by defining a new DTLS protocol
2021-01-20 09:53:30 +01:00
Luca Deri e9f43516ec Improves STUN dissection removing an invalid termination condition that prevented Skype calls to be properly identified 2021-01-13 16:46:51 +01:00
Luca Deri 68b6ac7da8 (C) Update 2021-01-07 11:13:36 +01:00
Luca Deri 367184628e Warning fix 2021-01-07 11:07:17 +01:00
Darryl Sokoloski 5f7b9d8024 Increase SNI hostname buffer length to 256. (#1111)
According to RFC 4366, SNI host names can be up to 255 bytes.
Previous size of 64 resulted in failed application matches due to truncation.

For example:
 0976e041e65b1aece3e720df36ac6bd7.safeframe.googlesyndication.co|m

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2021-01-07 10:58:48 +01:00
Ivan Nardi ec2735694b STUN: avoid false positives (#1110)
STUN traffic doesn't use multicast addresses
2021-01-07 10:58:22 +01:00
Ivan Nardi 1e2b57bed4 HTTP: fix compilation and a memory error when NDPI_ENABLE_DEBUG_MESSAGES is defined (#1109) 2021-01-07 10:58:02 +01:00
Ivan Nardi 2080cc7365 QUIC: add suppport for DNS-over-QUIC (#1107)
Even if it is only an early internet draft, DoQ has already (at least)
one deployed implementation.
See: https://www.zdnet.com/article/ad-blocker-adguard-deploys-worlds-first-dns-over-quic-resolver/
Draft: https://tools.ietf.org/html/draft-huitema-dprive-dnsoquic-00

In the future, if this protocol will be really used, it might be worth to
rename NDPI_PROTOCOL_DOH_DOT in NDPI_PROTOCOL_DOH_DOT_DOQ
2021-01-07 10:56:39 +01:00
Ivan Nardi 00dabce65e Quic fixes (#1106)
* QUIC: fix heap-buffer-overflow

* TLS: fix parsing of QUIC Transport Parameters
2021-01-07 10:55:55 +01:00
Ivan Nardi b8a5358e80 QUIC: improve handling of SNI (#1105)
* QUIC: SNI should be always saved in flow->protos.stun_ssl.ssl.client_requested_server_name

Close #1077

* QUIC: fix matching of custom categories

* QUIC: add NDPI_TLS_MISSING_SNI support for older GQUIC versions

* QUIC: fix serialization

* QUIC: add DGA check for older GQUIC versions
2021-01-07 10:55:23 +01:00
Luca Deri eb37f8f1fb Split HTTP request from response Content-Type. Request Content-Type should be present with POSTs and not with other methods such as GET 2021-01-06 18:28:24 +01:00
Luca Deri cd21f0d316 Added check for invalid HTTP content 2021-01-06 12:25:32 +01:00
Ivan Nardi 1b524f5538 QUIC: update to draft-33 (#1104)
QUIC (final!?) constants for v1 are defined in draft-33
2021-01-04 15:50:14 +01:00
Ivan Nardi 3aa16b63e8 Fix some warnings when compiling with "-W -Wall" flags (#1103) 2021-01-04 15:49:39 +01:00
Ivan Nardi 23b84cd3ee Remove FB_ZERO protocol (#1102)
FB_ZERO was an experimental protocol run by Facebook.
They switched to QUIC/TLS1.3 more than 2 years ago; no one ever used it but
them so it is definitely dead.
See: https://engineering.fb.com/2018/08/06/security/fizz/
2021-01-04 15:49:19 +01:00
Toni 53ee6db795 Added a new API function `ndpi_free_flow_data' which free's all members of ndpi_flow_struct but not the struct itself. (#1101)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-01-04 12:47:05 +01:00
Ivan Nardi 3bb12fde63 Fix memory leak introduced in b7376cc6 (#1100) 2021-01-04 12:46:30 +01:00
Luca Deri 4ddb5f4245 Added TLS test with long certificate 2021-01-04 11:31:25 +01:00
Luca Deri a91e6179c5 Fixed missing symbol 2021-01-02 21:21:15 +01:00
Luca Deri 05d76525b0 Added HTTP suspicious content securirty risk (useful for tracking trickbot) 2021-01-02 21:11:42 +01:00
Luca Deri 32f0446c9c Updated ndpi_ptree_match_addr() prototype 2020-12-30 18:23:40 +01:00
Luca Deri b7376cc690 Restored QUIC stats 2020-12-30 12:12:33 +01:00
Luca Deri dc401f8a74 Split ptree user data in 32 and 64 bit entries 2020-12-30 12:12:33 +01:00
pengtian ab037e2c82 Bugfix for host check (#1097)
this bug is from commit `427002d14` `2020-05-06 00:31:40`
2020-12-29 14:41:41 +01:00
Luca Deri 84132c6735 Added known protocol on unknown port for ntop 2020-12-28 16:11:39 +01:00
Luca Deri 7f944cc43b Initialization fix 2020-12-28 11:57:20 +01:00
Luca Deri 0fceb6576a Free flow fix 2020-12-28 09:32:06 +01:00
Luca Deri 9c1827a77b Fixed output when tLS (nad not QUIC) is used 2020-12-28 09:19:39 +01:00
Luca Deri cf00ffed32 Removed test code 2020-12-26 18:13:10 +01:00
Luca Deri e1be363037 Removed space from protocol name 2020-12-23 20:38:02 +00:00
Luca Deri 34fc9d5d50 Introduced fix on TLS for discarding traffic out of sequence that might invalidate dissection 2020-12-22 09:47:39 +01:00
Luca Deri 6462f4671c Fixed invalid TLS check for extra processing detection 2020-12-21 22:42:05 +01:00
Luca Deri b231982286 Win fixes 2020-12-17 15:34:17 +01:00
Luca Deri df1b3367a9 Type change to avoid Windows compilation issues 2020-12-17 12:58:12 +01:00
Luca Deri 2768da0637 Improved HTTP dissection 2020-12-16 14:45:29 +01:00
Luca Deri a89642ad04 Fixes bug introduced by https://github.com/ntop/nDPI/pull/1085 2020-12-12 12:32:44 +01:00
Zied Aouini 5bd5461f96 Fix minimum packet length condition (#1087) 2020-12-12 11:12:59 +01:00
Toni 74a77e7b3d Added --ignore-vlanid / -I to exclude VLAN ids for flow hash calculation. #1073 (#1085)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-12-11 21:01:51 +01:00
Toni edf3a57a6a Added `fast.com' host pattern to Netflix detection. #1080 (#1084)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-12-11 21:01:28 +01:00
Ivan Nardi 5587010cfb soulseek: fix heap buffer overflow (#1083)
Close #1082
2020-12-11 21:01:07 +01:00
Luca Deri 5cb6ddfd22 Rule changes work in progress 2020-12-11 17:25:57 +01:00
Luca Deri 21ad3a1775 Added initialization 2020-12-11 17:25:57 +01:00
Alfredo Cardigliano 499897219e Add NDPI_PROTOCOL_ANY_CATEGORY to ndpi_protocol_category_t enum 2020-12-11 15:56:57 +01:00
Luca Deri 9c5fbb5208 Added FreeBSD ibuils instructions 2020-12-10 14:19:59 +00:00
Luca Deri 4cb8712a35 Added --with-mipsel for building nDPI on mipsel devices 2020-12-09 11:01:40 +00:00
Toni 62cd852c6f Rename Jabber detection name as we are not sure if it is unencrypted e.g. if START_TLS used. (#1079)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-12-08 15:49:17 +01:00
rafaliuszandrafal 1ecc6d323e Add a connectionless DCE/RPC detection (#1078)
* Add connectionless DCE/RPC detection

* Add DCE/RPC pcap file as well as its test result

Co-authored-by: rafal <rafal.burzynski@cryptomage.com>
2020-12-08 15:48:53 +01:00
Ivan Nardi 860ef58ace QUIC: sync with Wireshark latest changes (#1074)
Most of the QUIC crypto code has been "copied-and-pasted" from Wireshark;
try to stay in sync with the original sources to ease backporting of fixes.

Only cosmetic changes and code refactoring; no behaviour changes or bugfixes.
See:
https://gitlab.com/wireshark/wireshark/-/commit/5e45f770fd79ca979c41ed397fee72d2e8fb5f1e
https://gitlab.com/wireshark/wireshark/-/commit/5798b91c1526747bf688b6746b33562c1b24a9e0
2020-12-08 15:47:58 +01:00
Luca Deri 1fb1848586 Added chec for avoiding doung ntop_config.h include 2020-12-01 08:57:09 +00:00
Luca Deri eb689b2069 nDPI rules (work in progress) implementation 2020-11-30 22:01:49 +01:00
Luca Deri b77a4276f9 Fixed JSON-C check 2020-11-30 16:41:58 +01:00
Luca Deri d4a42ed733 Improved JSON-C detection 2020-11-30 16:30:38 +01:00
Luca Deri 6b9155589a Removed unnecessary field 2020-11-29 19:04:26 +01:00
Luca Deri 23a15bae5f Fixes #1029 2020-11-27 18:51:56 +01:00
Luca Deri 6c119c0b06 Fixes #1032 2020-11-27 18:48:25 +01:00
Toni daa1c8ef8c Fix/overflow and libgerror check (#1068)
* Fixed stack overflow caused by missing buffer space for the trailing \0 added by sprintf()

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Remove the autoconf cache value from the previous and failed check before checking again.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-26 19:46:31 +01:00
Alfredo Cardigliano 7437ecaab0 Warning fix 2020-11-23 10:19:35 +01:00
Luca Deri 38984af03e Updated disney definition 2020-11-22 18:49:28 +01:00
Luca Deri 78f762bd36 Typo 2020-11-22 16:51:13 +01:00
Luca Deri 21a6112b91 Added 129.205.94.0/23 to Facebook network list 2020-11-22 11:33:31 +01:00
Ivan Nardi 53a5c354d8 Quic fixes (#1067)
* QUIC: fix return value on error path on quic_cipher_init()

* QUIC: allow dissection of sessions forcing version negotiation

Enhance heuristic to avoid false positives.
2020-11-22 11:04:10 +01:00
Ivan Nardi fb2027cc8e iec60870-5-104: fix heap-buffer-overflow error (#1066) 2020-11-22 11:03:26 +01:00
Luca Deri fe3d926741 Added prerequisites for Ubuntu/Debian systems 2020-11-18 16:40:45 +01:00
Alfredo Cardigliano e45f0a4b64 Install ndpi under /usr 2020-11-17 09:19:07 +01:00
Luca Deri d670850f55 Updated results 2020-11-16 21:25:59 +01:00
Luca Deri acffd7bd1e Compilation fix 2020-11-16 21:22:54 +01:00
Zied AouiniandLuca Deri bfabb0ddf4 Add Virtual Asssitant (Alexa, Siri) support. (#1057)
* Add AmazonAlexa protocol.

* Add AmazonAlexa test file and result.

* Include pcapng as file format.

* Rename Category to VirtualAssistant.

* Add AppleSiri virtual assistant.

* Fix pcapng test files format support.

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2020-11-16 21:19:38 +01:00
Oleksandr Natalenko 5c8e125f06 lib/ndpi_content_match: update TOR IPs (#1063)
Signed-off-by: Oleksandr Natalenko <oleksandr@natalenko.name>
2020-11-16 21:18:17 +01:00
Zied Aouini 3d8fd42307 Implement DGA detection performances tracking workflow. (#1064)
* Implement dga evaluation helper.

* Add test set for DGA classification.

* Add DGA classification performances tracking as part of Travis.

* Add DGA evaluation doc.

* Fix CI on OSX.

* Add missing backquote.
2020-11-16 21:17:16 +01:00
Zied Aouini 76bb83085b Improve subprotocols detection. (#1062)
* Improve Spotify detection.

* Improve Skype detection.

* Improve Microsoft detection.

* Fix Microsoft detection categories.

* Improve Waze detection.

* Improve Apple detection.

* Improve WindowsUpdate detection.

* Improve TikTok detection.

* Improve Teams detection.

* Improve Youtube detection.

* Improve Messenger detection.

* Improve Twitch detection.

* Improve Hulu detection.

* Improve Facebook detection.

* Improve AmazonVideo detection.
2020-11-16 21:16:38 +01:00
Zied AouiniandLuca Deri 3529268df8 Add Tumblr support. (#1061)
* Add Tumblr protocol.

* Add Tumblr test file and result.

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2020-11-16 21:14:06 +01:00
Zied AouiniandLuca Deri 22780da8d5 Add Reddit support. (#1060)
* Add Reddit protocol.

* Add Reddit test file and result.

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2020-11-16 21:13:01 +01:00
Zied AouiniandLuca Deri 13dab51cc7 Add Pinterest support. (#1059)
* Add Pinterest protocol.

* Add Pinterest test file and result.

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2020-11-16 21:11:43 +01:00
Zied Aouini f35fad13c3 Add DisneyPlus protocol. (#1058) 2020-11-16 21:10:11 +01:00
Luca Deri 419483f128 File nor necessary and generated by configure 2020-11-12 23:22:15 +01:00
Luca Deri 328ff24657 Renumbered AmongUs protocol 2020-11-09 16:23:01 +01:00
Toni 6b5bdf773d Added support for AmongUs. (#1054)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-09 16:19:00 +01:00
Toni af02ffb60f Support raw IPv4 / IPv6 pcap packet processing. (#1053)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-09 16:18:05 +01:00
Toni ac18d5a084 Improved SSH protocol detection. (#1052)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-09 16:17:42 +01:00
Don J. Rude bbf42c3610 fixes issue #1050 Syntax error caused buffer pointer to equal 0x1 (#1051)
* Syntax error caused buffer pointer to equal 0x1

Possible copy-paste from lines 141-142?

* Another comma operator

* whitespace matching

* another comma operator

* another comma operator

* another comma operator

* Check for non-zero payload
2020-11-09 16:17:10 +01:00
Luca Deri 89a363aff6 Updated ESNI/SNI alarm generation prolicy 2020-11-08 10:07:35 +01:00
Luca Deri f6cb869c63 Reworked IEC60870 dissector 2020-11-04 14:29:11 +01:00
Luca Deri 044a11faca IEC60870 dissection improvements 2020-11-04 12:40:31 +01:00
Luca Deri 017e395ed1 Cosmetic changes 2020-11-03 16:46:30 +01:00
Leonn 0576dc2a49 💡 Add mongodb protocol dissector (#1048) 2020-11-03 16:16:02 +01:00
Luca Deri fc58e564f0 Restored Ubuntu/Debian packaging 2020-11-03 16:08:57 +01:00
Toni 8e84abfa0e Fix/packaging (#1047)
* Fix OpenWrt build.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Fixed Debian/Ubuntu packaging.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Added DPKG package build to a CI job.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-03 11:36:40 +01:00
Ivan Nardi a9547da138 QUIC: fix dissection of Initial packets coalesced with 0-RTT one (#1044)
* QUIC: fix dissection of Initial packets coalesced with 0-RTT one

* QUIC: fix a memory leak
2020-11-03 11:35:52 +01:00
Luca Deri 92b80e2468 Updated results with numeric IP detection 2020-11-01 13:31:00 +01:00
Luca Deri b502bb08e0 Fix for detecting numeric IPs 2020-11-01 13:31:00 +01:00
Luca Deri 50ab34bfe6 Centos8 package fix 2020-10-29 18:36:56 +01:00
Alfredo Cardigliano 53f133f669 Fix libdir on centos8 2020-10-29 18:20:51 +01:00
Alfredo Cardigliano 29dbf78c4e Removed generation of /usr/lib/.build-id 2020-10-29 17:50:05 +01:00
Luca Deri efbfb7cbd6 Set /usr as installation prefix 2020-10-29 17:45:55 +01:00
Alfredo Cardigliano 1c4ce8b0f4 Move libraries from /usr/local to /usr 2020-10-29 12:43:33 +01:00
Vitaliy Ivanov 10506b5173 Generate NDPI_API_VERSION based on git commands.
https://github.com/ntop/PF_RING/issues/668

Check for .git directory works in case of usual clone of nDPI repo.
But if you integrate is as submodule then .git is a file.

Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com>
2020-10-29 12:35:05 +01:00
Luca Deri 94471193c2 Added boundary check 2020-10-27 08:48:12 +01:00
Igor Duarte ba6a48c9fe Improve skype detection (#1039)
* Add new skype pcap

PCAP extracted from SkypeIRC.cap (available in https://wiki.wireshark.org/SampleCaptures?action=AttachFile&do=get&target=SkypeIRC.cap)

* Improve skype detection
2020-10-27 08:45:09 +01:00
Luca Deri 48d640583a Moved global in reader_util.c 2020-10-27 08:40:00 +01:00
Luca Deri 948a906037 Added -D flag for detecting DoH in the wild
Removed heuristic from CiscoVPN as it leads to false positives
2020-10-26 21:40:59 +01:00
Luca Deri 9873972acb Various improvemement when using ndpi_pref_enable_tls_block_dissection:
application data TLS blocks are now ignored when exchanged before
- the end of certificate negotiation (up to TLS 1.2)
- change cipher
2020-10-24 19:22:56 +02:00
Luca Deri 9b85669a64 Added -x for checking patterns 2020-10-22 23:28:18 +02:00
Luca Deri 833d0eee53 Added CPHA - CheckPoint High Availability Protocol protocl support 2020-10-22 18:39:13 +02:00
Ivan Nardi 6027a7c799 Fix parsing of DLT_PPP datalink type (#1042) 2020-10-21 22:27:42 +02:00
Luca Deri 9dac9945c9 Fixes #1033 2020-10-21 20:59:02 +02:00
DouglasandDouglas Wordell efb712c2c0 Added new IPs for TeamViewer (#1040)
Co-authored-by: Douglas Wordell <douglas.wordell@seti.com.br>
2020-10-21 15:09:14 +02:00
Zied Aouini 43c1f6a3fd CAPWAP tunnel decoding fix (#1038)
* Fix CAPWAP processing.

* Update result.
2020-10-21 15:07:20 +02:00
Toni 822c8e56f7 Improved/autoconf (#1037)
* Switched to PKG_CHECK_MODULES to keep pkg-config checks more portable.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Improved GCrypt autoconf check to detect a possible gpg-error inter-library dependency.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-10-21 15:05:33 +02:00
Hosein Ghahremanzadeh bb33d57971 Fix a memory leak on fuzzing code (#1036)
After allocation of pcap_buffer it is necessary to free it
2020-10-21 15:05:11 +02:00
Igor Duarte 13d99a9390 Remove possible invalid read (#1035) 2020-10-21 15:04:55 +02:00
Luca Deri 6c0a809de8 Initial 3.5 commit 2020-10-19 16:23:03 +02:00
509 changed files with 103193 additions and 15605 deletions
+1 -1
View File
@@ -1,4 +1,4 @@
version: 3.1.{build}
mversion: 3.1.{build}
branches:
only:
- dev
+20
View File
@@ -0,0 +1,20 @@
name: Build
on:
push:
branches:
- master
pull_request:
types: [opened, synchronize, reopened]
jobs:
sonarcloud:
name: SonarCloud
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis
- name: SonarCloud Scan
uses: SonarSource/sonarcloud-github-action@master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Needed to get PR information, if any
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
+25 -1
View File
@@ -6,11 +6,30 @@ matrix:
compiler: clang
- os: linux
env: BUILD_DPKG_PACKAGE=y
compiler: gcc
dist: bionic
addons:
apt:
packages:
- fakeroot
- debhelper
- dpkg-dev
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
- os: linux
env: DISABLE_GCRYPT=y
compiler: gcc
dist: bionic
addons:
apt:
packages:
- fakeroot
- debhelper
- dpkg-dev
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
@@ -189,7 +208,11 @@ before_script:
# - lcov --directory . --zerocounters
script:
- if [ -n "$QA_FUZZ" ]; then ./configure --enable-fuzztargets ; else ./configure ; fi
- if [ -n "$QA_FUZZ" ]; then ./configure --enable-fuzztargets ; else
./configure ; fi
- if [ -n "$BUILD_DPKG_PACKAGE" ]; then cd ./packages/ubuntu ;
./configure && dpkg-buildpackage -b -us -uc ; cd ../.. ; fi
- if [ -n "$DISABLE_GCRYPT" ]; then ./configure --disable-gcrypt ; fi
- make
- make -C example ndpiSimpleIntegration
- make dist
@@ -198,6 +221,7 @@ script:
- cd tests
- ./do.sh
- ./do-unit.sh
- ./do-dga.sh
#after_success:
#- cd ${TRAVIS_BUILD_DIR}
+132
View File
@@ -1,5 +1,137 @@
# CHANGELOG
#### nDPI 4.0 (July 2021)
## New Features
- Add API for computing RSI (Relative Strenght Index)
- Add GeoIP support
- Add fragments management
- Add API for jitter calculation
- Add single exponential smoothing API
- Add timeseries forecasting support implementing Holt-Winters with confidence interval
- Add support for MAC to radi tree and expose the full API to applications
- Add JA3+, with ALPN and elliptic curve
- Add double exponential smoothing implementation
- Extended API for managing flow risks
- Add flow risk score
- New flow risks:
- Desktop or File Sharing Session
- HTTP suspicious content (useful for tracking trickbot)
- Malicious JA3
- Malicious SHA1
- Risky domain
- Risky AS
- TLS Certificate Validity Too Long
- TLS Suspicious Extension
## New Supported Protocols and Services
- New protocols:
- AmongUs
- AVAST SecureDNS
- CPHA (CheckPoint High Availability Protocol)
- DisneyPlus
- DTLS
- Genshin Impact
- HP Virtual Machine Group Management (hpvirtgrp)
- Mongodb
- Pinterest
- Reddit
- Snapchat VoIP calls
- Tumblr
- Virtual Asssitant (Alexa, Siri)
- Z39.50
- Add protocols to HTTP as subprotocols
- Add detection of TLS browser type
- Add connectionless DCE/RPC detection
## Improvements
- 2.5x speed bump. Example ndpiReader with a long mixed pcap
v3.4 - nDPI throughput: 1.29 M pps / 3.35 Gb/sec
v4.0 - nDPI throughput: 3.35 M pps / 8.68 Gb/sec
- Improve detection/dissection of:
- AnyDesk
- DNS
- Hulu
- DCE/RPC (avoid false positives)
- dnscrypt
- Facebook (add new networks)
- Fortigate
- FTP Control
- HTTP
- Fix user-agent parsing
- Fix logs when NDPI_ENABLE_DEBUG_MESSAGES is defined
- IEC104
- IEC60870
- IRC
- Netbios
- Netflix
- Ookla speedtest (detection over IPv6)
- openspeedtest.com
- Outlook / MicrosoftMail
- QUIC
- update to draft-33
- improve handling of SNI
- support for fragmented Client Hello
- support for DNS-over-QUIC
- RTSP
- RTSP via HTTP
- SNMP (reimplemented)
- Skype
- SSH
- Steam (Steam Datagram Relay - SDR)
- STUN (avoid false positives, improved Skype detection)
- TeamViewer (add new hosts)
- TOR (update hosts)
- TLS
- Certificate Subject matching
- Check for common ALPNs
- Reworked fingerprint calculation
- Fix extraction for TLS signature algorithms
- Fix ClientHello parsing
- UPnP
- wireguard
- Improve DGA detection
- Improve JA3
- Improve Mining detection
- Improve string matching algorithm
- Improve ndpi_pref_enable_tls_block_dissection
- Optimize speed and memory size
- Update ahocorasick library
- Improve subprotocols detection
## Fixes
- Fix partial application matching
- Fix multiple segfault and leaks
- Fix uninitialized memory use
- Fix release of patterns allocated in ndpi_add_string_to_automa
- Fix return value of ndpi_match_string_subprotocol
- Fix setting of flow risks on 32 bit machines
- Fix TLS certificate threshold
- Fix a memory error in TLS JA3 code
- Fix false positives in Z39.50
- Fix off-by-one memory error for TLS-JA3
- Fix bug in ndpi_lru_find_cache
- Fix invalid xbox and playstation port guesses
- Fix CAPWAP tunnel decoding
- Fix parsing of DLT_PPP datalink type
- Fix dissection of QUIC initial packets coalesced with 0-RTT one
- Fix parsing of GTP headers
- Add bitmap boundary checks
## Misc
- Update download category name
- Update category labels
- Renamed Skype in Skype_Teams (the protocol is now shared across these apps)
- Add IEC analysis wireshark plugin
- Flow risk visualization in Wireshark
- ndpiReader
- add statistics about nDPI performance
- fix memory leak
- fix collecting of risks statistics
- Move installed libraries from /usr/local to /usr
- Improve NDPI_API_VERSION generation
- Update ndpi_ptree_match_addr prototype
#### nDPI 3.4 (October 2020)
## New Features
+1 -1
View File
@@ -11,4 +11,4 @@ pkgconfig_DATA = libndpi.pc
EXTRA_DIST = README.md CHANGELOG.md CONTRIBUTING.md README.protocols autogen.sh configure.seed wireshark python windows utils packages doc/nDPI_QuickStartGuide.pages doc/nDPI_QuickStartGuide.pdf example/MacOS example/Win32
changelog:
git log --since={`curl -s https://github.com/ntop/ndpi/releases | grep datetime | head -n1 | egrep -o "[0-9]+\-[0-9]+\-[0-9]+"`} --name-only --pretty=format:" - %s" | grep "^ " > Changelog.latest
git log --since={`curl -s https://github.com/ntop/ndpi/releases | grep datetime | head -n1 | egrep -o "[0-9]+\-[0-9]+\-[0-9]+"`} --name-only --pretty=format:" - %s" | grep "^ " > CHANGELOG.latest
+12
View File
@@ -26,6 +26,14 @@ Please note that the pre-requisites for compilation include:
- GNU tools (autogen, automake, autoconf, libtool)
- GNU C compiler (gcc)
On Debian/Ubuntu systems do:
- sudo apt-get install build-essential git bison flex libpcap-dev libtool libtool-bin autoconf pkg-config automake autogen libjson-c-dev libnuma-dev libgcrypt20-dev libpcre2-dev
FreeBSD
- sudo pkg install gcc git autoconf automake libtool devel/pkgconf json-c gmake
Remember to use gmake and not make on FreeBSD
### How To Add A New Protocol Dissector
The entire procedure of adding new protocols in detail:
@@ -44,6 +52,10 @@ The entire procedure of adding new protocols in detail:
### How to use nDPI to Block Selected Traffic
You can use nDPI to selectively block selected Internet traffic by embedding it onto an application (remember that nDPI is just a library). Both [ntopng](https://github.com/ntop/ntopng) and [nProbe cento](http://www.ntop.org/products/netflow/nprobe-cento/) can do this.
### nDPI Paper Citation
- Deri, Luca, et al. [nDPI: Open-source high-speed deep packet inspection](http://luca.ntop.org/nDPI.pdf) 2014 International Wireless Communications and Mobile Computing Conference (IWCMC). IEEE, 2014.
### nDPI-Related Projects
- [nfstream](https://github.com/aouinizied/nfstream)
+3 -2
View File
@@ -1,7 +1,7 @@
#!/bin/sh
NDPI_MAJOR="3"
NDPI_MINOR="4"
NDPI_MAJOR="4"
NDPI_MINOR="0"
NDPI_PATCH="0"
NDPI_VERSION_SHORT="$NDPI_MAJOR.$NDPI_MINOR.$NDPI_PATCH"
@@ -57,6 +57,7 @@ autoreconf -ivf
cat configure | sed "s/#define PACKAGE/#define NDPI_PACKAGE/g" | sed "s/#define VERSION/#define NDPI_VERSION/g" > configure.tmp
cat configure.tmp > configure
echo "./configure $@"
chmod +x configure
./configure $@
+61 -12
View File
@@ -5,14 +5,25 @@ AC_CONFIG_MACRO_DIR([m4])
AM_INIT_AUTOMAKE([foreign subdir-objects])
EXTRA_TARGETS="example tests tests/unit"
AC_PREFIX_DEFAULT(/usr)
EXTRA_TARGETS="example tests tests/unit tests/dga"
AC_ARG_WITH(only-libndpi, AS_HELP_STRING([--with-only-libndpi], [Build only libndpi (no examples, tests etc)]))
AS_IF([test "${with_only_libndpi+set}" = set],[
EXTRA_TARGETS=""
])
GNU_PREFIX=""
AC_ARG_WITH(mipsel, AS_HELP_STRING([--with-mipsel], [Build only libndpi (no examples, tests etc) for mipsel]))
AS_IF([test "${with_mipsel+set}" = set],[
EXTRA_TARGETS=""
GNU_PREFIX="mipsel-linux-gnu-"
])
AC_ARG_WITH(sanitizer, AS_HELP_STRING([--with-sanitizer], [Build with support for address, undefined and leak sanitizer]))
AC_ARG_ENABLE(fuzztargets, AS_HELP_STRING([--enable-fuzztargets], [Enable fuzz targets]),[enable_fuzztargets=$enableval],[enable_fuzztargets=no])
AS_IF([test "x$enable_fuzztargets" = "xyes"], [BUILD_FUZZTARGETS=1], [BUILD_FUZZTARGETS=0])
AM_CONDITIONAL([BUILD_FUZZTARGETS], [test "x$enable_fuzztargets" = "xyes"])
AS_IF([test "${with_sanitizer+set}" = set],[
@@ -65,7 +76,10 @@ if test -d ".git" || test -f ".git" ; then :
else
GIT_RELEASE="${PACKAGE_VERSION}"
GIT_DATE=`date -u -r CHANGELOG.md`
NDPI_API_VERSION=`date +%s | cut -c7-10`
if test -z "$SOURCE_DATE_EPOCH" ; then :
SOURCE_DATE_EPOCH=`date +%s`
fi
NDPI_API_VERSION=`echo $SOURCE_DATE_EPOCH | cut -c7-10`
fi
NDPI_API_VERSION=`echo $NDPI_API_VERSION | sed 's/^0*//'`
@@ -73,9 +87,13 @@ NDPI_API_VERSION=`echo $NDPI_API_VERSION | sed 's/^0*//'`
AC_DEFINE_UNQUOTED(NDPI_GIT_RELEASE, "${GIT_RELEASE}", [GIT Release])
AC_DEFINE_UNQUOTED(NDPI_GIT_DATE, "${GIT_DATE}", [Last GIT change])
dnl> used by json-c
JSONC_INC=`pkg-config --cflags json-c`
CFLAGS="${CFLAGS} ${JSONC_INC}"
if ! test "${with_mipsel+set}" = set && ! test "${with_only_libndpi+set}" = set; then :
dnl> used by json-c for unit tests
PKG_CHECK_MODULES([JSONC], [json-c], [JSONC_LIBS=`pkg-config --libs json-c` JSONC_CFLAGS=`pkg-config --cflags json-c`])
fi
LIBS="$LIBS $JSONC_LIBS"
CFLAGS="$CFLAGS $JSONC_CFLAGS"
AC_CHECK_HEADERS([netinet/in.h stdint.h stdlib.h string.h unistd.h json.h])
@@ -110,6 +128,10 @@ if test -d ../nDPI-custom; then :
AC_MSG_RESULT([Compiling with custom nDPI protocols])
fi
if test -d ../ndpi-pro; then :
with_maxminddb=1
fi
case "$host" in
*-*-mingw32*|*-*-msys)
CFLAGS="${CFLAGS} -DOS_WIN32"
@@ -176,14 +198,23 @@ AM_CONDITIONAL([HAS_FUZZLDFLAGS], [test "x$has_sanitizefuzzer" = "xyes"])
AC_CHECK_LIB(pthread, pthread_setaffinity_np, AC_DEFINE_UNQUOTED(HAVE_PTHREAD_SETAFFINITY_NP, 1, [libc has pthread_setaffinity_np]))
dnl> GCRYPT
GCRYPT_ENABLED=1
AC_ARG_ENABLE([gcrypt],
[AS_HELP_STRING([--disable-gcrypt], [Avoid compiling with libgcrypt/libgpg-error, even if they are present. QUIC sub-classification may be missing])],
[:],
[AC_CHECK_LIB(gcrypt, gcry_cipher_checktag)
AC_CHECK_LIB(gpg-error, gpg_strerror_r)])
if test "x$ac_cv_lib_gcrypt_gcry_cipher_checktag" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lgcrypt"
fi
[GCRYPT_ENABLED=0],
[AC_CHECK_LIB(gcrypt, gcry_cipher_checktag)])
AS_IF([test "x$enable_gcrypt" = "xyes"], [
if test "x$ac_cv_lib_gcrypt_gcry_cipher_checktag" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lgcrypt"
else
$as_unset ac_cv_lib_gcrypt_gcry_cipher_checktag
AC_CHECK_LIB(gpg-error, gpg_strerror_r)
AC_CHECK_LIB(gcrypt, gcry_cipher_checktag)
if test "x$ac_cv_lib_gcrypt_gcry_cipher_checktag" = xyes -a "x$ac_cv_lib_gpg_error_gpg_strerror_r" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lgcrypt -lgpg-error"
fi
fi
])
dnl> PCRE
AC_ARG_WITH(pcre, [ --with-pcre Enable nDPI build with libpcre])
@@ -194,8 +225,20 @@ if test "${with_pcre+set}" = set; then :
fi
fi
dnl> GeoIP
AC_ARG_WITH(maxminddb, [ --with-maxminddb Enable nDPI build with libmaxminddb])
if test "${with_maxminddb+set}" = set; then :
AC_CHECK_LIB([maxminddb], [MMDB_lookup_sockaddr])
AC_HAVE_HEADERS(maxminddb.h)
if test ".${ac_cv_lib_maxminddb_MMDB_lookup_sockaddr}" = ".yes" && test ".${ac_cv_header_maxminddb_h}" = ".yes"; then
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lmaxminddb"
AC_DEFINE_UNQUOTED(HAVE_MAXMINDDB, 1, [MaxMind DB support])
fi
fi
AC_CONFIG_FILES([Makefile example/Makefile example/Makefile.dpdk tests/Makefile tests/unit/Makefile libndpi.pc src/include/ndpi_define.h src/lib/Makefile python/Makefile fuzz/Makefile src/include/ndpi_api.h])
AC_CONFIG_FILES([Makefile example/Makefile example/Makefile.dpdk tests/Makefile tests/unit/Makefile tests/dga/Makefile libndpi.pc src/include/ndpi_define.h src/lib/Makefile python/Makefile fuzz/Makefile src/include/ndpi_api.h])
AC_CONFIG_FILES([tests/do.sh], [chmod +x tests/do.sh])
AC_CONFIG_FILES([tests/do_valgrind.sh], [chmod +x tests/do_valgrind.sh])
AC_CONFIG_HEADERS(src/include/ndpi_config.h)
AC_SUBST(GIT_RELEASE)
AC_SUBST(NDPI_MAJOR)
@@ -212,4 +255,10 @@ AC_SUBST(CUSTOM_NDPI)
AC_SUBST(NDPI_API_VERSION)
AC_SUBST(EXTRA_TARGETS)
AC_SUBST(BUILD_MINGW)
AC_SUBST(BUILD_FUZZTARGETS)
AC_SUBST(JSONC_CFLAGS)
AC_SUBST(JSONC_LIBS)
AC_SUBST(GCRYPT_ENABLED)
AC_SUBST(GNU_PREFIX)
AC_SUBST(FRAG_MAN_ENABLED)
AC_OUTPUT
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* intrusion_detection.c
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* intrusion_detection.h
*
* Copyright (C) 2011-19 - ntop.org
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+109
View File
@@ -0,0 +1,109 @@
################################################################
# abuse.ch Suricata JA3 Fingerprint Blacklist (CSV) #
# For Suricata 4.1.0 or newer #
# Last updated: 2020-04-09 06:48:14 UTC #
# #
# Terms Of Use: https://sslbl.abuse.ch/blacklist/ #
# For questions please contact sslbl [at] abuse.ch #
################################################################
#
# ja3_md5,Firstseen,Lastseen,Listingreason
b386946a5a44d1ddcc843bc75336dfce,2017-07-14 18:08:15,2019-07-27 20:42:54,Dridex
8991a387e4cc841740f25d6f5139f92d,2017-07-14 19:02:03,2019-07-28 00:34:38,Adware
cb98a24ee4b9134448ffb5714fd870ac,2017-07-14 19:48:28,2019-05-22 03:22:38,Dridex
1aa7bf8b97e540ca5edd75f7b8384bfa,2017-07-14 20:23:38,2019-07-28 01:38:22,TrickBot
3d89c0dfb1fa44911b8fa7523ef8dedb,2017-07-15 04:23:45,2021-02-01 18:23:25,Adware
bc6c386f480ee97b9d9e52d472b772d8,2017-07-15 10:57:38,2021-02-21 13:13:30,Adware
8f52d1ce303fb4a6515836aec3cc16b1,2017-07-15 19:05:11,2019-07-27 20:00:57,TrickBot
d6f04b5a910115f4b50ecec09d40a1df,2017-07-15 19:42:24,2018-10-14 08:12:51,Dridex
35c0a31c481927f022a3b530255ac080,2017-07-15 19:43:19,2021-02-06 03:21:55,Tofsee
d551fafc4f40f1dec2bb45980bfa9492,2017-07-15 19:59:29,2020-11-16 13:06:20,Adware
e330bca99c8a5256ae126a55c4c725c5,2017-07-15 19:59:29,2021-01-13 00:29:37,Adware
b8f81673c0e1d29908346f3bab892b9b,2017-07-16 01:32:03,2018-12-17 06:08:03,Adware
83e04bc58d402f9633983cbf22724b02,2017-07-16 01:32:03,2021-01-14 14:23:42,Adware
70722097d1fe1d78d8c2164640ab6df4,2017-07-16 02:39:08,2021-02-22 04:56:57,Tofsee
9c2589e1c0e9f533a022c6205f9719e1,2017-07-16 08:37:17,2021-02-19 08:46:32,Adware
849b04bdbd1d2b983f6e8a457e0632a8,2017-07-16 08:37:17,2021-02-19 08:46:32,Adware
16efcf0e00504ddfedde13bfea997952,2017-07-16 19:45:45,2020-12-23 15:10:32,Adware
4d7a28d6f2263ed61de88ca66eb011e3,2017-07-16 21:20:29,2020-12-08 18:10:55,Tofsee
550dce18de1bb143e69d6dd9413b8355,2017-07-16 22:17:20,2018-12-21 07:04:50,Adware
c50f6a8b9173676b47ba6085bd0c6cee,2017-07-16 22:38:41,2019-05-21 09:42:17,TrickBot
20dd18bdd3209ea718989030a6f93364,2017-07-18 10:22:58,2019-04-28 09:23:31,Adware
8498fe4268764dbf926a38283e9d3d8f,2017-07-18 10:22:58,2020-12-31 02:06:27,Adware
590a232d04d56409fab72e752a8a2634,2017-07-18 18:53:24,2020-10-11 20:48:33,Tofsee
51a7ad14509fd614c7bb3a50c4982b8c,2017-07-19 07:28:19,2019-07-14 11:58:32,JBifrost
96eba628dcb2b47607192ba74a3b55ba,2017-07-19 18:53:48,2021-01-04 12:45:08,Tofsee
df5c30e670dba99f9270ed36060cf054,2017-07-20 17:44:07,2018-04-11 15:57:59,Tofsee
098f55e27d8c4b0a590102cbdb3a5f3a,2017-07-21 09:52:01,2019-04-08 01:09:54,Adware
46efd49abcca8ea9baa932da68fdb529,2017-07-22 14:07:36,2021-02-20 20:45:32,Adware
29085f03f8e8a03f0b399c5c7cf0b0b8,2017-07-22 14:07:36,2021-02-21 20:56:01,Adware
d7150af4514b868defb854db0f62a441,2017-07-23 09:39:24,2018-07-24 01:04:58,Tofsee
03e186a7f83285e93341de478334006e,2017-07-24 18:17:14,2021-02-13 23:55:58,Tofsee
3cda52da4ade09f1f781ad2e82dcfa20,2017-07-30 18:41:36,2019-05-21 17:34:18,Quakbot
b13d01846ad7a14a70bf030a16775c78,2017-08-08 07:12:49,2021-02-21 01:32:12,Adware
1543a7c46633acf71e8401baccbd0568,2017-08-08 21:32:28,2020-11-10 05:30:17,Tofsee
1d095e68489d3c535297cd8dffb06cb9,2017-08-12 19:56:28,2020-10-28 11:06:23,Tofsee
93d056782d649deb51cda44ecb714bb0,2017-08-28 12:20:47,2019-04-15 23:47:27,Adware
698e36219f3979420fa2581b21dac7ec,2017-08-28 12:20:47,2020-12-31 02:06:31,Adware
1712287800ac91b34cadd5884ce85568,2017-08-28 16:01:59,2021-01-23 09:36:32,TorrentLocker
5e573c9c9f8ba720ef9b18e9fce2e2f7,2017-08-30 13:44:56,2021-02-17 17:11:34,Adware
f6fd83a21f9f3c5f9ff7b5c63bbc179d,2017-10-20 08:03:21,2018-11-06 06:42:12,Adware
92579701f145605e9edc0b01a901c6d5,2017-10-23 00:10:48,2021-02-18 22:56:49,Adware
a61299f9b501adcf680b9275d79d4ac6,2017-11-04 18:03:59,2020-04-21 17:08:24,Tofsee
b2b61db7b9490a60d270ccb20b462826,2017-11-14 20:12:03,2021-02-18 22:56:49,Adware
7dcce5b76c8b17472d024758970a406b,2017-11-22 12:42:46,2020-12-22 15:15:49,Tofsee
534ce2dbc413c68e908363b5df0ae5e0,2017-12-22 09:36:21,2019-07-27 15:22:33,TrickBot
fb00055a1196aeea8d1bc609885ba953,2018-01-01 22:49:25,2019-04-09 06:58:58,TrickBot
a50a861119aceb0ccc74902e8fddb618,2018-01-02 08:16:23,2018-07-05 02:33:08,Tofsee
e7643725fcff971e3051fe0e47fc2c71,2018-01-31 08:06:13,2020-03-25 16:19:48,Tofsee
7c410ce832e848a3321432c9a82e972b,2018-01-31 20:04:25,2021-02-21 22:51:31,Tofsee
da949afd9bd6df820730f8f171584a71,2018-02-03 05:19:37,2021-02-21 05:20:43,Tofsee
906004246f3ba5e755b043c057254a29,2018-03-11 08:25:38,2018-04-14 00:59:16,Tofsee
fd80fa9c6120cdeea8520510f3c644ac,2018-03-11 09:34:30,2021-02-21 22:16:39,Tofsee
b90bdbe961a648f0427db21aaa6ccb59,2018-03-11 10:37:43,2020-05-29 23:39:01,Tofsee
1fe4c7a3544eb27afec2adfb3a3dbf60,2018-03-11 19:23:08,2021-02-20 11:19:52,Tofsee
c201b92f8b483fa388be174d6689f534,2018-03-12 13:43:52,2021-01-28 06:17:06,Gozi
9f62c4f26b90d3d757bea609e82f2eaf,2018-03-13 06:23:41,2020-12-18 16:07:31,Tofsee
1be3ecebe5aa9d3654e6e703d81f6928,2018-03-13 11:50:02,2021-02-14 23:03:13,Ransomware.Troldesh
e3b2ab1f9a56f2fb4c9248f2f41631fa,2018-03-15 01:06:34,2021-02-22 10:36:48,Tofsee
dff8a0aa1c904aaea76c5bf624e88333,2018-03-18 09:41:15,2020-10-27 09:50:24,Tofsee
17fd49722f8d11f3d76dce84f8e099a7,2018-03-19 23:02:27,2020-12-09 11:32:21,Tofsee
911479ac8a0813ed1241b3686ccdade9,2018-03-19 23:24:59,2020-03-30 04:09:18,Tofsee
c5deb9465d47232dd48772f9c4d14679,2018-03-22 15:42:48,2021-02-17 07:39:32,Tofsee
f22bdd57e3a52de86cda40da2d84e83b,2018-03-27 13:40:19,2019-01-20 14:31:39,Tofsee
d18a4da84af59e1108862a39bae7c9d4,2018-04-03 00:40:51,2021-02-06 01:53:12,Tofsee
2d8794cb7b52b777bee2695e79c15760,2018-04-04 06:56:37,2021-02-10 08:22:31,Ransomware
40adfd923eb82b89d8836ba37a19bca1,2018-04-15 15:49:08,2021-02-21 22:22:56,CoinMiner
1aee0238942d453d679fc1e37a303387,2018-05-13 01:59:49,2021-02-10 13:19:24,Tofsee
2092e1fffb45d7e4a19a57f9bc5e203a,2018-05-16 21:59:36,2018-09-05 01:58:33,Adware
bffa4501966196d3d6e90cee1f88fc89,2018-06-07 15:08:04,2020-03-16 00:03:44,Tofsee
807fca46d9d0cf63adf4e5e80e414bbe,2018-06-07 16:51:03,2021-02-20 03:15:38,Tofsee
fb58831f892190644fe44e25bc830b45,2018-06-08 12:07:59,2019-05-31 21:02:37,Adware
0cc1e84568e471aa1d62ad4158ade6b5,2018-06-24 10:50:47,2021-02-10 21:48:57,Tofsee
d2935c58fe676744fecc8614ee5356c7,2018-08-14 21:48:41,2021-02-21 21:44:55,Adwind
8916410db85077a5460817142dcbc8de,2018-08-21 12:32:28,2021-02-21 20:02:19,TrickBot
c5235d3a8b9934b7fbbd204d50bc058d,2018-08-23 17:36:08,2019-10-13 05:11:09,Gootkit
57f3642b4e37e28f5cbe3020c9331b4c,2018-08-28 15:54:53,2021-02-22 11:09:43,Gozi
e62a5f4d538cbf169c2af71bec2399b4,2018-08-30 15:45:40,2021-02-21 20:36:55,TrickBot
51c64c77e60f3980eea90869b68c58a8,2018-08-30 21:04:57,2021-02-21 22:09:16,Dridex
7691297bcb20a41233fd0a0baa0a3628,2018-09-17 02:50:05,2021-02-21 17:05:57,Adware
7dd50e112cd23734a310b90f6f44a7cd,2018-09-17 17:54:58,2021-02-22 11:14:04,Quakbot
52c7396a501e4fecbdfa99c5408334ac,2018-09-18 00:29:04,2019-12-03 17:24:02,Tofsee
f735bbc6b69723b9df7b0e7ef27872af,2018-10-02 18:04:16,2021-02-19 06:55:39,TrickBot
49ed2ef3f1321e5f044f1e71b0e6fdd5,2018-10-02 18:04:17,2021-02-19 06:55:39,TrickBot
d76ee64fb7273733cbe455ac81c292e6,2018-11-16 13:26:39,2018-11-18 19:19:36,Tofsee
8f6c918dcb585ebbea05e2cc94530e3d,2018-11-16 13:26:41,2020-05-06 15:45:21,Tofsee
34f14a69ad7009ca5863379218af17f3,2018-11-17 05:17:22,2021-01-28 08:19:18,Tofsee
c2b4710c6888a5d47befe865c8e6fb19,2018-11-29 20:46:04,2020-12-16 15:08:09,Tofsee
decfb48a53789ebe081b88aabb58ee34,2018-12-21 09:06:16,2021-02-16 12:28:53,Adwind
08a8a4e85b25ac42e1490bc85cfdb5ce,2019-01-30 02:48:34,2020-10-27 09:50:19,Tofsee
c0220cd64849a629397a9cb68f78a0ea,2019-03-24 00:12:32,2021-02-13 05:13:33,Tofsee
7a29c223fb122ec64d10f0a159e07996,2019-06-09 22:55:29,2020-10-27 09:50:26,Tofsee
44dab16d680ef93487bc16ad23b3ffb1,2019-06-09 22:55:29,2020-10-27 09:50:25,Tofsee
70a04365be5bbd4653698bebeb43ce68,2019-07-02 06:26:56,2020-05-30 04:19:00,Tofsee
d81d654effb94714a4086734fa0adad9,2019-07-16 23:29:02,2020-10-27 09:50:21,Tofsee
25d74b7b4b779eb1efd4b31d26d651c6,2019-08-03 20:15:33,2020-07-14 21:43:25,Tofsee
fc2299d5b2964cd242c5a2c8c531a5f0,2019-08-09 23:56:32,2021-02-22 08:49:18,Tofsee
32926ca3e59f0413d0b98725454594f5,2019-09-12 06:56:10,2020-10-27 21:49:31,Tofsee
ffefafdb86336d057eda5fdf02b3d5ce,2019-10-26 07:31:49,2020-07-25 00:14:09,Tofsee
# END (98) entries
1 ################################################################
2 # abuse.ch Suricata JA3 Fingerprint Blacklist (CSV) #
3 # For Suricata 4.1.0 or newer #
4 # Last updated: 2020-04-09 06:48:14 UTC #
5 # #
6 # Terms Of Use: https://sslbl.abuse.ch/blacklist/ #
7 # For questions please contact sslbl [at] abuse.ch #
8 ################################################################
9 #
10 # ja3_md5,Firstseen,Lastseen,Listingreason
11 b386946a5a44d1ddcc843bc75336dfce,2017-07-14 18:08:15,2019-07-27 20:42:54,Dridex
12 8991a387e4cc841740f25d6f5139f92d,2017-07-14 19:02:03,2019-07-28 00:34:38,Adware
13 cb98a24ee4b9134448ffb5714fd870ac,2017-07-14 19:48:28,2019-05-22 03:22:38,Dridex
14 1aa7bf8b97e540ca5edd75f7b8384bfa,2017-07-14 20:23:38,2019-07-28 01:38:22,TrickBot
15 3d89c0dfb1fa44911b8fa7523ef8dedb,2017-07-15 04:23:45,2021-02-01 18:23:25,Adware
16 bc6c386f480ee97b9d9e52d472b772d8,2017-07-15 10:57:38,2021-02-21 13:13:30,Adware
17 8f52d1ce303fb4a6515836aec3cc16b1,2017-07-15 19:05:11,2019-07-27 20:00:57,TrickBot
18 d6f04b5a910115f4b50ecec09d40a1df,2017-07-15 19:42:24,2018-10-14 08:12:51,Dridex
19 35c0a31c481927f022a3b530255ac080,2017-07-15 19:43:19,2021-02-06 03:21:55,Tofsee
20 d551fafc4f40f1dec2bb45980bfa9492,2017-07-15 19:59:29,2020-11-16 13:06:20,Adware
21 e330bca99c8a5256ae126a55c4c725c5,2017-07-15 19:59:29,2021-01-13 00:29:37,Adware
22 b8f81673c0e1d29908346f3bab892b9b,2017-07-16 01:32:03,2018-12-17 06:08:03,Adware
23 83e04bc58d402f9633983cbf22724b02,2017-07-16 01:32:03,2021-01-14 14:23:42,Adware
24 70722097d1fe1d78d8c2164640ab6df4,2017-07-16 02:39:08,2021-02-22 04:56:57,Tofsee
25 9c2589e1c0e9f533a022c6205f9719e1,2017-07-16 08:37:17,2021-02-19 08:46:32,Adware
26 849b04bdbd1d2b983f6e8a457e0632a8,2017-07-16 08:37:17,2021-02-19 08:46:32,Adware
27 16efcf0e00504ddfedde13bfea997952,2017-07-16 19:45:45,2020-12-23 15:10:32,Adware
28 4d7a28d6f2263ed61de88ca66eb011e3,2017-07-16 21:20:29,2020-12-08 18:10:55,Tofsee
29 550dce18de1bb143e69d6dd9413b8355,2017-07-16 22:17:20,2018-12-21 07:04:50,Adware
30 c50f6a8b9173676b47ba6085bd0c6cee,2017-07-16 22:38:41,2019-05-21 09:42:17,TrickBot
31 20dd18bdd3209ea718989030a6f93364,2017-07-18 10:22:58,2019-04-28 09:23:31,Adware
32 8498fe4268764dbf926a38283e9d3d8f,2017-07-18 10:22:58,2020-12-31 02:06:27,Adware
33 590a232d04d56409fab72e752a8a2634,2017-07-18 18:53:24,2020-10-11 20:48:33,Tofsee
34 51a7ad14509fd614c7bb3a50c4982b8c,2017-07-19 07:28:19,2019-07-14 11:58:32,JBifrost
35 96eba628dcb2b47607192ba74a3b55ba,2017-07-19 18:53:48,2021-01-04 12:45:08,Tofsee
36 df5c30e670dba99f9270ed36060cf054,2017-07-20 17:44:07,2018-04-11 15:57:59,Tofsee
37 098f55e27d8c4b0a590102cbdb3a5f3a,2017-07-21 09:52:01,2019-04-08 01:09:54,Adware
38 46efd49abcca8ea9baa932da68fdb529,2017-07-22 14:07:36,2021-02-20 20:45:32,Adware
39 29085f03f8e8a03f0b399c5c7cf0b0b8,2017-07-22 14:07:36,2021-02-21 20:56:01,Adware
40 d7150af4514b868defb854db0f62a441,2017-07-23 09:39:24,2018-07-24 01:04:58,Tofsee
41 03e186a7f83285e93341de478334006e,2017-07-24 18:17:14,2021-02-13 23:55:58,Tofsee
42 3cda52da4ade09f1f781ad2e82dcfa20,2017-07-30 18:41:36,2019-05-21 17:34:18,Quakbot
43 b13d01846ad7a14a70bf030a16775c78,2017-08-08 07:12:49,2021-02-21 01:32:12,Adware
44 1543a7c46633acf71e8401baccbd0568,2017-08-08 21:32:28,2020-11-10 05:30:17,Tofsee
45 1d095e68489d3c535297cd8dffb06cb9,2017-08-12 19:56:28,2020-10-28 11:06:23,Tofsee
46 93d056782d649deb51cda44ecb714bb0,2017-08-28 12:20:47,2019-04-15 23:47:27,Adware
47 698e36219f3979420fa2581b21dac7ec,2017-08-28 12:20:47,2020-12-31 02:06:31,Adware
48 1712287800ac91b34cadd5884ce85568,2017-08-28 16:01:59,2021-01-23 09:36:32,TorrentLocker
49 5e573c9c9f8ba720ef9b18e9fce2e2f7,2017-08-30 13:44:56,2021-02-17 17:11:34,Adware
50 f6fd83a21f9f3c5f9ff7b5c63bbc179d,2017-10-20 08:03:21,2018-11-06 06:42:12,Adware
51 92579701f145605e9edc0b01a901c6d5,2017-10-23 00:10:48,2021-02-18 22:56:49,Adware
52 a61299f9b501adcf680b9275d79d4ac6,2017-11-04 18:03:59,2020-04-21 17:08:24,Tofsee
53 b2b61db7b9490a60d270ccb20b462826,2017-11-14 20:12:03,2021-02-18 22:56:49,Adware
54 7dcce5b76c8b17472d024758970a406b,2017-11-22 12:42:46,2020-12-22 15:15:49,Tofsee
55 534ce2dbc413c68e908363b5df0ae5e0,2017-12-22 09:36:21,2019-07-27 15:22:33,TrickBot
56 fb00055a1196aeea8d1bc609885ba953,2018-01-01 22:49:25,2019-04-09 06:58:58,TrickBot
57 a50a861119aceb0ccc74902e8fddb618,2018-01-02 08:16:23,2018-07-05 02:33:08,Tofsee
58 e7643725fcff971e3051fe0e47fc2c71,2018-01-31 08:06:13,2020-03-25 16:19:48,Tofsee
59 7c410ce832e848a3321432c9a82e972b,2018-01-31 20:04:25,2021-02-21 22:51:31,Tofsee
60 da949afd9bd6df820730f8f171584a71,2018-02-03 05:19:37,2021-02-21 05:20:43,Tofsee
61 906004246f3ba5e755b043c057254a29,2018-03-11 08:25:38,2018-04-14 00:59:16,Tofsee
62 fd80fa9c6120cdeea8520510f3c644ac,2018-03-11 09:34:30,2021-02-21 22:16:39,Tofsee
63 b90bdbe961a648f0427db21aaa6ccb59,2018-03-11 10:37:43,2020-05-29 23:39:01,Tofsee
64 1fe4c7a3544eb27afec2adfb3a3dbf60,2018-03-11 19:23:08,2021-02-20 11:19:52,Tofsee
65 c201b92f8b483fa388be174d6689f534,2018-03-12 13:43:52,2021-01-28 06:17:06,Gozi
66 9f62c4f26b90d3d757bea609e82f2eaf,2018-03-13 06:23:41,2020-12-18 16:07:31,Tofsee
67 1be3ecebe5aa9d3654e6e703d81f6928,2018-03-13 11:50:02,2021-02-14 23:03:13,Ransomware.Troldesh
68 e3b2ab1f9a56f2fb4c9248f2f41631fa,2018-03-15 01:06:34,2021-02-22 10:36:48,Tofsee
69 dff8a0aa1c904aaea76c5bf624e88333,2018-03-18 09:41:15,2020-10-27 09:50:24,Tofsee
70 17fd49722f8d11f3d76dce84f8e099a7,2018-03-19 23:02:27,2020-12-09 11:32:21,Tofsee
71 911479ac8a0813ed1241b3686ccdade9,2018-03-19 23:24:59,2020-03-30 04:09:18,Tofsee
72 c5deb9465d47232dd48772f9c4d14679,2018-03-22 15:42:48,2021-02-17 07:39:32,Tofsee
73 f22bdd57e3a52de86cda40da2d84e83b,2018-03-27 13:40:19,2019-01-20 14:31:39,Tofsee
74 d18a4da84af59e1108862a39bae7c9d4,2018-04-03 00:40:51,2021-02-06 01:53:12,Tofsee
75 2d8794cb7b52b777bee2695e79c15760,2018-04-04 06:56:37,2021-02-10 08:22:31,Ransomware
76 40adfd923eb82b89d8836ba37a19bca1,2018-04-15 15:49:08,2021-02-21 22:22:56,CoinMiner
77 1aee0238942d453d679fc1e37a303387,2018-05-13 01:59:49,2021-02-10 13:19:24,Tofsee
78 2092e1fffb45d7e4a19a57f9bc5e203a,2018-05-16 21:59:36,2018-09-05 01:58:33,Adware
79 bffa4501966196d3d6e90cee1f88fc89,2018-06-07 15:08:04,2020-03-16 00:03:44,Tofsee
80 807fca46d9d0cf63adf4e5e80e414bbe,2018-06-07 16:51:03,2021-02-20 03:15:38,Tofsee
81 fb58831f892190644fe44e25bc830b45,2018-06-08 12:07:59,2019-05-31 21:02:37,Adware
82 0cc1e84568e471aa1d62ad4158ade6b5,2018-06-24 10:50:47,2021-02-10 21:48:57,Tofsee
83 d2935c58fe676744fecc8614ee5356c7,2018-08-14 21:48:41,2021-02-21 21:44:55,Adwind
84 8916410db85077a5460817142dcbc8de,2018-08-21 12:32:28,2021-02-21 20:02:19,TrickBot
85 c5235d3a8b9934b7fbbd204d50bc058d,2018-08-23 17:36:08,2019-10-13 05:11:09,Gootkit
86 57f3642b4e37e28f5cbe3020c9331b4c,2018-08-28 15:54:53,2021-02-22 11:09:43,Gozi
87 e62a5f4d538cbf169c2af71bec2399b4,2018-08-30 15:45:40,2021-02-21 20:36:55,TrickBot
88 51c64c77e60f3980eea90869b68c58a8,2018-08-30 21:04:57,2021-02-21 22:09:16,Dridex
89 7691297bcb20a41233fd0a0baa0a3628,2018-09-17 02:50:05,2021-02-21 17:05:57,Adware
90 7dd50e112cd23734a310b90f6f44a7cd,2018-09-17 17:54:58,2021-02-22 11:14:04,Quakbot
91 52c7396a501e4fecbdfa99c5408334ac,2018-09-18 00:29:04,2019-12-03 17:24:02,Tofsee
92 f735bbc6b69723b9df7b0e7ef27872af,2018-10-02 18:04:16,2021-02-19 06:55:39,TrickBot
93 49ed2ef3f1321e5f044f1e71b0e6fdd5,2018-10-02 18:04:17,2021-02-19 06:55:39,TrickBot
94 d76ee64fb7273733cbe455ac81c292e6,2018-11-16 13:26:39,2018-11-18 19:19:36,Tofsee
95 8f6c918dcb585ebbea05e2cc94530e3d,2018-11-16 13:26:41,2020-05-06 15:45:21,Tofsee
96 34f14a69ad7009ca5863379218af17f3,2018-11-17 05:17:22,2021-01-28 08:19:18,Tofsee
97 c2b4710c6888a5d47befe865c8e6fb19,2018-11-29 20:46:04,2020-12-16 15:08:09,Tofsee
98 decfb48a53789ebe081b88aabb58ee34,2018-12-21 09:06:16,2021-02-16 12:28:53,Adwind
99 08a8a4e85b25ac42e1490bc85cfdb5ce,2019-01-30 02:48:34,2020-10-27 09:50:19,Tofsee
100 c0220cd64849a629397a9cb68f78a0ea,2019-03-24 00:12:32,2021-02-13 05:13:33,Tofsee
101 7a29c223fb122ec64d10f0a159e07996,2019-06-09 22:55:29,2020-10-27 09:50:26,Tofsee
102 44dab16d680ef93487bc16ad23b3ffb1,2019-06-09 22:55:29,2020-10-27 09:50:25,Tofsee
103 70a04365be5bbd4653698bebeb43ce68,2019-07-02 06:26:56,2020-05-30 04:19:00,Tofsee
104 d81d654effb94714a4086734fa0adad9,2019-07-16 23:29:02,2020-10-27 09:50:21,Tofsee
105 25d74b7b4b779eb1efd4b31d26d651c6,2019-08-03 20:15:33,2020-07-14 21:43:25,Tofsee
106 fc2299d5b2964cd242c5a2c8c531a5f0,2019-08-09 23:56:32,2021-02-22 08:49:18,Tofsee
107 32926ca3e59f0413d0b98725454594f5,2019-09-12 06:56:10,2020-10-27 21:49:31,Tofsee
108 ffefafdb86336d057eda5fdf02b3d5ce,2019-10-26 07:31:49,2020-07-25 00:14:09,Tofsee
109 # END (98) entries
+1042 -178
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+16 -1
View File
@@ -10,7 +10,7 @@ tcp:3000@ntop
# Format:
# host:"<value>",host:"<value>",.....@<subproto>
host:"disneyplus.com"host:"cdn.registerdisney.go.com",host:"disney-portal.my.onetrust.com",host:"disneyplus.bn5x.net",host:"disney-plus.net"@DisneyPlus
host:"disneyplus.com",host:"cdn.registerdisney.go.com",host:"disney-portal.my.onetrust.com",host:"disneyplus.bn5x.net",host:"disney-plus.net"@DisneyPlus
host:"*.lvlt.dash.us.aiv-cdn.net.c.footprint.net"@AmazonVideo
host:"api-global.netflix.com"@Netflix
# IP based Subprotocols
@@ -39,3 +39,18 @@ ip:213.75.170.11/32:443@CustomProtocol
ip:8.248.73.247:443@AmazonPrime
ip:54.80.47.130@AmazonPrime
#
# Risk Exceptions
#
# ip_risk_mask: used to mask flow risks for IP addresses
# host_risk_mask: used to mask exceptions for domain names and hosts
#
# Syntax: <name>=<64 bit mask to be put in AND with the risk
#
# For IPs, the flow risk is put in AND (source IP mask OR destination IP mask)
# For Flows with a hostname (e.g. TLS) the risk is also put in AND with the host_risk_mask
#ip_risk_mask:192.168.1.0/24=0
#ip_risk_mask:10.196.157.228=0
host_risk_mask:".local"=0
host_risk_mask:".msftconnecttest.com"=0
+189 -89
View File
@@ -1,7 +1,7 @@
/*
* reader_util.c
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -38,6 +38,8 @@
#include <float.h>
#endif
#include "reader_util.h"
#ifndef ETH_P_IP
#define ETH_P_IP 0x0800 /* IPv4 */
#endif
@@ -77,20 +79,18 @@
#define DLT_LINUX_SLL 113
#endif
#define PLEN_MAX 1504
#define PLEN_BIN_LEN 32
#define PLEN_NUM_BINS 48 /* 47*32 = 1504 */
#define MAX_NUM_BIN_PKTS 256
#include "ndpi_main.h"
#include "reader_util.h"
#include "ndpi_classify.h"
extern u_int8_t enable_protocol_guess, enable_joy_stats, enable_payload_analyzer;
extern u_int8_t verbose, human_readeable_string_len;
extern u_int8_t max_num_udp_dissected_pkts /* 8 */, max_num_tcp_dissected_pkts /* 10 */;
extern u_int8_t max_num_udp_dissected_pkts /* 24 */, max_num_tcp_dissected_pkts /* 80 */;
static u_int32_t flow_id = 0;
u_int8_t enable_doh_dot_detection = 0;
u_int8_t enable_ja3_plus = 0;
/* ****************************************************** */
struct flow_id_stats {
@@ -415,7 +415,7 @@ struct ndpi_workflow* ndpi_workflow_init(const struct ndpi_workflow_prefs * pref
set_ndpi_flow_malloc(NULL), set_ndpi_flow_free(NULL);
/* TODO: just needed here to init ndpi ndpi_malloc wrapper */
module = ndpi_init_detection_module(ndpi_no_prefs);
module = ndpi_init_detection_module(enable_ja3_plus ? ndpi_enable_ja3_plus : ndpi_no_prefs);
if(module == NULL) {
LOG(NDPI_LOG_ERROR, "global structure initialization failed\n");
@@ -495,13 +495,13 @@ static void ndpi_free_flow_tls_data(struct ndpi_flow_info *flow) {
/* ***************************************************** */
static void ndpi_free_flow_data_analysis(struct ndpi_flow_info *flow) {
if(flow->iat_c_to_s) ndpi_free_data_analysis(flow->iat_c_to_s);
if(flow->iat_s_to_c) ndpi_free_data_analysis(flow->iat_s_to_c);
if(flow->iat_c_to_s) ndpi_free_data_analysis(flow->iat_c_to_s, 1);
if(flow->iat_s_to_c) ndpi_free_data_analysis(flow->iat_s_to_c, 1);
if(flow->pktlen_c_to_s) ndpi_free_data_analysis(flow->pktlen_c_to_s);
if(flow->pktlen_s_to_c) ndpi_free_data_analysis(flow->pktlen_s_to_c);
if(flow->pktlen_c_to_s) ndpi_free_data_analysis(flow->pktlen_c_to_s, 1);
if(flow->pktlen_s_to_c) ndpi_free_data_analysis(flow->pktlen_s_to_c, 1);
if(flow->iat_flow) ndpi_free_data_analysis(flow->iat_flow);
if(flow->iat_flow) ndpi_free_data_analysis(flow->iat_flow, 1);
}
/* ***************************************************** */
@@ -781,11 +781,11 @@ static struct ndpi_flow_info *get_ndpi_flow_info(struct ndpi_workflow * workflow
flow.protocol = iph->protocol, flow.vlan_id = vlan_id;
flow.src_ip = iph->saddr, flow.dst_ip = iph->daddr;
flow.src_port = htons(*sport), flow.dst_port = htons(*dport);
flow.hashval = hashval = flow.protocol + flow.vlan_id + flow.src_ip + flow.dst_ip + flow.src_port + flow.dst_port;
flow.hashval = hashval = flow.protocol + flow.src_ip + flow.dst_ip + flow.src_port + flow.dst_port;
#if 0
printf("hashval=%u [%u][%u][%u:%u][%u:%u]\n", hashval, flow.protocol, flow.vlan_id,
flow.src_ip, flow.src_port, flow.dst_ip, flow.dst_port);
flow.src_ip, flow.src_port, ntohs(flow.dst_ip), ntohs(flow.dst_port));
#endif
idx = hashval % workflow->prefs.num_roots;
@@ -814,7 +814,7 @@ static struct ndpi_flow_info *get_ndpi_flow_info(struct ndpi_workflow * workflow
LOG(NDPI_LOG_ERROR,
"maximum flow count (%u) has been exceeded\n",
workflow->prefs.max_ndpi_flows);
exit(-1);
return NULL;
} else {
struct ndpi_flow_info *newflow = (struct ndpi_flow_info*)ndpi_malloc(sizeof(struct ndpi_flow_info));
@@ -893,6 +893,12 @@ static struct ndpi_flow_info *get_ndpi_flow_info(struct ndpi_workflow * workflow
ndpi_tsearch(newflow, &workflow->ndpi_flows_root[idx], ndpi_workflow_node_cmp); /* Add */
workflow->stats.ndpi_flow_count++;
if(*proto == IPPROTO_TCP)
workflow->stats.flow_count[0]++;
else if(*proto == IPPROTO_UDP)
workflow->stats.flow_count[1]++;
else
workflow->stats.flow_count[2]++;
*src = newflow->src_id, *dst = newflow->dst_id;
newflow->entropy.src2dst_pkt_len[newflow->entropy.src2dst_pkt_count] = l4_data_len;
@@ -1036,7 +1042,7 @@ u_int8_t plen2slot(u_int16_t plen) {
/* ****************************************************** */
void process_ndpi_collected_info(struct ndpi_workflow * workflow, struct ndpi_flow_info *flow, FILE * csv_fp) {
u_int i;
u_int i, is_quic = 0;
if(!flow->ndpi_flow) return;
@@ -1047,7 +1053,7 @@ void process_ndpi_collected_info(struct ndpi_workflow * workflow, struct ndpi_fl
flow->ndpi_flow->flow_extra_info);
flow->risk = flow->ndpi_flow->risk;
if(is_ndpi_proto(flow, NDPI_PROTOCOL_DHCP)) {
snprintf(flow->dhcp_fingerprint, sizeof(flow->dhcp_fingerprint), "%s", flow->ndpi_flow->protos.dhcp.fingerprint);
} else if(is_ndpi_proto(flow, NDPI_PROTOCOL_BITTORRENT)) {
@@ -1122,6 +1128,7 @@ void process_ndpi_collected_info(struct ndpi_workflow * workflow, struct ndpi_fl
snprintf(flow->http.url, sizeof(flow->http.url), "%s", flow->ndpi_flow->http.url);
flow->http.response_status_code = flow->ndpi_flow->http.response_status_code;
snprintf(flow->http.content_type, sizeof(flow->http.content_type), "%s", flow->ndpi_flow->http.content_type ? flow->ndpi_flow->http.content_type : "");
snprintf(flow->http.request_content_type, sizeof(flow->http.request_content_type), "%s", flow->ndpi_flow->http.request_content_type ? flow->ndpi_flow->http.request_content_type : "");
snprintf(flow->http.user_agent, sizeof(flow->http.user_agent), "%s", flow->ndpi_flow->http.user_agent ? flow->ndpi_flow->http.user_agent : "");
}
} else if(is_ndpi_proto(flow, NDPI_PROTOCOL_TELNET)) {
@@ -1140,89 +1147,92 @@ void process_ndpi_collected_info(struct ndpi_workflow * workflow, struct ndpi_fl
}
/* TLS */
else if((is_ndpi_proto(flow, NDPI_PROTOCOL_TLS))
|| ((is_ndpi_proto(flow, NDPI_PROTOCOL_QUIC)))
|| ((is_quic = is_ndpi_proto(flow, NDPI_PROTOCOL_QUIC)))
|| (flow->detected_protocol.master_protocol == NDPI_PROTOCOL_TLS)
|| (flow->ndpi_flow->protos.stun_ssl.ssl.ja3_client[0] != '\0')
|| (flow->ndpi_flow->protos.tls_quic_stun.tls_quic.ja3_client[0] != '\0')
) {
flow->ssh_tls.ssl_version = flow->ndpi_flow->protos.stun_ssl.ssl.ssl_version;
flow->ssh_tls.ssl_version = flow->ndpi_flow->protos.tls_quic_stun.tls_quic.ssl_version;
snprintf(flow->ssh_tls.client_requested_server_name,
sizeof(flow->ssh_tls.client_requested_server_name), "%s",
flow->ndpi_flow->protos.stun_ssl.ssl.client_requested_server_name);
flow->ndpi_flow->protos.tls_quic_stun.tls_quic.client_requested_server_name);
snprintf(flow->http.user_agent, sizeof(flow->http.user_agent), "%s", flow->ndpi_flow->http.user_agent ? flow->ndpi_flow->http.user_agent : "");
if(flow->ndpi_flow->protos.stun_ssl.ssl.server_names_len > 0 && flow->ndpi_flow->protos.stun_ssl.ssl.server_names)
flow->ssh_tls.server_names = ndpi_strdup(flow->ndpi_flow->protos.stun_ssl.ssl.server_names);
flow->ssh_tls.notBefore = flow->ndpi_flow->protos.stun_ssl.ssl.notBefore;
flow->ssh_tls.notAfter = flow->ndpi_flow->protos.stun_ssl.ssl.notAfter;
if(flow->ndpi_flow->protos.tls_quic_stun.tls_quic.server_names_len > 0 && flow->ndpi_flow->protos.tls_quic_stun.tls_quic.server_names)
flow->ssh_tls.server_names = ndpi_strdup(flow->ndpi_flow->protos.tls_quic_stun.tls_quic.server_names);
flow->ssh_tls.notBefore = flow->ndpi_flow->protos.tls_quic_stun.tls_quic.notBefore;
flow->ssh_tls.notAfter = flow->ndpi_flow->protos.tls_quic_stun.tls_quic.notAfter;
snprintf(flow->ssh_tls.ja3_client, sizeof(flow->ssh_tls.ja3_client), "%s",
flow->ndpi_flow->protos.stun_ssl.ssl.ja3_client);
flow->ndpi_flow->protos.tls_quic_stun.tls_quic.ja3_client);
snprintf(flow->ssh_tls.ja3_server, sizeof(flow->ssh_tls.ja3_server), "%s",
flow->ndpi_flow->protos.stun_ssl.ssl.ja3_server);
flow->ssh_tls.server_unsafe_cipher = flow->ndpi_flow->protos.stun_ssl.ssl.server_unsafe_cipher;
flow->ssh_tls.server_cipher = flow->ndpi_flow->protos.stun_ssl.ssl.server_cipher;
flow->ndpi_flow->protos.tls_quic_stun.tls_quic.ja3_server);
flow->ssh_tls.server_unsafe_cipher = flow->ndpi_flow->protos.tls_quic_stun.tls_quic.server_unsafe_cipher;
flow->ssh_tls.server_cipher = flow->ndpi_flow->protos.tls_quic_stun.tls_quic.server_cipher;
if(flow->ndpi_flow->l4.tcp.tls.fingerprint_set) {
memcpy(flow->ssh_tls.sha1_cert_fingerprint,
flow->ndpi_flow->l4.tcp.tls.sha1_certificate_fingerprint, 20);
flow->ndpi_flow->protos.tls_quic_stun.tls_quic.sha1_certificate_fingerprint, 20);
flow->ssh_tls.sha1_cert_fingerprint_set = 1;
}
if(flow->ndpi_flow->protos.stun_ssl.ssl.alpn) {
if((flow->ssh_tls.tls_alpn = ndpi_strdup(flow->ndpi_flow->protos.stun_ssl.ssl.alpn)) != NULL)
correct_csv_data_field(flow->ssh_tls.tls_alpn);
flow->ssh_tls.browser_heuristics = flow->ndpi_flow->protos.tls_quic_stun.tls_quic.browser_heuristics;
if(flow->ndpi_flow->protos.tls_quic_stun.tls_quic.alpn) {
if((flow->ssh_tls.tls_alpn = ndpi_strdup(flow->ndpi_flow->protos.tls_quic_stun.tls_quic.alpn)) != NULL)
correct_csv_data_field(flow->ssh_tls.tls_alpn);
}
if(flow->ndpi_flow->protos.stun_ssl.ssl.issuerDN)
flow->ssh_tls.tls_issuerDN = strdup(flow->ndpi_flow->protos.stun_ssl.ssl.issuerDN);
if(flow->ndpi_flow->protos.tls_quic_stun.tls_quic.issuerDN)
flow->ssh_tls.tls_issuerDN = strdup(flow->ndpi_flow->protos.tls_quic_stun.tls_quic.issuerDN);
if(flow->ndpi_flow->protos.stun_ssl.ssl.subjectDN)
flow->ssh_tls.tls_subjectDN = strdup(flow->ndpi_flow->protos.stun_ssl.ssl.subjectDN);
if(flow->ndpi_flow->protos.tls_quic_stun.tls_quic.subjectDN)
flow->ssh_tls.tls_subjectDN = strdup(flow->ndpi_flow->protos.tls_quic_stun.tls_quic.subjectDN);
if(flow->ndpi_flow->protos.stun_ssl.ssl.encrypted_sni.esni) {
flow->ssh_tls.encrypted_sni.esni = strdup(flow->ndpi_flow->protos.stun_ssl.ssl.encrypted_sni.esni);
flow->ssh_tls.encrypted_sni.cipher_suite = flow->ndpi_flow->protos.stun_ssl.ssl.encrypted_sni.cipher_suite;
if(flow->ndpi_flow->protos.tls_quic_stun.tls_quic.encrypted_sni.esni) {
flow->ssh_tls.encrypted_sni.esni = strdup(flow->ndpi_flow->protos.tls_quic_stun.tls_quic.encrypted_sni.esni);
flow->ssh_tls.encrypted_sni.cipher_suite = flow->ndpi_flow->protos.tls_quic_stun.tls_quic.encrypted_sni.cipher_suite;
}
if(flow->ssh_tls.tls_supported_versions) {
if((flow->ssh_tls.tls_supported_versions = ndpi_strdup(flow->ndpi_flow->protos.stun_ssl.ssl.tls_supported_versions)) != NULL)
if((flow->ssh_tls.tls_supported_versions = ndpi_strdup(flow->ndpi_flow->protos.tls_quic_stun.tls_quic.tls_supported_versions)) != NULL)
correct_csv_data_field(flow->ssh_tls.tls_supported_versions);
}
if(flow->ndpi_flow->protos.stun_ssl.ssl.alpn
&& flow->ndpi_flow->protos.stun_ssl.ssl.tls_supported_versions) {
correct_csv_data_field(flow->ndpi_flow->protos.stun_ssl.ssl.alpn);
correct_csv_data_field(flow->ndpi_flow->protos.stun_ssl.ssl.tls_supported_versions);
if(flow->ndpi_flow->protos.tls_quic_stun.tls_quic.alpn
&& flow->ndpi_flow->protos.tls_quic_stun.tls_quic.tls_supported_versions) {
correct_csv_data_field(flow->ndpi_flow->protos.tls_quic_stun.tls_quic.alpn);
correct_csv_data_field(flow->ndpi_flow->protos.tls_quic_stun.tls_quic.tls_supported_versions);
if(csv_fp)
snprintf(flow->info, sizeof(flow->info), "%s",
flow->ndpi_flow->protos.stun_ssl.ssl.alpn);
flow->ndpi_flow->protos.tls_quic_stun.tls_quic.alpn);
else
snprintf(flow->info, sizeof(flow->info), "ALPN: %s][TLS Supported Versions: %s",
flow->ndpi_flow->protos.stun_ssl.ssl.alpn,
flow->ndpi_flow->protos.stun_ssl.ssl.tls_supported_versions);
}
else if(flow->ndpi_flow->protos.stun_ssl.ssl.alpn) {
correct_csv_data_field(flow->ndpi_flow->protos.stun_ssl.ssl.alpn);
flow->ndpi_flow->protos.tls_quic_stun.tls_quic.alpn,
flow->ndpi_flow->protos.tls_quic_stun.tls_quic.tls_supported_versions);
} else if(flow->ndpi_flow->protos.tls_quic_stun.tls_quic.alpn) {
correct_csv_data_field(flow->ndpi_flow->protos.tls_quic_stun.tls_quic.alpn);
if(csv_fp)
snprintf(flow->info, sizeof(flow->info), "%s,",
flow->ndpi_flow->protos.stun_ssl.ssl.alpn);
flow->ndpi_flow->protos.tls_quic_stun.tls_quic.alpn);
else
snprintf(flow->info, sizeof(flow->info), "ALPN: %s",
flow->ndpi_flow->protos.stun_ssl.ssl.alpn);
flow->ndpi_flow->protos.tls_quic_stun.tls_quic.alpn);
}
#ifdef USE_TLS_LEN
/* For TLS we use TLS block lenght instead of payload lenght */
ndpi_reset_bin(&flow->payload_len_bin);
for(i=0; i<flow->ndpi_flow->l4.tcp.tls.num_tls_blocks; i++) {
u_int16_t len = abs(flow->ndpi_flow->l4.tcp.tls.tls_application_blocks_len[i]);
printf("%u\n", len);
ndpi_inc_bin(&flow->payload_len_bin, plen2slot(len), 1);
if(enable_doh_dot_detection) {
/* For TLS we use TLS block lenght instead of payload lenght */
ndpi_reset_bin(&flow->payload_len_bin);
for(i=0; i<flow->ndpi_flow->l4.tcp.tls.num_tls_blocks; i++) {
u_int16_t len = abs(flow->ndpi_flow->l4.tcp.tls.tls_application_blocks_len[i]);
/* printf("[TLS_LEN] %u\n", len); */
ndpi_inc_bin(&flow->payload_len_bin, plen2slot(len), 1);
}
}
#endif
}
if(flow->detection_completed && (!flow->check_extra_packets)) {
@@ -1287,6 +1297,7 @@ void update_tcp_flags_count(struct ndpi_flow_info* flow, struct ndpi_tcphdr* tcp
}
/* ****************************************************** */
/**
Function to process the packet:
determine the flow of a packet and try to decode it
@@ -1304,8 +1315,9 @@ static struct ndpi_proto packet_processing(struct ndpi_workflow * workflow,
u_int16_t ipsize, u_int16_t rawsize,
const struct pcap_pkthdr *header,
const u_char *packet,
pkt_timeval when,
FILE * csv_fp) {
pkt_timeval when,
ndpi_risk *flow_risk,
FILE * csv_fp) {
struct ndpi_id_struct *src, *dst;
struct ndpi_flow_info *flow = NULL;
struct ndpi_flow_struct *ndpi_flow = NULL;
@@ -1318,6 +1330,9 @@ static struct ndpi_proto packet_processing(struct ndpi_workflow * workflow,
u_int8_t begin_or_end_tcp = 0;
struct ndpi_proto nproto = NDPI_PROTOCOL_NULL;
if(workflow->prefs.ignore_vlanid)
vlan_id = 0;
if(iph)
flow = get_ndpi_flow_info(workflow, IPVERSION, vlan_id,
tunnel_type, iph, NULL,
@@ -1457,8 +1472,8 @@ static struct ndpi_proto packet_processing(struct ndpi_workflow * workflow,
/* Copy packets entropy if num packets count == 10 */
ndpi_clear_entropy_stats(flow);
if(!flow->has_human_readeable_strings) {
if((human_readeable_string_len != 0) && (!flow->has_human_readeable_strings)) {
u_int8_t skip = 0;
if((proto == IPPROTO_TCP)
@@ -1469,10 +1484,10 @@ static struct ndpi_proto packet_processing(struct ndpi_workflow * workflow,
|| (flow->detected_protocol.master_protocol == NDPI_PROTOCOL_SSH))
) {
if((flow->src2dst_packets+flow->dst2src_packets) < 10 /* MIN_NUM_ENCRYPT_SKIP_PACKETS */)
skip = 1;
skip = 1; /* Skip initial negotiation packets */
}
if(!skip) {
if((!skip) && ((flow->src2dst_packets+flow->dst2src_packets) < 100)) {
if(ndpi_has_human_readeable_string(workflow->ndpi_struct, (char*)packet, header->caplen,
human_readeable_string_len,
flow->human_readeable_string_buffer,
@@ -1503,6 +1518,13 @@ static struct ndpi_proto packet_processing(struct ndpi_workflow * workflow,
printf("%s()\n", __FUNCTION__);
#endif
if(proto == IPPROTO_TCP)
workflow->stats.dpi_packet_count[0]++;
else if(proto == IPPROTO_UDP)
workflow->stats.dpi_packet_count[1]++;
else
workflow->stats.dpi_packet_count[2]++;
flow->detected_protocol = ndpi_detection_process_packet(workflow->ndpi_struct, ndpi_flow,
iph ? (uint8_t *)iph : (uint8_t *)iph6,
ipsize, time_ms, src, dst);
@@ -1526,6 +1548,7 @@ static struct ndpi_proto packet_processing(struct ndpi_workflow * workflow,
flow->detected_protocol = ndpi_detection_giveup(workflow->ndpi_struct, flow->ndpi_flow,
enable_protocol_guess, &proto_guessed);
if(enable_protocol_guess) workflow->stats.guessed_flow_protocols++;
}
process_ndpi_collected_info(workflow, flow, csv_fp);
@@ -1533,15 +1556,50 @@ static struct ndpi_proto packet_processing(struct ndpi_workflow * workflow,
}
}
#if 0
if(flow->risk != 0) {
FILE *r = fopen("/tmp/e", "a");
if(r) {
fprintf(r, "->>> %u [%08X]\n", flow->risk, flow->risk);
fclose(r);
}
}
#endif
*flow_risk = flow->risk;
return(flow->detected_protocol);
}
/* ****************************************************** */
int ndpi_is_datalink_supported(int datalink_type) {
/* Keep in sync with the similar switch in ndpi_workflow_process_packet */
switch(datalink_type) {
case DLT_NULL:
case DLT_PPP_SERIAL:
case DLT_C_HDLC:
case DLT_PPP:
case DLT_IPV4:
case DLT_IPV6:
case DLT_EN10MB:
case DLT_LINUX_SLL:
case DLT_IEEE802_11_RADIO:
case DLT_RAW:
return 1;
default:
return 0;
}
}
/* ****************************************************** */
struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
const struct pcap_pkthdr *header,
const u_char *packet,
FILE * csv_fp) {
ndpi_risk *flow_risk,
FILE * csv_fp) {
/*
* Declare pointers to packet headers
*/
@@ -1589,6 +1647,8 @@ struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
/* counters */
u_int8_t vlan_packet = 0;
*flow_risk = 0 /* NDPI_NO_RISK */;
/* Increment raw packet counter */
workflow->stats.raw_packet_count++;
@@ -1612,11 +1672,12 @@ struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
datalink_type = (int)pcap_datalink(workflow->pcap_handle);
#endif
datalink_check:
if(header->caplen < eth_offset + 40)
// 20 for min iph and 8 for min UDP
if(header->caplen < eth_offset + 28)
return(nproto); /* Too short */
/* Keep in sync with ndpi_is_datalink_supported() */
switch(datalink_type) {
case DLT_NULL:
if(ntohl(*((u_int32_t*)&packet[eth_offset])) == 2)
@@ -1637,9 +1698,24 @@ struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
/* Cisco PPP - 9 or 104 */
case DLT_C_HDLC:
case DLT_PPP:
chdlc = (struct ndpi_chdlc *) &packet[eth_offset];
ip_offset = sizeof(struct ndpi_chdlc); /* CHDLC_OFF = 4 */
type = ntohs(chdlc->proto_code);
if(packet[0] == 0x0f || packet[0] == 0x8f) {
chdlc = (struct ndpi_chdlc *) &packet[eth_offset];
ip_offset = sizeof(struct ndpi_chdlc); /* CHDLC_OFF = 4 */
type = ntohs(chdlc->proto_code);
} else {
ip_offset = 2;
type = ntohs(*((u_int16_t*)&packet[eth_offset]));
}
break;
case DLT_IPV4:
type = ETH_P_IP;
ip_offset = 0;
break;
case DLT_IPV6:
type = ETH_P_IPV6;
ip_offset = 0;
break;
/* IEEE 802.3 Ethernet - 1 */
@@ -1687,7 +1763,6 @@ struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
if(header->caplen < (eth_offset + radio_len + sizeof(struct ndpi_wifi_header)))
return(nproto);
/* Calculate 802.11 header length (variable) */
wifi = (struct ndpi_wifi_header*)( packet + eth_offset + radio_len);
fc = wifi->fc;
@@ -1716,7 +1791,11 @@ struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
break;
default:
/* printf("Unknown datalink %d\n", datalink_type); */
/*
* We shoudn't be here, because we already checked that this datalink is supported.
* Should ndpi_is_datalink_supported() be updated?
*/
printf("Unknown datalink %d\n", datalink_type);
return(nproto);
}
@@ -1821,16 +1900,20 @@ struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
} else if(iph->version == 6) {
if(header->caplen < ip_offset + sizeof(struct ndpi_ipv6hdr))
return(nproto); /* Too short for IPv6 header*/
iph6 = (struct ndpi_ipv6hdr *)&packet[ip_offset];
proto = iph6->ip6_hdr.ip6_un1_nxt;
ip_len = ntohs(iph6->ip6_hdr.ip6_un1_plen);
if(header->caplen < (ip_offset + sizeof(struct ndpi_ipv6hdr) + ntohs(iph6->ip6_hdr.ip6_un1_plen)))
return(nproto); /* Too short for IPv6 payload*/
const u_int8_t *l4ptr = (((const u_int8_t *) iph6) + sizeof(struct ndpi_ipv6hdr));
if(ndpi_handle_ipv6_extension_headers(NULL, &l4ptr, &ip_len, &proto) != 0) {
return(nproto);
}
if(proto == IPPROTO_IPV6 || proto == IPPROTO_IPIP) {
if(l4ptr > packet) { /* Better safe than sorry */
ip_offset = (l4ptr - packet);
@@ -1849,6 +1932,7 @@ struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
"\n\nWARNING: only IPv4/IPv6 packets are supported in this demo (nDPI supports both IPv4 and IPv6), all other packets will be discarded\n\n");
ipv4_warning_used = 1;
}
workflow->stats.total_discarded_bytes += header->len;
return(nproto);
}
@@ -1860,23 +1944,40 @@ struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
struct ndpi_udphdr *udp = (struct ndpi_udphdr *)&packet[ip_offset+ip_len];
u_int16_t sport = ntohs(udp->source), dport = ntohs(udp->dest);
if((sport == GTP_U_V1_PORT) || (dport == GTP_U_V1_PORT)) {
if(((sport == GTP_U_V1_PORT) || (dport == GTP_U_V1_PORT)) &&
(ip_offset + ip_len + sizeof(struct ndpi_udphdr) + 8 /* Minimum GTPv1 header len */ < header->caplen)) {
/* Check if it's GTPv1 */
u_int offset = ip_offset+ip_len+sizeof(struct ndpi_udphdr);
u_int8_t flags = packet[offset];
u_int8_t message_type = packet[offset+1];
tunnel_type = ndpi_gtp_tunnel;
u_int8_t exts_parsing_error = 0;
if((((flags & 0xE0) >> 5) == 1 /* GTPv1 */) &&
(message_type == 0xFF /* T-PDU */)) {
ip_offset = ip_offset+ip_len+sizeof(struct ndpi_udphdr)+8; /* GTPv1 header len */
if(flags & 0x04) ip_offset += 1; /* next_ext_header is present */
if(flags & 0x02) ip_offset += 4; /* sequence_number is present (it also includes next_ext_header and pdu_number) */
if(flags & 0x01) ip_offset += 1; /* pdu_number is present */
offset += 8; /* GTPv1 header len */
if(flags & 0x07)
offset += 4; /* sequence_number + pdu_number + next_ext_header fields */
/* Extensions parsing */
if(flags & 0x04) {
unsigned int ext_length = 0;
if(ip_offset < header->caplen) {
while(offset < header->caplen) {
ext_length = packet[offset] << 2;
offset += ext_length;
if(offset >= header->caplen || ext_length == 0) {
exts_parsing_error = 1;
break;
}
if(packet[offset - 1] == 0)
break;
}
}
if(offset < header->caplen && !exts_parsing_error) {
/* Ok, valid GTP-U */
tunnel_type = ndpi_gtp_tunnel;
ip_offset = offset;
iph = (struct ndpi_iphdr *)&packet[ip_offset];
if(iph->version == 6) {
iph6 = (struct ndpi_ipv6hdr *)&packet[ip_offset];
@@ -1942,9 +2043,8 @@ struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
offset += msg_len;
if((offset + 32 < header->caplen) && (packet[offset] == 0x02)) {
if((offset + 32 < header->caplen)) {
/* IEEE 802.11 Data */
offset += 24;
/* LLC header is 8 bytes */
type = ntohs((u_int16_t)*((u_int16_t*)&packet[offset+6]));
@@ -1964,7 +2064,7 @@ struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
return(packet_processing(workflow, time_ms, vlan_id, tunnel_type, iph, iph6,
ip_offset, header->caplen - ip_offset,
header->caplen, header, packet, header->ts,
csv_fp));
flow_risk, csv_fp));
}
/* ********************************************************** */
+44 -31
View File
@@ -1,7 +1,7 @@
/*
* ndpi_util.h
*
* Copyright (C) 2011-19 - ntop.org
* Copyright (C) 2011-21 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
@@ -29,7 +29,7 @@
#ifndef __NDPI_UTIL_H__
#define __NDPI_UTIL_H__
#include "uthash.h"
#include "../src/lib/third_party/include/uthash.h"
#include <pcap.h>
#include "ndpi_includes.h"
#include "ndpi_classify.h"
@@ -54,6 +54,11 @@ extern int dpdk_port_init(int port, struct rte_mempool *mbuf_pool);
extern int dpdk_port_deinit(int port);
#endif
#define PLEN_MAX 1504
#define PLEN_BIN_LEN 32
#define PLEN_NUM_BINS 48 /* 47*32 = 1504 */
#define MAX_NUM_BIN_PKTS 256
/* ETTA Spec defiintions for feature readiness */
#define ETTA_MIN_PACKETS 10
#define ETTA_MIN_OCTETS 4000
@@ -89,38 +94,38 @@ extern int dpdk_port_deinit(int port);
// inner hash table (ja3 -> security state)
typedef struct ndpi_ja3_info {
char * ja3;
ndpi_cipher_weakness unsafe_cipher;
UT_hash_handle hh;
char * ja3;
ndpi_cipher_weakness unsafe_cipher;
UT_hash_handle hh;
} ndpi_ja3_info;
// external hash table (host ip -> <ip string, hash table ja3c, hash table ja3s>)
// used to aggregate ja3 fingerprints by hosts
typedef struct ndpi_host_ja3_fingerprints {
u_int32_t ip;
char *ip_string;
char *dns_name;
ndpi_ja3_info *host_client_info_hasht;
ndpi_ja3_info *host_server_info_hasht;
u_int32_t ip;
char *ip_string;
char *dns_name;
ndpi_ja3_info *host_client_info_hasht;
ndpi_ja3_info *host_server_info_hasht;
UT_hash_handle hh;
UT_hash_handle hh;
} ndpi_host_ja3_fingerprints;
//inner hash table
typedef struct ndpi_ip_dns{
u_int32_t ip;
char *ip_string;
char *dns_name; //server name if any;
UT_hash_handle hh;
u_int32_t ip;
char *ip_string;
char *dns_name; //server name if any;
UT_hash_handle hh;
} ndpi_ip_dns;
//hash table ja3 -> <host, ip, security>, used to aggregate host by ja3 fingerprints
typedef struct ndpi_ja3_fingerprints_host{
char *ja3; //key
ndpi_cipher_weakness unsafe_cipher;
ndpi_ip_dns *ipToDNS_ht;
UT_hash_handle hh;
char *ja3; //key
ndpi_cipher_weakness unsafe_cipher;
ndpi_ip_dns *ipToDNS_ht;
UT_hash_handle hh;
} ndpi_ja3_fingerprints_host;
struct flow_metrics {
@@ -190,7 +195,7 @@ typedef struct ndpi_flow_info {
struct ndpi_analyze_struct *iat_c_to_s, *iat_s_to_c, *iat_flow,
*pktlen_c_to_s, *pktlen_s_to_c;
char info[160];
char info[255];
char flow_extra_info[16];
char host_server_name[240];
char bittorent_hash[41];
@@ -199,24 +204,27 @@ typedef struct ndpi_flow_info {
struct {
u_int16_t ssl_version;
char client_requested_server_name[64], server_info[64],
char client_requested_server_name[256], server_info[64],
client_hassh[33], server_hassh[33], *server_names,
*tls_alpn, *tls_supported_versions,
*tls_issuerDN, *tls_subjectDN,
ja3_client[33], ja3_server[33],
sha1_cert_fingerprint[20];
u_int8_t sha1_cert_fingerprint_set;
struct tls_heuristics browser_heuristics;
struct {
u_int16_t cipher_suite;
char *esni;
} encrypted_sni;
time_t notBefore, notAfter;
u_int16_t server_cipher;
ndpi_cipher_weakness client_unsafe_cipher, server_unsafe_cipher;
} ssh_tls;
struct {
char url[256], content_type[64], user_agent[128];
char url[256], request_content_type[64], content_type[64], user_agent[128];
u_int response_status_code;
} http;
@@ -248,10 +256,12 @@ typedef struct ndpi_stats {
u_int64_t protocol_counter_bytes[NDPI_MAX_SUPPORTED_PROTOCOLS + NDPI_MAX_NUM_CUSTOM_PROTOCOLS + 1];
u_int32_t protocol_flows[NDPI_MAX_SUPPORTED_PROTOCOLS + NDPI_MAX_NUM_CUSTOM_PROTOCOLS + 1];
u_int32_t ndpi_flow_count;
u_int32_t flow_count[3];
u_int64_t tcp_count, udp_count;
u_int64_t mpls_count, pppoe_count, vlan_count, fragmented_count;
u_int64_t packet_len[6];
u_int16_t max_packet_len;
u_int64_t dpi_packet_count[3];
} ndpi_stats_t;
@@ -259,6 +269,7 @@ typedef struct ndpi_stats {
typedef struct ndpi_workflow_prefs {
u_int8_t decode_tunnels;
u_int8_t quiet_mode;
u_int8_t ignore_vlanid;
u_int32_t num_roots;
u_int32_t max_ndpi_flows;
} ndpi_workflow_prefs_t;
@@ -288,14 +299,14 @@ typedef struct ndpi_workflow {
void **ndpi_flows_root;
struct ndpi_detection_module_struct *ndpi_struct;
u_int32_t num_allocated_flows;
} ndpi_workflow_t;
} ndpi_workflow_t;
/* TODO: remove wrappers parameters and use ndpi global, when their initialization will be fixed... */
struct ndpi_workflow * ndpi_workflow_init(const struct ndpi_workflow_prefs * prefs, pcap_t * pcap_handle);
/* workflow main free function */
/* workflow main free function */
void ndpi_workflow_free(struct ndpi_workflow * workflow);
@@ -310,8 +321,10 @@ void ndpi_free_flow_info_half(struct ndpi_flow_info *flow);
struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
const struct pcap_pkthdr *header,
const u_char *packet,
FILE * csv_fp);
ndpi_risk *flow_risk,
FILE * csv_fp);
int ndpi_is_datalink_supported(int datalink_type);
/* flow callbacks for complete detected flow
(ndpi_flow_info will be freed right after) */
@@ -327,7 +340,7 @@ static inline void ndpi_workflow_set_flow_giveup_callback(struct ndpi_workflow *
workflow->__flow_giveup_udata = udata;
}
/* compare two nodes in workflow */
/* compare two nodes in workflow */
int ndpi_workflow_node_cmp(const void *a, const void *b);
void process_ndpi_collected_info(struct ndpi_workflow * workflow, struct ndpi_flow_info *flow, FILE * csv_fp);
u_int32_t ethernet_crc32(const void* data, size_t n_bytes);
@@ -339,13 +352,13 @@ float ndpi_flow_get_byte_count_entropy(const uint32_t byte_count[256], unsigned
extern int nDPI_LogLevel;
#ifdef NDPI_ENABLE_DEBUG_MESSAGES
#define LOG(log_level, args...) \
{ \
if(log_level <= nDPI_LogLevel) \
printf(args); \
#define LOG(log_level, args...) \
{ \
if(log_level <= nDPI_LogLevel) \
printf(args); \
}
#else
#define LOG(...) {}
#define LOG(...) {}
#endif
#endif
+11
View File
@@ -0,0 +1,11 @@
# https://www.spamhaus.org/statistics/tlds/
.fail
.viajes
.exposed
.london
.work
.gq
.ml
.surk
.tk
.xyz
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -15,8 +15,8 @@ fuzz_process_packet_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
fuzz_ndpi_reader_SOURCES = fuzz_ndpi_reader.c
fuzz_ndpi_reader_CFLAGS = -I../example/
fuzz_ndpi_reader_LDADD = ../src/lib/libndpi.a
fuzz_ndpi_reader_LDFLAGS = ../example/libndpiReader.a $(PCAP_LIB) $(ADDITIONAL_LIBS) $(LIBS)
fuzz_ndpi_reader_LDADD = ../example/libndpiReader.a ../src/lib/libndpi.a
fuzz_ndpi_reader_LDFLAGS = $(PCAP_LIB) $(ADDITIONAL_LIBS) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_LDFLAGS += $(LIB_FUZZING_ENGINE)
+14 -2
View File
@@ -66,6 +66,14 @@ int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
free(pcap_path);
return 0;
}
if (ndpi_is_datalink_supported(pcap_datalink(pkts)) == 0)
{
/* Do not fail if the datalink type is not supported (may happen often during fuzzing). */
pcap_close(pkts);
remove(pcap_path);
free(pcap_path);
return 0;
}
struct ndpi_workflow * workflow = ndpi_workflow_init(prefs, pkts);
// enable all protocols
NDPI_BITMASK_SET_ALL(all);
@@ -76,8 +84,9 @@ int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
sizeof(workflow->stats.protocol_counter_bytes));
memset(workflow->stats.protocol_flows, 0,
sizeof(workflow->stats.protocol_flows));
ndpi_finalize_initalization(workflow->ndpi_struct);
ndpi_finalize_initialization(workflow->ndpi_struct);
header = NULL;
r = pcap_next_ex(pkts, &header, &pkt);
while (r > 0) {
if(header->caplen >= 42 /* ARP+ size */) {
@@ -85,8 +94,10 @@ int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
uint8_t *packet_checked = malloc(header->caplen);
if(packet_checked) {
ndpi_risk flow_risk;
memcpy(packet_checked, pkt, header->caplen);
ndpi_workflow_process_packet(workflow, header, packet_checked, NULL);
ndpi_workflow_process_packet(workflow, header, packet_checked, &flow_risk, NULL);
free(packet_checked);
}
}
@@ -151,6 +162,7 @@ int main(int argc, char ** argv)
if (fread(pcap_buffer, sizeof(*pcap_buffer), pcap_file_size, pcap_file) != pcap_file_size) {
perror("fread failed");
fclose(pcap_file);
free(pcap_buffer);
return 1;
}
+1 -1
View File
@@ -1 +1 @@
/usr/local/lib
/usr/lib
+65 -29
View File
@@ -1,57 +1,93 @@
#
# Copyright (C) 2018-20 - ntop.org
# Copyright (C) 2017 OpenWrt.org
#
# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#
include $(TOPDIR)/rules.mk
PKG_NAME:=libndpi
PKG_VERSION:=17022020
PKG_VERSION:=3.4
PKG_RELEASE:=1
PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz
PKG_SOURCE_URL:=https://github.com/ntop/nDPI.git
PKG_SOURCE_VERSION:=1f921562d1d7962f1d23ca5b59c25f9b65073460
PKG_SOURCE_SUBDIR:=$(PKG_NAME)-$(PKG_VERSION)
PKG_SOURCE_PROTO:=git
PKG_SOURCE_URL:=https://codeload.github.com/ntop/nDPI/tar.gz/$(PKG_VERSION)?
PKG_HASH:=dc9b291c7fde94edb45fb0f222e0d93c93f8d6d37f4efba20ebd9c655bfcedf9
PKG_BUILD_DIR:=$(BUILD_DIR)/nDPI-$(PKG_VERSION)
PKG_MAINTAINER:=Luca Deri <deri@ntop.org>
PKG_LICENSE:=GPL3
PKG_BUILD_DEPENDS:=
PKG_MAINTAINER:=Banglang Huang <banglang.huang@foxmail.com>, Toni Uhlig <matzeton@googlemail.com>
PKG_LICENSE:=LGPL-3.0-or-later
PKG_LICENSE_FILES:=COPYING
PKG_FIXUP:=autoreconf
PKG_REMOVE_FILES:=autogen.sh
PKG_BUILD_DEPENDS:=libpcap
PKG_BUILD_PARALLEL:=1
# autogen fix
PKG_FIXUP:=autoreconf
include $(INCLUDE_DIR)/uclibc++.mk
include $(INCLUDE_DIR)/package.mk
ifeq ($(CONFIG_LIBNDPI_GCRYPT),)
CONFIGURE_ARGS += --disable-gcrypt
endif
define Package/libndpi
SECTION:=network
CATEGORY:=Network
TITLE:=nDPI Deep Packet Inspection Library
URL:=https://www.ntop.org
SECTION:=libs
CATEGORY:=Libraries
TITLE:=Library for deep-packet inspection
URL:=https://github.com/ntop/nDPI
DEPENDS:=$(CXX_DEPENDS) +LIBNDPI_GCRYPT:libgcrypt +libpcap +libjson-c
endef
define Package/libndpi/description
Open and Extensible GPLv3 Deep Packet Inspection Library
nDPI is an open source LGPLv3 library for deep-packet inspection.
Based on OpenDPI it includes ntop extensions.
endef
CONFIGURE_ARGS += \
--with-only-libndpi
define Package/libndpi/config
config LIBNDPI_GCRYPT
bool "GCrypt support"
depends on PACKAGE_libndpi
default n
help
This option enables QUIC client hello decryption.
Disabled by default.
endef
define Build/Prepare
$(call Build/Prepare/Default)
endef
define Build/Configure
( cd $(PKG_BUILD_DIR); ./autogen.sh )
$(call Build/Configure/Default)
$(PKG_UNPACK)
$(Build/Patch)
mv $(PKG_BUILD_DIR)/configure.seed $(PKG_BUILD_DIR)/configure.ac
$(SED) "s/@NDPI_MAJOR@/3/g" \
-e "s/@NDPI_MINOR@/4/g" \
-e "s/@NDPI_PATCH@/0/g" \
-e "s/@NDPI_VERSION_SHORT@/3.4.0/g" \
-e "s/@FUZZY@/dnl> /g" \
$(PKG_BUILD_DIR)/configure.ac
endef
define Build/InstallDev
$(INSTALL_DIR) $(STAGING_DIR)/usr/local/include/libndpi
$(CP) $(PKG_BUILD_DIR)/src/include/* $(STAGING_DIR)/usr/local/include/libndpi
$(INSTALL_DIR) $(STAGING_DIR)/usr/local/lib
$(CP) $(PKG_BUILD_DIR)/src/lib/libndpi.* $(STAGING_DIR)/usr/local/lib
$(INSTALL_DIR) $(1)/usr/include/ndpi
$(CP) $(PKG_BUILD_DIR)/src/include/*.h \
$(1)/usr/include/ndpi/
$(INSTALL_DIR) $(1)/usr/lib
$(CP) $(PKG_BUILD_DIR)/src/lib/libndpi.so* \
$(1)/usr/lib/
$(INSTALL_DIR) $(1)/usr/lib/pkgconfig
$(CP) $(PKG_BUILD_DIR)/libndpi.pc \
$(1)/usr/lib/pkgconfig/
endef
define Package/libndpi/install
$(INSTALL_DIR) $(1)/usr/lib/
$(CP) $(PKG_BUILD_DIR)/src/lib/libndpi.so* \
$(1)/usr/lib/
$(INSTALL_DIR) $(1)/usr/bin/
$(CP) $(PKG_BUILD_DIR)/example/ndpiReader \
$(1)/usr/bin/
endef
$(eval $(call BuildPackage,libndpi))
+93
View File
@@ -0,0 +1,93 @@
#
# Copyright (C) 2017 OpenWrt.org
#
# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#
include $(TOPDIR)/rules.mk
PKG_NAME:=libndpi
# Avoid a specfic version
PKG_VERSION:=dev
PKG_RELEASE:=1
# Compile it from soucre
SOURCE_DIR:=/home/deri/openwrt-dependencies/nDPI/
PKG_MAINTAINER:=Luca Deri <deri@ntop.org>
PKG_LICENSE:=LGPL-3.0-or-later
PKG_LICENSE_FILES:=COPYING
PKG_FIXUP:=autoreconf
PKG_REMOVE_FILES:=autogen.sh
PKG_BUILD_DEPENDS:=libpcap
PKG_BUILD_PARALLEL:=1
include $(INCLUDE_DIR)/uclibc++.mk
include $(INCLUDE_DIR)/package.mk
ifeq ($(CONFIG_LIBNDPI_GCRYPT),)
CONFIGURE_ARGS += --disable-gcrypt
endif
define Package/libndpi
SECTION:=libs
CATEGORY:=Libraries
TITLE:=Library for deep-packet inspection
URL:=https://github.com/ntop/nDPI
DEPENDS:=$(CXX_DEPENDS) +LIBNDPI_GCRYPT:libgcrypt +libpcap +libjson-c
endef
define Package/libndpi/description
nDPI is an open source LGPLv3 library for deep-packet inspection.
Based on OpenDPI it includes ntop extensions.
endef
define Package/libndpi/config
config LIBNDPI_GCRYPT
bool "GCrypt support"
depends on PACKAGE_libndpi
default n
help
This option enables QUIC client hello decryption.
Disabled by default.
endef
define Build/Prepare
cp -r $(SOURCE_DIR)/* $(PKG_BUILD_DIR)
$(PKG_UNPACK)
$(Build/Patch)
mv $(PKG_BUILD_DIR)/configure.seed $(PKG_BUILD_DIR)/configure.ac
$(SED) "s/@NDPI_MAJOR@/3/g" \
-e "s/@NDPI_MINOR@/4/g" \
-e "s/@NDPI_PATCH@/0/g" \
-e "s/@NDPI_VERSION_SHORT@/3.4.0/g" \
-e "s/@FUZZY@/dnl> /g" \
$(PKG_BUILD_DIR)/configure.ac
endef
define Build/InstallDev
$(INSTALL_DIR) $(1)/usr/include/ndpi
$(CP) $(PKG_BUILD_DIR)/src/include/*.h \
$(1)/usr/include/ndpi/
$(INSTALL_DIR) $(1)/usr/lib
$(CP) $(PKG_BUILD_DIR)/src/lib/libndpi.so* \
$(1)/usr/lib/
$(INSTALL_DIR) $(1)/usr/lib/pkgconfig
$(CP) $(PKG_BUILD_DIR)/libndpi.pc \
$(1)/usr/lib/pkgconfig/
endef
define Package/libndpi/install
$(INSTALL_DIR) $(1)/usr/lib/
$(CP) $(PKG_BUILD_DIR)/src/lib/libndpi.so* \
$(1)/usr/lib/
$(INSTALL_DIR) $(1)/usr/bin/
$(CP) $(PKG_BUILD_DIR)/example/ndpiReader \
$(1)/usr/bin/
endef
$(eval $(call BuildPackage,libndpi))
+11 -4
View File
@@ -1,14 +1,20 @@
Howto Compile lindpi on OpenWRT
Howto Compile libndpi on OpenWRT
-------------------------------
cd myopenwrt_directory
mkdir package/network/services/libndpi
cd package/network/services/libndpi
cp ~/nDPI/packages/openwrt/Makefile .
# (a) Stable version
ln -s ~/nDPI/packages/openwrt/Makefile .
# (b) Dev version
cp ~/nDPI/packages/openwrt/Makefile.dev Makefile
cd myopenwrt_directory
make menuconfig
Go under network and select
Go under libraries and select
<M> libndpi.............................. nDPI Deep Packet Inspection Library
@@ -22,4 +28,5 @@ make -j1 V=s package/network/services/libndpi/compile
Other Documents
---------------
https://openwrt.org/packages/pkgdata/libndpi
https://openwrt.org/packages/pkgdata/libndpi
+1600 -2
View File
File diff suppressed because it is too large Load Diff
+8
View File
@@ -39,10 +39,18 @@ if test "$CENTOS_RELEASE" -ne 8; then
RPM_SIGN_CMD="./rpm-sign.exp"
fi
AC_CHECK_LIB([maxminddb], [MMDB_lookup_sockaddr])
AC_HAVE_HEADERS(maxminddb.h)
if test ".${ac_cv_lib_maxminddb_MMDB_lookup_sockaddr}" = ".yes" &&
test ".${ac_cv_header_maxminddb_h}" = ".yes"; then
GEOIP_DEP="libmaxminddb"
fi
AC_CONFIG_FILES([Makefile ndpi.spec])
AC_SUBST(PACKAGE_VERSION)
AC_SUBST(PLATFORM)
AC_SUBST(GEOIP_DEP)
AC_SUBST(MACHINE)
AC_SUBST(EXTN)
AC_SUBST(DATE)
+23 -17
View File
@@ -8,12 +8,16 @@ URL: http://www.ntop.org/products/deep-packet-inspection/ndpi/
Source: ndpi-%{version}.tgz
Packager: Luca Deri <deri@ntop.org>
BuildRoot: %{_tmppath}/%{name}-%{version}-root
%{?el7:Requires: glibc >= 2.3.4 numactl coreutils }
%{?el6:Requires: glibc >= 2.3.4 numactl coreutils }
%{?el8:Requires: glibc >= 2.3.4 numactl coreutils @GEOIP_DEP@ }
%{?el7:Requires: glibc >= 2.3.4 numactl coreutils @GEOIP_DEP@ }
%{?el6:Requires: glibc >= 2.3.4 numactl coreutils @GEOIP_DEP@ }
AutoReqProv: no
%define debug_package %{nil}
# Make sure .build-id is not part of the package
%define _build_id_links none
%description
nDPI Open and Extensible LGPLv3 Deep Packet Inspection Library.
@@ -43,9 +47,11 @@ you will need to install %{name}-dev.
%build
PATH=/usr/bin:/bin:/usr/sbin:/sbin
%define includedir /usr/local/include/ndpi
%define libdir /usr/local/lib
%define bindir /usr/local/bin
%define includedir /usr/include/ndpi
%{?el6:%define libdir /usr/lib}
%{?el7:%define libdir /usr/lib}
%{?el8:%define libdir /usr/lib64}
%define bindir /usr/bin
%install
PATH=/usr/bin:/bin:/usr/sbin:/sbin
@@ -56,13 +62,13 @@ mkdir -p $RPM_BUILD_ROOT%{includedir}
mkdir -p $RPM_BUILD_ROOT%{libdir}
mkdir -p $RPM_BUILD_ROOT%{bindir}
cd $HOME/nDPI; make DESTDIR=$RPM_BUILD_ROOT install;
rm $RPM_BUILD_ROOT%{libdir}/libndpi.so
rm $RPM_BUILD_ROOT%{libdir}/libndpi.so.?
cd $RPM_BUILD_ROOT%{libdir}/; ln -s libndpi.so.@PACKAGE_VERSION@ libndpi.so.@MAJOR_RELEASE@; cd -
cd $RPM_BUILD_ROOT%{libdir}/; ln -s libndpi.so.@PACKAGE_VERSION@ libndpi.so; cd -
rm $RPM_BUILD_ROOT/usr/lib/libndpi.so
rm $RPM_BUILD_ROOT/usr/lib/libndpi.so.?
cd $RPM_BUILD_ROOT/usr/lib/; ln -s libndpi.so.@PACKAGE_VERSION@ libndpi.so.@MAJOR_RELEASE@; cd -
cd $RPM_BUILD_ROOT/usr/lib/; ln -s libndpi.so.@PACKAGE_VERSION@ libndpi.so; cd -
mkdir -p $RPM_BUILD_ROOT/etc/ld.so.conf.d
cp $HOME/nDPI/packages/etc/ld.so.conf.d/ndpi.conf $RPM_BUILD_ROOT/etc/ld.so.conf.d
rm -rf $RPM_BUILD_ROOT/usr/local/sbin/ndpi
rm -rf $RPM_BUILD_ROOT/usr/sbin/ndpi
strip $RPM_BUILD_ROOT%{bindir}/*
rm -fr %{buildroot}%{includedir}/ndpi_win32.h
#sed -i -e "s/#include \"ndpi_win32.h\"//g" %{buildroot}%{includedir}/ndpi_main.h
@@ -73,12 +79,12 @@ rm -fr $RPM_BUILD_ROOT
%files
%defattr(-, root, root)
/usr/local/lib/libndpi.so
/usr/local/lib/libndpi.so.@MAJOR_RELEASE@
/usr/local/lib/libndpi.so.@PACKAGE_VERSION@
/usr/local/bin/ndpiReader
/usr/local/share/ndpi/ndpiCustomCategory.txt
/usr/local/share/ndpi/ndpiProtos.txt
/usr/lib/libndpi.so
/usr/lib/libndpi.so.@MAJOR_RELEASE@
/usr/lib/libndpi.so.@PACKAGE_VERSION@
/usr/bin/ndpiReader
/usr/share/ndpi/ndpiCustomCategory.txt
/usr/share/ndpi/ndpiProtos.txt
/etc/ld.so.conf.d/ndpi.conf
%{libdir}/pkgconfig/libndpi.pc
@@ -88,7 +94,7 @@ rm -fr $RPM_BUILD_ROOT
%defattr(-,root,root,-)
%{includedir}
%{libdir}/libndpi.a
/usr/lib/libndpi.a
%changelog
* Mon Nov 19 2018 Alfredo Cardigliano <cardigliano@ntop.org> 2.5
+14 -18
View File
@@ -1,24 +1,19 @@
#
# Change it according to your setup
#
NDPI_HOME=$(PWD)/../..
NDPI_BUILD=${NDPI_HOME}/packages/ubuntu
all: clean ndpi
NDPI_BUILD="$(realpath ${NDPI_HOME}/packages/ubuntu)"
ndpi:
\rm -rf ./debian/ndpi-tmp ./debian/ndpi-dev-tmp ./debian/ndpi ./debian/ndpi-dev
mkdir -p ./debian/ndpi-tmp ./debian/ndpi-dev-tmp
mkdir -p ./debian/ndpi-tmp/usr/local/lib ./debian/ndpi-tmp/usr/local/bin
mkdir -p ./debian/ndpi-dev-tmp/usr/local/lib ./debian/ndpi-dev-tmp/usr/local/include/ndpi
mkdir -p ./debian/ndpi-tmp/usr/lib ./debian/ndpi-tmp/usr/bin
mkdir -p ./debian/ndpi-dev-tmp/usr/lib ./debian/ndpi-dev-tmp/usr/include/ndpi
cd ${NDPI_HOME}; ./autogen.sh; ./configure; make
cp $(NDPI_HOME)/src/lib/libndpi.so.@NDPI_VERS@ ./debian/ndpi-tmp/usr/local/lib/
cd ./debian/ndpi-tmp/usr/local/lib/; ln -s libndpi.so.@NDPI_VERS@ libndpi.so; cd -
cd ./debian/ndpi-tmp/usr/local/lib/; ln -s libndpi.so.@NDPI_VERS@ libndpi.so.@MAJOR_RELEASE@; cd -
cp $(NDPI_HOME)/src/lib/libndpi.a ./debian/ndpi-dev-tmp/usr/local/lib/
cp $(NDPI_HOME)/example/ndpiReader ./debian/ndpi-tmp/usr/local/bin/
cp $(NDPI_HOME)/src/include/*.h ./debian/ndpi-dev-tmp/usr/local/include/ndpi/
-rm -fr ./debian/ndpi-dev-tmp/usr/local/include/ndpi/ndpi_win32.h*
cp $(NDPI_HOME)/src/lib/libndpi.so.@NDPI_VERS@ ./debian/ndpi-tmp/usr/lib/
cd ./debian/ndpi-tmp/usr/lib/; ln -s libndpi.so.@NDPI_VERS@ libndpi.so; cd -
cd ./debian/ndpi-tmp/usr/lib/; ln -s libndpi.so.@NDPI_VERS@ libndpi.so.@MAJOR_RELEASE@; cd -
cp $(NDPI_HOME)/src/lib/libndpi.a ./debian/ndpi-dev-tmp/usr/lib/
cp $(NDPI_HOME)/example/ndpiReader ./debian/ndpi-tmp/usr/bin/
cp $(NDPI_HOME)/src/include/*.h ./debian/ndpi-dev-tmp/usr/include/ndpi/
-rm -fr ./debian/ndpi-dev-tmp/usr/include/ndpi/ndpi_win32.h*
@echo
@find ./debian/ndpi-tmp -name "*~" -exec /bin/rm {} ';'
@find ./debian/ndpi-dev-tmp -name "*~" -exec /bin/rm {} ';'
@@ -37,10 +32,11 @@ ndpi:
@echo "-------------------------------"
distclean:
echo "dummy distclean"
-cd ${NDPI_HOME} && make distclean
install:
echo "dummy install"
cd ${NDPI_HOME}; make install DESTDIR="${NDPI_BUILD}/debian/tmp" AM_UPDATE_INFO_DIR=no
clean:
-rm -rf *~ *deb ./usr ./debian/ndpi ./debian/ndpi-dev ./debian/ndpi-tmp ./debian/ndpi-dev-tmp
-cd ${NDPI_HOME} && make clean
-rm -rf *~ *deb ./usr ./debian/ndpi ./debian/ndpi-dev ./debian/ndpi ./debian/ndpi-dev
+1604 -18
View File
File diff suppressed because it is too large Load Diff
+11 -3
View File
@@ -24,13 +24,21 @@ else
fi
fi
NDPI_VERS=`../version.sh --release`
MAJOR_RELEASE=`../version.sh --major-release`
GIT_REVISION=`../version.sh --revision`
NDPI_VERS=`$(dirname "${0}")/../version.sh --release`
MAJOR_RELEASE=`$(dirname "${0}")/../version.sh --major-release`
GIT_REVISION=`$(dirname "${0}")/../version.sh --revision`
AC_CHECK_LIB([maxminddb], [MMDB_lookup_sockaddr])
AC_HAVE_HEADERS(maxminddb.h)
if test ".${ac_cv_lib_maxminddb_MMDB_lookup_sockaddr}" = ".yes" &&
test ".${ac_cv_header_maxminddb_h}" = ".yes"; then
GEOIP_DEP=", libmaxminddb0"
fi
AC_CONFIG_FILES([Makefile debian/changelog debian/files debian/control])
AC_SUBST(NDPI_VERS)
AC_SUBST(GEOIP_DEP)
AC_SUBST(MACHINE)
AC_SUBST(EXTN)
AC_SUBST(DATE)
+1 -1
View File
@@ -8,7 +8,7 @@ Build-Conflicts:
Package: ndpi
Architecture: @EXTN@
Depends: libnuma1
Depends: libnuma1 @GEOIP_DEP@
Recommends:
Suggests:
Pre-Depends:
+1 -1
View File
@@ -2,7 +2,7 @@ CC=@CC@
CFLAGS=-I. -I../src/include -I./src/lib/third_party/include @CFLAGS@ @CUSTOM_NDPI@ -shared
#LIBNDPI=../src/lib/libndpi.so.@NDPI_VERSION_SHORT@
LIBNDPI=../src/lib/libndpi.a
LDFLAGS=$(CFILE) $(LIBNDPI) -lpcap @ADDITIONAL_LIBS@
LDFLAGS=$(CFILE) $(LIBNDPI) -lpcap @LIBS@ @ADDITIONAL_LIBS@
SHARE = -soname,ndpi_wrap
SO=ndpi_wrap.so
OBJS = ndpi_wrap.o
+12 -6
View File
@@ -316,6 +316,13 @@ typedef enum {
NDPI_UNSAFE_PROTOCOL,
NDPI_DNS_SUSPICIOUS_TRAFFIC,
NDPI_TLS_MISSING_SNI,
NDPI_HTTP_SUSPICIOUS_CONTENT,
NDPI_RISKY_ASN,
NDPI_RISKY_DOMAIN,
NDPI_MALICIOUS_JA3,
NDPI_MALICIOUS_SHA1_CERTIFICATE,
NDPI_DESKTOP_OR_FILE_SHARING_SESSION,
/* Leave this as last member */
NDPI_MAX_RISK
} ndpi_risk_enum;
@@ -666,7 +673,7 @@ struct ndpi_flow_udp_struct {
uint32_t halflife2_stage:2; // 0 - 2
/* NDPI_PROTOCOL_TFTP */
uint32_t tftp_stage:1;
uint32_t tftp_stage:2;
/* NDPI_PROTOCOL_AIMINI */
uint32_t aimini_stage:5;
@@ -715,7 +722,6 @@ struct ndpi_packet_struct {
uint64_t current_time_ms;
uint16_t detected_protocol_stack[2];
uint8_t detected_subprotocol_stack[2];
uint16_t protocol_stack_info;
struct ndpi_int_one_line_struct line[64];
@@ -869,9 +875,9 @@ typedef enum {
typedef struct ndpi_proto_defaults {
char *protoName;
ndpi_protocol_category_t protoCategory;
uint8_t can_have_a_subprotocol;
u_int16_t * subprotocols;
size_t subprotocol_count;
uint16_t protoId, protoIdx;
uint16_t master_tcp_protoId[2], master_udp_protoId[2]; /* The main protocols on which this sub-protocol sits on */
uint16_t tcp_default_ports[5], udp_default_ports[5];
ndpi_protocol_breed_t protoBreed;
void (*func) (struct ndpi_detection_module_struct *, struct ndpi_flow_struct *flow);
@@ -1067,6 +1073,8 @@ struct ndpi_flow_struct {
uint8_t num_request_headers, num_response_headers;
uint8_t request_version; /* 0=1.0 and 1=1.1. Create an enum for this? */
uint16_t response_status_code; /* 200, 404, etc. */
uint8_t detected_os[32]; /* Via HTTP/QUIC User-Agent */
} http;
/*
@@ -1144,8 +1152,6 @@ struct ndpi_flow_struct {
} ubntac2;
struct {
/* Via HTTP User-Agent */
uint8_t detected_os[32];
/* Via HTTP X-Forwarded-For */
uint8_t nat_ip[24];
} http;
+5 -7
View File
@@ -94,11 +94,10 @@ class NDPIProtoDefaultsT(Structure):
_fields_ = [
("protoName", c_char_p),
("protoCategory", c_uint),
("can_have_a_subprotocol", c_uint8),
("subprotocols", c_uint16_p),
("subprotocol_count", c_uint32),
("protoId", c_uint16),
("protoIdx", c_uint16),
("master_tcp_protoId", c_uint16 * 2),
("master_udp_protoId", c_uint16 * 2),
("protoBreed", c_uint),
("func", CFUNCTYPE(None, POINTER(NDPIDetectionModuleStruct), POINTER(NDPIFlowStruct))),
]
@@ -427,7 +426,7 @@ class NDPIFlowUdpStruct(Structure):
('snmp_stage', c_uint32, 2),
('ppstream_stage', c_uint32, 3),
('halflife2_stage', c_uint32, 2),
('tftp_stage', c_uint32, 1),
('tftp_stage', c_uint32, 2),
('aimini_stage', c_uint32, 5),
('xbox_stage', c_uint32, 1),
('wsus_stage', c_uint32, 1),
@@ -456,6 +455,7 @@ class Http(Structure):
("num_response_headers", c_uint8),
("request_version", c_uint8),
("response_status_code", c_uint16),
("detected_os", c_char * 32),
]
@@ -535,7 +535,6 @@ class Ubntac2(Structure):
class Http2(Structure):
_fields_ = [
("detected_os", c_char * 32),
("nat_ip", c_char * 24)
]
@@ -653,7 +652,6 @@ class NDPIUdpHdr(Structure):
class NDPIPacketStructStack(Structure):
_pack_ = 1
_fields_ = [
('detected_subprotocol_stack', c_uint8 * ndpi.ndpi_wrap_ndpi_procol_size()),
('protocol_stack_info', c_uint16)
]
@@ -861,4 +859,4 @@ ndpi.ndpi_set_protocol_detection_bitmask2.argtypes = [POINTER(NDPIDetectionModul
ndpi.ndpi_twalk.argtypes = [c_void_p, CFUNCTYPE(None, c_void_p, c_int32, c_int, c_void_p), c_void_p]
""" ndpi_tdestroy: node destroy. """
ndpi.ndpi_tdestroy.argtypes = [c_void_p, CFUNCTYPE(None, c_void_p)]
ndpi.ndpi_tdestroy.argtypes = [c_void_p, CFUNCTYPE(None, c_void_p)]
+12
View File
@@ -0,0 +1,12 @@
sonar.projectKey=ntop_nDPI
sonar.organization=ntop
# This is the name and version displayed in the SonarCloud UI.
#sonar.projectName=nDPI
#sonar.projectVersion=1.0
# Path is relative to the sonar-project.properties file. Replace "\" by "/" on Windows.
#sonar.sources=.
# Encoding of the source code. Default is default system encoding
#sonar.sourceEncoding=UTF-8
+273 -90
View File
@@ -47,6 +47,11 @@ extern "C" {
#define SAVE_DETECTION_BITMASK_AS_UNKNOWN 1
#define NO_SAVE_DETECTION_BITMASK_AS_UNKNOWN 0
/*
In case a custom DGA function is used, the fucntion
below must be overwritten,
*/
extern ndpi_custom_dga_predict_fctn ndpi_dga_function;
/**
* Check if a string is encoded with punycode
@@ -180,7 +185,7 @@ extern "C" {
* Returns a new initialized detection module
* Note that before you can use it you can still load
* hosts and do other things. As soon as you are ready to use
* it do not forget to call first ndpi_finalize_initalization()
* it do not forget to call first ndpi_finalize_initialization()
*
* You can call this function multiple times, (i.e. to create multiple
* indipendent detection contexts) but all these calls MUST NOT run
@@ -198,12 +203,20 @@ extern "C" {
* @par ndpi_str = the struct created for the protocol detection
*
*/
void ndpi_finalize_initalization(struct ndpi_detection_module_struct *ndpi_str);
void ndpi_finalize_initialization(struct ndpi_detection_module_struct *ndpi_str);
/**
* Frees the memory allocated in the specified flow
* Frees the dynamic memory allocated members in the specified flow
*
* @par flow = the flow to deallocate
* @par flow = the flow struct which dynamic allocated members should be deallocated
*
*/
void ndpi_free_flow_data(struct ndpi_flow_struct *flow);
/**
* Frees the dynamic memory allocated members in the specified flow and the flow struct itself
*
* @par flow = the flow struct and its dynamic allocated members that should be deallocated
*
*/
void ndpi_free_flow(struct ndpi_flow_struct *flow);
@@ -263,7 +276,7 @@ extern "C" {
void ndpi_set_protocol_detection_bitmask2(struct ndpi_detection_module_struct *ndpi_struct,
const NDPI_PROTOCOL_BITMASK * detection_bitmask);
/**
/**
* Function to be called before we give up with detection for a given flow.
* This function reduces the NDPI_UNKNOWN_PROTOCOL detection
*
@@ -351,7 +364,7 @@ extern "C" {
struct ndpi_flow_struct *flow,
NDPI_SELECTION_BITMASK_PROTOCOL_SIZE *ndpi_selection_packet);
/**
* Query the pointer to the layer 4 packet
*
@@ -362,7 +375,7 @@ extern "C" {
* @par l4_protocol_return = protocol of the layer 4 data if return value == 0, undefined otherwise
* @par flags = limit operation on ipv4 or ipv6 packets. Possible values: NDPI_DETECTION_ONLY_IPV4 - NDPI_DETECTION_ONLY_IPV6 - 0 (any)
* @return 0 if layer 4 data could be found correctly;
else != 0
else != 0
*
*/
u_int8_t ndpi_detection_get_l4(const u_int8_t *l3, u_int16_t l3_len, const u_int8_t **l4_return, u_int16_t *l4_len_return,
@@ -412,7 +425,9 @@ extern "C" {
* @par string_to_match_len = the length of the string
* @par ret_match = completed returned match information
* @par is_host_match = value of the second field of struct ndpi_automa
* @return the ID of the matched subprotocol
* @return the ID of the matched subprotocol;
* -1 if automa is not finalized;
* -2 if automa==NULL or string_to_match==NULL or empty string_to_match
*
*/
int ndpi_match_string_subprotocol(struct ndpi_detection_module_struct *ndpi_struct,
@@ -439,6 +454,16 @@ extern "C" {
ndpi_protocol_match_result *ret_match,
u_int16_t master_protocol_id);
/**
* Check if the string content passed match with a protocol
*
* @par ndpi_struct = the detection module
* @par flow = the flow where match the host
* @par subprotocol_id = subprotocol id
*/
void ndpi_check_subprotocol_risk(struct ndpi_detection_module_struct *ndpi_str,
struct ndpi_flow_struct *flow, u_int16_t subprotocol_id);
/**
* Check if the string content passed match with a protocol
*
@@ -466,9 +491,9 @@ extern "C" {
*
*/
void ndpi_exclude_protocol(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow,
u_int16_t master_protocol_id,
const char *_file, const char *_func,int _line);
struct ndpi_flow_struct *flow,
u_int16_t master_protocol_id,
const char *_file, const char *_func,int _line);
/**
* Check if the string -bigram_to_match- match with a bigram of -automa-
*
@@ -478,9 +503,7 @@ extern "C" {
* @return 0
*
*/
int ndpi_match_bigram(struct ndpi_detection_module_struct *ndpi_mod,
ndpi_automa *automa,
char *bigram_to_match);
int ndpi_match_bigram(const char *bigram_to_match);
/**
* Write the protocol name in the buffer -buf- as master_protocol.protocol
@@ -659,6 +682,13 @@ extern "C" {
*/
void ndpi_dump_protocols(struct ndpi_detection_module_struct *mod);
/**
* Write the list of the scores and their associated risks
*
* @par ndpi_mod = the detection module
*/
void ndpi_dump_risks_score();
/**
* Read a file and load the protocols
*
@@ -677,7 +707,29 @@ extern "C" {
int ndpi_load_protocols_file(struct ndpi_detection_module_struct *ndpi_mod,
const char* path);
/**
/**
* Add an IP-address based risk mask
*
* @par ndpi_mod = the detection module
* @par ip = the IP address for which you wanna set the mask
* @par mask = the IP risk mask
* @return 0 if the rule is loaded correctly;
* -1 else
*/
int ndpi_add_ip_risk_mask(struct ndpi_detection_module_struct *ndpi_mod, char *ip, ndpi_risk mask);
/**
* Add a host-address based risk mask
*
* @par ndpi_mod = the detection module
* @par host = the hostname/domain for which you wanna set the mask
* @par mask = the host risk mask
* @return 0 if the rule is loaded correctly;
* -1 else
*/
int ndpi_add_host_risk_mask(struct ndpi_detection_module_struct *ndpi_mod, char *host, ndpi_risk mask);
/**
* Read a file and load the categories
*
* @par ndpi_mod = the detection module
@@ -687,6 +739,35 @@ extern "C" {
*/
int ndpi_load_categories_file(struct ndpi_detection_module_struct *ndpi_str, const char* path);
/**
* Read a file and load the list of risky domains
*
* @par ndpi_mod = the detection module
* @par path = the path of the file
* @return 0 if the file is loaded correctly;
* -1 else
*/
int ndpi_load_risk_domain_file(struct ndpi_detection_module_struct *ndpi_str, const char* path);
/**
* Read a file and load the list of malicious JA3 signatures
*
* @par ndpi_mod = the detection module
* @par path = the path of the file
* @return 0 if the file is loaded correctly;
* -1 else
*/
int ndpi_load_malicious_ja3_file(struct ndpi_detection_module_struct *ndpi_str, const char *path);
/**
* Read a file and load the list of malicious SSL certificate SHA1 fingerprints.
* @par ndpi_mod = the detection module
* @par path = the path of the file
* @return 0 if the file is loaded correctly;
* -1 else
*/
int ndpi_load_malicious_sha1_file(struct ndpi_detection_module_struct *ndpi_str, const char *path);
/**
* Get the total number of the supported protocols
*
@@ -714,7 +795,7 @@ extern "C" {
void ndpi_set_automa(struct ndpi_detection_module_struct *ndpi_struct,
void* automa);
/* NDPI_PROTOCOL_HTTP */
/* NDPI_PROTOCOL_HTTP */
/**
* Retrieve information for HTTP flows
*
@@ -722,7 +803,7 @@ extern "C" {
* @par flow = the detected flow
* @return the HTTP method information about the flow
*
*/
*/
ndpi_http_method ndpi_get_http_method(struct ndpi_detection_module_struct *ndpi_mod,
struct ndpi_flow_struct *flow);
@@ -733,7 +814,7 @@ extern "C" {
* @par flow = the detected flow
* @return the HTTP method information about the flow
*
*/
*/
char* ndpi_get_http_url(struct ndpi_detection_module_struct *ndpi_mod,
struct ndpi_flow_struct *flow);
@@ -744,11 +825,11 @@ extern "C" {
* @par flow = the detected flow
* @return the HTTP method information about the flow
*
*/
*/
char* ndpi_get_http_content_type(struct ndpi_detection_module_struct *ndpi_mod,
struct ndpi_flow_struct *flow);
/* NDPI_PROTOCOL_TOR */
/* NDPI_PROTOCOL_TOR */
/**
* Check if the flow could be detected as TOR protocol
*
@@ -783,7 +864,7 @@ extern "C" {
* Add a string to match to an automata
*
* @par The automata initialized with ndpi_init_automa();
* @par The (sub)string to search
* @par The (sub)string to search (malloc'ed memory)
* @par The number associated with this string
* @return 0 in case of no error, or -1 if an error occurred.
*
@@ -794,7 +875,7 @@ extern "C" {
* Add a string to match to an automata. Same as ndpi_add_string_value_to_automa() with num set to 1
*
* @par The automata initialized with ndpi_init_automa();
* @par The (sub)string to search
* @par The (sub)string to search (malloc'ed memory)
* @return 0 in case of no error, or -1 if an error occurred.
*
*/
@@ -819,16 +900,16 @@ extern "C" {
int ndpi_match_string(void *_automa, char *string_to_match);
int ndpi_load_ip_category(struct ndpi_detection_module_struct *ndpi_struct,
const char *ip_address_and_mask, ndpi_protocol_category_t category);
const char *ip_address_and_mask, ndpi_protocol_category_t category);
int ndpi_load_hostname_category(struct ndpi_detection_module_struct *ndpi_struct,
const char *name_to_add, ndpi_protocol_category_t category);
const char *name_to_add, ndpi_protocol_category_t category);
int ndpi_load_category(struct ndpi_detection_module_struct *ndpi_struct,
const char *ip_or_name, ndpi_protocol_category_t category);
const char *ip_or_name, ndpi_protocol_category_t category);
int ndpi_enable_loaded_categories(struct ndpi_detection_module_struct *ndpi_struct);
int ndpi_fill_ip_protocol_category(struct ndpi_detection_module_struct *ndpi_struct,
u_int32_t saddr,
u_int32_t daddr,
ndpi_protocol *ret);
u_int32_t saddr,
u_int32_t daddr,
ndpi_protocol *ret);
int ndpi_match_custom_category(struct ndpi_detection_module_struct *ndpi_struct,
char *name, u_int name_len, ndpi_protocol_category_t *id);
void ndpi_fill_protocol_category(struct ndpi_detection_module_struct *ndpi_struct,
@@ -845,6 +926,9 @@ extern "C" {
ndpi_l4_proto_info ndpi_get_l4_proto_info(struct ndpi_detection_module_struct *ndpi_struct, u_int16_t ndpi_proto_id);
const char* ndpi_get_l4_proto_name(ndpi_l4_proto_info proto);
u_int16_t ndpi_get_lower_proto(ndpi_protocol proto);
u_int16_t ndpi_get_upper_proto(ndpi_protocol proto);
ndpi_proto_defaults_t* ndpi_get_proto_defaults(struct ndpi_detection_module_struct *ndpi_mod);
u_int ndpi_get_ndpi_num_supported_protocols(struct ndpi_detection_module_struct *ndpi_mod);
u_int ndpi_get_ndpi_num_custom_protocols(struct ndpi_detection_module_struct *ndpi_mod);
@@ -881,9 +965,6 @@ extern "C" {
void set_ndpi_flow_free(void (*__ndpi_flow_free)(void *ptr));
void set_ndpi_debug_function(struct ndpi_detection_module_struct *ndpi_str,
ndpi_debug_function_ptr ndpi_debug_printf);
//void * ndpi_malloc(size_t size);
//void * ndpi_calloc(unsigned long count, size_t size);
//void ndpi_free(void *ptr);
u_int16_t ndpi_get_api_version(void);
const char *ndpi_get_gcrypt_version(void);
@@ -904,8 +985,12 @@ extern "C" {
char *buffer, u_int buffer_size,
u_int8_t min_string_match_len, /* Will return 0 if no string > min_string_match_len have been found */
char *outbuf, u_int outbuf_len);
/* Return a flow info string (summarized). Does only work for DNS/HTTP/TLS/QUIC. */
const char* ndpi_get_flow_info(struct ndpi_flow_struct const * const flow,
ndpi_protocol const * const l7_protocol);
char* ndpi_ssl_version2str(struct ndpi_flow_struct *flow,
u_int16_t version, u_int8_t *unknown_tls_version);
int ndpi_netbios_name_interpret(u_char *in, u_int in_len, u_char *out, u_int out_len);
void ndpi_patchIPv6Address(char *str);
void ndpi_user_pwd_payload_copy(u_int8_t *dest, u_int dest_len, u_int offset,
const u_int8_t *src, u_int src_len);
@@ -930,14 +1015,40 @@ extern "C" {
ndpi_serializer *serializer);
void ndpi_md5(const u_char *data, size_t data_len, u_char hash[16]);
u_int32_t ndpi_quick_hash(unsigned char *str, u_int str_len);
const char* ndpi_http_method2str(ndpi_http_method m);
ndpi_http_method ndpi_http_str2method(const char* method, ssize_t method_len);
/* ptree (trie) API */
ndpi_http_method ndpi_http_str2method(const char* method, u_int16_t method_len);
/* Utility functions to fill prefix (used by the patricia tree) */
int ndpi_fill_prefix_v4(ndpi_prefix_t *p, const struct in_addr *a, int b, int mb);
int ndpi_fill_prefix_v6(ndpi_prefix_t *prefix, const struct in6_addr *addr, int bits, int maxbits);
int ndpi_fill_prefix_mac(ndpi_prefix_t *prefix, u_int8_t *mac, int bits, int maxbits);
/* Patricia tree API (radix tree supporting IPv4/IPv6/MAC) */
ndpi_patricia_tree_t *ndpi_patricia_new(u_int16_t maxbits);
ndpi_patricia_tree_t *ndpi_patricia_clone (const ndpi_patricia_tree_t * const from);
void ndpi_patricia_destroy(ndpi_patricia_tree_t *patricia, ndpi_void_fn_t func);
ndpi_patricia_node_t *ndpi_patricia_search_exact(ndpi_patricia_tree_t *patricia, ndpi_prefix_t *prefix);
ndpi_patricia_node_t *ndpi_patricia_search_best(ndpi_patricia_tree_t *patricia, ndpi_prefix_t *prefix);
ndpi_patricia_node_t *ndpi_patricia_lookup(ndpi_patricia_tree_t *patricia, ndpi_prefix_t *prefix);
size_t ndpi_patricia_walk_tree_inorder(ndpi_patricia_tree_t *patricia, ndpi_void_fn3_t func, void *data);
size_t ndpi_patricia_walk_inorder(ndpi_patricia_node_t *node, ndpi_void_fn3_t func, void *data);
void ndpi_patricia_remove(ndpi_patricia_tree_t *patricia, ndpi_patricia_node_t *node);
void ndpi_patricia_set_node_u64(ndpi_patricia_node_t *node, u_int64_t value);
u_int64_t ndpi_patricia_get_node_u64(ndpi_patricia_node_t *node);
void ndpi_patricia_set_node_data(ndpi_patricia_node_t *node, void *data);
void *ndpi_patricia_get_node_data(ndpi_patricia_node_t *node);
ndpi_prefix_t *ndpi_patricia_get_node_prefix(ndpi_patricia_node_t *node);
u_int16_t ndpi_patricia_get_node_bits(ndpi_patricia_node_t *node);
u_int16_t ndpi_patricia_get_maxbits(ndpi_patricia_tree_t *tree);
/* ptree (trie) API - a wrapper on top of Patricia that seamlessly handle IPv4 and IPv6 */
ndpi_ptree_t* ndpi_ptree_create(void);
int ndpi_ptree_insert(ndpi_ptree_t *tree, const ndpi_ip_addr_t *addr, u_int8_t bits, u_int32_t user_data);
int ndpi_ptree_match_addr(ndpi_ptree_t *tree, const ndpi_ip_addr_t *addr, u_int32_t *user_data);
int ndpi_ptree_insert(ndpi_ptree_t *tree, const ndpi_ip_addr_t *addr, u_int8_t bits, u_int64_t user_data);
int ndpi_ptree_match_addr(ndpi_ptree_t *tree, const ndpi_ip_addr_t *addr, u_int64_t *user_data);
void ndpi_ptree_destroy(ndpi_ptree_t *tree);
/* General purpose utilities */
@@ -953,7 +1064,7 @@ extern "C" {
/**
* Initialize a serializer handle (allocated by the caller).
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param fmt The serialization format (ndpi_serialization_format_json, ndpi_serialization_format_tlv, ndpi_serialization_format_csv)
* @return 0 on success, a negative number otherwise
*/
@@ -961,28 +1072,28 @@ extern "C" {
/**
* Initialize a serializer handle. Same as ndpi_init_serializer, but with some low-level settings.
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param fmt The serialization format (ndpi_serialization_format_json, ndpi_serialization_format_tlv, ndpi_serialization_format_csv)
* @param buffer_size The initial internal buffer_size
* @return 0 on success, a negative number otherwise
* @return 0 on success, a negative number otherwise
*/
int ndpi_init_serializer_ll(ndpi_serializer *serializer, ndpi_serialization_format fmt, u_int32_t buffer_size);
/**
* Release all allocated data structure.
* @param serializer The serializer handle
* @param serializer The serializer handle
*/
void ndpi_term_serializer(ndpi_serializer *serializer);
/**
* Reset the serializer (cleanup the internal buffer to start a new serialization)
* @param serializer The serializer handle
* @param serializer The serializer handle
*/
void ndpi_reset_serializer(ndpi_serializer *serializer);
/**
* Serialize a 32-bit unsigned int key and a 32-bit unsigned int value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @return 0 on success, a negative number otherwise
@@ -990,8 +1101,8 @@ extern "C" {
int ndpi_serialize_uint32_uint32(ndpi_serializer *serializer, u_int32_t key, u_int32_t value);
/**
* Serialize a 32-bit unsigned int key and a 64-bit unsigned int value
* @param serializer The serializer handle
* Serialize a 32-bit unsigned int key and a 64-bit unsigned int value
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @return 0 on success, a negative number otherwise
@@ -1000,7 +1111,7 @@ extern "C" {
/**
* Serialize a 32-bit unsigned int key and a 32-bit signed int value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @return 0 on success, a negative number otherwise
@@ -1009,7 +1120,7 @@ extern "C" {
/**
* Serialize a 32-bit unsigned int key and a 64-bit signed int value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @return 0 on success, a negative number otherwise
@@ -1018,7 +1129,7 @@ extern "C" {
/**
* Serialize a 32-bit unsigned int key and a float value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @param format The float value format
@@ -1028,7 +1139,7 @@ extern "C" {
/**
* Serialize a 32-bit unsigned int key and a string value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @return 0 on success, a negative number otherwise
@@ -1037,7 +1148,7 @@ extern "C" {
/**
* Serialize a 32-bit unsigned int key and a boolean value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @return 0 on success, a negative number otherwise
@@ -1046,7 +1157,7 @@ extern "C" {
/**
* Serialize an unterminated string key and a 32-bit signed int value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param klen The key length
* @param value The field value
@@ -1056,7 +1167,7 @@ extern "C" {
/**
* Serialize a string key and a 32-bit signed int value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @return 0 on success, a negative number otherwise
@@ -1065,7 +1176,7 @@ extern "C" {
/**
* Serialize an unterminated string key and a 64-bit signed int value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param klen The key length
* @param value The field value
@@ -1075,7 +1186,7 @@ extern "C" {
/**
* Serialize a string key and a 64-bit signed int value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @return 0 on success, a negative number otherwise
@@ -1084,7 +1195,7 @@ extern "C" {
/**
* Serialize an unterminated string key and a 32-bit unsigned int value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param klen The key length
* @param value The field value
@@ -1094,7 +1205,7 @@ extern "C" {
/**
* Serialize a string key and a 32-bit unsigned int value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @return 0 on success, a negative number otherwise
@@ -1103,7 +1214,7 @@ extern "C" {
/**
* Serialize a string key and a float value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @param format The float format
@@ -1113,7 +1224,7 @@ extern "C" {
/**
* Serialize an unterminated string key and a 64-bit unsigned int value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param klen The key length
* @param value The field value
@@ -1123,7 +1234,7 @@ extern "C" {
/**
* Serialize a string key and a 64-bit unsigned int value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @return 0 on success, a negative number otherwise
@@ -1132,7 +1243,7 @@ extern "C" {
/**
* Serialize an unterminated string key and an unterminated string value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param klen The key length
* @param value The field value
@@ -1143,7 +1254,7 @@ extern "C" {
/**
* Serialize a string key and a string value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @return 0 on success, a negative number otherwise
@@ -1152,7 +1263,7 @@ extern "C" {
/**
* Serialize a string key and an unterminated string value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @param vlen The value length
@@ -1162,7 +1273,7 @@ extern "C" {
/**
* Serialize a string key and a raw value (this is a string which is added to the JSON without any quote or escaping)
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @param vlen The value length
@@ -1172,7 +1283,7 @@ extern "C" {
/**
* Serialize an unterminated string key and a float value
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param klen The key length
* @param value The field value
@@ -1182,8 +1293,8 @@ extern "C" {
int ndpi_serialize_binary_float(ndpi_serializer *_serializer, const char *key, u_int16_t klen, float value, const char *format /* e.f. "%.2f" */);
/**
* Serialize a string key and a a float value
* @param serializer The serializer handle
* Serialize a string key and a a float value
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @param format The float format
@@ -1192,8 +1303,18 @@ extern "C" {
int ndpi_serialize_string_float(ndpi_serializer *serializer, const char *key, float value, const char *format /* e.f. "%.2f" */);
/**
* Serialize a string key and a boolean value
* @param serializer The serializer handle
* Serialize an unterminated string key and a boolean value (JSON/CSV only, not supported by TLV)
* @param serializer The serializer handle
* @param key The field name or ID
* @param klen The key length
* @param value The field value
* @return 0 on success, a negative number otherwise
*/
int ndpi_serialize_binary_boolean(ndpi_serializer *_serializer, const char *key, u_int16_t klen, u_int8_t value);
/**
* Serialize a string key and a boolean value (JSON/CSV only, not supported by TLV)
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @return 0 on success, a negative number otherwise
@@ -1202,7 +1323,7 @@ extern "C" {
/**
* Serialize a raw record in an array (this is a low-level function and its use is not recommended)
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param record The record value
* @param record_len The record length
* @return 0 on success, a negative number otherwise
@@ -1212,15 +1333,15 @@ extern "C" {
/**
* Serialize an End-Of-Record (the current object becomes is terminated and added to an array,
* and a new object is created where the next items will be added)
* @param serializer The serializer handle
* @param serializer The serializer handle
* @return 0 on success, a negative number otherwise
*/
int ndpi_serialize_end_of_record(ndpi_serializer *serializer);
/**
* Serialize the start of a list with an unterminated string key, where the next serialized items
* Serialize the start of a list with an unterminated string key, where the next serialized items
* will be added (note: keys for the new items are ignored)
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param klen The key length
* @return 0 on success, a negative number otherwise
@@ -1230,7 +1351,7 @@ extern "C" {
/**
* Serialize the start of a list, where the next serialized items will be added (note: keys for
* the new items are ignored)
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @return 0 on success, a negative number otherwise
@@ -1239,14 +1360,14 @@ extern "C" {
/**
* Serialize the end of a list
* @param serializer The serializer handle
* @param serializer The serializer handle
* @return 0 on success, a negative number otherwise
*/
int ndpi_serialize_end_of_list(ndpi_serializer *serializer);
/**
* Serialize the start of a block with an unterminated string key
* @param serializer The serializer handle
* Serialize the start of a block with an unterminated string key
* @param serializer The serializer handle
* @param key The field name or ID
* @param klen The key length
* @return 0 on success, a negative number otherwise
@@ -1254,16 +1375,16 @@ extern "C" {
int ndpi_serialize_start_of_block_binary(ndpi_serializer *_serializer, const char *key, u_int16_t klen);
/**
* Serialize the start of a block with a string key
* @param serializer The serializer handle
* Serialize the start of a block with a string key
* @param serializer The serializer handle
* @param key The field name or ID
* @return 0 on success, a negative number otherwise
*/
int ndpi_serialize_start_of_block(ndpi_serializer *serializer, const char *key);
/**
* Serialize the end of a block
* @param serializer The serializer handle
* Serialize the end of a block
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @return 0 on success, a negative number otherwise
@@ -1272,7 +1393,7 @@ extern "C" {
/**
* Return the serialized buffer
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param buffer_len The buffer length (out)
* @return The buffer
*/
@@ -1280,7 +1401,7 @@ extern "C" {
/**
* Return the current serialized buffer length
* @param serializer The serializer handle
* @param serializer The serializer handle
* @return The buffer length
*/
u_int32_t ndpi_serializer_get_buffer_len(ndpi_serializer *serializer);
@@ -1294,29 +1415,29 @@ extern "C" {
/**
* Change the serializer buffer length
* @param serializer The serializer handle
* @param l The new buffer length
* @param serializer The serializer handle
* @param l The new buffer length
* @return 0 on success, a negative number otherwise
*/
int ndpi_serializer_set_buffer_len(ndpi_serializer *serializer, u_int32_t l);
/**
* Return the configured serialization format
* @param serializer The serializer handle
* @param serializer The serializer handle
* @return The serialization format
*/
ndpi_serialization_format ndpi_serializer_get_format(ndpi_serializer *serializer);
/**
* Set the CSV separator
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param separator The separator
*/
void ndpi_serializer_set_csv_separator(ndpi_serializer *serializer, char separator);
/**
* Return the header automatically built from keys (CSV only)
* @param serializer The serializer handle
* @param serializer The serializer handle
* @param buffer_len The buffer length (out)
* @return The header
*/
@@ -1324,13 +1445,13 @@ extern "C" {
/**
* Create a snapshot of the internal buffer for later rollback (ndpi_serializer_rollback_snapshot)
* @param serializer The serializer handle
* @param serializer The serializer handle
*/
void ndpi_serializer_create_snapshot(ndpi_serializer *serializer);
/**
* Rollback to the latest snapshot
* @param serializer The serializer handle
* @param serializer The serializer handle
*/
void ndpi_serializer_rollback_snapshot(ndpi_serializer *serializer);
@@ -1362,7 +1483,7 @@ extern "C" {
/* Data analysis */
struct ndpi_analyze_struct* ndpi_alloc_data_analysis(u_int16_t _max_series_len);
void ndpi_init_data_analysis(struct ndpi_analyze_struct *s, u_int16_t _max_series_len);
void ndpi_free_data_analysis(struct ndpi_analyze_struct *d);
void ndpi_free_data_analysis(struct ndpi_analyze_struct *d, u_int8_t free_pointer);
void ndpi_reset_data_analysis(struct ndpi_analyze_struct *d);
void ndpi_data_add_value(struct ndpi_analyze_struct *s, const u_int32_t value);
@@ -1381,6 +1502,37 @@ extern "C" {
u_int32_t ndpi_data_max(struct ndpi_analyze_struct *s);
float ndpi_data_ratio(u_int32_t sent, u_int32_t rcvd);
/* ******************************* */
int ndpi_alloc_rsi(struct ndpi_rsi_struct *s, u_int16_t num_learning_values);
void ndpi_free_rsi(struct ndpi_rsi_struct *s);
float ndpi_rsi_add_value(struct ndpi_rsi_struct *s, const u_int32_t value);
/* ******************************* */
int ndpi_hw_init(struct ndpi_hw_struct *hw, u_int16_t num_periods, u_int8_t additive_seeasonal,
double alpha, double beta, double gamma, float significance);
void ndpi_hw_free(struct ndpi_hw_struct *hw);
int ndpi_hw_add_value(struct ndpi_hw_struct *hw, const u_int64_t value, double *forecast, double *confidence_band);
/* ******************************* */
int ndpi_ses_init(struct ndpi_ses_struct *ses, double alpha, float significance);
int ndpi_ses_add_value(struct ndpi_ses_struct *ses, const u_int64_t _value, double *forecast, double *confidence_band);
/* ******************************* */
int ndpi_des_init(struct ndpi_des_struct *des, double alpha, double beta, float significance);
int ndpi_des_add_value(struct ndpi_des_struct *des, const u_int64_t _value, double *forecast, double *confidence_band);
/* ******************************* */
int ndpi_jitter_init(struct ndpi_jitter_struct *hw, u_int16_t num_periods);
void ndpi_jitter_free(struct ndpi_jitter_struct *hw);
float ndpi_jitter_add_value(struct ndpi_jitter_struct *s, const float value);
/* ******************************* */
const char* ndpi_data_ratio2str(float ratio);
void ndpi_data_print_window_values(struct ndpi_analyze_struct *s); /* debug */
@@ -1392,6 +1544,10 @@ extern "C" {
void ndpi_serialize_risk(ndpi_serializer *serializer, struct ndpi_flow_struct *flow);
const char* ndpi_risk2str(ndpi_risk_enum risk);
const char* ndpi_severity2str(ndpi_risk_severity s);
ndpi_risk_info* ndpi_risk2severity(ndpi_risk_enum risk);
u_int16_t ndpi_risk2score(ndpi_risk risk,
u_int16_t *client_score, u_int16_t *server_score);
/* ******************************* */
@@ -1401,7 +1557,7 @@ extern "C" {
int ndpi_hll_init(struct ndpi_hll *hll, u_int8_t bits);
void ndpi_hll_destroy(struct ndpi_hll *hll);
void ndpi_hll_reset(struct ndpi_hll *hll);
/* Add values */
void ndpi_hll_add(struct ndpi_hll *hll, const char *data, size_t data_len);
void ndpi_hll_add_number(struct ndpi_hll *hll, u_int32_t value) ;
@@ -1425,7 +1581,34 @@ extern "C" {
u_int8_t num_clusters, u_int16_t *cluster_ids,
struct ndpi_bin *centroids);
/* ******************************* */
u_int32_t ndpi_quick_16_byte_hash(u_int8_t *in_16_bytes_long);
/* ******************************* */
ndpi_str_hash* ndpi_hash_alloc(u_int32_t max_num_entries);
void ndpi_hash_free(ndpi_str_hash *h);
int ndpi_hash_find_entry(ndpi_str_hash *h, char *key, u_int key_len, u_int8_t *value);
int ndpi_hash_add_entry(ndpi_str_hash *h, char *key, u_int8_t key_len, u_int8_t value);
/* ******************************* */
int ndpi_load_geoip(struct ndpi_detection_module_struct *ndpi_str,
const char *ip_city_data, const char *ip_as_data);
void ndpi_free_geoip(struct ndpi_detection_module_struct *ndpi_str);
int ndpi_get_geoip_asn(struct ndpi_detection_module_struct *ndpi_str,
char *ip, u_int32_t *asn);
int ndpi_get_geoip_country_continent(struct ndpi_detection_module_struct *ndpi_str, char *ip,
char *country_code, u_int8_t country_code_len,
char *continent, u_int8_t continent_len);
/* ******************************* */
char* ndpi_get_flow_name(struct ndpi_flow_struct *flow);
/* ******************************* */
#ifdef __cplusplus
}
#endif
+11 -7
View File
@@ -181,7 +181,10 @@
#define NDPI_JABBER_FT_TIMEOUT 5
#define NDPI_SOULSEEK_CONNECTION_IP_TICK_TIMEOUT 600
#ifndef _NDPI_CONFIG_H_
#include "ndpi_config.h" /* To have access to NDPI_ENABLE_DEBUG_MESSAGES */
#define _NDPI_CONFIG_H_
#endif
#ifdef NDPI_ENABLE_DEBUG_MESSAGES
#define NDPI_LOG(proto, m, log_level, args...) \
@@ -257,6 +260,7 @@
((x.u6_addr.u6_addr64[0] < y.u6_addr.u6_addr64[0]) || ((x.u6_addr.u6_addr64[0] == y.u6_addr.u6_addr64[0]) && (x.u6_addr.u6_addr64[1] < y.u6_addr.u6_addr64[1])))
#define NDPI_NUM_BITS 512
#define NDPI_NUM_BITS_MASK (512-1)
#define NDPI_BITS /* 32 */ (sizeof(ndpi_ndpi_mask) * 8 /* number of bits in a byte */) /* bits per mask */
#define howmanybits(x, y) (((x)+((y)-1))/(y))
@@ -278,18 +282,18 @@
#define NDPI_BITMASK_SET_ALL(a) NDPI_ONE(&a)
#define NDPI_BITMASK_SET(a, b) { memcpy(&a, &b, sizeof(NDPI_PROTOCOL_BITMASK)); }
#define NDPI_SET_BIT(num, n) num |= 1UL << ( n )
#define NDPI_CLR_BIT(num, n) num &= ~(1UL << ( n ))
#define NDPI_CLR_BIT(num, n) num &= ~(1UL << ( n ))
#define NDPI_ISSET_BIT(num, n) (num & (1UL << ( n )))
#define NDPI_SET_BIT(num, n) num |= 1ULL << ( n )
#define NDPI_CLR_BIT(num, n) num &= ~(1ULL << ( n ))
#define NDPI_CLR_BIT(num, n) num &= ~(1ULL << ( n ))
#define NDPI_ISSET_BIT(num, n) (num & (1ULL << ( n )))
#define NDPI_ZERO_BIT(num) num = 0
/* this is a very very tricky macro *g*,
* the compiler will remove all shifts here if the protocol is static...
*/
#define NDPI_ADD_PROTOCOL_TO_BITMASK(bmask,value) NDPI_SET(&bmask,value)
#define NDPI_DEL_PROTOCOL_FROM_BITMASK(bmask,value) NDPI_CLR(&bmask,value)
#define NDPI_COMPARE_PROTOCOL_TO_BITMASK(bmask,value) NDPI_ISSET(&bmask,value)
#define NDPI_ADD_PROTOCOL_TO_BITMASK(bmask,value) NDPI_SET(&bmask, value & NDPI_NUM_BITS_MASK)
#define NDPI_DEL_PROTOCOL_FROM_BITMASK(bmask,value) NDPI_CLR(&bmask, value & NDPI_NUM_BITS_MASK)
#define NDPI_COMPARE_PROTOCOL_TO_BITMASK(bmask,value) NDPI_ISSET(&bmask, value & NDPI_NUM_BITS_MASK)
#define NDPI_SAVE_AS_BITMASK(bmask,value) { NDPI_ZERO(&bmask) ; NDPI_ADD_PROTOCOL_TO_BITMASK(bmask, value); }
+408
View File
@@ -0,0 +1,408 @@
/*
* ndpi_encryption.h
*
* Copyright (C) 2016-21 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
/* https://wiki.mozilla.org/Security/Cipher_Suites */
/* https://www.oryx-embedded.com/doc/tls__cipher__suites_8h.html */
#define TLS_NULL_WITH_NULL_NULL 0x0000
#define TLS_RSA_EXPORT_WITH_RC4_40_MD5 0x0003
#define TLS_RSA_EXPORT_WITH_RC2_CBC_40_MD5 0x0006
#define TLS_RSA_EXPORT_WITH_DES40_CBC_SHA 0x0008
#define TLS_RSA_WITH_NULL_MD5 0x0001
#define TLS_RSA_WITH_NULL_SHA 0x0002
#define TLS_RSA_WITH_NULL_SHA256 0x003B
#define TLS_RSA_WITH_RC4_128_MD5 0x0004
#define TLS_RSA_WITH_RC4_128_SHA 0x0005
#define TLS_RSA_WITH_IDEA_CBC_SHA 0x0007
#define TLS_RSA_WITH_DES_CBC_SHA 0x0009
#define TLS_RSA_WITH_3DES_EDE_CBC_SHA 0x000A
#define TLS_RSA_WITH_AES_128_CBC_SHA 0x002F
#define TLS_RSA_WITH_AES_256_CBC_SHA 0x0035
#define TLS_RSA_WITH_AES_128_CBC_SHA256 0x003C
#define TLS_RSA_WITH_AES_256_CBC_SHA256 0x003D
#define TLS_RSA_WITH_AES_128_GCM_SHA256 0x009C
#define TLS_RSA_WITH_AES_256_GCM_SHA384 0x009D
#define TLS_RSA_WITH_AES_128_CCM 0xC09C
#define TLS_RSA_WITH_AES_256_CCM 0xC09D
#define TLS_RSA_WITH_AES_128_CCM_8 0xC0A0
#define TLS_RSA_WITH_AES_256_CCM_8 0xC0A1
#define TLS_RSA_WITH_CAMELLIA_128_CBC_SHA 0x0041
#define TLS_RSA_WITH_CAMELLIA_256_CBC_SHA 0x0084
#define TLS_RSA_WITH_CAMELLIA_128_CBC_SHA256 0x00BA
#define TLS_RSA_WITH_CAMELLIA_256_CBC_SHA256 0x00C0
#define TLS_RSA_WITH_CAMELLIA_128_GCM_SHA256 0xC07A
#define TLS_RSA_WITH_CAMELLIA_256_GCM_SHA384 0xC07B
#define TLS_RSA_WITH_SEED_CBC_SHA 0x0096
#define TLS_RSA_WITH_ARIA_128_CBC_SHA256 0xC03C
#define TLS_RSA_WITH_ARIA_256_CBC_SHA384 0xC03D
#define TLS_RSA_WITH_ARIA_128_GCM_SHA256 0xC050
#define TLS_RSA_WITH_ARIA_256_GCM_SHA384 0xC051
#define TLS_DH_RSA_EXPORT_WITH_DES40_CBC_SHA 0x000E
#define TLS_DH_RSA_WITH_DES_CBC_SHA 0x000F
#define TLS_DH_RSA_WITH_3DES_EDE_CBC_SHA 0x0010
#define TLS_DH_RSA_WITH_AES_128_CBC_SHA 0x0031
#define TLS_DH_RSA_WITH_AES_256_CBC_SHA 0x0037
#define TLS_DH_RSA_WITH_AES_128_CBC_SHA256 0x003F
#define TLS_DH_RSA_WITH_AES_256_CBC_SHA256 0x0069
#define TLS_DH_RSA_WITH_AES_128_GCM_SHA256 0x00A0
#define TLS_DH_RSA_WITH_AES_256_GCM_SHA384 0x00A1
#define TLS_DH_RSA_WITH_CAMELLIA_128_CBC_SHA 0x0043
#define TLS_DH_RSA_WITH_CAMELLIA_256_CBC_SHA 0x0086
#define TLS_DH_RSA_WITH_CAMELLIA_128_CBC_SHA256 0x00BC
#define TLS_DH_RSA_WITH_CAMELLIA_256_CBC_SHA256 0x00C2
#define TLS_DH_RSA_WITH_CAMELLIA_128_GCM_SHA256 0xC07E
#define TLS_DH_RSA_WITH_CAMELLIA_256_GCM_SHA384 0xC07F
#define TLS_DH_RSA_WITH_SEED_CBC_SHA 0x0098
#define TLS_DH_RSA_WITH_ARIA_128_CBC_SHA256 0xC040
#define TLS_DH_RSA_WITH_ARIA_256_CBC_SHA384 0xC041
#define TLS_DH_RSA_WITH_ARIA_128_GCM_SHA256 0xC054
#define TLS_DH_RSA_WITH_ARIA_256_GCM_SHA384 0xC055
#define TLS_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA 0x0014
#define TLS_DHE_RSA_WITH_DES_CBC_SHA 0x0015
#define TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA 0x0016
#define TLS_DHE_RSA_WITH_AES_128_CBC_SHA 0x0033
#define TLS_DHE_RSA_WITH_AES_256_CBC_SHA 0x0039
#define TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 0x0067
#define TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 0x006B
#define TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 0x009E
#define TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 0x009F
#define TLS_DHE_RSA_WITH_AES_128_CCM 0xC09E
#define TLS_DHE_RSA_WITH_AES_256_CCM 0xC09F
#define TLS_DHE_RSA_WITH_AES_128_CCM_8 0xC0A2
#define TLS_DHE_RSA_WITH_AES_256_CCM_8 0xC0A3
#define TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA 0x0045
#define TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA 0x0088
#define TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA256 0x00BE
#define TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA256 0x00C4
#define TLS_DHE_RSA_WITH_CAMELLIA_128_GCM_SHA256 0xC07C
#define TLS_DHE_RSA_WITH_CAMELLIA_256_GCM_SHA384 0xC07D
#define TLS_DHE_RSA_WITH_SEED_CBC_SHA 0x009A
#define TLS_DHE_RSA_WITH_ARIA_128_CBC_SHA256 0xC044
#define TLS_DHE_RSA_WITH_ARIA_256_CBC_SHA384 0xC045
#define TLS_DHE_RSA_WITH_ARIA_128_GCM_SHA256 0xC052
#define TLS_DHE_RSA_WITH_ARIA_256_GCM_SHA384 0xC053
#define TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256 0xCCAA
#define TLS_DH_DSS_EXPORT_WITH_DES40_CBC_SHA 0x000B
#define TLS_DH_DSS_WITH_DES_CBC_SHA 0x000C
#define TLS_DH_DSS_WITH_3DES_EDE_CBC_SHA 0x000D
#define TLS_DH_DSS_WITH_AES_128_CBC_SHA 0x0030
#define TLS_DH_DSS_WITH_AES_256_CBC_SHA 0x0036
#define TLS_DH_DSS_WITH_AES_128_CBC_SHA256 0x003E
#define TLS_DH_DSS_WITH_AES_256_CBC_SHA256 0x0068
#define TLS_DH_DSS_WITH_AES_128_GCM_SHA256 0x00A4
#define TLS_DH_DSS_WITH_AES_256_GCM_SHA384 0x00A5
#define TLS_DH_DSS_WITH_CAMELLIA_128_CBC_SHA 0x0042
#define TLS_DH_DSS_WITH_CAMELLIA_256_CBC_SHA 0x0085
#define TLS_DH_DSS_WITH_CAMELLIA_128_CBC_SHA256 0x00BB
#define TLS_DH_DSS_WITH_CAMELLIA_256_CBC_SHA256 0x00C1
#define TLS_DH_DSS_WITH_CAMELLIA_128_GCM_SHA256 0xC082
#define TLS_DH_DSS_WITH_CAMELLIA_256_GCM_SHA384 0xC083
#define TLS_DH_DSS_WITH_SEED_CBC_SHA 0x0097
#define TLS_DH_DSS_WITH_ARIA_128_CBC_SHA256 0xC03E
#define TLS_DH_DSS_WITH_ARIA_256_CBC_SHA384 0xC03F
#define TLS_DH_DSS_WITH_ARIA_128_GCM_SHA256 0xC058
#define TLS_DH_DSS_WITH_ARIA_256_GCM_SHA384 0xC059
#define TLS_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA 0x0011
#define TLS_DHE_DSS_WITH_DES_CBC_SHA 0x0012
#define TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA 0x0013
#define TLS_DHE_DSS_WITH_AES_128_CBC_SHA 0x0032
#define TLS_DHE_DSS_WITH_AES_256_CBC_SHA 0x0038
#define TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 0x0040
#define TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 0x006A
#define TLS_DHE_DSS_WITH_AES_128_GCM_SHA256 0x00A2
#define TLS_DHE_DSS_WITH_AES_256_GCM_SHA384 0x00A3
#define TLS_DHE_DSS_WITH_CAMELLIA_128_CBC_SHA 0x0044
#define TLS_DHE_DSS_WITH_CAMELLIA_256_CBC_SHA 0x0087
#define TLS_DHE_DSS_WITH_CAMELLIA_128_CBC_SHA256 0x00BD
#define TLS_DHE_DSS_WITH_CAMELLIA_256_CBC_SHA256 0x00C3
#define TLS_DHE_DSS_WITH_CAMELLIA_128_GCM_SHA256 0xC080
#define TLS_DHE_DSS_WITH_CAMELLIA_256_GCM_SHA384 0xC081
#define TLS_DHE_DSS_WITH_SEED_CBC_SHA 0x0099
#define TLS_DHE_DSS_WITH_ARIA_128_CBC_SHA256 0xC042
#define TLS_DHE_DSS_WITH_ARIA_256_CBC_SHA384 0xC043
#define TLS_DHE_DSS_WITH_ARIA_128_GCM_SHA256 0xC056
#define TLS_DHE_DSS_WITH_ARIA_256_GCM_SHA384 0xC057
#define TLS_DH_ANON_EXPORT_WITH_RC4_40_MD5 0x0017
#define TLS_DH_ANON_EXPORT_WITH_DES40_CBC_SHA 0x0019
#define TLS_DH_ANON_WITH_RC4_128_MD5 0x0018
#define TLS_DH_ANON_WITH_DES_CBC_SHA 0x001A
#define TLS_DH_ANON_WITH_3DES_EDE_CBC_SHA 0x001B
#define TLS_DH_ANON_WITH_AES_128_CBC_SHA 0x0034
#define TLS_DH_ANON_WITH_AES_256_CBC_SHA 0x003A
#define TLS_DH_ANON_WITH_AES_128_CBC_SHA256 0x006C
#define TLS_DH_ANON_WITH_AES_256_CBC_SHA256 0x006D
#define TLS_DH_ANON_WITH_AES_128_GCM_SHA256 0x00A6
#define TLS_DH_ANON_WITH_AES_256_GCM_SHA384 0x00A7
#define TLS_DH_ANON_WITH_CAMELLIA_128_CBC_SHA 0x0046
#define TLS_DH_ANON_WITH_CAMELLIA_256_CBC_SHA 0x0089
#define TLS_DH_ANON_WITH_CAMELLIA_128_CBC_SHA256 0x00BF
#define TLS_DH_ANON_WITH_CAMELLIA_256_CBC_SHA256 0x00C5
#define TLS_DH_ANON_WITH_CAMELLIA_128_GCM_SHA256 0xC084
#define TLS_DH_ANON_WITH_CAMELLIA_256_GCM_SHA384 0xC085
#define TLS_DH_ANON_WITH_SEED_CBC_SHA 0x009B
#define TLS_DH_ANON_WITH_ARIA_128_CBC_SHA256 0xC046
#define TLS_DH_ANON_WITH_ARIA_256_CBC_SHA384 0xC047
#define TLS_DH_ANON_WITH_ARIA_128_GCM_SHA256 0xC05A
#define TLS_DH_ANON_WITH_ARIA_256_GCM_SHA384 0xC05B
#define TLS_ECDH_RSA_WITH_NULL_SHA 0xC00B
#define TLS_ECDH_RSA_WITH_RC4_128_SHA 0xC00C
#define TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA 0xC00D
#define TLS_ECDH_RSA_WITH_AES_128_CBC_SHA 0xC00E
#define TLS_ECDH_RSA_WITH_AES_256_CBC_SHA 0xC00F
#define TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256 0xC029
#define TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 0xC02A
#define TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256 0xC031
#define TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384 0xC032
#define TLS_ECDH_RSA_WITH_CAMELLIA_128_CBC_SHA256 0xC078
#define TLS_ECDH_RSA_WITH_CAMELLIA_256_CBC_SHA384 0xC079
#define TLS_ECDH_RSA_WITH_CAMELLIA_128_GCM_SHA256 0xC08C
#define TLS_ECDH_RSA_WITH_CAMELLIA_256_GCM_SHA384 0xC08D
#define TLS_ECDH_RSA_WITH_ARIA_128_CBC_SHA256 0xC04E
#define TLS_ECDH_RSA_WITH_ARIA_256_CBC_SHA384 0xC04F
#define TLS_ECDH_RSA_WITH_ARIA_128_GCM_SHA256 0xC062
#define TLS_ECDH_RSA_WITH_ARIA_256_GCM_SHA384 0xC063
#define TLS_ECDHE_RSA_WITH_NULL_SHA 0xC010
#define TLS_ECDHE_RSA_WITH_RC4_128_SHA 0xC011
#define TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA 0xC012
#define TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA 0xC013
#define TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA 0xC014
#define TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 0xC027
#define TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 0xC028
#define TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 0xC02F
#define TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 0xC030
#define TLS_ECDHE_RSA_WITH_CAMELLIA_128_CBC_SHA256 0xC076
#define TLS_ECDHE_RSA_WITH_CAMELLIA_256_CBC_SHA384 0xC077
#define TLS_ECDHE_RSA_WITH_CAMELLIA_128_GCM_SHA256 0xC08A
#define TLS_ECDHE_RSA_WITH_CAMELLIA_256_GCM_SHA384 0xC08B
#define TLS_ECDHE_RSA_WITH_ARIA_128_CBC_SHA256 0xC04C
#define TLS_ECDHE_RSA_WITH_ARIA_256_CBC_SHA384 0xC04D
#define TLS_ECDHE_RSA_WITH_ARIA_128_GCM_SHA256 0xC060
#define TLS_ECDHE_RSA_WITH_ARIA_256_GCM_SHA384 0xC061
#define TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 0xCCA8
#define TLS_ECDH_ECDSA_WITH_NULL_SHA 0xC001
#define TLS_ECDH_ECDSA_WITH_RC4_128_SHA 0xC002
#define TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA 0xC003
#define TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA 0xC004
#define TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA 0xC005
#define TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256 0xC025
#define TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 0xC026
#define TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256 0xC02D
#define TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384 0xC02E
#define TLS_ECDH_ECDSA_WITH_CAMELLIA_128_CBC_SHA256 0xC074
#define TLS_ECDH_ECDSA_WITH_CAMELLIA_256_CBC_SHA384 0xC075
#define TLS_ECDH_ECDSA_WITH_CAMELLIA_128_GCM_SHA256 0xC088
#define TLS_ECDH_ECDSA_WITH_CAMELLIA_256_GCM_SHA384 0xC089
#define TLS_ECDH_ECDSA_WITH_ARIA_128_CBC_SHA256 0xC04A
#define TLS_ECDH_ECDSA_WITH_ARIA_256_CBC_SHA384 0xC04B
#define TLS_ECDH_ECDSA_WITH_ARIA_128_GCM_SHA256 0xC05E
#define TLS_ECDH_ECDSA_WITH_ARIA_256_GCM_SHA384 0xC05F
#define TLS_ECDHE_ECDSA_WITH_NULL_SHA 0xC006
#define TLS_ECDHE_ECDSA_WITH_RC4_128_SHA 0xC007
#define TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA 0xC008
#define TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA 0xC009
#define TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA 0xC00A
#define TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 0xC023
#define TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 0xC024
#define TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 0xC02B
#define TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 0xC02C
#define TLS_ECDHE_ECDSA_WITH_AES_128_CCM 0xC0AC
#define TLS_ECDHE_ECDSA_WITH_AES_256_CCM 0xC0AD
#define TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8 0xC0AE
#define TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8 0xC0AF
#define TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_CBC_SHA256 0xC072
#define TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_CBC_SHA384 0xC073
#define TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_GCM_SHA256 0xC086
#define TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_GCM_SHA384 0xC087
#define TLS_ECDHE_ECDSA_WITH_ARIA_128_CBC_SHA256 0xC048
#define TLS_ECDHE_ECDSA_WITH_ARIA_256_CBC_SHA384 0xC049
#define TLS_ECDHE_ECDSA_WITH_ARIA_128_GCM_SHA256 0xC05C
#define TLS_ECDHE_ECDSA_WITH_ARIA_256_GCM_SHA384 0xC05D
#define TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 0xCCA9
#define TLS_ECDH_ANON_WITH_NULL_SHA 0xC015
#define TLS_ECDH_ANON_WITH_RC4_128_SHA 0xC016
#define TLS_ECDH_ANON_WITH_3DES_EDE_CBC_SHA 0xC017
#define TLS_ECDH_ANON_WITH_AES_128_CBC_SHA 0xC018
#define TLS_ECDH_ANON_WITH_AES_256_CBC_SHA 0xC019
#define TLS_PSK_WITH_NULL_SHA 0x002C
#define TLS_PSK_WITH_NULL_SHA256 0x00B0
#define TLS_PSK_WITH_NULL_SHA384 0x00B1
#define TLS_PSK_WITH_RC4_128_SHA 0x008A
#define TLS_PSK_WITH_3DES_EDE_CBC_SHA 0x008B
#define TLS_PSK_WITH_AES_128_CBC_SHA 0x008C
#define TLS_PSK_WITH_AES_256_CBC_SHA 0x008D
#define TLS_PSK_WITH_AES_128_CBC_SHA256 0x00AE
#define TLS_PSK_WITH_AES_256_CBC_SHA384 0x00AF
#define TLS_PSK_WITH_AES_128_GCM_SHA256 0x00A8
#define TLS_PSK_WITH_AES_256_GCM_SHA384 0x00A9
#define TLS_PSK_WITH_AES_128_CCM 0xC0A4
#define TLS_PSK_WITH_AES_256_CCM 0xC0A5
#define TLS_PSK_WITH_AES_128_CCM_8 0xC0A8
#define TLS_PSK_WITH_AES_256_CCM_8 0xC0A9
#define TLS_PSK_WITH_CAMELLIA_128_CBC_SHA256 0xC094
#define TLS_PSK_WITH_CAMELLIA_256_CBC_SHA384 0xC095
#define TLS_PSK_WITH_CAMELLIA_128_GCM_SHA256 0xC08E
#define TLS_PSK_WITH_CAMELLIA_256_GCM_SHA384 0xC08F
#define TLS_PSK_WITH_ARIA_128_CBC_SHA256 0xC064
#define TLS_PSK_WITH_ARIA_256_CBC_SHA384 0xC065
#define TLS_PSK_WITH_ARIA_128_GCM_SHA256 0xC06A
#define TLS_PSK_WITH_ARIA_256_GCM_SHA384 0xC06B
#define TLS_PSK_WITH_CHACHA20_POLY1305_SHA256 0xCCAB
#define TLS_RSA_PSK_WITH_NULL_SHA 0x002E
#define TLS_RSA_PSK_WITH_NULL_SHA256 0x00B8
#define TLS_RSA_PSK_WITH_NULL_SHA384 0x00B9
#define TLS_RSA_PSK_WITH_RC4_128_SHA 0x0092
#define TLS_RSA_PSK_WITH_3DES_EDE_CBC_SHA 0x0093
#define TLS_RSA_PSK_WITH_AES_128_CBC_SHA 0x0094
#define TLS_RSA_PSK_WITH_AES_256_CBC_SHA 0x0095
#define TLS_RSA_PSK_WITH_AES_128_CBC_SHA256 0x00B6
#define TLS_RSA_PSK_WITH_AES_256_CBC_SHA384 0x00B7
#define TLS_RSA_PSK_WITH_AES_128_GCM_SHA256 0x00AC
#define TLS_RSA_PSK_WITH_AES_256_GCM_SHA384 0x00AD
#define TLS_RSA_PSK_WITH_CAMELLIA_128_CBC_SHA256 0xC098
#define TLS_RSA_PSK_WITH_CAMELLIA_256_CBC_SHA384 0xC099
#define TLS_RSA_PSK_WITH_CAMELLIA_128_GCM_SHA256 0xC092
#define TLS_RSA_PSK_WITH_CAMELLIA_256_GCM_SHA384 0xC093
#define TLS_RSA_PSK_WITH_ARIA_128_CBC_SHA256 0xC068
#define TLS_RSA_PSK_WITH_ARIA_256_CBC_SHA384 0xC069
#define TLS_RSA_PSK_WITH_ARIA_128_GCM_SHA256 0xC06E
#define TLS_RSA_PSK_WITH_ARIA_256_GCM_SHA384 0xC06F
#define TLS_RSA_PSK_WITH_CHACHA20_POLY1305_SHA256 0xCCAE
#define TLS_DHE_PSK_WITH_NULL_SHA 0x002D
#define TLS_DHE_PSK_WITH_NULL_SHA256 0x00B4
#define TLS_DHE_PSK_WITH_NULL_SHA384 0x00B5
#define TLS_DHE_PSK_WITH_RC4_128_SHA 0x008E
#define TLS_DHE_PSK_WITH_3DES_EDE_CBC_SHA 0x008F
#define TLS_DHE_PSK_WITH_AES_128_CBC_SHA 0x0090
#define TLS_DHE_PSK_WITH_AES_256_CBC_SHA 0x0091
#define TLS_DHE_PSK_WITH_AES_128_CBC_SHA256 0x00B2
#define TLS_DHE_PSK_WITH_AES_256_CBC_SHA384 0x00B3
#define TLS_DHE_PSK_WITH_AES_128_GCM_SHA256 0x00AA
#define TLS_DHE_PSK_WITH_AES_256_GCM_SHA384 0x00AB
#define TLS_DHE_PSK_WITH_AES_128_CCM 0xC0A6
#define TLS_DHE_PSK_WITH_AES_256_CCM 0xC0A7
#define TLS_DHE_PSK_WITH_AES_128_CCM_8 0xC0AA
#define TLS_DHE_PSK_WITH_AES_256_CCM_8 0xC0AB
#define TLS_DHE_PSK_WITH_CAMELLIA_128_CBC_SHA256 0xC096
#define TLS_DHE_PSK_WITH_CAMELLIA_256_CBC_SHA384 0xC097
#define TLS_DHE_PSK_WITH_CAMELLIA_128_GCM_SHA256 0xC090
#define TLS_DHE_PSK_WITH_CAMELLIA_256_GCM_SHA384 0xC091
#define TLS_DHE_PSK_WITH_ARIA_128_CBC_SHA256 0xC066
#define TLS_DHE_PSK_WITH_ARIA_256_CBC_SHA384 0xC067
#define TLS_DHE_PSK_WITH_ARIA_128_GCM_SHA256 0xC06C
#define TLS_DHE_PSK_WITH_ARIA_256_GCM_SHA384 0xC06D
#define TLS_DHE_PSK_WITH_CHACHA20_POLY1305_SHA256 0xCCAD
#define TLS_ECDHE_PSK_WITH_NULL_SHA 0xC039
#define TLS_ECDHE_PSK_WITH_NULL_SHA256 0xC03A
#define TLS_ECDHE_PSK_WITH_NULL_SHA384 0xC03B
#define TLS_ECDHE_PSK_WITH_RC4_128_SHA 0xC033
#define TLS_ECDHE_PSK_WITH_3DES_EDE_CBC_SHA 0xC034
#define TLS_ECDHE_PSK_WITH_AES_128_CBC_SHA 0xC035
#define TLS_ECDHE_PSK_WITH_AES_256_CBC_SHA 0xC036
#define TLS_ECDHE_PSK_WITH_AES_128_CBC_SHA256 0xC037
#define TLS_ECDHE_PSK_WITH_AES_256_CBC_SHA384 0xC038
#define TLS_ECDHE_PSK_WITH_AES_128_GCM_SHA256 0xD001
#define TLS_ECDHE_PSK_WITH_AES_256_GCM_SHA384 0xD002
#define TLS_ECDHE_PSK_WITH_AES_128_CCM_SHA256 0xD005
#define TLS_ECDHE_PSK_WITH_AES_128_CCM_8_SHA256 0xD003
#define TLS_ECDHE_PSK_WITH_CAMELLIA_128_CBC_SHA256 0xC09A
#define TLS_ECDHE_PSK_WITH_CAMELLIA_256_CBC_SHA384 0xC09B
#define TLS_ECDHE_PSK_WITH_ARIA_128_CBC_SHA256 0xC070
#define TLS_ECDHE_PSK_WITH_ARIA_256_CBC_SHA384 0xC071
#define TLS_ECDHE_PSK_WITH_CHACHA20_POLY1305_SHA256 0xCCAC
#define TLS_KRB5_EXPORT_WITH_RC4_40_MD5 0x002B
#define TLS_KRB5_EXPORT_WITH_RC4_40_SHA 0x0028
#define TLS_KRB5_EXPORT_WITH_RC2_CBC_40_MD5 0x002A
#define TLS_KRB5_EXPORT_WITH_RC2_CBC_40_SHA 0x0027
#define TLS_KRB5_EXPORT_WITH_DES_CBC_40_MD5 0x0029
#define TLS_KRB5_EXPORT_WITH_DES_CBC_40_SHA 0x0026
#define TLS_KRB5_WITH_RC4_128_MD5 0x0024
#define TLS_KRB5_WITH_RC4_128_SHA 0x0020
#define TLS_KRB5_WITH_IDEA_CBC_MD5 0x0025
#define TLS_KRB5_WITH_IDEA_CBC_SHA 0x0021
#define TLS_KRB5_WITH_DES_CBC_MD5 0x0022
#define TLS_KRB5_WITH_DES_CBC_SHA 0x001E
#define TLS_KRB5_WITH_3DES_EDE_CBC_MD5 0x0023
#define TLS_KRB5_WITH_3DES_EDE_CBC_SHA 0x001F
#define TLS_SRP_SHA_WITH_3DES_EDE_CBC_SHA 0xC01A
#define TLS_SRP_SHA_WITH_AES_128_CBC_SHA 0xC01D
#define TLS_SRP_SHA_WITH_AES_256_CBC_SHA 0xC020
#define TLS_SRP_SHA_RSA_WITH_3DES_EDE_CBC_SHA 0xC01B
#define TLS_SRP_SHA_RSA_WITH_AES_128_CBC_SHA 0xC01E
#define TLS_SRP_SHA_RSA_WITH_AES_256_CBC_SHA 0xC021
#define TLS_SRP_SHA_DSS_WITH_3DES_EDE_CBC_SHA 0xC01C
#define TLS_SRP_SHA_DSS_WITH_AES_128_CBC_SHA 0xC01F
#define TLS_SRP_SHA_DSS_WITH_AES_256_CBC_SHA 0xC022
#define TLS_ECCPWD_WITH_AES_128_GCM_SHA256 0xC0B0
#define TLS_ECCPWD_WITH_AES_256_GCM_SHA384 0xC0B1
#define TLS_ECCPWD_WITH_AES_128_CCM_SHA256 0xC0B2
#define TLS_ECCPWD_WITH_AES_256_CCM_SHA384 0xC0B3
#define TLS_GOSTR341112_256_WITH_KUZNYECHIK_CTR_OMAC 0xC100
#define TLS_GOSTR341112_256_WITH_MAGMA_CTR_OMAC 0xC101
#define TLS_GOSTR341112_256_WITH_28147_CNT_IMIT 0xC102
#define TLS_AES_128_GCM_SHA256 0x1301
#define TLS_AES_256_GCM_SHA384 0x1302
#define TLS_AES_128_CCM_SHA256 0x1304
#define TLS_AES_128_CCM_8_SHA256 0x1305
#define TLS_CHACHA20_POLY1305_SHA256 0x1303
#define TLS_SM4_GCM_SM3 0x00C6
#define TLS_SM4_CCM_SM3 0x00C7
#define TLS_SHA256_SHA256 0xC0B4
#define TLS_SHA384_SHA384 0xC0B5
#define TLS_EMPTY_RENEGOTIATION_INFO_SCSV 0x00FF
#define TLS_FALLBACK_SCSV 0x5600
/* https://datatracker.ietf.org/doc/html/rfc8701 */
#define TLS_CIPHER_GREASE_RESERVED_0 0x0A0A
#define TLS_CIPHER_GREASE_RESERVED_1 0x1A1A
#define TLS_CIPHER_GREASE_RESERVED_2 0x2A2A
#define TLS_CIPHER_GREASE_RESERVED_3 0x3A3A
#define TLS_CIPHER_GREASE_RESERVED_4 0x4A4A
#define TLS_CIPHER_GREASE_RESERVED_5 0x5A5A
#define TLS_CIPHER_GREASE_RESERVED_6 0x6A6A
#define TLS_CIPHER_GREASE_RESERVED_7 0x7A7A
#define TLS_CIPHER_GREASE_RESERVED_8 0x8A8A
#define TLS_CIPHER_GREASE_RESERVED_9 0x9A9A
#define TLS_CIPHER_GREASE_RESERVED_A 0xAAAA
#define TLS_CIPHER_GREASE_RESERVED_B 0xBABA
#define TLS_CIPHER_GREASE_RESERVED_C 0xCACA
#define TLS_CIPHER_GREASE_RESERVED_D 0xDADA
#define TLS_CIPHER_GREASE_RESERVED_E 0xEAEA
#define TLS_CIPHER_GREASE_RESERVED_F 0xFAFA
/* ********************************************** */
/* Signature algorithms */
#define RSA_PKCS1_SHA1 0x0201
#define ECDSA_SHA1 0x0203
#define RSA_PKCS1_SHA256 0x0401
#define ECDSA_SECP256R1_SHA256 0x0403
#define RSA_PKCS1_SHA384 0x0501
#define ECDSA_SECP384R1_SHA384 0x0503
#define RSA_PKCS1_SHA512 0x0601
#define ECDSA_SECP521R1_SHA512 0x0603
#define RSA_PSS_RSAE_SHA256 0x0804
#define RSA_PSS_RSAE_SHA384 0x0805
#define RSA_PSS_RSAE_SHA512 0x0806
#define ED25519 0x0807
#define ED448 0x0808
#define RSA_PSS_PSS_SHA256 0x0809
#define RSA_PSS_PSS_SHA384 0x080A
#define RSA_PSS_PSS_SHA512 0x080B
+20 -16
View File
@@ -1,7 +1,7 @@
/*
* ndpi_main.h
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -31,6 +31,10 @@
#include "ndpi_api.h"
#include "ndpi_protocols.h"
/* used by ndpi_set_proto_subprotocols */
#define NDPI_PROTOCOL_NO_MORE_SUBPROTOCOLS (-1)
#define NDPI_PROTOCOL_MATCHED_BY_CONTENT (-2)
#ifdef __cplusplus
extern "C" {
#endif
@@ -95,11 +99,11 @@ extern "C" {
struct ndpi_flow_struct *flow,
ndpi_protocol_category_t protocol_category);
extern void ndpi_set_proto_subprotocols(struct ndpi_detection_module_struct *ndpi_mod,
int protoId, ...);
extern void ndpi_set_proto_defaults(struct ndpi_detection_module_struct *ndpi_mod,
ndpi_protocol_breed_t protoBreed, u_int16_t protoId,
u_int8_t can_have_a_subprotocol,
u_int16_t tcp_alias_protoId[2],
u_int16_t udp_alias_protoId[2], char *protoName,
ndpi_protocol_breed_t protoBreed, u_int16_t protoId, char *protoName,
ndpi_protocol_category_t protoCategory,
ndpi_port_range *tcpDefPorts,
ndpi_port_range *udpDefPorts);
@@ -127,13 +131,7 @@ extern "C" {
extern u_int8_t ndpi_is_proto(ndpi_protocol proto, u_int16_t p);
extern u_int16_t ndpi_get_lower_proto(ndpi_protocol p);
extern int ndpi_get_protocol_id_master_proto(struct ndpi_detection_module_struct *ndpi_struct,
u_int16_t protocol_id,
u_int16_t** tcp_master_proto,
u_int16_t** udp_master_proto);
#/* NDPI_PROTOCOL_NETBIOS */
int ndpi_netbios_name_interpret(char *in, size_t inlen, char *out, u_int out_len);
#ifdef NDPI_ENABLE_DEBUG_MESSAGES
void ndpi_debug_get_last_log_function_line(struct ndpi_detection_module_struct *ndpi_struct,
const char **file, const char **func, u_int32_t * line);
@@ -150,10 +148,16 @@ extern "C" {
#define ndpi_match_strprefix(payload, payload_len, str) \
ndpi_match_prefix((payload), (payload_len), (str), (sizeof(str)-1))
#ifdef NDPI_DETECTION_SUPPORT_IPV6
int ndpi_handle_ipv6_extension_headers(struct ndpi_detection_module_struct *ndpi_str, const u_int8_t ** l4ptr, u_int16_t * l4len, u_int8_t * nxt_hdr);
#endif
int ndpi_handle_ipv6_extension_headers(struct ndpi_detection_module_struct *ndpi_str,
const u_int8_t ** l4ptr, u_int16_t * l4len,
u_int8_t * nxt_hdr);
void ndpi_set_risk(struct ndpi_detection_module_struct *ndpi_str,
struct ndpi_flow_struct *flow, ndpi_risk_enum r);
int ndpi_is_printable_string(char const * const str, size_t len);
float ndpi_calculate_entropy(u_int8_t const * const buf, size_t len);
void load_common_alpns(struct ndpi_detection_module_struct *ndpi_str);
u_int8_t is_a_common_alpn(struct ndpi_detection_module_struct *ndpi_str,
const char *alpn_to_check, u_int alpn_to_check_len);
#ifdef __cplusplus
}
#endif
+107
View File
@@ -0,0 +1,107 @@
/*
* ndpi_patricia_typedef.h
*
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
/*
* $Id: ndpi_patricia.h,v 1.6 2005/12/07 20:53:01 dplonka Exp $
* Dave Plonka <plonka@doit.wisc.edu>
*
* This product includes software developed by the University of Michigan,
* Merit Network, Inc., and their contributors.
*
* This file had been called "radix.h" in the MRT sources.
*
* I renamed it to "ndpi_patricia.h" since it's not an implementation of a general
* radix trie. Also, pulled in various requirements from "mrt.h" and added
* some other things it could be used as a standalone API.
https://github.com/deepfield/MRT/blob/master/COPYRIGHT
Copyright (c) 1999-2013
The Regents of the University of Michigan ("The Regents") and Merit
Network, Inc.
Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef _NDPI_PATRICIA_TYPEDEF_H_
#define _NDPI_PATRICIA_TYPEDEF_H_
/* pointer to usr data (ex. route flap info) */
union ndpi_patricia_node_value_t {
void *user_data;
/* User-defined values */
union {
struct {
u_int32_t user_value, additional_user_value;
} uv32;
u_int64_t uv64;
} u;
};
typedef struct _ndpi_prefix_t {
u_int16_t family; /* AF_INET | AF_INET6 */
u_int16_t bitlen; /* same as mask? */
int ref_count; /* reference count */
union {
struct in_addr sin;
struct in6_addr sin6;
u_int8_t mac[6];
} add;
} ndpi_prefix_t;
typedef struct _ndpi_patricia_node_t {
u_int16_t bit; /* flag if this node used */
ndpi_prefix_t *prefix; /* who we are in patricia tree */
struct _ndpi_patricia_node_t *l, *r; /* left and right children */
struct _ndpi_patricia_node_t *parent;/* may be used */
void *data; /* pointer to data */
union ndpi_patricia_node_value_t value;
} ndpi_patricia_node_t;
typedef struct _ndpi_patricia_tree_t {
ndpi_patricia_node_t *head;
u_int16_t maxbits; /* for IP, 32 bit addresses */
int num_active_node; /* for debug purpose */
} ndpi_patricia_tree_t;
#endif /* _NDPI_PATRICIA_TYPEDEF_H_ */
+27 -18
View File
@@ -1,7 +1,7 @@
/*
* ndpi_protocol_ids.h
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -25,7 +25,6 @@
#ifndef __NDPI_PROTOCOLS_IDS_H__
#define __NDPI_PROTOCOLS_IDS_H__
#define NDPI_DETECTION_SUPPORT_IPV6
#define NDPI_PROTOCOL_SIZE 2
typedef enum {
@@ -59,7 +58,7 @@ typedef enum {
NDPI_PROTOCOL_COAP = 27,
NDPI_PROTOCOL_VMWARE = 28,
NDPI_PROTOCOL_MAIL_SMTPS = 29,
NDPI_PROTOCOL_FBZERO = 30,
NDPI_PROTOCOL_DTLS = 30,
NDPI_PROTOCOL_UBNTAC2 = 31, /* Ubiquity UBNT AirControl = 2 - Thomas Fjellstrom <thomas+ndpi@fjellstrom.ca> */
NDPI_PROTOCOL_KONTIKI = 32,
NDPI_PROTOCOL_OPENFT = 33,
@@ -82,25 +81,25 @@ typedef enum {
NDPI_PROTOCOL_RTSP = 50,
NDPI_PROTOCOL_MAIL_IMAPS = 51,
NDPI_PROTOCOL_ICECAST = 52,
NDPI_PROTOCOL_FREE_53 = 53,
NDPI_PROTOCOL_CPHA = 53,
NDPI_PROTOCOL_PPSTREAM = 54,
NDPI_PROTOCOL_ZATTOO = 55,
NDPI_PROTOCOL_SHOUTCAST = 56,
NDPI_PROTOCOL_SOPCAST = 57,
NDPI_PROTOCOL_DISCORD = 58,
NDPI_PROTOCOL_TVUPLAYER = 59,
NDPI_PROTOCOL_HTTP_DOWNLOAD = 60,
NDPI_PROTOCOL_MONGODB = 60, /* Leonn Paiva <leonn.paiva@gmail.com> */
NDPI_PROTOCOL_QQLIVE = 61,
NDPI_PROTOCOL_THUNDER = 62,
NDPI_PROTOCOL_SOULSEEK = 63,
NDPI_PROTOCOL_PS_VUE = 64,
NDPI_PROTOCOL_IRC = 65,
NDPI_PROTOCOL_AYIYA = 66,
NDPI_PROTOCOL_UNENCRYPTED_JABBER = 67,
NDPI_PROTOCOL_JABBER = 67,
NDPI_PROTOCOL_NATS = 68,
NDPI_PROTOCOL_FREE_69 = 69, /* Free */
NDPI_PROTOCOL_AMONG_US = 69, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_YAHOO = 70,
NDPI_PROTOCOL_FREE_71 = 71, /* Free */
NDPI_PROTOCOL_DISNEYPLUS = 71, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_GOOGLE_PLUS = 72,
NDPI_PROTOCOL_IP_VRRP = 73,
NDPI_PROTOCOL_STEAM = 74, /* Tomasz Bujlow <tomasz@skatnet.dk> */
@@ -119,7 +118,7 @@ typedef enum {
NDPI_PROTOCOL_RTP = 87,
NDPI_PROTOCOL_RDP = 88,
NDPI_PROTOCOL_VNC = 89,
NDPI_PROTOCOL_FREE90 = 90, /* Free */
NDPI_PROTOCOL_TUMBLR = 90, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_TLS = 91,
NDPI_PROTOCOL_SSH = 92,
NDPI_PROTOCOL_USENET = 93,
@@ -139,7 +138,7 @@ typedef enum {
NDPI_PROTOCOL_FIESTA = 107,
NDPI_PROTOCOL_FLORENSIA = 108,
NDPI_PROTOCOL_GUILDWARS = 109,
NDPI_PROTOCOL_HTTP_ACTIVESYNC = 110,
NDPI_PROTOCOL_AMAZON_ALEXA = 110, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_KERBEROS = 111,
NDPI_PROTOCOL_LDAP = 112,
NDPI_PROTOCOL_MAPLESTORY = 113,
@@ -154,7 +153,7 @@ typedef enum {
NDPI_PROTOCOL_GMAIL = 122,
NDPI_PROTOCOL_GOOGLE_MAPS = 123,
NDPI_PROTOCOL_YOUTUBE = 124,
NDPI_PROTOCOL_SKYPE = 125,
NDPI_PROTOCOL_SKYPE_TEAMS = 125,
NDPI_PROTOCOL_GOOGLE = 126,
NDPI_PROTOCOL_DCERPC = 127,
NDPI_PROTOCOL_NETFLOW = 128,
@@ -182,7 +181,7 @@ typedef enum {
NDPI_PROTOCOL_LOTUS_NOTES = 150,
NDPI_PROTOCOL_SAP = 151,
NDPI_PROTOCOL_GTP = 152,
NDPI_PROTOCOL_UPNP = 153,
NDPI_PROTOCOL_WSD = 153,
NDPI_PROTOCOL_LLMNR = 154,
NDPI_PROTOCOL_REMOTE_SCAN = 155,
NDPI_PROTOCOL_SPOTIFY = 156,
@@ -212,7 +211,7 @@ typedef enum {
NDPI_PROTOCOL_CNN = 180, /* Tomasz Bujlow <tomasz@skatnet.dk> */
NDPI_PROTOCOL_MEGACO = 181, /* Gianluca Costa <g.costa@xplico.org> */
NDPI_PROTOCOL_REDIS = 182,
NDPI_PROTOCOL_FREE_183 = 183,
NDPI_PROTOCOL_PINTEREST = 183, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_VHUA = 184,
NDPI_PROTOCOL_TELEGRAM = 185, /* Gianluca Costa <g.costa@xplico.org> */
NDPI_PROTOCOL_VEVO = 186,
@@ -225,7 +224,7 @@ typedef enum {
NDPI_PROTOCOL_KAKAOTALK = 193, /* KakaoTalk Chat (no voice call) */
NDPI_PROTOCOL_KAKAOTALK_VOICE = 194, /* KakaoTalk Voice */
NDPI_PROTOCOL_TWITCH = 195, /* Edoardo Dominici <edoaramis@gmail.com> */
NDPI_PROTOCOL_DOH_DOT = 196, /* DoH (DNS over HTTPS), DoT (DNS over TLS) */
NDPI_PROTOCOL_DOH_DOT = 196, /* DoH (DNS over HTTPS), DoT (DNS over TLS), DoQ (DNS over QUIC). TODO: rename in NDPI_PROTOCOL_DOH_DOT_DOQ? */
NDPI_PROTOCOL_WECHAT = 197,
NDPI_PROTOCOL_MPEGTS = 198,
NDPI_PROTOCOL_SNAPCHAT = 199,
@@ -234,7 +233,7 @@ typedef enum {
NDPI_PROTOCOL_IFLIX = 202, /* www.vizuamatix.com R&D team & M.Mallawaarachchie <manoj_ws@yahoo.com> */
NDPI_PROTOCOL_GITHUB = 203,
NDPI_PROTOCOL_BJNP = 204,
NDPI_PROTOCOL_FREE_205 = 205,
NDPI_PROTOCOL_REDDIT = 205, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_WIREGUARD = 206,
NDPI_PROTOCOL_SMPP = 207, /* Damir Franusic <df@release14.org> */
NDPI_PROTOCOL_DNSCRYPT = 208, /* Toni Uhlig <matzeton@googlemail.com> */
@@ -265,8 +264,8 @@ typedef enum {
NDPI_PROTOCOL_LINKEDIN = 233, /* Paulo Angelo <pa@pauloangelo.com> */
NDPI_PROTOCOL_SOUNDCLOUD = 234,
NDPI_PROTOCOL_CSGO = 235, /* Counter-Strike Global Offensive, Dota = 2 */
NDPI_PROTOCOL_LISP = 236,
NDPI_PROTOCOL_DIAMETER = 237,
NDPI_PROTOCOL_LISP = 236,
NDPI_PROTOCOL_DIAMETER = 237,
NDPI_PROTOCOL_APPLE_PUSH = 238,
NDPI_PROTOCOL_GOOGLE_SERVICES = 239,
NDPI_PROTOCOL_AMAZON_VIDEO = 240,
@@ -283,10 +282,20 @@ typedef enum {
NDPI_PROTOCOL_WEBSOCKET = 251, /* Leonn Paiva <leonn.paiva@gmail.com> */
NDPI_PROTOCOL_ANYDESK = 252, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_SOAP = 253, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_APPLE_SIRI = 254, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_SNAPCHAT_CALL = 255,
NDPI_PROTOCOL_HPVIRTGRP = 256, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_GENSHIN_IMPACT = 257, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_ACTIVISION = 258,
NDPI_PROTOCOL_FORTICLIENT = 259,
NDPI_PROTOCOL_Z3950 = 260, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_LIKEE = 261,
NDPI_PROTOCOL_GITLAB = 262,
NDPI_PROTOCOL_AVAST_SECUREDNS = 263, /* Toni Uhlig <matzeton@googlemail.com> */
#ifdef CUSTOM_NDPI_PROTOCOLS
#include "../../../nDPI-custom/custom_ndpi_protocol_ids.h"
#endif
#endif
/*
IMPORTANT
+9 -5
View File
@@ -1,7 +1,7 @@
/*
* ndpi_protocols.h
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -87,7 +87,6 @@ void init_guildwars_dissector(struct ndpi_detection_module_struct *ndpi_struct,
void init_h323_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_halflife2_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_http_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_http_activesync_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_iax_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_icecast_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_ipp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
@@ -160,10 +159,9 @@ void init_telegram_dissector(struct ndpi_detection_module_struct *ndpi_struct, u
void init_telnet_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_tftp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_thunder_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_tor_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_tvuplayer_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_usenet_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_upnp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_wsd_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_veohtv_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_vhua_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_viber_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
@@ -193,11 +191,11 @@ void init_fix_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int3
void init_nintendo_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_csgo_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_checkmk_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_cpha_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_apple_push_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_amazon_video_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_whatsapp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_ajp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_fbzero_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_memcached_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_nest_log_sink_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_ookla_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
@@ -212,5 +210,11 @@ void init_s7comm_dissector(struct ndpi_detection_module_struct *ndpi_struct,u_in
void init_websocket_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_soap_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_dnscrypt_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_mongodb_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_among_us_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_hpvirtgrp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_genshin_impact_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_z3950_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_avast_securedns_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
#endif /* __NDPI_PROTOCOLS_H__ */
+365 -142
View File
@@ -1,7 +1,7 @@
/*
* ndpi_typedefs.h
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -26,6 +26,14 @@
#include "ndpi_define.h"
#include "ndpi_protocol_ids.h"
#include "ndpi_utils.h"
/* Used by both nDPI core and patricia code under third-party */
#include "ndpi_patricia_typedefs.h"
#ifdef HAVE_MAXMINDDB
#include <maxminddb.h>
#endif
/* NDPI_LOG_LEVEL */
typedef enum {
@@ -48,14 +56,17 @@ typedef enum {
ndpi_capwap_tunnel,
ndpi_tzsp_tunnel,
ndpi_l2tp_tunnel,
ndpi_vxlan_tunnel,
} ndpi_packet_tunnel;
/*
NOTE
When the typedef below is modified don't forget to update
- nDPI/wireshark/ndpi.lua
- ndpi_risk2str (in ndpi_utils.c)
- https://github.com/ntop/ntopng/blob/dev/scripts/lua/modules/flow_risk_utils.lua
- ndpi_risk_enum (in python/ndpi.py)
- ndpi_known_risks (ndpi_utils.c)
*/
typedef enum {
NDPI_NO_RISK = 0,
@@ -83,12 +94,50 @@ typedef enum {
NDPI_UNSAFE_PROTOCOL,
NDPI_DNS_SUSPICIOUS_TRAFFIC,
NDPI_TLS_MISSING_SNI,
NDPI_HTTP_SUSPICIOUS_CONTENT,
NDPI_RISKY_ASN,
NDPI_RISKY_DOMAIN,
NDPI_MALICIOUS_JA3,
NDPI_MALICIOUS_SHA1_CERTIFICATE,
NDPI_DESKTOP_OR_FILE_SHARING_SESSION, /* 30 */
NDPI_TLS_UNCOMMON_ALPN,
NDPI_TLS_CERT_VALIDITY_TOO_LONG,
NDPI_TLS_EXTENSION_SUSPICIOUS,
/* Leave this as last member */
NDPI_MAX_RISK /* must be <= 31 due to (**) */
NDPI_MAX_RISK /* must be <= 63 due to (**) */
} ndpi_risk_enum;
typedef u_int32_t ndpi_risk; /* (**) */
typedef u_int64_t ndpi_risk; /* (**) */
typedef enum {
NDPI_RISK_LOW,
NDPI_RISK_MEDIUM,
NDPI_RISK_HIGH,
NDPI_RISK_SEVERE
} ndpi_risk_severity;
typedef enum {
NDPI_SCORE_RISK_LOW = 10,
NDPI_SCORE_RISK_MEDIUM = 50,
NDPI_SCORE_RISK_HIGH = 100,
NDPI_SCORE_RISK_SEVERE = 250,
} ndpi_risk_score;
typedef enum {
CLIENT_NO_RISK_PERCENTAGE = 0, /* 100% server risk */
CLIENT_LOW_RISK_PERCENTAGE = 10, /* 90% server risk */
CLIENT_FAIR_RISK_PERCENTAGE = 50, /* 50% server risk */
CLIENT_HIGH_RISK_PERCENTAGE = 90, /* 10% server risk */
CLIENT_FULL_RISK_PERCENTAGE = 100 /* 0% server risk */
} risk_percentage;
typedef struct {
ndpi_risk_enum risk;
ndpi_risk_severity severity;
risk_percentage default_client_risk_pctg; /* 0-100 */
} ndpi_risk_info;
/* NDPI_VISIT */
typedef enum {
@@ -375,10 +424,7 @@ struct ndpi_dns_packet_header {
typedef union
{
u_int32_t ipv4;
u_int8_t ipv4_u_int8_t[4];
#ifdef NDPI_DETECTION_SUPPORT_IPV6
struct ndpi_in6_addr ipv6;
#endif
} ndpi_ip_addr_t;
@@ -430,6 +476,12 @@ struct ndpi_vxlanhdr {
/* ******************* ********************* ****************** */
/* ************************************************************ */
typedef struct message {
u_int8_t *buffer;
u_int buffer_len, buffer_used, max_expected;
u_int32_t next_seq[2]; /* Directions */
} message_t;
/* NDPI_PROTOCOL_BITTORRENT */
typedef struct spinlock {
volatile int val;
@@ -479,8 +531,8 @@ PACK_ON struct tinc_cache_entry {
u_int16_t dst_port;
} PACK_OFF;
/*
In case the typedef below is modified, please update
/*
In case the typedef below is modified, please update
ndpi_http_method2str (ndpi_utils.c)
*/
typedef enum {
@@ -500,7 +552,7 @@ struct ndpi_lru_cache_entry {
u_int32_t key; /* Store the whole key to avoid ambiguities */
u_int32_t is_full:1, value:16, pad:15;
};
struct ndpi_lru_cache {
u_int32_t num_entries;
struct ndpi_lru_cache_entry *entries;
@@ -539,7 +591,7 @@ struct ndpi_id_struct {
/* NDPI_PROTOCOL_ZATTOO */
u_int32_t zattoo_ts;
/* NDPI_PROTOCOL_UNENCRYPTED_JABBER */
/* NDPI_PROTOCOL_JABBER */
u_int32_t jabber_stun_or_ft_ts;
/* NDPI_PROTOCOL_DIRECTCONNECT */
@@ -558,7 +610,7 @@ struct ndpi_id_struct {
u_int16_t bt_port_t[NDPI_BT_PORTS];
u_int16_t bt_port_u[NDPI_BT_PORTS];
/* NDPI_PROTOCOL_UNENCRYPTED_JABBER */
/* NDPI_PROTOCOL_JABBER */
#define JABBER_MAX_STUN_PORTS 6
u_int16_t jabber_voice_stun_port[JABBER_MAX_STUN_PORTS];
u_int16_t jabber_file_transfer_port[2];
@@ -576,7 +628,7 @@ struct ndpi_id_struct {
/* NDPI_PROTOCOL_IRC */
u_int8_t irc_number_of_port;
/* NDPI_PROTOCOL_UNENCRYPTED_JABBER */
/* NDPI_PROTOCOL_JABBER */
u_int8_t jabber_voice_stun_used_ports;
/* NDPI_PROTOCOL_SIP */
@@ -667,20 +719,17 @@ struct ndpi_flow_tcp_struct {
u_int32_t telnet_stage:2; // 0 - 2
struct {
struct {
u_int8_t *buffer;
u_int buffer_len, buffer_used;
} message;
message_t message;
void* srv_cert_fingerprint_ctx; /* SHA-1 */
/* NDPI_PROTOCOL_TLS */
u_int8_t hello_processed:1, certificate_processed:1, subprotocol_detected:1,
fingerprint_set:1, _pad:4;
u_int8_t sha1_certificate_fingerprint[20], num_tls_blocks;
u_int8_t num_tls_blocks;
int16_t tls_application_blocks_len[NDPI_MAX_NUM_TLS_APPL_BLOCKS]; /* + = src->dst, - = dst->src */
} tls;
/* NDPI_PROTOCOL_POSTGRES */
u_int32_t postgres_stage:3;
@@ -762,7 +811,7 @@ struct ndpi_flow_udp_struct {
u_int32_t halflife2_stage:2; // 0 - 2
/* NDPI_PROTOCOL_TFTP */
u_int32_t tftp_stage:1;
u_int32_t tftp_stage:2;
/* NDPI_PROTOCOL_AIMINI */
u_int32_t aimini_stage:5;
@@ -775,6 +824,7 @@ struct ndpi_flow_udp_struct {
/* NDPI_PROTOCOL_SKYPE */
u_int8_t skype_packet_id;
u_int8_t skype_crc[4];
/* NDPI_PROTOCOL_TEAMVIEWER */
u_int8_t teamviewer_stage;
@@ -793,10 +843,32 @@ struct ndpi_flow_udp_struct {
/* NDPI_PROTOCOL_WIREGUARD */
u_int8_t wireguard_stage;
u_int32_t wireguard_peer_index[2];
/* NDPI_PROTOCOL_QUIC */
u_int8_t *quic_reasm_buf;
u_int32_t quic_reasm_buf_len;
};
/* ************************************************** */
#define LINE_EQUALS(ndpi_int_one_line_struct, string_to_compare) \
((ndpi_int_one_line_struct).len == strlen(string_to_compare) && \
LINE_CMP(ndpi_int_one_line_struct, string_to_compare, strlen(string_to_compare)) == 1)
#define LINE_STARTS(ndpi_int_one_line_struct, string_to_compare) \
((ndpi_int_one_line_struct).len >= strlen(string_to_compare) && \
LINE_CMP(ndpi_int_one_line_struct, string_to_compare, strlen(string_to_compare)) == 1)
#define LINE_ENDS(ndpi_int_one_line_struct, string_to_compare) \
((ndpi_int_one_line_struct).len >= strlen(string_to_compare) && \
memcmp((ndpi_int_one_line_struct).ptr + \
((ndpi_int_one_line_struct).len - strlen(string_to_compare)), \
string_to_compare, strlen(string_to_compare)) == 0)
#define LINE_CMP(ndpi_int_one_line_struct, string_to_compare, string_to_compare_length) \
((ndpi_int_one_line_struct).ptr != NULL && \
memcmp((ndpi_int_one_line_struct).ptr, string_to_compare, string_to_compare_length) == 0)
struct ndpi_int_one_line_struct {
const u_int8_t *ptr;
u_int16_t len;
@@ -804,9 +876,7 @@ struct ndpi_int_one_line_struct {
struct ndpi_packet_struct {
const struct ndpi_iphdr *iph;
#ifdef NDPI_DETECTION_SUPPORT_IPV6
const struct ndpi_ipv6hdr *iphv6;
#endif
const struct ndpi_tcphdr *tcp;
const struct ndpi_udphdr *udp;
const u_int8_t *generic_l4_ptr; /* is set only for non tcp-udp traffic */
@@ -815,7 +885,6 @@ struct ndpi_packet_struct {
u_int64_t current_time_ms;
u_int16_t detected_protocol_stack[NDPI_PROTOCOL_SIZE];
u_int8_t detected_subprotocol_stack[NDPI_PROTOCOL_SIZE];
u_int16_t protocol_stack_info;
struct ndpi_int_one_line_struct line[NDPI_MAX_PARSE_LINES_PER_PACKET];
@@ -853,18 +922,18 @@ struct ndpi_packet_struct {
u_int8_t tls_certificate_detected:4, tls_certificate_num_checks:4;
u_int8_t packet_lines_parsed_complete:1,
packet_direction:1, empty_line_position_set:1, pad:5;
packet_direction:1, empty_line_position_set:1, http_check_content:1, pad:4;
};
struct ndpi_detection_module_struct;
struct ndpi_flow_struct;
struct ndpi_call_function_struct {
u_int16_t ndpi_protocol_id;
NDPI_PROTOCOL_BITMASK detection_bitmask;
NDPI_PROTOCOL_BITMASK excluded_protocol_bitmask;
NDPI_SELECTION_BITMASK_PROTOCOL_SIZE ndpi_selection_bitmask;
void (*func) (struct ndpi_detection_module_struct *, struct ndpi_flow_struct *flow);
NDPI_SELECTION_BITMASK_PROTOCOL_SIZE ndpi_selection_bitmask;
u_int16_t ndpi_protocol_id;
u_int8_t detection_feature;
};
@@ -877,105 +946,109 @@ typedef struct {
} ndpi_port_range;
typedef enum {
NDPI_PROTOCOL_SAFE = 0, /* Surely doesn't provide risks for the network. (e.g., a news site) */
NDPI_PROTOCOL_ACCEPTABLE, /* Probably doesn't provide risks, but could be malicious (e.g., Dropbox) */
NDPI_PROTOCOL_FUN, /* Pure fun protocol, which may be prohibited by the user policy (e.g., Netflix) */
NDPI_PROTOCOL_UNSAFE, /* Probably provides risks, but could be a normal traffic. Unencrypted protocols with clear pass should be here (e.g., telnet) */
NDPI_PROTOCOL_POTENTIALLY_DANGEROUS, /* Possibly dangerous (ex. Tor). */
NDPI_PROTOCOL_DANGEROUS, /* Surely is dangerous (ex. smbv1). Be prepared to troubles */
NDPI_PROTOCOL_TRACKER_ADS, /* Trackers, Advertisements... */
NDPI_PROTOCOL_UNRATED /* No idea, not implemented or impossible to classify */
NDPI_PROTOCOL_SAFE = 0, /* Surely doesn't provide risks for the network. (e.g., a news site) */
NDPI_PROTOCOL_ACCEPTABLE, /* Probably doesn't provide risks, but could be malicious (e.g., Dropbox) */
NDPI_PROTOCOL_FUN, /* Pure fun protocol, which may be prohibited by the user policy (e.g., Netflix) */
NDPI_PROTOCOL_UNSAFE, /* Probably provides risks, but could be a normal traffic. Unencrypted protocols with clear pass should be here (e.g., telnet) */
NDPI_PROTOCOL_POTENTIALLY_DANGEROUS, /* Possibly dangerous (ex. Tor). */
NDPI_PROTOCOL_DANGEROUS, /* Surely is dangerous (ex. smbv1). Be prepared to troubles */
NDPI_PROTOCOL_TRACKER_ADS, /* Trackers, Advertisements... */
NDPI_PROTOCOL_UNRATED /* No idea, not implemented or impossible to classify */
} ndpi_protocol_breed_t;
#define NUM_BREEDS (NDPI_PROTOCOL_UNRATED+1)
/* Abstract categories to group the protocols. */
typedef enum {
NDPI_PROTOCOL_CATEGORY_UNSPECIFIED = 0, /* For general services and unknown protocols */
NDPI_PROTOCOL_CATEGORY_MEDIA, /* Multimedia and streaming */
NDPI_PROTOCOL_CATEGORY_VPN, /* Virtual Private Networks */
NDPI_PROTOCOL_CATEGORY_MAIL, /* Protocols to send/receive/sync emails */
NDPI_PROTOCOL_CATEGORY_DATA_TRANSFER, /* AFS/NFS and similar protocols */
NDPI_PROTOCOL_CATEGORY_WEB, /* Web/mobile protocols and services */
NDPI_PROTOCOL_CATEGORY_SOCIAL_NETWORK, /* Social networks */
NDPI_PROTOCOL_CATEGORY_DOWNLOAD_FT, /* Download, FTP, file transfer/sharing */
NDPI_PROTOCOL_CATEGORY_GAME, /* Online games */
NDPI_PROTOCOL_CATEGORY_CHAT, /* Instant messaging */
NDPI_PROTOCOL_CATEGORY_VOIP, /* Real-time communications and conferencing */
NDPI_PROTOCOL_CATEGORY_DATABASE, /* Protocols for database communication */
NDPI_PROTOCOL_CATEGORY_REMOTE_ACCESS, /* Remote access and control */
NDPI_PROTOCOL_CATEGORY_CLOUD, /* Online cloud services */
NDPI_PROTOCOL_CATEGORY_NETWORK, /* Network infrastructure protocols */
NDPI_PROTOCOL_CATEGORY_COLLABORATIVE, /* Software for collaborative development, including Webmail */
NDPI_PROTOCOL_CATEGORY_RPC, /* High level network communication protocols */
NDPI_PROTOCOL_CATEGORY_STREAMING, /* Streaming protocols */
NDPI_PROTOCOL_CATEGORY_SYSTEM_OS, /* System/Operating System level applications */
NDPI_PROTOCOL_CATEGORY_SW_UPDATE, /* Software update */
NDPI_PROTOCOL_CATEGORY_UNSPECIFIED = 0, /* For general services and unknown protocols */
NDPI_PROTOCOL_CATEGORY_MEDIA, /* Multimedia and streaming */
NDPI_PROTOCOL_CATEGORY_VPN, /* Virtual Private Networks */
NDPI_PROTOCOL_CATEGORY_MAIL, /* Protocols to send/receive/sync emails */
NDPI_PROTOCOL_CATEGORY_DATA_TRANSFER, /* AFS/NFS and similar protocols */
NDPI_PROTOCOL_CATEGORY_WEB, /* Web/mobile protocols and services */
NDPI_PROTOCOL_CATEGORY_SOCIAL_NETWORK, /* Social networks */
NDPI_PROTOCOL_CATEGORY_DOWNLOAD_FT, /* Download, FTP, file transfer/sharing */
NDPI_PROTOCOL_CATEGORY_GAME, /* Online games */
NDPI_PROTOCOL_CATEGORY_CHAT, /* Instant messaging */
NDPI_PROTOCOL_CATEGORY_VOIP, /* Real-time communications and conferencing */
NDPI_PROTOCOL_CATEGORY_DATABASE, /* Protocols for database communication */
NDPI_PROTOCOL_CATEGORY_REMOTE_ACCESS, /* Remote access and control */
NDPI_PROTOCOL_CATEGORY_CLOUD, /* Online cloud services */
NDPI_PROTOCOL_CATEGORY_NETWORK, /* Network infrastructure protocols */
NDPI_PROTOCOL_CATEGORY_COLLABORATIVE, /* Software for collaborative development, including Webmail */
NDPI_PROTOCOL_CATEGORY_RPC, /* High level network communication protocols */
NDPI_PROTOCOL_CATEGORY_STREAMING, /* Streaming protocols */
NDPI_PROTOCOL_CATEGORY_SYSTEM_OS, /* System/Operating System level applications */
NDPI_PROTOCOL_CATEGORY_SW_UPDATE, /* Software update */
/* See #define NUM_CUSTOM_CATEGORIES */
NDPI_PROTOCOL_CATEGORY_CUSTOM_1, /* User custom category 1 */
NDPI_PROTOCOL_CATEGORY_CUSTOM_2, /* User custom category 2 */
NDPI_PROTOCOL_CATEGORY_CUSTOM_3, /* User custom category 3 */
NDPI_PROTOCOL_CATEGORY_CUSTOM_4, /* User custom category 4 */
NDPI_PROTOCOL_CATEGORY_CUSTOM_5, /* User custom category 5 */
/* Further categories... */
NDPI_PROTOCOL_CATEGORY_MUSIC,
NDPI_PROTOCOL_CATEGORY_VIDEO,
NDPI_PROTOCOL_CATEGORY_SHOPPING,
NDPI_PROTOCOL_CATEGORY_PRODUCTIVITY,
NDPI_PROTOCOL_CATEGORY_FILE_SHARING,
/*
The category below is used by sites who are used
to test connectivity
*/
NDPI_PROTOCOL_CATEGORY_CONNECTIVITY_CHECK,
NDPI_PROTOCOL_CATEGORY_IOT_SCADA,
/*
The category below is used for vocal assistance services.
*/
NDPI_PROTOCOL_CATEGORY_VIRTUAL_ASSISTANT,
/* Some custom categories */
CUSTOM_CATEGORY_MINING = 99,
CUSTOM_CATEGORY_MALWARE = 100,
CUSTOM_CATEGORY_ADVERTISEMENT = 101,
CUSTOM_CATEGORY_BANNED_SITE = 102,
CUSTOM_CATEGORY_SITE_UNAVAILABLE = 103,
CUSTOM_CATEGORY_ALLOWED_SITE = 104,
/*
The category below is used to track communications made by
security applications (e.g. sophosxl.net, spamhaus.org)
to track malware, spam etc.
*/
CUSTOM_CATEGORY_ANTIMALWARE = 105,
/*
IMPORTANT
/* See #define NUM_CUSTOM_CATEGORIES */
NDPI_PROTOCOL_CATEGORY_CUSTOM_1, /* User custom category 1 */
NDPI_PROTOCOL_CATEGORY_CUSTOM_2, /* User custom category 2 */
NDPI_PROTOCOL_CATEGORY_CUSTOM_3, /* User custom category 3 */
NDPI_PROTOCOL_CATEGORY_CUSTOM_4, /* User custom category 4 */
NDPI_PROTOCOL_CATEGORY_CUSTOM_5, /* User custom category 5 */
Please keep in sync with
/* Further categories... */
NDPI_PROTOCOL_CATEGORY_MUSIC,
NDPI_PROTOCOL_CATEGORY_VIDEO,
NDPI_PROTOCOL_CATEGORY_SHOPPING,
NDPI_PROTOCOL_CATEGORY_PRODUCTIVITY,
NDPI_PROTOCOL_CATEGORY_FILE_SHARING,
/*
The category below is used by sites who are used
to test connectivity
*/
NDPI_PROTOCOL_CATEGORY_CONNECTIVITY_CHECK,
NDPI_PROTOCOL_CATEGORY_IOT_SCADA,
static const char* categories[] = { ..}
/* Some custom categories */
CUSTOM_CATEGORY_MINING = 99,
CUSTOM_CATEGORY_MALWARE = 100,
CUSTOM_CATEGORY_ADVERTISEMENT = 101,
CUSTOM_CATEGORY_BANNED_SITE = 102,
CUSTOM_CATEGORY_SITE_UNAVAILABLE = 103,
CUSTOM_CATEGORY_ALLOWED_SITE = 104,
/*
The category below is used to track communications made by
security applications (e.g. sophosxl.net, spamhaus.org)
to track malware, spam etc.
*/
CUSTOM_CATEGORY_ANTIMALWARE = 105,
in ndpi_main.c
*/
/*
IMPORTANT
Please keep in sync with
static const char* categories[] = { ..}
in ndpi_main.c
*/
NDPI_PROTOCOL_NUM_CATEGORIES, /*
NOTE: Keep this as last member
Unused as value but useful to getting the number of elements
in this datastructure
*/
NDPI_PROTOCOL_ANY_CATEGORY /* Used to handle wildcards */
NDPI_PROTOCOL_NUM_CATEGORIES, /*
NOTE: Keep this as last member
Unused as value but useful to getting the number of elements
in this datastructure
*/
NDPI_PROTOCOL_ANY_CATEGORY /* Used to handle wildcards */
} ndpi_protocol_category_t;
typedef enum {
ndpi_pref_direction_detect_disable = 0,
ndpi_pref_enable_tls_block_dissection
ndpi_pref_enable_tls_block_dissection /* nDPI considers only those blocks past the certificate exchange */
} ndpi_detection_preference;
/* ntop extensions */
typedef struct ndpi_proto_defaults {
char *protoName;
ndpi_protocol_category_t protoCategory;
u_int8_t can_have_a_subprotocol;
u_int16_t * subprotocols;
u_int32_t subprotocol_count;
u_int16_t protoId, protoIdx;
u_int16_t master_tcp_protoId[2], master_udp_protoId[2]; /* The main protocols on which this sub-protocol sits on */
u_int16_t tcp_default_ports[MAX_DEFAULT_PORTS], udp_default_ports[MAX_DEFAULT_PORTS];
ndpi_protocol_breed_t protoBreed;
void (*func) (struct ndpi_detection_module_struct *, struct ndpi_flow_struct *flow);
@@ -989,7 +1062,6 @@ typedef struct ndpi_default_ports_tree_node {
typedef struct _ndpi_automa {
void *ac_automa; /* Real type is AC_AUTOMATA_t */
u_int8_t ac_automa_finalized;
} ndpi_automa;
typedef struct ndpi_proto {
@@ -1010,7 +1082,10 @@ typedef struct ndpi_proto {
#ifdef NDPI_LIB_COMPILATION
/* Needed to have access to HAVE_* defines */
#ifndef _NDPI_CONFIG_H_
#include "ndpi_config.h"
#define _NDPI_CONFIG_H_
#endif
#ifdef HAVE_PCRE
#include <pcre.h>
@@ -1033,7 +1108,8 @@ struct ndpi_detection_module_struct {
u_int32_t current_ts;
u_int32_t ticks_per_second;
u_int16_t num_tls_blocks_to_follow;
u_int8_t skip_tls_blocks_until_change_cipher:1, enable_ja3_plus:1, _notused:6;
#ifdef NDPI_ENABLE_DEBUG_MESSAGES
void *user_data;
#endif
@@ -1078,12 +1154,17 @@ struct ndpi_detection_module_struct {
u_int ndpi_num_supported_protocols;
u_int ndpi_num_custom_protocols;
/* HTTP/DNS/HTTPS host matching */
int ac_automa_finalized;
/* HTTP/DNS/HTTPS/QUIC host matching */
ndpi_automa host_automa, /* Used for DNS/HTTPS */
content_automa, /* Used for HTTP subprotocol_detection */
subprotocol_automa, /* Used for HTTP subprotocol_detection */
bigrams_automa, impossible_bigrams_automa; /* TOR */
/* IMPORTANT: please update ndpi_finalize_initalization() whenever you add a new automa */
risky_domain_automa, tls_cert_subject_automa,
malicious_ja3_automa, malicious_sha1_automa,
host_risk_mask_automa, common_alpns_automa;
/* IMPORTANT: please update ndpi_finalize_initialization() whenever you add a new automa */
void *ip_risk_mask_ptree;
struct {
ndpi_automa hostnames, hostnames_shadow;
@@ -1114,10 +1195,7 @@ struct ndpi_detection_module_struct {
u_int32_t jabber_file_transfer_timeout;
u_int8_t ip_version_limit;
/* NDPI_PROTOCOL_BITTORRENT */
struct hash_ip4p_table *bt_ht;
#ifdef NDPI_DETECTION_SUPPORT_IPV6
struct hash_ip4p_table *bt6_ht;
#endif
struct hash_ip4p_table *bt_ht, *bt6_ht;
/* BT_ANNOUNCE */
struct bt_announce *bt_ann;
@@ -1132,6 +1210,12 @@ struct ndpi_detection_module_struct {
/* NDPI_PROTOCOL_STUN and subprotocols */
struct ndpi_lru_cache *stun_cache;
/* NDPI_PROTOCOL_TLS and subprotocols */
struct ndpi_lru_cache *tls_cert_cache;
/* NDPI_PROTOCOL_MINING and subprotocols */
struct ndpi_lru_cache *mining_cache;
/* NDPI_PROTOCOL_MSTEAMS */
struct ndpi_lru_cache *msteams_cache;
@@ -1140,10 +1224,15 @@ struct ndpi_detection_module_struct {
u_int8_t direction_detect_disable:1, /* disable internal detection of packet direction */ _pad:7;
void (*ndpi_notify_lru_add_handler_ptr)(ndpi_lru_cache_type cache_type, u_int32_t proto, u_int32_t app_proto);
#ifdef CUSTOM_NDPI_PROTOCOLS
#include "../../../nDPI-custom/custom_ndpi_typedefs.h"
#endif
#ifdef HAVE_MAXMINDDB
MMDB_s mmdb_city, mmdb_as;
u_int8_t mmdb_city_loaded, mmdb_as_loaded;
#endif
};
#endif /* NDPI_LIB_COMPILATION */
@@ -1154,6 +1243,16 @@ typedef enum {
ndpi_cipher_insecure = NDPI_CIPHER_INSECURE
} ndpi_cipher_weakness;
#define MAX_NUM_TLS_SIGNATURE_ALGORITHMS 16
struct tls_heuristics {
/*
TLS heuristics for detecting browsers usage
NOTE: expect false positives
*/
u_int8_t is_safari_tls:1, is_firefox_tls:1, is_chrome_tls:1, notused:5;
};
/*
NOTE
When the struct below is modified don't forget to update
@@ -1174,9 +1273,11 @@ struct ndpi_flow_struct {
*/
u_int32_t next_tcp_seq_nr[2];
// -----------------------------------------
u_int8_t max_extra_packets_to_check;
u_int8_t num_extra_packets_checked;
u_int8_t num_processed_pkts; /* <= WARNING it can wrap but we do expect people to giveup earlier */
u_int16_t num_processed_pkts; /* <= WARNING it can wrap but we do expect people to giveup earlier */
int (*extra_packets_func) (struct ndpi_detection_module_struct *, struct ndpi_flow_struct *flow);
@@ -1191,7 +1292,7 @@ struct ndpi_flow_struct {
/* Place textual flow info here */
char flow_extra_info[16];
/*
Pointer to src or dst that identifies the
server of this connection
@@ -1200,9 +1301,10 @@ struct ndpi_flow_struct {
/* HTTP host or DNS query */
u_char host_server_name[240];
u_int8_t initial_binary_bytes[8], initial_binary_bytes_len;
u_int8_t risk_checked;
u_int8_t risk_checked:1, ip_risk_mask_evaluated:1, host_risk_mask_evaluated:1, _notused:5;
ndpi_risk risk_mask; /* Stores the flow risk mask for flow peers */
ndpi_risk risk; /* Issues found with this flow [bitmask of ndpi_risk] */
/*
This structure below will not not stay inside the protos
structure below as HTTP is used by many subprotocols
@@ -1212,18 +1314,19 @@ struct ndpi_flow_struct {
*/
struct {
ndpi_http_method method;
char *url, *content_type, *user_agent;
char *url, *content_type /* response */, *request_content_type /* e.g. for POST */, *user_agent;
u_int8_t num_request_headers, num_response_headers;
u_int8_t request_version; /* 0=1.0 and 1=1.1. Create an enum for this? */
u_int16_t response_status_code; /* 200, 404, etc. */
u_char detected_os[32]; /* Via HTTP/QUIC User-Agent */
} http;
/*
/*
Put outside of the union to avoid issues in case the protocol
is remapped to somethign pther than Kerberos due to a faulty
dissector
*/
struct {
struct {
char *pktbuf;
u_int16_t pktbuf_maxlen, pktbuf_currlen;
} kerberos_buf;
@@ -1249,25 +1352,35 @@ struct ndpi_flow_struct {
struct {
char ssl_version_str[12];
u_int16_t ssl_version, server_names_len;
char client_requested_server_name[64], *server_names,
*alpn, *tls_supported_versions, *issuerDN, *subjectDN;
char client_requested_server_name[256], /* SNI hostname length: RFC 4366 */
*server_names, *alpn, *tls_supported_versions, *issuerDN, *subjectDN;
u_int32_t notBefore, notAfter;
char ja3_client[33], ja3_server[33];
u_int16_t server_cipher;
u_int8_t sha1_certificate_fingerprint[20];
#ifdef TLS_HANDLE_SIGNATURE_ALGORITMS
/* Under #ifdef to save memory for those who do not need them */
u_int8_t num_tls_signature_algorithms;
u_int16_t client_signature_algorithms[MAX_NUM_TLS_SIGNATURE_ALGORITHMS];
#endif
struct tls_heuristics browser_heuristics;
struct {
u_int16_t cipher_suite;
char *esni;
} encrypted_sni;
ndpi_cipher_weakness server_unsafe_cipher;
} ssl;
} tls_quic;
struct {
u_int8_t num_udp_pkts, num_processed_pkts, num_binding_requests;
u_int8_t num_udp_pkts, num_binding_requests;
u_int16_t num_processed_pkts;
} stun;
/* We can have STUN over SSL/TLS thus they need to live together */
} stun_ssl;
} tls_quic_stun;
struct {
char client_signature[48], server_signature[48];
@@ -1277,7 +1390,7 @@ struct ndpi_flow_struct {
struct {
u_int8_t last_one_byte_pkt, last_byte;
} imo;
struct {
u_int8_t username_detected:1, username_found:1,
password_detected:1, password_found:1,
@@ -1285,14 +1398,12 @@ struct ndpi_flow_struct {
u_int8_t character_id;
char username[32], password[32];
} telnet;
struct {
char version[32];
} ubntac2;
struct {
/* Via HTTP User-Agent */
u_char detected_os[32];
/* Via HTTP X-Forwarded-For */
u_char nat_ip[24];
} http;
@@ -1301,7 +1412,7 @@ struct ndpi_flow_struct {
u_int8_t auth_found:1, auth_failed:1, _pad:5;
char username[16], password[16];
} ftp_imap_pop_smtp;
struct {
/* Bittorrent hash */
u_char hash[20];
@@ -1371,6 +1482,9 @@ struct ndpi_flow_struct {
/* NDPI_PROTOCOL_STARCRAFT */
u_int8_t starcraft_udp_stage : 3; // 0-7
/* NDPI_PROTOCOL_Z3950 */
u_int8_t z3950_stage : 2; // 0-3
/* NDPI_PROTOCOL_OPENVPN */
u_int8_t ovpn_session_id[8];
u_int8_t ovpn_counter;
@@ -1382,7 +1496,7 @@ struct ndpi_flow_struct {
/* NDPI_PROTOCOL_CSGO */
u_int8_t csgo_strid[18],csgo_state,csgo_s2;
u_int32_t csgo_id2;
/* internal structures to save functions calls */
struct ndpi_packet_struct packet;
struct ndpi_flow_struct *flow;
@@ -1392,9 +1506,10 @@ struct ndpi_flow_struct {
typedef struct {
char *string_to_match, *proto_name;
int protocol_id;
u_int16_t protocol_id;
ndpi_protocol_category_t protocol_category;
ndpi_protocol_breed_t protocol_breed;
int level; /* 0 by default */
} ndpi_protocol_match;
typedef struct {
@@ -1402,6 +1517,11 @@ typedef struct {
ndpi_protocol_category_t protocol_category;
} ndpi_category_match;
typedef struct {
char *string_to_match;
u_int16_t protocol_id;
} ndpi_tls_cert_name_match;
typedef struct {
u_int32_t network;
u_int8_t cidr;
@@ -1412,13 +1532,14 @@ typedef u_int32_t ndpi_init_prefs;
typedef enum
{
ndpi_no_prefs = 0,
ndpi_dont_load_tor_hosts,
ndpi_dont_init_libgcrypt,
ndpi_no_prefs = 0,
ndpi_dont_load_tor_hosts = 1,
ndpi_dont_init_libgcrypt = 2,
ndpi_enable_ja3_plus = 4
} ndpi_prefs;
typedef struct {
int protocol_id;
u_int32_t protocol_id;
ndpi_protocol_category_t protocol_category;
ndpi_protocol_breed_t protocol_breed;
} ndpi_protocol_match_result;
@@ -1430,7 +1551,7 @@ typedef enum {
ndpi_serialization_format_csv
} ndpi_serialization_format;
/* Note:
/* Note:
* - up to 16 types (TLV encoding: "4 bit key type" << 4 | "4 bit value type")
* - key supports string and uint32 (compressed to uint8/uint16) only, this is also enforced by the API */
typedef enum {
@@ -1520,12 +1641,39 @@ struct ndpi_analyze_struct {
/* **************************************** */
struct ndpi_rsi_struct {
u_int8_t empty:1, rsi_ready:1, _notused:6;
u_int16_t num_values, next_index;
u_int32_t *gains, *losses;
u_int32_t last_value, total_gains, total_losses;
};
/* **************************************** */
struct ndpi_jitter_struct {
u_int8_t empty:1, jitter_ready:1, _notused:6;
u_int16_t num_values, next_index;
float *observations, last_value, jitter_total;
};
/* **************************************** */
#ifndef AF_MAC
#define AF_MAC 99
#endif
typedef void (*ndpi_void_fn_t)(void *data);
typedef void (*ndpi_void_fn2_t)(ndpi_prefix_t *prefix, void *data);
typedef void (*ndpi_void_fn3_t)(ndpi_patricia_node_t *node, void *data, void *user_data);
/* **************************************** */
typedef struct ndpi_ptree ndpi_ptree_t;
/* **************************************** */
struct ndpi_hll {
u_int8_t bits;
u_int8_t bits;
size_t size;
u_int8_t *registers;
};
@@ -1541,7 +1689,7 @@ enum ndpi_bin_family {
struct ndpi_bin {
u_int8_t num_bins, is_empty;
enum ndpi_bin_family family;
union {
u_int8_t *bins8; /* num_bins bins */
u_int16_t *bins16; /* num_bins bins */
@@ -1549,4 +1697,79 @@ struct ndpi_bin {
} u;
};
/* **************************************** */
struct ndpi_str_hash_info {
char *key; /* Key */
u_int8_t key_len;
u_int8_t value; /* Value */
struct ndpi_str_hash_info *next;
};
typedef struct {
u_int32_t num_buckets, max_num_entries;
struct ndpi_str_hash_info **buckets;
} ndpi_str_hash;
/* **************************************** */
#define HW_HISTORY_LEN 4
#define MAX_SQUARE_ERROR_ITERATIONS 64 /* MUST be < num_values_rollup (256 max) */
struct ndpi_hw_struct {
struct {
u_int8_t use_hw_additive_seasonal;
double alpha, beta, gamma, ro;
u_int16_t num_season_periods; /* num of values of a season */
} params;
struct {
double sum_square_error;
u_int8_t num_values_rollup;
} prev_error;
u_int32_t num_values;
double u, v, sum_square_error;
/* These two values need to store the signal history */
u_int64_t *y;
double *s;
};
struct ndpi_ses_struct {
struct {
double alpha, ro;
} params;
struct {
double sum_square_error;
u_int8_t num_values_rollup;
} prev_error;
u_int32_t num_values;
double sum_square_error, last_forecast, last_value;
};
struct ndpi_des_struct {
struct {
double alpha, beta, ro;
} params;
struct {
double sum_square_error;
u_int8_t num_values_rollup;
} prev_error;
u_int32_t num_values;
double sum_square_error, last_forecast, last_trend, last_value;
};
/* **************************************** */
/* Prototype used to define custom DGA detection function */
typedef int (*ndpi_custom_dga_predict_fctn)(const char* domain, int domain_length);
/* **************************************** */
#endif /* __NDPI_TYPEDEFS_H__ */
+32
View File
@@ -0,0 +1,32 @@
#ifndef __NDPI_UTILS_H__
#define __NDPI_UTILS_H__
#include "ndpi_define.h"
#define MYDBG(m, ...) \
printf(" DBG[%s:%s:%u]: \t" m "\n", __FILE__, __FUNCTION__, __LINE__, ##__VA_ARGS__);
// #define NDPI_ENABLE_DEBUG_POINTER_MESSAGES
// #define NDPI_ENABLE_DEBUG_INFO_MESSAGES
// #define NDPI_ENABLE_DEBUG_TRACE_MESSAGES
extern void printRawData(const uint8_t *ptr, size_t len);
//extern uint8_t add_segment_to_buffer( struct ndpi_flow_struct *flow, struct ndpi_tcphdr const * tcph, uint32_t waited);
//extern uint8_t check_for_sequence( struct ndpi_flow_struct *flow, struct ndpi_tcphdr const * tcph);
/* **************************************** */
/* Can't call libc functions from kernel space, define some stub instead */
#define ndpi_isalpha(ch) (((ch) >= 'a' && (ch) <= 'z') || ((ch) >= 'A' && (ch) <= 'Z'))
#define ndpi_isdigit(ch) ((ch) >= '0' && (ch) <= '9')
#define ndpi_isspace(ch) (((ch) >= '\t' && (ch) <= '\r') || ((ch) == ' '))
#define ndpi_isprint(ch) ((ch) >= 0x20 && (ch) <= 0x7e)
#define ndpi_ispunct(ch) (((ch) >= '!' && (ch) <= '/') || \
((ch) >= ':' && (ch) <= '@') || \
((ch) >= '[' && (ch) <= '`') || \
((ch) >= '{' && (ch) <= '~'))
#endif
+3 -6
View File
@@ -59,8 +59,10 @@
#define IPPROTO_SCTP 132
#endif
#if defined(__MINGW32__) || defined(__MINGW64__)
#undef gettimeofday
#define gettimeofday mingw_gettimeofday
#endif
extern char* strsep(char **sp, char *sep);
@@ -93,11 +95,6 @@ extern unsigned long waitForNextEvent(unsigned long ulDelay /* ms */);
#define strtok_r strtok_s
#define timegm _mkgmtime
static inline struct tm * localtime_r(const time_t *timep, struct tm * result)
{
struct tm *timeinfo;
timeinfo = localtime(timep);
return timeinfo;
}
extern struct tm* localtime_r(const time_t* timep, struct tm* r);
#endif /* __NDPI_WIN32_H__ */
+4 -5
View File
@@ -6,17 +6,16 @@
# make Makefile
#
#
# Installation directories
#
prefix = @prefix@
libdir = ${prefix}/lib
includedir = ${prefix}/include/ndpi
CC = @CC@
CC = @GNU_PREFIX@@CC@
CFLAGS += -fPIC -DPIC -I../include -Ithird_party/include -DNDPI_LIB_COMPILATION -Wall @CFLAGS@ @CUSTOM_NDPI@
LDFLAGS = @LDFLAGS@ @ADDITIONAL_LIBS@ @LIBS@
RANLIB = ranlib
LDFLAGS = @LDFLAGS@ @ADDITIONAL_LIBS@ @LIBS@ -lm
RANLIB = @GNU_PREFIX@ranlib
OBJECTS = $(patsubst protocols/%.c, protocols/%.o, $(wildcard protocols/*.c)) $(patsubst third_party/src/%.c, third_party/src/%.o, $(wildcard third_party/src/*.c)) $(patsubst ./%.c, ./%.o, $(wildcard ./*.c))
HEADERS = $(wildcard ../include/*.h)
@@ -48,7 +47,7 @@ all: $(NDPI_LIBS)
ndpi_main.c: ndpi_content_match.c.inc
$(NDPI_LIB_STATIC): $(OBJECTS)
ar rc $@ $(OBJECTS)
@GNU_PREFIX@ar rc $@ $(OBJECTS)
$(RANLIB) $@
$(NDPI_LIB_SHARED): $(OBJECTS)
+509 -19
View File
@@ -1,7 +1,7 @@
/*
* ndpi_analyze.c
*
* Copyright (C) 2019 - ntop.org
* Copyright (C) 2019-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library.
@@ -39,20 +39,16 @@
void ndpi_init_data_analysis(struct ndpi_analyze_struct *ret, u_int16_t _max_series_len) {
u_int32_t len;
memset(ret, 0, sizeof(struct ndpi_analyze_struct));
memset(ret, 0, sizeof(*ret));
if(_max_series_len > MAX_SERIES_LEN) _max_series_len = MAX_SERIES_LEN;
ret->num_values_array_len = _max_series_len;
if(ret->num_values_array_len > 0) {
len = sizeof(u_int32_t)*ret->num_values_array_len;
if((ret->values = ndpi_malloc(len)) == NULL) {
ndpi_free(ret);
ret = NULL;
} else
len = sizeof(u_int32_t) * ret->num_values_array_len;
if((ret->values = ndpi_malloc(len)) != NULL)
memset(ret->values, 0, len);
} else
ret->values = NULL;
}
}
/* ********************************************************************************* */
@@ -68,17 +64,23 @@ struct ndpi_analyze_struct* ndpi_alloc_data_analysis(u_int16_t _max_series_len)
/* ********************************************************************************* */
void ndpi_free_data_analysis(struct ndpi_analyze_struct *d) {
void ndpi_free_data_analysis(struct ndpi_analyze_struct *d, u_int8_t free_pointer) {
if(d->values) ndpi_free(d->values);
ndpi_free(d);
if(free_pointer) ndpi_free(d);
}
/* ********************************************************************************* */
void ndpi_reset_data_analysis(struct ndpi_analyze_struct *d) {
u_int32_t *values_bkp = d->values;
u_int32_t num_values_array_len_bpk = d->num_values_array_len;
memset(d, 0, sizeof(struct ndpi_analyze_struct));
d->values = values_bkp;
d->num_values_array_len = num_values_array_len_bpk;
memset(d->values, 0, sizeof(u_int32_t)*d->num_values_array_len);
d->num_data_entries = 0;
}
/* ********************************************************************************* */
@@ -105,12 +107,12 @@ void ndpi_data_add_value(struct ndpi_analyze_struct *s, const u_int32_t value) {
/*
Optimized stddev calculation
https://www.khanacademy.org/math/probability/data-distributions-a1/summarizing-spread-distributions/a/calculating-standard-deviation-step-by-step
https://math.stackexchange.com/questions/683297/how-to-calculate-standard-deviation-without-detailed-historical-data
http://mathcentral.uregina.ca/QQ/database/QQ.09.02/carlos1.html
*/
s->stddev.sum_square_total += value * value;
s->stddev.sum_square_total += (u_int64_t)value * (u_int64_t)value;
}
/* ********************************************************************************* */
@@ -140,7 +142,8 @@ u_int32_t ndpi_data_max(struct ndpi_analyze_struct *s) { return(s->max_val); }
/* Compute the variance on all values */
float ndpi_data_variance(struct ndpi_analyze_struct *s) {
return(s->num_data_entries ? (float)(s->stddev.sum_square_total - ((s->sum_total * s->sum_total) / (s->num_data_entries))) / (float)s->num_data_entries : 0);
float v = s->num_data_entries ? ((float)s->stddev.sum_square_total - ((float)s->sum_total * (float)s->sum_total / (float)s->num_data_entries)) / (float)s->num_data_entries : 0.0;
return((v < 0 /* rounding problem */) ? 0 : v);
}
/* ********************************************************************************* */
@@ -468,6 +471,7 @@ void ndpi_normalize_bin(struct ndpi_bin *b) {
b->u.bins8[i] = (b->u.bins8[i]*100) / tot;
}
break;
case ndpi_bin_family16:
for(i=0; i<b->num_bins; i++) tot += b->u.bins16[i];
@@ -476,6 +480,7 @@ void ndpi_normalize_bin(struct ndpi_bin *b) {
b->u.bins16[i] = (b->u.bins16[i]*100) / tot;
}
break;
case ndpi_bin_family32:
for(i=0; i<b->num_bins; i++) tot += b->u.bins32[i];
@@ -586,7 +591,7 @@ float ndpi_bin_similarity(struct ndpi_bin *b1, struct ndpi_bin *b2, u_int8_t nor
u_int32_t a = ndpi_get_bin_value(b1, i);
u_int32_t b = ndpi_get_bin_value(b2, i);
u_int32_t diff = (a > b) ? (a - b) : (b - a);
if(a != b) sum += pow(diff, 2);
// printf("[a: %u][b: %u][sum: %u]\n", a, b, sum);
@@ -707,7 +712,7 @@ int ndpi_cluster_bins(struct ndpi_bin *bins, u_int16_t num_bins,
#ifdef COSINE_SIMILARITY
best_similarity = -1;
#else
best_similarity = 99999999999;
best_similarity = 99999999999.0f;
#endif
for(j=0; j<num_clusters; j++) {
@@ -719,7 +724,7 @@ int ndpi_cluster_bins(struct ndpi_bin *bins, u_int16_t num_bins,
if(j == cluster_ids[i])
current_similarity = similarity;
if(verbose)
printf("Bin %u / centroid %u [similarity: %f]\n", i, j, similarity);
@@ -741,7 +746,7 @@ int ndpi_cluster_bins(struct ndpi_bin *bins, u_int16_t num_bins,
*/
cluster_id = cluster_ids[i];
}
bin_score[i] = best_similarity;
if(cluster_ids[i] != cluster_id) {
@@ -815,3 +820,488 @@ int ndpi_cluster_bins(struct ndpi_bin *bins, u_int16_t num_bins,
}
/* ********************************************************************************* */
/*
RSI (Relative Strength Index)
RSI = 100 − [ 100/ (1 + (Average gain/Average loss)) ]
https://www.investopedia.com/terms/r/rsi.asp
*/
int ndpi_alloc_rsi(struct ndpi_rsi_struct *s, u_int16_t num_learning_values) {
memset(s, 0, sizeof(struct ndpi_rsi_struct));
s->empty = 1, s->num_values = num_learning_values;
s->gains = (u_int32_t*)ndpi_calloc(num_learning_values, sizeof(u_int32_t));
s->losses = (u_int32_t*)ndpi_calloc(num_learning_values, sizeof(u_int32_t));
if(s->gains && s->losses) {
s->last_value = 0;
return(0);
} else {
if(s->gains) free(s->gains);
if(s->losses) free(s->losses);
return(-1);
}
}
/* ************************************* */
void ndpi_free_rsi(struct ndpi_rsi_struct *s) {
ndpi_free(s->gains), ndpi_free(s->losses);
}
/* ************************************* */
// #define DEBUG_RSI
/*
This function adds a new value and returns the computed RSI, or -1
if there are too few points (< num_learning_values)
RSI < 30 (too many losses)
RSI > 70 (too many gains)
*/
float ndpi_rsi_add_value(struct ndpi_rsi_struct *s, const u_int32_t value) {
float relative_strength;
if(!s->empty) {
u_int32_t val;
s->total_gains -= s->gains[s->next_index], s->total_losses -= s->losses[s->next_index];
if(value > s->last_value) {
val = value - s->last_value;
s->gains[s->next_index] = val, s->losses[s->next_index] = 0;
s->total_gains += val;
#ifdef DEBUG_RSI
printf("Gain: %u\n", val);
#endif
} else {
val = s->last_value - value;
s->losses[s->next_index] = val, s->gains[s->next_index] = 0;
s->total_losses += val;
#ifdef DEBUG_RSI
printf("Loss: %u\n", val);
#endif
}
#ifdef DEBUG_RSI
printf("[value: %u][total_gains: %u][total_losses: %u][cur_idx: %u]\n", value, s->total_gains, s->total_losses, s->next_index);
#endif
}
s->last_value = value, s->next_index = (s->next_index + 1) % s->num_values, s->empty = 0;
if(s->next_index == 0) s->rsi_ready = 1; /* We have completed one round */
if(!s->rsi_ready)
return(-1); /* Too early */
else if(s->total_losses == 0) /* Avoid division by zero (**) */
return(100.);
else {
relative_strength = (float)s->total_gains / (float)s->total_losses; /* (**) */
#ifdef DEBUG_RSI
printf("RSI: %f\n", relative_strength);
#endif
return(100. - (100. / (1. + relative_strength)));
}
}
/* *********************************************************** */
/* https://www.johndcook.com/blog/cpp_phi_inverse/ */
static double ndpi_rational_approximation(double t) {
// Abramowitz and Stegun formula 26.2.23.
// The absolute value of the error should be less than 4.5 e-4.
double c[] = { 2.515517, 0.802853, 0.010328 };
double d[] = { 1.432788, 0.189269, 0.001308 };
return(t - ((c[2]*t + c[1])*t + c[0]) / (((d[2]*t + d[1])*t + d[0])*t + 1.0));
}
static double ndpi_normal_cdf_inverse(double p) {
if(p <= 0.0 || p >= 1.0)
return(0); /* Invalid argument: valid range 0 < X < 1 */
if(p < 0.5) {
// F^-1(p) = - G^-1(p)
return -ndpi_rational_approximation( sqrt(-2.0*log(p)) );
} else {
// F^-1(p) = G^-1(1-p)
return ndpi_rational_approximation( sqrt(-2.0*log(1-p)) );
}
}
double ndpi_avg_inline(u_int64_t *v, u_int num) {
double avg = 0;
u_int i;
for(i=0; i<num; i++)
avg += v[i];
return(avg / (u_int32_t)num);
}
/* *********************************************************** */
/* *********************************************************** */
/*
Initializes Holt-Winters with Confidence Interval
Input
hw: Datastructure to initialize and that needs tobe freed with ndpi_hw_free()
num_periods Number of observations of a season, or in ML-parlance the number of points that are required to make the forecast
additive If set to 1 will use the Holt-Winters additive seasonal (should be the default), otherwise the multiplicative seasonal.
alpha Level: specifies the coefficient for the level smoothing. Range 0..1. The higher α, the faster the method forgets old values
beta Trend: specifies the coefficient for the trend smoothing. Range 0..1.
gamma Seasonal: specifies the coefficient for the seasonal smoothing. Range 0..1. With gamma = 0, seasonal correction is not used.
significance Significance level for the forecats sed for computing lower and upper bands. Range 0..1. Typical values 0.05 or less.
See https://en.wikipedia.org/wiki/Statistical_significance
NOTE (See https://otexts.com/fpp2/holt-winters.html)
The additive method is preferred when the seasonal variations are roughly constant through the series,
while the multiplicative method is preferred when the seasonal variations are changing proportional to the level of the series.
For learning more about timeseries forecasting see
https://www.real-statistics.com/time-series-analysis/basic-time-series-forecasting/
*/
int ndpi_hw_init(struct ndpi_hw_struct *hw,
u_int16_t num_periods, u_int8_t additive_seeasonal,
double alpha, double beta, double gamma, float significance) {
memset(hw, 0, sizeof(struct ndpi_hw_struct));
hw->params.num_season_periods = num_periods + 1;
hw->params.alpha = alpha;
hw->params.beta = beta;
hw->params.gamma = gamma;
hw->params.use_hw_additive_seasonal = additive_seeasonal;
if((significance < 0) || (significance > 1)) significance = 0.05;
hw->params.ro = ndpi_normal_cdf_inverse(1 - (significance / 2.));
if((hw->y = (u_int64_t*)ndpi_calloc(hw->params.num_season_periods, sizeof(u_int64_t))) == NULL)
return(-1);
if((hw->s = (double*)ndpi_calloc(hw->params.num_season_periods, sizeof(double))) == NULL) {
free(hw->y);
return(-1);
}
return(0);
}
/* *********************************************************** */
/* Frees the memory allocated by ndpi_hw_init() */
void ndpi_hw_free(struct ndpi_hw_struct *hw) {
if(hw->y) ndpi_free(hw->y);
if(hw->s) ndpi_free(hw->s);
}
/* *********************************************************** */
/*
Returns the forecast and the band (forecast +/- band are the upper and lower values)
Input
hw: Datastructure previously initialized
value The value to add to the measurement
Output
forecast The forecasted value
confidence_band The value +/- on which the value should fall is not an anomaly
Return code
0 Too early: we're still in the learning phase. Output values are zero.
1 Normal processing: forecast and confidence_band are meaningful
*/
int ndpi_hw_add_value(struct ndpi_hw_struct *hw, const u_int64_t _value, double *forecast, double *confidence_band) {
if(hw->num_values < hw->params.num_season_periods) {
hw->y[hw->num_values++] = _value;
*forecast = 0;
*confidence_band = 0;
return(0); /* Too early still training... */
} else {
u_int idx = hw->num_values % hw->params.num_season_periods;
double prev_u, prev_v, prev_s, value = (double)_value;
double sq, error, sq_error;
u_int observations;
if(hw->num_values == hw->params.num_season_periods) {
double avg = ndpi_avg_inline(hw->y, hw->params.num_season_periods);
u_int i;
if(avg == 0) avg = 1; /* Avoid divisions by zero */
for(i=0; i<hw->params.num_season_periods; i++)
hw->s[i] = hw->y[i] / avg;
i = hw->params.num_season_periods-1;
if(hw->s[i] == 0)
hw->u = 0;
else
hw->u = _value / hw->s[i];
hw->v = 0;
ndpi_free(hw->y);
hw->y = NULL;
}
idx = hw->num_values % hw->params.num_season_periods;
prev_u = hw->u, prev_v = hw->v, prev_s = hw->s[idx];
if(prev_s != 0)
hw->u = ((hw->params.alpha * value) / prev_s) + ( 1 - hw->params.alpha) * (hw->u + hw->v);
else
hw->u = 0; /* Avoid divisions by zero */
hw->v = (hw->params.beta * (hw->u - prev_u)) + ((1 - hw->params.beta ) * hw->v);
if(hw->u != 0)
hw->s[idx] = (hw->params.gamma * (value / hw->u)) + ((1 - hw->params.gamma) * prev_s);
else
hw->s[idx] = 0; /* Avoid divisions by zero */
if(hw->params.use_hw_additive_seasonal)
*forecast = (prev_u + prev_v) + prev_s;
else
*forecast = (prev_u + prev_v) * prev_s;
error = value - *forecast;
sq_error = error * error;
hw->sum_square_error += sq_error, hw->prev_error.sum_square_error += sq_error;;
observations = (hw->num_values < MAX_SQUARE_ERROR_ITERATIONS) ? hw->num_values : ((hw->num_values % MAX_SQUARE_ERROR_ITERATIONS) + MAX_SQUARE_ERROR_ITERATIONS);
sq = sqrt(hw->sum_square_error / (observations - hw->params.num_season_periods));
*confidence_band = hw->params.ro * sq;
#ifdef HW_DEBUG
printf("[num_values: %u][u: %.3f][v: %.3f][s: %.3f][error: %.3f][forecast: %.3f][sqe: %.3f][sq: %.3f][confidence_band: %.3f]\n",
hw->num_values, hw->u, hw->v, hw->s[idx], error,
*forecast, hw->sum_square_error,
sq, *confidence_band);
#endif
hw->num_values++, idx = (idx + 1) % hw->params.num_season_periods;
if(++hw->prev_error.num_values_rollup == MAX_SQUARE_ERROR_ITERATIONS) {
hw->sum_square_error = hw->prev_error.sum_square_error;
hw->prev_error.num_values_rollup = 0, hw->prev_error.sum_square_error = 0;
}
return(1); /* We're in business: forecast is meaningful now */
}
}
/* ********************************************************************************* */
/* ********************************************************************************* */
/*
Jitter calculator
Used to determine how noisy is a signal
*/
int ndpi_jitter_init(struct ndpi_jitter_struct *s, u_int16_t num_learning_values) {
memset(s, 0, sizeof(struct ndpi_jitter_struct));
if(num_learning_values < 2) num_learning_values = 2;
s->empty = 1, s->num_values = num_learning_values;
s->observations = (float*)ndpi_calloc(num_learning_values, sizeof(float));
if(s->observations) {
s->last_value = 0;
return(0);
} else
return(-1);
}
/* ************************************* */
void ndpi_jitter_free(struct ndpi_jitter_struct *s) {
ndpi_free(s->observations);
}
/* ************************************* */
/*
This function adds a new value and returns the computed Jitter
*/
float ndpi_jitter_add_value(struct ndpi_jitter_struct *s, const float value) {
float val = fabsf(value - s->last_value);
if(s->empty && (s->next_index == 0))
; /* Skip the first value as we are unable to calculate the difference */
else {
s->jitter_total -= s->observations[s->next_index];
s->observations[s->next_index] = val;
s->jitter_total += val;
}
s->last_value = value, s->next_index = (s->next_index + 1) % s->num_values;
if(s->next_index == 0) s->jitter_ready = 1; /* We have completed one round */
#ifdef DEBUG_JITTER
printf("[JITTER] [value: %.3f][diff: %.3f][jitter_total: %.3f] -> %.3f\n",
value, val, s->jitter_total,
s->jitter_ready ? (s->jitter_total / s->num_values) : -1);
#endif
if(!s->jitter_ready)
return(-1); /* Too early */
else
return(s->jitter_total / s->num_values);
}
/* *********************************************************** */
/* *********************************************************** */
/*
Single Exponential Smoothing
*/
int ndpi_ses_init(struct ndpi_ses_struct *ses, double alpha, float significance) {
memset(ses, 0, sizeof(struct ndpi_ses_struct));
ses->params.alpha = alpha;
if((significance < 0) || (significance > 1)) significance = 0.05;
ses->params.ro = ndpi_normal_cdf_inverse(1 - (significance / 2.));
return(0);
}
/* *********************************************************** */
/*
Returns the forecast and the band (forecast +/- band are the upper and lower values)
Input
ses: Datastructure previously initialized
value The value to add to the measurement
Output
forecast The forecasted value
confidence_band The value +/- on which the value should fall is not an anomaly
Return code
0 Too early: we're still in the learning phase. Output values are zero.
1 Normal processing: forecast and confidence_band are meaningful
*/
int ndpi_ses_add_value(struct ndpi_ses_struct *ses, const u_int64_t _value, double *forecast, double *confidence_band) {
double value = (double)_value, error, sq_error;
int rc;
if(ses->num_values == 0)
*forecast = value;
else
*forecast = (ses->params.alpha * (ses->last_value - ses->last_forecast)) + ses->last_forecast;
error = value - *forecast;
sq_error = error * error;
ses->sum_square_error += sq_error, ses->prev_error.sum_square_error += sq_error;
if(ses->num_values > 0) {
u_int observations = (ses->num_values < MAX_SQUARE_ERROR_ITERATIONS) ? (ses->num_values + 1) : ((ses->num_values % MAX_SQUARE_ERROR_ITERATIONS) + MAX_SQUARE_ERROR_ITERATIONS + 1);
double sq = sqrt(ses->sum_square_error / observations);
*confidence_band = ses->params.ro * sq;
rc = 1;
} else
*confidence_band = 0, rc = 0;
ses->num_values++, ses->last_value = value, ses->last_forecast = *forecast;
if(++ses->prev_error.num_values_rollup == MAX_SQUARE_ERROR_ITERATIONS) {
ses->sum_square_error = ses->prev_error.sum_square_error;
ses->prev_error.num_values_rollup = 0, ses->prev_error.sum_square_error = 0;
}
#ifdef SES_DEBUG
printf("[num_values: %u][[error: %.3f][forecast: %.3f][sqe: %.3f][sq: %.3f][confidence_band: %.3f]\n",
ses->num_values, error, *forecast, ses->sum_square_error, sq, *confidence_band);
#endif
return(rc);
}
/* *********************************************************** */
/* *********************************************************** */
/*
Double Exponential Smoothing
*/
int ndpi_des_init(struct ndpi_des_struct *des, double alpha, double beta, float significance) {
memset(des, 0, sizeof(struct ndpi_des_struct));
des->params.alpha = alpha;
if((significance < 0) || (significance > 1)) significance = 0.05;
des->params.ro = ndpi_normal_cdf_inverse(1 - (significance / 2.));
return(0);
}
/* *********************************************************** */
/*
Returns the forecast and the band (forecast +/- band are the upper and lower values)
Input
des: Datastructure previously initialized
value The value to add to the measurement
Output
forecast The forecasted value
confidence_band The value +/- on which the value should fall is not an anomaly
Return code
0 Too early: we're still in the learning phase. Output values are zero.
1 Normal processing: forecast and confidence_band are meaningful
*/
int ndpi_des_add_value(struct ndpi_des_struct *des, const u_int64_t _value, double *forecast, double *confidence_band) {
double value = (double)_value, error, sq_error;
int rc;
if(des->num_values == 0)
*forecast = value, des->last_trend = 0;
else {
*forecast = (des->params.alpha * value) + ((1 - des->params.alpha) * (des->last_forecast + des->last_trend));
des->last_trend = (des->params.beta * (*forecast - des->last_forecast)) + ((1 - des->params.beta) * des->last_trend);
}
error = value - *forecast;
sq_error = error * error;
des->sum_square_error += sq_error, des->prev_error.sum_square_error += sq_error;
if(des->num_values > 0) {
u_int observations = (des->num_values < MAX_SQUARE_ERROR_ITERATIONS) ? (des->num_values + 1) : ((des->num_values % MAX_SQUARE_ERROR_ITERATIONS) + MAX_SQUARE_ERROR_ITERATIONS + 1);
double sq = sqrt(des->sum_square_error / observations);
*confidence_band = des->params.ro * sq;
rc = 1;
} else
*confidence_band = 0, rc = 0;
des->num_values++, des->last_value = value, des->last_forecast = *forecast;
if(++des->prev_error.num_values_rollup == MAX_SQUARE_ERROR_ITERATIONS) {
des->sum_square_error = des->prev_error.sum_square_error;
des->prev_error.num_values_rollup = 0, des->prev_error.sum_square_error = 0;
}
#ifdef DES_DEBUG
printf("[num_values: %u][[error: %.3f][forecast: %.3f][trend: %.3f[sqe: %.3f][sq: %.3f][confidence_band: %.3f]\n",
des->num_values, error, *forecast, des->last_trend, des->sum_square_error, sq, *confidence_band);
#endif
return(rc);
}
+2 -2
View File
@@ -1,7 +1,7 @@
/*
* ndpi_community_id.c
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -70,7 +70,7 @@
/* **************************************************** */
static ssize_t ndpi_community_id_buf_copy(u_int8_t * const dst, const void * const src, ssize_t len) {
static u_int16_t ndpi_community_id_buf_copy(u_int8_t * const dst, const void * const src, u_int16_t len) {
if(src)
memcpy(dst, src, len);
else
File diff suppressed because one or more lines are too long
+154
View File
@@ -0,0 +1,154 @@
/*
* ndpi_geoip.c
*
* Copyright (C) 2021 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library.
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#ifdef HAVE_CONFIG_H
#include "ndpi_config.h"
#endif
#include <stdlib.h>
#include <errno.h>
#include <sys/types.h>
#include <stdint.h>
#include "ndpi_api.h"
#include "ndpi_config.h"
/* ********************************************************************************* */
int ndpi_load_geoip(struct ndpi_detection_module_struct *ndpi_str,
const char *ip_city_data, const char *ip_as_data) {
#ifdef HAVE_MAXMINDDB
int status;
/* Open the MMDB files */
if((status = MMDB_open(ip_city_data, MMDB_MODE_MMAP, &ndpi_str->mmdb_city)) != MMDB_SUCCESS)
return(-1);
else
ndpi_str->mmdb_city_loaded = 1;
if((status = MMDB_open(ip_as_data, MMDB_MODE_MMAP, &ndpi_str->mmdb_as)) != MMDB_SUCCESS)
return(-2);
else
ndpi_str->mmdb_as_loaded = 1;
return(0);
#else
return(-3);
#endif
}
/* ********************************************************************************* */
void ndpi_free_geoip(struct ndpi_detection_module_struct *ndpi_str) {
#ifdef HAVE_MAXMINDDB
if(ndpi_str->mmdb_city_loaded) MMDB_close(&ndpi_str->mmdb_city);
if(ndpi_str->mmdb_as_loaded) MMDB_close(&ndpi_str->mmdb_as);
#endif
}
/* ********************************************************************************* */
int ndpi_get_geoip_asn(struct ndpi_detection_module_struct *ndpi_str, char *ip, u_int32_t *asn) {
#ifdef HAVE_MAXMINDDB
int gai_error, mmdb_error, status;
MMDB_lookup_result_s result;
MMDB_entry_data_s entry_data;
if(ndpi_str->mmdb_as_loaded) {
result = MMDB_lookup_string(&ndpi_str->mmdb_as, ip, &gai_error, &mmdb_error);
if((gai_error != 0)
|| (mmdb_error != MMDB_SUCCESS)
|| (!result.found_entry))
*asn = 0;
else {
/* Get the ASN */
if((status = MMDB_get_value(&result.entry, &entry_data, "autonomous_system_number", NULL)) == MMDB_SUCCESS) {
if(entry_data.has_data && entry_data.type == MMDB_DATA_TYPE_UINT32)
*asn = entry_data.uint32;
else
*asn = 0;
}
}
return(0);
}
#endif
return(-2);
}
/* ********************************************************************************* */
int ndpi_get_geoip_country_continent(struct ndpi_detection_module_struct *ndpi_str, char *ip,
char *country_code, u_int8_t country_code_len,
char *continent, u_int8_t continent_len) {
#ifdef HAVE_MAXMINDDB
int gai_error, mmdb_error;
MMDB_lookup_result_s result;
MMDB_entry_data_s entry_data;
if(ndpi_str->mmdb_city_loaded) {
int status;
result = MMDB_lookup_string(&ndpi_str->mmdb_city, ip, &gai_error, &mmdb_error);
if((gai_error != 0)
|| (mmdb_error != MMDB_SUCCESS)
|| (!result.found_entry))
country_code[0] = '\0';
else {
if(country_code_len > 0) {
status = MMDB_get_value(&result.entry, &entry_data, "country", "iso_code", NULL);
if((status != MMDB_SUCCESS) || (!entry_data.has_data))
country_code[0] = '\0';
else {
int str_len = ndpi_min(entry_data.data_size, country_code_len);
memcpy(country_code, entry_data.utf8_string, str_len);
country_code[str_len] = '\0';
}
}
if(continent_len > 0) {
status = MMDB_get_value(&result.entry, &entry_data, "continent", "names", "en", NULL);
if((status != MMDB_SUCCESS) || (!entry_data.has_data))
continent[0] = '\0';
else {
int str_len = ndpi_min(entry_data.data_size, continent_len);
memcpy(continent, entry_data.utf8_string, str_len);
continent[str_len] = '\0';
}
}
}
return(0);
}
#endif
return(-2);
}
+2437 -1598
View File
File diff suppressed because it is too large Load Diff
+10 -4
View File
@@ -1,7 +1,7 @@
/*
* ndpi_serializer.c
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-21 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
@@ -1673,11 +1673,10 @@ int ndpi_serialize_string_raw(ndpi_serializer *_serializer,
/* ********************************** */
int ndpi_serialize_string_boolean(ndpi_serializer *_serializer,
const char *key, u_int8_t value) {
int ndpi_serialize_binary_boolean(ndpi_serializer *_serializer,
const char *key, u_int16_t klen, u_int8_t value) {
ndpi_private_serializer *serializer = (ndpi_private_serializer*)_serializer;
u_int32_t buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
u_int16_t klen = strlen(key);
u_int32_t needed;
if(serializer->fmt != ndpi_serialization_format_json &&
@@ -1726,6 +1725,13 @@ int ndpi_serialize_string_boolean(ndpi_serializer *_serializer,
/* ********************************** */
int ndpi_serialize_string_boolean(ndpi_serializer *_serializer,
const char *key, u_int8_t value) {
return(ndpi_serialize_binary_boolean(_serializer, key, strlen(key), value));
}
/* ********************************** */
int ndpi_serialize_start_of_list_binary(ndpi_serializer *_serializer,
const char *key, u_int16_t klen) {
ndpi_private_serializer *serializer = (ndpi_private_serializer*)_serializer;
+1001 -577
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -2,7 +2,7 @@
* afp.c
*
* Copyright (C) 2009-11 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+25 -50
View File
@@ -1,8 +1,8 @@
/*
* aimini.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -33,22 +33,10 @@
static void ndpi_int_aimini_add_connection(struct ndpi_detection_module_struct *ndpi_struct, struct ndpi_flow_struct *flow/* , */
/* ndpi_protocol_type_t protocol_type */)
{
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_AIMINI, NDPI_PROTOCOL_UNKNOWN);
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_HTTP, NDPI_PROTOCOL_AIMINI);
}
static u_int8_t is_special_aimini_host(struct ndpi_int_one_line_struct host_line)
{
if (host_line.ptr != NULL && host_line.len >= NDPI_STATICSTRING_LEN("X.X.X.X.aimini.net")) {
if ((get_u_int32_t(host_line.ptr, 0) & htonl(0x00ff00ff)) == htonl(0x002e002e) &&
(get_u_int32_t(host_line.ptr, 4) & htonl(0x00ff00ff)) == htonl(0x002e002e) &&
memcmp(&host_line.ptr[8], "aimini.net", NDPI_STATICSTRING_LEN("aimini.net")) == 0) {
return 1;
}
}
return 0;
}
void ndpi_search_aimini(struct ndpi_detection_module_struct *ndpi_struct, struct ndpi_flow_struct *flow)
{
struct ndpi_packet_struct *packet = &flow->packet;
@@ -232,45 +220,32 @@ void ndpi_search_aimini(struct ndpi_detection_module_struct *ndpi_struct, struct
ndpi_int_aimini_add_connection(ndpi_struct, flow);
return;
}
} else if (packet->tcp != NULL) {
if ((packet->payload_packet_len > NDPI_STATICSTRING_LEN("GET /player/") &&
(memcmp(packet->payload, "GET /player/", NDPI_STATICSTRING_LEN("GET /player/")) == 0)) ||
(packet->payload_packet_len > NDPI_STATICSTRING_LEN("GET /play/?fid=") &&
(memcmp(packet->payload, "GET /play/?fid=", NDPI_STATICSTRING_LEN("GET /play/?fid=")) == 0))) {
NDPI_LOG_DBG2(ndpi_struct, "HTTP packet detected\n");
ndpi_parse_packet_line_info(ndpi_struct, flow);
if (packet->host_line.ptr != NULL && packet->host_line.len > 11
&& (memcmp(&packet->host_line.ptr[packet->host_line.len - 11], ".aimini.net", 11) == 0)) {
}
if (flow->detected_protocol_stack[0] == NDPI_PROTOCOL_HTTP) {
if (flow->http.method == NDPI_HTTP_METHOD_GET)
{
if ((LINE_STARTS(packet->http_url_name, "/download/") == 1 ||
LINE_STARTS(packet->http_url_name, "/player/") == 1 ||
LINE_STARTS(packet->http_url_name, "/play/") == 1 ||
LINE_STARTS(packet->http_url_name, "/member/") == 1) &&
(LINE_ENDS(packet->host_line, ".aimini.net") == 1 ||
LINE_ENDS(packet->host_line, ".aimini.com") == 1))
{
NDPI_LOG_INFO(ndpi_struct, "found AIMINI HTTP traffic\n");
ndpi_int_aimini_add_connection(ndpi_struct, flow);
return;
}
}
if (packet->payload_packet_len > 100) {
if (memcmp(packet->payload, "GET /", NDPI_STATICSTRING_LEN("GET /")) == 0) {
if (memcmp(&packet->payload[NDPI_STATICSTRING_LEN("GET /")], "play/",
NDPI_STATICSTRING_LEN("play/")) == 0 ||
memcmp(&packet->payload[NDPI_STATICSTRING_LEN("GET /")], "download/",
NDPI_STATICSTRING_LEN("download/")) == 0) {
ndpi_parse_packet_line_info(ndpi_struct, flow);
if (is_special_aimini_host(packet->host_line) == 1) {
NDPI_LOG_INFO(ndpi_struct,
"found AIMINI HTTP traffic\n");
ndpi_int_aimini_add_connection(ndpi_struct, flow);
return;
}
}
} else if (memcmp(packet->payload, "POST /", NDPI_STATICSTRING_LEN("POST /")) == 0) {
if (memcmp(&packet->payload[NDPI_STATICSTRING_LEN("POST /")], "upload/",
NDPI_STATICSTRING_LEN("upload/")) == 0) {
ndpi_parse_packet_line_info(ndpi_struct, flow);
if (is_special_aimini_host(packet->host_line) == 1) {
NDPI_LOG_INFO(ndpi_struct,
"found AIMINI HTTP traffic detected.\n");
ndpi_int_aimini_add_connection(ndpi_struct, flow);
return;
}
}
} else if (flow->http.method == NDPI_HTTP_METHOD_POST)
{
if ((LINE_STARTS(packet->http_url_name, "/upload/") == 1 ||
LINE_STARTS(packet->http_url_name, "/member/") == 1) &&
(LINE_ENDS(packet->host_line, ".aimini.net") == 1 ||
LINE_ENDS(packet->host_line, ".aimini.com") == 1))
{
NDPI_LOG_INFO(ndpi_struct, "found AIMINI HTTP traffic\n");
ndpi_int_aimini_add_connection(ndpi_struct, flow);
return;
}
}
}
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* amazon_video.c
*
* Copyright (C) 2018-20 by ntop.org
* Copyright (C) 2018-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+58
View File
@@ -0,0 +1,58 @@
/*
* among_us.c
*
* Copyright (C) 2020 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#include "ndpi_protocol_ids.h"
#define NDPI_CURRENT_PROTO NDPI_PROTOCOL_AMONG_US
#include "ndpi_api.h"
static void ndpi_int_among_us_add_connection(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow)
{
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_AMONG_US, NDPI_PROTOCOL_UNKNOWN);
}
void ndpi_search_among_us(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow)
{
struct ndpi_packet_struct * const packet = &flow->packet;
/* handshake packet */
if (packet->payload_packet_len > 9 &&
ntohl(*(u_int32_t*)&packet->payload[0]) == 0x08000100 &&
ntohl(*(u_int32_t*)&packet->payload[4]) == 0x80d90203)
{
ndpi_int_among_us_add_connection(ndpi_struct, flow);
} else {
NDPI_EXCLUDE_PROTO(ndpi_struct, flow);
}
}
void init_among_us_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id,
NDPI_PROTOCOL_BITMASK *detection_bitmask)
{
ndpi_set_bitmask_protocol_detection(
"AmongUs", ndpi_struct, detection_bitmask, *id,
NDPI_PROTOCOL_AMONG_US, ndpi_search_among_us, NDPI_SELECTION_BITMASK_PROTOCOL_V4_V6_UDP_WITH_PAYLOAD,
SAVE_DETECTION_BITMASK_AS_UNKNOWN, ADD_TO_DETECTION_BITMASK);
*id += 1;
}
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* amqp.c
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-21 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* applejuice.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* armagetron.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* flash.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* ftp.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* manolito.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* popo.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* secondlife.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+68
View File
@@ -0,0 +1,68 @@
/*
* avast.c
*
* Copyright (C) 2012-21 - ntop.org
*
* This module is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This module is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License.
* If not, see <http://www.gnu.org/licenses/>.
*
*/
#include "ndpi_protocol_ids.h"
#define NDPI_CURRENT_PROTO NDPI_PROTOCOL_AVAST_SECUREDNS
#include <stdlib.h>
#include "ndpi_api.h"
static void ndpi_int_avast_securedns_add_connection(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow)
{
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_AVAST_SECUREDNS, NDPI_PROTOCOL_UNKNOWN);
}
static void ndpi_search_avast_securedns(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow)
{
struct ndpi_packet_struct * packet = &flow->packet;
if (packet->payload_packet_len < 34 ||
ntohl(get_u_int32_t(packet->payload, 11)) != 0x00013209 ||
flow->packet_counter > 1)
{
NDPI_EXCLUDE_PROTO(ndpi_struct, flow);
return;
}
if (strncasecmp((char *)&packet->payload[15], "securedns", strlen("securedns")) == 0)
{
ndpi_int_avast_securedns_add_connection(ndpi_struct, flow);
return;
}
NDPI_EXCLUDE_PROTO(ndpi_struct, flow);
}
void init_avast_securedns_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id,
NDPI_PROTOCOL_BITMASK *detection_bitmask)
{
ndpi_set_bitmask_protocol_detection("AVAST SecureDNS",
ndpi_struct, detection_bitmask, *id,
NDPI_PROTOCOL_AVAST_SECUREDNS,
ndpi_search_avast_securedns,
NDPI_SELECTION_BITMASK_PROTOCOL_UDP_WITH_PAYLOAD,
SAVE_DETECTION_BITMASK_AS_UNKNOWN,
ADD_TO_DETECTION_BITMASK);
*id += 1;
}
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* ayiya.c
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* bgp.c
*
* Copyright (C) 2016-20 - ntop.org
* Copyright (C) 2016-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+7 -2
View File
@@ -1,8 +1,8 @@
/*
* bittorrent.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -493,6 +493,11 @@ void ndpi_search_bittorrent(struct ndpi_detection_module_struct *ndpi_struct, st
NDPI_EXCLUDE_PROTO(ndpi_struct, flow);
}
}
if(flow->packet_counter > 8) {
NDPI_EXCLUDE_PROTO(ndpi_struct, flow);
return;
}
}
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* btlib.c
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-21 - ntop.org
* Contributed by Vitaly Lavrov <vel21ripn@gmail.com>
*
* This file is part of nDPI, an open source deep packet inspection
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* checkmk.c
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+12 -13
View File
@@ -1,7 +1,7 @@
/*
* ciscovpn.c
*
* Copyright (C) 2013-20 - ntop.org
* Copyright (C) 2013-21 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
@@ -57,7 +57,7 @@ void ndpi_search_ciscovpn(struct ndpi_detection_module_struct *ndpi_struct, stru
if((tdport == 10000 && tsport == 10000) ||
((tsport == 443 || tdport == 443) &&
(packet->payload_packet_len >= 4) &&
(packet->payload[0] == 0x17 &&
(packet->payload[0] == 0x17 /* TLS Application Data */ &&
packet->payload[1] == 0x01 &&
packet->payload[2] == 0x00 &&
packet->payload[3] == 0x00)
@@ -68,28 +68,27 @@ void ndpi_search_ciscovpn(struct ndpi_detection_module_struct *ndpi_struct, stru
ndpi_int_ciscovpn_add_connection(ndpi_struct, flow);
return;
}
#if 0
/* Code disabled as it is too generic and it can lead to false positives */
else if(((tsport == 443 || tdport == 443) ||
(tsport == 80 || tdport == 80)) &&
(packet->payload_packet_len >= 5) &&
((packet->payload[0] == 0x17 &&
packet->payload[1] == 0x03 &&
packet->payload[2] == 0x03 &&
packet->payload[3] == 0x00 &&
packet->payload[4] == 0x3A)))
((packet->payload[0] == 0x17 /* TLS Application Data */ &&
packet->payload[1] == 0x03 && packet->payload[2] == 0x03 && /* TLS 1.2 */
packet->payload[3] == 0x00 && packet->payload[4] == 0x3A /* Length */)))
{
/* TLS signature of Cisco AnyConnect 0X170303003A */
NDPI_LOG_INFO(ndpi_struct, "found CISCO Anyconnect VPN\n");
ndpi_int_ciscovpn_add_connection(ndpi_struct, flow);
return;
}
#endif
else if(((tsport == 8009 || tdport == 8009) ||
(tsport == 8008 || tdport == 8008)) &&
(packet->payload_packet_len >= 5) &&
((packet->payload[0] == 0x17 &&
packet->payload[1] == 0x03 &&
packet->payload[2] == 0x03 &&
packet->payload[3] == 0x00 &&
packet->payload[4] == 0x69)))
((packet->payload[0] == 0x17 /* TLS Application Data */ &&
packet->payload[1] == 0x03 && packet->payload[2] == 0x03 && /* TLS 1.2 */
packet->payload[3] == 0x00 && packet->payload[4] == 0x69 /* Length */)))
{
/* TCP signature of Cisco AnyConnect 0X1703030069 */
NDPI_LOG_INFO(ndpi_struct, "found CISCO Anyconnect VPN\n");
@@ -116,7 +115,7 @@ void ndpi_search_ciscovpn(struct ndpi_detection_module_struct *ndpi_struct, stru
(usport == 443 || udport == 443)
&&
(packet->payload_packet_len >= 5) &&
(packet->payload[0] == 0x17 &&
(packet->payload[0] == 0x17 /* TLS Application Data */ &&
packet->payload[1] == 0x01 &&
packet->payload[2] == 0x00 &&
packet->payload[3] == 0x00 &&
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* citrix.c
*
* Copyright (C) 2012-20 - ntop.org
* Copyright (C) 2012-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* collectd.c
*
* Copyright (C) 2014-20 - ntop.org
* Copyright (C) 2014-21 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
+64
View File
@@ -0,0 +1,64 @@
/*
* qq.c
*
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#include "ndpi_protocol_ids.h"
/* CPHA - CheckPoint High Availability Protocol */
#define NDPI_CURRENT_PROTO NDPI_PROTOCOL_CPHA
#include "ndpi_api.h"
void ndpi_search_cpha(struct ndpi_detection_module_struct *ndpi_struct, struct ndpi_flow_struct *flow) {
struct ndpi_packet_struct *packet = &flow->packet;
const u_int16_t cpha_port = htons(8116);
NDPI_LOG_DBG(ndpi_struct, "search CPHA\n");
if((packet->payload_packet_len > 20)
&& (packet->payload[0] == 0x1a)
&& (packet->payload[1] == 0x90)
&& packet->udp
&& packet->iph
&& (packet->udp->source == cpha_port)
&& (packet->udp->dest == cpha_port)
&& packet->iph->saddr == 0 /* 0.0.0.0 */
) {
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_CPHA, NDPI_PROTOCOL_UNKNOWN);
} else
NDPI_EXCLUDE_PROTO(ndpi_struct, flow);
}
void init_cpha_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id,
NDPI_PROTOCOL_BITMASK *detection_bitmask) {
ndpi_set_bitmask_protocol_detection("CPHA", ndpi_struct, detection_bitmask, *id,
NDPI_PROTOCOL_CPHA,
ndpi_search_cpha,
NDPI_SELECTION_BITMASK_PROTOCOL_UDP_WITH_PAYLOAD,
SAVE_DETECTION_BITMASK_AS_UNKNOWN,
ADD_TO_DETECTION_BITMASK);
*id += 1;
}
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* crossfire.c
*
* Copyright (C) 2012-20 - ntop.org
* Copyright (C) 2012-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+44 -8
View File
@@ -26,7 +26,7 @@
#define NDPI_CURRENT_PROTO NDPI_PROTOCOL_DCERPC
#include "ndpi_api.h"
#include <stdbool.h>
static void ndpi_int_dcerpc_add_connection(struct ndpi_detection_module_struct
*ndpi_struct, struct ndpi_flow_struct *flow)
@@ -34,18 +34,55 @@ static void ndpi_int_dcerpc_add_connection(struct ndpi_detection_module_struct
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_DCERPC, NDPI_PROTOCOL_UNKNOWN);
}
void ndpi_search_dcerpc(struct ndpi_detection_module_struct *ndpi_struct, struct ndpi_flow_struct *flow)
bool is_connection_oriented_dcerpc(struct ndpi_packet_struct *packet, struct ndpi_flow_struct *flow)
{
struct ndpi_packet_struct *packet = &flow->packet;
NDPI_LOG_DBG(ndpi_struct, "search DCERPC\n");
if((packet->tcp != NULL)
&& (packet->payload_packet_len >= 64)
&& (packet->payload[0] == 0x05) /* version 5 */
&& (packet->payload[2] < 16) /* Packet type */
&& (((packet->payload[9]<<8) | packet->payload[8]) == packet->payload_packet_len) /* Packet Length */
) {
return true;
}
return false;
}
bool is_connectionless_dcerpc(struct ndpi_packet_struct *packet, struct ndpi_flow_struct *flow)
{
u_int16_t fragment_len;
if (packet->udp == NULL)
return false;
if (packet->payload_packet_len < 80)
return false;
if (packet->payload[0] != 0x04) /* type must be equal to 4 */
return false;
if (packet->payload[1] > 10) /* must be <= CANCEL ACK or it's not connectionless DCE/RPC */
return false;
if (packet->payload[3] & 0xFC) /* flags2: bit 3:8 are reserved for future use and must be set to 0 */
return false;
if (packet->payload[4] & 0xEE) /* neither big endian nor little endian */
return false;
if (packet->payload[5] > 3) /* invalid floating point type */
return false;
if(packet->payload[4] == 0x10)
fragment_len = (packet->payload[75] << 8) + packet->payload[74]; /* Big endian */
else
fragment_len = (packet->payload[74] << 8) + packet->payload[75]; /* Little endian */
if(packet->payload_packet_len != (fragment_len+76 /* offset */ + 4 /* rest of the packet */))
return false; /* Too short or too long, bot RPC */
return true;
}
void ndpi_search_dcerpc(struct ndpi_detection_module_struct *ndpi_struct, struct ndpi_flow_struct *flow)
{
struct ndpi_packet_struct *packet = &flow->packet;
NDPI_LOG_DBG(ndpi_struct, "search DCERPC\n");
if (is_connection_oriented_dcerpc(packet, flow) || is_connectionless_dcerpc(packet, flow)) {
NDPI_LOG_INFO(ndpi_struct, "found DCERPC\n");
ndpi_int_dcerpc_add_connection(ndpi_struct, flow);
return;
@@ -61,9 +98,8 @@ void init_dcerpc_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_i
ndpi_set_bitmask_protocol_detection("DCE_RPC", ndpi_struct, detection_bitmask, *id,
NDPI_PROTOCOL_DCERPC,
ndpi_search_dcerpc,
NDPI_SELECTION_BITMASK_PROTOCOL_V4_V6_TCP_WITH_PAYLOAD_WITHOUT_RETRANSMISSION,
NDPI_SELECTION_BITMASK_PROTOCOL_V4_V6_TCP_OR_UDP_WITH_PAYLOAD_WITHOUT_RETRANSMISSION,
SAVE_DETECTION_BITMASK_AS_UNKNOWN,
ADD_TO_DETECTION_BITMASK);
*id += 1;
}
+4 -4
View File
@@ -1,7 +1,7 @@
/*
* dhcp.c
*
* Copyright (C) 2016-20 - ntop.org
* Copyright (C) 2016-21 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
@@ -66,18 +66,18 @@ void ndpi_search_dhcp_udp(struct ndpi_detection_module_struct *ndpi_struct, stru
/* this detection also works for asymmetric dhcp traffic */
/*check standard DHCP 0.0.0.0:68 -> 255.255.255.255:67 */
/* check standard DHCP 0.0.0.0:68 -> 255.255.255.255:67 */
if(packet->udp) {
dhcp_packet_t *dhcp = (dhcp_packet_t*)packet->payload;
if((packet->payload_packet_len >= 240 /* 240 is the offset of options[0] in dhcp_packet_t */)
if((packet->payload_packet_len >= 244 /* 244 is the offset of options[0] in dhcp_packet_t */)
&& (packet->udp->source == htons(67) || packet->udp->source == htons(68))
&& (packet->udp->dest == htons(67) || packet->udp->dest == htons(68))
&& (dhcp->magic == htonl(DHCP_OPTION_MAGIC_NUMBER))) {
u_int i = 0, foundValidMsgType = 0;
u_int dhcp_options_size = ndpi_min(DHCP_VEND_LEN /* maximum size of options in dhcp_packet_t */,
packet->payload_packet_len - 240);
packet->payload_packet_len - 244);
while(i + 1 /* for the len */ < dhcp_options_size) {
u_int8_t id = dhcp->options[i];
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* dhcpv6.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* directconnect.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* directdownloadlink.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* dnp3.c
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-21 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
+54 -30
View File
@@ -1,7 +1,7 @@
/*
* dns.c
*
* Copyright (C) 2012-20 - ntop.org
* Copyright (C) 2012-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -29,7 +29,7 @@
#define FLAGS_MASK 0x8000
/* #define DNS_DEBUG 1 */
// #define DNS_DEBUG 1
#define DNS_PORT 53
#define LLMNR_PORT 5355
@@ -89,7 +89,7 @@ static void ndpi_check_dns_type(struct ndpi_detection_module_struct *ndpi_struct
case 106:
case 107:
case 259:
NDPI_SET_BIT(flow->risk, NDPI_DNS_SUSPICIOUS_TRAFFIC);
ndpi_set_risk(ndpi_struct, flow, NDPI_DNS_SUSPICIOUS_TRAFFIC);
break;
}
}
@@ -194,14 +194,16 @@ static int search_valid_dns(struct ndpi_detection_module_struct *ndpi_struct,
else if((dns_header->flags & FLAGS_MASK) == 0x8000)
*is_query = 0;
else {
NDPI_SET_BIT(flow->risk, NDPI_MALFORMED_PACKET);
ndpi_set_risk(ndpi_struct, flow, NDPI_MALFORMED_PACKET);
return(1 /* invalid */);
}
if(*is_query) {
/* DNS Request */
if((dns_header->num_queries > 0) && (dns_header->num_queries <= NDPI_MAX_DNS_REQUESTS)
// && (dns_header->num_answers == 0)
&& (((dns_header->flags & 0x2800) == 0x2800 /* Dynamic DNS Update */)
|| ((dns_header->flags & 0xFCF0) == 0x00) /* Standard Query */
|| ((dns_header->num_answers == 0) && (dns_header->authority_rrs == 0)))) {
/* This is a good query */
while(x+2 < flow->packet.payload_packet_len) {
@@ -217,7 +219,7 @@ static int search_valid_dns(struct ndpi_detection_module_struct *ndpi_struct,
x++;
}
} else {
NDPI_SET_BIT(flow->risk, NDPI_MALFORMED_PACKET);
ndpi_set_risk(ndpi_struct, flow, NDPI_MALFORMED_PACKET);
return(1 /* invalid */);
}
} else {
@@ -273,7 +275,7 @@ static int search_valid_dns(struct ndpi_detection_module_struct *ndpi_struct,
#endif
ndpi_check_dns_type(ndpi_struct, flow, rsp_type);
flow->protos.dns.rsp_type = rsp_type;
/* here x points to the response "class" field */
@@ -290,9 +292,7 @@ static int search_valid_dns(struct ndpi_detection_module_struct *ndpi_struct,
}
if((((rsp_type == 0x1) && (data_len == 4)) /* A */
#ifdef NDPI_DETECTION_SUPPORT_IPV6
|| ((rsp_type == 0x1c) && (data_len == 16)) /* AAAA */
#endif
)) {
memcpy(&flow->protos.dns.rsp_addr, flow->packet.payload + x, data_len);
}
@@ -376,44 +376,68 @@ static void ndpi_search_dns(struct ndpi_detection_module_struct *ndpi_struct, st
/* Before continuing let's dissect the following queries to see if they are valid */
for(idx=off, num_queries=0; (num_queries < dns_header.num_queries) && (idx < flow->packet.payload_packet_len);) {
u_int8_t name_len = flow->packet.payload[idx];
u_int16_t i, tot_len = 0;
for(i=idx; i<flow->packet.payload_packet_len;) {
u_int8_t is_ptr = 0, name_len = flow->packet.payload[i]; /* Lenght of the individual name blocks aaa.bbb.com */
if(name_len == 0) {
tot_len++; /* \0 */
/* End of query */
break;
} else if((name_len & 0xC0) == 0xC0)
is_ptr = 1, name_len = 0; /* Pointer */
#ifdef DNS_DEBUG
printf("[DNS] [name_len: %u]\n", name_len);
if((!is_ptr) && (name_len > 0)) {
printf("[DNS] [name_len: %u][", name_len);
{
int idx;
for(idx=0; idx<name_len; idx++)
printf("%c", flow->packet.payload[i+1+idx]);
printf("]\n");
}
}
#endif
i += name_len+1, tot_len += name_len+1;
if(is_ptr) break;
} /* for */
#ifdef DNS_DEBUG
printf("[DNS] [tot_len: %u]\n\n", tot_len+4 /* type + class */);
#endif
if(name_len == 0) {
/* End of query */
num_queries++;
idx += 5;
continue;
}
if((name_len+idx) >= flow->packet.payload_packet_len) {
if(((i+4 /* Skip query type and class */) > flow->packet.payload_packet_len)
|| ((flow->packet.payload[i+1] == 0x0) && (flow->packet.payload[i+2] == 0x0)) /* Query type cannot be 0 */
|| (tot_len > 253)
) {
/* Invalid */
#ifdef DNS_DEBUG
printf("[DNS] Invalid query len [%u >= %u]\n",
(name_len+idx),
flow->packet.payload_packet_len);
printf("[DNS] Invalid query len [%u >= %u]\n", i+4, flow->packet.payload_packet_len);
#endif
NDPI_SET_BIT(flow->risk, NDPI_MALFORMED_PACKET);
ndpi_set_risk(ndpi_struct, flow, NDPI_MALFORMED_PACKET);
break;
} else
idx += name_len+1;
}
} else {
idx = i+5, num_queries++;
}
} /* for */
while(j < max_len && off < flow->packet.payload_packet_len && flow->packet.payload[off] != '\0') {
while((j < max_len) && (off < flow->packet.payload_packet_len) && (flow->packet.payload[off] != '\0')) {
uint8_t c, cl = flow->packet.payload[off++];
if( (cl & 0xc0) != 0 || // we not support compressed names in query
off + cl >= flow->packet.payload_packet_len) {
if(((cl & 0xc0) != 0) || // we not support compressed names in query
(off + cl >= flow->packet.payload_packet_len)) {
j = 0;
break;
}
if(j && j < max_len) flow->host_server_name[j++] = '.';
if(j && (j < max_len)) flow->host_server_name[j++] = '.';
while(j < max_len && cl != 0) {
while((j < max_len) && (cl != 0)) {
u_int32_t shift;
c = flow->packet.payload[off++];
+14 -5
View File
@@ -38,22 +38,31 @@ void ndpi_search_dnscrypt(struct ndpi_detection_module_struct *ndpi_struct,
NDPI_LOG_DBG(ndpi_struct, "search dnscrypt\n");
if (flow->packet_counter > 2)
{
NDPI_EXCLUDE_PROTO(ndpi_struct, flow);
}
/* dnscrypt protocol version 1: check magic */
if (packet->payload_packet_len >= 64 &&
strncmp((char*)packet->payload, "r6fnvWj8", strlen("r6fnvWj8")) == 0)
{
ndpi_int_dnscrypt_add_connection(ndpi_struct, flow);
return;
}
/* dnscrypt protocol version 1 and 2: resolver ping */
if (packet->payload_packet_len > 13 + strlen(dnscrypt_initial) &&
strncasecmp((char*)packet->payload + 13, dnscrypt_initial, strlen(dnscrypt_initial)) == 0)
{
ndpi_int_dnscrypt_add_connection(ndpi_struct, flow);
return;
}
if ((flow->packet_direction_counter[packet->packet_direction] >= 1 &&
flow->packet_direction_counter[1 - packet->packet_direction] >= 1) ||
flow->packet_counter >= 10)
{
/*
* Wait for at least one packet per direction, but not more then 10 packets.
* Required as we need to wait for the server response which contains the ASCII pattern below.
*/
NDPI_EXCLUDE_PROTO(ndpi_struct, flow);
return;
}
}
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* dofus.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* drda.c
*
* Copyright (C) 2012-20 - ntop.org
* Copyright (C) 2012-21 - ntop.org
*
* This module is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* eaq.c
*
* Copyright (C) 2015-20 - ntop.org
* Copyright (C) 2015-21 - ntop.org
*
* This module is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* fasttrack.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
-134
View File
@@ -1,134 +0,0 @@
/*
* fbzero.c
*
* Copyright (C) 2018 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#include "ndpi_protocol_ids.h"
/* https://code.facebook.com/posts/608854979307125/building-zero-protocol-for-fast-secure-mobile-connections/ */
#define NDPI_CURRENT_PROTO NDPI_PROTOCOL_FBZERO
#include "ndpi_api.h"
PACK_ON
struct fbzero_tag {
char tag[4];
u_int32_t tag_offset_len;
} PACK_OFF;
PACK_ON
struct fbzero_header {
u_int8_t flags;
char version[3];
u_int8_t unknown;
u_int32_t len;
char tag[4];
u_int16_t tag_number, _pad;
} PACK_OFF;
/* **************************************************************************** */
void ndpi_search_fbzero(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow) {
struct ndpi_packet_struct *packet = &flow->packet;
NDPI_LOG_DBG(ndpi_struct, "search FacebookZero\n");
if(packet->payload_packet_len > sizeof(struct fbzero_header)) {
struct fbzero_header *h = (struct fbzero_header*)packet->payload;
struct fbzero_tag *t;
u_int offset = sizeof(struct fbzero_header), i, data_offset, tag_number, data_prev_offset = 0;
if((h->flags & 0x01) == 0)
goto fbzero_not_found;
if((h->version[0] != 'Q')
|| (h->version[1] != 'T')
|| (h->version[2] != 'V'))
goto fbzero_not_found;
if(h->unknown != 0x30)
goto fbzero_not_found;
t = (struct fbzero_tag*)&packet->payload[offset];
tag_number = h->tag_number;
data_offset = offset + tag_number*sizeof(struct fbzero_tag);
if(strncmp(h->tag, "CHLO", 4))
goto fbzero_not_found;
for(i=0; i<h->tag_number; i++) {
#ifdef DEBUG
printf("[FBZERO] %u) %c%c%c%c\n", i, t->tag[0], t->tag[1], t->tag[2], t->tag[3]);
#endif
offset += sizeof(struct fbzero_tag);
if((t->tag[0] == 'S') && (t->tag[1] == 'N')
&& (t->tag[2] == 'I') && (t->tag[3] == '\0')) {
char *value = (char*)&packet->payload[data_offset + data_prev_offset];
u_int tag_len = t->tag_offset_len-data_prev_offset, max_len;
ndpi_protocol_match_result ret_match;
max_len = ndpi_min(tag_len, sizeof(flow->host_server_name)-1);
if (data_offset + data_prev_offset + max_len >= packet->payload_packet_len) {
return;
}
strncpy((char*)flow->host_server_name, value, max_len);
flow->host_server_name[max_len] = '\0';
#ifdef DEBUG
printf("[FBZERO] SNI [@%u][len: %u][%s]\n", tag_len,
t->tag_offset_len-data_prev_offset, flow->host_server_name);
#endif
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_FBZERO, NDPI_PROTOCOL_UNKNOWN);
ndpi_match_host_subprotocol(ndpi_struct, flow, (char *)flow->host_server_name,
strlen((const char *)flow->host_server_name),
&ret_match,
NDPI_PROTOCOL_FBZERO);
return;
}
data_prev_offset = t->tag_offset_len;
t = (struct fbzero_tag*)&packet->payload[offset];
}
return;
}
fbzero_not_found:
NDPI_EXCLUDE_PROTO(ndpi_struct, flow);
}
/* **************************************************************************** */
void init_fbzero_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id,
NDPI_PROTOCOL_BITMASK *detection_bitmask) {
ndpi_set_bitmask_protocol_detection("FacebookZero", ndpi_struct, detection_bitmask, *id,
NDPI_PROTOCOL_FBZERO,
ndpi_search_fbzero,
NDPI_SELECTION_BITMASK_PROTOCOL_V4_V6_TCP_WITH_PAYLOAD_WITHOUT_RETRANSMISSION,
SAVE_DETECTION_BITMASK_AS_UNKNOWN,
ADD_TO_DETECTION_BITMASK);
*id += 1;
}
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* fiesta.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* fix.c
*
* Copyright (C) 2017-20 - ntop.org
* Copyright (C) 2017-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* florensia.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+4 -4
View File
@@ -1,7 +1,7 @@
/*
* ftp_control.c
*
* Copyright (C) 2016-20 - ntop.org
* Copyright (C) 2016-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -590,8 +590,8 @@ static void ndpi_check_ftp_control(struct ndpi_detection_module_struct *ndpi_str
return;
}
/* Break after 20 packets. */
if(flow->packet_counter > 20) {
/* Break after 8 packets. */
if(flow->packet_counter > 8) {
NDPI_EXCLUDE_PROTO(ndpi_struct, flow);
return;
}
@@ -666,7 +666,7 @@ void init_ftp_control_dissector(struct ndpi_detection_module_struct *ndpi_struct
ndpi_set_bitmask_protocol_detection("FTP_CONTROL", ndpi_struct, detection_bitmask, *id,
NDPI_PROTOCOL_FTP_CONTROL,
ndpi_search_ftp_control,
NDPI_SELECTION_BITMASK_PROTOCOL_V4_V6_TCP_WITHOUT_RETRANSMISSION,
NDPI_SELECTION_BITMASK_PROTOCOL_V4_V6_TCP_WITH_PAYLOAD_WITHOUT_RETRANSMISSION,
SAVE_DETECTION_BITMASK_AS_UNKNOWN,
ADD_TO_DETECTION_BITMASK);
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* ftp_data.c
*
* Copyright (C) 2016-20 - ntop.org
* Copyright (C) 2016-21 - ntop.org
*
* The signature is based on the Libprotoident library.
*
+73
View File
@@ -0,0 +1,73 @@
/*
* genshin_impact.c
*
* Copyright (C) 2012-21 - ntop.org
*
* This module is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This module is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License.
* If not, see <http://www.gnu.org/licenses/>.
*
*/
#include "ndpi_protocol_ids.h"
#define NDPI_CURRENT_PROTO NDPI_PROTOCOL_GENSHIN_IMPACT
#include "ndpi_api.h"
static void ndpi_int_genshin_impact_add_connection(
struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow)
{
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_GENSHIN_IMPACT, NDPI_PROTOCOL_UNKNOWN);
}
static void ndpi_search_genshin_impact(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow)
{
struct ndpi_packet_struct * packet = &flow->packet;
NDPI_LOG_DBG(ndpi_struct, "search genshin-impact\n");
if (packet->udp != NULL)
{
if (flow->packet_counter == 1 && packet->payload_packet_len >= 20 &&
ntohl(*(u_int32_t*)&packet->payload[0]) == 0x000000FF &&
ntohl(*(u_int32_t*)&packet->payload[4]) == 0x00000000 &&
ntohl(*(u_int32_t*)&packet->payload[12]) == 0x499602D2 &&
ntohl(*(u_int32_t*)&packet->payload[16]) == 0xFFFFFFFF)
{
ndpi_int_genshin_impact_add_connection(ndpi_struct, flow);
return;
}
}
NDPI_EXCLUDE_PROTO(ndpi_struct, flow);
}
/* ***************************************************************** */
void init_genshin_impact_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id,
NDPI_PROTOCOL_BITMASK *detection_bitmask)
{
ndpi_set_bitmask_protocol_detection("Genshin Impact",
ndpi_struct, detection_bitmask, *id,
NDPI_PROTOCOL_GENSHIN_IMPACT,
ndpi_search_genshin_impact,
NDPI_SELECTION_BITMASK_PROTOCOL_UDP_WITH_PAYLOAD,
SAVE_DETECTION_BITMASK_AS_UNKNOWN,
ADD_TO_DETECTION_BITMASK);
*id += 1;
}
+1 -2
View File
@@ -1,7 +1,7 @@
/*
* git.c
*
* Copyright (C) 2012-20 - ntop.org
* Copyright (C) 2012-21 - ntop.org
*
* This module is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
@@ -22,7 +22,6 @@
#define NDPI_CURRENT_PROTO NDPI_PROTOCOL_GIT
#include <stdlib.h>
#include "ndpi_api.h"
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* gnutella.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+67 -18
View File
@@ -1,7 +1,7 @@
/*
* gtp.c
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-21 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
@@ -24,38 +24,87 @@
#include "ndpi_api.h"
/* This code handles: GTP-U (port 2152), GTP-C (v1 and v2; port 2123) and GTP-PRIME
(port 3386).
It should be fine to ignore v0, since it should not be used anymore.
Message length checks and basic headers are not uniform across these protocols.
For GTPv2 (GTP-C v2), see TS 29.274 Sec. 5.5.1:
"Octets 3 to 4 represent the Message Length field. This field shall indicate
the length of the message in octets excluding the mandatory part of the GTP-C
header (the first 4 octets). The TEID (if present) and the Sequence Number
shall be included in the length count."
For GTP-PRIME TS 32.295 Sec. 6.1.1
"The Length indicates the length of payload (number of octets after the GTP'
header). The Sequence Number of the packet is part of the GTP' header."
For GTPv1 (GTP-U and GTP-C v1) TS TS 29.060 Sec. 6
"Length: This field indicates the length in octets of the payload, i.e. the
rest of the packet following the mandatory part of the GTP header (that is
the first 8 octets). The Sequence Number, the N-PDU Number or any Extension
headers shall be considered to be part of the payload, i.e. included in the
length count."
*/
#define HEADER_LEN_GTP_U 8
#define HEADER_LEN_GTP_C_V1 8
#define HEADER_LEN_GTP_C_V2 4
#define HEADER_LEN_GTP_PRIME 6
/* Common header for all GTP types */
struct gtp_header_generic {
u_int8_t flags, message_type;
u_int16_t message_len;
u_int32_t teid;
};
static void ndpi_check_gtp(struct ndpi_detection_module_struct *ndpi_struct, struct ndpi_flow_struct *flow)
{
struct ndpi_packet_struct *packet = &flow->packet;
// const u_int8_t *packet_payload = packet->payload;
u_int32_t payload_len = packet->payload_packet_len;
if((packet->udp != NULL) && (payload_len > sizeof(struct gtp_header_generic))) {
u_int32_t gtp_u = ntohs(2152);
u_int32_t gtp_c = ntohs(2123);
u_int32_t gtp_v0 = ntohs(3386);
u_int32_t gtp_prime = ntohs(3386);
if((packet->udp->source == gtp_u) || (packet->udp->dest == gtp_u)
|| (packet->udp->source == gtp_c) || (packet->udp->dest == gtp_c)
|| (packet->udp->source == gtp_v0) || (packet->udp->dest == gtp_v0)
) {
struct gtp_header_generic *gtp = (struct gtp_header_generic*)packet->payload;
u_int8_t gtp_version = (gtp->flags & 0xE0) >> 5;
struct gtp_header_generic *gtp = (struct gtp_header_generic *)packet->payload;
u_int8_t version = (gtp->flags & 0xE0) >> 5;
u_int8_t pt = (gtp->flags & 0x10) >> 4;
u_int16_t message_len = ntohs(gtp->message_len);
if((gtp_version == 0) || (gtp_version == 1) || (gtp_version == 2)) {
u_int16_t message_len = ntohs(gtp->message_len);
if(message_len <= (payload_len-sizeof(struct gtp_header_generic))) {
NDPI_LOG_INFO(ndpi_struct, "found gtp\n");
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_GTP, NDPI_PROTOCOL_UNKNOWN);
return;
}
if((packet->udp->source == gtp_u) || (packet->udp->dest == gtp_u)) {
if((version == 1) && (pt == 1) &&
(payload_len >= HEADER_LEN_GTP_U) &&
(message_len <= (payload_len - HEADER_LEN_GTP_U))) {
NDPI_LOG_INFO(ndpi_struct, "found gtp-u\n");
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_GTP, NDPI_PROTOCOL_UNKNOWN);
return;
}
}
if((packet->udp->source == gtp_c) || (packet->udp->dest == gtp_c)) {
if(((version == 1) &&
(payload_len >= HEADER_LEN_GTP_C_V1) &&
(message_len == (payload_len - HEADER_LEN_GTP_C_V1))) ||
((version == 2) &&
/* payload_len is always valid, because HEADER_LEN_GTP_C_V2 == sizeof(struct gtp_header_generic) */
(message_len <= (payload_len - HEADER_LEN_GTP_C_V2)))) {
NDPI_LOG_INFO(ndpi_struct, "found gtp-c\n");
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_GTP, NDPI_PROTOCOL_UNKNOWN);
return;
}
}
if((packet->udp->source == gtp_prime) || (packet->udp->dest == gtp_prime)) {
if((pt == 0) &&
(payload_len >= HEADER_LEN_GTP_PRIME) &&
(message_len <= (payload_len - HEADER_LEN_GTP_PRIME))) {
NDPI_LOG_INFO(ndpi_struct, "found gtp-prime\n");
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_GTP, NDPI_PROTOCOL_UNKNOWN);
return;
}
}
}
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* guildwars.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
+2 -2
View File
@@ -1,8 +1,8 @@
/*
* halflife2_and_mods.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH

Some files were not shown because too many files have changed in this diff Show More