Compare commits

..
652 Commits
Author SHA1 Message Date
Luca 8b5c6af71b Updated release to 4.2 2022-02-01 09:14:05 +01:00
Alfredo Cardigliano 03cb8efad6 Update changelog 2022-01-31 18:38:02 +01:00
Ivan Nardi 5bb5bec477 Remove struct ndpi_id_struct (#1427)
Remove the last uses of `struct ndpi_id_struct`.
That code is not really used and it has not been updated for a very long
time: see #1279 for details.

Correlation among flows is achieved via LRU caches.

This change allows to further reduce memory consumption (see also
91bb77a8).

At nDPI 4.0 (more precisly, at a6b10cf, because memory stats
were wrong until that commit):
```
nDPI Memory statistics:
	nDPI Memory (once):      221.15 KB
	Flow Memory (per flow):  2.94 KB
```
Now:
```
nDPI Memory statistics:
	nDPI Memory (once):      235.27 KB
	Flow Memory (per flow):  688 B        <--------
```
i.e. memory usage per flow has been reduced by 77%.

Close #1279
2022-01-30 19:18:12 +01:00
Ivan Nardi 1fdcddb33d Remove Playstation VUE protocol (#1426)
PS VUE service has been discontinued on January 30, 2020
https://en.wikipedia.org/wiki/PlayStation_Vue
2022-01-30 19:17:53 +01:00
Luca Deri 5f745ee425 Commented old code (see https://github.com/ntop/nDPI/pull/1425) 2022-01-30 19:17:23 +01:00
Ivan Nardi f455421870 Improve protocol stacks (#1425)
We should have two protocols in classification results only when the
"master" protocol allows some sub-protocols.

Classifications like `AmazonAWS`, `TLS/AmazonAWS`, `DNS/AmazonAWS` are
fine. However classifications like `NTP/Apple`, `BitTorrent/Azure`,
`DNScrypt.AmazonAWS` or `NestLogSink.Google` are misleading.
For example, `ndpiReader`shows  `BitTorrent/Azure` flows under `Azure`
statistics; that seems to be wrong or, at least, very misleading.

This is quite important since we have lots of addresses from CDN
operators.

The only drawback of this solution is that right now ICMP traffic is
classified simply as `ICMP`; if we are really interested in ICMP stuff
we can restore the old behaviour later.
2022-01-30 19:15:42 +01:00
Ivan Nardi 513e386959 Extend protocols support (#1422)
Add detection of AccuWeather site/app and Google Classroom.
Improve detection of Azure, Zattoo, Whatsapp, MQTT and LDAP.

Fix some RX false positives.

Fix some "Uncommon TLS ALPN"-risk false positives.

Fix "confidence" value for some Zoom/Torrent classifications.

Minor fix in Lua script for Wireshark extcap.

Update .gitignore file.

Let GitHub correctly detect the language type of *.inc files.

Zattoo example has been provided by @subhajit-cdot in #1148.
2022-01-29 09:19:26 +01:00
Toni 9b8679a320 Fix some race conditions by using atomic operations. (#1420)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-29 09:18:51 +01:00
Ivan Nardi 0c70411b1b Make some protocols more "big-endian" friendly (#1402)
See #1312
2022-01-29 09:18:32 +01:00
Ivan Nardi 86b97ffb73 Sync unit tests results (#1423)
Fix: 7a3aa41a
2022-01-28 17:17:39 +01:00
Luca 7a3aa41afc Updated alert description case 2022-01-28 15:02:42 +01:00
Toni c53c82d482 Fixed heap overflow in nDPI realloc wrapper if new size < old size. (#1421)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-27 15:29:16 +01:00
Ivan Nardi 9dcc4a2334 Kerberos, TLS, example: fix some memory errors (#1419)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43823
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43921
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43925
2022-01-27 10:59:35 +01:00
Toni 011ee3ecbd Fixed wrong ip tuple comparison. #1386 (#1418)
* Added u32 pads to `union ip_tuple` so btree search should now work as expected.
   The bug caused new flow's when the remote answers, resulting in two Flows per direction. Fail.
 * Fixed a race condition during shutdown phase.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-26 15:47:38 +01:00
Luca Deri 6d185c052e Added support for the .goog Google TLD 2022-01-26 15:22:33 +01:00
Toni 2cd0479204 Serializing empty `risk blocks' pollutes the resulting string. (#1417)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-26 13:36:45 +01:00
Luca Deri 81b33f0d06 Updated test results after the risk NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE has been added 2022-01-26 09:46:46 +01:00
Luca Deri 58a9aff17c Added NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE flow risk
Added ndpi_set_tls_cert_expire_days() API call to modify the number of days for triggering the above alert that by default is set to 30 days
2022-01-26 09:23:23 +01:00
Luca Deri 14e8808f03 Removed pandora.tv from pandora protocol as they are different services that shouldd not be mixed 2022-01-25 19:11:16 +01:00
Luca Deri bbb3b8349f Improved pandora TV detection 2022-01-24 18:28:43 +01:00
Luca Deri b6b4967aa6 Improved Zoom protocol detection 2022-01-23 20:59:36 +01:00
Luca Deri a8fe74e502 Tool for generating automatically the Azure IP list 2022-01-23 18:56:51 +01:00
Alfredo Cardigliano 20b9f01f12 Fix ndpi_serialize_string_int64 2022-01-21 17:35:38 +01:00
Alfredo Cardigliano a0bb0ab0ca Add unit test for ndpi_serialize_string_int64 2022-01-21 13:05:58 +01:00
havsah e16b6a18be Fix Grease values parsing (#1416)
The check for grease was too broad and filtered some valid values.
In particular, the value 257 was skipped because it matched the previous check.
This has been discovered while parsing tests/pcap/443-firefox.pcap
expected ja3:
771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-51-57-47-53-10,0-23-65281-10-11-35-16-5-51-43-13-45-28-21,29-23-24-25-256-257,0
previously generated ja3:
771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-51-57-47-53-10,0-23-65281-10-11-35-16-5-51-43-13-45-28-21,29-23-24-25-256,0

Signed-off-by: Patrick Havelange <patrick.havelange_ext@softathome.com>
2022-01-21 12:40:35 +01:00
Luca Deri 6f336f8307 Added JA3 in risj exceptions 2022-01-20 22:22:56 +01:00
Luca Deri d91477b18d Fixed certificate mismatch check 2022-01-19 23:10:10 +01:00
Luca Deri 771b6606c8 Fixed an issue on CentOS 7 introduced by
https://github.com/ntop/nDPI/commit/c2b7d77784beeae5f86f19e33ba3da5ddce55898
2022-01-19 10:19:58 +01:00
Luca Deri 9eee257be0 Updated confidence type 2022-01-18 22:28:58 +01:00
Ivan Nardi 9f050fa0a6 TLS, H323, examples: fix some memory errors (#1414)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=26880
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=26906
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43782
https://oss-fuzz.com/testcase-detail/6334089358082048
2022-01-18 21:52:37 +01:00
Ivan Nardi bd036f96f9 Netbios, CSGO: fix two memory errors (#1413)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43754
https://oss-fuzz.com/testcase-detail/5329842395021312
2022-01-18 19:11:01 +01:00
Sam JamesandLuca Deri c2b7d77784 build: respect environment options more (#1392)
* build: update m4/ax_pthread.m4 from serial 23 -> serial 31

Update ax_pthread.m4 to the latest version from the autoconf-archive
project.

Signed-off-by: Sam James <sam@gentoo.org>

* build: properly detect AR, CC, RANLIB

It's necessary to be able to override choice of AR/CC/RANLIB and other toolchain
variables/tools for cross-compilation, testing with other toolchains, and
to ensure the compiler chosen by the user is actually used for the build.

Previously, GNU_PREFIX was kind-of used for this but this isn't a standard
variable (at all) and it wasn't applied consistently anyway.

We now use the standard autoconf mechanisms for finding these tools.

(RANLIB is already covered by LT_INIT.)

Signed-off-by: Sam James <sam@gentoo.org>

* build: use $(MAKE)

This ensures that parallel make works correctly, as otherwise, a fresh
make job will be started without the jobserver fd, and hence
not know about its parent, forcing -j1.

* build: respect CPPFLAGS, LDFLAGS

- CPPFLAGS is for the C preprocessor (usually for setting defines)
- LDFLAGS should be placed before objects for certain flags to work
  (e.g. -Wl,--as-needed)

Signed-off-by: Sam James <sam@gentoo.org>

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2022-01-18 14:30:14 +01:00
Ivan Nardi e4a7990a4e H323: fix a use-after-poison error (#1412)
Detected by oss-fuzz
See: https://oss-fuzz.com/testcase-detail/6730505580576768

Fix a function prototype
Update a unit test results
2022-01-17 19:47:27 +01:00
Luca Deri d19bd96555 Improved Badoo detection (missing mobile app domain) 2022-01-17 18:54:27 +01:00
Luca Deri 6877e46273 Added Badoo detection 2022-01-17 15:13:14 +01:00
Luca 6679453d86 Adds some risk exceptions for popular services and domain names
via a new (internal) function named ndpi_add_domain_risk_exceptions()
2022-01-17 09:00:25 +01:00
Ivan Nardi 1ff308af09 TLS: fix a use-of-uninitialized-value error (#1411)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43705
2022-01-16 20:31:48 +01:00
Ivan Nardi 05734f3d31 Zattoo: fix Null-dereference READ with ipv6 traffic (#1410)
Fix: 20b5f6d7
Detected by oss-fux:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43700
2022-01-16 19:28:39 +01:00
Ivan Nardi 611c753da5 XBox, Diameter: fix dissectors initialization (#1405)
These dissectors have *never* been triggered because their registration
functions use the wrong parameter/bitmask.
Diameter code is buggy since the origianl commit (1d108234), while
XBox code since 5266c726.

Fix some false positives in Xbox code.
2022-01-16 16:19:00 +01:00
Luca Deri f3af39ee42 Added performance tests tools 2022-01-16 12:47:56 +01:00
Luca Deri 42d74171b2 Minor cosmetic changes 2022-01-16 12:47:56 +01:00
Luca Deri dd325248f1 Reduced Patricia tree bucket memory footprint 2022-01-16 12:47:56 +01:00
Ivan Nardi e46b508a36 Kerberos: fix use-of-uninitialized-value error (#1409)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43677
2022-01-15 20:53:36 +01:00
Ivan Nardi eb5d7b07af TLS: fix heap-buffer-overflow error (#1408)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43664
2022-01-15 18:47:59 +01:00
Ivan Nardi 63c44a24b6 STUN: fix "confidence" value for some classifications (#1407) 2022-01-15 17:35:06 +01:00
Ivan Nardi 20b5f6d7cc Improve IPv6 support, enabling IPv6 traffic on (almost) all dissectors. (#1406)
Follow-up of 7cba34a1
2022-01-15 17:34:56 +01:00
Ivan Nardi 7f69de0b51 Restore a unit test result (#1403)
Deleted, probably by mistake, in 406ac7e8

Fix Makefile and add compilation of `rrdtool` in CI tests
2022-01-13 23:14:28 +01:00
Luca Deri 533b3ab0e3 FreeBSD fixes 2022-01-13 21:15:21 +01:00
Luca Deri 406ac7e8c8 Added the ability to specify trusted issueDN often used in companies to self-signed certificates
This allows to avoid triggering alerts for trusted albeit private certificate issuers.

Extended the example/protos.txt with the new syntax for specifying trusted issueDN.
Example:

trusted_issuer_dn:"CN=813845657003339838, O=Code42, OU=TEST, ST=MN, C=US"
2022-01-13 19:06:21 +01:00
Vinicius Silva Nogueira dc60cd09c4 fix ahocorasick on big-endian machines (#1401) 2022-01-13 17:35:06 +01:00
Luca Deri d59fefd06e Improved MicrosoftAzure detection 2022-01-12 21:49:31 +01:00
Luca Deri 1e1cfb89d2 Added EthernetIP dissector 2022-01-12 21:48:39 +01:00
Ivan Nardi d4da3b6516 Run fuzzing tests also on push event (like the CI tests) (#1400) 2022-01-12 21:04:01 +01:00
Ivan Nardi f30fbc3212 Fix DGA false positive (#1399)
Close #1397
2022-01-12 20:51:29 +01:00
Ivan Nardi b080a1c136 Fix two use-of-uninitialized-value errors (#1398)
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=40269
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=41432

Fix fuzz compilation (follow-up of f5545a80)
2022-01-12 20:24:57 +01:00
Luca Deri 552d199d2e Removed outdated comment 2022-01-11 21:46:30 +01:00
Luca Deri f5545a80f9 Removed legacy code 2022-01-11 21:45:27 +01:00
Ivan Nardi 06f3b07909 QUIC: fix an integer overflow (#1396)
Reported by oss-fuzz:
```
==685288==ERROR: AddressSanitizer: SEGV on unknown address 0x61a100000687 (pc 0x0000005aba64 bp 0x7ffe3f29f510 sp 0x7ffe3f29f400 T0)
==685288==The signal is caused by a READ memory access.
SCARINESS: 20 (wild-addr-read)
 #0 0x5aba64 in quic_len ndpi/src/lib/protocols/quic.c:203:12
 #1 0x5aba64 in decrypt_initial_packet ndpi/src/lib/protocols/quic.c:993:16
 #2 0x5aba64 in get_clear_payload ndpi/src/lib/protocols/quic.c:1302:21
 #3 0x5aba64 in ndpi_search_quic ndpi/src/lib/protocols/quic.c:1658:19
 #4 0x579f00 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4683:6
 #5 0x57abe6 in ndpi_check_flow_func ndpi/src/lib/ndpi_main.c:0
 #6 0x583b2c in ndpi_detection_process_packet ndpi/src/lib/ndpi_main.c:5545:15
 #7 0x55e75e in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_process_packet.c:30:3
[...]
```
2022-01-11 17:57:34 +01:00
Ivan Nardi 3a087e951d Add a "confidence" field about the reliability of the classification. (#1395)
As a general rule, the higher the confidence value, the higher the
"reliability/precision" of the classification.

In other words, this new field provides an hint about "how" the flow
classification has been obtained.
For example, the application may want to ignore classification "by-port"
(they are not real DPI classifications, after all) or give a second
glance at flows classified via LRU caches (because of false positives).

Setting only one value for the confidence field is a bit tricky: more
work is probably needed in the next future to tweak/fix/improve the logic.
2022-01-11 15:23:39 +01:00
Luca Deri a2916d2e4c Readme for using fuzzer with nDPI 2022-01-11 15:12:45 +01:00
Luca Deri c02574807e Updated result 2022-01-09 18:52:30 +01:00
Luca Deri 708d4ea33a Improved user agent analysis 2022-01-09 18:47:47 +01:00
Ivan Nardi bf69321a29 GTP: fix some false positives (#1394) 2022-01-08 20:40:24 +01:00
Ivan Nardi 4d27001d1d Remove some unused fields (#1393) 2022-01-08 20:40:10 +01:00
Luca Deri eac3ae9ded Improved bittorrent detection 2022-01-06 19:50:31 +01:00
Luca Deri f24f68f59d Invalid check 2022-01-06 07:15:45 +01:00
Luca Deri 8f805a8163 Added missing dependency 2022-01-05 11:11:58 +01:00
Alfredo Cardigliano 948c0367f9 Update issue tempalte 2022-01-03 17:26:23 +01:00
Alfredo Cardigliano 23a4761276 Update copyright 2022-01-03 11:00:45 +01:00
Luca Deri d72a40a0fc Improved BitTorrent guess 2021-12-26 19:41:35 +01:00
Luca Deri c4ac53a03f Added support for Log4J/Log4Shell detection in nDPI via a new flow risk named NDPI_POSSIBLE_EXPLOIT 2021-12-23 21:30:16 +01:00
Ivan Nardi fdb6481cd6 Add support for ICloud Private Relay (#1390)
See:
https://www.apple.com/privacy/docs/iCloud_Private_Relay_Overview_Dec2021.PDF

TODO: an up-to-date list of egress IP ranges is publicly available. Can
we use it somehow?
2021-12-22 19:55:57 +01:00
Ivan Nardi 91bb77a880 A final(?) effort to reduce memory usage per flow (#1389)
Remove some unused fields and re-organize other ones.
In particular:
* Update the parameters of `ndpi_ssl_version2str()` function
* Zattoo, Thunder: these timestamps aren't really used.
* Ftp/mail: these protocols are dissected only over TCP.
* Attention must be paid to TLS.Bittorrent flows to avoid invalid
read/write to `flow->protos.bittorrent.hash` field.

This is the last(?) commit of a long series (see 22241a1d, 227e586e,
730c2360, a8ffcd8b) aiming to reduce library memory consumption.

Before, at nDPI 4.0 (more precisly, at a6b10cf7, because memory stats
were wrong until that commit):
```
nDPI Memory statistics:
	nDPI Memory (once):      221.15 KB
	Flow Memory (per flow):  2.94 KB
```
Now:
```
nDPI Memory statistics:
	nDPI Memory (once):      231.71 KB
	Flow Memory (per flow):  1008 B       <---------
```
i.e. memory usage per flow has been reduced by 66%, dropping below the
psychological threshold of 1 KB.

To further reduce this value, we probably need to look into #1279:
let's fight this battle another day.
2021-12-22 19:54:06 +01:00
Alfredo Cardigliano 02da143e45 Add links to other user guides 2021-12-22 10:23:14 +01:00
Luca Deri 4c02c81bf4 Improved bittorrent heuristic 2021-12-21 23:04:27 +01:00
Ivan Nardi a16b4aa564 Fix a global-buffer-overflow when loading Azure list (#1388) 2021-12-19 22:29:25 +01:00
Luca Deri 7aad97fffb Added Microsoft Azure support 2021-12-19 18:59:45 +01:00
Ivan Nardi 7153b8933c Improve/add several protocols (#1383)
Improve Microsoft, GMail, Likee, Whatsapp, DisneyPlus and Tiktok
detection.
Add Vimeo, Fuze, Alibaba and Firebase Crashlytics detection.

Try to differentiate between Messenger/Signal standard flows (i.e chat)
and their VOIP (video)calls (like we already do for Whatsapp and
Snapchat).

Add a partial list of some ADS/Tracking stuff.

Fix Cassandra, Radius and GTP false positives.
Fix DNS, Syslog and SIP false negatives.

Improve GTP (sub)classification: differentiate among GTP-U, GTP_C and
GTP_PRIME.

Fix 3 LGTM warnings.
2021-12-18 13:24:51 +01:00
Ivan Nardi 58b33dcb2d TLS: add support for IPV6 in Subject Alt Names field (#1385) 2021-12-18 12:03:46 +01:00
Ivan Nardi ec6eea0c23 TLS: fix usage of certificate cache (#1384) 2021-12-18 12:03:32 +01:00
Luca Deri 74519f6e10 Improved bittorrent detection 2021-12-17 21:17:18 +01:00
Ivan Nardi f11b215cfb QUIC: fix old versions of GQUIC on big-endian machines (#1387)
Credits to @viniciussn (see #1312)
2021-12-17 12:10:34 +01:00
Luca Deri 8e512ff503 Improved BitTorrent classification 2021-12-07 21:49:00 +01:00
Luca Deri 519d70fa04 Optimized ndpi_serialize_string_int32 2021-12-07 15:54:23 +01:00
Luca Deri 85b396ca43 Fixed issue that prevented alt certificate names to be fully detected when ipAddress and rfc822Name were specified in certificates 2021-12-07 14:45:42 +01:00
Luca Deri cefcc25b13 Added missing install target 2021-12-07 09:00:58 +01:00
Ivan Nardi 947896ad7d Fix configure script (after fb85dac9) (#1381)
Fix/disable some LGTM warnings
2021-12-04 20:22:05 +01:00
Ivan Nardi 7cba34a1e3 Improve IPv6 support, enabling IPv6 traffic on (almost) all dissectors. (#1380)
There are no valid reasons for a (generic) protocol to ignore IPv6
traffic.
Note that:
* I have not found the specifications of "CheckPoint High Availability
Protocol", so I don't know how/if it supports IPv6
* all LRU caches are still IPv4 only

Even if src_id/dst_id stuff is probably useless (see #1279), the right
way to update the protocol classification is via `ndpi_set_detected_protocol()`
2021-12-04 13:30:10 +01:00
Ivan Nardi 6e86e6d924 QUIC: add support for QUICv2 (draft 00) (#1379)
It is already time to start looking at the new QUIC version.
See: https://datatracker.ietf.org/doc/html/draft-ietf-quic-v2-00
2021-12-04 13:29:30 +01:00
Toni 4d60c70c8b Make serialize risk and proto not dependant on any flow. (#1377)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-12-04 13:28:55 +01:00
Luca Deri fb85dac905 Configure improvements
Fixed invalid anomaly detection in rrd_anomaly
2021-12-04 13:21:57 +01:00
Luca Deri fe2822c6a8 Added example for finding similarities in RRDs using nDPI statistical APIs 2021-12-04 10:09:01 +01:00
Luca Deri 6ab1367846 Added quick mode (-q) 2021-12-03 21:18:03 +01:00
Luca Deri 8c4ae7527b Updated rrdtool makefile 2021-12-03 21:10:43 +01:00
Luca Deri 1b077ba65f Demo tool for finding anomalies in RRD files 2021-12-03 17:36:49 +01:00
Alfredo Cardigliano 3dd9b5176e Remove printf when not required 2021-12-01 10:24:12 +01:00
Alfredo Cardigliano 1d3428d538 Improve JSON serialization 2021-12-01 10:04:17 +01:00
Luca Deri 33557c5fbc Improved CSV serialization 2021-11-30 22:15:26 +01:00
Luca Deri 5e0befd56c Fixed seriealizer unit 2021-11-30 17:24:42 +01:00
Luca Deri 4f81cdc103 Removed spaces from some protocol names as they create problems with some applications 2021-11-30 10:12:24 +01:00
Ivan Nardi b1e9245d94 ndpiReader: slight simplificaton of the output (#1378) 2021-11-27 17:32:23 +01:00
Luca Deri 320bcfafa2 Added ndpi_serializer_skip_header() serialization API 2021-11-26 22:18:59 +01:00
Luca Deri a7b5e09195 Added Salesforce detection 2021-11-26 19:07:45 +01:00
Luca Deri 9d59b8b1a9 HTTP proxy improvement 2021-11-25 23:17:27 +01:00
Luca Deri ea435c46f5 Reworked HTTP protocol dissection including HTTP proxy and HTTP connect 2021-11-25 22:53:46 +01:00
Luca Deri 772957349b Aded default port to HTTP_CONNECT 2021-11-25 21:46:00 +01:00
Luca Deri 40ff2c54c1 Added googletagmanager.com to the host_match list 2021-11-25 21:45:38 +01:00
Ivan Nardi a8ffcd8bb0 Rework how hostname/SNI info is saved (#1330)
Looking at `struct ndpi_flow_struct` the two bigger fields are
`host_server_name[240]` (mainly for HTTP hostnames and DNS domains) and
`protos.tls_quic.client_requested_server_name[256]`
(for TLS/QUIC SNIs).

This commit aims to reduce `struct ndpi_flow_struct` size, according to
two simple observations:
 1) maximum one of these two fields is used for each flow. So it seems safe
to merge them;
 2) even if hostnames/SNIs might be very long, in practice they are rarely
longer than a fews tens of bytes. So, using a (single) large buffer is a
waste of memory for all kinds of flows. If we need to truncate the name,
we keep the *last* characters, easing domain matching.

Analyzing some real traffic, it seems safe to assume that the vast
majority of hostnames/SNIs is shorter than 80 bytes.

Hostnames/SNIs are always converted to lowercase.

Attention was given so as to be sure that unit-tests outputs are not
affected by this change.

Because of a bug, TLS/QUIC SNI were always truncated to 64 bytes (the
*first* 64 ones): as a consequence, there were some "Suspicious DGA
domain name" and "TLS Certificate Mismatch" false positives.
2021-11-24 10:46:48 +01:00
Ivan Nardi fd02e1b304 QUIC: fix extra dissection (#1376)
When we have fully reassembled the Client Hello, we need to stop extra
dissection.
2021-11-23 10:57:13 +01:00
Luca Deri 4c39ed293e BitTorrent dissector code cleanup and merge of https://github.com/ntop/nDPI/pull/1374 2021-11-17 23:18:22 +01:00
Luca Deri 2e2d4c911a BitTorrent detection improvements 2021-11-17 22:41:35 +01:00
Luca Deri 2a6e5c6341 Improved BT hashing to minimize false positives 2021-11-17 10:47:21 +01:00
Luca Deri a6545ea095 bittorrent old code cleanup. Enlarged BT cache 2021-11-16 10:05:18 +01:00
Luca Deri f2a279e441 Improved BitTorrent detection 2021-11-15 17:25:13 +01:00
Ivan Nardi afc2b641eb Fix writes to flow->protos union fields (#1354)
We can write to `flow->protos` only after a proper classification.

This issue has been found in Kerberos, DHCP, HTTP, STUN, IMO, FTP,
SMTP, IMAP and POP code.
There are two kinds of fixes:
 * write to `flow->protos` only if a final protocol has been detected
 * move protocol state out of `flow->protos`
The hard part is to find, for each protocol, the right tradeoff between
memory usage and code complexity.

Handle Kerberos like DNS: if we find a request, we set the protocol
and an extra callback to further parsing the reply.

For all the other protocols, move the state out of `flow->protos`. This
is an issue only for the FTP/MAIL stuff.

Add DHCP Class Identification value to the output of ndpiReader and to
the Jason serialization.

Extend code coverage of fuzz tests.

Close #1343
Close #1342
2021-11-15 16:20:57 +01:00
ragostino da47357762 Update ndpi_main.c for HTTP line parser optimization and cleaning (#1372)
* Update ndpi_main.c

HTTP line parser: optimization and cleaning

* Update ndpi_main.c
2021-11-11 12:38:31 +01:00
Ivan Nardi acb1de69aa Reduce memory used by ndpiReader (#1371)
`ndpiReader` is only an example, aiming to show nDPI capabilities
and integration, without any claim about performances.

Nonetheless its memory usage per flow is *huge*, limiting the kinds
of traces that we can test on a "normal" hardware (example: scan
attacks).

The key reason of that behaviour is that we preallocate all the memory
needed for *all* the available features.

Try to reduce memory usage simply allocating some structures only
when they are really needed. Most significant example: JOY algorithms.

This way we should use a lot less memory in the two most common
user-cases:
 * `ndpiReader` invoked without any particular flag (i.e `ndpiReader -i
$FILENAME_OR_IFACE`)
 * internal unit tests

Before (on x86_64):
```
struct ndpi_flow_info {
[...]
	/* size: 7320, cachelines: 115, members: 72 */
```
After:
```
struct ndpi_flow_info {
[...]
	/* size: 2128, cachelines: 34, members: 75 */
```
2021-11-11 12:37:25 +01:00
Ivan Nardi 3e5491fa10 Add detection of OCSP (#1370)
This protocol is detected via HTTP Content-Type header.

Until 89d548f9, nDPI had a dedicated automa (`content_automa`) to
classify a HTTP flow according to this header. Since then, this automa has
been useless because it is always empty.
Re-enable it to match only a string seems overkilling.

Remove all `content_automa` leftovers.
2021-11-11 12:36:55 +01:00
Ivan Nardi 0f168d9150 IMAP, POP3, SMTP: improve dissection (#1368)
Avoid NATS false positives
2021-11-11 11:55:56 +01:00
Luca Deri 9abf528c8e BitTorrent cache debug 2021-11-08 12:42:15 +01:00
Ivan Nardi 5464bad6db Differentiate between standard Amazon stuff (i.e market) and AWS (#1369) 2021-11-04 00:20:45 +01:00
Luca Deri 4173775b60 Improved BitTorrent detection 2021-11-04 00:20:26 +01:00
Ivan Nardi a8a3c6f7c2 Fix use-of-uninitialized-value (#1366)
Close #1346
2021-11-03 15:40:07 +01:00
Ivan Nardi 55880e4ae4 TLS: fix two warnings (#1365)
Disable unit tests on CI for big-endian target. We know we have multiple
issues on big-endian architectures (see #1312) and so the unit tests
always fail there. Ignore this error for the time being and let the CI
pass if we don't have other issues.

Remove an unused automa definition
2021-11-02 22:08:15 +01:00
Luca Deri 1f6c94e663 Updated test results 2021-11-02 12:08:20 +01:00
Luca Deri b11065b2e3 Fixed certificate mismatch for TLS flows with no client hello observed 2021-11-02 12:06:18 +01:00
Luca Deri 9f8c1e8630 BitTorrent detection improvements 2021-10-31 18:45:13 +01:00
Luca Deri a9aa46287d Little change in TLS debug traces 2021-10-28 10:16:33 +02:00
Luca Deri a25d9d871e Updated results 2021-10-27 23:04:29 +02:00
Luca Deri 5283755afa Fixed TLS certificate dissection of subjectAltName 2021-10-27 22:51:59 +02:00
Luca Deri aaa474a37d Minor code cleanup 2021-10-27 09:32:11 +02:00
Ivan Nardi 6edb7bedd7 Avoid overwriting valid protocol in ndpi_detection_giveup (#1360)
We should avoid updating any valid protocol in `ndpi_detection_giveup`; we
should try to find a proper classification only if the flow is still
completely unclassified.

For example in the attached pcap there is a valid TLS session, recognized
as such by TLS dissector. However, the `ndpi_detection_giveup`function
updates it to "HTTP/TLS" (!?) simply because the server port is 80.

Note that the real issue is not the wrong classification, but the
wrong access to `flow->protos` union. If we already set some fields of
`flow->protos` and we change the protocol in `ndpi_detection_giveup`, we
might end up freeing some invalid pointers in `ndpi_free_flow_data`
(no wonder this issue has been found while fuzzing #1354)

Fix GIT and TLS dissectors (issues found by CI fuzzer)
2021-10-27 09:23:07 +02:00
Toni c4eebaaf5e Serialize additional information stored in the flow struct. (#1362)
* Changed function signature of ndpi_flow2json (removed unused vlan_id; API break)
 * Serialize NTP information.
 * Improved QUIC serialization.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-27 09:22:19 +02:00
Luca Deri f41d8ac106 Improved STUN and RTP detection 2021-10-27 00:06:05 +02:00
Toni 41765efcf8 Detect invalid characters in text and set a risk. Fixes #1347. (#1363)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-26 21:34:01 +02:00
Luca Deri 5ccc61d1cb Fixed Git protocol dissection (#1355) 2021-10-25 18:52:27 +02:00
Ivan Nardi b6d9536533 Fixed cleartext protocol assignment (#1357) 2021-10-25 15:04:04 +02:00
Ivan Nardi 333a6d60e8 TLS: fix a heap-buffer-overflow (#1356)
Revert of c3d1c697
Error reproducible with the attached pcap and valgrind
2021-10-22 14:57:49 +02:00
Luca Deri c3d1c69722 Fixed heap-buffer-overflow in TLS dissector 2021-10-20 22:07:47 +02:00
Ivan Nardi 8db8c8ae34 Fix QUIC log and remove SoulSeek leftovers after b97dc6ba (#1351)
Update .gitignore file
Fix a function prototype
Close #1349
2021-10-19 21:42:55 +02:00
Ivan Nardi fea3991ad1 Fix some invalid memory reads (#1350)
`ndpi_detection_giveup()` (and any functions called by it) can't access
`ndpi_detection_module_struct->packet` anymore since 730c236.

Sync unit tests results

Close #1348
2021-10-19 12:07:47 +02:00
Luca Deri 06f13fd2de Added clertext alert with telnet 2021-10-19 08:44:29 +02:00
Toni ed51987e3a Fix broken fuzz_process_packet fuzzer by adding a call to ndpi_finalize_initialization(). (#1334)
* fixed several memory errors (heap-overflow, unitialized memory, etc)
 * ability to build fuzz_process_packet with a main()
   allowing to replay crash data generated with fuzz_process_packet
   by LLVMs libfuzzer
 * temporarily disable fuzzing if `tests/do.sh`
   executed with env FUZZY_TESTING_ENABLED=1

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-18 23:16:32 +02:00
Luca Deri 7d3c3b23f8 Implemented RDP over UDP dissection 2021-10-18 22:12:28 +02:00
Luca Deri 9e97d20c25 Refreshed results list 2021-10-16 12:03:16 +02:00
Luca Deri b08b6a6a28 Fixed cleartext protocol assignment 2021-10-16 12:02:04 +02:00
Luca Deri c2e7fbce94 Updated test results after latest commit 2021-10-16 11:57:32 +02:00
Luca Deri 1b3c41a752 Reworked flow risks asignment
Added esceptions for windows update and binary application transfer risk
2021-10-16 11:55:08 +02:00
Luca Deri b97dc6baa4 Removed outdated (and broken) soulseek dissector 2021-10-15 17:27:27 +02:00
Luca Deri e696f31487 Updated test results 2021-10-14 22:18:34 +02:00
Luca Deri b1a43cbcff Added missing call to ndpi_reconcile_protocols() with protocol guess 2021-10-14 22:08:20 +02:00
Luca Deri 6e649e0cb6 Fixed issue on DGA numeric IP detection 2021-10-13 19:21:27 +02:00
Luca Deri 09668496c3 Indemt 2021-10-13 19:18:05 +02:00
Zied Aouini a2e9730b23 Add issue templates. (#1340)
* Add issue template.
* Add config log instruction.
2021-10-13 18:39:10 +02:00
Luca Deri 937357e4bc Implemented ndpi_ses_fitting() and ndpi_des_fitting()
for comuting the best alpha/beta values for exponential smoothing
2021-10-12 13:08:58 +02:00
Alfredo Cardigliano d947437233 Return 0 on ndpi_init_serializer_ll success for consistency 2021-10-12 10:47:44 +00:00
Luca Deri d13bc0e01a Fixed DES initialisation 2021-10-12 12:38:40 +02:00
Ivan Nardi 550e6fe6fc QUIC: fix an integer overflow (#1337)
Long standing bug: credits to @lnslbrty for digging into it and to
@aouinizied for the CI improvements
2021-10-11 23:08:10 +02:00
Toni a8f938f7ac Removed README.protocols because: (#1333)
* Tor via TLS should be detectable via DGA as a risk
 * protocol limitations should be part of the official documentation in `doc/`

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-11 23:07:36 +02:00
Luca Deri 960e8ab22a Win fix 2021-10-11 20:59:28 +02:00
Zied Aouini ca827ae548 Fix FuzzCI. (#1338) 2021-10-11 20:17:12 +02:00
Zied Aouini 7e0724d576 Enable sanitizers on CIFuzz. (#1336) 2021-10-11 19:01:50 +02:00
Luca Deri 246d115d7b Cleaned up code moving specific includes in files their are using it. Thi prevents ndpi_config.h to be included everywhere in apps using nDPI that might leade to #define redefinitions after the latest changes 2021-10-11 11:45:45 +02:00
Toni 60eca79fdf Additional fix related to cf931fda6b. (#1332)
* configure.seed references removed

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-10 14:14:02 +02:00
Toni 549134ef10 Fix make dist and add it to the CI along with make -C doc text. Fixes #1324 (#1327)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-10 13:15:38 +02:00
Toni cf931fda6b Get rid of configure.seed as it comes with some disadvantages. (#1328)
* using Autotools best-practices to achieve (hopefully) the same result

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-10 13:13:52 +02:00
Toni 6ff67b5043 Add more WindowsUpdate URLs as requested in #698. (#1329)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-08 18:29:49 +02:00
Luca Deri 3ffb65a81c Added extra check 2021-10-07 08:21:40 +02:00
Ivan Nardi 2dfc478ad4 Fix compilation with clang-13 or if some debug macros are enabled (#1326) 2021-10-06 18:32:15 +02:00
Luca Deri 944e8ed105 Added check 2021-10-06 18:28:09 +02:00
Luca Deri 2b4821a97b Compilation fix for old Linux distributions 2021-10-06 09:28:04 +02:00
Luca Deri d9c72999ee Test update 2021-10-06 00:06:34 +02:00
Luca Deri 181a03c5ad Compilation fix 2021-10-05 17:17:12 +02:00
Luca Deri 8957c52a48 Numeric IPs are not considered for DGA checks 2021-10-05 17:14:31 +02:00
Luca Deri 408d78e628 Improved DGA detection for skipping potential DGAs of known/popular domain names 2021-10-05 16:51:24 +02:00
Luca Deri 10576eab0a TLS obsolete protocol is set when TLS < 1.2 (used to be 1.1) 2021-10-05 16:44:56 +02:00
Ivan Nardi 730c2360bd Remove struct ndpi_packet_struct from struct ndpi_flow_struct (#1319)
There are no real reasons to embed `struct ndpi_packet_struct` (i.e. "packet")
in `struct ndpi_flow_struct` (i.e. "flow"). In other words, we can avoid
saving dissection information of "current packet" into the "flow" state,
i.e. in the flow management table.

The nDPI detection module processes only one packet at the time, so it is
safe to save packet dissection information in `struct ndpi_detection_module_struct`,
reusing always the same "packet" instance and saving a huge amount of memory.
Bottom line: we need only one copy of "packet" (for detection module),
not one for each "flow".

It is not clear how/why "packet" ended up in "flow" in the first place.
It has been there since the beginning of the GIT history, but in the original
OpenDPI code `struct ipoque_packet_struct` was embedded in
`struct ipoque_detection_module_struct`, i.e. there was the same exact
situation this commit wants to achieve.

Most of the changes in this PR are some boilerplate to update something
like "flow->packet" into something like "module->packet" throughout the code.
Some attention has been paid to update `ndpi_init_packet()` since we need
to reset some "packet" fields before starting to process another packet.

There has been one important change, though, in ndpi_detection_giveup().
Nothing changed for the applications/users, but this function can't access
"packet" anymore.
The reason is that this function can be called "asynchronously" with respect
to the data processing, i.e in context where there is no valid notion of
"current packet"; for example ndpiReader calls it after having processed all
the traffic, iterating the entire session table.

Mining LRU stuff seems a bit odd (even before this patch): probably we need
to rethink it, as a follow-up.
2021-10-05 15:49:52 +02:00
Ivan Nardi f3fcf1e7c0 Update unit tests results after da8eed5a (#1323) 2021-10-05 15:06:34 +02:00
Luca Deri da8eed5a32 Updated description 2021-10-05 11:08:16 +02:00
Ivan Nardi c1e794366f WHOIS: enhance detection, avoiding false positives (#1320)
We are interested only in the domain name required, not in the long reply.
2021-10-05 09:35:04 +02:00
Luca Deri bb7aff6526 Added -a <num> to ndpiReader for generating OPNsense configuration
See https://github.com/ntop/opnsense
2021-10-04 22:34:49 +02:00
Luca Deri fd0e65cb57 Removed trace 2021-10-03 23:41:31 +02:00
Ivan Nardi bc5be1e76c Fix how some protocols handle tcp retransmissions (#1321)
Most (all?) protocols don't care about (tcp) retransmissions.

If a protocol registers itself with a
NDPI_SELECTION_BITMASK_PROTOCOL_*_WITHOUT_RETRANSMISSION value, its
callback is never triggered with a retransmitted packet.
2021-10-03 21:38:49 +02:00
Toni e84da386c7 Added nDPI pkg-config file to Debian / Ubuntu ndpi-dev packaging. (#1318)
* Added nDPI pkg-config file to Debian / Ubuntu ndpi-dev packaging.

 * fixed missing gcrypt library dependency in libndpi.pc

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-03 21:29:56 +02:00
Simone Mainardi 06f763e6c3 Adds sections labels with risk id to the docs 2021-10-01 15:02:14 +02:00
Luca Deri a9d4f5d8d0 Warning fix 2021-10-01 11:36:35 +02:00
Luca Deri 8e6de00ce3 Initial attempt to write nDPI documentation. Starting with flow risks. Please contribute 2021-10-01 11:32:27 +02:00
Ivan Nardi 5cf3fef8f7 Remove detected_protocol_stack field from ndpi_packet_struct (#1317)
This field is an exact copy of `ndpi_flow_struct->detected_protocol_stack[2]`:
* at the very beginning of packet dissection, the value saved in
`flow->detected_protocol_stack` is copied in `packet->detected_protocol_stack`
(via `ndpi_detection_process_packet()` -> `ndpi_init_packet_header()`)

* every time we update `flow->detected_protocol_stack` we update
`packet->detected_protocol_stack` too (via `ndpi_int_change_protocol()`
-> `ndpi_int_change_packet_protocol()`)

These two fields are always in sync: keeping the same value in two
different places is useless.
2021-09-29 13:11:32 +02:00
Alfredo Cardigliano 721031210b Fix warning 2021-09-28 09:44:31 +02:00
Luca Deri af35d6b963 Added unit test for bitmap iteration 2021-09-27 12:47:05 +02:00
Luca Deri b9ccff0761 Compilation fixed on CentOS 7
Bitmap APi changes
2021-09-27 12:41:40 +02:00
Luca Deri 241ee32cbc Bitmap code 2021-09-27 12:06:46 +02:00
Luca Deri 519c69c5cc Reworked bitmap code 2021-09-27 10:29:03 +02:00
Luca Deri 72df138a7d Warnign fix 2021-09-27 09:08:04 +02:00
eac7024489 Fix armhf (#1315)
* Fix unaligned memory accesses with get_u_int64_t at armhf

see: https://bugs.debian.org/993627

* Use get_u_int64_t to avoid unaligned memory access at armhf

see: https://bugs.debian.org/993627

* Update src/include/ndpi_define.h.in

Drop const type from get_u_int64_t, from lnslbrty

Co-authored-by: Bernhard Übelacker <bernhardu@mailbox.org>
Co-authored-by: Toni <matzeton@googlemail.com>
2021-09-26 23:53:00 +02:00
Luca Deri 5ed1930a93 Added include for defining bools (not present on all platforms) 2021-09-26 23:00:13 +02:00
Luca Deri 1efabef4cf Added API for handling compressed bitmaps
ndpi_bitmap* ndpi_bitmap_alloc();
void ndpi_bitmap_free(ndpi_bitmap* b);
u_int64_t ndpi_bitmap_cardinality(ndpi_bitmap* b);
void ndpi_bitmap_set(ndpi_bitmap* b, u_int32_t value);
void ndpi_bitmap_unset(ndpi_bitmap* b, u_int32_t value);
bool ndpi_bitmap_isset(ndpi_bitmap* b, u_int32_t value);
void ndpi_bitmap_clear(ndpi_bitmap* b);
size_t ndpi_bitmap_serialize(ndpi_bitmap* b, char **buf);
ndpi_bitmap* ndpi_bitmap_deserialize(char *buf);

based on https://github.com/RoaringBitmap/CRoaring
2021-09-26 22:55:15 +02:00
Zied Aouini c40cf4e0ec Add arm64, armhf and s390x as part of CI. (#1314)
* Add arm64, armhf and s390x as part on CI.

* Minor fix.

* Fix code inspector complaints.
2021-09-23 14:41:56 +02:00
Ivan Nardi 7eb73bc92a QUIC: fix old GQUIC versions on big-endian machines (#1313) 2021-09-23 13:24:49 +02:00
Zied Aouini 0994771974 Improve CI (#1303)
* Improve CI pipeline

* Fix branch name.

* Fix branch name.

* Fix libgcrypt configuration.

* Update build.yml

* Move to Github Actions instead of Travis CI.

* Fix mingw on ubuntu bionic.

* Reactivate cross compile on Ubuntu Bionic.

* Switch to single line steps.

* Add several compilers versions

* Minor fix.

* Fix build all and delete cxx

* Fix RCE detection.

* Fix PCRE configuration.

* Add condition on PCRE test pcap.

* Update WebattackRCE.pcap.out

* Add missing SUBST.

* Delete WebattackRCE.pcap.out

* Update WebAttackRCE result.

* Fix typo.

* Extend jobs with pcre+msan+maxminddb.

* Fix code inpector warnings.

* Delete .appveyor.yml
2021-09-22 12:10:52 +02:00
Toni 9717afb2da Merge pull request #1311 from IvanNardi/ftp-start-tls
FTP: fix support for START-TLS sessions
2021-09-22 12:08:04 +02:00
Nardi Ivan 017c1a4239 FTP: fix support for START-TLS sessions
When TLS-over-FTP is used, the credentials are encrypted. So we must not
wait for the username and the password commands, otherwise we elaborate a
lot of packets for nothing.
2021-09-21 20:53:04 +02:00
Toni bb52f1362c Merge pull request #1309 from IvanNardi/ipv6-headers
Fix parsing of ipv6 packets with extension headers
2021-09-20 12:54:01 +02:00
Toni d74c2739d5 Merge pull request #1310 from IvanNardi/stun-realm
STUN: fix extraction of Realm attribute
2021-09-20 11:53:40 +02:00
Nardi Ivan 95ac8fd41a STUN: fix extraction of Realm attribute
While at it, improve detection of Facebook Messenger
2021-09-20 11:04:54 +02:00
Nardi Ivan 03d3e1bafc Fix parsing of ipv6 packets with extension headers
Decoding of ipv6 traffic with extension headers was completely broken,
since the beginning of the L4 header was always set to a wrong value.

Handle the ipv6 fragments in the same way as the ipv4 ones: keep the first
one and drop the others.
2021-09-19 17:29:22 +02:00
Ivan Nardi 994bd0696b Small fixes after latest commits (#1308) 2021-09-18 09:31:00 +02:00
Ivan Nardi 227e586e63 A (tiny) effort to reduce memory usage per flow (#1304)
Follow-up of 22241a1d
Only trivial changes:
* remove completely unused fields
* remove fields only written (but never read)
* CSGO protocol only handles UDP traffic
2021-09-18 09:29:45 +02:00
Luca Deri 4539e96ce2 Added DNS fragmented test pcap 2021-09-17 15:56:18 +02:00
Luca Deri 3303dac7cf Improved fragmented DNS detection 2021-09-17 15:17:39 +02:00
Luca Deri 119a14c6e3 DNS dissection fixes 2021-09-17 12:20:19 +02:00
deboracerretiniandLuca Deri 8f113c1d37 Progetto esame Gestione di Reti - Debora Cerretini (#1290)
* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2021-09-17 11:01:50 +02:00
Ivan Nardi 6325aebda6 TLS: avoid zeroing large structures (#1300)
Zeroing large structures (i.e. size > KB) is quite costly (from a CPU point
of view): we can safely avoid doing that for a couple of big structures.

Standard and Valgrind tests have been diverging quite a lot: it is time
to re-sync them. Use the same script and enable Valgrind via an
enviroment variable:

  NDPI_TESTS_VALGRIND=1 ./tests/do.sh
2021-09-16 14:34:59 +02:00
Simone Mainardi 978c9cfda3 Adds instructions to update flow risks in ntopng 2021-09-15 17:49:47 +02:00
Ivan Nardi 0eb7a0388c Hangout: fix usage of STUN function (#1299) 2021-09-13 18:25:43 +02:00
Ivan Nardi 3e06e6f5b8 Fix some ndpi_malloc/ndpi_free pairs (#1298) 2021-09-13 18:25:22 +02:00
Ivan Nardi 31404f4865 Update the list of default ports for QUIC protocol (#1297)
There are no reasons to register UDP/80 as a default port for QUIC
2021-09-13 18:25:04 +02:00
Ivan Nardi 17774bf09c Bittorrent: fix a potential use-of-uninitialized-value error (#1296)
Not sure if this is the right fix (from a logical point-of-view): this
code hasn't changed since OpenDPI era (!) and I have't found a trace
triggering this code path.
Anyway, the use-of-uninitialized-value error itself should be fixed.
2021-09-13 18:24:42 +02:00
Vitaly Lavrov 22241a1d36 Removing unused fields from the id_struct structure. (#1293)
Reduced by 40 bytes.
2021-09-13 18:19:09 +02:00
Ivan Nardi 951bcb0c39 Cassandra: fix compilation when "--enable-debug-messages" option is used (#1294)
Let's try adding a dedicated compilation in GitHub Actions to easily
detect this kind of errors in the future
2021-09-11 11:10:46 +02:00
Ivan Nardi 083e464929 Fix unit test results after 00857abf (#1295) 2021-09-11 11:09:55 +02:00
Luca Deri 00857abf2c Added new risk for clear text credentials 2021-09-10 22:00:04 +02:00
lucasbaileandLucas Santos 1fadf4754a Add Cassandra protocol dissector (#1285)
Co-authored-by: Lucas Santos <lucas.santos@zerum.com>
2021-09-09 22:47:58 +02:00
Luca Deri 36f08b8d2e Win fix 2021-09-07 22:37:22 +02:00
Alfredo Cardigliano 57fce4e350 Fix unused var 2021-09-03 17:46:04 +02:00
Alfredo Cardigliano 89af9b7e3c Print API version 2021-09-01 09:48:37 +02:00
Luca Deri a6b10cf73f Fixed memory stats 2021-08-26 12:12:22 +02:00
Luca Deri 90c6fd99a7 ARM fixes 2021-08-25 09:46:29 +02:00
Luca Deri 88040f0449 Compilation fix 2021-08-20 22:47:46 +02:00
Ivan Nardi 9e0f0ce3df Fix access to some TLS fields in flow structure (#1277)
Fields 'tls.hello_processed` and `tls.subprotocol_detected` are used by
QUIC (i.e UDP...), too.
2021-08-20 18:11:37 +02:00
Ivan Nardi 8fdffbf3a1 Compile everything with "-W -Wall -Wno-unused-parameter" flags (#1276)
Fix all the warnings.

Getting rid of "-Wno-unused-parameter" is quite complex because some
parameters usage depends on compilation variable (i.e.
`--enable-debug-messages`).

The "-Werror" flag has been added only in Travis builds to avoid
breaking the builds to users using uncommon/untested
OS/compiler/enviroment.

Tested on:
* x86_64; Ubuntu 20.04; gcc 7,8,9,10,11; clang 7,8,9,10,11,12
* x86_64; CentOS 7.7; gcc 4.8.5 (with "--disable-gcrypt" flag)
* Raspberry 4; Debian 10.10; gcc 8.3.0
2021-08-20 18:11:13 +02:00
Ivan Nardi 55eec29c08 wireshark/lua: restore full flow risks dissection (#1275)
It was partially disabled in 3eba8cc5.
Wireshark and Wireshark-Lua bindings don't handle 64 bit integer very well.
(see https://www.wireshark.org/docs/wsdg_html_chunked/lua_module_Int64.html).

As workaround, only for visualization purpose, split the (64 bit) risk mask
into two 32 bit integer values.
2021-08-18 11:34:46 +02:00
Toni 8d0c7b1fae Fixed Mingw64 build, SonerCloud-CI and more. (#1273)
* Added ARM build and unit test run for SonarCloud-CI.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Fixed Mingw64 build.

 * adapted to SonarCloud-CI workflow
 * removed broken and incomplete Windows example (tested on VS2017/VS2019)
 * removed unnecessary include (e.g. pthread.h for the library which does not make use of it)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-08-18 11:34:16 +02:00
Luca Deri 677b513f70 Fixes a crash on ARM (Raspberry Pi 4 Model B Rev 1.1) 2021-08-18 11:12:06 +02:00
Luca Deri 821f4c9249 Fixed some invalid TLS guesses 2021-08-17 19:04:07 +02:00
Luca Deri a2ea67152c Added check to avoid clases on similar protocols (FTP adn SMTP) on setting hostname 2021-08-11 12:11:58 +02:00
Luca Deri 5c33fbf19b Added extraction of hostname in SMTP
Fixed mail incalid subprotocol calculation
2021-08-11 11:52:24 +02:00
Alfredo Cardigliano 846b546dbc Fix typo 2021-08-10 11:43:32 +02:00
Luca Deri c1acb6d801 Typo 2021-08-09 18:36:36 +02:00
Luca Deri 0fbe895103 Addef ETH_ARP definition 2021-08-09 18:33:33 +02:00
Luca Deri 05492ec8e8 Defined more standard ethernet protocol types 2021-08-09 17:01:10 +02:00
Luca Deri 58ca7b5a7f Added protocol defines 2021-08-09 16:45:03 +02:00
Luca Deri 0df8aa19ec Clode cleanup (after last merge) 2021-08-08 17:10:08 +02:00
ToniandLuca Deri 259b60f574 Added entropy calculation to check for suspicious (encrypted) payload. (#1270)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2021-08-08 17:03:23 +02:00
Toni 4e856a41d6 Skip whitespaces between HTTP method and URL. (#1271)
* be less case-restrictive, RFC2616 wants it that way

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-08-08 17:00:10 +02:00
Toni 8e996f7f7c Fixed broken SonarCloud-CI. (#1272)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-08-08 16:59:36 +02:00
Luca Deri 3706267d68 Updated test output 2021-08-07 19:43:55 +02:00
Luca Deri 0522e562e8 Added testing pcap for TLS fatal alert 2021-08-07 19:42:53 +02:00
Luca Deri 4183718952 Added TLS fatal alert flow risk 2021-08-07 19:40:44 +02:00
Luca Deri e8455236bd Updated output 2021-08-07 17:38:33 +02:00
Luca Deri a13f1fe52f Report whether a protocol is encrypted 2021-08-07 17:35:34 +02:00
Luca Deri 46ebd7128f Added ability to report whether a protocol is encrypted
Added new API calls
- u_int8_t ndpi_is_valid_protoId(u_int16_t protoId);
Cleaned up code and organized in a function.

- u_int8_t ndpi_is_encrypted_proto(struct ndpi_detection_module_struct *ndpi_str, ndpi_protocol proto);
Check whether a protocol is encrypted or not based on master and app protocols
2021-08-07 17:10:18 +02:00
Luca Deri dc841574af Win fix 2021-08-03 09:44:50 +02:00
Luca Deri 6b7e5fa8d2 Added legacy whatsapp rules used to match old pcaps that need to be removed in the future 2021-07-31 23:49:48 +02:00
Toni ad57af9f79 Improved RTSP detection and fixed HTTP false-positive. Fixes #1229. (#1266)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-31 23:31:49 +02:00
Fábio Depin ce597b4806 Updated WhatsAPP and Instagram addresses (#1264)
* Updated WhatsAPP addresses

* Added Instagram addresses
2021-07-31 23:30:34 +02:00
Toni 13c5d6801e Improved libm check via Autoconf. (#1263)
* libm will now be part of @ADDITIONAL_LIBS@ (if required)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-28 22:24:03 +02:00
Luca Deri cb26a8c82d Moved to 4.1.0 2021-07-26 09:40:50 +02:00
Luca Deri 4cafa7cb1e Improved risk detection mask algorithm 2021-07-26 09:38:53 +02:00
Toni 29ec34f66d Improved TFTP detection. Fixes #1242, #1256 (#1262)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-25 21:54:19 +02:00
Luca Deri a482e1c0be Typo (Thanks Ivan) 2021-07-24 18:12:09 +02:00
Luca Deri 1d9a830b2f Update changelog 2021-07-24 17:54:32 +02:00
Luca Deri 51e4888442 Implemented ALPN automa for checking uncommon ALPNs 2021-07-24 17:50:32 +02:00
Luca Deri 526568fcd5 Risk check improvement 2021-07-24 16:18:18 +02:00
Alfredo Cardigliano c73501f859 Typo 2021-07-23 18:14:11 +02:00
Alfredo Cardigliano d5a60880d0 Fix prototype 2021-07-23 18:12:54 +02:00
Luca Deri 61fc5be202 Reworked flow risk implementation 2021-07-23 17:27:15 +02:00
Ivan Nardi 4ffe1eb3c0 Fix unit tests when "--disable-gcrypt" flag is used (#1255) 2021-07-23 10:37:36 +02:00
Toni 6ad0d6666c Implemented function to retrieve flow information. #1253 (#1254)
* fixed [h]euristic typo

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-23 10:37:20 +02:00
Luca Deri 8ea8ba8e9b Fixed risk mask implementation 2021-07-23 08:29:36 +02:00
Luca Deri b01b60a2b5 Implementation of flow risk eception (work in progress) 2021-07-22 01:35:57 +02:00
Alfredo Cardigliano fc9901292d Update changelog 2021-07-20 17:04:35 +00:00
Toni 32275543c4 Added risk: TLS_EXTENSION_SUSPICIOUS (#1252)
* validates client/server hello TLS extensions
 * inspects content for some extensions

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-19 16:23:24 +02:00
Ivan Nardi 57b8969a3d Fix setting of flow risks on 32 bit machines (#1251)
Since 19a29e1e (NDPI_TLS_CERT_VALIDITY_TOO_LONG is 32), unit tests are
failing on 32 bit machines (i.e Raspberry 4)
2021-07-19 16:22:39 +02:00
Luca Deri 0adcd28450 Updated download category name 2021-07-16 14:04:47 +02:00
Vitaly Lavrov 295cb3ee6a Bug fix. (#1250)
Restore part of the fields flow->packet after cleaning flow structure.
2021-07-16 13:53:36 +02:00
pacantandpacant 488166833e Fixed TLS certificate threshold (#1248)
Co-authored-by: pacant <a.pace97@outlook.com>
2021-07-16 13:52:55 +02:00
Vitaly Lavrov daf1fbfc7b Code cleanup. (#1246)
ndpi_utils.c: use ndpi_malloc,ndpi_calloc,ndpi_free
genshin_impact.c, git.c, hpvirtgrp.c, http.c, z3950.c: removed "#include stdlib.h"
2021-07-16 13:51:54 +02:00
Luca Deri a6de94a930 hulu and netflix dissection improvements 2021-07-14 12:53:03 +02:00
Luca Deri 3eba8cc5b5 Temporary fix for avoiding wireshark errors 2021-07-14 12:51:42 +02:00
Luca Deri b95bd0358f Code cleanup 2021-07-14 11:19:24 +02:00
pacantandpacant 19a29e1e22 TLS Risks - Certificate Validity Too Long (#1239)
* Added flow risk: TLS certificate too long

* Added flow risk: TLS certificate too long

* Date for TLS limit added

* TLS certificate check fixed

Co-authored-by: pacant <a.pace97@outlook.com>
2021-07-14 11:13:22 +02:00
Toni c411df523e Added AVAST SecureDNS protocol. (#1244)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-14 11:11:59 +02:00
Toni 62bae30a4a Improved Steam detection (Steam Datagram Relay - SDR). (#1243)
* improved DNSCrypt midstream detection again (sufficient for all tested use-cases)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-14 11:09:58 +02:00
Toni e4453938d5 Improved dnscrypt midstream detection. (#1241)
* fixed skype false-positive detection of dnscrypt traffic

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-13 15:10:18 +02:00
Ivan Nardi cccf794265 ndpiReader: add statistics about nDPI performance (#1240)
The goal is to have a (roughly) idea about how many packets nDPI needs
to properly classify a flow.

Log this information (and guessed flows number too) during unit tests,
to keep track of improvements/regressions across commits.
2021-07-13 12:28:39 +02:00
Luca Deri 96b71def49 Minor change 2021-07-12 17:51:25 +02:00
Vitaly Lavrov c418b7110b ahoсorasick. Code review. Part 2. (#1236)
Simplified the process of adding lines to AC_AUTOMATA_t.
Use the ndpi_string_to_automa() function to add patterns with domain names.
For other cases can use ndpi_add_string_value_to_automa().

ac_automata_feature(ac_automa, AC_FEATURE_LC) allows adding
and compare data in a case insensitive manner. For mandatory pattern comparison
from the end of the line, the "ac_pattern.rep.at_end=1" flag is used.
This eliminated unnecessary conversions to lowercase and adding "$" for
end-of-line matching in domain name patterns.

ac_match_handler() has been renamed ac_domain_match_handler() and has been greatly simplified.
ac_domain_match_handler() looks for the template with the highest domain level.
For special cases it is possible to manually specify the domain level.
Added test for checking ambiguous domain names like:
 - short.weixin.qq.com is QQ, not Wechat
 - instagram.faae1-1.fna.fbcdn.net is Instagram, not Facebook

If you specify a NULL handler when creating the AC_AUTOMATA_t structure,
then a pattern with the maximum length that satisfies the search conditions will be found
(exact match, from the beginning of the string, from the end of the string, or a substring).

Added debugging for ac_automata_search.
To do this, you need to enable debugging globally using ac_automata_enable_debug(1) and
enable debugging in the AC_AUTOMATA_t structure using ac_automata_name("name", AC_FEATURE_DEBUG).
The search will display "name" and a list of matching patterns.
Running "AHO_DEBUG=1 ndpiReader ..." will show the lines that were searched for templates
and which templates were found.

The ac_automata_dump() prototype has been changed. Now it outputs data to a file.
If it is specified as NULL, then the output will be directed to stdout.
If you need to get data as a string, then use open_memstream().

Added the ability to run individual tests via the do.sh script
2021-07-12 17:39:43 +02:00
Luca Deri 78b1295dc1 Fixed false positives in Z39.50 2021-07-09 00:16:48 +02:00
Luca Deri bec8361638 Updated category labels 2021-07-08 20:05:50 +02:00
Vitaly Lavrov 398ad8e5ff Fix for #1230 (#1235)
* Revert "Fix return value of ndpi_match_string_subprotocol() (#1230)"

This reverts commit 58665e93a9.

* Checking the return code after calling ndpi_match_string_subprotocol()

ndpi_api.h: Description of the returned error codes for
the ndpi_match_string_subprotocol() function.

If the ndpi_match_string_subprotocol() function returned an error,
then return NDPI_PROTOCOL_UNKNOWN.

http: The "Content-type" header is only checked if it is not empty.
2021-07-07 14:32:57 +02:00
Toni 34d0718bca Improved RTSP detection the second. (#1232) (#1233)
* RTSP is no subprotocol of HTTP (most of the time)
 * detection patterns should stay in rtsp.c
 * set detected HTTP protocol only if at least a valid HTTP method detected

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-07 09:55:36 +02:00
Luca Deri c408710c52 Added test pcap 2021-07-06 18:57:52 +02:00
Toni 6f29733262 Improved RTSP via HTTP detection. (#1232)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-06 18:57:04 +02:00
Ivan Nardi 58665e93a9 Fix return value of ndpi_match_string_subprotocol() (#1230) 2021-07-06 18:55:14 +02:00
Luca Deri 1ee7355b91 Updated prototype 2021-07-05 22:35:11 +02:00
Luca Deri e5dacc28e7 Minor fix 2021-07-05 22:02:46 +02:00
Luca Deri 48db44a567 Code cleanup 2021-07-05 21:45:30 +02:00
Alfredo Cardigliano 8e7b063804 Fix possible segfault/leak 2021-07-05 15:09:48 +02:00
Toni 6f7acfcab5 Added protocols to HTTP as subprotocols. (#1228)
* if HTTP was detected,
   all of those defined subprotocol detection routines should called

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-05 09:43:15 +02:00
Ivan Nardi 46c32d2433 TLS: fix a memory error in JA3 code (#1227)
protocols/tls.c:1856:5: runtime error: index 256 out of bounds for type 'char [256]'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:1856:5
2021-07-05 09:42:52 +02:00
Ivan Nardi 1116d0e4b1 Improve content match lists (#1226)
Add support for Likee app (https://likee.video/) and GitLab
Improve detection of Google domains ("gtv1.com" added in 6dd42d19f was
likely a typo)
Improve Snapchat and Tiktok detection
2021-07-05 09:42:11 +02:00
Toni 21b2cce715 Improved Z39.50 detection. (#1225)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-05 09:41:35 +02:00
Toni ced6fca184 Fixed off-by-one memory error for TLS-JA3. (#1222)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-06-29 15:33:12 +02:00
Toni 8222ac5bdc Removed ht_hash as it is not used anymore. (#1220)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-06-29 15:32:48 +02:00
Toni 1c2a0c36f1 Added Z39.50 protocol. (#1219)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-06-29 15:32:16 +02:00
vpiserchiaandvpiserchia e5fd86393e reconcile PPSTREAM protocol category with content_match declarations (#1218)
* reconcile PPSTREAM protocol category with content_match declarations

* remove duplicate declaration for PPSTREAM protocol

Co-authored-by: vpiserchia <vito.piserchia@dreamlab.net>
2021-06-28 14:58:42 +02:00
Ivan Nardi f8fe3ee520 QUIC: add basic support for fragmented Client Hello (#1216)
Only in-order and non overlapping fragments are handled
See #1195
2021-06-24 18:30:34 +02:00
Luca Deri 6a1fd9ad97 Added missing check to prevent crashes 2021-06-23 12:17:21 +02:00
Vitaly Lavrov 2234b97149 ndpiReader: memory leak (#1215)
Non-critical bugs.
If a file list is used, then all files except the last are not closed.
Opening the next file loses the memory allocated via pcap_open_offline() for the previous file.
If a bpf filter is used, then no memory is freed after pcap_compile.
2021-06-23 12:04:03 +02:00
Vitaly Lavrov 35fc6a6de5 Speed and memory size optimisation (#1214)
Removed bigram_automata, impossible_bigram_automata, trigram_automata.
The ahocorasick structure is replaced with a bitmap.
The bitmap size for ndpi_en_bigram is 176 bytes.
The bitmap size for ndpi_en_trigram is 2201 bytes.
On the test machine, the test execution time was reduced from 27.3 seconds to 24.7 (9%).
2021-06-23 12:03:52 +02:00
Luca Deri f19937c8c9 Addesses partial application matching (#1207) 2021-06-21 21:59:15 +02:00
Luca Deri 0cbb4b2ffa Restored 32 bit value in ndpi_match_string_value() 2021-06-21 17:06:22 +02:00
Luca Deri 613e21002a Upgraded exponential smoothing to 64 bit values 2021-06-18 22:56:56 +02:00
Luca Deri 0afc8ace3d Added nDPI Score report 2021-06-17 23:52:01 +02:00
martinscheuandtinu 0b9b6a683d fixed lua errors in non-iec104 packets (#1209)
* Update iec.lua

fixed lua errors in non iec 104 packets

* Update iec.lua

Co-authored-by: tinu <martin.scheu@switch.ch>
2021-06-17 15:35:31 +02:00
Luca Deri b0b3e1bc6c Added bitmap boundary checks 2021-06-17 10:49:11 +02:00
Luca Deri e89c06256e Extende nDPI API with ndpi_netbios_name_interpret() 2021-06-17 09:35:19 +02:00
Luca Deri 2eba77e3b0 NetBIOS decoding changes 2021-06-16 19:57:20 +02:00
Vitaly Lavrov be808c30f3 Code review. (#1205)
The common actions required to call the ac_automata_search() function
have been moved to the ndpi_match_string_common function. This made it
possible to simplify the ndpi_match_string, ndpi_match_string_protocol_id,
ndpi_match_string_value, ndpi_match_custom_category, ndpi_match_string_subprotocol,
ndpi_match_bigram, ndpi_match_trigram functions.

Using u_int16_t type for protocol identifiers when working with the
ahocorasick library (changes src/include/ndpi_api.h.in and src/include/ndpi_typedefs.h).

Reworked "finalization" of all AC_AUTOMATA_t structures.

Changing the order of fields in the ndpi_call_function_struct structure
reduces the size of the ndpi_detection_module_struct structure by 10 kB (for x86_64).
2021-06-15 11:28:09 +02:00
Luca Deri 09293fabd9 Win fix 2021-06-15 11:26:36 +02:00
Luca Deri 43ccaef2f5 IEC analysis wireshark plugin 2021-06-14 21:18:37 +02:00
Alfredo Cardigliano 4aefbe0c7a Call ac_automata_release with free_pattern = 1 (malloc'ed patterns expected in ndpi_add_string_to_automa) 2021-06-14 14:41:14 +02:00
Luca Deri 380286c069 Fixes https://github.com/ntop/ntopng/issues/5482 2021-06-11 22:21:03 +02:00
Luca Deri a79b8ee834 Reimplememnted SNMP dissector 2021-06-11 21:44:51 +02:00
Luca Deri dee6f90d6c Syntax error fixes 2021-06-11 21:01:27 +02:00
Luca Deri c9f582805b Fixed flow score label 2021-06-11 19:53:07 +02:00
Matteo Biscosi a01a2b7443 Changed output for the sharkfest lua scripts 2021-06-11 11:00:06 +02:00
Vitaly Lavrov 6704649a75 Bug in ndpi_lru_find_cache (#1204)
Key value check is missing.
2021-06-10 11:06:19 +02:00
Luca 1a8ae710cf Minor code cleanup 2021-06-09 08:58:18 +02:00
Toni 65e67a2f88 autoconf: Do not check for json-c if --with-only-libndpi is set. (#1202)
* json-c is used by a unit test
 * required to fix some libnDPI cross compilation issues

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-06-09 08:01:10 +02:00
Luca Deri 77710ed8b4 OpenWRT document fix 2021-06-08 21:39:28 +00:00
Luca Deri 177f9456bd Win changes 2021-06-08 18:15:46 +02:00
Luca Deri d3aa3df975 Updated scripts 2021-06-08 17:52:03 +02:00
Luca 41ec807d79 New testing pcap with syn scan attack 2021-06-08 10:39:41 +02:00
Luca 2f8d3ac0b0 Companion scripts written for the Sharkfest conference 2021-06-08 10:39:19 +02:00
Luca Deri 2af7b33de0 Added report on score cli/srv breakdown 2021-06-07 14:59:18 +02:00
Vitaly Lavrov 65678dbeea New version of the ahocorasick library (#1200)
The new version is about 25% faster with -O2 and 45% faster with -O3.
No recursion is used (smaller stack size required).
Uses less memory (by valgrind info)
 bigram:
  - original 1796 allocs,   247864 bytes allocated
  - new      1232 allocs,   158880 bytes allocated
 host_match:
  - original 18038 allocs, 3004576 bytes allocated
  - new       6861 allocs,  396624 bytes allocated

The function ac_automata_search() is thread safe.
Optional case-insensitive comparison.
Matching at the beginning and at the end of the string is supported.
One code file and one header file.
2021-06-07 14:19:40 +02:00
Vitaly Lavrov 2fcf641e87 Fixed uninitialized memory use. (#1198)
The pointer "header" must be initialized before first call pcap_next_ex().
2021-06-07 10:08:39 +02:00
Luca Deri b241dd5e6b Fixed expected output 2021-06-03 09:06:01 +02:00
Luca Deri 03ae090d39 Warnign fix 2021-06-03 09:05:53 +02:00
Luca Deri 8761ac1887 Renamed Skyp in Skype_Teams as the protocol is now shared across these apps 2021-06-02 21:47:01 +02:00
Ivan Nardi 654e84ff02 wireshark/lua: improve flow risk visualization (#1194)
Create a separated proto field entry for each possible flow risk.
This way, filtering will be more natural: you can use something like
"ndpi.flow_risk.desktop_file_sharing_session"
2021-06-02 21:28:50 +02:00
Luca Deri 06c6941f03 Removed xbox and playstation invalid port guesses 2021-06-02 00:24:41 +02:00
Ivan Nardi 9d427faafe ndpiReader: fix collecting of risks statistics (#1192) 2021-06-01 16:50:46 +02:00
Ivan Nardi f4dbeecba9 TLS: improve check for common ALPNs (#1191)
Facebook is still using its own ALPN for HTTP2 as well
2021-06-01 16:50:28 +02:00
Luca c620858671 Reworked ndpi flow risk score adding client and server score 2021-06-01 09:17:26 +02:00
Alfredo Cardigliano c4084ca3c7 Fix warnings 2021-05-25 15:17:48 +02:00
Luca eb0115d4e0 Added TLS check to avoid crash with UDP-based traffic 2021-05-25 09:03:24 +02:00
Luca Deri d485c1cecb Improved TLS browser detection heuristics 2021-05-19 21:28:33 +02:00
Ivan Nardi 71b81d3799 wireshark/lua: fix offsets (#1187) 2021-05-18 21:36:46 +02:00
Luca Deri 732bcecd17 Added flow risk score 2021-05-18 21:05:47 +02:00
Luca Deri 86f3c29d03 Typo 2021-05-18 19:52:33 +02:00
Luca Deri ca15e3295e Added risk/score dump (ndpiReader -h)
Added ndpi_dump_risks_score() API score
2021-05-18 19:34:17 +02:00
Luca Deri 0b3d627185 Updated API for ndpi_risk2score()
Added ndpi_get_upper_proto() API call
2021-05-17 22:39:07 +02:00
Luca Deri 43a8576efb Reworked human readeable string search in flows
Removed fragment manager code
2021-05-17 20:55:06 +02:00
Luca Deri 1ec621c85b Added further checks 2021-05-15 19:53:46 +02:00
Luca Deri 9c5ba9c060 Updated protocol category 2021-05-15 11:20:15 +02:00
Luca Deri 5646ffa725 Added ndpi_check_subprotocol_risk() API call definition 2021-05-15 11:16:27 +02:00
Luca Deri abd6bce6f9 Added TLS certifiacate caching
Added Fortigate protocol
2021-05-15 10:52:16 +02:00
Luca Deri ca92660d98 Converted some test .pcapng files to pcap format 2021-05-13 20:51:11 +02:00
Luca Deri ac1eaca8a6 Added browser TLS heuristic 2021-05-13 20:00:27 +02:00
Luca Deri a62be9b8ec Implemented heuristic to detect Safari and Firefox TLS browsing 2021-05-13 12:37:07 +02:00
Toni 87076dcd5b Fixed obsolete error printing if CTRL-C is pressed. #1165 (#1184)
* This fix was proposed by @robertsong2019

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-11 21:38:56 +02:00
Toni 5918a6542d Improved SSL certificate name wildcard handling and risk. #1182 (#1183)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-11 21:38:26 +02:00
Luca Deri a5ecdf9df8 Changes for sonarcloud integration 2021-05-11 21:36:25 +02:00
Luca Deri 5079bb1106 Fixed typo 2021-05-11 21:33:48 +02:00
Luca Deri 4297a65ce8 Implemented flow score in Wireshark integration 2021-05-10 22:43:05 +02:00
Luca Deri fff60ec317 Added check to reduce MongoDB false positive detection 2021-05-10 21:14:55 +02:00
Luca Deri 66ebe444ae Added (partial) Activision protocol support (based on tencent cloud) 2021-05-10 15:41:52 +02:00
Ivan Nardi 1deae1132b Fix some warnings (#1181)
```
In file included from protocols/fasttrack.c:29:
../include/ndpi_api.h:1504:3: warning: type qualifiers ignored on function return type [-Wignored-qualifiers]
 1504 |   const ndpi_risk_severity ndpi_risk2severity(ndpi_risk_enum risk);
      |   ^~~~~
In file included from protocols/amazon_video.c:28:
../include/ndpi_api.h:1504:3: warning: type qualifiers ignored on function return type [-Wignored-qualifiers]
 1504 |   const ndpi_risk_severity ndpi_risk2severity(ndpi_risk_enum risk);
      |   ^~~~~

...

ndpi_utils.c: In function ‘ndpi_risk2severity’:
ndpi_utils.c:1834:1: warning: control reaches end of non-void function [-Wreturn-type]
 1834 | }
      | ^
```
2021-05-09 15:11:05 +02:00
Ivan Nardi 2ca37ff8be TLS: fix extraction for TLS signature algorithms (#1180)
```
==69562==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6100009000fb at pc 0x7f41882003a7 bp 0x7f4183cfbfc0 sp 0x7f4183cfb768
READ of size 32 at 0x6100009000fb thread T1
    #0 0x7f41882003a6 in __interceptor_memcpy ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827
    #1 0x560b2d7462a1 in processClientServerHello protocols/tls.c:1647
    #2 0x560b2d73be6a in processTLSBlock protocols/tls.c:712
    #3 0x560b2d73e61f in ndpi_search_tls_udp protocols/tls.c:968
```
2021-05-09 15:10:41 +02:00
Ivan Nardi 12c6ab768d TLS: fix another use-of-uninitialized-value error in ClientHello parsing (#1179)
Error detected with valgrind.

==13127== Conditional jump or move depends on uninitialised value(s)
==13127==    at 0x483EF58: strlen (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==13127==    by 0x1A93B6: ndpi_strdup (ndpi_main.c:159)
==13127==    by 0x1C07CC: processClientServerHello (tls.c:1678)
==13127==    by 0x1C0C4C: processTLSBlock (tls.c:712)
==13127==    by 0x1C0C4C: ndpi_search_tls_tcp.part.0 (tls.c:849)

See also 8c3674e9
2021-05-09 15:10:14 +02:00
Toni 8c28613eb2 Check datalink during fuzzing to prevent console / logfile spam. See #1175 for more information. (#1177)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-05-09 15:09:43 +02:00
Bernhard M. Wiedemann 0663544be8 Allow to override build date with SOURCE_DATE_EPOCH (#1176)
in order to make builds reproducible.
See https://reproducible-builds.org/ for why this is good
and https://reproducible-builds.org/specs/source-date-epoch/
for the definition of this variable.
2021-05-09 15:09:15 +02:00
Luca ae2470fad4 Initial work towards detection via TLS of browser types 2021-05-06 21:42:06 +02:00
Luca 808050e635 Add extraction for TLS signature algorithms 2021-05-06 08:56:01 +02:00
Luca ce54a6ab57 Added ndpi_risk2severity() API call 2021-05-02 19:35:01 +02:00
Luca Deri 153d11c920 Enhanced netbios decoding check 2021-04-28 22:58:22 +02:00
Luca Deri dd65142020 Compilation fix 2021-04-27 08:26:08 +02:00
Luca Deri 70686249c9 Updated code due to https://github.com/ntop/nDPI/pull/1175 2021-04-27 08:12:14 +02:00
Luca Deri 7b62db81c3 Removed DGA check for ipv6 reverse IPs 2021-04-27 07:23:11 +02:00
Toni da3e6bd61b Check for common ALPNs and set a flow risk if not known. (#1175)
* Increased risk bitmask to 64bit (instead of 32bit).
 * Removed annoying "Unknown datalink" error message for fuzzers.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-27 07:22:04 +02:00
Luca Deri d5b395f35a Fixed invalid DNS dissection 2021-04-26 23:08:00 +02:00
Luca Deri 87ad2b58dc Compilation fix 2021-04-26 10:50:19 +02:00
Luca Deri 4a09707e48 Added flow risk to wireshark dissection 2021-04-26 10:17:29 +02:00
Luca Deri a04efb7ce7 Added tshark description 2021-04-25 11:47:19 +02:00
Luca Deri fad246289f README for the tshark class 2021-04-25 11:45:35 +02:00
Luca Deri b586c223dd Lua tshark class and examples 2021-04-25 11:40:53 +02:00
Luca Deri 0ef7998348 Removed protocol space in Genshin Impact 2021-04-25 10:06:22 +02:00
Toni b4a8c8c63f Add Genshin Impact protocol. (#1173)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-25 10:02:07 +02:00
Alfredo Cardigliano d95cd9912c Add vxlan enum to ndpi_packet_tunnel 2021-04-21 12:43:12 +02:00
Toni 9377991263 Add HP Virtual Machine Group Management (hpvirtgrp) protocol. (#1170)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-04-20 14:12:16 +02:00
Luca Deri c6a835746e Reworked GeoIP API
int ndpi_get_geoip_asn(struct ndpi_detection_module_struct *ndpi_str,
                        char *ip, u_int32_t *asn);
int ndpi_get_geoip_country_continent(struct ndpi_detection_module_struct *ndpi_str, char *ip,
                        char *country_code, u_int8_t country_code_len,
                        char *continent, u_int8_t continent_len);
2021-04-19 21:39:55 +02:00
Ivan Nardi 8c3674e9a3 TLS: fix some use-of-uninitialized-value errors in ClientHello parsing (#1169)
Error detected with valgrind.

==125883== Conditional jump or move depends on uninitialised value(s)
==125883==    at 0x438F57: processClientServerHello (tls.c:1421)
==125883==    by 0x43B35A: processTLSBlock (tls.c:712)
==125883==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125883==    by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)
==125883==    by 0x42E920: ndpi_detection_process_packet (ndpi_main.c:5301)

==125916== Conditional jump or move depends on uninitialised value(s)
==125916==    at 0x438D7D: processClientServerHello (tls.c:1379)
==125916==    by 0x43B35A: processTLSBlock (tls.c:712)
==125916==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125916==    by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)

==125932== Conditional jump or move depends on uninitialised value(s)
==125932==    at 0x438C1D: processClientServerHello (tls.c:1298)
==125932==    by 0x43B35A: processTLSBlock (tls.c:712)
==125932==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125932==    by 0x42C60B: check_ndpi_detection_func (ndpi_main.c:4426)

==125950== Conditional jump or move depends on uninitialised value(s)
==125950==    at 0x438D4F: processClientServerHello (tls.c:1371)
==125950==    by 0x43B35A: processTLSBlock (tls.c:712)
==125950==    by 0x43B1C4: ndpi_search_tls_tcp (tls.c:849)
==125950==    by 0x42C079: check_ndpi_detection_func (ndpi_main.c:4443)
2021-04-18 21:38:01 +02:00
Ivan Nardi 9ca62ed7ac Fix detunneling of GTP-U traffic (#1168)
Fuzzing #1161 exposed some (completely unrelated) issues on GTP-U
detunneling code.
(see https://github.com/ntop/nDPI/actions/runs/719882047)
2021-04-18 21:37:51 +02:00
Ivan Nardi d9278d55c3 Improve classification of Outlook/MicrosoftMail traffic (#1167)
See #1148
2021-04-18 21:37:26 +02:00
Luca Deri 9fcf98c067 GeoIP handlign fixes 2021-04-18 21:36:03 +02:00
Luca Deri 18c6c1c2d6 Added NDPI_DESKTOP_OR_FILE_SHARING_SESSION risk to remote protocols for remote assistance sessions 2021-04-12 18:11:14 +02:00
Luca Deri bf318e0b86 Added NDPI_DESKTOP_OR_FILE_SHARING_SESSION flow risk 2021-04-11 14:42:27 +02:00
Ivan Nardi ceb98a3f9c wireshark/lua: fix handling of VLAN traffic (#1162) 2021-04-05 19:23:56 +02:00
Ivan Nardi 0bfe444362 GTP: fix parsing of GTP headers (#1161)
Message length checks and basic headers are not uniform across GTP-U,
GTP-C and GTP-PRIME.

Note that, even if the length checks were wrong, the GTP sessions were almost
always correctly classified because of the "guessing" algorithm.

This patch has been tested with GTP-U, GTP-C-V1, GTP-C-V2 and GPT-PRIME-V2
traffic using ndpiReader with "-d" flag (to avoid "guessing" algorithm) and
without "-t" flag (to avoid GTP-U de-tunneling).

See #1148
2021-04-05 19:23:03 +02:00
Ivan Nardi fb74785282 Fix some warnings about unused variables/functions (#1160) 2021-04-05 19:21:30 +02:00
Luca Deri a1dba74346 Trace fix 2021-04-02 12:55:15 +02:00
Luca Deri 4f8ca9485a Fixed incapoatibilities with the latest extcap/wireshark 2021-04-01 23:53:53 +02:00
Luca Deri db5aeb0cb1 Readme update 2021-04-01 10:24:44 +02:00
Luca Deri 28879c570e Reworked ndpi patricia includes to avoid compilation issues on some platforms 2021-03-31 19:43:55 +02:00
Ivan Nardi 1290c40968 Mining: lru cache is ipv4 only (for the time being) (#1159)
Fix memory error with ipv6 traffic
2021-03-31 14:13:07 +02:00
Luca Deri 48726301e1 Fixed mispelled word 2021-03-31 12:52:54 +02:00
Luca Deri 2b84f28df5 Improved mining results 2021-03-30 17:54:44 +02:00
Luca Deri c1d6e3f145 Improved mining detection support 2021-03-30 17:50:19 +02:00
Luca Deri 637b2063ed Added missing tracker/Ads breed 2021-03-30 17:50:03 +02:00
Luca Deri fcbc16da00 Fixed invalid guess stats 2021-03-30 17:49:48 +02:00
Luca Deri ffbce931b9 Ignore TLD .local .lan and .home in DGA domain check 2021-03-26 15:53:04 +01:00
Ivan Nardi a6029d250d ndpiReader: print an error msg if we found an unsupported datalink type (#1157) 2021-03-23 11:47:29 +01:00
Toni b040407683 Refactored nDPI subprotocol handling and aimini protocol detection. (#1156)
* Refactored and merged callback buffer routines for non-udp-tcp / udp / tcp / tcp-wo-payload.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Try to detect one subprotocol if a detected protocol can have one.

 * This adds a performance overhead due to much more protocol detection routine calls.
   See #1148 for more information.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactor subprotocol handling (1/2).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactor subprotocol handling (2/2).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Prevent some code duplication by using macros for ndpi_int_one_line_struct string comparision.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactored aimini HTTP detection parts (somehow related to #1148).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Added aimini client/server test pcap.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Removed master protocol as it was only used for STUN and via also removed API function ndpi_get_protocol_id_master_proto

 * Adjusted Python code to conform to the changes made during the refactoring process.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-03-23 11:46:12 +01:00
Luca Deri 0cae9bf4a4 Win compilation fix 2021-03-22 21:41:28 +01:00
Luca Deri 627299e4dd Better DGA detection (slightly decreased accuracy) 2021-03-20 17:56:24 +01:00
Luca Deri 6333bb1702 Removed duplicate extesions len 2021-03-19 22:35:08 +01:00
Luca Deri d96f4ca736 Added ALPN and elliptic curve in JA3S+ 2021-03-19 22:33:04 +01:00
Luca Deri fa238bbe42 Implemented JA3+ also for JA3S 2021-03-19 22:27:36 +01:00
Luca Deri e2c3445a20 Reworked JA3 2021-03-19 22:20:52 +01:00
Luca Deri 61f7ec1696 JA3 debug improvements 2021-03-19 19:10:31 +01:00
Luca Deri 9eed0a60d3 Fixed support for custom DGA detection libraryr 2021-03-18 11:32:56 +01:00
Luca Deri b22cb70a68 Added % of flows with risks 2021-03-14 21:59:33 +01:00
Luca Deri 1b6275005e Added in stats the number of flows with risks 2021-03-14 21:54:35 +01:00
Luca Deri 3b02cfab69 Added flows risks report 2021-03-14 21:46:23 +01:00
Luca Deri 565a7bfce3 Reworked extendal dependency across testing tools 2021-03-14 20:48:21 +01:00
Luca Deri e58527536d Help crash fix 2021-03-14 19:55:21 +01:00
Luca Deri 9419015711 Implemented square erro rollup to avoid overflow 2021-03-14 11:01:51 +01:00
Zied Aouini 7f9a18fd05 Make libmaxminddb build configurable. (#1154) 2021-03-14 10:47:28 +01:00
Alfredo Cardigliano efc5630378 Fix compilation warning 2021-03-12 09:25:00 +01:00
Luca 192fad4402 Added double exponential smoothing implementation 2021-03-11 09:39:52 +01:00
Luca Deri 6833ee2bbe Added single exponential smoothing API
int ndpi_ses_init(struct ndpi_ses_struct *ses, double alpha, float significance);
int ndpi_ses_add_value(struct ndpi_ses_struct *ses, const u_int32_t _value, double *forecast, double *confidence_band);
2021-03-11 00:04:33 +01:00
Luca Deri 5b7fe1360a Fixed JA3+ computation 2021-03-11 00:04:12 +01:00
Luca Deri 5c6f070097 Fixed nDPI prefs definition 2021-03-09 23:50:31 +01:00
Luca Deri f6ad16d8f8 Added experiemntal JA3+ implementation that can be used with -z i ndpiReader 2021-03-09 23:38:29 +01:00
Ivan Nardi 8074bc8201 HTTP: fix memory access in ndpi_http_parse_subprotocol() (#1151) 2021-03-09 19:46:32 +01:00
Luca Deri 574fc4f09e Ookla detection improvement 2021-03-09 17:43:12 +01:00
Luca Deri 3032864ec9 Added Ookla detection over IPv6 2021-03-09 12:55:14 +01:00
Luca Deri db716d0ab0 Ookla fixes 2021-03-09 11:42:23 +01:00
Luca Deri 477e4db650 Improved detection of Ookla speedtest and openspeedtest.com 2021-03-09 11:38:31 +01:00
Luca Deri bb6423a79f Added the ability to define a custom DGA detection function by overwriting
the value of the function pointer ndpi_dga_function curently set to NULL
(that means the nDPI internal DGA function will be used)
2021-03-08 22:57:30 +01:00
Ivan Nardi c3490e80a7 Fix some stack-use-after-return errors in automa code (#1150) 2021-03-08 21:14:01 +01:00
Ivan Nardi c50a8d4808 Add support for Snapchat voip calls (#1147)
* Add support for Snapchat voip calls

Snapchat multiplexes some of its audio/video real time traffic with QUIC
sessions. The peculiarity of these sessions is that they are Q046 and
don't have any SNI.

* Fix tests with libgcrypt disabled
2021-03-06 05:48:36 +01:00
Alfredo Cardigliano 73e4348570 Add ndpi_serialize_binary_boolean for consistency. Fix comments. 2021-03-04 10:31:03 +01:00
Luca Deri 0f8a994841 Improved DGA detection
Before
Accuracy 66%, Precision 86%, Recall 38%

After
Accuracy 71%, Precision 89%, Recall 49%
2021-03-03 19:30:01 +01:00
Luca 49843509e5 Added verbose option 2021-03-03 08:12:16 +01:00
Luca Deri 1a37595de0 Removed check for knowns protocols (major and app protocols) 2021-03-03 00:57:56 +01:00
Luca Deri 56bfb439f8 Improved DGA detection with trigrams. Disadvantage: slower startup time
Reworked Tor dissector embedded in TLS (fixes #1141)
Removed false positive on HTTP User-Agent
2021-03-03 00:41:07 +01:00
Ivan Nardi 4c00ff89df DTLS: improve support (#1146)
* DTLS: add some pcap tests

* DTLS: fix parsing of Client/Server Helllo message

* DTLS: add parsing of server certificates
2021-03-02 21:15:40 +01:00
Ivan Nardi 2b71a329c1 QUIC: fix mvfst-27 test (#1145)
Regardless of its name, quic-mvfst-27 trace doesn't contain mvfst-27
traffic
2021-03-02 21:15:02 +01:00
Ivan Nardi c12a697f2d TOR: update node list (#1144)
https://panwdbl.appspot.com/lists/ettor.txt list is no more available

Close #1141
2021-03-02 21:14:36 +01:00
Luca Deri 2d558afb54 Added HW checks 2021-03-01 19:59:14 +01:00
Luca Deri 2724a14bf7 Added further HW checks 2021-03-01 19:34:20 +01:00
Luca Deri 4bff595733 Holt-Winters calculation improvement 2021-02-27 11:32:51 +01:00
Toni 16890a6632 Added NDPI_MALICIOUS_SHA1 flow risk. (#1142)
* An external file which contains known malicious SSL certificate SHA-1 hashes
   can be loaded via ndpi_load_malicious_sha1_file(...)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-26 17:00:05 +01:00
Luca Deri fba61adf5e Improved DNS dissector 2021-02-26 00:42:15 +01:00
pengtian 28330edb7a [Fix] replace free to ndpi_free (#1140)
same as https://github.com/ntop/nDPI/issues/1096
2021-02-25 08:46:55 +01:00
Toni 70c35addcb Added protocol breed to JSON serializer. (#1137)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-25 08:46:38 +01:00
Alfredo Cardigliano d898fddea0 Fix ndpi_fill_prefix_v6 2021-02-24 14:43:12 +00:00
Luca Deri 545f270132 Windows code cleanup 2021-02-24 11:04:04 +01:00
Luca Deri 4bd175b07e Modified JA3 fingerprint message 2021-02-24 10:42:26 +01:00
Alfredo Cardigliano 566d5db402 Add more utility functions to work with patricia trees 2021-02-23 11:53:15 +01:00
Alfredo Cardigliano 888ddfcd17 Update ndpi_patricia_walk_inorder API 2021-02-23 10:23:52 +01:00
Alfredo Cardigliano f8e83f7e35 Add support for MAC to Patricia tree. Expose full API to applications. Add utility functions. 2021-02-23 10:01:56 +01:00
Luca Deri b757cf606b Fixed mispelled API call 2021-02-22 23:59:08 +01:00
Luca Deri f1b22b199f Added NDPI_MALICIOUS_JA3 flow risk
Added ndpi_load_malicious_ja3_file() API call
2021-02-22 23:19:23 +01:00
Luca Deri fc3db8f169 Implemented TLS Certificate Sibject matching
Improved AnyDesk detection
2021-02-22 22:37:33 +01:00
Luca Deri a21ad2fbe4 Added new data for risky domains test 2021-02-21 22:08:01 +01:00
Luca Deri 5ad056e99b Removed unused NDPI_RISKY_COUNTRY 2021-02-21 21:45:46 +01:00
Luca Deri fc16c9368e Added risky domain flow-risk support 2021-02-21 21:45:46 +01:00
Luca Deri 6db20b3ba9 Updated dependency 2021-02-19 17:59:40 +01:00
Luca Deri 09559611cc Fixes #1136 2021-02-19 15:15:02 +01:00
Luca Deri dee24f7acf Removed old unused code 2021-02-19 15:12:03 +01:00
Alfredo Cardigliano e9350d8fc7 Fix leak 2021-02-19 09:27:24 +01:00
Luca Deri 92e8d95f38 Added ndpi_get_geoip() APi call 2021-02-18 23:15:27 +01:00
Luca Deri bdf95a5c22 Package fix 2021-02-18 22:08:15 +00:00
Luca Deri 393a844c9a GeoIP support 2021-02-18 22:40:57 +01:00
Luca Deri bce54079d8 Initial geoip support 2021-02-18 22:35:25 +01:00
Luca Deri a2c5adc374 Improved nDPI string matching algorithm 2021-02-18 21:47:09 +01:00
Luca Deri fb4cdecb68 Fixes due to the fragment mananegr code 2021-02-18 13:31:02 +01:00
Luca Deri 413dc287b4 Added paper citation 2021-02-17 08:22:38 +01:00
Luca Deri 4609e2084b Added new risks (future use)
-  NDPI_RISKY_ASN
-  NDPI_RISKY_DOMAIN
-  NDPI_RISKY_COUNTRY
2021-02-16 18:23:19 +01:00
Alfredo Cardigliano 177f1785da Fix warning 2021-02-12 09:11:14 +01:00
Luca Deri 813df77d07 Performance optimization to avoid un-necessary calls and thus increase the overall performance 2021-02-11 17:54:55 +01:00
Luca Deri d99d636696 Some optimizations during flow guess 2021-02-10 19:51:26 +01:00
Luca Deri fffff3760b Code refactory 2021-02-10 19:28:35 +01:00
Luca Deri ea0309d21b Removed now obsolete NDPI_DETECTION_SUPPORT_IPV6: code is more readeable now 2021-02-10 19:09:11 +01:00
Luca Deri 76ac551ed3 Improved FTP_CONTROL detection 2021-02-10 17:55:07 +01:00
Luca Deri 79102ef4f2 Added check for avoiding long dissections 2021-02-10 17:36:09 +01:00
Ivan Nardi 421609475e Fix small memory leak (#1133)
Now function definition matches the prototype in ndpi_api.h.in
2021-02-10 15:24:34 +01:00
Toni 1e12c90c66 Fixed memory leaks caused by conditional free'ing for some TLS connec… (#1132)
* Fixed memory leaks caused by conditional free'ing for some TLS connections.

 * Members of tls_quic struct should also free'd if the detected master protocol is IMAPS / POPS / SMTPS / etc.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Prevent reader_util.c from exit()'ing if maximum flow count reached.
This confuses the fuzzer.

 * Improved fuzz/Makefile.am to use LDADD for ../example/libndpiReader.a instead of LDFLAGS.
   That way, fuzz_ndpi_reader re-links to ../example/libndpiReader.a if something changed there.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-02-10 15:24:11 +01:00
Luca Deri c408df1b0e STUN improvements 2021-02-10 15:22:20 +01:00
Luca Deri e2f6569adb Fixed CPHA missing protocol initialization
Improved IEC104 and IRC detection
2021-02-10 15:22:20 +01:00
Luca Deri 0de3d4c37a Dissection inprovements 2021-02-09 21:48:46 +01:00
Luca Deri 512f73f381 Added checks for giving up faster on IRC and SMTP 2021-02-09 21:27:09 +01:00
Luca Deri 1a557d0918 IRC test files 2021-02-09 21:25:48 +01:00
Luca Deri 1331e0aec9 Extended the API to calculate jitter
- ndpi_jitter_init()
- ndpi_jitter_free()
- ndpi_jitter_add_value()
2021-02-09 15:56:03 +01:00
Luca Deri b960809d3e Minor code improvements 2021-02-09 12:55:11 +01:00
Luca Deri 4abaf3e279 Removed debug statement 2021-02-09 10:05:29 +01:00
Luca Deri 732579b72b Added timeseries forecasting support implementing Holt-Winters with confidence interval
New API calls added
- ndpi_hw_init()
- ndpi_hw_add_value()
- ndpi_hw_free()
2021-02-08 19:10:25 +01:00
Luca Deri d9da752aa8 Updated skype addresses 2021-02-07 19:05:56 +01:00
Luca Deri 3ecda45bc4 IP address matching update 2021-02-07 17:42:31 +01:00
Luca Deri 8297dacfe5 Added Makefile for building the development version of libndpi for OpenWRT 2021-02-07 09:29:01 +00:00
Luca Deri be58d78417 (Temporarely) Disabled fragment manager (#1129) 2021-02-05 23:03:08 +01:00
Luca Deri 45e9c3c438 Partial fix for #1129 2021-02-05 22:22:33 +01:00
Alfredo Cardigliano 95a6a2072b Code cleanup and safety checks in the fragment manager (#1129) 2021-02-05 17:16:11 +01:00
Luca Deri 8dd7716ae5 Implemented more efficient and memory savvy RSI 2021-02-05 12:38:41 +01:00
Luca Deri 60b58dbd67 RSI enhancements 2021-02-05 10:59:09 +01:00
Luca Deri 1eedf734be Implemented API for computing RSI (Relative Strenght Index)
void  ndpi_init_rsi(struct ndpi_rsi_struct *s, u_int16_t num_learning_values);
void  ndpi_free_rsi(struct ndpi_rsi_struct *s);
float ndpi_rsi_add_value(struct ndpi_rsi_struct *s, const u_int32_t value);
2021-02-04 23:52:33 +01:00
Luca Deri 54636a3213 Improved (partial) TLS dissection 2021-02-04 22:06:18 +01:00
Ivan Nardi 679c317c57 Fix utests when "--disable-gcrypt" flag is used (#1128)
Fix: d6684f4b
2021-02-04 19:29:48 +01:00
Ivan Nardi 2a65321884 Fix some memory leakes in reassembler code (#1127) 2021-02-04 19:29:27 +01:00
ToniandLuca Deri 7255dbe4b7 Added fuzzy targets conditional in tests/do.sh.in which prevents the fuzzer from running if nDPI was configured previously --enable-fuzztargets but not for the current config (may produce invalid results). (#1126)
* fixed possible NULL pointer dereference for memcpy(), src pointer should never be NULL

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2021-02-04 19:28:52 +01:00
Luca Deri c68c8c79c9 Added missing check 2021-02-04 18:17:45 +01:00
Luca Deri 6c366d73e2 Fixed leak with DTLS 2021-02-03 23:36:11 +01:00
Luca Deri 6289413fea Updated results 2021-02-03 11:58:17 +01:00
Ivan Nardi a772e18977 Fix a warning (#1125)
Introduced in 5f7b9d802

reader_util.c: In function ‘process_ndpi_collected_info’:
reader_util.c:1148:60: warning: ‘%s’ directive output may be truncated writing up to 255 bytes into a region of size 64 [-Wformat-truncation=]
 1148 |       sizeof(flow->ssh_tls.client_requested_server_name), "%s",
      |                                                            ^~
reader_util.c:1147:5: note: ‘snprintf’ output between 1 and 256 bytes into a destination of size 64
 1147 |     snprintf(flow->ssh_tls.client_requested_server_name,
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 1148 |       sizeof(flow->ssh_tls.client_requested_server_name), "%s",
      |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 1149 |       flow->ndpi_flow->protos.tls_quic_stun.tls_quic.client_requested_server_name);
      |       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2021-02-03 11:56:37 +01:00
Luca Deri d6684f4b4c Added test pcap 2021-02-03 11:56:14 +01:00
Ivan Nardi 8c0ea694f8 HTTP: fix user-agent parsing (#1124)
User-agent information is used to try to detect the user OS; since the
UA is extracted for QUIC traffic too, the "detected_os" field must be
generic and not associated to HTTP flows only.

Otherwise, you might overwrite some "tls_quic_stun" fields (SNI...) with
random data.

Strangely enough, the "detected_os" field is never used: it is never
logged, or printed, or exported...
2021-02-03 11:54:11 +01:00
Ivan Nardi 8cee718e8b HTTP: fix logs when NDPI_ENABLE_DEBUG_MESSAGES is defined (#1123) 2021-02-03 11:53:41 +01:00
Luca Deri 155b1aaa68 Updated results due to the new fragment manager handler 2021-02-03 11:52:46 +01:00
Luca Deri 6e87daab69 Cosmetic fixes 2021-02-03 11:52:32 +01:00
Luca Deri 1f3ade5923 Increased number of extra packets that is necessary since the frgament mananger introduction 2021-02-03 11:51:55 +01:00
Luca Deri 4b181be58e Improved debug message 2021-02-03 11:49:14 +01:00
Luca Deri d19bad1581 Added pcap for testing fragments reassembly 2021-02-03 11:48:53 +01:00
Luca Deri 01f4a57118 Fixes an issue with https://github.com/ntop/nDPI/pull/1122 that misprocsssed packets belonging to flows whose initial part (e.g. the 3WH) was not observed by nDPI (e.g. capture started in the middle of the flow) 2021-02-03 11:47:21 +01:00
b70ad0e2f1 fragments management added (#1122)
Management of tcp segments managements.

Co-authored-by: ragostino <ragostino73@gmail.com>
Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2021-02-03 10:28:51 +01:00
ragostino ee94534906 debug message bugfix (#1108)
you can not look for memory enlargement if you print debug message after updating the variables
2021-02-03 09:31:17 +01:00
Luca Deri 50f4049b58 Improved wireguard dissection 2021-01-29 16:38:43 +01:00
Luca Deri bb74b903d0 DCE/RPC improvement to avoid false positives 2021-01-29 16:23:18 +01:00
Luca Deri aeeccee106 DGA name improvement 2021-01-27 11:43:35 +01:00
Luca Deri aa5486f5f8 Added missing library 2021-01-22 09:31:37 +01:00
Luca Deri 288ccd6215 Fixes due to datatype rename 2021-01-22 09:17:34 +01:00
Luca Deri a31bd5ac3c Cleaned up tls/quic datatypes 2021-01-21 19:17:33 +01:00
Luca Deri 15295ef4c5 Reworked TLS fingerprint calcolation
Modified TLS memory free
2021-01-21 19:06:05 +01:00
Toni 399755607d Disable tests that require libgcrypt if --disable-gcrypt set. (#1121)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-01-21 08:59:06 +01:00
morefigs 53415c8855 Added missing comma (#1116)
I presume there is a comma missing in this comma separated list.
2021-01-21 08:58:42 +01:00
Luca Deri 3e5e9569ff Added simple hash implementation to the nDPI API 2021-01-20 21:30:19 +01:00
Luca Deri d964c3e081 Code cleanup: third party uthash is at the right place 2021-01-20 19:11:36 +01:00
Luca 0809956e5f Rewored UPnP protocol that in essence was WSD hence it has been renamed
Cleaned up TLS code for DTLS detection by defining a new DTLS protocol
2021-01-20 09:53:30 +01:00
Luca Deri e9f43516ec Improves STUN dissection removing an invalid termination condition that prevented Skype calls to be properly identified 2021-01-13 16:46:51 +01:00
Luca Deri 68b6ac7da8 (C) Update 2021-01-07 11:13:36 +01:00
Luca Deri 367184628e Warning fix 2021-01-07 11:07:17 +01:00
Darryl Sokoloski 5f7b9d8024 Increase SNI hostname buffer length to 256. (#1111)
According to RFC 4366, SNI host names can be up to 255 bytes.
Previous size of 64 resulted in failed application matches due to truncation.

For example:
 0976e041e65b1aece3e720df36ac6bd7.safeframe.googlesyndication.co|m

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2021-01-07 10:58:48 +01:00
Ivan Nardi ec2735694b STUN: avoid false positives (#1110)
STUN traffic doesn't use multicast addresses
2021-01-07 10:58:22 +01:00
Ivan Nardi 1e2b57bed4 HTTP: fix compilation and a memory error when NDPI_ENABLE_DEBUG_MESSAGES is defined (#1109) 2021-01-07 10:58:02 +01:00
Ivan Nardi 2080cc7365 QUIC: add suppport for DNS-over-QUIC (#1107)
Even if it is only an early internet draft, DoQ has already (at least)
one deployed implementation.
See: https://www.zdnet.com/article/ad-blocker-adguard-deploys-worlds-first-dns-over-quic-resolver/
Draft: https://tools.ietf.org/html/draft-huitema-dprive-dnsoquic-00

In the future, if this protocol will be really used, it might be worth to
rename NDPI_PROTOCOL_DOH_DOT in NDPI_PROTOCOL_DOH_DOT_DOQ
2021-01-07 10:56:39 +01:00
Ivan Nardi 00dabce65e Quic fixes (#1106)
* QUIC: fix heap-buffer-overflow

* TLS: fix parsing of QUIC Transport Parameters
2021-01-07 10:55:55 +01:00
Ivan Nardi b8a5358e80 QUIC: improve handling of SNI (#1105)
* QUIC: SNI should be always saved in flow->protos.stun_ssl.ssl.client_requested_server_name

Close #1077

* QUIC: fix matching of custom categories

* QUIC: add NDPI_TLS_MISSING_SNI support for older GQUIC versions

* QUIC: fix serialization

* QUIC: add DGA check for older GQUIC versions
2021-01-07 10:55:23 +01:00
Luca Deri eb37f8f1fb Split HTTP request from response Content-Type. Request Content-Type should be present with POSTs and not with other methods such as GET 2021-01-06 18:28:24 +01:00
Luca Deri cd21f0d316 Added check for invalid HTTP content 2021-01-06 12:25:32 +01:00
Ivan Nardi 1b524f5538 QUIC: update to draft-33 (#1104)
QUIC (final!?) constants for v1 are defined in draft-33
2021-01-04 15:50:14 +01:00
Ivan Nardi 3aa16b63e8 Fix some warnings when compiling with "-W -Wall" flags (#1103) 2021-01-04 15:49:39 +01:00
Ivan Nardi 23b84cd3ee Remove FB_ZERO protocol (#1102)
FB_ZERO was an experimental protocol run by Facebook.
They switched to QUIC/TLS1.3 more than 2 years ago; no one ever used it but
them so it is definitely dead.
See: https://engineering.fb.com/2018/08/06/security/fizz/
2021-01-04 15:49:19 +01:00
Toni 53ee6db795 Added a new API function `ndpi_free_flow_data' which free's all members of ndpi_flow_struct but not the struct itself. (#1101)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-01-04 12:47:05 +01:00
Ivan Nardi 3bb12fde63 Fix memory leak introduced in b7376cc6 (#1100) 2021-01-04 12:46:30 +01:00
Luca Deri 4ddb5f4245 Added TLS test with long certificate 2021-01-04 11:31:25 +01:00
Luca Deri a91e6179c5 Fixed missing symbol 2021-01-02 21:21:15 +01:00
Luca Deri 05d76525b0 Added HTTP suspicious content securirty risk (useful for tracking trickbot) 2021-01-02 21:11:42 +01:00
Luca Deri 32f0446c9c Updated ndpi_ptree_match_addr() prototype 2020-12-30 18:23:40 +01:00
Luca Deri b7376cc690 Restored QUIC stats 2020-12-30 12:12:33 +01:00
Luca Deri dc401f8a74 Split ptree user data in 32 and 64 bit entries 2020-12-30 12:12:33 +01:00
pengtian ab037e2c82 Bugfix for host check (#1097)
this bug is from commit `427002d14` `2020-05-06 00:31:40`
2020-12-29 14:41:41 +01:00
Luca Deri 84132c6735 Added known protocol on unknown port for ntop 2020-12-28 16:11:39 +01:00
Luca Deri 7f944cc43b Initialization fix 2020-12-28 11:57:20 +01:00
Luca Deri 0fceb6576a Free flow fix 2020-12-28 09:32:06 +01:00
Luca Deri 9c1827a77b Fixed output when tLS (nad not QUIC) is used 2020-12-28 09:19:39 +01:00
Luca Deri cf00ffed32 Removed test code 2020-12-26 18:13:10 +01:00
Luca Deri e1be363037 Removed space from protocol name 2020-12-23 20:38:02 +00:00
Luca Deri 34fc9d5d50 Introduced fix on TLS for discarding traffic out of sequence that might invalidate dissection 2020-12-22 09:47:39 +01:00
Luca Deri 6462f4671c Fixed invalid TLS check for extra processing detection 2020-12-21 22:42:05 +01:00
Luca Deri b231982286 Win fixes 2020-12-17 15:34:17 +01:00
Luca Deri df1b3367a9 Type change to avoid Windows compilation issues 2020-12-17 12:58:12 +01:00
Luca Deri 2768da0637 Improved HTTP dissection 2020-12-16 14:45:29 +01:00
Luca Deri a89642ad04 Fixes bug introduced by https://github.com/ntop/nDPI/pull/1085 2020-12-12 12:32:44 +01:00
Zied Aouini 5bd5461f96 Fix minimum packet length condition (#1087) 2020-12-12 11:12:59 +01:00
Toni 74a77e7b3d Added --ignore-vlanid / -I to exclude VLAN ids for flow hash calculation. #1073 (#1085)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-12-11 21:01:51 +01:00
Toni edf3a57a6a Added `fast.com' host pattern to Netflix detection. #1080 (#1084)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-12-11 21:01:28 +01:00
Ivan Nardi 5587010cfb soulseek: fix heap buffer overflow (#1083)
Close #1082
2020-12-11 21:01:07 +01:00
Luca Deri 5cb6ddfd22 Rule changes work in progress 2020-12-11 17:25:57 +01:00
Luca Deri 21ad3a1775 Added initialization 2020-12-11 17:25:57 +01:00
Alfredo Cardigliano 499897219e Add NDPI_PROTOCOL_ANY_CATEGORY to ndpi_protocol_category_t enum 2020-12-11 15:56:57 +01:00
Luca Deri 9c5fbb5208 Added FreeBSD ibuils instructions 2020-12-10 14:19:59 +00:00
Luca Deri 4cb8712a35 Added --with-mipsel for building nDPI on mipsel devices 2020-12-09 11:01:40 +00:00
Toni 62cd852c6f Rename Jabber detection name as we are not sure if it is unencrypted e.g. if START_TLS used. (#1079)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-12-08 15:49:17 +01:00
rafaliuszandrafal 1ecc6d323e Add a connectionless DCE/RPC detection (#1078)
* Add connectionless DCE/RPC detection

* Add DCE/RPC pcap file as well as its test result

Co-authored-by: rafal <rafal.burzynski@cryptomage.com>
2020-12-08 15:48:53 +01:00
Ivan Nardi 860ef58ace QUIC: sync with Wireshark latest changes (#1074)
Most of the QUIC crypto code has been "copied-and-pasted" from Wireshark;
try to stay in sync with the original sources to ease backporting of fixes.

Only cosmetic changes and code refactoring; no behaviour changes or bugfixes.
See:
https://gitlab.com/wireshark/wireshark/-/commit/5e45f770fd79ca979c41ed397fee72d2e8fb5f1e
https://gitlab.com/wireshark/wireshark/-/commit/5798b91c1526747bf688b6746b33562c1b24a9e0
2020-12-08 15:47:58 +01:00
Luca Deri 1fb1848586 Added chec for avoiding doung ntop_config.h include 2020-12-01 08:57:09 +00:00
Luca Deri eb689b2069 nDPI rules (work in progress) implementation 2020-11-30 22:01:49 +01:00
Luca Deri b77a4276f9 Fixed JSON-C check 2020-11-30 16:41:58 +01:00
Luca Deri d4a42ed733 Improved JSON-C detection 2020-11-30 16:30:38 +01:00
Luca Deri 6b9155589a Removed unnecessary field 2020-11-29 19:04:26 +01:00
Luca Deri 23a15bae5f Fixes #1029 2020-11-27 18:51:56 +01:00
Luca Deri 6c119c0b06 Fixes #1032 2020-11-27 18:48:25 +01:00
Toni daa1c8ef8c Fix/overflow and libgerror check (#1068)
* Fixed stack overflow caused by missing buffer space for the trailing \0 added by sprintf()

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Remove the autoconf cache value from the previous and failed check before checking again.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-26 19:46:31 +01:00
Alfredo Cardigliano 7437ecaab0 Warning fix 2020-11-23 10:19:35 +01:00
Luca Deri 38984af03e Updated disney definition 2020-11-22 18:49:28 +01:00
Luca Deri 78f762bd36 Typo 2020-11-22 16:51:13 +01:00
Luca Deri 21a6112b91 Added 129.205.94.0/23 to Facebook network list 2020-11-22 11:33:31 +01:00
Ivan Nardi 53a5c354d8 Quic fixes (#1067)
* QUIC: fix return value on error path on quic_cipher_init()

* QUIC: allow dissection of sessions forcing version negotiation

Enhance heuristic to avoid false positives.
2020-11-22 11:04:10 +01:00
Ivan Nardi fb2027cc8e iec60870-5-104: fix heap-buffer-overflow error (#1066) 2020-11-22 11:03:26 +01:00
Luca Deri fe3d926741 Added prerequisites for Ubuntu/Debian systems 2020-11-18 16:40:45 +01:00
Alfredo Cardigliano e45f0a4b64 Install ndpi under /usr 2020-11-17 09:19:07 +01:00
Luca Deri d670850f55 Updated results 2020-11-16 21:25:59 +01:00
Luca Deri acffd7bd1e Compilation fix 2020-11-16 21:22:54 +01:00
Zied AouiniandLuca Deri bfabb0ddf4 Add Virtual Asssitant (Alexa, Siri) support. (#1057)
* Add AmazonAlexa protocol.

* Add AmazonAlexa test file and result.

* Include pcapng as file format.

* Rename Category to VirtualAssistant.

* Add AppleSiri virtual assistant.

* Fix pcapng test files format support.

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2020-11-16 21:19:38 +01:00
Oleksandr Natalenko 5c8e125f06 lib/ndpi_content_match: update TOR IPs (#1063)
Signed-off-by: Oleksandr Natalenko <oleksandr@natalenko.name>
2020-11-16 21:18:17 +01:00
Zied Aouini 3d8fd42307 Implement DGA detection performances tracking workflow. (#1064)
* Implement dga evaluation helper.

* Add test set for DGA classification.

* Add DGA classification performances tracking as part of Travis.

* Add DGA evaluation doc.

* Fix CI on OSX.

* Add missing backquote.
2020-11-16 21:17:16 +01:00
Zied Aouini 76bb83085b Improve subprotocols detection. (#1062)
* Improve Spotify detection.

* Improve Skype detection.

* Improve Microsoft detection.

* Fix Microsoft detection categories.

* Improve Waze detection.

* Improve Apple detection.

* Improve WindowsUpdate detection.

* Improve TikTok detection.

* Improve Teams detection.

* Improve Youtube detection.

* Improve Messenger detection.

* Improve Twitch detection.

* Improve Hulu detection.

* Improve Facebook detection.

* Improve AmazonVideo detection.
2020-11-16 21:16:38 +01:00
Zied AouiniandLuca Deri 3529268df8 Add Tumblr support. (#1061)
* Add Tumblr protocol.

* Add Tumblr test file and result.

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2020-11-16 21:14:06 +01:00
Zied AouiniandLuca Deri 22780da8d5 Add Reddit support. (#1060)
* Add Reddit protocol.

* Add Reddit test file and result.

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2020-11-16 21:13:01 +01:00
Zied AouiniandLuca Deri 13dab51cc7 Add Pinterest support. (#1059)
* Add Pinterest protocol.

* Add Pinterest test file and result.

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2020-11-16 21:11:43 +01:00
Zied Aouini f35fad13c3 Add DisneyPlus protocol. (#1058) 2020-11-16 21:10:11 +01:00
Luca Deri 419483f128 File nor necessary and generated by configure 2020-11-12 23:22:15 +01:00
Luca Deri 328ff24657 Renumbered AmongUs protocol 2020-11-09 16:23:01 +01:00
Toni 6b5bdf773d Added support for AmongUs. (#1054)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-09 16:19:00 +01:00
Toni af02ffb60f Support raw IPv4 / IPv6 pcap packet processing. (#1053)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-09 16:18:05 +01:00
Toni ac18d5a084 Improved SSH protocol detection. (#1052)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-09 16:17:42 +01:00
Don J. Rude bbf42c3610 fixes issue #1050 Syntax error caused buffer pointer to equal 0x1 (#1051)
* Syntax error caused buffer pointer to equal 0x1

Possible copy-paste from lines 141-142?

* Another comma operator

* whitespace matching

* another comma operator

* another comma operator

* another comma operator

* Check for non-zero payload
2020-11-09 16:17:10 +01:00
Luca Deri 89a363aff6 Updated ESNI/SNI alarm generation prolicy 2020-11-08 10:07:35 +01:00
Luca Deri f6cb869c63 Reworked IEC60870 dissector 2020-11-04 14:29:11 +01:00
Luca Deri 044a11faca IEC60870 dissection improvements 2020-11-04 12:40:31 +01:00
Luca Deri 017e395ed1 Cosmetic changes 2020-11-03 16:46:30 +01:00
Leonn 0576dc2a49 💡 Add mongodb protocol dissector (#1048) 2020-11-03 16:16:02 +01:00
Luca Deri fc58e564f0 Restored Ubuntu/Debian packaging 2020-11-03 16:08:57 +01:00
Toni 8e84abfa0e Fix/packaging (#1047)
* Fix OpenWrt build.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Fixed Debian/Ubuntu packaging.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Added DPKG package build to a CI job.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-11-03 11:36:40 +01:00
Ivan Nardi a9547da138 QUIC: fix dissection of Initial packets coalesced with 0-RTT one (#1044)
* QUIC: fix dissection of Initial packets coalesced with 0-RTT one

* QUIC: fix a memory leak
2020-11-03 11:35:52 +01:00
Luca Deri 92b80e2468 Updated results with numeric IP detection 2020-11-01 13:31:00 +01:00
Luca Deri b502bb08e0 Fix for detecting numeric IPs 2020-11-01 13:31:00 +01:00
Luca Deri 50ab34bfe6 Centos8 package fix 2020-10-29 18:36:56 +01:00
Alfredo Cardigliano 53f133f669 Fix libdir on centos8 2020-10-29 18:20:51 +01:00
Alfredo Cardigliano 29dbf78c4e Removed generation of /usr/lib/.build-id 2020-10-29 17:50:05 +01:00
Luca Deri efbfb7cbd6 Set /usr as installation prefix 2020-10-29 17:45:55 +01:00
Alfredo Cardigliano 1c4ce8b0f4 Move libraries from /usr/local to /usr 2020-10-29 12:43:33 +01:00
Vitaliy Ivanov 10506b5173 Generate NDPI_API_VERSION based on git commands.
https://github.com/ntop/PF_RING/issues/668

Check for .git directory works in case of usual clone of nDPI repo.
But if you integrate is as submodule then .git is a file.

Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com>
2020-10-29 12:35:05 +01:00
Luca Deri 94471193c2 Added boundary check 2020-10-27 08:48:12 +01:00
Igor Duarte ba6a48c9fe Improve skype detection (#1039)
* Add new skype pcap

PCAP extracted from SkypeIRC.cap (available in https://wiki.wireshark.org/SampleCaptures?action=AttachFile&do=get&target=SkypeIRC.cap)

* Improve skype detection
2020-10-27 08:45:09 +01:00
Luca Deri 48d640583a Moved global in reader_util.c 2020-10-27 08:40:00 +01:00
Luca Deri 948a906037 Added -D flag for detecting DoH in the wild
Removed heuristic from CiscoVPN as it leads to false positives
2020-10-26 21:40:59 +01:00
Luca Deri 9873972acb Various improvemement when using ndpi_pref_enable_tls_block_dissection:
application data TLS blocks are now ignored when exchanged before
- the end of certificate negotiation (up to TLS 1.2)
- change cipher
2020-10-24 19:22:56 +02:00
Luca Deri 9b85669a64 Added -x for checking patterns 2020-10-22 23:28:18 +02:00
Luca Deri 833d0eee53 Added CPHA - CheckPoint High Availability Protocol protocl support 2020-10-22 18:39:13 +02:00
Ivan Nardi 6027a7c799 Fix parsing of DLT_PPP datalink type (#1042) 2020-10-21 22:27:42 +02:00
Luca Deri 9dac9945c9 Fixes #1033 2020-10-21 20:59:02 +02:00
DouglasandDouglas Wordell efb712c2c0 Added new IPs for TeamViewer (#1040)
Co-authored-by: Douglas Wordell <douglas.wordell@seti.com.br>
2020-10-21 15:09:14 +02:00
Zied Aouini 43c1f6a3fd CAPWAP tunnel decoding fix (#1038)
* Fix CAPWAP processing.

* Update result.
2020-10-21 15:07:20 +02:00
Toni 822c8e56f7 Improved/autoconf (#1037)
* Switched to PKG_CHECK_MODULES to keep pkg-config checks more portable.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Improved GCrypt autoconf check to detect a possible gpg-error inter-library dependency.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-10-21 15:05:33 +02:00
Hosein Ghahremanzadeh bb33d57971 Fix a memory leak on fuzzing code (#1036)
After allocation of pcap_buffer it is necessary to free it
2020-10-21 15:05:11 +02:00
Igor Duarte 13d99a9390 Remove possible invalid read (#1035) 2020-10-21 15:04:55 +02:00
Luca Deri 6c0a809de8 Initial 3.5 commit 2020-10-19 16:23:03 +02:00
673 changed files with 176730 additions and 24416 deletions
-17
View File
@@ -1,17 +0,0 @@
version: 3.1.{build}
branches:
only:
- dev
environment:
matrix:
- COMPILER: mingw
MINGW_DIR: c:\msys64\mingw32
before_build:
- set Path=%MINGW_DIR%\bin;%Path%
build_script:
- set Path=%MINGW_DIR%\bin;c:\msys64\usr\bin;
- bash autogen.sh
- cd src/lib
- make
+3
View File
@@ -1,3 +1,6 @@
*.sh -crlf
*.ac -crlf
*.am -crlf
#Content files are C file
*.inc linguist-language=C
+50
View File
@@ -0,0 +1,50 @@
---
name: "🐞 Report a Bug"
about: Create a report to help us improve nDPI
title: ''
labels: 'bug'
assignees: ''
---
## Describe the bug
A clear and concise description of what the bug is.
### Expected behavior
A clear and concise description of what you expected to happen.
### Obtained behavior
A clear and concise description of what happening.
## nDPI Environment (please complete the following information):
* OS name: [e.g. Ubuntu].
* OS version: [e.g. 18.04]
* Architecture: [e.g. arm64]
* nDPI version or commit hash: [e.g. 4.0-stable or 937357e4bc55610f116f66d15a8e0fc1e260c02c].
* nDPI compilation flags used: if you are building from source [e.g. --with-pcre --disable-gcrypt].
* Attach the `config.log` file generated after `./configure` ran (if you are building from source).
## How to reproduce the reported bug
### Reproducible using ndpiReader?
- [x] The reported bug is reproducible using ndpiReader.
- [x] The reported bug is not reproducible using ndpiReader.
### If applicable, the used ndpiReader options:
``` shell
ndpiReader -q -t -i wlo1 -T 20 -U 20
```
### If your bug is reproducible using a pcap, please attach a pcap file (or a valid link to download it)
Example: test.pcap
### Steps to reproduce the behavior:
1. Run '...'
2. Set '....'
3. Do '....'
4. See error
## Additional context
Add any other context about the problem here.
@@ -0,0 +1,20 @@
---
name: "💡 Suggest an Improvement or Feature"
about: Suggest an idea or a feature for nDPI
title: ''
labels: 'enhancement'
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I always need to [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
@@ -0,0 +1,10 @@
---
name: General question
about: Ask a question about nDPI
title: ''
labels: 'question'
assignees: ''
---
Type your question in a clear and concise way to help the community answering it.
+289
View File
@@ -0,0 +1,289 @@
name: Build
on:
push:
branches:
- dev
pull_request:
branches:
- dev
types: [opened, synchronize, reopened]
release:
types: [created]
jobs:
test:
name: ${{ matrix.os }} ${{ matrix.arch }} ${{ matrix.gcrypt }} ${{ matrix.compiler }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} ${{ matrix.msan }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false # Debugging purposes, switch to true when all issues on big endian is solved
matrix:
os: ["ubuntu-latest", "ubuntu-18.04", "macOS-latest", "macos-11"]
arch: ["x86_64"]
gcrypt: ["--disable-gcrypt", ""]
compiler: ["default-cc"]
pcre: [""]
maxminddb: [""]
msan: [""]
include:
- compiler: "gcc-7"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "gcc-10"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "clang-7"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "clang-12"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "default-cc"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "default-cc"
os: ubuntu-18.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "default-cc"
os: macOS-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "" # Disable sanitizer on macos
- compiler: "default-cc"
os: macos-11
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "" # Disable sanitizer on macos
- compiler: "default-cc"
os: ubuntu-latest
arch: "arm64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "" # Disable sanitizer on arm64
- compiler: "default-cc"
os: ubuntu-latest
arch: "armhf"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "default-cc"
os: ubuntu-latest
arch: "s390x"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: ""
steps:
- name: Setup multiarch/qemu-user-static
if: startsWith(matrix.os, 'ubuntu') && !startsWith(matrix.arch, 'x86_64')
run: |
docker run --rm --privileged multiarch/qemu-user-static:register --reset
- uses: actions/checkout@v2
with:
fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis
- name: Install Ubuntu Prerequisites
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
sudo apt-get install gcc-mingw-w64 libc6-dev
sudo apt-get install doxygen python3-sphinx python3-sphinx-rtd-theme python3-breathe python3-pip
sudo apt-get install rrdtool librrd-dev
- name: Install Ubuntu Prerequisites (libgcrypt)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.gcrypt, '--disable-gcrypt')
run: |
sudo apt-get install libgcrypt20-dev
- name: Install Ubuntu Prerequisites (libpcre)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.pcre, '--with-pcre')
run: |
sudo apt-get install libpcre3-dev
- name: Install Ubuntu Prerequisites (maxminddb)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.maxminddb, '--with-maxminddb')
run: |
sudo apt-get install libmaxminddb-dev
- name: Setup Ubuntu specified compiler (gcc)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'gcc')
run: |
sudo apt-get install ${{ matrix.compiler }}
- name: Setup Ubuntu specified compiler (clang)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'clang')
run: |
sudo apt-get install ${{ matrix.compiler }}
- name: Installing MacOS prerequisites
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64')
run: |
brew install autoconf automake libtool pkg-config gettext json-c
brew install coreutils
brew install rrdtool
- name: Install MacOS Prerequisites (libgcrypt)
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.gcrypt, '--disable-gcrypt')
run: |
brew install libgcrypt
- name: Install MacOS Prerequisites (libpcre)
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.pcre, '--with-pcre')
run: |
brew install pcre
- name: Install MacOS Prerequisites (maxminddb)
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.maxminddb, '--with-maxminddb')
run: |
brew install libmaxminddb
- name: Configure nDPI on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'default-cc')
run: |
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }}
- name: Configure nDPI on MacOS
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'default-cc')
run: |
env CC=clang CFLAGS='-Werror' ./autogen.sh --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }}
- name: Configure nDPI with specified GCC version on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'gcc')
run: |
env CC=${{ matrix.compiler }} CFLAGS='-Werror' ./autogen.sh --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }}
- name: Configure nDPI with specified CLANG on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'clang')
run: |
env CC=${{ matrix.compiler }} CFLAGS='-Werror' ./autogen.sh --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }}
- name: Build nDPI
if: startsWith(matrix.arch, 'x86_64')
run: |
make all
make -C example ndpiSimpleIntegration
make -C rrdtool
make -C python
- name: Install nDPI
if: startsWith(matrix.arch, 'x86_64')
run: |
make install DESTDIR=$(realpath _install)
ls -alhHR _install
- name: Test nDPI [DIFF]
if: startsWith(matrix.arch, 'x86_64')
run: |
./tests/do.sh
- name: Test nDPI [UNIT]
if: startsWith(matrix.arch, 'x86_64')
run: |
./tests/do-unit.sh
- name: Test nDPI [DGA]
if: startsWith(matrix.arch, 'x86_64')
run: |
./tests/do-dga.sh
- name: Generate tarball
if: startsWith(matrix.os, 'ubuntu-latest') && startsWith(matrix.arch, 'x86_64')
run: |
make dist
- name: Generate documentation
if: startsWith(matrix.os, 'ubuntu-latest') && startsWith(matrix.arch, 'x86_64')
run: |
pip3 install --user sphinxcontrib.swaggerdoc
make -C doc text
make -C doc html
- name: Configure nDPI [Mingw-w64] (runs only on ubuntu jobs)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
run: |
make distclean && ./autogen.sh --host=x86_64-w64-mingw32 ${{ matrix.gcrypt }} ${{ matrix.pcre }} ${{ matrix.maxminddb }}
- name: Build nDPI [Mingw-w64] (runs only on ubuntu jobs)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
run: |
make all
- name: Display qemu specified architecture (arm64 - little endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'arm64')
uses: docker://multiarch/ubuntu-core:arm64-bionic
with:
args: >
bash -c
"uname -a &&
lscpu | grep Endian
"
- name: Configure, compile and test using qemu for the specified architecture (arm64 - little endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'arm64')
uses: docker://multiarch/ubuntu-core:arm64-bionic
with:
args: >
bash -c
"apt-get -y update &&
apt-get -y install git autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} &&
make all &&
make -C example ndpiSimpleIntegration &&
make -C rrdtool &&
make -C python &&
./tests/do.sh &&
./tests/do-unit.sh
"
- name: Display qemu specified architecture (armhf - little endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'armhf')
uses: docker://multiarch/ubuntu-core:armhf-bionic
with:
args: >
bash -c
"uname -a &&
lscpu | grep Endian
"
- name: Configure, compile and test using qemu for the specified architecture (armhf - little endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'armhf')
uses: docker://multiarch/ubuntu-core:armhf-bionic
with:
args: >
bash -c
"apt-get -y update &&
apt-get -y install git autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} &&
make all &&
make -C example ndpiSimpleIntegration &&
make -C rrdtool &&
make -C python &&
./tests/do.sh &&
./tests/do-unit.sh
"
- name: Display qemu specified architecture (s390x - big endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 's390x')
uses: docker://multiarch/ubuntu-core:s390x-bionic
with:
args: >
bash -c
"uname -a &&
lscpu | grep Endian
"
- name: Configure and compile (no tests) using qemu for the specified architecture (s390x - big endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 's390x')
uses: docker://multiarch/ubuntu-core:s390x-bionic
with: #./tests/do.sh disabled because we know we have some problems with big-endian machines
args: >
bash -c
"apt-get -y update &&
apt-get -y install git autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} &&
make all &&
make -C example ndpiSimpleIntegration &&
make -C rrdtool &&
make -C python &&
./tests/do-unit.sh
"
+32 -21
View File
@@ -1,23 +1,34 @@
name: CIFuzz
on: [pull_request]
on: [push, pull_request]
jobs:
Fuzzing:
runs-on: ubuntu-latest
steps:
- name: Build Fuzzers
uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@master
with:
oss-fuzz-project-name: 'ndpi'
dry-run: false
- name: Run Fuzzers
uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@master
with:
oss-fuzz-project-name: 'ndpi'
fuzz-seconds: 600
dry-run: false
- name: Upload Crash
uses: actions/upload-artifact@v1
if: failure()
with:
name: artifacts
path: ./out/artifacts
Fuzzing:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
sanitizer: [address, undefined, memory]
steps:
- name: Build Fuzzers - ${{ matrix.sanitizer }}
uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@master
with:
oss-fuzz-project-name: 'ndpi'
dry-run: false
sanitizer: ${{ matrix.sanitizer }}
- name: Run Fuzzers - ${{ matrix.sanitizer }}
uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@master
with:
oss-fuzz-project-name: 'ndpi'
fuzz-seconds: 600
dry-run: false
sanitizer: ${{ matrix.sanitizer }}
- name: Check Crash (fails when a crash is detected)
# Run Fuzzers return success even when setting dry-run to false.
# A temporal workaround is to trigger failure manually if we fing crash files.
run: |
exit $(ls out/artifacts |wc -l)
- name: Upload Crash (upload detected crash as artifacts)
uses: actions/upload-artifact@v1
if: failure()
with:
name: ${{ matrix.sanitizer }}-artifacts
path: ./out/artifacts
+26 -1
View File
@@ -11,7 +11,6 @@ compile
/config.status
/config.sub
/configure
/configure.ac
/configure.tmp
.cproject
/depcomp
@@ -21,6 +20,7 @@ compile
/example/ndpiReader
/example/Makefile.dpdk
/example/build
/example/ndpiSimpleIntegration
*.in
/install-sh
*.ko
@@ -45,6 +45,7 @@ compile
.settings
/src/include/Makefile
/src/include/ndpi_config.h
/src/include/ndpi_api.h
/src/include/ndpi_define.h
/src/lib/.deps/
/src/lib/Makefile
@@ -54,3 +55,27 @@ stamp-h1
*.swp
/test-driver
/tests/Makefile
/fuzz/.deps/
/fuzz/Makefile
/fuzz/fuzz_ndpi_reader
/fuzz/fuzz_ndpi_reader_with_main
/fuzz/fuzz_process_packet
/fuzz/fuzz_process_packet_with_main
/python/Makefile
/tests/dga/Makefile
/tests/do.sh
/tests/unit/Makefile
/example/libndpiReader.a
/example/ndpiReader.exe
/src/lib/libndpi.a
/src/lib/libndpi.so*
/src/lib/libndpi
/src/lib/libndpi-4.1.0.dll
/src/lib/libndpi.4
/tests/dga/dga_evaluate
/tests/unit/unit
/tags
/rrdtool/Makefile
/rrdtool/rrd_anomaly
/rrdtool/rrd_similarity
/tests/performance/Makefile
-214
View File
@@ -1,214 +0,0 @@
language: c
matrix:
include:
- os: osx
compiler: clang
- os: linux
compiler: gcc
dist: bionic
addons:
apt:
packages:
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
- os: linux
compiler: clang
dist: bionic
addons:
apt:
packages:
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
- os: linux
compiler: gcc
dist: bionic
arch: arm64
addons:
apt:
packages:
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
- os: linux
compiler: clang
dist: bionic
arch: arm64
addons:
apt:
packages:
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
# Targets below have been disabled as we have no way
# to debug on tsuch platforms at the moment
# - os: linux
# compiler: gcc
# dist: bionic
# arch: s390x
# addons:
# apt:
# packages:
# - libpcap-dev
# - libjson-c-dev
# - autogen
#
# - os: linux
# compiler: clang
# dist: bionic
# arch: s390x
# addons:
# apt:
# packages:
# - libpcap-dev
# - libjson-c-dev
# - autogen
- os: linux
compiler: gcc-8
addons:
apt:
sources:
- ubuntu-toolchain-r-test
packages:
- g++-8
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
env:
- MATRIX_EVAL="CC=gcc-8"
- os: linux
compiler: gcc-9
addons:
apt:
sources:
- ubuntu-toolchain-r-test
packages:
- g++-9
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
env:
- MATRIX_EVAL="CC=gcc-9"
- os: linux
compiler: clang-8
addons:
apt:
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty-8
packages:
- clang-8
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
env:
- MATRIX_EVAL="CC=clang-8"
- os: linux
compiler: clang-7
addons:
apt:
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty-7
packages:
- clang-7
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
env:
- MATRIX_EVAL="CC=clang-7"
- name: fuzza
env: CXXFLAGS="-g3 -O0 -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize=fuzzer-no-link" CFLAGS="-g3 -O0 -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize=fuzzer-no-link" LDFLAGS="-g3 -O0 -fsanitize=address" QA_FUZZ=asan CC=clang-7 && CXX=clang++-7 ASAN_SYMBOLIZER_PATH=/usr/local/clang-7.0.0/bin/llvm-symbolizer
os: linux
compiler: clang-7
addons:
apt:
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty-7
packages:
- clang-7
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
- name: fuzzm
env: CXXFLAGS="-g3 -O0 -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=memory -fsanitize=fuzzer-no-link" CFLAGS="-g3 -O0 -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=memory -fsanitize=fuzzer-no-link" LDFLAGS="-g3 -O0 -fsanitize=memory" QA_FUZZ=msan CC=clang && CXX=clang++
os: linux
compiler: clang
dist: bionic
addons:
apt:
packages:
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
- name: fuzzu
env: CXXFLAGS="-g3 -Og -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=undefined -fsanitize=fuzzer-no-link -fno-sanitize=alignment" CFLAGS="-g3 -Og -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=undefined -fno-sanitize-recover=undefined,integer -fsanitize=fuzzer-no-link -fno-sanitize=alignment" LDFLAGS="-g3 -Og -fsanitize=undefined -fno-sanitize=alignment" QA_FUZZ=ubsan CC=clang-7 && CXX=clang++-7
os: linux
compiler: clang-7
addons:
apt:
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty-7
packages:
- clang-7
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
before_install:
- eval "${MATRIX_EVAL}"
before_script:
- ./autogen.sh
# - lcov --directory . --zerocounters
script:
- if [ -n "$QA_FUZZ" ]; then ./configure --enable-fuzztargets ; else ./configure ; fi
- make
- make -C example ndpiSimpleIntegration
- make dist
#after_script:
- cd tests
- ./do.sh
- ./do-unit.sh
#after_success:
#- cd ${TRAVIS_BUILD_DIR}
#- lcov --directory . --capture --output-file coverage.info # capture coverage info
#- lcov --remove coverage.info 'tests/*' 'packages/*' 'm4/*' 'doc/*' --output-file coverage.info
#- lcov --list coverage.info # debug before upload
#- coveralls-lcov --repo-token ${COVERALLS_TOKEN} coverage.info # uploads to coveralls
notifications:
email:
recipients:
- packager@ntop.org
on_success: never
on_failure: always
+228
View File
@@ -1,5 +1,233 @@
# CHANGELOG
#### nDPI 4.2 (Feb 2022)
## New Features
- Add a "confidence" field indicating the reliability of the classification
- Add risk exceptions for services and domain names via ndpi_add_domain_risk_exceptions()
- Add ability to report whether a protocol is encrypted
## New Supported Protocols and Services
- Add protocol detection for:
- Badoo
- Cassandra
- EthernetIP
## Improvements
- Reduce memory footprint
- Improve protocol detection for:
- BitTorrent
- ICloud Private Relay
- IMAP, POP3, SMTP
- Log4J/Log4Shell
- Microsoft Azure
- Pandora TV
- RTP
- RTSP
- Salesforce
- STUN
- Whatsapp
- QUICv2
- Zoom
- Add flow risk:
- NDPI_CLEAR_TEXT_CREDENTIALS
- NDPI_POSSIBLE_EXPLOIT (Log4J)
- NDPI_TLS_FATAL_ALERT
- NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE
- Update WhatsAPP and Instagram addresses
- Update the list of default ports for QUIC
- Update WindowsUpdate URLs
- Add support for the .goog Google TLD
- Add googletagmanager.com
- Add bitmaps and API for handling compressed bitmaps
- Add JA3 in risk exceptions
- Add entropy calculation to check for suspicious (encrypted) payload
- Add extraction of hostname in SMTP
- Add RDP over UDP dissection
- Add support for TLS over IPV6 in Subject Alt Names field
- Improve JSON and CSV serialization
- Improve IPv6 support for almost all dissectors
- Improve CI and unit tests, add arm64, armhf and s390x as part of CI
- Improve WHOIS detection, reduce false positives
- Improve DGA detection for skipping potential DGAs of known/popular domain names
- Improve user agent analysis
- Reworked HTTP protocol dissection including HTTP proxy and HTTP connect
## Changes
- TLS obsolete protocol is set when TLS < 1.2 (used to be 1.1)
- Numeric IPs are not considered for DGA checks
- Differentiate between standard Amazon stuff (i.e market) and AWS
- Remove Playstation VUE protocol
- Remove pandora.tv from Pandora protocol
- Remove outdated SoulSeek dissector
## Fixes
- Fix race conditions
- Fix dissectors to be big-endian friendly
- Fix heap overflow in realloc wrapper
- Fix errors in Kerberos, TLS, H323, Netbios, CSGO, Bittorrent
- Fix wrong tuple comparison
- Fix ndpi_serialize_string_int64
- Fix Grease values parsing
- Fix certificate mismatch check
- Fix null-dereference read for Zattoo with IPv6
- Fix dissectors initialization for XBox, Diameter
- Fix confidence for STUN classifications
- Fix FreeBSD support
- Fix old GQUIC versions on big-endian machines
- Fix aho-corasick on big-endian machines
- Fix DGA false positive
- Fix integer overflow for QUIC
- Fix HTTP false positives
- Fix SonarCloud-CI support
- Fix clashes setting the hostname on similar protocols (FTP, SMTP)
- Fix some invalid TLS guesses
- Fix crash on ARM (Raspberry)
- Fix DNS (including fragmented DNS) dissection
- Fix parsing of IPv6 packets with extension headers
- Fix extraction of Realm attribute in STUN
- Fix support for START-TLS sessions in FTP
- Fix TCP retransmissions for multiple dissectors
- Fix DES initialisation
- Fix Git protocol dissection
- Fix certificate mismatch for TLS flows with no client hello observed
- Fix old versions of GQUIC on big-endian machines
## Misc
- Add tool for generating automatically the Azure IP list
#### nDPI 4.0 (July 2021)
## New Features
- Add API for computing RSI (Relative Strenght Index)
- Add GeoIP support
- Add fragments management
- Add API for jitter calculation
- Add single exponential smoothing API
- Add timeseries forecasting support implementing Holt-Winters with confidence interval
- Add support for MAC to radi tree and expose the full API to applications
- Add JA3+, with ALPN and elliptic curve
- Add double exponential smoothing implementation
- Extended API for managing flow risks
- Add flow risk score
- New flow risks:
- Desktop or File Sharing Session
- HTTP suspicious content (useful for tracking trickbot)
- Malicious JA3
- Malicious SHA1
- Risky domain
- Risky AS
- TLS Certificate Validity Too Long
- TLS Suspicious Extension
## New Supported Protocols and Services
- New protocols:
- AmongUs
- AVAST SecureDNS
- CPHA (CheckPoint High Availability Protocol)
- DisneyPlus
- DTLS
- Genshin Impact
- HP Virtual Machine Group Management (hpvirtgrp)
- Mongodb
- Pinterest
- Reddit
- Snapchat VoIP calls
- Tumblr
- Virtual Asssitant (Alexa, Siri)
- Z39.50
- Add protocols to HTTP as subprotocols
- Add detection of TLS browser type
- Add connectionless DCE/RPC detection
## Improvements
- 2.5x speed bump. Example ndpiReader with a long mixed pcap
v3.4 - nDPI throughput: 1.29 M pps / 3.35 Gb/sec
v4.0 - nDPI throughput: 3.35 M pps / 8.68 Gb/sec
- Improve detection/dissection of:
- AnyDesk
- DNS
- Hulu
- DCE/RPC (avoid false positives)
- dnscrypt
- Facebook (add new networks)
- Fortigate
- FTP Control
- HTTP
- Fix user-agent parsing
- Fix logs when NDPI_ENABLE_DEBUG_MESSAGES is defined
- IEC104
- IEC60870
- IRC
- Netbios
- Netflix
- Ookla speedtest (detection over IPv6)
- openspeedtest.com
- Outlook / MicrosoftMail
- QUIC
- update to draft-33
- improve handling of SNI
- support for fragmented Client Hello
- support for DNS-over-QUIC
- RTSP
- RTSP via HTTP
- SNMP (reimplemented)
- Skype
- SSH
- Steam (Steam Datagram Relay - SDR)
- STUN (avoid false positives, improved Skype detection)
- TeamViewer (add new hosts)
- TOR (update hosts)
- TLS
- Certificate Subject matching
- Check for common ALPNs
- Reworked fingerprint calculation
- Fix extraction for TLS signature algorithms
- Fix ClientHello parsing
- UPnP
- wireguard
- Improve DGA detection
- Improve JA3
- Improve Mining detection
- Improve string matching algorithm
- Improve ndpi_pref_enable_tls_block_dissection
- Optimize speed and memory size
- Update ahocorasick library
- Improve subprotocols detection
## Fixes
- Fix partial application matching
- Fix multiple segfault and leaks
- Fix uninitialized memory use
- Fix release of patterns allocated in ndpi_add_string_to_automa
- Fix return value of ndpi_match_string_subprotocol
- Fix setting of flow risks on 32 bit machines
- Fix TLS certificate threshold
- Fix a memory error in TLS JA3 code
- Fix false positives in Z39.50
- Fix off-by-one memory error for TLS-JA3
- Fix bug in ndpi_lru_find_cache
- Fix invalid xbox and playstation port guesses
- Fix CAPWAP tunnel decoding
- Fix parsing of DLT_PPP datalink type
- Fix dissection of QUIC initial packets coalesced with 0-RTT one
- Fix parsing of GTP headers
- Add bitmap boundary checks
## Misc
- Update download category name
- Update category labels
- Renamed Skype in Skype_Teams (the protocol is now shared across these apps)
- Add IEC analysis wireshark plugin
- Flow risk visualization in Wireshark
- ndpiReader
- add statistics about nDPI performance
- fix memory leak
- fix collecting of risks statistics
- Move installed libraries from /usr/local to /usr
- Improve NDPI_API_VERSION generation
- Update ndpi_ptree_match_addr prototype
#### nDPI 3.4 (October 2020)
## New Features
+2 -2
View File
@@ -8,7 +8,7 @@ endif
pkgconfigdir = $(libdir)/pkgconfig
pkgconfig_DATA = libndpi.pc
EXTRA_DIST = README.md CHANGELOG.md CONTRIBUTING.md README.protocols autogen.sh configure.seed wireshark python windows utils packages doc/nDPI_QuickStartGuide.pages doc/nDPI_QuickStartGuide.pdf example/MacOS example/Win32
EXTRA_DIST = README.md CHANGELOG.md CONTRIBUTING.md autogen.sh wireshark python windows utils packages example/MacOS doc/api.rst doc/conf.py doc/flow_risks.rst doc/guide/nDPI_QuickStartGuide.pages doc/guide/nDPI_QuickStartGuide.pdf doc/img/logo.png doc/index.rst doc/Makefile doc/what_is_ndpi.rst
changelog:
git log --since={`curl -s https://github.com/ntop/ndpi/releases | grep datetime | head -n1 | egrep -o "[0-9]+\-[0-9]+\-[0-9]+"`} --name-only --pretty=format:" - %s" | grep "^ " > Changelog.latest
git log --since={`curl -s https://github.com/ntop/ndpi/releases | grep datetime | head -n1 | egrep -o "[0-9]+\-[0-9]+\-[0-9]+"`} --name-only --pretty=format:" - %s" | grep "^ " > CHANGELOG.latest
+41
View File
@@ -0,0 +1,41 @@
## Compiling nDPI with Fuzzer
- Install the latest clang (sudo apt-get install clang-10)
- Export environamental variables to prefer clang over g++ (of installed)
- export CC=/usr/bin/clang-10
- export CPP=/usr/bin/clang-cpp-10
- export CXX=/usr/bin/clang++-10
- export LD=/usr/bin/ld.lld-10
- Run autogen.sh
- ./autogen.sh --with-sanitizer --enable-fuzztargets
## Testing nDPI with ClusterFuzz Artifacts
- Download the artifact (example clusterfuzz-testcase-fuzz_process_packet-4992218834796544)
- Run nDPI against the artifact
- Example: ./fuzz/fuzz_process_packet_with_main clusterfuzz-testcase-fuzz_process_packet-4992218834796544
The output is the error report
```
./fuzz/fuzz_process_packet_with_main /tmp/clusterfuzz-testcase-fuzz_process_packet-4992218834796544
AddressSanitizer:DEADLYSIGNAL
=================================================================
==11590==ERROR: AddressSanitizer: SEGV on unknown address 0x61a100000087 (pc 0x00000056e6a4 bp 0x7ffd624fa170 sp 0x7ffd624fa090 T0)
==11590==The signal is caused by a READ memory access.
#0 0x56e6a4 in quic_len /home/deri/nDPI/src/lib/protocols/quic.c:203:12
#1 0x575d6b in decrypt_initial_packet /home/deri/nDPI/src/lib/protocols/quic.c:993:16
#2 0x571776 in get_clear_payload /home/deri/nDPI/src/lib/protocols/quic.c:1302:21
#3 0x56f149 in ndpi_search_quic /home/deri/nDPI/src/lib/protocols/quic.c:1658:19
#4 0x503935 in check_ndpi_detection_func /home/deri/nDPI/src/lib/ndpi_main.c:4683:6
#5 0x5056fb in check_ndpi_udp_flow_func /home/deri/nDPI/src/lib/ndpi_main.c:4742:10
#6 0x505152 in ndpi_check_flow_func /home/deri/nDPI/src/lib/ndpi_main.c:4775:12
#7 0x5174cf in ndpi_detection_process_packet /home/deri/nDPI/src/lib/ndpi_main.c:5545:15
#8 0x4c709b in LLVMFuzzerTestOneInput /home/deri/nDPI/fuzz/fuzz_process_packet.c:30:3
#9 0x4c7640 in main /home/deri/nDPI/fuzz/fuzz_process_packet.c:90:17
#10 0x7f888e5dabf6 in __libc_start_main /build/glibc-S9d2JN/glibc-2.27/csu/../csu/libc-start.c:310
#11 0x41c399 in _start (/home/deri/nDPI/fuzz/fuzz_process_packet_with_main+0x41c399)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/deri/nDPI/src/lib/protocols/quic.c:203:12 in quic_len
==11590==ABORTING
```
+13 -1
View File
@@ -1,7 +1,7 @@
![ntop][ntopng_logo] ![ntop][ntop_logo]
# nDPI
[![Build Status](https://travis-ci.org/ntop/nDPI.png?branch=dev)](https://travis-ci.org/ntop/nDPI)
[![Build Status](https://img.shields.io/github/workflow/status/ntop/nDPI/Build/dev?logo=github)](https://github.com/ntop/nDPI/actions?query=workflow%3ABuild)
[![Code Quality: Cpp](https://img.shields.io/lgtm/grade/cpp/g/ntop/nDPI.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/ntop/nDPI/context:cpp)
[![Total Alerts](https://img.shields.io/lgtm/alerts/g/ntop/nDPI.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/ntop/nDPI/alerts)
[![Fuzzing Status](https://oss-fuzz-build-logs.storage.googleapis.com/badges/ndpi.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened&can=1&q=proj:ndpi)
@@ -26,6 +26,14 @@ Please note that the pre-requisites for compilation include:
- GNU tools (autogen, automake, autoconf, libtool)
- GNU C compiler (gcc)
On Debian/Ubuntu systems do:
- sudo apt-get install build-essential git bison flex libpcap-dev libtool libtool-bin autoconf pkg-config automake autogen libjson-c-dev libnuma-dev libgcrypt20-dev libpcre2-dev
FreeBSD
- sudo pkg install gcc git autoconf automake libtool devel/pkgconf json-c gmake
Remember to use gmake and not make on FreeBSD
### How To Add A New Protocol Dissector
The entire procedure of adding new protocols in detail:
@@ -44,6 +52,10 @@ The entire procedure of adding new protocols in detail:
### How to use nDPI to Block Selected Traffic
You can use nDPI to selectively block selected Internet traffic by embedding it onto an application (remember that nDPI is just a library). Both [ntopng](https://github.com/ntop/ntopng) and [nProbe cento](http://www.ntop.org/products/netflow/nprobe-cento/) can do this.
### nDPI Paper Citation
- Deri, Luca, et al. [nDPI: Open-source high-speed deep packet inspection](http://luca.ntop.org/nDPI.pdf) 2014 International Wireless Communications and Mobile Computing Conference (IWCMC). IEEE, 2014.
### nDPI-Related Projects
- [nfstream](https://github.com/aouinizied/nfstream)
-18
View File
@@ -1,18 +0,0 @@
Tor
---
Tor protocol can use SSL to hide itself. These are examples:
TCP 37.128.208.46:9001 <-> 172.16.253.130:2078 [VLAN: 0][proto: 91/SSL][132 pkts/93834 bytes][SSL client: www.jwrpsthzrih.com]
TCP 172.16.253.130:2021 <-> 75.147.140.249:443 [VLAN: 0][proto: 91/SSL][28 pkts/8053 bytes][SSL client: www.5akw23dx.com]
TCP 172.16.253.130:2077 <-> 77.247.181.163:443 [VLAN: 0][proto: 91/SSL][136 pkts/94329 bytes][SSL client: www.fk4pprq42hsvl2wey.com]
It can be detected by analyzing the SSL client certificate and checking the name that does not match to a real host in
addition of being a bit weird. As doing DNS resolution is not a task for nDPI we let applications do and then recognize
SSL-tunnelled connections.
See http://www.netresec.com/?page=Blog&month=2013-04&post=Detecting-TOR-Communication-in-Network-Traffic
For this reason nDPI uses a heuristic, non-DNS based, approach to detect tor communications. If possible, apps
should validate the certificate using the DNS. This is not something nDPI can afford to do for performance
reasons
+1 -21
View File
@@ -1,10 +1,5 @@
#!/bin/sh
NDPI_MAJOR="3"
NDPI_MINOR="4"
NDPI_PATCH="0"
NDPI_VERSION_SHORT="$NDPI_MAJOR.$NDPI_MINOR.$NDPI_PATCH"
rm -f configure config.h config.h.in
AUTOCONF=$(command -v autoconf)
@@ -13,16 +8,10 @@ LIBTOOL=$(command -v libtool)
LIBTOOLIZE=$(command -v libtoolize)
AUTORECONF=$(command -v autoreconf)
PKG_CONFIG=$(command -v pkg-config)
FUZZY=
if test -z $AUTOCONF; then
echo "autoconf is missing: please install it and try again"
exit
else
V=`autoconf --version | head -1 | cut -d ' ' -f 4`
if [ "$V" = '2.63' ]; then
FUZZY="dnl> "
fi
fi
if test -z $AUTOMAKE; then
@@ -45,18 +34,9 @@ if test -z $PKG_CONFIG; then
exit
fi
cat configure.seed | sed \
-e "s/@NDPI_MAJOR@/$NDPI_MAJOR/g" \
-e "s/@NDPI_MINOR@/$NDPI_MINOR/g" \
-e "s/@NDPI_PATCH@/$NDPI_PATCH/g" \
-e "s/@NDPI_VERSION_SHORT@/$NDPI_VERSION_SHORT/g" \
-e "s/@FUZZY@/$FUZZY/g" \
> configure.ac
autoreconf -ivf
cat configure | sed "s/#define PACKAGE/#define NDPI_PACKAGE/g" | sed "s/#define VERSION/#define NDPI_VERSION/g" > configure.tmp
cat configure.tmp > configure
echo "./configure $@"
chmod +x configure
./configure $@
+309
View File
@@ -0,0 +1,309 @@
AC_INIT([libndpi], [4.2.0])
AC_CONFIG_AUX_DIR([.])
AC_CONFIG_MACRO_DIR([m4])
AM_INIT_AUTOMAKE([foreign subdir-objects])
AC_PREFIX_DEFAULT(/usr)
EXTRA_TARGETS="example tests tests/unit tests/dga"
AC_ARG_WITH(only-libndpi, AS_HELP_STRING([--with-only-libndpi], [Build only libndpi (no examples, tests etc)]))
AS_IF([test "${with_only_libndpi+set}" = set],[
EXTRA_TARGETS=""
])
AC_ARG_WITH(sanitizer, AS_HELP_STRING([--with-sanitizer], [Build with support for address, undefined and leak sanitizer]))
AC_ARG_ENABLE(fuzztargets, AS_HELP_STRING([--enable-fuzztargets], [Enable fuzz targets]),[enable_fuzztargets=$enableval],[enable_fuzztargets=no])
AS_IF([test "x$enable_fuzztargets" = "xyes"], [BUILD_FUZZTARGETS=1], [BUILD_FUZZTARGETS=0])
AM_CONDITIONAL([BUILD_FUZZTARGETS], [test "x$enable_fuzztargets" = "xyes"])
AS_IF([test "${with_sanitizer+set}" = set],[
CFLAGS="${CFLAGS} -g3 -O0 -fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fsanitize=leak -fno-omit-frame-pointer"
LDFLAGS="${LDFLAGS} -fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fsanitize=leak"
])
LT_INIT
LT_LIB_M
PKG_PROG_PKG_CONFIG
SYSTEM=`uname -s`
if test $SYSTEM = "Darwin"; then
dnl> AC_PROG_CC(clang gcc)
AM_PROG_CC_C_O(clang gcc)
AC_PROG_CXX(clang++ g++)
AC_PROG_CC_STDC(clang gcc)
else
AC_PROG_CC_C_O
AC_PROG_CXX
AC_PROG_CC_STDC
fi
dnl> Can't iuse AM_PROG_AR because not all of our Makefiles are automake (yet?)
AC_CHECK_TOOL(AR, ar, [false])
AC_LANG_WERROR
NDPI_MAJOR=`echo "${PACKAGE_VERSION}" | cut -d . -f 1`
NDPI_MINOR=`echo "${PACKAGE_VERSION}" | cut -d . -f 2`
NDPI_PATCH=`echo "${PACKAGE_VERSION}" | cut -d . -f 3`
NDPI_VERSION_SHORT="$NDPI_MAJOR.$NDPI_MINOR.$NDPI_PATCH"
AC_DEFINE_UNQUOTED(NDPI_MAJOR_RELEASE, "${NDPI_MAJOR}", [nDPI major release])
AC_DEFINE_UNQUOTED(NDPI_MINOR_RELEASE, "${NDPI_MINOR}", [nDPI minor release])
AC_DEFINE_UNQUOTED(NDPI_PATCH_LEVEL, "${NDPI_PATCH}", [nDPI patch level])
# .git as directory in a cloned repo
# .git as file in submodule based integration
if test -d ".git" || test -f ".git" ; then :
GIT_TAG=`git log -1 --format=%h`
GIT_DATE=`git log -1 --format=%cd`
#
# On CentOS 6 `git rev-list HEAD --count` does not work
#
#
GIT_NUM=`git log --pretty=oneline | wc -l | tr -d '[[:space:]]'`
GIT_RELEASE="${PACKAGE_VERSION}-${GIT_NUM}-${GIT_TAG}"
A=`git log src/include/ndpi_typedefs.h|wc -l`
B=`git log src/include/ndpi_protocol_ids.h|wc -l`
C=`git log src/include/ndpi_api.h.in|wc -l`
NDPI_API_VERSION=$((A+B+C))
else
GIT_RELEASE="${PACKAGE_VERSION}"
GIT_DATE=`date -u -r CHANGELOG.md`
if test -z "$SOURCE_DATE_EPOCH" ; then :
SOURCE_DATE_EPOCH=`date +%s`
fi
NDPI_API_VERSION=`echo $SOURCE_DATE_EPOCH | cut -c7-10`
fi
NDPI_API_VERSION=`echo $NDPI_API_VERSION | sed 's/^0*//'`
echo "Setting API version to $NDPI_API_VERSION"
AC_DEFINE_UNQUOTED(NDPI_GIT_RELEASE, "${GIT_RELEASE}", [GIT Release])
AC_DEFINE_UNQUOTED(NDPI_GIT_DATE, "${GIT_DATE}", [Last GIT change])
if ! test "${with_only_libndpi+set}" = set; then :
dnl> used by json-c for unit tests
PKG_CHECK_MODULES([JSONC], [json-c], [JSONC_LIBS="${pkg_cv_JSONC_LIBS}" JSONC_CFLAGS="${pkg_cv_JSONC_CFLAGS}"])
AC_CHECK_LIB([json-c], [json_object_put], [], [
AC_MSG_WARN([JSON-C not available. Disabled unit test.])
JSONC_LIBS=""
JSONC_CFLAGS=""
EXTRA_TARGETS="example tests tests/dga"
])
AX_PTHREAD([],[
AC_MSG_WARN([POSIX Threads not available. Building library only.])
JSONC_LIBS=""
JSONC_CFLAGS=""
EXTRA_TARGETS=""
])
fi
LIBS="$LIBS $JSONC_LIBS"
CFLAGS="-W -Wall -Wno-unused-parameter -Wno-unused-function $CFLAGS $JSONC_CFLAGS"
AC_CHECK_HEADERS([netinet/in.h stdint.h stdlib.h string.h unistd.h math.h float.h])
AC_CHECK_LIB([m], [sqrt], [], [LIBM="-lm"])
AC_CHECK_LIB([rrd], [rrd_fetch_r], [LIBRRD=-lrrd])
if test "x$ac_cv_lib_rrd_rrd_fetch_r" = xyes; then :
AC_DEFINE_UNQUOTED(HAVE_RRDTOOL, "1", [rrdtool is present])
EXTRA_TARGETS="$EXTRA_TARGETS rrdtool"
fi
ADDITIONAL_LIBS="$LIBRRD $LIBM"
PCAP_HOME=$HOME/PF_RING/userland
DPDK_TARGET=
AC_MSG_CHECKING([DPDK (used by ndpiReader)])
if test -d $HOME/DPDK; then :
AC_MSG_RESULT(yes)
DPDK_TARGET=dpdk
else
AC_MSG_RESULT([no (missing $HOME/DPDK)])
fi
if ! test -d $PCAP_HOME; then :
PCAP_HOME=`pwd`/../../PF_RING/userland
fi
SHORT_MACHINE=`uname -m | cut -b1-3`
if test $SHORT_MACHINE = "arm"; then
LIBNUMA=""
else
AC_CHECK_LIB([numa], [numa_available], [LIBNUMA="-lnuma"])
fi
MACHINE=`uname -m`
CUSTOM_NDPI=
if test -d ../nDPI-custom; then :
CUSTOM_NDPI="-DCUSTOM_NDPI_PROTOCOLS"
AC_MSG_RESULT([Compiling with custom nDPI protocols])
fi
if test -d ../ndpi-pro; then :
with_maxminddb=1
fi
case "$host" in
*-*-mingw32*|*-*-msys)
PCAP_INC=""
PCAP_LIB=""
case "$host" in
x86_64-*)
BUILD_MINGW_X64=1
;;
i686-*)
;;
*)
AC_MSG_ERROR([Unsupported MingW cross compiler.])
;;
esac
CFLAGS="${CFLAGS}"
LIBS="${LIBS} -lws2_32"
BUILD_MINGW=1
;;
*)
if test -f $PCAP_HOME/libpcap/libpcap.a; then :
echo "Using libpcap from $PCAP_HOME"
PCAP_INC="-I $PCAP_HOME/libpcap"
PFRING_LIB=
if test -f $PCAP_HOME/lib/libpfring.a; then :
PFRING_LIB=$PCAP_HOME/lib/libpfring.a
fi
PCAP_LIB="$PCAP_HOME/libpcap/libpcap.a $PFRING_LIB $LIBNUMA `$PCAP_HOME/lib/pfring_config --libs`"
AC_CHECK_LIB([rt], [clock_gettime], [PCAP_LIB="$PCAP_LIB -lrt"])
AC_CHECK_LIB([nl], [nl_handle_alloc], [PCAP_LIB="$PCAP_LIB -lnl"])
# The dlopen() function is in libdl on GLIBC-based systems
# and in the C library for *BSD systems
AC_CHECK_LIB([dl], [dlopen], [DL_LIB="-ldl"],[AC_CHECK_LIB([c],
[dlopen], [DL_LIB="-lc"],
[AC_MSG_ERROR([unable to find the dlopen(), dlsym() functions]) ]) ])
elif test "${with_only_libndpi+set}" != set; then :
AC_CHECK_LIB([pcap], [pcap_open_live], [PCAP_LIB="-lpcap"])
if test $ac_cv_lib_pcap_pcap_open_live = "no"; then :
AC_MSG_ERROR([Missing libpcap(-dev) library required to compile the example application.])
fi
fi
;;
esac
AC_ARG_ENABLE([debug-messages],
AS_HELP_STRING([--enable-debug-messages], [Define NDPI_ENABLE_DEBUG_MESSAGES=1]), [
AC_DEFINE(NDPI_ENABLE_DEBUG_MESSAGES, 1, [Enable ndpi_debug_messages]) ])
AS_IF([test "x$enable_fuzztargets" = "xyes"], [
AC_PROG_CXX
AC_LANG_PUSH(C++)
tmp_saved_flags=$[]_AC_LANG_PREFIX[]FLAGS
AX_CHECK_COMPILE_FLAG([-fsanitize=fuzzer],,
[AC_MSG_ERROR([--enable-fuzztargets requires -fsanitize=fuzzer which is only supported by LLVM])],
[-Werror])
AS_IF([test "x$LIB_FUZZING_ENGINE" = "x"], [
LIB_FUZZING_ENGINE=-fsanitize=fuzzer
AC_SUBST(LIB_FUZZING_ENGINE)
])
_AC_LANG_PREFIX[]FLAGS="$[]_AC_LANG_PREFIX[]FLAGS $LIB_FUZZING_ENGINE"
AC_MSG_CHECKING([whether $CXX accepts $LIB_FUZZING_ENGINE])
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#include <sys/types.h>
extern "C" int LLVMFuzzerTestOneInput(const unsigned char *Data, size_t Size);
extern "C" int LLVMFuzzerTestOneInput(const unsigned char *Data, size_t Size) {
(void)Data;
(void)Size;
return 0;
}
]])],
[ AC_MSG_RESULT(yes)
has_sanitizefuzzer=yes],
[ AC_MSG_RESULT(no) ]
)
_AC_LANG_PREFIX[]FLAGS=$tmp_saved_flags
AC_LANG_POP()
])
AM_CONDITIONAL([HAS_FUZZLDFLAGS], [test "x$has_sanitizefuzzer" = "xyes"])
AC_CHECK_LIB(pthread, pthread_setaffinity_np, AC_DEFINE_UNQUOTED(HAVE_PTHREAD_SETAFFINITY_NP, 1, [libc has pthread_setaffinity_np]))
dnl> GCRYPT
GCRYPT_ENABLED=1
AC_ARG_ENABLE([gcrypt],
[AS_HELP_STRING([--disable-gcrypt], [Avoid compiling with libgcrypt/libgpg-error, even if they are present. QUIC sub-classification may be missing])],
[GCRYPT_ENABLED=0],
[AC_CHECK_LIB(gcrypt, gcry_cipher_checktag)])
AS_IF([test ${GCRYPT_ENABLED} -eq 1], [
if test "x$ac_cv_lib_gcrypt_gcry_cipher_checktag" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lgcrypt"
else
$as_unset ac_cv_lib_gcrypt_gcry_cipher_checktag
AC_CHECK_LIB(gpg-error, gpg_strerror_r)
AC_CHECK_LIB(gcrypt, gcry_cipher_checktag)
if test "x$ac_cv_lib_gcrypt_gcry_cipher_checktag" = xyes -a "x$ac_cv_lib_gpg_error_gpg_strerror_r" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lgcrypt -lgpg-error"
else
GCRYPT_ENABLED=0
fi
fi
])
dnl> PCRE
PCRE_ENABLED=0
AC_ARG_WITH(pcre, [ --with-pcre Enable nDPI build with libpcre])
if test "${with_pcre+set}" = set; then :
AC_CHECK_LIB(pcre, pcre_compile, AC_DEFINE_UNQUOTED(HAVE_PCRE, 1, [libpcre(-dev) is present]))
if test "x$ac_cv_lib_pcre_pcre_compile" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lpcre"
PCRE_ENABLED=1
fi
fi
dnl> GeoIP
AC_ARG_WITH(maxminddb, [ --with-maxminddb Enable nDPI build with libmaxminddb])
if test "${with_maxminddb+set}" = set; then :
AC_CHECK_LIB([maxminddb], [MMDB_lookup_sockaddr])
AC_HAVE_HEADERS(maxminddb.h)
if test ".${ac_cv_lib_maxminddb_MMDB_lookup_sockaddr}" = ".yes" && test ".${ac_cv_header_maxminddb_h}" = ".yes"; then
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lmaxminddb"
AC_DEFINE_UNQUOTED(HAVE_MAXMINDDB, 1, [MaxMind DB support])
fi
fi
dnl> Curl
dnl> AC_CHECK_LIB([curl], [curl_easy_perform])
dnl> if test ${ac_cv_lib_curl_curl_easy_perform} = "no"; then
dnl> echo "Please install libcurl4-openssl-dev (http://curl.haxx.se/) in order to enable curl support"
dnl> else
dnl> ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lcurl"
dnl> AC_DEFINE_UNQUOTED(HAVE_CURL, 1, [curl is present])
dnl> fi
AC_CONFIG_FILES([Makefile example/Makefile example/Makefile.dpdk tests/Makefile tests/unit/Makefile tests/performance/Makefile tests/dga/Makefile rrdtool/Makefile libndpi.pc src/include/ndpi_define.h src/lib/Makefile python/Makefile fuzz/Makefile src/include/ndpi_api.h])
AC_CONFIG_FILES([tests/do.sh], [chmod +x tests/do.sh])
AC_CONFIG_HEADERS(src/include/ndpi_config.h)
AC_SUBST(GIT_RELEASE)
AC_SUBST(NDPI_MAJOR)
AC_SUBST(NDPI_MINOR)
AC_SUBST(NDPI_PATCH)
AC_SUBST(NDPI_VERSION_SHORT)
AC_SUBST(PCAP_INC)
AC_SUBST(PCAP_LIB)
AC_SUBST(ADDITIONAL_LIBS)
AC_SUBST(DL_LIB)
AC_SUBST(DPDK_TARGET)
AC_SUBST(HAVE_PTHREAD_SETAFFINITY_NP)
AC_SUBST(CUSTOM_NDPI)
AC_SUBST(NDPI_API_VERSION)
AC_SUBST(EXTRA_TARGETS)
AC_SUBST(BUILD_MINGW)
AC_SUBST(BUILD_MINGW_X64)
AC_SUBST(BUILD_FUZZTARGETS)
AC_SUBST(JSONC_CFLAGS)
AC_SUBST(JSONC_LIBS)
AC_SUBST(GCRYPT_ENABLED)
AC_SUBST(PCRE_ENABLED)
AC_OUTPUT
-213
View File
@@ -1,213 +0,0 @@
AC_INIT([libndpi], [@NDPI_VERSION_SHORT@])
AC_CONFIG_AUX_DIR([.])
AC_CONFIG_MACRO_DIR([m4])
AM_INIT_AUTOMAKE([foreign subdir-objects])
EXTRA_TARGETS="example tests tests/unit"
AC_ARG_WITH(only-libndpi, AS_HELP_STRING([--with-only-libndpi], [Build only libndpi (no examples, tests etc)]))
AS_IF([test "${with_only_libndpi+set}" = set],[
EXTRA_TARGETS=""
])
AC_ARG_WITH(sanitizer, AS_HELP_STRING([--with-sanitizer], [Build with support for address, undefined and leak sanitizer]))
AC_ARG_ENABLE(fuzztargets, AS_HELP_STRING([--enable-fuzztargets], [Enable fuzz targets]),[enable_fuzztargets=$enableval],[enable_fuzztargets=no])
AM_CONDITIONAL([BUILD_FUZZTARGETS], [test "x$enable_fuzztargets" = "xyes"])
AS_IF([test "${with_sanitizer+set}" = set],[
CFLAGS="${CFLAGS} -g3 -O0 -Wno-unused-value -fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fsanitize=leak -fno-omit-frame-pointer"
LDFLAGS="${LDFLAGS} -fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fsanitize=leak"
])
LT_INIT
SYSTEM=`uname -s`
if test $SYSTEM = "Darwin"; then
dnl> AC_PROG_CC(clang gcc)
AM_PROG_CC_C_O(clang gcc)
AC_PROG_CXX(clang++ g++)
AC_PROG_CC_STDC(clang gcc)
else
dnl> AC_PROG_CC
AM_PROG_CC_C_O
AC_PROG_CXX
AC_PROG_CC_STDC
fi
AC_LANG_WERROR
AX_PTHREAD
NDPI_VERSION_SHORT="@NDPI_VERSION_SHORT@"
NDPI_MAJOR="@NDPI_MAJOR@"
NDPI_MINOR="@NDPI_MINOR@"
NDPI_PATCH="@NDPI_PATCH@"
AC_DEFINE_UNQUOTED(NDPI_MAJOR_RELEASE, "${NDPI_MAJOR}", [nDPI major release])
AC_DEFINE_UNQUOTED(NDPI_MINOR_RELEASE, "${NDPI_MINOR}", [nDPI minor release])
AC_DEFINE_UNQUOTED(NDPI_PATCH_LEVEL, "${NDPI_PATCH}", [nDPI patch level])
if test -d ".git"; then :
GIT_TAG=`git log -1 --format=%h`
GIT_DATE=`git log -1 --format=%cd`
#
# On CentOS 6 `git rev-list HEAD --count` does not work
#
#
GIT_NUM=`git log --pretty=oneline | wc -l | tr -d '[[:space:]]'`
GIT_RELEASE="${PACKAGE_VERSION}-${GIT_NUM}-${GIT_TAG}"
A=`git log src/include/ndpi_typedefs.h|wc -l`
B=`git log src/include/ndpi_protocol_ids.h|wc -l`
C=`git log src/include/ndpi_api.h.in|wc -l`
NDPI_API_VERSION=$((A+B+C))
else
GIT_RELEASE="${PACKAGE_VERSION}"
GIT_DATE=`date -u -r CHANGELOG.md`
NDPI_API_VERSION=`date +%s | cut -c7-10`
fi
NDPI_API_VERSION=`echo $NDPI_API_VERSION | sed 's/^0*//'`
AC_DEFINE_UNQUOTED(NDPI_GIT_RELEASE, "${GIT_RELEASE}", [GIT Release])
AC_DEFINE_UNQUOTED(NDPI_GIT_DATE, "${GIT_DATE}", [Last GIT change])
dnl> used by json-c
JSONC_INC=`pkg-config --cflags json-c`
CFLAGS="${CFLAGS} ${JSONC_INC}"
AC_CHECK_HEADERS([netinet/in.h stdint.h stdlib.h string.h unistd.h json.h])
ADDITIONAL_LIBS=
PCAP_HOME=$HOME/PF_RING/userland
DPDK_TARGET=
AC_MSG_CHECKING([DPDK (used by ndpiReader)])
if test -d $HOME/DPDK; then :
AC_MSG_RESULT(yes)
DPDK_TARGET=dpdk
else
AC_MSG_RESULT([no (missing $HOME/DPDK)])
fi
if ! test -d $PCAP_HOME; then :
PCAP_HOME=`pwd`/../../PF_RING/userland
fi
SHORT_MACHINE=`uname -m | cut -b1-3`
if test $SHORT_MACHINE = "arm"; then
LIBNUMA=""
else
AC_CHECK_LIB([numa], [numa_available], [LIBNUMA="-lnuma"])
fi
MACHINE=`uname -m`
CUSTOM_NDPI=
if test -d ../nDPI-custom; then :
CUSTOM_NDPI="-DCUSTOM_NDPI_PROTOCOLS"
AC_MSG_RESULT([Compiling with custom nDPI protocols])
fi
case "$host" in
*-*-mingw32*|*-*-msys)
CFLAGS="${CFLAGS} -DOS_WIN32"
LDFLAGS="${LDFLAGS} -lws2_32 -lucrtbase"
BUILD_MINGW=1
;;
*)
if test -f $PCAP_HOME/libpcap/libpcap.a; then :
echo "Using libpcap from $PCAP_HOME"
PCAP_INC="-I $PCAP_HOME/libpcap"
PCAP_LIB="$PCAP_HOME/libpcap/libpcap.a $PCAP_HOME/lib/libpfring.a $LIBNUMA `$PCAP_HOME/lib/pfring_config --libs`"
AC_CHECK_LIB([rt], [clock_gettime], [PCAP_LIB="$PCAP_LIB -lrt"])
AC_CHECK_LIB([nl], [nl_handle_alloc], [PCAP_LIB="$PCAP_LIB -lnl"])
# The dlopen() function is in libdl on GLIBC-based systems
# and in the C library for *BSD systems
AC_CHECK_LIB([dl], [dlopen, dlsym], [DL_LIB="-ldl"],[AC_CHECK_LIB([c], [dlopen, dlsym], [DL_LIB="-lc"],[AC_MSG_ERROR([unable to find the dlopen(), dlsym() functions]) ]) ])
else
AC_CHECK_LIB([pcap], [pcap_open_live], [PCAP_LIB="-lpcap"])
if test $ac_cv_lib_pcap_pcap_open_live = "no"; then :
echo ""
echo "ERROR: Missing libpcap(-dev) library required to compile the example application"
echo "ERROR: Please install it and try again"
exit
fi
fi
;;
esac
AC_ARG_ENABLE([debug-messages],
AS_HELP_STRING([--enable-debug-messages], [Define NDPI_ENABLE_DEBUG_MESSAGES=1]), [
AC_DEFINE(NDPI_ENABLE_DEBUG_MESSAGES, 1, [Enable ndpi_debug_messages]) ])
@FUZZY@ AS_IF([test "x$enable_fuzztargets" = "xyes"], [
@FUZZY@ AC_PROG_CXX
@FUZZY@ AC_LANG_PUSH(C++)
@FUZZY@ tmp_saved_flags=$[]_AC_LANG_PREFIX[]FLAGS
@FUZZY@ AX_CHECK_COMPILE_FLAG([-fsanitize=fuzzer],,
@FUZZY@ [AC_MSG_ERROR([--enable-fuzztargets requires -fsanitize=fuzzer which is only supported by LLVM])],
@FUZZY@ [-Werror])
@FUZZY@ AS_IF([test "x$LIB_FUZZING_ENGINE" = "x"], [
@FUZZY@ LIB_FUZZING_ENGINE=-fsanitize=fuzzer
@FUZZY@ AC_SUBST(LIB_FUZZING_ENGINE)
@FUZZY@ ])
@FUZZY@ _AC_LANG_PREFIX[]FLAGS="$[]_AC_LANG_PREFIX[]FLAGS $LIB_FUZZING_ENGINE"
@FUZZY@ AC_MSG_CHECKING([whether $CXX accepts $LIB_FUZZING_ENGINE])
@FUZZY@ AC_LINK_IFELSE([AC_LANG_SOURCE([[
@FUZZY@ #include <sys/types.h>
@FUZZY@ extern "C" int LLVMFuzzerTestOneInput(const unsigned char *Data, size_t Size);
@FUZZY@ extern "C" int LLVMFuzzerTestOneInput(const unsigned char *Data, size_t Size) {
@FUZZY@ (void)Data;
@FUZZY@ (void)Size;
@FUZZY@ return 0;
@FUZZY@ }
@FUZZY@ ]])],
@FUZZY@ [ AC_MSG_RESULT(yes)
@FUZZY@ has_sanitizefuzzer=yes],
@FUZZY@ [ AC_MSG_RESULT(no) ]
@FUZZY@ )
@FUZZY@ _AC_LANG_PREFIX[]FLAGS=$tmp_saved_flags
@FUZZY@ AC_LANG_POP()
@FUZZY@ ])
AM_CONDITIONAL([HAS_FUZZLDFLAGS], [test "x$has_sanitizefuzzer" = "xyes"])
AC_CHECK_LIB(pthread, pthread_setaffinity_np, AC_DEFINE_UNQUOTED(HAVE_PTHREAD_SETAFFINITY_NP, 1, [libc has pthread_setaffinity_np]))
dnl> GCRYPT
AC_ARG_ENABLE([gcrypt],
[AS_HELP_STRING([--disable-gcrypt], [Avoid compiling with libgcrypt/libgpg-error, even if they are present. QUIC sub-classification may be missing])],
[:],
[AC_CHECK_LIB(gcrypt, gcry_cipher_checktag)
AC_CHECK_LIB(gpg-error, gpg_strerror_r)])
if test "x$ac_cv_lib_gcrypt_gcry_cipher_checktag" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lgcrypt"
fi
dnl> PCRE
AC_ARG_WITH(pcre, [ --with-pcre Enable nDPI build with libpcre])
if test "${with_pcre+set}" = set; then :
AC_CHECK_LIB(pcre, pcre_compile, AC_DEFINE_UNQUOTED(HAVE_PCRE, 1, [libpcre(-dev) is present]))
if test "x$ac_cv_lib_pcre_pcre_compile" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lpcre"
fi
fi
AC_CONFIG_FILES([Makefile example/Makefile example/Makefile.dpdk tests/Makefile tests/unit/Makefile libndpi.pc src/include/ndpi_define.h src/lib/Makefile python/Makefile fuzz/Makefile src/include/ndpi_api.h])
AC_CONFIG_HEADERS(src/include/ndpi_config.h)
AC_SUBST(GIT_RELEASE)
AC_SUBST(NDPI_MAJOR)
AC_SUBST(NDPI_MINOR)
AC_SUBST(NDPI_PATCH)
AC_SUBST(NDPI_VERSION_SHORT)
AC_SUBST(PCAP_INC)
AC_SUBST(PCAP_LIB)
AC_SUBST(ADDITIONAL_LIBS)
AC_SUBST(DL_LIB)
AC_SUBST(DPDK_TARGET)
AC_SUBST(HAVE_PTHREAD_SETAFFINITY_NP)
AC_SUBST(CUSTOM_NDPI)
AC_SUBST(NDPI_API_VERSION)
AC_SUBST(EXTRA_TARGETS)
AC_SUBST(BUILD_MINGW)
AC_OUTPUT
+231
View File
@@ -0,0 +1,231 @@
# Makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
# User-friendly check for sphinx-build
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
endif
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " release to make standalone HTML files for ntop documentation release"
@echo " html to make standalone HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " singlehtml to make a single large HTML file"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " qthelp to make HTML files and a qthelp project"
@echo " applehelp to make an Apple Help Book"
@echo " devhelp to make HTML files and a Devhelp project"
@echo " epub to make an epub"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " latexpdf to make LaTeX files and run them through pdflatex"
@echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
@echo " text to make text files"
@echo " man to make manual pages"
@echo " texinfo to make Texinfo files"
@echo " info to make Texinfo files and run them through makeinfo"
@echo " gettext to make PO message catalogs"
@echo " changes to make an overview of all changed/added/deprecated items"
@echo " xml to make Docutils-native XML files"
@echo " pseudoxml to make pseudoxml-XML files for display purposes"
@echo " linkcheck to check all external links for integrity"
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
@echo " coverage to run coverage check of the documentation (if enabled)"
.PHONY: clean
clean:
rm -rf $(BUILDDIR)/*
cd doxygen; make clean
.PHONY: html
html:
cd doxygen; make
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
.PHONY: release
release:
make clean
@{ \
git_root=$$(git rev-parse --show-toplevel);\
version=$$(grep -Po '^NTOPNG_SHORT_VERSION="\K.+(?=")' $${git_root}/configure.ac);\
echo Release version $${version};\
sed -i "s/version = u'.*'/version = u'$${version}'/" conf.py;\
sed -i "s/release = u'.*'/release = u'$${version}'/" conf.py;\
}
make html
.PHONY: dirhtml
dirhtml:
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
.PHONY: singlehtml
singlehtml:
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
@echo
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
.PHONY: pickle
pickle:
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
@echo
@echo "Build finished; now you can process the pickle files."
.PHONY: json
json:
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
@echo
@echo "Build finished; now you can process the JSON files."
.PHONY: htmlhelp
htmlhelp:
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
@echo
@echo "Build finished; now you can run HTML Help Workshop with the" \
".hhp project file in $(BUILDDIR)/htmlhelp."
.PHONY: qthelp
qthelp:
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
@echo
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/ntop.qhcp"
@echo "To view the help file:"
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/ntop.qhc"
.PHONY: applehelp
applehelp:
$(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp
@echo
@echo "Build finished. The help book is in $(BUILDDIR)/applehelp."
@echo "N.B. You won't be able to view it unless you put it in" \
"~/Library/Documentation/Help or install it in your application" \
"bundle."
.PHONY: devhelp
devhelp:
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
@echo
@echo "Build finished."
@echo "To view the help file:"
@echo "# mkdir -p $$HOME/.local/share/devhelp/ntop"
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/ntop"
@echo "# devhelp"
.PHONY: epub
epub:
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
@echo
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
.PHONY: latex
latex:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
@echo "Run \`make' in that directory to run these through (pdf)latex" \
"(use \`make latexpdf' here to do that automatically)."
.PHONY: latexpdf
latexpdf:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through pdflatex..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
.PHONY: latexpdfja
latexpdfja:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through platex and dvipdfmx..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
.PHONY: text
text:
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
@echo
@echo "Build finished. The text files are in $(BUILDDIR)/text."
.PHONY: man
man:
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
@echo
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
.PHONY: texinfo
texinfo:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
@echo "Run \`make' in that directory to run these through makeinfo" \
"(use \`make info' here to do that automatically)."
.PHONY: info
info:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo "Running Texinfo files through makeinfo..."
make -C $(BUILDDIR)/texinfo info
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
.PHONY: gettext
gettext:
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
@echo
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
.PHONY: changes
changes:
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
@echo
@echo "The overview file is in $(BUILDDIR)/changes."
.PHONY: linkcheck
linkcheck:
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or in $(BUILDDIR)/linkcheck/output.txt."
.PHONY: doctest
doctest:
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
@echo "Testing of doctests in the sources finished, look at the " \
"results in $(BUILDDIR)/doctest/output.txt."
.PHONY: coverage
coverage:
$(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage
@echo "Testing of coverage in the sources finished, look at the " \
"results in $(BUILDDIR)/coverage/python.txt."
.PHONY: xml
xml:
$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
@echo
@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
.PHONY: pseudoxml
pseudoxml:
$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
@echo
@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."
+4
View File
@@ -0,0 +1,4 @@
nDPI Programming API
####################
Placeholder
+291
View File
@@ -0,0 +1,291 @@
# -*- coding: utf-8 -*-
import sys
import os
import sphinx_rtd_theme
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#sys.path.insert(0, os.path.abspath('.'))
sys.path.append( "breathe/" )
# -- General configuration ------------------------------------------------
# If documentation needs a minimal Sphinx version, state it here.
#needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.intersphinx',
'breathe'
]
# Workaround for platforms where swaggerdoc is not available
if not os.environ.get("PYTHON_SKIP_SWAGGERDOC"):
extensions.append('sphinxcontrib.swaggerdoc')
breathe_projects = { "apidoc" : "doxygen/xml/" }
breathe_default_project = "apidoc"
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
source_suffix = ['.rst', '.md']
#source_suffix = '.rst'
# The encoding of source files.
#source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'nDPI'
copyright = u'2021, ntop and contributors'
author = u'ntop.org'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = u'4.1'
# The full version, including alpha/beta/rc tags.
release = u'4.1'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ['_build', 'README.md', 'example.rst']
# The reST default role (used for this markup: `text`) to use for all
# documents.
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
#add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
#show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
#modindex_common_prefix = []
# If true, keep warnings as "system message" paragraphs in the built documents.
#keep_warnings = False
# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = False
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#html_theme = 'alabaster'
html_theme = "sphinx_rtd_theme"
#html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#html_theme_options = {}
# Add any paths that contain custom themes here, relative to this directory.
#html_theme_path = []
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
#html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
html_logo = "img/logo.png"
# The name of an image file (relative to this directory) to use as a favicon of
# the docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
#html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
# directly to the root of the documentation.
#html_extra_path = []
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
#html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}
# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# If false, no module index is generated.
#html_domain_indices = True
# If false, no index is generated.
#html_use_index = True
# If true, the index is split into individual pages for each letter.
#html_split_index = False
# If true, links to the reST sources are added to the pages.
#html_show_sourcelink = True
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
#html_show_sphinx = True
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
#html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = None
# Language to be used for generating the HTML full-text search index.
# Sphinx supports the following languages:
# 'da', 'de', 'en', 'es', 'fi', 'fr', 'hu', 'it', 'ja'
# 'nl', 'no', 'pt', 'ro', 'ru', 'sv', 'tr'
#html_search_language = 'en'
# A dictionary with options for the search language support, empty by default.
# Now only 'ja' uses this config value
#html_search_options = {'type': 'default'}
# The name of a javascript file (relative to the configuration directory) that
# implements a search results scorer. If empty, the default will be used.
#html_search_scorer = 'scorer.js'
# Output file base name for HTML help builder.
htmlhelp_basename = 'ntopdoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
# Latex figure (float) alignment
#'figure_align': 'htbp',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'nDPI.tex', u'nDPI Documentation',
u'ntop', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
#latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# If true, show page references after internal links.
#latex_show_pagerefs = False
# If true, show URL addresses after external links.
#latex_show_urls = False
# Documents to append as an appendix to all manuals.
#latex_appendices = []
# If false, no module index is generated.
#latex_domain_indices = True
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'nDPI', u'nDPI Documentation',
[author], 1)
]
# If true, show URL addresses after external links.
#man_show_urls = False
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'nDPI', u'nDPI Documentation',
author, 'ntop and contributors', 'Deep Packet Inspection toolkit',
'Miscellaneous'),
]
# Documents to append as an appendix to all manuals.
#texinfo_appendices = []
# If false, no module index is generated.
#texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
#texinfo_show_urls = 'footnote'
# If true, do not generate a @detailmenu in the "Top" node's menu.
#texinfo_no_detailmenu = False
intersphinx_mapping = {'https://docs.python.org/': None}
def setup(app):
app.add_stylesheet('css/ntop.css')
+255
View File
@@ -0,0 +1,255 @@
nDPI Flow Risks
###############
nDPI is designed not just to detect application protocols in traffic flows but also to evaluate potential security risks associated with the traffic. In nDPI parlance this is called a "flow risk". Flows can have multiple risks detected hence nDPI reports them with a bitmap. Each risk detected corresponds to a bit in the flow risk bitmap. You can read more about `ndpi_risk_enum <https://github.com/ntop/nDPI/blob/dev/src/include/ndpi_typedefs.h>`_ for the list of all numeric risks currently supported.
Below you can find a description of each flow risk so that you can easily understand when a risk is triggered and its meaning. The flow risks are listed in numerical order as they are defined in ndpi_risk_enum.
.. _Risk 001:
NDPI_URL_POSSIBLE_XSS
=====================
HTTP only: this risk indicates a possible `XSS (Cross Side Scripting) <https://en.wikipedia.org/wiki/Cross-site_scripting>`_ attack.
.. _Risk 002:
NDPI_URL_POSSIBLE_SQL_INJECTION
===============================
HTTP only: this risk indicates a possible `SQL Injection attack <https://en.wikipedia.org/wiki/SQL_injection>`_.
.. _Risk 003:
NDPI_URL_POSSIBLE_RCE_INJECTION
===============================
HTTP only: this risk indicates a possible `RCE (Remote Code Execution) attack <https://en.wikipedia.org/wiki/Arbitrary_code_execution>`_.
.. _Risk 004:
NDPI_BINARY_APPLICATION_TRANSFER
================================
HTTP only: this risk indicates that a binary application is downloaded/uploaded. Detected applications include Windows binaries, Linux executables, Unix scripts and Android apps.
.. _Risk 005:
NDPI_KNOWN_PROTOCOL_ON_NON_STANDARD_PORT
========================================
This risk indicates a known protocol used on a non standard port. Example HTTP is supposed to use TCP/80, and in case it is detected on TCP/1234 this risk is detected.
.. _Risk 006:
NDPI_TLS_SELFSIGNED_CERTIFICATE
===============================
TLS/QUIC only: this risk is triggered when a `self-signed certificate <https://en.wikipedia.org/wiki/Self-signed_certificate>`_ is used.
.. _Risk 007:
NDPI_TLS_OBSOLETE_VERSION
=========================
Risk 0triggered when TLS version is older than 1.1.
.. _Risk 008:
NDPI_TLS_WEAK_CIPHER
====================
Risk 0triggered when an unsafe TLS cipher is used. See `this page <https://community.qualys.com/thread/18212-how-does-qualys-determine-the-server-cipher-suites>`_ for a list of insecure ciphers.
.. _Risk 009:
NDPI_TLS_CERTIFICATE_EXPIRED
============================
Risk 0triggered when a TLS certificate is expired, i.e. the current date falls outside of the certificate validity dates.
.. _Risk 010:
NDPI_TLS_CERTIFICATE_MISMATCH
=============================
Risk 0triggered when a TLS certificate does not match the hostname we're accessing. Example you do http://www.aaa.com and the TLS certificate returned is for www.bbb.com.
.. _Risk 011:
NDPI_HTTP_SUSPICIOUS_USER_AGENT
===============================
HTTP only: this risk is triggered whenever the user agent contains suspicious characters or its format is suspicious. Example: <?php something ?> is a typical suspicious user agent.
.. _Risk 012:
NDPI_HTTP_NUMERIC_IP_HOST
=========================
HTTP only: this risk is triggered whenever we're accessing a host using its IP rather than its symbolic name. Example http://1.2.3.4.
.. _Risk 013:
NDPI_HTTP_SUSPICIOUS_URL
========================
HTTP only: this risk is triggered whenever the accessed URL is suspicious. Example: http://127.0.0.1/msadc/..%255c../..%255c../..%255c../winnt/system32/cmd.exe.
.. _Risk 014:
NDPI_HTTP_SUSPICIOUS_HEADER
===========================
HTTP only: this risk is triggered whenever the HTTP peader contains suspicious entries such as Uuid, TLS_version, Osname that are unexpected on the HTTP header.
.. _Risk 015:
NDPI_TLS_NOT_CARRYING_HTTPS
===========================
TLS only: this risk indicates that this TLS flow will not be used to transport HTTP content. Example VPNs use TLS to encrypt data rather to carry HTTP. This is useful to spot this type of cases.
.. _Risk 016:
NDPI_SUSPICIOUS_DGA_DOMAIN
==========================
A `DGA <https://en.wikipedia.org/wiki/Domain_generation_algorithm>`_ is used to generate domain names often used by malwares. This risk indicates that this domain name can (but it's not 100% sure) a DGA as its name is suspicious.
.. _Risk 017:
NDPI_MALFORMED_PACKET
=====================
This risk is generated when a packet (e.g. a DNS packet) has an unexpected formt. This can indicate a protocol error or more often an attempt to jeopardize a valid protocol to carry other type of data.
.. _Risk 018:
NDPI_SSH_OBSOLETE_CLIENT_VERSION_OR_CIPHER
==========================================
This risk is generated whenever a SSH client uses an obsolete SSH protocol version or insecure ciphers.
.. _Risk 019:
NDPI_SSH_OBSOLETE_SERVER_VERSION_OR_CIPHER
==========================================
This risk is generated whenever a SSH server uses an obsolete SSH protocol version or insecure ciphers.
.. _Risk 020:
NDPI_SMB_INSECURE_VERSION
=========================
This risk indicates that the `SMB <https://en.wikipedia.org/wiki/Server_Message_Block>`_ version used is insecure (i.e. v1).
.. _Risk 021:
NDPI_TLS_SUSPICIOUS_ESNI_USAGE
==============================
`SNI <https://en.wikipedia.org/wiki/Server_Name_Indication>`_ is a way to carry in TLS the host/domain name we're accessing. ESNI means encrypted SNI and it is a way to mask SNI (carried in clear text in the TLS header) with encryption. While this practice is legal, it could be used for hiding data or for attacks such as a suspicious `domain fronting <https://github.com/SixGenInc/Noctilucent/blob/master/docs/>`_.
.. _Risk 022:
NDPI_UNSAFE_PROTOCOL
====================
This risk indicates that the protocol used is insecure and that a secure protocol should be used (e.g. Telnet vs SSH).
.. _Risk 023:
NDPI_DNS_SUSPICIOUS_TRAFFIC
===========================
This risk is returned when DNS traffic returns an unexpected/obsolete `record type <https://en.wikipedia.org/wiki/List_of_DNS_record_types>`_.
.. _Risk 024:
NDPI_TLS_MISSING_SNI
====================
TLS needs to carry the the `SNI <https://en.wikipedia.org/wiki/Server_Name_Indication>`_ of the remote server we're accessing. Unfortunately SNI is optional in TLS so it can be omitted. In this case this risk is triggered as this is a non-standard situation that indicates a potential security problem or a protocol using TLS for other purposes (or a protocol bug).
.. _Risk 025:
NDPI_HTTP_SUSPICIOUS_CONTENT
============================
HTTP only: risk reported when HTTP carries content in expected format. Example the HTTP header indicates that the context is text/html but the real content is not readeable (i.e. it can transport binary data). In general this is an attempt to use a valid MIME type to carry data that does not match the type.
.. _Risk 026:
NDPI_RISKY_ASN
==============
This is a placeholder for traffic exchanged with `ASN <https://en.wikipedia.org/wiki/Autonomous_system_(Internet)>`_ that are considered risky. nDPI does not fill this risk that instead should be filled by aplications sitting on top of nDPI (e.g. ntopng).
.. _Risk 027:
NDPI_RISKY_DOMAIN
=================
This is a placeholder for traffic exchanged with domain names that are considered risky. nDPI does not fill this risk that instead should be filled by aplications sitting on top of nDPI (e.g. ntopng).
.. _Risk 028:
NDPI_MALICIOUS_JA3
==================
`JA3 <https://engineering.salesforce.com/tls-fingerprinting-with-ja3-and-ja3s-247362855967>`_ is a method to fingerprint TLS traffic. This risk indicates that the JA3 of the TLS connection is considered suspicious (i.e. it has been found in known malware JA3 blacklists). nDPI does not fill this risk that instead should be filled by aplications sitting on top of nDPI (e.g. ntopng).
.. _Risk 029:
NDPI_MALICIOUS_SHA1_CERTIFICATE
===============================
TLS certificates are uniquely identified with a `SHA1 <https://en.wikipedia.org/wiki/SHA-1>`_ hash value. If such hash is found on a blacklist, this risk can be used. As for other risks, this is a placeholder as nDPI does not fill this risk that instead should be filled by aplications sitting on top of nDPI (e.g. ntopng).
.. _Risk 030:
NDPI_DESKTOP_OR_FILE_SHARING_SESSION
====================================
This risk is set when the flow carries desktop or file sharing sessions (e.g. TeamViewer or AnyDesk just to mention two).
.. _Risk 031:
NDPI_TLS_UNCOMMON_ALPN
======================
This risk is set when the `ALPN <https://en.wikipedia.org/wiki/Application-Layer_Protocol_Negotiation>`_ (it indicates the protocol carried into this TLS flow, for instance HTTP/1.1) is uncommon with respect to the list of expected values.
.. _Risk 032:
NDPI_TLS_CERT_VALIDITY_TOO_LONG
===============================
From 01/09/2020 TLS certificates lifespan is limited to 13 months. This risk is triggered for certificates not respecting this directive.
.. _Risk 033:
NDPI_TLS_SUSPICIOUS_EXTENSION
=============================
This risk is triggered when the domain name (SNI extension) is not printable and thus it is a problem.
.. _Risk 034:
NDPI_TLS_FATAL_ALERT
====================
This risk is triggered when a TLS fatal alert is detected in the TLS flow. See `this page <https://techcommunity.microsoft.com/t5/iis-support-blog/ssl-tls-alert-protocol-and-the-alert-codes/ba-p/377132>`_ for details.
.. _Risk 035:
NDPI_SUSPICIOUS_ENTROPY
=======================
This risk is used to detect suspicious data carried in ICMP packets whose entropy (used to measure how data is distributed, hence to indirectly guess the type of data carried on) is suspicious and thus that it can indicate a data leak.
.. _Risk 036:
NDPI_CLEAR_TEXT_CREDENTIALS
===========================
Clear text protocols are not bad per-se, but they should be avoided when they carry credentials as they can be intercepted by malicious users. This risk is triggered whenever clear text protocols (e.g. FTP, HTTP, IMAP...) contain credentials in clear text (read it as nDPI does not trigger this risk for HTTP connections that do not carry credentials).
.. _Risk 037:
NDPI_DNS_LARGE_PACKET
=====================
`DNS <https://en.wikipedia.org/wiki/Domain_Name_System>`_ packets over UDP should be limited to 512 bytes. DNS packets over this threshold indicate a potential security risk (e.g. use DNS to carry data) or a misconfiguration.
.. _Risk 038:
NDPI_DNS_FRAGMENTED
===================
UDP `DNS <https://en.wikipedia.org/wiki/Domain_Name_System>`_ packets cannot be fragmented. If so, this indicates a potential security risk (e.g. use DNS to carry data) or a misconfiguration.
.. _Risk 039:
NDPI_INVALID_CHARACTERS
=======================
The risk is set whenever a dissected protocol contains characters not allowed in that protocol field.
For example a DNS hostname must only contain a subset of all printable characters or else this risk is set.
Additionally, some TLS protocol fields are checked for printable characters as well.
.. _Risk 040:
NDPI_POSSIBLE_EXPLOIT
=====================
The risk is set whenever a possible exploit (e.g. Log4J/Log4Shell) is detected.
.. _Risk 041:
NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE
===================================
The risk is set whenever a TLS certificate is close to the expiration date.
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 47 KiB

+37
View File
@@ -0,0 +1,37 @@
nDPI Documentation
====================
nDPI is an open source DPI (Deep Packet Inspection) toolkit for traffic analysis.
.. toctree::
:maxdepth: 2
:caption: User's Guide
what_is_ndpi
.. toctree::
:maxdepth: 2
:caption: Developer's Guide
api
flow_risks
.. toctree::
:caption: Other Products
ntopng <https://www.ntop.org/guides/ntopng/>
nProbe <https://www.ntop.org/guides/nprobe/>
nProbe Cento <https://www.ntop.org/guides/cento/>
n2disk <https://www.ntop.org/guides/n2disk/>
PF_RING <https://www.ntop.org/guides/pf_ring/>
nEdge <https://www.ntop.org/guides/nedge/>
nScrub <https://www.ntop.org/guides/nscrub/>
nBox <https://www.ntop.org/guides/nbox/>
.. Indices and tables
.. ==================
..
.. * :ref:`genindex`
.. * :ref:`modindex`
.. * :ref:`search`
+56
View File
@@ -0,0 +1,56 @@
What is nDPI
##############
nDPI is a DPI (Deep Packet Inspection) toolkit able to:
- Detect application protocol on traffic flows
- Analize encrypted traffic flows via ET (Encrypted Traffic Analysis)
- Extract selected protocol metadata from traffic
- Implement APIs for analysing traffic
Releases and Features
---------------------
nDPI development lifecycle is typically 6 to 9 months. The history of changes and features implemented by every release, is available on its `Changelog <https://github.com/ntop/nDPI/blob/dev/CHANGELOG.md>`_.
Installation
============
nDPI is open source and available on `GitHub
<https://github.com/ntop/nDPI>`_. In addition, pre-compiled, binary
nDPI packages are available both for Linux and other platforms. Installation
instructions for binary packages are available below.
Installing on Linux
-------------------
Installation instructions can be found at
http://packages.ntop.org/. Development and stable builds are
available. Stable builds are intended for production environments whereas
development builds are intended for testing or early feature access.
Software Updates
================
General instructions for updating the software can be found at
http://packages.ntop.org/ together with the installation instructions.
Depending on the Operating System, nDPI supports also automatic updates
through the GUI as described in the below sections.
Updating the Software on Linux
------------------------------
Instructions for updating the software via command line can be found
at http://packages.ntop.org/. For example on Ubuntu/Debian systems the
below commands will update the repository, check for updates and install
the latest software update if any:
.. code:: bash
apt-get update
apt-get upgrade
.. _AvailableVersions:
+1 -1
View File
@@ -21,7 +21,7 @@ SRCS-y := reader_util.c intrusion_detection.c ndpiReader.c
CFLAGS += -g
CFLAGS += -Wno-strict-prototypes -Wno-missing-prototypes -Wno-missing-declarations -Wno-unused-parameter -I $(PWD)/../src/include @CFLAGS@ -DUSE_DPDK
LDLIBS = $(LIBNDPI) @PCAP_LIB@ @LIBS@ -lpthread -lm @LDFLAGS@
LDLIBS = $(LIBNDPI) @PCAP_LIB@ @LIBS@ @ADDITIONAL_LIBS@ -lpthread @LDFLAGS@
include $(RTE_SDK)/mk/rte.extapp.mk
+29 -16
View File
@@ -1,55 +1,70 @@
AR=@AR@
CC=@CC@
CXX=@CXX@
BUILD_MINGW=@BUILD_MINGW@
BUILD_MINGW_X64=@BUILD_MINGW_X64@
SRCHOME=../src
CFLAGS=-g -fPIC -DPIC -I$(SRCHOME)/include @PCAP_INC@ @CFLAGS@
LDFLAGS=@LDFLAGS@
LIBNDPI=$(SRCHOME)/lib/libndpi.a
LDFLAGS=$(LIBNDPI) @PCAP_LIB@ @LIBS@ @ADDITIONAL_LIBS@ -lpthread -lm @LDFLAGS@
LIBS=$(LIBNDPI) @PCAP_LIB@ @LIBS@ @ADDITIONAL_LIBS@
HEADERS=intrusion_detection.h reader_util.h $(SRCHOME)/include/ndpi_api.h \
$(SRCHOME)/include/ndpi_typedefs.h $(SRCHOME)/include/ndpi_protocol_ids.h
OBJS=ndpiReader.o reader_util.o intrusion_detection.o
PREFIX?=@prefix@
ifneq ($(BUILD_MINGW),)
all:
@echo 'Examples disabled due to mingw build.'
SUFFIX:=.exe
CFLAGS+=-I@srcdir@/../windows/WpdPack/Include -I@srcdir@/../windows/WpdPack/Include/pcap
LIBS+=-Wl,-Bstatic -lpthread -Wl,-Bdynamic
ifneq ($(BUILD_MINGW_X64),)
LIBS+=@srcdir@/../windows/WpdPack/Lib/x64/wpcap.lib
else
LIBS+=@srcdir@/../windows/WpdPack/Lib/wpcap.lib
endif
else
all: ndpiReader @DPDK_TARGET@
LIBS+=-lpthread
endif
all: ndpiReader$(SUFFIX) @DPDK_TARGET@
EXECUTABLE_SOURCES := ndpiReader.c ndpiSimpleIntegration.c
COMMON_SOURCES := $(filter-out $(EXECUTABLE_SOURCES),$(wildcard *.c ))
libndpiReader.a: $(COMMON_SOURCES:%.c=%.o) $(LIBNDPI)
ar rsv libndpiReader.a $(COMMON_SOURCES:%.c=%.o)
$(AR) rsv libndpiReader.a $(COMMON_SOURCES:%.c=%.o)
ndpiReader: libndpiReader.a $(LIBNDPI) ndpiReader.o
$(CC) $(CFLAGS) ndpiReader.o libndpiReader.a -o $@ $(LDFLAGS)
ndpiReader$(SUFFIX): libndpiReader.a $(LIBNDPI) ndpiReader.o
$(CC) $(CFLAGS) $(LDFLAGS) ndpiReader.o libndpiReader.a $(LIBS) -o $@
ndpiSimpleIntegration: ndpiSimpleIntegration.o
$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)
ndpiSimpleIntegration$(SUFFIX): ndpiSimpleIntegration.o
$(CC) $(CFLAGS) $(LDFLAGS) $< $(LIBS) -o $@
%.o: %.c $(HEADERS) Makefile
$(CC) $(CFLAGS) -c $< -o $@
$(CC) $(CFLAGS) $(CPPFLAGS) -c $< -o $@
install: ndpiReader
install: ndpiReader$(SUFFIX)
mkdir -p $(DESTDIR)$(PREFIX)/bin/
mkdir -p $(DESTDIR)$(PREFIX)/share/ndpi
cp ndpiReader $(DESTDIR)$(PREFIX)/bin/
cp ndpiReader$(SUFFIX) $(DESTDIR)$(PREFIX)/bin/
cp protos.txt $(DESTDIR)$(PREFIX)/share/ndpi/ndpiProtos.txt
cp mining_hosts.txt $(DESTDIR)$(PREFIX)/share/ndpi/ndpiCustomCategory.txt
[ -f ndpiSimpleIntegration$(SUFFIX) ] && cp ndpiSimpleIntegration$(SUFFIX) $(DESTDIR)$(PREFIX)/bin/ndpiSimpleIntegration$(SUFFIX) || true
[ -f build/app/ndpiReader.dpdk ] && cp build/app/ndpiReader.dpdk $(DESTDIR)$(PREFIX)/bin/ || true
[ -f ndpiReader.dpdk ] && cp ndpiReader.dpdk $(DESTDIR)$(PREFIX)/bin/ || true
dpdk:
make -f Makefile.dpdk
$(MAKE) -f Makefile.dpdk
check:
cppcheck --template='{file}:{line}:{severity}:{message}' --quiet --enable=all --force -I$(SRCHOME)/include *.c
clean:
/bin/rm -f *.o ndpiReader ndpiReader.dpdk
/bin/rm -f *.o ndpiReader ndpiSimpleIntegration ndpiReader$(SUFFIX) ndpiSimpleIntegration$(SUFFIX) ndpiReader.dpdk
/bin/rm -f .*.dpdk.cmd .*.o.cmd *.dpdk.map .*.o.d
/bin/rm -f _install _postbuild _postinstall _preinstall
/bin/rm -rf build
@@ -61,5 +76,3 @@ distdir:
distclean: clean
/bin/rm -f Makefile.dpdk
/bin/rm -f Makefile
endif
-20
View File
@@ -1,20 +0,0 @@
Microsoft Visual Studio Solution File, Format Version 11.00
# Visual C++ Express 2010
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pcapExample", "pcapExample\pcapExample.vcxproj", "{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Win32 = Debug|Win32
Release|Win32 = Release|Win32
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}.Debug|Win32.ActiveCfg = Debug|Win32
{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}.Debug|Win32.Build.0 = Debug|Win32
{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}.Release|Win32.ActiveCfg = Release|Win32
{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}.Release|Win32.Build.0 = Release|Win32
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
EndGlobal
Binary file not shown.
-11
View File
@@ -1,11 +0,0 @@
// pcapExample.cpp : Defines the entry point for the console application.
//
#include "stdafx.h"
int _tmain(int argc, _TCHAR* argv[])
{
return 0;
}
@@ -1,231 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|Win32">
<Configuration>Release</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
</ItemGroup>
<PropertyGroup Label="Globals">
<ProjectGuid>{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}</ProjectGuid>
<Keyword>Win32Proj</Keyword>
<RootNamespace>pcapExample</RootNamespace>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<WholeProgramOptimization>true</WholeProgramOptimization>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<PropertyGroup Label="UserMacros" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<LinkIncremental>true</LinkIncremental>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<LinkIncremental>false</LinkIncremental>
</PropertyGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<ClCompile>
<PrecompiledHeader>NotUsing</PrecompiledHeader>
<WarningLevel>Level3</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>C:\ntop\nprobe\nprobe-win-32;C:\ntop\winpcap\Include;C:\ntop\dependencies\nDPI\src\lib\third_party\include;C:\ntop\dependencies\nDPI\src\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<AdditionalDependencies>Wsock32.lib;Ws2_32.lib;C:\ntop\winpcap\Lib\Packet.lib;C:\ntop\winpcap\Lib\wpcap.lib;%(AdditionalDependencies)</AdditionalDependencies>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<ClCompile>
<WarningLevel>Level3</WarningLevel>
<PrecompiledHeader>Use</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
</Link>
</ItemDefinitionGroup>
<ItemGroup>
<ClCompile Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt.c" />
<ClCompile Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt1.c" />
<ClCompile Include="..\..\..\src\lib\ndpi_main.c" />
<ClCompile Include="..\..\..\src\lib\protocols\afp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\aimini.c" />
<ClCompile Include="..\..\..\src\lib\protocols\applejuice.c" />
<ClCompile Include="..\..\..\src\lib\protocols\armagetron.c" />
<ClCompile Include="..\..\..\src\lib\protocols\battlefield.c" />
<ClCompile Include="..\..\..\src\lib\protocols\bgp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\bittorrent.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ciscovpn.c" />
<ClCompile Include="..\..\..\src\lib\protocols\citrix.c" />
<ClCompile Include="..\..\..\src\lib\protocols\corba.c" />
<ClCompile Include="..\..\..\src\lib\protocols\crossfire.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dcerpc.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dhcp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dhcpv6.c" />
<ClCompile Include="..\..\..\src\lib\protocols\directconnect.c" />
<ClCompile Include="..\..\..\src\lib\protocols\directdownloadlink.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dns.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dofus.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dropbox.c" />
<ClCompile Include="..\..\..\src\lib\protocols\edonkey.c" />
<ClCompile Include="..\..\..\src\lib\protocols\fasttrack.c" />
<ClCompile Include="..\..\..\src\lib\protocols\feidian.c" />
<ClCompile Include="..\..\..\src\lib\protocols\fiesta.c" />
<ClCompile Include="..\..\..\src\lib\protocols\filetopia.c" />
<ClCompile Include="..\..\..\src\lib\protocols\flash.c" />
<ClCompile Include="..\..\..\src\lib\protocols\florensia.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ftp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\gadu_gadu.c" />
<ClCompile Include="..\..\..\src\lib\protocols\gnutella.c" />
<ClCompile Include="..\..\..\src\lib\protocols\gtp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\guildwars.c" />
<ClCompile Include="..\..\..\src\lib\protocols\h323.c" />
<ClCompile Include="..\..\..\src\lib\protocols\halflife2_and_mods.c" />
<ClCompile Include="..\..\..\src\lib\protocols\http.c" />
<ClCompile Include="..\..\..\src\lib\protocols\http_activesync.c" />
<ClCompile Include="..\..\..\src\lib\protocols\i23v5.c" />
<ClCompile Include="..\..\..\src\lib\protocols\iax.c" />
<ClCompile Include="..\..\..\src\lib\protocols\icecast.c" />
<ClCompile Include="..\..\..\src\lib\protocols\imesh.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ipp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\irc.c" />
<ClCompile Include="..\..\..\src\lib\protocols\jabber.c" />
<ClCompile Include="..\..\..\src\lib\protocols\kerberos.c" />
<ClCompile Include="..\..\..\src\lib\protocols\kontiki.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ldap.c" />
<ClCompile Include="..\..\..\src\lib\protocols\lotus_notes.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mail_imap.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mail_pop.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mail_smtp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\maplestory.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mdns.c" />
<ClCompile Include="..\..\..\src\lib\protocols\meebo.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mgcp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mms.c" />
<ClCompile Include="..\..\..\src\lib\protocols\msn.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mssql.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mysql.c" />
<ClCompile Include="..\..\..\src\lib\protocols\netbios.c" />
<ClCompile Include="..\..\..\src\lib\protocols\netflow.c" />
<ClCompile Include="..\..\..\src\lib\protocols\nfs.c" />
<ClCompile Include="..\..\..\src\lib\protocols\noe.c" />
<ClCompile Include="..\..\..\src\lib\protocols\non_tcp_udp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ntp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\openft.c" />
<ClCompile Include="..\..\..\src\lib\protocols\openvpn.c" />
<ClCompile Include="..\..\..\src\lib\protocols\oracle.c" />
<ClCompile Include="..\..\..\src\lib\protocols\oscar.c" />
<ClCompile Include="..\..\..\src\lib\protocols\pando.c" />
<ClCompile Include="..\..\..\src\lib\protocols\pcanywhere.c" />
<ClCompile Include="..\..\..\src\lib\protocols\popo.c" />
<ClCompile Include="..\..\..\src\lib\protocols\postgres.c" />
<ClCompile Include="..\..\..\src\lib\protocols\pplive.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ppstream.c" />
<ClCompile Include="..\..\..\src\lib\protocols\pptp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\qq.c" />
<ClCompile Include="..\..\..\src\lib\protocols\quake.c" />
<ClCompile Include="..\..\..\src\lib\protocols\quic.c" />
<ClCompile Include="..\..\..\src\lib\protocols\radius.c" />
<ClCompile Include="..\..\..\src\lib\protocols\rdp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\rsync.c" />
<ClCompile Include="..\..\..\src\lib\protocols\rtcp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\rtp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\rtsp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\sflow.c" />
<ClCompile Include="..\..\..\src\lib\protocols\shoutcast.c" />
<ClCompile Include="..\..\..\src\lib\protocols\sip.c" />
<ClCompile Include="..\..\..\src\lib\protocols\skinny.c" />
<ClCompile Include="..\..\..\src\lib\protocols\skype.c" />
<ClCompile Include="..\..\..\src\lib\protocols\smb.c" />
<ClCompile Include="..\..\..\src\lib\protocols\snmp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\socrates.c" />
<ClCompile Include="..\..\..\src\lib\protocols\sopcast.c" />
<ClCompile Include="..\..\..\src\lib\protocols\soulseek.c" />
<ClCompile Include="..\..\..\src\lib\protocols\spotify.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ssdp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ssh.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ssl.c" />
<ClCompile Include="..\..\..\src\lib\protocols\stealthnet.c" />
<ClCompile Include="..\..\..\src\lib\protocols\steam.c" />
<ClCompile Include="..\..\..\src\lib\protocols\stun.c" />
<ClCompile Include="..\..\..\src\lib\protocols\syslog.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tcp_udp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tds.c" />
<ClCompile Include="..\..\..\src\lib\protocols\teamspeak.c" />
<ClCompile Include="..\..\..\src\lib\protocols\teamviewer.c" />
<ClCompile Include="..\..\..\src\lib\protocols\telnet.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tftp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\thunder.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tor.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tvants.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tvuplayer.c" />
<ClCompile Include="..\..\..\src\lib\protocols\usenet.c" />
<ClCompile Include="..\..\..\src\lib\protocols\veohtv.c" />
<ClCompile Include="..\..\..\src\lib\protocols\viber.c" />
<ClCompile Include="..\..\..\src\lib\protocols\vmware.c" />
<ClCompile Include="..\..\..\src\lib\protocols\vnc.c" />
<ClCompile Include="..\..\..\src\lib\protocols\warcraft3.c" />
<ClCompile Include="..\..\..\src\lib\protocols\winmx.c" />
<ClCompile Include="..\..\..\src\lib\protocols\world_of_kung_fu.c" />
<ClCompile Include="..\..\..\src\lib\protocols\world_of_warcraft.c" />
<ClCompile Include="..\..\..\src\lib\protocols\xbox.c" />
<ClCompile Include="..\..\..\src\lib\protocols\xdmcp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\yahoo.c" />
<ClCompile Include="..\..\..\src\lib\protocols\zattoo.c" />
<ClCompile Include="..\..\..\src\lib\third_party\src\ahocorasick.c" />
<ClCompile Include="..\..\..\src\lib\third_party\src\node.c" />
<ClCompile Include="..\..\..\src\lib\third_party\src\sort.c" />
<ClCompile Include="..\..\pcapReader.c" />
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt.h" />
<ClInclude Include="..\..\..\src\include\linux_compat.h" />
<ClInclude Include="..\..\..\src\include\ndpi_api.h" />
<ClInclude Include="..\..\..\src\include\ndpi_debug_functions.h" />
<ClInclude Include="..\..\..\src\include\ndpi_define.h" />
<ClInclude Include="..\..\..\src\include\ndpi_macros.h" />
<ClInclude Include="..\..\..\src\include\ndpi_main.h" />
<ClInclude Include="..\..\..\src\include\ndpi_protocols.h" />
<ClInclude Include="..\..\..\src\include\ndpi_protocols_osdpi.h" />
<ClInclude Include="..\..\..\src\include\ndpi_protocol_history.h" />
<ClInclude Include="..\..\..\src\include\ndpi_public_functions.h" />
<ClInclude Include="..\..\..\src\include\ndpi_structs.h" />
<ClInclude Include="..\..\..\src\include\ndpi_utils.h" />
<ClInclude Include="..\..\..\src\lib\third_party\include\actypes.h" />
<ClInclude Include="..\..\..\src\lib\third_party\include\ahocorasick.h" />
<ClInclude Include="..\..\..\src\lib\third_party\include\node.h" />
<ClInclude Include="..\..\..\src\lib\third_party\include\sort.h" />
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
</Project>
@@ -1,465 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Filter Include="Source Files">
<UniqueIdentifier>{4FC737F1-C7A5-4376-A066-2A32D752A2FF}</UniqueIdentifier>
<Extensions>cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx</Extensions>
</Filter>
<Filter Include="Header Files">
<UniqueIdentifier>{93995380-89BD-4b04-88EB-625FBE52EBFB}</UniqueIdentifier>
<Extensions>h;hpp;hxx;hm;inl;inc;xsd</Extensions>
</Filter>
<Filter Include="Resource Files">
<UniqueIdentifier>{67DA6AB6-F800-4c08-8B7A-83BB121AAD01}</UniqueIdentifier>
<Extensions>rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav;mfcribbon-ms</Extensions>
</Filter>
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\pcapReader.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\ndpi_main.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\zattoo.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\afp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\aimini.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\applejuice.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\armagetron.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\battlefield.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\bgp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\bittorrent.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ciscovpn.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\citrix.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\corba.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\crossfire.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dcerpc.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dhcp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dhcpv6.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\directconnect.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\directdownloadlink.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dns.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dofus.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dropbox.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\edonkey.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\fasttrack.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\feidian.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\fiesta.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\filetopia.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\flash.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\florensia.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ftp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\gadu_gadu.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\gnutella.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\gtp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\guildwars.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\h323.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\halflife2_and_mods.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\http.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\http_activesync.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\i23v5.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\iax.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\icecast.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\imesh.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ipp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\irc.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\jabber.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\kerberos.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\kontiki.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ldap.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\lotus_notes.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mail_imap.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mail_pop.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mail_smtp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\maplestory.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mdns.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\meebo.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mgcp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mms.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\msn.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mssql.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mysql.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\netbios.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\netflow.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\nfs.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\noe.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\non_tcp_udp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ntp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\openft.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\openvpn.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\oracle.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\oscar.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\pando.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\pcanywhere.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\popo.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\postgres.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\pplive.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ppstream.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\pptp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\qq.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\quake.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\quic.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\radius.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\rdp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\rsync.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\rtcp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\rtp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\rtsp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\sflow.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\shoutcast.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\sip.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\skinny.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\skype.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\smb.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\snmp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\socrates.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\sopcast.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\soulseek.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\spotify.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ssdp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ssh.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ssl.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\stealthnet.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\steam.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\stun.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\syslog.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tcp_udp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tds.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\teamspeak.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\teamviewer.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\telnet.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tftp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\thunder.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tor.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tvants.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tvuplayer.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\usenet.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\veohtv.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\viber.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\vmware.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\vnc.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\warcraft3.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\winmx.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\world_of_kung_fu.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\world_of_warcraft.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\xbox.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\xdmcp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\yahoo.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\third_party\src\ahocorasick.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\third_party\src\node.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\third_party\src\sort.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt1.c">
<Filter>Source Files</Filter>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\src\lib\third_party\include\actypes.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\lib\third_party\include\ahocorasick.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\lib\third_party\include\node.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\lib\third_party\include\sort.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_api.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_debug_functions.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_define.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_macros.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_main.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_protocol_history.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_protocols.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_protocols_osdpi.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_public_functions.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_structs.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_utils.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\linux_compat.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt.h">
<Filter>Header Files</Filter>
</ClInclude>
</ItemGroup>
</Project>
@@ -1,3 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
</Project>
-462
View File
@@ -1,462 +0,0 @@
/*
* intrusion_detection.c
*
* Copyright (C) 2011-20 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#include "intrusion_detection.h"
double normalize(ndpi_norm_value* tresholds){
if(tresholds->upper_bound != tresholds->lower_bound){
tresholds->norm_value = (tresholds->value - tresholds->lower_bound) / (tresholds->upper_bound - tresholds->lower_bound);
}else{
if(tresholds->value > tresholds->upper_bound){
tresholds->norm_value = 1 + (tresholds->value - tresholds->lower_bound) / tresholds->upper_bound;
}else{
tresholds->norm_value = 1 - (tresholds->value - tresholds->lower_bound) / tresholds->upper_bound;
}
}
if(tresholds->norm_value >= 0){
return tresholds->norm_value * tresholds->weight;
}
else{
return (1 - tresholds->norm_value) * tresholds->weight;
}
}
double get_flow_score(ndpi_norm_value* scores, int n_metrics){
double flow_score = 0;
for(int i=0; i<n_metrics; i++){
flow_score += normalize(&scores[i]);
}
return flow_score;
}
/* ********************************** */
double Ddos_score(struct ndpi_flow_info* flow){
int n_metrics = 6;
ndpi_norm_value* scores = malloc(n_metrics * sizeof(ndpi_norm_value));
/* pktlen_c_to_s_avg */
int i = 0;
scores[i].lower_bound = 70.0;
scores[i].upper_bound = 263.4799999999999;
scores[i].weight = 0.21257330032661592;
scores[i].value = ndpi_data_average(flow->pktlen_c_to_s);
/* pktlen_s_to_c_max */
i++;
scores[i].lower_bound = 90.0;
scores[i].upper_bound = 2974.0;
scores[i].weight = 0.21073785073559176;
scores[i].value = ndpi_data_max(flow->pktlen_s_to_c);
/* pktlen_s_to_c_avg */
i++;
scores[i].lower_bound = 72.7;
scores[i].upper_bound = 1130.4199999999996;
scores[i].weight = 0.21257330032661592;
scores[i].value = ndpi_data_average(flow->pktlen_s_to_c);
/* pktlen_s_to_c_stddev */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 906.0;
scores[i].weight = 0.20990954527912953;
scores[i].value = ndpi_data_stddev(flow->pktlen_s_to_c);
/* fin */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.07710300166602348;
scores[i].value = flow->fin_count;
/* s_to_c_fin */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.07710300166602348;
scores[i].value = flow->dst2src_fin_count;
// sum = 1.0
double flow_score = get_flow_score(scores, n_metrics);
free(scores);
return flow_score;
}
double Dos_goldeneye_score(struct ndpi_flow_info* flow){
int n_metrics = 6;
ndpi_norm_value* scores = malloc(n_metrics * sizeof(ndpi_norm_value));
/* pktlen_s_to_c_max */
int i = 0;
scores[i].lower_bound = 74.0;
scores[i].upper_bound = 3292.6699999999764;
scores[i].weight = 0.3123007140611667;
scores[i].value = ndpi_data_max(flow->pktlen_s_to_c);
/* pktlen_s_to_c_avg */
i++;
scores[i].lower_bound = 68.7;
scores[i].upper_bound = 1354.0569999999987;
scores[i].weight = 0.23802038891633356;
scores[i].value = ndpi_data_average(flow->pktlen_s_to_c);
/* pktlen_s_to_c_stddev */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 959.4469999999993;
scores[i].weight = 0.3111779763775991;
scores[i].value = ndpi_data_stddev(flow->pktlen_s_to_c);
/* syn */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.0464364305923564;
scores[i].value = flow->syn_count;
/* c_to_s_syn */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 1.0;
scores[i].weight = 0.04562805946018772;
scores[i].value = flow->src2dst_syn_count;
/* s_to_c_syn */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.0464364305923564;
scores[i].value = flow->dst2src_syn_count;
// sum = 0.9999999999999998
double flow_score = get_flow_score(scores, n_metrics);
free(scores);
return flow_score;
}
double Dos_hulk_score(struct ndpi_flow_info* flow){
double f = (double)flow->first_seen_ms/1000.0, l = (double)flow->last_seen_ms/1000.0;
int n_metrics = 6;
ndpi_norm_value* scores = malloc(n_metrics * sizeof(ndpi_norm_value));
/* duration */
int i = 0;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 539.40668006422;
scores[i].weight = 0.16666666666666666;
scores[i].value = (l - f);
/* src2dst_packets */
i++;
scores[i].lower_bound = 2.0;
scores[i].upper_bound = 41.0;
scores[i].weight = 0.16666666666666666;
scores[i].value = flow->src2dst_packets;
/* dst2src_packets */
i++;
scores[i].lower_bound = 2.0;
scores[i].upper_bound = 45.0;
scores[i].weight = 0.16666666666666666;
scores[i].value = flow->dst2src_packets;
/* src2dst_bytes */
i++;
scores[i].lower_bound = 146.0;
scores[i].upper_bound = 6306.300000000001;
scores[i].weight = 0.16666666666666666;
scores[i].value = flow->src2dst_bytes;
/* ack */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 82.0;
scores[i].weight = 0.16666666666666666;
scores[i].value = flow->ack_count;
/* syn */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.16666666666666666;
scores[i].value = flow->syn_count;
// sum = 0.9999999999999999
double flow_score = get_flow_score(scores, n_metrics);
free(scores);
return flow_score;
}
double Dos_slow_score(struct ndpi_flow_info* flow){
int n_metrics = 6;
ndpi_norm_value* scores = malloc(n_metrics * sizeof(ndpi_norm_value));
/* pktlen_s_to_c_max */
int i = 0;
scores[i].lower_bound = 90.0;
scores[i].upper_bound = 3135.0;
scores[i].weight = 0.1760747755022144;
scores[i].value = ndpi_data_max(flow->pktlen_s_to_c);
/* pktlen_s_to_c_avg */
i++;
scores[i].lower_bound = 80.37100000000001;
scores[i].upper_bound = 1292.5900000000008;
scores[i].weight = 0.17600137023171597;
scores[i].value = ndpi_data_average(flow->pktlen_s_to_c);
/* dst2src_bytes */
i++;
scores[i].lower_bound = 262.0;
scores[i].upper_bound = 53227.80000000002;
scores[i].weight = 0.16919914849886225;
scores[i].value = flow->dst2src_bytes;
/* syn */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.168000195747388;
scores[i].value = flow->syn_count;
/* c_to_s_syn */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 1.0;
scores[i].weight = 0.14272431427243143;
scores[i].value = flow->src2dst_syn_count;
/* s_to_c_syn */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.168000195747388;
scores[i].value = flow->dst2src_syn_count;
// sum = 1.0
double flow_score = get_flow_score(scores, n_metrics);
free(scores);
return flow_score;
}
double Ftp_patator_score(struct ndpi_flow_info* flow){
int n_metrics = 6;
ndpi_norm_value* scores = malloc(n_metrics * sizeof(ndpi_norm_value));
/* iat_flow_min */
int i = 0;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 24.0;
scores[i].weight = 0.002732919254658385;
scores[i].value = ndpi_data_min(flow->iat_flow);
/* pktlen_s_to_c_max */
i++;
scores[i].lower_bound = 90.0;
scores[i].upper_bound = 3393.0;
scores[i].weight = 0.007453416149068323;
scores[i].value = ndpi_data_max(flow->pktlen_s_to_c);
/* pktlen_s_to_c_avg */
i++;
scores[i].lower_bound = 81.3;
scores[i].upper_bound = 1315.021;
scores[i].weight = 0.9833540372670807;
scores[i].value = ndpi_data_average(flow->pktlen_s_to_c);
/* dst2src_bytes */
i++;
scores[i].lower_bound = 256.0;
scores[i].upper_bound = 56434.0;
scores[i].weight = 0.0034782608695652175;
scores[i].value = flow->dst2src_bytes;
/* fin */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.0014906832298136647;
scores[i].value = flow->fin_count;
/* rst */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.0014906832298136647;
scores[i].value = flow->rst_count;
// sum = 1.0
double flow_score = get_flow_score(scores, n_metrics);
free(scores);
return flow_score;
}
double Hearthbleed_score(struct ndpi_flow_info* flow){
double f = (double)flow->first_seen_ms/1000.0, l = (double)flow->last_seen_ms/1000.0;
int n_metrics = 6;
ndpi_norm_value* scores = malloc(n_metrics * sizeof(ndpi_norm_value));
/* iat_flow_max */
int i = 0;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 595213.3999999999;
scores[i].weight = 0.16666666666666666;
scores[i].value = ndpi_data_max(flow->iat_flow);
/* iat_flow_stddev */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 245377.74799999973;
scores[i].weight = 0.16666666666666666;
scores[i].value = ndpi_data_stddev(flow->iat_flow);
/* pktlen_s_to_c_max */
i++;
scores[i].lower_bound = 74.0;
scores[i].upper_bound = 3380.0;
scores[i].weight = 0.16666666666666666;
scores[i].value = ndpi_data_max(flow->pktlen_s_to_c);
/* pktlen_s_to_c_avg */
i++;
scores[i].lower_bound = 70.0;
scores[i].upper_bound = 1344.6399999999996;
scores[i].weight = 0.16666666666666666;
scores[i].value = ndpi_data_average(flow->pktlen_s_to_c);
/* pktlen_s_to_c_stddev */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 944.6399999999996;
scores[i].weight = 0.16666666666666666;
scores[i].value = ndpi_data_stddev(flow->pktlen_s_to_c);
/* duration */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 711.6677598000391;
scores[i].weight = 0.16666666666666666;
scores[i].value = (l - f);
// sum = 0.9999999999999999
double flow_score = get_flow_score(scores, n_metrics);
free(scores);
return flow_score;
}
double Infiltration_score(struct ndpi_flow_info* flow){
int n_metrics = 6;
ndpi_norm_value* scores = malloc(n_metrics * sizeof(ndpi_norm_value));
/* pktlen_c_to_s_max */
int i = 0;
scores[i].lower_bound = 72.0;
scores[i].upper_bound = 1840.739999999998;
scores[i].weight = 0.11937557392102846;
scores[i].value = ndpi_data_max(flow->pktlen_c_to_s);
/* pktlen_c_to_s_avg */
i++;
scores[i].lower_bound = 70.0;
scores[i].upper_bound = 296.56599999999816;
scores[i].weight = 0.12526782981328435;
scores[i].value = ndpi_data_average(flow->pktlen_c_to_s);
/* pktlen_s_to_c_max */
i++;
scores[i].lower_bound = 90.0;
scores[i].upper_bound = 3496.1399999999776;
scores[i].weight = 0.13927150290786652;
scores[i].value = ndpi_data_max(flow->pktlen_s_to_c);
/* pktlen_s_to_c_avg */
i++;
scores[i].lower_bound = 72.6;
scores[i].upper_bound = 1367.7959999999991;
scores[i].weight = 0.12182430364248545;
scores[i].value = ndpi_data_average(flow->pktlen_s_to_c);
/* src2dst_bytes */
i++;
scores[i].lower_bound = 144.0;
scores[i].upper_bound = 7847.69999999999;
scores[i].weight = 0.12059993878175697;
scores[i].value = flow->src2dst_bytes;
/* dst2src_bytes */
i++;
scores[i].lower_bound = 236.0;
scores[i].upper_bound = 74486.7799999998;
scores[i].weight = 0.3736608509335782;
scores[i].value = flow->dst2src_bytes;
// sum = 1.0
double flow_score = get_flow_score(scores, n_metrics);
free(scores);
return flow_score;
}
double Ssh_patator_score(struct ndpi_flow_info* flow){
int n_metrics = 6;
ndpi_norm_value* scores = malloc(n_metrics * sizeof(ndpi_norm_value));
/* fin */
int i = 0;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.0033738191632928477;
scores[i].value = flow->fin_count;
/* psh */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 30.0;
scores[i].weight = 0.33076923076923076;
scores[i].value = flow->psh_count;
/* c_to_s_syn */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 1.0;
scores[i].weight = 0.0004048582995951417;
scores[i].value = flow->src2dst_syn_count;
/* c_to_s_psh */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 12.0;
scores[i].weight = 0.33130904183535764;
scores[i].value = flow->src2dst_psh_count;
/* s_to_c_fin */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.0033738191632928477;
scores[i].value = flow->dst2src_fin_count;
/* s_to_c_psh */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 30.0;
scores[i].weight = 0.33076923076923076;
scores[i].value = flow->dst2src_psh_count;
// sum = 1.0
double flow_score = get_flow_score(scores, n_metrics);
free(scores);
return flow_score;
}
-69
View File
@@ -1,69 +0,0 @@
/*
* intrusion_detection.h
*
* Copyright (C) 2011-19 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#ifndef _INTRUSION_DETECTION_H_
#define _INTRUSION_DETECTION_H_
/*
Code to detect attacks reported in
https://www.unb.ca/cic/datasets/ids-2017.html
https://www.unb.ca/cic/datasets/ids-2018.html
*/
#include <stdio.h>
#include <stdlib.h>
#include "reader_util.h"
#include "ndpi_api.h"
typedef struct norm_values{
double upper_bound;
double lower_bound;
double weight;
double value;
double norm_value;
}ndpi_norm_value;
double normalize(ndpi_norm_value* tresholds);
double get_flow_score(ndpi_norm_value* scores, int n_metrics);
/* ********************************** */
double Ddos_score(struct ndpi_flow_info* flow);
double Dos_goldeneye_score(struct ndpi_flow_info* flow);
double Dos_hulk_score(struct ndpi_flow_info* flow);
double Dos_slow_score(struct ndpi_flow_info* flow);
double Ftp_patator_score(struct ndpi_flow_info* flow);
double Hearthbleed_score(struct ndpi_flow_info* flow);
double Infiltration_score(struct ndpi_flow_info* flow);
double Ssh_patator_score(struct ndpi_flow_info* flow);
#endif /* _INTRUSION_DETECTION_H_ */
+109
View File
@@ -0,0 +1,109 @@
################################################################
# abuse.ch Suricata JA3 Fingerprint Blacklist (CSV) #
# For Suricata 4.1.0 or newer #
# Last updated: 2020-04-09 06:48:14 UTC #
# #
# Terms Of Use: https://sslbl.abuse.ch/blacklist/ #
# For questions please contact sslbl [at] abuse.ch #
################################################################
#
# ja3_md5,Firstseen,Lastseen,Listingreason
b386946a5a44d1ddcc843bc75336dfce,2017-07-14 18:08:15,2019-07-27 20:42:54,Dridex
8991a387e4cc841740f25d6f5139f92d,2017-07-14 19:02:03,2019-07-28 00:34:38,Adware
cb98a24ee4b9134448ffb5714fd870ac,2017-07-14 19:48:28,2019-05-22 03:22:38,Dridex
1aa7bf8b97e540ca5edd75f7b8384bfa,2017-07-14 20:23:38,2019-07-28 01:38:22,TrickBot
3d89c0dfb1fa44911b8fa7523ef8dedb,2017-07-15 04:23:45,2021-02-01 18:23:25,Adware
bc6c386f480ee97b9d9e52d472b772d8,2017-07-15 10:57:38,2021-02-21 13:13:30,Adware
8f52d1ce303fb4a6515836aec3cc16b1,2017-07-15 19:05:11,2019-07-27 20:00:57,TrickBot
d6f04b5a910115f4b50ecec09d40a1df,2017-07-15 19:42:24,2018-10-14 08:12:51,Dridex
35c0a31c481927f022a3b530255ac080,2017-07-15 19:43:19,2021-02-06 03:21:55,Tofsee
d551fafc4f40f1dec2bb45980bfa9492,2017-07-15 19:59:29,2020-11-16 13:06:20,Adware
e330bca99c8a5256ae126a55c4c725c5,2017-07-15 19:59:29,2021-01-13 00:29:37,Adware
b8f81673c0e1d29908346f3bab892b9b,2017-07-16 01:32:03,2018-12-17 06:08:03,Adware
83e04bc58d402f9633983cbf22724b02,2017-07-16 01:32:03,2021-01-14 14:23:42,Adware
70722097d1fe1d78d8c2164640ab6df4,2017-07-16 02:39:08,2021-02-22 04:56:57,Tofsee
9c2589e1c0e9f533a022c6205f9719e1,2017-07-16 08:37:17,2021-02-19 08:46:32,Adware
849b04bdbd1d2b983f6e8a457e0632a8,2017-07-16 08:37:17,2021-02-19 08:46:32,Adware
16efcf0e00504ddfedde13bfea997952,2017-07-16 19:45:45,2020-12-23 15:10:32,Adware
4d7a28d6f2263ed61de88ca66eb011e3,2017-07-16 21:20:29,2020-12-08 18:10:55,Tofsee
550dce18de1bb143e69d6dd9413b8355,2017-07-16 22:17:20,2018-12-21 07:04:50,Adware
c50f6a8b9173676b47ba6085bd0c6cee,2017-07-16 22:38:41,2019-05-21 09:42:17,TrickBot
20dd18bdd3209ea718989030a6f93364,2017-07-18 10:22:58,2019-04-28 09:23:31,Adware
8498fe4268764dbf926a38283e9d3d8f,2017-07-18 10:22:58,2020-12-31 02:06:27,Adware
590a232d04d56409fab72e752a8a2634,2017-07-18 18:53:24,2020-10-11 20:48:33,Tofsee
51a7ad14509fd614c7bb3a50c4982b8c,2017-07-19 07:28:19,2019-07-14 11:58:32,JBifrost
96eba628dcb2b47607192ba74a3b55ba,2017-07-19 18:53:48,2021-01-04 12:45:08,Tofsee
df5c30e670dba99f9270ed36060cf054,2017-07-20 17:44:07,2018-04-11 15:57:59,Tofsee
098f55e27d8c4b0a590102cbdb3a5f3a,2017-07-21 09:52:01,2019-04-08 01:09:54,Adware
46efd49abcca8ea9baa932da68fdb529,2017-07-22 14:07:36,2021-02-20 20:45:32,Adware
29085f03f8e8a03f0b399c5c7cf0b0b8,2017-07-22 14:07:36,2021-02-21 20:56:01,Adware
d7150af4514b868defb854db0f62a441,2017-07-23 09:39:24,2018-07-24 01:04:58,Tofsee
03e186a7f83285e93341de478334006e,2017-07-24 18:17:14,2021-02-13 23:55:58,Tofsee
3cda52da4ade09f1f781ad2e82dcfa20,2017-07-30 18:41:36,2019-05-21 17:34:18,Quakbot
b13d01846ad7a14a70bf030a16775c78,2017-08-08 07:12:49,2021-02-21 01:32:12,Adware
1543a7c46633acf71e8401baccbd0568,2017-08-08 21:32:28,2020-11-10 05:30:17,Tofsee
1d095e68489d3c535297cd8dffb06cb9,2017-08-12 19:56:28,2020-10-28 11:06:23,Tofsee
93d056782d649deb51cda44ecb714bb0,2017-08-28 12:20:47,2019-04-15 23:47:27,Adware
698e36219f3979420fa2581b21dac7ec,2017-08-28 12:20:47,2020-12-31 02:06:31,Adware
1712287800ac91b34cadd5884ce85568,2017-08-28 16:01:59,2021-01-23 09:36:32,TorrentLocker
5e573c9c9f8ba720ef9b18e9fce2e2f7,2017-08-30 13:44:56,2021-02-17 17:11:34,Adware
f6fd83a21f9f3c5f9ff7b5c63bbc179d,2017-10-20 08:03:21,2018-11-06 06:42:12,Adware
92579701f145605e9edc0b01a901c6d5,2017-10-23 00:10:48,2021-02-18 22:56:49,Adware
a61299f9b501adcf680b9275d79d4ac6,2017-11-04 18:03:59,2020-04-21 17:08:24,Tofsee
b2b61db7b9490a60d270ccb20b462826,2017-11-14 20:12:03,2021-02-18 22:56:49,Adware
7dcce5b76c8b17472d024758970a406b,2017-11-22 12:42:46,2020-12-22 15:15:49,Tofsee
534ce2dbc413c68e908363b5df0ae5e0,2017-12-22 09:36:21,2019-07-27 15:22:33,TrickBot
fb00055a1196aeea8d1bc609885ba953,2018-01-01 22:49:25,2019-04-09 06:58:58,TrickBot
a50a861119aceb0ccc74902e8fddb618,2018-01-02 08:16:23,2018-07-05 02:33:08,Tofsee
e7643725fcff971e3051fe0e47fc2c71,2018-01-31 08:06:13,2020-03-25 16:19:48,Tofsee
7c410ce832e848a3321432c9a82e972b,2018-01-31 20:04:25,2021-02-21 22:51:31,Tofsee
da949afd9bd6df820730f8f171584a71,2018-02-03 05:19:37,2021-02-21 05:20:43,Tofsee
906004246f3ba5e755b043c057254a29,2018-03-11 08:25:38,2018-04-14 00:59:16,Tofsee
fd80fa9c6120cdeea8520510f3c644ac,2018-03-11 09:34:30,2021-02-21 22:16:39,Tofsee
b90bdbe961a648f0427db21aaa6ccb59,2018-03-11 10:37:43,2020-05-29 23:39:01,Tofsee
1fe4c7a3544eb27afec2adfb3a3dbf60,2018-03-11 19:23:08,2021-02-20 11:19:52,Tofsee
c201b92f8b483fa388be174d6689f534,2018-03-12 13:43:52,2021-01-28 06:17:06,Gozi
9f62c4f26b90d3d757bea609e82f2eaf,2018-03-13 06:23:41,2020-12-18 16:07:31,Tofsee
1be3ecebe5aa9d3654e6e703d81f6928,2018-03-13 11:50:02,2021-02-14 23:03:13,Ransomware.Troldesh
e3b2ab1f9a56f2fb4c9248f2f41631fa,2018-03-15 01:06:34,2021-02-22 10:36:48,Tofsee
dff8a0aa1c904aaea76c5bf624e88333,2018-03-18 09:41:15,2020-10-27 09:50:24,Tofsee
17fd49722f8d11f3d76dce84f8e099a7,2018-03-19 23:02:27,2020-12-09 11:32:21,Tofsee
911479ac8a0813ed1241b3686ccdade9,2018-03-19 23:24:59,2020-03-30 04:09:18,Tofsee
c5deb9465d47232dd48772f9c4d14679,2018-03-22 15:42:48,2021-02-17 07:39:32,Tofsee
f22bdd57e3a52de86cda40da2d84e83b,2018-03-27 13:40:19,2019-01-20 14:31:39,Tofsee
d18a4da84af59e1108862a39bae7c9d4,2018-04-03 00:40:51,2021-02-06 01:53:12,Tofsee
2d8794cb7b52b777bee2695e79c15760,2018-04-04 06:56:37,2021-02-10 08:22:31,Ransomware
40adfd923eb82b89d8836ba37a19bca1,2018-04-15 15:49:08,2021-02-21 22:22:56,CoinMiner
1aee0238942d453d679fc1e37a303387,2018-05-13 01:59:49,2021-02-10 13:19:24,Tofsee
2092e1fffb45d7e4a19a57f9bc5e203a,2018-05-16 21:59:36,2018-09-05 01:58:33,Adware
bffa4501966196d3d6e90cee1f88fc89,2018-06-07 15:08:04,2020-03-16 00:03:44,Tofsee
807fca46d9d0cf63adf4e5e80e414bbe,2018-06-07 16:51:03,2021-02-20 03:15:38,Tofsee
fb58831f892190644fe44e25bc830b45,2018-06-08 12:07:59,2019-05-31 21:02:37,Adware
0cc1e84568e471aa1d62ad4158ade6b5,2018-06-24 10:50:47,2021-02-10 21:48:57,Tofsee
d2935c58fe676744fecc8614ee5356c7,2018-08-14 21:48:41,2021-02-21 21:44:55,Adwind
8916410db85077a5460817142dcbc8de,2018-08-21 12:32:28,2021-02-21 20:02:19,TrickBot
c5235d3a8b9934b7fbbd204d50bc058d,2018-08-23 17:36:08,2019-10-13 05:11:09,Gootkit
57f3642b4e37e28f5cbe3020c9331b4c,2018-08-28 15:54:53,2021-02-22 11:09:43,Gozi
e62a5f4d538cbf169c2af71bec2399b4,2018-08-30 15:45:40,2021-02-21 20:36:55,TrickBot
51c64c77e60f3980eea90869b68c58a8,2018-08-30 21:04:57,2021-02-21 22:09:16,Dridex
7691297bcb20a41233fd0a0baa0a3628,2018-09-17 02:50:05,2021-02-21 17:05:57,Adware
7dd50e112cd23734a310b90f6f44a7cd,2018-09-17 17:54:58,2021-02-22 11:14:04,Quakbot
52c7396a501e4fecbdfa99c5408334ac,2018-09-18 00:29:04,2019-12-03 17:24:02,Tofsee
f735bbc6b69723b9df7b0e7ef27872af,2018-10-02 18:04:16,2021-02-19 06:55:39,TrickBot
49ed2ef3f1321e5f044f1e71b0e6fdd5,2018-10-02 18:04:17,2021-02-19 06:55:39,TrickBot
d76ee64fb7273733cbe455ac81c292e6,2018-11-16 13:26:39,2018-11-18 19:19:36,Tofsee
8f6c918dcb585ebbea05e2cc94530e3d,2018-11-16 13:26:41,2020-05-06 15:45:21,Tofsee
34f14a69ad7009ca5863379218af17f3,2018-11-17 05:17:22,2021-01-28 08:19:18,Tofsee
c2b4710c6888a5d47befe865c8e6fb19,2018-11-29 20:46:04,2020-12-16 15:08:09,Tofsee
decfb48a53789ebe081b88aabb58ee34,2018-12-21 09:06:16,2021-02-16 12:28:53,Adwind
08a8a4e85b25ac42e1490bc85cfdb5ce,2019-01-30 02:48:34,2020-10-27 09:50:19,Tofsee
c0220cd64849a629397a9cb68f78a0ea,2019-03-24 00:12:32,2021-02-13 05:13:33,Tofsee
7a29c223fb122ec64d10f0a159e07996,2019-06-09 22:55:29,2020-10-27 09:50:26,Tofsee
44dab16d680ef93487bc16ad23b3ffb1,2019-06-09 22:55:29,2020-10-27 09:50:25,Tofsee
70a04365be5bbd4653698bebeb43ce68,2019-07-02 06:26:56,2020-05-30 04:19:00,Tofsee
d81d654effb94714a4086734fa0adad9,2019-07-16 23:29:02,2020-10-27 09:50:21,Tofsee
25d74b7b4b779eb1efd4b31d26d651c6,2019-08-03 20:15:33,2020-07-14 21:43:25,Tofsee
fc2299d5b2964cd242c5a2c8c531a5f0,2019-08-09 23:56:32,2021-02-22 08:49:18,Tofsee
32926ca3e59f0413d0b98725454594f5,2019-09-12 06:56:10,2020-10-27 21:49:31,Tofsee
ffefafdb86336d057eda5fdf02b3d5ce,2019-10-26 07:31:49,2020-07-25 00:14:09,Tofsee
# END (98) entries
1 ################################################################
2 # abuse.ch Suricata JA3 Fingerprint Blacklist (CSV) #
3 # For Suricata 4.1.0 or newer #
4 # Last updated: 2020-04-09 06:48:14 UTC #
5 # #
6 # Terms Of Use: https://sslbl.abuse.ch/blacklist/ #
7 # For questions please contact sslbl [at] abuse.ch #
8 ################################################################
9 #
10 # ja3_md5,Firstseen,Lastseen,Listingreason
11 b386946a5a44d1ddcc843bc75336dfce,2017-07-14 18:08:15,2019-07-27 20:42:54,Dridex
12 8991a387e4cc841740f25d6f5139f92d,2017-07-14 19:02:03,2019-07-28 00:34:38,Adware
13 cb98a24ee4b9134448ffb5714fd870ac,2017-07-14 19:48:28,2019-05-22 03:22:38,Dridex
14 1aa7bf8b97e540ca5edd75f7b8384bfa,2017-07-14 20:23:38,2019-07-28 01:38:22,TrickBot
15 3d89c0dfb1fa44911b8fa7523ef8dedb,2017-07-15 04:23:45,2021-02-01 18:23:25,Adware
16 bc6c386f480ee97b9d9e52d472b772d8,2017-07-15 10:57:38,2021-02-21 13:13:30,Adware
17 8f52d1ce303fb4a6515836aec3cc16b1,2017-07-15 19:05:11,2019-07-27 20:00:57,TrickBot
18 d6f04b5a910115f4b50ecec09d40a1df,2017-07-15 19:42:24,2018-10-14 08:12:51,Dridex
19 35c0a31c481927f022a3b530255ac080,2017-07-15 19:43:19,2021-02-06 03:21:55,Tofsee
20 d551fafc4f40f1dec2bb45980bfa9492,2017-07-15 19:59:29,2020-11-16 13:06:20,Adware
21 e330bca99c8a5256ae126a55c4c725c5,2017-07-15 19:59:29,2021-01-13 00:29:37,Adware
22 b8f81673c0e1d29908346f3bab892b9b,2017-07-16 01:32:03,2018-12-17 06:08:03,Adware
23 83e04bc58d402f9633983cbf22724b02,2017-07-16 01:32:03,2021-01-14 14:23:42,Adware
24 70722097d1fe1d78d8c2164640ab6df4,2017-07-16 02:39:08,2021-02-22 04:56:57,Tofsee
25 9c2589e1c0e9f533a022c6205f9719e1,2017-07-16 08:37:17,2021-02-19 08:46:32,Adware
26 849b04bdbd1d2b983f6e8a457e0632a8,2017-07-16 08:37:17,2021-02-19 08:46:32,Adware
27 16efcf0e00504ddfedde13bfea997952,2017-07-16 19:45:45,2020-12-23 15:10:32,Adware
28 4d7a28d6f2263ed61de88ca66eb011e3,2017-07-16 21:20:29,2020-12-08 18:10:55,Tofsee
29 550dce18de1bb143e69d6dd9413b8355,2017-07-16 22:17:20,2018-12-21 07:04:50,Adware
30 c50f6a8b9173676b47ba6085bd0c6cee,2017-07-16 22:38:41,2019-05-21 09:42:17,TrickBot
31 20dd18bdd3209ea718989030a6f93364,2017-07-18 10:22:58,2019-04-28 09:23:31,Adware
32 8498fe4268764dbf926a38283e9d3d8f,2017-07-18 10:22:58,2020-12-31 02:06:27,Adware
33 590a232d04d56409fab72e752a8a2634,2017-07-18 18:53:24,2020-10-11 20:48:33,Tofsee
34 51a7ad14509fd614c7bb3a50c4982b8c,2017-07-19 07:28:19,2019-07-14 11:58:32,JBifrost
35 96eba628dcb2b47607192ba74a3b55ba,2017-07-19 18:53:48,2021-01-04 12:45:08,Tofsee
36 df5c30e670dba99f9270ed36060cf054,2017-07-20 17:44:07,2018-04-11 15:57:59,Tofsee
37 098f55e27d8c4b0a590102cbdb3a5f3a,2017-07-21 09:52:01,2019-04-08 01:09:54,Adware
38 46efd49abcca8ea9baa932da68fdb529,2017-07-22 14:07:36,2021-02-20 20:45:32,Adware
39 29085f03f8e8a03f0b399c5c7cf0b0b8,2017-07-22 14:07:36,2021-02-21 20:56:01,Adware
40 d7150af4514b868defb854db0f62a441,2017-07-23 09:39:24,2018-07-24 01:04:58,Tofsee
41 03e186a7f83285e93341de478334006e,2017-07-24 18:17:14,2021-02-13 23:55:58,Tofsee
42 3cda52da4ade09f1f781ad2e82dcfa20,2017-07-30 18:41:36,2019-05-21 17:34:18,Quakbot
43 b13d01846ad7a14a70bf030a16775c78,2017-08-08 07:12:49,2021-02-21 01:32:12,Adware
44 1543a7c46633acf71e8401baccbd0568,2017-08-08 21:32:28,2020-11-10 05:30:17,Tofsee
45 1d095e68489d3c535297cd8dffb06cb9,2017-08-12 19:56:28,2020-10-28 11:06:23,Tofsee
46 93d056782d649deb51cda44ecb714bb0,2017-08-28 12:20:47,2019-04-15 23:47:27,Adware
47 698e36219f3979420fa2581b21dac7ec,2017-08-28 12:20:47,2020-12-31 02:06:31,Adware
48 1712287800ac91b34cadd5884ce85568,2017-08-28 16:01:59,2021-01-23 09:36:32,TorrentLocker
49 5e573c9c9f8ba720ef9b18e9fce2e2f7,2017-08-30 13:44:56,2021-02-17 17:11:34,Adware
50 f6fd83a21f9f3c5f9ff7b5c63bbc179d,2017-10-20 08:03:21,2018-11-06 06:42:12,Adware
51 92579701f145605e9edc0b01a901c6d5,2017-10-23 00:10:48,2021-02-18 22:56:49,Adware
52 a61299f9b501adcf680b9275d79d4ac6,2017-11-04 18:03:59,2020-04-21 17:08:24,Tofsee
53 b2b61db7b9490a60d270ccb20b462826,2017-11-14 20:12:03,2021-02-18 22:56:49,Adware
54 7dcce5b76c8b17472d024758970a406b,2017-11-22 12:42:46,2020-12-22 15:15:49,Tofsee
55 534ce2dbc413c68e908363b5df0ae5e0,2017-12-22 09:36:21,2019-07-27 15:22:33,TrickBot
56 fb00055a1196aeea8d1bc609885ba953,2018-01-01 22:49:25,2019-04-09 06:58:58,TrickBot
57 a50a861119aceb0ccc74902e8fddb618,2018-01-02 08:16:23,2018-07-05 02:33:08,Tofsee
58 e7643725fcff971e3051fe0e47fc2c71,2018-01-31 08:06:13,2020-03-25 16:19:48,Tofsee
59 7c410ce832e848a3321432c9a82e972b,2018-01-31 20:04:25,2021-02-21 22:51:31,Tofsee
60 da949afd9bd6df820730f8f171584a71,2018-02-03 05:19:37,2021-02-21 05:20:43,Tofsee
61 906004246f3ba5e755b043c057254a29,2018-03-11 08:25:38,2018-04-14 00:59:16,Tofsee
62 fd80fa9c6120cdeea8520510f3c644ac,2018-03-11 09:34:30,2021-02-21 22:16:39,Tofsee
63 b90bdbe961a648f0427db21aaa6ccb59,2018-03-11 10:37:43,2020-05-29 23:39:01,Tofsee
64 1fe4c7a3544eb27afec2adfb3a3dbf60,2018-03-11 19:23:08,2021-02-20 11:19:52,Tofsee
65 c201b92f8b483fa388be174d6689f534,2018-03-12 13:43:52,2021-01-28 06:17:06,Gozi
66 9f62c4f26b90d3d757bea609e82f2eaf,2018-03-13 06:23:41,2020-12-18 16:07:31,Tofsee
67 1be3ecebe5aa9d3654e6e703d81f6928,2018-03-13 11:50:02,2021-02-14 23:03:13,Ransomware.Troldesh
68 e3b2ab1f9a56f2fb4c9248f2f41631fa,2018-03-15 01:06:34,2021-02-22 10:36:48,Tofsee
69 dff8a0aa1c904aaea76c5bf624e88333,2018-03-18 09:41:15,2020-10-27 09:50:24,Tofsee
70 17fd49722f8d11f3d76dce84f8e099a7,2018-03-19 23:02:27,2020-12-09 11:32:21,Tofsee
71 911479ac8a0813ed1241b3686ccdade9,2018-03-19 23:24:59,2020-03-30 04:09:18,Tofsee
72 c5deb9465d47232dd48772f9c4d14679,2018-03-22 15:42:48,2021-02-17 07:39:32,Tofsee
73 f22bdd57e3a52de86cda40da2d84e83b,2018-03-27 13:40:19,2019-01-20 14:31:39,Tofsee
74 d18a4da84af59e1108862a39bae7c9d4,2018-04-03 00:40:51,2021-02-06 01:53:12,Tofsee
75 2d8794cb7b52b777bee2695e79c15760,2018-04-04 06:56:37,2021-02-10 08:22:31,Ransomware
76 40adfd923eb82b89d8836ba37a19bca1,2018-04-15 15:49:08,2021-02-21 22:22:56,CoinMiner
77 1aee0238942d453d679fc1e37a303387,2018-05-13 01:59:49,2021-02-10 13:19:24,Tofsee
78 2092e1fffb45d7e4a19a57f9bc5e203a,2018-05-16 21:59:36,2018-09-05 01:58:33,Adware
79 bffa4501966196d3d6e90cee1f88fc89,2018-06-07 15:08:04,2020-03-16 00:03:44,Tofsee
80 807fca46d9d0cf63adf4e5e80e414bbe,2018-06-07 16:51:03,2021-02-20 03:15:38,Tofsee
81 fb58831f892190644fe44e25bc830b45,2018-06-08 12:07:59,2019-05-31 21:02:37,Adware
82 0cc1e84568e471aa1d62ad4158ade6b5,2018-06-24 10:50:47,2021-02-10 21:48:57,Tofsee
83 d2935c58fe676744fecc8614ee5356c7,2018-08-14 21:48:41,2021-02-21 21:44:55,Adwind
84 8916410db85077a5460817142dcbc8de,2018-08-21 12:32:28,2021-02-21 20:02:19,TrickBot
85 c5235d3a8b9934b7fbbd204d50bc058d,2018-08-23 17:36:08,2019-10-13 05:11:09,Gootkit
86 57f3642b4e37e28f5cbe3020c9331b4c,2018-08-28 15:54:53,2021-02-22 11:09:43,Gozi
87 e62a5f4d538cbf169c2af71bec2399b4,2018-08-30 15:45:40,2021-02-21 20:36:55,TrickBot
88 51c64c77e60f3980eea90869b68c58a8,2018-08-30 21:04:57,2021-02-21 22:09:16,Dridex
89 7691297bcb20a41233fd0a0baa0a3628,2018-09-17 02:50:05,2021-02-21 17:05:57,Adware
90 7dd50e112cd23734a310b90f6f44a7cd,2018-09-17 17:54:58,2021-02-22 11:14:04,Quakbot
91 52c7396a501e4fecbdfa99c5408334ac,2018-09-18 00:29:04,2019-12-03 17:24:02,Tofsee
92 f735bbc6b69723b9df7b0e7ef27872af,2018-10-02 18:04:16,2021-02-19 06:55:39,TrickBot
93 49ed2ef3f1321e5f044f1e71b0e6fdd5,2018-10-02 18:04:17,2021-02-19 06:55:39,TrickBot
94 d76ee64fb7273733cbe455ac81c292e6,2018-11-16 13:26:39,2018-11-18 19:19:36,Tofsee
95 8f6c918dcb585ebbea05e2cc94530e3d,2018-11-16 13:26:41,2020-05-06 15:45:21,Tofsee
96 34f14a69ad7009ca5863379218af17f3,2018-11-17 05:17:22,2021-01-28 08:19:18,Tofsee
97 c2b4710c6888a5d47befe865c8e6fb19,2018-11-29 20:46:04,2020-12-16 15:08:09,Tofsee
98 decfb48a53789ebe081b88aabb58ee34,2018-12-21 09:06:16,2021-02-16 12:28:53,Adwind
99 08a8a4e85b25ac42e1490bc85cfdb5ce,2019-01-30 02:48:34,2020-10-27 09:50:19,Tofsee
100 c0220cd64849a629397a9cb68f78a0ea,2019-03-24 00:12:32,2021-02-13 05:13:33,Tofsee
101 7a29c223fb122ec64d10f0a159e07996,2019-06-09 22:55:29,2020-10-27 09:50:26,Tofsee
102 44dab16d680ef93487bc16ad23b3ffb1,2019-06-09 22:55:29,2020-10-27 09:50:25,Tofsee
103 70a04365be5bbd4653698bebeb43ce68,2019-07-02 06:26:56,2020-05-30 04:19:00,Tofsee
104 d81d654effb94714a4086734fa0adad9,2019-07-16 23:29:02,2020-10-27 09:50:21,Tofsee
105 25d74b7b4b779eb1efd4b31d26d651c6,2019-08-03 20:15:33,2020-07-14 21:43:25,Tofsee
106 fc2299d5b2964cd242c5a2c8c531a5f0,2019-08-09 23:56:32,2021-02-22 08:49:18,Tofsee
107 32926ca3e59f0413d0b98725454594f5,2019-09-12 06:56:10,2020-10-27 21:49:31,Tofsee
108 ffefafdb86336d057eda5fdf02b3d5ce,2019-10-26 07:31:49,2020-07-25 00:14:09,Tofsee
109 # END (98) entries
+1231 -320
View File
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+18 -1
View File
@@ -10,7 +10,7 @@ tcp:3000@ntop
# Format:
# host:"<value>",host:"<value>",.....@<subproto>
host:"disneyplus.com"host:"cdn.registerdisney.go.com",host:"disney-portal.my.onetrust.com",host:"disneyplus.bn5x.net",host:"disney-plus.net"@DisneyPlus
host:"disneyplus.com",host:"cdn.registerdisney.go.com",host:"disney-portal.my.onetrust.com",host:"disneyplus.bn5x.net",host:"disney-plus.net"@DisneyPlus
host:"*.lvlt.dash.us.aiv-cdn.net.c.footprint.net"@AmazonVideo
host:"api-global.netflix.com"@Netflix
# IP based Subprotocols
@@ -39,3 +39,20 @@ ip:213.75.170.11/32:443@CustomProtocol
ip:8.248.73.247:443@AmazonPrime
ip:54.80.47.130@AmazonPrime
#
# Risk Exceptions
#
# ip_risk_mask: used to mask flow risks for IP addresses
# host_risk_mask: used to mask exceptions for domain names and hosts
#
# Syntax: <name>=<64 bit mask to be put in AND with the risk
#
# For IPs, the flow risk is put in AND (source IP mask OR destination IP mask)
# For Flows with a hostname (e.g. TLS) the risk is also put in AND with the host_risk_mask
#ip_risk_mask:192.168.1.0/24=0
#ip_risk_mask:10.196.157.228=0
host_risk_mask:".home"=0
# Custom certification autorities we trust
trusted_issuer_dn:"CN=813845657003339838, O=Code42, OU=TEST, ST=MN, C=US"
+393 -265
View File
File diff suppressed because it is too large Load Diff
+58 -42
View File
@@ -1,7 +1,7 @@
/*
* ndpi_util.h
*
* Copyright (C) 2011-19 - ntop.org
* Copyright (C) 2011-22 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
@@ -29,7 +29,7 @@
#ifndef __NDPI_UTIL_H__
#define __NDPI_UTIL_H__
#include "uthash.h"
#include "../src/lib/third_party/include/uthash.h"
#include <pcap.h>
#include "ndpi_includes.h"
#include "ndpi_classify.h"
@@ -54,6 +54,11 @@ extern int dpdk_port_init(int port, struct rte_mempool *mbuf_pool);
extern int dpdk_port_deinit(int port);
#endif
#define PLEN_MAX 1504
#define PLEN_BIN_LEN 32
#define PLEN_NUM_BINS 48 /* 47*32 = 1504 */
#define MAX_NUM_BIN_PKTS 256
/* ETTA Spec defiintions for feature readiness */
#define ETTA_MIN_PACKETS 10
#define ETTA_MIN_OCTETS 4000
@@ -89,38 +94,38 @@ extern int dpdk_port_deinit(int port);
// inner hash table (ja3 -> security state)
typedef struct ndpi_ja3_info {
char * ja3;
ndpi_cipher_weakness unsafe_cipher;
UT_hash_handle hh;
char * ja3;
ndpi_cipher_weakness unsafe_cipher;
UT_hash_handle hh;
} ndpi_ja3_info;
// external hash table (host ip -> <ip string, hash table ja3c, hash table ja3s>)
// used to aggregate ja3 fingerprints by hosts
typedef struct ndpi_host_ja3_fingerprints {
u_int32_t ip;
char *ip_string;
char *dns_name;
ndpi_ja3_info *host_client_info_hasht;
ndpi_ja3_info *host_server_info_hasht;
u_int32_t ip;
char *ip_string;
char *dns_name;
ndpi_ja3_info *host_client_info_hasht;
ndpi_ja3_info *host_server_info_hasht;
UT_hash_handle hh;
UT_hash_handle hh;
} ndpi_host_ja3_fingerprints;
//inner hash table
typedef struct ndpi_ip_dns{
u_int32_t ip;
char *ip_string;
char *dns_name; //server name if any;
UT_hash_handle hh;
u_int32_t ip;
char *ip_string;
char *dns_name; //server name if any;
UT_hash_handle hh;
} ndpi_ip_dns;
//hash table ja3 -> <host, ip, security>, used to aggregate host by ja3 fingerprints
typedef struct ndpi_ja3_fingerprints_host{
char *ja3; //key
ndpi_cipher_weakness unsafe_cipher;
ndpi_ip_dns *ipToDNS_ht;
UT_hash_handle hh;
char *ja3; //key
ndpi_cipher_weakness unsafe_cipher;
ndpi_ip_dns *ipToDNS_ht;
UT_hash_handle hh;
} ndpi_ja3_fingerprints_host;
struct flow_metrics {
@@ -129,7 +134,6 @@ struct flow_metrics {
struct ndpi_entropy {
// Entropy fields
pkt_timeval src2dst_last_pkt_time, dst2src_last_pkt_time, flow_last_pkt_time;
u_int16_t src2dst_pkt_len[MAX_NUM_PKTS]; /*!< array of packet appdata lengths */
pkt_timeval src2dst_pkt_time[MAX_NUM_PKTS]; /*!< array of arrival times */
u_int16_t dst2src_pkt_len[MAX_NUM_PKTS]; /*!< array of packet appdata lengths */
@@ -157,10 +161,10 @@ struct ndpi_entropy {
typedef struct ndpi_flow_info {
u_int32_t flow_id;
u_int32_t hashval;
u_int32_t src_ip;
u_int32_t dst_ip;
u_int16_t src_port;
u_int16_t dst_port;
u_int32_t src_ip; /* network order */
u_int32_t dst_ip; /* network order */
u_int16_t src_port; /* network order */
u_int16_t dst_port; /* network order */
u_int8_t detection_completed, protocol, bidirectional, check_extra_packets;
u_int16_t vlan_id;
ndpi_packet_tunnel tunnel_type;
@@ -185,49 +189,55 @@ typedef struct ndpi_flow_info {
// result only, not used for flow identification
ndpi_protocol detected_protocol;
ndpi_confidence_t confidence;
// Flow data analysis
pkt_timeval src2dst_last_pkt_time, dst2src_last_pkt_time, flow_last_pkt_time;
struct ndpi_analyze_struct *iat_c_to_s, *iat_s_to_c, *iat_flow,
*pktlen_c_to_s, *pktlen_s_to_c;
char info[160];
char info[255];
char flow_extra_info[16];
char host_server_name[240];
char bittorent_hash[41];
char dhcp_fingerprint[48];
char host_server_name[80]; /* Hostname/SNI */
char *bittorent_hash;
char *dhcp_fingerprint;
char *dhcp_class_ident;
ndpi_risk risk;
struct {
u_int16_t ssl_version;
char client_requested_server_name[64], server_info[64],
char server_info[64],
client_hassh[33], server_hassh[33], *server_names,
*tls_alpn, *tls_supported_versions,
*tls_issuerDN, *tls_subjectDN,
ja3_client[33], ja3_server[33],
sha1_cert_fingerprint[20];
u_int8_t sha1_cert_fingerprint_set;
struct tls_heuristics browser_heuristics;
struct {
u_int16_t cipher_suite;
char *esni;
} encrypted_sni;
time_t notBefore, notAfter;
u_int16_t server_cipher;
ndpi_cipher_weakness client_unsafe_cipher, server_unsafe_cipher;
} ssh_tls;
struct {
char url[256], content_type[64], user_agent[128];
char url[256], request_content_type[64], content_type[64], user_agent[256];
u_int response_status_code;
} http;
struct {
char username[32], password[32];
char *username, *password;
} telnet;
void *src_id, *dst_id;
struct ndpi_entropy entropy;
struct ndpi_entropy last_entropy;
struct ndpi_entropy *entropy;
struct ndpi_entropy *last_entropy;
/* Payload lenght bins */
#ifdef DIRECTION_BINS
@@ -248,10 +258,13 @@ typedef struct ndpi_stats {
u_int64_t protocol_counter_bytes[NDPI_MAX_SUPPORTED_PROTOCOLS + NDPI_MAX_NUM_CUSTOM_PROTOCOLS + 1];
u_int32_t protocol_flows[NDPI_MAX_SUPPORTED_PROTOCOLS + NDPI_MAX_NUM_CUSTOM_PROTOCOLS + 1];
u_int32_t ndpi_flow_count;
u_int32_t flow_count[3];
u_int64_t tcp_count, udp_count;
u_int64_t mpls_count, pppoe_count, vlan_count, fragmented_count;
u_int64_t packet_len[6];
u_int16_t max_packet_len;
u_int64_t dpi_packet_count[3];
u_int64_t flow_confidence[NDPI_CONFIDENCE_MAX];
} ndpi_stats_t;
@@ -259,6 +272,7 @@ typedef struct ndpi_stats {
typedef struct ndpi_workflow_prefs {
u_int8_t decode_tunnels;
u_int8_t quiet_mode;
u_int8_t ignore_vlanid;
u_int32_t num_roots;
u_int32_t max_ndpi_flows;
} ndpi_workflow_prefs_t;
@@ -288,14 +302,14 @@ typedef struct ndpi_workflow {
void **ndpi_flows_root;
struct ndpi_detection_module_struct *ndpi_struct;
u_int32_t num_allocated_flows;
} ndpi_workflow_t;
} ndpi_workflow_t;
/* TODO: remove wrappers parameters and use ndpi global, when their initialization will be fixed... */
struct ndpi_workflow * ndpi_workflow_init(const struct ndpi_workflow_prefs * prefs, pcap_t * pcap_handle);
/* workflow main free function */
/* workflow main free function */
void ndpi_workflow_free(struct ndpi_workflow * workflow);
@@ -310,8 +324,10 @@ void ndpi_free_flow_info_half(struct ndpi_flow_info *flow);
struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
const struct pcap_pkthdr *header,
const u_char *packet,
FILE * csv_fp);
ndpi_risk *flow_risk,
FILE * csv_fp);
int ndpi_is_datalink_supported(int datalink_type);
/* flow callbacks for complete detected flow
(ndpi_flow_info will be freed right after) */
@@ -327,7 +343,7 @@ static inline void ndpi_workflow_set_flow_giveup_callback(struct ndpi_workflow *
workflow->__flow_giveup_udata = udata;
}
/* compare two nodes in workflow */
/* compare two nodes in workflow */
int ndpi_workflow_node_cmp(const void *a, const void *b);
void process_ndpi_collected_info(struct ndpi_workflow * workflow, struct ndpi_flow_info *flow, FILE * csv_fp);
u_int32_t ethernet_crc32(const void* data, size_t n_bytes);
@@ -339,13 +355,13 @@ float ndpi_flow_get_byte_count_entropy(const uint32_t byte_count[256], unsigned
extern int nDPI_LogLevel;
#ifdef NDPI_ENABLE_DEBUG_MESSAGES
#define LOG(log_level, args...) \
{ \
if(log_level <= nDPI_LogLevel) \
printf(args); \
#define LOG(log_level, args...) \
{ \
if(log_level <= nDPI_LogLevel) \
printf(args); \
}
#else
#define LOG(...) {}
#define LOG(...) {}
#endif
#endif
+11
View File
@@ -0,0 +1,11 @@
# https://www.spamhaus.org/statistics/tlds/
.fail
.viajes
.exposed
.london
.work
.gq
.ml
.surk
.tk
.xyz
File diff suppressed because it is too large Load Diff
+13 -4
View File
@@ -1,4 +1,4 @@
bin_PROGRAMS = fuzz_process_packet fuzz_ndpi_reader fuzz_ndpi_reader_with_main
bin_PROGRAMS = fuzz_process_packet fuzz_process_packet_with_main fuzz_ndpi_reader fuzz_ndpi_reader_with_main
fuzz_process_packet_SOURCES = fuzz_process_packet.c
fuzz_process_packet_CFLAGS =
@@ -13,10 +13,19 @@ fuzz_process_packet_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_process_packet_LDFLAGS) $(LDFLAGS) -o $@
fuzz_process_packet_with_main_SOURCES = fuzz_process_packet.c
fuzz_process_packet_with_main_CFLAGS = -DBUILD_MAIN
fuzz_process_packet_with_main_LDADD = ../src/lib/libndpi.a
fuzz_process_packet_with_main_LDFLAGS = $(ADDITIONAL_LIBS) $(LIBS)
# force usage of CXX for linker
fuzz_process_packet_with_main_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_process_packet_with_main_LDFLAGS) $(LDFLAGS) -o $@
fuzz_ndpi_reader_SOURCES = fuzz_ndpi_reader.c
fuzz_ndpi_reader_CFLAGS = -I../example/
fuzz_ndpi_reader_LDADD = ../src/lib/libndpi.a
fuzz_ndpi_reader_LDFLAGS = ../example/libndpiReader.a $(PCAP_LIB) $(ADDITIONAL_LIBS) $(LIBS)
fuzz_ndpi_reader_LDADD = ../example/libndpiReader.a ../src/lib/libndpi.a
fuzz_ndpi_reader_LDFLAGS = $(PCAP_LIB) $(ADDITIONAL_LIBS) $(LIBS)
if HAS_FUZZLDFLAGS
fuzz_ndpi_reader_CFLAGS += $(LIB_FUZZING_ENGINE)
fuzz_ndpi_reader_LDFLAGS += $(LIB_FUZZING_ENGINE)
@@ -37,7 +46,7 @@ fuzz_ndpi_reader_with_main_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS
# required for Google oss-fuzz
# see https://github.com/google/oss-fuzz/tree/master/projects/ndpi
testpcaps := $(wildcard ../tests/pcap/*.pcap)
testpcaps := $(wildcard ../tests/pcap/*.pcap*)
fuzz_ndpi_reader_seed_corpus.zip: $(testpcaps)
zip -r fuzz_ndpi_reader_seed_corpus.zip $(testpcaps)
+16 -4
View File
@@ -13,7 +13,7 @@ int nDPI_LogLevel = 0;
char *_debug_protocols = NULL;
u_int32_t current_ndpi_memory = 0, max_ndpi_memory = 0;
u_int8_t enable_protocol_guess = 1, enable_payload_analyzer = 0;
u_int8_t enable_joy_stats = 0;
u_int8_t enable_flow_stats = 0;
u_int8_t human_readeable_string_len = 5;
u_int8_t max_num_udp_dissected_pkts = 16 /* 8 is enough for most protocols, Signal requires more */, max_num_tcp_dissected_pkts = 80 /* due to telnet */;
@@ -66,6 +66,14 @@ int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
free(pcap_path);
return 0;
}
if (ndpi_is_datalink_supported(pcap_datalink(pkts)) == 0)
{
/* Do not fail if the datalink type is not supported (may happen often during fuzzing). */
pcap_close(pkts);
remove(pcap_path);
free(pcap_path);
return 0;
}
struct ndpi_workflow * workflow = ndpi_workflow_init(prefs, pkts);
// enable all protocols
NDPI_BITMASK_SET_ALL(all);
@@ -76,8 +84,9 @@ int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
sizeof(workflow->stats.protocol_counter_bytes));
memset(workflow->stats.protocol_flows, 0,
sizeof(workflow->stats.protocol_flows));
ndpi_finalize_initalization(workflow->ndpi_struct);
ndpi_finalize_initialization(workflow->ndpi_struct);
header = NULL;
r = pcap_next_ex(pkts, &header, &pkt);
while (r > 0) {
if(header->caplen >= 42 /* ARP+ size */) {
@@ -85,8 +94,10 @@ int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
uint8_t *packet_checked = malloc(header->caplen);
if(packet_checked) {
ndpi_risk flow_risk;
memcpy(packet_checked, pkt, header->caplen);
ndpi_workflow_process_packet(workflow, header, packet_checked, NULL);
ndpi_workflow_process_packet(workflow, header, packet_checked, &flow_risk, NULL);
free(packet_checked);
}
}
@@ -148,9 +159,10 @@ int main(int argc, char ** argv)
return 1;
}
if (fread(pcap_buffer, sizeof(*pcap_buffer), pcap_file_size, pcap_file) != pcap_file_size) {
if (fread(pcap_buffer, sizeof(*pcap_buffer), pcap_file_size, pcap_file) != (size_t)pcap_file_size) {
perror("fread failed");
fclose(pcap_file);
free(pcap_buffer);
return 1;
}
+71 -9
View File
@@ -4,25 +4,87 @@
#include <stdio.h>
struct ndpi_detection_module_struct *ndpi_info_mod = NULL;
struct ndpi_id_struct *src;
struct ndpi_id_struct *dst;
int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
uint8_t protocol_was_guessed;
if (ndpi_info_mod == NULL) {
ndpi_info_mod = ndpi_init_detection_module(ndpi_no_prefs);
NDPI_PROTOCOL_BITMASK all;
ndpi_info_mod = ndpi_init_detection_module(ndpi_enable_ja3_plus);
NDPI_PROTOCOL_BITMASK all, debug_bitmask;
NDPI_BITMASK_SET_ALL(all);
NDPI_BITMASK_SET_ALL(debug_bitmask);
ndpi_set_protocol_detection_bitmask2(ndpi_info_mod, &all);
src = ndpi_malloc(SIZEOF_ID_STRUCT);
dst = ndpi_malloc(SIZEOF_ID_STRUCT);
ndpi_set_log_level(ndpi_info_mod, 4);
ndpi_set_debug_bitmask(ndpi_info_mod, debug_bitmask);
ndpi_finalize_initialization(ndpi_info_mod);
}
struct ndpi_flow_struct *ndpi_flow = ndpi_flow_malloc(SIZEOF_FLOW_STRUCT);
memset(ndpi_flow, 0, SIZEOF_FLOW_STRUCT);
memset(src, 0, SIZEOF_ID_STRUCT);
memset(dst, 0, SIZEOF_ID_STRUCT);
ndpi_detection_process_packet(ndpi_info_mod, ndpi_flow, Data, Size, 0, src, dst);
ndpi_detection_process_packet(ndpi_info_mod, ndpi_flow, Data, Size, 0);
ndpi_detection_giveup(ndpi_info_mod, ndpi_flow, 1, &protocol_was_guessed);
ndpi_free_flow(ndpi_flow);
return 0;
}
#ifdef BUILD_MAIN
int main(int argc, char ** argv)
{
FILE * data_file;
long data_file_size;
uint8_t * data_buffer;
int test_retval;
if (argc != 2) {
fprintf(stderr, "usage: %s: [data-file]\n",
(argc > 0 ? argv[0] : "fuzz_process_packet_with_main"));
return 1;
}
data_file = fopen(argv[1], "r");
if (data_file == NULL) {
perror("fopen failed");
return 1;
}
if (fseek(data_file, 0, SEEK_END) != 0) {
perror("fseek(SEEK_END) failed");
fclose(data_file);
return 1;
}
data_file_size = ftell(data_file);
if (data_file_size < 0) {
perror("ftell failed");
fclose(data_file);
return 1;
}
if (fseek(data_file, 0, SEEK_SET) != 0) {
perror("fseek(0, SEEK_SET) failed");
fclose(data_file);
return 1;
}
data_buffer = malloc(data_file_size);
if (data_buffer == NULL) {
perror("malloc failed");
fclose(data_file);
return 1;
}
if (fread(data_buffer, sizeof(*data_buffer), data_file_size, data_file) != (size_t)data_file_size) {
perror("fread failed");
fclose(data_file);
free(data_buffer);
return 1;
}
test_retval = LLVMFuzzerTestOneInput(data_buffer, data_file_size);
fclose(data_file);
free(data_buffer);
return test_retval;
}
#endif
+2
View File
@@ -0,0 +1,2 @@
queries:
- exclude: cpp/cleartext-storage-file
+153 -115
View File
@@ -1,5 +1,5 @@
# ===========================================================================
# http://www.gnu.org/software/autoconf-archive/ax_pthread.html
# https://www.gnu.org/software/autoconf-archive/ax_pthread.html
# ===========================================================================
#
# SYNOPSIS
@@ -14,20 +14,24 @@
# flags that are needed. (The user can also force certain compiler
# flags/libs to be tested by setting these environment variables.)
#
# Also sets PTHREAD_CC to any special C compiler that is needed for
# multi-threaded programs (defaults to the value of CC otherwise). (This
# is necessary on AIX to use the special cc_r compiler alias.)
# Also sets PTHREAD_CC and PTHREAD_CXX to any special C compiler that is
# needed for multi-threaded programs (defaults to the value of CC
# respectively CXX otherwise). (This is necessary on e.g. AIX to use the
# special cc_r/CC_r compiler alias.)
#
# NOTE: You are assumed to not only compile your program with these flags,
# but also to link with them as well. For example, you might link with
# $PTHREAD_CC $CFLAGS $PTHREAD_CFLAGS $LDFLAGS ... $PTHREAD_LIBS $LIBS
# $PTHREAD_CXX $CXXFLAGS $PTHREAD_CFLAGS $LDFLAGS ... $PTHREAD_LIBS $LIBS
#
# If you are only building threaded programs, you may wish to use these
# variables in your default LIBS, CFLAGS, and CC:
#
# LIBS="$PTHREAD_LIBS $LIBS"
# CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
# CXXFLAGS="$CXXFLAGS $PTHREAD_CFLAGS"
# CC="$PTHREAD_CC"
# CXX="$PTHREAD_CXX"
#
# In addition, if the PTHREAD_CREATE_JOINABLE thread-attribute constant
# has a nonstandard name, this macro defines PTHREAD_CREATE_JOINABLE to
@@ -55,6 +59,7 @@
#
# Copyright (c) 2008 Steven G. Johnson <stevenj@alum.mit.edu>
# Copyright (c) 2011 Daniel Richard G. <skunk@iSKUNK.ORG>
# Copyright (c) 2019 Marc Stevens <marc.stevens@cwi.nl>
#
# This program is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
@@ -67,7 +72,7 @@
# Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program. If not, see <http://www.gnu.org/licenses/>.
# with this program. If not, see <https://www.gnu.org/licenses/>.
#
# As a special exception, the respective Autoconf Macro's copyright owner
# gives unlimited permission to copy, distribute and modify the configure
@@ -82,7 +87,7 @@
# modified version of the Autoconf Macro, you may extend this special
# exception to the GPL to apply to your modified version as well.
#serial 23
#serial 31
AU_ALIAS([ACX_PTHREAD], [AX_PTHREAD])
AC_DEFUN([AX_PTHREAD], [
@@ -104,6 +109,7 @@ if test "x$PTHREAD_CFLAGS$PTHREAD_LIBS" != "x"; then
ax_pthread_save_CFLAGS="$CFLAGS"
ax_pthread_save_LIBS="$LIBS"
AS_IF([test "x$PTHREAD_CC" != "x"], [CC="$PTHREAD_CC"])
AS_IF([test "x$PTHREAD_CXX" != "x"], [CXX="$PTHREAD_CXX"])
CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
LIBS="$PTHREAD_LIBS $LIBS"
AC_MSG_CHECKING([for pthread_join using $CC $PTHREAD_CFLAGS $PTHREAD_LIBS])
@@ -123,10 +129,12 @@ fi
# (e.g. DEC) have both -lpthread and -lpthreads, where one of the
# libraries is broken (non-POSIX).
# Create a list of thread flags to try. Items starting with a "-" are
# C compiler flags, and other items are library names, except for "none"
# which indicates that we try without any flags at all, and "pthread-config"
# which is a program returning the flags for the Pth emulation library.
# Create a list of thread flags to try. Items with a "," contain both
# C compiler flags (before ",") and linker flags (after ","). Other items
# starting with a "-" are C compiler flags, and remaining items are
# library names, except for "none" which indicates that we try without
# any flags at all, and "pthread-config" which is a program returning
# the flags for the Pth emulation library.
ax_pthread_flags="pthreads none -Kthread -pthread -pthreads -mthreads pthread --thread-safe -mt pthread-config"
@@ -194,14 +202,47 @@ case $host_os in
# that too in a future libc.) So we'll check first for the
# standard Solaris way of linking pthreads (-mt -lpthread).
ax_pthread_flags="-mt,pthread pthread $ax_pthread_flags"
ax_pthread_flags="-mt,-lpthread pthread $ax_pthread_flags"
;;
esac
# Are we compiling with Clang?
AC_CACHE_CHECK([whether $CC is Clang],
[ax_cv_PTHREAD_CLANG],
[ax_cv_PTHREAD_CLANG=no
# Note that Autoconf sets GCC=yes for Clang as well as GCC
if test "x$GCC" = "xyes"; then
AC_EGREP_CPP([AX_PTHREAD_CC_IS_CLANG],
[/* Note: Clang 2.7 lacks __clang_[a-z]+__ */
# if defined(__clang__) && defined(__llvm__)
AX_PTHREAD_CC_IS_CLANG
# endif
],
[ax_cv_PTHREAD_CLANG=yes])
fi
])
ax_pthread_clang="$ax_cv_PTHREAD_CLANG"
# GCC generally uses -pthread, or -pthreads on some platforms (e.g. SPARC)
# Note that for GCC and Clang -pthread generally implies -lpthread,
# except when -nostdlib is passed.
# This is problematic using libtool to build C++ shared libraries with pthread:
# [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25460
# [2] https://bugzilla.redhat.com/show_bug.cgi?id=661333
# [3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=468555
# To solve this, first try -pthread together with -lpthread for GCC
AS_IF([test "x$GCC" = "xyes"],
[ax_pthread_flags="-pthread -pthreads $ax_pthread_flags"])
[ax_pthread_flags="-pthread,-lpthread -pthread -pthreads $ax_pthread_flags"])
# Clang takes -pthread (never supported any other flag), but we'll try with -lpthread first
AS_IF([test "x$ax_pthread_clang" = "xyes"],
[ax_pthread_flags="-pthread,-lpthread -pthread"])
# The presence of a feature test macro requesting re-entrant function
# definitions is, on some systems, a strong hint that pthreads support is
@@ -224,101 +265,6 @@ AS_IF([test "x$ax_pthread_check_macro" = "x--"],
[ax_pthread_check_cond=0],
[ax_pthread_check_cond="!defined($ax_pthread_check_macro)"])
# Are we compiling with Clang?
AC_CACHE_CHECK([whether $CC is Clang],
[ax_cv_PTHREAD_CLANG],
[ax_cv_PTHREAD_CLANG=no
# Note that Autoconf sets GCC=yes for Clang as well as GCC
if test "x$GCC" = "xyes"; then
AC_EGREP_CPP([AX_PTHREAD_CC_IS_CLANG],
[/* Note: Clang 2.7 lacks __clang_[a-z]+__ */
# if defined(__clang__) && defined(__llvm__)
AX_PTHREAD_CC_IS_CLANG
# endif
],
[ax_cv_PTHREAD_CLANG=yes])
fi
])
ax_pthread_clang="$ax_cv_PTHREAD_CLANG"
ax_pthread_clang_warning=no
# Clang needs special handling, because older versions handle the -pthread
# option in a rather... idiosyncratic way
if test "x$ax_pthread_clang" = "xyes"; then
# Clang takes -pthread; it has never supported any other flag
# (Note 1: This will need to be revisited if a system that Clang
# supports has POSIX threads in a separate library. This tends not
# to be the way of modern systems, but it's conceivable.)
# (Note 2: On some systems, notably Darwin, -pthread is not needed
# to get POSIX threads support; the API is always present and
# active. We could reasonably leave PTHREAD_CFLAGS empty. But
# -pthread does define _REENTRANT, and while the Darwin headers
# ignore this macro, third-party headers might not.)
PTHREAD_CFLAGS="-pthread"
PTHREAD_LIBS=
ax_pthread_ok=yes
# However, older versions of Clang make a point of warning the user
# that, in an invocation where only linking and no compilation is
# taking place, the -pthread option has no effect ("argument unused
# during compilation"). They expect -pthread to be passed in only
# when source code is being compiled.
#
# Problem is, this is at odds with the way Automake and most other
# C build frameworks function, which is that the same flags used in
# compilation (CFLAGS) are also used in linking. Many systems
# supported by AX_PTHREAD require exactly this for POSIX threads
# support, and in fact it is often not straightforward to specify a
# flag that is used only in the compilation phase and not in
# linking. Such a scenario is extremely rare in practice.
#
# Even though use of the -pthread flag in linking would only print
# a warning, this can be a nuisance for well-run software projects
# that build with -Werror. So if the active version of Clang has
# this misfeature, we search for an option to squash it.
AC_CACHE_CHECK([whether Clang needs flag to prevent "argument unused" warning when linking with -pthread],
[ax_cv_PTHREAD_CLANG_NO_WARN_FLAG],
[ax_cv_PTHREAD_CLANG_NO_WARN_FLAG=unknown
# Create an alternate version of $ac_link that compiles and
# links in two steps (.c -> .o, .o -> exe) instead of one
# (.c -> exe), because the warning occurs only in the second
# step
ax_pthread_save_ac_link="$ac_link"
ax_pthread_sed='s/conftest\.\$ac_ext/conftest.$ac_objext/g'
ax_pthread_link_step=`$as_echo "$ac_link" | sed "$ax_pthread_sed"`
ax_pthread_2step_ac_link="($ac_compile) && (echo ==== >&5) && ($ax_pthread_link_step)"
ax_pthread_save_CFLAGS="$CFLAGS"
for ax_pthread_try in '' -Qunused-arguments -Wno-unused-command-line-argument unknown; do
AS_IF([test "x$ax_pthread_try" = "xunknown"], [break])
CFLAGS="-Werror -Wunknown-warning-option $ax_pthread_try -pthread $ax_pthread_save_CFLAGS"
ac_link="$ax_pthread_save_ac_link"
AC_LINK_IFELSE([AC_LANG_SOURCE([[int main(void){return 0;}]])],
[ac_link="$ax_pthread_2step_ac_link"
AC_LINK_IFELSE([AC_LANG_SOURCE([[int main(void){return 0;}]])],
[break])
])
done
ac_link="$ax_pthread_save_ac_link"
CFLAGS="$ax_pthread_save_CFLAGS"
AS_IF([test "x$ax_pthread_try" = "x"], [ax_pthread_try=no])
ax_cv_PTHREAD_CLANG_NO_WARN_FLAG="$ax_pthread_try"
])
case "$ax_cv_PTHREAD_CLANG_NO_WARN_FLAG" in
no | unknown) ;;
*) PTHREAD_CFLAGS="$ax_cv_PTHREAD_CLANG_NO_WARN_FLAG $PTHREAD_CFLAGS" ;;
esac
fi # $ax_pthread_clang = yes
if test "x$ax_pthread_ok" = "xno"; then
for ax_pthread_try_flag in $ax_pthread_flags; do
@@ -328,10 +274,10 @@ for ax_pthread_try_flag in $ax_pthread_flags; do
AC_MSG_CHECKING([whether pthreads work without any flags])
;;
-mt,pthread)
AC_MSG_CHECKING([whether pthreads work with -mt -lpthread])
PTHREAD_CFLAGS="-mt"
PTHREAD_LIBS="-lpthread"
*,*)
PTHREAD_CFLAGS=`echo $ax_pthread_try_flag | sed "s/^\(.*\),\(.*\)$/\1/"`
PTHREAD_LIBS=`echo $ax_pthread_try_flag | sed "s/^\(.*\),\(.*\)$/\2/"`
AC_MSG_CHECKING([whether pthreads work with "$PTHREAD_CFLAGS" and "$PTHREAD_LIBS"])
;;
-*)
@@ -371,7 +317,13 @@ for ax_pthread_try_flag in $ax_pthread_flags; do
# if $ax_pthread_check_cond
# error "$ax_pthread_check_macro must be defined"
# endif
static void routine(void *a) { a = 0; }
static void *some_global = NULL;
static void routine(void *a)
{
/* To avoid any unused-parameter or
unused-but-set-parameter warning. */
some_global = a;
}
static void *start_routine(void *a) { return a; }],
[pthread_t th; pthread_attr_t attr;
pthread_create(&th, 0, start_routine, 0);
@@ -393,6 +345,80 @@ for ax_pthread_try_flag in $ax_pthread_flags; do
done
fi
# Clang needs special handling, because older versions handle the -pthread
# option in a rather... idiosyncratic way
if test "x$ax_pthread_clang" = "xyes"; then
# Clang takes -pthread; it has never supported any other flag
# (Note 1: This will need to be revisited if a system that Clang
# supports has POSIX threads in a separate library. This tends not
# to be the way of modern systems, but it's conceivable.)
# (Note 2: On some systems, notably Darwin, -pthread is not needed
# to get POSIX threads support; the API is always present and
# active. We could reasonably leave PTHREAD_CFLAGS empty. But
# -pthread does define _REENTRANT, and while the Darwin headers
# ignore this macro, third-party headers might not.)
# However, older versions of Clang make a point of warning the user
# that, in an invocation where only linking and no compilation is
# taking place, the -pthread option has no effect ("argument unused
# during compilation"). They expect -pthread to be passed in only
# when source code is being compiled.
#
# Problem is, this is at odds with the way Automake and most other
# C build frameworks function, which is that the same flags used in
# compilation (CFLAGS) are also used in linking. Many systems
# supported by AX_PTHREAD require exactly this for POSIX threads
# support, and in fact it is often not straightforward to specify a
# flag that is used only in the compilation phase and not in
# linking. Such a scenario is extremely rare in practice.
#
# Even though use of the -pthread flag in linking would only print
# a warning, this can be a nuisance for well-run software projects
# that build with -Werror. So if the active version of Clang has
# this misfeature, we search for an option to squash it.
AC_CACHE_CHECK([whether Clang needs flag to prevent "argument unused" warning when linking with -pthread],
[ax_cv_PTHREAD_CLANG_NO_WARN_FLAG],
[ax_cv_PTHREAD_CLANG_NO_WARN_FLAG=unknown
# Create an alternate version of $ac_link that compiles and
# links in two steps (.c -> .o, .o -> exe) instead of one
# (.c -> exe), because the warning occurs only in the second
# step
ax_pthread_save_ac_link="$ac_link"
ax_pthread_sed='s/conftest\.\$ac_ext/conftest.$ac_objext/g'
ax_pthread_link_step=`AS_ECHO(["$ac_link"]) | sed "$ax_pthread_sed"`
ax_pthread_2step_ac_link="($ac_compile) && (echo ==== >&5) && ($ax_pthread_link_step)"
ax_pthread_save_CFLAGS="$CFLAGS"
for ax_pthread_try in '' -Qunused-arguments -Wno-unused-command-line-argument unknown; do
AS_IF([test "x$ax_pthread_try" = "xunknown"], [break])
CFLAGS="-Werror -Wunknown-warning-option $ax_pthread_try -pthread $ax_pthread_save_CFLAGS"
ac_link="$ax_pthread_save_ac_link"
AC_LINK_IFELSE([AC_LANG_SOURCE([[int main(void){return 0;}]])],
[ac_link="$ax_pthread_2step_ac_link"
AC_LINK_IFELSE([AC_LANG_SOURCE([[int main(void){return 0;}]])],
[break])
])
done
ac_link="$ax_pthread_save_ac_link"
CFLAGS="$ax_pthread_save_CFLAGS"
AS_IF([test "x$ax_pthread_try" = "x"], [ax_pthread_try=no])
ax_cv_PTHREAD_CLANG_NO_WARN_FLAG="$ax_pthread_try"
])
case "$ax_cv_PTHREAD_CLANG_NO_WARN_FLAG" in
no | unknown) ;;
*) PTHREAD_CFLAGS="$ax_cv_PTHREAD_CLANG_NO_WARN_FLAG $PTHREAD_CFLAGS" ;;
esac
fi # $ax_pthread_clang = yes
# Various other checks:
if test "x$ax_pthread_ok" = "xyes"; then
ax_pthread_save_CFLAGS="$CFLAGS"
@@ -438,7 +464,8 @@ if test "x$ax_pthread_ok" = "xyes"; then
AC_CACHE_CHECK([for PTHREAD_PRIO_INHERIT],
[ax_cv_PTHREAD_PRIO_INHERIT],
[AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include <pthread.h>]],
[[int i = PTHREAD_PRIO_INHERIT;]])],
[[int i = PTHREAD_PRIO_INHERIT;
return i;]])],
[ax_cv_PTHREAD_PRIO_INHERIT=yes],
[ax_cv_PTHREAD_PRIO_INHERIT=no])
])
@@ -460,18 +487,28 @@ if test "x$ax_pthread_ok" = "xyes"; then
[#handle absolute path differently from PATH based program lookup
AS_CASE(["x$CC"],
[x/*],
[AS_IF([AS_EXECUTABLE_P([${CC}_r])],[PTHREAD_CC="${CC}_r"])],
[AC_CHECK_PROGS([PTHREAD_CC],[${CC}_r],[$CC])])])
[
AS_IF([AS_EXECUTABLE_P([${CC}_r])],[PTHREAD_CC="${CC}_r"])
AS_IF([test "x${CXX}" != "x"], [AS_IF([AS_EXECUTABLE_P([${CXX}_r])],[PTHREAD_CXX="${CXX}_r"])])
],
[
AC_CHECK_PROGS([PTHREAD_CC],[${CC}_r],[$CC])
AS_IF([test "x${CXX}" != "x"], [AC_CHECK_PROGS([PTHREAD_CXX],[${CXX}_r],[$CXX])])
]
)
])
;;
esac
fi
fi
test -n "$PTHREAD_CC" || PTHREAD_CC="$CC"
test -n "$PTHREAD_CXX" || PTHREAD_CXX="$CXX"
AC_SUBST([PTHREAD_LIBS])
AC_SUBST([PTHREAD_CFLAGS])
AC_SUBST([PTHREAD_CC])
AC_SUBST([PTHREAD_CXX])
# Finally, execute ACTION-IF-FOUND/ACTION-IF-NOT-FOUND:
if test "x$ax_pthread_ok" = "xyes"; then
@@ -483,3 +520,4 @@ else
fi
AC_LANG_POP
])dnl AX_PTHREAD
+1 -1
View File
@@ -1 +1 @@
/usr/local/lib
/usr/lib
+58 -29
View File
@@ -1,57 +1,86 @@
#
# Copyright (C) 2018-20 - ntop.org
# Copyright (C) 2017 OpenWrt.org
#
# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#
include $(TOPDIR)/rules.mk
PKG_NAME:=libndpi
PKG_VERSION:=17022020
PKG_VERSION:=4.0
PKG_RELEASE:=1
PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz
PKG_SOURCE_URL:=https://github.com/ntop/nDPI.git
PKG_SOURCE_VERSION:=1f921562d1d7962f1d23ca5b59c25f9b65073460
PKG_SOURCE_SUBDIR:=$(PKG_NAME)-$(PKG_VERSION)
PKG_SOURCE_PROTO:=git
PKG_SOURCE_URL:=https://codeload.github.com/ntop/nDPI/tar.gz/$(PKG_VERSION)?
PKG_HASH:=dc9b291c7fde94edb45fb0f222e0d93c93f8d6d37f4efba20ebd9c655bfcedf9
PKG_BUILD_DIR:=$(BUILD_DIR)/nDPI-$(PKG_VERSION)
PKG_MAINTAINER:=Luca Deri <deri@ntop.org>
PKG_LICENSE:=GPL3
PKG_BUILD_DEPENDS:=
PKG_MAINTAINER:=Banglang Huang <banglang.huang@foxmail.com>, Toni Uhlig <matzeton@googlemail.com>
PKG_LICENSE:=LGPL-3.0-or-later
PKG_LICENSE_FILES:=COPYING
PKG_FIXUP:=autoreconf
PKG_REMOVE_FILES:=autogen.sh
PKG_BUILD_DEPENDS:=libpcap
PKG_BUILD_PARALLEL:=1
# autogen fix
PKG_FIXUP:=autoreconf
include $(INCLUDE_DIR)/uclibc++.mk
include $(INCLUDE_DIR)/package.mk
ifeq ($(CONFIG_LIBNDPI_GCRYPT),)
CONFIGURE_ARGS += --disable-gcrypt
endif
define Package/libndpi
SECTION:=network
CATEGORY:=Network
TITLE:=nDPI Deep Packet Inspection Library
URL:=https://www.ntop.org
SECTION:=libs
CATEGORY:=Libraries
TITLE:=Library for deep-packet inspection
URL:=https://github.com/ntop/nDPI
DEPENDS:=$(CXX_DEPENDS) +LIBNDPI_GCRYPT:libgcrypt +libpcap +libjson-c
endef
define Package/libndpi/description
Open and Extensible GPLv3 Deep Packet Inspection Library
nDPI is an open source LGPLv3 library for deep-packet inspection.
Based on OpenDPI it includes ntop extensions.
endef
CONFIGURE_ARGS += \
--with-only-libndpi
define Package/libndpi/config
config LIBNDPI_GCRYPT
bool "GCrypt support"
depends on PACKAGE_libndpi
default n
help
This option enables QUIC client hello decryption.
Disabled by default.
endef
define Build/Prepare
$(call Build/Prepare/Default)
endef
define Build/Configure
( cd $(PKG_BUILD_DIR); ./autogen.sh )
$(call Build/Configure/Default)
$(PKG_UNPACK)
$(Build/Patch)
endef
define Build/InstallDev
$(INSTALL_DIR) $(STAGING_DIR)/usr/local/include/libndpi
$(CP) $(PKG_BUILD_DIR)/src/include/* $(STAGING_DIR)/usr/local/include/libndpi
$(INSTALL_DIR) $(STAGING_DIR)/usr/local/lib
$(CP) $(PKG_BUILD_DIR)/src/lib/libndpi.* $(STAGING_DIR)/usr/local/lib
$(INSTALL_DIR) $(1)/usr/include/ndpi
$(CP) $(PKG_BUILD_DIR)/src/include/*.h \
$(1)/usr/include/ndpi/
$(INSTALL_DIR) $(1)/usr/lib
$(CP) $(PKG_BUILD_DIR)/src/lib/libndpi.so* \
$(1)/usr/lib/
$(INSTALL_DIR) $(1)/usr/lib/pkgconfig
$(CP) $(PKG_BUILD_DIR)/libndpi.pc \
$(1)/usr/lib/pkgconfig/
endef
define Package/libndpi/install
$(INSTALL_DIR) $(1)/usr/lib/
$(CP) $(PKG_BUILD_DIR)/src/lib/libndpi.so* \
$(1)/usr/lib/
$(INSTALL_DIR) $(1)/usr/bin/
$(CP) $(PKG_BUILD_DIR)/example/ndpiReader \
$(1)/usr/bin/
endef
$(eval $(call BuildPackage,libndpi))
+86
View File
@@ -0,0 +1,86 @@
#
# Copyright (C) 2017 OpenWrt.org
#
# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#
include $(TOPDIR)/rules.mk
PKG_NAME:=libndpi
# Avoid a specfic version
PKG_VERSION:=dev
PKG_RELEASE:=1
# Compile it from soucre
SOURCE_DIR:=/home/deri/openwrt-dependencies/nDPI/
PKG_MAINTAINER:=Luca Deri <deri@ntop.org>
PKG_LICENSE:=LGPL-3.0-or-later
PKG_LICENSE_FILES:=COPYING
PKG_FIXUP:=autoreconf
PKG_REMOVE_FILES:=autogen.sh
PKG_BUILD_DEPENDS:=libpcap
PKG_BUILD_PARALLEL:=1
include $(INCLUDE_DIR)/uclibc++.mk
include $(INCLUDE_DIR)/package.mk
ifeq ($(CONFIG_LIBNDPI_GCRYPT),)
CONFIGURE_ARGS += --disable-gcrypt
endif
define Package/libndpi
SECTION:=libs
CATEGORY:=Libraries
TITLE:=Library for deep-packet inspection
URL:=https://github.com/ntop/nDPI
DEPENDS:=$(CXX_DEPENDS) +LIBNDPI_GCRYPT:libgcrypt +libpcap +libjson-c
endef
define Package/libndpi/description
nDPI is an open source LGPLv3 library for deep-packet inspection.
Based on OpenDPI it includes ntop extensions.
endef
define Package/libndpi/config
config LIBNDPI_GCRYPT
bool "GCrypt support"
depends on PACKAGE_libndpi
default n
help
This option enables QUIC client hello decryption.
Disabled by default.
endef
define Build/Prepare
cp -r $(SOURCE_DIR)/* $(PKG_BUILD_DIR)
$(PKG_UNPACK)
$(Build/Patch)
endef
define Build/InstallDev
$(INSTALL_DIR) $(1)/usr/include/ndpi
$(CP) $(PKG_BUILD_DIR)/src/include/*.h \
$(1)/usr/include/ndpi/
$(INSTALL_DIR) $(1)/usr/lib
$(CP) $(PKG_BUILD_DIR)/src/lib/libndpi.so* \
$(1)/usr/lib/
$(INSTALL_DIR) $(1)/usr/lib/pkgconfig
$(CP) $(PKG_BUILD_DIR)/libndpi.pc \
$(1)/usr/lib/pkgconfig/
endef
define Package/libndpi/install
$(INSTALL_DIR) $(1)/usr/lib/
$(CP) $(PKG_BUILD_DIR)/src/lib/libndpi.so* \
$(1)/usr/lib/
$(INSTALL_DIR) $(1)/usr/bin/
$(CP) $(PKG_BUILD_DIR)/example/ndpiReader \
$(1)/usr/bin/
endef
$(eval $(call BuildPackage,libndpi))
+11 -4
View File
@@ -1,14 +1,20 @@
Howto Compile lindpi on OpenWRT
Howto Compile libndpi on OpenWRT
-------------------------------
cd myopenwrt_directory
mkdir package/network/services/libndpi
cd package/network/services/libndpi
cp ~/nDPI/packages/openwrt/Makefile .
# (a) Stable version
ln -s ~/nDPI/packages/openwrt/Makefile .
# (b) Dev version
cp ~/nDPI/packages/openwrt/Makefile.dev Makefile
cd myopenwrt_directory
make menuconfig
Go under network and select
Go under libraries and select
<M> libndpi.............................. nDPI Deep Packet Inspection Library
@@ -22,4 +28,5 @@ make -j1 V=s package/network/services/libndpi/compile
Other Documents
---------------
https://openwrt.org/packages/pkgdata/libndpi
https://openwrt.org/packages/pkgdata/libndpi
+1600 -2
View File
File diff suppressed because it is too large Load Diff
+8
View File
@@ -39,10 +39,18 @@ if test "$CENTOS_RELEASE" -ne 8; then
RPM_SIGN_CMD="./rpm-sign.exp"
fi
AC_CHECK_LIB([maxminddb], [MMDB_lookup_sockaddr])
AC_HAVE_HEADERS(maxminddb.h)
if test ".${ac_cv_lib_maxminddb_MMDB_lookup_sockaddr}" = ".yes" &&
test ".${ac_cv_header_maxminddb_h}" = ".yes"; then
GEOIP_DEP="libmaxminddb"
fi
AC_CONFIG_FILES([Makefile ndpi.spec])
AC_SUBST(PACKAGE_VERSION)
AC_SUBST(PLATFORM)
AC_SUBST(GEOIP_DEP)
AC_SUBST(MACHINE)
AC_SUBST(EXTN)
AC_SUBST(DATE)
+23 -17
View File
@@ -8,12 +8,16 @@ URL: http://www.ntop.org/products/deep-packet-inspection/ndpi/
Source: ndpi-%{version}.tgz
Packager: Luca Deri <deri@ntop.org>
BuildRoot: %{_tmppath}/%{name}-%{version}-root
%{?el7:Requires: glibc >= 2.3.4 numactl coreutils }
%{?el6:Requires: glibc >= 2.3.4 numactl coreutils }
%{?el8:Requires: glibc >= 2.3.4 numactl coreutils @GEOIP_DEP@ }
%{?el7:Requires: glibc >= 2.3.4 numactl coreutils @GEOIP_DEP@ }
%{?el6:Requires: glibc >= 2.3.4 numactl coreutils @GEOIP_DEP@ }
AutoReqProv: no
%define debug_package %{nil}
# Make sure .build-id is not part of the package
%define _build_id_links none
%description
nDPI Open and Extensible LGPLv3 Deep Packet Inspection Library.
@@ -43,9 +47,11 @@ you will need to install %{name}-dev.
%build
PATH=/usr/bin:/bin:/usr/sbin:/sbin
%define includedir /usr/local/include/ndpi
%define libdir /usr/local/lib
%define bindir /usr/local/bin
%define includedir /usr/include/ndpi
%{?el6:%define libdir /usr/lib}
%{?el7:%define libdir /usr/lib}
%{?el8:%define libdir /usr/lib64}
%define bindir /usr/bin
%install
PATH=/usr/bin:/bin:/usr/sbin:/sbin
@@ -56,13 +62,13 @@ mkdir -p $RPM_BUILD_ROOT%{includedir}
mkdir -p $RPM_BUILD_ROOT%{libdir}
mkdir -p $RPM_BUILD_ROOT%{bindir}
cd $HOME/nDPI; make DESTDIR=$RPM_BUILD_ROOT install;
rm $RPM_BUILD_ROOT%{libdir}/libndpi.so
rm $RPM_BUILD_ROOT%{libdir}/libndpi.so.?
cd $RPM_BUILD_ROOT%{libdir}/; ln -s libndpi.so.@PACKAGE_VERSION@ libndpi.so.@MAJOR_RELEASE@; cd -
cd $RPM_BUILD_ROOT%{libdir}/; ln -s libndpi.so.@PACKAGE_VERSION@ libndpi.so; cd -
rm $RPM_BUILD_ROOT/usr/lib/libndpi.so
rm $RPM_BUILD_ROOT/usr/lib/libndpi.so.?
cd $RPM_BUILD_ROOT/usr/lib/; ln -s libndpi.so.@PACKAGE_VERSION@ libndpi.so.@MAJOR_RELEASE@; cd -
cd $RPM_BUILD_ROOT/usr/lib/; ln -s libndpi.so.@PACKAGE_VERSION@ libndpi.so; cd -
mkdir -p $RPM_BUILD_ROOT/etc/ld.so.conf.d
cp $HOME/nDPI/packages/etc/ld.so.conf.d/ndpi.conf $RPM_BUILD_ROOT/etc/ld.so.conf.d
rm -rf $RPM_BUILD_ROOT/usr/local/sbin/ndpi
rm -rf $RPM_BUILD_ROOT/usr/sbin/ndpi
strip $RPM_BUILD_ROOT%{bindir}/*
rm -fr %{buildroot}%{includedir}/ndpi_win32.h
#sed -i -e "s/#include \"ndpi_win32.h\"//g" %{buildroot}%{includedir}/ndpi_main.h
@@ -73,12 +79,12 @@ rm -fr $RPM_BUILD_ROOT
%files
%defattr(-, root, root)
/usr/local/lib/libndpi.so
/usr/local/lib/libndpi.so.@MAJOR_RELEASE@
/usr/local/lib/libndpi.so.@PACKAGE_VERSION@
/usr/local/bin/ndpiReader
/usr/local/share/ndpi/ndpiCustomCategory.txt
/usr/local/share/ndpi/ndpiProtos.txt
/usr/lib/libndpi.so
/usr/lib/libndpi.so.@MAJOR_RELEASE@
/usr/lib/libndpi.so.@PACKAGE_VERSION@
/usr/bin/ndpiReader
/usr/share/ndpi/ndpiCustomCategory.txt
/usr/share/ndpi/ndpiProtos.txt
/etc/ld.so.conf.d/ndpi.conf
%{libdir}/pkgconfig/libndpi.pc
@@ -88,7 +94,7 @@ rm -fr $RPM_BUILD_ROOT
%defattr(-,root,root,-)
%{includedir}
%{libdir}/libndpi.a
/usr/lib/libndpi.a
%changelog
* Mon Nov 19 2018 Alfredo Cardigliano <cardigliano@ntop.org> 2.5
+16 -19
View File
@@ -1,24 +1,20 @@
#
# Change it according to your setup
#
NDPI_HOME=$(PWD)/../..
NDPI_BUILD=${NDPI_HOME}/packages/ubuntu
all: clean ndpi
NDPI_BUILD="$(realpath ${NDPI_HOME}/packages/ubuntu)"
ndpi:
\rm -rf ./debian/ndpi-tmp ./debian/ndpi-dev-tmp ./debian/ndpi ./debian/ndpi-dev
mkdir -p ./debian/ndpi-tmp ./debian/ndpi-dev-tmp
mkdir -p ./debian/ndpi-tmp/usr/local/lib ./debian/ndpi-tmp/usr/local/bin
mkdir -p ./debian/ndpi-dev-tmp/usr/local/lib ./debian/ndpi-dev-tmp/usr/local/include/ndpi
cd ${NDPI_HOME}; ./autogen.sh; ./configure; make
cp $(NDPI_HOME)/src/lib/libndpi.so.@NDPI_VERS@ ./debian/ndpi-tmp/usr/local/lib/
cd ./debian/ndpi-tmp/usr/local/lib/; ln -s libndpi.so.@NDPI_VERS@ libndpi.so; cd -
cd ./debian/ndpi-tmp/usr/local/lib/; ln -s libndpi.so.@NDPI_VERS@ libndpi.so.@MAJOR_RELEASE@; cd -
cp $(NDPI_HOME)/src/lib/libndpi.a ./debian/ndpi-dev-tmp/usr/local/lib/
cp $(NDPI_HOME)/example/ndpiReader ./debian/ndpi-tmp/usr/local/bin/
cp $(NDPI_HOME)/src/include/*.h ./debian/ndpi-dev-tmp/usr/local/include/ndpi/
-rm -fr ./debian/ndpi-dev-tmp/usr/local/include/ndpi/ndpi_win32.h*
mkdir -p ./debian/ndpi-tmp/usr/lib ./debian/ndpi-tmp/usr/bin
mkdir -p ./debian/ndpi-dev-tmp/usr/lib ./debian/ndpi-dev-tmp/usr/include/ndpi ./debian/ndpi-dev-tmp/usr/lib/pkgconfig
cd ${NDPI_HOME}; ./autogen.sh; ./configure; $(MAKE)
cp $(NDPI_HOME)/src/lib/libndpi.so.@NDPI_VERS@ ./debian/ndpi-tmp/usr/lib/
cd ./debian/ndpi-tmp/usr/lib/; ln -s libndpi.so.@NDPI_VERS@ libndpi.so; cd -
cd ./debian/ndpi-tmp/usr/lib/; ln -s libndpi.so.@NDPI_VERS@ libndpi.so.@MAJOR_RELEASE@; cd -
cp $(NDPI_HOME)/src/lib/libndpi.a ./debian/ndpi-dev-tmp/usr/lib/
cp $(NDPI_HOME)/example/ndpiReader ./debian/ndpi-tmp/usr/bin/
cp $(NDPI_HOME)/src/include/*.h ./debian/ndpi-dev-tmp/usr/include/ndpi/
cp $(NDPI_HOME)/libndpi.pc ./debian/ndpi-dev-tmp/usr/lib/pkgconfig/
-rm -fr ./debian/ndpi-dev-tmp/usr/include/ndpi/ndpi_win32.h*
@echo
@find ./debian/ndpi-tmp -name "*~" -exec /bin/rm {} ';'
@find ./debian/ndpi-dev-tmp -name "*~" -exec /bin/rm {} ';'
@@ -37,10 +33,11 @@ ndpi:
@echo "-------------------------------"
distclean:
echo "dummy distclean"
-cd ${NDPI_HOME} && $(MAKE) distclean
install:
echo "dummy install"
cd ${NDPI_HOME}; $(MAKE) install DESTDIR="${NDPI_BUILD}/debian/tmp" AM_UPDATE_INFO_DIR=no
clean:
-rm -rf *~ *deb ./usr ./debian/ndpi ./debian/ndpi-dev ./debian/ndpi-tmp ./debian/ndpi-dev-tmp
-cd ${NDPI_HOME} && $(MAKE) clean
-rm -rf *~ *deb ./usr ./debian/ndpi ./debian/ndpi-dev ./debian/ndpi ./debian/ndpi-dev
+1604 -18
View File
File diff suppressed because it is too large Load Diff
+11 -3
View File
@@ -24,13 +24,21 @@ else
fi
fi
NDPI_VERS=`../version.sh --release`
MAJOR_RELEASE=`../version.sh --major-release`
GIT_REVISION=`../version.sh --revision`
NDPI_VERS=`$(dirname "${0}")/../version.sh --release`
MAJOR_RELEASE=`$(dirname "${0}")/../version.sh --major-release`
GIT_REVISION=`$(dirname "${0}")/../version.sh --revision`
AC_CHECK_LIB([maxminddb], [MMDB_lookup_sockaddr])
AC_HAVE_HEADERS(maxminddb.h)
if test ".${ac_cv_lib_maxminddb_MMDB_lookup_sockaddr}" = ".yes" &&
test ".${ac_cv_header_maxminddb_h}" = ".yes"; then
GEOIP_DEP=", libmaxminddb0"
fi
AC_CONFIG_FILES([Makefile debian/changelog debian/files debian/control])
AC_SUBST(NDPI_VERS)
AC_SUBST(GEOIP_DEP)
AC_SUBST(MACHINE)
AC_SUBST(EXTN)
AC_SUBST(DATE)
+1 -1
View File
@@ -8,7 +8,7 @@ Build-Conflicts:
Package: ndpi
Architecture: @EXTN@
Depends: libnuma1
Depends: libnuma1 @GEOIP_DEP@
Recommends:
Suggests:
Pre-Depends:
+3 -2
View File
@@ -2,7 +2,8 @@ CC=@CC@
CFLAGS=-I. -I../src/include -I./src/lib/third_party/include @CFLAGS@ @CUSTOM_NDPI@ -shared
#LIBNDPI=../src/lib/libndpi.so.@NDPI_VERSION_SHORT@
LIBNDPI=../src/lib/libndpi.a
LDFLAGS=$(CFILE) $(LIBNDPI) -lpcap @ADDITIONAL_LIBS@
LDFLAGS=@LDFLAGS@
LIBS=$(CFILE) $(LIBNDPI) -lpcap @LIBS@ @ADDITIONAL_LIBS@
SHARE = -soname,ndpi_wrap
SO=ndpi_wrap.so
OBJS = ndpi_wrap.o
@@ -19,7 +20,7 @@ endif
all: $(SO)
$(SO): $(CFILE) $(LIBNDPI) Makefile
$(CC) $(CFLAGS) -Wl,$(SHARE) -o $@ $(PIC) $(LDFLAGS)
$(CC) $(CPPFLAGS) $(CFLAGS) $(LDFLAGS) $(PIC) -Wl,$(SHARE) -o $@ $(LIBS)
# ln -s $(LIBNDPI) .
clean:
+69 -165
View File
@@ -316,6 +316,24 @@ typedef enum {
NDPI_UNSAFE_PROTOCOL,
NDPI_DNS_SUSPICIOUS_TRAFFIC,
NDPI_TLS_MISSING_SNI,
NDPI_HTTP_SUSPICIOUS_CONTENT,
NDPI_RISKY_ASN,
NDPI_RISKY_DOMAIN,
NDPI_MALICIOUS_JA3,
NDPI_MALICIOUS_SHA1_CERTIFICATE,
NDPI_DESKTOP_OR_FILE_SHARING_SESSION,
NDPI_TLS_UNCOMMON_ALPN,
NDPI_TLS_CERT_VALIDITY_TOO_LONG,
NDPI_TLS_SUSPICIOUS_EXTENSION,
NDPI_TLS_FATAL_ALERT,
NDPI_SUSPICIOUS_ENTROPY,
NDPI_CLEAR_TEXT_CREDENTIALS,
NDPI_DNS_LARGE_PACKET,
NDPI_DNS_FRAGMENTED,
NDPI_INVALID_CHARACTERS,
NDPI_POSSIBLE_EXPLOIT,
NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE,
/* Leave this as last member */
NDPI_MAX_RISK
} ndpi_risk_enum;
@@ -426,11 +444,6 @@ struct ndpi_id_struct {
to compare this, use:
**/
NDPI_PROTOCOL_BITMASK detected_protocol_bitmask;
/* NDPI_PROTOCOL_RTSP */
ndpi_ip_addr_t rtsp_ip_address;
/* NDPI_PROTOCOL_YAHOO */
uint32_t yahoo_video_lan_timer;
/* NDPI_PROTOCOL_IRC_MAXPORT % 2 must be 0 */
/* NDPI_PROTOCOL_IRC */
@@ -442,65 +455,31 @@ struct ndpi_id_struct {
/* NDPI_PROTOCOL_GNUTELLA */
uint32_t gnutella_ts;
/* NDPI_PROTOCOL_THUNDER */
uint32_t thunder_ts;
/* NDPI_PROTOCOL_RTSP */
uint32_t rtsp_timer;
/* NDPI_PROTOCOL_ZATTOO */
uint32_t zattoo_ts;
/* NDPI_PROTOCOL_UNENCRYPTED_JABBER */
uint32_t jabber_stun_or_ft_ts;
/* NDPI_PROTOCOL_DIRECTCONNECT */
uint32_t directconnect_last_safe_access_time;
/* NDPI_PROTOCOL_SOULSEEK */
uint32_t soulseek_last_safe_access_time;
/* NDPI_PROTOCOL_DIRECTCONNECT */
uint16_t detected_directconnect_port;
uint16_t detected_directconnect_udp_port;
uint16_t detected_directconnect_ssl_port;
/* NDPI_PROTOCOL_BITTORRENT */
#define NDPI_BT_PORTS 8
uint16_t bt_port_t[NDPI_BT_PORTS];
uint16_t bt_port_u[NDPI_BT_PORTS];
/* NDPI_PROTOCOL_UNENCRYPTED_JABBER */
#define JABBER_MAX_STUN_PORTS 6
uint16_t jabber_voice_stun_port[JABBER_MAX_STUN_PORTS];
uint16_t jabber_file_transfer_port[2];
/* NDPI_PROTOCOL_GNUTELLA */
uint16_t detected_gnutella_port;
/* NDPI_PROTOCOL_GNUTELLA */
uint16_t detected_gnutella_udp_port1;
uint16_t detected_gnutella_udp_port2;
/* NDPI_PROTOCOL_SOULSEEK */
uint16_t soulseek_listen_port;
/* NDPI_PROTOCOL_IRC */
uint8_t irc_number_of_port;
/* NDPI_PROTOCOL_UNENCRYPTED_JABBER */
uint8_t jabber_voice_stun_used_ports;
/* NDPI_PROTOCOL_SIP */
/* NDPI_PROTOCOL_YAHOO */
uint32_t yahoo_video_lan_dir:1;
/* NDPI_PROTOCOL_YAHOO */
uint32_t yahoo_conf_logged_in:1;
uint32_t yahoo_voice_conf_logged_in:1;
/* NDPI_PROTOCOL_RTSP */
uint32_t rtsp_ts_set:1;
};
struct ndpi_flow_tcp_struct {
@@ -510,18 +489,11 @@ struct ndpi_flow_tcp_struct {
/* NDPI_PROTOCOL_MAIL_POP */
uint16_t pop_command_bitmask;
/* NDPI_PROTOCOL_QQ */
uint16_t qq_nxt_len;
/* NDPI_PROTOCOL_WHATSAPP */
uint8_t wa_matched_so_far;
/* NDPI_PROTOCOL_TDS */
uint8_t tds_login_version;
/* NDPI_PROTOCOL_IRC */
uint8_t irc_stage;
uint8_t irc_port;
/* NDPI_PROTOCOL_H323 */
uint8_t h323_valid_packets;
@@ -535,38 +507,16 @@ struct ndpi_flow_tcp_struct {
uint32_t irc_direction:2;
uint32_t irc_0x1000_full:1;
/* NDPI_PROTOCOL_SOULSEEK */
uint32_t soulseek_stage:2;
/* NDPI_PROTOCOL_TDS */
uint32_t tds_stage:3;
/* NDPI_PROTOCOL_USENET */
uint32_t usenet_stage:2;
/* NDPI_PROTOCOL_IMESH */
uint32_t imesh_stage:4;
/* NDPI_PROTOCOL_HTTP */
uint32_t http_setup_dir:2;
uint32_t http_stage:2;
uint32_t http_empty_line_seen:1;
uint32_t http_wait_for_retransmission:1;
/* NDPI_PROTOCOL_GNUTELLA */
uint32_t gnutella_stage:2; // 0 - 2
/* NDPI_CONTENT_MMS */
uint32_t mms_stage:2;
/* NDPI_PROTOCOL_YAHOO */
uint32_t yahoo_sip_comm:1;
uint32_t yahoo_http_proxy_stage:2;
/* NDPI_PROTOCOL_MSN */
uint32_t msn_stage:3;
uint32_t msn_ssl_ft:2;
/* NDPI_PROTOCOL_SSH */
uint32_t ssh_stage:3;
@@ -582,10 +532,8 @@ struct ndpi_flow_tcp_struct {
unsigned buffer_len, buffer_used;
} message;
void* srv_cert_fingerprint_ctx; /* SHA-1 */
/* NDPI_PROTOCOL_TLS */
uint8_t hello_processed:1, certificate_processed:1, subprotocol_detected:1, fingerprint_set:1, _pad:4;
uint8_t certificate_processed:1, fingerprint_set:1, _pad:6;
uint8_t sha1_certificate_fingerprint[20], num_tls_blocks;
int16_t tls_application_blocks_len[NDPI_MAX_NUM_TLS_APPL_BLOCKS];
} tls;
@@ -593,8 +541,7 @@ struct ndpi_flow_tcp_struct {
/* NDPI_PROTOCOL_POSTGRES */
uint32_t postgres_stage:3;
/* NDPI_PROTOCOL_DIRECT_DOWNLOAD_LINK */
uint32_t ddlink_server_direction:1;
/* Part of the TCP header. */
uint32_t seen_syn:1;
uint32_t seen_syn_ack:1;
uint32_t seen_ack:1;
@@ -611,9 +558,6 @@ struct ndpi_flow_tcp_struct {
/* NDPI_PROTOCOL_WORLDOFWARCRAFT */
uint32_t wow_stage:2;
/* NDPI_PROTOCOL_HTTP_APPLICATION_VEOHTV */
uint32_t veoh_tv_stage:2;
/* NDPI_PROTOCOL_SHOUTCAST */
uint32_t shoutcast_stage:2;
@@ -653,12 +597,6 @@ struct ndpi_flow_tcp_struct {
};
struct ndpi_flow_udp_struct {
/* NDPI_PROTOCOL_SNMP */
uint32_t snmp_msg_id;
/* NDPI_PROTOCOL_SNMP */
uint32_t snmp_stage:2;
/* NDPI_PROTOCOL_PPSTREAM */
uint32_t ppstream_stage:3; // 0 - 7
@@ -666,7 +604,7 @@ struct ndpi_flow_udp_struct {
uint32_t halflife2_stage:2; // 0 - 2
/* NDPI_PROTOCOL_TFTP */
uint32_t tftp_stage:1;
uint32_t tftp_stage:2;
/* NDPI_PROTOCOL_AIMINI */
uint32_t aimini_stage:5;
@@ -674,9 +612,6 @@ struct ndpi_flow_udp_struct {
/* NDPI_PROTOCOL_XBOX */
uint32_t xbox_stage:1;
/* NDPI_PROTOCOL_WINDOWS_UPDATE */
uint32_t wsus_stage:1;
/* NDPI_PROTOCOL_SKYPE */
uint8_t skype_packet_id;
@@ -697,6 +632,21 @@ struct ndpi_flow_udp_struct {
/* NDPI_PROTOCOL_WIREGUARD */
uint8_t wireguard_stage;
uint32_t wireguard_peer_index[2];
/* NDPI_PROTOCOL_QUIC */
u_int8_t *quic_reasm_buf;
u_int32_t quic_reasm_buf_len;
/* NDPI_PROTOCOL_CSGO */
uint8_t csgo_strid[18],csgo_state,csgo_s2;
uint32_t csgo_id2;
/* NDPI_PROTOCOL_RDP */
u_int8_t rdp_to_srv[3], rdp_from_srv[3], rdp_to_srv_pkts, rdp_from_srv_pkts;
/* NDPI_PROTOCOL_IMO */
uint8_t imo_last_one_byte_pkt, imo_last_byte;
};
struct ndpi_int_one_line_struct {
@@ -715,8 +665,6 @@ struct ndpi_packet_struct {
uint64_t current_time_ms;
uint16_t detected_protocol_stack[2];
uint8_t detected_subprotocol_stack[2];
uint16_t protocol_stack_info;
struct ndpi_int_one_line_struct line[64];
/* HTTP headers */
@@ -740,19 +688,13 @@ struct ndpi_packet_struct {
uint8_t http_num_headers; /* number of found (valid) header lines in HTTP request or response */
uint16_t l3_packet_len;
uint16_t l4_packet_len;
uint16_t payload_packet_len;
uint16_t actual_payload_len;
uint16_t num_retried_bytes;
uint16_t parsed_lines;
uint16_t parsed_unix_lines;
uint16_t empty_line_position;
uint8_t tcp_retransmission;
uint8_t l4_protocol;
uint8_t tls_certificate_detected:4, tls_certificate_num_checks:4;
uint8_t packet_lines_parsed_complete:1,
packet_direction:1, empty_line_position_set:1, pad:5;
packet_direction:1, empty_line_position_set:1, http_check_content:1, pad:4;
};
struct ndpi_detection_module_struct;
@@ -869,9 +811,9 @@ typedef enum {
typedef struct ndpi_proto_defaults {
char *protoName;
ndpi_protocol_category_t protoCategory;
uint8_t can_have_a_subprotocol;
u_int16_t * subprotocols;
size_t subprotocol_count;
uint16_t protoId, protoIdx;
uint16_t master_tcp_protoId[2], master_udp_protoId[2]; /* The main protocols on which this sub-protocol sits on */
uint16_t tcp_default_ports[5], udp_default_ports[5];
ndpi_protocol_breed_t protoBreed;
void (*func) (struct ndpi_detection_module_struct *, struct ndpi_flow_struct *flow);
@@ -946,8 +888,6 @@ struct ndpi_detection_module_struct {
/* HTTP/DNS/HTTPS host matching */
ndpi_automa host_automa, /* Used for DNS/HTTPS */
content_automa, /* Used for HTTP subprotocol_detection */
subprotocol_automa, /* Used for HTTP subprotocol_detection */
bigrams_automa, impossible_bigrams_automa; /* TOR */
/* IMPORTANT: please update ndpi_finalize_initalization() whenever you add a new automa */
@@ -964,18 +904,7 @@ struct ndpi_detection_module_struct {
uint32_t irc_timeout;
/* gnutella parameters */
uint32_t gnutella_timeout;
/* thunder parameters */
uint32_t thunder_timeout;
/* SoulSeek parameters */
uint32_t soulseek_connection_ip_tick_timeout;
/* rtsp parameters */
uint32_t rtsp_connection_timeout;
/* rstp */
uint32_t orb_rstp_ts_timeout;
/* yahoo */
uint8_t yahoo_detect_http_connections;
uint32_t yahoo_lan_video_timeout;
uint32_t zattoo_connection_timeout;
uint32_t jabber_stun_timeout;
uint32_t jabber_file_transfer_timeout;
uint8_t ip_version_limit;
@@ -1002,6 +931,9 @@ struct ndpi_detection_module_struct {
uint8_t direction_detect_disable:1, /* disable internal detection of packet direction */
_pad:7;
/* Current packet */
struct ndpi_packet_struct packet;
};
#define NDPI_CIPHER_SAFE 0
@@ -1016,7 +948,6 @@ typedef enum {
struct ndpi_flow_struct {
uint16_t detected_protocol_stack[2];
uint16_t protocol_stack_info;
/* init parameter, internal used to set up timestamp,... */
uint16_t guessed_protocol_id, guessed_host_protocol_id, guessed_category, guessed_header_category;
uint8_t l4_proto, protocol_id_already_guessed:1, host_already_guessed:1,
@@ -1043,13 +974,8 @@ struct ndpi_flow_struct {
/* Place textual flow info here */
char flow_extra_info[16];
/*
Pointer to src or dst that identifies the
server of this connection
*/
struct ndpi_id_struct *server_id;
/* HTTP host or DNS query */
uint8_t host_server_name[240];
char host_server_name[80];
uint8_t initial_binary_bytes[8], initial_binary_bytes_len;
uint8_t risk_checked;
ndpi_risk risk; /* Issues found with this flow [bitmask of ndpi_risk] */
@@ -1063,10 +989,12 @@ struct ndpi_flow_struct {
*/
struct {
ndpi_http_method method;
char *url, *content_type, *user_agent;
uint8_t num_request_headers, num_response_headers;
uint8_t request_version; /* 0=1.0 and 1=1.1. Create an enum for this? */
uint16_t response_status_code; /* 200, 404, etc. */
char *url, *content_type, *user_agent;
char *detected_os; /* Via HTTP/QUIC User-Agent */
char *nat_ip; /* Via HTTP X-Forwarded-For */
} http;
/*
@@ -1078,6 +1006,18 @@ struct ndpi_flow_struct {
char *pktbuf;
uint16_t pktbuf_maxlen, pktbuf_currlen;
} kerberos_buf;
struct {
u_int8_t num_udp_pkts, num_binding_requests;
u_int16_t num_processed_pkts;
} stun;
/* TODO: something clever to save memory */
struct {
uint8_t auth_found:1, auth_failed:1, auth_tls:1, auth_done:1, _pad:4;
char username[32], password[16];
} ftp_imap_pop_smtp;
union {
/* the only fields useful for nDPI and ntopng */
struct {
@@ -1096,37 +1036,25 @@ struct ndpi_flow_struct {
} kerberos;
struct {
struct {
char ssl_version_str[12];
uint16_t ssl_version, server_names_len;
char client_requested_server_name[64], *server_names,
*alpn, *tls_supported_versions, *issuerDN, *subjectDN;
char *server_names, *alpn, *tls_supported_versions, *issuerDN, *subjectDN;
uint32_t notBefore, notAfter;
char ja3_client[33], ja3_server[33];
uint16_t server_cipher;
u_int8_t sha1_certificate_fingerprint[20];
u_int8_t hello_processed:1, subprotocol_detected:1, _pad:6;
struct {
uint16_t cipher_suite;
char *esni;
} encrypted_sni;
ndpi_cipher_weakness server_unsafe_cipher;
} ssl;
struct {
uint8_t num_udp_pkts, num_processed_pkts, num_binding_requests;
} stun;
/* We can have STUN over SSL/TLS thus they need to live together */
} stun_ssl;
uint16_t ssl_version, server_names_len;
} tls_quic;
struct {
char client_signature[48], server_signature[48];
char hassh_client[33], hassh_server[33];
} ssh;
struct {
uint8_t last_one_byte_pkt, last_byte;
} imo;
struct {
uint8_t username_detected:1, username_found:1,
password_detected:1, password_found:1,
@@ -1143,18 +1071,6 @@ struct ndpi_flow_struct {
char version[32];
} ubntac2;
struct {
/* Via HTTP User-Agent */
uint8_t detected_os[32];
/* Via HTTP X-Forwarded-For */
uint8_t nat_ip[24];
} http;
struct {
uint8_t auth_found:1, auth_failed:1, _pad:5;
char username[16], password[16];
} ftp_imap_pop_smtp;
struct {
/* Bittorrent hash */
uint8_t hash[20];
@@ -1185,24 +1101,15 @@ struct ndpi_flow_struct {
/* NDPI_PROTOCOL_DIRECTCONNECT */
uint8_t directconnect_stage:2; // 0 - 1
/* NDPI_PROTOCOL_YAHOO */
uint8_t sip_yahoo_voice:1;
/* NDPI_PROTOCOL_HTTP */
uint8_t http_detected:1;
/* NDPI_PROTOCOL_RTSP */
uint8_t rtsprdt_stage:2, rtsp_control_flow:1;
/* NDPI_PROTOCOL_YAHOO */
uint8_t yahoo_detection_finished:2;
uint8_t rtsprdt_stage:2;
/* NDPI_PROTOCOL_ZATTOO */
uint8_t zattoo_stage:3;
/* NDPI_PROTOCOL_QQ */
uint8_t qq_stage:3;
/* NDPI_PROTOCOL_THUNDER */
uint8_t thunder_stage:2; // 0 - 3
@@ -1237,16 +1144,13 @@ struct ndpi_flow_struct {
uint8_t ovpn_session_id[8];
uint8_t ovpn_counter;
/* Flow key used to search a match into the mining cache */
u_int32_t key_mining_cache;
/* NDPI_PROTOCOL_TINC */
uint8_t tinc_state;
struct tinc_cache_entry tinc_cache_entry;
/* NDPI_PROTOCOL_CSGO */
uint8_t csgo_strid[18],csgo_state,csgo_s2;
uint32_t csgo_id2;
/* internal structures to save functions calls */
struct ndpi_packet_struct packet;
struct ndpi_flow_struct *flow;
struct ndpi_id_struct *src;
struct ndpi_id_struct *dst;
};
+40 -111
View File
@@ -94,11 +94,10 @@ class NDPIProtoDefaultsT(Structure):
_fields_ = [
("protoName", c_char_p),
("protoCategory", c_uint),
("can_have_a_subprotocol", c_uint8),
("subprotocols", c_uint16_p),
("subprotocol_count", c_uint32),
("protoId", c_uint16),
("protoIdx", c_uint16),
("master_tcp_protoId", c_uint16 * 2),
("master_udp_protoId", c_uint16 * 2),
("protoBreed", c_uint),
("func", CFUNCTYPE(None, POINTER(NDPIDetectionModuleStruct), POINTER(NDPIFlowStruct))),
]
@@ -252,22 +251,12 @@ NDPIDetectionModuleStruct._fields_ = [
("ndpi_num_supported_protocols", c_uint),
("ndpi_num_custom_protocols", c_uint),
("host_automa", NDPIAutoma),
("content_automa", NDPIAutoma),
("subprotocol_automa", NDPIAutoma),
("bigrams_automa", NDPIAutoma),
("impossible_bigrams_automa", NDPIAutoma),
("custom_categories", CustomCategories),
("protocols_ptree", c_void_p),
("irc_timeout", c_uint32),
("gnutella_timeout", c_uint32),
("battlefield_timeout", c_uint32),
("thunder_timeout", c_uint32),
("soulseek_connection_ip_tick_timeout", c_uint32),
("rtsp_connection_timeout", c_uint32),
("tvants_connection_timeout", c_uint32),
("orb_rstp_ts_timeout", c_uint32),
("yahoo_detect_http_connections", c_uint8),
("yahoo_lan_video_timeout", c_uint32),
("zattoo_connection_timeout", c_uint32),
("jabber_stun_timeout", c_uint32),
("jabber_file_transfer_timeout", c_uint32),
@@ -280,11 +269,9 @@ NDPIDetectionModuleStruct._fields_ = [
("tinc_cache", POINTER(Cache)),
("proto_defaults", NDPIProtoDefaultsT * (ndpi.ndpi_wrap_ndpi_max_supported_protocols() +
ndpi.ndpi_wrap_ndpi_max_num_custom_protocols())),
("http_dont_dissect_response", c_uint8, 1),
("dns_dont_dissect_response", c_uint8, 1),
("direction_detect_disable", c_uint8, 1),
("disable_metadata_export", c_uint8, 1),
("hyperscan", c_void_p)
('_pad', c_uint8, 7),
('packet', NDPIPacketStruct),
]
@@ -313,38 +300,24 @@ class NDPIIpAddrT(Union):
class NDPIIdStruct(Structure):
_fields_ = [
('detected_protocol_bitmask', NDPIProtocolBitMask),
('rtsp_ip_address', NDPIIpAddrT),
('yahoo_video_lan_timer', c_uint32),
('irc_port', c_uint16 * 8),
('last_time_port_used', c_uint32 * 8),
('irc_ts', c_uint32),
('gnutella_ts', c_uint32),
('battlefield_ts', c_uint32),
('thunder_ts', c_uint32),
('rtsp_timer', c_uint32),
('oscar_last_safe_access_time', c_uint32),
('zattoo_ts', c_uint32),
('jabber_stun_or_ft_ts', c_uint32),
('directconnect_last_safe_access_time', c_uint32),
('soulseek_last_safe_access_time', c_uint32),
('detected_directconnect_port', c_uint16),
('detected_directconnect_udp_port', c_uint16),
('detected_directconnect_ssl_port', c_uint16),
('bt_port_t', c_uint16 * 8),
('bt_port_u', c_uint16 * 8),
('jabber_voice_stun_port', c_uint16 * 6),
('jabber_file_transfer_port', c_uint16 * 2),
('detected_gnutella_port', c_uint16),
('detected_gnutella_udp_port1', c_uint16),
('detected_gnutella_udp_port2', c_uint16),
('soulseek_listen_port', c_uint16),
('irc_number_of_port', c_uint8),
('oscar_ssl_session_id', c_uint8 * 33),
('jabber_voice_stun_used_ports', c_uint8),
('yahoo_video_lan_dir', c_uint32, 1),
('yahoo_conf_logged_in', c_uint32, 1),
('yahoo_voice_conf_logged_in', c_uint32, 1),
('rtsp_ts_set', c_uint32, 1),
]
@@ -353,35 +326,21 @@ class NDPIFlowTcpStruct(Structure):
_fields_ = [
('smtp_command_bitmask', c_uint16),
('pop_command_bitmask', c_uint16),
('qq_nxt_len', c_uint16),
('wa_matched_so_far', c_uint8),
('tds_login_version', c_uint8),
('irc_stage', c_uint8),
('irc_port', c_uint8),
('h323_valid_packets', c_uint8),
('gnutella_msg_id', c_uint8 * 3),
('irc_3a_counter', c_uint32, 3),
('irc_stage2', c_uint32, 5),
('irc_direction', c_uint32, 2),
('irc_0x1000_full', c_uint32, 1),
('soulseek_stage', c_uint32, 2),
('tds_stage', c_uint32, 3),
('usenet_stage', c_uint32, 2),
('imesh_stage', c_uint32, 4),
('http_setup_dir', c_uint32, 2),
('http_stage', c_uint32, 2),
('http_empty_line_seen', c_uint32, 1),
('http_wait_for_retransmission', c_uint32, 1),
('gnutella_stage', c_uint32, 2),
('mms_stage', c_uint32, 2),
('yahoo_sip_comm', c_uint32, 1),
('yahoo_http_proxy_stage', c_uint32, 2),
('msn_stage', c_uint32, 3),
('msn_ssl_ft', c_uint32, 2),
('ssh_stage', c_uint32, 3),
('vnc_stage', c_uint32, 2),
('telnet_stage', c_uint32, 2),
('tls_srv_cert_fingerprint_ctx', c_void_p),
('tls_seen_client_cert', c_uint8, 1),
('tls_seen_server_cert', c_uint8, 1),
('tls_seen_certificate', c_uint8, 1),
@@ -392,7 +351,6 @@ class NDPIFlowTcpStruct(Structure):
('tls_fingerprint_len', c_int16),
('tls_sha1_certificate_fingerprint', c_uint8 * 20),
('postgres_stage', c_uint32, 3),
('ddlink_server_direction', c_uint32, 1),
('seen_syn', c_uint32, 1),
('seen_syn_ack', c_uint32, 1),
('seen_ack', c_uint32, 1),
@@ -400,7 +358,6 @@ class NDPIFlowTcpStruct(Structure):
('dofus_stage', c_uint32, 1),
('fiesta_stage', c_uint32, 2),
('wow_stage', c_uint32, 2),
('veoh_tv_stage', c_uint32, 2),
('shoutcast_stage', c_uint32, 2),
('rtp_special_packets_seen', c_uint32, 1),
('mail_pop_stage', c_uint32, 2),
@@ -421,16 +378,11 @@ class NDPIFlowTcpStruct(Structure):
class NDPIFlowUdpStruct(Structure):
_pack_ = 1
_fields_ = [
('battlefield_msg_id', c_uint32),
('snmp_msg_id', c_uint32),
('battlefield_stage', c_uint32, 3),
('snmp_stage', c_uint32, 2),
('ppstream_stage', c_uint32, 3),
('halflife2_stage', c_uint32, 2),
('tftp_stage', c_uint32, 1),
('tftp_stage', c_uint32, 2),
('aimini_stage', c_uint32, 5),
('xbox_stage', c_uint32, 1),
('wsus_stage', c_uint32, 1),
('skype_packet_id', c_uint8),
('teamviewer_stage', c_uint8),
('eaq_pkt_id', c_uint8),
@@ -440,6 +392,18 @@ class NDPIFlowUdpStruct(Structure):
('memcached_matches', c_uint8),
('wireguard_stage', c_uint8),
('wireguard_peer_index', c_uint32 * 2),
('quic_reasm_buf', POINTER(c_uint8)),
('quic_reasm_buf_len', c_uint32),
('csgo_strid', c_uint8 * 18),
('csgo_state', c_uint8),
('csgo_s2', c_uint8),
('csgo_id2', c_uint32),
('rdp_to_srv', c_uint8 * 3),
('rdp_from_srv', c_uint8 * 3),
('rdp_to_srv_pkts,', c_uint8),
('rdp_from_srv_pkts', c_uint8),
('imo_last_one_byte_pkt,', c_uint8),
('imo_last_byte', c_uint8),
]
@@ -450,12 +414,13 @@ class L4(Union):
class Http(Structure):
_fields_ = [
("method", c_int),
("url", c_char_p),
("content_type", c_char_p),
("num_request_headers", c_uint8),
("num_response_headers", c_uint8),
("request_version", c_uint8),
("response_status_code", c_uint16),
("url", c_char_p),
("content_type", c_char_p),
("user_agent", c_char_p),
("detected_os", c_char_p),
("nat_ip", c_char_p),
]
@@ -482,7 +447,7 @@ class Kerberos(Structure):
("realm", c_char * 24)]
class Ssl(Structure):
class QuicSsl(Structure):
_fields_ = [
("ssl_version", c_uint16),
("client_certificate", c_char * 64),
@@ -505,10 +470,6 @@ class Stun(Structure):
]
class StunSsl(Structure):
_fields_ = [("ssl", Ssl), ("stun", Stun)]
class Ssh(Structure):
_fields_ = [
("client_signature", c_char * 48),
@@ -518,13 +479,6 @@ class Ssh(Structure):
]
class Imo(Structure):
_fields_ = [
("last_one_byte_pkt", c_uint8),
("last_byte", c_uint8)
]
class Mdns(Structure):
_fields_ = [("answer", c_char * 96)]
@@ -533,13 +487,17 @@ class Ubntac2(Structure):
_fields_ = [("version", c_char * 32)]
class Http2(Structure):
class FtpImapPopSmtp(Structure):
_fields_ = [
("detected_os", c_char * 32),
("nat_ip", c_char * 24)
("auth_found", c_uint8, 1),
("auth_failed", c_uint8, 1),
("auth_tls", c_uint8, 1),
("auth_done", c_uint8, 1),
("_pad", c_uint8, 4),
("username", c_char * 32),
("password", c_char * 16)
]
class Bittorrent(Structure):
_fields_ = [("hash", c_char * 20)]
@@ -555,12 +513,10 @@ class Protos(Union):
_fields_ = [
("dns", Dns),
("kerberos", Kerberos),
("stun_ssl", StunSsl),
("quic_ssl", QuicSsl),
("ssh", Ssh),
("imo", Imo),
("mdns", Mdns),
("ubntac2", Ubntac2),
("http", Http2),
("bittorrent", Bittorrent),
("dhcp", Dhcp)
]
@@ -650,14 +606,6 @@ class NDPIUdpHdr(Structure):
]
class NDPIPacketStructStack(Structure):
_pack_ = 1
_fields_ = [
('detected_subprotocol_stack', c_uint8 * ndpi.ndpi_wrap_ndpi_procol_size()),
('protocol_stack_info', c_uint16)
]
class NDPIPacketStruct(Structure):
_fields_ = [
('iph', POINTER(NDPIIphdr)),
@@ -666,10 +614,8 @@ class NDPIPacketStruct(Structure):
('udp', POINTER(NDPIUdpHdr)),
('generic_l4_ptr', POINTER(c_uint8)),
('payload', POINTER(c_uint8)),
('tick_timestamp', c_uint32),
('tick_timestamp_l', c_uint64),
('current_timestamp_ms', c_uint64),
('detected_protocol_stack', c_uint16 * ndpi.ndpi_wrap_ndpi_procol_size()),
('ndpi_packet_stack', NDPIPacketStructStack),
('line', NDPIIntOneLineStruct * 64),
('host_line', NDPIIntOneLineStruct),
('forwarded_line', NDPIIntOneLineStruct),
@@ -689,21 +635,15 @@ class NDPIPacketStruct(Structure):
('http_response', NDPIIntOneLineStruct),
('http_num_headers', c_uint8),
('l3_packet_len', c_uint16),
('l4_packet_len', c_uint16),
('payload_packet_len', c_uint16),
('actual_payload_len', c_uint16),
('num_retried_bytes', c_uint16),
('parsed_lines', c_uint16),
('parsed_unix_lines', c_uint16),
('empty_line_position', c_uint16),
('tcp_retransmission', c_uint8),
('l4_protocol', c_uint8),
('ssl_certificate_detected', c_uint8, 4),
('ssl_certificate_num_checks', c_uint8, 4),
('packet_lines_parsed_complete', c_uint8, 1),
('packet_direction', c_uint8, 1),
('empty_line_position_set', c_uint8, 1),
('pad', c_uint8, 5),
('http_check_content', c_uint8, 1),
('pad', c_uint8, 4),
]
@@ -711,7 +651,6 @@ class NDPIFlowStructStack(Structure):
_pack_ = 1
_fields_ = [
("detected_protocol_stack", c_uint16 * ndpi.ndpi_wrap_ndpi_procol_size()),
("protocol_stack_info", c_uint16)
]
@@ -734,9 +673,10 @@ NDPIFlowStruct._fields_ = [
("num_processed_pkts", c_uint8),
("extra_packets_func", CFUNCTYPE(c_int, POINTER(NDPIDetectionModuleStruct), POINTER(NDPIFlowStruct))),
("l4", L4),
("server_id", POINTER(NDPIIdStruct)),
("host_server_name", c_ubyte * 256),
("host_server_name", c_char * 80),
("http", Http),
("stun", Stun),
("ftp_imap_pop_smtp", FtpImapPopSmtp),
("protos", Protos),
("excluded_protocol_bitmask", NDPIProtocolBitMask),
("category", c_int),
@@ -747,15 +687,11 @@ NDPIFlowStruct._fields_ = [
('byte_counter', c_uint16 * 2),
('bittorrent_stage', c_uint8),
('directconnect_stage', c_uint8, 2),
('sip_yahoo_voice', c_uint8, 1),
('http_detected', c_uint8, 1),
('http_upper_protocol', c_uint16),
('http_lower_protocol', c_uint16),
('rtsprdt_stage', c_uint8, 2),
('rtsp_control_flow', c_uint8, 1),
('yahoo_detection_finished', c_uint8, 2),
('zattoo_stage', c_uint8, 3),
('qq_stage', c_uint8, 3),
('thunder_stage', c_uint8, 2),
('oscar_ssl_voice_stage', c_uint8, 3),
('oscar_video_voice', c_uint8, 1),
@@ -775,16 +711,9 @@ NDPIFlowStruct._fields_ = [
('starcraft_udp_stage', c_uint8, 3),
('ovpn_session_id', c_uint8 * 8),
('ovpn_counter', c_uint8),
('key_mining_cache', c_uint32),
('tinc_state', c_uint8),
('TincCacheEntry', TincCacheEntry),
('csgo_strid', c_uint8 * 18),
('csgo_state', c_uint8),
('csgo_s2', c_uint8),
('csgo_id2', c_uint32),
('kxun_counter', c_uint16),
('iqiyi_counter', c_uint16),
('packet', NDPIPacketStruct),
('flow', POINTER(NDPIFlowStruct)),
('src', POINTER(NDPIIdStruct)),
('dst', POINTER(NDPIIdStruct))
]
@@ -861,4 +790,4 @@ ndpi.ndpi_set_protocol_detection_bitmask2.argtypes = [POINTER(NDPIDetectionModul
ndpi.ndpi_twalk.argtypes = [c_void_p, CFUNCTYPE(None, c_void_p, c_int32, c_int, c_void_p), c_void_p]
""" ndpi_tdestroy: node destroy. """
ndpi.ndpi_tdestroy.argtypes = [c_void_p, CFUNCTYPE(None, c_void_p)]
ndpi.ndpi_tdestroy.argtypes = [c_void_p, CFUNCTYPE(None, c_void_p)]
+26
View File
@@ -0,0 +1,26 @@
CC=@CC@
INC=-I ../src/include -I/usr/local/include
LIBDPI=../src/lib/libndpi.a
LDFLAGS=@LDFLAGS@
LIB=$(LIBDPI) -lrrd -lm @LIBS@ @ADDITIONAL_LIBS@
TOOLS=rrd_anomaly rrd_similarity
all: $(TOOLS)
rrd_anomaly: rrd_anomaly.c Makefile $(LIBDPI)
$(CC) $(CPPFLAGS) -g $(INC) $(LDFLAGS) rrd_anomaly.c -o rrd_anomaly $(LIB)
rrd_similarity: rrd_similarity.c Makefile $(LIBDPI)
$(CC) $(CPPFLAGS) -g $(INC) $(LDFLAGS) rrd_similarity.c -o rrd_similarity $(LIB)
clean:
/bin/rm -f *.o $(TOOLS) *~
distclean: clean
/bin/rm -f Makefile
distdir:
install:
@echo -n ""
+10
View File
@@ -0,0 +1,10 @@
This directory contains a tool that allows to identify anomalies and similarities in RRD files
Prerequisite
- rrdtool (https://oss.oetiker.ch/rrdtool/)
If you have Ubuntu-based systems you need to do
- sudo apt-get install rrdtool librrd-dev
+184
View File
@@ -0,0 +1,184 @@
/*
* rrd_anomaly.c
*
* Copyright (C) 2011-22 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <getopt.h>
#include <rrd.h>
#include "ndpi_api.h"
#define DEFAULT_ALPHA 0.5
#define DEFAULT_RO 0.05
#define DEFAULT_START "now-1d"
#define DEFAULT_END "now"
/* *************************************************** */
static void help() {
printf("Usage: rrd_anomaly [-v][-a <alpha>][-e <end>][-q][-s <start>] -f <filename>\n"
"-a | Set alpha. Valid range >0 .. <1. Default %.2f\n"
"-e <end> | RRD end time. Default %s\n"
"-q | Quick output (only anomalies are reported)\n"
"-s <start> | RRD start time. Default %s\n"
"-f <rrd path> | Path of the RRD filename to analyze\n"
"-v | Verbose\n"
,
DEFAULT_ALPHA, DEFAULT_END, DEFAULT_START);
printf("\n\nExample: rrd_anomaly -q -f hum.rrd\n");
exit(0);
}
/* *************************************************** */
int main(int argc, char *argv[]) {
rrd_time_value_t start_tv, end_tv;
unsigned long step = 0, ds_cnt = 0;
rrd_value_t *data, *p;
char **names, *filename = NULL, *start_s, *end_s, *cf;
u_int i, j, t, first = 1, quick_mode = 0, verbose = 0;
time_t start, end;
struct ndpi_ses_struct ses;
float alpha, ro;
char c;
/* Defaults */
alpha = DEFAULT_ALPHA;
start_s = DEFAULT_START;
end_s = DEFAULT_END;
cf = "AVERAGE";
ro = DEFAULT_RO;
while((c = getopt(argc, argv, "s:e:a:qf:r:v")) != '?') {
if(c == -1) break;
switch(c) {
case 's':
start_s = optarg;
break;
case 'e':
end_s = optarg;
break;
case 'q':
quick_mode = 1;
break;
case 'a':
{
float f = atof(optarg);
if((f > 0) && (f < 1))
alpha = f;
else
printf("Discarding -a: valid range is >0 .. <1\n");
}
break;
case 'f':
filename = optarg;
break;
case 'r':
ro = atof(optarg);
if((ro <= 0) || (ro >= 1))
ro = DEFAULT_RO;
break;
case 'v':
verbose = 1;
break;
default:
help();
break;
}
}
if(filename == NULL)
help();
ndpi_ses_init(&ses, alpha, ro);
if((rrd_parsetime(start_s, &start_tv) != NULL)) {
printf("Unable to parse start time %s\n", start_s);
return(-1);
}
if((rrd_parsetime(end_s, &end_tv) != NULL)) {
printf("Unable to parse end time %s\n", end_s);
return(-1);
}
rrd_proc_start_end(&start_tv, &end_tv, &start, &end);
if(rrd_fetch_r(filename, cf, &start, &end, &step, &ds_cnt, &names, &data) != 0) {
printf("Unable to extract data from rrd %s\n", filename);
return(-2);
}
p = data;
for(t=start+1, i=0; t<end; t+=step, i++) {
j = 0; /* Consider only the first DS */
/* for(j=0; j<ds_cnt; j++) */ {
rrd_value_t value = *p++;
if(!isnan(value)) {
double prediction, confidence_band;
double lower, upper;
char buf[32];
int rc;
u_int is_anomaly;
value *= 100; /* trick to avoid dealing with floats */
rc = ndpi_ses_add_value(&ses, value, &prediction, &confidence_band);
lower = prediction - confidence_band, upper = prediction + confidence_band;
is_anomaly = ((rc == 0) || (confidence_band == 0) || ((value >= lower) && (value <= upper))) ? 0 : 1;
if(verbose || is_anomaly) {
if(quick_mode) {
printf("%u\n", t);
} else {
const time_t _t = t;
struct tm *t_info = localtime((const time_t*)&_t);
strftime(buf, sizeof(buf), "%d/%b/%Y %H:%M:%S", t_info);
if(first) {
first = 0;
printf("%s %s\t%s %s %s\t %s [%s]\n",
"When", "Value", "Prediction", "Lower", "Upper", "Out", "Band");
}
printf("%s %12.3f\t%.3f\t%12.3f\t%12.3f\t %s [%.3f]\n",
buf, value/100., prediction/100., lower/100., upper/100., is_anomaly? "ANOMALY" : "OK",
confidence_band/100.);
}
}
}
}
}
rrd_freemem(data);
return(0);
}
+319
View File
@@ -0,0 +1,319 @@
/*
* rrd_similarity.c
*
* Copyright (C) 2011-22 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <getopt.h>
#include <dirent.h>
#include <sys/stat.h>
#include "rrd.h"
#include "ndpi_api.h"
#define DEFAULT_ALPHA 0.5
#define DEFAULT_START "now-1d"
#define DEFAULT_END "now"
#define MAX_NUM_RRDS 8192
#ifndef PATH_MAX
#define PATH_MAX 4096
#endif
typedef struct {
char *path;
float average, stddev;
struct ndpi_bin b;
} rrd_file_stats;
u_int verbose = 0, similarity_threshold = 100, skip_zero = 0;
/* *************************************************** */
static void help() {
printf("Usage: rrd_similarity [-v][-a <alpha>][-e <end>][-q][-s <start>]\n"
" -f <filename> -d <basedir> [-t <threshold>]\n"
"-a | Set alpha. Valid range >0 .. <1. Default %.2f\n"
"-e <end> | RRD end time. Default %s\n"
"-q | Quick output (only anomalies are reported)\n"
"-s <start> | RRD start time. Default %s\n"
"-d <basedir> | Base directory where RRD filename is searched\n"
"-f <rrd path> | Path of the RRD filename to analyze\n"
"-t <threshold> | Similarity threshold. Default %u (0 == alike)\n"
"-v | Verbose\n"
"-z | Skip zero RRDs during comparison\n"
,
DEFAULT_ALPHA, DEFAULT_END, DEFAULT_START, similarity_threshold);
printf("\n\nExample: rrd_similarity -q -f bytes.rrd -d /var/lib/ntopng/-1/snmpstats\n");
printf("\n\nGoal: find similar RRDs\n");
exit(0);
}
/* *************************************************** */
void analyze_rrd(rrd_file_stats *rrd, time_t start, time_t end) {
unsigned long step = 0, ds_cnt = 0;
rrd_value_t *data, *p;
char **names;
u_int t, i, j, num_points;
struct ndpi_analyze_struct *s;
if(rrd_fetch_r(rrd->path, "AVERAGE", &start, &end, &step, &ds_cnt, &names, &data) != 0) {
printf("Unable to extract data from rrd %s\n", rrd->path);
return;
}
p = data;
num_points = (end-start)/step;
if((s = ndpi_alloc_data_analysis(num_points)) == NULL)
return;
ndpi_init_bin(&rrd->b, ndpi_bin_family32, num_points);
/* Step 1 - Compute average and stddev */
for(t=start+1, i=0; t<end; t+=step, i++) {
double value = (double)*p++;
if(isnan(value)) value = 0;
ndpi_data_add_value(s, value);
ndpi_set_bin(&rrd->b, i, value);
}
rrd->average = ndpi_data_average(s);
rrd->stddev = ndpi_data_stddev(s);
/* Step 2 - Bin analysis */
ndpi_free_data_analysis(s, 1);
rrd_freemem(data);
}
/* *************************************************** */
int circles_touch(int x1, int r1, int x2, int r2) {
int radius_sum = r1+r2;
int x_diff = abs(x1 - x2);
return((radius_sum < x_diff) ? 0 : 1);
}
/* *************************************************** */
void find_rrd_similarities(rrd_file_stats *rrd, u_int num_rrds) {
u_int i, j, num_similar_rrds = 0, num_potentially_zero_equal = 0;
for(i=0; i<num_rrds; i++) {
for(j=i+1; j<num_rrds; j++) {
/*
Average is the circle center, and stddev is the radius
if circles touch each other then there is a chance that
the two rrds are similar
*/
if((rrd[i].average == 0) && (rrd[i].average == rrd[j].average)) {
if(!skip_zero)
printf("%s [%.1f/%.1f] - %s [%.1f/%.1f] are alike\n",
rrd[i].path, rrd[i].average, rrd[i].stddev,
rrd[j].path, rrd[j].average, rrd[j].stddev);
num_potentially_zero_equal++;
} else if(circles_touch(rrd[i].average, rrd[i].stddev, rrd[j].average, rrd[j].stddev)
) {
float similarity = ndpi_bin_similarity(&rrd[i].b, &rrd[j].b, 0, similarity_threshold);
if((similarity >= 0) && (similarity < similarity_threshold)) {
if(verbose)
printf("%s [%.1f/%.1f] - %s [%.1f/%.1f] are %s [%.1f]\n",
rrd[i].path, rrd[i].average, rrd[i].stddev,
rrd[j].path, rrd[j].average, rrd[j].stddev,
(similarity == 0) ? "alike" : "similar",
similarity
);
num_similar_rrds++;
}
}
}
}
printf("Found %u (%.3f %%) similar RRDs / %u zero alike RRDs [num_rrds: %u]\n",
num_similar_rrds,
(num_similar_rrds*100.)/(float)(num_rrds*num_rrds),
num_potentially_zero_equal,
num_rrds);
}
/* *************************************************** */
void find_rrds(char *basedir, char *filename, rrd_file_stats *rrds, u_int *num_rrds) {
struct dirent **namelist;
int n = scandir(basedir, &namelist, 0, NULL);
if(n < 0)
return; /* End of the tree */
while(n--) {
if(namelist[n]->d_name[0] != '.') {
char path[PATH_MAX];
struct stat s;
snprintf(path, sizeof(path), "%s/%s", basedir, namelist[n]->d_name);
if(stat(path, &s) == 0) {
if(S_ISDIR(s.st_mode))
find_rrds(path, filename, rrds, num_rrds);
else if(strcmp(namelist[n]->d_name, filename) == 0) {
if(*num_rrds < MAX_NUM_RRDS) {
rrds[*num_rrds].path = strdup(path);
if(rrds[*num_rrds].path != NULL)
(*num_rrds)++;
}
}
}
}
free(namelist[n]);
}
free(namelist);
}
/* *************************************************** */
int main(int argc, char *argv[]) {
rrd_time_value_t start_tv, end_tv;
char *filename = NULL, *start_s, *end_s, *dirname = NULL, *basedir = NULL;
u_int first = 1, quick_mode = 0;
float alpha;
char c;
time_t start, end;
u_int num_rrds = 0, i;
rrd_file_stats *rrds;
/* Defaults */
alpha = DEFAULT_ALPHA;
start_s = DEFAULT_START;
end_s = DEFAULT_END;
while((c = getopt(argc, argv, "d:s:e:a:qf:t:vz")) != '?') {
if(c == -1) break;
switch(c) {
case 's':
start_s = optarg;
break;
case 'd':
basedir = optarg;
break;
case 'e':
end_s = optarg;
break;
case 'q':
quick_mode = 1;
break;
case 'v':
verbose = 1;
break;
case 'a':
{
float f = atof(optarg);
if((f > 0) && (f < 1))
alpha = f;
else
printf("Discarding -a: valid range is >0 .. <1\n");
}
break;
case 'f':
filename = optarg;
break;
case 't':
similarity_threshold = atoi(optarg);
break;
case 'z':
skip_zero = 1;
break;
default:
help();
break;
}
}
if((filename == NULL) || (basedir == NULL))
help();
if((rrd_parsetime(start_s, &start_tv) != NULL)) {
printf("Unable to parse start time %s\n", start_s);
return(-1);
}
if((rrd_parsetime(end_s, &end_tv) != NULL)) {
printf("Unable to parse end time %s\n", end_s);
return(-1);
}
rrd_proc_start_end(&start_tv, &end_tv, &start, &end);
if((rrds = ndpi_calloc(sizeof(rrd_file_stats), MAX_NUM_RRDS)) == NULL) {
printf("Not enough memory !\n");
return(-1);
}
/* Find all rrd's */
find_rrds(basedir, filename, rrds, &num_rrds);
/* Read RRD's data */
for(i=0; i<num_rrds; i++)
analyze_rrd(&rrds[i], start, end);
find_rrd_similarities(rrds, num_rrds);
#if 0
if(verbose) {
for(i=0; i<num_rrds; i++)
printf("%s\t%.1f\t%.1f\n", rrds[i].path, rrds[i].average, rrds[i].stddev);
}
#endif
for(i=0; i<num_rrds; i++) {
ndpi_free_bin(&rrds[i].b);
free(rrds[i].path);
}
ndpi_free(rrds);
return(0);
}
+12
View File
@@ -0,0 +1,12 @@
sonar.projectKey=ntop_nDPI
sonar.organization=ntop
# This is the name and version displayed in the SonarCloud UI.
#sonar.projectName=nDPI
#sonar.projectVersion=1.0
# Path is relative to the sonar-project.properties file. Replace "\" by "/" on Windows.
#sonar.sources=.
# Encoding of the source code. Default is default system encoding
#sonar.sourceEncoding=UTF-8
+352 -136
View File
File diff suppressed because it is too large Load Diff
+101 -15
View File
@@ -27,8 +27,8 @@
gcc -E -dM - < /dev/null |grep ENDIAN
*/
#if defined(__FreeBSD__) || defined(__NetBSD__) || defined(__OpenBSD__)
#include <machine/endian.h>
#if defined(__FreeBSD__) || defined(__NetBSD__)
#include <sys/endian.h>
#endif
#ifdef __OpenBSD__
@@ -156,6 +156,7 @@
/* misc definitions */
#define NDPI_DEFAULT_MAX_TCP_RETRANSMISSION_WINDOW_SIZE 0x10000
#define NDPI_MAX_NUM_PKTS_PER_FLOW_TO_DISSECT 32
/* TODO: rebuild all memory areas to have a more aligned memory block here */
@@ -171,17 +172,11 @@
#define NDPI_GNUTELLA_CONNECTION_TIMEOUT 60
#define NDPI_BATTLEFIELD_CONNECTION_TIMEOUT 60
#define NDPI_THUNDER_CONNECTION_TIMEOUT 30
#define NDPI_RTSP_CONNECTION_TIMEOUT 5
#define NDPI_TVANTS_CONNECTION_TIMEOUT 5
#define NDPI_YAHOO_DETECT_HTTP_CONNECTIONS 1
#define NDPI_YAHOO_LAN_VIDEO_TIMEOUT 30
#define NDPI_ZATTOO_CONNECTION_TIMEOUT 120
#define NDPI_ZATTOO_FLASH_TIMEOUT 5
#define NDPI_JABBER_STUN_TIMEOUT 30
#define NDPI_JABBER_FT_TIMEOUT 5
#define NDPI_SOULSEEK_CONNECTION_IP_TICK_TIMEOUT 600
#include "ndpi_config.h" /* To have access to NDPI_ENABLE_DEBUG_MESSAGES */
#ifdef NDPI_ENABLE_DEBUG_MESSAGES
#define NDPI_LOG(proto, m, log_level, args...) \
@@ -257,6 +252,7 @@
((x.u6_addr.u6_addr64[0] < y.u6_addr.u6_addr64[0]) || ((x.u6_addr.u6_addr64[0] == y.u6_addr.u6_addr64[0]) && (x.u6_addr.u6_addr64[1] < y.u6_addr.u6_addr64[1])))
#define NDPI_NUM_BITS 512
#define NDPI_NUM_BITS_MASK (512-1)
#define NDPI_BITS /* 32 */ (sizeof(ndpi_ndpi_mask) * 8 /* number of bits in a byte */) /* bits per mask */
#define howmanybits(x, y) (((x)+((y)-1))/(y))
@@ -278,18 +274,18 @@
#define NDPI_BITMASK_SET_ALL(a) NDPI_ONE(&a)
#define NDPI_BITMASK_SET(a, b) { memcpy(&a, &b, sizeof(NDPI_PROTOCOL_BITMASK)); }
#define NDPI_SET_BIT(num, n) num |= 1UL << ( n )
#define NDPI_CLR_BIT(num, n) num &= ~(1UL << ( n ))
#define NDPI_CLR_BIT(num, n) num &= ~(1UL << ( n ))
#define NDPI_ISSET_BIT(num, n) (num & (1UL << ( n )))
#define NDPI_SET_BIT(num, n) num |= 1ULL << ( n )
#define NDPI_CLR_BIT(num, n) num &= ~(1ULL << ( n ))
#define NDPI_CLR_BIT(num, n) num &= ~(1ULL << ( n ))
#define NDPI_ISSET_BIT(num, n) (num & (1ULL << ( n )))
#define NDPI_ZERO_BIT(num) num = 0
/* this is a very very tricky macro *g*,
* the compiler will remove all shifts here if the protocol is static...
*/
#define NDPI_ADD_PROTOCOL_TO_BITMASK(bmask,value) NDPI_SET(&bmask,value)
#define NDPI_DEL_PROTOCOL_FROM_BITMASK(bmask,value) NDPI_CLR(&bmask,value)
#define NDPI_COMPARE_PROTOCOL_TO_BITMASK(bmask,value) NDPI_ISSET(&bmask,value)
#define NDPI_ADD_PROTOCOL_TO_BITMASK(bmask,value) NDPI_SET(&bmask, value & NDPI_NUM_BITS_MASK)
#define NDPI_DEL_PROTOCOL_FROM_BITMASK(bmask,value) NDPI_CLR(&bmask, value & NDPI_NUM_BITS_MASK)
#define NDPI_COMPARE_PROTOCOL_TO_BITMASK(bmask,value) NDPI_ISSET(&bmask, value & NDPI_NUM_BITS_MASK)
#define NDPI_SAVE_AS_BITMASK(bmask,value) { NDPI_ZERO(&bmask) ; NDPI_ADD_PROTOCOL_TO_BITMASK(bmask, value); }
@@ -317,7 +313,16 @@
#define get_u_int8_t(X,O) (*(u_int8_t *)((&(((u_int8_t *)X)[O]))))
#define get_u_int16_t(X,O) (*(u_int16_t *)((&(((u_int8_t *)X)[O]))))
#define get_u_int32_t(X,O) (*(u_int32_t *)((&(((u_int8_t *)X)[O]))))
#if defined(__arm__)
static inline u_int64_t get_u_int64_t(const u_int8_t* X, int O)
{
u_int64_t tmp;
memcpy(&tmp, X + O, sizeof(tmp));
return tmp;
}
#else
#define get_u_int64_t(X,O) (*(u_int64_t *)((&(((u_int8_t *)X)[O]))))
#endif // __arm__
/* new definitions to get little endian from network bytes */
#define get_ul8(X,O) get_u_int8_t(X,O)
@@ -341,6 +346,19 @@
#define snprintf _snprintf
#endif
#if defined(WIN32)
#undef strtok_r
#define strtok_r strtok_s
#if BYTE_ORDER == LITTLE_ENDIAN
#define le16toh(x) (x)
#define le32toh(x) (x)
#else
#error "byte order not supported"
#endif
#endif /* WIN32 */
#define NDPI_MAX_DNS_REQUESTS 16
#define NDPI_MIN_NUM_STUN_DETECTION 8
@@ -378,4 +396,72 @@
#endif /* __APPLE__ */
#if defined(__MINGW32__)
#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
#define htobe16(x) htons(x)
#define htole16(x) (x)
#define be16toh(x) ntohs(x)
#define le16toh(x) (x)
#define htobe32(x) htonl(x)
#define htole32(x) (x)
#define be32toh(x) ntohl(x)
#define le32toh(x) (x)
#define htobe64(x) htonll(x)
#define htole64(x) (x)
#define be64toh(x) ntohll(x)
#define le64toh(x) (x)
#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
#define htobe16(x) (x)
#define htole16(x) __builtin_bswap16(x)
#define be16toh(x) (x)
#define le16toh(x) __builtin_bswap16(x)
#define htobe32(x) (x)
#define htole32(x) __builtin_bswap32(x)
#define be32toh(x) (x)
#define le32toh(x) __builtin_bswap32(x)
#define htobe64(x) (x)
#define htole64(x) __builtin_bswap64(x)
#define be64toh(x) (x)
#define le64toh(x) __builtin_bswap64(x)
#else
#error Unexpected __BYTE_ORDER__
#endif /* __BYTE_ORDER__ */
#endif /* __MINGW32__ */
#ifndef ETH_ARP
#define ETH_ARP 0x0806
#endif
#ifndef ETH_P_IP
#define ETH_P_IP 0x0800 /* IPv4 */
#endif
#ifndef ETH_P_IPV6
#define ETH_P_IPV6 0x86dd /* IPv6 */
#endif
#ifndef ETH_P_VLAN
#define ETH_P_VLAN 0x8100
#endif
#ifndef ETH_P_MPLS_UNI
#define ETH_P_MPLS_UNI 0x8847
#endif
#ifndef ETH_P_MPLS_MULTI
#define ETH_P_MPLS_MULTI 0x8848
#endif
#ifndef ETH_P_PPPoE
#define ETH_P_PPPoE 0x8864
#endif
#endif /* __NDPI_DEFINE_INCLUDE_FILE__ */
+408
View File
@@ -0,0 +1,408 @@
/*
* ndpi_encryption.h
*
* Copyright (C) 2016-22 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
/* https://wiki.mozilla.org/Security/Cipher_Suites */
/* https://www.oryx-embedded.com/doc/tls__cipher__suites_8h.html */
#define TLS_NULL_WITH_NULL_NULL 0x0000
#define TLS_RSA_EXPORT_WITH_RC4_40_MD5 0x0003
#define TLS_RSA_EXPORT_WITH_RC2_CBC_40_MD5 0x0006
#define TLS_RSA_EXPORT_WITH_DES40_CBC_SHA 0x0008
#define TLS_RSA_WITH_NULL_MD5 0x0001
#define TLS_RSA_WITH_NULL_SHA 0x0002
#define TLS_RSA_WITH_NULL_SHA256 0x003B
#define TLS_RSA_WITH_RC4_128_MD5 0x0004
#define TLS_RSA_WITH_RC4_128_SHA 0x0005
#define TLS_RSA_WITH_IDEA_CBC_SHA 0x0007
#define TLS_RSA_WITH_DES_CBC_SHA 0x0009
#define TLS_RSA_WITH_3DES_EDE_CBC_SHA 0x000A
#define TLS_RSA_WITH_AES_128_CBC_SHA 0x002F
#define TLS_RSA_WITH_AES_256_CBC_SHA 0x0035
#define TLS_RSA_WITH_AES_128_CBC_SHA256 0x003C
#define TLS_RSA_WITH_AES_256_CBC_SHA256 0x003D
#define TLS_RSA_WITH_AES_128_GCM_SHA256 0x009C
#define TLS_RSA_WITH_AES_256_GCM_SHA384 0x009D
#define TLS_RSA_WITH_AES_128_CCM 0xC09C
#define TLS_RSA_WITH_AES_256_CCM 0xC09D
#define TLS_RSA_WITH_AES_128_CCM_8 0xC0A0
#define TLS_RSA_WITH_AES_256_CCM_8 0xC0A1
#define TLS_RSA_WITH_CAMELLIA_128_CBC_SHA 0x0041
#define TLS_RSA_WITH_CAMELLIA_256_CBC_SHA 0x0084
#define TLS_RSA_WITH_CAMELLIA_128_CBC_SHA256 0x00BA
#define TLS_RSA_WITH_CAMELLIA_256_CBC_SHA256 0x00C0
#define TLS_RSA_WITH_CAMELLIA_128_GCM_SHA256 0xC07A
#define TLS_RSA_WITH_CAMELLIA_256_GCM_SHA384 0xC07B
#define TLS_RSA_WITH_SEED_CBC_SHA 0x0096
#define TLS_RSA_WITH_ARIA_128_CBC_SHA256 0xC03C
#define TLS_RSA_WITH_ARIA_256_CBC_SHA384 0xC03D
#define TLS_RSA_WITH_ARIA_128_GCM_SHA256 0xC050
#define TLS_RSA_WITH_ARIA_256_GCM_SHA384 0xC051
#define TLS_DH_RSA_EXPORT_WITH_DES40_CBC_SHA 0x000E
#define TLS_DH_RSA_WITH_DES_CBC_SHA 0x000F
#define TLS_DH_RSA_WITH_3DES_EDE_CBC_SHA 0x0010
#define TLS_DH_RSA_WITH_AES_128_CBC_SHA 0x0031
#define TLS_DH_RSA_WITH_AES_256_CBC_SHA 0x0037
#define TLS_DH_RSA_WITH_AES_128_CBC_SHA256 0x003F
#define TLS_DH_RSA_WITH_AES_256_CBC_SHA256 0x0069
#define TLS_DH_RSA_WITH_AES_128_GCM_SHA256 0x00A0
#define TLS_DH_RSA_WITH_AES_256_GCM_SHA384 0x00A1
#define TLS_DH_RSA_WITH_CAMELLIA_128_CBC_SHA 0x0043
#define TLS_DH_RSA_WITH_CAMELLIA_256_CBC_SHA 0x0086
#define TLS_DH_RSA_WITH_CAMELLIA_128_CBC_SHA256 0x00BC
#define TLS_DH_RSA_WITH_CAMELLIA_256_CBC_SHA256 0x00C2
#define TLS_DH_RSA_WITH_CAMELLIA_128_GCM_SHA256 0xC07E
#define TLS_DH_RSA_WITH_CAMELLIA_256_GCM_SHA384 0xC07F
#define TLS_DH_RSA_WITH_SEED_CBC_SHA 0x0098
#define TLS_DH_RSA_WITH_ARIA_128_CBC_SHA256 0xC040
#define TLS_DH_RSA_WITH_ARIA_256_CBC_SHA384 0xC041
#define TLS_DH_RSA_WITH_ARIA_128_GCM_SHA256 0xC054
#define TLS_DH_RSA_WITH_ARIA_256_GCM_SHA384 0xC055
#define TLS_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA 0x0014
#define TLS_DHE_RSA_WITH_DES_CBC_SHA 0x0015
#define TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA 0x0016
#define TLS_DHE_RSA_WITH_AES_128_CBC_SHA 0x0033
#define TLS_DHE_RSA_WITH_AES_256_CBC_SHA 0x0039
#define TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 0x0067
#define TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 0x006B
#define TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 0x009E
#define TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 0x009F
#define TLS_DHE_RSA_WITH_AES_128_CCM 0xC09E
#define TLS_DHE_RSA_WITH_AES_256_CCM 0xC09F
#define TLS_DHE_RSA_WITH_AES_128_CCM_8 0xC0A2
#define TLS_DHE_RSA_WITH_AES_256_CCM_8 0xC0A3
#define TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA 0x0045
#define TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA 0x0088
#define TLS_DHE_RSA_WITH_CAMELLIA_128_CBC_SHA256 0x00BE
#define TLS_DHE_RSA_WITH_CAMELLIA_256_CBC_SHA256 0x00C4
#define TLS_DHE_RSA_WITH_CAMELLIA_128_GCM_SHA256 0xC07C
#define TLS_DHE_RSA_WITH_CAMELLIA_256_GCM_SHA384 0xC07D
#define TLS_DHE_RSA_WITH_SEED_CBC_SHA 0x009A
#define TLS_DHE_RSA_WITH_ARIA_128_CBC_SHA256 0xC044
#define TLS_DHE_RSA_WITH_ARIA_256_CBC_SHA384 0xC045
#define TLS_DHE_RSA_WITH_ARIA_128_GCM_SHA256 0xC052
#define TLS_DHE_RSA_WITH_ARIA_256_GCM_SHA384 0xC053
#define TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256 0xCCAA
#define TLS_DH_DSS_EXPORT_WITH_DES40_CBC_SHA 0x000B
#define TLS_DH_DSS_WITH_DES_CBC_SHA 0x000C
#define TLS_DH_DSS_WITH_3DES_EDE_CBC_SHA 0x000D
#define TLS_DH_DSS_WITH_AES_128_CBC_SHA 0x0030
#define TLS_DH_DSS_WITH_AES_256_CBC_SHA 0x0036
#define TLS_DH_DSS_WITH_AES_128_CBC_SHA256 0x003E
#define TLS_DH_DSS_WITH_AES_256_CBC_SHA256 0x0068
#define TLS_DH_DSS_WITH_AES_128_GCM_SHA256 0x00A4
#define TLS_DH_DSS_WITH_AES_256_GCM_SHA384 0x00A5
#define TLS_DH_DSS_WITH_CAMELLIA_128_CBC_SHA 0x0042
#define TLS_DH_DSS_WITH_CAMELLIA_256_CBC_SHA 0x0085
#define TLS_DH_DSS_WITH_CAMELLIA_128_CBC_SHA256 0x00BB
#define TLS_DH_DSS_WITH_CAMELLIA_256_CBC_SHA256 0x00C1
#define TLS_DH_DSS_WITH_CAMELLIA_128_GCM_SHA256 0xC082
#define TLS_DH_DSS_WITH_CAMELLIA_256_GCM_SHA384 0xC083
#define TLS_DH_DSS_WITH_SEED_CBC_SHA 0x0097
#define TLS_DH_DSS_WITH_ARIA_128_CBC_SHA256 0xC03E
#define TLS_DH_DSS_WITH_ARIA_256_CBC_SHA384 0xC03F
#define TLS_DH_DSS_WITH_ARIA_128_GCM_SHA256 0xC058
#define TLS_DH_DSS_WITH_ARIA_256_GCM_SHA384 0xC059
#define TLS_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA 0x0011
#define TLS_DHE_DSS_WITH_DES_CBC_SHA 0x0012
#define TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA 0x0013
#define TLS_DHE_DSS_WITH_AES_128_CBC_SHA 0x0032
#define TLS_DHE_DSS_WITH_AES_256_CBC_SHA 0x0038
#define TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 0x0040
#define TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 0x006A
#define TLS_DHE_DSS_WITH_AES_128_GCM_SHA256 0x00A2
#define TLS_DHE_DSS_WITH_AES_256_GCM_SHA384 0x00A3
#define TLS_DHE_DSS_WITH_CAMELLIA_128_CBC_SHA 0x0044
#define TLS_DHE_DSS_WITH_CAMELLIA_256_CBC_SHA 0x0087
#define TLS_DHE_DSS_WITH_CAMELLIA_128_CBC_SHA256 0x00BD
#define TLS_DHE_DSS_WITH_CAMELLIA_256_CBC_SHA256 0x00C3
#define TLS_DHE_DSS_WITH_CAMELLIA_128_GCM_SHA256 0xC080
#define TLS_DHE_DSS_WITH_CAMELLIA_256_GCM_SHA384 0xC081
#define TLS_DHE_DSS_WITH_SEED_CBC_SHA 0x0099
#define TLS_DHE_DSS_WITH_ARIA_128_CBC_SHA256 0xC042
#define TLS_DHE_DSS_WITH_ARIA_256_CBC_SHA384 0xC043
#define TLS_DHE_DSS_WITH_ARIA_128_GCM_SHA256 0xC056
#define TLS_DHE_DSS_WITH_ARIA_256_GCM_SHA384 0xC057
#define TLS_DH_ANON_EXPORT_WITH_RC4_40_MD5 0x0017
#define TLS_DH_ANON_EXPORT_WITH_DES40_CBC_SHA 0x0019
#define TLS_DH_ANON_WITH_RC4_128_MD5 0x0018
#define TLS_DH_ANON_WITH_DES_CBC_SHA 0x001A
#define TLS_DH_ANON_WITH_3DES_EDE_CBC_SHA 0x001B
#define TLS_DH_ANON_WITH_AES_128_CBC_SHA 0x0034
#define TLS_DH_ANON_WITH_AES_256_CBC_SHA 0x003A
#define TLS_DH_ANON_WITH_AES_128_CBC_SHA256 0x006C
#define TLS_DH_ANON_WITH_AES_256_CBC_SHA256 0x006D
#define TLS_DH_ANON_WITH_AES_128_GCM_SHA256 0x00A6
#define TLS_DH_ANON_WITH_AES_256_GCM_SHA384 0x00A7
#define TLS_DH_ANON_WITH_CAMELLIA_128_CBC_SHA 0x0046
#define TLS_DH_ANON_WITH_CAMELLIA_256_CBC_SHA 0x0089
#define TLS_DH_ANON_WITH_CAMELLIA_128_CBC_SHA256 0x00BF
#define TLS_DH_ANON_WITH_CAMELLIA_256_CBC_SHA256 0x00C5
#define TLS_DH_ANON_WITH_CAMELLIA_128_GCM_SHA256 0xC084
#define TLS_DH_ANON_WITH_CAMELLIA_256_GCM_SHA384 0xC085
#define TLS_DH_ANON_WITH_SEED_CBC_SHA 0x009B
#define TLS_DH_ANON_WITH_ARIA_128_CBC_SHA256 0xC046
#define TLS_DH_ANON_WITH_ARIA_256_CBC_SHA384 0xC047
#define TLS_DH_ANON_WITH_ARIA_128_GCM_SHA256 0xC05A
#define TLS_DH_ANON_WITH_ARIA_256_GCM_SHA384 0xC05B
#define TLS_ECDH_RSA_WITH_NULL_SHA 0xC00B
#define TLS_ECDH_RSA_WITH_RC4_128_SHA 0xC00C
#define TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA 0xC00D
#define TLS_ECDH_RSA_WITH_AES_128_CBC_SHA 0xC00E
#define TLS_ECDH_RSA_WITH_AES_256_CBC_SHA 0xC00F
#define TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256 0xC029
#define TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 0xC02A
#define TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256 0xC031
#define TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384 0xC032
#define TLS_ECDH_RSA_WITH_CAMELLIA_128_CBC_SHA256 0xC078
#define TLS_ECDH_RSA_WITH_CAMELLIA_256_CBC_SHA384 0xC079
#define TLS_ECDH_RSA_WITH_CAMELLIA_128_GCM_SHA256 0xC08C
#define TLS_ECDH_RSA_WITH_CAMELLIA_256_GCM_SHA384 0xC08D
#define TLS_ECDH_RSA_WITH_ARIA_128_CBC_SHA256 0xC04E
#define TLS_ECDH_RSA_WITH_ARIA_256_CBC_SHA384 0xC04F
#define TLS_ECDH_RSA_WITH_ARIA_128_GCM_SHA256 0xC062
#define TLS_ECDH_RSA_WITH_ARIA_256_GCM_SHA384 0xC063
#define TLS_ECDHE_RSA_WITH_NULL_SHA 0xC010
#define TLS_ECDHE_RSA_WITH_RC4_128_SHA 0xC011
#define TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA 0xC012
#define TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA 0xC013
#define TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA 0xC014
#define TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 0xC027
#define TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 0xC028
#define TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 0xC02F
#define TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 0xC030
#define TLS_ECDHE_RSA_WITH_CAMELLIA_128_CBC_SHA256 0xC076
#define TLS_ECDHE_RSA_WITH_CAMELLIA_256_CBC_SHA384 0xC077
#define TLS_ECDHE_RSA_WITH_CAMELLIA_128_GCM_SHA256 0xC08A
#define TLS_ECDHE_RSA_WITH_CAMELLIA_256_GCM_SHA384 0xC08B
#define TLS_ECDHE_RSA_WITH_ARIA_128_CBC_SHA256 0xC04C
#define TLS_ECDHE_RSA_WITH_ARIA_256_CBC_SHA384 0xC04D
#define TLS_ECDHE_RSA_WITH_ARIA_128_GCM_SHA256 0xC060
#define TLS_ECDHE_RSA_WITH_ARIA_256_GCM_SHA384 0xC061
#define TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 0xCCA8
#define TLS_ECDH_ECDSA_WITH_NULL_SHA 0xC001
#define TLS_ECDH_ECDSA_WITH_RC4_128_SHA 0xC002
#define TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA 0xC003
#define TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA 0xC004
#define TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA 0xC005
#define TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256 0xC025
#define TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 0xC026
#define TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256 0xC02D
#define TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384 0xC02E
#define TLS_ECDH_ECDSA_WITH_CAMELLIA_128_CBC_SHA256 0xC074
#define TLS_ECDH_ECDSA_WITH_CAMELLIA_256_CBC_SHA384 0xC075
#define TLS_ECDH_ECDSA_WITH_CAMELLIA_128_GCM_SHA256 0xC088
#define TLS_ECDH_ECDSA_WITH_CAMELLIA_256_GCM_SHA384 0xC089
#define TLS_ECDH_ECDSA_WITH_ARIA_128_CBC_SHA256 0xC04A
#define TLS_ECDH_ECDSA_WITH_ARIA_256_CBC_SHA384 0xC04B
#define TLS_ECDH_ECDSA_WITH_ARIA_128_GCM_SHA256 0xC05E
#define TLS_ECDH_ECDSA_WITH_ARIA_256_GCM_SHA384 0xC05F
#define TLS_ECDHE_ECDSA_WITH_NULL_SHA 0xC006
#define TLS_ECDHE_ECDSA_WITH_RC4_128_SHA 0xC007
#define TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA 0xC008
#define TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA 0xC009
#define TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA 0xC00A
#define TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 0xC023
#define TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 0xC024
#define TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 0xC02B
#define TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 0xC02C
#define TLS_ECDHE_ECDSA_WITH_AES_128_CCM 0xC0AC
#define TLS_ECDHE_ECDSA_WITH_AES_256_CCM 0xC0AD
#define TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8 0xC0AE
#define TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8 0xC0AF
#define TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_CBC_SHA256 0xC072
#define TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_CBC_SHA384 0xC073
#define TLS_ECDHE_ECDSA_WITH_CAMELLIA_128_GCM_SHA256 0xC086
#define TLS_ECDHE_ECDSA_WITH_CAMELLIA_256_GCM_SHA384 0xC087
#define TLS_ECDHE_ECDSA_WITH_ARIA_128_CBC_SHA256 0xC048
#define TLS_ECDHE_ECDSA_WITH_ARIA_256_CBC_SHA384 0xC049
#define TLS_ECDHE_ECDSA_WITH_ARIA_128_GCM_SHA256 0xC05C
#define TLS_ECDHE_ECDSA_WITH_ARIA_256_GCM_SHA384 0xC05D
#define TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 0xCCA9
#define TLS_ECDH_ANON_WITH_NULL_SHA 0xC015
#define TLS_ECDH_ANON_WITH_RC4_128_SHA 0xC016
#define TLS_ECDH_ANON_WITH_3DES_EDE_CBC_SHA 0xC017
#define TLS_ECDH_ANON_WITH_AES_128_CBC_SHA 0xC018
#define TLS_ECDH_ANON_WITH_AES_256_CBC_SHA 0xC019
#define TLS_PSK_WITH_NULL_SHA 0x002C
#define TLS_PSK_WITH_NULL_SHA256 0x00B0
#define TLS_PSK_WITH_NULL_SHA384 0x00B1
#define TLS_PSK_WITH_RC4_128_SHA 0x008A
#define TLS_PSK_WITH_3DES_EDE_CBC_SHA 0x008B
#define TLS_PSK_WITH_AES_128_CBC_SHA 0x008C
#define TLS_PSK_WITH_AES_256_CBC_SHA 0x008D
#define TLS_PSK_WITH_AES_128_CBC_SHA256 0x00AE
#define TLS_PSK_WITH_AES_256_CBC_SHA384 0x00AF
#define TLS_PSK_WITH_AES_128_GCM_SHA256 0x00A8
#define TLS_PSK_WITH_AES_256_GCM_SHA384 0x00A9
#define TLS_PSK_WITH_AES_128_CCM 0xC0A4
#define TLS_PSK_WITH_AES_256_CCM 0xC0A5
#define TLS_PSK_WITH_AES_128_CCM_8 0xC0A8
#define TLS_PSK_WITH_AES_256_CCM_8 0xC0A9
#define TLS_PSK_WITH_CAMELLIA_128_CBC_SHA256 0xC094
#define TLS_PSK_WITH_CAMELLIA_256_CBC_SHA384 0xC095
#define TLS_PSK_WITH_CAMELLIA_128_GCM_SHA256 0xC08E
#define TLS_PSK_WITH_CAMELLIA_256_GCM_SHA384 0xC08F
#define TLS_PSK_WITH_ARIA_128_CBC_SHA256 0xC064
#define TLS_PSK_WITH_ARIA_256_CBC_SHA384 0xC065
#define TLS_PSK_WITH_ARIA_128_GCM_SHA256 0xC06A
#define TLS_PSK_WITH_ARIA_256_GCM_SHA384 0xC06B
#define TLS_PSK_WITH_CHACHA20_POLY1305_SHA256 0xCCAB
#define TLS_RSA_PSK_WITH_NULL_SHA 0x002E
#define TLS_RSA_PSK_WITH_NULL_SHA256 0x00B8
#define TLS_RSA_PSK_WITH_NULL_SHA384 0x00B9
#define TLS_RSA_PSK_WITH_RC4_128_SHA 0x0092
#define TLS_RSA_PSK_WITH_3DES_EDE_CBC_SHA 0x0093
#define TLS_RSA_PSK_WITH_AES_128_CBC_SHA 0x0094
#define TLS_RSA_PSK_WITH_AES_256_CBC_SHA 0x0095
#define TLS_RSA_PSK_WITH_AES_128_CBC_SHA256 0x00B6
#define TLS_RSA_PSK_WITH_AES_256_CBC_SHA384 0x00B7
#define TLS_RSA_PSK_WITH_AES_128_GCM_SHA256 0x00AC
#define TLS_RSA_PSK_WITH_AES_256_GCM_SHA384 0x00AD
#define TLS_RSA_PSK_WITH_CAMELLIA_128_CBC_SHA256 0xC098
#define TLS_RSA_PSK_WITH_CAMELLIA_256_CBC_SHA384 0xC099
#define TLS_RSA_PSK_WITH_CAMELLIA_128_GCM_SHA256 0xC092
#define TLS_RSA_PSK_WITH_CAMELLIA_256_GCM_SHA384 0xC093
#define TLS_RSA_PSK_WITH_ARIA_128_CBC_SHA256 0xC068
#define TLS_RSA_PSK_WITH_ARIA_256_CBC_SHA384 0xC069
#define TLS_RSA_PSK_WITH_ARIA_128_GCM_SHA256 0xC06E
#define TLS_RSA_PSK_WITH_ARIA_256_GCM_SHA384 0xC06F
#define TLS_RSA_PSK_WITH_CHACHA20_POLY1305_SHA256 0xCCAE
#define TLS_DHE_PSK_WITH_NULL_SHA 0x002D
#define TLS_DHE_PSK_WITH_NULL_SHA256 0x00B4
#define TLS_DHE_PSK_WITH_NULL_SHA384 0x00B5
#define TLS_DHE_PSK_WITH_RC4_128_SHA 0x008E
#define TLS_DHE_PSK_WITH_3DES_EDE_CBC_SHA 0x008F
#define TLS_DHE_PSK_WITH_AES_128_CBC_SHA 0x0090
#define TLS_DHE_PSK_WITH_AES_256_CBC_SHA 0x0091
#define TLS_DHE_PSK_WITH_AES_128_CBC_SHA256 0x00B2
#define TLS_DHE_PSK_WITH_AES_256_CBC_SHA384 0x00B3
#define TLS_DHE_PSK_WITH_AES_128_GCM_SHA256 0x00AA
#define TLS_DHE_PSK_WITH_AES_256_GCM_SHA384 0x00AB
#define TLS_DHE_PSK_WITH_AES_128_CCM 0xC0A6
#define TLS_DHE_PSK_WITH_AES_256_CCM 0xC0A7
#define TLS_DHE_PSK_WITH_AES_128_CCM_8 0xC0AA
#define TLS_DHE_PSK_WITH_AES_256_CCM_8 0xC0AB
#define TLS_DHE_PSK_WITH_CAMELLIA_128_CBC_SHA256 0xC096
#define TLS_DHE_PSK_WITH_CAMELLIA_256_CBC_SHA384 0xC097
#define TLS_DHE_PSK_WITH_CAMELLIA_128_GCM_SHA256 0xC090
#define TLS_DHE_PSK_WITH_CAMELLIA_256_GCM_SHA384 0xC091
#define TLS_DHE_PSK_WITH_ARIA_128_CBC_SHA256 0xC066
#define TLS_DHE_PSK_WITH_ARIA_256_CBC_SHA384 0xC067
#define TLS_DHE_PSK_WITH_ARIA_128_GCM_SHA256 0xC06C
#define TLS_DHE_PSK_WITH_ARIA_256_GCM_SHA384 0xC06D
#define TLS_DHE_PSK_WITH_CHACHA20_POLY1305_SHA256 0xCCAD
#define TLS_ECDHE_PSK_WITH_NULL_SHA 0xC039
#define TLS_ECDHE_PSK_WITH_NULL_SHA256 0xC03A
#define TLS_ECDHE_PSK_WITH_NULL_SHA384 0xC03B
#define TLS_ECDHE_PSK_WITH_RC4_128_SHA 0xC033
#define TLS_ECDHE_PSK_WITH_3DES_EDE_CBC_SHA 0xC034
#define TLS_ECDHE_PSK_WITH_AES_128_CBC_SHA 0xC035
#define TLS_ECDHE_PSK_WITH_AES_256_CBC_SHA 0xC036
#define TLS_ECDHE_PSK_WITH_AES_128_CBC_SHA256 0xC037
#define TLS_ECDHE_PSK_WITH_AES_256_CBC_SHA384 0xC038
#define TLS_ECDHE_PSK_WITH_AES_128_GCM_SHA256 0xD001
#define TLS_ECDHE_PSK_WITH_AES_256_GCM_SHA384 0xD002
#define TLS_ECDHE_PSK_WITH_AES_128_CCM_SHA256 0xD005
#define TLS_ECDHE_PSK_WITH_AES_128_CCM_8_SHA256 0xD003
#define TLS_ECDHE_PSK_WITH_CAMELLIA_128_CBC_SHA256 0xC09A
#define TLS_ECDHE_PSK_WITH_CAMELLIA_256_CBC_SHA384 0xC09B
#define TLS_ECDHE_PSK_WITH_ARIA_128_CBC_SHA256 0xC070
#define TLS_ECDHE_PSK_WITH_ARIA_256_CBC_SHA384 0xC071
#define TLS_ECDHE_PSK_WITH_CHACHA20_POLY1305_SHA256 0xCCAC
#define TLS_KRB5_EXPORT_WITH_RC4_40_MD5 0x002B
#define TLS_KRB5_EXPORT_WITH_RC4_40_SHA 0x0028
#define TLS_KRB5_EXPORT_WITH_RC2_CBC_40_MD5 0x002A
#define TLS_KRB5_EXPORT_WITH_RC2_CBC_40_SHA 0x0027
#define TLS_KRB5_EXPORT_WITH_DES_CBC_40_MD5 0x0029
#define TLS_KRB5_EXPORT_WITH_DES_CBC_40_SHA 0x0026
#define TLS_KRB5_WITH_RC4_128_MD5 0x0024
#define TLS_KRB5_WITH_RC4_128_SHA 0x0020
#define TLS_KRB5_WITH_IDEA_CBC_MD5 0x0025
#define TLS_KRB5_WITH_IDEA_CBC_SHA 0x0021
#define TLS_KRB5_WITH_DES_CBC_MD5 0x0022
#define TLS_KRB5_WITH_DES_CBC_SHA 0x001E
#define TLS_KRB5_WITH_3DES_EDE_CBC_MD5 0x0023
#define TLS_KRB5_WITH_3DES_EDE_CBC_SHA 0x001F
#define TLS_SRP_SHA_WITH_3DES_EDE_CBC_SHA 0xC01A
#define TLS_SRP_SHA_WITH_AES_128_CBC_SHA 0xC01D
#define TLS_SRP_SHA_WITH_AES_256_CBC_SHA 0xC020
#define TLS_SRP_SHA_RSA_WITH_3DES_EDE_CBC_SHA 0xC01B
#define TLS_SRP_SHA_RSA_WITH_AES_128_CBC_SHA 0xC01E
#define TLS_SRP_SHA_RSA_WITH_AES_256_CBC_SHA 0xC021
#define TLS_SRP_SHA_DSS_WITH_3DES_EDE_CBC_SHA 0xC01C
#define TLS_SRP_SHA_DSS_WITH_AES_128_CBC_SHA 0xC01F
#define TLS_SRP_SHA_DSS_WITH_AES_256_CBC_SHA 0xC022
#define TLS_ECCPWD_WITH_AES_128_GCM_SHA256 0xC0B0
#define TLS_ECCPWD_WITH_AES_256_GCM_SHA384 0xC0B1
#define TLS_ECCPWD_WITH_AES_128_CCM_SHA256 0xC0B2
#define TLS_ECCPWD_WITH_AES_256_CCM_SHA384 0xC0B3
#define TLS_GOSTR341112_256_WITH_KUZNYECHIK_CTR_OMAC 0xC100
#define TLS_GOSTR341112_256_WITH_MAGMA_CTR_OMAC 0xC101
#define TLS_GOSTR341112_256_WITH_28147_CNT_IMIT 0xC102
#define TLS_AES_128_GCM_SHA256 0x1301
#define TLS_AES_256_GCM_SHA384 0x1302
#define TLS_AES_128_CCM_SHA256 0x1304
#define TLS_AES_128_CCM_8_SHA256 0x1305
#define TLS_CHACHA20_POLY1305_SHA256 0x1303
#define TLS_SM4_GCM_SM3 0x00C6
#define TLS_SM4_CCM_SM3 0x00C7
#define TLS_SHA256_SHA256 0xC0B4
#define TLS_SHA384_SHA384 0xC0B5
#define TLS_EMPTY_RENEGOTIATION_INFO_SCSV 0x00FF
#define TLS_FALLBACK_SCSV 0x5600
/* https://datatracker.ietf.org/doc/html/rfc8701 */
#define TLS_CIPHER_GREASE_RESERVED_0 0x0A0A
#define TLS_CIPHER_GREASE_RESERVED_1 0x1A1A
#define TLS_CIPHER_GREASE_RESERVED_2 0x2A2A
#define TLS_CIPHER_GREASE_RESERVED_3 0x3A3A
#define TLS_CIPHER_GREASE_RESERVED_4 0x4A4A
#define TLS_CIPHER_GREASE_RESERVED_5 0x5A5A
#define TLS_CIPHER_GREASE_RESERVED_6 0x6A6A
#define TLS_CIPHER_GREASE_RESERVED_7 0x7A7A
#define TLS_CIPHER_GREASE_RESERVED_8 0x8A8A
#define TLS_CIPHER_GREASE_RESERVED_9 0x9A9A
#define TLS_CIPHER_GREASE_RESERVED_A 0xAAAA
#define TLS_CIPHER_GREASE_RESERVED_B 0xBABA
#define TLS_CIPHER_GREASE_RESERVED_C 0xCACA
#define TLS_CIPHER_GREASE_RESERVED_D 0xDADA
#define TLS_CIPHER_GREASE_RESERVED_E 0xEAEA
#define TLS_CIPHER_GREASE_RESERVED_F 0xFAFA
/* ********************************************** */
/* Signature algorithms */
#define RSA_PKCS1_SHA1 0x0201
#define ECDSA_SHA1 0x0203
#define RSA_PKCS1_SHA256 0x0401
#define ECDSA_SECP256R1_SHA256 0x0403
#define RSA_PKCS1_SHA384 0x0501
#define ECDSA_SECP384R1_SHA384 0x0503
#define RSA_PKCS1_SHA512 0x0601
#define ECDSA_SECP521R1_SHA512 0x0603
#define RSA_PSS_RSAE_SHA256 0x0804
#define RSA_PSS_RSAE_SHA384 0x0805
#define RSA_PSS_RSAE_SHA512 0x0806
#define ED25519 0x0807
#define ED448 0x0808
#define RSA_PSS_PSS_SHA256 0x0809
#define RSA_PSS_PSS_SHA384 0x080A
#define RSA_PSS_PSS_SHA512 0x080B
+1 -5
View File
@@ -32,13 +32,13 @@
#include <ctype.h>
#include <time.h>
#include <limits.h>
#include <stdbool.h>
#ifdef WIN32
#include "ndpi_win32.h"
#else
#include <sys/types.h>
#include <sys/param.h>
#include <pthread.h>
#include <arpa/inet.h>
#include <sys/time.h>
#include <sys/socket.h>
@@ -57,10 +57,6 @@
#if defined __NetBSD__ || defined __OpenBSD__
#include <netinet/in_systm.h>
#if defined __OpenBSD__
#include <pthread.h>
#endif
#endif
#endif
#endif
+28 -25
View File
@@ -1,7 +1,7 @@
/*
* ndpi_main.h
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -31,6 +31,10 @@
#include "ndpi_api.h"
#include "ndpi_protocols.h"
/* used by ndpi_set_proto_subprotocols */
#define NDPI_PROTOCOL_NO_MORE_SUBPROTOCOLS (-1)
#define NDPI_PROTOCOL_MATCHED_BY_CONTENT (-2)
#ifdef __cplusplus
extern "C" {
#endif
@@ -73,7 +77,8 @@ extern "C" {
void ndpi_set_detected_protocol(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow,
u_int16_t upper_detected_protocol,
u_int16_t lower_detected_protocol);
u_int16_t lower_detected_protocol,
ndpi_confidence_t confidence);
extern void ndpi_parse_packet_line_info(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow);
@@ -83,28 +88,20 @@ extern "C" {
extern u_int16_t ndpi_check_for_email_address(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow, u_int16_t counter);
extern void ndpi_int_change_packet_protocol(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow,
u_int16_t upper_detected_protocol,
u_int16_t lower_detected_protocol);
extern void ndpi_int_change_protocol(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow,
u_int16_t upper_detected_protocol,
u_int16_t lower_detected_protocol);
extern void ndpi_int_change_category(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow,
ndpi_protocol_category_t protocol_category);
extern void ndpi_set_proto_subprotocols(struct ndpi_detection_module_struct *ndpi_mod,
int protoId, ...);
extern void ndpi_set_proto_defaults(struct ndpi_detection_module_struct *ndpi_mod,
ndpi_protocol_breed_t protoBreed, u_int16_t protoId,
u_int8_t can_have_a_subprotocol,
u_int16_t tcp_alias_protoId[2],
u_int16_t udp_alias_protoId[2], char *protoName,
u_int8_t is_cleartext,
ndpi_protocol_breed_t protoBreed, u_int16_t protoId, char *protoName,
ndpi_protocol_category_t protoCategory,
ndpi_port_range *tcpDefPorts,
ndpi_port_range *udpDefPorts);
extern void ndpi_int_reset_packet_protocol(struct ndpi_packet_struct *packet);
extern void ndpi_int_reset_protocol(struct ndpi_flow_struct *flow);
extern int ndpi_packet_src_ip_eql(const struct ndpi_packet_struct *packet, const ndpi_ip_addr_t * ip);
@@ -127,13 +124,7 @@ extern "C" {
extern u_int8_t ndpi_is_proto(ndpi_protocol proto, u_int16_t p);
extern u_int16_t ndpi_get_lower_proto(ndpi_protocol p);
extern int ndpi_get_protocol_id_master_proto(struct ndpi_detection_module_struct *ndpi_struct,
u_int16_t protocol_id,
u_int16_t** tcp_master_proto,
u_int16_t** udp_master_proto);
#/* NDPI_PROTOCOL_NETBIOS */
int ndpi_netbios_name_interpret(char *in, size_t inlen, char *out, u_int out_len);
#ifdef NDPI_ENABLE_DEBUG_MESSAGES
void ndpi_debug_get_last_log_function_line(struct ndpi_detection_module_struct *ndpi_struct,
const char **file, const char **func, u_int32_t * line);
@@ -150,9 +141,21 @@ extern "C" {
#define ndpi_match_strprefix(payload, payload_len, str) \
ndpi_match_prefix((payload), (payload_len), (str), (sizeof(str)-1))
#ifdef NDPI_DETECTION_SUPPORT_IPV6
int ndpi_handle_ipv6_extension_headers(struct ndpi_detection_module_struct *ndpi_str, const u_int8_t ** l4ptr, u_int16_t * l4len, u_int8_t * nxt_hdr);
#endif
int ndpi_handle_ipv6_extension_headers(u_int16_t l3len,
const u_int8_t ** l4ptr, u_int16_t * l4len,
u_int8_t * nxt_hdr);
void ndpi_set_risk(struct ndpi_detection_module_struct *ndpi_str,
struct ndpi_flow_struct *flow, ndpi_risk_enum r);
int ndpi_isset_risk(struct ndpi_detection_module_struct *ndpi_str,
struct ndpi_flow_struct *flow, ndpi_risk_enum r);
int ndpi_is_printable_string(char * const str, size_t len);
#define NDPI_ENTROPY_ENCRYPTED_OR_RANDOM(entropy) (entropy > 7.0f)
float ndpi_entropy(u_int8_t const * const buf, size_t len);
void load_common_alpns(struct ndpi_detection_module_struct *ndpi_str);
u_int8_t is_a_common_alpn(struct ndpi_detection_module_struct *ndpi_str,
const char *alpn_to_check, u_int alpn_to_check_len);
char *ndpi_hostname_sni_set(struct ndpi_flow_struct *flow, const u_int8_t *value, size_t value_len);
#ifdef __cplusplus
}
+107
View File
@@ -0,0 +1,107 @@
/*
* ndpi_patricia_typedef.h
*
* Copyright (C) 2011-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
/*
* $Id: ndpi_patricia.h,v 1.6 2005/12/07 20:53:01 dplonka Exp $
* Dave Plonka <plonka@doit.wisc.edu>
*
* This product includes software developed by the University of Michigan,
* Merit Network, Inc., and their contributors.
*
* This file had been called "radix.h" in the MRT sources.
*
* I renamed it to "ndpi_patricia.h" since it's not an implementation of a general
* radix trie. Also, pulled in various requirements from "mrt.h" and added
* some other things it could be used as a standalone API.
https://github.com/deepfield/MRT/blob/master/COPYRIGHT
Copyright (c) 1999-2013
The Regents of the University of Michigan ("The Regents") and Merit
Network, Inc.
Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef _NDPI_PATRICIA_TYPEDEF_H_
#define _NDPI_PATRICIA_TYPEDEF_H_
/* pointer to usr data (ex. route flap info) */
union ndpi_patricia_node_value_t {
/* User-defined values */
union {
struct {
u_int32_t user_value, additional_user_value;
} uv32;
u_int64_t uv64;
void *user_data;
} u;
};
typedef struct _ndpi_prefix_t {
u_int16_t family; /* AF_INET | AF_INET6 */
u_int16_t bitlen; /* same as mask? */
int ref_count; /* reference count */
union {
struct in_addr sin;
struct in6_addr sin6;
u_int8_t mac[6];
} add;
} ndpi_prefix_t;
typedef struct _ndpi_patricia_node_t {
u_int16_t bit; /* flag if this node used */
ndpi_prefix_t *prefix; /* who we are in patricia tree */
struct _ndpi_patricia_node_t *l, *r; /* left and right children */
struct _ndpi_patricia_node_t *parent;/* may be used */
void *data; /* pointer to data */
union ndpi_patricia_node_value_t value;
} ndpi_patricia_node_t;
typedef struct _ndpi_patricia_tree_t {
ndpi_patricia_node_t *head;
u_int16_t maxbits; /* for IP, 32 bit addresses */
int num_active_node; /* for debug purpose */
} ndpi_patricia_tree_t;
#endif /* _NDPI_PATRICIA_TYPEDEF_H_ */
+47 -20
View File
@@ -1,7 +1,7 @@
/*
* ndpi_protocol_ids.h
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -25,7 +25,6 @@
#ifndef __NDPI_PROTOCOLS_IDS_H__
#define __NDPI_PROTOCOLS_IDS_H__
#define NDPI_DETECTION_SUPPORT_IPV6
#define NDPI_PROTOCOL_SIZE 2
typedef enum {
@@ -59,7 +58,7 @@ typedef enum {
NDPI_PROTOCOL_COAP = 27,
NDPI_PROTOCOL_VMWARE = 28,
NDPI_PROTOCOL_MAIL_SMTPS = 29,
NDPI_PROTOCOL_FBZERO = 30,
NDPI_PROTOCOL_DTLS = 30,
NDPI_PROTOCOL_UBNTAC2 = 31, /* Ubiquity UBNT AirControl = 2 - Thomas Fjellstrom <thomas+ndpi@fjellstrom.ca> */
NDPI_PROTOCOL_KONTIKI = 32,
NDPI_PROTOCOL_OPENFT = 33,
@@ -82,25 +81,25 @@ typedef enum {
NDPI_PROTOCOL_RTSP = 50,
NDPI_PROTOCOL_MAIL_IMAPS = 51,
NDPI_PROTOCOL_ICECAST = 52,
NDPI_PROTOCOL_FREE_53 = 53,
NDPI_PROTOCOL_CPHA = 53,
NDPI_PROTOCOL_PPSTREAM = 54,
NDPI_PROTOCOL_ZATTOO = 55,
NDPI_PROTOCOL_SHOUTCAST = 56,
NDPI_PROTOCOL_SOPCAST = 57,
NDPI_PROTOCOL_DISCORD = 58,
NDPI_PROTOCOL_TVUPLAYER = 59,
NDPI_PROTOCOL_HTTP_DOWNLOAD = 60,
NDPI_PROTOCOL_MONGODB = 60, /* Leonn Paiva <leonn.paiva@gmail.com> */
NDPI_PROTOCOL_QQLIVE = 61,
NDPI_PROTOCOL_THUNDER = 62,
NDPI_PROTOCOL_SOULSEEK = 63,
NDPI_PROTOCOL_PS_VUE = 64,
NDPI_PROTOCOL_OCSP = 63,
NDPI_PROTOCOL_FREE_64 = 64, /* FREE */
NDPI_PROTOCOL_IRC = 65,
NDPI_PROTOCOL_AYIYA = 66,
NDPI_PROTOCOL_UNENCRYPTED_JABBER = 67,
NDPI_PROTOCOL_JABBER = 67,
NDPI_PROTOCOL_NATS = 68,
NDPI_PROTOCOL_FREE_69 = 69, /* Free */
NDPI_PROTOCOL_AMONG_US = 69, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_YAHOO = 70,
NDPI_PROTOCOL_FREE_71 = 71, /* Free */
NDPI_PROTOCOL_DISNEYPLUS = 71, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_GOOGLE_PLUS = 72,
NDPI_PROTOCOL_IP_VRRP = 73,
NDPI_PROTOCOL_STEAM = 74, /* Tomasz Bujlow <tomasz@skatnet.dk> */
@@ -119,7 +118,7 @@ typedef enum {
NDPI_PROTOCOL_RTP = 87,
NDPI_PROTOCOL_RDP = 88,
NDPI_PROTOCOL_VNC = 89,
NDPI_PROTOCOL_FREE90 = 90, /* Free */
NDPI_PROTOCOL_TUMBLR = 90, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_TLS = 91,
NDPI_PROTOCOL_SSH = 92,
NDPI_PROTOCOL_USENET = 93,
@@ -139,7 +138,7 @@ typedef enum {
NDPI_PROTOCOL_FIESTA = 107,
NDPI_PROTOCOL_FLORENSIA = 108,
NDPI_PROTOCOL_GUILDWARS = 109,
NDPI_PROTOCOL_HTTP_ACTIVESYNC = 110,
NDPI_PROTOCOL_AMAZON_ALEXA = 110, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_KERBEROS = 111,
NDPI_PROTOCOL_LDAP = 112,
NDPI_PROTOCOL_MAPLESTORY = 113,
@@ -154,7 +153,7 @@ typedef enum {
NDPI_PROTOCOL_GMAIL = 122,
NDPI_PROTOCOL_GOOGLE_MAPS = 123,
NDPI_PROTOCOL_YOUTUBE = 124,
NDPI_PROTOCOL_SKYPE = 125,
NDPI_PROTOCOL_SKYPE_TEAMS = 125,
NDPI_PROTOCOL_GOOGLE = 126,
NDPI_PROTOCOL_DCERPC = 127,
NDPI_PROTOCOL_NETFLOW = 128,
@@ -182,7 +181,7 @@ typedef enum {
NDPI_PROTOCOL_LOTUS_NOTES = 150,
NDPI_PROTOCOL_SAP = 151,
NDPI_PROTOCOL_GTP = 152,
NDPI_PROTOCOL_UPNP = 153,
NDPI_PROTOCOL_WSD = 153,
NDPI_PROTOCOL_LLMNR = 154,
NDPI_PROTOCOL_REMOTE_SCAN = 155,
NDPI_PROTOCOL_SPOTIFY = 156,
@@ -212,7 +211,7 @@ typedef enum {
NDPI_PROTOCOL_CNN = 180, /* Tomasz Bujlow <tomasz@skatnet.dk> */
NDPI_PROTOCOL_MEGACO = 181, /* Gianluca Costa <g.costa@xplico.org> */
NDPI_PROTOCOL_REDIS = 182,
NDPI_PROTOCOL_FREE_183 = 183,
NDPI_PROTOCOL_PINTEREST = 183, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_VHUA = 184,
NDPI_PROTOCOL_TELEGRAM = 185, /* Gianluca Costa <g.costa@xplico.org> */
NDPI_PROTOCOL_VEVO = 186,
@@ -225,7 +224,7 @@ typedef enum {
NDPI_PROTOCOL_KAKAOTALK = 193, /* KakaoTalk Chat (no voice call) */
NDPI_PROTOCOL_KAKAOTALK_VOICE = 194, /* KakaoTalk Voice */
NDPI_PROTOCOL_TWITCH = 195, /* Edoardo Dominici <edoaramis@gmail.com> */
NDPI_PROTOCOL_DOH_DOT = 196, /* DoH (DNS over HTTPS), DoT (DNS over TLS) */
NDPI_PROTOCOL_DOH_DOT = 196, /* DoH (DNS over HTTPS), DoT (DNS over TLS), DoQ (DNS over QUIC). TODO: rename in NDPI_PROTOCOL_DOH_DOT_DOQ? */
NDPI_PROTOCOL_WECHAT = 197,
NDPI_PROTOCOL_MPEGTS = 198,
NDPI_PROTOCOL_SNAPCHAT = 199,
@@ -234,7 +233,7 @@ typedef enum {
NDPI_PROTOCOL_IFLIX = 202, /* www.vizuamatix.com R&D team & M.Mallawaarachchie <manoj_ws@yahoo.com> */
NDPI_PROTOCOL_GITHUB = 203,
NDPI_PROTOCOL_BJNP = 204,
NDPI_PROTOCOL_FREE_205 = 205,
NDPI_PROTOCOL_REDDIT = 205, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_WIREGUARD = 206,
NDPI_PROTOCOL_SMPP = 207, /* Damir Franusic <df@release14.org> */
NDPI_PROTOCOL_DNSCRYPT = 208, /* Toni Uhlig <matzeton@googlemail.com> */
@@ -265,8 +264,8 @@ typedef enum {
NDPI_PROTOCOL_LINKEDIN = 233, /* Paulo Angelo <pa@pauloangelo.com> */
NDPI_PROTOCOL_SOUNDCLOUD = 234,
NDPI_PROTOCOL_CSGO = 235, /* Counter-Strike Global Offensive, Dota = 2 */
NDPI_PROTOCOL_LISP = 236,
NDPI_PROTOCOL_DIAMETER = 237,
NDPI_PROTOCOL_LISP = 236,
NDPI_PROTOCOL_DIAMETER = 237,
NDPI_PROTOCOL_APPLE_PUSH = 238,
NDPI_PROTOCOL_GOOGLE_SERVICES = 239,
NDPI_PROTOCOL_AMAZON_VIDEO = 240,
@@ -283,10 +282,38 @@ typedef enum {
NDPI_PROTOCOL_WEBSOCKET = 251, /* Leonn Paiva <leonn.paiva@gmail.com> */
NDPI_PROTOCOL_ANYDESK = 252, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_SOAP = 253, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_APPLE_SIRI = 254, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_SNAPCHAT_CALL = 255,
NDPI_PROTOCOL_HPVIRTGRP = 256, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_GENSHIN_IMPACT = 257, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_ACTIVISION = 258,
NDPI_PROTOCOL_FORTICLIENT = 259,
NDPI_PROTOCOL_Z3950 = 260, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_LIKEE = 261,
NDPI_PROTOCOL_GITLAB = 262,
NDPI_PROTOCOL_AVAST_SECUREDNS = 263, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_CASSANDRA = 264, /* Lucas Santos <lfneiva.santos@gmail.com> */
NDPI_PROTOCOL_AMAZON_AWS = 265,
NDPI_PROTOCOL_SALESFORCE = 266,
NDPI_PROTOCOL_VIMEO = 267,
NDPI_PROTOCOL_FACEBOOK_VOIP = 268,
NDPI_PROTOCOL_SIGNAL_VOIP = 269,
NDPI_PROTOCOL_FUZE = 270,
NDPI_PROTOCOL_GTP_U = 271,
NDPI_PROTOCOL_GTP_C = 272,
NDPI_PROTOCOL_GTP_PRIME = 273,
NDPI_PROTOCOL_ALIBABA = 274,
NDPI_PROTOCOL_CRASHLYSTICS = 275,
NDPI_PROTOCOL_MICROSOFT_AZURE = 276,
NDPI_PROTOCOL_ICLOUD_PRIVATE_RELAY = 277,
NDPI_PROTOCOL_ETHERNET_IP = 278,
NDPI_PROTOCOL_BADOO = 279,
NDPI_PROTOCOL_ACCUWEATHER = 280,
NDPI_PROTOCOL_GOOGLE_CLASSROOM = 281,
#ifdef CUSTOM_NDPI_PROTOCOLS
#include "../../../nDPI-custom/custom_ndpi_protocol_ids.h"
#endif
#endif
/*
IMPORTANT
+15 -6
View File
@@ -1,7 +1,7 @@
/*
* ndpi_protocols.h
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -60,6 +60,8 @@ void init_ayiya_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_in
void init_amqp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_bgp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_bittorrent_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
int ndpi_search_into_bittorrent_cache(struct ndpi_detection_module_struct *ndpi_struct, struct ndpi_flow_struct *flow,
u_int32_t saddr, u_int16_t sport, u_int32_t daddr, u_int16_t dport);
void init_lisp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_teredo_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_ciscovpn_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
@@ -87,7 +89,6 @@ void init_guildwars_dissector(struct ndpi_detection_module_struct *ndpi_struct,
void init_h323_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_halflife2_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_http_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_http_activesync_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_iax_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_icecast_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_ipp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
@@ -143,7 +144,6 @@ void init_smb_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int3
void init_snmp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_socrates_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_sopcast_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_soulseek_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_socks_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_spotify_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_ssh_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
@@ -160,10 +160,9 @@ void init_telegram_dissector(struct ndpi_detection_module_struct *ndpi_struct, u
void init_telnet_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_tftp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_thunder_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_tor_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_tvuplayer_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_usenet_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_upnp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_wsd_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_veohtv_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_vhua_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_viber_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
@@ -193,11 +192,11 @@ void init_fix_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int3
void init_nintendo_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_csgo_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_checkmk_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_cpha_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_apple_push_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_amazon_video_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_whatsapp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_ajp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_fbzero_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_memcached_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_nest_log_sink_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_ookla_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
@@ -212,5 +211,15 @@ void init_s7comm_dissector(struct ndpi_detection_module_struct *ndpi_struct,u_in
void init_websocket_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_soap_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_dnscrypt_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_mongodb_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_among_us_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_hpvirtgrp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_genshin_impact_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_z3950_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_avast_securedns_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_cassandra_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
/* ndpi_main.c */
extern u_int32_t ndpi_ip_port_hash_funct(u_int32_t ip, u_int16_t port);
#endif /* __NDPI_PROTOCOLS_H__ */
File diff suppressed because it is too large Load Diff
-3
View File
@@ -28,9 +28,6 @@
#include <netinet/in.h>
#if defined(__NetBSD__) || defined(__OpenBSD__)
#include <netinet/in_systm.h>
#if defined(__OpenBSD__)
#include <pthread.h>
#endif
#endif
#endif
+34
View File
@@ -0,0 +1,34 @@
#ifndef __NDPI_UTILS_H__
#define __NDPI_UTILS_H__
#include "ndpi_define.h"
#define MYDBG(m, ...) \
printf(" DBG[%s:%s:%u]: \t" m "\n", __FILE__, __FUNCTION__, __LINE__, ##__VA_ARGS__);
// #define NDPI_ENABLE_DEBUG_POINTER_MESSAGES
// #define NDPI_ENABLE_DEBUG_INFO_MESSAGES
// #define NDPI_ENABLE_DEBUG_TRACE_MESSAGES
extern void printRawData(const uint8_t *ptr, size_t len);
//extern uint8_t add_segment_to_buffer( struct ndpi_flow_struct *flow, struct ndpi_tcphdr const * tcph, uint32_t waited);
//extern uint8_t check_for_sequence( struct ndpi_flow_struct *flow, struct ndpi_tcphdr const * tcph);
extern u_int8_t ndpi_ends_with(char *str, char *ends);
/* **************************************** */
/* Can't call libc functions from kernel space, define some stub instead */
#define ndpi_isalpha(ch) (((ch) >= 'a' && (ch) <= 'z') || ((ch) >= 'A' && (ch) <= 'Z'))
#define ndpi_isdigit(ch) ((ch) >= '0' && (ch) <= '9')
#define ndpi_isspace(ch) (((ch) >= '\t' && (ch) <= '\r') || ((ch) == ' '))
#define ndpi_isprint(ch) ((ch) >= 0x20 && (ch) <= 0x7e)
#define ndpi_ispunct(ch) (((ch) >= '!' && (ch) <= '/') || \
((ch) >= ':' && (ch) <= '@') || \
((ch) >= '[' && (ch) <= '`') || \
((ch) >= '{' && (ch) <= '~'))
#endif
+4 -22
View File
@@ -59,8 +59,10 @@
#define IPPROTO_SCTP 132
#endif
#if defined(__MINGW32__) || defined(__MINGW64__)
#undef gettimeofday
#define gettimeofday mingw_gettimeofday
#endif
extern char* strsep(char **sp, char *sep);
@@ -74,30 +76,10 @@ typedef uint u_int32_t;
typedef uint u_int;
typedef unsigned __int64 u_int64_t;
#define pthread_t HANDLE
#define pthread_mutex_t HANDLE
#define pthread_rwlock_t pthread_mutex_t
#define pthread_rwlock_init pthread_mutex_init
#define pthread_rwlock_wrlock pthread_mutex_lock
#define pthread_rwlock_rdlock pthread_mutex_lock
#define pthread_rwlock_unlock pthread_mutex_unlock
#define pthread_rwlock_destroy pthread_mutex_destroy
#define gmtime_r(a, b) memcpy(b, gmtime(a), sizeof(struct tm))
#define gmtime_r(a, b) memcpy(b, gmtime(a), sizeof(struct tm))
#define in_addr_t unsigned long
extern unsigned long waitForNextEvent(unsigned long ulDelay /* ms */);
#define sleep(a /* sec */) waitForNextEvent(1000*a /* ms */)
#define strtok_r strtok_s
#define timegm _mkgmtime
static inline struct tm * localtime_r(const time_t *timep, struct tm * result)
{
struct tm *timeinfo;
timeinfo = localtime(timep);
return timeinfo;
}
#define sleep(a /* sec */) Sleep(1000*a /* ms */)
#endif /* __NDPI_WIN32_H__ */
+9 -9
View File
@@ -5,18 +5,18 @@
# cd src/lib
# make Makefile
#
AR = @AR@
CC = @CC@
RANLIB = @RANLIB@
#
# Installation directories
#
prefix = @prefix@
libdir = ${prefix}/lib
includedir = ${prefix}/include/ndpi
CC = @CC@
CFLAGS += -fPIC -DPIC -I../include -Ithird_party/include -DNDPI_LIB_COMPILATION -Wall @CFLAGS@ @CUSTOM_NDPI@
LDFLAGS = @LDFLAGS@ @ADDITIONAL_LIBS@ @LIBS@
RANLIB = ranlib
CFLAGS += -fPIC -DPIC -I../include -Ithird_party/include -DNDPI_LIB_COMPILATION @CFLAGS@ @CUSTOM_NDPI@
LDFLAGS = @LDFLAGS@
LIBS = @ADDITIONAL_LIBS@ @LIBS@
OBJECTS = $(patsubst protocols/%.c, protocols/%.o, $(wildcard protocols/*.c)) $(patsubst third_party/src/%.c, third_party/src/%.o, $(wildcard third_party/src/*.c)) $(patsubst ./%.c, ./%.o, $(wildcard ./*.c))
HEADERS = $(wildcard ../include/*.h)
@@ -48,16 +48,16 @@ all: $(NDPI_LIBS)
ndpi_main.c: ndpi_content_match.c.inc
$(NDPI_LIB_STATIC): $(OBJECTS)
ar rc $@ $(OBJECTS)
$(AR) rc $@ $(OBJECTS)
$(RANLIB) $@
$(NDPI_LIB_SHARED): $(OBJECTS)
$(CC) -shared -fPIC $(SONAME_FLAG) -o $@ $(OBJECTS) $(LDFLAGS)
$(CC) -shared -fPIC $(SONAME_FLAG) -o $@ $(LDFLAGS) $(OBJECTS) $(LIBS)
ln -fs $(NDPI_LIB_SHARED) $(NDPI_LIB_SHARED_BASE)
ln -fs $(NDPI_LIB_SHARED) $(NDPI_LIB_SHARED_BASE).$(NDPI_VERSION_MAJOR)
%.o: %.c $(HEADERS) Makefile
$(CC) $(CFLAGS) -c $< -o $@
$(CC) $(CPPFLAGS) $(CFLAGS) -c $< -o $@
clean:
/bin/rm -f $(NDPI_LIB_STATIC) $(OBJECTS) *.o *.so *.lo libndpi.so*
+646 -36
View File
@@ -1,7 +1,7 @@
/*
* ndpi_analyze.c
*
* Copyright (C) 2019 - ntop.org
* Copyright (C) 2019-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library.
@@ -39,20 +39,16 @@
void ndpi_init_data_analysis(struct ndpi_analyze_struct *ret, u_int16_t _max_series_len) {
u_int32_t len;
memset(ret, 0, sizeof(struct ndpi_analyze_struct));
memset(ret, 0, sizeof(*ret));
if(_max_series_len > MAX_SERIES_LEN) _max_series_len = MAX_SERIES_LEN;
ret->num_values_array_len = _max_series_len;
if(ret->num_values_array_len > 0) {
len = sizeof(u_int32_t)*ret->num_values_array_len;
if((ret->values = ndpi_malloc(len)) == NULL) {
ndpi_free(ret);
ret = NULL;
} else
len = sizeof(u_int32_t) * ret->num_values_array_len;
if((ret->values = ndpi_malloc(len)) != NULL)
memset(ret->values, 0, len);
} else
ret->values = NULL;
}
}
/* ********************************************************************************* */
@@ -68,17 +64,23 @@ struct ndpi_analyze_struct* ndpi_alloc_data_analysis(u_int16_t _max_series_len)
/* ********************************************************************************* */
void ndpi_free_data_analysis(struct ndpi_analyze_struct *d) {
void ndpi_free_data_analysis(struct ndpi_analyze_struct *d, u_int8_t free_pointer) {
if(d->values) ndpi_free(d->values);
ndpi_free(d);
if(free_pointer) ndpi_free(d);
}
/* ********************************************************************************* */
void ndpi_reset_data_analysis(struct ndpi_analyze_struct *d) {
u_int32_t *values_bkp = d->values;
u_int32_t num_values_array_len_bpk = d->num_values_array_len;
memset(d, 0, sizeof(struct ndpi_analyze_struct));
d->values = values_bkp;
d->num_values_array_len = num_values_array_len_bpk;
memset(d->values, 0, sizeof(u_int32_t)*d->num_values_array_len);
d->num_data_entries = 0;
}
/* ********************************************************************************* */
@@ -105,12 +107,12 @@ void ndpi_data_add_value(struct ndpi_analyze_struct *s, const u_int32_t value) {
/*
Optimized stddev calculation
https://www.khanacademy.org/math/probability/data-distributions-a1/summarizing-spread-distributions/a/calculating-standard-deviation-step-by-step
https://math.stackexchange.com/questions/683297/how-to-calculate-standard-deviation-without-detailed-historical-data
http://mathcentral.uregina.ca/QQ/database/QQ.09.02/carlos1.html
*/
s->stddev.sum_square_total += value * value;
s->stddev.sum_square_total += (u_int64_t)value * (u_int64_t)value;
}
/* ********************************************************************************* */
@@ -140,7 +142,8 @@ u_int32_t ndpi_data_max(struct ndpi_analyze_struct *s) { return(s->max_val); }
/* Compute the variance on all values */
float ndpi_data_variance(struct ndpi_analyze_struct *s) {
return(s->num_data_entries ? (float)(s->stddev.sum_square_total - ((s->sum_total * s->sum_total) / (s->num_data_entries))) / (float)s->num_data_entries : 0);
float v = s->num_data_entries ? ((float)s->stddev.sum_square_total - ((float)s->sum_total * (float)s->sum_total / (float)s->num_data_entries)) / (float)s->num_data_entries : 0.0;
return((v < 0 /* rounding problem */) ? 0 : v);
}
/* ********************************************************************************* */
@@ -296,7 +299,7 @@ double ndpi_hll_count(struct ndpi_hll *hll) {
/* ********************************************************************************* */
/* ********************************************************************************* */
int ndpi_init_bin(struct ndpi_bin *b, enum ndpi_bin_family f, u_int8_t num_bins) {
int ndpi_init_bin(struct ndpi_bin *b, enum ndpi_bin_family f, u_int16_t num_bins) {
b->num_bins = num_bins, b->family = f, b->is_empty = 1;
switch(f) {
@@ -324,13 +327,13 @@ int ndpi_init_bin(struct ndpi_bin *b, enum ndpi_bin_family f, u_int8_t num_bins)
void ndpi_free_bin(struct ndpi_bin *b) {
switch(b->family) {
case ndpi_bin_family8:
free(b->u.bins8);
ndpi_free(b->u.bins8);
break;
case ndpi_bin_family16:
free(b->u.bins16);
ndpi_free(b->u.bins16);
break;
case ndpi_bin_family32:
free(b->u.bins32);
ndpi_free(b->u.bins32);
break;
}
}
@@ -347,7 +350,7 @@ struct ndpi_bin* ndpi_clone_bin(struct ndpi_bin *b) {
switch(out->family) {
case ndpi_bin_family8:
if((out->u.bins8 = (u_int8_t*)ndpi_calloc(out->num_bins, sizeof(u_int8_t))) == NULL) {
free(out);
ndpi_free(out);
return(NULL);
} else
memcpy(out->u.bins8, b->u.bins8, out->num_bins*sizeof(u_int8_t));
@@ -355,7 +358,7 @@ struct ndpi_bin* ndpi_clone_bin(struct ndpi_bin *b) {
case ndpi_bin_family16:
if((out->u.bins16 = (u_int16_t*)ndpi_calloc(out->num_bins, sizeof(u_int16_t))) == NULL) {
free(out);
ndpi_free(out);
return(NULL);
} else
memcpy(out->u.bins16, b->u.bins16, out->num_bins*sizeof(u_int16_t));
@@ -363,7 +366,7 @@ struct ndpi_bin* ndpi_clone_bin(struct ndpi_bin *b) {
case ndpi_bin_family32:
if((out->u.bins32 = (u_int32_t*)ndpi_calloc(out->num_bins, sizeof(u_int32_t))) == NULL) {
free(out);
ndpi_free(out);
return(NULL);
} else
memcpy(out->u.bins32, b->u.bins32, out->num_bins*sizeof(u_int32_t));
@@ -375,7 +378,7 @@ struct ndpi_bin* ndpi_clone_bin(struct ndpi_bin *b) {
/* ********************************************************************************* */
void ndpi_set_bin(struct ndpi_bin *b, u_int8_t slot_id, u_int32_t val) {
void ndpi_set_bin(struct ndpi_bin *b, u_int16_t slot_id, u_int32_t val) {
if(slot_id >= b->num_bins) slot_id = 0;
switch(b->family) {
@@ -393,7 +396,7 @@ void ndpi_set_bin(struct ndpi_bin *b, u_int8_t slot_id, u_int32_t val) {
/* ********************************************************************************* */
void ndpi_inc_bin(struct ndpi_bin *b, u_int8_t slot_id, u_int32_t val) {
void ndpi_inc_bin(struct ndpi_bin *b, u_int16_t slot_id, u_int32_t val) {
b->is_empty = 0;
if(slot_id >= b->num_bins) slot_id = 0;
@@ -413,7 +416,7 @@ void ndpi_inc_bin(struct ndpi_bin *b, u_int8_t slot_id, u_int32_t val) {
/* ********************************************************************************* */
u_int32_t ndpi_get_bin_value(struct ndpi_bin *b, u_int8_t slot_id) {
u_int32_t ndpi_get_bin_value(struct ndpi_bin *b, u_int16_t slot_id) {
if(slot_id >= b->num_bins) slot_id = 0;
switch(b->family) {
@@ -454,7 +457,7 @@ void ndpi_reset_bin(struct ndpi_bin *b) {
Each bin slot is transformed in a % with respect to the value total
*/
void ndpi_normalize_bin(struct ndpi_bin *b) {
u_int8_t i;
u_int16_t i;
u_int32_t tot = 0;
if(b->is_empty) return;
@@ -468,6 +471,7 @@ void ndpi_normalize_bin(struct ndpi_bin *b) {
b->u.bins8[i] = (b->u.bins8[i]*100) / tot;
}
break;
case ndpi_bin_family16:
for(i=0; i<b->num_bins; i++) tot += b->u.bins16[i];
@@ -476,6 +480,7 @@ void ndpi_normalize_bin(struct ndpi_bin *b) {
b->u.bins16[i] = (b->u.bins16[i]*100) / tot;
}
break;
case ndpi_bin_family32:
for(i=0; i<b->num_bins; i++) tot += b->u.bins32[i];
@@ -490,7 +495,7 @@ void ndpi_normalize_bin(struct ndpi_bin *b) {
/* ********************************************************************************* */
char* ndpi_print_bin(struct ndpi_bin *b, u_int8_t normalize_first, char *out_buf, u_int out_buf_len) {
u_int8_t i;
u_int16_t i;
u_int len = 0;
if(!out_buf) return(out_buf); else out_buf[0] = '\0';
@@ -550,10 +555,14 @@ char* ndpi_print_bin(struct ndpi_bin *b, u_int8_t normalize_first, char *out_buf
0 = alike
...
the higher the more different
*/
float ndpi_bin_similarity(struct ndpi_bin *b1, struct ndpi_bin *b2, u_int8_t normalize_first) {
u_int8_t i;
if similarity_max_threshold != 0, we assume that bins arent similar
*/
float ndpi_bin_similarity(struct ndpi_bin *b1, struct ndpi_bin *b2,
u_int8_t normalize_first, float similarity_max_threshold) {
u_int16_t i;
float threshold = similarity_max_threshold*similarity_max_threshold;
if(
// (b1->family != b2->family) ||
(b1->num_bins != b2->num_bins))
@@ -586,10 +595,13 @@ float ndpi_bin_similarity(struct ndpi_bin *b1, struct ndpi_bin *b2, u_int8_t nor
u_int32_t a = ndpi_get_bin_value(b1, i);
u_int32_t b = ndpi_get_bin_value(b2, i);
u_int32_t diff = (a > b) ? (a - b) : (b - a);
if(a != b) sum += pow(diff, 2);
// printf("[a: %u][b: %u][sum: %u]\n", a, b, sum);
if(threshold && (sum > threshold))
return(-2); /* Sorry they are not similar */
// printf("%u/%u) [a: %u][b: %u][sum: %u]\n", i, b1->num_bins, a, b, sum);
}
/* The lower the more similar */
@@ -707,7 +719,7 @@ int ndpi_cluster_bins(struct ndpi_bin *bins, u_int16_t num_bins,
#ifdef COSINE_SIMILARITY
best_similarity = -1;
#else
best_similarity = 99999999999;
best_similarity = 99999999999.0f;
#endif
for(j=0; j<num_clusters; j++) {
@@ -715,11 +727,11 @@ int ndpi_cluster_bins(struct ndpi_bin *bins, u_int16_t num_bins,
if(centroids[j].is_empty) continue;
similarity = ndpi_bin_similarity(&bins[i], &centroids[j], 0);
similarity = ndpi_bin_similarity(&bins[i], &centroids[j], 0, 0);
if(j == cluster_ids[i])
current_similarity = similarity;
if(verbose)
printf("Bin %u / centroid %u [similarity: %f]\n", i, j, similarity);
@@ -741,7 +753,7 @@ int ndpi_cluster_bins(struct ndpi_bin *bins, u_int16_t num_bins,
*/
cluster_id = cluster_ids[i];
}
bin_score[i] = best_similarity;
if(cluster_ids[i] != cluster_id) {
@@ -815,3 +827,601 @@ int ndpi_cluster_bins(struct ndpi_bin *bins, u_int16_t num_bins,
}
/* ********************************************************************************* */
/*
RSI (Relative Strength Index)
RSI = 100 − [ 100/ (1 + (Average gain/Average loss)) ]
https://www.investopedia.com/terms/r/rsi.asp
*/
int ndpi_alloc_rsi(struct ndpi_rsi_struct *s, u_int16_t num_learning_values) {
memset(s, 0, sizeof(struct ndpi_rsi_struct));
s->empty = 1, s->num_values = num_learning_values;
s->gains = (u_int32_t*)ndpi_calloc(num_learning_values, sizeof(u_int32_t));
s->losses = (u_int32_t*)ndpi_calloc(num_learning_values, sizeof(u_int32_t));
if(s->gains && s->losses) {
s->last_value = 0;
return(0);
} else {
if(s->gains) ndpi_free(s->gains);
if(s->losses) ndpi_free(s->losses);
return(-1);
}
}
/* ************************************* */
void ndpi_free_rsi(struct ndpi_rsi_struct *s) {
ndpi_free(s->gains), ndpi_free(s->losses);
}
/* ************************************* */
// #define DEBUG_RSI
/*
This function adds a new value and returns the computed RSI, or -1
if there are too few points (< num_learning_values)
RSI < 30 (too many losses)
RSI > 70 (too many gains)
*/
float ndpi_rsi_add_value(struct ndpi_rsi_struct *s, const u_int32_t value) {
float relative_strength;
if(!s->empty) {
u_int32_t val;
s->total_gains -= s->gains[s->next_index], s->total_losses -= s->losses[s->next_index];
if(value > s->last_value) {
val = value - s->last_value;
s->gains[s->next_index] = val, s->losses[s->next_index] = 0;
s->total_gains += val;
#ifdef DEBUG_RSI
printf("Gain: %u\n", val);
#endif
} else {
val = s->last_value - value;
s->losses[s->next_index] = val, s->gains[s->next_index] = 0;
s->total_losses += val;
#ifdef DEBUG_RSI
printf("Loss: %u\n", val);
#endif
}
#ifdef DEBUG_RSI
printf("[value: %u][total_gains: %u][total_losses: %u][cur_idx: %u]\n", value, s->total_gains, s->total_losses, s->next_index);
#endif
}
s->last_value = value, s->next_index = (s->next_index + 1) % s->num_values, s->empty = 0;
if(s->next_index == 0) s->rsi_ready = 1; /* We have completed one round */
if(!s->rsi_ready)
return(-1); /* Too early */
else if(s->total_losses == 0) /* Avoid division by zero (**) */
return(100.);
else {
relative_strength = (float)s->total_gains / (float)s->total_losses; /* (**) */
#ifdef DEBUG_RSI
printf("RSI: %f\n", relative_strength);
#endif
return(100. - (100. / (1. + relative_strength)));
}
}
/* *********************************************************** */
/* https://www.johndcook.com/blog/cpp_phi_inverse/ */
static double ndpi_rational_approximation(double t) {
// Abramowitz and Stegun formula 26.2.23.
// The absolute value of the error should be less than 4.5 e-4.
double c[] = { 2.515517, 0.802853, 0.010328 };
double d[] = { 1.432788, 0.189269, 0.001308 };
return(t - ((c[2]*t + c[1])*t + c[0]) / (((d[2]*t + d[1])*t + d[0])*t + 1.0));
}
static double ndpi_normal_cdf_inverse(double p) {
if(p <= 0.0 || p >= 1.0)
return(0); /* Invalid argument: valid range 0 < X < 1 */
if(p < 0.5) {
// F^-1(p) = - G^-1(p)
return -ndpi_rational_approximation( sqrt(-2.0*log(p)) );
} else {
// F^-1(p) = G^-1(1-p)
return ndpi_rational_approximation( sqrt(-2.0*log(1-p)) );
}
}
double ndpi_avg_inline(u_int64_t *v, u_int num) {
double avg = 0;
u_int i;
for(i=0; i<num; i++)
avg += v[i];
return(avg / (u_int32_t)num);
}
/* *********************************************************** */
/* *********************************************************** */
/*
Initializes Holt-Winters with Confidence Interval
Input
hw: Datastructure to initialize and that needs tobe freed with ndpi_hw_free()
num_periods Number of observations of a season, or in ML-parlance the number of points that are required to make the forecast
additive If set to 1 will use the Holt-Winters additive seasonal (should be the default), otherwise the multiplicative seasonal.
alpha Level: specifies the coefficient for the level smoothing. Range 0..1. The higher α, the faster the method forgets old values
beta Trend: specifies the coefficient for the trend smoothing. Range 0..1.
gamma Seasonal: specifies the coefficient for the seasonal smoothing. Range 0..1. With gamma = 0, seasonal correction is not used.
significance Significance level for the forecats sed for computing lower and upper bands. Range 0..1. Typical values 0.05 or less.
See https://en.wikipedia.org/wiki/Statistical_significance
NOTE (See https://otexts.com/fpp2/holt-winters.html)
The additive method is preferred when the seasonal variations are roughly constant through the series,
while the multiplicative method is preferred when the seasonal variations are changing proportional to the level of the series.
For learning more about timeseries forecasting see
https://www.real-statistics.com/time-series-analysis/basic-time-series-forecasting/
*/
int ndpi_hw_init(struct ndpi_hw_struct *hw,
u_int16_t num_periods, u_int8_t additive_seeasonal,
double alpha, double beta, double gamma, float significance) {
memset(hw, 0, sizeof(struct ndpi_hw_struct));
hw->params.num_season_periods = num_periods + 1;
hw->params.alpha = alpha;
hw->params.beta = beta;
hw->params.gamma = gamma;
hw->params.use_hw_additive_seasonal = additive_seeasonal;
if((significance < 0) || (significance > 1)) significance = 0.05;
hw->params.ro = ndpi_normal_cdf_inverse(1 - (significance / 2.));
if((hw->y = (u_int64_t*)ndpi_calloc(hw->params.num_season_periods, sizeof(u_int64_t))) == NULL)
return(-1);
if((hw->s = (double*)ndpi_calloc(hw->params.num_season_periods, sizeof(double))) == NULL) {
ndpi_free(hw->y);
return(-1);
}
return(0);
}
/* *********************************************************** */
/* Frees the memory allocated by ndpi_hw_init() */
void ndpi_hw_free(struct ndpi_hw_struct *hw) {
if(hw->y) ndpi_free(hw->y);
if(hw->s) ndpi_free(hw->s);
}
/* *********************************************************** */
/*
Returns the forecast and the band (forecast +/- band are the upper and lower values)
Input
hw: Datastructure previously initialized
value The value to add to the measurement
Output
forecast The forecasted value
confidence_band The value +/- on which the value should fall is not an anomaly
Return code
0 Too early: we're still in the learning phase. Output values are zero.
1 Normal processing: forecast and confidence_band are meaningful
*/
int ndpi_hw_add_value(struct ndpi_hw_struct *hw, const u_int64_t _value, double *forecast, double *confidence_band) {
if(hw->num_values < hw->params.num_season_periods) {
hw->y[hw->num_values++] = _value;
*forecast = 0;
*confidence_band = 0;
return(0); /* Too early still training... */
} else {
u_int idx = hw->num_values % hw->params.num_season_periods;
double prev_u, prev_v, prev_s, value = (double)_value;
double sq, error, sq_error;
u_int observations;
if(hw->num_values == hw->params.num_season_periods) {
double avg = ndpi_avg_inline(hw->y, hw->params.num_season_periods);
u_int i;
if(avg == 0) avg = 1; /* Avoid divisions by zero */
for(i=0; i<hw->params.num_season_periods; i++)
hw->s[i] = hw->y[i] / avg;
i = hw->params.num_season_periods-1;
if(hw->s[i] == 0)
hw->u = 0;
else
hw->u = _value / hw->s[i];
hw->v = 0;
ndpi_free(hw->y);
hw->y = NULL;
}
idx = hw->num_values % hw->params.num_season_periods;
prev_u = hw->u, prev_v = hw->v, prev_s = hw->s[idx];
if(prev_s != 0)
hw->u = ((hw->params.alpha * value) / prev_s) + ( 1 - hw->params.alpha) * (hw->u + hw->v);
else
hw->u = 0; /* Avoid divisions by zero */
hw->v = (hw->params.beta * (hw->u - prev_u)) + ((1 - hw->params.beta ) * hw->v);
if(hw->u != 0)
hw->s[idx] = (hw->params.gamma * (value / hw->u)) + ((1 - hw->params.gamma) * prev_s);
else
hw->s[idx] = 0; /* Avoid divisions by zero */
if(hw->params.use_hw_additive_seasonal)
*forecast = (prev_u + prev_v) + prev_s;
else
*forecast = (prev_u + prev_v) * prev_s;
error = value - *forecast;
sq_error = error * error;
hw->sum_square_error += sq_error, hw->prev_error.sum_square_error += sq_error;;
observations = (hw->num_values < MAX_SQUARE_ERROR_ITERATIONS) ? hw->num_values : ((hw->num_values % MAX_SQUARE_ERROR_ITERATIONS) + MAX_SQUARE_ERROR_ITERATIONS);
sq = sqrt(hw->sum_square_error / (observations - hw->params.num_season_periods));
*confidence_band = hw->params.ro * sq;
#ifdef HW_DEBUG
printf("[num_values: %u][u: %.3f][v: %.3f][s: %.3f][error: %.3f][forecast: %.3f][sqe: %.3f][sq: %.3f][confidence_band: %.3f]\n",
hw->num_values, hw->u, hw->v, hw->s[idx], error,
*forecast, hw->sum_square_error,
sq, *confidence_band);
#endif
hw->num_values++, idx = (idx + 1) % hw->params.num_season_periods;
if(++hw->prev_error.num_values_rollup == MAX_SQUARE_ERROR_ITERATIONS) {
hw->sum_square_error = hw->prev_error.sum_square_error;
hw->prev_error.num_values_rollup = 0, hw->prev_error.sum_square_error = 0;
}
return(1); /* We're in business: forecast is meaningful now */
}
}
/* ********************************************************************************* */
/* ********************************************************************************* */
/*
Jitter calculator
Used to determine how noisy is a signal
*/
int ndpi_jitter_init(struct ndpi_jitter_struct *s, u_int16_t num_learning_values) {
memset(s, 0, sizeof(struct ndpi_jitter_struct));
if(num_learning_values < 2) num_learning_values = 2;
s->empty = 1, s->num_values = num_learning_values;
s->observations = (float*)ndpi_calloc(num_learning_values, sizeof(float));
if(s->observations) {
s->last_value = 0;
return(0);
} else
return(-1);
}
/* ************************************* */
void ndpi_jitter_free(struct ndpi_jitter_struct *s) {
ndpi_free(s->observations);
}
/* ************************************* */
/*
This function adds a new value and returns the computed Jitter
*/
float ndpi_jitter_add_value(struct ndpi_jitter_struct *s, const float value) {
float val = fabsf(value - s->last_value);
if(s->empty && (s->next_index == 0))
; /* Skip the first value as we are unable to calculate the difference */
else {
s->jitter_total -= s->observations[s->next_index];
s->observations[s->next_index] = val;
s->jitter_total += val;
}
s->last_value = value, s->next_index = (s->next_index + 1) % s->num_values;
if(s->next_index == 0) s->jitter_ready = 1; /* We have completed one round */
#ifdef DEBUG_JITTER
printf("[JITTER] [value: %.3f][diff: %.3f][jitter_total: %.3f] -> %.3f\n",
value, val, s->jitter_total,
s->jitter_ready ? (s->jitter_total / s->num_values) : -1);
#endif
if(!s->jitter_ready)
return(-1); /* Too early */
else
return(s->jitter_total / s->num_values);
}
/* *********************************************************** */
/* *********************************************************** */
/*
Single Exponential Smoothing
*/
int ndpi_ses_init(struct ndpi_ses_struct *ses, double alpha, float significance) {
memset(ses, 0, sizeof(struct ndpi_ses_struct));
ses->params.alpha = alpha;
if((significance < 0) || (significance > 1)) significance = 0.05;
ses->params.ro = ndpi_normal_cdf_inverse(1 - (significance / 2.));
return(0);
}
/* *********************************************************** */
/*
Returns the forecast and the band (forecast +/- band are the upper and lower values)
Input
ses: Datastructure previously initialized
value The value to add to the measurement
Output
forecast The forecasted value
confidence_band The value +/- on which the value should fall is not an anomaly
Return code
0 Too early: we're still in the learning phase. Output values are zero.
1 Normal processing: forecast and confidence_band are meaningful
*/
int ndpi_ses_add_value(struct ndpi_ses_struct *ses, const u_int64_t _value, double *forecast, double *confidence_band) {
double value = (double)_value, error, sq_error;
int rc;
if(ses->num_values == 0)
*forecast = value;
else
*forecast = (ses->params.alpha * (ses->last_value - ses->last_forecast)) + ses->last_forecast;
error = value - *forecast;
sq_error = error * error;
ses->sum_square_error += sq_error, ses->prev_error.sum_square_error += sq_error;
if(ses->num_values > 0) {
u_int observations = (ses->num_values < MAX_SQUARE_ERROR_ITERATIONS) ? (ses->num_values + 1) : ((ses->num_values % MAX_SQUARE_ERROR_ITERATIONS) + MAX_SQUARE_ERROR_ITERATIONS + 1);
double sq = sqrt(ses->sum_square_error / observations);
*confidence_band = ses->params.ro * sq;
rc = 1;
} else
*confidence_band = 0, rc = 0;
ses->num_values++, ses->last_value = value, ses->last_forecast = *forecast;
if(++ses->prev_error.num_values_rollup == MAX_SQUARE_ERROR_ITERATIONS) {
ses->sum_square_error = ses->prev_error.sum_square_error;
ses->prev_error.num_values_rollup = 0, ses->prev_error.sum_square_error = 0;
}
#ifdef SES_DEBUG
printf("[num_values: %u][[error: %.3f][forecast: %.3f][sqe: %.3f][sq: %.3f][confidence_band: %.3f]\n",
ses->num_values, error, *forecast, ses->sum_square_error, sq_error, *confidence_band);
#endif
return(rc);
}
/* *********************************************************** */
/*
Computes the best alpha value using the specified values used for training
*/
void ndpi_ses_fitting(double *values, u_int32_t num_values, float *ret_alpha) {
u_int i;
float alpha, best_alpha;
double sse, lowest_sse;
int trace = 0;
lowest_sse = 0, best_alpha = 0;
for(alpha=0.1; alpha<0.99; alpha += 0.05) {
struct ndpi_ses_struct ses;
ndpi_ses_init(&ses, alpha, 0.05);
if(trace)
printf("\nDouble Exponential Smoothing [alpha: %.2f]\n", alpha);
sse = 0;
for(i=0; i<num_values; i++) {
double prediction, confidence_band;
double diff;
if(ndpi_ses_add_value(&ses, values[i], &prediction, &confidence_band) != 0) {
diff = fabs(prediction-values[i]);
if(trace)
printf("%2u)\t%12.3f\t%.3f\t%.3f\n", i, values[i], prediction, diff);
sse += diff*diff;
}
}
if(lowest_sse == 0)
lowest_sse = sse, best_alpha = alpha; /* first run */
else {
if(sse <= lowest_sse)
lowest_sse = sse, best_alpha = alpha;
}
if(trace)
printf("[alpha: %.2f] - SSE: %.2f [BEST: alpha: %.2f/SSE: %.2f]\n", alpha, sse,
best_alpha, lowest_sse);
} /* for (alpha) */
if(trace)
printf("BEST [alpha: %.2f][SSE: %.2f]\n", best_alpha, lowest_sse);
*ret_alpha = best_alpha;
}
/* *********************************************************** */
/* *********************************************************** */
/*
Double Exponential Smoothing
*/
int ndpi_des_init(struct ndpi_des_struct *des, double alpha, double beta, float significance) {
memset(des, 0, sizeof(struct ndpi_des_struct));
des->params.alpha = alpha;
des->params.beta = beta;
if((significance < 0) || (significance > 1)) significance = 0.05;
des->params.ro = ndpi_normal_cdf_inverse(1 - (significance / 2.));
return(0);
}
/* *********************************************************** */
/*
Returns the forecast and the band (forecast +/- band are the upper and lower values)
Input
des: Datastructure previously initialized
value The value to add to the measurement
Output
forecast The forecasted value
confidence_band The value +/- on which the value should fall is not an anomaly
Return code
0 Too early: we're still in the learning phase. Output values are zero.
1 Normal processing: forecast and confidence_band are meaningful
*/
int ndpi_des_add_value(struct ndpi_des_struct *des, const u_int64_t _value, double *forecast, double *confidence_band) {
double value = (double)_value, error, sq_error;
int rc;
if(des->num_values == 0)
*forecast = value, des->last_trend = 0;
else {
*forecast = (des->params.alpha * value) + ((1 - des->params.alpha) * (des->last_forecast + des->last_trend));
des->last_trend = (des->params.beta * (*forecast - des->last_forecast)) + ((1 - des->params.beta) * des->last_trend);
}
error = value - *forecast;
sq_error = error * error;
des->sum_square_error += sq_error, des->prev_error.sum_square_error += sq_error;
if(des->num_values > 0) {
u_int observations = (des->num_values < MAX_SQUARE_ERROR_ITERATIONS) ? (des->num_values + 1) : ((des->num_values % MAX_SQUARE_ERROR_ITERATIONS) + MAX_SQUARE_ERROR_ITERATIONS + 1);
double sq = sqrt(des->sum_square_error / observations);
*confidence_band = des->params.ro * sq;
rc = 1;
} else
*confidence_band = 0, rc = 0;
des->num_values++, des->last_value = value, des->last_forecast = *forecast;
if(++des->prev_error.num_values_rollup == MAX_SQUARE_ERROR_ITERATIONS) {
des->sum_square_error = des->prev_error.sum_square_error;
des->prev_error.num_values_rollup = 0, des->prev_error.sum_square_error = 0;
}
#ifdef DES_DEBUG
printf("[num_values: %u][[error: %.3f][forecast: %.3f][trend: %.3f[sqe: %.3f][sq: %.3f][confidence_band: %.3f]\n",
des->num_values, error, *forecast, des->last_trend, des->sum_square_error, sq, *confidence_band);
#endif
return(rc);
}
/* *********************************************************** */
/*
Computes the best alpha and beta values using the specified values used for training
*/
void ndpi_des_fitting(double *values, u_int32_t num_values, float *ret_alpha, float *ret_beta) {
u_int i;
float alpha, best_alpha, best_beta, beta = 0;
double sse, lowest_sse;
int trace = 0;
lowest_sse = 0, best_alpha = 0, best_beta = 0;
for(beta=0.1; beta<0.99; beta += 0.05) {
for(alpha=0.1; alpha<0.99; alpha += 0.05) {
struct ndpi_des_struct des;
ndpi_des_init(&des, alpha, beta, 0.05);
if(trace)
printf("\nDouble Exponential Smoothing [alpha: %.2f][beta: %.2f]\n", alpha, beta);
sse = 0;
for(i=0; i<num_values; i++) {
double prediction, confidence_band;
double diff;
if(ndpi_des_add_value(&des, values[i], &prediction, &confidence_band) != 0) {
diff = fabs(prediction-values[i]);
if(trace)
printf("%2u)\t%12.3f\t%.3f\t%.3f\n", i, values[i], prediction, diff);
sse += diff*diff;
}
}
if(lowest_sse == 0)
lowest_sse = sse, best_alpha = alpha, best_beta = beta; /* first run */
else {
if(sse <= lowest_sse)
lowest_sse = sse, best_alpha = alpha, best_beta = beta;
}
if(trace)
printf("[alpha: %.2f][beta: %.2f] - SSE: %.2f [BEST: alpha: %.2f/beta: %.2f/SSE: %.2f]\n", alpha, beta, sse,
best_alpha, best_beta, lowest_sse);
} /* for (alpha) */
} /* for (beta) */
if(trace)
printf("BEST [alpha: %.2f][beta: %.2f][SSE: %.2f]\n", best_alpha, best_beta, lowest_sse);
*ret_alpha = best_alpha, *ret_beta = best_beta;
}
File diff suppressed because it is too large Load Diff
+139
View File
@@ -0,0 +1,139 @@
/*
* ndpi_utils.c
*
* Copyright (C) 2011-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#include <stdlib.h>
#include <errno.h>
#include <math.h>
#include <sys/types.h>
#define NDPI_CURRENT_PROTO NDPI_PROTOCOL_UNKNOWN
#include "ndpi_config.h"
#include "ndpi_api.h"
#include "ndpi_includes.h"
#include "ndpi_encryption.h"
#include "third_party/include/roaring.h"
#include "third_party/src/roaring.cc"
/* ******************************************* */
ndpi_bitmap* ndpi_bitmap_alloc() {
return((ndpi_bitmap*)roaring_bitmap_create());
}
/* ******************************************* */
void ndpi_bitmap_free(ndpi_bitmap* b) {
roaring_bitmap_free((const roaring_bitmap_t *)b);
}
/* ******************************************* */
u_int64_t ndpi_bitmap_cardinality(ndpi_bitmap* b) {
return(roaring_bitmap_get_cardinality((const roaring_bitmap_t *)b));
}
/* ******************************************* */
void ndpi_bitmap_set(ndpi_bitmap* b, u_int32_t value) {
roaring_bitmap_add((roaring_bitmap_t *)b, value);
}
/* ******************************************* */
void ndpi_bitmap_unset(ndpi_bitmap* b, u_int32_t value) {
roaring_bitmap_remove((roaring_bitmap_t *)b, value);
}
/* ******************************************* */
bool ndpi_bitmap_isset(ndpi_bitmap* b, u_int32_t value) {
return(roaring_bitmap_contains((const roaring_bitmap_t *)b, value));
}
/* ******************************************* */
void ndpi_bitmap_clear(ndpi_bitmap* b) {
roaring_bitmap_clear((roaring_bitmap_t *)b);
}
/* ******************************************* */
size_t ndpi_bitmap_serialize(ndpi_bitmap* b, char **buf) {
const roaring_bitmap_t *r = (const roaring_bitmap_t *)b;
size_t s = roaring_bitmap_size_in_bytes(r);
*buf = (char*)ndpi_malloc(s);
if((*buf) == NULL) return(0);
return(roaring_bitmap_serialize(r, *buf));
}
/* ******************************************* */
ndpi_bitmap* ndpi_bitmap_deserialize(char *buf) {
return((ndpi_bitmap*)roaring_bitmap_deserialize(buf));
}
/* ******************************************* */
/* b = b & b_and */
void ndpi_bitmap_and(ndpi_bitmap* a, ndpi_bitmap* b_and) {
roaring_bitmap_and_inplace((ndpi_bitmap*)a, (ndpi_bitmap*)b_and);
}
/* ******************************************* */
/* b = b | b_or */
void ndpi_bitmap_or(ndpi_bitmap* a, ndpi_bitmap* b_or) {
roaring_bitmap_or_inplace((ndpi_bitmap*)a, (ndpi_bitmap*)b_or);
}
/* ******************************************* */
ndpi_bitmap_iterator* ndpi_bitmap_iterator_alloc(ndpi_bitmap* b) {
return(roaring_create_iterator((ndpi_bitmap*)b));
}
/* ******************************************* */
void ndpi_bitmap_iterator_free(ndpi_bitmap* b) {
return(roaring_free_uint32_iterator((ndpi_bitmap*)b));
}
/* ******************************************* */
/* Return the next value in the bitmap iterator
true is returned when a value is present, false when we reached the end
*/
bool ndpi_bitmap_iterator_next(ndpi_bitmap_iterator* i, uint32_t *value) {
uint32_t num = roaring_read_uint32_iterator((roaring_uint32_iterator_t*)i, value, 1);
return((num == 1) ? true /* found */ : false /* not found */);
}
+6
View File
@@ -688,7 +688,13 @@ ndpi_log_timestamp(char *log_ts, uint32_t log_ts_len)
gettimeofday(&tv, NULL);
nowtime = tv.tv_sec;
#ifdef WIN32
/* localtime() on Windows is thread-safe */
struct tm * nowtm_r_ptr = localtime(&nowtime);
nowtm_r = *nowtm_r_ptr;
#else
localtime_r(&nowtime, &nowtm_r);
#endif
strftime(tmbuf, NDPI_TIMESTAMP_LEN, "%H:%M:%S", &nowtm_r);
snprintf(log_ts, log_ts_len, "%s.%06ld", tmbuf, (long)tv.tv_usec);
}
+4 -4
View File
@@ -1,7 +1,7 @@
/*
* ndpi_community_id.c
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -70,7 +70,7 @@
/* **************************************************** */
static ssize_t ndpi_community_id_buf_copy(u_int8_t * const dst, const void * const src, ssize_t len) {
static u_int16_t ndpi_community_id_buf_copy(u_int8_t * const dst, const void * const src, u_int16_t len) {
if(src)
memcpy(dst, src, len);
else
@@ -185,11 +185,11 @@ void ndpi_string_sha1_hash(const uint8_t *message, size_t len, u_char *hash /* 2
/* **************************************************** */
/*
https://github.com/corelight/community-id-spec/blob/bda913f617389df07cdaa23606e11bbd318e265c/community-id.py#L285
https://github.com/corelight/community-id-spec/blob/bda913f617389df07cdaa23606e11bbd318e265c/community-id.py#L285
*/
static int ndpi_community_id_finalize_and_compute_hash(u_int8_t *comm_buf, u_int16_t off, u_int8_t l4_proto,
u_int16_t src_port, u_int16_t dst_port,
char *hash_buf, u_int8_t hash_buf_len) {
char *hash_buf, size_t hash_buf_len) {
u_int8_t pad = 0;
uint32_t hash[5];
char *community_id;
File diff suppressed because one or more lines are too long
+163
View File
@@ -0,0 +1,163 @@
/*
* ndpi_geoip.c
*
* Copyright (C) 2021 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library.
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#include <stdlib.h>
#include <errno.h>
#include <sys/types.h>
#include <stdint.h>
#include "ndpi_api.h"
#include "ndpi_config.h"
#ifdef HAVE_MAXMINDDB
#include <maxminddb.h>
#endif
/* ********************************************************************************* */
int ndpi_load_geoip(struct ndpi_detection_module_struct *ndpi_str,
const char *ip_city_data, const char *ip_as_data) {
#ifdef HAVE_MAXMINDDB
int status;
ndpi_str->mmdb_city = (void*)ndpi_malloc(sizeof(MMDB_s));
ndpi_str->mmdb_as = (void*)ndpi_malloc(sizeof(MMDB_s));
if((ndpi_str->mmdb_city == NULL) || (ndpi_str->mmdb_as == NULL))
return(-1);
/* Open the MMDB files */
if((status = MMDB_open(ip_city_data, MMDB_MODE_MMAP, (MMDB_s*)ndpi_str->mmdb_city)) != MMDB_SUCCESS)
return(-1);
else
ndpi_str->mmdb_city_loaded = 1;
if((status = MMDB_open(ip_as_data, MMDB_MODE_MMAP, (MMDB_s*)ndpi_str->mmdb_as)) != MMDB_SUCCESS)
return(-2);
else
ndpi_str->mmdb_as_loaded = 1;
return(0);
#else
return(-3);
#endif
}
/* ********************************************************************************* */
void ndpi_free_geoip(struct ndpi_detection_module_struct *ndpi_str) {
#ifdef HAVE_MAXMINDDB
if(ndpi_str->mmdb_city_loaded) MMDB_close((MMDB_s*)ndpi_str->mmdb_city);
if(ndpi_str->mmdb_as_loaded) MMDB_close((MMDB_s*)ndpi_str->mmdb_as);
ndpi_free(ndpi_str->mmdb_city);
ndpi_free(ndpi_str->mmdb_as);
#endif
}
/* ********************************************************************************* */
int ndpi_get_geoip_asn(struct ndpi_detection_module_struct *ndpi_str, char *ip, u_int32_t *asn) {
#ifdef HAVE_MAXMINDDB
int gai_error, mmdb_error, status;
MMDB_lookup_result_s result;
MMDB_entry_data_s entry_data;
if(ndpi_str->mmdb_as_loaded) {
result = MMDB_lookup_string((MMDB_s*)ndpi_str->mmdb_as, ip, &gai_error, &mmdb_error);
if((gai_error != 0)
|| (mmdb_error != MMDB_SUCCESS)
|| (!result.found_entry))
*asn = 0;
else {
/* Get the ASN */
if((status = MMDB_get_value(&result.entry, &entry_data, "autonomous_system_number", NULL)) == MMDB_SUCCESS) {
if(entry_data.has_data && entry_data.type == MMDB_DATA_TYPE_UINT32)
*asn = entry_data.uint32;
else
*asn = 0;
}
}
return(0);
}
#endif
return(-2);
}
/* ********************************************************************************* */
int ndpi_get_geoip_country_continent(struct ndpi_detection_module_struct *ndpi_str, char *ip,
char *country_code, u_int8_t country_code_len,
char *continent, u_int8_t continent_len) {
#ifdef HAVE_MAXMINDDB
int gai_error, mmdb_error;
MMDB_lookup_result_s result;
MMDB_entry_data_s entry_data;
if(ndpi_str->mmdb_city_loaded) {
int status;
result = MMDB_lookup_string((MMDB_s*)ndpi_str->mmdb_city, ip, &gai_error, &mmdb_error);
if((gai_error != 0)
|| (mmdb_error != MMDB_SUCCESS)
|| (!result.found_entry))
country_code[0] = '\0';
else {
if(country_code_len > 0) {
status = MMDB_get_value(&result.entry, &entry_data, "country", "iso_code", NULL);
if((status != MMDB_SUCCESS) || (!entry_data.has_data))
country_code[0] = '\0';
else {
int str_len = ndpi_min(entry_data.data_size, country_code_len);
memcpy(country_code, entry_data.utf8_string, str_len);
country_code[str_len] = '\0';
}
}
if(continent_len > 0) {
status = MMDB_get_value(&result.entry, &entry_data, "continent", "names", "en", NULL);
if((status != MMDB_SUCCESS) || (!entry_data.has_data))
continent[0] = '\0';
else {
int str_len = ndpi_min(entry_data.data_size, continent_len);
memcpy(continent, entry_data.utf8_string, str_len);
continent[str_len] = '\0';
}
}
}
return(0);
}
#endif
return(-2);
}
+3254 -2102
View File
File diff suppressed because it is too large Load Diff
+236 -114
View File
@@ -1,7 +1,7 @@
/*
* ndpi_serializer.c
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-22 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
@@ -47,14 +47,17 @@
#define NDPI_U64_FORMAT "%llu"
#endif
#define OPTIMIZE_NUMERIC_KEYS /* Convert numeric string keys into int keys */
#define OPTIMIZE_CSV_SERIALIZATION /* Optimize serialization speed in case of CSV */
/* ********************************** */
u_int64_t ndpi_htonll(u_int64_t v) {
union { u_int32_t lv[2]; u_int64_t llv; } u;
u.lv[0] = htonl(v >> 32);
u.lv[1] = htonl(v & 0xFFFFFFFFULL);
return(u.llv);
}
@@ -62,22 +65,24 @@ u_int64_t ndpi_htonll(u_int64_t v) {
u_int64_t ndpi_ntohll(u_int64_t v) {
union { u_int32_t lv[2]; u_int64_t llv; } u;
u.llv = v;
return((u_int64_t)ntohl(u.lv[0]) << 32) | (u_int64_t)ntohl(u.lv[1]);
}
/* ********************************** */
#ifdef OPTIMIZE_NUMERIC_KEYS
static int ndpi_is_number(const char *str, u_int32_t str_len) {
int i;
unsigned int i;
for(i = 0; i < str_len; i++)
if(!isdigit(str[i])) return(0);
return(1);
}
#endif
/* ********************************** */
@@ -159,7 +164,7 @@ static inline int ndpi_snappend(char *buf, size_t size, const char *fmt, ...) {
else if (wlen >= size)
wlen = size-1;
return wlen;
return(wlen);
}
#endif
@@ -188,12 +193,21 @@ void ndpi_reset_serializer(ndpi_serializer *_serializer) {
/* ********************************** */
void ndpi_serializer_skip_header(ndpi_serializer *_serializer) {
ndpi_private_serializer *serializer = (ndpi_private_serializer*)_serializer;
serializer->status.flags |= NDPI_SERIALIZER_STATUS_HDR_DONE;
}
/* ********************************** */
static int ndpi_init_serializer_buffer(ndpi_private_serializer_buffer *buffer, u_int32_t buffer_size) {
buffer->initial_size = buffer->size = buffer_size;
buffer->data = (u_int8_t *) calloc(buffer->size, sizeof(u_int8_t));
if(buffer->data == NULL)
return -1;
return 0;
return(-1);
return(0);
}
/* ********************************** */
@@ -209,7 +223,7 @@ int ndpi_init_serializer_ll(ndpi_serializer *_serializer,
if (ndpi_init_serializer_buffer(&serializer->buffer, buffer_size) != 0)
return(-1);
if(serializer->fmt == ndpi_serialization_format_json) {
/* nothing to do */
@@ -227,7 +241,7 @@ int ndpi_init_serializer_ll(ndpi_serializer *_serializer,
ndpi_reset_serializer(_serializer);
return(1);
return(0);
}
/* ********************************** */
@@ -272,12 +286,12 @@ static inline int ndpi_serializer_check_header_room(ndpi_private_serializer *ser
u_int32_t buff_diff = serializer->header.size - serializer->status.header.size_used;
if (buff_diff < needed)
if (ndpi_extend_serializer_buffer(&serializer->header, needed - buff_diff) < 0)
return -1;
if (ndpi_extend_serializer_buffer(&serializer->header, needed - buff_diff) < 0)
return(-1);
buff_diff = serializer->header.size - serializer->status.header.size_used;
return buff_diff;
return(buff_diff);
}
/* ********************************** */
@@ -286,17 +300,17 @@ static inline int ndpi_serializer_header_uint32(ndpi_private_serializer *seriali
int room;
if (serializer->status.flags & NDPI_SERIALIZER_STATUS_HDR_DONE)
return 0;
return(0);
room = ndpi_serializer_check_header_room(serializer, 12);
if (room < 0)
return -1;
return(-1);
serializer->status.header.size_used += snprintf((char *) &serializer->header.data[serializer->status.header.size_used],
serializer->status.header.size_used += snprintf((char *) &serializer->header.data[serializer->status.header.size_used],
room, "%s%u", (serializer->status.header.size_used > 0) ? serializer->csv_separator : "", key);
return 0;
return(0);
}
/* ********************************** */
@@ -305,12 +319,12 @@ static inline int ndpi_serializer_header_string(ndpi_private_serializer *seriali
int room;
if (serializer->status.flags & NDPI_SERIALIZER_STATUS_HDR_DONE)
return 0;
return(0);
room = ndpi_serializer_check_header_room(serializer, klen + 4);
if (room < 0)
return -1;
return(-1);
if (serializer->status.header.size_used > 0) {
int slen = strlen(serializer->csv_separator);
@@ -325,7 +339,7 @@ static inline int ndpi_serializer_header_string(ndpi_private_serializer *seriali
serializer->header.data[serializer->status.header.size_used] = '\0';
return 0;
return(0);
}
/* ********************************** */
@@ -385,7 +399,7 @@ char* ndpi_serializer_get_header(ndpi_serializer *_serializer, u_int32_t *buffer
if(buf == NULL) {
*buffer_len = 0;
return "";
return("");
}
/* NULL terminate the buffer */
@@ -401,7 +415,8 @@ char* ndpi_serializer_get_header(ndpi_serializer *_serializer, u_int32_t *buffer
ndpi_serialization_format ndpi_serializer_get_format(ndpi_serializer *_serializer) {
ndpi_private_serializer *serializer = (ndpi_private_serializer*)_serializer;
return serializer->fmt;
return(serializer->fmt);
}
/* ********************************** */
@@ -579,7 +594,7 @@ int ndpi_serialize_raw_record(ndpi_serializer *_serializer,
u_int32_t buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
u_int16_t needed = record_len;
u_int8_t add_comma = 0;
if(serializer->fmt == ndpi_serialization_format_json) {
needed += 1;
@@ -588,21 +603,21 @@ int ndpi_serialize_raw_record(ndpi_serializer *_serializer,
else
needed += 2, add_comma = 1;
}
if(buff_diff < needed) {
if(ndpi_extend_serializer_buffer(&serializer->buffer, needed - buff_diff) < 0)
return(-1);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
if(serializer->fmt == ndpi_serialization_format_json) {
if(serializer->fmt == ndpi_serialization_format_json) {
if(add_comma)
serializer->buffer.data[serializer->status.buffer.size_used-1] = ',';
else
serializer->status.buffer.size_used--;
}
memcpy(&serializer->buffer.data[serializer->status.buffer.size_used], record, record_len);
serializer->status.buffer.size_used += record_len;
@@ -610,9 +625,9 @@ int ndpi_serialize_raw_record(ndpi_serializer *_serializer,
serializer->buffer.data[serializer->status.buffer.size_used] = ']';
if(add_comma) serializer->status.buffer.size_used++;
}
ndpi_serialize_end_of_record(_serializer);
return(0);
}
@@ -653,7 +668,7 @@ int ndpi_serialize_end_of_record(ndpi_serializer *_serializer) {
}
serializer->status.flags &= ~NDPI_SERIALIZER_STATUS_NOT_EMPTY;
return(0);
}
@@ -665,10 +680,10 @@ static inline void ndpi_serialize_csv_pre(ndpi_private_serializer *serializer) {
} else if (serializer->status.buffer.size_used == 0) {
/* nothing to do */
} else {
u_int32_t buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used], buff_diff, "%s",
serializer->csv_separator);
if(serializer->buffer.size > serializer->status.buffer.size_used)
serializer->buffer.data[serializer->status.buffer.size_used] = serializer->csv_separator[0];
serializer->status.buffer.size_used++;
}
}
@@ -692,7 +707,7 @@ static inline void ndpi_serialize_json_pre(ndpi_serializer *_serializer) {
serializer->status.buffer.size_used--; /* Remove ']' */
if(serializer->status.flags & NDPI_SERIALIZER_STATUS_SOL)
serializer->status.flags &= ~NDPI_SERIALIZER_STATUS_SOL;
else
else
serializer->buffer.data[serializer->status.buffer.size_used++] = ',';
} else {
if(serializer->status.flags & NDPI_SERIALIZER_STATUS_SOB)
@@ -763,13 +778,13 @@ int ndpi_serialize_uint32_uint32(ndpi_serializer *_serializer,
if (!(serializer->status.flags & NDPI_SERIALIZER_STATUS_LIST)) {
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used],
&serializer->buffer.data[serializer->status.buffer.size_used],
buff_diff, "\"%u\":", key);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used],
&serializer->buffer.data[serializer->status.buffer.size_used],
buff_diff, "%u", value);
ndpi_serialize_json_post(_serializer);
@@ -830,13 +845,13 @@ int ndpi_serialize_uint32_uint64(ndpi_serializer *_serializer,
ndpi_serialize_json_pre(_serializer);
if (!(serializer->status.flags & NDPI_SERIALIZER_STATUS_LIST)) {
serializer->status.buffer.size_used += snprintf((char *)
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used], buff_diff,
"\"%u\":", key);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
serializer->status.buffer.size_used += snprintf((char *)
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used], buff_diff,
NDPI_U64_FORMAT, (unsigned long long)value);
@@ -892,16 +907,16 @@ int ndpi_serialize_uint32_int32(ndpi_serializer *_serializer,
if(serializer->fmt == ndpi_serialization_format_json) {
ndpi_serialize_json_pre(_serializer);
if (!(serializer->status.flags & NDPI_SERIALIZER_STATUS_LIST)) {
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used],
&serializer->buffer.data[serializer->status.buffer.size_used],
buff_diff, "\"%u\":", key);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used],
&serializer->buffer.data[serializer->status.buffer.size_used],
buff_diff, "%d", value);
ndpi_serialize_json_post(_serializer);
@@ -909,7 +924,7 @@ int ndpi_serialize_uint32_int32(ndpi_serializer *_serializer,
if (ndpi_serializer_header_uint32(serializer, key) < 0) return(-1);
ndpi_serialize_csv_pre(serializer);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
serializer->status.buffer.size_used += snprintf((char *)
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used], buff_diff,
"%d", value);
} else {
@@ -962,14 +977,14 @@ int ndpi_serialize_uint32_int64(ndpi_serializer *_serializer,
ndpi_serialize_json_pre(_serializer);
if (!(serializer->status.flags & NDPI_SERIALIZER_STATUS_LIST)) {
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used],
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used],
buff_diff, "\"%u\":", key);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used],
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used],
buff_diff, NDPI_I64_FORMAT, (long long int)value);
ndpi_serialize_json_post(_serializer);
@@ -1084,7 +1099,7 @@ static int ndpi_serialize_uint32_binary(ndpi_serializer *_serializer,
ndpi_serialize_json_pre(_serializer);
if (!(serializer->status.flags & NDPI_SERIALIZER_STATUS_LIST)) {
serializer->status.buffer.size_used += snprintf((char *) &serializer->buffer.data[serializer->status.buffer.size_used],
serializer->status.buffer.size_used += snprintf((char *) &serializer->buffer.data[serializer->status.buffer.size_used],
buff_diff, "\"%u\":", key);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
@@ -1096,9 +1111,9 @@ static int ndpi_serialize_uint32_binary(ndpi_serializer *_serializer,
if (ndpi_serializer_header_uint32(serializer, key) < 0) return(-1);
ndpi_serialize_csv_pre(serializer);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used], buff_diff,
"%s", value);
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used], buff_diff,
"%s", value);
} else {
ndpi_serialization_type kt;
u_int8_t type = 0;
@@ -1135,7 +1150,7 @@ int ndpi_serialize_uint32_boolean(ndpi_serializer *_serializer,
if(serializer->fmt != ndpi_serialization_format_json &&
serializer->fmt != ndpi_serialization_format_csv)
return -1;
return(-1);
if(buff_diff < needed) {
if(ndpi_extend_serializer_buffer(&serializer->buffer, needed - buff_diff) < 0)
@@ -1147,14 +1162,14 @@ int ndpi_serialize_uint32_boolean(ndpi_serializer *_serializer,
ndpi_serialize_json_pre(_serializer);
if (!(serializer->status.flags & NDPI_SERIALIZER_STATUS_LIST)) {
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used],
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used],
buff_diff, "\"%u\":", key);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used],
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used],
buff_diff, "%s", value ? "true" : "false");
ndpi_serialize_json_post(_serializer);
@@ -1180,8 +1195,10 @@ int ndpi_serialize_binary_int32(ndpi_serializer *_serializer,
u_int32_t buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
u_int32_t needed;
#ifdef OPTIMIZE_NUMERIC_KEYS
if(ndpi_is_number(key, klen))
return(ndpi_serialize_uint32_int32(_serializer, atoi(key), value));
#endif
needed =
sizeof(u_int8_t) /* type */ +
@@ -1204,13 +1221,12 @@ int ndpi_serialize_binary_int32(ndpi_serializer *_serializer,
if (!(serializer->status.flags & NDPI_SERIALIZER_STATUS_LIST)) {
serializer->status.buffer.size_used += ndpi_json_string_escape(key, klen,
(char *) &serializer->buffer.data[serializer->status.buffer.size_used], buff_diff);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used], buff_diff, ":");
serializer->buffer.data[serializer->status.buffer.size_used] = ':';
serializer->status.buffer.size_used++;
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
serializer->status.buffer.size_used += snprintf((char *)
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used], buff_diff, "%d", value);
ndpi_serialize_json_post(_serializer);
@@ -1238,7 +1254,7 @@ int ndpi_serialize_binary_int32(ndpi_serializer *_serializer,
}
serializer->status.flags |= NDPI_SERIALIZER_STATUS_NOT_EMPTY;
return(0);
}
@@ -1246,6 +1262,38 @@ int ndpi_serialize_binary_int32(ndpi_serializer *_serializer,
int ndpi_serialize_string_int32(ndpi_serializer *_serializer,
const char *key, int32_t value) {
#ifdef OPTIMIZE_CSV_SERIALIZATION
ndpi_private_serializer *serializer = (ndpi_private_serializer*)_serializer;
if(serializer->fmt == ndpi_serialization_format_csv) {
/* Key is ignored */
u_int32_t buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
u_int16_t needed = 11 /* 10 (billion) + CVS separator */;
int rc;
if(buff_diff < needed) {
if(ndpi_extend_serializer_buffer(&serializer->buffer, needed - buff_diff) < 0)
return(-1);
else
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
if(!(serializer->status.flags & NDPI_SERIALIZER_STATUS_HDR_DONE)) {
if(ndpi_serializer_header_string(serializer, key, strlen(key)) < 0)
return(-1);
}
ndpi_serialize_csv_pre(serializer);
needed--;
rc = snprintf((char*)&serializer->buffer.data[serializer->status.buffer.size_used],
needed, "%u", value);
if(rc > 0)
serializer->status.buffer.size_used += rc;
return(0);
} else
#endif
return(ndpi_serialize_binary_int32(_serializer, key, strlen(key), value));
}
@@ -1258,8 +1306,10 @@ int ndpi_serialize_binary_int64(ndpi_serializer *_serializer,
u_int32_t buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
u_int32_t needed;
#ifdef OPTIMIZE_NUMERIC_KEYS
if(ndpi_is_number(key, klen))
return(ndpi_serialize_uint32_int64(_serializer, atoi(key), value));
#endif
needed =
sizeof(u_int8_t) /* type */ +
@@ -1283,13 +1333,12 @@ int ndpi_serialize_binary_int64(ndpi_serializer *_serializer,
serializer->status.buffer.size_used += ndpi_json_string_escape(key, klen,
(char *) &serializer->buffer.data[serializer->status.buffer.size_used], buff_diff);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used], buff_diff,
":");
serializer->buffer.data[serializer->status.buffer.size_used] = ':';
serializer->status.buffer.size_used++;
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
serializer->status.buffer.size_used += snprintf((char *)
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used], buff_diff,
NDPI_I64_FORMAT, (long long int)value);
@@ -1306,7 +1355,7 @@ int ndpi_serialize_binary_int64(ndpi_serializer *_serializer,
} else {
serializer->buffer.data[serializer->status.buffer.size_used++] = (ndpi_serialization_string << 4) | ndpi_serialization_int64;
ndpi_serialize_single_string(serializer, key, klen);
ndpi_serialize_single_uint32(serializer, value);
ndpi_serialize_single_uint64(serializer, value);
}
}
@@ -1329,17 +1378,48 @@ int ndpi_serialize_binary_uint32(ndpi_serializer *_serializer,
u_int32_t buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
u_int32_t needed;
#ifdef OPTIMIZE_CSV_SERIALIZATION
if(serializer->fmt == ndpi_serialization_format_csv) {
/* Key is ignored */
u_int32_t buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
u_int16_t needed;
char _value[16];
snprintf(_value, sizeof(_value), "%u", value);
needed = strlen(_value) + 1 /* CVS separator */;
if(buff_diff < needed) {
if(ndpi_extend_serializer_buffer(&serializer->buffer, needed - buff_diff) < 0)
return(-1);
else
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
if(!(serializer->status.flags & NDPI_SERIALIZER_STATUS_HDR_DONE)) {
if(ndpi_serializer_header_string(serializer, key, klen) < 0)
return(-1);
}
ndpi_serialize_csv_pre(serializer);
needed--;
memcpy(&serializer->buffer.data[serializer->status.buffer.size_used], _value, needed);
serializer->status.buffer.size_used += needed;
return(0);
}
#endif
#ifdef OPTIMIZE_NUMERIC_KEYS
if(ndpi_is_number(key, klen))
return(ndpi_serialize_uint32_uint32(_serializer, atoi(key), value));
#endif
needed =
sizeof(u_int8_t) /* type */ +
sizeof(u_int16_t) /* key len */ +
klen /* key */ +
sizeof(u_int32_t);
if(serializer->fmt == ndpi_serialization_format_json)
needed += 16 + klen;
sizeof(u_int32_t) + /* value (as int) */
20 /* value (as string, in case of CSV) */ +
16 /* extra overhead for JSON */;
if(buff_diff < needed) {
if(ndpi_extend_serializer_buffer(&serializer->buffer, needed - buff_diff) < 0)
@@ -1354,16 +1434,14 @@ int ndpi_serialize_binary_uint32(ndpi_serializer *_serializer,
serializer->status.buffer.size_used += ndpi_json_string_escape(key, klen,
(char *) &serializer->buffer.data[serializer->status.buffer.size_used], buff_diff);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used],
buff_diff, ":");
serializer->buffer.data[serializer->status.buffer.size_used] = ':';
serializer->status.buffer.size_used++;
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used],
buff_diff, "%u", value);
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used],
buff_diff, "%u", value);
ndpi_serialize_json_post(_serializer);
} else if(serializer->fmt == ndpi_serialization_format_csv) {
@@ -1371,8 +1449,8 @@ int ndpi_serialize_binary_uint32(ndpi_serializer *_serializer,
ndpi_serialize_csv_pre(serializer);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used], buff_diff,
"%u", value);
&serializer->buffer.data[serializer->status.buffer.size_used], buff_diff,
"%u", value);
} else {
if(value <= 0xff) {
serializer->buffer.data[serializer->status.buffer.size_used++] = (ndpi_serialization_string << 4) | ndpi_serialization_uint8;
@@ -1431,8 +1509,10 @@ int ndpi_serialize_binary_uint64(ndpi_serializer *_serializer,
u_int32_t buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
u_int32_t needed;
#ifdef OPTIMIZE_NUMERIC_KEYS
if(ndpi_is_number(key, klen))
return(ndpi_serialize_uint32_uint64(_serializer, atoi(key), value));
#endif
needed =
sizeof(u_int8_t) /* type */ +
@@ -1456,13 +1536,12 @@ int ndpi_serialize_binary_uint64(ndpi_serializer *_serializer,
serializer->status.buffer.size_used += ndpi_json_string_escape(key, klen,
(char *) &serializer->buffer.data[serializer->status.buffer.size_used], buff_diff);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used], buff_diff,
":");
serializer->buffer.data[serializer->status.buffer.size_used] = ':';
serializer->status.buffer.size_used++;
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
serializer->status.buffer.size_used += snprintf((char *)
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used], buff_diff,
NDPI_U64_FORMAT, (unsigned long long)value);
@@ -1506,8 +1585,10 @@ int ndpi_serialize_binary_float(ndpi_serializer *_serializer,
u_int32_t buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
u_int32_t needed;
#ifdef OPTIMIZE_NUMERIC_KEYS
if(ndpi_is_number(key, klen))
return(ndpi_serialize_uint32_float(_serializer, atoi(key), value, format));
#endif
needed =
sizeof(u_int8_t) /* type */ +
@@ -1530,9 +1611,9 @@ int ndpi_serialize_binary_float(ndpi_serializer *_serializer,
if (!(serializer->status.flags & NDPI_SERIALIZER_STATUS_LIST)) {
serializer->status.buffer.size_used += ndpi_json_string_escape(key, klen,
(char *) &serializer->buffer.data[serializer->status.buffer.size_used], buff_diff);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
serializer->buffer.data[serializer->status.buffer.size_used] = ':';
serializer->status.buffer.size_used++;
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
serializer->status.buffer.size_used += snprintf((char *) &serializer->buffer.data[serializer->status.buffer.size_used], buff_diff, format, value);
@@ -1598,26 +1679,31 @@ static int ndpi_serialize_binary_raw(ndpi_serializer *_serializer,
if (!(serializer->status.flags & NDPI_SERIALIZER_STATUS_LIST)) {
serializer->status.buffer.size_used += ndpi_json_string_escape(key, klen,
(char *) &serializer->buffer.data[serializer->status.buffer.size_used], buff_diff);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
serializer->status.buffer.size_used += snprintf((char *) &serializer->buffer.data[serializer->status.buffer.size_used], buff_diff, ":");
serializer->buffer.data[serializer->status.buffer.size_used] = ':';
serializer->status.buffer.size_used++;
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
if (escape)
serializer->status.buffer.size_used += ndpi_json_string_escape(value, vlen,
(char *) &serializer->buffer.data[serializer->status.buffer.size_used], buff_diff);
else
serializer->status.buffer.size_used += snprintf((char *) &serializer->buffer.data[serializer->status.buffer.size_used], buff_diff,
value, vlen);
else {
//serializer->status.buffer.size_used += snprintf((char *) &serializer->buffer.data[serializer->status.buffer.size_used], buff_diff, value, vlen);
memcpy(&serializer->buffer.data[serializer->status.buffer.size_used], value, vlen);
serializer->status.buffer.size_used += vlen;
}
ndpi_serialize_json_post(_serializer);
} else if(serializer->fmt == ndpi_serialization_format_csv) {
if (ndpi_serializer_header_string(serializer, key, klen) < 0) return(-1);
ndpi_serialize_csv_pre(serializer);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used], buff_diff,
"%s", value);
//serializer->status.buffer.size_used += snprintf((char *)
// &serializer->buffer.data[serializer->status.buffer.size_used], buff_diff, "%s", value);
memcpy(&serializer->buffer.data[serializer->status.buffer.size_used], value, vlen);
serializer->status.buffer.size_used += vlen;
} else {
serializer->buffer.data[serializer->status.buffer.size_used++] = (ndpi_serialization_string << 4) | ndpi_serialization_string;
@@ -1631,7 +1717,7 @@ static int ndpi_serialize_binary_raw(ndpi_serializer *_serializer,
/* ********************************** */
/* Key is a <string, len> pair, value is a <string, len> pair */
/* Key is a <string, len> pair, value is a <string, len> pair */
int ndpi_serialize_binary_binary(ndpi_serializer *_serializer,
const char *key,
u_int16_t klen,
@@ -1639,10 +1725,12 @@ int ndpi_serialize_binary_binary(ndpi_serializer *_serializer,
u_int16_t vlen) {
const char *value = _value ? _value : "";
#ifdef OPTIMIZE_NUMERIC_KEYS
if(ndpi_is_number(key, klen))
return(ndpi_serialize_uint32_binary(_serializer, atoi(key), value, vlen));
#endif
return ndpi_serialize_binary_raw(_serializer, key, klen, value, vlen, 1 /* escape */);
return(ndpi_serialize_binary_raw(_serializer, key, klen, value, vlen, 1 /* escape */));
}
/* ********************************** */
@@ -1660,6 +1748,33 @@ int ndpi_serialize_string_binary(ndpi_serializer *_serializer,
int ndpi_serialize_string_string(ndpi_serializer *_serializer,
const char *key, const char *_value) {
const char *value = _value ? _value : "";
ndpi_private_serializer *serializer = (ndpi_private_serializer*)_serializer;
#ifdef OPTIMIZE_CSV_SERIALIZATION
if(serializer->fmt == ndpi_serialization_format_csv) {
/* Key is ignored */
u_int32_t buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
u_int16_t needed = strlen(_value) + 1 /* CVS separator */;
if(buff_diff < needed) {
if(ndpi_extend_serializer_buffer(&serializer->buffer, needed - buff_diff) < 0)
return(-1);
else
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
if(!(serializer->status.flags & NDPI_SERIALIZER_STATUS_HDR_DONE)) {
if(ndpi_serializer_header_string(serializer, key, strlen(key)) < 0)
return(-1);
}
ndpi_serialize_csv_pre(serializer);
needed--;
memcpy(&serializer->buffer.data[serializer->status.buffer.size_used], _value, needed);
serializer->status.buffer.size_used += needed;
return(0);
} else
#endif
return(ndpi_serialize_binary_binary(_serializer, key, strlen(key), value, strlen(value)));
}
@@ -1673,19 +1788,20 @@ int ndpi_serialize_string_raw(ndpi_serializer *_serializer,
/* ********************************** */
int ndpi_serialize_string_boolean(ndpi_serializer *_serializer,
const char *key, u_int8_t value) {
int ndpi_serialize_binary_boolean(ndpi_serializer *_serializer,
const char *key, u_int16_t klen, u_int8_t value) {
ndpi_private_serializer *serializer = (ndpi_private_serializer*)_serializer;
u_int32_t buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
u_int16_t klen = strlen(key);
u_int32_t needed;
if(serializer->fmt != ndpi_serialization_format_json &&
serializer->fmt != ndpi_serialization_format_csv)
return -1;
return(-1);
#ifdef OPTIMIZE_NUMERIC_KEYS
if(ndpi_is_number(key, klen))
return(ndpi_serialize_uint32_boolean(_serializer, atoi(key), value));
#endif
needed = klen + 16;
@@ -1701,9 +1817,8 @@ int ndpi_serialize_string_boolean(ndpi_serializer *_serializer,
if (!(serializer->status.flags & NDPI_SERIALIZER_STATUS_LIST)) {
serializer->status.buffer.size_used += ndpi_json_string_escape(key, klen,
(char *) &serializer->buffer.data[serializer->status.buffer.size_used], buff_diff);
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
serializer->status.buffer.size_used += snprintf((char *)
&serializer->buffer.data[serializer->status.buffer.size_used], buff_diff, ":");
serializer->buffer.data[serializer->status.buffer.size_used] = ':';
serializer->status.buffer.size_used++;
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
}
@@ -1726,6 +1841,13 @@ int ndpi_serialize_string_boolean(ndpi_serializer *_serializer,
/* ********************************** */
int ndpi_serialize_string_boolean(ndpi_serializer *_serializer,
const char *key, u_int8_t value) {
return(ndpi_serialize_binary_boolean(_serializer, key, strlen(key), value));
}
/* ********************************** */
int ndpi_serialize_start_of_list_binary(ndpi_serializer *_serializer,
const char *key, u_int16_t klen) {
ndpi_private_serializer *serializer = (ndpi_private_serializer*)_serializer;
@@ -1773,7 +1895,7 @@ int ndpi_serialize_start_of_list(ndpi_serializer *_serializer,
const char *key = _key ? _key : "";
u_int16_t klen = strlen(key);
return ndpi_serialize_start_of_list_binary(_serializer, key, klen);
return(ndpi_serialize_start_of_list_binary(_serializer, key, klen));
}
/* ********************************** */
@@ -1807,7 +1929,7 @@ int ndpi_serialize_start_of_block_binary(ndpi_serializer *_serializer,
u_int32_t buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
u_int32_t needed;
if(serializer->fmt != ndpi_serialization_format_json &&
if(serializer->fmt != ndpi_serialization_format_json &&
serializer->fmt != ndpi_serialization_format_tlv)
return(-1);
@@ -1827,7 +1949,7 @@ int ndpi_serialize_start_of_block_binary(ndpi_serializer *_serializer,
serializer->status.buffer.size_used += snprintf((char *) &serializer->buffer.data[serializer->status.buffer.size_used], buff_diff, ": {");
buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
ndpi_serialize_json_post(_serializer);
serializer->status.flags |= NDPI_SERIALIZER_STATUS_SOB;
} else /* ndpi_serialization_format_tlv */ {
serializer->buffer.data[serializer->status.buffer.size_used++] = ndpi_serialization_start_of_block;
@@ -1845,7 +1967,7 @@ int ndpi_serialize_start_of_block(ndpi_serializer *_serializer,
const char *key = _key ? _key : "";
u_int16_t klen = strlen(key);
return ndpi_serialize_start_of_block_binary(_serializer, key, klen);
return(ndpi_serialize_start_of_block_binary(_serializer, key, klen));
}
/* ********************************** */
@@ -1856,7 +1978,7 @@ int ndpi_serialize_end_of_block(ndpi_serializer *_serializer) {
u_int32_t buff_diff = serializer->buffer.size - serializer->status.buffer.size_used;
u_int32_t needed;
if(serializer->fmt != ndpi_serialization_format_json &&
if(serializer->fmt != ndpi_serialization_format_json &&
serializer->fmt != ndpi_serialization_format_tlv)
return(-1);
@@ -1893,8 +2015,8 @@ void ndpi_serializer_create_snapshot(ndpi_serializer *_serializer) {
(serializer->status.flags & NDPI_SERIALIZER_STATUS_ARRAY) ? " ARRAY" : "",
(serializer->status.flags & NDPI_SERIALIZER_STATUS_EOR) ? " EOR" : ""
);
#endif
#endif
memcpy(&serializer->snapshot, &serializer->status, sizeof(ndpi_private_serializer_status));
serializer->has_snapshot = 1;
}
@@ -1903,7 +2025,7 @@ void ndpi_serializer_create_snapshot(ndpi_serializer *_serializer) {
void ndpi_serializer_rollback_snapshot(ndpi_serializer *_serializer) {
ndpi_private_serializer *serializer = (ndpi_private_serializer*)_serializer;
if(serializer->has_snapshot) {
memcpy(&serializer->status, &serializer->snapshot, sizeof(ndpi_private_serializer_status));
serializer->has_snapshot = 0;
@@ -1915,7 +2037,7 @@ void ndpi_serializer_rollback_snapshot(ndpi_serializer *_serializer) {
serializer->buffer.data[0] = ' ';
serializer->buffer.data[serializer->status.buffer.size_used-1] = '}';
}
}
}
}
}
+1170 -637
View File
File diff suppressed because it is too large Load Diff
+3 -3
View File
@@ -2,7 +2,7 @@
* afp.c
*
* Copyright (C) 2009-11 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -37,13 +37,13 @@ struct afpHeader {
static void ndpi_int_afp_add_connection(struct ndpi_detection_module_struct *ndpi_struct, struct ndpi_flow_struct *flow)
{
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_AFP, NDPI_PROTOCOL_UNKNOWN);
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_AFP, NDPI_PROTOCOL_UNKNOWN, NDPI_CONFIDENCE_DPI);
}
void ndpi_search_afp(struct ndpi_detection_module_struct *ndpi_struct, struct ndpi_flow_struct *flow)
{
struct ndpi_packet_struct *packet = &flow->packet;
struct ndpi_packet_struct *packet = &ndpi_struct->packet;
NDPI_LOG_DBG(ndpi_struct, "search AFP\n");
+26 -51
View File
@@ -1,8 +1,8 @@
/*
* aimini.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -33,25 +33,13 @@
static void ndpi_int_aimini_add_connection(struct ndpi_detection_module_struct *ndpi_struct, struct ndpi_flow_struct *flow/* , */
/* ndpi_protocol_type_t protocol_type */)
{
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_AIMINI, NDPI_PROTOCOL_UNKNOWN);
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_HTTP, NDPI_PROTOCOL_AIMINI, NDPI_CONFIDENCE_DPI);
}
static u_int8_t is_special_aimini_host(struct ndpi_int_one_line_struct host_line)
{
if (host_line.ptr != NULL && host_line.len >= NDPI_STATICSTRING_LEN("X.X.X.X.aimini.net")) {
if ((get_u_int32_t(host_line.ptr, 0) & htonl(0x00ff00ff)) == htonl(0x002e002e) &&
(get_u_int32_t(host_line.ptr, 4) & htonl(0x00ff00ff)) == htonl(0x002e002e) &&
memcmp(&host_line.ptr[8], "aimini.net", NDPI_STATICSTRING_LEN("aimini.net")) == 0) {
return 1;
}
}
return 0;
}
void ndpi_search_aimini(struct ndpi_detection_module_struct *ndpi_struct, struct ndpi_flow_struct *flow)
{
struct ndpi_packet_struct *packet = &flow->packet;
struct ndpi_packet_struct *packet = &ndpi_struct->packet;
NDPI_LOG_DBG(ndpi_struct, "search aimini\n");
@@ -232,45 +220,32 @@ void ndpi_search_aimini(struct ndpi_detection_module_struct *ndpi_struct, struct
ndpi_int_aimini_add_connection(ndpi_struct, flow);
return;
}
} else if (packet->tcp != NULL) {
if ((packet->payload_packet_len > NDPI_STATICSTRING_LEN("GET /player/") &&
(memcmp(packet->payload, "GET /player/", NDPI_STATICSTRING_LEN("GET /player/")) == 0)) ||
(packet->payload_packet_len > NDPI_STATICSTRING_LEN("GET /play/?fid=") &&
(memcmp(packet->payload, "GET /play/?fid=", NDPI_STATICSTRING_LEN("GET /play/?fid=")) == 0))) {
NDPI_LOG_DBG2(ndpi_struct, "HTTP packet detected\n");
ndpi_parse_packet_line_info(ndpi_struct, flow);
if (packet->host_line.ptr != NULL && packet->host_line.len > 11
&& (memcmp(&packet->host_line.ptr[packet->host_line.len - 11], ".aimini.net", 11) == 0)) {
}
if (flow->detected_protocol_stack[0] == NDPI_PROTOCOL_HTTP) {
if (flow->http.method == NDPI_HTTP_METHOD_GET)
{
if ((LINE_STARTS(packet->http_url_name, "/download/") == 1 ||
LINE_STARTS(packet->http_url_name, "/player/") == 1 ||
LINE_STARTS(packet->http_url_name, "/play/") == 1 ||
LINE_STARTS(packet->http_url_name, "/member/") == 1) &&
(LINE_ENDS(packet->host_line, ".aimini.net") == 1 ||
LINE_ENDS(packet->host_line, ".aimini.com") == 1))
{
NDPI_LOG_INFO(ndpi_struct, "found AIMINI HTTP traffic\n");
ndpi_int_aimini_add_connection(ndpi_struct, flow);
return;
}
}
if (packet->payload_packet_len > 100) {
if (memcmp(packet->payload, "GET /", NDPI_STATICSTRING_LEN("GET /")) == 0) {
if (memcmp(&packet->payload[NDPI_STATICSTRING_LEN("GET /")], "play/",
NDPI_STATICSTRING_LEN("play/")) == 0 ||
memcmp(&packet->payload[NDPI_STATICSTRING_LEN("GET /")], "download/",
NDPI_STATICSTRING_LEN("download/")) == 0) {
ndpi_parse_packet_line_info(ndpi_struct, flow);
if (is_special_aimini_host(packet->host_line) == 1) {
NDPI_LOG_INFO(ndpi_struct,
"found AIMINI HTTP traffic\n");
ndpi_int_aimini_add_connection(ndpi_struct, flow);
return;
}
}
} else if (memcmp(packet->payload, "POST /", NDPI_STATICSTRING_LEN("POST /")) == 0) {
if (memcmp(&packet->payload[NDPI_STATICSTRING_LEN("POST /")], "upload/",
NDPI_STATICSTRING_LEN("upload/")) == 0) {
ndpi_parse_packet_line_info(ndpi_struct, flow);
if (is_special_aimini_host(packet->host_line) == 1) {
NDPI_LOG_INFO(ndpi_struct,
"found AIMINI HTTP traffic detected.\n");
ndpi_int_aimini_add_connection(ndpi_struct, flow);
return;
}
}
} else if (flow->http.method == NDPI_HTTP_METHOD_POST)
{
if ((LINE_STARTS(packet->http_url_name, "/upload/") == 1 ||
LINE_STARTS(packet->http_url_name, "/member/") == 1) &&
(LINE_ENDS(packet->host_line, ".aimini.net") == 1 ||
LINE_ENDS(packet->host_line, ".aimini.com") == 1))
{
NDPI_LOG_INFO(ndpi_struct, "found AIMINI HTTP traffic\n");
ndpi_int_aimini_add_connection(ndpi_struct, flow);
return;
}
}
}
+3 -5
View File
@@ -63,7 +63,7 @@ static void set_ajp_detected(struct ndpi_detection_module_struct *ndpi_struct,
/* If no custom protocol has been detected */
/* if(flow->detected_protocol_stack[0] == NDPI_PROTOCOL_UNKNOWN) */
ndpi_int_reset_protocol(flow);
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_AJP, flow->guessed_host_protocol_id);
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_AJP, flow->guessed_host_protocol_id, NDPI_CONFIDENCE_DPI);
}
}
@@ -73,7 +73,7 @@ static void set_ajp_detected(struct ndpi_detection_module_struct *ndpi_struct,
static void ndpi_check_ajp(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow) {
struct ajp_header ajp_hdr;
struct ndpi_packet_struct *packet = &flow->packet;
struct ndpi_packet_struct *packet = &ndpi_struct->packet;
if (packet->payload_packet_len < sizeof(ajp_hdr)) {
NDPI_EXCLUDE_PROTO(ndpi_struct, flow);
@@ -115,15 +115,13 @@ static void ndpi_check_ajp(struct ndpi_detection_module_struct *ndpi_struct,
void ndpi_search_ajp(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow)
{
struct ndpi_packet_struct *packet = &flow->packet;
// Break after 20 packets.
if(flow->packet_counter > 20) {
NDPI_EXCLUDE_PROTO(ndpi_struct, flow);
return;
}
if(packet->detected_protocol_stack[0] != NDPI_PROTOCOL_UNKNOWN) {
if(flow->detected_protocol_stack[0] != NDPI_PROTOCOL_UNKNOWN) {
return;
}
+6 -8
View File
@@ -1,7 +1,7 @@
/*
* amazon_video.c
*
* Copyright (C) 2018-20 by ntop.org
* Copyright (C) 2018-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -29,7 +29,7 @@
static void ndpi_check_amazon_video(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow) {
struct ndpi_packet_struct *packet = &flow->packet;
struct ndpi_packet_struct *packet = &ndpi_struct->packet;
NDPI_LOG_DBG(ndpi_struct, "search Amazon Prime\n");
@@ -40,7 +40,7 @@ static void ndpi_check_amazon_video(struct ndpi_detection_module_struct *ndpi_st
packet->payload[2] == 0xFA &&
packet->payload[3] == 0xCE)) {
NDPI_LOG_INFO(ndpi_struct, "found Amazon Video on TCP\n");
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_AMAZON_VIDEO, NDPI_PROTOCOL_UNKNOWN);
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_AMAZON_VIDEO, NDPI_PROTOCOL_UNKNOWN, NDPI_CONFIDENCE_DPI);
return;
} else if((packet->udp != NULL) &&
(packet->payload[0] == 0xDE &&
@@ -48,7 +48,7 @@ static void ndpi_check_amazon_video(struct ndpi_detection_module_struct *ndpi_st
packet->payload[2] == 0xBE &&
packet->payload[3] == 0xEF)) {
NDPI_LOG_INFO(ndpi_struct, "found Amazon Video on UDP\n");
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_AMAZON_VIDEO, NDPI_PROTOCOL_UNKNOWN);
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_AMAZON_VIDEO, NDPI_PROTOCOL_UNKNOWN, NDPI_CONFIDENCE_DPI);
return;
}
}
@@ -58,12 +58,10 @@ static void ndpi_check_amazon_video(struct ndpi_detection_module_struct *ndpi_st
void ndpi_search_amazon_video(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow) {
struct ndpi_packet_struct *packet = &flow->packet;
NDPI_LOG_DBG(ndpi_struct, "search amazon_video\n");
/* skip marked packets */
if(packet->detected_protocol_stack[0] != NDPI_PROTOCOL_AMAZON_VIDEO)
if(flow->detected_protocol_stack[0] != NDPI_PROTOCOL_AMAZON_VIDEO)
ndpi_check_amazon_video(ndpi_struct, flow);
}
@@ -73,7 +71,7 @@ void init_amazon_video_dissector(struct ndpi_detection_module_struct *ndpi_struc
ndpi_set_bitmask_protocol_detection("AMAZON_VIDEO", ndpi_struct, detection_bitmask, *id,
NDPI_PROTOCOL_AMAZON_VIDEO,
ndpi_search_amazon_video,
NDPI_SELECTION_BITMASK_PROTOCOL_TCP_OR_UDP_WITH_PAYLOAD,
NDPI_SELECTION_BITMASK_PROTOCOL_V4_V6_TCP_OR_UDP_WITH_PAYLOAD,
SAVE_DETECTION_BITMASK_AS_UNKNOWN,
ADD_TO_DETECTION_BITMASK);
*id += 1;
+58
View File
@@ -0,0 +1,58 @@
/*
* among_us.c
*
* Copyright (C) 2020 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#include "ndpi_protocol_ids.h"
#define NDPI_CURRENT_PROTO NDPI_PROTOCOL_AMONG_US
#include "ndpi_api.h"
static void ndpi_int_among_us_add_connection(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow)
{
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_AMONG_US, NDPI_PROTOCOL_UNKNOWN, NDPI_CONFIDENCE_DPI);
}
void ndpi_search_among_us(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow)
{
struct ndpi_packet_struct * const packet = &ndpi_struct->packet;
/* handshake packet */
if (packet->payload_packet_len > 9 &&
ntohl(*(u_int32_t*)&packet->payload[0]) == 0x08000100 &&
ntohl(*(u_int32_t*)&packet->payload[4]) == 0x80d90203)
{
ndpi_int_among_us_add_connection(ndpi_struct, flow);
} else {
NDPI_EXCLUDE_PROTO(ndpi_struct, flow);
}
}
void init_among_us_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id,
NDPI_PROTOCOL_BITMASK *detection_bitmask)
{
ndpi_set_bitmask_protocol_detection(
"AmongUs", ndpi_struct, detection_bitmask, *id,
NDPI_PROTOCOL_AMONG_US, ndpi_search_among_us, NDPI_SELECTION_BITMASK_PROTOCOL_V4_V6_UDP_WITH_PAYLOAD,
SAVE_DETECTION_BITMASK_AS_UNKNOWN, ADD_TO_DETECTION_BITMASK);
*id += 1;
}
+3 -3
View File
@@ -1,7 +1,7 @@
/*
* amqp.c
*
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2011-22 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
@@ -36,11 +36,11 @@ struct amqp_header {
static void ndpi_int_amqp_add_connection(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow/* , */
/* ndpi_protocol_type_t protocol_type */) {
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_AMQP, NDPI_PROTOCOL_UNKNOWN);
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_AMQP, NDPI_PROTOCOL_UNKNOWN, NDPI_CONFIDENCE_DPI);
}
void ndpi_search_amqp(struct ndpi_detection_module_struct *ndpi_struct, struct ndpi_flow_struct *flow) {
struct ndpi_packet_struct *packet = &flow->packet;
struct ndpi_packet_struct *packet = &ndpi_struct->packet;
NDPI_LOG_DBG(ndpi_struct, "search amqp\n");
+50 -20
View File
@@ -27,26 +27,58 @@
#include "ndpi_api.h"
static int is_apple_push_addr(const struct ndpi_packet_struct *packet)
{
if(packet->iph) {
/* 17.0.0.0/8 */
if(((ntohl(packet->iph->saddr) & 0xFF000000 /* 255.0.0.0 */) == 0x11000000) ||
((ntohl(packet->iph->daddr) & 0xFF000000 /* 255.0.0.0 */) == 0x11000000))
return 1;
} else if(packet->iphv6) {
/* 2620:149:a44::/48 */
if(((packet->iphv6->ip6_src.u6_addr.u6_addr32[0] == ntohl(0x26200149)) &&
((packet->iphv6->ip6_src.u6_addr.u6_addr32[1] & htonl (0xffff0000)) == ntohl(0x0a440000))) ||
((packet->iphv6->ip6_dst.u6_addr.u6_addr32[0] == ntohl(0x26200149)) &&
((packet->iphv6->ip6_dst.u6_addr.u6_addr32[1] & htonl (0xffff0000)) == ntohl(0x0a440000))))
return 1;
/* 2403:300:a42::/48 */
if(((packet->iphv6->ip6_src.u6_addr.u6_addr32[0] == ntohl(0x24030300)) &&
((packet->iphv6->ip6_src.u6_addr.u6_addr32[1] & htonl (0xffff0000)) == ntohl(0x0a420000))) ||
((packet->iphv6->ip6_dst.u6_addr.u6_addr32[0] == ntohl(0x24030300)) &&
((packet->iphv6->ip6_dst.u6_addr.u6_addr32[1] & htonl (0xffff0000)) == ntohl(0x0a420000))))
return 1;
/* 2403:300:a51::/48 */
if(((packet->iphv6->ip6_src.u6_addr.u6_addr32[0] == ntohl(0x24030300)) &&
((packet->iphv6->ip6_src.u6_addr.u6_addr32[1] & htonl (0xffff0000)) == ntohl(0x0a510000))) ||
((packet->iphv6->ip6_dst.u6_addr.u6_addr32[0] == ntohl(0x24030300)) &&
((packet->iphv6->ip6_dst.u6_addr.u6_addr32[1] & htonl (0xffff0000)) == ntohl(0x0a510000))))
return 1;
/* 2a01:b740:a42::/48 */
if(((packet->iphv6->ip6_src.u6_addr.u6_addr32[0] == ntohl(0x2a0ab740)) &&
((packet->iphv6->ip6_src.u6_addr.u6_addr32[1] & htonl (0xffff0000)) == ntohl(0x0a420000))) ||
((packet->iphv6->ip6_dst.u6_addr.u6_addr32[0] == ntohl(0x2a0ab740)) &&
((packet->iphv6->ip6_dst.u6_addr.u6_addr32[1] & htonl (0xffff0000)) == ntohl(0x0a420000))))
return 1;
}
return 0;
}
static void ndpi_check_apple_push(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow) {
struct ndpi_packet_struct *packet = &flow->packet;
struct ndpi_packet_struct *packet = &ndpi_struct->packet;
if(packet->iph) {
/* https://support.apple.com/en-us/HT203609 */
if(((ntohl(packet->iph->saddr) & 0xFF000000 /* 255.0.0.0 */) == 0x11000000 /* 17.0.0.0/8 */)
|| ((ntohl(packet->iph->daddr) & 0xFF000000 /* 255.0.0.0 */) == 0x11000000 /* 17.0.0.0/8 */)) {
u_int16_t apple_push_port = ntohs(5223);
u_int16_t notification_apn_port = ntohs(2195);
u_int16_t apn_feedback_port = ntohs(2196);
/* https://support.apple.com/en-us/HT203609 */
if(is_apple_push_addr(packet)) {
u_int16_t apple_push_port = ntohs(5223);
u_int16_t notification_apn_port = ntohs(2197);
if(((packet->tcp->source == apple_push_port) || (packet->tcp->dest == apple_push_port))
|| ((packet->tcp->source == notification_apn_port) || (packet->tcp->dest == notification_apn_port))
|| ((packet->tcp->source == apn_feedback_port) || (packet->tcp->dest == apn_feedback_port))
) {
NDPI_LOG_INFO(ndpi_struct, "found apple_push\n");
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_APPLE_PUSH, NDPI_PROTOCOL_UNKNOWN);
return;
}
if((packet->tcp->source == apple_push_port) || (packet->tcp->dest == apple_push_port) ||
(packet->tcp->source == notification_apn_port) || (packet->tcp->dest == notification_apn_port)) {
NDPI_LOG_INFO(ndpi_struct, "found apple_push\n");
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_APPLE_PUSH, NDPI_PROTOCOL_UNKNOWN, NDPI_CONFIDENCE_DPI);
return;
}
}
@@ -55,12 +87,10 @@ static void ndpi_check_apple_push(struct ndpi_detection_module_struct *ndpi_stru
void ndpi_search_apple_push(struct ndpi_detection_module_struct *ndpi_struct, struct ndpi_flow_struct *flow)
{
struct ndpi_packet_struct *packet = &flow->packet;
NDPI_LOG_DBG(ndpi_struct, "search apple_push\n");
/* skip marked packets */
if(packet->detected_protocol_stack[0] != NDPI_PROTOCOL_APPLE_PUSH)
if(flow->detected_protocol_stack[0] != NDPI_PROTOCOL_APPLE_PUSH)
ndpi_check_apple_push(ndpi_struct, flow);
}
@@ -70,7 +100,7 @@ void init_apple_push_dissector(struct ndpi_detection_module_struct *ndpi_struct,
ndpi_set_bitmask_protocol_detection("APPLE_PUSH", ndpi_struct, detection_bitmask, *id,
NDPI_PROTOCOL_APPLE_PUSH,
ndpi_search_apple_push,
NDPI_SELECTION_BITMASK_PROTOCOL_TCP,
NDPI_SELECTION_BITMASK_PROTOCOL_V4_V6_TCP_WITH_PAYLOAD,
SAVE_DETECTION_BITMASK_AS_UNKNOWN,
ADD_TO_DETECTION_BITMASK);
*id += 1;
+4 -4
View File
@@ -1,8 +1,8 @@
/*
* applejuice.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -32,13 +32,13 @@
static void ndpi_int_applejuice_add_connection(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow)
{
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_APPLEJUICE, NDPI_PROTOCOL_UNKNOWN);
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_APPLEJUICE, NDPI_PROTOCOL_UNKNOWN, NDPI_CONFIDENCE_DPI);
}
void ndpi_search_applejuice_tcp(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow)
{
struct ndpi_packet_struct *packet = &flow->packet;
struct ndpi_packet_struct *packet = &ndpi_struct->packet;
NDPI_LOG_DBG(ndpi_struct, "search applejuice\n");
+4 -4
View File
@@ -1,8 +1,8 @@
/*
* armagetron.c
*
* Copyright (C) 2009-2011 by ipoque GmbH
* Copyright (C) 2011-20 - ntop.org
* Copyright (C) 2009-11 - ipoque GmbH
* Copyright (C) 2011-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -32,12 +32,12 @@
static void ndpi_int_armagetron_add_connection(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow)
{
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_ARMAGETRON, NDPI_PROTOCOL_UNKNOWN);
ndpi_set_detected_protocol(ndpi_struct, flow, NDPI_PROTOCOL_ARMAGETRON, NDPI_PROTOCOL_UNKNOWN, NDPI_CONFIDENCE_DPI);
}
void ndpi_search_armagetron_udp(struct ndpi_detection_module_struct *ndpi_struct, struct ndpi_flow_struct *flow)
{
struct ndpi_packet_struct *packet = &flow->packet;
struct ndpi_packet_struct *packet = &ndpi_struct->packet;
NDPI_LOG_DBG(ndpi_struct, "search armagetron\n");

Some files were not shown because too many files have changed in this diff Show More