Compare commits

...
534 Commits
Author SHA1 Message Date
Luca Deri 2c5258b19d Version 4.4 2022-07-06 12:44:31 +02:00
Toni 175f863665 Label SMTP w/ STARTTLS as SMTPS *and* dissect TLS clho. (#1639)
* Label SMTP w/ STARTTLS as SMTPS *and* dissect TLS clho.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Revert "SMTP with STARTTLS is now identified as SMTPS"

This reverts commit 52d987b603.

* Revert "Compilation fix"

This reverts commit c019946f60.

* Sync unit tests.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-06 12:40:25 +02:00
Luca Deri c019946f60 Compilation fix 2022-07-05 17:38:32 +02:00
Ivan Nardi 7645909460 Fix handling of NDPI_UNIDIRECTIONAL_TRAFFIC risk (#1636) 2022-07-05 17:01:00 +02:00
Luca Deri 52d987b603 SMTP with STARTTLS is now identified as SMTPS 2022-07-05 17:00:21 +02:00
Toni f4a1739f9c Detect SMTPs w/ STARTTLS as TLS and dissect client/server hello. Fixes #1630. (#1637)
* FTP needs to get updated as well as it has similiar STARTTLS semantics -> follow-up

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-05 16:35:23 +02:00
Toni 388dfb8e13 Run regression tests from different locations at the same time w/o side effects on the results. (#1638)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-05 15:08:59 +02:00
Luca Deri 7fa8d882d8 Exported username in flow information 2022-07-04 22:52:54 +02:00
Luca Deri 461589517e Updated ndpi_check_flow_risk_exceptions() signature 2022-07-04 21:38:38 +02:00
Luca Deri e7a5eaecde Cleaned-up issuer DN check code adding
u_int8_t ndpi_check_issuerdn_risk_exception(struct ndpi_detection_module_struct *ndpi_str, char *issuerDN);

Added new API function for checking nDPI-configured exceptions
u_int8_t ndpi_check_flow_risk_exception(struct ndpi_detection_module_struct *ndpi_str,
                                        u_int num_params,
                                        ndpi_risk_params **params);
2022-07-04 18:41:01 +02:00
Luca Deri 8ff2860601 Set CiscoVPN as a network protocol 2022-07-04 18:41:01 +02:00
Toni Uhlig b3ab66020f Updated JA3/SSL fingerprints.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 16:05:22 +02:00
Toni Uhlig b3d3e3b210 Replaced malicious JA3-md5/SSL-cert-sha1 ac automata with hashmaps.
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 16:05:22 +02:00
Toni 4ff8aa48b2 Added UltraSurf protocol dissector. (#1618)
* TLSv1.3 UltraSurf flows are not detected by now

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 16:04:53 +02:00
Ivan Nardi 5aa3d9126f Add two new confidence values: confidence by partial DPI (#1632)
Used for all classifications based on partial/incomplete DPI
information, i.e. all classifications done in `ndpi_detection_giveup()`.
2022-07-04 13:56:51 +02:00
Ivan Nardi 4445989588 Update host content list match (#1633)
Improve classifications of Outlook, Cachefly, Cloudflare, Tiktok and
Cybersecurity.
2022-07-04 13:21:11 +02:00
Toni a1de0285eb Sync Psiphon unit test. (#1634)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 13:20:44 +02:00
Toni 75f7da5c26 Added Psiphon detection patterns. See #566 and #1099. (#1631)
* The traces are not up to date, but this is the best we got so far.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-04 10:34:54 +02:00
Ivan Nardi 192a32207c OCSP: improve detection (#1629) 2022-07-04 07:22:56 +02:00
Toni a74fc089c4 Added i3D and RiotGames protocol dissectors. (#1609)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 20:43:30 +02:00
Ivan Nardi faaff58620 TargusDataspeed: avoid false positives (#1628)
TargusDataspeed dissector doesn't perform any real DPI checks but it only
looks at the TCP/UDP ports.
Delete it, and use standard logic to classify these flows by port.
2022-07-03 20:28:58 +02:00
Ivan Nardi 50c0212df1 Update ASN/IPs lists (#1627) 2022-07-03 19:50:47 +02:00
Ivan Nardi b5fb2066cb bins: add support for 64bit bins (#1626) 2022-07-03 19:25:15 +02:00
Ivan Nardi 422d002542 Skinny: rework and improve classification (#1625) 2022-07-03 19:25:00 +02:00
Ivan Nardi eed47acfc8 Skype_Teams, Mining, SnapchatCall: fix flow category (#1624) 2022-07-03 18:51:16 +02:00
Ivan Nardi 77ac58e553 Minor changes in how classification results are set (#1623)
Protocol classification should always be set via
`ndpi_set_detected_protocol()`: this way, the values in
`flow->detected_protocol_stack[]` are always coherent.
2022-07-03 18:45:24 +02:00
Ivan Nardi 060e894d5b Usenet: improve dissection (#1622) 2022-07-03 18:08:04 +02:00
Ivan Nardi fdb1649a49 Fix category for mail sessions (#1621)
Close #629
2022-07-03 17:47:58 +02:00
Ivan Nardi 5fe6087686 TLS: add support for old DTLS versions and for detection of mid-sessions (#1619) 2022-07-03 17:44:17 +02:00
Ivan Nardi 5f6fa6d164 Fix a compilation warning (#1620)
With clang-15 (nightly build)
```
In file included from ndpi_bitmap.c:39:
./third_party/src/roaring.cc:14233:13: warning: variable 'run_count' set but not used [-Wunused-but-set-variable]
        int run_count = 0;
```
2022-07-03 17:43:38 +02:00
Toni Uhlig 69ccb39741 Generate profiling results as PNG.
* use -ltcmalloc_and_profiler and try to get rid of LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libprofiler.so

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 17:38:43 +02:00
lns f2d1edbedf gprof test/CI integration
Signed-off-by: lns <matzeton@googlemail.com>
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 17:38:43 +02:00
Toni 1a01e8dc68 Improved TFTP. Dissect Read/Write Request filenames. (#1617)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 14:37:05 +02:00
Toni 5ca82ad84b Added TSAN support. (#1613)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 14:35:21 +02:00
Toni 59b00b00a7 Fix byte-order issue during ndpiReader tcp/udp src/dst port serialization. Fixes #1608. (#1614)
* fixed possible memory leak caused by an invalid call to `node_proto_guess_walker()` during serialization
 * execute serialization code while running regression tests

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-03 11:16:52 +02:00
Toni 7c5c811eb0 Added Cloudflare WARP detection patterns. (#1615) (#1616)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-02 14:57:56 +02:00
Luca Deri 008a1790e4 Fixed SMTP default port 587 2022-07-02 11:49:22 +02:00
Toni bb72aa4767 Added TunnelBear VPN detection patterns. (#1615)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-07-01 13:19:17 +02:00
Luca Deri 8f6a006e36 Updated (C) 2022-06-30 14:53:47 +02:00
Luca Deri 6cf7838295 Removed space from "Genshin Impact" 2022-06-30 14:50:05 +02:00
Toni c96f4512fa sync unit tests (#1612)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-06-30 09:37:30 +02:00
Luca Deri 5f1caeb54e Fix after the protocol name update 2022-06-28 17:26:11 +02:00
Luca d393fb969d Renamed Z39.50 -> Z3950 as the '.' breaks the naming convention
QUIC is a network protocol
2022-06-28 17:20:58 +02:00
Luca Deri 227ab5c105 Enhanced TLS risk info reported to users 2022-06-28 00:01:00 +02:00
Luca Deri 4296ec969a Added default port for syslog TCP 2022-06-27 18:36:56 +02:00
Ivan Nardi b2b61011ae Fix compilation and sync unit tests results (#1606) 2022-06-20 14:27:13 +02:00
Luca Deri ab09b8ce2e Added unidirectional traffic flow risk 2022-06-20 00:22:13 +02:00
Toni c287eb835b Improved SOAP via HTTP. (#1605)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-18 17:19:16 +02:00
Toni 6cd8f8cc6d Improved GenshinImpact protocol dissector. (#1604)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-18 15:11:59 +02:00
Toni 432de5eb57 Added collectd dissector (again). (#1601)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-17 19:56:33 +02:00
Toni 9c8b2d63da Replaced nDPI's internal hashmap with uthash. (#1602)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-17 19:50:31 +02:00
Toni 20a29c393f Improved IPSec/ISAKMP detection. (#1600)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-16 09:16:18 +02:00
Luca c43360f8e6 Added new test pcaps 2022-06-15 18:57:46 +02:00
claudio_burrafatoandToni c73a0988cd Add some statistics to ndpiReader (#1587)
* Add some statistics to ndpiReader

The purpose of this version of ndpiReader is too adding some other statistics printed by ndpiReader. In this simple version the domain names(in the flows) that are collected are:
flow-> ssh_tls.server_info
flow-> host_server_name
and are placed in a UT_hash_table, ordering them by number of occurrences.

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update example/ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>

* Update ndpiReader.c

* Update ndpiReader.c

* Update ndpiReader.c

* Update ndpiReader.c

Co-authored-by: Toni <matzeton@googlemail.com>
2022-06-15 16:36:25 +02:00
Ivan Nardi e2cc08bfe5 Add support for PIM (Protocol Indipendent Multicast) protocol (#1599)
Close #1598
2022-06-15 12:25:26 +02:00
Toni d1773cc8e3 Improved WhatsApp detection. (#1595)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-14 20:06:48 +02:00
Ivan Nardi bdf54d725b Fix invalid memory access (#1596)
We can access `flow->protos` union only after checking the protocol.

Checking `flow->detected_protocol.master_protocol` is redundant because
we already check it in `is_ndpi_proto`

```
AddressSanitizer:DEADLYSIGNAL
=================================================================
==29739==ERROR: AddressSanitizer: SEGV on unknown address 0x000000353820 (pc 0x7f9b64dd2717 bp 0x7fff161a52f0 sp 0x7fff161a4aa8 T0)
==29739==The signal is caused by a READ memory access.
    #0 0x7f9b64dd2717  /build/glibc-SzIz7B/glibc-2.31/string/../sysdeps/x86_64/multiarch/strlen-avx2.S:96
    #1 0x555c65e597d8 in __interceptor_strlen (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x6407d8) (BuildId: 11ac8ec30f1d49fb0276c9b03368e491505d2bba)
    #2 0x555c65fd85fa in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:269:13
    #3 0x555c65f3e8c6 in process_ndpi_collected_info /home/ivan/svnrepos/nDPI/example/reader_util.c:1188:36
    #4 0x555c65f52cab in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1567:2
    #5 0x555c65f4b632 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2110:10
    #6 0x555c65f04d29 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:109:7
    #7 0x555c65f054bb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:181:17
    #8 0x7f9b64c6e082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #9 0x555c65e4253d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x62953d) (BuildId: 11ac8ec30f1d49fb0276c9b03368e491505d2bba)

```

Found by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=48020
2022-06-14 11:44:32 +02:00
Ivan Nardi 8dcaa5c0e1 DNS: fix TTL check and sync unit test results (#1594) 2022-06-14 09:29:16 +02:00
Luca Deri defe7d7f79 Updated DNS alert triggered only with TTL == 0 2022-06-14 00:13:05 +02:00
Luca Deri 831562c141 Restored ndpi_set_proto_defaults() prototype
Updated test results
2022-06-13 23:53:32 +02:00
Luca Deri 7a172ce11e Added check for DGA names that resolve to a valid record 2022-06-13 23:42:07 +02:00
Luca Deri cf5873ffd7 Improved DNS traffic analysis
Added ability to identify application and network protocols
2022-06-13 23:19:47 +02:00
Luca Deri 32428026d6 Added DNS record TTL check 2022-06-13 17:05:56 +02:00
Toni df0ff9bcbd Added gprof CPU/HEAP profiling support. (#1592)
* Some small auto{conf,make} improvements

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-12 21:00:41 +02:00
Toni 341f58fd80 Removed Makefile references to legacy code. (#1589)
- feature removed with f5545a80f9

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-08 13:37:11 +02:00
Luca Deri 11babc7ea3 Added Pragmatic General Multicast (PGM) protocol detection 2022-06-08 09:11:22 +02:00
Toni bb3c2e82de Dissect host line if SSDP contains such. (#1586)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-07 11:15:47 +02:00
Toni 938e89ca33 Reimplemented 1kxun application protocol. (#1585)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-06 18:04:49 +02:00
Toni df78e67518 Prevent compilation failure if, for whatever reason, NDPI_API_VERSION is empty. (#1584)
- This is not a fix for #1494, just an improvment.

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-06 15:53:48 +02:00
Toni 0b3f8ed849 Fixed syslog false negatives. (#1582)
- RSH vs Syslog may still happen for midstream traffic

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-05 23:01:56 +02:00
Ivan Nardi 1d492c9ed8 Fix some debug messages (#1583)
Increase max number of flows handled during fuzzing
2022-06-05 14:45:21 +02:00
Luca Deri 59c084b166 Updated test results 2022-06-05 11:34:10 +02:00
Luca Deri 2202365b95 Fixed invalid DHCP dissection 2022-06-05 11:19:28 +02:00
Luca Deri b6ae9642d3 Fixed DHCP dissection bug 2022-06-05 11:14:23 +02:00
Toni 7419cfee64 Added RSH dissector. Fixes #202. (#1581)
- added syslog false-positive pcap that was missing in 09fbe0a64a
 - added NDPI_ARRAY_LENGTH() macro, usable on `type var[]` declarations

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-04 19:12:53 +02:00
Ivan Nardi 338427d712 Add support for GoTo products (mainly GoToMeeting) (#1580)
There is some overlap with Citrix protocol.
2022-06-04 14:30:52 +02:00
Toni ff8e1e197e Fix syslog heap overflow introduced in 09fbe0a64a. (#1579)
- fixes #1578

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-04 08:47:47 +02:00
Toni 09fbe0a64a Fixed syslog false positives. (#1577)
* syslog: removed unnecessary/unreliable printable string check
 * added `ndpi_isalnum()`
 * splitted `ndpi_is_printable_string()` into `ndpi_is_printable_buffer()` and `ndpi_normalize_printable_string()`

Signed-off-by: lns <matzeton@googlemail.com>
2022-06-03 18:21:29 +02:00
Toni 6149c0f880 Fix heap buffer overflow mentioned in #1574. (#1576)
Signed-off-by: lns <matzeton@googlemail.com>
2022-06-02 13:07:09 +02:00
Ivan Nardi 4ff1bf29df TLS: fix use-of-uninitialized-value error (#1573)
Proper fix for the error already reported in 9040bc74

```
Uninitialized bytes in __interceptor_strlen at offset 3 inside [0x7ffc7a147390, 4)
==111876==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x55e3e4f32e5b in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:268:13
    #1 0x55e3e4ef7391 in ndpi_set_risk /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:2254:12
    #2 0x55e3e5022fdf in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1523:8
    #3 0x55e3e503af44 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:865:5
    #4 0x55e3e50397cd in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1024:2
    #5 0x55e3e503570c in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2453:5
    #6 0x55e3e4f84a6a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5150:6
    #7 0x55e3e4f85778 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5198:12
    #8 0x55e3e4f851e1 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5217:12
    #9 0x55e3e4f96c7a in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6076:15
    #10 0x55e3e4ed91ef in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #11 0x55e3e4eda27e in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #12 0x7f5cb3146082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #13 0x55e3e4e5140d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0xa340d) (BuildId: 0c02c433e039970dd13a60382b94dd5a8e19f625)
```
2022-06-02 00:15:52 +02:00
Toni eac11030ac Removed README.nDPI as it does not provide any new information not covered by README.md (#1572)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-31 22:11:48 +02:00
Toni 00c62a7c3c Removed LGTM ql query for packet payload integer arithmetic. (#1570)
* Detection is unreliable and results in too much false-positives (and false-negatives!)

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-31 16:46:44 +02:00
Toni d8d806c79f Force roaring bitmap to use ndpi memory wrappers. (#1569)
GCC analyzer won't complain about possible use-after-free (false positive).

 * tests/do.sh prints word diff's only once and not the same over and over again
 * sync unit tests

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-31 12:06:01 +02:00
Ivan Nardi 9040bc74c9 TLS: fix stack-buffer-overflow error (#1567)
```
==300852==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f108951f060 at pc 0x5641db0ee78c bp 0x7fff3b10b910 sp 0x7fff3b10b0d0
WRITE of size 116 at 0x7f108951f060 thread T0
    #0 0x5641db0ee78b in __interceptor_strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x64e78b) (BuildId: 23cb34bbaf8ac11eb97563bbdc12e29ead9fb0fa)
    #1 0x5641db28efad in tlsCheckUncommonALPN /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1224:7
    #2 0x5641db27ec76 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1533:6
    #3 0x5641db295677 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:865:5
    #4 0x5641db2935bb in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1024:2
    #5 0x5641db28f692 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2446:5
    #6 0x5641db1d87ce in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5150:6
    #7 0x5641db1d91e4 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5198:12
    #8 0x5641db1d8e87 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5217:12
    #9 0x5641db1eb4a7 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6076:15
    #10 0x5641db140b45 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #11 0x5641db14130b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #12 0x7f108bcab082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #13 0x5641db07f46d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x5df46d) (BuildId: 23cb34bbaf8ac11eb97563bbdc12e29ead9fb0fa)

Address 0x7f108951f060 is located in stack of thread T0 at offset 96 in frame
    #0 0x5641db28ec4f in tlsCheckUncommonALPN /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1204

  This frame has 1 object(s):
    [32, 96) 'str' (line 1218) <== Memory access at offset 96 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x64e78b) (BuildId: 23cb34bbaf8ac11eb97563bbdc12e29ead9fb0fa) in __interceptor_strncpy
Shadow bytes around the buggy address:
```

Avoid zeroing the entire string.

Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47730
2022-05-31 10:33:42 +02:00
Luca Deri 354addd693 Updated risk results 2022-05-30 23:28:59 +02:00
Luca Deri b306a4b8ed Improved message for known proto on non std port 2022-05-30 23:23:18 +02:00
Luca Deri b3bf695723 Added check 2022-05-30 22:39:23 +02:00
Toni 00ee364a7e Updated README.md (#1562)
* make check great again (not so much)
 * make doc/doc-view
 * CI updates

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-30 19:48:05 +02:00
Ivan Nardi ca115774c9 TLS: fix use-of-uninitialized-value error (#1566)
```
Uninitialized bytes in __interceptor_strlen at offset 0 inside [0x7ffc987443a0, 7)
==282918==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x5651277e001b in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:268:13
    #1 0x5651277a45c1 in ndpi_set_risk /home/ivan/svnrepos/nDPI/src/lib/ndpi_utils.c:2254:12
    #2 0x5651278cfaa6 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1515:8
    #3 0x5651278e7834 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:865:5
    #4 0x5651278e60bd in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1024:2
    #5 0x5651278e1ffc in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2445:5
    #6 0x565127831c2a in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5150:6
    #7 0x565127832938 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5198:12
    #8 0x5651278323a1 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5217:12
    #9 0x565127843e3a in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6076:15
    #10 0x56512778641f in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:29:5
    #11 0x5651277874ae in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:101:17
    #12 0x7f4c46794082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #13 0x5651276fe40d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0xa340d)
```
Found by oss-fuzzer
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47728
2022-05-30 19:40:47 +02:00
Luca Deri 912b69af60 Redefined type name to avoid conflicts 2022-05-30 18:29:08 +02:00
Luca 34ede63c15 Added ability to return risk info in JSON format in ndpi_get_flow_risk_info() 2022-05-30 17:54:30 +02:00
Toni 48065d80e9 Support word diff for tests/do.sh for better readability. (#1565)
* Sync unit tests

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-30 17:33:03 +02:00
Toni 32750271c3 Prohibit MPEG-DASH to set HTTP as application protocol. (#1560)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-30 14:54:27 +02:00
Ivan Nardi c4f50b2cda HTTP: fix heap-buffer-overflow error (#1564)
```
==222479==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60b000014739 at pc 0x55af06f2364f bp 0x7ffd7b6f4bf0 sp 0x7ffd7b6f4378
READ of size 12 at 0x60b000014739 thread T0
    #0 0x55af06f2364e in printf_common(void*, char const*, __va_list_tag*) asan_interceptors.cpp.o
    #1 0x55af06f24f70 in __interceptor_snprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x613f70) (BuildId: f6545ec2bd7663bc3f16aeeb87bddc64d173a2a8)
    #2 0x55af0720927f in ndpi_check_http_header /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:982:2
    #3 0x55af071f4797 in ndpi_check_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1219:5
    #4 0x55af071f05c5 in ndpi_search_http_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/http.c:1402:3
    #5 0x55af07080d1e in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5150:6
    #6 0x55af07081734 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5198:12
    #7 0x55af070813d7 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5217:12
    #8 0x55af070939f7 in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6076:15
    #9 0x55af06fc7e1f in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1541
    #10 0x55af06fc7e1f in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2110
    #11 0x55af06fc2859 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:109:7
    #12 0x55af06fc2feb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:181:17
    #13 0x7efe5eaac082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16
    #14 0x55af06f0055d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x5ef55d) (BuildId: f6545ec2bd7663bc3f16aeeb87bddc64d173a2a8)

0x60b000014739 is located 0 bytes to the right of 105-byte region [0x60b0000146d0,0x60b000014739)
allocated by thread T0 here:
    #0 0x55af06f84bae in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x673bae) (BuildId: f6545ec2bd7663bc3f16aeeb87bddc64d173a2a8)
    #1 0x55af06fc2673 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:103:31
    #2 0x55af06fc2feb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:181:17
    #3 0x7efe5eaac082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/../csu/libc-start.c:308:16

```
Fiund by oss-fuzzer.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47724
2022-05-30 14:39:01 +02:00
Ivan Nardi 3b825fca6d Certificate timestamps should be printed in UTC (#1563)
This way, the CI results are consistent
2022-05-30 13:43:05 +02:00
Luca Deri 897f8314b2 Fixed dispay bug for risk_info 2022-05-30 01:12:38 +02:00
Luca Deri 1da9f1a36f Updated tests results
Code cleanup
2022-05-30 00:54:17 +02:00
Luca Deri f25deeccb1 Added RiskInfo string 2022-05-30 00:32:32 +02:00
Ivan Nardi a9d7cc4841 Fix dissection of IPv4 header (#1561)
See: https://github.com/ntop/nDPI/runs/6643914510?check_suite_focus=true

Convert al the `MIN(a,b)` calls to `ndpi_min(a,b)`
2022-05-29 22:39:48 +02:00
Ivan Nardi 9c1a53f39f Dazn: add support for Dazn streaming service (#1559)
Update .gitignore file
2022-05-29 17:47:16 +02:00
Luca Deri 406ac81a37 Compilation fixes for old ggc's 2022-05-29 16:40:16 +02:00
Luca Deri dfbe863f4e Comment 2022-05-29 16:35:08 +02:00
Luca Deri 1093aafa5f Added detection for WordPress exploits
Fixed ndpi_iph_is_valid_and_not_fragmented() that was bugged with non UDP traffic
2022-05-29 16:31:26 +02:00
Toni 6b7b23b01d Use Doxygen to generate the API documentation. (#1558)
* Integrated Doxygen documentation into Sphinx

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-29 13:44:52 +02:00
Toni 33f9729ee4 Added MPEG-DASH dissector. Fixes #1223. (#1555)
* Improved HTTP POST detection
 * Refactored subprotocol detection

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-29 13:12:13 +02:00
Toni cfd23e8819 Fixed HTTP lower/upper protocol mess for Aimini/IPP. (#1557)
* If HTTP was already detected, there is no need to overwrite the detection again

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-28 16:31:35 +02:00
Luca Deri 77dc884a4c Compilation fixes for old gcc compiler 2022-05-28 16:06:34 +02:00
Luca Deri 7e3d061428 Compilation fixes 2022-05-27 17:35:30 +02:00
Luca Deri 6d543808ad Version cut fix 2022-05-27 16:58:39 +02:00
Luca Deri 02ba12b0de Fixes compilation issues on RedHat systems 2022-05-26 23:34:34 +02:00
Ivan Nardi f81f0c5e69 Sync unit test results (#1554) 2022-05-26 20:15:30 +02:00
Luca Deri cedbd32030 Updated SkypeCall -> Skype_TeamsCall 2022-05-26 19:27:12 +02:00
Luca Deri 1645fea85c Fixed false positives with NATS 2022-05-26 19:01:13 +02:00
Toni 78dfe959cb Added script to compare and verify the output of `make dist'. (#1551)
This fixes some build/test issues resulting when using tarballs.

 * nDPI uses autotools (especially autoconf) in a wrong way, see #1163

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-26 15:07:22 +02:00
Toni 9614b66b44 Replaced obsolete autoconf macros. (#1553)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-26 11:31:09 +02:00
Toni 7010d17ccc Fixed windows-latest build error. (#1552)
* The warning itself looks like a bug

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-26 10:19:57 +02:00
Luca Deri 83e1ea2506 Improved invalid host detection 2022-05-24 21:17:33 +02:00
Luca Deri 0c703ef8eb Added invalid SNI check in QUIC 2022-05-24 19:51:49 +02:00
Luca Deri 4f9dee164e Improved detection of invalid SNI and hostnames in TLS, HTTP 2022-05-24 19:47:54 +02:00
Luca Deri 2560260a41 Added room for storing information used by custom third-party dissectors 2022-05-19 23:24:42 +02:00
lns 8b750686c9 Moved RTSP http patterns to the protocol source file.
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-19 19:58:11 +02:00
Toni Uhlig dae7986a37 Yet another approach to fix #1499 (basically a copy&pasta from @socketpair).
* Related to #1545, #1494 and #1189 as well

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-05-16 16:54:18 +02:00
Toni Uhlig 3c08a3c875 Removed MacOS XCode integration.
* It is outdated (from 2018) and will most likely not work anymore
   due to changed protocol files and outdated API usage in the example integration
 * Removal is the only option besides fixing those issues and
   provide a CI integration for future automatic checks

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-05-16 16:54:18 +02:00
Toni Uhlig 7162bf6abf Moved mgcp.pcapng to tests/pcap/ instead of tests/
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-05-16 16:54:18 +02:00
Ivan Nardi f822aad8b0 DNS-over-QUIC: update default port (#1548)
Final RFC (https://www.rfc-editor.org/rfc/rfc9250.txt) changed the
default port to 853 (from the 784 used on the early drafts).
Since there is at least one deployed implementation still stuck at
draft-02, keep also the old value, for the time being.
2022-05-12 14:31:26 +02:00
Toni 054d151373 Improved Viber (TCP) detection. (#1547)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-10 21:37:03 +02:00
Toni 704920414a Improved Xiaomi HTTP detection. (#1546)
* Merged Xiaomi pcap files

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-10 11:10:04 +02:00
Toni 6fc29b3ae8 Removed TLS patterns in the CiscoVPN aka Anyconnect dissector as mentioned in PR #1534. (#1543)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-09 10:29:52 +02:00
Toni 915ffebade Added Softether(-VPN) DDNS service detection. (#1544)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-09 08:16:19 +02:00
Toni 4319d760e0 Improved TLS alert detection. (#1542)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-08 20:33:59 +02:00
Toni 34882d9cf0 Improved TLS application data detection. (#1541)
* #1532 did fx TLS appdata detection only partially
 * use flow->l4.tcp.tls.message.buffer_used instead of packet->payload

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-08 19:56:08 +02:00
Toni b2648a45a3 Added Edgecast and Cachefly CDNs. (#1540)
* Improved ASN update script
 * Ran `utils/update_every_lists.sh'
 * `tests/do.sh.in' prints the amount of failed pcap(s)
 * `utils/asn_update.sh' prints the amount of failed download(s)

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-07 09:33:25 +02:00
Toni 87f93ea4fd Replaced ndpiReader's libjson-c support with libnDPI's internal serialization interface. (#1535)
* Fixes #1528
 * Serialization Interface should also fuzzed
 * libjson-c may only be used in the unit test to verify the internal serialization interface
 * Serialization Interface supports tlv(broken), csv and json
 * Unit test does work again and requires libjson-c

Signed-off-by: lns <matzeton@googlemail.com>
2022-05-07 09:26:09 +02:00
Ivan Nardi 2e0dedbaae Fix compilation (if --enable-debug-messages is used) (#1539) 2022-05-04 18:43:37 +02:00
Luca Deri 434c38c74d Added extra check to make sure that the guessed protocol is the one we expect and not another one 2022-05-04 16:27:36 +02:00
Luca Deri 5996cf1725 Fixes bug that prevents triggering alerts for traffic on non-standard ports that have been defined in the custom protocols file
Fixes https://github.com/ntop/ntopng/issues/6458
2022-05-04 16:07:05 +02:00
Luca Deri 81a367f1ae Fixes outdated description 2022-05-04 16:06:50 +02:00
Luca Deri 5dcb9c1c16 Modified risk labels 2022-05-04 12:10:01 +02:00
Toni d5b922cfe4 Added some Pluralsight Hostnames/SNIs. May fix #1501. (#1538)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-03 13:20:44 +02:00
Luca Deri 65dd052a61 Updated RRD dependencies 2022-05-03 12:37:22 +02:00
Toni 47d6a65522 Improved suspicious http user agent detection. (#1537)
Signed-off-by: lns <matzeton@googlemail.com>
2022-05-02 19:17:32 +02:00
Luca Deri c8f72ef764 Added ndpi_get_flow_error_code() API call
Fixed typo
2022-05-02 10:51:08 +02:00
Toni 02d0b5fe13 Improved AES-NI check. (#1536)
* A library should not open a subshell

Signed-off-by: lns <matzeton@googlemail.com>
2022-04-30 20:46:20 +02:00
Luca Deri e4318ffc2d Improved AES-NI check on Linux to avoid crashes on CPUs that do
not support it (e.g. Intel Celeron N2930)
2022-04-29 14:50:21 +02:00
Ivan Nardi 71636dcafd Sync unit tests results (#1533) 2022-04-27 18:22:11 +02:00
Toni 10161448bc Improved TLS application data detection. (#1532)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-27 17:22:53 +02:00
Luca Deri 3ad989f6a8 Added BPF filtering for discarding non-IP packets 2022-04-27 17:05:33 +02:00
Luca Deri 3d755fce73 String messages have been shrinked 2022-04-27 10:28:06 +02:00
Luca Deri a7c1152397 Added ability to store custom category file in patricia tree 2022-04-26 14:42:31 +02:00
Alfredo Cardigliano e8d81123cc Add ndpi_json_string_escape to the API 2022-04-26 12:25:34 +02:00
Ivan Nardi d5005f584c Raknet: fix heap-buffer-overflow (#1531)
```
==120637==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x606000000488 at pc 0x55a0598d97ec bp 0x7ffcfe13f2e0 sp 0x7ffcfe13f2d8
READ of size 2 at 0x606000000488 thread T0
    #0 0x55a0598d97eb in ndpi_search_raknet /home/ivan/svnrepos/nDPI/src/lib/protocols/raknet.c:152:38
    #1 0x55a05966c48e in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5060:6
    #2 0x55a05966e428 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5119:10
    #3 0x55a05966dddc in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5152:12
    #4 0x55a05967fa7a in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5971:15
    #5 0x55a05962b860 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:24:3
    #6 0x55a05962bd9b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:84:17
    #7 0x7f4aad7d80b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
    #8 0x55a05956b46d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x5d746d) (BuildId: 9429d3d08edc3836e5536f93c07f140716d8b82e)

0x606000000488 is located 9 bytes to the right of 63-byte region [0x606000000440,0x60600000047f)
allocated by thread T0 here:
    #0 0x55a0595ee80e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x65a80e) (BuildId: 9429d3d08edc3836e5536f93c07f140716d8b82e)
    #1 0x55a05962bd06 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:70:17
    #2 0x7f4aad7d80b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=47000
2022-04-25 19:01:11 +02:00
Toni bc2ad3407a Added generic user agent setter. (#1530)
* ndpiReader: Print user agent if one was set and not just for certain protocols.

Signed-off-by: lns <matzeton@googlemail.com>
2022-04-25 13:00:50 +02:00
Ivan Nardi 075bce5f3d XIAOMI: add detection of Xiaomi traffic (#1529)
Most of the credits should go to @utoni (see #1521)
2022-04-25 11:00:02 +02:00
Toni ac0f50b561 Added RakNet protocol dissector. (#1527)
* Frame Set PDU's do not get fully dissected for the sake of simplicity

Signed-off-by: lns <matzeton@googlemail.com>
2022-04-24 13:08:57 +02:00
Luca Deri 9d31b7450b Code cleanup (removed redundancy) 2022-04-22 19:03:50 +02:00
Toni d5ae9170ba Tiny gnutella improvement if gtk-gnutella used. (#1525)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-22 18:33:27 +02:00
Toni 824c343602 Updated `utils/whatsapp_ip_addresses_download.sh' to scrape the required IP addresses/ranges. (#1524)
* Replaced return statements in `utils/*.sh' with exit's (such scripts should never source'd)
 * Ran `utils/update_every_lists.sh'

Signed-off-by: lns <matzeton@googlemail.com>
2022-04-22 13:27:41 +02:00
Ivan Nardi 42909673ce Add some scripts to easily update some IPs lists (#1522)
Follow-up of 8b062295

Add a new protocol id for generic Tencent/Wechat flows
2022-04-21 20:43:52 +02:00
Toni ddc256b09e Reduce ndpiReader's -h' spam. -H' does this job now. (#1523)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-21 20:43:36 +02:00
Toni c1d46ebc4e Added proprietary Agora Software Defined Real-time Network (SD-RTN) protocol dissector. (#1520)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-20 18:31:28 +02:00
Toni dbf2fb8c9f Added Toca Boca protocol dissector. (#1517)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-19 19:32:49 +02:00
Toni a83ca555fb Removed superfluous ifdef'd includes. (#1519)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-19 17:51:48 +02:00
Toni fa79f07d15 Improved sflow protocol detection false-positives. (#1518)
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-19 17:46:40 +02:00
Ivan Nardi 739dfc54b0 Kerberos: fix Undefined-shift error (#1516)
Similar to the error fixed in 4775be3d
Found by oss-fuzz.
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=46713
2022-04-18 20:54:13 +02:00
Luca Deri 8b2c9860be DGA improvements 2022-04-16 09:56:08 +02:00
Zied Aouini 9c8a3f270a Minor fix. 2022-04-15 18:01:34 +02:00
Zied Aouini fba75a3cf5 Merge pull request #1491 from utoni/fix/windows-msys2
Fixed msys2 build and re-activated CI Mingw-w64 build.
2022-04-15 13:22:25 +02:00
Toni Uhlig c3df3a12aa Fixed msys2 build warnings and re-activated CI Mingw64 build.
* Removed Visual Studio leftovers. Maintaining an autotools project with VS integration requires some additional overhead.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-14 19:17:48 +02:00
Ivan Nardi 4775be3d85 Kerberos: fix some memory access errors (#1514)
```
==19724==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60e00000045e at pc 0x5620b8b3d3cc bp 0x7ffe0fda6b50 sp 0x7ffe0fda6310
READ of size 2 at 0x60e00000045e thread T0
    #0 0x5620b8b3d3cb in __interceptor_strncpy (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x63f3cb) (BuildId: ee53ff920c8cd4c226d8520a0d4846d8864726b6)
    #1 0x5620b8d9b69c in strncpy_lower /home/ivan/svnrepos/nDPI/src/lib/protocols/kerberos.c:208:4
    #2 0x5620b8d995a0 in krb_parse /home/ivan/svnrepos/nDPI/src/lib/protocols/kerberos.c:316:5
    #3 0x5620b8d97a90 in ndpi_search_kerberos /home/ivan/svnrepos/nDPI/src/lib/protocols/kerberos.c:687:12
    #4 0x5620b8bcef35 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4996:4
    #5 0x5620b8bd1be8 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5072:10
    #6 0x5620b8bd159c in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5105:12
    #7 0x5620b8be323a in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5924:15
    #8 0x5620b8b8f7e0 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:24:3
    #9 0x5620b8b8fd1b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:84:17
    #10 0x7f45b32b90b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
    #11 0x5620b8acf47d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x5d147d) (BuildId: ee53ff920c8cd4c226d8520a0d4846d8864726b6)

0x60e00000045e is located 0 bytes to the right of 158-byte region [0x60e0000003c0,0x60e00000045e)
allocated by thread T0 here:
    #0 0x5620b8b5283e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet_with_main+0x65483e) (BuildId: ee53ff920c8cd4c226d8520a0d4846d8864726b6)
    #1 0x5620b8b8fc86 in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:70:17
    #2 0x7f45b32b90b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
```

```
protocols/kerberos.c:79:52: runtime error: left shift of 255 by 24 places cannot be represented in type 'int'
```

Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=46670
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=46636
2022-04-13 18:10:56 +02:00
Luca Deri e00997ea7c Extended list of cybersecurity domains 2022-04-13 18:05:29 +02:00
Wayneand林文烽 939572c61e fix(ndpi_main):Fix memory leak about ndpi_str; (#1513)
Co-authored-by: 林文烽 <wenfeng.lin@baishan.com>
2022-04-12 15:34:37 +02:00
Ivan Nardi 06a0abb6fd TINC: fix invalid memory read (#1512)
```
=================================================================
==19324==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600061be96 at pc 0x55b4a4cb4460 bp 0x7ffc7b461a70 sp 0x7ffc7b461a68
READ of size 1 at 0x60600061be96 thread T0
    #0 0x55b4a4cb445f in ndpi_check_tinc /home/ivan/svnrepos/nDPI/src/lib/protocols/tinc.c:105:9
    #1 0x55b4a4cb1888 in ndpi_search_tinc /home/ivan/svnrepos/nDPI/src/lib/protocols/tinc.c:135:5
    #2 0x55b4a4b4a6e1 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5013:6
    #3 0x55b4a4b4c2d4 in check_ndpi_tcp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5084:12
    #4 0x55b4a4b4bf77 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5103:12
    #5 0x55b4a4b5dcca in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:5924:15
    #6 0x55b4a4a87734 in packet_processing /home/ivan/svnrepos/nDPI/example/reader_util.c:1519:31
    #7 0x55b4a4a80761 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2093:10
    #8 0x55b4a4a39c8d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #9 0x55b4a4a3a46b in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17
    #10 0x7f69c63760b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
    #11 0x55b4a497954d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x61654d) (BuildId: 705ebc5c412d267294a65cb01f03a1f012aeaf20)

0x60600061be96 is located 0 bytes to the right of 54-byte region [0x60600061be60,0x60600061be96)
allocated by thread T0 here:
[...]
```
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=46499
2022-04-10 16:59:34 +02:00
Toni e3faecf9de Improved ASN.1 parsing for Keberos. Fixes #1492. (#1497)
* This is a quick fix, the Kerberos protocol dissector requires some refactoring effort.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
Signed-off-by: lns <matzeton@googlemail.com>
2022-04-10 10:36:10 +02:00
Vinicius Silva Nogueira 713cdd0b13 QUIC: handle retransmissions and overlapping fragments in reassembler (#1195) (#1498)
* QUIC: handle retransmissions and overlapping fragments in reassembler

* Trigger CI

* minor fix: parentheses

* Changing ndpi_malloc to ndpi_calloc

* fix memory leak

* quic_reasm_buf calloc to malloc

* change order of is_ch_complete && is_reasm_buf_complete call

* is_reasm_buf_complete: added handling for case where frame size is not multiple of 8

* add extra check
2022-04-07 21:18:58 +02:00
aouinizied 9537788ade Fix JSON-C. 2022-04-07 17:59:11 +02:00
aouinizied ff993e2f57 Python bindings fix. 2022-04-07 17:54:41 +02:00
Luca Deri a6e2f4a15a Added ndpi_find_outliers() API call using Z-Score 2022-04-04 10:02:45 +02:00
Luca Deri 19019383f0 Added -z flag 2022-04-03 11:45:23 +02:00
Ivan Nardi e0e231756c ndpiReader: fix compilation (#1510)
Not sure why Windows started complaining... anyway, the fixes has been
taken from https://github.com/ntop/nDPI/pull/1491: credits to @lnslbrty
2022-04-01 22:01:39 +02:00
Luca Deri 8080b3d14f Removed un-necessary guess in mining 2022-04-01 19:23:13 +02:00
Luca Deri 9f3a6f131b update 2022-04-01 19:07:46 +02:00
Luca Deri 32a7d42469 Fixed incompatibilities due to https://github.com/ntop/nDPI/pull/1509 2022-04-01 19:07:00 +02:00
Luca Deri 6eb5bd88a7 DGA improvements 2022-04-01 17:39:29 +02:00
Luca Deri a75d77b536 Waring fixes 2022-04-01 17:35:25 +02:00
Vitaliy Ivanov 61bc9815d5 ndpireader: add json output back. (#1509)
- partial revert of:
    commit 51cfdfb0d8
    Author: Luca Deri <deri@ntop.org>
    Date: Sun Nov 17 17:51:45 2019 +0100
      Removed unused JSON-C code

- Json option is changed from 'j' to 'k' as it's used in the new codebase.
- use HAVE_LIBJSON_C instead of HAVE_JSON_C.
- tabs vs spaces clean ups.

Signed-off-by: Vitaliy Ivanov <vitaliyi@interfacemasters.com>

 Conflicts:
	example/ndpiReader.c
2022-04-01 11:47:01 +02:00
Luca Deri 8238e68275 Improvements for CUSTOM_NDPI_PROTOCOLS 2022-04-01 09:02:05 +02:00
Luca Deri 1e8cdd893c Moved geneated file to a separate folder 2022-04-01 08:58:53 +02:00
Luca Deri 0ff3e77c42 Improved twitter detection 2022-04-01 08:50:56 +02:00
Luca Deri 6677d692c4 Removed SRV record from suspicious DNS traffic 2022-03-31 11:32:30 +02:00
Luca Deri 98a548c642 Improved DGA detection 2022-03-31 11:29:30 +02:00
Darryl Sokoloski 46dc8b5bda [autoconf] Fixed .git submodule detection test. (#1507)
Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2022-03-31 03:21:51 +02:00
Luca Deri 4e199abd39 Added code for identifiying anomalies with metrics stored in InfluxDB 2022-03-30 22:12:39 +02:00
Ivan Nardi 6bcba118d4 reader_util: add support for userAgent in SSDP (#1502)
Update unit tests results
Follow-up of d668ab4b
2022-03-28 04:25:34 +02:00
Ivan Nardi 6acf7a6abe Add support for Pluralsight site (#1503) 2022-03-27 15:13:12 +02:00
Ivan Nardi ec80f1fb3b Fix CI tests results (#1504)
CI integration is failing since 856d7d2.
2022-03-27 15:12:39 +02:00
Vitaly Lavrov 4896dabb6b Reducing the size of the ndpi_detection_module_struct structure. (#1490)
The ndpi_detection_module_struct structure contains 5 arrays "struct
ndpi_call_function_struct" size 286*144=41 kB size,
which are occupied by a small number of elements.

At the moment we have callback_buffer_size = 172, tcp_with_payload=114,
tcp_no_payload=8, udp=93, other 8.
NDPI_MAX_SUPPORTED_PROTOCOLS = 285.

Size of struct ndpi_detection_module_struct is 253136 bytes.
Size of all structs ndpi_call_function_struct 5*286*144=205920 bytes.
Real use memory size for struct ndpi_call_function_struct is
(173+224)*144=57168 bytes.
2022-03-27 12:00:21 +02:00
Darryl Sokoloski d668ab4bec [SSDP] Extract HTTP user-agent when available. (#1500)
[SSDP] Added capture file with UA header.
[SSDP] Added pcap test output log file.

Signed-off-by: Darryl Sokoloski <darryl@sokoloski.ca>
2022-03-27 11:10:44 +02:00
Luca Deri 856d7d2916 Improved DGA detection skipping names containign at least 3 consecutive digits in the first word 2022-03-26 09:59:55 +01:00
Ivan Nardi aeb5f1f947 QUIC: add support for version 2 draft 01 (#1493)
Support for v2-00 has been removed (it has never been used in real
networks and it is incompatible with v2-01).

Chrome already supports v2-01 in latest versions in Chrome Beta channel.
2022-03-25 10:16:30 +01:00
Ivan Nardi af1d20bca1 Mining: cleanup registration (#1496)
Use the same pattern of all the other dissectors: one registration and
one callback.
Spotted by @dsokoloski
2022-03-25 10:07:47 +01:00
Vinicius Silva Nogueira 26df1403e6 Trying to improve QUIC reassembler (#1195) (#1489)
* handling QUIC out-of-order fragments

* minor fix

* updated quic_frags_ch_out_of_order_same_packet_craziness.pcapng.out

* quic test: buf_len + last_pos

* QUIC: comment update in __reassemble function and minor change is_ch_complete function
2022-03-24 10:51:31 +01:00
aouinizied bb12837ca7 Update Python bindings guide. 2022-03-22 15:01:55 +01:00
aouinizied c7059db84a Fix typo. 2022-03-22 14:58:22 +01:00
aouinizied 1207ea118a Add HOWTO Python. 2022-03-22 14:56:44 +01:00
aouinizied c47d710d8e Fix python bindings CI. 2022-03-22 13:40:31 +01:00
aouinizied beef4f997b Complete rework of nDPI Python bindings (cffi API, automatic generation, packaging and CI integration) 2022-03-22 13:19:27 +01:00
Luca Deri 93f723d50f Extended the list of cybersecurity protocol 2022-03-21 15:12:50 +01:00
Vitaly Lavrov 8dd70b7023 Bug fixing. (#1487)
Using the protocol_id instead of its index.
2022-03-15 19:40:18 +01:00
Ivan Nardi 9e22c88e0a QUIC: convert logs to standard mechanism (#1485) 2022-03-15 16:55:47 +01:00
Ivan Nardi b83e9f20e7 QUIC: fix dissection of draft-34 (#1484)
QUIC-34 is probably not used in production, but fixing it is trivial and
it doesn't add any noise to the already complex QUIC code.
2022-03-09 22:37:56 +01:00
Ivan Nardi 7aee856aa0 Extend tests coverage (#1476)
Now there is at least one flow under `tests/pcap` for 249 protocols out
of the 284 ones supported by nDPI.

The 35 protocols without any tests are:

* P2P/sharing protocols: DIRECT_DOWNLOAD_LINK, OPENFT, FASTTRACK,
EDONKEY, SOPCAST, THUNDER, APPLEJUICE, DIRECTCONNECT, STEALTHNET

* games: CSGO, HALFLIFE2, ARMAGETRON, CROSSFIRE, DOFUS, FIESTA,
FLORENSIA, GUILDWARS, MAPLESTORY, WORLD_OF_KUNG_FU

* voip/streaming: VHUA, ICECAST, SHOUTCAST, TVUPLAYER, TRUPHONE

* other: AYIYA, SOAP, TARGUS_GETDATA, RPC, ZMQ, REDIS, VMWARE, NOE,
LOTUS_NOTES, EGP, SAP

Most of these protocols (expecially the P2P and games ones) have been
inherited by OpenDPI and have not been updated since then: even if they
are still used, the detection rules might be outdated.

However code coverage (of `lib/protocols`) only increases from 65.6% to
68.9%.

Improve Citrix, Corba, Fix, Aimini, Megaco, PPStream, SNMP and Some/IP
dissection.
Treat IPP as a HTTP sub protocol.
Fix Cassandra false positives.

Remove `NDPI_PROTOCOL_QQLIVE` and `NDPI_PROTOCOL_REMOTE_SCAN`:
these protocol ids are defined but they are never used.

Remove Collectd support: its code has never been called. If someone is
really interested in this protocol, we can re-add it later, updating the
dissector.

Add decoding of PPI (Per-Packet Information) data link type.
2022-03-09 22:37:35 +01:00
Toni f646a4bce0 Improved ASN/IP update scripts and CI integration. (#1474)
* CI will print a warning if ASN/IP addresses changed.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-09 13:53:04 +01:00
Zied Aouini 74ae315e36 Implement CI on Windows. (#1483)
* Switch fail fast to True.
* Windows CI.
2022-03-09 10:57:21 +01:00
Ivan Nardi 269be6c7ef Some small fixes (#1481)
FTP: if the authentication fails, stop analyzing the flow
WSD: call the initialization routine; the dissector code has never been
triggered
MINING: fix dissection
2022-03-08 00:21:51 +01:00
sharonenochandSharon Enoch f91218360b Extracting the Azure Origin url from the download link (#1480)
* Extracting the Azure Origin url from the download link

* Response code check to address double quotes

Co-authored-by: Sharon Enoch <sharone@amzetta.com>
2022-03-08 00:21:32 +01:00
Vitaly Lavrov a1451935b8 Errors fixed (#1482)
Fixed errors for bigendian platforms in ndpiReader.
All address and port comparisons and hash calculations are done with
endian in mind.
The get_ndpi_flow_info() function searched for an existing flow for the
forward and reverse direction of the packet.
The ndpi_workflow_node_cmp() function looked for a flow regardless of
the packet's direction. This is what led to an error in determining the
direction of transmission of the packet.

Fixed error in "synscan" test: the number of packets in the forward and
reverse direction is incorrectly defined (verified via tcpdump).

Fixed bug with icmp protocol checksum check for big endian platforms.
2022-03-08 00:20:56 +01:00
Ivan Nardi c345b3c7af EthernetIP: fix integer conversion on big-endian archs (#1477) 2022-03-05 15:05:05 +01:00
Vitaly Lavrov 438f04066b Fixed a bug for BE architectures (#1478)
Fixed a bug in the internal implementation of libgcrypt for bigendian architectures
2022-03-05 11:44:21 +01:00
Ivan Nardi a173d3e98f configure: fix usage of libgpg-error with --with-local-libgcrypt (#1472)
Right now, using external libgcrypt, nDPI is not linked to libgpg-error
because configure script never checks for it.

```
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ CC=gcc-11 CXX=g++-11 CFLAGS="-O3 -g -Werror" ./autogen.sh --enable-debug-messages --with-pcre  --with-local-libgcrypt && make -s -j
[...]
checking for numa_available in -lnuma... yes
checking for pcap_open_live in -lpcap... yes
checking for pthread_setaffinity_np in -lpthread... yes
checking for gcry_cipher_checktag in -lgcrypt... yes             <------- missing check for libgpg-error
checking for pcre_compile in -lpcre... yes
checking that generated files are newer than configure... done
[...]
ivan@ivan-Latitude-E6540:~/svnrepos/nDPI(dev)$ grep HAVE_LIBGPG_ERROR src/include/ndpi_config.h
/* #undef HAVE_LIBGPG_ERROR */

```

Make both libgcrypt and libgpg-error mandatory if
`--with-local-libgcrypt` is used.

Technically speaking, libgpg-error might be optional, because it is used
only for debug messages. However having both libraries mandatory
slightly simplified the logic.
In most environments, libgpg-error is a dependency of libgcrypt anyway,
so having both libraries should be the standard case.
2022-03-05 11:41:37 +01:00
Toni 95a3d4fffe Added autoconf option `--enable-tls-sigs'. (#1471)
* Testing more code in CI environments.
 * Added strict option checking for `./configure' in CI environments.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-03 10:16:08 +01:00
Toni cb62dfd249 Drop support for non-gcrypt builds. (#1469)
* As there is now a builtin, lightweight libgcrypt
   there is no need to disable tls-clho decryption.
 * It is still possible to use a host libgcrypt
   with `--with-local-libgcrypt'.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-02 19:48:46 +01:00
Ivan Nardi e23507d3e9 Internal crypto: increase size of authentication buffer (#1468)
Some QUIC flows are not properly decoded while using internal crypto
code: the authentication buffer is too small.
The new value (like the old one) is arbitrary.

Close #1463
2022-03-02 15:14:57 +01:00
Ivan Nardi 6f431057c5 reader_util: fix parsing of MPLS packets (#1467)
```
==20492==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60300000578c at pc 0x55c47455e3ea bp 0x7ffc62ca1eb0 sp 0x7ffc62ca1ea8
READ of size 4 at 0x60300000578c thread T0
    #0 0x55c47455e3e9 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:1840:16
    #1 0x55c47451b9cd in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #2 0x55c47451c1ab in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17
    #3 0x7f661b50e0b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16
    #4 0x55c47445b54d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x61054d) (BuildId: eba4bb4cd43b7101e4f0028ec0fb79087bae0e37)

0x60300000578d is located 0 bytes to the right of 29-byte region [0x603000005770,0x60300000578d)

```
2022-03-02 15:14:41 +01:00
Toni e8559a4127 Add ICMP checksum check and set risk if mismatch detected. (#1464)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-02 13:12:01 +01:00
Luca Deri 6c4df21238 Typo 2022-03-02 12:47:04 +01:00
Toni 4da1453d90 Added configureable ndpi packet processing limit. (#1466)
* The current behaviour ignores any user preferences
   and was also incorrectly implemented, because the
   flow->num_processed_pkts wraps every 65535 and nDPI
   will process packets again until
   NDPI_MAX_NUM_PKTS_PER_FLOW_TO_DISSECT reached.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-02 12:02:09 +01:00
Toni 736a80f9ce Fix libgcrypt(-light/-internal) compile error. (#1465)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-03-02 11:03:47 +01:00
Ivan Nardi 7a7e4ee69f Add a new flow risk NDPI_ANONYMOUS_SUBSCRIBER (#1462)
The main goal of a DPI engine is usually to determine "what", i.e. which
types of traffic flow on the network.
However the applications using DPI are often interested also in "who",
i.e. which "user/subscriber" generated that traffic.

The association between a flow and a subscriber is usually done via some
kind of DHCP/GTP/RADIUS/NAT mappings. In all these cases the key element
of the flow used to identify the user is the source ip address.

That usually happens for the vast majority of the traffic.

However, depending on the protocols involved and on the position on the net
where the traffic is captured, the source ip address might have been
changed/anonymized. In that case, that address is useless for any
flow-username association.

Example: iCloud Private Relay traffic captured between the exit relay and
the server.
See the picture at page 5 on:
https://www.apple.com/privacy/docs/iCloud_Private_Relay_Overview_Dec2021.PDF

This commit adds new generic flow risk `NDPI_ANONYMOUS_SUBSCRIBER` hinting
that the ip addresses shouldn't be used to identify the user associated
with the flow.
As a first example of this new feature, the entire list of the relay ip
addresses used by Private Relay is added.

A key point to note is that list is NOT used for flow classification
(unlike all the other ip lists present in nDPI) but only for setting this
new flow risk.

TODO: IPv6
2022-02-28 15:25:58 +01:00
Luca Deri 1a8a6aa161 Unless --with-libgcrypt is used, nDPI now uses its internal gcrypt implementation 2022-02-28 15:18:10 +01:00
Ivan Nardi e193f3713d Removed some unused fields (#1461) 2022-02-28 15:03:49 +01:00
Vitaly Lavrov c390085f91 Bug fixing. (#1459)
The '--enable-debug-messages' option works again.
Fixed warning in ahocorasick.c
Fixed integer overflow in ndpiReader.c for 32bit systems.
2022-02-28 15:01:00 +01:00
Toni 00d9a16b6a Added `--enable-code-coverage' build using lcov for coverage generation. (#1430)
* CI job generates a coverage report

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-02-28 14:58:45 +01:00
Ivan Nardi 61a3c2eb5b reader_util: fix TZSP decapsulation (#1460)
```
==38674==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60400008957f at pc 0x5653fcee6434 bp 0x7ffe9b554b50 sp 0x7ffe9b554b48
READ of size 1 at 0x60400008957f thread T0
    #0 0x5653fcee6433 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2050:18
    #1 0x5653fce9fc0d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7
    #2 0x5653fcea03eb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17
    #3 0x7fe71dc3d0b2 in __libc_start_main /build/glibc-YbNSs7/glibc-2.31/csu/../csu/libc-start.c:308:16
    #4 0x5653fcddf67d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x58a67d) (BuildId: 525418a27e8c37d6c492cc3220e0e97809c40f98)

0x60400008957f is located 0 bytes to the right of 47-byte region [0x604000089550,0x60400008957f)
allocated by thread T0 here:
```

Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=45036
2022-02-27 14:18:23 +01:00
Ivan Nardi 8b062295cc Add some scripts to easily update some IPs lists (#1449)
While the lists in a6ff0dd0 and 2f5f445f are somehow provided by the
companies themselves (or by some interested parties), these new lists
are directly extracted from BGP information, via AS prefixes.

*Usually*, these new lists are far more stable than the previous ones.

TODO:
* add some other ASNs (see `src/lib/ndpi_content_match.c.inc`)
* IPv6, as usual :-(
2022-02-25 14:26:26 +01:00
Toni f14dcb8344 Provide some API functions for convenience. (#1456)
* Extended JSON serializsation: risk, risk score, confidence

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-02-25 14:26:04 +01:00
Luca Deri 34e020ac35 Win fixes 2022-02-23 21:00:53 +01:00
Luca Deri 16ed0fd3f8 Replaced strdup with ndpi_strup 2022-02-23 21:37:52 +01:00
Ivan Nardi d506a0e12f Directly drop malformed packets (#1455)
TCP/UDP/ICMP/ICMPv6 packets with invalid L4 header length should be
ignored.
2022-02-22 22:15:46 +01:00
Ivan Nardi 96f8942f75 reader_util: fix parsing of IPv6 extension headers (#1453)
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44942
2022-02-22 19:58:17 +01:00
Ivan Nardi 279779062c reader_util: fix infinite loop in packet dissection (#1454)
Found by oss-fuzz
See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44944
2022-02-22 19:56:41 +01:00
Ivan Nardi fbb9700086 fuzz: purge old sessions (#1451)
At every fuzz iteration (i.e for every trace file):
* keep the same ndpi context (`ndpi_init_detection_module` is very
slow);
* reset the flow table, otherwise it grows indefinitely.

This change should fix the "out-of-memory" errors reported by oss-fuzz.
2022-02-21 20:32:50 +01:00
Ivan Nardi 6c1accd2bd DTLS: fix access to certificate cache (#1450)
```
protocols/tls.c:650:54: runtime error: member access within null pointer of type 'const struct ndpi_tcphdr'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in
protocols/tls.c:650:54: runtime error: load of null pointer of type 'const u_int16_t' (aka 'const unsigned short')
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:650:54 in
AddressSanitizer:DEADLYSIGNAL
=================================================================
==47401==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x55f7a61b661c bp 0x7f38190f91b0 sp 0x7f38190f70e0 T1)
==47401==The signal is caused by a READ memory access.
==47401==Hint: address points to the zero page.
    #0 0x55f7a61b661c in processCertificateElements /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:650:41
    #1 0x55f7a61ac3cc in processCertificate /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:792:7
    #2 0x55f7a61d34e1 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:846:13
```
2022-02-21 20:30:22 +01:00
Ivan Nardi 59993e294e EthernetIP: add missing initialization (#1448)
Fix:1e1cfb89
2022-02-20 10:20:46 +01:00
Ivan Nardi 2f5f445f72 Add support for Google Cloud (#1447)
Differentiate between Google its own apps/services and Google Cloud.
We already do something similar for Amazon vs AWS and Microsoft vs Azure.
2022-02-20 10:19:52 +01:00
Ivan Nardi 75a53ad064 fuzz: make fuzz_ndpi_reader faster (#1446)
Initialize ndpi_workflow_init context only once.

On a quite old notebook, before:
```
$ ./fuzz/fuzz_ndpi_reader -max_total_time="${MAX_TOTAL_TIME:-360}" -print_pcs=1 -workers="${FUZZY_WORKERS:-0}" -jobs="${FUZZY_JOBS:-0}" ./tests/pcap/
[...]
Done 3256 runs in 361 second(s)
```
after:
```
Done 5032771 runs in 361 second(s)   <----------- ~1400X
```
oss-fuzz will be happy!
2022-02-20 10:19:22 +01:00
Vitaly Lavrov f2411958d7 Added lightweight implementation of libgcrypt. (#1444)
Implementation borrowed from the
https://github.com/ARMmbed/mbedtls.git project (v3.1.0)

Speed testing (Xeon(R) CPU E3-1230 V2 @ 3.30GHz):

gcrypt-gnu         Test md   2897 ms enc   2777 ms dec    942 ms
gcrypt-int         Test md   3668 ms enc   1312 ms dec   2836 ms
gcrypt-int-noaesni Test md   3652 ms enc   1916 ms dec   4458 ms

gcrypt-gnu-nonopt  Test md   3763 ms enc   4978 ms dec   3999 ms

gcrypt-gnu-nonopt - libgcrypt compiled without hardware acceleration
  --disable-padlock-support --disable-aesni-support \
  --disable-shaext-support --disable-pclmul-support \
  --disable-sse41-support --disable-drng-support \
  --disable-avx-support --disable-avx2-support \
  --disable-neon-support --disable-arm-crypto-support \
  --disable-ppc-crypto-support
  --disable-amd64-as-feature-detection
2022-02-20 10:16:46 +01:00
Ivan Nardi f28a3b293e Fix compilation and sync unit tests results (#1445)
'strcasestr' is not defined in all enviroments: quicker fix is to use
'ndpi_strncasestr' instead.
2022-02-19 19:18:02 +01:00
Luca Deri a2878af1ee Added newflow risk NDPI_HTTP_CRAWLER_BOT 2022-02-17 17:20:52 +01:00
Luca Deri 8a2a47e62a Silenced
NDPI_SUSPICIOUS_DGA_DOMAIN,
   NDPI_BINARY_APPLICATION_TRANSFER,
   NDPI_HTTP_NUMERIC_IP_HOST,
   NDPI_MALICIOUS_JA3,

for predefined connectivity check and cybersec categories
2022-02-14 23:38:21 +01:00
Luca Deri 92da30f017 Extended cybersecurity protocol dissection 2022-02-14 23:19:21 +01:00
Luca Deri c4113ffd7e Added SNMP error code check 2022-02-14 23:17:51 +01:00
Luca Deri de6905f41b Exteended cybersecurity list 2022-02-10 20:16:48 +01:00
Luca Deri 023618c13a Invalid prototupe fix 2022-02-10 19:38:11 +01:00
Ivan Nardi ada7d32c50 HSRP: fix dissection over IPv6 (#1443)
Handle all message types.
2022-02-10 11:46:32 +01:00
Luca Deri 1431aa3d1c Added cybersecurity category mapping to string 2022-02-10 10:35:34 +01:00
Luca Deri 5ddbbf4dd2 Added cybersecurity protocol and category that groups traffic towards leading cybersecurity companies and CDNs, useful to make destinations that should be marked as trusted in firewalls and security gateways 2022-02-10 09:50:06 +01:00
Ivan Nardi b5eea436ae HSRP: add support for IPv6 (#1440) 2022-02-09 11:47:37 +01:00
Dmytrii Vitman 4cf853548c Added VXLAN dissector (#1439)
* RFC 7348
2022-02-09 11:47:09 +01:00
Ivan Nardi f229068fd3 Fix memory access in ndpi_strnstr() (#1438)
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44482

It should be the same error reported (and only partially fixed) in
79968f32
2022-02-09 11:46:39 +01:00
Ivan NardiandLuca Deri a6ff0dd0e3 Add few scripts to easily update some IPs lists (#1436)
* Add few scripts to easily update some IPs lists

Some IPs lists should be updated frequently: try to easy the process.
The basic idea is taken from d59fefd0 and a8fe74e5 (for Azure
addresses): one specific .c.inc file and one script for each protocol.

Add the possibility to don't load a specific list.

Rename the old NDPI_PROTOCOL_HOTMAIL id to NDPI_PROTOCOL_MS_OUTLOOK,
to identify Hotmail/Outlook/Exchange flows.

TODO: ipv6

Remove the 9 addresses associated to BitTorrent: they have been added in
e2f21116 but it is not clear why all the traffic to/from these ips
should be classified as BitTorrent.

* Added quotes

* Added quotes

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2022-02-09 11:45:48 +01:00
ol-andreyizrailevandAndrey Izrailev 8cc5cb9f76 Increment current/total number of active flows on successful flow insertion (#1434)
Memory allocation or ndpi_tsearch might fail, so the two values should be
incremented only when insertion actually happened.

Co-authored-by: Andrey Izrailev <Andrey.Izrailev@oktetlabs.ru>
2022-02-09 11:43:10 +01:00
Luca Deri 9838ceb7cd Added ndpi_serialize_string_string_len() APi call
Fixed CSV string serialization
2022-02-08 22:17:26 +01:00
Luca Deri be5a29ba9d Added HSRP protocol detection
Removed attic directory now obsolete
2022-02-08 18:04:57 +01:00
Luca Deri 922a7be897 Added check to ignore multicast packets marking the as Skype 2022-02-08 17:23:25 +01:00
Toni d59f0327a5 Improved MDNS/LLMNR detection. (#1437)
* Checking for port 5353/5355 is not enough.
 * Added additional multicast address and header checks.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-02-07 18:05:23 +01:00
Ivan Nardi ff38d4b980 TLS: fix parsing of certificate elements (#1435)
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44280
```
==263603==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x592478 in ndpi_is_printable_string ndpi/src/lib/ndpi_utils.c:2200:9
    #1 0x5b047c in processCertificateElements ndpi/src/lib/protocols/tls.c:400:7
    #2 0x5ac880 in processCertificate ndpi/src/lib/protocols/tls.c:790:7
    #3 0x5c3a32 in processTLSBlock ndpi/src/lib/protocols/tls.c:844:13
    #4 0x5c2c61 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:973:2
    #5 0x5c117d in ndpi_search_tls_wrapper ndpi/src/lib/protocols/tls.c:2367:5
    #6 0x552a50 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4792:6
```
2022-02-07 18:05:12 +01:00
Ivan Nardi 79968f3245 Sync utests (#1433)
* Sync utest results

* Fix read-heap-buffer-overflow error reported by CI

See: https://github.com/ntop/nDPI/runs/5055876515?check_suite_focus=true
2022-02-04 15:57:17 +01:00
Alfredo Cardigliano f5035521e8 Add comment 2022-02-03 18:43:31 +01:00
Luca Deri c40496eaac Updated test results 2022-02-03 13:54:38 +01:00
Luca Deri 7aef27f85e Added NDPI_ERROR_CODE_DETECTED risk 2022-02-03 13:20:54 +01:00
Luca Deri 0dd5ac89f8 Renamed DCERPC to more generic RPC protocol so we can use also for other types of RPCs (not limited to DCE)
Extended HTTP plugin to support RPC
Improved HTTP crear text detection to limit it to Basic and Digest
2022-02-03 13:06:27 +01:00
Luca Deri 6975f265ec Typo 2022-02-03 12:21:51 +01:00
Luca Deri ae09e7fb14 Improved risks description 2022-02-03 12:19:01 +01:00
Luca Deri a96d7eb180 Updated risk documentation 2022-02-03 12:10:18 +01:00
Luca 37ff626e78 Added new IDN/Punycode risk for spotting internationalized domain names 2022-02-03 09:17:54 +01:00
Luca Deri cd3d720ae3 Added missing __sync_fetch_and_add() definition in Windows 2022-02-02 10:36:50 +01:00
Luca 46bd093934 Moved to 4.3 2022-02-01 09:13:55 +01:00
Alfredo Cardigliano 03cb8efad6 Update changelog 2022-01-31 18:38:02 +01:00
Ivan Nardi 5bb5bec477 Remove struct ndpi_id_struct (#1427)
Remove the last uses of `struct ndpi_id_struct`.
That code is not really used and it has not been updated for a very long
time: see #1279 for details.

Correlation among flows is achieved via LRU caches.

This change allows to further reduce memory consumption (see also
91bb77a8).

At nDPI 4.0 (more precisly, at a6b10cf, because memory stats
were wrong until that commit):
```
nDPI Memory statistics:
	nDPI Memory (once):      221.15 KB
	Flow Memory (per flow):  2.94 KB
```
Now:
```
nDPI Memory statistics:
	nDPI Memory (once):      235.27 KB
	Flow Memory (per flow):  688 B        <--------
```
i.e. memory usage per flow has been reduced by 77%.

Close #1279
2022-01-30 19:18:12 +01:00
Ivan Nardi 1fdcddb33d Remove Playstation VUE protocol (#1426)
PS VUE service has been discontinued on January 30, 2020
https://en.wikipedia.org/wiki/PlayStation_Vue
2022-01-30 19:17:53 +01:00
Luca Deri 5f745ee425 Commented old code (see https://github.com/ntop/nDPI/pull/1425) 2022-01-30 19:17:23 +01:00
Ivan Nardi f455421870 Improve protocol stacks (#1425)
We should have two protocols in classification results only when the
"master" protocol allows some sub-protocols.

Classifications like `AmazonAWS`, `TLS/AmazonAWS`, `DNS/AmazonAWS` are
fine. However classifications like `NTP/Apple`, `BitTorrent/Azure`,
`DNScrypt.AmazonAWS` or `NestLogSink.Google` are misleading.
For example, `ndpiReader`shows  `BitTorrent/Azure` flows under `Azure`
statistics; that seems to be wrong or, at least, very misleading.

This is quite important since we have lots of addresses from CDN
operators.

The only drawback of this solution is that right now ICMP traffic is
classified simply as `ICMP`; if we are really interested in ICMP stuff
we can restore the old behaviour later.
2022-01-30 19:15:42 +01:00
Ivan Nardi 513e386959 Extend protocols support (#1422)
Add detection of AccuWeather site/app and Google Classroom.
Improve detection of Azure, Zattoo, Whatsapp, MQTT and LDAP.

Fix some RX false positives.

Fix some "Uncommon TLS ALPN"-risk false positives.

Fix "confidence" value for some Zoom/Torrent classifications.

Minor fix in Lua script for Wireshark extcap.

Update .gitignore file.

Let GitHub correctly detect the language type of *.inc files.

Zattoo example has been provided by @subhajit-cdot in #1148.
2022-01-29 09:19:26 +01:00
Toni 9b8679a320 Fix some race conditions by using atomic operations. (#1420)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-29 09:18:51 +01:00
Ivan Nardi 0c70411b1b Make some protocols more "big-endian" friendly (#1402)
See #1312
2022-01-29 09:18:32 +01:00
Ivan Nardi 86b97ffb73 Sync unit tests results (#1423)
Fix: 7a3aa41a
2022-01-28 17:17:39 +01:00
Luca 7a3aa41afc Updated alert description case 2022-01-28 15:02:42 +01:00
Toni c53c82d482 Fixed heap overflow in nDPI realloc wrapper if new size < old size. (#1421)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-27 15:29:16 +01:00
Ivan Nardi 9dcc4a2334 Kerberos, TLS, example: fix some memory errors (#1419)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43823
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43921
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43925
2022-01-27 10:59:35 +01:00
Toni 011ee3ecbd Fixed wrong ip tuple comparison. #1386 (#1418)
* Added u32 pads to `union ip_tuple` so btree search should now work as expected.
   The bug caused new flow's when the remote answers, resulting in two Flows per direction. Fail.
 * Fixed a race condition during shutdown phase.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-26 15:47:38 +01:00
Luca Deri 6d185c052e Added support for the .goog Google TLD 2022-01-26 15:22:33 +01:00
Toni 2cd0479204 Serializing empty `risk blocks' pollutes the resulting string. (#1417)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2022-01-26 13:36:45 +01:00
Luca Deri 81b33f0d06 Updated test results after the risk NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE has been added 2022-01-26 09:46:46 +01:00
Luca Deri 58a9aff17c Added NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE flow risk
Added ndpi_set_tls_cert_expire_days() API call to modify the number of days for triggering the above alert that by default is set to 30 days
2022-01-26 09:23:23 +01:00
Luca Deri 14e8808f03 Removed pandora.tv from pandora protocol as they are different services that shouldd not be mixed 2022-01-25 19:11:16 +01:00
Luca Deri bbb3b8349f Improved pandora TV detection 2022-01-24 18:28:43 +01:00
Luca Deri b6b4967aa6 Improved Zoom protocol detection 2022-01-23 20:59:36 +01:00
Luca Deri a8fe74e502 Tool for generating automatically the Azure IP list 2022-01-23 18:56:51 +01:00
Alfredo Cardigliano 20b9f01f12 Fix ndpi_serialize_string_int64 2022-01-21 17:35:38 +01:00
Alfredo Cardigliano a0bb0ab0ca Add unit test for ndpi_serialize_string_int64 2022-01-21 13:05:58 +01:00
havsah e16b6a18be Fix Grease values parsing (#1416)
The check for grease was too broad and filtered some valid values.
In particular, the value 257 was skipped because it matched the previous check.
This has been discovered while parsing tests/pcap/443-firefox.pcap
expected ja3:
771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-51-57-47-53-10,0-23-65281-10-11-35-16-5-51-43-13-45-28-21,29-23-24-25-256-257,0
previously generated ja3:
771,4865-4867-4866-49195-49199-52393-52392-49196-49200-49162-49161-49171-49172-51-57-47-53-10,0-23-65281-10-11-35-16-5-51-43-13-45-28-21,29-23-24-25-256,0

Signed-off-by: Patrick Havelange <patrick.havelange_ext@softathome.com>
2022-01-21 12:40:35 +01:00
Luca Deri 6f336f8307 Added JA3 in risj exceptions 2022-01-20 22:22:56 +01:00
Luca Deri d91477b18d Fixed certificate mismatch check 2022-01-19 23:10:10 +01:00
Luca Deri 771b6606c8 Fixed an issue on CentOS 7 introduced by
https://github.com/ntop/nDPI/commit/c2b7d77784beeae5f86f19e33ba3da5ddce55898
2022-01-19 10:19:58 +01:00
Luca Deri 9eee257be0 Updated confidence type 2022-01-18 22:28:58 +01:00
Ivan Nardi 9f050fa0a6 TLS, H323, examples: fix some memory errors (#1414)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=26880
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=26906
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43782
https://oss-fuzz.com/testcase-detail/6334089358082048
2022-01-18 21:52:37 +01:00
Ivan Nardi bd036f96f9 Netbios, CSGO: fix two memory errors (#1413)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43754
https://oss-fuzz.com/testcase-detail/5329842395021312
2022-01-18 19:11:01 +01:00
Sam JamesandLuca Deri c2b7d77784 build: respect environment options more (#1392)
* build: update m4/ax_pthread.m4 from serial 23 -> serial 31

Update ax_pthread.m4 to the latest version from the autoconf-archive
project.

Signed-off-by: Sam James <sam@gentoo.org>

* build: properly detect AR, CC, RANLIB

It's necessary to be able to override choice of AR/CC/RANLIB and other toolchain
variables/tools for cross-compilation, testing with other toolchains, and
to ensure the compiler chosen by the user is actually used for the build.

Previously, GNU_PREFIX was kind-of used for this but this isn't a standard
variable (at all) and it wasn't applied consistently anyway.

We now use the standard autoconf mechanisms for finding these tools.

(RANLIB is already covered by LT_INIT.)

Signed-off-by: Sam James <sam@gentoo.org>

* build: use $(MAKE)

This ensures that parallel make works correctly, as otherwise, a fresh
make job will be started without the jobserver fd, and hence
not know about its parent, forcing -j1.

* build: respect CPPFLAGS, LDFLAGS

- CPPFLAGS is for the C preprocessor (usually for setting defines)
- LDFLAGS should be placed before objects for certain flags to work
  (e.g. -Wl,--as-needed)

Signed-off-by: Sam James <sam@gentoo.org>

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2022-01-18 14:30:14 +01:00
Ivan Nardi e4a7990a4e H323: fix a use-after-poison error (#1412)
Detected by oss-fuzz
See: https://oss-fuzz.com/testcase-detail/6730505580576768

Fix a function prototype
Update a unit test results
2022-01-17 19:47:27 +01:00
Luca Deri d19bd96555 Improved Badoo detection (missing mobile app domain) 2022-01-17 18:54:27 +01:00
Luca Deri 6877e46273 Added Badoo detection 2022-01-17 15:13:14 +01:00
Luca 6679453d86 Adds some risk exceptions for popular services and domain names
via a new (internal) function named ndpi_add_domain_risk_exceptions()
2022-01-17 09:00:25 +01:00
Ivan Nardi 1ff308af09 TLS: fix a use-of-uninitialized-value error (#1411)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43705
2022-01-16 20:31:48 +01:00
Ivan Nardi 05734f3d31 Zattoo: fix Null-dereference READ with ipv6 traffic (#1410)
Fix: 20b5f6d7
Detected by oss-fux:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43700
2022-01-16 19:28:39 +01:00
Ivan Nardi 611c753da5 XBox, Diameter: fix dissectors initialization (#1405)
These dissectors have *never* been triggered because their registration
functions use the wrong parameter/bitmask.
Diameter code is buggy since the origianl commit (1d108234), while
XBox code since 5266c726.

Fix some false positives in Xbox code.
2022-01-16 16:19:00 +01:00
Luca Deri f3af39ee42 Added performance tests tools 2022-01-16 12:47:56 +01:00
Luca Deri 42d74171b2 Minor cosmetic changes 2022-01-16 12:47:56 +01:00
Luca Deri dd325248f1 Reduced Patricia tree bucket memory footprint 2022-01-16 12:47:56 +01:00
Ivan Nardi e46b508a36 Kerberos: fix use-of-uninitialized-value error (#1409)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43677
2022-01-15 20:53:36 +01:00
Ivan Nardi eb5d7b07af TLS: fix heap-buffer-overflow error (#1408)
Detected by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=43664
2022-01-15 18:47:59 +01:00
Ivan Nardi 63c44a24b6 STUN: fix "confidence" value for some classifications (#1407) 2022-01-15 17:35:06 +01:00
Ivan Nardi 20b5f6d7cc Improve IPv6 support, enabling IPv6 traffic on (almost) all dissectors. (#1406)
Follow-up of 7cba34a1
2022-01-15 17:34:56 +01:00
Ivan Nardi 7f69de0b51 Restore a unit test result (#1403)
Deleted, probably by mistake, in 406ac7e8

Fix Makefile and add compilation of `rrdtool` in CI tests
2022-01-13 23:14:28 +01:00
Luca Deri 533b3ab0e3 FreeBSD fixes 2022-01-13 21:15:21 +01:00
Luca Deri 406ac7e8c8 Added the ability to specify trusted issueDN often used in companies to self-signed certificates
This allows to avoid triggering alerts for trusted albeit private certificate issuers.

Extended the example/protos.txt with the new syntax for specifying trusted issueDN.
Example:

trusted_issuer_dn:"CN=813845657003339838, O=Code42, OU=TEST, ST=MN, C=US"
2022-01-13 19:06:21 +01:00
Vinicius Silva Nogueira dc60cd09c4 fix ahocorasick on big-endian machines (#1401) 2022-01-13 17:35:06 +01:00
Luca Deri d59fefd06e Improved MicrosoftAzure detection 2022-01-12 21:49:31 +01:00
Luca Deri 1e1cfb89d2 Added EthernetIP dissector 2022-01-12 21:48:39 +01:00
Ivan Nardi d4da3b6516 Run fuzzing tests also on push event (like the CI tests) (#1400) 2022-01-12 21:04:01 +01:00
Ivan Nardi f30fbc3212 Fix DGA false positive (#1399)
Close #1397
2022-01-12 20:51:29 +01:00
Ivan Nardi b080a1c136 Fix two use-of-uninitialized-value errors (#1398)
Found by oss-fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=40269
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=41432

Fix fuzz compilation (follow-up of f5545a80)
2022-01-12 20:24:57 +01:00
Luca Deri 552d199d2e Removed outdated comment 2022-01-11 21:46:30 +01:00
Luca Deri f5545a80f9 Removed legacy code 2022-01-11 21:45:27 +01:00
Ivan Nardi 06f3b07909 QUIC: fix an integer overflow (#1396)
Reported by oss-fuzz:
```
==685288==ERROR: AddressSanitizer: SEGV on unknown address 0x61a100000687 (pc 0x0000005aba64 bp 0x7ffe3f29f510 sp 0x7ffe3f29f400 T0)
==685288==The signal is caused by a READ memory access.
SCARINESS: 20 (wild-addr-read)
 #0 0x5aba64 in quic_len ndpi/src/lib/protocols/quic.c:203:12
 #1 0x5aba64 in decrypt_initial_packet ndpi/src/lib/protocols/quic.c:993:16
 #2 0x5aba64 in get_clear_payload ndpi/src/lib/protocols/quic.c:1302:21
 #3 0x5aba64 in ndpi_search_quic ndpi/src/lib/protocols/quic.c:1658:19
 #4 0x579f00 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4683:6
 #5 0x57abe6 in ndpi_check_flow_func ndpi/src/lib/ndpi_main.c:0
 #6 0x583b2c in ndpi_detection_process_packet ndpi/src/lib/ndpi_main.c:5545:15
 #7 0x55e75e in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_process_packet.c:30:3
[...]
```
2022-01-11 17:57:34 +01:00
Ivan Nardi 3a087e951d Add a "confidence" field about the reliability of the classification. (#1395)
As a general rule, the higher the confidence value, the higher the
"reliability/precision" of the classification.

In other words, this new field provides an hint about "how" the flow
classification has been obtained.
For example, the application may want to ignore classification "by-port"
(they are not real DPI classifications, after all) or give a second
glance at flows classified via LRU caches (because of false positives).

Setting only one value for the confidence field is a bit tricky: more
work is probably needed in the next future to tweak/fix/improve the logic.
2022-01-11 15:23:39 +01:00
Luca Deri a2916d2e4c Readme for using fuzzer with nDPI 2022-01-11 15:12:45 +01:00
Luca Deri c02574807e Updated result 2022-01-09 18:52:30 +01:00
Luca Deri 708d4ea33a Improved user agent analysis 2022-01-09 18:47:47 +01:00
Ivan Nardi bf69321a29 GTP: fix some false positives (#1394) 2022-01-08 20:40:24 +01:00
Ivan Nardi 4d27001d1d Remove some unused fields (#1393) 2022-01-08 20:40:10 +01:00
Luca Deri eac3ae9ded Improved bittorrent detection 2022-01-06 19:50:31 +01:00
Luca Deri f24f68f59d Invalid check 2022-01-06 07:15:45 +01:00
Luca Deri 8f805a8163 Added missing dependency 2022-01-05 11:11:58 +01:00
Alfredo Cardigliano 948c0367f9 Update issue tempalte 2022-01-03 17:26:23 +01:00
Alfredo Cardigliano 23a4761276 Update copyright 2022-01-03 11:00:45 +01:00
Luca Deri d72a40a0fc Improved BitTorrent guess 2021-12-26 19:41:35 +01:00
Luca Deri c4ac53a03f Added support for Log4J/Log4Shell detection in nDPI via a new flow risk named NDPI_POSSIBLE_EXPLOIT 2021-12-23 21:30:16 +01:00
Ivan Nardi fdb6481cd6 Add support for ICloud Private Relay (#1390)
See:
https://www.apple.com/privacy/docs/iCloud_Private_Relay_Overview_Dec2021.PDF

TODO: an up-to-date list of egress IP ranges is publicly available. Can
we use it somehow?
2021-12-22 19:55:57 +01:00
Ivan Nardi 91bb77a880 A final(?) effort to reduce memory usage per flow (#1389)
Remove some unused fields and re-organize other ones.
In particular:
* Update the parameters of `ndpi_ssl_version2str()` function
* Zattoo, Thunder: these timestamps aren't really used.
* Ftp/mail: these protocols are dissected only over TCP.
* Attention must be paid to TLS.Bittorrent flows to avoid invalid
read/write to `flow->protos.bittorrent.hash` field.

This is the last(?) commit of a long series (see 22241a1d, 227e586e,
730c2360, a8ffcd8b) aiming to reduce library memory consumption.

Before, at nDPI 4.0 (more precisly, at a6b10cf7, because memory stats
were wrong until that commit):
```
nDPI Memory statistics:
	nDPI Memory (once):      221.15 KB
	Flow Memory (per flow):  2.94 KB
```
Now:
```
nDPI Memory statistics:
	nDPI Memory (once):      231.71 KB
	Flow Memory (per flow):  1008 B       <---------
```
i.e. memory usage per flow has been reduced by 66%, dropping below the
psychological threshold of 1 KB.

To further reduce this value, we probably need to look into #1279:
let's fight this battle another day.
2021-12-22 19:54:06 +01:00
Alfredo Cardigliano 02da143e45 Add links to other user guides 2021-12-22 10:23:14 +01:00
Luca Deri 4c02c81bf4 Improved bittorrent heuristic 2021-12-21 23:04:27 +01:00
Ivan Nardi a16b4aa564 Fix a global-buffer-overflow when loading Azure list (#1388) 2021-12-19 22:29:25 +01:00
Luca Deri 7aad97fffb Added Microsoft Azure support 2021-12-19 18:59:45 +01:00
Ivan Nardi 7153b8933c Improve/add several protocols (#1383)
Improve Microsoft, GMail, Likee, Whatsapp, DisneyPlus and Tiktok
detection.
Add Vimeo, Fuze, Alibaba and Firebase Crashlytics detection.

Try to differentiate between Messenger/Signal standard flows (i.e chat)
and their VOIP (video)calls (like we already do for Whatsapp and
Snapchat).

Add a partial list of some ADS/Tracking stuff.

Fix Cassandra, Radius and GTP false positives.
Fix DNS, Syslog and SIP false negatives.

Improve GTP (sub)classification: differentiate among GTP-U, GTP_C and
GTP_PRIME.

Fix 3 LGTM warnings.
2021-12-18 13:24:51 +01:00
Ivan Nardi 58b33dcb2d TLS: add support for IPV6 in Subject Alt Names field (#1385) 2021-12-18 12:03:46 +01:00
Ivan Nardi ec6eea0c23 TLS: fix usage of certificate cache (#1384) 2021-12-18 12:03:32 +01:00
Luca Deri 74519f6e10 Improved bittorrent detection 2021-12-17 21:17:18 +01:00
Ivan Nardi f11b215cfb QUIC: fix old versions of GQUIC on big-endian machines (#1387)
Credits to @viniciussn (see #1312)
2021-12-17 12:10:34 +01:00
Luca Deri 8e512ff503 Improved BitTorrent classification 2021-12-07 21:49:00 +01:00
Luca Deri 519d70fa04 Optimized ndpi_serialize_string_int32 2021-12-07 15:54:23 +01:00
Luca Deri 85b396ca43 Fixed issue that prevented alt certificate names to be fully detected when ipAddress and rfc822Name were specified in certificates 2021-12-07 14:45:42 +01:00
Luca Deri cefcc25b13 Added missing install target 2021-12-07 09:00:58 +01:00
Ivan Nardi 947896ad7d Fix configure script (after fb85dac9) (#1381)
Fix/disable some LGTM warnings
2021-12-04 20:22:05 +01:00
Ivan Nardi 7cba34a1e3 Improve IPv6 support, enabling IPv6 traffic on (almost) all dissectors. (#1380)
There are no valid reasons for a (generic) protocol to ignore IPv6
traffic.
Note that:
* I have not found the specifications of "CheckPoint High Availability
Protocol", so I don't know how/if it supports IPv6
* all LRU caches are still IPv4 only

Even if src_id/dst_id stuff is probably useless (see #1279), the right
way to update the protocol classification is via `ndpi_set_detected_protocol()`
2021-12-04 13:30:10 +01:00
Ivan Nardi 6e86e6d924 QUIC: add support for QUICv2 (draft 00) (#1379)
It is already time to start looking at the new QUIC version.
See: https://datatracker.ietf.org/doc/html/draft-ietf-quic-v2-00
2021-12-04 13:29:30 +01:00
Toni 4d60c70c8b Make serialize risk and proto not dependant on any flow. (#1377)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-12-04 13:28:55 +01:00
Luca Deri fb85dac905 Configure improvements
Fixed invalid anomaly detection in rrd_anomaly
2021-12-04 13:21:57 +01:00
Luca Deri fe2822c6a8 Added example for finding similarities in RRDs using nDPI statistical APIs 2021-12-04 10:09:01 +01:00
Luca Deri 6ab1367846 Added quick mode (-q) 2021-12-03 21:18:03 +01:00
Luca Deri 8c4ae7527b Updated rrdtool makefile 2021-12-03 21:10:43 +01:00
Luca Deri 1b077ba65f Demo tool for finding anomalies in RRD files 2021-12-03 17:36:49 +01:00
Alfredo Cardigliano 3dd9b5176e Remove printf when not required 2021-12-01 10:24:12 +01:00
Alfredo Cardigliano 1d3428d538 Improve JSON serialization 2021-12-01 10:04:17 +01:00
Luca Deri 33557c5fbc Improved CSV serialization 2021-11-30 22:15:26 +01:00
Luca Deri 5e0befd56c Fixed seriealizer unit 2021-11-30 17:24:42 +01:00
Luca Deri 4f81cdc103 Removed spaces from some protocol names as they create problems with some applications 2021-11-30 10:12:24 +01:00
Ivan Nardi b1e9245d94 ndpiReader: slight simplificaton of the output (#1378) 2021-11-27 17:32:23 +01:00
Luca Deri 320bcfafa2 Added ndpi_serializer_skip_header() serialization API 2021-11-26 22:18:59 +01:00
Luca Deri a7b5e09195 Added Salesforce detection 2021-11-26 19:07:45 +01:00
Luca Deri 9d59b8b1a9 HTTP proxy improvement 2021-11-25 23:17:27 +01:00
Luca Deri ea435c46f5 Reworked HTTP protocol dissection including HTTP proxy and HTTP connect 2021-11-25 22:53:46 +01:00
Luca Deri 772957349b Aded default port to HTTP_CONNECT 2021-11-25 21:46:00 +01:00
Luca Deri 40ff2c54c1 Added googletagmanager.com to the host_match list 2021-11-25 21:45:38 +01:00
Ivan Nardi a8ffcd8bb0 Rework how hostname/SNI info is saved (#1330)
Looking at `struct ndpi_flow_struct` the two bigger fields are
`host_server_name[240]` (mainly for HTTP hostnames and DNS domains) and
`protos.tls_quic.client_requested_server_name[256]`
(for TLS/QUIC SNIs).

This commit aims to reduce `struct ndpi_flow_struct` size, according to
two simple observations:
 1) maximum one of these two fields is used for each flow. So it seems safe
to merge them;
 2) even if hostnames/SNIs might be very long, in practice they are rarely
longer than a fews tens of bytes. So, using a (single) large buffer is a
waste of memory for all kinds of flows. If we need to truncate the name,
we keep the *last* characters, easing domain matching.

Analyzing some real traffic, it seems safe to assume that the vast
majority of hostnames/SNIs is shorter than 80 bytes.

Hostnames/SNIs are always converted to lowercase.

Attention was given so as to be sure that unit-tests outputs are not
affected by this change.

Because of a bug, TLS/QUIC SNI were always truncated to 64 bytes (the
*first* 64 ones): as a consequence, there were some "Suspicious DGA
domain name" and "TLS Certificate Mismatch" false positives.
2021-11-24 10:46:48 +01:00
Ivan Nardi fd02e1b304 QUIC: fix extra dissection (#1376)
When we have fully reassembled the Client Hello, we need to stop extra
dissection.
2021-11-23 10:57:13 +01:00
Luca Deri 4c39ed293e BitTorrent dissector code cleanup and merge of https://github.com/ntop/nDPI/pull/1374 2021-11-17 23:18:22 +01:00
Luca Deri 2e2d4c911a BitTorrent detection improvements 2021-11-17 22:41:35 +01:00
Luca Deri 2a6e5c6341 Improved BT hashing to minimize false positives 2021-11-17 10:47:21 +01:00
Luca Deri a6545ea095 bittorrent old code cleanup. Enlarged BT cache 2021-11-16 10:05:18 +01:00
Luca Deri f2a279e441 Improved BitTorrent detection 2021-11-15 17:25:13 +01:00
Ivan Nardi afc2b641eb Fix writes to flow->protos union fields (#1354)
We can write to `flow->protos` only after a proper classification.

This issue has been found in Kerberos, DHCP, HTTP, STUN, IMO, FTP,
SMTP, IMAP and POP code.
There are two kinds of fixes:
 * write to `flow->protos` only if a final protocol has been detected
 * move protocol state out of `flow->protos`
The hard part is to find, for each protocol, the right tradeoff between
memory usage and code complexity.

Handle Kerberos like DNS: if we find a request, we set the protocol
and an extra callback to further parsing the reply.

For all the other protocols, move the state out of `flow->protos`. This
is an issue only for the FTP/MAIL stuff.

Add DHCP Class Identification value to the output of ndpiReader and to
the Jason serialization.

Extend code coverage of fuzz tests.

Close #1343
Close #1342
2021-11-15 16:20:57 +01:00
ragostino da47357762 Update ndpi_main.c for HTTP line parser optimization and cleaning (#1372)
* Update ndpi_main.c

HTTP line parser: optimization and cleaning

* Update ndpi_main.c
2021-11-11 12:38:31 +01:00
Ivan Nardi acb1de69aa Reduce memory used by ndpiReader (#1371)
`ndpiReader` is only an example, aiming to show nDPI capabilities
and integration, without any claim about performances.

Nonetheless its memory usage per flow is *huge*, limiting the kinds
of traces that we can test on a "normal" hardware (example: scan
attacks).

The key reason of that behaviour is that we preallocate all the memory
needed for *all* the available features.

Try to reduce memory usage simply allocating some structures only
when they are really needed. Most significant example: JOY algorithms.

This way we should use a lot less memory in the two most common
user-cases:
 * `ndpiReader` invoked without any particular flag (i.e `ndpiReader -i
$FILENAME_OR_IFACE`)
 * internal unit tests

Before (on x86_64):
```
struct ndpi_flow_info {
[...]
	/* size: 7320, cachelines: 115, members: 72 */
```
After:
```
struct ndpi_flow_info {
[...]
	/* size: 2128, cachelines: 34, members: 75 */
```
2021-11-11 12:37:25 +01:00
Ivan Nardi 3e5491fa10 Add detection of OCSP (#1370)
This protocol is detected via HTTP Content-Type header.

Until 89d548f9, nDPI had a dedicated automa (`content_automa`) to
classify a HTTP flow according to this header. Since then, this automa has
been useless because it is always empty.
Re-enable it to match only a string seems overkilling.

Remove all `content_automa` leftovers.
2021-11-11 12:36:55 +01:00
Ivan Nardi 0f168d9150 IMAP, POP3, SMTP: improve dissection (#1368)
Avoid NATS false positives
2021-11-11 11:55:56 +01:00
Luca Deri 9abf528c8e BitTorrent cache debug 2021-11-08 12:42:15 +01:00
Ivan Nardi 5464bad6db Differentiate between standard Amazon stuff (i.e market) and AWS (#1369) 2021-11-04 00:20:45 +01:00
Luca Deri 4173775b60 Improved BitTorrent detection 2021-11-04 00:20:26 +01:00
Ivan Nardi a8a3c6f7c2 Fix use-of-uninitialized-value (#1366)
Close #1346
2021-11-03 15:40:07 +01:00
Ivan Nardi 55880e4ae4 TLS: fix two warnings (#1365)
Disable unit tests on CI for big-endian target. We know we have multiple
issues on big-endian architectures (see #1312) and so the unit tests
always fail there. Ignore this error for the time being and let the CI
pass if we don't have other issues.

Remove an unused automa definition
2021-11-02 22:08:15 +01:00
Luca Deri 1f6c94e663 Updated test results 2021-11-02 12:08:20 +01:00
Luca Deri b11065b2e3 Fixed certificate mismatch for TLS flows with no client hello observed 2021-11-02 12:06:18 +01:00
Luca Deri 9f8c1e8630 BitTorrent detection improvements 2021-10-31 18:45:13 +01:00
Luca Deri a9aa46287d Little change in TLS debug traces 2021-10-28 10:16:33 +02:00
Luca Deri a25d9d871e Updated results 2021-10-27 23:04:29 +02:00
Luca Deri 5283755afa Fixed TLS certificate dissection of subjectAltName 2021-10-27 22:51:59 +02:00
Luca Deri aaa474a37d Minor code cleanup 2021-10-27 09:32:11 +02:00
Ivan Nardi 6edb7bedd7 Avoid overwriting valid protocol in ndpi_detection_giveup (#1360)
We should avoid updating any valid protocol in `ndpi_detection_giveup`; we
should try to find a proper classification only if the flow is still
completely unclassified.

For example in the attached pcap there is a valid TLS session, recognized
as such by TLS dissector. However, the `ndpi_detection_giveup`function
updates it to "HTTP/TLS" (!?) simply because the server port is 80.

Note that the real issue is not the wrong classification, but the
wrong access to `flow->protos` union. If we already set some fields of
`flow->protos` and we change the protocol in `ndpi_detection_giveup`, we
might end up freeing some invalid pointers in `ndpi_free_flow_data`
(no wonder this issue has been found while fuzzing #1354)

Fix GIT and TLS dissectors (issues found by CI fuzzer)
2021-10-27 09:23:07 +02:00
Toni c4eebaaf5e Serialize additional information stored in the flow struct. (#1362)
* Changed function signature of ndpi_flow2json (removed unused vlan_id; API break)
 * Serialize NTP information.
 * Improved QUIC serialization.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-27 09:22:19 +02:00
Luca Deri f41d8ac106 Improved STUN and RTP detection 2021-10-27 00:06:05 +02:00
Toni 41765efcf8 Detect invalid characters in text and set a risk. Fixes #1347. (#1363)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-26 21:34:01 +02:00
Luca Deri 5ccc61d1cb Fixed Git protocol dissection (#1355) 2021-10-25 18:52:27 +02:00
Ivan Nardi b6d9536533 Fixed cleartext protocol assignment (#1357) 2021-10-25 15:04:04 +02:00
Ivan Nardi 333a6d60e8 TLS: fix a heap-buffer-overflow (#1356)
Revert of c3d1c697
Error reproducible with the attached pcap and valgrind
2021-10-22 14:57:49 +02:00
Luca Deri c3d1c69722 Fixed heap-buffer-overflow in TLS dissector 2021-10-20 22:07:47 +02:00
Ivan Nardi 8db8c8ae34 Fix QUIC log and remove SoulSeek leftovers after b97dc6ba (#1351)
Update .gitignore file
Fix a function prototype
Close #1349
2021-10-19 21:42:55 +02:00
Ivan Nardi fea3991ad1 Fix some invalid memory reads (#1350)
`ndpi_detection_giveup()` (and any functions called by it) can't access
`ndpi_detection_module_struct->packet` anymore since 730c236.

Sync unit tests results

Close #1348
2021-10-19 12:07:47 +02:00
Luca Deri 06f13fd2de Added clertext alert with telnet 2021-10-19 08:44:29 +02:00
Toni ed51987e3a Fix broken fuzz_process_packet fuzzer by adding a call to ndpi_finalize_initialization(). (#1334)
* fixed several memory errors (heap-overflow, unitialized memory, etc)
 * ability to build fuzz_process_packet with a main()
   allowing to replay crash data generated with fuzz_process_packet
   by LLVMs libfuzzer
 * temporarily disable fuzzing if `tests/do.sh`
   executed with env FUZZY_TESTING_ENABLED=1

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-18 23:16:32 +02:00
Luca Deri 7d3c3b23f8 Implemented RDP over UDP dissection 2021-10-18 22:12:28 +02:00
Luca Deri 9e97d20c25 Refreshed results list 2021-10-16 12:03:16 +02:00
Luca Deri b08b6a6a28 Fixed cleartext protocol assignment 2021-10-16 12:02:04 +02:00
Luca Deri c2e7fbce94 Updated test results after latest commit 2021-10-16 11:57:32 +02:00
Luca Deri 1b3c41a752 Reworked flow risks asignment
Added esceptions for windows update and binary application transfer risk
2021-10-16 11:55:08 +02:00
Luca Deri b97dc6baa4 Removed outdated (and broken) soulseek dissector 2021-10-15 17:27:27 +02:00
Luca Deri e696f31487 Updated test results 2021-10-14 22:18:34 +02:00
Luca Deri b1a43cbcff Added missing call to ndpi_reconcile_protocols() with protocol guess 2021-10-14 22:08:20 +02:00
Luca Deri 6e649e0cb6 Fixed issue on DGA numeric IP detection 2021-10-13 19:21:27 +02:00
Luca Deri 09668496c3 Indemt 2021-10-13 19:18:05 +02:00
Zied Aouini a2e9730b23 Add issue templates. (#1340)
* Add issue template.
* Add config log instruction.
2021-10-13 18:39:10 +02:00
Luca Deri 937357e4bc Implemented ndpi_ses_fitting() and ndpi_des_fitting()
for comuting the best alpha/beta values for exponential smoothing
2021-10-12 13:08:58 +02:00
Alfredo Cardigliano d947437233 Return 0 on ndpi_init_serializer_ll success for consistency 2021-10-12 10:47:44 +00:00
Luca Deri d13bc0e01a Fixed DES initialisation 2021-10-12 12:38:40 +02:00
Ivan Nardi 550e6fe6fc QUIC: fix an integer overflow (#1337)
Long standing bug: credits to @lnslbrty for digging into it and to
@aouinizied for the CI improvements
2021-10-11 23:08:10 +02:00
Toni a8f938f7ac Removed README.protocols because: (#1333)
* Tor via TLS should be detectable via DGA as a risk
 * protocol limitations should be part of the official documentation in `doc/`

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-11 23:07:36 +02:00
Luca Deri 960e8ab22a Win fix 2021-10-11 20:59:28 +02:00
Zied Aouini ca827ae548 Fix FuzzCI. (#1338) 2021-10-11 20:17:12 +02:00
Zied Aouini 7e0724d576 Enable sanitizers on CIFuzz. (#1336) 2021-10-11 19:01:50 +02:00
Luca Deri 246d115d7b Cleaned up code moving specific includes in files their are using it. Thi prevents ndpi_config.h to be included everywhere in apps using nDPI that might leade to #define redefinitions after the latest changes 2021-10-11 11:45:45 +02:00
Toni 60eca79fdf Additional fix related to cf931fda6b. (#1332)
* configure.seed references removed

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-10 14:14:02 +02:00
Toni 549134ef10 Fix make dist and add it to the CI along with make -C doc text. Fixes #1324 (#1327)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-10 13:15:38 +02:00
Toni cf931fda6b Get rid of configure.seed as it comes with some disadvantages. (#1328)
* using Autotools best-practices to achieve (hopefully) the same result

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-10 13:13:52 +02:00
Toni 6ff67b5043 Add more WindowsUpdate URLs as requested in #698. (#1329)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-08 18:29:49 +02:00
Luca Deri 3ffb65a81c Added extra check 2021-10-07 08:21:40 +02:00
Ivan Nardi 2dfc478ad4 Fix compilation with clang-13 or if some debug macros are enabled (#1326) 2021-10-06 18:32:15 +02:00
Luca Deri 944e8ed105 Added check 2021-10-06 18:28:09 +02:00
Luca Deri 2b4821a97b Compilation fix for old Linux distributions 2021-10-06 09:28:04 +02:00
Luca Deri d9c72999ee Test update 2021-10-06 00:06:34 +02:00
Luca Deri 181a03c5ad Compilation fix 2021-10-05 17:17:12 +02:00
Luca Deri 8957c52a48 Numeric IPs are not considered for DGA checks 2021-10-05 17:14:31 +02:00
Luca Deri 408d78e628 Improved DGA detection for skipping potential DGAs of known/popular domain names 2021-10-05 16:51:24 +02:00
Luca Deri 10576eab0a TLS obsolete protocol is set when TLS < 1.2 (used to be 1.1) 2021-10-05 16:44:56 +02:00
Ivan Nardi 730c2360bd Remove struct ndpi_packet_struct from struct ndpi_flow_struct (#1319)
There are no real reasons to embed `struct ndpi_packet_struct` (i.e. "packet")
in `struct ndpi_flow_struct` (i.e. "flow"). In other words, we can avoid
saving dissection information of "current packet" into the "flow" state,
i.e. in the flow management table.

The nDPI detection module processes only one packet at the time, so it is
safe to save packet dissection information in `struct ndpi_detection_module_struct`,
reusing always the same "packet" instance and saving a huge amount of memory.
Bottom line: we need only one copy of "packet" (for detection module),
not one for each "flow".

It is not clear how/why "packet" ended up in "flow" in the first place.
It has been there since the beginning of the GIT history, but in the original
OpenDPI code `struct ipoque_packet_struct` was embedded in
`struct ipoque_detection_module_struct`, i.e. there was the same exact
situation this commit wants to achieve.

Most of the changes in this PR are some boilerplate to update something
like "flow->packet" into something like "module->packet" throughout the code.
Some attention has been paid to update `ndpi_init_packet()` since we need
to reset some "packet" fields before starting to process another packet.

There has been one important change, though, in ndpi_detection_giveup().
Nothing changed for the applications/users, but this function can't access
"packet" anymore.
The reason is that this function can be called "asynchronously" with respect
to the data processing, i.e in context where there is no valid notion of
"current packet"; for example ndpiReader calls it after having processed all
the traffic, iterating the entire session table.

Mining LRU stuff seems a bit odd (even before this patch): probably we need
to rethink it, as a follow-up.
2021-10-05 15:49:52 +02:00
Ivan Nardi f3fcf1e7c0 Update unit tests results after da8eed5a (#1323) 2021-10-05 15:06:34 +02:00
Luca Deri da8eed5a32 Updated description 2021-10-05 11:08:16 +02:00
Ivan Nardi c1e794366f WHOIS: enhance detection, avoiding false positives (#1320)
We are interested only in the domain name required, not in the long reply.
2021-10-05 09:35:04 +02:00
Luca Deri bb7aff6526 Added -a <num> to ndpiReader for generating OPNsense configuration
See https://github.com/ntop/opnsense
2021-10-04 22:34:49 +02:00
Luca Deri fd0e65cb57 Removed trace 2021-10-03 23:41:31 +02:00
Ivan Nardi bc5be1e76c Fix how some protocols handle tcp retransmissions (#1321)
Most (all?) protocols don't care about (tcp) retransmissions.

If a protocol registers itself with a
NDPI_SELECTION_BITMASK_PROTOCOL_*_WITHOUT_RETRANSMISSION value, its
callback is never triggered with a retransmitted packet.
2021-10-03 21:38:49 +02:00
Toni e84da386c7 Added nDPI pkg-config file to Debian / Ubuntu ndpi-dev packaging. (#1318)
* Added nDPI pkg-config file to Debian / Ubuntu ndpi-dev packaging.

 * fixed missing gcrypt library dependency in libndpi.pc

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-10-03 21:29:56 +02:00
Simone Mainardi 06f763e6c3 Adds sections labels with risk id to the docs 2021-10-01 15:02:14 +02:00
Luca Deri a9d4f5d8d0 Warning fix 2021-10-01 11:36:35 +02:00
Luca Deri 8e6de00ce3 Initial attempt to write nDPI documentation. Starting with flow risks. Please contribute 2021-10-01 11:32:27 +02:00
Ivan Nardi 5cf3fef8f7 Remove detected_protocol_stack field from ndpi_packet_struct (#1317)
This field is an exact copy of `ndpi_flow_struct->detected_protocol_stack[2]`:
* at the very beginning of packet dissection, the value saved in
`flow->detected_protocol_stack` is copied in `packet->detected_protocol_stack`
(via `ndpi_detection_process_packet()` -> `ndpi_init_packet_header()`)

* every time we update `flow->detected_protocol_stack` we update
`packet->detected_protocol_stack` too (via `ndpi_int_change_protocol()`
-> `ndpi_int_change_packet_protocol()`)

These two fields are always in sync: keeping the same value in two
different places is useless.
2021-09-29 13:11:32 +02:00
Alfredo Cardigliano 721031210b Fix warning 2021-09-28 09:44:31 +02:00
Luca Deri af35d6b963 Added unit test for bitmap iteration 2021-09-27 12:47:05 +02:00
Luca Deri b9ccff0761 Compilation fixed on CentOS 7
Bitmap APi changes
2021-09-27 12:41:40 +02:00
Luca Deri 241ee32cbc Bitmap code 2021-09-27 12:06:46 +02:00
Luca Deri 519c69c5cc Reworked bitmap code 2021-09-27 10:29:03 +02:00
Luca Deri 72df138a7d Warnign fix 2021-09-27 09:08:04 +02:00
eac7024489 Fix armhf (#1315)
* Fix unaligned memory accesses with get_u_int64_t at armhf

see: https://bugs.debian.org/993627

* Use get_u_int64_t to avoid unaligned memory access at armhf

see: https://bugs.debian.org/993627

* Update src/include/ndpi_define.h.in

Drop const type from get_u_int64_t, from lnslbrty

Co-authored-by: Bernhard Übelacker <bernhardu@mailbox.org>
Co-authored-by: Toni <matzeton@googlemail.com>
2021-09-26 23:53:00 +02:00
Luca Deri 5ed1930a93 Added include for defining bools (not present on all platforms) 2021-09-26 23:00:13 +02:00
Luca Deri 1efabef4cf Added API for handling compressed bitmaps
ndpi_bitmap* ndpi_bitmap_alloc();
void ndpi_bitmap_free(ndpi_bitmap* b);
u_int64_t ndpi_bitmap_cardinality(ndpi_bitmap* b);
void ndpi_bitmap_set(ndpi_bitmap* b, u_int32_t value);
void ndpi_bitmap_unset(ndpi_bitmap* b, u_int32_t value);
bool ndpi_bitmap_isset(ndpi_bitmap* b, u_int32_t value);
void ndpi_bitmap_clear(ndpi_bitmap* b);
size_t ndpi_bitmap_serialize(ndpi_bitmap* b, char **buf);
ndpi_bitmap* ndpi_bitmap_deserialize(char *buf);

based on https://github.com/RoaringBitmap/CRoaring
2021-09-26 22:55:15 +02:00
Zied Aouini c40cf4e0ec Add arm64, armhf and s390x as part of CI. (#1314)
* Add arm64, armhf and s390x as part on CI.

* Minor fix.

* Fix code inspector complaints.
2021-09-23 14:41:56 +02:00
Ivan Nardi 7eb73bc92a QUIC: fix old GQUIC versions on big-endian machines (#1313) 2021-09-23 13:24:49 +02:00
Zied Aouini 0994771974 Improve CI (#1303)
* Improve CI pipeline

* Fix branch name.

* Fix branch name.

* Fix libgcrypt configuration.

* Update build.yml

* Move to Github Actions instead of Travis CI.

* Fix mingw on ubuntu bionic.

* Reactivate cross compile on Ubuntu Bionic.

* Switch to single line steps.

* Add several compilers versions

* Minor fix.

* Fix build all and delete cxx

* Fix RCE detection.

* Fix PCRE configuration.

* Add condition on PCRE test pcap.

* Update WebattackRCE.pcap.out

* Add missing SUBST.

* Delete WebattackRCE.pcap.out

* Update WebAttackRCE result.

* Fix typo.

* Extend jobs with pcre+msan+maxminddb.

* Fix code inpector warnings.

* Delete .appveyor.yml
2021-09-22 12:10:52 +02:00
Toni 9717afb2da Merge pull request #1311 from IvanNardi/ftp-start-tls
FTP: fix support for START-TLS sessions
2021-09-22 12:08:04 +02:00
Nardi Ivan 017c1a4239 FTP: fix support for START-TLS sessions
When TLS-over-FTP is used, the credentials are encrypted. So we must not
wait for the username and the password commands, otherwise we elaborate a
lot of packets for nothing.
2021-09-21 20:53:04 +02:00
Toni bb52f1362c Merge pull request #1309 from IvanNardi/ipv6-headers
Fix parsing of ipv6 packets with extension headers
2021-09-20 12:54:01 +02:00
Toni d74c2739d5 Merge pull request #1310 from IvanNardi/stun-realm
STUN: fix extraction of Realm attribute
2021-09-20 11:53:40 +02:00
Nardi Ivan 95ac8fd41a STUN: fix extraction of Realm attribute
While at it, improve detection of Facebook Messenger
2021-09-20 11:04:54 +02:00
Nardi Ivan 03d3e1bafc Fix parsing of ipv6 packets with extension headers
Decoding of ipv6 traffic with extension headers was completely broken,
since the beginning of the L4 header was always set to a wrong value.

Handle the ipv6 fragments in the same way as the ipv4 ones: keep the first
one and drop the others.
2021-09-19 17:29:22 +02:00
Ivan Nardi 994bd0696b Small fixes after latest commits (#1308) 2021-09-18 09:31:00 +02:00
Ivan Nardi 227e586e63 A (tiny) effort to reduce memory usage per flow (#1304)
Follow-up of 22241a1d
Only trivial changes:
* remove completely unused fields
* remove fields only written (but never read)
* CSGO protocol only handles UDP traffic
2021-09-18 09:29:45 +02:00
Luca Deri 4539e96ce2 Added DNS fragmented test pcap 2021-09-17 15:56:18 +02:00
Luca Deri 3303dac7cf Improved fragmented DNS detection 2021-09-17 15:17:39 +02:00
Luca Deri 119a14c6e3 DNS dissection fixes 2021-09-17 12:20:19 +02:00
deboracerretiniandLuca Deri 8f113c1d37 Progetto esame Gestione di Reti - Debora Cerretini (#1290)
* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2021-09-17 11:01:50 +02:00
Ivan Nardi 6325aebda6 TLS: avoid zeroing large structures (#1300)
Zeroing large structures (i.e. size > KB) is quite costly (from a CPU point
of view): we can safely avoid doing that for a couple of big structures.

Standard and Valgrind tests have been diverging quite a lot: it is time
to re-sync them. Use the same script and enable Valgrind via an
enviroment variable:

  NDPI_TESTS_VALGRIND=1 ./tests/do.sh
2021-09-16 14:34:59 +02:00
Simone Mainardi 978c9cfda3 Adds instructions to update flow risks in ntopng 2021-09-15 17:49:47 +02:00
Ivan Nardi 0eb7a0388c Hangout: fix usage of STUN function (#1299) 2021-09-13 18:25:43 +02:00
Ivan Nardi 3e06e6f5b8 Fix some ndpi_malloc/ndpi_free pairs (#1298) 2021-09-13 18:25:22 +02:00
Ivan Nardi 31404f4865 Update the list of default ports for QUIC protocol (#1297)
There are no reasons to register UDP/80 as a default port for QUIC
2021-09-13 18:25:04 +02:00
Ivan Nardi 17774bf09c Bittorrent: fix a potential use-of-uninitialized-value error (#1296)
Not sure if this is the right fix (from a logical point-of-view): this
code hasn't changed since OpenDPI era (!) and I have't found a trace
triggering this code path.
Anyway, the use-of-uninitialized-value error itself should be fixed.
2021-09-13 18:24:42 +02:00
Vitaly Lavrov 22241a1d36 Removing unused fields from the id_struct structure. (#1293)
Reduced by 40 bytes.
2021-09-13 18:19:09 +02:00
Ivan Nardi 951bcb0c39 Cassandra: fix compilation when "--enable-debug-messages" option is used (#1294)
Let's try adding a dedicated compilation in GitHub Actions to easily
detect this kind of errors in the future
2021-09-11 11:10:46 +02:00
Ivan Nardi 083e464929 Fix unit test results after 00857abf (#1295) 2021-09-11 11:09:55 +02:00
Luca Deri 00857abf2c Added new risk for clear text credentials 2021-09-10 22:00:04 +02:00
lucasbaileandLucas Santos 1fadf4754a Add Cassandra protocol dissector (#1285)
Co-authored-by: Lucas Santos <lucas.santos@zerum.com>
2021-09-09 22:47:58 +02:00
Luca Deri 36f08b8d2e Win fix 2021-09-07 22:37:22 +02:00
Alfredo Cardigliano 57fce4e350 Fix unused var 2021-09-03 17:46:04 +02:00
Alfredo Cardigliano 89af9b7e3c Print API version 2021-09-01 09:48:37 +02:00
Luca Deri a6b10cf73f Fixed memory stats 2021-08-26 12:12:22 +02:00
Luca Deri 90c6fd99a7 ARM fixes 2021-08-25 09:46:29 +02:00
Luca Deri 88040f0449 Compilation fix 2021-08-20 22:47:46 +02:00
Ivan Nardi 9e0f0ce3df Fix access to some TLS fields in flow structure (#1277)
Fields 'tls.hello_processed` and `tls.subprotocol_detected` are used by
QUIC (i.e UDP...), too.
2021-08-20 18:11:37 +02:00
Ivan Nardi 8fdffbf3a1 Compile everything with "-W -Wall -Wno-unused-parameter" flags (#1276)
Fix all the warnings.

Getting rid of "-Wno-unused-parameter" is quite complex because some
parameters usage depends on compilation variable (i.e.
`--enable-debug-messages`).

The "-Werror" flag has been added only in Travis builds to avoid
breaking the builds to users using uncommon/untested
OS/compiler/enviroment.

Tested on:
* x86_64; Ubuntu 20.04; gcc 7,8,9,10,11; clang 7,8,9,10,11,12
* x86_64; CentOS 7.7; gcc 4.8.5 (with "--disable-gcrypt" flag)
* Raspberry 4; Debian 10.10; gcc 8.3.0
2021-08-20 18:11:13 +02:00
Ivan Nardi 55eec29c08 wireshark/lua: restore full flow risks dissection (#1275)
It was partially disabled in 3eba8cc5.
Wireshark and Wireshark-Lua bindings don't handle 64 bit integer very well.
(see https://www.wireshark.org/docs/wsdg_html_chunked/lua_module_Int64.html).

As workaround, only for visualization purpose, split the (64 bit) risk mask
into two 32 bit integer values.
2021-08-18 11:34:46 +02:00
Toni 8d0c7b1fae Fixed Mingw64 build, SonerCloud-CI and more. (#1273)
* Added ARM build and unit test run for SonarCloud-CI.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Fixed Mingw64 build.

 * adapted to SonarCloud-CI workflow
 * removed broken and incomplete Windows example (tested on VS2017/VS2019)
 * removed unnecessary include (e.g. pthread.h for the library which does not make use of it)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-08-18 11:34:16 +02:00
Luca Deri 677b513f70 Fixes a crash on ARM (Raspberry Pi 4 Model B Rev 1.1) 2021-08-18 11:12:06 +02:00
Luca Deri 821f4c9249 Fixed some invalid TLS guesses 2021-08-17 19:04:07 +02:00
Luca Deri a2ea67152c Added check to avoid clases on similar protocols (FTP adn SMTP) on setting hostname 2021-08-11 12:11:58 +02:00
Luca Deri 5c33fbf19b Added extraction of hostname in SMTP
Fixed mail incalid subprotocol calculation
2021-08-11 11:52:24 +02:00
Alfredo Cardigliano 846b546dbc Fix typo 2021-08-10 11:43:32 +02:00
Luca Deri c1acb6d801 Typo 2021-08-09 18:36:36 +02:00
Luca Deri 0fbe895103 Addef ETH_ARP definition 2021-08-09 18:33:33 +02:00
Luca Deri 05492ec8e8 Defined more standard ethernet protocol types 2021-08-09 17:01:10 +02:00
Luca Deri 58ca7b5a7f Added protocol defines 2021-08-09 16:45:03 +02:00
Luca Deri 0df8aa19ec Clode cleanup (after last merge) 2021-08-08 17:10:08 +02:00
ToniandLuca Deri 259b60f574 Added entropy calculation to check for suspicious (encrypted) payload. (#1270)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

Co-authored-by: Luca Deri <lucaderi@users.noreply.github.com>
2021-08-08 17:03:23 +02:00
Toni 4e856a41d6 Skip whitespaces between HTTP method and URL. (#1271)
* be less case-restrictive, RFC2616 wants it that way

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-08-08 17:00:10 +02:00
Toni 8e996f7f7c Fixed broken SonarCloud-CI. (#1272)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-08-08 16:59:36 +02:00
Luca Deri 3706267d68 Updated test output 2021-08-07 19:43:55 +02:00
Luca Deri 0522e562e8 Added testing pcap for TLS fatal alert 2021-08-07 19:42:53 +02:00
Luca Deri 4183718952 Added TLS fatal alert flow risk 2021-08-07 19:40:44 +02:00
Luca Deri e8455236bd Updated output 2021-08-07 17:38:33 +02:00
Luca Deri a13f1fe52f Report whether a protocol is encrypted 2021-08-07 17:35:34 +02:00
Luca Deri 46ebd7128f Added ability to report whether a protocol is encrypted
Added new API calls
- u_int8_t ndpi_is_valid_protoId(u_int16_t protoId);
Cleaned up code and organized in a function.

- u_int8_t ndpi_is_encrypted_proto(struct ndpi_detection_module_struct *ndpi_str, ndpi_protocol proto);
Check whether a protocol is encrypted or not based on master and app protocols
2021-08-07 17:10:18 +02:00
Luca Deri dc841574af Win fix 2021-08-03 09:44:50 +02:00
Luca Deri 6b7e5fa8d2 Added legacy whatsapp rules used to match old pcaps that need to be removed in the future 2021-07-31 23:49:48 +02:00
Toni ad57af9f79 Improved RTSP detection and fixed HTTP false-positive. Fixes #1229. (#1266)
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-31 23:31:49 +02:00
Fábio Depin ce597b4806 Updated WhatsAPP and Instagram addresses (#1264)
* Updated WhatsAPP addresses

* Added Instagram addresses
2021-07-31 23:30:34 +02:00
Toni 13c5d6801e Improved libm check via Autoconf. (#1263)
* libm will now be part of @ADDITIONAL_LIBS@ (if required)

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-07-28 22:24:03 +02:00
Luca Deri cb26a8c82d Moved to 4.1.0 2021-07-26 09:40:50 +02:00
878 changed files with 121730 additions and 33625 deletions
-17
View File
@@ -1,17 +0,0 @@
mversion: 3.1.{build}
branches:
only:
- dev
environment:
matrix:
- COMPILER: mingw
MINGW_DIR: c:\msys64\mingw32
before_build:
- set Path=%MINGW_DIR%\bin;%Path%
build_script:
- set Path=%MINGW_DIR%\bin;c:\msys64\usr\bin;
- bash autogen.sh
- cd src/lib
- make
+4
View File
@@ -0,0 +1,4 @@
# Codige Code Review ignore
# Prevent Codiga complaining about Sphinx config.py
doc/conf.py
+3
View File
@@ -1,3 +1,6 @@
*.sh -crlf
*.ac -crlf
*.am -crlf
#Content files are C file
*.inc linguist-language=C
+50
View File
@@ -0,0 +1,50 @@
---
name: "🐞 Report a Bug"
about: Create a report to help us improve nDPI
title: ''
labels: 'bug'
assignees: ''
---
## Describe the bug
A clear and concise description of what the bug is.
### Expected behavior
A clear and concise description of what you expected to happen.
### Obtained behavior
A clear and concise description of what happening.
## nDPI Environment (please complete the following information):
* OS name: [e.g. Ubuntu].
* OS version: [e.g. 18.04]
* Architecture: [e.g. arm64]
* nDPI version or commit hash: [e.g. 4.0-stable or 937357e4bc55610f116f66d15a8e0fc1e260c02c].
* nDPI compilation flags used: if you are building from source [e.g. --with-pcre --with-local-libgcrypt].
* Attach the `config.log` file generated after `./configure` ran (if you are building from source).
## How to reproduce the reported bug
### Reproducible using ndpiReader?
- [x] The reported bug is reproducible using ndpiReader.
- [x] The reported bug is not reproducible using ndpiReader.
### If applicable, the used ndpiReader options:
``` shell
ndpiReader -q -t -i wlo1 -T 20 -U 20
```
### If your bug is reproducible using a pcap, please attach a pcap file (or a valid link to download it)
Example: test.pcap
### Steps to reproduce the behavior:
1. Run '...'
2. Set '....'
3. Do '....'
4. See error
## Additional context
Add any other context about the problem here.
@@ -0,0 +1,20 @@
---
name: "💡 Suggest an Improvement or Feature"
about: Suggest an idea or a feature for nDPI
title: ''
labels: 'enhancement'
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I always need to [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
@@ -0,0 +1,10 @@
---
name: General question
about: Ask a question about nDPI
title: ''
labels: 'question'
assignees: ''
---
Type your question in a clear and concise way to help the community answering it.
+470 -8
View File
@@ -2,19 +2,481 @@ name: Build
on:
push:
branches:
- master
- dev
pull_request:
branches:
- dev
types: [opened, synchronize, reopened]
release:
types: [created]
jobs:
sonarcloud:
name: SonarCloud
coverage:
name: Coverage (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
sudo apt-get install rrdtool librrd-dev
sudo apt-get install libpcre3-dev libmaxminddb-dev lcov
sudo apt-get install wdiff colordiff
- name: Configure
run: env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-code-coverage --with-pcre --with-maxminddb --enable-tls-sigs
- name: Build
run: make all
- name: Test
run: |
make check VERBOSE=1
- name: Generate Coverage Report
run: |
mkdir -vp coverage_report/ndpi_coverage_report
lcov --directory . --capture --output-file lcov.info
genhtml -o coverage_report/ndpi_coverage_report lcov.info
- uses: actions/upload-artifact@v2
with:
name: ndpi-coverage-report
path: coverage_report
retention-days: 7
python-bindings:
name: Python Bindings (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: "3.9"
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
- name: Build nDPI library
run: |
env CC=gcc CFLAGS='-Werror' ./autogen.sh
make
sudo make install
- name: Generate Python bindings
run: |
pip install --upgrade pip
pip install -r python/requirements.txt
cd python
python setup.py install
cd ..
- name: Test Python Bindings
run: |
cd python
python tests.py
documentation:
name: Documentation (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: "3.9"
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison doxygen
- name: Configure nDPI library
run: |
./autogen.sh --with-only-libndpi --enable-option-checking=fatal
- name: Generate Documentation
run: |
pip install --upgrade pip
pip install -r doc/requirements.txt
make doc
mkdir -vp doc/_build/ndpi-documentation-upload/ndpi-documentation
mv -v doc/_build/html doc/_build/ndpi-documentation-upload/ndpi-documentation/html
- uses: actions/upload-artifact@v2
with:
name: ndpi-documentation
path: doc/_build/ndpi-documentation-upload
retention-days: 7
performance:
name: Performance (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison libpcap-dev
sudo apt-get install golang-1.16-go libgoogle-perftools-dev graphviz
env GO111MODULE=on go install github.com/google/pprof@latest
sudo ln -s ${HOME}/go/bin/pprof /usr/bin/pprof
pprof -h
- name: Configure nDPI library
run: |
./autogen.sh --enable-gprof --enable-option-checking=fatal --with-pcre --with-maxminddb --enable-tls-sigs
- name: Build nDPI library
run: |
make
- name: Performance Profiling
run: |
./tests/do.sh
mkdir ndpi-performance-upload
mv -v tests/result/cpu_profile.png ndpi-performance-upload/cpu_profile.png
mv -v tests/result/heap_profile.png ndpi-performance-upload/heap_profile.png
- uses: actions/upload-artifact@v2
with:
name: ndpi-performance
path: ndpi-performance-upload
retention-days: 7
test-scripts:
name: Test Utils (ubuntu-latest)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- name: Install Ubuntu Prerequisites
run: |
sudo apt-get update
sudo apt-get install python3-netaddr git
- name: Run Scripts
run: |
echo 'Running ./utils/bitcoinnodes.sh'
./utils/bitcoinnodes.sh >/dev/null
echo 'Running ./utils/get_routes_by_asn.sh AS714'
./utils/get_routes_by_asn.sh AS714 >/dev/null
echo 'Running ./utils/update_every_lists.sh'
./utils/update_every_lists.sh
echo 'Checking for changes in the git tree..'
git update-index --refresh || echo "::warning file=utils/update_every_lists.sh::Please re-run utils/update_every_lists.sh and commit any changes."
git diff-index --quiet HEAD -- || true
test:
name: ${{ matrix.os }} ${{ matrix.arch }} ${{ matrix.gcrypt }} ${{ matrix.compiler }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} ${{ matrix.msan }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: true
matrix:
os: ["ubuntu-latest", "ubuntu-18.04", "macOS-latest", "macos-11", "windows-latest"]
arch: ["x86_64"]
gcrypt: ["--with-local-libgcrypt", ""]
compiler: ["default-cc"]
pcre: [""]
maxminddb: [""]
msan: [""]
include:
- compiler: "gcc-7"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "gcc-10"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "clang-7"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "clang-12"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "default-cc"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "default-cc"
os: ubuntu-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-thread-sanitizer"
- compiler: "default-cc"
os: ubuntu-18.04
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "default-cc"
os: macOS-latest
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "" # Disable sanitizer on macos
- compiler: "default-cc"
os: macos-11
arch: "x86_64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "" # Disable sanitizer on macos
- compiler: "default-cc"
os: ubuntu-latest
arch: "arm64"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "" # Disable sanitizer on arm64
- compiler: "default-cc"
os: ubuntu-latest
arch: "armhf"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: "--with-sanitizer"
- compiler: "default-cc"
os: ubuntu-latest
arch: "s390x"
gcrypt: ""
pcre: "--with-pcre"
maxminddb: "--with-maxminddb"
msan: ""
steps:
- name: Setup multiarch/qemu-user-static
if: startsWith(matrix.os, 'ubuntu') && !startsWith(matrix.arch, 'x86_64')
run: |
docker run --rm --privileged multiarch/qemu-user-static:register --reset
- uses: actions/checkout@v2
with:
fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis
- name: SonarCloud Scan
uses: SonarSource/sonarcloud-github-action@master
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # Needed to get PR information, if any
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
- name: Install Ubuntu Prerequisites
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
run: |
sudo apt-get update
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
sudo apt-get install gcc-mingw-w64 libc6-dev
sudo apt-get install doxygen python3-sphinx python3-sphinx-rtd-theme python3-breathe python3-pip
sudo apt-get install rrdtool librrd-dev
- name: Install Ubuntu Prerequisites (libgcrypt)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.gcrypt, '--with-local-libgcrypt')
run: |
sudo apt-get install libgcrypt20-dev
- name: Install Ubuntu Prerequisites (libpcre)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.pcre, '--with-pcre')
run: |
sudo apt-get install libpcre3-dev
- name: Install Ubuntu Prerequisites (maxminddb)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.maxminddb, '--with-maxminddb')
run: |
sudo apt-get install libmaxminddb-dev
- name: Setup Ubuntu specified compiler (gcc)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'gcc')
run: |
sudo apt-get install ${{ matrix.compiler }}
- name: Setup Ubuntu specified compiler (clang)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'clang')
run: |
sudo apt-get install ${{ matrix.compiler }}
- name: Install Windows prerequisites
if: startsWith(matrix.os, 'windows')
uses: msys2/setup-msys2@v2
with:
msystem: MINGW64
update: true
install: git mingw-w64-x86_64-toolchain automake1.16 automake-wrapper autoconf libtool make mingw-w64-x86_64-json-c mingw-w64-x86_64-crt-git mingw-w64-x86_64-pcre mingw-w64-x86_64-libpcap
- name: Setup Windows prerequisites (libgcrypt)
if: startsWith(matrix.os, 'windows') && startsWith(matrix.gcrypt, '--with-local-libgcrypt')
run: |
msys2 -c 'git clone --branch libgpg-error-1.42 https://github.com/gpg/libgpg-error'
cd libgpg-error
msys2 -c './autogen.sh'
msys2 -c './configure -enable-maintainer-mode --enable-static --enable-shared --with-pic --disable-doc --disable-nls'
msys2 -c 'make'
msys2 -c 'make install'
msys2 -c 'cd ..'
msys2 -c 'rm -rf libgpg-error/'
msys2 -c 'git clone --branch libgcrypt-1.8.8 https://github.com/gpg/libgcrypt'
cd libgcrypt
msys2 -c './autogen.sh'
msys2 -c './configure -enable-maintainer-mode --enable-static --enable-shared --with-pic --disable-doc'
msys2 -c 'make'
msys2 -c 'make install'
msys2 -c 'cd ..'
msys2 -c 'rm -rf libgcrypt/'
- name: Installing MacOS prerequisites
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64')
run: |
brew install coreutils wdiff colordiff autoconf automake libtool pkg-config gettext json-c rrdtool
- name: Install MacOS Prerequisites (libgcrypt)
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.gcrypt, '--with-local-libgcrypt')
run: |
brew install libgcrypt
- name: Install MacOS Prerequisites (libpcre)
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.pcre, '--with-pcre')
run: |
brew install pcre
- name: Install MacOS Prerequisites (maxminddb)
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.maxminddb, '--with-maxminddb')
run: |
brew install libmaxminddb
- name: Configure nDPI on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'default-cc')
run: |
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs
- name: Configure nDPI on MacOS
if: startsWith(matrix.os, 'macOS') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'default-cc')
run: |
env CC=clang CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs
- name: Configure nDPI on Windows
if: startsWith(matrix.os, 'windows') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'default-cc')
run: |
msys2 -c 'env CFLAGS=\"-Werror\" ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --disable-npcap ${{ matrix.gcrypt }}'
- name: Build nDPI on Windows
if: startsWith(matrix.os, 'windows') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'default-cc')
run: |
msys2 -c 'make all'
msys2 -c 'ldd ./example/ndpiReader.exe'
- name: Configure nDPI with specified GCC version on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'gcc')
run: |
env CC=${{ matrix.compiler }} CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs
- name: Configure nDPI with specified CLANG on Ubuntu
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64') && startsWith(matrix.compiler, 'clang')
run: |
env CC=${{ matrix.compiler }} CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs
- name: Build nDPI
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
make all
make -C example ndpiSimpleIntegration
make -C rrdtool
- name: Install nDPI
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
make install DESTDIR=$(realpath _install)
ls -alhHR _install
- name: Test nDPI [DIFF]
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
./tests/do.sh
- name: Test nDPI [UNIT]
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
./tests/do-unit.sh
- name: Test nDPI [DGA]
if: startsWith(matrix.arch, 'x86_64') && !startsWith(matrix.os, 'windows')
run: |
./tests/do-dga.sh
- name: Test nDPI [DIFF] (runs only on windows jobs)
if: startsWith(matrix.arch, 'x86_64') && startsWith(matrix.os, 'windows')
run: |
msys2 -c './tests/do.sh'
- name: Test nDPI [UNIT] (runs only on windows jobs)
if: startsWith(matrix.arch, 'x86_64') && startsWith(matrix.os, 'windows')
run: |
msys2 -c './tests/do-unit.sh'
- name: Test nDPI [DGA] (runs only on windows jobs)
if: startsWith(matrix.arch, 'x86_64') && startsWith(matrix.os, 'windows')
run: |
msys2 -c './tests/do-dga.sh'
- name: Generate/Verify tarball
if: startsWith(matrix.os, 'ubuntu-latest') && startsWith(matrix.arch, 'x86_64')
run: |
make dist
./utils/verify_dist_tarball.sh
- name: Build nDPI [Mingw-w64] (runs only on ubuntu jobs)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'x86_64')
run: |
make distclean
env CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --host=x86_64-w64-mingw32
make all
- name: Display qemu specified architecture (arm64 - little endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'arm64')
uses: docker://multiarch/ubuntu-core:arm64-bionic
with:
args: >
bash -c
"uname -a &&
lscpu | grep Endian
"
- name: Configure, compile and test using qemu for the specified architecture (arm64 - little endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'arm64')
uses: docker://multiarch/ubuntu-core:arm64-bionic
with:
args: >
bash -c
"apt-get -y update &&
apt-get -y install git wdiff colordiff autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
git config --global --add safe.directory $(realpath .) &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs &&
make all &&
make -C example ndpiSimpleIntegration &&
make -C rrdtool &&
make check VERBOSE=1
"
- name: Display qemu specified architecture (armhf - little endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'armhf')
uses: docker://multiarch/ubuntu-core:armhf-bionic
with:
args: >
bash -c
"uname -a &&
lscpu | grep Endian
"
- name: Configure, compile and test using qemu for the specified architecture (armhf - little endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 'armhf')
uses: docker://multiarch/ubuntu-core:armhf-bionic
with:
args: >
bash -c
"apt-get -y update &&
apt-get -y install git wdiff colordiff autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
git config --global --add safe.directory $(realpath .) &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs &&
make all &&
make -C example ndpiSimpleIntegration &&
make -C rrdtool &&
make check VERBOSE=1
"
- name: Display qemu specified architecture (s390x - big endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 's390x')
uses: docker://multiarch/ubuntu-core:s390x-bionic
with:
args: >
bash -c
"uname -a &&
lscpu | grep Endian
"
- name: Configure and compile using qemu for the specified architecture (s390x - big endian)
if: startsWith(matrix.os, 'ubuntu') && startsWith(matrix.arch, 's390x')
uses: docker://multiarch/ubuntu-core:s390x-bionic
with:
args: >
bash -c
"apt-get -y update &&
apt-get -y install git wdiff colordiff autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev libgcrypt20-dev libpcre3-dev libmaxminddb-dev rrdtool librrd-dev &&
git config --global --add safe.directory $(realpath .) &&
env CC=gcc CFLAGS='-Werror' ./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.gcrypt }} ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs &&
make all &&
make -C example ndpiSimpleIntegration &&
make -C rrdtool &&
make check VERBOSE=1
"
+32 -21
View File
@@ -1,23 +1,34 @@
name: CIFuzz
on: [pull_request]
on: [push, pull_request]
jobs:
Fuzzing:
runs-on: ubuntu-latest
steps:
- name: Build Fuzzers
uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@master
with:
oss-fuzz-project-name: 'ndpi'
dry-run: false
- name: Run Fuzzers
uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@master
with:
oss-fuzz-project-name: 'ndpi'
fuzz-seconds: 600
dry-run: false
- name: Upload Crash
uses: actions/upload-artifact@v1
if: failure()
with:
name: artifacts
path: ./out/artifacts
Fuzzing:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
sanitizer: [address, undefined, memory]
steps:
- name: Build Fuzzers - ${{ matrix.sanitizer }}
uses: google/oss-fuzz/infra/cifuzz/actions/build_fuzzers@master
with:
oss-fuzz-project-name: 'ndpi'
dry-run: false
sanitizer: ${{ matrix.sanitizer }}
- name: Run Fuzzers - ${{ matrix.sanitizer }}
uses: google/oss-fuzz/infra/cifuzz/actions/run_fuzzers@master
with:
oss-fuzz-project-name: 'ndpi'
fuzz-seconds: 600
dry-run: false
sanitizer: ${{ matrix.sanitizer }}
- name: Check Crash (fails when a crash is detected)
# Run Fuzzers return success even when setting dry-run to false.
# A temporal workaround is to trigger failure manually if we fing crash files.
run: |
exit $(ls out/artifacts |wc -l)
- name: Upload Crash (upload detected crash as artifacts)
uses: actions/upload-artifact@v1
if: failure()
with:
name: ${{ matrix.sanitizer }}-artifacts
path: ./out/artifacts
+27 -1
View File
@@ -11,7 +11,6 @@ compile
/config.status
/config.sub
/configure
/configure.ac
/configure.tmp
.cproject
/depcomp
@@ -21,6 +20,7 @@ compile
/example/ndpiReader
/example/Makefile.dpdk
/example/build
/example/ndpiSimpleIntegration
*.in
/install-sh
*.ko
@@ -45,6 +45,7 @@ compile
.settings
/src/include/Makefile
/src/include/ndpi_config.h
/src/include/ndpi_api.h
/src/include/ndpi_define.h
/src/lib/.deps/
/src/lib/Makefile
@@ -54,3 +55,28 @@ stamp-h1
*.swp
/test-driver
/tests/Makefile
/fuzz/.deps/
/fuzz/Makefile
/fuzz/fuzz_ndpi_reader
/fuzz/fuzz_ndpi_reader_with_main
/fuzz/fuzz_process_packet
/fuzz/fuzz_process_packet_with_main
/tests/dga/Makefile
/tests/do.sh
/tests/unit/Makefile
/example/libndpiReader.a
/example/ndpiReader.exe
/src/lib/libndpi.a
/src/lib/libndpi.so*
/src/lib/libndpi
/src/lib/libndpi-4.1.0.dll
/src/lib/libndpi.4
/tests/dga/dga_evaluate
/tests/unit/unit
/tags
/rrdtool/Makefile
/rrdtool/rrd_anomaly
/rrdtool/rrd_similarity
/tests/performance/Makefile
/influxdb/Makefile
/doc/Doxyfile.cfg
@@ -1,83 +0,0 @@
/**
* @name Suspicious packet->payload based integer arithmetic
* @description An arithmetic operation influenced array access is suspicious
* if it uses an integer value that is likely to be network-controlled, and
* may require a closer manual audit.
* @kind problem
* @problem.severity warning
* @id cpp/packet-payload-integer-arithmetic
* @tags audit security
*/
import cpp
import semmle.code.cpp.dataflow.TaintTracking
import semmle.code.cpp.rangeanalysis.SimpleRangeAnalysis
/** A source of an integer value that is likely to come from the network.
* This is produced by an invocation of a macro of the form `ntoh*` or `get_u_int*_t`,
* called with `packet->payload` as an argument.
*/
class NetworkMacro extends Macro {
NetworkMacro() { this.getName().regexpMatch("^ntoh(ll|l|s)") }
}
class NetworkIntegerSource extends Expr {
NetworkIntegerSource() {
exists(MacroInvocation mi |
this = mi.getExpr() and
mi.getUnexpandedArgument(0).regexpMatch(".*packet->payload.*") |
// catch all get_u_int*_t(x)
mi.getMacroName().regexpMatch("^get_u_int(64|32|16|8)_t") and
// dedup ntoh*(get_u_int*_t(x)) since we'll catch those in the next case
not mi.getOutermostMacroAccess().getMacro() instanceof NetworkMacro
or
// catch all ntoh*(x) ... this will also catch the nested cases
mi.getMacro() instanceof NetworkMacro
)
}
}
class ArithmeticOperation extends Operation {
ArithmeticOperation() {
this instanceof UnaryArithmeticOperation or this instanceof BinaryArithmeticOperation
}
}
class NetworkToArrayAccess extends TaintTracking::Configuration {
NetworkToArrayAccess() { this = "NetworkToArrayAccess" }
override predicate isSource(DataFlow::Node source) {
source.asExpr() instanceof NetworkIntegerSource
}
override predicate isSink(DataFlow::Node sink) {
exists(ArrayExpr ae | sink.asExpr() = ae.getArrayOffset())
}
}
class NetworkToArithmetic extends TaintTracking::Configuration {
NetworkToArithmetic() { this = "NetworkToArithmetic" }
override predicate isSource(DataFlow::Node source) {
source.asExpr() instanceof NetworkIntegerSource
}
override predicate isSink(DataFlow::Node sink) {
exists (Assignment assign |
sink.asExpr() = assign.getRValue().(ArithmeticOperation) or
sink.asExpr() = assign.(AssignArithmeticOperation)
) or
exists(LocalVariable var |
sink.asExpr() = var.getInitializer().getExpr().(ArithmeticOperation)
)
}
}
// find audit candidates based on suspicious network integer use
from NetworkIntegerSource source, Expr sink1, Expr sink2, NetworkToArithmetic config1, NetworkToArrayAccess config2
where config1.hasFlow(DataFlow::exprNode(source), DataFlow::exprNode(sink1))
// or this if you want integer arithmeric _OR_ array accesses
and config2.hasFlow(DataFlow::exprNode(source), DataFlow::exprNode(sink2))
select source, "Suspicious use of network integer arithmetic."
-238
View File
@@ -1,238 +0,0 @@
language: c
matrix:
include:
- os: osx
compiler: clang
- os: linux
env: BUILD_DPKG_PACKAGE=y
compiler: gcc
dist: bionic
addons:
apt:
packages:
- fakeroot
- debhelper
- dpkg-dev
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
- os: linux
env: DISABLE_GCRYPT=y
compiler: gcc
dist: bionic
addons:
apt:
packages:
- fakeroot
- debhelper
- dpkg-dev
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
- os: linux
compiler: clang
dist: bionic
addons:
apt:
packages:
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
- os: linux
compiler: gcc
dist: bionic
arch: arm64
addons:
apt:
packages:
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
- os: linux
compiler: clang
dist: bionic
arch: arm64
addons:
apt:
packages:
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
# Targets below have been disabled as we have no way
# to debug on tsuch platforms at the moment
# - os: linux
# compiler: gcc
# dist: bionic
# arch: s390x
# addons:
# apt:
# packages:
# - libpcap-dev
# - libjson-c-dev
# - autogen
#
# - os: linux
# compiler: clang
# dist: bionic
# arch: s390x
# addons:
# apt:
# packages:
# - libpcap-dev
# - libjson-c-dev
# - autogen
- os: linux
compiler: gcc-8
addons:
apt:
sources:
- ubuntu-toolchain-r-test
packages:
- g++-8
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
env:
- MATRIX_EVAL="CC=gcc-8"
- os: linux
compiler: gcc-9
addons:
apt:
sources:
- ubuntu-toolchain-r-test
packages:
- g++-9
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
env:
- MATRIX_EVAL="CC=gcc-9"
- os: linux
compiler: clang-8
addons:
apt:
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty-8
packages:
- clang-8
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
env:
- MATRIX_EVAL="CC=clang-8"
- os: linux
compiler: clang-7
addons:
apt:
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty-7
packages:
- clang-7
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
env:
- MATRIX_EVAL="CC=clang-7"
- name: fuzza
env: CXXFLAGS="-g3 -O0 -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize=fuzzer-no-link" CFLAGS="-g3 -O0 -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize=fuzzer-no-link" LDFLAGS="-g3 -O0 -fsanitize=address" QA_FUZZ=asan CC=clang-7 && CXX=clang++-7 ASAN_SYMBOLIZER_PATH=/usr/local/clang-7.0.0/bin/llvm-symbolizer
os: linux
compiler: clang-7
addons:
apt:
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty-7
packages:
- clang-7
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
- name: fuzzm
env: CXXFLAGS="-g3 -O0 -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=memory -fsanitize=fuzzer-no-link" CFLAGS="-g3 -O0 -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=memory -fsanitize=fuzzer-no-link" LDFLAGS="-g3 -O0 -fsanitize=memory" QA_FUZZ=msan CC=clang && CXX=clang++
os: linux
compiler: clang
dist: bionic
addons:
apt:
packages:
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
- name: fuzzu
env: CXXFLAGS="-g3 -Og -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=undefined -fsanitize=fuzzer-no-link -fno-sanitize=alignment" CFLAGS="-g3 -Og -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=undefined -fno-sanitize-recover=undefined,integer -fsanitize=fuzzer-no-link -fno-sanitize=alignment" LDFLAGS="-g3 -Og -fsanitize=undefined -fno-sanitize=alignment" QA_FUZZ=ubsan CC=clang-7 && CXX=clang++-7
os: linux
compiler: clang-7
addons:
apt:
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-trusty-7
packages:
- clang-7
- libpcap-dev
- libgcrypt20-dev
- libjson-c-dev
- autogen
before_install:
- eval "${MATRIX_EVAL}"
before_script:
- ./autogen.sh
# - lcov --directory . --zerocounters
script:
- if [ -n "$QA_FUZZ" ]; then ./configure --enable-fuzztargets ; else
./configure ; fi
- if [ -n "$BUILD_DPKG_PACKAGE" ]; then cd ./packages/ubuntu ;
./configure && dpkg-buildpackage -b -us -uc ; cd ../.. ; fi
- if [ -n "$DISABLE_GCRYPT" ]; then ./configure --disable-gcrypt ; fi
- make
- make -C example ndpiSimpleIntegration
- make dist
#after_script:
- cd tests
- ./do.sh
- ./do-unit.sh
- ./do-dga.sh
#after_success:
#- cd ${TRAVIS_BUILD_DIR}
#- lcov --directory . --capture --output-file coverage.info # capture coverage info
#- lcov --remove coverage.info 'tests/*' 'packages/*' 'm4/*' 'doc/*' --output-file coverage.info
#- lcov --list coverage.info # debug before upload
#- coveralls-lcov --repo-token ${COVERALLS_TOKEN} coverage.info # uploads to coveralls
notifications:
email:
recipients:
- packager@ntop.org
on_success: never
on_failure: always
+96
View File
@@ -1,5 +1,101 @@
# CHANGELOG
#### nDPI 4.2 (Feb 2022)
## New Features
- Add a "confidence" field indicating the reliability of the classification
- Add risk exceptions for services and domain names via ndpi_add_domain_risk_exceptions()
- Add ability to report whether a protocol is encrypted
## New Supported Protocols and Services
- Add protocol detection for:
- Badoo
- Cassandra
- EthernetIP
## Improvements
- Reduce memory footprint
- Improve protocol detection for:
- BitTorrent
- ICloud Private Relay
- IMAP, POP3, SMTP
- Log4J/Log4Shell
- Microsoft Azure
- Pandora TV
- RTP
- RTSP
- Salesforce
- STUN
- Whatsapp
- QUICv2
- Zoom
- Add flow risk:
- NDPI_CLEAR_TEXT_CREDENTIALS
- NDPI_POSSIBLE_EXPLOIT (Log4J)
- NDPI_TLS_FATAL_ALERT
- NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE
- Update WhatsAPP and Instagram addresses
- Update the list of default ports for QUIC
- Update WindowsUpdate URLs
- Add support for the .goog Google TLD
- Add googletagmanager.com
- Add bitmaps and API for handling compressed bitmaps
- Add JA3 in risk exceptions
- Add entropy calculation to check for suspicious (encrypted) payload
- Add extraction of hostname in SMTP
- Add RDP over UDP dissection
- Add support for TLS over IPV6 in Subject Alt Names field
- Improve JSON and CSV serialization
- Improve IPv6 support for almost all dissectors
- Improve CI and unit tests, add arm64, armhf and s390x as part of CI
- Improve WHOIS detection, reduce false positives
- Improve DGA detection for skipping potential DGAs of known/popular domain names
- Improve user agent analysis
- Reworked HTTP protocol dissection including HTTP proxy and HTTP connect
## Changes
- TLS obsolete protocol is set when TLS < 1.2 (used to be 1.1)
- Numeric IPs are not considered for DGA checks
- Differentiate between standard Amazon stuff (i.e market) and AWS
- Remove Playstation VUE protocol
- Remove pandora.tv from Pandora protocol
- Remove outdated SoulSeek dissector
## Fixes
- Fix race conditions
- Fix dissectors to be big-endian friendly
- Fix heap overflow in realloc wrapper
- Fix errors in Kerberos, TLS, H323, Netbios, CSGO, Bittorrent
- Fix wrong tuple comparison
- Fix ndpi_serialize_string_int64
- Fix Grease values parsing
- Fix certificate mismatch check
- Fix null-dereference read for Zattoo with IPv6
- Fix dissectors initialization for XBox, Diameter
- Fix confidence for STUN classifications
- Fix FreeBSD support
- Fix old GQUIC versions on big-endian machines
- Fix aho-corasick on big-endian machines
- Fix DGA false positive
- Fix integer overflow for QUIC
- Fix HTTP false positives
- Fix SonarCloud-CI support
- Fix clashes setting the hostname on similar protocols (FTP, SMTP)
- Fix some invalid TLS guesses
- Fix crash on ARM (Raspberry)
- Fix DNS (including fragmented DNS) dissection
- Fix parsing of IPv6 packets with extension headers
- Fix extraction of Realm attribute in STUN
- Fix support for START-TLS sessions in FTP
- Fix TCP retransmissions for multiple dissectors
- Fix DES initialisation
- Fix Git protocol dissection
- Fix certificate mismatch for TLS flows with no client hello observed
- Fix old versions of GQUIC on big-endian machines
## Misc
- Add tool for generating automatically the Azure IP list
#### nDPI 4.0 (July 2021)
## New Features
+19 -1
View File
@@ -8,7 +8,25 @@ endif
pkgconfigdir = $(libdir)/pkgconfig
pkgconfig_DATA = libndpi.pc
EXTRA_DIST = README.md CHANGELOG.md CONTRIBUTING.md README.protocols autogen.sh configure.seed wireshark python windows utils packages doc/nDPI_QuickStartGuide.pages doc/nDPI_QuickStartGuide.pdf example/MacOS example/Win32
EXTRA_DIST = README.md README.fuzzer.md CHANGELOG.md CONTRIBUTING.md \
autogen.sh wireshark windows utils packages \
influxdb/Makefile.in influxdb/metric_anomaly.c influxdb/README.txt \
rrdtool/Makefile.in rrdtool/README.txt rrdtool/rrd_anomaly.c rrdtool/rrd_similarity.c \
doc/requirements.txt doc/conf.py doc/flow_risks.rst doc/guide/nDPI_QuickStartGuide.pages \
doc/guide/nDPI_QuickStartGuide.pdf doc/img/logo.png doc/index.rst \
doc/Makefile doc/what_is_ndpi.rst \
python/DEV_GUIDE.md python/dev_requirements.txt python/ndpi_example.py python/ndpi/__init__.py \
python/ndpi/ndpi_build.py python/ndpi/ndpi.py python/README.md \
python/requirements.txt python/setup.py python/tests.py \
sonar-project.properties .github .ci-ignore lgtm.yml
.PHONY: doc doc-view
doc:
make -C doc html
doc-view:
python3 -m http.server --bind 127.0.0.1 --directory doc/_build/html 8080
changelog:
git log --since={`curl -s https://github.com/ntop/ndpi/releases | grep datetime | head -n1 | egrep -o "[0-9]+\-[0-9]+\-[0-9]+"`} --name-only --pretty=format:" - %s" | grep "^ " > CHANGELOG.latest
+41
View File
@@ -0,0 +1,41 @@
## Compiling nDPI with Fuzzer
- Install the latest clang (sudo apt-get install clang-10)
- Export environamental variables to prefer clang over g++ (of installed)
- export CC=/usr/bin/clang-10
- export CPP=/usr/bin/clang-cpp-10
- export CXX=/usr/bin/clang++-10
- export LD=/usr/bin/ld.lld-10
- Run autogen.sh
- ./autogen.sh --with-sanitizer --enable-fuzztargets
## Testing nDPI with ClusterFuzz Artifacts
- Download the artifact (example clusterfuzz-testcase-fuzz_process_packet-4992218834796544)
- Run nDPI against the artifact
- Example: ./fuzz/fuzz_process_packet_with_main clusterfuzz-testcase-fuzz_process_packet-4992218834796544
The output is the error report
```
./fuzz/fuzz_process_packet_with_main /tmp/clusterfuzz-testcase-fuzz_process_packet-4992218834796544
AddressSanitizer:DEADLYSIGNAL
=================================================================
==11590==ERROR: AddressSanitizer: SEGV on unknown address 0x61a100000087 (pc 0x00000056e6a4 bp 0x7ffd624fa170 sp 0x7ffd624fa090 T0)
==11590==The signal is caused by a READ memory access.
#0 0x56e6a4 in quic_len /home/deri/nDPI/src/lib/protocols/quic.c:203:12
#1 0x575d6b in decrypt_initial_packet /home/deri/nDPI/src/lib/protocols/quic.c:993:16
#2 0x571776 in get_clear_payload /home/deri/nDPI/src/lib/protocols/quic.c:1302:21
#3 0x56f149 in ndpi_search_quic /home/deri/nDPI/src/lib/protocols/quic.c:1658:19
#4 0x503935 in check_ndpi_detection_func /home/deri/nDPI/src/lib/ndpi_main.c:4683:6
#5 0x5056fb in check_ndpi_udp_flow_func /home/deri/nDPI/src/lib/ndpi_main.c:4742:10
#6 0x505152 in ndpi_check_flow_func /home/deri/nDPI/src/lib/ndpi_main.c:4775:12
#7 0x5174cf in ndpi_detection_process_packet /home/deri/nDPI/src/lib/ndpi_main.c:5545:15
#8 0x4c709b in LLVMFuzzerTestOneInput /home/deri/nDPI/fuzz/fuzz_process_packet.c:30:3
#9 0x4c7640 in main /home/deri/nDPI/fuzz/fuzz_process_packet.c:90:17
#10 0x7f888e5dabf6 in __libc_start_main /build/glibc-S9d2JN/glibc-2.27/csu/../csu/libc-start.c:310
#11 0x41c399 in _start (/home/deri/nDPI/fuzz/fuzz_process_packet_with_main+0x41c399)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/deri/nDPI/src/lib/protocols/quic.c:203:12 in quic_len
==11590==ABORTING
```
+50 -21
View File
@@ -1,7 +1,7 @@
![ntop][ntopng_logo] ![ntop][ntop_logo]
# nDPI
[![Build Status](https://travis-ci.org/ntop/nDPI.png?branch=dev)](https://travis-ci.org/ntop/nDPI)
[![Build Status](https://img.shields.io/github/workflow/status/ntop/nDPI/Build/dev?logo=github)](https://github.com/ntop/nDPI/actions?query=workflow%3ABuild)
[![Code Quality: Cpp](https://img.shields.io/lgtm/grade/cpp/g/ntop/nDPI.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/ntop/nDPI/context:cpp)
[![Total Alerts](https://img.shields.io/lgtm/alerts/g/ntop/nDPI.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/ntop/nDPI/alerts)
[![Fuzzing Status](https://oss-fuzz-build-logs.storage.googleapis.com/badges/ndpi.svg)](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened&can=1&q=proj:ndpi)
@@ -12,44 +12,70 @@ nDPI® is an open source LGPLv3 library for deep-packet inspection. Based on Ope
### How To Compile nDPI
In order to compile this library do
In order to compile this project do
- ./autogen.sh
- ./configure
- make
To compile the library w/o any tools or tests:
- ./autogen.sh --with-only-libndpi
- make
To run tests do additionally:
- cd tests; ./do.sh
- ./tests/do.sh # Generate and check for diff's in PCAP files
- ./tests/do-unit.sh # Run unit tests
- ./tests/do-dga.sh # Run DGA detection test
Please note that the pre-requisites for compilation include:
- GNU tools (autogen, automake, autoconf, libtool)
- GNU C compiler (gcc)
or run all with: `make check`
Please note that the (minimal) pre-requisites for compilation include:
- GNU tools (autoconf automake libtool pkg-config gettext flex bison)
- GNU C compiler (gcc) or Clang
On Debian/Ubuntu systems do:
- sudo apt-get install build-essential git bison flex libpcap-dev libtool libtool-bin autoconf pkg-config automake autogen libjson-c-dev libnuma-dev libgcrypt20-dev libpcre2-dev
- sudo apt-get install build-essential git gettext flex bison libtool autoconf automake pkg-config libpcap-dev libjson-c-dev libnuma-dev libpcre2-dev libmaxminddb-dev librrd-dev
FreeBSD
- sudo pkg install gcc git autoconf automake libtool devel/pkgconf json-c gmake
On Arch Linux:
- sudo pacman -S gcc git gettext flex bison libtool autoconf automake pkg-config libpcap json-c numactl pcre2 libmaxminddb rrdtool
Remember to use gmake and not make on FreeBSD
On FreeBSD:
- sudo pkg install gcc git gettext flex bison libtool autoconf automake devel/pkgconf gmake libpcap json-c pcre2 libmaxminddb rrdtool
Remember to use `gmake` and not `make` on FreeBSD
On MacOS:
- brew install coreutils gcc git gettext flex bison libtool autoconf automake pkg-config libpcap json-c pcre2 libmaxminddb rrdtool
On Windows (assuming [MSYS2](https://www.msys2.org/) already installed):
- msys2 -c "pacman --noconfirm -S --needed --overwrite '\*' git mingw-w64-x86\_64-toolchain automake1.16 automake-wrapper autoconf libtool make mingw-w64-x86\_64-json-c mingw-w64-x86\_64-crt-git mingw-w64-x86\_64-pcre mingw-w64-x86\_64-libpcap"
### How To Build The Documentation
- pip install --upgrade pip
- pip install -r doc/requirements.txt
- make doc
Use the builtin python3 webserver to view documentation:
- make doc-view
### How To Add A New Protocol Dissector
The entire procedure of adding new protocols in detail:
1. Add new protocol together with its unique ID to: src/include/ndpi_protocol_ids.h
2. Create a new protocol in: src/lib/protocols/
3. Variables to be kept for the duration of the entire flow (as state variables) need to be placed in: src/include/ndpi_typedefs.h in ndpi_flow_tcp_struct (for TCP only), ndpi_flow_udp_struct (for UDP only), or ndpi_flow_struct (for both).
4. Add a new entry for the search function for the new protocol in: src/include/ndpi_protocols.h
5. Choose (do not change anything) a selection bitmask from: src/include/ndpi_define.h
6. Add a new entry in ndpi_set_protocol_detection_bitmask2 in: src/lib/ndpi_main.c
7. Set protocol default ports in ndpi_init_protocol_defaults in: src/lib/ndpi_main.c
8. ./autogen.sh
9. make
10. make check
1. Add new protocol together with its unique ID to: `src/include/ndpi_protocol_ids.h`
2. Create a new protocol in: `src/lib/protocols/`
3. Variables to be kept for the duration of the entire flow (as state variables) need to be placed in: `src/include/ndpi_typedefs.h` in `ndpi_flow_tcp_struct` (for TCP only), `ndpi_flow_udp_struct` (for UDP only), or `ndpi_flow_struct` (for both).
4. Add a new entry for the search function for the new protocol in: `src/include/ndpi_protocols.h`
5. Choose (do not change anything) a selection bitmask from: `src/include/ndpi_define.h`
6. Set protocol default ports in `ndpi_init_protocol_defaults` in: `src/lib/ndpi_main.c`
7. `./autogen.sh`
8. `make`
9. `make check`
### How to use nDPI to Block Selected Traffic
You can use nDPI to selectively block selected Internet traffic by embedding it onto an application (remember that nDPI is just a library). Both [ntopng](https://github.com/ntop/ntopng) and [nProbe cento](http://www.ntop.org/products/netflow/nprobe-cento/) can do this.
### nDPI Paper Citation
@@ -57,9 +83,12 @@ You can use nDPI to selectively block selected Internet traffic by embedding it
- Deri, Luca, et al. [nDPI: Open-source high-speed deep packet inspection](http://luca.ntop.org/nDPI.pdf) 2014 International Wireless Communications and Mobile Computing Conference (IWCMC). IEEE, 2014.
### nDPI-Related Projects
- [nfstream](https://github.com/aouinizied/nfstream)
- [nDPId](https://github.com/utoni/nDPId)
### DISCLAIMER
While we do our best to detect network protocols, we cannot guarantee that our software is error free and 100% accurate in protocol detection. Please make sure that you respect the privacy of users and you have proper authorization to listen, capture and inspect network traffic.
nDPI is a registered trademark in the US and EU.
-22
View File
@@ -1,22 +0,0 @@
Prerequisites for Compilation
-----------------------------
Prerequisites
- GNU autotools/libtool
- libpcap or PF_RING (optional but recommended)
- apt-get install libjson-c-dev (for JSON support)
- libgcrypt20-dev (for QUIC support)
On Ubuntu/Debian
- apt-get install build-essential
- apt-get install git autoconf automake autogen libpcap-dev libtool libgcrypt20-dev
On Fedora/CentOS
- yum groupinstall "Development tools"
- yum install git autoconf automake autogen libpcap-devel libtool libgcrypt-devel
On MacOSX (using http://brew.sh)
brew install autoconf automake libtool git libgcrypt
On FreeBSD
- pkg install autoconf automake libtool gmake git libgcrypt
-18
View File
@@ -1,18 +0,0 @@
Tor
---
Tor protocol can use SSL to hide itself. These are examples:
TCP 37.128.208.46:9001 <-> 172.16.253.130:2078 [VLAN: 0][proto: 91/SSL][132 pkts/93834 bytes][SSL client: www.jwrpsthzrih.com]
TCP 172.16.253.130:2021 <-> 75.147.140.249:443 [VLAN: 0][proto: 91/SSL][28 pkts/8053 bytes][SSL client: www.5akw23dx.com]
TCP 172.16.253.130:2077 <-> 77.247.181.163:443 [VLAN: 0][proto: 91/SSL][136 pkts/94329 bytes][SSL client: www.fk4pprq42hsvl2wey.com]
It can be detected by analyzing the SSL client certificate and checking the name that does not match to a real host in
addition of being a bit weird. As doing DNS resolution is not a task for nDPI we let applications do and then recognize
SSL-tunnelled connections.
See http://www.netresec.com/?page=Blog&month=2013-04&post=Detecting-TOR-Communication-in-Network-Traffic
For this reason nDPI uses a heuristic, non-DNS based, approach to detect tor communications. If possible, apps
should validate the certificate using the DNS. This is not something nDPI can afford to do for performance
reasons
+10 -25
View File
@@ -1,10 +1,5 @@
#!/bin/sh
NDPI_MAJOR="3"
NDPI_MINOR="5"
NDPI_PATCH="0"
NDPI_VERSION_SHORT="$NDPI_MAJOR.$NDPI_MINOR.$NDPI_PATCH"
rm -f configure config.h config.h.in
AUTOCONF=$(command -v autoconf)
@@ -13,49 +8,39 @@ LIBTOOL=$(command -v libtool)
LIBTOOLIZE=$(command -v libtoolize)
AUTORECONF=$(command -v autoreconf)
PKG_CONFIG=$(command -v pkg-config)
FUZZY=
MAKE=$(command -v make)
if test -z $AUTOCONF; then
if test -z "${AUTOCONF}"; then
echo "autoconf is missing: please install it and try again"
exit
else
V=`autoconf --version | head -1 | cut -d ' ' -f 4`
if [ "$V" = '2.63' ]; then
FUZZY="dnl> "
fi
fi
if test -z $AUTOMAKE; then
if test -z "${AUTOMAKE}"; then
echo "automake is missing: please install it and try again"
exit
fi
if test -z $LIBTOOL && test -z $LIBTOOLIZE ; then
if test -z "${LIBTOOL}" && test -z "${LIBTOOLIZE}"; then
echo "libtool and libtoolize is missing: please install it and try again"
exit
fi
if test -z $AUTORECONF; then
if test -z "${AUTORECONF}"; then
echo "autoreconf is missing: please install it and try again"
exit
fi
if test -z $PKG_CONFIG; then
if test -z "${PKG_CONFIG}"; then
echo "pkg-config is missing: please install it (apt-get install pkg-config) and try again"
exit
fi
cat configure.seed | sed \
-e "s/@NDPI_MAJOR@/$NDPI_MAJOR/g" \
-e "s/@NDPI_MINOR@/$NDPI_MINOR/g" \
-e "s/@NDPI_PATCH@/$NDPI_PATCH/g" \
-e "s/@NDPI_VERSION_SHORT@/$NDPI_VERSION_SHORT/g" \
-e "s/@FUZZY@/$FUZZY/g" \
> configure.ac
if test -z "${MAKE}"; then
echo "make is missing; please install it (apt-get install make) and try again"
exit
fi
autoreconf -ivf
cat configure | sed "s/#define PACKAGE/#define NDPI_PACKAGE/g" | sed "s/#define VERSION/#define NDPI_VERSION/g" > configure.tmp
cat configure.tmp > configure
echo "./configure $@"
chmod +x configure
+367
View File
@@ -0,0 +1,367 @@
AC_INIT([libndpi],[4.4.0])
AC_CONFIG_AUX_DIR([.])
AC_CONFIG_MACRO_DIR([m4])
AM_INIT_AUTOMAKE([foreign subdir-objects])
AC_PREFIX_DEFAULT(/usr)
EXTRA_TARGETS="example tests tests/dga"
AC_ARG_WITH(only-libndpi, AS_HELP_STRING([--with-only-libndpi], [Build only libndpi (no examples, tests etc)]))
AS_IF([test "${with_only_libndpi+set}" = set],[
EXTRA_TARGETS=""
])
AC_ARG_WITH(sanitizer, AS_HELP_STRING([--with-sanitizer], [Build with support for address, undefined and leak sanitizer]))
AC_ARG_WITH(thread-sanitizer, AS_HELP_STRING([--with-thread-sanitizer], [Build with support for thread sanitizer]))
AC_ARG_ENABLE(fuzztargets, AS_HELP_STRING([--enable-fuzztargets], [Enable fuzz targets]),[enable_fuzztargets=$enableval],[enable_fuzztargets=no])
AC_ARG_ENABLE(gprof, AS_HELP_STRING([--enable-gprof], [Enable CPU/HEAP profiling with gperftools]),[enable_gprof=$enableval],[enable_gprof=no])
AC_ARG_ENABLE(code-coverage, AS_HELP_STRING([--enable-code-coverage], [Generate Code Coverage report]))
AC_ARG_WITH(local-libgcrypt, AS_HELP_STRING([--with-local-libgcrypt], [Build with libgcrypt (if present) instead of the enclosed gcrypt light]))
AC_ARG_ENABLE(tls-sigs, AS_HELP_STRING([--enable-tls-sigs], [Enable TLS Client signature algorithm dissection. Rarely used, but requires significantly more memory.]))
AC_ARG_ENABLE(npcap, AS_HELP_STRING([--disable-npcap], [msys2 only: Disable linkage against the wpcap/npcap import library in windows/WpdPack/Lib.]))
AS_IF([test "x$enable_fuzztargets" = "xyes"], [BUILD_FUZZTARGETS=1], [BUILD_FUZZTARGETS=0])
AM_CONDITIONAL([BUILD_FUZZTARGETS], [test "x$enable_fuzztargets" = "xyes"])
AS_IF([test "${with_sanitizer+set}" = set -a "${with_thread_sanitizer+set}" = set],[
AC_MSG_ERROR([Configure options `--with-sanitizer' and `--with_thread_sanitizer' can not used at the same time.])
])
AS_IF([test "${with_sanitizer+set}" = set -o "${with_thread_sanitizer+set}" = set],[
CFLAGS="${CFLAGS} -O0 -g3"
],[
CFLAGS="${CFLAGS} -g"
])
AS_IF([test "${with_sanitizer+set}" = set],[
CFLAGS="${CFLAGS} -fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fsanitize=leak -fno-omit-frame-pointer"
LDFLAGS="${LDFLAGS} -fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fsanitize=leak"
])
AS_IF([test "${with_thread_sanitizer+set}" = set],[
CFLAGS="${CFLAGS} -fsanitize=thread -fno-omit-frame-pointer"
LDFLAGS="${LDFLAGS} -fsanitize=thread"
])
AS_IF([test "x${enable_code_coverage}" = "xyes"],[
CFLAGS="${CFLAGS} -fprofile-arcs -ftest-coverage"
LDFLAGS="${LDFLAGS} --coverage"
])
AS_IF([test "${enable_npcap+set}" = set],[DISABLE_NPCAP=1],[DISABLE_NPCAP=0])
LT_INIT
LT_LIB_M
PKG_PROG_PKG_CONFIG
AC_PROG_CC
SYSTEM=`uname -s`
if test $SYSTEM = "Darwin"; then
dnl> AC_PROG_CC(clang gcc)
AM_PROG_CC_C_O(clang gcc)
AC_PROG_CXX(clang++ g++)
else
AC_PROG_CC_C_O
AC_PROG_CXX
fi
dnl> Can't iuse AM_PROG_AR because not all of our Makefiles are automake (yet?)
AC_CHECK_TOOL(AR, ar, [false])
AC_LANG_WERROR
NDPI_MAJOR=`echo "${PACKAGE_VERSION}" | cut -d . -f 1`
NDPI_MINOR=`echo "${PACKAGE_VERSION}" | cut -d . -f 2`
NDPI_PATCH=`echo "${PACKAGE_VERSION}" | cut -d . -f 3`
NDPI_VERSION_SHORT="$NDPI_MAJOR.$NDPI_MINOR.$NDPI_PATCH"
AC_DEFINE_UNQUOTED(NDPI_MAJOR_RELEASE, "${NDPI_MAJOR}", [nDPI major release])
AC_DEFINE_UNQUOTED(NDPI_MINOR_RELEASE, "${NDPI_MINOR}", [nDPI minor release])
AC_DEFINE_UNQUOTED(NDPI_PATCH_LEVEL, "${NDPI_PATCH}", [nDPI patch level])
# .git as directory in a cloned repo
# .git as file in submodule based integration
if test -d ".git" || test -r ".git"; then
GIT_TAG=`git log -1 --format=%h`
GIT_DATE=`git log -1 --format=%cd`
#
# On CentOS 6 `git rev-list HEAD --count` does not work
#
#
GIT_NUM=`git log --pretty=oneline | wc -l | tr -d '[[:space:]]'`
GIT_RELEASE="${PACKAGE_VERSION}-${GIT_NUM}-${GIT_TAG}"
A=`git log src/include/ndpi_typedefs.h|wc -l`
B=`git log src/include/ndpi_protocol_ids.h|wc -l`
C=`git log src/include/ndpi_api.h.in|wc -l`
NDPI_API_VERSION=$((A+B+C))
else
GIT_RELEASE="${PACKAGE_VERSION}"
GIT_DATE=`date -u -r CHANGELOG.md`
if test -z "$SOURCE_DATE_EPOCH" ; then :
SOURCE_DATE_EPOCH=`date +%s`
fi
NDPI_API_VERSION=$((SOURCE_DATE_EPOCH % 65536))
fi
AS_IF([test "x${NDPI_API_VERSION}" = "x"],[
AC_MSG_WARN([Could not get NDPI_API_VERSION, defaulting to 0.])
NDPI_API_VERSION=0
])
echo "Setting API version to ${NDPI_API_VERSION}"
AC_DEFINE_UNQUOTED(NDPI_GIT_RELEASE, "${GIT_RELEASE}", [GIT Release])
AC_DEFINE_UNQUOTED(NDPI_GIT_DATE, "${GIT_DATE}", [Last GIT change])
if ! test "${with_only_libndpi+set}" = set; then :
dnl> used by json-c for unit tests
PKG_CHECK_MODULES([JSONC], [json-c], [
AC_DEFINE(HAVE_LIBJSON_C, 1, [libjson-c is present])
JSONC_LIBS="${pkg_cv_JSONC_LIBS}"
JSONC_CFLAGS="${pkg_cv_JSONC_CFLAGS}"
], [AC_MSG_WARN([JSON-C not available. Disabled unit test.])])
AC_CHECK_LIB([json-c], [json_object_put], [
EXTRA_TARGETS="$EXTRA_TARGETS tests/unit"
build_unittests=yes
],[
AC_MSG_WARN([JSON-C not available. Disabled unit test.])
JSONC_LIBS=""
JSONC_CFLAGS=""
])
AX_PTHREAD([],[
AC_MSG_WARN([POSIX Threads not available. Building library only.])
JSONC_LIBS=""
JSONC_CFLAGS=""
EXTRA_TARGETS=""
])
fi
AM_CONDITIONAL([BUILD_UNITTESTS], [test "x$build_unittests" = "xyes"])
HANDLE_TLS_SIGS="//"
AS_IF([test "x${enable_tls_sigs}" = "xyes"],[
HANDLE_TLS_SIGS=""
])
CFLAGS="-W -Wall -Wno-unused-parameter -Wno-unused-function -Wno-address-of-packed-member ${CFLAGS}"
AC_CHECK_HEADERS([netinet/in.h stdint.h stdlib.h string.h unistd.h math.h float.h])
AC_CHECK_LIB([m], [sqrt], [], [LIBM="-lm"])
AC_CHECK_LIB([rrd], [rrd_fetch_r], [LIBRRD=-lrrd])
if test "x$ac_cv_lib_rrd_rrd_fetch_r" = xyes; then :
AC_DEFINE_UNQUOTED(HAVE_RRDTOOL, "1", [rrdtool is present])
EXTRA_TARGETS="$EXTRA_TARGETS rrdtool"
fi
ADDITIONAL_LIBS="$LIBM"
PCAP_HOME=$HOME/PF_RING/userland
DPDK_TARGET=
AC_MSG_CHECKING([DPDK (used by ndpiReader)])
if test -d $HOME/DPDK; then :
AC_MSG_RESULT(yes)
DPDK_TARGET=dpdk
else
AC_MSG_RESULT([no (missing $HOME/DPDK)])
fi
if ! test -d $PCAP_HOME; then :
PCAP_HOME=`pwd`/../../PF_RING/userland
fi
SHORT_MACHINE=`uname -m | cut -b1-3`
if test $SHORT_MACHINE = "arm"; then
LIBNUMA=""
else
AC_CHECK_LIB([numa], [numa_available], [LIBNUMA="-lnuma"])
fi
MACHINE=`uname -m`
CUSTOM_NDPI=
if test -d ../nDPI-custom; then :
CUSTOM_NDPI="-DCUSTOM_NDPI_PROTOCOLS"
AC_MSG_RESULT([Compiling with custom nDPI protocols])
fi
if test -d ../ndpi-pro; then :
with_maxminddb=1
fi
case "$host" in
*-*-mingw32*|*-*-msys)
PCAP_INC=""
PCAP_LIB=""
case "$host" in
x86_64-*)
BUILD_MINGW_X64=1
LDFLAGS="${LDFLAGS} -L/mingw64/lib"
;;
i686-*)
LDFLAGS="${LDFLAGS} -L/mingw32/lib"
;;
*)
AC_MSG_ERROR([Unsupported MingW cross compiler.])
;;
esac
CFLAGS="-D__USE_MINGW_ANSI_STDIO -D__STDC_FORMAT_MACROS ${CFLAGS}"
LIBS="${LIBS} -lws2_32"
BUILD_MINGW=1
EXE_SUFFIX=".exe"
AS_IF([test "${enable_npcap+set}" != set && test "${with_only_libndpi+set}" != set],,
[PKG_CHECK_MODULES([PCAP], [libpcap], [PCAP_LIB="" PCAP_INC="${pkg_cv_PCAP_CFLAGS}"])
AC_CHECK_LIB([pcap], [pcap_open_live],, [AC_MSG_ERROR([Missing msys2/mingw libpcap library. Install it with `pacman -S mingw-w64-x86_64-libpcap' (msys2).])])])
;;
*)
if test -f $PCAP_HOME/libpcap/libpcap.a; then :
echo "Using libpcap from $PCAP_HOME"
PCAP_INC="-I $PCAP_HOME/libpcap"
PFRING_LIB=
if test -f $PCAP_HOME/lib/libpfring.a; then :
PFRING_LIB=$PCAP_HOME/lib/libpfring.a
fi
PCAP_LIB="$PCAP_HOME/libpcap/libpcap.a $PFRING_LIB $LIBNUMA `$PCAP_HOME/lib/pfring_config --libs`"
AC_CHECK_LIB([rt], [clock_gettime], [PCAP_LIB="$PCAP_LIB -lrt"])
AC_CHECK_LIB([nl], [nl_handle_alloc], [PCAP_LIB="$PCAP_LIB -lnl"])
# The dlopen() function is in libdl on GLIBC-based systems
# and in the C library for *BSD systems
AC_CHECK_LIB([dl], [dlopen], [DL_LIB="-ldl"],[AC_CHECK_LIB([c],
[dlopen], [DL_LIB="-lc"],
[AC_MSG_ERROR([unable to find the dlopen(), dlsym() functions]) ]) ])
elif test "${with_only_libndpi+set}" != set; then :
AC_CHECK_LIB([pcap], [pcap_open_live], [PCAP_LIB="-lpcap"])
if test $ac_cv_lib_pcap_pcap_open_live = "no"; then :
AC_MSG_ERROR([Missing libpcap(-dev) library required to compile the example application.])
fi
fi
EXE_SUFFIX=""
AS_IF([test "${enable_npcap+set}" = set],AC_MSG_ERROR([--disable-npcap does only work with msys2/mingw CCs]))
;;
esac
AC_ARG_ENABLE([debug-messages],
AS_HELP_STRING([--enable-debug-messages], [Define NDPI_ENABLE_DEBUG_MESSAGES=1]), [
AC_DEFINE(NDPI_ENABLE_DEBUG_MESSAGES, 1, [Enable ndpi_debug_messages]) ])
AS_IF([test "x$enable_fuzztargets" = "xyes"], [
AC_PROG_CXX
AC_LANG_PUSH(C++)
tmp_saved_flags=$[]_AC_LANG_PREFIX[]FLAGS
AX_CHECK_COMPILE_FLAG([-fsanitize=fuzzer],,
[AC_MSG_ERROR([--enable-fuzztargets requires -fsanitize=fuzzer which is only supported by LLVM])],
[-Werror])
AS_IF([test "x$LIB_FUZZING_ENGINE" = "x"], [
LIB_FUZZING_ENGINE=-fsanitize=fuzzer
AC_SUBST(LIB_FUZZING_ENGINE)
])
_AC_LANG_PREFIX[]FLAGS="$[]_AC_LANG_PREFIX[]FLAGS $LIB_FUZZING_ENGINE"
AC_MSG_CHECKING([whether $CXX accepts $LIB_FUZZING_ENGINE])
AC_LINK_IFELSE([AC_LANG_SOURCE([[
#include <sys/types.h>
extern "C" int LLVMFuzzerTestOneInput(const unsigned char *Data, size_t Size);
extern "C" int LLVMFuzzerTestOneInput(const unsigned char *Data, size_t Size) {
(void)Data;
(void)Size;
return 0;
}
]])],
[ AC_MSG_RESULT(yes)
has_sanitizefuzzer=yes],
[ AC_MSG_RESULT(no) ]
)
_AC_LANG_PREFIX[]FLAGS=$tmp_saved_flags
AC_LANG_POP()
])
AM_CONDITIONAL([HAS_FUZZLDFLAGS], [test "x$has_sanitizefuzzer" = "xyes"])
GPROF_ENABLED=0
AS_IF([test "x$enable_gprof" = "xyes"], [
PKG_CHECK_MODULES([PROFILER], [libprofiler],,[
AC_MSG_ERROR([libprofiler not available. Required for profiling support.])
])
PKG_CHECK_MODULES([TCMALLOC], [libtcmalloc],,[
AC_MSG_ERROR([libtcmalloc not available. Required for profiling support.])
])
GPROF_ENABLED=1
GPROF_LIBS="-ltcmalloc_and_profiler"
GPROF_CFLAGS="${pkg_cv_PROFILER_CFLAGS} ${pkg_cv_TCMALLOC_CFLAGS}"
])
AC_CHECK_LIB(pthread, pthread_setaffinity_np, AC_DEFINE_UNQUOTED(HAVE_PTHREAD_SETAFFINITY_NP, 1, [libc has pthread_setaffinity_np]))
dnl> libgcrypt (external)
USE_HOST_LIBGCRYPT=0
AS_IF([test "${with_local_libgcrypt+set}" = set],[
USE_HOST_LIBGCRYPT=1
AC_CHECK_LIB(gpg-error, gpg_strerror_r, [], AC_MSG_ERROR([libgpg-error required (because of --with-local-libgcrypt) but not found or too old.]))
AC_CHECK_LIB(gcrypt, gcry_cipher_checktag, [], AC_MSG_ERROR([libgcrypt required (because of --with-local-libgcrypt) but not found or too old.]))
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lgcrypt -lgpg-error"
AC_DEFINE_UNQUOTED(USE_HOST_LIBGCRYPT, 1, [Use locally installed libgcrypt instead of builtin gcrypt-light])
])
dnl> PCRE
PCRE_ENABLED=0
AC_ARG_WITH(pcre, [ --with-pcre Enable nDPI build with libpcre])
if test "${with_pcre+set}" = set; then :
AC_CHECK_LIB(pcre, pcre_compile, AC_DEFINE_UNQUOTED(HAVE_PCRE, 1, [libpcre(-dev) is present]))
if test "x$ac_cv_lib_pcre_pcre_compile" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lpcre"
PCRE_ENABLED=1
fi
fi
dnl> GeoIP
AC_ARG_WITH(maxminddb, [ --with-maxminddb Enable nDPI build with libmaxminddb])
if test "${with_maxminddb+set}" = set; then :
AC_CHECK_LIB([maxminddb], [MMDB_lookup_sockaddr])
AC_CHECK_HEADERS([maxminddb.h])
if test ".${ac_cv_lib_maxminddb_MMDB_lookup_sockaddr}" = ".yes" && test ".${ac_cv_header_maxminddb_h}" = ".yes"; then
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lmaxminddb"
AC_DEFINE_UNQUOTED(HAVE_MAXMINDDB, 1, [MaxMind DB support])
fi
fi
dnl> Curl
dnl> AC_CHECK_LIB([curl], [curl_easy_perform])
dnl> if test ${ac_cv_lib_curl_curl_easy_perform} = "no"; then
dnl> echo "Please install libcurl4-openssl-dev (http://curl.haxx.se/) in order to enable curl support"
dnl> else
dnl> ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lcurl"
dnl> AC_DEFINE_UNQUOTED(HAVE_CURL, 1, [curl is present])
dnl> fi
AC_CONFIG_FILES([Makefile example/Makefile example/Makefile.dpdk tests/Makefile tests/unit/Makefile tests/performance/Makefile tests/dga/Makefile rrdtool/Makefile influxdb/Makefile libndpi.pc src/include/ndpi_define.h src/lib/Makefile fuzz/Makefile src/include/ndpi_api.h doc/Doxyfile.cfg])
AC_CONFIG_FILES([tests/do.sh], [chmod +x tests/do.sh])
AC_CONFIG_HEADERS(src/include/ndpi_config.h)
AC_SUBST(GIT_RELEASE)
AC_SUBST(NDPI_MAJOR)
AC_SUBST(NDPI_MINOR)
AC_SUBST(NDPI_PATCH)
AC_SUBST(NDPI_VERSION_SHORT)
AC_SUBST(PCAP_INC)
AC_SUBST(PCAP_LIB)
AC_SUBST(LIBRRD)
AC_SUBST(ADDITIONAL_LIBS)
AC_SUBST(DL_LIB)
AC_SUBST(DPDK_TARGET)
AC_SUBST(HAVE_PTHREAD_SETAFFINITY_NP)
AC_SUBST(CUSTOM_NDPI)
AC_SUBST(NDPI_API_VERSION)
AC_SUBST(EXTRA_TARGETS)
AC_SUBST(BUILD_MINGW)
AC_SUBST(BUILD_MINGW_X64)
AC_SUBST(BUILD_FUZZTARGETS)
AC_SUBST(JSONC_CFLAGS)
AC_SUBST(JSONC_LIBS)
AC_SUBST(GPROF_CFLAGS)
AC_SUBST(GPROF_LIBS)
AC_SUBST(GPROF_ENABLED)
AC_SUBST(USE_HOST_LIBGCRYPT)
AC_SUBST(PCRE_ENABLED)
AC_SUBST(HANDLE_TLS_SIGS)
AC_SUBST(DISABLE_NPCAP)
AC_SUBST(EXE_SUFFIX)
AC_OUTPUT
-264
View File
@@ -1,264 +0,0 @@
AC_INIT([libndpi], [@NDPI_VERSION_SHORT@])
AC_CONFIG_AUX_DIR([.])
AC_CONFIG_MACRO_DIR([m4])
AM_INIT_AUTOMAKE([foreign subdir-objects])
AC_PREFIX_DEFAULT(/usr)
EXTRA_TARGETS="example tests tests/unit tests/dga"
AC_ARG_WITH(only-libndpi, AS_HELP_STRING([--with-only-libndpi], [Build only libndpi (no examples, tests etc)]))
AS_IF([test "${with_only_libndpi+set}" = set],[
EXTRA_TARGETS=""
])
GNU_PREFIX=""
AC_ARG_WITH(mipsel, AS_HELP_STRING([--with-mipsel], [Build only libndpi (no examples, tests etc) for mipsel]))
AS_IF([test "${with_mipsel+set}" = set],[
EXTRA_TARGETS=""
GNU_PREFIX="mipsel-linux-gnu-"
])
AC_ARG_WITH(sanitizer, AS_HELP_STRING([--with-sanitizer], [Build with support for address, undefined and leak sanitizer]))
AC_ARG_ENABLE(fuzztargets, AS_HELP_STRING([--enable-fuzztargets], [Enable fuzz targets]),[enable_fuzztargets=$enableval],[enable_fuzztargets=no])
AS_IF([test "x$enable_fuzztargets" = "xyes"], [BUILD_FUZZTARGETS=1], [BUILD_FUZZTARGETS=0])
AM_CONDITIONAL([BUILD_FUZZTARGETS], [test "x$enable_fuzztargets" = "xyes"])
AS_IF([test "${with_sanitizer+set}" = set],[
CFLAGS="${CFLAGS} -g3 -O0 -Wno-unused-value -fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fsanitize=leak -fno-omit-frame-pointer"
LDFLAGS="${LDFLAGS} -fsanitize=address -fsanitize=undefined -fno-sanitize=alignment -fsanitize=leak"
])
LT_INIT
SYSTEM=`uname -s`
if test $SYSTEM = "Darwin"; then
dnl> AC_PROG_CC(clang gcc)
AM_PROG_CC_C_O(clang gcc)
AC_PROG_CXX(clang++ g++)
AC_PROG_CC_STDC(clang gcc)
else
dnl> AC_PROG_CC
AM_PROG_CC_C_O
AC_PROG_CXX
AC_PROG_CC_STDC
fi
AC_LANG_WERROR
AX_PTHREAD
NDPI_VERSION_SHORT="@NDPI_VERSION_SHORT@"
NDPI_MAJOR="@NDPI_MAJOR@"
NDPI_MINOR="@NDPI_MINOR@"
NDPI_PATCH="@NDPI_PATCH@"
AC_DEFINE_UNQUOTED(NDPI_MAJOR_RELEASE, "${NDPI_MAJOR}", [nDPI major release])
AC_DEFINE_UNQUOTED(NDPI_MINOR_RELEASE, "${NDPI_MINOR}", [nDPI minor release])
AC_DEFINE_UNQUOTED(NDPI_PATCH_LEVEL, "${NDPI_PATCH}", [nDPI patch level])
# .git as directory in a cloned repo
# .git as file in submodule based integration
if test -d ".git" || test -f ".git" ; then :
GIT_TAG=`git log -1 --format=%h`
GIT_DATE=`git log -1 --format=%cd`
#
# On CentOS 6 `git rev-list HEAD --count` does not work
#
#
GIT_NUM=`git log --pretty=oneline | wc -l | tr -d '[[:space:]]'`
GIT_RELEASE="${PACKAGE_VERSION}-${GIT_NUM}-${GIT_TAG}"
A=`git log src/include/ndpi_typedefs.h|wc -l`
B=`git log src/include/ndpi_protocol_ids.h|wc -l`
C=`git log src/include/ndpi_api.h.in|wc -l`
NDPI_API_VERSION=$((A+B+C))
else
GIT_RELEASE="${PACKAGE_VERSION}"
GIT_DATE=`date -u -r CHANGELOG.md`
if test -z "$SOURCE_DATE_EPOCH" ; then :
SOURCE_DATE_EPOCH=`date +%s`
fi
NDPI_API_VERSION=`echo $SOURCE_DATE_EPOCH | cut -c7-10`
fi
NDPI_API_VERSION=`echo $NDPI_API_VERSION | sed 's/^0*//'`
AC_DEFINE_UNQUOTED(NDPI_GIT_RELEASE, "${GIT_RELEASE}", [GIT Release])
AC_DEFINE_UNQUOTED(NDPI_GIT_DATE, "${GIT_DATE}", [Last GIT change])
if ! test "${with_mipsel+set}" = set && ! test "${with_only_libndpi+set}" = set; then :
dnl> used by json-c for unit tests
PKG_CHECK_MODULES([JSONC], [json-c], [JSONC_LIBS=`pkg-config --libs json-c` JSONC_CFLAGS=`pkg-config --cflags json-c`])
fi
LIBS="$LIBS $JSONC_LIBS"
CFLAGS="$CFLAGS $JSONC_CFLAGS"
AC_CHECK_HEADERS([netinet/in.h stdint.h stdlib.h string.h unistd.h json.h])
ADDITIONAL_LIBS=
PCAP_HOME=$HOME/PF_RING/userland
DPDK_TARGET=
AC_MSG_CHECKING([DPDK (used by ndpiReader)])
if test -d $HOME/DPDK; then :
AC_MSG_RESULT(yes)
DPDK_TARGET=dpdk
else
AC_MSG_RESULT([no (missing $HOME/DPDK)])
fi
if ! test -d $PCAP_HOME; then :
PCAP_HOME=`pwd`/../../PF_RING/userland
fi
SHORT_MACHINE=`uname -m | cut -b1-3`
if test $SHORT_MACHINE = "arm"; then
LIBNUMA=""
else
AC_CHECK_LIB([numa], [numa_available], [LIBNUMA="-lnuma"])
fi
MACHINE=`uname -m`
CUSTOM_NDPI=
if test -d ../nDPI-custom; then :
CUSTOM_NDPI="-DCUSTOM_NDPI_PROTOCOLS"
AC_MSG_RESULT([Compiling with custom nDPI protocols])
fi
if test -d ../ndpi-pro; then :
with_maxminddb=1
fi
case "$host" in
*-*-mingw32*|*-*-msys)
CFLAGS="${CFLAGS} -DOS_WIN32"
LDFLAGS="${LDFLAGS} -lws2_32 -lucrtbase"
BUILD_MINGW=1
;;
*)
if test -f $PCAP_HOME/libpcap/libpcap.a; then :
echo "Using libpcap from $PCAP_HOME"
PCAP_INC="-I $PCAP_HOME/libpcap"
PCAP_LIB="$PCAP_HOME/libpcap/libpcap.a $PCAP_HOME/lib/libpfring.a $LIBNUMA `$PCAP_HOME/lib/pfring_config --libs`"
AC_CHECK_LIB([rt], [clock_gettime], [PCAP_LIB="$PCAP_LIB -lrt"])
AC_CHECK_LIB([nl], [nl_handle_alloc], [PCAP_LIB="$PCAP_LIB -lnl"])
# The dlopen() function is in libdl on GLIBC-based systems
# and in the C library for *BSD systems
AC_CHECK_LIB([dl], [dlopen, dlsym], [DL_LIB="-ldl"],[AC_CHECK_LIB([c], [dlopen, dlsym], [DL_LIB="-lc"],[AC_MSG_ERROR([unable to find the dlopen(), dlsym() functions]) ]) ])
else
AC_CHECK_LIB([pcap], [pcap_open_live], [PCAP_LIB="-lpcap"])
if test $ac_cv_lib_pcap_pcap_open_live = "no"; then :
echo ""
echo "ERROR: Missing libpcap(-dev) library required to compile the example application"
echo "ERROR: Please install it and try again"
exit
fi
fi
;;
esac
AC_ARG_ENABLE([debug-messages],
AS_HELP_STRING([--enable-debug-messages], [Define NDPI_ENABLE_DEBUG_MESSAGES=1]), [
AC_DEFINE(NDPI_ENABLE_DEBUG_MESSAGES, 1, [Enable ndpi_debug_messages]) ])
@FUZZY@ AS_IF([test "x$enable_fuzztargets" = "xyes"], [
@FUZZY@ AC_PROG_CXX
@FUZZY@ AC_LANG_PUSH(C++)
@FUZZY@ tmp_saved_flags=$[]_AC_LANG_PREFIX[]FLAGS
@FUZZY@ AX_CHECK_COMPILE_FLAG([-fsanitize=fuzzer],,
@FUZZY@ [AC_MSG_ERROR([--enable-fuzztargets requires -fsanitize=fuzzer which is only supported by LLVM])],
@FUZZY@ [-Werror])
@FUZZY@ AS_IF([test "x$LIB_FUZZING_ENGINE" = "x"], [
@FUZZY@ LIB_FUZZING_ENGINE=-fsanitize=fuzzer
@FUZZY@ AC_SUBST(LIB_FUZZING_ENGINE)
@FUZZY@ ])
@FUZZY@ _AC_LANG_PREFIX[]FLAGS="$[]_AC_LANG_PREFIX[]FLAGS $LIB_FUZZING_ENGINE"
@FUZZY@ AC_MSG_CHECKING([whether $CXX accepts $LIB_FUZZING_ENGINE])
@FUZZY@ AC_LINK_IFELSE([AC_LANG_SOURCE([[
@FUZZY@ #include <sys/types.h>
@FUZZY@ extern "C" int LLVMFuzzerTestOneInput(const unsigned char *Data, size_t Size);
@FUZZY@ extern "C" int LLVMFuzzerTestOneInput(const unsigned char *Data, size_t Size) {
@FUZZY@ (void)Data;
@FUZZY@ (void)Size;
@FUZZY@ return 0;
@FUZZY@ }
@FUZZY@ ]])],
@FUZZY@ [ AC_MSG_RESULT(yes)
@FUZZY@ has_sanitizefuzzer=yes],
@FUZZY@ [ AC_MSG_RESULT(no) ]
@FUZZY@ )
@FUZZY@ _AC_LANG_PREFIX[]FLAGS=$tmp_saved_flags
@FUZZY@ AC_LANG_POP()
@FUZZY@ ])
AM_CONDITIONAL([HAS_FUZZLDFLAGS], [test "x$has_sanitizefuzzer" = "xyes"])
AC_CHECK_LIB(pthread, pthread_setaffinity_np, AC_DEFINE_UNQUOTED(HAVE_PTHREAD_SETAFFINITY_NP, 1, [libc has pthread_setaffinity_np]))
dnl> GCRYPT
GCRYPT_ENABLED=1
AC_ARG_ENABLE([gcrypt],
[AS_HELP_STRING([--disable-gcrypt], [Avoid compiling with libgcrypt/libgpg-error, even if they are present. QUIC sub-classification may be missing])],
[GCRYPT_ENABLED=0],
[AC_CHECK_LIB(gcrypt, gcry_cipher_checktag)])
AS_IF([test "x$enable_gcrypt" = "xyes"], [
if test "x$ac_cv_lib_gcrypt_gcry_cipher_checktag" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lgcrypt"
else
$as_unset ac_cv_lib_gcrypt_gcry_cipher_checktag
AC_CHECK_LIB(gpg-error, gpg_strerror_r)
AC_CHECK_LIB(gcrypt, gcry_cipher_checktag)
if test "x$ac_cv_lib_gcrypt_gcry_cipher_checktag" = xyes -a "x$ac_cv_lib_gpg_error_gpg_strerror_r" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lgcrypt -lgpg-error"
fi
fi
])
dnl> PCRE
AC_ARG_WITH(pcre, [ --with-pcre Enable nDPI build with libpcre])
if test "${with_pcre+set}" = set; then :
AC_CHECK_LIB(pcre, pcre_compile, AC_DEFINE_UNQUOTED(HAVE_PCRE, 1, [libpcre(-dev) is present]))
if test "x$ac_cv_lib_pcre_pcre_compile" = xyes; then :
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lpcre"
fi
fi
dnl> GeoIP
AC_ARG_WITH(maxminddb, [ --with-maxminddb Enable nDPI build with libmaxminddb])
if test "${with_maxminddb+set}" = set; then :
AC_CHECK_LIB([maxminddb], [MMDB_lookup_sockaddr])
AC_HAVE_HEADERS(maxminddb.h)
if test ".${ac_cv_lib_maxminddb_MMDB_lookup_sockaddr}" = ".yes" && test ".${ac_cv_header_maxminddb_h}" = ".yes"; then
ADDITIONAL_LIBS="${ADDITIONAL_LIBS} -lmaxminddb"
AC_DEFINE_UNQUOTED(HAVE_MAXMINDDB, 1, [MaxMind DB support])
fi
fi
AC_CONFIG_FILES([Makefile example/Makefile example/Makefile.dpdk tests/Makefile tests/unit/Makefile tests/dga/Makefile libndpi.pc src/include/ndpi_define.h src/lib/Makefile python/Makefile fuzz/Makefile src/include/ndpi_api.h])
AC_CONFIG_FILES([tests/do.sh], [chmod +x tests/do.sh])
AC_CONFIG_FILES([tests/do_valgrind.sh], [chmod +x tests/do_valgrind.sh])
AC_CONFIG_HEADERS(src/include/ndpi_config.h)
AC_SUBST(GIT_RELEASE)
AC_SUBST(NDPI_MAJOR)
AC_SUBST(NDPI_MINOR)
AC_SUBST(NDPI_PATCH)
AC_SUBST(NDPI_VERSION_SHORT)
AC_SUBST(PCAP_INC)
AC_SUBST(PCAP_LIB)
AC_SUBST(ADDITIONAL_LIBS)
AC_SUBST(DL_LIB)
AC_SUBST(DPDK_TARGET)
AC_SUBST(HAVE_PTHREAD_SETAFFINITY_NP)
AC_SUBST(CUSTOM_NDPI)
AC_SUBST(NDPI_API_VERSION)
AC_SUBST(EXTRA_TARGETS)
AC_SUBST(BUILD_MINGW)
AC_SUBST(BUILD_FUZZTARGETS)
AC_SUBST(JSONC_CFLAGS)
AC_SUBST(JSONC_LIBS)
AC_SUBST(GCRYPT_ENABLED)
AC_SUBST(GNU_PREFIX)
AC_SUBST(FRAG_MAN_ENABLED)
AC_OUTPUT
+2658
View File
File diff suppressed because it is too large Load Diff
+234
View File
@@ -0,0 +1,234 @@
# Makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
# User-friendly check for sphinx-build
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
endif
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " release to make standalone Doxygen/HTML files for ntop documentation release"
@echo " html to make standalone Doxygen/HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " singlehtml to make a single large HTML file"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " qthelp to make HTML files and a qthelp project"
@echo " applehelp to make an Apple Help Book"
@echo " devhelp to make HTML files and a Devhelp project"
@echo " epub to make an epub"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " latexpdf to make LaTeX files and run them through pdflatex"
@echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
@echo " text to make text files"
@echo " man to make manual pages"
@echo " texinfo to make Texinfo files"
@echo " info to make Texinfo files and run them through makeinfo"
@echo " gettext to make PO message catalogs"
@echo " changes to make an overview of all changed/added/deprecated items"
@echo " xml to make Docutils-native XML files"
@echo " pseudoxml to make pseudoxml-XML files for display purposes"
@echo " linkcheck to check all external links for integrity"
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
@echo " coverage to run coverage check of the documentation (if enabled)"
.PHONY: clean
clean:
rm -rf $(BUILDDIR)/* api doxygen
.PHONY: html
html:
@test -r Doxyfile.cfg || { \
echo 'Doxyfile.cfg does not exist; Please run $(realpath ..)/autogen.sh or $(realpath ..)/configure first.'; \
false; \
}
doxygen Doxyfile.cfg
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
.PHONY: release
release:
make clean
@{ \
git_root=$$(git rev-parse --show-toplevel);\
version=$$(grep -Po '^NTOPNG_SHORT_VERSION="\K.+(?=")' $${git_root}/configure.ac);\
echo Release version $${version};\
sed -i "s/version = u'.*'/version = u'$${version}'/" conf.py;\
sed -i "s/release = u'.*'/release = u'$${version}'/" conf.py;\
}
make html
.PHONY: dirhtml
dirhtml:
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
.PHONY: singlehtml
singlehtml:
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
@echo
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
.PHONY: pickle
pickle:
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
@echo
@echo "Build finished; now you can process the pickle files."
.PHONY: json
json:
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
@echo
@echo "Build finished; now you can process the JSON files."
.PHONY: htmlhelp
htmlhelp:
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
@echo
@echo "Build finished; now you can run HTML Help Workshop with the" \
".hhp project file in $(BUILDDIR)/htmlhelp."
.PHONY: qthelp
qthelp:
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
@echo
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/ntop.qhcp"
@echo "To view the help file:"
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/ntop.qhc"
.PHONY: applehelp
applehelp:
$(SPHINXBUILD) -b applehelp $(ALLSPHINXOPTS) $(BUILDDIR)/applehelp
@echo
@echo "Build finished. The help book is in $(BUILDDIR)/applehelp."
@echo "N.B. You won't be able to view it unless you put it in" \
"~/Library/Documentation/Help or install it in your application" \
"bundle."
.PHONY: devhelp
devhelp:
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
@echo
@echo "Build finished."
@echo "To view the help file:"
@echo "# mkdir -p $$HOME/.local/share/devhelp/ntop"
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/ntop"
@echo "# devhelp"
.PHONY: epub
epub:
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
@echo
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
.PHONY: latex
latex:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
@echo "Run \`make' in that directory to run these through (pdf)latex" \
"(use \`make latexpdf' here to do that automatically)."
.PHONY: latexpdf
latexpdf:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through pdflatex..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
.PHONY: latexpdfja
latexpdfja:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through platex and dvipdfmx..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
.PHONY: text
text:
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
@echo
@echo "Build finished. The text files are in $(BUILDDIR)/text."
.PHONY: man
man:
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
@echo
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
.PHONY: texinfo
texinfo:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
@echo "Run \`make' in that directory to run these through makeinfo" \
"(use \`make info' here to do that automatically)."
.PHONY: info
info:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo "Running Texinfo files through makeinfo..."
make -C $(BUILDDIR)/texinfo info
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
.PHONY: gettext
gettext:
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
@echo
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
.PHONY: changes
changes:
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
@echo
@echo "The overview file is in $(BUILDDIR)/changes."
.PHONY: linkcheck
linkcheck:
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or in $(BUILDDIR)/linkcheck/output.txt."
.PHONY: doctest
doctest:
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
@echo "Testing of doctests in the sources finished, look at the " \
"results in $(BUILDDIR)/doctest/output.txt."
.PHONY: coverage
coverage:
$(SPHINXBUILD) -b coverage $(ALLSPHINXOPTS) $(BUILDDIR)/coverage
@echo "Testing of coverage in the sources finished, look at the " \
"results in $(BUILDDIR)/coverage/python.txt."
.PHONY: xml
xml:
$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
@echo
@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
.PHONY: pseudoxml
pseudoxml:
$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
@echo
@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."
+304
View File
@@ -0,0 +1,304 @@
# -*- coding: utf-8 -*-
import sys
import os
import sphinx_rtd_theme
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#sys.path.insert(0, os.path.abspath('.'))
sys.path.append( "breathe/" )
# -- General configuration ------------------------------------------------
# If documentation needs a minimal Sphinx version, state it here.
#needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.intersphinx',
'breathe',
'exhale'
]
# Workaround for platforms where swaggerdoc is not available
if not os.environ.get("PYTHON_SKIP_SWAGGERDOC"):
extensions.append('sphinxcontrib.swaggerdoc')
breathe_projects = { "apidoc" : "doxygen/xml/" }
breathe_default_project = "apidoc"
exhale_args = {
"containmentFolder": "./api",
"rootFileName": "library_root.rst",
"rootFileTitle": "Library API",
"doxygenStripFromPath": ".",
"createTreeView": True,
# "treeViewIsBootstrap": True,
}
primary_domain = 'c'
highlight_language = 'c'
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
source_suffix = ['.rst', '.md']
#source_suffix = '.rst'
# The encoding of source files.
#source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'nDPI'
copyright = u'2021, ntop and contributors'
author = u'ntop.org'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = u'4.1'
# The full version, including alpha/beta/rc tags.
release = u'4.1'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ['_build', 'README.md', 'example.rst']
# The reST default role (used for this markup: `text`) to use for all
# documents.
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
#add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
#show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
#modindex_common_prefix = []
# If true, keep warnings as "system message" paragraphs in the built documents.
#keep_warnings = False
# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = False
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#html_theme = 'alabaster'
html_theme = "sphinx_rtd_theme"
#html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#html_theme_options = {}
# Add any paths that contain custom themes here, relative to this directory.
#html_theme_path = []
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
#html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
html_logo = "img/logo.png"
# The name of an image file (relative to this directory) to use as a favicon of
# the docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
#html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
#html_static_path = ['_static']
# Add any extra paths that contain custom files (such as robots.txt or
# .htaccess) here, relative to this directory. These files are copied
# directly to the root of the documentation.
#html_extra_path = []
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
#html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}
# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# If false, no module index is generated.
#html_domain_indices = True
# If false, no index is generated.
#html_use_index = True
# If true, the index is split into individual pages for each letter.
#html_split_index = False
# If true, links to the reST sources are added to the pages.
#html_show_sourcelink = True
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
#html_show_sphinx = True
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
#html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = None
# Language to be used for generating the HTML full-text search index.
# Sphinx supports the following languages:
# 'da', 'de', 'en', 'es', 'fi', 'fr', 'hu', 'it', 'ja'
# 'nl', 'no', 'pt', 'ro', 'ru', 'sv', 'tr'
#html_search_language = 'en'
# A dictionary with options for the search language support, empty by default.
# Now only 'ja' uses this config value
#html_search_options = {'type': 'default'}
# The name of a javascript file (relative to the configuration directory) that
# implements a search results scorer. If empty, the default will be used.
#html_search_scorer = 'scorer.js'
# Output file base name for HTML help builder.
htmlhelp_basename = 'ntopdoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
# Latex figure (float) alignment
#'figure_align': 'htbp',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'nDPI.tex', u'nDPI Documentation',
u'ntop', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
#latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# If true, show page references after internal links.
#latex_show_pagerefs = False
# If true, show URL addresses after external links.
#latex_show_urls = False
# Documents to append as an appendix to all manuals.
#latex_appendices = []
# If false, no module index is generated.
#latex_domain_indices = True
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'nDPI', u'nDPI Documentation',
[author], 1)
]
# If true, show URL addresses after external links.
#man_show_urls = False
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'nDPI', u'nDPI Documentation',
author, 'ntop and contributors', 'Deep Packet Inspection toolkit',
'Miscellaneous'),
]
# Documents to append as an appendix to all manuals.
#texinfo_appendices = []
# If false, no module index is generated.
#texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
#texinfo_show_urls = 'footnote'
# If true, do not generate a @detailmenu in the "Top" node's menu.
#texinfo_no_detailmenu = False
intersphinx_mapping = {'https://docs.python.org/': None}
def setup(app):
app.add_css_file('css/ntop.css')
+287
View File
@@ -0,0 +1,287 @@
nDPI Flow Risks
###############
nDPI is designed not just to detect application protocols in traffic flows but also to evaluate potential security risks associated with the traffic. In nDPI parlance this is called a "flow risk". Flows can have multiple risks detected hence nDPI reports them with a bitmap. Each risk detected corresponds to a bit in the flow risk bitmap. You can read more about `ndpi_risk_enum <https://github.com/ntop/nDPI/blob/dev/src/include/ndpi_typedefs.h>`_ for the list of all numeric risks currently supported.
Below you can find a description of each flow risk so that you can easily understand when a risk is triggered and its meaning. The flow risks are listed in numerical order as they are defined in ndpi_risk_enum.
.. _Risk 001:
NDPI_URL_POSSIBLE_XSS
=====================
HTTP only: this risk indicates a possible `XSS (Cross Side Scripting) <https://en.wikipedia.org/wiki/Cross-site_scripting>`_ attack.
.. _Risk 002:
NDPI_URL_POSSIBLE_SQL_INJECTION
===============================
HTTP only: this risk indicates a possible `SQL Injection attack <https://en.wikipedia.org/wiki/SQL_injection>`_.
.. _Risk 003:
NDPI_URL_POSSIBLE_RCE_INJECTION
===============================
HTTP only: this risk indicates a possible `RCE (Remote Code Execution) attack <https://en.wikipedia.org/wiki/Arbitrary_code_execution>`_.
.. _Risk 004:
NDPI_BINARY_APPLICATION_TRANSFER
================================
HTTP only: this risk indicates that a binary application is downloaded/uploaded. Detected applications include Windows binaries, Linux executables, Unix scripts and Android apps.
.. _Risk 005:
NDPI_KNOWN_PROTOCOL_ON_NON_STANDARD_PORT
========================================
This risk indicates a known protocol used on a non standard port. Example HTTP is supposed to use TCP/80, and in case it is detected on TCP/1234 this risk is detected.
.. _Risk 006:
NDPI_TLS_SELFSIGNED_CERTIFICATE
===============================
TLS/QUIC only: this risk is triggered when a `self-signed certificate <https://en.wikipedia.org/wiki/Self-signed_certificate>`_ is used.
.. _Risk 007:
NDPI_TLS_OBSOLETE_VERSION
=========================
Risk triggered when TLS version is older than 1.1.
.. _Risk 008:
NDPI_TLS_WEAK_CIPHER
====================
Risk triggered when an unsafe TLS cipher is used. See `this page <https://community.qualys.com/thread/18212-how-does-qualys-determine-the-server-cipher-suites>`_ for a list of insecure ciphers.
.. _Risk 009:
NDPI_TLS_CERTIFICATE_EXPIRED
============================
Risk triggered when a TLS certificate is expired, i.e. the current date falls outside of the certificate validity dates.
.. _Risk 010:
NDPI_TLS_CERTIFICATE_MISMATCH
=============================
Risk triggered when a TLS certificate does not match the hostname we're accessing. Example you do http://www.aaa.com and the TLS certificate returned is for www.bbb.com.
.. _Risk 011:
NDPI_HTTP_SUSPICIOUS_USER_AGENT
===============================
HTTP only: this risk is triggered whenever the user agent contains suspicious characters or its format is suspicious. Example: <?php something ?> is a typical suspicious user agent.
.. _Risk 012:
NDPI_HTTP_NUMERIC_IP_HOST
=========================
HTTP only: this risk is triggered whenever we're accessing a host using its IP rather than its symbolic name. Example http://1.2.3.4.
.. _Risk 013:
NDPI_HTTP_SUSPICIOUS_URL
========================
HTTP only: this risk is triggered whenever the accessed URL is suspicious. Example: http://127.0.0.1/msadc/..%255c../..%255c../..%255c../winnt/system32/cmd.exe.
.. _Risk 014:
NDPI_HTTP_SUSPICIOUS_HEADER
===========================
HTTP only: this risk is triggered whenever the HTTP peader contains suspicious entries such as Uuid, TLS_version, Osname that are unexpected on the HTTP header.
.. _Risk 015:
NDPI_TLS_NOT_CARRYING_HTTPS
===========================
TLS only: this risk indicates that this TLS flow will not be used to transport HTTP content. Example VPNs use TLS to encrypt data rather to carry HTTP. This is useful to spot this type of cases.
.. _Risk 016:
NDPI_SUSPICIOUS_DGA_DOMAIN
==========================
A `DGA <https://en.wikipedia.org/wiki/Domain_generation_algorithm>`_ is used to generate domain names often used by malwares. This risk indicates that this domain name can (but it's not 100% sure) a DGA as its name is suspicious.
.. _Risk 017:
NDPI_MALFORMED_PACKET
=====================
This risk is generated when a packet (e.g. a DNS packet) has an unexpected formt. This can indicate a protocol error or more often an attempt to jeopardize a valid protocol to carry other type of data.
.. _Risk 018:
NDPI_SSH_OBSOLETE_CLIENT_VERSION_OR_CIPHER
==========================================
This risk is generated whenever a SSH client uses an obsolete SSH protocol version or insecure ciphers.
.. _Risk 019:
NDPI_SSH_OBSOLETE_SERVER_VERSION_OR_CIPHER
==========================================
This risk is generated whenever a SSH server uses an obsolete SSH protocol version or insecure ciphers.
.. _Risk 020:
NDPI_SMB_INSECURE_VERSION
=========================
This risk indicates that the `SMB <https://en.wikipedia.org/wiki/Server_Message_Block>`_ version used is insecure (i.e. v1).
.. _Risk 021:
NDPI_TLS_SUSPICIOUS_ESNI_USAGE
==============================
`SNI <https://en.wikipedia.org/wiki/Server_Name_Indication>`_ is a way to carry in TLS the host/domain name we're accessing. ESNI means encrypted SNI and it is a way to mask SNI (carried in clear text in the TLS header) with encryption. While this practice is legal, it could be used for hiding data or for attacks such as a suspicious `domain fronting <https://github.com/SixGenInc/Noctilucent/blob/master/docs/>`_.
.. _Risk 022:
NDPI_UNSAFE_PROTOCOL
====================
This risk indicates that the protocol used is insecure and that a secure protocol should be used (e.g. Telnet vs SSH).
.. _Risk 023:
NDPI_DNS_SUSPICIOUS_TRAFFIC
===========================
This risk is returned when DNS traffic returns an unexpected/obsolete `record type <https://en.wikipedia.org/wiki/List_of_DNS_record_types>`_.
.. _Risk 024:
NDPI_TLS_MISSING_SNI
====================
TLS needs to carry the the `SNI <https://en.wikipedia.org/wiki/Server_Name_Indication>`_ of the remote server we're accessing. Unfortunately SNI is optional in TLS so it can be omitted. In this case this risk is triggered as this is a non-standard situation that indicates a potential security problem or a protocol using TLS for other purposes (or a protocol bug).
.. _Risk 025:
NDPI_HTTP_SUSPICIOUS_CONTENT
============================
HTTP only: risk reported when HTTP carries content in expected format. Example the HTTP header indicates that the context is text/html but the real content is not readeable (i.e. it can transport binary data). In general this is an attempt to use a valid MIME type to carry data that does not match the type.
.. _Risk 026:
NDPI_RISKY_ASN
==============
This is a placeholder for traffic exchanged with `ASN <https://en.wikipedia.org/wiki/Autonomous_system_(Internet)>`_ that are considered risky. nDPI does not fill this risk that instead should be filled by aplications sitting on top of nDPI (e.g. ntopng).
.. _Risk 027:
NDPI_RISKY_DOMAIN
=================
This is a placeholder for traffic exchanged with domain names that are considered risky. nDPI does not fill this risk that instead should be filled by aplications sitting on top of nDPI (e.g. ntopng).
.. _Risk 028:
NDPI_MALICIOUS_JA3
==================
`JA3 <https://engineering.salesforce.com/tls-fingerprinting-with-ja3-and-ja3s-247362855967>`_ is a method to fingerprint TLS traffic. This risk indicates that the JA3 of the TLS connection is considered suspicious (i.e. it has been found in known malware JA3 blacklists). nDPI does not fill this risk that instead should be filled by aplications sitting on top of nDPI (e.g. ntopng).
.. _Risk 029:
NDPI_MALICIOUS_SHA1_CERTIFICATE
===============================
TLS certificates are uniquely identified with a `SHA1 <https://en.wikipedia.org/wiki/SHA-1>`_ hash value. If such hash is found on a blacklist, this risk can be used. As for other risks, this is a placeholder as nDPI does not fill this risk that instead should be filled by aplications sitting on top of nDPI (e.g. ntopng).
.. _Risk 030:
NDPI_DESKTOP_OR_FILE_SHARING_SESSION
====================================
This risk is set when the flow carries desktop or file sharing sessions (e.g. TeamViewer or AnyDesk just to mention two).
.. _Risk 031:
NDPI_TLS_UNCOMMON_ALPN
======================
This risk is set when the `ALPN <https://en.wikipedia.org/wiki/Application-Layer_Protocol_Negotiation>`_ (it indicates the protocol carried into this TLS flow, for instance HTTP/1.1) is uncommon with respect to the list of expected values.
.. _Risk 032:
NDPI_TLS_CERT_VALIDITY_TOO_LONG
===============================
From 01/09/2020 TLS certificates lifespan is limited to `13 months <https://www.appviewx.com/blogs/tls-certificate-lifespans-now-capped-at-13-months/>`_. This risk is triggered for certificates not respecting this directive.
.. _Risk 033:
NDPI_TLS_SUSPICIOUS_EXTENSION
=============================
This risk is triggered when the domain name (SNI extension) is not printable and thus it is a problem. In TLS extensions can be dynamically specified by the client in the hello packet.
.. _Risk 034:
NDPI_TLS_FATAL_ALERT
====================
This risk is triggered when a TLS fatal alert is detected in the TLS flow. See `this page <https://techcommunity.microsoft.com/t5/iis-support-blog/ssl-tls-alert-protocol-and-the-alert-codes/ba-p/377132>`_ for details.
.. _Risk 035:
NDPI_SUSPICIOUS_ENTROPY
=======================
This risk is used to detect suspicious data carried in ICMP packets whose entropy (used to measure how data is distributed, hence to indirectly guess the type of data carried on) is suspicious and thus that it can indicate a data leak.
.. _Risk 036:
NDPI_CLEAR_TEXT_CREDENTIALS
===========================
Clear text protocols are not bad per-se, but they should be avoided when they carry credentials as they can be intercepted by malicious users. This risk is triggered whenever clear text protocols (e.g. FTP, HTTP, IMAP...) contain credentials in clear text (read it as nDPI does not trigger this risk for HTTP connections that do not carry credentials).
.. _Risk 037:
NDPI_DNS_LARGE_PACKET
=====================
`DNS <https://en.wikipedia.org/wiki/Domain_Name_System>`_ packets over UDP should be limited to 512 bytes. DNS packets over this threshold indicate a potential security risk (e.g. use DNS to carry data) or a misconfiguration.
.. _Risk 038:
NDPI_DNS_FRAGMENTED
===================
UDP `DNS <https://en.wikipedia.org/wiki/Domain_Name_System>`_ packets cannot be fragmented. If so, this indicates a potential security risk (e.g. use DNS to carry data) or a misconfiguration.
.. _Risk 039:
NDPI_INVALID_CHARACTERS
=======================
The risk is set whenever a dissected protocol contains characters not allowed in that protocol field.
For example a DNS hostname must only contain a subset of all printable characters or else this risk is set.
Additionally, some TLS protocol fields are checked for printable characters as well.
.. _Risk 040:
NDPI_POSSIBLE_EXPLOIT
=====================
The risk is set whenever a possible exploit (e.g. `Log4J/Log4Shell <https://en.wikipedia.org/wiki/Log4Shell>`_) is detected.
.. _Risk 041:
NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE
===================================
The risk is set whenever a TLS certificate is close to the expiration date.
.. _Risk 042:
NDPI_PUNYCODE_IDN
===================================
The risk is set whenever a domain name is specified in IDN format as they are sometimes used in `IDN homograph attacks <https://en.wikipedia.org/wiki/IDN_homograph_attack>`_.
.. _Risk 043:
NDPI_ERROR_CODE_DETECTED
===================================
The risk is set whenever an error code is detected in the underlying protocol (e.g. HTTP and DNS).
.. _Risk 044:
NDPI_HTTP_CRAWLER_BOT
===================================
The risk is set whenever a crawler/bot/robot has been detected
.. _Risk 045:
NDPI_ANONYMOUS_SUBSCRIBER
===================================
The risk is set whenever the (source) ip address has been anonymized and it can't be used to identify the subscriber.
Example: the flow is generated by an iCloud-private-relay exit node.
.. _Risk 046:
NDPI_UNIDIRECTIONAL_TRAFFIC_SUBSCRIBER
===================================
The risk is set whenever the flow has unidirectional traffic (typically no traffic on the server to client direction). THis
risk is not triggered for multicast/broadcast destinations.
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 47 KiB

+37
View File
@@ -0,0 +1,37 @@
nDPI Documentation
====================
nDPI is an open source DPI (Deep Packet Inspection) toolkit for traffic analysis.
.. toctree::
:maxdepth: 2
:caption: User's Guide
what_is_ndpi
.. toctree::
:maxdepth: 2
:caption: Developer's Guide
api/library_root
flow_risks
.. toctree::
:caption: Other Products
ntopng <https://www.ntop.org/guides/ntopng/>
nProbe <https://www.ntop.org/guides/nprobe/>
nProbe Cento <https://www.ntop.org/guides/cento/>
n2disk <https://www.ntop.org/guides/n2disk/>
PF_RING <https://www.ntop.org/guides/pf_ring/>
nEdge <https://www.ntop.org/guides/nedge/>
nScrub <https://www.ntop.org/guides/nscrub/>
nBox <https://www.ntop.org/guides/nbox/>
.. Indices and tables
.. ==================
..
.. * :ref:`genindex`
.. * :ref:`modindex`
.. * :ref:`search`
+4
View File
@@ -0,0 +1,4 @@
breathe
exhale
sphinx-rtd-theme
sphinxcontrib-swaggerdoc
+56
View File
@@ -0,0 +1,56 @@
What is nDPI
##############
nDPI is a DPI (Deep Packet Inspection) toolkit able to:
- Detect application protocol on traffic flows
- Analize encrypted traffic flows via ET (Encrypted Traffic Analysis)
- Extract selected protocol metadata from traffic
- Implement APIs for analysing traffic
Releases and Features
---------------------
nDPI development lifecycle is typically 6 to 9 months. The history of changes and features implemented by every release, is available on its `Changelog <https://github.com/ntop/nDPI/blob/dev/CHANGELOG.md>`_.
Installation
============
nDPI is open source and available on `GitHub
<https://github.com/ntop/nDPI>`_. In addition, pre-compiled, binary
nDPI packages are available both for Linux and other platforms. Installation
instructions for binary packages are available below.
Installing on Linux
-------------------
Installation instructions can be found at
http://packages.ntop.org/. Development and stable builds are
available. Stable builds are intended for production environments whereas
development builds are intended for testing or early feature access.
Software Updates
================
General instructions for updating the software can be found at
http://packages.ntop.org/ together with the installation instructions.
Depending on the Operating System, nDPI supports also automatic updates
through the GUI as described in the below sections.
Updating the Software on Linux
------------------------------
Instructions for updating the software via command line can be found
at http://packages.ntop.org/. For example on Ubuntu/Debian systems the
below commands will update the repository, check for updates and install
the latest software update if any:
.. code:: bash
apt-get update
apt-get upgrade
.. _AvailableVersions:
-18
View File
@@ -1,18 +0,0 @@
# How to use?
You need to first compile the nDPI library as usual:
- ./autogen.sh
- ./configure
- make
Then open the Xcode project and you are ready to go. The default behavior is to analyze an embeded pcap file `capture.pcap`. You can change the behavior by changing command line input in `ViewController.m` file.
# What does the XCode project do?
It's a dummy Mac App project with a **Run** button. It doesn't modify any nDPI code except that it renamed the `main` function to `orginal_main` in `ndpiReader.c` (because the Mac App has it's own main function) and call the `orginal_main` with synthetic command line input from `ViewController.m` file when the **Run** button is clicked.
It also fixes some problems when compiling with Xcode. Some are listed below:
- Add missed `NDPI_LOG_DEBUG2` macro definition implementation (defined as `NDPI_LOG_DEBUG2_XCODE_PROJ` in `ViewController.m`)
- Add an empty ndpi_utils.h file to make `protocols/attic/ftp.c` and `protocols/attic/secondlife.c` can compile
- Specially treat `ndpi_patricia.c` by not adding it into compilation source, since it's directly included in `ndpi_main.c`
File diff suppressed because it is too large Load Diff
@@ -1,7 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<Workspace
version = "1.0">
<FileRef
location = "self:ndpiExample.xcodeproj">
</FileRef>
</Workspace>
@@ -1,58 +0,0 @@
{
"images" : [
{
"idiom" : "mac",
"size" : "16x16",
"scale" : "1x"
},
{
"idiom" : "mac",
"size" : "16x16",
"scale" : "2x"
},
{
"idiom" : "mac",
"size" : "32x32",
"scale" : "1x"
},
{
"idiom" : "mac",
"size" : "32x32",
"scale" : "2x"
},
{
"idiom" : "mac",
"size" : "128x128",
"scale" : "1x"
},
{
"idiom" : "mac",
"size" : "128x128",
"scale" : "2x"
},
{
"idiom" : "mac",
"size" : "256x256",
"scale" : "1x"
},
{
"idiom" : "mac",
"size" : "256x256",
"scale" : "2x"
},
{
"idiom" : "mac",
"size" : "512x512",
"scale" : "1x"
},
{
"idiom" : "mac",
"size" : "512x512",
"scale" : "2x"
}
],
"info" : {
"version" : 1,
"author" : "xcode"
}
}
@@ -1,732 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<document type="com.apple.InterfaceBuilder3.Cocoa.Storyboard.XIB" version="3.0" toolsVersion="13771" targetRuntime="MacOSX.Cocoa" propertyAccessControl="none" useAutolayout="YES" initialViewController="B8D-0N-5wS">
<dependencies>
<deployment identifier="macosx"/>
<plugIn identifier="com.apple.InterfaceBuilder.CocoaPlugin" version="13771"/>
<capability name="documents saved in the Xcode 8 format" minToolsVersion="8.0"/>
</dependencies>
<scenes>
<!--Application-->
<scene sceneID="JPo-4y-FX3">
<objects>
<application id="hnw-xV-0zn" sceneMemberID="viewController">
<menu key="mainMenu" title="Main Menu" systemMenu="main" id="AYu-sK-qS6">
<items>
<menuItem title="ndpiExample" id="1Xt-HY-uBw">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="ndpiExample" systemMenu="apple" id="uQy-DD-JDr">
<items>
<menuItem title="About ndpiExample" id="5kV-Vb-QxS">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="orderFrontStandardAboutPanel:" target="Ady-hI-5gd" id="Exp-CZ-Vem"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="VOq-y0-SEH"/>
<menuItem title="Preferences…" keyEquivalent="," id="BOF-NM-1cW"/>
<menuItem isSeparatorItem="YES" id="wFC-TO-SCJ"/>
<menuItem title="Services" id="NMo-om-nkz">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Services" systemMenu="services" id="hz9-B4-Xy5"/>
</menuItem>
<menuItem isSeparatorItem="YES" id="4je-JR-u6R"/>
<menuItem title="Hide ndpiExample" keyEquivalent="h" id="Olw-nP-bQN">
<connections>
<action selector="hide:" target="Ady-hI-5gd" id="PnN-Uc-m68"/>
</connections>
</menuItem>
<menuItem title="Hide Others" keyEquivalent="h" id="Vdr-fp-XzO">
<modifierMask key="keyEquivalentModifierMask" option="YES" command="YES"/>
<connections>
<action selector="hideOtherApplications:" target="Ady-hI-5gd" id="VT4-aY-XCT"/>
</connections>
</menuItem>
<menuItem title="Show All" id="Kd2-mp-pUS">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="unhideAllApplications:" target="Ady-hI-5gd" id="Dhg-Le-xox"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="kCx-OE-vgT"/>
<menuItem title="Quit ndpiExample" keyEquivalent="q" id="4sb-4s-VLi">
<connections>
<action selector="terminate:" target="Ady-hI-5gd" id="Te7-pn-YzF"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="File" id="dMs-cI-mzQ">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="File" id="bib-Uj-vzu">
<items>
<menuItem title="New" keyEquivalent="n" id="Was-JA-tGl">
<connections>
<action selector="newDocument:" target="Ady-hI-5gd" id="4Si-XN-c54"/>
</connections>
</menuItem>
<menuItem title="Open…" keyEquivalent="o" id="IAo-SY-fd9">
<connections>
<action selector="openDocument:" target="Ady-hI-5gd" id="bVn-NM-KNZ"/>
</connections>
</menuItem>
<menuItem title="Open Recent" id="tXI-mr-wws">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Open Recent" systemMenu="recentDocuments" id="oas-Oc-fiZ">
<items>
<menuItem title="Clear Menu" id="vNY-rz-j42">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="clearRecentDocuments:" target="Ady-hI-5gd" id="Daa-9d-B3U"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem isSeparatorItem="YES" id="m54-Is-iLE"/>
<menuItem title="Close" keyEquivalent="w" id="DVo-aG-piG">
<connections>
<action selector="performClose:" target="Ady-hI-5gd" id="HmO-Ls-i7Q"/>
</connections>
</menuItem>
<menuItem title="Save…" keyEquivalent="s" id="pxx-59-PXV">
<connections>
<action selector="saveDocument:" target="Ady-hI-5gd" id="teZ-XB-qJY"/>
</connections>
</menuItem>
<menuItem title="Save As…" keyEquivalent="S" id="Bw7-FT-i3A">
<connections>
<action selector="saveDocumentAs:" target="Ady-hI-5gd" id="mDf-zr-I0C"/>
</connections>
</menuItem>
<menuItem title="Revert to Saved" keyEquivalent="r" id="KaW-ft-85H">
<connections>
<action selector="revertDocumentToSaved:" target="Ady-hI-5gd" id="iJ3-Pv-kwq"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="aJh-i4-bef"/>
<menuItem title="Page Setup…" keyEquivalent="P" id="qIS-W8-SiK">
<modifierMask key="keyEquivalentModifierMask" shift="YES" command="YES"/>
<connections>
<action selector="runPageLayout:" target="Ady-hI-5gd" id="Din-rz-gC5"/>
</connections>
</menuItem>
<menuItem title="Print…" keyEquivalent="p" id="aTl-1u-JFS">
<connections>
<action selector="print:" target="Ady-hI-5gd" id="qaZ-4w-aoO"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Edit" id="5QF-Oa-p0T">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Edit" id="W48-6f-4Dl">
<items>
<menuItem title="Undo" keyEquivalent="z" id="dRJ-4n-Yzg">
<connections>
<action selector="undo:" target="Ady-hI-5gd" id="M6e-cu-g7V"/>
</connections>
</menuItem>
<menuItem title="Redo" keyEquivalent="Z" id="6dh-zS-Vam">
<connections>
<action selector="redo:" target="Ady-hI-5gd" id="oIA-Rs-6OD"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="WRV-NI-Exz"/>
<menuItem title="Cut" keyEquivalent="x" id="uRl-iY-unG">
<connections>
<action selector="cut:" target="Ady-hI-5gd" id="YJe-68-I9s"/>
</connections>
</menuItem>
<menuItem title="Copy" keyEquivalent="c" id="x3v-GG-iWU">
<connections>
<action selector="copy:" target="Ady-hI-5gd" id="G1f-GL-Joy"/>
</connections>
</menuItem>
<menuItem title="Paste" keyEquivalent="v" id="gVA-U4-sdL">
<connections>
<action selector="paste:" target="Ady-hI-5gd" id="UvS-8e-Qdg"/>
</connections>
</menuItem>
<menuItem title="Paste and Match Style" keyEquivalent="V" id="WeT-3V-zwk">
<modifierMask key="keyEquivalentModifierMask" option="YES" command="YES"/>
<connections>
<action selector="pasteAsPlainText:" target="Ady-hI-5gd" id="cEh-KX-wJQ"/>
</connections>
</menuItem>
<menuItem title="Delete" id="pa3-QI-u2k">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="delete:" target="Ady-hI-5gd" id="0Mk-Ml-PaM"/>
</connections>
</menuItem>
<menuItem title="Select All" keyEquivalent="a" id="Ruw-6m-B2m">
<connections>
<action selector="selectAll:" target="Ady-hI-5gd" id="VNm-Mi-diN"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="uyl-h8-XO2"/>
<menuItem title="Find" id="4EN-yA-p0u">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Find" id="1b7-l0-nxx">
<items>
<menuItem title="Find…" tag="1" keyEquivalent="f" id="Xz5-n4-O0W">
<connections>
<action selector="performFindPanelAction:" target="Ady-hI-5gd" id="cD7-Qs-BN4"/>
</connections>
</menuItem>
<menuItem title="Find and Replace…" tag="12" keyEquivalent="f" id="YEy-JH-Tfz">
<modifierMask key="keyEquivalentModifierMask" option="YES" command="YES"/>
<connections>
<action selector="performFindPanelAction:" target="Ady-hI-5gd" id="WD3-Gg-5AJ"/>
</connections>
</menuItem>
<menuItem title="Find Next" tag="2" keyEquivalent="g" id="q09-fT-Sye">
<connections>
<action selector="performFindPanelAction:" target="Ady-hI-5gd" id="NDo-RZ-v9R"/>
</connections>
</menuItem>
<menuItem title="Find Previous" tag="3" keyEquivalent="G" id="OwM-mh-QMV">
<connections>
<action selector="performFindPanelAction:" target="Ady-hI-5gd" id="HOh-sY-3ay"/>
</connections>
</menuItem>
<menuItem title="Use Selection for Find" tag="7" keyEquivalent="e" id="buJ-ug-pKt">
<connections>
<action selector="performFindPanelAction:" target="Ady-hI-5gd" id="U76-nv-p5D"/>
</connections>
</menuItem>
<menuItem title="Jump to Selection" keyEquivalent="j" id="S0p-oC-mLd">
<connections>
<action selector="centerSelectionInVisibleArea:" target="Ady-hI-5gd" id="IOG-6D-g5B"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Spelling and Grammar" id="Dv1-io-Yv7">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Spelling" id="3IN-sU-3Bg">
<items>
<menuItem title="Show Spelling and Grammar" keyEquivalent=":" id="HFo-cy-zxI">
<connections>
<action selector="showGuessPanel:" target="Ady-hI-5gd" id="vFj-Ks-hy3"/>
</connections>
</menuItem>
<menuItem title="Check Document Now" keyEquivalent=";" id="hz2-CU-CR7">
<connections>
<action selector="checkSpelling:" target="Ady-hI-5gd" id="fz7-VC-reM"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="bNw-od-mp5"/>
<menuItem title="Check Spelling While Typing" id="rbD-Rh-wIN">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleContinuousSpellChecking:" target="Ady-hI-5gd" id="7w6-Qz-0kB"/>
</connections>
</menuItem>
<menuItem title="Check Grammar With Spelling" id="mK6-2p-4JG">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleGrammarChecking:" target="Ady-hI-5gd" id="muD-Qn-j4w"/>
</connections>
</menuItem>
<menuItem title="Correct Spelling Automatically" id="78Y-hA-62v">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleAutomaticSpellingCorrection:" target="Ady-hI-5gd" id="2lM-Qi-WAP"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Substitutions" id="9ic-FL-obx">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Substitutions" id="FeM-D8-WVr">
<items>
<menuItem title="Show Substitutions" id="z6F-FW-3nz">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="orderFrontSubstitutionsPanel:" target="Ady-hI-5gd" id="oku-mr-iSq"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="gPx-C9-uUO"/>
<menuItem title="Smart Copy/Paste" id="9yt-4B-nSM">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleSmartInsertDelete:" target="Ady-hI-5gd" id="3IJ-Se-DZD"/>
</connections>
</menuItem>
<menuItem title="Smart Quotes" id="hQb-2v-fYv">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleAutomaticQuoteSubstitution:" target="Ady-hI-5gd" id="ptq-xd-QOA"/>
</connections>
</menuItem>
<menuItem title="Smart Dashes" id="rgM-f4-ycn">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleAutomaticDashSubstitution:" target="Ady-hI-5gd" id="oCt-pO-9gS"/>
</connections>
</menuItem>
<menuItem title="Smart Links" id="cwL-P1-jid">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleAutomaticLinkDetection:" target="Ady-hI-5gd" id="Gip-E3-Fov"/>
</connections>
</menuItem>
<menuItem title="Data Detectors" id="tRr-pd-1PS">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleAutomaticDataDetection:" target="Ady-hI-5gd" id="R1I-Nq-Kbl"/>
</connections>
</menuItem>
<menuItem title="Text Replacement" id="HFQ-gK-NFA">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleAutomaticTextReplacement:" target="Ady-hI-5gd" id="DvP-Fe-Py6"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Transformations" id="2oI-Rn-ZJC">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Transformations" id="c8a-y6-VQd">
<items>
<menuItem title="Make Upper Case" id="vmV-6d-7jI">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="uppercaseWord:" target="Ady-hI-5gd" id="sPh-Tk-edu"/>
</connections>
</menuItem>
<menuItem title="Make Lower Case" id="d9M-CD-aMd">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="lowercaseWord:" target="Ady-hI-5gd" id="iUZ-b5-hil"/>
</connections>
</menuItem>
<menuItem title="Capitalize" id="UEZ-Bs-lqG">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="capitalizeWord:" target="Ady-hI-5gd" id="26H-TL-nsh"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Speech" id="xrE-MZ-jX0">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Speech" id="3rS-ZA-NoH">
<items>
<menuItem title="Start Speaking" id="Ynk-f8-cLZ">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="startSpeaking:" target="Ady-hI-5gd" id="654-Ng-kyl"/>
</connections>
</menuItem>
<menuItem title="Stop Speaking" id="Oyz-dy-DGm">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="stopSpeaking:" target="Ady-hI-5gd" id="dX8-6p-jy9"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Format" id="jxT-CU-nIS">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Format" id="GEO-Iw-cKr">
<items>
<menuItem title="Font" id="Gi5-1S-RQB">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Font" systemMenu="font" id="aXa-aM-Jaq">
<items>
<menuItem title="Show Fonts" keyEquivalent="t" id="Q5e-8K-NDq">
<connections>
<action selector="orderFrontFontPanel:" target="YLy-65-1bz" id="WHr-nq-2xA"/>
</connections>
</menuItem>
<menuItem title="Bold" tag="2" keyEquivalent="b" id="GB9-OM-e27">
<connections>
<action selector="addFontTrait:" target="YLy-65-1bz" id="hqk-hr-sYV"/>
</connections>
</menuItem>
<menuItem title="Italic" tag="1" keyEquivalent="i" id="Vjx-xi-njq">
<connections>
<action selector="addFontTrait:" target="YLy-65-1bz" id="IHV-OB-c03"/>
</connections>
</menuItem>
<menuItem title="Underline" keyEquivalent="u" id="WRG-CD-K1S">
<connections>
<action selector="underline:" target="Ady-hI-5gd" id="FYS-2b-JAY"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="5gT-KC-WSO"/>
<menuItem title="Bigger" tag="3" keyEquivalent="+" id="Ptp-SP-VEL">
<connections>
<action selector="modifyFont:" target="YLy-65-1bz" id="Uc7-di-UnL"/>
</connections>
</menuItem>
<menuItem title="Smaller" tag="4" keyEquivalent="-" id="i1d-Er-qST">
<connections>
<action selector="modifyFont:" target="YLy-65-1bz" id="HcX-Lf-eNd"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="kx3-Dk-x3B"/>
<menuItem title="Kern" id="jBQ-r6-VK2">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Kern" id="tlD-Oa-oAM">
<items>
<menuItem title="Use Default" id="GUa-eO-cwY">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="useStandardKerning:" target="Ady-hI-5gd" id="6dk-9l-Ckg"/>
</connections>
</menuItem>
<menuItem title="Use None" id="cDB-IK-hbR">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="turnOffKerning:" target="Ady-hI-5gd" id="U8a-gz-Maa"/>
</connections>
</menuItem>
<menuItem title="Tighten" id="46P-cB-AYj">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="tightenKerning:" target="Ady-hI-5gd" id="hr7-Nz-8ro"/>
</connections>
</menuItem>
<menuItem title="Loosen" id="ogc-rX-tC1">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="loosenKerning:" target="Ady-hI-5gd" id="8i4-f9-FKE"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Ligatures" id="o6e-r0-MWq">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Ligatures" id="w0m-vy-SC9">
<items>
<menuItem title="Use Default" id="agt-UL-0e3">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="useStandardLigatures:" target="Ady-hI-5gd" id="7uR-wd-Dx6"/>
</connections>
</menuItem>
<menuItem title="Use None" id="J7y-lM-qPV">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="turnOffLigatures:" target="Ady-hI-5gd" id="iX2-gA-Ilz"/>
</connections>
</menuItem>
<menuItem title="Use All" id="xQD-1f-W4t">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="useAllLigatures:" target="Ady-hI-5gd" id="KcB-kA-TuK"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Baseline" id="OaQ-X3-Vso">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Baseline" id="ijk-EB-dga">
<items>
<menuItem title="Use Default" id="3Om-Ey-2VK">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="unscript:" target="Ady-hI-5gd" id="0vZ-95-Ywn"/>
</connections>
</menuItem>
<menuItem title="Superscript" id="Rqc-34-cIF">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="superscript:" target="Ady-hI-5gd" id="3qV-fo-wpU"/>
</connections>
</menuItem>
<menuItem title="Subscript" id="I0S-gh-46l">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="subscript:" target="Ady-hI-5gd" id="Q6W-4W-IGz"/>
</connections>
</menuItem>
<menuItem title="Raise" id="2h7-ER-AoG">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="raiseBaseline:" target="Ady-hI-5gd" id="4sk-31-7Q9"/>
</connections>
</menuItem>
<menuItem title="Lower" id="1tx-W0-xDw">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="lowerBaseline:" target="Ady-hI-5gd" id="OF1-bc-KW4"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem isSeparatorItem="YES" id="Ndw-q3-faq"/>
<menuItem title="Show Colors" keyEquivalent="C" id="bgn-CT-cEk">
<connections>
<action selector="orderFrontColorPanel:" target="Ady-hI-5gd" id="mSX-Xz-DV3"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="iMs-zA-UFJ"/>
<menuItem title="Copy Style" keyEquivalent="c" id="5Vv-lz-BsD">
<modifierMask key="keyEquivalentModifierMask" option="YES" command="YES"/>
<connections>
<action selector="copyFont:" target="Ady-hI-5gd" id="GJO-xA-L4q"/>
</connections>
</menuItem>
<menuItem title="Paste Style" keyEquivalent="v" id="vKC-jM-MkH">
<modifierMask key="keyEquivalentModifierMask" option="YES" command="YES"/>
<connections>
<action selector="pasteFont:" target="Ady-hI-5gd" id="JfD-CL-leO"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Text" id="Fal-I4-PZk">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Text" id="d9c-me-L2H">
<items>
<menuItem title="Align Left" keyEquivalent="{" id="ZM1-6Q-yy1">
<connections>
<action selector="alignLeft:" target="Ady-hI-5gd" id="zUv-R1-uAa"/>
</connections>
</menuItem>
<menuItem title="Center" keyEquivalent="|" id="VIY-Ag-zcb">
<connections>
<action selector="alignCenter:" target="Ady-hI-5gd" id="spX-mk-kcS"/>
</connections>
</menuItem>
<menuItem title="Justify" id="J5U-5w-g23">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="alignJustified:" target="Ady-hI-5gd" id="ljL-7U-jND"/>
</connections>
</menuItem>
<menuItem title="Align Right" keyEquivalent="}" id="wb2-vD-lq4">
<connections>
<action selector="alignRight:" target="Ady-hI-5gd" id="r48-bG-YeY"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="4s2-GY-VfK"/>
<menuItem title="Writing Direction" id="H1b-Si-o9J">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Writing Direction" id="8mr-sm-Yjd">
<items>
<menuItem title="Paragraph" enabled="NO" id="ZvO-Gk-QUH">
<modifierMask key="keyEquivalentModifierMask"/>
</menuItem>
<menuItem id="YGs-j5-SAR">
<string key="title"> Default</string>
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="makeBaseWritingDirectionNatural:" target="Ady-hI-5gd" id="qtV-5e-UBP"/>
</connections>
</menuItem>
<menuItem id="Lbh-J2-qVU">
<string key="title"> Left to Right</string>
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="makeBaseWritingDirectionLeftToRight:" target="Ady-hI-5gd" id="S0X-9S-QSf"/>
</connections>
</menuItem>
<menuItem id="jFq-tB-4Kx">
<string key="title"> Right to Left</string>
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="makeBaseWritingDirectionRightToLeft:" target="Ady-hI-5gd" id="5fk-qB-AqJ"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="swp-gr-a21"/>
<menuItem title="Selection" enabled="NO" id="cqv-fj-IhA">
<modifierMask key="keyEquivalentModifierMask"/>
</menuItem>
<menuItem id="Nop-cj-93Q">
<string key="title"> Default</string>
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="makeTextWritingDirectionNatural:" target="Ady-hI-5gd" id="lPI-Se-ZHp"/>
</connections>
</menuItem>
<menuItem id="BgM-ve-c93">
<string key="title"> Left to Right</string>
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="makeTextWritingDirectionLeftToRight:" target="Ady-hI-5gd" id="caW-Bv-w94"/>
</connections>
</menuItem>
<menuItem id="RB4-Sm-HuC">
<string key="title"> Right to Left</string>
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="makeTextWritingDirectionRightToLeft:" target="Ady-hI-5gd" id="EXD-6r-ZUu"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem isSeparatorItem="YES" id="fKy-g9-1gm"/>
<menuItem title="Show Ruler" id="vLm-3I-IUL">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="toggleRuler:" target="Ady-hI-5gd" id="FOx-HJ-KwY"/>
</connections>
</menuItem>
<menuItem title="Copy Ruler" keyEquivalent="c" id="MkV-Pr-PK5">
<modifierMask key="keyEquivalentModifierMask" control="YES" command="YES"/>
<connections>
<action selector="copyRuler:" target="Ady-hI-5gd" id="71i-fW-3W2"/>
</connections>
</menuItem>
<menuItem title="Paste Ruler" keyEquivalent="v" id="LVM-kO-fVI">
<modifierMask key="keyEquivalentModifierMask" control="YES" command="YES"/>
<connections>
<action selector="pasteRuler:" target="Ady-hI-5gd" id="cSh-wd-qM2"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="View" id="H8h-7b-M4v">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="View" id="HyV-fh-RgO">
<items>
<menuItem title="Show Toolbar" keyEquivalent="t" id="snW-S8-Cw5">
<modifierMask key="keyEquivalentModifierMask" option="YES" command="YES"/>
<connections>
<action selector="toggleToolbarShown:" target="Ady-hI-5gd" id="BXY-wc-z0C"/>
</connections>
</menuItem>
<menuItem title="Customize Toolbar…" id="1UK-8n-QPP">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="runToolbarCustomizationPalette:" target="Ady-hI-5gd" id="pQI-g3-MTW"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="hB3-LF-h0Y"/>
<menuItem title="Show Sidebar" keyEquivalent="s" id="kIP-vf-haE">
<modifierMask key="keyEquivalentModifierMask" control="YES" command="YES"/>
<connections>
<action selector="toggleSourceList:" target="Ady-hI-5gd" id="iwa-gc-5KM"/>
</connections>
</menuItem>
<menuItem title="Enter Full Screen" keyEquivalent="f" id="4J7-dP-txa">
<modifierMask key="keyEquivalentModifierMask" control="YES" command="YES"/>
<connections>
<action selector="toggleFullScreen:" target="Ady-hI-5gd" id="dU3-MA-1Rq"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Window" id="aUF-d1-5bR">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Window" systemMenu="window" id="Td7-aD-5lo">
<items>
<menuItem title="Minimize" keyEquivalent="m" id="OY7-WF-poV">
<connections>
<action selector="performMiniaturize:" target="Ady-hI-5gd" id="VwT-WD-YPe"/>
</connections>
</menuItem>
<menuItem title="Zoom" id="R4o-n2-Eq4">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="performZoom:" target="Ady-hI-5gd" id="DIl-cC-cCs"/>
</connections>
</menuItem>
<menuItem isSeparatorItem="YES" id="eu3-7i-yIM"/>
<menuItem title="Bring All to Front" id="LE2-aR-0XJ">
<modifierMask key="keyEquivalentModifierMask"/>
<connections>
<action selector="arrangeInFront:" target="Ady-hI-5gd" id="DRN-fu-gQh"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
<menuItem title="Help" id="wpr-3q-Mcd">
<modifierMask key="keyEquivalentModifierMask"/>
<menu key="submenu" title="Help" systemMenu="help" id="F2S-fz-NVQ">
<items>
<menuItem title="ndpiExample Help" keyEquivalent="?" id="FKE-Sm-Kum">
<connections>
<action selector="showHelp:" target="Ady-hI-5gd" id="y7X-2Q-9no"/>
</connections>
</menuItem>
</items>
</menu>
</menuItem>
</items>
</menu>
<connections>
<outlet property="delegate" destination="Voe-Tx-rLC" id="PrD-fu-P6m"/>
</connections>
</application>
<customObject id="Voe-Tx-rLC" customClass="AppDelegate"/>
<customObject id="YLy-65-1bz" customClass="NSFontManager"/>
<customObject id="Ady-hI-5gd" userLabel="First Responder" customClass="NSResponder" sceneMemberID="firstResponder"/>
</objects>
<point key="canvasLocation" x="75" y="0.0"/>
</scene>
<!--Window Controller-->
<scene sceneID="R2V-B0-nI4">
<objects>
<windowController id="B8D-0N-5wS" sceneMemberID="viewController">
<window key="window" title="Window" allowsToolTipsWhenApplicationIsInactive="NO" autorecalculatesKeyViewLoop="NO" oneShot="NO" releasedWhenClosed="NO" showsToolbarButton="NO" visibleAtLaunch="NO" animationBehavior="default" id="IQv-IB-iLA">
<windowStyleMask key="styleMask" titled="YES" closable="YES" miniaturizable="YES" resizable="YES"/>
<windowPositionMask key="initialPositionMask" leftStrut="YES" rightStrut="YES" topStrut="YES" bottomStrut="YES"/>
<rect key="contentRect" x="196" y="240" width="480" height="270"/>
<rect key="screenRect" x="0.0" y="0.0" width="1680" height="1027"/>
<connections>
<outlet property="delegate" destination="B8D-0N-5wS" id="98r-iN-zZc"/>
</connections>
</window>
<connections>
<segue destination="XfG-lQ-9wD" kind="relationship" relationship="window.shadowedContentViewController" id="cq2-FE-JQM"/>
</connections>
</windowController>
<customObject id="Oky-zY-oP4" userLabel="First Responder" customClass="NSResponder" sceneMemberID="firstResponder"/>
</objects>
<point key="canvasLocation" x="75" y="250"/>
</scene>
<!--View Controller-->
<scene sceneID="hIz-AP-VOD">
<objects>
<viewController id="XfG-lQ-9wD" customClass="ViewController" sceneMemberID="viewController">
<view key="view" wantsLayer="YES" id="m2S-Jp-Qdl">
<rect key="frame" x="0.0" y="0.0" width="480" height="270"/>
<autoresizingMask key="autoresizingMask"/>
<subviews>
<button verticalHuggingPriority="750" fixedFrame="YES" translatesAutoresizingMaskIntoConstraints="NO" id="Jdf-YK-4JB">
<rect key="frame" x="23" y="173" width="64" height="32"/>
<autoresizingMask key="autoresizingMask" flexibleMaxX="YES" flexibleMinY="YES"/>
<buttonCell key="cell" type="push" title="Run" bezelStyle="rounded" alignment="center" borderStyle="border" imageScaling="proportionallyDown" inset="2" id="1eG-Fb-Ybk">
<behavior key="behavior" pushIn="YES" lightByBackground="YES" lightByGray="YES"/>
<font key="font" metaFont="system"/>
</buttonCell>
<connections>
<action selector="onRunButtonClicked:" target="XfG-lQ-9wD" id="zyK-t7-vwt"/>
</connections>
</button>
</subviews>
</view>
</viewController>
<customObject id="rPt-NT-nkU" userLabel="First Responder" customClass="NSResponder" sceneMemberID="firstResponder"/>
</objects>
<point key="canvasLocation" x="75" y="655"/>
</scene>
</scenes>
</document>
@@ -1,32 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>CFBundleDevelopmentRegion</key>
<string>$(DEVELOPMENT_LANGUAGE)</string>
<key>CFBundleExecutable</key>
<string>$(EXECUTABLE_NAME)</string>
<key>CFBundleIconFile</key>
<string></string>
<key>CFBundleIdentifier</key>
<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
<key>CFBundleInfoDictionaryVersion</key>
<string>6.0</string>
<key>CFBundleName</key>
<string>$(PRODUCT_NAME)</string>
<key>CFBundlePackageType</key>
<string>APPL</string>
<key>CFBundleShortVersionString</key>
<string>1.0</string>
<key>CFBundleVersion</key>
<string>1</string>
<key>LSMinimumSystemVersion</key>
<string>$(MACOSX_DEPLOYMENT_TARGET)</string>
<key>NSHumanReadableCopyright</key>
<string>Copyright © 2018年 ZengYingpei. All rights reserved.</string>
<key>NSMainStoryboardFile</key>
<string>Main</string>
<key>NSPrincipalClass</key>
<string>NSApplication</string>
</dict>
</plist>
@@ -1,106 +0,0 @@
/*
*
* Copyright (C) 2011-18 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#import "ViewController.h"
#include "ndpi_api.h"
// Declare the orginal_main defined in ndpiReader.c here
extern int orginal_main(int argc, char **argv);
@implementation ViewController
- (void)viewDidLoad {
[super viewDidLoad];
// Do any additional setup after loading the view.
}
- (void)setRepresentedObject:(id)representedObject {
[super setRepresentedObject:representedObject];
// Update the view, if already loaded.
}
- (IBAction)onRunButtonClicked:(id)sender
{
char* args[10];
extern int optind;
optind = 1; // reset the parse of getopt_long
// Check the "nDPI_QuickStartGuide.pdf" for comand option explanation.
/* Following code it to execute below command (remember to change args[2] to
* absolute path):
* ./ndpiReader -i capture.pcap
*/
args[0] = (char*)"ndpiReader";
args[1] = (char*)"-i";
NSString* pcap_file = [[NSBundle mainBundle]pathForResource:@"capture" ofType:@"pcap"];
args[2] = (char*)[pcap_file cStringUsingEncoding:NSUTF8StringEncoding];
// Change to you pcap file path if you want.
//args[2] = (char*)"/Users/zengyingpei/Documents/code/nDPI/example/MacOS/ndpiExample/ndpiExample/capture.pcap";
// Remember to change below number of args when you change to other command inputs.
orginal_main(3, args);
/* Following code it to execute below command:
* ./ndpiReader -i en1 -s 10 -p protos.txt
* The process seems to be not support re-entering. You may have to re-run the App.
*/
/*
args[0] = (char*)"ndpiReader";
args[1] = (char*)"-i";
args[2] = (char*)"en0";
args[3] = (char*)"-s";
args[4] = (char*)"10";
args[5] = (char*)"-p";
NSString* proto_file = [[NSBundle mainBundle]pathForResource:@"protos" ofType:@"txt"];
args[6] = (char*)[proto_file cStringUsingEncoding:NSUTF8StringEncoding];
//args[6] = (char*)"/Users/zengyingpei/Documents/code/nDPI/example/protos.txt";
orginal_main(7, args);
*/
}
// In order to fix the missing of NDPI_LOG_DEBUG2 (used in ndpi_main.c), we define
// NDPI_LOG_DEBUG2 as NDPI_LOG_DEBUG2_XCODE_PROJ.
void vNDPI_LOG_DEBUG2_XCODE_PROJ(struct ndpi_detection_module_struct * ndpi_struct,
const char *format, va_list ap)
{
vprintf(format, ap);
}
void NDPI_LOG_DEBUG2_XCODE_PROJ(struct ndpi_detection_module_struct * ndpi_struct,
const char *format, ...)
{
va_list ap;
va_start (ap, format);
vNDPI_LOG_DEBUG2_XCODE_PROJ(ndpi_struct, format, ap);
va_end (ap);
}
@end
Binary file not shown.
@@ -1,27 +0,0 @@
/*
*
* Copyright (C) 2011-18 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#import <Cocoa/Cocoa.h>
int main(int argc, const char * argv[]) {
return NSApplicationMain(argc, argv);
}
@@ -1,5 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict/>
</plist>
+2 -2
View File
@@ -17,11 +17,11 @@ include $(RTE_SDK)/mk/rte.vars.mk
APP = ndpiReader.dpdk
LIBNDPI = $(PWD)/../src/lib/libndpi.a
SRCS-y := reader_util.c intrusion_detection.c ndpiReader.c
SRCS-y := reader_util.c ndpiReader.c
CFLAGS += -g
CFLAGS += -Wno-strict-prototypes -Wno-missing-prototypes -Wno-missing-declarations -Wno-unused-parameter -I $(PWD)/../src/include @CFLAGS@ -DUSE_DPDK
LDLIBS = $(LIBNDPI) @PCAP_LIB@ @LIBS@ -lpthread -lm @LDFLAGS@
LDLIBS = $(LIBNDPI) @PCAP_LIB@ @LIBS@ @ADDITIONAL_LIBS@ -lpthread @LDFLAGS@
include $(RTE_SDK)/mk/rte.extapp.mk
+53 -21
View File
@@ -1,65 +1,97 @@
AR=@AR@
CC=@CC@
CXX=@CXX@
BUILD_MINGW=@BUILD_MINGW@
BUILD_MINGW_X64=@BUILD_MINGW_X64@
DISABLE_NPCAP=@DISABLE_NPCAP@
EXE_SUFFIX=@EXE_SUFFIX@
SRCHOME=../src
CFLAGS=-g -fPIC -DPIC -I$(SRCHOME)/include @PCAP_INC@ @CFLAGS@
ifneq ($(OS),Windows_NT)
CFLAGS+=-fPIC -DPIC
endif
CFLAGS+=-I$(SRCHOME)/include @PCAP_INC@ @CFLAGS@ @GPROF_CFLAGS@
LDFLAGS=@LDFLAGS@
LIBNDPI=$(SRCHOME)/lib/libndpi.a
LDFLAGS=$(LIBNDPI) @PCAP_LIB@ @LIBS@ @ADDITIONAL_LIBS@ -lpthread -lm @LDFLAGS@
HEADERS=intrusion_detection.h reader_util.h $(SRCHOME)/include/ndpi_api.h \
LIBS=$(LIBNDPI) @PCAP_LIB@ @ADDITIONAL_LIBS@ @LIBS@ @GPROF_LIBS@
HEADERS=reader_util.h $(SRCHOME)/include/ndpi_api.h \
$(SRCHOME)/include/ndpi_typedefs.h $(SRCHOME)/include/ndpi_protocol_ids.h
OBJS=ndpiReader.o reader_util.o intrusion_detection.o
PREFIX?=@prefix@
ifneq ($(BUILD_MINGW),)
all:
@echo 'Examples disabled due to mingw build.'
ifeq ($(DISABLE_NPCAP),0)
CFLAGS+=-I@srcdir@/../windows/WpdPack/Include -I@srcdir@/../windows/WpdPack/Include/pcap
else
CFLAGS+=-DDISABLE_NPCAP
endif
all: ndpiReader @DPDK_TARGET@
ifeq ($(DISABLE_NPCAP),0)
ifneq ($(BUILD_MINGW_X64),)
LIBS+=@srcdir@/../windows/WpdPack/Lib/x64/wpcap.lib
else
LIBS+=@srcdir@/../windows/WpdPack/Lib/wpcap.lib
endif
endif
LIBS+=-Wl,-Bstatic -lpthread -Wl,-Bdynamic
else
LIBS+=-pthread
endif
CFLAGS+=-pthread
all: ndpiReader$(EXE_SUFFIX) @DPDK_TARGET@
EXECUTABLE_SOURCES := ndpiReader.c ndpiSimpleIntegration.c
COMMON_SOURCES := $(filter-out $(EXECUTABLE_SOURCES),$(wildcard *.c ))
libndpiReader.a: $(COMMON_SOURCES:%.c=%.o) $(LIBNDPI)
ar rsv libndpiReader.a $(COMMON_SOURCES:%.c=%.o)
$(AR) rsv libndpiReader.a $(COMMON_SOURCES:%.c=%.o)
ndpiReader: libndpiReader.a $(LIBNDPI) ndpiReader.o
$(CC) $(CFLAGS) ndpiReader.o libndpiReader.a -o $@ $(LDFLAGS)
ndpiReader$(EXE_SUFFIX): libndpiReader.a $(LIBNDPI) ndpiReader.o
$(CC) $(CFLAGS) $(LDFLAGS) ndpiReader.o libndpiReader.a $(LIBS) -o $@
ndpiSimpleIntegration: ndpiSimpleIntegration.o
$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)
ndpiSimpleIntegration$(EXE_SUFFIX): ndpiSimpleIntegration.o
$(CC) $(CFLAGS) $(LDFLAGS) $< $(LIBS) -o $@
%.o: %.c $(HEADERS) Makefile
$(CC) $(CFLAGS) -c $< -o $@
$(CC) $(CFLAGS) $(CPPFLAGS) -c $< -o $@
install: ndpiReader
install: ndpiReader$(EXE_SUFFIX)
mkdir -p $(DESTDIR)$(PREFIX)/bin/
mkdir -p $(DESTDIR)$(PREFIX)/share/ndpi
cp ndpiReader $(DESTDIR)$(PREFIX)/bin/
cp ndpiReader$(EXE_SUFFIX) $(DESTDIR)$(PREFIX)/bin/
cp protos.txt $(DESTDIR)$(PREFIX)/share/ndpi/ndpiProtos.txt
cp mining_hosts.txt $(DESTDIR)$(PREFIX)/share/ndpi/ndpiCustomCategory.txt
[ -f ndpiSimpleIntegration$(EXE_SUFFIX) ] && cp ndpiSimpleIntegration$(EXE_SUFFIX) $(DESTDIR)$(PREFIX)/bin/ndpiSimpleIntegration$(EXE_SUFFIX) || true
[ -f build/app/ndpiReader.dpdk ] && cp build/app/ndpiReader.dpdk $(DESTDIR)$(PREFIX)/bin/ || true
[ -f ndpiReader.dpdk ] && cp ndpiReader.dpdk $(DESTDIR)$(PREFIX)/bin/ || true
dpdk:
make -f Makefile.dpdk
$(MAKE) -f Makefile.dpdk
check:
true # nothing to do here
cppcheck:
cppcheck --template='{file}:{line}:{severity}:{message}' --quiet --enable=all --force -I$(SRCHOME)/include *.c
clean:
/bin/rm -f *.o ndpiReader ndpiReader.dpdk
/bin/rm -f *.o ndpiReader ndpiSimpleIntegration ndpiReader$(EXE_SUFFIX) ndpiSimpleIntegration$(EXE_SUFFIX) ndpiReader.dpdk
/bin/rm -f .*.dpdk.cmd .*.o.cmd *.dpdk.map .*.o.d
/bin/rm -f _install _postbuild _postinstall _preinstall
/bin/rm -rf build
distdir:
cp categories.txt mining_hosts.txt protos.txt README.DPDK '$(distdir)/'
find . -maxdepth 1 -type f -name '*.c' -o -name '*.h' -o -name '*.py' | xargs -I'{}' cp '{}' '$(distdir)/{}'
cp README.DPDK '$(distdir)/'
find . -maxdepth 1 -type f -name '*.c' \
-o -name '*.h' \
-o -name '*.py' \
-o -name '*.txt' \
-o -name '*.csv' | xargs -I'{}' cp '{}' '$(distdir)/{}'
distclean: clean
/bin/rm -f Makefile.dpdk
/bin/rm -f Makefile
endif
-20
View File
@@ -1,20 +0,0 @@
Microsoft Visual Studio Solution File, Format Version 11.00
# Visual C++ Express 2010
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pcapExample", "pcapExample\pcapExample.vcxproj", "{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Win32 = Debug|Win32
Release|Win32 = Release|Win32
EndGlobalSection
GlobalSection(ProjectConfigurationPlatforms) = postSolution
{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}.Debug|Win32.ActiveCfg = Debug|Win32
{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}.Debug|Win32.Build.0 = Debug|Win32
{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}.Release|Win32.ActiveCfg = Release|Win32
{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}.Release|Win32.Build.0 = Release|Win32
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
EndGlobal
Binary file not shown.
-11
View File
@@ -1,11 +0,0 @@
// pcapExample.cpp : Defines the entry point for the console application.
//
#include "stdafx.h"
int _tmain(int argc, _TCHAR* argv[])
{
return 0;
}
@@ -1,231 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug|Win32">
<Configuration>Debug</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
<ProjectConfiguration Include="Release|Win32">
<Configuration>Release</Configuration>
<Platform>Win32</Platform>
</ProjectConfiguration>
</ItemGroup>
<PropertyGroup Label="Globals">
<ProjectGuid>{F6A2C0AE-2110-438A-87E4-7C1CFCE064C6}</ProjectGuid>
<Keyword>Win32Proj</Keyword>
<RootNamespace>pcapExample</RootNamespace>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>true</UseDebugLibraries>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<UseDebugLibraries>false</UseDebugLibraries>
<WholeProgramOptimization>true</WholeProgramOptimization>
<CharacterSet>Unicode</CharacterSet>
</PropertyGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<ImportGroup Label="ExtensionSettings">
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<ImportGroup Label="PropertySheets" Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
</ImportGroup>
<PropertyGroup Label="UserMacros" />
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<LinkIncremental>true</LinkIncremental>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<LinkIncremental>false</LinkIncremental>
</PropertyGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<ClCompile>
<PrecompiledHeader>NotUsing</PrecompiledHeader>
<WarningLevel>Level3</WarningLevel>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>C:\ntop\nprobe\nprobe-win-32;C:\ntop\winpcap\Include;C:\ntop\dependencies\nDPI\src\lib\third_party\include;C:\ntop\dependencies\nDPI\src\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<AdditionalDependencies>Wsock32.lib;Ws2_32.lib;C:\ntop\winpcap\Lib\Packet.lib;C:\ntop\winpcap\Lib\wpcap.lib;%(AdditionalDependencies)</AdditionalDependencies>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<ClCompile>
<WarningLevel>Level3</WarningLevel>
<PrecompiledHeader>Use</PrecompiledHeader>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;%(PreprocessorDefinitions)</PreprocessorDefinitions>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
</Link>
</ItemDefinitionGroup>
<ItemGroup>
<ClCompile Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt.c" />
<ClCompile Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt1.c" />
<ClCompile Include="..\..\..\src\lib\ndpi_main.c" />
<ClCompile Include="..\..\..\src\lib\protocols\afp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\aimini.c" />
<ClCompile Include="..\..\..\src\lib\protocols\applejuice.c" />
<ClCompile Include="..\..\..\src\lib\protocols\armagetron.c" />
<ClCompile Include="..\..\..\src\lib\protocols\battlefield.c" />
<ClCompile Include="..\..\..\src\lib\protocols\bgp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\bittorrent.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ciscovpn.c" />
<ClCompile Include="..\..\..\src\lib\protocols\citrix.c" />
<ClCompile Include="..\..\..\src\lib\protocols\corba.c" />
<ClCompile Include="..\..\..\src\lib\protocols\crossfire.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dcerpc.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dhcp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dhcpv6.c" />
<ClCompile Include="..\..\..\src\lib\protocols\directconnect.c" />
<ClCompile Include="..\..\..\src\lib\protocols\directdownloadlink.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dns.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dofus.c" />
<ClCompile Include="..\..\..\src\lib\protocols\dropbox.c" />
<ClCompile Include="..\..\..\src\lib\protocols\edonkey.c" />
<ClCompile Include="..\..\..\src\lib\protocols\fasttrack.c" />
<ClCompile Include="..\..\..\src\lib\protocols\feidian.c" />
<ClCompile Include="..\..\..\src\lib\protocols\fiesta.c" />
<ClCompile Include="..\..\..\src\lib\protocols\filetopia.c" />
<ClCompile Include="..\..\..\src\lib\protocols\flash.c" />
<ClCompile Include="..\..\..\src\lib\protocols\florensia.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ftp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\gadu_gadu.c" />
<ClCompile Include="..\..\..\src\lib\protocols\gnutella.c" />
<ClCompile Include="..\..\..\src\lib\protocols\gtp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\guildwars.c" />
<ClCompile Include="..\..\..\src\lib\protocols\h323.c" />
<ClCompile Include="..\..\..\src\lib\protocols\halflife2_and_mods.c" />
<ClCompile Include="..\..\..\src\lib\protocols\http.c" />
<ClCompile Include="..\..\..\src\lib\protocols\http_activesync.c" />
<ClCompile Include="..\..\..\src\lib\protocols\i23v5.c" />
<ClCompile Include="..\..\..\src\lib\protocols\iax.c" />
<ClCompile Include="..\..\..\src\lib\protocols\icecast.c" />
<ClCompile Include="..\..\..\src\lib\protocols\imesh.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ipp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\irc.c" />
<ClCompile Include="..\..\..\src\lib\protocols\jabber.c" />
<ClCompile Include="..\..\..\src\lib\protocols\kerberos.c" />
<ClCompile Include="..\..\..\src\lib\protocols\kontiki.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ldap.c" />
<ClCompile Include="..\..\..\src\lib\protocols\lotus_notes.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mail_imap.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mail_pop.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mail_smtp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\maplestory.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mdns.c" />
<ClCompile Include="..\..\..\src\lib\protocols\meebo.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mgcp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mms.c" />
<ClCompile Include="..\..\..\src\lib\protocols\msn.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mssql.c" />
<ClCompile Include="..\..\..\src\lib\protocols\mysql.c" />
<ClCompile Include="..\..\..\src\lib\protocols\netbios.c" />
<ClCompile Include="..\..\..\src\lib\protocols\netflow.c" />
<ClCompile Include="..\..\..\src\lib\protocols\nfs.c" />
<ClCompile Include="..\..\..\src\lib\protocols\noe.c" />
<ClCompile Include="..\..\..\src\lib\protocols\non_tcp_udp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ntp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\openft.c" />
<ClCompile Include="..\..\..\src\lib\protocols\openvpn.c" />
<ClCompile Include="..\..\..\src\lib\protocols\oracle.c" />
<ClCompile Include="..\..\..\src\lib\protocols\oscar.c" />
<ClCompile Include="..\..\..\src\lib\protocols\pando.c" />
<ClCompile Include="..\..\..\src\lib\protocols\pcanywhere.c" />
<ClCompile Include="..\..\..\src\lib\protocols\popo.c" />
<ClCompile Include="..\..\..\src\lib\protocols\postgres.c" />
<ClCompile Include="..\..\..\src\lib\protocols\pplive.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ppstream.c" />
<ClCompile Include="..\..\..\src\lib\protocols\pptp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\qq.c" />
<ClCompile Include="..\..\..\src\lib\protocols\quake.c" />
<ClCompile Include="..\..\..\src\lib\protocols\quic.c" />
<ClCompile Include="..\..\..\src\lib\protocols\radius.c" />
<ClCompile Include="..\..\..\src\lib\protocols\rdp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\rsync.c" />
<ClCompile Include="..\..\..\src\lib\protocols\rtcp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\rtp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\rtsp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\sflow.c" />
<ClCompile Include="..\..\..\src\lib\protocols\shoutcast.c" />
<ClCompile Include="..\..\..\src\lib\protocols\sip.c" />
<ClCompile Include="..\..\..\src\lib\protocols\skinny.c" />
<ClCompile Include="..\..\..\src\lib\protocols\skype.c" />
<ClCompile Include="..\..\..\src\lib\protocols\smb.c" />
<ClCompile Include="..\..\..\src\lib\protocols\snmp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\socrates.c" />
<ClCompile Include="..\..\..\src\lib\protocols\sopcast.c" />
<ClCompile Include="..\..\..\src\lib\protocols\soulseek.c" />
<ClCompile Include="..\..\..\src\lib\protocols\spotify.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ssdp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ssh.c" />
<ClCompile Include="..\..\..\src\lib\protocols\ssl.c" />
<ClCompile Include="..\..\..\src\lib\protocols\stealthnet.c" />
<ClCompile Include="..\..\..\src\lib\protocols\steam.c" />
<ClCompile Include="..\..\..\src\lib\protocols\stun.c" />
<ClCompile Include="..\..\..\src\lib\protocols\syslog.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tcp_udp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tds.c" />
<ClCompile Include="..\..\..\src\lib\protocols\teamspeak.c" />
<ClCompile Include="..\..\..\src\lib\protocols\teamviewer.c" />
<ClCompile Include="..\..\..\src\lib\protocols\telnet.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tftp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\thunder.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tor.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tvants.c" />
<ClCompile Include="..\..\..\src\lib\protocols\tvuplayer.c" />
<ClCompile Include="..\..\..\src\lib\protocols\usenet.c" />
<ClCompile Include="..\..\..\src\lib\protocols\veohtv.c" />
<ClCompile Include="..\..\..\src\lib\protocols\viber.c" />
<ClCompile Include="..\..\..\src\lib\protocols\vmware.c" />
<ClCompile Include="..\..\..\src\lib\protocols\vnc.c" />
<ClCompile Include="..\..\..\src\lib\protocols\warcraft3.c" />
<ClCompile Include="..\..\..\src\lib\protocols\winmx.c" />
<ClCompile Include="..\..\..\src\lib\protocols\world_of_kung_fu.c" />
<ClCompile Include="..\..\..\src\lib\protocols\world_of_warcraft.c" />
<ClCompile Include="..\..\..\src\lib\protocols\xbox.c" />
<ClCompile Include="..\..\..\src\lib\protocols\xdmcp.c" />
<ClCompile Include="..\..\..\src\lib\protocols\yahoo.c" />
<ClCompile Include="..\..\..\src\lib\protocols\zattoo.c" />
<ClCompile Include="..\..\..\src\lib\third_party\src\ahocorasick.c" />
<ClCompile Include="..\..\..\src\lib\third_party\src\node.c" />
<ClCompile Include="..\..\..\src\lib\third_party\src\sort.c" />
<ClCompile Include="..\..\pcapReader.c" />
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt.h" />
<ClInclude Include="..\..\..\src\include\linux_compat.h" />
<ClInclude Include="..\..\..\src\include\ndpi_api.h" />
<ClInclude Include="..\..\..\src\include\ndpi_debug_functions.h" />
<ClInclude Include="..\..\..\src\include\ndpi_define.h" />
<ClInclude Include="..\..\..\src\include\ndpi_macros.h" />
<ClInclude Include="..\..\..\src\include\ndpi_main.h" />
<ClInclude Include="..\..\..\src\include\ndpi_protocols.h" />
<ClInclude Include="..\..\..\src\include\ndpi_protocols_osdpi.h" />
<ClInclude Include="..\..\..\src\include\ndpi_protocol_history.h" />
<ClInclude Include="..\..\..\src\include\ndpi_public_functions.h" />
<ClInclude Include="..\..\..\src\include\ndpi_structs.h" />
<ClInclude Include="..\..\..\src\include\ndpi_utils.h" />
<ClInclude Include="..\..\..\src\lib\third_party\include\actypes.h" />
<ClInclude Include="..\..\..\src\lib\third_party\include\ahocorasick.h" />
<ClInclude Include="..\..\..\src\lib\third_party\include\node.h" />
<ClInclude Include="..\..\..\src\lib\third_party\include\sort.h" />
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
</Project>
@@ -1,465 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup>
<Filter Include="Source Files">
<UniqueIdentifier>{4FC737F1-C7A5-4376-A066-2A32D752A2FF}</UniqueIdentifier>
<Extensions>cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx</Extensions>
</Filter>
<Filter Include="Header Files">
<UniqueIdentifier>{93995380-89BD-4b04-88EB-625FBE52EBFB}</UniqueIdentifier>
<Extensions>h;hpp;hxx;hm;inl;inc;xsd</Extensions>
</Filter>
<Filter Include="Resource Files">
<UniqueIdentifier>{67DA6AB6-F800-4c08-8B7A-83BB121AAD01}</UniqueIdentifier>
<Extensions>rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav;mfcribbon-ms</Extensions>
</Filter>
</ItemGroup>
<ItemGroup>
<ClCompile Include="..\..\pcapReader.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\ndpi_main.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\zattoo.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\afp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\aimini.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\applejuice.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\armagetron.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\battlefield.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\bgp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\bittorrent.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ciscovpn.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\citrix.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\corba.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\crossfire.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dcerpc.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dhcp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dhcpv6.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\directconnect.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\directdownloadlink.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dns.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dofus.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\dropbox.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\edonkey.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\fasttrack.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\feidian.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\fiesta.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\filetopia.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\flash.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\florensia.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ftp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\gadu_gadu.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\gnutella.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\gtp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\guildwars.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\h323.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\halflife2_and_mods.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\http.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\http_activesync.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\i23v5.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\iax.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\icecast.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\imesh.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ipp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\irc.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\jabber.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\kerberos.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\kontiki.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ldap.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\lotus_notes.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mail_imap.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mail_pop.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mail_smtp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\maplestory.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mdns.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\meebo.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mgcp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mms.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\msn.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mssql.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\mysql.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\netbios.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\netflow.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\nfs.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\noe.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\non_tcp_udp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ntp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\openft.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\openvpn.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\oracle.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\oscar.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\pando.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\pcanywhere.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\popo.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\postgres.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\pplive.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ppstream.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\pptp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\qq.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\quake.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\quic.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\radius.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\rdp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\rsync.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\rtcp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\rtp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\rtsp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\sflow.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\shoutcast.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\sip.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\skinny.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\skype.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\smb.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\snmp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\socrates.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\sopcast.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\soulseek.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\spotify.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ssdp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ssh.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\ssl.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\stealthnet.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\steam.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\stun.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\syslog.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tcp_udp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tds.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\teamspeak.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\teamviewer.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\telnet.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tftp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\thunder.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tor.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tvants.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\tvuplayer.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\usenet.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\veohtv.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\viber.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\vmware.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\vnc.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\warcraft3.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\winmx.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\world_of_kung_fu.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\world_of_warcraft.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\xbox.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\xdmcp.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\protocols\yahoo.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\third_party\src\ahocorasick.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\third_party\src\node.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\src\lib\third_party\src\sort.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClCompile Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt1.c">
<Filter>Source Files</Filter>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ClInclude Include="..\..\..\src\lib\third_party\include\actypes.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\lib\third_party\include\ahocorasick.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\lib\third_party\include\node.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\lib\third_party\include\sort.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_api.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_debug_functions.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_define.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_macros.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_main.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_protocol_history.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_protocols.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_protocols_osdpi.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_public_functions.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_structs.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\ndpi_utils.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\src\include\linux_compat.h">
<Filter>Header Files</Filter>
</ClInclude>
<ClInclude Include="..\..\..\..\..\nprobe\nprobe-win-32\getopt.h">
<Filter>Header Files</Filter>
</ClInclude>
</ItemGroup>
</Project>
@@ -1,3 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
</Project>
-462
View File
@@ -1,462 +0,0 @@
/*
* intrusion_detection.c
*
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#include "intrusion_detection.h"
double normalize(ndpi_norm_value* tresholds){
if(tresholds->upper_bound != tresholds->lower_bound){
tresholds->norm_value = (tresholds->value - tresholds->lower_bound) / (tresholds->upper_bound - tresholds->lower_bound);
}else{
if(tresholds->value > tresholds->upper_bound){
tresholds->norm_value = 1 + (tresholds->value - tresholds->lower_bound) / tresholds->upper_bound;
}else{
tresholds->norm_value = 1 - (tresholds->value - tresholds->lower_bound) / tresholds->upper_bound;
}
}
if(tresholds->norm_value >= 0){
return tresholds->norm_value * tresholds->weight;
}
else{
return (1 - tresholds->norm_value) * tresholds->weight;
}
}
double get_flow_score(ndpi_norm_value* scores, int n_metrics){
double flow_score = 0;
for(int i=0; i<n_metrics; i++){
flow_score += normalize(&scores[i]);
}
return flow_score;
}
/* ********************************** */
double Ddos_score(struct ndpi_flow_info* flow){
int n_metrics = 6;
ndpi_norm_value* scores = malloc(n_metrics * sizeof(ndpi_norm_value));
/* pktlen_c_to_s_avg */
int i = 0;
scores[i].lower_bound = 70.0;
scores[i].upper_bound = 263.4799999999999;
scores[i].weight = 0.21257330032661592;
scores[i].value = ndpi_data_average(flow->pktlen_c_to_s);
/* pktlen_s_to_c_max */
i++;
scores[i].lower_bound = 90.0;
scores[i].upper_bound = 2974.0;
scores[i].weight = 0.21073785073559176;
scores[i].value = ndpi_data_max(flow->pktlen_s_to_c);
/* pktlen_s_to_c_avg */
i++;
scores[i].lower_bound = 72.7;
scores[i].upper_bound = 1130.4199999999996;
scores[i].weight = 0.21257330032661592;
scores[i].value = ndpi_data_average(flow->pktlen_s_to_c);
/* pktlen_s_to_c_stddev */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 906.0;
scores[i].weight = 0.20990954527912953;
scores[i].value = ndpi_data_stddev(flow->pktlen_s_to_c);
/* fin */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.07710300166602348;
scores[i].value = flow->fin_count;
/* s_to_c_fin */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.07710300166602348;
scores[i].value = flow->dst2src_fin_count;
// sum = 1.0
double flow_score = get_flow_score(scores, n_metrics);
free(scores);
return flow_score;
}
double Dos_goldeneye_score(struct ndpi_flow_info* flow){
int n_metrics = 6;
ndpi_norm_value* scores = malloc(n_metrics * sizeof(ndpi_norm_value));
/* pktlen_s_to_c_max */
int i = 0;
scores[i].lower_bound = 74.0;
scores[i].upper_bound = 3292.6699999999764;
scores[i].weight = 0.3123007140611667;
scores[i].value = ndpi_data_max(flow->pktlen_s_to_c);
/* pktlen_s_to_c_avg */
i++;
scores[i].lower_bound = 68.7;
scores[i].upper_bound = 1354.0569999999987;
scores[i].weight = 0.23802038891633356;
scores[i].value = ndpi_data_average(flow->pktlen_s_to_c);
/* pktlen_s_to_c_stddev */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 959.4469999999993;
scores[i].weight = 0.3111779763775991;
scores[i].value = ndpi_data_stddev(flow->pktlen_s_to_c);
/* syn */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.0464364305923564;
scores[i].value = flow->syn_count;
/* c_to_s_syn */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 1.0;
scores[i].weight = 0.04562805946018772;
scores[i].value = flow->src2dst_syn_count;
/* s_to_c_syn */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.0464364305923564;
scores[i].value = flow->dst2src_syn_count;
// sum = 0.9999999999999998
double flow_score = get_flow_score(scores, n_metrics);
free(scores);
return flow_score;
}
double Dos_hulk_score(struct ndpi_flow_info* flow){
double f = (double)flow->first_seen_ms/1000.0, l = (double)flow->last_seen_ms/1000.0;
int n_metrics = 6;
ndpi_norm_value* scores = malloc(n_metrics * sizeof(ndpi_norm_value));
/* duration */
int i = 0;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 539.40668006422;
scores[i].weight = 0.16666666666666666;
scores[i].value = (l - f);
/* src2dst_packets */
i++;
scores[i].lower_bound = 2.0;
scores[i].upper_bound = 41.0;
scores[i].weight = 0.16666666666666666;
scores[i].value = flow->src2dst_packets;
/* dst2src_packets */
i++;
scores[i].lower_bound = 2.0;
scores[i].upper_bound = 45.0;
scores[i].weight = 0.16666666666666666;
scores[i].value = flow->dst2src_packets;
/* src2dst_bytes */
i++;
scores[i].lower_bound = 146.0;
scores[i].upper_bound = 6306.300000000001;
scores[i].weight = 0.16666666666666666;
scores[i].value = flow->src2dst_bytes;
/* ack */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 82.0;
scores[i].weight = 0.16666666666666666;
scores[i].value = flow->ack_count;
/* syn */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.16666666666666666;
scores[i].value = flow->syn_count;
// sum = 0.9999999999999999
double flow_score = get_flow_score(scores, n_metrics);
free(scores);
return flow_score;
}
double Dos_slow_score(struct ndpi_flow_info* flow){
int n_metrics = 6;
ndpi_norm_value* scores = malloc(n_metrics * sizeof(ndpi_norm_value));
/* pktlen_s_to_c_max */
int i = 0;
scores[i].lower_bound = 90.0;
scores[i].upper_bound = 3135.0;
scores[i].weight = 0.1760747755022144;
scores[i].value = ndpi_data_max(flow->pktlen_s_to_c);
/* pktlen_s_to_c_avg */
i++;
scores[i].lower_bound = 80.37100000000001;
scores[i].upper_bound = 1292.5900000000008;
scores[i].weight = 0.17600137023171597;
scores[i].value = ndpi_data_average(flow->pktlen_s_to_c);
/* dst2src_bytes */
i++;
scores[i].lower_bound = 262.0;
scores[i].upper_bound = 53227.80000000002;
scores[i].weight = 0.16919914849886225;
scores[i].value = flow->dst2src_bytes;
/* syn */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.168000195747388;
scores[i].value = flow->syn_count;
/* c_to_s_syn */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 1.0;
scores[i].weight = 0.14272431427243143;
scores[i].value = flow->src2dst_syn_count;
/* s_to_c_syn */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.168000195747388;
scores[i].value = flow->dst2src_syn_count;
// sum = 1.0
double flow_score = get_flow_score(scores, n_metrics);
free(scores);
return flow_score;
}
double Ftp_patator_score(struct ndpi_flow_info* flow){
int n_metrics = 6;
ndpi_norm_value* scores = malloc(n_metrics * sizeof(ndpi_norm_value));
/* iat_flow_min */
int i = 0;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 24.0;
scores[i].weight = 0.002732919254658385;
scores[i].value = ndpi_data_min(flow->iat_flow);
/* pktlen_s_to_c_max */
i++;
scores[i].lower_bound = 90.0;
scores[i].upper_bound = 3393.0;
scores[i].weight = 0.007453416149068323;
scores[i].value = ndpi_data_max(flow->pktlen_s_to_c);
/* pktlen_s_to_c_avg */
i++;
scores[i].lower_bound = 81.3;
scores[i].upper_bound = 1315.021;
scores[i].weight = 0.9833540372670807;
scores[i].value = ndpi_data_average(flow->pktlen_s_to_c);
/* dst2src_bytes */
i++;
scores[i].lower_bound = 256.0;
scores[i].upper_bound = 56434.0;
scores[i].weight = 0.0034782608695652175;
scores[i].value = flow->dst2src_bytes;
/* fin */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.0014906832298136647;
scores[i].value = flow->fin_count;
/* rst */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.0014906832298136647;
scores[i].value = flow->rst_count;
// sum = 1.0
double flow_score = get_flow_score(scores, n_metrics);
free(scores);
return flow_score;
}
double Hearthbleed_score(struct ndpi_flow_info* flow){
double f = (double)flow->first_seen_ms/1000.0, l = (double)flow->last_seen_ms/1000.0;
int n_metrics = 6;
ndpi_norm_value* scores = malloc(n_metrics * sizeof(ndpi_norm_value));
/* iat_flow_max */
int i = 0;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 595213.3999999999;
scores[i].weight = 0.16666666666666666;
scores[i].value = ndpi_data_max(flow->iat_flow);
/* iat_flow_stddev */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 245377.74799999973;
scores[i].weight = 0.16666666666666666;
scores[i].value = ndpi_data_stddev(flow->iat_flow);
/* pktlen_s_to_c_max */
i++;
scores[i].lower_bound = 74.0;
scores[i].upper_bound = 3380.0;
scores[i].weight = 0.16666666666666666;
scores[i].value = ndpi_data_max(flow->pktlen_s_to_c);
/* pktlen_s_to_c_avg */
i++;
scores[i].lower_bound = 70.0;
scores[i].upper_bound = 1344.6399999999996;
scores[i].weight = 0.16666666666666666;
scores[i].value = ndpi_data_average(flow->pktlen_s_to_c);
/* pktlen_s_to_c_stddev */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 944.6399999999996;
scores[i].weight = 0.16666666666666666;
scores[i].value = ndpi_data_stddev(flow->pktlen_s_to_c);
/* duration */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 711.6677598000391;
scores[i].weight = 0.16666666666666666;
scores[i].value = (l - f);
// sum = 0.9999999999999999
double flow_score = get_flow_score(scores, n_metrics);
free(scores);
return flow_score;
}
double Infiltration_score(struct ndpi_flow_info* flow){
int n_metrics = 6;
ndpi_norm_value* scores = malloc(n_metrics * sizeof(ndpi_norm_value));
/* pktlen_c_to_s_max */
int i = 0;
scores[i].lower_bound = 72.0;
scores[i].upper_bound = 1840.739999999998;
scores[i].weight = 0.11937557392102846;
scores[i].value = ndpi_data_max(flow->pktlen_c_to_s);
/* pktlen_c_to_s_avg */
i++;
scores[i].lower_bound = 70.0;
scores[i].upper_bound = 296.56599999999816;
scores[i].weight = 0.12526782981328435;
scores[i].value = ndpi_data_average(flow->pktlen_c_to_s);
/* pktlen_s_to_c_max */
i++;
scores[i].lower_bound = 90.0;
scores[i].upper_bound = 3496.1399999999776;
scores[i].weight = 0.13927150290786652;
scores[i].value = ndpi_data_max(flow->pktlen_s_to_c);
/* pktlen_s_to_c_avg */
i++;
scores[i].lower_bound = 72.6;
scores[i].upper_bound = 1367.7959999999991;
scores[i].weight = 0.12182430364248545;
scores[i].value = ndpi_data_average(flow->pktlen_s_to_c);
/* src2dst_bytes */
i++;
scores[i].lower_bound = 144.0;
scores[i].upper_bound = 7847.69999999999;
scores[i].weight = 0.12059993878175697;
scores[i].value = flow->src2dst_bytes;
/* dst2src_bytes */
i++;
scores[i].lower_bound = 236.0;
scores[i].upper_bound = 74486.7799999998;
scores[i].weight = 0.3736608509335782;
scores[i].value = flow->dst2src_bytes;
// sum = 1.0
double flow_score = get_flow_score(scores, n_metrics);
free(scores);
return flow_score;
}
double Ssh_patator_score(struct ndpi_flow_info* flow){
int n_metrics = 6;
ndpi_norm_value* scores = malloc(n_metrics * sizeof(ndpi_norm_value));
/* fin */
int i = 0;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.0033738191632928477;
scores[i].value = flow->fin_count;
/* psh */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 30.0;
scores[i].weight = 0.33076923076923076;
scores[i].value = flow->psh_count;
/* c_to_s_syn */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 1.0;
scores[i].weight = 0.0004048582995951417;
scores[i].value = flow->src2dst_syn_count;
/* c_to_s_psh */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 12.0;
scores[i].weight = 0.33130904183535764;
scores[i].value = flow->src2dst_psh_count;
/* s_to_c_fin */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 2.0;
scores[i].weight = 0.0033738191632928477;
scores[i].value = flow->dst2src_fin_count;
/* s_to_c_psh */
i++;
scores[i].lower_bound = 0.0;
scores[i].upper_bound = 30.0;
scores[i].weight = 0.33076923076923076;
scores[i].value = flow->dst2src_psh_count;
// sum = 1.0
double flow_score = get_flow_score(scores, n_metrics);
free(scores);
return flow_score;
}
-69
View File
@@ -1,69 +0,0 @@
/*
* intrusion_detection.h
*
* Copyright (C) 2011-21 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#ifndef _INTRUSION_DETECTION_H_
#define _INTRUSION_DETECTION_H_
/*
Code to detect attacks reported in
https://www.unb.ca/cic/datasets/ids-2017.html
https://www.unb.ca/cic/datasets/ids-2018.html
*/
#include <stdio.h>
#include <stdlib.h>
#include "reader_util.h"
#include "ndpi_api.h"
typedef struct norm_values{
double upper_bound;
double lower_bound;
double weight;
double value;
double norm_value;
}ndpi_norm_value;
double normalize(ndpi_norm_value* tresholds);
double get_flow_score(ndpi_norm_value* scores, int n_metrics);
/* ********************************** */
double Ddos_score(struct ndpi_flow_info* flow);
double Dos_goldeneye_score(struct ndpi_flow_info* flow);
double Dos_hulk_score(struct ndpi_flow_info* flow);
double Dos_slow_score(struct ndpi_flow_info* flow);
double Ftp_patator_score(struct ndpi_flow_info* flow);
double Hearthbleed_score(struct ndpi_flow_info* flow);
double Infiltration_score(struct ndpi_flow_info* flow);
double Ssh_patator_score(struct ndpi_flow_info* flow);
#endif /* _INTRUSION_DETECTION_H_ */
+108 -109
View File
@@ -1,109 +1,108 @@
################################################################
# abuse.ch Suricata JA3 Fingerprint Blacklist (CSV) #
# For Suricata 4.1.0 or newer #
# Last updated: 2020-04-09 06:48:14 UTC #
# #
# Terms Of Use: https://sslbl.abuse.ch/blacklist/ #
# For questions please contact sslbl [at] abuse.ch #
################################################################
#
# ja3_md5,Firstseen,Lastseen,Listingreason
b386946a5a44d1ddcc843bc75336dfce,2017-07-14 18:08:15,2019-07-27 20:42:54,Dridex
8991a387e4cc841740f25d6f5139f92d,2017-07-14 19:02:03,2019-07-28 00:34:38,Adware
cb98a24ee4b9134448ffb5714fd870ac,2017-07-14 19:48:28,2019-05-22 03:22:38,Dridex
1aa7bf8b97e540ca5edd75f7b8384bfa,2017-07-14 20:23:38,2019-07-28 01:38:22,TrickBot
3d89c0dfb1fa44911b8fa7523ef8dedb,2017-07-15 04:23:45,2021-02-01 18:23:25,Adware
bc6c386f480ee97b9d9e52d472b772d8,2017-07-15 10:57:38,2021-02-21 13:13:30,Adware
8f52d1ce303fb4a6515836aec3cc16b1,2017-07-15 19:05:11,2019-07-27 20:00:57,TrickBot
d6f04b5a910115f4b50ecec09d40a1df,2017-07-15 19:42:24,2018-10-14 08:12:51,Dridex
35c0a31c481927f022a3b530255ac080,2017-07-15 19:43:19,2021-02-06 03:21:55,Tofsee
d551fafc4f40f1dec2bb45980bfa9492,2017-07-15 19:59:29,2020-11-16 13:06:20,Adware
e330bca99c8a5256ae126a55c4c725c5,2017-07-15 19:59:29,2021-01-13 00:29:37,Adware
b8f81673c0e1d29908346f3bab892b9b,2017-07-16 01:32:03,2018-12-17 06:08:03,Adware
83e04bc58d402f9633983cbf22724b02,2017-07-16 01:32:03,2021-01-14 14:23:42,Adware
70722097d1fe1d78d8c2164640ab6df4,2017-07-16 02:39:08,2021-02-22 04:56:57,Tofsee
9c2589e1c0e9f533a022c6205f9719e1,2017-07-16 08:37:17,2021-02-19 08:46:32,Adware
849b04bdbd1d2b983f6e8a457e0632a8,2017-07-16 08:37:17,2021-02-19 08:46:32,Adware
16efcf0e00504ddfedde13bfea997952,2017-07-16 19:45:45,2020-12-23 15:10:32,Adware
4d7a28d6f2263ed61de88ca66eb011e3,2017-07-16 21:20:29,2020-12-08 18:10:55,Tofsee
550dce18de1bb143e69d6dd9413b8355,2017-07-16 22:17:20,2018-12-21 07:04:50,Adware
c50f6a8b9173676b47ba6085bd0c6cee,2017-07-16 22:38:41,2019-05-21 09:42:17,TrickBot
20dd18bdd3209ea718989030a6f93364,2017-07-18 10:22:58,2019-04-28 09:23:31,Adware
8498fe4268764dbf926a38283e9d3d8f,2017-07-18 10:22:58,2020-12-31 02:06:27,Adware
590a232d04d56409fab72e752a8a2634,2017-07-18 18:53:24,2020-10-11 20:48:33,Tofsee
51a7ad14509fd614c7bb3a50c4982b8c,2017-07-19 07:28:19,2019-07-14 11:58:32,JBifrost
96eba628dcb2b47607192ba74a3b55ba,2017-07-19 18:53:48,2021-01-04 12:45:08,Tofsee
df5c30e670dba99f9270ed36060cf054,2017-07-20 17:44:07,2018-04-11 15:57:59,Tofsee
098f55e27d8c4b0a590102cbdb3a5f3a,2017-07-21 09:52:01,2019-04-08 01:09:54,Adware
46efd49abcca8ea9baa932da68fdb529,2017-07-22 14:07:36,2021-02-20 20:45:32,Adware
29085f03f8e8a03f0b399c5c7cf0b0b8,2017-07-22 14:07:36,2021-02-21 20:56:01,Adware
d7150af4514b868defb854db0f62a441,2017-07-23 09:39:24,2018-07-24 01:04:58,Tofsee
03e186a7f83285e93341de478334006e,2017-07-24 18:17:14,2021-02-13 23:55:58,Tofsee
3cda52da4ade09f1f781ad2e82dcfa20,2017-07-30 18:41:36,2019-05-21 17:34:18,Quakbot
b13d01846ad7a14a70bf030a16775c78,2017-08-08 07:12:49,2021-02-21 01:32:12,Adware
1543a7c46633acf71e8401baccbd0568,2017-08-08 21:32:28,2020-11-10 05:30:17,Tofsee
1d095e68489d3c535297cd8dffb06cb9,2017-08-12 19:56:28,2020-10-28 11:06:23,Tofsee
93d056782d649deb51cda44ecb714bb0,2017-08-28 12:20:47,2019-04-15 23:47:27,Adware
698e36219f3979420fa2581b21dac7ec,2017-08-28 12:20:47,2020-12-31 02:06:31,Adware
1712287800ac91b34cadd5884ce85568,2017-08-28 16:01:59,2021-01-23 09:36:32,TorrentLocker
5e573c9c9f8ba720ef9b18e9fce2e2f7,2017-08-30 13:44:56,2021-02-17 17:11:34,Adware
f6fd83a21f9f3c5f9ff7b5c63bbc179d,2017-10-20 08:03:21,2018-11-06 06:42:12,Adware
92579701f145605e9edc0b01a901c6d5,2017-10-23 00:10:48,2021-02-18 22:56:49,Adware
a61299f9b501adcf680b9275d79d4ac6,2017-11-04 18:03:59,2020-04-21 17:08:24,Tofsee
b2b61db7b9490a60d270ccb20b462826,2017-11-14 20:12:03,2021-02-18 22:56:49,Adware
7dcce5b76c8b17472d024758970a406b,2017-11-22 12:42:46,2020-12-22 15:15:49,Tofsee
534ce2dbc413c68e908363b5df0ae5e0,2017-12-22 09:36:21,2019-07-27 15:22:33,TrickBot
fb00055a1196aeea8d1bc609885ba953,2018-01-01 22:49:25,2019-04-09 06:58:58,TrickBot
a50a861119aceb0ccc74902e8fddb618,2018-01-02 08:16:23,2018-07-05 02:33:08,Tofsee
e7643725fcff971e3051fe0e47fc2c71,2018-01-31 08:06:13,2020-03-25 16:19:48,Tofsee
7c410ce832e848a3321432c9a82e972b,2018-01-31 20:04:25,2021-02-21 22:51:31,Tofsee
da949afd9bd6df820730f8f171584a71,2018-02-03 05:19:37,2021-02-21 05:20:43,Tofsee
906004246f3ba5e755b043c057254a29,2018-03-11 08:25:38,2018-04-14 00:59:16,Tofsee
fd80fa9c6120cdeea8520510f3c644ac,2018-03-11 09:34:30,2021-02-21 22:16:39,Tofsee
b90bdbe961a648f0427db21aaa6ccb59,2018-03-11 10:37:43,2020-05-29 23:39:01,Tofsee
1fe4c7a3544eb27afec2adfb3a3dbf60,2018-03-11 19:23:08,2021-02-20 11:19:52,Tofsee
c201b92f8b483fa388be174d6689f534,2018-03-12 13:43:52,2021-01-28 06:17:06,Gozi
9f62c4f26b90d3d757bea609e82f2eaf,2018-03-13 06:23:41,2020-12-18 16:07:31,Tofsee
1be3ecebe5aa9d3654e6e703d81f6928,2018-03-13 11:50:02,2021-02-14 23:03:13,Ransomware.Troldesh
e3b2ab1f9a56f2fb4c9248f2f41631fa,2018-03-15 01:06:34,2021-02-22 10:36:48,Tofsee
dff8a0aa1c904aaea76c5bf624e88333,2018-03-18 09:41:15,2020-10-27 09:50:24,Tofsee
17fd49722f8d11f3d76dce84f8e099a7,2018-03-19 23:02:27,2020-12-09 11:32:21,Tofsee
911479ac8a0813ed1241b3686ccdade9,2018-03-19 23:24:59,2020-03-30 04:09:18,Tofsee
c5deb9465d47232dd48772f9c4d14679,2018-03-22 15:42:48,2021-02-17 07:39:32,Tofsee
f22bdd57e3a52de86cda40da2d84e83b,2018-03-27 13:40:19,2019-01-20 14:31:39,Tofsee
d18a4da84af59e1108862a39bae7c9d4,2018-04-03 00:40:51,2021-02-06 01:53:12,Tofsee
2d8794cb7b52b777bee2695e79c15760,2018-04-04 06:56:37,2021-02-10 08:22:31,Ransomware
40adfd923eb82b89d8836ba37a19bca1,2018-04-15 15:49:08,2021-02-21 22:22:56,CoinMiner
1aee0238942d453d679fc1e37a303387,2018-05-13 01:59:49,2021-02-10 13:19:24,Tofsee
2092e1fffb45d7e4a19a57f9bc5e203a,2018-05-16 21:59:36,2018-09-05 01:58:33,Adware
bffa4501966196d3d6e90cee1f88fc89,2018-06-07 15:08:04,2020-03-16 00:03:44,Tofsee
807fca46d9d0cf63adf4e5e80e414bbe,2018-06-07 16:51:03,2021-02-20 03:15:38,Tofsee
fb58831f892190644fe44e25bc830b45,2018-06-08 12:07:59,2019-05-31 21:02:37,Adware
0cc1e84568e471aa1d62ad4158ade6b5,2018-06-24 10:50:47,2021-02-10 21:48:57,Tofsee
d2935c58fe676744fecc8614ee5356c7,2018-08-14 21:48:41,2021-02-21 21:44:55,Adwind
8916410db85077a5460817142dcbc8de,2018-08-21 12:32:28,2021-02-21 20:02:19,TrickBot
c5235d3a8b9934b7fbbd204d50bc058d,2018-08-23 17:36:08,2019-10-13 05:11:09,Gootkit
57f3642b4e37e28f5cbe3020c9331b4c,2018-08-28 15:54:53,2021-02-22 11:09:43,Gozi
e62a5f4d538cbf169c2af71bec2399b4,2018-08-30 15:45:40,2021-02-21 20:36:55,TrickBot
51c64c77e60f3980eea90869b68c58a8,2018-08-30 21:04:57,2021-02-21 22:09:16,Dridex
7691297bcb20a41233fd0a0baa0a3628,2018-09-17 02:50:05,2021-02-21 17:05:57,Adware
7dd50e112cd23734a310b90f6f44a7cd,2018-09-17 17:54:58,2021-02-22 11:14:04,Quakbot
52c7396a501e4fecbdfa99c5408334ac,2018-09-18 00:29:04,2019-12-03 17:24:02,Tofsee
f735bbc6b69723b9df7b0e7ef27872af,2018-10-02 18:04:16,2021-02-19 06:55:39,TrickBot
49ed2ef3f1321e5f044f1e71b0e6fdd5,2018-10-02 18:04:17,2021-02-19 06:55:39,TrickBot
d76ee64fb7273733cbe455ac81c292e6,2018-11-16 13:26:39,2018-11-18 19:19:36,Tofsee
8f6c918dcb585ebbea05e2cc94530e3d,2018-11-16 13:26:41,2020-05-06 15:45:21,Tofsee
34f14a69ad7009ca5863379218af17f3,2018-11-17 05:17:22,2021-01-28 08:19:18,Tofsee
c2b4710c6888a5d47befe865c8e6fb19,2018-11-29 20:46:04,2020-12-16 15:08:09,Tofsee
decfb48a53789ebe081b88aabb58ee34,2018-12-21 09:06:16,2021-02-16 12:28:53,Adwind
08a8a4e85b25ac42e1490bc85cfdb5ce,2019-01-30 02:48:34,2020-10-27 09:50:19,Tofsee
c0220cd64849a629397a9cb68f78a0ea,2019-03-24 00:12:32,2021-02-13 05:13:33,Tofsee
7a29c223fb122ec64d10f0a159e07996,2019-06-09 22:55:29,2020-10-27 09:50:26,Tofsee
44dab16d680ef93487bc16ad23b3ffb1,2019-06-09 22:55:29,2020-10-27 09:50:25,Tofsee
70a04365be5bbd4653698bebeb43ce68,2019-07-02 06:26:56,2020-05-30 04:19:00,Tofsee
d81d654effb94714a4086734fa0adad9,2019-07-16 23:29:02,2020-10-27 09:50:21,Tofsee
25d74b7b4b779eb1efd4b31d26d651c6,2019-08-03 20:15:33,2020-07-14 21:43:25,Tofsee
fc2299d5b2964cd242c5a2c8c531a5f0,2019-08-09 23:56:32,2021-02-22 08:49:18,Tofsee
32926ca3e59f0413d0b98725454594f5,2019-09-12 06:56:10,2020-10-27 21:49:31,Tofsee
ffefafdb86336d057eda5fdf02b3d5ce,2019-10-26 07:31:49,2020-07-25 00:14:09,Tofsee
# END (98) entries
################################################################
# abuse.ch Suricata JA3 Fingerprint Blacklist (CSV) #
# For Suricata 4.1.0 or newer #
# Last updated: 2021-08-03 14:33:44 UTC #
# #
# Terms Of Use: https://sslbl.abuse.ch/blacklist/ #
# For questions please contact sslbl [at] abuse.ch #
################################################################
#
# ja3_md5,Firstseen,Lastseen,Listingreason
b386946a5a44d1ddcc843bc75336dfce,2017-07-14 18:08:15,2019-07-27 20:42:54,Dridex
8991a387e4cc841740f25d6f5139f92d,2017-07-14 19:02:03,2019-07-28 00:34:38,Adware
cb98a24ee4b9134448ffb5714fd870ac,2017-07-14 19:48:28,2019-05-22 03:22:38,Dridex
1aa7bf8b97e540ca5edd75f7b8384bfa,2017-07-14 20:23:38,2019-07-28 01:38:22,TrickBot
3d89c0dfb1fa44911b8fa7523ef8dedb,2017-07-15 04:23:45,2021-02-01 18:23:25,Adware
bc6c386f480ee97b9d9e52d472b772d8,2017-07-15 10:57:38,2021-03-13 07:33:39,Adware
8f52d1ce303fb4a6515836aec3cc16b1,2017-07-15 19:05:11,2019-07-27 20:00:57,TrickBot
d6f04b5a910115f4b50ecec09d40a1df,2017-07-15 19:42:24,2018-10-14 08:12:51,Dridex
35c0a31c481927f022a3b530255ac080,2017-07-15 19:43:19,2021-04-10 12:54:04,Tofsee
e330bca99c8a5256ae126a55c4c725c5,2017-07-15 19:59:29,2021-01-13 00:29:37,Adware
d551fafc4f40f1dec2bb45980bfa9492,2017-07-15 19:59:29,2020-11-16 13:06:20,Adware
83e04bc58d402f9633983cbf22724b02,2017-07-16 01:32:03,2021-03-02 04:07:43,Adware
b8f81673c0e1d29908346f3bab892b9b,2017-07-16 01:32:03,2021-03-02 04:07:36,Adware
70722097d1fe1d78d8c2164640ab6df4,2017-07-16 02:39:08,2021-05-04 09:52:20,Tofsee
9c2589e1c0e9f533a022c6205f9719e1,2017-07-16 08:37:17,2021-07-25 08:33:18,Adware
849b04bdbd1d2b983f6e8a457e0632a8,2017-07-16 08:37:17,2021-07-25 08:33:18,Adware
16efcf0e00504ddfedde13bfea997952,2017-07-16 19:45:45,2020-12-23 15:10:32,Adware
4d7a28d6f2263ed61de88ca66eb011e3,2017-07-16 21:20:29,2020-12-08 18:10:55,Tofsee
550dce18de1bb143e69d6dd9413b8355,2017-07-16 22:17:20,2018-12-21 07:04:50,Adware
c50f6a8b9173676b47ba6085bd0c6cee,2017-07-16 22:38:41,2019-05-21 09:42:17,TrickBot
590a232d04d56409fab72e752a8a2634,2017-07-18 18:53:24,2020-10-11 20:48:33,Tofsee
51a7ad14509fd614c7bb3a50c4982b8c,2017-07-19 07:28:19,2019-07-14 11:58:32,JBifrost
96eba628dcb2b47607192ba74a3b55ba,2017-07-19 18:53:48,2021-07-31 01:48:32,Tofsee
df5c30e670dba99f9270ed36060cf054,2017-07-20 17:44:07,2018-04-11 15:57:59,Tofsee
098f55e27d8c4b0a590102cbdb3a5f3a,2017-07-21 09:52:01,2019-04-08 01:09:54,Adware
29085f03f8e8a03f0b399c5c7cf0b0b8,2017-07-22 14:07:36,2021-04-11 06:42:45,Adware
46efd49abcca8ea9baa932da68fdb529,2017-07-22 14:07:36,2021-04-11 05:54:57,Adware
d7150af4514b868defb854db0f62a441,2017-07-23 09:39:24,2018-07-24 01:04:58,Tofsee
03e186a7f83285e93341de478334006e,2017-07-24 18:17:14,2021-03-20 07:45:40,Tofsee
3cda52da4ade09f1f781ad2e82dcfa20,2017-07-30 18:41:36,2019-05-21 17:34:18,Quakbot
b13d01846ad7a14a70bf030a16775c78,2017-08-08 07:12:49,2021-04-10 00:03:33,Adware
1543a7c46633acf71e8401baccbd0568,2017-08-08 21:32:28,2021-04-08 19:50:46,Tofsee
1d095e68489d3c535297cd8dffb06cb9,2017-08-12 19:56:28,2020-10-28 11:06:23,Tofsee
698e36219f3979420fa2581b21dac7ec,2017-08-28 12:20:47,2020-12-31 02:06:31,Adware
93d056782d649deb51cda44ecb714bb0,2017-08-28 12:20:47,2019-04-15 23:47:27,Adware
1712287800ac91b34cadd5884ce85568,2017-08-28 16:01:59,2021-07-28 14:16:00,TorrentLocker
5e573c9c9f8ba720ef9b18e9fce2e2f7,2017-08-30 13:44:56,2021-03-13 07:33:38,Adware
f6fd83a21f9f3c5f9ff7b5c63bbc179d,2017-10-20 08:03:21,2018-11-06 06:42:12,Adware
92579701f145605e9edc0b01a901c6d5,2017-10-23 00:10:48,2021-07-25 17:07:34,Adware
a61299f9b501adcf680b9275d79d4ac6,2017-11-04 18:03:59,2020-04-21 17:08:24,Tofsee
b2b61db7b9490a60d270ccb20b462826,2017-11-14 20:12:03,2021-06-06 20:27:10,Adware
7dcce5b76c8b17472d024758970a406b,2017-11-22 12:42:46,2021-03-16 12:53:35,Tofsee
534ce2dbc413c68e908363b5df0ae5e0,2017-12-22 09:36:21,2019-07-27 15:22:33,TrickBot
fb00055a1196aeea8d1bc609885ba953,2018-01-01 22:49:25,2019-04-09 06:58:58,TrickBot
a50a861119aceb0ccc74902e8fddb618,2018-01-02 08:16:23,2018-07-05 02:33:08,Tofsee
e7643725fcff971e3051fe0e47fc2c71,2018-01-31 08:06:13,2020-03-25 16:19:48,Tofsee
7c410ce832e848a3321432c9a82e972b,2018-01-31 20:04:25,2021-08-01 06:13:14,Tofsee
da949afd9bd6df820730f8f171584a71,2018-02-03 05:19:37,2021-03-08 22:10:10,Tofsee
906004246f3ba5e755b043c057254a29,2018-03-11 08:25:38,2018-04-14 00:59:16,Tofsee
fd80fa9c6120cdeea8520510f3c644ac,2018-03-11 09:34:30,2021-08-11 12:34:00,Tofsee
b90bdbe961a648f0427db21aaa6ccb59,2018-03-11 10:37:43,2020-05-29 23:39:01,Tofsee
1fe4c7a3544eb27afec2adfb3a3dbf60,2018-03-11 19:23:08,2021-08-09 11:42:58,Tofsee
c201b92f8b483fa388be174d6689f534,2018-03-12 13:43:52,2021-01-28 06:17:06,Gozi
9f62c4f26b90d3d757bea609e82f2eaf,2018-03-13 06:23:41,2021-05-20 23:01:06,Tofsee
1be3ecebe5aa9d3654e6e703d81f6928,2018-03-13 11:50:02,2021-08-11 13:02:35,Ransomware.Troldesh
e3b2ab1f9a56f2fb4c9248f2f41631fa,2018-03-15 01:06:34,2021-07-02 21:51:49,Tofsee
dff8a0aa1c904aaea76c5bf624e88333,2018-03-18 09:41:15,2020-10-27 09:50:24,Tofsee
17fd49722f8d11f3d76dce84f8e099a7,2018-03-19 23:02:27,2021-08-01 00:40:52,Tofsee
911479ac8a0813ed1241b3686ccdade9,2018-03-19 23:24:59,2020-03-30 04:09:18,Tofsee
c5deb9465d47232dd48772f9c4d14679,2018-03-22 15:42:48,2021-03-23 00:34:25,Tofsee
f22bdd57e3a52de86cda40da2d84e83b,2018-03-27 13:40:19,2019-01-20 14:31:39,Tofsee
d18a4da84af59e1108862a39bae7c9d4,2018-04-03 00:40:51,2021-02-06 01:53:12,Tofsee
2d8794cb7b52b777bee2695e79c15760,2018-04-04 06:56:37,2021-07-26 08:07:00,Ransomware
40adfd923eb82b89d8836ba37a19bca1,2018-04-15 15:49:08,2021-04-11 04:42:47,CoinMiner
1aee0238942d453d679fc1e37a303387,2018-05-13 01:59:49,2021-07-30 12:27:07,Tofsee
2092e1fffb45d7e4a19a57f9bc5e203a,2018-05-16 21:59:36,2018-09-05 01:58:33,Adware
bffa4501966196d3d6e90cee1f88fc89,2018-06-07 15:08:04,2020-03-16 00:03:44,Tofsee
807fca46d9d0cf63adf4e5e80e414bbe,2018-06-07 16:51:03,2021-08-07 03:15:42,Tofsee
fb58831f892190644fe44e25bc830b45,2018-06-08 12:07:59,2021-07-20 01:39:05,Adware
0cc1e84568e471aa1d62ad4158ade6b5,2018-06-24 10:50:47,2021-06-21 02:35:57,Tofsee
d2935c58fe676744fecc8614ee5356c7,2018-08-14 21:48:41,2021-08-11 11:54:42,Adwind
8916410db85077a5460817142dcbc8de,2018-08-21 12:32:28,2021-08-11 15:00:50,TrickBot
c5235d3a8b9934b7fbbd204d50bc058d,2018-08-23 17:36:08,2019-10-13 05:11:09,Gootkit
57f3642b4e37e28f5cbe3020c9331b4c,2018-08-28 15:54:53,2021-08-11 13:05:18,Gozi
e62a5f4d538cbf169c2af71bec2399b4,2018-08-30 15:45:40,2021-08-11 09:48:52,TrickBot
51c64c77e60f3980eea90869b68c58a8,2018-08-30 21:04:57,2021-08-11 08:13:08,Dridex
7691297bcb20a41233fd0a0baa0a3628,2018-09-17 02:50:05,2021-08-11 12:20:33,Adware
7dd50e112cd23734a310b90f6f44a7cd,2018-09-17 17:54:58,2021-08-01 11:28:46,Quakbot
52c7396a501e4fecbdfa99c5408334ac,2018-09-18 00:29:04,2019-12-03 17:24:02,Tofsee
fc54e0d16d9764783542f0146a98b300,2018-09-24 12:33:44,2021-08-11 12:51:10,AsyncRAT
f735bbc6b69723b9df7b0e7ef27872af,2018-10-02 18:04:16,2021-08-11 07:25:14,TrickBot
49ed2ef3f1321e5f044f1e71b0e6fdd5,2018-10-02 18:04:17,2021-08-08 22:08:01,TrickBot
d76ee64fb7273733cbe455ac81c292e6,2018-11-16 13:26:39,2018-11-18 19:19:36,Tofsee
8f6c918dcb585ebbea05e2cc94530e3d,2018-11-16 13:26:41,2020-05-06 15:45:21,Tofsee
34f14a69ad7009ca5863379218af17f3,2018-11-17 05:17:22,2021-01-28 08:19:18,Tofsee
c2b4710c6888a5d47befe865c8e6fb19,2018-11-29 20:46:04,2021-08-03 23:37:22,Tofsee
decfb48a53789ebe081b88aabb58ee34,2018-12-21 09:06:16,2021-06-14 05:27:16,Adwind
08a8a4e85b25ac42e1490bc85cfdb5ce,2019-01-30 02:48:34,2020-10-27 09:50:19,Tofsee
c0220cd64849a629397a9cb68f78a0ea,2019-03-24 00:12:32,2021-07-31 00:26:06,Tofsee
7a29c223fb122ec64d10f0a159e07996,2019-06-09 22:55:29,2020-10-27 09:50:26,None
70a04365be5bbd4653698bebeb43ce68,2019-07-02 06:26:56,2020-05-30 04:19:00,Tofsee
d81d654effb94714a4086734fa0adad9,2019-07-16 23:29:02,2020-10-27 09:50:21,Tofsee
25d74b7b4b779eb1efd4b31d26d651c6,2019-08-03 20:15:33,2020-07-14 21:43:25,Tofsee
fc2299d5b2964cd242c5a2c8c531a5f0,2019-08-09 23:56:32,2021-08-11 02:20:22,Tofsee
32926ca3e59f0413d0b98725454594f5,2019-09-12 06:56:10,2021-04-08 19:50:43,Tofsee
ffefafdb86336d057eda5fdf02b3d5ce,2019-10-26 07:31:49,2020-07-25 00:14:09,Tofsee
8515076cbbca9dce33151b798f782456,2020-12-27 16:53:04,2021-08-11 15:06:36,BitRAT
# END (97) entries
1 ################################################################
2 # abuse.ch Suricata JA3 Fingerprint Blacklist (CSV) #
3 # For Suricata 4.1.0 or newer #
4 # Last updated: 2020-04-09 06:48:14 UTC # # Last updated: 2021-08-03 14:33:44 UTC #
5 # #
6 # Terms Of Use: https://sslbl.abuse.ch/blacklist/ #
7 # For questions please contact sslbl [at] abuse.ch #
8 ################################################################
9 #
10 # ja3_md5,Firstseen,Lastseen,Listingreason
11 b386946a5a44d1ddcc843bc75336dfce,2017-07-14 18:08:15,2019-07-27 20:42:54,Dridex
12 8991a387e4cc841740f25d6f5139f92d,2017-07-14 19:02:03,2019-07-28 00:34:38,Adware
13 cb98a24ee4b9134448ffb5714fd870ac,2017-07-14 19:48:28,2019-05-22 03:22:38,Dridex
14 1aa7bf8b97e540ca5edd75f7b8384bfa,2017-07-14 20:23:38,2019-07-28 01:38:22,TrickBot
15 3d89c0dfb1fa44911b8fa7523ef8dedb,2017-07-15 04:23:45,2021-02-01 18:23:25,Adware
16 bc6c386f480ee97b9d9e52d472b772d8,2017-07-15 10:57:38,2021-02-21 13:13:30,Adware bc6c386f480ee97b9d9e52d472b772d8,2017-07-15 10:57:38,2021-03-13 07:33:39,Adware
17 8f52d1ce303fb4a6515836aec3cc16b1,2017-07-15 19:05:11,2019-07-27 20:00:57,TrickBot
18 d6f04b5a910115f4b50ecec09d40a1df,2017-07-15 19:42:24,2018-10-14 08:12:51,Dridex
19 35c0a31c481927f022a3b530255ac080,2017-07-15 19:43:19,2021-02-06 03:21:55,Tofsee 35c0a31c481927f022a3b530255ac080,2017-07-15 19:43:19,2021-04-10 12:54:04,Tofsee
20 d551fafc4f40f1dec2bb45980bfa9492,2017-07-15 19:59:29,2020-11-16 13:06:20,Adware e330bca99c8a5256ae126a55c4c725c5,2017-07-15 19:59:29,2021-01-13 00:29:37,Adware
21 e330bca99c8a5256ae126a55c4c725c5,2017-07-15 19:59:29,2021-01-13 00:29:37,Adware d551fafc4f40f1dec2bb45980bfa9492,2017-07-15 19:59:29,2020-11-16 13:06:20,Adware
22 b8f81673c0e1d29908346f3bab892b9b,2017-07-16 01:32:03,2018-12-17 06:08:03,Adware 83e04bc58d402f9633983cbf22724b02,2017-07-16 01:32:03,2021-03-02 04:07:43,Adware
23 83e04bc58d402f9633983cbf22724b02,2017-07-16 01:32:03,2021-01-14 14:23:42,Adware b8f81673c0e1d29908346f3bab892b9b,2017-07-16 01:32:03,2021-03-02 04:07:36,Adware
24 70722097d1fe1d78d8c2164640ab6df4,2017-07-16 02:39:08,2021-02-22 04:56:57,Tofsee 70722097d1fe1d78d8c2164640ab6df4,2017-07-16 02:39:08,2021-05-04 09:52:20,Tofsee
25 9c2589e1c0e9f533a022c6205f9719e1,2017-07-16 08:37:17,2021-02-19 08:46:32,Adware 9c2589e1c0e9f533a022c6205f9719e1,2017-07-16 08:37:17,2021-07-25 08:33:18,Adware
26 849b04bdbd1d2b983f6e8a457e0632a8,2017-07-16 08:37:17,2021-02-19 08:46:32,Adware 849b04bdbd1d2b983f6e8a457e0632a8,2017-07-16 08:37:17,2021-07-25 08:33:18,Adware
27 16efcf0e00504ddfedde13bfea997952,2017-07-16 19:45:45,2020-12-23 15:10:32,Adware
28 4d7a28d6f2263ed61de88ca66eb011e3,2017-07-16 21:20:29,2020-12-08 18:10:55,Tofsee
29 550dce18de1bb143e69d6dd9413b8355,2017-07-16 22:17:20,2018-12-21 07:04:50,Adware
30 c50f6a8b9173676b47ba6085bd0c6cee,2017-07-16 22:38:41,2019-05-21 09:42:17,TrickBot
31 20dd18bdd3209ea718989030a6f93364,2017-07-18 10:22:58,2019-04-28 09:23:31,Adware 590a232d04d56409fab72e752a8a2634,2017-07-18 18:53:24,2020-10-11 20:48:33,Tofsee
32 8498fe4268764dbf926a38283e9d3d8f,2017-07-18 10:22:58,2020-12-31 02:06:27,Adware 51a7ad14509fd614c7bb3a50c4982b8c,2017-07-19 07:28:19,2019-07-14 11:58:32,JBifrost
33 590a232d04d56409fab72e752a8a2634,2017-07-18 18:53:24,2020-10-11 20:48:33,Tofsee 96eba628dcb2b47607192ba74a3b55ba,2017-07-19 18:53:48,2021-07-31 01:48:32,Tofsee
34 51a7ad14509fd614c7bb3a50c4982b8c,2017-07-19 07:28:19,2019-07-14 11:58:32,JBifrost df5c30e670dba99f9270ed36060cf054,2017-07-20 17:44:07,2018-04-11 15:57:59,Tofsee
35 96eba628dcb2b47607192ba74a3b55ba,2017-07-19 18:53:48,2021-01-04 12:45:08,Tofsee 098f55e27d8c4b0a590102cbdb3a5f3a,2017-07-21 09:52:01,2019-04-08 01:09:54,Adware
36 df5c30e670dba99f9270ed36060cf054,2017-07-20 17:44:07,2018-04-11 15:57:59,Tofsee 29085f03f8e8a03f0b399c5c7cf0b0b8,2017-07-22 14:07:36,2021-04-11 06:42:45,Adware
37 098f55e27d8c4b0a590102cbdb3a5f3a,2017-07-21 09:52:01,2019-04-08 01:09:54,Adware 46efd49abcca8ea9baa932da68fdb529,2017-07-22 14:07:36,2021-04-11 05:54:57,Adware
38 46efd49abcca8ea9baa932da68fdb529,2017-07-22 14:07:36,2021-02-20 20:45:32,Adware d7150af4514b868defb854db0f62a441,2017-07-23 09:39:24,2018-07-24 01:04:58,Tofsee
39 29085f03f8e8a03f0b399c5c7cf0b0b8,2017-07-22 14:07:36,2021-02-21 20:56:01,Adware 03e186a7f83285e93341de478334006e,2017-07-24 18:17:14,2021-03-20 07:45:40,Tofsee
40 d7150af4514b868defb854db0f62a441,2017-07-23 09:39:24,2018-07-24 01:04:58,Tofsee 3cda52da4ade09f1f781ad2e82dcfa20,2017-07-30 18:41:36,2019-05-21 17:34:18,Quakbot
41 03e186a7f83285e93341de478334006e,2017-07-24 18:17:14,2021-02-13 23:55:58,Tofsee b13d01846ad7a14a70bf030a16775c78,2017-08-08 07:12:49,2021-04-10 00:03:33,Adware
42 3cda52da4ade09f1f781ad2e82dcfa20,2017-07-30 18:41:36,2019-05-21 17:34:18,Quakbot 1543a7c46633acf71e8401baccbd0568,2017-08-08 21:32:28,2021-04-08 19:50:46,Tofsee
43 b13d01846ad7a14a70bf030a16775c78,2017-08-08 07:12:49,2021-02-21 01:32:12,Adware 1d095e68489d3c535297cd8dffb06cb9,2017-08-12 19:56:28,2020-10-28 11:06:23,Tofsee
44 1543a7c46633acf71e8401baccbd0568,2017-08-08 21:32:28,2020-11-10 05:30:17,Tofsee 698e36219f3979420fa2581b21dac7ec,2017-08-28 12:20:47,2020-12-31 02:06:31,Adware
45 1d095e68489d3c535297cd8dffb06cb9,2017-08-12 19:56:28,2020-10-28 11:06:23,Tofsee 93d056782d649deb51cda44ecb714bb0,2017-08-28 12:20:47,2019-04-15 23:47:27,Adware
46 93d056782d649deb51cda44ecb714bb0,2017-08-28 12:20:47,2019-04-15 23:47:27,Adware 1712287800ac91b34cadd5884ce85568,2017-08-28 16:01:59,2021-07-28 14:16:00,TorrentLocker
47 698e36219f3979420fa2581b21dac7ec,2017-08-28 12:20:47,2020-12-31 02:06:31,Adware 5e573c9c9f8ba720ef9b18e9fce2e2f7,2017-08-30 13:44:56,2021-03-13 07:33:38,Adware
48 1712287800ac91b34cadd5884ce85568,2017-08-28 16:01:59,2021-01-23 09:36:32,TorrentLocker f6fd83a21f9f3c5f9ff7b5c63bbc179d,2017-10-20 08:03:21,2018-11-06 06:42:12,Adware
49 5e573c9c9f8ba720ef9b18e9fce2e2f7,2017-08-30 13:44:56,2021-02-17 17:11:34,Adware 92579701f145605e9edc0b01a901c6d5,2017-10-23 00:10:48,2021-07-25 17:07:34,Adware
50 f6fd83a21f9f3c5f9ff7b5c63bbc179d,2017-10-20 08:03:21,2018-11-06 06:42:12,Adware a61299f9b501adcf680b9275d79d4ac6,2017-11-04 18:03:59,2020-04-21 17:08:24,Tofsee
51 92579701f145605e9edc0b01a901c6d5,2017-10-23 00:10:48,2021-02-18 22:56:49,Adware b2b61db7b9490a60d270ccb20b462826,2017-11-14 20:12:03,2021-06-06 20:27:10,Adware
52 a61299f9b501adcf680b9275d79d4ac6,2017-11-04 18:03:59,2020-04-21 17:08:24,Tofsee 7dcce5b76c8b17472d024758970a406b,2017-11-22 12:42:46,2021-03-16 12:53:35,Tofsee
53 b2b61db7b9490a60d270ccb20b462826,2017-11-14 20:12:03,2021-02-18 22:56:49,Adware 534ce2dbc413c68e908363b5df0ae5e0,2017-12-22 09:36:21,2019-07-27 15:22:33,TrickBot
54 7dcce5b76c8b17472d024758970a406b,2017-11-22 12:42:46,2020-12-22 15:15:49,Tofsee fb00055a1196aeea8d1bc609885ba953,2018-01-01 22:49:25,2019-04-09 06:58:58,TrickBot
55 534ce2dbc413c68e908363b5df0ae5e0,2017-12-22 09:36:21,2019-07-27 15:22:33,TrickBot a50a861119aceb0ccc74902e8fddb618,2018-01-02 08:16:23,2018-07-05 02:33:08,Tofsee
56 fb00055a1196aeea8d1bc609885ba953,2018-01-01 22:49:25,2019-04-09 06:58:58,TrickBot e7643725fcff971e3051fe0e47fc2c71,2018-01-31 08:06:13,2020-03-25 16:19:48,Tofsee
57 a50a861119aceb0ccc74902e8fddb618,2018-01-02 08:16:23,2018-07-05 02:33:08,Tofsee 7c410ce832e848a3321432c9a82e972b,2018-01-31 20:04:25,2021-08-01 06:13:14,Tofsee
58 e7643725fcff971e3051fe0e47fc2c71,2018-01-31 08:06:13,2020-03-25 16:19:48,Tofsee da949afd9bd6df820730f8f171584a71,2018-02-03 05:19:37,2021-03-08 22:10:10,Tofsee
59 7c410ce832e848a3321432c9a82e972b,2018-01-31 20:04:25,2021-02-21 22:51:31,Tofsee 906004246f3ba5e755b043c057254a29,2018-03-11 08:25:38,2018-04-14 00:59:16,Tofsee
60 da949afd9bd6df820730f8f171584a71,2018-02-03 05:19:37,2021-02-21 05:20:43,Tofsee fd80fa9c6120cdeea8520510f3c644ac,2018-03-11 09:34:30,2021-08-11 12:34:00,Tofsee
61 906004246f3ba5e755b043c057254a29,2018-03-11 08:25:38,2018-04-14 00:59:16,Tofsee b90bdbe961a648f0427db21aaa6ccb59,2018-03-11 10:37:43,2020-05-29 23:39:01,Tofsee
62 fd80fa9c6120cdeea8520510f3c644ac,2018-03-11 09:34:30,2021-02-21 22:16:39,Tofsee 1fe4c7a3544eb27afec2adfb3a3dbf60,2018-03-11 19:23:08,2021-08-09 11:42:58,Tofsee
63 b90bdbe961a648f0427db21aaa6ccb59,2018-03-11 10:37:43,2020-05-29 23:39:01,Tofsee c201b92f8b483fa388be174d6689f534,2018-03-12 13:43:52,2021-01-28 06:17:06,Gozi
64 1fe4c7a3544eb27afec2adfb3a3dbf60,2018-03-11 19:23:08,2021-02-20 11:19:52,Tofsee 9f62c4f26b90d3d757bea609e82f2eaf,2018-03-13 06:23:41,2021-05-20 23:01:06,Tofsee
65 c201b92f8b483fa388be174d6689f534,2018-03-12 13:43:52,2021-01-28 06:17:06,Gozi 1be3ecebe5aa9d3654e6e703d81f6928,2018-03-13 11:50:02,2021-08-11 13:02:35,Ransomware.Troldesh
66 9f62c4f26b90d3d757bea609e82f2eaf,2018-03-13 06:23:41,2020-12-18 16:07:31,Tofsee e3b2ab1f9a56f2fb4c9248f2f41631fa,2018-03-15 01:06:34,2021-07-02 21:51:49,Tofsee
67 1be3ecebe5aa9d3654e6e703d81f6928,2018-03-13 11:50:02,2021-02-14 23:03:13,Ransomware.Troldesh dff8a0aa1c904aaea76c5bf624e88333,2018-03-18 09:41:15,2020-10-27 09:50:24,Tofsee
68 e3b2ab1f9a56f2fb4c9248f2f41631fa,2018-03-15 01:06:34,2021-02-22 10:36:48,Tofsee 17fd49722f8d11f3d76dce84f8e099a7,2018-03-19 23:02:27,2021-08-01 00:40:52,Tofsee
69 dff8a0aa1c904aaea76c5bf624e88333,2018-03-18 09:41:15,2020-10-27 09:50:24,Tofsee 911479ac8a0813ed1241b3686ccdade9,2018-03-19 23:24:59,2020-03-30 04:09:18,Tofsee
70 17fd49722f8d11f3d76dce84f8e099a7,2018-03-19 23:02:27,2020-12-09 11:32:21,Tofsee c5deb9465d47232dd48772f9c4d14679,2018-03-22 15:42:48,2021-03-23 00:34:25,Tofsee
71 911479ac8a0813ed1241b3686ccdade9,2018-03-19 23:24:59,2020-03-30 04:09:18,Tofsee f22bdd57e3a52de86cda40da2d84e83b,2018-03-27 13:40:19,2019-01-20 14:31:39,Tofsee
72 c5deb9465d47232dd48772f9c4d14679,2018-03-22 15:42:48,2021-02-17 07:39:32,Tofsee d18a4da84af59e1108862a39bae7c9d4,2018-04-03 00:40:51,2021-02-06 01:53:12,Tofsee
73 f22bdd57e3a52de86cda40da2d84e83b,2018-03-27 13:40:19,2019-01-20 14:31:39,Tofsee 2d8794cb7b52b777bee2695e79c15760,2018-04-04 06:56:37,2021-07-26 08:07:00,Ransomware
74 d18a4da84af59e1108862a39bae7c9d4,2018-04-03 00:40:51,2021-02-06 01:53:12,Tofsee 40adfd923eb82b89d8836ba37a19bca1,2018-04-15 15:49:08,2021-04-11 04:42:47,CoinMiner
75 2d8794cb7b52b777bee2695e79c15760,2018-04-04 06:56:37,2021-02-10 08:22:31,Ransomware 1aee0238942d453d679fc1e37a303387,2018-05-13 01:59:49,2021-07-30 12:27:07,Tofsee
76 40adfd923eb82b89d8836ba37a19bca1,2018-04-15 15:49:08,2021-02-21 22:22:56,CoinMiner 2092e1fffb45d7e4a19a57f9bc5e203a,2018-05-16 21:59:36,2018-09-05 01:58:33,Adware
77 1aee0238942d453d679fc1e37a303387,2018-05-13 01:59:49,2021-02-10 13:19:24,Tofsee bffa4501966196d3d6e90cee1f88fc89,2018-06-07 15:08:04,2020-03-16 00:03:44,Tofsee
78 2092e1fffb45d7e4a19a57f9bc5e203a,2018-05-16 21:59:36,2018-09-05 01:58:33,Adware 807fca46d9d0cf63adf4e5e80e414bbe,2018-06-07 16:51:03,2021-08-07 03:15:42,Tofsee
79 bffa4501966196d3d6e90cee1f88fc89,2018-06-07 15:08:04,2020-03-16 00:03:44,Tofsee fb58831f892190644fe44e25bc830b45,2018-06-08 12:07:59,2021-07-20 01:39:05,Adware
80 807fca46d9d0cf63adf4e5e80e414bbe,2018-06-07 16:51:03,2021-02-20 03:15:38,Tofsee 0cc1e84568e471aa1d62ad4158ade6b5,2018-06-24 10:50:47,2021-06-21 02:35:57,Tofsee
81 fb58831f892190644fe44e25bc830b45,2018-06-08 12:07:59,2019-05-31 21:02:37,Adware d2935c58fe676744fecc8614ee5356c7,2018-08-14 21:48:41,2021-08-11 11:54:42,Adwind
82 0cc1e84568e471aa1d62ad4158ade6b5,2018-06-24 10:50:47,2021-02-10 21:48:57,Tofsee 8916410db85077a5460817142dcbc8de,2018-08-21 12:32:28,2021-08-11 15:00:50,TrickBot
83 d2935c58fe676744fecc8614ee5356c7,2018-08-14 21:48:41,2021-02-21 21:44:55,Adwind c5235d3a8b9934b7fbbd204d50bc058d,2018-08-23 17:36:08,2019-10-13 05:11:09,Gootkit
84 8916410db85077a5460817142dcbc8de,2018-08-21 12:32:28,2021-02-21 20:02:19,TrickBot 57f3642b4e37e28f5cbe3020c9331b4c,2018-08-28 15:54:53,2021-08-11 13:05:18,Gozi
85 c5235d3a8b9934b7fbbd204d50bc058d,2018-08-23 17:36:08,2019-10-13 05:11:09,Gootkit e62a5f4d538cbf169c2af71bec2399b4,2018-08-30 15:45:40,2021-08-11 09:48:52,TrickBot
86 57f3642b4e37e28f5cbe3020c9331b4c,2018-08-28 15:54:53,2021-02-22 11:09:43,Gozi 51c64c77e60f3980eea90869b68c58a8,2018-08-30 21:04:57,2021-08-11 08:13:08,Dridex
87 e62a5f4d538cbf169c2af71bec2399b4,2018-08-30 15:45:40,2021-02-21 20:36:55,TrickBot 7691297bcb20a41233fd0a0baa0a3628,2018-09-17 02:50:05,2021-08-11 12:20:33,Adware
88 51c64c77e60f3980eea90869b68c58a8,2018-08-30 21:04:57,2021-02-21 22:09:16,Dridex 7dd50e112cd23734a310b90f6f44a7cd,2018-09-17 17:54:58,2021-08-01 11:28:46,Quakbot
89 7691297bcb20a41233fd0a0baa0a3628,2018-09-17 02:50:05,2021-02-21 17:05:57,Adware 52c7396a501e4fecbdfa99c5408334ac,2018-09-18 00:29:04,2019-12-03 17:24:02,Tofsee
90 7dd50e112cd23734a310b90f6f44a7cd,2018-09-17 17:54:58,2021-02-22 11:14:04,Quakbot fc54e0d16d9764783542f0146a98b300,2018-09-24 12:33:44,2021-08-11 12:51:10,AsyncRAT
91 52c7396a501e4fecbdfa99c5408334ac,2018-09-18 00:29:04,2019-12-03 17:24:02,Tofsee f735bbc6b69723b9df7b0e7ef27872af,2018-10-02 18:04:16,2021-08-11 07:25:14,TrickBot
92 f735bbc6b69723b9df7b0e7ef27872af,2018-10-02 18:04:16,2021-02-19 06:55:39,TrickBot 49ed2ef3f1321e5f044f1e71b0e6fdd5,2018-10-02 18:04:17,2021-08-08 22:08:01,TrickBot
93 49ed2ef3f1321e5f044f1e71b0e6fdd5,2018-10-02 18:04:17,2021-02-19 06:55:39,TrickBot d76ee64fb7273733cbe455ac81c292e6,2018-11-16 13:26:39,2018-11-18 19:19:36,Tofsee
94 d76ee64fb7273733cbe455ac81c292e6,2018-11-16 13:26:39,2018-11-18 19:19:36,Tofsee 8f6c918dcb585ebbea05e2cc94530e3d,2018-11-16 13:26:41,2020-05-06 15:45:21,Tofsee
95 8f6c918dcb585ebbea05e2cc94530e3d,2018-11-16 13:26:41,2020-05-06 15:45:21,Tofsee 34f14a69ad7009ca5863379218af17f3,2018-11-17 05:17:22,2021-01-28 08:19:18,Tofsee
96 34f14a69ad7009ca5863379218af17f3,2018-11-17 05:17:22,2021-01-28 08:19:18,Tofsee c2b4710c6888a5d47befe865c8e6fb19,2018-11-29 20:46:04,2021-08-03 23:37:22,Tofsee
97 c2b4710c6888a5d47befe865c8e6fb19,2018-11-29 20:46:04,2020-12-16 15:08:09,Tofsee decfb48a53789ebe081b88aabb58ee34,2018-12-21 09:06:16,2021-06-14 05:27:16,Adwind
98 decfb48a53789ebe081b88aabb58ee34,2018-12-21 09:06:16,2021-02-16 12:28:53,Adwind 08a8a4e85b25ac42e1490bc85cfdb5ce,2019-01-30 02:48:34,2020-10-27 09:50:19,Tofsee
99 08a8a4e85b25ac42e1490bc85cfdb5ce,2019-01-30 02:48:34,2020-10-27 09:50:19,Tofsee c0220cd64849a629397a9cb68f78a0ea,2019-03-24 00:12:32,2021-07-31 00:26:06,Tofsee
100 c0220cd64849a629397a9cb68f78a0ea,2019-03-24 00:12:32,2021-02-13 05:13:33,Tofsee 7a29c223fb122ec64d10f0a159e07996,2019-06-09 22:55:29,2020-10-27 09:50:26,None
101 7a29c223fb122ec64d10f0a159e07996,2019-06-09 22:55:29,2020-10-27 09:50:26,Tofsee 70a04365be5bbd4653698bebeb43ce68,2019-07-02 06:26:56,2020-05-30 04:19:00,Tofsee
102 44dab16d680ef93487bc16ad23b3ffb1,2019-06-09 22:55:29,2020-10-27 09:50:25,Tofsee d81d654effb94714a4086734fa0adad9,2019-07-16 23:29:02,2020-10-27 09:50:21,Tofsee
103 70a04365be5bbd4653698bebeb43ce68,2019-07-02 06:26:56,2020-05-30 04:19:00,Tofsee 25d74b7b4b779eb1efd4b31d26d651c6,2019-08-03 20:15:33,2020-07-14 21:43:25,Tofsee
104 d81d654effb94714a4086734fa0adad9,2019-07-16 23:29:02,2020-10-27 09:50:21,Tofsee fc2299d5b2964cd242c5a2c8c531a5f0,2019-08-09 23:56:32,2021-08-11 02:20:22,Tofsee
105 25d74b7b4b779eb1efd4b31d26d651c6,2019-08-03 20:15:33,2020-07-14 21:43:25,Tofsee 32926ca3e59f0413d0b98725454594f5,2019-09-12 06:56:10,2021-04-08 19:50:43,Tofsee
106 fc2299d5b2964cd242c5a2c8c531a5f0,2019-08-09 23:56:32,2021-02-22 08:49:18,Tofsee ffefafdb86336d057eda5fdf02b3d5ce,2019-10-26 07:31:49,2020-07-25 00:14:09,Tofsee
107 32926ca3e59f0413d0b98725454594f5,2019-09-12 06:56:10,2020-10-27 21:49:31,Tofsee 8515076cbbca9dce33151b798f782456,2020-12-27 16:53:04,2021-08-11 15:06:36,BitRAT
108 ffefafdb86336d057eda5fdf02b3d5ce,2019-10-26 07:31:49,2020-07-25 00:14:09,Tofsee # END (97) entries
# END (98) entries
+1711 -1038
View File
File diff suppressed because it is too large Load Diff
+196 -159
View File
@@ -1,6 +1,27 @@
/*
*
* Copyright (C) 2011-22 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#ifndef WIN32
#include <arpa/inet.h>
#include <errno.h>
#include <netinet/in.h>
#endif
#include <errno.h>
#include <ndpi_api.h>
#include <ndpi_main.h>
#include <ndpi_typedefs.h>
@@ -12,6 +33,11 @@
#include <string.h>
#include <unistd.h>
#ifdef WIN32
#include <windows.h>
#endif
//#define VERBOSE 1
#define MAX_FLOW_ROOTS_PER_THREAD 2048
#define MAX_IDLE_FLOWS_PER_THREAD 64
#define TICK_RESOLUTION 1000
@@ -47,12 +73,19 @@ struct nDPI_flow_info {
union {
struct {
uint32_t src;
uint32_t pad_00[3];
uint32_t dst;
uint32_t pad_01[3];
} v4;
struct {
uint64_t src[2];
uint64_t dst[2];
} v6;
struct {
uint32_t src[4];
uint32_t dst[4];
} u32;
} ip_tuple;
unsigned long long int total_l4_data_len;
@@ -73,16 +106,12 @@ struct nDPI_flow_info {
struct ndpi_proto guessed_protocol;
struct ndpi_flow_struct * ndpi_flow;
struct ndpi_id_struct * ndpi_src;
struct ndpi_id_struct * ndpi_dst;
};
struct nDPI_workflow {
pcap_t * pcap_handle;
uint8_t error_or_eof:1;
uint8_t reserved_00:7;
uint8_t reserved_01[3];
volatile long int error_or_eof;
unsigned long long int packets_captured;
unsigned long long int packets_processed;
@@ -113,8 +142,8 @@ struct nDPI_reader_thread {
static struct nDPI_reader_thread reader_threads[MAX_READER_THREADS] = {};
static int reader_thread_count = MAX_READER_THREADS;
static int main_thread_shutdown = 0;
static uint32_t flow_id = 0;
static volatile long int main_thread_shutdown = 0;
static volatile long int flow_id = 0;
static void free_workflow(struct nDPI_workflow ** const workflow);
@@ -122,7 +151,10 @@ static struct nDPI_workflow * init_workflow(char const * const file_or_device)
{
char pcap_error_buffer[PCAP_ERRBUF_SIZE];
struct nDPI_workflow * workflow = (struct nDPI_workflow *)ndpi_calloc(1, sizeof(*workflow));
const char *bpfFilter = "ip or ip6";
static struct bpf_program bpf_code;
static struct bpf_program *bpf_cfilter = NULL;
if (workflow == NULL) {
return NULL;
}
@@ -130,17 +162,32 @@ static struct nDPI_workflow * init_workflow(char const * const file_or_device)
if (access(file_or_device, R_OK) != 0 && errno == ENOENT) {
workflow->pcap_handle = pcap_open_live(file_or_device, /* 1536 */ 65535, 1, 250, pcap_error_buffer);
} else {
#ifdef WIN32
workflow->pcap_handle = pcap_open_offline(file_or_device, pcap_error_buffer);
#else
workflow->pcap_handle = pcap_open_offline_with_tstamp_precision(file_or_device, PCAP_TSTAMP_PRECISION_MICRO,
pcap_error_buffer);
#endif
}
if (workflow->pcap_handle == NULL) {
fprintf(stderr, "pcap_open_live / pcap_open_offline_with_tstamp_precision: %.*s\n",
fprintf(stderr, "pcap_open_live / pcap_open_offline: %.*s\n",
(int) PCAP_ERRBUF_SIZE, pcap_error_buffer);
free_workflow(&workflow);
return NULL;
}
if(pcap_compile(workflow->pcap_handle, &bpf_code, bpfFilter, 1, 0xFFFFFF00) < 0) {
printf("pcap_compile error: '%s'\n", pcap_geterr(workflow->pcap_handle));
exit(-1);
}
bpf_cfilter = &bpf_code;
if(pcap_setfilter(workflow->pcap_handle, bpf_cfilter) < 0) {
printf("pcap_setfilter error: '%s'\n", pcap_geterr(workflow->pcap_handle));
}
ndpi_init_prefs init_prefs = ndpi_no_prefs;
workflow->ndpi_struct = ndpi_init_detection_module(init_prefs);
if (workflow->ndpi_struct == NULL) {
@@ -176,8 +223,6 @@ static void ndpi_flow_info_freer(void * const node)
{
struct nDPI_flow_info * const flow = (struct nDPI_flow_info *)node;
ndpi_free(flow->ndpi_dst);
ndpi_free(flow->ndpi_src);
ndpi_flow_free(flow->ndpi_flow);
ndpi_free(flow);
}
@@ -290,7 +335,7 @@ static void print_packet_info(struct nDPI_reader_thread const * const reader_thr
char buf[256];
int used = 0, ret;
ret = snprintf(buf, sizeof(buf), "[%8llu, %d, %4u] %4u bytes: ",
ret = ndpi_snprintf(buf, sizeof(buf), "[%8llu, %d, %4u] %4u bytes: ",
workflow->packets_captured, reader_thread->array_index,
flow->flow_id, header->caplen);
if (ret > 0) {
@@ -298,9 +343,9 @@ static void print_packet_info(struct nDPI_reader_thread const * const reader_thr
}
if (ip_tuple_to_string(flow, src_addr_str, sizeof(src_addr_str), dst_addr_str, sizeof(dst_addr_str)) != 0) {
ret = snprintf(buf + used, sizeof(buf) - used, "IP[%s -> %s]", src_addr_str, dst_addr_str);
ret = ndpi_snprintf(buf + used, sizeof(buf) - used, "IP[%s -> %s]", src_addr_str, dst_addr_str);
} else {
ret = snprintf(buf + used, sizeof(buf) - used, "IP[ERROR]");
ret = ndpi_snprintf(buf + used, sizeof(buf) - used, "IP[ERROR]");
}
if (ret > 0) {
used += ret;
@@ -308,24 +353,24 @@ static void print_packet_info(struct nDPI_reader_thread const * const reader_thr
switch (flow->l4_protocol) {
case IPPROTO_UDP:
ret = snprintf(buf + used, sizeof(buf) - used, " -> UDP[%u -> %u, %u bytes]",
ret = ndpi_snprintf(buf + used, sizeof(buf) - used, " -> UDP[%u -> %u, %u bytes]",
flow->src_port, flow->dst_port, l4_data_len);
break;
case IPPROTO_TCP:
ret = snprintf(buf + used, sizeof(buf) - used, " -> TCP[%u -> %u, %u bytes]",
ret = ndpi_snprintf(buf + used, sizeof(buf) - used, " -> TCP[%u -> %u, %u bytes]",
flow->src_port, flow->dst_port, l4_data_len);
break;
case IPPROTO_ICMP:
ret = snprintf(buf + used, sizeof(buf) - used, " -> ICMP");
ret = ndpi_snprintf(buf + used, sizeof(buf) - used, " -> ICMP");
break;
case IPPROTO_ICMPV6:
ret = snprintf(buf + used, sizeof(buf) - used, " -> ICMP6");
ret = ndpi_snprintf(buf + used, sizeof(buf) - used, " -> ICMP6");
break;
case IPPROTO_HOPOPTS:
ret = snprintf(buf + used, sizeof(buf) - used, " -> ICMP6 Hop-By-Hop");
ret = ndpi_snprintf(buf + used, sizeof(buf) - used, " -> ICMP6 Hop-By-Hop");
break;
default:
ret = snprintf(buf + used, sizeof(buf) - used, " -> Unknown[0x%X]", flow->l4_protocol);
ret = ndpi_snprintf(buf + used, sizeof(buf) - used, " -> Unknown[0x%X]", flow->l4_protocol);
break;
}
if (ret > 0) {
@@ -336,60 +381,72 @@ static void print_packet_info(struct nDPI_reader_thread const * const reader_thr
}
#endif
static int ip_tuples_equal(struct nDPI_flow_info const * const A,
struct nDPI_flow_info const * const B)
static int ip_tuples_compare(struct nDPI_flow_info const * const A, struct nDPI_flow_info const * const B)
{
if (A->l3_type == L3_IP && B->l3_type == L3_IP6) {
return A->ip_tuple.v4.src == B->ip_tuple.v4.src &&
A->ip_tuple.v4.dst == B->ip_tuple.v4.dst;
} else if (A->l3_type == L3_IP6 && B->l3_type == L3_IP6) {
return A->ip_tuple.v6.src[0] == B->ip_tuple.v6.src[0] &&
A->ip_tuple.v6.src[1] == B->ip_tuple.v6.src[1] &&
A->ip_tuple.v6.dst[0] == B->ip_tuple.v6.dst[0] &&
A->ip_tuple.v6.dst[1] == B->ip_tuple.v6.dst[1];
// generate a warning if the enum changes
switch (A->l3_type)
{
case L3_IP:
case L3_IP6:
break;
}
return 0;
}
static int ip_tuples_compare(struct nDPI_flow_info const * const A,
struct nDPI_flow_info const * const B)
{
if (A->l3_type == L3_IP && B->l3_type == L3_IP6) {
if (A->ip_tuple.v4.src < B->ip_tuple.v4.src ||
A->ip_tuple.v4.dst < B->ip_tuple.v4.dst)
{
return -1;
}
if (A->ip_tuple.v4.src > B->ip_tuple.v4.src ||
A->ip_tuple.v4.dst > B->ip_tuple.v4.dst)
{
return 1;
}
} else if (A->l3_type == L3_IP6 && B->l3_type == L3_IP6) {
if ((A->ip_tuple.v6.src[0] < B->ip_tuple.v6.src[0] &&
A->ip_tuple.v6.src[1] < B->ip_tuple.v6.src[1]) ||
(A->ip_tuple.v6.dst[0] < B->ip_tuple.v6.dst[0] &&
A->ip_tuple.v6.dst[1] < B->ip_tuple.v6.dst[1]))
{
return -1;
}
if ((A->ip_tuple.v6.src[0] > B->ip_tuple.v6.src[0] &&
A->ip_tuple.v6.src[1] > B->ip_tuple.v6.src[1]) ||
(A->ip_tuple.v6.dst[0] > B->ip_tuple.v6.dst[0] &&
A->ip_tuple.v6.dst[1] > B->ip_tuple.v6.dst[1]))
{
return 1;
}
}
if (A->src_port < B->src_port ||
A->dst_port < B->dst_port)
if (A->l3_type == L3_IP && B->l3_type == L3_IP)
{
if (A->ip_tuple.v4.src < B->ip_tuple.v4.src)
{
return -1;
} else if (A->src_port > B->src_port ||
A->dst_port > B->dst_port)
}
if (A->ip_tuple.v4.src > B->ip_tuple.v4.src)
{
return 1;
}
if (A->ip_tuple.v4.dst < B->ip_tuple.v4.dst)
{
return -1;
}
if (A->ip_tuple.v4.dst > B->ip_tuple.v4.dst)
{
return 1;
}
}
else if (A->l3_type == L3_IP6 && B->l3_type == L3_IP6)
{
if (A->ip_tuple.v6.src[0] < B->ip_tuple.v6.src[0] && A->ip_tuple.v6.src[1] < B->ip_tuple.v6.src[1])
{
return -1;
}
if (A->ip_tuple.v6.src[0] > B->ip_tuple.v6.src[0] && A->ip_tuple.v6.src[1] > B->ip_tuple.v6.src[1])
{
return 1;
}
if (A->ip_tuple.v6.dst[0] < B->ip_tuple.v6.dst[0] && A->ip_tuple.v6.dst[1] < B->ip_tuple.v6.dst[1])
{
return -1;
}
if (A->ip_tuple.v6.dst[0] > B->ip_tuple.v6.dst[0] && A->ip_tuple.v6.dst[1] > B->ip_tuple.v6.dst[1])
{
return 1;
}
}
if (A->src_port < B->src_port)
{
return -1;
}
if (A->src_port > B->src_port)
{
return 1;
}
if (A->dst_port < B->dst_port)
{
return -1;
}
if (A->dst_port > B->dst_port)
{
return 1;
}
return 0;
}
@@ -438,13 +495,6 @@ static int ndpi_workflow_node_cmp(void const * const A, void const * const B) {
return(1);
}
if (ip_tuples_equal(flow_info_a, flow_info_b) != 0 &&
flow_info_a->src_port == flow_info_b->src_port &&
flow_info_a->dst_port == flow_info_b->dst_port)
{
return(0);
}
return ip_tuples_compare(flow_info_a, flow_info_b);
}
@@ -486,10 +536,6 @@ static void ndpi_process_packet(uint8_t * const args,
void * tree_result;
struct nDPI_flow_info * flow_to_process;
int direction_changed = 0;
struct ndpi_id_struct * ndpi_src;
struct ndpi_id_struct * ndpi_dst;
const struct ndpi_ethhdr * ethernet;
const struct ndpi_iphdr * ip;
struct ndpi_ipv6hdr * ip6;
@@ -687,7 +733,7 @@ static void ndpi_process_packet(uint8_t * const args,
workflow->total_l4_data_len += l4_len;
#ifdef VERBOSE
print_packet_info(reader_thread, header, l4_data_len, &flow);
print_packet_info(reader_thread, header, l4_len, &flow);
#endif
/* calculate flow hash for btree find, search(insert) */
@@ -713,27 +759,38 @@ static void ndpi_process_packet(uint8_t * const args,
tree_result = ndpi_tfind(&flow, &workflow->ndpi_flows_active[hashed_index], ndpi_workflow_node_cmp);
if (tree_result == NULL) {
/* flow not found in btree: switch src <-> dst and try to find it again */
uint64_t orig_src_ip[2] = { flow.ip_tuple.v6.src[0], flow.ip_tuple.v6.src[1] };
uint64_t orig_dst_ip[2] = { flow.ip_tuple.v6.dst[0], flow.ip_tuple.v6.dst[1] };
uint32_t orig_src_ip[4] = { flow.ip_tuple.u32.src[0], flow.ip_tuple.u32.src[1],
flow.ip_tuple.u32.src[2], flow.ip_tuple.u32.src[3] };
uint32_t orig_dst_ip[4] = { flow.ip_tuple.u32.dst[0], flow.ip_tuple.u32.dst[1],
flow.ip_tuple.u32.dst[2], flow.ip_tuple.u32.dst[3] };
uint16_t orig_src_port = flow.src_port;
uint16_t orig_dst_port = flow.dst_port;
flow.ip_tuple.v6.src[0] = orig_dst_ip[0];
flow.ip_tuple.v6.src[1] = orig_dst_ip[1];
flow.ip_tuple.v6.dst[0] = orig_src_ip[0];
flow.ip_tuple.v6.dst[1] = orig_src_ip[1];
flow.ip_tuple.u32.src[0] = orig_dst_ip[0];
flow.ip_tuple.u32.src[1] = orig_dst_ip[1];
flow.ip_tuple.u32.src[2] = orig_dst_ip[2];
flow.ip_tuple.u32.src[3] = orig_dst_ip[3];
flow.ip_tuple.u32.dst[0] = orig_src_ip[0];
flow.ip_tuple.u32.dst[1] = orig_src_ip[1];
flow.ip_tuple.u32.dst[2] = orig_src_ip[2];
flow.ip_tuple.u32.dst[3] = orig_src_ip[3];
flow.src_port = orig_dst_port;
flow.dst_port = orig_src_port;
tree_result = ndpi_tfind(&flow, &workflow->ndpi_flows_active[hashed_index], ndpi_workflow_node_cmp);
if (tree_result != NULL) {
direction_changed = 1;
}
flow.ip_tuple.v6.src[0] = orig_src_ip[0];
flow.ip_tuple.v6.src[1] = orig_src_ip[1];
flow.ip_tuple.v6.dst[0] = orig_dst_ip[0];
flow.ip_tuple.v6.dst[1] = orig_dst_ip[1];
flow.ip_tuple.u32.src[0] = orig_src_ip[0];
flow.ip_tuple.u32.src[1] = orig_src_ip[1];
flow.ip_tuple.u32.src[2] = orig_src_ip[2];
flow.ip_tuple.u32.src[3] = orig_src_ip[3];
flow.ip_tuple.u32.dst[0] = orig_dst_ip[0];
flow.ip_tuple.u32.dst[1] = orig_dst_ip[1];
flow.ip_tuple.u32.dst[2] = orig_dst_ip[2];
flow.ip_tuple.u32.dst[3] = orig_dst_ip[3];
flow.src_port = orig_src_port;
flow.dst_port = orig_dst_port;
}
@@ -754,10 +811,8 @@ static void ndpi_process_packet(uint8_t * const args,
return;
}
workflow->cur_active_flows++;
workflow->total_active_flows++;
memcpy(flow_to_process, &flow, sizeof(*flow_to_process));
flow_to_process->flow_id = flow_id++;
flow_to_process->flow_id = __sync_fetch_and_add(&flow_id, 1);
flow_to_process->ndpi_flow = (struct ndpi_flow_struct *)ndpi_flow_malloc(SIZEOF_FLOW_STRUCT);
if (flow_to_process->ndpi_flow == NULL) {
@@ -767,20 +822,6 @@ static void ndpi_process_packet(uint8_t * const args,
}
memset(flow_to_process->ndpi_flow, 0, SIZEOF_FLOW_STRUCT);
flow_to_process->ndpi_src = (struct ndpi_id_struct *)ndpi_calloc(1, SIZEOF_ID_STRUCT);
if (flow_to_process->ndpi_src == NULL) {
fprintf(stderr, "[%8llu, %d, %4u] Not enough memory for src id struct\n",
workflow->packets_captured, reader_thread->array_index, flow_to_process->flow_id);
return;
}
flow_to_process->ndpi_dst = (struct ndpi_id_struct *)ndpi_calloc(1, SIZEOF_ID_STRUCT);
if (flow_to_process->ndpi_dst == NULL) {
fprintf(stderr, "[%8llu, %d, %4u] Not enough memory for dst id struct\n",
workflow->packets_captured, reader_thread->array_index, flow_to_process->flow_id);
return;
}
printf("[%8llu, %d, %4u] new %sflow\n", workflow->packets_captured, thread_index,
flow_to_process->flow_id,
(flow_to_process->is_midstream_flow != 0 ? "midstream-" : ""));
@@ -789,18 +830,10 @@ static void ndpi_process_packet(uint8_t * const args,
return;
}
ndpi_src = flow_to_process->ndpi_src;
ndpi_dst = flow_to_process->ndpi_dst;
workflow->cur_active_flows++;
workflow->total_active_flows++;
} else {
flow_to_process = *(struct nDPI_flow_info **)tree_result;
if (direction_changed != 0) {
ndpi_src = flow_to_process->ndpi_dst;
ndpi_dst = flow_to_process->ndpi_src;
} else {
ndpi_src = flow_to_process->ndpi_src;
ndpi_dst = flow_to_process->ndpi_dst;
}
}
flow_to_process->packets_processed++;
@@ -851,7 +884,7 @@ static void ndpi_process_packet(uint8_t * const args,
flow_to_process->detected_l7_protocol =
ndpi_detection_process_packet(workflow->ndpi_struct, flow_to_process->ndpi_flow,
ip != NULL ? (uint8_t *)ip : (uint8_t *)ip6,
ip_size, time_ms, ndpi_src, ndpi_dst);
ip_size, time_ms);
if (ndpi_is_protocol_detected(workflow->ndpi_struct,
flow_to_process->detected_l7_protocol) != 0 &&
@@ -906,41 +939,43 @@ static void ndpi_process_packet(uint8_t * const args,
flow_to_process->detected_l7_protocol.app_protocol == NDPI_PROTOCOL_TLS)
{
if (flow_to_process->tls_client_hello_seen == 0 &&
flow_to_process->ndpi_flow->l4.tcp.tls.hello_processed != 0)
flow_to_process->ndpi_flow->protos.tls_quic.hello_processed != 0)
{
uint8_t unknown_tls_version = 0;
char buf_ver[16];
printf("[%8llu, %d, %4d][TLS-CLIENT-HELLO] version: %s | sni: %s | alpn: %s\n",
workflow->packets_captured,
reader_thread->array_index,
flow_to_process->flow_id,
ndpi_ssl_version2str(flow_to_process->ndpi_flow,
flow_to_process->ndpi_flow->protos.tls_quic_stun.tls_quic.ssl_version,
ndpi_ssl_version2str(buf_ver, sizeof(buf_ver),
flow_to_process->ndpi_flow->protos.tls_quic.ssl_version,
&unknown_tls_version),
flow_to_process->ndpi_flow->protos.tls_quic_stun.tls_quic.client_requested_server_name,
(flow_to_process->ndpi_flow->protos.tls_quic_stun.tls_quic.alpn != NULL ?
flow_to_process->ndpi_flow->protos.tls_quic_stun.tls_quic.alpn : "-"));
flow_to_process->ndpi_flow->host_server_name,
(flow_to_process->ndpi_flow->protos.tls_quic.alpn != NULL ?
flow_to_process->ndpi_flow->protos.tls_quic.alpn : "-"));
flow_to_process->tls_client_hello_seen = 1;
}
if (flow_to_process->tls_server_hello_seen == 0 &&
flow_to_process->ndpi_flow->l4.tcp.tls.certificate_processed != 0)
{
uint8_t unknown_tls_version = 0;
char buf_ver[16];
printf("[%8llu, %d, %4d][TLS-SERVER-HELLO] version: %s | common-name(s): %.*s | "
"issuer: %s | subject: %s\n",
workflow->packets_captured,
reader_thread->array_index,
flow_to_process->flow_id,
ndpi_ssl_version2str(flow_to_process->ndpi_flow,
flow_to_process->ndpi_flow->protos.tls_quic_stun.tls_quic.ssl_version,
ndpi_ssl_version2str(buf_ver, sizeof(buf_ver),
flow_to_process->ndpi_flow->protos.tls_quic.ssl_version,
&unknown_tls_version),
(flow_to_process->ndpi_flow->protos.tls_quic_stun.tls_quic.server_names_len == 0 ?
1 : flow_to_process->ndpi_flow->protos.tls_quic_stun.tls_quic.server_names_len),
(flow_to_process->ndpi_flow->protos.tls_quic_stun.tls_quic.server_names == NULL ?
"-" : flow_to_process->ndpi_flow->protos.tls_quic_stun.tls_quic.server_names),
(flow_to_process->ndpi_flow->protos.tls_quic_stun.tls_quic.issuerDN != NULL ?
flow_to_process->ndpi_flow->protos.tls_quic_stun.tls_quic.issuerDN : "-"),
(flow_to_process->ndpi_flow->protos.tls_quic_stun.tls_quic.subjectDN != NULL ?
flow_to_process->ndpi_flow->protos.tls_quic_stun.tls_quic.subjectDN : "-"));
(flow_to_process->ndpi_flow->protos.tls_quic.server_names_len == 0 ?
1 : flow_to_process->ndpi_flow->protos.tls_quic.server_names_len),
(flow_to_process->ndpi_flow->protos.tls_quic.server_names == NULL ?
"-" : flow_to_process->ndpi_flow->protos.tls_quic.server_names),
(flow_to_process->ndpi_flow->protos.tls_quic.issuerDN != NULL ?
flow_to_process->ndpi_flow->protos.tls_quic.issuerDN : "-"),
(flow_to_process->ndpi_flow->protos.tls_quic.subjectDN != NULL ?
flow_to_process->ndpi_flow->protos.tls_quic.subjectDN : "-"));
flow_to_process->tls_server_hello_seen = 1;
}
}
@@ -957,7 +992,7 @@ static void run_pcap_loop(struct nDPI_reader_thread const * const reader_thread)
fprintf(stderr, "Error while reading pcap file: '%s'\n",
pcap_geterr(reader_thread->workflow->pcap_handle));
reader_thread->workflow->error_or_eof = 1;
__sync_fetch_and_add(&reader_thread->workflow->error_or_eof, 1);
}
}
}
@@ -978,14 +1013,14 @@ static void * processing_thread(void * const ndpi_thread_arg)
printf("Starting Thread %d\n", reader_thread->array_index);
run_pcap_loop(reader_thread);
reader_thread->workflow->error_or_eof = 1;
__sync_fetch_and_add(&reader_thread->workflow->error_or_eof, 1);
return NULL;
}
static int processing_threads_error_or_eof(void)
{
for (int i = 0; i < reader_thread_count; ++i) {
if (reader_threads[i].workflow->error_or_eof == 0) {
if (__sync_fetch_and_add(&reader_threads[i].workflow->error_or_eof, 0) == 0) {
return 0;
}
}
@@ -994,6 +1029,7 @@ static int processing_threads_error_or_eof(void)
static int start_reader_threads(void)
{
#ifndef WIN32
sigset_t thread_signal_set, old_signal_set;
sigfillset(&thread_signal_set);
@@ -1003,6 +1039,7 @@ static int start_reader_threads(void)
fprintf(stderr, "pthread_sigmask: %s\n", strerror(errno));
return 1;
}
#endif
for (int i = 0; i < reader_thread_count; ++i) {
reader_threads[i].array_index = i;
@@ -1030,6 +1067,7 @@ static int start_reader_threads(void)
static int stop_reader_threads(void)
{
unsigned long long int total_packets_captured = 0;
unsigned long long int total_packets_processed = 0;
unsigned long long int total_l4_data_len = 0;
unsigned long long int total_flows_captured = 0;
@@ -1047,6 +1085,10 @@ static int stop_reader_threads(void)
continue;
}
if (pthread_join(reader_threads[i].thread_id, NULL) != 0) {
fprintf(stderr, "pthread_join: %s\n", strerror(errno));
}
total_packets_processed += reader_threads[i].workflow->packets_processed;
total_l4_data_len += reader_threads[i].workflow->total_l4_data_len;
total_flows_captured += reader_threads[i].workflow->total_active_flows;
@@ -1059,27 +1101,25 @@ static int stop_reader_threads(void)
reader_threads[i].workflow->total_l4_data_len, reader_threads[i].workflow->total_active_flows,
reader_threads[i].workflow->total_idle_flows, reader_threads[i].workflow->detected_flow_protocols);
}
/* total packets captured: same value for all threads as packet2thread distribution happens later */
printf("Total packets captured.: %llu\n",
reader_threads[0].workflow->packets_captured);
printf("Total packets processed: %llu\n", total_packets_processed);
printf("Total layer4 data size.: %llu\n", total_l4_data_len);
printf("Total flows captured...: %llu\n", total_flows_captured);
printf("Total flows timed out..: %llu\n", total_flows_idle);
printf("Total flows detected...: %llu\n", total_flows_detected);
total_packets_captured = reader_threads[0].workflow->packets_captured;
for (int i = 0; i < reader_thread_count; ++i) {
if (reader_threads[i].workflow == NULL) {
continue;
}
if (pthread_join(reader_threads[i].thread_id, NULL) != 0) {
fprintf(stderr, "pthread_join: %s\n", strerror(errno));
}
free_workflow(&reader_threads[i].workflow);
}
printf("Total packets captured.: %llu\n", total_packets_captured);
printf("Total packets processed: %llu\n", total_packets_processed);
printf("Total layer4 data size.: %llu\n", total_l4_data_len);
printf("Total flows captured...: %llu\n", total_flows_captured);
printf("Total flows timed out..: %llu\n", total_flows_idle);
printf("Total flows detected...: %llu\n", total_flows_detected);
return 0;
}
@@ -1087,12 +1127,8 @@ static void sighandler(int signum)
{
fprintf(stderr, "Received SIGNAL %d\n", signum);
if (main_thread_shutdown == 0) {
main_thread_shutdown = 1;
if (stop_reader_threads() != 0) {
fprintf(stderr, "Failed to stop reader threads!\n");
exit(EXIT_FAILURE);
}
if (__sync_fetch_and_add(&main_thread_shutdown, 0) == 0) {
__sync_fetch_and_add(&main_thread_shutdown, 1);
} else {
fprintf(stderr, "Reader threads are already shutting down, please be patient.\n");
}
@@ -1101,9 +1137,10 @@ static void sighandler(int signum)
int main(int argc, char ** argv)
{
if (argc == 0) {
printf("usage: ndpiSimpleIntegration Mdevice name>\n");
return 1;
}
printf("usage: %s [PCAP-FILE-OR-INTERFACE]\n"
"----------------------------------\n"
"nDPI version: %s\n"
@@ -1126,11 +1163,11 @@ int main(int argc, char ** argv)
signal(SIGINT, sighandler);
signal(SIGTERM, sighandler);
while (main_thread_shutdown == 0 && processing_threads_error_or_eof() == 0) {
while (__sync_fetch_and_add(&main_thread_shutdown, 0) == 0 && processing_threads_error_or_eof() == 0) {
sleep(1);
}
if (main_thread_shutdown == 0 && stop_reader_threads() != 0) {
if (stop_reader_threads() != 0) {
fprintf(stderr, "%s: stop_reader_threads\n", argv[0]);
return 1;
}
+4 -2
View File
@@ -51,6 +51,8 @@ ip:54.80.47.130@AmazonPrime
# For Flows with a hostname (e.g. TLS) the risk is also put in AND with the host_risk_mask
#ip_risk_mask:192.168.1.0/24=0
#ip_risk_mask:10.196.157.228=0
host_risk_mask:".local"=0
host_risk_mask:".msftconnecttest.com"=0
host_risk_mask:".home"=0
# Custom certification autorities we trust
trusted_issuer_dn:"CN=813845657003339838, O=Code42, OU=TEST, ST=MN, C=US"
+398 -332
View File
File diff suppressed because it is too large Load Diff
+55 -19
View File
@@ -1,7 +1,7 @@
/*
* ndpi_util.h
*
* Copyright (C) 2011-21 - ntop.org
* Copyright (C) 2011-22 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
@@ -90,6 +90,7 @@ extern int dpdk_port_deinit(int port);
#define MAX_TABLE_SIZE_1 4096
#define MAX_TABLE_SIZE_2 8192
#define INIT_VAL -1
#define SERIALIZATION_BUFSIZ (8192 * 2)
// inner hash table (ja3 -> security state)
@@ -134,7 +135,6 @@ struct flow_metrics {
struct ndpi_entropy {
// Entropy fields
pkt_timeval src2dst_last_pkt_time, dst2src_last_pkt_time, flow_last_pkt_time;
u_int16_t src2dst_pkt_len[MAX_NUM_PKTS]; /*!< array of packet appdata lengths */
pkt_timeval src2dst_pkt_time[MAX_NUM_PKTS]; /*!< array of arrival times */
u_int16_t dst2src_pkt_len[MAX_NUM_PKTS]; /*!< array of packet appdata lengths */
@@ -158,14 +158,25 @@ struct ndpi_entropy {
float score;
};
enum info_type {
INFO_INVALID = 0,
INFO_GENERIC,
INFO_KERBEROS,
INFO_FTP_IMAP_POP_SMTP,
INFO_TLS_QUIC_ALPN_VERSION,
INFO_TLS_QUIC_ALPN_ONLY,
};
// flow tracking
typedef struct ndpi_flow_info {
u_int32_t flow_id;
u_int32_t hashval;
u_int32_t src_ip;
u_int32_t dst_ip;
u_int16_t src_port;
u_int16_t dst_port;
u_int32_t src_ip; /* network order */
u_int32_t dst_ip; /* network order */
struct ndpi_in6_addr src_ip6; /* network order */
struct ndpi_in6_addr dst_ip6; /* network order */
u_int16_t src_port; /* network order */
u_int16_t dst_port; /* network order */
u_int8_t detection_completed, protocol, bidirectional, check_extra_packets;
u_int16_t vlan_id;
ndpi_packet_tunnel tunnel_type;
@@ -187,24 +198,46 @@ typedef struct ndpi_flow_info {
u_int32_t src2dst_packets, dst2src_packets;
u_int32_t has_human_readeable_strings;
char human_readeable_string_buffer[32];
char *risk_str;
// result only, not used for flow identification
ndpi_protocol detected_protocol;
ndpi_confidence_t confidence;
// Flow data analysis
pkt_timeval src2dst_last_pkt_time, dst2src_last_pkt_time, flow_last_pkt_time;
struct ndpi_analyze_struct *iat_c_to_s, *iat_s_to_c, *iat_flow,
*pktlen_c_to_s, *pktlen_s_to_c;
char info[255];
enum info_type info_type;
union {
char info[256];
struct {
char alpn[128];
char tls_supported_versions[128];
} tls_quic;
struct {
unsigned char auth_failed;
char username[127];
char password[128];
} ftp_imap_pop_smtp;
struct {
char domain[85];
char hostname[85];
char username[86];
} kerberos;
};
char flow_extra_info[16];
char host_server_name[240];
char bittorent_hash[41];
char dhcp_fingerprint[48];
char host_server_name[80]; /* Hostname/SNI */
char *bittorent_hash;
char *dhcp_fingerprint;
char *dhcp_class_ident;
ndpi_risk risk;
struct {
u_int16_t ssl_version;
char client_requested_server_name[256], server_info[64],
char server_info[64],
client_hassh[33], server_hassh[33], *server_names,
*tls_alpn, *tls_supported_versions,
*tls_issuerDN, *tls_subjectDN,
@@ -224,18 +257,18 @@ typedef struct ndpi_flow_info {
} ssh_tls;
struct {
char url[256], request_content_type[64], content_type[64], user_agent[128];
char url[256], request_content_type[64], content_type[64], user_agent[256];
u_int response_status_code;
} http;
struct {
char username[32], password[32];
char *username, *password;
} telnet;
void *src_id, *dst_id;
struct ndpi_entropy entropy;
struct ndpi_entropy last_entropy;
struct ndpi_entropy *entropy;
struct ndpi_entropy *last_entropy;
/* Payload lenght bins */
#ifdef DIRECTION_BINS
@@ -262,6 +295,7 @@ typedef struct ndpi_stats {
u_int64_t packet_len[6];
u_int16_t max_packet_len;
u_int64_t dpi_packet_count[3];
u_int64_t flow_confidence[NDPI_CONFIDENCE_MAX];
} ndpi_stats_t;
@@ -299,11 +333,14 @@ typedef struct ndpi_workflow {
void **ndpi_flows_root;
struct ndpi_detection_module_struct *ndpi_struct;
u_int32_t num_allocated_flows;
/* CSV,TLV,JSON serialization interface */
ndpi_serializer ndpi_serializer;
} ndpi_workflow_t;
/* TODO: remove wrappers parameters and use ndpi global, when their initialization will be fixed... */
struct ndpi_workflow * ndpi_workflow_init(const struct ndpi_workflow_prefs * prefs, pcap_t * pcap_handle);
struct ndpi_workflow * ndpi_workflow_init(const struct ndpi_workflow_prefs * prefs, pcap_t * pcap_handle, int do_init_flows_root, ndpi_serialization_format serialization_format);
/* workflow main free function */
@@ -321,8 +358,7 @@ void ndpi_free_flow_info_half(struct ndpi_flow_info *flow);
struct ndpi_proto ndpi_workflow_process_packet(struct ndpi_workflow * workflow,
const struct pcap_pkthdr *header,
const u_char *packet,
ndpi_risk *flow_risk,
FILE * csv_fp);
ndpi_risk *flow_risk);
int ndpi_is_datalink_supported(int datalink_type);
@@ -342,7 +378,7 @@ static inline void ndpi_workflow_set_flow_giveup_callback(struct ndpi_workflow *
/* compare two nodes in workflow */
int ndpi_workflow_node_cmp(const void *a, const void *b);
void process_ndpi_collected_info(struct ndpi_workflow * workflow, struct ndpi_flow_info *flow, FILE * csv_fp);
void process_ndpi_collected_info(struct ndpi_workflow * workflow, struct ndpi_flow_info *flow);
u_int32_t ethernet_crc32(const void* data, size_t n_bytes);
void ndpi_flow_info_free_data(struct ndpi_flow_info *flow);
void ndpi_flow_info_freer(void *node);
File diff suppressed because it is too large Load Diff
+11 -2
View File
@@ -1,4 +1,4 @@
bin_PROGRAMS = fuzz_process_packet fuzz_ndpi_reader fuzz_ndpi_reader_with_main
bin_PROGRAMS = fuzz_process_packet fuzz_process_packet_with_main fuzz_ndpi_reader fuzz_ndpi_reader_with_main
fuzz_process_packet_SOURCES = fuzz_process_packet.c
fuzz_process_packet_CFLAGS =
@@ -13,6 +13,15 @@ fuzz_process_packet_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_process_packet_LDFLAGS) $(LDFLAGS) -o $@
fuzz_process_packet_with_main_SOURCES = fuzz_process_packet.c
fuzz_process_packet_with_main_CFLAGS = -DBUILD_MAIN
fuzz_process_packet_with_main_LDADD = ../src/lib/libndpi.a
fuzz_process_packet_with_main_LDFLAGS = $(ADDITIONAL_LIBS) $(LIBS)
# force usage of CXX for linker
fuzz_process_packet_with_main_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
$(LIBTOOLFLAGS) --mode=link $(CXX) $(AM_CXXFLAGS) $(CXXFLAGS) \
$(fuzz_process_packet_with_main_LDFLAGS) $(LDFLAGS) -o $@
fuzz_ndpi_reader_SOURCES = fuzz_ndpi_reader.c
fuzz_ndpi_reader_CFLAGS = -I../example/
fuzz_ndpi_reader_LDADD = ../example/libndpiReader.a ../src/lib/libndpi.a
@@ -37,7 +46,7 @@ fuzz_ndpi_reader_with_main_LINK=$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS
# required for Google oss-fuzz
# see https://github.com/google/oss-fuzz/tree/master/projects/ndpi
testpcaps := $(wildcard ../tests/pcap/*.pcap)
testpcaps := $(wildcard ../tests/pcap/*.pcap*)
fuzz_ndpi_reader_seed_corpus.zip: $(testpcaps)
zip -r fuzz_ndpi_reader_seed_corpus.zip $(testpcaps)
+37 -25
View File
@@ -8,12 +8,13 @@
#include <stdio.h>
struct ndpi_workflow_prefs *prefs = NULL;
struct ndpi_workflow *workflow = NULL;
int nDPI_LogLevel = 0;
char *_debug_protocols = NULL;
u_int32_t current_ndpi_memory = 0, max_ndpi_memory = 0;
u_int8_t enable_protocol_guess = 1, enable_payload_analyzer = 0;
u_int8_t enable_joy_stats = 0;
u_int8_t enable_flow_stats = 0;
u_int8_t human_readeable_string_len = 5;
u_int8_t max_num_udp_dissected_pkts = 16 /* 8 is enough for most protocols, Signal requires more */, max_num_tcp_dissected_pkts = 80 /* due to telnet */;
@@ -46,6 +47,7 @@ int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
char errbuf[PCAP_ERRBUF_SIZE];
NDPI_PROTOCOL_BITMASK all;
char * pcap_path = tempnam("/tmp", "fuzz-ndpi-reader");
u_int i;
if (prefs == NULL) {
prefs = calloc(sizeof(struct ndpi_workflow_prefs), 1);
@@ -55,15 +57,29 @@ int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
}
prefs->decode_tunnels = 1;
prefs->num_roots = 16;
prefs->max_ndpi_flows = 1024;
prefs->max_ndpi_flows = 16 * 1024 * 1024;
prefs->quiet_mode = 0;
workflow = ndpi_workflow_init(prefs, NULL /* pcap handler will be set later */, 0, ndpi_serialization_format_json);
// enable all protocols
NDPI_BITMASK_SET_ALL(all);
ndpi_set_protocol_detection_bitmask2(workflow->ndpi_struct, &all);
memset(workflow->stats.protocol_counter, 0,
sizeof(workflow->stats.protocol_counter));
memset(workflow->stats.protocol_counter_bytes, 0,
sizeof(workflow->stats.protocol_counter_bytes));
memset(workflow->stats.protocol_flows, 0,
sizeof(workflow->stats.protocol_flows));
ndpi_finalize_initialization(workflow->ndpi_struct);
}
bufferToFile(pcap_path, Data, Size);
pkts = pcap_open_offline(pcap_path, errbuf);
if (pkts == NULL) {
remove(pcap_path);
free(pcap_path);
ndpi_term_serializer(&workflow->ndpi_serializer);
return 0;
}
if (ndpi_is_datalink_supported(pcap_datalink(pkts)) == 0)
@@ -72,41 +88,37 @@ int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
pcap_close(pkts);
remove(pcap_path);
free(pcap_path);
ndpi_term_serializer(&workflow->ndpi_serializer);
return 0;
}
struct ndpi_workflow * workflow = ndpi_workflow_init(prefs, pkts);
// enable all protocols
NDPI_BITMASK_SET_ALL(all);
ndpi_set_protocol_detection_bitmask2(workflow->ndpi_struct, &all);
memset(workflow->stats.protocol_counter, 0,
sizeof(workflow->stats.protocol_counter));
memset(workflow->stats.protocol_counter_bytes, 0,
sizeof(workflow->stats.protocol_counter_bytes));
memset(workflow->stats.protocol_flows, 0,
sizeof(workflow->stats.protocol_flows));
ndpi_finalize_initialization(workflow->ndpi_struct);
workflow->pcap_handle = pkts;
/* Init flow tree */
workflow->ndpi_flows_root = ndpi_calloc(workflow->prefs.num_roots, sizeof(void *));
header = NULL;
r = pcap_next_ex(pkts, &header, &pkt);
while (r > 0) {
if(header->caplen >= 42 /* ARP+ size */) {
/* allocate an exact size buffer to check overflows */
uint8_t *packet_checked = malloc(header->caplen);
/* allocate an exact size buffer to check overflows */
uint8_t *packet_checked = malloc(header->caplen);
if(packet_checked) {
ndpi_risk flow_risk;
memcpy(packet_checked, pkt, header->caplen);
ndpi_workflow_process_packet(workflow, header, packet_checked, &flow_risk, NULL);
free(packet_checked);
}
if(packet_checked) {
ndpi_risk flow_risk;
memcpy(packet_checked, pkt, header->caplen);
ndpi_workflow_process_packet(workflow, header, packet_checked, &flow_risk);
free(packet_checked);
}
r = pcap_next_ex(pkts, &header, &pkt);
}
ndpi_workflow_free(workflow);
pcap_close(pkts);
/* Free flow trees */
for(i = 0; i < workflow->prefs.num_roots; i++)
ndpi_tdestroy(workflow->ndpi_flows_root[i], ndpi_flow_info_freer);
ndpi_free(workflow->ndpi_flows_root);
remove(pcap_path);
free(pcap_path);
@@ -159,7 +171,7 @@ int main(int argc, char ** argv)
return 1;
}
if (fread(pcap_buffer, sizeof(*pcap_buffer), pcap_file_size, pcap_file) != pcap_file_size) {
if (fread(pcap_buffer, sizeof(*pcap_buffer), pcap_file_size, pcap_file) != (size_t)pcap_file_size) {
perror("fread failed");
fclose(pcap_file);
free(pcap_buffer);
+88 -9
View File
@@ -4,25 +4,104 @@
#include <stdio.h>
struct ndpi_detection_module_struct *ndpi_info_mod = NULL;
struct ndpi_id_struct *src;
struct ndpi_id_struct *dst;
static ndpi_serializer json_serializer = {};
static ndpi_serializer csv_serializer = {};
int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) {
uint8_t protocol_was_guessed;
if (ndpi_info_mod == NULL) {
ndpi_info_mod = ndpi_init_detection_module(ndpi_no_prefs);
NDPI_PROTOCOL_BITMASK all;
ndpi_info_mod = ndpi_init_detection_module(ndpi_enable_ja3_plus);
NDPI_PROTOCOL_BITMASK all, debug_bitmask;
NDPI_BITMASK_SET_ALL(all);
NDPI_BITMASK_SET_ALL(debug_bitmask);
ndpi_set_protocol_detection_bitmask2(ndpi_info_mod, &all);
src = ndpi_malloc(SIZEOF_ID_STRUCT);
dst = ndpi_malloc(SIZEOF_ID_STRUCT);
ndpi_set_log_level(ndpi_info_mod, 4);
ndpi_set_debug_bitmask(ndpi_info_mod, debug_bitmask);
ndpi_finalize_initialization(ndpi_info_mod);
ndpi_init_serializer(&json_serializer, ndpi_serialization_format_json);
ndpi_init_serializer(&csv_serializer, ndpi_serialization_format_csv);
}
struct ndpi_flow_struct *ndpi_flow = ndpi_flow_malloc(SIZEOF_FLOW_STRUCT);
memset(ndpi_flow, 0, SIZEOF_FLOW_STRUCT);
memset(src, 0, SIZEOF_ID_STRUCT);
memset(dst, 0, SIZEOF_ID_STRUCT);
ndpi_detection_process_packet(ndpi_info_mod, ndpi_flow, Data, Size, 0, src, dst);
ndpi_protocol detected_protocol =
ndpi_detection_process_packet(ndpi_info_mod, ndpi_flow, Data, Size, 0);
ndpi_protocol guessed_protocol =
ndpi_detection_giveup(ndpi_info_mod, ndpi_flow, 1, &protocol_was_guessed);
ndpi_reset_serializer(&json_serializer);
ndpi_reset_serializer(&csv_serializer);
if (protocol_was_guessed == 0)
{
ndpi_dpi2json(ndpi_info_mod, ndpi_flow, detected_protocol, &json_serializer);
ndpi_dpi2json(ndpi_info_mod, ndpi_flow, detected_protocol, &csv_serializer);
} else {
ndpi_dpi2json(ndpi_info_mod, ndpi_flow, guessed_protocol, &json_serializer);
ndpi_dpi2json(ndpi_info_mod, ndpi_flow, guessed_protocol, &csv_serializer);
}
ndpi_free_flow(ndpi_flow);
return 0;
}
#ifdef BUILD_MAIN
int main(int argc, char ** argv)
{
FILE * data_file;
long data_file_size;
uint8_t * data_buffer;
int test_retval;
if (argc != 2) {
fprintf(stderr, "usage: %s: [data-file]\n",
(argc > 0 ? argv[0] : "fuzz_process_packet_with_main"));
return 1;
}
data_file = fopen(argv[1], "r");
if (data_file == NULL) {
perror("fopen failed");
return 1;
}
if (fseek(data_file, 0, SEEK_END) != 0) {
perror("fseek(SEEK_END) failed");
fclose(data_file);
return 1;
}
data_file_size = ftell(data_file);
if (data_file_size < 0) {
perror("ftell failed");
fclose(data_file);
return 1;
}
if (fseek(data_file, 0, SEEK_SET) != 0) {
perror("fseek(0, SEEK_SET) failed");
fclose(data_file);
return 1;
}
data_buffer = malloc(data_file_size);
if (data_buffer == NULL) {
perror("malloc failed");
fclose(data_file);
return 1;
}
if (fread(data_buffer, sizeof(*data_buffer), data_file_size, data_file) != (size_t)data_file_size) {
perror("fread failed");
fclose(data_file);
free(data_buffer);
return 1;
}
test_retval = LLVMFuzzerTestOneInput(data_buffer, data_file_size);
fclose(data_file);
free(data_buffer);
return test_retval;
}
#endif
+28
View File
@@ -0,0 +1,28 @@
CC=@CC@
INC=-I ../src/include -I/usr/local/include
LIBDPI=../src/lib/libndpi.a
LDFLAGS=@LDFLAGS@
LIB=$(LIBDPI) -lm @ADDITIONAL_LIBS@ @LIBS@
TOOLS=metric_anomaly
all: $(TOOLS)
metric_anomaly: metric_anomaly.c Makefile $(LIBDPI)
$(CC) $(CPPFLAGS) -g $(INC) $(LDFLAGS) metric_anomaly.c -o metric_anomaly $(LIB)
clean:
/bin/rm -f *.o $(TOOLS) *~
distclean: clean
/bin/rm -f Makefile
distdir:
find . -type f -name '*.c' \
-o -name '*.txt' | xargs -I'{}' cp '{}' '$(distdir)/{}'
install:
@echo -n ""
check:
true # nothing to do here
+4
View File
@@ -0,0 +1,4 @@
This directory contains a tool that allows anomalies to be identified with InfluxDB
+166
View File
@@ -0,0 +1,166 @@
/*
* metric_anomaly.c
*
* Copyright (C) 2011-22 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <getopt.h>
#include "ndpi_api.h"
#define DEFAULT_ALPHA 0.5
#define DEFAULT_RO 0.05
/* *************************************************** */
static void help() {
printf("Usage: metric_anomaly [-Q][-v][-z][-a <alpha>][-q] -d <database> -q <query>\n"
"-a | Set alpha. Valid range >0 .. <1. Default %.2f\n"
"-Q | Quick output (only anomalies are reported)\n"
"-d <database> | InfluxDB database name\n"
"-q <query> | InfluxQL query\n"
"-v | Verbose\n"
"-z | Bottom metric value set to zero\n"
,
DEFAULT_ALPHA);
printf("\n\nExample: metric_anomaly -d ntopng -q \"%s\"\n",
"SELECT mean(\"cpu0\") FROM \"cpu_load\" WHERE time > 1648634807000000000 GROUP BY time(60s) fill(previous)");
exit(0);
}
/* *************************************************** */
int main(int argc, char *argv[]) {
char *database = NULL, *query = NULL, cmd[512], buf[256];
u_int i, j, first = 1, quick_mode = 0, verbose = 0;
struct ndpi_ses_struct ses;
float alpha, ro;
char c;
FILE *fd;
bool go_below_zero = true;
/* Defaults */
alpha = DEFAULT_ALPHA;
ro = DEFAULT_RO;
while((c = getopt(argc, argv, "a:Qd:q:vz")) != '?') {
if(c == -1) break;
switch(c) {
case 'a':
{
float f = atof(optarg);
if((f > 0) && (f < 1))
alpha = f;
else
printf("Discarding -a: valid range is >0 .. <1\n");
}
break;
case 'Q':
quick_mode = 1;
break;
case 'd':
database = optarg;
break;
case 'q':
query = optarg;
break;
case 'v':
verbose = 1;
break;
case 'z':
go_below_zero = false;
break;
default:
help();
break;
}
}
if((database == NULL) || (query == NULL))
help();
ndpi_snprintf(cmd, sizeof(cmd), "influx -database '%s' -precision s -execute '%s'", database, query);
if(verbose) printf("%s\n", cmd);
if ((fd = popen(cmd, "r")) == NULL) {
printf("Unable to execute '%s'\n", cmd);
return(-1);
}
ndpi_ses_init(&ses, alpha, ro);
while(fgets(buf, sizeof(buf), fd) != NULL) {
u_int32_t epoch;
float value;
double prediction, confidence_band;
double lower, upper;
int rc;
bool is_anomaly;
if(sscanf(buf, "%u %f", &epoch, &value) != 2)
continue;
// printf("->>> '%s'", buf);
value *= 100; /* trick to avoid dealing with floats */
rc = ndpi_ses_add_value(&ses, value, &prediction, &confidence_band);
lower = prediction - confidence_band, upper = prediction + confidence_band;
if(!go_below_zero) lower = ndpi_max(lower, 0), upper = ndpi_max(upper, 0);
is_anomaly = ((rc == 0) || (confidence_band == 0) || ((value >= lower) && (value <= upper))) ? false : true;
if(verbose || is_anomaly) {
const time_t _t = epoch;
struct tm *t_info = localtime((const time_t*)&_t);
strftime(buf, sizeof(buf), "%d/%b/%Y %H:%M:%S", t_info);
if(quick_mode) {
if(is_anomaly) {
printf("%u [%s]\n", epoch, buf);
}
} else {
if(first) {
first = 0;
printf("%s %s\t%s %s %s\t %s [%s]\n",
"When", "Value", "Prediction", "Lower", "Upper", "Out", "Band");
}
printf("%s %12.3f\t%.3f\t%12.3f\t%12.3f\t %s [%.3f][rc: %d]\n",
buf, value/100., prediction/100., lower/100., upper/100., is_anomaly? "ANOMALY" : "OK",
confidence_band/100., rc);
}
}
}
(void)pclose(fd);
return(0);
}
+2
View File
@@ -0,0 +1,2 @@
queries:
- exclude: cpp/cleartext-storage-file
+153 -115
View File
@@ -1,5 +1,5 @@
# ===========================================================================
# http://www.gnu.org/software/autoconf-archive/ax_pthread.html
# https://www.gnu.org/software/autoconf-archive/ax_pthread.html
# ===========================================================================
#
# SYNOPSIS
@@ -14,20 +14,24 @@
# flags that are needed. (The user can also force certain compiler
# flags/libs to be tested by setting these environment variables.)
#
# Also sets PTHREAD_CC to any special C compiler that is needed for
# multi-threaded programs (defaults to the value of CC otherwise). (This
# is necessary on AIX to use the special cc_r compiler alias.)
# Also sets PTHREAD_CC and PTHREAD_CXX to any special C compiler that is
# needed for multi-threaded programs (defaults to the value of CC
# respectively CXX otherwise). (This is necessary on e.g. AIX to use the
# special cc_r/CC_r compiler alias.)
#
# NOTE: You are assumed to not only compile your program with these flags,
# but also to link with them as well. For example, you might link with
# $PTHREAD_CC $CFLAGS $PTHREAD_CFLAGS $LDFLAGS ... $PTHREAD_LIBS $LIBS
# $PTHREAD_CXX $CXXFLAGS $PTHREAD_CFLAGS $LDFLAGS ... $PTHREAD_LIBS $LIBS
#
# If you are only building threaded programs, you may wish to use these
# variables in your default LIBS, CFLAGS, and CC:
#
# LIBS="$PTHREAD_LIBS $LIBS"
# CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
# CXXFLAGS="$CXXFLAGS $PTHREAD_CFLAGS"
# CC="$PTHREAD_CC"
# CXX="$PTHREAD_CXX"
#
# In addition, if the PTHREAD_CREATE_JOINABLE thread-attribute constant
# has a nonstandard name, this macro defines PTHREAD_CREATE_JOINABLE to
@@ -55,6 +59,7 @@
#
# Copyright (c) 2008 Steven G. Johnson <stevenj@alum.mit.edu>
# Copyright (c) 2011 Daniel Richard G. <skunk@iSKUNK.ORG>
# Copyright (c) 2019 Marc Stevens <marc.stevens@cwi.nl>
#
# This program is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
@@ -67,7 +72,7 @@
# Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program. If not, see <http://www.gnu.org/licenses/>.
# with this program. If not, see <https://www.gnu.org/licenses/>.
#
# As a special exception, the respective Autoconf Macro's copyright owner
# gives unlimited permission to copy, distribute and modify the configure
@@ -82,7 +87,7 @@
# modified version of the Autoconf Macro, you may extend this special
# exception to the GPL to apply to your modified version as well.
#serial 23
#serial 31
AU_ALIAS([ACX_PTHREAD], [AX_PTHREAD])
AC_DEFUN([AX_PTHREAD], [
@@ -104,6 +109,7 @@ if test "x$PTHREAD_CFLAGS$PTHREAD_LIBS" != "x"; then
ax_pthread_save_CFLAGS="$CFLAGS"
ax_pthread_save_LIBS="$LIBS"
AS_IF([test "x$PTHREAD_CC" != "x"], [CC="$PTHREAD_CC"])
AS_IF([test "x$PTHREAD_CXX" != "x"], [CXX="$PTHREAD_CXX"])
CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
LIBS="$PTHREAD_LIBS $LIBS"
AC_MSG_CHECKING([for pthread_join using $CC $PTHREAD_CFLAGS $PTHREAD_LIBS])
@@ -123,10 +129,12 @@ fi
# (e.g. DEC) have both -lpthread and -lpthreads, where one of the
# libraries is broken (non-POSIX).
# Create a list of thread flags to try. Items starting with a "-" are
# C compiler flags, and other items are library names, except for "none"
# which indicates that we try without any flags at all, and "pthread-config"
# which is a program returning the flags for the Pth emulation library.
# Create a list of thread flags to try. Items with a "," contain both
# C compiler flags (before ",") and linker flags (after ","). Other items
# starting with a "-" are C compiler flags, and remaining items are
# library names, except for "none" which indicates that we try without
# any flags at all, and "pthread-config" which is a program returning
# the flags for the Pth emulation library.
ax_pthread_flags="pthreads none -Kthread -pthread -pthreads -mthreads pthread --thread-safe -mt pthread-config"
@@ -194,14 +202,47 @@ case $host_os in
# that too in a future libc.) So we'll check first for the
# standard Solaris way of linking pthreads (-mt -lpthread).
ax_pthread_flags="-mt,pthread pthread $ax_pthread_flags"
ax_pthread_flags="-mt,-lpthread pthread $ax_pthread_flags"
;;
esac
# Are we compiling with Clang?
AC_CACHE_CHECK([whether $CC is Clang],
[ax_cv_PTHREAD_CLANG],
[ax_cv_PTHREAD_CLANG=no
# Note that Autoconf sets GCC=yes for Clang as well as GCC
if test "x$GCC" = "xyes"; then
AC_EGREP_CPP([AX_PTHREAD_CC_IS_CLANG],
[/* Note: Clang 2.7 lacks __clang_[a-z]+__ */
# if defined(__clang__) && defined(__llvm__)
AX_PTHREAD_CC_IS_CLANG
# endif
],
[ax_cv_PTHREAD_CLANG=yes])
fi
])
ax_pthread_clang="$ax_cv_PTHREAD_CLANG"
# GCC generally uses -pthread, or -pthreads on some platforms (e.g. SPARC)
# Note that for GCC and Clang -pthread generally implies -lpthread,
# except when -nostdlib is passed.
# This is problematic using libtool to build C++ shared libraries with pthread:
# [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25460
# [2] https://bugzilla.redhat.com/show_bug.cgi?id=661333
# [3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=468555
# To solve this, first try -pthread together with -lpthread for GCC
AS_IF([test "x$GCC" = "xyes"],
[ax_pthread_flags="-pthread -pthreads $ax_pthread_flags"])
[ax_pthread_flags="-pthread,-lpthread -pthread -pthreads $ax_pthread_flags"])
# Clang takes -pthread (never supported any other flag), but we'll try with -lpthread first
AS_IF([test "x$ax_pthread_clang" = "xyes"],
[ax_pthread_flags="-pthread,-lpthread -pthread"])
# The presence of a feature test macro requesting re-entrant function
# definitions is, on some systems, a strong hint that pthreads support is
@@ -224,101 +265,6 @@ AS_IF([test "x$ax_pthread_check_macro" = "x--"],
[ax_pthread_check_cond=0],
[ax_pthread_check_cond="!defined($ax_pthread_check_macro)"])
# Are we compiling with Clang?
AC_CACHE_CHECK([whether $CC is Clang],
[ax_cv_PTHREAD_CLANG],
[ax_cv_PTHREAD_CLANG=no
# Note that Autoconf sets GCC=yes for Clang as well as GCC
if test "x$GCC" = "xyes"; then
AC_EGREP_CPP([AX_PTHREAD_CC_IS_CLANG],
[/* Note: Clang 2.7 lacks __clang_[a-z]+__ */
# if defined(__clang__) && defined(__llvm__)
AX_PTHREAD_CC_IS_CLANG
# endif
],
[ax_cv_PTHREAD_CLANG=yes])
fi
])
ax_pthread_clang="$ax_cv_PTHREAD_CLANG"
ax_pthread_clang_warning=no
# Clang needs special handling, because older versions handle the -pthread
# option in a rather... idiosyncratic way
if test "x$ax_pthread_clang" = "xyes"; then
# Clang takes -pthread; it has never supported any other flag
# (Note 1: This will need to be revisited if a system that Clang
# supports has POSIX threads in a separate library. This tends not
# to be the way of modern systems, but it's conceivable.)
# (Note 2: On some systems, notably Darwin, -pthread is not needed
# to get POSIX threads support; the API is always present and
# active. We could reasonably leave PTHREAD_CFLAGS empty. But
# -pthread does define _REENTRANT, and while the Darwin headers
# ignore this macro, third-party headers might not.)
PTHREAD_CFLAGS="-pthread"
PTHREAD_LIBS=
ax_pthread_ok=yes
# However, older versions of Clang make a point of warning the user
# that, in an invocation where only linking and no compilation is
# taking place, the -pthread option has no effect ("argument unused
# during compilation"). They expect -pthread to be passed in only
# when source code is being compiled.
#
# Problem is, this is at odds with the way Automake and most other
# C build frameworks function, which is that the same flags used in
# compilation (CFLAGS) are also used in linking. Many systems
# supported by AX_PTHREAD require exactly this for POSIX threads
# support, and in fact it is often not straightforward to specify a
# flag that is used only in the compilation phase and not in
# linking. Such a scenario is extremely rare in practice.
#
# Even though use of the -pthread flag in linking would only print
# a warning, this can be a nuisance for well-run software projects
# that build with -Werror. So if the active version of Clang has
# this misfeature, we search for an option to squash it.
AC_CACHE_CHECK([whether Clang needs flag to prevent "argument unused" warning when linking with -pthread],
[ax_cv_PTHREAD_CLANG_NO_WARN_FLAG],
[ax_cv_PTHREAD_CLANG_NO_WARN_FLAG=unknown
# Create an alternate version of $ac_link that compiles and
# links in two steps (.c -> .o, .o -> exe) instead of one
# (.c -> exe), because the warning occurs only in the second
# step
ax_pthread_save_ac_link="$ac_link"
ax_pthread_sed='s/conftest\.\$ac_ext/conftest.$ac_objext/g'
ax_pthread_link_step=`$as_echo "$ac_link" | sed "$ax_pthread_sed"`
ax_pthread_2step_ac_link="($ac_compile) && (echo ==== >&5) && ($ax_pthread_link_step)"
ax_pthread_save_CFLAGS="$CFLAGS"
for ax_pthread_try in '' -Qunused-arguments -Wno-unused-command-line-argument unknown; do
AS_IF([test "x$ax_pthread_try" = "xunknown"], [break])
CFLAGS="-Werror -Wunknown-warning-option $ax_pthread_try -pthread $ax_pthread_save_CFLAGS"
ac_link="$ax_pthread_save_ac_link"
AC_LINK_IFELSE([AC_LANG_SOURCE([[int main(void){return 0;}]])],
[ac_link="$ax_pthread_2step_ac_link"
AC_LINK_IFELSE([AC_LANG_SOURCE([[int main(void){return 0;}]])],
[break])
])
done
ac_link="$ax_pthread_save_ac_link"
CFLAGS="$ax_pthread_save_CFLAGS"
AS_IF([test "x$ax_pthread_try" = "x"], [ax_pthread_try=no])
ax_cv_PTHREAD_CLANG_NO_WARN_FLAG="$ax_pthread_try"
])
case "$ax_cv_PTHREAD_CLANG_NO_WARN_FLAG" in
no | unknown) ;;
*) PTHREAD_CFLAGS="$ax_cv_PTHREAD_CLANG_NO_WARN_FLAG $PTHREAD_CFLAGS" ;;
esac
fi # $ax_pthread_clang = yes
if test "x$ax_pthread_ok" = "xno"; then
for ax_pthread_try_flag in $ax_pthread_flags; do
@@ -328,10 +274,10 @@ for ax_pthread_try_flag in $ax_pthread_flags; do
AC_MSG_CHECKING([whether pthreads work without any flags])
;;
-mt,pthread)
AC_MSG_CHECKING([whether pthreads work with -mt -lpthread])
PTHREAD_CFLAGS="-mt"
PTHREAD_LIBS="-lpthread"
*,*)
PTHREAD_CFLAGS=`echo $ax_pthread_try_flag | sed "s/^\(.*\),\(.*\)$/\1/"`
PTHREAD_LIBS=`echo $ax_pthread_try_flag | sed "s/^\(.*\),\(.*\)$/\2/"`
AC_MSG_CHECKING([whether pthreads work with "$PTHREAD_CFLAGS" and "$PTHREAD_LIBS"])
;;
-*)
@@ -371,7 +317,13 @@ for ax_pthread_try_flag in $ax_pthread_flags; do
# if $ax_pthread_check_cond
# error "$ax_pthread_check_macro must be defined"
# endif
static void routine(void *a) { a = 0; }
static void *some_global = NULL;
static void routine(void *a)
{
/* To avoid any unused-parameter or
unused-but-set-parameter warning. */
some_global = a;
}
static void *start_routine(void *a) { return a; }],
[pthread_t th; pthread_attr_t attr;
pthread_create(&th, 0, start_routine, 0);
@@ -393,6 +345,80 @@ for ax_pthread_try_flag in $ax_pthread_flags; do
done
fi
# Clang needs special handling, because older versions handle the -pthread
# option in a rather... idiosyncratic way
if test "x$ax_pthread_clang" = "xyes"; then
# Clang takes -pthread; it has never supported any other flag
# (Note 1: This will need to be revisited if a system that Clang
# supports has POSIX threads in a separate library. This tends not
# to be the way of modern systems, but it's conceivable.)
# (Note 2: On some systems, notably Darwin, -pthread is not needed
# to get POSIX threads support; the API is always present and
# active. We could reasonably leave PTHREAD_CFLAGS empty. But
# -pthread does define _REENTRANT, and while the Darwin headers
# ignore this macro, third-party headers might not.)
# However, older versions of Clang make a point of warning the user
# that, in an invocation where only linking and no compilation is
# taking place, the -pthread option has no effect ("argument unused
# during compilation"). They expect -pthread to be passed in only
# when source code is being compiled.
#
# Problem is, this is at odds with the way Automake and most other
# C build frameworks function, which is that the same flags used in
# compilation (CFLAGS) are also used in linking. Many systems
# supported by AX_PTHREAD require exactly this for POSIX threads
# support, and in fact it is often not straightforward to specify a
# flag that is used only in the compilation phase and not in
# linking. Such a scenario is extremely rare in practice.
#
# Even though use of the -pthread flag in linking would only print
# a warning, this can be a nuisance for well-run software projects
# that build with -Werror. So if the active version of Clang has
# this misfeature, we search for an option to squash it.
AC_CACHE_CHECK([whether Clang needs flag to prevent "argument unused" warning when linking with -pthread],
[ax_cv_PTHREAD_CLANG_NO_WARN_FLAG],
[ax_cv_PTHREAD_CLANG_NO_WARN_FLAG=unknown
# Create an alternate version of $ac_link that compiles and
# links in two steps (.c -> .o, .o -> exe) instead of one
# (.c -> exe), because the warning occurs only in the second
# step
ax_pthread_save_ac_link="$ac_link"
ax_pthread_sed='s/conftest\.\$ac_ext/conftest.$ac_objext/g'
ax_pthread_link_step=`AS_ECHO(["$ac_link"]) | sed "$ax_pthread_sed"`
ax_pthread_2step_ac_link="($ac_compile) && (echo ==== >&5) && ($ax_pthread_link_step)"
ax_pthread_save_CFLAGS="$CFLAGS"
for ax_pthread_try in '' -Qunused-arguments -Wno-unused-command-line-argument unknown; do
AS_IF([test "x$ax_pthread_try" = "xunknown"], [break])
CFLAGS="-Werror -Wunknown-warning-option $ax_pthread_try -pthread $ax_pthread_save_CFLAGS"
ac_link="$ax_pthread_save_ac_link"
AC_LINK_IFELSE([AC_LANG_SOURCE([[int main(void){return 0;}]])],
[ac_link="$ax_pthread_2step_ac_link"
AC_LINK_IFELSE([AC_LANG_SOURCE([[int main(void){return 0;}]])],
[break])
])
done
ac_link="$ax_pthread_save_ac_link"
CFLAGS="$ax_pthread_save_CFLAGS"
AS_IF([test "x$ax_pthread_try" = "x"], [ax_pthread_try=no])
ax_cv_PTHREAD_CLANG_NO_WARN_FLAG="$ax_pthread_try"
])
case "$ax_cv_PTHREAD_CLANG_NO_WARN_FLAG" in
no | unknown) ;;
*) PTHREAD_CFLAGS="$ax_cv_PTHREAD_CLANG_NO_WARN_FLAG $PTHREAD_CFLAGS" ;;
esac
fi # $ax_pthread_clang = yes
# Various other checks:
if test "x$ax_pthread_ok" = "xyes"; then
ax_pthread_save_CFLAGS="$CFLAGS"
@@ -438,7 +464,8 @@ if test "x$ax_pthread_ok" = "xyes"; then
AC_CACHE_CHECK([for PTHREAD_PRIO_INHERIT],
[ax_cv_PTHREAD_PRIO_INHERIT],
[AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include <pthread.h>]],
[[int i = PTHREAD_PRIO_INHERIT;]])],
[[int i = PTHREAD_PRIO_INHERIT;
return i;]])],
[ax_cv_PTHREAD_PRIO_INHERIT=yes],
[ax_cv_PTHREAD_PRIO_INHERIT=no])
])
@@ -460,18 +487,28 @@ if test "x$ax_pthread_ok" = "xyes"; then
[#handle absolute path differently from PATH based program lookup
AS_CASE(["x$CC"],
[x/*],
[AS_IF([AS_EXECUTABLE_P([${CC}_r])],[PTHREAD_CC="${CC}_r"])],
[AC_CHECK_PROGS([PTHREAD_CC],[${CC}_r],[$CC])])])
[
AS_IF([AS_EXECUTABLE_P([${CC}_r])],[PTHREAD_CC="${CC}_r"])
AS_IF([test "x${CXX}" != "x"], [AS_IF([AS_EXECUTABLE_P([${CXX}_r])],[PTHREAD_CXX="${CXX}_r"])])
],
[
AC_CHECK_PROGS([PTHREAD_CC],[${CC}_r],[$CC])
AS_IF([test "x${CXX}" != "x"], [AC_CHECK_PROGS([PTHREAD_CXX],[${CXX}_r],[$CXX])])
]
)
])
;;
esac
fi
fi
test -n "$PTHREAD_CC" || PTHREAD_CC="$CC"
test -n "$PTHREAD_CXX" || PTHREAD_CXX="$CXX"
AC_SUBST([PTHREAD_LIBS])
AC_SUBST([PTHREAD_CFLAGS])
AC_SUBST([PTHREAD_CC])
AC_SUBST([PTHREAD_CXX])
# Finally, execute ACTION-IF-FOUND/ACTION-IF-NOT-FOUND:
if test "x$ax_pthread_ok" = "xyes"; then
@@ -483,3 +520,4 @@ else
fi
AC_LANG_POP
])dnl AX_PTHREAD
+6 -12
View File
@@ -8,7 +8,7 @@
include $(TOPDIR)/rules.mk
PKG_NAME:=libndpi
PKG_VERSION:=3.4
PKG_VERSION:=4.0
PKG_RELEASE:=1
PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz
@@ -28,8 +28,8 @@ PKG_BUILD_PARALLEL:=1
include $(INCLUDE_DIR)/uclibc++.mk
include $(INCLUDE_DIR)/package.mk
ifeq ($(CONFIG_LIBNDPI_GCRYPT),)
CONFIGURE_ARGS += --disable-gcrypt
ifneq ($(CONFIG_LIBNDPI_GCRYPT),)
CONFIGURE_ARGS += --with-local-libgcrypt
endif
define Package/libndpi
@@ -47,24 +47,18 @@ endef
define Package/libndpi/config
config LIBNDPI_GCRYPT
bool "GCrypt support"
bool "Use external libgcrypt"
depends on PACKAGE_libndpi
default n
help
This option enables QUIC client hello decryption.
This option enables QUIC client hello decryption through
an external libgcrypt instead of a lightweight builtin version.
Disabled by default.
endef
define Build/Prepare
$(PKG_UNPACK)
$(Build/Patch)
mv $(PKG_BUILD_DIR)/configure.seed $(PKG_BUILD_DIR)/configure.ac
$(SED) "s/@NDPI_MAJOR@/3/g" \
-e "s/@NDPI_MINOR@/4/g" \
-e "s/@NDPI_PATCH@/0/g" \
-e "s/@NDPI_VERSION_SHORT@/3.4.0/g" \
-e "s/@FUZZY@/dnl> /g" \
$(PKG_BUILD_DIR)/configure.ac
endef
define Build/InstallDev
+5 -11
View File
@@ -27,8 +27,8 @@ PKG_BUILD_PARALLEL:=1
include $(INCLUDE_DIR)/uclibc++.mk
include $(INCLUDE_DIR)/package.mk
ifeq ($(CONFIG_LIBNDPI_GCRYPT),)
CONFIGURE_ARGS += --disable-gcrypt
ifneq ($(CONFIG_LIBNDPI_GCRYPT),)
CONFIGURE_ARGS += --with-local-libgcrypt
endif
define Package/libndpi
@@ -46,11 +46,12 @@ endef
define Package/libndpi/config
config LIBNDPI_GCRYPT
bool "GCrypt support"
bool "Use external libgcrypt"
depends on PACKAGE_libndpi
default n
help
This option enables QUIC client hello decryption.
This option enables QUIC client hello decryption through
an external libgcrypt instead of a lightweight builtin version.
Disabled by default.
endef
@@ -58,13 +59,6 @@ define Build/Prepare
cp -r $(SOURCE_DIR)/* $(PKG_BUILD_DIR)
$(PKG_UNPACK)
$(Build/Patch)
mv $(PKG_BUILD_DIR)/configure.seed $(PKG_BUILD_DIR)/configure.ac
$(SED) "s/@NDPI_MAJOR@/3/g" \
-e "s/@NDPI_MINOR@/4/g" \
-e "s/@NDPI_PATCH@/0/g" \
-e "s/@NDPI_VERSION_SHORT@/3.4.0/g" \
-e "s/@FUZZY@/dnl> /g" \
$(PKG_BUILD_DIR)/configure.ac
endef
define Build/InstallDev
+6 -5
View File
@@ -5,14 +5,15 @@ ndpi:
\rm -rf ./debian/ndpi-tmp ./debian/ndpi-dev-tmp ./debian/ndpi ./debian/ndpi-dev
mkdir -p ./debian/ndpi-tmp ./debian/ndpi-dev-tmp
mkdir -p ./debian/ndpi-tmp/usr/lib ./debian/ndpi-tmp/usr/bin
mkdir -p ./debian/ndpi-dev-tmp/usr/lib ./debian/ndpi-dev-tmp/usr/include/ndpi
cd ${NDPI_HOME}; ./autogen.sh; ./configure; make
mkdir -p ./debian/ndpi-dev-tmp/usr/lib ./debian/ndpi-dev-tmp/usr/include/ndpi ./debian/ndpi-dev-tmp/usr/lib/pkgconfig
cd ${NDPI_HOME}; ./autogen.sh; ./configure; $(MAKE)
cp $(NDPI_HOME)/src/lib/libndpi.so.@NDPI_VERS@ ./debian/ndpi-tmp/usr/lib/
cd ./debian/ndpi-tmp/usr/lib/; ln -s libndpi.so.@NDPI_VERS@ libndpi.so; cd -
cd ./debian/ndpi-tmp/usr/lib/; ln -s libndpi.so.@NDPI_VERS@ libndpi.so.@MAJOR_RELEASE@; cd -
cp $(NDPI_HOME)/src/lib/libndpi.a ./debian/ndpi-dev-tmp/usr/lib/
cp $(NDPI_HOME)/example/ndpiReader ./debian/ndpi-tmp/usr/bin/
cp $(NDPI_HOME)/src/include/*.h ./debian/ndpi-dev-tmp/usr/include/ndpi/
cp $(NDPI_HOME)/libndpi.pc ./debian/ndpi-dev-tmp/usr/lib/pkgconfig/
-rm -fr ./debian/ndpi-dev-tmp/usr/include/ndpi/ndpi_win32.h*
@echo
@find ./debian/ndpi-tmp -name "*~" -exec /bin/rm {} ';'
@@ -32,11 +33,11 @@ ndpi:
@echo "-------------------------------"
distclean:
-cd ${NDPI_HOME} && make distclean
-cd ${NDPI_HOME} && $(MAKE) distclean
install:
cd ${NDPI_HOME}; make install DESTDIR="${NDPI_BUILD}/debian/tmp" AM_UPDATE_INFO_DIR=no
cd ${NDPI_HOME}; $(MAKE) install DESTDIR="${NDPI_BUILD}/debian/tmp" AM_UPDATE_INFO_DIR=no
clean:
-cd ${NDPI_HOME} && make clean
-cd ${NDPI_HOME} && $(MAKE) clean
-rm -rf *~ *deb ./usr ./debian/ndpi ./debian/ndpi-dev ./debian/ndpi ./debian/ndpi-dev
+2 -2
View File
@@ -1,8 +1,8 @@
#!/bin/bash
SCRIPTPATH="$(cd "$(dirname "$0")"; pwd -P)"
RELEASE="$(cd ${SCRIPTPATH}; cat ../configure.ac|grep C_INIT|cut -c 21-|rev|cut -c 3-|rev)"
MAJOR_RELEASE="$(cd ${SCRIPTPATH}; cat ../configure.ac|grep C_INIT|cut -c 21-|rev|cut -c 3-|rev|cut -d. -f1)"
RELEASE="$(cd ${SCRIPTPATH}; cat ../configure.ac|grep C_INIT|cut -c 20-|rev|cut -c 3-|rev)"
MAJOR_RELEASE="$(cd ${SCRIPTPATH}; cat ../configure.ac|grep C_INIT|cut -c 20-|rev|cut -c 3-|rev|cut -d. -f1)"
REVISION="$(cd ${SCRIPTPATH}; git rev-list --all |wc -l | tr -d '[[:space:]]')"
get_release() {
+34
View File
@@ -0,0 +1,34 @@
# Python Bindings Development Guide
The aim of this document is to guide when extending these bindings with additional nDPI API.
In the following we suppose that we want to add the following API to ndpi python package.
``` c
int ndpi_des_init(struct ndpi_des_struct *des, double alpha, double beta, float significance);
```
## Add it to NDPI_APIS Python definition
[**NDPI_APIS**][py_ndpi_api] must be updated with the this new API your want to add.
## Regenerate bindings
``` bash
python3 setup.py install
```
## That's it!
Now this API can be called and used on python side
``` python
from ndpi import lib, ffi
des = ffi.new("struct ndpi_des_struct *")
alpha = 0.9
beta = 0.5
lib.ndpi_des_init(des, alpha, beta, 0.05)
```
[py_ndpi_api]: https://github.com/ntop/nDPI/blob/c47d710d8e5281fff2f1f90ff5462c16ac91d50c/python/ndpi/ndpi_build.py#L49
-26
View File
@@ -1,26 +0,0 @@
CC=@CC@
CFLAGS=-I. -I../src/include -I./src/lib/third_party/include @CFLAGS@ @CUSTOM_NDPI@ -shared
#LIBNDPI=../src/lib/libndpi.so.@NDPI_VERSION_SHORT@
LIBNDPI=../src/lib/libndpi.a
LDFLAGS=$(CFILE) $(LIBNDPI) -lpcap @LIBS@ @ADDITIONAL_LIBS@
SHARE = -soname,ndpi_wrap
SO=ndpi_wrap.so
OBJS = ndpi_wrap.o
CFILE = ndpi_wrap.c
PIC = -fPIC
PREFIX?=/usr/local
.PHONY: all
UNAME_S := $(shell uname -s)
ifeq ($(UNAME_S),Darwin)#do something for linux
SHARE=-install_name,ndpiReader.so
endif
all: $(SO)
$(SO): $(CFILE) $(LIBNDPI) Makefile
$(CC) $(CFLAGS) -Wl,$(SHARE) -o $@ $(PIC) $(LDFLAGS)
# ln -s $(LIBNDPI) .
clean:
/bin/rm -f $(SO)
+81
View File
@@ -0,0 +1,81 @@
# ndpi
This package contains Python bindings for nDPI. nDPI is an Open and Extensible LGPLv3 Deep Packet Inspection Library.
**ndpi** is implemented using [**CFFI**][cffi] (out-of-line API mode). Consequently, it is fast and [**PyPy**][pypy]
compliant.
## Installation
### Build nDPI
``` bash
git clone --branch dev https://github.com/ntop/nDPI.git
cd nDPI
./autogen.sh
./configure
make
sudo make install
```
### Install ndpi package
``` bash
cd python
python3 -m pip install --upgrade pip
python3 -m install -r dev_requirements.txt
python3 -m pip install .
```
## Usage
### API
``` python
from ndpi import NDPI, NDPIFlow
nDPI = NDPI()
# You per flow processing here
# ...
ndpi_flow = NDPIFlow()
nDPI.process_packet(ndpi_flow, ip_bytes, time_ms)
nDPI.giveup(ndpi_flow) # If you want to guess it instead (DPI fallback)
```
### Example Application
[ndpi_example.py][ndpi_example] is provided to demonstrate how **ndpi** can be integrated within your Python application.
``` bash
Using nDPI 4.3.0-3532-8dd70b70
usage: ndpi_example.py [-h] [-u] input
positional arguments:
input input pcap file path
optional arguments:
-h, --help show this help message and exit
-u, --include-unknowns
```
Example with a Skype capture file
``` bash
python3 ndpi_example.py -u ../tests/pcap/skype.pcap
```
## Related projects
The provided example is for demo purposes only, For additional features (live capture, multiplatform support,
multiprocessing, ML based classification, system visibility, etc.), please check nDPI based
framework, [**NFStream**][nfstream].
## License
This project is licensed under the LGPLv3 License - see the [**License**][license] file for details.
[license]: https://github.com/ntop/nDPI/blob/dev/COPYING
[cffi]: https://cffi.readthedocs.io/en/latest/
[pypy]: https://www.pypy.org/
[nfstream]: https://github.com/nfstream/nfstream
[ndpi_example]: https://github.com/ntop/nDPI/blob/dev/python/ndpi_example.py
-43
View File
@@ -1,43 +0,0 @@
nDPI Python bindings
--------------------
This directory contains the Python3 bindings for nDPI. We provide both cffi and ctypes based bindings.
**cffi bindings**
Files:
* ndpi.py
Example (using `nfstream <https://github.com/aouinizied/nfstream>`_ package):
.. code-block:: bash
pip3 install nfstream
python3 flow_printer.py <interface>
python3 flow_printer.py <pcap_file>
Code courtesy:
* Zied Aouini
**ctypes bindings**
Files:
* ndpi_typestruct.py
* ndpi_wrap.c
* Makefile.in
Example:
.. code-block:: bash
pip3 install scapy
python3 ndpi_example.py <interface>
python3 ndpi_example.py <pcap_file>
Code courtesy:
* Massimo Puddu
* Zied Aouini
+2
View File
@@ -0,0 +1,2 @@
cffi>=1.15.0
dpkt>=1.9.7.2
-38
View File
@@ -1,38 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
------------------------------------------------------------------------------------------------------------------------
flow_printer.py
Copyright (C) 2019-20 - NFStream Developers
This file is part of NFStream, a Flexible Network Data Analysis Framework (https://www.nfstream.org/).
NFStream is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
NFStream is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
"""
from nfstream import NFStreamer
import sys
# Example must run with nfstream >= 6.1.1
path = sys.argv[1]
flow_streamer = NFStreamer(source=path, statistical_analysis=False, performance_report=1)
result = {}
try:
for flow in flow_streamer:
print(flow)
try:
result[flow.application_name] += flow.bidirectional_packets
except KeyError:
result[flow.application_name] = flow.bidirectional_packets
print("\nSummary (Application Name: Packets):")
print(result)
except KeyboardInterrupt:
print("\nSummary (Application Name: Packets):")
print(result)
print("Terminated.")
-1465
View File
File diff suppressed because it is too large Load Diff
+21
View File
@@ -0,0 +1,21 @@
"""
------------------------------------------------------------------------------------------------------------------------
__init__.py
Copyright (C) 2011-22 - ntop.org
This file is part of nDPI, an open source deep packet inspection library.
nDPI is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
nDPI is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
"""
from .ndpi import *
__author__ = """Zied Aouini"""
__email__ = 'aouinizied@gmail.com'
__version__ = '4.3.0'
+101
View File
@@ -0,0 +1,101 @@
"""
------------------------------------------------------------------------------------------------------------------------
ndpi.py
Copyright (C) 2011-22 - ntop.org
This file is part of nDPI, an open source deep packet inspection library.
nDPI is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
nDPI is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
"""
from collections import namedtuple
from _ndpi import ffi, lib
ndpi_protocol = namedtuple('NDPIProtocol', ['C',
'master_protocol',
'app_protocol',
'category'])
ndpi_confidence = namedtuple('NDPIConfidence', ['id',
'name'])
class NDPI(object):
__slots__ = ("_api_version",
"_revision",
"_detection_module")
def __init__(self):
self._detection_module = lib.ndpi_init_detection_module(0)
if self._detection_module == ffi.NULL:
raise MemoryError("Unable to instantiate NDPI object")
lib.ndpi_py_setup_detection_module(self._detection_module)
@property
def api_version(self):
return lib.ndpi_get_api_version()
@property
def revision(self):
return ffi.string(lib.ndpi_revision()).decode('utf-8', errors='ignore')
def process_packet(self, flow, packet, packet_time_ms):
p = lib.ndpi_detection_process_packet(self._detection_module,
flow.C,
packet,
len(packet),
int(packet_time_ms))
return ndpi_protocol(C=p,
master_protocol=p.master_protocol,
app_protocol=p.app_protocol,
category=p.category)
def giveup(self, flow, enable_guess=True):
p = lib.ndpi_detection_giveup(self._detection_module,
flow.C,
enable_guess,
ffi.new("uint8_t*", 0))
return ndpi_protocol(C=p,
master_protocol=p.master_protocol,
app_protocol=p.app_protocol,
category=p.category)
def protocol_name(self, protocol):
buf = ffi.new("char[40]")
lib.ndpi_protocol2name(self._detection_module, protocol.C, buf, ffi.sizeof(buf))
return ffi.string(buf).decode('utf-8', errors='ignore')
def protocol_category_name(self, protocol):
return ffi.string(lib.ndpi_category_get_name(self._detection_module,
protocol.C.category)).decode('utf-8',
errors='ignore')
def __del__(self):
if self._detection_module != ffi.NULL:
lib.ndpi_exit_detection_module(self._detection_module)
class NDPIFlow(object):
__slots__ = "C"
@property
def confidence(self):
confidence = self.C.confidence
return ndpi_confidence(id=confidence,
name=ffi.string(lib.ndpi_confidence_get_name(confidence)).decode('utf-8',
errors='ignore'))
def __init__(self):
self.C = lib.ndpi_py_initialize_flow()
def __del__(self):
if self.C != ffi.NULL:
lib.ndpi_flow_free(self.C)
self.C = ffi.NULL
+137
View File
@@ -0,0 +1,137 @@
"""
------------------------------------------------------------------------------------------------------------------------
ndpi_build.py
Copyright (C) 2011-22 - ntop.org
This file is part of nDPI, an open source deep packet inspection library.
nDPI is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
nDPI is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
"""
from cffi import FFI
import subprocess
import pathlib
NDPI_INCLUDES = """
#include "ndpi_main.h"
#include "ndpi_typedefs.h"
#include "ndpi_api.h"
"""
NDPI_HELPERS = """
// nDPI cffi helper functions (function naming convention ndpi_py_*)
void ndpi_py_setup_detection_module(struct ndpi_detection_module_struct *mod) {
if (mod == NULL) {
return;
} else {
NDPI_PROTOCOL_BITMASK protos;
NDPI_BITMASK_SET_ALL(protos); // Set bitmask for ALL protocols
ndpi_set_protocol_detection_bitmask2(mod, &protos);
ndpi_finalize_initialization(mod);
}
};
struct ndpi_flow_struct * ndpi_py_initialize_flow(void) {
struct ndpi_flow_struct * ndpi_flow = NULL;
ndpi_flow = (struct ndpi_flow_struct *)ndpi_flow_malloc(SIZEOF_FLOW_STRUCT);
memset(ndpi_flow, 0, SIZEOF_FLOW_STRUCT);
return ndpi_flow;
};
"""
NDPI_APIS = """
u_int16_t ndpi_get_api_version(void);
char* ndpi_revision(void);
struct ndpi_detection_module_struct *ndpi_init_detection_module(ndpi_init_prefs prefs);
void ndpi_exit_detection_module(struct ndpi_detection_module_struct *ndpi_struct);
void ndpi_flow_free(void *ptr);
ndpi_protocol ndpi_detection_process_packet(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow,
const unsigned char *packet,
const unsigned short packetlen,
const u_int64_t packet_time_ms);
ndpi_protocol ndpi_detection_giveup(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow,
u_int8_t enable_guess,
u_int8_t *protocol_was_guessed);
void ndpi_py_setup_detection_module(struct ndpi_detection_module_struct *mod);
struct ndpi_flow_struct * ndpi_py_initialize_flow(void);
char* ndpi_protocol2name(struct ndpi_detection_module_struct *ndpi_mod, ndpi_protocol proto, char *buf, u_int buf_len);
const char* ndpi_category_get_name(struct ndpi_detection_module_struct *ndpi_mod, ndpi_protocol_category_t category);
const char* ndpi_confidence_get_name(ndpi_confidence_t confidence);
"""
ffi_builder = FFI()
INCLUDE_DIR = pathlib.Path(__file__)\
.parent.resolve().parent.resolve().parent.resolve().\
joinpath("src").joinpath("include")
LIBRARY_DIR = pathlib.Path(__file__)\
.parent.resolve().parent.resolve().parent.resolve().\
joinpath("src").joinpath("lib")
NDPI_CDEF = subprocess.run(["gcc",
"-DNDPI_LIB_COMPILATION",
"-DNDPI_CFFI_PREPROCESSING",
"-DNDPI_CFFI_PREPROCESSING_EXCLUDE_PACKED",
"-E", "-x", "c", "-P", "-C",
str(INCLUDE_DIR.joinpath("ndpi_typedefs.h"))],
capture_output=True
).stdout.decode('utf-8',
errors='ignore')
NDPI_MODULE_STRUCT_CDEF = NDPI_CDEF.split("//CFFI.NDPI_MODULE_STRUCT")[1]
NDPI_PACKED = subprocess.run(["gcc",
"-DNDPI_LIB_COMPILATION", "-DNDPI_CFFI_PREPROCESSING",
"-E", "-x", "c", "-P", "-C",
str(INCLUDE_DIR.joinpath("ndpi_typedefs.h"))],
capture_output=True
).stdout.decode('utf-8',
errors='ignore')
NDPI_PACKED_STRUCTURES = NDPI_PACKED.split("//CFFI.NDPI_PACKED_STRUCTURES")[1]
NDPI_SOURCE = NDPI_INCLUDES + NDPI_MODULE_STRUCT_CDEF + NDPI_HELPERS
ffi_builder.set_source("_ndpi",
NDPI_SOURCE,
libraries=["ndpi"],
library_dirs=[str(LIBRARY_DIR)],
include_dirs=[str(INCLUDE_DIR)])
ffi_builder.cdef("""
typedef uint64_t u_int64_t;
typedef uint32_t u_int32_t;
typedef uint16_t u_int16_t;
typedef uint8_t u_int8_t;
typedef uint8_t u_char;
typedef unsigned u_int;
struct in_addr {
unsigned long s_addr;
};
struct in6_addr {
unsigned char s6_addr[16];
};
""")
ffi_builder.cdef(NDPI_PACKED_STRUCTURES, packed=True)
ffi_builder.cdef(NDPI_CDEF)
ffi_builder.cdef(NDPI_APIS)
if __name__ == "__main__":
ffi_builder.compile(verbose=True)
Executable → Regular
+144 -250
View File
@@ -1,273 +1,167 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
file: ndpi_example.py
This file is part of nDPI.
Copyright (C) 2011-19 - ntop.org
Copyright (C) 2019 - Zied Aouini <aouinizied@gmail.com> (Incremental improvements)
nDPI is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License
as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
------------------------------------------------------------------------------------------------------------------------
ndpi_example.py
Copyright (C) 2011-22 - ntop.org
This file is part of nDPI, an open source deep packet inspection library.
nDPI is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
nDPI is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with nDPI.
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
"""
from ndpi_typestruct import *
from ctypes import *
from scapy.all import *
import sys
# ------- return type & pcapstruct to declare -------
from collections import namedtuple
from ndpi import NDPI, NDPIFlow
import argparse
import socket
import dpkt
class WorkFlow(Structure):
_fields_ = [("src_ip", c_uint32),
("dst_ip", c_uint32),
("src_port", c_uint16),
("dst_port", c_uint16),
("protocol", c_uint8),
("packets", c_uint32),
("detected_protocol", NDPIProtocol),
("detection_completed", c_uint8),
("id", c_uint32),
("src_id", POINTER(NDPIIdStruct)),
("dst_id", POINTER(NDPIIdStruct)),
("flow", POINTER(NDPIFlowStruct))]
FLOW_KEY = "{} {}:{} <-> {}:{}"
FLOW_STR = " {} {} [protocol:{}] [category:{}] [confidence:{}] [{} packets/{} bytes]"
CMCFUN = CFUNCTYPE(c_int, c_void_p, c_void_p)
GUESS = CFUNCTYPE(None, c_void_p, c_int32, c_int, c_void_p)
FREE = CFUNCTYPE(None, c_void_p)
PROTOCOL_UNKNWON = 0
def node_proto_guess_walker(nodo, which, depth, user_data):
global ndpi_info_mod
class Flow(object):
__slots__ = ("index",
"pkts",
"bytes",
"detected_protocol",
"ndpi_flow")
flow = cast(nodo, POINTER(POINTER(WorkFlow))).contents.contents
if which == 0 or which == 3: # execute only preorder operation of the tree
if flow.detection_completed == 0: # order for tree operation
flow.detected_protocol = ndpi.ndpi_detection_giveup(ndpi_info_mod,
flow.flow,
1,
cast(addressof(c_uint8(0)), POINTER(c_uint8)))
count_protocol[flow.detected_protocol.app_protocol] += flow.packets
def __init__(self):
self.pkts = 0
self.detected_protocol = None
self.bytes = 0
self.ndpi_flow = None
def py_cmp_fun(a, b):
fa = cast(a, POINTER(WorkFlow))
fb = cast(b, POINTER(WorkFlow))
if fa.contents.src_ip < fb.contents.src_ip: return -1
elif fa.contents.src_ip > fb.contents.src_ip: return 1
if fa.contents.src_port < fb.contents.src_port: return -1
elif fa.contents.src_port > fb.contents.src_port: return 1
if fa.contents.dst_ip < fb.contents.dst_ip: return -1
elif fa.contents.dst_ip > fb.contents.dst_ip: return 1
if fa.contents.dst_port < fb.contents.dst_port: return -1
elif fa.contents.dst_port > fb.contents.dst_port: return 1
if fa.contents.protocol < fb.contents.protocol: return -1
elif fa.contents.protocol > fb.contents.protocol: return 1
return 0
ppacket = namedtuple('ParsedPacket', ['src_ip',
'src_port',
'dst_ip',
'dst_port',
'protocol',
'ip_version',
'ip_bytes'])
def freer(a):
pass
cmp_func = CMCFUN(py_cmp_fun)
guess_walker = GUESS(node_proto_guess_walker)
free_walk = FREE(freer)
# --------------------------------------
# number of analyzed packet
packet_number = 0
flow_count = 0
max_num_udp_dissected_pkts = 16
max_num_tcp_dissected_pkts = 10
flows_root = c_void_p(None)
flows_root_ref = pointer(flows_root)
count_protocol = (c_int32 * (ndpi.ndpi_wrap_ndpi_max_supported_protocols() + ndpi.ndpi_wrap_ndpi_max_num_custom_protocols() + 1))()
lista = [] # used to avoid impropriate memory deallocation from python
# check ndpi version
if ndpi.ndpi_get_api_version() != ndpi.ndpi_wrap_get_api_version():
print("nDPI Library version mismatch: please make sure this code and the nDPI library are in sync\n")
sys.exit(-1)
# create data structure of ndpi
ndpi_info_mod = ndpi.ndpi_init_detection_module()
if ndpi_info_mod is None:
sys.exit(-1)
else:
ndpi_ndpi_finalize_initalization(ndpi_info_mod)
def ip2int(ip):
"""
Convert an IP string to long and then c_uint32
"""
packedIP = socket.inet_aton(ip)
return int(struct.unpack("!I", packedIP)[0])
def get_flow(pkt):
global flows_root
global flows_root_ref
global flow_count
ip_packet = pkt[1]
ip_protocol = ip_packet.proto
transport_packet = None
if ip_protocol == 6 or ip_protocol == 17:
transport_packet = pkt[2]
if transport_packet is not None:
# to avoid two nodes in one binary tree for a flow
ip_src = ip2int(ip_packet.src)
ip_dst = ip2int(ip_packet.dst)
src_port = transport_packet.sport
dst_port = transport_packet.dport
else:
return None
# set flows to correct type and data
ndpi_flow = pointer(NDPIFlowStruct())
memset(ndpi_flow, 0, sizeof(NDPIFlowStruct))
if ip_src > ip_dst:
flow = WorkFlow(ip_src, ip_dst, src_port, dst_port, int(ip_packet.proto), 0, NDPIProtocol(), 0, 0,
pointer(NDPIIdStruct()), pointer(NDPIIdStruct()), ndpi_flow)
else:
flow = WorkFlow(ip_dst, ip_src, dst_port, src_port, int(ip_packet.proto), 0, NDPIProtocol(), 0, 0,
pointer(NDPIIdStruct()), pointer(NDPIIdStruct()), ndpi_flow)
flow_ref = pointer(flow)
res = ndpi.ndpi_tfind(flow_ref, flows_root_ref, cmp_func)
if res is None:
ndpi.ndpi_tsearch(flow_ref, pointer(flows_root), cmp_func) # add
lista.append(flow)
flow_count += 1
return pointer(flow)
flow = cast(res, POINTER(POINTER(WorkFlow))).contents
return flow
def packetcaptured(pkt):
global packet_number
global ndpi_info_mod
flow = None
h = PcapPktHdr()
# getting flow
def inet_to_str(inet):
""" get string representation of IP address """
try:
flow = get_flow(pkt)
except AttributeError:
pass # ignore packet
if flow is None: return
# filling pcap_pkthdr
h.len = h.caplen = len(pkt)
h.ts.tv_sec = int(pkt["IP"].time/1000000)
h.ts.tv_usec = int(pkt["IP"].time)
# real work
if int(pkt[1].frag) == 0: # not fragmented packet
flow.contents.packets += 1
packet_number += 1
# get ndpi_iphdr address
iphdr_addr = cast(c_char_p(pkt[1].build()), c_void_p)
ndpi_flow = flow.contents.flow
if flow.contents.detection_completed is 0:
flow.contents.detected_protocol = ndpi.ndpi_detection_process_packet(ndpi_info_mod,
ndpi_flow,
cast(iphdr_addr, POINTER(c_uint8)),
int(pkt[1].len),
h.ts.tv_usec,
flow.contents.src_id,
flow.contents.dst_id)
flow1 = flow.contents.detected_protocol
valid = False
if flow.contents.protocol == 6: valid = flow.contents.packets > max_num_tcp_dissected_pkts
elif flow.contents.protocol == 17: valid = flow.contents.packets > max_num_udp_dissected_pkts
# should we continue anylizing packet or not?
if valid or flow1.app_protocol is not 0:
if valid or flow1.master_protocol is not 91: # or # 91 is NDPI_PROTOCOL_TLS
flow.contents.detection_completed = 1 # protocol found
if flow1.app_protocol is 0:
flow.contents.detected_protocol = ndpi.ndpi_detection_giveup(ndpi_info_mod,
ndpi_flow,
1,
cast(addressof(c_uint8(0)),
POINTER(c_uint8)))
return socket.inet_ntop(socket.AF_INET, inet)
except ValueError:
return socket.inet_ntop(socket.AF_INET6, inet)
def result():
global flows_root_ref
global ndpi_info_mod
print('\nnumber of analyzed packet: ' + str(packet_number))
print('number of flows: ' + str(flow_count))
ndpi.ndpi_twalk(flows_root_ref.contents, guess_walker, None)
print('\nDetected protocols:')
for i in range(0, ndpi.ndpi_get_num_supported_protocols(ndpi_info_mod)):
if count_protocol[i] > 0:
print("{}: {} packets".format(
cast(ndpi.ndpi_get_proto_name(ndpi_info_mod, i), c_char_p).value.decode('utf-8'),
str(count_protocol[i])))
def free(ndpi_struct):
ndpi.ndpi_tdestroy(flows_root, free_walk)
ndpi.ndpi_exit_detection_module(ndpi_struct)
def initialize(ndpi_struct):
all = NDPIProtocolBitMask()
ndpi.ndpi_wrap_NDPI_BITMASK_SET_ALL(pointer(all))
ndpi.ndpi_set_protocol_detection_bitmask2(ndpi_struct, pointer(all))
print('Using nDPI ' + cast(ndpi.ndpi_revision(), c_char_p).value.decode("utf-8"))
initialize(ndpi_info_mod)
if len(sys.argv) != 2:
print("\nUsage: ndpi_example.py <device>")
sys.exit(0)
if "." in sys.argv[1]:
print('Reading pcap from file ' + sys.argv[1] + '...')
scapy_cap = None
def parse_packet(pkt):
""" parse packet and extract 5 tuple and IP bytes """
try:
scapy_cap = rdpcap(sys.argv[1])
except FileNotFoundError:
print("\nFile not found")
except Scapy_Exception:
print("\nBad pcap")
l2 = dpkt.ethernet.Ethernet(pkt)
if isinstance(l2.data, dpkt.ip.IP):
ip_version = 4
elif isinstance(l2.data, dpkt.ip6.IP6):
ip_version = 6
else:
return
except dpkt.dpkt.NeedData:
return
l3 = l2.data
stop_decoding = False
while not stop_decoding:
if isinstance(l3.data, dpkt.tcp.TCP):
l4 = l3.data
proto = "TCP"
stop_decoding = True
elif isinstance(l3.data, dpkt.udp.UDP):
l4 = l3.data
proto = "UDP"
stop_decoding = True
elif isinstance(l3.data, dpkt.ip6.IP6):
l3 = l3.data
else:
return
return ppacket(src_ip=inet_to_str(l3.src), src_port=l4.sport,
dst_ip=inet_to_str(l3.dst), dst_port=l4.dport,
protocol=proto, ip_version=ip_version,
ip_bytes=bytes(l3))
def ppkt_to_flow_key(ppkt):
""" create a consistent direction agnostic flow keyfrom a parsed packet """
if ppkt.src_ip < ppkt.dst_ip:
k = FLOW_KEY.format(ppkt.protocol, ppkt.src_ip, ppkt.src_port, ppkt.dst_ip, ppkt.dst_port)
else:
for packet in scapy_cap:
packetcaptured(packet)
else:
print('Capturing live traffic from device ' + sys.argv[1] + '...')
try:
sniff(iface=sys.argv[1], prn=packetcaptured)
except KeyboardInterrupt:
print('\nInterrupted\n')
except PermissionError:
sys.exit('\nRoot privilege required for live capture on interface: {}\n'.format(sys.argv[1]))
if ppkt.src_ip == ppkt.dst_ip:
if ppkt.src_port <= ppkt.dst_port:
k = FLOW_KEY.format(ppkt.protocol, ppkt.src_ip, ppkt.src_port, ppkt.dst_ip, ppkt.dst_port)
else:
k = FLOW_KEY.format(ppkt.protocol, ppkt.dst_ip, ppkt.dst_port, ppkt.src_ip, ppkt.src_port)
else:
k = FLOW_KEY.format(ppkt.protocol, ppkt.dst_ip, ppkt.dst_port, ppkt.src_ip, ppkt.src_port)
return k
result()
free(ndpi_info_mod)
def parse_arguments():
parser = argparse.ArgumentParser()
parser.add_argument("input", help="input pcap file path")
parser.add_argument('-u', '--include-unknowns', action='store_true')
return parser.parse_args()
if __name__ == "__main__":
nDPI = NDPI() # As simple as that. :)
flow_cache = {} # We store the flows in a dictionary.
flow_count = 0 # Flow counter
print("Using nDPI {}".format(nDPI.revision))
args = parse_arguments()
with open(args.input, 'rb') as pcap_file:
capture = dpkt.pcap.Reader(pcap_file) # We use dpkt pcap capture handler
for time, packet in capture:
time_ms = int(time * 1000) # Convert packet timestamp to milliseconds
ppkt = parse_packet(packet)
if ppkt is not None: # If we succeed to parse the packet
key = ppkt_to_flow_key(ppkt)
try: # Try a Flow update
flow = flow_cache[key]
flow.detected_protocol = nDPI.process_packet(flow.ndpi_flow, ppkt.ip_bytes, time_ms)
flow.pkts += 1
flow.bytes += len(packet)
except KeyError: # New Flow
flow = Flow()
flow.index = flow_count
flow_count += 1
flow.ndpi_flow = NDPIFlow() # We create an nDPIFlow object per Flow
flow.detected_protocol = nDPI.process_packet(flow.ndpi_flow, ppkt.ip_bytes, time_ms)
flow.pkts += 1
flow.bytes += len(packet)
flow_cache[key] = flow
print(" Detected flows:")
unknown_flows = []
for key, flow in flow_cache.items(): # Iterate over all flows in flow cache
if flow.detected_protocol.app_protocol == PROTOCOL_UNKNWON: # Didn't succeed to identigy it using DPI
flow.detected_protocol = nDPI.giveup(flow.ndpi_flow) # We try to guess it (port matching, LRU, etc.)
FLOW_EXPORT = FLOW_STR.format(flow.index,
key,
nDPI.protocol_name(flow.detected_protocol),
nDPI.protocol_category_name(flow.detected_protocol),
flow.ndpi_flow.confidence.name,
flow.pkts,
flow.bytes)
if flow.detected_protocol.app_protocol != PROTOCOL_UNKNWON:
print(FLOW_EXPORT) # We start by printing detected flows
else:
# Format it for later
unknown_flows.append(FLOW_EXPORT)
if args.include_unknowns:
print(" Unknown flows:")
for unknown_flow in unknown_flows: # Dump unknown flows
print(unknown_flow)
-862
View File
@@ -1,862 +0,0 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
file: ndpi_typestruct.py
This file is part of nDPI.
Copyright (C) 2011-19 - ntop.org
Copyright (C) 2019 - Zied Aouini <aouinizied@gmail.com> (Incremental improvements)
nDPI is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License
as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
nDPI is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with nDPI.
If not, see <http://www.gnu.org/licenses/>.
"""
from ctypes import CDLL, Structure, c_uint16, c_int, c_ulong, c_uint32, CFUNCTYPE, c_void_p, POINTER, c_char_p, c_uint8
from ctypes import c_char, c_uint, c_int16, c_longlong, c_size_t, Union, c_ubyte, c_uint64, c_int32, c_ushort, cast
from os.path import abspath, dirname
ndpi = CDLL(dirname(abspath(__file__)) + '/ndpi_wrap.so')
# ----------------------------------------------- Structures -----------------------------------------------------------
class NDPIDetectionModuleStruct(Structure):
pass
class NDPIFlowStruct(Structure):
pass
class NDPIProtocol(Structure):
_fields_ = [
("master_protocol", c_uint16),
("app_protocol", c_uint16),
("category", c_int)
]
class TimeVal(Structure):
_fields_ = [("tv_sec", c_ulong), ("tv_usec", c_ulong)]
class PcapPktHdr(Structure):
_fields_ = [("ts", TimeVal), ("caplen", c_uint32), ("len", c_uint32)]
class NDPIMask(Structure):
_fields_ = [("fds_bits", c_uint32)]
class NDPIProtocolBitMask(Structure):
_fields_ = [("fds_bits", NDPIMask * ndpi.ndpi_wrap_ndpi_num_fds_bits())]
class NDPISubprotocolConfStruct(Structure):
_fields_ = [("func", CFUNCTYPE(c_void_p, POINTER(NDPIDetectionModuleStruct), c_char_p, c_char_p, c_int))]
class NDPIAutoma(Structure):
_fields_ = [
("ac_automa", c_void_p),
("ac_automa_finalized", c_uint8)
]
class NDPINode(Structure):
pass
NDPINode._fields_ = [
('key', POINTER(c_char)),
('left', POINTER(NDPINode)),
('right', POINTER(NDPINode)),
]
class NDPICallFunctionStruct(Structure):
_fields_ = [
("detection_bitmask", NDPIProtocolBitMask),
("excluded_protocol_bitmask", NDPIProtocolBitMask),
("ndpi_selection_bitmask", c_uint32),
("func", CFUNCTYPE(None, POINTER(NDPIDetectionModuleStruct), POINTER(NDPIFlowStruct))),
("detection_feature", c_uint8)
]
class NDPIProtoDefaultsT(Structure):
_fields_ = [
("protoName", c_char_p),
("protoCategory", c_uint),
("subprotocols", c_uint16_p),
("subprotocol_count", c_uint32),
("protoId", c_uint16),
("protoIdx", c_uint16),
("protoBreed", c_uint),
("func", CFUNCTYPE(None, POINTER(NDPIDetectionModuleStruct), POINTER(NDPIFlowStruct))),
]
class NDPIDefaultsPortsTreeNodeT(Structure):
_fields_ = [
("proto", NDPIProtoDefaultsT),
("customUserProto", c_uint8),
("default_port", c_int16)
]
class SpinlockT(Structure):
_fields_ = [("val", c_int)]
class AtomicT(Structure):
_fields_ = [("counter", c_int)]
class TimeT(Structure):
_fields_ = [("counter", c_longlong)]
class HashIp4pNode(Structure):
pass
HashIp4pNode._fields_ = [
("next", POINTER(HashIp4pNode)),
("prev", POINTER(HashIp4pNode)),
("lchg", TimeT),
("port", c_uint16),
("count", c_uint16, 12),
("flag", c_uint16, 4),
("ip", c_uint32)
]
class HashIp4p(Structure):
_fields_ = [
("top", POINTER(HashIp4pNode)),
("lock",SpinlockT),
("len", c_size_t)
]
class HashIp4pTable(Structure):
_fields_ = [
("size", c_size_t),
("ipv6", c_int),
("lock", SpinlockT),
("count", AtomicT),
("tbl", HashIp4p)
]
class BtAnnounce(Structure):
_fields_ = [
("hash", c_uint32 * 5),
("ip", c_uint32 * 4),
("time", c_uint32),
("port", c_uint16),
("name_len", c_uint8),
("name", c_uint8 * 149)
]
class NDPILruCacheEntry(Structure):
_fields_ = [
("key", c_uint32),
("is_full", c_uint32, 1),
("value", c_uint32, 16),
("pad", c_uint32, 15)
]
class NDPILruCache(Structure):
_fields_ = [
("num_entries", c_uint32),
("entries", POINTER(NDPILruCacheEntry)),
]
class CacheEntry(Structure):
pass
CacheEntry._fields_ = [
("item", c_void_p),
("item_size", c_uint32),
("prev", POINTER(CacheEntry)),
("next", POINTER(CacheEntry))
]
class CacheEntryMap(Structure):
pass
CacheEntryMap._fields_ = [
("entry", POINTER(CacheEntry)),
("next", POINTER(CacheEntryMap)),
]
class Cache(Structure):
_fields_ = [
("size", c_uint32),
("max_size", c_uint32),
("head", POINTER(CacheEntry)),
("tail", POINTER(CacheEntry)),
("map", POINTER(POINTER(CacheEntryMap)))
]
class CustomCategories(Structure):
_fields_ = [
("hostnames", NDPIAutoma),
("hostnames_shadow", NDPIAutoma),
("ipAddresses", c_void_p),
("ipAddresses_shadow", c_void_p),
("categories_loaded", c_uint8),
]
NDPIDetectionModuleStruct._fields_ = [
("detection_bitmask", NDPIProtocolBitMask),
("generic_http_packet_bitmask", NDPIProtocolBitMask),
("current_ts", c_uint32),
("ticks_per_second", c_uint32),
("custom_category_labels",
(c_char * ndpi.ndpi_wrap_num_custom_categories()) * ndpi.ndpi_wrap_custom_category_label_len()),
("callback_buffer", NDPICallFunctionStruct * (ndpi.ndpi_wrap_ndpi_max_supported_protocols() + 1)),
("callback_buffer_size", c_uint32),
("callback_buffer_tcp_no_payload", NDPICallFunctionStruct * (ndpi.ndpi_wrap_ndpi_max_supported_protocols() + 1)),
("callback_buffer_size_tcp_no_payload", c_uint32),
("callback_buffer_tcp_payload", NDPICallFunctionStruct * (ndpi.ndpi_wrap_ndpi_max_supported_protocols() + 1)),
("callback_buffer_size_tcp_payload", c_uint32),
("callback_buffer_udp", NDPICallFunctionStruct * (ndpi.ndpi_wrap_ndpi_max_supported_protocols() + 1)),
("callback_buffer_size_udp", c_uint32),
("callback_buffer_non_tcp_udp", NDPICallFunctionStruct * (ndpi.ndpi_wrap_ndpi_max_supported_protocols() + 1)),
("callback_buffer_size_non_tcp_udp", c_uint32),
("tcpRoot", POINTER(NDPIDefaultsPortsTreeNodeT)),
("udpRoot", POINTER(NDPIDefaultsPortsTreeNodeT)),
("ndpi_log_level", c_uint),
("tcp_max_retransmission_window_size", c_uint32),
("directconnect_connection_ip_tick_timeout", c_uint32),
("subprotocol_conf", NDPISubprotocolConfStruct * (ndpi.ndpi_wrap_ndpi_max_supported_protocols() + 1)),
("ndpi_num_supported_protocols", c_uint),
("ndpi_num_custom_protocols", c_uint),
("host_automa", NDPIAutoma),
("content_automa", NDPIAutoma),
("subprotocol_automa", NDPIAutoma),
("bigrams_automa", NDPIAutoma),
("impossible_bigrams_automa", NDPIAutoma),
("custom_categories", CustomCategories),
("protocols_ptree", c_void_p),
("irc_timeout", c_uint32),
("gnutella_timeout", c_uint32),
("battlefield_timeout", c_uint32),
("thunder_timeout", c_uint32),
("soulseek_connection_ip_tick_timeout", c_uint32),
("rtsp_connection_timeout", c_uint32),
("tvants_connection_timeout", c_uint32),
("orb_rstp_ts_timeout", c_uint32),
("yahoo_detect_http_connections", c_uint8),
("yahoo_lan_video_timeout", c_uint32),
("zattoo_connection_timeout", c_uint32),
("jabber_stun_timeout", c_uint32),
("jabber_file_transfer_timeout", c_uint32),
("ip_version_limit", c_uint8),
("bt_ht", POINTER(HashIp4pTable)),
("bt6_ht", POINTER(HashIp4pTable)),
("bt_ann", POINTER(BtAnnounce)),
("bt_ann_len", c_int),
("ookla_cache", POINTER(NDPILruCache)),
("tinc_cache", POINTER(Cache)),
("proto_defaults", NDPIProtoDefaultsT * (ndpi.ndpi_wrap_ndpi_max_supported_protocols() +
ndpi.ndpi_wrap_ndpi_max_num_custom_protocols())),
("http_dont_dissect_response", c_uint8, 1),
("dns_dont_dissect_response", c_uint8, 1),
("direction_detect_disable", c_uint8, 1),
("disable_metadata_export", c_uint8, 1),
("hyperscan", c_void_p)
]
class U6Addr(Union):
_fields_ = [
("u6_addr8", c_uint8 * 16),
("u6_addr16", c_uint16 * 8),
("u6_addr32", c_uint32 * 4),
("u6_addr64", c_uint64 * 2)
]
class NDPIIn6Addr(Structure):
_pack_ = 1
_fields_ = [("u6_addr", U6Addr)]
class NDPIIpAddrT(Union):
_fields_ = [
('ipv4', c_uint32),
('ipv4_u_int8_t', c_uint8 * 4),
('ipv6', NDPIIn6Addr),
]
class NDPIIdStruct(Structure):
_fields_ = [
('detected_protocol_bitmask', NDPIProtocolBitMask),
('rtsp_ip_address', NDPIIpAddrT),
('yahoo_video_lan_timer', c_uint32),
('irc_port', c_uint16 * 8),
('last_time_port_used', c_uint32 * 8),
('irc_ts', c_uint32),
('gnutella_ts', c_uint32),
('battlefield_ts', c_uint32),
('thunder_ts', c_uint32),
('rtsp_timer', c_uint32),
('oscar_last_safe_access_time', c_uint32),
('zattoo_ts', c_uint32),
('jabber_stun_or_ft_ts', c_uint32),
('directconnect_last_safe_access_time', c_uint32),
('soulseek_last_safe_access_time', c_uint32),
('detected_directconnect_port', c_uint16),
('detected_directconnect_udp_port', c_uint16),
('detected_directconnect_ssl_port', c_uint16),
('bt_port_t', c_uint16 * 8),
('bt_port_u', c_uint16 * 8),
('jabber_voice_stun_port', c_uint16 * 6),
('jabber_file_transfer_port', c_uint16 * 2),
('detected_gnutella_port', c_uint16),
('detected_gnutella_udp_port1', c_uint16),
('detected_gnutella_udp_port2', c_uint16),
('soulseek_listen_port', c_uint16),
('irc_number_of_port', c_uint8),
('oscar_ssl_session_id', c_uint8 * 33),
('jabber_voice_stun_used_ports', c_uint8),
('yahoo_video_lan_dir', c_uint32, 1),
('yahoo_conf_logged_in', c_uint32, 1),
('yahoo_voice_conf_logged_in', c_uint32, 1),
('rtsp_ts_set', c_uint32, 1),
]
class NDPIFlowTcpStruct(Structure):
_pack_ = 1
_fields_ = [
('smtp_command_bitmask', c_uint16),
('pop_command_bitmask', c_uint16),
('qq_nxt_len', c_uint16),
('wa_matched_so_far', c_uint8),
('tds_login_version', c_uint8),
('irc_stage', c_uint8),
('irc_port', c_uint8),
('h323_valid_packets', c_uint8),
('gnutella_msg_id', c_uint8 * 3),
('irc_3a_counter', c_uint32, 3),
('irc_stage2', c_uint32, 5),
('irc_direction', c_uint32, 2),
('irc_0x1000_full', c_uint32, 1),
('soulseek_stage', c_uint32, 2),
('tds_stage', c_uint32, 3),
('usenet_stage', c_uint32, 2),
('imesh_stage', c_uint32, 4),
('http_setup_dir', c_uint32, 2),
('http_stage', c_uint32, 2),
('http_empty_line_seen', c_uint32, 1),
('http_wait_for_retransmission', c_uint32, 1),
('gnutella_stage', c_uint32, 2),
('mms_stage', c_uint32, 2),
('yahoo_sip_comm', c_uint32, 1),
('yahoo_http_proxy_stage', c_uint32, 2),
('msn_stage', c_uint32, 3),
('msn_ssl_ft', c_uint32, 2),
('ssh_stage', c_uint32, 3),
('vnc_stage', c_uint32, 2),
('telnet_stage', c_uint32, 2),
('tls_srv_cert_fingerprint_ctx', c_void_p),
('tls_seen_client_cert', c_uint8, 1),
('tls_seen_server_cert', c_uint8, 1),
('tls_seen_certificate', c_uint8, 1),
('tls_srv_cert_fingerprint_found', c_uint8, 1),
('tls_srv_cert_fingerprint_processed', c_uint8, 1),
('tls_stage', c_uint8, 2),
('tls_record_offset', c_int16),
('tls_fingerprint_len', c_int16),
('tls_sha1_certificate_fingerprint', c_uint8 * 20),
('postgres_stage', c_uint32, 3),
('ddlink_server_direction', c_uint32, 1),
('seen_syn', c_uint32, 1),
('seen_syn_ack', c_uint32, 1),
('seen_ack', c_uint32, 1),
('icecast_stage', c_uint32, 1),
('dofus_stage', c_uint32, 1),
('fiesta_stage', c_uint32, 2),
('wow_stage', c_uint32, 2),
('veoh_tv_stage', c_uint32, 2),
('shoutcast_stage', c_uint32, 2),
('rtp_special_packets_seen', c_uint32, 1),
('mail_pop_stage', c_uint32, 2),
('mail_imap_stage', c_uint32, 3),
('mail_imap_starttls', c_uint32, 2),
('skype_packet_id', c_uint8),
('citrix_packet_id', c_uint8),
('lotus_notes_packet_id', c_uint8),
('teamviewer_stage', c_uint8),
('prev_zmq_pkt_len', c_uint8),
('prev_zmq_pkt', c_char * 10),
('ppstream_stage', c_uint32, 3),
('memcached_matches', c_uint8),
('nest_log_sink_matches', c_uint8),
]
class NDPIFlowUdpStruct(Structure):
_pack_ = 1
_fields_ = [
('battlefield_msg_id', c_uint32),
('snmp_msg_id', c_uint32),
('battlefield_stage', c_uint32, 3),
('snmp_stage', c_uint32, 2),
('ppstream_stage', c_uint32, 3),
('halflife2_stage', c_uint32, 2),
('tftp_stage', c_uint32, 2),
('aimini_stage', c_uint32, 5),
('xbox_stage', c_uint32, 1),
('wsus_stage', c_uint32, 1),
('skype_packet_id', c_uint8),
('teamviewer_stage', c_uint8),
('eaq_pkt_id', c_uint8),
('eaq_sequence', c_uint32),
('rx_conn_epoch', c_uint32),
('rx_conn_id', c_uint32),
('memcached_matches', c_uint8),
('wireguard_stage', c_uint8),
('wireguard_peer_index', c_uint32 * 2),
]
class L4(Union):
_fields_ = [("tcp", NDPIFlowTcpStruct), ("udp", NDPIFlowUdpStruct)]
class Http(Structure):
_fields_ = [
("method", c_int),
("url", c_char_p),
("content_type", c_char_p),
("num_request_headers", c_uint8),
("num_response_headers", c_uint8),
("request_version", c_uint8),
("response_status_code", c_uint16),
("detected_os", c_char * 32),
]
class Dns(Structure):
_fields_ = [
("num_queries", c_uint8),
("num_answers", c_uint8),
("reply_code", c_uint8),
("is_query", c_uint8),
("query_type", c_uint16),
("query_class", c_uint16),
("rsp_type", c_uint16),
("rsp_addr", NDPIIpAddrT)
]
class Ntp(Structure):
_fields_ = [("request_code", c_uint8),
("version", c_uint8)]
class Kerberos(Structure):
_fields_ = [("cname", c_char * 24),
("realm", c_char * 24)]
class Ssl(Structure):
_fields_ = [
("ssl_version", c_uint16),
("client_certificate", c_char * 64),
("server_certificate", c_char * 64),
("server_organization", c_char * 64),
('notBefore', c_uint32),
('notAfter', c_uint32),
("ja3_client", c_char * 33),
("ja3_server", c_char * 33),
("server_cipher", c_uint16),
("server_unsafe_cipher", c_int)
]
class Stun(Structure):
_fields_ = [
("num_udp_pkts", c_uint8),
("num_processed_pkts", c_uint8),
("num_binding_requests", c_uint8),
]
class StunSsl(Structure):
_fields_ = [("ssl", Ssl), ("stun", Stun)]
class Ssh(Structure):
_fields_ = [
("client_signature", c_char * 48),
("server_signature", c_char * 48),
("hassh_client", c_char * 33),
("hassh_server", c_char * 33)
]
class Imo(Structure):
_fields_ = [
("last_one_byte_pkt", c_uint8),
("last_byte", c_uint8)
]
class Mdns(Structure):
_fields_ = [("answer", c_char * 96)]
class Ubntac2(Structure):
_fields_ = [("version", c_char * 32)]
class Http2(Structure):
_fields_ = [
("nat_ip", c_char * 24)
]
class Bittorrent(Structure):
_fields_ = [("hash", c_char * 20)]
class Dhcp(Structure):
_fields_ = [
("fingerprint", c_char * 48),
("class_ident", c_char * 48)
]
class Protos(Union):
_fields_ = [
("dns", Dns),
("kerberos", Kerberos),
("stun_ssl", StunSsl),
("ssh", Ssh),
("imo", Imo),
("mdns", Mdns),
("ubntac2", Ubntac2),
("http", Http2),
("bittorrent", Bittorrent),
("dhcp", Dhcp)
]
class TincCacheEntry(Structure):
_pack_ = 1
_fields_ = [
('src_address', c_uint32),
('dst_address', c_uint32),
('dst_port', c_uint16),
]
class NDPIIntOneLineStruct(Structure):
_fields_ = [
('ptr', POINTER(c_uint8)),
('len', c_uint16),
]
class NDPIIphdr(Structure):
_pack_ = 1
_fields_ = [
('ihl', c_uint8, 4),
('version', c_uint8, 4),
('tos', c_uint8),
('tot_len', c_uint16),
('id', c_uint16),
('frag_off', c_uint16),
('ttl', c_uint8),
('protocol', c_uint8),
('check', c_uint16),
('saddr', c_uint32),
('daddr', c_uint32)]
class NDPIIp6Hdrctl(Structure):
_pack_ = 1
_fields_ = [
('ip6_un1_flow', c_uint32),
('ip6_un1_plen', c_uint16),
('ip6_un1_nxt', c_uint8),
('ip6_un1_hlim', c_uint8),
]
class NDPIIpv6hdr(Structure):
_pack_ = 1
_fields_ = [
('ip6_hdr', NDPIIp6Hdrctl),
('ip6_src', NDPIIn6Addr),
('ip6_dst', NDPIIn6Addr),
]
class NDPITcpHdr(Structure):
_pack_ = 1
_fields_ = [
('source', c_uint16),
('dest', c_uint16),
('seq', c_uint32),
('ack_seq', c_uint32),
('res1', c_uint16, 4),
('doff', c_uint16, 4),
('fin', c_uint16, 1),
('syn', c_uint16, 1),
('rst', c_uint16, 1),
('psh', c_uint16, 1),
('ack', c_uint16, 1),
('urg', c_uint16, 1),
('ece', c_uint16, 1),
('cwr', c_uint16, 1),
('window', c_uint16),
('check', c_uint16),
('urg_ptr', c_uint16),
]
class NDPIUdpHdr(Structure):
_pack_ = 1
_fields_ = [
('source', c_uint16),
('dest', c_uint16),
('len', c_uint16),
('check', c_uint16),
]
class NDPIPacketStructStack(Structure):
_pack_ = 1
_fields_ = [
('protocol_stack_info', c_uint16)
]
class NDPIPacketStruct(Structure):
_fields_ = [
('iph', POINTER(NDPIIphdr)),
('iphv6', POINTER(NDPIIpv6hdr)),
('tcp', POINTER(NDPITcpHdr)),
('udp', POINTER(NDPIUdpHdr)),
('generic_l4_ptr', POINTER(c_uint8)),
('payload', POINTER(c_uint8)),
('tick_timestamp', c_uint32),
('tick_timestamp_l', c_uint64),
('detected_protocol_stack', c_uint16 * ndpi.ndpi_wrap_ndpi_procol_size()),
('ndpi_packet_stack', NDPIPacketStructStack),
('line', NDPIIntOneLineStruct * 64),
('host_line', NDPIIntOneLineStruct),
('forwarded_line', NDPIIntOneLineStruct),
('referer_line', NDPIIntOneLineStruct),
('content_line', NDPIIntOneLineStruct),
('accept_line', NDPIIntOneLineStruct),
('user_agent_line', NDPIIntOneLineStruct),
('http_url_name', NDPIIntOneLineStruct),
('http_encoding', NDPIIntOneLineStruct),
('http_transfer_encoding', NDPIIntOneLineStruct),
('http_contentlen', NDPIIntOneLineStruct),
('http_cookie', NDPIIntOneLineStruct),
('http_origin', NDPIIntOneLineStruct),
('http_x_session_type', NDPIIntOneLineStruct),
('server_line', NDPIIntOneLineStruct),
('http_method', NDPIIntOneLineStruct),
('http_response', NDPIIntOneLineStruct),
('http_num_headers', c_uint8),
('l3_packet_len', c_uint16),
('l4_packet_len', c_uint16),
('payload_packet_len', c_uint16),
('actual_payload_len', c_uint16),
('num_retried_bytes', c_uint16),
('parsed_lines', c_uint16),
('parsed_unix_lines', c_uint16),
('empty_line_position', c_uint16),
('tcp_retransmission', c_uint8),
('l4_protocol', c_uint8),
('ssl_certificate_detected', c_uint8, 4),
('ssl_certificate_num_checks', c_uint8, 4),
('packet_lines_parsed_complete', c_uint8, 1),
('packet_direction', c_uint8, 1),
('empty_line_position_set', c_uint8, 1),
('pad', c_uint8, 5),
]
class NDPIFlowStructStack(Structure):
_pack_ = 1
_fields_ = [
("detected_protocol_stack", c_uint16 * ndpi.ndpi_wrap_ndpi_procol_size()),
("protocol_stack_info", c_uint16)
]
NDPIFlowStruct._fields_ = [
("ndpi_flow_stack", NDPIFlowStructStack),
("guessed_protocol_id", c_uint16),
("guessed_host_protocol_id", c_uint16),
("guessed_category", c_uint16),
("guessed_header_category", c_uint16),
("l4_proto", c_uint8),
("protocol_id_already_guessed", c_uint8, 1),
("host_already_guessed", c_uint8, 1),
("init_finished", c_uint8, 1),
("setup_packet_direction", c_uint8, 1),
("packet_direction", c_uint8, 1),
("check_extra_packets", c_uint8, 1),
("next_tcp_seq_nr", c_uint32 * 2),
("max_extra_packets_to_check", c_uint8),
("num_extra_packets_checked", c_uint8),
("num_processed_pkts", c_uint8),
("extra_packets_func", CFUNCTYPE(c_int, POINTER(NDPIDetectionModuleStruct), POINTER(NDPIFlowStruct))),
("l4", L4),
("server_id", POINTER(NDPIIdStruct)),
("host_server_name", c_ubyte * 256),
("http", Http),
("protos", Protos),
("excluded_protocol_bitmask", NDPIProtocolBitMask),
("category", c_int),
('redis_s2d_first_char', c_uint8),
('redis_d2s_first_char', c_uint8),
('packet_counter', c_uint16),
('packet_direction_counter', c_uint16 * 2),
('byte_counter', c_uint16 * 2),
('bittorrent_stage', c_uint8),
('directconnect_stage', c_uint8, 2),
('sip_yahoo_voice', c_uint8, 1),
('http_detected', c_uint8, 1),
('http_upper_protocol', c_uint16),
('http_lower_protocol', c_uint16),
('rtsprdt_stage', c_uint8, 2),
('rtsp_control_flow', c_uint8, 1),
('yahoo_detection_finished', c_uint8, 2),
('zattoo_stage', c_uint8, 3),
('qq_stage', c_uint8, 3),
('thunder_stage', c_uint8, 2),
('oscar_ssl_voice_stage', c_uint8, 3),
('oscar_video_voice', c_uint8, 1),
('florensia_stage', c_uint8, 1),
('socks5_stage', c_uint8, 2),
('socks4_stage', c_uint8, 2),
('edonkey_stage', c_uint8, 2),
('ftp_control_stage', c_uint8, 2),
('rtmp_stage', c_uint8, 2),
('pando_stage', c_uint8, 3),
('steam_stage1', c_uint16, 3),
('steam_stage2', c_uint16, 2),
('steam_stage3', c_uint16, 2),
('pplive_stage1', c_uint8, 3),
('pplive_stage2', c_uint8, 2),
('pplive_stage3', c_uint8, 2),
('starcraft_udp_stage', c_uint8, 3),
('ovpn_session_id', c_uint8 * 8),
('ovpn_counter', c_uint8),
('tinc_state', c_uint8),
('TincCacheEntry', TincCacheEntry),
('csgo_strid', c_uint8 * 18),
('csgo_state', c_uint8),
('csgo_s2', c_uint8),
('csgo_id2', c_uint32),
('kxun_counter', c_uint16),
('iqiyi_counter', c_uint16),
('packet', NDPIPacketStruct),
('flow', POINTER(NDPIFlowStruct)),
('src', POINTER(NDPIIdStruct)),
('dst', POINTER(NDPIIdStruct))
]
# ----------------------------------------------- nDPI APIs ------------------------------------------------------------
""" ndpi_detection_giveup: Function to be called before we give up with detection for a given flow.
This function reduces the NDPI_UNKNOWN_PROTOCOL detection. """
ndpi.ndpi_detection_giveup.restype = NDPIProtocol
ndpi.ndpi_detection_giveup.argtypes = [POINTER(NDPIDetectionModuleStruct),
POINTER(NDPIFlowStruct), c_uint8,
POINTER(c_uint8)]
""" ndpi_detection_process_packet: Processes one packet and returns the ID of the detected protocol.
This is the MAIN PACKET PROCESSING FUNCTION. """
ndpi.ndpi_detection_process_packet.restype = NDPIProtocol
ndpi.ndpi_detection_process_packet.argtypes = [POINTER(NDPIDetectionModuleStruct),
POINTER(NDPIFlowStruct),
POINTER(c_ubyte),
c_ushort,
c_uint64,
POINTER(NDPIIdStruct),
POINTER(NDPIIdStruct)]
""" ndpi_ssl_version2str : Converts ssl version to readable string """
ndpi.ndpi_ssl_version2str.restype = c_char_p
ndpi.ndpi_ssl_version2str.argtypes = [c_int16, POINTER(c_uint8)]
""" ndpi_init_detection_module: Returns a new initialized detection module.
Note that before you can use it you can still load hosts and do other things. As soon as you are ready to use
it do not forget to call first ndpi_finalize_initalization() """
ndpi.ndpi_init_detection_module.restype = POINTER(NDPIDetectionModuleStruct)
def ndpi_ndpi_finalize_initalization(detection_module):
""" ndpi_finalize_initalization: Completes the initialization (ndpi_revision >= 3.1)"""
if cast(ndpi.ndpi_revision(), c_char_p).value.decode("utf-8")[:3] >= '3.1':
ndpi.ndpi_finalize_initalization.restype = c_void_p
ndpi.ndpi_finalize_initalization.argtypes = [POINTER(NDPIDetectionModuleStruct)]
return ndpi.ndpi_finalize_initalization(detection_module)
else:
# ignore it
return None
""" ndpi_tfind: find a node, or return 0. """
ndpi.ndpi_tfind.restype = c_void_p
""" ndpi_tsearch: ftp://ftp.cc.uoc.gr/mirrors/OpenBSD/src/lib/libc/stdlib/tsearch.c
find or insert datum into search tree. """
ndpi.ndpi_tsearch.restype = c_void_p
ndpi.ndpi_tsearch.argtypes = [c_void_p, POINTER(c_void_p), CFUNCTYPE(c_int, c_void_p, c_void_p)]
""" ndpi_revision: Get the nDPI version release. """
ndpi.ndpi_revision.restype = c_void_p
""" ndpi_get_proto_name: Get the protocol name associated to the ID."""
ndpi.ndpi_get_proto_name.restype = c_void_p
""" ndpi_category_get_name: Get protocol category as string."""
ndpi.ndpi_category_get_name.restype = c_void_p
""" ndpi_get_num_supported_protocols: Get the total number of the supported protocols."""
ndpi.ndpi_get_num_supported_protocols.restype = c_uint
""" ndpi_wrap_NDPI_BITMASK_SET_ALL: memset((char *)(p), 0xFF, sizeof(*(p)))"""
ndpi.ndpi_wrap_NDPI_BITMASK_SET_ALL.argtypes = [POINTER(NDPIProtocolBitMask)]
""" ndpi_set_protocol_detection_bitmask2: Sets the protocol bitmask2."""
ndpi.ndpi_set_protocol_detection_bitmask2.argtypes = [POINTER(NDPIDetectionModuleStruct),
POINTER(NDPIProtocolBitMask)]
""" ndpi_twalk: Walk the nodes of a tree. """
ndpi.ndpi_twalk.argtypes = [c_void_p, CFUNCTYPE(None, c_void_p, c_int32, c_int, c_void_p), c_void_p]
""" ndpi_tdestroy: node destroy. """
ndpi.ndpi_tdestroy.argtypes = [c_void_p, CFUNCTYPE(None, c_void_p)]
-60
View File
@@ -1,60 +0,0 @@
/*
* ndpi_wrap.c
*
* Copyright (C) 2011-19 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#include "sys/types.h"
#include "ndpi_config.h"
#include "ndpi_main.h"
int ndpi_wrap_get_api_version(){
return NDPI_API_VERSION;
}
int ndpi_wrap_ndpi_num_fds_bits(){
return NDPI_NUM_FDS_BITS;
}
int ndpi_wrap_num_custom_categories(){
return NUM_CUSTOM_CATEGORIES;
}
int ndpi_wrap_custom_category_label_len(){
return CUSTOM_CATEGORY_LABEL_LEN;
}
int ndpi_wrap_ndpi_max_supported_protocols(){
return NDPI_MAX_SUPPORTED_PROTOCOLS;
}
int ndpi_wrap_ndpi_max_num_custom_protocols(){
return NDPI_MAX_NUM_CUSTOM_PROTOCOLS;
}
int ndpi_wrap_ndpi_procol_size(){
return NDPI_PROTOCOL_SIZE;
}
void ndpi_wrap_NDPI_BITMASK_SET_ALL(NDPI_PROTOCOL_BITMASK* bitmask){
NDPI_ONE(bitmask);
}
void dummy() {
/* Dummy call just to cause linker to include the ndpi library */
ndpi_tfind(NULL, NULL, NULL);
}
Binary file not shown.
+1
View File
@@ -0,0 +1 @@
cffi>=1.15.0
+57
View File
@@ -0,0 +1,57 @@
"""
------------------------------------------------------------------------------------------------------------------------
setup.py
Copyright (C) 2011-22 - ntop.org
This file is part of nDPI, an open source deep packet inspection library.
nDPI is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
nDPI is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
"""
from setuptools import setup
import os
this_directory = os.path.abspath(os.path.dirname(__file__))
with open(os.path.join(this_directory, 'README.md'), encoding='utf-8') as f:
long_description = f.read()
setup(
name="ndpi",
version='4.3.0',
url='https://www.ntop.org/products/deep-packet-inspection/ndpi/',
license='LGPLv3',
description="Open and Extensible LGPLv3 Deep Packet Inspection Library",
long_description=long_description,
long_description_content_type='text/markdown',
author='Zied Aouini',
author_email='aouinizied@gmail.com',
packages=['ndpi'],
setup_requires=["cffi>=1.15.0"],
cffi_modules=["ndpi/ndpi_build.py:ffi_builder"],
install_requires=["cffi>=1.15.0"],
platforms=["Linux",
"Mac OS-X",
"Windows",
"Unix"],
classifiers=[
'Development Status :: 5 - Production/Stable',
'License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+)',
'Programming Language :: Python :: 3 :: Only',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Programming Language :: Python :: 3.10',
'Topic :: System :: Networking :: Monitoring',
],
project_urls={
'GitHub': 'https://github.com/ntop/nDPI',
}
)
+28
View File
@@ -0,0 +1,28 @@
"""
------------------------------------------------------------------------------------------------------------------------
tests.py
Copyright (C) 2011-22 - ntop.org
This file is part of nDPI, an open source deep packet inspection library.
nDPI is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
nDPI is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty
of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with NFStream.
If not, see <http://www.gnu.org/licenses/>.
------------------------------------------------------------------------------------------------------------------------
"""
from ndpi import NDPI, NDPIFlow
import time
if __name__ == '__main__':
try:
nDPI = NDPI()
ndpi_flow = NDPIFlow()
nDPI.process_packet(ndpi_flow, b'', time.time())
nDPI.giveup(ndpi_flow)
print("nDPI Python bindings: OK")
except Exception:
raise AssertionError("nDPI Python bindings: KO")
+31
View File
@@ -0,0 +1,31 @@
CC=@CC@
INC=-I ../src/include -I/usr/local/include
LIBDPI=../src/lib/libndpi.a
LDFLAGS=@LDFLAGS@
LIB=$(LIBDPI) -lm @ADDITIONAL_LIBS@ @LIBRRD@ @LIBS@
TOOLS=rrd_anomaly rrd_similarity
all: $(TOOLS)
rrd_anomaly: rrd_anomaly.c Makefile $(LIBDPI)
$(CC) $(CPPFLAGS) -g $(INC) $(LDFLAGS) rrd_anomaly.c -o rrd_anomaly $(LIB)
rrd_similarity: rrd_similarity.c Makefile $(LIBDPI)
$(CC) $(CPPFLAGS) -g $(INC) $(LDFLAGS) rrd_similarity.c -o rrd_similarity $(LIB)
clean:
/bin/rm -f *.o $(TOOLS) *~
distclean: clean
/bin/rm -f Makefile
distdir:
find . -type f -name '*.c' \
-o -name '*.txt' | xargs -I'{}' cp '{}' '$(distdir)/{}'
install:
@echo -n ""
check:
true # nothing to do here
+10
View File
@@ -0,0 +1,10 @@
This directory contains a tool that allows anomalies and similarities in RRD files to be identified
Prerequisite
- rrdtool (https://oss.oetiker.ch/rrdtool/)
If you have Ubuntu-based systems you need to do
- sudo apt-get install rrdtool librrd-dev
+184
View File
@@ -0,0 +1,184 @@
/*
* rrd_anomaly.c
*
* Copyright (C) 2011-22 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <getopt.h>
#include <rrd.h>
#include "ndpi_api.h"
#define DEFAULT_ALPHA 0.5
#define DEFAULT_RO 0.05
#define DEFAULT_START "now-1d"
#define DEFAULT_END "now"
/* *************************************************** */
static void help() {
printf("Usage: rrd_anomaly [-v][-a <alpha>][-e <end>][-q][-s <start>] -f <filename>\n"
"-a | Set alpha. Valid range >0 .. <1. Default %.2f\n"
"-e <end> | RRD end time. Default %s\n"
"-q | Quick output (only anomalies are reported)\n"
"-s <start> | RRD start time. Default %s\n"
"-f <rrd path> | Path of the RRD filename to analyze\n"
"-v | Verbose\n"
,
DEFAULT_ALPHA, DEFAULT_END, DEFAULT_START);
printf("\n\nExample: rrd_anomaly -q -f hum.rrd\n");
exit(0);
}
/* *************************************************** */
int main(int argc, char *argv[]) {
rrd_time_value_t start_tv, end_tv;
unsigned long step = 0, ds_cnt = 0;
rrd_value_t *data, *p;
char **names, *filename = NULL, *start_s, *end_s, *cf;
u_int i, j, t, first = 1, quick_mode = 0, verbose = 0;
time_t start, end;
struct ndpi_ses_struct ses;
float alpha, ro;
char c;
/* Defaults */
alpha = DEFAULT_ALPHA;
start_s = DEFAULT_START;
end_s = DEFAULT_END;
cf = "AVERAGE";
ro = DEFAULT_RO;
while((c = getopt(argc, argv, "s:e:a:qf:r:v")) != '?') {
if(c == -1) break;
switch(c) {
case 's':
start_s = optarg;
break;
case 'e':
end_s = optarg;
break;
case 'q':
quick_mode = 1;
break;
case 'a':
{
float f = atof(optarg);
if((f > 0) && (f < 1))
alpha = f;
else
printf("Discarding -a: valid range is >0 .. <1\n");
}
break;
case 'f':
filename = optarg;
break;
case 'r':
ro = atof(optarg);
if((ro <= 0) || (ro >= 1))
ro = DEFAULT_RO;
break;
case 'v':
verbose = 1;
break;
default:
help();
break;
}
}
if(filename == NULL)
help();
ndpi_ses_init(&ses, alpha, ro);
if((rrd_parsetime(start_s, &start_tv) != NULL)) {
printf("Unable to parse start time %s\n", start_s);
return(-1);
}
if((rrd_parsetime(end_s, &end_tv) != NULL)) {
printf("Unable to parse end time %s\n", end_s);
return(-1);
}
rrd_proc_start_end(&start_tv, &end_tv, &start, &end);
if(rrd_fetch_r(filename, cf, &start, &end, &step, &ds_cnt, &names, &data) != 0) {
printf("Unable to extract data from rrd %s\n", filename);
return(-2);
}
p = data;
for(t=start+1, i=0; t<end; t+=step, i++) {
j = 0; /* Consider only the first DS */
/* for(j=0; j<ds_cnt; j++) */ {
rrd_value_t value = *p++;
if(!isnan(value)) {
double prediction, confidence_band;
double lower, upper;
char buf[32];
int rc;
u_int is_anomaly;
value *= 100; /* trick to avoid dealing with floats */
rc = ndpi_ses_add_value(&ses, value, &prediction, &confidence_band);
lower = prediction - confidence_band, upper = prediction + confidence_band;
is_anomaly = ((rc == 0) || (confidence_band == 0) || ((value >= lower) && (value <= upper))) ? 0 : 1;
if(verbose || is_anomaly) {
if(quick_mode) {
printf("%u\n", t);
} else {
const time_t _t = t;
struct tm *t_info = localtime((const time_t*)&_t);
strftime(buf, sizeof(buf), "%d/%b/%Y %H:%M:%S", t_info);
if(first) {
first = 0;
printf("%s %s\t%s %s %s\t %s [%s]\n",
"When", "Value", "Prediction", "Lower", "Upper", "Out", "Band");
}
printf("%s %12.3f\t%.3f\t%12.3f\t%12.3f\t %s [%.3f]\n",
buf, value/100., prediction/100., lower/100., upper/100., is_anomaly? "ANOMALY" : "OK",
confidence_band/100.);
}
}
}
}
}
rrd_freemem(data);
return(0);
}
+319
View File
@@ -0,0 +1,319 @@
/*
* rrd_similarity.c
*
* Copyright (C) 2011-22 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* nDPI is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with nDPI. If not, see <http://www.gnu.org/licenses/>.
*
*/
#include <stdio.h>
#include <stdlib.h>
#include <math.h>
#include <getopt.h>
#include <dirent.h>
#include <sys/stat.h>
#include "rrd.h"
#include "ndpi_api.h"
#define DEFAULT_ALPHA 0.5
#define DEFAULT_START "now-1d"
#define DEFAULT_END "now"
#define MAX_NUM_RRDS 8192
#ifndef PATH_MAX
#define PATH_MAX 4096
#endif
typedef struct {
char *path;
float average, stddev;
struct ndpi_bin b;
} rrd_file_stats;
u_int verbose = 0, similarity_threshold = 100, skip_zero = 0;
/* *************************************************** */
static void help() {
printf("Usage: rrd_similarity [-v][-a <alpha>][-e <end>][-q][-s <start>]\n"
" -f <filename> -d <basedir> [-t <threshold>]\n"
"-a | Set alpha. Valid range >0 .. <1. Default %.2f\n"
"-e <end> | RRD end time. Default %s\n"
"-q | Quick output (only anomalies are reported)\n"
"-s <start> | RRD start time. Default %s\n"
"-d <basedir> | Base directory where RRD filename is searched\n"
"-f <rrd path> | Path of the RRD filename to analyze\n"
"-t <threshold> | Similarity threshold. Default %u (0 == alike)\n"
"-v | Verbose\n"
"-z | Skip zero RRDs during comparison\n"
,
DEFAULT_ALPHA, DEFAULT_END, DEFAULT_START, similarity_threshold);
printf("\n\nExample: rrd_similarity -q -f bytes.rrd -d /var/lib/ntopng/-1/snmpstats\n");
printf("\n\nGoal: find similar RRDs\n");
exit(0);
}
/* *************************************************** */
void analyze_rrd(rrd_file_stats *rrd, time_t start, time_t end) {
unsigned long step = 0, ds_cnt = 0;
rrd_value_t *data, *p;
char **names;
u_int t, i, j, num_points;
struct ndpi_analyze_struct *s;
if(rrd_fetch_r(rrd->path, "AVERAGE", &start, &end, &step, &ds_cnt, &names, &data) != 0) {
printf("Unable to extract data from rrd %s\n", rrd->path);
return;
}
p = data;
num_points = (end-start)/step;
if((s = ndpi_alloc_data_analysis(num_points)) == NULL)
return;
ndpi_init_bin(&rrd->b, ndpi_bin_family32, num_points);
/* Step 1 - Compute average and stddev */
for(t=start+1, i=0; t<end; t+=step, i++) {
double value = (double)*p++;
if(isnan(value)) value = 0;
ndpi_data_add_value(s, value);
ndpi_set_bin(&rrd->b, i, value);
}
rrd->average = ndpi_data_average(s);
rrd->stddev = ndpi_data_stddev(s);
/* Step 2 - Bin analysis */
ndpi_free_data_analysis(s, 1);
rrd_freemem(data);
}
/* *************************************************** */
int circles_touch(int x1, int r1, int x2, int r2) {
int radius_sum = r1+r2;
int x_diff = abs(x1 - x2);
return((radius_sum < x_diff) ? 0 : 1);
}
/* *************************************************** */
void find_rrd_similarities(rrd_file_stats *rrd, u_int num_rrds) {
u_int i, j, num_similar_rrds = 0, num_potentially_zero_equal = 0;
for(i=0; i<num_rrds; i++) {
for(j=i+1; j<num_rrds; j++) {
/*
Average is the circle center, and stddev is the radius
if circles touch each other then there is a chance that
the two rrds are similar
*/
if((rrd[i].average == 0) && (rrd[i].average == rrd[j].average)) {
if(!skip_zero)
printf("%s [%.1f/%.1f] - %s [%.1f/%.1f] are alike\n",
rrd[i].path, rrd[i].average, rrd[i].stddev,
rrd[j].path, rrd[j].average, rrd[j].stddev);
num_potentially_zero_equal++;
} else if(circles_touch(rrd[i].average, rrd[i].stddev, rrd[j].average, rrd[j].stddev)
) {
float similarity = ndpi_bin_similarity(&rrd[i].b, &rrd[j].b, 0, similarity_threshold);
if((similarity >= 0) && (similarity < similarity_threshold)) {
if(verbose)
printf("%s [%.1f/%.1f] - %s [%.1f/%.1f] are %s [%.1f]\n",
rrd[i].path, rrd[i].average, rrd[i].stddev,
rrd[j].path, rrd[j].average, rrd[j].stddev,
(similarity == 0) ? "alike" : "similar",
similarity
);
num_similar_rrds++;
}
}
}
}
printf("Found %u (%.3f %%) similar RRDs / %u zero alike RRDs [num_rrds: %u]\n",
num_similar_rrds,
(num_similar_rrds*100.)/(float)(num_rrds*num_rrds),
num_potentially_zero_equal,
num_rrds);
}
/* *************************************************** */
void find_rrds(char *basedir, char *filename, rrd_file_stats *rrds, u_int *num_rrds) {
struct dirent **namelist;
int n = scandir(basedir, &namelist, 0, NULL);
if(n < 0)
return; /* End of the tree */
while(n--) {
if(namelist[n]->d_name[0] != '.') {
char path[PATH_MAX];
struct stat s;
ndpi_snprintf(path, sizeof(path), "%s/%s", basedir, namelist[n]->d_name);
if(stat(path, &s) == 0) {
if(S_ISDIR(s.st_mode))
find_rrds(path, filename, rrds, num_rrds);
else if(strcmp(namelist[n]->d_name, filename) == 0) {
if(*num_rrds < MAX_NUM_RRDS) {
rrds[*num_rrds].path = strdup(path);
if(rrds[*num_rrds].path != NULL)
(*num_rrds)++;
}
}
}
}
free(namelist[n]);
}
free(namelist);
}
/* *************************************************** */
int main(int argc, char *argv[]) {
rrd_time_value_t start_tv, end_tv;
char *filename = NULL, *start_s, *end_s, *dirname = NULL, *basedir = NULL;
u_int first = 1, quick_mode = 0;
float alpha;
char c;
time_t start, end;
u_int num_rrds = 0, i;
rrd_file_stats *rrds;
/* Defaults */
alpha = DEFAULT_ALPHA;
start_s = DEFAULT_START;
end_s = DEFAULT_END;
while((c = getopt(argc, argv, "d:s:e:a:qf:t:vz")) != '?') {
if(c == -1) break;
switch(c) {
case 's':
start_s = optarg;
break;
case 'd':
basedir = optarg;
break;
case 'e':
end_s = optarg;
break;
case 'q':
quick_mode = 1;
break;
case 'v':
verbose = 1;
break;
case 'a':
{
float f = atof(optarg);
if((f > 0) && (f < 1))
alpha = f;
else
printf("Discarding -a: valid range is >0 .. <1\n");
}
break;
case 'f':
filename = optarg;
break;
case 't':
similarity_threshold = atoi(optarg);
break;
case 'z':
skip_zero = 1;
break;
default:
help();
break;
}
}
if((filename == NULL) || (basedir == NULL))
help();
if((rrd_parsetime(start_s, &start_tv) != NULL)) {
printf("Unable to parse start time %s\n", start_s);
return(-1);
}
if((rrd_parsetime(end_s, &end_tv) != NULL)) {
printf("Unable to parse end time %s\n", end_s);
return(-1);
}
rrd_proc_start_end(&start_tv, &end_tv, &start, &end);
if((rrds = ndpi_calloc(sizeof(rrd_file_stats), MAX_NUM_RRDS)) == NULL) {
printf("Not enough memory !\n");
return(-1);
}
/* Find all rrd's */
find_rrds(basedir, filename, rrds, &num_rrds);
/* Read RRD's data */
for(i=0; i<num_rrds; i++)
analyze_rrd(&rrds[i], start, end);
find_rrd_similarities(rrds, num_rrds);
#if 0
if(verbose) {
for(i=0; i<num_rrds; i++)
printf("%s\t%.1f\t%.1f\n", rrds[i].path, rrds[i].average, rrds[i].stddev);
}
#endif
for(i=0; i<num_rrds; i++) {
ndpi_free_bin(&rrds[i].b);
free(rrds[i].path);
}
ndpi_free(rrds);
return(0);
}
+239 -61
View File
@@ -36,7 +36,6 @@ extern "C" {
*/
#define NDPI_API_VERSION @NDPI_API_VERSION@
#define SIZEOF_ID_STRUCT ( sizeof(struct ndpi_id_struct) )
#define SIZEOF_FLOW_STRUCT ( sizeof(struct ndpi_flow_struct) )
#define NDPI_DETECTION_ONLY_IPV4 ( 1 << 0 )
@@ -75,15 +74,6 @@ extern "C" {
u_int32_t ndpi_detection_get_sizeof_ndpi_flow_struct(void);
/**
* Get the size of the id struct
*
* @return the size of the id struct
*
*/
u_int32_t ndpi_detection_get_sizeof_ndpi_id_struct(void);
/**
* Get the size of the flow tcp struct
*
@@ -108,6 +98,15 @@ extern "C" {
int ndpi_match_string_value(void *_automa, char *string_to_match,
u_int match_len, u_int32_t *num);
/**
* Return the protocol error code of a given flow
*
* @par flow = the flow to analyze
* @return the error code or 0 otherwise
*
*/
u_int32_t ndpi_get_flow_error_code(struct ndpi_flow_struct *flow);
/**
* nDPI personal allocation and free functions
**/
@@ -118,6 +117,7 @@ extern "C" {
void ndpi_free(void *ptr);
void * ndpi_flow_malloc(size_t size);
void ndpi_flow_free(void *ptr);
u_int32_t ndpi_get_tot_allocated_memory(void);
/**
* Search the first occurrence of substring -find- in -s-
@@ -302,8 +302,6 @@ extern "C" {
* @par packet = unsigned char pointer to the Layer 3 (IP header)
* @par packetlen = the length of the packet
* @par packet_time_ms = the current timestamp for the packet (expressed in msec)
* @par src = pointer to the source subscriber state machine
* @par dst = pointer to the destination subscriber state machine
* @return void
*
*/
@@ -311,9 +309,7 @@ extern "C" {
struct ndpi_flow_struct *flow,
const unsigned char *packet,
const unsigned short packetlen,
const u_int64_t packet_time_ms,
struct ndpi_id_struct *src,
struct ndpi_id_struct *dst);
const u_int64_t packet_time_ms);
/**
* Processes one packet and returns the ID of the detected protocol.
@@ -324,8 +320,6 @@ extern "C" {
* @par packet = unsigned char pointer to the Layer 3 (IP header)
* @par packetlen = the length of the packet
* @par packet_time_ms = the current timestamp for the packet (expressed in msec)
* @par src = pointer to the source subscriber state machine
* @par dst = pointer to the destination subscriber state machine
* @return the detected ID of the protocol
*
*/
@@ -333,9 +327,7 @@ extern "C" {
struct ndpi_flow_struct *flow,
const unsigned char *packet,
const unsigned short packetlen,
const u_int64_t packet_time_ms,
struct ndpi_id_struct *src,
struct ndpi_id_struct *dst);
const u_int64_t packet_time_ms);
/**
* Get the main protocol of the passed flows for the detected module
*
@@ -348,6 +340,40 @@ extern "C" {
u_int16_t ndpi_get_flow_masterprotocol(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow);
/**
* Get the app protocol of the passed flows for the detected module
*
*
* @par ndpi_struct = the detection module
* @par flow = the flow given for the detection module
* @return the ID of the app protocol detected
*
*/
u_int16_t ndpi_get_flow_appprotocol(struct ndpi_detection_module_struct *ndpi_str, struct ndpi_flow_struct *flow);
/**
* Get the category of the passed flows for the detected module
*
*
* @par ndpi_struct = the detection module
* @par flow = the flow given for the detection module
* @return the ID of the category
*
*/
ndpi_protocol_category_t ndpi_get_flow_category(struct ndpi_detection_module_struct *ndpi_str, struct ndpi_flow_struct *flow);
/**
* Get the ndpi protocol data of the passed flows for the detected module
*
*
* @par ndpi_struct = the detection module
* @par flow = the flow given for the detection module
* @par ndpi_proto = the output struct where to store the requested information
*
*/
void ndpi_get_flow_ndpi_proto(struct ndpi_detection_module_struct *ndpi_str, struct ndpi_flow_struct *flow,
struct ndpi_proto * ndpi_proto);
/**
* API call that is called internally by ndpi_detection_process_packet or by apps
* that want to avoid calling ndpi_detection_process_packet as they have already
@@ -424,7 +450,6 @@ extern "C" {
* @par string_to_match = the string to match
* @par string_to_match_len = the length of the string
* @par ret_match = completed returned match information
* @par is_host_match = value of the second field of struct ndpi_automa
* @return the ID of the matched subprotocol;
* -1 if automa is not finalized;
* -2 if automa==NULL or string_to_match==NULL or empty string_to_match
@@ -433,8 +458,7 @@ extern "C" {
int ndpi_match_string_subprotocol(struct ndpi_detection_module_struct *ndpi_struct,
char *string_to_match,
u_int string_to_match_len,
ndpi_protocol_match_result *ret_match,
u_int8_t is_host_match);
ndpi_protocol_match_result *ret_match);
/**
* Check if the host passed match with a protocol
*
@@ -464,24 +488,6 @@ extern "C" {
void ndpi_check_subprotocol_risk(struct ndpi_detection_module_struct *ndpi_str,
struct ndpi_flow_struct *flow, u_int16_t subprotocol_id);
/**
* Check if the string content passed match with a protocol
*
* @par ndpi_struct = the detection module
* @par flow = the flow where match the host
* @par string_to_match = the string to match
* @par string_to_match_len = the length of the string
* @par ret_match = completed returned match information
* @par master_protocol_id = value of the ID associated to the master protocol detected
* @return the ID of the matched subprotocol
*
*/
u_int16_t ndpi_match_content_subprotocol(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow,
char *string_to_match,
u_int string_to_match_len,
ndpi_protocol_match_result *ret_match,
u_int16_t master_protocol_id);
/**
* Exclude protocol from search
*
@@ -601,6 +607,15 @@ extern "C" {
const char* ndpi_category_get_name(struct ndpi_detection_module_struct *ndpi_mod,
ndpi_protocol_category_t category);
/**
* Get classification confidence as string
*
* @par confidence = the confidence value
* @return the string name of the confidence result
*
*/
const char* ndpi_confidence_get_name(ndpi_confidence_t confidence);
/**
* Set protocol category string
*
@@ -682,12 +697,19 @@ extern "C" {
*/
void ndpi_dump_protocols(struct ndpi_detection_module_struct *mod);
/**
* Generate Options list used in OPNsense firewall plugin
*
* @par opt = The Option list to generate
*/
void ndpi_generate_options(u_int opt);
/**
* Write the list of the scores and their associated risks
*
* @par ndpi_mod = the detection module
*/
void ndpi_dump_risks_score();
void ndpi_dump_risks_score(void);
/**
* Read a file and load the protocols
@@ -729,15 +751,25 @@ extern "C" {
*/
int ndpi_add_host_risk_mask(struct ndpi_detection_module_struct *ndpi_mod, char *host, ndpi_risk mask);
/**
* Add a trusted certificate issuer DN
*
* @par ndpi_mod = the detection module
* @par dn = the issuer DN as it appears in the certificate (example "CN=813845657003339838, O=Code42, OU=TEST, ST=MN, C=US")
* @return 0 if the rule is loaded correctly; < 0 in case an error is detected
*/
int ndpi_add_trusted_issuer_dn(struct ndpi_detection_module_struct *ndpi_mod, char *dn);
/**
* Read a file and load the categories
*
* @par ndpi_mod = the detection module
* @par path = the path of the file
* @par user_data = pointer to some user data value
* @return 0 if the file is loaded correctly;
* -1 else
*/
int ndpi_load_categories_file(struct ndpi_detection_module_struct *ndpi_str, const char* path);
int ndpi_load_categories_file(struct ndpi_detection_module_struct *ndpi_str, const char* path, void *user_data);
/**
* Read a file and load the list of risky domains
@@ -900,12 +932,16 @@ extern "C" {
int ndpi_match_string(void *_automa, char *string_to_match);
int ndpi_load_ip_category(struct ndpi_detection_module_struct *ndpi_struct,
const char *ip_address_and_mask, ndpi_protocol_category_t category);
const char *ip_address_and_mask, ndpi_protocol_category_t category,
void *user_data);
int ndpi_load_hostname_category(struct ndpi_detection_module_struct *ndpi_struct,
const char *name_to_add, ndpi_protocol_category_t category);
int ndpi_load_category(struct ndpi_detection_module_struct *ndpi_struct,
const char *ip_or_name, ndpi_protocol_category_t category);
const char *ip_or_name, ndpi_protocol_category_t category,
void *user_data);
int ndpi_enable_loaded_categories(struct ndpi_detection_module_struct *ndpi_struct);
void* ndpi_find_ipv4_category_userdata(struct ndpi_detection_module_struct *ndpi_str,
u_int32_t saddr);
int ndpi_fill_ip_protocol_category(struct ndpi_detection_module_struct *ndpi_struct,
u_int32_t saddr,
u_int32_t daddr,
@@ -958,6 +994,18 @@ extern "C" {
ndpi_protocol_category_t *category,
ndpi_protocol_breed_t *breed);
/**
* Specifies the threshold used to trigger the NDPI_TLS_CERTIFICATE_ABOUT_TO_EXPIRE
* flow risk that by default is set to 30 days
*
* @par ndpi_struct = the struct created for the protocol detection
* @par days = the number of days threshold for emitting the alert
*
*/
void ndpi_set_tls_cert_expire_days(struct ndpi_detection_module_struct *ndpi_str,
u_int8_t days);
/* Utility functions to set ndpi malloc/free/print wrappers */
void set_ndpi_malloc(void* (*__ndpi_malloc)(size_t size));
void set_ndpi_free(void (*__ndpi_free)(void *ptr));
@@ -988,7 +1036,7 @@ extern "C" {
/* Return a flow info string (summarized). Does only work for DNS/HTTP/TLS/QUIC. */
const char* ndpi_get_flow_info(struct ndpi_flow_struct const * const flow,
ndpi_protocol const * const l7_protocol);
char* ndpi_ssl_version2str(struct ndpi_flow_struct *flow,
char* ndpi_ssl_version2str(char *buf, int buf_len,
u_int16_t version, u_int8_t *unknown_tls_version);
int ndpi_netbios_name_interpret(u_char *in, u_int in_len, u_char *out, u_int out_len);
void ndpi_patchIPv6Address(char *str);
@@ -1007,7 +1055,7 @@ extern "C" {
int ndpi_flow2json(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow,
u_int8_t ip_version,
u_int8_t l4_protocol, u_int16_t vlan_id,
u_int8_t l4_protocol,
u_int32_t src_v4, u_int32_t dst_v4,
struct ndpi_in6_addr *src_v6, struct ndpi_in6_addr *dst_v6,
u_int16_t src_port, u_int16_t dst_port,
@@ -1054,6 +1102,8 @@ extern "C" {
/* General purpose utilities */
u_int64_t ndpi_htonll(u_int64_t v);
u_int64_t ndpi_ntohll(u_int64_t v);
u_int8_t ndpi_is_valid_protoId(u_int16_t protoId);
u_int8_t ndpi_is_encrypted_proto(struct ndpi_detection_module_struct *ndpi_str, ndpi_protocol proto);
/* DGA */
int ndpi_check_dga_name(struct ndpi_detection_module_struct *ndpi_str,
@@ -1091,6 +1141,12 @@ extern "C" {
*/
void ndpi_reset_serializer(ndpi_serializer *serializer);
/**
* Hint to not create the header (used to avoid creaign the header when not used)
* @param serializer The serializer handle
*/
void ndpi_serializer_skip_header(ndpi_serializer *serializer);
/**
* Serialize a 32-bit unsigned int key and a 32-bit unsigned int value
* @param serializer The serializer handle
@@ -1147,7 +1203,7 @@ extern "C" {
int ndpi_serialize_uint32_string(ndpi_serializer *serializer, u_int32_t key, const char *value);
/**
* Serialize a 32-bit unsigned int key and a boolean value
* Serialize a 32-bit unsigned int key and a boolean value (JSON/CSV only, not supported by TLV)
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
@@ -1261,6 +1317,17 @@ extern "C" {
*/
int ndpi_serialize_string_string(ndpi_serializer *serializer, const char *key, const char *value);
/**
* Serialize a string key and a string value
* @param serializer The serializer handle
* @param key The field name or ID
* @param value The field value
* @param value_len The field value length
* @return 0 on success, a negative number otherwise
*/
int ndpi_serialize_string_string_len(ndpi_serializer *serializer, const char *key,
const char *value, u_int16_t value_len);
/**
* Serialize a string key and an unterminated string value
* @param serializer The serializer handle
@@ -1382,6 +1449,14 @@ extern "C" {
*/
int ndpi_serialize_start_of_block(ndpi_serializer *serializer, const char *key);
/**
* Serialize the start of a block with a numeric key
* @param serializer The serializer handle
* @param key The numeric key as 32-bit unsigned integer.
* @return 0 on success, a negative number otherwise
*/
int ndpi_serialize_start_of_block_uint32(ndpi_serializer *serializer, u_int32_t key);
/**
* Serialize the end of a block
* @param serializer The serializer handle
@@ -1480,6 +1555,13 @@ extern "C" {
int ndpi_deserialize_clone_item(ndpi_deserializer *deserializer, ndpi_serializer *serializer);
int ndpi_deserialize_clone_all(ndpi_deserializer *deserializer, ndpi_serializer *serializer);
/*
* Escape a string to be suitable for a JSON value, adding double quotes, and terminating the string with a null byte.
* It is recommended to provide a destination buffer (dst) which is as large as double the source buffer (src) at least.
* Upon successful return, these functions return the number of characters printed (excluding the null byte used to terminate the string).
*/
int ndpi_json_string_escape(const char *src, int src_len, char *dst, int dst_max_len);
/* Data analysis */
struct ndpi_analyze_struct* ndpi_alloc_data_analysis(u_int16_t _max_series_len);
void ndpi_init_data_analysis(struct ndpi_analyze_struct *s, u_int16_t _max_series_len);
@@ -1497,6 +1579,7 @@ extern "C" {
float ndpi_data_entropy(struct ndpi_analyze_struct *s);
float ndpi_data_variance(struct ndpi_analyze_struct *s);
float ndpi_data_stddev(struct ndpi_analyze_struct *s);
float ndpi_data_mean(struct ndpi_analyze_struct *s);
u_int32_t ndpi_data_last(struct ndpi_analyze_struct *s);
u_int32_t ndpi_data_min(struct ndpi_analyze_struct *s);
u_int32_t ndpi_data_max(struct ndpi_analyze_struct *s);
@@ -1519,11 +1602,13 @@ extern "C" {
int ndpi_ses_init(struct ndpi_ses_struct *ses, double alpha, float significance);
int ndpi_ses_add_value(struct ndpi_ses_struct *ses, const u_int64_t _value, double *forecast, double *confidence_band);
void ndpi_ses_fitting(double *values, u_int32_t num_values, float *ret_alpha);
/* ******************************* */
int ndpi_des_init(struct ndpi_des_struct *des, double alpha, double beta, float significance);
int ndpi_des_add_value(struct ndpi_des_struct *des, const u_int64_t _value, double *forecast, double *confidence_band);
void ndpi_des_fitting(double *values, u_int32_t num_values, float *ret_alpha, float *ret_beta);
/* ******************************* */
@@ -1541,14 +1626,26 @@ extern "C" {
u_int8_t ndpi_is_protocol_detected(struct ndpi_detection_module_struct *ndpi_str,
ndpi_protocol proto);
void ndpi_serialize_risk(ndpi_serializer *serializer, struct ndpi_flow_struct *flow);
void ndpi_serialize_risk(ndpi_serializer *serializer, ndpi_risk risk);
void ndpi_serialize_risk_score(ndpi_serializer *serializer, ndpi_risk_enum risk);
void ndpi_serialize_confidence(ndpi_serializer *serializer, ndpi_confidence_t confidence);
void ndpi_serialize_proto(struct ndpi_detection_module_struct *ndpi_struct,
ndpi_serializer *serializer,
ndpi_risk_enum risk,
ndpi_confidence_t confidence,
ndpi_protocol l7_protocol);
const char* ndpi_risk2str(ndpi_risk_enum risk);
const char* ndpi_severity2str(ndpi_risk_severity s);
ndpi_risk_info* ndpi_risk2severity(ndpi_risk_enum risk);
u_int16_t ndpi_risk2score(ndpi_risk risk,
u_int16_t *client_score, u_int16_t *server_score);
u_int8_t ndpi_check_issuerdn_risk_exception(struct ndpi_detection_module_struct *ndpi_str,
char *issuerDN);
u_int8_t ndpi_check_flow_risk_exceptions(struct ndpi_detection_module_struct *ndpi_str,
u_int num_params,
ndpi_risk_params params[]);
/* ******************************* */
/* HyperLogLog cardinality estimator */
@@ -1567,30 +1664,87 @@ extern "C" {
/* ******************************* */
int ndpi_init_bin(struct ndpi_bin *b, enum ndpi_bin_family f, u_int8_t num_bins);
int ndpi_init_bin(struct ndpi_bin *b, enum ndpi_bin_family f, u_int16_t num_bins);
void ndpi_free_bin(struct ndpi_bin *b);
struct ndpi_bin* ndpi_clone_bin(struct ndpi_bin *b);
void ndpi_inc_bin(struct ndpi_bin *b, u_int8_t slot_id, u_int32_t val);
void ndpi_set_bin(struct ndpi_bin *b, u_int8_t slot_id, u_int32_t value);
u_int32_t ndpi_get_bin_value(struct ndpi_bin *b, u_int8_t slot_id);
void ndpi_inc_bin(struct ndpi_bin *b, u_int16_t slot_id, u_int64_t val);
void ndpi_set_bin(struct ndpi_bin *b, u_int16_t slot_id, u_int64_t value);
u_int64_t ndpi_get_bin_value(struct ndpi_bin *b, u_int16_t slot_id);
void ndpi_reset_bin(struct ndpi_bin *b);
void ndpi_normalize_bin(struct ndpi_bin *b);
char* ndpi_print_bin(struct ndpi_bin *b, u_int8_t normalize_first, char *out_buf, u_int out_buf_len);
float ndpi_bin_similarity(struct ndpi_bin *b1, struct ndpi_bin *b2, u_int8_t normalize_first);
float ndpi_bin_similarity(struct ndpi_bin *b1, struct ndpi_bin *b2,
u_int8_t normalize_first, float similarity_max_threshold);
int ndpi_cluster_bins(struct ndpi_bin *bins, u_int16_t num_bins,
u_int8_t num_clusters, u_int16_t *cluster_ids,
struct ndpi_bin *centroids);
/* ******************************* */
/*
* Finds outliers using Z-score
* Z-Score = (Value - Mean) / StdDev
*
* @par values = pointer to the individual values to be analyzed [in]
* @par outliers = pointer to a list of outliers identified [out]
* @par num_values = lenght of values and outliers that MUST have the same lenght [in]
*
* @return The number of outliers found
*/
u_int ndpi_find_outliers(u_int32_t *values, bool *outliers, u_int32_t num_values);
/* ******************************* */
u_int32_t ndpi_quick_16_byte_hash(u_int8_t *in_16_bytes_long);
/* ******************************* */
ndpi_str_hash* ndpi_hash_alloc(u_int32_t max_num_entries);
void ndpi_hash_free(ndpi_str_hash *h);
int ndpi_hash_find_entry(ndpi_str_hash *h, char *key, u_int key_len, u_int8_t *value);
int ndpi_hash_add_entry(ndpi_str_hash *h, char *key, u_int8_t key_len, u_int8_t value);
/**
* Initialize the hashmap.
*
* @par h = pointer to the hash map [in, out]
*
* @return 0 on success, 1 otherwise
*
*/
int ndpi_hash_init(ndpi_str_hash **h);
/**
* Free the hashmap.
*
* @par h = pointer to the hash map [in, out]
* @par cleanup_func = pointer to a optional callback function
* called for each element in the hashmap [in]
*
*/
void ndpi_hash_free(ndpi_str_hash **h, void (*cleanup_func)(ndpi_str_hash *h));
/**
* Search for an entry in the hashmap.
*
* @par h = pointer to the hash map [in]
* @par key = character string (no '\0' required) [in]
* @par key_len = length of the character string @key [in]
* @par value = pointer to a pointer to the value, which contains a
* previously added hash entry [in, out]
*
* @return 0 if an entry with that key was found, 1 otherwise
*
*/
int ndpi_hash_find_entry(ndpi_str_hash *h, char *key, u_int key_len, void **value);
/**
* Add an entry to the hashmap.
*
* @par h = pointer to the hash map [in, out]
* @par key = character string (no '\0' required) [in]
* @par key_len = length of the character string @key [in]
* @par value = pointer to the value to add [in]
*
* @return 0 if the entry was added, 1 otherwise
*
*/
int ndpi_hash_add_entry(ndpi_str_hash **h, char *key, u_int8_t key_len, void *value);
/* ******************************* */
@@ -1606,9 +1760,33 @@ extern "C" {
/* ******************************* */
char* ndpi_get_flow_name(struct ndpi_flow_struct *flow);
/* ******************************* */
ndpi_bitmap* ndpi_bitmap_alloc(void);
void ndpi_bitmap_free(ndpi_bitmap* b);
u_int64_t ndpi_bitmap_cardinality(ndpi_bitmap* b);
void ndpi_bitmap_set(ndpi_bitmap* b, u_int32_t value);
void ndpi_bitmap_unset(ndpi_bitmap* b, u_int32_t value);
bool ndpi_bitmap_isset(ndpi_bitmap* b, u_int32_t value);
void ndpi_bitmap_clear(ndpi_bitmap* b);
size_t ndpi_bitmap_serialize(ndpi_bitmap* b, char **buf);
ndpi_bitmap* ndpi_bitmap_deserialize(char *buf);
void ndpi_bitmap_and(ndpi_bitmap* a, ndpi_bitmap* b_and);
void ndpi_bitmap_or(ndpi_bitmap* a, ndpi_bitmap* b_or);
ndpi_bitmap_iterator* ndpi_bitmap_iterator_alloc(ndpi_bitmap* b);
void ndpi_bitmap_iterator_free(ndpi_bitmap* b);
bool ndpi_bitmap_iterator_next(ndpi_bitmap_iterator* i, uint32_t *value);
/* ******************************* */
char* ndpi_get_flow_risk_info(struct ndpi_flow_struct *flow,
char *out, u_int out_len,
u_int8_t use_json);
#ifdef __cplusplus
}
#endif
+97 -26
View File
@@ -27,8 +27,8 @@
gcc -E -dM - < /dev/null |grep ENDIAN
*/
#if defined(__FreeBSD__) || defined(__NetBSD__) || defined(__OpenBSD__)
#include <machine/endian.h>
#if defined(__FreeBSD__) || defined(__NetBSD__)
#include <sys/endian.h>
#endif
#ifdef __OpenBSD__
@@ -156,7 +156,7 @@
/* misc definitions */
#define NDPI_DEFAULT_MAX_TCP_RETRANSMISSION_WINDOW_SIZE 0x10000
#define NDPI_DEFAULT_MAX_NUM_PKTS_PER_FLOW_TO_DISSECT 32
/* TODO: rebuild all memory areas to have a more aligned memory block here */
@@ -166,26 +166,6 @@
#define MAX_PACKET_COUNTER 65000
#define MAX_DEFAULT_PORTS 5
#define NDPI_DIRECTCONNECT_CONNECTION_IP_TICK_TIMEOUT 600
#define NDPI_IRC_CONNECTION_TIMEOUT 120
#define NDPI_GNUTELLA_CONNECTION_TIMEOUT 60
#define NDPI_BATTLEFIELD_CONNECTION_TIMEOUT 60
#define NDPI_THUNDER_CONNECTION_TIMEOUT 30
#define NDPI_RTSP_CONNECTION_TIMEOUT 5
#define NDPI_TVANTS_CONNECTION_TIMEOUT 5
#define NDPI_YAHOO_DETECT_HTTP_CONNECTIONS 1
#define NDPI_YAHOO_LAN_VIDEO_TIMEOUT 30
#define NDPI_ZATTOO_CONNECTION_TIMEOUT 120
#define NDPI_ZATTOO_FLASH_TIMEOUT 5
#define NDPI_JABBER_STUN_TIMEOUT 30
#define NDPI_JABBER_FT_TIMEOUT 5
#define NDPI_SOULSEEK_CONNECTION_IP_TICK_TIMEOUT 600
#ifndef _NDPI_CONFIG_H_
#include "ndpi_config.h" /* To have access to NDPI_ENABLE_DEBUG_MESSAGES */
#define _NDPI_CONFIG_H_
#endif
#ifdef NDPI_ENABLE_DEBUG_MESSAGES
#define NDPI_LOG(proto, m, log_level, args...) \
{ \
@@ -316,17 +296,29 @@
#define NDPI_SCTP_PROTOCOL_TYPE 132
#define NDPI_IPIP_PROTOCOL_TYPE 0x04
#define NDPI_ICMPV6_PROTOCOL_TYPE 0x3a
#define NDPI_PGM_PROTOCOL_TYPE 0x71
#define NDPI_PIM_PROTOCOL_TYPE 0x67
#define NDPI_ARRAY_LENGTH(array) (sizeof(array) / sizeof((array)[0]))
/* the get_uXX will return raw network packet bytes !! */
#define get_u_int8_t(X,O) (*(u_int8_t *)((&(((u_int8_t *)X)[O]))))
#define get_u_int16_t(X,O) (*(u_int16_t *)((&(((u_int8_t *)X)[O]))))
#define get_u_int32_t(X,O) (*(u_int32_t *)((&(((u_int8_t *)X)[O]))))
#if defined(__arm__)
static inline u_int64_t get_u_int64_t(const u_int8_t* X, int O)
{
u_int64_t tmp;
memcpy(&tmp, X + O, sizeof(tmp));
return tmp;
}
#else
#define get_u_int64_t(X,O) (*(u_int64_t *)((&(((u_int8_t *)X)[O]))))
#endif // __arm__
/* new definitions to get little endian from network bytes */
#define get_ul8(X,O) get_u_int8_t(X,O)
#if defined(__LITTLE_ENDIAN__) || defined(_LITTLE_ENDIAN)
#define get_l16(X,O) get_u_int16_t(X,O)
#define get_l32(X,O) get_u_int32_t(X,O)
@@ -341,10 +333,19 @@
/* define memory callback function */
#define match_first_bytes(payload,st) (memcmp((payload),(st),(sizeof(st)-1))==0)
#if defined(WIN32) && !defined(snprintf)
#define snprintf _snprintf
#if defined(WIN32)
#undef strtok_r
#define strtok_r strtok_s
#if BYTE_ORDER == LITTLE_ENDIAN
#define le16toh(x) (x)
#define le32toh(x) (x)
#else
#error "byte order not supported"
#endif
#endif /* WIN32 */
#define NDPI_MAX_DNS_REQUESTS 16
#define NDPI_MIN_NUM_STUN_DETECTION 8
@@ -361,6 +362,8 @@
#define NDPI_MAX_NUM_TLS_APPL_BLOCKS 8
@HANDLE_TLS_SIGS@#define TLS_HANDLE_SIGNATURE_ALGORITMS 1
#ifdef __APPLE__
#include <libkern/OSByteOrder.h>
@@ -382,4 +385,72 @@
#endif /* __APPLE__ */
#if defined(__MINGW32__)
#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
#define htobe16(x) htons(x)
#define htole16(x) (x)
#define be16toh(x) ntohs(x)
#define le16toh(x) (x)
#define htobe32(x) htonl(x)
#define htole32(x) (x)
#define be32toh(x) ntohl(x)
#define le32toh(x) (x)
#define htobe64(x) htonll(x)
#define htole64(x) (x)
#define be64toh(x) ntohll(x)
#define le64toh(x) (x)
#elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
#define htobe16(x) (x)
#define htole16(x) __builtin_bswap16(x)
#define be16toh(x) (x)
#define le16toh(x) __builtin_bswap16(x)
#define htobe32(x) (x)
#define htole32(x) __builtin_bswap32(x)
#define be32toh(x) (x)
#define le32toh(x) __builtin_bswap32(x)
#define htobe64(x) (x)
#define htole64(x) __builtin_bswap64(x)
#define be64toh(x) (x)
#define le64toh(x) __builtin_bswap64(x)
#else
#error Unexpected __BYTE_ORDER__
#endif /* __BYTE_ORDER__ */
#endif /* __MINGW32__ */
#ifndef ETH_ARP
#define ETH_ARP 0x0806
#endif
#ifndef ETH_P_IP
#define ETH_P_IP 0x0800 /* IPv4 */
#endif
#ifndef ETH_P_IPV6
#define ETH_P_IPV6 0x86dd /* IPv6 */
#endif
#ifndef ETH_P_VLAN
#define ETH_P_VLAN 0x8100
#endif
#ifndef ETH_P_MPLS_UNI
#define ETH_P_MPLS_UNI 0x8847
#endif
#ifndef ETH_P_MPLS_MULTI
#define ETH_P_MPLS_MULTI 0x8848
#endif
#ifndef ETH_P_PPPoE
#define ETH_P_PPPoE 0x8864
#endif
#endif /* __NDPI_DEFINE_INCLUDE_FILE__ */
+1 -1
View File
@@ -1,7 +1,7 @@
/*
* ndpi_encryption.h
*
* Copyright (C) 2016-21 - ntop.org
* Copyright (C) 2016-22 - ntop.org
*
* nDPI is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
+2 -6
View File
@@ -1,7 +1,7 @@
/*
* ndpi_includes.h
*
* Copyright (C) 2011-16 - ntop.org
* Copyright (C) 2011-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -32,13 +32,13 @@
#include <ctype.h>
#include <time.h>
#include <limits.h>
#include <stdbool.h>
#ifdef WIN32
#include "ndpi_win32.h"
#else
#include <sys/types.h>
#include <sys/param.h>
#include <pthread.h>
#include <arpa/inet.h>
#include <sys/time.h>
#include <sys/socket.h>
@@ -57,10 +57,6 @@
#if defined __NetBSD__ || defined __OpenBSD__
#include <netinet/in_systm.h>
#if defined __OpenBSD__
#include <pthread.h>
#endif
#endif
#endif
#endif
+1 -5
View File
@@ -1,7 +1,7 @@
/*
* ndpi_includes_OpenBSD.h
*
* Copyright (C) 2011-16 - ntop.org
* Copyright (C) 2011-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -24,10 +24,6 @@
#ifndef __NDPI_INCLUDES_OPENBSD_H__
#define __NDPI_INCLUDES_OPENBSD_H__
#ifndef IPPROTO_SCTP
#define IPPROTO_SCTP 132
#endif /* IPPROTO_SCTP */
#include <net/bpf.h>
typedef struct bpf_timeval pkt_timeval;
+30 -23
View File
@@ -1,7 +1,7 @@
/*
* ndpi_main.h
*
* Copyright (C) 2011-21 - ntop.org
* Copyright (C) 2011-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -25,6 +25,10 @@
#define __NDPI_MAIN_H__
#include "ndpi_includes.h"
#ifdef NDPI_LIB_COMPILATION
/* for macros NDPI_LOG_* in ndpi_define.h */
#include "ndpi_config.h"
#endif
#include "ndpi_define.h"
#include "ndpi_protocol_ids.h"
#include "ndpi_typedefs.h"
@@ -77,7 +81,8 @@ extern "C" {
void ndpi_set_detected_protocol(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow,
u_int16_t upper_detected_protocol,
u_int16_t lower_detected_protocol);
u_int16_t lower_detected_protocol,
ndpi_confidence_t confidence);
extern void ndpi_parse_packet_line_info(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow);
@@ -87,14 +92,6 @@ extern "C" {
extern u_int16_t ndpi_check_for_email_address(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow, u_int16_t counter);
extern void ndpi_int_change_packet_protocol(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow,
u_int16_t upper_detected_protocol,
u_int16_t lower_detected_protocol);
extern void ndpi_int_change_protocol(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow,
u_int16_t upper_detected_protocol,
u_int16_t lower_detected_protocol);
extern void ndpi_int_change_category(struct ndpi_detection_module_struct *ndpi_struct,
struct ndpi_flow_struct *flow,
ndpi_protocol_category_t protocol_category);
@@ -102,13 +99,6 @@ extern "C" {
extern void ndpi_set_proto_subprotocols(struct ndpi_detection_module_struct *ndpi_mod,
int protoId, ...);
extern void ndpi_set_proto_defaults(struct ndpi_detection_module_struct *ndpi_mod,
ndpi_protocol_breed_t protoBreed, u_int16_t protoId, char *protoName,
ndpi_protocol_category_t protoCategory,
ndpi_port_range *tcpDefPorts,
ndpi_port_range *udpDefPorts);
extern void ndpi_int_reset_packet_protocol(struct ndpi_packet_struct *packet);
extern void ndpi_int_reset_protocol(struct ndpi_flow_struct *flow);
extern int ndpi_packet_src_ip_eql(const struct ndpi_packet_struct *packet, const ndpi_ip_addr_t * ip);
@@ -130,8 +120,6 @@ extern "C" {
extern u_int8_t ndpi_is_proto(ndpi_protocol proto, u_int16_t p);
extern u_int16_t ndpi_get_lower_proto(ndpi_protocol p);
#ifdef NDPI_ENABLE_DEBUG_MESSAGES
void ndpi_debug_get_last_log_function_line(struct ndpi_detection_module_struct *ndpi_struct,
const char **file, const char **func, u_int32_t * line);
@@ -148,16 +136,35 @@ extern "C" {
#define ndpi_match_strprefix(payload, payload_len, str) \
ndpi_match_prefix((payload), (payload_len), (str), (sizeof(str)-1))
int ndpi_handle_ipv6_extension_headers(struct ndpi_detection_module_struct *ndpi_str,
int ndpi_handle_ipv6_extension_headers(u_int16_t l3len,
const u_int8_t ** l4ptr, u_int16_t * l4len,
u_int8_t * nxt_hdr);
void ndpi_set_proto_defaults(struct ndpi_detection_module_struct *ndpi_str,
u_int8_t is_cleartext, u_int8_t is_app_protocol,
ndpi_protocol_breed_t breed,
u_int16_t protoId, char *protoName,
ndpi_protocol_category_t protoCategory,
ndpi_port_range *tcpDefPorts,
ndpi_port_range *udpDefPorts);
void ndpi_set_risk(struct ndpi_detection_module_struct *ndpi_str,
struct ndpi_flow_struct *flow, ndpi_risk_enum r);
int ndpi_is_printable_string(char const * const str, size_t len);
float ndpi_calculate_entropy(u_int8_t const * const buf, size_t len);
struct ndpi_flow_struct *flow, ndpi_risk_enum r,
char *risk_message);
int ndpi_isset_risk(struct ndpi_detection_module_struct *ndpi_str,
struct ndpi_flow_struct *flow, ndpi_risk_enum r);
int ndpi_is_printable_buffer(uint8_t const * const buf, size_t len);
int ndpi_normalize_printable_string(char * const str, size_t len);
int ndpi_is_valid_hostname(char * const str, size_t len);
#define NDPI_ENTROPY_ENCRYPTED_OR_RANDOM(entropy) (entropy > 7.0f)
float ndpi_entropy(u_int8_t const * const buf, size_t len);
u_int16_t ndpi_calculate_icmp4_checksum(u_int8_t const * const buf, size_t len);
void load_common_alpns(struct ndpi_detection_module_struct *ndpi_str);
u_int8_t is_a_common_alpn(struct ndpi_detection_module_struct *ndpi_str,
const char *alpn_to_check, u_int alpn_to_check_len);
char *ndpi_hostname_sni_set(struct ndpi_flow_struct *flow, const u_int8_t *value, size_t value_len);
char *ndpi_user_agent_set(struct ndpi_flow_struct *flow, const u_int8_t *value, size_t value_len);
#ifdef __cplusplus
}
#endif
+5 -4
View File
@@ -1,7 +1,7 @@
/*
* ndpi_patricia_typedef.h
*
* Copyright (C) 2011-21 - ntop.org
* Copyright (C) 2011-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -65,9 +65,7 @@
#define _NDPI_PATRICIA_TYPEDEF_H_
/* pointer to usr data (ex. route flap info) */
union ndpi_patricia_node_value_t {
void *user_data;
union ndpi_patricia_node_value_t {
/* User-defined values */
union {
struct {
@@ -75,6 +73,8 @@ union ndpi_patricia_node_value_t {
} uv32;
u_int64_t uv64;
void *user_data;
} u;
};
@@ -95,6 +95,7 @@ typedef struct _ndpi_patricia_node_t {
struct _ndpi_patricia_node_t *l, *r; /* left and right children */
struct _ndpi_patricia_node_t *parent;/* may be used */
void *data; /* pointer to data */
void *custom_user_data; /* pointer to custom userdata */
union ndpi_patricia_node_value_t value;
} ndpi_patricia_node_t;
+105 -64
View File
@@ -1,7 +1,7 @@
/*
* ndpi_protocol_ids.h
*
* Copyright (C) 2011-21 - ntop.org
* Copyright (C) 2011-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -29,7 +29,7 @@
typedef enum {
NDPI_PROTOCOL_UNKNOWN = 0,
NDPI_PROTOCOL_FTP_CONTROL = 1, /* Tomasz Bujlow <tomasz@skatnet.dk> */
NDPI_PROTOCOL_FTP_CONTROL = 1,
NDPI_PROTOCOL_MAIL_POP = 2,
NDPI_PROTOCOL_MAIL_SMTP = 3,
NDPI_PROTOCOL_MAIL_IMAP = 4,
@@ -49,7 +49,7 @@ typedef enum {
NDPI_PROTOCOL_DHCP = 18,
NDPI_PROTOCOL_POSTGRES = 19,
NDPI_PROTOCOL_MYSQL = 20,
NDPI_PROTOCOL_HOTMAIL = 21,
NDPI_PROTOCOL_MS_OUTLOOK = 21, /* Hotmail / Microsoft Outlook / Exchange */
NDPI_PROTOCOL_DIRECT_DOWNLOAD_LINK = 22,
NDPI_PROTOCOL_MAIL_POPS = 23,
NDPI_PROTOCOL_APPLEJUICE = 24,
@@ -59,19 +59,19 @@ typedef enum {
NDPI_PROTOCOL_VMWARE = 28,
NDPI_PROTOCOL_MAIL_SMTPS = 29,
NDPI_PROTOCOL_DTLS = 30,
NDPI_PROTOCOL_UBNTAC2 = 31, /* Ubiquity UBNT AirControl = 2 - Thomas Fjellstrom <thomas+ndpi@fjellstrom.ca> */
NDPI_PROTOCOL_UBNTAC2 = 31, /* Ubiquity UBNT AirControl = 2 */
NDPI_PROTOCOL_KONTIKI = 32,
NDPI_PROTOCOL_OPENFT = 33,
NDPI_PROTOCOL_FASTTRACK = 34,
NDPI_PROTOCOL_GNUTELLA = 35,
NDPI_PROTOCOL_EDONKEY = 36, /* Tomasz Bujlow <tomasz@skatnet.dk> */
NDPI_PROTOCOL_EDONKEY = 36,
NDPI_PROTOCOL_BITTORRENT = 37,
NDPI_PROTOCOL_SKYPE_CALL = 38, /* Skype call and videocalls */
NDPI_PROTOCOL_SKYPE_TEAMS_CALL = 38, /* Skype call and videocalls */
NDPI_PROTOCOL_SIGNAL = 39,
NDPI_PROTOCOL_MEMCACHED = 40, /* Memcached - Darryl Sokoloski <darryl@egloo.ca> */
NDPI_PROTOCOL_MEMCACHED = 40, /* Memcached */
NDPI_PROTOCOL_SMBV23 = 41, /* SMB version 2/3 */
NDPI_PROTOCOL_MINING = 42, /* Bitcoin, Ethereum, ZCash, Monero */
NDPI_PROTOCOL_NEST_LOG_SINK = 43, /* Nest Log Sink (Nest Protect) - Darryl Sokoloski <darryl@egloo.ca> */
NDPI_PROTOCOL_NEST_LOG_SINK = 43, /* Nest Log Sink (Nest Protect) */
NDPI_PROTOCOL_MODBUS = 44, /* Modbus */
NDPI_PROTOCOL_WHATSAPP_CALL = 45, /* WhatsApp video ad audio calls go here */
NDPI_PROTOCOL_DATASAVER = 46, /* Protocols used to save data on Internet communications */
@@ -88,26 +88,26 @@ typedef enum {
NDPI_PROTOCOL_SOPCAST = 57,
NDPI_PROTOCOL_DISCORD = 58,
NDPI_PROTOCOL_TVUPLAYER = 59,
NDPI_PROTOCOL_MONGODB = 60, /* Leonn Paiva <leonn.paiva@gmail.com> */
NDPI_PROTOCOL_QQLIVE = 61,
NDPI_PROTOCOL_MONGODB = 60,
NDPI_PROTOCOL_PLURALSIGHT = 61,
NDPI_PROTOCOL_THUNDER = 62,
NDPI_PROTOCOL_SOULSEEK = 63,
NDPI_PROTOCOL_PS_VUE = 64,
NDPI_PROTOCOL_OCSP = 63,
NDPI_PROTOCOL_VXLAN = 64,
NDPI_PROTOCOL_IRC = 65,
NDPI_PROTOCOL_AYIYA = 66,
NDPI_PROTOCOL_JABBER = 67,
NDPI_PROTOCOL_NATS = 68,
NDPI_PROTOCOL_AMONG_US = 69, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_AMONG_US = 69,
NDPI_PROTOCOL_YAHOO = 70,
NDPI_PROTOCOL_DISNEYPLUS = 71, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_DISNEYPLUS = 71,
NDPI_PROTOCOL_GOOGLE_PLUS = 72,
NDPI_PROTOCOL_IP_VRRP = 73,
NDPI_PROTOCOL_STEAM = 74, /* Tomasz Bujlow <tomasz@skatnet.dk> */
NDPI_PROTOCOL_STEAM = 74,
NDPI_PROTOCOL_HALFLIFE2 = 75,
NDPI_PROTOCOL_WORLDOFWARCRAFT = 76,
NDPI_PROTOCOL_TELNET = 77,
NDPI_PROTOCOL_STUN = 78,
NDPI_PROTOCOL_IP_IPSEC = 79,
NDPI_PROTOCOL_IPSEC = 79,
NDPI_PROTOCOL_IP_GRE = 80,
NDPI_PROTOCOL_IP_ICMP = 81,
NDPI_PROTOCOL_IP_IGMP = 82,
@@ -118,7 +118,7 @@ typedef enum {
NDPI_PROTOCOL_RTP = 87,
NDPI_PROTOCOL_RDP = 88,
NDPI_PROTOCOL_VNC = 89,
NDPI_PROTOCOL_TUMBLR = 90, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_TUMBLR = 90,
NDPI_PROTOCOL_TLS = 91,
NDPI_PROTOCOL_SSH = 92,
NDPI_PROTOCOL_USENET = 93,
@@ -138,7 +138,7 @@ typedef enum {
NDPI_PROTOCOL_FIESTA = 107,
NDPI_PROTOCOL_FLORENSIA = 108,
NDPI_PROTOCOL_GUILDWARS = 109,
NDPI_PROTOCOL_AMAZON_ALEXA = 110, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_AMAZON_ALEXA = 110,
NDPI_PROTOCOL_KERBEROS = 111,
NDPI_PROTOCOL_LDAP = 112,
NDPI_PROTOCOL_MAPLESTORY = 113,
@@ -155,7 +155,7 @@ typedef enum {
NDPI_PROTOCOL_YOUTUBE = 124,
NDPI_PROTOCOL_SKYPE_TEAMS = 125,
NDPI_PROTOCOL_GOOGLE = 126,
NDPI_PROTOCOL_DCERPC = 127,
NDPI_PROTOCOL_RPC = 127,
NDPI_PROTOCOL_NETFLOW = 128,
NDPI_PROTOCOL_SFLOW = 129,
NDPI_PROTOCOL_HTTP_CONNECT = 130,
@@ -167,7 +167,7 @@ typedef enum {
NDPI_PROTOCOL_YOUTUBE_UPLOAD = 136, /* Upload files to youtube */
NDPI_PROTOCOL_HULU = 137,
NDPI_PROTOCOL_CHECKMK = 138,
NDPI_PROTOCOL_AJP = 139, /* Leonn Paiva <leonn.paiva@gmail.com> */
NDPI_PROTOCOL_AJP = 139,
NDPI_PROTOCOL_APPLE = 140,
NDPI_PROTOCOL_WEBEX = 141,
NDPI_PROTOCOL_WHATSAPP = 142,
@@ -176,72 +176,72 @@ typedef enum {
NDPI_PROTOCOL_APPLE_ITUNES = 145,
NDPI_PROTOCOL_RADIUS = 146,
NDPI_PROTOCOL_WINDOWS_UPDATE = 147,
NDPI_PROTOCOL_TEAMVIEWER = 148, /* xplico.org */
NDPI_PROTOCOL_TEAMVIEWER = 148,
NDPI_PROTOCOL_TUENTI = 149,
NDPI_PROTOCOL_LOTUS_NOTES = 150,
NDPI_PROTOCOL_SAP = 151,
NDPI_PROTOCOL_GTP = 152,
NDPI_PROTOCOL_WSD = 153,
NDPI_PROTOCOL_LLMNR = 154,
NDPI_PROTOCOL_REMOTE_SCAN = 155,
NDPI_PROTOCOL_TOCA_BOCA = 155,
NDPI_PROTOCOL_SPOTIFY = 156,
NDPI_PROTOCOL_MESSENGER = 157,
NDPI_PROTOCOL_H323 = 158, /* Remy Mudingay <mudingay@ill.fr> */
NDPI_PROTOCOL_OPENVPN = 159, /* Remy Mudingay <mudingay@ill.fr> */
NDPI_PROTOCOL_NOE = 160, /* Remy Mudingay <mudingay@ill.fr> */
NDPI_PROTOCOL_CISCOVPN = 161, /* Remy Mudingay <mudingay@ill.fr> */
NDPI_PROTOCOL_TEAMSPEAK = 162, /* Remy Mudingay <mudingay@ill.fr> */
NDPI_PROTOCOL_TOR = 163, /* Remy Mudingay <mudingay@ill.fr> */
NDPI_PROTOCOL_SKINNY = 164, /* Remy Mudingay <mudingay@ill.fr> */
NDPI_PROTOCOL_RTCP = 165, /* Remy Mudingay <mudingay@ill.fr> */
NDPI_PROTOCOL_RSYNC = 166, /* Remy Mudingay <mudingay@ill.fr> */
NDPI_PROTOCOL_ORACLE = 167, /* Remy Mudingay <mudingay@ill.fr> */
NDPI_PROTOCOL_CORBA = 168, /* Remy Mudingay <mudingay@ill.fr> */
NDPI_PROTOCOL_UBUNTUONE = 169, /* Remy Mudingay <mudingay@ill.fr> */
NDPI_PROTOCOL_H323 = 158,
NDPI_PROTOCOL_OPENVPN = 159,
NDPI_PROTOCOL_NOE = 160,
NDPI_PROTOCOL_CISCOVPN = 161,
NDPI_PROTOCOL_TEAMSPEAK = 162,
NDPI_PROTOCOL_TOR = 163,
NDPI_PROTOCOL_SKINNY = 164,
NDPI_PROTOCOL_RTCP = 165,
NDPI_PROTOCOL_RSYNC = 166,
NDPI_PROTOCOL_ORACLE = 167,
NDPI_PROTOCOL_CORBA = 168,
NDPI_PROTOCOL_UBUNTUONE = 169,
NDPI_PROTOCOL_WHOIS_DAS = 170,
NDPI_PROTOCOL_COLLECTD = 171,
NDPI_PROTOCOL_SOCKS = 172, /* Tomasz Bujlow <tomasz@skatnet.dk> */
NDPI_PROTOCOL_SD_RTN = 171, /* Agora SD-RTN: https://www.agora.io/en */
NDPI_PROTOCOL_SOCKS = 172,
NDPI_PROTOCOL_NINTENDO = 173,
NDPI_PROTOCOL_RTMP = 174, /* Tomasz Bujlow <tomasz@skatnet.dk> */
NDPI_PROTOCOL_FTP_DATA = 175, /* Tomasz Bujlow <tomasz@skatnet.dk> */
NDPI_PROTOCOL_WIKIPEDIA = 176, /* Tomasz Bujlow <tomasz@skatnet.dk> */
NDPI_PROTOCOL_RTMP = 174,
NDPI_PROTOCOL_FTP_DATA = 175,
NDPI_PROTOCOL_WIKIPEDIA = 176,
NDPI_PROTOCOL_ZMQ = 177,
NDPI_PROTOCOL_AMAZON = 178, /* Tomasz Bujlow <tomasz@skatnet.dk> */
NDPI_PROTOCOL_EBAY = 179, /* Tomasz Bujlow <tomasz@skatnet.dk> */
NDPI_PROTOCOL_CNN = 180, /* Tomasz Bujlow <tomasz@skatnet.dk> */
NDPI_PROTOCOL_MEGACO = 181, /* Gianluca Costa <g.costa@xplico.org> */
NDPI_PROTOCOL_AMAZON = 178,
NDPI_PROTOCOL_EBAY = 179,
NDPI_PROTOCOL_CNN = 180,
NDPI_PROTOCOL_MEGACO = 181,
NDPI_PROTOCOL_REDIS = 182,
NDPI_PROTOCOL_PINTEREST = 183, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_PINTEREST = 183,
NDPI_PROTOCOL_VHUA = 184,
NDPI_PROTOCOL_TELEGRAM = 185, /* Gianluca Costa <g.costa@xplico.org> */
NDPI_PROTOCOL_TELEGRAM = 185,
NDPI_PROTOCOL_VEVO = 186,
NDPI_PROTOCOL_PANDORA = 187,
NDPI_PROTOCOL_QUIC = 188, /* Andrea Buscarinu <andrea.buscarinu@gmail.com> - Michele Campus <michelecampus5@gmail.com> */
NDPI_PROTOCOL_QUIC = 188,
NDPI_PROTOCOL_ZOOM = 189, /* Zoom video conference. */
NDPI_PROTOCOL_EAQ = 190,
NDPI_PROTOCOL_OOKLA = 191,
NDPI_PROTOCOL_AMQP = 192,
NDPI_PROTOCOL_KAKAOTALK = 193, /* KakaoTalk Chat (no voice call) */
NDPI_PROTOCOL_KAKAOTALK_VOICE = 194, /* KakaoTalk Voice */
NDPI_PROTOCOL_TWITCH = 195, /* Edoardo Dominici <edoaramis@gmail.com> */
NDPI_PROTOCOL_TWITCH = 195,
NDPI_PROTOCOL_DOH_DOT = 196, /* DoH (DNS over HTTPS), DoT (DNS over TLS), DoQ (DNS over QUIC). TODO: rename in NDPI_PROTOCOL_DOH_DOT_DOQ? */
NDPI_PROTOCOL_WECHAT = 197,
NDPI_PROTOCOL_MPEGTS = 198,
NDPI_PROTOCOL_SNAPCHAT = 199,
NDPI_PROTOCOL_SINA = 200,
NDPI_PROTOCOL_HANGOUT_DUO = 201, /* Google Hangout ad Duo (merged as they are very similar) */
NDPI_PROTOCOL_IFLIX = 202, /* www.vizuamatix.com R&D team & M.Mallawaarachchie <manoj_ws@yahoo.com> */
NDPI_PROTOCOL_IFLIX = 202,
NDPI_PROTOCOL_GITHUB = 203,
NDPI_PROTOCOL_BJNP = 204,
NDPI_PROTOCOL_REDDIT = 205, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_REDDIT = 205,
NDPI_PROTOCOL_WIREGUARD = 206,
NDPI_PROTOCOL_SMPP = 207, /* Damir Franusic <df@release14.org> */
NDPI_PROTOCOL_DNSCRYPT = 208, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_TINC = 209, /* William Guglielmo <william@deselmo.com> */
NDPI_PROTOCOL_SMPP = 207,
NDPI_PROTOCOL_DNSCRYPT = 208,
NDPI_PROTOCOL_TINC = 209,
NDPI_PROTOCOL_DEEZER = 210,
NDPI_PROTOCOL_INSTAGRAM = 211, /* Andrea Buscarinu <andrea.buscarinu@gmail.com> */
NDPI_PROTOCOL_INSTAGRAM = 211,
NDPI_PROTOCOL_MICROSOFT = 212,
NDPI_PROTOCOL_STARCRAFT = 213, /* Matteo Bracci <matteobracci1@gmail.com> */
NDPI_PROTOCOL_STARCRAFT = 213,
NDPI_PROTOCOL_TEREDO = 214,
NDPI_PROTOCOL_HOTSPOT_SHIELD = 215,
NDPI_PROTOCOL_IMO = 216,
@@ -260,8 +260,8 @@ typedef enum {
NDPI_PROTOCOL_SOMEIP = 229,
NDPI_PROTOCOL_FIX = 230,
NDPI_PROTOCOL_PLAYSTATION = 231,
NDPI_PROTOCOL_PASTEBIN = 232, /* Paulo Angelo <pa@pauloangelo.com> */
NDPI_PROTOCOL_LINKEDIN = 233, /* Paulo Angelo <pa@pauloangelo.com> */
NDPI_PROTOCOL_PASTEBIN = 232,
NDPI_PROTOCOL_LINKEDIN = 233,
NDPI_PROTOCOL_SOUNDCLOUD = 234,
NDPI_PROTOCOL_CSGO = 235, /* Counter-Strike Global Offensive, Dota = 2 */
NDPI_PROTOCOL_LISP = 236,
@@ -279,19 +279,60 @@ typedef enum {
NDPI_PROTOCOL_ZABBIX = 248,
NDPI_PROTOCOL_S7COMM = 249,
NDPI_PROTOCOL_MSTEAMS = 250,
NDPI_PROTOCOL_WEBSOCKET = 251, /* Leonn Paiva <leonn.paiva@gmail.com> */
NDPI_PROTOCOL_ANYDESK = 252, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_SOAP = 253, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_APPLE_SIRI = 254, /* Zied Aouini <aouinizied@gmail.com> */
NDPI_PROTOCOL_WEBSOCKET = 251,
NDPI_PROTOCOL_ANYDESK = 252,
NDPI_PROTOCOL_SOAP = 253,
NDPI_PROTOCOL_APPLE_SIRI = 254,
NDPI_PROTOCOL_SNAPCHAT_CALL = 255,
NDPI_PROTOCOL_HPVIRTGRP = 256, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_GENSHIN_IMPACT = 257, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_HPVIRTGRP = 256,
NDPI_PROTOCOL_GENSHIN_IMPACT = 257,
NDPI_PROTOCOL_ACTIVISION = 258,
NDPI_PROTOCOL_FORTICLIENT = 259,
NDPI_PROTOCOL_Z3950 = 260, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_Z3950 = 260,
NDPI_PROTOCOL_LIKEE = 261,
NDPI_PROTOCOL_GITLAB = 262,
NDPI_PROTOCOL_AVAST_SECUREDNS = 263, /* Toni Uhlig <matzeton@googlemail.com> */
NDPI_PROTOCOL_AVAST_SECUREDNS = 263,
NDPI_PROTOCOL_CASSANDRA = 264,
NDPI_PROTOCOL_AMAZON_AWS = 265,
NDPI_PROTOCOL_SALESFORCE = 266,
NDPI_PROTOCOL_VIMEO = 267,
NDPI_PROTOCOL_FACEBOOK_VOIP = 268,
NDPI_PROTOCOL_SIGNAL_VOIP = 269,
NDPI_PROTOCOL_FUZE = 270,
NDPI_PROTOCOL_GTP_U = 271,
NDPI_PROTOCOL_GTP_C = 272,
NDPI_PROTOCOL_GTP_PRIME = 273,
NDPI_PROTOCOL_ALIBABA = 274,
NDPI_PROTOCOL_CRASHLYSTICS = 275,
NDPI_PROTOCOL_MICROSOFT_AZURE = 276,
NDPI_PROTOCOL_ICLOUD_PRIVATE_RELAY = 277,
NDPI_PROTOCOL_ETHERNET_IP = 278,
NDPI_PROTOCOL_BADOO = 279,
NDPI_PROTOCOL_ACCUWEATHER = 280,
NDPI_PROTOCOL_GOOGLE_CLASSROOM = 281,
NDPI_PROTOCOL_HSRP = 282,
NDPI_PROTOCOL_CYBERSECURITY = 283, /* Cybersecurity companies */
NDPI_PROTOCOL_GOOGLE_CLOUD = 284,
NDPI_PROTOCOL_TENCENT = 285,
NDPI_PROTOCOL_RAKNET = 286,
NDPI_PROTOCOL_XIAOMI = 287,
NDPI_PROTOCOL_EDGECAST = 288,
NDPI_PROTOCOL_CACHEFLY = 289,
NDPI_PROTOCOL_SOFTETHER = 290,
NDPI_PROTOCOL_MPEGDASH = 291,
NDPI_PROTOCOL_DAZN = 292,
NDPI_PROTOCOL_GOTO = 293, /* GoTo products, mainly GoToMeeting */
NDPI_PROTOCOL_RSH = 294,
NDPI_PROTOCOL_1KXUN = 295,
NDPI_PROTOCOL_IP_PGM = 296,
NDPI_PROTOCOL_IP_PIM = 297,
NDPI_PROTOCOL_COLLECTD = 298,
NDPI_PROTOCOL_TUNNELBEAR = 299,
NDPI_PROTOCOL_CLOUDFLARE_WARP = 300,
NDPI_PROTOCOL_I3D = 301, /* i3d.net: Game Hosting service */
NDPI_PROTOCOL_RIOTGAMES = 302,
NDPI_PROTOCOL_PSIPHON = 303,
NDPI_PROTOCOL_ULTRASURF = 304,
#ifdef CUSTOM_NDPI_PROTOCOLS
#include "../../../nDPI-custom/custom_ndpi_protocol_ids.h"
+21 -3
View File
@@ -1,7 +1,7 @@
/*
* ndpi_protocols.h
*
* Copyright (C) 2011-21 - ntop.org
* Copyright (C) 2011-22 - ntop.org
*
* This file is part of nDPI, an open source deep packet inspection
* library based on the OpenDPI and PACE technology by ipoque GmbH
@@ -60,6 +60,8 @@ void init_ayiya_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_in
void init_amqp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_bgp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_bittorrent_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
int ndpi_search_into_bittorrent_cache(struct ndpi_detection_module_struct *ndpi_struct, struct ndpi_flow_struct *flow,
u_int32_t saddr, u_int16_t sport, u_int32_t daddr, u_int16_t dport);
void init_lisp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_teredo_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_ciscovpn_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
@@ -83,6 +85,7 @@ void init_ftp_control_dissector(struct ndpi_detection_module_struct *ndpi_struct
void init_ftp_data_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_gnutella_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_gtp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_hsrp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_guildwars_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_h323_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_halflife2_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
@@ -142,7 +145,6 @@ void init_smb_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int3
void init_snmp_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_socrates_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_sopcast_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_soulseek_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_socks_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_spotify_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_ssh_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
@@ -167,6 +169,7 @@ void init_vhua_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int
void init_viber_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_vmware_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_vnc_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_vxlan_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_warcraft3_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_whois_das_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_world_of_warcraft_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
@@ -203,7 +206,6 @@ void init_modbus_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_i
void init_capwap_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_zabbix_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_wireguard_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_targus_getdata_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_dnp3_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_104_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_s7comm_dissector(struct ndpi_detection_module_struct *ndpi_struct,u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
@@ -216,5 +218,21 @@ void init_hpvirtgrp_dissector(struct ndpi_detection_module_struct *ndpi_struct,
void init_genshin_impact_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_z3950_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_avast_securedns_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_cassandra_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_ethernet_ip_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_toca_boca_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_sd_rtn_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_raknet_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_xiaomi_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_mpegdash_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_rsh_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_ipsec_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_collectd_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_i3d_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_riotgames_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
void init_ultrasurf_dissector(struct ndpi_detection_module_struct *ndpi_struct, u_int32_t *id, NDPI_PROTOCOL_BITMASK *detection_bitmask);
/* ndpi_main.c */
extern u_int32_t ndpi_ip_port_hash_funct(u_int32_t ip, u_int16_t port);
#endif /* __NDPI_PROTOCOLS_H__ */
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More