Luca
47fb5e9f3d
Cleanup
2023-09-07 11:12:37 +02:00
Luca Deri
2bd986a00a
Disabled debug message
2023-09-05 18:39:26 +02:00
Luca Deri
ad9ed4f954
Minor warning fixes
2023-09-05 18:08:16 +02:00
Luca Deri
978df906b3
Improved classification further reducing memory used
2023-09-05 17:03:20 +02:00
Luca Deri
0080d0d092
Compilation fixes for old compilers
2023-09-05 10:43:36 +02:00
Luca Deri
feaaf83bed
Added sub-domain classification fix
2023-09-05 07:21:48 +02:00
Luca Deri
cf67f483f4
Classification fixes
2023-09-05 01:17:02 +02:00
Luca Deri
c5ebd84fd8
Added ndpi_bitmap64 support
2023-09-05 00:21:16 +02:00
Luca Deri
4f2ce2d43b
Added ndpi_murmur_hash to the nDPI API
2023-09-04 12:53:42 +02:00
Luca Deri
f0dc3347ec
Merged new and old version of ndpi_domain_classify.c code
2023-09-02 19:27:19 +02:00
Luca Deri
1d480c18e3
Reworked domain classification based on binary filters
2023-09-02 19:16:40 +02:00
Luca Deri
854c2d80f1
Improvement for reducing false positives
2023-09-01 10:26:07 +02:00
Luca Deri
74fdac7d77
Code cleanup
2023-09-01 08:50:55 +02:00
Luca Deri
133f45f3f3
Added ndpi_binary_bitmap datastruture
...
It is similar to ndpi_filter but based on binary search and with
the ability to store a category per value (as ndpi_domain_classify)
2023-08-31 23:51:50 +02:00
Luca Deri
19381f330a
Fixed heap-buffer-overflow issue
2023-08-31 18:35:49 +02:00
Luca Deri
16b0ce3710
Code cleanup
2023-08-31 09:14:17 +02:00
Luca Deri
f824939662
Added comment
2023-08-31 09:00:36 +02:00
Luca Deri
3c7b2a714e
Added missing lists files
2023-08-29 19:21:57 +02:00
Luca Deri
c5c610c941
Fixes dist
2023-08-29 18:49:43 +02:00
Luca Deri
65142b2a53
Added -G flag
2023-08-29 18:32:57 +02:00
Luca Deri
1de7fd9bf0
Warning fixes
2023-08-29 18:08:56 +02:00
Luca Deri
3dbc4f65fc
Added README file
2023-08-29 17:45:36 +02:00
Luca Deri
36abf06c6f
Swap from Aho-Corasick to an experimental/home-grown algorithm that uses a probabilistic
...
approach for handling Internet domain names.
For switching back to Aho-Corasick it is necessary to edit
ndpi-typedefs.h and uncomment the line
// #define USE_LEGACY_AHO_CORASICK
[1] With Aho-Corasick
$ ./example/ndpiReader -G ./lists/ -i tests/pcap/ookla.pcap | grep Memory
nDPI Memory statistics:
nDPI Memory (once): 37.34 KB
Flow Memory (per flow): 960 B
Actual Memory: 33.09 MB
Peak Memory: 33.09 MB
[2] With the new algorithm
$ ./example/ndpiReader -G ./lists/ -i tests/pcap/ookla.pcap | grep Memory
nDPI Memory statistics:
nDPI Memory (once): 37.31 KB
Flow Memory (per flow): 960 B
Actual Memory: 7.42 MB
Peak Memory: 7.42 MB
In essence from ~33 MB to ~7 MB
This new algorithm will enable larger lists to be loaded (e.g. top 1M domans
https://s3-us-west-1.amazonaws.com/umbrella-static/index.html )
In ./lists there are file names that are named as <category>_<string>.list
With -G ndpiReader can load all of them at startup
2023-08-29 17:34:04 +02:00
Toni Uhlig
1f693c3f5a
Added lists/gambling.list to extra dist.
...
* make dist in `./fuzz`: fixed inconsistent `*.dict` file pattern
Signed-off-by: Toni Uhlig <matzeton@googlemail.com >
2023-08-27 15:53:31 +02:00
Toni Uhlig
afd250f5d9
fix compilation and symbol check
...
Signed-off-by: Toni Uhlig <matzeton@googlemail.com >
2023-08-27 15:32:42 +02:00
Toni Uhlig
25f3a7cd1d
fix compilation
...
Signed-off-by: Toni Uhlig <matzeton@googlemail.com >
2023-08-27 15:17:56 +02:00
Luca Deri
34986de297
Search fixes
2023-08-26 19:47:50 +02:00
Luca Deri
eeeee46b1e
Changes for supporinng more efficient sub-string matching
2023-08-26 17:55:50 +02:00
Luca Deri
4ca94369e1
Improved domain search tet unit
2023-08-26 10:43:19 +02:00
Luca Deri
ac00542df2
LEak fix
2023-08-26 10:10:49 +02:00
Luca Deri
2c565c77c9
Added ndpi_domain_classify_XXX(0 API
2023-08-26 00:24:33 +02:00
Luca Deri
42fcb011ad
Warning fix
2023-08-25 19:29:43 +02:00
Luca Deri
eb9932164f
added bimap and/or with allocation
2023-08-24 07:45:38 +02:00
Luca Deri
ff4f291afe
Minor improvements
2023-08-23 11:04:31 +02:00
Luca
95fcd08687
Added ndpi_bitmap_is_empty() and ndpi_bitmap_optimize() API calls
2023-08-23 11:03:59 +02:00
Luca
9933b5910d
Boundary check
2023-08-21 23:19:57 +02:00
Luca
bd25df3334
Improved Wireguard detection
2023-08-21 13:40:42 +02:00
Luca
9e4bcc7771
Removed empty line
2023-08-21 13:40:30 +02:00
Luca
6e5466f9e5
Added ndpi_bitmap_andnot API call
2023-08-21 13:26:38 +02:00
Ivan Nardi
cc4461f424
fuzz: extend coverage ( #2073 )
2023-08-20 15:18:19 +02:00
Ivan Nardi
5867f43fae
Fix compilation on Windows ( #2072 )
2023-08-14 22:51:09 +02:00
Luca Deri
cfbb7a423e
Replaces free() with ndpi_free()
2023-08-14 18:51:11 +02:00
Luca Deri
049fbf819c
Reworked ndpi_filter_xxx implementation using compressed bitmaps
2023-08-14 12:50:54 +02:00
Luca Deri
3b34941be1
Typo
2023-08-12 11:39:19 +02:00
Luca Deri
9a44954365
ARM compilation fix
2023-08-12 07:39:35 +02:00
Luca Deri
fe634b8d55
Fixed to address issue
...
Run ./utils/check_symbols.sh || { FAILED=$?; echo "::error file=${NDPI_LIB}::Unwanted libc symbols found: ${FAILED}. Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of malloc/calloc/realloc/free."; false; }
[ndpi_filter.o]: calloc
Unwanted symbols found: 1
Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of malloc/calloc/realloc/free
Error: Unwanted libc symbols found: 1. Please make sure to use only ndpi_malloc/ndpi_calloc/ndpi_realloc/ndpi_free wrapper instead of
2023-08-12 00:27:37 +02:00
Luca Deri
8e17f50853
Added ndpi_filter_add_multi() API call
2023-08-11 23:55:44 +02:00
Luca Deri
890dcf91d9
Minor initialization fixes
2023-08-11 23:51:16 +02:00
Luca Deri
ec7adc212e
Added new API calls for implementing Bloom-filter like data structures
...
ndpi_filter* ndpi_filter_alloc(uint32_t elements_number);
bool ndpi_filter_add(ndpi_filter *f, uint64_t value);
bool ndpi_filter_contains(ndpi_filter *f, uint64_t value);
void ndpi_filter_free(ndpi_filter *f);
2023-08-11 22:49:31 +02:00
Luca
5fad72ebd9
Typo fix
2023-08-07 10:20:21 +02:00