Commit Graph
51 Commits
Author SHA1 Message Date
Luca Deri fe2822c6a8 Added example for finding similarities in RRDs using nDPI statistical APIs 2021-12-04 10:09:01 +01:00
Luca Deri 1b077ba65f Demo tool for finding anomalies in RRD files 2021-12-03 17:36:49 +01:00
Luca Deri 937357e4bc Implemented ndpi_ses_fitting() and ndpi_des_fitting()
for comuting the best alpha/beta values for exponential smoothing
2021-10-12 13:08:58 +02:00
Luca Deri d13bc0e01a Fixed DES initialisation 2021-10-12 12:38:40 +02:00
Ivan Nardi 3e06e6f5b8 Fix some ndpi_malloc/ndpi_free pairs (#1298) 2021-09-13 18:25:22 +02:00
Alfredo Cardigliano 8e7b063804 Fix possible segfault/leak 2021-07-05 15:09:48 +02:00
Luca Deri 613e21002a Upgraded exponential smoothing to 64 bit values 2021-06-18 22:56:56 +02:00
Toni b040407683 Refactored nDPI subprotocol handling and aimini protocol detection. (#1156)
* Refactored and merged callback buffer routines for non-udp-tcp / udp / tcp / tcp-wo-payload.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Try to detect one subprotocol if a detected protocol can have one.

 * This adds a performance overhead due to much more protocol detection routine calls.
   See #1148 for more information.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactor subprotocol handling (1/2).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactor subprotocol handling (2/2).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Prevent some code duplication by using macros for ndpi_int_one_line_struct string comparision.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Refactored aimini HTTP detection parts (somehow related to #1148).

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Added aimini client/server test pcap.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>

* Removed master protocol as it was only used for STUN and via also removed API function ndpi_get_protocol_id_master_proto

 * Adjusted Python code to conform to the changes made during the refactoring process.

Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2021-03-23 11:46:12 +01:00
Luca Deri 9419015711 Implemented square erro rollup to avoid overflow 2021-03-14 11:01:51 +01:00
Luca 192fad4402 Added double exponential smoothing implementation 2021-03-11 09:39:52 +01:00
Luca Deri 6833ee2bbe Added single exponential smoothing API
int ndpi_ses_init(struct ndpi_ses_struct *ses, double alpha, float significance);
int ndpi_ses_add_value(struct ndpi_ses_struct *ses, const u_int32_t _value, double *forecast, double *confidence_band);
2021-03-11 00:04:33 +01:00
Luca Deri 2d558afb54 Added HW checks 2021-03-01 19:59:14 +01:00
Luca Deri 2724a14bf7 Added further HW checks 2021-03-01 19:34:20 +01:00
Luca Deri 4bff595733 Holt-Winters calculation improvement 2021-02-27 11:32:51 +01:00
Luca Deri 09559611cc Fixes #1136 2021-02-19 15:15:02 +01:00
Ivan Nardi 421609475e Fix small memory leak (#1133)
Now function definition matches the prototype in ndpi_api.h.in
2021-02-10 15:24:34 +01:00
Luca Deri 1331e0aec9 Extended the API to calculate jitter
- ndpi_jitter_init()
- ndpi_jitter_free()
- ndpi_jitter_add_value()
2021-02-09 15:56:03 +01:00
Luca Deri b960809d3e Minor code improvements 2021-02-09 12:55:11 +01:00
Luca Deri 732579b72b Added timeseries forecasting support implementing Holt-Winters with confidence interval
New API calls added
- ndpi_hw_init()
- ndpi_hw_add_value()
- ndpi_hw_free()
2021-02-08 19:10:25 +01:00
Luca Deri 8dd7716ae5 Implemented more efficient and memory savvy RSI 2021-02-05 12:38:41 +01:00
Luca Deri 60b58dbd67 RSI enhancements 2021-02-05 10:59:09 +01:00
Luca Deri 1eedf734be Implemented API for computing RSI (Relative Strenght Index)
void  ndpi_init_rsi(struct ndpi_rsi_struct *s, u_int16_t num_learning_values);
void  ndpi_free_rsi(struct ndpi_rsi_struct *s);
float ndpi_rsi_add_value(struct ndpi_rsi_struct *s, const u_int32_t value);
2021-02-04 23:52:33 +01:00
Luca Deri 23a15bae5f Fixes #1029 2020-11-27 18:51:56 +01:00
Luca Deri 6c119c0b06 Fixes #1032 2020-11-27 18:48:25 +01:00
Luca Deri 948a906037 Added -D flag for detecting DoH in the wild
Removed heuristic from CiscoVPN as it leads to false positives
2020-10-26 21:40:59 +01:00
Luca Deri f597086386 Stddev calculation changes 2020-08-30 12:48:32 +02:00
Luca Deri dd75060932 Fixed false positive in suspicous user agent
Optimized stddev calculation
2020-08-30 12:25:15 +02:00
Luca Deri 21e2e57614 Fixed bin similarity 2020-07-24 01:30:58 +02:00
Luca Deri 439558f6a3 Improved bin clustering 2020-07-22 23:56:50 +02:00
Luca Deri 64fb40d21e Added ndpi_reset_data_analysis() API call 2020-07-17 13:16:01 +02:00
Luca Deri acd05b171e Added ndpi_data_last() API call 2020-07-16 11:22:06 +02:00
Luca Deri 46e97d5da3 Added ndpi_data_window_variance() and ndpi_data_window_stddev() API calls 2020-07-15 18:13:06 +02:00
Luca Deri 9f3e3e8456 Add ndpi_hll_reset() API call
Fixes bug in ndpi_data_window_average() with zero points
2020-07-15 10:30:45 +02:00
Luca Deri 014fdd9a02 Various fixes in bins implementation
Added -b flag in ndpiReader to test bins
2020-07-09 17:28:02 +02:00
Luca Deri d658ac9e61 Added -b to ndpiReader to evaluate bins 2020-07-09 00:15:21 +02:00
Alfredo Cardigliano 5be9370cd2 Fix warnings 2020-07-08 09:15:48 +02:00
Toni Uhlig e308e59002 fixed memory leak in ndpi_cluster_bins / binUnitTest
Signed-off-by: Toni Uhlig <matzeton@googlemail.com>
2020-07-07 17:07:20 +02:00
Luca Deri 1c60c22893 Added ndpi_cluster_bins() for clustering bins and ancillary functions for bins manipulation 2020-07-07 15:10:51 +02:00
Luca Deri 503e3bb495 Added comment on stddev 2020-07-03 14:45:58 +02:00
Luca Deri 392ce44573 Added ndpi_print_bin() API call 2020-06-29 22:36:46 +02:00
Luca Deri 2c263bc726 Added ndpi_bin_similarity() for computing bin similarity 2020-06-29 19:09:38 +02:00
Luca Deri 1a62f4c799 Added ndpi_bin_XXX API
Added packet lenght distribution bins
2020-06-22 01:02:54 +02:00
Luca Deri 60aaa80570 Added HyperLogLog cardinality estimator API calls
/* Memory lifecycle */
int ndpi_hll_init(struct ndpi_hll *hll, u_int8_t bits);
void ndpi_hll_destroy(struct ndpi_hll *hll);

/* Add values */
void ndpi_hll_add(struct ndpi_hll *hll, const char *data, size_t data_len);
void ndpi_hll_add_number(struct ndpi_hll *hll, u_int32_t value) ;

/* Get cardinality estimation */
double ndpi_hll_count(struct ndpi_hll *hll);
2020-06-10 23:43:35 +02:00
Luca Deri d665df64e2 Added ndpi_serialize_raw_record() API call 2020-04-10 21:39:23 +02:00
Luca Deri d457b07f9f Refreshed Tor node list 2019-09-08 21:42:33 +02:00
Luca Deri eb154bac69 Added ndpi_alloc_data_analysis APi call 2019-09-02 22:12:46 +02:00
Luca Deri d0436e7b29 Added new parameter to -P 2019-09-02 14:00:18 +02:00
Luca c328452687 Average calculation fix 2019-08-29 16:08:18 +02:00
Luca 9a6f6d9fe4 Implemented IAT (Inter Arrival Time) stats 2019-08-29 13:40:44 +02:00
Luca e4e40e3c70 Added entropy, average, stddev, variance, bytes ratio calculation 2019-08-28 14:02:39 +02:00