]>
source.dussan.org Git - rspamd.git/log
Vsevolod Stakhov [Wed, 1 Apr 2020 13:57:55 +0000 (14:57 +0100)]
Release 2.5
* [Conf] Mark Rspamd emailbl as ignore whitelist
* [Conf] RBL: Add missing emails = true option
* [Feature] Add support for scripts in fuzzy storage
* [Feature] Arc: Add whitelisted_signers_map option
* [Feature] Implement hosts file processing
* [Feature] Neural: Introduce classes bias that allows non-equal classes learning
* [Feature] Update libev to 4.33
* [Fix] Another brain damage html standard adoptions
* [Fix] Another fix for brain damaged obs-fws state
* [Fix] Fix flags that caused force_actions failure
* [Fix] Fix logging issue
* [Fix] Fix lua symbols scores registration when config does not define scores
* [Fix] Fix opaque maps logic
* [Fix] Fix parsing of the html tags with no spaces after attributes
* [Fix] Fix some corner cases in urls parsing, add limits
* [Fix] Fix tlds extraction if custom composition rules are used
* [Fix] Fix variables replacement in mempool
* [Fix] Improve base64 detection
* [Fix] Normalize dynamic scores in ANN correctly
* [Fix] Plug memory leak introduced by #3153
* [Fix] Stat_redis_backend: Fix memory leak and simplify learn path
* [Fix] Try hard to deal with ghost workers
* [Fix] metadata_exporter default formatter
* [Rework] Change the way to extract URLs when dealing with alternative parts
* [Rework] Fix various url extraction issues
* [Rework] Re cache: Load compiled hyperscan in the main process as well
* [Rework] Re cache: Load hyperscan early
* [Rework] Rework URL structure: adjust tld part
* [Rework] Rework URL structure: host field
* [Rework] Rework URL structure: more structure optimisations
* [Rework] Rework URL structure: user field
* [Rework] URL: Another update for urls extraction logic
* [Rework] Urls: Improve query urls handling
* [Rework] Urls: adopt html related stuff
* [Rework] Urls: more rework of the urls sets
* [Rework] Urls: process query urls in HTML urls correctly
* [Rework] Urls: rework urls hash structure
* [Rework] Urls: update lua libraries
* [Rework] Use multiple search tries for different url extraction types
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
Vsevolod Stakhov [Tue, 31 Mar 2020 13:38:48 +0000 (14:38 +0100)]
[Minor] Use stack for keep-alive pool
Vsevolod Stakhov [Tue, 31 Mar 2020 12:12:35 +0000 (13:12 +0100)]
[Minor] Show count of workers to be killed
Vsevolod Stakhov [Tue, 31 Mar 2020 11:19:59 +0000 (12:19 +0100)]
[Minor] Check socket error when reusing redis pool connections
Vsevolod Stakhov [Tue, 31 Mar 2020 11:09:24 +0000 (12:09 +0100)]
[Minor] Check for socket error before reusing keep-alive connection
Vsevolod Stakhov [Tue, 31 Mar 2020 10:25:27 +0000 (11:25 +0100)]
[Fix] Another fix for brain damaged obs-fws state
Vsevolod Stakhov [Tue, 31 Mar 2020 10:25:16 +0000 (11:25 +0100)]
[Minor] Try to fix OOB reads
Vsevolod Stakhov [Tue, 31 Mar 2020 09:26:11 +0000 (10:26 +0100)]
Merge pull request #3315 from moisseev/patch-2
[Minor] Add missing CT badness values
Alexander Moisseev [Tue, 31 Mar 2020 06:19:45 +0000 (09:19 +0300)]
[Minor] Add missing CT badness values
Vsevolod Stakhov [Mon, 30 Mar 2020 11:18:58 +0000 (12:18 +0100)]
[Minor] Fix another close/ev_modify race
Vsevolod Stakhov [Mon, 30 Mar 2020 11:18:47 +0000 (12:18 +0100)]
[Minor] Add missing function
Vsevolod Stakhov [Mon, 30 Mar 2020 11:18:22 +0000 (12:18 +0100)]
[Minor] Slghtly improve cmake reporting + use -O0 with full debug
Vsevolod Stakhov [Mon, 30 Mar 2020 11:18:09 +0000 (12:18 +0100)]
[Minor] Fix build with leak sanitizer
Vsevolod Stakhov [Sun, 29 Mar 2020 13:19:22 +0000 (14:19 +0100)]
[Minor] Fix order of objects destruction for http keepalive conns
Vsevolod Stakhov [Sun, 29 Mar 2020 12:10:14 +0000 (13:10 +0100)]
[Minor] Fix double close in rspamc
Vsevolod Stakhov [Sat, 28 Mar 2020 16:37:50 +0000 (16:37 +0000)]
[Minor] Try harder to avoid races on child termination
Vsevolod Stakhov [Thu, 26 Mar 2020 17:59:23 +0000 (17:59 +0000)]
[Minor] Do not use signalfd - it is totally broken in libev
Vsevolod Stakhov [Thu, 26 Mar 2020 17:36:53 +0000 (17:36 +0000)]
[Fix] Try hard to deal with ghost workers
Vsevolod Stakhov [Thu, 26 Mar 2020 17:36:34 +0000 (17:36 +0000)]
[Minor] Do not try to set title on osx: it is totally broken
Vsevolod Stakhov [Thu, 26 Mar 2020 17:05:02 +0000 (17:05 +0000)]
[Minor] Fix usage of signalfd and timerfd
Vsevolod Stakhov [Thu, 26 Mar 2020 15:51:27 +0000 (15:51 +0000)]
[Conf] Mark Rspamd emailbl as ignore whitelist
Vsevolod Stakhov [Thu, 26 Mar 2020 15:50:42 +0000 (15:50 +0000)]
Merge pull request #3309 from ricalfieri/patch-1
Update rbl.conf
Vsevolod Stakhov [Thu, 26 Mar 2020 12:36:31 +0000 (12:36 +0000)]
[Minor] Settings: Fix parsing of the settings map
Vsevolod Stakhov [Thu, 26 Mar 2020 09:45:36 +0000 (09:45 +0000)]
[Minor] Fix return values
Vsevolod Stakhov [Wed, 25 Mar 2020 16:43:52 +0000 (16:43 +0000)]
[Conf] RBL: Add missing emails = true option
Vsevolod Stakhov [Wed, 25 Mar 2020 16:40:36 +0000 (16:40 +0000)]
[Feature] Arc: Add whitelisted_signers_map option
Issue: #3308
Vsevolod Stakhov [Wed, 25 Mar 2020 15:36:14 +0000 (15:36 +0000)]
[Minor] Arc: Improve symbols options format
Riccardo Alfieri [Wed, 25 Mar 2020 13:44:20 +0000 (14:44 +0100)]
Update rbl.conf
MSBL list a lot of gmail dropboxes, but these are being excluded from the checks due to gmail.com being whitelisted. Same happens for other freemail providers.
Ignoring the whitelist in this case should be safe enough.
Vsevolod Stakhov [Wed, 25 Mar 2020 10:19:56 +0000 (10:19 +0000)]
[Minor] Slightly improve ev backend selection
Vsevolod Stakhov [Tue, 24 Mar 2020 16:59:07 +0000 (16:59 +0000)]
[Minor] Disable EVBACKEND_IOURING for now as recommended
Vsevolod Stakhov [Tue, 24 Mar 2020 16:36:43 +0000 (16:36 +0000)]
[Feature] Update libev to 4.33
Changelog: http://cvs.schmorp.de/libev/Changes?pathrev=rel-4_33
Vsevolod Stakhov [Tue, 24 Mar 2020 11:02:13 +0000 (11:02 +0000)]
Merge pull request #3307 from moisseev/patch-1
[WebUI] Fix favicon path
Vsevolod Stakhov [Tue, 24 Mar 2020 10:56:33 +0000 (10:56 +0000)]
[Minor] Add logging
Vsevolod Stakhov [Tue, 24 Mar 2020 10:45:58 +0000 (10:45 +0000)]
[Minor] Add sanity check
Vsevolod Stakhov [Mon, 23 Mar 2020 16:13:45 +0000 (16:13 +0000)]
[Minor] Reduce log noise
Vsevolod Stakhov [Mon, 23 Mar 2020 16:01:28 +0000 (16:01 +0000)]
[Rework] Re cache: Load compiled hyperscan in the main process as well
Vsevolod Stakhov [Mon, 23 Mar 2020 15:34:04 +0000 (15:34 +0000)]
[Rework] Re cache: Load hyperscan early
Vsevolod Stakhov [Mon, 23 Mar 2020 14:50:24 +0000 (14:50 +0000)]
[Rework] URL: Another update for urls extraction logic
URL extraction from HTML parts should look like this:
1. Extract href links
2. Convert HTML to plain text and extract:
a) (http|https|ftp)://foo.bar and www.foo
b) email like strings \bfoo@bar.baz\b .
For all extracted strings check if we have host with a domain from the public suffix.
Vsevolod Stakhov [Mon, 23 Mar 2020 08:29:35 +0000 (08:29 +0000)]
[Minor] Oops, forgot to fill struct field
Alexander Moisseev [Sun, 22 Mar 2020 19:26:17 +0000 (22:26 +0300)]
[WebUI] Fix favicon path
Issue: #3207
Vsevolod Stakhov [Sun, 22 Mar 2020 17:25:32 +0000 (17:25 +0000)]
[Rework] Urls: process query urls in HTML urls correctly
Vsevolod Stakhov [Sun, 22 Mar 2020 13:08:26 +0000 (13:08 +0000)]
[Rework] Urls: Improve query urls handling
Vsevolod Stakhov [Fri, 20 Mar 2020 14:58:21 +0000 (14:58 +0000)]
[Minor] Replace g_abort with abort
Vsevolod Stakhov [Thu, 19 Mar 2020 18:33:16 +0000 (18:33 +0000)]
[Rework] Fix various url extraction issues
Vsevolod Stakhov [Thu, 19 Mar 2020 17:07:33 +0000 (17:07 +0000)]
[Minor] Improve dkim failures logging
Vsevolod Stakhov [Thu, 19 Mar 2020 16:40:51 +0000 (16:40 +0000)]
[Rework] Use multiple search tries for different url extraction types
Vsevolod Stakhov [Thu, 19 Mar 2020 16:06:42 +0000 (16:06 +0000)]
[Rework] Change the way to extract URLs when dealing with alternative parts
Vsevolod Stakhov [Thu, 19 Mar 2020 15:26:45 +0000 (15:26 +0000)]
[Fix] Fix some corner cases in urls parsing, add limits
Vsevolod Stakhov [Thu, 19 Mar 2020 15:26:31 +0000 (15:26 +0000)]
[Minor] Remove UCP from TLD parsing
Vsevolod Stakhov [Thu, 19 Mar 2020 13:37:31 +0000 (13:37 +0000)]
[Minor] Use a more strict pattern for tld lookup
Vsevolod Stakhov [Wed, 18 Mar 2020 15:57:53 +0000 (15:57 +0000)]
[Fix] Fix flags that caused force_actions failure
Issue: #3299
Vsevolod Stakhov [Wed, 18 Mar 2020 10:42:23 +0000 (10:42 +0000)]
[Minor] Fix learning with symbols that have no static scores
Vsevolod Stakhov [Tue, 17 Mar 2020 17:07:31 +0000 (17:07 +0000)]
Merge pull request #3302 from korgoth1/master
[TEST] Force Actions Settings
korgoth1 [Tue, 17 Mar 2020 17:04:47 +0000 (20:04 +0300)]
[TEST] Force Actions Settings
Vsevolod Stakhov [Tue, 17 Mar 2020 15:41:05 +0000 (15:41 +0000)]
[Minor] Implement `hosts` parameter in DNS config
Vsevolod Stakhov [Tue, 17 Mar 2020 15:36:54 +0000 (15:36 +0000)]
[Feature] Implement hosts file processing
Vsevolod Stakhov [Tue, 17 Mar 2020 11:45:57 +0000 (11:45 +0000)]
Merge pull request #3301 from citrin/MISSING_MIME_VERSION
[Minor] Improve MISSING_MIME_VERSION rule
Anton Yuzhaninov [Tue, 17 Mar 2020 11:34:12 +0000 (11:34 +0000)]
[Minor] Improve MISSING_MIME_VERSION rule
Previous condition have a complex condition which prevented adding the
symbol for some MIME messages without MIME-Version header.
Vsevolod Stakhov [Mon, 16 Mar 2020 17:08:07 +0000 (17:08 +0000)]
[Minor] Firther relax href parsing
Vsevolod Stakhov [Mon, 16 Mar 2020 14:10:48 +0000 (14:10 +0000)]
[Minor] Neural: Fix vectors lengths checking
Vsevolod Stakhov [Mon, 16 Mar 2020 14:10:37 +0000 (14:10 +0000)]
[Test] Fix race condition
Vsevolod Stakhov [Mon, 16 Mar 2020 13:31:01 +0000 (13:31 +0000)]
[Fix] Improve base64 detection
Vsevolod Stakhov [Mon, 16 Mar 2020 11:43:54 +0000 (11:43 +0000)]
[Minor] Fix table unpacking
Vsevolod Stakhov [Mon, 16 Mar 2020 11:32:49 +0000 (11:32 +0000)]
[Minor] Try to fix brain damaged libev timers
Vsevolod Stakhov [Mon, 16 Mar 2020 11:18:20 +0000 (11:18 +0000)]
[Minor] Improve logging of the invalid charset case
Vsevolod Stakhov [Mon, 16 Mar 2020 11:15:12 +0000 (11:15 +0000)]
[Feature] Neural: Introduce classes bias that allows non-equal classes learning
Vsevolod Stakhov [Mon, 16 Mar 2020 10:12:46 +0000 (10:12 +0000)]
[Fix] Normalize dynamic scores in ANN correctly
Vsevolod Stakhov [Sun, 15 Mar 2020 20:57:07 +0000 (20:57 +0000)]
[Minor] Add some docs
Vsevolod Stakhov [Fri, 13 Mar 2020 13:51:34 +0000 (13:51 +0000)]
[Minor] Clickhouse: Try to fix order of extra columns
Vsevolod Stakhov [Thu, 12 Mar 2020 14:56:38 +0000 (14:56 +0000)]
[Minor] Fix old broken logic compatibility
Vsevolod Stakhov [Thu, 12 Mar 2020 14:26:53 +0000 (14:26 +0000)]
[Minor] Oops, fix html urls processing
Vsevolod Stakhov [Thu, 12 Mar 2020 12:27:40 +0000 (12:27 +0000)]
[Fix] Fix lua symbols scores registration when config does not define scores
Vsevolod Stakhov [Wed, 11 Mar 2020 19:50:21 +0000 (19:50 +0000)]
Merge pull request #3298 from korgoth1/master
[Test] URL compose map
korgoth1 [Wed, 11 Mar 2020 19:45:07 +0000 (22:45 +0300)]
[Test] URL compose map
Vsevolod Stakhov [Wed, 11 Mar 2020 17:56:34 +0000 (17:56 +0000)]
[Minor] Zero terminate id string
Vsevolod Stakhov [Wed, 11 Mar 2020 17:53:43 +0000 (17:53 +0000)]
[Fix] Plug memory leak introduced by #3153
Vsevolod Stakhov [Wed, 11 Mar 2020 14:56:29 +0000 (14:56 +0000)]
Revert "[Minor] Filter nil values in rspamd_config:get_all_opt"
This reverts commit
af30ca1eb035d6824107c1f55613099c6331cbdb .
Vsevolod Stakhov [Wed, 11 Mar 2020 14:41:13 +0000 (14:41 +0000)]
[Minor] Fix bitset size
Vsevolod Stakhov [Wed, 11 Mar 2020 14:32:33 +0000 (14:32 +0000)]
[Minor] Filter nil values in rspamd_config:get_all_opt
Vsevolod Stakhov [Wed, 11 Mar 2020 14:31:38 +0000 (14:31 +0000)]
[Minor] Allow to push objects to lua replacing UCL_NULL
Vsevolod Stakhov [Wed, 11 Mar 2020 09:32:25 +0000 (09:32 +0000)]
[Minor] Oops, remove g_free dtor
Vsevolod Stakhov [Tue, 10 Mar 2020 16:31:39 +0000 (16:31 +0000)]
Merge pull request #3292 from tech-ozon-io/segfault_01_master
[Minor] Segfault for some configuration.
Thierry Fournier [Sat, 7 Mar 2020 13:48:31 +0000 (14:48 +0100)]
[Minor] Segfault for some configuration.
When the configuration file contains only these two lines, rspamd
try to send log which announce that it can bind the soket, but
"bind_conf" is NULL and it is dereferenced, so we have a segfault.
worker "normal" {
}
This patch fix the segfault.
Vsevolod Stakhov [Tue, 10 Mar 2020 15:59:20 +0000 (15:59 +0000)]
[Minor] Allocate variable in the pool and avoid malloc
Vsevolod Stakhov [Tue, 10 Mar 2020 15:48:56 +0000 (15:48 +0000)]
[Fix] Fix variables replacement in mempool
Vsevolod Stakhov [Tue, 10 Mar 2020 12:08:46 +0000 (12:08 +0000)]
Merge pull request #3294 from tech-ozon-io/compile_error
[Minor] Compilation error with __has_attribuute operator
Vsevolod Stakhov [Mon, 9 Mar 2020 16:59:54 +0000 (16:59 +0000)]
[Fix] Stat_redis_backend: Fix memory leak and simplify learn path
Vsevolod Stakhov [Mon, 9 Mar 2020 13:09:52 +0000 (13:09 +0000)]
[Minor] Fuzzy_backend_redis: Improve logging
Thierry Fournier [Sun, 8 Mar 2020 09:16:13 +0000 (10:16 +0100)]
[Minor] Compilation error with __has_attribuute operator
GCC recommands to add alternative to __has_attribute operator:
https://gcc.gnu.org/onlinedocs/cpp/_005f_005fhas_005fattribute.html
Vsevolod Stakhov [Mon, 9 Mar 2020 12:37:22 +0000 (12:37 +0000)]
[Minor] Another dkim logs improvement
Vsevolod Stakhov [Mon, 9 Mar 2020 12:11:11 +0000 (12:11 +0000)]
[Test] Sort hash tables in tests
Vsevolod Stakhov [Mon, 9 Mar 2020 10:45:26 +0000 (10:45 +0000)]
[Rework] Urls: update lua libraries
Vsevolod Stakhov [Sat, 7 Mar 2020 16:32:14 +0000 (16:32 +0000)]
[Rework] Urls: adopt html related stuff
Vsevolod Stakhov [Sat, 7 Mar 2020 12:15:51 +0000 (12:15 +0000)]
[Rework] Urls: more rework of the urls sets
Vsevolod Stakhov [Fri, 6 Mar 2020 15:58:22 +0000 (15:58 +0000)]
[Rework] Urls: rework urls hash structure
Vsevolod Stakhov [Fri, 6 Mar 2020 14:03:20 +0000 (14:03 +0000)]
[Rework] Rework URL structure: adjust tld part
Vsevolod Stakhov [Fri, 6 Mar 2020 13:14:41 +0000 (13:14 +0000)]
[Rework] Rework URL structure: more structure optimisations
Vsevolod Stakhov [Fri, 6 Mar 2020 12:01:37 +0000 (12:01 +0000)]
[Rework] Rework URL structure: host field
Vsevolod Stakhov [Fri, 6 Mar 2020 11:27:24 +0000 (11:27 +0000)]
[Rework] Rework URL structure: user field
Vsevolod Stakhov [Sun, 8 Mar 2020 11:45:40 +0000 (11:45 +0000)]
Merge pull request #3293 from xf-/task/apple-touch-icon
[WebUI] Add apple-touch-icon