From a20e33c22d63d6d5a53b4aeda7a65c9301709dbc Mon Sep 17 00:00:00 2001 From: Vsevolod Stakhov Date: Mon, 2 Jul 2018 15:23:22 +0100 Subject: [PATCH] Release 1.7.7 * [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov --- ChangeLog | 8345 +++++++++++++++++++++++++++-------------------------- 1 file changed, 4194 insertions(+), 4151 deletions(-) diff --git a/ChangeLog b/ChangeLog index 7f7fbd0c5..b9d6f3299 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3837 +1,3880 @@ +1.7.7: 02 Jul 2018 + * [CritFix] Check NM part of pubkey to match it with rotating keypairs + * [CritFix] Do not overwrite PID of the main process + * [CritFix] Fix maps after reload + * [CritFix] Fix maps race conditions on reload + * [CritFix] Fix shmem leak in encrypting proxy mode + * [Feature] Add a concept of ignored symbols to avoid race conditions + * [Feature] Add ability to print bayes tokens in rspamadm mime + * [Feature] Add method to get statistical tokens in Lua API + * [Feature] Add preliminary mime stat command + * [Feature] Add rspamadm mime tool + * [Feature] Add urls extraction tool + * [Feature] Address ZeroFont exploit + * [Feature] Allow rspamadm mime to process multiple files + * [Feature] Allow to extract words in `rspamadm mime` + * [Feature] Allow to print mime part data + * [Feature] Allow to show HTML structure on extraction + * [Feature] Distinguish IP failures from connection failures + * [Feature] Improve output for mime command + * [Feature] Improve styles propagation + * [Feature] Main process crash will now cleanup all children + * [Feature] Preload file and static maps in main process + * [Feature] Print stack trace on crash + * [Feature] Process font size in HTML parser + * [Feature] Propagate content length of invisible tags + * [Feature] Read ordinary file maps in chunks to be more safe on rewrites + * [Feature] Support base tag in HTML + * [Feature] Support more size suffixes when parsing HTML styles + * [Feature] Support opacity style + * [Fix] Another fix for nested composites + * [Fix] Fill nm id in keypairs cache code + * [Fix] Fix colors alpha channel handling + * [Fix] Fix destruction logic + * [Fix] Fix double free + * [Fix] Fix maps preload logic + * [Fix] Fix nested composites process + * [Fix] Fix proxying of Exim connections + * [Fix] Fix reload crash + * [Fix] Fix rspamadm -l command + * [Fix] Update ed25519 signing schema + * [WebUI] Stop using "const" declaration + * [WebUI] Update RequireJS to 2.3.5 + 1.7.6: 15 Jun 2018 - * [CritFix] Fix multiple neural networks support - * [Feature] Add decryption function to keypair command - * [Feature] Add gzip compression for HTTP requests in elastic module - * [Feature] Add gzip methods to lua util - * [Feature] Add maps based on Top Level Domains - * [Feature] Add pubkey checks for dkim_signing - * [Feature] Add support of fake DNS records - * [Feature] Add tool to encrypt files - * [Feature] Allow to add symbols using settings directly - * [Feature] Allow to match private and public keys for DKIM signatures - * [Feature] Allow to set task flags via settings - * [Feature] Allow to specify fake DNS address from the config - * [Feature] Implement signatures verification using rspamadm keypair - * [Feature] Implement signing using `rspamadm keypair` - * [Feature] Improve error reporting for DKIM key access issues - * [Feature] Provide $HOSTNAME variable in UCL - * [Feature] Rework levenshtein distance computation - * [Feature] Split message parsing and processing - * [Feature] Support ED25519 DKIM signatures - * [Feature] Support encrypted configs in UCL - * [Feature] Suppress duplicate warning on very large radix tries - * [Feature] Use OSB to combine header names - * [Fix] Cleanup maps data on shutdown - * [Fix] Fix '~' behaviour in composites - * [Fix] Fix HTTP maps updates - * [Fix] Fix NIST signatures - * [Fix] Fix RFC822 comments when processing a mime address - * [Fix] Fix double free - * [Fix] Fix dynamic settings application - * [Fix] Fix for CommuniGate Pro maillist - * [Fix] Fix keypair creation method to actually create keypair... - * [Fix] Fix matching patterns with no paths - * [Fix] Fix memory leak in parsing comments - * [Fix] Fix parsing of urls with numeric password - * [Fix] Fix plugins intialisation in configwizard - * [Fix] Fix potential crash on reload - * [Fix] Fix potential race condition for a finished HTTP connections - * [Fix] Fix race-condition leak on processes reload - * [Fix] Fix signing in openssl mode - * [Fix] Free language detector structures - * [Fix] Relax alignment requirements - * [Fix] Send DMARC reports compressed - * [Fix] Try to fix leak in dmarc module - * [Fix] Try to plug memory leak in metric exporter - * [Project] Convert rspamadm subcommands to Lua - * [WebUI] Display smtp sender/recipient in history - * [WebUI] Fix elements disabling in "Symbols" tab - * [WebUI] Limit recipients list in history column to 3 - * [WebUI] Match envelope and mime addresses following in arbitrary order - * [WebUI] Update column header - * [WebUI] Wrap addresses in history + * [CritFix] Fix multiple neural networks support + * [Feature] Add decryption function to keypair command + * [Feature] Add gzip compression for HTTP requests in elastic module + * [Feature] Add gzip methods to lua util + * [Feature] Add maps based on Top Level Domains + * [Feature] Add pubkey checks for dkim_signing + * [Feature] Add support of fake DNS records + * [Feature] Add tool to encrypt files + * [Feature] Allow to add symbols using settings directly + * [Feature] Allow to match private and public keys for DKIM signatures + * [Feature] Allow to set task flags via settings + * [Feature] Allow to specify fake DNS address from the config + * [Feature] Implement signatures verification using rspamadm keypair + * [Feature] Implement signing using `rspamadm keypair` + * [Feature] Improve error reporting for DKIM key access issues + * [Feature] Provide $HOSTNAME variable in UCL + * [Feature] Rework levenshtein distance computation + * [Feature] Split message parsing and processing + * [Feature] Support ED25519 DKIM signatures + * [Feature] Support encrypted configs in UCL + * [Feature] Suppress duplicate warning on very large radix tries + * [Feature] Use OSB to combine header names + * [Fix] Cleanup maps data on shutdown + * [Fix] Fix '~' behaviour in composites + * [Fix] Fix HTTP maps updates + * [Fix] Fix NIST signatures + * [Fix] Fix RFC822 comments when processing a mime address + * [Fix] Fix double free + * [Fix] Fix dynamic settings application + * [Fix] Fix for CommuniGate Pro maillist + * [Fix] Fix keypair creation method to actually create keypair... + * [Fix] Fix matching patterns with no paths + * [Fix] Fix memory leak in parsing comments + * [Fix] Fix parsing of urls with numeric password + * [Fix] Fix plugins intialisation in configwizard + * [Fix] Fix potential crash on reload + * [Fix] Fix potential race condition for a finished HTTP connections + * [Fix] Fix race-condition leak on processes reload + * [Fix] Fix signing in openssl mode + * [Fix] Free language detector structures + * [Fix] Relax alignment requirements + * [Fix] Send DMARC reports compressed + * [Fix] Try to fix leak in dmarc module + * [Fix] Try to plug memory leak in metric exporter + * [Project] Convert rspamadm subcommands to Lua + * [WebUI] Display smtp sender/recipient in history + * [WebUI] Fix elements disabling in "Symbols" tab + * [WebUI] Limit recipients list in history column to 3 + * [WebUI] Match envelope and mime addresses following in arbitrary order + * [WebUI] Update column header + * [WebUI] Wrap addresses in history 1.7.5: 18 May 2018 - * [Conf] Add MSBL proposed return codes - * [Conf] Add additional groups for policies - * [CritFix] Do not use volatile Lua strings as UCL keys - * [Feature] Add ability to add fuzzy hashes to headers - * [Feature] Add function to extract most meaningful urls - * [Feature] Add rule to block mixed text and encrypted parts - * [Feature] Allow multiple groups for symbols - * [Feature] Allow to disable lua squeezing logic - * [Feature] Allow to get multipart children in Lua - * [Feature] Allow to insert multiple headers from milter headers - * [Feature] Allow to print scores in subject and further extensions - * [Feature] Be more error-prone in squeezed rules - * [Feature] Support multiple return codes in emails module - * [Feature] Use EMA for calculating averages - * [Feature] Use common jit cache for all regexps - * [Feature] support for CommuniGate Pro self-generated messages - * [Fix] Allow to have multiple values for headers as arrays - * [Fix] Do not open sockets for disabled workers - * [Fix] Fix AuthservId - * [Fix] Fix base64 folding in Lua API - * [Fix] Fix build on non-x86 platforms - * [Fix] Fix cached maps logic - * [Fix] Fix compatibility with old maps query logic - * [Fix] Fix crash if skip_map is used - * [Fix] Fix importing static maps from UCL - * [Fix] Fix parsing of unix sockets - * [Fix] Fix raw_mime regexp on HTML part with no text content - * [Fix] Fix tables logging - * [Fix] Fix vertical tab handling in libucl - * [Fix] Try to fix frequency counters - * [Fix] Use better sharding for ip_score - * [Fix] Use multiple results from SURBL DNS reply - * [Fix] When doing AV scan select a different server for retransmit + * [Conf] Add MSBL proposed return codes + * [Conf] Add additional groups for policies + * [CritFix] Do not use volatile Lua strings as UCL keys + * [Feature] Add ability to add fuzzy hashes to headers + * [Feature] Add function to extract most meaningful urls + * [Feature] Add rule to block mixed text and encrypted parts + * [Feature] Allow multiple groups for symbols + * [Feature] Allow to disable lua squeezing logic + * [Feature] Allow to get multipart children in Lua + * [Feature] Allow to insert multiple headers from milter headers + * [Feature] Allow to print scores in subject and further extensions + * [Feature] Be more error-prone in squeezed rules + * [Feature] Support multiple return codes in emails module + * [Feature] Use EMA for calculating averages + * [Feature] Use common jit cache for all regexps + * [Feature] support for CommuniGate Pro self-generated messages + * [Fix] Allow to have multiple values for headers as arrays + * [Fix] Do not open sockets for disabled workers + * [Fix] Fix AuthservId + * [Fix] Fix base64 folding in Lua API + * [Fix] Fix build on non-x86 platforms + * [Fix] Fix cached maps logic + * [Fix] Fix compatibility with old maps query logic + * [Fix] Fix crash if skip_map is used + * [Fix] Fix importing static maps from UCL + * [Fix] Fix parsing of unix sockets + * [Fix] Fix raw_mime regexp on HTML part with no text content + * [Fix] Fix tables logging + * [Fix] Fix vertical tab handling in libucl + * [Fix] Try to fix frequency counters + * [Fix] Use better sharding for ip_score + * [Fix] Use multiple results from SURBL DNS reply + * [Fix] When doing AV scan select a different server for retransmit 1.7.4: 01 May 2018 - * [Conf] Major stock config updates: - - Workers are now specified in a new format worker "type" { ... } - - Enable fuzzy worker to simplify local fuzzy storages configuration - - Bind all workers to localhost by default to avoid security flaws - * [Conf] Make more sane fuzzy_check default settings - * [CritFix] Fix ucl escape for bad symbols - * [Feature] Add failure symbol for AV module - * [Feature] Add lazy expiration mode for new classifier schema - * [Feature] Add preliminary version of maps stats plugin - * [Feature] Allow to block fuzzy requests from specific networks - * [Feature] Allow to change `expire` of live statistics - * [Feature] Distinguish AV failure from clean result - * [Feature] Further improvements of language detector by using khash - * [Feature] Further optimization of the lang_detection - * [Feature] Implement cluster-aware bayes expiry - * [Feature] Implement exclude patterns in rspamc - * [Feature] Implement glob maps in addition to regexp maps - * [Feature] Implement map statistics function for lua API - * [Feature] Implement stop symbols for Clickhouse collection - * [Feature] Support recipients separated by commas - * [Feature] Try harder to upload scripts to the Redis server - * [Feature] Upgrade t1ha distribution - * [Feature] use_domain_sign_inbound - * [Feature] Use scores from maps if `symbols_set` is not defined - * [Fix] Add resolving version of radix map helper - * [Fix] Check URL before adding implicit prefix - * [Fix] Do not check pid/state when using PRNG - * [Fix] Fix CentOS logrotate script for systemd - * [Fix] Fix slash + dot in urls - * [Fix] Fix systemd version of the logrotate script - * [Fix] Propagate key when import implicit array from Lua - * [Fix] Strip spaces from map keys and values - * [Fix] Try to fix a specific case when processing milter protocol - * [Fix] Try to fix crash when a tcp connection cannot be set - * [Fix] Typo use_domain_local --> use_domain_sign_local - * [Fix] Various fixes to once_received module - * [Project] Store hits counters for map elements + * [Conf] Major stock config updates: + - Workers are now specified in a new format worker "type" { ... } + - Enable fuzzy worker to simplify local fuzzy storages configuration + - Bind all workers to localhost by default to avoid security flaws + * [Conf] Make more sane fuzzy_check default settings + * [CritFix] Fix ucl escape for bad symbols + * [Feature] Add failure symbol for AV module + * [Feature] Add lazy expiration mode for new classifier schema + * [Feature] Add preliminary version of maps stats plugin + * [Feature] Allow to block fuzzy requests from specific networks + * [Feature] Allow to change `expire` of live statistics + * [Feature] Distinguish AV failure from clean result + * [Feature] Further improvements of language detector by using khash + * [Feature] Further optimization of the lang_detection + * [Feature] Implement cluster-aware bayes expiry + * [Feature] Implement exclude patterns in rspamc + * [Feature] Implement glob maps in addition to regexp maps + * [Feature] Implement map statistics function for lua API + * [Feature] Implement stop symbols for Clickhouse collection + * [Feature] Support recipients separated by commas + * [Feature] Try harder to upload scripts to the Redis server + * [Feature] Upgrade t1ha distribution + * [Feature] use_domain_sign_inbound + * [Feature] Use scores from maps if `symbols_set` is not defined + * [Fix] Add resolving version of radix map helper + * [Fix] Check URL before adding implicit prefix + * [Fix] Do not check pid/state when using PRNG + * [Fix] Fix CentOS logrotate script for systemd + * [Fix] Fix slash + dot in urls + * [Fix] Fix systemd version of the logrotate script + * [Fix] Propagate key when import implicit array from Lua + * [Fix] Strip spaces from map keys and values + * [Fix] Try to fix a specific case when processing milter protocol + * [Fix] Try to fix crash when a tcp connection cannot be set + * [Fix] Typo use_domain_local --> use_domain_sign_local + * [Fix] Various fixes to once_received module + * [Project] Store hits counters for map elements 1.7.3: 10 Apr 2018 - * [CritFix] Plug bad memory leak in protocol reply - * [Feature] Add avx2 codec for base64 - * [Feature] Add method to receive all URL flags from Lua API - * [Feature] Allow to fold headers on stop characters - * [Feature] Allow to set lua_cpath from options - * [Feature] Allow to specify custom rejection message in milter - * [Feature] Deal with unnormalised Unicode obfuscation - * [Feature] Do not detect language twice for relative parts - * [Feature] Implement oversigning feature - * [Feature] Implement silent logging level to minimize noise in logs - * [Feature] Improve URL_IN_SUBJECT rule - * [Feature] Use hashing to reduce redis attack surface - * [Fix] Add oversigning for the most important headers - * [Fix] add 'rewrite subject' to History dropdown - * [Fix] Another fix in folding algorithm - * [Fix] Do not call multimap addr for parts of addr if filter is presented - * [Fix] Do not clean hostname on generic reset - * [Fix] Do not create pid file in no-fork mode - * [Fix] Fix fold_after case to preserve multiple spaces - * [Fix] Fix folding and folding tests - * [Fix] Fix hostname usage in milter mode - * [Fix] Fix lua RSA verify and its tests - * [Fix] Fix metadata exporter send_mail backend (#2124) - * [Fix] Fix processing of '\v' in libucl - * [Fix] Fix shemaless URLs detection - * [Fix] Fix support of multiple headers in sign_header - * [Fix] Fix usage of util.parse_mail_address - * [Fix] Fix weights of dynamic squeezed rules - * [Fix] Leak from bucket before checking the burst - * [Fix] Stop using own localtime as DST could be messy in many cases - * [Fix] Treat unnormalised URLs as obscured - * [Rework] Restore leaky bucket model in ratelimit plugin - * [WebUI] Add messages total to throughput summary - * [WebUI] Add symbols order selector to history - * [WebUI] Config: Load list on demand - * [WebUI] Fix modalBody for maps that appear more than once - * [WebUI] History: Fix Tooltips on paging, filtering and sorting - * [WebUI] Remove a previously-attached event handler - * [WebUI] Update D3 to v5.0.0 and jQuery to v3.3.1 + * [CritFix] Plug bad memory leak in protocol reply + * [Feature] Add avx2 codec for base64 + * [Feature] Add method to receive all URL flags from Lua API + * [Feature] Allow to fold headers on stop characters + * [Feature] Allow to set lua_cpath from options + * [Feature] Allow to specify custom rejection message in milter + * [Feature] Deal with unnormalised Unicode obfuscation + * [Feature] Do not detect language twice for relative parts + * [Feature] Implement oversigning feature + * [Feature] Implement silent logging level to minimize noise in logs + * [Feature] Improve URL_IN_SUBJECT rule + * [Feature] Use hashing to reduce redis attack surface + * [Fix] Add oversigning for the most important headers + * [Fix] add 'rewrite subject' to History dropdown + * [Fix] Another fix in folding algorithm + * [Fix] Do not call multimap addr for parts of addr if filter is presented + * [Fix] Do not clean hostname on generic reset + * [Fix] Do not create pid file in no-fork mode + * [Fix] Fix fold_after case to preserve multiple spaces + * [Fix] Fix folding and folding tests + * [Fix] Fix hostname usage in milter mode + * [Fix] Fix lua RSA verify and its tests + * [Fix] Fix metadata exporter send_mail backend (#2124) + * [Fix] Fix processing of '\v' in libucl + * [Fix] Fix shemaless URLs detection + * [Fix] Fix support of multiple headers in sign_header + * [Fix] Fix usage of util.parse_mail_address + * [Fix] Fix weights of dynamic squeezed rules + * [Fix] Leak from bucket before checking the burst + * [Fix] Stop using own localtime as DST could be messy in many cases + * [Fix] Treat unnormalised URLs as obscured + * [Rework] Restore leaky bucket model in ratelimit plugin + * [WebUI] Add messages total to throughput summary + * [WebUI] Add symbols order selector to history + * [WebUI] Config: Load list on demand + * [WebUI] Fix modalBody for maps that appear more than once + * [WebUI] History: Fix Tooltips on paging, filtering and sorting + * [WebUI] Remove a previously-attached event handler + * [WebUI] Update D3 to v5.0.0 and jQuery to v3.3.1 1.7.2: 23 Mar 2018 - * [Feature] Store emails in Clickhouse - * [Feature] Support single quotes in config - * [Feature] Use templates when publishing CH schema - * [Feature] Improve Docker image - * [Fix] Add rounding when printing a lot of FP variables - * [Fix] Allow to disable certain actions by assigning null to them - * [Fix] Disable results caching - * [Fix] Fix disabling of squeezed symbols - * [Fix] Fix scan time set - * [Fix] Rework logic of actions setting - * [Fix] Try to fix various Lua stack issues - * [WebUI] Add link tag for favicon.ico - * [WebUI] Display hostname:port/path in the page title + * [Feature] Store emails in Clickhouse + * [Feature] Support single quotes in config + * [Feature] Use templates when publishing CH schema + * [Feature] Improve Docker image + * [Fix] Add rounding when printing a lot of FP variables + * [Fix] Allow to disable certain actions by assigning null to them + * [Fix] Disable results caching + * [Fix] Fix disabling of squeezed symbols + * [Fix] Fix scan time set + * [Fix] Rework logic of actions setting + * [Fix] Try to fix various Lua stack issues + * [WebUI] Add link tag for favicon.ico + * [WebUI] Display hostname:port/path in the page title 1.7.1: 20 Mar 2018 - * [CritFix] Fix lowercase comparison - * [CritFix] Timezone defines seconds WEST UTC not East - * [Feature] Add filename to log format - * [Feature] Add lua rules squeezing - * [Feature] Add related symbols analysis to rspamd_stats - * [Feature] Remove upstream `X-Spam: Yes` header by default - * [Feature] rspamd_stats: Output progress info on STDERR - * [Feature] Whitelist for emails module - * [Fix] Do not allow dependencies on self - * [Fix] Do not cache metric result - * [Fix] Do not trust all issuers as a client certificate - * [Fix] Fix dependencies in lua squeeze - * [Fix] Fix enabling/disabling squeezed rules - * [Fix] Fix enabling/disabling symbols - * [Fix] Fix external dependencies - * [Fix] Fix processing of a single compressed file - * [Fix] Fix some typos - * [Fix] Fix various modules in case of empty message - * [Fix] Handle callbacks that returns table of options - * [Fix] Improve cached action interaction - * [Fix] Make dynamic conf more NaN aware - * [Fix] Never hide actions from WebUI `configuration` tab - * [Project] Implementation of Lua rules squeezing + * [CritFix] Fix lowercase comparison + * [CritFix] Timezone defines seconds WEST UTC not East + * [Feature] Add filename to log format + * [Feature] Add lua rules squeezing + * [Feature] Add related symbols analysis to rspamd_stats + * [Feature] Remove upstream `X-Spam: Yes` header by default + * [Feature] rspamd_stats: Output progress info on STDERR + * [Feature] Whitelist for emails module + * [Fix] Do not allow dependencies on self + * [Fix] Do not cache metric result + * [Fix] Do not trust all issuers as a client certificate + * [Fix] Fix dependencies in lua squeeze + * [Fix] Fix enabling/disabling squeezed rules + * [Fix] Fix enabling/disabling symbols + * [Fix] Fix external dependencies + * [Fix] Fix processing of a single compressed file + * [Fix] Fix some typos + * [Fix] Fix various modules in case of empty message + * [Fix] Handle callbacks that returns table of options + * [Fix] Improve cached action interaction + * [Fix] Make dynamic conf more NaN aware + * [Fix] Never hide actions from WebUI `configuration` tab + * [Project] Implementation of Lua rules squeezing 1.7.0: 12 Mar 2018 - * [Conf] Add bayes_expiry as explicit module - * [Conf] Adjust names and weights for neural network plugin - * [Conf] Change updates url - * [Conf] Default statistics is stored in Redis now - * [Conf] Disable fann_redis module by default - * [Conf] Fix default elastic configuration - * [Conf] Fix double quote position - * [Conf] Massive config rework for new structure of symbols and scores - * [Conf] Rename Rambler BLs as they are now Rspamd's ones - * [Conf] Use dedicated rspamd.com subdomains - * [Conf] Use more data from rspamd.com fuzzy storage - * [CritFix] Add sanity guards for badly broken HTML - * [CritFix] Another errors path handling fix - * [CritFix] Another portion of tokenization fixes - * [CritFix] Do not send reject messages after set reply - * [CritFix] Fix ARC chain verification - * [CritFix] Fix crash in milter errors handler - * [CritFix] Fix memory leak in spf caching logic - * [CritFix] Fix milter commands pipelining - * [CritFix] Fix newlines detection - * [CritFix] Fix semicolons parsing in the content type - * [CritFix] Plug memory leak in zstd protocol compression - * [Feature] Add ability to match score in force_actions module - * [Feature] Add aes-rng PRF to libottery - * [Feature] Add 'composites' debug module - * [Feature] Add concept of experimental modules - * [Feature] Add DKIM trace symbol - * [Feature] Add EBL to the default config - * [Feature] Add expected ip check for emails plugin - * [Feature] Add framework to manage Redis scripts - * [Feature] Add framing for the new reputation generic plugin - * [Feature] Add function to show plugins stat - * [Feature] Add gzip compression support for clickhouse module - * [Feature] Add gzip compression support for rspamd controller - * [Feature] Add gzip support when sending lua http requests - * [Feature] Add json output for rspamd_stats - * [Feature] Add method to do a synchronous Redis connection - * [Feature] Add method to get all content-type attributes in Lua - * [Feature] Add `-m` flag to configdump to show modules states - * [Feature] Add mime types to extensions map - * [Feature] Add more features to rescore utility - * [Feature] Add more gtube like patterns to test other spam actions - * [Feature] Add more metafunctions, improve logging - * [Feature] Add more text attributes - * [Feature] Add new configwizard command to rspamadm - * [Feature] Add new tooling for stats conversation - * [Feature] Add old groups migration tool - * [Feature] Add plugins state variable - * [Feature] Add preliminary ecdsa keys support in DKIM - * [Feature] Add preliminary support of idempotent symbols - * [Feature] Add Redis server wizard - * [Feature] Add routine to convert old style stats to a new one - * [Feature] Add some sanity checks for actions and controller - * [Feature] Add statistic convertation module to configwizard - * [Feature] Add sugestions logic to mempool allocator - * [Feature] Add support of config transform in Lua - * [Feature] Add timeout to rspamc when doing corpus test - * [Feature] Add tooling to convert bayes schemas - * [Feature] Add torch conditional to configuration - * [Feature] Add torch-decisiontree package - * [Feature] Add torch-optim contrib package - * [Feature] Add TTL autodetection - * [Feature] Add urls reputation to the reputation framework - * [Feature] Allow floating and negative values in expressions limits - * [Feature] Allow multiple CTs in full extensions map - * [Feature] Allow multiple fann rules - * [Feature] Allow randomly select User-Agent from a list - * [Feature] Allow rspamadm commands to export methods in Lua - * [Feature] Allow rule specific min_bytes in fuzzy check - * [Feature] Allow to adjust symbols scores from Lua - * [Feature] Allow to attach stat signature to messages - * [Feature] Allow to change SMTP from via milter headers - * [Feature] Allow to configure monitored - * [Feature] Allow to create directories in Lua API - * [Feature] Allow to disable torch and skip train samples for ANN - * [Feature] Allow to discard messages dynamically - * [Feature] Allow to enable/disable languages from the detector - * [Feature] Allow to generate DKIM keys from rspamadm API - * [Feature] Allow to get CPU flags from Lua - * [Feature] Allow to have high precision timestamps in logs - * [Feature] Allow to insert headers into specific position - * [Feature] Allow to limit redirector requests per task - * [Feature] Allow to load and use dynamic ANNs with torch - * [Feature] Allow to quarantine rejected messages using milter interface - * [Feature] Allow to receive signing keys from mempool vars - * [Feature] Allow to reserve elements in libucl - * [Feature] Allow to reuse signal handlers chains - * [Feature] Allow to set custom mempool variables from settings - * [Feature] Allow to set headers from settings - * [Feature] Allow to set Settings-Id for all connections - * [Feature] Allow to skip real action and add a header instead - * [Feature] Allow to skip specific hashes in fuzzy storage - * [Feature] Allow to spawn asynchronous processes from Lua - * [Feature] Allow to specify number of threads for ANN learning - * [Feature] Allow to use global lua maps in settings - * [Feature] Allow to use postfilters in composites - * [Feature] Allow to verify signatures from HTTP headers in maps - * [Feature] Antivirus: ordered pattern matches - * [Feature] Authentication-Results: support hiding usernames - * [Feature] Automatically create tables in clickhouse - * [Feature] Catch next-to-last bad extension - * [Feature] Check cached maps more frequently - * [Feature] Check groups sanity - * [Feature] Deal with obscured URLs with @ symbols - * [Feature] Enhance task:store_in_file method - * [Feature] Export password encryption routines to Redis - * [Feature] Filter nan and inf when adding scores - * [Feature] Finalize 7zip files support - * [Feature] Further improvements in language detection - * [Feature] Further improvements in language detection algorithm - * [Feature] Generic key name expansion for Redis keys - * [Feature] Hash whitelist for fuzzy_check - * [Feature] Implement bayes signatures storage - * [Feature] Implement buckets for Redis backend - * [Feature] Implement DKIM reputation adjustments - * [Feature] Implement forked workers children monitoring - * [Feature] Implement headers flags in mime parser - * [Feature] Implement l1/l2 regularization against the current weights - * [Feature] Implement manual ANN train mode - * [Feature] Implement per-user ANN support - * [Feature] Implement torch based ANN learning - * [Feature] Implement upstreams logic for clickhouse exporter - * [Feature] Import torch to Rspamd... - * [Feature] Improve allocation policy when interacting with Lua - * [Feature] Improve Lua/C interaction in history_redis - * [Feature] Improve multiple fuzzy results combining - * [Feature] Improve parsing of DKIM keys: parse algorithm - * [Feature] Improve subprocesses termination handle - * [Feature] Improve symbol type parsing in Lua API - * [Feature] Metadata Exporter: e-Mail Alerts: support multiple recipients; alerting senders/recipients/users (#1600) - * [Feature] Milter headers: support adding/removing arbitrary headers from config - * [Feature] More metatokens - * [Feature] Multimap: checking of symbol options - * [Feature] Multimap: template URL filter - * [Feature] New bayes expiry plugin - * [Feature] Periodically save rspamd stats to disk - * [Feature] Preliminary import of the elasticsearch module - * [Feature] Ratelimit: allow full addresses in whitelisted_rcpts - * [Feature] Ratelimit: support fetching limits from Redis - * [Feature] RBL: received: filtering by position & flags - * [Feature] Read global maps for lua - * [Feature] Redis settings: support checking multiple keys - * [Feature] Rework fann plugin to be a normal post-filter - * [Feature] Rework logging configuration for rspamadm case - * [Feature] Rework short hashes generation to avoid FP - * [Feature] Save real ucl types when exporting to Lua - * [Feature] Set TCP_NODELAY for milter sockets - * [Feature] Setup DKIM signing from configwizard - * [Feature] Skip certain symbols from ANN classify - * [Feature] Store plugins state - * [Feature] Support etag for HTTP maps - * [Feature] Support Expires header when using HTTP maps - * [Feature] Support sending given header multiple times in lua_http - * [Feature] Support sha512 in DKIM signatures - * [Feature] Try to detect HTML messages better - * [Feature] Use array instead of queue to reduce memory fragmentation - * [Feature] Use controller port by default when connecting to local IP - * [Feature] Use rdtsc where possible - * [Fix] Actively load skip hashes map in fuzzy storage - * [Fix] Add another workaround to display history properly - * [Fix] Add definition for old glib compatibility method - * [Fix] Add missing rspamadm control options to help - * [Fix] Add workaround for IPv6 in sendmail - * [Fix] Add workaround for system with non-XSI compatible tzset - * [Fix] Allow oversigning in DKIM signatures - * [Fix] Allow to check negative scores in force_actions - * [Fix] Allow to have negative actions limits - * [Fix] Allow to set any layers number for fann rules - * [Fix] Another fix for rdtcs - * [Fix] Another fix to lua xmlrpc - * [Fix] Another try to deal with #1998 - * [Fix] Another try to fix #1998 - * [Fix] Another try to fix threading in torch - * [Fix] Apply language detection when adding fuzzy hashes - * [Fix] ARC: Fix Lua 5.3 compatibility; timestamp should be integer - * [Fix] Authentication Results: Fix SPF smtp.mail_from - * [Fix] Auth-Results: Multiple DKIM signatures - * [Fix] Avoid changing content-transfer-encoding header's value - * [Fix] Better handling of the legacy protocol - * [Fix] Check decoded headers sanity (e.g. by excluding \0) - * [Fix] Check for magic when checking for an archive - * [Fix] Cleanup mess with groups - * [Fix] Clickhouse: Insertion in the symbols table - * [Fix] Crash in URL processing - * [Fix] Deal with another case when processing exceptions - * [Fix] Deal with deeply nested messages more aggressively - * [Fix] Deal with nan and inf encoding in json/ucl - * [Fix] Deal with non-key arguments in lua_redis.exec_script - * [Fix] Deal with unknown weight - * [Fix] Deal with URLs with no slashes after protocol - * [Fix] Deal with URLs wrapped in [] in text parts - * [Fix] Deal with zero scores symbols - * [Fix] Default monitoring domain for surbl plugin - * [Fix] Delay upstream re-resolving when one upstream is defined - * [Fix] Detection of maillist optimized and fixed - * [Fix] DKIM signing: allow for auth_only to be false - * [Fix] DMARC: require report_settings for sending reports only - * [Fix] Do not allow garbadge when checking url domain - * [Fix] Do not cache SPF records with PTR elements - * [Fix] Do not constantly re-resolve failed upstreams with a single element - * [Fix] Do not crash if no words defined - * [Fix] Do not crash on empty subtype - * [Fix] Do not expose spamtrap messages to SMTP reply - * [Fix] Do not fail rbl plugin when there are no received or emails - * [Fix] Do not ignore short words - * [Fix] Do not include idempotent/nostat symbols to checksum - * [Fix] Do not override groups when converting metrics - * [Fix] Do not override unix socket group when group comes before owner - * [Fix] Do not skip the last character - * [Fix] Do not spawn too many workers by default - * [Fix] Do not stop monitored on dns errors - * [Fix] Do not stop parsing headers on bad IP header - * [Fix] Do not strip last character in the last word - * [Fix] Do not treat script content as text - * [Fix] Do not try to connect to non-supported addresses - * [Fix] Do not try to dereference last character - * [Fix] Do not try to sign unknown domains - * [Fix] Don't use whitelist/greylist maps as regexp, but as map - * [Fix] Erase unknown HTML entities - * [Fix] Exim Received header protocol parsing - * [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false - * [Fix] Fix a lot of FP in chartable in mixed languages - * [Fix] Fix ANN checks - * [Fix] Fix ANN loading logic - * [Fix] Fix another tokenization issue - * [Fix] Fix autolearn parameters reading - * [Fix] Fix bad archive characters stripping - * [Fix] Fix bad extension check - * [Fix] Fix bayes schema conversion - * [Fix] Fix blacklists and DMARC in whitelist - * [Fix] Fix brain-damaged torch build system - * [Fix] Fix build on FreeBSD - * [Fix] Fix clickhouse exporter - * [Fix] Fix clickhouse schema - * [Fix] Fix comparision - * [Fix] Fix composites processing - * [Fix] Fix connecting to a unix socket in rspamadm statconvert - * [Fix] Fix couple of warnings - * [Fix] Fix crashes in the rspamd_control path - * [Fix] Fix deletion from hash - * [Fix] Fix DKIM forgeries via multiple headers - * [Fix] FIx dynamic conf plugin - * [Fix] Fix emails detection - * [Fix] Fix empty headers simple canonicalization - * [Fix] Fix empty threshold check in greylisting module - * [Fix] Fix encrypted legacy reply in fuzzy storage - * [Fix] Fix enormous scores for R_WHITE_ON_WHITE - * [Fix] Fix exceptions list in surbl - * [Fix] Fix *_EXCESS_BASE64 rules - * [Fix] Fix expire rounding - * [Fix] Fix extra hits in PCRE mode for regular expressions - * [Fix] Fix format strings - * [Fix] Fix get_content method - * [Fix] Fix groups override when defining symbols - * [Fix] Fix learned count in new schema - * [Fix] Fix learn errors propagation - * [Fix] Fix loading of per-user redis backend for statistics - * [Fix] Fix logging buffer corruption in case of repeated messages - * [Fix] Fix lua cached elements invalidation - * [Fix] Fix merging of the implicit arrays - * [Fix] Fix mime_types scoring - * [Fix] Fix multiple headers in DKIM headers list - * [Fix] Fix null callee case in clang plugin - * [Fix] Fix obscured url in format user@@example.com - * [Fix] Fix parsing of the per-user script - * [Fix] Fix priorities in rspamd_update, disable rules execution - * [Fix] Fix processing of closed tags - * [Fix] Fix processing of idempotent rules when autolearn fails - * [Fix] Fix processing of multipart parts with no headers - * [Fix] Fix processing of skip-hashes in fuzzy storage - * [Fix] Fix PTR processing in SPF - * [Fix] Fix pushing country to clickhouse asn table - * [Fix] Fix random forests module - * [Fix] Fix real IP parsing for some strange Exim received - * [Fix] Fix Redis timeout setup - * [Fix] Fix reload crash when hyperscan is enabled - * [Fix] Fix reusing of redis connection after exec - * [Fix] Fix sanity checks on macro value - * [Fix] Fix setting of path and cpath for Lua - * [Fix] Fix setting of signals when spawning a thread - * [Fix] Fix text splitting: stack overflow (too many captures) - * [Fix] Fix ticks processing - * [Fix] Fix upstream addrs updating - * [Fix] Fix urls/emails distinguishing found in queries - * [Fix] Fix user settings check - * [Fix] Fix variable increment - * [Fix] Fix various issues in stat_convert - * [Fix] F-PROT Antivirus infection string for all known occurences - * [Fix] F-PROT Antivirus: only check return code to determine infection - * [Fix] Further fixes around floating point expressions - * [Fix] Further fixes to ANN module - * [Fix] Further fixes to rescore tool - * [Fix] Further fixes to support ES 6 - * [Fix] Further tokenization fixes - * [Fix] Greylisting set phase is not idempotent - * [Fix] Handle proxy copy errors - * [Fix] Header checks: Fix get_raw_header method - * [Fix] Header checks: REPLYTO_UNPARSEABLE rule - * [Fix] Kill spawned processes on termination - * [Fix] Load skip map from all processes as shared cache is unavailable - * [Fix] Lowercase HTTP headers to make them searchable from Lua - * [Fix] Lowercase words - * [Fix] Lua_http: freeing - * [Fix] Lua: lpeg to be loaded with rspamd_lua_add_preload, to avoid "rspamd_config_read: rcl parse error: cannot init lua file […] module 'lpeg' not found" - * [Fix] Map absence is not an error - * [Fix] Metadata exporter: check IP sanity - * [Fix] Milter headers: custom headers: removing headers - * [Fix] Milter headers: skip_local / skip_authenticated settings - * [Fix] Milter headers: X-Spamd-Result header if X-Virus ran first - * [Fix] mime_types: fix next-to-last extension length check - * [Fix] More hacks to deal with old configs - * [Fix] Move composites second pass to the dedicated stage - * [Fix] Multimap: received: filtering of artificial header - * [Fix] Multiple fixes in torch based ANN plugins - * [Fix] Once more (#1879) fix bad extension check - * [Fix] Optimize rspamd_fstring_t reallocations - * [Fix] options.local_networks setting - * [Fix] Parse HREF urls without explicit prefix - * [Fix] Plan new event on HTTP errors - * [Fix] Plug another possible memory leak - * [Fix] Plug memory leak - * [Fix] Plug memory leak in lua_tcp - * [Fix] Plug memory leak when setting email addresses from Lua - * [Fix] Propagate learn/stat errors more precisely - * [Fix] Ratelimit: fix whitelisted_rcpts matching - * [Fix] Ratelimit: lowercase email addresses - * [Fix] RBL: received: deal with missing data (#1965) - * [Fix] Rebalance and slightly rework MX check plugin - * [Fix] Redis key expansion: EVAL: deal with strings - * [Fix] Redis script loading in DMARC; URL tags; URL reputation - * [Fix] Reject invalid bh for DKIM signatures earlier - * [Fix] Relax pem signature detection - * [Fix] Relax unicode properties requirements for chartable module - * [Fix] Remove extra noise from dkim and arc signing - * [Fix] Remove hop-by-hop headers in proxy - * [Fix] Remove incorrect method `task:set_metric_subject` - * [Fix] Replace space like characters in headers with plain space - * [Fix] Restore old style ratelimits support - * [Fix] Rework elasticsearch plugin - * [Fix] Rewriting subjects via force actions module - * [Fix] RPM postinstall - * [Fix] Sanitize IP in history redis - * [Fix] Select the correct signature when doing simple canon - * [Fix] Set CLOEXEC flag on files opened - * [Fix] Setting check_local / check_authed in plugins (#1954) - * [Fix] Settings: avoid checking invalid IP (#1981) - * [Fix] Settings: header: deal with multiple settings (#1988) - * [Fix] Skip checks if both extensions are not bad - * [Fix] Skip nostat tokens when get number of tokens - * [Fix] Some more fixes towards emails detection - * [Fix] SpamAssassin: Fail check_freemail_header if regexp didn't match - * [Fix] Stop using of g_slice... - * [Fix] Switch rspamadm logging to message level - * [Fix] Symbol 'FANNR_SPAM' has its score defined.. - * [Fix] Table parameter for rspamd_config:add_doc() - * [Fix] Treat 'rewrite subject' as spam action - * [Fix] Try harder in passing IPv6 addresses - * [Fix] Try harder to find rfc822 notifications - * [Fix] Try harder to find urls - * [Fix] Use decoded values when parsing mime addresses - * [Fix] Use full URL when making an HTTP request - * [Fix] Use greylisting threshold in greylisting module - * [Fix] Use n_words attribute from ngramms - * [Fix] Use raw urls when sending requests to redirector - * [Fix] Use the right boolean operator on error check - * [Fix] Use weight from map for fuzzy scoring - * [Fix] Various fixes to elastic plugin - * [Fix] Various fixes to fann_redis instantiation - * [Fix] Various improvements in language detection - * [Fix] Virus infection string for F-PROT Antivirus - * [Fix] Virus infetction string for F-PROT Antivirus - * [Fix] WebUI: use relative path for savemap (#1943) - * [Fix] WHITE_ON_WHITE: Ensure score is matched to part that fired the rule - * [Fix] Write configuration changes as UCL config - * [Project] Add detection logic for words - * [Project] Add fast debug logging infrastructure - * [Project] Add more flags to languages - * [Project] Add n-gramms data files - * [Project] Add ngramms frequencies detector - * [Project] Add random words selection logic - * [Project] Add unigramms to language detection as well - * [Project] Convert all C modules to fast debug infrastructure - * [Project] Detect some languages based on unicode script - * [Project] Enable fast debug lookup for some modules - * [Project] Enable language detector init in scanner workers - * [Project] Further improvements to language detector - * [Project] Implement logic of ngramms application - * [Project] Improve weighting in lang_detection - * [Project] Initialize language detector - * [Project] Preliminary version of ngramms based language detector - * [Project] Preliminary version of the new stat_convert - * [Project] Remove old language detector - * [Project] Rework language detection ngramms structure - * [Project] Start language detection project - * [Project] Start rework of language detection to improve quality - * [Project] Use fast debug logging check - * [Rework] Add frame for new reputation based IP score module - * [Rework] Continue stat_convert rework task - * [Rework] Implement new version of fuzzy replies - * [Rework] Improve readability of xmlrpc API - * [Rework] Kill metrics!11 - * [Rework] Ratelimit module - * [Rework] Rename fann_redis to neural plugin - * [Rework] Reorganize mime_types module - * [Rework] Rework rescore utility - * [Rework] Rewrite model and learning logic for rescore - * [Rework] Run post-loads when all initialization is completed - * [Rework] Simplify lua path initialization - * [Rework] Start major stat_convert rework - * [Rework] Start mempool fragmentation reduce project - * [Rework] Start moving of fann redis to torch - * [Rework] Stop embedding rspamadm scripts into C - * [Rework] Use floating point arithmetics in Rspamd expressions - * [Rework] Use frequencies distribution in language detector - * [Rules] Penalise R_BAD_CTE_7BIT for utf8 messages - * [WebUI] Compact graph selectors - * [WebUI] Escape strings inside HTML in history - * [WebUI] Fix message count in throughput summary (#1724) - * [WebUI] Fix NaNs display on Throughput graph - * [WebUI] Migrate widgets to D3 v4 - * [WebUI] Restore passwordless login support (#2003) - * [WebUI] Show symbol descriptions as tooltips in history - * [WebUI] Stop using commas in pie chart tooltips - * [WebUI] Update D3 and jQuery - * [WebUI] Update D3Evolution 1.0.0 -> 1.1.0 + * [Conf] Add bayes_expiry as explicit module + * [Conf] Adjust names and weights for neural network plugin + * [Conf] Change updates url + * [Conf] Default statistics is stored in Redis now + * [Conf] Disable fann_redis module by default + * [Conf] Fix default elastic configuration + * [Conf] Fix double quote position + * [Conf] Massive config rework for new structure of symbols and scores + * [Conf] Rename Rambler BLs as they are now Rspamd's ones + * [Conf] Use dedicated rspamd.com subdomains + * [Conf] Use more data from rspamd.com fuzzy storage + * [CritFix] Add sanity guards for badly broken HTML + * [CritFix] Another errors path handling fix + * [CritFix] Another portion of tokenization fixes + * [CritFix] Do not send reject messages after set reply + * [CritFix] Fix ARC chain verification + * [CritFix] Fix crash in milter errors handler + * [CritFix] Fix memory leak in spf caching logic + * [CritFix] Fix milter commands pipelining + * [CritFix] Fix newlines detection + * [CritFix] Fix semicolons parsing in the content type + * [CritFix] Plug memory leak in zstd protocol compression + * [Feature] Add ability to match score in force_actions module + * [Feature] Add aes-rng PRF to libottery + * [Feature] Add 'composites' debug module + * [Feature] Add concept of experimental modules + * [Feature] Add DKIM trace symbol + * [Feature] Add EBL to the default config + * [Feature] Add expected ip check for emails plugin + * [Feature] Add framework to manage Redis scripts + * [Feature] Add framing for the new reputation generic plugin + * [Feature] Add function to show plugins stat + * [Feature] Add gzip compression support for clickhouse module + * [Feature] Add gzip compression support for rspamd controller + * [Feature] Add gzip support when sending lua http requests + * [Feature] Add json output for rspamd_stats + * [Feature] Add method to do a synchronous Redis connection + * [Feature] Add method to get all content-type attributes in Lua + * [Feature] Add `-m` flag to configdump to show modules states + * [Feature] Add mime types to extensions map + * [Feature] Add more features to rescore utility + * [Feature] Add more gtube like patterns to test other spam actions + * [Feature] Add more metafunctions, improve logging + * [Feature] Add more text attributes + * [Feature] Add new configwizard command to rspamadm + * [Feature] Add new tooling for stats conversation + * [Feature] Add old groups migration tool + * [Feature] Add plugins state variable + * [Feature] Add preliminary ecdsa keys support in DKIM + * [Feature] Add preliminary support of idempotent symbols + * [Feature] Add Redis server wizard + * [Feature] Add routine to convert old style stats to a new one + * [Feature] Add some sanity checks for actions and controller + * [Feature] Add statistic convertation module to configwizard + * [Feature] Add sugestions logic to mempool allocator + * [Feature] Add support of config transform in Lua + * [Feature] Add timeout to rspamc when doing corpus test + * [Feature] Add tooling to convert bayes schemas + * [Feature] Add torch conditional to configuration + * [Feature] Add torch-decisiontree package + * [Feature] Add torch-optim contrib package + * [Feature] Add TTL autodetection + * [Feature] Add urls reputation to the reputation framework + * [Feature] Allow floating and negative values in expressions limits + * [Feature] Allow multiple CTs in full extensions map + * [Feature] Allow multiple fann rules + * [Feature] Allow randomly select User-Agent from a list + * [Feature] Allow rspamadm commands to export methods in Lua + * [Feature] Allow rule specific min_bytes in fuzzy check + * [Feature] Allow to adjust symbols scores from Lua + * [Feature] Allow to attach stat signature to messages + * [Feature] Allow to change SMTP from via milter headers + * [Feature] Allow to configure monitored + * [Feature] Allow to create directories in Lua API + * [Feature] Allow to disable torch and skip train samples for ANN + * [Feature] Allow to discard messages dynamically + * [Feature] Allow to enable/disable languages from the detector + * [Feature] Allow to generate DKIM keys from rspamadm API + * [Feature] Allow to get CPU flags from Lua + * [Feature] Allow to have high precision timestamps in logs + * [Feature] Allow to insert headers into specific position + * [Feature] Allow to limit redirector requests per task + * [Feature] Allow to load and use dynamic ANNs with torch + * [Feature] Allow to quarantine rejected messages using milter interface + * [Feature] Allow to receive signing keys from mempool vars + * [Feature] Allow to reserve elements in libucl + * [Feature] Allow to reuse signal handlers chains + * [Feature] Allow to set custom mempool variables from settings + * [Feature] Allow to set headers from settings + * [Feature] Allow to set Settings-Id for all connections + * [Feature] Allow to skip real action and add a header instead + * [Feature] Allow to skip specific hashes in fuzzy storage + * [Feature] Allow to spawn asynchronous processes from Lua + * [Feature] Allow to specify number of threads for ANN learning + * [Feature] Allow to use global lua maps in settings + * [Feature] Allow to use postfilters in composites + * [Feature] Allow to verify signatures from HTTP headers in maps + * [Feature] Antivirus: ordered pattern matches + * [Feature] Authentication-Results: support hiding usernames + * [Feature] Automatically create tables in clickhouse + * [Feature] Catch next-to-last bad extension + * [Feature] Check cached maps more frequently + * [Feature] Check groups sanity + * [Feature] Deal with obscured URLs with @ symbols + * [Feature] Enhance task:store_in_file method + * [Feature] Export password encryption routines to Redis + * [Feature] Filter nan and inf when adding scores + * [Feature] Finalize 7zip files support + * [Feature] Further improvements in language detection + * [Feature] Further improvements in language detection algorithm + * [Feature] Generic key name expansion for Redis keys + * [Feature] Hash whitelist for fuzzy_check + * [Feature] Implement bayes signatures storage + * [Feature] Implement buckets for Redis backend + * [Feature] Implement DKIM reputation adjustments + * [Feature] Implement forked workers children monitoring + * [Feature] Implement headers flags in mime parser + * [Feature] Implement l1/l2 regularization against the current weights + * [Feature] Implement manual ANN train mode + * [Feature] Implement per-user ANN support + * [Feature] Implement torch based ANN learning + * [Feature] Implement upstreams logic for clickhouse exporter + * [Feature] Import torch to Rspamd... + * [Feature] Improve allocation policy when interacting with Lua + * [Feature] Improve Lua/C interaction in history_redis + * [Feature] Improve multiple fuzzy results combining + * [Feature] Improve parsing of DKIM keys: parse algorithm + * [Feature] Improve subprocesses termination handle + * [Feature] Improve symbol type parsing in Lua API + * [Feature] Metadata Exporter: e-Mail Alerts: support multiple recipients; alerting senders/recipients/users (#1600) + * [Feature] Milter headers: support adding/removing arbitrary headers from config + * [Feature] More metatokens + * [Feature] Multimap: checking of symbol options + * [Feature] Multimap: template URL filter + * [Feature] New bayes expiry plugin + * [Feature] Periodically save rspamd stats to disk + * [Feature] Preliminary import of the elasticsearch module + * [Feature] Ratelimit: allow full addresses in whitelisted_rcpts + * [Feature] Ratelimit: support fetching limits from Redis + * [Feature] RBL: received: filtering by position & flags + * [Feature] Read global maps for lua + * [Feature] Redis settings: support checking multiple keys + * [Feature] Rework fann plugin to be a normal post-filter + * [Feature] Rework logging configuration for rspamadm case + * [Feature] Rework short hashes generation to avoid FP + * [Feature] Save real ucl types when exporting to Lua + * [Feature] Set TCP_NODELAY for milter sockets + * [Feature] Setup DKIM signing from configwizard + * [Feature] Skip certain symbols from ANN classify + * [Feature] Store plugins state + * [Feature] Support etag for HTTP maps + * [Feature] Support Expires header when using HTTP maps + * [Feature] Support sending given header multiple times in lua_http + * [Feature] Support sha512 in DKIM signatures + * [Feature] Try to detect HTML messages better + * [Feature] Use array instead of queue to reduce memory fragmentation + * [Feature] Use controller port by default when connecting to local IP + * [Feature] Use rdtsc where possible + * [Fix] Actively load skip hashes map in fuzzy storage + * [Fix] Add another workaround to display history properly + * [Fix] Add definition for old glib compatibility method + * [Fix] Add missing rspamadm control options to help + * [Fix] Add workaround for IPv6 in sendmail + * [Fix] Add workaround for system with non-XSI compatible tzset + * [Fix] Allow oversigning in DKIM signatures + * [Fix] Allow to check negative scores in force_actions + * [Fix] Allow to have negative actions limits + * [Fix] Allow to set any layers number for fann rules + * [Fix] Another fix for rdtcs + * [Fix] Another fix to lua xmlrpc + * [Fix] Another try to deal with #1998 + * [Fix] Another try to fix #1998 + * [Fix] Another try to fix threading in torch + * [Fix] Apply language detection when adding fuzzy hashes + * [Fix] ARC: Fix Lua 5.3 compatibility; timestamp should be integer + * [Fix] Authentication Results: Fix SPF smtp.mail_from + * [Fix] Auth-Results: Multiple DKIM signatures + * [Fix] Avoid changing content-transfer-encoding header's value + * [Fix] Better handling of the legacy protocol + * [Fix] Check decoded headers sanity (e.g. by excluding \0) + * [Fix] Check for magic when checking for an archive + * [Fix] Cleanup mess with groups + * [Fix] Clickhouse: Insertion in the symbols table + * [Fix] Crash in URL processing + * [Fix] Deal with another case when processing exceptions + * [Fix] Deal with deeply nested messages more aggressively + * [Fix] Deal with nan and inf encoding in json/ucl + * [Fix] Deal with non-key arguments in lua_redis.exec_script + * [Fix] Deal with unknown weight + * [Fix] Deal with URLs with no slashes after protocol + * [Fix] Deal with URLs wrapped in [] in text parts + * [Fix] Deal with zero scores symbols + * [Fix] Default monitoring domain for surbl plugin + * [Fix] Delay upstream re-resolving when one upstream is defined + * [Fix] Detection of maillist optimized and fixed + * [Fix] DKIM signing: allow for auth_only to be false + * [Fix] DMARC: require report_settings for sending reports only + * [Fix] Do not allow garbadge when checking url domain + * [Fix] Do not cache SPF records with PTR elements + * [Fix] Do not constantly re-resolve failed upstreams with a single element + * [Fix] Do not crash if no words defined + * [Fix] Do not crash on empty subtype + * [Fix] Do not expose spamtrap messages to SMTP reply + * [Fix] Do not fail rbl plugin when there are no received or emails + * [Fix] Do not ignore short words + * [Fix] Do not include idempotent/nostat symbols to checksum + * [Fix] Do not override groups when converting metrics + * [Fix] Do not override unix socket group when group comes before owner + * [Fix] Do not skip the last character + * [Fix] Do not spawn too many workers by default + * [Fix] Do not stop monitored on dns errors + * [Fix] Do not stop parsing headers on bad IP header + * [Fix] Do not strip last character in the last word + * [Fix] Do not treat script content as text + * [Fix] Do not try to connect to non-supported addresses + * [Fix] Do not try to dereference last character + * [Fix] Do not try to sign unknown domains + * [Fix] Don't use whitelist/greylist maps as regexp, but as map + * [Fix] Erase unknown HTML entities + * [Fix] Exim Received header protocol parsing + * [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false + * [Fix] Fix a lot of FP in chartable in mixed languages + * [Fix] Fix ANN checks + * [Fix] Fix ANN loading logic + * [Fix] Fix another tokenization issue + * [Fix] Fix autolearn parameters reading + * [Fix] Fix bad archive characters stripping + * [Fix] Fix bad extension check + * [Fix] Fix bayes schema conversion + * [Fix] Fix blacklists and DMARC in whitelist + * [Fix] Fix brain-damaged torch build system + * [Fix] Fix build on FreeBSD + * [Fix] Fix clickhouse exporter + * [Fix] Fix clickhouse schema + * [Fix] Fix comparision + * [Fix] Fix composites processing + * [Fix] Fix connecting to a unix socket in rspamadm statconvert + * [Fix] Fix couple of warnings + * [Fix] Fix crashes in the rspamd_control path + * [Fix] Fix deletion from hash + * [Fix] Fix DKIM forgeries via multiple headers + * [Fix] FIx dynamic conf plugin + * [Fix] Fix emails detection + * [Fix] Fix empty headers simple canonicalization + * [Fix] Fix empty threshold check in greylisting module + * [Fix] Fix encrypted legacy reply in fuzzy storage + * [Fix] Fix enormous scores for R_WHITE_ON_WHITE + * [Fix] Fix exceptions list in surbl + * [Fix] Fix *_EXCESS_BASE64 rules + * [Fix] Fix expire rounding + * [Fix] Fix extra hits in PCRE mode for regular expressions + * [Fix] Fix format strings + * [Fix] Fix get_content method + * [Fix] Fix groups override when defining symbols + * [Fix] Fix learned count in new schema + * [Fix] Fix learn errors propagation + * [Fix] Fix loading of per-user redis backend for statistics + * [Fix] Fix logging buffer corruption in case of repeated messages + * [Fix] Fix lua cached elements invalidation + * [Fix] Fix merging of the implicit arrays + * [Fix] Fix mime_types scoring + * [Fix] Fix multiple headers in DKIM headers list + * [Fix] Fix null callee case in clang plugin + * [Fix] Fix obscured url in format user@@example.com + * [Fix] Fix parsing of the per-user script + * [Fix] Fix priorities in rspamd_update, disable rules execution + * [Fix] Fix processing of closed tags + * [Fix] Fix processing of idempotent rules when autolearn fails + * [Fix] Fix processing of multipart parts with no headers + * [Fix] Fix processing of skip-hashes in fuzzy storage + * [Fix] Fix PTR processing in SPF + * [Fix] Fix pushing country to clickhouse asn table + * [Fix] Fix random forests module + * [Fix] Fix real IP parsing for some strange Exim received + * [Fix] Fix Redis timeout setup + * [Fix] Fix reload crash when hyperscan is enabled + * [Fix] Fix reusing of redis connection after exec + * [Fix] Fix sanity checks on macro value + * [Fix] Fix setting of path and cpath for Lua + * [Fix] Fix setting of signals when spawning a thread + * [Fix] Fix text splitting: stack overflow (too many captures) + * [Fix] Fix ticks processing + * [Fix] Fix upstream addrs updating + * [Fix] Fix urls/emails distinguishing found in queries + * [Fix] Fix user settings check + * [Fix] Fix variable increment + * [Fix] Fix various issues in stat_convert + * [Fix] F-PROT Antivirus infection string for all known occurences + * [Fix] F-PROT Antivirus: only check return code to determine infection + * [Fix] Further fixes around floating point expressions + * [Fix] Further fixes to ANN module + * [Fix] Further fixes to rescore tool + * [Fix] Further fixes to support ES 6 + * [Fix] Further tokenization fixes + * [Fix] Greylisting set phase is not idempotent + * [Fix] Handle proxy copy errors + * [Fix] Header checks: Fix get_raw_header method + * [Fix] Header checks: REPLYTO_UNPARSEABLE rule + * [Fix] Kill spawned processes on termination + * [Fix] Load skip map from all processes as shared cache is unavailable + * [Fix] Lowercase HTTP headers to make them searchable from Lua + * [Fix] Lowercase words + * [Fix] Lua_http: freeing + * [Fix] Lua: lpeg to be loaded with rspamd_lua_add_preload, to avoid "rspamd_config_read: rcl parse error: cannot init lua file […] module 'lpeg' not found" + * [Fix] Map absence is not an error + * [Fix] Metadata exporter: check IP sanity + * [Fix] Milter headers: custom headers: removing headers + * [Fix] Milter headers: skip_local / skip_authenticated settings + * [Fix] Milter headers: X-Spamd-Result header if X-Virus ran first + * [Fix] mime_types: fix next-to-last extension length check + * [Fix] More hacks to deal with old configs + * [Fix] Move composites second pass to the dedicated stage + * [Fix] Multimap: received: filtering of artificial header + * [Fix] Multiple fixes in torch based ANN plugins + * [Fix] Once more (#1879) fix bad extension check + * [Fix] Optimize rspamd_fstring_t reallocations + * [Fix] options.local_networks setting + * [Fix] Parse HREF urls without explicit prefix + * [Fix] Plan new event on HTTP errors + * [Fix] Plug another possible memory leak + * [Fix] Plug memory leak + * [Fix] Plug memory leak in lua_tcp + * [Fix] Plug memory leak when setting email addresses from Lua + * [Fix] Propagate learn/stat errors more precisely + * [Fix] Ratelimit: fix whitelisted_rcpts matching + * [Fix] Ratelimit: lowercase email addresses + * [Fix] RBL: received: deal with missing data (#1965) + * [Fix] Rebalance and slightly rework MX check plugin + * [Fix] Redis key expansion: EVAL: deal with strings + * [Fix] Redis script loading in DMARC; URL tags; URL reputation + * [Fix] Reject invalid bh for DKIM signatures earlier + * [Fix] Relax pem signature detection + * [Fix] Relax unicode properties requirements for chartable module + * [Fix] Remove extra noise from dkim and arc signing + * [Fix] Remove hop-by-hop headers in proxy + * [Fix] Remove incorrect method `task:set_metric_subject` + * [Fix] Replace space like characters in headers with plain space + * [Fix] Restore old style ratelimits support + * [Fix] Rework elasticsearch plugin + * [Fix] Rewriting subjects via force actions module + * [Fix] RPM postinstall + * [Fix] Sanitize IP in history redis + * [Fix] Select the correct signature when doing simple canon + * [Fix] Set CLOEXEC flag on files opened + * [Fix] Setting check_local / check_authed in plugins (#1954) + * [Fix] Settings: avoid checking invalid IP (#1981) + * [Fix] Settings: header: deal with multiple settings (#1988) + * [Fix] Skip checks if both extensions are not bad + * [Fix] Skip nostat tokens when get number of tokens + * [Fix] Some more fixes towards emails detection + * [Fix] SpamAssassin: Fail check_freemail_header if regexp didn't match + * [Fix] Stop using of g_slice... + * [Fix] Switch rspamadm logging to message level + * [Fix] Symbol 'FANNR_SPAM' has its score defined.. + * [Fix] Table parameter for rspamd_config:add_doc() + * [Fix] Treat 'rewrite subject' as spam action + * [Fix] Try harder in passing IPv6 addresses + * [Fix] Try harder to find rfc822 notifications + * [Fix] Try harder to find urls + * [Fix] Use decoded values when parsing mime addresses + * [Fix] Use full URL when making an HTTP request + * [Fix] Use greylisting threshold in greylisting module + * [Fix] Use n_words attribute from ngramms + * [Fix] Use raw urls when sending requests to redirector + * [Fix] Use the right boolean operator on error check + * [Fix] Use weight from map for fuzzy scoring + * [Fix] Various fixes to elastic plugin + * [Fix] Various fixes to fann_redis instantiation + * [Fix] Various improvements in language detection + * [Fix] Virus infection string for F-PROT Antivirus + * [Fix] Virus infetction string for F-PROT Antivirus + * [Fix] WebUI: use relative path for savemap (#1943) + * [Fix] WHITE_ON_WHITE: Ensure score is matched to part that fired the rule + * [Fix] Write configuration changes as UCL config + * [Project] Add detection logic for words + * [Project] Add fast debug logging infrastructure + * [Project] Add more flags to languages + * [Project] Add n-gramms data files + * [Project] Add ngramms frequencies detector + * [Project] Add random words selection logic + * [Project] Add unigramms to language detection as well + * [Project] Convert all C modules to fast debug infrastructure + * [Project] Detect some languages based on unicode script + * [Project] Enable fast debug lookup for some modules + * [Project] Enable language detector init in scanner workers + * [Project] Further improvements to language detector + * [Project] Implement logic of ngramms application + * [Project] Improve weighting in lang_detection + * [Project] Initialize language detector + * [Project] Preliminary version of ngramms based language detector + * [Project] Preliminary version of the new stat_convert + * [Project] Remove old language detector + * [Project] Rework language detection ngramms structure + * [Project] Start language detection project + * [Project] Start rework of language detection to improve quality + * [Project] Use fast debug logging check + * [Rework] Add frame for new reputation based IP score module + * [Rework] Continue stat_convert rework task + * [Rework] Implement new version of fuzzy replies + * [Rework] Improve readability of xmlrpc API + * [Rework] Kill metrics!11 + * [Rework] Ratelimit module + * [Rework] Rename fann_redis to neural plugin + * [Rework] Reorganize mime_types module + * [Rework] Rework rescore utility + * [Rework] Rewrite model and learning logic for rescore + * [Rework] Run post-loads when all initialization is completed + * [Rework] Simplify lua path initialization + * [Rework] Start major stat_convert rework + * [Rework] Start mempool fragmentation reduce project + * [Rework] Start moving of fann redis to torch + * [Rework] Stop embedding rspamadm scripts into C + * [Rework] Use floating point arithmetics in Rspamd expressions + * [Rework] Use frequencies distribution in language detector + * [Rules] Penalise R_BAD_CTE_7BIT for utf8 messages + * [WebUI] Compact graph selectors + * [WebUI] Escape strings inside HTML in history + * [WebUI] Fix message count in throughput summary (#1724) + * [WebUI] Fix NaNs display on Throughput graph + * [WebUI] Migrate widgets to D3 v4 + * [WebUI] Restore passwordless login support (#2003) + * [WebUI] Show symbol descriptions as tooltips in history + * [WebUI] Stop using commas in pie chart tooltips + * [WebUI] Update D3 and jQuery + * [WebUI] Update D3Evolution 1.0.0 -> 1.1.0 1.6.6: 16 Feb 2018 - * [CritFix] Add sanity guards for badly broken HTML - * [CritFix] Another errors path handling fix - * [CritFix] Fix ARC chain verification - * [CritFix] Fix crash in milter errors handler - * [Feature] Allow to insert headers into specific position - * [Feature] Allow to receive signing keys from mempool vars - * [Feature] Authentication-Results: support hiding usernames - * [Fix] Another try to deal with #1998 - * [Fix] Another try to fix #1998 - * [Fix] Better handling of the legacy protocol - * [Fix] Check decoded headers sanity (e.g. by excluding \0) - * [Fix] Deal with nan and inf encoding in json/ucl - * [Fix] Deal with URLs wrapped in [] in text parts - * [Fix] DKIM signing: allow for auth_only to be false - * [Fix] Do not crash on empty subtype - * [Fix] Do not fail rbl plugin when there are no received or emails - * [Fix] Do not skip the last character - * [Fix] Do not try to dereference last character - * [Fix] Do not try to sign unknown domains - * [Fix] Exim Received header protocol parsing - * [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false - * [Fix] Fix bad archive characters stripping - * [Fix] Fix comparision - * [Fix] Fix connecting to a unix socket in rspamadm statconvert - * [Fix] Fix empty headers simple canonicalization - * [Fix] Fix extra hits in PCRE mode for regular expressions - * [Fix] Fix parsing of the per-user script - * [Fix] Fix processing of skip-hashes in fuzzy storage - * [Fix] Fix Redis timeout setup - * [Fix] Fix sanity checks on macro value - * [Fix] Fix text splitting: stack overflow (too many captures) - * [Fix] Fix urls/emails distinguishing found in queries - * [Fix] F-PROT Antivirus: only check return code to determine infection - * [Fix] Metadata exporter: check IP sanity - * [Fix] Multimap: received: filtering of artificial header - * [Fix] Plan new event on HTTP errors - * [Fix] Plug another possible memory leak - * [Fix] Remove hop-by-hop headers in proxy - * [Fix] Sanitize IP in history redis - * [Fix] Setting check_local / check_authed in plugins (#1954) - * [Fix] Settings: avoid checking invalid IP (#1981) - * [Fix] Try harder in passing IPv6 addresses - * [Fix] WebUI: use relative path for savemap (#1943) - * [WebUI] Fix message count in throughput summary (#1724) - * [WebUI] Fix NaNs display on Throughput graph - * [WebUI] Restore passwordless login support (#2003) + * [CritFix] Add sanity guards for badly broken HTML + * [CritFix] Another errors path handling fix + * [CritFix] Fix ARC chain verification + * [CritFix] Fix crash in milter errors handler + * [Feature] Allow to insert headers into specific position + * [Feature] Allow to receive signing keys from mempool vars + * [Feature] Authentication-Results: support hiding usernames + * [Fix] Another try to deal with #1998 + * [Fix] Another try to fix #1998 + * [Fix] Better handling of the legacy protocol + * [Fix] Check decoded headers sanity (e.g. by excluding \0) + * [Fix] Deal with nan and inf encoding in json/ucl + * [Fix] Deal with URLs wrapped in [] in text parts + * [Fix] DKIM signing: allow for auth_only to be false + * [Fix] Do not crash on empty subtype + * [Fix] Do not fail rbl plugin when there are no received or emails + * [Fix] Do not skip the last character + * [Fix] Do not try to dereference last character + * [Fix] Do not try to sign unknown domains + * [Fix] Exim Received header protocol parsing + * [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false + * [Fix] Fix bad archive characters stripping + * [Fix] Fix comparision + * [Fix] Fix connecting to a unix socket in rspamadm statconvert + * [Fix] Fix empty headers simple canonicalization + * [Fix] Fix extra hits in PCRE mode for regular expressions + * [Fix] Fix parsing of the per-user script + * [Fix] Fix processing of skip-hashes in fuzzy storage + * [Fix] Fix Redis timeout setup + * [Fix] Fix sanity checks on macro value + * [Fix] Fix text splitting: stack overflow (too many captures) + * [Fix] Fix urls/emails distinguishing found in queries + * [Fix] F-PROT Antivirus: only check return code to determine infection + * [Fix] Metadata exporter: check IP sanity + * [Fix] Multimap: received: filtering of artificial header + * [Fix] Plan new event on HTTP errors + * [Fix] Plug another possible memory leak + * [Fix] Remove hop-by-hop headers in proxy + * [Fix] Sanitize IP in history redis + * [Fix] Setting check_local / check_authed in plugins (#1954) + * [Fix] Settings: avoid checking invalid IP (#1981) + * [Fix] Try harder in passing IPv6 addresses + * [Fix] WebUI: use relative path for savemap (#1943) + * [WebUI] Fix message count in throughput summary (#1724) + * [WebUI] Fix NaNs display on Throughput graph + * [WebUI] Restore passwordless login support (#2003) 1.6.5: 22 Oct 2017 - * [CritFix] Another portion of tokenization fixes - * [CritFix] Fix memory leak in spf caching logic - * [CritFix] Fix milter commands pipelining - * [CritFix] Fix newlines detection - * [Feature] Filter nan and inf when adding scores - * [Feature] Implement headers flags in mime parser - * [Feature] Support Expires header when using HTTP maps - * [Fix] Actively load skip hashes map in fuzzy storage - * [Fix] Add workaround for IPv6 in sendmail - * [Fix] Authentication Results: Fix SPF smtp.mail_from - * [Fix] Check for magic when checking for an archive - * [Fix] Deal with another case when processing exceptions - * [Fix] Deal with URLs with no slashes after protocol - * [Fix] Do not allow garbadge when checking url domain - * [Fix] Do not ignore short words - * [Fix] Do not strip last character in the last word - * [Fix] Do not treat script content as text - * [Fix] Erase unknown HTML entities - * [Fix] Fix another tokenization issue - * [Fix] Fix DKIM forgeries via multiple headers - * [Fix] Fix emails detection - * [Fix] Fix empty threshold check in greylisting module - * [Fix] Fix enormous scores for R_WHITE_ON_WHITE - * [Fix] Fix loading of per-user redis backend for statistics - * [Fix] Fix multiple headers in DKIM headers list - * [Fix] Fix obscured url in format user@@example.com - * [Fix] Further tokenization fixes - * [Fix] Load skip map from all processes as shared cache is unavailable - * [Fix] Lowercase words - * [Fix] Milter headers: skip_local / skip_authenticated settings - * [Fix] Milter headers: X-Spamd-Result header if X-Virus ran first - * [Fix] Ratelimit: fix whitelisted_rcpts matching - * [Fix] Some more fixes towards emails detection - * [Fix] SpamAssassin: Fail check_freemail_header if regexp didn't match - * [Fix] Use greylisting threshold in greylisting module + * [CritFix] Another portion of tokenization fixes + * [CritFix] Fix memory leak in spf caching logic + * [CritFix] Fix milter commands pipelining + * [CritFix] Fix newlines detection + * [Feature] Filter nan and inf when adding scores + * [Feature] Implement headers flags in mime parser + * [Feature] Support Expires header when using HTTP maps + * [Fix] Actively load skip hashes map in fuzzy storage + * [Fix] Add workaround for IPv6 in sendmail + * [Fix] Authentication Results: Fix SPF smtp.mail_from + * [Fix] Check for magic when checking for an archive + * [Fix] Deal with another case when processing exceptions + * [Fix] Deal with URLs with no slashes after protocol + * [Fix] Do not allow garbadge when checking url domain + * [Fix] Do not ignore short words + * [Fix] Do not strip last character in the last word + * [Fix] Do not treat script content as text + * [Fix] Erase unknown HTML entities + * [Fix] Fix another tokenization issue + * [Fix] Fix DKIM forgeries via multiple headers + * [Fix] Fix emails detection + * [Fix] Fix empty threshold check in greylisting module + * [Fix] Fix enormous scores for R_WHITE_ON_WHITE + * [Fix] Fix loading of per-user redis backend for statistics + * [Fix] Fix multiple headers in DKIM headers list + * [Fix] Fix obscured url in format user@@example.com + * [Fix] Further tokenization fixes + * [Fix] Load skip map from all processes as shared cache is unavailable + * [Fix] Lowercase words + * [Fix] Milter headers: skip_local / skip_authenticated settings + * [Fix] Milter headers: X-Spamd-Result header if X-Virus ran first + * [Fix] Ratelimit: fix whitelisted_rcpts matching + * [Fix] Some more fixes towards emails detection + * [Fix] SpamAssassin: Fail check_freemail_header if regexp didn't match + * [Fix] Use greylisting threshold in greylisting module 1.6.4: 10 Sep 2017 - * [Feature] Add method to get all content-type attributes in Lua - * [Feature] Add some sanity checks for actions and controller - * [Feature] Allow randomly select User-Agent from a list - * [Feature] Deal with obscured URLs with @ symbols - * [Feature] Milter headers: support adding/removing arbitrary headers from config - * [Fix] Add another workaround to display history properly - * [Fix] Add missing rspamadm control options to help - * [Fix] Auth-Results: Multiple DKIM signatures - * [Fix] Crash in URL processing - * [Fix] Default monitoring domain for surbl plugin - * [Fix] Detection of maillist optimized and fixed - * [Fix] Do not cache SPF records with PTR elements - * [Fix] Fix blacklists and DMARC in whitelist - * [Fix] Fix exceptions list in surbl - * [Fix] Fix processing of closed tags - * [Fix] Fix PTR processing in SPF - * [Fix] Lowercase HTTP headers to make them searchable from Lua - * [Fix] options.local_networks setting - * [Fix] Ratelimit: lowercase email addresses - * [Fix] Rebalance and slightly rework MX check plugin - * [Fix] Redis script loading in DMARC; URL tags; URL reputation - * [Fix] Reject invalid bh for DKIM signatures earlier - * [Fix] Remove incorrect method `task:set_metric_subject` - * [Fix] Rewriting subjects via force actions module - * [Fix] RPM postinstall - * [Fix] Treat 'rewrite subject' as spam action - * [Fix] Try harder to find urls - * [Fix] Use full URL when making an HTTP request - * [Fix] Use raw urls when sending requests to redirector - * [Fix] Use weight from map for fuzzy scoring - * [Rules] Penalise R_BAD_CTE_7BIT for utf8 messages + * [Feature] Add method to get all content-type attributes in Lua + * [Feature] Add some sanity checks for actions and controller + * [Feature] Allow randomly select User-Agent from a list + * [Feature] Deal with obscured URLs with @ symbols + * [Feature] Milter headers: support adding/removing arbitrary headers from config + * [Fix] Add another workaround to display history properly + * [Fix] Add missing rspamadm control options to help + * [Fix] Auth-Results: Multiple DKIM signatures + * [Fix] Crash in URL processing + * [Fix] Default monitoring domain for surbl plugin + * [Fix] Detection of maillist optimized and fixed + * [Fix] Do not cache SPF records with PTR elements + * [Fix] Fix blacklists and DMARC in whitelist + * [Fix] Fix exceptions list in surbl + * [Fix] Fix processing of closed tags + * [Fix] Fix PTR processing in SPF + * [Fix] Lowercase HTTP headers to make them searchable from Lua + * [Fix] options.local_networks setting + * [Fix] Ratelimit: lowercase email addresses + * [Fix] Rebalance and slightly rework MX check plugin + * [Fix] Redis script loading in DMARC; URL tags; URL reputation + * [Fix] Reject invalid bh for DKIM signatures earlier + * [Fix] Remove incorrect method `task:set_metric_subject` + * [Fix] Rewriting subjects via force actions module + * [Fix] RPM postinstall + * [Fix] Treat 'rewrite subject' as spam action + * [Fix] Try harder to find urls + * [Fix] Use full URL when making an HTTP request + * [Fix] Use raw urls when sending requests to redirector + * [Fix] Use weight from map for fuzzy scoring + * [Rules] Penalise R_BAD_CTE_7BIT for utf8 messages 1.6.3: 26 Jul 2017 - * [CritFix] Fix semicolons parsing in the content type - * [Feature] Add EBL to the default config - * [Feature] Allow to configure monitored - * [Feature] Allow to skip specific hashes in fuzzy storage - * [Feature] Multimap: checking of symbol options - * [Feature] Redis settings: support checking multiple keys - * [Fix] ARC: Fix Lua 5.3 compatibility; timestamp should be integer - * [Fix] Avoid changing content-transfer-encoding header's value - * [Fix] Don't use whitelist/greylist maps as regexp, but as map - * [Fix] Fix get_content method - * [Fix] Header checks: Fix get_raw_header method - * [Fix] Header checks: REPLYTO_UNPARSEABLE rule - * [Fix] Lua_http: freeing - * [Fix] Milter headers: custom headers: removing headers - * [Fix] Parse HREF urls without explicit prefix - * [Fix] WHITE_ON_WHITE: Ensure score is matched to part that fired the rule - * [WebUI] Escape strings inside HTML in history + * [CritFix] Fix semicolons parsing in the content type + * [Feature] Add EBL to the default config + * [Feature] Allow to configure monitored + * [Feature] Allow to skip specific hashes in fuzzy storage + * [Feature] Multimap: checking of symbol options + * [Feature] Redis settings: support checking multiple keys + * [Fix] ARC: Fix Lua 5.3 compatibility; timestamp should be integer + * [Fix] Avoid changing content-transfer-encoding header's value + * [Fix] Don't use whitelist/greylist maps as regexp, but as map + * [Fix] Fix get_content method + * [Fix] Header checks: Fix get_raw_header method + * [Fix] Header checks: REPLYTO_UNPARSEABLE rule + * [Fix] Lua_http: freeing + * [Fix] Milter headers: custom headers: removing headers + * [Fix] Parse HREF urls without explicit prefix + * [Fix] WHITE_ON_WHITE: Ensure score is matched to part that fired the rule + * [WebUI] Escape strings inside HTML in history 1.6.2: 08 Jul 2017 - * [Conf] Remove Rambler email bl for now - * [Conf] Switch RAMBLER_URIBL to a locally managed source - * [CritFix] Switch from ragel to C for Content-Type parsing - * [Feature] Add `-e` option for lua_repl - * [Feature] Add per-domain emails normalisation rules - * [Feature] Add sessions cache to debug dangling sessions - * [Feature] Add short_text_direct_hash for fuzzy check module - * [Feature] Add text_part:get_stats function - * [Feature] Allow to add custom processing script for surbl - * [Feature] Allow to check reply-to email - * [Feature] Allow to customize spam header, remove existing spam headers - * [Feature] Allow to disable specific workers in the config - * [Feature] Allow to discard messages instead of rejection - * [Feature] Allow to specify custom delimiter in emails plugin - * [Feature] Allow to specify custom User-Agent for rspamc - * [Feature] Allow to store symbols data in Clickhouse - * [Feature] Allow to use HTTPS when connecting to Clickhouse - * [Feature] Enable sessions cache tracking for milter connections - * [Feature] Implement per-line mode in lua_repl (like `perl -p`) - * [Feature] Implement rdns-curve plugin based on rspamd cryptobox - * [Feature] Improve maps cached data lifetime - * [Feature] Improve maps checking frequency - * [Feature] Improve monitored timeouts logic - * [Feature] milter_headers: add `extended_headers_rcpt` option - * [Feature] Milter headers: Add X-Spam-Flag to rmilter-compatibility headers - * [Feature] Milter headers: remove-header routine - * [Feature] Multimap: received filters for extracting TLDs from hostnames - * [Feature] Normalize email aliases in emails module - * [Feature] Re-add rambler email bl (as hashed list) - * [Feature] Reload file maps more frequently - * [Feature] Rework newlines strip parser one more time - * [Feature] Skip updates for messages scanned via controller - * [Feature] Split long DKIM public keys - * [Feature] Store more data when stripping newlines - * [Feature] Support SPF macros transformations - * [Feature] Support suppressing DMARC reports for some domains - * [Fix] Add missing `break` statement - * [Fix] Allow modifiers in SPF macros - * [Fix] DKIM sign tools: edge-cases around use_esld - * [Fix] Do not cache SPF records with macros - * [Fix] Do not overwrite score when setting pre-action - * [Fix] Fix comparison logic - * [Fix] Fix DKIM base64 folding for milter flagged messages - * [Fix] Fix emails module configuration - * [Fix] Fix folding for arc headers when milter interface is used - * [Fix] Fix gmail dots removal - * [Fix] Fix rspamc detection in greylist module - * [Fix] Fix some more issues with HTTP maps - * [Fix] Milter sessions can live forever - * [Fix] Normalize fuzzy probability better - * [Fix] Plug memory leak - * [Fix] RBL: Fixed hashed email address lookups - * [Fix] Try to deal with brain-damaged milter behaviour - * [Fix] Use `\n` to fold headers for milter - * [Rework] Allow to use custom callback for monitored checks - * [Rework] Further steps towards one process monitoring - * [Rework] Send health checks from a single worker - * [WebUI] Round-up throughput summary values + * [Conf] Remove Rambler email bl for now + * [Conf] Switch RAMBLER_URIBL to a locally managed source + * [CritFix] Switch from ragel to C for Content-Type parsing + * [Feature] Add `-e` option for lua_repl + * [Feature] Add per-domain emails normalisation rules + * [Feature] Add sessions cache to debug dangling sessions + * [Feature] Add short_text_direct_hash for fuzzy check module + * [Feature] Add text_part:get_stats function + * [Feature] Allow to add custom processing script for surbl + * [Feature] Allow to check reply-to email + * [Feature] Allow to customize spam header, remove existing spam headers + * [Feature] Allow to disable specific workers in the config + * [Feature] Allow to discard messages instead of rejection + * [Feature] Allow to specify custom delimiter in emails plugin + * [Feature] Allow to specify custom User-Agent for rspamc + * [Feature] Allow to store symbols data in Clickhouse + * [Feature] Allow to use HTTPS when connecting to Clickhouse + * [Feature] Enable sessions cache tracking for milter connections + * [Feature] Implement per-line mode in lua_repl (like `perl -p`) + * [Feature] Implement rdns-curve plugin based on rspamd cryptobox + * [Feature] Improve maps cached data lifetime + * [Feature] Improve maps checking frequency + * [Feature] Improve monitored timeouts logic + * [Feature] milter_headers: add `extended_headers_rcpt` option + * [Feature] Milter headers: Add X-Spam-Flag to rmilter-compatibility headers + * [Feature] Milter headers: remove-header routine + * [Feature] Multimap: received filters for extracting TLDs from hostnames + * [Feature] Normalize email aliases in emails module + * [Feature] Re-add rambler email bl (as hashed list) + * [Feature] Reload file maps more frequently + * [Feature] Rework newlines strip parser one more time + * [Feature] Skip updates for messages scanned via controller + * [Feature] Split long DKIM public keys + * [Feature] Store more data when stripping newlines + * [Feature] Support SPF macros transformations + * [Feature] Support suppressing DMARC reports for some domains + * [Fix] Add missing `break` statement + * [Fix] Allow modifiers in SPF macros + * [Fix] DKIM sign tools: edge-cases around use_esld + * [Fix] Do not cache SPF records with macros + * [Fix] Do not overwrite score when setting pre-action + * [Fix] Fix comparison logic + * [Fix] Fix DKIM base64 folding for milter flagged messages + * [Fix] Fix emails module configuration + * [Fix] Fix folding for arc headers when milter interface is used + * [Fix] Fix gmail dots removal + * [Fix] Fix rspamc detection in greylist module + * [Fix] Fix some more issues with HTTP maps + * [Fix] Milter sessions can live forever + * [Fix] Normalize fuzzy probability better + * [Fix] Plug memory leak + * [Fix] RBL: Fixed hashed email address lookups + * [Fix] Try to deal with brain-damaged milter behaviour + * [Fix] Use `\n` to fold headers for milter + * [Rework] Allow to use custom callback for monitored checks + * [Rework] Further steps towards one process monitoring + * [Rework] Send health checks from a single worker + * [WebUI] Round-up throughput summary values 1.6.1: 14 Jun 2017 - * [Fix] Allow to init resolver without rspamd_config - * [Fix] Do not crash when resolver failed to initialize - * [Fix] Fix abstract context layout - * [Fix] Fix CGP helper reply parsing - * [Fix] Fix crashes when socket write errors occur - * [Fix] Fix parsing IPv6 nameservers in resolv.conf - * [Fix] Milter: Don't defer on "greylist" action + * [Fix] Allow to init resolver without rspamd_config + * [Fix] Do not crash when resolver failed to initialize + * [Fix] Fix abstract context layout + * [Fix] Fix CGP helper reply parsing + * [Fix] Fix crashes when socket write errors occur + * [Fix] Fix parsing IPv6 nameservers in resolv.conf + * [Fix] Milter: Don't defer on "greylist" action 1.6.0: 12 Jun 2017 - * [Conf] Add rspamd_proxy to the default configuration set - * [Conf] Add sample arc module config - * [Conf] Do away with systemd specifics completely - * [Conf] Increase min_bytes to avoid FP - * [Conf] Remove ratelimits from default configuration - * [CritFix] Fix accepting on IPv6 sockets - * [CritFix] Fix corruption when multiple fuzzy are defined - * [CritFix] Fix learn condition in fuzzy check - * [CritFix] Fix memory leak in fuzzy check - * [CritFix] Fix memory leak in maps scheduling - * [CritFix] Paese the last character in DKIM signature correctly - * [CritFix] Zero fill sockaddr_un - * [Feature] Add ability to add doc strings by example - * [Feature] Add API to verify DKIM (and ARC) signatures - * [Feature] Add compression/decompression to proxy - * [Feature] Add count to url structure - * [Feature] Add initial support of the new protocol reply - * [Feature] Add Lua plugin spamtrap - * [Feature] Add `monitored_address` for rbls - * [Feature] Add new schema for bayes tokens - * [Feature] Add preliminary ARC support to dkim code - * [Feature] Add preliminary support of ARC signing - * [Feature] Add rules to detect bad 8bit characters in From and To - * [Feature] Add scanning support for milter protocol - * [Feature] Add support for bidirectional symbols in rspamd_stats - * [Feature] Add support for static maps - * [Feature] Add support of maps with multiple regexps matches - * [Feature] Add `text_multiplier` param - * [Feature] Add the preliminary ARC plugin - * [Feature] Add top redirector targets rank - * [Feature] Allow async events to be registered from LUA rules - * [Feature] Allow storing bayes tokens in Redis - * [Feature] Allow to exclude specific domains from mx check - * [Feature] Allow to have a stack of watcher finalisers - * [Feature] Allow to pass hostname to `-i` flag in Rspamc - * [Feature] Allow to set custom user agent in url redirector - * [Feature] Allow to use custom callback when parsing resolv.conf - * [Feature] Allow to use domain from authenticated user - * [Feature] Bayes expiry plugin - * [Feature] Check dkim sign keys for modifications - * [Feature] DKIM signing: sign_networks/local address specific use_domain settings - * [Feature] DMARC: Support excluding domains from sampling - * [Feature] Expire processing items for URL redirector aggressively - * [Feature] Fix surbl monitored for IP lists, add `monitored_domain` option - * [Feature] Implement caching for dkim body hashes - * [Feature] Implement milter protocol scan reply - * [Feature] Improve omograph phishing detection - * [Feature] Initial support of self-scan in Rspamd proxy - * [Feature] Keep track of headers in milter interface - * [Feature] Milter headers: better controls for local/authenticated - * [Feature] Multimap: email:domain:tld filter - * [Feature] Preliminary DMARC reporting implementation - * [Feature] Reuse stemmers in the cache - * [Feature] Rework confighelp to load Lua plugins - * [Feature] Rework hfilter to use hyperscan if possible - * [Feature] Rework lua RSA API - * [Feature] Rmilter_headers: approximate rmilter's extended_spam_headers - * [Feature] Start integration of milter support in proxy - * [Feature] Store average words length and short words count - * [Feature] Store hash of headers order and names - * [Feature] Support MTA name header - * [Feature] Support multiple types of dkim signing in Lua - * [Feature] Support numeric arguments for Redis requests - * [Feature] Use headers hash in bayes metatokens - * [Feature] Use normal resolv.conf rules of rotation in Rspamd - * [Feature] Use version 2 proto for checking messages - * [Fix] Allow to follow symlinks when safe - * [Fix] Append MX name for authentication results as required - * [Fix] Change default text multiplier from 0.5 to 2.0 - * [Fix] Check min_bytes for images as well - * [Fix] Deal with 7bit charsets properly - * [Fix] Deal with 8bit characters in email addresses - * [Fix] Deal with unpaired tags - * [Fix] Detect confighelp in plugins initialisation - * [Fix] Disable certain checks for utf spoof detection - * [Fix] DKIM Signing: avoid nil index when From header is missing - * [Fix] Do not add exact hashes from different parts - * [Fix] Do not check DMARC if SPF or DKIM were not checked - * [Fix] Do not check URLs that are resolved to be redirected - * [Fix] Do not set bayes probability if we don't use it - * [Fix] Do not stop on illegal unicode points - replace them - * [Fix] Fix another race condition in arc checks - * [Fix] Fix arc count logic - * [Fix] Fix ARC signing - * [Fix] Fix brain-damaged spamc protocol for now - * [Fix] Fix calling for peak functions - * [Fix] Fix couple of issues in FORWARDED rule - * [Fix] Fix CTE propagation from parent containers to children parts - * [Fix] Fix errors processing in the controller - * [Fix] Fix format string in milter - * [Fix] Fix issues in SPF macros parsing - * [Fix] Fix logging format string - * [Fix] Fix logic of cached passwords check - * [Fix] Fix lowercasing of stemmed words - * [Fix] Fix LRU elements removal - * [Fix] Fix memory leak when accepting from unix sockets - * [Fix] Fix milter connections persistence - * [Fix] Fix objects merging in UCL - * [Fix] Fix order of operations to avoid race condition - * [Fix] Fix parsing of long regexp types - * [Fix] Fix passing data to log helper when many symbols defined - * [Fix] Fix pools management for milter session - * [Fix] Fix processing of the watchers - * [Fix] Fix queue id macro in milter - * [Fix] Fix R_BAD_CTE_7BIT rule - * [Fix] Fix Redis timeout set - * [Fix] Fix REPLYTO_UNPARSEABLE rule - * [Fix] Fix setting of email address - * [Fix] Fix some more issues about duplicated fuzzy requests - * [Fix] Fix spamc support in rspamd proxy - * [Fix] Fix syntax error in spamtrap plugin - * [Fix] Fix url counts for href urls - * [Fix] Fix url handling in the protocol - * [Fix] Multimap: Received IP filters with Redis - * [Fix] Oops, fix d9d0fa5e86db2f4470d34395a233b450478b2f60 - * [Fix] Parse rgb[a](x,x,x[,x]) css colors - * [Fix] Phishing: strict_domains - * [Fix] Reduce maps aggressiveness - * [Fix] Reresolve upstreams even if there is a single server there - * [Fix] Rspamadm grep: Disable Lua patterns in string search by default - * [Fix] Skip text parts when checking binary parts in fuzzy check - * [Fix] Support v2 checks in controller - * [Fix] Treat empty address as valid - * [Fix] Try harder to detect CTE - * [Fix] Try to deal with v4 mapped to v6 addresses on accept - * [Fix] Use dkim signing callback properly - * [Fix] Use non-volatile memory for storing data - * [Fix] Use static maps instead of ugly hack for radix_from_config - * [Fix] Use the same pool for related sessions - * [Rework] Continue modularisation for lua library - * [Rework] Initial milter protocol support - * [Rework] Make log pipes worker agnostic, add scanners API - * [Rework] Move authentication results generation to a separate routine - * [Rework] Move common DKIM functions to a separate lua module - * [Rework] Move global functions to a separate directory - * [Rework] Prepare dkim module for ARC checks - * [Rework] Propagate ucl variables from the command line - * [Rework] Remove multiple metrics support from Rspamd - * [Rework] Stop using name 'rmilter' for the modern protocol - * [Rework] Use LFU algorithm in LRU cache - * [Rules] Fix received TLS rules - * [Rules] Improve URL_COUNT_ODD rule - * [WebUI] Fix add header filter in history - * [WebUI] Use modern protocol for checking messages + * [Conf] Add rspamd_proxy to the default configuration set + * [Conf] Add sample arc module config + * [Conf] Do away with systemd specifics completely + * [Conf] Increase min_bytes to avoid FP + * [Conf] Remove ratelimits from default configuration + * [CritFix] Fix accepting on IPv6 sockets + * [CritFix] Fix corruption when multiple fuzzy are defined + * [CritFix] Fix learn condition in fuzzy check + * [CritFix] Fix memory leak in fuzzy check + * [CritFix] Fix memory leak in maps scheduling + * [CritFix] Paese the last character in DKIM signature correctly + * [CritFix] Zero fill sockaddr_un + * [Feature] Add ability to add doc strings by example + * [Feature] Add API to verify DKIM (and ARC) signatures + * [Feature] Add compression/decompression to proxy + * [Feature] Add count to url structure + * [Feature] Add initial support of the new protocol reply + * [Feature] Add Lua plugin spamtrap + * [Feature] Add `monitored_address` for rbls + * [Feature] Add new schema for bayes tokens + * [Feature] Add preliminary ARC support to dkim code + * [Feature] Add preliminary support of ARC signing + * [Feature] Add rules to detect bad 8bit characters in From and To + * [Feature] Add scanning support for milter protocol + * [Feature] Add support for bidirectional symbols in rspamd_stats + * [Feature] Add support for static maps + * [Feature] Add support of maps with multiple regexps matches + * [Feature] Add `text_multiplier` param + * [Feature] Add the preliminary ARC plugin + * [Feature] Add top redirector targets rank + * [Feature] Allow async events to be registered from LUA rules + * [Feature] Allow storing bayes tokens in Redis + * [Feature] Allow to exclude specific domains from mx check + * [Feature] Allow to have a stack of watcher finalisers + * [Feature] Allow to pass hostname to `-i` flag in Rspamc + * [Feature] Allow to set custom user agent in url redirector + * [Feature] Allow to use custom callback when parsing resolv.conf + * [Feature] Allow to use domain from authenticated user + * [Feature] Bayes expiry plugin + * [Feature] Check dkim sign keys for modifications + * [Feature] DKIM signing: sign_networks/local address specific use_domain settings + * [Feature] DMARC: Support excluding domains from sampling + * [Feature] Expire processing items for URL redirector aggressively + * [Feature] Fix surbl monitored for IP lists, add `monitored_domain` option + * [Feature] Implement caching for dkim body hashes + * [Feature] Implement milter protocol scan reply + * [Feature] Improve omograph phishing detection + * [Feature] Initial support of self-scan in Rspamd proxy + * [Feature] Keep track of headers in milter interface + * [Feature] Milter headers: better controls for local/authenticated + * [Feature] Multimap: email:domain:tld filter + * [Feature] Preliminary DMARC reporting implementation + * [Feature] Reuse stemmers in the cache + * [Feature] Rework confighelp to load Lua plugins + * [Feature] Rework hfilter to use hyperscan if possible + * [Feature] Rework lua RSA API + * [Feature] Rmilter_headers: approximate rmilter's extended_spam_headers + * [Feature] Start integration of milter support in proxy + * [Feature] Store average words length and short words count + * [Feature] Store hash of headers order and names + * [Feature] Support MTA name header + * [Feature] Support multiple types of dkim signing in Lua + * [Feature] Support numeric arguments for Redis requests + * [Feature] Use headers hash in bayes metatokens + * [Feature] Use normal resolv.conf rules of rotation in Rspamd + * [Feature] Use version 2 proto for checking messages + * [Fix] Allow to follow symlinks when safe + * [Fix] Append MX name for authentication results as required + * [Fix] Change default text multiplier from 0.5 to 2.0 + * [Fix] Check min_bytes for images as well + * [Fix] Deal with 7bit charsets properly + * [Fix] Deal with 8bit characters in email addresses + * [Fix] Deal with unpaired tags + * [Fix] Detect confighelp in plugins initialisation + * [Fix] Disable certain checks for utf spoof detection + * [Fix] DKIM Signing: avoid nil index when From header is missing + * [Fix] Do not add exact hashes from different parts + * [Fix] Do not check DMARC if SPF or DKIM were not checked + * [Fix] Do not check URLs that are resolved to be redirected + * [Fix] Do not set bayes probability if we don't use it + * [Fix] Do not stop on illegal unicode points - replace them + * [Fix] Fix another race condition in arc checks + * [Fix] Fix arc count logic + * [Fix] Fix ARC signing + * [Fix] Fix brain-damaged spamc protocol for now + * [Fix] Fix calling for peak functions + * [Fix] Fix couple of issues in FORWARDED rule + * [Fix] Fix CTE propagation from parent containers to children parts + * [Fix] Fix errors processing in the controller + * [Fix] Fix format string in milter + * [Fix] Fix issues in SPF macros parsing + * [Fix] Fix logging format string + * [Fix] Fix logic of cached passwords check + * [Fix] Fix lowercasing of stemmed words + * [Fix] Fix LRU elements removal + * [Fix] Fix memory leak when accepting from unix sockets + * [Fix] Fix milter connections persistence + * [Fix] Fix objects merging in UCL + * [Fix] Fix order of operations to avoid race condition + * [Fix] Fix parsing of long regexp types + * [Fix] Fix passing data to log helper when many symbols defined + * [Fix] Fix pools management for milter session + * [Fix] Fix processing of the watchers + * [Fix] Fix queue id macro in milter + * [Fix] Fix R_BAD_CTE_7BIT rule + * [Fix] Fix Redis timeout set + * [Fix] Fix REPLYTO_UNPARSEABLE rule + * [Fix] Fix setting of email address + * [Fix] Fix some more issues about duplicated fuzzy requests + * [Fix] Fix spamc support in rspamd proxy + * [Fix] Fix syntax error in spamtrap plugin + * [Fix] Fix url counts for href urls + * [Fix] Fix url handling in the protocol + * [Fix] Multimap: Received IP filters with Redis + * [Fix] Oops, fix d9d0fa5e86db2f4470d34395a233b450478b2f60 + * [Fix] Parse rgb[a](x,x,x[,x]) css colors + * [Fix] Phishing: strict_domains + * [Fix] Reduce maps aggressiveness + * [Fix] Reresolve upstreams even if there is a single server there + * [Fix] Rspamadm grep: Disable Lua patterns in string search by default + * [Fix] Skip text parts when checking binary parts in fuzzy check + * [Fix] Support v2 checks in controller + * [Fix] Treat empty address as valid + * [Fix] Try harder to detect CTE + * [Fix] Try to deal with v4 mapped to v6 addresses on accept + * [Fix] Use dkim signing callback properly + * [Fix] Use non-volatile memory for storing data + * [Fix] Use static maps instead of ugly hack for radix_from_config + * [Fix] Use the same pool for related sessions + * [Rework] Continue modularisation for lua library + * [Rework] Initial milter protocol support + * [Rework] Make log pipes worker agnostic, add scanners API + * [Rework] Move authentication results generation to a separate routine + * [Rework] Move common DKIM functions to a separate lua module + * [Rework] Move global functions to a separate directory + * [Rework] Prepare dkim module for ARC checks + * [Rework] Propagate ucl variables from the command line + * [Rework] Remove multiple metrics support from Rspamd + * [Rework] Stop using name 'rmilter' for the modern protocol + * [Rework] Use LFU algorithm in LRU cache + * [Rules] Fix received TLS rules + * [Rules] Improve URL_COUNT_ODD rule + * [WebUI] Fix add header filter in history + * [WebUI] Use modern protocol for checking messages 1.5.9: - * [Conf] Increase min_bytes to avoid FP - * [Conf] Remove ratelimits from default configuration - * [CritFix] Fix accepting on IPv6 sockets - * [CritFix] Zero fill sockaddr_un - * [Feature] Add `text_multiplier` param - * [Fix] Check min_bytes for images as well - * [Fix] Do not add exact hashes from different parts - * [Fix] Fix memory leak when accepting from unix sockets - * [Fix] Fix some more issues about duplicated fuzzy requests - * [Fix] Phishing: strict_domains - * [Fix] Skip text parts when checking binary parts in fuzzy check - * [Minor] Add the same duplicates protection for all fuzzy hashes types - * [Minor] Fix braces - * [Minor] Fix test - * [Minor] SPOOF_DISPLAY_NAME: Use all SMTP/MIME recipients - * [Minor] Validate assumed spoofed display name domains to contain a dot + * [Conf] Increase min_bytes to avoid FP + * [Conf] Remove ratelimits from default configuration + * [CritFix] Fix accepting on IPv6 sockets + * [CritFix] Zero fill sockaddr_un + * [Feature] Add `text_multiplier` param + * [Fix] Check min_bytes for images as well + * [Fix] Do not add exact hashes from different parts + * [Fix] Fix memory leak when accepting from unix sockets + * [Fix] Fix some more issues about duplicated fuzzy requests + * [Fix] Phishing: strict_domains + * [Fix] Skip text parts when checking binary parts in fuzzy check + * [Minor] Add the same duplicates protection for all fuzzy hashes types + * [Minor] Fix braces + * [Minor] Fix test + * [Minor] SPOOF_DISPLAY_NAME: Use all SMTP/MIME recipients + * [Minor] Validate assumed spoofed display name domains to contain a dot 1.5.8: - * [CritFix] Fix memory leak in fuzzy check - * [CritFix] Fix memory leak in maps scheduling - * [Feature] Multimap: email:domain:tld filter - * [Fix] DKIM Signing: avoid nil index when From header is missing - * [Fix] Do not set bayes probability if we don't use it - * [Fix] Do not stop on illegal unicode points - replace them - * [Fix] Fix brain-damaged spamc protocol for now - * [Fix] Fix Redis timeout set - * [Fix] Fix spamc support in rspamd proxy - * [Fix] Multimap: Received IP filters with Redis - * [Fix] Parse rgb[a](x,x,x[,x]) css colors - * [Fix] Reresolve upstreams even if there is a single server there - * [Fix] Treat empty address as valid - * [Fix] Try harder to detect CTE - * [Fix] Try to deal with v4 mapped to v6 addresses on accept - * [Minor] Add `wsf` and `hta` bad archive extensions - * [Minor] Fix configuration option - * [Minor] Fix result parsing for SAVAPI - * [Minor] Further logging improvements - * [Minor] Improve logging of errors - * [Minor] Prevent MID_CONTAINS_FROM from firing on empty address - * [Minor] Reduce digit->number transmission penalty - * [Minor] Relax CTYPE_MISSING_DISPOSITION rule + * [CritFix] Fix memory leak in fuzzy check + * [CritFix] Fix memory leak in maps scheduling + * [Feature] Multimap: email:domain:tld filter + * [Fix] DKIM Signing: avoid nil index when From header is missing + * [Fix] Do not set bayes probability if we don't use it + * [Fix] Do not stop on illegal unicode points - replace them + * [Fix] Fix brain-damaged spamc protocol for now + * [Fix] Fix Redis timeout set + * [Fix] Fix spamc support in rspamd proxy + * [Fix] Multimap: Received IP filters with Redis + * [Fix] Parse rgb[a](x,x,x[,x]) css colors + * [Fix] Reresolve upstreams even if there is a single server there + * [Fix] Treat empty address as valid + * [Fix] Try harder to detect CTE + * [Fix] Try to deal with v4 mapped to v6 addresses on accept + * [Minor] Add `wsf` and `hta` bad archive extensions + * [Minor] Fix configuration option + * [Minor] Fix result parsing for SAVAPI + * [Minor] Further logging improvements + * [Minor] Improve logging of errors + * [Minor] Prevent MID_CONTAINS_FROM from firing on empty address + * [Minor] Reduce digit->number transmission penalty + * [Minor] Relax CTYPE_MISSING_DISPOSITION rule 1.5.7: - * [CritFix] Fix corruption when multiple fuzzy are defined - * [CritFix] Fix learn condition in fuzzy check - * [Feature] Add rules to detect bad 8bit characters in From and To - * [Feature] DKIM signing: sign_networks/local address specific use_domain settings - * [Feature] Support numeric arguments for Redis requests - * [Fix] Deal with 8bit characters in email addresses - * [Fix] Fix couple of issues in FORWARDED rule - * [Fix] Fix passing data to log helper when many symbols defined - * [Fix] Fix R_BAD_CTE_7BIT rule - * [Fix] Fix REPLYTO_UNPARSEABLE rule - * [Fix] Fix setting of email address - * [Fix] Rspamadm grep: Disable Lua patterns in string search by default - * [Minor] Add Lua 5.3 workaround - * [Minor] Add lua methods to detect if a part has 8bit characters - * [Minor] Allow session-less lua dns requests - * [Minor] Allow to append greylist end time to message reported - * [Minor] Avoid `nil` table - * [Minor] Disable dkim_signing if redis is specified but not configured - * [Minor] Fix build with pcre2 - * [Minor] Fix rule - * [Minor] Fix warnings - * [Minor] Format floating point number - * [Minor] Push email flags to the lua API - * [Minor] Silence some warnings - * [Minor] Silence warning - * [Minor] Try all hostname regexps to find the most significant one - * [WebUI] Fix add header filter in history + * [CritFix] Fix corruption when multiple fuzzy are defined + * [CritFix] Fix learn condition in fuzzy check + * [Feature] Add rules to detect bad 8bit characters in From and To + * [Feature] DKIM signing: sign_networks/local address specific use_domain settings + * [Feature] Support numeric arguments for Redis requests + * [Fix] Deal with 8bit characters in email addresses + * [Fix] Fix couple of issues in FORWARDED rule + * [Fix] Fix passing data to log helper when many symbols defined + * [Fix] Fix R_BAD_CTE_7BIT rule + * [Fix] Fix REPLYTO_UNPARSEABLE rule + * [Fix] Fix setting of email address + * [Fix] Rspamadm grep: Disable Lua patterns in string search by default + * [Minor] Add Lua 5.3 workaround + * [Minor] Add lua methods to detect if a part has 8bit characters + * [Minor] Allow session-less lua dns requests + * [Minor] Allow to append greylist end time to message reported + * [Minor] Avoid `nil` table + * [Minor] Disable dkim_signing if redis is specified but not configured + * [Minor] Fix build with pcre2 + * [Minor] Fix rule + * [Minor] Fix warnings + * [Minor] Format floating point number + * [Minor] Push email flags to the lua API + * [Minor] Silence some warnings + * [Minor] Silence warning + * [Minor] Try all hostname regexps to find the most significant one + * [WebUI] Fix add header filter in history 1.5.6: - * [Feature] Add unigramms support in bayes - * [Feature] Allow configurable sign headers for DKIM - * [Feature] Allow to add unigramm metatokens from Lua - * [Feature] DKIM Signing: envelope match exception for local IPs - * [Feature] UCL: register parser variables from Lua - * [Fix] Always try to adjust filename - * [Fix] Do extra copy to ensure that original content is never touched - * [Fix] Fix SPOOF_REPLYTO rule - * [Fix] Ignore Rmilter added Received - * [Fix] More fixes for hashed email dnsbls - * [Fix] Plug memory leak in chartable module - * [WebUI] Display multiple alerts at once + * [Feature] Add unigramms support in bayes + * [Feature] Allow configurable sign headers for DKIM + * [Feature] Allow to add unigramm metatokens from Lua + * [Feature] DKIM Signing: envelope match exception for local IPs + * [Feature] UCL: register parser variables from Lua + * [Fix] Always try to adjust filename + * [Fix] Do extra copy to ensure that original content is never touched + * [Fix] Fix SPOOF_REPLYTO rule + * [Fix] Ignore Rmilter added Received + * [Fix] More fixes for hashed email dnsbls + * [Fix] Plug memory leak in chartable module + * [WebUI] Display multiple alerts at once 1.5.5: - * [CritFix] Fix classifier learning with Redis backend - * [CritFix] Fix issue when parsing encoded rfc822/messages - * [Feature] Add escaped version of lua_ucl import - * [Feature] Add task:headers_foreach function - * [Feature] Allow to process filenames from content type - * [Feature] Allow to query hashed emails - * [Feature] Ignore bayes with mostly metatokens or with too few text - * [Feature] Probabilistically skip metatokens - * [Feature] Retrieve all virus names from SAVAPI - * [Feature] Rework classifiers lua metatokens - * [Feature] Store headers order - * [Feature] Store text tokens inside bayes tokens - * [Feature] Use cached shingles keys - * [Fix] Add missing score normalisation for HFILTER_URL_ONLY - * [Fix] Avoid lookup in absent hash - * [Fix] Check return values from Lua functions called from C - * [Fix] Do not count sending and loading time in rspamc - * [Fix] Escape json strings for controller rejplies from Lua - * [Fix] Fix archive scans for savapi - * [Fix] Fix domain_only emails RBL - * [Fix] Fix ip_score map configuration - * [Fix] Fix JSON output for history_redis - * [Fix] Fix one character length substrings search - * [Fix] Fix parsing of non-RFC compatible Exim received - * [Fix] Fix parsing of options for workers with the same type - * [Fix] Fix processing of small tokens vectors - * [Fix] Fix rfc2047 tokenization - * [Fix] Fix typo - * [Fix] More fixes for inplace decoding - * [Fix] Try to avoid modifications of the original data - * [Fix] URL redirector: Fix call to is_redirector - * [Rework] Set token data as uint64_t instead of chars array - * [WebUI] Check if neighbours' history backend versions match - * [WebUI] Disable phrase connectors replacement in history filtering - * [WebUI] Disable phrase connectors replacement in symbols filtering - * [WebUI] Do not hide messages with bad subject, just replace it with '???' - * [WebUI] Fix error message - * [WebUI] Fix history v2 display - * [WebUI] Fix legacy history - * [WebUI] history: break To address lists on commas - * [WebUI] Increase default timeout to 20 seconds - * [WebUI] Save some history table space + * [CritFix] Fix classifier learning with Redis backend + * [CritFix] Fix issue when parsing encoded rfc822/messages + * [Feature] Add escaped version of lua_ucl import + * [Feature] Add task:headers_foreach function + * [Feature] Allow to process filenames from content type + * [Feature] Allow to query hashed emails + * [Feature] Ignore bayes with mostly metatokens or with too few text + * [Feature] Probabilistically skip metatokens + * [Feature] Retrieve all virus names from SAVAPI + * [Feature] Rework classifiers lua metatokens + * [Feature] Store headers order + * [Feature] Store text tokens inside bayes tokens + * [Feature] Use cached shingles keys + * [Fix] Add missing score normalisation for HFILTER_URL_ONLY + * [Fix] Avoid lookup in absent hash + * [Fix] Check return values from Lua functions called from C + * [Fix] Do not count sending and loading time in rspamc + * [Fix] Escape json strings for controller rejplies from Lua + * [Fix] Fix archive scans for savapi + * [Fix] Fix domain_only emails RBL + * [Fix] Fix ip_score map configuration + * [Fix] Fix JSON output for history_redis + * [Fix] Fix one character length substrings search + * [Fix] Fix parsing of non-RFC compatible Exim received + * [Fix] Fix parsing of options for workers with the same type + * [Fix] Fix processing of small tokens vectors + * [Fix] Fix rfc2047 tokenization + * [Fix] Fix typo + * [Fix] More fixes for inplace decoding + * [Fix] Try to avoid modifications of the original data + * [Fix] URL redirector: Fix call to is_redirector + * [Rework] Set token data as uint64_t instead of chars array + * [WebUI] Check if neighbours' history backend versions match + * [WebUI] Disable phrase connectors replacement in history filtering + * [WebUI] Disable phrase connectors replacement in symbols filtering + * [WebUI] Do not hide messages with bad subject, just replace it with '???' + * [WebUI] Fix error message + * [WebUI] Fix history v2 display + * [WebUI] Fix legacy history + * [WebUI] history: break To address lists on commas + * [WebUI] Increase default timeout to 20 seconds + * [WebUI] Save some history table space 1.5.4: - * [Conf] Add history_redis default configuration - * [Feature] Add spoofed rules - * [Feature] Add URL_IN_SUBJECT rule - * [Feature] Allow to get task's subject - * [Feature] Allow to specify maximum number of shots for symbols - * [Feature] Distinguish URLs found in Subject - * [Feature] Memoize LPEG grammars - * [Feature] Parse else parts in SA rules - * [Feature] Process subject for mixed characters - * [Feature] Resolve url chains in url_redirector module - * [Feature] Stat greylisted messages as greylisted not soft-rejected - * [Feature] Support checking for redirector in Lua SURBL - * [Feature] Support tag_exists SA function - * [Feature] Work with broken rfc2047 tokens - * [Fix] Check all watcher's dependencies - * [Fix] Do not compile hyperscan with no SSSE3 support - * [Fix] Do not crash if cannot decode qp encoded part - * [Fix] Fix dependencies of DKIM when multiple signatures are found - * [Fix] Fix lists in whitelist plugin - * [Fix] Fix one-shot symbols weight calculations - * [Fix] Fix options and shots match - * [Fix] Fix order of symbol options - * [Fix] Fix parsing of dot at the end of the address - * [Fix] Fix parsing of lua table arguments - * [Fix] Fix processing of subject words - * [Fix] Fix string split memoization - * [Fix] Fix templates grammar usage - * [Fix] Fix various issues related to Lua stack manipulation - * [Fix] Force actions: Use postfilter if we have honor_action / require_action - * [Fix] Further fixes to avoid PHISHING FP - * [Fix] Preserve order of options in symbols - * [Fix] Rspamadm grep: deal with unusually-formatted logs - * [Fix] Use hostname suffix when dealing with history - * [Rework] Remove outdated SA rules - * [WebUI] Add flexible columns - * [WebUI] Add footable - * [WebUI] Add sender, recipients and subject columns - * [WebUI] Allow message-id break - * [WebUI] Fix history clustering - * [WebUI] Fix history display - * [WebUI] Fix sorting - * [WebUI] Humanize sizes - * [WebUI] Initial move towards footable - * [WebUI] Remove datatables - * [WebUI] Replace `.values` method with `.map` - * [WebUI] Rework v2 symbols display - * [WebUI] Try to normalize frequencies - * [WebUI] Unbreak WebUI - * [WebUI] Use Footable to draw Throughput summary table + * [Conf] Add history_redis default configuration + * [Feature] Add spoofed rules + * [Feature] Add URL_IN_SUBJECT rule + * [Feature] Allow to get task's subject + * [Feature] Allow to specify maximum number of shots for symbols + * [Feature] Distinguish URLs found in Subject + * [Feature] Memoize LPEG grammars + * [Feature] Parse else parts in SA rules + * [Feature] Process subject for mixed characters + * [Feature] Resolve url chains in url_redirector module + * [Feature] Stat greylisted messages as greylisted not soft-rejected + * [Feature] Support checking for redirector in Lua SURBL + * [Feature] Support tag_exists SA function + * [Feature] Work with broken rfc2047 tokens + * [Fix] Check all watcher's dependencies + * [Fix] Do not compile hyperscan with no SSSE3 support + * [Fix] Do not crash if cannot decode qp encoded part + * [Fix] Fix dependencies of DKIM when multiple signatures are found + * [Fix] Fix lists in whitelist plugin + * [Fix] Fix one-shot symbols weight calculations + * [Fix] Fix options and shots match + * [Fix] Fix order of symbol options + * [Fix] Fix parsing of dot at the end of the address + * [Fix] Fix parsing of lua table arguments + * [Fix] Fix processing of subject words + * [Fix] Fix string split memoization + * [Fix] Fix templates grammar usage + * [Fix] Fix various issues related to Lua stack manipulation + * [Fix] Force actions: Use postfilter if we have honor_action / require_action + * [Fix] Further fixes to avoid PHISHING FP + * [Fix] Preserve order of options in symbols + * [Fix] Rspamadm grep: deal with unusually-formatted logs + * [Fix] Use hostname suffix when dealing with history + * [Rework] Remove outdated SA rules + * [WebUI] Add flexible columns + * [WebUI] Add footable + * [WebUI] Add sender, recipients and subject columns + * [WebUI] Allow message-id break + * [WebUI] Fix history clustering + * [WebUI] Fix history display + * [WebUI] Fix sorting + * [WebUI] Humanize sizes + * [WebUI] Initial move towards footable + * [WebUI] Remove datatables + * [WebUI] Replace `.values` method with `.map` + * [WebUI] Rework v2 symbols display + * [WebUI] Try to normalize frequencies + * [WebUI] Unbreak WebUI + * [WebUI] Use Footable to draw Throughput summary table 1.5.3: - * [Conf] Add composite for hacked wordpress phishing - * [CritFix] Fix base64 decoding when there are unparseable characters - * [Feature] Additional symbol metadata in metadata exporter - * [Feature] Add method to get protocol reply from Lua - * [Feature] Add symbols when tagged rcpt/sender are normalised - * [Feature] Add task:get_symbols_all() function - * [Feature] Allow multiple formats of DKIM signing key - * [Feature] Allow to cache and use flexible protocol reply - * [Feature] Allow to set one_shot flag from register_symbol - * [Feature] Allow to skip certain types of hashes when learning fuzzy - * [Feature] Cache and insert scan time into the protocol - * [Feature] Detect newlines in rspamc --mime - * [Feature] DKIM signing: support use of maps - * [Feature] Greylist: Support excluding low-scoring messages from greylisting - * [Feature] Implement lua history in controller - * [Feature] Implement redis history querying - * [Feature] Preliminary implementation of redis history plugin - * [Feature] Support using request headers in settings - * [Fix] Change default template to deal with non-ASCII characters - * [Fix] Deal with lists of maps in whitelist module - * [Fix] DKIM signing: use domain-specific signing key - * [Fix] Do not reallocate completed zstd buffer - * [Fix] Do not use local_addrs in proxy - * [Fix] Fix crash when resolver is undefined - * [Fix] Fix double free when closing lua_tcp connections - * [Fix] Fix for lua 5.3 - * [Fix] Fix freeing of arrays iterators - * [Fix] Fix issue with task:get_symbol and symbols with no metric - * [Fix] Fix log line duplication in `rspamadm grep` - * [Fix] Fix memory corruption on termination - * [Fix] Fix out-of-bound access in base64 decode - * [Fix] Fix ratelimit + greylisting - * [Fix] Fix subject rewriting - * [Fix] Fix task:set_recipients function - * [Fix] Fix URI_COUNT_ODD rule - * [Fix] Follow the traditional symbols conventions in RCPT_COUNT rule - * [Fix] Greylist: Suppress greylist action for whitelisted hosts too - * [Fix] Metadata exporter: use rule-specific settings for emails - * [Fix] Properly set missing fields in exporter - * [Fix] Proxy: max_retries option - * [Fix] RCPT_COUNT fixes - * [Fix] Rework HAS_X_PRIO rule to match symbols conventions - * [Fix] Update issues in ac-trie - * [Fix] Use optimised base64 decoding in DKIM - * [WebUI] Add preliminary v2 history parser - * [WebUI] Allow different history parsers - * [WebUI] Display symbols - * [WebUI] Rework history v2 function + * [Conf] Add composite for hacked wordpress phishing + * [CritFix] Fix base64 decoding when there are unparseable characters + * [Feature] Additional symbol metadata in metadata exporter + * [Feature] Add method to get protocol reply from Lua + * [Feature] Add symbols when tagged rcpt/sender are normalised + * [Feature] Add task:get_symbols_all() function + * [Feature] Allow multiple formats of DKIM signing key + * [Feature] Allow to cache and use flexible protocol reply + * [Feature] Allow to set one_shot flag from register_symbol + * [Feature] Allow to skip certain types of hashes when learning fuzzy + * [Feature] Cache and insert scan time into the protocol + * [Feature] Detect newlines in rspamc --mime + * [Feature] DKIM signing: support use of maps + * [Feature] Greylist: Support excluding low-scoring messages from greylisting + * [Feature] Implement lua history in controller + * [Feature] Implement redis history querying + * [Feature] Preliminary implementation of redis history plugin + * [Feature] Support using request headers in settings + * [Fix] Change default template to deal with non-ASCII characters + * [Fix] Deal with lists of maps in whitelist module + * [Fix] DKIM signing: use domain-specific signing key + * [Fix] Do not reallocate completed zstd buffer + * [Fix] Do not use local_addrs in proxy + * [Fix] Fix crash when resolver is undefined + * [Fix] Fix double free when closing lua_tcp connections + * [Fix] Fix for lua 5.3 + * [Fix] Fix freeing of arrays iterators + * [Fix] Fix issue with task:get_symbol and symbols with no metric + * [Fix] Fix log line duplication in `rspamadm grep` + * [Fix] Fix memory corruption on termination + * [Fix] Fix out-of-bound access in base64 decode + * [Fix] Fix ratelimit + greylisting + * [Fix] Fix subject rewriting + * [Fix] Fix task:set_recipients function + * [Fix] Fix URI_COUNT_ODD rule + * [Fix] Follow the traditional symbols conventions in RCPT_COUNT rule + * [Fix] Greylist: Suppress greylist action for whitelisted hosts too + * [Fix] Metadata exporter: use rule-specific settings for emails + * [Fix] Properly set missing fields in exporter + * [Fix] Proxy: max_retries option + * [Fix] RCPT_COUNT fixes + * [Fix] Rework HAS_X_PRIO rule to match symbols conventions + * [Fix] Update issues in ac-trie + * [Fix] Use optimised base64 decoding in DKIM + * [WebUI] Add preliminary v2 history parser + * [WebUI] Allow different history parsers + * [WebUI] Display symbols + * [WebUI] Rework history v2 function 1.5.2: - * [Conf] Add default config for spamassasssin plugin - * [Conf] Add default configuration for antivirus module - * [Conf] Add dkim signing docs - * [Conf] Add mx_check default config - * [Conf] Add replies config - * [Conf] Add trie default config - * [Feature] Add heuristic to find text parts in files - * [Feature] Add rule to detect broken content type - * [Feature] Allow to extract CTE in Lua API - * [Feature] Allow to set from address for a lua_task - * [Feature] Allow to set recipients of a task from Lua - * [Feature] Enchance text_part:get_content method - * [Feature] Remove + aliases from emails - * [Feature] Support rmilter block and dkim signature in CGP helper - * [Feature] Support running event loop from Lua - * [Fix] Antivirus: use scanner-specific redis prefix - * [Fix] Couple of fixes for DKIM signing module - * [Fix] Distinguish missing and broken mandatory headers - * [Fix] Do more heuristical detection for missing CTE - * [Fix] Do not resort cache on each check - * [Fix] Fix CGP escaping - * [Fix] Fix MISSING_MIME_VERSION rule for plain messages - * [Fix] Fix parsing of cte in expressions - * [Fix] Fix partial matches in rspamadm grep - * [Fix] Fix setting class on style field - * [WebUI] Auto-switch Throughput units to `msg/min` for very low rate - * [WebUI] Update D3Evolution to 0.0.2 + * [Conf] Add default config for spamassasssin plugin + * [Conf] Add default configuration for antivirus module + * [Conf] Add dkim signing docs + * [Conf] Add mx_check default config + * [Conf] Add replies config + * [Conf] Add trie default config + * [Feature] Add heuristic to find text parts in files + * [Feature] Add rule to detect broken content type + * [Feature] Allow to extract CTE in Lua API + * [Feature] Allow to set from address for a lua_task + * [Feature] Allow to set recipients of a task from Lua + * [Feature] Enchance text_part:get_content method + * [Feature] Remove + aliases from emails + * [Feature] Support rmilter block and dkim signature in CGP helper + * [Feature] Support running event loop from Lua + * [Fix] Antivirus: use scanner-specific redis prefix + * [Fix] Couple of fixes for DKIM signing module + * [Fix] Distinguish missing and broken mandatory headers + * [Fix] Do more heuristical detection for missing CTE + * [Fix] Do not resort cache on each check + * [Fix] Fix CGP escaping + * [Fix] Fix MISSING_MIME_VERSION rule for plain messages + * [Fix] Fix parsing of cte in expressions + * [Fix] Fix partial matches in rspamadm grep + * [Fix] Fix setting class on style field + * [WebUI] Auto-switch Throughput units to `msg/min` for very low rate + * [WebUI] Update D3Evolution to 0.0.2 1.5.1: - * [CritFix] Fix processing of stop_patterns with `\0` character - * [CritFix] Fix setting of raw key for signing - * [Fix] Fix lua exports from plugins during reload - * [Fix] Fix prefilters action scores - * [Fix] Fix symbols processing order - * [Minor] Help cmake find gthread - * [Minor] Some cmake fixes + * [CritFix] Fix processing of stop_patterns with `\0` character + * [CritFix] Fix setting of raw key for signing + * [Fix] Fix lua exports from plugins during reload + * [Fix] Fix prefilters action scores + * [Fix] Fix symbols processing order + * [Minor] Help cmake find gthread + * [Minor] Some cmake fixes 1.5.0: - * [Conf] Add configurations for asn, clickhouse and dcc - * [Conf] Add default config for url redirector plugin - * [Conf] Add the default config for greylist module - * [Conf] Allow to edit all local maps from WebUI by default - * [CritFix] Deal with absent headers in DKIM - * [CritFix] Do not trust remote shingles count - * [CritFix] Fix bad memory leak in TLS certificates validation - * [CritFix] Fix critical memory issues with radix maps - * [CritFix] Fix descriptors leak on reload - * [CritFix] Fix headers selection in DKIM verification - * [CritFix] Fix parsing of boundaries that end with `--` - * [CritFix] Repair PTR_ARRAY_FOREACH macro - * [Feature] Add CORS support to the controller - * [Feature] Add FROM_NAME_EXCESS_SPACE rule - * [Feature] Add REPLYTO_EMAIL_HAS_TITLE rule - * [Feature] Add `caseless_hash` method to `lua_util` - * [Feature] Add `rip` keyword to ratelimit module - * [Feature] Add a simple benchmark for content type parsing - * [Feature] Add boundaries parsing in content type - * [Feature] Add charset detection for text parts - * [Feature] Add content disposition parser - * [Feature] Add fallback if too many updates are failing - * [Feature] Add function to convert struct tm to time using timezone - * [Feature] Add function to normalize HTTP paths - * [Feature] Add fuzzy collection plugin - * [Feature] Add fuzzy logic for images - * [Feature] Add gmime parser to mime_tool - * [Feature] Add heuristic to detect broken messages - * [Feature] Add heuristic to find displayed URLs - * [Feature] Add heuristic to process broken email addresses - * [Feature] Add images normalization - * [Feature] Add mechanism for disabling composites (Fixes #1270) - * [Feature] Add method to create regexp from a glob pattern - * [Feature] Add mime encoding manipulation routines - * [Feature] Add mime tool to explore messages - * [Feature] Add more meta tokens from received headers - * [Feature] Add neighbours option to support Rspamd cluster in WebUI - * [Feature] Add new function to parse mime addresses - * [Feature] Add new methods for lua_tcp - * [Feature] Add own headers decoding routine - * [Feature] Add own routine to generate a message id - * [Feature] Add parser for SMTP date - * [Feature] Add per-task lua cache to reuse 'heavy' objects - * [Feature] Add plugins list path in WebUI - * [Feature] Add preliminary multipart support - * [Feature] Add preliminary version of DKIM signing module - * [Feature] Add profiling support in client output - * [Feature] Add rfc2047 grammar - * [Feature] Add rfc2047 variant for QP decoding - * [Feature] Add rmilter_headers module (Fixes #1227) - * [Feature] Add sse42 version of base64 decoding - * [Feature] Add ssse3 and avx2 base64 decoders - * [Feature] Add support of libgd - * [Feature] Add the preliminary version of redirects resolver in Lua - * [Feature] Add ucl_object_iterate_full function - * [Feature] Add url encoding function - * [Feature] Allow SOA requests in lua dns - * [Feature] Allow custom parse types in lua ucl - * [Feature] Allow plugins to register webui handlers - * [Feature] Allow to add options explicitly to symbols - * [Feature] Allow to call a callback when symbol frequency is on peak - * [Feature] Allow to call redirector script from SURBL - * [Feature] Allow to create variable length dkim keys - * [Feature] Allow to have module specific options for Redis in plugins - * [Feature] Allow to pass sign key directly from Lua - * [Feature] Allow to register configuration docs from Lua API - * [Feature] Allow to return options as a table - * [Feature] Allow to set peak callbacks from Lua - * [Feature] Allow to specify custom method for a message - * [Feature] Allow to store dkim keys in Redis - * [Feature] Allow to store messages in files - * [Feature] Apply DCT using AAN for fuzzy signature - * [Feature] Avira SAVAPI support - * [Feature] Cache and simplify DCT and jpeg decode - * [Feature] Cache libicu converters - * [Feature] Detect URLs with suspicious omographs - * [Feature] Do not increase score for duplicate options - * [Feature] Do not trust CTE, check base64 and qp strictly - * [Feature] Dynamic reputation in URL reputation plugin - * [Feature] Extend redis lock when learning spawned - * [Feature] Filter non-utf chars from all decoded headers - * [Feature] Fix phishing detection for IDNA urls - * [Feature] Ignore bad symbols on base64 decoding - * [Feature] Ignore too wide elements in SPF - * [Feature] Implement fuzzy collection mode - * [Feature] Implement helo maps in multimap - * [Feature] Implement human readable buckets configuration - * [Feature] Implement min-hash shingles for DCT data from images - * [Feature] Implement new algorithm for fuzzy hashes of images - * [Feature] Implement new unicode normalizer - * [Feature] Implement quoted printable decoding - * [Feature] Implement received headers flags - * [Feature] Implement rspamdgrep tool - * [Feature] Implement sane checksum for config file - * [Feature] Implement url tags concept - * [Feature] Improve detection of omographs using libicu - * [Feature] Improve url redirector module - * [Feature] Multimap: Received header processing - * [Feature] Multiple improvements in the maps - * [Feature] New URL filters in multimap - * [Feature] Plugin to force actions on selected symbols - * [Feature] RBL module: support hashing for emails and helo RBL - * [Feature] Reuse URL tags in SURBL module - * [Feature] Rework RRD ds count, add conversion path - * [Feature] Rework surbl module to avoid extra redirector calls - * [Feature] Send config id to the WebUI - * [Feature] Simplify HTTPCrypt client support - * [Feature] Skip processing for large images - * [Feature] Start collection only mode implementation for fuzzy storage - * [Feature] Start import of the optimized base64 decode - * [Feature] Store all received headers in lua - * [Feature] Store relational order of all headers in a message - * [Feature] Support DKIM signing in Lua plugins - * [Feature] Support HTTPCrypt client in lua_http - * [Feature] Support setting SMTP message in multimap - * [Feature] Support setting metric subject from Lua - * [Feature] Support setting subject in force actions module - * [Feature] Treat v6 mapped addresses as v4 addresses - * [Feature] URL reputation plugin - * [Feature] Use Redis instead of memcached in URLs redirector - * [Feature] Use Rspamd rfc2047 decoder instead of gmime one - * [Feature] Use a different normalization for fuzzy images - * [Feature] Use normalized images in fuzzy hashes - * [Feature] Use own code for parsing of date - * [Feature] Use shingles for images fuzzying - * [Feature] Use t1ha for hashes, allow inlining - * [Feature] Use t1ha instead of metrohash and xxhash32 - * [Feature] Various new features in metadata exporter module - * [Feature] rmilter_headers: authentication-results (#78) - * [Fix] Add additional check to mark redis connection inactive - * [Fix] Add packed attribute for protocol structure - * [Fix] Adopt OMOGRAPH_URL rule - * [Fix] Allow static maps - * [Fix] Allow to disable classifiers checks using settings and conditions - * [Fix] Another try to fix 0 length maps - * [Fix] Another try to fix corruption during maps reload - * [Fix] Another try to fix descriptors leak - * [Fix] Another try to fix reload and logger - * [Fix] Antivirus module: register virtual symbols for patterns - * [Fix] Avoid extensive reallocs - * [Fix] Avoid mempool leak in SA plugin on reload - * [Fix] Avoid race condition on saving cache and reload - * [Fix] Avoid reusing g_error (Fixes #1262) - * [Fix] Break pool connection on fatal redis errors - * [Fix] Check for NaN properly - * [Fix] Couple of fixes for date parsing - * [Fix] Date header timezone adjustments (#1279) - * [Fix] Deal with EOF properly - * [Fix] Decode filename in content disposition - * [Fix] Disable fuzzy images by default - * [Fix] Disable zero-copy mode for text parts to avoid crashes - * [Fix] Do not destroy session when not all finish scripts are done - * [Fix] Do not greyscale images - * [Fix] Do not leave parent-less workers processes on fatal errors - * [Fix] Do not lowercase Content-Disposition to perform decoding - * [Fix] Do not penalize characters just after numeric prefix - * [Fix] Do not refork workers that are intended to die - * [Fix] Do not set pre-result and update records for no `Queue-ID` messages - * [Fix] Do not skip post-filters when pre-filters have set some results - * [Fix] Do not stop symbols planning if async events are pending - * [Fix] Do not try to set keys for unencrypted requests in proxy - * [Fix] Encode URLs according to rfc3986 - * [Fix] Encode URLs before sending them to the protocol - * [Fix] Filter bad characters from message id - * [Fix] Fix CTE detection heuristic - * [Fix] Fix Content-Type in HTTP requests - * [Fix] Fix IDN eslds phishing checks - * [Fix] Fix adding maps from config in Lua - * [Fix] Fix another reload memory issue - * [Fix] Fix argument returned on redis backend errors - * [Fix] Fix assertion in graph handling - * [Fix] Fix body trie matching - * [Fix] Fix build - * [Fix] Fix byte array expansion during toutf8 conversion - * [Fix] Fix charset normalisation - * [Fix] Fix checking of DKIM bodies that needs just `\n` to be added - * [Fix] Fix couple of cornercases with email addresses - * [Fix] Fix couple of issues - * [Fix] Fix dependencies tracking for callback symbols - * [Fix] Fix detection of jpeg size - * [Fix] Fix errors handling in fuzzy backend initialization - * [Fix] Fix fuzzy hashes count - * [Fix] Fix globbing and convert lists to arrays in fuzzy_check - * [Fix] Fix heuristical CTE detection for QP encoding - * [Fix] Fix ignoring of bad text parts - * [Fix] Fix indexes in array access, interleave loop - * [Fix] Fix int64 -> double conversion - * [Fix] Fix invalid memory access on reload - * [Fix] Fix issues with empty updates - * [Fix] Fix issues with quoted-printable encoding - * [Fix] Fix keys names - * [Fix] Fix lots of issues in mime parser code - * [Fix] Fix lua maps load - * [Fix] Fix macro name - * [Fix] Fix mas group score calculations - * [Fix] Fix matching of the same patterns from different tries - * [Fix] Fix memory corruprtion and leak - * [Fix] Fix memory leak in HTTP maps - * [Fix] Fix memory leak in expression destroying - * [Fix] Fix memory leak in parsing of mime names - * [Fix] Fix memory leak in safe ucl iterators - * [Fix] Fix memory leak on reload in plugins - * [Fix] Fix modules reconfigure on reload - * [Fix] Fix monitored setup fro URLBLs with IP addresses - * [Fix] Fix name of var - * [Fix] Fix new rrd updates - * [Fix] Fix out of bounds access - * [Fix] Fix parsing messages with no body - * [Fix] Fix parsing of '=' character in headers - * [Fix] Fix parsing of messages with no content type - * [Fix] Fix plugins callbacks in webui - * [Fix] Fix possible memory corruption in redis pool - * [Fix] Fix probability calculations for fuzzy redis backend - * [Fix] Fix processing errors in lua_tcp - * [Fix] Fix processing of emails with name only - * [Fix] Fix processing of non-multipart messages - * [Fix] Fix processing of parts with no valid content type - * [Fix] Fix race condition in SIGUSR2 handler - * [Fix] Fix redis options parsing when no redis servers are defined - * [Fix] Fix reload and hyperscan ready event - * [Fix] Fix reload memory issue - * [Fix] Fix rra_ptr conversion - * [Fix] Fix rrd file conversion - * [Fix] Fix setting of content-type attributes - * [Fix] Fix signing headers creation in DKIM - * [Fix] Fix stddev calculations - * [Fix] Fix surbl plugin to work with composite maps - * [Fix] Fix timezones parsing - * [Fix] Fix tokens usage - * [Fix] Fix urls and emails hashes - * [Fix] Fix usage of unsafe ucl iterators - * [Fix] Fix work with broken utf8 tokens - * [Fix] Fix writing of user to roll history - * [Fix] Forgotten worker - * [Fix] Further memory leaks fixes - * [Fix] Ignore lua metatokens in bayes for now - * [Fix] Improve OMOGRAPH_URL rule - * [Fix] Lua IP from string should be invalid if parsing failed - * [Fix] Miltiple fixes to new lua_tcp, add debugging - * [Fix] More fixes for iterators cleanup - * [Fix] More fixes to logger initialization - * [Fix] More heuristic fixes for phishing detection - * [Fix] More leaks eliminated - * [Fix] More leaks... - * [Fix] More random fixes for reload... - * [Fix] Multimap: Fixes for email filters - * [Fix] Multiple fixes for fann module - * [Fix] Multiple memory corruption fixes - * [Fix] Normalize path in HTTP router - * [Fix] Plug memory leak - * [Fix] Plug memory leak in adding radix trees - * [Fix] Plug memory leak in configuration parser - * [Fix] Plug memory leak in expressions parsing during reload - * [Fix] Plug memory leak in learning fuzzy storage - * [Fix] Plug memory leak in lua_tcp - * [Fix] Plug reload leaks - * [Fix] Plug termination memory leaks - * [Fix] Really increase lock lifetime - * [Fix] Replies module: fix symbol weight - * [Fix] Restore content type params related functions - * [Fix] Set task's subject from mime subject - * [Fix] Sigh, one more reload leak - * [Fix] Simplify images shingles - * [Fix] Some more memory issues are fixed - * [Fix] Stop hardcoding of lua in C - * [Fix] Stop processing of bad parts as text parts - * [Fix] Strictly filter bad characters when emittin json - * [Fix] Strings returned from lua are ephemeral - * [Fix] Support unix sockets for lua redis - * [Fix] Try to fix issues with reloading config - * [Fix] Try to fix race condition in redis_pool - * [Fix] Use checksum to avoid intersection between different ANNs - * [Fix] Use rspamd hashes in embedded ucl - * [Fix] Use sane default rewrite subject (*** SPAM *** %s) - * [Fix] Various collection mode fixes - * [Fix] Various fixes to mime parser - * [Fix] Various reload leak fixing - * [Fix] Whitelist certain extensions from archive checks - * [Rework] Add preliminary implementation of the mime parser - * [Rework] Adopt code for the new options - * [Rework] Change logger setup interface - * [Rework] Composite configuration (#1270) - * [Rework] Finally remove gmime dependency from Rspamd - * [Rework] Further fixes to symbols frequencies - * [Rework] Implement content type parser for mime - * [Rework] Kill all InternetAddressList usages - * [Rework] Multiple fixes for symbols cache statistics - * [Rework] Refactor struct names - * [Rework] Rework images fuzzy hashes algorithm - * [Rework] Rework lua_tcp to allow TCP dialog - * [Rework] Start massive rework to get rid of gmime - * [Rework] Start new approach for multiparts parsing - * [Rework] Start rework of mime addresses - * [Rework] Start rework of symbols cache updates - * [Rework] Start switching to libicu - * [Rework] Use a special structure for stats tokens - * [Rework] Use hash tables for symbols options - * [Rework] Use libicu instead of iconv for conversions - * [Rework] Use new scheme to parse mime parts - * [WebUI] Add Access-Control-Allow-Origin for cluster management - * [WebUI] Add Throughput graph autorefreshing (#820) - * [WebUI] Add Visibility.js library - * [WebUI] Add basic cluster support to Throughput tab - * [WebUI] Add graph legend entries for new DSes - * [WebUI] Add graph tab - * [WebUI] Add neighbours RRD data consolidation - * [WebUI] Add preliminary save symbols clustering - * [WebUI] Add server selector to navbar - * [WebUI] Add soft reject to auth stats - * [WebUI] Add summary to the Throughput tab - * [WebUI] Allow to save maps on the cluster - * [WebUI] Avoid extra graph redraw and alerts glitching - * [WebUI] Be more generous with AJAX timeout - * [WebUI] Disable error ring loading in `read only` mode - * [WebUI] Enclose table header cells with `tr`s - * [WebUI] Finish interface rework - * [WebUI] Fix RRD summary pie chart position - * [WebUI] Fix `All SERVERS` graph fot just one available server - * [WebUI] Fix case when no cluster is defined - * [WebUI] Fix compatibility with non-ES6 compliant browsers - * [WebUI] Fix config ID - * [WebUI] Fix configuration page partially - * [WebUI] Fix disabled state - * [WebUI] Fix graph dataset selector initialization - * [WebUI] Fix graph selectors state resetting - * [WebUI] Fix mouse events on throughput summary table area - * [WebUI] Fix multiple JS issues - * [WebUI] Fix pie chart displaying - * [WebUI] Fix read only - * [WebUI] Fix read only2 - * [WebUI] Fix retarded datatables - * [WebUI] Fix soft reject in pie chart - * [WebUI] Fix stat widgets timers multiplication on `Refresh` click - * [WebUI] Fix symbols config - * [WebUI] Fix various errors with login form - * [WebUI] Further fixes - * [WebUI] Hide learning tab in read-only mode - * [WebUI] Initial clusters support - * [WebUI] Make legend entry colours more contrast - * [WebUI] Move configuration tab to a separate module - * [WebUI] Move history tab - * [WebUI] Move symbols config as well - * [WebUI] New sec to time function - * [WebUI] Prevent multiple clicks on `Refresh` - * [WebUI] RRD summary: Hide inner labels of tiny pie sectors - * [WebUI] RRD summary: Respect undefined values - * [WebUI] Reduce font size of graph's legend - * [WebUI] Remove orphaned font duplicates - * [WebUI] Remove unused code - * [WebUI] Replace spinner with animated glyphicon - * [WebUI] Reset refresh timer on server switching - * [WebUI] Rework interface to use requirejs - * [WebUI] Rework neighbours query function - * [WebUI] Separate attributes by space - * [WebUI] Set focus to password field (#1230) - * [WebUI] Simplify neighbours table populating - * [WebUI] Start rework of modules - * [WebUI] Stop stats refreshing if the page is hidden - * [WebUI] Turn d3pie's stuff into a reusable function, - * [WebUI] Unify send data functions - * [WebUI] Update D3Evolution to 0.0.1 - * [WebUI] Update d3.js - * [WebUI] Update datatables to work with the requirejs - * [WebUI] Use unified tab click event handler, - * [WebUI] clusters for the chart - * [WebUI] fix uptime + * [Conf] Add configurations for asn, clickhouse and dcc + * [Conf] Add default config for url redirector plugin + * [Conf] Add the default config for greylist module + * [Conf] Allow to edit all local maps from WebUI by default + * [CritFix] Deal with absent headers in DKIM + * [CritFix] Do not trust remote shingles count + * [CritFix] Fix bad memory leak in TLS certificates validation + * [CritFix] Fix critical memory issues with radix maps + * [CritFix] Fix descriptors leak on reload + * [CritFix] Fix headers selection in DKIM verification + * [CritFix] Fix parsing of boundaries that end with `--` + * [CritFix] Repair PTR_ARRAY_FOREACH macro + * [Feature] Add CORS support to the controller + * [Feature] Add FROM_NAME_EXCESS_SPACE rule + * [Feature] Add REPLYTO_EMAIL_HAS_TITLE rule + * [Feature] Add `caseless_hash` method to `lua_util` + * [Feature] Add `rip` keyword to ratelimit module + * [Feature] Add a simple benchmark for content type parsing + * [Feature] Add boundaries parsing in content type + * [Feature] Add charset detection for text parts + * [Feature] Add content disposition parser + * [Feature] Add fallback if too many updates are failing + * [Feature] Add function to convert struct tm to time using timezone + * [Feature] Add function to normalize HTTP paths + * [Feature] Add fuzzy collection plugin + * [Feature] Add fuzzy logic for images + * [Feature] Add gmime parser to mime_tool + * [Feature] Add heuristic to detect broken messages + * [Feature] Add heuristic to find displayed URLs + * [Feature] Add heuristic to process broken email addresses + * [Feature] Add images normalization + * [Feature] Add mechanism for disabling composites (Fixes #1270) + * [Feature] Add method to create regexp from a glob pattern + * [Feature] Add mime encoding manipulation routines + * [Feature] Add mime tool to explore messages + * [Feature] Add more meta tokens from received headers + * [Feature] Add neighbours option to support Rspamd cluster in WebUI + * [Feature] Add new function to parse mime addresses + * [Feature] Add new methods for lua_tcp + * [Feature] Add own headers decoding routine + * [Feature] Add own routine to generate a message id + * [Feature] Add parser for SMTP date + * [Feature] Add per-task lua cache to reuse 'heavy' objects + * [Feature] Add plugins list path in WebUI + * [Feature] Add preliminary multipart support + * [Feature] Add preliminary version of DKIM signing module + * [Feature] Add profiling support in client output + * [Feature] Add rfc2047 grammar + * [Feature] Add rfc2047 variant for QP decoding + * [Feature] Add rmilter_headers module (Fixes #1227) + * [Feature] Add sse42 version of base64 decoding + * [Feature] Add ssse3 and avx2 base64 decoders + * [Feature] Add support of libgd + * [Feature] Add the preliminary version of redirects resolver in Lua + * [Feature] Add ucl_object_iterate_full function + * [Feature] Add url encoding function + * [Feature] Allow SOA requests in lua dns + * [Feature] Allow custom parse types in lua ucl + * [Feature] Allow plugins to register webui handlers + * [Feature] Allow to add options explicitly to symbols + * [Feature] Allow to call a callback when symbol frequency is on peak + * [Feature] Allow to call redirector script from SURBL + * [Feature] Allow to create variable length dkim keys + * [Feature] Allow to have module specific options for Redis in plugins + * [Feature] Allow to pass sign key directly from Lua + * [Feature] Allow to register configuration docs from Lua API + * [Feature] Allow to return options as a table + * [Feature] Allow to set peak callbacks from Lua + * [Feature] Allow to specify custom method for a message + * [Feature] Allow to store dkim keys in Redis + * [Feature] Allow to store messages in files + * [Feature] Apply DCT using AAN for fuzzy signature + * [Feature] Avira SAVAPI support + * [Feature] Cache and simplify DCT and jpeg decode + * [Feature] Cache libicu converters + * [Feature] Detect URLs with suspicious omographs + * [Feature] Do not increase score for duplicate options + * [Feature] Do not trust CTE, check base64 and qp strictly + * [Feature] Dynamic reputation in URL reputation plugin + * [Feature] Extend redis lock when learning spawned + * [Feature] Filter non-utf chars from all decoded headers + * [Feature] Fix phishing detection for IDNA urls + * [Feature] Ignore bad symbols on base64 decoding + * [Feature] Ignore too wide elements in SPF + * [Feature] Implement fuzzy collection mode + * [Feature] Implement helo maps in multimap + * [Feature] Implement human readable buckets configuration + * [Feature] Implement min-hash shingles for DCT data from images + * [Feature] Implement new algorithm for fuzzy hashes of images + * [Feature] Implement new unicode normalizer + * [Feature] Implement quoted printable decoding + * [Feature] Implement received headers flags + * [Feature] Implement rspamdgrep tool + * [Feature] Implement sane checksum for config file + * [Feature] Implement url tags concept + * [Feature] Improve detection of omographs using libicu + * [Feature] Improve url redirector module + * [Feature] Multimap: Received header processing + * [Feature] Multiple improvements in the maps + * [Feature] New URL filters in multimap + * [Feature] Plugin to force actions on selected symbols + * [Feature] RBL module: support hashing for emails and helo RBL + * [Feature] Reuse URL tags in SURBL module + * [Feature] Rework RRD ds count, add conversion path + * [Feature] Rework surbl module to avoid extra redirector calls + * [Feature] Send config id to the WebUI + * [Feature] Simplify HTTPCrypt client support + * [Feature] Skip processing for large images + * [Feature] Start collection only mode implementation for fuzzy storage + * [Feature] Start import of the optimized base64 decode + * [Feature] Store all received headers in lua + * [Feature] Store relational order of all headers in a message + * [Feature] Support DKIM signing in Lua plugins + * [Feature] Support HTTPCrypt client in lua_http + * [Feature] Support setting SMTP message in multimap + * [Feature] Support setting metric subject from Lua + * [Feature] Support setting subject in force actions module + * [Feature] Treat v6 mapped addresses as v4 addresses + * [Feature] URL reputation plugin + * [Feature] Use Redis instead of memcached in URLs redirector + * [Feature] Use Rspamd rfc2047 decoder instead of gmime one + * [Feature] Use a different normalization for fuzzy images + * [Feature] Use normalized images in fuzzy hashes + * [Feature] Use own code for parsing of date + * [Feature] Use shingles for images fuzzying + * [Feature] Use t1ha for hashes, allow inlining + * [Feature] Use t1ha instead of metrohash and xxhash32 + * [Feature] Various new features in metadata exporter module + * [Feature] rmilter_headers: authentication-results (#78) + * [Fix] Add additional check to mark redis connection inactive + * [Fix] Add packed attribute for protocol structure + * [Fix] Adopt OMOGRAPH_URL rule + * [Fix] Allow static maps + * [Fix] Allow to disable classifiers checks using settings and conditions + * [Fix] Another try to fix 0 length maps + * [Fix] Another try to fix corruption during maps reload + * [Fix] Another try to fix descriptors leak + * [Fix] Another try to fix reload and logger + * [Fix] Antivirus module: register virtual symbols for patterns + * [Fix] Avoid extensive reallocs + * [Fix] Avoid mempool leak in SA plugin on reload + * [Fix] Avoid race condition on saving cache and reload + * [Fix] Avoid reusing g_error (Fixes #1262) + * [Fix] Break pool connection on fatal redis errors + * [Fix] Check for NaN properly + * [Fix] Couple of fixes for date parsing + * [Fix] Date header timezone adjustments (#1279) + * [Fix] Deal with EOF properly + * [Fix] Decode filename in content disposition + * [Fix] Disable fuzzy images by default + * [Fix] Disable zero-copy mode for text parts to avoid crashes + * [Fix] Do not destroy session when not all finish scripts are done + * [Fix] Do not greyscale images + * [Fix] Do not leave parent-less workers processes on fatal errors + * [Fix] Do not lowercase Content-Disposition to perform decoding + * [Fix] Do not penalize characters just after numeric prefix + * [Fix] Do not refork workers that are intended to die + * [Fix] Do not set pre-result and update records for no `Queue-ID` messages + * [Fix] Do not skip post-filters when pre-filters have set some results + * [Fix] Do not stop symbols planning if async events are pending + * [Fix] Do not try to set keys for unencrypted requests in proxy + * [Fix] Encode URLs according to rfc3986 + * [Fix] Encode URLs before sending them to the protocol + * [Fix] Filter bad characters from message id + * [Fix] Fix CTE detection heuristic + * [Fix] Fix Content-Type in HTTP requests + * [Fix] Fix IDN eslds phishing checks + * [Fix] Fix adding maps from config in Lua + * [Fix] Fix another reload memory issue + * [Fix] Fix argument returned on redis backend errors + * [Fix] Fix assertion in graph handling + * [Fix] Fix body trie matching + * [Fix] Fix build + * [Fix] Fix byte array expansion during toutf8 conversion + * [Fix] Fix charset normalisation + * [Fix] Fix checking of DKIM bodies that needs just `\n` to be added + * [Fix] Fix couple of cornercases with email addresses + * [Fix] Fix couple of issues + * [Fix] Fix dependencies tracking for callback symbols + * [Fix] Fix detection of jpeg size + * [Fix] Fix errors handling in fuzzy backend initialization + * [Fix] Fix fuzzy hashes count + * [Fix] Fix globbing and convert lists to arrays in fuzzy_check + * [Fix] Fix heuristical CTE detection for QP encoding + * [Fix] Fix ignoring of bad text parts + * [Fix] Fix indexes in array access, interleave loop + * [Fix] Fix int64 -> double conversion + * [Fix] Fix invalid memory access on reload + * [Fix] Fix issues with empty updates + * [Fix] Fix issues with quoted-printable encoding + * [Fix] Fix keys names + * [Fix] Fix lots of issues in mime parser code + * [Fix] Fix lua maps load + * [Fix] Fix macro name + * [Fix] Fix mas group score calculations + * [Fix] Fix matching of the same patterns from different tries + * [Fix] Fix memory corruprtion and leak + * [Fix] Fix memory leak in HTTP maps + * [Fix] Fix memory leak in expression destroying + * [Fix] Fix memory leak in parsing of mime names + * [Fix] Fix memory leak in safe ucl iterators + * [Fix] Fix memory leak on reload in plugins + * [Fix] Fix modules reconfigure on reload + * [Fix] Fix monitored setup fro URLBLs with IP addresses + * [Fix] Fix name of var + * [Fix] Fix new rrd updates + * [Fix] Fix out of bounds access + * [Fix] Fix parsing messages with no body + * [Fix] Fix parsing of '=' character in headers + * [Fix] Fix parsing of messages with no content type + * [Fix] Fix plugins callbacks in webui + * [Fix] Fix possible memory corruption in redis pool + * [Fix] Fix probability calculations for fuzzy redis backend + * [Fix] Fix processing errors in lua_tcp + * [Fix] Fix processing of emails with name only + * [Fix] Fix processing of non-multipart messages + * [Fix] Fix processing of parts with no valid content type + * [Fix] Fix race condition in SIGUSR2 handler + * [Fix] Fix redis options parsing when no redis servers are defined + * [Fix] Fix reload and hyperscan ready event + * [Fix] Fix reload memory issue + * [Fix] Fix rra_ptr conversion + * [Fix] Fix rrd file conversion + * [Fix] Fix setting of content-type attributes + * [Fix] Fix signing headers creation in DKIM + * [Fix] Fix stddev calculations + * [Fix] Fix surbl plugin to work with composite maps + * [Fix] Fix timezones parsing + * [Fix] Fix tokens usage + * [Fix] Fix urls and emails hashes + * [Fix] Fix usage of unsafe ucl iterators + * [Fix] Fix work with broken utf8 tokens + * [Fix] Fix writing of user to roll history + * [Fix] Forgotten worker + * [Fix] Further memory leaks fixes + * [Fix] Ignore lua metatokens in bayes for now + * [Fix] Improve OMOGRAPH_URL rule + * [Fix] Lua IP from string should be invalid if parsing failed + * [Fix] Miltiple fixes to new lua_tcp, add debugging + * [Fix] More fixes for iterators cleanup + * [Fix] More fixes to logger initialization + * [Fix] More heuristic fixes for phishing detection + * [Fix] More leaks eliminated + * [Fix] More leaks... + * [Fix] More random fixes for reload... + * [Fix] Multimap: Fixes for email filters + * [Fix] Multiple fixes for fann module + * [Fix] Multiple memory corruption fixes + * [Fix] Normalize path in HTTP router + * [Fix] Plug memory leak + * [Fix] Plug memory leak in adding radix trees + * [Fix] Plug memory leak in configuration parser + * [Fix] Plug memory leak in expressions parsing during reload + * [Fix] Plug memory leak in learning fuzzy storage + * [Fix] Plug memory leak in lua_tcp + * [Fix] Plug reload leaks + * [Fix] Plug termination memory leaks + * [Fix] Really increase lock lifetime + * [Fix] Replies module: fix symbol weight + * [Fix] Restore content type params related functions + * [Fix] Set task's subject from mime subject + * [Fix] Sigh, one more reload leak + * [Fix] Simplify images shingles + * [Fix] Some more memory issues are fixed + * [Fix] Stop hardcoding of lua in C + * [Fix] Stop processing of bad parts as text parts + * [Fix] Strictly filter bad characters when emittin json + * [Fix] Strings returned from lua are ephemeral + * [Fix] Support unix sockets for lua redis + * [Fix] Try to fix issues with reloading config + * [Fix] Try to fix race condition in redis_pool + * [Fix] Use checksum to avoid intersection between different ANNs + * [Fix] Use rspamd hashes in embedded ucl + * [Fix] Use sane default rewrite subject (*** SPAM *** %s) + * [Fix] Various collection mode fixes + * [Fix] Various fixes to mime parser + * [Fix] Various reload leak fixing + * [Fix] Whitelist certain extensions from archive checks + * [Rework] Add preliminary implementation of the mime parser + * [Rework] Adopt code for the new options + * [Rework] Change logger setup interface + * [Rework] Composite configuration (#1270) + * [Rework] Finally remove gmime dependency from Rspamd + * [Rework] Further fixes to symbols frequencies + * [Rework] Implement content type parser for mime + * [Rework] Kill all InternetAddressList usages + * [Rework] Multiple fixes for symbols cache statistics + * [Rework] Refactor struct names + * [Rework] Rework images fuzzy hashes algorithm + * [Rework] Rework lua_tcp to allow TCP dialog + * [Rework] Start massive rework to get rid of gmime + * [Rework] Start new approach for multiparts parsing + * [Rework] Start rework of mime addresses + * [Rework] Start rework of symbols cache updates + * [Rework] Start switching to libicu + * [Rework] Use a special structure for stats tokens + * [Rework] Use hash tables for symbols options + * [Rework] Use libicu instead of iconv for conversions + * [Rework] Use new scheme to parse mime parts + * [WebUI] Add Access-Control-Allow-Origin for cluster management + * [WebUI] Add Throughput graph autorefreshing (#820) + * [WebUI] Add Visibility.js library + * [WebUI] Add basic cluster support to Throughput tab + * [WebUI] Add graph legend entries for new DSes + * [WebUI] Add graph tab + * [WebUI] Add neighbours RRD data consolidation + * [WebUI] Add preliminary save symbols clustering + * [WebUI] Add server selector to navbar + * [WebUI] Add soft reject to auth stats + * [WebUI] Add summary to the Throughput tab + * [WebUI] Allow to save maps on the cluster + * [WebUI] Avoid extra graph redraw and alerts glitching + * [WebUI] Be more generous with AJAX timeout + * [WebUI] Disable error ring loading in `read only` mode + * [WebUI] Enclose table header cells with `tr`s + * [WebUI] Finish interface rework + * [WebUI] Fix RRD summary pie chart position + * [WebUI] Fix `All SERVERS` graph fot just one available server + * [WebUI] Fix case when no cluster is defined + * [WebUI] Fix compatibility with non-ES6 compliant browsers + * [WebUI] Fix config ID + * [WebUI] Fix configuration page partially + * [WebUI] Fix disabled state + * [WebUI] Fix graph dataset selector initialization + * [WebUI] Fix graph selectors state resetting + * [WebUI] Fix mouse events on throughput summary table area + * [WebUI] Fix multiple JS issues + * [WebUI] Fix pie chart displaying + * [WebUI] Fix read only + * [WebUI] Fix read only2 + * [WebUI] Fix retarded datatables + * [WebUI] Fix soft reject in pie chart + * [WebUI] Fix stat widgets timers multiplication on `Refresh` click + * [WebUI] Fix symbols config + * [WebUI] Fix various errors with login form + * [WebUI] Further fixes + * [WebUI] Hide learning tab in read-only mode + * [WebUI] Initial clusters support + * [WebUI] Make legend entry colours more contrast + * [WebUI] Move configuration tab to a separate module + * [WebUI] Move history tab + * [WebUI] Move symbols config as well + * [WebUI] New sec to time function + * [WebUI] Prevent multiple clicks on `Refresh` + * [WebUI] RRD summary: Hide inner labels of tiny pie sectors + * [WebUI] RRD summary: Respect undefined values + * [WebUI] Reduce font size of graph's legend + * [WebUI] Remove orphaned font duplicates + * [WebUI] Remove unused code + * [WebUI] Replace spinner with animated glyphicon + * [WebUI] Reset refresh timer on server switching + * [WebUI] Rework interface to use requirejs + * [WebUI] Rework neighbours query function + * [WebUI] Separate attributes by space + * [WebUI] Set focus to password field (#1230) + * [WebUI] Simplify neighbours table populating + * [WebUI] Start rework of modules + * [WebUI] Stop stats refreshing if the page is hidden + * [WebUI] Turn d3pie's stuff into a reusable function, + * [WebUI] Unify send data functions + * [WebUI] Update D3Evolution to 0.0.1 + * [WebUI] Update d3.js + * [WebUI] Update datatables to work with the requirejs + * [WebUI] Use unified tab click event handler, + * [WebUI] clusters for the chart + * [WebUI] fix uptime 1.4.2: - * [CritFix] Deal with absent headers in DKIM - * [CritFix] Do not trust remote shingles count - * [CritFix] Fix headers selection in DKIM verification - * [Feature] Add EXT_CSS rule - * [Feature] Add toggle for disabling SURBLs - * [Feature] Extend redis lock when learning spawned - * [Feature] Parse HTML tags - * [Fix] Avoid reusing g_error (Fixes #1262) - * [Fix] Do not reset loaded ANN when learning is requested - * [Fix] Fix another issue with external deps in SA - * [Fix] Fix body trie matching - * [Fix] Fix checking of DKIM bodies that needs just `\n` to be added - * [Fix] Fix fuzzy hashes count - * [Fix] Fix keys names - * [Fix] Fix length calculations for url encoded urls - * [Fix] Fix matching of the same patterns from different tries - * [Fix] Fix name of var - * [Fix] Fix parsing of URLs with spaces and other bad chars - * [Fix] Fix probability calculations for fuzzy redis backend - * [Fix] Fix signing headers creation in DKIM - * [Fix] Plug memory leak - * [Fix] Really fix chained SA dependencies - * [Fix] Really increase lock lifetime - * [Fix] Use checksum to avoid intersection between different ANNs - * [Fix] Use rspamd hashes in embedded ucl - * [Fix] Yet another change for testing external deps + * [CritFix] Deal with absent headers in DKIM + * [CritFix] Do not trust remote shingles count + * [CritFix] Fix headers selection in DKIM verification + * [Feature] Add EXT_CSS rule + * [Feature] Add toggle for disabling SURBLs + * [Feature] Extend redis lock when learning spawned + * [Feature] Parse HTML tags + * [Fix] Avoid reusing g_error (Fixes #1262) + * [Fix] Do not reset loaded ANN when learning is requested + * [Fix] Fix another issue with external deps in SA + * [Fix] Fix body trie matching + * [Fix] Fix checking of DKIM bodies that needs just `\n` to be added + * [Fix] Fix fuzzy hashes count + * [Fix] Fix keys names + * [Fix] Fix length calculations for url encoded urls + * [Fix] Fix matching of the same patterns from different tries + * [Fix] Fix name of var + * [Fix] Fix parsing of URLs with spaces and other bad chars + * [Fix] Fix probability calculations for fuzzy redis backend + * [Fix] Fix signing headers creation in DKIM + * [Fix] Plug memory leak + * [Fix] Really fix chained SA dependencies + * [Fix] Really increase lock lifetime + * [Fix] Use checksum to avoid intersection between different ANNs + * [Fix] Use rspamd hashes in embedded ucl + * [Fix] Yet another change for testing external deps 1.4.1: - * [Feature] ASN support in Clickhouse module - * [Feature] Add clickhouse plugin - * [Feature] Add generic tool to add universal maps for lua modules - * [Feature] Add logger.debugm to debug lua modules - * [Feature] Allow to register metrics symbols using register_symbol - * [Feature] Allow to specify prefix for fann_redis - * [Feature] Clickhouse: support different masks for IPv4/IPv6 - * [Feature] Support forcing action in antivirus plugin - * [Fix] Add handling of regexp maps - * [Fix] Allow backslashes in http urls - * [Fix] Avoid mapping of empty files - * [Fix] Do not load tld file to speed up rspamadm - * [Fix] Do not resolve numeric IP addresses due to ipv6 insanity - * [Fix] Filter incorrect training data - * [Fix] Fix Fuzzyconvert tool when password or DB is given - * [Fix] Fix build with custom glib/gmime - * [Fix] Fix converting of learn count from sqlite to redis - * [Fix] Fix crashes with invalid received and task:set_from_ip - * [Fix] Fix external dependencies for SA module - * [Fix] Fix fann_redis when number of scores has been changed - * [Fix] Fix hyperscan usage for non compatible platforms - * [Fix] Fix loading of maps from UCL objects - * [Fix] Fix memory leak for task-less redis requests - * [Fix] Fix mid module with new maps syntax - * [Fix] Fix parsing of URLs with username - * [Fix] Fix re cache initialisation - * [Fix] Fix replacements to sanitize '%' character - * [Fix] Fix set and regexp like static maps - * [Fix] Fix some issues in redis settings - * [Fix] Fix static IP maps - * [Fix] Fix total learns counter for redis stats - * [Fix] Fix usage of config during reload - * [Fix] Fix various warnings and issues - * [Fix] Invalidate ANN if training data is incorrect - * [Fix] Miltiple fixes to fann_redis module - * [Fix] More fixes for URLs with backslashes - * [Fix] Properly get options for ip_score module - * [Fix] Relax requirements for Received as gmail cannot RFC - * [Fix] Remove or fix hyperscan incompatible regexps - * [Fix] Settings: correctly read redis config - * [Rework] Rework lua logger interface slightly - * [Rework] Use new maps add function + * [Feature] ASN support in Clickhouse module + * [Feature] Add clickhouse plugin + * [Feature] Add generic tool to add universal maps for lua modules + * [Feature] Add logger.debugm to debug lua modules + * [Feature] Allow to register metrics symbols using register_symbol + * [Feature] Allow to specify prefix for fann_redis + * [Feature] Clickhouse: support different masks for IPv4/IPv6 + * [Feature] Support forcing action in antivirus plugin + * [Fix] Add handling of regexp maps + * [Fix] Allow backslashes in http urls + * [Fix] Avoid mapping of empty files + * [Fix] Do not load tld file to speed up rspamadm + * [Fix] Do not resolve numeric IP addresses due to ipv6 insanity + * [Fix] Filter incorrect training data + * [Fix] Fix Fuzzyconvert tool when password or DB is given + * [Fix] Fix build with custom glib/gmime + * [Fix] Fix converting of learn count from sqlite to redis + * [Fix] Fix crashes with invalid received and task:set_from_ip + * [Fix] Fix external dependencies for SA module + * [Fix] Fix fann_redis when number of scores has been changed + * [Fix] Fix hyperscan usage for non compatible platforms + * [Fix] Fix loading of maps from UCL objects + * [Fix] Fix memory leak for task-less redis requests + * [Fix] Fix mid module with new maps syntax + * [Fix] Fix parsing of URLs with username + * [Fix] Fix re cache initialisation + * [Fix] Fix replacements to sanitize '%' character + * [Fix] Fix set and regexp like static maps + * [Fix] Fix some issues in redis settings + * [Fix] Fix static IP maps + * [Fix] Fix total learns counter for redis stats + * [Fix] Fix usage of config during reload + * [Fix] Fix various warnings and issues + * [Fix] Invalidate ANN if training data is incorrect + * [Fix] Miltiple fixes to fann_redis module + * [Fix] More fixes for URLs with backslashes + * [Fix] Properly get options for ip_score module + * [Fix] Relax requirements for Received as gmail cannot RFC + * [Fix] Remove or fix hyperscan incompatible regexps + * [Fix] Settings: correctly read redis config + * [Rework] Rework lua logger interface slightly + * [Rework] Use new maps add function 1.4.0: - * [CritFix] Add guards for inactive redis connections - * [CritFix] Another fix for proxying files using rspamd_proxy - * [CritFix] Cleanup inactive redis connections - * [CritFix] Do not sometimes try to exec posfilters before classification - * [CritFix] Fix application of IPv6 mask - * [CritFix] Fix chunked encoding when reading messages - * [CritFix] Fix file mode for rspamd_proxy - * [CritFix] Fix hyperscan compilation on regexp change - * [CritFix] Fix issue with finding of end of lines pointers - * [CritFix] Fix iteration over headers array (introduced in 1.4) - * [CritFix] Fix processing of learned tokens count for redis backend - * [CritFix] Fix race condition in checking of cached maps - * [CritFix] Fix workers scripts by sharing workers configs - * [CritFix] Introduce raw content to text parts - * [CritFix] Plug memory leak and potential memory corruption - * [Feature] Adaptive ratelimits - * [Feature] Add ASN -> rbldnsd script for asn.rspamd.com - * [Feature] Add DMARC_NA symbol - * [Feature] Add F-Prot support to antivirus module - * [Feature] Add HTTP backend to metadata exporter - * [Feature] Add Lua API module for monitored objects - * [Feature] Add R_DKIM_NA / R_SPF_NA / AUTH_NA symbols - * [Feature] Add R_DKIM_PERMFAIL symbol - * [Feature] Add R_SPF_PERMFAIL symbol - * [Feature] Add Sophos antivirus support - * [Feature] Add ZSTD compression to Lua API - * [Feature] Add `mid` Lua module - * [Feature] Add `one_param` flag for metric symbols - * [Feature] Add a generic lua classifier - * [Feature] Add a very basic interface to access workers data from on_load - * [Feature] Add ability to delete a hash by its data to fuzzy_check plugin - * [Feature] Add ability to enable/disable symbols via dynamic_conf - * [Feature] Add ability to lookup settings by key - * [Feature] Add common way to disable Lua modules - * [Feature] Add compression support to rspamd client - * [Feature] Add condition to do antiviral check - * [Feature] Add configuration for lua classifiers - * [Feature] Add configuration knobs for the errors circular buffer - * [Feature] Add decompression support in rspamd client - * [Feature] Add errors exporter to the controller - * [Feature] Add expected value for monitored DNS resources - * [Feature] Add exporter from error ringbuf to ucl - * [Feature] Add extended version for fann creation function - * [Feature] Add ffi friendly version of process_regexp function - * [Feature] Add frequency and time display to webui - * [Feature] Add fuzzy_delhash command to rspamc client - * [Feature] Add implementation of redis connections pool - * [Feature] Add latency and offline time monitoring - * [Feature] Add learning support for lua classifiers - * [Feature] Add max-size and timeout options to CGP helper - * [Feature] Add method to enable/disable symbols in config - * [Feature] Add methods to get metric's actions and symbols from Lua - * [Feature] Add mmap support to lua_text - * [Feature] Add monitored object for surbl plugin - * [Feature] Add more exceptions to surbl whitelist - * [Feature] Add more meta-tokens to bayes - * [Feature] Add neural net classifier to fann_scores module - * [Feature] Add neural net serialization/deserialization - * [Feature] Add new dynamic conf module - * [Feature] Add periodic events support for lua_config - * [Feature] Add plugin to check MX'es for the sender's domain - * [Feature] Add preliminary monitored module - * [Feature] Add preliminary support of dynamic conf updates in Redis - * [Feature] Add preliminary version of clamav plugin - * [Feature] Add redis cache to asn module - * [Feature] Add replies compression - * [Feature] Add spamhaus DROP dnsbl - * [Feature] Add support for dictionary in client compression - * [Feature] Add support for fuzzy learn and unlearn from lua - * [Feature] Add support for input encryption - * [Feature] Add support of min_learns to neural net classifier - * [Feature] Add termination callbacks for workers - * [Feature] Add user-agent for rspamc - * [Feature] Add utility to perform classifier tests - * [Feature] Add zstd compression library - * [Feature] Allow HTTPS requests in lua_http - * [Feature] Allow conditions for pre and postfilters - * [Feature] Allow custom functions for ratelimits - * [Feature] Allow for excluding messages from AV scanning based on size - * [Feature] Allow for getting worker stats from Lua - * [Feature] Allow getting task UID from Lua - * [Feature] Allow parsing of mailbox messages from the commandline - * [Feature] Allow plugins to publish their lua API via rspamd_plugins - * [Feature] Allow to compare other systems with Rspamd - * [Feature] Allow to execute Lua scripts by controller - * [Feature] Allow to have a function to set custom greylist message - * [Feature] Allow to iterate over multiple tags - * [Feature] Allow to pass extra data from plugins to log helper - * [Feature] Allow to plan new periodics at different time - * [Feature] Allow to reset hashes - * [Feature] Allow to run rspamadm lua just as a lua interpreter - * [Feature] Allow to store settings in redis - * [Feature] Allow to update dynamic conf in Redis - * [Feature] Allow to use dictionaries for compression - * [Feature] Allow to use md5, sha1, sha256, sha384 and sha512 hashes in Lua - * [Feature] Allow whitelisting by IP for greylisting plugin - * [Feature] Antivirus: Support whitelists & pattern-matching sig names - * [Feature] Backport pack/unpack routines from Lua 5.3 - * [Feature] Check settings with equal priopities in alphabetical order - * [Feature] Compress neural net in redis - * [Feature] Consider more tags when doing WHITE_ON_WHITE rule - * [Feature] Descriptive options for DMARC failure symbols - * [Feature] Descriptive options for RBL symbols - * [Feature] Enable configuration for monitored objects - * [Feature] Execute on_load scripts with ev_base ready - * [Feature] Fann scores now uses metadata from a message - * [Feature] Implement FANN threaded learning - * [Feature] Implement classifying for lua classifiers - * [Feature] Implement finish scripts for worker processes - * [Feature] Implement monitoring for DNS resources - * [Feature] Implement real priorities for pre and post filters - * [Feature] Insert two symbols: FANN_HAM and FANN_SPAM instead of one - * [Feature] Module to push metadata/messages to redis pubsub - * [Feature] Monitor RBL records - * [Feature] Move fann_classifier to a separate plugin - * [Feature] Normalize all ANN inputs - * [Feature] Preliminary version of metric exporter module - * [Feature] Preserve decompression context between tasks - * [Feature] Ratelimit: Support dynamic bucket size/leak rate - * [Feature] Relax FORGED_RECIPIENTS: allow senders to BCC themselves - * [Feature] Remove symbols weights on composites processing - * [Feature] Return symbol scores when getting resulting symbols - * [Feature] Rework lua tcp module - * [Feature] Rule to detect some obvious X-PHP-Originating-Script forgeries - * [Feature] Rule to identify some X-PHP-Script forgeries - * [Feature] Rules for scoring Google Message-ID fixes - * [Feature] Send hashes values to reply - * [Feature] Set expire for dmarc reports - * [Feature] Stop using cymru zone as it is unstable - * [Feature] Stop using of GLists for headers, improve performance - * [Feature] Store `for` in task:get_received_headers - * [Feature] Store `for` part in received headers - * [Feature] Store enabled flag for webui session - * [Feature] Store error messages in ring buffer - * [Feature] Support compressed maps - * [Feature] Support excluding selected users from ratelimits - * [Feature] Support looking up NS records in lua_dns - * [Feature] Support modern style SURBL configuration - * [Feature] Support multiple hashes in delhash path - * [Feature] Support new messages in rspamc - * [Feature] Support requests without reads in lua_tcp - * [Feature] Support setting task message from Lua - * [Feature] Track visibility of HTML elements - * [Feature] Try to add CRLF when checking DKIM - * [Feature] Try to guess line endings when folding headers - * [Feature] Try to improve normalization function for bayes - * [Feature] Use FFI to optimize SA module - * [Feature] Use length based arguments for redis, allow lua_text as arg - * [Feature] Use more layers for fann and another normalization - * [Feature] User-defined ratelimits - * [Feature] Utility to convert fuzzy storage from sqlite to redis - * [Feature] Yield DMARC_DNSFAIL on lookup failure - * [Fix] Adopt fuzzy storage for flexible backends - * [Fix] Allow plain IP addresses in Rspamd maps - * [Fix] Another fix for brain-damaged hiredis - * [Fix] Another fix for rdns write errors - * [Fix] Another fix for rdns_make_request_full invocation - * [Fix] Another fix in DKIM canonicalization - * [Fix] Another memory leak plugged - * [Fix] Another try to deal with posix idiotizm - * [Fix] Another try to fix RDNS events processing logic - * [Fix] Avoid double frees in HEAD requests - * [Fix] Avoid extra symbols for RBLs - * [Fix] Banish table.maxn from Lua parts - * [Fix] Check for socket error before connection in lua_tcp - * [Fix] Correctly propagate redis timeouts to Lua - * [Fix] Do not add extra newline in MIME mode - * [Fix] Do not be cheated by system hiredis - * [Fix] Do not classify when a message has not enough tokens - * [Fix] Do not crash on redis errors - * [Fix] Do not distinguish NXDOMAIN and NOREC for monitored - * [Fix] Do not replan retransmits if merely one server is defined - * [Fix] Do not use headers to calculate messages digests - * [Fix] Don't force action in replies module for authenticated users/local networks - * [Fix] Explicitly ban default passwords in webui - * [Fix] Finally fix ambiguity between parsed and resolved spf elts - * [Fix] Fix 'decoded' value in task:get_header_full() - * [Fix] Fix DKIM calculations - * [Fix] Fix DKIM signing for messages with no newline at the end - * [Fix] Fix DNS request in monitored - * [Fix] Fix DNS write errors processing - * [Fix] Fix HTTP methods other than GET and POST - * [Fix] Fix PERMFAIL for v6/v4 ambiguities - * [Fix] Fix absurdic scores for HFILTER_URL_ONLY - * [Fix] Fix actions in rolling history - * [Fix] Fix actrie patterns - * [Fix] Fix applying of lua dynamic confg - * [Fix] Fix autolearning errors and redis cache - * [Fix] Fix bayes learn_condition - * [Fix] Fix build with the recent OpenSSL - * [Fix] Fix caching and compressed maps - * [Fix] Fix check plain text part - * [Fix] Fix crash on OpenBSD in `url_email_start` - * [Fix] Fix double free in SPF - * [Fix] Fix extraction of shingles from redis fuzzy storage - * [Fix] Fix false sharing for symbols in the cache - * [Fix] Fix float usage in util:get_time - * [Fix] Fix folding algorithm to deal with empty tokens - * [Fix] Fix format string - * [Fix] Fix format string usage in controller errors handling - * [Fix] Fix handling of '\0' in lua_tcp - * [Fix] Fix handling of HTTP HEAD methods - * [Fix] Fix hash creation - * [Fix] Fix hiredis stupidity - * [Fix] Fix implicit settings module settingsup - * [Fix] Fix interaction with lua GC to avoid craches - * [Fix] Fix ip_score module registration - * [Fix] Fix issue with empty messages and dkim - * [Fix] Fix issues with CGP helper - * [Fix] Fix issues with the recent SPF changes - * [Fix] Fix key name to load ANN correctly - * [Fix] Fix lua tcp module by saving `do_read` in callback data - * [Fix] Fix memory leak in client when using compression - * [Fix] Fix min_learns option - * [Fix] Fix on_finish scripts and async handlers - * [Fix] Fix options for SPF dnsfail symbol - * [Fix] Fix parsing includes and redirects in SPF - * [Fix] Fix parsing of lua comments with empty lines - * [Fix] Fix parsing of unquoted HTML attributes - * [Fix] Fix periodic events and redis - * [Fix] Fix processing of fuzzy learns from Lua - * [Fix] Fix processing of redirect in SPF includes - * [Fix] Fix processing of symbols when reject limit is reached - * [Fix] Fix refcounts when map is specified by IP - * [Fix] Fix rspamd{session} class in Lua API - * [Fix] Fix setting ratelimit key for 'ip' bucket - * [Fix] Fix some cases of TLD urls detector - * [Fix] Fix statconvert tool - * [Fix] Fix stats for backend-less classifiers - * [Fix] Fix training script for fann_redis - * [Fix] Fix variable in ann module - * [Fix] Fix various errors in lua dynamic conf plugin - * [Fix] Forget old ANN when max_usages is reached to avoid overtrain - * [Fix] Further canonicalization fixes - * [Fix] Further fixes for fann_redis prefixes - * [Fix] Handle failures for inactive pooled connections - * [Fix] Improve multimap info message - * [Fix] More fixes in ANN loading - * [Fix] More fixes to fann_redis - * [Fix] More issues in fann_redis - * [Fix] More spaces fix in DKIM signature - * [Fix] Multiple fixes to asn script, add IPv6 support - * [Fix] Multiple issues in fann_redis - * [Fix] No greylist rejected messages - * [Fix] One more attempt to fix lua_redis - * [Fix] One more check for readdir... - * [Fix] Params should be treated as a hash - * [Fix] Plug memory leak in regexp desctructor - * [Fix] Process headers only once - * [Fix] Properly handle nil values in ratelimit plugin - * [Fix] Really fix redis shingles check - * [Fix] Remove fann with incorrect layers count - * [Fix] Remove mentions of deleted include - * [Fix] Remove some incompatible functions - * [Fix] Settings: fix `authenticated` parameter (#886) - * [Fix] Skip MX check for authenticated users and local networks - * [Fix] Slightly fix ANN routines - * [Fix] Stop caching records with DNS failures - * [Fix] Treat all errors in redis_pool as fatal errors for a connection - * [Fix] Try avoid false-positives in HEADER_FORGED_MDN rule - * [Fix] Try to avoid race condition when using rrd - * [Fix] Try to reload redis scripts if they are missing - * [Fix] Unbreak once_received skipping for local networks - * [Fix] Unlock ANN on error - * [Fix] Use memmove for overlapping regions - * [Fix] Use real size instead of displayed for core limits - * [Fix] Use the correct macro to get the size of control - * [Fix] Various fixes for errors ringbuffer - * [Fix] Yield R_SPF_DNSFAIL if lookup of included record fails - * [Fix] mid: fix map initialization - * [Fix] mid: handle incorrect rgexps in the map - * [Rework] Add extract training data function to fann_redis - * [Rework] Add preliminary train tests - * [Rework] Add redis storage feature to fann_redis - * [Rework] Adopt fuzzy storage for abstract backend - * [Rework] Adopt plugins - * [Rework] First reiteration on fann scores - * [Rework] Implement loading/invalidating - * [Rework] Make lua_redis task agnostic - * [Rework] Make rspamd protocol messages useful - * [Rework] Massive removal of legacy code - * [Rework] More cleanup actions - * [Rework] Remove legacy code never used for classifiers - * [Rework] Remove outdated and unused lua_session module - * [Rework] Reorganize fuzzy backend structure - * [Rework] Reorganize the internal backend structure - * [Rework] Restore old fann_scores, move common parts - * [Rework] Rework and simplify rbl plugin - * [Rework] Rework parsing of DMARC records + * [CritFix] Add guards for inactive redis connections + * [CritFix] Another fix for proxying files using rspamd_proxy + * [CritFix] Cleanup inactive redis connections + * [CritFix] Do not sometimes try to exec posfilters before classification + * [CritFix] Fix application of IPv6 mask + * [CritFix] Fix chunked encoding when reading messages + * [CritFix] Fix file mode for rspamd_proxy + * [CritFix] Fix hyperscan compilation on regexp change + * [CritFix] Fix issue with finding of end of lines pointers + * [CritFix] Fix iteration over headers array (introduced in 1.4) + * [CritFix] Fix processing of learned tokens count for redis backend + * [CritFix] Fix race condition in checking of cached maps + * [CritFix] Fix workers scripts by sharing workers configs + * [CritFix] Introduce raw content to text parts + * [CritFix] Plug memory leak and potential memory corruption + * [Feature] Adaptive ratelimits + * [Feature] Add ASN -> rbldnsd script for asn.rspamd.com + * [Feature] Add DMARC_NA symbol + * [Feature] Add F-Prot support to antivirus module + * [Feature] Add HTTP backend to metadata exporter + * [Feature] Add Lua API module for monitored objects + * [Feature] Add R_DKIM_NA / R_SPF_NA / AUTH_NA symbols + * [Feature] Add R_DKIM_PERMFAIL symbol + * [Feature] Add R_SPF_PERMFAIL symbol + * [Feature] Add Sophos antivirus support + * [Feature] Add ZSTD compression to Lua API + * [Feature] Add `mid` Lua module + * [Feature] Add `one_param` flag for metric symbols + * [Feature] Add a generic lua classifier + * [Feature] Add a very basic interface to access workers data from on_load + * [Feature] Add ability to delete a hash by its data to fuzzy_check plugin + * [Feature] Add ability to enable/disable symbols via dynamic_conf + * [Feature] Add ability to lookup settings by key + * [Feature] Add common way to disable Lua modules + * [Feature] Add compression support to rspamd client + * [Feature] Add condition to do antiviral check + * [Feature] Add configuration for lua classifiers + * [Feature] Add configuration knobs for the errors circular buffer + * [Feature] Add decompression support in rspamd client + * [Feature] Add errors exporter to the controller + * [Feature] Add expected value for monitored DNS resources + * [Feature] Add exporter from error ringbuf to ucl + * [Feature] Add extended version for fann creation function + * [Feature] Add ffi friendly version of process_regexp function + * [Feature] Add frequency and time display to webui + * [Feature] Add fuzzy_delhash command to rspamc client + * [Feature] Add implementation of redis connections pool + * [Feature] Add latency and offline time monitoring + * [Feature] Add learning support for lua classifiers + * [Feature] Add max-size and timeout options to CGP helper + * [Feature] Add method to enable/disable symbols in config + * [Feature] Add methods to get metric's actions and symbols from Lua + * [Feature] Add mmap support to lua_text + * [Feature] Add monitored object for surbl plugin + * [Feature] Add more exceptions to surbl whitelist + * [Feature] Add more meta-tokens to bayes + * [Feature] Add neural net classifier to fann_scores module + * [Feature] Add neural net serialization/deserialization + * [Feature] Add new dynamic conf module + * [Feature] Add periodic events support for lua_config + * [Feature] Add plugin to check MX'es for the sender's domain + * [Feature] Add preliminary monitored module + * [Feature] Add preliminary support of dynamic conf updates in Redis + * [Feature] Add preliminary version of clamav plugin + * [Feature] Add redis cache to asn module + * [Feature] Add replies compression + * [Feature] Add spamhaus DROP dnsbl + * [Feature] Add support for dictionary in client compression + * [Feature] Add support for fuzzy learn and unlearn from lua + * [Feature] Add support for input encryption + * [Feature] Add support of min_learns to neural net classifier + * [Feature] Add termination callbacks for workers + * [Feature] Add user-agent for rspamc + * [Feature] Add utility to perform classifier tests + * [Feature] Add zstd compression library + * [Feature] Allow HTTPS requests in lua_http + * [Feature] Allow conditions for pre and postfilters + * [Feature] Allow custom functions for ratelimits + * [Feature] Allow for excluding messages from AV scanning based on size + * [Feature] Allow for getting worker stats from Lua + * [Feature] Allow getting task UID from Lua + * [Feature] Allow parsing of mailbox messages from the commandline + * [Feature] Allow plugins to publish their lua API via rspamd_plugins + * [Feature] Allow to compare other systems with Rspamd + * [Feature] Allow to execute Lua scripts by controller + * [Feature] Allow to have a function to set custom greylist message + * [Feature] Allow to iterate over multiple tags + * [Feature] Allow to pass extra data from plugins to log helper + * [Feature] Allow to plan new periodics at different time + * [Feature] Allow to reset hashes + * [Feature] Allow to run rspamadm lua just as a lua interpreter + * [Feature] Allow to store settings in redis + * [Feature] Allow to update dynamic conf in Redis + * [Feature] Allow to use dictionaries for compression + * [Feature] Allow to use md5, sha1, sha256, sha384 and sha512 hashes in Lua + * [Feature] Allow whitelisting by IP for greylisting plugin + * [Feature] Antivirus: Support whitelists & pattern-matching sig names + * [Feature] Backport pack/unpack routines from Lua 5.3 + * [Feature] Check settings with equal priopities in alphabetical order + * [Feature] Compress neural net in redis + * [Feature] Consider more tags when doing WHITE_ON_WHITE rule + * [Feature] Descriptive options for DMARC failure symbols + * [Feature] Descriptive options for RBL symbols + * [Feature] Enable configuration for monitored objects + * [Feature] Execute on_load scripts with ev_base ready + * [Feature] Fann scores now uses metadata from a message + * [Feature] Implement FANN threaded learning + * [Feature] Implement classifying for lua classifiers + * [Feature] Implement finish scripts for worker processes + * [Feature] Implement monitoring for DNS resources + * [Feature] Implement real priorities for pre and post filters + * [Feature] Insert two symbols: FANN_HAM and FANN_SPAM instead of one + * [Feature] Module to push metadata/messages to redis pubsub + * [Feature] Monitor RBL records + * [Feature] Move fann_classifier to a separate plugin + * [Feature] Normalize all ANN inputs + * [Feature] Preliminary version of metric exporter module + * [Feature] Preserve decompression context between tasks + * [Feature] Ratelimit: Support dynamic bucket size/leak rate + * [Feature] Relax FORGED_RECIPIENTS: allow senders to BCC themselves + * [Feature] Remove symbols weights on composites processing + * [Feature] Return symbol scores when getting resulting symbols + * [Feature] Rework lua tcp module + * [Feature] Rule to detect some obvious X-PHP-Originating-Script forgeries + * [Feature] Rule to identify some X-PHP-Script forgeries + * [Feature] Rules for scoring Google Message-ID fixes + * [Feature] Send hashes values to reply + * [Feature] Set expire for dmarc reports + * [Feature] Stop using cymru zone as it is unstable + * [Feature] Stop using of GLists for headers, improve performance + * [Feature] Store `for` in task:get_received_headers + * [Feature] Store `for` part in received headers + * [Feature] Store enabled flag for webui session + * [Feature] Store error messages in ring buffer + * [Feature] Support compressed maps + * [Feature] Support excluding selected users from ratelimits + * [Feature] Support looking up NS records in lua_dns + * [Feature] Support modern style SURBL configuration + * [Feature] Support multiple hashes in delhash path + * [Feature] Support new messages in rspamc + * [Feature] Support requests without reads in lua_tcp + * [Feature] Support setting task message from Lua + * [Feature] Track visibility of HTML elements + * [Feature] Try to add CRLF when checking DKIM + * [Feature] Try to guess line endings when folding headers + * [Feature] Try to improve normalization function for bayes + * [Feature] Use FFI to optimize SA module + * [Feature] Use length based arguments for redis, allow lua_text as arg + * [Feature] Use more layers for fann and another normalization + * [Feature] User-defined ratelimits + * [Feature] Utility to convert fuzzy storage from sqlite to redis + * [Feature] Yield DMARC_DNSFAIL on lookup failure + * [Fix] Adopt fuzzy storage for flexible backends + * [Fix] Allow plain IP addresses in Rspamd maps + * [Fix] Another fix for brain-damaged hiredis + * [Fix] Another fix for rdns write errors + * [Fix] Another fix for rdns_make_request_full invocation + * [Fix] Another fix in DKIM canonicalization + * [Fix] Another memory leak plugged + * [Fix] Another try to deal with posix idiotizm + * [Fix] Another try to fix RDNS events processing logic + * [Fix] Avoid double frees in HEAD requests + * [Fix] Avoid extra symbols for RBLs + * [Fix] Banish table.maxn from Lua parts + * [Fix] Check for socket error before connection in lua_tcp + * [Fix] Correctly propagate redis timeouts to Lua + * [Fix] Do not add extra newline in MIME mode + * [Fix] Do not be cheated by system hiredis + * [Fix] Do not classify when a message has not enough tokens + * [Fix] Do not crash on redis errors + * [Fix] Do not distinguish NXDOMAIN and NOREC for monitored + * [Fix] Do not replan retransmits if merely one server is defined + * [Fix] Do not use headers to calculate messages digests + * [Fix] Don't force action in replies module for authenticated users/local networks + * [Fix] Explicitly ban default passwords in webui + * [Fix] Finally fix ambiguity between parsed and resolved spf elts + * [Fix] Fix 'decoded' value in task:get_header_full() + * [Fix] Fix DKIM calculations + * [Fix] Fix DKIM signing for messages with no newline at the end + * [Fix] Fix DNS request in monitored + * [Fix] Fix DNS write errors processing + * [Fix] Fix HTTP methods other than GET and POST + * [Fix] Fix PERMFAIL for v6/v4 ambiguities + * [Fix] Fix absurdic scores for HFILTER_URL_ONLY + * [Fix] Fix actions in rolling history + * [Fix] Fix actrie patterns + * [Fix] Fix applying of lua dynamic confg + * [Fix] Fix autolearning errors and redis cache + * [Fix] Fix bayes learn_condition + * [Fix] Fix build with the recent OpenSSL + * [Fix] Fix caching and compressed maps + * [Fix] Fix check plain text part + * [Fix] Fix crash on OpenBSD in `url_email_start` + * [Fix] Fix double free in SPF + * [Fix] Fix extraction of shingles from redis fuzzy storage + * [Fix] Fix false sharing for symbols in the cache + * [Fix] Fix float usage in util:get_time + * [Fix] Fix folding algorithm to deal with empty tokens + * [Fix] Fix format string + * [Fix] Fix format string usage in controller errors handling + * [Fix] Fix handling of '\0' in lua_tcp + * [Fix] Fix handling of HTTP HEAD methods + * [Fix] Fix hash creation + * [Fix] Fix hiredis stupidity + * [Fix] Fix implicit settings module settingsup + * [Fix] Fix interaction with lua GC to avoid craches + * [Fix] Fix ip_score module registration + * [Fix] Fix issue with empty messages and dkim + * [Fix] Fix issues with CGP helper + * [Fix] Fix issues with the recent SPF changes + * [Fix] Fix key name to load ANN correctly + * [Fix] Fix lua tcp module by saving `do_read` in callback data + * [Fix] Fix memory leak in client when using compression + * [Fix] Fix min_learns option + * [Fix] Fix on_finish scripts and async handlers + * [Fix] Fix options for SPF dnsfail symbol + * [Fix] Fix parsing includes and redirects in SPF + * [Fix] Fix parsing of lua comments with empty lines + * [Fix] Fix parsing of unquoted HTML attributes + * [Fix] Fix periodic events and redis + * [Fix] Fix processing of fuzzy learns from Lua + * [Fix] Fix processing of redirect in SPF includes + * [Fix] Fix processing of symbols when reject limit is reached + * [Fix] Fix refcounts when map is specified by IP + * [Fix] Fix rspamd{session} class in Lua API + * [Fix] Fix setting ratelimit key for 'ip' bucket + * [Fix] Fix some cases of TLD urls detector + * [Fix] Fix statconvert tool + * [Fix] Fix stats for backend-less classifiers + * [Fix] Fix training script for fann_redis + * [Fix] Fix variable in ann module + * [Fix] Fix various errors in lua dynamic conf plugin + * [Fix] Forget old ANN when max_usages is reached to avoid overtrain + * [Fix] Further canonicalization fixes + * [Fix] Further fixes for fann_redis prefixes + * [Fix] Handle failures for inactive pooled connections + * [Fix] Improve multimap info message + * [Fix] More fixes in ANN loading + * [Fix] More fixes to fann_redis + * [Fix] More issues in fann_redis + * [Fix] More spaces fix in DKIM signature + * [Fix] Multiple fixes to asn script, add IPv6 support + * [Fix] Multiple issues in fann_redis + * [Fix] No greylist rejected messages + * [Fix] One more attempt to fix lua_redis + * [Fix] One more check for readdir... + * [Fix] Params should be treated as a hash + * [Fix] Plug memory leak in regexp desctructor + * [Fix] Process headers only once + * [Fix] Properly handle nil values in ratelimit plugin + * [Fix] Really fix redis shingles check + * [Fix] Remove fann with incorrect layers count + * [Fix] Remove mentions of deleted include + * [Fix] Remove some incompatible functions + * [Fix] Settings: fix `authenticated` parameter (#886) + * [Fix] Skip MX check for authenticated users and local networks + * [Fix] Slightly fix ANN routines + * [Fix] Stop caching records with DNS failures + * [Fix] Treat all errors in redis_pool as fatal errors for a connection + * [Fix] Try avoid false-positives in HEADER_FORGED_MDN rule + * [Fix] Try to avoid race condition when using rrd + * [Fix] Try to reload redis scripts if they are missing + * [Fix] Unbreak once_received skipping for local networks + * [Fix] Unlock ANN on error + * [Fix] Use memmove for overlapping regions + * [Fix] Use real size instead of displayed for core limits + * [Fix] Use the correct macro to get the size of control + * [Fix] Various fixes for errors ringbuffer + * [Fix] Yield R_SPF_DNSFAIL if lookup of included record fails + * [Fix] mid: fix map initialization + * [Fix] mid: handle incorrect rgexps in the map + * [Rework] Add extract training data function to fann_redis + * [Rework] Add preliminary train tests + * [Rework] Add redis storage feature to fann_redis + * [Rework] Adopt fuzzy storage for abstract backend + * [Rework] Adopt plugins + * [Rework] First reiteration on fann scores + * [Rework] Implement loading/invalidating + * [Rework] Make lua_redis task agnostic + * [Rework] Make rspamd protocol messages useful + * [Rework] Massive removal of legacy code + * [Rework] More cleanup actions + * [Rework] Remove legacy code never used for classifiers + * [Rework] Remove outdated and unused lua_session module + * [Rework] Reorganize fuzzy backend structure + * [Rework] Reorganize the internal backend structure + * [Rework] Restore old fann_scores, move common parts + * [Rework] Rework and simplify rbl plugin + * [Rework] Rework parsing of DMARC records 1.3.4: - * [Feature] ASN module; support matching ASN/country in multimap - * [Feature] Add SPF method in spf return result - * [Feature] Add Yandex and Mail.ru forwarding rules - * [Feature] Add mempool maps in multimap - * [Feature] Add rule for identifying mail sent by eval()'d PHP code - * [Feature] Add support of stub DNSSEC resolver to rdns - * [Feature] Add task:get_digest method - * [Feature] Allow for more fine-grained scoring for ip_score - * [Feature] Allow to get digest of a mime part from lua - * [Feature] Allow to print message digest in logs - * [Feature] Fold DKIM-Signature header - * [Feature] Implement encrypted logs - * [Feature] Log URLs encrypted if we have log encryption pubkey - * [Feature] Pass authenticated bit to lua - * [Feature] Read redis backend statistics configuration from global section - * [Feature] Show the exact value matched for multima symbols - * [Feature] Store task checksum - * [Fix] Avoid setting limits when required elements are missing - * [Fix] DMARC: Fix alignment checking for subdomains - * [Fix] DMARC: deal with missing and spurious spaces - * [Fix] Defer insertion of results in ip_score to avoid skewing stats - * [Fix] Disable DMARC for local/authorized mail - * [Fix] Fix handling of proxied headers in controller - * [Fix] Fix hex printing of strings - * [Fix] Fix issue with spaces in maps - * [Fix] Fix parsing of forwarded IP - * [Fix] Fix reload in some plugins and workers - * [Fix] Fix reloading on SIGHUP - * [Fix] Fix some border cases for DKIM canonicalization - * [Fix] Fix url maps - * [Fix] Make dnssec configurable option disabled by default for now - * [Fix] rspamadm statconvert: force db to be a string - * [Fix] rspamadm statconvert: use db/password for learn cache - * [Rework] Rework flags in rspamd logger + * [Feature] ASN module; support matching ASN/country in multimap + * [Feature] Add SPF method in spf return result + * [Feature] Add Yandex and Mail.ru forwarding rules + * [Feature] Add mempool maps in multimap + * [Feature] Add rule for identifying mail sent by eval()'d PHP code + * [Feature] Add support of stub DNSSEC resolver to rdns + * [Feature] Add task:get_digest method + * [Feature] Allow for more fine-grained scoring for ip_score + * [Feature] Allow to get digest of a mime part from lua + * [Feature] Allow to print message digest in logs + * [Feature] Fold DKIM-Signature header + * [Feature] Implement encrypted logs + * [Feature] Log URLs encrypted if we have log encryption pubkey + * [Feature] Pass authenticated bit to lua + * [Feature] Read redis backend statistics configuration from global section + * [Feature] Show the exact value matched for multima symbols + * [Feature] Store task checksum + * [Fix] Avoid setting limits when required elements are missing + * [Fix] DMARC: Fix alignment checking for subdomains + * [Fix] DMARC: deal with missing and spurious spaces + * [Fix] Defer insertion of results in ip_score to avoid skewing stats + * [Fix] Disable DMARC for local/authorized mail + * [Fix] Fix handling of proxied headers in controller + * [Fix] Fix hex printing of strings + * [Fix] Fix issue with spaces in maps + * [Fix] Fix parsing of forwarded IP + * [Fix] Fix reload in some plugins and workers + * [Fix] Fix reloading on SIGHUP + * [Fix] Fix some border cases for DKIM canonicalization + * [Fix] Fix url maps + * [Fix] Make dnssec configurable option disabled by default for now + * [Fix] rspamadm statconvert: force db to be a string + * [Fix] rspamadm statconvert: use db/password for learn cache + * [Rework] Rework flags in rspamd logger 1.3.3: - * [CritFix] Check hyperscan cache sanity before loading - * [CritFix] Fix setting of fuzzy keys (completely breaks fuzzy storage) - * [Feature] Add SARBL (sarbl.org) uribl - * [Feature] Add `--search-pattern` option to rspamd_stats - * [Feature] Add some sanity check for very long from/to log elements - * [Feature] Allow to create hashes from string in a single step - * [Feature] Fix order of pre and postfilters - * [Feature] Improve lua URLs API - * [Feature] Improve message about fuzzy rules - * [Feature] Pre-calculate blake2 digest for all parts - * [Feature] Print radix duplicate keys as IP addresses - * [Feature] Simple mechanism for disabling RBLs in local.d/rbl.conf - * [Feature] Use faster hash function for fuzzy storage - * [Feature] rspamd_stats: support log directory reading - * [Fix] Add sanity check for url filters - * [Fix] Do not show rmilter section as a fake metric in rspamc - * [Fix] Fix URL filters - * [Fix] Fix a stupid mistake in util.strequal_caseless - * [Fix] Fix blake2b hash of the string "rspamd" - * [Fix] Fix filename maps filter - * [Fix] Fix finding tld in util.get_tld - * [Fix] Fix multimap content filters - * [Fix] Fix returning boolean from Lua - * [Fix] Fix returning of REDIS_NIL - * [Fix] Try to deal with multiple workers terminated - * [Fix] Use forced DNS request when calling for lua_http - * [Rework] Rework multimap filters, add redis maps + * [CritFix] Check hyperscan cache sanity before loading + * [CritFix] Fix setting of fuzzy keys (completely breaks fuzzy storage) + * [Feature] Add SARBL (sarbl.org) uribl + * [Feature] Add `--search-pattern` option to rspamd_stats + * [Feature] Add some sanity check for very long from/to log elements + * [Feature] Allow to create hashes from string in a single step + * [Feature] Fix order of pre and postfilters + * [Feature] Improve lua URLs API + * [Feature] Improve message about fuzzy rules + * [Feature] Pre-calculate blake2 digest for all parts + * [Feature] Print radix duplicate keys as IP addresses + * [Feature] Simple mechanism for disabling RBLs in local.d/rbl.conf + * [Feature] Use faster hash function for fuzzy storage + * [Feature] rspamd_stats: support log directory reading + * [Fix] Add sanity check for url filters + * [Fix] Do not show rmilter section as a fake metric in rspamc + * [Fix] Fix URL filters + * [Fix] Fix a stupid mistake in util.strequal_caseless + * [Fix] Fix blake2b hash of the string "rspamd" + * [Fix] Fix filename maps filter + * [Fix] Fix finding tld in util.get_tld + * [Fix] Fix multimap content filters + * [Fix] Fix returning boolean from Lua + * [Fix] Fix returning of REDIS_NIL + * [Fix] Try to deal with multiple workers terminated + * [Fix] Use forced DNS request when calling for lua_http + * [Rework] Rework multimap filters, add redis maps 1.3.2: - * [Feature] Add a special symbol for SPF DNS errors: R_SPF_DNSFAIL - * [Feature] Add correlations report in fuzzy stats - * [Feature] Add experimental CGP integration - * [Feature] Add method to get urls length in a text part - * [Feature] Add new methods to lua_html to access HTML tags - * [Feature] Allow all types of symbols to be added via __newindex method - * [Feature] Allow to create settings for authenticated users - * [Feature] Allow to get block content for HTML tags - * [Feature] Improve DNS failures when dealing with SPF - * [Feature] Properly implement R_WHITE_ON_WHITE rule - * [Feature] Remove old ugly rules - * [Feature] Rspamc can now add dkim signature in mime mode - * [Feature] Store content length for HTML tags - * [Feature] Support reacher set of HTML colors - * [Feature] Try to avoid FP for low contrast fonts detection - * [Fix] Add missing HTML colors - * [Fix] Add spaces to dkim signature to allow folding - * [Fix] Avoid returning NaN as score on scan - * [Fix] Decode entitles in href parts - * [Fix] Do not cache SPF records with DNS errors - * [Fix] Do not crash on cyclic depends - * [Fix] Do not insert HELO/HOSTNAME unknown when they are not passed - * [Fix] Do not set absent hostname to "unknown" - * [Fix] Do not stress redis with KEYS command (#791) - * [Fix] Fix DMARC_BAD_POLICY symbol - * [Fix] Fix HFILTER_URL module - * [Fix] Fix HFILTER_URL_ONELINE rule - * [Fix] Fix buffering in CGP integration - * [Fix] Fix colors propagation from parent nodes - * [Fix] Fix confusing OpenSSL API usage of i2d_RSAPublicKey - * [Fix] Fix dependencies id sanity check - * [Fix] Fix folding for semicolon separated tokens - * [Fix] Fix largest possible TLD behaviour - * [Fix] Fix last token folding - * [Fix] Fix length calculations in white on white rule - * [Fix] Fix multiple request headers structure - * [Fix] Fix multiple values headers freeing - * [Fix] Fix parsing of background color - * [Fix] Fix printing from field in log_urls - * [Fix] Fix processing of last element of DMARC policies - * [Fix] Further fixes for HTML colors - * [Fix] Further fixes for multiple values headers - * [Fix] Further fixes for white on white rule - * [Fix] Further fixes in HTML tags parsing - * [Fix] Ignore content type/subtype case - * [Fix] Increase score of R_WHITE_ON_WHITE - * [Fix] Parse CGP envelope data - * [Fix] Propagate colors in HTML - * [Fix] Restore multiple values headers in protocol - * [Fix] Restore multiple values in headers processing - * [Fix] Some more changes to tag's content length calculations - * [Fix] Some more fixes for low contrast fonts detector - * [Fix] SpamAssassin plugin: support check_freemail_header('EnvelopeFrom', [..]) - * [Fix] Trigger HTML_SHORT_LINK_IMG on any external image - * [Fix] rspamd_stats: remove deprecated defined(@array) + * [Feature] Add a special symbol for SPF DNS errors: R_SPF_DNSFAIL + * [Feature] Add correlations report in fuzzy stats + * [Feature] Add experimental CGP integration + * [Feature] Add method to get urls length in a text part + * [Feature] Add new methods to lua_html to access HTML tags + * [Feature] Allow all types of symbols to be added via __newindex method + * [Feature] Allow to create settings for authenticated users + * [Feature] Allow to get block content for HTML tags + * [Feature] Improve DNS failures when dealing with SPF + * [Feature] Properly implement R_WHITE_ON_WHITE rule + * [Feature] Remove old ugly rules + * [Feature] Rspamc can now add dkim signature in mime mode + * [Feature] Store content length for HTML tags + * [Feature] Support reacher set of HTML colors + * [Feature] Try to avoid FP for low contrast fonts detection + * [Fix] Add missing HTML colors + * [Fix] Add spaces to dkim signature to allow folding + * [Fix] Avoid returning NaN as score on scan + * [Fix] Decode entitles in href parts + * [Fix] Do not cache SPF records with DNS errors + * [Fix] Do not crash on cyclic depends + * [Fix] Do not insert HELO/HOSTNAME unknown when they are not passed + * [Fix] Do not set absent hostname to "unknown" + * [Fix] Do not stress redis with KEYS command (#791) + * [Fix] Fix DMARC_BAD_POLICY symbol + * [Fix] Fix HFILTER_URL module + * [Fix] Fix HFILTER_URL_ONELINE rule + * [Fix] Fix buffering in CGP integration + * [Fix] Fix colors propagation from parent nodes + * [Fix] Fix confusing OpenSSL API usage of i2d_RSAPublicKey + * [Fix] Fix dependencies id sanity check + * [Fix] Fix folding for semicolon separated tokens + * [Fix] Fix largest possible TLD behaviour + * [Fix] Fix last token folding + * [Fix] Fix length calculations in white on white rule + * [Fix] Fix multiple request headers structure + * [Fix] Fix multiple values headers freeing + * [Fix] Fix parsing of background color + * [Fix] Fix printing from field in log_urls + * [Fix] Fix processing of last element of DMARC policies + * [Fix] Further fixes for HTML colors + * [Fix] Further fixes for multiple values headers + * [Fix] Further fixes for white on white rule + * [Fix] Further fixes in HTML tags parsing + * [Fix] Ignore content type/subtype case + * [Fix] Increase score of R_WHITE_ON_WHITE + * [Fix] Parse CGP envelope data + * [Fix] Propagate colors in HTML + * [Fix] Restore multiple values headers in protocol + * [Fix] Restore multiple values in headers processing + * [Fix] Some more changes to tag's content length calculations + * [Fix] Some more fixes for low contrast fonts detector + * [Fix] SpamAssassin plugin: support check_freemail_header('EnvelopeFrom', [..]) + * [Fix] Trigger HTML_SHORT_LINK_IMG on any external image + * [Fix] rspamd_stats: remove deprecated defined(@array) 1.3.1: - * [CritFix] Fix catena passwords validation - * [CritFix] Fix crash when the first received is faked - * [Feature] Add DMARC_BAD_POLICY symbol when DMARC policy was invalid - * [Feature] Allow for matching hostnames in multimap (#773) - * [Feature] Allow for setting action based on DMARC disposition - * [Feature] Allow limiting of the inbound message size - * [Feature] Allow maps with multiple symbols and scores - * [Feature] Allow regexps in the emails maps - * [Feature] Allow to register metric symbols from multimap - * [Feature] Allow to reset redis tokens instead of appendig values - * [Feature] Allow to store strings in radix maps - * [Feature] Check UTF validity when there are utf regexps in a map - * [Feature] Correctly work when there is no hard reject action - * [Feature] Implement dependencies for maps - * [Fix] Another effort to unbreak sqlite locking - * [Fix] Avoid crash when closing mmapped file - * [Fix] Do not break history on NaN in required score - * [Fix] Ensure that hyperscan cache written is written properly - * [Fix] Filter NaN from scores in history - * [Fix] Fix DNSBL maps - * [Fix] Fix another locking issue in sqlite - * [Fix] Fix another locking issue with mapped files - * [Fix] Fix deadlock in mmaped file stats - * [Fix] Fix dependencies in multimap plugin - * [Fix] Fix emails module configuration - * [Fix] Fix greylist plugin (#755) - * [Fix] Fix greylisting plugin variable usage - * [Fix] Fix installed permissions for rspamd_stats - * [Fix] Fix locking in mmapped statistics - * [Fix] Fix paths in tests - * [Fix] Fix prefilter mode for multimap - * [Fix] Forgot to commit leftover changes - * [Fix] Really fix local.d includes - * [Fix] Restore selective greylisting behaviour - * [Fix] Set max size on per connection basis - * [Fix] Use temporary storage for hyperscan cache - * [Rework] Remove systemd socket activation + * [CritFix] Fix catena passwords validation + * [CritFix] Fix crash when the first received is faked + * [Feature] Add DMARC_BAD_POLICY symbol when DMARC policy was invalid + * [Feature] Allow for matching hostnames in multimap (#773) + * [Feature] Allow for setting action based on DMARC disposition + * [Feature] Allow limiting of the inbound message size + * [Feature] Allow maps with multiple symbols and scores + * [Feature] Allow regexps in the emails maps + * [Feature] Allow to register metric symbols from multimap + * [Feature] Allow to reset redis tokens instead of appendig values + * [Feature] Allow to store strings in radix maps + * [Feature] Check UTF validity when there are utf regexps in a map + * [Feature] Correctly work when there is no hard reject action + * [Feature] Implement dependencies for maps + * [Fix] Another effort to unbreak sqlite locking + * [Fix] Avoid crash when closing mmapped file + * [Fix] Do not break history on NaN in required score + * [Fix] Ensure that hyperscan cache written is written properly + * [Fix] Filter NaN from scores in history + * [Fix] Fix DNSBL maps + * [Fix] Fix another locking issue in sqlite + * [Fix] Fix another locking issue with mapped files + * [Fix] Fix deadlock in mmaped file stats + * [Fix] Fix dependencies in multimap plugin + * [Fix] Fix emails module configuration + * [Fix] Fix greylist plugin (#755) + * [Fix] Fix greylisting plugin variable usage + * [Fix] Fix installed permissions for rspamd_stats + * [Fix] Fix locking in mmapped statistics + * [Fix] Fix paths in tests + * [Fix] Fix prefilter mode for multimap + * [Fix] Forgot to commit leftover changes + * [Fix] Really fix local.d includes + * [Fix] Restore selective greylisting behaviour + * [Fix] Set max size on per connection basis + * [Fix] Use temporary storage for hyperscan cache + * [Rework] Remove systemd socket activation 1.3.0: - * [CritFix] Fix SA rawbody processing - exclude top part - * [CritFix] Fix decoding of UTF HTML entitles - * [CritFix] Fix encrypted fuzzy requests - * [CritFix] Fix leak of shared memory fds and files - * [CritFix] Fix levenshtein distance calculations - * [CritFix] Fix mime headers processing - * [CritFix] Fix parsing of URLs in texts - * [CritFix] Fix parsing of missing classes - * [CritFix] Fix redis structure by adding {NULL, NULL} member - * [CritFix] Fix some more URL detector issues - * [CritFix] Fix systemd sockets activation - * [CritFix] Fix unencrypted passwords processing in the controller - * [CritFix] Fix writing CDPs to the database - * [CritFix] Fix writing of encrypted HTTP requests - * [CritFix] Plug memory leak in dkim module - * [CritFix] Plug memory leak in headers getting code - * [CritFix] Pre-filters and post-filters were completely broken - * [CritFix] Properly support SA body regexps - * [CritFix] Really skip filters in case of pre-result set - * [CritFix] Restore the intended pre-filters behaviour - * [Rework] Adopt new maps code - * [Rework] Compile ragel sources when building rspamd - * [Rework] Finish rework for the rest of places that use HTTP - * [Rework] Fix DKIM headers canonicalization - * [Rework] Fix lua maps API - * [Rework] Import linenoise for line editing - * [Rework] Include config structure to all rcl handlers - * [Rework] Make chartable module useful - * [Rework] Move http internal structures to a private header - * [Rework] Partly fix controller - * [Rework] Remove dedicated images list - * [Rework] Rename http proxy to rspamd proxy - * [Rework] Rename mime parts structures - * [Rework] Rework HTTP code - * [Rework] Rework exceptions and newlines processing - * [Rework] Rework pre and postfilters system - * [Rework] Separate method to close backend connections - * [Rework] Start the complete maps rework - * [Rework] Use dynamically generated ragel C sources - * [Feature] Add 'blacklist' and 'strict' modes for whitelists - * [Feature] Add 'symbols_enabled' and 'groups_enabled' to settings - * [Feature] Add ESMTPSA received type - * [Feature] Add a simple script to evaluate rspamd rules in the logs - * [Feature] Add a simple tool to generate DKIM keys - * [Feature] Add a trivial heuristic for codepages - * [Feature] Add and use mumhash for non-crypto hashing - * [Feature] Add better method to check lua userdata types - * [Feature] Add better zip files search algorithm - * [Feature] Add catena PBKDF function - * [Feature] Add configuration knobs for in and out parser scripts - * [Feature] Add content filtering support to multimap - * [Feature] Add different timeouts for proxy connections - * [Feature] Add dot commands for lua REPL: - * [Feature] Add execution of lua global functions script - * [Feature] Add function for pretty printing of inet addresses - * [Feature] Add function to convert fstring_t to c string - * [Feature] Add function to create temporary shared memory mapping - * [Feature] Add function to generate random hex string - * [Feature] Add generic fucnction to parse IP maps - * [Feature] Add initial version of HTTP lua repl - * [Feature] Add learn condition to the default configuration - * [Feature] Add learn conditions for classifiers - * [Feature] Add limit for dkim signatures to be checked - * [Feature] Add locking routines for lua_util - * [Feature] Add lua API for getting info from archives - * [Feature] Add lua utility to decode URL encoding - * [Feature] Add method to copy message from http connection - * [Feature] Add mirrors feature - * [Feature] Add more algorithms for shingles generation - * [Feature] Add more domains to redirectors list - * [Feature] Add more encodingsto cryptobox hash API - * [Feature] Add more file utilities to lua_util - * [Feature] Add more functions to extract data from text parts - * [Feature] Add more methods to get headers from a task - * [Feature] Add more methods to init http message body - * [Feature] Add more options for redis config parsing function - * [Feature] Add new representation of email address - * [Feature] Add new symbols to filter bad extensions in messages - * [Feature] Add new utility methods to mimepart object - * [Feature] Add openphish support to rspamd phishing module - * [Feature] Add parsers for SMTP address in ragel - * [Feature] Add parsing of mirror hosts for fuzzy storage - * [Feature] Add parsing scripts for master connection as well - * [Feature] Add preliminary greylist plugin - * [Feature] Add preliminary phishtank support - * [Feature] Add preliminary rarv5 support - * [Feature] Add preliminary version of ssl toolbox - * [Feature] Add protection against open files limit and accepting sockets - * [Feature] Add rar v4 support - * [Feature] Add reading scripts for master connection - * [Feature] Add replies plugin - * [Feature] Add results parsing code - * [Feature] Add routines to compare and check pubkeys - * [Feature] Add simpler versions of refcounts - * [Feature] Add some time manipulation functions for lua APi - * [Feature] Add support for non-standard BATV signatures - * [Feature] Add support of address with at-domain list - * [Feature] Add support to search archives by magic - * [Feature] Add task:get_rawbody method - * [Feature] Add test to check shared memory support sanity - * [Feature] Add the initial version of LUA repl to rspamadm - * [Feature] Add throughput graph for RRD backend to WebUI - * [Feature] Add universal function to make a proper redis request - * [Feature] Add universal function to parse redis servers for plugins - * [Feature] Add util.unlink function - * [Feature] Add utility function to return random number from 0 to 1 - * [Feature] Add utility method to convert ftok to C string - * [Feature] Add utility to map shared memory segments - * [Feature] Add versions to fuzzy storage - * [Feature] Add workaround for legacy clients in rspamd proxy - * [Feature] Add workaround for systems without sane shmem support - * [Feature] Add xoroshiro+ fast rng for non-crypto purposes - * [Feature] Adopt plugins for new maps API - * [Feature] Allow SPF to be checked for empty tasks - * [Feature] Allow binary patterns in lua_trie - * [Feature] Allow catena encrypted passwords in controller - * [Feature] Allow client ip match in the settings - * [Feature] Allow easy adding and overriding of fuzzy rules - * [Feature] Allow empty tasks to be processed - * [Feature] Allow hostnames in IP maps - * [Feature] Allow https maps - * [Feature] Allow multiple PBKDF types in `rspamadm pw` - * [Feature] Allow named fuzzy rules - * [Feature] Allow non zero terminated patterns in multipattern - * [Feature] Allow partial hash updates - * [Feature] Allow pipelining for redis.make_request - * [Feature] Allow sending empty requests using client - * [Feature] Allow setting fuzzy flag by symbol not by value - * [Feature] Allow setting scores and actions from lua - * [Feature] Allow shared memory simple http client - * [Feature] Allow static lua files in any parts of rspamd sources - * [Feature] Allow to change flag from fuzzy learn condition - * [Feature] Allow to check rspamd_text using maps - * [Feature] Allow to disable composite rules from settings - * [Feature] Allow to disable some modules from common redis setup - * [Feature] Allow to extract ucl_object from lua using common API - * [Feature] Allow to get settings and settings id hash from lua_task - * [Feature] Allow to have specific settings for mirrored traffic - * [Feature] Allow to open message from a shared memory segment - * [Feature] Allow to parse pubkeys from the rcl config - * [Feature] Allow to pass extradata from rspamd to rmilter - * [Feature] Allow to query storage about number of fuzzy hashes stored - * [Feature] Allow to read logs without symbols scores - * [Feature] Allow to read password from console for rspamc - * [Feature] Allow to set ciphers and CA paths in config - * [Feature] Allow to skip some initialization phases to speed up rspamadm - * [Feature] Allow underscore separated names in settings - * [Feature] Allow versioning for sqlite databases - * [Feature] Always allow to terminate rspamd - * [Feature] Better deal with backend errors - * [Feature] Better lua_redis logging - * [Feature] Configure CA path and ciphers - * [Feature] Create a dedicated parser to strip newlines - * [Feature] Deduplicate the same urls in multimap module - * [Feature] Distinguish luajit from lua - * [Feature] Do not print garbadge in --compact output - * [Feature] Dynamically detect if a CPU is incompatible with hyperscan - * [Feature] Enable forced resolving for some lua plugins - * [Feature] Enable rrd by default - * [Feature] Enable workaround for exim - * [Feature] Fix task functions to work without rspamd_config - * [Feature] Further improvements to chartable module - * [Feature] Further micro-optimizations for hashing and shingles - * [Feature] Further relax parser - * [Feature] Humanize numbers in stats widgets - * [Feature] Implement HTTPS client - * [Feature] Implement SSL support in http client - * [Feature] Implement body rules for the trie plugin - * [Feature] Implement braced regexp quantifiers - * [Feature] Implement compare scripts for mirrors results - * [Feature] Implement compare scripts setup - * [Feature] Implement composites policies - * [Feature] Implement conditional learning for classifiers - * [Feature] Implement constructing of map from UCL - * [Feature] Implement dkim signing in dkim check plugin - * [Feature] Implement fuzzy storage updates - * [Feature] Implement fuzzy updates push protocol - * [Feature] Implement https maps - * [Feature] Implement inter-process maps caching - * [Feature] Implement mapping of remote flags to local flags - * [Feature] Implement mirroring in rspamd proxy - * [Feature] Implement multi-flags fuzzy replies - * [Feature] Implement multiple-sources fuzzy storage - * [Feature] Implement order of pre/post filters - * [Feature] Implement partial deleting for multi-flags - * [Feature] Implement pipelining for redis async interface - * [Feature] Implement ragel parser for received headers - * [Feature] Implement reading of messages to shared memory - * [Feature] Implement refcount for messages - * [Feature] Implement retransmits for master connection - * [Feature] Implement zero-copy mode for HTTP reading - * [Feature] Improve SPF domain detection logic - * [Feature] Improve config:register_symbol function - * [Feature] Improve error report for type mismatch in lua - * [Feature] Improve fstrings API - * [Feature] Improve getting SMTP data from lua_task - * [Feature] Improve levenshtein distance function - * [Feature] Improve logging in proxy and add refcounts - * [Feature] Improve logging lua types - * [Feature] Improve master/slave logging - * [Feature] Improve phishing plugin - * [Feature] Improve phishtank and openphish support - * [Feature] Improve ragel build target - * [Feature] Improve statistics script - * [Feature] Initialize ssl library to use SSL connections - * [Feature] Interpolate points sent to webui - * [Feature] Limit logging of elements that could have too many items - * [Feature] Lock ANN file when loading - * [Feature] New abstract hashing API in cryptobox - * [Feature] Normalize quoted addresses - * [Feature] Now cryptobox lua API accepts rspamd text as input - * [Feature] Optimize alignment to speed up hashing - * [Feature] Parse received date and ESMTPA proto - * [Feature] Parse received timestamp - * [Feature] Pass settings id to log helper - * [Feature] Pass settings id to lua script from log helper - * [Feature] Perform files expansion on proxying - * [Feature] Preliminary implementation of fuzzy master/slave updates - * [Feature] Print userdata using tostring if possible - * [Feature] Propagate HTTP result string when using proxy - * [Feature] Properly implement unweighted round-robin algorithm - * [Feature] Reduce number of timers queries - * [Feature] Rework and improve fuzzy storage - * [Feature] Rework dns resolving API for lua, add 'forced' option - * [Feature] Rework fann module to understand settings - * [Feature] Rework listening system to allow multiple socket types per worker - * [Feature] Rework ratelimit module to set expiration - * [Feature] Save bayes probability in memory pool var - * [Feature] Save settings id hash for convenience - * [Feature] Search for SSL_set_tlsext_host_name support - * [Feature] Send DKIM signature to protocol reply - * [Feature] Show DKIM signature in rspamc client - * [Feature] Show symbols description in scan output - * [Feature] Sign message merely after DKIM check - * [Feature] Simplify machines by assuming that headers are unfolded - * [Feature] Sort symbols when displaying them in log - * [Feature] Split main connection from mirrored connections - * [Feature] Start moving to the new email address structure - * [Feature] Store HTTP headers in a hash table - * [Feature] Store more information about compressed files - * [Feature] Store raw headers value to use them in DKIM - * [Feature] Store text parts content with newlines stripped - * [Feature] Support DKIM signing - * [Feature] Support EXIF jpeg images - * [Feature] Support archive files list extraction - * [Feature] Support archives when matching patterns in multimap - * [Feature] Support premium/academic feed for openphish - * [Feature] Support rspamd_updates via https - * [Feature] Supprort FQDNs in phishing module maps - * [Feature] Try to read on fuzzy timeout to avoid fake timeouts - * [Feature] Try to select the optimal possible function for input - * [Feature] Unescape and unquote smtp addresses - * [Feature] Update fuzzy timestamp when adding value - * [Feature] Update mumhash - * [Feature] Use -flto if possible when optimizations are enabled - * [Feature] Use extended map types in lua map, unify code - * [Feature] Use file lock in logger to avoid deadlocks - * [Feature] Use generic global string split function - * [Feature] Use metrohash as well - * [Feature] Use mumhash by default for incremental hashing - * [Feature] Use mumhash for words hashing - * [Feature] Use new ip parsing API - * [Feature] Use new maps API for local addrs - * [Feature] Use new ragel parser in message parsing code - * [Feature] Use new received parser instead of old one - * [Feature] Use new redis API in DMARC plugin - * [Feature] Use new redis API in greylist plugin - * [Feature] Use new redis API in ip_score plugin - * [Feature] Use new redis API in ratelimit plugin - * [Feature] Use new redis API in replies plugin - * [Feature] Use new version of register_symbol in rspamd plugins - * [Feature] Use offset when passing shm to deal with encrypted requests - * [Feature] Use one pass to remove newlines and store their positions - * [Feature] Use rspamd specific type checks for userdata - * [Feature] Use shared memory storage for http maps - * [Feature] Use universal redis definitions in rspamd plugins - * [Feature] Various improvements in greylist module - * [Feature] Wait for sqlite if locked when switching to WAL mode - * [Fix] Add filenames sanity filtering for mime types - * [Fix] Add guards for empty parts - * [Fix] Add missing types - * [Fix] Add more priority for config file symbols registered from UCL - * [Fix] Add sanity checks when compiling regexp maps - * [Fix] Add spaces instead of newlines to the normalized content - * [Fix] Add workaround for ancient openssl - * [Fix] Add workaround for gmime CTE stupidity - * [Fix] Add workaround for hex digits - * [Fix] Adjust MISSING_MIMEOLE score - * [Fix] Adjust body/body_buf when stealing encrypted message - * [Fix] Adopt lua task API for the new email addresses structure - * [Fix] Allow for disabling DMARC reporting when Redis is configured - * [Fix] Always register openphish and phishtank virtual symbols - * [Fix] Always use shmem on linux - * [Fix] Another change of newlines policy - * [Fix] Another d3evolution update - * [Fix] Another fix for exim workaround - * [Fix] Another fix for legacy clients - * [Fix] Another fix for maps scheduling - * [Fix] Another fix for marking upstreams inactive - * [Fix] Another fix for postfilters - * [Fix] Another fix for redis timeouts - * [Fix] Avoid `table.getn` method as it has been removed in lua 5.3 - * [Fix] Avoid double hashing for images - * [Fix] Avoid linking with actrie if hyperscan is enabled - * [Fix] Check copy result when sending message to mirrors - * [Fix] Cleanup message when assiging body - * [Fix] Cleanup stack from global vars - * [Fix] Correctly parse query type - * [Fix] Disable all symbols if enable_groups is found in settings - * [Fix] Disable fts as it is completely broken in bloody linux - * [Fix] Disable multiple autolearn checks - * [Fix] Disallow updates by default - * [Fix] Do not abort when cannot load a map - * [Fix] Do not check recursion for non-DNS SPF record types - * [Fix] Do not delete uninitialized events - * [Fix] Do not die if shmem_mkstemp fails - * [Fix] Do not die when no metrics defined - * [Fix] Do not even try pcre in case of regexp maps - * [Fix] Do not greylist messages if redis has failed somehow - * [Fix] Do not greylist on rejection - * [Fix] Do not leave temporary maps cached - * [Fix] Do not output meaningless errors - * [Fix] Do not send NaN in json - * [Fix] Don't mix hyperscan and pcre processing within a same task - * [Fix] Finally rework and simplify redis backend for statistics - * [Fix] Fix Exim shutdown patch - * [Fix] Fix JIT compilation for PCRE2 expressions - * [Fix] Fix JIT usage for PCRE2 - * [Fix] Fix REPL output - * [Fix] Fix SMTP address parsing machine - * [Fix] Fix UTF8 mode in PCRE2 - * [Fix] Fix a stupid misprint in word 'phishing' - * [Fix] Fix accepting the first update when local idx is -1 - * [Fix] Fix adding maps from ucl - * [Fix] Fix adding upstream to an active queue - * [Fix] Fix and rescore R_PARTS_DIFFER logic - * [Fix] Fix body rules in SA plugin - * [Fix] Fix body start position - * [Fix] Fix border case in urls detector - * [Fix] Fix border cases for incremental hashing - * [Fix] Fix caseless uthash application - * [Fix] Fix chartable issue with starting digits - * [Fix] Fix client_ip in users settings - * [Fix] Fix compilation issue - * [Fix] Fix conditional learning - * [Fix] Fix crash on empty maps - * [Fix] Fix creating of URLs from LUA - * [Fix] Fix creating of temporary shmem segment - * [Fix] Fix creation of mmapped statfiles - * [Fix] Fix descriptors leak on shmem detaching - * [Fix] Fix detaching of shared memory segments - * [Fix] Fix detection of URLs in text parts - * [Fix] Fix directories processing for rspamc - * [Fix] Fix displaying of rewrite subject in WebUI - * [Fix] Fix dkim private keys operations - * [Fix] Fix dkim signing - * [Fix] Fix dynamic scoring of subject symbols - * [Fix] Fix email address build - * [Fix] Fix encrypted proxy requests - * [Fix] Fix errors counting in upstreams - * [Fix] Fix errors handling in the proxy - * [Fix] Fix event bases for IO events - * [Fix] Fix events handling when scheduling map wacth - * [Fix] Fix fann rewrite - * [Fix] Fix files fallback for shmem transfer - * [Fix] Fix fuzzy adding in webui - * [Fix] Fix fuzzy storage encrypted mirroring - * [Fix] Fix fuzzy storage sync replies - * [Fix] Fix handling of the same words - * [Fix] Fix inserting values to the sources list - * [Fix] Fix ipv6 mask application - * [Fix] Fix issue with missing recipients - * [Fix] Fix issues with multiple returns from lua - * [Fix] Fix learning for non-existent backend - * [Fix] Fix legacy clients support in proxy - * [Fix] Fix length calculations for shared memory segments - * [Fix] Fix listening on UDP sockets - * [Fix] Fix loading of file maps - * [Fix] Fix long regexp flags (e.g. {sa_body}) - * [Fix] Fix lua compare function init - * [Fix] Fix maps descriptions - * [Fix] Fix maps locking - * [Fix] Fix max_train setup in ANN module - * [Fix] Fix memory corruption - * [Fix] Fix memory leak in unsigned maps reading - * [Fix] Fix misprints for lto usage - * [Fix] Fix more issues with scripts processing - * [Fix] Fix next-to-last extension length check - * [Fix] Fix openssl initialization - * [Fix] Fix order of arguments - * [Fix] Fix order of initialization - * [Fix] Fix parser - * [Fix] Fix parsing of binary tries - * [Fix] Fix parsing of braced IPv6 addresses - * [Fix] Fix parsing of nested braces in SMTP comments - * [Fix] Fix parsing of rarv5 archives - * [Fix] Fix periodic scheduling when a map is not modified - * [Fix] Fix possible FP in TRACKER_ID rule - * [Fix] Fix post-filters processing - * [Fix] Fix potential NULL dereference - * [Fix] Fix processing of
and
tags - * [Fix] Fix processing of addresses in protocol - * [Fix] Fix processing of messages without received headers - * [Fix] Fix proxying issue for unconnected sessions - * [Fix] Fix proxying of the encrypted messages - * [Fix] Fix race condition with shared memory by refcounts - * [Fix] Fix ratelimit initialization - * [Fix] Fix redis set request in replies plugin - * [Fix] Fix redis timeout events handling - * [Fix] Fix redis timeouts processing logic - * [Fix] Fix refcounts in lua_redis - * [Fix] Fix results checking if no master connection is active - * [Fix] Fix return value for couple of lua functions - * [Fix] Fix round-robin selection when upstreams have no weight - * [Fix] Fix rows calculation in graph - * [Fix] Fix rspamd_redis_make_request syntax in replies plugin - * [Fix] Fix scheduling of locked map events - * [Fix] Fix scores detection - * [Fix] Fix searching for newline positions - * [Fix] Fix secure_ip setting in controller - * [Fix] Fix sending data to graph command - * [Fix] Fix setting of score for parts differ - * [Fix] Fix setting of the lua top - * [Fix] Fix setting path for lua - * [Fix] Fix setting path for phishtank - * [Fix] Fix settings application - * [Fix] Fix shm_open call as described in POSIX - * [Fix] Fix size of length in fuzzy mirror wire protocol - * [Fix] Fix smtp grammar issues - * [Fix] Fix some issues with redis API - * [Fix] Fix some issues with retries in the proxy - * [Fix] Fix stack growing - * [Fix] Fix start of body detection in DKIM - * [Fix] Fix state on timeout - * [Fix] Fix stats script - * [Fix] Fix substring search when there are '\0' in strings - * [Fix] Fix symbol name for spf soft fail - * [Fix] Fix symbol type's check - * [Fix] Fix symbols registration and execution - * [Fix] Fix the case of multiple values keys - * [Fix] Fix the default symbol names according to metric - * [Fix] Fix timeout setup on learning - * [Fix] Fix timeouts in redis cache processing - * [Fix] Fix timeouts processing in lua_redis - * [Fix] Fix upstreams interaction for rspamd proxy - * [Fix] Fix usage of rdns reply structure - * [Fix] Fix varargs loop - * [Fix] Fix whitelists and blacklists in SA rules - * [Fix] Fix write servers setup for redis - * [Fix] Fix writing of HTTP messages - * [Fix] Force rspamd to upgrade fuzzy db on opening - * [Fix] Free the correct pointer - * [Fix] Further fixes for lto and static linking - * [Fix] Further fixes for surbl extensions map - * [Fix] Further fixes in maps code - * [Fix] Further improvements to error messages in fuzzy check - * [Fix] Further tweaks to redis garbadge collection - * [Fix] Groups are now case insensitive - * [Fix] Handle log pipe read errors - * [Fix] Handle nested dependencies in SpamAssassin plugin - * [Fix] Implement new automata to skip empty lines for dkim signing - * [Fix] Improve error messages on fuzzy add - * [Fix] Improve lua redis handling - * [Fix] Improve phishing module logging - * [Fix] Improve printing of fuzzy errors - * [Fix] Improve rrd diagnostic errors - * [Fix] Improve strcase hash used in uthash - * [Fix] Include fuzzy key to distinguish storages with different keys - * [Fix] Include slave cluster name into http request - * [Fix] Include some more information about archives - * [Fix] Indicate upstream error on timeout - * [Fix] Initialize hash tables array to avoid crashes - * [Fix] Initialize parser scripts properly - * [Fix] Initialize vars to avoid warnings - * [Fix] Inverse logic for saving ANN - * [Fix] Link lpeg to rspamd lua library - * [Fix] Make extension checks case-insensitive - * [Fix] Mark expired hashes as not found and not as zero flag - * [Fix] Match archive name as well - * [Fix] More and more fixes to redis states - * [Fix] More fixes about shared memory in proxy - * [Fix] More fixes for redis refcounts - * [Fix] More fixes to end of headers detection - * [Fix] More fixes to events logic - * [Fix] More fixes to multi-flag fuzzy storage - * [Fix] More fixes to parts distance calculations - * [Fix] More guards for redis free - * [Fix] One more fix in redis destructor - * [Fix] One more try to fix redis - * [Fix] PIE is required for static build - * [Fix] Partial fix for mmap'd statistics tests - * [Fix] Plug memory leak in proxy - * [Fix] Properly detect end of headers position - * [Fix] Properly init and free session structures - * [Fix] Reduce PRECEDENCE_BULK rule weight - * [Fix] Reduce the default thresholds for learning - * [Fix] Remove Type=forking from systemd unit file (#709) - * [Fix] Remove bad FANN file to save computational resources - * [Fix] Remove event before closing of fd to avoid race conditions - * [Fix] Remove parsing of 'from' variable in redis backend - * [Fix] Remove some bad domains from whitelists - * [Fix] Repair optional dependencies - * [Fix] Reset master connection when retransmitting scan request - * [Fix] Restore ONCE_RECEIVED symbol - * [Fix] Restore compatibility with old lua API behaviour - * [Fix] Restore redis runtime state - * [Fix] Reverse options when received - * [Fix] Send updates to mirrors only if we have some changes - * [Fix] Set host attribute properly when making HTTP request from lua - * [Fix] Set terminated state before calling of async free - * [Fix] Simplify MISSING_MIMEOLE rule - * [Fix] Simplify state machine by ignoring multiple spaces - * [Fix] Skip setting RPATH for static builds - * [Fix] Slightly reduce weights of rules with high FP rate - * [Fix] Some fixes to libmagic initialization - * [Fix] Some more fixes to ratelimit plugin - * [Fix] Strip '\r\n' properly - * [Fix] Switch hashes to mumhash - * [Fix] Treat NaN values properly in graph command - * [Fix] Try to avoid FP when checking for phished URLs - * [Fix] Try to avoid recursive events deletions - * [Fix] Try to fix false positive URL detections in text parts - * [Fix] Try to fix issue in redis stats backend when task is closed - * [Fix] Try to fix proxying of stupid spamc protocol to HTTP mirrors - * [Fix] Try to fix redis crashes - * [Fix] Try to fix upstreams with one element - * [Fix] Try to handle multiline history in a more sane way - * [Fix] Unbreak build on gcc < 4.9 - * [Fix] Update RPM spec/sources (#700) - * [Fix] Update d3evolution version - * [Fix] Update mumhash implementation - * [Fix] Use custom error function for pre and post filters - * [Fix] Use new postfilters and prefilters API in the plugins - * [Fix] Use non-blocking mode for systemd sockets - * [Fix] Use shared memory merely for local backends in the proxy - * [Fix] Use watchers for spf plugin - * [Fix] Varioud fixes to the maps code + * [CritFix] Fix SA rawbody processing - exclude top part + * [CritFix] Fix decoding of UTF HTML entitles + * [CritFix] Fix encrypted fuzzy requests + * [CritFix] Fix leak of shared memory fds and files + * [CritFix] Fix levenshtein distance calculations + * [CritFix] Fix mime headers processing + * [CritFix] Fix parsing of URLs in texts + * [CritFix] Fix parsing of missing classes + * [CritFix] Fix redis structure by adding {NULL, NULL} member + * [CritFix] Fix some more URL detector issues + * [CritFix] Fix systemd sockets activation + * [CritFix] Fix unencrypted passwords processing in the controller + * [CritFix] Fix writing CDPs to the database + * [CritFix] Fix writing of encrypted HTTP requests + * [CritFix] Plug memory leak in dkim module + * [CritFix] Plug memory leak in headers getting code + * [CritFix] Pre-filters and post-filters were completely broken + * [CritFix] Properly support SA body regexps + * [CritFix] Really skip filters in case of pre-result set + * [CritFix] Restore the intended pre-filters behaviour + * [Rework] Adopt new maps code + * [Rework] Compile ragel sources when building rspamd + * [Rework] Finish rework for the rest of places that use HTTP + * [Rework] Fix DKIM headers canonicalization + * [Rework] Fix lua maps API + * [Rework] Import linenoise for line editing + * [Rework] Include config structure to all rcl handlers + * [Rework] Make chartable module useful + * [Rework] Move http internal structures to a private header + * [Rework] Partly fix controller + * [Rework] Remove dedicated images list + * [Rework] Rename http proxy to rspamd proxy + * [Rework] Rename mime parts structures + * [Rework] Rework HTTP code + * [Rework] Rework exceptions and newlines processing + * [Rework] Rework pre and postfilters system + * [Rework] Separate method to close backend connections + * [Rework] Start the complete maps rework + * [Rework] Use dynamically generated ragel C sources + * [Feature] Add 'blacklist' and 'strict' modes for whitelists + * [Feature] Add 'symbols_enabled' and 'groups_enabled' to settings + * [Feature] Add ESMTPSA received type + * [Feature] Add a simple script to evaluate rspamd rules in the logs + * [Feature] Add a simple tool to generate DKIM keys + * [Feature] Add a trivial heuristic for codepages + * [Feature] Add and use mumhash for non-crypto hashing + * [Feature] Add better method to check lua userdata types + * [Feature] Add better zip files search algorithm + * [Feature] Add catena PBKDF function + * [Feature] Add configuration knobs for in and out parser scripts + * [Feature] Add content filtering support to multimap + * [Feature] Add different timeouts for proxy connections + * [Feature] Add dot commands for lua REPL: + * [Feature] Add execution of lua global functions script + * [Feature] Add function for pretty printing of inet addresses + * [Feature] Add function to convert fstring_t to c string + * [Feature] Add function to create temporary shared memory mapping + * [Feature] Add function to generate random hex string + * [Feature] Add generic fucnction to parse IP maps + * [Feature] Add initial version of HTTP lua repl + * [Feature] Add learn condition to the default configuration + * [Feature] Add learn conditions for classifiers + * [Feature] Add limit for dkim signatures to be checked + * [Feature] Add locking routines for lua_util + * [Feature] Add lua API for getting info from archives + * [Feature] Add lua utility to decode URL encoding + * [Feature] Add method to copy message from http connection + * [Feature] Add mirrors feature + * [Feature] Add more algorithms for shingles generation + * [Feature] Add more domains to redirectors list + * [Feature] Add more encodingsto cryptobox hash API + * [Feature] Add more file utilities to lua_util + * [Feature] Add more functions to extract data from text parts + * [Feature] Add more methods to get headers from a task + * [Feature] Add more methods to init http message body + * [Feature] Add more options for redis config parsing function + * [Feature] Add new representation of email address + * [Feature] Add new symbols to filter bad extensions in messages + * [Feature] Add new utility methods to mimepart object + * [Feature] Add openphish support to rspamd phishing module + * [Feature] Add parsers for SMTP address in ragel + * [Feature] Add parsing of mirror hosts for fuzzy storage + * [Feature] Add parsing scripts for master connection as well + * [Feature] Add preliminary greylist plugin + * [Feature] Add preliminary phishtank support + * [Feature] Add preliminary rarv5 support + * [Feature] Add preliminary version of ssl toolbox + * [Feature] Add protection against open files limit and accepting sockets + * [Feature] Add rar v4 support + * [Feature] Add reading scripts for master connection + * [Feature] Add replies plugin + * [Feature] Add results parsing code + * [Feature] Add routines to compare and check pubkeys + * [Feature] Add simpler versions of refcounts + * [Feature] Add some time manipulation functions for lua APi + * [Feature] Add support for non-standard BATV signatures + * [Feature] Add support of address with at-domain list + * [Feature] Add support to search archives by magic + * [Feature] Add task:get_rawbody method + * [Feature] Add test to check shared memory support sanity + * [Feature] Add the initial version of LUA repl to rspamadm + * [Feature] Add throughput graph for RRD backend to WebUI + * [Feature] Add universal function to make a proper redis request + * [Feature] Add universal function to parse redis servers for plugins + * [Feature] Add util.unlink function + * [Feature] Add utility function to return random number from 0 to 1 + * [Feature] Add utility method to convert ftok to C string + * [Feature] Add utility to map shared memory segments + * [Feature] Add versions to fuzzy storage + * [Feature] Add workaround for legacy clients in rspamd proxy + * [Feature] Add workaround for systems without sane shmem support + * [Feature] Add xoroshiro+ fast rng for non-crypto purposes + * [Feature] Adopt plugins for new maps API + * [Feature] Allow SPF to be checked for empty tasks + * [Feature] Allow binary patterns in lua_trie + * [Feature] Allow catena encrypted passwords in controller + * [Feature] Allow client ip match in the settings + * [Feature] Allow easy adding and overriding of fuzzy rules + * [Feature] Allow empty tasks to be processed + * [Feature] Allow hostnames in IP maps + * [Feature] Allow https maps + * [Feature] Allow multiple PBKDF types in `rspamadm pw` + * [Feature] Allow named fuzzy rules + * [Feature] Allow non zero terminated patterns in multipattern + * [Feature] Allow partial hash updates + * [Feature] Allow pipelining for redis.make_request + * [Feature] Allow sending empty requests using client + * [Feature] Allow setting fuzzy flag by symbol not by value + * [Feature] Allow setting scores and actions from lua + * [Feature] Allow shared memory simple http client + * [Feature] Allow static lua files in any parts of rspamd sources + * [Feature] Allow to change flag from fuzzy learn condition + * [Feature] Allow to check rspamd_text using maps + * [Feature] Allow to disable composite rules from settings + * [Feature] Allow to disable some modules from common redis setup + * [Feature] Allow to extract ucl_object from lua using common API + * [Feature] Allow to get settings and settings id hash from lua_task + * [Feature] Allow to have specific settings for mirrored traffic + * [Feature] Allow to open message from a shared memory segment + * [Feature] Allow to parse pubkeys from the rcl config + * [Feature] Allow to pass extradata from rspamd to rmilter + * [Feature] Allow to query storage about number of fuzzy hashes stored + * [Feature] Allow to read logs without symbols scores + * [Feature] Allow to read password from console for rspamc + * [Feature] Allow to set ciphers and CA paths in config + * [Feature] Allow to skip some initialization phases to speed up rspamadm + * [Feature] Allow underscore separated names in settings + * [Feature] Allow versioning for sqlite databases + * [Feature] Always allow to terminate rspamd + * [Feature] Better deal with backend errors + * [Feature] Better lua_redis logging + * [Feature] Configure CA path and ciphers + * [Feature] Create a dedicated parser to strip newlines + * [Feature] Deduplicate the same urls in multimap module + * [Feature] Distinguish luajit from lua + * [Feature] Do not print garbadge in --compact output + * [Feature] Dynamically detect if a CPU is incompatible with hyperscan + * [Feature] Enable forced resolving for some lua plugins + * [Feature] Enable rrd by default + * [Feature] Enable workaround for exim + * [Feature] Fix task functions to work without rspamd_config + * [Feature] Further improvements to chartable module + * [Feature] Further micro-optimizations for hashing and shingles + * [Feature] Further relax parser + * [Feature] Humanize numbers in stats widgets + * [Feature] Implement HTTPS client + * [Feature] Implement SSL support in http client + * [Feature] Implement body rules for the trie plugin + * [Feature] Implement braced regexp quantifiers + * [Feature] Implement compare scripts for mirrors results + * [Feature] Implement compare scripts setup + * [Feature] Implement composites policies + * [Feature] Implement conditional learning for classifiers + * [Feature] Implement constructing of map from UCL + * [Feature] Implement dkim signing in dkim check plugin + * [Feature] Implement fuzzy storage updates + * [Feature] Implement fuzzy updates push protocol + * [Feature] Implement https maps + * [Feature] Implement inter-process maps caching + * [Feature] Implement mapping of remote flags to local flags + * [Feature] Implement mirroring in rspamd proxy + * [Feature] Implement multi-flags fuzzy replies + * [Feature] Implement multiple-sources fuzzy storage + * [Feature] Implement order of pre/post filters + * [Feature] Implement partial deleting for multi-flags + * [Feature] Implement pipelining for redis async interface + * [Feature] Implement ragel parser for received headers + * [Feature] Implement reading of messages to shared memory + * [Feature] Implement refcount for messages + * [Feature] Implement retransmits for master connection + * [Feature] Implement zero-copy mode for HTTP reading + * [Feature] Improve SPF domain detection logic + * [Feature] Improve config:register_symbol function + * [Feature] Improve error report for type mismatch in lua + * [Feature] Improve fstrings API + * [Feature] Improve getting SMTP data from lua_task + * [Feature] Improve levenshtein distance function + * [Feature] Improve logging in proxy and add refcounts + * [Feature] Improve logging lua types + * [Feature] Improve master/slave logging + * [Feature] Improve phishing plugin + * [Feature] Improve phishtank and openphish support + * [Feature] Improve ragel build target + * [Feature] Improve statistics script + * [Feature] Initialize ssl library to use SSL connections + * [Feature] Interpolate points sent to webui + * [Feature] Limit logging of elements that could have too many items + * [Feature] Lock ANN file when loading + * [Feature] New abstract hashing API in cryptobox + * [Feature] Normalize quoted addresses + * [Feature] Now cryptobox lua API accepts rspamd text as input + * [Feature] Optimize alignment to speed up hashing + * [Feature] Parse received date and ESMTPA proto + * [Feature] Parse received timestamp + * [Feature] Pass settings id to log helper + * [Feature] Pass settings id to lua script from log helper + * [Feature] Perform files expansion on proxying + * [Feature] Preliminary implementation of fuzzy master/slave updates + * [Feature] Print userdata using tostring if possible + * [Feature] Propagate HTTP result string when using proxy + * [Feature] Properly implement unweighted round-robin algorithm + * [Feature] Reduce number of timers queries + * [Feature] Rework and improve fuzzy storage + * [Feature] Rework dns resolving API for lua, add 'forced' option + * [Feature] Rework fann module to understand settings + * [Feature] Rework listening system to allow multiple socket types per worker + * [Feature] Rework ratelimit module to set expiration + * [Feature] Save bayes probability in memory pool var + * [Feature] Save settings id hash for convenience + * [Feature] Search for SSL_set_tlsext_host_name support + * [Feature] Send DKIM signature to protocol reply + * [Feature] Show DKIM signature in rspamc client + * [Feature] Show symbols description in scan output + * [Feature] Sign message merely after DKIM check + * [Feature] Simplify machines by assuming that headers are unfolded + * [Feature] Sort symbols when displaying them in log + * [Feature] Split main connection from mirrored connections + * [Feature] Start moving to the new email address structure + * [Feature] Store HTTP headers in a hash table + * [Feature] Store more information about compressed files + * [Feature] Store raw headers value to use them in DKIM + * [Feature] Store text parts content with newlines stripped + * [Feature] Support DKIM signing + * [Feature] Support EXIF jpeg images + * [Feature] Support archive files list extraction + * [Feature] Support archives when matching patterns in multimap + * [Feature] Support premium/academic feed for openphish + * [Feature] Support rspamd_updates via https + * [Feature] Supprort FQDNs in phishing module maps + * [Feature] Try to read on fuzzy timeout to avoid fake timeouts + * [Feature] Try to select the optimal possible function for input + * [Feature] Unescape and unquote smtp addresses + * [Feature] Update fuzzy timestamp when adding value + * [Feature] Update mumhash + * [Feature] Use -flto if possible when optimizations are enabled + * [Feature] Use extended map types in lua map, unify code + * [Feature] Use file lock in logger to avoid deadlocks + * [Feature] Use generic global string split function + * [Feature] Use metrohash as well + * [Feature] Use mumhash by default for incremental hashing + * [Feature] Use mumhash for words hashing + * [Feature] Use new ip parsing API + * [Feature] Use new maps API for local addrs + * [Feature] Use new ragel parser in message parsing code + * [Feature] Use new received parser instead of old one + * [Feature] Use new redis API in DMARC plugin + * [Feature] Use new redis API in greylist plugin + * [Feature] Use new redis API in ip_score plugin + * [Feature] Use new redis API in ratelimit plugin + * [Feature] Use new redis API in replies plugin + * [Feature] Use new version of register_symbol in rspamd plugins + * [Feature] Use offset when passing shm to deal with encrypted requests + * [Feature] Use one pass to remove newlines and store their positions + * [Feature] Use rspamd specific type checks for userdata + * [Feature] Use shared memory storage for http maps + * [Feature] Use universal redis definitions in rspamd plugins + * [Feature] Various improvements in greylist module + * [Feature] Wait for sqlite if locked when switching to WAL mode + * [Fix] Add filenames sanity filtering for mime types + * [Fix] Add guards for empty parts + * [Fix] Add missing types + * [Fix] Add more priority for config file symbols registered from UCL + * [Fix] Add sanity checks when compiling regexp maps + * [Fix] Add spaces instead of newlines to the normalized content + * [Fix] Add workaround for ancient openssl + * [Fix] Add workaround for gmime CTE stupidity + * [Fix] Add workaround for hex digits + * [Fix] Adjust MISSING_MIMEOLE score + * [Fix] Adjust body/body_buf when stealing encrypted message + * [Fix] Adopt lua task API for the new email addresses structure + * [Fix] Allow for disabling DMARC reporting when Redis is configured + * [Fix] Always register openphish and phishtank virtual symbols + * [Fix] Always use shmem on linux + * [Fix] Another change of newlines policy + * [Fix] Another d3evolution update + * [Fix] Another fix for exim workaround + * [Fix] Another fix for legacy clients + * [Fix] Another fix for maps scheduling + * [Fix] Another fix for marking upstreams inactive + * [Fix] Another fix for postfilters + * [Fix] Another fix for redis timeouts + * [Fix] Avoid `table.getn` method as it has been removed in lua 5.3 + * [Fix] Avoid double hashing for images + * [Fix] Avoid linking with actrie if hyperscan is enabled + * [Fix] Check copy result when sending message to mirrors + * [Fix] Cleanup message when assiging body + * [Fix] Cleanup stack from global vars + * [Fix] Correctly parse query type + * [Fix] Disable all symbols if enable_groups is found in settings + * [Fix] Disable fts as it is completely broken in bloody linux + * [Fix] Disable multiple autolearn checks + * [Fix] Disallow updates by default + * [Fix] Do not abort when cannot load a map + * [Fix] Do not check recursion for non-DNS SPF record types + * [Fix] Do not delete uninitialized events + * [Fix] Do not die if shmem_mkstemp fails + * [Fix] Do not die when no metrics defined + * [Fix] Do not even try pcre in case of regexp maps + * [Fix] Do not greylist messages if redis has failed somehow + * [Fix] Do not greylist on rejection + * [Fix] Do not leave temporary maps cached + * [Fix] Do not output meaningless errors + * [Fix] Do not send NaN in json + * [Fix] Don't mix hyperscan and pcre processing within a same task + * [Fix] Finally rework and simplify redis backend for statistics + * [Fix] Fix Exim shutdown patch + * [Fix] Fix JIT compilation for PCRE2 expressions + * [Fix] Fix JIT usage for PCRE2 + * [Fix] Fix REPL output + * [Fix] Fix SMTP address parsing machine + * [Fix] Fix UTF8 mode in PCRE2 + * [Fix] Fix a stupid misprint in word 'phishing' + * [Fix] Fix accepting the first update when local idx is -1 + * [Fix] Fix adding maps from ucl + * [Fix] Fix adding upstream to an active queue + * [Fix] Fix and rescore R_PARTS_DIFFER logic + * [Fix] Fix body rules in SA plugin + * [Fix] Fix body start position + * [Fix] Fix border case in urls detector + * [Fix] Fix border cases for incremental hashing + * [Fix] Fix caseless uthash application + * [Fix] Fix chartable issue with starting digits + * [Fix] Fix client_ip in users settings + * [Fix] Fix compilation issue + * [Fix] Fix conditional learning + * [Fix] Fix crash on empty maps + * [Fix] Fix creating of URLs from LUA + * [Fix] Fix creating of temporary shmem segment + * [Fix] Fix creation of mmapped statfiles + * [Fix] Fix descriptors leak on shmem detaching + * [Fix] Fix detaching of shared memory segments + * [Fix] Fix detection of URLs in text parts + * [Fix] Fix directories processing for rspamc + * [Fix] Fix displaying of rewrite subject in WebUI + * [Fix] Fix dkim private keys operations + * [Fix] Fix dkim signing + * [Fix] Fix dynamic scoring of subject symbols + * [Fix] Fix email address build + * [Fix] Fix encrypted proxy requests + * [Fix] Fix errors counting in upstreams + * [Fix] Fix errors handling in the proxy + * [Fix] Fix event bases for IO events + * [Fix] Fix events handling when scheduling map wacth + * [Fix] Fix fann rewrite + * [Fix] Fix files fallback for shmem transfer + * [Fix] Fix fuzzy adding in webui + * [Fix] Fix fuzzy storage encrypted mirroring + * [Fix] Fix fuzzy storage sync replies + * [Fix] Fix handling of the same words + * [Fix] Fix inserting values to the sources list + * [Fix] Fix ipv6 mask application + * [Fix] Fix issue with missing recipients + * [Fix] Fix issues with multiple returns from lua + * [Fix] Fix learning for non-existent backend + * [Fix] Fix legacy clients support in proxy + * [Fix] Fix length calculations for shared memory segments + * [Fix] Fix listening on UDP sockets + * [Fix] Fix loading of file maps + * [Fix] Fix long regexp flags (e.g. {sa_body}) + * [Fix] Fix lua compare function init + * [Fix] Fix maps descriptions + * [Fix] Fix maps locking + * [Fix] Fix max_train setup in ANN module + * [Fix] Fix memory corruption + * [Fix] Fix memory leak in unsigned maps reading + * [Fix] Fix misprints for lto usage + * [Fix] Fix more issues with scripts processing + * [Fix] Fix next-to-last extension length check + * [Fix] Fix openssl initialization + * [Fix] Fix order of arguments + * [Fix] Fix order of initialization + * [Fix] Fix parser + * [Fix] Fix parsing of binary tries + * [Fix] Fix parsing of braced IPv6 addresses + * [Fix] Fix parsing of nested braces in SMTP comments + * [Fix] Fix parsing of rarv5 archives + * [Fix] Fix periodic scheduling when a map is not modified + * [Fix] Fix possible FP in TRACKER_ID rule + * [Fix] Fix post-filters processing + * [Fix] Fix potential NULL dereference + * [Fix] Fix processing of
and
tags + * [Fix] Fix processing of addresses in protocol + * [Fix] Fix processing of messages without received headers + * [Fix] Fix proxying issue for unconnected sessions + * [Fix] Fix proxying of the encrypted messages + * [Fix] Fix race condition with shared memory by refcounts + * [Fix] Fix ratelimit initialization + * [Fix] Fix redis set request in replies plugin + * [Fix] Fix redis timeout events handling + * [Fix] Fix redis timeouts processing logic + * [Fix] Fix refcounts in lua_redis + * [Fix] Fix results checking if no master connection is active + * [Fix] Fix return value for couple of lua functions + * [Fix] Fix round-robin selection when upstreams have no weight + * [Fix] Fix rows calculation in graph + * [Fix] Fix rspamd_redis_make_request syntax in replies plugin + * [Fix] Fix scheduling of locked map events + * [Fix] Fix scores detection + * [Fix] Fix searching for newline positions + * [Fix] Fix secure_ip setting in controller + * [Fix] Fix sending data to graph command + * [Fix] Fix setting of score for parts differ + * [Fix] Fix setting of the lua top + * [Fix] Fix setting path for lua + * [Fix] Fix setting path for phishtank + * [Fix] Fix settings application + * [Fix] Fix shm_open call as described in POSIX + * [Fix] Fix size of length in fuzzy mirror wire protocol + * [Fix] Fix smtp grammar issues + * [Fix] Fix some issues with redis API + * [Fix] Fix some issues with retries in the proxy + * [Fix] Fix stack growing + * [Fix] Fix start of body detection in DKIM + * [Fix] Fix state on timeout + * [Fix] Fix stats script + * [Fix] Fix substring search when there are '\0' in strings + * [Fix] Fix symbol name for spf soft fail + * [Fix] Fix symbol type's check + * [Fix] Fix symbols registration and execution + * [Fix] Fix the case of multiple values keys + * [Fix] Fix the default symbol names according to metric + * [Fix] Fix timeout setup on learning + * [Fix] Fix timeouts in redis cache processing + * [Fix] Fix timeouts processing in lua_redis + * [Fix] Fix upstreams interaction for rspamd proxy + * [Fix] Fix usage of rdns reply structure + * [Fix] Fix varargs loop + * [Fix] Fix whitelists and blacklists in SA rules + * [Fix] Fix write servers setup for redis + * [Fix] Fix writing of HTTP messages + * [Fix] Force rspamd to upgrade fuzzy db on opening + * [Fix] Free the correct pointer + * [Fix] Further fixes for lto and static linking + * [Fix] Further fixes for surbl extensions map + * [Fix] Further fixes in maps code + * [Fix] Further improvements to error messages in fuzzy check + * [Fix] Further tweaks to redis garbadge collection + * [Fix] Groups are now case insensitive + * [Fix] Handle log pipe read errors + * [Fix] Handle nested dependencies in SpamAssassin plugin + * [Fix] Implement new automata to skip empty lines for dkim signing + * [Fix] Improve error messages on fuzzy add + * [Fix] Improve lua redis handling + * [Fix] Improve phishing module logging + * [Fix] Improve printing of fuzzy errors + * [Fix] Improve rrd diagnostic errors + * [Fix] Improve strcase hash used in uthash + * [Fix] Include fuzzy key to distinguish storages with different keys + * [Fix] Include slave cluster name into http request + * [Fix] Include some more information about archives + * [Fix] Indicate upstream error on timeout + * [Fix] Initialize hash tables array to avoid crashes + * [Fix] Initialize parser scripts properly + * [Fix] Initialize vars to avoid warnings + * [Fix] Inverse logic for saving ANN + * [Fix] Link lpeg to rspamd lua library + * [Fix] Make extension checks case-insensitive + * [Fix] Mark expired hashes as not found and not as zero flag + * [Fix] Match archive name as well + * [Fix] More and more fixes to redis states + * [Fix] More fixes about shared memory in proxy + * [Fix] More fixes for redis refcounts + * [Fix] More fixes to end of headers detection + * [Fix] More fixes to events logic + * [Fix] More fixes to multi-flag fuzzy storage + * [Fix] More fixes to parts distance calculations + * [Fix] More guards for redis free + * [Fix] One more fix in redis destructor + * [Fix] One more try to fix redis + * [Fix] PIE is required for static build + * [Fix] Partial fix for mmap'd statistics tests + * [Fix] Plug memory leak in proxy + * [Fix] Properly detect end of headers position + * [Fix] Properly init and free session structures + * [Fix] Reduce PRECEDENCE_BULK rule weight + * [Fix] Reduce the default thresholds for learning + * [Fix] Remove Type=forking from systemd unit file (#709) + * [Fix] Remove bad FANN file to save computational resources + * [Fix] Remove event before closing of fd to avoid race conditions + * [Fix] Remove parsing of 'from' variable in redis backend + * [Fix] Remove some bad domains from whitelists + * [Fix] Repair optional dependencies + * [Fix] Reset master connection when retransmitting scan request + * [Fix] Restore ONCE_RECEIVED symbol + * [Fix] Restore compatibility with old lua API behaviour + * [Fix] Restore redis runtime state + * [Fix] Reverse options when received + * [Fix] Send updates to mirrors only if we have some changes + * [Fix] Set host attribute properly when making HTTP request from lua + * [Fix] Set terminated state before calling of async free + * [Fix] Simplify MISSING_MIMEOLE rule + * [Fix] Simplify state machine by ignoring multiple spaces + * [Fix] Skip setting RPATH for static builds + * [Fix] Slightly reduce weights of rules with high FP rate + * [Fix] Some fixes to libmagic initialization + * [Fix] Some more fixes to ratelimit plugin + * [Fix] Strip '\r\n' properly + * [Fix] Switch hashes to mumhash + * [Fix] Treat NaN values properly in graph command + * [Fix] Try to avoid FP when checking for phished URLs + * [Fix] Try to avoid recursive events deletions + * [Fix] Try to fix false positive URL detections in text parts + * [Fix] Try to fix issue in redis stats backend when task is closed + * [Fix] Try to fix proxying of stupid spamc protocol to HTTP mirrors + * [Fix] Try to fix redis crashes + * [Fix] Try to fix upstreams with one element + * [Fix] Try to handle multiline history in a more sane way + * [Fix] Unbreak build on gcc < 4.9 + * [Fix] Update RPM spec/sources (#700) + * [Fix] Update d3evolution version + * [Fix] Update mumhash implementation + * [Fix] Use custom error function for pre and post filters + * [Fix] Use new postfilters and prefilters API in the plugins + * [Fix] Use non-blocking mode for systemd sockets + * [Fix] Use shared memory merely for local backends in the proxy + * [Fix] Use watchers for spf plugin + * [Fix] Varioud fixes to the maps code 1.2.8: - * Another fix for exim workaround (#637) - * Fix unencrypted passwords processing in the controller - * Fix setting path for lua (#652) - * Fix usage of rdns reply structure (#654) - * Use file lock in logger to avoid deadlocks - * Add `application/octet-stream` mime type for `pdf` extension (by @moisseev) - * Implement new automata to skip empty lines for dkim signing (#651) - * Fix parsing of missing classes - * Clarify some rspamc arguments (by @fatalbanana) - * Correct suppress spelling + * Another fix for exim workaround (#637) + * Fix unencrypted passwords processing in the controller + * Fix setting path for lua (#652) + * Fix usage of rdns reply structure (#654) + * Use file lock in logger to avoid deadlocks + * Add `application/octet-stream` mime type for `pdf` extension (by @moisseev) + * Implement new automata to skip empty lines for dkim signing (#651) + * Fix parsing of missing classes + * Clarify some rspamc arguments (by @fatalbanana) + * Correct suppress spelling 1.2.7: - * Slightly reduce weights of rules with high FP rate - * Add workround for rspamd-1.3 - * Fix possible FP in TRACKER_ID rule - * Simplify MISSING_MIMEOLE rule - * Add workaround for gmime CTE stupidity - * Fix mime headers processing - * Fix false positive URL detections in text parts - * Fix Exim shutdown patch - * Enable workaround for exim mailbox format - * Backport shingles static test - * Fix levenshtein distance calculations - * Fix max_train setup in ANN module - * Fix redis structure by adding {NULL, NULL} member - * Fix build with unmodified LibreSSL opensslv.h - * Repair optional dependencies - * Really skip filters in case of pre-result set - * Restore the intended pre-filters behaviour - * Fix ipv6 mask application + * Slightly reduce weights of rules with high FP rate + * Add workround for rspamd-1.3 + * Fix possible FP in TRACKER_ID rule + * Simplify MISSING_MIMEOLE rule + * Add workaround for gmime CTE stupidity + * Fix mime headers processing + * Fix false positive URL detections in text parts + * Fix Exim shutdown patch + * Enable workaround for exim mailbox format + * Backport shingles static test + * Fix levenshtein distance calculations + * Fix max_train setup in ANN module + * Fix redis structure by adding {NULL, NULL} member + * Fix build with unmodified LibreSSL opensslv.h + * Repair optional dependencies + * Really skip filters in case of pre-result set + * Restore the intended pre-filters behaviour + * Fix ipv6 mask application 1.2.6: - * Fix parsing of URLs in texts - * Fix creating of URLs from LUA - * Fix some more URL detector issues - * Fix unit tests - * Fix JIT compilation for PCRE2 expressions - * Fix JIT usage for PCRE2 - * Fix UTF8 mode in PCRE2 - * Add workaround for pre-historic compilers (#605) - * Fix and rescore R_PARTS_DIFFER logic - * Properly set lua paths for tests - * Fix SA rawbody processing - exclude top part - * Store text parts content with newlines stripped - * Properly support SA body regexps - * Fix body rules in SA plugin - * Fix setting of score for parts differ - * More fixes to parts distance calculations - - Use hashed words instead of full words for speed - - Improve levenstein distance calculations and penalise replaces - - Always return number from 0 to 1 - - Use g_malloc instead of alloca - * Fix percents output in R_PARTS_DIFFER - * Plug memory leak in dkim module - * Plug minor memory leak in regexps creation + * Fix parsing of URLs in texts + * Fix creating of URLs from LUA + * Fix some more URL detector issues + * Fix unit tests + * Fix JIT compilation for PCRE2 expressions + * Fix JIT usage for PCRE2 + * Fix UTF8 mode in PCRE2 + * Add workaround for pre-historic compilers (#605) + * Fix and rescore R_PARTS_DIFFER logic + * Properly set lua paths for tests + * Fix SA rawbody processing - exclude top part + * Store text parts content with newlines stripped + * Properly support SA body regexps + * Fix body rules in SA plugin + * Fix setting of score for parts differ + * More fixes to parts distance calculations + - Use hashed words instead of full words for speed + - Improve levenstein distance calculations and penalise replaces + - Always return number from 0 to 1 + - Use g_malloc instead of alloca + * Fix percents output in R_PARTS_DIFFER + * Plug memory leak in dkim module + * Plug minor memory leak in regexps creation 1.2.5: - * Plug an important memory leak in headers getting code - * Remove some bad domains from whitelists + * Plug an important memory leak in headers getting code + * Remove some bad domains from whitelists 1.2.4: - * Implement new multipattern matcher that uses hyperscan if possible - * Use mutlipattern for lua_trie code - * Add utility methods for multipattern - * Use multipattern in url matcher - * Add escape functions for hyperscan - * Allow to optimize lua -> C transition by flattening table args - * Optimize hot paths in SA plugin - * Optimize rspamd_re_cache_type_from_string - * Allow empty tries - * Fix extraction of URLs from Subject - * Allow to have different flags for different patterns in multipattern - * Add common directory for hyperscan cache to config - * Implement caching for hyperscan multipattern - * Attach domain part to `R_SUSPICIOUS_URL` (by @moisseev) - * Allow multipattern scans to be nested for the case of hyperscan - * Simplify SURBL redirector search code and avoid ac_trie - * Add two way substring search algorithm - * Avoid acism usage to find gtube pattern - * Fix processing of empty headers - * Allow to disable pthread mutexes on broken platforms - * Make web interface not send password in query strings (#585) by @fatalbanana - * Add maximum delay to ratelimit module - * Backport fix for empty files inclusion from libucl - * Fix settings id setup - * Add min_learns option to classifiers - * Use more clever to utf8 conversion strategy - * Fix disabling of virtual symbols in the settings - * Rework settings to work properly in metric-less configuration - * Set the default limit for classifier - * Fix ttl based expiration from LRU cache - * Rework DKIM module to use OpenSSL for digests - * Fix mailto urls parsing with hyperscan - * Do not set obscured flag for urls starting with spaces - * Fix crash on redis learn - * Fix ratelimit ctime setting + * Implement new multipattern matcher that uses hyperscan if possible + * Use mutlipattern for lua_trie code + * Add utility methods for multipattern + * Use multipattern in url matcher + * Add escape functions for hyperscan + * Allow to optimize lua -> C transition by flattening table args + * Optimize hot paths in SA plugin + * Optimize rspamd_re_cache_type_from_string + * Allow empty tries + * Fix extraction of URLs from Subject + * Allow to have different flags for different patterns in multipattern + * Add common directory for hyperscan cache to config + * Implement caching for hyperscan multipattern + * Attach domain part to `R_SUSPICIOUS_URL` (by @moisseev) + * Allow multipattern scans to be nested for the case of hyperscan + * Simplify SURBL redirector search code and avoid ac_trie + * Add two way substring search algorithm + * Avoid acism usage to find gtube pattern + * Fix processing of empty headers + * Allow to disable pthread mutexes on broken platforms + * Make web interface not send password in query strings (#585) by @fatalbanana + * Add maximum delay to ratelimit module + * Backport fix for empty files inclusion from libucl + * Fix settings id setup + * Add min_learns option to classifiers + * Use more clever to utf8 conversion strategy + * Fix disabling of virtual symbols in the settings + * Rework settings to work properly in metric-less configuration + * Set the default limit for classifier + * Fix ttl based expiration from LRU cache + * Rework DKIM module to use OpenSSL for digests + * Fix mailto urls parsing with hyperscan + * Do not set obscured flag for urls starting with spaces + * Fix crash on redis learn + * Fix ratelimit ctime setting 1.2.3: - * New DCC module (by @smfreegard) - * Rework whitelist module: - - Now we check different elements for different checks - - MIME from for DMARC - - DKIM signature domain for DKIM - - SMTP from or HELO for SPF - * Fix regexps results combination (*critical*) - * Fix issue with expressions processing (*critical*) - * Optimize strlcpy for aligned input - * Add support of half-closed connection in lua_tcp - * Allow to print compact json in client - * Save required score in history (#581) - * Allow to attach file descriptors to control commands - * Allow to send descriptors from workers to main - * Allow to attach fd when broadcasting to workers - * Implement log pipe feature for rspamd logs analysis - * Add `log_helper` worker - * Add `URIBL_SBL_CSS` (by @smfreegard) - * Add worker scripts functionality - * Add on load hooks for rspamd_config - * Add lua scripts for log_helper worker - * Add generic maillist detector (#584) - * Implement FANN autolearn using log_helper worker - * Rework metrics configuration to allow includes - * Change default value of forced removal in composite rules - * Allow to use assembly version of blake2b on x86 cpu - * Use less precise (but faster) clock if possible - * Insert redirected URL to the urls list - * Allow to get and set callback data for rspamd symbols - * Add binary heap implementation - * Use binary heap for expire algorithms in the hash - * Use `least frequent used` expiration strategy - * Allow to get mime headers from a task - * Add support for mime headers in `regexp` module - * Update Exim patches (by @fatalbanana) - * Allow building rspamd with jemalloc - * Save multipart boundaries - * SA plugin changes: - - Properly handle MIME headers - - Fix eval:check_for_missing_to_header rule - - Implement SA compatible body regexps - - Use sabody rules in SA plugin - * LUA API changes: - - Add util.get_ticks function - - Add util.stat function - - Add task:get_symbols_numeric method - - Add method to get number of symbols in the cache - - Add lua methods to get redirected urls - - Allow to get callbacks for lua symbols - - Add config:set_symbol_callback function + * New DCC module (by @smfreegard) + * Rework whitelist module: + - Now we check different elements for different checks + - MIME from for DMARC + - DKIM signature domain for DKIM + - SMTP from or HELO for SPF + * Fix regexps results combination (*critical*) + * Fix issue with expressions processing (*critical*) + * Optimize strlcpy for aligned input + * Add support of half-closed connection in lua_tcp + * Allow to print compact json in client + * Save required score in history (#581) + * Allow to attach file descriptors to control commands + * Allow to send descriptors from workers to main + * Allow to attach fd when broadcasting to workers + * Implement log pipe feature for rspamd logs analysis + * Add `log_helper` worker + * Add `URIBL_SBL_CSS` (by @smfreegard) + * Add worker scripts functionality + * Add on load hooks for rspamd_config + * Add lua scripts for log_helper worker + * Add generic maillist detector (#584) + * Implement FANN autolearn using log_helper worker + * Rework metrics configuration to allow includes + * Change default value of forced removal in composite rules + * Allow to use assembly version of blake2b on x86 cpu + * Use less precise (but faster) clock if possible + * Insert redirected URL to the urls list + * Allow to get and set callback data for rspamd symbols + * Add binary heap implementation + * Use binary heap for expire algorithms in the hash + * Use `least frequent used` expiration strategy + * Allow to get mime headers from a task + * Add support for mime headers in `regexp` module + * Update Exim patches (by @fatalbanana) + * Allow building rspamd with jemalloc + * Save multipart boundaries + * SA plugin changes: + - Properly handle MIME headers + - Fix eval:check_for_missing_to_header rule + - Implement SA compatible body regexps + - Use sabody rules in SA plugin + * LUA API changes: + - Add util.get_ticks function + - Add util.stat function + - Add task:get_symbols_numeric method + - Add method to get number of symbols in the cache + - Add lua methods to get redirected urls + - Allow to get callbacks for lua symbols + - Add config:set_symbol_callback function 1.2.2: - * Use HTTP Content-Type on non mime input if possible - * Save log level when compressing log messages - * Further rework of composite rules (add '^' prefix) - * Add tracking for rspamd expressions - * Store actions limits in metric result - * Fix parsing of include/redirect with many records in SPF - * Add method to disable symbols execution in the cache - * Allow to disable checks from settings - * Allow to select settings by id in HTTP query - * Find URLs with '\r' and '\n' inside href attribute - * Implement vectored mode for hyperscan (experimental) - * Improve client connection errors diagnostics - * Allow to edit new files with signtool - * Improve hashes performance on 32 bit platforms - * Fix sorting of limits - * Remove slow and unused rules `INVALID_EXIM_RECEIVED*` - * Add expression:process_traced lua method - * Allow tables in task:insert_result - * Save trace for SA metas - * Do not parse broken TLD parts in URLs - * Investigate many border cases in URLs parser + * Use HTTP Content-Type on non mime input if possible + * Save log level when compressing log messages + * Further rework of composite rules (add '^' prefix) + * Add tracking for rspamd expressions + * Store actions limits in metric result + * Fix parsing of include/redirect with many records in SPF + * Add method to disable symbols execution in the cache + * Allow to disable checks from settings + * Allow to select settings by id in HTTP query + * Find URLs with '\r' and '\n' inside href attribute + * Implement vectored mode for hyperscan (experimental) + * Improve client connection errors diagnostics + * Allow to edit new files with signtool + * Improve hashes performance on 32 bit platforms + * Fix sorting of limits + * Remove slow and unused rules `INVALID_EXIM_RECEIVED*` + * Add expression:process_traced lua method + * Allow tables in task:insert_result + * Save trace for SA metas + * Do not parse broken TLD parts in URLs + * Investigate many border cases in URLs parser 1.2.1: - * Add list support to `mime types` module configuration (by @moisseev) - * Allow symbols params to be printed in logs - * Fix `MIME_BAD_ATTACHMENT` false positives for MDN/DSN - * Fix crashes on arm32 - * Do not classify message if some class is missing - * Fix cryptobox cleanup - * Remove multipart/report from bad mime types (#569) - * Improve logging for fuzzy hashes - * Show map URLs in webui - * Sort symbols in webui + * Add list support to `mime types` module configuration (by @moisseev) + * Allow symbols params to be printed in logs + * Fix `MIME_BAD_ATTACHMENT` false positives for MDN/DSN + * Fix crashes on arm32 + * Do not classify message if some class is missing + * Fix cryptobox cleanup + * Remove multipart/report from bad mime types (#569) + * Improve logging for fuzzy hashes + * Show map URLs in webui + * Sort symbols in webui 1.2.0: - * New dynamic updates plugin - * Regular expressions map support - * Faster radix trie algorithm - * Faster siphash for AVX2 supporing CPUs (used in fuzzy hashes) - * PCRE2 support - * Allow quoted and slashed keys in map - * Add proper support of DNS resolvers balancing (#552) - * Rework includes and configuration system for better local changes support - * New keypairs framework for signing and encryption - * Added support for dynamic modules and workers - * Allow to dump configuration with help comments - * Rework once_received module - - Fix priority for `good_hosts` - - If a good host has been found do not add once_received symbols - - Fix priorities for strict once_received - - Add ability to whitelist IP addresses - * Implement support of signed maps for HTTP and file maps - * Add command to sync fuzzy storage (#533) - * Rework system of symbols and actions registration - It is possible now to use priorities when adding symbols to metrics and - override scores for symbols with lower priority with the scores with - high priority. - * Add auth support and db selection for redis stats - * Improve composite rules application - * Add ignore_received option - * Fix critical issue with inconsistent resorting - * Fix `all` in spf redirects - * Add punycoded versions for IDN domains (#554) - * Improve sorting order for symbols cache - * Add lockless logging for processes management - * Allow to specify flags for metric symbols - * Load images height and width from style attribute (#538) - * Override DNS requests limits for SPF and DKIM - * Fix resetting symbols to their default values in WebUI - * Improve configuration agility for redis stats - * Allow to set db and password for redis in stat_convert - * Import the latest libucl - * LUA API changes: - - Add rspamd_version function to LUA API - - Add lua_cryptobox module - - Add lua_map module - - Add task:set_metric_action lua API method - - Fix race condition in lua_tcp module - - Fix a lot of issues in lua_redis module - - Rework and abstract lua maps API - - Add util.strlen_utf8 lua function - - Add lua functions for caseless comparison - - Allow optional symbols registration - - Add config:add_map table form method, add regexp maps - - Add task:has_urls method - - Add task:has_flag method - - Add html tags methods to lua_html - - Add task:get_dns_req - * Plugins changes: - - Add support for WLBLEval SA plugin - - Use caseless comparison in SA and DMARC plugins - - Allow SA plugin to set scores for rspamd symbols - - Add regexp maps support to multimap - - Allow filenames match in multimap - - Add more filters for the existing map types - - Fix html images rules to reduce FP rates - * New rules: - - LONG_SUBJ - too long subject - - MIME_BAD_ATTACHMENT - bad attachment type - - RDNS_NONE - no reverse DNS record for sender's IP - - Fix MISSING_MIMEOLE rule for modern OE - * Many other bugfixes, memory leaks plugs thanks to: - - Coverity scan - - New gcc-6 warnings - - valgrind manual iterations - * Documentation improvements: - - FAQ list: https://rspamd.com/doc/faq.html - - Reworked quick start guide - - Added documentation for all active modules - * Other changes: - - Dropped Ubuntu Vivid support - - Added Ubuntu Xenial support - - Rework build system for rspamd and rmilter + * New dynamic updates plugin + * Regular expressions map support + * Faster radix trie algorithm + * Faster siphash for AVX2 supporing CPUs (used in fuzzy hashes) + * PCRE2 support + * Allow quoted and slashed keys in map + * Add proper support of DNS resolvers balancing (#552) + * Rework includes and configuration system for better local changes support + * New keypairs framework for signing and encryption + * Added support for dynamic modules and workers + * Allow to dump configuration with help comments + * Rework once_received module + - Fix priority for `good_hosts` + - If a good host has been found do not add once_received symbols + - Fix priorities for strict once_received + - Add ability to whitelist IP addresses + * Implement support of signed maps for HTTP and file maps + * Add command to sync fuzzy storage (#533) + * Rework system of symbols and actions registration + It is possible now to use priorities when adding symbols to metrics and + override scores for symbols with lower priority with the scores with + high priority. + * Add auth support and db selection for redis stats + * Improve composite rules application + * Add ignore_received option + * Fix critical issue with inconsistent resorting + * Fix `all` in spf redirects + * Add punycoded versions for IDN domains (#554) + * Improve sorting order for symbols cache + * Add lockless logging for processes management + * Allow to specify flags for metric symbols + * Load images height and width from style attribute (#538) + * Override DNS requests limits for SPF and DKIM + * Fix resetting symbols to their default values in WebUI + * Improve configuration agility for redis stats + * Allow to set db and password for redis in stat_convert + * Import the latest libucl + * LUA API changes: + - Add rspamd_version function to LUA API + - Add lua_cryptobox module + - Add lua_map module + - Add task:set_metric_action lua API method + - Fix race condition in lua_tcp module + - Fix a lot of issues in lua_redis module + - Rework and abstract lua maps API + - Add util.strlen_utf8 lua function + - Add lua functions for caseless comparison + - Allow optional symbols registration + - Add config:add_map table form method, add regexp maps + - Add task:has_urls method + - Add task:has_flag method + - Add html tags methods to lua_html + - Add task:get_dns_req + * Plugins changes: + - Add support for WLBLEval SA plugin + - Use caseless comparison in SA and DMARC plugins + - Allow SA plugin to set scores for rspamd symbols + - Add regexp maps support to multimap + - Allow filenames match in multimap + - Add more filters for the existing map types + - Fix html images rules to reduce FP rates + * New rules: + - LONG_SUBJ - too long subject + - MIME_BAD_ATTACHMENT - bad attachment type + - RDNS_NONE - no reverse DNS record for sender's IP + - Fix MISSING_MIMEOLE rule for modern OE + * Many other bugfixes, memory leaks plugs thanks to: + - Coverity scan + - New gcc-6 warnings + - valgrind manual iterations + * Documentation improvements: + - FAQ list: https://rspamd.com/doc/faq.html + - Reworked quick start guide + - Added documentation for all active modules + * Other changes: + - Dropped Ubuntu Vivid support + - Added Ubuntu Xenial support + - Rework build system for rspamd and rmilter 1.1.4: - * Print traceback on lua errors in lua config - * Fix leaks in lua error paths - * Improve 'R_EMPTY_IMAGE' rule - * Fix metas memoization in SA plugin - * Properly set `flag` in fuzzy replies - * Fix arguments order - * Fix issue with out-of-boundary reading - * Fix issues found by coverity - * Same result checking error found by coverity - * Fix varargs processing (found by coverity) - * Fix error in printing hex - * Reduce weights for some hfilter patterns - * Add aliases for task:get_from_ip: - - task:get_addr - - task:get_from_addr - - task:get_ip - * Rework once_received module - - Fix priority for `good_hosts` - - If a good host has been found do not add once_received symbols - - Fix priorities for strict once_received - - Add ability to whitelist IP addresses - * Fix `MISSING_MIMEOLE` rule for modern OE - * Treat meta tags as embedded tags (#501) + * Print traceback on lua errors in lua config + * Fix leaks in lua error paths + * Improve 'R_EMPTY_IMAGE' rule + * Fix metas memoization in SA plugin + * Properly set `flag` in fuzzy replies + * Fix arguments order + * Fix issue with out-of-boundary reading + * Fix issues found by coverity + * Same result checking error found by coverity + * Fix varargs processing (found by coverity) + * Fix error in printing hex + * Reduce weights for some hfilter patterns + * Add aliases for task:get_from_ip: + - task:get_addr + - task:get_from_addr + - task:get_ip + * Rework once_received module + - Fix priority for `good_hosts` + - If a good host has been found do not add once_received symbols + - Fix priorities for strict once_received + - Add ability to whitelist IP addresses + * Fix `MISSING_MIMEOLE` rule for modern OE + * Treat meta tags as embedded tags (#501) 1.1.3: - * Fix DSN rules when SMTP from is unavailable - * Fix statconvert routine to avoid lua module usage - * Set a sane quark for configtest to avoid NULL to be printed in logs - * Support c11 if available - * Fix parsing of ip:port strings - * Add more diagnostic for lua subr errors - * Fix task:set_from_ip lua method - * Add basic routines for digital signatures - * Add tool for digital signatures - * Add plain open file API method for atomic open - * Fix parsing nested braces inside logger vars - * Pre filters now actually skip processing - * Add pre-filter mode for multimap - * Switch to apache 2 license + * Fix DSN rules when SMTP from is unavailable + * Fix statconvert routine to avoid lua module usage + * Set a sane quark for configtest to avoid NULL to be printed in logs + * Support c11 if available + * Fix parsing of ip:port strings + * Add more diagnostic for lua subr errors + * Fix task:set_from_ip lua method + * Add basic routines for digital signatures + * Add tool for digital signatures + * Add plain open file API method for atomic open + * Fix parsing nested braces inside logger vars + * Pre filters now actually skip processing + * Add pre-filter mode for multimap + * Switch to apache 2 license 1.1.2: - * Fix stat_cache closing - * Add checkpoints to sqlite3 learn cache - * Do not recompile lua generated headers all the time - * Increase number of messages learned - * Fix issues with dual stack and hfilter - * Disable MID checks for hfilter by default - * Fix cache definitions in multiple classifier and no type - * Don't crash if learn cache failed to initialize - * Fix googlegroups support in maillist plugin - * Rework flags LUA API: - - Allow to check for a specific flag - - Add `learn_spam`, `learn_ham` and `broken_headers` flags - - Unify internal functions - * Add `BROKEN_HEADERS` rule - * Add support for forged confirmation headers (by @AdUser) - * Allow `any`, `mime` and `smtp` for get_from/get_recipients - * Add mime types checking plugin - * Add rule to detect spammers attempts to cheat mime parsing - * Rework parsing of IP addresses in configuration (better IPv6 support) - * Add `util.parse_mail_address` function to LUA API - * Add lua sqlite3 module - * Implement synchronous redis call - * Ratelimit: avoid possible indexing of nil value (Fixes #498) (by @fatalbanana) - * Add stat_convert command to convert stats tokens from sqlite3 to redis - * Implement redis advanced lua api with pipelining - * Fix memory leak on redis stat (#500) - * Fix user/language learn count in sqlite statistics (#496) (by @fatalbanana) - * Fix build with custom pcre - * Fix fuzzy relearning (#498) - * Improve planning of asynchronous tasks - * Show slow rules in log - * Add warning for slow regexps - * Add base32 decode/encode routines to lua util - * Allow converting of learn cache from sqlite to redis - * Add methods to check if a messages has from/rcpts - * Improve and fix multimap plugin: - - Restore 'header' maps - - Add filters for headers - - Add 'email:addr', 'email:user', 'email:domain' and 'email:name' filters - - Add generic regexp filters - * Disable reload command in rc scripts - * Improve runtime CPU dispatcher for libcryptobox - * Add preliminary support of digital signatures via ed25519 - * Add detection for RDRAND support - * Print configuration of crypto on start - * A in SPF presumes AAAA lookup as well + * Fix stat_cache closing + * Add checkpoints to sqlite3 learn cache + * Do not recompile lua generated headers all the time + * Increase number of messages learned + * Fix issues with dual stack and hfilter + * Disable MID checks for hfilter by default + * Fix cache definitions in multiple classifier and no type + * Don't crash if learn cache failed to initialize + * Fix googlegroups support in maillist plugin + * Rework flags LUA API: + - Allow to check for a specific flag + - Add `learn_spam`, `learn_ham` and `broken_headers` flags + - Unify internal functions + * Add `BROKEN_HEADERS` rule + * Add support for forged confirmation headers (by @AdUser) + * Allow `any`, `mime` and `smtp` for get_from/get_recipients + * Add mime types checking plugin + * Add rule to detect spammers attempts to cheat mime parsing + * Rework parsing of IP addresses in configuration (better IPv6 support) + * Add `util.parse_mail_address` function to LUA API + * Add lua sqlite3 module + * Implement synchronous redis call + * Ratelimit: avoid possible indexing of nil value (Fixes #498) (by @fatalbanana) + * Add stat_convert command to convert stats tokens from sqlite3 to redis + * Implement redis advanced lua api with pipelining + * Fix memory leak on redis stat (#500) + * Fix user/language learn count in sqlite statistics (#496) (by @fatalbanana) + * Fix build with custom pcre + * Fix fuzzy relearning (#498) + * Improve planning of asynchronous tasks + * Show slow rules in log + * Add warning for slow regexps + * Add base32 decode/encode routines to lua util + * Allow converting of learn cache from sqlite to redis + * Add methods to check if a messages has from/rcpts + * Improve and fix multimap plugin: + - Restore 'header' maps + - Add filters for headers + - Add 'email:addr', 'email:user', 'email:domain' and 'email:name' filters + - Add generic regexp filters + * Disable reload command in rc scripts + * Improve runtime CPU dispatcher for libcryptobox + * Add preliminary support of digital signatures via ed25519 + * Add detection for RDRAND support + * Print configuration of crypto on start + * A in SPF presumes AAAA lookup as well 1.1.1: - * Fix duplicated XBL symbol - * Reduce log severity for ratelimit missing servers - * Fix XBL composite to avoid duplicate symbols - * Reduce weight of URL_ONLY rule due to FP rate - * Disable fuzzy hashes from the metadata for now - * Fix processing of empty messages (#486) - * Always treat DNS timeouts as temporary fail for SPF - * Fix issue with SPF double IP stack (#483) - * Use X-Forwarded-For when checking secure_ip (#488) - * Fix hash calculation for sqlite stats - * Fix memory corruption on punycode - * Fix strings allocation in punycode - * Fix error message (#491) + * Fix duplicated XBL symbol + * Reduce log severity for ratelimit missing servers + * Fix XBL composite to avoid duplicate symbols + * Reduce weight of URL_ONLY rule due to FP rate + * Disable fuzzy hashes from the metadata for now + * Fix processing of empty messages (#486) + * Always treat DNS timeouts as temporary fail for SPF + * Fix issue with SPF double IP stack (#483) + * Use X-Forwarded-For when checking secure_ip (#488) + * Fix hash calculation for sqlite stats + * Fix memory corruption on punycode + * Fix strings allocation in punycode + * Fix error message (#491) 1.1.0: - * Incompatible change: sqlite3 and per_user behaviour: - Now both redis and sqlite3 follows the common principles for per-user - statistics: - 1) If per-user statistics is enabled check per-user tokens ONLY - 2) If per-user statistics is not enabled then check common tokens ONLY - If you need old behaviour, then you'd need to use separate classifier - for per-user statistics. - * Implement redis statistics backend and cache - * Implement autolearning for statistics - * Reworked statistics architecture from scratch - * Add hyperscan (https://github.com/01org/hyperscan) engine for regular - expressions: - - add lazy loader for hyperscan databases - - rework regexp cache to have joint pcre/hyperscan scanning - - implement hyperscan pre-filter support - - add compilation guards for bad expressions - - implement `rspamadm control recompile` command - - implement hyperscan cache monitoring - - slides: - * Implement flexible task logging - * Rework fuzzy worker: - - it is now possible to run multiple fuzzy workers; - - implement lazy writing as sqlite3 is bad at concurrent writing; - - add retries for simple sql commands in fuzzy backend; - - use fine-grained transactions for fuzzy; - - implement new multi-pubkeys mode; - - allow encrypted only storages; - - rework statistics for fuzzy; - - add `rspamadm control fuzzystat` command for extended statistics; - - implement human readable output for the previous command; - - add condition script for learning fuzzy storage; - * Various fixes to SPF: - - fix `redirect` records; - - fix domains when parsing mx/ptr/a records in includes/redirects; - - fix issues with multiple addresses in SPF records; - - ignore SPF results in case of DNS failure; - - adjust TTL of records when resolving subelements of SPF records; - - always select `v=spf1` line if it is available - - do not cache records with DNS failure in subrequests; - - ignore records with temporary fails during subrequests resolving; - - fix `RDNS_RC_NOREC` support; - * Add clang plugin for static analysis: - - implement static checks for `rspamd_printf` format strings; - * Add 'allow_raw_input' option for non-mime messages - * Recognize types using libmagic - * Fix parsing of IPv6 received headers - * Add new interface of communication between workers in rspamd - * Add support for named socketpairs - * Don't write URLs by default as it is too verbose - * Set status for HTTP replies - * Try load `rspamd.conf.override` - * Implement words decaying for text parts to limit many checks - * Improve support of SA rules and plugins: - - add check_for_shifted_date and check_for_missing_to_header eval rules; - - add 'check_relays_unparseable' support; - - add `check_for_mime('mime_attachement')` function; - - use new re_cache interface for all SA rules; - - add support for `Mail::SpamAssassin::Plugin::MIMEHeader`; - - add support of 'special' SA headers to `exists` function; - - fix issue when SA metas contain other metas; - - fix freemail rules; - * Many fixes to the URL parser - * Match any newline character in regexps - * Fix resolving of upstreams and detection of poor IPv6 configurations - * Parse upstreams selection algorithm from the configuration line - * Add `reresolve` command to the control interface - * Generate fuzzy hashes from task metadata (URLs and headers) - * Add method to check if IP is local and `local_addrs` option - * Implement forced timeout for delayed filters - * Disable fast path of pcre-jit as it seems to be broken - * Bayes fixes: - - new normalizer function; - - really use weights of tokens from the OSB algorithm; - - restore multiple classifiers support; - * Rules changes: - - add `R_SUSPICIOUS_URL` rule that detects obfuscated URL's; - - improve empty image rule; - - rework `FORGED_RECIPIENTS` rule; - - reduce weight of `SUSPICIOUS_RECIPS`; - - fix `*_NORESOLVE_MX` symbols in hfilter; - - add `SUBJ_ALL_CAPS` rule with support of UTF8 - - add spamhaus SBL to uribl - - fix `SUSPICIOUS_RECIPS` and `SORTED_RECIPS` rules - - remove `R_TO_SEEMS_AUTO` as it generates a lot of FP; - - add new Message-ID regexp for Thunderbird (by @moisseev); - * Plugins changes: - - allow ratelimit plugin to set symbol instead of pre-result - - support IP DNS black lists for URIBL (e.g spamhaus SBL); - - drop deprecated SURBL bits (by @fatalbanana) - - rename `JP_SURBL_MULTI` to `ABUSE_SURBL` (by @fatalbanana) - - add `SURBL_BLOCKED` (by @fatalbanana) - - add `CR_SURBL` - - SURBL: allow fallthrough to default symbol (by @fatalbanana) - - Settings: fix IP match (by @fatalbanana) - - SURBL: add missing symbols to metric (by @fatalbanana) - - allow processing images urls for SURBL - - unconditionally disable SPF for authenticated users and local networks - * Rework ratelimit plugin - - switch to `rates` instead of old and stupid strings to setup; - - check if a bucket is zero and disable the corresponding limits' - - turn off all buckets by default; - - check either `rcpt` or `user` buckets, not all together' - - document new `rates` and `symbol` options; - - inform user about what buckets are used in the configuration; - * Add neural network **experimental** plugin - * Add a sample script to learn neural network from rspamd logs - * Add documentation strings support to rspamd: - - add strings for the main configuration options; - - document workers options; - - add internal plugin options; - - create `rspamadm confighelp` routine; - - implement human readable output for the previous command; - - add subtree search support; - - add keyword search support; - * Documentation improvements, tutorials section, statistics description - * Many other minor and major bugfixes not noted here + * Incompatible change: sqlite3 and per_user behaviour: + Now both redis and sqlite3 follows the common principles for per-user + statistics: + 1) If per-user statistics is enabled check per-user tokens ONLY + 2) If per-user statistics is not enabled then check common tokens ONLY + If you need old behaviour, then you'd need to use separate classifier + for per-user statistics. + * Implement redis statistics backend and cache + * Implement autolearning for statistics + * Reworked statistics architecture from scratch + * Add hyperscan (https://github.com/01org/hyperscan) engine for regular + expressions: + - add lazy loader for hyperscan databases + - rework regexp cache to have joint pcre/hyperscan scanning + - implement hyperscan pre-filter support + - add compilation guards for bad expressions + - implement `rspamadm control recompile` command + - implement hyperscan cache monitoring + - slides: + * Implement flexible task logging + * Rework fuzzy worker: + - it is now possible to run multiple fuzzy workers; + - implement lazy writing as sqlite3 is bad at concurrent writing; + - add retries for simple sql commands in fuzzy backend; + - use fine-grained transactions for fuzzy; + - implement new multi-pubkeys mode; + - allow encrypted only storages; + - rework statistics for fuzzy; + - add `rspamadm control fuzzystat` command for extended statistics; + - implement human readable output for the previous command; + - add condition script for learning fuzzy storage; + * Various fixes to SPF: + - fix `redirect` records; + - fix domains when parsing mx/ptr/a records in includes/redirects; + - fix issues with multiple addresses in SPF records; + - ignore SPF results in case of DNS failure; + - adjust TTL of records when resolving subelements of SPF records; + - always select `v=spf1` line if it is available + - do not cache records with DNS failure in subrequests; + - ignore records with temporary fails during subrequests resolving; + - fix `RDNS_RC_NOREC` support; + * Add clang plugin for static analysis: + - implement static checks for `rspamd_printf` format strings; + * Add 'allow_raw_input' option for non-mime messages + * Recognize types using libmagic + * Fix parsing of IPv6 received headers + * Add new interface of communication between workers in rspamd + * Add support for named socketpairs + * Don't write URLs by default as it is too verbose + * Set status for HTTP replies + * Try load `rspamd.conf.override` + * Implement words decaying for text parts to limit many checks + * Improve support of SA rules and plugins: + - add check_for_shifted_date and check_for_missing_to_header eval rules; + - add 'check_relays_unparseable' support; + - add `check_for_mime('mime_attachement')` function; + - use new re_cache interface for all SA rules; + - add support for `Mail::SpamAssassin::Plugin::MIMEHeader`; + - add support of 'special' SA headers to `exists` function; + - fix issue when SA metas contain other metas; + - fix freemail rules; + * Many fixes to the URL parser + * Match any newline character in regexps + * Fix resolving of upstreams and detection of poor IPv6 configurations + * Parse upstreams selection algorithm from the configuration line + * Add `reresolve` command to the control interface + * Generate fuzzy hashes from task metadata (URLs and headers) + * Add method to check if IP is local and `local_addrs` option + * Implement forced timeout for delayed filters + * Disable fast path of pcre-jit as it seems to be broken + * Bayes fixes: + - new normalizer function; + - really use weights of tokens from the OSB algorithm; + - restore multiple classifiers support; + * Rules changes: + - add `R_SUSPICIOUS_URL` rule that detects obfuscated URL's; + - improve empty image rule; + - rework `FORGED_RECIPIENTS` rule; + - reduce weight of `SUSPICIOUS_RECIPS`; + - fix `*_NORESOLVE_MX` symbols in hfilter; + - add `SUBJ_ALL_CAPS` rule with support of UTF8 + - add spamhaus SBL to uribl + - fix `SUSPICIOUS_RECIPS` and `SORTED_RECIPS` rules + - remove `R_TO_SEEMS_AUTO` as it generates a lot of FP; + - add new Message-ID regexp for Thunderbird (by @moisseev); + * Plugins changes: + - allow ratelimit plugin to set symbol instead of pre-result + - support IP DNS black lists for URIBL (e.g spamhaus SBL); + - drop deprecated SURBL bits (by @fatalbanana) + - rename `JP_SURBL_MULTI` to `ABUSE_SURBL` (by @fatalbanana) + - add `SURBL_BLOCKED` (by @fatalbanana) + - add `CR_SURBL` + - SURBL: allow fallthrough to default symbol (by @fatalbanana) + - Settings: fix IP match (by @fatalbanana) + - SURBL: add missing symbols to metric (by @fatalbanana) + - allow processing images urls for SURBL + - unconditionally disable SPF for authenticated users and local networks + * Rework ratelimit plugin + - switch to `rates` instead of old and stupid strings to setup; + - check if a bucket is zero and disable the corresponding limits' + - turn off all buckets by default; + - check either `rcpt` or `user` buckets, not all together' + - document new `rates` and `symbol` options; + - inform user about what buckets are used in the configuration; + * Add neural network **experimental** plugin + * Add a sample script to learn neural network from rspamd logs + * Add documentation strings support to rspamd: + - add strings for the main configuration options; + - document workers options; + - add internal plugin options; + - create `rspamadm confighelp` routine; + - implement human readable output for the previous command; + - add subtree search support; + - add keyword search support; + * Documentation improvements, tutorials section, statistics description + * Many other minor and major bugfixes not noted here 1.0.11: - * Fix spf redirects - * Fix domains when parsing mx/ptr/a records in includes/redirects - * Fix unfolded base64 encoding - * Fix GError use-after-free - * Do not rewrite the original url when using redirector - * Fix parsing of fragment in urls - * Fix processing of HTML tags - * Improve empty image rule - * Avoid long double type - * Fix tokens weights in OSB algorithm - * Improve debugging for bayes + * Fix spf redirects + * Fix domains when parsing mx/ptr/a records in includes/redirects + * Fix unfolded base64 encoding + * Fix GError use-after-free + * Do not rewrite the original url when using redirector + * Fix parsing of fragment in urls + * Fix processing of HTML tags + * Improve empty image rule + * Avoid long double type + * Fix tokens weights in OSB algorithm + * Improve debugging for bayes 1.0.10: - * Fix settings application (#416) - * Fix another issue with fixed strings - * Fix hash function invocation - * Use the proper string for make_dns_request in lua_http - * Fix scan time output - * Update webui: - - fix labels for greylisting - - fix dimension of scan time + * Fix settings application (#416) + * Fix another issue with fixed strings + * Fix hash function invocation + * Use the proper string for make_dns_request in lua_http + * Fix scan time output + * Update webui: + - fix labels for greylisting + - fix dimension of scan time 1.0.9: - * Emergency fix in keyed blake2 to fix fuzzy hashes and encrypted password - * Support passwords longer than 64 symbols + * Emergency fix in keyed blake2 to fix fuzzy hashes and encrypted password + * Support passwords longer than 64 symbols 1.0.8: - * Add function to traverse AST atoms - * Allow dependencies on rspamd symbols for SA metas - * Fix memory corruption when timeout is removed in fuzzy check - * Fix encrypted fuzzy add processing - * Avoid use-after-free in controller session destructor - * Use session pool instead of task pool in fuzzy check - * Fix assembly in i386 mode (#413, #412) + * Add function to traverse AST atoms + * Allow dependencies on rspamd symbols for SA metas + * Fix memory corruption when timeout is removed in fuzzy check + * Fix encrypted fuzzy add processing + * Avoid use-after-free in controller session destructor + * Use session pool instead of task pool in fuzzy check + * Fix assembly in i386 mode (#413, #412) 1.0.7: - * Plugged memory leaks in internet address object & html parser - * Fixed static build - * Fixed multiple sigchld processing - * Fixed deletion of signal events after event processing loop - * Fixed build on ARM (#404 - reported by @Gottox) - * Fixed setting the default mask for SPF. - * Fixed sanitisation of HTTP query values - * Fixed parsing of the last header in encrypted HTTP messages - * Additions and fixes for test suite & benchmarks - * Added openssl aes-256-gcm support to libcryptobox & HTTP server - * Implemented support for starting multiple HTTP servers - * Implemented batch accept in HTTP server - * Added module to get data from HTTP headers (#285 - reported by @msimerson) - * Added `rspamadm control` command - * Added ability to sort counters output. - * Added ability to specify custom headers for rspamc client - * Fix architecture detection - * Converted history storage to the UCL format - * Allow flexible number of rows in history - * Fix action badges in WebUI - * Add universal cryptobox hash API - * Migrated to the optimized blake2b implementation adopted from Andrew Moon - * Allow explicit loading of specific modules - * Always load settings module - * Allow to add symbols from settings - * Fix double free in the controller fuzzy learn command - * Avoid endless loop when cannot open sqlite db - * Updated libucl + * Plugged memory leaks in internet address object & html parser + * Fixed static build + * Fixed multiple sigchld processing + * Fixed deletion of signal events after event processing loop + * Fixed build on ARM (#404 - reported by @Gottox) + * Fixed setting the default mask for SPF. + * Fixed sanitisation of HTTP query values + * Fixed parsing of the last header in encrypted HTTP messages + * Additions and fixes for test suite & benchmarks + * Added openssl aes-256-gcm support to libcryptobox & HTTP server + * Implemented support for starting multiple HTTP servers + * Implemented batch accept in HTTP server + * Added module to get data from HTTP headers (#285 - reported by @msimerson) + * Added `rspamadm control` command + * Added ability to sort counters output. + * Added ability to specify custom headers for rspamc client + * Fix architecture detection + * Converted history storage to the UCL format + * Allow flexible number of rows in history + * Fix action badges in WebUI + * Add universal cryptobox hash API + * Migrated to the optimized blake2b implementation adopted from Andrew Moon + * Allow explicit loading of specific modules + * Always load settings module + * Allow to add symbols from settings + * Fix double free in the controller fuzzy learn command + * Avoid endless loop when cannot open sqlite db + * Updated libucl 1.0.6: - * Fix build on i386 - * Update CentOS7 service file patch (by @fatalbanana) - * Fix path to rspamadm in Debian init script (by @fatalbanana) - * Fix broken '_SC_GETPW_R_SIZE_MAX' on FreeBSD - * Fix portability issues - * Use cryptobox chacha for libottery - * Better support of 32 bit builds - * Fix header name tokens setup - * Fix levenstein distance method for words - * Add workaround for old libevent (#400) - * Fix microseconds in termination timer - * Fix some more issues with fixed strings - * Explicitly test CPU instructions even after CPUID call - * Do not check out of boundary memory - * Do not output broken emails - * Fix unknown symbols registration - * Handle SIGILL using longjmp - * Block signals when exiting event loop - * Fix incorrect allocation size - * Slightly optimize alignment - * Restore rspamd -t for compatibility - * Add more sanity checks for emails + * Fix build on i386 + * Update CentOS7 service file patch (by @fatalbanana) + * Fix path to rspamadm in Debian init script (by @fatalbanana) + * Fix broken '_SC_GETPW_R_SIZE_MAX' on FreeBSD + * Fix portability issues + * Use cryptobox chacha for libottery + * Better support of 32 bit builds + * Fix header name tokens setup + * Fix levenstein distance method for words + * Add workaround for old libevent (#400) + * Fix microseconds in termination timer + * Fix some more issues with fixed strings + * Explicitly test CPU instructions even after CPUID call + * Do not check out of boundary memory + * Do not output broken emails + * Fix unknown symbols registration + * Handle SIGILL using longjmp + * Block signals when exiting event loop + * Fix incorrect allocation size + * Slightly optimize alignment + * Restore rspamd -t for compatibility + * Add more sanity checks for emails 1.0.5: - * Add rspamd control interface: - - support `stat` command to get runtime stats of rspamd workers - - support `reload` command to reload runtime elements (e.g. sqlite3 databases) - * Rework curve25519 library for modular design: - - add Sandy2x implementation by Tung Chou - - fix CPU detection for variables loading assembly - - add testing for curve25519 ECDH - * New fixed strings library - * Add `R_SUSPICIOUS_IMAGES` rule - * Enable mmap in sqlite3 - * Use new strings in the HTTP code - * Improve google perftools invocation - * Improve performance profiling in http test - * Reorganize includes to reduce namespace pollution - * Allow specific sections printing in configdump command - * Rework workers signals handlers to be chained if needed - * Update socketpair utility function - * Add control_path option for rspamd control protocol - * Fix ownership when listening on UNIX sockets - * Rework signals processing in main - * Remove extra tools from rspamd (they live in rspamadm now) - * Remove global rspamd_main - * Add global timeout for the overall task processing (8 seconds by default) - * Sanitize NULL values for fuzzy backend - * Store NM between encrypt/decrypt - * Add textpart:get_words_count method - * Fix generic DNS request in lua - * Tune hfilter weights - * Add support of IPv6 in hfilter - * Fix parsing of HTTP headers with IP addresses - * Sync with the recent libucl - * Various minor bugfixes + * Add rspamd control interface: + - support `stat` command to get runtime stats of rspamd workers + - support `reload` command to reload runtime elements (e.g. sqlite3 databases) + * Rework curve25519 library for modular design: + - add Sandy2x implementation by Tung Chou + - fix CPU detection for variables loading assembly + - add testing for curve25519 ECDH + * New fixed strings library + * Add `R_SUSPICIOUS_IMAGES` rule + * Enable mmap in sqlite3 + * Use new strings in the HTTP code + * Improve google perftools invocation + * Improve performance profiling in http test + * Reorganize includes to reduce namespace pollution + * Allow specific sections printing in configdump command + * Rework workers signals handlers to be chained if needed + * Update socketpair utility function + * Add control_path option for rspamd control protocol + * Fix ownership when listening on UNIX sockets + * Rework signals processing in main + * Remove extra tools from rspamd (they live in rspamadm now) + * Remove global rspamd_main + * Add global timeout for the overall task processing (8 seconds by default) + * Sanitize NULL values for fuzzy backend + * Store NM between encrypt/decrypt + * Add textpart:get_words_count method + * Fix generic DNS request in lua + * Tune hfilter weights + * Add support of IPv6 in hfilter + * Fix parsing of HTTP headers with IP addresses + * Sync with the recent libucl + * Various minor bugfixes 1.0.4: - * Add configdump routine to rspamadm - * Implement retransmits for fuzzy_check plugin - * Fix events processing for learning anf checking fuzzy hashes - * Avoid dependency on unneeded and uncompatible glib include - * Add `historyreset` command to the controller - * Fix loading of tokenizer config from dump (#389) - * Add sorting hints for the history - * Allow custom lua scripts for users/languages extraction (#388) - * Do not add FORGED_RECIPIENTS when 'To' is missing (#387) - * Do not add R_UNDISC_RCPT when 'To' is missing (#387) - * Add encryption to fuzzy check plugin - * Add encryption for fuzzy storage - * Add new epoch for encrypted fuzzy request - * Add encryption for `rspamd.com` storage - * Remove gmime processing for LDA mode as it is deadly broken - * Add routine to find end of headers position in mime messages - * Fix LDA headers folding - * Init libraries in rspamc client as well to avoid locale issues - * Avoid collision with locally installed includes - * Allocate and free memory with the same allocator in rspamadm (#385) - * Preserve expired fuzzy hashes counter - * Improvements in webui: - - Add favicon.ico - - Rework history table - - Fix sorting for the history - - Migrate to bootstrap 3 and jquery 2 - - Fix css bugs - - Add glyphicons - - Add reset history - - Improve history buttons - - Redraw graph to avoid display issues - - Webui is now MIT licensed to match licensing policy of rspamd + * Add configdump routine to rspamadm + * Implement retransmits for fuzzy_check plugin + * Fix events processing for learning anf checking fuzzy hashes + * Avoid dependency on unneeded and uncompatible glib include + * Add `historyreset` command to the controller + * Fix loading of tokenizer config from dump (#389) + * Add sorting hints for the history + * Allow custom lua scripts for users/languages extraction (#388) + * Do not add FORGED_RECIPIENTS when 'To' is missing (#387) + * Do not add R_UNDISC_RCPT when 'To' is missing (#387) + * Add encryption to fuzzy check plugin + * Add encryption for fuzzy storage + * Add new epoch for encrypted fuzzy request + * Add encryption for `rspamd.com` storage + * Remove gmime processing for LDA mode as it is deadly broken + * Add routine to find end of headers position in mime messages + * Fix LDA headers folding + * Init libraries in rspamc client as well to avoid locale issues + * Avoid collision with locally installed includes + * Allocate and free memory with the same allocator in rspamadm (#385) + * Preserve expired fuzzy hashes counter + * Improvements in webui: + - Add favicon.ico + - Rework history table + - Fix sorting for the history + - Migrate to bootstrap 3 and jquery 2 + - Fix css bugs + - Add glyphicons + - Add reset history + - Improve history buttons + - Redraw graph to avoid display issues + - Webui is now MIT licensed to match licensing policy of rspamd 1.0.3: - * Fix piechart clean slice (#380) - * Fix controller crashes when GString is reallocated (#381) - * Correctly set locale before start - * Set C locale for numeric values - * Add rspamadm routine: - - add `pw` command to manage passwords - - add `help` command for displaying help - - add `configtest` command to check configuration files - - add `keypair` command for generating encryption keys - - add `fuzzy_merge` routine to merge fuzzy sqlite databases - - add a simple manual page for rspamadm - * Allow metric registration for composite expressions - * Add strict mode for configtest - * Add logger counters - * Save and show learned messages count (#383) - * Add `no_stat` flag - * Add `task:set_flag` and `task:get_flags` (#382) - * Enable foreign keys in sqlite3 - * Remove orphaned shingles from fuzzy storage - * Optimize synchronization steps for fuzzy storage - * Allow delayed conditions registration - * Add lua API for conditions registering + * Fix piechart clean slice (#380) + * Fix controller crashes when GString is reallocated (#381) + * Correctly set locale before start + * Set C locale for numeric values + * Add rspamadm routine: + - add `pw` command to manage passwords + - add `help` command for displaying help + - add `configtest` command to check configuration files + - add `keypair` command for generating encryption keys + - add `fuzzy_merge` routine to merge fuzzy sqlite databases + - add a simple manual page for rspamadm + * Allow metric registration for composite expressions + * Add strict mode for configtest + * Add logger counters + * Save and show learned messages count (#383) + * Add `no_stat` flag + * Add `task:set_flag` and `task:get_flags` (#382) + * Enable foreign keys in sqlite3 + * Remove orphaned shingles from fuzzy storage + * Optimize synchronization steps for fuzzy storage + * Allow delayed conditions registration + * Add lua API for conditions registering 1.0.2: - * Fix critical bug in webui that prevents password from being sent - * Rework webui view: - - Switch to d3.js for graphs - - Improve piechart look - - Rework colors for piechart - - Fix layout for symbols - - Fix refresh button - * Add descriptions for whitelist maps - * Fix build on arm (#379) - * Fix issue with the last element in the radix trie - * Add more tests for radix trie algorithm - * Allow to extract URLs from query strings of other URLs (#361) - * Initialize rrd fields before writing to file - * Fix double free if no password has been specified + * Fix critical bug in webui that prevents password from being sent + * Rework webui view: + - Switch to d3.js for graphs + - Improve piechart look + - Rework colors for piechart + - Fix layout for symbols + - Fix refresh button + * Add descriptions for whitelist maps + * Fix build on arm (#379) + * Fix issue with the last element in the radix trie + * Add more tests for radix trie algorithm + * Allow to extract URLs from query strings of other URLs (#361) + * Initialize rrd fields before writing to file + * Fix double free if no password has been specified 1.0.1: - * Add writing to rrd from the controller - * Fixed lots of bugs in rrd code - * Adopt new DNS API in hfilter plugin (by @AlexeySa) - * Allow only one controller process to manage rrd file - * Set event base for fuzzy calls - * Improve fuzzy IO errors logging - * Add rra extraction function to rrd library - * Add graph handler to the controller - * Cache correct passwords to avoid too high CPU usage when working with webui - * Controller sockets are owned by router do not export them to task - * Optimize logging by skipping hash table search if it's empty - * Fix loading issue with broken statfiles - * Print assertions from glib to rspamd logger - * Load legacy `lua/rspamd.local.lua` - * Update webui with some fixes to learning and scanning + * Add writing to rrd from the controller + * Fixed lots of bugs in rrd code + * Adopt new DNS API in hfilter plugin (by @AlexeySa) + * Allow only one controller process to manage rrd file + * Set event base for fuzzy calls + * Improve fuzzy IO errors logging + * Add rra extraction function to rrd library + * Add graph handler to the controller + * Cache correct passwords to avoid too high CPU usage when working with webui + * Controller sockets are owned by router do not export them to task + * Optimize logging by skipping hash table search if it's empty + * Fix loading issue with broken statfiles + * Print assertions from glib to rspamd logger + * Load legacy `lua/rspamd.local.lua` + * Update webui with some fixes to learning and scanning 1.0.0: - * Rework symbols processing: - - Improve sorting logic for symbols - - Organize processing into multiple stages - - Added asynchronous watchers for symbols - - Added ability to organize dependencies between symbols - * Fixed URL redirector: - - Use optimized POE loop - - Organize dependencies - - Fix startup - * New sqlite3 backend: - - Allow to have per-languages and per-user statistics - - Allow sqlite3 to be used as statistics backend - * Store tokenizer configuration within statfiles - * Improve bayes statistics: - - Use headers and images metainformation in bayes - - Suggest using of pre-processed tokens for statistics - - Fix tokens normalization for OSB algorithm - * Rewrite url parsing: - - Fix numerous issues with url extraction and normalization - - Fix mailto urls - * Fix settings plugin to allow custom actions scores - * Improve rbl plugin - * Allow capturing patterns in rspamd lua regexp library - * Add GTUBE support - * Fix spamc legacy support - * Add DKIM support to RBL module - * Fix issues with multiple DKIM signatures - * Fix issue if rspamd cannot create statfiles (#331) - * Rework parts and task structure: - - Now text_parts, parts and received are arrays - - Pre-allocate arrays with some reasonable defaults - - Use arrays instead of lists in plugins and checks - - Remove unused fields from task structure - - Rework mime_foreach callback function - - Remove deprecated scan_milliseconds field - * Add ip_score plugin support (not enabled by default): - - Can check for asn/country and network using DNS lookups - - Can store and load reputation from redis server - * Improve PARTS_DIFFER rule to count merely different words - * New HTML parser: - - Parses HTML parts using a set of state machines - - Extracts useful data and exports it to lua functions: - + Styles - + Images - + URLs - + Colors - + Structure elements - - Added HTML rules for some checks - * New version of LUA DNS API - * Table versions of many functions in LUA API - * Improve rspamc client: - - Print execution time - - Allow executing of external commands and passing output to them - - Allow mime output mode when rspamc alters message according to rspamd - checks and send it to an external command or stdout - * Allow scanning of local files using HTTP requests - * Rework configuration system: - - Rules are now moved from the $CONFDIR to $RULESDIR to avoid ambiguity - - All modules configurations are now split in $CONFDIR/modules.d/* to - simplify upgrades - - Move hfilter to plugins - - Allow plugins and rules to define default scores to simplify metrics - setup - - Include overrides for all modules to honor local/automatic parameters - - Tune scores for many modules - * Rework and enable DMARC plugin - * Add whitelist plugin for SPF/DKIM/DMARC based whitelisting - * Add some common domains to whitelists shipped with rspamd - * Rework logging: - - Now each log entry supports module name and a `tag`. Tag is used to - identify unique objects (such as tasks) when checking log files - - It is possible to turn on debugging for the specific modules - - Systemd logging is fixed - * Improve spamassassin plugin. - - Now headers are matched more like SA - - Improve support of Message-ID - - Add support of ToCc header type - - Fix :addr and :name in headers regexps - * Resurrect rrd support code - * Save controller stats between restarts - * Fixed tonns of bugs - * Added tonns of minor improvements and features - * Added more unit tests - * Create functional tests framework - * Added documentation for missing modules - * Added rpm/deb repositories and scripts - * Updated WebUI and libucl externals + * Rework symbols processing: + - Improve sorting logic for symbols + - Organize processing into multiple stages + - Added asynchronous watchers for symbols + - Added ability to organize dependencies between symbols + * Fixed URL redirector: + - Use optimized POE loop + - Organize dependencies + - Fix startup + * New sqlite3 backend: + - Allow to have per-languages and per-user statistics + - Allow sqlite3 to be used as statistics backend + * Store tokenizer configuration within statfiles + * Improve bayes statistics: + - Use headers and images metainformation in bayes + - Suggest using of pre-processed tokens for statistics + - Fix tokens normalization for OSB algorithm + * Rewrite url parsing: + - Fix numerous issues with url extraction and normalization + - Fix mailto urls + * Fix settings plugin to allow custom actions scores + * Improve rbl plugin + * Allow capturing patterns in rspamd lua regexp library + * Add GTUBE support + * Fix spamc legacy support + * Add DKIM support to RBL module + * Fix issues with multiple DKIM signatures + * Fix issue if rspamd cannot create statfiles (#331) + * Rework parts and task structure: + - Now text_parts, parts and received are arrays + - Pre-allocate arrays with some reasonable defaults + - Use arrays instead of lists in plugins and checks + - Remove unused fields from task structure + - Rework mime_foreach callback function + - Remove deprecated scan_milliseconds field + * Add ip_score plugin support (not enabled by default): + - Can check for asn/country and network using DNS lookups + - Can store and load reputation from redis server + * Improve PARTS_DIFFER rule to count merely different words + * New HTML parser: + - Parses HTML parts using a set of state machines + - Extracts useful data and exports it to lua functions: + + Styles + + Images + + URLs + + Colors + + Structure elements + - Added HTML rules for some checks + * New version of LUA DNS API + * Table versions of many functions in LUA API + * Improve rspamc client: + - Print execution time + - Allow executing of external commands and passing output to them + - Allow mime output mode when rspamc alters message according to rspamd + checks and send it to an external command or stdout + * Allow scanning of local files using HTTP requests + * Rework configuration system: + - Rules are now moved from the $CONFDIR to $RULESDIR to avoid ambiguity + - All modules configurations are now split in $CONFDIR/modules.d/* to + simplify upgrades + - Move hfilter to plugins + - Allow plugins and rules to define default scores to simplify metrics + setup + - Include overrides for all modules to honor local/automatic parameters + - Tune scores for many modules + * Rework and enable DMARC plugin + * Add whitelist plugin for SPF/DKIM/DMARC based whitelisting + * Add some common domains to whitelists shipped with rspamd + * Rework logging: + - Now each log entry supports module name and a `tag`. Tag is used to + identify unique objects (such as tasks) when checking log files + - It is possible to turn on debugging for the specific modules + - Systemd logging is fixed + * Improve spamassassin plugin. + - Now headers are matched more like SA + - Improve support of Message-ID + - Add support of ToCc header type + - Fix :addr and :name in headers regexps + * Resurrect rrd support code + * Save controller stats between restarts + * Fixed tonns of bugs + * Added tonns of minor improvements and features + * Added more unit tests + * Create functional tests framework + * Added documentation for missing modules + * Added rpm/deb repositories and scripts + * Updated WebUI and libucl externals 0.9.10: - * Do not dereference null pointer on learning. - * Fix some extreme cases in BAYES. - * Add a workaround to avoid bad HTML messages breaking. - * Build with -O2 flags by default. - * Add constraints to limit DNS requests count per task. - * Add workaround for SURBL DNS flood. - * Set error if rspamd cannot learn anything. + * Do not dereference null pointer on learning. + * Fix some extreme cases in BAYES. + * Add a workaround to avoid bad HTML messages breaking. + * Build with -O2 flags by default. + * Add constraints to limit DNS requests count per task. + * Add workaround for SURBL DNS flood. + * Set error if rspamd cannot learn anything. 0.9.9: - * Don't use RWL_SPAMHAUS_WL (unknown result) for whitelisting (by @fatalbanana) - * Import updated public suffix list (by @fatalbanana) - * Remove debug message - * Fix settings (by @fatalbanana) - * Remove duplicated symbol registration - * Use WAL for fuzzy storage - * RBL fixes (by @fatalbanana): - - silence errors; - - yield unknown results from RBLs; - - fix scoring for DNSWL; - - fix use of RBL name as symbol; - - ignore RBL names that would not be yielded; - * Support captures in regular expressions - * Add captures support to lua_regexp - * Support dist on FreeBSD and Darwin - * Add RCVD_IN_DNSWL_NONE as whitelisting exclusion (by @fatalbanana) - * Multiple fixes to URL detection: - - support port definition; - - fix query and path recognition; - - fix parsing of multiple slashes in URL; - - fix parsing query just after port; - - fix path field in `url:to_table` method; - - improve support of IP based URLs. - * Set ignore_whitelists = true for RECEIVED_SPAMHAUS_XBL (by @fatalbanana) - * Add GTUBE support - * Ignore User header in SA mode + * Don't use RWL_SPAMHAUS_WL (unknown result) for whitelisting (by @fatalbanana) + * Import updated public suffix list (by @fatalbanana) + * Remove debug message + * Fix settings (by @fatalbanana) + * Remove duplicated symbol registration + * Use WAL for fuzzy storage + * RBL fixes (by @fatalbanana): + - silence errors; + - yield unknown results from RBLs; + - fix scoring for DNSWL; + - fix use of RBL name as symbol; + - ignore RBL names that would not be yielded; + * Support captures in regular expressions + * Add captures support to lua_regexp + * Support dist on FreeBSD and Darwin + * Add RCVD_IN_DNSWL_NONE as whitelisting exclusion (by @fatalbanana) + * Multiple fixes to URL detection: + - support port definition; + - fix query and path recognition; + - fix parsing of multiple slashes in URL; + - fix parsing query just after port; + - fix path field in `url:to_table` method; + - improve support of IP based URLs. + * Set ignore_whitelists = true for RECEIVED_SPAMHAUS_XBL (by @fatalbanana) + * Add GTUBE support + * Ignore User header in SA mode 0.9.8: - * Fix critical bug in bayes classifier (#305) - * Fix critical bug in RBL module (by @fatalbanana) - * Fix and rework settings plugin. - * Fix get_all_opts for a case of non-iterable options. - * Use tld for redirector's matching. + * Fix critical bug in bayes classifier (#305) + * Fix critical bug in RBL module (by @fatalbanana) + * Fix and rework settings plugin. + * Fix get_all_opts for a case of non-iterable options. + * Use tld for redirector's matching. 0.9.7: - * Add whitelist_exception setting to RBL module (by @fatalbanana) - * Don't use RWL_MAILSPIKE_POSSIBLE or DNSWL_BLOCKED for whitelisting (by - @fatalbanana) - * Fix extreme cases in bayes classifier. - * Fix parsing of urls with '?' at the end of hostname. - * Update interface. - * Fix number of issues with webui interaction. - * Fix saving maps. - * Allow user@ and @domain matches in multimap. - * Fix issues with bounces From processing. - * Fix abs/fabs misuse. - * Fix builds on suse and arch linux distributions. + * Add whitelist_exception setting to RBL module (by @fatalbanana) + * Don't use RWL_MAILSPIKE_POSSIBLE or DNSWL_BLOCKED for whitelisting (by + @fatalbanana) + * Fix extreme cases in bayes classifier. + * Fix parsing of urls with '?' at the end of hostname. + * Update interface. + * Fix number of issues with webui interaction. + * Fix saving maps. + * Allow user@ and @domain matches in multimap. + * Fix issues with bounces From processing. + * Fix abs/fabs misuse. + * Fix builds on suse and arch linux distributions. 0.9.6: - * Fix memory leak if mime cannot be parsed. - * Fix dkim cache expiration. - * Fix issues with redirector HTTP response. - * Fix abnormal connection closing with certains messages with a high score - (issue #296) - * Fix redirector installation. - * Use specific POE loop for some systems. - * Fix number of issues in URL redirector. - * Fix selecting URLs for sending to redirector. + * Fix memory leak if mime cannot be parsed. + * Fix dkim cache expiration. + * Fix issues with redirector HTTP response. + * Fix abnormal connection closing with certains messages with a high score + (issue #296) + * Fix redirector installation. + * Use specific POE loop for some systems. + * Fix number of issues in URL redirector. + * Fix selecting URLs for sending to redirector. 0.9.5: - * Avoid double free when extending HTTP message. - * Fix double free if multiple classifiers are defined. - * Fix misprint in spamassassin plugin. - * Fix cpuid invocation on i386. - * Fix ownership issues for zero-copy decode. - * Allow __len metamethod on rspamd{text}. - * Add base64 decoding lua utility. - * Fix build on FreeBSD - * Skip spaces at the beginning of mime messages. - * DBL_ABUSE_REDIR should not have significant weight. - * Allow to split by lua_regexp rspamd{text} objects. - * Allow to specify custom stop pattern for lua_tcp. + * Avoid double free when extending HTTP message. + * Fix double free if multiple classifiers are defined. + * Fix misprint in spamassassin plugin. + * Fix cpuid invocation on i386. + * Fix ownership issues for zero-copy decode. + * Allow __len metamethod on rspamd{text}. + * Add base64 decoding lua utility. + * Fix build on FreeBSD + * Skip spaces at the beginning of mime messages. + * DBL_ABUSE_REDIR should not have significant weight. + * Allow to split by lua_regexp rspamd{text} objects. + * Allow to specify custom stop pattern for lua_tcp. 0.9.4: - * Fix critical bugs in tokenization algorithm - * Write unit tests for tokenization - * Add documentation for lua_tcp - * Switch off legacy tokenization by default. - * Fix critical bugs in words normalization - * Add lua bindings to tokenizer. - * Implement storing of HTTP headers inside task - * Add lua API to accerss HTTP headers data - * Implemented base64 encoding suitable for MIME - * Use caseless hash and equal functions for HTTP request headers. - * Improve debian architectures support (by @dottedmag) + * Fix critical bugs in tokenization algorithm + * Write unit tests for tokenization + * Add documentation for lua_tcp + * Switch off legacy tokenization by default. + * Fix critical bugs in words normalization + * Add lua bindings to tokenizer. + * Implement storing of HTTP headers inside task + * Add lua API to accerss HTTP headers data + * Implemented base64 encoding suitable for MIME + * Use caseless hash and equal functions for HTTP request headers. + * Improve debian architectures support (by @dottedmag) 0.9.3: - * Revert incorrect regexp change that broke the default rules - * Fix lua_tcp module + * Revert incorrect regexp change that broke the default rules + * Fix lua_tcp module 0.9.2: - * Fix error on spawning unique workers. - * Add preliminary version of generic LUA TCP requests API. - * Use lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc) - * Fix fuzzy mime strings with only type. - * Improve thunderbird sanity checks. - * Fix critical bug on matching regular expressions. - * Make hiredis optional dependency. - * Fix multiple bugs in daemon reloading + * Fix error on spawning unique workers. + * Add preliminary version of generic LUA TCP requests API. + * Use lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc) + * Fix fuzzy mime strings with only type. + * Improve thunderbird sanity checks. + * Fix critical bug on matching regular expressions. + * Make hiredis optional dependency. + * Fix multiple bugs in daemon reloading 0.9.1: - * Restore utf8 validation for regular expressions to avoid crashes - * Fix symbols displaying in the interface - * Add symbol groups to the interface - * Fix maps ID parsing in the controller - * Add multimap and regexp modules documentation - * Backport fixes from libucl - * Fix debian package (by @dottedmag) - * Rework XXH32 invocations + * Restore utf8 validation for regular expressions to avoid crashes + * Fix symbols displaying in the interface + * Add symbol groups to the interface + * Fix maps ID parsing in the controller + * Add multimap and regexp modules documentation + * Backport fixes from libucl + * Fix debian package (by @dottedmag) + * Rework XXH32 invocations 0.9.0: - * Add support of the fast and secure protocol level encryption: - - curve25519 is used for key exchange; - - chacha20/poly1305 cryptobox construction for bulk encryption; - - zero latency overhead; - - encrypting and balancing HTTP proxy worker - * Rework expressions and create new expressions library: - - aggressive optimizations based on the abstract syntax tree; - - abstract expressions support (regular expressions, functions, lua modules - composites and so on) - - New comparison and '+' operators support - - New greedy algorithm to minimize execution time of expressions and - all symbols - - Dynamic expressions benchmark and reoptimizations - * Many improvements to the LUA API: - - reworked logger module allowing to do pretty print of the most of lua - types (including tables and userdata classes) - - reworked lua redis and lua HTTP to support more features - - added opaque type for passing large text chunks without copying - - new regexp module with many auxiliary functions (e.g. `re:split`) - * LuaJIT is now the default requirement for rspamd allowing to speed up lua - execution by a large margin (however, plain lua is still supported) - * New plugins: - - spamassassin rules plugin that allows to load and re-use the most of - SA rules natively - - DMARC plugin that evaluates SPF and DKIM policies to the domain policies - - many old plugins has been reworked to implement new features and improve - stability - * New aho-corasic trie implementation from @mischasan that allows to load and - use hundreds of thousands of patterns with no influence on load - * Support of PCRE JIT and PCRE JIT fast path modes that significantly improves - the performance of regular expressions if supported by PCRE - * New URLs parser and extractor: - - removed legacy code that was useless for url finding - - reworked algorithms of URL parsing for more precise and accurate results - - added top-level-domains tree from http://publicsuffix.org - - improved emails parsing - - removed many phishing false positives due to TLD tree check - * New statistics infrastructure: - - created a separate layer of statistic library - - improved OSB-Bayes by re-weighting tokens according to the original - academic paper and `crm114` implementation, which reduced false positives - rate significantly - - created learn cache to avoid double learning of statistics and providing - an efficient way to re-learn class for a message - - created abstract layers for different statistics backends - - implemented new tokenization algorithms with fast or secure (siphash) - hashes to generate statistics features - * Reworked utf8 tokenization that previously corrupted all UTF8 words (minor - incompatibility with old fuzzy hashes with utf-8 symbols) - * SPF module has been completely rewritten to support complex cases of - `include` and `redirect` within SPF records - * DKIM module now supports multiple signatures - * Controller passwords can now be stored encrypted by `PBKDF2-HMAC` in the - configuration file - * Many hand-written HTTP clients has been replaced with the common rspamd - http module - * New test framework: - - import lua `telescope` test framework - - add unit tests for many rspamd modules and routines - - create a unit test for each possible bug found - - use luajit ffi for testing C code - - added preliminary support of functional testing by creating tasks from lua - * Randomize hash seed to avoid certain hash tables vulnerabilities - * Documentation improvements: - - added documentation for the vast majority of rspamd modules - - added documentation for rspamd protocol - - added documentation for the most of rspamd LUA extensions - * Fixed tonns of bugs and memory leaks - * Added tonns of minor features + * Add support of the fast and secure protocol level encryption: + - curve25519 is used for key exchange; + - chacha20/poly1305 cryptobox construction for bulk encryption; + - zero latency overhead; + - encrypting and balancing HTTP proxy worker + * Rework expressions and create new expressions library: + - aggressive optimizations based on the abstract syntax tree; + - abstract expressions support (regular expressions, functions, lua modules + composites and so on) + - New comparison and '+' operators support + - New greedy algorithm to minimize execution time of expressions and + all symbols + - Dynamic expressions benchmark and reoptimizations + * Many improvements to the LUA API: + - reworked logger module allowing to do pretty print of the most of lua + types (including tables and userdata classes) + - reworked lua redis and lua HTTP to support more features + - added opaque type for passing large text chunks without copying + - new regexp module with many auxiliary functions (e.g. `re:split`) + * LuaJIT is now the default requirement for rspamd allowing to speed up lua + execution by a large margin (however, plain lua is still supported) + * New plugins: + - spamassassin rules plugin that allows to load and re-use the most of + SA rules natively + - DMARC plugin that evaluates SPF and DKIM policies to the domain policies + - many old plugins has been reworked to implement new features and improve + stability + * New aho-corasic trie implementation from @mischasan that allows to load and + use hundreds of thousands of patterns with no influence on load + * Support of PCRE JIT and PCRE JIT fast path modes that significantly improves + the performance of regular expressions if supported by PCRE + * New URLs parser and extractor: + - removed legacy code that was useless for url finding + - reworked algorithms of URL parsing for more precise and accurate results + - added top-level-domains tree from http://publicsuffix.org + - improved emails parsing + - removed many phishing false positives due to TLD tree check + * New statistics infrastructure: + - created a separate layer of statistic library + - improved OSB-Bayes by re-weighting tokens according to the original + academic paper and `crm114` implementation, which reduced false positives + rate significantly + - created learn cache to avoid double learning of statistics and providing + an efficient way to re-learn class for a message + - created abstract layers for different statistics backends + - implemented new tokenization algorithms with fast or secure (siphash) + hashes to generate statistics features + * Reworked utf8 tokenization that previously corrupted all UTF8 words (minor + incompatibility with old fuzzy hashes with utf-8 symbols) + * SPF module has been completely rewritten to support complex cases of + `include` and `redirect` within SPF records + * DKIM module now supports multiple signatures + * Controller passwords can now be stored encrypted by `PBKDF2-HMAC` in the + configuration file + * Many hand-written HTTP clients has been replaced with the common rspamd + http module + * New test framework: + - import lua `telescope` test framework + - add unit tests for many rspamd modules and routines + - create a unit test for each possible bug found + - use luajit ffi for testing C code + - added preliminary support of functional testing by creating tasks from lua + * Randomize hash seed to avoid certain hash tables vulnerabilities + * Documentation improvements: + - added documentation for the vast majority of rspamd modules + - added documentation for rspamd protocol + - added documentation for the most of rspamd LUA extensions + * Fixed tonns of bugs and memory leaks + * Added tonns of minor features 0.8.3: - * Various critical fixes in distribution (by @dottedmag and @fatalbanana) - * Fixed bugs in url detector to parse certain patterns - * Add default host and helo for a client - * Some sanity checks for tokenizer and classifier - * Reiterate on systemd support - * Fix missing symbol registration - * Add support of spamc compatible output - * Filter double-dots in rbl.lua validate_dns (by @fatalbanana) - * Update ucl submodule due to critical bugfix + * Various critical fixes in distribution (by @dottedmag and @fatalbanana) + * Fixed bugs in url detector to parse certain patterns + * Add default host and helo for a client + * Some sanity checks for tokenizer and classifier + * Reiterate on systemd support + * Fix missing symbol registration + * Add support of spamc compatible output + * Filter double-dots in rbl.lua validate_dns (by @fatalbanana) + * Update ucl submodule due to critical bugfix 0.8.2: - * Create fuzzy db if it does not exist - * Fix: Centos init script: configtest() (by @AlexeySa) - * Enable one_shot for RECEIVED_SPAMHAUS_XBL - Fixes #102 (by @fatalbanana) - * Update Exim patch (by @fatalbanana) - * Fix processing of unix sockets. - * Allow applying settings to authenticated users (by @fatalbanana) - * Make settings priorities work as documented (by @fatalbanana) - * Fix race condition in symbols planner - * Add DNSWL_BLOCKED symbol (by @fatalbanana) - * Make Exim pass usernames to rspamd (by @fatalbanana) - * Update RBL module (by @fatalbanana): - - fix indentation; - - collapse loops; - - avoid calling for un-needed information; - - allow disabling RBLs for authenticated users - * once_received.lua: Fix indentation & add exclusion for authenticated users (by @fatalbanana) - * hfilter.lua: Add exclusion for authenticated users (by @AlexeySa) - * Updates to hfilter rules (by @AlexeySa) - * Set empty <> user or addr for msgs without FROM (by @eneq123) - * Fix: attempt to index field '?' (a nil value) (by @eneq123) - * Fix: if not exist Date-header (by @AlexeySa) - * Add task:get_content() method. - * rbl.lua: Ignore private IP space (by @fatalbanana) - * Allow to check radix maps from lua by rspamd{ip} - * Make local exclusions configurable per-RBL (by @fatalbanana) - * Add rspamd_config:radix_from_config() (by @fatalbanana) - * Support emails dnsbl in rbl (by @fatalbanana) - * Complete rework of url extraction logic - * Allow customizations for unix sockets. (fixes #182) - * Set lua path according to rspamd settings. - * Import lua-functional for plugins stuff. - * Completely rewrite multimap plugin in functional style. - * Fix FORGED_MUA_THUNDERBIRD_MSGID (fixes #186) - * Check IPv6 addresses at dnswl.org and Spamhaus whitelist (by @fatalbanana) - * Add lowercase utility for utf8 strings. - * Various fixes to build system - * Updated debian configuration infrastructure (by @dottedmag) + * Create fuzzy db if it does not exist + * Fix: Centos init script: configtest() (by @AlexeySa) + * Enable one_shot for RECEIVED_SPAMHAUS_XBL - Fixes #102 (by @fatalbanana) + * Update Exim patch (by @fatalbanana) + * Fix processing of unix sockets. + * Allow applying settings to authenticated users (by @fatalbanana) + * Make settings priorities work as documented (by @fatalbanana) + * Fix race condition in symbols planner + * Add DNSWL_BLOCKED symbol (by @fatalbanana) + * Make Exim pass usernames to rspamd (by @fatalbanana) + * Update RBL module (by @fatalbanana): + - fix indentation; + - collapse loops; + - avoid calling for un-needed information; + - allow disabling RBLs for authenticated users + * once_received.lua: Fix indentation & add exclusion for authenticated users (by @fatalbanana) + * hfilter.lua: Add exclusion for authenticated users (by @AlexeySa) + * Updates to hfilter rules (by @AlexeySa) + * Set empty <> user or addr for msgs without FROM (by @eneq123) + * Fix: attempt to index field '?' (a nil value) (by @eneq123) + * Fix: if not exist Date-header (by @AlexeySa) + * Add task:get_content() method. + * rbl.lua: Ignore private IP space (by @fatalbanana) + * Allow to check radix maps from lua by rspamd{ip} + * Make local exclusions configurable per-RBL (by @fatalbanana) + * Add rspamd_config:radix_from_config() (by @fatalbanana) + * Support emails dnsbl in rbl (by @fatalbanana) + * Complete rework of url extraction logic + * Allow customizations for unix sockets. (fixes #182) + * Set lua path according to rspamd settings. + * Import lua-functional for plugins stuff. + * Completely rewrite multimap plugin in functional style. + * Fix FORGED_MUA_THUNDERBIRD_MSGID (fixes #186) + * Check IPv6 addresses at dnswl.org and Spamhaus whitelist (by @fatalbanana) + * Add lowercase utility for utf8 strings. + * Various fixes to build system + * Updated debian configuration infrastructure (by @dottedmag) 0.8.1: - * Add sqlite and perl as dependencies for RPM/Debian packages (by @fatalbanana) - * Remove whitelist.lua from RPM file list (by @fatalbanana) - * Make Exim pass hostnames to rspamd (by @fatalbanana) - * Fix building on Fedora (by @fatalbanana) - * Add toggle for disabling installation of systemd units on Linux (by @fatalbanana) - * Fix double format rounding that caused output corruption (reported by @fatalbanana) - * Revert broken change for destructors ordering that led to memory corruption - * Do not reset symbols case of settings if parsed from lua (reported by @andrejzverev) - * Fix build on SunOS (by @wiedi) - * Fix multiple crashes on broken DKIM DNS records - * Fix critical issue with composites weights removing - * Fix memory corruption in composites processing code - * Ignore non-SPF TXT records when parsing SPF includes + * Add sqlite and perl as dependencies for RPM/Debian packages (by @fatalbanana) + * Remove whitelist.lua from RPM file list (by @fatalbanana) + * Make Exim pass hostnames to rspamd (by @fatalbanana) + * Fix building on Fedora (by @fatalbanana) + * Add toggle for disabling installation of systemd units on Linux (by @fatalbanana) + * Fix double format rounding that caused output corruption (reported by @fatalbanana) + * Revert broken change for destructors ordering that led to memory corruption + * Do not reset symbols case of settings if parsed from lua (reported by @andrejzverev) + * Fix build on SunOS (by @wiedi) + * Fix multiple crashes on broken DKIM DNS records + * Fix critical issue with composites weights removing + * Fix memory corruption in composites processing code + * Ignore non-SPF TXT records when parsing SPF includes 0.8.0: - * New fuzzy check logic: - - use shingles algorithm for fuzzy matching - - use blake2 instead of md5 for larger output space - - combine fuzzy and strict matching - - allow to organize private storages by means of keys - - preserve compatibility with previous versions - * New fuzzy storage: - - use sqlite instead of own memory based hash tables - - rework commands interface - - add conversion from the old format - - add fuzzy match by shignles - - support old rspamd versions - * Add lemmatizing for words used in fuzzy hashes that allows to improve match - quality by using of the first forms of all words - * Rework language detection - * Fix several critical bugs, memory leaks and deadlocks: - - memory leak in HTML nodes parsing - - deadlock in logger code - - deadlock in signals processing - - crashes in fuzzy_storage - - crashes in tokenizers if the input was empty - * Import new libucl with several bugfixes and improvements - * Support listening on ipv6 addresses only - * Fix macro expansion in SPF module - * Several bugfixes in DKIM module - * Add load headers support for mime parts to the lua API - * Add documentation for: - - workers in general - - fuzzy_storage worker - - fuzzy_check plugin - - mimepart and textpart lua API modules + * New fuzzy check logic: + - use shingles algorithm for fuzzy matching + - use blake2 instead of md5 for larger output space + - combine fuzzy and strict matching + - allow to organize private storages by means of keys + - preserve compatibility with previous versions + * New fuzzy storage: + - use sqlite instead of own memory based hash tables + - rework commands interface + - add conversion from the old format + - add fuzzy match by shignles + - support old rspamd versions + * Add lemmatizing for words used in fuzzy hashes that allows to improve match + quality by using of the first forms of all words + * Rework language detection + * Fix several critical bugs, memory leaks and deadlocks: + - memory leak in HTML nodes parsing + - deadlock in logger code + - deadlock in signals processing + - crashes in fuzzy_storage + - crashes in tokenizers if the input was empty + * Import new libucl with several bugfixes and improvements + * Support listening on ipv6 addresses only + * Fix macro expansion in SPF module + * Several bugfixes in DKIM module + * Add load headers support for mime parts to the lua API + * Add documentation for: + - workers in general + - fuzzy_storage worker + - fuzzy_check plugin + - mimepart and textpart lua API modules 0.7.6: - * Apply boundary fix for dkim simple canonization - * Fix ping command - * Return nil if header was not found in lua_task - * Fix hang in upstreams revive logic - * Decode entitles when normalizing HTML parts - * Fix logic of finding URLs in HTML parts - * Do not include \0 into length of text when performing conversion to utf8 - * Fix raw vs parsed reperesentations - Raw parts are now: - - decoded b64/qp, but *NOT* converted to utf-8 - Processed parts are now: - - converted to UTF-8 - - normalized if needed (e.g. HTML tags are stripped) - * Rework DKIM canonization to line based - * Fix fuzzy hashes adding - * Use more specific hash function for fuzzy - * Fix leaking of iconv descriptors - * Fix PTR resolving in lua resolver - * Rework spf module. - - Copy data to memory pool as cached record might be destroyed causing - freed memory being passed to the protocol output (use after free) - - Allow SPF_NEUTRAL policy to be handled separately - - Add R_SPF_NEUTRAL to the default config - * Rework `register_symbols` function - * Allow to disable components of hfilter + * Apply boundary fix for dkim simple canonization + * Fix ping command + * Return nil if header was not found in lua_task + * Fix hang in upstreams revive logic + * Decode entitles when normalizing HTML parts + * Fix logic of finding URLs in HTML parts + * Do not include \0 into length of text when performing conversion to utf8 + * Fix raw vs parsed reperesentations + Raw parts are now: + - decoded b64/qp, but *NOT* converted to utf-8 + Processed parts are now: + - converted to UTF-8 + - normalized if needed (e.g. HTML tags are stripped) + * Rework DKIM canonization to line based + * Fix fuzzy hashes adding + * Use more specific hash function for fuzzy + * Fix leaking of iconv descriptors + * Fix PTR resolving in lua resolver + * Rework spf module. + - Copy data to memory pool as cached record might be destroyed causing + freed memory being passed to the protocol output (use after free) + - Allow SPF_NEUTRAL policy to be handled separately + - Add R_SPF_NEUTRAL to the default config + * Rework `register_symbols` function + * Allow to disable components of hfilter 0.7.5: - * Fix owner when creating folder /run/rspamd (by @sfirmery) - * Fix IP validity checks - * Decode URLs obtained from HTML tags - * Fix crash with unweighted upstreams - * Stop processing headers in parts - * Set sockaddr.sa_family properly when connectig to upstreams - * Fix reload issues in surbl and fuzzy_check (reported by @citrin) - * Fix timeouts in redirector - * Improve lua errors reporting - * Fix lua closures processing in libucl - * Rework calling of lua functions from regexp module - * Choose raw regexp for raw headers - * Rework conversion to utf since glib one is broken - * Ignore SGML style tags in html - * Fix old bug with non-capturing https urls - * Fix memory corruption on fuzzy reload (reported by @citrin) - * Fix percents display in rspamc - * Fix buffer update for DKIM - * Do not validate utf for raw headers + * Fix owner when creating folder /run/rspamd (by @sfirmery) + * Fix IP validity checks + * Decode URLs obtained from HTML tags + * Fix crash with unweighted upstreams + * Stop processing headers in parts + * Set sockaddr.sa_family properly when connectig to upstreams + * Fix reload issues in surbl and fuzzy_check (reported by @citrin) + * Fix timeouts in redirector + * Improve lua errors reporting + * Fix lua closures processing in libucl + * Rework calling of lua functions from regexp module + * Choose raw regexp for raw headers + * Rework conversion to utf since glib one is broken + * Ignore SGML style tags in html + * Fix old bug with non-capturing https urls + * Fix memory corruption on fuzzy reload (reported by @citrin) + * Fix percents display in rspamc + * Fix buffer update for DKIM + * Do not validate utf for raw headers 0.7.4: - * Fix build under *BSD - * Detect HAN unicode script - * Implement language detection heuristic for text parts - * Fix time output in history - * Improve piechart coloring - * Fix \r\n conversion in DKIM module (reported by @citrin) - * Try to detect systems with no IPv6 support - * Fix multiple/single values in use settings (reported by @citrin) - * Rework IP addresses in upstreams: - - Select ipv4/unix addresses if they exist and use ipv6 for ipv6 only - upstreams (since the support of ipv6 is poor in many OSes and - environments) - - Free IP list on upstream destruction - - Add test cases for addresses selection - - Allow adding of free form IP addresses to upstreams + * Fix build under *BSD + * Detect HAN unicode script + * Implement language detection heuristic for text parts + * Fix time output in history + * Improve piechart coloring + * Fix \r\n conversion in DKIM module (reported by @citrin) + * Try to detect systems with no IPv6 support + * Fix multiple/single values in use settings (reported by @citrin) + * Rework IP addresses in upstreams: + - Select ipv4/unix addresses if they exist and use ipv6 for ipv6 only + upstreams (since the support of ipv6 is poor in many OSes and + environments) + - Free IP list on upstream destruction + - Add test cases for addresses selection + - Allow adding of free form IP addresses to upstreams * Fix endianness in lua_radix search (reported by @citrin) * Soft shutdown should also set wanna_die flag (reported by @citrin) * Stop use-after-free in event loop termination @@ -3840,26 +3883,26 @@ * Fix issues with PTR and MX elements in SPF parser (reported by @citrin) 0.7.3: - * New upstreams code: - - simplify upstreams API; - - unify strings parsing in upstreams definition; - - add configuration options for the upstreams; - - for failed upstreams re-resolve their addresses; - - use all resolved addresses for an upstream (round-robin); - - implement stable hashing and use it by default for upstreams; - - add unit test for upstreams module. - * Rework signals processing in all rspamd workers: - - signals are now processed in the event loop; - - implement the most common signal handlers for all workers; - - add callbacks for workers specific signal handlers - * Fix critical issue with fuzzy storage: - Fuzzy stroage could not save any hashes on termination due to bugged - signals handling + * New upstreams code: + - simplify upstreams API; + - unify strings parsing in upstreams definition; + - add configuration options for the upstreams; + - for failed upstreams re-resolve their addresses; + - use all resolved addresses for an upstream (round-robin); + - implement stable hashing and use it by default for upstreams; + - add unit test for upstreams module. + * Rework signals processing in all rspamd workers: + - signals are now processed in the event loop; + - implement the most common signal handlers for all workers; + - add callbacks for workers specific signal handlers + * Fix critical issue with fuzzy storage: + Fuzzy stroage could not save any hashes on termination due to bugged + signals handling * Fix roll history IP storage * Rework ipv4/ipv6 handling in parsing addresses: - - turn off support of IPV6_V6ONLY socket option; - - create ipv6 socket prior to ipv4 one to handle systems with v6/v4 - sockets enabled (Linux) + - turn off support of IPV6_V6ONLY socket option; + - create ipv6 socket prior to ipv4 one to handle systems with v6/v4 + sockets enabled (Linux) * Remove CBL as it's wholly included in Spamhaus XBL (by @fatalbanana) * Remove nszones.com fake RBL (by @citrin) * Fix upstreams interaction for fuzzy_check @@ -3869,21 +3912,21 @@ * Sync bugfixes from libucl 0.7.2: - * Convert all maps to the compressed radix trie - * Allow IPv6 addresses in IP maps - * Remove dynamic items support from symbols cache - * Allow hex encoded output of strings - * Fix bug with control connections count - * Process fuzzy weight correctly (reported by @fatalbanana) - * Remove extra reference retain of http connection on error - * Remove deprecated options from the default config - * Add `one_shot` attr to metric's symbols - * Doc: add documentation for metrics - * Add Upstart job to debian packaging (by @CameronNemo) - * Config: improve SURBL symbols descriptions (by @citrin) - * Config: reflect SURBL changes (by @citrin): - - Outblaze removed, malware moved to separate list: - http://www.surbl.org/news/internal/MW-malware-sublist-added-to-multi + * Convert all maps to the compressed radix trie + * Allow IPv6 addresses in IP maps + * Remove dynamic items support from symbols cache + * Allow hex encoded output of strings + * Fix bug with control connections count + * Process fuzzy weight correctly (reported by @fatalbanana) + * Remove extra reference retain of http connection on error + * Remove deprecated options from the default config + * Add `one_shot` attr to metric's symbols + * Doc: add documentation for metrics + * Add Upstart job to debian packaging (by @CameronNemo) + * Config: improve SURBL symbols descriptions (by @citrin) + * Config: reflect SURBL changes (by @citrin): + - Outblaze removed, malware moved to separate list: + http://www.surbl.org/news/internal/MW-malware-sublist-added-to-multi * Fix C modules initialization on restart * Treat single IP as a single IP in radix lists (reported by @citrin) * Do not touch file and core limits if not asked explicitly (reported by @citrin) @@ -3895,478 +3938,478 @@ * Sync with libucl include features 0.7.1: - * Fix typo in stat output. - * Fix issues with includes crossing with the system includes - * Restore testing framework - * Add radix trie test suite - * Implement new path-compressed radix trie. - - The performance benefit over the old algorithm is about 1.5 times. - - Memory usage is significantly lower as well. - - Now radix trie can accept any IPv4/IPv6 values - * Various improvements to the memory pools code - * Fix writing reply to a client when no filters are defined - * Write base32 encoded fuzzy - * Fix 'soft reject' action - * Fix rspamd reload and modules reconfiguration - * Fix subject rewriting for the default subject - * Fix states for processing task and pre-filters - * Fix issues with connection closing - * Fix crashes in rdns - * Fix ratelimit pre-filter - * Update exim patch. - - Update to the recent exim version - - Strip extra leading src/ from the patch - - Remove sendfile since it was broken - - Fix rspamd spam report for exim - * Improve documentation + * Fix typo in stat output. + * Fix issues with includes crossing with the system includes + * Restore testing framework + * Add radix trie test suite + * Implement new path-compressed radix trie. + - The performance benefit over the old algorithm is about 1.5 times. + - Memory usage is significantly lower as well. + - Now radix trie can accept any IPv4/IPv6 values + * Various improvements to the memory pools code + * Fix writing reply to a client when no filters are defined + * Write base32 encoded fuzzy + * Fix 'soft reject' action + * Fix rspamd reload and modules reconfiguration + * Fix subject rewriting for the default subject + * Fix states for processing task and pre-filters + * Fix issues with connection closing + * Fix crashes in rdns + * Fix ratelimit pre-filter + * Update exim patch. + - Update to the recent exim version + - Strip extra leading src/ from the patch + - Remove sendfile since it was broken + - Fix rspamd spam report for exim + * Improve documentation 0.7.0: - * Use HTTP protocol for all operatiosn - * Webui worker is now removed and controller works as webui - * Allow to serve static files via controller by option `static_dir` - * Rspamd interface is now a part of rspamd - * Rspamc client has been rewritten to use HTTP and non-blocking mode - allowing to start multiple operations simultaneously (see `-n` option) - * Lua API was completely reworked to satisfy modern standards of LUA: - * Module `lua-message` was removed - * Reduced number of superglobals registered by rspamd - * Many functions has been redesigned - * Symbols registration is now more convenient - * Users settings has been rewritten as lua plugin - * Reworked headers system as gmime's based one misses many headers and is - very slow to get headers values - * Reorganized code and removed embedded jannsson by using UCL for all json - parsing - * Migrated to `librdns` for DNS resolving that improves concurrency for - DNS requests significantly - * Fixed tonns of bugs in MIME processing - * Improved metrcis and default symbol's weights - * Added new RBL's - * Fixed a number of issues in the modules - * Removed several memory leaks found - * Fix unicode processing - * Fix fuzzy checking for unicode parts - * Significantly improve documentation and especially LUA API docs - * See migration notes at https://rspamd.com/doc/migration.html + * Use HTTP protocol for all operatiosn + * Webui worker is now removed and controller works as webui + * Allow to serve static files via controller by option `static_dir` + * Rspamd interface is now a part of rspamd + * Rspamc client has been rewritten to use HTTP and non-blocking mode + allowing to start multiple operations simultaneously (see `-n` option) + * Lua API was completely reworked to satisfy modern standards of LUA: + * Module `lua-message` was removed + * Reduced number of superglobals registered by rspamd + * Many functions has been redesigned + * Symbols registration is now more convenient + * Users settings has been rewritten as lua plugin + * Reworked headers system as gmime's based one misses many headers and is + very slow to get headers values + * Reorganized code and removed embedded jannsson by using UCL for all json + parsing + * Migrated to `librdns` for DNS resolving that improves concurrency for + DNS requests significantly + * Fixed tonns of bugs in MIME processing + * Improved metrcis and default symbol's weights + * Added new RBL's + * Fixed a number of issues in the modules + * Removed several memory leaks found + * Fix unicode processing + * Fix fuzzy checking for unicode parts + * Significantly improve documentation and especially LUA API docs + * See migration notes at https://rspamd.com/doc/migration.html 0.6.8: - * Controller now listen for localhost and not for 127.0.0.1 by default - * Allow FCrDNS-style RBL lookups (by @fatalbanana) - * Reduce threshold for parts_differ function. - * Fix hostname lookup for rdns rbl (by @AlexeySa) - * Fix HFILTER_URL_ONELINE to reduce false positive rate. - * Fix whitelist module. - * Allow override system predefined settings without touching system ones - by .try_include macro (by @andrejzverev) - * Check for [ip.address]-style HELO and suppress lookups. (by - @fatalbanana) - * Optimize hfilter (by @AlexeySa) - * Fix issue with random numbers generator in dns. - * Use more clever time values to setup entropy. - * Synced with the recent libucl. + * Controller now listen for localhost and not for 127.0.0.1 by default + * Allow FCrDNS-style RBL lookups (by @fatalbanana) + * Reduce threshold for parts_differ function. + * Fix hostname lookup for rdns rbl (by @AlexeySa) + * Fix HFILTER_URL_ONELINE to reduce false positive rate. + * Fix whitelist module. + * Allow override system predefined settings without touching system ones + by .try_include macro (by @andrejzverev) + * Check for [ip.address]-style HELO and suppress lookups. (by + @fatalbanana) + * Optimize hfilter (by @AlexeySa) + * Fix issue with random numbers generator in dns. + * Use more clever time values to setup entropy. + * Synced with the recent libucl. 0.6.7: - * Use ChaCha20 for DNS generator (more secure DNS id) - * Unknown symbols now has zero weight and not 1.0 - * Fix fuzzy hashes expire time - * Fix critical issue in statfiles on FreeBSD 9 (and some other platforms) - * Add .include_map macro to ucl parser - * Update libucl - * Fix headers end detection for DKIM module - * Fix a bug in received headers parser - * Validate IP addresses before pushing them to lua - * Start new documentation project - * Fixed tonns of other minor bugs - * Start to prepare for 0.7 with HTTP protocol and new settings + * Use ChaCha20 for DNS generator (more secure DNS id) + * Unknown symbols now has zero weight and not 1.0 + * Fix fuzzy hashes expire time + * Fix critical issue in statfiles on FreeBSD 9 (and some other platforms) + * Add .include_map macro to ucl parser + * Update libucl + * Fix headers end detection for DKIM module + * Fix a bug in received headers parser + * Validate IP addresses before pushing them to lua + * Start new documentation project + * Fixed tonns of other minor bugs + * Start to prepare for 0.7 with HTTP protocol and new settings 0.6.6: - * Removed issue with BUFSIZ limitation in the controller output - * Simplify logging symbols escaping - * Adjusted weights for several rules - * Improve spamhaus rbl support - * Removed PBL for received headers checks - * Added hfilter module that performs various HELO and IP checks. - * Rspamd can now be reloaded by HUP signal - * Fuzzy storage should expire hashes properly - * Build system has been reworked for better supportof pkg-config - * Various minor bugfixes + * Removed issue with BUFSIZ limitation in the controller output + * Simplify logging symbols escaping + * Adjusted weights for several rules + * Improve spamhaus rbl support + * Removed PBL for received headers checks + * Added hfilter module that performs various HELO and IP checks. + * Rspamd can now be reloaded by HUP signal + * Fuzzy storage should expire hashes properly + * Build system has been reworked for better supportof pkg-config + * Various minor bugfixes 0.6.5: - * Fixed critical bug in DNS resolver, introduced in 0.6.4 - * Improved multimap and rbl plugins to skip - * Add dns_sockets option for tuning sockets per server in DNS resolver - * Improved packages for rspamd + * Fixed critical bug in DNS resolver, introduced in 0.6.4 + * Improved multimap and rbl plugins to skip + * Add dns_sockets option for tuning sockets per server in DNS resolver + * Improved packages for rspamd 0.6.4: - * Added io channels for DNS request to balance load and reduce id - collisions chance - * Fixed a bug in SPF filter that may cause core dump in specific - circumstances - * FIxed default config for rbl module - * It is possible to get a list of rspamc commands with their descriptions - * Added SORBS bl to the default config - * 2tld file for surbl module has been significantly extended - * Perl modules has been removed from the code. - * Fixed an issue in libucl when parsing macros + * Added io channels for DNS request to balance load and reduce id + collisions chance + * Fixed a bug in SPF filter that may cause core dump in specific + circumstances + * FIxed default config for rbl module + * It is possible to get a list of rspamc commands with their descriptions + * Added SORBS bl to the default config + * 2tld file for surbl module has been significantly extended + * Perl modules has been removed from the code. + * Fixed an issue in libucl when parsing macros 0.6.3: - * Fixed issues with DNS: - - labels decompression algorithm was fixed; - - added resolve_mx to lua API; - - fixed modules that use DNS. - * Lua modules once_received and emails reworked for new resolver API and UCL. - * Debian package was polished. - * Fixed a bug in fuzzy_check module that prevents correct processing messages - without valid parts. + * Fixed issues with DNS: + - labels decompression algorithm was fixed; + - added resolve_mx to lua API; + - fixed modules that use DNS. + * Lua modules once_received and emails reworked for new resolver API and UCL. + * Debian package was polished. + * Fixed a bug in fuzzy_check module that prevents correct processing messages + without valid parts. 0.6.2: - * Fuzzy check module has been reworked: - - now fuzzy_check operates with a group of rules, that define which - servers sre to be checked; - - it is possible to specify read_only groups to avoid learning errors; - - turn fuzzy_check to one_shot mode permanently; - - fuzzy_check module configuration is now incompatible with the previous - versions. - * Imported bugfixes from libucl. - * Fixed whitelist plugin. - * Fixed statfiles resizing. - * Improved logging initialization order. - * Fixed race condition in the controller worker. + * Fuzzy check module has been reworked: + - now fuzzy_check operates with a group of rules, that define which + servers sre to be checked; + - it is possible to specify read_only groups to avoid learning errors; + - turn fuzzy_check to one_shot mode permanently; + - fuzzy_check module configuration is now incompatible with the previous + versions. + * Imported bugfixes from libucl. + * Fixed whitelist plugin. + * Fixed statfiles resizing. + * Improved logging initialization order. + * Fixed race condition in the controller worker. 0.6.1: - * Critical bugfixes: - - fixed build system; - - fixed in_class setting in bayes learning; + * Critical bugfixes: + - fixed build system; + - fixed in_class setting in bayes learning; 0.6.0: - * Use UCL instead xml for configuration (https://github.com/vstakhov/libucl) - * Fix statistics module normalization - * Rework the many modules for the new configuration: - - surbl has incompatible configuration; - - fuzzy_check has incompatible configuration; - - multimap has now new configuration; - - received_rbl is removed and replaced by rbl module. - * Removed deprecated options: - - statfile_pool_size; - - action and required/reject score for a metric. - * Simplify build system and unify configuration for all platforms. - * Improved URL detector (reduced false positive rate). - * Lua 5.2 is now the default and fully supported version. - * Tons of bugfixes and minor improvements. + * Use UCL instead xml for configuration (https://github.com/vstakhov/libucl) + * Fix statistics module normalization + * Rework the many modules for the new configuration: + - surbl has incompatible configuration; + - fuzzy_check has incompatible configuration; + - multimap has now new configuration; + - received_rbl is removed and replaced by rbl module. + * Removed deprecated options: + - statfile_pool_size; + - action and required/reject score for a metric. + * Simplify build system and unify configuration for all platforms. + * Improved URL detector (reduced false positive rate). + * Lua 5.2 is now the default and fully supported version. + * Tons of bugfixes and minor improvements. 0.5.6: - * Fix bug with counters incrementing in rolling history - * Detect expl and exp2l as some systems do not have it - * Support input streams without Content-Length - * Implement counters output via rspamc and controller interface - * Fix bug with udp sockets in fuzzy storage + * Fix bug with counters incrementing in rolling history + * Detect expl and exp2l as some systems do not have it + * Support input streams without Content-Length + * Implement counters output via rspamc and controller interface + * Fix bug with udp sockets in fuzzy storage 0.5.5: - * New bayes normalizator based on inverse chi-square function - * Various fixes to fuzzy storage - * Allow update fuzzy storage only from specific IP addresses - * Better support of IPv6 and address selection algorithms - * Add CentOS spec file + * New bayes normalizator based on inverse chi-square function + * Various fixes to fuzzy storage + * Allow update fuzzy storage only from specific IP addresses + * Better support of IPv6 and address selection algorithms + * Add CentOS spec file 0.5.4: - * Fixed issues with diff algorithm - * Added support of RRD statistics - * Add webui worker for interface interaction - * Fix a lot of issues with dynamic conf - * Fix critical memory leak in settings code - * Improve stability of the system + * Fixed issues with diff algorithm + * Added support of RRD statistics + * Add webui worker for interface interaction + * Fix a lot of issues with dynamic conf + * Fix critical memory leak in settings code + * Improve stability of the system 0.5.3: - * Added dynamic options - * Added advanced metaclassfication - * Added RESTfull API for controller - * Improved hashing algorithms - * Various fixes for rspamc client: - - allow interacting with unix sockets - - librspamdclient major cleanup - - bayes is now default classifier + * Added dynamic options + * Added advanced metaclassfication + * Added RESTfull API for controller + * Improved hashing algorithms + * Various fixes for rspamc client: + - allow interacting with unix sockets + - librspamdclient major cleanup + - bayes is now default classifier 0.5.2: - * Added lua bindings for: - - basic mime parts, that allows checkign attachements for example; - - DNS resolver; - * Existing lua bindings now works without task object allowing to use them - in custom code. - * Threads system was reworked to avoid global lua interpreter lock. - * DKIM module now converts all line endings to CRLF how opendkim does. - * URL detector is now more accurate for text parts. - * Several critical bugs and memory leaks were fixed. + * Added lua bindings for: + - basic mime parts, that allows checkign attachements for example; + - DNS resolver; + * Existing lua bindings now works without task object allowing to use them + in custom code. + * Threads system was reworked to avoid global lua interpreter lock. + * DKIM module now converts all line endings to CRLF how opendkim does. + * URL detector is now more accurate for text parts. + * Several critical bugs and memory leaks were fixed. 0.5.1: - * Added lua worker type to handle network connections in lua. - * Added lua bindings for async_session, IO dispatcher, memory_pool, and - worker. - * Composites can now uses other composites in expressions. - * Fixes for debian package and for FreeBSD >= 9.1 - * Add support of gmime-2.6 if gmime-2.4 is not found. - * Improve url detection and phishing detection. - * Add lua mime_part library to get an access to all message part - attributes (like filename, length, type). + * Added lua worker type to handle network connections in lua. + * Added lua bindings for async_session, IO dispatcher, memory_pool, and + worker. + * Composites can now uses other composites in expressions. + * Fixes for debian package and for FreeBSD >= 9.1 + * Add support of gmime-2.6 if gmime-2.4 is not found. + * Improve url detection and phishing detection. + * Add lua mime_part library to get an access to all message part + attributes (like filename, length, type). 0.5.0: - * Added SMTP lightweight balancing proxy with XCLIENT support. - * Added lua bindings for upstreams objects and API. - * New pre-filters are implemented to support initial checking for messages. - * Added ratelimit plugin that uses redis protocol to store data. - * Added ipv6 support to spf and some other modules. - * Unbreak spf plugin. - * Allow options with the same name be threated as list. - * DKIM plugin an parsing code was added. - * Separate build system to put logic in several shared libraries. - * Many bugfixes. + * Added SMTP lightweight balancing proxy with XCLIENT support. + * Added lua bindings for upstreams objects and API. + * New pre-filters are implemented to support initial checking for messages. + * Added ratelimit plugin that uses redis protocol to store data. + * Added ipv6 support to spf and some other modules. + * Unbreak spf plugin. + * Allow options with the same name be threated as list. + * DKIM plugin an parsing code was added. + * Separate build system to put logic in several shared libraries. + * Many bugfixes. 0.3.2: - * Add error handling for regexps - * Fix quit command in controller interface - * Write symbols weights to rspamc output - * Improve logic of selecting rspamc version - * Do not try to parse broken DNS replies - * Add 'raw' flag to FROM_EXCESS_BASE64 rule (requested by citrin) - * Output message id in rspamc reply - * Fix inserting composite symbol - * Fix output of log line - * Document composites - * Add logging for fuzzy checks - * Add logging for learning - * Improve logic of learning messages - do not learn more than specific threshold - * Fix inserting results for symbols that were incorrectly (for example more than 1 time) defined in config file - * Do not output control characters if output is not terminal - * Fix some logic errors in learning - * Consider lua plugins errors as fatal configuration errors - * Fix writing message id during fuzzy_add command - * Display weight of symbols correctly - * Fixes to winnow learning - * One more try to improve accuracy of winnow algorithm - * Add bayesian classifier (initial version) - * Remove normalizer as it is winnow specific thing, so all statistic algorithms now returns value from 0 to 1 - * Some fixes to fuzzy hashes expiration: - * Fix assertion while look up value in NULL hash (found by cirtin) - * Fix normalization for systems that have not tanhl function - * Ignore rfc822 group addresses - * Move images library to core rspamd - * Add lua api to access images properties - * Add post filters to lua API - filters that would be called after all message's processing - * Add ability to check for specified symbol in task results from lua - * Add ability to check for metric's results from lua - * Add ability to learn specified statfile form lua - * Add ability to extract filename and size of images from lua - * Fix assertion while extracting internet address - * Fix races in fuzzy storage - * Make spf parser case insensitive - * Add ability to check hashes of selected mime types - * Add ability to set minimum size in bytes for mime types - * Add ability to set minimum dimensions for images - * Assume all text/* content types as text/plain - * Fix getting data wrapper for gmime24 - * Many fixes to statfile synchronization system - * Fixed statfile pool initialization and synchronization with disk - * Prepare 0.3.2 - * Fix Mail::Rspamd::Config for new rspamd features - * Use Mail::Rspamd::Config in rspamc client - * Write user's name to rspamd log - * Prepare rspamd build infrastructure for creating rpm and deb packages - * Fix depends - * Add start script for linux systems - * Fix shared usage of statfiles - * Add invalidation of statfiles in case of learning, so now statfiles - * This should fix shared usage of statfile pool by several processes - * Fix misprint (reported by az) - * Fix stupid error when all checks can be done in a single pass - * New trie based url scanner (based on libcamel) - * Small fixes to rspamd perl client - * Write fuzzy hashes info to log - * Add trie interface to lua api - * Explain sample config and cleanup it + * Add error handling for regexps + * Fix quit command in controller interface + * Write symbols weights to rspamc output + * Improve logic of selecting rspamc version + * Do not try to parse broken DNS replies + * Add 'raw' flag to FROM_EXCESS_BASE64 rule (requested by citrin) + * Output message id in rspamc reply + * Fix inserting composite symbol + * Fix output of log line + * Document composites + * Add logging for fuzzy checks + * Add logging for learning + * Improve logic of learning messages - do not learn more than specific threshold + * Fix inserting results for symbols that were incorrectly (for example more than 1 time) defined in config file + * Do not output control characters if output is not terminal + * Fix some logic errors in learning + * Consider lua plugins errors as fatal configuration errors + * Fix writing message id during fuzzy_add command + * Display weight of symbols correctly + * Fixes to winnow learning + * One more try to improve accuracy of winnow algorithm + * Add bayesian classifier (initial version) + * Remove normalizer as it is winnow specific thing, so all statistic algorithms now returns value from 0 to 1 + * Some fixes to fuzzy hashes expiration: + * Fix assertion while look up value in NULL hash (found by cirtin) + * Fix normalization for systems that have not tanhl function + * Ignore rfc822 group addresses + * Move images library to core rspamd + * Add lua api to access images properties + * Add post filters to lua API - filters that would be called after all message's processing + * Add ability to check for specified symbol in task results from lua + * Add ability to check for metric's results from lua + * Add ability to learn specified statfile form lua + * Add ability to extract filename and size of images from lua + * Fix assertion while extracting internet address + * Fix races in fuzzy storage + * Make spf parser case insensitive + * Add ability to check hashes of selected mime types + * Add ability to set minimum size in bytes for mime types + * Add ability to set minimum dimensions for images + * Assume all text/* content types as text/plain + * Fix getting data wrapper for gmime24 + * Many fixes to statfile synchronization system + * Fixed statfile pool initialization and synchronization with disk + * Prepare 0.3.2 + * Fix Mail::Rspamd::Config for new rspamd features + * Use Mail::Rspamd::Config in rspamc client + * Write user's name to rspamd log + * Prepare rspamd build infrastructure for creating rpm and deb packages + * Fix depends + * Add start script for linux systems + * Fix shared usage of statfiles + * Add invalidation of statfiles in case of learning, so now statfiles + * This should fix shared usage of statfile pool by several processes + * Fix misprint (reported by az) + * Fix stupid error when all checks can be done in a single pass + * New trie based url scanner (based on libcamel) + * Small fixes to rspamd perl client + * Write fuzzy hashes info to log + * Add trie interface to lua api + * Explain sample config and cleanup it 0.3.1: - * Add modules documentation - * Continue implementing smtp proxy - * Implement new learning system, now rspamd should be much more intelligent while learning messages - * Convert statistic sums to use long double for counters - * Use hyperbolic tangent for internal normalizer - * In classify normalize result after comparing, not before - * New symbols sorter - * Fix strict aliasing while compiling with optimization - * Fix tanhl detection for platforms that have not implementation of it - * Remove several compile warnings - * Add experimental support of dynamic rules to regexp module - * Document views configuration - * Several fixes to documentation - * Add more logic for dynamic rules - * Add documentation for dynamic rules - * Add ability to make negations in networks in dynamic rules - * Clean up cache items correctly - * Implement basic SMTP dialog: - * Implement interaction with smtp upstream (with support of XCLIENT) - * Check messages received via smtp proxy - * Add support for sendfile in io dispatcher - * Fix issues with compatibility of worker_task and smtp proxy - * Proxy DATA command - * Fix SMTP - * Change metric logic - * Completely remove lex/yacc readers for config - * Make common sense of metric/action and symbols - * Sync changes with all plugins - * Incorrectly removed in previous commit - * Fix misprint (by Andrej Zverev) - * announce the "password" keyword in usage list - * Implement initial version of greylisting triplets storage - * Fix issues with smtp worker - * Fix QUIT command in SMTP worker - * Some fixes about new metrics system (may be incomplete) - * Get weights of symbol from default metric for symbols cache - * Fix setting task->from/task->rctp in smtp client - * Copy from and rcpt correctly - * Some performance improvements to IO dispatcher (do not drain the whole buffer after a single line readed) - * Fix smtp data input - * Fix misprint - * Add limit of maximum allowed smtp session errors - * New logic of SURBL module: - * Use system mkstemp(3) on systems where it is available as glib implementation - * Try to fix memmove issues in io dispatcher - * Remove debug from SURBL module - * Rewrite buffered input for line policy (again) - * Fix issue with links that are ip addresses in numeric form in surbl - * On Darwin use BSD style sendfile definition - * Reorganize platform specific knobs in CMakeLists - * Use gettimeofday on systems that have not clock_getres - * Use ftime for dns trans id generation on systems without clock_getres - * Darwin sendfile(2) support - * TIMEDB->TIMEB - * More to previous commit - * Pass env from main() arguments instead of platform specific global environ - * Fix compatibility issues - * Fix -lintl detection - * Init some variables to avoid problems - * Remove garbadge (gnome terminal sucks) - * Add more information about why we drop smtp connection - * Fix mkstemp call - * Send to upstream QUIT command at the end of session - * Check return value of each rspamd_dispatcher_write as in case of write errors sessions can be destroyed early - * Fix states in smtp dialog - * Use rspamd_snprintf instead of libc one - * Fix URLS command - * Fix reconfigure process of surbl module - * Fix destroying smtp session (unmap memory and do not delete pool early) - * Delete pool after using its variables - * Delay timer must be registered in async session to correctly handle connection termination - * Register dns requests in session too - * Make session before registering events - * Remove events in handlers - * Add ability to set filters for smtp worker for each smtp stage - * Add very initial version of DNS resolver (many things to be done) - * Announce weights and sync - * Fix few typo - * Understand short names of facility in logging config - * Add ability to make whitelist for spf checks - * Misprint != -> == - * Handle lua tag in way that it is not required to write additional text: - * Strip all starting whitespace symbols from xml texts - * Fix stupid bug in calculating buffer length while reading file maps - * Add resolv.conf parsing into dns.c - * Fix microseconds<->milliseconds conversions - * Take callback argument in Mail::Rspamd::Client for processing files and directories - * Print results if rspamc is called for a directory - * Fix stupid error with surbl module reconfig (another one, blame me) - * Do not show duplicate urls in url header - * Fix detection of numeric urls (reported by citrin) - * Write real time of message's scan to log (not only virtual) - * Fix chartable module in utf mode - * Fix parsing of some broken urls - * Add ability to test regexp with 'T' flag - * Write more code for DNS resolver: - * Make DNS resolver working - * Many improvements to rspamd test suite: now it CAN be used for testing rspamd functionality - * Write DNS resolver tests - * Fix issues with memory_pool mutexes and with creating of statfiles - * Forgotten in previous commit - * Add support for parsing SPF and SRV records - * Fix PTR parsing - * Add tests - * Make SURBL module to use rspamd dns module - * Several fixes to DNS logic - * Remove evdns and use only rspamd resolver - * Very hard to detect problem with race among error in socket and destroying task while we are writing to socket and go through a hash table - * Fix resolving in smtp module - * Init events before configuring resolver in smtp worker - * Set resolver inside task - * Fix reload signal (reported by citrin) - * Some improvements to redirector - * Call has_forked method to inform POE about fork - * Fix lua DNS code - * Decompress labels in DNS packets more strictly - * Fix some problems with TXT records - * Try to fix removing of DNS events - * Do not insert unparsed RR's into reply - * Calling callbacks may cause destroying session from which we are calling callback so we MUST call callback as the latest action - * Fix check_smtp_data function - * Add ability to make views by recipient - * Add ability to set metric's action from config file - * Fix bug with writing garbadge if message has no urls or no messages - * Fix bug with incorrect behaviour of compare_parts_distance function - * Add ability to assign several actions to one metric - * Report action in rspamc protocol - * Mail::Rspamd::Client and rspamc can now understand Action header too - * Write action to log as well - * Make valgrind happy about comparing symbols - * Add more debug to comparing parts distance function - * Write action even if message has no symbols - * Make improvements to HTML entities decoder: now it replaces entities with common characters and - * Add -d option to force debug output - * Assume 7bit as default transfer encoding - * Do not overwrite lua plugins and configs if they already exists in target directory - * Improve logging - * Write queue id to log - * Remove test messages from cmake - * Reopen log file by USR1 signal - * Add reopenlog method to FreeBSD rc script - * Adopt foreach for cmake 2.6 - * Fix to rc script - * Do not try to resolve names with several dots in a row - * Fix surbl request formatting for ip addresses - * Handle cases of broken requests - * Fix problems with parsing compressed names - * Fix TXT records parsing - * Fix expanding spf macros that may fail in rare cases - * Fix another error with early task destroying - * Handle empty from header - * Improve reopenlog command in rc script - * Strip trailing whitespace characters in Mail::Rspamd::Client - * Use ungreedy match to strip trailing whitespaces - * Stupid error in calculation compressed label length - * Some optimizations to client library - * Do not compare empty parts - * Empty and non-empty parts are always different - * Save in regexp cache the whole regexp with header name (if exists) and with flags - * Add rspamd_log variable to lua plugins to access logging functions - * Each part in rspamd task now can have parent part - * Check for parts distance only for multipart/alternative subparts - * Do not check attachements even if they are text (but attached as file) - * Do not die if write (2) returned ENOSPACE while doing logging, turn on throttling mode instead (1 write try in a second) - * Add ability to turn on debug for specific symbols - * Add ability to configure dns timeouts and dns retransmits in config file - * More debug - * Fix extracting arguments in lua logger interface - * Turn off debug_ip during reload if it was disabled by new config - * Improve lua logging - * Pre-init symbols cache when rereading config - * Fix lua representing of invalid ip (nil, not 255.255.255.255) - * Fix R_TO_SEEMS_AUTO rule (by citrin) - * Add multimap lua plugin - * Fix some multimap issues - * Try to save images hashes to fuzzy storage to stop some annoying spammers - * Allocate some more bytes for read buffer to avoid incorrect behavoiur - * Add ability to check dns black lists by multimap module - * Add multimap documentation - * Fix labels parsing - * Another try to save regexps in cache correctly - * Improve test logs for regexps - * Fix parsing txt records to avoid reading of uninitialized data - * Fix error with writing symbols cache file - * Fix error while working in utf mode when raw regexps was not created properly - * Do not add extra byte while converting text to utf - * Add error handling for regexps + * Add modules documentation + * Continue implementing smtp proxy + * Implement new learning system, now rspamd should be much more intelligent while learning messages + * Convert statistic sums to use long double for counters + * Use hyperbolic tangent for internal normalizer + * In classify normalize result after comparing, not before + * New symbols sorter + * Fix strict aliasing while compiling with optimization + * Fix tanhl detection for platforms that have not implementation of it + * Remove several compile warnings + * Add experimental support of dynamic rules to regexp module + * Document views configuration + * Several fixes to documentation + * Add more logic for dynamic rules + * Add documentation for dynamic rules + * Add ability to make negations in networks in dynamic rules + * Clean up cache items correctly + * Implement basic SMTP dialog: + * Implement interaction with smtp upstream (with support of XCLIENT) + * Check messages received via smtp proxy + * Add support for sendfile in io dispatcher + * Fix issues with compatibility of worker_task and smtp proxy + * Proxy DATA command + * Fix SMTP + * Change metric logic + * Completely remove lex/yacc readers for config + * Make common sense of metric/action and symbols + * Sync changes with all plugins + * Incorrectly removed in previous commit + * Fix misprint (by Andrej Zverev) + * announce the "password" keyword in usage list + * Implement initial version of greylisting triplets storage + * Fix issues with smtp worker + * Fix QUIT command in SMTP worker + * Some fixes about new metrics system (may be incomplete) + * Get weights of symbol from default metric for symbols cache + * Fix setting task->from/task->rctp in smtp client + * Copy from and rcpt correctly + * Some performance improvements to IO dispatcher (do not drain the whole buffer after a single line readed) + * Fix smtp data input + * Fix misprint + * Add limit of maximum allowed smtp session errors + * New logic of SURBL module: + * Use system mkstemp(3) on systems where it is available as glib implementation + * Try to fix memmove issues in io dispatcher + * Remove debug from SURBL module + * Rewrite buffered input for line policy (again) + * Fix issue with links that are ip addresses in numeric form in surbl + * On Darwin use BSD style sendfile definition + * Reorganize platform specific knobs in CMakeLists + * Use gettimeofday on systems that have not clock_getres + * Use ftime for dns trans id generation on systems without clock_getres + * Darwin sendfile(2) support + * TIMEDB->TIMEB + * More to previous commit + * Pass env from main() arguments instead of platform specific global environ + * Fix compatibility issues + * Fix -lintl detection + * Init some variables to avoid problems + * Remove garbadge (gnome terminal sucks) + * Add more information about why we drop smtp connection + * Fix mkstemp call + * Send to upstream QUIT command at the end of session + * Check return value of each rspamd_dispatcher_write as in case of write errors sessions can be destroyed early + * Fix states in smtp dialog + * Use rspamd_snprintf instead of libc one + * Fix URLS command + * Fix reconfigure process of surbl module + * Fix destroying smtp session (unmap memory and do not delete pool early) + * Delete pool after using its variables + * Delay timer must be registered in async session to correctly handle connection termination + * Register dns requests in session too + * Make session before registering events + * Remove events in handlers + * Add ability to set filters for smtp worker for each smtp stage + * Add very initial version of DNS resolver (many things to be done) + * Announce weights and sync + * Fix few typo + * Understand short names of facility in logging config + * Add ability to make whitelist for spf checks + * Misprint != -> == + * Handle lua tag in way that it is not required to write additional text: + * Strip all starting whitespace symbols from xml texts + * Fix stupid bug in calculating buffer length while reading file maps + * Add resolv.conf parsing into dns.c + * Fix microseconds<->milliseconds conversions + * Take callback argument in Mail::Rspamd::Client for processing files and directories + * Print results if rspamc is called for a directory + * Fix stupid error with surbl module reconfig (another one, blame me) + * Do not show duplicate urls in url header + * Fix detection of numeric urls (reported by citrin) + * Write real time of message's scan to log (not only virtual) + * Fix chartable module in utf mode + * Fix parsing of some broken urls + * Add ability to test regexp with 'T' flag + * Write more code for DNS resolver: + * Make DNS resolver working + * Many improvements to rspamd test suite: now it CAN be used for testing rspamd functionality + * Write DNS resolver tests + * Fix issues with memory_pool mutexes and with creating of statfiles + * Forgotten in previous commit + * Add support for parsing SPF and SRV records + * Fix PTR parsing + * Add tests + * Make SURBL module to use rspamd dns module + * Several fixes to DNS logic + * Remove evdns and use only rspamd resolver + * Very hard to detect problem with race among error in socket and destroying task while we are writing to socket and go through a hash table + * Fix resolving in smtp module + * Init events before configuring resolver in smtp worker + * Set resolver inside task + * Fix reload signal (reported by citrin) + * Some improvements to redirector + * Call has_forked method to inform POE about fork + * Fix lua DNS code + * Decompress labels in DNS packets more strictly + * Fix some problems with TXT records + * Try to fix removing of DNS events + * Do not insert unparsed RR's into reply + * Calling callbacks may cause destroying session from which we are calling callback so we MUST call callback as the latest action + * Fix check_smtp_data function + * Add ability to make views by recipient + * Add ability to set metric's action from config file + * Fix bug with writing garbadge if message has no urls or no messages + * Fix bug with incorrect behaviour of compare_parts_distance function + * Add ability to assign several actions to one metric + * Report action in rspamc protocol + * Mail::Rspamd::Client and rspamc can now understand Action header too + * Write action to log as well + * Make valgrind happy about comparing symbols + * Add more debug to comparing parts distance function + * Write action even if message has no symbols + * Make improvements to HTML entities decoder: now it replaces entities with common characters and + * Add -d option to force debug output + * Assume 7bit as default transfer encoding + * Do not overwrite lua plugins and configs if they already exists in target directory + * Improve logging + * Write queue id to log + * Remove test messages from cmake + * Reopen log file by USR1 signal + * Add reopenlog method to FreeBSD rc script + * Adopt foreach for cmake 2.6 + * Fix to rc script + * Do not try to resolve names with several dots in a row + * Fix surbl request formatting for ip addresses + * Handle cases of broken requests + * Fix problems with parsing compressed names + * Fix TXT records parsing + * Fix expanding spf macros that may fail in rare cases + * Fix another error with early task destroying + * Handle empty from header + * Improve reopenlog command in rc script + * Strip trailing whitespace characters in Mail::Rspamd::Client + * Use ungreedy match to strip trailing whitespaces + * Stupid error in calculation compressed label length + * Some optimizations to client library + * Do not compare empty parts + * Empty and non-empty parts are always different + * Save in regexp cache the whole regexp with header name (if exists) and with flags + * Add rspamd_log variable to lua plugins to access logging functions + * Each part in rspamd task now can have parent part + * Check for parts distance only for multipart/alternative subparts + * Do not check attachements even if they are text (but attached as file) + * Do not die if write (2) returned ENOSPACE while doing logging, turn on throttling mode instead (1 write try in a second) + * Add ability to turn on debug for specific symbols + * Add ability to configure dns timeouts and dns retransmits in config file + * More debug + * Fix extracting arguments in lua logger interface + * Turn off debug_ip during reload if it was disabled by new config + * Improve lua logging + * Pre-init symbols cache when rereading config + * Fix lua representing of invalid ip (nil, not 255.255.255.255) + * Fix R_TO_SEEMS_AUTO rule (by citrin) + * Add multimap lua plugin + * Fix some multimap issues + * Try to save images hashes to fuzzy storage to stop some annoying spammers + * Allocate some more bytes for read buffer to avoid incorrect behavoiur + * Add ability to check dns black lists by multimap module + * Add multimap documentation + * Fix labels parsing + * Another try to save regexps in cache correctly + * Improve test logs for regexps + * Fix parsing txt records to avoid reading of uninitialized data + * Fix error with writing symbols cache file + * Fix error while working in utf mode when raw regexps was not created properly + * Do not add extra byte while converting text to utf + * Add error handling for regexps -- 2.39.5