* [Conf] Add R_DKIM_PERMFAIL to the metric
* [CritFix] Dkim: Fix simple canonicalisation if multiple signatures are presented
* [CritFix] Fix controller paths normalisation
* [Feature] Add INVALID_DATE rule
* [Feature] Add controller endpoint for training neural
* [Feature] Add sanity checks for actions thresholds
* [Feature] Add support of '==' and '!=' in Rspamd expressions
* [Feature] Composites: Improve composite atoms parser
* [Feature] Docker: use Debian slim variant
* [Feature] Elastic: Add some missing fields
* [Feature] Extract text from img alt attributes
* [Feature] Improve charset detection logic
* [Feature] Lua_clickhouse: Add optional row callback for large selections
* [Feature] Lua_dns_resolver: Add idna_convert_utf8 method
* [Feature] Lua_mime: Add ability to do multipattern replacement
* [Feature] Lua_trie: Allow to report start of the match
* [Feature] Multimap: support adding map values as extra options
* [Feature] Neural: Move PCA learning to a subprocess
* [Feature] RBL: support matching content/image URLs only
* [Feature] RBL: support use of multiple selectors
* [Feature] Reputation: Allow to specify ip masks
* [Feature] Support SMIME signed messages container
* [Feature] Support multiple conditions for symbols
* [Feature] Support ping in milter mode
* [Feature] Support rspamd_text in selector regexps
* [Feature] Use own daemonization routine
* [Feature] Vadesecure: Implement settings_outbound feature as recommended by Vade
* [Feature] `rspamadm clickhouse` command
* [Feature] allow hyperscan for aarch64
* [Fix] Allow to set priorities between post init scripts
* [Fix] Allow to use maps for strings that are not zero terminated
* [Fix] Apply max_lua_urls limit for emails as well
* [Fix] Arc: Fix CV check on signing
* [Fix] Arc: Fix signing of the broken ARC chains
* [Fix] Clickhouse: escape carriage return
* [Fix] Composites: Allow partial match
* [Fix] Deduct type of a table methods
* [Fix] Do not load errored hyperscan database
* [Fix] Do not process links in ignored html tags
* [Fix] Fix ClamAV result for cached encrypted file (#3395)
* [Fix] Fix canonicalisation when l= tag is presented
* [Fix] Fix flag shift
* [Fix] Fix handling of skip/skip_process http flags
* [Fix] Fix html attachments checks
* [Fix] Fix issue with pushing binary formats to Lua strings
* [Fix] Fix logging for rspamadm
* [Fix] Fix off-by-one with init check
* [Fix] Fix parsing of escape characters in quoted pairs
* [Fix] Fix pushing ucl strings with \0 inside
* [Fix] Fix quoted-printable soft newlines bugged case
* [Fix] Fix settings in case actions are set to null (#3415)
* [Fix] Fix several issues with auth results producing
* [Fix] Fix smtp comments exclusion
* [Fix] Fix smtp date syntax definition
* [Fix] Fix substring search in case if srchlen == inlen
* [Fix] Fix text selectors
* [Fix] Honour `systemd` setting when logging to console (#3514)
* [Fix] Html: Add entities collisions prevention logic (e.g. for mathml entities)
* [Fix] Lua_auth_results: Quote potentially bad values in AR header
* [Fix] Multimap: Fix flags usage
* [Fix] Multimap: Fix scoring for combined maps
* [Fix] Plug GList * leak in redis pool
* [Fix] RBL: allow for multiple matches of the same label if types are different
* [Fix] Rely on libev checks for file maps
* [Fix] Restore simple dkim canonicalisation mode
* [Fix] Return MimeCharset as we work with emails...
* [Fix] Spamassassin: Fix pcre_only flags
* [Fix] Spamassassin: Preserve 'pcre_only' flag when dealing with regexp replacements
* [Fix] Try to fix GError leak
* [Fix] Try to fix a mess with settings loading by adding priorities
* [Fix] Try to move setings initialisation to a later stage
* [Fix] Use dup fd in milter handler to avoid races with the proxy
* [Fix] Use message pointer to avoid obsolete data to be cached
* [Project] Rbl: Migrate to `checks`
* [Project] Rbl: Move config code outside of the plugin
* [Project] Ressurect empty prefilters as connection filters
* [Project] Support connection filters registration from Lua
* [Rework] Add final cleanup logic
* [Rework] Add preliminary support of hyperscan caching for re maps
* [Rework] Add stale cache removal
* [Rework] Clickhouse: Improve performance
* [Rework] Distinguish between strict config test mode
* [Rework] Furhter logging improvements
* [Rework] Milter_headers: improve extended_headers_rcpt support
* [Rework] Move parsers to a separate lua library
* [Rework] Neural: Skip composite symbols
* [Rework] Rbl: Rework defaults logic
* [Rework] Some tunes to cache saving
* [Rework] Track maps origins
* [Rework] Use full crypto hash for regexp maps
* [Rules] Remove broken rule

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Add missing symbols
* [Conf] Add missing symbols
* [Conf] Fix fat-fingers typo
* [Conf] Fix wrong comment in options.inc
* [Conf] Neural: Fix the default name for max_trains
* [Conf] Register a known symbol
* [Conf] Spf: Add R_SPF_PERMFAIL symbol
* [CritFix] Arc: Fix ARC validation for chains of signatures
* [CritFix] Distinguish socketpairs between different fuzzy workers
* [CritFix] Fix IDNA dots parsing
* [CritFix] Fix test assertion method
* [CritFix] Fix usage of crypto_sign it should be crypto_sign_detached!
* [Feature] Add BOUNCE rule
* [Feature] Add controller plugins support and selectors plugin
* [Feature] Add maps query method
* [Feature] Add minimal delay to fuzzy storage
* [Feature] Add multiple base32 alphabets for decoding
* [Feature] Add preliminary support of BCH addresses
* [Feature] Add query_specific endpoint
* [Feature] Allow multiple base32 encodings in Lua API
* [Feature] Allow to specify nonces manually
* [Feature] Controller: Allow to pass query arguments to the lua webui plugins
* [Feature] Fuzzy_check: Add gen_hashes command
* [Feature] Fuzzy_check: Add weight_threshold option for fuzzy rules
* [Feature] Implement address retry on connection failure
* [Feature] Improve limits in pdf scanning
* [Feature] Initial support of subscribe command in lua_redis
* [Feature] Lua_cryptobox: Add secretbox API
* [Feature] Lua_text: Add encoding methods
* [Feature] Milter_headers: Allow to activate routines via users settings
* [Feature] PDF: Add timeouts for expensive operations
* [Feature] Preliminary maps addon for controller
* [Feature] Split pdf processing object and output object to allow GC
* [Feature] Support BLIS blas library
* [Feature] Support input vectorisation by recvmmsg call
* [Feature] Support multiple base32 alphabets
* [Feature] add queueid, uid, messageid and specific symbols to selectors [Minor] use only selectors to fill vars in force_actions message
* [Feature] allow variables in force_actions messages
* [Feature] extend lua api
* [Fix] #3249
* [Fix] Allow to adjust neurons in the hidden layer
* [Fix] Another try to fix email names parsing
* [Fix] Arc: Allow to reuse authentication results when doing multi-stage signing
* [Fix] Arc: Fix bug with arc chains verification where i>1
* [Fix] Arc: Sort headers by their i= value
* [Fix] Change neural plugin's loss function
* [Fix] Deal with double eqsigns when decoding headers
* [Fix] Default ANN names in clickhouse
* [Fix] Disable reuseport for TCP sockets as it causes too many troubles
* [Fix] Disable text detection heuristics for encrypted parts
* [Fix] Distinguish DKIM keys by md5
* [Fix] Distinguish type from flags in register_symbol
* [Fix] Dmarc: Unbreak reporting after cf2ae3292a
* [Fix] Do not flag pre-result of virus scanners as least if action is reject
* [Fix] Do not use GC64 workaround on 32bit platforms, omg
* [Fix] Exclude damaged urls from html parser
* [Fix] Fix FREEMAIL_REPLYTO_NEQ_FROM_DOM
* [Fix] Fix FROM_NEQ_ENVFROM
* [Fix] Fix FWD_GOOGLE rule (#1815)
* [Fix] Fix adding of the empty archive file for gzip
* [Fix] Fix aliases in forged recipients and limit number of iterations
* [Fix] Fix authentication results insertion
* [Fix] Fix calling of methods in selectors
* [Fix] Fix clen length for hiredis...
* [Fix] Fix endless loop if broken arc chain has been found
* [Fix] Fix false - operation
* [Fix] Fix get_urls table invocation
* [Fix] Fix group based composites
* [Fix] Fix headers passing in rspamd_proxy
* [Fix] Fix incomplete utf8 sequences handling
* [Fix] Fix lua_next invocation
* [Fix] Fix lua_parse_symbol_type function logic
* [Fix] Fix multiple listen configuration
* [Fix] Fix occasional encryption of the cached data
* [Fix] Fix parsing boundaries with spaces
* [Fix] Fix passing of methods arguments
* [Fix] Fix poor man allocator algorithm
* [Fix] Fix regexp selector and add flattening
* [Fix] Fix rfc base32 encode ordering (skip inverse bits)
* [Fix] Fix rfc based base32 decoding
* [Fix] Fix sockets leak in the client
* [Fix] Fix storing of the original smtp from
* [Fix] Fix types check and types usage in lua_cryptobox
* [Fix] Fix unused results
* [Fix] Fuzzy_check: Disable shingles for short texts (really)
* [Fix] Ical: Fix identation grammar
* [Fix] Improve part:is_attachment logic
* [Fix] Mmap return value must be checked versus MAP_FAILED
* [Fix] One more fix to skip images that are not urls
* [Fix] Pdf: Support some weird objects with no newline before endobj
* [Fix] Rbl: Fix ignore_defaults in conjunction with ignore_whitelists
* [Fix] Restore support for `for` and `id` parts in received headers
* [Fix] Segmentation fault in contrib/lua-lpeg/lpvm.c on ppc64el
* [Fix] Skip spaces at the boundary end
* [Fix] Slashing fix: fix captures matching API
* [Fix] Spamassassin: Rework metas processing
* [Fix] Store reference of upstream list in upstreams objects
* [Fix] Understand utf8 in content-disposition parser
* [Fix] Unify selectors digest functions
* [Fix] Use `abs` value when checking composites
* [Fix] Use strict IDNA for utf8 DNS names + add sanity checks for DNS names
* [Fix] Use unsigned char and better support of utf8 in ragel parser
* [Fix] add missing selector_cache declaration
* [Project] Add `L` flag for regexps to save start of the match in Hyperscan
* [Project] Add `lower` method to lua_text
* [Project] Add a simple matrix Lua library
* [Project] Add implicit bitcoincash prefix
* [Project] Add linalg ffi library for prototyping
* [Project] Add methods to append data to fuzzy requests
* [Project] Add routine to call a generic lua function
* [Project] Add ssyev method interface
* [Project] Add tensors index method
* [Project] Add text:sub method
* [Project] Allow rspamd_text based selectors
* [Project] Allow to specify re_conditions for regular expressions
* [Project] Attach extensions to the binary fuzzy commands
* [Project] Bitcoin: BTC cash addresses needs some checksum validation
* [Project] Cleanup the redis script
* [Project] Convert bitcoin rules to the new regexp conditions feature
* [Project] Detect memrchr in systems that supports it
* [Project] Do not listen sockets in the main process
* [Project] Implement 'probabilistic' learn mode for ANN
* [Project] Implement BTC polymod in C as it requires 64 bit ops
* [Project] Implement bitcoin cash validation in a proper way
* [Project] Implement extensions logic for fuzzy storage
* [Project] Implement symbols insertion in multiple results mode
* [Project] Lua_text: Add method memchr
* [Project] Neural: Add PCA loading logic
* [Project] Neural: Fix PCA based learning
* [Project] Neural: Fix matrix gemm
* [Project] Neural: Further PCA fixes
* [Project] Neural: Implement PCA in learning
* [Project] Neural: Implement PCA learning
* [Project] Neural: Implement PCA on ANN forward
* [Project] Neural: Implement PCA serialisation
* [Project] Neural: Start PCA implementation
* [Project] Neural: Use C version of scatter matrix producing
* [Project] Preliminary support of lua conditions for regexps
* [Project] Preliminary usage of the reuseport
* [Project] Process composites separately for each shadow result
* [Project] Remove old code
* [Project] Rework scan result functions to support shadow results
* [Project] Rework some more functions to work with shadow results
* [Project] Some more fixes
* [Project] Start results chain implementation
* [Project] Support fun iterators on rspamd_text objects
* [Project] Support multiply, minus and divide operators in expressions
* [Project] Tensor: Move scatter matrix calculation to C
* [Rework] Allow to specify exat metric result when adding a symbol
* [Rework] Change and improve openblas detection and usage
* [Rework] Close listen sockets in main after fork
* [Rework] Further rework of lua urls extraction API
* [Rework] Lua_cryptobox: Allow to store output of the hash function
* [Rework] Lua_task: Add more methods to deal with shadow results
* [Rework] Modernize logging for expressions
* [Rework] Remove empty prefilters feature - we are not prepared...
* [Rework] Remove old FindLua module, disable lua fallback when LuaJIT is enabled
* [Rework] Rework and refactor forged recipients plugin
* [Rework] Rework expressions processing
* [Rework] Rework fuzzy commands processing
* [Rework] Rework url flags handling API
* [Rework] Rework urls extraction
* [Rework] Split operations processing and add more debug logs
* [Rework] Update zstd to 1.4.5
* [Rework] Use google-ced instead of libicu chardet as the former sucks
* [Rework] add alias util:parse_addr for util:parse_mail_address
* [Rework] get rid of util:parse_addr duplicating the util:parse_mail_address, replace where used
* [Rules] Allow prefix for bitcoin cash addresses
* [Rules] More fixes for bitcoin cash addresses decoding
* [Rules] Refactor bleach32 addresses handling

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Mark Rspamd emailbl as ignore whitelist
* [Conf] RBL: Add missing emails = true option
* [Feature] Add support for scripts in fuzzy storage
* [Feature] Arc: Add whitelisted_signers_map option
* [Feature] Implement hosts file processing
* [Feature] Neural: Introduce classes bias that allows non-equal classes learning
* [Feature] Update libev to 4.33
* [Fix] Another brain damage html standard adoptions
* [Fix] Another fix for brain damaged obs-fws state
* [Fix] Fix flags that caused force_actions failure
* [Fix] Fix logging issue
* [Fix] Fix lua symbols scores registration when config does not define scores
* [Fix] Fix opaque maps logic
* [Fix] Fix parsing of the html tags with no spaces after attributes
* [Fix] Fix some corner cases in urls parsing, add limits
* [Fix] Fix tlds extraction if custom composition rules are used
* [Fix] Fix variables replacement in mempool
* [Fix] Improve base64 detection
* [Fix] Normalize dynamic scores in ANN correctly
* [Fix] Plug memory leak introduced by #3153
* [Fix] Stat_redis_backend: Fix memory leak and simplify learn path
* [Fix] Try hard to deal with ghost workers
* [Fix] metadata_exporter default formatter
* [Rework] Change the way to extract URLs when dealing with alternative parts
* [Rework] Fix various url extraction issues
* [Rework] Re cache: Load compiled hyperscan in the main process as well
* [Rework] Re cache: Load hyperscan early
* [Rework] Rework URL structure: adjust tld part
* [Rework] Rework URL structure: host field
* [Rework] Rework URL structure: more structure optimisations
* [Rework] Rework URL structure: user field
* [Rework] URL: Another update for urls extraction logic
* [Rework] Urls: Improve query urls handling
* [Rework] Urls: adopt html related stuff
* [Rework] Urls: more rework of the urls sets
* [Rework] Urls: process query urls in HTML urls correctly
* [Rework] Urls: rework urls hash structure
* [Rework] Urls: update lua libraries
* [Rework] Use multiple search tries for different url extraction types

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [CritFix] Fix parsing of the content type attributes
* [Feature] Clickhouse: Add extra columns support
* [Feature] Rbl: Add url_compose_map option for RBL rules
* [Fix] 'R' flag is for all headers regexp
* [Fix] Allow to reset settings id from Lua (e.g. because of the priority)
* [Fix] Avoid collisions in mempool variables by changing fuzzy caching logic
* [Fix] Avoid strdup usage for symbols options
* [Fix] Do not trust stat(2) it lies
* [Fix] Filter all options for symbols to have sane characters
* [Fix] Fix all headers iteration
* [Fix] Fix allowed_settings for neural
* [Fix] Fix listen socket parsing
* [Fix] Fix maps expressions evaluation
* [Fix] Fix sentinel connections leak by using async connections
* [Fix] Fix smtp message on passthrough result
* [Fix] Fix tld compositon rules
* [Fix] Fuzzy_storage: Do not check for shingles if a direct hash has been found
* [Fix] Lua_mime: Do not perform QP encoding for 7bit parts
* [Fix] Neural: Distinguish missing symbols from symbols with low scores
* [Fix] Support listening on systemd sockets by name
* [Project] Add lua_urls_compose library
* [Project] Allow to set a custom log function to the logger
* [Project] CDB maps: Start making cdb a first class citizen
* [Project] Clickhouse: Add extra columns concept
* [Project] Fix urls composition rules, add unit tests
* [Project] Unify cdb maps
* [Rework] Logger infrastructure rework
* [Rework] Refactor libraries structure
* [Rework] Rework SSL caching
* [Rework] Update snowball stemmer to 2.0 and remove all crap aside of UTF8

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] SPF is no longer a C module
* [Conf] Update spamtrap map path example
* [CritFix] Fix html entities decoding
* [CritFix] Fix re cache when mix of pcre and hyperscan is used
* [Feature] Allow milter code to deal with multiple headers
* [Feature] Antivirus: Add avast support
* [Feature] Dkim_signing: Allow to sign via milter_headers
* [Feature] Implement content hashes
* [Feature] Lua_text: Add regexp split iterator method
* [Feature] Lua_text: Implement flattening of the input tables
* [Feature] Send quit command to Redis
* [Feature] Speed up is_ascii function
* [Feature] Spf: Add external_relay option
* [Fix] Avoid double escaping
* [Fix] Fix O(N^2) algorithm
* [Fix] Fix arc seal validation
* [Fix] Fix base tag processing according to stupid HTML renderer behaviour
* [Fix] Fix dealing with `\0` in ucl strings and JSON
* [Fix] Fix gpg parts misdetection
* [Fix] Fix ignored symbols exporting
* [Fix] Fix processing of numeric url's
* [Fix] Fix processing of the closed tcp connections
* [Fix] Fix regexp type check for pcre2
* [Fix] Fix urls encode function
* [Fix] Fix urls shifting when doing decode to include separators
* [Fix] Fix white on white rule and add is_leaf flag
* [Fix] Further fixes in charset detection
* [Fix] Ignore diacritics in chartable module for specific languages
* [Fix] Limit size of symbols options by max_opts_len option
* [Fix] More fixes in html tag content calculations
* [Fix] Plug memory leak in fuzzy storage
* [Fix] Process high priority settings even if settings/id has been specified
* [Fix] Select a different upstream on last retransmit
* [Fix] Treat soft hyphen as zero width space
* [Fix] Try harder to watch the lifetime of the key_stat
* [Fix] Use ipv6-mapped-ipv4 addresses in radix trie
* [Project] Add logic to break execution when processing symbols*
* [Project] Add methods to set specific content for mime parts from Lua
* [Project] Lua_content: support PDF files
* [Project] Move dns_tool to using of the rspamd_spf from FFI module
* [Project] Preliminary SPF plugin in Lua
* [Project] Show debug stat for memory pool
* [Project] Some rework about specific data that is now tagged
* [Project] Start reworking of the mempool structure
* [Rework] Allow to add userdata as symbols options
* [Rework] Change mime part specifics handling
* [Rework] Move LRU SPF cache from spf plugin
* [Rework] Rework HTML tags content attachment
* [Rework] Rework options hash structure
* [Rework] Start lua_content library
* [Rework] Stop using of uthash for http headers
* [Rework] Use faster hashing approach for memory pools variables
* [Rules] Add PDF related rules

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Antivirus: Fix the default config
* [Feature] Add verdict library in lua
* [Feature] Allow exception when choosing upstream
* [Feature] Allow to disable symbols from the metric config
* [Feature] Allow to limit maps per specific worker
* [Feature] Always validate Rspamd protocol output
* [Feature] Antivirus: Add preliminary virustotal support
* [Feature] Clickhouse: Rework Clickhouse collection logic
* [Feature] Improve base64 usage
* [Feature] Shutdown timeout is now associated with task timeout
* [Fix] #3129 Multiple classifiers on redis working incorrectly
* [Fix] Allow real upstreams configuration
* [Fix] Another try to fix slow callbacks and timers
* [Fix] Check results of write message as SSL can bork them
* [Fix] Clickhouse: Avoid potential races in collection
* [Fix] Clickhouse: Fix periodic script
* [Fix] Fail DNS upstream on each retransmit attempt
* [Fix] Fix consistent hashing when upstreams are marked inactive
* [Fix] Fix issues found
* [Fix] Fix off-by-one in retries for the proxy
* [Fix] Fix termination
* [Fix] Fix upstreams exclusion logic
* [Fix] Fix utf8 validation for symbols options and empty strings
* [Fix] Oops, fix maps reload
* [Fix] Rbl: Allow utf8 lookups for IDN domains
* [Fix] Sigh, another try to fix brain-damaged openssl
* [Project] Add fast utf8 validation library
* [Project] Use own utf8 validation instead of glib
* [Rework] Another phase of finish actions rework
* [Rework] Further cmake system rework
* [Rework] Further isolation of the controller's functions
* [Rework] Make cmake structure more modular
* [Rework] Move cmake modules to a dedicated path
* [Rework] Replace controller functions by any scanner worker if needed
* [Rework] Rework final scripts logic
* [Rework] Rewrite rspamd_str_make_utf_valid function

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Update neural.conf
* [CritFix] Fix dkim verification for multiple headers listed
* [Feature] Add support of uudecode
* [Feature] Allow to explicitly set events backend
* [Feature] Implement configurable limits for SPF lookups
* [Feature] Lua_scanners: Use lua magic for inclusion/exclusion logic
* [Feature] Multimap: Do not check files in office archives
* [Feature] Neural: Add sampling when storing training vectors
* [Feature] SPF: Allow to disable AAAA checks in configuration
* [Feature] Spf: Add limits configuration support
* [Feature] Store etag in cached HTTP maps + better logging
* [Feature] Support segwit BTC addresses, fix LTC verification
* [Feature] Support uuencoding
* [Fix] Add configurable number of threads for OpenBLAS
* [Fix] Add workaround for ragel 7 in hyperscan related maps code
* [Fix] Another fix for numeric urls parsing
* [Fix] Correct EMA time calculations
* [Fix] Do not treat archives as text
* [Fix] Do not use strdup on data extracted from lua
* [Fix] Fix a failure calcuating URL reputation.
* [Fix] Fix crash due to constructors init order
* [Fix] Fix crash on parts with no cd
* [Fix] Fix empty prefilters that require mime structures
* [Fix] Fix event loop creation
* [Fix] Fix issues sending DMARC reports.
* [Fix] Fix misprint
* [Fix] Fix saving of the file maps
* [Fix] Fix size calculations when converting from utf16
* [Fix] Fix support of disable_monitoring in rbl
* [Fix] Fix use-after-free
* [Fix] Fix zip files check to relax requirements
* [Fix] Important hiredis fixes
* [Fix] Lot's of fixes in maps check logic
* [Fix] Lua_tcp: Deal with temporary fails on write
* [Fix] Lua_tcp: Make write errors fatal and rework error handlers
* [Fix] Meta: Filter some more values
* [Fix] Neural: Add protection agains infinities
* [Fix] Oops, fix math.huge invocation
* [Fix] Plug memory leak
* [Fix] Sigh, another email to string fix
* [Fix] Try to fix another ownership race in ssl connection
* [Fix] Uuencode: Fix parsing of corrupted uuencode
* [Fix] lua_scanners - razor rename need_check function
* [Rework] Require CMake 3.9 to work, remove manual lto crap

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Add BROKEN_HEADERS_MAILLIST composite
* [Conf] Add path to greylist-whitelist-domains.inc
* [Conf] Clarify documentation in the config files
* [Conf] Introduce maps.d directories
* [Conf] Log settings id by default
* [Conf] Make LEAKED_PASSWORD_SCAM a composite rule again
* [Conf] Move all surbl/emails rules to rbl
* [Conf] Register new Spamhaus codes
* [Conf] Remove configs for deleted modules
* [Conf] Remove surbl parts, fix hash_format attribute
* [Conf] Show autolearn sample
* [Conf] Slashing: Change default stats backend to Redis
* [Conf] Surbl: Utilise new `check_emails` option
* [Conf] Update header
* [Conf] Use multi-prefixes RBLs in the default config
* [CritFix] Deal with case-sensivity in Content-Disposition parser
* [CritFix] Eliminate old endpoint
* [CritFix] Fix case sensivity when parsing Content-Type
* [CritFix] Fix loading of DKIM public keys
* [CritFix] Fix procesing of urls
* [CritFix] Fix whitelisting when both spf and dkim are required to be valid
* [CritFix] Langdet: Fix language detection where no stop words found
* [Feature] Add description to the groups
* [Feature] Add limit for number of URLs in Lua
* [Feature] Add logging of groups to the log_format
* [Feature] Add lua_smtp library
* [Feature] Add maps cache and type refinement
* [Feature] Add p0f scanner
* [Feature] Adopt emails module to use lua_maps
* [Feature] Allow options matching in composites
* [Feature] Allow selectors in rbl module
* [Feature] Allow to output group results
* [Feature] Asn: Allow to use bgpdump when NET::MRT is broken
* [Feature] Calculate tokens occurrences distribution
* [Feature] Clickhouse: Add authenticated user and settings id columns
* [Feature] Clickhouse: Store groups data
* [Feature] Clickhouse: Utilise LowCardinality feature
* [Feature] Implement Redis prefixes registration logic
* [Feature] Implement settings id propagation between deps
* [Feature] Improve AV results caching
* [Feature] Improve autolearning
* [Feature] Improve logging locking logic (remove it actually)
* [Feature] Improve settings processing
* [Feature] Langdet: Limit number of stop words to be checked
* [Feature] Libucl: Allow to sort keys in ucl objects
* [Feature] Lua_config: Extend get symbols method
* [Feature] Lua_maps: Allow static maps for key-value pairs
* [Feature] Lua_mimepart: Add function filter_words
* [Feature] Lua_selectors: Add `words` selector
* [Feature] Lua_selectors: Add sort and uniq transform functions
* [Feature] Lua_selectors: Allow table arguments for selectors
* [Feature] Lua_tcp: Add preliminary support of SSL connections
* [Feature] Lua_trie: More flexible API
* [Feature] Lua_util: Add filter_specific_url function
* [Feature] Lua_util: table_digest can now recursively traverse tables
* [Feature] Maillist: Improve detection
* [Feature] Maps: Allow caching for complex maps
* [Feature] Monitored: Support random lookups
* [Feature] Multimap: Add combined maps prototype
* [Feature] Multimap: Add dependend maps via redis keys selectors
* [Feature] Multimap: Allow multiple email addresses matches
* [Feature] Multimap: Also check detected charset when do filename checks
* [Feature] Output number of messages processed to proctitle
* [Feature] Perform clean SSL shutdown
* [Feature] Performance: Do not use base64 SIMD version for bad inputs
* [Feature] RBL: Support bit results in replies
* [Feature] RBL: Support type specific prefixes
* [Feature] Ratelimit: Consider number of SMTP recipients
* [Feature] Rbl: Add ability to check urls
* [Feature] Rbl: Add resolve_ip based RBLs
* [Feature] Rbl: Make config checks much more strict
* [Feature] Rbl: Support per-rule whitelists
* [Feature] Rbl: Support process script
* [Feature] Rbl: Support replyto addresses
* [Feature] SURBL: Allow to check email domains
* [Feature] Selectors: Add `list` generator
* [Feature] Selectors: Add `specific_urls` extractor
* [Feature] Selectors: Add flatten function
* [Feature] Selectors: Support filter_map and apply_map functions
* [Feature] Store Clickhouse data outside of lua alloc
* [Feature] Support caching for encrypted files and macros
* [Feature] Support images when extracting urls
* [Feature] Support more hyperscan flags
* [Feature] Support protocol flags
* [Feature] URL: Apply stringprep to hostnames to filter garbage
* [Feature] Upstreams: Add lazy resolving logic to all upstreams
* [Feature] Upstreams: Set noresolve flag on numeric upstreams
* [Feature] Use `scores` in apply section
* [Feature] Use maps logic from lua_maps for multimap
* [Feature] Use random monitored in rbl module
* [Feature] lua_scanners - add Razor support
* [Fix] Add another safe-guard in urls processing
* [Fix] Add debug to ssl, fixed write hangs
* [Fix] Add missing groups to C callback symbols
* [Fix] Add more checks for ghosts symbols
* [Fix] Allow to enable or add new actions via settings
* [Fix] Allow to set 0 size for spf/dkim caches
* [Fix] Another bunch of fixes towards protocol mess
* [Fix] Another fix to deal with bad URLs
* [Fix] Arc: Another bunch of fixes for arc signing
* [Fix] Arc: More arc signing fixes
* [Fix] Avoid another overflow in fpconv
* [Fix] Clickhouse: Fix quoting
* [Fix] Clickhouse: Fix retention query quoting
* [Fix] Distinguish empty and non-empty prefilters
* [Fix] Distinguish remote and local addrs parsing
* [Fix] Do not assert if length of sig is bad, just fail verification
* [Fix] Do not assert if we have broken mime boundary in the headers
* [Fix] Do not call implicit strlen to avoid issues
* [Fix] Do not count images urls when checking url regexps for compatibility
* [Fix] Do not output rbl suffix in symbol option
* [Fix] Do not use config pool to avoid issues with double reload
* [Fix] Do not use ephemeral string
* [Fix] Do not use lightuserdata for traceback
* [Fix] Do not use priority in metric registration
* [Fix] Emails: Check email sanity before testing on BL
* [Fix] Emails: Fix misprint in key name
* [Fix] Escape utf in regexp to dodge ragel/hyperscan issue
* [Fix] Extend task_timeout to postfilters stage
* [Fix] Fix ARC signing after fixing another bug in it...
* [Fix] Fix AV scan logic
* [Fix] Fix DMARC_NA behaviour in case of no valid policies
* [Fix] Fix LRU hash iteration logic
* [Fix] Fix alignment mess
* [Fix] Fix configuring symbols without scores
* [Fix] Fix disabling of the actions
* [Fix] Fix dkim signing exceptions
* [Fix] Fix embedded images linking logic
* [Fix] Fix events leak
* [Fix] Fix eviction corner case
* [Fix] Fix fuzzy image score calculation #2962
* [Fix] Fix hang in fuzzy_learn when explicit rotation is set
* [Fix] Fix headers propagation logic
* [Fix] Fix hearbeats restart issue
* [Fix] Fix history reset
* [Fix] Fix log parameter
* [Fix] Fix lua_ip_equal logic
* [Fix] Fix more issues with nested messages + tests
* [Fix] Fix normalization of non-alphabet based languages
* [Fix] Fix offsets when parsing message/rfc822 in multipart
* [Fix] Fix options in rbl symbols
* [Fix] Fix out of bound access in lua logger
* [Fix] Fix out-of-bound read in qp decode
* [Fix] Fix parent CTE propagation
* [Fix] Fix parsing of the received headers with empty part
* [Fix] Fix pending checks for events
* [Fix] Fix printing of NULL pointer with fixed length
* [Fix] Fix race condition in watcher handler
* [Fix] Fix read-after-end in quoted printable decoding
* [Fix] Fix redis sentinel support
* [Fix] Fix registry leak in case of DNS errors
* [Fix] Fix reload logic
* [Fix] Fix sending of large entries via HTTPS
* [Fix] Fix settings reload
* [Fix] Fix some more corner cases for fpconv
* [Fix] Fix trie code when there are regexps and Hyperscan is absent
* [Fix] Further fixes to printing of the FP numbers
* [Fix] Fuzzy_check: Fix timeouts
* [Fix] Grrr, fix empty ip case
* [Fix] Html: Fix processing of fjlig entity
* [Fix] Lang_det: Try better to distinguish Chinese and Japanese
* [Fix] Lua_mime: Fix reversed extensions map
* [Fix] Lua_task: Fix message-less API
* [Fix] Lua_tcp: Report connection failures
* [Fix] Lua_tcp: Various fixes and debugging improvements
* [Fix] Metadata_exporter: This plugin is idempotent not a postfilter
* [Fix] More fixes to extract_specific_urls
* [Fix] More stages fixes
* [Fix] Neural: Another bunch of fixes
* [Fix] Neural: use version in ANN key profile
* [Fix] Postpone lua state destruction to allow lua dtors to be used
* [Fix] Prefer surbl/emails rule on rbl to preserve compatibility
* [Fix] RBL: Fix behaviour of emails_domainonly
* [Fix] Ratelimit: Fix dynamic score
* [Fix] Rbl: Fix emailbl functions
* [Fix] Really fix hyperscan workaround
* [Fix] Set sanity limits for pcre2
* [Fix] Settings: Fix settings check flags
* [Fix] Sort keys when getting data from Lua when filling rules
* [Fix] Statistics: Do not query Redis tokens when there are no learns
* [Fix] Stop IO event on write finished in http connection
* [Fix] Use heuristically detected text parts data
* [Fix] Various fixes to QP encoding algorithm
* [Fix] Various fixes to SSL state machine handler
* [Fix] Various fixes to asn module
* [Fix] Workaround for empty charset in rfc2231 encoding
* [Project] Switch from torch to KANN
* [Project] Add heartbeat events
* [Project] Add preliminary support of the Kaspersky Scan Engine
* [Project] Add preliminary version of maps expressions
* [Project] Add preprocessed settings to the config structure
* [Project] Add simple forward propagation function
* [Project] Add small helpers for migration simplifications
* [Project] Allow to replace body in milter
* [Project] Bundle libev
* [Project] First refactoring step libevent->libev
* [Project] Implement syntax highlighting for Lua
* [Project] Lua_magic: Adopt lua_magic stuff in mime_types
* [Project] Remove libfann, gd and other unsupported stuff
* [Project] Remove torch
* [Project] Rework upstreams
* [Rework] Allow execution of async events when hs compiles regexps
* [Rework] Bayes expiry: eliminate `default` expiration mode
* [Rework] Dkim: Remove signing code
* [Rework] Dkim_signing: Move sign condition to dkim_signing
* [Rework] Do not lowercase all data send to ClickHouse
* [Rework] Drop url tags
* [Rework] Eliminate lua_squeeze as it has shown no improvements
* [Rework] Eliminate virtual scan time as it is useless
* [Rework] Lua core: Use lightuserdata to index classes
* [Rework] Lua_util: Another rework for extract_specific_urls
* [Rework] Migrate from ip_score to reputation
* [Rework] Move mime modification functions to lua_mime library
* [Rework] Rbl: Major whitelisting logic rework
* [Rework] Remove deprecated plugins
* [Rework] Remove log helper worker
* [Rework] Remove rspamd.classifiers.lua
* [Rework] Rename filter.h to a more sane name
* [Rework] Reorganise selectors implementation
* [Rework] Replace linenoise with replxx
* [Rework] Reputation: Remove ipnet from the ip reputation
* [Rework] Reputation: Slashing - change name of symbols
* [Rework] Rework children operations
* [Rework] Rework config reload
* [Rework] Rework expression API
* [Rework] Rework image urls processing
* [Rework] Rework initialisation to reduce static leaks count
* [Rework] Rework request headers processing
* [Rework] Slashing: Change versioning schema - move to 2.0
* [Rework] Slashing: Turn off postfilters when passthrough result is set
* [Rework] Start moving to replxx
* [Rework] Stop support of signed HTTP maps to simplify code
* [Rework] Store ASN as UInt32 in ClickHouse
* [Rework] Url_redirector: Rewrite plugin
* [Rework] Use a dedicated library for autolearn
* [Rework] Use libsodium instead of hand crafted crypto implementations
* [Rework] Use opaque structure to store a table of mime headers
* [Rules] Add dedicated bitcoin addresses filter rule
* [Rules] Add more detection to LEAKED_PASSWORD_SCAM
* [Rules] Catch LTC addresses
* [Rules] Reduce weight of RSPAMD_EMAILBL
* [Rules] Rework LEAKED_PASSWORD_SCAM rule one more time

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Add IP_SCORE_FREEMAIL composite rule
* [Feature] Add cryptobox method to generate dkim keypairs
* [Feature] Add fast hashes to lua cryptobox hash
* [Feature] Add least passthrough results
* [Feature] Allow oversign if exists mode
* [Feature] Clickhouse: Modernise table initial schema
* [Feature] Implement IUF interface for specific fast hashes
* [Feature] Lua_util: Allow to obfuscate different fields
* [Feature] Tune memory management in Rspamd and Lua
* [Fix] Avoid buffer overflow when printing long lua strings
* [Fix] Change the default oversigning headers to a more sane list
* [Fix] Clickhouse: Do not store digest as it is not needed now
* [Fix] Clickhouse: Fix lots of storage issues
* [Fix] Clickhouse: Support custom actions
* [Fix] Deny URLs where hostname is bogus
* [Fix] Do not blacklist mail by SPF/DMARC for local/authed users
* [Fix] Fix DoS caused by bug in glib
* [Fix] Fix UCL parsing of the multiline strings
* [Fix] Fix buffer overflow when printing small floats
* [Fix] Fix init code for servers keypairs cache
* [Fix] Fix issue with urls with no tld (e.g. IP)
* [Fix] Fix memory in arc signing logic
* [Fix] Fix memory leak in language detector during reloads
* [Fix] Fix mixed case content type processing
* [Fix] Fix processing of the ip urls in file
* [Fix] Fix use after free
* [Fix] HTML: Fix `size` attribute processing
* [Fix] Hum, it seems that 99ff1c8 was not correct
* [Fix] Lua_task: Fix task:get_from method
* [Fix] Preserve fd when mapping file to scan
* [Fix] Re-use milter_headers settings when doing arc signing
* [Fix] Set dmarc force action as least action
* [Fix] Switch to GMT
* [Fix] allow PKCS7 signatures to be text/plain, too
* [Project] Add initial version of the vault management tool
* [Project] Add vault support for DKIM and ARC signing
* [Project] Implement keys rotation in the vault
* [Project] Improve dkim keys generation for vault
* [Project] Improve keys creation in rspamadm vault
* [Rework] Move lua_worker to a dedicated unit
* [WebUI] Add URL fragments (#) support
* [WebUI] Fix AJAX request URL

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Allow to load users plugins from plugins.d
* [Conf] oversign openpgp and autocrypt headers
* [Feature] Add SPF FFI library for Lua
* [Feature] Add more verbosity for SPF caching
* [Feature] Antivirus: Handle encrypted files specially
* [Feature] Clickhouse: Slashing - add new fields to CH
* [Feature] Dkim_signing: Add OpenDKIM like signing_table and key_table
* [Feature] Dkim_signing: Allow to use new options as maps
* [Feature] Import fpconv library
* [Feature] Lua_maps: Allow static regexp and glob maps
* [Feature] Parse ical files
* [Feature] Rspamadm: Add dns_tool utility
* [Feature] Store SPF records digests
* [Feature] Use fpconv girsu2 implementation for printing floats
* [Fix] Clickhouse: Use integer seconds when inserting rows
* [Fix] Fix floating point printing
* [Fix] Fix processing of embedded urls
* [Fix] Lua_clickhouse: Fix CH errors processing
* [Fix] Make spf digest stable
* [Fix] Properly detect encrypted files in zip archives
* [Fix] Slashing: Store times in GMT timezone in ClickHouse
* [Rules] Add additional conditions to perform BTC checks
* [Rules] Fix pay-to-hash addresses validation

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Add vendor groups for symbols
* [Feature] Add `rspamadm template` command
* [Feature] Allow to add messages from settings
* [Feature] Allow unconnected DNS servers operations
* [Feature] Check limits after being set, migrate to uint64
* [Feature] Greylist: Allow to disable greylisting depending on symbols
* [Feature] Improve lua binary strings output
* [Feature] Mime_types: Implement user configurable extension filters
* [Feature] Mime_types: When no extension defined, detect it by content
* [Feature] Preprocess config files using jinja templates
* [Feature] Replies: Filter replies sender to limit whitelisting to direct messages
* [Feature] Treat all tags with HREF as a potential hyperlinks
* [Feature] Validate BTC addresses in LEAKED_PASSWORD_SCAM
* [Fix] Add crash safety for HTTP async routines
* [Fix] Another fix for Redis sentinel
* [Fix] Clickhouse: Fix table schema upload
* [Fix] Core: Fix squeezed dependencies handling for virtual symbols
* [Fix] Finally fix default parameters parsing in actions section
* [Fix] Fix ES sending logic (restore from coroutines mess)
* [Fix] Fix finishing script for clickhouse collection
* [Fix] Fix priority for regexp symbols registriation
* [Fix] Fix various issues found by PVS Studio
* [Fix] Initialize lua debugging earlier
* [Fix] Neural: Fix training
* [Fix] Rework cached Redis logic to avoid sentinels breaking
* [Fix] SURBL: Fix regression in surbl module
* [Fix] Fix double signing in the milter
* [Project] Add support of HTTP proxy in requests
* [Rework] Change lua global variables registration
* [Rework] Rework HTML content urls extraction
* [Rework] Start rework of aliasing in Rspamd
* [WebUI] Combine Scan and Learning into one tab
* [WebUI] Fix symbol score input type
* [WebUI] Show grayed out pie
* [WebUI] Update Throughput summary values dynamically

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Add missing includes
* [Conf] Move to options
* [Conf] Rbl: DWL is actually special whitelist
* [Conf] Relax some uribl rules
* [Conf] Remove abuse.ch
* [CritFix] Html: Entities are not valid within tag params values
* [Feature] Add `rspamadm mime sign` tool
* [Feature] Add configgraph utility
* [Feature] Add dedicated ZW spaces detection for URLs
* [Feature] Add flag to url object when visible part is url_like
* [Feature] Add method task:lookup_words
* [Feature] Add pyzor support (by crosenberg)
* [Feature] Allow to add upstream watchers to Lua API
* [Feature] Allow to set rewrite subject pattern from settings
* [Feature] Better escaping of unicode
* [Feature] Clickhouse: Allow to store subject in Clickhouse
* [Feature] Core: Add QP encoding utility
* [Feature] Core: Add libmagic detection for all parts
* [Feature] Core: Add support for gzip archives
* [Feature] Core: Allow to construct scan tasks from raw data
* [Feature] Core: Detect charset in archived files
* [Feature] Core: Ignore and mark invisible spaces
* [Feature] Core: Normalise zero-width spaces in urls
* [Feature] Core: Process data urls for images
* [Feature] Core: Relax quoted-printable encoding
* [Feature] Core: Support RFC2231 encoding in headers
* [Feature] Core: Support telephone URLs
* [Feature] Core: allow to emit soft reject on task timeout
* [Feature] DCC: Add bulkness and reputation checks to dcc
* [Feature] Elastic: Modernize plugin
* [Feature] Export visible part of url to lua
* [Feature] Fuzzy_storage: add preliminary support of rate limits
* [Feature] HTML: Specially treat data urls in HTML
* [Feature] Implement event watchers for upstreams
* [Feature] Implement includes tracing in Lua
* [Feature] Improve dkim part in configwizard
* [Feature] Lua_scanners: Add VadeSecure engine support
* [Feature] Lua_task: Add flexible method to get specific urls
* [Feature] Mime_types: Add MIME_BAD_UNICODE rule
* [Feature] Mime_types: Use detected content type as well
* [Feature] Plugins: Add preliminary version of the external services plugin
* [Feature] Query sentinel on master errors
* [Feature] Regexp: Allow local lua functions in Rspamd regexp module
* [Feature] Rspamadm: Allow to append footers to plain messages
* [Feature] Rspamadm: Allow to rewrite headers in messages
* [Feature] Selectors: Add `ipmask` processor
* [Feature] Settings: Allow hostname match
* [Feature] Settings: Allow local when selecting settings
* [Feature] Settings: Allow multiple selectors
* [Feature] Settings: Allow to inverse conditions
* [Feature] Support User-Agent in HTTP requests
* [Feature] Support ed25519 dkim keys generation
* [Feature] Try to filter bad unicode types during normalisation
* [Feature] external_services - oletools (olefy) support
* [Feature] lua_scanners - icap protocol support
* [Feature] lua_scanners - spamassassin spam scanner
* [Fix] Add filter for absurdic URLs
* [Fix] Add some more cases for Received header
* [Fix] Allow to disable/enable composite symbols
* [Fix] Arc: Use a separated list of headers for arc signing
* [Fix] Archive: Final fixes for 7z archives
* [Fix] Clickhouse: Fix database usage
* [Fix] Controller: Make save stats timer persistent
* [Fix] Core: Detect encrypted rarv5 archives
* [Fix] Core: Don't detect language twice
* [Fix] Core: Fix address rotation bug
* [Fix] Core: Fix content calculations for message parts
* [Fix] Core: Fix emails comments parsing and other issues
* [Fix] Core: Fix etags support
* [Fix] Core: Fix headers folding on the last token
* [Fix] Core: Fix iso-8859-16 encoding
* [Fix] Core: Fix log_urls flag (and encrypted logging)
* [Fix] Core: Fix part length when dealing with boundaries
* [Fix] Core: Fix parts distance calculations
* [Fix] Core: Fix processing of NDNs of certain type
* [Fix] Core: Implement logic to find some bad characters in URLs
* [Fix] Core: treat nodes with ttl properly in lru cache
* [Fix] Fix Content-Type parsing
* [Fix] Fix HTTP headers signing case
* [Fix] Fix control interface
* [Fix] Fix deletion of the duplicate headers
* [Fix] Fix emails filtering in emails module
* [Fix] Fix greylisting log message and logic
* [Fix] Fix issues with storing of the accepted addr in rspamd control
* [Fix] Fix maps object update race condition
* [Fix] Fix memor leaks and whitespace processing
* [Fix] Fix processing of null bytes in headers
* [Fix] Fix rcpt_mime and from_mime in user settings
* [Fix] Fix rfc2047 decoding for CD headers
* [Fix] Fix rfc2231 for Content-Disposition header
* [Fix] Fix setting of the subject pattern in config
* [Fix] Greylist: fix records checking
* [Fix] HTML: Another HTML comments exception fix
* [Fix] HTML: Another entities decoding logic fix
* [Fix] HTML: Fix HTML comments with many dashes
* [Fix] HTML: Fix entities in HTML attributes
* [Fix] HTML: Fix some more SGML tags issues
* [Fix] Ignore whitespaces at the end of value in DKIM records
* [Fix] MID module: Fix DKIM domain matching
* [Fix] Milter_headers: Fix remove_upstream_spam_flag and modernise config
* [Fix] Mime_parser: Fix issue with parsing of the trailing garbadge
* [Fix] Mime_parser: Fix parsing of mime parts without closing boundary
* [Fix] Multimap: Fix operating with userdata
* [Fix] Process orphaned `symbols` section
* [Fix] Rdns: Fix multiple replies in fake replies
* [Fix] Rework groups scores definitions
* [Fix] Set proper element when reading data from Sentinel
* [Fix] Set rspamd user to initialise supplementary groups on reload
* [Fix] Settings: Fix selectors usage
* [Fix] Sort data received from Sentinel to avoid constant replacing
* [Fix] groups.conf - filename typo
* [Fix] lua_scanner - oletools typos, logging
* [Fix] lua_scanners - actions and symbol_fail
* [Fix] lua_scanners - fix luacheck
* [Fix] lua_scanners - kaspersky - response with fname
* [Fix] lua_scanners - savapi redis prefix
* [Fix] tests - antivirus - fprot symbols
* [Project] Add concept of flexible actions
* [Project] Add heuristical from parser to received parser
* [Project] Add new flags to clickhouse, redis and elastic exporters
* [Project] Attach new received parser
* [Project] Fallback to callbacks from coroutines
* [Project] Implement keep-alive support in lua_http
* [Project] Lua_udp: Implement fully functional client
* [Project] Plug keepalive knobs into http connection handling
* [Project] Rspamadm: Add `modify` tool
* [Rework] Convert rspamd-server to a shared library
* [Rework] Dcc: Rework DCC plugin
* [Rework] Enable explicit coroutines symbols
* [Rework] Rework telephone urls parsing logic
* [Rework] Rewrite RBL module
* [Rework] Settings: Rework settings check
* [Rework] Slashing: Distinguish lualibdir, pluginsdir and sharedir
* [Rework] Unify task_timeout
* [Rework] Use VEX instructions in assembly, relocate
* [WebUI] Notify user if uploaded data was not learned
* [WebUI] Remove redundant condition

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [CritFix] Make flags mutually exclusive for mime parts
* [CritFix] Strictly deny unencoded bad utf8 sequences in headers
* [Feature] Add Kaspersky antivirus support
* [Feature] Add method to get dkim results
* [Feature] Add more words regexp classes
* [Feature] Allow to choose words format in `rspamadm mime`
* [Feature] Allow to get all types of words from Lua
* [Feature] Allow to get task flags in C expressions
* [Feature] Allow to require encryption when accepting connections
* [Feature] Ignore bogus whitespaces in the words
* [Feature] Implement more strict configuration tests
* [Feature] Improve SPF results in Authentication-Results
* [Feature] Support ClickHouse database
* [Fix] Add failsafety for utf8 regexps
* [Fix] Do not trigger BROKEN_CONTENT_TYPE on innocent text parts
* [Fix] Emit error if connection has been terminated with no stop pattern
* [Fix] Fix boundaries checks in embedded messages
* [Fix] Fix double free
* [Fix] Perform policy downgrade on sample out, add tests
* [Fix] Properly escape utf8 regexps in hyperscan mode
* [Fix] Selectors - attachments args condition
* [Fix] Some fixes for raw parts
* [Fix] Treat learning errors as non-fatal
* [Fix] Use tld when looking for DKIM domains
* [Project] Words unicode structure rework
* [Project] Add preliminary Redis Sentinel support
* [Project] Improve Authentication-Results header
* [Project] Rework DKIM checks results
* [Project] Use more generalised API to produce meta words

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Add DWL support in the default configuration
* [Conf] Disable rspamd_update by default (again)
* [Conf] Fix configuration sample for ratelimit
* [CritFix] Disable broken url tags by default
* [CritFix] Fix \0 processing when doing RSA sign
* [CritFix] Fix adding symbols to their primary groups
* [Feature] Add `rspamadm cookie` utility
* [Feature] Add specialised functions for generating encrypted cookies
* [Feature] Add support of cookies in replies module
* [Feature] Add support of words regexps
* [Feature] Allow to add 3rd party clang plugins
* [Feature] Allow to create lua regexps from glob or plain patterns
* [Feature] Allow to set custom limits for upstream lists
* [Feature] Detect orphaned parts and attach them to message
* [Feature] Filter tokens in bayes
* [Feature] Fold b= value when doing arc sealing
* [Feature] Ignore cookies in the future and too old in the past
* [Feature] Skip stop words in statistics
* [Feature] Store stop words and allow to query them
* [Feature] Support query arguments in controller's custom commands
* [Feature] Tune upstream limits in Rspamd proxy
* [Feature] Use different callback symbols for different uribls
* [Feature] Write DKIM selector in dkim allow/reject symbols
* [Fix] Add obs_fws state support to eoh state machine
* [Fix] Add sanity check when applying mime boundaries heuristic
* [Fix] Antivirus - virus names with 0 were recognized as tables
* [Fix] Disable headernames in bayes temporarily
* [Fix] Do not allow syntax errors in include files...
* [Fix] Do not allow to merge an object with an array (or vice versa)
* [Fix] Don't perform forged recipients check for missing recipients
* [Fix] Fix DKIM based RBLs
* [Fix] Fix actrie implementation (sync from upstream), fixed OOB read
* [Fix] Fix explicit methods call in selectors
* [Fix] Fix extraction of additional parts
* [Fix] Fix finalization for internal plugins
* [Fix] Fix override_defaults function
* [Fix] Fix squeezed symbols when using settings
* [Fix] Fix urls insertion in Clickhouse module
* [Fix] Furhter fixes to ratelimits logic
* [Fix] Ignore signatures when looking for boundaries
* [Fix] Properly set learned count
* [Fix] Really fix ratelimits configuration and work
* [Fix] Remove ambigious format flag from printf
* [Fix] Restore URLs exporting in ClickHouse plugin
* [Fix] Rework bayes calculations...
* [Fix] Switch from chi-square to naive for large Fisher value
* [Fix] Treat normal password as enable password if there is no enable password
* [Fix] Use proper syntax for making DNS requests
* [Fix] Various fixes in embedded plugins
* [Project] Change fuzzy check selection logic to lua_fuzzy library
* [Project] Rework async events and symbols
* [Project] Move all metatokens in Bayes to lua_stat from C
* [WebUI] Add history rows per page control

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [CritFix] Fix options insertion
* [CritFix] Fix words decay one more time (affects long messages)
* [CritFix] Increase default words_decay
* [CritFix] Plug memory leak in redis pool
* [Feature] Add `check_violation` feature to DKIM/ARC signing
* [Feature] Add only unique elements to Clickhouse url arrays
* [Feature] Allow `g+:` and `g-:` composite atoms
* [Feature] Allow dkim domains check in surbl
* [Feature] Allow maps with HTTP auth
* [Feature] Allow to disable actions by users settings
* [Feature] Extend whitelisting options
* [Feature] Store url object in images
* [Feature] Use verdict instead of the plain action in plugins
* [Fix] Allow to call fstring append with NULL string
* [Fix] DCC - luacheck
* [Fix] Do not load torch on each rspamadm invocation
* [Fix] Fix boundaries detection and rework stop words algorithm
* [Fix] Fix dependencies for DNS_SIGNED symbol
* [Fix] Fix errors when dealing with dynamic rates/bursts in Ratelimit
* [Fix] Fix groups mess
* [Fix] Fix groups mess
* [Fix] Fix parsing address with comments
* [Fix] Fix resolving in DMARC reports
* [Fix] Fix various issues with parsing of the received headers
* [Fix] Fix watchers issue in lua_tcp when doing no resolving
* [Fix] Plug memory leak in language detector (affects reloads)
* [Fix] Remove one letter stop words
* [Fix] Slashing: backport chunk logic from libucl
* [Fix] Stop libevent from using cached time in rspamadm
* [Fix] Try to fix watchers chaining
* [Fix] Various fixes in redis sync interface
* [Fix] ip_score - respect check_authed and check_local settings from config
* [Project] Rework passthrough actions
* [Project] Clustering module
* [Rework] Always create result for a task
* [Rework] Completely rewrite DMARC checks logic
* [Rework] Rework and fix whitelist plugin
* [WebUI] Add symbols sorting buttons
* [WebUI] Change symbols order without updating history
* [WebUI] Colorize symbols
* [WebUI] Do not display password form when secure_ip is set
* [WebUI] Fix symbol description tooltips display
* [WebUI] History: add sorting by symbol score value

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Feature] Add arguments schemas to processors and extractors
* [Feature] Add functional selectors library
* [Feature] Add generic selector to reputation module
* [Feature] Add more ratelimits: by digest, by attachments data, by filenames
* [Feature] Add preliminary stop words detection support
* [Feature] Add pure Lua debugm function
* [Feature] Add schema validation for Redis settings
* [Feature] Add selectors combine function
* [Feature] Add some recursion protection to lua logger
* [Feature] Add support for Lua API tracing
* [Feature] Allow to apply schema to arguments
* [Feature] Allow to get dkim signing data directly from HTTP headers
* [Feature] Allow to reuse existing authentication results
* [Feature] Cache selectors results in re runtime
* [Feature] Implement new text tokenizer based on libicu
* [Feature] Integrate selectors framework to multimap
* [Feature] Relax FORGED_RECIPIENTS
* [Feature] Support (almost) all html entities
* [Feature] Support adding and deletion of recipients in the milter block
* [Feature] Support gathering HTTP body from fragments in lua_http
* [Feature] Support multi flag in regexp and glob maps
* [Feature] Support selectors in ratelimit module
* [Feature] Support selectors in settings
* [Feature] Use khash in HTML parser
* [Feature] Use pure Lua debugm function
* [Fix] Add fail-safety for destroying sessions
* [Fix] Allow to add result-less fake DNS records
* [Fix] Another try to fix race conditions on config unload
* [Fix] Call Lua callback on DNS timeouts
* [Fix] Deprecate task:inc_dns_req as it is redundant
* [Fix] Do not allow events deletions on cleanup
* [Fix] Do not try to process skipped messages
* [Fix] Fix HTTP requests with no body
* [Fix] Fix another cleanup race condition
* [Fix] Fix bug in processing of pcre regexps
* [Fix] Fix byte array allocation in the pool
* [Fix] Fix crashes on task cleanup
* [Fix] Fix dynamic buckets in ratelimits
* [Fix] Fix endless loop when waiting for Rspamd to stop
* [Fix] Fix lua_util.str_split in case of delimiters set
* [Fix] Fix more issues with watching of async events
* [Fix] Fix stop words detection and loading logic
* [Fix] Fix various corner cases for language detection
* [Fix] Fix watchers in lua_tcp
* [Fix] Fix words decay algorithm
* [Fix] Implement watchers replacement to handle nested calls
* [Fix] Save faked code into fake dns record
* [Fix] Show the proper frame when using lua_util.debugm
* [Fix] Use fake dns records in tests
* [Fix] Use unicode replacements for HTML entities
* [Fix] fixed "cannot find dependency on symbol 1" issue when using replaced symbols in spamassassin rules
* [Fix] partition_id is not available in old versions of CH
* [Project] Add implicit conversion logic to selectors
* [Project] Add initial support for selectors in regexps
* [Project] Add method concept
* [Project] Further changes in unicode operations
* [Project] Implement Clickhouse migrations
* [Project] Implement implicit conversions to userdata
* [Project] Implement insert method
* [Project] Implement selectors registration for regular expressions
* [Project] Implement selectors support in re_cache
* [Project] Improve language detector: cleanup unused files, categorize
* [Project] Migrate CH data to a fat table
* [Project] Rework selectors logic
* [Project] Start Clickhouse utilities library
* [Project] Start unicode rework
* [Project] coroutine threaded model for API calls: thread pool
* [Rework] Move phishtank to a DNS based service
* [Rework] Rework Clickhouse plugin to use the new API
* [Rework] Rework language detector
* [Rework] Rework utf content processing in text parts
* [WebUI] Add progress bar for AJAX requests
* [WebUI] Avoid errors table reinitialization
* [WebUI] Avoid history table reinitialization
* [WebUI] Avoid throughput summary table reinitialization
* [WebUI] Destroy summary table on disconnect
* [WebUI] Fix "auth" request URL
* [WebUI] Fix disabling and hiding controls on page reload
* [WebUI] Fix maps loading from neighbours
* [WebUI] Fix symbols sorting by score
* [WebUI] Fix tables destroying
* [WebUI] Fix throughput data consolidation
* [WebUI] Fix upload buttons disabling
* [WebUI] Notify user on module loading failure
* [WebUI] Update FooTable 3.1.4 -> 3.1.6

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [CritFix] Fix caseless comparison of equal length strings
* [Feature] Add HTTP basic auth support to elastic and clickhouse plugins
* [Feature] Add SPF selector to reputation
* [Feature] Add support of the fallback backends for HTTP maps
* [Feature] Allow to print full mime structure when extracting mime data
* [Feature] Allow to split symbols in reputation plugin
* [Feature] Check attachments only on AV scanners in attachments_only mode
* [Feature] Disable all SSL checks if ssl_no_verify flag is set
* [Feature] Implement parsing of scoped IPv6 addresses
* [Feature] Improve `rspamc counters` output
* [Fix] Add sanity checks when expanding SPF macros
* [Fix] Allow to parse SA rules with no spaces around =~ (dirty hack)
* [Fix] Avoid one extra byte writing
* [Fix] Deal with direct hash table
* [Fix] Detect empty text part as text, not HTML
* [Fix] Do not reduce map watch timeout for mixed http/file maps
* [Fix] Fix HTML part detection heuristic
* [Fix] Fix double free in redirectors cleanup
* [Fix] Fix legacy history handling in the controller
* [Fix] Fix messages insertion
* [Fix] Fix sending string method
* [Fix] Fix statconver command line arguments
* [Fix] Fixed argument checking for being null
* [Fix] Fixed issues reported by luacheck
* [Fix] Freeze updates queue when do actual storage update
* [Fix] HTTP map hash is per-backend and not per-map
* [Fix] Plug memory leak in fuzzy updates
* [Fix] Prefer 'MTA-Name' when producing authentication results
* [Fix] Replace bad unicode sequences instead of stopping on them
* [Fix] Set classifier version on learning
* [Project] Reworked ratelimits
* [Project] Apply topological sorting for symbols in Rspamd
* [Project] Remove global contexts from C modules
* [Project] Move performance critical hash tables to khash
* [WebUI] Avoid unused indexes
* [WebUI] Do not execute `on_success` callback
* [WebUI] Fix history reset for "All SERVERS" (#2346)
* [WebUI] Fix query URL for selected server
* [WebUI] Fix symbols display in legacy history,
* [WebUI] Hide symbols order selector for legacy history
* [WebUI] Refactor query functions into one
* [WebUI] Remove previously-attached event handlers
* [WebUI] Save symbols to the selected server
* [WebUI] Unify arguments of query functions
* [WebUI] Use common query functions to get graph data
* [WebUI] Use common query functions to save symbols

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Feature] Add more extended statistics about fuzzy updates
* [Feature] Add more non-conformant Received headers support
* [Feature] Add preliminary function to get fuzzy hashes from text in Lua
* [Feature] Allow to configure AV module rejection message
* [Feature] Implement fuzzy hashes extraction in mime tool
* [Feature] Improve WHITE_ON_WHITE rule
* [Feature] Improve integer -> string conversion
* [Feature] Reuse maps in multimap module more aggressively
* [Fix] Avoid race condition in skip map as pool lifetime is not enough
* [Fix] Eliminate all specific C plugins pools
* [Fix] Fix DKIM check rule if DNS is unavailable
* [Fix] Fix build where ucontext is defined in ucontext.h
* [Fix] Fix crash in base url handling
* [Fix] Fix descriptors leak in sqlite3 locking code
* [Fix] Fix messages quarantine
* [Fix] Fix padded numbers printing
* [Fix] Fix race condition on maps reinit
* [Fix] Fix regexp functions when no data is passed
* [Fix] Fix specific urls extraction
* [Fix] Fix styles propagation
* [Fix] Improve resetting of the limit buckets
* [Fix] Initialize sqlite3 properly
* [Fix] Work with broken resolvers in resolv.conf
* [Project] Implement HTTP maps caching
* [Project] Refresh fuzzy hashes when matched
* [Project] Add logic to deduplicate fuzzy updates queue
* [WebUI] Add missed declarations
* [WebUI] Avoid using "undefined" property
* [WebUI] Do not accept passwords containing control characters
* [WebUI] Do not redeclare variables
* [WebUI] Enable strict mode,
* [WebUI] Fix variable assignment
* [WebUI] Initialize variables at declaration
* [WebUI] Remove duplicated path from RequireJS config
* [WebUI] Remove unused block
* [WebUI] Remove unused variable
* [WebUI] Remove unused variables
* [WebUI] Use self-explanatory notation
* [WebUI] Use type-safe equality operators

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [CritFix] Check NM part of pubkey to match it with rotating keypairs
* [CritFix] Do not overwrite PID of the main process
* [CritFix] Fix maps after reload
* [CritFix] Fix maps race conditions on reload
* [CritFix] Fix shmem leak in encrypting proxy mode
* [Feature] Add a concept of ignored symbols to avoid race conditions
* [Feature] Add ability to print bayes tokens in rspamadm mime
* [Feature] Add method to get statistical tokens in Lua API
* [Feature] Add preliminary mime stat command
* [Feature] Add rspamadm mime tool
* [Feature] Add urls extraction tool
* [Feature] Address ZeroFont exploit
* [Feature] Allow rspamadm mime to process multiple files
* [Feature] Allow to extract words in `rspamadm mime`
* [Feature] Allow to print mime part data
* [Feature] Allow to show HTML structure on extraction
* [Feature] Distinguish IP failures from connection failures
* [Feature] Improve output for mime command
* [Feature] Improve styles propagation
* [Feature] Main process crash will now cleanup all children
* [Feature] Preload file and static maps in main process
* [Feature] Print stack trace on crash
* [Feature] Process font size in HTML parser
* [Feature] Propagate content length of invisible tags
* [Feature] Read ordinary file maps in chunks to be more safe on rewrites
* [Feature] Support base tag in HTML
* [Feature] Support more size suffixes when parsing HTML styles
* [Feature] Support opacity style
* [Fix] Another fix for nested composites
* [Fix] Fill nm id in keypairs cache code
* [Fix] Fix colors alpha channel handling
* [Fix] Fix destruction logic
* [Fix] Fix double free
* [Fix] Fix maps preload logic
* [Fix] Fix nested composites process
* [Fix] Fix proxying of Exim connections
* [Fix] Fix reload crash
* [Fix] Fix rspamadm -l command
* [Fix] Update ed25519 signing schema
* [WebUI] Stop using "const" declaration
* [WebUI] Update RequireJS to 2.3.5

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [CritFix] Fix multiple neural networks support
* [Feature] Add decryption function to keypair command
* [Feature] Add gzip compression for HTTP requests in elastic module
* [Feature] Add gzip methods to lua util
* [Feature] Add maps based on Top Level Domains
* [Feature] Add pubkey checks for dkim_signing
* [Feature] Add support of fake DNS records
* [Feature] Add tool to encrypt files
* [Feature] Allow to add symbols using settings directly
* [Feature] Allow to match private and public keys for DKIM signatures
* [Feature] Allow to set task flags via settings
* [Feature] Allow to specify fake DNS address from the config
* [Feature] Implement signatures verification using rspamadm keypair
* [Feature] Implement signing using `rspamadm keypair`
* [Feature] Improve error reporting for DKIM key access issues
* [Feature] Provide $HOSTNAME variable in UCL
* [Feature] Rework levenshtein distance computation
* [Feature] Split message parsing and processing
* [Feature] Support ED25519 DKIM signatures
* [Feature] Support encrypted configs in UCL
* [Feature] Suppress duplicate warning on very large radix tries
* [Feature] Use OSB to combine header names
* [Fix] Cleanup maps data on shutdown
* [Fix] Fix '~' behaviour in composites
* [Fix] Fix HTTP maps updates
* [Fix] Fix NIST signatures
* [Fix] Fix RFC822 comments when processing a mime address
* [Fix] Fix double free
* [Fix] Fix dynamic settings application
* [Fix] Fix for CommuniGate Pro maillist
* [Fix] Fix keypair creation method to actually create keypair...
* [Fix] Fix matching patterns with no paths
* [Fix] Fix memory leak in parsing comments
* [Fix] Fix parsing of urls with numeric password
* [Fix] Fix plugins intialisation in configwizard
* [Fix] Fix potential crash on reload
* [Fix] Fix potential race condition for a finished HTTP connections
* [Fix] Fix race-condition leak on processes reload
* [Fix] Fix signing in openssl mode
* [Fix] Free language detector structures
* [Fix] Relax alignment requirements
* [Fix] Send DMARC reports compressed
* [Fix] Try to fix leak in dmarc module
* [Fix] Try to plug memory leak in metric exporter
* [Project] Convert rspamadm subcommands to Lua
* [WebUI] Display smtp sender/recipient in history
* [WebUI] Fix elements disabling in "Symbols" tab
* [WebUI] Limit recipients list in history column to 3
* [WebUI] Match envelope and mime addresses following in arbitrary order
* [WebUI] Update column header
* [WebUI] Wrap addresses in history

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Add MSBL proposed return codes
* [Conf] Add additional groups for policies
* [CritFix] Do not use volatile Lua strings as UCL keys
* [Feature] Add ability to add fuzzy hashes to headers
* [Feature] Add function to extract most meaningful urls
* [Feature] Add rule to block mixed text and encrypted parts
* [Feature] Allow multiple groups for symbols
* [Feature] Allow to disable lua squeezing logic
* [Feature] Allow to get multipart children in Lua
* [Feature] Allow to insert multiple headers from milter headers
* [Feature] Allow to print scores in subject and further extensions
* [Feature] Be more error-prone in squeezed rules
* [Feature] Support multiple return codes in emails module
* [Feature] Use EMA for calculating averages
* [Feature] Use common jit cache for all regexps
* [Feature] support for CommuniGate Pro self-generated messages
* [Fix] Allow to have multiple values for headers as arrays
* [Fix] Do not open sockets for disabled workers
* [Fix] Fix AuthservId
* [Fix] Fix base64 folding in Lua API
* [Fix] Fix build on non-x86 platforms
* [Fix] Fix cached maps logic
* [Fix] Fix compatibility with old maps query logic
* [Fix] Fix crash if skip_map is used
* [Fix] Fix importing static maps from UCL
* [Fix] Fix parsing of unix sockets
* [Fix] Fix raw_mime regexp on HTML part with no text content
* [Fix] Fix tables logging
* [Fix] Fix vertical tab handling in libucl
* [Fix] Try to fix frequency counters
* [Fix] Use better sharding for ip_score
* [Fix] Use multiple results from SURBL DNS reply
* [Fix] When doing AV scan select a different server for retransmit

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Major stock config updates:
    - Workers are now specified in a new format worker "type" { ... }
    - Enable fuzzy worker to simplify local fuzzy storages configuration
    - Bind all workers to localhost by default to avoid security flaws
* [Conf] Make more sane fuzzy_check default settings
* [CritFix] Fix ucl escape for bad symbols
* [Feature] Add failure symbol for AV module
* [Feature] Add lazy expiration mode for new classifier schema
* [Feature] Add preliminary version of maps stats plugin
* [Feature] Allow to block fuzzy requests from specific networks
* [Feature] Allow to change `expire` of live statistics
* [Feature] Distinguish AV failure from clean result
* [Feature] Further improvements of language detector by using khash
* [Feature] Further optimization of the lang_detection
* [Feature] Implement cluster-aware bayes expiry
* [Feature] Implement exclude patterns in rspamc
* [Feature] Implement glob maps in addition to regexp maps
* [Feature] Implement map statistics function for lua API
* [Feature] Implement stop symbols for Clickhouse collection
* [Feature] Support recipients separated by commas
* [Feature] Try harder to upload scripts to the Redis server
* [Feature] Upgrade t1ha distribution
* [Feature] use_domain_sign_inbound
* [Feature] Use scores from maps if `symbols_set` is not defined
* [Fix] Add resolving version of radix map helper
* [Fix] Check URL before adding implicit prefix
* [Fix] Do not check pid/state when using PRNG
* [Fix] Fix CentOS logrotate script for systemd
* [Fix] Fix slash + dot in urls
* [Fix] Fix systemd version of the logrotate script
* [Fix] Propagate key when import implicit array from Lua
* [Fix] Strip spaces from map keys and values
* [Fix] Try to fix a specific case when processing milter protocol
* [Fix] Try to fix crash when a tcp connection cannot be set
* [Fix] Typo use_domain_local --> use_domain_sign_local
* [Fix] Various fixes to once_received module
* [Project] Store hits counters for map elements

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [CritFix] Plug bad memory leak in protocol reply
* [Feature] Add avx2 codec for base64
* [Feature] Add method to receive all URL flags from Lua API
* [Feature] Allow to fold headers on stop characters
* [Feature] Allow to set lua_cpath from options
* [Feature] Allow to specify custom rejection message in milter
* [Feature] Deal with unnormalised Unicode obfuscation
* [Feature] Do not detect language twice for relative parts
* [Feature] Implement oversigning feature
* [Feature] Implement silent logging level to minimize noise in logs
* [Feature] Improve URL_IN_SUBJECT rule
* [Feature] Use hashing to reduce redis attack surface
* [Fix] Add oversigning for the most important headers
* [Fix] add 'rewrite subject' to History dropdown
* [Fix] Another fix in folding algorithm
* [Fix] Do not call multimap addr for parts of addr if filter is presented
* [Fix] Do not clean hostname on generic reset
* [Fix] Do not create pid file in no-fork mode
* [Fix] Fix fold_after case to preserve multiple spaces
* [Fix] Fix folding and folding tests
* [Fix] Fix hostname usage in milter mode
* [Fix] Fix lua RSA verify and its tests
* [Fix] Fix metadata exporter send_mail backend (#2124)
* [Fix] Fix processing of '\v' in libucl
* [Fix] Fix shemaless URLs detection
* [Fix] Fix support of multiple headers in sign_header
* [Fix] Fix usage of util.parse_mail_address
* [Fix] Fix weights of dynamic squeezed rules
* [Fix] Leak from bucket before checking the burst
* [Fix] Stop using own localtime as DST could be messy in many cases
* [Fix] Treat unnormalised URLs as obscured
* [Rework] Restore leaky bucket model in ratelimit plugin
* [WebUI] Add messages total to throughput summary
* [WebUI] Add symbols order selector to history
* [WebUI] Config: Load list on demand
* [WebUI] Fix modalBody for maps that appear more than once
* [WebUI] History: Fix Tooltips on paging, filtering and sorting
* [WebUI] Remove a previously-attached event handler
* [WebUI] Update D3 to v5.0.0 and jQuery to v3.3.1

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Feature] Store emails in Clickhouse
* [Feature] Support single quotes in config
* [Feature] Use templates when publishing CH schema
* [Feature] Improve Docker image
* [Fix] Add rounding when printing a lot of FP variables
* [Fix] Allow to disable certain actions by assigning null to them
* [Fix] Disable results caching
* [Fix] Fix disabling of squeezed symbols
* [Fix] Fix scan time set
* [Fix] Rework logic of actions setting
* [Fix] Try to fix various Lua stack issues
* [WebUI] Add link tag for favicon.ico
* [WebUI] Display hostname:port/path in the page title

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [CritFix] Fix lowercase comparison
* [CritFix] Timezone defines seconds WEST UTC not East
* [Feature] Add filename to log format
* [Feature] Add lua rules squeezing
* [Feature] Add related symbols analysis to rspamd_stats
* [Feature] Remove upstream `X-Spam: Yes` header by default
* [Feature] rspamd_stats: Output progress info on STDERR
* [Feature] Whitelist for emails module
* [Fix] Do not allow dependencies on self
* [Fix] Do not cache metric result
* [Fix] Do not trust all issuers as a client certificate
* [Fix] Fix dependencies in lua squeeze
* [Fix] Fix enabling/disabling squeezed rules
* [Fix] Fix enabling/disabling symbols
* [Fix] Fix external dependencies
* [Fix] Fix processing of a single compressed file
* [Fix] Fix some typos
* [Fix] Fix various modules in case of empty message
* [Fix] Handle callbacks that returns table of options
* [Fix] Improve cached action interaction
* [Fix] Make dynamic conf more NaN aware
* [Fix] Never hide actions from WebUI `configuration` tab
* [Project] Implementation of Lua rules squeezing

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Add bayes_expiry as explicit module
* [Conf] Adjust names and weights for neural network plugin
* [Conf] Change updates url
* [Conf] Default statistics is stored in Redis now
* [Conf] Disable fann_redis module by default
* [Conf] Fix default elastic configuration
* [Conf] Fix double quote position
* [Conf] Massive config rework for new structure of symbols and scores
* [Conf] Rename Rambler BLs as they are now Rspamd's ones
* [Conf] Use dedicated rspamd.com subdomains
* [Conf] Use more data from rspamd.com fuzzy storage
* [CritFix] Add sanity guards for badly broken HTML
* [CritFix] Another errors path handling fix
* [CritFix] Another portion of tokenization fixes
* [CritFix] Do not send reject messages after set reply
* [CritFix] Fix ARC chain verification
* [CritFix] Fix crash in milter errors handler
* [CritFix] Fix memory leak in spf caching logic
* [CritFix] Fix milter commands pipelining
* [CritFix] Fix newlines detection
* [CritFix] Fix semicolons parsing in the content type
* [CritFix] Plug memory leak in zstd protocol compression
* [Feature] Add ability to match score in force_actions module
* [Feature] Add aes-rng PRF to libottery
* [Feature] Add 'composites' debug module
* [Feature] Add concept of experimental modules
* [Feature] Add DKIM trace symbol
* [Feature] Add EBL to the default config
* [Feature] Add expected ip check for emails plugin
* [Feature] Add framework to manage Redis scripts
* [Feature] Add framing for the new reputation generic plugin
* [Feature] Add function to show plugins stat
* [Feature] Add gzip compression support for clickhouse module
* [Feature] Add gzip compression support for rspamd controller
* [Feature] Add gzip support when sending lua http requests
* [Feature] Add json output for rspamd_stats
* [Feature] Add method to do a synchronous Redis connection
* [Feature] Add method to get all content-type attributes in Lua
* [Feature] Add `-m` flag to configdump to show modules states
* [Feature] Add mime types to extensions map
* [Feature] Add more features to rescore utility
* [Feature] Add more gtube like patterns to test other spam actions
* [Feature] Add more metafunctions, improve logging
* [Feature] Add more text attributes
* [Feature] Add new configwizard command to rspamadm
* [Feature] Add new tooling for stats conversation
* [Feature] Add old groups migration tool
* [Feature] Add plugins state variable
* [Feature] Add preliminary ecdsa keys support in DKIM
* [Feature] Add preliminary support of idempotent symbols
* [Feature] Add Redis server wizard
* [Feature] Add routine to convert old style stats to a new one
* [Feature] Add some sanity checks for actions and controller
* [Feature] Add statistic convertation module to configwizard
* [Feature] Add sugestions logic to mempool allocator
* [Feature] Add support of config transform in Lua
* [Feature] Add timeout to rspamc when doing corpus test
* [Feature] Add tooling to convert bayes schemas
* [Feature] Add torch conditional to configuration
* [Feature] Add torch-decisiontree package
* [Feature] Add torch-optim contrib package
* [Feature] Add TTL autodetection
* [Feature] Add urls reputation to the reputation framework
* [Feature] Allow floating and negative values in expressions limits
* [Feature] Allow multiple CTs in full extensions map
* [Feature] Allow multiple fann rules
* [Feature] Allow randomly select User-Agent from a list
* [Feature] Allow rspamadm commands to export methods in Lua
* [Feature] Allow rule specific min_bytes in fuzzy check
* [Feature] Allow to adjust symbols scores from Lua
* [Feature] Allow to attach stat signature to messages
* [Feature] Allow to change SMTP from via milter headers
* [Feature] Allow to configure monitored
* [Feature] Allow to create directories in Lua API
* [Feature] Allow to disable torch and skip train samples for ANN
* [Feature] Allow to discard messages dynamically
* [Feature] Allow to enable/disable languages from the detector
* [Feature] Allow to generate DKIM keys from rspamadm API
* [Feature] Allow to get CPU flags from Lua
* [Feature] Allow to have high precision timestamps in logs
* [Feature] Allow to insert headers into specific position
* [Feature] Allow to limit redirector requests per task
* [Feature] Allow to load and use dynamic ANNs with torch
* [Feature] Allow to quarantine rejected messages using milter interface
* [Feature] Allow to receive signing keys from mempool vars
* [Feature] Allow to reserve elements in libucl
* [Feature] Allow to reuse signal handlers chains
* [Feature] Allow to set custom mempool variables from settings
* [Feature] Allow to set headers from settings
* [Feature] Allow to set Settings-Id for all connections
* [Feature] Allow to skip real action and add a header instead
* [Feature] Allow to skip specific hashes in fuzzy storage
* [Feature] Allow to spawn asynchronous processes from Lua
* [Feature] Allow to specify number of threads for ANN learning
* [Feature] Allow to use global lua maps in settings
* [Feature] Allow to use postfilters in composites
* [Feature] Allow to verify signatures from HTTP headers in maps
* [Feature] Antivirus: ordered pattern matches
* [Feature] Authentication-Results: support hiding usernames
* [Feature] Automatically create tables in clickhouse
* [Feature] Catch next-to-last bad extension
* [Feature] Check cached maps more frequently
* [Feature] Check groups sanity
* [Feature] Deal with obscured URLs with @ symbols
* [Feature] Enhance task:store_in_file method
* [Feature] Export password encryption routines to Redis
* [Feature] Filter nan and inf when adding scores
* [Feature] Finalize 7zip files support
* [Feature] Further improvements in language detection
* [Feature] Further improvements in language detection algorithm
* [Feature] Generic key name expansion for Redis keys
* [Feature] Hash whitelist for fuzzy_check
* [Feature] Implement bayes signatures storage
* [Feature] Implement buckets for Redis backend
* [Feature] Implement DKIM reputation adjustments
* [Feature] Implement forked workers children monitoring
* [Feature] Implement headers flags in mime parser
* [Feature] Implement l1/l2 regularization against the current weights
* [Feature] Implement manual ANN train mode
* [Feature] Implement per-user ANN support
* [Feature] Implement torch based ANN learning
* [Feature] Implement upstreams logic for clickhouse exporter
* [Feature] Import torch to Rspamd...
* [Feature] Improve allocation policy when interacting with Lua
* [Feature] Improve Lua/C interaction in history_redis
* [Feature] Improve multiple fuzzy results combining
* [Feature] Improve parsing of DKIM keys: parse algorithm
* [Feature] Improve subprocesses termination handle
* [Feature] Improve symbol type parsing in Lua API
* [Feature] Metadata Exporter: e-Mail Alerts: support multiple recipients; alerting senders/recipients/users (#1600)
* [Feature] Milter headers: support adding/removing arbitrary headers from config
* [Feature] More metatokens
* [Feature] Multimap: checking of symbol options
* [Feature] Multimap: template URL filter
* [Feature] New bayes expiry plugin
* [Feature] Periodically save rspamd stats to disk
* [Feature] Preliminary import of the elasticsearch module
* [Feature] Ratelimit: allow full addresses in whitelisted_rcpts
* [Feature] Ratelimit: support fetching limits from Redis
* [Feature] RBL: received: filtering by position & flags
* [Feature] Read global maps for lua
* [Feature] Redis settings: support checking multiple keys
* [Feature] Rework fann plugin to be a normal post-filter
* [Feature] Rework logging configuration for rspamadm case
* [Feature] Rework short hashes generation to avoid FP
* [Feature] Save real ucl types when exporting to Lua
* [Feature] Set TCP_NODELAY for milter sockets
* [Feature] Setup DKIM signing from configwizard
* [Feature] Skip certain symbols from ANN classify
* [Feature] Store plugins state
* [Feature] Support etag for HTTP maps
* [Feature] Support Expires header when using HTTP maps
* [Feature] Support sending given header multiple times in lua_http
* [Feature] Support sha512 in DKIM signatures
* [Feature] Try to detect HTML messages better
* [Feature] Use array instead of queue to reduce memory fragmentation
* [Feature] Use controller port by default when connecting to local IP
* [Feature] Use rdtsc where possible
* [Fix] Actively load skip hashes map in fuzzy storage
* [Fix] Add another workaround to display history properly
* [Fix] Add definition for old glib compatibility method
* [Fix] Add missing rspamadm control options to help
* [Fix] Add workaround for IPv6 in sendmail
* [Fix] Add workaround for system with non-XSI compatible tzset
* [Fix] Allow oversigning in DKIM signatures
* [Fix] Allow to check negative scores in force_actions
* [Fix] Allow to have negative actions limits
* [Fix] Allow to set any layers number for fann rules
* [Fix] Another fix for rdtcs
* [Fix] Another fix to lua xmlrpc
* [Fix] Another try to deal with #1998
* [Fix] Another try to fix #1998
* [Fix] Another try to fix threading in torch
* [Fix] Apply language detection when adding fuzzy hashes
* [Fix] ARC: Fix Lua 5.3 compatibility; timestamp should be integer
* [Fix] Authentication Results: Fix SPF smtp.mail_from
* [Fix] Auth-Results: Multiple DKIM signatures
* [Fix] Avoid changing content-transfer-encoding header's value
* [Fix] Better handling of the legacy protocol
* [Fix] Check decoded headers sanity (e.g. by excluding \0)
* [Fix] Check for magic when checking for an archive
* [Fix] Cleanup mess with groups
* [Fix] Clickhouse: Insertion in the symbols table
* [Fix] Crash in URL processing
* [Fix] Deal with another case when processing exceptions
* [Fix] Deal with deeply nested messages more aggressively
* [Fix] Deal with nan and inf encoding in json/ucl
* [Fix] Deal with non-key arguments in lua_redis.exec_script
* [Fix] Deal with unknown weight
* [Fix] Deal with URLs with no slashes after protocol
* [Fix] Deal with URLs wrapped in [] in text parts
* [Fix] Deal with zero scores symbols
* [Fix] Default monitoring domain for surbl plugin
* [Fix] Delay upstream re-resolving when one upstream is defined
* [Fix] Detection of maillist optimized and fixed
* [Fix] DKIM signing: allow for auth_only to be false
* [Fix] DMARC: require report_settings for sending reports only
* [Fix] Do not allow garbadge when checking url domain
* [Fix] Do not cache SPF records with PTR elements
* [Fix] Do not constantly re-resolve failed upstreams with a single element
* [Fix] Do not crash if no words defined
* [Fix] Do not crash on empty subtype
* [Fix] Do not expose spamtrap messages to SMTP reply
* [Fix] Do not fail rbl plugin when there are no received or emails
* [Fix] Do not ignore short words
* [Fix] Do not include idempotent/nostat symbols to checksum
* [Fix] Do not override groups when converting metrics
* [Fix] Do not override unix socket group when group comes before owner
* [Fix] Do not skip the last character
* [Fix] Do not spawn too many workers by default
* [Fix] Do not stop monitored on dns errors
* [Fix] Do not stop parsing headers on bad IP header
* [Fix] Do not strip last character in the last word
* [Fix] Do not treat script content as text
* [Fix] Do not try to connect to non-supported addresses
* [Fix] Do not try to dereference last character
* [Fix] Do not try to sign unknown domains
* [Fix] Don't use whitelist/greylist maps as regexp, but as map
* [Fix] Erase unknown HTML entities
* [Fix] Exim Received header protocol parsing
* [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false
* [Fix] Fix a lot of FP in chartable in mixed languages
* [Fix] Fix ANN checks
* [Fix] Fix ANN loading logic
* [Fix] Fix another tokenization issue
* [Fix] Fix autolearn parameters reading
* [Fix] Fix bad archive characters stripping
* [Fix] Fix bad extension check
* [Fix] Fix bayes schema conversion
* [Fix] Fix blacklists and DMARC in whitelist
* [Fix] Fix brain-damaged torch build system
* [Fix] Fix build on FreeBSD
* [Fix] Fix clickhouse exporter
* [Fix] Fix clickhouse schema
* [Fix] Fix comparision
* [Fix] Fix composites processing
* [Fix] Fix connecting to a unix socket in rspamadm statconvert
* [Fix] Fix couple of warnings
* [Fix] Fix crashes in the rspamd_control path
* [Fix] Fix deletion from hash
* [Fix] Fix DKIM forgeries via multiple headers
* [Fix] FIx dynamic conf plugin
* [Fix] Fix emails detection
* [Fix] Fix empty headers simple canonicalization
* [Fix] Fix empty threshold check in greylisting module
* [Fix] Fix encrypted legacy reply in fuzzy storage
* [Fix] Fix enormous scores for R_WHITE_ON_WHITE
* [Fix] Fix exceptions list in surbl
* [Fix] Fix *_EXCESS_BASE64 rules
* [Fix] Fix expire rounding
* [Fix] Fix extra hits in PCRE mode for regular expressions
* [Fix] Fix format strings
* [Fix] Fix get_content method
* [Fix] Fix groups override when defining symbols
* [Fix] Fix learned count in new schema
* [Fix] Fix learn errors propagation
* [Fix] Fix loading of per-user redis backend for statistics
* [Fix] Fix logging buffer corruption in case of repeated messages
* [Fix] Fix lua cached elements invalidation
* [Fix] Fix merging of the implicit arrays
* [Fix] Fix mime_types scoring
* [Fix] Fix multiple headers in DKIM headers list
* [Fix] Fix null callee case in clang plugin
* [Fix] Fix obscured url in format user@@example.com
* [Fix] Fix parsing of the per-user script
* [Fix] Fix priorities in rspamd_update, disable rules execution
* [Fix] Fix processing of closed tags
* [Fix] Fix processing of idempotent rules when autolearn fails
* [Fix] Fix processing of multipart parts with no headers
* [Fix] Fix processing of skip-hashes in fuzzy storage
* [Fix] Fix PTR processing in SPF
* [Fix] Fix pushing country to clickhouse asn table
* [Fix] Fix random forests module
* [Fix] Fix real IP parsing for some strange Exim received
* [Fix] Fix Redis timeout setup
* [Fix] Fix reload crash when hyperscan is enabled
* [Fix] Fix reusing of redis connection after exec
* [Fix] Fix sanity checks on macro value
* [Fix] Fix setting of path and cpath for Lua
* [Fix] Fix setting of signals when spawning a thread
* [Fix] Fix text splitting: stack overflow (too many captures)
* [Fix] Fix ticks processing
* [Fix] Fix upstream addrs updating
* [Fix] Fix urls/emails distinguishing found in queries
* [Fix] Fix user settings check
* [Fix] Fix variable increment
* [Fix] Fix various issues in stat_convert
* [Fix] F-PROT Antivirus infection string for all known occurences
* [Fix] F-PROT Antivirus: only check return code to determine infection
* [Fix] Further fixes around floating point expressions
* [Fix] Further fixes to ANN module
* [Fix] Further fixes to rescore tool
* [Fix] Further fixes to support ES 6
* [Fix] Further tokenization fixes
* [Fix] Greylisting set phase is not idempotent
* [Fix] Handle proxy copy errors
* [Fix] Header checks: Fix get_raw_header method
* [Fix] Header checks: REPLYTO_UNPARSEABLE rule
* [Fix] Kill spawned processes on termination
* [Fix] Load skip map from all processes as shared cache is unavailable
* [Fix] Lowercase HTTP headers to make them searchable from Lua
* [Fix] Lowercase words
* [Fix] Lua_http: freeing
* [Fix] Lua: lpeg to be loaded with rspamd_lua_add_preload, to avoid "rspamd_config_read: rcl parse error: cannot init lua file […] module 'lpeg' not found"
* [Fix] Map absence is not an error
* [Fix] Metadata exporter: check IP sanity
* [Fix] Milter headers: custom headers: removing headers
* [Fix] Milter headers: skip_local / skip_authenticated settings
* [Fix] Milter headers: X-Spamd-Result header if X-Virus ran first
* [Fix] mime_types: fix next-to-last extension length check
* [Fix] More hacks to deal with old configs
* [Fix] Move composites second pass to the dedicated stage
* [Fix] Multimap: received: filtering of artificial header
* [Fix] Multiple fixes in torch based ANN plugins
* [Fix] Once more (#1879) fix bad extension check
* [Fix] Optimize rspamd_fstring_t reallocations
* [Fix] options.local_networks setting
* [Fix] Parse HREF urls without explicit prefix
* [Fix] Plan new event on HTTP errors
* [Fix] Plug another possible memory leak
* [Fix] Plug memory leak
* [Fix] Plug memory leak in lua_tcp
* [Fix] Plug memory leak when setting email addresses from Lua
* [Fix] Propagate learn/stat errors more precisely
* [Fix] Ratelimit: fix whitelisted_rcpts matching
* [Fix] Ratelimit: lowercase email addresses
* [Fix] RBL: received: deal with missing data (#1965)
* [Fix] Rebalance and slightly rework MX check plugin
* [Fix] Redis key expansion: EVAL: deal with strings
* [Fix] Redis script loading in DMARC; URL tags; URL reputation
* [Fix] Reject invalid bh for DKIM signatures earlier
* [Fix] Relax pem signature detection
* [Fix] Relax unicode properties requirements for chartable module
* [Fix] Remove extra noise from dkim and arc signing
* [Fix] Remove hop-by-hop headers in proxy
* [Fix] Remove incorrect method `task:set_metric_subject`
* [Fix] Replace space like characters in headers with plain space
* [Fix] Restore old style ratelimits support
* [Fix] Rework elasticsearch plugin
* [Fix] Rewriting subjects via force actions module
* [Fix] RPM postinstall
* [Fix] Sanitize IP in history redis
* [Fix] Select the correct signature when doing simple canon
* [Fix] Set CLOEXEC flag on files opened
* [Fix] Setting check_local / check_authed in plugins (#1954)
* [Fix] Settings: avoid checking invalid IP (#1981)
* [Fix] Settings: header: deal with multiple settings (#1988)
* [Fix] Skip checks if both extensions are not bad
* [Fix] Skip nostat tokens when get number of tokens
* [Fix] Some more fixes towards emails detection
* [Fix] SpamAssassin: Fail check_freemail_header if regexp didn't match
* [Fix] Stop using of g_slice...
* [Fix] Switch rspamadm logging to message level
* [Fix] Symbol 'FANNR_SPAM' has its score defined..
* [Fix] Table parameter for rspamd_config:add_doc()
* [Fix] Treat 'rewrite subject' as spam action
* [Fix] Try harder in passing IPv6 addresses
* [Fix] Try harder to find rfc822 notifications
* [Fix] Try harder to find urls
* [Fix] Use decoded values when parsing mime addresses
* [Fix] Use full URL when making an HTTP request
* [Fix] Use greylisting threshold in greylisting module
* [Fix] Use n_words attribute from ngramms
* [Fix] Use raw urls when sending requests to redirector
* [Fix] Use the right boolean operator on error check
* [Fix] Use weight from map for fuzzy scoring
* [Fix] Various fixes to elastic plugin
* [Fix] Various fixes to fann_redis instantiation
* [Fix] Various improvements in language detection
* [Fix] Virus infection string for F-PROT Antivirus
* [Fix] Virus infetction string for F-PROT Antivirus
* [Fix] WebUI: use relative path for savemap (#1943)
* [Fix] WHITE_ON_WHITE: Ensure score is matched to part that fired the rule
* [Fix] Write configuration changes as UCL config
* [Project] Add detection logic for words
* [Project] Add fast debug logging infrastructure
* [Project] Add more flags to languages
* [Project] Add n-gramms data files
* [Project] Add ngramms frequencies detector
* [Project] Add random words selection logic
* [Project] Add unigramms to language detection as well
* [Project] Convert all C modules to fast debug infrastructure
* [Project] Detect some languages based on unicode script
* [Project] Enable fast debug lookup for some modules
* [Project] Enable language detector init in scanner workers
* [Project] Further improvements to language detector
* [Project] Implement logic of ngramms application
* [Project] Improve weighting in lang_detection
* [Project] Initialize language detector
* [Project] Preliminary version of ngramms based language detector
* [Project] Preliminary version of the new stat_convert
* [Project] Remove old language detector
* [Project] Rework language detection ngramms structure
* [Project] Start language detection project
* [Project] Start rework of language detection to improve quality
* [Project] Use fast debug logging check
* [Rework] Add frame for new reputation based IP score module
* [Rework] Continue stat_convert rework task
* [Rework] Implement new version of fuzzy replies
* [Rework] Improve readability of xmlrpc API
* [Rework] Kill metrics!11
* [Rework] Ratelimit module
* [Rework] Rename fann_redis to neural plugin
* [Rework] Reorganize mime_types module
* [Rework] Rework rescore utility
* [Rework] Rewrite model and learning logic for rescore
* [Rework] Run post-loads when all initialization is completed
* [Rework] Simplify lua path initialization
* [Rework] Start major stat_convert rework
* [Rework] Start mempool fragmentation reduce project
* [Rework] Start moving of fann redis to torch
* [Rework] Stop embedding rspamadm scripts into C
* [Rework] Use floating point arithmetics in Rspamd expressions
* [Rework] Use frequencies distribution in language detector
* [Rules] Penalise R_BAD_CTE_7BIT for utf8 messages
* [WebUI] Compact graph selectors
* [WebUI] Escape strings inside HTML in history
* [WebUI] Fix message count in throughput summary (#1724)
* [WebUI] Fix NaNs display on Throughput graph
* [WebUI] Migrate widgets to D3 v4
* [WebUI] Restore passwordless login support (#2003)
* [WebUI] Show symbol descriptions as tooltips in history
* [WebUI] Stop using commas in pie chart tooltips
* [WebUI] Update D3 and jQuery
* [WebUI] Update D3Evolution 1.0.0 -> 1.1.0

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [CritFix] Add sanity guards for badly broken HTML
* [CritFix] Another errors path handling fix
* [CritFix] Fix ARC chain verification
* [CritFix] Fix crash in milter errors handler
* [Feature] Allow to insert headers into specific position
* [Feature] Allow to receive signing keys from mempool vars
* [Feature] Authentication-Results: support hiding usernames
* [Fix] Another try to deal with #1998
* [Fix] Another try to fix #1998
* [Fix] Better handling of the legacy protocol
* [Fix] Check decoded headers sanity (e.g. by excluding \0)
* [Fix] Deal with nan and inf encoding in json/ucl
* [Fix] Deal with URLs wrapped in [] in text parts
* [Fix] DKIM signing: allow for auth_only to be false
* [Fix] Do not crash on empty subtype
* [Fix] Do not fail rbl plugin when there are no received or emails
* [Fix] Do not skip the last character
* [Fix] Do not try to dereference last character
* [Fix] Do not try to sign unknown domains
* [Fix] Exim Received header protocol parsing
* [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false
* [Fix] Fix bad archive characters stripping
* [Fix] Fix comparision
* [Fix] Fix connecting to a unix socket in rspamadm statconvert
* [Fix] Fix empty headers simple canonicalization
* [Fix] Fix extra hits in PCRE mode for regular expressions
* [Fix] Fix parsing of the per-user script
* [Fix] Fix processing of skip-hashes in fuzzy storage
* [Fix] Fix Redis timeout setup
* [Fix] Fix sanity checks on macro value
* [Fix] Fix text splitting: stack overflow (too many captures)
* [Fix] Fix urls/emails distinguishing found in queries
* [Fix] F-PROT Antivirus: only check return code to determine infection
* [Fix] Metadata exporter: check IP sanity
* [Fix] Multimap: received: filtering of artificial header
* [Fix] Plan new event on HTTP errors
* [Fix] Plug another possible memory leak
* [Fix] Remove hop-by-hop headers in proxy
* [Fix] Sanitize IP in history redis
* [Fix] Setting check_local / check_authed in plugins (#1954)
* [Fix] Settings: avoid checking invalid IP (#1981)
* [Fix] Try harder in passing IPv6 addresses
* [Fix] WebUI: use relative path for savemap (#1943)
* [WebUI] Fix message count in throughput summary (#1724)
* [WebUI] Fix NaNs display on Throughput graph
* [WebUI] Restore passwordless login support (#2003)

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [CritFix] Fix semicolons parsing in the content type
* [Feature] Add EBL to the default config
* [Feature] Allow to configure monitored
* [Feature] Allow to skip specific hashes in fuzzy storage
* [Feature] Multimap: checking of symbol options
* [Feature] Redis settings: support checking multiple keys
* [Fix] ARC: Fix Lua 5.3 compatibility; timestamp should be integer
* [Fix] Avoid changing content-transfer-encoding header's value
* [Fix] Don't use whitelist/greylist maps as regexp, but as map
* [Fix] Fix get_content method
* [Fix] Header checks: Fix get_raw_header method
* [Fix] Header checks: REPLYTO_UNPARSEABLE rule
* [Fix] Lua_http: freeing
* [Fix] Milter headers: custom headers: removing headers
* [Fix] Parse HREF urls without explicit prefix
* [Fix] WHITE_ON_WHITE: Ensure score is matched to part that fired the rule
* [WebUI] Escape strings inside HTML in history

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Remove Rambler email bl for now
* [Conf] Switch RAMBLER_URIBL to a locally managed source
* [CritFix] Switch from ragel to C for Content-Type parsing
* [Feature] Add `-e` option for lua_repl
* [Feature] Add per-domain emails normalisation rules
* [Feature] Add sessions cache to debug dangling sessions
* [Feature] Add short_text_direct_hash for fuzzy check module
* [Feature] Add text_part:get_stats function
* [Feature] Allow to add custom processing script for surbl
* [Feature] Allow to check reply-to email
* [Feature] Allow to customize spam header, remove existing spam headers
* [Feature] Allow to disable specific workers in the config
* [Feature] Allow to discard messages instead of rejection
* [Feature] Allow to specify custom delimiter in emails plugin
* [Feature] Allow to specify custom User-Agent for rspamc
* [Feature] Allow to store symbols data in Clickhouse
* [Feature] Allow to use HTTPS when connecting to Clickhouse
* [Feature] Enable sessions cache tracking for milter connections
* [Feature] Implement per-line mode in lua_repl (like `perl -p`)
* [Feature] Implement rdns-curve plugin based on rspamd cryptobox
* [Feature] Improve maps cached data lifetime
* [Feature] Improve maps checking frequency
* [Feature] Improve monitored timeouts logic
* [Feature] milter_headers: add `extended_headers_rcpt` option
* [Feature] Milter headers: Add X-Spam-Flag to rmilter-compatibility headers
* [Feature] Milter headers: remove-header routine
* [Feature] Multimap: received filters for extracting TLDs from hostnames
* [Feature] Normalize email aliases in emails module
* [Feature] Re-add rambler email bl (as hashed list)
* [Feature] Reload file maps more frequently
* [Feature] Rework newlines strip parser one more time
* [Feature] Skip updates for messages scanned via controller
* [Feature] Split long DKIM public keys
* [Feature] Store more data when stripping newlines
* [Feature] Support SPF macros transformations
* [Feature] Support suppressing DMARC reports for some domains
* [Fix] Add missing `break` statement
* [Fix] Allow modifiers in SPF macros
* [Fix] DKIM sign tools: edge-cases around use_esld
* [Fix] Do not cache SPF records with macros
* [Fix] Do not overwrite score when setting pre-action
* [Fix] Fix comparision logic
* [Fix] Fix DKIM base64 folding for milter flagged messages
* [Fix] Fix emails module configuration
* [Fix] Fix folding for arc headers when milter interface is used
* [Fix] Fix gmail dots removal
* [Fix] Fix rspamc detection in greylist module
* [Fix] Fix some more issues with HTTP maps
* [Fix] Milter sessions can live forever
* [Fix] Normalize fuzzy probability better
* [Fix] Plug memory leak
* [Fix] RBL: Fixed hashed email address lookups
* [Fix] Try to deal with brain-damaged milter behaviour
* [Fix] Use `\n` to fold headers for milter
* [Rework] Allow to use custom callback for monitored checks
* [Rework] Further steps towards one process monitoring
* [Rework] Send health checks from a single worker
* [WebUI] Round-up throughput summary values

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

by a bot https://github.com/ka7/misspell_fixer

* [Fix] Allow to init resolver without rspamd_config
* [Fix] Do not crash when resolver failed to initialize
* [Fix] Fix abstract context layout
* [Fix] Fix CGP helper reply parsing
* [Fix] Fix crashes when socket write errors occur
* [Fix] Fix parsing IPv6 nameservers in resolv.conf
* [Fix] Milter: Don't defer on "greylist" action

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Add rspamd_proxy to the default configuration set
* [Conf] Add sample arc module config
* [Conf] Do away with systemd specifics completely
* [Conf] Increase min_bytes to avoid FP
* [Conf] Remove ratelimits from default configuration
* [CritFix] Fix accepting on IPv6 sockets
* [CritFix] Fix corruption when multiple fuzzy are defined
* [CritFix] Fix learn condition in fuzzy check
* [CritFix] Fix memory leak in fuzzy check
* [CritFix] Fix memory leak in maps scheduling
* [CritFix] Paese the last character in DKIM signature correctly
* [CritFix] Zero fill sockaddr_un
* [Feature] Add ability to add doc strings by example
* [Feature] Add API to verify DKIM (and ARC) signatures
* [Feature] Add compression/decompression to proxy
* [Feature] Add count to url structure
* [Feature] Add initial support of the new protocol reply
* [Feature] Add Lua plugin spamtrap
* [Feature] Add `monitored_address` for rbls
* [Feature] Add new schema for bayes tokens
* [Feature] Add preliminary ARC support to dkim code
* [Feature] Add preliminary support of ARC signing
* [Feature] Add rules to detect bad 8bit characters in From and To
* [Feature] Add scanning support for milter protocol
* [Feature] Add support for bidirectional symbols in rspamd_stats
* [Feature] Add support for static maps
* [Feature] Add support of maps with multiple regexps matches
* [Feature] Add `text_multiplier` param
* [Feature] Add the preliminary ARC plugin
* [Feature] Add top redirector targets rank
* [Feature] Allow async events to be registered from LUA rules
* [Feature] Allow storing bayes tokens in Redis
* [Feature] Allow to exclude specific domains from mx check
* [Feature] Allow to have a stack of watcher finalisers
* [Feature] Allow to pass hostname to `-i` flag in Rspamc
* [Feature] Allow to set custom user agent in url redirector
* [Feature] Allow to use custom callback when parsing resolv.conf
* [Feature] Allow to use domain from authenticated user
* [Feature] Bayes expiry plugin
* [Feature] Check dkim sign keys for modifications
* [Feature] DKIM signing: sign_networks/local address specific use_domain settings
* [Feature] DMARC: Support excluding domains from sampling
* [Feature] Expire processing items for URL redirector aggressively
* [Feature] Fix surbl monitored for IP lists, add `monitored_domain` option
* [Feature] Implement caching for dkim body hashes
* [Feature] Implement milter protocol scan reply
* [Feature] Improve omograph phishing detection
* [Feature] Initial support of self-scan in Rspamd proxy
* [Feature] Keep track of headers in milter interface
* [Feature] Milter headers: better controls for local/authenticated
* [Feature] Multimap: email:domain:tld filter
* [Feature] Preliminary DMARC reporting implementation
* [Feature] Reuse stemmers in the cache
* [Feature] Rework confighelp to load Lua plugins
* [Feature] Rework hfilter to use hyperscan if possible
* [Feature] Rework lua RSA API
* [Feature] Rmilter_headers: approximate rmilter's extended_spam_headers
* [Feature] Start integration of milter support in proxy
* [Feature] Store average words length and short words count
* [Feature] Store hash of headers order and names
* [Feature] Support MTA name header
* [Feature] Support multiple types of dkim signing in Lua
* [Feature] Support numeric arguments for Redis requests
* [Feature] Use headers hash in bayes metatokens
* [Feature] Use normal resolv.conf rules of rotation in Rspamd
* [Feature] Use version 2 proto for checking messages
* [Fix] Allow to follow symlinks when safe
* [Fix] Append MX name for authentication results as required
* [Fix] Change default text multiplier from 0.5 to 2.0
* [Fix] Check min_bytes for images as well
* [Fix] Deal with 7bit charsets properly
* [Fix] Deal with 8bit characters in email addresses
* [Fix] Deal with unpaired <a> tags
* [Fix] Detect confighelp in plugins initialisation
* [Fix] Disable certain checks for utf spoof detection
* [Fix] DKIM Signing: avoid nil index when From header is missing
* [Fix] Do not add exact hashes from different parts
* [Fix] Do not check DMARC if SPF or DKIM were not checked
* [Fix] Do not check URLs that are resolved to be redirected
* [Fix] Do not set bayes probability if we don't use it
* [Fix] Do not stop on illegal unicode points - replace them
* [Fix] Fix another race condition in arc checks
* [Fix] Fix arc count logic
* [Fix] Fix ARC signing
* [Fix] Fix brain-damaged spamc protocol for now
* [Fix] Fix calling for peak functions
* [Fix] Fix couple of issues in FORWARDED rule
* [Fix] Fix CTE propagation from parent containers to children parts
* [Fix] Fix errors processing in the controller
* [Fix] Fix format string in milter
* [Fix] Fix issues in SPF macros parsing
* [Fix] Fix logging format string
* [Fix] Fix logic of cached passwords check
* [Fix] Fix lowercasing of stemmed words
* [Fix] Fix LRU elements removal
* [Fix] Fix memory leak when accepting from unix sockets
* [Fix] Fix milter connections persistence
* [Fix] Fix objects merging in UCL
* [Fix] Fix order of operations to avoid race condition
* [Fix] Fix parsing of long regexp types
* [Fix] Fix passing data to log helper when many symbols defined
* [Fix] Fix pools management for milter session
* [Fix] Fix processing of the watchers
* [Fix] Fix queue id macro in milter
* [Fix] Fix R_BAD_CTE_7BIT rule
* [Fix] Fix Redis timeout set
* [Fix] Fix REPLYTO_UNPARSEABLE rule
* [Fix] Fix setting of email address
* [Fix] Fix some more issues about duplicated fuzzy requests
* [Fix] Fix spamc support in rspamd proxy
* [Fix] Fix syntax error in spamtrap plugin
* [Fix] Fix url counts for href urls
* [Fix] Fix url handling in the protocol
* [Fix] Multimap: Received IP filters with Redis
* [Fix] Oops, fix d9d0fa5e86
* [Fix] Parse rgb[a](x,x,x[,x]) css colors
* [Fix] Phishing: strict_domains
* [Fix] Reduce maps aggressiveness
* [Fix] Reresolve upstreams even if there is a single server there
* [Fix] Rspamadm grep: Disable Lua patterns in string search by default
* [Fix] Skip text parts when checking binary parts in fuzzy check
* [Fix] Support v2 checks in controller
* [Fix] Treat empty address as valid
* [Fix] Try harder to detect CTE
* [Fix] Try to deal with v4 mapped to v6 addresses on accept
* [Fix] Use dkim signing callback properly
* [Fix] Use non-volatile memory for storing data
* [Fix] Use static maps instead of ugly hack for radix_from_config
* [Fix] Use the same pool for related sessions
* [Rework] Continue modularisation for lua library
* [Rework] Initial milter protocol support
* [Rework] Make log pipes worker agnostic, add scanners API
* [Rework] Move authentication results generation to a separate routine
* [Rework] Move common DKIM functions to a separate lua module
* [Rework] Move global functions to a separate directory
* [Rework] Prepare dkim module for ARC checks
* [Rework] Propagate ucl variables from the command line
* [Rework] Remove multiple metrics support from Rspamd
* [Rework] Stop using name 'rmilter' for the modern protocol
* [Rework] Use LFU algorithm in LRU cache
* [Rules] Fix received TLS rules
* [Rules] Improve URL_COUNT_ODD rule
* [WebUI] Fix add header filter in history
* [WebUI] Use modern protocol for checking messages

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Feature] Add unigramms support in bayes
* [Feature] Allow configurable sign headers for DKIM
* [Feature] Allow to add unigramm metatokens from Lua
* [Feature] DKIM Signing: envelope match exception for local IPs
* [Feature] UCL: register parser variables from Lua
* [Fix] Always try to adjust filename
* [Fix] Do extra copy to ensure that original content is never touched
* [Fix] Fix SPOOF_REPLYTO rule
* [Fix] Ignore Rmilter added Received
* [Fix] More fixes for hashed email dnsbls
* [Fix] Plug memory leak in chartable module
* [WebUI] Display multiple alerts at once

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [CritFix] Fix classifier learning with Redis backend
* [CritFix] Fix issue when parsing encoded rfc822/messages
* [Feature] Add escaped version of lua_ucl import
* [Feature] Add task:headers_foreach function
* [Feature] Allow to process filenames from content type
* [Feature] Allow to query hashed emails
* [Feature] Ignore bayes with mostly metatokens or with too few text
* [Feature] Probabilistically skip metatokens
* [Feature] Retrieve all virus names from SAVAPI
* [Feature] Rework classifiers lua metatokens
* [Feature] Store headers order
* [Feature] Store text tokens inside bayes tokens
* [Feature] Use cached shingles keys
* [Fix] Add missing score normalisation for HFILTER_URL_ONLY
* [Fix] Avoid lookup in absent hash
* [Fix] Check return values from Lua functions called from C
* [Fix] Do not count sending and loading time in rspamc
* [Fix] Escape json strings for controller rejplies from Lua
* [Fix] Fix archive scans for savapi
* [Fix] Fix domain_only emails RBL
* [Fix] Fix ip_score map configuration
* [Fix] Fix JSON output for history_redis
* [Fix] Fix one character length substrings search
* [Fix] Fix parsing of non-RFC compatible Exim received
* [Fix] Fix parsing of options for workers with the same type
* [Fix] Fix processing of small tokens vectors
* [Fix] Fix rfc2047 tokenization
* [Fix] Fix typo
* [Fix] More fixes for inplace decoding
* [Fix] Try to avoid modifications of the original data
* [Fix] URL redirector: Fix call to is_redirector
* [Rework] Set token data as uint64_t instead of chars array
* [WebUI] Check if neighbours' history backend versions match
* [WebUI] Disable phrase connectors replacement in history filtering
* [WebUI] Disable phrase connectors replacement in symbols filtering
* [WebUI] Do not hide messages with bad subject, just replace it with '???'
* [WebUI] Fix error message
* [WebUI] Fix history v2 display
* [WebUI] Fix legacy history
* [WebUI] history: break To address lists on commas
* [WebUI] Increase default timeout to 20 seconds
* [WebUI] Save some history table space

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Add history_redis default configuration
* [Feature] Add spoofed rules
* [Feature] Add URL_IN_SUBJECT rule
* [Feature] Allow to get task's subject
* [Feature] Allow to specify maximum number of shots for symbols
* [Feature] Distinguish URLs found in Subject
* [Feature] Memoize LPEG grammars
* [Feature] Parse else parts in SA rules
* [Feature] Process subject for mixed characters
* [Feature] Resolve url chains in url_redirector module
* [Feature] Stat greylisted messages as greylisted not soft-rejected
* [Feature] Support checking for redirector in Lua SURBL
* [Feature] Support tag_exists SA function
* [Feature] Work with broken rfc2047 tokens
* [Fix] Check all watcher's dependencies
* [Fix] Do not compile hyperscan with no SSSE3 support
* [Fix] Do not crash if cannot decode qp encoded part
* [Fix] Fix dependencies of DKIM when multiple signatures are found
* [Fix] Fix lists in whitelist plugin
* [Fix] Fix one-shot symbols weight calculations
* [Fix] Fix options and shots match
* [Fix] Fix order of symbol options
* [Fix] Fix parsing of dot at the end of the address
* [Fix] Fix parsing of lua table arguments
* [Fix] Fix processing of subject words
* [Fix] Fix string split memoization
* [Fix] Fix templates grammar usage
* [Fix] Fix various issues related to Lua stack manipulation
* [Fix] Force actions: Use postfilter if we have honor_action / require_action
* [Fix] Further fixes to avoid PHISHING FP
* [Fix] Preserve order of options in symbols
* [Fix] Rspamadm grep: deal with unusually-formatted logs
* [Fix] Use hostname suffix when dealing with history
* [Rework] Remove outdated SA rules
* [WebUI] Add flexible columns
* [WebUI] Add footable
* [WebUI] Add sender, recipients and subject columns
* [WebUI] Allow message-id break
* [WebUI] Fix history clustering
* [WebUI] Fix history display
* [WebUI] Fix sorting
* [WebUI] Humanize sizes
* [WebUI] Initial move towards footable
* [WebUI] Remove datatables
* [WebUI] Replace `.values` method with `.map`
* [WebUI] Rework v2 symbols display
* [WebUI] Try to normalize frequencies
* [WebUI] Unbreak WebUI
* [WebUI] Use Footable to draw Throughput summary table

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Add composite for hacked wordpress phishing
* [CritFix] Fix base64 decoding when there are unparseable characters
* [Feature] Additional symbol metadata in metadata exporter
* [Feature] Add method to get protocol reply from Lua
* [Feature] Add symbols when tagged rcpt/sender are normalised
* [Feature] Add task:get_symbols_all() function
* [Feature] Allow multiple formats of DKIM signing key
* [Feature] Allow to cache and use flexible protocol reply
* [Feature] Allow to set one_shot flag from register_symbol
* [Feature] Allow to skip certain types of hashes when learning fuzzy
* [Feature] Cache and insert scan time into the protocol
* [Feature] Detect newlines in rspamc --mime
* [Feature] DKIM signing: support use of maps
* [Feature] Greylist: Support excluding low-scoring messages from greylisting
* [Feature] Implement lua history in controller
* [Feature] Implement redis history querying
* [Feature] Preliminary implementation of redis history plugin
* [Feature] Support using request headers in settings
* [Fix] Change default template to deal with non-ASCII characters
* [Fix] Deal with lists of maps in whitelist module
* [Fix] DKIM signing: use domain-specific signing key
* [Fix] Do not reallocate completed zstd buffer
* [Fix] Do not use local_addrs in proxy
* [Fix] Fix crash when resolver is undefined
* [Fix] Fix double free when closing lua_tcp connections
* [Fix] Fix for lua 5.3
* [Fix] Fix freeing of arrays iterators
* [Fix] Fix issue with task:get_symbol and symbols with no metric
* [Fix] Fix log line duplication in `rspamadm grep`
* [Fix] Fix memory corruption on termination
* [Fix] Fix out-of-bound access in base64 decode
* [Fix] Fix ratelimit + greylisting
* [Fix] Fix subject rewriting
* [Fix] Fix task:set_recipients function
* [Fix] Fix URI_COUNT_ODD rule
* [Fix] Follow the traditional symbols conventions in RCPT_COUNT rule
* [Fix] Greylist: Suppress greylist action for whitelisted hosts too
* [Fix] Metadata exporter: use rule-specific settings for emails
* [Fix] Properly set missing fields in exporter
* [Fix] Proxy: max_retries option
* [Fix] RCPT_COUNT fixes
* [Fix] Rework HAS_X_PRIO rule to match symbols conventions
* [Fix] Update issues in ac-trie
* [Fix] Use optimised base64 decoding in DKIM
* [WebUI] Add preliminary v2 history parser
* [WebUI] Allow different history parsers
* [WebUI] Display symbols
* [WebUI] Rework history v2 function

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Add default config for spamassasssin plugin
* [Conf] Add default configuration for antivirus module
* [Conf] Add dkim signing docs
* [Conf] Add mx_check default config
* [Conf] Add replies config
* [Conf] Add trie default config
* [Feature] Add heuristic to find text parts in files
* [Feature] Add rule to detect broken content type
* [Feature] Allow to extract CTE in Lua API
* [Feature] Allow to set from address for a lua_task
* [Feature] Allow to set recipients of a task from Lua
* [Feature] Enchance text_part:get_content method
* [Feature] Remove + aliases from emails
* [Feature] Support rmilter block and dkim signature in CGP helper
* [Feature] Support running event loop from Lua
* [Fix] Antivirus: use scanner-specific redis prefix
* [Fix] Couple of fixes for DKIM signing module
* [Fix] Distinguish missing and broken mandatory headers
* [Fix] Do more heuristical detection for missing CTE
* [Fix] Do not resort cache on each check
* [Fix] Fix CGP escaping
* [Fix] Fix MISSING_MIME_VERSION rule for plain messages
* [Fix] Fix parsing of cte in expressions
* [Fix] Fix partial matches in rspamadm grep
* [Fix] Fix setting class on style field
* [WebUI] Auto-switch Throughput units to `msg/min` for very low rate
* [WebUI] Update D3Evolution to 0.0.2

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [CritFix] Fix processing of stop_patterns with `\0` character
* [CritFix] Fix setting of raw key for signing
* [Fix] Fix lua exports from plugins during reload
* [Fix] Fix prefilters action scores
* [Fix] Fix symbols processing order
* [Minor] Help cmake find gthread
* [Minor] Some cmake fixes

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Conf] Add configurations for asn, clickhouse and dcc
* [Conf] Add default config for url redirector plugin
* [Conf] Add the default config for greylist module
* [Conf] Allow to edit all local maps from WebUI by default
* [CritFix] Deal with absent headers in DKIM
* [CritFix] Do not trust remote shingles count
* [CritFix] Fix bad memory leak in TLS certificates validation
* [CritFix] Fix critical memory issues with radix maps
* [CritFix] Fix descriptors leak on reload
* [CritFix] Fix headers selection in DKIM verification
* [CritFix] Fix parsing of boundaries that end with `--`
* [CritFix] Repair PTR_ARRAY_FOREACH macro
* [Feature] Add CORS support to the controller
* [Feature] Add FROM_NAME_EXCESS_SPACE rule
* [Feature] Add REPLYTO_EMAIL_HAS_TITLE rule
* [Feature] Add `caseless_hash` method to `lua_util`
* [Feature] Add `rip` keyword to ratelimit module
* [Feature] Add a simple benchmark for content type parsing
* [Feature] Add boundaries parsing in content type
* [Feature] Add charset detection for text parts
* [Feature] Add content disposition parser
* [Feature] Add fallback if too many updates are failing
* [Feature] Add function to convert struct tm to time using timezone
* [Feature] Add function to normalize HTTP paths
* [Feature] Add fuzzy collection plugin
* [Feature] Add fuzzy logic for images
* [Feature] Add gmime parser to mime_tool
* [Feature] Add heuristic to detect broken messages
* [Feature] Add heuristic to find displayed URLs
* [Feature] Add heuristic to process broken email addresses
* [Feature] Add images normalization
* [Feature] Add mechanism for disabling composites (Fixes #1270)
* [Feature] Add method to create regexp from a glob pattern
* [Feature] Add mime encoding manipulation routines
* [Feature] Add mime tool to explore messages
* [Feature] Add more meta tokens from received headers
* [Feature] Add neighbours option to support Rspamd cluster in WebUI
* [Feature] Add new function to parse mime addresses
* [Feature] Add new methods for lua_tcp
* [Feature] Add own headers decoding routine
* [Feature] Add own routine to generate a message id
* [Feature] Add parser for SMTP date
* [Feature] Add per-task lua cache to reuse 'heavy' objects
* [Feature] Add plugins list path in WebUI
* [Feature] Add preliminary multipart support
* [Feature] Add preliminary version of DKIM signing module
* [Feature] Add profiling support in client output
* [Feature] Add rfc2047 grammar
* [Feature] Add rfc2047 variant for QP decoding
* [Feature] Add rmilter_headers module (Fixes #1227)
* [Feature] Add sse42 version of base64 decoding
* [Feature] Add ssse3 and avx2 base64 decoders
* [Feature] Add support of libgd
* [Feature] Add the preliminary version of redirects resolver in Lua
* [Feature] Add ucl_object_iterate_full function
* [Feature] Add url encoding function
* [Feature] Allow SOA requests in lua dns
* [Feature] Allow custom parse types in lua ucl
* [Feature] Allow plugins to register webui handlers
* [Feature] Allow to add options explicitly to symbols
* [Feature] Allow to call a callback when symbol frequency is on peak
* [Feature] Allow to call redirector script from SURBL
* [Feature] Allow to create variable length dkim keys
* [Feature] Allow to have module specific options for Redis in plugins
* [Feature] Allow to pass sign key directly from Lua
* [Feature] Allow to register configuration docs from Lua API
* [Feature] Allow to return options as a table
* [Feature] Allow to set peak callbacks from Lua
* [Feature] Allow to specify custom method for a message
* [Feature] Allow to store dkim keys in Redis
* [Feature] Allow to store messages in files
* [Feature] Apply DCT using AAN for fuzzy signature
* [Feature] Avira SAVAPI support
* [Feature] Cache and simplify DCT and jpeg decode
* [Feature] Cache libicu convertors
* [Feature] Detect URLs with suspicious omographs
* [Feature] Do not increase score for duplicate options
* [Feature] Do not trust CTE, check base64 and qp strictly
* [Feature] Dynamic reputation in URL reputation plugin
* [Feature] Extend redis lock when learning spawned
* [Feature] Filter non-utf chars from all decoded headers
* [Feature] Fix phishing detection for IDNA urls
* [Feature] Ignore bad symbols on base64 decoding
* [Feature] Ignore too wide elements in SPF
* [Feature] Implement fuzzy collection mode
* [Feature] Implement helo maps in multimap
* [Feature] Implement human readable buckets configuration
* [Feature] Implement min-hash shingles for DCT data from images
* [Feature] Implement new algorithm for fuzzy hashes of images
* [Feature] Implement new unicode normalizer
* [Feature] Implement quoted printable decoding
* [Feature] Implement received headers flags
* [Feature] Implement rspamdgrep tool
* [Feature] Implement sane checksum for config file
* [Feature] Implement url tags concept
* [Feature] Improve detection of omographs using libicu
* [Feature] Improve url redirector module
* [Feature] Multimap: Received header processing
* [Feature] Multiple improvements in the maps
* [Feature] New URL filters in multimap
* [Feature] Plugin to force actions on selected symbols
* [Feature] RBL module: support hashing for emails and helo RBL
* [Feature] Reuse URL tags in SURBL module
* [Feature] Rework RRD ds count, add conversion path
* [Feature] Rework surbl module to avoid extra redirector calls
* [Feature] Send config id to the WebUI
* [Feature] Simplify HTTPCrypt client support
* [Feature] Skip processing for large images
* [Feature] Start collection only mode implementation for fuzzy storage
* [Feature] Start import of the optimized base64 decode
* [Feature] Store all received headers in lua
* [Feature] Store relational order of all headers in a message
* [Feature] Support DKIM signing in Lua plugins
* [Feature] Support HTTPCrypt client in lua_http
* [Feature] Support setting SMTP message in multimap
* [Feature] Support setting metric subject from Lua
* [Feature] Support setting subject in force actions module
* [Feature] Treat v6 mapped addresses as v4 addresses
* [Feature] URL reputation plugin
* [Feature] Use Redis instead of memcached in URLs redirector
* [Feature] Use Rspamd rfc2047 decoder instead of gmime one
* [Feature] Use a different normalization for fuzzy images
* [Feature] Use normalized images in fuzzy hashes
* [Feature] Use own code for parsing of date
* [Feature] Use shingles for images fuzzying
* [Feature] Use t1ha for hashes, allow inlining
* [Feature] Use t1ha instead of metrohash and xxhash32
* [Feature] Various new features in metadata exporter module
* [Feature] rmilter_headers: authentication-results (#78)
* [Fix] Add additional check to mark redis connection inactive
* [Fix] Add packed attribute for protocol structure
* [Fix] Adopt OMOGRAPH_URL rule
* [Fix] Allow static maps
* [Fix] Allow to disable classifiers checks using settings and conditions
* [Fix] Another try to fix 0 length maps
* [Fix] Another try to fix corruption during maps reload
* [Fix] Another try to fix descriptors leak
* [Fix] Another try to fix reload and logger
* [Fix] Antivirus module: register virtual symbols for patterns
* [Fix] Avoid extensive reallocs
* [Fix] Avoid mempool leak in SA plugin on reload
* [Fix] Avoid race condition on saving cache and reload
* [Fix] Avoid reusing g_error (Fixes #1262)
* [Fix] Break pool connection on fatal redis errors
* [Fix] Check for NaN properly
* [Fix] Couple of fixes for date parsing
* [Fix] Date header timezone adjustments (#1279)
* [Fix] Deal with EOF properly
* [Fix] Decode filename in content disposition
* [Fix] Disable fuzzy images by default
* [Fix] Disable zero-copy mode for text parts to avoid crashes
* [Fix] Do not destroy session when not all finish scripts are done
* [Fix] Do not greyscale images
* [Fix] Do not leave parent-less workers processes on fatal errors
* [Fix] Do not lowercase Content-Disposition to perform decoding
* [Fix] Do not penalize characters just after numeric prefix
* [Fix] Do not refork workers that are intended to die
* [Fix] Do not set pre-result and update records for no `Queue-ID` messages
* [Fix] Do not skip post-filters when pre-filters have set some results
* [Fix] Do not stop symbols planning if async events are pending
* [Fix] Do not try to set keys for unencrypted requests in proxy
* [Fix] Encode URLs according to rfc3986
* [Fix] Encode URLs before sending them to the protocol
* [Fix] Filter bad characters from message id
* [Fix] Fix CTE detection heuristic
* [Fix] Fix Content-Type in HTTP requests
* [Fix] Fix IDN eslds phishing checks
* [Fix] Fix adding maps from config in Lua
* [Fix] Fix another reload memory issue
* [Fix] Fix argument returned on redis backend errors
* [Fix] Fix assertion in graph handling
* [Fix] Fix body trie matching
* [Fix] Fix build
* [Fix] Fix byte array expansion during toutf8 conversion
* [Fix] Fix charset normalisation
* [Fix] Fix checking of DKIM bodies that needs just `\n` to be added
* [Fix] Fix couple of cornercases with email addresses
* [Fix] Fix couple of issues
* [Fix] Fix dependencies tracking for callback symbols
* [Fix] Fix detection of jpeg size
* [Fix] Fix errors handling in fuzzy backend initialization
* [Fix] Fix fuzzy hashes count
* [Fix] Fix globbing and convert lists to arrays in fuzzy_check
* [Fix] Fix heuristical CTE detection for QP encoding
* [Fix] Fix ignoring of bad text parts
* [Fix] Fix indexes in array access, interleave loop
* [Fix] Fix int64 -> double conversion
* [Fix] Fix invalid memory access on reload
* [Fix] Fix issues with empty updates
* [Fix] Fix issues with quoted-printable encoding
* [Fix] Fix keys names
* [Fix] Fix lots of issues in mime parser code
* [Fix] Fix lua maps load
* [Fix] Fix macro name
* [Fix] Fix mas group score calculations
* [Fix] Fix matching of the same patterns from different tries
* [Fix] Fix memory corruprtion and leak
* [Fix] Fix memory leak in HTTP maps
* [Fix] Fix memory leak in expression destroying
* [Fix] Fix memory leak in parsing of mime names
* [Fix] Fix memory leak in safe ucl iterators
* [Fix] Fix memory leak on reload in plugins
* [Fix] Fix modules reconfigure on reload
* [Fix] Fix monitored setup fro URLBLs with IP addresses
* [Fix] Fix name of var
* [Fix] Fix new rrd updates
* [Fix] Fix out of bounds access
* [Fix] Fix parsing messages with no body
* [Fix] Fix parsing of '=' character in headers
* [Fix] Fix parsing of messages with no content type
* [Fix] Fix plugins callbacks in webui
* [Fix] Fix possible memory corruption in redis pool
* [Fix] Fix probability calculations for fuzzy redis backend
* [Fix] Fix processing errors in lua_tcp
* [Fix] Fix processing of emails with name only
* [Fix] Fix processing of non-multipart messages
* [Fix] Fix processing of parts with no valid content type
* [Fix] Fix race condition in SIGUSR2 handler
* [Fix] Fix redis options parsing when no redis servers are defined
* [Fix] Fix reload and hyperscan ready event
* [Fix] Fix reload memory issue
* [Fix] Fix rra_ptr conversion
* [Fix] Fix rrd file conversion
* [Fix] Fix setting of content-type attributes
* [Fix] Fix signing headers creation in DKIM
* [Fix] Fix stddev calculations
* [Fix] Fix surbl plugin to work with composite maps
* [Fix] Fix timezones parsing
* [Fix] Fix tokens usage
* [Fix] Fix urls and emails hashes
* [Fix] Fix usage of unsafe ucl iterators
* [Fix] Fix work with broken utf8 tokens
* [Fix] Fix writing of user to roll history
* [Fix] Forgotten worker
* [Fix] Further memory leaks fixes
* [Fix] Ignore lua metatokens in bayes for now
* [Fix] Improve OMOGRAPH_URL rule
* [Fix] Lua IP from string should be invalid if parsing failed
* [Fix] Miltiple fixes to new lua_tcp, add debugging
* [Fix] More fixes for iterators cleanup
* [Fix] More fixes to logger initialization
* [Fix] More heuristic fixes for phishing detection
* [Fix] More leaks eliminated
* [Fix] More leaks...
* [Fix] More random fixes for reload...
* [Fix] Multimap: Fixes for email filters
* [Fix] Multiple fixes for fann module
* [Fix] Multiple memory corruption fixes
* [Fix] Normalize path in HTTP router
* [Fix] Plug memory leak
* [Fix] Plug memory leak in adding radix trees
* [Fix] Plug memory leak in configuration parser
* [Fix] Plug memory leak in expressions parsing during reload
* [Fix] Plug memory leak in learning fuzzy storage
* [Fix] Plug memory leak in lua_tcp
* [Fix] Plug reload leaks
* [Fix] Plug termination memory leaks
* [Fix] Really increase lock lifetime
* [Fix] Replies module: fix symbol weight
* [Fix] Restore content type params related functions
* [Fix] Set task's subject from mime subject
* [Fix] Sigh, one more reload leak
* [Fix] Simplify images shingles
* [Fix] Some more memory issues are fixed
* [Fix] Stop hardcoding of lua in C
* [Fix] Stop processing of bad parts as text parts
* [Fix] Strictly filter bad characters when emittin json
* [Fix] Strings returned from lua are ephemeral
* [Fix] Support unix sockets for lua redis
* [Fix] Try to fix issues with reloading config
* [Fix] Try to fix race condition in redis_pool
* [Fix] Use checksum to avoid intersection between different ANNs
* [Fix] Use rspamd hashes in embedded ucl
* [Fix] Use sane default rewrite subject (*** SPAM *** %s)
* [Fix] Various collection mode fixes
* [Fix] Various fixes to mime parser
* [Fix] Various reload leak fixing
* [Fix] Whitelist certain extensions from archive checks
* [Rework] Add preliminary implementation of the mime parser
* [Rework] Adopt code for the new options
* [Rework] Change logger setup interface
* [Rework] Composite configuration (#1270)
* [Rework] Finally remove gmime dependency from Rspamd
* [Rework] Further fixes to symbols frequencies
* [Rework] Implement content type parser for mime
* [Rework] Kill all InternetAddressList usages
* [Rework] Multiple fixes for symbols cache statistics
* [Rework] Refactor struct names
* [Rework] Rework images fuzzy hashes algorithm
* [Rework] Rework lua_tcp to allow TCP dialog
* [Rework] Start massive rework to get rid of gmime
* [Rework] Start new approach for multiparts parsing
* [Rework] Start rework of mime addresses
* [Rework] Start rework of symbols cache updates
* [Rework] Start switching to libicu
* [Rework] Use a special structure for stats tokens
* [Rework] Use hash tables for symbols options
* [Rework] Use libicu instead of iconv for conversions
* [Rework] Use new scheme to parse mime parts
* [WebUI] Add Access-Control-Allow-Origin for cluster management
* [WebUI] Add Throughput graph autorefreshing (#820)
* [WebUI] Add Visibility.js library
* [WebUI] Add basic cluster support to Throughput tab
* [WebUI] Add graph legend entries for new DSes
* [WebUI] Add graph tab
* [WebUI] Add neighbours RRD data consolidation
* [WebUI] Add preliminary save symbols clustering
* [WebUI] Add server selector to navbar
* [WebUI] Add soft reject to auth stats
* [WebUI] Add summary to the Throughput tab
* [WebUI] Allow to save maps on the cluster
* [WebUI] Avoid extra graph redraw and alerts glitching
* [WebUI] Be more generous with AJAX timeout
* [WebUI] Disable error ring loading in `read only` mode
* [WebUI] Enclose table header cells with `tr`s
* [WebUI] Finish interface rework
* [WebUI] Fix RRD summary pie chart position
* [WebUI] Fix `All SERVERS` graph fot just one available server
* [WebUI] Fix case when no cluster is defined
* [WebUI] Fix compatibility with non-ES6 compliant browsers
* [WebUI] Fix config ID
* [WebUI] Fix configuration page partially
* [WebUI] Fix disabled state
* [WebUI] Fix graph dataset selector initialization
* [WebUI] Fix graph selectors state resetting
* [WebUI] Fix mouse events on throughput summary table area
* [WebUI] Fix multiple JS issues
* [WebUI] Fix pie chart displaying
* [WebUI] Fix read only
* [WebUI] Fix read only2
* [WebUI] Fix retarded datatables
* [WebUI] Fix soft reject in pie chart
* [WebUI] Fix stat widgets timers multiplication on `Refresh` click
* [WebUI] Fix symbols config
* [WebUI] Fix various errors with login form
* [WebUI] Further fixes
* [WebUI] Hide learning tab in read-only mode
* [WebUI] Initial clusters support
* [WebUI] Make legend entry colours more contrast
* [WebUI] Move configuration tab to a separate module
* [WebUI] Move history tab
* [WebUI] Move symbols config as well
* [WebUI] New sec to time function
* [WebUI] Prevent multiple clicks on `Refresh`
* [WebUI] RRD summary: Hide inner labels of tiny pie sectors
* [WebUI] RRD summary: Respect undefined values
* [WebUI] Reduce font size of graph's legend
* [WebUI] Remove orphaned font duplicates
* [WebUI] Remove unused code
* [WebUI] Replace spinner with animated glyphicon
* [WebUI] Reset refresh timer on server switching
* [WebUI] Rework interface to use requirejs
* [WebUI] Rework neighbours query function
* [WebUI] Separate attributes by space
* [WebUI] Set focus to password field (#1230)
* [WebUI] Simplify neighbours table populating
* [WebUI] Start rework of modules
* [WebUI] Stop stats refreshing if the page is hidden
* [WebUI] Turn d3pie's stuff into a reusable function,
* [WebUI] Unify send data functions
* [WebUI] Update D3Evolution to 0.0.1
* [WebUI] Update d3.js
* [WebUI] Update datatables to work with the requirejs
* [WebUI] Use unified tab click event handler,
* [WebUI] clusters for the chart
* [WebUI] fix uptime

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [CritFix] Deal with absent headers in DKIM
* [CritFix] Do not trust remote shingles count
* [CritFix] Fix headers selection in DKIM verification
* [Feature] Add EXT_CSS rule
* [Feature] Add toggle for disabling SURBLs
* [Feature] Extend redis lock when learning spawned
* [Feature] Parse <link> HTML tags
* [Fix] Avoid reusing g_error (Fixes #1262)
* [Fix] Do not reset loaded ANN when learning is requested
* [Fix] Fix another issue with external deps in SA
* [Fix] Fix body trie matching
* [Fix] Fix checking of DKIM bodies that needs just `\n` to be added
* [Fix] Fix fuzzy hashes count
* [Fix] Fix keys names
* [Fix] Fix length calculations for url encoded urls
* [Fix] Fix matching of the same patterns from different tries
* [Fix] Fix name of var
* [Fix] Fix parsing of URLs with spaces and other bad chars
* [Fix] Fix probability calculations for fuzzy redis backend
* [Fix] Fix signing headers creation in DKIM
* [Fix] Plug memory leak
* [Fix] Really fix chained SA dependencies
* [Fix] Really increase lock lifetime
* [Fix] Use checksum to avoid intersection between different ANNs
* [Fix] Use rspamd hashes in embedded ucl
* [Fix] Yet another change for testing external deps

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Feature] ASN support in Clickhouse module
* [Feature] Add clickhouse plugin
* [Feature] Add generic tool to add universal maps for lua modules
* [Feature] Add logger.debugm to debug lua modules
* [Feature] Allow to register metrics symbols using register_symbol
* [Feature] Allow to specify prefix for fann_redis
* [Feature] Clickhouse: support different masks for IPv4/IPv6
* [Feature] Support forcing action in antivirus plugin
* [Fix] Add handling of regexp maps
* [Fix] Allow backslashes in http urls
* [Fix] Avoid mapping of empty files
* [Fix] Do not load tld file to speed up rspamadm
* [Fix] Do not resolve numeric IP addresses due to ipv6 insanity
* [Fix] Filter incorrect training data
* [Fix] Fix Fuzzyconvert tool when password or DB is given
* [Fix] Fix build with custom glib/gmime
* [Fix] Fix converting of learn count from sqlite to redis
* [Fix] Fix crashes with invalid received and task:set_from_ip
* [Fix] Fix external dependencies for SA module
* [Fix] Fix fann_redis when number of scores has been changed
* [Fix] Fix hyperscan usage for non compatible platforms
* [Fix] Fix loading of maps from UCL objects
* [Fix] Fix memory leak for task-less redis requests
* [Fix] Fix mid module with new maps syntax
* [Fix] Fix parsing of URLs with username
* [Fix] Fix re cache initialisation
* [Fix] Fix replacements to sanitize '%' character
* [Fix] Fix set and regexp like static maps
* [Fix] Fix some issues in redis settings
* [Fix] Fix static IP maps
* [Fix] Fix total learns counter for redis stats
* [Fix] Fix usage of config during reload
* [Fix] Fix various warnings and issues
* [Fix] Invalidate ANN if training data is incorrect
* [Fix] Miltiple fixes to fann_redis module
* [Fix] More fixes for URLs with backslashes
* [Fix] Properly get options for ip_score module
* [Fix] Relax requirements for Received as gmail cannot RFC
* [Fix] Remove or fix hyperscan incompatible regexps
* [Fix] Settings: correctly read redis config
* [Rework] Rework lua logger interface slightly
* [Rework] Use new maps add function

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [CritFix] Add guards for inactive redis connections
* [CritFix] Another fix for proxying files using rspamd_proxy
* [CritFix] Cleanup inactive redis connections
* [CritFix] Do not sometimes try to exec posfilters before classification
* [CritFix] Fix application of IPv6 mask
* [CritFix] Fix chunked encoding when reading messages
* [CritFix] Fix file mode for rspamd_proxy
* [CritFix] Fix hyperscan compilation on regexp change
* [CritFix] Fix issue with finding of end of lines pointers
* [CritFix] Fix iteration over headers array (introduced in 1.4)
* [CritFix] Fix processing of learned tokens count for redis backend
* [CritFix] Fix race condition in checking of cached maps
* [CritFix] Fix workers scripts by sharing workers configs
* [CritFix] Introduce raw content to text parts
* [CritFix] Plug memory leak and potential memory corruption
* [Feature] Adaptive ratelimits
* [Feature] Add ASN -> rbldnsd script for asn.rspamd.com
* [Feature] Add DMARC_NA symbol
* [Feature] Add F-Prot support to antivirus module
* [Feature] Add HTTP backend to metadata exporter
* [Feature] Add Lua API module for monitored objects
* [Feature] Add R_DKIM_NA / R_SPF_NA / AUTH_NA symbols
* [Feature] Add R_DKIM_PERMFAIL symbol
* [Feature] Add R_SPF_PERMFAIL symbol
* [Feature] Add Sophos antivirus support
* [Feature] Add ZSTD compression to Lua API
* [Feature] Add `mid` Lua module
* [Feature] Add `one_param` flag for metric symbols
* [Feature] Add a generic lua classifier
* [Feature] Add a very basic interface to access workers data from on_load
* [Feature] Add ability to delete a hash by its data to fuzzy_check plugin
* [Feature] Add ability to enable/disable symbols via dynamic_conf
* [Feature] Add ability to lookup settings by key
* [Feature] Add common way to disable Lua modules
* [Feature] Add compression support to rspamd client
* [Feature] Add condition to do antiviral check
* [Feature] Add configuration for lua classifiers
* [Feature] Add configuration knobs for the errors circular buffer
* [Feature] Add decompression support in rspamd client
* [Feature] Add errors exporter to the controller
* [Feature] Add expected value for monitored DNS resources
* [Feature] Add exporter from error ringbuf to ucl
* [Feature] Add extended version for fann creation function
* [Feature] Add ffi friendly version of process_regexp function
* [Feature] Add frequency and time display to webui
* [Feature] Add fuzzy_delhash command to rspamc client
* [Feature] Add implementation of redis connections pool
* [Feature] Add latency and offline time monitoring
* [Feature] Add learning support for lua classifiers
* [Feature] Add max-size and timeout options to CGP helper
* [Feature] Add method to enable/disable symbols in config
* [Feature] Add methods to get metric's actions and symbols from Lua
* [Feature] Add mmap support to lua_text
* [Feature] Add monitored object for surbl plugin
* [Feature] Add more exceptions to surbl whitelist
* [Feature] Add more meta-tokens to bayes
* [Feature] Add neural net classifier to fann_scores module
* [Feature] Add neural net serialization/deserialization
* [Feature] Add new dynamic conf module
* [Feature] Add periodic events support for lua_config
* [Feature] Add plugin to check MX'es for the sender's domain
* [Feature] Add preliminary monitored module
* [Feature] Add preliminary support of dynamic conf updates in Redis
* [Feature] Add preliminary version of clamav plugin
* [Feature] Add redis cache to asn module
* [Feature] Add replies compression
* [Feature] Add spamhaus DROP dnsbl
* [Feature] Add support for dictionary in client compression
* [Feature] Add support for fuzzy learn and unlearn from lua
* [Feature] Add support for input encryption
* [Feature] Add support of min_learns to neural net classifier
* [Feature] Add termination callbacks for workers
* [Feature] Add user-agent for rspamc
* [Feature] Add utility to perform classifier tests
* [Feature] Add zstd compression library
* [Feature] Allow HTTPS requests in lua_http
* [Feature] Allow conditions for pre and postfilters
* [Feature] Allow custom functions for ratelimits
* [Feature] Allow for excluding messages from AV scanning based on size
* [Feature] Allow for getting worker stats from Lua
* [Feature] Allow getting task UID from Lua
* [Feature] Allow parsing of mailbox messages from the commandline
* [Feature] Allow plugins to publish their lua API via rspamd_plugins
* [Feature] Allow to compare other systems with Rspamd
* [Feature] Allow to execute Lua scripts by controller
* [Feature] Allow to have a function to set custom greylist message
* [Feature] Allow to iterate over multiple tags
* [Feature] Allow to pass extra data from plugins to log helper
* [Feature] Allow to plan new periodics at different time
* [Feature] Allow to reset hashes
* [Feature] Allow to run rspamadm lua just as a lua interpreter
* [Feature] Allow to store settings in redis
* [Feature] Allow to update dynamic conf in Redis
* [Feature] Allow to use dictionaries for compression
* [Feature] Allow to use md5, sha1, sha256, sha384 and sha512 hashes in Lua
* [Feature] Allow whitelisting by IP for greylisting plugin
* [Feature] Antivirus: Support whitelists & pattern-matching sig names
* [Feature] Backport pack/unpack routines from Lua 5.3
* [Feature] Check settings with equal priopities in alphabetical order
* [Feature] Compress neural net in redis
* [Feature] Consider more tags when doing WHITE_ON_WHITE rule
* [Feature] Descriptive options for DMARC failure symbols
* [Feature] Descriptive options for RBL symbols
* [Feature] Enable configuration for monitored objects
* [Feature] Execute on_load scripts with ev_base ready
* [Feature] Fann scores now uses metadata from a message
* [Feature] Implement FANN threaded learning
* [Feature] Implement classifying for lua classifiers
* [Feature] Implement finish scripts for worker processes
* [Feature] Implement monitoring for DNS resources
* [Feature] Implement real priorities for pre and post filters
* [Feature] Insert two symbols: FANN_HAM and FANN_SPAM instead of one
* [Feature] Module to push metadata/messages to redis pubsub
* [Feature] Monitor RBL records
* [Feature] Move fann_classifier to a separate plugin
* [Feature] Normalize all ANN inputs
* [Feature] Preliminary version of metric exporter module
* [Feature] Preserve decompression context between tasks
* [Feature] Ratelimit: Support dynamic bucket size/leak rate
* [Feature] Relax FORGED_RECIPIENTS: allow senders to BCC themselves
* [Feature] Remove symbols weights on composites processing
* [Feature] Return symbol scores when getting resulting symbols
* [Feature] Rework lua tcp module
* [Feature] Rule to detect some obvious X-PHP-Originating-Script forgeries
* [Feature] Rule to identify some X-PHP-Script forgeries
* [Feature] Rules for scoring Google Message-ID fixes
* [Feature] Send hashes values to reply
* [Feature] Set expire for dmarc reports
* [Feature] Stop using cymru zone as it is unstable
* [Feature] Stop using of GLists for headers, improve performance
* [Feature] Store `for` in task:get_received_headers
* [Feature] Store `for` part in received headers
* [Feature] Store enabled flag for webui session
* [Feature] Store error messages in ring buffer
* [Feature] Support compressed maps
* [Feature] Support excluding selected users from ratelimits
* [Feature] Support looking up NS records in lua_dns
* [Feature] Support modern style SURBL configuration
* [Feature] Support multiple hashes in delhash path
* [Feature] Support new messages in rspamc
* [Feature] Support requests without reads in lua_tcp
* [Feature] Support setting task message from Lua
* [Feature] Track visibility of HTML elements
* [Feature] Try to add CRLF when checking DKIM
* [Feature] Try to guess line endings when folding headers
* [Feature] Try to improve normalization function for bayes
* [Feature] Use FFI to optimize SA module
* [Feature] Use length based arguments for redis, allow lua_text as arg
* [Feature] Use more layers for fann and another normalization
* [Feature] User-defined ratelimits
* [Feature] Utility to convert fuzzy storage from sqlite to redis
* [Feature] Yield DMARC_DNSFAIL on lookup failure
* [Fix] Adopt fuzzy storage for flexible backends
* [Fix] Allow plain IP addresses in Rspamd maps
* [Fix] Another fix for brain-damaged hiredis
* [Fix] Another fix for rdns write errors
* [Fix] Another fix for rdns_make_request_full invocation
* [Fix] Another fix in DKIM canonicalization
* [Fix] Another memory leak plugged
* [Fix] Another try to deal with posix idiotizm
* [Fix] Another try to fix RDNS events processing logic
* [Fix] Avoid double frees in HEAD requests
* [Fix] Avoid extra symbols for RBLs
* [Fix] Banish table.maxn from Lua parts
* [Fix] Check for socket error before connection in lua_tcp
* [Fix] Correctly propagate redis timeouts to Lua
* [Fix] Do not add extra newline in MIME mode
* [Fix] Do not be cheated by system hiredis
* [Fix] Do not classify when a message has not enough tokens
* [Fix] Do not crash on redis errors
* [Fix] Do not distinguish NXDOMAIN and NOREC for monitored
* [Fix] Do not replan retransmits if merely one server is defined
* [Fix] Do not use headers to calculate messages digests
* [Fix] Don't force action in replies module for authenticated users/local networks
* [Fix] Explicitly ban default passwords in webui
* [Fix] Finally fix ambiguity between parsed and resolved spf elts
* [Fix] Fix 'decoded' value in task:get_header_full()
* [Fix] Fix DKIM calculations
* [Fix] Fix DKIM signing for messages with no newline at the end
* [Fix] Fix DNS request in monitored
* [Fix] Fix DNS write errors processing
* [Fix] Fix HTTP methods other than GET and POST
* [Fix] Fix PERMFAIL for v6/v4 ambiguities
* [Fix] Fix absurdic scores for HFILTER_URL_ONLY
* [Fix] Fix actions in rolling history
* [Fix] Fix actrie patterns
* [Fix] Fix applying of lua dynamic confg
* [Fix] Fix autolearning errors and redis cache
* [Fix] Fix bayes learn_condition
* [Fix] Fix build with the recent OpenSSL
* [Fix] Fix caching and compressed maps
* [Fix] Fix check plain text part
* [Fix] Fix crash on OpenBSD in `url_email_start`
* [Fix] Fix double free in SPF
* [Fix] Fix extraction of shingles from redis fuzzy storage
* [Fix] Fix false sharing for symbols in the cache
* [Fix] Fix float usage in util:get_time
* [Fix] Fix folding algorithm to deal with empty tokens
* [Fix] Fix format string
* [Fix] Fix format string usage in controller errors handling
* [Fix] Fix handling of '\0' in lua_tcp
* [Fix] Fix handling of HTTP HEAD methods
* [Fix] Fix hash creation
* [Fix] Fix hiredis stupidity
* [Fix] Fix implicit settings module settingsup
* [Fix] Fix interaction with lua GC to avoid craches
* [Fix] Fix ip_score module registration
* [Fix] Fix issue with empty messages and dkim
* [Fix] Fix issues with CGP helper
* [Fix] Fix issues with the recent SPF changes
* [Fix] Fix key name to load ANN correctly
* [Fix] Fix lua tcp module by saving `do_read` in callback data
* [Fix] Fix memory leak in client when using compression
* [Fix] Fix min_learns option
* [Fix] Fix on_finish scripts and async handlers
* [Fix] Fix options for SPF dnsfail symbol
* [Fix] Fix parsing includes and redirects in SPF
* [Fix] Fix parsing of lua comments with empty lines
* [Fix] Fix parsing of unquoted HTML attributes
* [Fix] Fix periodic events and redis
* [Fix] Fix processing of fuzzy learns from Lua
* [Fix] Fix processing of redirect in SPF includes
* [Fix] Fix processing of symbols when reject limit is reached
* [Fix] Fix refcounts when map is specified by IP
* [Fix] Fix rspamd{session} class in Lua API
* [Fix] Fix setting ratelimit key for 'ip' bucket
* [Fix] Fix some cases of TLD urls detector
* [Fix] Fix statconvert tool
* [Fix] Fix stats for backend-less classifiers
* [Fix] Fix training script for fann_redis
* [Fix] Fix variable in ann module
* [Fix] Fix various errors in lua dynamic conf plugin
* [Fix] Forget old ANN when max_usages is reached to avoid overtrain
* [Fix] Further canonicalization fixes
* [Fix] Further fixes for fann_redis prefixes
* [Fix] Handle failures for inactive pooled connections
* [Fix] Improve multimap info message
* [Fix] More fixes in ANN loading
* [Fix] More fixes to fann_redis
* [Fix] More issues in fann_redis
* [Fix] More spaces fix in DKIM signature
* [Fix] Multiple fixes to asn script, add IPv6 support
* [Fix] Multiple issues in fann_redis
* [Fix] No greylist rejected messages
* [Fix] One more attempt to fix lua_redis
* [Fix] One more check for readdir...
* [Fix] Params should be treated as a hash
* [Fix] Plug memory leak in regexp desctructor
* [Fix] Process headers only once
* [Fix] Properly handle nil values in ratelimit plugin
* [Fix] Really fix redis shingles check
* [Fix] Remove fann with incorrect layers count
* [Fix] Remove mentions of deleted include
* [Fix] Remove some incompatible functions
* [Fix] Settings: fix `authenticated` parameter (#886)
* [Fix] Skip MX check for authenticated users and local networks
* [Fix] Slightly fix ANN routines
* [Fix] Stop caching records with DNS failures
* [Fix] Treat all errors in redis_pool as fatal errors for a connection
* [Fix] Try avoid false-positives in HEADER_FORGED_MDN rule
* [Fix] Try to avoid race condition when using rrd
* [Fix] Try to reload redis scripts if they are missing
* [Fix] Unbreak once_received skipping for local networks
* [Fix] Unlock ANN on error
* [Fix] Use memmove for overlapping regions
* [Fix] Use real size instead of displayed for core limits
* [Fix] Use the correct macro to get the size of control
* [Fix] Various fixes for errors ringbuffer
* [Fix] Yield R_SPF_DNSFAIL if lookup of included record fails
* [Fix] mid: fix map initialization
* [Fix] mid: handle incorrect rgexps in the map
* [Rework] Add extract training data function to fann_redis
* [Rework] Add preliminary train tests
* [Rework] Add redis storage feature to fann_redis
* [Rework] Adopt fuzzy storage for abstract backend
* [Rework] Adopt plugins
* [Rework] First reiteration on fann scores
* [Rework] Implement loading/invalidating
* [Rework] Make lua_redis task agnostic
* [Rework] Make rspamd protocol messages useful
* [Rework] Massive removal of legacy code
* [Rework] More cleanup actions
* [Rework] Remove legacy code never used for classifiers
* [Rework] Remove outdated and unused lua_session module
* [Rework] Reorganize fuzzy backend structure
* [Rework] Reorganize the internal backend structure
* [Rework] Restore old fann_scores, move common parts
* [Rework] Rework and simplify rbl plugin
* [Rework] Rework parsing of DMARC records

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Feature] ASN module; support matching ASN/country in multimap
* [Feature] Add SPF method in spf return result
* [Feature] Add Yandex and Mail.ru forwarding rules
* [Feature] Add mempool maps in multimap
* [Feature] Add rule for identifying mail sent by eval()'d PHP code
* [Feature] Add support of stub DNSSEC resolver to rdns
* [Feature] Add task:get_digest method
* [Feature] Allow for more fine-grained scoring for ip_score
* [Feature] Allow to get digest of a mime part from lua
* [Feature] Allow to print message digest in logs
* [Feature] Fold DKIM-Signature header
* [Feature] Implement encrypted logs
* [Feature] Log URLs encrypted if we have log encryption pubkey
* [Feature] Pass authenticated bit to lua
* [Feature] Read redis backend statistics configuration from global section
* [Feature] Show the exact value matched for multima symbols
* [Feature] Store task checksum
* [Fix] Avoid setting limits when required elements are missing
* [Fix] DMARC: Fix alignment checking for subdomains
* [Fix] DMARC: deal with missing and spurious spaces
* [Fix] Defer insertion of results in ip_score to avoid skewing stats
* [Fix] Disable DMARC for local/authorized mail
* [Fix] Fix handling of proxied headers in controller
* [Fix] Fix hex printing of strings
* [Fix] Fix issue with spaces in maps
* [Fix] Fix parsing of forwarded IP
* [Fix] Fix reload in some plugins and workers
* [Fix] Fix reloading on SIGHUP
* [Fix] Fix some border cases for DKIM canonicalization
* [Fix] Fix url maps
* [Fix] Make dnssec configurable option disabled by default for now
* [Fix] rspamadm statconvert: force db to be a string
* [Fix] rspamadm statconvert: use db/password for learn cache
* [Rework] Rework flags in rspamd logger

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [CritFix] Check hyperscan cache sanity before loading
* [CritFix] Fix setting of fuzzy keys (completely breaks fuzzy storage)
* [Feature] Add SARBL (sarbl.org) uribl
* [Feature] Add `--search-pattern` option to rspamd_stats
* [Feature] Add some sanity check for very long from/to log elements
* [Feature] Allow to create hashes from string in a single step
* [Feature] Fix order of pre and postfilters
* [Feature] Improve lua URLs API
* [Feature] Improve message about fuzzy rules
* [Feature] Pre-calculate blake2 digest for all parts
* [Feature] Print radix duplicate keys as IP addresses
* [Feature] Simple mechanism for disabling RBLs in local.d/rbl.conf
* [Feature] Use faster hash function for fuzzy storage
* [Feature] rspamd_stats: support log directory reading
* [Fix] Add sanity check for url filters
* [Fix] Do not show rmilter section as a fake metric in rspamc
* [Fix] Fix URL filters
* [Fix] Fix a stupid mistake in util.strequal_caseless
* [Fix] Fix blake2b hash of the string "rspamd"
* [Fix] Fix filename maps filter
* [Fix] Fix finding tld in util.get_tld
* [Fix] Fix multimap content filters
* [Fix] Fix returning boolean from Lua
* [Fix] Fix returning of REDIS_NIL
* [Fix] Try to deal with multiple workers terminated
* [Fix] Use forced DNS request when calling for lua_http
* [Rework] Rework multimap filters, add redis maps

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>

* [Feature] Add a special symbol for SPF DNS errors: R_SPF_DNSFAIL
* [Feature] Add correlations report in fuzzy stats
* [Feature] Add experimental CGP integration
* [Feature] Add method to get urls length in a text part
* [Feature] Add new methods to lua_html to access HTML tags
* [Feature] Allow all types of symbols to be added via __newindex method
* [Feature] Allow to create settings for authenticated users
* [Feature] Allow to get block content for HTML tags
* [Feature] Improve DNS failures when dealing with SPF
* [Feature] Properly implement R_WHITE_ON_WHITE rule
* [Feature] Remove old ugly rules
* [Feature] Rspamc can now add dkim signature in mime mode
* [Feature] Store content length for HTML tags
* [Feature] Support reacher set of HTML colors
* [Feature] Try to avoid FP for low contrast fonts detection
* [Fix] Add missing HTML colors
* [Fix] Add spaces to dkim signature to allow folding
* [Fix] Avoid returning NaN as score on scan
* [Fix] Decode entitles in href parts
* [Fix] Do not cache SPF records with DNS errors
* [Fix] Do not crash on cyclic depends
* [Fix] Do not insert HELO/HOSTNAME unknown when they are not passed
* [Fix] Do not set absent hostname to "unknown"
* [Fix] Do not stress redis with KEYS command (#791)
* [Fix] Fix DMARC_BAD_POLICY symbol
* [Fix] Fix HFILTER_URL module
* [Fix] Fix HFILTER_URL_ONELINE rule
* [Fix] Fix buffering in CGP integration
* [Fix] Fix colors propagation from parent nodes
* [Fix] Fix confusing OpenSSL API usage of i2d_RSAPublicKey
* [Fix] Fix dependencies id sanity check
* [Fix] Fix folding for semicolon separated tokens
* [Fix] Fix largest possible TLD behaviour
* [Fix] Fix last token folding
* [Fix] Fix length calculations in white on white rule
* [Fix] Fix multiple request headers structure
* [Fix] Fix multiple values headers freeing
* [Fix] Fix parsing of background color
* [Fix] Fix printing from field in log_urls
* [Fix] Fix processing of last element of DMARC policies
* [Fix] Further fixes for HTML colors
* [Fix] Further fixes for multiple values headers
* [Fix] Further fixes for white on white rule
* [Fix] Further fixes in HTML tags parsing
* [Fix] Ignore content type/subtype case
* [Fix] Increase score of R_WHITE_ON_WHITE
* [Fix] Parse CGP envelope data
* [Fix] Propagate colors in HTML
* [Fix] Restore multiple values headers in protocol
* [Fix] Restore multiple values in headers processing
* [Fix] Some more changes to tag's content length calculations
* [Fix] Some more fixes for low contrast fonts detector
* [Fix] SpamAssassin plugin: support check_freemail_header('EnvelopeFrom', [..])
* [Fix] Trigger HTML_SHORT_LINK_IMG on any external image
* [Fix] rspamd_stats: remove deprecated defined(@array)

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>