rspamd/ChangeLog

1.0.4:
	* Add configdump routine to rspamadm
	* Implement retransmits for fuzzy_check plugin
	* Fix events processing for learning anf checking fuzzy hashes
	* Avoid dependency on unneeded and uncompatible glib include
	* Add `historyreset` command to the controller
	* Fix loading of tokenizer config from dump (#389)
	* Add sorting hints for the history
	* Allow custom lua scripts for users/languages extraction (#388)
	* Do not add FORGED_RECIPIENTS when 'To' is missing (#387)
	* Do not add R_UNDISC_RCPT when 'To' is missing (#387)
	* Add encryption to fuzzy check plugin
	* Add encryption for fuzzy storage
	* Add new epoch for encrypted fuzzy request
	* Add encryption for `rspamd.com` storage
	* Remove gmime processing for LDA mode as it is deadly broken
	* Add routine to find end of headers position in mime messages
	* Fix LDA headers folding
	* Init libraries in rspamc client as well to avoid locale issues
	* Avoid collision with locally installed includes
	* Allocate and free memory with the same allocator in rspamadm (#385)
	* Preserve expired fuzzy hashes counter
	* Improvements in webui:
		- Add favicon.ico
		- Rework history table
		- Fix sorting for the history
		- Migrate to bootstrap 3 and jquery 2
		- Fix css bugs
		- Add glyphicons
		- Add reset history
		- Improve history buttons
		- Redraw graph to avoid display issues
		- Webui is now MIT licensed to match licensing policy of rspamd

1.0.3:
	* Fix piechart clean slice (#380)
	* Fix controller crashes when GString is reallocated (#381)
	* Correctly set locale before start
	* Set C locale for numeric values
	* Add rspamadm routine:
		- add `pw` command to manage passwords
		- add `help` command for displaying help
		- add `configtest` command to check configuration files
		- add `keypair` command for generating encryption keys
		- add `fuzzy_merge` routine to merge fuzzy sqlite databases
		- add a simple manual page for rspamadm
	* Allow metric registration for composite expressions
	* Add strict mode for configtest
	* Add logger counters
	* Save and show learned messages count (#383)
	* Add `no_stat` flag
	* Add `task:set_flag` and `task:get_flags` (#382)
	* Enable foreign keys in sqlite3
	* Remove orphaned shingles from fuzzy storage
	* Optimize synchronization steps for fuzzy storage
	* Allow delayed conditions registration
	* Add lua API for conditions registering

1.0.2:
	* Fix critical bug in webui that prevents password from being sent
	* Rework webui view:
		- Switch to d3.js for graphs
		- Improve piechart look
		- Rework colors for piechart
		- Fix layout for symbols
		- Fix refresh button
	* Add descriptions for whitelist maps
	* Fix build on arm (#379)
	* Fix issue with the last element in the radix trie
	* Add more tests for radix trie algorithm
	* Allow to extract URLs from query strings of other URLs (#361)
	* Initialize rrd fields before writing to file
	* Fix double free if no password has been specified

1.0.1:
	* Add writing to rrd from the controller
	* Fixed lots of bugs in rrd code
	* Adopt new DNS API in hfilter plugin (by @AlexeySa)
	* Allow only one controller process to manage rrd file
	* Set event base for fuzzy calls
	* Improve fuzzy IO errors logging
	* Add rra extraction function to rrd library
	* Add graph handler to the controller
	* Cache correct passwords to avoid too high CPU usage when working with webui
	* Controller sockets are owned by router do not export them to task
	* Optimize logging by skipping hash table search if it's empty
	* Fix loading issue whith broken statfiles
	* Print assertions from glib to rspamd logger
	* Load legacy `lua/rspamd.local.lua`
	* Update webui with some fixes to learning and scanning

1.0.0:
	* Rework symbols processing:
		- Improve sorting logic for symbols
		- Organize processing into multiple stages
		- Added asynchronous watchers for symbols
		- Added ability to organize dependencies between symbols
	* Fixed URL redirector:
		- Use optimized POE loop
		- Organize dependencies
		- Fix startup
	* New sqlite3 backend:
		- Allow to have per-languages and per-user statistics
		- Allow sqlite3 to be used as statistics backend
	* Store tokenizer configuration within statfiles
	* Improve bayes statistics:
		- Use headers and images metainformation in bayes
		- Suggest using of pre-processed tokens for statistics
		- Fix tokens normalization for OSB algorithm
	* Rewrite url parsing:
		- Fix numerous issues with url extraction and normalization
		- Fix mailto urls
	* Fix settings plugin to allow custom actions scores
	* Improve rbl plugin
	* Allow capturing patterns in rspamd lua regexp library
	* Add GTUBE support
	* Fix spamc legacy support
	* Add DKIM support to RBL module
	* Fix issues with multiple DKIM signatures
	* Fix issue if rspamd cannot create statfiles (#331)
	* Rework parts and task structure:
		- Now text_parts, parts and received are arrays
		- Pre-allocate arrays with some reasonable defaults
		- Use arrays instead of lists in plugins and checks
		- Remove unused fields from task structure
		- Rework mime_foreach callback function
		- Remove deprecated scan_milliseconds field
	* Add ip_score plugin support (not enabled by default):
		- Can check for asn/country and network using DNS lookups
		- Can store and load reputation from redis server
	* Improve PARTS_DIFFER rule to count merely different words
	* New HTML parser:
		- Parses HTML parts using a set of state machines
		- Extracts useful data and exports it to lua functions:
			+ Styles
			+ Images
			+ URLs
			+ Colors
			+ Structure elements
		- Added HTML rules for some checks
	* New version of LUA DNS API
	* Table versions of many functions in LUA API
	* Improve rspamc client:
		- Print execution time
		- Allow executing of external commands and passing output to them
		- Allow mime output mode when rspamc alters message according to rspamd
		  checks and send it to an external command or stdout
	* Allow scanning of local files using HTTP requests
	* Rework configuration system:
		- Rules are now moved from the $CONFDIR to $RULESDIR to avoid ambiguity
		- All modules configurations are now split in $CONFDIR/modules.d/* to
		  simplify upgrades
		- Move hfilter to plugins
		- Allow plugins and rules to define default scores to simplify metrics
		  setup
		- Include overrides for all modules to honor local/automatic parameters
		- Tune scores for many modules
	* Rework and enable DMARC plugin
	* Add whitelist plugin for SPF/DKIM/DMARC based whitelisting
	* Add some common domains to whitelists shipped with rspamd
	* Rework logging:
		- Now each log entry supports module name and a `tag`. Tag is used to
		  identify unique objects (such as tasks) when checking log files
		- It is possible to turn on debugging for the specific modules
		- Systemd logging is fixed
	* Improve spamassassin plugin.
		- Now headers are matched more like SA
		- Improve support of Message-ID
		- Add support of ToCc header type
		- Fix :addr and :name in headers regexps
	* Resurrect rrd support code
	* Save controller stats between restarts
	* Fixed tonns of bugs
	* Added tonns of minor improvements and features
	* Added more unit tests
	* Create functional tests framework
	* Added documentation for missing modules
	* Added rpm/deb repositories and scripts
	* Updated WebUI and libucl externals

0.9.10:
	* Do not dereference null pointer on learning.
	* Fix some extreme cases in BAYES.
	* Add a workaround to avoid bad HTML messages breaking.
	* Build with -O2 flags by default.
	* Add constraints to limit DNS requests count per task.
	* Add workaround for SURBL DNS flood.
	* Set error if rspamd cannot learn anything.

0.9.9:
	* Don't use RWL_SPAMHAUS_WL (unknown result) for whitelisting (by @fatalbanana)
	* Import updated public suffix list (by @fatalbanana)
	* Remove debug message
	* Fix settings (by @fatalbanana)
	* Remove duplicated symbol registration
	* Use WAL for fuzzy storage
	* RBL fixes (by @fatalbanana):
		- silence errors;
		- yield unknown results from RBLs;
		- fix scoring for DNSWL;
		- fix use of RBL name as symbol;
		- ignore RBL names that would not be yielded;
	* Support captures in regular expressions
	* Add captures support to lua_regexp
	* Support dist on FreeBSD and Darwin
	* Add RCVD_IN_DNSWL_NONE as whitelisting exclusion (by @fatalbanana)
	* Multiple fixes to URL detection:
		- support port definition;
		- fix query and path recognition;
		- fix parsing of multiple slashes in URL;
		- fix parsing query just after port;
		- fix path field in `url:to_table` method;
		- improve support of IP based URLs.
	* Set ignore_whitelists = true for RECEIVED_SPAMHAUS_XBL (by @fatalbanana)
	* Add GTUBE support
	* Ignore User header in SA mode

0.9.8:
	* Fix critical bug in bayes classifier (#305)
	* Fix critical bug in RBL module (by @fatalbanana)
	* Fix and rework settings plugin.
	* Fix get_all_opts for a case of non-iterable options.
	* Use tld for redirector's matching.

0.9.7:
	* Add whitelist_exception setting to RBL module (by @fatalbanana)
	* Don't use RWL_MAILSPIKE_POSSIBLE or DNSWL_BLOCKED for whitelisting (by
	  @fatalbanana)
	* Fix extreme cases in bayes classifier.
	* Fix parsing of urls with '?' at the end of hostname.
	* Update interface.
	* Fix number of issues with webui interaction.
	* Fix saving maps.
	* Allow user@ and @domain matches in multimap.
	* Fix issues with bounces From processing.
	* Fix abs/fabs misuse.
	* Fix builds on suse and arch linux distributions.

0.9.6:
	* Fix memory leak if mime cannot be parsed.
	* Fix dkim cache expiration.
	* Fix issues with redirector HTTP response.
	* Fix abnormal connection closing with certains messages with a high score
	  (issue #296)
	* Fix redirector installation.
	* Use specific POE loop for some systems.
	* Fix number of issues in URL redirector.
	* Fix selecting URLs for sending to redirector.

0.9.5:
	* Avoid double free when extending HTTP message.
	* Fix double free if multiple classifiers are defined.
	* Fix misprint in spamassassin plugin.
	* Fix cpuid invocation on i386.
	* Fix ownership issues for zero-copy decode.
	* Allow __len metamethod on rspamd{text}.
	* Add base64 decoding lua utility.
	* Fix build on FreeBSD
	* Skip spaces at the beginning of mime messages.
	* DBL_ABUSE_REDIR should not have significant weight.
	* Allow to split by lua_regexp rspamd{text} objects.
	* Allow to specify custom stop pattern for lua_tcp.

0.9.4:
	* Fix critical bugs in tokenization algorithm
	* Write unit tests for tokenization
	* Add documentation for lua_tcp
	* Switch off legacy tokenization by default.
	* Fix critical bugs in words normalization
	* Add lua bindings to tokenizer.
	* Implement storing of HTTP headers inside task
	* Add lua API to accerss HTTP headers data
	* Implemented base64 encoding suitable for MIME
	* Use caseless hash and equal functions for HTTP request headers.
	* Improve debian architectures support (by @dottedmag)

0.9.3:
	* Revert incorrect regexp change that broke the default rules
	* Fix lua_tcp module

0.9.2:
	* Fix error on spawning unique workers.
	* Add preliminary version of generic LUA TCP requests API.
	* Use lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc)
	* Fix fuzzy mime strings with only type.
	* Improve thunderbird sanity checks.
	* Fix critical bug on matching regular expressions.
	* Make hiredis optional dependency.
	* Fix multiple bugs in daemon reloading

0.9.1:
	* Restore utf8 validation for regular expressions to avoid crashes
	* Fix symbols displaying in the interface
	* Add symbol groups to the interface
	* Fix maps ID parsing in the controller
	* Add multimap and regexp modules documentation
	* Backport fixes from libucl
	* Fix debian package (by @dottedmag)
	* Rework XXH32 invocations

0.9.0:
	* Add support of the fast and secure protocol level encryption:
		- curve25519 is used for key exchange;
		- chacha20/poly1305 cryptobox construction for bulk encryption;
		- zero latency overhead;
		- encrypting and balancing HTTP proxy worker
	* Rework expressions and create new expressions library:
		- aggressive optimizations based on the abstract syntax tree;
		- abstract expressions support (regular expressions, functions, lua modules
		  composites and so on)
		- New comparision and '+' operators support
		- New greedy algorithm to minimize execution time of expressions and
		  all symbols
		- Dynamic expressions benchmark and reoptimizations
	* Many improvements to the LUA API:
		- reworked logger module allowing to do pretty print of the most of lua
		  types (including tables and userdata classes)
		- reworked lua redis and lua HTTP to support more features
		- added opaque type for passing large text chunks without copying
		- new regexp module with many auxiliary functions (e.g. `re:split`)
	* LuaJIT is now the default requirement for rspamd allowing to speed up lua
	  execution by a large margin (however, plain lua is still supported)
	* New plugins:
		- spamassassin rules plugin that allows to load and re-use the most of
		  SA rules natively
		- DMARC plugin that evaluates SPF and DKIM policies to the domain policies
		- many old plugins has been reworked to implement new features and improve
		  stability
	* New aho-corasic trie implementation from @mischasan that allows to load and
	  use hundreds of thousands of patterns with no influence on load
	* Support of PCRE JIT and PCRE JIT fast path modes that significantly improves
	  the performance of regular expressions if supported by PCRE
	* New URLs parser and extractor:
		- removed legacy code that was useless for url finding
		- reworked algorithms of URL parsing for more precise and accurate results
		- added top-level-domains tree from http://publicsuffix.org
		- improved emails parsing
		- removed many phishing false positives due to TLD tree check
	* New statistics infrastructure:
		- created a separate layer of statistic library
		- improved OSB-Bayes by re-weighting tokens according to the original
		  academic paper and `crm114` implementation, which reduced false positives
		  rate significantly
		- created learn cache to avoid double learning of statistics and providing
		  an efficient way to re-learn class for a message
		- created abstract layers for different statistics backends
		- implemented new tokenization algorithms with fast or secure (siphash)
		  hashes to generate statistics features
	* Reworked utf8 tokenization that previously corrupted all UTF8 words (minor
	  incompatibility with old fuzzy hashes with utf-8 symbols)
	* SPF module has been completely rewritten to support complex cases of
	  `include` and `redirect` within SPF records
	* DKIM module now supports multiple signatures
	* Controller passwords can now be stored encrypted by `PBKDF2-HMAC` in the
	  configuration file
	* Many hand-written HTTP clients has been replaced with the common rspamd
	  http module
	* New test framework:
		- import lua `telescope` test framework
		- add unit tests for many rspamd modules and routines
		- create a unit test for each possible bug found
		- use luajit ffi for testing C code
		- added preliminary support of functional testing by creating tasks from lua
	* Randomize hash seed to avoid certain hash tables vulnerabilities
	* Documentation improvements:
		- added documentation for the vast majority of rspamd modules
		- added documentation for rspamd protocol
		- added documentation for the most of rspamd LUA extensions
	* Fixed tonns of bugs and memory leaks
	* Added tonns of minor features

0.8.3:
	* Various critical fixes in distribution (by @dottedmag and @fatalbanana)
	* Fixed bugs in url detector to parse certain patterns
	* Add default host and helo for a client
	* Some sanity checks for tokenizer and classifier
	* Reiterate on systemd support
	* Fix missing symbol registration
	* Add support of spamc compatible output
	* Filter double-dots in rbl.lua validate_dns (by @fatalbanana)
	* Update ucl submodule due to critical bugfix

0.8.2:
	* Create fuzzy db if it does not exist
	* Fix: Centos init script: configtest() (by @AlexeySa)
	* Enable one_shot for RECEIVED_SPAMHAUS_XBL - Fixes #102 (by @fatalbanana)
	* Update Exim patch (by @fatalbanana)
	* Fix processing of unix sockets.
	* Allow applying settings to authenticated users (by @fatalbanana)
	* Make settings priorities work as documented (by @fatalbanana)
	* Fix race condition in symbols planner
	* Add DNSWL_BLOCKED symbol (by @fatalbanana)
	* Make Exim pass usernames to rspamd (by @fatalbanana)
	* Update RBL module (by @fatalbanana):
		- fix indentation;
		- collapse loops;
		- avoid calling for un-needed information;
		- allow disabling RBLs for authenticated users
	* once_received.lua: Fix indentation & add exclusion for authenticated users (by @fatalbanana)
	* hfilter.lua: Add exclusion for authenticated users (by @AlexeySa)
	* Updates to hfilter rules (by @AlexeySa)
	* Set empty <> user or addr for msgs without FROM (by @eneq123)
	* Fix: attempt to index field '?' (a nil value) (by @eneq123)
	* Fix: if not exist Date-header (by @AlexeySa)
	* Add task:get_content() method.
	* rbl.lua: Ignore private IP space (by @fatalbanana)
	* Allow to check radix maps from lua by rspamd{ip}
	* Make local exclusions configurable per-RBL (by @fatalbanana)
	* Add rspamd_config:radix_from_config() (by @fatalbanana)
	* Support emails dnsbl in rbl (by @fatalbanana)
	* Complete rework of url extraction logic
	* Allow customizations for unix sockets. (fixes #182)
	* Set lua path according to rspamd settings.
	* Import lua-functional for plugins stuff.
	* Completely rewrite multimap plugin in functional style.
	* Fix FORGED_MUA_THUNDERBIRD_MSGID (fixes #186)
	* Check IPv6 addresses at dnswl.org and Spamhaus whitelist (by @fatalbanana)
	* Add lowercase utility for utf8 strings.
	* Various fixes to build system
	* Updated debian configuration infrastructure (by @dottedmag)

0.8.1:
	* Add sqlite and perl as dependencies for RPM/Debian packages (by @fatalbanana)
	* Remove whitelist.lua from RPM file list (by @fatalbanana)
	* Make Exim pass hostnames to rspamd (by @fatalbanana)
	* Fix building on Fedora (by @fatalbanana)
	* Add toggle for disabling installation of systemd units on Linux (by @fatalbanana)
	* Fix double format rounding that caused output corruption (reported by @fatalbanana)
	* Revert broken change for destructors ordering that led to memory corruption
	* Do not reset symbols case of settings if parsed from lua (reported by @andrejzverev)
	* Fix build on SunOS (by @wiedi)
	* Fix multiple crashes on broken DKIM DNS records
	* Fix critical issue with composites weights removing
	* Fix memory corruption in composites processing code
	* Ignore non-SPF TXT records when parsing SPF includes

0.8.0:
	* New fuzzy check logic:
		- use shingles algorithm for fuzzy matching
		- use blake2 instead of md5 for larger output space
		- combine fuzzy and strict matching
		- allow to organize private storages by means of keys
		- preserve compatibility with previous versions
	* New fuzzy storage:
		- use sqlite instead of own memory based hash tables
		- rework commands interface
		- add conversion from the old format
		- add fuzzy match by shignles
		- support old rspamd versions
	* Add lemmatizing for words used in fuzzy hashes that allows to improve match
	quality by using of the first forms of all words
	* Rework language detection
	* Fix several critical bugs, memory leaks and deadlocks:
		- memory leak in HTML nodes parsing
		- deadlock in logger code
		- deadlock in signals processing
		- crashes in fuzzy_storage
		- crashes in tokenizers if the input was empty
	* Import new libucl with several bugfixes and improvements
	* Support listening on ipv6 addresses only
	* Fix macro expansion in SPF module
	* Several bugfixes in DKIM module
	* Add load headers support for mime parts to the lua API
	* Add documentation for:
		- workers in general
		- fuzzy_storage worker
		- fuzzy_check plugin
		- mimepart and textpart lua API modules

0.7.6:
	* Apply boundary fix for dkim simple canonization
	* Fix ping command
	* Return nil if header was not found in lua_task
	* Fix hang in upstreams revive logic
	* Decode entitles when normalizing HTML parts
	* Fix logic of finding URLs in HTML parts
	* Do not include \0 into length of text when performing conversion to utf8
	* Fix raw vs parsed reperesentations
		Raw parts are now:
			- decoded b64/qp, but *NOT* converted to utf-8
		Processed parts are now:
			- converted to UTF-8
			- normalized if needed (e.g. HTML tags are stripped)
	* Rework DKIM canonization to line based
	* Fix fuzzy hashes addding
	* Use more specific hash function for fuzzy
	* Fix leaking of iconv descriptors
	* Fix PTR resolving in lua resolver
	* Rework spf module.
		- Copy data to memory pool as cached record might be destroyed causing
		  freed memory being passed to the protocol output (use after free)
		- Allow SPF_NEUTRAL policy to be handled separately
		- Add R_SPF_NEUTRAL to the default config
	* Rework `register_symbols` function
	* Allow to disable components of hfilter

0.7.5:
	* Fix owner when creating folder /run/rspamd (by @sfirmery)
	* Fix IP validity checks
	* Decode URLs obtained from HTML tags
	* Fix crash with unweighted upstreams
	* Stop processing headers in parts
	* Set sockaddr.sa_family properly when connectig to upstreams
	* Fix reload issues in surbl and fuzzy_check (reported by @citrin)
	* Fix timeouts in redirector
	* Improve lua errors reporting
	* Fix lua closures processing in libucl
	* Rework calling of lua functions from regexp module
	* Choose raw regexp for raw headers
	* Rework conversion to utf since glib one is broken
	* Ignore SGML style tags in html
	* Fix old bug with non-capturing https urls
	* Fix memory corruption on fuzzy reload (reported by @citrin)
	* Fix percents display in rspamc
	* Fix buffer update for DKIM
	* Do not validate utf for raw headers

0.7.4:
	* Fix build under *BSD
	* Detect HAN unicode script
	* Implement language detection heuristic for text parts
	* Fix time output in history
	* Improve piechart coloring
	* Fix \r\n conversion in DKIM module (reported by @citrin)
	* Try to detect systems with no IPv6 support
	* Fix multiple/single values in use settings (reported by @citrin)
	*  Rework IP addresses in upstreams:
		- Select ipv4/unix addresses if they exist and use ipv6 for ipv6 only
		  upstreams (since the support of ipv6 is poor in many OSes and
		  environments)
		- Free IP list on upstream destruction
		- Add test cases for addresses selection
		- Allow adding of free form IP addresses to upstreams
    * Fix endiannes in lua_radix search (reported by @citrin)
    * Soft shutdown should also set wanna_die flag (reported by @citrin)
    * Stop use-after-free in event loop termination
    * Fix processing of very short messages in DKIM (reported by @citrin)
    * Detect systems without shared mutexes
    * Fix issues with PTR and MX elements in SPF parser (reported by @citrin)

0.7.3:
	* New upstreams code:
		- simplify upstreams API;
		- unify strings parsing in upstreams definition;
		- add configuration options for the upstreams;
		- for failed upstreams re-resolve their addresses;
		- use all resolved addresses for an upstream (round-robin);
		- implement stable hashing and use it by default for upstreams;
		- add unit test for upstreams module.
	* Rework signals processing in all rspamd workers:
		- signals are now processed in the event loop;
		- implement the most common signal handlers for all workers;
		- add callbacks for workers specific signal handlers
	*  Fix critical issue with fuzzy storage:
		Fuzzy stroage could not save any hashes on termination due to bugged
		signals handling
    * Fix roll history IP storage
    * Rework ipv4/ipv6 handling in parsing addresses:
		- turn off support of IPV6_V6ONLY socket option;
		- create ipv6 socket prior to ipv4 one to handle systems with v6/v4 
		sockets enabled (Linux)
    * Remove CBL as it's wholly included in Spamhaus XBL (by @fatalbanana)
    * Remove nszones.com fake RBL (by @citrin)
    * Fix upstreams interaction for fuzzy_check
    * Verify spf PTR records (reported by @citrin)
    * Fix spf MX records parsing
    * Add compatibility for old libevent (by @yellowman)
    * Sync bugfixes from libucl

0.7.2:
	* Convert all maps to the compressed radix trie
	* Allow IPv6 addresses in IP maps
	* Remove dynamic items support from symbols cache
	* Allow hex encoded output of strings
	* Fix bug with control connections count
	* Process fuzzy weight correctly (reported by @fatalbanana)
	* Remove extra reference retain of http connection on error
	* Remove deprecated options from the default config
	* Add `one_shot` attr to metric's symbols
	* Doc: add documentation for metrics
	* Add Upstart job to debian packaging (by @CameronNemo)
	* Config: improve SURBL symbols descriptions (by @citrin)
	* Config: reflect SURBL changes (by @citrin):
    	- Outblaze removed, malware moved to separate list:
    	http://www.surbl.org/news/internal/MW-malware-sublist-added-to-multi
    * Fix C modules initialization on restart
    * Treat single IP as a single IP in radix lists (reported by @citrin)
    * Do not touch file and core limits if not asked explicitly (reported by @citrin)
    * Improve logging for fuzzy errors
    * Block SIGPIPE for HTTP writing
    * Doc: update manual pages
    * Fix HTTP connection termination
    * Reduce default number of parallel requests to 8
    * Sync with libucl include features

0.7.1:
	* Fix typo in stat output.
	* Fix issues with includes crossing with the system includes
	* Restore testing framework
	* Add radix trie test suite
	* Implement new path-compressed radix trie.
	    - The performance benefit over the old algorithm is about 1.5 times.
		- Memory usage is significantly lower as well.
	    - Now radix trie can accept any IPv4/IPv6 values
	* Various improvements to the memory pools code
	* Fix writing reply to a client when no filters are defined
	* Write base32 encoded fuzzy
	* Fix 'soft reject' action
	* Fix rspamd reload and modules reconfiguration
	* Fix subject rewriting for the default subject
	* Fix states for processing task and pre-filters
	* Fix issues with connection closing
	* Fix crashes in rdns
	* Fix ratelimit pre-filter
	* Update exim patch.
		- Update to the recent exim version
		- Strip extra leading src/ from the patch
		- Remove sendfile since it was broken
		- Fix rspamd spam report for exim
	* Improve documentation

0.7.0:
	* Use HTTP protocol for all operatiosn
	* Webui worker is now removed and controller works as webui
	* Allow to serve static files via controller by option `static_dir`
	* Rspamd interface is now a part of rspamd
	* Rspamc client has been rewritten to use HTTP and non-blocking mode
	allowing to start multiple operations simultaneously (see `-n` option)
	* Lua API was completely reworked to satisfy modern standards of LUA:
		* Module `lua-message` was removed
		* Reduced number of superglobals registered by rspamd
		* Many functions has been redesigned
		* Symbols registration is now more convenient
	* Users settings has been rewritten as lua plugin
	* Reworked headers system as gmime's based one misses many headers and is
	very slow to get headers values
	* Reorganized code and removed embedded jannsson by using UCL for all json
	parsing
	* Migrated to `librdns` for DNS resolving that improves concurrency for
	DNS requests significantly
	* Fixed tonns of bugs in MIME processing
	* Improved metrcis and default symbol's weights
	* Added new RBL's
	* Fixed a number of issues in the modules
	* Removed several memory leaks found
	* Fix unicode processing
	* Fix fuzzy checking for unicode parts
	* Significantly improve documentation and especially LUA API docs
	* See migration notes at https://rspamd.com/doc/migration.html

0.6.8:
	* Controller now listen for localhost and not for 127.0.0.1 by default
	* Allow FCrDNS-style RBL lookups (by @fatalbanana)
	* Reduce threshold for parts_differ function.
	* Fix hostname lookup for rdns rbl (by @AlexeySa)
	* Fix HFILTER_URL_ONELINE to reduce false positive rate.
	* Fix whitelist module.
	* Allow override system predefined settings without touching system ones
	by .try_include macro (by @andrejzverev)
	* Check for [ip.address]-style HELO and suppress lookups. (by
	@fatalbanana)
	* Optimize hfilter (by @AlexeySa)
	* Fix issue with random numbers generator in dns.
	* Use more clever time values to setup entropy.
	* Synced with the recent libucl.

0.6.7:
	* Use ChaCha20 for DNS generator (more secure DNS id)
	* Unknown symbols now has zero weight and not 1.0
	* Fix fuzzy hashes expire time
	* Fix critical issue in statfiles on FreeBSD 9 (and some other platforms)
	* Add .include_map macro to ucl parser
	* Update libucl
	* Fix headers end detection for DKIM module
	* Fix a bug in received headers parser
	* Validate IP addresses before pushing them to lua
	* Start new documentation project
	* Fixed tonns of other minor bugs
	* Start to prepare for 0.7 with HTTP protocol and new settings

0.6.6:
	* Removed issue with BUFSIZ limitation in the controller output
	* Simplify logging symbols escaping
	* Adjusted weights for several rules
	* Improve spamhaus rbl support
	* Removed PBL for received headers checks
	* Added hfilter module that performs various HELO and IP checks.
	* Rspamd can now be reloaded by HUP signal
	* Fuzzy storage should expire hashes properly
	* Build system has been reworked for better supportof pkg-config
	* Various minor bugfixes

0.6.5:
	* Fixed critical bug in DNS resolver, introduced in 0.6.4
	* Improved multimap and rbl plugins to skip
	* Add dns_sockets option for tuning sockets per server in DNS resolver
	* Improved packages for rspamd

0.6.4:
	* Added io channels for DNS request to balance load and reduce id
	collisions chance
	* Fixed a bug in SPF filter that may cause core dump in specific
	circumstances
	* FIxed default config for rbl module
	* It is possible to get a list of rspamc commands with their descriptions
	* Added SORBS bl to the default config
	* 2tld file for surbl module has been significantly extended
	* Perl modules has been removed from the code.
	* Fixed an issue in libucl when parsing macros

0.6.3:
	* Fixed issues with DNS:
	 - labels decompression algorithm was fixed;
	 - added resolve_mx to lua API;
	 - fixed modules that use DNS.
	* Lua modules once_received and emails reworked for new resolver API and UCL.
	* Debian package was polished.
	* Fixed a bug in fuzzy_check module that prevents correct processing messages 
	  without valid parts.

0.6.2:
	* Fuzzy check module has been reworked:
	 - now fuzzy_check operates with a group of rules, that define which
	 servers sre to be checked;
	 - it is possible to specify read_only groups to avoid learning errors;
	 - turn fuzzy_check to one_shot mode permanently;
	 - fuzzy_check module configuration is now incompatible with the previous
	 versions.
	* Imported bugfixes from libucl.
	* Fixed whitelist plugin.
	* Fixed statfiles resizing.
	* Improved logging initialization order.
	* Fixed race condition in the controller worker.

0.6.1:
	* Critical bugfixes:
	 - fixed build system;
	 - fixed in_class setting in bayes learning;

0.6.0:
	* Use UCL instead xml for configuration (https://github.com/vstakhov/libucl)
	* Fix statistics module normalization
	* Rework the many modules for the new configuration:
	 - surbl has incompatible configuration;
	 - fuzzy_check has incompatible configuration;
	 - multimap has now new configuration;
	 - received_rbl is removed and replaced by rbl module.
	* Removed deprecated options:
	 - statfile_pool_size;
	 - action and required/reject score for a metric.
	* Simplify build system and unify configuration for all platforms.
	* Improved URL detector (reduced false positive rate).
	* Lua 5.2 is now the default and fully supported version.
	* Tons of bugfixes and minor improvements.	

0.5.6:
	* Fix bug with counters incrementing in rolling history
	* Detect expl and exp2l as some systems do not have it
	* Support input streams without Content-Length
	* Implement counters output via rspamc and controller interface
	* Fix bug with udp sockets in fuzzy storage

0.5.5:
	* New bayes normalizator based on inverse chi-square function
	* Various fixes to fuzzy storage
	* Allow update fuzzy storage only from specific IP addresses
	* Better support of IPv6 and address selection algorithms
	* Add CentOS spec file

0.5.4:
	* Fixed issues with diff algorithm
	* Added support of RRD statistics
	* Add webui worker for interface interaction
	* Fix a lot of issues with dynamic conf
	* Fix critical memory leak in settings code
	* Improve stability of the system

0.5.3:
	* Added dynamic options
	* Added advanced metaclassfication
	* Added RESTfull API for controller
	* Improved hashing algorithms
	* Various fixes for rspamc client:
	 - allow interacting with unix sockets
	 - librspamdclient major cleanup
	 - bayes is now default classifier
0.5.2:
	* Added lua bindings for:
	 - basic mime parts, that allows checkign attachements for example;
	 - DNS resolver;
	* Existing lua bindings now works without task object allowing to use them
	in custom code.
	* Threads system was reworked to avoid global lua interpreter lock.
	* DKIM module now converts all line endings to CRLF how opendkim does.
	* URL detector is now more accurate for text parts.
	* Several critical bugs and memory leaks were fixed.

0.5.1:
	* Added lua worker type to handle network connections in lua.
	* Added lua bindings for async_session, IO dispatcher, memory_pool, and
	worker.
	* Composites can now uses other composites in expressions.
	* Fixes for debian package and for FreeBSD >= 9.1
	* Add support of gmime-2.6 if gmime-2.4 is not found.
	* Improve url detection and phishing detection.
	* Add lua mime_part library to get an access to all message part
	attributes (like filename, length, type).

0.5.0:
	* Added SMTP lightweight balancing proxy with XCLIENT support.
	* Added lua bindings for upstreams objects and API.
	* New pre-filters are implemented to support initial checking for messages.
	* Added ratelimit plugin that uses redis protocol to store data.
	* Added ipv6 support to spf and some other modules.
	* Unbreak spf plugin.
	* Allow options with the same name be threated as list.
	* DKIM plugin an parsing code was added.
	* Separate build system to put logic in several shared libraries.
	* Many bugfixes.

0.3.2:
	* Add error handling for regexps
	* Fix quit command in controller interface
	* Write symbols weights to rspamc output
	* Improve logic of selecting rspamc version
	* Do not try to parse broken DNS replies
	* Add 'raw' flag to FROM_EXCESS_BASE64 rule (requested by citrin)
	* Output message id in rspamc reply
	* Fix inserting composite symbol
	* Fix output of log line
	* Document composites
	* Add logging for fuzzy checks
	* Add logging for learning
	* Improve logic of learning messages - do not learn more than specific threshold
	* Fix inserting results for symbols that were incorrectly (for example more than 1 time) defined in config file
	* Do not output control characters if output is not terminal
	* Fix some logic errors in learning
	* Consider lua plugins errors as fatal configuration errors
	* Fix wirting message id during fuzzy_add command
	* Display weight of symbols correctly
	* Fixes to winnow learning
	* One more try to improve accuracy of winnow algorithm
	* Add bayesian classifier (initial version)
	* Remove normalizer as it is winnow specific thing, so all statistic algorithms now returns value from 0 to 1
	* Some fixes to fuzzy hashes expiration:
	* Fix assertion while look up value in NULL hash (found by cirtin)
	* Fix normalization for systems that have not tanhl function
	* Ignore rfc822 group addresses
	* Move images library to core rspamd
	* Add lua api to access images properties
	* Add post filters to lua API - filters that would be called after all message's processing
	* Add ability to check for specified symbol in task results from lua
	* Add ability to check for metric's results from lua
	* Add ability to learn specified statfile form lua
	* Add ability to extract filename and size of images from lua
	* Fix assertion while extracting internet address
	* Fix races in fuzzy storage
	* Make spf parser case insensitive
	* Add ability to check hashes of selected mime types
	* Add ability to set minimum size in bytes for mime types
	* Add ability to set minimum dimensions for images
	* Assume all text/* content types as text/plain
	* Fix getting data wrapper for gmime24
	* Many fixes to statfile syncronization system
	* Fixed statfile pool initialization and synchronization with disk
	* Prepare 0.3.2
	* Fix Mail::Rspamd::Config for new rspamd features
	* Use Mail::Rspamd::Config in rspamc client
	* Write user's name to rspamd log
	* Prepare rspamd build infrastructure for creating rpm and deb packages
	* Fix depends
	* Add start script for linux systems
	* Fix shared usage of statfiles
	* Add invalidation of statfiles in case of learning, so now statfiles
	* This should fix shared usage of statfile pool by several processes
	* Fix misprint (reported by az)
	* Fix stupid error when all checks can be done in a single pass
	* New trie based url scanner (based on libcamel)
	* Small fixes to rspamd perl client
	* Write fuzzy hashes info to log
	* Add trie interface to lua api
	* Explain sample config and cleanup it

0.3.1:
	* Add modules documentation
	* Continue implementing smtp proxy
	* Implement new learning system, now rspamd should be much more intelligent while learning messages
	* Convert statistic sums to use long double for counters
	* Use hyperbolic tangent for internal normalizer
	* In classify normalize result after comparing, not before
	* New symbols sorter
	* Fix strict aliasing while compiling with optimization
	* Fix tanhl detection for platforms that have not implementation of it
	* Remove several compile warnings
	* Add experimental support of dynamic rules to regexp module
	* Document views configuration
	* Several fixes to documentation
	* Add more logic for dynamic rules
	* Add documentation for dynamic rules
	* Add ability to make negations in networks in dynamic rules
	* Clean up cache items correctly
	* Implement basic SMTP dialog:
	* Implement interaction with smtp upstream (with support of XCLIENT)
	* Check messages received via smtp proxy
	* Add support for sendfile in io dispatcher
	* Fix issues with compatibility of worker_task and smtp proxy
	* Proxy DATA command
	* Fix SMTP
	* Change metric logic
	* Completely remove lex/yacc readers for config
	* Make common sense of metric/action and symbols
	* Sync changes with all plugins
	* Incorrectly removed in previous commit
	* Fix misprint (by Andrej Zverev)
	* announce the "password" keyword in usage list
	* Implement initial version of greylisting triplets storage
	* Fix issues with smtp worker
	* Fix QUIT command in SMTP worker
	* Some fixes about new metrics system (may be incomplete)
	* Get weights of symbol from default metric for symbols cache
	* Fix setting task->from/task->rctp in smtp client
	* Copy from and rcpt correctly
	* Some performance improvements to IO dispatcher (do not drain the whole buffer after a single line readed)
	* Fix smtp data input
	* Fix misprint
	* Add limit of maximum allowed smtp session errors
	* New logic of SURBL module:
	* Use system mkstemp(3) on systems where it is available as glib implementation
	* Try to fix memmove issues in io dispatcher
	* Remove debug from SURBL module
	* Rewrite buffered input for line policy (again)
	* Fix issue with links that are ip addresses in numeric form in surbl
	* On Darwin use BSD style sendfile definition
	* Reorganize platform specific knobs in CMakeLists
	* Use gettimeofday on systems that have not clock_getres
	* Use ftime for dns trans id generation on systems without clock_getres
	* Darwin sendfile(2) support
	* TIMEDB->TIMEB
	* More to previous commit
	* Pass env from main() arguments instead of platform specific global environ
	* Fix compatibility issues
	* Fix -lintl detection
	* Init some variables to avoid problems
	* Remove garbadge (gnome terminal sucks)
	* Add more information about why we drop smtp connection
	* Fix mkstemp call
	* Send to upstream QUIT command at the end of session
	* Check return value of each rspamd_dispatcher_write as in case of write errors sessions can be destroyed early
	* Fix states in smtp dialog
	* Use rspamd_snprintf instead of libc one
	* Fix URLS command
	* Fix reconfigure process of surbl module
	* Fix destroying smtp session (unmap memory and do not delete pool early)
	* Delete pool after using its variables
	* Delay timer must be registered in async session to correctly handle connection termination
	* Register dns requests in session too
	* Make session before registering events
	* Remove events in handlers
	* Add ability to set filters for smtp worker for each smtp stage
	* Add very initial version of DNS resolver (many things to be done)
	* Announce weights and sync
	* Fix few typo
	* Understand short names of facility in logging config
	* Add ability to make whitelist for spf checks
	* Misprint != -> ==
	* Handle lua tag in way that it is not required to write additional text:
	* Strip all starting whitespace symbols from xml texts
	* Fix stupid bug in calculating buffer length while reading file maps
	* Add resolv.conf parsing into dns.c
	* Fix microseconds<->milliseconds conversions
	* Take callback argument in Mail::Rspamd::Client for processing files and directories
	* Print results if rspamc is called for a directory
	* Fix stupid error with surbl module reconfig (another one, blame me)
	* Do not show duplicate urls in url header
	* Fix detection of numeric urls (reported by citrin)
	* Write real time of message's scan to log (not only virtual)
	* Fix chartable module in utf mode
	* Fix parsing of some broken urls
	* Add ability to test regexp with 'T' flag
	* Write more code for DNS resolver:
	* Make DNS resolver working
	* Many improvements to rspamd test suite: now it CAN be used for testing rspamd functionality
	* Write DNS resolver tests
	* Fix issues with memory_pool mutexes and with creating of statfiles
	* Forgotten in previous commit
	* Add support for parsing SPF and SRV records
	* Fix PTR parsing
	* Add tests
	* Make SURBL module to use rspamd dns module
	* Several fixes to DNS logic
	* Remove evdns and use only rspamd resolver
	* Very hard to detect problem with race among error in socket and destroying task while we are writing to socket and go through a hash table
	* Fix resolving in smtp module
	* Init events before configuring resolver in smtp worker
	* Set resolver inside task
	* Fix reload signal (reported by citrin)
	* Some improvements to redirector
	* Call has_forked method to inform POE about fork
	* Fix lua DNS code
	* Decompress labels in DNS packets more strictly
	* Fix some problems with TXT records
	* Try to fix removing of DNS events
	* Do not insert unparsed RR's into reply
	* Calling callbacks may cause destroying session from which we are calling callback so we MUST call callback as the latest action
	* Fix check_smtp_data function
	* Add ability to make views by recipient
	* Add ability to set metric's action from config file
	* Fix bug with writing garbadge if message has no urls or no messages
	* Fix bug with incorrect behaviour of compare_parts_distance function
	* Add ability to assign several actions to one metric
	* Report action in rspamc protocol
	* Mail::Rspamd::Client and rspamc can now understand Action header too
	* Write action to log as well
	* Make valgrind happy about comparing symbols
	* Add more debug to comparing parts distance function
	* Write action even if message has no symbols
	* Make improvements to HTML entites decoder: now it replaces entities with common characters and
	* Add -d option to force debug output
	* Assume 7bit as default transfer encoding
	* Do not overwrite lua plugins and configs if they already exists in target directory
	* Improve logging
	* Write queue id to log
	* Remove test messages from cmake
	* Reopen log file by USR1 signal
	* Add reopenlog method to FreeBSD rc script
	* Adopt foreach for cmake 2.6
	* Fix to rc script
	* Do not try to resolve names with several dots in a row
	* Fix surbl request formatting for ip addresses
	* Handle cases of broken requests
	* Fix problems with parsing compressed names
	* Fix TXT records parsing
	* Fix expanding spf macros that may fail in rare cases
	* Fix another error with early task destroying
	* Handle empty from header
	* Improve reopenlog command in rc script
	* Strip trailing whitespace characters in Mail::Rspamd::Client
	* Use ungreedy match to strip trailing whitespaces
	* Stupid error in calculation compressed label length
	* Some optimizations to client library
	* Do not compare empty parts
	* Empty and non-empty parts are allways different
	* Save in regexp cache the whole regexp with header name (if exists) and with flags
	* Add rspamd_log variable to lua plugins to access logging functions
	* Each part in rspamd task now can have parent part
	* Check for parts distance only for multipart/alternative subparts
	* Do not check attachements even if they are text (but attached as file)
	* Do not die if write (2) returned ENOSPACE while doing logging, turn on throttling mode instead (1 write try in a second)
	* Add ability to turn on debug for specific symbols
	* Add ability to configure dns timeouts and dns retransmits in config file
	* More debug
	* Fix extracting arguments in lua logger interface
	* Turn off debug_ip during reload if it was disabled by new config
	* Improve lua logging
	* Pre-init symbols cache when rereading config
	* Fix lua representing of invalid ip (nil, not 255.255.255.255)
	* Fix R_TO_SEEMS_AUTO rule (by citrin)
	* Add multimap lua plugin
	* Fix some multimap issues
	* Try to save images hashes to fuzzy storage to stop some annoying spammers
	* Allocate some more bytes for read buffer to avoid incorrect behavoiur
	* Add ability to check dns black lists by multimap module
	* Add multimap documentation
	* Fix labels parsing
	* Another try to save regexps in cache correctly
	* Improve test logs for regexps
	* Fix parsing txt records to avoid reading of uninitialized data
	* Fix error with writing symbols cache file
	* Fix error while working in utf mode when raw regexps was not created properly
	* Do not add extra byte while converting text to utf
	* Add error handling for regexps