path: root/ChangeLog

0.7.5:
	* Fix owner when creating folder /run/rspamd (by @sfirmery)
	* Fix IP validity checks
	* Decode URLs obtained from HTML tags
	* Fix crash with unweighted upstreams
	* Stop processing headers in parts
	* Set sockaddr.sa_family properly when connectig to upstreams
	* Fix reload issues in surbl and fuzzy_check (reported by @citrin)
	* Fix timeouts in redirector
	* Improve lua errors reporting
	* Fix lua closures processing in libucl
	* Rework calling of lua functions from regexp module
	* Choose raw regexp for raw headers
	* Rework conversion to utf since glib one is broken
	* Ignore SGML style tags in html
	* Fix old bug with non-capturing https urls
	* Fix memory corruption on fuzzy reload (reported by @citrin)
	* Fix percents display in rspamc
	* Fix buffer update for DKIM
	* Do not validate utf for raw headers

0.7.4:
	* Fix build under *BSD
	* Detect HAN unicode script
	* Implement language detection heuristic for text parts
	* Fix time output in history
	* Improve piechart coloring
	* Fix \r\n conversion in DKIM module (reported by @citrin)
	* Try to detect systems with no IPv6 support
	* Fix multiple/single values in use settings (reported by @citrin)
	*  Rework IP addresses in upstreams:
		- Select ipv4/unix addresses if they exist and use ipv6 for ipv6 only
		  upstreams (since the support of ipv6 is poor in many OSes and
		  environments)
		- Free IP list on upstream destruction
		- Add test cases for addresses selection
		- Allow adding of free form IP addresses to upstreams
    * Fix endiannes in lua_radix search (reported by @citrin)
    * Soft shutdown should also set wanna_die flag (reported by @citrin)
    * Stop use-after-free in event loop termination
    * Fix processing of very short messages in DKIM (reported by @citrin)
    * Detect systems without shared mutexes
    * Fix issues with PTR and MX elements in SPF parser (reported by @citrin)

0.7.3:
	* New upstreams code:
		- simplify upstreams API;
		- unify strings parsing in upstreams definition;
		- add configuration options for the upstreams;
		- for failed upstreams re-resolve their addresses;
		- use all resolved addresses for an upstream (round-robin);
		- implement stable hashing and use it by default for upstreams;
		- add unit test for upstreams module.
	* Rework signals processing in all rspamd workers:
		- signals are now processed in the event loop;
		- implement the most common signal handlers for all workers;
		- add callbacks for workers specific signal handlers
	*  Fix critical issue with fuzzy storage:
		Fuzzy stroage could not save any hashes on termination due to bugged
		signals handling
    * Fix roll history IP storage
    * Rework ipv4/ipv6 handling in parsing addresses:
		- turn off support of IPV6_V6ONLY socket option;
		- create ipv6 socket prior to ipv4 one to handle systems with v6/v4 
		sockets enabled (Linux)
    * Remove CBL as it's wholly included in Spamhaus XBL (by @fatalbanana)
    * Remove nszones.com fake RBL (by @citrin)
    * Fix upstreams interaction for fuzzy_check
    * Verify spf PTR records (reported by @citrin)
    * Fix spf MX records parsing
    * Add compatibility for old libevent (by @yellowman)
    * Sync bugfixes from libucl

0.7.2:
	* Convert all maps to the compressed radix trie
	* Allow IPv6 addresses in IP maps
	* Remove dynamic items support from symbols cache
	* Allow hex encoded output of strings
	* Fix bug with control connections count
	* Process fuzzy weight correctly (reported by @fatalbanana)
	* Remove extra reference retain of http connection on error
	* Remove deprecated options from the default config
	* Add `one_shot` attr to metric's symbols
	* Doc: add documentation for metrics
	* Add Upstart job to debian packaging (by @CameronNemo)
	* Config: improve SURBL symbols descriptions (by @citrin)
	* Config: reflect SURBL changes (by @citrin):
    	- Outblaze removed, malware moved to separate list:
    	http://www.surbl.org/news/internal/MW-malware-sublist-added-to-multi
    * Fix C modules initialization on restart
    * Treat single IP as a single IP in radix lists (reported by @citrin)
    * Do not touch file and core limits if not asked explicitly (reported by @citrin)
    * Improve logging for fuzzy errors
    * Block SIGPIPE for HTTP writing
    * Doc: update manual pages
    * Fix HTTP connection termination
    * Reduce default number of parallel requests to 8
    * Sync with libucl include features

0.7.1:
	* Fix typo in stat output.
	* Fix issues with includes crossing with the system includes
	* Restore testing framework
	* Add radix trie test suite
	* Implement new path-compressed radix trie.
	    - The performance benefit over the old algorithm is about 1.5 times.
		- Memory usage is significantly lower as well.
	    - Now radix trie can accept any IPv4/IPv6 values
	* Various improvements to the memory pools code
	* Fix writing reply to a client when no filters are defined
	* Write base32 encoded fuzzy
	* Fix 'soft reject' action
	* Fix rspamd reload and modules reconfiguration
	* Fix subject rewriting for the default subject
	* Fix states for processing task and pre-filters
	* Fix issues with connection closing
	* Fix crashes in rdns
	* Fix ratelimit pre-filter
	* Update exim patch.
		- Update to the recent exim version
		- Strip extra leading src/ from the patch
		- Remove sendfile since it was broken
		- Fix rspamd spam report for exim
	* Improve documentation

0.7.0:
	* Use HTTP protocol for all operatiosn
	* Webui worker is now removed and controller works as webui
	* Allow to serve static files via controller by option `static_dir`
	* Rspamd interface is now a part of rspamd
	* Rspamc client has been rewritten to use HTTP and non-blocking mode
	allowing to start multiple operations simultaneously (see `-n` option)
	* Lua API was completely reworked to satisfy modern standards of LUA:
		* Module `lua-message` was removed
		* Reduced number of superglobals registered by rspamd
		* Many functions has been redesigned
		* Symbols registration is now more convenient
	* Users settings has been rewritten as lua plugin
	* Reworked headers system as gmime's based one misses many headers and is
	very slow to get headers values
	* Reorganized code and removed embedded jannsson by using UCL for all json
	parsing
	* Migrated to `librdns` for DNS resolving that improves concurrency for
	DNS requests significantly
	* Fixed tonns of bugs in MIME processing
	* Improved metrcis and default symbol's weights
	* Added new RBL's
	* Fixed a number of issues in the modules
	* Removed several memory leaks found
	* Fix unicode processing
	* Fix fuzzy checking for unicode parts
	* Significantly improve documentation and especially LUA API docs
	* See migration notes at https://rspamd.com/doc/migration.html

0.6.8:
	* Controller now listen for localhost and not for 127.0.0.1 by default
	* Allow FCrDNS-style RBL lookups (by @fatalbanana)
	* Reduce threshold for parts_differ function.
	* Fix hostname lookup for rdns rbl (by @AlexeySa)
	* Fix HFILTER_URL_ONELINE to reduce false positive rate.
	* Fix whitelist module.
	* Allow override system predefined settings without touching system ones
	by .try_include macro (by @andrejzverev)
	* Check for [ip.address]-style HELO and suppress lookups. (by
	@fatalbanana)
	* Optimize hfilter (by @AlexeySa)
	* Fix issue with random numbers generator in dns.
	* Use more clever time values to setup entropy.
	* Synced with the recent libucl.

0.6.7:
	* Use ChaCha20 for DNS generator (more secure DNS id)
	* Unknown symbols now has zero weight and not 1.0
	* Fix fuzzy hashes expire time
	* Fix critical issue in statfiles on FreeBSD 9 (and some other platforms)
	* Add .include_map macro to ucl parser
	* Update libucl
	* Fix headers end detection for DKIM module
	* Fix a bug in received headers parser
	* Validate IP addresses before pushing them to lua
	* Start new documentation project
	* Fixed tonns of other minor bugs
	* Start to prepare for 0.7 with HTTP protocol and new settings

0.6.6:
	* Removed issue with BUFSIZ limitation in the controller output
	* Simplify logging symbols escaping
	* Adjusted weights for several rules
	* Improve spamhaus rbl support
	* Removed PBL for received headers checks
	* Added hfilter module that performs various HELO and IP checks.
	* Rspamd can now be reloaded by HUP signal
	* Fuzzy storage should expire hashes properly
	* Build system has been reworked for better supportof pkg-config
	* Various minor bugfixes

0.6.5:
	* Fixed critical bug in DNS resolver, introduced in 0.6.4
	* Improved multimap and rbl plugins to skip
	* Add dns_sockets option for tuning sockets per server in DNS resolver
	* Improved packages for rspamd

0.6.4:
	* Added io channels for DNS request to balance load and reduce id
	collisions chance
	* Fixed a bug in SPF filter that may cause core dump in specific
	circumstances
	* FIxed default config for rbl module
	* It is possible to get a list of rspamc commands with their descriptions
	* Added SORBS bl to the default config
	* 2tld file for surbl module has been significantly extended
	* Perl modules has been removed from the code.
	* Fixed an issue in libucl when parsing macros

0.6.3:
	* Fixed issues with DNS:
	 - labels decompression algorithm was fixed;
	 - added resolve_mx to lua API;
	 - fixed modules that use DNS.
	* Lua modules once_received and emails reworked for new resolver API and UCL.
	* Debian package was polished.
	* Fixed a bug in fuzzy_check module that prevents correct processing messages 
	  without valid parts.

0.6.2:
	* Fuzzy check module has been reworked:
	 - now fuzzy_check operates with a group of rules, that define which
	 servers sre to be checked;
	 - it is possible to specify read_only groups to avoid learning errors;
	 - turn fuzzy_check to one_shot mode permanently;
	 - fuzzy_check module configuration is now incompatible with the previous
	 versions.
	* Imported bugfixes from libucl.
	* Fixed whitelist plugin.
	* Fixed statfiles resizing.
	* Improved logging initialization order.
	* Fixed race condition in the controller worker.

0.6.1:
	* Critical bugfixes:
	 - fixed build system;
	 - fixed in_class setting in bayes learning;

0.6.0:
	* Use UCL instead xml for configuration (https://github.com/vstakhov/libucl)
	* Fix statistics module normalization
	* Rework the many modules for the new configuration:
	 - surbl has incompatible configuration;
	 - fuzzy_check has incompatible configuration;
	 - multimap has now new configuration;
	 - received_rbl is removed and replaced by rbl module.
	* Removed deprecated options:
	 - statfile_pool_size;
	 - action and required/reject score for a metric.
	* Simplify build system and unify configuration for all platforms.
	* Improved URL detector (reduced false positive rate).
	* Lua 5.2 is now the default and fully supported version.
	* Tons of bugfixes and minor improvements.	

0.5.6:
	* Fix bug with counters incrementing in rolling history
	* Detect expl and exp2l as some systems do not have it
	* Support input streams without Content-Length
	* Implement counters output via rspamc and controller interface
	* Fix bug with udp sockets in fuzzy storage

0.5.5:
	* New bayes normalizator based on inverse chi-square function
	* Various fixes to fuzzy storage
	* Allow update fuzzy storage only from specific IP addresses
	* Better support of IPv6 and address selection algorithms
	* Add CentOS spec file

0.5.4:
	* Fixed issues with diff algorithm
	* Added support of RRD statistics
	* Add webui worker for interface interaction
	* Fix a lot of issues with dynamic conf
	* Fix critical memory leak in settings code
	* Improve stability of the system

0.5.3:
	* Added dynamic options
	* Added advanced metaclassfication
	* Added RESTfull API for controller
	* Improved hashing algorithms
	* Various fixes for rspamc client:
	 - allow interacting with unix sockets
	 - librspamdclient major cleanup
	 - bayes is now default classifier
0.5.2:
	* Added lua bindings for:
	 - basic mime parts, that allows checkign attachements for example;
	 - DNS resolver;
	* Existing lua bindings now works without task object allowing to use them
	in custom code.
	* Threads system was reworked to avoid global lua interpreter lock.
	* DKIM module now converts all line endings to CRLF how opendkim does.
	* URL detector is now more accurate for text parts.
	* Several critical bugs and memory leaks were fixed.

0.5.1:
	* Added lua worker type to handle network connections in lua.
	* Added lua bindings for async_session, IO dispatcher, memory_pool, and
	worker.
	* Composites can now uses other composites in expressions.
	* Fixes for debian package and for FreeBSD >= 9.1
	* Add support of gmime-2.6 if gmime-2.4 is not found.
	* Improve url detection and phishing detection.
	* Add lua mime_part library to get an access to all message part
	attributes (like filename, length, type).

0.5.0:
	* Added SMTP lightweight balancing proxy with XCLIENT support.
	* Added lua bindings for upstreams objects and API.
	* New pre-filters are implemented to support initial checking for messages.
	* Added ratelimit plugin that uses redis protocol to store data.
	* Added ipv6 support to spf and some other modules.
	* Unbreak spf plugin.
	* Allow options with the same name be threated as list.
	* DKIM plugin an parsing code was added.
	* Separate build system to put logic in several shared libraries.
	* Many bugfixes.

0.3.2:
	* Add error handling for regexps
	* Fix quit command in controller interface
	* Write symbols weights to rspamc output
	* Improve logic of selecting rspamc version
	* Do not try to parse broken DNS replies
	* Add 'raw' flag to FROM_EXCESS_BASE64 rule (requested by citrin)
	* Output message id in rspamc reply
	* Fix inserting composite symbol
	* Fix output of log line
	* Document composites
	* Add logging for fuzzy checks
	* Add logging for learning
	* Improve logic of learning messages - do not learn more than specific threshold
	* Fix inserting results for symbols that were incorrectly (for example more than 1 time) defined in config file
	* Do not output control characters if output is not terminal
	* Fix some logic errors in learning
	* Consider lua plugins errors as fatal configuration errors
	* Fix wirting message id during fuzzy_add command
	* Display weight of symbols correctly
	* Fixes to winnow learning
	* One more try to improve accuracy of winnow algorithm
	* Add bayesian classifier (initial version)
	* Remove normalizer as it is winnow specific thing, so all statistic algorithms now returns value from 0 to 1
	* Some fixes to fuzzy hashes expiration:
	* Fix assertion while look up value in NULL hash (found by cirtin)
	* Fix normalization for systems that have not tanhl function
	* Ignore rfc822 group addresses
	* Move images library to core rspamd
	* Add lua api to access images properties
	* Add post filters to lua API - filters that would be called after all message's processing
	* Add ability to check for specified symbol in task results from lua
	* Add ability to check for metric's results from lua
	* Add ability to learn specified statfile form lua
	* Add ability to extract filename and size of images from lua
	* Fix assertion while extracting internet address
	* Fix races in fuzzy storage
	* Make spf parser case insensitive
	* Add ability to check hashes of selected mime types
	* Add ability to set minimum size in bytes for mime types
	* Add ability to set minimum dimensions for images
	* Assume all text/* content types as text/plain
	* Fix getting data wrapper for gmime24
	* Many fixes to statfile syncronization system
	* Fixed statfile pool initialization and synchronization with disk
	* Prepare 0.3.2
	* Fix Mail::Rspamd::Config for new rspamd features
	* Use Mail::Rspamd::Config in rspamc client
	* Write user's name to rspamd log
	* Prepare rspamd build infrastructure for creating rpm and deb packages
	* Fix depends
	* Add start script for linux systems
	* Fix shared usage of statfiles
	* Add invalidation of statfiles in case of learning, so now statfiles
	* This should fix shared usage of statfile pool by several processes
	* Fix misprint (reported by az)
	* Fix stupid error when all checks can be done in a single pass
	* New trie based url scanner (based on libcamel)
	* Small fixes to rspamd perl client
	* Write fuzzy hashes info to log
	* Add trie interface to lua api
	* Explain sample config and cleanup it

0.3.1:
	* Add modules documentation
	* Continue implementing smtp proxy
	* Implement new learning system, now rspamd should be much more intelligent while learning messages
	* Convert statistic sums to use long double for counters
	* Use hyperbolic tangent for internal normalizer
	* In classify normalize result after comparing, not before
	* New symbols sorter
	* Fix strict aliasing while compiling with optimization
	* Fix tanhl detection for platforms that have not implementation of it
	* Remove several compile warnings
	* Add experimental support of dynamic rules to regexp module
	* Document views configuration
	* Several fixes to documentation
	* Add more logic for dynamic rules
	* Add documentation for dynamic rules
	* Add ability to make negations in networks in dynamic rules
	* Clean up cache items correctly
	* Implement basic SMTP dialog:
	* Implement interaction with smtp upstream (with support of XCLIENT)
	* Check messages received via smtp proxy
	* Add support for sendfile in io dispatcher
	* Fix issues with compatibility of worker_task and smtp proxy
	* Proxy DATA command
	* Fix SMTP
	* Change metric logic
	* Completely remove lex/yacc readers for config
	* Make common sense of metric/action and symbols
	* Sync changes with all plugins
	* Incorrectly removed in previous commit
	* Fix misprint (by Andrej Zverev)
	* announce the "password" keyword in usage list
	* Implement initial version of greylisting triplets storage
	* Fix issues with smtp worker
	* Fix QUIT command in SMTP worker
	* Some fixes about new metrics system (may be incomplete)
	* Get weights of symbol from default metric for symbols cache
	* Fix setting task->from/task->rctp in smtp client
	* Copy from and rcpt correctly
	* Some performance improvements to IO dispatcher (do not drain the whole buffer after a single line readed)
	* Fix smtp data input
	* Fix misprint
	* Add limit of maximum allowed smtp session errors
	* New logic of SURBL module:
	* Use system mkstemp(3) on systems where it is available as glib implementation
	* Try to fix memmove issues in io dispatcher
	* Remove debug from SURBL module
	* Rewrite buffered input for line policy (again)
	* Fix issue with links that are ip addresses in numeric form in surbl
	* On Darwin use BSD style sendfile definition
	* Reorganize platform specific knobs in CMakeLists
	* Use gettimeofday on systems that have not clock_getres
	* Use ftime for dns trans id generation on systems without clock_getres
	* Darwin sendfile(2) support
	* TIMEDB->TIMEB
	* More to previous commit
	* Pass env from main() arguments instead of platform specific global environ
	* Fix compatibility issues
	* Fix -lintl detection
	* Init some variables to avoid problems
	* Remove garbadge (gnome terminal sucks)
	* Add more information about why we drop smtp connection
	* Fix mkstemp call
	* Send to upstream QUIT command at the end of session
	* Check return value of each rspamd_dispatcher_write as in case of write errors sessions can be destroyed early
	* Fix states in smtp dialog
	* Use rspamd_snprintf instead of libc one
	* Fix URLS command
	* Fix reconfigure process of surbl module
	* Fix destroying smtp session (unmap memory and do not delete pool early)
	* Delete pool after using its variables
	* Delay timer must be registered in async session to correctly handle connection termination
	* Register dns requests in session too
	* Make session before registering events
	* Remove events in handlers
	* Add ability to set filters for smtp worker for each smtp stage
	* Add very initial version of DNS resolver (many things to be done)
	* Announce weights and sync
	* Fix few typo
	* Understand short names of facility in logging config
	* Add ability to make whitelist for spf checks
	* Misprint != -> ==
	* Handle lua tag in way that it is not required to write additional text:
	* Strip all starting whitespace symbols from xml texts
	* Fix stupid bug in calculating buffer length while reading file maps
	* Add resolv.conf parsing into dns.c
	* Fix microseconds<->milliseconds conversions
	* Take callback argument in Mail::Rspamd::Client for processing files and directories
	* Print results if rspamc is called for a directory
	* Fix stupid error with surbl module reconfig (another one, blame me)
	* Do not show duplicate urls in url header
	* Fix detection of numeric urls (reported by citrin)
	* Write real time of message's scan to log (not only virtual)
	* Fix chartable module in utf mode
	* Fix parsing of some broken urls
	* Add ability to test regexp with 'T' flag
	* Write more code for DNS resolver:
	* Make DNS resolver working
	* Many improvements to rspamd test suite: now it CAN be used for testing rspamd functionality
	* Write DNS resolver tests
	* Fix issues with memory_pool mutexes and with creating of statfiles
	* Forgotten in previous commit
	* Add support for parsing SPF and SRV records
	* Fix PTR parsing
	* Add tests
	* Make SURBL module to use rspamd dns module
	* Several fixes to DNS logic
	* Remove evdns and use only rspamd resolver
	* Very hard to detect problem with race among error in socket and destroying task while we are writing to socket and go through a hash table
	* Fix resolving in smtp module
	* Init events before configuring resolver in smtp worker
	* Set resolver inside task
	* Fix reload signal (reported by citrin)
	* Some improvements to redirector
	* Call has_forked method to inform POE about fork
	* Fix lua DNS code
	* Decompress labels in DNS packets more strictly
	* Fix some problems with TXT records
	* Try to fix removing of DNS events
	* Do not insert unparsed RR's into reply
	* Calling callbacks may cause destroying session from which we are calling callback so we MUST call callback as the latest action
	* Fix check_smtp_data function
	* Add ability to make views by recipient
	* Add ability to set metric's action from config file
	* Fix bug with writing garbadge if message has no urls or no messages
	* Fix bug with incorrect behaviour of compare_parts_distance function
	* Add ability to assign several actions to one metric
	* Report action in rspamc protocol
	* Mail::Rspamd::Client and rspamc can now understand Action header too
	* Write action to log as well
	* Make valgrind happy about comparing symbols
	* Add more debug to comparing parts distance function
	* Write action even if message has no symbols
	* Make improvements to HTML entites decoder: now it replaces entities with common characters and
	* Add -d option to force debug output
	* Assume 7bit as default transfer encoding
	* Do not overwrite lua plugins and configs if they already exists in target directory
	* Improve logging
	* Write queue id to log
	* Remove test messages from cmake
	* Reopen log file by USR1 signal
	* Add reopenlog method to FreeBSD rc script
	* Adopt foreach for cmake 2.6
	* Fix to rc script
	* Do not try to resolve names with several dots in a row
	* Fix surbl request formatting for ip addresses
	* Handle cases of broken requests
	* Fix problems with parsing compressed names
	* Fix TXT records parsing
	* Fix expanding spf macros that may fail in rare cases
	* Fix another error with early task destroying
	* Handle empty from header
	* Improve reopenlog command in rc script
	* Strip trailing whitespace characters in Mail::Rspamd::Client
	* Use ungreedy match to strip trailing whitespaces
	* Stupid error in calculation compressed label length
	* Some optimizations to client library
	* Do not compare empty parts
	* Empty and non-empty parts are allways different
	* Save in regexp cache the whole regexp with header name (if exists) and with flags
	* Add rspamd_log variable to lua plugins to access logging functions
	* Each part in rspamd task now can have parent part
	* Check for parts distance only for multipart/alternative subparts
	* Do not check attachements even if they are text (but attached as file)
	* Do not die if write (2) returned ENOSPACE while doing logging, turn on throttling mode instead (1 write try in a second)
	* Add ability to turn on debug for specific symbols
	* Add ability to configure dns timeouts and dns retransmits in config file
	* More debug
	* Fix extracting arguments in lua logger interface
	* Turn off debug_ip during reload if it was disabled by new config
	* Improve lua logging
	* Pre-init symbols cache when rereading config
	* Fix lua representing of invalid ip (nil, not 255.255.255.255)
	* Fix R_TO_SEEMS_AUTO rule (by citrin)
	* Add multimap lua plugin
	* Fix some multimap issues
	* Try to save images hashes to fuzzy storage to stop some annoying spammers
	* Allocate some more bytes for read buffer to avoid incorrect behavoiur
	* Add ability to check dns black lists by multimap module
	* Add multimap documentation
	* Fix labels parsing
	* Another try to save regexps in cache correctly
	* Improve test logs for regexps
	* Fix parsing txt records to avoid reading of uninitialized data
	* Fix error with writing symbols cache file
	* Fix error while working in utf mode when raw regexps was not created properly
	* Do not add extra byte while converting text to utf
	* Add error handling for regexps