]> source.dussan.org Git - rspamd.git/log
rspamd.git
9 years agoUpdate version number to 0.9.6.
Vsevolod Stakhov [Tue, 16 Jun 2015 15:17:28 +0000 (16:17 +0100)]
Update version number to 0.9.6.

9 years agoUpdate->add
Vsevolod Stakhov [Tue, 16 Jun 2015 14:32:44 +0000 (15:32 +0100)]
Update->add

9 years agoFix digest invocation.
Vsevolod Stakhov [Tue, 16 Jun 2015 14:16:53 +0000 (15:16 +0100)]
Fix digest invocation.

9 years agoFix variable name.
Vsevolod Stakhov [Tue, 16 Jun 2015 13:33:33 +0000 (14:33 +0100)]
Fix variable name.

9 years agoFix issues with redirector HTTP response.
Vsevolod Stakhov [Tue, 16 Jun 2015 13:22:38 +0000 (14:22 +0100)]
Fix issues with redirector HTTP response.

9 years agoAdd workaround for #296.
Vsevolod Stakhov [Tue, 16 Jun 2015 12:33:19 +0000 (13:33 +0100)]
Add workaround for #296.

9 years agoFix dkim cache expiration.
Vsevolod Stakhov [Fri, 12 Jun 2015 17:57:00 +0000 (13:57 -0400)]
Fix dkim cache expiration.

9 years agoFix memory leak if mime cannot be parsed.
Vsevolod Stakhov [Tue, 2 Jun 2015 15:42:48 +0000 (16:42 +0100)]
Fix memory leak if mime cannot be parsed.

9 years agoRelease 0.9.5 0.9.5
Vsevolod Stakhov [Fri, 29 May 2015 12:08:35 +0000 (13:08 +0100)]
Release 0.9.5

* Avoid double free when extending HTTP message.
* Fix double free if multiple classifiers are defined.
* Fix misprint in spamassassin plugin.
* Fix cpuid invocation on i386.
* Fix ownership issues for zero-copy decode.
* Allow __len metamethod on rspamd{text}.
* Add base64 decoding lua utility.
* Fix build on FreeBSD
* Skip spaces at the beginning of mime messages.
* DBL_ABUSE_REDIR should not have significant weight.
* Allow to split by lua_regexp rspamd{text} objects.
* Allow to specify custom stop pattern for lua_tcp.

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years agoAvoid double free when extending HTTP message.
Vsevolod Stakhov [Fri, 29 May 2015 10:47:22 +0000 (11:47 +0100)]
Avoid double free when extending HTTP message.

9 years agoFix double free if multiple classifiers are defined.
Vsevolod Stakhov [Wed, 27 May 2015 10:08:41 +0000 (11:08 +0100)]
Fix double free if multiple classifiers are defined.

9 years agoIf replacement cannot be applied, just remove regexp.
Vsevolod Stakhov [Tue, 26 May 2015 16:37:45 +0000 (17:37 +0100)]
If replacement cannot be applied, just remove regexp.

9 years agoFix misprint in spamassassin plugin.
Vsevolod Stakhov [Tue, 26 May 2015 16:03:09 +0000 (17:03 +0100)]
Fix misprint in spamassassin plugin.

9 years agoFix i386 cpuid invocation.
Vsevolod Stakhov [Tue, 26 May 2015 11:33:04 +0000 (12:33 +0100)]
Fix i386 cpuid invocation.

9 years agoAdd unit tests for b64 decoding and native encoding.
Vsevolod Stakhov [Mon, 25 May 2015 16:52:38 +0000 (17:52 +0100)]
Add unit tests for b64 decoding and native encoding.

9 years agoFix ownership issues for zero-copy decode.
Vsevolod Stakhov [Mon, 25 May 2015 16:52:14 +0000 (17:52 +0100)]
Fix ownership issues for zero-copy decode.

9 years agoAllow __len metamethod on rspamd{text}.
Vsevolod Stakhov [Mon, 25 May 2015 16:51:57 +0000 (17:51 +0100)]
Allow __len metamethod on rspamd{text}.

9 years agoAdd base64 decoding lua utility.
Vsevolod Stakhov [Mon, 25 May 2015 15:44:24 +0000 (16:44 +0100)]
Add base64 decoding lua utility.

9 years agoFix build on FreeBSD < 10.
Vsevolod Stakhov [Sun, 24 May 2015 17:00:41 +0000 (18:00 +0100)]
Fix build on FreeBSD < 10.

9 years agoSkip spaces at the beginning of mime message.
Vsevolod Stakhov [Sun, 24 May 2015 15:53:13 +0000 (16:53 +0100)]
Skip spaces at the beginning of mime message.

9 years agoDBL_ABUSE_REDIR should not have significant weight.
Vsevolod Stakhov [Sat, 23 May 2015 21:18:59 +0000 (22:18 +0100)]
DBL_ABUSE_REDIR should not have significant weight.

9 years agoAllow to split by lua_regexp rspamd{text} objects.
Vsevolod Stakhov [Sat, 23 May 2015 20:59:00 +0000 (21:59 +0100)]
Allow to split by lua_regexp rspamd{text} objects.

9 years agoAllow to specify custom stop pattern for lua_tcp.
Vsevolod Stakhov [Thu, 21 May 2015 17:26:20 +0000 (18:26 +0100)]
Allow to specify custom stop pattern for lua_tcp.

9 years agoRelease 0.9.4 0.9.4
Vsevolod Stakhov [Thu, 21 May 2015 13:28:22 +0000 (14:28 +0100)]
Release 0.9.4

* Fix critical bugs in tokenization algorithm
* Write unit tests for tokenization
* Add documentation for lua_tcp
* Switch off legacy tokenization by default.
* Fix critical bugs in words normalization
* Add lua bindings to tokenizer.
* Implement storing of HTTP headers inside task
* Add lua API to accerss HTTP headers data
* Implemented base64 encoding suitable for MIME
* Use caseless hash and equal functions for HTTP request headers.
* Improve debian architectures support (by @dottedmag)

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years agoFix possible illegal memory access.
Vsevolod Stakhov [Thu, 21 May 2015 13:10:52 +0000 (14:10 +0100)]
Fix possible illegal memory access.

9 years agoMerge pull request #287 from dottedmag/master
Vsevolod Stakhov [Thu, 21 May 2015 13:20:31 +0000 (14:20 +0100)]
Merge pull request #287 from dottedmag/master

Debian changes for 0.9.4

9 years agoSwitch off legacy tokenization by default.
Vsevolod Stakhov [Thu, 21 May 2015 12:31:39 +0000 (13:31 +0100)]
Switch off legacy tokenization by default.

9 years agoDo not stem exceptions.
Vsevolod Stakhov [Thu, 21 May 2015 12:31:28 +0000 (13:31 +0100)]
Do not stem exceptions.

9 years agoUse not common name for tokenization exceptions.
Vsevolod Stakhov [Thu, 21 May 2015 11:52:54 +0000 (12:52 +0100)]
Use not common name for tokenization exceptions.

9 years agoFix memory issues with tokens normalization.
Vsevolod Stakhov [Thu, 21 May 2015 11:47:40 +0000 (12:47 +0100)]
Fix memory issues with tokens normalization.

9 years agoAdd more unit tests for tokenization.
Vsevolod Stakhov [Thu, 21 May 2015 09:23:52 +0000 (10:23 +0100)]
Add more unit tests for tokenization.

9 years agoMore fixes to tokenization.
Vsevolod Stakhov [Thu, 21 May 2015 09:23:41 +0000 (10:23 +0100)]
More fixes to tokenization.

9 years agoReverse list of exceptions.
Vsevolod Stakhov [Thu, 21 May 2015 09:23:26 +0000 (10:23 +0100)]
Reverse list of exceptions.

9 years agoAdd tests for tokenizer bug.
Vsevolod Stakhov [Wed, 20 May 2015 21:53:09 +0000 (22:53 +0100)]
Add tests for tokenizer bug.

9 years agoFix critical bug in tokenization logic.
Vsevolod Stakhov [Wed, 20 May 2015 21:52:46 +0000 (22:52 +0100)]
Fix critical bug in tokenization logic.

9 years agoAdd a simple unit test for tokenizer.
Vsevolod Stakhov [Wed, 20 May 2015 21:37:33 +0000 (22:37 +0100)]
Add a simple unit test for tokenizer.

9 years agoAdd lua bindings to tokenizer.
Vsevolod Stakhov [Wed, 20 May 2015 21:17:45 +0000 (22:17 +0100)]
Add lua bindings to tokenizer.

9 years agoUpdate changelog 287/head
Mikhail Gusarov [Wed, 20 May 2015 18:37:14 +0000 (18:37 +0000)]
Update changelog

9 years agoRestrict luajit B-D to architectures it is available
Mikhail Gusarov [Wed, 20 May 2015 18:35:44 +0000 (18:35 +0000)]
Restrict luajit B-D to architectures it is available

9 years agoUse caseless hash and equal functions for request headers.
Vsevolod Stakhov [Wed, 20 May 2015 16:34:15 +0000 (17:34 +0100)]
Use caseless hash and equal functions for request headers.

9 years agoAdd API to get/set request headers from lua.
Vsevolod Stakhov [Wed, 20 May 2015 16:28:36 +0000 (17:28 +0100)]
Add API to get/set request headers from lua.

9 years agoSave all HTTP headers as request headers.
Vsevolod Stakhov [Wed, 20 May 2015 15:30:56 +0000 (16:30 +0100)]
Save all HTTP headers as request headers.

9 years agoAdd unit test for base64 encoding.
Vsevolod Stakhov [Wed, 20 May 2015 13:59:24 +0000 (14:59 +0100)]
Add unit test for base64 encoding.

9 years agoFix issues in base64 encoding.
Vsevolod Stakhov [Wed, 20 May 2015 13:59:10 +0000 (14:59 +0100)]
Fix issues in base64 encoding.

9 years agoAdd lua bindings for base64.
Vsevolod Stakhov [Wed, 20 May 2015 11:40:19 +0000 (12:40 +0100)]
Add lua bindings for base64.

9 years agoAllow owning lua_text.
Vsevolod Stakhov [Wed, 20 May 2015 11:03:46 +0000 (12:03 +0100)]
Allow owning lua_text.

9 years agoAdd base64 encoding routine.
Vsevolod Stakhov [Wed, 20 May 2015 10:49:51 +0000 (11:49 +0100)]
Add base64 encoding routine.

9 years agoAdd documentation for lua_tcp.
Vsevolod Stakhov [Tue, 19 May 2015 17:38:09 +0000 (18:38 +0100)]
Add documentation for lua_tcp.

9 years agoRelease 0.9.3 0.9.3
Vsevolod Stakhov [Tue, 19 May 2015 16:20:19 +0000 (17:20 +0100)]
Release 0.9.3

* Revert incorrect regexp change that broke the default rules
* Fix lua_tcp module

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years agoFix issues with lua_tcp.
Vsevolod Stakhov [Tue, 19 May 2015 16:16:31 +0000 (17:16 +0100)]
Fix issues with lua_tcp.

9 years agoRevert automatic capture in regexps as it breaks things.
Vsevolod Stakhov [Tue, 19 May 2015 15:56:27 +0000 (16:56 +0100)]
Revert automatic capture in regexps as it breaks things.

9 years agoRelease 0.9.2 0.9.2
Vsevolod Stakhov [Tue, 19 May 2015 15:34:21 +0000 (16:34 +0100)]
Release 0.9.2

* Fix error on spawning unique workers.
* Add preliminary version of generic LUA TCP requests API.
* Use lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc)
* Fix fuzzy mime strings with only type.
* Improve thunderbird sanity checks.
* Fix critical bug on matching regular expressions.
* Make hiredis optional dependency.
* Fix multiple bugs in daemon reloading

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years agoFix reload.
Vsevolod Stakhov [Tue, 19 May 2015 15:29:03 +0000 (16:29 +0100)]
Fix reload.

9 years agoFix reloading in logger.
Vsevolod Stakhov [Tue, 19 May 2015 15:28:56 +0000 (16:28 +0100)]
Fix reloading in logger.

9 years agoOne more try to fix memset.
Vsevolod Stakhov [Tue, 19 May 2015 15:28:42 +0000 (16:28 +0100)]
One more try to fix memset.

9 years agoFix reloading of C plugins.
Vsevolod Stakhov [Tue, 19 May 2015 14:37:31 +0000 (15:37 +0100)]
Fix reloading of C plugins.

9 years agoFix reloading of modules.
Vsevolod Stakhov [Tue, 19 May 2015 14:37:20 +0000 (15:37 +0100)]
Fix reloading of modules.

9 years agoDo not try dereference NULL pointer.
Vsevolod Stakhov [Tue, 19 May 2015 14:37:02 +0000 (15:37 +0100)]
Do not try dereference NULL pointer.

9 years agoMerge pull request #284 from dottedmag/master
Vsevolod Stakhov [Tue, 19 May 2015 13:34:18 +0000 (14:34 +0100)]
Merge pull request #284 from dottedmag/master

Updates for 0.9.2 Debian release

9 years agoAdd method to detect the full size of message.
Vsevolod Stakhov [Tue, 19 May 2015 12:58:07 +0000 (13:58 +0100)]
Add method to detect the full size of message.

9 years agoFallback to plain lua if luajit has not been found.
Vsevolod Stakhov [Tue, 19 May 2015 12:51:50 +0000 (13:51 +0100)]
Fallback to plain lua if luajit has not been found.

9 years agoMake hiredis optional dependency.
Vsevolod Stakhov [Tue, 19 May 2015 12:43:26 +0000 (13:43 +0100)]
Make hiredis optional dependency.

9 years agoFix critical bug on matching regular expressions.
Vsevolod Stakhov [Tue, 19 May 2015 12:32:09 +0000 (13:32 +0100)]
Fix critical bug on matching regular expressions.

Issue: #186
Reported by: @moisseev

9 years agoImprove thunderbird sanity checks.
Vsevolod Stakhov [Tue, 19 May 2015 12:31:40 +0000 (13:31 +0100)]
Improve thunderbird sanity checks.

Submitted by: corvax

9 years agoFix fuzzy mime strings with only type.
Vsevolod Stakhov [Tue, 19 May 2015 10:23:21 +0000 (11:23 +0100)]
Fix fuzzy mime strings with only type.

Issue: #283
Reported by: @moisseev

9 years agoUpdate changelog 284/head
Mikhail Gusarov [Mon, 18 May 2015 16:57:10 +0000 (16:57 +0000)]
Update changelog

9 years agoUse lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc)
Mikhail Gusarov [Mon, 18 May 2015 16:35:05 +0000 (16:35 +0000)]
Use lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc)

9 years agoRequire fixed version of libhiredis to drop workaround
Mikhail Gusarov [Mon, 18 May 2015 16:32:25 +0000 (16:32 +0000)]
Require fixed version of libhiredis to drop workaround

9 years agoUpdate changelog
Mikhail Gusarov [Sun, 17 May 2015 15:46:43 +0000 (15:46 +0000)]
Update changelog

9 years agoImplement IO in lua tcp.
Vsevolod Stakhov [Mon, 18 May 2015 15:54:55 +0000 (16:54 +0100)]
Implement IO in lua tcp.

9 years agoAdd helper to use GString in pool destructors.
Vsevolod Stakhov [Mon, 18 May 2015 15:54:29 +0000 (16:54 +0100)]
Add helper to use GString in pool destructors.

9 years agoAdd preliminary version of generic LUA TCP requests.
Vsevolod Stakhov [Mon, 18 May 2015 15:09:03 +0000 (16:09 +0100)]
Add preliminary version of generic LUA TCP requests.

9 years agoFix error on spawning unique workers.
Vsevolod Stakhov [Mon, 18 May 2015 12:22:18 +0000 (13:22 +0100)]
Fix error on spawning unique workers.

Issue: #282
Reported by: @moisseev

9 years agoRelease 0.9.1. 0.9.1
Vsevolod Stakhov [Sun, 17 May 2015 15:24:08 +0000 (16:24 +0100)]
Release 0.9.1.

* Restore utf8 validation for regular expressions to avoid crashes
* Fix symbols displaying in the interface
* Add symbol groups to the interface
* Fix maps ID parsing in the controller
* Add multimap and regexp modules documentation
* Backport fixes from libucl
* Fix debian package (by @dottedmag)
* Rework XXH32 invocations

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years agoAdopt xxhash invocation.
Vsevolod Stakhov [Sun, 17 May 2015 15:11:12 +0000 (16:11 +0100)]
Adopt xxhash invocation.

9 years agoDo not free xxh32_state to work with static states.
Vsevolod Stakhov [Sun, 17 May 2015 15:10:47 +0000 (16:10 +0100)]
Do not free xxh32_state to work with static states.

9 years agoBackport fixes from libucl.
Vsevolod Stakhov [Sun, 17 May 2015 15:10:29 +0000 (16:10 +0100)]
Backport fixes from libucl.

9 years agoMerge pull request #281 from dottedmag/master
Vsevolod Stakhov [Sun, 17 May 2015 14:36:02 +0000 (15:36 +0100)]
Merge pull request #281 from dottedmag/master

Debian changes for 0.9.x

9 years agoUpdate changelog 281/head
Mikhail Gusarov [Sat, 16 May 2015 18:56:53 +0000 (18:56 +0000)]
Update changelog

9 years agoUpdate debian/copyright
Mikhail Gusarov [Sat, 16 May 2015 23:44:10 +0000 (23:44 +0000)]
Update debian/copyright

9 years agoRestart rspamd on upgrade, stop on removal
Mikhail Gusarov [Sat, 16 May 2015 19:28:57 +0000 (19:28 +0000)]
Restart rspamd on upgrade, stop on removal

9 years agoRestore utf8 validation as it causes segfaults sometimes.
Vsevolod Stakhov [Fri, 15 May 2015 15:36:15 +0000 (16:36 +0100)]
Restore utf8 validation as it causes segfaults sometimes.

9 years agoSync from interface.
Vsevolod Stakhov [Fri, 15 May 2015 14:21:04 +0000 (15:21 +0100)]
Sync from interface.

9 years agoFix symbols iteration.
Vsevolod Stakhov [Fri, 15 May 2015 14:20:34 +0000 (15:20 +0100)]
Fix symbols iteration.

9 years agoAnother sync.
Vsevolod Stakhov [Fri, 15 May 2015 14:04:03 +0000 (15:04 +0100)]
Another sync.

9 years agoSync with interface.
Vsevolod Stakhov [Fri, 15 May 2015 14:00:15 +0000 (15:00 +0100)]
Sync with interface.

9 years agoFix symbols output in the webui.
Vsevolod Stakhov [Fri, 15 May 2015 13:56:31 +0000 (14:56 +0100)]
Fix symbols output in the webui.

9 years agoFix misprint.
Vsevolod Stakhov [Fri, 15 May 2015 12:38:36 +0000 (13:38 +0100)]
Fix misprint.

9 years agoFix getting map ID.
Vsevolod Stakhov [Fri, 15 May 2015 12:24:54 +0000 (13:24 +0100)]
Fix getting map ID.

9 years agoAdd preliminary regexp module documentation.
Vsevolod Stakhov [Thu, 14 May 2015 18:08:51 +0000 (19:08 +0100)]
Add preliminary regexp module documentation.

9 years agoAdd multimap documentation.
Vsevolod Stakhov [Thu, 14 May 2015 17:33:07 +0000 (18:33 +0100)]
Add multimap documentation.

9 years agoRelease 0.9.0 0.9.0
Vsevolod Stakhov [Wed, 13 May 2015 21:04:47 +0000 (22:04 +0100)]
Release 0.9.0

* Add support of the fast and secure protocol level encryption:
- curve25519 is used for key exchange;
- chacha20/poly1305 cryptobox construction for bulk encryption;
- zero latency overhead;
- encrypting and balancing HTTP proxy worker
* Rework expressions and create new expressions library:
- aggressive optimizations based on the abstract syntax tree;
- abstract expressions support (regular expressions, functions, lua modules
composites and so on)
- New comparision and '+' operators support
- New greedy algorithm to minimize execution time of expressions and
all symbols
- Dynamic expressions benchmark and reoptimizations
* Many improvements to the LUA API:
- reworked logger module allowing to do pretty print of the most of lua
types (including tables and userdata classes)
- reworked lua redis and lua HTTP to support more features
- added opaque type for passing large text chunks without copying
- new regexp module with many auxiliary functions (e.g. `re:split`)
* LuaJIT is now the default requirement for rspamd allowing to speed up lua
execution by a large margin (however, plain lua is still supported)
* New plugins:
- spamassassin rules plugin that allows to load and re-use the most of
SA rules natively
- DMARC plugin that evaluates SPF and DKIM policies to the domain policies
- many old plugins has been reworked to implement new features and improve
stability
* New aho-corasic trie implementation from @mischasan that allows to load and
use hundreds of thousands of patterns with no influence on load
* Support of PCRE JIT and PCRE JIT fast path modes that significantly improves
the performance of regular expressions if supported by PCRE
* New URLs parser and extractor:
- removed legacy code that was useless for url finding
- reworked algorithms of URL parsing for more precise and accurate results
- added top-level-domains tree from http://publicsuffix.org
- improved emails parsing
- removed many phishing false positives due to TLD tree check
* New statistics infrastructure:
- created a separate layer of statistic library
- improved OSB-Bayes by re-weighting tokens according to the original
academic paper and `crm114` implementation, which reduced false positives
rate significantly
- created learn cache to avoid double learning of statistics and providing
an efficient way to re-learn class for a message
- created abstract layers for different statistics backends
- implemented new tokenization algorithms with fast or secure (siphash)
hashes to generate statistics features
* Reworked utf8 tokenization that previously corrupted all UTF8 words (minor
incompatibility with old fuzzy hashes with utf-8 symbols)
* SPF module has been completely rewritten to support complex cases of
`include` and `redirect` within SPF records
* DKIM module now supports multiple signatures
* Controller passwords can now be stored encrypted by `PBKDF2-HMAC` in the
configuration file
* Many hand-written HTTP clients has been replaced with the common rspamd
http module
* New test framework:
- import lua `telescope` test framework
- add unit tests for many rspamd modules and routines
- create a unit test for each possible bug found
- use luajit ffi for testing C code
- added preliminary support of functional testing by creating tasks from lua
* Randomize hash seed to avoid certain hash tables vulnerabilities
* Documentation improvements:
- added documentation for the vast majority of rspamd modules
- added documentation for rspamd protocol
- added documentation for the most of rspamd LUA extensions
* Fixed tonns of bugs and memory leaks
* Added tonns of minor features

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years agoBackport changelog entries from 0.8 branch.
Vsevolod Stakhov [Wed, 13 May 2015 20:18:22 +0000 (21:18 +0100)]
Backport changelog entries from 0.8 branch.

9 years agoMerge pull request #279 from fatalbanana/master
Vsevolod Stakhov [Wed, 13 May 2015 20:49:50 +0000 (21:49 +0100)]
Merge pull request #279 from fatalbanana/master

Update RPM BuildRequires for 0.9

9 years agoUpdate RPM BuildRequires for 0.9 279/head
Andrew Lewis [Wed, 13 May 2015 20:37:58 +0000 (22:37 +0200)]
Update RPM BuildRequires for 0.9

9 years agoFix spec for 0.9.0
Vsevolod Stakhov [Wed, 13 May 2015 20:15:02 +0000 (21:15 +0100)]
Fix spec for 0.9.0

9 years agoFix regexp splitting in lua.
Vsevolod Stakhov [Wed, 13 May 2015 16:41:50 +0000 (17:41 +0100)]
Fix regexp splitting in lua.

9 years agoPlug another memory leak.
Vsevolod Stakhov [Wed, 13 May 2015 15:41:18 +0000 (16:41 +0100)]
Plug another memory leak.

9 years agoMore workarounds for hiredis.
Vsevolod Stakhov [Wed, 13 May 2015 15:19:58 +0000 (16:19 +0100)]
More workarounds for hiredis.

9 years agoBreak deadlock in redis timeout logic.
Vsevolod Stakhov [Wed, 13 May 2015 14:30:31 +0000 (15:30 +0100)]
Break deadlock in redis timeout logic.