]> source.dussan.org Git - rspamd.git/log
rspamd.git
9 years agoReverse list of exceptions.
Vsevolod Stakhov [Thu, 21 May 2015 09:23:26 +0000 (10:23 +0100)]
Reverse list of exceptions.

9 years agoAdd tests for tokenizer bug.
Vsevolod Stakhov [Wed, 20 May 2015 21:53:09 +0000 (22:53 +0100)]
Add tests for tokenizer bug.

9 years agoFix critical bug in tokenization logic.
Vsevolod Stakhov [Wed, 20 May 2015 21:52:46 +0000 (22:52 +0100)]
Fix critical bug in tokenization logic.

9 years agoAdd a simple unit test for tokenizer.
Vsevolod Stakhov [Wed, 20 May 2015 21:37:33 +0000 (22:37 +0100)]
Add a simple unit test for tokenizer.

9 years agoAdd lua bindings to tokenizer.
Vsevolod Stakhov [Wed, 20 May 2015 21:17:45 +0000 (22:17 +0100)]
Add lua bindings to tokenizer.

9 years agoUse caseless hash and equal functions for request headers.
Vsevolod Stakhov [Wed, 20 May 2015 16:34:15 +0000 (17:34 +0100)]
Use caseless hash and equal functions for request headers.

9 years agoAdd API to get/set request headers from lua.
Vsevolod Stakhov [Wed, 20 May 2015 16:28:36 +0000 (17:28 +0100)]
Add API to get/set request headers from lua.

9 years agoSave all HTTP headers as request headers.
Vsevolod Stakhov [Wed, 20 May 2015 15:30:56 +0000 (16:30 +0100)]
Save all HTTP headers as request headers.

9 years agoAdd unit test for base64 encoding.
Vsevolod Stakhov [Wed, 20 May 2015 13:59:24 +0000 (14:59 +0100)]
Add unit test for base64 encoding.

9 years agoFix issues in base64 encoding.
Vsevolod Stakhov [Wed, 20 May 2015 13:59:10 +0000 (14:59 +0100)]
Fix issues in base64 encoding.

9 years agoAdd lua bindings for base64.
Vsevolod Stakhov [Wed, 20 May 2015 11:40:19 +0000 (12:40 +0100)]
Add lua bindings for base64.

9 years agoAllow owning lua_text.
Vsevolod Stakhov [Wed, 20 May 2015 11:03:46 +0000 (12:03 +0100)]
Allow owning lua_text.

9 years agoAdd base64 encoding routine.
Vsevolod Stakhov [Wed, 20 May 2015 10:49:51 +0000 (11:49 +0100)]
Add base64 encoding routine.

9 years agoAdd documentation for lua_tcp.
Vsevolod Stakhov [Tue, 19 May 2015 17:38:09 +0000 (18:38 +0100)]
Add documentation for lua_tcp.

9 years agoRelease 0.9.3 0.9.3
Vsevolod Stakhov [Tue, 19 May 2015 16:20:19 +0000 (17:20 +0100)]
Release 0.9.3

* Revert incorrect regexp change that broke the default rules
* Fix lua_tcp module

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years agoFix issues with lua_tcp.
Vsevolod Stakhov [Tue, 19 May 2015 16:16:31 +0000 (17:16 +0100)]
Fix issues with lua_tcp.

9 years agoRevert automatic capture in regexps as it breaks things.
Vsevolod Stakhov [Tue, 19 May 2015 15:56:27 +0000 (16:56 +0100)]
Revert automatic capture in regexps as it breaks things.

9 years agoRelease 0.9.2 0.9.2
Vsevolod Stakhov [Tue, 19 May 2015 15:34:21 +0000 (16:34 +0100)]
Release 0.9.2

* Fix error on spawning unique workers.
* Add preliminary version of generic LUA TCP requests API.
* Use lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc)
* Fix fuzzy mime strings with only type.
* Improve thunderbird sanity checks.
* Fix critical bug on matching regular expressions.
* Make hiredis optional dependency.
* Fix multiple bugs in daemon reloading

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years agoFix reload.
Vsevolod Stakhov [Tue, 19 May 2015 15:29:03 +0000 (16:29 +0100)]
Fix reload.

9 years agoFix reloading in logger.
Vsevolod Stakhov [Tue, 19 May 2015 15:28:56 +0000 (16:28 +0100)]
Fix reloading in logger.

9 years agoOne more try to fix memset.
Vsevolod Stakhov [Tue, 19 May 2015 15:28:42 +0000 (16:28 +0100)]
One more try to fix memset.

9 years agoFix reloading of C plugins.
Vsevolod Stakhov [Tue, 19 May 2015 14:37:31 +0000 (15:37 +0100)]
Fix reloading of C plugins.

9 years agoFix reloading of modules.
Vsevolod Stakhov [Tue, 19 May 2015 14:37:20 +0000 (15:37 +0100)]
Fix reloading of modules.

9 years agoDo not try dereference NULL pointer.
Vsevolod Stakhov [Tue, 19 May 2015 14:37:02 +0000 (15:37 +0100)]
Do not try dereference NULL pointer.

9 years agoMerge pull request #284 from dottedmag/master
Vsevolod Stakhov [Tue, 19 May 2015 13:34:18 +0000 (14:34 +0100)]
Merge pull request #284 from dottedmag/master

Updates for 0.9.2 Debian release

9 years agoAdd method to detect the full size of message.
Vsevolod Stakhov [Tue, 19 May 2015 12:58:07 +0000 (13:58 +0100)]
Add method to detect the full size of message.

9 years agoFallback to plain lua if luajit has not been found.
Vsevolod Stakhov [Tue, 19 May 2015 12:51:50 +0000 (13:51 +0100)]
Fallback to plain lua if luajit has not been found.

9 years agoMake hiredis optional dependency.
Vsevolod Stakhov [Tue, 19 May 2015 12:43:26 +0000 (13:43 +0100)]
Make hiredis optional dependency.

9 years agoFix critical bug on matching regular expressions.
Vsevolod Stakhov [Tue, 19 May 2015 12:32:09 +0000 (13:32 +0100)]
Fix critical bug on matching regular expressions.

Issue: #186
Reported by: @moisseev

9 years agoImprove thunderbird sanity checks.
Vsevolod Stakhov [Tue, 19 May 2015 12:31:40 +0000 (13:31 +0100)]
Improve thunderbird sanity checks.

Submitted by: corvax

9 years agoFix fuzzy mime strings with only type.
Vsevolod Stakhov [Tue, 19 May 2015 10:23:21 +0000 (11:23 +0100)]
Fix fuzzy mime strings with only type.

Issue: #283
Reported by: @moisseev

9 years agoUpdate changelog 284/head
Mikhail Gusarov [Mon, 18 May 2015 16:57:10 +0000 (16:57 +0000)]
Update changelog

9 years agoUse lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc)
Mikhail Gusarov [Mon, 18 May 2015 16:35:05 +0000 (16:35 +0000)]
Use lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc)

9 years agoRequire fixed version of libhiredis to drop workaround
Mikhail Gusarov [Mon, 18 May 2015 16:32:25 +0000 (16:32 +0000)]
Require fixed version of libhiredis to drop workaround

9 years agoUpdate changelog
Mikhail Gusarov [Sun, 17 May 2015 15:46:43 +0000 (15:46 +0000)]
Update changelog

9 years agoImplement IO in lua tcp.
Vsevolod Stakhov [Mon, 18 May 2015 15:54:55 +0000 (16:54 +0100)]
Implement IO in lua tcp.

9 years agoAdd helper to use GString in pool destructors.
Vsevolod Stakhov [Mon, 18 May 2015 15:54:29 +0000 (16:54 +0100)]
Add helper to use GString in pool destructors.

9 years agoAdd preliminary version of generic LUA TCP requests.
Vsevolod Stakhov [Mon, 18 May 2015 15:09:03 +0000 (16:09 +0100)]
Add preliminary version of generic LUA TCP requests.

9 years agoFix error on spawning unique workers.
Vsevolod Stakhov [Mon, 18 May 2015 12:22:18 +0000 (13:22 +0100)]
Fix error on spawning unique workers.

Issue: #282
Reported by: @moisseev

9 years agoRelease 0.9.1. 0.9.1
Vsevolod Stakhov [Sun, 17 May 2015 15:24:08 +0000 (16:24 +0100)]
Release 0.9.1.

* Restore utf8 validation for regular expressions to avoid crashes
* Fix symbols displaying in the interface
* Add symbol groups to the interface
* Fix maps ID parsing in the controller
* Add multimap and regexp modules documentation
* Backport fixes from libucl
* Fix debian package (by @dottedmag)
* Rework XXH32 invocations

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years agoAdopt xxhash invocation.
Vsevolod Stakhov [Sun, 17 May 2015 15:11:12 +0000 (16:11 +0100)]
Adopt xxhash invocation.

9 years agoDo not free xxh32_state to work with static states.
Vsevolod Stakhov [Sun, 17 May 2015 15:10:47 +0000 (16:10 +0100)]
Do not free xxh32_state to work with static states.

9 years agoBackport fixes from libucl.
Vsevolod Stakhov [Sun, 17 May 2015 15:10:29 +0000 (16:10 +0100)]
Backport fixes from libucl.

9 years agoMerge pull request #281 from dottedmag/master
Vsevolod Stakhov [Sun, 17 May 2015 14:36:02 +0000 (15:36 +0100)]
Merge pull request #281 from dottedmag/master

Debian changes for 0.9.x

9 years agoUpdate changelog 281/head
Mikhail Gusarov [Sat, 16 May 2015 18:56:53 +0000 (18:56 +0000)]
Update changelog

9 years agoUpdate debian/copyright
Mikhail Gusarov [Sat, 16 May 2015 23:44:10 +0000 (23:44 +0000)]
Update debian/copyright

9 years agoRestart rspamd on upgrade, stop on removal
Mikhail Gusarov [Sat, 16 May 2015 19:28:57 +0000 (19:28 +0000)]
Restart rspamd on upgrade, stop on removal

9 years agoRestore utf8 validation as it causes segfaults sometimes.
Vsevolod Stakhov [Fri, 15 May 2015 15:36:15 +0000 (16:36 +0100)]
Restore utf8 validation as it causes segfaults sometimes.

9 years agoSync from interface.
Vsevolod Stakhov [Fri, 15 May 2015 14:21:04 +0000 (15:21 +0100)]
Sync from interface.

9 years agoFix symbols iteration.
Vsevolod Stakhov [Fri, 15 May 2015 14:20:34 +0000 (15:20 +0100)]
Fix symbols iteration.

9 years agoAnother sync.
Vsevolod Stakhov [Fri, 15 May 2015 14:04:03 +0000 (15:04 +0100)]
Another sync.

9 years agoSync with interface.
Vsevolod Stakhov [Fri, 15 May 2015 14:00:15 +0000 (15:00 +0100)]
Sync with interface.

9 years agoFix symbols output in the webui.
Vsevolod Stakhov [Fri, 15 May 2015 13:56:31 +0000 (14:56 +0100)]
Fix symbols output in the webui.

9 years agoFix misprint.
Vsevolod Stakhov [Fri, 15 May 2015 12:38:36 +0000 (13:38 +0100)]
Fix misprint.

9 years agoFix getting map ID.
Vsevolod Stakhov [Fri, 15 May 2015 12:24:54 +0000 (13:24 +0100)]
Fix getting map ID.

9 years agoAdd preliminary regexp module documentation.
Vsevolod Stakhov [Thu, 14 May 2015 18:08:51 +0000 (19:08 +0100)]
Add preliminary regexp module documentation.

9 years agoAdd multimap documentation.
Vsevolod Stakhov [Thu, 14 May 2015 17:33:07 +0000 (18:33 +0100)]
Add multimap documentation.

9 years agoRelease 0.9.0 0.9.0
Vsevolod Stakhov [Wed, 13 May 2015 21:04:47 +0000 (22:04 +0100)]
Release 0.9.0

* Add support of the fast and secure protocol level encryption:
- curve25519 is used for key exchange;
- chacha20/poly1305 cryptobox construction for bulk encryption;
- zero latency overhead;
- encrypting and balancing HTTP proxy worker
* Rework expressions and create new expressions library:
- aggressive optimizations based on the abstract syntax tree;
- abstract expressions support (regular expressions, functions, lua modules
composites and so on)
- New comparision and '+' operators support
- New greedy algorithm to minimize execution time of expressions and
all symbols
- Dynamic expressions benchmark and reoptimizations
* Many improvements to the LUA API:
- reworked logger module allowing to do pretty print of the most of lua
types (including tables and userdata classes)
- reworked lua redis and lua HTTP to support more features
- added opaque type for passing large text chunks without copying
- new regexp module with many auxiliary functions (e.g. `re:split`)
* LuaJIT is now the default requirement for rspamd allowing to speed up lua
execution by a large margin (however, plain lua is still supported)
* New plugins:
- spamassassin rules plugin that allows to load and re-use the most of
SA rules natively
- DMARC plugin that evaluates SPF and DKIM policies to the domain policies
- many old plugins has been reworked to implement new features and improve
stability
* New aho-corasic trie implementation from @mischasan that allows to load and
use hundreds of thousands of patterns with no influence on load
* Support of PCRE JIT and PCRE JIT fast path modes that significantly improves
the performance of regular expressions if supported by PCRE
* New URLs parser and extractor:
- removed legacy code that was useless for url finding
- reworked algorithms of URL parsing for more precise and accurate results
- added top-level-domains tree from http://publicsuffix.org
- improved emails parsing
- removed many phishing false positives due to TLD tree check
* New statistics infrastructure:
- created a separate layer of statistic library
- improved OSB-Bayes by re-weighting tokens according to the original
academic paper and `crm114` implementation, which reduced false positives
rate significantly
- created learn cache to avoid double learning of statistics and providing
an efficient way to re-learn class for a message
- created abstract layers for different statistics backends
- implemented new tokenization algorithms with fast or secure (siphash)
hashes to generate statistics features
* Reworked utf8 tokenization that previously corrupted all UTF8 words (minor
incompatibility with old fuzzy hashes with utf-8 symbols)
* SPF module has been completely rewritten to support complex cases of
`include` and `redirect` within SPF records
* DKIM module now supports multiple signatures
* Controller passwords can now be stored encrypted by `PBKDF2-HMAC` in the
configuration file
* Many hand-written HTTP clients has been replaced with the common rspamd
http module
* New test framework:
- import lua `telescope` test framework
- add unit tests for many rspamd modules and routines
- create a unit test for each possible bug found
- use luajit ffi for testing C code
- added preliminary support of functional testing by creating tasks from lua
* Randomize hash seed to avoid certain hash tables vulnerabilities
* Documentation improvements:
- added documentation for the vast majority of rspamd modules
- added documentation for rspamd protocol
- added documentation for the most of rspamd LUA extensions
* Fixed tonns of bugs and memory leaks
* Added tonns of minor features

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years agoBackport changelog entries from 0.8 branch.
Vsevolod Stakhov [Wed, 13 May 2015 20:18:22 +0000 (21:18 +0100)]
Backport changelog entries from 0.8 branch.

9 years agoMerge pull request #279 from fatalbanana/master
Vsevolod Stakhov [Wed, 13 May 2015 20:49:50 +0000 (21:49 +0100)]
Merge pull request #279 from fatalbanana/master

Update RPM BuildRequires for 0.9

9 years agoUpdate RPM BuildRequires for 0.9 279/head
Andrew Lewis [Wed, 13 May 2015 20:37:58 +0000 (22:37 +0200)]
Update RPM BuildRequires for 0.9

9 years agoFix spec for 0.9.0
Vsevolod Stakhov [Wed, 13 May 2015 20:15:02 +0000 (21:15 +0100)]
Fix spec for 0.9.0

9 years agoFix regexp splitting in lua.
Vsevolod Stakhov [Wed, 13 May 2015 16:41:50 +0000 (17:41 +0100)]
Fix regexp splitting in lua.

9 years agoPlug another memory leak.
Vsevolod Stakhov [Wed, 13 May 2015 15:41:18 +0000 (16:41 +0100)]
Plug another memory leak.

9 years agoMore workarounds for hiredis.
Vsevolod Stakhov [Wed, 13 May 2015 15:19:58 +0000 (16:19 +0100)]
More workarounds for hiredis.

9 years agoBreak deadlock in redis timeout logic.
Vsevolod Stakhov [Wed, 13 May 2015 14:30:31 +0000 (15:30 +0100)]
Break deadlock in redis timeout logic.

9 years agoDo not add invalid URLs from HTML tags.
Vsevolod Stakhov [Wed, 13 May 2015 14:10:52 +0000 (15:10 +0100)]
Do not add invalid URLs from HTML tags.

9 years agoFix method name in the documentation.
Vsevolod Stakhov [Wed, 13 May 2015 13:58:29 +0000 (14:58 +0100)]
Fix method name in the documentation.

9 years agoFix memory leak in DKIM.
Vsevolod Stakhov [Wed, 13 May 2015 13:52:18 +0000 (14:52 +0100)]
Fix memory leak in DKIM.

9 years agoOne more memory leak in lua_redis.
Vsevolod Stakhov [Wed, 13 May 2015 13:41:06 +0000 (14:41 +0100)]
One more memory leak in lua_redis.

9 years agoDo not add IP maps twice.
Vsevolod Stakhov [Wed, 13 May 2015 13:09:23 +0000 (14:09 +0100)]
Do not add IP maps twice.

9 years agoOutput some statistics about maps.
Vsevolod Stakhov [Wed, 13 May 2015 13:04:31 +0000 (14:04 +0100)]
Output some statistics about maps.

9 years agoFix critical bug with reading file maps.
Vsevolod Stakhov [Wed, 13 May 2015 13:04:11 +0000 (14:04 +0100)]
Fix critical bug with reading file maps.

9 years agoAdd routine to check radix trie size.
Vsevolod Stakhov [Wed, 13 May 2015 13:03:46 +0000 (14:03 +0100)]
Add routine to check radix trie size.

9 years agoPrint userdata in format 'class(address)'
Vsevolod Stakhov [Wed, 13 May 2015 12:44:55 +0000 (13:44 +0100)]
Print userdata in format 'class(address)'

9 years agoFix JITless world.
Vsevolod Stakhov [Wed, 13 May 2015 12:32:22 +0000 (13:32 +0100)]
Fix JITless world.

9 years agoFix another issues with raw/utf regexps.
Vsevolod Stakhov [Wed, 13 May 2015 12:00:52 +0000 (13:00 +0100)]
Fix another issues with raw/utf regexps.

9 years agoAllow to push raw pointer from the text boxed type.
Vsevolod Stakhov [Wed, 13 May 2015 10:22:03 +0000 (11:22 +0100)]
Allow to push raw pointer from the text boxed type.

9 years agoFix symbols removal in composites.
Vsevolod Stakhov [Wed, 13 May 2015 09:58:02 +0000 (10:58 +0100)]
Fix symbols removal in composites.

9 years agoFix forged recipients plugin for the modern rspamd.
Vsevolod Stakhov [Wed, 13 May 2015 09:57:14 +0000 (10:57 +0100)]
Fix forged recipients plugin for the modern rspamd.

9 years agoTry to load all lua modules ignoring failed ones.
Vsevolod Stakhov [Wed, 13 May 2015 09:53:28 +0000 (10:53 +0100)]
Try to load all lua modules ignoring failed ones.

9 years agoWhitelisted ip map should be radix, not hash.
Vsevolod Stakhov [Tue, 12 May 2015 18:19:54 +0000 (19:19 +0100)]
Whitelisted ip map should be radix, not hash.

9 years agoAdd documentation fro ratelimit module.
Vsevolod Stakhov [Tue, 12 May 2015 18:17:50 +0000 (19:17 +0100)]
Add documentation fro ratelimit module.

9 years agoRework composite symbols removal.
Vsevolod Stakhov [Tue, 12 May 2015 17:47:00 +0000 (18:47 +0100)]
Rework composite symbols removal.

Reported by: @fatalbanana

9 years agoSkip unparseable DKIM signatures.
Vsevolod Stakhov [Tue, 12 May 2015 14:40:14 +0000 (15:40 +0100)]
Skip unparseable DKIM signatures.

9 years agoFix zero-copy mode in lua_http (double free).
Vsevolod Stakhov [Tue, 12 May 2015 14:26:42 +0000 (15:26 +0100)]
Fix zero-copy mode in lua_http (double free).

9 years agoFix finding headers in HTTP messages.
Vsevolod Stakhov [Tue, 12 May 2015 14:11:42 +0000 (15:11 +0100)]
Fix finding headers in HTTP messages.

9 years agoFix coredump in a client (double free).
Vsevolod Stakhov [Tue, 12 May 2015 13:55:31 +0000 (14:55 +0100)]
Fix coredump in a client (double free).

9 years agoMore fixes to luajit detection.
Vsevolod Stakhov [Tue, 12 May 2015 12:24:03 +0000 (13:24 +0100)]
More fixes to luajit detection.

9 years agoFix converting of old storage.
Vsevolod Stakhov [Tue, 12 May 2015 11:51:36 +0000 (12:51 +0100)]
Fix converting of old storage.

9 years agoUse raw regexp instead of utf ones by default.
Vsevolod Stakhov [Tue, 12 May 2015 11:46:10 +0000 (12:46 +0100)]
Use raw regexp instead of utf ones by default.

Now to enable utf8 match, one should specify 'u' modifier, for example,
by adding '/u' to the slashed patterm.

9 years agoFix finding of luajit library.
Vsevolod Stakhov [Tue, 12 May 2015 11:10:42 +0000 (12:10 +0100)]
Fix finding of luajit library.

9 years agoRemove old radix code (no functional changes).
Vsevolod Stakhov [Tue, 12 May 2015 08:28:25 +0000 (09:28 +0100)]
Remove old radix code (no functional changes).

9 years agoAdd key rotation for http proxy.
Vsevolod Stakhov [Tue, 12 May 2015 08:16:21 +0000 (09:16 +0100)]
Add key rotation for http proxy.

9 years agoLibutil is useless everywhere but FreeBSD.
Vsevolod Stakhov [Mon, 11 May 2015 16:08:58 +0000 (17:08 +0100)]
Libutil is useless everywhere but FreeBSD.

9 years agoTry to fix libutil...
Vsevolod Stakhov [Mon, 11 May 2015 14:38:25 +0000 (15:38 +0100)]
Try to fix libutil...

9 years agoAnother workaround for pcre jit.
Vsevolod Stakhov [Mon, 11 May 2015 14:10:56 +0000 (15:10 +0100)]
Another workaround for pcre jit.

9 years agoAnother try to fixup the mess.
Vsevolod Stakhov [Mon, 11 May 2015 13:53:48 +0000 (14:53 +0100)]
Another try to fixup the mess.

9 years agoAnother iteration of libraries detection rework.
Vsevolod Stakhov [Mon, 11 May 2015 13:17:15 +0000 (14:17 +0100)]
Another iteration of libraries detection rework.

9 years agoAdd workaround for glib < 2.30.
Vsevolod Stakhov [Mon, 11 May 2015 13:17:04 +0000 (14:17 +0100)]
Add workaround for glib < 2.30.