]> source.dussan.org Git - rspamd.git/log
rspamd.git
9 years agoChoose raw regexp for raw headers.
Vsevolod Stakhov [Wed, 12 Nov 2014 18:18:00 +0000 (18:18 +0000)]
Choose raw regexp for raw headers.

9 years agoRework calling of lua functions from regexp module.
Vsevolod Stakhov [Wed, 12 Nov 2014 16:32:58 +0000 (16:32 +0000)]
Rework calling of lua functions from regexp module.

9 years agoSync libucl.
Vsevolod Stakhov [Wed, 12 Nov 2014 16:24:27 +0000 (16:24 +0000)]
Sync libucl.

9 years agoBe more informative about lua errors.
Vsevolod Stakhov [Tue, 11 Nov 2014 17:36:03 +0000 (17:36 +0000)]
Be more informative about lua errors.

9 years agoFix timeouts in redirector.
Vsevolod Stakhov [Tue, 11 Nov 2014 17:27:14 +0000 (17:27 +0000)]
Fix timeouts in redirector.

9 years agoFix reload issues in surbl and fuzzy_check.
Vsevolod Stakhov [Tue, 11 Nov 2014 17:13:37 +0000 (17:13 +0000)]
Fix reload issues in surbl and fuzzy_check.

9 years agoSet AF properly in one more place.
Vsevolod Stakhov [Tue, 11 Nov 2014 17:13:25 +0000 (17:13 +0000)]
Set AF properly in one more place.

9 years agoSet sockaddr.sa_family properly.
Vsevolod Stakhov [Tue, 11 Nov 2014 16:15:04 +0000 (16:15 +0000)]
Set sockaddr.sa_family properly.

9 years agoRemove unused variable.
Vsevolod Stakhov [Tue, 11 Nov 2014 15:59:02 +0000 (15:59 +0000)]
Remove unused variable.

9 years agoStop processing headers in parts.
Vsevolod Stakhov [Tue, 11 Nov 2014 15:58:03 +0000 (15:58 +0000)]
Stop processing headers in parts.

9 years agoFix crash with unweighted upstreams.
Vsevolod Stakhov [Tue, 11 Nov 2014 15:54:18 +0000 (15:54 +0000)]
Fix crash with unweighted upstreams.

9 years agoDecode URL obtained from HTML tags.
Vsevolod Stakhov [Tue, 11 Nov 2014 13:01:16 +0000 (13:01 +0000)]
Decode URL obtained from HTML tags.

9 years agoFix IP validity checks.
Vsevolod Stakhov [Mon, 10 Nov 2014 23:21:14 +0000 (23:21 +0000)]
Fix IP validity checks.

9 years agoMerge pull request #114 from sfirmery/master
Vsevolod Stakhov [Mon, 10 Nov 2014 22:01:35 +0000 (22:01 +0000)]
Merge pull request #114 from sfirmery/master

Fix owner when creating folder /run/rspamd in Debian init script

9 years agoFix owner when creating folder /run/rspamd 114/head
Sylvain Firmery [Sat, 8 Nov 2014 19:05:54 +0000 (20:05 +0100)]
Fix owner when creating folder /run/rspamd

9 years agoRelease 0.7.4 0.7.4
Vsevolod Stakhov [Sat, 8 Nov 2014 19:01:31 +0000 (19:01 +0000)]
Release 0.7.4

* Fix build under *BSD
* Detect HAN unicode script
* Implement language detection heuristic for text parts
* Fix time output in history
* Improve piechart coloring
* Fix \r\n conversion in DKIM module (reported by @citrin)
* Try to detect systems with no IPv6 support
* Fix multiple/single values in use settings (reported by @citrin)
*  Rework IP addresses in upstreams:
- Select ipv4/unix addresses if they exist and use ipv6 for ipv6 only
    upstreams (since the support of ipv6 is poor in many OSes and
    environments)
- Free IP list on upstream destruction
- Add test cases for addresses selection
- Allow adding of free form IP addresses to upstreams
* Fix endiannes in lua_radix search (reported by @citrin)
* Soft shutdown should also set wanna_die flag (reported by @citrin)
* Stop use-after-free in event loop termination
* Fix processing of very short messages in DKIM (reported by @citrin)
* Detect systems without shared mutexes
* Fix issues with PTR and MX elements in SPF parser (reported by @citrin)

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years agoImprove coloring.
Vsevolod Stakhov [Sat, 8 Nov 2014 18:31:18 +0000 (18:31 +0000)]
Improve coloring.

9 years agoUpdate versions.
Vsevolod Stakhov [Sat, 8 Nov 2014 16:15:47 +0000 (16:15 +0000)]
Update versions.

9 years agoUpdate version to 0.7.4.
Vsevolod Stakhov [Sat, 8 Nov 2014 15:53:44 +0000 (15:53 +0000)]
Update version to 0.7.4.

9 years agoMerge pull request #113 from citrin/master
Vsevolod Stakhov [Sat, 8 Nov 2014 18:37:53 +0000 (18:37 +0000)]
Merge pull request #113 from citrin/master

Fix typo in symbol description.

9 years agoTry to detect systems without shared mutexes.
Vsevolod Stakhov [Sat, 8 Nov 2014 15:46:03 +0000 (15:46 +0000)]
Try to detect systems without shared mutexes.

9 years agoFix processing of very short messages in DKIM.
Vsevolod Stakhov [Fri, 7 Nov 2014 17:08:41 +0000 (17:08 +0000)]
Fix processing of very short messages in DKIM.

Reported by: @citrin

9 years agoAnother fix for PTR records in SPF.
Vsevolod Stakhov [Fri, 7 Nov 2014 16:00:59 +0000 (16:00 +0000)]
Another fix for PTR records in SPF.

9 years agoStop use-after-free in event loop termination.
Vsevolod Stakhov [Fri, 7 Nov 2014 15:08:09 +0000 (15:08 +0000)]
Stop use-after-free in event loop termination.

9 years agoSoft shutdown should also set wanna_die flag.
Vsevolod Stakhov [Fri, 7 Nov 2014 14:36:10 +0000 (14:36 +0000)]
Soft shutdown should also set wanna_die flag.

9 years agoFix endiannes in lua_radix search.
Vsevolod Stakhov [Fri, 7 Nov 2014 14:22:12 +0000 (14:22 +0000)]
Fix endiannes in lua_radix search.

9 years agoRework IP addresses in upstreams.
Vsevolod Stakhov [Fri, 7 Nov 2014 13:33:45 +0000 (13:33 +0000)]
Rework IP addresses in upstreams.

- Select ipv4/unix addresses if they exist and use ipv6 for ipv6 only
  upstreams (since the support of ipv6 is poor in many OSes and
  environments).
- Free IP list on upstream destruction.
- Add test cases for addresses selection.
- Allow adding of free form IP addresses to upstreams.

9 years agoFix multiple/single values in use settings.
Vsevolod Stakhov [Fri, 7 Nov 2014 11:00:18 +0000 (11:00 +0000)]
Fix multiple/single values in use settings.

Reported by: @citrin

9 years agoTry to detect systems with no IPv6 support.
Vsevolod Stakhov [Thu, 6 Nov 2014 22:54:31 +0000 (22:54 +0000)]
Try to detect systems with no IPv6 support.

9 years agoFix typo in symbol description. 113/head
Anton Yuzhaninov [Wed, 5 Nov 2014 18:32:32 +0000 (21:32 +0300)]
Fix typo in symbol description.

9 years agoDisable extra debug.
Vsevolod Stakhov [Wed, 5 Nov 2014 16:14:23 +0000 (16:14 +0000)]
Disable extra debug.

9 years agoMore fixes to dkim \r\n converting.
Vsevolod Stakhov [Wed, 5 Nov 2014 16:12:03 +0000 (16:12 +0000)]
More fixes to dkim \r\n converting.

Found by: @citrin

9 years agoFix \r\n conversion in DKIM module.
Vsevolod Stakhov [Wed, 5 Nov 2014 15:27:34 +0000 (15:27 +0000)]
Fix \r\n conversion in DKIM module.

9 years agoImprove piechart coloring.
Vsevolod Stakhov [Tue, 4 Nov 2014 13:52:53 +0000 (13:52 +0000)]
Improve piechart coloring.

9 years agoFix time output in history.
Vsevolod Stakhov [Tue, 4 Nov 2014 13:14:41 +0000 (13:14 +0000)]
Fix time output in history.

9 years agoDetect language heuristic for text parts.
Vsevolod Stakhov [Tue, 4 Nov 2014 12:51:55 +0000 (12:51 +0000)]
Detect language heuristic for text parts.

9 years agoDetect HAN unicode script.
Vsevolod Stakhov [Tue, 4 Nov 2014 12:32:49 +0000 (12:32 +0000)]
Detect HAN unicode script.

9 years agoFix stupid error for FreeBSD build.
Vsevolod Stakhov [Mon, 3 Nov 2014 12:47:50 +0000 (12:47 +0000)]
Fix stupid error for FreeBSD build.

9 years agoRelease 0.7.3. 0.7.3
Vsevolod Stakhov [Mon, 3 Nov 2014 12:23:36 +0000 (12:23 +0000)]
Release 0.7.3.

* New upstreams code:
- simplify upstreams API;
- unify strings parsing in upstreams definition;
- add configuration options for the upstreams;
- for failed upstreams re-resolve their addresses;
- use all resolved addresses for an upstream (round-robin);
- implement stable hashing and use it by default for upstreams;
- add unit test for upstreams module.
* Rework signals processing in all rspamd workers:
- signals are now processed in the event loop;
- implement the most common signal handlers for all workers;
- add callbacks for workers specific signal handlers
*  Fix critical issue with fuzzy storage:
Fuzzy stroage could not save any hashes on termination due to bugged
signals handling
* Fix roll history IP storage
* Rework ipv4/ipv6 handling in parsing addresses:
    - turn off support of IPV6_V6ONLY socket option;
    - create ipv6 socket prior to ipv4 one to handle systems with v6/v4
    sockets enabled (Linux)
* Remove CBL as it's wholly included in Spamhaus XBL (by @fatalbanana)
* Remove nszones.com fake RBL (by @citrin)
* Fix upstreams interaction for fuzzy_check
* Verify spf PTR records (reported by @citrin)
* Fix spf MX records parsing
* Add compatibility for old libevent (by @yellowman)
* Sync bugfixes from libucl

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
9 years agoImprove upstream tests.
Vsevolod Stakhov [Mon, 3 Nov 2014 11:41:07 +0000 (11:41 +0000)]
Improve upstream tests.

9 years agoFix tv_to_msec macro priority.
Vsevolod Stakhov [Mon, 3 Nov 2014 10:54:44 +0000 (10:54 +0000)]
Fix tv_to_msec macro priority.

9 years agoConfigure upstreams in workers.
Vsevolod Stakhov [Mon, 3 Nov 2014 10:23:32 +0000 (10:23 +0000)]
Configure upstreams in workers.

9 years agoParse unsigned integers in the configuration.
Vsevolod Stakhov [Mon, 3 Nov 2014 10:23:13 +0000 (10:23 +0000)]
Parse unsigned integers in the configuration.

9 years agoUse upstreams configuration.
Vsevolod Stakhov [Mon, 3 Nov 2014 10:19:13 +0000 (10:19 +0000)]
Use upstreams configuration.

9 years agoAdd upstreams configuration.
Vsevolod Stakhov [Mon, 3 Nov 2014 10:18:55 +0000 (10:18 +0000)]
Add upstreams configuration.

9 years agoAdd consistent hash test.
Vsevolod Stakhov [Sun, 2 Nov 2014 21:42:53 +0000 (21:42 +0000)]
Add consistent hash test.

P value is calculated as following:
when we add/remove M upstreams from the list, the probability of hash
miss should be close to the relation N / (N + M), where N is the size of
the previous upstreams list.

So far, the algorithm described in http://arxiv.org/abs/1406.2294 shows
p value more than 0.99.

9 years agoSome more fixes to the test.
Vsevolod Stakhov [Sun, 2 Nov 2014 19:03:15 +0000 (19:03 +0000)]
Some more fixes to the test.

9 years agoFix some upstream issues.
Vsevolod Stakhov [Sun, 2 Nov 2014 18:55:24 +0000 (18:55 +0000)]
Fix some upstream issues.

9 years agoAdd a simple upstreams test.
Vsevolod Stakhov [Sun, 2 Nov 2014 18:38:43 +0000 (18:38 +0000)]
Add a simple upstreams test.

9 years agoRemove old radix tests.
Vsevolod Stakhov [Sun, 2 Nov 2014 17:39:45 +0000 (17:39 +0000)]
Remove old radix tests.

9 years agoUpdate version to 0.7.3
Vsevolod Stakhov [Sun, 2 Nov 2014 17:19:00 +0000 (17:19 +0000)]
Update version to 0.7.3

9 years agoUse PTHREAD_PROCESS_SHARED mutexes if possible.
Vsevolod Stakhov [Sat, 1 Nov 2014 22:11:15 +0000 (22:11 +0000)]
Use PTHREAD_PROCESS_SHARED mutexes if possible.

9 years agoUse process-level mutex to avoid log races.
Vsevolod Stakhov [Sat, 1 Nov 2014 17:52:15 +0000 (17:52 +0000)]
Use process-level mutex to avoid log races.

9 years agoFix critical issue with fuzzy storage.
Vsevolod Stakhov [Sat, 1 Nov 2014 17:22:55 +0000 (17:22 +0000)]
Fix critical issue with fuzzy storage.

Fuzzy stroage could not save any hashes on termination due to bugged
signals handling.

Reported by: at0r at #rspamd

9 years agoFix signals initializing.
Vsevolod Stakhov [Sat, 1 Nov 2014 17:22:45 +0000 (17:22 +0000)]
Fix signals initializing.

9 years agoAdd post-signal handler.
Vsevolod Stakhov [Sat, 1 Nov 2014 17:05:35 +0000 (17:05 +0000)]
Add post-signal handler.

9 years agoRework signals processing in workers.
Vsevolod Stakhov [Sat, 1 Nov 2014 16:41:38 +0000 (16:41 +0000)]
Rework signals processing in workers.

9 years agoUse constant for hash seed for upstreams selection.
Vsevolod Stakhov [Sat, 1 Nov 2014 15:18:48 +0000 (15:18 +0000)]
Use constant for hash seed for upstreams selection.

9 years agoAdd jitter for upstream reviving.
Vsevolod Stakhov [Sat, 1 Nov 2014 15:18:33 +0000 (15:18 +0000)]
Add jitter for upstream reviving.

9 years agoFix some issues in the upstream code.
Vsevolod Stakhov [Fri, 31 Oct 2014 12:14:32 +0000 (12:14 +0000)]
Fix some issues in the upstream code.

9 years agoFix roll history IP storage.
Vsevolod Stakhov [Fri, 31 Oct 2014 11:46:48 +0000 (11:46 +0000)]
Fix roll history IP storage.

9 years agoFix upstreams in ratelimit.
Vsevolod Stakhov [Thu, 30 Oct 2014 22:29:28 +0000 (22:29 +0000)]
Fix upstreams in ratelimit.

9 years agoFuck ipv6 inventors for ipv6/v4 sockets.
Vsevolod Stakhov [Wed, 29 Oct 2014 15:40:05 +0000 (15:40 +0000)]
Fuck ipv6 inventors for ipv6/v4 sockets.

9 years agoSet length properly.
Vsevolod Stakhov [Wed, 29 Oct 2014 15:39:53 +0000 (15:39 +0000)]
Set length properly.

9 years agoFix build.
Vsevolod Stakhov [Wed, 29 Oct 2014 15:02:51 +0000 (15:02 +0000)]
Fix build.

9 years agoFix fuzzy_check.
Vsevolod Stakhov [Wed, 29 Oct 2014 14:54:28 +0000 (14:54 +0000)]
Fix fuzzy_check.

9 years agoSome more auxiliary functions.
Vsevolod Stakhov [Wed, 29 Oct 2014 14:54:18 +0000 (14:54 +0000)]
Some more auxiliary functions.

9 years agoFix surbl.
Vsevolod Stakhov [Wed, 29 Oct 2014 14:10:57 +0000 (14:10 +0000)]
Fix surbl.

9 years agoFix smtp proxy code.
Vsevolod Stakhov [Wed, 29 Oct 2014 14:02:13 +0000 (14:02 +0000)]
Fix smtp proxy code.

9 years agoAdd parsing of multiple upstreams string.
Vsevolod Stakhov [Wed, 29 Oct 2014 13:58:33 +0000 (13:58 +0000)]
Add parsing of multiple upstreams string.

9 years agoRework and simplify listen sockets creation.
Vsevolod Stakhov [Wed, 29 Oct 2014 12:18:13 +0000 (12:18 +0000)]
Rework and simplify listen sockets creation.

9 years agoAllow addresses to be allocated in rspamd_mempool.
Vsevolod Stakhov [Wed, 29 Oct 2014 11:26:08 +0000 (11:26 +0000)]
Allow addresses to be allocated in rspamd_mempool.

9 years agoFix some more old upstreams code.
Vsevolod Stakhov [Wed, 29 Oct 2014 01:38:22 +0000 (01:38 +0000)]
Fix some more old upstreams code.

9 years agoDisable binlog configuration as it is anyway broken.
Vsevolod Stakhov [Tue, 28 Oct 2014 23:24:53 +0000 (23:24 +0000)]
Disable binlog configuration as it is anyway broken.

9 years agoMerge branch 'upstream-rework'
Vsevolod Stakhov [Tue, 28 Oct 2014 17:27:59 +0000 (17:27 +0000)]
Merge branch 'upstream-rework'

9 years agoFix lua upstream code.
Vsevolod Stakhov [Tue, 28 Oct 2014 17:27:06 +0000 (17:27 +0000)]
Fix lua upstream code.

9 years agoUpstreams get implementation.
Vsevolod Stakhov [Tue, 28 Oct 2014 17:04:43 +0000 (17:04 +0000)]
Upstreams get implementation.

9 years agoAllow multiple addresses per upstream.
Vsevolod Stakhov [Tue, 28 Oct 2014 16:04:33 +0000 (16:04 +0000)]
Allow multiple addresses per upstream.

9 years agoAdd some basic logic for upstreams.
Vsevolod Stakhov [Tue, 28 Oct 2014 15:19:06 +0000 (15:19 +0000)]
Add some basic logic for upstreams.

9 years agoRework host/port/priority parsing.
Vsevolod Stakhov [Tue, 28 Oct 2014 14:03:46 +0000 (14:03 +0000)]
Rework host/port/priority parsing.

9 years agoSplit addr definition from the utils code.
Vsevolod Stakhov [Tue, 28 Oct 2014 13:36:15 +0000 (13:36 +0000)]
Split addr definition from the utils code.

9 years agoStart new upstreams implementation.
Vsevolod Stakhov [Tue, 28 Oct 2014 13:29:13 +0000 (13:29 +0000)]
Start new upstreams implementation.

9 years agoRework API completely.
Vsevolod Stakhov [Tue, 28 Oct 2014 12:18:50 +0000 (12:18 +0000)]
Rework API completely.

9 years agoUpdate readme.
Vsevolod Stakhov [Sun, 26 Oct 2014 18:07:20 +0000 (18:07 +0000)]
Update readme.

9 years agoMerge pull request #112 from fatalbanana/master
Vsevolod Stakhov [Thu, 23 Oct 2014 15:27:42 +0000 (16:27 +0100)]
Merge pull request #112 from fatalbanana/master

Remove CBL as it's wholly included in Spamhaus XBL

9 years agoRemove CBL as it's wholly included in Spamhaus XBL which we're querying already ... 112/head
Andrew Lewis [Thu, 23 Oct 2014 14:23:21 +0000 (16:23 +0200)]
Remove CBL as it's wholly included in Spamhaus XBL which we're querying already - http://cbl.abuseat.org/faq.html

9 years agoMerge pull request #111 from citrin/master
Vsevolod Stakhov [Thu, 23 Oct 2014 12:21:46 +0000 (13:21 +0100)]
Merge pull request #111 from citrin/master

Remove nszones.com fake RBL

9 years agoRemove /0 mask for SPF mx as well.
Vsevolod Stakhov [Wed, 22 Oct 2014 16:11:47 +0000 (17:11 +0100)]
Remove /0 mask for SPF mx as well.

Reported by: @citrin

9 years agoFix upstreams interaction for fuzzy_check.
Vsevolod Stakhov [Wed, 22 Oct 2014 16:07:38 +0000 (17:07 +0100)]
Fix upstreams interaction for fuzzy_check.

9 years agoRevert ucl resynchronization.
Vsevolod Stakhov [Wed, 22 Oct 2014 16:06:29 +0000 (17:06 +0100)]
Revert ucl resynchronization.

Reported by: @citrin

9 years agoProperly set address for PTR components.
Vsevolod Stakhov [Wed, 22 Oct 2014 15:52:40 +0000 (16:52 +0100)]
Properly set address for PTR components.

9 years agoVerify spf PTR records.
Vsevolod Stakhov [Wed, 22 Oct 2014 15:21:18 +0000 (16:21 +0100)]
Verify spf PTR records.

Reported by: @citrin

9 years agoRemove nszones.com fake RBL: 111/head
Anton Yuzhaninov [Tue, 21 Oct 2014 15:19:19 +0000 (19:19 +0400)]
Remove nszones.com fake RBL:
http://www.spamhaus.org/organization/statement/8/

10 years agoAdd compatibility for old libevent.
Vsevolod Stakhov [Fri, 17 Oct 2014 00:10:30 +0000 (01:10 +0100)]
Add compatibility for old libevent.

Issue: #109
Suggested by: @yellowman

10 years agoSync bugfix from libucl.
Vsevolod Stakhov [Thu, 16 Oct 2014 13:12:38 +0000 (14:12 +0100)]
Sync bugfix from libucl.

10 years agoRelease 0.7.2. 0.7.2
Vsevolod Stakhov [Wed, 15 Oct 2014 16:30:22 +0000 (17:30 +0100)]
Release 0.7.2.

* Convert all maps to the compressed radix trie
* Allow IPv6 addresses in IP maps
* Remove dynamic items support from symbols cache
* Allow hex encoded output of strings
* Fix bug with control connections count
* Process fuzzy weight correctly (reported by @fatalbanana)
* Remove extra reference retain of http connection on error
* Remove deprecated options from the default config
* Add `one_shot` attr to metric's symbols
* Doc: add documentation for metrics
* Add Upstart job to debian packaging (by @CameronNemo)
* Config: improve SURBL symbols descriptions (by @citrin)
* Config: reflect SURBL changes (by @citrin):
    - Outblaze removed, malware moved to separate list:
http://www.surbl.org/news/internal/MW-malware-sublist-added-to-multi
* Fix C modules initialization on restart
* Treat single IP as a single IP in radix lists (reported by @citrin)
* Do not touch file and core limits if not asked explicitly (reported by @citrin)
* Improve logging for fuzzy errors
* Block SIGPIPE for HTTP writing
* Doc: update manual pages
* Fix HTTP connection termination
* Reduce default number of parallel requests to 8
* Sync with libucl include features

Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
10 years agoUpdate rpm spec.
Vsevolod Stakhov [Wed, 15 Oct 2014 16:29:58 +0000 (17:29 +0100)]
Update rpm spec.

10 years agoAdded by a mistake.
Vsevolod Stakhov [Wed, 15 Oct 2014 16:29:38 +0000 (17:29 +0100)]
Added by a mistake.

10 years agoUpdate debian packaging.
Vsevolod Stakhov [Wed, 15 Oct 2014 16:29:01 +0000 (17:29 +0100)]
Update debian packaging.

10 years agoUpdate to 0.7.2.
Vsevolod Stakhov [Wed, 15 Oct 2014 16:18:46 +0000 (17:18 +0100)]
Update to 0.7.2.