1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
|
0.9.6:
* Fix memory leak if mime cannot be parsed.
* Fix dkim cache expiration.
* Fix issues with redirector HTTP response.
* Fix abnormal connection closing with certains messages with a high score
(issue #296)
* Fix redirector installation.
* Use specific POE loop for some systems.
* Fix number of issues in URL redirector.
* Fix selecting URLs for sending to redirector.
0.9.5:
* Avoid double free when extending HTTP message.
* Fix double free if multiple classifiers are defined.
* Fix misprint in spamassassin plugin.
* Fix cpuid invocation on i386.
* Fix ownership issues for zero-copy decode.
* Allow __len metamethod on rspamd{text}.
* Add base64 decoding lua utility.
* Fix build on FreeBSD
* Skip spaces at the beginning of mime messages.
* DBL_ABUSE_REDIR should not have significant weight.
* Allow to split by lua_regexp rspamd{text} objects.
* Allow to specify custom stop pattern for lua_tcp.
0.9.4:
* Fix critical bugs in tokenization algorithm
* Write unit tests for tokenization
* Add documentation for lua_tcp
* Switch off legacy tokenization by default.
* Fix critical bugs in words normalization
* Add lua bindings to tokenizer.
* Implement storing of HTTP headers inside task
* Add lua API to accerss HTTP headers data
* Implemented base64 encoding suitable for MIME
* Use caseless hash and equal functions for HTTP request headers.
* Improve debian architectures support (by @dottedmag)
0.9.3:
* Revert incorrect regexp change that broke the default rules
* Fix lua_tcp module
0.9.2:
* Fix error on spawning unique workers.
* Add preliminary version of generic LUA TCP requests API.
* Use lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc)
* Fix fuzzy mime strings with only type.
* Improve thunderbird sanity checks.
* Fix critical bug on matching regular expressions.
* Make hiredis optional dependency.
* Fix multiple bugs in daemon reloading
0.9.1:
* Restore utf8 validation for regular expressions to avoid crashes
* Fix symbols displaying in the interface
* Add symbol groups to the interface
* Fix maps ID parsing in the controller
* Add multimap and regexp modules documentation
* Backport fixes from libucl
* Fix debian package (by @dottedmag)
* Rework XXH32 invocations
0.9.0:
* Add support of the fast and secure protocol level encryption:
- curve25519 is used for key exchange;
- chacha20/poly1305 cryptobox construction for bulk encryption;
- zero latency overhead;
- encrypting and balancing HTTP proxy worker
* Rework expressions and create new expressions library:
- aggressive optimizations based on the abstract syntax tree;
- abstract expressions support (regular expressions, functions, lua modules
composites and so on)
- New comparision and '+' operators support
- New greedy algorithm to minimize execution time of expressions and
all symbols
- Dynamic expressions benchmark and reoptimizations
* Many improvements to the LUA API:
- reworked logger module allowing to do pretty print of the most of lua
types (including tables and userdata classes)
- reworked lua redis and lua HTTP to support more features
- added opaque type for passing large text chunks without copying
- new regexp module with many auxiliary functions (e.g. `re:split`)
* LuaJIT is now the default requirement for rspamd allowing to speed up lua
execution by a large margin (however, plain lua is still supported)
* New plugins:
- spamassassin rules plugin that allows to load and re-use the most of
SA rules natively
- DMARC plugin that evaluates SPF and DKIM policies to the domain policies
- many old plugins has been reworked to implement new features and improve
stability
* New aho-corasic trie implementation from @mischasan that allows to load and
use hundreds of thousands of patterns with no influence on load
* Support of PCRE JIT and PCRE JIT fast path modes that significantly improves
the performance of regular expressions if supported by PCRE
* New URLs parser and extractor:
- removed legacy code that was useless for url finding
- reworked algorithms of URL parsing for more precise and accurate results
- added top-level-domains tree from http://publicsuffix.org
- improved emails parsing
- removed many phishing false positives due to TLD tree check
* New statistics infrastructure:
- created a separate layer of statistic library
- improved OSB-Bayes by re-weighting tokens according to the original
academic paper and `crm114` implementation, which reduced false positives
rate significantly
- created learn cache to avoid double learning of statistics and providing
an efficient way to re-learn class for a message
- created abstract layers for different statistics backends
- implemented new tokenization algorithms with fast or secure (siphash)
hashes to generate statistics features
* Reworked utf8 tokenization that previously corrupted all UTF8 words (minor
incompatibility with old fuzzy hashes with utf-8 symbols)
* SPF module has been completely rewritten to support complex cases of
`include` and `redirect` within SPF records
* DKIM module now supports multiple signatures
* Controller passwords can now be stored encrypted by `PBKDF2-HMAC` in the
configuration file
* Many hand-written HTTP clients has been replaced with the common rspamd
http module
* New test framework:
- import lua `telescope` test framework
- add unit tests for many rspamd modules and routines
- create a unit test for each possible bug found
- use luajit ffi for testing C code
- added preliminary support of functional testing by creating tasks from lua
* Randomize hash seed to avoid certain hash tables vulnerabilities
* Documentation improvements:
- added documentation for the vast majority of rspamd modules
- added documentation for rspamd protocol
- added documentation for the most of rspamd LUA extensions
* Fixed tonns of bugs and memory leaks
* Added tonns of minor features
0.8.3:
* Various critical fixes in distribution (by @dottedmag and @fatalbanana)
* Fixed bugs in url detector to parse certain patterns
* Add default host and helo for a client
* Some sanity checks for tokenizer and classifier
* Reiterate on systemd support
* Fix missing symbol registration
* Add support of spamc compatible output
* Filter double-dots in rbl.lua validate_dns (by @fatalbanana)
* Update ucl submodule due to critical bugfix
0.8.2:
* Create fuzzy db if it does not exist
* Fix: Centos init script: configtest() (by @AlexeySa)
* Enable one_shot for RECEIVED_SPAMHAUS_XBL - Fixes #102 (by @fatalbanana)
* Update Exim patch (by @fatalbanana)
* Fix processing of unix sockets.
* Allow applying settings to authenticated users (by @fatalbanana)
* Make settings priorities work as documented (by @fatalbanana)
* Fix race condition in symbols planner
* Add DNSWL_BLOCKED symbol (by @fatalbanana)
* Make Exim pass usernames to rspamd (by @fatalbanana)
* Update RBL module (by @fatalbanana):
- fix indentation;
- collapse loops;
- avoid calling for un-needed information;
- allow disabling RBLs for authenticated users
* once_received.lua: Fix indentation & add exclusion for authenticated users (by @fatalbanana)
* hfilter.lua: Add exclusion for authenticated users (by @AlexeySa)
* Updates to hfilter rules (by @AlexeySa)
* Set empty <> user or addr for msgs without FROM (by @eneq123)
* Fix: attempt to index field '?' (a nil value) (by @eneq123)
* Fix: if not exist Date-header (by @AlexeySa)
* Add task:get_content() method.
* rbl.lua: Ignore private IP space (by @fatalbanana)
* Allow to check radix maps from lua by rspamd{ip}
* Make local exclusions configurable per-RBL (by @fatalbanana)
* Add rspamd_config:radix_from_config() (by @fatalbanana)
* Support emails dnsbl in rbl (by @fatalbanana)
* Complete rework of url extraction logic
* Allow customizations for unix sockets. (fixes #182)
* Set lua path according to rspamd settings.
* Import lua-functional for plugins stuff.
* Completely rewrite multimap plugin in functional style.
* Fix FORGED_MUA_THUNDERBIRD_MSGID (fixes #186)
* Check IPv6 addresses at dnswl.org and Spamhaus whitelist (by @fatalbanana)
* Add lowercase utility for utf8 strings.
* Various fixes to build system
* Updated debian configuration infrastructure (by @dottedmag)
0.8.1:
* Add sqlite and perl as dependencies for RPM/Debian packages (by @fatalbanana)
* Remove whitelist.lua from RPM file list (by @fatalbanana)
* Make Exim pass hostnames to rspamd (by @fatalbanana)
* Fix building on Fedora (by @fatalbanana)
* Add toggle for disabling installation of systemd units on Linux (by @fatalbanana)
* Fix double format rounding that caused output corruption (reported by @fatalbanana)
* Revert broken change for destructors ordering that led to memory corruption
* Do not reset symbols case of settings if parsed from lua (reported by @andrejzverev)
* Fix build on SunOS (by @wiedi)
* Fix multiple crashes on broken DKIM DNS records
* Fix critical issue with composites weights removing
* Fix memory corruption in composites processing code
* Ignore non-SPF TXT records when parsing SPF includes
0.8.0:
* New fuzzy check logic:
- use shingles algorithm for fuzzy matching
- use blake2 instead of md5 for larger output space
- combine fuzzy and strict matching
- allow to organize private storages by means of keys
- preserve compatibility with previous versions
* New fuzzy storage:
- use sqlite instead of own memory based hash tables
- rework commands interface
- add conversion from the old format
- add fuzzy match by shignles
- support old rspamd versions
* Add lemmatizing for words used in fuzzy hashes that allows to improve match
quality by using of the first forms of all words
* Rework language detection
* Fix several critical bugs, memory leaks and deadlocks:
- memory leak in HTML nodes parsing
- deadlock in logger code
- deadlock in signals processing
- crashes in fuzzy_storage
- crashes in tokenizers if the input was empty
* Import new libucl with several bugfixes and improvements
* Support listening on ipv6 addresses only
* Fix macro expansion in SPF module
* Several bugfixes in DKIM module
* Add load headers support for mime parts to the lua API
* Add documentation for:
- workers in general
- fuzzy_storage worker
- fuzzy_check plugin
- mimepart and textpart lua API modules
0.7.6:
* Apply boundary fix for dkim simple canonization
* Fix ping command
* Return nil if header was not found in lua_task
* Fix hang in upstreams revive logic
* Decode entitles when normalizing HTML parts
* Fix logic of finding URLs in HTML parts
* Do not include \0 into length of text when performing conversion to utf8
* Fix raw vs parsed reperesentations
Raw parts are now:
- decoded b64/qp, but *NOT* converted to utf-8
Processed parts are now:
- converted to UTF-8
- normalized if needed (e.g. HTML tags are stripped)
* Rework DKIM canonization to line based
* Fix fuzzy hashes addding
* Use more specific hash function for fuzzy
* Fix leaking of iconv descriptors
* Fix PTR resolving in lua resolver
* Rework spf module.
- Copy data to memory pool as cached record might be destroyed causing
freed memory being passed to the protocol output (use after free)
- Allow SPF_NEUTRAL policy to be handled separately
- Add R_SPF_NEUTRAL to the default config
* Rework `register_symbols` function
* Allow to disable components of hfilter
0.7.5:
* Fix owner when creating folder /run/rspamd (by @sfirmery)
* Fix IP validity checks
* Decode URLs obtained from HTML tags
* Fix crash with unweighted upstreams
* Stop processing headers in parts
* Set sockaddr.sa_family properly when connectig to upstreams
* Fix reload issues in surbl and fuzzy_check (reported by @citrin)
* Fix timeouts in redirector
* Improve lua errors reporting
* Fix lua closures processing in libucl
* Rework calling of lua functions from regexp module
* Choose raw regexp for raw headers
* Rework conversion to utf since glib one is broken
* Ignore SGML style tags in html
* Fix old bug with non-capturing https urls
* Fix memory corruption on fuzzy reload (reported by @citrin)
* Fix percents display in rspamc
* Fix buffer update for DKIM
* Do not validate utf for raw headers
0.7.4:
* Fix build under *BSD
* Detect HAN unicode script
* Implement language detection heuristic for text parts
* Fix time output in history
* Improve piechart coloring
* Fix \r\n conversion in DKIM module (reported by @citrin)
* Try to detect systems with no IPv6 support
* Fix multiple/single values in use settings (reported by @citrin)
* Rework IP addresses in upstreams:
- Select ipv4/unix addresses if they exist and use ipv6 for ipv6 only
upstreams (since the support of ipv6 is poor in many OSes and
environments)
- Free IP list on upstream destruction
- Add test cases for addresses selection
- Allow adding of free form IP addresses to upstreams
* Fix endiannes in lua_radix search (reported by @citrin)
* Soft shutdown should also set wanna_die flag (reported by @citrin)
* Stop use-after-free in event loop termination
* Fix processing of very short messages in DKIM (reported by @citrin)
* Detect systems without shared mutexes
* Fix issues with PTR and MX elements in SPF parser (reported by @citrin)
0.7.3:
* New upstreams code:
- simplify upstreams API;
- unify strings parsing in upstreams definition;
- add configuration options for the upstreams;
- for failed upstreams re-resolve their addresses;
- use all resolved addresses for an upstream (round-robin);
- implement stable hashing and use it by default for upstreams;
- add unit test for upstreams module.
* Rework signals processing in all rspamd workers:
- signals are now processed in the event loop;
- implement the most common signal handlers for all workers;
- add callbacks for workers specific signal handlers
* Fix critical issue with fuzzy storage:
Fuzzy stroage could not save any hashes on termination due to bugged
signals handling
* Fix roll history IP storage
* Rework ipv4/ipv6 handling in parsing addresses:
- turn off support of IPV6_V6ONLY socket option;
- create ipv6 socket prior to ipv4 one to handle systems with v6/v4
sockets enabled (Linux)
* Remove CBL as it's wholly included in Spamhaus XBL (by @fatalbanana)
* Remove nszones.com fake RBL (by @citrin)
* Fix upstreams interaction for fuzzy_check
* Verify spf PTR records (reported by @citrin)
* Fix spf MX records parsing
* Add compatibility for old libevent (by @yellowman)
* Sync bugfixes from libucl
0.7.2:
* Convert all maps to the compressed radix trie
* Allow IPv6 addresses in IP maps
* Remove dynamic items support from symbols cache
* Allow hex encoded output of strings
* Fix bug with control connections count
* Process fuzzy weight correctly (reported by @fatalbanana)
* Remove extra reference retain of http connection on error
* Remove deprecated options from the default config
* Add `one_shot` attr to metric's symbols
* Doc: add documentation for metrics
* Add Upstart job to debian packaging (by @CameronNemo)
* Config: improve SURBL symbols descriptions (by @citrin)
* Config: reflect SURBL changes (by @citrin):
- Outblaze removed, malware moved to separate list:
http://www.surbl.org/news/internal/MW-malware-sublist-added-to-multi
* Fix C modules initialization on restart
* Treat single IP as a single IP in radix lists (reported by @citrin)
* Do not touch file and core limits if not asked explicitly (reported by @citrin)
* Improve logging for fuzzy errors
* Block SIGPIPE for HTTP writing
* Doc: update manual pages
* Fix HTTP connection termination
* Reduce default number of parallel requests to 8
* Sync with libucl include features
0.7.1:
* Fix typo in stat output.
* Fix issues with includes crossing with the system includes
* Restore testing framework
* Add radix trie test suite
* Implement new path-compressed radix trie.
- The performance benefit over the old algorithm is about 1.5 times.
- Memory usage is significantly lower as well.
- Now radix trie can accept any IPv4/IPv6 values
* Various improvements to the memory pools code
* Fix writing reply to a client when no filters are defined
* Write base32 encoded fuzzy
* Fix 'soft reject' action
* Fix rspamd reload and modules reconfiguration
* Fix subject rewriting for the default subject
* Fix states for processing task and pre-filters
* Fix issues with connection closing
* Fix crashes in rdns
* Fix ratelimit pre-filter
* Update exim patch.
- Update to the recent exim version
- Strip extra leading src/ from the patch
- Remove sendfile since it was broken
- Fix rspamd spam report for exim
* Improve documentation
0.7.0:
* Use HTTP protocol for all operatiosn
* Webui worker is now removed and controller works as webui
* Allow to serve static files via controller by option `static_dir`
* Rspamd interface is now a part of rspamd
* Rspamc client has been rewritten to use HTTP and non-blocking mode
allowing to start multiple operations simultaneously (see `-n` option)
* Lua API was completely reworked to satisfy modern standards of LUA:
* Module `lua-message` was removed
* Reduced number of superglobals registered by rspamd
* Many functions has been redesigned
* Symbols registration is now more convenient
* Users settings has been rewritten as lua plugin
* Reworked headers system as gmime's based one misses many headers and is
very slow to get headers values
* Reorganized code and removed embedded jannsson by using UCL for all json
parsing
* Migrated to `librdns` for DNS resolving that improves concurrency for
DNS requests significantly
* Fixed tonns of bugs in MIME processing
* Improved metrcis and default symbol's weights
* Added new RBL's
* Fixed a number of issues in the modules
* Removed several memory leaks found
* Fix unicode processing
* Fix fuzzy checking for unicode parts
* Significantly improve documentation and especially LUA API docs
* See migration notes at https://rspamd.com/doc/migration.html
0.6.8:
* Controller now listen for localhost and not for 127.0.0.1 by default
* Allow FCrDNS-style RBL lookups (by @fatalbanana)
* Reduce threshold for parts_differ function.
* Fix hostname lookup for rdns rbl (by @AlexeySa)
* Fix HFILTER_URL_ONELINE to reduce false positive rate.
* Fix whitelist module.
* Allow override system predefined settings without touching system ones
by .try_include macro (by @andrejzverev)
* Check for [ip.address]-style HELO and suppress lookups. (by
@fatalbanana)
* Optimize hfilter (by @AlexeySa)
* Fix issue with random numbers generator in dns.
* Use more clever time values to setup entropy.
* Synced with the recent libucl.
0.6.7:
* Use ChaCha20 for DNS generator (more secure DNS id)
* Unknown symbols now has zero weight and not 1.0
* Fix fuzzy hashes expire time
* Fix critical issue in statfiles on FreeBSD 9 (and some other platforms)
* Add .include_map macro to ucl parser
* Update libucl
* Fix headers end detection for DKIM module
* Fix a bug in received headers parser
* Validate IP addresses before pushing them to lua
* Start new documentation project
* Fixed tonns of other minor bugs
* Start to prepare for 0.7 with HTTP protocol and new settings
0.6.6:
* Removed issue with BUFSIZ limitation in the controller output
* Simplify logging symbols escaping
* Adjusted weights for several rules
* Improve spamhaus rbl support
* Removed PBL for received headers checks
* Added hfilter module that performs various HELO and IP checks.
* Rspamd can now be reloaded by HUP signal
* Fuzzy storage should expire hashes properly
* Build system has been reworked for better supportof pkg-config
* Various minor bugfixes
0.6.5:
* Fixed critical bug in DNS resolver, introduced in 0.6.4
* Improved multimap and rbl plugins to skip
* Add dns_sockets option for tuning sockets per server in DNS resolver
* Improved packages for rspamd
0.6.4:
* Added io channels for DNS request to balance load and reduce id
collisions chance
* Fixed a bug in SPF filter that may cause core dump in specific
circumstances
* FIxed default config for rbl module
* It is possible to get a list of rspamc commands with their descriptions
* Added SORBS bl to the default config
* 2tld file for surbl module has been significantly extended
* Perl modules has been removed from the code.
* Fixed an issue in libucl when parsing macros
0.6.3:
* Fixed issues with DNS:
- labels decompression algorithm was fixed;
- added resolve_mx to lua API;
- fixed modules that use DNS.
* Lua modules once_received and emails reworked for new resolver API and UCL.
* Debian package was polished.
* Fixed a bug in fuzzy_check module that prevents correct processing messages
without valid parts.
0.6.2:
* Fuzzy check module has been reworked:
- now fuzzy_check operates with a group of rules, that define which
servers sre to be checked;
- it is possible to specify read_only groups to avoid learning errors;
- turn fuzzy_check to one_shot mode permanently;
- fuzzy_check module configuration is now incompatible with the previous
versions.
* Imported bugfixes from libucl.
* Fixed whitelist plugin.
* Fixed statfiles resizing.
* Improved logging initialization order.
* Fixed race condition in the controller worker.
0.6.1:
* Critical bugfixes:
- fixed build system;
- fixed in_class setting in bayes learning;
0.6.0:
* Use UCL instead xml for configuration (https://github.com/vstakhov/libucl)
* Fix statistics module normalization
* Rework the many modules for the new configuration:
- surbl has incompatible configuration;
- fuzzy_check has incompatible configuration;
- multimap has now new configuration;
- received_rbl is removed and replaced by rbl module.
* Removed deprecated options:
- statfile_pool_size;
- action and required/reject score for a metric.
* Simplify build system and unify configuration for all platforms.
* Improved URL detector (reduced false positive rate).
* Lua 5.2 is now the default and fully supported version.
* Tons of bugfixes and minor improvements.
0.5.6:
* Fix bug with counters incrementing in rolling history
* Detect expl and exp2l as some systems do not have it
* Support input streams without Content-Length
* Implement counters output via rspamc and controller interface
* Fix bug with udp sockets in fuzzy storage
0.5.5:
* New bayes normalizator based on inverse chi-square function
* Various fixes to fuzzy storage
* Allow update fuzzy storage only from specific IP addresses
* Better support of IPv6 and address selection algorithms
* Add CentOS spec file
0.5.4:
* Fixed issues with diff algorithm
* Added support of RRD statistics
* Add webui worker for interface interaction
* Fix a lot of issues with dynamic conf
* Fix critical memory leak in settings code
* Improve stability of the system
0.5.3:
* Added dynamic options
* Added advanced metaclassfication
* Added RESTfull API for controller
* Improved hashing algorithms
* Various fixes for rspamc client:
- allow interacting with unix sockets
- librspamdclient major cleanup
- bayes is now default classifier
0.5.2:
* Added lua bindings for:
- basic mime parts, that allows checkign attachements for example;
- DNS resolver;
* Existing lua bindings now works without task object allowing to use them
in custom code.
* Threads system was reworked to avoid global lua interpreter lock.
* DKIM module now converts all line endings to CRLF how opendkim does.
* URL detector is now more accurate for text parts.
* Several critical bugs and memory leaks were fixed.
0.5.1:
* Added lua worker type to handle network connections in lua.
* Added lua bindings for async_session, IO dispatcher, memory_pool, and
worker.
* Composites can now uses other composites in expressions.
* Fixes for debian package and for FreeBSD >= 9.1
* Add support of gmime-2.6 if gmime-2.4 is not found.
* Improve url detection and phishing detection.
* Add lua mime_part library to get an access to all message part
attributes (like filename, length, type).
0.5.0:
* Added SMTP lightweight balancing proxy with XCLIENT support.
* Added lua bindings for upstreams objects and API.
* New pre-filters are implemented to support initial checking for messages.
* Added ratelimit plugin that uses redis protocol to store data.
* Added ipv6 support to spf and some other modules.
* Unbreak spf plugin.
* Allow options with the same name be threated as list.
* DKIM plugin an parsing code was added.
* Separate build system to put logic in several shared libraries.
* Many bugfixes.
0.3.2:
* Add error handling for regexps
* Fix quit command in controller interface
* Write symbols weights to rspamc output
* Improve logic of selecting rspamc version
* Do not try to parse broken DNS replies
* Add 'raw' flag to FROM_EXCESS_BASE64 rule (requested by citrin)
* Output message id in rspamc reply
* Fix inserting composite symbol
* Fix output of log line
* Document composites
* Add logging for fuzzy checks
* Add logging for learning
* Improve logic of learning messages - do not learn more than specific threshold
* Fix inserting results for symbols that were incorrectly (for example more than 1 time) defined in config file
* Do not output control characters if output is not terminal
* Fix some logic errors in learning
* Consider lua plugins errors as fatal configuration errors
* Fix wirting message id during fuzzy_add command
* Display weight of symbols correctly
* Fixes to winnow learning
* One more try to improve accuracy of winnow algorithm
* Add bayesian classifier (initial version)
* Remove normalizer as it is winnow specific thing, so all statistic algorithms now returns value from 0 to 1
* Some fixes to fuzzy hashes expiration:
* Fix assertion while look up value in NULL hash (found by cirtin)
* Fix normalization for systems that have not tanhl function
* Ignore rfc822 group addresses
* Move images library to core rspamd
* Add lua api to access images properties
* Add post filters to lua API - filters that would be called after all message's processing
* Add ability to check for specified symbol in task results from lua
* Add ability to check for metric's results from lua
* Add ability to learn specified statfile form lua
* Add ability to extract filename and size of images from lua
* Fix assertion while extracting internet address
* Fix races in fuzzy storage
* Make spf parser case insensitive
* Add ability to check hashes of selected mime types
* Add ability to set minimum size in bytes for mime types
* Add ability to set minimum dimensions for images
* Assume all text/* content types as text/plain
* Fix getting data wrapper for gmime24
* Many fixes to statfile syncronization system
* Fixed statfile pool initialization and synchronization with disk
* Prepare 0.3.2
* Fix Mail::Rspamd::Config for new rspamd features
* Use Mail::Rspamd::Config in rspamc client
* Write user's name to rspamd log
* Prepare rspamd build infrastructure for creating rpm and deb packages
* Fix depends
* Add start script for linux systems
* Fix shared usage of statfiles
* Add invalidation of statfiles in case of learning, so now statfiles
* This should fix shared usage of statfile pool by several processes
* Fix misprint (reported by az)
* Fix stupid error when all checks can be done in a single pass
* New trie based url scanner (based on libcamel)
* Small fixes to rspamd perl client
* Write fuzzy hashes info to log
* Add trie interface to lua api
* Explain sample config and cleanup it
0.3.1:
* Add modules documentation
* Continue implementing smtp proxy
* Implement new learning system, now rspamd should be much more intelligent while learning messages
* Convert statistic sums to use long double for counters
* Use hyperbolic tangent for internal normalizer
* In classify normalize result after comparing, not before
* New symbols sorter
* Fix strict aliasing while compiling with optimization
* Fix tanhl detection for platforms that have not implementation of it
* Remove several compile warnings
* Add experimental support of dynamic rules to regexp module
* Document views configuration
* Several fixes to documentation
* Add more logic for dynamic rules
* Add documentation for dynamic rules
* Add ability to make negations in networks in dynamic rules
* Clean up cache items correctly
* Implement basic SMTP dialog:
* Implement interaction with smtp upstream (with support of XCLIENT)
* Check messages received via smtp proxy
* Add support for sendfile in io dispatcher
* Fix issues with compatibility of worker_task and smtp proxy
* Proxy DATA command
* Fix SMTP
* Change metric logic
* Completely remove lex/yacc readers for config
* Make common sense of metric/action and symbols
* Sync changes with all plugins
* Incorrectly removed in previous commit
* Fix misprint (by Andrej Zverev)
* announce the "password" keyword in usage list
* Implement initial version of greylisting triplets storage
* Fix issues with smtp worker
* Fix QUIT command in SMTP worker
* Some fixes about new metrics system (may be incomplete)
* Get weights of symbol from default metric for symbols cache
* Fix setting task->from/task->rctp in smtp client
* Copy from and rcpt correctly
* Some performance improvements to IO dispatcher (do not drain the whole buffer after a single line readed)
* Fix smtp data input
* Fix misprint
* Add limit of maximum allowed smtp session errors
* New logic of SURBL module:
* Use system mkstemp(3) on systems where it is available as glib implementation
* Try to fix memmove issues in io dispatcher
* Remove debug from SURBL module
* Rewrite buffered input for line policy (again)
* Fix issue with links that are ip addresses in numeric form in surbl
* On Darwin use BSD style sendfile definition
* Reorganize platform specific knobs in CMakeLists
* Use gettimeofday on systems that have not clock_getres
* Use ftime for dns trans id generation on systems without clock_getres
* Darwin sendfile(2) support
* TIMEDB->TIMEB
* More to previous commit
* Pass env from main() arguments instead of platform specific global environ
* Fix compatibility issues
* Fix -lintl detection
* Init some variables to avoid problems
* Remove garbadge (gnome terminal sucks)
* Add more information about why we drop smtp connection
* Fix mkstemp call
* Send to upstream QUIT command at the end of session
* Check return value of each rspamd_dispatcher_write as in case of write errors sessions can be destroyed early
* Fix states in smtp dialog
* Use rspamd_snprintf instead of libc one
* Fix URLS command
* Fix reconfigure process of surbl module
* Fix destroying smtp session (unmap memory and do not delete pool early)
* Delete pool after using its variables
* Delay timer must be registered in async session to correctly handle connection termination
* Register dns requests in session too
* Make session before registering events
* Remove events in handlers
* Add ability to set filters for smtp worker for each smtp stage
* Add very initial version of DNS resolver (many things to be done)
* Announce weights and sync
* Fix few typo
* Understand short names of facility in logging config
* Add ability to make whitelist for spf checks
* Misprint != -> ==
* Handle lua tag in way that it is not required to write additional text:
* Strip all starting whitespace symbols from xml texts
* Fix stupid bug in calculating buffer length while reading file maps
* Add resolv.conf parsing into dns.c
* Fix microseconds<->milliseconds conversions
* Take callback argument in Mail::Rspamd::Client for processing files and directories
* Print results if rspamc is called for a directory
* Fix stupid error with surbl module reconfig (another one, blame me)
* Do not show duplicate urls in url header
* Fix detection of numeric urls (reported by citrin)
* Write real time of message's scan to log (not only virtual)
* Fix chartable module in utf mode
* Fix parsing of some broken urls
* Add ability to test regexp with 'T' flag
* Write more code for DNS resolver:
* Make DNS resolver working
* Many improvements to rspamd test suite: now it CAN be used for testing rspamd functionality
* Write DNS resolver tests
* Fix issues with memory_pool mutexes and with creating of statfiles
* Forgotten in previous commit
* Add support for parsing SPF and SRV records
* Fix PTR parsing
* Add tests
* Make SURBL module to use rspamd dns module
* Several fixes to DNS logic
* Remove evdns and use only rspamd resolver
* Very hard to detect problem with race among error in socket and destroying task while we are writing to socket and go through a hash table
* Fix resolving in smtp module
* Init events before configuring resolver in smtp worker
* Set resolver inside task
* Fix reload signal (reported by citrin)
* Some improvements to redirector
* Call has_forked method to inform POE about fork
* Fix lua DNS code
* Decompress labels in DNS packets more strictly
* Fix some problems with TXT records
* Try to fix removing of DNS events
* Do not insert unparsed RR's into reply
* Calling callbacks may cause destroying session from which we are calling callback so we MUST call callback as the latest action
* Fix check_smtp_data function
* Add ability to make views by recipient
* Add ability to set metric's action from config file
* Fix bug with writing garbadge if message has no urls or no messages
* Fix bug with incorrect behaviour of compare_parts_distance function
* Add ability to assign several actions to one metric
* Report action in rspamc protocol
* Mail::Rspamd::Client and rspamc can now understand Action header too
* Write action to log as well
* Make valgrind happy about comparing symbols
* Add more debug to comparing parts distance function
* Write action even if message has no symbols
* Make improvements to HTML entites decoder: now it replaces entities with common characters and
* Add -d option to force debug output
* Assume 7bit as default transfer encoding
* Do not overwrite lua plugins and configs if they already exists in target directory
* Improve logging
* Write queue id to log
* Remove test messages from cmake
* Reopen log file by USR1 signal
* Add reopenlog method to FreeBSD rc script
* Adopt foreach for cmake 2.6
* Fix to rc script
* Do not try to resolve names with several dots in a row
* Fix surbl request formatting for ip addresses
* Handle cases of broken requests
* Fix problems with parsing compressed names
* Fix TXT records parsing
* Fix expanding spf macros that may fail in rare cases
* Fix another error with early task destroying
* Handle empty from header
* Improve reopenlog command in rc script
* Strip trailing whitespace characters in Mail::Rspamd::Client
* Use ungreedy match to strip trailing whitespaces
* Stupid error in calculation compressed label length
* Some optimizations to client library
* Do not compare empty parts
* Empty and non-empty parts are allways different
* Save in regexp cache the whole regexp with header name (if exists) and with flags
* Add rspamd_log variable to lua plugins to access logging functions
* Each part in rspamd task now can have parent part
* Check for parts distance only for multipart/alternative subparts
* Do not check attachements even if they are text (but attached as file)
* Do not die if write (2) returned ENOSPACE while doing logging, turn on throttling mode instead (1 write try in a second)
* Add ability to turn on debug for specific symbols
* Add ability to configure dns timeouts and dns retransmits in config file
* More debug
* Fix extracting arguments in lua logger interface
* Turn off debug_ip during reload if it was disabled by new config
* Improve lua logging
* Pre-init symbols cache when rereading config
* Fix lua representing of invalid ip (nil, not 255.255.255.255)
* Fix R_TO_SEEMS_AUTO rule (by citrin)
* Add multimap lua plugin
* Fix some multimap issues
* Try to save images hashes to fuzzy storage to stop some annoying spammers
* Allocate some more bytes for read buffer to avoid incorrect behavoiur
* Add ability to check dns black lists by multimap module
* Add multimap documentation
* Fix labels parsing
* Another try to save regexps in cache correctly
* Improve test logs for regexps
* Fix parsing txt records to avoid reading of uninitialized data
* Fix error with writing symbols cache file
* Fix error while working in utf mode when raw regexps was not created properly
* Do not add extra byte while converting text to utf
* Add error handling for regexps
|