peilaus alkaen
https://github.com/rspamd/rspamd.git
synced 2024-08-19 02:10:31 +02:00
b8e108bc9c
101 Commitit
| Tekijä | SHA1 | Viesti | Päivämäärä | |
|---|---|---|---|---|
Vsevolod Stakhov
|
cc1aef5ea4 | [Minor] Add Changelog for 1.9.4 | ||
|
Vsevolod Stakhov
|
fe91396af3
|
Release 1.9.3
* [Conf] Add IP_SCORE_FREEMAIL composite rule
* [Feature] Add cryptobox method to generate dkim keypairs
* [Feature] Add fast hashes to lua cryptobox hash
* [Feature] Add least passthrough results
* [Feature] Allow oversign if exists mode
* [Feature] Clickhouse: Modernise table initial schema
* [Feature] Implement IUF interface for specific fast hashes
* [Feature] Lua_util: Allow to obfuscate different fields
* [Feature] Tune memory management in Rspamd and Lua
* [Fix] Avoid buffer overflow when printing long lua strings
* [Fix] Change the default oversigning headers to a more sane list
* [Fix] Clickhouse: Do not store digest as it is not needed now
* [Fix] Clickhouse: Fix lots of storage issues
* [Fix] Clickhouse: Support custom actions
* [Fix] Deny URLs where hostname is bogus
* [Fix] Do not blacklist mail by SPF/DMARC for local/authed users
* [Fix] Fix DoS caused by bug in glib
* [Fix] Fix UCL parsing of the multiline strings
* [Fix] Fix buffer overflow when printing small floats
* [Fix] Fix init code for servers keypairs cache
* [Fix] Fix issue with urls with no tld (e.g. IP)
* [Fix] Fix memory in arc signing logic
* [Fix] Fix memory leak in language detector during reloads
* [Fix] Fix mixed case content type processing
* [Fix] Fix processing of the ip urls in file
* [Fix] Fix use after free
* [Fix] HTML: Fix `size` attribute processing
* [Fix] Hum, it seems that
|
||
|
Vsevolod Stakhov
|
fb577bd5a3
|
Release 1.9.2
* [Conf] Allow to load users plugins from plugins.d * [Conf] oversign openpgp and autocrypt headers * [Feature] Add SPF FFI library for Lua * [Feature] Add more verbosity for SPF caching * [Feature] Antivirus: Handle encrypted files specially * [Feature] Clickhouse: Slashing - add new fields to CH * [Feature] Dkim_signing: Add OpenDKIM like signing_table and key_table * [Feature] Dkim_signing: Allow to use new options as maps * [Feature] Import fpconv library * [Feature] Lua_maps: Allow static regexp and glob maps * [Feature] Parse ical files * [Feature] Rspamadm: Add dns_tool utility * [Feature] Store SPF records digests * [Feature] Use fpconv girsu2 implementation for printing floats * [Fix] Clickhouse: Use integer seconds when inserting rows * [Fix] Fix floating point printing * [Fix] Fix processing of embedded urls * [Fix] Lua_clickhouse: Fix CH errors processing * [Fix] Make spf digest stable * [Fix] Properly detect encrypted files in zip archives * [Fix] Slashing: Store times in GMT timezone in ClickHouse * [Rules] Add additional conditions to perform BTC checks * [Rules] Fix pay-to-hash addresses validation Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
38ca7b5e10
|
Release 1.9.1
* [Conf] Add vendor groups for symbols * [Feature] Add `rspamadm template` command * [Feature] Allow to add messages from settings * [Feature] Allow unconnected DNS servers operations * [Feature] Check limits after being set, migrate to uint64 * [Feature] Greylist: Allow to disable greylisting depending on symbols * [Feature] Improve lua binary strings output * [Feature] Mime_types: Implement user configurable extension filters * [Feature] Mime_types: When no extension defined, detect it by content * [Feature] Preprocess config files using jinja templates * [Feature] Replies: Filter replies sender to limit whitelisting to direct messages * [Feature] Treat all tags with HREF as a potential hyperlinks * [Feature] Validate BTC addresses in LEAKED_PASSWORD_SCAM * [Fix] Add crash safety for HTTP async routines * [Fix] Another fix for Redis sentinel * [Fix] Clickhouse: Fix table schema upload * [Fix] Core: Fix squeezed dependencies handling for virtual symbols * [Fix] Finally fix default parameters parsing in actions section * [Fix] Fix ES sending logic (restore from coroutines mess) * [Fix] Fix finishing script for clickhouse collection * [Fix] Fix priority for regexp symbols registriation * [Fix] Fix various issues found by PVS Studio * [Fix] Initialize lua debugging earlier * [Fix] Neural: Fix training * [Fix] Rework cached Redis logic to avoid sentinels breaking * [Fix] SURBL: Fix regression in surbl module * [Fix] Fix double signing in the milter * [Project] Add support of HTTP proxy in requests * [Rework] Change lua global variables registration * [Rework] Rework HTML content urls extraction * [Rework] Start rework of aliasing in Rspamd * [WebUI] Combine Scan and Learning into one tab * [WebUI] Fix symbol score input type * [WebUI] Show grayed out pie * [WebUI] Update Throughput summary values dynamically Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
0e2437493e
|
Release 1.9.0
* [Conf] Add missing includes * [Conf] Move to options * [Conf] Rbl: DWL is actually special whitelist * [Conf] Relax some uribl rules * [Conf] Remove abuse.ch * [CritFix] Html: Entities are not valid within tag params values * [Feature] Add `rspamadm mime sign` tool * [Feature] Add configgraph utility * [Feature] Add dedicated ZW spaces detection for URLs * [Feature] Add flag to url object when visible part is url_like * [Feature] Add method task:lookup_words * [Feature] Add pyzor support (by crosenberg) * [Feature] Allow to add upstream watchers to Lua API * [Feature] Allow to set rewrite subject pattern from settings * [Feature] Better escaping of unicode * [Feature] Clickhouse: Allow to store subject in Clickhouse * [Feature] Core: Add QP encoding utility * [Feature] Core: Add libmagic detection for all parts * [Feature] Core: Add support for gzip archives * [Feature] Core: Allow to construct scan tasks from raw data * [Feature] Core: Detect charset in archived files * [Feature] Core: Ignore and mark invisible spaces * [Feature] Core: Normalise zero-width spaces in urls * [Feature] Core: Process data urls for images * [Feature] Core: Relax quoted-printable encoding * [Feature] Core: Support RFC2231 encoding in headers * [Feature] Core: Support telephone URLs * [Feature] Core: allow to emit soft reject on task timeout * [Feature] DCC: Add bulkness and reputation checks to dcc * [Feature] Elastic: Modernize plugin * [Feature] Export visible part of url to lua * [Feature] Fuzzy_storage: add preliminary support of rate limits * [Feature] HTML: Specially treat data urls in HTML * [Feature] Implement event watchers for upstreams * [Feature] Implement includes tracing in Lua * [Feature] Improve dkim part in configwizard * [Feature] Lua_scanners: Add VadeSecure engine support * [Feature] Lua_task: Add flexible method to get specific urls * [Feature] Mime_types: Add MIME_BAD_UNICODE rule * [Feature] Mime_types: Use detected content type as well * [Feature] Plugins: Add preliminary version of the external services plugin * [Feature] Query sentinel on master errors * [Feature] Regexp: Allow local lua functions in Rspamd regexp module * [Feature] Rspamadm: Allow to append footers to plain messages * [Feature] Rspamadm: Allow to rewrite headers in messages * [Feature] Selectors: Add `ipmask` processor * [Feature] Settings: Allow hostname match * [Feature] Settings: Allow local when selecting settings * [Feature] Settings: Allow multiple selectors * [Feature] Settings: Allow to inverse conditions * [Feature] Support User-Agent in HTTP requests * [Feature] Support ed25519 dkim keys generation * [Feature] Try to filter bad unicode types during normalisation * [Feature] external_services - oletools (olefy) support * [Feature] lua_scanners - icap protocol support * [Feature] lua_scanners - spamassassin spam scanner * [Fix] Add filter for absurdic URLs * [Fix] Add some more cases for Received header * [Fix] Allow to disable/enable composite symbols * [Fix] Arc: Use a separated list of headers for arc signing * [Fix] Archive: Final fixes for 7z archives * [Fix] Clickhouse: Fix database usage * [Fix] Controller: Make save stats timer persistent * [Fix] Core: Detect encrypted rarv5 archives * [Fix] Core: Don't detect language twice * [Fix] Core: Fix address rotation bug * [Fix] Core: Fix content calculations for message parts * [Fix] Core: Fix emails comments parsing and other issues * [Fix] Core: Fix etags support * [Fix] Core: Fix headers folding on the last token * [Fix] Core: Fix iso-8859-16 encoding * [Fix] Core: Fix log_urls flag (and encrypted logging) * [Fix] Core: Fix part length when dealing with boundaries * [Fix] Core: Fix parts distance calculations * [Fix] Core: Fix processing of NDNs of certain type * [Fix] Core: Implement logic to find some bad characters in URLs * [Fix] Core: treat nodes with ttl properly in lru cache * [Fix] Fix Content-Type parsing * [Fix] Fix HTTP headers signing case * [Fix] Fix control interface * [Fix] Fix deletion of the duplicate headers * [Fix] Fix emails filtering in emails module * [Fix] Fix greylisting log message and logic * [Fix] Fix issues with storing of the accepted addr in rspamd control * [Fix] Fix maps object update race condition * [Fix] Fix memor leaks and whitespace processing * [Fix] Fix processing of null bytes in headers * [Fix] Fix rcpt_mime and from_mime in user settings * [Fix] Fix rfc2047 decoding for CD headers * [Fix] Fix rfc2231 for Content-Disposition header * [Fix] Fix setting of the subject pattern in config * [Fix] Greylist: fix records checking * [Fix] HTML: Another HTML comments exception fix * [Fix] HTML: Another entities decoding logic fix * [Fix] HTML: Fix HTML comments with many dashes * [Fix] HTML: Fix entities in HTML attributes * [Fix] HTML: Fix some more SGML tags issues * [Fix] Ignore whitespaces at the end of value in DKIM records * [Fix] MID module: Fix DKIM domain matching * [Fix] Milter_headers: Fix remove_upstream_spam_flag and modernise config * [Fix] Mime_parser: Fix issue with parsing of the trailing garbadge * [Fix] Mime_parser: Fix parsing of mime parts without closing boundary * [Fix] Multimap: Fix operating with userdata * [Fix] Process orphaned `symbols` section * [Fix] Rdns: Fix multiple replies in fake replies * [Fix] Rework groups scores definitions * [Fix] Set proper element when reading data from Sentinel * [Fix] Set rspamd user to initialise supplementary groups on reload * [Fix] Settings: Fix selectors usage * [Fix] Sort data received from Sentinel to avoid constant replacing * [Fix] groups.conf - filename typo * [Fix] lua_scanner - oletools typos, logging * [Fix] lua_scanners - actions and symbol_fail * [Fix] lua_scanners - fix luacheck * [Fix] lua_scanners - kaspersky - response with fname * [Fix] lua_scanners - savapi redis prefix * [Fix] tests - antivirus - fprot symbols * [Project] Add concept of flexible actions * [Project] Add heuristical from parser to received parser * [Project] Add new flags to clickhouse, redis and elastic exporters * [Project] Attach new received parser * [Project] Fallback to callbacks from coroutines * [Project] Implement keep-alive support in lua_http * [Project] Lua_udp: Implement fully functional client * [Project] Plug keepalive knobs into http connection handling * [Project] Rspamadm: Add `modify` tool * [Rework] Convert rspamd-server to a shared library * [Rework] Dcc: Rework DCC plugin * [Rework] Enable explicit coroutines symbols * [Rework] Rework telephone urls parsing logic * [Rework] Rewrite RBL module * [Rework] Settings: Rework settings check * [Rework] Slashing: Distinguish lualibdir, pluginsdir and sharedir * [Rework] Unify task_timeout * [Rework] Use VEX instructions in assembly, relocate * [WebUI] Notify user if uploaded data was not learned * [WebUI] Remove redundant condition Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
1b9f2f3026
|
Release 1.8.3
* [CritFix] Make flags mutually exclusive for mime parts * [CritFix] Strictly deny unencoded bad utf8 sequences in headers * [Feature] Add Kaspersky antivirus support * [Feature] Add method to get dkim results * [Feature] Add more words regexp classes * [Feature] Allow to choose words format in `rspamadm mime` * [Feature] Allow to get all types of words from Lua * [Feature] Allow to get task flags in C expressions * [Feature] Allow to require encryption when accepting connections * [Feature] Ignore bogus whitespaces in the words * [Feature] Implement more strict configuration tests * [Feature] Improve SPF results in Authentication-Results * [Feature] Support ClickHouse database * [Fix] Add failsafety for utf8 regexps * [Fix] Do not trigger BROKEN_CONTENT_TYPE on innocent text parts * [Fix] Emit error if connection has been terminated with no stop pattern * [Fix] Fix boundaries checks in embedded messages * [Fix] Fix double free * [Fix] Perform policy downgrade on sample out, add tests * [Fix] Properly escape utf8 regexps in hyperscan mode * [Fix] Selectors - attachments args condition * [Fix] Some fixes for raw parts * [Fix] Treat learning errors as non-fatal * [Fix] Use tld when looking for DKIM domains * [Project] Words unicode structure rework * [Project] Add preliminary Redis Sentinel support * [Project] Improve Authentication-Results header * [Project] Rework DKIM checks results * [Project] Use more generalised API to produce meta words Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
dcab7362e0
|
Release 1.8.2
* [Conf] Add DWL support in the default configuration * [Conf] Disable rspamd_update by default (again) * [Conf] Fix configuration sample for ratelimit * [CritFix] Disable broken url tags by default * [CritFix] Fix \0 processing when doing RSA sign * [CritFix] Fix adding symbols to their primary groups * [Feature] Add `rspamadm cookie` utility * [Feature] Add specialised functions for generating encrypted cookies * [Feature] Add support of cookies in replies module * [Feature] Add support of words regexps * [Feature] Allow to add 3rd party clang plugins * [Feature] Allow to create lua regexps from glob or plain patterns * [Feature] Allow to set custom limits for upstream lists * [Feature] Detect orphaned parts and attach them to message * [Feature] Filter tokens in bayes * [Feature] Fold b= value when doing arc sealing * [Feature] Ignore cookies in the future and too old in the past * [Feature] Skip stop words in statistics * [Feature] Store stop words and allow to query them * [Feature] Support query arguments in controller's custom commands * [Feature] Tune upstream limits in Rspamd proxy * [Feature] Use different callback symbols for different uribls * [Feature] Write DKIM selector in dkim allow/reject symbols * [Fix] Add obs_fws state support to eoh state machine * [Fix] Add sanity check when applying mime boundaries heuristic * [Fix] Antivirus - virus names with 0 were recognized as tables * [Fix] Disable headernames in bayes temporarily * [Fix] Do not allow syntax errors in include files... * [Fix] Do not allow to merge an object with an array (or vice versa) * [Fix] Don't perform forged recipients check for missing recipients * [Fix] Fix DKIM based RBLs * [Fix] Fix actrie implementation (sync from upstream), fixed OOB read * [Fix] Fix explicit methods call in selectors * [Fix] Fix extraction of additional parts * [Fix] Fix finalization for internal plugins * [Fix] Fix override_defaults function * [Fix] Fix squeezed symbols when using settings * [Fix] Fix urls insertion in Clickhouse module * [Fix] Furhter fixes to ratelimits logic * [Fix] Ignore signatures when looking for boundaries * [Fix] Properly set learned count * [Fix] Really fix ratelimits configuration and work * [Fix] Remove ambigious format flag from printf * [Fix] Restore URLs exporting in ClickHouse plugin * [Fix] Rework bayes calculations... * [Fix] Switch from chi-square to naive for large Fisher value * [Fix] Treat normal password as enable password if there is no enable password * [Fix] Use proper syntax for making DNS requests * [Fix] Various fixes in embedded plugins * [Project] Change fuzzy check selection logic to lua_fuzzy library * [Project] Rework async events and symbols * [Project] Move all metatokens in Bayes to lua_stat from C * [WebUI] Add history rows per page control Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
6fe627f31f
|
Release 1.8.1
* [CritFix] Fix options insertion * [CritFix] Fix words decay one more time (affects long messages) * [CritFix] Increase default words_decay * [CritFix] Plug memory leak in redis pool * [Feature] Add `check_violation` feature to DKIM/ARC signing * [Feature] Add only unique elements to Clickhouse url arrays * [Feature] Allow `g+:` and `g-:` composite atoms * [Feature] Allow dkim domains check in surbl * [Feature] Allow maps with HTTP auth * [Feature] Allow to disable actions by users settings * [Feature] Extend whitelisting options * [Feature] Store url object in images * [Feature] Use verdict instead of the plain action in plugins * [Fix] Allow to call fstring append with NULL string * [Fix] DCC - luacheck * [Fix] Do not load torch on each rspamadm invocation * [Fix] Fix boundaries detection and rework stop words algorithm * [Fix] Fix dependencies for DNS_SIGNED symbol * [Fix] Fix errors when dealing with dynamic rates/bursts in Ratelimit * [Fix] Fix groups mess * [Fix] Fix groups mess * [Fix] Fix parsing address with comments * [Fix] Fix resolving in DMARC reports * [Fix] Fix various issues with parsing of the received headers * [Fix] Fix watchers issue in lua_tcp when doing no resolving * [Fix] Plug memory leak in language detector (affects reloads) * [Fix] Remove one letter stop words * [Fix] Slashing: backport chunk logic from libucl * [Fix] Stop libevent from using cached time in rspamadm * [Fix] Try to fix watchers chaining * [Fix] Various fixes in redis sync interface * [Fix] ip_score - respect check_authed and check_local settings from config * [Project] Rework passthrough actions * [Project] Clustering module * [Rework] Always create result for a task * [Rework] Completely rewrite DMARC checks logic * [Rework] Rework and fix whitelist plugin * [WebUI] Add symbols sorting buttons * [WebUI] Change symbols order without updating history * [WebUI] Colorize symbols * [WebUI] Do not display password form when secure_ip is set * [WebUI] Fix symbol description tooltips display * [WebUI] History: add sorting by symbol score value Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
f7f7588e22
|
Release 1.8.0
* [Feature] Add arguments schemas to processors and extractors * [Feature] Add functional selectors library * [Feature] Add generic selector to reputation module * [Feature] Add more ratelimits: by digest, by attachments data, by filenames * [Feature] Add preliminary stop words detection support * [Feature] Add pure Lua debugm function * [Feature] Add schema validation for Redis settings * [Feature] Add selectors combine function * [Feature] Add some recursion protection to lua logger * [Feature] Add support for Lua API tracing * [Feature] Allow to apply schema to arguments * [Feature] Allow to get dkim signing data directly from HTTP headers * [Feature] Allow to reuse existing authentication results * [Feature] Cache selectors results in re runtime * [Feature] Implement new text tokenizer based on libicu * [Feature] Integrate selectors framework to multimap * [Feature] Relax FORGED_RECIPIENTS * [Feature] Support (almost) all html entities * [Feature] Support adding and deletion of recipients in the milter block * [Feature] Support gathering HTTP body from fragments in lua_http * [Feature] Support multi flag in regexp and glob maps * [Feature] Support selectors in ratelimit module * [Feature] Support selectors in settings * [Feature] Use khash in HTML parser * [Feature] Use pure Lua debugm function * [Fix] Add fail-safety for destroying sessions * [Fix] Allow to add result-less fake DNS records * [Fix] Another try to fix race conditions on config unload * [Fix] Call Lua callback on DNS timeouts * [Fix] Deprecate task:inc_dns_req as it is redundant * [Fix] Do not allow events deletions on cleanup * [Fix] Do not try to process skipped messages * [Fix] Fix HTTP requests with no body * [Fix] Fix another cleanup race condition * [Fix] Fix bug in processing of pcre regexps * [Fix] Fix byte array allocation in the pool * [Fix] Fix crashes on task cleanup * [Fix] Fix dynamic buckets in ratelimits * [Fix] Fix endless loop when waiting for Rspamd to stop * [Fix] Fix lua_util.str_split in case of delimiters set * [Fix] Fix more issues with watching of async events * [Fix] Fix stop words detection and loading logic * [Fix] Fix various corner cases for language detection * [Fix] Fix watchers in lua_tcp * [Fix] Fix words decay algorithm * [Fix] Implement watchers replacement to handle nested calls * [Fix] Save faked code into fake dns record * [Fix] Show the proper frame when using lua_util.debugm * [Fix] Use fake dns records in tests * [Fix] Use unicode replacements for HTML entities * [Fix] fixed "cannot find dependency on symbol 1" issue when using replaced symbols in spamassassin rules * [Fix] partition_id is not available in old versions of CH * [Project] Add implicit conversion logic to selectors * [Project] Add initial support for selectors in regexps * [Project] Add method concept * [Project] Further changes in unicode operations * [Project] Implement Clickhouse migrations * [Project] Implement implicit conversions to userdata * [Project] Implement insert method * [Project] Implement selectors registration for regular expressions * [Project] Implement selectors support in re_cache * [Project] Improve language detector: cleanup unused files, categorize * [Project] Migrate CH data to a fat table * [Project] Rework selectors logic * [Project] Start Clickhouse utilities library * [Project] Start unicode rework * [Project] coroutine threaded model for API calls: thread pool * [Rework] Move phishtank to a DNS based service * [Rework] Rework Clickhouse plugin to use the new API * [Rework] Rework language detector * [Rework] Rework utf content processing in text parts * [WebUI] Add progress bar for AJAX requests * [WebUI] Avoid errors table reinitialization * [WebUI] Avoid history table reinitialization * [WebUI] Avoid throughput summary table reinitialization * [WebUI] Destroy summary table on disconnect * [WebUI] Fix "auth" request URL * [WebUI] Fix disabling and hiding controls on page reload * [WebUI] Fix maps loading from neighbours * [WebUI] Fix symbols sorting by score * [WebUI] Fix tables destroying * [WebUI] Fix throughput data consolidation * [WebUI] Fix upload buttons disabling * [WebUI] Notify user on module loading failure * [WebUI] Update FooTable 3.1.4 -> 3.1.6 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
f63ec0e39b
|
Release 1.7.9
* [CritFix] Fix caseless comparison of equal length strings * [Feature] Add HTTP basic auth support to elastic and clickhouse plugins * [Feature] Add SPF selector to reputation * [Feature] Add support of the fallback backends for HTTP maps * [Feature] Allow to print full mime structure when extracting mime data * [Feature] Allow to split symbols in reputation plugin * [Feature] Check attachments only on AV scanners in attachments_only mode * [Feature] Disable all SSL checks if ssl_no_verify flag is set * [Feature] Implement parsing of scoped IPv6 addresses * [Feature] Improve `rspamc counters` output * [Fix] Add sanity checks when expanding SPF macros * [Fix] Allow to parse SA rules with no spaces around =~ (dirty hack) * [Fix] Avoid one extra byte writing * [Fix] Deal with direct hash table * [Fix] Detect empty text part as text, not HTML * [Fix] Do not reduce map watch timeout for mixed http/file maps * [Fix] Fix HTML part detection heuristic * [Fix] Fix double free in redirectors cleanup * [Fix] Fix legacy history handling in the controller * [Fix] Fix messages insertion * [Fix] Fix sending string method * [Fix] Fix statconver command line arguments * [Fix] Fixed argument checking for being null * [Fix] Fixed issues reported by luacheck * [Fix] Freeze updates queue when do actual storage update * [Fix] HTTP map hash is per-backend and not per-map * [Fix] Plug memory leak in fuzzy updates * [Fix] Prefer 'MTA-Name' when producing authentication results * [Fix] Replace bad unicode sequences instead of stopping on them * [Fix] Set classifier version on learning * [Project] Reworked ratelimits * [Project] Apply topological sorting for symbols in Rspamd * [Project] Remove global contexts from C modules * [Project] Move performance critical hash tables to khash * [WebUI] Avoid unused indexes * [WebUI] Do not execute `on_success` callback * [WebUI] Fix history reset for "All SERVERS" (#2346) * [WebUI] Fix query URL for selected server * [WebUI] Fix symbols display in legacy history, * [WebUI] Hide symbols order selector for legacy history * [WebUI] Refactor query functions into one * [WebUI] Remove previously-attached event handlers * [WebUI] Save symbols to the selected server * [WebUI] Unify arguments of query functions * [WebUI] Use common query functions to get graph data * [WebUI] Use common query functions to save symbols Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
4c80975059
|
Release 1.7.8
* [Feature] Add more extended statistics about fuzzy updates * [Feature] Add more non-conformant Received headers support * [Feature] Add preliminary function to get fuzzy hashes from text in Lua * [Feature] Allow to configure AV module rejection message * [Feature] Implement fuzzy hashes extraction in mime tool * [Feature] Improve WHITE_ON_WHITE rule * [Feature] Improve integer -> string conversion * [Feature] Reuse maps in multimap module more aggressively * [Fix] Avoid race condition in skip map as pool lifetime is not enough * [Fix] Eliminate all specific C plugins pools * [Fix] Fix DKIM check rule if DNS is unavailable * [Fix] Fix build where ucontext is defined in ucontext.h * [Fix] Fix crash in base url handling * [Fix] Fix descriptors leak in sqlite3 locking code * [Fix] Fix messages quarantine * [Fix] Fix padded numbers printing * [Fix] Fix race condition on maps reinit * [Fix] Fix regexp functions when no data is passed * [Fix] Fix specific urls extraction * [Fix] Fix styles propagation * [Fix] Improve resetting of the limit buckets * [Fix] Initialize sqlite3 properly * [Fix] Work with broken resolvers in resolv.conf * [Project] Implement HTTP maps caching * [Project] Refresh fuzzy hashes when matched * [Project] Add logic to deduplicate fuzzy updates queue * [WebUI] Add missed declarations * [WebUI] Avoid using "undefined" property * [WebUI] Do not accept passwords containing control characters * [WebUI] Do not redeclare variables * [WebUI] Enable strict mode, * [WebUI] Fix variable assignment * [WebUI] Initialize variables at declaration * [WebUI] Remove duplicated path from RequireJS config * [WebUI] Remove unused block * [WebUI] Remove unused variable * [WebUI] Remove unused variables * [WebUI] Use self-explanatory notation * [WebUI] Use type-safe equality operators Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
a20e33c22d
|
Release 1.7.7
* [CritFix] Check NM part of pubkey to match it with rotating keypairs * [CritFix] Do not overwrite PID of the main process * [CritFix] Fix maps after reload * [CritFix] Fix maps race conditions on reload * [CritFix] Fix shmem leak in encrypting proxy mode * [Feature] Add a concept of ignored symbols to avoid race conditions * [Feature] Add ability to print bayes tokens in rspamadm mime * [Feature] Add method to get statistical tokens in Lua API * [Feature] Add preliminary mime stat command * [Feature] Add rspamadm mime tool * [Feature] Add urls extraction tool * [Feature] Address ZeroFont exploit * [Feature] Allow rspamadm mime to process multiple files * [Feature] Allow to extract words in `rspamadm mime` * [Feature] Allow to print mime part data * [Feature] Allow to show HTML structure on extraction * [Feature] Distinguish IP failures from connection failures * [Feature] Improve output for mime command * [Feature] Improve styles propagation * [Feature] Main process crash will now cleanup all children * [Feature] Preload file and static maps in main process * [Feature] Print stack trace on crash * [Feature] Process font size in HTML parser * [Feature] Propagate content length of invisible tags * [Feature] Read ordinary file maps in chunks to be more safe on rewrites * [Feature] Support base tag in HTML * [Feature] Support more size suffixes when parsing HTML styles * [Feature] Support opacity style * [Fix] Another fix for nested composites * [Fix] Fill nm id in keypairs cache code * [Fix] Fix colors alpha channel handling * [Fix] Fix destruction logic * [Fix] Fix double free * [Fix] Fix maps preload logic * [Fix] Fix nested composites process * [Fix] Fix proxying of Exim connections * [Fix] Fix reload crash * [Fix] Fix rspamadm -l command * [Fix] Update ed25519 signing schema * [WebUI] Stop using "const" declaration * [WebUI] Update RequireJS to 2.3.5 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
d9c84b4bd0
|
Release 1.7.6
* [CritFix] Fix multiple neural networks support * [Feature] Add decryption function to keypair command * [Feature] Add gzip compression for HTTP requests in elastic module * [Feature] Add gzip methods to lua util * [Feature] Add maps based on Top Level Domains * [Feature] Add pubkey checks for dkim_signing * [Feature] Add support of fake DNS records * [Feature] Add tool to encrypt files * [Feature] Allow to add symbols using settings directly * [Feature] Allow to match private and public keys for DKIM signatures * [Feature] Allow to set task flags via settings * [Feature] Allow to specify fake DNS address from the config * [Feature] Implement signatures verification using rspamadm keypair * [Feature] Implement signing using `rspamadm keypair` * [Feature] Improve error reporting for DKIM key access issues * [Feature] Provide $HOSTNAME variable in UCL * [Feature] Rework levenshtein distance computation * [Feature] Split message parsing and processing * [Feature] Support ED25519 DKIM signatures * [Feature] Support encrypted configs in UCL * [Feature] Suppress duplicate warning on very large radix tries * [Feature] Use OSB to combine header names * [Fix] Cleanup maps data on shutdown * [Fix] Fix '~' behaviour in composites * [Fix] Fix HTTP maps updates * [Fix] Fix NIST signatures * [Fix] Fix RFC822 comments when processing a mime address * [Fix] Fix double free * [Fix] Fix dynamic settings application * [Fix] Fix for CommuniGate Pro maillist * [Fix] Fix keypair creation method to actually create keypair... * [Fix] Fix matching patterns with no paths * [Fix] Fix memory leak in parsing comments * [Fix] Fix parsing of urls with numeric password * [Fix] Fix plugins intialisation in configwizard * [Fix] Fix potential crash on reload * [Fix] Fix potential race condition for a finished HTTP connections * [Fix] Fix race-condition leak on processes reload * [Fix] Fix signing in openssl mode * [Fix] Free language detector structures * [Fix] Relax alignment requirements * [Fix] Send DMARC reports compressed * [Fix] Try to fix leak in dmarc module * [Fix] Try to plug memory leak in metric exporter * [Project] Convert rspamadm subcommands to Lua * [WebUI] Display smtp sender/recipient in history * [WebUI] Fix elements disabling in "Symbols" tab * [WebUI] Limit recipients list in history column to 3 * [WebUI] Match envelope and mime addresses following in arbitrary order * [WebUI] Update column header * [WebUI] Wrap addresses in history Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
341222861d
|
Release 1.7.5
* [Conf] Add MSBL proposed return codes * [Conf] Add additional groups for policies * [CritFix] Do not use volatile Lua strings as UCL keys * [Feature] Add ability to add fuzzy hashes to headers * [Feature] Add function to extract most meaningful urls * [Feature] Add rule to block mixed text and encrypted parts * [Feature] Allow multiple groups for symbols * [Feature] Allow to disable lua squeezing logic * [Feature] Allow to get multipart children in Lua * [Feature] Allow to insert multiple headers from milter headers * [Feature] Allow to print scores in subject and further extensions * [Feature] Be more error-prone in squeezed rules * [Feature] Support multiple return codes in emails module * [Feature] Use EMA for calculating averages * [Feature] Use common jit cache for all regexps * [Feature] support for CommuniGate Pro self-generated messages * [Fix] Allow to have multiple values for headers as arrays * [Fix] Do not open sockets for disabled workers * [Fix] Fix AuthservId * [Fix] Fix base64 folding in Lua API * [Fix] Fix build on non-x86 platforms * [Fix] Fix cached maps logic * [Fix] Fix compatibility with old maps query logic * [Fix] Fix crash if skip_map is used * [Fix] Fix importing static maps from UCL * [Fix] Fix parsing of unix sockets * [Fix] Fix raw_mime regexp on HTML part with no text content * [Fix] Fix tables logging * [Fix] Fix vertical tab handling in libucl * [Fix] Try to fix frequency counters * [Fix] Use better sharding for ip_score * [Fix] Use multiple results from SURBL DNS reply * [Fix] When doing AV scan select a different server for retransmit Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
72b3e29432
|
Release 1.7.4
* [Conf] Major stock config updates:
- Workers are now specified in a new format worker "type" { ... }
- Enable fuzzy worker to simplify local fuzzy storages configuration
- Bind all workers to localhost by default to avoid security flaws
* [Conf] Make more sane fuzzy_check default settings
* [CritFix] Fix ucl escape for bad symbols
* [Feature] Add failure symbol for AV module
* [Feature] Add lazy expiration mode for new classifier schema
* [Feature] Add preliminary version of maps stats plugin
* [Feature] Allow to block fuzzy requests from specific networks
* [Feature] Allow to change `expire` of live statistics
* [Feature] Distinguish AV failure from clean result
* [Feature] Further improvements of language detector by using khash
* [Feature] Further optimization of the lang_detection
* [Feature] Implement cluster-aware bayes expiry
* [Feature] Implement exclude patterns in rspamc
* [Feature] Implement glob maps in addition to regexp maps
* [Feature] Implement map statistics function for lua API
* [Feature] Implement stop symbols for Clickhouse collection
* [Feature] Support recipients separated by commas
* [Feature] Try harder to upload scripts to the Redis server
* [Feature] Upgrade t1ha distribution
* [Feature] use_domain_sign_inbound
* [Feature] Use scores from maps if `symbols_set` is not defined
* [Fix] Add resolving version of radix map helper
* [Fix] Check URL before adding implicit prefix
* [Fix] Do not check pid/state when using PRNG
* [Fix] Fix CentOS logrotate script for systemd
* [Fix] Fix slash + dot in urls
* [Fix] Fix systemd version of the logrotate script
* [Fix] Propagate key when import implicit array from Lua
* [Fix] Strip spaces from map keys and values
* [Fix] Try to fix a specific case when processing milter protocol
* [Fix] Try to fix crash when a tcp connection cannot be set
* [Fix] Typo use_domain_local --> use_domain_sign_local
* [Fix] Various fixes to once_received module
* [Project] Store hits counters for map elements
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
||
|
Vsevolod Stakhov
|
4f481f93bc
|
Release 1.7.3
* [CritFix] Plug bad memory leak in protocol reply * [Feature] Add avx2 codec for base64 * [Feature] Add method to receive all URL flags from Lua API * [Feature] Allow to fold headers on stop characters * [Feature] Allow to set lua_cpath from options * [Feature] Allow to specify custom rejection message in milter * [Feature] Deal with unnormalised Unicode obfuscation * [Feature] Do not detect language twice for relative parts * [Feature] Implement oversigning feature * [Feature] Implement silent logging level to minimize noise in logs * [Feature] Improve URL_IN_SUBJECT rule * [Feature] Use hashing to reduce redis attack surface * [Fix] Add oversigning for the most important headers * [Fix] add 'rewrite subject' to History dropdown * [Fix] Another fix in folding algorithm * [Fix] Do not call multimap addr for parts of addr if filter is presented * [Fix] Do not clean hostname on generic reset * [Fix] Do not create pid file in no-fork mode * [Fix] Fix fold_after case to preserve multiple spaces * [Fix] Fix folding and folding tests * [Fix] Fix hostname usage in milter mode * [Fix] Fix lua RSA verify and its tests * [Fix] Fix metadata exporter send_mail backend (#2124) * [Fix] Fix processing of '\v' in libucl * [Fix] Fix shemaless URLs detection * [Fix] Fix support of multiple headers in sign_header * [Fix] Fix usage of util.parse_mail_address * [Fix] Fix weights of dynamic squeezed rules * [Fix] Leak from bucket before checking the burst * [Fix] Stop using own localtime as DST could be messy in many cases * [Fix] Treat unnormalised URLs as obscured * [Rework] Restore leaky bucket model in ratelimit plugin * [WebUI] Add messages total to throughput summary * [WebUI] Add symbols order selector to history * [WebUI] Config: Load list on demand * [WebUI] Fix modalBody for maps that appear more than once * [WebUI] History: Fix Tooltips on paging, filtering and sorting * [WebUI] Remove a previously-attached event handler * [WebUI] Update D3 to v5.0.0 and jQuery to v3.3.1 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
8073aece67
|
Release 1.7.2
* [Feature] Store emails in Clickhouse * [Feature] Support single quotes in config * [Feature] Use templates when publishing CH schema * [Feature] Improve Docker image * [Fix] Add rounding when printing a lot of FP variables * [Fix] Allow to disable certain actions by assigning null to them * [Fix] Disable results caching * [Fix] Fix disabling of squeezed symbols * [Fix] Fix scan time set * [Fix] Rework logic of actions setting * [Fix] Try to fix various Lua stack issues * [WebUI] Add link tag for favicon.ico * [WebUI] Display hostname:port/path in the page title Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
3ba34325ab
|
Release 1.7.1
* [CritFix] Fix lowercase comparison * [CritFix] Timezone defines seconds WEST UTC not East * [Feature] Add filename to log format * [Feature] Add lua rules squeezing * [Feature] Add related symbols analysis to rspamd_stats * [Feature] Remove upstream `X-Spam: Yes` header by default * [Feature] rspamd_stats: Output progress info on STDERR * [Feature] Whitelist for emails module * [Fix] Do not allow dependencies on self * [Fix] Do not cache metric result * [Fix] Do not trust all issuers as a client certificate * [Fix] Fix dependencies in lua squeeze * [Fix] Fix enabling/disabling squeezed rules * [Fix] Fix enabling/disabling symbols * [Fix] Fix external dependencies * [Fix] Fix processing of a single compressed file * [Fix] Fix some typos * [Fix] Fix various modules in case of empty message * [Fix] Handle callbacks that returns table of options * [Fix] Improve cached action interaction * [Fix] Make dynamic conf more NaN aware * [Fix] Never hide actions from WebUI `configuration` tab * [Project] Implementation of Lua rules squeezing Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
720ae30a31
|
Release 1.7.0
* [Conf] Add bayes_expiry as explicit module * [Conf] Adjust names and weights for neural network plugin * [Conf] Change updates url * [Conf] Default statistics is stored in Redis now * [Conf] Disable fann_redis module by default * [Conf] Fix default elastic configuration * [Conf] Fix double quote position * [Conf] Massive config rework for new structure of symbols and scores * [Conf] Rename Rambler BLs as they are now Rspamd's ones * [Conf] Use dedicated rspamd.com subdomains * [Conf] Use more data from rspamd.com fuzzy storage * [CritFix] Add sanity guards for badly broken HTML * [CritFix] Another errors path handling fix * [CritFix] Another portion of tokenization fixes * [CritFix] Do not send reject messages after set reply * [CritFix] Fix ARC chain verification * [CritFix] Fix crash in milter errors handler * [CritFix] Fix memory leak in spf caching logic * [CritFix] Fix milter commands pipelining * [CritFix] Fix newlines detection * [CritFix] Fix semicolons parsing in the content type * [CritFix] Plug memory leak in zstd protocol compression * [Feature] Add ability to match score in force_actions module * [Feature] Add aes-rng PRF to libottery * [Feature] Add 'composites' debug module * [Feature] Add concept of experimental modules * [Feature] Add DKIM trace symbol * [Feature] Add EBL to the default config * [Feature] Add expected ip check for emails plugin * [Feature] Add framework to manage Redis scripts * [Feature] Add framing for the new reputation generic plugin * [Feature] Add function to show plugins stat * [Feature] Add gzip compression support for clickhouse module * [Feature] Add gzip compression support for rspamd controller * [Feature] Add gzip support when sending lua http requests * [Feature] Add json output for rspamd_stats * [Feature] Add method to do a synchronous Redis connection * [Feature] Add method to get all content-type attributes in Lua * [Feature] Add `-m` flag to configdump to show modules states * [Feature] Add mime types to extensions map * [Feature] Add more features to rescore utility * [Feature] Add more gtube like patterns to test other spam actions * [Feature] Add more metafunctions, improve logging * [Feature] Add more text attributes * [Feature] Add new configwizard command to rspamadm * [Feature] Add new tooling for stats conversation * [Feature] Add old groups migration tool * [Feature] Add plugins state variable * [Feature] Add preliminary ecdsa keys support in DKIM * [Feature] Add preliminary support of idempotent symbols * [Feature] Add Redis server wizard * [Feature] Add routine to convert old style stats to a new one * [Feature] Add some sanity checks for actions and controller * [Feature] Add statistic convertation module to configwizard * [Feature] Add sugestions logic to mempool allocator * [Feature] Add support of config transform in Lua * [Feature] Add timeout to rspamc when doing corpus test * [Feature] Add tooling to convert bayes schemas * [Feature] Add torch conditional to configuration * [Feature] Add torch-decisiontree package * [Feature] Add torch-optim contrib package * [Feature] Add TTL autodetection * [Feature] Add urls reputation to the reputation framework * [Feature] Allow floating and negative values in expressions limits * [Feature] Allow multiple CTs in full extensions map * [Feature] Allow multiple fann rules * [Feature] Allow randomly select User-Agent from a list * [Feature] Allow rspamadm commands to export methods in Lua * [Feature] Allow rule specific min_bytes in fuzzy check * [Feature] Allow to adjust symbols scores from Lua * [Feature] Allow to attach stat signature to messages * [Feature] Allow to change SMTP from via milter headers * [Feature] Allow to configure monitored * [Feature] Allow to create directories in Lua API * [Feature] Allow to disable torch and skip train samples for ANN * [Feature] Allow to discard messages dynamically * [Feature] Allow to enable/disable languages from the detector * [Feature] Allow to generate DKIM keys from rspamadm API * [Feature] Allow to get CPU flags from Lua * [Feature] Allow to have high precision timestamps in logs * [Feature] Allow to insert headers into specific position * [Feature] Allow to limit redirector requests per task * [Feature] Allow to load and use dynamic ANNs with torch * [Feature] Allow to quarantine rejected messages using milter interface * [Feature] Allow to receive signing keys from mempool vars * [Feature] Allow to reserve elements in libucl * [Feature] Allow to reuse signal handlers chains * [Feature] Allow to set custom mempool variables from settings * [Feature] Allow to set headers from settings * [Feature] Allow to set Settings-Id for all connections * [Feature] Allow to skip real action and add a header instead * [Feature] Allow to skip specific hashes in fuzzy storage * [Feature] Allow to spawn asynchronous processes from Lua * [Feature] Allow to specify number of threads for ANN learning * [Feature] Allow to use global lua maps in settings * [Feature] Allow to use postfilters in composites * [Feature] Allow to verify signatures from HTTP headers in maps * [Feature] Antivirus: ordered pattern matches * [Feature] Authentication-Results: support hiding usernames * [Feature] Automatically create tables in clickhouse * [Feature] Catch next-to-last bad extension * [Feature] Check cached maps more frequently * [Feature] Check groups sanity * [Feature] Deal with obscured URLs with @ symbols * [Feature] Enhance task:store_in_file method * [Feature] Export password encryption routines to Redis * [Feature] Filter nan and inf when adding scores * [Feature] Finalize 7zip files support * [Feature] Further improvements in language detection * [Feature] Further improvements in language detection algorithm * [Feature] Generic key name expansion for Redis keys * [Feature] Hash whitelist for fuzzy_check * [Feature] Implement bayes signatures storage * [Feature] Implement buckets for Redis backend * [Feature] Implement DKIM reputation adjustments * [Feature] Implement forked workers children monitoring * [Feature] Implement headers flags in mime parser * [Feature] Implement l1/l2 regularization against the current weights * [Feature] Implement manual ANN train mode * [Feature] Implement per-user ANN support * [Feature] Implement torch based ANN learning * [Feature] Implement upstreams logic for clickhouse exporter * [Feature] Import torch to Rspamd... * [Feature] Improve allocation policy when interacting with Lua * [Feature] Improve Lua/C interaction in history_redis * [Feature] Improve multiple fuzzy results combining * [Feature] Improve parsing of DKIM keys: parse algorithm * [Feature] Improve subprocesses termination handle * [Feature] Improve symbol type parsing in Lua API * [Feature] Metadata Exporter: e-Mail Alerts: support multiple recipients; alerting senders/recipients/users (#1600) * [Feature] Milter headers: support adding/removing arbitrary headers from config * [Feature] More metatokens * [Feature] Multimap: checking of symbol options * [Feature] Multimap: template URL filter * [Feature] New bayes expiry plugin * [Feature] Periodically save rspamd stats to disk * [Feature] Preliminary import of the elasticsearch module * [Feature] Ratelimit: allow full addresses in whitelisted_rcpts * [Feature] Ratelimit: support fetching limits from Redis * [Feature] RBL: received: filtering by position & flags * [Feature] Read global maps for lua * [Feature] Redis settings: support checking multiple keys * [Feature] Rework fann plugin to be a normal post-filter * [Feature] Rework logging configuration for rspamadm case * [Feature] Rework short hashes generation to avoid FP * [Feature] Save real ucl types when exporting to Lua * [Feature] Set TCP_NODELAY for milter sockets * [Feature] Setup DKIM signing from configwizard * [Feature] Skip certain symbols from ANN classify * [Feature] Store plugins state * [Feature] Support etag for HTTP maps * [Feature] Support Expires header when using HTTP maps * [Feature] Support sending given header multiple times in lua_http * [Feature] Support sha512 in DKIM signatures * [Feature] Try to detect HTML messages better * [Feature] Use array instead of queue to reduce memory fragmentation * [Feature] Use controller port by default when connecting to local IP * [Feature] Use rdtsc where possible * [Fix] Actively load skip hashes map in fuzzy storage * [Fix] Add another workaround to display history properly * [Fix] Add definition for old glib compatibility method * [Fix] Add missing rspamadm control options to help * [Fix] Add workaround for IPv6 in sendmail * [Fix] Add workaround for system with non-XSI compatible tzset * [Fix] Allow oversigning in DKIM signatures * [Fix] Allow to check negative scores in force_actions * [Fix] Allow to have negative actions limits * [Fix] Allow to set any layers number for fann rules * [Fix] Another fix for rdtcs * [Fix] Another fix to lua xmlrpc * [Fix] Another try to deal with #1998 * [Fix] Another try to fix #1998 * [Fix] Another try to fix threading in torch * [Fix] Apply language detection when adding fuzzy hashes * [Fix] ARC: Fix Lua 5.3 compatibility; timestamp should be integer * [Fix] Authentication Results: Fix SPF smtp.mail_from * [Fix] Auth-Results: Multiple DKIM signatures * [Fix] Avoid changing content-transfer-encoding header's value * [Fix] Better handling of the legacy protocol * [Fix] Check decoded headers sanity (e.g. by excluding \0) * [Fix] Check for magic when checking for an archive * [Fix] Cleanup mess with groups * [Fix] Clickhouse: Insertion in the symbols table * [Fix] Crash in URL processing * [Fix] Deal with another case when processing exceptions * [Fix] Deal with deeply nested messages more aggressively * [Fix] Deal with nan and inf encoding in json/ucl * [Fix] Deal with non-key arguments in lua_redis.exec_script * [Fix] Deal with unknown weight * [Fix] Deal with URLs with no slashes after protocol * [Fix] Deal with URLs wrapped in [] in text parts * [Fix] Deal with zero scores symbols * [Fix] Default monitoring domain for surbl plugin * [Fix] Delay upstream re-resolving when one upstream is defined * [Fix] Detection of maillist optimized and fixed * [Fix] DKIM signing: allow for auth_only to be false * [Fix] DMARC: require report_settings for sending reports only * [Fix] Do not allow garbadge when checking url domain * [Fix] Do not cache SPF records with PTR elements * [Fix] Do not constantly re-resolve failed upstreams with a single element * [Fix] Do not crash if no words defined * [Fix] Do not crash on empty subtype * [Fix] Do not expose spamtrap messages to SMTP reply * [Fix] Do not fail rbl plugin when there are no received or emails * [Fix] Do not ignore short words * [Fix] Do not include idempotent/nostat symbols to checksum * [Fix] Do not override groups when converting metrics * [Fix] Do not override unix socket group when group comes before owner * [Fix] Do not skip the last character * [Fix] Do not spawn too many workers by default * [Fix] Do not stop monitored on dns errors * [Fix] Do not stop parsing headers on bad IP header * [Fix] Do not strip last character in the last word * [Fix] Do not treat script content as text * [Fix] Do not try to connect to non-supported addresses * [Fix] Do not try to dereference last character * [Fix] Do not try to sign unknown domains * [Fix] Don't use whitelist/greylist maps as regexp, but as map * [Fix] Erase unknown HTML entities * [Fix] Exim Received header protocol parsing * [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false * [Fix] Fix a lot of FP in chartable in mixed languages * [Fix] Fix ANN checks * [Fix] Fix ANN loading logic * [Fix] Fix another tokenization issue * [Fix] Fix autolearn parameters reading * [Fix] Fix bad archive characters stripping * [Fix] Fix bad extension check * [Fix] Fix bayes schema conversion * [Fix] Fix blacklists and DMARC in whitelist * [Fix] Fix brain-damaged torch build system * [Fix] Fix build on FreeBSD * [Fix] Fix clickhouse exporter * [Fix] Fix clickhouse schema * [Fix] Fix comparision * [Fix] Fix composites processing * [Fix] Fix connecting to a unix socket in rspamadm statconvert * [Fix] Fix couple of warnings * [Fix] Fix crashes in the rspamd_control path * [Fix] Fix deletion from hash * [Fix] Fix DKIM forgeries via multiple headers * [Fix] FIx dynamic conf plugin * [Fix] Fix emails detection * [Fix] Fix empty headers simple canonicalization * [Fix] Fix empty threshold check in greylisting module * [Fix] Fix encrypted legacy reply in fuzzy storage * [Fix] Fix enormous scores for R_WHITE_ON_WHITE * [Fix] Fix exceptions list in surbl * [Fix] Fix *_EXCESS_BASE64 rules * [Fix] Fix expire rounding * [Fix] Fix extra hits in PCRE mode for regular expressions * [Fix] Fix format strings * [Fix] Fix get_content method * [Fix] Fix groups override when defining symbols * [Fix] Fix learned count in new schema * [Fix] Fix learn errors propagation * [Fix] Fix loading of per-user redis backend for statistics * [Fix] Fix logging buffer corruption in case of repeated messages * [Fix] Fix lua cached elements invalidation * [Fix] Fix merging of the implicit arrays * [Fix] Fix mime_types scoring * [Fix] Fix multiple headers in DKIM headers list * [Fix] Fix null callee case in clang plugin * [Fix] Fix obscured url in format user@@example.com * [Fix] Fix parsing of the per-user script * [Fix] Fix priorities in rspamd_update, disable rules execution * [Fix] Fix processing of closed tags * [Fix] Fix processing of idempotent rules when autolearn fails * [Fix] Fix processing of multipart parts with no headers * [Fix] Fix processing of skip-hashes in fuzzy storage * [Fix] Fix PTR processing in SPF * [Fix] Fix pushing country to clickhouse asn table * [Fix] Fix random forests module * [Fix] Fix real IP parsing for some strange Exim received * [Fix] Fix Redis timeout setup * [Fix] Fix reload crash when hyperscan is enabled * [Fix] Fix reusing of redis connection after exec * [Fix] Fix sanity checks on macro value * [Fix] Fix setting of path and cpath for Lua * [Fix] Fix setting of signals when spawning a thread * [Fix] Fix text splitting: stack overflow (too many captures) * [Fix] Fix ticks processing * [Fix] Fix upstream addrs updating * [Fix] Fix urls/emails distinguishing found in queries * [Fix] Fix user settings check * [Fix] Fix variable increment * [Fix] Fix various issues in stat_convert * [Fix] F-PROT Antivirus infection string for all known occurences * [Fix] F-PROT Antivirus: only check return code to determine infection * [Fix] Further fixes around floating point expressions * [Fix] Further fixes to ANN module * [Fix] Further fixes to rescore tool * [Fix] Further fixes to support ES 6 * [Fix] Further tokenization fixes * [Fix] Greylisting set phase is not idempotent * [Fix] Handle proxy copy errors * [Fix] Header checks: Fix get_raw_header method * [Fix] Header checks: REPLYTO_UNPARSEABLE rule * [Fix] Kill spawned processes on termination * [Fix] Load skip map from all processes as shared cache is unavailable * [Fix] Lowercase HTTP headers to make them searchable from Lua * [Fix] Lowercase words * [Fix] Lua_http: freeing * [Fix] Lua: lpeg to be loaded with rspamd_lua_add_preload, to avoid "rspamd_config_read: rcl parse error: cannot init lua file […] module 'lpeg' not found" * [Fix] Map absence is not an error * [Fix] Metadata exporter: check IP sanity * [Fix] Milter headers: custom headers: removing headers * [Fix] Milter headers: skip_local / skip_authenticated settings * [Fix] Milter headers: X-Spamd-Result header if X-Virus ran first * [Fix] mime_types: fix next-to-last extension length check * [Fix] More hacks to deal with old configs * [Fix] Move composites second pass to the dedicated stage * [Fix] Multimap: received: filtering of artificial header * [Fix] Multiple fixes in torch based ANN plugins * [Fix] Once more (#1879) fix bad extension check * [Fix] Optimize rspamd_fstring_t reallocations * [Fix] options.local_networks setting * [Fix] Parse HREF urls without explicit prefix * [Fix] Plan new event on HTTP errors * [Fix] Plug another possible memory leak * [Fix] Plug memory leak * [Fix] Plug memory leak in lua_tcp * [Fix] Plug memory leak when setting email addresses from Lua * [Fix] Propagate learn/stat errors more precisely * [Fix] Ratelimit: fix whitelisted_rcpts matching * [Fix] Ratelimit: lowercase email addresses * [Fix] RBL: received: deal with missing data (#1965) * [Fix] Rebalance and slightly rework MX check plugin * [Fix] Redis key expansion: EVAL: deal with strings * [Fix] Redis script loading in DMARC; URL tags; URL reputation * [Fix] Reject invalid bh for DKIM signatures earlier * [Fix] Relax pem signature detection * [Fix] Relax unicode properties requirements for chartable module * [Fix] Remove extra noise from dkim and arc signing * [Fix] Remove hop-by-hop headers in proxy * [Fix] Remove incorrect method `task:set_metric_subject` * [Fix] Replace space like characters in headers with plain space * [Fix] Restore old style ratelimits support * [Fix] Rework elasticsearch plugin * [Fix] Rewriting subjects via force actions module * [Fix] RPM postinstall * [Fix] Sanitize IP in history redis * [Fix] Select the correct signature when doing simple canon * [Fix] Set CLOEXEC flag on files opened * [Fix] Setting check_local / check_authed in plugins (#1954) * [Fix] Settings: avoid checking invalid IP (#1981) * [Fix] Settings: header: deal with multiple settings (#1988) * [Fix] Skip checks if both extensions are not bad * [Fix] Skip nostat tokens when get number of tokens * [Fix] Some more fixes towards emails detection * [Fix] SpamAssassin: Fail check_freemail_header if regexp didn't match * [Fix] Stop using of g_slice... * [Fix] Switch rspamadm logging to message level * [Fix] Symbol 'FANNR_SPAM' has its score defined.. * [Fix] Table parameter for rspamd_config:add_doc() * [Fix] Treat 'rewrite subject' as spam action * [Fix] Try harder in passing IPv6 addresses * [Fix] Try harder to find rfc822 notifications * [Fix] Try harder to find urls * [Fix] Use decoded values when parsing mime addresses * [Fix] Use full URL when making an HTTP request * [Fix] Use greylisting threshold in greylisting module * [Fix] Use n_words attribute from ngramms * [Fix] Use raw urls when sending requests to redirector * [Fix] Use the right boolean operator on error check * [Fix] Use weight from map for fuzzy scoring * [Fix] Various fixes to elastic plugin * [Fix] Various fixes to fann_redis instantiation * [Fix] Various improvements in language detection * [Fix] Virus infection string for F-PROT Antivirus * [Fix] Virus infetction string for F-PROT Antivirus * [Fix] WebUI: use relative path for savemap (#1943) * [Fix] WHITE_ON_WHITE: Ensure score is matched to part that fired the rule * [Fix] Write configuration changes as UCL config * [Project] Add detection logic for words * [Project] Add fast debug logging infrastructure * [Project] Add more flags to languages * [Project] Add n-gramms data files * [Project] Add ngramms frequencies detector * [Project] Add random words selection logic * [Project] Add unigramms to language detection as well * [Project] Convert all C modules to fast debug infrastructure * [Project] Detect some languages based on unicode script * [Project] Enable fast debug lookup for some modules * [Project] Enable language detector init in scanner workers * [Project] Further improvements to language detector * [Project] Implement logic of ngramms application * [Project] Improve weighting in lang_detection * [Project] Initialize language detector * [Project] Preliminary version of ngramms based language detector * [Project] Preliminary version of the new stat_convert * [Project] Remove old language detector * [Project] Rework language detection ngramms structure * [Project] Start language detection project * [Project] Start rework of language detection to improve quality * [Project] Use fast debug logging check * [Rework] Add frame for new reputation based IP score module * [Rework] Continue stat_convert rework task * [Rework] Implement new version of fuzzy replies * [Rework] Improve readability of xmlrpc API * [Rework] Kill metrics!11 * [Rework] Ratelimit module * [Rework] Rename fann_redis to neural plugin * [Rework] Reorganize mime_types module * [Rework] Rework rescore utility * [Rework] Rewrite model and learning logic for rescore * [Rework] Run post-loads when all initialization is completed * [Rework] Simplify lua path initialization * [Rework] Start major stat_convert rework * [Rework] Start mempool fragmentation reduce project * [Rework] Start moving of fann redis to torch * [Rework] Stop embedding rspamadm scripts into C * [Rework] Use floating point arithmetics in Rspamd expressions * [Rework] Use frequencies distribution in language detector * [Rules] Penalise R_BAD_CTE_7BIT for utf8 messages * [WebUI] Compact graph selectors * [WebUI] Escape strings inside HTML in history * [WebUI] Fix message count in throughput summary (#1724) * [WebUI] Fix NaNs display on Throughput graph * [WebUI] Migrate widgets to D3 v4 * [WebUI] Restore passwordless login support (#2003) * [WebUI] Show symbol descriptions as tooltips in history * [WebUI] Stop using commas in pie chart tooltips * [WebUI] Update D3 and jQuery * [WebUI] Update D3Evolution 1.0.0 -> 1.1.0 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
8fb0ef19f4 |
Release 1.6.6
* [CritFix] Add sanity guards for badly broken HTML * [CritFix] Another errors path handling fix * [CritFix] Fix ARC chain verification * [CritFix] Fix crash in milter errors handler * [Feature] Allow to insert headers into specific position * [Feature] Allow to receive signing keys from mempool vars * [Feature] Authentication-Results: support hiding usernames * [Fix] Another try to deal with #1998 * [Fix] Another try to fix #1998 * [Fix] Better handling of the legacy protocol * [Fix] Check decoded headers sanity (e.g. by excluding \0) * [Fix] Deal with nan and inf encoding in json/ucl * [Fix] Deal with URLs wrapped in [] in text parts * [Fix] DKIM signing: allow for auth_only to be false * [Fix] Do not crash on empty subtype * [Fix] Do not fail rbl plugin when there are no received or emails * [Fix] Do not skip the last character * [Fix] Do not try to dereference last character * [Fix] Do not try to sign unknown domains * [Fix] Exim Received header protocol parsing * [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false * [Fix] Fix bad archive characters stripping * [Fix] Fix comparision * [Fix] Fix connecting to a unix socket in rspamadm statconvert * [Fix] Fix empty headers simple canonicalization * [Fix] Fix extra hits in PCRE mode for regular expressions * [Fix] Fix parsing of the per-user script * [Fix] Fix processing of skip-hashes in fuzzy storage * [Fix] Fix Redis timeout setup * [Fix] Fix sanity checks on macro value * [Fix] Fix text splitting: stack overflow (too many captures) * [Fix] Fix urls/emails distinguishing found in queries * [Fix] F-PROT Antivirus: only check return code to determine infection * [Fix] Metadata exporter: check IP sanity * [Fix] Multimap: received: filtering of artificial header * [Fix] Plan new event on HTTP errors * [Fix] Plug another possible memory leak * [Fix] Remove hop-by-hop headers in proxy * [Fix] Sanitize IP in history redis * [Fix] Setting check_local / check_authed in plugins (#1954) * [Fix] Settings: avoid checking invalid IP (#1981) * [Fix] Try harder in passing IPv6 addresses * [Fix] WebUI: use relative path for savemap (#1943) * [WebUI] Fix message count in throughput summary (#1724) * [WebUI] Fix NaNs display on Throughput graph * [WebUI] Restore passwordless login support (#2003) Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Viktor Szépe
|
b3e626e260
|
Typo: comparision | ||
|
Vsevolod Stakhov
|
111dfde1ad | [Minor] Sync ChangeLog | ||
|
Vsevolod Stakhov
|
fa9b2fc2d8 |
Release 1.6.3
* [CritFix] Fix semicolons parsing in the content type * [Feature] Add EBL to the default config * [Feature] Allow to configure monitored * [Feature] Allow to skip specific hashes in fuzzy storage * [Feature] Multimap: checking of symbol options * [Feature] Redis settings: support checking multiple keys * [Fix] ARC: Fix Lua 5.3 compatibility; timestamp should be integer * [Fix] Avoid changing content-transfer-encoding header's value * [Fix] Don't use whitelist/greylist maps as regexp, but as map * [Fix] Fix get_content method * [Fix] Header checks: Fix get_raw_header method * [Fix] Header checks: REPLYTO_UNPARSEABLE rule * [Fix] Lua_http: freeing * [Fix] Milter headers: custom headers: removing headers * [Fix] Parse HREF urls without explicit prefix * [Fix] WHITE_ON_WHITE: Ensure score is matched to part that fired the rule * [WebUI] Escape strings inside HTML in history Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
e5cefe177e
|
Release 1.6.2
* [Conf] Remove Rambler email bl for now * [Conf] Switch RAMBLER_URIBL to a locally managed source * [CritFix] Switch from ragel to C for Content-Type parsing * [Feature] Add `-e` option for lua_repl * [Feature] Add per-domain emails normalisation rules * [Feature] Add sessions cache to debug dangling sessions * [Feature] Add short_text_direct_hash for fuzzy check module * [Feature] Add text_part:get_stats function * [Feature] Allow to add custom processing script for surbl * [Feature] Allow to check reply-to email * [Feature] Allow to customize spam header, remove existing spam headers * [Feature] Allow to disable specific workers in the config * [Feature] Allow to discard messages instead of rejection * [Feature] Allow to specify custom delimiter in emails plugin * [Feature] Allow to specify custom User-Agent for rspamc * [Feature] Allow to store symbols data in Clickhouse * [Feature] Allow to use HTTPS when connecting to Clickhouse * [Feature] Enable sessions cache tracking for milter connections * [Feature] Implement per-line mode in lua_repl (like `perl -p`) * [Feature] Implement rdns-curve plugin based on rspamd cryptobox * [Feature] Improve maps cached data lifetime * [Feature] Improve maps checking frequency * [Feature] Improve monitored timeouts logic * [Feature] milter_headers: add `extended_headers_rcpt` option * [Feature] Milter headers: Add X-Spam-Flag to rmilter-compatibility headers * [Feature] Milter headers: remove-header routine * [Feature] Multimap: received filters for extracting TLDs from hostnames * [Feature] Normalize email aliases in emails module * [Feature] Re-add rambler email bl (as hashed list) * [Feature] Reload file maps more frequently * [Feature] Rework newlines strip parser one more time * [Feature] Skip updates for messages scanned via controller * [Feature] Split long DKIM public keys * [Feature] Store more data when stripping newlines * [Feature] Support SPF macros transformations * [Feature] Support suppressing DMARC reports for some domains * [Fix] Add missing `break` statement * [Fix] Allow modifiers in SPF macros * [Fix] DKIM sign tools: edge-cases around use_esld * [Fix] Do not cache SPF records with macros * [Fix] Do not overwrite score when setting pre-action * [Fix] Fix comparision logic * [Fix] Fix DKIM base64 folding for milter flagged messages * [Fix] Fix emails module configuration * [Fix] Fix folding for arc headers when milter interface is used * [Fix] Fix gmail dots removal * [Fix] Fix rspamc detection in greylist module * [Fix] Fix some more issues with HTTP maps * [Fix] Milter sessions can live forever * [Fix] Normalize fuzzy probability better * [Fix] Plug memory leak * [Fix] RBL: Fixed hashed email address lookups * [Fix] Try to deal with brain-damaged milter behaviour * [Fix] Use `\n` to fold headers for milter * [Rework] Allow to use custom callback for monitored checks * [Rework] Further steps towards one process monitoring * [Rework] Send health checks from a single worker * [WebUI] Round-up throughput summary values Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Alexander Moisseev
|
bb5a9f8e45 |
[Minor] Massive spelling correction
by a bot https://github.com/ka7/misspell_fixer |
||
|
Vsevolod Stakhov
|
8895fc6fab
|
Release 1.6.1
* [Fix] Allow to init resolver without rspamd_config * [Fix] Do not crash when resolver failed to initialize * [Fix] Fix abstract context layout * [Fix] Fix CGP helper reply parsing * [Fix] Fix crashes when socket write errors occur * [Fix] Fix parsing IPv6 nameservers in resolv.conf * [Fix] Milter: Don't defer on "greylist" action Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
9c0fd5e439
|
Release 1.6.0
* [Conf] Add rspamd_proxy to the default configuration set
* [Conf] Add sample arc module config
* [Conf] Do away with systemd specifics completely
* [Conf] Increase min_bytes to avoid FP
* [Conf] Remove ratelimits from default configuration
* [CritFix] Fix accepting on IPv6 sockets
* [CritFix] Fix corruption when multiple fuzzy are defined
* [CritFix] Fix learn condition in fuzzy check
* [CritFix] Fix memory leak in fuzzy check
* [CritFix] Fix memory leak in maps scheduling
* [CritFix] Paese the last character in DKIM signature correctly
* [CritFix] Zero fill sockaddr_un
* [Feature] Add ability to add doc strings by example
* [Feature] Add API to verify DKIM (and ARC) signatures
* [Feature] Add compression/decompression to proxy
* [Feature] Add count to url structure
* [Feature] Add initial support of the new protocol reply
* [Feature] Add Lua plugin spamtrap
* [Feature] Add `monitored_address` for rbls
* [Feature] Add new schema for bayes tokens
* [Feature] Add preliminary ARC support to dkim code
* [Feature] Add preliminary support of ARC signing
* [Feature] Add rules to detect bad 8bit characters in From and To
* [Feature] Add scanning support for milter protocol
* [Feature] Add support for bidirectional symbols in rspamd_stats
* [Feature] Add support for static maps
* [Feature] Add support of maps with multiple regexps matches
* [Feature] Add `text_multiplier` param
* [Feature] Add the preliminary ARC plugin
* [Feature] Add top redirector targets rank
* [Feature] Allow async events to be registered from LUA rules
* [Feature] Allow storing bayes tokens in Redis
* [Feature] Allow to exclude specific domains from mx check
* [Feature] Allow to have a stack of watcher finalisers
* [Feature] Allow to pass hostname to `-i` flag in Rspamc
* [Feature] Allow to set custom user agent in url redirector
* [Feature] Allow to use custom callback when parsing resolv.conf
* [Feature] Allow to use domain from authenticated user
* [Feature] Bayes expiry plugin
* [Feature] Check dkim sign keys for modifications
* [Feature] DKIM signing: sign_networks/local address specific use_domain settings
* [Feature] DMARC: Support excluding domains from sampling
* [Feature] Expire processing items for URL redirector aggressively
* [Feature] Fix surbl monitored for IP lists, add `monitored_domain` option
* [Feature] Implement caching for dkim body hashes
* [Feature] Implement milter protocol scan reply
* [Feature] Improve omograph phishing detection
* [Feature] Initial support of self-scan in Rspamd proxy
* [Feature] Keep track of headers in milter interface
* [Feature] Milter headers: better controls for local/authenticated
* [Feature] Multimap: email:domain:tld filter
* [Feature] Preliminary DMARC reporting implementation
* [Feature] Reuse stemmers in the cache
* [Feature] Rework confighelp to load Lua plugins
* [Feature] Rework hfilter to use hyperscan if possible
* [Feature] Rework lua RSA API
* [Feature] Rmilter_headers: approximate rmilter's extended_spam_headers
* [Feature] Start integration of milter support in proxy
* [Feature] Store average words length and short words count
* [Feature] Store hash of headers order and names
* [Feature] Support MTA name header
* [Feature] Support multiple types of dkim signing in Lua
* [Feature] Support numeric arguments for Redis requests
* [Feature] Use headers hash in bayes metatokens
* [Feature] Use normal resolv.conf rules of rotation in Rspamd
* [Feature] Use version 2 proto for checking messages
* [Fix] Allow to follow symlinks when safe
* [Fix] Append MX name for authentication results as required
* [Fix] Change default text multiplier from 0.5 to 2.0
* [Fix] Check min_bytes for images as well
* [Fix] Deal with 7bit charsets properly
* [Fix] Deal with 8bit characters in email addresses
* [Fix] Deal with unpaired <a> tags
* [Fix] Detect confighelp in plugins initialisation
* [Fix] Disable certain checks for utf spoof detection
* [Fix] DKIM Signing: avoid nil index when From header is missing
* [Fix] Do not add exact hashes from different parts
* [Fix] Do not check DMARC if SPF or DKIM were not checked
* [Fix] Do not check URLs that are resolved to be redirected
* [Fix] Do not set bayes probability if we don't use it
* [Fix] Do not stop on illegal unicode points - replace them
* [Fix] Fix another race condition in arc checks
* [Fix] Fix arc count logic
* [Fix] Fix ARC signing
* [Fix] Fix brain-damaged spamc protocol for now
* [Fix] Fix calling for peak functions
* [Fix] Fix couple of issues in FORWARDED rule
* [Fix] Fix CTE propagation from parent containers to children parts
* [Fix] Fix errors processing in the controller
* [Fix] Fix format string in milter
* [Fix] Fix issues in SPF macros parsing
* [Fix] Fix logging format string
* [Fix] Fix logic of cached passwords check
* [Fix] Fix lowercasing of stemmed words
* [Fix] Fix LRU elements removal
* [Fix] Fix memory leak when accepting from unix sockets
* [Fix] Fix milter connections persistence
* [Fix] Fix objects merging in UCL
* [Fix] Fix order of operations to avoid race condition
* [Fix] Fix parsing of long regexp types
* [Fix] Fix passing data to log helper when many symbols defined
* [Fix] Fix pools management for milter session
* [Fix] Fix processing of the watchers
* [Fix] Fix queue id macro in milter
* [Fix] Fix R_BAD_CTE_7BIT rule
* [Fix] Fix Redis timeout set
* [Fix] Fix REPLYTO_UNPARSEABLE rule
* [Fix] Fix setting of email address
* [Fix] Fix some more issues about duplicated fuzzy requests
* [Fix] Fix spamc support in rspamd proxy
* [Fix] Fix syntax error in spamtrap plugin
* [Fix] Fix url counts for href urls
* [Fix] Fix url handling in the protocol
* [Fix] Multimap: Received IP filters with Redis
* [Fix] Oops, fix
|
||
|
Vsevolod Stakhov
|
a92e55249f | [Minor] Update ChangeLog from 1.5 branch | ||
|
Sebastien Badia
|
dd64721ae6
|
[Minor] debian: Update BD and refresh packaging | ||
|
Vsevolod Stakhov
|
27ea426985
|
Release 1.5.6
* [Feature] Add unigramms support in bayes * [Feature] Allow configurable sign headers for DKIM * [Feature] Allow to add unigramm metatokens from Lua * [Feature] DKIM Signing: envelope match exception for local IPs * [Feature] UCL: register parser variables from Lua * [Fix] Always try to adjust filename * [Fix] Do extra copy to ensure that original content is never touched * [Fix] Fix SPOOF_REPLYTO rule * [Fix] Ignore Rmilter added Received * [Fix] More fixes for hashed email dnsbls * [Fix] Plug memory leak in chartable module * [WebUI] Display multiple alerts at once Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
939ab1342e |
Release 1.5.5
* [CritFix] Fix classifier learning with Redis backend * [CritFix] Fix issue when parsing encoded rfc822/messages * [Feature] Add escaped version of lua_ucl import * [Feature] Add task:headers_foreach function * [Feature] Allow to process filenames from content type * [Feature] Allow to query hashed emails * [Feature] Ignore bayes with mostly metatokens or with too few text * [Feature] Probabilistically skip metatokens * [Feature] Retrieve all virus names from SAVAPI * [Feature] Rework classifiers lua metatokens * [Feature] Store headers order * [Feature] Store text tokens inside bayes tokens * [Feature] Use cached shingles keys * [Fix] Add missing score normalisation for HFILTER_URL_ONLY * [Fix] Avoid lookup in absent hash * [Fix] Check return values from Lua functions called from C * [Fix] Do not count sending and loading time in rspamc * [Fix] Escape json strings for controller rejplies from Lua * [Fix] Fix archive scans for savapi * [Fix] Fix domain_only emails RBL * [Fix] Fix ip_score map configuration * [Fix] Fix JSON output for history_redis * [Fix] Fix one character length substrings search * [Fix] Fix parsing of non-RFC compatible Exim received * [Fix] Fix parsing of options for workers with the same type * [Fix] Fix processing of small tokens vectors * [Fix] Fix rfc2047 tokenization * [Fix] Fix typo * [Fix] More fixes for inplace decoding * [Fix] Try to avoid modifications of the original data * [Fix] URL redirector: Fix call to is_redirector * [Rework] Set token data as uint64_t instead of chars array * [WebUI] Check if neighbours' history backend versions match * [WebUI] Disable phrase connectors replacement in history filtering * [WebUI] Disable phrase connectors replacement in symbols filtering * [WebUI] Do not hide messages with bad subject, just replace it with '???' * [WebUI] Fix error message * [WebUI] Fix history v2 display * [WebUI] Fix legacy history * [WebUI] history: break To address lists on commas * [WebUI] Increase default timeout to 20 seconds * [WebUI] Save some history table space Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
0839f87a25
|
Release 1.5.4
* [Conf] Add history_redis default configuration * [Feature] Add spoofed rules * [Feature] Add URL_IN_SUBJECT rule * [Feature] Allow to get task's subject * [Feature] Allow to specify maximum number of shots for symbols * [Feature] Distinguish URLs found in Subject * [Feature] Memoize LPEG grammars * [Feature] Parse else parts in SA rules * [Feature] Process subject for mixed characters * [Feature] Resolve url chains in url_redirector module * [Feature] Stat greylisted messages as greylisted not soft-rejected * [Feature] Support checking for redirector in Lua SURBL * [Feature] Support tag_exists SA function * [Feature] Work with broken rfc2047 tokens * [Fix] Check all watcher's dependencies * [Fix] Do not compile hyperscan with no SSSE3 support * [Fix] Do not crash if cannot decode qp encoded part * [Fix] Fix dependencies of DKIM when multiple signatures are found * [Fix] Fix lists in whitelist plugin * [Fix] Fix one-shot symbols weight calculations * [Fix] Fix options and shots match * [Fix] Fix order of symbol options * [Fix] Fix parsing of dot at the end of the address * [Fix] Fix parsing of lua table arguments * [Fix] Fix processing of subject words * [Fix] Fix string split memoization * [Fix] Fix templates grammar usage * [Fix] Fix various issues related to Lua stack manipulation * [Fix] Force actions: Use postfilter if we have honor_action / require_action * [Fix] Further fixes to avoid PHISHING FP * [Fix] Preserve order of options in symbols * [Fix] Rspamadm grep: deal with unusually-formatted logs * [Fix] Use hostname suffix when dealing with history * [Rework] Remove outdated SA rules * [WebUI] Add flexible columns * [WebUI] Add footable * [WebUI] Add sender, recipients and subject columns * [WebUI] Allow message-id break * [WebUI] Fix history clustering * [WebUI] Fix history display * [WebUI] Fix sorting * [WebUI] Humanize sizes * [WebUI] Initial move towards footable * [WebUI] Remove datatables * [WebUI] Replace `.values` method with `.map` * [WebUI] Rework v2 symbols display * [WebUI] Try to normalize frequencies * [WebUI] Unbreak WebUI * [WebUI] Use Footable to draw Throughput summary table Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
ca1a968cf5
|
Release 1.5.3
* [Conf] Add composite for hacked wordpress phishing * [CritFix] Fix base64 decoding when there are unparseable characters * [Feature] Additional symbol metadata in metadata exporter * [Feature] Add method to get protocol reply from Lua * [Feature] Add symbols when tagged rcpt/sender are normalised * [Feature] Add task:get_symbols_all() function * [Feature] Allow multiple formats of DKIM signing key * [Feature] Allow to cache and use flexible protocol reply * [Feature] Allow to set one_shot flag from register_symbol * [Feature] Allow to skip certain types of hashes when learning fuzzy * [Feature] Cache and insert scan time into the protocol * [Feature] Detect newlines in rspamc --mime * [Feature] DKIM signing: support use of maps * [Feature] Greylist: Support excluding low-scoring messages from greylisting * [Feature] Implement lua history in controller * [Feature] Implement redis history querying * [Feature] Preliminary implementation of redis history plugin * [Feature] Support using request headers in settings * [Fix] Change default template to deal with non-ASCII characters * [Fix] Deal with lists of maps in whitelist module * [Fix] DKIM signing: use domain-specific signing key * [Fix] Do not reallocate completed zstd buffer * [Fix] Do not use local_addrs in proxy * [Fix] Fix crash when resolver is undefined * [Fix] Fix double free when closing lua_tcp connections * [Fix] Fix for lua 5.3 * [Fix] Fix freeing of arrays iterators * [Fix] Fix issue with task:get_symbol and symbols with no metric * [Fix] Fix log line duplication in `rspamadm grep` * [Fix] Fix memory corruption on termination * [Fix] Fix out-of-bound access in base64 decode * [Fix] Fix ratelimit + greylisting * [Fix] Fix subject rewriting * [Fix] Fix task:set_recipients function * [Fix] Fix URI_COUNT_ODD rule * [Fix] Follow the traditional symbols conventions in RCPT_COUNT rule * [Fix] Greylist: Suppress greylist action for whitelisted hosts too * [Fix] Metadata exporter: use rule-specific settings for emails * [Fix] Properly set missing fields in exporter * [Fix] Proxy: max_retries option * [Fix] RCPT_COUNT fixes * [Fix] Rework HAS_X_PRIO rule to match symbols conventions * [Fix] Update issues in ac-trie * [Fix] Use optimised base64 decoding in DKIM * [WebUI] Add preliminary v2 history parser * [WebUI] Allow different history parsers * [WebUI] Display symbols * [WebUI] Rework history v2 function Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
898c9c7b5f
|
Release 1.5.2
* [Conf] Add default config for spamassasssin plugin * [Conf] Add default configuration for antivirus module * [Conf] Add dkim signing docs * [Conf] Add mx_check default config * [Conf] Add replies config * [Conf] Add trie default config * [Feature] Add heuristic to find text parts in files * [Feature] Add rule to detect broken content type * [Feature] Allow to extract CTE in Lua API * [Feature] Allow to set from address for a lua_task * [Feature] Allow to set recipients of a task from Lua * [Feature] Enchance text_part:get_content method * [Feature] Remove + aliases from emails * [Feature] Support rmilter block and dkim signature in CGP helper * [Feature] Support running event loop from Lua * [Fix] Antivirus: use scanner-specific redis prefix * [Fix] Couple of fixes for DKIM signing module * [Fix] Distinguish missing and broken mandatory headers * [Fix] Do more heuristical detection for missing CTE * [Fix] Do not resort cache on each check * [Fix] Fix CGP escaping * [Fix] Fix MISSING_MIME_VERSION rule for plain messages * [Fix] Fix parsing of cte in expressions * [Fix] Fix partial matches in rspamadm grep * [Fix] Fix setting class on style field * [WebUI] Auto-switch Throughput units to `msg/min` for very low rate * [WebUI] Update D3Evolution to 0.0.2 Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
f6c7e98b74
|
Release 1.5.1
* [CritFix] Fix processing of stop_patterns with `\0` character * [CritFix] Fix setting of raw key for signing * [Fix] Fix lua exports from plugins during reload * [Fix] Fix prefilters action scores * [Fix] Fix symbols processing order * [Minor] Help cmake find gthread * [Minor] Some cmake fixes Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
9d2e7e0e2c |
Release 1.5.0
* [Conf] Add configurations for asn, clickhouse and dcc * [Conf] Add default config for url redirector plugin * [Conf] Add the default config for greylist module * [Conf] Allow to edit all local maps from WebUI by default * [CritFix] Deal with absent headers in DKIM * [CritFix] Do not trust remote shingles count * [CritFix] Fix bad memory leak in TLS certificates validation * [CritFix] Fix critical memory issues with radix maps * [CritFix] Fix descriptors leak on reload * [CritFix] Fix headers selection in DKIM verification * [CritFix] Fix parsing of boundaries that end with `--` * [CritFix] Repair PTR_ARRAY_FOREACH macro * [Feature] Add CORS support to the controller * [Feature] Add FROM_NAME_EXCESS_SPACE rule * [Feature] Add REPLYTO_EMAIL_HAS_TITLE rule * [Feature] Add `caseless_hash` method to `lua_util` * [Feature] Add `rip` keyword to ratelimit module * [Feature] Add a simple benchmark for content type parsing * [Feature] Add boundaries parsing in content type * [Feature] Add charset detection for text parts * [Feature] Add content disposition parser * [Feature] Add fallback if too many updates are failing * [Feature] Add function to convert struct tm to time using timezone * [Feature] Add function to normalize HTTP paths * [Feature] Add fuzzy collection plugin * [Feature] Add fuzzy logic for images * [Feature] Add gmime parser to mime_tool * [Feature] Add heuristic to detect broken messages * [Feature] Add heuristic to find displayed URLs * [Feature] Add heuristic to process broken email addresses * [Feature] Add images normalization * [Feature] Add mechanism for disabling composites (Fixes #1270) * [Feature] Add method to create regexp from a glob pattern * [Feature] Add mime encoding manipulation routines * [Feature] Add mime tool to explore messages * [Feature] Add more meta tokens from received headers * [Feature] Add neighbours option to support Rspamd cluster in WebUI * [Feature] Add new function to parse mime addresses * [Feature] Add new methods for lua_tcp * [Feature] Add own headers decoding routine * [Feature] Add own routine to generate a message id * [Feature] Add parser for SMTP date * [Feature] Add per-task lua cache to reuse 'heavy' objects * [Feature] Add plugins list path in WebUI * [Feature] Add preliminary multipart support * [Feature] Add preliminary version of DKIM signing module * [Feature] Add profiling support in client output * [Feature] Add rfc2047 grammar * [Feature] Add rfc2047 variant for QP decoding * [Feature] Add rmilter_headers module (Fixes #1227) * [Feature] Add sse42 version of base64 decoding * [Feature] Add ssse3 and avx2 base64 decoders * [Feature] Add support of libgd * [Feature] Add the preliminary version of redirects resolver in Lua * [Feature] Add ucl_object_iterate_full function * [Feature] Add url encoding function * [Feature] Allow SOA requests in lua dns * [Feature] Allow custom parse types in lua ucl * [Feature] Allow plugins to register webui handlers * [Feature] Allow to add options explicitly to symbols * [Feature] Allow to call a callback when symbol frequency is on peak * [Feature] Allow to call redirector script from SURBL * [Feature] Allow to create variable length dkim keys * [Feature] Allow to have module specific options for Redis in plugins * [Feature] Allow to pass sign key directly from Lua * [Feature] Allow to register configuration docs from Lua API * [Feature] Allow to return options as a table * [Feature] Allow to set peak callbacks from Lua * [Feature] Allow to specify custom method for a message * [Feature] Allow to store dkim keys in Redis * [Feature] Allow to store messages in files * [Feature] Apply DCT using AAN for fuzzy signature * [Feature] Avira SAVAPI support * [Feature] Cache and simplify DCT and jpeg decode * [Feature] Cache libicu convertors * [Feature] Detect URLs with suspicious omographs * [Feature] Do not increase score for duplicate options * [Feature] Do not trust CTE, check base64 and qp strictly * [Feature] Dynamic reputation in URL reputation plugin * [Feature] Extend redis lock when learning spawned * [Feature] Filter non-utf chars from all decoded headers * [Feature] Fix phishing detection for IDNA urls * [Feature] Ignore bad symbols on base64 decoding * [Feature] Ignore too wide elements in SPF * [Feature] Implement fuzzy collection mode * [Feature] Implement helo maps in multimap * [Feature] Implement human readable buckets configuration * [Feature] Implement min-hash shingles for DCT data from images * [Feature] Implement new algorithm for fuzzy hashes of images * [Feature] Implement new unicode normalizer * [Feature] Implement quoted printable decoding * [Feature] Implement received headers flags * [Feature] Implement rspamdgrep tool * [Feature] Implement sane checksum for config file * [Feature] Implement url tags concept * [Feature] Improve detection of omographs using libicu * [Feature] Improve url redirector module * [Feature] Multimap: Received header processing * [Feature] Multiple improvements in the maps * [Feature] New URL filters in multimap * [Feature] Plugin to force actions on selected symbols * [Feature] RBL module: support hashing for emails and helo RBL * [Feature] Reuse URL tags in SURBL module * [Feature] Rework RRD ds count, add conversion path * [Feature] Rework surbl module to avoid extra redirector calls * [Feature] Send config id to the WebUI * [Feature] Simplify HTTPCrypt client support * [Feature] Skip processing for large images * [Feature] Start collection only mode implementation for fuzzy storage * [Feature] Start import of the optimized base64 decode * [Feature] Store all received headers in lua * [Feature] Store relational order of all headers in a message * [Feature] Support DKIM signing in Lua plugins * [Feature] Support HTTPCrypt client in lua_http * [Feature] Support setting SMTP message in multimap * [Feature] Support setting metric subject from Lua * [Feature] Support setting subject in force actions module * [Feature] Treat v6 mapped addresses as v4 addresses * [Feature] URL reputation plugin * [Feature] Use Redis instead of memcached in URLs redirector * [Feature] Use Rspamd rfc2047 decoder instead of gmime one * [Feature] Use a different normalization for fuzzy images * [Feature] Use normalized images in fuzzy hashes * [Feature] Use own code for parsing of date * [Feature] Use shingles for images fuzzying * [Feature] Use t1ha for hashes, allow inlining * [Feature] Use t1ha instead of metrohash and xxhash32 * [Feature] Various new features in metadata exporter module * [Feature] rmilter_headers: authentication-results (#78) * [Fix] Add additional check to mark redis connection inactive * [Fix] Add packed attribute for protocol structure * [Fix] Adopt OMOGRAPH_URL rule * [Fix] Allow static maps * [Fix] Allow to disable classifiers checks using settings and conditions * [Fix] Another try to fix 0 length maps * [Fix] Another try to fix corruption during maps reload * [Fix] Another try to fix descriptors leak * [Fix] Another try to fix reload and logger * [Fix] Antivirus module: register virtual symbols for patterns * [Fix] Avoid extensive reallocs * [Fix] Avoid mempool leak in SA plugin on reload * [Fix] Avoid race condition on saving cache and reload * [Fix] Avoid reusing g_error (Fixes #1262) * [Fix] Break pool connection on fatal redis errors * [Fix] Check for NaN properly * [Fix] Couple of fixes for date parsing * [Fix] Date header timezone adjustments (#1279) * [Fix] Deal with EOF properly * [Fix] Decode filename in content disposition * [Fix] Disable fuzzy images by default * [Fix] Disable zero-copy mode for text parts to avoid crashes * [Fix] Do not destroy session when not all finish scripts are done * [Fix] Do not greyscale images * [Fix] Do not leave parent-less workers processes on fatal errors * [Fix] Do not lowercase Content-Disposition to perform decoding * [Fix] Do not penalize characters just after numeric prefix * [Fix] Do not refork workers that are intended to die * [Fix] Do not set pre-result and update records for no `Queue-ID` messages * [Fix] Do not skip post-filters when pre-filters have set some results * [Fix] Do not stop symbols planning if async events are pending * [Fix] Do not try to set keys for unencrypted requests in proxy * [Fix] Encode URLs according to rfc3986 * [Fix] Encode URLs before sending them to the protocol * [Fix] Filter bad characters from message id * [Fix] Fix CTE detection heuristic * [Fix] Fix Content-Type in HTTP requests * [Fix] Fix IDN eslds phishing checks * [Fix] Fix adding maps from config in Lua * [Fix] Fix another reload memory issue * [Fix] Fix argument returned on redis backend errors * [Fix] Fix assertion in graph handling * [Fix] Fix body trie matching * [Fix] Fix build * [Fix] Fix byte array expansion during toutf8 conversion * [Fix] Fix charset normalisation * [Fix] Fix checking of DKIM bodies that needs just `\n` to be added * [Fix] Fix couple of cornercases with email addresses * [Fix] Fix couple of issues * [Fix] Fix dependencies tracking for callback symbols * [Fix] Fix detection of jpeg size * [Fix] Fix errors handling in fuzzy backend initialization * [Fix] Fix fuzzy hashes count * [Fix] Fix globbing and convert lists to arrays in fuzzy_check * [Fix] Fix heuristical CTE detection for QP encoding * [Fix] Fix ignoring of bad text parts * [Fix] Fix indexes in array access, interleave loop * [Fix] Fix int64 -> double conversion * [Fix] Fix invalid memory access on reload * [Fix] Fix issues with empty updates * [Fix] Fix issues with quoted-printable encoding * [Fix] Fix keys names * [Fix] Fix lots of issues in mime parser code * [Fix] Fix lua maps load * [Fix] Fix macro name * [Fix] Fix mas group score calculations * [Fix] Fix matching of the same patterns from different tries * [Fix] Fix memory corruprtion and leak * [Fix] Fix memory leak in HTTP maps * [Fix] Fix memory leak in expression destroying * [Fix] Fix memory leak in parsing of mime names * [Fix] Fix memory leak in safe ucl iterators * [Fix] Fix memory leak on reload in plugins * [Fix] Fix modules reconfigure on reload * [Fix] Fix monitored setup fro URLBLs with IP addresses * [Fix] Fix name of var * [Fix] Fix new rrd updates * [Fix] Fix out of bounds access * [Fix] Fix parsing messages with no body * [Fix] Fix parsing of '=' character in headers * [Fix] Fix parsing of messages with no content type * [Fix] Fix plugins callbacks in webui * [Fix] Fix possible memory corruption in redis pool * [Fix] Fix probability calculations for fuzzy redis backend * [Fix] Fix processing errors in lua_tcp * [Fix] Fix processing of emails with name only * [Fix] Fix processing of non-multipart messages * [Fix] Fix processing of parts with no valid content type * [Fix] Fix race condition in SIGUSR2 handler * [Fix] Fix redis options parsing when no redis servers are defined * [Fix] Fix reload and hyperscan ready event * [Fix] Fix reload memory issue * [Fix] Fix rra_ptr conversion * [Fix] Fix rrd file conversion * [Fix] Fix setting of content-type attributes * [Fix] Fix signing headers creation in DKIM * [Fix] Fix stddev calculations * [Fix] Fix surbl plugin to work with composite maps * [Fix] Fix timezones parsing * [Fix] Fix tokens usage * [Fix] Fix urls and emails hashes * [Fix] Fix usage of unsafe ucl iterators * [Fix] Fix work with broken utf8 tokens * [Fix] Fix writing of user to roll history * [Fix] Forgotten worker * [Fix] Further memory leaks fixes * [Fix] Ignore lua metatokens in bayes for now * [Fix] Improve OMOGRAPH_URL rule * [Fix] Lua IP from string should be invalid if parsing failed * [Fix] Miltiple fixes to new lua_tcp, add debugging * [Fix] More fixes for iterators cleanup * [Fix] More fixes to logger initialization * [Fix] More heuristic fixes for phishing detection * [Fix] More leaks eliminated * [Fix] More leaks... * [Fix] More random fixes for reload... * [Fix] Multimap: Fixes for email filters * [Fix] Multiple fixes for fann module * [Fix] Multiple memory corruption fixes * [Fix] Normalize path in HTTP router * [Fix] Plug memory leak * [Fix] Plug memory leak in adding radix trees * [Fix] Plug memory leak in configuration parser * [Fix] Plug memory leak in expressions parsing during reload * [Fix] Plug memory leak in learning fuzzy storage * [Fix] Plug memory leak in lua_tcp * [Fix] Plug reload leaks * [Fix] Plug termination memory leaks * [Fix] Really increase lock lifetime * [Fix] Replies module: fix symbol weight * [Fix] Restore content type params related functions * [Fix] Set task's subject from mime subject * [Fix] Sigh, one more reload leak * [Fix] Simplify images shingles * [Fix] Some more memory issues are fixed * [Fix] Stop hardcoding of lua in C * [Fix] Stop processing of bad parts as text parts * [Fix] Strictly filter bad characters when emittin json * [Fix] Strings returned from lua are ephemeral * [Fix] Support unix sockets for lua redis * [Fix] Try to fix issues with reloading config * [Fix] Try to fix race condition in redis_pool * [Fix] Use checksum to avoid intersection between different ANNs * [Fix] Use rspamd hashes in embedded ucl * [Fix] Use sane default rewrite subject (*** SPAM *** %s) * [Fix] Various collection mode fixes * [Fix] Various fixes to mime parser * [Fix] Various reload leak fixing * [Fix] Whitelist certain extensions from archive checks * [Rework] Add preliminary implementation of the mime parser * [Rework] Adopt code for the new options * [Rework] Change logger setup interface * [Rework] Composite configuration (#1270) * [Rework] Finally remove gmime dependency from Rspamd * [Rework] Further fixes to symbols frequencies * [Rework] Implement content type parser for mime * [Rework] Kill all InternetAddressList usages * [Rework] Multiple fixes for symbols cache statistics * [Rework] Refactor struct names * [Rework] Rework images fuzzy hashes algorithm * [Rework] Rework lua_tcp to allow TCP dialog * [Rework] Start massive rework to get rid of gmime * [Rework] Start new approach for multiparts parsing * [Rework] Start rework of mime addresses * [Rework] Start rework of symbols cache updates * [Rework] Start switching to libicu * [Rework] Use a special structure for stats tokens * [Rework] Use hash tables for symbols options * [Rework] Use libicu instead of iconv for conversions * [Rework] Use new scheme to parse mime parts * [WebUI] Add Access-Control-Allow-Origin for cluster management * [WebUI] Add Throughput graph autorefreshing (#820) * [WebUI] Add Visibility.js library * [WebUI] Add basic cluster support to Throughput tab * [WebUI] Add graph legend entries for new DSes * [WebUI] Add graph tab * [WebUI] Add neighbours RRD data consolidation * [WebUI] Add preliminary save symbols clustering * [WebUI] Add server selector to navbar * [WebUI] Add soft reject to auth stats * [WebUI] Add summary to the Throughput tab * [WebUI] Allow to save maps on the cluster * [WebUI] Avoid extra graph redraw and alerts glitching * [WebUI] Be more generous with AJAX timeout * [WebUI] Disable error ring loading in `read only` mode * [WebUI] Enclose table header cells with `tr`s * [WebUI] Finish interface rework * [WebUI] Fix RRD summary pie chart position * [WebUI] Fix `All SERVERS` graph fot just one available server * [WebUI] Fix case when no cluster is defined * [WebUI] Fix compatibility with non-ES6 compliant browsers * [WebUI] Fix config ID * [WebUI] Fix configuration page partially * [WebUI] Fix disabled state * [WebUI] Fix graph dataset selector initialization * [WebUI] Fix graph selectors state resetting * [WebUI] Fix mouse events on throughput summary table area * [WebUI] Fix multiple JS issues * [WebUI] Fix pie chart displaying * [WebUI] Fix read only * [WebUI] Fix read only2 * [WebUI] Fix retarded datatables * [WebUI] Fix soft reject in pie chart * [WebUI] Fix stat widgets timers multiplication on `Refresh` click * [WebUI] Fix symbols config * [WebUI] Fix various errors with login form * [WebUI] Further fixes * [WebUI] Hide learning tab in read-only mode * [WebUI] Initial clusters support * [WebUI] Make legend entry colours more contrast * [WebUI] Move configuration tab to a separate module * [WebUI] Move history tab * [WebUI] Move symbols config as well * [WebUI] New sec to time function * [WebUI] Prevent multiple clicks on `Refresh` * [WebUI] RRD summary: Hide inner labels of tiny pie sectors * [WebUI] RRD summary: Respect undefined values * [WebUI] Reduce font size of graph's legend * [WebUI] Remove orphaned font duplicates * [WebUI] Remove unused code * [WebUI] Replace spinner with animated glyphicon * [WebUI] Reset refresh timer on server switching * [WebUI] Rework interface to use requirejs * [WebUI] Rework neighbours query function * [WebUI] Separate attributes by space * [WebUI] Set focus to password field (#1230) * [WebUI] Simplify neighbours table populating * [WebUI] Start rework of modules * [WebUI] Stop stats refreshing if the page is hidden * [WebUI] Turn d3pie's stuff into a reusable function, * [WebUI] Unify send data functions * [WebUI] Update D3Evolution to 0.0.1 * [WebUI] Update d3.js * [WebUI] Update datatables to work with the requirejs * [WebUI] Use unified tab click event handler, * [WebUI] clusters for the chart * [WebUI] fix uptime Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
37573488e2 |
Release 1.4.2
* [CritFix] Deal with absent headers in DKIM * [CritFix] Do not trust remote shingles count * [CritFix] Fix headers selection in DKIM verification * [Feature] Add EXT_CSS rule * [Feature] Add toggle for disabling SURBLs * [Feature] Extend redis lock when learning spawned * [Feature] Parse <link> HTML tags * [Fix] Avoid reusing g_error (Fixes #1262) * [Fix] Do not reset loaded ANN when learning is requested * [Fix] Fix another issue with external deps in SA * [Fix] Fix body trie matching * [Fix] Fix checking of DKIM bodies that needs just `\n` to be added * [Fix] Fix fuzzy hashes count * [Fix] Fix keys names * [Fix] Fix length calculations for url encoded urls * [Fix] Fix matching of the same patterns from different tries * [Fix] Fix name of var * [Fix] Fix parsing of URLs with spaces and other bad chars * [Fix] Fix probability calculations for fuzzy redis backend * [Fix] Fix signing headers creation in DKIM * [Fix] Plug memory leak * [Fix] Really fix chained SA dependencies * [Fix] Really increase lock lifetime * [Fix] Use checksum to avoid intersection between different ANNs * [Fix] Use rspamd hashes in embedded ucl * [Fix] Yet another change for testing external deps Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
cbdf99ae2e
|
Release 1.4.1
* [Feature] ASN support in Clickhouse module * [Feature] Add clickhouse plugin * [Feature] Add generic tool to add universal maps for lua modules * [Feature] Add logger.debugm to debug lua modules * [Feature] Allow to register metrics symbols using register_symbol * [Feature] Allow to specify prefix for fann_redis * [Feature] Clickhouse: support different masks for IPv4/IPv6 * [Feature] Support forcing action in antivirus plugin * [Fix] Add handling of regexp maps * [Fix] Allow backslashes in http urls * [Fix] Avoid mapping of empty files * [Fix] Do not load tld file to speed up rspamadm * [Fix] Do not resolve numeric IP addresses due to ipv6 insanity * [Fix] Filter incorrect training data * [Fix] Fix Fuzzyconvert tool when password or DB is given * [Fix] Fix build with custom glib/gmime * [Fix] Fix converting of learn count from sqlite to redis * [Fix] Fix crashes with invalid received and task:set_from_ip * [Fix] Fix external dependencies for SA module * [Fix] Fix fann_redis when number of scores has been changed * [Fix] Fix hyperscan usage for non compatible platforms * [Fix] Fix loading of maps from UCL objects * [Fix] Fix memory leak for task-less redis requests * [Fix] Fix mid module with new maps syntax * [Fix] Fix parsing of URLs with username * [Fix] Fix re cache initialisation * [Fix] Fix replacements to sanitize '%' character * [Fix] Fix set and regexp like static maps * [Fix] Fix some issues in redis settings * [Fix] Fix static IP maps * [Fix] Fix total learns counter for redis stats * [Fix] Fix usage of config during reload * [Fix] Fix various warnings and issues * [Fix] Invalidate ANN if training data is incorrect * [Fix] Miltiple fixes to fann_redis module * [Fix] More fixes for URLs with backslashes * [Fix] Properly get options for ip_score module * [Fix] Relax requirements for Received as gmail cannot RFC * [Fix] Remove or fix hyperscan incompatible regexps * [Fix] Settings: correctly read redis config * [Rework] Rework lua logger interface slightly * [Rework] Use new maps add function Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
919cbd477d
|
Release 1.4.0
* [CritFix] Add guards for inactive redis connections
* [CritFix] Another fix for proxying files using rspamd_proxy
* [CritFix] Cleanup inactive redis connections
* [CritFix] Do not sometimes try to exec posfilters before classification
* [CritFix] Fix application of IPv6 mask
* [CritFix] Fix chunked encoding when reading messages
* [CritFix] Fix file mode for rspamd_proxy
* [CritFix] Fix hyperscan compilation on regexp change
* [CritFix] Fix issue with finding of end of lines pointers
* [CritFix] Fix iteration over headers array (introduced in 1.4)
* [CritFix] Fix processing of learned tokens count for redis backend
* [CritFix] Fix race condition in checking of cached maps
* [CritFix] Fix workers scripts by sharing workers configs
* [CritFix] Introduce raw content to text parts
* [CritFix] Plug memory leak and potential memory corruption
* [Feature] Adaptive ratelimits
* [Feature] Add ASN -> rbldnsd script for asn.rspamd.com
* [Feature] Add DMARC_NA symbol
* [Feature] Add F-Prot support to antivirus module
* [Feature] Add HTTP backend to metadata exporter
* [Feature] Add Lua API module for monitored objects
* [Feature] Add R_DKIM_NA / R_SPF_NA / AUTH_NA symbols
* [Feature] Add R_DKIM_PERMFAIL symbol
* [Feature] Add R_SPF_PERMFAIL symbol
* [Feature] Add Sophos antivirus support
* [Feature] Add ZSTD compression to Lua API
* [Feature] Add `mid` Lua module
* [Feature] Add `one_param` flag for metric symbols
* [Feature] Add a generic lua classifier
* [Feature] Add a very basic interface to access workers data from on_load
* [Feature] Add ability to delete a hash by its data to fuzzy_check plugin
* [Feature] Add ability to enable/disable symbols via dynamic_conf
* [Feature] Add ability to lookup settings by key
* [Feature] Add common way to disable Lua modules
* [Feature] Add compression support to rspamd client
* [Feature] Add condition to do antiviral check
* [Feature] Add configuration for lua classifiers
* [Feature] Add configuration knobs for the errors circular buffer
* [Feature] Add decompression support in rspamd client
* [Feature] Add errors exporter to the controller
* [Feature] Add expected value for monitored DNS resources
* [Feature] Add exporter from error ringbuf to ucl
* [Feature] Add extended version for fann creation function
* [Feature] Add ffi friendly version of process_regexp function
* [Feature] Add frequency and time display to webui
* [Feature] Add fuzzy_delhash command to rspamc client
* [Feature] Add implementation of redis connections pool
* [Feature] Add latency and offline time monitoring
* [Feature] Add learning support for lua classifiers
* [Feature] Add max-size and timeout options to CGP helper
* [Feature] Add method to enable/disable symbols in config
* [Feature] Add methods to get metric's actions and symbols from Lua
* [Feature] Add mmap support to lua_text
* [Feature] Add monitored object for surbl plugin
* [Feature] Add more exceptions to surbl whitelist
* [Feature] Add more meta-tokens to bayes
* [Feature] Add neural net classifier to fann_scores module
* [Feature] Add neural net serialization/deserialization
* [Feature] Add new dynamic conf module
* [Feature] Add periodic events support for lua_config
* [Feature] Add plugin to check MX'es for the sender's domain
* [Feature] Add preliminary monitored module
* [Feature] Add preliminary support of dynamic conf updates in Redis
* [Feature] Add preliminary version of clamav plugin
* [Feature] Add redis cache to asn module
* [Feature] Add replies compression
* [Feature] Add spamhaus DROP dnsbl
* [Feature] Add support for dictionary in client compression
* [Feature] Add support for fuzzy learn and unlearn from lua
* [Feature] Add support for input encryption
* [Feature] Add support of min_learns to neural net classifier
* [Feature] Add termination callbacks for workers
* [Feature] Add user-agent for rspamc
* [Feature] Add utility to perform classifier tests
* [Feature] Add zstd compression library
* [Feature] Allow HTTPS requests in lua_http
* [Feature] Allow conditions for pre and postfilters
* [Feature] Allow custom functions for ratelimits
* [Feature] Allow for excluding messages from AV scanning based on size
* [Feature] Allow for getting worker stats from Lua
* [Feature] Allow getting task UID from Lua
* [Feature] Allow parsing of mailbox messages from the commandline
* [Feature] Allow plugins to publish their lua API via rspamd_plugins
* [Feature] Allow to compare other systems with Rspamd
* [Feature] Allow to execute Lua scripts by controller
* [Feature] Allow to have a function to set custom greylist message
* [Feature] Allow to iterate over multiple tags
* [Feature] Allow to pass extra data from plugins to log helper
* [Feature] Allow to plan new periodics at different time
* [Feature] Allow to reset hashes
* [Feature] Allow to run rspamadm lua just as a lua interpreter
* [Feature] Allow to store settings in redis
* [Feature] Allow to update dynamic conf in Redis
* [Feature] Allow to use dictionaries for compression
* [Feature] Allow to use md5, sha1, sha256, sha384 and sha512 hashes in Lua
* [Feature] Allow whitelisting by IP for greylisting plugin
* [Feature] Antivirus: Support whitelists & pattern-matching sig names
* [Feature] Backport pack/unpack routines from Lua 5.3
* [Feature] Check settings with equal priopities in alphabetical order
* [Feature] Compress neural net in redis
* [Feature] Consider more tags when doing WHITE_ON_WHITE rule
* [Feature] Descriptive options for DMARC failure symbols
* [Feature] Descriptive options for RBL symbols
* [Feature] Enable configuration for monitored objects
* [Feature] Execute on_load scripts with ev_base ready
* [Feature] Fann scores now uses metadata from a message
* [Feature] Implement FANN threaded learning
* [Feature] Implement classifying for lua classifiers
* [Feature] Implement finish scripts for worker processes
* [Feature] Implement monitoring for DNS resources
* [Feature] Implement real priorities for pre and post filters
* [Feature] Insert two symbols: FANN_HAM and FANN_SPAM instead of one
* [Feature] Module to push metadata/messages to redis pubsub
* [Feature] Monitor RBL records
* [Feature] Move fann_classifier to a separate plugin
* [Feature] Normalize all ANN inputs
* [Feature] Preliminary version of metric exporter module
* [Feature] Preserve decompression context between tasks
* [Feature] Ratelimit: Support dynamic bucket size/leak rate
* [Feature] Relax FORGED_RECIPIENTS: allow senders to BCC themselves
* [Feature] Remove symbols weights on composites processing
* [Feature] Return symbol scores when getting resulting symbols
* [Feature] Rework lua tcp module
* [Feature] Rule to detect some obvious X-PHP-Originating-Script forgeries
* [Feature] Rule to identify some X-PHP-Script forgeries
* [Feature] Rules for scoring Google Message-ID fixes
* [Feature] Send hashes values to reply
* [Feature] Set expire for dmarc reports
* [Feature] Stop using cymru zone as it is unstable
* [Feature] Stop using of GLists for headers, improve performance
* [Feature] Store `for` in task:get_received_headers
* [Feature] Store `for` part in received headers
* [Feature] Store enabled flag for webui session
* [Feature] Store error messages in ring buffer
* [Feature] Support compressed maps
* [Feature] Support excluding selected users from ratelimits
* [Feature] Support looking up NS records in lua_dns
* [Feature] Support modern style SURBL configuration
* [Feature] Support multiple hashes in delhash path
* [Feature] Support new messages in rspamc
* [Feature] Support requests without reads in lua_tcp
* [Feature] Support setting task message from Lua
* [Feature] Track visibility of HTML elements
* [Feature] Try to add CRLF when checking DKIM
* [Feature] Try to guess line endings when folding headers
* [Feature] Try to improve normalization function for bayes
* [Feature] Use FFI to optimize SA module
* [Feature] Use length based arguments for redis, allow lua_text as arg
* [Feature] Use more layers for fann and another normalization
* [Feature] User-defined ratelimits
* [Feature] Utility to convert fuzzy storage from sqlite to redis
* [Feature] Yield DMARC_DNSFAIL on lookup failure
* [Fix] Adopt fuzzy storage for flexible backends
* [Fix] Allow plain IP addresses in Rspamd maps
* [Fix] Another fix for brain-damaged hiredis
* [Fix] Another fix for rdns write errors
* [Fix] Another fix for rdns_make_request_full invocation
* [Fix] Another fix in DKIM canonicalization
* [Fix] Another memory leak plugged
* [Fix] Another try to deal with posix idiotizm
* [Fix] Another try to fix RDNS events processing logic
* [Fix] Avoid double frees in HEAD requests
* [Fix] Avoid extra symbols for RBLs
* [Fix] Banish table.maxn from Lua parts
* [Fix] Check for socket error before connection in lua_tcp
* [Fix] Correctly propagate redis timeouts to Lua
* [Fix] Do not add extra newline in MIME mode
* [Fix] Do not be cheated by system hiredis
* [Fix] Do not classify when a message has not enough tokens
* [Fix] Do not crash on redis errors
* [Fix] Do not distinguish NXDOMAIN and NOREC for monitored
* [Fix] Do not replan retransmits if merely one server is defined
* [Fix] Do not use headers to calculate messages digests
* [Fix] Don't force action in replies module for authenticated users/local networks
* [Fix] Explicitly ban default passwords in webui
* [Fix] Finally fix ambiguity between parsed and resolved spf elts
* [Fix] Fix 'decoded' value in task:get_header_full()
* [Fix] Fix DKIM calculations
* [Fix] Fix DKIM signing for messages with no newline at the end
* [Fix] Fix DNS request in monitored
* [Fix] Fix DNS write errors processing
* [Fix] Fix HTTP methods other than GET and POST
* [Fix] Fix PERMFAIL for v6/v4 ambiguities
* [Fix] Fix absurdic scores for HFILTER_URL_ONLY
* [Fix] Fix actions in rolling history
* [Fix] Fix actrie patterns
* [Fix] Fix applying of lua dynamic confg
* [Fix] Fix autolearning errors and redis cache
* [Fix] Fix bayes learn_condition
* [Fix] Fix build with the recent OpenSSL
* [Fix] Fix caching and compressed maps
* [Fix] Fix check plain text part
* [Fix] Fix crash on OpenBSD in `url_email_start`
* [Fix] Fix double free in SPF
* [Fix] Fix extraction of shingles from redis fuzzy storage
* [Fix] Fix false sharing for symbols in the cache
* [Fix] Fix float usage in util:get_time
* [Fix] Fix folding algorithm to deal with empty tokens
* [Fix] Fix format string
* [Fix] Fix format string usage in controller errors handling
* [Fix] Fix handling of '\0' in lua_tcp
* [Fix] Fix handling of HTTP HEAD methods
* [Fix] Fix hash creation
* [Fix] Fix hiredis stupidity
* [Fix] Fix implicit settings module settingsup
* [Fix] Fix interaction with lua GC to avoid craches
* [Fix] Fix ip_score module registration
* [Fix] Fix issue with empty messages and dkim
* [Fix] Fix issues with CGP helper
* [Fix] Fix issues with the recent SPF changes
* [Fix] Fix key name to load ANN correctly
* [Fix] Fix lua tcp module by saving `do_read` in callback data
* [Fix] Fix memory leak in client when using compression
* [Fix] Fix min_learns option
* [Fix] Fix on_finish scripts and async handlers
* [Fix] Fix options for SPF dnsfail symbol
* [Fix] Fix parsing includes and redirects in SPF
* [Fix] Fix parsing of lua comments with empty lines
* [Fix] Fix parsing of unquoted HTML attributes
* [Fix] Fix periodic events and redis
* [Fix] Fix processing of fuzzy learns from Lua
* [Fix] Fix processing of redirect in SPF includes
* [Fix] Fix processing of symbols when reject limit is reached
* [Fix] Fix refcounts when map is specified by IP
* [Fix] Fix rspamd{session} class in Lua API
* [Fix] Fix setting ratelimit key for 'ip' bucket
* [Fix] Fix some cases of TLD urls detector
* [Fix] Fix statconvert tool
* [Fix] Fix stats for backend-less classifiers
* [Fix] Fix training script for fann_redis
* [Fix] Fix variable in ann module
* [Fix] Fix various errors in lua dynamic conf plugin
* [Fix] Forget old ANN when max_usages is reached to avoid overtrain
* [Fix] Further canonicalization fixes
* [Fix] Further fixes for fann_redis prefixes
* [Fix] Handle failures for inactive pooled connections
* [Fix] Improve multimap info message
* [Fix] More fixes in ANN loading
* [Fix] More fixes to fann_redis
* [Fix] More issues in fann_redis
* [Fix] More spaces fix in DKIM signature
* [Fix] Multiple fixes to asn script, add IPv6 support
* [Fix] Multiple issues in fann_redis
* [Fix] No greylist rejected messages
* [Fix] One more attempt to fix lua_redis
* [Fix] One more check for readdir...
* [Fix] Params should be treated as a hash
* [Fix] Plug memory leak in regexp desctructor
* [Fix] Process headers only once
* [Fix] Properly handle nil values in ratelimit plugin
* [Fix] Really fix redis shingles check
* [Fix] Remove fann with incorrect layers count
* [Fix] Remove mentions of deleted include
* [Fix] Remove some incompatible functions
* [Fix] Settings: fix `authenticated` parameter (#886)
* [Fix] Skip MX check for authenticated users and local networks
* [Fix] Slightly fix ANN routines
* [Fix] Stop caching records with DNS failures
* [Fix] Treat all errors in redis_pool as fatal errors for a connection
* [Fix] Try avoid false-positives in HEADER_FORGED_MDN rule
* [Fix] Try to avoid race condition when using rrd
* [Fix] Try to reload redis scripts if they are missing
* [Fix] Unbreak once_received skipping for local networks
* [Fix] Unlock ANN on error
* [Fix] Use memmove for overlapping regions
* [Fix] Use real size instead of displayed for core limits
* [Fix] Use the correct macro to get the size of control
* [Fix] Various fixes for errors ringbuffer
* [Fix] Yield R_SPF_DNSFAIL if lookup of included record fails
* [Fix] mid: fix map initialization
* [Fix] mid: handle incorrect rgexps in the map
* [Rework] Add extract training data function to fann_redis
* [Rework] Add preliminary train tests
* [Rework] Add redis storage feature to fann_redis
* [Rework] Adopt fuzzy storage for abstract backend
* [Rework] Adopt plugins
* [Rework] First reiteration on fann scores
* [Rework] Implement loading/invalidating
* [Rework] Make lua_redis task agnostic
* [Rework] Make rspamd protocol messages useful
* [Rework] Massive removal of legacy code
* [Rework] More cleanup actions
* [Rework] Remove legacy code never used for classifiers
* [Rework] Remove outdated and unused lua_session module
* [Rework] Reorganize fuzzy backend structure
* [Rework] Reorganize the internal backend structure
* [Rework] Restore old fann_scores, move common parts
* [Rework] Rework and simplify rbl plugin
* [Rework] Rework parsing of DMARC records
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
||
|
Vsevolod Stakhov
|
c30af6a8c9
|
Release 1.3.4
* [Feature] ASN module; support matching ASN/country in multimap * [Feature] Add SPF method in spf return result * [Feature] Add Yandex and Mail.ru forwarding rules * [Feature] Add mempool maps in multimap * [Feature] Add rule for identifying mail sent by eval()'d PHP code * [Feature] Add support of stub DNSSEC resolver to rdns * [Feature] Add task:get_digest method * [Feature] Allow for more fine-grained scoring for ip_score * [Feature] Allow to get digest of a mime part from lua * [Feature] Allow to print message digest in logs * [Feature] Fold DKIM-Signature header * [Feature] Implement encrypted logs * [Feature] Log URLs encrypted if we have log encryption pubkey * [Feature] Pass authenticated bit to lua * [Feature] Read redis backend statistics configuration from global section * [Feature] Show the exact value matched for multima symbols * [Feature] Store task checksum * [Fix] Avoid setting limits when required elements are missing * [Fix] DMARC: Fix alignment checking for subdomains * [Fix] DMARC: deal with missing and spurious spaces * [Fix] Defer insertion of results in ip_score to avoid skewing stats * [Fix] Disable DMARC for local/authorized mail * [Fix] Fix handling of proxied headers in controller * [Fix] Fix hex printing of strings * [Fix] Fix issue with spaces in maps * [Fix] Fix parsing of forwarded IP * [Fix] Fix reload in some plugins and workers * [Fix] Fix reloading on SIGHUP * [Fix] Fix some border cases for DKIM canonicalization * [Fix] Fix url maps * [Fix] Make dnssec configurable option disabled by default for now * [Fix] rspamadm statconvert: force db to be a string * [Fix] rspamadm statconvert: use db/password for learn cache * [Rework] Rework flags in rspamd logger Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
8466d1816d |
Release 1.3.3
* [CritFix] Check hyperscan cache sanity before loading * [CritFix] Fix setting of fuzzy keys (completely breaks fuzzy storage) * [Feature] Add SARBL (sarbl.org) uribl * [Feature] Add `--search-pattern` option to rspamd_stats * [Feature] Add some sanity check for very long from/to log elements * [Feature] Allow to create hashes from string in a single step * [Feature] Fix order of pre and postfilters * [Feature] Improve lua URLs API * [Feature] Improve message about fuzzy rules * [Feature] Pre-calculate blake2 digest for all parts * [Feature] Print radix duplicate keys as IP addresses * [Feature] Simple mechanism for disabling RBLs in local.d/rbl.conf * [Feature] Use faster hash function for fuzzy storage * [Feature] rspamd_stats: support log directory reading * [Fix] Add sanity check for url filters * [Fix] Do not show rmilter section as a fake metric in rspamc * [Fix] Fix URL filters * [Fix] Fix a stupid mistake in util.strequal_caseless * [Fix] Fix blake2b hash of the string "rspamd" * [Fix] Fix filename maps filter * [Fix] Fix finding tld in util.get_tld * [Fix] Fix multimap content filters * [Fix] Fix returning boolean from Lua * [Fix] Fix returning of REDIS_NIL * [Fix] Try to deal with multiple workers terminated * [Fix] Use forced DNS request when calling for lua_http * [Rework] Rework multimap filters, add redis maps Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
cb856117ab
|
Release 1.3.2
* [Feature] Add a special symbol for SPF DNS errors: R_SPF_DNSFAIL * [Feature] Add correlations report in fuzzy stats * [Feature] Add experimental CGP integration * [Feature] Add method to get urls length in a text part * [Feature] Add new methods to lua_html to access HTML tags * [Feature] Allow all types of symbols to be added via __newindex method * [Feature] Allow to create settings for authenticated users * [Feature] Allow to get block content for HTML tags * [Feature] Improve DNS failures when dealing with SPF * [Feature] Properly implement R_WHITE_ON_WHITE rule * [Feature] Remove old ugly rules * [Feature] Rspamc can now add dkim signature in mime mode * [Feature] Store content length for HTML tags * [Feature] Support reacher set of HTML colors * [Feature] Try to avoid FP for low contrast fonts detection * [Fix] Add missing HTML colors * [Fix] Add spaces to dkim signature to allow folding * [Fix] Avoid returning NaN as score on scan * [Fix] Decode entitles in href parts * [Fix] Do not cache SPF records with DNS errors * [Fix] Do not crash on cyclic depends * [Fix] Do not insert HELO/HOSTNAME unknown when they are not passed * [Fix] Do not set absent hostname to "unknown" * [Fix] Do not stress redis with KEYS command (#791) * [Fix] Fix DMARC_BAD_POLICY symbol * [Fix] Fix HFILTER_URL module * [Fix] Fix HFILTER_URL_ONELINE rule * [Fix] Fix buffering in CGP integration * [Fix] Fix colors propagation from parent nodes * [Fix] Fix confusing OpenSSL API usage of i2d_RSAPublicKey * [Fix] Fix dependencies id sanity check * [Fix] Fix folding for semicolon separated tokens * [Fix] Fix largest possible TLD behaviour * [Fix] Fix last token folding * [Fix] Fix length calculations in white on white rule * [Fix] Fix multiple request headers structure * [Fix] Fix multiple values headers freeing * [Fix] Fix parsing of background color * [Fix] Fix printing from field in log_urls * [Fix] Fix processing of last element of DMARC policies * [Fix] Further fixes for HTML colors * [Fix] Further fixes for multiple values headers * [Fix] Further fixes for white on white rule * [Fix] Further fixes in HTML tags parsing * [Fix] Ignore content type/subtype case * [Fix] Increase score of R_WHITE_ON_WHITE * [Fix] Parse CGP envelope data * [Fix] Propagate colors in HTML * [Fix] Restore multiple values headers in protocol * [Fix] Restore multiple values in headers processing * [Fix] Some more changes to tag's content length calculations * [Fix] Some more fixes for low contrast fonts detector * [Fix] SpamAssassin plugin: support check_freemail_header('EnvelopeFrom', [..]) * [Fix] Trigger HTML_SHORT_LINK_IMG on any external image * [Fix] rspamd_stats: remove deprecated defined(@array) Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
68b3702150
|
Release 1.3.1
* [CritFix] Fix catena passwords validation * [CritFix] Fix crash when the first received is faked * [Feature] Add DMARC_BAD_POLICY symbol when DMARC policy was invalid * [Feature] Allow for matching hostnames in multimap (#773) * [Feature] Allow for setting action based on DMARC disposition * [Feature] Allow limiting of the inbound message size * [Feature] Allow maps with multiple symbols and scores * [Feature] Allow regexps in the emails maps * [Feature] Allow to register metric symbols from multimap * [Feature] Allow to reset redis tokens instead of appendig values * [Feature] Allow to store strings in radix maps * [Feature] Check UTF validity when there are utf regexps in a map * [Feature] Correctly work when there is no hard reject action * [Feature] Implement dependencies for maps * [Fix] Another effort to unbreak sqlite locking * [Fix] Avoid crash when closing mmapped file * [Fix] Do not break history on NaN in required score * [Fix] Ensure that hyperscan cache written is written properly * [Fix] Filter NaN from scores in history * [Fix] Fix DNSBL maps * [Fix] Fix another locking issue in sqlite * [Fix] Fix another locking issue with mapped files * [Fix] Fix deadlock in mmaped file stats * [Fix] Fix dependencies in multimap plugin * [Fix] Fix emails module configuration * [Fix] Fix greylist plugin (#755) * [Fix] Fix greylisting plugin variable usage * [Fix] Fix installed permissions for rspamd_stats * [Fix] Fix locking in mmapped statistics * [Fix] Fix paths in tests * [Fix] Fix prefilter mode for multimap * [Fix] Forgot to commit leftover changes * [Fix] Really fix local.d includes * [Fix] Restore selective greylisting behaviour * [Fix] Set max size on per connection basis * [Fix] Use temporary storage for hyperscan cache * [Rework] Remove systemd socket activation Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
2842a5ffb1
|
Release 1.3.0
https://rspamd.com/announce/2016/07/25/rspamd-1.3.0.html Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
f39c98bff5 |
Release 1.2.8
* Another fix for exim workaround (#637) * Fix unencrypted passwords processing in the controller * Fix setting path for lua (#652) * Fix usage of rdns reply structure (#654) * Use file lock in logger to avoid deadlocks * Add `application/octet-stream` mime type for `pdf` extension (by @moisseev) * Implement new automata to skip empty lines for dkim signing (#651) * Fix parsing of missing classes * Clarify some rspamc arguments (by @fatalbanana) * Correct suppress spelling Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
54313a6369 |
Release 1.2.7
* Slightly reduce weights of rules with high FP rate
* Add workround for rspamd-1.3
* Fix possible FP in TRACKER_ID rule
* Simplify MISSING_MIMEOLE rule
* Add workaround for gmime CTE stupidity
* Fix mime headers processing
* Fix false positive URL detections in text parts
* Fix Exim shutdown patch
* Enable workaround for exim mailbox format
* Backport shingles static test
* Fix levenshtein distance calculations
* Fix max_train setup in ANN module
* Fix redis structure by adding {NULL, NULL} member
* Fix build with unmodified LibreSSL opensslv.h
* Repair optional dependencies
* Really skip filters in case of pre-result set
* Restore the intended pre-filters behaviour
* Fix ipv6 mask application
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
||
|
Vsevolod Stakhov
|
bb58ea6af4 |
Release 1.2.6
* Fix parsing of URLs in texts * Fix creating of URLs from LUA * Fix some more URL detector issues * Fix unit tests * Fix JIT compilation for PCRE2 expressions * Fix JIT usage for PCRE2 * Fix UTF8 mode in PCRE2 * Add workaround for pre-historic compilers (#605) * Fix and rescore R_PARTS_DIFFER logic * Properly set lua paths for tests * Fix SA rawbody processing - exclude top part * Store text parts content with newlines stripped * Properly support SA body regexps * Fix body rules in SA plugin * Fix setting of score for parts differ * More fixes to parts distance calculations - Use hashed words instead of full words for speed - Improve levenstein distance calculations and penalise replaces - Always return number from 0 to 1 - Use g_malloc instead of alloca * Fix percents output in R_PARTS_DIFFER * Plug memory leak in dkim module * Plug minor memory leak in regexps creation Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
0c026a12be |
Release 1.2.5
* Plug an important memory leak in headers getting code * Remove some bad domains from whitelists Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
78dc764f55
|
Release 1.2.4
* Implement new multipattern matcher that uses hyperscan if possible * Use mutlipattern for lua_trie code * Add utility methods for multipattern * Use multipattern in url matcher * Add escape functions for hyperscan * Allow to optimize lua -> C transition by flattening table args * Optimize hot paths in SA plugin * Optimize rspamd_re_cache_type_from_string * Allow empty tries * Fix extraction of URLs from Subject * Allow to have different flags for different patterns in multipattern * Add common directory for hyperscan cache to config * Implement caching for hyperscan multipattern * Attach domain part to `R_SUSPICIOUS_URL` (by @moisseev) * Allow multipattern scans to be nested for the case of hyperscan * Simplify SURBL redirector search code and avoid ac_trie * Add two way substring search algorithm * Avoid acism usage to find gtube pattern * Fix processing of empty headers * Allow to disable pthread mutexes on broken platforms * Make web interface not send password in query strings (#585) by @fatalbanana * Add maximum delay to ratelimit module * Backport fix for empty files inclusion from libucl * Fix settings id setup * Add min_learns option to classifiers * Use more clever to utf8 conversion strategy * Fix disabling of virtual symbols in the settings * Rework settings to work properly in metric-less configuration * Set the default limit for classifier * Fix ttl based expiration from LRU cache * Rework DKIM module to use OpenSSL for digests * Fix mailto urls parsing with hyperscan * Do not set obscured flag for urls starting with spaces * Fix crash on redis learn * Fix ratelimit ctime setting Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |
||
|
Vsevolod Stakhov
|
1aa4a012f0
|
Release 1.2.3
* New DCC module (by @smfreegard) * Rework whitelist module: - Now we check different elements for different checks - MIME from for DMARC - DKIM signature domain for DKIM - SMTP from or HELO for SPF * Fix regexps results combination (*critical*) * Fix issue with expressions processing (*critical*) * Optimize strlcpy for aligned input * Add support of half-closed connection in lua_tcp * Allow to print compact json in client * Save required score in history (#581) * Allow to attach file descriptors to control commands * Allow to send descriptors from workers to main * Allow to attach fd when broadcasting to workers * Implement log pipe feature for rspamd logs analysis * Add `log_helper` worker * Add `URIBL_SBL_CSS` (by @smfreegard) * Add worker scripts functionality * Add on load hooks for rspamd_config * Add lua scripts for log_helper worker * Add generic maillist detector (#584) * Implement FANN autolearn using log_helper worker * Rework metrics configuration to allow includes * Change default value of forced removal in composite rules * Allow to use assembly version of blake2b on x86 cpu * Use less precise (but faster) clock if possible * Insert redirected URL to the urls list * Allow to get and set callback data for rspamd symbols * Add binary heap implementation * Use binary heap for expire algorithms in the hash * Use `least frequent used` expiration strategy * Allow to get mime headers from a task * Add support for mime headers in `regexp` module * Update Exim patches (by @fatalbanana) * Allow building rspamd with jemalloc * Save multipart boundaries * SA plugin changes: - Properly handle MIME headers - Fix eval:check_for_missing_to_header rule - Implement SA compatible body regexps - Use sabody rules in SA plugin * LUA API changes: - Add util.get_ticks function - Add util.stat function - Add task:get_symbols_numeric method - Add method to get number of symbols in the cache - Add lua methods to get redirected urls - Allow to get callbacks for lua symbols - Add config:set_symbol_callback function Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru> |