| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* [Feature] ASN support in Clickhouse module
* [Feature] Add clickhouse plugin
* [Feature] Add generic tool to add universal maps for lua modules
* [Feature] Add logger.debugm to debug lua modules
* [Feature] Allow to register metrics symbols using register_symbol
* [Feature] Allow to specify prefix for fann_redis
* [Feature] Clickhouse: support different masks for IPv4/IPv6
* [Feature] Support forcing action in antivirus plugin
* [Fix] Add handling of regexp maps
* [Fix] Allow backslashes in http urls
* [Fix] Avoid mapping of empty files
* [Fix] Do not load tld file to speed up rspamadm
* [Fix] Do not resolve numeric IP addresses due to ipv6 insanity
* [Fix] Filter incorrect training data
* [Fix] Fix Fuzzyconvert tool when password or DB is given
* [Fix] Fix build with custom glib/gmime
* [Fix] Fix converting of learn count from sqlite to redis
* [Fix] Fix crashes with invalid received and task:set_from_ip
* [Fix] Fix external dependencies for SA module
* [Fix] Fix fann_redis when number of scores has been changed
* [Fix] Fix hyperscan usage for non compatible platforms
* [Fix] Fix loading of maps from UCL objects
* [Fix] Fix memory leak for task-less redis requests
* [Fix] Fix mid module with new maps syntax
* [Fix] Fix parsing of URLs with username
* [Fix] Fix re cache initialisation
* [Fix] Fix replacements to sanitize '%' character
* [Fix] Fix set and regexp like static maps
* [Fix] Fix some issues in redis settings
* [Fix] Fix static IP maps
* [Fix] Fix total learns counter for redis stats
* [Fix] Fix usage of config during reload
* [Fix] Fix various warnings and issues
* [Fix] Invalidate ANN if training data is incorrect
* [Fix] Miltiple fixes to fann_redis module
* [Fix] More fixes for URLs with backslashes
* [Fix] Properly get options for ip_score module
* [Fix] Relax requirements for Received as gmail cannot RFC
* [Fix] Remove or fix hyperscan incompatible regexps
* [Fix] Settings: correctly read redis config
* [Rework] Rework lua logger interface slightly
* [Rework] Use new maps add function
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* [CritFix] Add guards for inactive redis connections
* [CritFix] Another fix for proxying files using rspamd_proxy
* [CritFix] Cleanup inactive redis connections
* [CritFix] Do not sometimes try to exec posfilters before classification
* [CritFix] Fix application of IPv6 mask
* [CritFix] Fix chunked encoding when reading messages
* [CritFix] Fix file mode for rspamd_proxy
* [CritFix] Fix hyperscan compilation on regexp change
* [CritFix] Fix issue with finding of end of lines pointers
* [CritFix] Fix iteration over headers array (introduced in 1.4)
* [CritFix] Fix processing of learned tokens count for redis backend
* [CritFix] Fix race condition in checking of cached maps
* [CritFix] Fix workers scripts by sharing workers configs
* [CritFix] Introduce raw content to text parts
* [CritFix] Plug memory leak and potential memory corruption
* [Feature] Adaptive ratelimits
* [Feature] Add ASN -> rbldnsd script for asn.rspamd.com
* [Feature] Add DMARC_NA symbol
* [Feature] Add F-Prot support to antivirus module
* [Feature] Add HTTP backend to metadata exporter
* [Feature] Add Lua API module for monitored objects
* [Feature] Add R_DKIM_NA / R_SPF_NA / AUTH_NA symbols
* [Feature] Add R_DKIM_PERMFAIL symbol
* [Feature] Add R_SPF_PERMFAIL symbol
* [Feature] Add Sophos antivirus support
* [Feature] Add ZSTD compression to Lua API
* [Feature] Add `mid` Lua module
* [Feature] Add `one_param` flag for metric symbols
* [Feature] Add a generic lua classifier
* [Feature] Add a very basic interface to access workers data from on_load
* [Feature] Add ability to delete a hash by its data to fuzzy_check plugin
* [Feature] Add ability to enable/disable symbols via dynamic_conf
* [Feature] Add ability to lookup settings by key
* [Feature] Add common way to disable Lua modules
* [Feature] Add compression support to rspamd client
* [Feature] Add condition to do antiviral check
* [Feature] Add configuration for lua classifiers
* [Feature] Add configuration knobs for the errors circular buffer
* [Feature] Add decompression support in rspamd client
* [Feature] Add errors exporter to the controller
* [Feature] Add expected value for monitored DNS resources
* [Feature] Add exporter from error ringbuf to ucl
* [Feature] Add extended version for fann creation function
* [Feature] Add ffi friendly version of process_regexp function
* [Feature] Add frequency and time display to webui
* [Feature] Add fuzzy_delhash command to rspamc client
* [Feature] Add implementation of redis connections pool
* [Feature] Add latency and offline time monitoring
* [Feature] Add learning support for lua classifiers
* [Feature] Add max-size and timeout options to CGP helper
* [Feature] Add method to enable/disable symbols in config
* [Feature] Add methods to get metric's actions and symbols from Lua
* [Feature] Add mmap support to lua_text
* [Feature] Add monitored object for surbl plugin
* [Feature] Add more exceptions to surbl whitelist
* [Feature] Add more meta-tokens to bayes
* [Feature] Add neural net classifier to fann_scores module
* [Feature] Add neural net serialization/deserialization
* [Feature] Add new dynamic conf module
* [Feature] Add periodic events support for lua_config
* [Feature] Add plugin to check MX'es for the sender's domain
* [Feature] Add preliminary monitored module
* [Feature] Add preliminary support of dynamic conf updates in Redis
* [Feature] Add preliminary version of clamav plugin
* [Feature] Add redis cache to asn module
* [Feature] Add replies compression
* [Feature] Add spamhaus DROP dnsbl
* [Feature] Add support for dictionary in client compression
* [Feature] Add support for fuzzy learn and unlearn from lua
* [Feature] Add support for input encryption
* [Feature] Add support of min_learns to neural net classifier
* [Feature] Add termination callbacks for workers
* [Feature] Add user-agent for rspamc
* [Feature] Add utility to perform classifier tests
* [Feature] Add zstd compression library
* [Feature] Allow HTTPS requests in lua_http
* [Feature] Allow conditions for pre and postfilters
* [Feature] Allow custom functions for ratelimits
* [Feature] Allow for excluding messages from AV scanning based on size
* [Feature] Allow for getting worker stats from Lua
* [Feature] Allow getting task UID from Lua
* [Feature] Allow parsing of mailbox messages from the commandline
* [Feature] Allow plugins to publish their lua API via rspamd_plugins
* [Feature] Allow to compare other systems with Rspamd
* [Feature] Allow to execute Lua scripts by controller
* [Feature] Allow to have a function to set custom greylist message
* [Feature] Allow to iterate over multiple tags
* [Feature] Allow to pass extra data from plugins to log helper
* [Feature] Allow to plan new periodics at different time
* [Feature] Allow to reset hashes
* [Feature] Allow to run rspamadm lua just as a lua interpreter
* [Feature] Allow to store settings in redis
* [Feature] Allow to update dynamic conf in Redis
* [Feature] Allow to use dictionaries for compression
* [Feature] Allow to use md5, sha1, sha256, sha384 and sha512 hashes in Lua
* [Feature] Allow whitelisting by IP for greylisting plugin
* [Feature] Antivirus: Support whitelists & pattern-matching sig names
* [Feature] Backport pack/unpack routines from Lua 5.3
* [Feature] Check settings with equal priopities in alphabetical order
* [Feature] Compress neural net in redis
* [Feature] Consider more tags when doing WHITE_ON_WHITE rule
* [Feature] Descriptive options for DMARC failure symbols
* [Feature] Descriptive options for RBL symbols
* [Feature] Enable configuration for monitored objects
* [Feature] Execute on_load scripts with ev_base ready
* [Feature] Fann scores now uses metadata from a message
* [Feature] Implement FANN threaded learning
* [Feature] Implement classifying for lua classifiers
* [Feature] Implement finish scripts for worker processes
* [Feature] Implement monitoring for DNS resources
* [Feature] Implement real priorities for pre and post filters
* [Feature] Insert two symbols: FANN_HAM and FANN_SPAM instead of one
* [Feature] Module to push metadata/messages to redis pubsub
* [Feature] Monitor RBL records
* [Feature] Move fann_classifier to a separate plugin
* [Feature] Normalize all ANN inputs
* [Feature] Preliminary version of metric exporter module
* [Feature] Preserve decompression context between tasks
* [Feature] Ratelimit: Support dynamic bucket size/leak rate
* [Feature] Relax FORGED_RECIPIENTS: allow senders to BCC themselves
* [Feature] Remove symbols weights on composites processing
* [Feature] Return symbol scores when getting resulting symbols
* [Feature] Rework lua tcp module
* [Feature] Rule to detect some obvious X-PHP-Originating-Script forgeries
* [Feature] Rule to identify some X-PHP-Script forgeries
* [Feature] Rules for scoring Google Message-ID fixes
* [Feature] Send hashes values to reply
* [Feature] Set expire for dmarc reports
* [Feature] Stop using cymru zone as it is unstable
* [Feature] Stop using of GLists for headers, improve performance
* [Feature] Store `for` in task:get_received_headers
* [Feature] Store `for` part in received headers
* [Feature] Store enabled flag for webui session
* [Feature] Store error messages in ring buffer
* [Feature] Support compressed maps
* [Feature] Support excluding selected users from ratelimits
* [Feature] Support looking up NS records in lua_dns
* [Feature] Support modern style SURBL configuration
* [Feature] Support multiple hashes in delhash path
* [Feature] Support new messages in rspamc
* [Feature] Support requests without reads in lua_tcp
* [Feature] Support setting task message from Lua
* [Feature] Track visibility of HTML elements
* [Feature] Try to add CRLF when checking DKIM
* [Feature] Try to guess line endings when folding headers
* [Feature] Try to improve normalization function for bayes
* [Feature] Use FFI to optimize SA module
* [Feature] Use length based arguments for redis, allow lua_text as arg
* [Feature] Use more layers for fann and another normalization
* [Feature] User-defined ratelimits
* [Feature] Utility to convert fuzzy storage from sqlite to redis
* [Feature] Yield DMARC_DNSFAIL on lookup failure
* [Fix] Adopt fuzzy storage for flexible backends
* [Fix] Allow plain IP addresses in Rspamd maps
* [Fix] Another fix for brain-damaged hiredis
* [Fix] Another fix for rdns write errors
* [Fix] Another fix for rdns_make_request_full invocation
* [Fix] Another fix in DKIM canonicalization
* [Fix] Another memory leak plugged
* [Fix] Another try to deal with posix idiotizm
* [Fix] Another try to fix RDNS events processing logic
* [Fix] Avoid double frees in HEAD requests
* [Fix] Avoid extra symbols for RBLs
* [Fix] Banish table.maxn from Lua parts
* [Fix] Check for socket error before connection in lua_tcp
* [Fix] Correctly propagate redis timeouts to Lua
* [Fix] Do not add extra newline in MIME mode
* [Fix] Do not be cheated by system hiredis
* [Fix] Do not classify when a message has not enough tokens
* [Fix] Do not crash on redis errors
* [Fix] Do not distinguish NXDOMAIN and NOREC for monitored
* [Fix] Do not replan retransmits if merely one server is defined
* [Fix] Do not use headers to calculate messages digests
* [Fix] Don't force action in replies module for authenticated users/local networks
* [Fix] Explicitly ban default passwords in webui
* [Fix] Finally fix ambiguity between parsed and resolved spf elts
* [Fix] Fix 'decoded' value in task:get_header_full()
* [Fix] Fix DKIM calculations
* [Fix] Fix DKIM signing for messages with no newline at the end
* [Fix] Fix DNS request in monitored
* [Fix] Fix DNS write errors processing
* [Fix] Fix HTTP methods other than GET and POST
* [Fix] Fix PERMFAIL for v6/v4 ambiguities
* [Fix] Fix absurdic scores for HFILTER_URL_ONLY
* [Fix] Fix actions in rolling history
* [Fix] Fix actrie patterns
* [Fix] Fix applying of lua dynamic confg
* [Fix] Fix autolearning errors and redis cache
* [Fix] Fix bayes learn_condition
* [Fix] Fix build with the recent OpenSSL
* [Fix] Fix caching and compressed maps
* [Fix] Fix check plain text part
* [Fix] Fix crash on OpenBSD in `url_email_start`
* [Fix] Fix double free in SPF
* [Fix] Fix extraction of shingles from redis fuzzy storage
* [Fix] Fix false sharing for symbols in the cache
* [Fix] Fix float usage in util:get_time
* [Fix] Fix folding algorithm to deal with empty tokens
* [Fix] Fix format string
* [Fix] Fix format string usage in controller errors handling
* [Fix] Fix handling of '\0' in lua_tcp
* [Fix] Fix handling of HTTP HEAD methods
* [Fix] Fix hash creation
* [Fix] Fix hiredis stupidity
* [Fix] Fix implicit settings module settingsup
* [Fix] Fix interaction with lua GC to avoid craches
* [Fix] Fix ip_score module registration
* [Fix] Fix issue with empty messages and dkim
* [Fix] Fix issues with CGP helper
* [Fix] Fix issues with the recent SPF changes
* [Fix] Fix key name to load ANN correctly
* [Fix] Fix lua tcp module by saving `do_read` in callback data
* [Fix] Fix memory leak in client when using compression
* [Fix] Fix min_learns option
* [Fix] Fix on_finish scripts and async handlers
* [Fix] Fix options for SPF dnsfail symbol
* [Fix] Fix parsing includes and redirects in SPF
* [Fix] Fix parsing of lua comments with empty lines
* [Fix] Fix parsing of unquoted HTML attributes
* [Fix] Fix periodic events and redis
* [Fix] Fix processing of fuzzy learns from Lua
* [Fix] Fix processing of redirect in SPF includes
* [Fix] Fix processing of symbols when reject limit is reached
* [Fix] Fix refcounts when map is specified by IP
* [Fix] Fix rspamd{session} class in Lua API
* [Fix] Fix setting ratelimit key for 'ip' bucket
* [Fix] Fix some cases of TLD urls detector
* [Fix] Fix statconvert tool
* [Fix] Fix stats for backend-less classifiers
* [Fix] Fix training script for fann_redis
* [Fix] Fix variable in ann module
* [Fix] Fix various errors in lua dynamic conf plugin
* [Fix] Forget old ANN when max_usages is reached to avoid overtrain
* [Fix] Further canonicalization fixes
* [Fix] Further fixes for fann_redis prefixes
* [Fix] Handle failures for inactive pooled connections
* [Fix] Improve multimap info message
* [Fix] More fixes in ANN loading
* [Fix] More fixes to fann_redis
* [Fix] More issues in fann_redis
* [Fix] More spaces fix in DKIM signature
* [Fix] Multiple fixes to asn script, add IPv6 support
* [Fix] Multiple issues in fann_redis
* [Fix] No greylist rejected messages
* [Fix] One more attempt to fix lua_redis
* [Fix] One more check for readdir...
* [Fix] Params should be treated as a hash
* [Fix] Plug memory leak in regexp desctructor
* [Fix] Process headers only once
* [Fix] Properly handle nil values in ratelimit plugin
* [Fix] Really fix redis shingles check
* [Fix] Remove fann with incorrect layers count
* [Fix] Remove mentions of deleted include
* [Fix] Remove some incompatible functions
* [Fix] Settings: fix `authenticated` parameter (#886)
* [Fix] Skip MX check for authenticated users and local networks
* [Fix] Slightly fix ANN routines
* [Fix] Stop caching records with DNS failures
* [Fix] Treat all errors in redis_pool as fatal errors for a connection
* [Fix] Try avoid false-positives in HEADER_FORGED_MDN rule
* [Fix] Try to avoid race condition when using rrd
* [Fix] Try to reload redis scripts if they are missing
* [Fix] Unbreak once_received skipping for local networks
* [Fix] Unlock ANN on error
* [Fix] Use memmove for overlapping regions
* [Fix] Use real size instead of displayed for core limits
* [Fix] Use the correct macro to get the size of control
* [Fix] Various fixes for errors ringbuffer
* [Fix] Yield R_SPF_DNSFAIL if lookup of included record fails
* [Fix] mid: fix map initialization
* [Fix] mid: handle incorrect rgexps in the map
* [Rework] Add extract training data function to fann_redis
* [Rework] Add preliminary train tests
* [Rework] Add redis storage feature to fann_redis
* [Rework] Adopt fuzzy storage for abstract backend
* [Rework] Adopt plugins
* [Rework] First reiteration on fann scores
* [Rework] Implement loading/invalidating
* [Rework] Make lua_redis task agnostic
* [Rework] Make rspamd protocol messages useful
* [Rework] Massive removal of legacy code
* [Rework] More cleanup actions
* [Rework] Remove legacy code never used for classifiers
* [Rework] Remove outdated and unused lua_session module
* [Rework] Reorganize fuzzy backend structure
* [Rework] Reorganize the internal backend structure
* [Rework] Restore old fann_scores, move common parts
* [Rework] Rework and simplify rbl plugin
* [Rework] Rework parsing of DMARC records
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* [Feature] ASN module; support matching ASN/country in multimap
* [Feature] Add SPF method in spf return result
* [Feature] Add Yandex and Mail.ru forwarding rules
* [Feature] Add mempool maps in multimap
* [Feature] Add rule for identifying mail sent by eval()'d PHP code
* [Feature] Add support of stub DNSSEC resolver to rdns
* [Feature] Add task:get_digest method
* [Feature] Allow for more fine-grained scoring for ip_score
* [Feature] Allow to get digest of a mime part from lua
* [Feature] Allow to print message digest in logs
* [Feature] Fold DKIM-Signature header
* [Feature] Implement encrypted logs
* [Feature] Log URLs encrypted if we have log encryption pubkey
* [Feature] Pass authenticated bit to lua
* [Feature] Read redis backend statistics configuration from global section
* [Feature] Show the exact value matched for multima symbols
* [Feature] Store task checksum
* [Fix] Avoid setting limits when required elements are missing
* [Fix] DMARC: Fix alignment checking for subdomains
* [Fix] DMARC: deal with missing and spurious spaces
* [Fix] Defer insertion of results in ip_score to avoid skewing stats
* [Fix] Disable DMARC for local/authorized mail
* [Fix] Fix handling of proxied headers in controller
* [Fix] Fix hex printing of strings
* [Fix] Fix issue with spaces in maps
* [Fix] Fix parsing of forwarded IP
* [Fix] Fix reload in some plugins and workers
* [Fix] Fix reloading on SIGHUP
* [Fix] Fix some border cases for DKIM canonicalization
* [Fix] Fix url maps
* [Fix] Make dnssec configurable option disabled by default for now
* [Fix] rspamadm statconvert: force db to be a string
* [Fix] rspamadm statconvert: use db/password for learn cache
* [Rework] Rework flags in rspamd logger
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* [CritFix] Check hyperscan cache sanity before loading
* [CritFix] Fix setting of fuzzy keys (completely breaks fuzzy storage)
* [Feature] Add SARBL (sarbl.org) uribl
* [Feature] Add `--search-pattern` option to rspamd_stats
* [Feature] Add some sanity check for very long from/to log elements
* [Feature] Allow to create hashes from string in a single step
* [Feature] Fix order of pre and postfilters
* [Feature] Improve lua URLs API
* [Feature] Improve message about fuzzy rules
* [Feature] Pre-calculate blake2 digest for all parts
* [Feature] Print radix duplicate keys as IP addresses
* [Feature] Simple mechanism for disabling RBLs in local.d/rbl.conf
* [Feature] Use faster hash function for fuzzy storage
* [Feature] rspamd_stats: support log directory reading
* [Fix] Add sanity check for url filters
* [Fix] Do not show rmilter section as a fake metric in rspamc
* [Fix] Fix URL filters
* [Fix] Fix a stupid mistake in util.strequal_caseless
* [Fix] Fix blake2b hash of the string "rspamd"
* [Fix] Fix filename maps filter
* [Fix] Fix finding tld in util.get_tld
* [Fix] Fix multimap content filters
* [Fix] Fix returning boolean from Lua
* [Fix] Fix returning of REDIS_NIL
* [Fix] Try to deal with multiple workers terminated
* [Fix] Use forced DNS request when calling for lua_http
* [Rework] Rework multimap filters, add redis maps
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* [Feature] Add a special symbol for SPF DNS errors: R_SPF_DNSFAIL
* [Feature] Add correlations report in fuzzy stats
* [Feature] Add experimental CGP integration
* [Feature] Add method to get urls length in a text part
* [Feature] Add new methods to lua_html to access HTML tags
* [Feature] Allow all types of symbols to be added via __newindex method
* [Feature] Allow to create settings for authenticated users
* [Feature] Allow to get block content for HTML tags
* [Feature] Improve DNS failures when dealing with SPF
* [Feature] Properly implement R_WHITE_ON_WHITE rule
* [Feature] Remove old ugly rules
* [Feature] Rspamc can now add dkim signature in mime mode
* [Feature] Store content length for HTML tags
* [Feature] Support reacher set of HTML colors
* [Feature] Try to avoid FP for low contrast fonts detection
* [Fix] Add missing HTML colors
* [Fix] Add spaces to dkim signature to allow folding
* [Fix] Avoid returning NaN as score on scan
* [Fix] Decode entitles in href parts
* [Fix] Do not cache SPF records with DNS errors
* [Fix] Do not crash on cyclic depends
* [Fix] Do not insert HELO/HOSTNAME unknown when they are not passed
* [Fix] Do not set absent hostname to "unknown"
* [Fix] Do not stress redis with KEYS command (#791)
* [Fix] Fix DMARC_BAD_POLICY symbol
* [Fix] Fix HFILTER_URL module
* [Fix] Fix HFILTER_URL_ONELINE rule
* [Fix] Fix buffering in CGP integration
* [Fix] Fix colors propagation from parent nodes
* [Fix] Fix confusing OpenSSL API usage of i2d_RSAPublicKey
* [Fix] Fix dependencies id sanity check
* [Fix] Fix folding for semicolon separated tokens
* [Fix] Fix largest possible TLD behaviour
* [Fix] Fix last token folding
* [Fix] Fix length calculations in white on white rule
* [Fix] Fix multiple request headers structure
* [Fix] Fix multiple values headers freeing
* [Fix] Fix parsing of background color
* [Fix] Fix printing from field in log_urls
* [Fix] Fix processing of last element of DMARC policies
* [Fix] Further fixes for HTML colors
* [Fix] Further fixes for multiple values headers
* [Fix] Further fixes for white on white rule
* [Fix] Further fixes in HTML tags parsing
* [Fix] Ignore content type/subtype case
* [Fix] Increase score of R_WHITE_ON_WHITE
* [Fix] Parse CGP envelope data
* [Fix] Propagate colors in HTML
* [Fix] Restore multiple values headers in protocol
* [Fix] Restore multiple values in headers processing
* [Fix] Some more changes to tag's content length calculations
* [Fix] Some more fixes for low contrast fonts detector
* [Fix] SpamAssassin plugin: support check_freemail_header('EnvelopeFrom', [..])
* [Fix] Trigger HTML_SHORT_LINK_IMG on any external image
* [Fix] rspamd_stats: remove deprecated defined(@array)
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* [CritFix] Fix catena passwords validation
* [CritFix] Fix crash when the first received is faked
* [Feature] Add DMARC_BAD_POLICY symbol when DMARC policy was invalid
* [Feature] Allow for matching hostnames in multimap (#773)
* [Feature] Allow for setting action based on DMARC disposition
* [Feature] Allow limiting of the inbound message size
* [Feature] Allow maps with multiple symbols and scores
* [Feature] Allow regexps in the emails maps
* [Feature] Allow to register metric symbols from multimap
* [Feature] Allow to reset redis tokens instead of appendig values
* [Feature] Allow to store strings in radix maps
* [Feature] Check UTF validity when there are utf regexps in a map
* [Feature] Correctly work when there is no hard reject action
* [Feature] Implement dependencies for maps
* [Fix] Another effort to unbreak sqlite locking
* [Fix] Avoid crash when closing mmapped file
* [Fix] Do not break history on NaN in required score
* [Fix] Ensure that hyperscan cache written is written properly
* [Fix] Filter NaN from scores in history
* [Fix] Fix DNSBL maps
* [Fix] Fix another locking issue in sqlite
* [Fix] Fix another locking issue with mapped files
* [Fix] Fix deadlock in mmaped file stats
* [Fix] Fix dependencies in multimap plugin
* [Fix] Fix emails module configuration
* [Fix] Fix greylist plugin (#755)
* [Fix] Fix greylisting plugin variable usage
* [Fix] Fix installed permissions for rspamd_stats
* [Fix] Fix locking in mmapped statistics
* [Fix] Fix paths in tests
* [Fix] Fix prefilter mode for multimap
* [Fix] Forgot to commit leftover changes
* [Fix] Really fix local.d includes
* [Fix] Restore selective greylisting behaviour
* [Fix] Set max size on per connection basis
* [Fix] Use temporary storage for hyperscan cache
* [Rework] Remove systemd socket activation
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
| |
https://rspamd.com/announce/2016/07/25/rspamd-1.3.0.html
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Another fix for exim workaround (#637)
* Fix unencrypted passwords processing in the controller
* Fix setting path for lua (#652)
* Fix usage of rdns reply structure (#654)
* Use file lock in logger to avoid deadlocks
* Add `application/octet-stream` mime type for `pdf` extension (by @moisseev)
* Implement new automata to skip empty lines for dkim signing (#651)
* Fix parsing of missing classes
* Clarify some rspamc arguments (by @fatalbanana)
* Correct suppress spelling
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Slightly reduce weights of rules with high FP rate
* Add workround for rspamd-1.3
* Fix possible FP in TRACKER_ID rule
* Simplify MISSING_MIMEOLE rule
* Add workaround for gmime CTE stupidity
* Fix mime headers processing
* Fix false positive URL detections in text parts
* Fix Exim shutdown patch
* Enable workaround for exim mailbox format
* Backport shingles static test
* Fix levenshtein distance calculations
* Fix max_train setup in ANN module
* Fix redis structure by adding {NULL, NULL} member
* Fix build with unmodified LibreSSL opensslv.h
* Repair optional dependencies
* Really skip filters in case of pre-result set
* Restore the intended pre-filters behaviour
* Fix ipv6 mask application
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix parsing of URLs in texts
* Fix creating of URLs from LUA
* Fix some more URL detector issues
* Fix unit tests
* Fix JIT compilation for PCRE2 expressions
* Fix JIT usage for PCRE2
* Fix UTF8 mode in PCRE2
* Add workaround for pre-historic compilers (#605)
* Fix and rescore R_PARTS_DIFFER logic
* Properly set lua paths for tests
* Fix SA rawbody processing - exclude top part
* Store text parts content with newlines stripped
* Properly support SA body regexps
* Fix body rules in SA plugin
* Fix setting of score for parts differ
* More fixes to parts distance calculations
- Use hashed words instead of full words for speed
- Improve levenstein distance calculations and penalise replaces
- Always return number from 0 to 1
- Use g_malloc instead of alloca
* Fix percents output in R_PARTS_DIFFER
* Plug memory leak in dkim module
* Plug minor memory leak in regexps creation
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
| |
* Plug an important memory leak in headers getting code
* Remove some bad domains from whitelists
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Implement new multipattern matcher that uses hyperscan if possible
* Use mutlipattern for lua_trie code
* Add utility methods for multipattern
* Use multipattern in url matcher
* Add escape functions for hyperscan
* Allow to optimize lua -> C transition by flattening table args
* Optimize hot paths in SA plugin
* Optimize rspamd_re_cache_type_from_string
* Allow empty tries
* Fix extraction of URLs from Subject
* Allow to have different flags for different patterns in multipattern
* Add common directory for hyperscan cache to config
* Implement caching for hyperscan multipattern
* Attach domain part to `R_SUSPICIOUS_URL` (by @moisseev)
* Allow multipattern scans to be nested for the case of hyperscan
* Simplify SURBL redirector search code and avoid ac_trie
* Add two way substring search algorithm
* Avoid acism usage to find gtube pattern
* Fix processing of empty headers
* Allow to disable pthread mutexes on broken platforms
* Make web interface not send password in query strings (#585) by @fatalbanana
* Add maximum delay to ratelimit module
* Backport fix for empty files inclusion from libucl
* Fix settings id setup
* Add min_learns option to classifiers
* Use more clever to utf8 conversion strategy
* Fix disabling of virtual symbols in the settings
* Rework settings to work properly in metric-less configuration
* Set the default limit for classifier
* Fix ttl based expiration from LRU cache
* Rework DKIM module to use OpenSSL for digests
* Fix mailto urls parsing with hyperscan
* Do not set obscured flag for urls starting with spaces
* Fix crash on redis learn
* Fix ratelimit ctime setting
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* New DCC module (by @smfreegard)
* Rework whitelist module:
- Now we check different elements for different checks
- MIME from for DMARC
- DKIM signature domain for DKIM
- SMTP from or HELO for SPF
* Fix regexps results combination (*critical*)
* Fix issue with expressions processing (*critical*)
* Optimize strlcpy for aligned input
* Add support of half-closed connection in lua_tcp
* Allow to print compact json in client
* Save required score in history (#581)
* Allow to attach file descriptors to control commands
* Allow to send descriptors from workers to main
* Allow to attach fd when broadcasting to workers
* Implement log pipe feature for rspamd logs analysis
* Add `log_helper` worker
* Add `URIBL_SBL_CSS` (by @smfreegard)
* Add worker scripts functionality
* Add on load hooks for rspamd_config
* Add lua scripts for log_helper worker
* Add generic maillist detector (#584)
* Implement FANN autolearn using log_helper worker
* Rework metrics configuration to allow includes
* Change default value of forced removal in composite rules
* Allow to use assembly version of blake2b on x86 cpu
* Use less precise (but faster) clock if possible
* Insert redirected URL to the urls list
* Allow to get and set callback data for rspamd symbols
* Add binary heap implementation
* Use binary heap for expire algorithms in the hash
* Use `least frequent used` expiration strategy
* Allow to get mime headers from a task
* Add support for mime headers in `regexp` module
* Update Exim patches (by @fatalbanana)
* Allow building rspamd with jemalloc
* Save multipart boundaries
* SA plugin changes:
- Properly handle MIME headers
- Fix eval:check_for_missing_to_header rule
- Implement SA compatible body regexps
- Use sabody rules in SA plugin
* LUA API changes:
- Add util.get_ticks function
- Add util.stat function
- Add task:get_symbols_numeric method
- Add method to get number of symbols in the cache
- Add lua methods to get redirected urls
- Allow to get callbacks for lua symbols
- Add config:set_symbol_callback function
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Use HTTP Content-Type on non mime input if possible
* Save log level when compressing log messages
* Further rework of composite rules (add '^' prefix)
* Add tracking for rspamd expressions
* Store actions limits in metric result
* Fix parsing of include/redirect with many records in SPF
* Add method to disable symbols execution in the cache
* Allow to disable checks from settings
* Allow to select settings by id in HTTP query
* Find URLs with '\r' and '\n' inside href attribute
* Implement vectored mode for hyperscan (experimental)
* Improve client connection errors diagnostics
* Allow to edit new files with signtool
* Improve hashes performance on 32 bit platforms
* Fix sorting of limits
* Remove slow and unused rules `INVALID_EXIM_RECEIVED*`
* Add expression:process_traced lua method
* Allow tables in task:insert_result
* Save trace for SA metas
* Do not parse broken TLD parts in URLs
* Investigate many border cases in URLs parser
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add list support to `mime types` module configuration (by @moisseev)
* Allow symbols params to be printed in logs
* Fix `MIME_BAD_ATTACHMENT` false positives for MDN/DSN
* Fix crashes on arm32
* Do not classify message if some class is missing
* Fix cryptobox cleanup
* Remove multipart/report from bad mime types (#569)
* Improve logging for fuzzy hashes
* Show map URLs in webui
* Sort symbols in webui
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* New dynamic updates plugin
* Regular expressions map support
* Faster radix trie algorithm
* Faster siphash for AVX2 supporing CPUs (used in fuzzy hashes)
* PCRE2 support
* Allow quoted and slashed keys in map
* Add proper support of DNS resolvers balancing (#552)
* Rework includes and configuration system for better local changes support
* New keypairs framework for signing and encryption
* Added support for dynamic modules and workers
* Allow to dump configuration with help comments
* Rework once_received module
- Fix priority for `good_hosts`
- If a good host has been found do not add once_received symbols
- Fix priorities for strict once_received
- Add ability to whitelist IP addresses
* Implement support of signed maps for HTTP and file maps
* Add command to sync fuzzy storage (#533)
* Rework system of symbols and actions registration
It is possible now to use priorities when adding symbols to metrics and
override scores for symbols with lower priority with the scores with
high priority.
* Add auth support and db selection for redis stats
* Improve composite rules application
* Add ignore_received option
* Fix critical issue with inconsistent resorting
* Fix `all` in spf redirects
* Add punycoded versions for IDN domains (#554)
* Improve sorting order for symbols cache
* Add lockless logging for processes management
* Allow to specify flags for metric symbols
* Load images height and width from style attribute (#538)
* Override DNS requests limits for SPF and DKIM
* Fix resetting symbols to their default values in WebUI
* Improve configuration agility for redis stats
* Allow to set db and password for redis in stat_convert
* Import the latest libucl
* LUA API changes:
- Add rspamd_version function to LUA API
- Add lua_cryptobox module
- Add lua_map module
- Add task:set_metric_action lua API method
- Fix race condition in lua_tcp module
- Fix a lot of issues in lua_redis module
- Rework and abstract lua maps API
- Add util.strlen_utf8 lua function
- Add lua functions for caseless comparison
- Allow optional symbols registration
- Add config:add_map table form method, add regexp maps
- Add task:has_urls method
- Add task:has_flag method
- Add html tags methods to lua_html
- Add task:get_dns_req
* Plugins changes:
- Add support for WLBLEval SA plugin
- Use caseless comparison in SA and DMARC plugins
- Allow SA plugin to set scores for rspamd symbols
- Add regexp maps support to multimap
- Allow filenames match in multimap
- Add more filters for the existing map types
- Fix html images rules to reduce FP rates
* New rules:
- LONG_SUBJ - too long subject
- MIME_BAD_ATTACHMENT - bad attachement type
- RDNS_NONE - no reverse DNS record for sender's IP
- Fix MISSING_MIMEOLE rule for modern OE
* Many other bugfixes, memory leaks plugs thanks to:
- Coverity scan
- New gcc-6 warnings
- valgrind manual iterations
* Documentation improvements:
- FAQ list: https://rspamd.com/doc/faq.html
- Reworked quick start guide
- Added documentation for all active modules
* Other changes:
- Dropped Ubuntu Vivid support
- Added Ubuntu Xenial support
- Rework build system for rspamd and rmilter
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Print traceback on lua errors in lua config
* Fix leaks in lua error paths
* Improve 'R_EMPTY_IMAGE' rule
* Fix metas memoization in SA plugin
* Properly set `flag` in fuzzy replies
* Fix arguments order
* Fix issue with out-of-boundary reading
* Fix issues found by coverity
* Same result checking error found by coverity
* Fix varargs processing (found by coverity)
* Fix error in printing hex
* Reduce weights for some hfilter patterns
* Add aliases for task:get_from_ip:
- task:get_addr
- task:get_from_addr
- task:get_ip
* Rework once_received module
- Fix priority for `good_hosts`
- If a good host has been found do not add once_received symbols
- Fix priorities for strict once_received
- Add ability to whitelist IP addresses
* Fix `MISSING_MIMEOLE` rule for modern OE
* Treat meta tags as embedded tags (#501)
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix DSN rules when SMTP from is unavailable
* Fix statconvert routine to avoid lua module usage
* Set a sane quark for configtest to avoid NULL to be printed in logs
* Support c11 if available
* Fix parsing of ip:port strings
* Add more diagnostic for lua subr errors
* Fix task:set_from_ip lua method
* Add basic routines for digital signatures
* Add tool for digital signatures
* Add plain open file API method for atomic open
* Fix parsing nested braces inside logger vars
* Pre filters now actually skip processing
* Add pre-filter mode for multimap
* Switch to apache 2 license
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix stat_cache closing
* Add checkpoints to sqlite3 learn cache
* Do not recompile lua generated headers all the time
* Increase number of messages learned
* Fix issues with dual stack and hfilter
* Disable MID checks for hfilter by default
* Fix cache definitions in multiple classifier and no type
* Don't crash if learn cache failed to initialize
* Fix googlegroups support in maillist plugin
* Rework flags LUA API:
- Allow to check for a specific flag
- Add `learn_spam`, `learn_ham` and `broken_headers` flags
- Unify internal functions
* Add `BROKEN_HEADERS` rule
* Add support for forged confirmation headers (by @AdUser)
* Allow `any`, `mime` and `smtp` for get_from/get_recipients
* Add mime types checking plugin
* Add rule to detect spammers attempts to cheat mime parsing
* Rework parsing of IP addresses in configuration (better IPv6 support)
* Add `util.parse_mail_address` function to LUA API
* Add lua sqlite3 module
* Implement synchronous redis call
* Ratelimit: avoid possible indexing of nil value (Fixes #498) (by @fatalbanana)
* Add stat_convert command to convert stats tokens from sqlite3 to redis
* Implement redis advanced lua api with pipelining
* Fix memory leak on redis stat (#500)
* Fix user/language learn count in sqlite statistics (#496) (by @fatalbanana)
* Fix build with custom pcre
* Fix fuzzy relearning (#498)
* Improve planning of asynchronous tasks
* Show slow rules in log
* Add warning for slow regexps
* Add base32 decode/encode routines to lua util
* Allow converting of learn cache from sqlite to redis
* Add methods to check if a messages has from/rcpts
* Improve and fix multimap plugin:
- Restore 'header' maps
- Add filters for headers
- Add 'email:addr', 'email:user', 'email:domain' and 'email:name' filters
- Add generic regexp filters
* Disable reload command in rc scripts
* Improve runtime CPU dispatcher for libcryptobox
* Add preliminary support of digital signatures via ed25519
* Add detection for RDRAND support
* Print configuration of crypto on start
* A in SPF presumes AAAA lookup as well
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix duplicated XBL symbol
* Reduce log severity for ratelimit missing servers
* Fix XBL composite to avoid duplicate symbols
* Reduce weight of URL_ONLY rule due to FP rate
* Disable fuzzy hashes from the metadata for now
* Fix processing of empty messages (#486)
* Always treat DNS timeouts as temporary fail for SPF
* Fix issue with SPF double IP stack (#483)
* Use X-Forwarded-For when checking secure_ip (#488)
* Fix hash calculation for sqlite stats
* Fix memory corruption on punycode
* Fix strings allocation in punycode
* Fix error message (#491)
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Incompatible change: sqlite3 and per_user behaviour:
Now both redis and sqlite3 follows the common principles for per-user
statistics:
1) If per-user statistics is enabled check per-user tokens ONLY
2) If per-user statistics is not enabled then check common tokens ONLY
If you need old behaviour, then you'd need to use separate classifier
for per-user statistics.
* Implement redis statistics backend and cache
* Implement autolearning for statistics
* Reworked statistics architecture from scratch
* Add hyperscan (https://github.com/01org/hyperscan) engine for regular
expressions:
- add lazy loader for hyperscan databases
- rework regexp cache to have joint pcre/hyperscan scanning
- implement hyperscan pre-filter support
- add compilation guards for bad expressions
- implement `rspamadm control recompile` command
- implement hyperscan cache monitoring
- slides: <https://highsecure.ru/rspamd-hyperscan.pdf>
* Implement flexible task logging
* Rework fuzzy worker:
- it is now possible to run multiple fuzzy workers;
- implement lazy writing as sqlite3 is bad at concurrent writing;
- add retries for simple sql commands in fuzzy backend;
- use fine-grained transactions for fuzzy;
- implement new multi-pubkeys mode;
- allow encrypted only storages;
- rework statistics for fuzzy;
- add `rspamadm control fuzzystat` command for extended statistics;
- implement human readable output for the previous command;
- add condition script for learning fuzzy storage;
* Various fixes to SPF:
- fix `redirect` records;
- fix domains when parsing mx/ptr/a records in includes/redirects;
- fix issues with multiple addresses in SPF records;
- ignore SPF results in case of DNS failure;
- adjust TTL of records when resolving subelements of SPF records;
- always select `v=spf1` line if it is available
- do not cache records with DNS failure in subrequests;
- ignore records with temporary fails during subrequests resolving;
- fix `RDNS_RC_NOREC` support;
* Add clang plugin for static analysis:
- implement static checks for `rspamd_printf` format strings;
* Add 'allow_raw_input' option for non-mime messages
* Recognize types using libmagic
* Fix parsing of IPv6 received headers
* Add new interface of communication between workers in rspamd
* Add support for named socketpairs
* Don't write URLs by default as it is too verbose
* Set status for HTTP replies
* Try load `rspamd.conf.override`
* Implement words decaying for text parts to limit many checks
* Improve support of SA rules and plugins:
- add check_for_shifted_date and check_for_missing_to_header eval rules;
- add 'check_relays_unparseable' support;
- add `check_for_mime('mime_attachement')` function;
- use new re_cache interface for all SA rules;
- add support for `Mail::SpamAssassin::Plugin::MIMEHeader`;
- add support of 'special' SA headers to `exists` function;
- fix issue when SA metas contain other metas;
- fix freemail rules;
* Many fixes to the URL parser
* Match any newline character in regexps
* Fix resolving of upstreams and detection of poor IPv6 configurations
* Parse upstreams selection algorithm from the configuration line
* Add `reresolve` command to the control interface
* Generate fuzzy hashes from task metadata (URLs and headers)
* Add method to check if IP is local and `local_addrs` option
* Implement forced timeout for delayed filters
* Disable fast path of pcre-jit as it seems to be broken
* Bayes fixes:
- new normalizer function;
- really use weights of tokens from the OSB algorithm;
- restore multiple classifiers support;
* Rules changes:
- add `R_SUSPICIOUS_URL` rule that detects obfuscated URL's;
- improve empty image rule;
- rework `FORGED_RECIPIENTS` rule;
- reduce weight of `SUSPICIOUS_RECIPS`;
- fix `*_NORESOLVE_MX` symbols in hfilter;
- add `SUBJ_ALL_CAPS` rule with support of UTF8
- add spamhaus SBL to uribl
- fix `SUSPICIOUS_RECIPS` and `SORTED_RECIPS` rules
- remove `R_TO_SEEMS_AUTO` as it generates a lot of FP;
- add new Message-ID regexp for Thunderbird (by @moisseev);
* Plugins changes:
- allow ratelimit plugin to set symbol instead of pre-result
- support IP DNS black lists for URIBL (e.g spamhaus SBL);
- drop deprecated SURBL bits (by @fatalbanana)
- rename `JP_SURBL_MULTI` to `ABUSE_SURBL` (by @fatalbanana)
- add `SURBL_BLOCKED` (by @fatalbanana)
- add `CR_SURBL`
- SURBL: allow fallthrough to default symbol (by @fatalbanana)
- Settings: fix IP match (by @fatalbanana)
- SURBL: add missing symbols to metric (by @fatalbanana)
- allow processing images urls for SURBL
- unconditionally disable SPF for authenticated users and local networks
* Rework ratelimit plugin
- switch to `rates` instead of old and stupid strings to setup;
- check if a bucket is zero and disable the corresponding limits'
- turn off all buckets by default;
- check either `rcpt` or `user` buckets, not all together'
- document new `rates` and `symbol` options;
- inform user about what buckets are used in the configuration;
* Add neural network **experimental** plugin
* Add a sample script to learn neural network from rspamd logs
* Add documentation strings support to rspamd:
- add strings for the main configuration options;
- document workers options;
- add internal plugin options;
- create `rspamadm confighelp` routine;
- implement human readable output for the previous command;
- add subtree search support;
- add keyword search support;
* Documentation improvements, tutorials section, statistics description
* Many other minor and major bugfixes not noted here
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix spf redirects
* Fix domains when parsing mx/ptr/a records in includes/redirects
* Fix unfolded base64 encoding
* Fix GError use-after-free
* Do not rewrite the original url when using redirector
* Fix parsing of fragment in urls
* Fix processing of HTML tags
* Improve empty image rule
* Avoid long double type
* Fix tokens weights in OSB algorithm
* Improve debugging for bayes
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix settings application (#416)
* Fix another issue with fixed strings
* Fix hash function invocation
* Use the proper string for make_dns_request in lua_http
* Fix scan time output
* Update webui:
- fix labels for greylisting
- fix dimension of scan time
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
| |
* Emergency fix in keyed blake2 to fix fuzzy hashes and encrypted password
* Support passwords longer than 64 symbols
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add function to traverse AST atoms
* Allow dependencies on rspamd symbols for SA metas
* Fix memory corruption when timeout is removed in fuzzy check
* Fix encrypted fuzzy add processing
* Avoid use-after-free in controller session destructor
* Use session pool instead of task pool in fuzzy check
* Fix assembly in i386 mode (#413, #412)
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Plugged memory leaks in internet address object & html parser
* Fixed static build
* Fixed multiple sigchld processing
* Fixed deletion of signal events after event processing loop
* Fixed build on ARM (#404 - reported by @Gottox)
* Fixed setting the default mask for SPF.
* Fixed sanitisation of HTTP query values
* Fixed parsing of the last header in encrypted HTTP messages
* Additions and fixes for test suite & benchmarks
* Added openssl aes-256-gcm support to libcryptobox & HTTP server
* Implemented support for starting multiple HTTP servers
* Implemented batch accept in HTTP server
* Added module to get data from HTTP headers (#285 - reported by @msimerson)
* Added `rspamadm control` command
* Added ability to sort counters output.
* Added ability to specify custom headers for rspamc client
* Fix architecture detection
* Converted history storage to the UCL format
* Allow flexible number of rows in history
* Fix action badges in WebUI
* Add universal cryptobox hash API
* Migrated to the optimized blake2b implementation adopted from Andrew Moon
* Allow explicit loading of specific modules
* Always load settings module
* Allow to add symbols from settings
* Fix double free in the controller fuzzy learn command
* Avoid endless loop when cannot open sqlite db
* Updated libucl
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix build on i386
* Update CentOS7 service file patch (by @fatalbanana)
* Fix path to rspamadm in Debian init script (by @fatalbanana)
* Fix broken '_SC_GETPW_R_SIZE_MAX' on FreeBSD
* Fix portability issues
* Use cryptobox chacha for libottery
* Better support of 32 bit builds
* Fix header name tokens setup
* Fix levenstein distance method for words
* Add workaround for old libevent (#400)
* Fix microseconds in termination timer
* Fix some more issues with fixed strings
* Explicitly test CPU instructions even after CPUID call
* Do not check out of boundary memory
* Do not output broken emails
* Fix unknown symbols registration
* Handle SIGILL using longjmp
* Block signals when exiting event loop
* Fix incorrect allocation size
* Slightly optimize alignment
* Restore rspamd -t for compatibility
* Add more sanity checks for emails
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add rspamd control interface:
- support `stat` command to get runtime stats of rspamd workers
- support `reload` command to reload runtime elements (e.g. sqlite3 databases)
* Rework curve25519 library for modular design:
- add Sandy2x implementation by Tung Chou
- fix CPU detection for variables loading assembly
- add testing for curve25519 ECDH
* New fixed strings library
* Add `R_SUSPICIOUS_IMAGES` rule
* Enable mmap in sqlite3
* Use new strings in the HTTP code
* Improve google perftools invocation
* Improve performance profiling in http test
* Reorganize includes to reduce namespace pollution
* Allow specific sections printing in configdump command
* Rework workers signals handlers to be chained if needed
* Update socketpair utility function
* Add control_path option for rspamd control protocol
* Fix ownership when listening on UNIX sockets
* Rework signals processing in main
* Remove extra tools from rspamd (they live in rspamadm now)
* Remove global rspamd_main
* Add global timeout for the overall task processing (8 seconds by default)
* Sanitize NULL values for fuzzy backend
* Store NM between encrypt/decrypt
* Add textpart:get_words_count method
* Fix generic DNS request in lua
* Tune hfilter weights
* Add support of IPv6 in hfilter
* Fix parsing of HTTP headers with IP addresses
* Sync with the recent libucl
* Various minor bugfixes
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add configdump routine to rspamadm
* Implement retransmits for fuzzy_check plugin
* Fix events processing for learning anf checking fuzzy hashes
* Avoid dependency on unneeded and uncompatible glib include
* Add `historyreset` command to the controller
* Fix loading of tokenizer config from dump (#389)
* Add sorting hints for the history
* Allow custom lua scripts for users/languages extraction (#388)
* Do not add FORGED_RECIPIENTS when 'To' is missing (#387)
* Do not add R_UNDISC_RCPT when 'To' is missing (#387)
* Add encryption to fuzzy check plugin
* Add encryption for fuzzy storage
* Add new epoch for encrypted fuzzy request
* Add encryption for `rspamd.com` storage
* Remove gmime processing for LDA mode as it is deadly broken
* Add routine to find end of headers position in mime messages
* Fix LDA headers folding
* Init libraries in rspamc client as well to avoid locale issues
* Avoid collision with locally installed includes
* Allocate and free memory with the same allocator in rspamadm (#385)
* Preserve expired fuzzy hashes counter
* Improvements in webui:
- Add favicon.ico
- Rework history table
- Fix sorting for the history
- Migrate to bootstrap 3 and jquery 2
- Fix css bugs
- Add glyphicons
- Add reset history
- Improve history buttons
- Redraw graph to avoid display issues
- Webui is now MIT licensed to match licensing policy of rspamd
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix piechart clean slice (#380)
* Fix controller crashes when GString is reallocated (#381)
* Correctly set locale before start
* Set C locale for numeric values
* Add rspamadm routine:
- add `pw` command to manage passwords
- add `help` command for displaying help
- add `configtest` command to check configuration files
- add `keypair` command for generating encryption keys
- add `fuzzy_merge` routine to merge fuzzy sqlite databases
- add a simple manual page for rspamadm
* Allow metric registration for composite expressions
* Add strict mode for configtest
* Add logger counters
* Save and show learned messages count (#383)
* Add `no_stat` flag
* Add `task:set_flag` and `task:get_flags` (#382)
* Enable foreign keys in sqlite3
* Remove orphaned shingles from fuzzy storage
* Optimize synchronization steps for fuzzy storage
* Allow delayed conditions registration
* Add lua API for conditions registering
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix critical bug in webui that prevents password from being sent
* Rework webui view:
- Switch to d3.js for graphs
- Improve piechart look
- Rework colors for piechart
- Fix layout for symbols
- Fix refresh button
* Add descriptions for whitelist maps
* Fix build on arm (#379)
* Fix issue with the last element in the radix trie
* Add more tests for radix trie algorithm
* Allow to extract URLs from query strings of other URLs (#361)
* Initialize rrd fields before writing to file
* Fix double free if no password has been specified
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add writing to rrd from the controller
* Fixed lots of bugs in rrd code
* Adopt new DNS API in hfilter plugin (by @AlexeySa)
* Allow only one controller process to manage rrd file
* Set event base for fuzzy calls
* Improve fuzzy IO errors logging
* Add rra extraction function to rrd library
* Add graph handler to the controller
* Cache correct passwords to avoid too high CPU usage when working with webui
* Controller sockets are owned by router do not export them to task
* Optimize logging by skipping hash table search if it's empty
* Fix loading issue whith broken statfiles
* Print assertions from glib to rspamd logger
* Load legacy `lua/rspamd.local.lua`
* Update webui with some fixes to learning and scanning
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Rework symbols processing:
- Improve sorting logic for symbols
- Organize processing into multiple stages
- Added asynchronous watchers for symbols
- Added ability to organize dependencies between symbols
* Fixed URL redirector:
- Use optimized POE loop
- Organize dependencies
- Fix startup
* New sqlite3 backend:
- Allow to have per-languages and per-user statistics
- Allow sqlite3 to be used as statistics backend
* Store tokenizer configuration within statfiles
* Improve bayes statistics:
- Use headers and images metainformation in bayes
- Suggest using of pre-processed tokens for statistics
- Fix tokens normalization for OSB algorithm
* Rewrite url parsing:
- Fix numerous issues with url extraction and normalization
- Fix mailto urls
* Fix settings plugin to allow custom actions scores
* Improve rbl plugin
* Allow capturing patterns in rspamd lua regexp library
* Add GTUBE support
* Fix spamc legacy support
* Add DKIM support to RBL module
* Fix issues with multiple DKIM signatures
* Fix issue if rspamd cannot create statfiles (#331)
* Rework parts and task structure:
- Now text_parts, parts and received are arrays
- Pre-allocate arrays with some reasonable defaults
- Use arrays instead of lists in plugins and checks
- Remove unused fields from task structure
- Rework mime_foreach callback function
- Remove deprecated scan_milliseconds field
* Add ip_score plugin support (not enabled by default):
- Can check for asn/country and network using DNS lookups
- Can store and load reputation from redis server
* Improve PARTS_DIFFER rule to count merely different words
* New HTML parser:
- Parses HTML parts using a set of state machines
- Extracts useful data and exports it to lua functions:
+ Styles
+ Images
+ URLs
+ Colors
+ Structure elements
- Added HTML rules for some checks
* New version of LUA DNS API
* Table versions of many functions in LUA API
* Improve rspamc client:
- Print execution time
- Allow executing of external commands and passing output to them
- Allow mime output mode when rspamc alters message according to rspamd
checks and send it to an external command or stdout
* Allow scanning of local files using HTTP requests
* Rework configuration system:
- Rules are now moved from the $CONFDIR to $RULESDIR to avoid ambiguity
- All modules configurations are now split in $CONFDIR/modules.d/* to
simplify upgrades
- Move hfilter to plugins
- Allow plugins and rules to define default scores to simplify metrics
setup
- Include overrides for all modules to honor local/automatic parameters
- Tune scores for many modules
* Rework and enable DMARC plugin
* Add whitelist plugin for SPF/DKIM/DMARC based whitelisting
* Add some common domains to whitelists shipped with rspamd
* Rework logging:
- Now each log entry supports module name and a `tag`. Tag is used to
identify unique objects (such as tasks) when checking log files
- It is possible to turn on debugging for the specific modules
- Systemd logging is fixed
* Improve spamassassin plugin.
- Now headers are matched more like SA
- Improve support of Message-ID
- Add support of ToCc header type
- Fix :addr and :name in headers regexps
* Resurrect rrd support code
* Save controller stats between restarts
* Fixed tonns of bugs
* Added tonns of minor improvements and features
* Added more unit tests
* Create functional tests framework
* Added documentation for missing modules
* Added rpm/deb repositories and scripts
* Updated WebUI and libucl externals
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix critical bugs in tokenization algorithm
* Write unit tests for tokenization
* Add documentation for lua_tcp
* Switch off legacy tokenization by default.
* Fix critical bugs in words normalization
* Add lua bindings to tokenizer.
* Implement storing of HTTP headers inside task
* Add lua API to accerss HTTP headers data
* Implemented base64 encoding suitable for MIME
* Use caseless hash and equal functions for HTTP request headers.
* Improve debian architectures support (by @dottedmag)
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
| |
* Revert incorrect regexp change that broke the default rules
* Fix lua_tcp module
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix error on spawning unique workers.
* Add preliminary version of generic LUA TCP requests API.
* Use lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc)
* Fix fuzzy mime strings with only type.
* Improve thunderbird sanity checks.
* Fix critical bug on matching regular expressions.
* Make hiredis optional dependency.
* Fix multiple bugs in daemon reloading
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Restore utf8 validation for regular expressions to avoid crashes
* Fix symbols displaying in the interface
* Add symbol groups to the interface
* Fix maps ID parsing in the controller
* Add multimap and regexp modules documentation
* Backport fixes from libucl
* Fix debian package (by @dottedmag)
* Rework XXH32 invocations
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add support of the fast and secure protocol level encryption:
- curve25519 is used for key exchange;
- chacha20/poly1305 cryptobox construction for bulk encryption;
- zero latency overhead;
- encrypting and balancing HTTP proxy worker
* Rework expressions and create new expressions library:
- aggressive optimizations based on the abstract syntax tree;
- abstract expressions support (regular expressions, functions, lua modules
composites and so on)
- New comparision and '+' operators support
- New greedy algorithm to minimize execution time of expressions and
all symbols
- Dynamic expressions benchmark and reoptimizations
* Many improvements to the LUA API:
- reworked logger module allowing to do pretty print of the most of lua
types (including tables and userdata classes)
- reworked lua redis and lua HTTP to support more features
- added opaque type for passing large text chunks without copying
- new regexp module with many auxiliary functions (e.g. `re:split`)
* LuaJIT is now the default requirement for rspamd allowing to speed up lua
execution by a large margin (however, plain lua is still supported)
* New plugins:
- spamassassin rules plugin that allows to load and re-use the most of
SA rules natively
- DMARC plugin that evaluates SPF and DKIM policies to the domain policies
- many old plugins has been reworked to implement new features and improve
stability
* New aho-corasic trie implementation from @mischasan that allows to load and
use hundreds of thousands of patterns with no influence on load
* Support of PCRE JIT and PCRE JIT fast path modes that significantly improves
the performance of regular expressions if supported by PCRE
* New URLs parser and extractor:
- removed legacy code that was useless for url finding
- reworked algorithms of URL parsing for more precise and accurate results
- added top-level-domains tree from http://publicsuffix.org
- improved emails parsing
- removed many phishing false positives due to TLD tree check
* New statistics infrastructure:
- created a separate layer of statistic library
- improved OSB-Bayes by re-weighting tokens according to the original
academic paper and `crm114` implementation, which reduced false positives
rate significantly
- created learn cache to avoid double learning of statistics and providing
an efficient way to re-learn class for a message
- created abstract layers for different statistics backends
- implemented new tokenization algorithms with fast or secure (siphash)
hashes to generate statistics features
* Reworked utf8 tokenization that previously corrupted all UTF8 words (minor
incompatibility with old fuzzy hashes with utf-8 symbols)
* SPF module has been completely rewritten to support complex cases of
`include` and `redirect` within SPF records
* DKIM module now supports multiple signatures
* Controller passwords can now be stored encrypted by `PBKDF2-HMAC` in the
configuration file
* Many hand-written HTTP clients has been replaced with the common rspamd
http module
* New test framework:
- import lua `telescope` test framework
- add unit tests for many rspamd modules and routines
- create a unit test for each possible bug found
- use luajit ffi for testing C code
- added preliminary support of functional testing by creating tasks from lua
* Randomize hash seed to avoid certain hash tables vulnerabilities
* Documentation improvements:
- added documentation for the vast majority of rspamd modules
- added documentation for rspamd protocol
- added documentation for the most of rspamd LUA extensions
* Fixed tonns of bugs and memory leaks
* Added tonns of minor features
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* New fuzzy check logic:
- use shingles algorithm for fuzzy matching
- use blake2 instead of md5 for larger output space
- combine fuzzy and strict matching
- allow to organize private storages by means of keys
- preserve compatibility with previous versions
* New fuzzy storage:
- use sqlite instead of own memory based hash tables
- rework commands interface
- add conversion from the old format
- add fuzzy match by shignles
- support old rspamd versions
* Add lemmatizing for words used in fuzzy hashes that allows to improve match
quality by using of the first forms of all words
* Rework language detection
* Fix several critical bugs, memory leaks and deadlocks:
- memory leak in HTML nodes parsing
- deadlock in logger code
- deadlock in signals processing
- crashes in fuzzy_storage
- crashes in tokenizers if the input was empty
* Import new libucl with several bugfixes and improvements
* Support listening on ipv6 addresses only
* Fix macro expansion in SPF module
* Several bugfixes in DKIM module
* Add load headers support for mime parts to the lua API
* Add documentation for:
- workers in general
- fuzzy_storage worker
- fuzzy_check plugin
- mimepart and textpart lua API modules
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Apply boundary fix for dkim simple canonization
* Fix ping command
* Return nil if header was not found in lua_task
* Fix hang in upstreams revive logic
* Decode entitles when normalizing HTML parts
* Fix logic of finding URLs in HTML parts
* Do not include \0 into length of text when performing conversion to utf8
* Fix raw vs parsed reperesentations
Raw parts are now:
- decoded b64/qp, but *NOT* converted to utf-8
Processed parts are now:
- converted to UTF-8
- normalized if needed (e.g. HTML tags are stripped)
* Rework DKIM canonization to line based
* Fix fuzzy hashes addding
* Use more specific hash function for fuzzy
* Fix leaking of iconv descriptors
* Fix PTR resolving in lua resolver
* Rework spf module.
- Copy data to memory pool as cached record might be destroyed causing
freed memory being passed to the protocol output (use after free)
- Allow SPF_NEUTRAL policy to be handled separately
- Add R_SPF_NEUTRAL to the default config
* Rework `register_symbols` function
* Allow to disable components of hfilter
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix owner when creating folder /run/rspamd (by @sfirmery)
* Fix IP validity checks
* Decode URLs obtained from HTML tags
* Fix crash with unweighted upstreams
* Stop processing headers in parts
* Set sockaddr.sa_family properly when connectig to upstreams
* Fix reload issues in surbl and fuzzy_check (reported by @citrin)
* Fix timeouts in redirector
* Improve lua errors reporting
* Fix lua closures processing in libucl
* Rework calling of lua functions from regexp module
* Choose raw regexp for raw headers
* Rework conversion to utf since glib one is broken
* Ignore SGML style tags in html
* Fix old bug with non-capturing https urls
* Fix memory corruption on fuzzy reload (reported by @citrin)
* Fix percents display in rspamc
* Fix buffer update for DKIM
* Do not validate utf for raw headers
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix build under *BSD
* Detect HAN unicode script
* Implement language detection heuristic for text parts
* Fix time output in history
* Improve piechart coloring
* Fix \r\n conversion in DKIM module (reported by @citrin)
* Try to detect systems with no IPv6 support
* Fix multiple/single values in use settings (reported by @citrin)
* Rework IP addresses in upstreams:
- Select ipv4/unix addresses if they exist and use ipv6 for ipv6 only
upstreams (since the support of ipv6 is poor in many OSes and
environments)
- Free IP list on upstream destruction
- Add test cases for addresses selection
- Allow adding of free form IP addresses to upstreams
* Fix endiannes in lua_radix search (reported by @citrin)
* Soft shutdown should also set wanna_die flag (reported by @citrin)
* Stop use-after-free in event loop termination
* Fix processing of very short messages in DKIM (reported by @citrin)
* Detect systems without shared mutexes
* Fix issues with PTR and MX elements in SPF parser (reported by @citrin)
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* New upstreams code:
- simplify upstreams API;
- unify strings parsing in upstreams definition;
- add configuration options for the upstreams;
- for failed upstreams re-resolve their addresses;
- use all resolved addresses for an upstream (round-robin);
- implement stable hashing and use it by default for upstreams;
- add unit test for upstreams module.
* Rework signals processing in all rspamd workers:
- signals are now processed in the event loop;
- implement the most common signal handlers for all workers;
- add callbacks for workers specific signal handlers
* Fix critical issue with fuzzy storage:
Fuzzy stroage could not save any hashes on termination due to bugged
signals handling
* Fix roll history IP storage
* Rework ipv4/ipv6 handling in parsing addresses:
- turn off support of IPV6_V6ONLY socket option;
- create ipv6 socket prior to ipv4 one to handle systems with v6/v4
sockets enabled (Linux)
* Remove CBL as it's wholly included in Spamhaus XBL (by @fatalbanana)
* Remove nszones.com fake RBL (by @citrin)
* Fix upstreams interaction for fuzzy_check
* Verify spf PTR records (reported by @citrin)
* Fix spf MX records parsing
* Add compatibility for old libevent (by @yellowman)
* Sync bugfixes from libucl
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Convert all maps to the compressed radix trie
* Allow IPv6 addresses in IP maps
* Remove dynamic items support from symbols cache
* Allow hex encoded output of strings
* Fix bug with control connections count
* Process fuzzy weight correctly (reported by @fatalbanana)
* Remove extra reference retain of http connection on error
* Remove deprecated options from the default config
* Add `one_shot` attr to metric's symbols
* Doc: add documentation for metrics
* Add Upstart job to debian packaging (by @CameronNemo)
* Config: improve SURBL symbols descriptions (by @citrin)
* Config: reflect SURBL changes (by @citrin):
- Outblaze removed, malware moved to separate list:
http://www.surbl.org/news/internal/MW-malware-sublist-added-to-multi
* Fix C modules initialization on restart
* Treat single IP as a single IP in radix lists (reported by @citrin)
* Do not touch file and core limits if not asked explicitly (reported by @citrin)
* Improve logging for fuzzy errors
* Block SIGPIPE for HTTP writing
* Doc: update manual pages
* Fix HTTP connection termination
* Reduce default number of parallel requests to 8
* Sync with libucl include features
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Fix typo in stat output.
* Fix issues with includes crossing with the system includes
* Restore testing framework
* Add radix trie test suite
* Implement new path-compressed radix trie.
- The performance benefit over the old algorithm is about 1.5 times.
- Memory usage is significantly lower as well.
- Now radix trie can accept any IPv4/IPv6 values
* Various improvements to the memory pools code
* Fix writing reply to a client when no filters are defined
* Write base32 encoded fuzzy
* Fix 'soft reject' action
* Fix rspamd reload and modules reconfiguration
* Fix subject rewriting for the default subject
* Fix states for processing task and pre-filters
* Fix issues with connection closing
* Fix crashes in rdns
* Fix ratelimit pre-filter
* Update exim patch.
- Update to the recent exim version
- Strip extra leading src/ from the patch
- Remove sendfile since it was broken
- Fix rspamd spam report for exim
* Improve documentation
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|
|
|
|
| |
Signed-off-by: Vsevolod Stakhov <vsevolod@highsecure.ru>
|