You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

ChangeLog 70KB

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670
  1. 1.2.7:
  2. * Slightly reduce weights of rules with high FP rate
  3. * Add workround for rspamd-1.3
  4. * Fix possible FP in TRACKER_ID rule
  5. * Simplify MISSING_MIMEOLE rule
  6. * Add workaround for gmime CTE stupidity
  7. * Fix mime headers processing
  8. * Fix false positive URL detections in text parts
  9. * Fix Exim shutdown patch
  10. * Enable workaround for exim mailbox format
  11. * Backport shingles static test
  12. * Fix levenshtein distance calculations
  13. * Fix max_train setup in ANN module
  14. * Fix redis structure by adding {NULL, NULL} member
  15. * Fix build with unmodified LibreSSL opensslv.h
  16. * Repair optional dependencies
  17. * Really skip filters in case of pre-result set
  18. * Restore the intended pre-filters behaviour
  19. * Fix ipv6 mask application
  20. 1.2.6:
  21. * Fix parsing of URLs in texts
  22. * Fix creating of URLs from LUA
  23. * Fix some more URL detector issues
  24. * Fix unit tests
  25. * Fix JIT compilation for PCRE2 expressions
  26. * Fix JIT usage for PCRE2
  27. * Fix UTF8 mode in PCRE2
  28. * Add workaround for pre-historic compilers (#605)
  29. * Fix and rescore R_PARTS_DIFFER logic
  30. * Properly set lua paths for tests
  31. * Fix SA rawbody processing - exclude top part
  32. * Store text parts content with newlines stripped
  33. * Properly support SA body regexps
  34. * Fix body rules in SA plugin
  35. * Fix setting of score for parts differ
  36. * More fixes to parts distance calculations
  37. - Use hashed words instead of full words for speed
  38. - Improve levenstein distance calculations and penalise replaces
  39. - Always return number from 0 to 1
  40. - Use g_malloc instead of alloca
  41. * Fix percents output in R_PARTS_DIFFER
  42. * Plug memory leak in dkim module
  43. * Plug minor memory leak in regexps creation
  44. 1.2.5:
  45. * Plug an important memory leak in headers getting code
  46. * Remove some bad domains from whitelists
  47. 1.2.4:
  48. * Implement new multipattern matcher that uses hyperscan if possible
  49. * Use mutlipattern for lua_trie code
  50. * Add utility methods for multipattern
  51. * Use multipattern in url matcher
  52. * Add escape functions for hyperscan
  53. * Allow to optimize lua -> C transition by flattening table args
  54. * Optimize hot paths in SA plugin
  55. * Optimize rspamd_re_cache_type_from_string
  56. * Allow empty tries
  57. * Fix extraction of URLs from Subject
  58. * Allow to have different flags for different patterns in multipattern
  59. * Add common directory for hyperscan cache to config
  60. * Implement caching for hyperscan multipattern
  61. * Attach domain part to `R_SUSPICIOUS_URL` (by @moisseev)
  62. * Allow multipattern scans to be nested for the case of hyperscan
  63. * Simplify SURBL redirector search code and avoid ac_trie
  64. * Add two way substring search algorithm
  65. * Avoid acism usage to find gtube pattern
  66. * Fix processing of empty headers
  67. * Allow to disable pthread mutexes on broken platforms
  68. * Make web interface not send password in query strings (#585) by @fatalbanana
  69. * Add maximum delay to ratelimit module
  70. * Backport fix for empty files inclusion from libucl
  71. * Fix settings id setup
  72. * Add min_learns option to classifiers
  73. * Use more clever to utf8 conversion strategy
  74. * Fix disabling of virtual symbols in the settings
  75. * Rework settings to work properly in metric-less configuration
  76. * Set the default limit for classifier
  77. * Fix ttl based expiration from LRU cache
  78. * Rework DKIM module to use OpenSSL for digests
  79. * Fix mailto urls parsing with hyperscan
  80. * Do not set obscured flag for urls starting with spaces
  81. * Fix crash on redis learn
  82. * Fix ratelimit ctime setting
  83. 1.2.3:
  84. * New DCC module (by @smfreegard)
  85. * Rework whitelist module:
  86. - Now we check different elements for different checks
  87. - MIME from for DMARC
  88. - DKIM signature domain for DKIM
  89. - SMTP from or HELO for SPF
  90. * Fix regexps results combination (*critical*)
  91. * Fix issue with expressions processing (*critical*)
  92. * Optimize strlcpy for aligned input
  93. * Add support of half-closed connection in lua_tcp
  94. * Allow to print compact json in client
  95. * Save required score in history (#581)
  96. * Allow to attach file descriptors to control commands
  97. * Allow to send descriptors from workers to main
  98. * Allow to attach fd when broadcasting to workers
  99. * Implement log pipe feature for rspamd logs analysis
  100. * Add `log_helper` worker
  101. * Add `URIBL_SBL_CSS` (by @smfreegard)
  102. * Add worker scripts functionality
  103. * Add on load hooks for rspamd_config
  104. * Add lua scripts for log_helper worker
  105. * Add generic maillist detector (#584)
  106. * Implement FANN autolearn using log_helper worker
  107. * Rework metrics configuration to allow includes
  108. * Change default value of forced removal in composite rules
  109. * Allow to use assembly version of blake2b on x86 cpu
  110. * Use less precise (but faster) clock if possible
  111. * Insert redirected URL to the urls list
  112. * Allow to get and set callback data for rspamd symbols
  113. * Add binary heap implementation
  114. * Use binary heap for expire algorithms in the hash
  115. * Use `least frequent used` expiration strategy
  116. * Allow to get mime headers from a task
  117. * Add support for mime headers in `regexp` module
  118. * Update Exim patches (by @fatalbanana)
  119. * Allow building rspamd with jemalloc
  120. * Save multipart boundaries
  121. * SA plugin changes:
  122. - Properly handle MIME headers
  123. - Fix eval:check_for_missing_to_header rule
  124. - Implement SA compatible body regexps
  125. - Use sabody rules in SA plugin
  126. * LUA API changes:
  127. - Add util.get_ticks function
  128. - Add util.stat function
  129. - Add task:get_symbols_numeric method
  130. - Add method to get number of symbols in the cache
  131. - Add lua methods to get redirected urls
  132. - Allow to get callbacks for lua symbols
  133. - Add config:set_symbol_callback function
  134. 1.2.2:
  135. * Use HTTP Content-Type on non mime input if possible
  136. * Save log level when compressing log messages
  137. * Further rework of composite rules (add '^' prefix)
  138. * Add tracking for rspamd expressions
  139. * Store actions limits in metric result
  140. * Fix parsing of include/redirect with many records in SPF
  141. * Add method to disable symbols execution in the cache
  142. * Allow to disable checks from settings
  143. * Allow to select settings by id in HTTP query
  144. * Find URLs with '\r' and '\n' inside href attribute
  145. * Implement vectored mode for hyperscan (experimental)
  146. * Improve client connection errors diagnostics
  147. * Allow to edit new files with signtool
  148. * Improve hashes performance on 32 bit platforms
  149. * Fix sorting of limits
  150. * Remove slow and unused rules `INVALID_EXIM_RECEIVED*`
  151. * Add expression:process_traced lua method
  152. * Allow tables in task:insert_result
  153. * Save trace for SA metas
  154. * Do not parse broken TLD parts in URLs
  155. * Investigate many border cases in URLs parser
  156. 1.2.1:
  157. * Add list support to `mime types` module configuration (by @moisseev)
  158. * Allow symbols params to be printed in logs
  159. * Fix `MIME_BAD_ATTACHMENT` false positives for MDN/DSN
  160. * Fix crashes on arm32
  161. * Do not classify message if some class is missing
  162. * Fix cryptobox cleanup
  163. * Remove multipart/report from bad mime types (#569)
  164. * Improve logging for fuzzy hashes
  165. * Show map URLs in webui
  166. * Sort symbols in webui
  167. 1.2.0:
  168. * New dynamic updates plugin
  169. * Regular expressions map support
  170. * Faster radix trie algorithm
  171. * Faster siphash for AVX2 supporing CPUs (used in fuzzy hashes)
  172. * PCRE2 support
  173. * Allow quoted and slashed keys in map
  174. * Add proper support of DNS resolvers balancing (#552)
  175. * Rework includes and configuration system for better local changes support
  176. * New keypairs framework for signing and encryption
  177. * Added support for dynamic modules and workers
  178. * Allow to dump configuration with help comments
  179. * Rework once_received module
  180. - Fix priority for `good_hosts`
  181. - If a good host has been found do not add once_received symbols
  182. - Fix priorities for strict once_received
  183. - Add ability to whitelist IP addresses
  184. * Implement support of signed maps for HTTP and file maps
  185. * Add command to sync fuzzy storage (#533)
  186. * Rework system of symbols and actions registration
  187. It is possible now to use priorities when adding symbols to metrics and
  188. override scores for symbols with lower priority with the scores with
  189. high priority.
  190. * Add auth support and db selection for redis stats
  191. * Improve composite rules application
  192. * Add ignore_received option
  193. * Fix critical issue with inconsistent resorting
  194. * Fix `all` in spf redirects
  195. * Add punycoded versions for IDN domains (#554)
  196. * Improve sorting order for symbols cache
  197. * Add lockless logging for processes management
  198. * Allow to specify flags for metric symbols
  199. * Load images height and width from style attribute (#538)
  200. * Override DNS requests limits for SPF and DKIM
  201. * Fix resetting symbols to their default values in WebUI
  202. * Improve configuration agility for redis stats
  203. * Allow to set db and password for redis in stat_convert
  204. * Import the latest libucl
  205. * LUA API changes:
  206. - Add rspamd_version function to LUA API
  207. - Add lua_cryptobox module
  208. - Add lua_map module
  209. - Add task:set_metric_action lua API method
  210. - Fix race condition in lua_tcp module
  211. - Fix a lot of issues in lua_redis module
  212. - Rework and abstract lua maps API
  213. - Add util.strlen_utf8 lua function
  214. - Add lua functions for caseless comparison
  215. - Allow optional symbols registration
  216. - Add config:add_map table form method, add regexp maps
  217. - Add task:has_urls method
  218. - Add task:has_flag method
  219. - Add html tags methods to lua_html
  220. - Add task:get_dns_req
  221. * Plugins changes:
  222. - Add support for WLBLEval SA plugin
  223. - Use caseless comparison in SA and DMARC plugins
  224. - Allow SA plugin to set scores for rspamd symbols
  225. - Add regexp maps support to multimap
  226. - Allow filenames match in multimap
  227. - Add more filters for the existing map types
  228. - Fix html images rules to reduce FP rates
  229. * New rules:
  230. - LONG_SUBJ - too long subject
  231. - MIME_BAD_ATTACHMENT - bad attachement type
  232. - RDNS_NONE - no reverse DNS record for sender's IP
  233. - Fix MISSING_MIMEOLE rule for modern OE
  234. * Many other bugfixes, memory leaks plugs thanks to:
  235. - Coverity scan
  236. - New gcc-6 warnings
  237. - valgrind manual iterations
  238. * Documentation improvements:
  239. - FAQ list: https://rspamd.com/doc/faq.html
  240. - Reworked quick start guide
  241. - Added documentation for all active modules
  242. * Other changes:
  243. - Dropped Ubuntu Vivid support
  244. - Added Ubuntu Xenial support
  245. - Rework build system for rspamd and rmilter
  246. 1.1.4:
  247. * Print traceback on lua errors in lua config
  248. * Fix leaks in lua error paths
  249. * Improve 'R_EMPTY_IMAGE' rule
  250. * Fix metas memoization in SA plugin
  251. * Properly set `flag` in fuzzy replies
  252. * Fix arguments order
  253. * Fix issue with out-of-boundary reading
  254. * Fix issues found by coverity
  255. * Same result checking error found by coverity
  256. * Fix varargs processing (found by coverity)
  257. * Fix error in printing hex
  258. * Reduce weights for some hfilter patterns
  259. * Add aliases for task:get_from_ip:
  260. - task:get_addr
  261. - task:get_from_addr
  262. - task:get_ip
  263. * Rework once_received module
  264. - Fix priority for `good_hosts`
  265. - If a good host has been found do not add once_received symbols
  266. - Fix priorities for strict once_received
  267. - Add ability to whitelist IP addresses
  268. * Fix `MISSING_MIMEOLE` rule for modern OE
  269. * Treat meta tags as embedded tags (#501)
  270. 1.1.3:
  271. * Fix DSN rules when SMTP from is unavailable
  272. * Fix statconvert routine to avoid lua module usage
  273. * Set a sane quark for configtest to avoid NULL to be printed in logs
  274. * Support c11 if available
  275. * Fix parsing of ip:port strings
  276. * Add more diagnostic for lua subr errors
  277. * Fix task:set_from_ip lua method
  278. * Add basic routines for digital signatures
  279. * Add tool for digital signatures
  280. * Add plain open file API method for atomic open
  281. * Fix parsing nested braces inside logger vars
  282. * Pre filters now actually skip processing
  283. * Add pre-filter mode for multimap
  284. * Switch to apache 2 license
  285. 1.1.2:
  286. * Fix stat_cache closing
  287. * Add checkpoints to sqlite3 learn cache
  288. * Do not recompile lua generated headers all the time
  289. * Increase number of messages learned
  290. * Fix issues with dual stack and hfilter
  291. * Disable MID checks for hfilter by default
  292. * Fix cache definitions in multiple classifier and no type
  293. * Don't crash if learn cache failed to initialize
  294. * Fix googlegroups support in maillist plugin
  295. * Rework flags LUA API:
  296. - Allow to check for a specific flag
  297. - Add `learn_spam`, `learn_ham` and `broken_headers` flags
  298. - Unify internal functions
  299. * Add `BROKEN_HEADERS` rule
  300. * Add support for forged confirmation headers (by @AdUser)
  301. * Allow `any`, `mime` and `smtp` for get_from/get_recipients
  302. * Add mime types checking plugin
  303. * Add rule to detect spammers attempts to cheat mime parsing
  304. * Rework parsing of IP addresses in configuration (better IPv6 support)
  305. * Add `util.parse_mail_address` function to LUA API
  306. * Add lua sqlite3 module
  307. * Implement synchronous redis call
  308. * Ratelimit: avoid possible indexing of nil value (Fixes #498) (by @fatalbanana)
  309. * Add stat_convert command to convert stats tokens from sqlite3 to redis
  310. * Implement redis advanced lua api with pipelining
  311. * Fix memory leak on redis stat (#500)
  312. * Fix user/language learn count in sqlite statistics (#496) (by @fatalbanana)
  313. * Fix build with custom pcre
  314. * Fix fuzzy relearning (#498)
  315. * Improve planning of asynchronous tasks
  316. * Show slow rules in log
  317. * Add warning for slow regexps
  318. * Add base32 decode/encode routines to lua util
  319. * Allow converting of learn cache from sqlite to redis
  320. * Add methods to check if a messages has from/rcpts
  321. * Improve and fix multimap plugin:
  322. - Restore 'header' maps
  323. - Add filters for headers
  324. - Add 'email:addr', 'email:user', 'email:domain' and 'email:name' filters
  325. - Add generic regexp filters
  326. * Disable reload command in rc scripts
  327. * Improve runtime CPU dispatcher for libcryptobox
  328. * Add preliminary support of digital signatures via ed25519
  329. * Add detection for RDRAND support
  330. * Print configuration of crypto on start
  331. * A in SPF presumes AAAA lookup as well
  332. 1.1.1:
  333. * Fix duplicated XBL symbol
  334. * Reduce log severity for ratelimit missing servers
  335. * Fix XBL composite to avoid duplicate symbols
  336. * Reduce weight of URL_ONLY rule due to FP rate
  337. * Disable fuzzy hashes from the metadata for now
  338. * Fix processing of empty messages (#486)
  339. * Always treat DNS timeouts as temporary fail for SPF
  340. * Fix issue with SPF double IP stack (#483)
  341. * Use X-Forwarded-For when checking secure_ip (#488)
  342. * Fix hash calculation for sqlite stats
  343. * Fix memory corruption on punycode
  344. * Fix strings allocation in punycode
  345. * Fix error message (#491)
  346. 1.1.0:
  347. * Incompatible change: sqlite3 and per_user behaviour:
  348. Now both redis and sqlite3 follows the common principles for per-user
  349. statistics:
  350. 1) If per-user statistics is enabled check per-user tokens ONLY
  351. 2) If per-user statistics is not enabled then check common tokens ONLY
  352. If you need old behaviour, then you'd need to use separate classifier
  353. for per-user statistics.
  354. * Implement redis statistics backend and cache
  355. * Implement autolearning for statistics
  356. * Reworked statistics architecture from scratch
  357. * Add hyperscan (https://github.com/01org/hyperscan) engine for regular
  358. expressions:
  359. - add lazy loader for hyperscan databases
  360. - rework regexp cache to have joint pcre/hyperscan scanning
  361. - implement hyperscan pre-filter support
  362. - add compilation guards for bad expressions
  363. - implement `rspamadm control recompile` command
  364. - implement hyperscan cache monitoring
  365. - slides: <https://highsecure.ru/rspamd-hyperscan.pdf>
  366. * Implement flexible task logging
  367. * Rework fuzzy worker:
  368. - it is now possible to run multiple fuzzy workers;
  369. - implement lazy writing as sqlite3 is bad at concurrent writing;
  370. - add retries for simple sql commands in fuzzy backend;
  371. - use fine-grained transactions for fuzzy;
  372. - implement new multi-pubkeys mode;
  373. - allow encrypted only storages;
  374. - rework statistics for fuzzy;
  375. - add `rspamadm control fuzzystat` command for extended statistics;
  376. - implement human readable output for the previous command;
  377. - add condition script for learning fuzzy storage;
  378. * Various fixes to SPF:
  379. - fix `redirect` records;
  380. - fix domains when parsing mx/ptr/a records in includes/redirects;
  381. - fix issues with multiple addresses in SPF records;
  382. - ignore SPF results in case of DNS failure;
  383. - adjust TTL of records when resolving subelements of SPF records;
  384. - always select `v=spf1` line if it is available
  385. - do not cache records with DNS failure in subrequests;
  386. - ignore records with temporary fails during subrequests resolving;
  387. - fix `RDNS_RC_NOREC` support;
  388. * Add clang plugin for static analysis:
  389. - implement static checks for `rspamd_printf` format strings;
  390. * Add 'allow_raw_input' option for non-mime messages
  391. * Recognize types using libmagic
  392. * Fix parsing of IPv6 received headers
  393. * Add new interface of communication between workers in rspamd
  394. * Add support for named socketpairs
  395. * Don't write URLs by default as it is too verbose
  396. * Set status for HTTP replies
  397. * Try load `rspamd.conf.override`
  398. * Implement words decaying for text parts to limit many checks
  399. * Improve support of SA rules and plugins:
  400. - add check_for_shifted_date and check_for_missing_to_header eval rules;
  401. - add 'check_relays_unparseable' support;
  402. - add `check_for_mime('mime_attachement')` function;
  403. - use new re_cache interface for all SA rules;
  404. - add support for `Mail::SpamAssassin::Plugin::MIMEHeader`;
  405. - add support of 'special' SA headers to `exists` function;
  406. - fix issue when SA metas contain other metas;
  407. - fix freemail rules;
  408. * Many fixes to the URL parser
  409. * Match any newline character in regexps
  410. * Fix resolving of upstreams and detection of poor IPv6 configurations
  411. * Parse upstreams selection algorithm from the configuration line
  412. * Add `reresolve` command to the control interface
  413. * Generate fuzzy hashes from task metadata (URLs and headers)
  414. * Add method to check if IP is local and `local_addrs` option
  415. * Implement forced timeout for delayed filters
  416. * Disable fast path of pcre-jit as it seems to be broken
  417. * Bayes fixes:
  418. - new normalizer function;
  419. - really use weights of tokens from the OSB algorithm;
  420. - restore multiple classifiers support;
  421. * Rules changes:
  422. - add `R_SUSPICIOUS_URL` rule that detects obfuscated URL's;
  423. - improve empty image rule;
  424. - rework `FORGED_RECIPIENTS` rule;
  425. - reduce weight of `SUSPICIOUS_RECIPS`;
  426. - fix `*_NORESOLVE_MX` symbols in hfilter;
  427. - add `SUBJ_ALL_CAPS` rule with support of UTF8
  428. - add spamhaus SBL to uribl
  429. - fix `SUSPICIOUS_RECIPS` and `SORTED_RECIPS` rules
  430. - remove `R_TO_SEEMS_AUTO` as it generates a lot of FP;
  431. - add new Message-ID regexp for Thunderbird (by @moisseev);
  432. * Plugins changes:
  433. - allow ratelimit plugin to set symbol instead of pre-result
  434. - support IP DNS black lists for URIBL (e.g spamhaus SBL);
  435. - drop deprecated SURBL bits (by @fatalbanana)
  436. - rename `JP_SURBL_MULTI` to `ABUSE_SURBL` (by @fatalbanana)
  437. - add `SURBL_BLOCKED` (by @fatalbanana)
  438. - add `CR_SURBL`
  439. - SURBL: allow fallthrough to default symbol (by @fatalbanana)
  440. - Settings: fix IP match (by @fatalbanana)
  441. - SURBL: add missing symbols to metric (by @fatalbanana)
  442. - allow processing images urls for SURBL
  443. - unconditionally disable SPF for authenticated users and local networks
  444. * Rework ratelimit plugin
  445. - switch to `rates` instead of old and stupid strings to setup;
  446. - check if a bucket is zero and disable the corresponding limits'
  447. - turn off all buckets by default;
  448. - check either `rcpt` or `user` buckets, not all together'
  449. - document new `rates` and `symbol` options;
  450. - inform user about what buckets are used in the configuration;
  451. * Add neural network **experimental** plugin
  452. * Add a sample script to learn neural network from rspamd logs
  453. * Add documentation strings support to rspamd:
  454. - add strings for the main configuration options;
  455. - document workers options;
  456. - add internal plugin options;
  457. - create `rspamadm confighelp` routine;
  458. - implement human readable output for the previous command;
  459. - add subtree search support;
  460. - add keyword search support;
  461. * Documentation improvements, tutorials section, statistics description
  462. * Many other minor and major bugfixes not noted here
  463. 1.0.11:
  464. * Fix spf redirects
  465. * Fix domains when parsing mx/ptr/a records in includes/redirects
  466. * Fix unfolded base64 encoding
  467. * Fix GError use-after-free
  468. * Do not rewrite the original url when using redirector
  469. * Fix parsing of fragment in urls
  470. * Fix processing of HTML tags
  471. * Improve empty image rule
  472. * Avoid long double type
  473. * Fix tokens weights in OSB algorithm
  474. * Improve debugging for bayes
  475. 1.0.10:
  476. * Fix settings application (#416)
  477. * Fix another issue with fixed strings
  478. * Fix hash function invocation
  479. * Use the proper string for make_dns_request in lua_http
  480. * Fix scan time output
  481. * Update webui:
  482. - fix labels for greylisting
  483. - fix dimension of scan time
  484. 1.0.9:
  485. * Emergency fix in keyed blake2 to fix fuzzy hashes and encrypted password
  486. * Support passwords longer than 64 symbols
  487. 1.0.8:
  488. * Add function to traverse AST atoms
  489. * Allow dependencies on rspamd symbols for SA metas
  490. * Fix memory corruption when timeout is removed in fuzzy check
  491. * Fix encrypted fuzzy add processing
  492. * Avoid use-after-free in controller session destructor
  493. * Use session pool instead of task pool in fuzzy check
  494. * Fix assembly in i386 mode (#413, #412)
  495. 1.0.7:
  496. * Plugged memory leaks in internet address object & html parser
  497. * Fixed static build
  498. * Fixed multiple sigchld processing
  499. * Fixed deletion of signal events after event processing loop
  500. * Fixed build on ARM (#404 - reported by @Gottox)
  501. * Fixed setting the default mask for SPF.
  502. * Fixed sanitisation of HTTP query values
  503. * Fixed parsing of the last header in encrypted HTTP messages
  504. * Additions and fixes for test suite & benchmarks
  505. * Added openssl aes-256-gcm support to libcryptobox & HTTP server
  506. * Implemented support for starting multiple HTTP servers
  507. * Implemented batch accept in HTTP server
  508. * Added module to get data from HTTP headers (#285 - reported by @msimerson)
  509. * Added `rspamadm control` command
  510. * Added ability to sort counters output.
  511. * Added ability to specify custom headers for rspamc client
  512. * Fix architecture detection
  513. * Converted history storage to the UCL format
  514. * Allow flexible number of rows in history
  515. * Fix action badges in WebUI
  516. * Add universal cryptobox hash API
  517. * Migrated to the optimized blake2b implementation adopted from Andrew Moon
  518. * Allow explicit loading of specific modules
  519. * Always load settings module
  520. * Allow to add symbols from settings
  521. * Fix double free in the controller fuzzy learn command
  522. * Avoid endless loop when cannot open sqlite db
  523. * Updated libucl
  524. 1.0.6:
  525. * Fix build on i386
  526. * Update CentOS7 service file patch (by @fatalbanana)
  527. * Fix path to rspamadm in Debian init script (by @fatalbanana)
  528. * Fix broken '_SC_GETPW_R_SIZE_MAX' on FreeBSD
  529. * Fix portability issues
  530. * Use cryptobox chacha for libottery
  531. * Better support of 32 bit builds
  532. * Fix header name tokens setup
  533. * Fix levenstein distance method for words
  534. * Add workaround for old libevent (#400)
  535. * Fix microseconds in termination timer
  536. * Fix some more issues with fixed strings
  537. * Explicitly test CPU instructions even after CPUID call
  538. * Do not check out of boundary memory
  539. * Do not output broken emails
  540. * Fix unknown symbols registration
  541. * Handle SIGILL using longjmp
  542. * Block signals when exiting event loop
  543. * Fix incorrect allocation size
  544. * Slightly optimize alignment
  545. * Restore rspamd -t for compatibility
  546. * Add more sanity checks for emails
  547. 1.0.5:
  548. * Add rspamd control interface:
  549. - support `stat` command to get runtime stats of rspamd workers
  550. - support `reload` command to reload runtime elements (e.g. sqlite3 databases)
  551. * Rework curve25519 library for modular design:
  552. - add Sandy2x implementation by Tung Chou
  553. - fix CPU detection for variables loading assembly
  554. - add testing for curve25519 ECDH
  555. * New fixed strings library
  556. * Add `R_SUSPICIOUS_IMAGES` rule
  557. * Enable mmap in sqlite3
  558. * Use new strings in the HTTP code
  559. * Improve google perftools invocation
  560. * Improve performance profiling in http test
  561. * Reorganize includes to reduce namespace pollution
  562. * Allow specific sections printing in configdump command
  563. * Rework workers signals handlers to be chained if needed
  564. * Update socketpair utility function
  565. * Add control_path option for rspamd control protocol
  566. * Fix ownership when listening on UNIX sockets
  567. * Rework signals processing in main
  568. * Remove extra tools from rspamd (they live in rspamadm now)
  569. * Remove global rspamd_main
  570. * Add global timeout for the overall task processing (8 seconds by default)
  571. * Sanitize NULL values for fuzzy backend
  572. * Store NM between encrypt/decrypt
  573. * Add textpart:get_words_count method
  574. * Fix generic DNS request in lua
  575. * Tune hfilter weights
  576. * Add support of IPv6 in hfilter
  577. * Fix parsing of HTTP headers with IP addresses
  578. * Sync with the recent libucl
  579. * Various minor bugfixes
  580. 1.0.4:
  581. * Add configdump routine to rspamadm
  582. * Implement retransmits for fuzzy_check plugin
  583. * Fix events processing for learning anf checking fuzzy hashes
  584. * Avoid dependency on unneeded and uncompatible glib include
  585. * Add `historyreset` command to the controller
  586. * Fix loading of tokenizer config from dump (#389)
  587. * Add sorting hints for the history
  588. * Allow custom lua scripts for users/languages extraction (#388)
  589. * Do not add FORGED_RECIPIENTS when 'To' is missing (#387)
  590. * Do not add R_UNDISC_RCPT when 'To' is missing (#387)
  591. * Add encryption to fuzzy check plugin
  592. * Add encryption for fuzzy storage
  593. * Add new epoch for encrypted fuzzy request
  594. * Add encryption for `rspamd.com` storage
  595. * Remove gmime processing for LDA mode as it is deadly broken
  596. * Add routine to find end of headers position in mime messages
  597. * Fix LDA headers folding
  598. * Init libraries in rspamc client as well to avoid locale issues
  599. * Avoid collision with locally installed includes
  600. * Allocate and free memory with the same allocator in rspamadm (#385)
  601. * Preserve expired fuzzy hashes counter
  602. * Improvements in webui:
  603. - Add favicon.ico
  604. - Rework history table
  605. - Fix sorting for the history
  606. - Migrate to bootstrap 3 and jquery 2
  607. - Fix css bugs
  608. - Add glyphicons
  609. - Add reset history
  610. - Improve history buttons
  611. - Redraw graph to avoid display issues
  612. - Webui is now MIT licensed to match licensing policy of rspamd
  613. 1.0.3:
  614. * Fix piechart clean slice (#380)
  615. * Fix controller crashes when GString is reallocated (#381)
  616. * Correctly set locale before start
  617. * Set C locale for numeric values
  618. * Add rspamadm routine:
  619. - add `pw` command to manage passwords
  620. - add `help` command for displaying help
  621. - add `configtest` command to check configuration files
  622. - add `keypair` command for generating encryption keys
  623. - add `fuzzy_merge` routine to merge fuzzy sqlite databases
  624. - add a simple manual page for rspamadm
  625. * Allow metric registration for composite expressions
  626. * Add strict mode for configtest
  627. * Add logger counters
  628. * Save and show learned messages count (#383)
  629. * Add `no_stat` flag
  630. * Add `task:set_flag` and `task:get_flags` (#382)
  631. * Enable foreign keys in sqlite3
  632. * Remove orphaned shingles from fuzzy storage
  633. * Optimize synchronization steps for fuzzy storage
  634. * Allow delayed conditions registration
  635. * Add lua API for conditions registering
  636. 1.0.2:
  637. * Fix critical bug in webui that prevents password from being sent
  638. * Rework webui view:
  639. - Switch to d3.js for graphs
  640. - Improve piechart look
  641. - Rework colors for piechart
  642. - Fix layout for symbols
  643. - Fix refresh button
  644. * Add descriptions for whitelist maps
  645. * Fix build on arm (#379)
  646. * Fix issue with the last element in the radix trie
  647. * Add more tests for radix trie algorithm
  648. * Allow to extract URLs from query strings of other URLs (#361)
  649. * Initialize rrd fields before writing to file
  650. * Fix double free if no password has been specified
  651. 1.0.1:
  652. * Add writing to rrd from the controller
  653. * Fixed lots of bugs in rrd code
  654. * Adopt new DNS API in hfilter plugin (by @AlexeySa)
  655. * Allow only one controller process to manage rrd file
  656. * Set event base for fuzzy calls
  657. * Improve fuzzy IO errors logging
  658. * Add rra extraction function to rrd library
  659. * Add graph handler to the controller
  660. * Cache correct passwords to avoid too high CPU usage when working with webui
  661. * Controller sockets are owned by router do not export them to task
  662. * Optimize logging by skipping hash table search if it's empty
  663. * Fix loading issue whith broken statfiles
  664. * Print assertions from glib to rspamd logger
  665. * Load legacy `lua/rspamd.local.lua`
  666. * Update webui with some fixes to learning and scanning
  667. 1.0.0:
  668. * Rework symbols processing:
  669. - Improve sorting logic for symbols
  670. - Organize processing into multiple stages
  671. - Added asynchronous watchers for symbols
  672. - Added ability to organize dependencies between symbols
  673. * Fixed URL redirector:
  674. - Use optimized POE loop
  675. - Organize dependencies
  676. - Fix startup
  677. * New sqlite3 backend:
  678. - Allow to have per-languages and per-user statistics
  679. - Allow sqlite3 to be used as statistics backend
  680. * Store tokenizer configuration within statfiles
  681. * Improve bayes statistics:
  682. - Use headers and images metainformation in bayes
  683. - Suggest using of pre-processed tokens for statistics
  684. - Fix tokens normalization for OSB algorithm
  685. * Rewrite url parsing:
  686. - Fix numerous issues with url extraction and normalization
  687. - Fix mailto urls
  688. * Fix settings plugin to allow custom actions scores
  689. * Improve rbl plugin
  690. * Allow capturing patterns in rspamd lua regexp library
  691. * Add GTUBE support
  692. * Fix spamc legacy support
  693. * Add DKIM support to RBL module
  694. * Fix issues with multiple DKIM signatures
  695. * Fix issue if rspamd cannot create statfiles (#331)
  696. * Rework parts and task structure:
  697. - Now text_parts, parts and received are arrays
  698. - Pre-allocate arrays with some reasonable defaults
  699. - Use arrays instead of lists in plugins and checks
  700. - Remove unused fields from task structure
  701. - Rework mime_foreach callback function
  702. - Remove deprecated scan_milliseconds field
  703. * Add ip_score plugin support (not enabled by default):
  704. - Can check for asn/country and network using DNS lookups
  705. - Can store and load reputation from redis server
  706. * Improve PARTS_DIFFER rule to count merely different words
  707. * New HTML parser:
  708. - Parses HTML parts using a set of state machines
  709. - Extracts useful data and exports it to lua functions:
  710. + Styles
  711. + Images
  712. + URLs
  713. + Colors
  714. + Structure elements
  715. - Added HTML rules for some checks
  716. * New version of LUA DNS API
  717. * Table versions of many functions in LUA API
  718. * Improve rspamc client:
  719. - Print execution time
  720. - Allow executing of external commands and passing output to them
  721. - Allow mime output mode when rspamc alters message according to rspamd
  722. checks and send it to an external command or stdout
  723. * Allow scanning of local files using HTTP requests
  724. * Rework configuration system:
  725. - Rules are now moved from the $CONFDIR to $RULESDIR to avoid ambiguity
  726. - All modules configurations are now split in $CONFDIR/modules.d/* to
  727. simplify upgrades
  728. - Move hfilter to plugins
  729. - Allow plugins and rules to define default scores to simplify metrics
  730. setup
  731. - Include overrides for all modules to honor local/automatic parameters
  732. - Tune scores for many modules
  733. * Rework and enable DMARC plugin
  734. * Add whitelist plugin for SPF/DKIM/DMARC based whitelisting
  735. * Add some common domains to whitelists shipped with rspamd
  736. * Rework logging:
  737. - Now each log entry supports module name and a `tag`. Tag is used to
  738. identify unique objects (such as tasks) when checking log files
  739. - It is possible to turn on debugging for the specific modules
  740. - Systemd logging is fixed
  741. * Improve spamassassin plugin.
  742. - Now headers are matched more like SA
  743. - Improve support of Message-ID
  744. - Add support of ToCc header type
  745. - Fix :addr and :name in headers regexps
  746. * Resurrect rrd support code
  747. * Save controller stats between restarts
  748. * Fixed tonns of bugs
  749. * Added tonns of minor improvements and features
  750. * Added more unit tests
  751. * Create functional tests framework
  752. * Added documentation for missing modules
  753. * Added rpm/deb repositories and scripts
  754. * Updated WebUI and libucl externals
  755. 0.9.10:
  756. * Do not dereference null pointer on learning.
  757. * Fix some extreme cases in BAYES.
  758. * Add a workaround to avoid bad HTML messages breaking.
  759. * Build with -O2 flags by default.
  760. * Add constraints to limit DNS requests count per task.
  761. * Add workaround for SURBL DNS flood.
  762. * Set error if rspamd cannot learn anything.
  763. 0.9.9:
  764. * Don't use RWL_SPAMHAUS_WL (unknown result) for whitelisting (by @fatalbanana)
  765. * Import updated public suffix list (by @fatalbanana)
  766. * Remove debug message
  767. * Fix settings (by @fatalbanana)
  768. * Remove duplicated symbol registration
  769. * Use WAL for fuzzy storage
  770. * RBL fixes (by @fatalbanana):
  771. - silence errors;
  772. - yield unknown results from RBLs;
  773. - fix scoring for DNSWL;
  774. - fix use of RBL name as symbol;
  775. - ignore RBL names that would not be yielded;
  776. * Support captures in regular expressions
  777. * Add captures support to lua_regexp
  778. * Support dist on FreeBSD and Darwin
  779. * Add RCVD_IN_DNSWL_NONE as whitelisting exclusion (by @fatalbanana)
  780. * Multiple fixes to URL detection:
  781. - support port definition;
  782. - fix query and path recognition;
  783. - fix parsing of multiple slashes in URL;
  784. - fix parsing query just after port;
  785. - fix path field in `url:to_table` method;
  786. - improve support of IP based URLs.
  787. * Set ignore_whitelists = true for RECEIVED_SPAMHAUS_XBL (by @fatalbanana)
  788. * Add GTUBE support
  789. * Ignore User header in SA mode
  790. 0.9.8:
  791. * Fix critical bug in bayes classifier (#305)
  792. * Fix critical bug in RBL module (by @fatalbanana)
  793. * Fix and rework settings plugin.
  794. * Fix get_all_opts for a case of non-iterable options.
  795. * Use tld for redirector's matching.
  796. 0.9.7:
  797. * Add whitelist_exception setting to RBL module (by @fatalbanana)
  798. * Don't use RWL_MAILSPIKE_POSSIBLE or DNSWL_BLOCKED for whitelisting (by
  799. @fatalbanana)
  800. * Fix extreme cases in bayes classifier.
  801. * Fix parsing of urls with '?' at the end of hostname.
  802. * Update interface.
  803. * Fix number of issues with webui interaction.
  804. * Fix saving maps.
  805. * Allow user@ and @domain matches in multimap.
  806. * Fix issues with bounces From processing.
  807. * Fix abs/fabs misuse.
  808. * Fix builds on suse and arch linux distributions.
  809. 0.9.6:
  810. * Fix memory leak if mime cannot be parsed.
  811. * Fix dkim cache expiration.
  812. * Fix issues with redirector HTTP response.
  813. * Fix abnormal connection closing with certains messages with a high score
  814. (issue #296)
  815. * Fix redirector installation.
  816. * Use specific POE loop for some systems.
  817. * Fix number of issues in URL redirector.
  818. * Fix selecting URLs for sending to redirector.
  819. 0.9.5:
  820. * Avoid double free when extending HTTP message.
  821. * Fix double free if multiple classifiers are defined.
  822. * Fix misprint in spamassassin plugin.
  823. * Fix cpuid invocation on i386.
  824. * Fix ownership issues for zero-copy decode.
  825. * Allow __len metamethod on rspamd{text}.
  826. * Add base64 decoding lua utility.
  827. * Fix build on FreeBSD
  828. * Skip spaces at the beginning of mime messages.
  829. * DBL_ABUSE_REDIR should not have significant weight.
  830. * Allow to split by lua_regexp rspamd{text} objects.
  831. * Allow to specify custom stop pattern for lua_tcp.
  832. 0.9.4:
  833. * Fix critical bugs in tokenization algorithm
  834. * Write unit tests for tokenization
  835. * Add documentation for lua_tcp
  836. * Switch off legacy tokenization by default.
  837. * Fix critical bugs in words normalization
  838. * Add lua bindings to tokenizer.
  839. * Implement storing of HTTP headers inside task
  840. * Add lua API to accerss HTTP headers data
  841. * Implemented base64 encoding suitable for MIME
  842. * Use caseless hash and equal functions for HTTP request headers.
  843. * Improve debian architectures support (by @dottedmag)
  844. 0.9.3:
  845. * Revert incorrect regexp change that broke the default rules
  846. * Fix lua_tcp module
  847. 0.9.2:
  848. * Fix error on spawning unique workers.
  849. * Add preliminary version of generic LUA TCP requests API.
  850. * Use lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc)
  851. * Fix fuzzy mime strings with only type.
  852. * Improve thunderbird sanity checks.
  853. * Fix critical bug on matching regular expressions.
  854. * Make hiredis optional dependency.
  855. * Fix multiple bugs in daemon reloading
  856. 0.9.1:
  857. * Restore utf8 validation for regular expressions to avoid crashes
  858. * Fix symbols displaying in the interface
  859. * Add symbol groups to the interface
  860. * Fix maps ID parsing in the controller
  861. * Add multimap and regexp modules documentation
  862. * Backport fixes from libucl
  863. * Fix debian package (by @dottedmag)
  864. * Rework XXH32 invocations
  865. 0.9.0:
  866. * Add support of the fast and secure protocol level encryption:
  867. - curve25519 is used for key exchange;
  868. - chacha20/poly1305 cryptobox construction for bulk encryption;
  869. - zero latency overhead;
  870. - encrypting and balancing HTTP proxy worker
  871. * Rework expressions and create new expressions library:
  872. - aggressive optimizations based on the abstract syntax tree;
  873. - abstract expressions support (regular expressions, functions, lua modules
  874. composites and so on)
  875. - New comparision and '+' operators support
  876. - New greedy algorithm to minimize execution time of expressions and
  877. all symbols
  878. - Dynamic expressions benchmark and reoptimizations
  879. * Many improvements to the LUA API:
  880. - reworked logger module allowing to do pretty print of the most of lua
  881. types (including tables and userdata classes)
  882. - reworked lua redis and lua HTTP to support more features
  883. - added opaque type for passing large text chunks without copying
  884. - new regexp module with many auxiliary functions (e.g. `re:split`)
  885. * LuaJIT is now the default requirement for rspamd allowing to speed up lua
  886. execution by a large margin (however, plain lua is still supported)
  887. * New plugins:
  888. - spamassassin rules plugin that allows to load and re-use the most of
  889. SA rules natively
  890. - DMARC plugin that evaluates SPF and DKIM policies to the domain policies
  891. - many old plugins has been reworked to implement new features and improve
  892. stability
  893. * New aho-corasic trie implementation from @mischasan that allows to load and
  894. use hundreds of thousands of patterns with no influence on load
  895. * Support of PCRE JIT and PCRE JIT fast path modes that significantly improves
  896. the performance of regular expressions if supported by PCRE
  897. * New URLs parser and extractor:
  898. - removed legacy code that was useless for url finding
  899. - reworked algorithms of URL parsing for more precise and accurate results
  900. - added top-level-domains tree from http://publicsuffix.org
  901. - improved emails parsing
  902. - removed many phishing false positives due to TLD tree check
  903. * New statistics infrastructure:
  904. - created a separate layer of statistic library
  905. - improved OSB-Bayes by re-weighting tokens according to the original
  906. academic paper and `crm114` implementation, which reduced false positives
  907. rate significantly
  908. - created learn cache to avoid double learning of statistics and providing
  909. an efficient way to re-learn class for a message
  910. - created abstract layers for different statistics backends
  911. - implemented new tokenization algorithms with fast or secure (siphash)
  912. hashes to generate statistics features
  913. * Reworked utf8 tokenization that previously corrupted all UTF8 words (minor
  914. incompatibility with old fuzzy hashes with utf-8 symbols)
  915. * SPF module has been completely rewritten to support complex cases of
  916. `include` and `redirect` within SPF records
  917. * DKIM module now supports multiple signatures
  918. * Controller passwords can now be stored encrypted by `PBKDF2-HMAC` in the
  919. configuration file
  920. * Many hand-written HTTP clients has been replaced with the common rspamd
  921. http module
  922. * New test framework:
  923. - import lua `telescope` test framework
  924. - add unit tests for many rspamd modules and routines
  925. - create a unit test for each possible bug found
  926. - use luajit ffi for testing C code
  927. - added preliminary support of functional testing by creating tasks from lua
  928. * Randomize hash seed to avoid certain hash tables vulnerabilities
  929. * Documentation improvements:
  930. - added documentation for the vast majority of rspamd modules
  931. - added documentation for rspamd protocol
  932. - added documentation for the most of rspamd LUA extensions
  933. * Fixed tonns of bugs and memory leaks
  934. * Added tonns of minor features
  935. 0.8.3:
  936. * Various critical fixes in distribution (by @dottedmag and @fatalbanana)
  937. * Fixed bugs in url detector to parse certain patterns
  938. * Add default host and helo for a client
  939. * Some sanity checks for tokenizer and classifier
  940. * Reiterate on systemd support
  941. * Fix missing symbol registration
  942. * Add support of spamc compatible output
  943. * Filter double-dots in rbl.lua validate_dns (by @fatalbanana)
  944. * Update ucl submodule due to critical bugfix
  945. 0.8.2:
  946. * Create fuzzy db if it does not exist
  947. * Fix: Centos init script: configtest() (by @AlexeySa)
  948. * Enable one_shot for RECEIVED_SPAMHAUS_XBL - Fixes #102 (by @fatalbanana)
  949. * Update Exim patch (by @fatalbanana)
  950. * Fix processing of unix sockets.
  951. * Allow applying settings to authenticated users (by @fatalbanana)
  952. * Make settings priorities work as documented (by @fatalbanana)
  953. * Fix race condition in symbols planner
  954. * Add DNSWL_BLOCKED symbol (by @fatalbanana)
  955. * Make Exim pass usernames to rspamd (by @fatalbanana)
  956. * Update RBL module (by @fatalbanana):
  957. - fix indentation;
  958. - collapse loops;
  959. - avoid calling for un-needed information;
  960. - allow disabling RBLs for authenticated users
  961. * once_received.lua: Fix indentation & add exclusion for authenticated users (by @fatalbanana)
  962. * hfilter.lua: Add exclusion for authenticated users (by @AlexeySa)
  963. * Updates to hfilter rules (by @AlexeySa)
  964. * Set empty <> user or addr for msgs without FROM (by @eneq123)
  965. * Fix: attempt to index field '?' (a nil value) (by @eneq123)
  966. * Fix: if not exist Date-header (by @AlexeySa)
  967. * Add task:get_content() method.
  968. * rbl.lua: Ignore private IP space (by @fatalbanana)
  969. * Allow to check radix maps from lua by rspamd{ip}
  970. * Make local exclusions configurable per-RBL (by @fatalbanana)
  971. * Add rspamd_config:radix_from_config() (by @fatalbanana)
  972. * Support emails dnsbl in rbl (by @fatalbanana)
  973. * Complete rework of url extraction logic
  974. * Allow customizations for unix sockets. (fixes #182)
  975. * Set lua path according to rspamd settings.
  976. * Import lua-functional for plugins stuff.
  977. * Completely rewrite multimap plugin in functional style.
  978. * Fix FORGED_MUA_THUNDERBIRD_MSGID (fixes #186)
  979. * Check IPv6 addresses at dnswl.org and Spamhaus whitelist (by @fatalbanana)
  980. * Add lowercase utility for utf8 strings.
  981. * Various fixes to build system
  982. * Updated debian configuration infrastructure (by @dottedmag)
  983. 0.8.1:
  984. * Add sqlite and perl as dependencies for RPM/Debian packages (by @fatalbanana)
  985. * Remove whitelist.lua from RPM file list (by @fatalbanana)
  986. * Make Exim pass hostnames to rspamd (by @fatalbanana)
  987. * Fix building on Fedora (by @fatalbanana)
  988. * Add toggle for disabling installation of systemd units on Linux (by @fatalbanana)
  989. * Fix double format rounding that caused output corruption (reported by @fatalbanana)
  990. * Revert broken change for destructors ordering that led to memory corruption
  991. * Do not reset symbols case of settings if parsed from lua (reported by @andrejzverev)
  992. * Fix build on SunOS (by @wiedi)
  993. * Fix multiple crashes on broken DKIM DNS records
  994. * Fix critical issue with composites weights removing
  995. * Fix memory corruption in composites processing code
  996. * Ignore non-SPF TXT records when parsing SPF includes
  997. 0.8.0:
  998. * New fuzzy check logic:
  999. - use shingles algorithm for fuzzy matching
  1000. - use blake2 instead of md5 for larger output space
  1001. - combine fuzzy and strict matching
  1002. - allow to organize private storages by means of keys
  1003. - preserve compatibility with previous versions
  1004. * New fuzzy storage:
  1005. - use sqlite instead of own memory based hash tables
  1006. - rework commands interface
  1007. - add conversion from the old format
  1008. - add fuzzy match by shignles
  1009. - support old rspamd versions
  1010. * Add lemmatizing for words used in fuzzy hashes that allows to improve match
  1011. quality by using of the first forms of all words
  1012. * Rework language detection
  1013. * Fix several critical bugs, memory leaks and deadlocks:
  1014. - memory leak in HTML nodes parsing
  1015. - deadlock in logger code
  1016. - deadlock in signals processing
  1017. - crashes in fuzzy_storage
  1018. - crashes in tokenizers if the input was empty
  1019. * Import new libucl with several bugfixes and improvements
  1020. * Support listening on ipv6 addresses only
  1021. * Fix macro expansion in SPF module
  1022. * Several bugfixes in DKIM module
  1023. * Add load headers support for mime parts to the lua API
  1024. * Add documentation for:
  1025. - workers in general
  1026. - fuzzy_storage worker
  1027. - fuzzy_check plugin
  1028. - mimepart and textpart lua API modules
  1029. 0.7.6:
  1030. * Apply boundary fix for dkim simple canonization
  1031. * Fix ping command
  1032. * Return nil if header was not found in lua_task
  1033. * Fix hang in upstreams revive logic
  1034. * Decode entitles when normalizing HTML parts
  1035. * Fix logic of finding URLs in HTML parts
  1036. * Do not include \0 into length of text when performing conversion to utf8
  1037. * Fix raw vs parsed reperesentations
  1038. Raw parts are now:
  1039. - decoded b64/qp, but *NOT* converted to utf-8
  1040. Processed parts are now:
  1041. - converted to UTF-8
  1042. - normalized if needed (e.g. HTML tags are stripped)
  1043. * Rework DKIM canonization to line based
  1044. * Fix fuzzy hashes addding
  1045. * Use more specific hash function for fuzzy
  1046. * Fix leaking of iconv descriptors
  1047. * Fix PTR resolving in lua resolver
  1048. * Rework spf module.
  1049. - Copy data to memory pool as cached record might be destroyed causing
  1050. freed memory being passed to the protocol output (use after free)
  1051. - Allow SPF_NEUTRAL policy to be handled separately
  1052. - Add R_SPF_NEUTRAL to the default config
  1053. * Rework `register_symbols` function
  1054. * Allow to disable components of hfilter
  1055. 0.7.5:
  1056. * Fix owner when creating folder /run/rspamd (by @sfirmery)
  1057. * Fix IP validity checks
  1058. * Decode URLs obtained from HTML tags
  1059. * Fix crash with unweighted upstreams
  1060. * Stop processing headers in parts
  1061. * Set sockaddr.sa_family properly when connectig to upstreams
  1062. * Fix reload issues in surbl and fuzzy_check (reported by @citrin)
  1063. * Fix timeouts in redirector
  1064. * Improve lua errors reporting
  1065. * Fix lua closures processing in libucl
  1066. * Rework calling of lua functions from regexp module
  1067. * Choose raw regexp for raw headers
  1068. * Rework conversion to utf since glib one is broken
  1069. * Ignore SGML style tags in html
  1070. * Fix old bug with non-capturing https urls
  1071. * Fix memory corruption on fuzzy reload (reported by @citrin)
  1072. * Fix percents display in rspamc
  1073. * Fix buffer update for DKIM
  1074. * Do not validate utf for raw headers
  1075. 0.7.4:
  1076. * Fix build under *BSD
  1077. * Detect HAN unicode script
  1078. * Implement language detection heuristic for text parts
  1079. * Fix time output in history
  1080. * Improve piechart coloring
  1081. * Fix \r\n conversion in DKIM module (reported by @citrin)
  1082. * Try to detect systems with no IPv6 support
  1083. * Fix multiple/single values in use settings (reported by @citrin)
  1084. * Rework IP addresses in upstreams:
  1085. - Select ipv4/unix addresses if they exist and use ipv6 for ipv6 only
  1086. upstreams (since the support of ipv6 is poor in many OSes and
  1087. environments)
  1088. - Free IP list on upstream destruction
  1089. - Add test cases for addresses selection
  1090. - Allow adding of free form IP addresses to upstreams
  1091. * Fix endiannes in lua_radix search (reported by @citrin)
  1092. * Soft shutdown should also set wanna_die flag (reported by @citrin)
  1093. * Stop use-after-free in event loop termination
  1094. * Fix processing of very short messages in DKIM (reported by @citrin)
  1095. * Detect systems without shared mutexes
  1096. * Fix issues with PTR and MX elements in SPF parser (reported by @citrin)
  1097. 0.7.3:
  1098. * New upstreams code:
  1099. - simplify upstreams API;
  1100. - unify strings parsing in upstreams definition;
  1101. - add configuration options for the upstreams;
  1102. - for failed upstreams re-resolve their addresses;
  1103. - use all resolved addresses for an upstream (round-robin);
  1104. - implement stable hashing and use it by default for upstreams;
  1105. - add unit test for upstreams module.
  1106. * Rework signals processing in all rspamd workers:
  1107. - signals are now processed in the event loop;
  1108. - implement the most common signal handlers for all workers;
  1109. - add callbacks for workers specific signal handlers
  1110. * Fix critical issue with fuzzy storage:
  1111. Fuzzy stroage could not save any hashes on termination due to bugged
  1112. signals handling
  1113. * Fix roll history IP storage
  1114. * Rework ipv4/ipv6 handling in parsing addresses:
  1115. - turn off support of IPV6_V6ONLY socket option;
  1116. - create ipv6 socket prior to ipv4 one to handle systems with v6/v4
  1117. sockets enabled (Linux)
  1118. * Remove CBL as it's wholly included in Spamhaus XBL (by @fatalbanana)
  1119. * Remove nszones.com fake RBL (by @citrin)
  1120. * Fix upstreams interaction for fuzzy_check
  1121. * Verify spf PTR records (reported by @citrin)
  1122. * Fix spf MX records parsing
  1123. * Add compatibility for old libevent (by @yellowman)
  1124. * Sync bugfixes from libucl
  1125. 0.7.2:
  1126. * Convert all maps to the compressed radix trie
  1127. * Allow IPv6 addresses in IP maps
  1128. * Remove dynamic items support from symbols cache
  1129. * Allow hex encoded output of strings
  1130. * Fix bug with control connections count
  1131. * Process fuzzy weight correctly (reported by @fatalbanana)
  1132. * Remove extra reference retain of http connection on error
  1133. * Remove deprecated options from the default config
  1134. * Add `one_shot` attr to metric's symbols
  1135. * Doc: add documentation for metrics
  1136. * Add Upstart job to debian packaging (by @CameronNemo)
  1137. * Config: improve SURBL symbols descriptions (by @citrin)
  1138. * Config: reflect SURBL changes (by @citrin):
  1139. - Outblaze removed, malware moved to separate list:
  1140. http://www.surbl.org/news/internal/MW-malware-sublist-added-to-multi
  1141. * Fix C modules initialization on restart
  1142. * Treat single IP as a single IP in radix lists (reported by @citrin)
  1143. * Do not touch file and core limits if not asked explicitly (reported by @citrin)
  1144. * Improve logging for fuzzy errors
  1145. * Block SIGPIPE for HTTP writing
  1146. * Doc: update manual pages
  1147. * Fix HTTP connection termination
  1148. * Reduce default number of parallel requests to 8
  1149. * Sync with libucl include features
  1150. 0.7.1:
  1151. * Fix typo in stat output.
  1152. * Fix issues with includes crossing with the system includes
  1153. * Restore testing framework
  1154. * Add radix trie test suite
  1155. * Implement new path-compressed radix trie.
  1156. - The performance benefit over the old algorithm is about 1.5 times.
  1157. - Memory usage is significantly lower as well.
  1158. - Now radix trie can accept any IPv4/IPv6 values
  1159. * Various improvements to the memory pools code
  1160. * Fix writing reply to a client when no filters are defined
  1161. * Write base32 encoded fuzzy
  1162. * Fix 'soft reject' action
  1163. * Fix rspamd reload and modules reconfiguration
  1164. * Fix subject rewriting for the default subject
  1165. * Fix states for processing task and pre-filters
  1166. * Fix issues with connection closing
  1167. * Fix crashes in rdns
  1168. * Fix ratelimit pre-filter
  1169. * Update exim patch.
  1170. - Update to the recent exim version
  1171. - Strip extra leading src/ from the patch
  1172. - Remove sendfile since it was broken
  1173. - Fix rspamd spam report for exim
  1174. * Improve documentation
  1175. 0.7.0:
  1176. * Use HTTP protocol for all operatiosn
  1177. * Webui worker is now removed and controller works as webui
  1178. * Allow to serve static files via controller by option `static_dir`
  1179. * Rspamd interface is now a part of rspamd
  1180. * Rspamc client has been rewritten to use HTTP and non-blocking mode
  1181. allowing to start multiple operations simultaneously (see `-n` option)
  1182. * Lua API was completely reworked to satisfy modern standards of LUA:
  1183. * Module `lua-message` was removed
  1184. * Reduced number of superglobals registered by rspamd
  1185. * Many functions has been redesigned
  1186. * Symbols registration is now more convenient
  1187. * Users settings has been rewritten as lua plugin
  1188. * Reworked headers system as gmime's based one misses many headers and is
  1189. very slow to get headers values
  1190. * Reorganized code and removed embedded jannsson by using UCL for all json
  1191. parsing
  1192. * Migrated to `librdns` for DNS resolving that improves concurrency for
  1193. DNS requests significantly
  1194. * Fixed tonns of bugs in MIME processing
  1195. * Improved metrcis and default symbol's weights
  1196. * Added new RBL's
  1197. * Fixed a number of issues in the modules
  1198. * Removed several memory leaks found
  1199. * Fix unicode processing
  1200. * Fix fuzzy checking for unicode parts
  1201. * Significantly improve documentation and especially LUA API docs
  1202. * See migration notes at https://rspamd.com/doc/migration.html
  1203. 0.6.8:
  1204. * Controller now listen for localhost and not for 127.0.0.1 by default
  1205. * Allow FCrDNS-style RBL lookups (by @fatalbanana)
  1206. * Reduce threshold for parts_differ function.
  1207. * Fix hostname lookup for rdns rbl (by @AlexeySa)
  1208. * Fix HFILTER_URL_ONELINE to reduce false positive rate.
  1209. * Fix whitelist module.
  1210. * Allow override system predefined settings without touching system ones
  1211. by .try_include macro (by @andrejzverev)
  1212. * Check for [ip.address]-style HELO and suppress lookups. (by
  1213. @fatalbanana)
  1214. * Optimize hfilter (by @AlexeySa)
  1215. * Fix issue with random numbers generator in dns.
  1216. * Use more clever time values to setup entropy.
  1217. * Synced with the recent libucl.
  1218. 0.6.7:
  1219. * Use ChaCha20 for DNS generator (more secure DNS id)
  1220. * Unknown symbols now has zero weight and not 1.0
  1221. * Fix fuzzy hashes expire time
  1222. * Fix critical issue in statfiles on FreeBSD 9 (and some other platforms)
  1223. * Add .include_map macro to ucl parser
  1224. * Update libucl
  1225. * Fix headers end detection for DKIM module
  1226. * Fix a bug in received headers parser
  1227. * Validate IP addresses before pushing them to lua
  1228. * Start new documentation project
  1229. * Fixed tonns of other minor bugs
  1230. * Start to prepare for 0.7 with HTTP protocol and new settings
  1231. 0.6.6:
  1232. * Removed issue with BUFSIZ limitation in the controller output
  1233. * Simplify logging symbols escaping
  1234. * Adjusted weights for several rules
  1235. * Improve spamhaus rbl support
  1236. * Removed PBL for received headers checks
  1237. * Added hfilter module that performs various HELO and IP checks.
  1238. * Rspamd can now be reloaded by HUP signal
  1239. * Fuzzy storage should expire hashes properly
  1240. * Build system has been reworked for better supportof pkg-config
  1241. * Various minor bugfixes
  1242. 0.6.5:
  1243. * Fixed critical bug in DNS resolver, introduced in 0.6.4
  1244. * Improved multimap and rbl plugins to skip
  1245. * Add dns_sockets option for tuning sockets per server in DNS resolver
  1246. * Improved packages for rspamd
  1247. 0.6.4:
  1248. * Added io channels for DNS request to balance load and reduce id
  1249. collisions chance
  1250. * Fixed a bug in SPF filter that may cause core dump in specific
  1251. circumstances
  1252. * FIxed default config for rbl module
  1253. * It is possible to get a list of rspamc commands with their descriptions
  1254. * Added SORBS bl to the default config
  1255. * 2tld file for surbl module has been significantly extended
  1256. * Perl modules has been removed from the code.
  1257. * Fixed an issue in libucl when parsing macros
  1258. 0.6.3:
  1259. * Fixed issues with DNS:
  1260. - labels decompression algorithm was fixed;
  1261. - added resolve_mx to lua API;
  1262. - fixed modules that use DNS.
  1263. * Lua modules once_received and emails reworked for new resolver API and UCL.
  1264. * Debian package was polished.
  1265. * Fixed a bug in fuzzy_check module that prevents correct processing messages
  1266. without valid parts.
  1267. 0.6.2:
  1268. * Fuzzy check module has been reworked:
  1269. - now fuzzy_check operates with a group of rules, that define which
  1270. servers sre to be checked;
  1271. - it is possible to specify read_only groups to avoid learning errors;
  1272. - turn fuzzy_check to one_shot mode permanently;
  1273. - fuzzy_check module configuration is now incompatible with the previous
  1274. versions.
  1275. * Imported bugfixes from libucl.
  1276. * Fixed whitelist plugin.
  1277. * Fixed statfiles resizing.
  1278. * Improved logging initialization order.
  1279. * Fixed race condition in the controller worker.
  1280. 0.6.1:
  1281. * Critical bugfixes:
  1282. - fixed build system;
  1283. - fixed in_class setting in bayes learning;
  1284. 0.6.0:
  1285. * Use UCL instead xml for configuration (https://github.com/vstakhov/libucl)
  1286. * Fix statistics module normalization
  1287. * Rework the many modules for the new configuration:
  1288. - surbl has incompatible configuration;
  1289. - fuzzy_check has incompatible configuration;
  1290. - multimap has now new configuration;
  1291. - received_rbl is removed and replaced by rbl module.
  1292. * Removed deprecated options:
  1293. - statfile_pool_size;
  1294. - action and required/reject score for a metric.
  1295. * Simplify build system and unify configuration for all platforms.
  1296. * Improved URL detector (reduced false positive rate).
  1297. * Lua 5.2 is now the default and fully supported version.
  1298. * Tons of bugfixes and minor improvements.
  1299. 0.5.6:
  1300. * Fix bug with counters incrementing in rolling history
  1301. * Detect expl and exp2l as some systems do not have it
  1302. * Support input streams without Content-Length
  1303. * Implement counters output via rspamc and controller interface
  1304. * Fix bug with udp sockets in fuzzy storage
  1305. 0.5.5:
  1306. * New bayes normalizator based on inverse chi-square function
  1307. * Various fixes to fuzzy storage
  1308. * Allow update fuzzy storage only from specific IP addresses
  1309. * Better support of IPv6 and address selection algorithms
  1310. * Add CentOS spec file
  1311. 0.5.4:
  1312. * Fixed issues with diff algorithm
  1313. * Added support of RRD statistics
  1314. * Add webui worker for interface interaction
  1315. * Fix a lot of issues with dynamic conf
  1316. * Fix critical memory leak in settings code
  1317. * Improve stability of the system
  1318. 0.5.3:
  1319. * Added dynamic options
  1320. * Added advanced metaclassfication
  1321. * Added RESTfull API for controller
  1322. * Improved hashing algorithms
  1323. * Various fixes for rspamc client:
  1324. - allow interacting with unix sockets
  1325. - librspamdclient major cleanup
  1326. - bayes is now default classifier
  1327. 0.5.2:
  1328. * Added lua bindings for:
  1329. - basic mime parts, that allows checkign attachements for example;
  1330. - DNS resolver;
  1331. * Existing lua bindings now works without task object allowing to use them
  1332. in custom code.
  1333. * Threads system was reworked to avoid global lua interpreter lock.
  1334. * DKIM module now converts all line endings to CRLF how opendkim does.
  1335. * URL detector is now more accurate for text parts.
  1336. * Several critical bugs and memory leaks were fixed.
  1337. 0.5.1:
  1338. * Added lua worker type to handle network connections in lua.
  1339. * Added lua bindings for async_session, IO dispatcher, memory_pool, and
  1340. worker.
  1341. * Composites can now uses other composites in expressions.
  1342. * Fixes for debian package and for FreeBSD >= 9.1
  1343. * Add support of gmime-2.6 if gmime-2.4 is not found.
  1344. * Improve url detection and phishing detection.
  1345. * Add lua mime_part library to get an access to all message part
  1346. attributes (like filename, length, type).
  1347. 0.5.0:
  1348. * Added SMTP lightweight balancing proxy with XCLIENT support.
  1349. * Added lua bindings for upstreams objects and API.
  1350. * New pre-filters are implemented to support initial checking for messages.
  1351. * Added ratelimit plugin that uses redis protocol to store data.
  1352. * Added ipv6 support to spf and some other modules.
  1353. * Unbreak spf plugin.
  1354. * Allow options with the same name be threated as list.
  1355. * DKIM plugin an parsing code was added.
  1356. * Separate build system to put logic in several shared libraries.
  1357. * Many bugfixes.
  1358. 0.3.2:
  1359. * Add error handling for regexps
  1360. * Fix quit command in controller interface
  1361. * Write symbols weights to rspamc output
  1362. * Improve logic of selecting rspamc version
  1363. * Do not try to parse broken DNS replies
  1364. * Add 'raw' flag to FROM_EXCESS_BASE64 rule (requested by citrin)
  1365. * Output message id in rspamc reply
  1366. * Fix inserting composite symbol
  1367. * Fix output of log line
  1368. * Document composites
  1369. * Add logging for fuzzy checks
  1370. * Add logging for learning
  1371. * Improve logic of learning messages - do not learn more than specific threshold
  1372. * Fix inserting results for symbols that were incorrectly (for example more than 1 time) defined in config file
  1373. * Do not output control characters if output is not terminal
  1374. * Fix some logic errors in learning
  1375. * Consider lua plugins errors as fatal configuration errors
  1376. * Fix wirting message id during fuzzy_add command
  1377. * Display weight of symbols correctly
  1378. * Fixes to winnow learning
  1379. * One more try to improve accuracy of winnow algorithm
  1380. * Add bayesian classifier (initial version)
  1381. * Remove normalizer as it is winnow specific thing, so all statistic algorithms now returns value from 0 to 1
  1382. * Some fixes to fuzzy hashes expiration:
  1383. * Fix assertion while look up value in NULL hash (found by cirtin)
  1384. * Fix normalization for systems that have not tanhl function
  1385. * Ignore rfc822 group addresses
  1386. * Move images library to core rspamd
  1387. * Add lua api to access images properties
  1388. * Add post filters to lua API - filters that would be called after all message's processing
  1389. * Add ability to check for specified symbol in task results from lua
  1390. * Add ability to check for metric's results from lua
  1391. * Add ability to learn specified statfile form lua
  1392. * Add ability to extract filename and size of images from lua
  1393. * Fix assertion while extracting internet address
  1394. * Fix races in fuzzy storage
  1395. * Make spf parser case insensitive
  1396. * Add ability to check hashes of selected mime types
  1397. * Add ability to set minimum size in bytes for mime types
  1398. * Add ability to set minimum dimensions for images
  1399. * Assume all text/* content types as text/plain
  1400. * Fix getting data wrapper for gmime24
  1401. * Many fixes to statfile syncronization system
  1402. * Fixed statfile pool initialization and synchronization with disk
  1403. * Prepare 0.3.2
  1404. * Fix Mail::Rspamd::Config for new rspamd features
  1405. * Use Mail::Rspamd::Config in rspamc client
  1406. * Write user's name to rspamd log
  1407. * Prepare rspamd build infrastructure for creating rpm and deb packages
  1408. * Fix depends
  1409. * Add start script for linux systems
  1410. * Fix shared usage of statfiles
  1411. * Add invalidation of statfiles in case of learning, so now statfiles
  1412. * This should fix shared usage of statfile pool by several processes
  1413. * Fix misprint (reported by az)
  1414. * Fix stupid error when all checks can be done in a single pass
  1415. * New trie based url scanner (based on libcamel)
  1416. * Small fixes to rspamd perl client
  1417. * Write fuzzy hashes info to log
  1418. * Add trie interface to lua api
  1419. * Explain sample config and cleanup it
  1420. 0.3.1:
  1421. * Add modules documentation
  1422. * Continue implementing smtp proxy
  1423. * Implement new learning system, now rspamd should be much more intelligent while learning messages
  1424. * Convert statistic sums to use long double for counters
  1425. * Use hyperbolic tangent for internal normalizer
  1426. * In classify normalize result after comparing, not before
  1427. * New symbols sorter
  1428. * Fix strict aliasing while compiling with optimization
  1429. * Fix tanhl detection for platforms that have not implementation of it
  1430. * Remove several compile warnings
  1431. * Add experimental support of dynamic rules to regexp module
  1432. * Document views configuration
  1433. * Several fixes to documentation
  1434. * Add more logic for dynamic rules
  1435. * Add documentation for dynamic rules
  1436. * Add ability to make negations in networks in dynamic rules
  1437. * Clean up cache items correctly
  1438. * Implement basic SMTP dialog:
  1439. * Implement interaction with smtp upstream (with support of XCLIENT)
  1440. * Check messages received via smtp proxy
  1441. * Add support for sendfile in io dispatcher
  1442. * Fix issues with compatibility of worker_task and smtp proxy
  1443. * Proxy DATA command
  1444. * Fix SMTP
  1445. * Change metric logic
  1446. * Completely remove lex/yacc readers for config
  1447. * Make common sense of metric/action and symbols
  1448. * Sync changes with all plugins
  1449. * Incorrectly removed in previous commit
  1450. * Fix misprint (by Andrej Zverev)
  1451. * announce the "password" keyword in usage list
  1452. * Implement initial version of greylisting triplets storage
  1453. * Fix issues with smtp worker
  1454. * Fix QUIT command in SMTP worker
  1455. * Some fixes about new metrics system (may be incomplete)
  1456. * Get weights of symbol from default metric for symbols cache
  1457. * Fix setting task->from/task->rctp in smtp client
  1458. * Copy from and rcpt correctly
  1459. * Some performance improvements to IO dispatcher (do not drain the whole buffer after a single line readed)
  1460. * Fix smtp data input
  1461. * Fix misprint
  1462. * Add limit of maximum allowed smtp session errors
  1463. * New logic of SURBL module:
  1464. * Use system mkstemp(3) on systems where it is available as glib implementation
  1465. * Try to fix memmove issues in io dispatcher
  1466. * Remove debug from SURBL module
  1467. * Rewrite buffered input for line policy (again)
  1468. * Fix issue with links that are ip addresses in numeric form in surbl
  1469. * On Darwin use BSD style sendfile definition
  1470. * Reorganize platform specific knobs in CMakeLists
  1471. * Use gettimeofday on systems that have not clock_getres
  1472. * Use ftime for dns trans id generation on systems without clock_getres
  1473. * Darwin sendfile(2) support
  1474. * TIMEDB->TIMEB
  1475. * More to previous commit
  1476. * Pass env from main() arguments instead of platform specific global environ
  1477. * Fix compatibility issues
  1478. * Fix -lintl detection
  1479. * Init some variables to avoid problems
  1480. * Remove garbadge (gnome terminal sucks)
  1481. * Add more information about why we drop smtp connection
  1482. * Fix mkstemp call
  1483. * Send to upstream QUIT command at the end of session
  1484. * Check return value of each rspamd_dispatcher_write as in case of write errors sessions can be destroyed early
  1485. * Fix states in smtp dialog
  1486. * Use rspamd_snprintf instead of libc one
  1487. * Fix URLS command
  1488. * Fix reconfigure process of surbl module
  1489. * Fix destroying smtp session (unmap memory and do not delete pool early)
  1490. * Delete pool after using its variables
  1491. * Delay timer must be registered in async session to correctly handle connection termination
  1492. * Register dns requests in session too
  1493. * Make session before registering events
  1494. * Remove events in handlers
  1495. * Add ability to set filters for smtp worker for each smtp stage
  1496. * Add very initial version of DNS resolver (many things to be done)
  1497. * Announce weights and sync
  1498. * Fix few typo
  1499. * Understand short names of facility in logging config
  1500. * Add ability to make whitelist for spf checks
  1501. * Misprint != -> ==
  1502. * Handle lua tag in way that it is not required to write additional text:
  1503. * Strip all starting whitespace symbols from xml texts
  1504. * Fix stupid bug in calculating buffer length while reading file maps
  1505. * Add resolv.conf parsing into dns.c
  1506. * Fix microseconds<->milliseconds conversions
  1507. * Take callback argument in Mail::Rspamd::Client for processing files and directories
  1508. * Print results if rspamc is called for a directory
  1509. * Fix stupid error with surbl module reconfig (another one, blame me)
  1510. * Do not show duplicate urls in url header
  1511. * Fix detection of numeric urls (reported by citrin)
  1512. * Write real time of message's scan to log (not only virtual)
  1513. * Fix chartable module in utf mode
  1514. * Fix parsing of some broken urls
  1515. * Add ability to test regexp with 'T' flag
  1516. * Write more code for DNS resolver:
  1517. * Make DNS resolver working
  1518. * Many improvements to rspamd test suite: now it CAN be used for testing rspamd functionality
  1519. * Write DNS resolver tests
  1520. * Fix issues with memory_pool mutexes and with creating of statfiles
  1521. * Forgotten in previous commit
  1522. * Add support for parsing SPF and SRV records
  1523. * Fix PTR parsing
  1524. * Add tests
  1525. * Make SURBL module to use rspamd dns module
  1526. * Several fixes to DNS logic
  1527. * Remove evdns and use only rspamd resolver
  1528. * Very hard to detect problem with race among error in socket and destroying task while we are writing to socket and go through a hash table
  1529. * Fix resolving in smtp module
  1530. * Init events before configuring resolver in smtp worker
  1531. * Set resolver inside task
  1532. * Fix reload signal (reported by citrin)
  1533. * Some improvements to redirector
  1534. * Call has_forked method to inform POE about fork
  1535. * Fix lua DNS code
  1536. * Decompress labels in DNS packets more strictly
  1537. * Fix some problems with TXT records
  1538. * Try to fix removing of DNS events
  1539. * Do not insert unparsed RR's into reply
  1540. * Calling callbacks may cause destroying session from which we are calling callback so we MUST call callback as the latest action
  1541. * Fix check_smtp_data function
  1542. * Add ability to make views by recipient
  1543. * Add ability to set metric's action from config file
  1544. * Fix bug with writing garbadge if message has no urls or no messages
  1545. * Fix bug with incorrect behaviour of compare_parts_distance function
  1546. * Add ability to assign several actions to one metric
  1547. * Report action in rspamc protocol
  1548. * Mail::Rspamd::Client and rspamc can now understand Action header too
  1549. * Write action to log as well
  1550. * Make valgrind happy about comparing symbols
  1551. * Add more debug to comparing parts distance function
  1552. * Write action even if message has no symbols
  1553. * Make improvements to HTML entites decoder: now it replaces entities with common characters and
  1554. * Add -d option to force debug output
  1555. * Assume 7bit as default transfer encoding
  1556. * Do not overwrite lua plugins and configs if they already exists in target directory
  1557. * Improve logging
  1558. * Write queue id to log
  1559. * Remove test messages from cmake
  1560. * Reopen log file by USR1 signal
  1561. * Add reopenlog method to FreeBSD rc script
  1562. * Adopt foreach for cmake 2.6
  1563. * Fix to rc script
  1564. * Do not try to resolve names with several dots in a row
  1565. * Fix surbl request formatting for ip addresses
  1566. * Handle cases of broken requests
  1567. * Fix problems with parsing compressed names
  1568. * Fix TXT records parsing
  1569. * Fix expanding spf macros that may fail in rare cases
  1570. * Fix another error with early task destroying
  1571. * Handle empty from header
  1572. * Improve reopenlog command in rc script
  1573. * Strip trailing whitespace characters in Mail::Rspamd::Client
  1574. * Use ungreedy match to strip trailing whitespaces
  1575. * Stupid error in calculation compressed label length
  1576. * Some optimizations to client library
  1577. * Do not compare empty parts
  1578. * Empty and non-empty parts are allways different
  1579. * Save in regexp cache the whole regexp with header name (if exists) and with flags
  1580. * Add rspamd_log variable to lua plugins to access logging functions
  1581. * Each part in rspamd task now can have parent part
  1582. * Check for parts distance only for multipart/alternative subparts
  1583. * Do not check attachements even if they are text (but attached as file)
  1584. * Do not die if write (2) returned ENOSPACE while doing logging, turn on throttling mode instead (1 write try in a second)
  1585. * Add ability to turn on debug for specific symbols
  1586. * Add ability to configure dns timeouts and dns retransmits in config file
  1587. * More debug
  1588. * Fix extracting arguments in lua logger interface
  1589. * Turn off debug_ip during reload if it was disabled by new config
  1590. * Improve lua logging
  1591. * Pre-init symbols cache when rereading config
  1592. * Fix lua representing of invalid ip (nil, not 255.255.255.255)
  1593. * Fix R_TO_SEEMS_AUTO rule (by citrin)
  1594. * Add multimap lua plugin
  1595. * Fix some multimap issues
  1596. * Try to save images hashes to fuzzy storage to stop some annoying spammers
  1597. * Allocate some more bytes for read buffer to avoid incorrect behavoiur
  1598. * Add ability to check dns black lists by multimap module
  1599. * Add multimap documentation
  1600. * Fix labels parsing
  1601. * Another try to save regexps in cache correctly
  1602. * Improve test logs for regexps
  1603. * Fix parsing txt records to avoid reading of uninitialized data
  1604. * Fix error with writing symbols cache file
  1605. * Fix error while working in utf mode when raw regexps was not created properly
  1606. * Do not add extra byte while converting text to utf
  1607. * Add error handling for regexps