1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144
2145
2146
2147
2148
2149
2150
2151
2152
2153
2154
2155
2156
2157
2158
2159
2160
2161
2162
2163
2164
2165
2166
2167
2168
2169
2170
2171
2172
2173
2174
2175
2176
2177
2178
2179
2180
2181
2182
2183
2184
2185
2186
2187
2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
2214
2215
2216
2217
2218
2219
2220
2221
2222
2223
2224
2225
2226
2227
2228
2229
2230
2231
2232
2233
2234
2235
2236
2237
2238
2239
2240
2241
2242
2243
2244
2245
2246
2247
2248
2249
2250
2251
2252
2253
2254
2255
2256
2257
2258
2259
2260
2261
2262
2263
2264
2265
2266
2267
2268
2269
2270
2271
2272
2273
2274
2275
2276
2277
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
2291
2292
2293
2294
2295
2296
2297
2298
2299
2300
2301
2302
2303
2304
2305
2306
2307
2308
2309
2310
2311
2312
2313
2314
2315
2316
2317
2318
2319
2320
2321
2322
2323
2324
2325
2326
2327
2328
2329
2330
2331
2332
2333
2334
2335
2336
2337
2338
2339
2340
2341
2342
2343
2344
2345
2346
2347
2348
2349
2350
2351
2352
2353
2354
2355
2356
2357
2358
2359
2360
2361
2362
2363
2364
2365
2366
2367
2368
2369
2370
2371
2372
2373
2374
2375
2376
2377
2378
2379
2380
2381
2382
2383
2384
2385
2386
2387
2388
2389
2390
2391
2392
2393
2394
2395
2396
2397
2398
2399
2400
2401
2402
2403
2404
2405
2406
2407
2408
2409
2410
2411
2412
2413
2414
2415
2416
2417
2418
2419
2420
2421
2422
2423
2424
2425
2426
2427
2428
2429
2430
2431
2432
2433
2434
2435
2436
2437
2438
2439
2440
2441
2442
2443
2444
2445
2446
2447
2448
2449
2450
2451
2452
2453
2454
2455
2456
2457
2458
2459
2460
2461
2462
2463
2464
2465
2466
2467
2468
2469
2470
2471
2472
2473
2474
2475
2476
2477
2478
2479
2480
2481
2482
2483
2484
2485
2486
2487
2488
2489
2490
2491
2492
2493
2494
2495
2496
2497
2498
2499
2500
2501
2502
2503
2504
2505
2506
2507
2508
2509
2510
2511
2512
2513
2514
2515
2516
2517
2518
2519
2520
2521
2522
2523
2524
2525
2526
2527
2528
2529
2530
2531
2532
2533
2534
2535
2536
2537
2538
2539
2540
2541
2542
2543
2544
2545
2546
2547
2548
2549
2550
2551
2552
2553
2554
2555
2556
2557
2558
2559
2560
2561
2562
2563
2564
2565
2566
2567
2568
2569
2570
2571
2572
2573
2574
2575
2576
2577
2578
2579
2580
2581
2582
2583
2584
2585
2586
2587
2588
2589
2590
2591
2592
2593
2594
2595
2596
2597
2598
2599
2600
2601
2602
2603
2604
2605
2606
2607
2608
2609
2610
2611
2612
2613
2614
2615
2616
2617
2618
2619
2620
2621
2622
2623
2624
2625
2626
2627
2628
2629
2630
2631
2632
2633
2634
2635
2636
2637
2638
2639
2640
2641
2642
2643
2644
2645
2646
2647
2648
2649
2650
2651
2652
2653
2654
2655
2656
2657
2658
2659
2660
2661
2662
2663
2664
2665
2666
2667
2668
2669
2670
2671
2672
2673
|
1.4.0:
* [CritFix] Add guards for inactive redis connections
* [CritFix] Another fix for proxying files using rspamd_proxy
* [CritFix] Cleanup inactive redis connections
* [CritFix] Do not sometimes try to exec posfilters before classification
* [CritFix] Fix application of IPv6 mask
* [CritFix] Fix chunked encoding when reading messages
* [CritFix] Fix file mode for rspamd_proxy
* [CritFix] Fix hyperscan compilation on regexp change
* [CritFix] Fix issue with finding of end of lines pointers
* [CritFix] Fix iteration over headers array (introduced in 1.4)
* [CritFix] Fix processing of learned tokens count for redis backend
* [CritFix] Fix race condition in checking of cached maps
* [CritFix] Fix workers scripts by sharing workers configs
* [CritFix] Introduce raw content to text parts
* [CritFix] Plug memory leak and potential memory corruption
* [Feature] Adaptive ratelimits
* [Feature] Add ASN -> rbldnsd script for asn.rspamd.com
* [Feature] Add DMARC_NA symbol
* [Feature] Add F-Prot support to antivirus module
* [Feature] Add HTTP backend to metadata exporter
* [Feature] Add Lua API module for monitored objects
* [Feature] Add R_DKIM_NA / R_SPF_NA / AUTH_NA symbols
* [Feature] Add R_DKIM_PERMFAIL symbol
* [Feature] Add R_SPF_PERMFAIL symbol
* [Feature] Add Sophos antivirus support
* [Feature] Add ZSTD compression to Lua API
* [Feature] Add `mid` Lua module
* [Feature] Add `one_param` flag for metric symbols
* [Feature] Add a generic lua classifier
* [Feature] Add a very basic interface to access workers data from on_load
* [Feature] Add ability to delete a hash by its data to fuzzy_check plugin
* [Feature] Add ability to enable/disable symbols via dynamic_conf
* [Feature] Add ability to lookup settings by key
* [Feature] Add common way to disable Lua modules
* [Feature] Add compression support to rspamd client
* [Feature] Add condition to do antiviral check
* [Feature] Add configuration for lua classifiers
* [Feature] Add configuration knobs for the errors circular buffer
* [Feature] Add decompression support in rspamd client
* [Feature] Add errors exporter to the controller
* [Feature] Add expected value for monitored DNS resources
* [Feature] Add exporter from error ringbuf to ucl
* [Feature] Add extended version for fann creation function
* [Feature] Add ffi friendly version of process_regexp function
* [Feature] Add frequency and time display to webui
* [Feature] Add fuzzy_delhash command to rspamc client
* [Feature] Add implementation of redis connections pool
* [Feature] Add latency and offline time monitoring
* [Feature] Add learning support for lua classifiers
* [Feature] Add max-size and timeout options to CGP helper
* [Feature] Add method to enable/disable symbols in config
* [Feature] Add methods to get metric's actions and symbols from Lua
* [Feature] Add mmap support to lua_text
* [Feature] Add monitored object for surbl plugin
* [Feature] Add more exceptions to surbl whitelist
* [Feature] Add more meta-tokens to bayes
* [Feature] Add neural net classifier to fann_scores module
* [Feature] Add neural net serialization/deserialization
* [Feature] Add new dynamic conf module
* [Feature] Add periodic events support for lua_config
* [Feature] Add plugin to check MX'es for the sender's domain
* [Feature] Add preliminary monitored module
* [Feature] Add preliminary support of dynamic conf updates in Redis
* [Feature] Add preliminary version of clamav plugin
* [Feature] Add redis cache to asn module
* [Feature] Add replies compression
* [Feature] Add spamhaus DROP dnsbl
* [Feature] Add support for dictionary in client compression
* [Feature] Add support for fuzzy learn and unlearn from lua
* [Feature] Add support for input encryption
* [Feature] Add support of min_learns to neural net classifier
* [Feature] Add termination callbacks for workers
* [Feature] Add user-agent for rspamc
* [Feature] Add utility to perform classifier tests
* [Feature] Add zstd compression library
* [Feature] Allow HTTPS requests in lua_http
* [Feature] Allow conditions for pre and postfilters
* [Feature] Allow custom functions for ratelimits
* [Feature] Allow for excluding messages from AV scanning based on size
* [Feature] Allow for getting worker stats from Lua
* [Feature] Allow getting task UID from Lua
* [Feature] Allow parsing of mailbox messages from the commandline
* [Feature] Allow plugins to publish their lua API via rspamd_plugins
* [Feature] Allow to compare other systems with Rspamd
* [Feature] Allow to execute Lua scripts by controller
* [Feature] Allow to have a function to set custom greylist message
* [Feature] Allow to iterate over multiple tags
* [Feature] Allow to pass extra data from plugins to log helper
* [Feature] Allow to plan new periodics at different time
* [Feature] Allow to reset hashes
* [Feature] Allow to run rspamadm lua just as a lua interpreter
* [Feature] Allow to store settings in redis
* [Feature] Allow to update dynamic conf in Redis
* [Feature] Allow to use dictionaries for compression
* [Feature] Allow to use md5, sha1, sha256, sha384 and sha512 hashes in Lua
* [Feature] Allow whitelisting by IP for greylisting plugin
* [Feature] Antivirus: Support whitelists & pattern-matching sig names
* [Feature] Backport pack/unpack routines from Lua 5.3
* [Feature] Check settings with equal priopities in alphabetical order
* [Feature] Compress neural net in redis
* [Feature] Consider more tags when doing WHITE_ON_WHITE rule
* [Feature] Descriptive options for DMARC failure symbols
* [Feature] Descriptive options for RBL symbols
* [Feature] Enable configuration for monitored objects
* [Feature] Execute on_load scripts with ev_base ready
* [Feature] Fann scores now uses metadata from a message
* [Feature] Implement FANN threaded learning
* [Feature] Implement classifying for lua classifiers
* [Feature] Implement finish scripts for worker processes
* [Feature] Implement monitoring for DNS resources
* [Feature] Implement real priorities for pre and post filters
* [Feature] Insert two symbols: FANN_HAM and FANN_SPAM instead of one
* [Feature] Module to push metadata/messages to redis pubsub
* [Feature] Monitor RBL records
* [Feature] Move fann_classifier to a separate plugin
* [Feature] Normalize all ANN inputs
* [Feature] Preliminary version of metric exporter module
* [Feature] Preserve decompression context between tasks
* [Feature] Ratelimit: Support dynamic bucket size/leak rate
* [Feature] Relax FORGED_RECIPIENTS: allow senders to BCC themselves
* [Feature] Remove symbols weights on composites processing
* [Feature] Return symbol scores when getting resulting symbols
* [Feature] Rework lua tcp module
* [Feature] Rule to detect some obvious X-PHP-Originating-Script forgeries
* [Feature] Rule to identify some X-PHP-Script forgeries
* [Feature] Rules for scoring Google Message-ID fixes
* [Feature] Send hashes values to reply
* [Feature] Set expire for dmarc reports
* [Feature] Stop using cymru zone as it is unstable
* [Feature] Stop using of GLists for headers, improve performance
* [Feature] Store `for` in task:get_received_headers
* [Feature] Store `for` part in received headers
* [Feature] Store enabled flag for webui session
* [Feature] Store error messages in ring buffer
* [Feature] Support compressed maps
* [Feature] Support excluding selected users from ratelimits
* [Feature] Support looking up NS records in lua_dns
* [Feature] Support modern style SURBL configuration
* [Feature] Support multiple hashes in delhash path
* [Feature] Support new messages in rspamc
* [Feature] Support requests without reads in lua_tcp
* [Feature] Support setting task message from Lua
* [Feature] Track visibility of HTML elements
* [Feature] Try to add CRLF when checking DKIM
* [Feature] Try to guess line endings when folding headers
* [Feature] Try to improve normalization function for bayes
* [Feature] Use FFI to optimize SA module
* [Feature] Use length based arguments for redis, allow lua_text as arg
* [Feature] Use more layers for fann and another normalization
* [Feature] User-defined ratelimits
* [Feature] Utility to convert fuzzy storage from sqlite to redis
* [Feature] Yield DMARC_DNSFAIL on lookup failure
* [Fix] Adopt fuzzy storage for flexible backends
* [Fix] Allow plain IP addresses in Rspamd maps
* [Fix] Another fix for brain-damaged hiredis
* [Fix] Another fix for rdns write errors
* [Fix] Another fix for rdns_make_request_full invocation
* [Fix] Another fix in DKIM canonicalization
* [Fix] Another memory leak plugged
* [Fix] Another try to deal with posix idiotizm
* [Fix] Another try to fix RDNS events processing logic
* [Fix] Avoid double frees in HEAD requests
* [Fix] Avoid extra symbols for RBLs
* [Fix] Banish table.maxn from Lua parts
* [Fix] Check for socket error before connection in lua_tcp
* [Fix] Correctly propagate redis timeouts to Lua
* [Fix] Do not add extra newline in MIME mode
* [Fix] Do not be cheated by system hiredis
* [Fix] Do not classify when a message has not enough tokens
* [Fix] Do not crash on redis errors
* [Fix] Do not distinguish NXDOMAIN and NOREC for monitored
* [Fix] Do not replan retransmits if merely one server is defined
* [Fix] Do not use headers to calculate messages digests
* [Fix] Don't force action in replies module for authenticated users/local networks
* [Fix] Explicitly ban default passwords in webui
* [Fix] Finally fix ambiguity between parsed and resolved spf elts
* [Fix] Fix 'decoded' value in task:get_header_full()
* [Fix] Fix DKIM calculations
* [Fix] Fix DKIM signing for messages with no newline at the end
* [Fix] Fix DNS request in monitored
* [Fix] Fix DNS write errors processing
* [Fix] Fix HTTP methods other than GET and POST
* [Fix] Fix PERMFAIL for v6/v4 ambiguities
* [Fix] Fix absurdic scores for HFILTER_URL_ONLY
* [Fix] Fix actions in rolling history
* [Fix] Fix actrie patterns
* [Fix] Fix applying of lua dynamic confg
* [Fix] Fix autolearning errors and redis cache
* [Fix] Fix bayes learn_condition
* [Fix] Fix build with the recent OpenSSL
* [Fix] Fix caching and compressed maps
* [Fix] Fix check plain text part
* [Fix] Fix crash on OpenBSD in `url_email_start`
* [Fix] Fix double free in SPF
* [Fix] Fix extraction of shingles from redis fuzzy storage
* [Fix] Fix false sharing for symbols in the cache
* [Fix] Fix float usage in util:get_time
* [Fix] Fix folding algorithm to deal with empty tokens
* [Fix] Fix format string
* [Fix] Fix format string usage in controller errors handling
* [Fix] Fix handling of '\0' in lua_tcp
* [Fix] Fix handling of HTTP HEAD methods
* [Fix] Fix hash creation
* [Fix] Fix hiredis stupidity
* [Fix] Fix implicit settings module settingsup
* [Fix] Fix interaction with lua GC to avoid craches
* [Fix] Fix ip_score module registration
* [Fix] Fix issue with empty messages and dkim
* [Fix] Fix issues with CGP helper
* [Fix] Fix issues with the recent SPF changes
* [Fix] Fix key name to load ANN correctly
* [Fix] Fix lua tcp module by saving `do_read` in callback data
* [Fix] Fix memory leak in client when using compression
* [Fix] Fix min_learns option
* [Fix] Fix on_finish scripts and async handlers
* [Fix] Fix options for SPF dnsfail symbol
* [Fix] Fix parsing includes and redirects in SPF
* [Fix] Fix parsing of lua comments with empty lines
* [Fix] Fix parsing of unquoted HTML attributes
* [Fix] Fix periodic events and redis
* [Fix] Fix processing of fuzzy learns from Lua
* [Fix] Fix processing of redirect in SPF includes
* [Fix] Fix processing of symbols when reject limit is reached
* [Fix] Fix refcounts when map is specified by IP
* [Fix] Fix rspamd{session} class in Lua API
* [Fix] Fix setting ratelimit key for 'ip' bucket
* [Fix] Fix some cases of TLD urls detector
* [Fix] Fix statconvert tool
* [Fix] Fix stats for backend-less classifiers
* [Fix] Fix training script for fann_redis
* [Fix] Fix variable in ann module
* [Fix] Fix various errors in lua dynamic conf plugin
* [Fix] Forget old ANN when max_usages is reached to avoid overtrain
* [Fix] Further canonicalization fixes
* [Fix] Further fixes for fann_redis prefixes
* [Fix] Handle failures for inactive pooled connections
* [Fix] Improve multimap info message
* [Fix] More fixes in ANN loading
* [Fix] More fixes to fann_redis
* [Fix] More issues in fann_redis
* [Fix] More spaces fix in DKIM signature
* [Fix] Multiple fixes to asn script, add IPv6 support
* [Fix] Multiple issues in fann_redis
* [Fix] No greylist rejected messages
* [Fix] One more attempt to fix lua_redis
* [Fix] One more check for readdir...
* [Fix] Params should be treated as a hash
* [Fix] Plug memory leak in regexp desctructor
* [Fix] Process headers only once
* [Fix] Properly handle nil values in ratelimit plugin
* [Fix] Really fix redis shingles check
* [Fix] Remove fann with incorrect layers count
* [Fix] Remove mentions of deleted include
* [Fix] Remove some incompatible functions
* [Fix] Settings: fix `authenticated` parameter (#886)
* [Fix] Skip MX check for authenticated users and local networks
* [Fix] Slightly fix ANN routines
* [Fix] Stop caching records with DNS failures
* [Fix] Treat all errors in redis_pool as fatal errors for a connection
* [Fix] Try avoid false-positives in HEADER_FORGED_MDN rule
* [Fix] Try to avoid race condition when using rrd
* [Fix] Try to reload redis scripts if they are missing
* [Fix] Unbreak once_received skipping for local networks
* [Fix] Unlock ANN on error
* [Fix] Use memmove for overlapping regions
* [Fix] Use real size instead of displayed for core limits
* [Fix] Use the correct macro to get the size of control
* [Fix] Various fixes for errors ringbuffer
* [Fix] Yield R_SPF_DNSFAIL if lookup of included record fails
* [Fix] mid: fix map initialization
* [Fix] mid: handle incorrect rgexps in the map
* [Rework] Add extract training data function to fann_redis
* [Rework] Add preliminary train tests
* [Rework] Add redis storage feature to fann_redis
* [Rework] Adopt fuzzy storage for abstract backend
* [Rework] Adopt plugins
* [Rework] First reiteration on fann scores
* [Rework] Implement loading/invalidating
* [Rework] Make lua_redis task agnostic
* [Rework] Make rspamd protocol messages useful
* [Rework] Massive removal of legacy code
* [Rework] More cleanup actions
* [Rework] Remove legacy code never used for classifiers
* [Rework] Remove outdated and unused lua_session module
* [Rework] Reorganize fuzzy backend structure
* [Rework] Reorganize the internal backend structure
* [Rework] Restore old fann_scores, move common parts
* [Rework] Rework and simplify rbl plugin
* [Rework] Rework parsing of DMARC records
1.3.4:
* [Feature] ASN module; support matching ASN/country in multimap
* [Feature] Add SPF method in spf return result
* [Feature] Add Yandex and Mail.ru forwarding rules
* [Feature] Add mempool maps in multimap
* [Feature] Add rule for identifying mail sent by eval()'d PHP code
* [Feature] Add support of stub DNSSEC resolver to rdns
* [Feature] Add task:get_digest method
* [Feature] Allow for more fine-grained scoring for ip_score
* [Feature] Allow to get digest of a mime part from lua
* [Feature] Allow to print message digest in logs
* [Feature] Fold DKIM-Signature header
* [Feature] Implement encrypted logs
* [Feature] Log URLs encrypted if we have log encryption pubkey
* [Feature] Pass authenticated bit to lua
* [Feature] Read redis backend statistics configuration from global section
* [Feature] Show the exact value matched for multima symbols
* [Feature] Store task checksum
* [Fix] Avoid setting limits when required elements are missing
* [Fix] DMARC: Fix alignment checking for subdomains
* [Fix] DMARC: deal with missing and spurious spaces
* [Fix] Defer insertion of results in ip_score to avoid skewing stats
* [Fix] Disable DMARC for local/authorized mail
* [Fix] Fix handling of proxied headers in controller
* [Fix] Fix hex printing of strings
* [Fix] Fix issue with spaces in maps
* [Fix] Fix parsing of forwarded IP
* [Fix] Fix reload in some plugins and workers
* [Fix] Fix reloading on SIGHUP
* [Fix] Fix some border cases for DKIM canonicalization
* [Fix] Fix url maps
* [Fix] Make dnssec configurable option disabled by default for now
* [Fix] rspamadm statconvert: force db to be a string
* [Fix] rspamadm statconvert: use db/password for learn cache
* [Rework] Rework flags in rspamd logger
1.3.3:
* [CritFix] Check hyperscan cache sanity before loading
* [CritFix] Fix setting of fuzzy keys (completely breaks fuzzy storage)
* [Feature] Add SARBL (sarbl.org) uribl
* [Feature] Add `--search-pattern` option to rspamd_stats
* [Feature] Add some sanity check for very long from/to log elements
* [Feature] Allow to create hashes from string in a single step
* [Feature] Fix order of pre and postfilters
* [Feature] Improve lua URLs API
* [Feature] Improve message about fuzzy rules
* [Feature] Pre-calculate blake2 digest for all parts
* [Feature] Print radix duplicate keys as IP addresses
* [Feature] Simple mechanism for disabling RBLs in local.d/rbl.conf
* [Feature] Use faster hash function for fuzzy storage
* [Feature] rspamd_stats: support log directory reading
* [Fix] Add sanity check for url filters
* [Fix] Do not show rmilter section as a fake metric in rspamc
* [Fix] Fix URL filters
* [Fix] Fix a stupid mistake in util.strequal_caseless
* [Fix] Fix blake2b hash of the string "rspamd"
* [Fix] Fix filename maps filter
* [Fix] Fix finding tld in util.get_tld
* [Fix] Fix multimap content filters
* [Fix] Fix returning boolean from Lua
* [Fix] Fix returning of REDIS_NIL
* [Fix] Try to deal with multiple workers terminated
* [Fix] Use forced DNS request when calling for lua_http
* [Rework] Rework multimap filters, add redis maps
1.3.2:
* [Feature] Add a special symbol for SPF DNS errors: R_SPF_DNSFAIL
* [Feature] Add correlations report in fuzzy stats
* [Feature] Add experimental CGP integration
* [Feature] Add method to get urls length in a text part
* [Feature] Add new methods to lua_html to access HTML tags
* [Feature] Allow all types of symbols to be added via __newindex method
* [Feature] Allow to create settings for authenticated users
* [Feature] Allow to get block content for HTML tags
* [Feature] Improve DNS failures when dealing with SPF
* [Feature] Properly implement R_WHITE_ON_WHITE rule
* [Feature] Remove old ugly rules
* [Feature] Rspamc can now add dkim signature in mime mode
* [Feature] Store content length for HTML tags
* [Feature] Support reacher set of HTML colors
* [Feature] Try to avoid FP for low contrast fonts detection
* [Fix] Add missing HTML colors
* [Fix] Add spaces to dkim signature to allow folding
* [Fix] Avoid returning NaN as score on scan
* [Fix] Decode entitles in href parts
* [Fix] Do not cache SPF records with DNS errors
* [Fix] Do not crash on cyclic depends
* [Fix] Do not insert HELO/HOSTNAME unknown when they are not passed
* [Fix] Do not set absent hostname to "unknown"
* [Fix] Do not stress redis with KEYS command (#791)
* [Fix] Fix DMARC_BAD_POLICY symbol
* [Fix] Fix HFILTER_URL module
* [Fix] Fix HFILTER_URL_ONELINE rule
* [Fix] Fix buffering in CGP integration
* [Fix] Fix colors propagation from parent nodes
* [Fix] Fix confusing OpenSSL API usage of i2d_RSAPublicKey
* [Fix] Fix dependencies id sanity check
* [Fix] Fix folding for semicolon separated tokens
* [Fix] Fix largest possible TLD behaviour
* [Fix] Fix last token folding
* [Fix] Fix length calculations in white on white rule
* [Fix] Fix multiple request headers structure
* [Fix] Fix multiple values headers freeing
* [Fix] Fix parsing of background color
* [Fix] Fix printing from field in log_urls
* [Fix] Fix processing of last element of DMARC policies
* [Fix] Further fixes for HTML colors
* [Fix] Further fixes for multiple values headers
* [Fix] Further fixes for white on white rule
* [Fix] Further fixes in HTML tags parsing
* [Fix] Ignore content type/subtype case
* [Fix] Increase score of R_WHITE_ON_WHITE
* [Fix] Parse CGP envelope data
* [Fix] Propagate colors in HTML
* [Fix] Restore multiple values headers in protocol
* [Fix] Restore multiple values in headers processing
* [Fix] Some more changes to tag's content length calculations
* [Fix] Some more fixes for low contrast fonts detector
* [Fix] SpamAssassin plugin: support check_freemail_header('EnvelopeFrom', [..])
* [Fix] Trigger HTML_SHORT_LINK_IMG on any external image
* [Fix] rspamd_stats: remove deprecated defined(@array)
1.3.1:
* [CritFix] Fix catena passwords validation
* [CritFix] Fix crash when the first received is faked
* [Feature] Add DMARC_BAD_POLICY symbol when DMARC policy was invalid
* [Feature] Allow for matching hostnames in multimap (#773)
* [Feature] Allow for setting action based on DMARC disposition
* [Feature] Allow limiting of the inbound message size
* [Feature] Allow maps with multiple symbols and scores
* [Feature] Allow regexps in the emails maps
* [Feature] Allow to register metric symbols from multimap
* [Feature] Allow to reset redis tokens instead of appendig values
* [Feature] Allow to store strings in radix maps
* [Feature] Check UTF validity when there are utf regexps in a map
* [Feature] Correctly work when there is no hard reject action
* [Feature] Implement dependencies for maps
* [Fix] Another effort to unbreak sqlite locking
* [Fix] Avoid crash when closing mmapped file
* [Fix] Do not break history on NaN in required score
* [Fix] Ensure that hyperscan cache written is written properly
* [Fix] Filter NaN from scores in history
* [Fix] Fix DNSBL maps
* [Fix] Fix another locking issue in sqlite
* [Fix] Fix another locking issue with mapped files
* [Fix] Fix deadlock in mmaped file stats
* [Fix] Fix dependencies in multimap plugin
* [Fix] Fix emails module configuration
* [Fix] Fix greylist plugin (#755)
* [Fix] Fix greylisting plugin variable usage
* [Fix] Fix installed permissions for rspamd_stats
* [Fix] Fix locking in mmapped statistics
* [Fix] Fix paths in tests
* [Fix] Fix prefilter mode for multimap
* [Fix] Forgot to commit leftover changes
* [Fix] Really fix local.d includes
* [Fix] Restore selective greylisting behaviour
* [Fix] Set max size on per connection basis
* [Fix] Use temporary storage for hyperscan cache
* [Rework] Remove systemd socket activation
1.3.0:
* [CritFix] Fix SA rawbody processing - exclude top part
* [CritFix] Fix decoding of UTF HTML entitles
* [CritFix] Fix encrypted fuzzy requests
* [CritFix] Fix leak of shared memory fds and files
* [CritFix] Fix levenshtein distance calculations
* [CritFix] Fix mime headers processing
* [CritFix] Fix parsing of URLs in texts
* [CritFix] Fix parsing of missing classes
* [CritFix] Fix redis structure by adding {NULL, NULL} member
* [CritFix] Fix some more URL detector issues
* [CritFix] Fix systemd sockets activation
* [CritFix] Fix unencrypted passwords processing in the controller
* [CritFix] Fix writing CDPs to the database
* [CritFix] Fix writing of encrypted HTTP requests
* [CritFix] Plug memory leak in dkim module
* [CritFix] Plug memory leak in headers getting code
* [CritFix] Pre-filters and post-filters were completely broken
* [CritFix] Properly support SA body regexps
* [CritFix] Really skip filters in case of pre-result set
* [CritFix] Restore the intended pre-filters behaviour
* [Rework] Adopt new maps code
* [Rework] Compile ragel sources when building rspamd
* [Rework] Finish rework for the rest of places that use HTTP
* [Rework] Fix DKIM headers canonicalization
* [Rework] Fix lua maps API
* [Rework] Import linenoise for line editing
* [Rework] Include config structure to all rcl handlers
* [Rework] Make chartable module useful
* [Rework] Move http internal structures to a private header
* [Rework] Partly fix controller
* [Rework] Remove dedicated images list
* [Rework] Rename http proxy to rspamd proxy
* [Rework] Rename mime parts structures
* [Rework] Rework HTTP code
* [Rework] Rework exceptions and newlines processing
* [Rework] Rework pre and postfilters system
* [Rework] Separate method to close backend connections
* [Rework] Start the complete maps rework
* [Rework] Use dynamically generated ragel C sources
* [Feature] Add 'blacklist' and 'strict' modes for whitelists
* [Feature] Add 'symbols_enabled' and 'groups_enabled' to settings
* [Feature] Add ESMTPSA received type
* [Feature] Add a simple script to evaluate rspamd rules in the logs
* [Feature] Add a simple tool to generate DKIM keys
* [Feature] Add a trivial heuristic for codepages
* [Feature] Add and use mumhash for non-crypto hashing
* [Feature] Add better method to check lua userdata types
* [Feature] Add better zip files search algorithm
* [Feature] Add catena PBKDF function
* [Feature] Add configuration knobs for in and out parser scripts
* [Feature] Add content filtering support to multimap
* [Feature] Add different timeouts for proxy connections
* [Feature] Add dot commands for lua REPL:
* [Feature] Add execution of lua global functions script
* [Feature] Add function for pretty printing of inet addresses
* [Feature] Add function to convert fstring_t to c string
* [Feature] Add function to create temporary shared memory mapping
* [Feature] Add function to generate random hex string
* [Feature] Add generic fucnction to parse IP maps
* [Feature] Add initial version of HTTP lua repl
* [Feature] Add learn condition to the default configuration
* [Feature] Add learn conditions for classifiers
* [Feature] Add limit for dkim signatures to be checked
* [Feature] Add locking routines for lua_util
* [Feature] Add lua API for getting info from archives
* [Feature] Add lua utility to decode URL encoding
* [Feature] Add method to copy message from http connection
* [Feature] Add mirrors feature
* [Feature] Add more algorithms for shingles generation
* [Feature] Add more domains to redirectors list
* [Feature] Add more encodingsto cryptobox hash API
* [Feature] Add more file utilities to lua_util
* [Feature] Add more functions to extract data from text parts
* [Feature] Add more methods to get headers from a task
* [Feature] Add more methods to init http message body
* [Feature] Add more options for redis config parsing function
* [Feature] Add new representation of email address
* [Feature] Add new symbols to filter bad extensions in messages
* [Feature] Add new utility methods to mimepart object
* [Feature] Add openphish support to rspamd phishing module
* [Feature] Add parsers for SMTP address in ragel
* [Feature] Add parsing of mirror hosts for fuzzy storage
* [Feature] Add parsing scripts for master connection as well
* [Feature] Add preliminary greylist plugin
* [Feature] Add preliminary phishtank support
* [Feature] Add preliminary rarv5 support
* [Feature] Add preliminary version of ssl toolbox
* [Feature] Add protection against open files limit and accepting sockets
* [Feature] Add rar v4 support
* [Feature] Add reading scripts for master connection
* [Feature] Add replies plugin
* [Feature] Add results parsing code
* [Feature] Add routines to compare and check pubkeys
* [Feature] Add simplier versions of refcounts
* [Feature] Add some time manipulation functions for lua APi
* [Feature] Add support for non-standard BATV signatures
* [Feature] Add support of address with at-domain list
* [Feature] Add support to search archives by magic
* [Feature] Add task:get_rawbody method
* [Feature] Add test to check shared memory support sanity
* [Feature] Add the initial version of LUA repl to rspamadm
* [Feature] Add throughput graph for RRD backend to WebUI
* [Feature] Add universal function to make a proper redis request
* [Feature] Add universal function to parse redis servers for plugins
* [Feature] Add util.unlink function
* [Feature] Add utility function to return random number from 0 to 1
* [Feature] Add utility method to convert ftok to C string
* [Feature] Add utility to map shared memory segments
* [Feature] Add versions to fuzzy storage
* [Feature] Add workaround for legacy clients in rspamd proxy
* [Feature] Add workaround for systems without sane shmem support
* [Feature] Add xoroshiro+ fast rng for non-crypto purposes
* [Feature] Adopt plugins for new maps API
* [Feature] Allow SPF to be checked for empty tasks
* [Feature] Allow binary patterns in lua_trie
* [Feature] Allow catena encrypted passwords in controller
* [Feature] Allow client ip match in the settings
* [Feature] Allow easy adding and overriding of fuzzy rules
* [Feature] Allow empty tasks to be processed
* [Feature] Allow hostnames in IP maps
* [Feature] Allow https maps
* [Feature] Allow multiple PBKDF types in `rspamadm pw`
* [Feature] Allow named fuzzy rules
* [Feature] Allow non zero terminated patterns in multipattern
* [Feature] Allow partial hash updates
* [Feature] Allow pipelining for redis.make_request
* [Feature] Allow sending empty requests using client
* [Feature] Allow setting fuzzy flag by symbol not by value
* [Feature] Allow setting scores and actions from lua
* [Feature] Allow shared memory simple http client
* [Feature] Allow static lua files in any parts of rspamd sources
* [Feature] Allow to change flag from fuzzy learn condition
* [Feature] Allow to check rspamd_text using maps
* [Feature] Allow to disable composite rules from settings
* [Feature] Allow to disable some modules from common redis setup
* [Feature] Allow to extract ucl_object from lua using common API
* [Feature] Allow to get settings and settings id hash from lua_task
* [Feature] Allow to have specific settings for mirrored traffic
* [Feature] Allow to open message from a shared memory segment
* [Feature] Allow to parse pubkeys from the rcl config
* [Feature] Allow to pass extradata from rspamd to rmilter
* [Feature] Allow to query storage about number of fuzzy hashes stored
* [Feature] Allow to read logs without symbols scores
* [Feature] Allow to read password from console for rspamc
* [Feature] Allow to set ciphers and CA paths in config
* [Feature] Allow to skip some initialization phases to speed up rspamadm
* [Feature] Allow underscore separated names in settings
* [Feature] Allow versioning for sqlite databases
* [Feature] Always allow to terminate rspamd
* [Feature] Better deal with backend errors
* [Feature] Better lua_redis logging
* [Feature] Configure CA path and ciphers
* [Feature] Create a dedicated parser to strip newlines
* [Feature] Deduplicate the same urls in multimap module
* [Feature] Distinguish luajit from lua
* [Feature] Do not print garbadge in --compact output
* [Feature] Dynamically detect if a CPU is incompatible with hyperscan
* [Feature] Enable forced resolving for some lua plugins
* [Feature] Enable rrd by default
* [Feature] Enable workaround for exim
* [Feature] Fix task functions to work without rspamd_config
* [Feature] Further improvements to chartable module
* [Feature] Further micro-optimizations for hashing and shingles
* [Feature] Further relax parser
* [Feature] Humanize numbers in stats widgets
* [Feature] Implement HTTPS client
* [Feature] Implement SSL support in http client
* [Feature] Implement body rules for the trie plugin
* [Feature] Implement braced regexp quantifiers
* [Feature] Implement compare scripts for mirrors results
* [Feature] Implement compare scripts setup
* [Feature] Implement composites policies
* [Feature] Implement conditional learning for classifiers
* [Feature] Implement constructing of map from UCL
* [Feature] Implement dkim signing in dkim check plugin
* [Feature] Implement fuzzy storage updates
* [Feature] Implement fuzzy updates push protocol
* [Feature] Implement https maps
* [Feature] Implement inter-process maps caching
* [Feature] Implement mapping of remote flags to local flags
* [Feature] Implement mirroring in rspamd proxy
* [Feature] Implement multi-flags fuzzy replies
* [Feature] Implement multiple-sources fuzzy storage
* [Feature] Implement order of pre/post filters
* [Feature] Implement partial deleting for multi-flags
* [Feature] Implement pipelining for redis async interface
* [Feature] Implement ragel parser for received headers
* [Feature] Implement reading of messages to shared memory
* [Feature] Implement refcount for messages
* [Feature] Implement retransmits for master connection
* [Feature] Implement zero-copy mode for HTTP reading
* [Feature] Improve SPF domain detection logic
* [Feature] Improve config:register_symbol function
* [Feature] Improve error report for type missmatch in lua
* [Feature] Improve fstrings API
* [Feature] Improve getting SMTP data from lua_task
* [Feature] Improve levenshtein distance function
* [Feature] Improve logging in proxy and add refcounts
* [Feature] Improve logging lua types
* [Feature] Improve master/slave logging
* [Feature] Improve phishing plugin
* [Feature] Improve phishtank and openphish support
* [Feature] Improve ragel build target
* [Feature] Improve statistics script
* [Feature] Initialize ssl library to use SSL connections
* [Feature] Interpolate points sent to webui
* [Feature] Limit logging of elements that could have too many items
* [Feature] Lock ANN file when loading
* [Feature] New abstract hashing API in cryptobox
* [Feature] Normalize quoted addresses
* [Feature] Now cryptobox lua API accepts rspamd text as input
* [Feature] Optimize alignment to speed up hashing
* [Feature] Parse received date and ESMTPA proto
* [Feature] Parse received timestamp
* [Feature] Pass settings id to log helper
* [Feature] Pass settings id to lua script from log helper
* [Feature] Perform files expansion on proxying
* [Feature] Preliminary implementation of fuzzy master/slave updates
* [Feature] Print userdata using tostring if possible
* [Feature] Propagate HTTP result string when using proxy
* [Feature] Properly implement unweighted round-robin algorithm
* [Feature] Reduce number of timers queries
* [Feature] Rework and improve fuzzy storage
* [Feature] Rework dns resolving API for lua, add 'forced' option
* [Feature] Rework fann module to understand settings
* [Feature] Rework listening system to allow multiple socket types per worker
* [Feature] Rework ratelimit module to set expiration
* [Feature] Save bayes probability in memory pool var
* [Feature] Save settings id hash for convenience
* [Feature] Search for SSL_set_tlsext_host_name support
* [Feature] Send DKIM signature to protocol reply
* [Feature] Show DKIM signature in rspamc client
* [Feature] Show symbols description in scan output
* [Feature] Sign message merely after DKIM check
* [Feature] Simplify machines by assuming that headers are unfolded
* [Feature] Sort symbols when displaying them in log
* [Feature] Split main connection from mirrored connections
* [Feature] Start moving to the new email address structure
* [Feature] Store HTTP headers in a hash table
* [Feature] Store more information about compressed files
* [Feature] Store raw headers value to use them in DKIM
* [Feature] Store text parts content with newlines stripped
* [Feature] Support DKIM signing
* [Feature] Support EXIF jpeg images
* [Feature] Support archive files list extraction
* [Feature] Support archives when matching patterns in multimap
* [Feature] Support premium/academic feed for openphish
* [Feature] Support rspamd_updates via https
* [Feature] Supprort FQDNs in phishing module maps
* [Feature] Try to read on fuzzy timeout to avoid fake timeouts
* [Feature] Try to select the optimal possible function for input
* [Feature] Unescape and unquote smtp addresses
* [Feature] Update fuzzy timestamp when adding value
* [Feature] Update mumhash
* [Feature] Use -flto if possible when optimizations are enabled
* [Feature] Use extended map types in lua map, unify code
* [Feature] Use file lock in logger to avoid deadlocks
* [Feature] Use generic global string split function
* [Feature] Use metrohash as well
* [Feature] Use mumhash by default for incremental hashing
* [Feature] Use mumhash for words hashing
* [Feature] Use new ip parsing API
* [Feature] Use new maps API for local addrs
* [Feature] Use new ragel parser in message parsing code
* [Feature] Use new received parser instead of old one
* [Feature] Use new redis API in DMARC plugin
* [Feature] Use new redis API in greylist plugin
* [Feature] Use new redis API in ip_score plugin
* [Feature] Use new redis API in ratelimit plugin
* [Feature] Use new redis API in replies plugin
* [Feature] Use new version of register_symbol in rspamd plugins
* [Feature] Use offset when passing shm to deal with encrypted requests
* [Feature] Use one pass to remove newlines and store their positions
* [Feature] Use rspamd specific type checks for userdata
* [Feature] Use shared memory storage for http maps
* [Feature] Use universal redis definitions in rspamd plugins
* [Feature] Various improvements in greylist module
* [Feature] Wait for sqlite if locked when switching to WAL mode
* [Fix] Add filenames sanity filtering for mime types
* [Fix] Add guards for empty parts
* [Fix] Add missing types
* [Fix] Add more priority for config file symbols registered from UCL
* [Fix] Add sanity checks when compiling regexp maps
* [Fix] Add spaces instead of newlines to the normalized content
* [Fix] Add workaround for ancient openssl
* [Fix] Add workaround for gmime CTE stupidity
* [Fix] Add workaround for hex digits
* [Fix] Adjust MISSING_MIMEOLE score
* [Fix] Adjust body/body_buf when stealing encrypted message
* [Fix] Adopt lua task API for the new email addresses structure
* [Fix] Allow for disabling DMARC reporting when Redis is configured
* [Fix] Always register openphish and phishtank virtual symbols
* [Fix] Always use shmem on linux
* [Fix] Another change of newlines policy
* [Fix] Another d3evolution update
* [Fix] Another fix for exim workaround
* [Fix] Another fix for legacy clients
* [Fix] Another fix for maps scheduling
* [Fix] Another fix for marking upstreams inactive
* [Fix] Another fix for postfilters
* [Fix] Another fix for redis timeouts
* [Fix] Avoid `table.getn` method as it has been removed in lua 5.3
* [Fix] Avoid double hashing for images
* [Fix] Avoid linking with actrie if hyperscan is enabled
* [Fix] Check copy result when sending message to mirrors
* [Fix] Cleanup message when assiging body
* [Fix] Cleanup stack from global vars
* [Fix] Correctly parse query type
* [Fix] Disable all symbols if enable_groups is found in settings
* [Fix] Disable fts as it is completely broken in bloody linux
* [Fix] Disable multiple autolearn checks
* [Fix] Disallow updates by default
* [Fix] Do not abort when cannot load a map
* [Fix] Do not check recursion for non-DNS SPF record types
* [Fix] Do not delete uninitialized events
* [Fix] Do not die if shmem_mkstemp fails
* [Fix] Do not die when no metrics defined
* [Fix] Do not even try pcre in case of regexp maps
* [Fix] Do not greylist messages if redis has failed somehow
* [Fix] Do not greylist on rejection
* [Fix] Do not leave temporary maps cached
* [Fix] Do not output meaningless errors
* [Fix] Do not send NaN in json
* [Fix] Don't mix hyperscan and pcre processing within a same task
* [Fix] Finally rework and simplify redis backend for statistics
* [Fix] Fix Exim shutdown patch
* [Fix] Fix JIT compilation for PCRE2 expressions
* [Fix] Fix JIT usage for PCRE2
* [Fix] Fix REPL output
* [Fix] Fix SMTP address parsing machine
* [Fix] Fix UTF8 mode in PCRE2
* [Fix] Fix a stupid misprint in word 'phishing'
* [Fix] Fix accepting the first update when local idx is -1
* [Fix] Fix adding maps from ucl
* [Fix] Fix adding upstream to an active queue
* [Fix] Fix and rescore R_PARTS_DIFFER logic
* [Fix] Fix body rules in SA plugin
* [Fix] Fix body start position
* [Fix] Fix border case in urls detector
* [Fix] Fix border cases for incremental hashing
* [Fix] Fix caseless uthash application
* [Fix] Fix chartable issue with starting digits
* [Fix] Fix client_ip in users settings
* [Fix] Fix compilation issue
* [Fix] Fix conditional learning
* [Fix] Fix crash on empty maps
* [Fix] Fix creating of URLs from LUA
* [Fix] Fix creating of temporary shmem segment
* [Fix] Fix creation of mmapped statfiles
* [Fix] Fix descriptors leak on shmem detaching
* [Fix] Fix detaching of shared memory segments
* [Fix] Fix detection of URLs in text parts
* [Fix] Fix directories processing for rspamc
* [Fix] Fix displaying of rewrite subject in WebUI
* [Fix] Fix dkim private keys operations
* [Fix] Fix dkim signing
* [Fix] Fix dynamic scoring of subject symbols
* [Fix] Fix email address build
* [Fix] Fix encrypted proxy requests
* [Fix] Fix errors counting in upstreams
* [Fix] Fix errors handling in the proxy
* [Fix] Fix event bases for IO events
* [Fix] Fix events handling when scheduling map wacth
* [Fix] Fix fann rewrite
* [Fix] Fix files fallback for shmem transfer
* [Fix] Fix fuzzy adding in webui
* [Fix] Fix fuzzy storage encrypted mirroring
* [Fix] Fix fuzzy storage sync replies
* [Fix] Fix handling of the same words
* [Fix] Fix inserting values to the sources list
* [Fix] Fix ipv6 mask application
* [Fix] Fix issue with missing recipients
* [Fix] Fix issues with multiple returns from lua
* [Fix] Fix learning for non-existent backend
* [Fix] Fix legacy clients support in proxy
* [Fix] Fix length calculations for shared memory segments
* [Fix] Fix listening on UDP sockets
* [Fix] Fix loading of file maps
* [Fix] Fix long regexp flags (e.g. {sa_body})
* [Fix] Fix lua compare function init
* [Fix] Fix maps descriptions
* [Fix] Fix maps locking
* [Fix] Fix max_train setup in ANN module
* [Fix] Fix memory corruption
* [Fix] Fix memory leak in unsigned maps reading
* [Fix] Fix misprints for lto usage
* [Fix] Fix more issues with scripts processing
* [Fix] Fix next-to-last extension length check
* [Fix] Fix openssl initialization
* [Fix] Fix order of arguments
* [Fix] Fix order of initialization
* [Fix] Fix parser
* [Fix] Fix parsing of binary tries
* [Fix] Fix parsing of braced IPv6 addresses
* [Fix] Fix parsing of nested braces in SMTP comments
* [Fix] Fix parsing of rarv5 archives
* [Fix] Fix periodic scheduling when a map is not modified
* [Fix] Fix possible FP in TRACKER_ID rule
* [Fix] Fix post-filters processing
* [Fix] Fix potential NULL dereference
* [Fix] Fix processing of <br> and <hr> tags
* [Fix] Fix processing of addresses in protocol
* [Fix] Fix processing of messages without received headers
* [Fix] Fix proxying issue for unconnected sessions
* [Fix] Fix proxying of the encrypted messages
* [Fix] Fix race condition with shared memory by refcounts
* [Fix] Fix ratelimit initialization
* [Fix] Fix redis set request in replies plugin
* [Fix] Fix redis timeout events handling
* [Fix] Fix redis timeouts processing logic
* [Fix] Fix refcounts in lua_redis
* [Fix] Fix results checking if no master connection is active
* [Fix] Fix return value for couple of lua functions
* [Fix] Fix round-robin selection when upstreams have no weight
* [Fix] Fix rows calculation in graph
* [Fix] Fix rspamd_redis_make_request syntax in replies plugin
* [Fix] Fix scheduling of locked map events
* [Fix] Fix scores detection
* [Fix] Fix searching for newline positions
* [Fix] Fix secure_ip setting in controller
* [Fix] Fix sending data to graph command
* [Fix] Fix setting of score for parts differ
* [Fix] Fix setting of the lua top
* [Fix] Fix setting path for lua
* [Fix] Fix setting path for phishtank
* [Fix] Fix settings application
* [Fix] Fix shm_open call as described in POSIX
* [Fix] Fix size of length in fuzzy mirror wire protocol
* [Fix] Fix smtp grammar issues
* [Fix] Fix some issues with redis API
* [Fix] Fix some issues with retries in the proxy
* [Fix] Fix stack growing
* [Fix] Fix start of body detection in DKIM
* [Fix] Fix state on timeout
* [Fix] Fix stats script
* [Fix] Fix substring search when there are '\0' in strings
* [Fix] Fix symbol name for spf soft fail
* [Fix] Fix symbol type's check
* [Fix] Fix symbols registration and execution
* [Fix] Fix the case of multiple values keys
* [Fix] Fix the default symbol names according to metric
* [Fix] Fix timeout setup on learning
* [Fix] Fix timeouts in redis cache processing
* [Fix] Fix timeouts processing in lua_redis
* [Fix] Fix upstreams interaction for rspamd proxy
* [Fix] Fix usage of rdns reply structure
* [Fix] Fix varargs loop
* [Fix] Fix whitelists and blacklists in SA rules
* [Fix] Fix write servers setup for redis
* [Fix] Fix writing of HTTP messages
* [Fix] Force rspamd to upgrade fuzzy db on opening
* [Fix] Free the correct pointer
* [Fix] Further fixes for lto and static linking
* [Fix] Further fixes for surbl extensions map
* [Fix] Further fixes in maps code
* [Fix] Further improvements to error messages in fuzzy check
* [Fix] Further tweaks to redis garbadge collection
* [Fix] Groups are now case insensitive
* [Fix] Handle log pipe read errors
* [Fix] Handle nested dependencies in SpamAssassin plugin
* [Fix] Implement new automata to skip empty lines for dkim signing
* [Fix] Improve error messages on fuzzy add
* [Fix] Improve lua redis handling
* [Fix] Improve phishing module logging
* [Fix] Improve printing of fuzzy errors
* [Fix] Improve rrd diagnostic errors
* [Fix] Improve strcase hash used in uthash
* [Fix] Include fuzzy key to distinguish storages with different keys
* [Fix] Include slave cluster name into http request
* [Fix] Include some more information about archives
* [Fix] Indicate upstream error on timeout
* [Fix] Initialize hash tables array to avoid crashes
* [Fix] Initialize parser scripts properly
* [Fix] Initialize vars to avoid warnings
* [Fix] Inverse logic for saving ANN
* [Fix] Link lpeg to rspamd lua library
* [Fix] Make extension checks case-insensitive
* [Fix] Mark expired hashes as not found and not as zero flag
* [Fix] Match archive name as well
* [Fix] More and more fixes to redis states
* [Fix] More fixes about shared memory in proxy
* [Fix] More fixes for redis refcounts
* [Fix] More fixes to end of headers detection
* [Fix] More fixes to events logic
* [Fix] More fixes to multi-flag fuzzy storage
* [Fix] More fixes to parts distance calculations
* [Fix] More guards for redis free
* [Fix] One more fix in redis destructor
* [Fix] One more try to fix redis
* [Fix] PIE is required for static build
* [Fix] Partial fix for mmap'd statistics tests
* [Fix] Plug memory leak in proxy
* [Fix] Properly detect end of headers position
* [Fix] Properly init and free session structures
* [Fix] Reduce PRECEDENCE_BULK rule weight
* [Fix] Reduce the default thresholds for learning
* [Fix] Remove Type=forking from systemd unit file (#709)
* [Fix] Remove bad FANN file to save computational resources
* [Fix] Remove event before closing of fd to avoid race conditions
* [Fix] Remove parsing of 'from' variable in redis backend
* [Fix] Remove some bad domains from whitelists
* [Fix] Repair optional dependencies
* [Fix] Reset master connection when retransmitting scan request
* [Fix] Restore ONCE_RECEIVED symbol
* [Fix] Restore compatibility with old lua API behaviour
* [Fix] Restore redis runtime state
* [Fix] Reverse options when received
* [Fix] Send updates to mirrors only if we have some changes
* [Fix] Set host attribute properly when making HTTP request from lua
* [Fix] Set terminated state before calling of async free
* [Fix] Simplify MISSING_MIMEOLE rule
* [Fix] Simplify state machine by ignoring multiple spaces
* [Fix] Skip setting RPATH for static builds
* [Fix] Slightly reduce weights of rules with high FP rate
* [Fix] Some fixes to libmagic initialization
* [Fix] Some more fixes to ratelimit plugin
* [Fix] Strip '\r\n' properly
* [Fix] Switch hashes to mumhash
* [Fix] Treat NaN values properly in graph command
* [Fix] Try to avoid FP when checking for phished URLs
* [Fix] Try to avoid recursive events deletions
* [Fix] Try to fix false positive URL detections in text parts
* [Fix] Try to fix issue in redis stats backend when task is closed
* [Fix] Try to fix proxying of stupid spamc protocol to HTTP mirrors
* [Fix] Try to fix redis crashes
* [Fix] Try to fix upstreams with one element
* [Fix] Try to handle multiline history in a more sane way
* [Fix] Unbreak build on gcc < 4.9
* [Fix] Update RPM spec/sources (#700)
* [Fix] Update d3evolution version
* [Fix] Update mumhash implementation
* [Fix] Use custom error function for pre and post filters
* [Fix] Use new postfilters and prefilters API in the plugins
* [Fix] Use non-blocking mode for systemd sockets
* [Fix] Use shared memory merely for local backends in the proxy
* [Fix] Use watchers for spf plugin
* [Fix] Varioud fixes to the maps code
1.2.8:
* Another fix for exim workaround (#637)
* Fix unencrypted passwords processing in the controller
* Fix setting path for lua (#652)
* Fix usage of rdns reply structure (#654)
* Use file lock in logger to avoid deadlocks
* Add `application/octet-stream` mime type for `pdf` extension (by @moisseev)
* Implement new automata to skip empty lines for dkim signing (#651)
* Fix parsing of missing classes
* Clarify some rspamc arguments (by @fatalbanana)
* Correct suppress spelling
1.2.7:
* Slightly reduce weights of rules with high FP rate
* Add workround for rspamd-1.3
* Fix possible FP in TRACKER_ID rule
* Simplify MISSING_MIMEOLE rule
* Add workaround for gmime CTE stupidity
* Fix mime headers processing
* Fix false positive URL detections in text parts
* Fix Exim shutdown patch
* Enable workaround for exim mailbox format
* Backport shingles static test
* Fix levenshtein distance calculations
* Fix max_train setup in ANN module
* Fix redis structure by adding {NULL, NULL} member
* Fix build with unmodified LibreSSL opensslv.h
* Repair optional dependencies
* Really skip filters in case of pre-result set
* Restore the intended pre-filters behaviour
* Fix ipv6 mask application
1.2.6:
* Fix parsing of URLs in texts
* Fix creating of URLs from LUA
* Fix some more URL detector issues
* Fix unit tests
* Fix JIT compilation for PCRE2 expressions
* Fix JIT usage for PCRE2
* Fix UTF8 mode in PCRE2
* Add workaround for pre-historic compilers (#605)
* Fix and rescore R_PARTS_DIFFER logic
* Properly set lua paths for tests
* Fix SA rawbody processing - exclude top part
* Store text parts content with newlines stripped
* Properly support SA body regexps
* Fix body rules in SA plugin
* Fix setting of score for parts differ
* More fixes to parts distance calculations
- Use hashed words instead of full words for speed
- Improve levenstein distance calculations and penalise replaces
- Always return number from 0 to 1
- Use g_malloc instead of alloca
* Fix percents output in R_PARTS_DIFFER
* Plug memory leak in dkim module
* Plug minor memory leak in regexps creation
1.2.5:
* Plug an important memory leak in headers getting code
* Remove some bad domains from whitelists
1.2.4:
* Implement new multipattern matcher that uses hyperscan if possible
* Use mutlipattern for lua_trie code
* Add utility methods for multipattern
* Use multipattern in url matcher
* Add escape functions for hyperscan
* Allow to optimize lua -> C transition by flattening table args
* Optimize hot paths in SA plugin
* Optimize rspamd_re_cache_type_from_string
* Allow empty tries
* Fix extraction of URLs from Subject
* Allow to have different flags for different patterns in multipattern
* Add common directory for hyperscan cache to config
* Implement caching for hyperscan multipattern
* Attach domain part to `R_SUSPICIOUS_URL` (by @moisseev)
* Allow multipattern scans to be nested for the case of hyperscan
* Simplify SURBL redirector search code and avoid ac_trie
* Add two way substring search algorithm
* Avoid acism usage to find gtube pattern
* Fix processing of empty headers
* Allow to disable pthread mutexes on broken platforms
* Make web interface not send password in query strings (#585) by @fatalbanana
* Add maximum delay to ratelimit module
* Backport fix for empty files inclusion from libucl
* Fix settings id setup
* Add min_learns option to classifiers
* Use more clever to utf8 conversion strategy
* Fix disabling of virtual symbols in the settings
* Rework settings to work properly in metric-less configuration
* Set the default limit for classifier
* Fix ttl based expiration from LRU cache
* Rework DKIM module to use OpenSSL for digests
* Fix mailto urls parsing with hyperscan
* Do not set obscured flag for urls starting with spaces
* Fix crash on redis learn
* Fix ratelimit ctime setting
1.2.3:
* New DCC module (by @smfreegard)
* Rework whitelist module:
- Now we check different elements for different checks
- MIME from for DMARC
- DKIM signature domain for DKIM
- SMTP from or HELO for SPF
* Fix regexps results combination (*critical*)
* Fix issue with expressions processing (*critical*)
* Optimize strlcpy for aligned input
* Add support of half-closed connection in lua_tcp
* Allow to print compact json in client
* Save required score in history (#581)
* Allow to attach file descriptors to control commands
* Allow to send descriptors from workers to main
* Allow to attach fd when broadcasting to workers
* Implement log pipe feature for rspamd logs analysis
* Add `log_helper` worker
* Add `URIBL_SBL_CSS` (by @smfreegard)
* Add worker scripts functionality
* Add on load hooks for rspamd_config
* Add lua scripts for log_helper worker
* Add generic maillist detector (#584)
* Implement FANN autolearn using log_helper worker
* Rework metrics configuration to allow includes
* Change default value of forced removal in composite rules
* Allow to use assembly version of blake2b on x86 cpu
* Use less precise (but faster) clock if possible
* Insert redirected URL to the urls list
* Allow to get and set callback data for rspamd symbols
* Add binary heap implementation
* Use binary heap for expire algorithms in the hash
* Use `least frequent used` expiration strategy
* Allow to get mime headers from a task
* Add support for mime headers in `regexp` module
* Update Exim patches (by @fatalbanana)
* Allow building rspamd with jemalloc
* Save multipart boundaries
* SA plugin changes:
- Properly handle MIME headers
- Fix eval:check_for_missing_to_header rule
- Implement SA compatible body regexps
- Use sabody rules in SA plugin
* LUA API changes:
- Add util.get_ticks function
- Add util.stat function
- Add task:get_symbols_numeric method
- Add method to get number of symbols in the cache
- Add lua methods to get redirected urls
- Allow to get callbacks for lua symbols
- Add config:set_symbol_callback function
1.2.2:
* Use HTTP Content-Type on non mime input if possible
* Save log level when compressing log messages
* Further rework of composite rules (add '^' prefix)
* Add tracking for rspamd expressions
* Store actions limits in metric result
* Fix parsing of include/redirect with many records in SPF
* Add method to disable symbols execution in the cache
* Allow to disable checks from settings
* Allow to select settings by id in HTTP query
* Find URLs with '\r' and '\n' inside href attribute
* Implement vectored mode for hyperscan (experimental)
* Improve client connection errors diagnostics
* Allow to edit new files with signtool
* Improve hashes performance on 32 bit platforms
* Fix sorting of limits
* Remove slow and unused rules `INVALID_EXIM_RECEIVED*`
* Add expression:process_traced lua method
* Allow tables in task:insert_result
* Save trace for SA metas
* Do not parse broken TLD parts in URLs
* Investigate many border cases in URLs parser
1.2.1:
* Add list support to `mime types` module configuration (by @moisseev)
* Allow symbols params to be printed in logs
* Fix `MIME_BAD_ATTACHMENT` false positives for MDN/DSN
* Fix crashes on arm32
* Do not classify message if some class is missing
* Fix cryptobox cleanup
* Remove multipart/report from bad mime types (#569)
* Improve logging for fuzzy hashes
* Show map URLs in webui
* Sort symbols in webui
1.2.0:
* New dynamic updates plugin
* Regular expressions map support
* Faster radix trie algorithm
* Faster siphash for AVX2 supporing CPUs (used in fuzzy hashes)
* PCRE2 support
* Allow quoted and slashed keys in map
* Add proper support of DNS resolvers balancing (#552)
* Rework includes and configuration system for better local changes support
* New keypairs framework for signing and encryption
* Added support for dynamic modules and workers
* Allow to dump configuration with help comments
* Rework once_received module
- Fix priority for `good_hosts`
- If a good host has been found do not add once_received symbols
- Fix priorities for strict once_received
- Add ability to whitelist IP addresses
* Implement support of signed maps for HTTP and file maps
* Add command to sync fuzzy storage (#533)
* Rework system of symbols and actions registration
It is possible now to use priorities when adding symbols to metrics and
override scores for symbols with lower priority with the scores with
high priority.
* Add auth support and db selection for redis stats
* Improve composite rules application
* Add ignore_received option
* Fix critical issue with inconsistent resorting
* Fix `all` in spf redirects
* Add punycoded versions for IDN domains (#554)
* Improve sorting order for symbols cache
* Add lockless logging for processes management
* Allow to specify flags for metric symbols
* Load images height and width from style attribute (#538)
* Override DNS requests limits for SPF and DKIM
* Fix resetting symbols to their default values in WebUI
* Improve configuration agility for redis stats
* Allow to set db and password for redis in stat_convert
* Import the latest libucl
* LUA API changes:
- Add rspamd_version function to LUA API
- Add lua_cryptobox module
- Add lua_map module
- Add task:set_metric_action lua API method
- Fix race condition in lua_tcp module
- Fix a lot of issues in lua_redis module
- Rework and abstract lua maps API
- Add util.strlen_utf8 lua function
- Add lua functions for caseless comparison
- Allow optional symbols registration
- Add config:add_map table form method, add regexp maps
- Add task:has_urls method
- Add task:has_flag method
- Add html tags methods to lua_html
- Add task:get_dns_req
* Plugins changes:
- Add support for WLBLEval SA plugin
- Use caseless comparison in SA and DMARC plugins
- Allow SA plugin to set scores for rspamd symbols
- Add regexp maps support to multimap
- Allow filenames match in multimap
- Add more filters for the existing map types
- Fix html images rules to reduce FP rates
* New rules:
- LONG_SUBJ - too long subject
- MIME_BAD_ATTACHMENT - bad attachement type
- RDNS_NONE - no reverse DNS record for sender's IP
- Fix MISSING_MIMEOLE rule for modern OE
* Many other bugfixes, memory leaks plugs thanks to:
- Coverity scan
- New gcc-6 warnings
- valgrind manual iterations
* Documentation improvements:
- FAQ list: https://rspamd.com/doc/faq.html
- Reworked quick start guide
- Added documentation for all active modules
* Other changes:
- Dropped Ubuntu Vivid support
- Added Ubuntu Xenial support
- Rework build system for rspamd and rmilter
1.1.4:
* Print traceback on lua errors in lua config
* Fix leaks in lua error paths
* Improve 'R_EMPTY_IMAGE' rule
* Fix metas memoization in SA plugin
* Properly set `flag` in fuzzy replies
* Fix arguments order
* Fix issue with out-of-boundary reading
* Fix issues found by coverity
* Same result checking error found by coverity
* Fix varargs processing (found by coverity)
* Fix error in printing hex
* Reduce weights for some hfilter patterns
* Add aliases for task:get_from_ip:
- task:get_addr
- task:get_from_addr
- task:get_ip
* Rework once_received module
- Fix priority for `good_hosts`
- If a good host has been found do not add once_received symbols
- Fix priorities for strict once_received
- Add ability to whitelist IP addresses
* Fix `MISSING_MIMEOLE` rule for modern OE
* Treat meta tags as embedded tags (#501)
1.1.3:
* Fix DSN rules when SMTP from is unavailable
* Fix statconvert routine to avoid lua module usage
* Set a sane quark for configtest to avoid NULL to be printed in logs
* Support c11 if available
* Fix parsing of ip:port strings
* Add more diagnostic for lua subr errors
* Fix task:set_from_ip lua method
* Add basic routines for digital signatures
* Add tool for digital signatures
* Add plain open file API method for atomic open
* Fix parsing nested braces inside logger vars
* Pre filters now actually skip processing
* Add pre-filter mode for multimap
* Switch to apache 2 license
1.1.2:
* Fix stat_cache closing
* Add checkpoints to sqlite3 learn cache
* Do not recompile lua generated headers all the time
* Increase number of messages learned
* Fix issues with dual stack and hfilter
* Disable MID checks for hfilter by default
* Fix cache definitions in multiple classifier and no type
* Don't crash if learn cache failed to initialize
* Fix googlegroups support in maillist plugin
* Rework flags LUA API:
- Allow to check for a specific flag
- Add `learn_spam`, `learn_ham` and `broken_headers` flags
- Unify internal functions
* Add `BROKEN_HEADERS` rule
* Add support for forged confirmation headers (by @AdUser)
* Allow `any`, `mime` and `smtp` for get_from/get_recipients
* Add mime types checking plugin
* Add rule to detect spammers attempts to cheat mime parsing
* Rework parsing of IP addresses in configuration (better IPv6 support)
* Add `util.parse_mail_address` function to LUA API
* Add lua sqlite3 module
* Implement synchronous redis call
* Ratelimit: avoid possible indexing of nil value (Fixes #498) (by @fatalbanana)
* Add stat_convert command to convert stats tokens from sqlite3 to redis
* Implement redis advanced lua api with pipelining
* Fix memory leak on redis stat (#500)
* Fix user/language learn count in sqlite statistics (#496) (by @fatalbanana)
* Fix build with custom pcre
* Fix fuzzy relearning (#498)
* Improve planning of asynchronous tasks
* Show slow rules in log
* Add warning for slow regexps
* Add base32 decode/encode routines to lua util
* Allow converting of learn cache from sqlite to redis
* Add methods to check if a messages has from/rcpts
* Improve and fix multimap plugin:
- Restore 'header' maps
- Add filters for headers
- Add 'email:addr', 'email:user', 'email:domain' and 'email:name' filters
- Add generic regexp filters
* Disable reload command in rc scripts
* Improve runtime CPU dispatcher for libcryptobox
* Add preliminary support of digital signatures via ed25519
* Add detection for RDRAND support
* Print configuration of crypto on start
* A in SPF presumes AAAA lookup as well
1.1.1:
* Fix duplicated XBL symbol
* Reduce log severity for ratelimit missing servers
* Fix XBL composite to avoid duplicate symbols
* Reduce weight of URL_ONLY rule due to FP rate
* Disable fuzzy hashes from the metadata for now
* Fix processing of empty messages (#486)
* Always treat DNS timeouts as temporary fail for SPF
* Fix issue with SPF double IP stack (#483)
* Use X-Forwarded-For when checking secure_ip (#488)
* Fix hash calculation for sqlite stats
* Fix memory corruption on punycode
* Fix strings allocation in punycode
* Fix error message (#491)
1.1.0:
* Incompatible change: sqlite3 and per_user behaviour:
Now both redis and sqlite3 follows the common principles for per-user
statistics:
1) If per-user statistics is enabled check per-user tokens ONLY
2) If per-user statistics is not enabled then check common tokens ONLY
If you need old behaviour, then you'd need to use separate classifier
for per-user statistics.
* Implement redis statistics backend and cache
* Implement autolearning for statistics
* Reworked statistics architecture from scratch
* Add hyperscan (https://github.com/01org/hyperscan) engine for regular
expressions:
- add lazy loader for hyperscan databases
- rework regexp cache to have joint pcre/hyperscan scanning
- implement hyperscan pre-filter support
- add compilation guards for bad expressions
- implement `rspamadm control recompile` command
- implement hyperscan cache monitoring
- slides: <https://highsecure.ru/rspamd-hyperscan.pdf>
* Implement flexible task logging
* Rework fuzzy worker:
- it is now possible to run multiple fuzzy workers;
- implement lazy writing as sqlite3 is bad at concurrent writing;
- add retries for simple sql commands in fuzzy backend;
- use fine-grained transactions for fuzzy;
- implement new multi-pubkeys mode;
- allow encrypted only storages;
- rework statistics for fuzzy;
- add `rspamadm control fuzzystat` command for extended statistics;
- implement human readable output for the previous command;
- add condition script for learning fuzzy storage;
* Various fixes to SPF:
- fix `redirect` records;
- fix domains when parsing mx/ptr/a records in includes/redirects;
- fix issues with multiple addresses in SPF records;
- ignore SPF results in case of DNS failure;
- adjust TTL of records when resolving subelements of SPF records;
- always select `v=spf1` line if it is available
- do not cache records with DNS failure in subrequests;
- ignore records with temporary fails during subrequests resolving;
- fix `RDNS_RC_NOREC` support;
* Add clang plugin for static analysis:
- implement static checks for `rspamd_printf` format strings;
* Add 'allow_raw_input' option for non-mime messages
* Recognize types using libmagic
* Fix parsing of IPv6 received headers
* Add new interface of communication between workers in rspamd
* Add support for named socketpairs
* Don't write URLs by default as it is too verbose
* Set status for HTTP replies
* Try load `rspamd.conf.override`
* Implement words decaying for text parts to limit many checks
* Improve support of SA rules and plugins:
- add check_for_shifted_date and check_for_missing_to_header eval rules;
- add 'check_relays_unparseable' support;
- add `check_for_mime('mime_attachement')` function;
- use new re_cache interface for all SA rules;
- add support for `Mail::SpamAssassin::Plugin::MIMEHeader`;
- add support of 'special' SA headers to `exists` function;
- fix issue when SA metas contain other metas;
- fix freemail rules;
* Many fixes to the URL parser
* Match any newline character in regexps
* Fix resolving of upstreams and detection of poor IPv6 configurations
* Parse upstreams selection algorithm from the configuration line
* Add `reresolve` command to the control interface
* Generate fuzzy hashes from task metadata (URLs and headers)
* Add method to check if IP is local and `local_addrs` option
* Implement forced timeout for delayed filters
* Disable fast path of pcre-jit as it seems to be broken
* Bayes fixes:
- new normalizer function;
- really use weights of tokens from the OSB algorithm;
- restore multiple classifiers support;
* Rules changes:
- add `R_SUSPICIOUS_URL` rule that detects obfuscated URL's;
- improve empty image rule;
- rework `FORGED_RECIPIENTS` rule;
- reduce weight of `SUSPICIOUS_RECIPS`;
- fix `*_NORESOLVE_MX` symbols in hfilter;
- add `SUBJ_ALL_CAPS` rule with support of UTF8
- add spamhaus SBL to uribl
- fix `SUSPICIOUS_RECIPS` and `SORTED_RECIPS` rules
- remove `R_TO_SEEMS_AUTO` as it generates a lot of FP;
- add new Message-ID regexp for Thunderbird (by @moisseev);
* Plugins changes:
- allow ratelimit plugin to set symbol instead of pre-result
- support IP DNS black lists for URIBL (e.g spamhaus SBL);
- drop deprecated SURBL bits (by @fatalbanana)
- rename `JP_SURBL_MULTI` to `ABUSE_SURBL` (by @fatalbanana)
- add `SURBL_BLOCKED` (by @fatalbanana)
- add `CR_SURBL`
- SURBL: allow fallthrough to default symbol (by @fatalbanana)
- Settings: fix IP match (by @fatalbanana)
- SURBL: add missing symbols to metric (by @fatalbanana)
- allow processing images urls for SURBL
- unconditionally disable SPF for authenticated users and local networks
* Rework ratelimit plugin
- switch to `rates` instead of old and stupid strings to setup;
- check if a bucket is zero and disable the corresponding limits'
- turn off all buckets by default;
- check either `rcpt` or `user` buckets, not all together'
- document new `rates` and `symbol` options;
- inform user about what buckets are used in the configuration;
* Add neural network **experimental** plugin
* Add a sample script to learn neural network from rspamd logs
* Add documentation strings support to rspamd:
- add strings for the main configuration options;
- document workers options;
- add internal plugin options;
- create `rspamadm confighelp` routine;
- implement human readable output for the previous command;
- add subtree search support;
- add keyword search support;
* Documentation improvements, tutorials section, statistics description
* Many other minor and major bugfixes not noted here
1.0.11:
* Fix spf redirects
* Fix domains when parsing mx/ptr/a records in includes/redirects
* Fix unfolded base64 encoding
* Fix GError use-after-free
* Do not rewrite the original url when using redirector
* Fix parsing of fragment in urls
* Fix processing of HTML tags
* Improve empty image rule
* Avoid long double type
* Fix tokens weights in OSB algorithm
* Improve debugging for bayes
1.0.10:
* Fix settings application (#416)
* Fix another issue with fixed strings
* Fix hash function invocation
* Use the proper string for make_dns_request in lua_http
* Fix scan time output
* Update webui:
- fix labels for greylisting
- fix dimension of scan time
1.0.9:
* Emergency fix in keyed blake2 to fix fuzzy hashes and encrypted password
* Support passwords longer than 64 symbols
1.0.8:
* Add function to traverse AST atoms
* Allow dependencies on rspamd symbols for SA metas
* Fix memory corruption when timeout is removed in fuzzy check
* Fix encrypted fuzzy add processing
* Avoid use-after-free in controller session destructor
* Use session pool instead of task pool in fuzzy check
* Fix assembly in i386 mode (#413, #412)
1.0.7:
* Plugged memory leaks in internet address object & html parser
* Fixed static build
* Fixed multiple sigchld processing
* Fixed deletion of signal events after event processing loop
* Fixed build on ARM (#404 - reported by @Gottox)
* Fixed setting the default mask for SPF.
* Fixed sanitisation of HTTP query values
* Fixed parsing of the last header in encrypted HTTP messages
* Additions and fixes for test suite & benchmarks
* Added openssl aes-256-gcm support to libcryptobox & HTTP server
* Implemented support for starting multiple HTTP servers
* Implemented batch accept in HTTP server
* Added module to get data from HTTP headers (#285 - reported by @msimerson)
* Added `rspamadm control` command
* Added ability to sort counters output.
* Added ability to specify custom headers for rspamc client
* Fix architecture detection
* Converted history storage to the UCL format
* Allow flexible number of rows in history
* Fix action badges in WebUI
* Add universal cryptobox hash API
* Migrated to the optimized blake2b implementation adopted from Andrew Moon
* Allow explicit loading of specific modules
* Always load settings module
* Allow to add symbols from settings
* Fix double free in the controller fuzzy learn command
* Avoid endless loop when cannot open sqlite db
* Updated libucl
1.0.6:
* Fix build on i386
* Update CentOS7 service file patch (by @fatalbanana)
* Fix path to rspamadm in Debian init script (by @fatalbanana)
* Fix broken '_SC_GETPW_R_SIZE_MAX' on FreeBSD
* Fix portability issues
* Use cryptobox chacha for libottery
* Better support of 32 bit builds
* Fix header name tokens setup
* Fix levenstein distance method for words
* Add workaround for old libevent (#400)
* Fix microseconds in termination timer
* Fix some more issues with fixed strings
* Explicitly test CPU instructions even after CPUID call
* Do not check out of boundary memory
* Do not output broken emails
* Fix unknown symbols registration
* Handle SIGILL using longjmp
* Block signals when exiting event loop
* Fix incorrect allocation size
* Slightly optimize alignment
* Restore rspamd -t for compatibility
* Add more sanity checks for emails
1.0.5:
* Add rspamd control interface:
- support `stat` command to get runtime stats of rspamd workers
- support `reload` command to reload runtime elements (e.g. sqlite3 databases)
* Rework curve25519 library for modular design:
- add Sandy2x implementation by Tung Chou
- fix CPU detection for variables loading assembly
- add testing for curve25519 ECDH
* New fixed strings library
* Add `R_SUSPICIOUS_IMAGES` rule
* Enable mmap in sqlite3
* Use new strings in the HTTP code
* Improve google perftools invocation
* Improve performance profiling in http test
* Reorganize includes to reduce namespace pollution
* Allow specific sections printing in configdump command
* Rework workers signals handlers to be chained if needed
* Update socketpair utility function
* Add control_path option for rspamd control protocol
* Fix ownership when listening on UNIX sockets
* Rework signals processing in main
* Remove extra tools from rspamd (they live in rspamadm now)
* Remove global rspamd_main
* Add global timeout for the overall task processing (8 seconds by default)
* Sanitize NULL values for fuzzy backend
* Store NM between encrypt/decrypt
* Add textpart:get_words_count method
* Fix generic DNS request in lua
* Tune hfilter weights
* Add support of IPv6 in hfilter
* Fix parsing of HTTP headers with IP addresses
* Sync with the recent libucl
* Various minor bugfixes
1.0.4:
* Add configdump routine to rspamadm
* Implement retransmits for fuzzy_check plugin
* Fix events processing for learning anf checking fuzzy hashes
* Avoid dependency on unneeded and uncompatible glib include
* Add `historyreset` command to the controller
* Fix loading of tokenizer config from dump (#389)
* Add sorting hints for the history
* Allow custom lua scripts for users/languages extraction (#388)
* Do not add FORGED_RECIPIENTS when 'To' is missing (#387)
* Do not add R_UNDISC_RCPT when 'To' is missing (#387)
* Add encryption to fuzzy check plugin
* Add encryption for fuzzy storage
* Add new epoch for encrypted fuzzy request
* Add encryption for `rspamd.com` storage
* Remove gmime processing for LDA mode as it is deadly broken
* Add routine to find end of headers position in mime messages
* Fix LDA headers folding
* Init libraries in rspamc client as well to avoid locale issues
* Avoid collision with locally installed includes
* Allocate and free memory with the same allocator in rspamadm (#385)
* Preserve expired fuzzy hashes counter
* Improvements in webui:
- Add favicon.ico
- Rework history table
- Fix sorting for the history
- Migrate to bootstrap 3 and jquery 2
- Fix css bugs
- Add glyphicons
- Add reset history
- Improve history buttons
- Redraw graph to avoid display issues
- Webui is now MIT licensed to match licensing policy of rspamd
1.0.3:
* Fix piechart clean slice (#380)
* Fix controller crashes when GString is reallocated (#381)
* Correctly set locale before start
* Set C locale for numeric values
* Add rspamadm routine:
- add `pw` command to manage passwords
- add `help` command for displaying help
- add `configtest` command to check configuration files
- add `keypair` command for generating encryption keys
- add `fuzzy_merge` routine to merge fuzzy sqlite databases
- add a simple manual page for rspamadm
* Allow metric registration for composite expressions
* Add strict mode for configtest
* Add logger counters
* Save and show learned messages count (#383)
* Add `no_stat` flag
* Add `task:set_flag` and `task:get_flags` (#382)
* Enable foreign keys in sqlite3
* Remove orphaned shingles from fuzzy storage
* Optimize synchronization steps for fuzzy storage
* Allow delayed conditions registration
* Add lua API for conditions registering
1.0.2:
* Fix critical bug in webui that prevents password from being sent
* Rework webui view:
- Switch to d3.js for graphs
- Improve piechart look
- Rework colors for piechart
- Fix layout for symbols
- Fix refresh button
* Add descriptions for whitelist maps
* Fix build on arm (#379)
* Fix issue with the last element in the radix trie
* Add more tests for radix trie algorithm
* Allow to extract URLs from query strings of other URLs (#361)
* Initialize rrd fields before writing to file
* Fix double free if no password has been specified
1.0.1:
* Add writing to rrd from the controller
* Fixed lots of bugs in rrd code
* Adopt new DNS API in hfilter plugin (by @AlexeySa)
* Allow only one controller process to manage rrd file
* Set event base for fuzzy calls
* Improve fuzzy IO errors logging
* Add rra extraction function to rrd library
* Add graph handler to the controller
* Cache correct passwords to avoid too high CPU usage when working with webui
* Controller sockets are owned by router do not export them to task
* Optimize logging by skipping hash table search if it's empty
* Fix loading issue whith broken statfiles
* Print assertions from glib to rspamd logger
* Load legacy `lua/rspamd.local.lua`
* Update webui with some fixes to learning and scanning
1.0.0:
* Rework symbols processing:
- Improve sorting logic for symbols
- Organize processing into multiple stages
- Added asynchronous watchers for symbols
- Added ability to organize dependencies between symbols
* Fixed URL redirector:
- Use optimized POE loop
- Organize dependencies
- Fix startup
* New sqlite3 backend:
- Allow to have per-languages and per-user statistics
- Allow sqlite3 to be used as statistics backend
* Store tokenizer configuration within statfiles
* Improve bayes statistics:
- Use headers and images metainformation in bayes
- Suggest using of pre-processed tokens for statistics
- Fix tokens normalization for OSB algorithm
* Rewrite url parsing:
- Fix numerous issues with url extraction and normalization
- Fix mailto urls
* Fix settings plugin to allow custom actions scores
* Improve rbl plugin
* Allow capturing patterns in rspamd lua regexp library
* Add GTUBE support
* Fix spamc legacy support
* Add DKIM support to RBL module
* Fix issues with multiple DKIM signatures
* Fix issue if rspamd cannot create statfiles (#331)
* Rework parts and task structure:
- Now text_parts, parts and received are arrays
- Pre-allocate arrays with some reasonable defaults
- Use arrays instead of lists in plugins and checks
- Remove unused fields from task structure
- Rework mime_foreach callback function
- Remove deprecated scan_milliseconds field
* Add ip_score plugin support (not enabled by default):
- Can check for asn/country and network using DNS lookups
- Can store and load reputation from redis server
* Improve PARTS_DIFFER rule to count merely different words
* New HTML parser:
- Parses HTML parts using a set of state machines
- Extracts useful data and exports it to lua functions:
+ Styles
+ Images
+ URLs
+ Colors
+ Structure elements
- Added HTML rules for some checks
* New version of LUA DNS API
* Table versions of many functions in LUA API
* Improve rspamc client:
- Print execution time
- Allow executing of external commands and passing output to them
- Allow mime output mode when rspamc alters message according to rspamd
checks and send it to an external command or stdout
* Allow scanning of local files using HTTP requests
* Rework configuration system:
- Rules are now moved from the $CONFDIR to $RULESDIR to avoid ambiguity
- All modules configurations are now split in $CONFDIR/modules.d/* to
simplify upgrades
- Move hfilter to plugins
- Allow plugins and rules to define default scores to simplify metrics
setup
- Include overrides for all modules to honor local/automatic parameters
- Tune scores for many modules
* Rework and enable DMARC plugin
* Add whitelist plugin for SPF/DKIM/DMARC based whitelisting
* Add some common domains to whitelists shipped with rspamd
* Rework logging:
- Now each log entry supports module name and a `tag`. Tag is used to
identify unique objects (such as tasks) when checking log files
- It is possible to turn on debugging for the specific modules
- Systemd logging is fixed
* Improve spamassassin plugin.
- Now headers are matched more like SA
- Improve support of Message-ID
- Add support of ToCc header type
- Fix :addr and :name in headers regexps
* Resurrect rrd support code
* Save controller stats between restarts
* Fixed tonns of bugs
* Added tonns of minor improvements and features
* Added more unit tests
* Create functional tests framework
* Added documentation for missing modules
* Added rpm/deb repositories and scripts
* Updated WebUI and libucl externals
0.9.10:
* Do not dereference null pointer on learning.
* Fix some extreme cases in BAYES.
* Add a workaround to avoid bad HTML messages breaking.
* Build with -O2 flags by default.
* Add constraints to limit DNS requests count per task.
* Add workaround for SURBL DNS flood.
* Set error if rspamd cannot learn anything.
0.9.9:
* Don't use RWL_SPAMHAUS_WL (unknown result) for whitelisting (by @fatalbanana)
* Import updated public suffix list (by @fatalbanana)
* Remove debug message
* Fix settings (by @fatalbanana)
* Remove duplicated symbol registration
* Use WAL for fuzzy storage
* RBL fixes (by @fatalbanana):
- silence errors;
- yield unknown results from RBLs;
- fix scoring for DNSWL;
- fix use of RBL name as symbol;
- ignore RBL names that would not be yielded;
* Support captures in regular expressions
* Add captures support to lua_regexp
* Support dist on FreeBSD and Darwin
* Add RCVD_IN_DNSWL_NONE as whitelisting exclusion (by @fatalbanana)
* Multiple fixes to URL detection:
- support port definition;
- fix query and path recognition;
- fix parsing of multiple slashes in URL;
- fix parsing query just after port;
- fix path field in `url:to_table` method;
- improve support of IP based URLs.
* Set ignore_whitelists = true for RECEIVED_SPAMHAUS_XBL (by @fatalbanana)
* Add GTUBE support
* Ignore User header in SA mode
0.9.8:
* Fix critical bug in bayes classifier (#305)
* Fix critical bug in RBL module (by @fatalbanana)
* Fix and rework settings plugin.
* Fix get_all_opts for a case of non-iterable options.
* Use tld for redirector's matching.
0.9.7:
* Add whitelist_exception setting to RBL module (by @fatalbanana)
* Don't use RWL_MAILSPIKE_POSSIBLE or DNSWL_BLOCKED for whitelisting (by
@fatalbanana)
* Fix extreme cases in bayes classifier.
* Fix parsing of urls with '?' at the end of hostname.
* Update interface.
* Fix number of issues with webui interaction.
* Fix saving maps.
* Allow user@ and @domain matches in multimap.
* Fix issues with bounces From processing.
* Fix abs/fabs misuse.
* Fix builds on suse and arch linux distributions.
0.9.6:
* Fix memory leak if mime cannot be parsed.
* Fix dkim cache expiration.
* Fix issues with redirector HTTP response.
* Fix abnormal connection closing with certains messages with a high score
(issue #296)
* Fix redirector installation.
* Use specific POE loop for some systems.
* Fix number of issues in URL redirector.
* Fix selecting URLs for sending to redirector.
0.9.5:
* Avoid double free when extending HTTP message.
* Fix double free if multiple classifiers are defined.
* Fix misprint in spamassassin plugin.
* Fix cpuid invocation on i386.
* Fix ownership issues for zero-copy decode.
* Allow __len metamethod on rspamd{text}.
* Add base64 decoding lua utility.
* Fix build on FreeBSD
* Skip spaces at the beginning of mime messages.
* DBL_ABUSE_REDIR should not have significant weight.
* Allow to split by lua_regexp rspamd{text} objects.
* Allow to specify custom stop pattern for lua_tcp.
0.9.4:
* Fix critical bugs in tokenization algorithm
* Write unit tests for tokenization
* Add documentation for lua_tcp
* Switch off legacy tokenization by default.
* Fix critical bugs in words normalization
* Add lua bindings to tokenizer.
* Implement storing of HTTP headers inside task
* Add lua API to accerss HTTP headers data
* Implemented base64 encoding suitable for MIME
* Use caseless hash and equal functions for HTTP request headers.
* Improve debian architectures support (by @dottedmag)
0.9.3:
* Revert incorrect regexp change that broke the default rules
* Fix lua_tcp module
0.9.2:
* Fix error on spawning unique workers.
* Add preliminary version of generic LUA TCP requests API.
* Use lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc)
* Fix fuzzy mime strings with only type.
* Improve thunderbird sanity checks.
* Fix critical bug on matching regular expressions.
* Make hiredis optional dependency.
* Fix multiple bugs in daemon reloading
0.9.1:
* Restore utf8 validation for regular expressions to avoid crashes
* Fix symbols displaying in the interface
* Add symbol groups to the interface
* Fix maps ID parsing in the controller
* Add multimap and regexp modules documentation
* Backport fixes from libucl
* Fix debian package (by @dottedmag)
* Rework XXH32 invocations
0.9.0:
* Add support of the fast and secure protocol level encryption:
- curve25519 is used for key exchange;
- chacha20/poly1305 cryptobox construction for bulk encryption;
- zero latency overhead;
- encrypting and balancing HTTP proxy worker
* Rework expressions and create new expressions library:
- aggressive optimizations based on the abstract syntax tree;
- abstract expressions support (regular expressions, functions, lua modules
composites and so on)
- New comparision and '+' operators support
- New greedy algorithm to minimize execution time of expressions and
all symbols
- Dynamic expressions benchmark and reoptimizations
* Many improvements to the LUA API:
- reworked logger module allowing to do pretty print of the most of lua
types (including tables and userdata classes)
- reworked lua redis and lua HTTP to support more features
- added opaque type for passing large text chunks without copying
- new regexp module with many auxiliary functions (e.g. `re:split`)
* LuaJIT is now the default requirement for rspamd allowing to speed up lua
execution by a large margin (however, plain lua is still supported)
* New plugins:
- spamassassin rules plugin that allows to load and re-use the most of
SA rules natively
- DMARC plugin that evaluates SPF and DKIM policies to the domain policies
- many old plugins has been reworked to implement new features and improve
stability
* New aho-corasic trie implementation from @mischasan that allows to load and
use hundreds of thousands of patterns with no influence on load
* Support of PCRE JIT and PCRE JIT fast path modes that significantly improves
the performance of regular expressions if supported by PCRE
* New URLs parser and extractor:
- removed legacy code that was useless for url finding
- reworked algorithms of URL parsing for more precise and accurate results
- added top-level-domains tree from http://publicsuffix.org
- improved emails parsing
- removed many phishing false positives due to TLD tree check
* New statistics infrastructure:
- created a separate layer of statistic library
- improved OSB-Bayes by re-weighting tokens according to the original
academic paper and `crm114` implementation, which reduced false positives
rate significantly
- created learn cache to avoid double learning of statistics and providing
an efficient way to re-learn class for a message
- created abstract layers for different statistics backends
- implemented new tokenization algorithms with fast or secure (siphash)
hashes to generate statistics features
* Reworked utf8 tokenization that previously corrupted all UTF8 words (minor
incompatibility with old fuzzy hashes with utf-8 symbols)
* SPF module has been completely rewritten to support complex cases of
`include` and `redirect` within SPF records
* DKIM module now supports multiple signatures
* Controller passwords can now be stored encrypted by `PBKDF2-HMAC` in the
configuration file
* Many hand-written HTTP clients has been replaced with the common rspamd
http module
* New test framework:
- import lua `telescope` test framework
- add unit tests for many rspamd modules and routines
- create a unit test for each possible bug found
- use luajit ffi for testing C code
- added preliminary support of functional testing by creating tasks from lua
* Randomize hash seed to avoid certain hash tables vulnerabilities
* Documentation improvements:
- added documentation for the vast majority of rspamd modules
- added documentation for rspamd protocol
- added documentation for the most of rspamd LUA extensions
* Fixed tonns of bugs and memory leaks
* Added tonns of minor features
0.8.3:
* Various critical fixes in distribution (by @dottedmag and @fatalbanana)
* Fixed bugs in url detector to parse certain patterns
* Add default host and helo for a client
* Some sanity checks for tokenizer and classifier
* Reiterate on systemd support
* Fix missing symbol registration
* Add support of spamc compatible output
* Filter double-dots in rbl.lua validate_dns (by @fatalbanana)
* Update ucl submodule due to critical bugfix
0.8.2:
* Create fuzzy db if it does not exist
* Fix: Centos init script: configtest() (by @AlexeySa)
* Enable one_shot for RECEIVED_SPAMHAUS_XBL - Fixes #102 (by @fatalbanana)
* Update Exim patch (by @fatalbanana)
* Fix processing of unix sockets.
* Allow applying settings to authenticated users (by @fatalbanana)
* Make settings priorities work as documented (by @fatalbanana)
* Fix race condition in symbols planner
* Add DNSWL_BLOCKED symbol (by @fatalbanana)
* Make Exim pass usernames to rspamd (by @fatalbanana)
* Update RBL module (by @fatalbanana):
- fix indentation;
- collapse loops;
- avoid calling for un-needed information;
- allow disabling RBLs for authenticated users
* once_received.lua: Fix indentation & add exclusion for authenticated users (by @fatalbanana)
* hfilter.lua: Add exclusion for authenticated users (by @AlexeySa)
* Updates to hfilter rules (by @AlexeySa)
* Set empty <> user or addr for msgs without FROM (by @eneq123)
* Fix: attempt to index field '?' (a nil value) (by @eneq123)
* Fix: if not exist Date-header (by @AlexeySa)
* Add task:get_content() method.
* rbl.lua: Ignore private IP space (by @fatalbanana)
* Allow to check radix maps from lua by rspamd{ip}
* Make local exclusions configurable per-RBL (by @fatalbanana)
* Add rspamd_config:radix_from_config() (by @fatalbanana)
* Support emails dnsbl in rbl (by @fatalbanana)
* Complete rework of url extraction logic
* Allow customizations for unix sockets. (fixes #182)
* Set lua path according to rspamd settings.
* Import lua-functional for plugins stuff.
* Completely rewrite multimap plugin in functional style.
* Fix FORGED_MUA_THUNDERBIRD_MSGID (fixes #186)
* Check IPv6 addresses at dnswl.org and Spamhaus whitelist (by @fatalbanana)
* Add lowercase utility for utf8 strings.
* Various fixes to build system
* Updated debian configuration infrastructure (by @dottedmag)
0.8.1:
* Add sqlite and perl as dependencies for RPM/Debian packages (by @fatalbanana)
* Remove whitelist.lua from RPM file list (by @fatalbanana)
* Make Exim pass hostnames to rspamd (by @fatalbanana)
* Fix building on Fedora (by @fatalbanana)
* Add toggle for disabling installation of systemd units on Linux (by @fatalbanana)
* Fix double format rounding that caused output corruption (reported by @fatalbanana)
* Revert broken change for destructors ordering that led to memory corruption
* Do not reset symbols case of settings if parsed from lua (reported by @andrejzverev)
* Fix build on SunOS (by @wiedi)
* Fix multiple crashes on broken DKIM DNS records
* Fix critical issue with composites weights removing
* Fix memory corruption in composites processing code
* Ignore non-SPF TXT records when parsing SPF includes
0.8.0:
* New fuzzy check logic:
- use shingles algorithm for fuzzy matching
- use blake2 instead of md5 for larger output space
- combine fuzzy and strict matching
- allow to organize private storages by means of keys
- preserve compatibility with previous versions
* New fuzzy storage:
- use sqlite instead of own memory based hash tables
- rework commands interface
- add conversion from the old format
- add fuzzy match by shignles
- support old rspamd versions
* Add lemmatizing for words used in fuzzy hashes that allows to improve match
quality by using of the first forms of all words
* Rework language detection
* Fix several critical bugs, memory leaks and deadlocks:
- memory leak in HTML nodes parsing
- deadlock in logger code
- deadlock in signals processing
- crashes in fuzzy_storage
- crashes in tokenizers if the input was empty
* Import new libucl with several bugfixes and improvements
* Support listening on ipv6 addresses only
* Fix macro expansion in SPF module
* Several bugfixes in DKIM module
* Add load headers support for mime parts to the lua API
* Add documentation for:
- workers in general
- fuzzy_storage worker
- fuzzy_check plugin
- mimepart and textpart lua API modules
0.7.6:
* Apply boundary fix for dkim simple canonization
* Fix ping command
* Return nil if header was not found in lua_task
* Fix hang in upstreams revive logic
* Decode entitles when normalizing HTML parts
* Fix logic of finding URLs in HTML parts
* Do not include \0 into length of text when performing conversion to utf8
* Fix raw vs parsed reperesentations
Raw parts are now:
- decoded b64/qp, but *NOT* converted to utf-8
Processed parts are now:
- converted to UTF-8
- normalized if needed (e.g. HTML tags are stripped)
* Rework DKIM canonization to line based
* Fix fuzzy hashes addding
* Use more specific hash function for fuzzy
* Fix leaking of iconv descriptors
* Fix PTR resolving in lua resolver
* Rework spf module.
- Copy data to memory pool as cached record might be destroyed causing
freed memory being passed to the protocol output (use after free)
- Allow SPF_NEUTRAL policy to be handled separately
- Add R_SPF_NEUTRAL to the default config
* Rework `register_symbols` function
* Allow to disable components of hfilter
0.7.5:
* Fix owner when creating folder /run/rspamd (by @sfirmery)
* Fix IP validity checks
* Decode URLs obtained from HTML tags
* Fix crash with unweighted upstreams
* Stop processing headers in parts
* Set sockaddr.sa_family properly when connectig to upstreams
* Fix reload issues in surbl and fuzzy_check (reported by @citrin)
* Fix timeouts in redirector
* Improve lua errors reporting
* Fix lua closures processing in libucl
* Rework calling of lua functions from regexp module
* Choose raw regexp for raw headers
* Rework conversion to utf since glib one is broken
* Ignore SGML style tags in html
* Fix old bug with non-capturing https urls
* Fix memory corruption on fuzzy reload (reported by @citrin)
* Fix percents display in rspamc
* Fix buffer update for DKIM
* Do not validate utf for raw headers
0.7.4:
* Fix build under *BSD
* Detect HAN unicode script
* Implement language detection heuristic for text parts
* Fix time output in history
* Improve piechart coloring
* Fix \r\n conversion in DKIM module (reported by @citrin)
* Try to detect systems with no IPv6 support
* Fix multiple/single values in use settings (reported by @citrin)
* Rework IP addresses in upstreams:
- Select ipv4/unix addresses if they exist and use ipv6 for ipv6 only
upstreams (since the support of ipv6 is poor in many OSes and
environments)
- Free IP list on upstream destruction
- Add test cases for addresses selection
- Allow adding of free form IP addresses to upstreams
* Fix endiannes in lua_radix search (reported by @citrin)
* Soft shutdown should also set wanna_die flag (reported by @citrin)
* Stop use-after-free in event loop termination
* Fix processing of very short messages in DKIM (reported by @citrin)
* Detect systems without shared mutexes
* Fix issues with PTR and MX elements in SPF parser (reported by @citrin)
0.7.3:
* New upstreams code:
- simplify upstreams API;
- unify strings parsing in upstreams definition;
- add configuration options for the upstreams;
- for failed upstreams re-resolve their addresses;
- use all resolved addresses for an upstream (round-robin);
- implement stable hashing and use it by default for upstreams;
- add unit test for upstreams module.
* Rework signals processing in all rspamd workers:
- signals are now processed in the event loop;
- implement the most common signal handlers for all workers;
- add callbacks for workers specific signal handlers
* Fix critical issue with fuzzy storage:
Fuzzy stroage could not save any hashes on termination due to bugged
signals handling
* Fix roll history IP storage
* Rework ipv4/ipv6 handling in parsing addresses:
- turn off support of IPV6_V6ONLY socket option;
- create ipv6 socket prior to ipv4 one to handle systems with v6/v4
sockets enabled (Linux)
* Remove CBL as it's wholly included in Spamhaus XBL (by @fatalbanana)
* Remove nszones.com fake RBL (by @citrin)
* Fix upstreams interaction for fuzzy_check
* Verify spf PTR records (reported by @citrin)
* Fix spf MX records parsing
* Add compatibility for old libevent (by @yellowman)
* Sync bugfixes from libucl
0.7.2:
* Convert all maps to the compressed radix trie
* Allow IPv6 addresses in IP maps
* Remove dynamic items support from symbols cache
* Allow hex encoded output of strings
* Fix bug with control connections count
* Process fuzzy weight correctly (reported by @fatalbanana)
* Remove extra reference retain of http connection on error
* Remove deprecated options from the default config
* Add `one_shot` attr to metric's symbols
* Doc: add documentation for metrics
* Add Upstart job to debian packaging (by @CameronNemo)
* Config: improve SURBL symbols descriptions (by @citrin)
* Config: reflect SURBL changes (by @citrin):
- Outblaze removed, malware moved to separate list:
http://www.surbl.org/news/internal/MW-malware-sublist-added-to-multi
* Fix C modules initialization on restart
* Treat single IP as a single IP in radix lists (reported by @citrin)
* Do not touch file and core limits if not asked explicitly (reported by @citrin)
* Improve logging for fuzzy errors
* Block SIGPIPE for HTTP writing
* Doc: update manual pages
* Fix HTTP connection termination
* Reduce default number of parallel requests to 8
* Sync with libucl include features
0.7.1:
* Fix typo in stat output.
* Fix issues with includes crossing with the system includes
* Restore testing framework
* Add radix trie test suite
* Implement new path-compressed radix trie.
- The performance benefit over the old algorithm is about 1.5 times.
- Memory usage is significantly lower as well.
- Now radix trie can accept any IPv4/IPv6 values
* Various improvements to the memory pools code
* Fix writing reply to a client when no filters are defined
* Write base32 encoded fuzzy
* Fix 'soft reject' action
* Fix rspamd reload and modules reconfiguration
* Fix subject rewriting for the default subject
* Fix states for processing task and pre-filters
* Fix issues with connection closing
* Fix crashes in rdns
* Fix ratelimit pre-filter
* Update exim patch.
- Update to the recent exim version
- Strip extra leading src/ from the patch
- Remove sendfile since it was broken
- Fix rspamd spam report for exim
* Improve documentation
0.7.0:
* Use HTTP protocol for all operatiosn
* Webui worker is now removed and controller works as webui
* Allow to serve static files via controller by option `static_dir`
* Rspamd interface is now a part of rspamd
* Rspamc client has been rewritten to use HTTP and non-blocking mode
allowing to start multiple operations simultaneously (see `-n` option)
* Lua API was completely reworked to satisfy modern standards of LUA:
* Module `lua-message` was removed
* Reduced number of superglobals registered by rspamd
* Many functions has been redesigned
* Symbols registration is now more convenient
* Users settings has been rewritten as lua plugin
* Reworked headers system as gmime's based one misses many headers and is
very slow to get headers values
* Reorganized code and removed embedded jannsson by using UCL for all json
parsing
* Migrated to `librdns` for DNS resolving that improves concurrency for
DNS requests significantly
* Fixed tonns of bugs in MIME processing
* Improved metrcis and default symbol's weights
* Added new RBL's
* Fixed a number of issues in the modules
* Removed several memory leaks found
* Fix unicode processing
* Fix fuzzy checking for unicode parts
* Significantly improve documentation and especially LUA API docs
* See migration notes at https://rspamd.com/doc/migration.html
0.6.8:
* Controller now listen for localhost and not for 127.0.0.1 by default
* Allow FCrDNS-style RBL lookups (by @fatalbanana)
* Reduce threshold for parts_differ function.
* Fix hostname lookup for rdns rbl (by @AlexeySa)
* Fix HFILTER_URL_ONELINE to reduce false positive rate.
* Fix whitelist module.
* Allow override system predefined settings without touching system ones
by .try_include macro (by @andrejzverev)
* Check for [ip.address]-style HELO and suppress lookups. (by
@fatalbanana)
* Optimize hfilter (by @AlexeySa)
* Fix issue with random numbers generator in dns.
* Use more clever time values to setup entropy.
* Synced with the recent libucl.
0.6.7:
* Use ChaCha20 for DNS generator (more secure DNS id)
* Unknown symbols now has zero weight and not 1.0
* Fix fuzzy hashes expire time
* Fix critical issue in statfiles on FreeBSD 9 (and some other platforms)
* Add .include_map macro to ucl parser
* Update libucl
* Fix headers end detection for DKIM module
* Fix a bug in received headers parser
* Validate IP addresses before pushing them to lua
* Start new documentation project
* Fixed tonns of other minor bugs
* Start to prepare for 0.7 with HTTP protocol and new settings
0.6.6:
* Removed issue with BUFSIZ limitation in the controller output
* Simplify logging symbols escaping
* Adjusted weights for several rules
* Improve spamhaus rbl support
* Removed PBL for received headers checks
* Added hfilter module that performs various HELO and IP checks.
* Rspamd can now be reloaded by HUP signal
* Fuzzy storage should expire hashes properly
* Build system has been reworked for better supportof pkg-config
* Various minor bugfixes
0.6.5:
* Fixed critical bug in DNS resolver, introduced in 0.6.4
* Improved multimap and rbl plugins to skip
* Add dns_sockets option for tuning sockets per server in DNS resolver
* Improved packages for rspamd
0.6.4:
* Added io channels for DNS request to balance load and reduce id
collisions chance
* Fixed a bug in SPF filter that may cause core dump in specific
circumstances
* FIxed default config for rbl module
* It is possible to get a list of rspamc commands with their descriptions
* Added SORBS bl to the default config
* 2tld file for surbl module has been significantly extended
* Perl modules has been removed from the code.
* Fixed an issue in libucl when parsing macros
0.6.3:
* Fixed issues with DNS:
- labels decompression algorithm was fixed;
- added resolve_mx to lua API;
- fixed modules that use DNS.
* Lua modules once_received and emails reworked for new resolver API and UCL.
* Debian package was polished.
* Fixed a bug in fuzzy_check module that prevents correct processing messages
without valid parts.
0.6.2:
* Fuzzy check module has been reworked:
- now fuzzy_check operates with a group of rules, that define which
servers sre to be checked;
- it is possible to specify read_only groups to avoid learning errors;
- turn fuzzy_check to one_shot mode permanently;
- fuzzy_check module configuration is now incompatible with the previous
versions.
* Imported bugfixes from libucl.
* Fixed whitelist plugin.
* Fixed statfiles resizing.
* Improved logging initialization order.
* Fixed race condition in the controller worker.
0.6.1:
* Critical bugfixes:
- fixed build system;
- fixed in_class setting in bayes learning;
0.6.0:
* Use UCL instead xml for configuration (https://github.com/vstakhov/libucl)
* Fix statistics module normalization
* Rework the many modules for the new configuration:
- surbl has incompatible configuration;
- fuzzy_check has incompatible configuration;
- multimap has now new configuration;
- received_rbl is removed and replaced by rbl module.
* Removed deprecated options:
- statfile_pool_size;
- action and required/reject score for a metric.
* Simplify build system and unify configuration for all platforms.
* Improved URL detector (reduced false positive rate).
* Lua 5.2 is now the default and fully supported version.
* Tons of bugfixes and minor improvements.
0.5.6:
* Fix bug with counters incrementing in rolling history
* Detect expl and exp2l as some systems do not have it
* Support input streams without Content-Length
* Implement counters output via rspamc and controller interface
* Fix bug with udp sockets in fuzzy storage
0.5.5:
* New bayes normalizator based on inverse chi-square function
* Various fixes to fuzzy storage
* Allow update fuzzy storage only from specific IP addresses
* Better support of IPv6 and address selection algorithms
* Add CentOS spec file
0.5.4:
* Fixed issues with diff algorithm
* Added support of RRD statistics
* Add webui worker for interface interaction
* Fix a lot of issues with dynamic conf
* Fix critical memory leak in settings code
* Improve stability of the system
0.5.3:
* Added dynamic options
* Added advanced metaclassfication
* Added RESTfull API for controller
* Improved hashing algorithms
* Various fixes for rspamc client:
- allow interacting with unix sockets
- librspamdclient major cleanup
- bayes is now default classifier
0.5.2:
* Added lua bindings for:
- basic mime parts, that allows checkign attachements for example;
- DNS resolver;
* Existing lua bindings now works without task object allowing to use them
in custom code.
* Threads system was reworked to avoid global lua interpreter lock.
* DKIM module now converts all line endings to CRLF how opendkim does.
* URL detector is now more accurate for text parts.
* Several critical bugs and memory leaks were fixed.
0.5.1:
* Added lua worker type to handle network connections in lua.
* Added lua bindings for async_session, IO dispatcher, memory_pool, and
worker.
* Composites can now uses other composites in expressions.
* Fixes for debian package and for FreeBSD >= 9.1
* Add support of gmime-2.6 if gmime-2.4 is not found.
* Improve url detection and phishing detection.
* Add lua mime_part library to get an access to all message part
attributes (like filename, length, type).
0.5.0:
* Added SMTP lightweight balancing proxy with XCLIENT support.
* Added lua bindings for upstreams objects and API.
* New pre-filters are implemented to support initial checking for messages.
* Added ratelimit plugin that uses redis protocol to store data.
* Added ipv6 support to spf and some other modules.
* Unbreak spf plugin.
* Allow options with the same name be threated as list.
* DKIM plugin an parsing code was added.
* Separate build system to put logic in several shared libraries.
* Many bugfixes.
0.3.2:
* Add error handling for regexps
* Fix quit command in controller interface
* Write symbols weights to rspamc output
* Improve logic of selecting rspamc version
* Do not try to parse broken DNS replies
* Add 'raw' flag to FROM_EXCESS_BASE64 rule (requested by citrin)
* Output message id in rspamc reply
* Fix inserting composite symbol
* Fix output of log line
* Document composites
* Add logging for fuzzy checks
* Add logging for learning
* Improve logic of learning messages - do not learn more than specific threshold
* Fix inserting results for symbols that were incorrectly (for example more than 1 time) defined in config file
* Do not output control characters if output is not terminal
* Fix some logic errors in learning
* Consider lua plugins errors as fatal configuration errors
* Fix wirting message id during fuzzy_add command
* Display weight of symbols correctly
* Fixes to winnow learning
* One more try to improve accuracy of winnow algorithm
* Add bayesian classifier (initial version)
* Remove normalizer as it is winnow specific thing, so all statistic algorithms now returns value from 0 to 1
* Some fixes to fuzzy hashes expiration:
* Fix assertion while look up value in NULL hash (found by cirtin)
* Fix normalization for systems that have not tanhl function
* Ignore rfc822 group addresses
* Move images library to core rspamd
* Add lua api to access images properties
* Add post filters to lua API - filters that would be called after all message's processing
* Add ability to check for specified symbol in task results from lua
* Add ability to check for metric's results from lua
* Add ability to learn specified statfile form lua
* Add ability to extract filename and size of images from lua
* Fix assertion while extracting internet address
* Fix races in fuzzy storage
* Make spf parser case insensitive
* Add ability to check hashes of selected mime types
* Add ability to set minimum size in bytes for mime types
* Add ability to set minimum dimensions for images
* Assume all text/* content types as text/plain
* Fix getting data wrapper for gmime24
* Many fixes to statfile syncronization system
* Fixed statfile pool initialization and synchronization with disk
* Prepare 0.3.2
* Fix Mail::Rspamd::Config for new rspamd features
* Use Mail::Rspamd::Config in rspamc client
* Write user's name to rspamd log
* Prepare rspamd build infrastructure for creating rpm and deb packages
* Fix depends
* Add start script for linux systems
* Fix shared usage of statfiles
* Add invalidation of statfiles in case of learning, so now statfiles
* This should fix shared usage of statfile pool by several processes
* Fix misprint (reported by az)
* Fix stupid error when all checks can be done in a single pass
* New trie based url scanner (based on libcamel)
* Small fixes to rspamd perl client
* Write fuzzy hashes info to log
* Add trie interface to lua api
* Explain sample config and cleanup it
0.3.1:
* Add modules documentation
* Continue implementing smtp proxy
* Implement new learning system, now rspamd should be much more intelligent while learning messages
* Convert statistic sums to use long double for counters
* Use hyperbolic tangent for internal normalizer
* In classify normalize result after comparing, not before
* New symbols sorter
* Fix strict aliasing while compiling with optimization
* Fix tanhl detection for platforms that have not implementation of it
* Remove several compile warnings
* Add experimental support of dynamic rules to regexp module
* Document views configuration
* Several fixes to documentation
* Add more logic for dynamic rules
* Add documentation for dynamic rules
* Add ability to make negations in networks in dynamic rules
* Clean up cache items correctly
* Implement basic SMTP dialog:
* Implement interaction with smtp upstream (with support of XCLIENT)
* Check messages received via smtp proxy
* Add support for sendfile in io dispatcher
* Fix issues with compatibility of worker_task and smtp proxy
* Proxy DATA command
* Fix SMTP
* Change metric logic
* Completely remove lex/yacc readers for config
* Make common sense of metric/action and symbols
* Sync changes with all plugins
* Incorrectly removed in previous commit
* Fix misprint (by Andrej Zverev)
* announce the "password" keyword in usage list
* Implement initial version of greylisting triplets storage
* Fix issues with smtp worker
* Fix QUIT command in SMTP worker
* Some fixes about new metrics system (may be incomplete)
* Get weights of symbol from default metric for symbols cache
* Fix setting task->from/task->rctp in smtp client
* Copy from and rcpt correctly
* Some performance improvements to IO dispatcher (do not drain the whole buffer after a single line readed)
* Fix smtp data input
* Fix misprint
* Add limit of maximum allowed smtp session errors
* New logic of SURBL module:
* Use system mkstemp(3) on systems where it is available as glib implementation
* Try to fix memmove issues in io dispatcher
* Remove debug from SURBL module
* Rewrite buffered input for line policy (again)
* Fix issue with links that are ip addresses in numeric form in surbl
* On Darwin use BSD style sendfile definition
* Reorganize platform specific knobs in CMakeLists
* Use gettimeofday on systems that have not clock_getres
* Use ftime for dns trans id generation on systems without clock_getres
* Darwin sendfile(2) support
* TIMEDB->TIMEB
* More to previous commit
* Pass env from main() arguments instead of platform specific global environ
* Fix compatibility issues
* Fix -lintl detection
* Init some variables to avoid problems
* Remove garbadge (gnome terminal sucks)
* Add more information about why we drop smtp connection
* Fix mkstemp call
* Send to upstream QUIT command at the end of session
* Check return value of each rspamd_dispatcher_write as in case of write errors sessions can be destroyed early
* Fix states in smtp dialog
* Use rspamd_snprintf instead of libc one
* Fix URLS command
* Fix reconfigure process of surbl module
* Fix destroying smtp session (unmap memory and do not delete pool early)
* Delete pool after using its variables
* Delay timer must be registered in async session to correctly handle connection termination
* Register dns requests in session too
* Make session before registering events
* Remove events in handlers
* Add ability to set filters for smtp worker for each smtp stage
* Add very initial version of DNS resolver (many things to be done)
* Announce weights and sync
* Fix few typo
* Understand short names of facility in logging config
* Add ability to make whitelist for spf checks
* Misprint != -> ==
* Handle lua tag in way that it is not required to write additional text:
* Strip all starting whitespace symbols from xml texts
* Fix stupid bug in calculating buffer length while reading file maps
* Add resolv.conf parsing into dns.c
* Fix microseconds<->milliseconds conversions
* Take callback argument in Mail::Rspamd::Client for processing files and directories
* Print results if rspamc is called for a directory
* Fix stupid error with surbl module reconfig (another one, blame me)
* Do not show duplicate urls in url header
* Fix detection of numeric urls (reported by citrin)
* Write real time of message's scan to log (not only virtual)
* Fix chartable module in utf mode
* Fix parsing of some broken urls
* Add ability to test regexp with 'T' flag
* Write more code for DNS resolver:
* Make DNS resolver working
* Many improvements to rspamd test suite: now it CAN be used for testing rspamd functionality
* Write DNS resolver tests
* Fix issues with memory_pool mutexes and with creating of statfiles
* Forgotten in previous commit
* Add support for parsing SPF and SRV records
* Fix PTR parsing
* Add tests
* Make SURBL module to use rspamd dns module
* Several fixes to DNS logic
* Remove evdns and use only rspamd resolver
* Very hard to detect problem with race among error in socket and destroying task while we are writing to socket and go through a hash table
* Fix resolving in smtp module
* Init events before configuring resolver in smtp worker
* Set resolver inside task
* Fix reload signal (reported by citrin)
* Some improvements to redirector
* Call has_forked method to inform POE about fork
* Fix lua DNS code
* Decompress labels in DNS packets more strictly
* Fix some problems with TXT records
* Try to fix removing of DNS events
* Do not insert unparsed RR's into reply
* Calling callbacks may cause destroying session from which we are calling callback so we MUST call callback as the latest action
* Fix check_smtp_data function
* Add ability to make views by recipient
* Add ability to set metric's action from config file
* Fix bug with writing garbadge if message has no urls or no messages
* Fix bug with incorrect behaviour of compare_parts_distance function
* Add ability to assign several actions to one metric
* Report action in rspamc protocol
* Mail::Rspamd::Client and rspamc can now understand Action header too
* Write action to log as well
* Make valgrind happy about comparing symbols
* Add more debug to comparing parts distance function
* Write action even if message has no symbols
* Make improvements to HTML entites decoder: now it replaces entities with common characters and
* Add -d option to force debug output
* Assume 7bit as default transfer encoding
* Do not overwrite lua plugins and configs if they already exists in target directory
* Improve logging
* Write queue id to log
* Remove test messages from cmake
* Reopen log file by USR1 signal
* Add reopenlog method to FreeBSD rc script
* Adopt foreach for cmake 2.6
* Fix to rc script
* Do not try to resolve names with several dots in a row
* Fix surbl request formatting for ip addresses
* Handle cases of broken requests
* Fix problems with parsing compressed names
* Fix TXT records parsing
* Fix expanding spf macros that may fail in rare cases
* Fix another error with early task destroying
* Handle empty from header
* Improve reopenlog command in rc script
* Strip trailing whitespace characters in Mail::Rspamd::Client
* Use ungreedy match to strip trailing whitespaces
* Stupid error in calculation compressed label length
* Some optimizations to client library
* Do not compare empty parts
* Empty and non-empty parts are allways different
* Save in regexp cache the whole regexp with header name (if exists) and with flags
* Add rspamd_log variable to lua plugins to access logging functions
* Each part in rspamd task now can have parent part
* Check for parts distance only for multipart/alternative subparts
* Do not check attachements even if they are text (but attached as file)
* Do not die if write (2) returned ENOSPACE while doing logging, turn on throttling mode instead (1 write try in a second)
* Add ability to turn on debug for specific symbols
* Add ability to configure dns timeouts and dns retransmits in config file
* More debug
* Fix extracting arguments in lua logger interface
* Turn off debug_ip during reload if it was disabled by new config
* Improve lua logging
* Pre-init symbols cache when rereading config
* Fix lua representing of invalid ip (nil, not 255.255.255.255)
* Fix R_TO_SEEMS_AUTO rule (by citrin)
* Add multimap lua plugin
* Fix some multimap issues
* Try to save images hashes to fuzzy storage to stop some annoying spammers
* Allocate some more bytes for read buffer to avoid incorrect behavoiur
* Add ability to check dns black lists by multimap module
* Add multimap documentation
* Fix labels parsing
* Another try to save regexps in cache correctly
* Improve test logs for regexps
* Fix parsing txt records to avoid reading of uninitialized data
* Fix error with writing symbols cache file
* Fix error while working in utf mode when raw regexps was not created properly
* Do not add extra byte while converting text to utf
* Add error handling for regexps
|