1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144
2145
2146
2147
2148
2149
2150
2151
2152
2153
2154
2155
2156
2157
2158
2159
2160
2161
2162
2163
2164
2165
2166
2167
2168
2169
2170
2171
2172
2173
2174
2175
2176
2177
2178
2179
2180
2181
2182
2183
2184
2185
2186
2187
2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
2214
2215
2216
2217
2218
2219
2220
2221
2222
2223
2224
2225
2226
2227
2228
2229
2230
2231
2232
2233
2234
2235
2236
2237
2238
2239
2240
2241
2242
2243
2244
2245
2246
2247
2248
2249
2250
2251
2252
2253
2254
2255
2256
2257
2258
2259
2260
2261
2262
2263
2264
2265
2266
2267
2268
2269
2270
2271
2272
2273
2274
2275
2276
2277
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
2291
2292
2293
2294
2295
2296
2297
2298
2299
2300
2301
2302
2303
2304
2305
2306
2307
2308
2309
2310
2311
2312
2313
2314
2315
2316
2317
2318
2319
2320
2321
2322
2323
2324
2325
2326
2327
2328
2329
2330
2331
2332
2333
2334
2335
2336
2337
2338
2339
2340
2341
2342
2343
2344
2345
2346
2347
2348
2349
2350
2351
2352
2353
2354
2355
2356
2357
2358
2359
2360
2361
2362
2363
2364
2365
2366
2367
2368
2369
2370
2371
2372
2373
2374
2375
2376
2377
2378
2379
2380
2381
2382
2383
2384
2385
2386
2387
2388
2389
2390
2391
2392
2393
2394
2395
2396
2397
2398
2399
2400
2401
2402
2403
2404
2405
2406
2407
2408
2409
2410
2411
2412
2413
2414
2415
2416
2417
2418
2419
2420
2421
2422
2423
2424
2425
2426
2427
2428
2429
2430
2431
2432
2433
2434
2435
2436
2437
2438
2439
2440
2441
2442
2443
2444
2445
2446
2447
2448
2449
2450
2451
2452
2453
2454
2455
2456
2457
2458
2459
2460
2461
2462
2463
2464
2465
2466
2467
2468
2469
2470
2471
2472
2473
2474
2475
2476
2477
2478
2479
2480
2481
2482
2483
2484
2485
2486
2487
2488
2489
2490
2491
2492
2493
2494
2495
2496
2497
2498
2499
2500
2501
2502
2503
2504
2505
2506
2507
2508
2509
2510
2511
2512
2513
2514
2515
2516
2517
2518
2519
2520
2521
2522
2523
2524
2525
2526
2527
2528
2529
2530
2531
2532
2533
2534
2535
2536
2537
2538
2539
2540
2541
2542
2543
2544
2545
2546
2547
2548
2549
2550
2551
2552
2553
2554
2555
2556
2557
2558
2559
2560
2561
2562
2563
2564
2565
2566
2567
2568
2569
2570
2571
2572
2573
2574
2575
2576
2577
2578
2579
2580
2581
2582
2583
2584
2585
2586
2587
2588
2589
2590
2591
2592
2593
2594
2595
2596
2597
2598
2599
2600
2601
2602
2603
2604
2605
2606
2607
2608
2609
2610
2611
2612
2613
2614
2615
2616
2617
2618
2619
2620
2621
2622
2623
2624
2625
2626
2627
2628
2629
2630
2631
2632
2633
2634
2635
2636
2637
2638
2639
2640
2641
2642
2643
2644
2645
2646
2647
2648
2649
2650
2651
2652
2653
2654
2655
2656
2657
2658
2659
2660
2661
2662
2663
2664
2665
2666
2667
2668
2669
2670
2671
2672
2673
2674
2675
2676
2677
2678
2679
2680
2681
2682
2683
2684
2685
2686
2687
2688
2689
2690
2691
2692
2693
2694
2695
2696
2697
2698
2699
2700
2701
2702
2703
2704
2705
2706
2707
2708
2709
2710
2711
2712
2713
2714
2715
2716
2717
2718
2719
2720
2721
2722
2723
2724
2725
2726
2727
2728
2729
2730
2731
2732
2733
2734
2735
2736
2737
2738
2739
2740
2741
2742
2743
2744
2745
2746
2747
2748
2749
2750
2751
2752
2753
2754
2755
2756
2757
2758
2759
2760
2761
2762
2763
2764
2765
2766
2767
2768
2769
2770
2771
2772
2773
2774
2775
2776
2777
2778
2779
2780
2781
2782
2783
2784
2785
2786
2787
2788
2789
2790
2791
2792
2793
2794
2795
2796
2797
2798
2799
2800
2801
2802
2803
2804
2805
2806
2807
2808
2809
2810
2811
2812
2813
2814
2815
2816
2817
2818
2819
2820
2821
2822
2823
2824
2825
2826
2827
2828
2829
2830
2831
2832
2833
2834
2835
2836
2837
2838
2839
2840
2841
2842
2843
2844
2845
2846
2847
2848
2849
2850
2851
2852
2853
2854
2855
2856
2857
2858
2859
2860
2861
2862
2863
2864
2865
2866
2867
2868
2869
2870
2871
2872
2873
2874
2875
2876
2877
2878
2879
2880
2881
2882
2883
2884
2885
2886
2887
2888
2889
2890
2891
2892
2893
2894
2895
2896
2897
2898
2899
2900
2901
2902
2903
2904
2905
2906
2907
2908
2909
2910
2911
2912
2913
2914
2915
2916
2917
2918
2919
2920
2921
2922
2923
2924
2925
2926
2927
2928
2929
2930
2931
2932
2933
2934
2935
2936
2937
2938
2939
2940
2941
2942
2943
2944
2945
2946
2947
2948
2949
2950
2951
2952
2953
2954
2955
2956
2957
2958
2959
2960
2961
2962
2963
2964
2965
2966
2967
2968
2969
2970
2971
2972
2973
2974
2975
2976
2977
2978
2979
2980
2981
2982
2983
2984
2985
2986
2987
2988
2989
2990
2991
2992
2993
2994
2995
2996
2997
2998
2999
3000
3001
3002
3003
3004
3005
3006
3007
3008
3009
3010
3011
3012
3013
3014
3015
3016
3017
3018
3019
3020
3021
3022
3023
3024
3025
3026
3027
3028
3029
3030
3031
3032
3033
3034
3035
3036
3037
3038
3039
3040
3041
3042
3043
3044
3045
3046
3047
3048
3049
3050
3051
3052
3053
3054
3055
3056
3057
3058
3059
3060
3061
3062
3063
3064
3065
3066
3067
3068
3069
3070
3071
3072
3073
3074
3075
3076
3077
3078
3079
3080
3081
3082
3083
3084
3085
3086
3087
3088
3089
3090
3091
3092
3093
3094
3095
3096
3097
3098
3099
3100
3101
3102
3103
3104
3105
3106
3107
3108
3109
3110
3111
3112
3113
3114
3115
3116
3117
3118
3119
3120
3121
3122
3123
3124
3125
3126
3127
3128
3129
3130
3131
3132
3133
3134
3135
3136
3137
3138
3139
3140
3141
3142
3143
3144
3145
3146
3147
3148
3149
3150
3151
3152
3153
3154
3155
3156
3157
3158
3159
3160
3161
3162
3163
3164
3165
3166
3167
3168
3169
3170
3171
3172
3173
3174
3175
3176
3177
3178
3179
3180
3181
3182
3183
3184
3185
3186
3187
3188
3189
3190
3191
3192
3193
3194
3195
3196
3197
3198
3199
3200
3201
3202
3203
3204
3205
3206
3207
3208
3209
3210
3211
3212
3213
3214
3215
3216
3217
3218
3219
3220
3221
3222
3223
3224
3225
3226
3227
3228
3229
3230
3231
3232
3233
3234
3235
3236
3237
3238
3239
3240
3241
3242
3243
3244
3245
3246
3247
3248
3249
3250
3251
3252
3253
3254
3255
3256
3257
3258
3259
3260
3261
3262
3263
3264
3265
3266
3267
3268
3269
3270
3271
3272
3273
3274
3275
3276
3277
3278
3279
3280
3281
3282
3283
3284
3285
3286
3287
3288
3289
3290
3291
3292
3293
3294
3295
3296
3297
3298
3299
3300
3301
3302
3303
3304
3305
3306
3307
3308
3309
3310
3311
3312
3313
3314
3315
3316
3317
3318
3319
3320
3321
3322
3323
3324
3325
3326
3327
3328
3329
3330
3331
3332
3333
3334
3335
3336
3337
3338
3339
3340
3341
3342
3343
3344
3345
3346
3347
3348
3349
3350
3351
3352
3353
3354
3355
3356
3357
3358
3359
3360
3361
3362
3363
3364
3365
3366
3367
3368
3369
3370
3371
3372
3373
3374
3375
3376
3377
3378
3379
3380
3381
3382
3383
3384
3385
3386
3387
3388
3389
3390
3391
3392
3393
3394
3395
3396
3397
3398
3399
3400
3401
3402
3403
3404
3405
3406
3407
3408
3409
3410
3411
3412
3413
3414
3415
3416
3417
3418
3419
3420
3421
3422
3423
3424
3425
3426
3427
3428
3429
3430
3431
3432
3433
3434
3435
3436
3437
3438
3439
3440
3441
3442
3443
3444
3445
3446
3447
3448
3449
3450
3451
3452
3453
3454
3455
3456
3457
3458
3459
3460
3461
3462
3463
3464
3465
3466
3467
3468
3469
3470
3471
3472
3473
3474
3475
3476
3477
3478
3479
3480
3481
3482
3483
3484
3485
3486
3487
3488
3489
3490
3491
3492
3493
3494
3495
3496
3497
3498
3499
3500
3501
3502
3503
3504
3505
3506
3507
3508
3509
3510
3511
3512
3513
3514
3515
3516
3517
3518
3519
3520
3521
3522
3523
3524
3525
3526
3527
3528
3529
3530
3531
3532
3533
3534
3535
3536
3537
3538
3539
3540
3541
3542
3543
3544
3545
3546
3547
3548
3549
3550
3551
3552
3553
3554
3555
3556
3557
3558
3559
3560
3561
3562
3563
3564
3565
3566
3567
3568
3569
3570
3571
3572
3573
3574
3575
3576
3577
3578
3579
3580
3581
3582
3583
3584
3585
3586
3587
3588
3589
3590
3591
3592
3593
3594
3595
3596
3597
3598
3599
3600
3601
3602
3603
3604
3605
3606
3607
3608
3609
3610
3611
3612
3613
3614
3615
3616
3617
3618
3619
3620
3621
3622
3623
3624
3625
3626
3627
3628
3629
3630
3631
3632
3633
3634
3635
3636
3637
3638
3639
3640
3641
3642
3643
3644
3645
3646
3647
3648
3649
3650
3651
3652
3653
3654
3655
3656
3657
3658
3659
3660
3661
3662
3663
3664
3665
3666
3667
3668
3669
3670
3671
3672
3673
3674
3675
3676
3677
3678
3679
3680
3681
3682
3683
3684
3685
3686
3687
3688
3689
3690
3691
3692
3693
3694
3695
3696
3697
3698
3699
3700
3701
3702
3703
3704
3705
3706
3707
3708
3709
3710
3711
3712
3713
3714
3715
3716
3717
3718
3719
3720
3721
3722
3723
3724
3725
3726
3727
3728
3729
3730
3731
3732
3733
3734
3735
3736
3737
3738
3739
3740
3741
3742
3743
3744
3745
3746
3747
3748
3749
3750
3751
3752
3753
3754
3755
3756
3757
3758
3759
3760
3761
3762
3763
3764
3765
3766
3767
3768
3769
3770
3771
3772
3773
3774
3775
3776
3777
3778
3779
3780
3781
3782
3783
3784
3785
3786
3787
3788
3789
3790
3791
3792
3793
3794
3795
3796
3797
3798
3799
3800
3801
3802
3803
3804
3805
3806
3807
3808
3809
3810
3811
3812
3813
3814
3815
3816
3817
3818
3819
3820
3821
3822
3823
3824
3825
3826
3827
3828
3829
3830
3831
3832
3833
3834
3835
3836
3837
3838
3839
3840
3841
3842
3843
3844
3845
3846
3847
3848
3849
3850
3851
3852
3853
3854
3855
3856
3857
3858
3859
3860
3861
3862
3863
3864
3865
3866
3867
3868
3869
3870
3871
3872
3873
3874
3875
3876
3877
3878
3879
3880
3881
3882
3883
3884
3885
3886
3887
3888
3889
3890
3891
3892
3893
3894
3895
3896
3897
3898
3899
3900
3901
3902
3903
3904
3905
3906
3907
3908
3909
3910
3911
3912
3913
3914
3915
3916
3917
3918
3919
3920
3921
3922
3923
3924
3925
3926
3927
3928
3929
3930
3931
3932
3933
3934
3935
3936
3937
3938
3939
3940
3941
3942
3943
3944
3945
3946
3947
3948
3949
3950
3951
3952
3953
3954
3955
3956
3957
3958
3959
3960
3961
3962
3963
3964
3965
3966
3967
3968
3969
3970
3971
3972
3973
3974
3975
3976
3977
3978
3979
3980
3981
3982
3983
3984
3985
3986
3987
3988
3989
3990
3991
3992
3993
3994
3995
3996
3997
3998
3999
4000
4001
4002
4003
4004
4005
4006
4007
4008
4009
4010
4011
4012
4013
4014
4015
4016
4017
4018
4019
4020
4021
4022
4023
4024
4025
4026
4027
4028
4029
4030
4031
4032
4033
4034
4035
4036
4037
4038
4039
4040
4041
4042
4043
4044
4045
4046
4047
4048
4049
4050
4051
4052
4053
4054
4055
4056
4057
4058
4059
4060
4061
4062
4063
4064
4065
4066
4067
4068
4069
4070
4071
4072
4073
4074
4075
4076
4077
4078
4079
4080
4081
4082
4083
4084
4085
4086
4087
4088
4089
4090
4091
4092
4093
4094
4095
4096
4097
4098
4099
4100
4101
4102
4103
4104
4105
4106
4107
4108
4109
4110
4111
4112
4113
4114
4115
4116
4117
4118
4119
4120
4121
4122
4123
4124
4125
4126
4127
4128
4129
4130
4131
4132
4133
4134
4135
4136
4137
4138
4139
4140
4141
4142
4143
4144
4145
4146
4147
4148
4149
4150
4151
4152
4153
4154
4155
4156
4157
4158
4159
4160
4161
4162
4163
4164
4165
4166
4167
4168
4169
4170
4171
4172
4173
4174
4175
4176
4177
4178
4179
4180
4181
4182
4183
4184
4185
4186
4187
4188
4189
4190
4191
4192
4193
4194
4195
4196
4197
4198
4199
4200
4201
4202
4203
4204
4205
4206
4207
4208
4209
4210
4211
4212
4213
4214
4215
4216
4217
4218
4219
4220
4221
4222
4223
4224
4225
4226
4227
4228
4229
4230
4231
4232
4233
4234
4235
4236
4237
4238
4239
4240
4241
4242
4243
4244
4245
4246
4247
4248
4249
4250
4251
4252
4253
4254
4255
4256
4257
4258
4259
4260
4261
4262
4263
4264
4265
4266
4267
4268
4269
4270
4271
4272
4273
4274
4275
4276
4277
4278
4279
4280
4281
4282
4283
4284
4285
4286
4287
4288
4289
4290
4291
4292
4293
4294
4295
4296
4297
4298
4299
4300
4301
4302
4303
4304
4305
4306
4307
4308
4309
4310
4311
4312
4313
4314
4315
4316
4317
4318
4319
4320
4321
4322
4323
4324
4325
4326
4327
4328
4329
4330
4331
4332
4333
4334
4335
4336
4337
4338
4339
4340
4341
4342
4343
4344
4345
4346
4347
4348
4349
4350
4351
4352
4353
4354
4355
4356
4357
4358
4359
4360
4361
4362
4363
4364
4365
4366
4367
4368
4369
4370
4371
4372
4373
4374
4375
4376
4377
4378
4379
4380
4381
4382
4383
4384
4385
4386
4387
4388
4389
4390
4391
4392
4393
4394
4395
4396
4397
4398
4399
4400
4401
4402
4403
4404
4405
4406
4407
4408
4409
4410
4411
4412
4413
4414
4415
4416
4417
4418
4419
4420
4421
4422
4423
4424
4425
4426
4427
4428
4429
4430
4431
4432
4433
4434
4435
4436
4437
4438
4439
4440
4441
4442
4443
4444
4445
4446
4447
4448
4449
4450
4451
4452
4453
4454
4455
4456
4457
4458
4459
4460
4461
4462
4463
4464
4465
4466
4467
4468
4469
4470
4471
4472
4473
4474
4475
4476
4477
4478
4479
4480
4481
4482
4483
4484
4485
4486
4487
4488
4489
4490
4491
4492
4493
4494
4495
4496
4497
4498
4499
4500
4501
4502
4503
4504
4505
4506
4507
4508
4509
4510
4511
4512
4513
4514
4515
4516
4517
4518
4519
4520
4521
4522
4523
4524
4525
4526
4527
4528
4529
4530
4531
4532
4533
4534
4535
4536
4537
4538
4539
4540
4541
4542
4543
4544
4545
4546
4547
4548
4549
4550
4551
4552
4553
4554
4555
4556
4557
4558
4559
4560
4561
4562
4563
4564
4565
4566
4567
4568
4569
4570
4571
4572
4573
4574
4575
4576
4577
4578
4579
4580
4581
4582
4583
4584
4585
4586
4587
4588
4589
4590
4591
4592
4593
4594
4595
4596
4597
4598
4599
4600
4601
4602
4603
4604
4605
4606
4607
4608
4609
4610
4611
4612
4613
4614
4615
4616
4617
4618
4619
4620
4621
4622
4623
4624
4625
4626
4627
4628
4629
4630
4631
4632
4633
4634
4635
4636
4637
4638
4639
4640
4641
4642
4643
4644
4645
4646
4647
4648
4649
4650
4651
4652
4653
4654
4655
4656
4657
4658
4659
4660
4661
4662
4663
4664
4665
4666
4667
4668
4669
4670
4671
4672
4673
4674
4675
4676
4677
4678
4679
4680
4681
4682
4683
4684
4685
4686
4687
4688
4689
4690
4691
4692
4693
4694
4695
4696
4697
4698
4699
4700
4701
4702
4703
4704
4705
4706
4707
4708
4709
4710
4711
4712
4713
4714
4715
4716
4717
4718
4719
4720
4721
4722
4723
4724
4725
4726
4727
4728
4729
4730
4731
4732
4733
4734
4735
4736
4737
4738
4739
4740
4741
4742
4743
4744
4745
4746
4747
4748
4749
4750
4751
4752
4753
4754
4755
4756
4757
4758
4759
4760
4761
4762
4763
4764
4765
4766
4767
4768
4769
4770
4771
4772
4773
4774
4775
4776
4777
4778
4779
4780
4781
4782
4783
4784
4785
4786
4787
4788
4789
4790
4791
4792
4793
4794
4795
4796
4797
4798
4799
4800
4801
4802
4803
4804
4805
4806
4807
4808
4809
4810
4811
4812
4813
4814
4815
4816
4817
4818
4819
4820
4821
4822
4823
4824
4825
4826
4827
4828
4829
4830
4831
4832
4833
4834
4835
4836
4837
4838
4839
4840
4841
4842
4843
4844
4845
4846
4847
4848
4849
4850
4851
4852
4853
4854
4855
4856
4857
4858
4859
4860
4861
4862
4863
4864
4865
4866
4867
4868
4869
4870
4871
4872
4873
4874
4875
4876
4877
4878
4879
4880
4881
4882
4883
4884
4885
4886
4887
4888
4889
4890
4891
4892
4893
4894
4895
4896
4897
4898
4899
4900
4901
4902
4903
4904
4905
4906
4907
4908
4909
4910
4911
4912
4913
4914
4915
4916
4917
4918
4919
4920
4921
4922
4923
4924
4925
4926
4927
4928
4929
4930
4931
4932
4933
4934
4935
4936
4937
4938
4939
4940
4941
4942
4943
4944
4945
4946
4947
4948
4949
4950
4951
4952
4953
4954
4955
4956
4957
4958
4959
4960
4961
4962
4963
4964
4965
4966
4967
4968
4969
4970
4971
4972
4973
4974
4975
4976
4977
4978
4979
4980
4981
4982
4983
4984
4985
4986
4987
4988
4989
4990
4991
4992
4993
4994
4995
4996
4997
4998
4999
5000
5001
5002
5003
5004
5005
5006
5007
5008
5009
5010
5011
5012
5013
5014
5015
5016
5017
5018
5019
5020
5021
5022
5023
5024
5025
5026
5027
5028
5029
5030
5031
5032
5033
5034
5035
5036
5037
5038
5039
5040
5041
5042
5043
5044
5045
5046
5047
5048
5049
5050
5051
5052
5053
5054
5055
5056
5057
5058
5059
5060
5061
5062
5063
5064
5065
5066
5067
5068
5069
5070
5071
5072
5073
5074
5075
5076
5077
5078
5079
5080
5081
5082
5083
5084
5085
5086
5087
5088
5089
5090
5091
5092
5093
5094
5095
5096
5097
5098
5099
5100
5101
5102
5103
5104
5105
5106
5107
5108
5109
5110
5111
5112
5113
5114
5115
5116
5117
5118
5119
5120
5121
5122
5123
5124
5125
5126
5127
5128
5129
5130
5131
5132
5133
5134
5135
5136
5137
5138
5139
5140
5141
5142
5143
5144
5145
5146
5147
5148
5149
5150
5151
5152
5153
5154
5155
5156
5157
5158
5159
5160
5161
5162
5163
5164
5165
5166
5167
5168
5169
5170
5171
5172
5173
5174
5175
5176
5177
5178
5179
5180
5181
5182
5183
5184
5185
5186
5187
5188
5189
5190
5191
5192
5193
5194
5195
5196
5197
5198
5199
5200
5201
5202
5203
5204
5205
5206
5207
5208
5209
5210
5211
5212
5213
5214
5215
5216
5217
5218
5219
5220
5221
5222
5223
5224
5225
5226
5227
5228
5229
5230
5231
5232
5233
5234
5235
5236
5237
5238
5239
5240
5241
5242
5243
5244
5245
5246
5247
5248
5249
5250
5251
5252
5253
5254
5255
5256
5257
5258
5259
5260
5261
5262
5263
5264
5265
5266
5267
5268
5269
5270
5271
5272
5273
5274
5275
5276
5277
5278
5279
5280
5281
5282
5283
5284
5285
5286
5287
5288
5289
5290
5291
5292
5293
5294
5295
5296
5297
5298
5299
5300
5301
5302
5303
5304
5305
5306
5307
5308
5309
5310
5311
5312
5313
5314
5315
5316
5317
5318
5319
5320
5321
5322
5323
5324
5325
5326
5327
5328
5329
5330
5331
5332
5333
5334
5335
5336
5337
5338
5339
5340
5341
5342
5343
5344
5345
5346
5347
5348
5349
5350
5351
5352
5353
5354
5355
5356
5357
5358
5359
5360
5361
5362
5363
5364
5365
5366
5367
5368
5369
5370
5371
5372
5373
5374
5375
5376
5377
5378
5379
5380
5381
5382
5383
5384
5385
5386
5387
5388
5389
5390
5391
5392
5393
5394
5395
5396
5397
5398
5399
5400
5401
5402
5403
5404
5405
5406
5407
5408
5409
5410
5411
5412
5413
5414
5415
5416
5417
5418
5419
5420
5421
5422
5423
5424
5425
5426
5427
5428
5429
5430
5431
5432
5433
5434
5435
5436
5437
5438
5439
5440
5441
5442
5443
5444
5445
5446
5447
5448
5449
5450
5451
5452
5453
5454
5455
5456
5457
5458
5459
5460
5461
5462
5463
5464
5465
5466
5467
5468
5469
5470
5471
5472
5473
5474
5475
5476
5477
5478
5479
5480
5481
5482
5483
5484
5485
5486
5487
5488
5489
5490
5491
5492
5493
5494
5495
5496
5497
5498
5499
5500
5501
5502
5503
5504
5505
5506
5507
5508
5509
5510
5511
5512
5513
5514
5515
5516
5517
5518
5519
5520
5521
5522
5523
5524
5525
5526
5527
5528
5529
5530
5531
5532
5533
5534
5535
5536
5537
5538
5539
5540
5541
5542
5543
5544
5545
5546
5547
5548
5549
5550
5551
5552
5553
5554
5555
5556
5557
5558
5559
5560
5561
5562
5563
5564
5565
5566
5567
5568
5569
5570
5571
5572
5573
5574
5575
5576
5577
5578
5579
5580
5581
5582
5583
5584
5585
5586
5587
5588
5589
5590
5591
5592
5593
5594
5595
5596
5597
5598
5599
5600
5601
5602
5603
5604
5605
5606
5607
5608
5609
5610
5611
5612
5613
5614
5615
5616
5617
5618
5619
5620
5621
5622
5623
5624
5625
5626
5627
5628
5629
5630
5631
5632
5633
5634
5635
5636
5637
5638
5639
5640
5641
5642
5643
5644
5645
5646
5647
5648
5649
5650
5651
5652
5653
5654
5655
5656
5657
5658
5659
5660
5661
5662
5663
5664
5665
5666
5667
5668
5669
5670
5671
5672
5673
5674
5675
5676
5677
5678
5679
5680
5681
5682
5683
5684
5685
5686
5687
5688
5689
5690
5691
5692
5693
5694
5695
5696
5697
5698
5699
5700
5701
5702
5703
5704
5705
5706
5707
5708
5709
5710
5711
5712
5713
5714
5715
5716
5717
5718
5719
5720
5721
5722
5723
5724
5725
5726
5727
5728
5729
5730
5731
5732
5733
5734
5735
5736
5737
5738
5739
5740
5741
5742
5743
5744
5745
5746
5747
5748
5749
5750
5751
5752
5753
5754
5755
5756
5757
5758
5759
5760
5761
5762
5763
5764
5765
5766
5767
5768
5769
5770
5771
5772
5773
5774
5775
5776
5777
5778
5779
5780
5781
5782
5783
5784
5785
5786
5787
5788
5789
5790
5791
5792
5793
5794
5795
5796
5797
5798
5799
5800
5801
5802
5803
5804
5805
5806
5807
5808
5809
5810
5811
5812
5813
5814
5815
5816
5817
5818
5819
5820
5821
5822
5823
5824
5825
5826
5827
5828
5829
5830
5831
5832
5833
5834
5835
5836
5837
5838
5839
5840
5841
5842
5843
5844
5845
5846
5847
5848
5849
5850
5851
5852
5853
5854
5855
5856
5857
5858
5859
5860
5861
5862
5863
5864
5865
5866
5867
5868
5869
5870
5871
5872
5873
5874
5875
5876
5877
5878
5879
5880
5881
5882
5883
5884
5885
5886
5887
5888
5889
5890
5891
5892
5893
5894
5895
5896
5897
5898
5899
5900
5901
5902
5903
5904
5905
5906
5907
5908
5909
5910
5911
5912
5913
5914
5915
5916
5917
5918
5919
5920
5921
5922
5923
5924
5925
5926
5927
5928
5929
5930
5931
5932
5933
5934
5935
5936
5937
5938
5939
5940
5941
5942
5943
5944
5945
5946
5947
5948
5949
5950
5951
5952
5953
5954
5955
5956
5957
5958
5959
5960
5961
5962
5963
5964
5965
5966
5967
5968
5969
5970
5971
5972
5973
5974
5975
5976
5977
5978
5979
5980
5981
5982
5983
5984
5985
5986
5987
5988
5989
5990
5991
5992
5993
5994
5995
5996
5997
5998
5999
6000
6001
6002
6003
6004
6005
6006
6007
6008
6009
6010
6011
6012
6013
6014
6015
6016
6017
6018
6019
6020
6021
6022
6023
6024
6025
6026
6027
6028
6029
6030
6031
6032
6033
6034
6035
6036
6037
6038
6039
6040
6041
6042
6043
6044
6045
6046
6047
6048
6049
6050
6051
6052
6053
6054
6055
6056
6057
6058
6059
6060
6061
6062
6063
6064
6065
6066
6067
6068
6069
6070
6071
6072
6073
6074
6075
6076
6077
6078
6079
6080
6081
6082
6083
6084
6085
6086
6087
6088
6089
6090
6091
6092
6093
6094
6095
6096
6097
6098
6099
6100
6101
6102
6103
6104
6105
6106
6107
6108
6109
6110
6111
6112
6113
6114
6115
6116
6117
6118
6119
6120
6121
6122
6123
6124
6125
6126
6127
6128
6129
6130
6131
6132
6133
6134
6135
6136
6137
6138
6139
6140
6141
6142
6143
6144
6145
6146
6147
6148
6149
6150
6151
6152
6153
6154
6155
6156
6157
6158
6159
6160
6161
6162
6163
6164
6165
6166
6167
6168
6169
6170
6171
6172
6173
6174
6175
6176
6177
6178
6179
6180
6181
6182
6183
6184
6185
6186
6187
6188
6189
6190
6191
6192
6193
6194
6195
6196
6197
6198
6199
6200
6201
6202
6203
6204
6205
6206
6207
6208
6209
6210
6211
6212
6213
6214
6215
6216
6217
6218
6219
6220
6221
6222
6223
6224
6225
6226
6227
6228
6229
6230
6231
6232
6233
6234
6235
6236
6237
6238
6239
6240
6241
6242
6243
6244
6245
6246
6247
6248
6249
6250
6251
6252
6253
6254
6255
6256
6257
6258
6259
6260
6261
6262
6263
6264
6265
6266
6267
6268
6269
6270
6271
6272
6273
6274
6275
6276
6277
6278
6279
6280
6281
6282
6283
6284
6285
6286
6287
6288
6289
6290
6291
6292
6293
6294
6295
6296
6297
6298
6299
6300
6301
6302
6303
6304
6305
6306
6307
6308
6309
6310
6311
6312
6313
6314
6315
6316
6317
6318
6319
6320
6321
6322
6323
6324
6325
6326
6327
6328
6329
6330
6331
6332
6333
6334
6335
6336
6337
6338
6339
6340
6341
6342
6343
6344
6345
6346
6347
6348
6349
6350
6351
6352
6353
6354
6355
6356
6357
6358
6359
6360
6361
6362
6363
6364
6365
6366
6367
6368
6369
6370
6371
6372
6373
6374
6375
6376
6377
6378
6379
6380
6381
6382
6383
6384
6385
6386
6387
6388
6389
6390
6391
6392
6393
6394
6395
6396
6397
6398
6399
6400
6401
6402
6403
6404
6405
6406
6407
6408
6409
6410
6411
6412
6413
6414
6415
6416
6417
6418
6419
6420
6421
6422
6423
6424
6425
6426
6427
6428
6429
6430
6431
6432
6433
6434
6435
6436
6437
6438
6439
6440
6441
6442
6443
6444
6445
6446
6447
6448
6449
6450
6451
6452
6453
6454
6455
6456
6457
6458
6459
6460
6461
6462
6463
6464
6465
6466
6467
6468
6469
6470
6471
6472
6473
6474
6475
6476
6477
6478
6479
6480
6481
6482
6483
6484
6485
6486
6487
6488
6489
6490
6491
6492
6493
6494
6495
6496
6497
6498
6499
6500
6501
6502
6503
6504
6505
6506
6507
6508
6509
6510
6511
6512
6513
6514
6515
6516
6517
6518
6519
6520
6521
6522
6523
6524
6525
6526
6527
6528
6529
6530
6531
6532
6533
6534
6535
6536
6537
6538
6539
6540
6541
6542
6543
6544
6545
6546
6547
6548
6549
6550
6551
6552
6553
6554
6555
6556
6557
6558
6559
6560
6561
6562
6563
6564
6565
6566
6567
6568
6569
6570
|
3.9.1: 23 Jul 2024
* [Conf] Spf: Add R_SPF_PLUSALL symbol with some score
* [Feature] Spf: Treat SPF +all in a special way
* [Minor] Ensure some safety when checking weights
* [Minor] Fix several issues with flag propagation
* [Minor] Gpt: Improve prompt and add some conversion heursitics
* [Minor] Gpt: Remove top_p reduce temperature to 0
* [Minor] Gpt: Set response_format
* [Minor] Gpt: Use gpt-4o-mini by default
3.9.0: 12 Jul 2024
* [CritFix] Protect regexp matcher from regexps with empty patterns
* [Feature] Allow adding X-CMAE-Score header
* [Feature] Allow custom milter quarantine and tempfail messages
* [Feature] Allow to specify minimum weight in GPT plugin
* [Feature] Cloudmark: Add scores_symbols setting
* [Feature] Further optimization to the hot path
* [Feature] Make min_resolve_interval configurable
* [Feature] Report slow synchronous rules
* [Feature] Rspamc: add `files-list` option
* [Feature] Support metrics command in normal/proxy workers
* [Feature] Support reply in message pack format
* [Feature] Use msgpack in the client
* [Feature] Verify sanity of l= tag in DKIM
* [Feature] milter_headers: support not removing headers
* [Fix] Allow autolearn for the controller worker
* [Fix] Allow spaces in DMARC records
* [Fix] Allow to set 0 as number of rows to disable roll history
* [Fix] Always set the unlearn flag when relearning
* [Fix] Apply detection phase if fasttext could not detect language
* [Fix] Backport fixes from libucl
* [Fix] Backport multiple fixes from libucl
* [Fix] Change expiration logic for redis_cache
* [Fix] Check nconns when firing the final termination event
* [Fix] Do not apply tableshape for known senders
* [Fix] Do not crash if symbol is missing in the metric
* [Fix] Do not read out-of-boundary when doing base64 encoding
* [Fix] Do not save multipatterns to FS in certain cases
* [Fix] Fix another corner case that allows candidates to be freed without init
* [Fix] Fix documentation nesting
* [Fix] Fix dynamic_symbols in the multimap plugin
* [Fix] Fix inconsistent nesting in mime parts
* [Fix] Fix parsing of maps definitions
* [Fix] Fix shared memory proxying when compression is set
* [Fix] Fuzzy add/delete handlers are badly broken for compression
* [Fix] Honor dynamic thresholds for greylisting module
* [Fix] If we have one statfile disabled we need to disable all
* [Fix] Increase/decrease hash_key value for tokens depending on is_unlearn
* [Fix] Libucl: Fix messagepack autodetection
* [Fix] Resgore the collaterally removed line
* [Fix] Unset autolearn flag if we learn explicitly
* [Fix] Update `nconns` in proxy
* [Fix] Update greylisting threshold as it can be easily dynamic
* [Fix] Update is_spam in learned_ids when relearning
* [Fix] Use explicit timeouts in the metadata exporter
* [Fix] metric_exporter: avoid sending `nil` in metric values
* [Project] Add GPT plugin
* [Project] Add a tool to perform statistical analysis of classifiers
* [Project] Finally fix slow timer behaviour
* [Project] Implement fuzzy check retransmits backpressure
* [Project] Improve stats processing
* [Project] Reduce default window size of OSB tokenizer to 2
* [Project] Rework grow factor
* [Project] Start optimization of userdata hashing
* [Project] Use libarchive for troublesome archives
* [Rework] Allow metrics endpoint to be enabled for each scanner
* [Rework] Allow multiple conditions for dkim signing when using vault/redis
* [Rework] Introduce dynamic_rate_limit for ratelimits
* [Rework] Serialize control commands
* [Rework] Update fmt to version 11
* [Rework] Use direct hashing instead of string hashing
* [Rules] Added rules for detecting likely malware
3.8.2: 20 Feb 2024
* [Feature] Add extraction type for `from` maps
* [Feature] Allow to add templates to redis history prefix
* [Feature] Implement dynamic keys map in fuzzy storage
* [Feature] Lua_url: Add `to_http` method
* [Feature] Support JSON logging when in syslog mode
* [Fix] Deal with `Connection` and `Host` headers on proxying
* [Fix] Encode headers in metadata exporter
* [Fix] Fix initial maps load
* [Fix] Make stat tokens allocation consistent
* [Fix] Resolve issue with bayes stat in `rspamadm` mode
* [Fix] Try to fix url path issue
* [Rework] Breaking: Do not report module as action
* [Rework] Use khash instead of glib hashes for many reasons
3.8.1: 25 Jan 2024
* [Fix] Fix headers insertion in the ordered list
* [Fix] Fix learn error propagation
* [Fix] Fix learning with long prefixes
* [Fix] Fix potential double free by expclicitly zeroing pointers
* [Fix] Fix storing of the bayes tokens
* [Fix] Use short comments rather than long one
3.8.0: 19 Jan 2024
* [Project] Rspamadm fuzzyping command
* [Project] Use Redis scripts for Bayes statistics and cache
* [Project] Support JSON logging
* [CritFix] - Fix reported length of logging structure
* [Feature] Allow to set `max_users` for Redis bayes backend
* [Feature] Escape JSON when needed
* [Feature] Proxy: Allow `encrypted_only` option
* [Feature] Reiterate on gtube patterns
* [Feature] Support ping command in fuzzy storage
* [Feature] Support suppressing DMARC reporting for particular recipients
* [Feature] rbl: support disabling or replacing url_whitelist per RBL
* [Fix] Another try to fix setproctitle
* [Fix] Cleanup session on exit
* [Fix] Do not cleanup hyperscan files unless new ones are loaded
* [Fix] Fix bad access when printing pending events on failure path
* [Fix] Fix issues with the raw header on header insertion/modification
* [Fix] Fix overflow in fuzzy_stats command
* [Fix] Fix some corner cases when parsing maps with no newlines
* [Fix] Fix various issues with canonicalisation of the paths
* [Fix] Fix wrong raw_len usage
* [Fix] Make words selection random deterministic upon content
* [Fix] Properly set config field when creating tasks from Lua
* [Fix] Really fix the language detector statistical heuristic
* [Fix] Set loaded variable explicitly
* [Fix] dkim_signing: siging_table: lowercase before lookup
* [Fix] known_senders: fix config handling
* [Fix] rbl: fix `exclude_local`
3.7.5: 15 Dec 2023
* Add support for loongarch
* [Fix] Fix rspamadm lua logger
* [Fix] Fix systemd logging
* [Fix] dkim_signing: siging_table: lowercase before lookup
* [Minor] Fix build with no hyperscan
* [Minor] Try improve test stability
3.7.4: 14 Nov 2023
* [Enhancement] Add composite rule for suspicious URLs in suspicious messages
* [Fix] Another try to fix setproctitle
* [Fix] Do not cleanup hyperscan files unless new ones are loaded
* [Fix] Fix various issues with canonicalisation of the paths
* [Fix] Properly set config field when creating tasks from Lua
* [Fix] Set loaded variable explicitly
* [Fix] known_senders: fix config handling
* [Fix] rbl: fix `exclude_local`
* [Minor] Add missing <algorithm> include for gcc 14
* [Minor] Add rule for messages missing both X-Mailer and User-Agent header
* [Minor] Bundle fasttext as linux distributives are just useless
* [Minor] Enable fasttext on RPM based linux
* [Minor] Forgot it in one more place
* [Minor] Improve FREEMAIL_AFF capture rates
* [Minor] Limit fasttext to amd64 only
* [Minor] Properly search for fasttext include
* [Minor] Treat *.zpaq attachments as archives and harmful
* [Minor] Update to 3.7.4
* [Minor] force_actions: set a group for symbols
* [WebUI] Update map editor
3.7.3: 27 Oct 2023
* [Fix] Emergency fix for the hyperscan path error
3.7.2: 26 Oct 2023
* [Feature] rbl: support checking returncodes by CIDR
* [Feature] rbl: support checking returncodes by regex
* [Feature] rbl: support globbed return codes
* [Fix] DMARC reporting: fix reporting for subdomains
* [Fix] Deal with fmtlib exceptions properly
* [Fix] backport fix for dlfcn.h from backward-cpp
* [Rules] Blank spam detection
3.7.1: 07 Oct 2023
* [CritFix] Fix leak in `gzip` function
* [Feature] Add ICAP Content-Type and Filename
* [Feature] Add `logging`->`task_max_elts` option
* [Feature] Add utility to split string like stuff for C++ code
* [Feature] Allow to set HTTP auth parameters for the maps
* [Feature] Check for plugin configuration errors on `configtest`
* [Feature] `known_senders` plugin
* [Feature] Use backward-cpp instead of manual libunwind stuff
* [Feature] rbl: support checking numeric URLs in isolation
* [Fix] CMakeLists.txt remove whitespace added by linter as it makes tests fail
* [Fix] Change Date: header location to conform with RFC
* [Fix] Correct format pattern for RE tree tempfile name
* [Fix] Correct format string for unw_word_t
* [Fix] Do not accept invalid ucl object types
* [Fix] Do not pollute public headers with libev internals
* [Fix] Do not set output type if list application failed
* [Fix] Fix `url:set_redirected` method
* [Fix] Fix format string and some length issues
* [Fix] Fix grammar definition for content-disposition attributes
* [Fix] Fix lua schema enrichment logic for Redis params
* [Fix] Fix lua stack corruption when logging large tables
* [Fix] Fix merge table utility
* [Fix] Fix output of non-RSA DKIM keys
* [Fix] Fix some corner cases of single-host urls parsing
* [Fix] Fix various issues in the `url_redirector` plugin
* [Fix] MISSING_MIMEOLE: avoid matching messages from Android GMail app (#4561)
* [Fix] Prevent DNSWL sabotage
* [Fix] Try to fix unzip function
* [Fix] rbl: really fix dependency registration when symbols_prefixes is used
* [Fix] rspamadm mime: arguments beginning with letter `t`
* [Rework] Breaking: return back to semver
* [Rework] Move rcl logic to C++
3.6: 03 Aug 2023
* [Conf] Add `one_shot` to some specific multimap rules
* [Conf] Add language detection configuration
* [Conf] Add missing attributes for the language detection configuration
* [Conf] Remove outdated composite rules
* [Feature] Add `sentinel_password` option
* [Feature] Add ability to deny specific fuzzy flags by default
* [Feature] Add controller endpoint to get fuzzy hashes from messages
* [Feature] Add extra symbol when URL redirector reaches nested limit
* [Feature] Add function to transliterate utf8 to ascii with some normalisation
* [Feature] Add html parsing limit
* [Feature] Add order to urls structure
* [Feature] Add some missing functions to `lua_rsa` library
* [Feature] Allow fuzzy workers to exchange blocked information
* [Feature] Allow to have weak flags in fuzzy storage
* [Feature] Allow to read options from maps in the multimap plugin
* [Feature] Allow to use other methods when fasttext detection is enabled
* [Feature] Count stats per key per flag
* [Feature] Finish all features of dkim_keygen in Lua
* [Feature] Khash: Allow static initialisation
* [Feature] Maps: Add on_load support
* [Feature] Preliminary implementation of dynamic composites
* [Feature] Process HTML parts before text ones
* [Feature] Reorganise struct rspamd_url to be 64 bytes size
* [Feature] Save fuzzy ratelimit buckets
* [Feature] Use in-place deflation for strings
* [Feature] external_relay: add ip_map strategy
* [Fix] Avoid race between config new/free by using a counter
* [Fix] Do not use `rspamadm.dkim_keygen`
* [Fix] Feed fasttext language model with the pre-tokenized words
* [Fix] Fix `rspamd_has_only_html_part`
* [Fix] Fix an old issue with order of destruction race between redis pool and lua
* [Fix] Fix format string usage
* [Fix] Fix parsing due to old bug revealed
* [Fix] Fix parsing of the mask values that are invalid
* [Fix] Ignore non-unique stop words
* [Fix] Include the last character when parsing the last header with no value
* [Fix] More fixes to fuzzystat
* [Fix] Set proper counter
* [Fix] Try harder to clean pending bucket
* [Fix] Try harder to remove bad hyperscan files
* [Fix] Update stats before encryption...
* [Fix] dmarc gramar - allow spaces before ";"
* [Fix] rbl: fix dependency registration when symbols_prefixes is used
* [Fix] remove obsolete rspamd-redirector files
* [Fix] test external_relay: count should always be the last rule, as it have no matching condition
* [Project] Allow to register multimap symbols dynamically
* [Project] Implement fasttext language detection
* [Rework] Default max shots must not influence options
* [Rework] Write dkim keygen tool in lua
* [Rules] Add thread hijacking composite rule
3.5: 17 Mar 2023
* [Conf] Clarify `timeout` in rspamd_proxy
* [Conf] Fix hashbl
* [Conf] Add SURBL hashbl support (WIP)
* [Conf] RBL: Fix selector
* [Conf] Reduce the default timeout
* [CritFix] Deserialise hyperscan to the page-aligned space to prevent alignment issues
* [CritFix] Fill path field in hyperscan notice command
* [Feature] Add `thresholds` field to the scan result
* [Feature] Add possibility to execute lua scripts for blocked fuzzy clients
* [Feature] Add preliminary support of the external maps in the multimap plugin
* [Feature] Allow to build a map by combining tuples of selectors
* [Feature] Allow to query external maps for settings
* [Feature] Allow to specify `selector_alias` in the maps definition
* [Feature] Enable Mime part filters on antivirus module
* [Feature] Improve ratelimit redis scripts
* [Feature] Selectors: Add specific_urls_filter_map extractor
* [Feature] Some rework of the selectors framework
* [Fix] Add O_CREAT flag when creating a file
* [Fix] Add a database check function unless we have anything from Hyperscan
* [Fix] Add hyperscan databases sanity check
* [Fix] Add workaround for ENOBUFS error on sending
* [Fix] Do not do `lstat` when we are creating file
* [Fix] Finally get rid of RSPAMD_USE_47BIT_LIGHTUSERDATA_HACK
* [Fix] Fix boundaries that contain only dashes
* [Fix] Fix off-by-one error in css tokenizer
* [Fix] Fix url reputation plugin
* [Fix] Fix usage of the Redis config schema as `extra_fields`
* [Fix] Further checks for the hs_scratch_alloc
* [Fix] Honor group flag for one shot
* [Fix] Normalize glob paths to avoid hash table misses
* [Fix] Ratelimit: Use unpack for `HMGET` return value
* [Fix] Rbl: Fix helo check pipeline
* [Fix] Replace broken strict_domains with phishing_exceptions
* [Fix] Restore `strict_domains` support
* [Fix] Return true from has_urls(true) if only emails are present
* [Fix] Rework lists applications
* [Fix] Set symcache item in coroutine calls
* [Fix] Treat `hs_allocate_scratch` errors as non-fatal
* [Fix] Treat hostnames with no dots as eSLD of their own
* [Fix] received: filtering of artificial header
* [Project] Add external maps support
* [Rework] Breaking: Do not report soft reject in history
* [Rework] Convert chartable plugin to c++ for convenience
* [Rework] Use a different approach for customization of the settings
* [Rules] Mid: Add MID_END_EQ_FROM_USER_PART rule
3.4: 01 Nov 2022
* [CritFix] Restore compatibility with the integrations and headers alterations
* [Feature] Milter_headers: Add `x-rspamd-action` routine
* [Feature] Share hyperscan database among processes
* [Fix] Another corner case in url parsing
* [Fix] Another fix for the enable password
* [Fix] Another try to fix close method in lua_tcp
* [Fix] Fix additional fields in the Redis schema
* [Fix] Fix emoji joiner FP
* [Fix] Fix favicon.ico Content-Type header
* [Fix] Fix hang when close is used
* [Fix] Lua_tcp: Sigh, another try to fix `close` invocation
* [Fix] Mx_check: Cache the fact of a missing MX record
* [Fix] Try to fix parsing of the unencoded `>` characters in html attributes
* [Fix] Try to fix the case where password == enable_password
* [Project] (Re)implement hyperscan caching
* [Project] Rework cleanup
* [Project] Synchronize hyperscan caches via the main process
* [Rework] Convert multipattern to use hyperscan tools
* [Rework] Make http normalize path function a generic function
* [Rework] Split locked and unlocked files, as mmap does not need flock normally
* [Rework] Start movement of the hyperscan related routines into a single unit
* [Rework] Store the current worker, so other libraries could use this information
* [Rework] Use blocking socket for IPC between main and workers
* [Rework] Use more predictable size for commands buffers
* [Rules] Do not insert ONCE_RECEIVED_STRICT on RDNS missing
* [Rules] Reduce score of HTTP_TO_HTTPS - subject to remove completely
3.3: 01 Oct 2022
* [Conf] Add missing groups for whitelist module symbols
* [CritFix] Neural: Fix keys regression after #3968
* [Feature] Accept upstream in lua_tcp
* [Feature] Add ability to statically maintain disabled/enabled patterns
* [Feature] Add function to store upstreams for HTTP urls
* [Feature] Allow augmentations set in Lua API
* [Feature] Allow lua_http module to accept upstreams
* [Feature] Allow to limit write access to fuzzy storage by key
* [Feature] Allow to sort symbols output
* [Feature] Check content for binary stuff before dumping it to Lua
* [Feature] Implement symbols augmentations
* [Fix] Add missing flags
* [Fix] Add more sanity checks for rua in dmarc_report
* [Fix] Adjust length of the fuzzy checks for short text parts
* [Fix] Another try to fix add headers compatibility logic
* [Fix] Another try to fix race condition in the runtime destruction
* [Fix] Avoid cyclic references in symcache and fix memory leaks
* [Fix] Avoid overriding IP with Sender IP
* [Fix] BAD_REP_POLICIES did not trigger when message was classified as spam by Bayes
* [Fix] Bind AF_UNIX DGRAM client connection to annonymous address
* [Fix] Disable IPv6 lookups for Blocklist.de RBL
* [Fix] Distinguish dynamic and static items
* [Fix] Dkim: Ignore unknown DKIM kv pairs as stated in RFC
* [Fix] Dmarc report: Use local timezone instead of GMT
* [Fix] Do not exclude authenticated users from URIBL lookups
* [Fix] Empty envelopes should not be emitted as arrays (json+messagepack) when populated envelopes are objects. This greatly complicates decoding in strictly typed languages.
* [Fix] External_relay: Restore the originating hostname check
* [Fix] Fix DKIM keys with spaces still allowing errors on invalid base64
* [Fix] Fix copying of sockaddr_un addresses
* [Fix] Fix crash with cname replies
* [Fix] Fix dependencies propagation
* [Fix] Fix iteration over milter headers
* [Fix] Fix ordering when sorting symcache
* [Fix] Fix reading of the cached maps
* [Fix] Fix several issues with the HTTP keepalive parsing
* [Fix] Fix stack smashing
* [Fix] Fix synchronous auth/select in lua_redis
* [Fix] Fix various symcache issues
* [Fix] Ignore all (I hope) unknown DKIM signature KV pairs
* [Fix] Ignore directories in RarV5 archives
* [Fix] Libucl: avoid memory leak on objects merging
* [Fix] Lua_tcp: Another try to fix closing logic
* [Fix] Mempool: Fix alloc_array function to actually multiply nmembers by size
* [Fix] Only check allowed fuzzy worker update ips for non-unix sockets
* [Fix] Plug memory leak in regexp destruction with pcre2
* [Fix] Properly check the original email flag
* [Fix] Properly deal with `get_symbol/get_metric_symbol` ambiguity
* [Fix] Properly parse expressions atoms
* [Fix] Properly set `Host` in rspamd_proxy
* [Fix] Rbl: Fix received positioned checks
* [Fix] Remove check for a score with no symbol being registered
* [Fix] Same fix for lua_tcp
* [Fix] Skip cname records when processing SPF records
* [Fix] Skip sending dmarc reports in no-opt mode fixes https://github.com/rspamd/rspamd/issues/4241
* [Fix] Stop slow timer on task destruction
* [Fix] Symcache: Do not use C style comparators in C++ sorts
* [Fix] Try to avoid a corner case for `@` pattern
* [Fix] Try to fix dkim reputation adjustements
* [Fix] Try to fix passthrough results processing logic
* [Fix] Try to fix the mess with read only flag
* [Fix] Upstreams: Don't ignore revive_time config option
* [Fix] Use proper format string, sigh...
* [Fix] Use space category in ragel automata to resolve space characters
* [Fix] Zstd: Fix compression with the new Zstd API
* [Fix] milter_headers: Header fields may be inserted at wrong position.
* [Project] Add experimental HTTP statistics backend
* [Project] Add more methods for symbols addition
* [Project] Add raii_sink file helper
* [Project] Add some more methods
* [Project] Add symbols processing methods
* [Project] Allow `=` separated augmentations to be treated as kv pairs
* [Project] Allow to extract augmentation values
* [Project] Few more methods
* [Project] Fix on conditions
* [Project] Further efforts to make a more consistent architecture
* [Project] Further rework
* [Project] Further rework tracking
* [Project] Further split of the code
* [Project] Get rid of C style ctors/dtors
* [Project] Http_stat: Notice statfiles when creating runtime
* [Project] Implement dynamic items lookup and processing
* [Project] Implement item finalization
* [Project] Implement more methods
* [Project] Implement runtime creation
* [Project] Implement settings processing + some neats
* [Project] Implement some conditions checks
* [Project] Implement validation logic
* [Project] More methods
* [Project] Move runtime cache part to a separate unit
* [Project] Move some more methods
* [Project] Re-implement counters method
* [Project] Reimplement dependencies processing
* [Project] Remove obsoleted methods
* [Project] Remove old code (finally)
* [Project] Rework symbols execution
* [Project] Some more adjustments in symbols registration
* [Project] Start rewrite symcache in c++
* [Project] Support augmentations with values
* [Project] Symcache: Use ordered filters to avoid extra lookups
* [Rework] Another movement
* [Rework] Augmentations can now imply flags
* [Rework] Further steps
* [Rework] Further work on deps processing
* [Rework] Implement cache resorting
* [Rework] Isolate disable/enable logic for the configuration ucl objects
* [Rework] Move item implementation to a separate header
* [Rework] Multimap: Avoid prefilters usage where augmentations can be used
* [Rework] Pass upstream when sending TCP requests
* [Rework] Re-implement cache sorting
* [Rework] Reimplement saving/loading the cache items
* [Rework] Reiterate on priorities
* [Rework] Rework files structure
* [Rework] Rewrite rspamc in C++
* [Rework] Simplify scores check and extend it to pre/post filters
* [Rework] Switch minimum C++ standard version to C++20
* [Rework] Try to fix the mess with types & flags
* [Rework] Use another version of hash table from the same author
* [Rework] Use dynamic items for calling callbacks
* [Rework] Use dynamic items in the callbacks
* [Rework] Use hash map for id->symbol mappings
* [Rework] Use khash instead of uthash in rdns compression logic
3.2: 26 Mar 2022
* [Conf] Score MIME_OBFUSCATED_ARCHIVE to 8 points
* [Conf] Set one_shot for URIBL rules by default
* [CritFix] Fix upstreams name resolution when there is also a port
* [Feature] Add ROC feature to neural network plugin
* [Feature] Add public suffic compilation utility
* [Feature] Add support of Cloudmark
* [Feature] Allow hyperscan for ppc64, as vectorscan now suports it.
* [Feature] Allow to skip DNS resolution for keep-alive connections
* [Feature] Aws_s3: Allow to store large parts separately
* [Feature] BIMI: Add preliminary version of the BIMI plugin
* [Feature] JSON endpoint for querying maps
* [Feature] Lua_magic: Add a sane CSV heuristic
* [Feature] Lua_mime: Add schema for message transfer
* [Feature] Output average scan time in /stat endpoint
* [Feature] Show average scan time in `rspamc stat` output
* [Fix] Add guards to avoid race condition on TCP connection
* [Fix] Allow spaces in DKIM key records
* [Fix] Apply the similar fix to the url_reputation
* [Fix] Avoid overwriting whitelisted_signers_map
* [Fix] Backport PR from libucl
* [Fix] Clear SSL errors
* [Fix] ClickHouse cleanup of old partitions
* [Fix] Do not double call error handler on ssl errors in the timeout path
* [Fix] Do not forget to clear pointers on IOC reset
* [Fix] External_relay: Remove useless check of the map value
* [Fix] Find suspicious url encodings that could break url extraction
* [Fix] Fix HTTP(s) client timeout
* [Fix] Fix exclude flags setting
* [Fix] Fix expanding of the variables
* [Fix] Fix host header usage in lua_http
* [Fix] Fix http maps shared memory cache cleanup
* [Fix] Fix logic in HTML processing FSM
* [Fix] Fix parsing of the compound mailto urls
* [Fix] Fix processing captures from pcre2
* [Fix] Fix removing from khash
* [Fix] Fix stuctured headers pushing
* [Fix] Further fix for i386 compilation
* [Fix] Improve duplicate settings error reporting
* [Fix] Lua: task:remove_result didn't work in some cases
* [Fix] Output service parts as well
* [Fix] Phishing: Deal with phishing + redirected URL
* [Fix] Phishing: Fix finding domains in the phishing map
* [Fix] Plug memory leak by using mempool for a copied address
* [Fix] Properly find the request and the number of requested entries
* [Fix] Rbl: Fix inversed logic of the url_full_hostname
* [Fix] Read file maps if they were not pre-read during preload
* [Fix] Restrict x86_64 assembly to x86_64
* [Fix] Return a real number of recipients when dealing with aliases
* [Fix] Rework unshedule DNS request function
* [Fix] Support definition of ungrouped symbol in conf file, use group info from lua or other conf file
* [Fix] Unschedule DNS request when clearing IO channel
* [Fix] When checking for phishing, we need to convert punicode -> UTF8, not vice versa
* [Fix] lua_cfg_transform - actions without score (discard)
* [Fix] lua_cfg_transform - silly break break actions
* [Fix] ratelimit - symbol per bucket
* [Project] BIMI: Fix helper integration issues
* [Project] Further DNS over TCP architecturing
* [Project] Rdns: Add more functions for TCP based requests
* [Project] Rdns: Add preliminary reading logic for TCP channels
* [Project] Rdns: Add reaper for inactive TCP connections
* [Project] Rdns: Add timeout logic for TCP requests
* [Project] Rdns: Do not treat TCP channels failure as fatal
* [Project] Rdns: Fix TCP connection mess
* [Project] Rdns: Fix TCP stuff cleanup
* [Project] Rdns: Fix various ownership issues
* [Project] Rdns: Implement TCP writing logic
* [Project] Rdns: Initial support of TCP IO channels
* [Project] Rdns: More fixes in TCP handling
* [Project] Rdns: Restore the previous EDNS0 size
* [Project] Rdns: Send truncated replies via TCP
* [Project] Rdns: Unregister TCP requests
* [Rework] Allow to restore SSL handlers after keepalive pooling
* [Rework] Allow to set a different behaviour for actions from settings
* [Rework] Include SSL flag into keepalive hash
* [Rework] Make `rspamadm dmarc_report` default behaviour more sane
* [Rework] Mempool: Use explicit alignment
* [Rework] Rdns: Use faster and more compact hash table for DNS requests
* [Rework] Rework SSL flag operations
* [Rework] Take disabled flag into account
* [Rework] Timeouts are now global per event and not reseted by IO activity
* [Rework] Use xxh3 as a default hash and fix memory/alignment issues
* [Rules] Fix old rules to stop global functions usage
* [Rules] Fix symbol for DKIM temporary failure
* [Rules] Remove ancient and inefficient rules
* [Rules] Slightly reduce MULTIPLE_FROM score
3.1: 01 Nov 2021
* [Feature] Add junk_threshold for autolearn
* [Feature] Add neural test command
* [Feature] Antivirus: Allow to set fake eicar patterns for testing AV engines
* [Feature] Lua_cdb: Add cdb building interface
* [Feature] Ratelimit: Add per bucket configurations
* [Feature] S3: Allow to store structured data in messagepack
* [Fix] Add concept of uncancellable events to prevent use-after-free
* [Fix] Add temporary guard to prevent linked list exploitation
* [Fix] Another rework of the ucl hashing
* [Fix] Another try to fix references safety
* [Fix] Another try to fix rspamd_text passing in the selectors
* [Fix] Avoid copy for received structure as it has raw C pointers
* [Fix] Avoid dangling reference
* [Fix] Correctly check numeric URLs in URL DNS lists
* [Fix] Delete the correct pointer type
* [Fix] Dmarc: Always lowercase domain
* [Fix] Fix compilation of the hyperscan databases with errors
* [Fix] Fix hash table lookup
* [Fix] Fix http message flag shift
* [Fix] Fix parsing of the from_hostname when it is an IP address
* [Fix] Fix parsing of the unquoted attributes in HTML
* [Fix] Fix passing of rspamd_text in selectors pipelines
* [Fix] Fix rubbish QP sequences decoding
* [Fix] Fix some complicated case with the closing tags parsing
* [Fix] Fix the case when l tag is too small
* [Fix] Html: Fix the case where only bgcolor is explicitly set
* [Fix] Libucl: Fix deletion from ucl objects
* [Fix] Namespace and add metadata for OpenMetrics, fix interleaving
* [Fix] Plug memory leak in http settings reload
* [Fix] Preserve SPF top record in the mempool variable
* [Fix] Remove aarch64 GC64 workaround
* [Fix] Remove bogus G_LIKELY
* [Fix] Spf: Do not parse non TXT DNS replies as TXT replies
* [Fix] Try to use on_connect/on_disconnect callbacks to handle internal Redis failures
* [Fix] buffer overflow in rspamc counters
* [Fix] fix static building
* [Fix] lua_scanners - message_min_words logic
* [Fix] src/lua/lua_mimepart.c: fix null dereference
* [Project] Add constant iterators
* [Project] Add helper library to handle mime strings in a more safe matter
* [Project] Add preliminary support of CDB bayes dump
* [Project] Add trim operations
* [Project] Allow mempool allocated mime strings
* [Project] Cdb: Finish backend implementation
* [Project] Cdb: Fix configuration load
* [Project] Cdb: Use shared data between cdb statfiles
* [Project] Cdb: continue statistics backend implementation
* [Project] Finish received headers rework part
* [Project] Move C++ specific declarations to C++ header
* [Project] Rework received headers parsing to C++
* [Project] Start using of the new received structure
* [Project] Start work on cdb backend
* [Rework] Further rework of the redis pool
* [Rework] Redis_pool: fix issues found
* [Rework] Rework learn and add classify condition
* [Rework] Save invisible content to a separate buffer
* [Rework] Start rewriting of the redis pool logic
* [Rules] Improve zero font rule
3.0: 19 Aug 2021
* [Conf] Align ARC scores with DKIM scores
* [CritFix] Neural: Fix sorting application
* [Feature] Add a simple dumper for bayes tokens
* [Feature] Add lua_maps.fill_config_maps function
* [Feature] Add preliminary exporter to AWS S3
* [Feature] Add preliminary restore bayes support
* [Feature] Add race condition protection against hs_helper restarts
* [Feature] Add rspamd_utf8_strcmp utility
* [Feature] Add zstd streaming API
* [Feature] Allow to log severity level explicitly
* [Feature] Allow to save and show attachment name when inserting AV scan results
* [Feature] Allow to sort urls for Lua
* [Feature] Allow to specify different timeouts/retransmits for fuzzy rules
* [Feature] Aws_s3: Allow to compress data stored
* [Feature] CMakeLists.txt: Change check and run-test to use rspamd-test-cxx * fixes #3807
* [Feature] Dmarc_report: allow sending reports in batches
* [Feature] Fuzzy_check: Allow to disable subject when making short text hash
* [Feature] Lua_cryptobox: Add keyed ssl hash functions via HMAC
* [Feature] Lua_task: Add get_urls_filtered method
* [Feature] Make monitored checks less frequent
* [Feature] Milter_headers: Add x-rspamd-pre-result header
* [Feature] Neural: Allow to balance FP/FN for the network
* [Feature] Ppopagate monitored errors from rbl module
* [Feature] Pyzor calculate score dynamically Count - WL-Count of default_score in percent
* [Feature] Rbl: Distinguish flattened and non-flattened selectors in RBL requests
* [Feature] Re-add pyzor support
* [Feature] Settings: add ip_map check and rework structure slightly
* [Feature] Spamassassin: Allow to set the default priority for SA scores
* [Feature] Strip smtp comments from message id
* [Feature] add SYSTEM_ZSTD cmake option To use the system zstd instead on the bundled version
* [Feature] external_relay plugin
* [Feature] rspamadm clickhouse neural_train subcommand
* [Fix] #3400 milter_headers: fix inverted logic for extended_headers_rcpt
* [Fix] ASN: fix _FAIL symbol for when main symbol is disabled
* [Fix] Add a special logic for text part with no text extraction
* [Fix] Add diacritics flag for several eu languages
* [Fix] Another FSM fix to accomodate possibility of multiple consequent ?
* [Fix] Avoid curse of dynamic array referencing
* [Fix] Avoid reinitialising neural settings
* [Fix] Check remain before processing TXT records
* [Fix] Enable error multiplier on http errors
* [Fix] Finally rework parsing entities logic
* [Fix] Fix '==' parsing in the content type attributes parser
* [Fix] Fix IPv6 expansion for SPF macros
* [Fix] Fix Mozilla Message-ID detection
* [Fix] Fix an edge case in BITCOIN_ADDR rule
* [Fix] Fix brain-damaged behaviour when http request has a custom Host header
* [Fix] Fix check of limits in email address parsing
* [Fix] Fix copy&paste error and rework
* [Fix] Fix expressions logic for and/or and float values
* [Fix] Fix fuzzy retransmits
* [Fix] Fix http maps with no or invalid expires data
* [Fix] Fix last quote character parsing in the content-type state machine
* [Fix] Fix normalisation flags propagation
* [Fix] Fix overflow when appending many broken tags
* [Fix] Fix parsing of rfc2047 tokens with '?' inside
* [Fix] Fix phishing flag set
* [Fix] Fix rfc2047 embedded into rfc2231 pieces in special headers
* [Fix] Fix round-robin rotation
* [Fix] Fix searching for symbols
* [Fix] Fix storing of the regexps inside variant
* [Fix] Fix tokenization near exceptions
* [Fix] Fix visibility calculations
* [Fix] Html: Attach inline tags to the structure
* [Fix] Html: Do not treat empty tags as block tags
* [Fix] Ical: Do not extract urls from all flags using merely specific ones
* [Fix] Initialise symcache even if it cannot be loaded properly
* [Fix] Lua_fuzzy: Remove text parts check when checking image dimensions
* [Fix] Lua_maps: Fix adjustments for the map type in the complex map definitions
* [Fix] Lua_task: Fix deleted symbols in has_symbol/get_symbol
* [Fix] Move metric and symcache link from validation to the init stage
* [Fix] Oletools: Another try to fix table sorting
* [Fix] One more default behaviour fix
* [Fix] Phishing: Rework urls processing
* [Fix] RBL: was missing some config schema
* [Fix] Replies: Fix 'Reply-To' handling in task:get_reply_sender
* [Fix] Rework metrics handling
* [Fix] Save symcache on exit
* [Fix] Selectors: Filter nil elements in lists
* [Fix] Selectors: Properly fix implicit tostring for nils
* [Fix] Try to fix some broken code in DMARC reporting plugin
* [Fix] Urls: Fix processing of html urls when it comes to the flags
* [Fix] Use proper buffer length
* [Fix] Various visibility fixes
* [Fix]: ASN: dns cb func should also return in case of an error
* [Project] Add a simple css rule definition
* [Project] Add css style skeleton
* [Project] Add css syntax (adopted from ebnf)
* [Project] Add css_selectors
* [Project] Add doctest unit testing library
* [Project] Add expected library
* [Project] Add fmt library for simple string ops
* [Project] Add fu2 library to better functions abstractions
* [Project] Add hashing method
* [Project] Add parsers skeleton
* [Project] Add preliminary support of vcard parser
* [Project] Add process exceptions for invisible text
* [Project] Add some methods for css parser
* [Project] Allow static libstdc++
* [Project] Another whitespace hack
* [Project] CSS: Various fixes in the declarations and values parsing
* [Project] Cpp: Add robin-hood hash map library
* [Project] Css: Add AST debug
* [Project] Css: Add colors conversion functions
* [Project] Css: Add dimensions handling
* [Project] Css: Add display value support
* [Project] Css: Add frozen library from https://github.com/serge-sans-paille/frozen/
* [Project] Css: Add opacity support
* [Project] Css: Add parser helpers to simplify debugging
* [Project] Css: Add preliminary stylesheet support
* [Project] Css: Add rules processing functions and tests
* [Project] Css: Add simple selectors unit tests
* [Project] Css: Add some c++ unit tests
* [Project] Css: Add some debug methods
* [Project] Css: Add some debug statements for the css parser
* [Project] Css: Add some logical skeleton for declarations parser
* [Project] Css: Add url/function tokens
* [Project] Css: Allow at rules parsing
* [Project] Css: Declarations parsing logic skeleton
* [Project] Css: Enable conditional css parsing support from the HTML parser
* [Project] Css: Finish generic lexer cases
* [Project] Css: Fix HSL conversion
* [Project] Css: Fix minus parsing
* [Project] Css: Fix parser consumers nesting
* [Project] Css: Fix parsing of the qualified rules
* [Project] Css: Fix rules merging
* [Project] Css: Further fixes to lexer
* [Project] Css: Further steps to parse css colors + rework
* [Project] Css: Further work on parser's methods
* [Project] Css: Implement backlog of css tokens
* [Project] Css: Implement numbers and ident parsers
* [Project] Css: Implement simple css selectors lookup
* [Project] Css: Implement styles merging
* [Project] Css: Make debug strings json like to simplify tests
* [Project] Css: Minor adjustments
* [Project] Css: More meat to the lexer
* [Project] Css: Move some of the tests to the doctest
* [Project] Css: Projected a parser
* [Project] Css: Properties attachment logic
* [Project] Css: Remove ragel from build targets (maybe keep for reference)
* [Project] Css: Rework css block structure
* [Project] Css: Rework flags of css properties
* [Project] Css: Rework tokens structure
* [Project] Css: Several fixes + tests
* [Project] Css: Simplify checks
* [Project] Css: Simplify debug code
* [Project] Css: Start css selectors parsing logic
* [Project] Css: Start semantic parsing for rules
* [Project] Css: Start stylesheet implementation
* [Project] Css: Tidy up lambdas
* [Project] Css: rework tokeniser
* [Project] Dmarc: Add dmarc report tool (WIP)
* [Project] Dmarc: Add munging configuration
* [Project] Dmarc: Add preliminary munging logic
* [Project] Dmarc: Fix header removal
* [Project] Dmarc: Fix munging logic
* [Project] Dmarc: Use full recipient address instead of a domain map
* [Project] Dmarc: Use zlists for dmarc reports
* [Project] Dmarc_report: Add message generation logic
* [Project] Dmarc_report: Add preliminary sending support
* [Project] Fix lua bindings
* [Project] Fix xml/sgml tags processing
* [Project] Handle new modification
* [Project] Html/CSS: Add transform from a CSS rule to html block
* [Project] Html/CSS: Link html and css styles
* [Project] Html/CSS: Switch styles parsing to css parser
* [Project] Html/Css: Fix some issues found
* [Project] Html/Css: Implement visibility rules for a block
* [Project] Html: Add more tests cases and fix some more corner issues
* [Project] Html: Add rows display type support
* [Project] Html: Allow decode entities function to normalise spaces + unit tests
* [Project] Html: Another rework of the tags structure
* [Project] Html: Another try to fix unbalanced cases
* [Project] Html: Fix crossing spans
* [Project] Html: Fix parent propagation
* [Project] Html: Further rework of the html parsing stuff
* [Project] Html: Implement logic for tags pairing
* [Project] Html: Implement rawtext state machine
* [Project] Html: Insert closing tags as well :(
* [Project] Html: More fixes
* [Project] Html: More fixes
* [Project] Html: More spaces logic fixes
* [Project] Html: One more attempt to write text content
* [Project] Html: Replace \0 in html content
* [Project] Html: Rework img/a tags handling
* [Project] Html: Rework propagation method
* [Project] Html: Rework tags placement
* [Project] Html: Rework transparency logic
* [Project] Html: Support 'hidden' attribute
* [Project] Html: Try another approach to append tags content
* [Project] Html: Try to deal with bad unknown tags properly
* [Project] Lua_aws: Add canonicalisation utility
* [Project] Lua_aws: Add function to produce AWS Authorisation header
* [Project] Lua_aws: Implement request signing
* [Project] Lua_mime: Add lua_mime.modify_headers routine
* [Project] Lua_task: Add modify_header method
* [Project] Lua_task: Allow to extract modified headers
* [Project] Make unescape code public for unit testing
* [Project] More fixes for closed tags
* [Project] More fixes to calculations
* [Project] Rework API for the modified headers
* [Project] Rework html visibility rule
* [Project] Skeleton of the css library
* [Project] Start headers modification API structure
* [Project] Start working on AWS Lua API
* [Project] Use lua_mime to modify headers
* [Project] Use modified headers on dkim signing
* [Project] Use string_view to constexpr variant unpacking
* [Rework] Add composites manager concept
* [Rework] Add tags definitions
* [Rework] Allow C code to be compiled with C++ compiler
* [Rework] Clickhouse: Store url flags
* [Rework] Composites: Rewrite the composites logic
* [Rework] Composites: Start rework of the composites framework
* [Rework] Dmarc: Move check policy function to the common utils
* [Rework] Dmarc: Rework reports keys structure
* [Rework] Further work to make html content private
* [Rework] Html/CSS: Remove css C bindings as they are useless now
* [Rework] Html/CSS: Rework Lua bindings
* [Rework] Html/Css: Start rework of the html blocks
* [Rework] Html: Add images processing logic
* [Rework] Html: Add traverse function
* [Rework] Html: Another steps to get rid of gnode
* [Rework] Html: Convert to variant
* [Rework] Html: Deal with the utf_content part
* [Rework] Html: Final rework part for the html processing code
* [Rework] Html: Fix Lua bindings
* [Rework] Html: Forgot to add the internal include
* [Rework] Html: Further html urls rework
* [Rework] Html: Further rework of the tags content extraction
* [Rework] Html: Make parameters as a vector again
* [Rework] Html: Move blocks part
* [Rework] Html: Move images processing stuff
* [Rework] Html: Rework lua bindings
* [Rework] Html: Start html text extraction rework
* [Rework] Html: Start refactoring of the html tags handling
* [Rework] Html: Start removing of GNode stuff
* [Rework] Html: Start rework of the html content structure
* [Rework] Lua_magic: Try to detect text parts with 8bit characters for non-utf8 encodings
* [Rework] Move HTML url functions and rework them
* [Rework] Move and adopt entities handling logic
* [Rework] Move common and rarely used dmarc code to the library
* [Rework] Move compression routines outside of rspamd_util library
* [Rework] Move entities/tags handling
* [Rework] Phishing: Split from redirectors usage
* [Rework] Redesign html blocks propagation logic
* [Rework] Remove tag name string
* [Rework] Rename phished url to a linked url
* [Rework] Reorganize dmarc plugin and remove unsupported reporting code
* [Rework] Reputation: Use more flexible types in get/set functions
* [Rework] Require proper C++ environment for Rspamd build
* [Rework] Rework extended urls output
* [Rework] Rework tags parsing machine
* [Rework] Slightly improve old regexp API
* [Rework] Start conversion of the redis pool code to c++
* [Rework] Try to resolve failed upstreams more agressively
* [Rework] Use C++ utf8 library with unit tests to trim whitespaces
* [Rework] Use C++ version for unicode normalisation
* [Rework] Use C++ version of the lua threads pool
* [Rules] Add raw addresses to MULTIPLE_FROM options
* [Rules] Another fix to HTTP_TO_HTTPS rule
* [Rules] Do not trigger HTML_SHORT_LINK_IMG on external images
* [Rules] Extend FORGED_X_MAILER
* [Rules] Extend OLD_X_MAILER
* [Rules] Fix CTYPE_MIXED_BOGUS for text attachments
* [Rules] Fix FPs for CTYPE_MIXED_BOGUS
* [Rules] Fix HTTP_TO_HTTPS rule
* [Rules] Fix HTTP_TO_HTTPS rule
* [Rules] Fix zerofont rule (partially)
* [Rules] Micro-optimize X_PHP_EVAL
* [Rules] Reduce default weight for R_MISSING_CHARSET
2.7: 08 Jan 2021
* [Conf] Add R_DKIM_PERMFAIL to the metric
* [CritFix] Dkim: Fix simple canonicalisation if multiple signatures are presented
* [CritFix] Fix controller paths normalisation
* [Feature] Add INVALID_DATE rule
* [Feature] Add controller endpoint for training neural
* [Feature] Add sanity checks for actions thresholds
* [Feature] Add support of '==' and '!=' in Rspamd expressions
* [Feature] Composites: Improve composite atoms parser
* [Feature] Docker: use Debian slim variant
* [Feature] Elastic: Add some missing fields
* [Feature] Extract text from img alt attributes
* [Feature] Improve charset detection logic
* [Feature] Lua_clickhouse: Add optional row callback for large selections
* [Feature] Lua_dns_resolver: Add idna_convert_utf8 method
* [Feature] Lua_mime: Add ability to do multipattern replacement
* [Feature] Lua_trie: Allow to report start of the match
* [Feature] Multimap: support adding map values as extra options
* [Feature] Neural: Move PCA learning to a subprocess
* [Feature] RBL: support matching content/image URLs only
* [Feature] RBL: support use of multiple selectors
* [Feature] Reputation: Allow to specify ip masks
* [Feature] Support SMIME signed messages container
* [Feature] Support multiple conditions for symbols
* [Feature] Support ping in milter mode
* [Feature] Support rspamd_text in selector regexps
* [Feature] Use own daemonization routine
* [Feature] Vadesecure: Implement settings_outbound feature as recommended by Vade
* [Feature] `rspamadm clickhouse` command
* [Feature] allow hyperscan for aarch64
* [Fix] Allow to set priorities between post init scripts
* [Fix] Allow to use maps for strings that are not zero terminated
* [Fix] Apply max_lua_urls limit for emails as well
* [Fix] Arc: Fix CV check on signing
* [Fix] Arc: Fix signing of the broken ARC chains
* [Fix] Clickhouse: escape carriage return
* [Fix] Composites: Allow partial match
* [Fix] Deduct type of a table methods
* [Fix] Do not load errored hyperscan database
* [Fix] Do not process links in ignored html tags
* [Fix] Fix ClamAV result for cached encrypted file (#3395)
* [Fix] Fix canonicalisation when l= tag is presented
* [Fix] Fix flag shift
* [Fix] Fix handling of skip/skip_process http flags
* [Fix] Fix html attachments checks
* [Fix] Fix issue with pushing binary formats to Lua strings
* [Fix] Fix logging for rspamadm
* [Fix] Fix off-by-one with init check
* [Fix] Fix parsing of escape characters in quoted pairs
* [Fix] Fix pushing ucl strings with \0 inside
* [Fix] Fix quoted-printable soft newlines bugged case
* [Fix] Fix settings in case actions are set to null (#3415)
* [Fix] Fix several issues with auth results producing
* [Fix] Fix smtp comments exclusion
* [Fix] Fix smtp date syntax definition
* [Fix] Fix substring search in case if srchlen == inlen
* [Fix] Fix text selectors
* [Fix] Honour `systemd` setting when logging to console (#3514)
* [Fix] Html: Add entities collisions prevention logic (e.g. for mathml entities)
* [Fix] Lua_auth_results: Quote potentially bad values in AR header
* [Fix] Multimap: Fix flags usage
* [Fix] Multimap: Fix scoring for combined maps
* [Fix] Plug GList * leak in redis pool
* [Fix] RBL: allow for multiple matches of the same label if types are different
* [Fix] Rely on libev checks for file maps
* [Fix] Restore simple dkim canonicalisation mode
* [Fix] Return MimeCharset as we work with emails...
* [Fix] Spamassassin: Fix pcre_only flags
* [Fix] Spamassassin: Preserve 'pcre_only' flag when dealing with regexp replacements
* [Fix] Try to fix GError leak
* [Fix] Try to fix a mess with settings loading by adding priorities
* [Fix] Try to move setings initialisation to a later stage
* [Fix] Use dup fd in milter handler to avoid races with the proxy
* [Fix] Use message pointer to avoid obsolete data to be cached
* [Project] Rbl: Migrate to `checks`
* [Project] Rbl: Move config code outside of the plugin
* [Project] Ressurect empty prefilters as connection filters
* [Project] Support connection filters registration from Lua
* [Rework] Add final cleanup logic
* [Rework] Add preliminary support of hyperscan caching for re maps
* [Rework] Add stale cache removal
* [Rework] Clickhouse: Improve performance
* [Rework] Distinguish between strict config test mode
* [Rework] Furhter logging improvements
* [Rework] Milter_headers: improve extended_headers_rcpt support
* [Rework] Move parsers to a separate lua library
* [Rework] Neural: Skip composite symbols
* [Rework] Rbl: Rework defaults logic
* [Rework] Some tunes to cache saving
* [Rework] Track maps origins
* [Rework] Use full crypto hash for regexp maps
* [Rules] Remove broken rule
2.6: 30 Sep 2020
* [Conf] Add missing symbols
* [Conf] Add missing symbols
* [Conf] Fix fat-fingers typo
* [Conf] Fix wrong comment in options.inc
* [Conf] Neural: Fix the default name for max_trains
* [Conf] Register a known symbol
* [Conf] Spf: Add R_SPF_PERMFAIL symbol
* [CritFix] Arc: Fix ARC validation for chains of signatures
* [CritFix] Distinguish socketpairs between different fuzzy workers
* [CritFix] Fix IDNA dots parsing
* [CritFix] Fix test assertion method
* [CritFix] Fix usage of crypto_sign it should be crypto_sign_detached!
* [Feature] Add BOUNCE rule
* [Feature] Add controller plugins support and selectors plugin
* [Feature] Add maps query method
* [Feature] Add minimal delay to fuzzy storage
* [Feature] Add multiple base32 alphabets for decoding
* [Feature] Add preliminary support of BCH addresses
* [Feature] Add query_specific endpoint
* [Feature] Allow multiple base32 encodings in Lua API
* [Feature] Allow to specify nonces manually
* [Feature] Controller: Allow to pass query arguments to the lua webui plugins
* [Feature] Fuzzy_check: Add gen_hashes command
* [Feature] Fuzzy_check: Add weight_threshold option for fuzzy rules
* [Feature] Implement address retry on connection failure
* [Feature] Improve limits in pdf scanning
* [Feature] Initial support of subscribe command in lua_redis
* [Feature] Lua_cryptobox: Add secretbox API
* [Feature] Lua_text: Add encoding methods
* [Feature] Milter_headers: Allow to activate routines via users settings
* [Feature] PDF: Add timeouts for expensive operations
* [Feature] Preliminary maps addon for controller
* [Feature] Split pdf processing object and output object to allow GC
* [Feature] Support BLIS blas library
* [Feature] Support input vectorisation by recvmmsg call
* [Feature] Support multiple base32 alphabets
* [Feature] add queueid, uid, messageid and specific symbols to selectors [Minor] use only selectors to fill vars in force_actions message
* [Feature] allow variables in force_actions messages
* [Feature] extend lua api
* [Fix] #3249
* [Fix] Allow to adjust neurons in the hidden layer
* [Fix] Another try to fix email names parsing
* [Fix] Arc: Allow to reuse authentication results when doing multi-stage signing
* [Fix] Arc: Fix bug with arc chains verification where i>1
* [Fix] Arc: Sort headers by their i= value
* [Fix] Change neural plugin's loss function
* [Fix] Deal with double eqsigns when decoding headers
* [Fix] Default ANN names in clickhouse
* [Fix] Disable reuseport for TCP sockets as it causes too many troubles
* [Fix] Disable text detection heuristics for encrypted parts
* [Fix] Distinguish DKIM keys by md5
* [Fix] Distinguish type from flags in register_symbol
* [Fix] Dmarc: Unbreak reporting after cf2ae3292ac93da8b6e0624b48a62828a51803c9
* [Fix] Do not flag pre-result of virus scanners as least if action is reject
* [Fix] Do not use GC64 workaround on 32bit platforms, omg
* [Fix] Exclude damaged urls from html parser
* [Fix] Fix FREEMAIL_REPLYTO_NEQ_FROM_DOM
* [Fix] Fix FROM_NEQ_ENVFROM
* [Fix] Fix FWD_GOOGLE rule (#1815)
* [Fix] Fix adding of the empty archive file for gzip
* [Fix] Fix aliases in forged recipients and limit number of iterations
* [Fix] Fix authentication results insertion
* [Fix] Fix calling of methods in selectors
* [Fix] Fix clen length for hiredis...
* [Fix] Fix endless loop if broken arc chain has been found
* [Fix] Fix false - operation
* [Fix] Fix get_urls table invocation
* [Fix] Fix group based composites
* [Fix] Fix headers passing in rspamd_proxy
* [Fix] Fix incomplete utf8 sequences handling
* [Fix] Fix lua_next invocation
* [Fix] Fix lua_parse_symbol_type function logic
* [Fix] Fix multiple listen configuration
* [Fix] Fix occasional encryption of the cached data
* [Fix] Fix parsing boundaries with spaces
* [Fix] Fix passing of methods arguments
* [Fix] Fix poor man allocator algorithm
* [Fix] Fix regexp selector and add flattening
* [Fix] Fix rfc base32 encode ordering (skip inverse bits)
* [Fix] Fix rfc based base32 decoding
* [Fix] Fix sockets leak in the client
* [Fix] Fix storing of the original smtp from
* [Fix] Fix types check and types usage in lua_cryptobox
* [Fix] Fix unused results
* [Fix] Fuzzy_check: Disable shingles for short texts (really)
* [Fix] Ical: Fix identation grammar
* [Fix] Improve part:is_attachment logic
* [Fix] Mmap return value must be checked versus MAP_FAILED
* [Fix] One more fix to skip images that are not urls
* [Fix] Pdf: Support some weird objects with no newline before endobj
* [Fix] Rbl: Fix ignore_defaults in conjunction with ignore_whitelists
* [Fix] Restore support for `for` and `id` parts in received headers
* [Fix] Segmentation fault in contrib/lua-lpeg/lpvm.c on ppc64el
* [Fix] Skip spaces at the boundary end
* [Fix] Slashing fix: fix captures matching API
* [Fix] Spamassassin: Rework metas processing
* [Fix] Store reference of upstream list in upstreams objects
* [Fix] Understand utf8 in content-disposition parser
* [Fix] Unify selectors digest functions
* [Fix] Use `abs` value when checking composites
* [Fix] Use strict IDNA for utf8 DNS names + add sanity checks for DNS names
* [Fix] Use unsigned char and better support of utf8 in ragel parser
* [Fix] add missing selector_cache declaration
* [Project] Add `L` flag for regexps to save start of the match in Hyperscan
* [Project] Add `lower` method to lua_text
* [Project] Add a simple matrix Lua library
* [Project] Add implicit bitcoincash prefix
* [Project] Add linalg ffi library for prototyping
* [Project] Add methods to append data to fuzzy requests
* [Project] Add routine to call a generic lua function
* [Project] Add ssyev method interface
* [Project] Add tensors index method
* [Project] Add text:sub method
* [Project] Allow rspamd_text based selectors
* [Project] Allow to specify re_conditions for regular expressions
* [Project] Attach extensions to the binary fuzzy commands
* [Project] Bitcoin: BTC cash addresses needs some checksum validation
* [Project] Cleanup the redis script
* [Project] Convert bitcoin rules to the new regexp conditions feature
* [Project] Detect memrchr in systems that supports it
* [Project] Do not listen sockets in the main process
* [Project] Implement 'probabilistic' learn mode for ANN
* [Project] Implement BTC polymod in C as it requires 64 bit ops
* [Project] Implement bitcoin cash validation in a proper way
* [Project] Implement extensions logic for fuzzy storage
* [Project] Implement symbols insertion in multiple results mode
* [Project] Lua_text: Add method memchr
* [Project] Neural: Add PCA loading logic
* [Project] Neural: Fix PCA based learning
* [Project] Neural: Fix matrix gemm
* [Project] Neural: Further PCA fixes
* [Project] Neural: Implement PCA in learning
* [Project] Neural: Implement PCA learning
* [Project] Neural: Implement PCA on ANN forward
* [Project] Neural: Implement PCA serialisation
* [Project] Neural: Start PCA implementation
* [Project] Neural: Use C version of scatter matrix producing
* [Project] Preliminary support of lua conditions for regexps
* [Project] Preliminary usage of the reuseport
* [Project] Process composites separately for each shadow result
* [Project] Remove old code
* [Project] Rework scan result functions to support shadow results
* [Project] Rework some more functions to work with shadow results
* [Project] Some more fixes
* [Project] Start results chain implementation
* [Project] Support fun iterators on rspamd_text objects
* [Project] Support multiply, minus and divide operators in expressions
* [Project] Tensor: Move scatter matrix calculation to C
* [Rework] Allow to specify exat metric result when adding a symbol
* [Rework] Change and improve openblas detection and usage
* [Rework] Close listen sockets in main after fork
* [Rework] Further rework of lua urls extraction API
* [Rework] Lua_cryptobox: Allow to store output of the hash function
* [Rework] Lua_task: Add more methods to deal with shadow results
* [Rework] Modernize logging for expressions
* [Rework] Remove empty prefilters feature - we are not prepared...
* [Rework] Remove old FindLua module, disable lua fallback when LuaJIT is enabled
* [Rework] Rework and refactor forged recipients plugin
* [Rework] Rework expressions processing
* [Rework] Rework fuzzy commands processing
* [Rework] Rework url flags handling API
* [Rework] Rework urls extraction
* [Rework] Split operations processing and add more debug logs
* [Rework] Update zstd to 1.4.5
* [Rework] Use google-ced instead of libicu chardet as the former sucks
* [Rework] add alias util:parse_addr for util:parse_mail_address
* [Rework] get rid of util:parse_addr duplicating the util:parse_mail_address, replace where used
* [Rules] Allow prefix for bitcoin cash addresses
* [Rules] More fixes for bitcoin cash addresses decoding
* [Rules] Refactor bleach32 addresses handling
2.5: 01 Apr 2020
* [Conf] Mark Rspamd emailbl as ignore whitelist
* [Conf] RBL: Add missing emails = true option
* [Feature] Add support for scripts in fuzzy storage
* [Feature] Arc: Add whitelisted_signers_map option
* [Feature] Implement hosts file processing
* [Feature] Neural: Introduce classes bias that allows non-equal classes learning
* [Feature] Update libev to 4.33
* [Fix] Another brain damage html standard adoptions
* [Fix] Another fix for brain damaged obs-fws state
* [Fix] Fix flags that caused force_actions failure
* [Fix] Fix logging issue
* [Fix] Fix lua symbols scores registration when config does not define scores
* [Fix] Fix opaque maps logic
* [Fix] Fix parsing of the html tags with no spaces after attributes
* [Fix] Fix some corner cases in urls parsing, add limits
* [Fix] Fix tlds extraction if custom composition rules are used
* [Fix] Fix variables replacement in mempool
* [Fix] Improve base64 detection
* [Fix] Normalize dynamic scores in ANN correctly
* [Fix] Plug memory leak introduced by #3153
* [Fix] Stat_redis_backend: Fix memory leak and simplify learn path
* [Fix] Try hard to deal with ghost workers
* [Fix] metadata_exporter default formatter
* [Rework] Change the way to extract URLs when dealing with alternative parts
* [Rework] Fix various url extraction issues
* [Rework] Re cache: Load compiled hyperscan in the main process as well
* [Rework] Re cache: Load hyperscan early
* [Rework] Rework URL structure: adjust tld part
* [Rework] Rework URL structure: host field
* [Rework] Rework URL structure: more structure optimisations
* [Rework] Rework URL structure: user field
* [Rework] URL: Another update for urls extraction logic
* [Rework] Urls: Improve query urls handling
* [Rework] Urls: adopt html related stuff
* [Rework] Urls: more rework of the urls sets
* [Rework] Urls: process query urls in HTML urls correctly
* [Rework] Urls: rework urls hash structure
* [Rework] Urls: update lua libraries
* [Rework] Use multiple search tries for different url extraction types
2.4: 26 Feb 2020
* [CritFix] Fix parsing of the content type attributes
* [Feature] Clickhouse: Add extra columns support
* [Feature] Rbl: Add url_compose_map option for RBL rules
* [Fix] 'R' flag is for all headers regexp
* [Fix] Allow to reset settings id from Lua (e.g. because of the priority)
* [Fix] Avoid collisions in mempool variables by changing fuzzy caching logic
* [Fix] Avoid strdup usage for symbols options
* [Fix] Do not trust stat(2) it lies
* [Fix] Filter all options for symbols to have sane characters
* [Fix] Fix all headers iteration
* [Fix] Fix allowed_settings for neural
* [Fix] Fix listen socket parsing
* [Fix] Fix maps expressions evaluation
* [Fix] Fix sentinel connections leak by using async connections
* [Fix] Fix smtp message on passthrough result
* [Fix] Fix tld compositon rules
* [Fix] Fuzzy_storage: Do not check for shingles if a direct hash has been found
* [Fix] Lua_mime: Do not perform QP encoding for 7bit parts
* [Fix] Neural: Distinguish missing symbols from symbols with low scores
* [Fix] Support listening on systemd sockets by name
* [Project] Add lua_urls_compose library
* [Project] Allow to set a custom log function to the logger
* [Project] CDB maps: Start making cdb a first class citizen
* [Project] Clickhouse: Add extra columns concept
* [Project] Fix urls composition rules, add unit tests
* [Project] Unify cdb maps
* [Rework] Logger infrastructure rework
* [Rework] Refactor libraries structure
* [Rework] Rework SSL caching
* [Rework] Update snowball stemmer to 2.0 and remove all crap aside of UTF8
2.3: 04 Feb 2020
* [Conf] SPF is no longer a C module
* [Conf] Update spamtrap map path example
* [CritFix] Fix html entities decoding
* [CritFix] Fix re cache when mix of pcre and hyperscan is used
* [Feature] Allow milter code to deal with multiple headers
* [Feature] Antivirus: Add avast support
* [Feature] Dkim_signing: Allow to sign via milter_headers
* [Feature] Implement content hashes
* [Feature] Lua_text: Add regexp split iterator method
* [Feature] Lua_text: Implement flattening of the input tables
* [Feature] Send quit command to Redis
* [Feature] Speed up is_ascii function
* [Feature] Spf: Add external_relay option
* [Fix] Avoid double escaping
* [Fix] Fix O(N^2) algorithm
* [Fix] Fix arc seal validation
* [Fix] Fix base tag processing according to stupid HTML renderer behaviour
* [Fix] Fix dealing with `\0` in ucl strings and JSON
* [Fix] Fix gpg parts misdetection
* [Fix] Fix ignored symbols exporting
* [Fix] Fix processing of numeric url's
* [Fix] Fix processing of the closed tcp connections
* [Fix] Fix regexp type check for pcre2
* [Fix] Fix urls encode function
* [Fix] Fix urls shifting when doing decode to include separators
* [Fix] Fix white on white rule and add is_leaf flag
* [Fix] Further fixes in charset detection
* [Fix] Ignore diacritics in chartable module for specific languages
* [Fix] Limit size of symbols options by max_opts_len option
* [Fix] More fixes in html tag content calculations
* [Fix] Plug memory leak in fuzzy storage
* [Fix] Process high priority settings even if settings/id has been specified
* [Fix] Select a different upstream on last retransmit
* [Fix] Treat soft hyphen as zero width space
* [Fix] Try harder to watch the lifetime of the key_stat
* [Fix] Use ipv6-mapped-ipv4 addresses in radix trie
* [Project] Add logic to break execution when processing symbols*
* [Project] Add methods to set specific content for mime parts from Lua
* [Project] Lua_content: support PDF files
* [Project] Move dns_tool to using of the rspamd_spf from FFI module
* [Project] Preliminary SPF plugin in Lua
* [Project] Show debug stat for memory pool
* [Project] Some rework about specific data that is now tagged
* [Project] Start reworking of the mempool structure
* [Rework] Allow to add userdata as symbols options
* [Rework] Change mime part specifics handling
* [Rework] Move LRU SPF cache from spf plugin
* [Rework] Rework HTML tags content attachment
* [Rework] Rework options hash structure
* [Rework] Start lua_content library
* [Rework] Stop using of uthash for http headers
* [Rework] Use faster hashing approach for memory pools variables
* [Rules] Add PDF related rules
2.2: 19 Nov 2019
* [Conf] Antivirus: Fix the default config
* [Feature] Add verdict library in lua
* [Feature] Allow exception when choosing upstream
* [Feature] Allow to disable symbols from the metric config
* [Feature] Allow to limit maps per specific worker
* [Feature] Always validate Rspamd protocol output
* [Feature] Antivirus: Add preliminary virustotal support
* [Feature] Clickhouse: Rework Clickhouse collection logic
* [Feature] Improve base64 usage
* [Feature] Shutdown timeout is now associated with task timeout
* [Fix] #3129 Multiple classifiers on redis working incorrectly
* [Fix] Allow real upstreams configuration
* [Fix] Another try to fix slow callbacks and timers
* [Fix] Check results of write message as SSL can bork them
* [Fix] Clickhouse: Avoid potential races in collection
* [Fix] Clickhouse: Fix periodic script
* [Fix] Fail DNS upstream on each retransmit attempt
* [Fix] Fix consistent hashing when upstreams are marked inactive
* [Fix] Fix issues found
* [Fix] Fix off-by-one in retries for the proxy
* [Fix] Fix termination
* [Fix] Fix upstreams exclusion logic
* [Fix] Fix utf8 validation for symbols options and empty strings
* [Fix] Oops, fix maps reload
* [Fix] Rbl: Allow utf8 lookups for IDN domains
* [Fix] Sigh, another try to fix brain-damaged openssl
* [Project] Add fast utf8 validation library
* [Project] Use own utf8 validation instead of glib
* [Rework] Another phase of finish actions rework
* [Rework] Further cmake system rework
* [Rework] Further isolation of the controller's functions
* [Rework] Make cmake structure more modular
* [Rework] Move cmake modules to a dedicated path
* [Rework] Replace controller functions by any scanner worker if needed
* [Rework] Rework final scripts logic
* [Rework] Rewrite rspamd_str_make_utf_valid function
2.1: 28 Oct 2019
* [Conf] Update neural.conf
* [CritFix] Fix dkim verification for multiple headers listed
* [Feature] Add support of uudecode
* [Feature] Allow to explicitly set events backend
* [Feature] Implement configurable limits for SPF lookups
* [Feature] Lua_scanners: Use lua magic for inclusion/exclusion logic
* [Feature] Multimap: Do not check files in office archives
* [Feature] Neural: Add sampling when storing training vectors
* [Feature] SPF: Allow to disable AAAA checks in configuration
* [Feature] Spf: Add limits configuration support
* [Feature] Store etag in cached HTTP maps + better logging
* [Feature] Support segwit BTC addresses, fix LTC verification
* [Feature] Support uuencoding
* [Fix] Add configurable number of threads for OpenBLAS
* [Fix] Add workaround for ragel 7 in hyperscan related maps code
* [Fix] Another fix for numeric urls parsing
* [Fix] Correct EMA time calculations
* [Fix] Do not treat archives as text
* [Fix] Do not use strdup on data extracted from lua
* [Fix] Fix a failure calcuating URL reputation.
* [Fix] Fix crash due to constructors init order
* [Fix] Fix crash on parts with no cd
* [Fix] Fix empty prefilters that require mime structures
* [Fix] Fix event loop creation
* [Fix] Fix issues sending DMARC reports.
* [Fix] Fix misprint
* [Fix] Fix saving of the file maps
* [Fix] Fix size calculations when converting from utf16
* [Fix] Fix support of disable_monitoring in rbl
* [Fix] Fix use-after-free
* [Fix] Fix zip files check to relax requirements
* [Fix] Important hiredis fixes
* [Fix] Lot's of fixes in maps check logic
* [Fix] Lua_tcp: Deal with temporary fails on write
* [Fix] Lua_tcp: Make write errors fatal and rework error handlers
* [Fix] Meta: Filter some more values
* [Fix] Neural: Add protection agains infinities
* [Fix] Oops, fix math.huge invocation
* [Fix] Plug memory leak
* [Fix] Sigh, another email to string fix
* [Fix] Try to fix another ownership race in ssl connection
* [Fix] Uuencode: Fix parsing of corrupted uuencode
* [Fix] lua_scanners - razor rename need_check function
* [Rework] Require CMake 3.9 to work, remove manual lto crap
2.0: 11 Oct 2019
* [Conf] Add BROKEN_HEADERS_MAILLIST composite
* [Conf] Add path to greylist-whitelist-domains.inc
* [Conf] Clarify documentation in the config files
* [Conf] Introduce maps.d directories
* [Conf] Log settings id by default
* [Conf] Make LEAKED_PASSWORD_SCAM a composite rule again
* [Conf] Move all surbl/emails rules to rbl
* [Conf] Register new Spamhaus codes
* [Conf] Remove configs for deleted modules
* [Conf] Remove surbl parts, fix hash_format attribute
* [Conf] Show autolearn sample
* [Conf] Slashing: Change default stats backend to Redis
* [Conf] Surbl: Utilise new `check_emails` option
* [Conf] Update header
* [Conf] Use multi-prefixes RBLs in the default config
* [CritFix] Deal with case-sensivity in Content-Disposition parser
* [CritFix] Eliminate old endpoint
* [CritFix] Fix case sensivity when parsing Content-Type
* [CritFix] Fix loading of DKIM public keys
* [CritFix] Fix procesing of urls
* [CritFix] Fix whitelisting when both spf and dkim are required to be valid
* [CritFix] Langdet: Fix language detection where no stop words found
* [Feature] Add description to the groups
* [Feature] Add limit for number of URLs in Lua
* [Feature] Add logging of groups to the log_format
* [Feature] Add lua_smtp library
* [Feature] Add maps cache and type refinement
* [Feature] Add p0f scanner
* [Feature] Adopt emails module to use lua_maps
* [Feature] Allow options matching in composites
* [Feature] Allow selectors in rbl module
* [Feature] Allow to output group results
* [Feature] Asn: Allow to use bgpdump when NET::MRT is broken
* [Feature] Calculate tokens occurrences distribution
* [Feature] Clickhouse: Add authenticated user and settings id columns
* [Feature] Clickhouse: Store groups data
* [Feature] Clickhouse: Utilise LowCardinality feature
* [Feature] Implement Redis prefixes registration logic
* [Feature] Implement settings id propagation between deps
* [Feature] Improve AV results caching
* [Feature] Improve autolearning
* [Feature] Improve logging locking logic (remove it actually)
* [Feature] Improve settings processing
* [Feature] Langdet: Limit number of stop words to be checked
* [Feature] Libucl: Allow to sort keys in ucl objects
* [Feature] Lua_config: Extend get symbols method
* [Feature] Lua_maps: Allow static maps for key-value pairs
* [Feature] Lua_mimepart: Add function filter_words
* [Feature] Lua_selectors: Add `words` selector
* [Feature] Lua_selectors: Add sort and uniq transform functions
* [Feature] Lua_selectors: Allow table arguments for selectors
* [Feature] Lua_tcp: Add preliminary support of SSL connections
* [Feature] Lua_trie: More flexible API
* [Feature] Lua_util: Add filter_specific_url function
* [Feature] Lua_util: table_digest can now recursively traverse tables
* [Feature] Maillist: Improve detection
* [Feature] Maps: Allow caching for complex maps
* [Feature] Monitored: Support random lookups
* [Feature] Multimap: Add combined maps prototype
* [Feature] Multimap: Add dependend maps via redis keys selectors
* [Feature] Multimap: Allow multiple email addresses matches
* [Feature] Multimap: Also check detected charset when do filename checks
* [Feature] Output number of messages processed to proctitle
* [Feature] Perform clean SSL shutdown
* [Feature] Performance: Do not use base64 SIMD version for bad inputs
* [Feature] RBL: Support bit results in replies
* [Feature] RBL: Support type specific prefixes
* [Feature] Ratelimit: Consider number of SMTP recipients
* [Feature] Rbl: Add ability to check urls
* [Feature] Rbl: Add resolve_ip based RBLs
* [Feature] Rbl: Make config checks much more strict
* [Feature] Rbl: Support per-rule whitelists
* [Feature] Rbl: Support process script
* [Feature] Rbl: Support replyto addresses
* [Feature] SURBL: Allow to check email domains
* [Feature] Selectors: Add `list` generator
* [Feature] Selectors: Add `specific_urls` extractor
* [Feature] Selectors: Add flatten function
* [Feature] Selectors: Support filter_map and apply_map functions
* [Feature] Store Clickhouse data outside of lua alloc
* [Feature] Support caching for encrypted files and macros
* [Feature] Support images when extracting urls
* [Feature] Support more hyperscan flags
* [Feature] Support protocol flags
* [Feature] URL: Apply stringprep to hostnames to filter garbage
* [Feature] Upstreams: Add lazy resolving logic to all upstreams
* [Feature] Upstreams: Set noresolve flag on numeric upstreams
* [Feature] Use `scores` in apply section
* [Feature] Use maps logic from lua_maps for multimap
* [Feature] Use random monitored in rbl module
* [Feature] lua_scanners - add Razor support
* [Fix] Add another safe-guard in urls processing
* [Fix] Add debug to ssl, fixed write hangs
* [Fix] Add missing groups to C callback symbols
* [Fix] Add more checks for ghosts symbols
* [Fix] Allow to enable or add new actions via settings
* [Fix] Allow to set 0 size for spf/dkim caches
* [Fix] Another bunch of fixes towards protocol mess
* [Fix] Another fix to deal with bad URLs
* [Fix] Arc: Another bunch of fixes for arc signing
* [Fix] Arc: More arc signing fixes
* [Fix] Avoid another overflow in fpconv
* [Fix] Clickhouse: Fix quoting
* [Fix] Clickhouse: Fix retention query quoting
* [Fix] Distinguish empty and non-empty prefilters
* [Fix] Distinguish remote and local addrs parsing
* [Fix] Do not assert if length of sig is bad, just fail verification
* [Fix] Do not assert if we have broken mime boundary in the headers
* [Fix] Do not call implicit strlen to avoid issues
* [Fix] Do not count images urls when checking url regexps for compatibility
* [Fix] Do not output rbl suffix in symbol option
* [Fix] Do not use config pool to avoid issues with double reload
* [Fix] Do not use ephemeral string
* [Fix] Do not use lightuserdata for traceback
* [Fix] Do not use priority in metric registration
* [Fix] Emails: Check email sanity before testing on BL
* [Fix] Emails: Fix misprint in key name
* [Fix] Escape utf in regexp to dodge ragel/hyperscan issue
* [Fix] Extend task_timeout to postfilters stage
* [Fix] Fix ARC signing after fixing another bug in it...
* [Fix] Fix AV scan logic
* [Fix] Fix DMARC_NA behaviour in case of no valid policies
* [Fix] Fix LRU hash iteration logic
* [Fix] Fix alignment mess
* [Fix] Fix configuring symbols without scores
* [Fix] Fix disabling of the actions
* [Fix] Fix dkim signing exceptions
* [Fix] Fix embedded images linking logic
* [Fix] Fix events leak
* [Fix] Fix eviction corner case
* [Fix] Fix fuzzy image score calculation #2962
* [Fix] Fix hang in fuzzy_learn when explicit rotation is set
* [Fix] Fix headers propagation logic
* [Fix] Fix hearbeats restart issue
* [Fix] Fix history reset
* [Fix] Fix log parameter
* [Fix] Fix lua_ip_equal logic
* [Fix] Fix more issues with nested messages + tests
* [Fix] Fix normalization of non-alphabet based languages
* [Fix] Fix offsets when parsing message/rfc822 in multipart
* [Fix] Fix options in rbl symbols
* [Fix] Fix out of bound access in lua logger
* [Fix] Fix out-of-bound read in qp decode
* [Fix] Fix parent CTE propagation
* [Fix] Fix parsing of the received headers with empty part
* [Fix] Fix pending checks for events
* [Fix] Fix printing of NULL pointer with fixed length
* [Fix] Fix race condition in watcher handler
* [Fix] Fix read-after-end in quoted printable decoding
* [Fix] Fix redis sentinel support
* [Fix] Fix registry leak in case of DNS errors
* [Fix] Fix reload logic
* [Fix] Fix sending of large entries via HTTPS
* [Fix] Fix settings reload
* [Fix] Fix some more corner cases for fpconv
* [Fix] Fix trie code when there are regexps and Hyperscan is absent
* [Fix] Further fixes to printing of the FP numbers
* [Fix] Fuzzy_check: Fix timeouts
* [Fix] Grrr, fix empty ip case
* [Fix] Html: Fix processing of fjlig entity
* [Fix] Lang_det: Try better to distinguish Chinese and Japanese
* [Fix] Lua_mime: Fix reversed extensions map
* [Fix] Lua_task: Fix message-less API
* [Fix] Lua_tcp: Report connection failures
* [Fix] Lua_tcp: Various fixes and debugging improvements
* [Fix] Metadata_exporter: This plugin is idempotent not a postfilter
* [Fix] More fixes to extract_specific_urls
* [Fix] More stages fixes
* [Fix] Neural: Another bunch of fixes
* [Fix] Neural: use version in ANN key profile
* [Fix] Postpone lua state destruction to allow lua dtors to be used
* [Fix] Prefer surbl/emails rule on rbl to preserve compatibility
* [Fix] RBL: Fix behaviour of emails_domainonly
* [Fix] Ratelimit: Fix dynamic score
* [Fix] Rbl: Fix emailbl functions
* [Fix] Really fix hyperscan workaround
* [Fix] Set sanity limits for pcre2
* [Fix] Settings: Fix settings check flags
* [Fix] Sort keys when getting data from Lua when filling rules
* [Fix] Statistics: Do not query Redis tokens when there are no learns
* [Fix] Stop IO event on write finished in http connection
* [Fix] Use heuristically detected text parts data
* [Fix] Various fixes to QP encoding algorithm
* [Fix] Various fixes to SSL state machine handler
* [Fix] Various fixes to asn module
* [Fix] Workaround for empty charset in rfc2231 encoding
* [Project] Switch from torch to KANN
* [Project] Add heartbeat events
* [Project] Add preliminary support of the Kaspersky Scan Engine
* [Project] Add preliminary version of maps expressions
* [Project] Add preprocessed settings to the config structure
* [Project] Add simple forward propagation function
* [Project] Add small helpers for migration simplifications
* [Project] Allow to replace body in milter
* [Project] Bundle libev
* [Project] First refactoring step libevent->libev
* [Project] Implement syntax highlighting for Lua
* [Project] Lua_magic: Adopt lua_magic stuff in mime_types
* [Project] Remove libfann, gd and other unsupported stuff
* [Project] Remove torch
* [Project] Rework upstreams
* [Rework] Allow execution of async events when hs compiles regexps
* [Rework] Bayes expiry: eliminate `default` expiration mode
* [Rework] Dkim: Remove signing code
* [Rework] Dkim_signing: Move sign condition to dkim_signing
* [Rework] Do not lowercase all data send to ClickHouse
* [Rework] Drop url tags
* [Rework] Eliminate lua_squeeze as it has shown no improvements
* [Rework] Eliminate virtual scan time as it is useless
* [Rework] Lua core: Use lightuserdata to index classes
* [Rework] Lua_util: Another rework for extract_specific_urls
* [Rework] Migrate from ip_score to reputation
* [Rework] Move mime modification functions to lua_mime library
* [Rework] Rbl: Major whitelisting logic rework
* [Rework] Remove deprecated plugins
* [Rework] Remove log helper worker
* [Rework] Remove rspamd.classifiers.lua
* [Rework] Rename filter.h to a more sane name
* [Rework] Reorganise selectors implementation
* [Rework] Replace linenoise with replxx
* [Rework] Reputation: Remove ipnet from the ip reputation
* [Rework] Reputation: Slashing - change name of symbols
* [Rework] Rework children operations
* [Rework] Rework config reload
* [Rework] Rework expression API
* [Rework] Rework image urls processing
* [Rework] Rework initialisation to reduce static leaks count
* [Rework] Rework request headers processing
* [Rework] Slashing: Change versioning schema - move to 2.0
* [Rework] Slashing: Turn off postfilters when passthrough result is set
* [Rework] Start moving to replxx
* [Rework] Stop support of signed HTTP maps to simplify code
* [Rework] Store ASN as UInt32 in ClickHouse
* [Rework] Url_redirector: Rewrite plugin
* [Rework] Use a dedicated library for autolearn
* [Rework] Use libsodium instead of hand crafted crypto implementations
* [Rework] Use opaque structure to store a table of mime headers
* [Rules] Add dedicated bitcoin addresses filter rule
* [Rules] Add more detection to LEAKED_PASSWORD_SCAM
* [Rules] Catch LTC addresses
* [Rules] Reduce weight of RSPAMD_EMAILBL
* [Rules] Rework LEAKED_PASSWORD_SCAM rule one more time
1.9.4: 23 May 2019
* [CritFix] Fix case sensitivity when parsing Content-Type
* [Fix] Arc: Another bunch of fixes for arc signing
* [Fix] Arc: More arc signing fixes
* [Fix] Avoid another overflow in fpconv
* [Fix] Fix ARC signing after fixing another bug in it...
* [Fix] Fix dkim signing exceptions
* [Fix] Fix some more corner cases for fpconv
* [Fix] Further fixes to printing of the FP numbers
* [Fix] Ratelimit: Fix dynamic score
1.9.3: 13 May 2019
* [Conf] Add IP_SCORE_FREEMAIL composite rule
* [Feature] Add cryptobox method to generate dkim keypairs
* [Feature] Add fast hashes to lua cryptobox hash
* [Feature] Add least passthrough results
* [Feature] Allow oversign if exists mode
* [Feature] Clickhouse: Modernise table initial schema
* [Feature] Implement IUF interface for specific fast hashes
* [Feature] Lua_util: Allow to obfuscate different fields
* [Feature] Tune memory management in Rspamd and Lua
* [Fix] Avoid buffer overflow when printing long lua strings
* [Fix] Change the default oversigning headers to a more sane list
* [Fix] Clickhouse: Do not store digest as it is not needed now
* [Fix] Clickhouse: Fix lots of storage issues
* [Fix] Clickhouse: Support custom actions
* [Fix] Deny URLs where hostname is bogus
* [Fix] Do not blacklist mail by SPF/DMARC for local/authed users
* [Fix] Fix DoS caused by bug in glib
* [Fix] Fix UCL parsing of the multiline strings
* [Fix] Fix buffer overflow when printing small floats
* [Fix] Fix init code for servers keypairs cache
* [Fix] Fix issue with urls with no tld (e.g. IP)
* [Fix] Fix memory in arc signing logic
* [Fix] Fix memory leak in language detector during reloads
* [Fix] Fix mixed case content type processing
* [Fix] Fix processing of the ip urls in file
* [Fix] Fix use after free
* [Fix] HTML: Fix `size` attribute processing
* [Fix] Hum, it seems that 99ff1c8 was not correct
* [Fix] Lua_task: Fix task:get_from method
* [Fix] Preserve fd when mapping file to scan
* [Fix] Re-use milter_headers settings when doing arc signing
* [Fix] Set dmarc force action as least action
* [Fix] Switch to GMT
* [Fix] allow PKCS7 signatures to be text/plain, too
* [Project] Add initial version of the vault management tool
* [Project] Add vault support for DKIM and ARC signing
* [Project] Implement keys rotation in the vault
* [Project] Improve dkim keys generation for vault
* [Project] Improve keys creation in rspamadm vault
* [Rework] Move lua_worker to a dedicated unit
* [WebUI] Add URL fragments (#) support
* [WebUI] Fix AJAX request URL
1.9.2: 16 Apr 2019
* [Conf] Allow to load users plugins from plugins.d
* [Conf] oversign openpgp and autocrypt headers
* [Feature] Add SPF FFI library for Lua
* [Feature] Add more verbosity for SPF caching
* [Feature] Antivirus: Handle encrypted files specially
* [Feature] Clickhouse: Slashing - add new fields to CH
* [Feature] Dkim_signing: Add OpenDKIM like signing_table and key_table
* [Feature] Dkim_signing: Allow to use new options as maps
* [Feature] Import fpconv library
* [Feature] Lua_maps: Allow static regexp and glob maps
* [Feature] Parse ical files
* [Feature] Rspamadm: Add dns_tool utility
* [Feature] Store SPF records digests
* [Feature] Use fpconv girsu2 implementation for printing floats
* [Fix] Clickhouse: Use integer seconds when inserting rows
* [Fix] Fix floating point printing
* [Fix] Fix processing of embedded urls
* [Fix] Lua_clickhouse: Fix CH errors processing
* [Fix] Make spf digest stable
* [Fix] Properly detect encrypted files in zip archives
* [Fix] Slashing: Store times in GMT timezone in ClickHouse
* [Rules] Add additional conditions to perform BTC checks
* [Rules] Fix pay-to-hash addresses validation
1.9.1: 5 Apr 2019
* [Conf] Add vendor groups for symbols
* [Feature] Add `rspamadm template` command
* [Feature] Allow to add messages from settings
* [Feature] Allow unconnected DNS servers operations
* [Feature] Check limits after being set, migrate to uint64
* [Feature] Greylist: Allow to disable greylisting depending on symbols
* [Feature] Improve lua binary strings output
* [Feature] Mime_types: Implement user configurable extension filters
* [Feature] Mime_types: When no extension defined, detect it by content
* [Feature] Preprocess config files using jinja templates
* [Feature] Replies: Filter replies sender to limit whitelisting to direct messages
* [Feature] Treat all tags with HREF as a potential hyperlinks
* [Feature] Validate BTC addresses in LEAKED_PASSWORD_SCAM
* [Fix] Add crash safety for HTTP async routines
* [Fix] Another fix for Redis sentinel
* [Fix] Clickhouse: Fix table schema upload
* [Fix] Core: Fix squeezed dependencies handling for virtual symbols
* [Fix] Finally fix default parameters parsing in actions section
* [Fix] Fix ES sending logic (restore from coroutines mess)
* [Fix] Fix finishing script for clickhouse collection
* [Fix] Fix priority for regexp symbols registriation
* [Fix] Fix various issues found by PVS Studio
* [Fix] Initialize lua debugging earlier
* [Fix] Neural: Fix training
* [Fix] Rework cached Redis logic to avoid sentinels breaking
* [Fix] SURBL: Fix regression in surbl module
* [Fix] Fix double signing in the milter
* [Project] Add support of HTTP proxy in requests
* [Rework] Change lua global variables registration
* [Rework] Rework HTML content urls extraction
* [Rework] Start rework of aliasing in Rspamd
* [WebUI] Combine Scan and Learning into one tab
* [WebUI] Fix symbol score input type
* [WebUI] Show grayed out pie
* [WebUI] Update Throughput summary values dynamically
1.9.0: 12 Mar 2019
* [Conf] Add missing includes
* [Conf] Move to options
* [Conf] Rbl: DWL is actually special whitelist
* [Conf] Relax some uribl rules
* [Conf] Remove abuse.ch
* [CritFix] Html: Entities are not valid within tag params values
* [Feature] Add `rspamadm mime sign` tool
* [Feature] Add configgraph utility
* [Feature] Add dedicated ZW spaces detection for URLs
* [Feature] Add flag to url object when visible part is url_like
* [Feature] Add method task:lookup_words
* [Feature] Add pyzor support (by crosenberg)
* [Feature] Allow to add upstream watchers to Lua API
* [Feature] Allow to set rewrite subject pattern from settings
* [Feature] Better escaping of unicode
* [Feature] Clickhouse: Allow to store subject in Clickhouse
* [Feature] Core: Add QP encoding utility
* [Feature] Core: Add libmagic detection for all parts
* [Feature] Core: Add support for gzip archives
* [Feature] Core: Allow to construct scan tasks from raw data
* [Feature] Core: Detect charset in archived files
* [Feature] Core: Ignore and mark invisible spaces
* [Feature] Core: Normalise zero-width spaces in urls
* [Feature] Core: Process data urls for images
* [Feature] Core: Relax quoted-printable encoding
* [Feature] Core: Support RFC2231 encoding in headers
* [Feature] Core: Support telephone URLs
* [Feature] Core: allow to emit soft reject on task timeout
* [Feature] DCC: Add bulkness and reputation checks to dcc
* [Feature] Elastic: Modernize plugin
* [Feature] Export visible part of url to lua
* [Feature] Fuzzy_storage: add preliminary support of rate limits
* [Feature] HTML: Specially treat data urls in HTML
* [Feature] Implement event watchers for upstreams
* [Feature] Implement includes tracing in Lua
* [Feature] Improve dkim part in configwizard
* [Feature] Lua_scanners: Add VadeSecure engine support
* [Feature] Lua_task: Add flexible method to get specific urls
* [Feature] Mime_types: Add MIME_BAD_UNICODE rule
* [Feature] Mime_types: Use detected content type as well
* [Feature] Plugins: Add preliminary version of the external services plugin
* [Feature] Query sentinel on master errors
* [Feature] Regexp: Allow local lua functions in Rspamd regexp module
* [Feature] Rspamadm: Allow to append footers to plain messages
* [Feature] Rspamadm: Allow to rewrite headers in messages
* [Feature] Selectors: Add `ipmask` processor
* [Feature] Settings: Allow hostname match
* [Feature] Settings: Allow local when selecting settings
* [Feature] Settings: Allow multiple selectors
* [Feature] Settings: Allow to inverse conditions
* [Feature] Support User-Agent in HTTP requests
* [Feature] Support ed25519 dkim keys generation
* [Feature] Try to filter bad unicode types during normalisation
* [Feature] external_services - oletools (olefy) support
* [Feature] lua_scanners - icap protocol support
* [Feature] lua_scanners - spamassassin spam scanner
* [Fix] Add filter for absurdic URLs
* [Fix] Add some more cases for Received header
* [Fix] Allow to disable/enable composite symbols
* [Fix] Arc: Use a separated list of headers for arc signing
* [Fix] Archive: Final fixes for 7z archives
* [Fix] Clickhouse: Fix database usage
* [Fix] Controller: Make save stats timer persistent
* [Fix] Core: Detect encrypted rarv5 archives
* [Fix] Core: Don't detect language twice
* [Fix] Core: Fix address rotation bug
* [Fix] Core: Fix content calculations for message parts
* [Fix] Core: Fix emails comments parsing and other issues
* [Fix] Core: Fix etags support
* [Fix] Core: Fix headers folding on the last token
* [Fix] Core: Fix iso-8859-16 encoding
* [Fix] Core: Fix log_urls flag (and encrypted logging)
* [Fix] Core: Fix part length when dealing with boundaries
* [Fix] Core: Fix parts distance calculations
* [Fix] Core: Fix processing of NDNs of certain type
* [Fix] Core: Implement logic to find some bad characters in URLs
* [Fix] Core: treat nodes with ttl properly in lru cache
* [Fix] Fix Content-Type parsing
* [Fix] Fix HTTP headers signing case
* [Fix] Fix control interface
* [Fix] Fix deletion of the duplicate headers
* [Fix] Fix emails filtering in emails module
* [Fix] Fix greylisting log message and logic
* [Fix] Fix issues with storing of the accepted addr in rspamd control
* [Fix] Fix maps object update race condition
* [Fix] Fix memor leaks and whitespace processing
* [Fix] Fix processing of null bytes in headers
* [Fix] Fix rcpt_mime and from_mime in user settings
* [Fix] Fix rfc2047 decoding for CD headers
* [Fix] Fix rfc2231 for Content-Disposition header
* [Fix] Fix setting of the subject pattern in config
* [Fix] Greylist: fix records checking
* [Fix] HTML: Another HTML comments exception fix
* [Fix] HTML: Another entities decoding logic fix
* [Fix] HTML: Fix HTML comments with many dashes
* [Fix] HTML: Fix entities in HTML attributes
* [Fix] HTML: Fix some more SGML tags issues
* [Fix] Ignore whitespaces at the end of value in DKIM records
* [Fix] MID module: Fix DKIM domain matching
* [Fix] Milter_headers: Fix remove_upstream_spam_flag and modernise config
* [Fix] Mime_parser: Fix issue with parsing of the trailing garbadge
* [Fix] Mime_parser: Fix parsing of mime parts without closing boundary
* [Fix] Multimap: Fix operating with userdata
* [Fix] Process orphaned `symbols` section
* [Fix] Rdns: Fix multiple replies in fake replies
* [Fix] Rework groups scores definitions
* [Fix] Set proper element when reading data from Sentinel
* [Fix] Set rspamd user to initialise supplementary groups on reload
* [Fix] Settings: Fix selectors usage
* [Fix] Sort data received from Sentinel to avoid constant replacing
* [Fix] groups.conf - filename typo
* [Fix] lua_scanner - oletools typos, logging
* [Fix] lua_scanners - actions and symbol_fail
* [Fix] lua_scanners - fix luacheck
* [Fix] lua_scanners - kaspersky - response with fname
* [Fix] lua_scanners - savapi redis prefix
* [Fix] tests - antivirus - fprot symbols
* [Project] Add concept of flexible actions
* [Project] Add heuristical from parser to received parser
* [Project] Add new flags to clickhouse, redis and elastic exporters
* [Project] Attach new received parser
* [Project] Fallback to callbacks from coroutines
* [Project] Implement keep-alive support in lua_http
* [Project] Lua_udp: Implement fully functional client
* [Project] Plug keepalive knobs into http connection handling
* [Project] Rspamadm: Add `modify` tool
* [Rework] Convert rspamd-server to a shared library
* [Rework] Dcc: Rework DCC plugin
* [Rework] Enable explicit coroutines symbols
* [Rework] Rework telephone urls parsing logic
* [Rework] Rewrite RBL module
* [Rework] Settings: Rework settings check
* [Rework] Slashing: Distinguish lualibdir, pluginsdir and sharedir
* [Rework] Unify task_timeout
* [Rework] Use VEX instructions in assembly, relocate
* [WebUI] Notify user if uploaded data was not learned
* [WebUI] Remove redundant condition
1.8.3: 03 Dec 2018
* [CritFix] Make flags mutually exclusive for mime parts
* [CritFix] Strictly deny unencoded bad utf8 sequences in headers
* [Feature] Add Kaspersky antivirus support
* [Feature] Add method to get dkim results
* [Feature] Add more words regexp classes
* [Feature] Allow to choose words format in `rspamadm mime`
* [Feature] Allow to get all types of words from Lua
* [Feature] Allow to get task flags in C expressions
* [Feature] Allow to require encryption when accepting connections
* [Feature] Ignore bogus whitespaces in the words
* [Feature] Implement more strict configuration tests
* [Feature] Improve SPF results in Authentication-Results
* [Feature] Support ClickHouse database
* [Fix] Add failsafety for utf8 regexps
* [Fix] Do not trigger BROKEN_CONTENT_TYPE on innocent text parts
* [Fix] Emit error if connection has been terminated with no stop pattern
* [Fix] Fix boundaries checks in embedded messages
* [Fix] Fix double free
* [Fix] Perform policy downgrade on sample out, add tests
* [Fix] Properly escape utf8 regexps in hyperscan mode
* [Fix] Selectors - attachments args condition
* [Fix] Some fixes for raw parts
* [Fix] Treat learning errors as non-fatal
* [Fix] Use tld when looking for DKIM domains
* [Project] Words unicode structure rework
* [Project] Add preliminary Redis Sentinel support
* [Project] Improve Authentication-Results header
* [Project] Rework DKIM checks results
* [Project] Use more generalised API to produce meta words
1.8.2: 19 Nov 2018
* [Conf] Add DWL support in the default configuration
* [Conf] Disable rspamd_update by default (again)
* [Conf] Fix configuration sample for ratelimit
* [CritFix] Disable broken url tags by default
* [CritFix] Fix \0 processing when doing RSA sign
* [CritFix] Fix adding symbols to their primary groups
* [Feature] Add `rspamadm cookie` utility
* [Feature] Add specialised functions for generating encrypted cookies
* [Feature] Add support of cookies in replies module
* [Feature] Add support of words regexps
* [Feature] Allow to add 3rd party clang plugins
* [Feature] Allow to create lua regexps from glob or plain patterns
* [Feature] Allow to set custom limits for upstream lists
* [Feature] Detect orphaned parts and attach them to message
* [Feature] Filter tokens in bayes
* [Feature] Fold b= value when doing arc sealing
* [Feature] Ignore cookies in the future and too old in the past
* [Feature] Skip stop words in statistics
* [Feature] Store stop words and allow to query them
* [Feature] Support query arguments in controller's custom commands
* [Feature] Tune upstream limits in Rspamd proxy
* [Feature] Use different callback symbols for different uribls
* [Feature] Write DKIM selector in dkim allow/reject symbols
* [Fix] Add obs_fws state support to eoh state machine
* [Fix] Add sanity check when applying mime boundaries heuristic
* [Fix] Antivirus - virus names with 0 were recognized as tables
* [Fix] Disable headernames in bayes temporarily
* [Fix] Do not allow syntax errors in include files...
* [Fix] Do not allow to merge an object with an array (or vice versa)
* [Fix] Don't perform forged recipients check for missing recipients
* [Fix] Fix DKIM based RBLs
* [Fix] Fix actrie implementation (sync from upstream), fixed OOB read
* [Fix] Fix explicit methods call in selectors
* [Fix] Fix extraction of additional parts
* [Fix] Fix finalization for internal plugins
* [Fix] Fix override_defaults function
* [Fix] Fix squeezed symbols when using settings
* [Fix] Fix urls insertion in Clickhouse module
* [Fix] Furhter fixes to ratelimits logic
* [Fix] Ignore signatures when looking for boundaries
* [Fix] Properly set learned count
* [Fix] Really fix ratelimits configuration and work
* [Fix] Remove ambigious format flag from printf
* [Fix] Restore URLs exporting in ClickHouse plugin
* [Fix] Rework bayes calculations...
* [Fix] Switch from chi-square to naive for large Fisher value
* [Fix] Treat normal password as enable password if there is no enable password
* [Fix] Use proper syntax for making DNS requests
* [Fix] Various fixes in embedded plugins
* [Project] Change fuzzy check selection logic to lua_fuzzy library
* [Project] Rework async events and symbols
* [Project] Move all metatokens in Bayes to lua_stat from C
* [WebUI] Add history rows per page control
1.8.1: 16 Oct 2018
* [CritFix] Fix options insertion
* [CritFix] Fix words decay one more time (affects long messages)
* [CritFix] Increase default words_decay
* [CritFix] Plug memory leak in redis pool
* [Feature] Add `check_violation` feature to DKIM/ARC signing
* [Feature] Add only unique elements to Clickhouse url arrays
* [Feature] Allow `g+:` and `g-:` composite atoms
* [Feature] Allow dkim domains check in surbl
* [Feature] Allow maps with HTTP auth
* [Feature] Allow to disable actions by users settings
* [Feature] Extend whitelisting options
* [Feature] Store url object in images
* [Feature] Use verdict instead of the plain action in plugins
* [Fix] Allow to call fstring append with NULL string
* [Fix] DCC - luacheck
* [Fix] Do not load torch on each rspamadm invocation
* [Fix] Fix boundaries detection and rework stop words algorithm
* [Fix] Fix dependencies for DNS_SIGNED symbol
* [Fix] Fix errors when dealing with dynamic rates/bursts in Ratelimit
* [Fix] Fix groups mess
* [Fix] Fix groups mess
* [Fix] Fix parsing address with comments
* [Fix] Fix resolving in DMARC reports
* [Fix] Fix various issues with parsing of the received headers
* [Fix] Fix watchers issue in lua_tcp when doing no resolving
* [Fix] Plug memory leak in language detector (affects reloads)
* [Fix] Remove one letter stop words
* [Fix] Slashing: backport chunk logic from libucl
* [Fix] Stop libevent from using cached time in rspamadm
* [Fix] Try to fix watchers chaining
* [Fix] Various fixes in redis sync interface
* [Fix] ip_score - respect check_authed and check_local settings from config
* [Project] Rework passthrough actions
* [Project] Clustering module
* [Rework] Always create result for a task
* [Rework] Completely rewrite DMARC checks logic
* [Rework] Rework and fix whitelist plugin
* [WebUI] Add symbols sorting buttons
* [WebUI] Change symbols order without updating history
* [WebUI] Colorize symbols
* [WebUI] Do not display password form when secure_ip is set
* [WebUI] Fix symbol description tooltips display
* [WebUI] History: add sorting by symbol score value
1.8.0: 24 Sep 2018
* [Feature] Add arguments schemas to processors and extractors
* [Feature] Add functional selectors library
* [Feature] Add generic selector to reputation module
* [Feature] Add more ratelimits: by digest, by attachments data, by filenames
* [Feature] Add preliminary stop words detection support
* [Feature] Add pure Lua debugm function
* [Feature] Add schema validation for Redis settings
* [Feature] Add selectors combine function
* [Feature] Add some recursion protection to lua logger
* [Feature] Add support for Lua API tracing
* [Feature] Allow to apply schema to arguments
* [Feature] Allow to get dkim signing data directly from HTTP headers
* [Feature] Allow to reuse existing authentication results
* [Feature] Cache selectors results in re runtime
* [Feature] Implement new text tokenizer based on libicu
* [Feature] Integrate selectors framework to multimap
* [Feature] Relax FORGED_RECIPIENTS
* [Feature] Support (almost) all html entities
* [Feature] Support adding and deletion of recipients in the milter block
* [Feature] Support gathering HTTP body from fragments in lua_http
* [Feature] Support multi flag in regexp and glob maps
* [Feature] Support selectors in ratelimit module
* [Feature] Support selectors in settings
* [Feature] Use khash in HTML parser
* [Feature] Use pure Lua debugm function
* [Fix] Add fail-safety for destroying sessions
* [Fix] Allow to add result-less fake DNS records
* [Fix] Another try to fix race conditions on config unload
* [Fix] Call Lua callback on DNS timeouts
* [Fix] Deprecate task:inc_dns_req as it is redundant
* [Fix] Do not allow events deletions on cleanup
* [Fix] Do not try to process skipped messages
* [Fix] Fix HTTP requests with no body
* [Fix] Fix another cleanup race condition
* [Fix] Fix bug in processing of pcre regexps
* [Fix] Fix byte array allocation in the pool
* [Fix] Fix crashes on task cleanup
* [Fix] Fix dynamic buckets in ratelimits
* [Fix] Fix endless loop when waiting for Rspamd to stop
* [Fix] Fix lua_util.str_split in case of delimiters set
* [Fix] Fix more issues with watching of async events
* [Fix] Fix stop words detection and loading logic
* [Fix] Fix various corner cases for language detection
* [Fix] Fix watchers in lua_tcp
* [Fix] Fix words decay algorithm
* [Fix] Implement watchers replacement to handle nested calls
* [Fix] Save faked code into fake dns record
* [Fix] Show the proper frame when using lua_util.debugm
* [Fix] Use fake dns records in tests
* [Fix] Use unicode replacements for HTML entities
* [Fix] fixed "cannot find dependency on symbol 1" issue when using replaced symbols in spamassassin rules
* [Fix] partition_id is not available in old versions of CH
* [Project] Add implicit conversion logic to selectors
* [Project] Add initial support for selectors in regexps
* [Project] Add method concept
* [Project] Further changes in unicode operations
* [Project] Implement Clickhouse migrations
* [Project] Implement implicit conversions to userdata
* [Project] Implement insert method
* [Project] Implement selectors registration for regular expressions
* [Project] Implement selectors support in re_cache
* [Project] Improve language detector: cleanup unused files, categorize
* [Project] Migrate CH data to a fat table
* [Project] Rework selectors logic
* [Project] Start Clickhouse utilities library
* [Project] Start unicode rework
* [Project] coroutine threaded model for API calls: thread pool
* [Rework] Move phishtank to a DNS based service
* [Rework] Rework Clickhouse plugin to use the new API
* [Rework] Rework language detector
* [Rework] Rework utf content processing in text parts
* [WebUI] Add progress bar for AJAX requests
* [WebUI] Avoid errors table reinitialization
* [WebUI] Avoid history table reinitialization
* [WebUI] Avoid throughput summary table reinitialization
* [WebUI] Destroy summary table on disconnect
* [WebUI] Fix "auth" request URL
* [WebUI] Fix disabling and hiding controls on page reload
* [WebUI] Fix maps loading from neighbours
* [WebUI] Fix symbols sorting by score
* [WebUI] Fix tables destroying
* [WebUI] Fix throughput data consolidation
* [WebUI] Fix upload buttons disabling
* [WebUI] Notify user on module loading failure
* [WebUI] Update FooTable 3.1.4 -> 3.1.6
1.7.9: 01 Aug 2018
* [CritFix] Fix caseless comparison of equal length strings
* [Feature] Add HTTP basic auth support to elastic and clickhouse plugins
* [Feature] Add SPF selector to reputation
* [Feature] Add support of the fallback backends for HTTP maps
* [Feature] Allow to print full mime structure when extracting mime data
* [Feature] Allow to split symbols in reputation plugin
* [Feature] Check attachments only on AV scanners in attachments_only mode
* [Feature] Disable all SSL checks if ssl_no_verify flag is set
* [Feature] Implement parsing of scoped IPv6 addresses
* [Feature] Improve `rspamc counters` output
* [Fix] Add sanity checks when expanding SPF macros
* [Fix] Allow to parse SA rules with no spaces around =~ (dirty hack)
* [Fix] Avoid one extra byte writing
* [Fix] Deal with direct hash table
* [Fix] Detect empty text part as text, not HTML
* [Fix] Do not reduce map watch timeout for mixed http/file maps
* [Fix] Fix HTML part detection heuristic
* [Fix] Fix double free in redirectors cleanup
* [Fix] Fix legacy history handling in the controller
* [Fix] Fix messages insertion
* [Fix] Fix sending string method
* [Fix] Fix statconver command line arguments
* [Fix] Fixed argument checking for being null
* [Fix] Fixed issues reported by luacheck
* [Fix] Freeze updates queue when do actual storage update
* [Fix] HTTP map hash is per-backend and not per-map
* [Fix] Plug memory leak in fuzzy updates
* [Fix] Prefer 'MTA-Name' when producing authentication results
* [Fix] Replace bad unicode sequences instead of stopping on them
* [Fix] Set classifier version on learning
* [Project] Reworked ratelimits
* [Project] Apply topological sorting for symbols in Rspamd
* [Project] Remove global contexts from C modules
* [Project] Move performance critical hash tables to khash
* [WebUI] Avoid unused indexes
* [WebUI] Do not execute `on_success` callback
* [WebUI] Fix history reset for "All SERVERS" (#2346)
* [WebUI] Fix query URL for selected server
* [WebUI] Fix symbols display in legacy history,
* [WebUI] Hide symbols order selector for legacy history
* [WebUI] Refactor query functions into one
* [WebUI] Remove previously-attached event handlers
* [WebUI] Save symbols to the selected server
* [WebUI] Unify arguments of query functions
* [WebUI] Use common query functions to get graph data
* [WebUI] Use common query functions to save symbols
1.7.8: 12 Jul 2018
* [Feature] Add more extended statistics about fuzzy updates
* [Feature] Add more non-conformant Received headers support
* [Feature] Add preliminary function to get fuzzy hashes from text in Lua
* [Feature] Allow to configure AV module rejection message
* [Feature] Implement fuzzy hashes extraction in mime tool
* [Feature] Improve WHITE_ON_WHITE rule
* [Feature] Improve integer -> string conversion
* [Feature] Reuse maps in multimap module more aggressively
* [Fix] Avoid race condition in skip map as pool lifetime is not enough
* [Fix] Eliminate all specific C plugins pools
* [Fix] Fix DKIM check rule if DNS is unavailable
* [Fix] Fix build where ucontext is defined in ucontext.h
* [Fix] Fix crash in base url handling
* [Fix] Fix descriptors leak in sqlite3 locking code
* [Fix] Fix messages quarantine
* [Fix] Fix padded numbers printing
* [Fix] Fix race condition on maps reinit
* [Fix] Fix regexp functions when no data is passed
* [Fix] Fix specific urls extraction
* [Fix] Fix styles propagation
* [Fix] Improve resetting of the limit buckets
* [Fix] Initialize sqlite3 properly
* [Fix] Work with broken resolvers in resolv.conf
* [Project] Implement HTTP maps caching
* [Project] Refresh fuzzy hashes when matched
* [Project] Add logic to deduplicate fuzzy updates queue
* [WebUI] Add missed declarations
* [WebUI] Avoid using "undefined" property
* [WebUI] Do not accept passwords containing control characters
* [WebUI] Do not redeclare variables
* [WebUI] Enable strict mode,
* [WebUI] Fix variable assignment
* [WebUI] Initialize variables at declaration
* [WebUI] Remove duplicated path from RequireJS config
* [WebUI] Remove unused block
* [WebUI] Remove unused variable
* [WebUI] Remove unused variables
* [WebUI] Use self-explanatory notation
* [WebUI] Use type-safe equality operators
1.7.7: 02 Jul 2018
* [CritFix] Check NM part of pubkey to match it with rotating keypairs
* [CritFix] Do not overwrite PID of the main process
* [CritFix] Fix maps after reload
* [CritFix] Fix maps race conditions on reload
* [CritFix] Fix shmem leak in encrypting proxy mode
* [Feature] Add a concept of ignored symbols to avoid race conditions
* [Feature] Add ability to print bayes tokens in rspamadm mime
* [Feature] Add method to get statistical tokens in Lua API
* [Feature] Add preliminary mime stat command
* [Feature] Add rspamadm mime tool
* [Feature] Add urls extraction tool
* [Feature] Address ZeroFont exploit
* [Feature] Allow rspamadm mime to process multiple files
* [Feature] Allow to extract words in `rspamadm mime`
* [Feature] Allow to print mime part data
* [Feature] Allow to show HTML structure on extraction
* [Feature] Distinguish IP failures from connection failures
* [Feature] Improve output for mime command
* [Feature] Improve styles propagation
* [Feature] Main process crash will now cleanup all children
* [Feature] Preload file and static maps in main process
* [Feature] Print stack trace on crash
* [Feature] Process font size in HTML parser
* [Feature] Propagate content length of invisible tags
* [Feature] Read ordinary file maps in chunks to be more safe on rewrites
* [Feature] Support base tag in HTML
* [Feature] Support more size suffixes when parsing HTML styles
* [Feature] Support opacity style
* [Fix] Another fix for nested composites
* [Fix] Fill nm id in keypairs cache code
* [Fix] Fix colors alpha channel handling
* [Fix] Fix destruction logic
* [Fix] Fix double free
* [Fix] Fix maps preload logic
* [Fix] Fix nested composites process
* [Fix] Fix proxying of Exim connections
* [Fix] Fix reload crash
* [Fix] Fix rspamadm -l command
* [Fix] Update ed25519 signing schema
* [WebUI] Stop using "const" declaration
* [WebUI] Update RequireJS to 2.3.5
1.7.6: 15 Jun 2018
* [CritFix] Fix multiple neural networks support
* [Feature] Add decryption function to keypair command
* [Feature] Add gzip compression for HTTP requests in elastic module
* [Feature] Add gzip methods to lua util
* [Feature] Add maps based on Top Level Domains
* [Feature] Add pubkey checks for dkim_signing
* [Feature] Add support of fake DNS records
* [Feature] Add tool to encrypt files
* [Feature] Allow to add symbols using settings directly
* [Feature] Allow to match private and public keys for DKIM signatures
* [Feature] Allow to set task flags via settings
* [Feature] Allow to specify fake DNS address from the config
* [Feature] Implement signatures verification using rspamadm keypair
* [Feature] Implement signing using `rspamadm keypair`
* [Feature] Improve error reporting for DKIM key access issues
* [Feature] Provide $HOSTNAME variable in UCL
* [Feature] Rework levenshtein distance computation
* [Feature] Split message parsing and processing
* [Feature] Support ED25519 DKIM signatures
* [Feature] Support encrypted configs in UCL
* [Feature] Suppress duplicate warning on very large radix tries
* [Feature] Use OSB to combine header names
* [Fix] Cleanup maps data on shutdown
* [Fix] Fix '~' behaviour in composites
* [Fix] Fix HTTP maps updates
* [Fix] Fix NIST signatures
* [Fix] Fix RFC822 comments when processing a mime address
* [Fix] Fix double free
* [Fix] Fix dynamic settings application
* [Fix] Fix for CommuniGate Pro maillist
* [Fix] Fix keypair creation method to actually create keypair...
* [Fix] Fix matching patterns with no paths
* [Fix] Fix memory leak in parsing comments
* [Fix] Fix parsing of urls with numeric password
* [Fix] Fix plugins intialisation in configwizard
* [Fix] Fix potential crash on reload
* [Fix] Fix potential race condition for a finished HTTP connections
* [Fix] Fix race-condition leak on processes reload
* [Fix] Fix signing in openssl mode
* [Fix] Free language detector structures
* [Fix] Relax alignment requirements
* [Fix] Send DMARC reports compressed
* [Fix] Try to fix leak in dmarc module
* [Fix] Try to plug memory leak in metric exporter
* [Project] Convert rspamadm subcommands to Lua
* [WebUI] Display smtp sender/recipient in history
* [WebUI] Fix elements disabling in "Symbols" tab
* [WebUI] Limit recipients list in history column to 3
* [WebUI] Match envelope and mime addresses following in arbitrary order
* [WebUI] Update column header
* [WebUI] Wrap addresses in history
1.7.5: 18 May 2018
* [Conf] Add MSBL proposed return codes
* [Conf] Add additional groups for policies
* [CritFix] Do not use volatile Lua strings as UCL keys
* [Feature] Add ability to add fuzzy hashes to headers
* [Feature] Add function to extract most meaningful urls
* [Feature] Add rule to block mixed text and encrypted parts
* [Feature] Allow multiple groups for symbols
* [Feature] Allow to disable lua squeezing logic
* [Feature] Allow to get multipart children in Lua
* [Feature] Allow to insert multiple headers from milter headers
* [Feature] Allow to print scores in subject and further extensions
* [Feature] Be more error-prone in squeezed rules
* [Feature] Support multiple return codes in emails module
* [Feature] Use EMA for calculating averages
* [Feature] Use common jit cache for all regexps
* [Feature] support for CommuniGate Pro self-generated messages
* [Fix] Allow to have multiple values for headers as arrays
* [Fix] Do not open sockets for disabled workers
* [Fix] Fix AuthservId
* [Fix] Fix base64 folding in Lua API
* [Fix] Fix build on non-x86 platforms
* [Fix] Fix cached maps logic
* [Fix] Fix compatibility with old maps query logic
* [Fix] Fix crash if skip_map is used
* [Fix] Fix importing static maps from UCL
* [Fix] Fix parsing of unix sockets
* [Fix] Fix raw_mime regexp on HTML part with no text content
* [Fix] Fix tables logging
* [Fix] Fix vertical tab handling in libucl
* [Fix] Try to fix frequency counters
* [Fix] Use better sharding for ip_score
* [Fix] Use multiple results from SURBL DNS reply
* [Fix] When doing AV scan select a different server for retransmit
1.7.4: 01 May 2018
* [Conf] Major stock config updates:
- Workers are now specified in a new format worker "type" { ... }
- Enable fuzzy worker to simplify local fuzzy storages configuration
- Bind all workers to localhost by default to avoid security flaws
* [Conf] Make more sane fuzzy_check default settings
* [CritFix] Fix ucl escape for bad symbols
* [Feature] Add failure symbol for AV module
* [Feature] Add lazy expiration mode for new classifier schema
* [Feature] Add preliminary version of maps stats plugin
* [Feature] Allow to block fuzzy requests from specific networks
* [Feature] Allow to change `expire` of live statistics
* [Feature] Distinguish AV failure from clean result
* [Feature] Further improvements of language detector by using khash
* [Feature] Further optimization of the lang_detection
* [Feature] Implement cluster-aware bayes expiry
* [Feature] Implement exclude patterns in rspamc
* [Feature] Implement glob maps in addition to regexp maps
* [Feature] Implement map statistics function for lua API
* [Feature] Implement stop symbols for Clickhouse collection
* [Feature] Support recipients separated by commas
* [Feature] Try harder to upload scripts to the Redis server
* [Feature] Upgrade t1ha distribution
* [Feature] use_domain_sign_inbound
* [Feature] Use scores from maps if `symbols_set` is not defined
* [Fix] Add resolving version of radix map helper
* [Fix] Check URL before adding implicit prefix
* [Fix] Do not check pid/state when using PRNG
* [Fix] Fix CentOS logrotate script for systemd
* [Fix] Fix slash + dot in urls
* [Fix] Fix systemd version of the logrotate script
* [Fix] Propagate key when import implicit array from Lua
* [Fix] Strip spaces from map keys and values
* [Fix] Try to fix a specific case when processing milter protocol
* [Fix] Try to fix crash when a tcp connection cannot be set
* [Fix] Typo use_domain_local --> use_domain_sign_local
* [Fix] Various fixes to once_received module
* [Project] Store hits counters for map elements
1.7.3: 10 Apr 2018
* [CritFix] Plug bad memory leak in protocol reply
* [Feature] Add avx2 codec for base64
* [Feature] Add method to receive all URL flags from Lua API
* [Feature] Allow to fold headers on stop characters
* [Feature] Allow to set lua_cpath from options
* [Feature] Allow to specify custom rejection message in milter
* [Feature] Deal with unnormalised Unicode obfuscation
* [Feature] Do not detect language twice for relative parts
* [Feature] Implement oversigning feature
* [Feature] Implement silent logging level to minimize noise in logs
* [Feature] Improve URL_IN_SUBJECT rule
* [Feature] Use hashing to reduce redis attack surface
* [Fix] Add oversigning for the most important headers
* [Fix] add 'rewrite subject' to History dropdown
* [Fix] Another fix in folding algorithm
* [Fix] Do not call multimap addr for parts of addr if filter is presented
* [Fix] Do not clean hostname on generic reset
* [Fix] Do not create pid file in no-fork mode
* [Fix] Fix fold_after case to preserve multiple spaces
* [Fix] Fix folding and folding tests
* [Fix] Fix hostname usage in milter mode
* [Fix] Fix lua RSA verify and its tests
* [Fix] Fix metadata exporter send_mail backend (#2124)
* [Fix] Fix processing of '\v' in libucl
* [Fix] Fix shemaless URLs detection
* [Fix] Fix support of multiple headers in sign_header
* [Fix] Fix usage of util.parse_mail_address
* [Fix] Fix weights of dynamic squeezed rules
* [Fix] Leak from bucket before checking the burst
* [Fix] Stop using own localtime as DST could be messy in many cases
* [Fix] Treat unnormalised URLs as obscured
* [Rework] Restore leaky bucket model in ratelimit plugin
* [WebUI] Add messages total to throughput summary
* [WebUI] Add symbols order selector to history
* [WebUI] Config: Load list on demand
* [WebUI] Fix modalBody for maps that appear more than once
* [WebUI] History: Fix Tooltips on paging, filtering and sorting
* [WebUI] Remove a previously-attached event handler
* [WebUI] Update D3 to v5.0.0 and jQuery to v3.3.1
1.7.2: 23 Mar 2018
* [Feature] Store emails in Clickhouse
* [Feature] Support single quotes in config
* [Feature] Use templates when publishing CH schema
* [Feature] Improve Docker image
* [Fix] Add rounding when printing a lot of FP variables
* [Fix] Allow to disable certain actions by assigning null to them
* [Fix] Disable results caching
* [Fix] Fix disabling of squeezed symbols
* [Fix] Fix scan time set
* [Fix] Rework logic of actions setting
* [Fix] Try to fix various Lua stack issues
* [WebUI] Add link tag for favicon.ico
* [WebUI] Display hostname:port/path in the page title
1.7.1: 20 Mar 2018
* [CritFix] Fix lowercase comparison
* [CritFix] Timezone defines seconds WEST UTC not East
* [Feature] Add filename to log format
* [Feature] Add lua rules squeezing
* [Feature] Add related symbols analysis to rspamd_stats
* [Feature] Remove upstream `X-Spam: Yes` header by default
* [Feature] rspamd_stats: Output progress info on STDERR
* [Feature] Whitelist for emails module
* [Fix] Do not allow dependencies on self
* [Fix] Do not cache metric result
* [Fix] Do not trust all issuers as a client certificate
* [Fix] Fix dependencies in lua squeeze
* [Fix] Fix enabling/disabling squeezed rules
* [Fix] Fix enabling/disabling symbols
* [Fix] Fix external dependencies
* [Fix] Fix processing of a single compressed file
* [Fix] Fix some typos
* [Fix] Fix various modules in case of empty message
* [Fix] Handle callbacks that returns table of options
* [Fix] Improve cached action interaction
* [Fix] Make dynamic conf more NaN aware
* [Fix] Never hide actions from WebUI `configuration` tab
* [Project] Implementation of Lua rules squeezing
1.7.0: 12 Mar 2018
* [Conf] Add bayes_expiry as explicit module
* [Conf] Adjust names and weights for neural network plugin
* [Conf] Change updates url
* [Conf] Default statistics is stored in Redis now
* [Conf] Disable fann_redis module by default
* [Conf] Fix default elastic configuration
* [Conf] Fix double quote position
* [Conf] Massive config rework for new structure of symbols and scores
* [Conf] Rename Rambler BLs as they are now Rspamd's ones
* [Conf] Use dedicated rspamd.com subdomains
* [Conf] Use more data from rspamd.com fuzzy storage
* [CritFix] Add sanity guards for badly broken HTML
* [CritFix] Another errors path handling fix
* [CritFix] Another portion of tokenization fixes
* [CritFix] Do not send reject messages after set reply
* [CritFix] Fix ARC chain verification
* [CritFix] Fix crash in milter errors handler
* [CritFix] Fix memory leak in spf caching logic
* [CritFix] Fix milter commands pipelining
* [CritFix] Fix newlines detection
* [CritFix] Fix semicolons parsing in the content type
* [CritFix] Plug memory leak in zstd protocol compression
* [Feature] Add ability to match score in force_actions module
* [Feature] Add aes-rng PRF to libottery
* [Feature] Add 'composites' debug module
* [Feature] Add concept of experimental modules
* [Feature] Add DKIM trace symbol
* [Feature] Add EBL to the default config
* [Feature] Add expected ip check for emails plugin
* [Feature] Add framework to manage Redis scripts
* [Feature] Add framing for the new reputation generic plugin
* [Feature] Add function to show plugins stat
* [Feature] Add gzip compression support for clickhouse module
* [Feature] Add gzip compression support for rspamd controller
* [Feature] Add gzip support when sending lua http requests
* [Feature] Add json output for rspamd_stats
* [Feature] Add method to do a synchronous Redis connection
* [Feature] Add method to get all content-type attributes in Lua
* [Feature] Add `-m` flag to configdump to show modules states
* [Feature] Add mime types to extensions map
* [Feature] Add more features to rescore utility
* [Feature] Add more gtube like patterns to test other spam actions
* [Feature] Add more metafunctions, improve logging
* [Feature] Add more text attributes
* [Feature] Add new configwizard command to rspamadm
* [Feature] Add new tooling for stats conversation
* [Feature] Add old groups migration tool
* [Feature] Add plugins state variable
* [Feature] Add preliminary ecdsa keys support in DKIM
* [Feature] Add preliminary support of idempotent symbols
* [Feature] Add Redis server wizard
* [Feature] Add routine to convert old style stats to a new one
* [Feature] Add some sanity checks for actions and controller
* [Feature] Add statistic convertation module to configwizard
* [Feature] Add sugestions logic to mempool allocator
* [Feature] Add support of config transform in Lua
* [Feature] Add timeout to rspamc when doing corpus test
* [Feature] Add tooling to convert bayes schemas
* [Feature] Add torch conditional to configuration
* [Feature] Add torch-decisiontree package
* [Feature] Add torch-optim contrib package
* [Feature] Add TTL autodetection
* [Feature] Add urls reputation to the reputation framework
* [Feature] Allow floating and negative values in expressions limits
* [Feature] Allow multiple CTs in full extensions map
* [Feature] Allow multiple fann rules
* [Feature] Allow randomly select User-Agent from a list
* [Feature] Allow rspamadm commands to export methods in Lua
* [Feature] Allow rule specific min_bytes in fuzzy check
* [Feature] Allow to adjust symbols scores from Lua
* [Feature] Allow to attach stat signature to messages
* [Feature] Allow to change SMTP from via milter headers
* [Feature] Allow to configure monitored
* [Feature] Allow to create directories in Lua API
* [Feature] Allow to disable torch and skip train samples for ANN
* [Feature] Allow to discard messages dynamically
* [Feature] Allow to enable/disable languages from the detector
* [Feature] Allow to generate DKIM keys from rspamadm API
* [Feature] Allow to get CPU flags from Lua
* [Feature] Allow to have high precision timestamps in logs
* [Feature] Allow to insert headers into specific position
* [Feature] Allow to limit redirector requests per task
* [Feature] Allow to load and use dynamic ANNs with torch
* [Feature] Allow to quarantine rejected messages using milter interface
* [Feature] Allow to receive signing keys from mempool vars
* [Feature] Allow to reserve elements in libucl
* [Feature] Allow to reuse signal handlers chains
* [Feature] Allow to set custom mempool variables from settings
* [Feature] Allow to set headers from settings
* [Feature] Allow to set Settings-Id for all connections
* [Feature] Allow to skip real action and add a header instead
* [Feature] Allow to skip specific hashes in fuzzy storage
* [Feature] Allow to spawn asynchronous processes from Lua
* [Feature] Allow to specify number of threads for ANN learning
* [Feature] Allow to use global lua maps in settings
* [Feature] Allow to use postfilters in composites
* [Feature] Allow to verify signatures from HTTP headers in maps
* [Feature] Antivirus: ordered pattern matches
* [Feature] Authentication-Results: support hiding usernames
* [Feature] Automatically create tables in clickhouse
* [Feature] Catch next-to-last bad extension
* [Feature] Check cached maps more frequently
* [Feature] Check groups sanity
* [Feature] Deal with obscured URLs with @ symbols
* [Feature] Enhance task:store_in_file method
* [Feature] Export password encryption routines to Redis
* [Feature] Filter nan and inf when adding scores
* [Feature] Finalize 7zip files support
* [Feature] Further improvements in language detection
* [Feature] Further improvements in language detection algorithm
* [Feature] Generic key name expansion for Redis keys
* [Feature] Hash whitelist for fuzzy_check
* [Feature] Implement bayes signatures storage
* [Feature] Implement buckets for Redis backend
* [Feature] Implement DKIM reputation adjustments
* [Feature] Implement forked workers children monitoring
* [Feature] Implement headers flags in mime parser
* [Feature] Implement l1/l2 regularization against the current weights
* [Feature] Implement manual ANN train mode
* [Feature] Implement per-user ANN support
* [Feature] Implement torch based ANN learning
* [Feature] Implement upstreams logic for clickhouse exporter
* [Feature] Import torch to Rspamd...
* [Feature] Improve allocation policy when interacting with Lua
* [Feature] Improve Lua/C interaction in history_redis
* [Feature] Improve multiple fuzzy results combining
* [Feature] Improve parsing of DKIM keys: parse algorithm
* [Feature] Improve subprocesses termination handle
* [Feature] Improve symbol type parsing in Lua API
* [Feature] Metadata Exporter: e-Mail Alerts: support multiple recipients; alerting senders/recipients/users (#1600)
* [Feature] Milter headers: support adding/removing arbitrary headers from config
* [Feature] More metatokens
* [Feature] Multimap: checking of symbol options
* [Feature] Multimap: template URL filter
* [Feature] New bayes expiry plugin
* [Feature] Periodically save rspamd stats to disk
* [Feature] Preliminary import of the elasticsearch module
* [Feature] Ratelimit: allow full addresses in whitelisted_rcpts
* [Feature] Ratelimit: support fetching limits from Redis
* [Feature] RBL: received: filtering by position & flags
* [Feature] Read global maps for lua
* [Feature] Redis settings: support checking multiple keys
* [Feature] Rework fann plugin to be a normal post-filter
* [Feature] Rework logging configuration for rspamadm case
* [Feature] Rework short hashes generation to avoid FP
* [Feature] Save real ucl types when exporting to Lua
* [Feature] Set TCP_NODELAY for milter sockets
* [Feature] Setup DKIM signing from configwizard
* [Feature] Skip certain symbols from ANN classify
* [Feature] Store plugins state
* [Feature] Support etag for HTTP maps
* [Feature] Support Expires header when using HTTP maps
* [Feature] Support sending given header multiple times in lua_http
* [Feature] Support sha512 in DKIM signatures
* [Feature] Try to detect HTML messages better
* [Feature] Use array instead of queue to reduce memory fragmentation
* [Feature] Use controller port by default when connecting to local IP
* [Feature] Use rdtsc where possible
* [Fix] Actively load skip hashes map in fuzzy storage
* [Fix] Add another workaround to display history properly
* [Fix] Add definition for old glib compatibility method
* [Fix] Add missing rspamadm control options to help
* [Fix] Add workaround for IPv6 in sendmail
* [Fix] Add workaround for system with non-XSI compatible tzset
* [Fix] Allow oversigning in DKIM signatures
* [Fix] Allow to check negative scores in force_actions
* [Fix] Allow to have negative actions limits
* [Fix] Allow to set any layers number for fann rules
* [Fix] Another fix for rdtcs
* [Fix] Another fix to lua xmlrpc
* [Fix] Another try to deal with #1998
* [Fix] Another try to fix #1998
* [Fix] Another try to fix threading in torch
* [Fix] Apply language detection when adding fuzzy hashes
* [Fix] ARC: Fix Lua 5.3 compatibility; timestamp should be integer
* [Fix] Authentication Results: Fix SPF smtp.mail_from
* [Fix] Auth-Results: Multiple DKIM signatures
* [Fix] Avoid changing content-transfer-encoding header's value
* [Fix] Better handling of the legacy protocol
* [Fix] Check decoded headers sanity (e.g. by excluding \0)
* [Fix] Check for magic when checking for an archive
* [Fix] Cleanup mess with groups
* [Fix] Clickhouse: Insertion in the symbols table
* [Fix] Crash in URL processing
* [Fix] Deal with another case when processing exceptions
* [Fix] Deal with deeply nested messages more aggressively
* [Fix] Deal with nan and inf encoding in json/ucl
* [Fix] Deal with non-key arguments in lua_redis.exec_script
* [Fix] Deal with unknown weight
* [Fix] Deal with URLs with no slashes after protocol
* [Fix] Deal with URLs wrapped in [] in text parts
* [Fix] Deal with zero scores symbols
* [Fix] Default monitoring domain for surbl plugin
* [Fix] Delay upstream re-resolving when one upstream is defined
* [Fix] Detection of maillist optimized and fixed
* [Fix] DKIM signing: allow for auth_only to be false
* [Fix] DMARC: require report_settings for sending reports only
* [Fix] Do not allow garbadge when checking url domain
* [Fix] Do not cache SPF records with PTR elements
* [Fix] Do not constantly re-resolve failed upstreams with a single element
* [Fix] Do not crash if no words defined
* [Fix] Do not crash on empty subtype
* [Fix] Do not expose spamtrap messages to SMTP reply
* [Fix] Do not fail rbl plugin when there are no received or emails
* [Fix] Do not ignore short words
* [Fix] Do not include idempotent/nostat symbols to checksum
* [Fix] Do not override groups when converting metrics
* [Fix] Do not override unix socket group when group comes before owner
* [Fix] Do not skip the last character
* [Fix] Do not spawn too many workers by default
* [Fix] Do not stop monitored on dns errors
* [Fix] Do not stop parsing headers on bad IP header
* [Fix] Do not strip last character in the last word
* [Fix] Do not treat script content as text
* [Fix] Do not try to connect to non-supported addresses
* [Fix] Do not try to dereference last character
* [Fix] Do not try to sign unknown domains
* [Fix] Don't use whitelist/greylist maps as regexp, but as map
* [Fix] Erase unknown HTML entities
* [Fix] Exim Received header protocol parsing
* [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false
* [Fix] Fix a lot of FP in chartable in mixed languages
* [Fix] Fix ANN checks
* [Fix] Fix ANN loading logic
* [Fix] Fix another tokenization issue
* [Fix] Fix autolearn parameters reading
* [Fix] Fix bad archive characters stripping
* [Fix] Fix bad extension check
* [Fix] Fix bayes schema conversion
* [Fix] Fix blacklists and DMARC in whitelist
* [Fix] Fix brain-damaged torch build system
* [Fix] Fix build on FreeBSD
* [Fix] Fix clickhouse exporter
* [Fix] Fix clickhouse schema
* [Fix] Fix comparision
* [Fix] Fix composites processing
* [Fix] Fix connecting to a unix socket in rspamadm statconvert
* [Fix] Fix couple of warnings
* [Fix] Fix crashes in the rspamd_control path
* [Fix] Fix deletion from hash
* [Fix] Fix DKIM forgeries via multiple headers
* [Fix] FIx dynamic conf plugin
* [Fix] Fix emails detection
* [Fix] Fix empty headers simple canonicalization
* [Fix] Fix empty threshold check in greylisting module
* [Fix] Fix encrypted legacy reply in fuzzy storage
* [Fix] Fix enormous scores for R_WHITE_ON_WHITE
* [Fix] Fix exceptions list in surbl
* [Fix] Fix *_EXCESS_BASE64 rules
* [Fix] Fix expire rounding
* [Fix] Fix extra hits in PCRE mode for regular expressions
* [Fix] Fix format strings
* [Fix] Fix get_content method
* [Fix] Fix groups override when defining symbols
* [Fix] Fix learned count in new schema
* [Fix] Fix learn errors propagation
* [Fix] Fix loading of per-user redis backend for statistics
* [Fix] Fix logging buffer corruption in case of repeated messages
* [Fix] Fix lua cached elements invalidation
* [Fix] Fix merging of the implicit arrays
* [Fix] Fix mime_types scoring
* [Fix] Fix multiple headers in DKIM headers list
* [Fix] Fix null callee case in clang plugin
* [Fix] Fix obscured url in format user@@example.com
* [Fix] Fix parsing of the per-user script
* [Fix] Fix priorities in rspamd_update, disable rules execution
* [Fix] Fix processing of closed tags
* [Fix] Fix processing of idempotent rules when autolearn fails
* [Fix] Fix processing of multipart parts with no headers
* [Fix] Fix processing of skip-hashes in fuzzy storage
* [Fix] Fix PTR processing in SPF
* [Fix] Fix pushing country to clickhouse asn table
* [Fix] Fix random forests module
* [Fix] Fix real IP parsing for some strange Exim received
* [Fix] Fix Redis timeout setup
* [Fix] Fix reload crash when hyperscan is enabled
* [Fix] Fix reusing of redis connection after exec
* [Fix] Fix sanity checks on macro value
* [Fix] Fix setting of path and cpath for Lua
* [Fix] Fix setting of signals when spawning a thread
* [Fix] Fix text splitting: stack overflow (too many captures)
* [Fix] Fix ticks processing
* [Fix] Fix upstream addrs updating
* [Fix] Fix urls/emails distinguishing found in queries
* [Fix] Fix user settings check
* [Fix] Fix variable increment
* [Fix] Fix various issues in stat_convert
* [Fix] F-PROT Antivirus infection string for all known occurences
* [Fix] F-PROT Antivirus: only check return code to determine infection
* [Fix] Further fixes around floating point expressions
* [Fix] Further fixes to ANN module
* [Fix] Further fixes to rescore tool
* [Fix] Further fixes to support ES 6
* [Fix] Further tokenization fixes
* [Fix] Greylisting set phase is not idempotent
* [Fix] Handle proxy copy errors
* [Fix] Header checks: Fix get_raw_header method
* [Fix] Header checks: REPLYTO_UNPARSEABLE rule
* [Fix] Kill spawned processes on termination
* [Fix] Load skip map from all processes as shared cache is unavailable
* [Fix] Lowercase HTTP headers to make them searchable from Lua
* [Fix] Lowercase words
* [Fix] Lua_http: freeing
* [Fix] Lua: lpeg to be loaded with rspamd_lua_add_preload, to avoid "rspamd_config_read: rcl parse error: cannot init lua file […] module 'lpeg' not found"
* [Fix] Map absence is not an error
* [Fix] Metadata exporter: check IP sanity
* [Fix] Milter headers: custom headers: removing headers
* [Fix] Milter headers: skip_local / skip_authenticated settings
* [Fix] Milter headers: X-Spamd-Result header if X-Virus ran first
* [Fix] mime_types: fix next-to-last extension length check
* [Fix] More hacks to deal with old configs
* [Fix] Move composites second pass to the dedicated stage
* [Fix] Multimap: received: filtering of artificial header
* [Fix] Multiple fixes in torch based ANN plugins
* [Fix] Once more (#1879) fix bad extension check
* [Fix] Optimize rspamd_fstring_t reallocations
* [Fix] options.local_networks setting
* [Fix] Parse HREF urls without explicit prefix
* [Fix] Plan new event on HTTP errors
* [Fix] Plug another possible memory leak
* [Fix] Plug memory leak
* [Fix] Plug memory leak in lua_tcp
* [Fix] Plug memory leak when setting email addresses from Lua
* [Fix] Propagate learn/stat errors more precisely
* [Fix] Ratelimit: fix whitelisted_rcpts matching
* [Fix] Ratelimit: lowercase email addresses
* [Fix] RBL: received: deal with missing data (#1965)
* [Fix] Rebalance and slightly rework MX check plugin
* [Fix] Redis key expansion: EVAL: deal with strings
* [Fix] Redis script loading in DMARC; URL tags; URL reputation
* [Fix] Reject invalid bh for DKIM signatures earlier
* [Fix] Relax pem signature detection
* [Fix] Relax unicode properties requirements for chartable module
* [Fix] Remove extra noise from dkim and arc signing
* [Fix] Remove hop-by-hop headers in proxy
* [Fix] Remove incorrect method `task:set_metric_subject`
* [Fix] Replace space like characters in headers with plain space
* [Fix] Restore old style ratelimits support
* [Fix] Rework elasticsearch plugin
* [Fix] Rewriting subjects via force actions module
* [Fix] RPM postinstall
* [Fix] Sanitize IP in history redis
* [Fix] Select the correct signature when doing simple canon
* [Fix] Set CLOEXEC flag on files opened
* [Fix] Setting check_local / check_authed in plugins (#1954)
* [Fix] Settings: avoid checking invalid IP (#1981)
* [Fix] Settings: header: deal with multiple settings (#1988)
* [Fix] Skip checks if both extensions are not bad
* [Fix] Skip nostat tokens when get number of tokens
* [Fix] Some more fixes towards emails detection
* [Fix] SpamAssassin: Fail check_freemail_header if regexp didn't match
* [Fix] Stop using of g_slice...
* [Fix] Switch rspamadm logging to message level
* [Fix] Symbol 'FANNR_SPAM' has its score defined..
* [Fix] Table parameter for rspamd_config:add_doc()
* [Fix] Treat 'rewrite subject' as spam action
* [Fix] Try harder in passing IPv6 addresses
* [Fix] Try harder to find rfc822 notifications
* [Fix] Try harder to find urls
* [Fix] Use decoded values when parsing mime addresses
* [Fix] Use full URL when making an HTTP request
* [Fix] Use greylisting threshold in greylisting module
* [Fix] Use n_words attribute from ngramms
* [Fix] Use raw urls when sending requests to redirector
* [Fix] Use the right boolean operator on error check
* [Fix] Use weight from map for fuzzy scoring
* [Fix] Various fixes to elastic plugin
* [Fix] Various fixes to fann_redis instantiation
* [Fix] Various improvements in language detection
* [Fix] Virus infection string for F-PROT Antivirus
* [Fix] Virus infetction string for F-PROT Antivirus
* [Fix] WebUI: use relative path for savemap (#1943)
* [Fix] WHITE_ON_WHITE: Ensure score is matched to part that fired the rule
* [Fix] Write configuration changes as UCL config
* [Project] Add detection logic for words
* [Project] Add fast debug logging infrastructure
* [Project] Add more flags to languages
* [Project] Add n-gramms data files
* [Project] Add ngramms frequencies detector
* [Project] Add random words selection logic
* [Project] Add unigramms to language detection as well
* [Project] Convert all C modules to fast debug infrastructure
* [Project] Detect some languages based on unicode script
* [Project] Enable fast debug lookup for some modules
* [Project] Enable language detector init in scanner workers
* [Project] Further improvements to language detector
* [Project] Implement logic of ngramms application
* [Project] Improve weighting in lang_detection
* [Project] Initialize language detector
* [Project] Preliminary version of ngramms based language detector
* [Project] Preliminary version of the new stat_convert
* [Project] Remove old language detector
* [Project] Rework language detection ngramms structure
* [Project] Start language detection project
* [Project] Start rework of language detection to improve quality
* [Project] Use fast debug logging check
* [Rework] Add frame for new reputation based IP score module
* [Rework] Continue stat_convert rework task
* [Rework] Implement new version of fuzzy replies
* [Rework] Improve readability of xmlrpc API
* [Rework] Kill metrics!11
* [Rework] Ratelimit module
* [Rework] Rename fann_redis to neural plugin
* [Rework] Reorganize mime_types module
* [Rework] Rework rescore utility
* [Rework] Rewrite model and learning logic for rescore
* [Rework] Run post-loads when all initialization is completed
* [Rework] Simplify lua path initialization
* [Rework] Start major stat_convert rework
* [Rework] Start mempool fragmentation reduce project
* [Rework] Start moving of fann redis to torch
* [Rework] Stop embedding rspamadm scripts into C
* [Rework] Use floating point arithmetics in Rspamd expressions
* [Rework] Use frequencies distribution in language detector
* [Rules] Penalise R_BAD_CTE_7BIT for utf8 messages
* [WebUI] Compact graph selectors
* [WebUI] Escape strings inside HTML in history
* [WebUI] Fix message count in throughput summary (#1724)
* [WebUI] Fix NaNs display on Throughput graph
* [WebUI] Migrate widgets to D3 v4
* [WebUI] Restore passwordless login support (#2003)
* [WebUI] Show symbol descriptions as tooltips in history
* [WebUI] Stop using commas in pie chart tooltips
* [WebUI] Update D3 and jQuery
* [WebUI] Update D3Evolution 1.0.0 -> 1.1.0
1.6.6: 16 Feb 2018
* [CritFix] Add sanity guards for badly broken HTML
* [CritFix] Another errors path handling fix
* [CritFix] Fix ARC chain verification
* [CritFix] Fix crash in milter errors handler
* [Feature] Allow to insert headers into specific position
* [Feature] Allow to receive signing keys from mempool vars
* [Feature] Authentication-Results: support hiding usernames
* [Fix] Another try to deal with #1998
* [Fix] Another try to fix #1998
* [Fix] Better handling of the legacy protocol
* [Fix] Check decoded headers sanity (e.g. by excluding \0)
* [Fix] Deal with nan and inf encoding in json/ucl
* [Fix] Deal with URLs wrapped in [] in text parts
* [Fix] DKIM signing: allow for auth_only to be false
* [Fix] Do not crash on empty subtype
* [Fix] Do not fail rbl plugin when there are no received or emails
* [Fix] Do not skip the last character
* [Fix] Do not try to dereference last character
* [Fix] Do not try to sign unknown domains
* [Fix] Exim Received header protocol parsing
* [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false
* [Fix] Fix bad archive characters stripping
* [Fix] Fix comparision
* [Fix] Fix connecting to a unix socket in rspamadm statconvert
* [Fix] Fix empty headers simple canonicalization
* [Fix] Fix extra hits in PCRE mode for regular expressions
* [Fix] Fix parsing of the per-user script
* [Fix] Fix processing of skip-hashes in fuzzy storage
* [Fix] Fix Redis timeout setup
* [Fix] Fix sanity checks on macro value
* [Fix] Fix text splitting: stack overflow (too many captures)
* [Fix] Fix urls/emails distinguishing found in queries
* [Fix] F-PROT Antivirus: only check return code to determine infection
* [Fix] Metadata exporter: check IP sanity
* [Fix] Multimap: received: filtering of artificial header
* [Fix] Plan new event on HTTP errors
* [Fix] Plug another possible memory leak
* [Fix] Remove hop-by-hop headers in proxy
* [Fix] Sanitize IP in history redis
* [Fix] Setting check_local / check_authed in plugins (#1954)
* [Fix] Settings: avoid checking invalid IP (#1981)
* [Fix] Try harder in passing IPv6 addresses
* [Fix] WebUI: use relative path for savemap (#1943)
* [WebUI] Fix message count in throughput summary (#1724)
* [WebUI] Fix NaNs display on Throughput graph
* [WebUI] Restore passwordless login support (#2003)
1.6.5: 22 Oct 2017
* [CritFix] Another portion of tokenization fixes
* [CritFix] Fix memory leak in spf caching logic
* [CritFix] Fix milter commands pipelining
* [CritFix] Fix newlines detection
* [Feature] Filter nan and inf when adding scores
* [Feature] Implement headers flags in mime parser
* [Feature] Support Expires header when using HTTP maps
* [Fix] Actively load skip hashes map in fuzzy storage
* [Fix] Add workaround for IPv6 in sendmail
* [Fix] Authentication Results: Fix SPF smtp.mail_from
* [Fix] Check for magic when checking for an archive
* [Fix] Deal with another case when processing exceptions
* [Fix] Deal with URLs with no slashes after protocol
* [Fix] Do not allow garbadge when checking url domain
* [Fix] Do not ignore short words
* [Fix] Do not strip last character in the last word
* [Fix] Do not treat script content as text
* [Fix] Erase unknown HTML entities
* [Fix] Fix another tokenization issue
* [Fix] Fix DKIM forgeries via multiple headers
* [Fix] Fix emails detection
* [Fix] Fix empty threshold check in greylisting module
* [Fix] Fix enormous scores for R_WHITE_ON_WHITE
* [Fix] Fix loading of per-user redis backend for statistics
* [Fix] Fix multiple headers in DKIM headers list
* [Fix] Fix obscured url in format user@@example.com
* [Fix] Further tokenization fixes
* [Fix] Load skip map from all processes as shared cache is unavailable
* [Fix] Lowercase words
* [Fix] Milter headers: skip_local / skip_authenticated settings
* [Fix] Milter headers: X-Spamd-Result header if X-Virus ran first
* [Fix] Ratelimit: fix whitelisted_rcpts matching
* [Fix] Some more fixes towards emails detection
* [Fix] SpamAssassin: Fail check_freemail_header if regexp didn't match
* [Fix] Use greylisting threshold in greylisting module
1.6.4: 10 Sep 2017
* [Feature] Add method to get all content-type attributes in Lua
* [Feature] Add some sanity checks for actions and controller
* [Feature] Allow randomly select User-Agent from a list
* [Feature] Deal with obscured URLs with @ symbols
* [Feature] Milter headers: support adding/removing arbitrary headers from config
* [Fix] Add another workaround to display history properly
* [Fix] Add missing rspamadm control options to help
* [Fix] Auth-Results: Multiple DKIM signatures
* [Fix] Crash in URL processing
* [Fix] Default monitoring domain for surbl plugin
* [Fix] Detection of maillist optimized and fixed
* [Fix] Do not cache SPF records with PTR elements
* [Fix] Fix blacklists and DMARC in whitelist
* [Fix] Fix exceptions list in surbl
* [Fix] Fix processing of closed tags
* [Fix] Fix PTR processing in SPF
* [Fix] Lowercase HTTP headers to make them searchable from Lua
* [Fix] options.local_networks setting
* [Fix] Ratelimit: lowercase email addresses
* [Fix] Rebalance and slightly rework MX check plugin
* [Fix] Redis script loading in DMARC; URL tags; URL reputation
* [Fix] Reject invalid bh for DKIM signatures earlier
* [Fix] Remove incorrect method `task:set_metric_subject`
* [Fix] Rewriting subjects via force actions module
* [Fix] RPM postinstall
* [Fix] Treat 'rewrite subject' as spam action
* [Fix] Try harder to find urls
* [Fix] Use full URL when making an HTTP request
* [Fix] Use raw urls when sending requests to redirector
* [Fix] Use weight from map for fuzzy scoring
* [Rules] Penalise R_BAD_CTE_7BIT for utf8 messages
1.6.3: 26 Jul 2017
* [CritFix] Fix semicolons parsing in the content type
* [Feature] Add EBL to the default config
* [Feature] Allow to configure monitored
* [Feature] Allow to skip specific hashes in fuzzy storage
* [Feature] Multimap: checking of symbol options
* [Feature] Redis settings: support checking multiple keys
* [Fix] ARC: Fix Lua 5.3 compatibility; timestamp should be integer
* [Fix] Avoid changing content-transfer-encoding header's value
* [Fix] Don't use whitelist/greylist maps as regexp, but as map
* [Fix] Fix get_content method
* [Fix] Header checks: Fix get_raw_header method
* [Fix] Header checks: REPLYTO_UNPARSEABLE rule
* [Fix] Lua_http: freeing
* [Fix] Milter headers: custom headers: removing headers
* [Fix] Parse HREF urls without explicit prefix
* [Fix] WHITE_ON_WHITE: Ensure score is matched to part that fired the rule
* [WebUI] Escape strings inside HTML in history
1.6.2: 08 Jul 2017
* [Conf] Remove Rambler email bl for now
* [Conf] Switch RAMBLER_URIBL to a locally managed source
* [CritFix] Switch from ragel to C for Content-Type parsing
* [Feature] Add `-e` option for lua_repl
* [Feature] Add per-domain emails normalisation rules
* [Feature] Add sessions cache to debug dangling sessions
* [Feature] Add short_text_direct_hash for fuzzy check module
* [Feature] Add text_part:get_stats function
* [Feature] Allow to add custom processing script for surbl
* [Feature] Allow to check reply-to email
* [Feature] Allow to customize spam header, remove existing spam headers
* [Feature] Allow to disable specific workers in the config
* [Feature] Allow to discard messages instead of rejection
* [Feature] Allow to specify custom delimiter in emails plugin
* [Feature] Allow to specify custom User-Agent for rspamc
* [Feature] Allow to store symbols data in Clickhouse
* [Feature] Allow to use HTTPS when connecting to Clickhouse
* [Feature] Enable sessions cache tracking for milter connections
* [Feature] Implement per-line mode in lua_repl (like `perl -p`)
* [Feature] Implement rdns-curve plugin based on rspamd cryptobox
* [Feature] Improve maps cached data lifetime
* [Feature] Improve maps checking frequency
* [Feature] Improve monitored timeouts logic
* [Feature] milter_headers: add `extended_headers_rcpt` option
* [Feature] Milter headers: Add X-Spam-Flag to rmilter-compatibility headers
* [Feature] Milter headers: remove-header routine
* [Feature] Multimap: received filters for extracting TLDs from hostnames
* [Feature] Normalize email aliases in emails module
* [Feature] Re-add rambler email bl (as hashed list)
* [Feature] Reload file maps more frequently
* [Feature] Rework newlines strip parser one more time
* [Feature] Skip updates for messages scanned via controller
* [Feature] Split long DKIM public keys
* [Feature] Store more data when stripping newlines
* [Feature] Support SPF macros transformations
* [Feature] Support suppressing DMARC reports for some domains
* [Fix] Add missing `break` statement
* [Fix] Allow modifiers in SPF macros
* [Fix] DKIM sign tools: edge-cases around use_esld
* [Fix] Do not cache SPF records with macros
* [Fix] Do not overwrite score when setting pre-action
* [Fix] Fix comparison logic
* [Fix] Fix DKIM base64 folding for milter flagged messages
* [Fix] Fix emails module configuration
* [Fix] Fix folding for arc headers when milter interface is used
* [Fix] Fix gmail dots removal
* [Fix] Fix rspamc detection in greylist module
* [Fix] Fix some more issues with HTTP maps
* [Fix] Milter sessions can live forever
* [Fix] Normalize fuzzy probability better
* [Fix] Plug memory leak
* [Fix] RBL: Fixed hashed email address lookups
* [Fix] Try to deal with brain-damaged milter behaviour
* [Fix] Use `\n` to fold headers for milter
* [Rework] Allow to use custom callback for monitored checks
* [Rework] Further steps towards one process monitoring
* [Rework] Send health checks from a single worker
* [WebUI] Round-up throughput summary values
1.6.1: 14 Jun 2017
* [Fix] Allow to init resolver without rspamd_config
* [Fix] Do not crash when resolver failed to initialize
* [Fix] Fix abstract context layout
* [Fix] Fix CGP helper reply parsing
* [Fix] Fix crashes when socket write errors occur
* [Fix] Fix parsing IPv6 nameservers in resolv.conf
* [Fix] Milter: Don't defer on "greylist" action
1.6.0: 12 Jun 2017
* [Conf] Add rspamd_proxy to the default configuration set
* [Conf] Add sample arc module config
* [Conf] Do away with systemd specifics completely
* [Conf] Increase min_bytes to avoid FP
* [Conf] Remove ratelimits from default configuration
* [CritFix] Fix accepting on IPv6 sockets
* [CritFix] Fix corruption when multiple fuzzy are defined
* [CritFix] Fix learn condition in fuzzy check
* [CritFix] Fix memory leak in fuzzy check
* [CritFix] Fix memory leak in maps scheduling
* [CritFix] Paese the last character in DKIM signature correctly
* [CritFix] Zero fill sockaddr_un
* [Feature] Add ability to add doc strings by example
* [Feature] Add API to verify DKIM (and ARC) signatures
* [Feature] Add compression/decompression to proxy
* [Feature] Add count to url structure
* [Feature] Add initial support of the new protocol reply
* [Feature] Add Lua plugin spamtrap
* [Feature] Add `monitored_address` for rbls
* [Feature] Add new schema for bayes tokens
* [Feature] Add preliminary ARC support to dkim code
* [Feature] Add preliminary support of ARC signing
* [Feature] Add rules to detect bad 8bit characters in From and To
* [Feature] Add scanning support for milter protocol
* [Feature] Add support for bidirectional symbols in rspamd_stats
* [Feature] Add support for static maps
* [Feature] Add support of maps with multiple regexps matches
* [Feature] Add `text_multiplier` param
* [Feature] Add the preliminary ARC plugin
* [Feature] Add top redirector targets rank
* [Feature] Allow async events to be registered from LUA rules
* [Feature] Allow storing bayes tokens in Redis
* [Feature] Allow to exclude specific domains from mx check
* [Feature] Allow to have a stack of watcher finalisers
* [Feature] Allow to pass hostname to `-i` flag in Rspamc
* [Feature] Allow to set custom user agent in url redirector
* [Feature] Allow to use custom callback when parsing resolv.conf
* [Feature] Allow to use domain from authenticated user
* [Feature] Bayes expiry plugin
* [Feature] Check dkim sign keys for modifications
* [Feature] DKIM signing: sign_networks/local address specific use_domain settings
* [Feature] DMARC: Support excluding domains from sampling
* [Feature] Expire processing items for URL redirector aggressively
* [Feature] Fix surbl monitored for IP lists, add `monitored_domain` option
* [Feature] Implement caching for dkim body hashes
* [Feature] Implement milter protocol scan reply
* [Feature] Improve omograph phishing detection
* [Feature] Initial support of self-scan in Rspamd proxy
* [Feature] Keep track of headers in milter interface
* [Feature] Milter headers: better controls for local/authenticated
* [Feature] Multimap: email:domain:tld filter
* [Feature] Preliminary DMARC reporting implementation
* [Feature] Reuse stemmers in the cache
* [Feature] Rework confighelp to load Lua plugins
* [Feature] Rework hfilter to use hyperscan if possible
* [Feature] Rework lua RSA API
* [Feature] Rmilter_headers: approximate rmilter's extended_spam_headers
* [Feature] Start integration of milter support in proxy
* [Feature] Store average words length and short words count
* [Feature] Store hash of headers order and names
* [Feature] Support MTA name header
* [Feature] Support multiple types of dkim signing in Lua
* [Feature] Support numeric arguments for Redis requests
* [Feature] Use headers hash in bayes metatokens
* [Feature] Use normal resolv.conf rules of rotation in Rspamd
* [Feature] Use version 2 proto for checking messages
* [Fix] Allow to follow symlinks when safe
* [Fix] Append MX name for authentication results as required
* [Fix] Change default text multiplier from 0.5 to 2.0
* [Fix] Check min_bytes for images as well
* [Fix] Deal with 7bit charsets properly
* [Fix] Deal with 8bit characters in email addresses
* [Fix] Deal with unpaired <a> tags
* [Fix] Detect confighelp in plugins initialisation
* [Fix] Disable certain checks for utf spoof detection
* [Fix] DKIM Signing: avoid nil index when From header is missing
* [Fix] Do not add exact hashes from different parts
* [Fix] Do not check DMARC if SPF or DKIM were not checked
* [Fix] Do not check URLs that are resolved to be redirected
* [Fix] Do not set bayes probability if we don't use it
* [Fix] Do not stop on illegal unicode points - replace them
* [Fix] Fix another race condition in arc checks
* [Fix] Fix arc count logic
* [Fix] Fix ARC signing
* [Fix] Fix brain-damaged spamc protocol for now
* [Fix] Fix calling for peak functions
* [Fix] Fix couple of issues in FORWARDED rule
* [Fix] Fix CTE propagation from parent containers to children parts
* [Fix] Fix errors processing in the controller
* [Fix] Fix format string in milter
* [Fix] Fix issues in SPF macros parsing
* [Fix] Fix logging format string
* [Fix] Fix logic of cached passwords check
* [Fix] Fix lowercasing of stemmed words
* [Fix] Fix LRU elements removal
* [Fix] Fix memory leak when accepting from unix sockets
* [Fix] Fix milter connections persistence
* [Fix] Fix objects merging in UCL
* [Fix] Fix order of operations to avoid race condition
* [Fix] Fix parsing of long regexp types
* [Fix] Fix passing data to log helper when many symbols defined
* [Fix] Fix pools management for milter session
* [Fix] Fix processing of the watchers
* [Fix] Fix queue id macro in milter
* [Fix] Fix R_BAD_CTE_7BIT rule
* [Fix] Fix Redis timeout set
* [Fix] Fix REPLYTO_UNPARSEABLE rule
* [Fix] Fix setting of email address
* [Fix] Fix some more issues about duplicated fuzzy requests
* [Fix] Fix spamc support in rspamd proxy
* [Fix] Fix syntax error in spamtrap plugin
* [Fix] Fix url counts for href urls
* [Fix] Fix url handling in the protocol
* [Fix] Multimap: Received IP filters with Redis
* [Fix] Oops, fix d9d0fa5e86db2f4470d34395a233b450478b2f60
* [Fix] Parse rgb[a](x,x,x[,x]) css colors
* [Fix] Phishing: strict_domains
* [Fix] Reduce maps aggressiveness
* [Fix] Reresolve upstreams even if there is a single server there
* [Fix] Rspamadm grep: Disable Lua patterns in string search by default
* [Fix] Skip text parts when checking binary parts in fuzzy check
* [Fix] Support v2 checks in controller
* [Fix] Treat empty address as valid
* [Fix] Try harder to detect CTE
* [Fix] Try to deal with v4 mapped to v6 addresses on accept
* [Fix] Use dkim signing callback properly
* [Fix] Use non-volatile memory for storing data
* [Fix] Use static maps instead of ugly hack for radix_from_config
* [Fix] Use the same pool for related sessions
* [Rework] Continue modularisation for lua library
* [Rework] Initial milter protocol support
* [Rework] Make log pipes worker agnostic, add scanners API
* [Rework] Move authentication results generation to a separate routine
* [Rework] Move common DKIM functions to a separate lua module
* [Rework] Move global functions to a separate directory
* [Rework] Prepare dkim module for ARC checks
* [Rework] Propagate ucl variables from the command line
* [Rework] Remove multiple metrics support from Rspamd
* [Rework] Stop using name 'rmilter' for the modern protocol
* [Rework] Use LFU algorithm in LRU cache
* [Rules] Fix received TLS rules
* [Rules] Improve URL_COUNT_ODD rule
* [WebUI] Fix add header filter in history
* [WebUI] Use modern protocol for checking messages
1.5.9:
* [Conf] Increase min_bytes to avoid FP
* [Conf] Remove ratelimits from default configuration
* [CritFix] Fix accepting on IPv6 sockets
* [CritFix] Zero fill sockaddr_un
* [Feature] Add `text_multiplier` param
* [Fix] Check min_bytes for images as well
* [Fix] Do not add exact hashes from different parts
* [Fix] Fix memory leak when accepting from unix sockets
* [Fix] Fix some more issues about duplicated fuzzy requests
* [Fix] Phishing: strict_domains
* [Fix] Skip text parts when checking binary parts in fuzzy check
* [Minor] Add the same duplicates protection for all fuzzy hashes types
* [Minor] Fix braces
* [Minor] Fix test
* [Minor] SPOOF_DISPLAY_NAME: Use all SMTP/MIME recipients
* [Minor] Validate assumed spoofed display name domains to contain a dot
1.5.8:
* [CritFix] Fix memory leak in fuzzy check
* [CritFix] Fix memory leak in maps scheduling
* [Feature] Multimap: email:domain:tld filter
* [Fix] DKIM Signing: avoid nil index when From header is missing
* [Fix] Do not set bayes probability if we don't use it
* [Fix] Do not stop on illegal unicode points - replace them
* [Fix] Fix brain-damaged spamc protocol for now
* [Fix] Fix Redis timeout set
* [Fix] Fix spamc support in rspamd proxy
* [Fix] Multimap: Received IP filters with Redis
* [Fix] Parse rgb[a](x,x,x[,x]) css colors
* [Fix] Reresolve upstreams even if there is a single server there
* [Fix] Treat empty address as valid
* [Fix] Try harder to detect CTE
* [Fix] Try to deal with v4 mapped to v6 addresses on accept
* [Minor] Add `wsf` and `hta` bad archive extensions
* [Minor] Fix configuration option
* [Minor] Fix result parsing for SAVAPI
* [Minor] Further logging improvements
* [Minor] Improve logging of errors
* [Minor] Prevent MID_CONTAINS_FROM from firing on empty address
* [Minor] Reduce digit->number transmission penalty
* [Minor] Relax CTYPE_MISSING_DISPOSITION rule
1.5.7:
* [CritFix] Fix corruption when multiple fuzzy are defined
* [CritFix] Fix learn condition in fuzzy check
* [Feature] Add rules to detect bad 8bit characters in From and To
* [Feature] DKIM signing: sign_networks/local address specific use_domain settings
* [Feature] Support numeric arguments for Redis requests
* [Fix] Deal with 8bit characters in email addresses
* [Fix] Fix couple of issues in FORWARDED rule
* [Fix] Fix passing data to log helper when many symbols defined
* [Fix] Fix R_BAD_CTE_7BIT rule
* [Fix] Fix REPLYTO_UNPARSEABLE rule
* [Fix] Fix setting of email address
* [Fix] Rspamadm grep: Disable Lua patterns in string search by default
* [Minor] Add Lua 5.3 workaround
* [Minor] Add lua methods to detect if a part has 8bit characters
* [Minor] Allow session-less lua dns requests
* [Minor] Allow to append greylist end time to message reported
* [Minor] Avoid `nil` table
* [Minor] Disable dkim_signing if redis is specified but not configured
* [Minor] Fix build with pcre2
* [Minor] Fix rule
* [Minor] Fix warnings
* [Minor] Format floating point number
* [Minor] Push email flags to the lua API
* [Minor] Silence some warnings
* [Minor] Silence warning
* [Minor] Try all hostname regexps to find the most significant one
* [WebUI] Fix add header filter in history
1.5.6:
* [Feature] Add unigramms support in bayes
* [Feature] Allow configurable sign headers for DKIM
* [Feature] Allow to add unigramm metatokens from Lua
* [Feature] DKIM Signing: envelope match exception for local IPs
* [Feature] UCL: register parser variables from Lua
* [Fix] Always try to adjust filename
* [Fix] Do extra copy to ensure that original content is never touched
* [Fix] Fix SPOOF_REPLYTO rule
* [Fix] Ignore Rmilter added Received
* [Fix] More fixes for hashed email dnsbls
* [Fix] Plug memory leak in chartable module
* [WebUI] Display multiple alerts at once
1.5.5:
* [CritFix] Fix classifier learning with Redis backend
* [CritFix] Fix issue when parsing encoded rfc822/messages
* [Feature] Add escaped version of lua_ucl import
* [Feature] Add task:headers_foreach function
* [Feature] Allow to process filenames from content type
* [Feature] Allow to query hashed emails
* [Feature] Ignore bayes with mostly metatokens or with too few text
* [Feature] Probabilistically skip metatokens
* [Feature] Retrieve all virus names from SAVAPI
* [Feature] Rework classifiers lua metatokens
* [Feature] Store headers order
* [Feature] Store text tokens inside bayes tokens
* [Feature] Use cached shingles keys
* [Fix] Add missing score normalisation for HFILTER_URL_ONLY
* [Fix] Avoid lookup in absent hash
* [Fix] Check return values from Lua functions called from C
* [Fix] Do not count sending and loading time in rspamc
* [Fix] Escape json strings for controller rejplies from Lua
* [Fix] Fix archive scans for savapi
* [Fix] Fix domain_only emails RBL
* [Fix] Fix ip_score map configuration
* [Fix] Fix JSON output for history_redis
* [Fix] Fix one character length substrings search
* [Fix] Fix parsing of non-RFC compatible Exim received
* [Fix] Fix parsing of options for workers with the same type
* [Fix] Fix processing of small tokens vectors
* [Fix] Fix rfc2047 tokenization
* [Fix] Fix typo
* [Fix] More fixes for inplace decoding
* [Fix] Try to avoid modifications of the original data
* [Fix] URL redirector: Fix call to is_redirector
* [Rework] Set token data as uint64_t instead of chars array
* [WebUI] Check if neighbours' history backend versions match
* [WebUI] Disable phrase connectors replacement in history filtering
* [WebUI] Disable phrase connectors replacement in symbols filtering
* [WebUI] Do not hide messages with bad subject, just replace it with '???'
* [WebUI] Fix error message
* [WebUI] Fix history v2 display
* [WebUI] Fix legacy history
* [WebUI] history: break To address lists on commas
* [WebUI] Increase default timeout to 20 seconds
* [WebUI] Save some history table space
1.5.4:
* [Conf] Add history_redis default configuration
* [Feature] Add spoofed rules
* [Feature] Add URL_IN_SUBJECT rule
* [Feature] Allow to get task's subject
* [Feature] Allow to specify maximum number of shots for symbols
* [Feature] Distinguish URLs found in Subject
* [Feature] Memoize LPEG grammars
* [Feature] Parse else parts in SA rules
* [Feature] Process subject for mixed characters
* [Feature] Resolve url chains in url_redirector module
* [Feature] Stat greylisted messages as greylisted not soft-rejected
* [Feature] Support checking for redirector in Lua SURBL
* [Feature] Support tag_exists SA function
* [Feature] Work with broken rfc2047 tokens
* [Fix] Check all watcher's dependencies
* [Fix] Do not compile hyperscan with no SSSE3 support
* [Fix] Do not crash if cannot decode qp encoded part
* [Fix] Fix dependencies of DKIM when multiple signatures are found
* [Fix] Fix lists in whitelist plugin
* [Fix] Fix one-shot symbols weight calculations
* [Fix] Fix options and shots match
* [Fix] Fix order of symbol options
* [Fix] Fix parsing of dot at the end of the address
* [Fix] Fix parsing of lua table arguments
* [Fix] Fix processing of subject words
* [Fix] Fix string split memoization
* [Fix] Fix templates grammar usage
* [Fix] Fix various issues related to Lua stack manipulation
* [Fix] Force actions: Use postfilter if we have honor_action / require_action
* [Fix] Further fixes to avoid PHISHING FP
* [Fix] Preserve order of options in symbols
* [Fix] Rspamadm grep: deal with unusually-formatted logs
* [Fix] Use hostname suffix when dealing with history
* [Rework] Remove outdated SA rules
* [WebUI] Add flexible columns
* [WebUI] Add footable
* [WebUI] Add sender, recipients and subject columns
* [WebUI] Allow message-id break
* [WebUI] Fix history clustering
* [WebUI] Fix history display
* [WebUI] Fix sorting
* [WebUI] Humanize sizes
* [WebUI] Initial move towards footable
* [WebUI] Remove datatables
* [WebUI] Replace `.values` method with `.map`
* [WebUI] Rework v2 symbols display
* [WebUI] Try to normalize frequencies
* [WebUI] Unbreak WebUI
* [WebUI] Use Footable to draw Throughput summary table
1.5.3:
* [Conf] Add composite for hacked wordpress phishing
* [CritFix] Fix base64 decoding when there are unparseable characters
* [Feature] Additional symbol metadata in metadata exporter
* [Feature] Add method to get protocol reply from Lua
* [Feature] Add symbols when tagged rcpt/sender are normalised
* [Feature] Add task:get_symbols_all() function
* [Feature] Allow multiple formats of DKIM signing key
* [Feature] Allow to cache and use flexible protocol reply
* [Feature] Allow to set one_shot flag from register_symbol
* [Feature] Allow to skip certain types of hashes when learning fuzzy
* [Feature] Cache and insert scan time into the protocol
* [Feature] Detect newlines in rspamc --mime
* [Feature] DKIM signing: support use of maps
* [Feature] Greylist: Support excluding low-scoring messages from greylisting
* [Feature] Implement lua history in controller
* [Feature] Implement redis history querying
* [Feature] Preliminary implementation of redis history plugin
* [Feature] Support using request headers in settings
* [Fix] Change default template to deal with non-ASCII characters
* [Fix] Deal with lists of maps in whitelist module
* [Fix] DKIM signing: use domain-specific signing key
* [Fix] Do not reallocate completed zstd buffer
* [Fix] Do not use local_addrs in proxy
* [Fix] Fix crash when resolver is undefined
* [Fix] Fix double free when closing lua_tcp connections
* [Fix] Fix for lua 5.3
* [Fix] Fix freeing of arrays iterators
* [Fix] Fix issue with task:get_symbol and symbols with no metric
* [Fix] Fix log line duplication in `rspamadm grep`
* [Fix] Fix memory corruption on termination
* [Fix] Fix out-of-bound access in base64 decode
* [Fix] Fix ratelimit + greylisting
* [Fix] Fix subject rewriting
* [Fix] Fix task:set_recipients function
* [Fix] Fix URI_COUNT_ODD rule
* [Fix] Follow the traditional symbols conventions in RCPT_COUNT rule
* [Fix] Greylist: Suppress greylist action for whitelisted hosts too
* [Fix] Metadata exporter: use rule-specific settings for emails
* [Fix] Properly set missing fields in exporter
* [Fix] Proxy: max_retries option
* [Fix] RCPT_COUNT fixes
* [Fix] Rework HAS_X_PRIO rule to match symbols conventions
* [Fix] Update issues in ac-trie
* [Fix] Use optimised base64 decoding in DKIM
* [WebUI] Add preliminary v2 history parser
* [WebUI] Allow different history parsers
* [WebUI] Display symbols
* [WebUI] Rework history v2 function
1.5.2:
* [Conf] Add default config for spamassasssin plugin
* [Conf] Add default configuration for antivirus module
* [Conf] Add dkim signing docs
* [Conf] Add mx_check default config
* [Conf] Add replies config
* [Conf] Add trie default config
* [Feature] Add heuristic to find text parts in files
* [Feature] Add rule to detect broken content type
* [Feature] Allow to extract CTE in Lua API
* [Feature] Allow to set from address for a lua_task
* [Feature] Allow to set recipients of a task from Lua
* [Feature] Enchance text_part:get_content method
* [Feature] Remove + aliases from emails
* [Feature] Support rmilter block and dkim signature in CGP helper
* [Feature] Support running event loop from Lua
* [Fix] Antivirus: use scanner-specific redis prefix
* [Fix] Couple of fixes for DKIM signing module
* [Fix] Distinguish missing and broken mandatory headers
* [Fix] Do more heuristical detection for missing CTE
* [Fix] Do not resort cache on each check
* [Fix] Fix CGP escaping
* [Fix] Fix MISSING_MIME_VERSION rule for plain messages
* [Fix] Fix parsing of cte in expressions
* [Fix] Fix partial matches in rspamadm grep
* [Fix] Fix setting class on style field
* [WebUI] Auto-switch Throughput units to `msg/min` for very low rate
* [WebUI] Update D3Evolution to 0.0.2
1.5.1:
* [CritFix] Fix processing of stop_patterns with `\0` character
* [CritFix] Fix setting of raw key for signing
* [Fix] Fix lua exports from plugins during reload
* [Fix] Fix prefilters action scores
* [Fix] Fix symbols processing order
* [Minor] Help cmake find gthread
* [Minor] Some cmake fixes
1.5.0:
* [Conf] Add configurations for asn, clickhouse and dcc
* [Conf] Add default config for url redirector plugin
* [Conf] Add the default config for greylist module
* [Conf] Allow to edit all local maps from WebUI by default
* [CritFix] Deal with absent headers in DKIM
* [CritFix] Do not trust remote shingles count
* [CritFix] Fix bad memory leak in TLS certificates validation
* [CritFix] Fix critical memory issues with radix maps
* [CritFix] Fix descriptors leak on reload
* [CritFix] Fix headers selection in DKIM verification
* [CritFix] Fix parsing of boundaries that end with `--`
* [CritFix] Repair PTR_ARRAY_FOREACH macro
* [Feature] Add CORS support to the controller
* [Feature] Add FROM_NAME_EXCESS_SPACE rule
* [Feature] Add REPLYTO_EMAIL_HAS_TITLE rule
* [Feature] Add `caseless_hash` method to `lua_util`
* [Feature] Add `rip` keyword to ratelimit module
* [Feature] Add a simple benchmark for content type parsing
* [Feature] Add boundaries parsing in content type
* [Feature] Add charset detection for text parts
* [Feature] Add content disposition parser
* [Feature] Add fallback if too many updates are failing
* [Feature] Add function to convert struct tm to time using timezone
* [Feature] Add function to normalize HTTP paths
* [Feature] Add fuzzy collection plugin
* [Feature] Add fuzzy logic for images
* [Feature] Add gmime parser to mime_tool
* [Feature] Add heuristic to detect broken messages
* [Feature] Add heuristic to find displayed URLs
* [Feature] Add heuristic to process broken email addresses
* [Feature] Add images normalization
* [Feature] Add mechanism for disabling composites (Fixes #1270)
* [Feature] Add method to create regexp from a glob pattern
* [Feature] Add mime encoding manipulation routines
* [Feature] Add mime tool to explore messages
* [Feature] Add more meta tokens from received headers
* [Feature] Add neighbours option to support Rspamd cluster in WebUI
* [Feature] Add new function to parse mime addresses
* [Feature] Add new methods for lua_tcp
* [Feature] Add own headers decoding routine
* [Feature] Add own routine to generate a message id
* [Feature] Add parser for SMTP date
* [Feature] Add per-task lua cache to reuse 'heavy' objects
* [Feature] Add plugins list path in WebUI
* [Feature] Add preliminary multipart support
* [Feature] Add preliminary version of DKIM signing module
* [Feature] Add profiling support in client output
* [Feature] Add rfc2047 grammar
* [Feature] Add rfc2047 variant for QP decoding
* [Feature] Add rmilter_headers module (Fixes #1227)
* [Feature] Add sse42 version of base64 decoding
* [Feature] Add ssse3 and avx2 base64 decoders
* [Feature] Add support of libgd
* [Feature] Add the preliminary version of redirects resolver in Lua
* [Feature] Add ucl_object_iterate_full function
* [Feature] Add url encoding function
* [Feature] Allow SOA requests in lua dns
* [Feature] Allow custom parse types in lua ucl
* [Feature] Allow plugins to register webui handlers
* [Feature] Allow to add options explicitly to symbols
* [Feature] Allow to call a callback when symbol frequency is on peak
* [Feature] Allow to call redirector script from SURBL
* [Feature] Allow to create variable length dkim keys
* [Feature] Allow to have module specific options for Redis in plugins
* [Feature] Allow to pass sign key directly from Lua
* [Feature] Allow to register configuration docs from Lua API
* [Feature] Allow to return options as a table
* [Feature] Allow to set peak callbacks from Lua
* [Feature] Allow to specify custom method for a message
* [Feature] Allow to store dkim keys in Redis
* [Feature] Allow to store messages in files
* [Feature] Apply DCT using AAN for fuzzy signature
* [Feature] Avira SAVAPI support
* [Feature] Cache and simplify DCT and jpeg decode
* [Feature] Cache libicu converters
* [Feature] Detect URLs with suspicious omographs
* [Feature] Do not increase score for duplicate options
* [Feature] Do not trust CTE, check base64 and qp strictly
* [Feature] Dynamic reputation in URL reputation plugin
* [Feature] Extend redis lock when learning spawned
* [Feature] Filter non-utf chars from all decoded headers
* [Feature] Fix phishing detection for IDNA urls
* [Feature] Ignore bad symbols on base64 decoding
* [Feature] Ignore too wide elements in SPF
* [Feature] Implement fuzzy collection mode
* [Feature] Implement helo maps in multimap
* [Feature] Implement human readable buckets configuration
* [Feature] Implement min-hash shingles for DCT data from images
* [Feature] Implement new algorithm for fuzzy hashes of images
* [Feature] Implement new unicode normalizer
* [Feature] Implement quoted printable decoding
* [Feature] Implement received headers flags
* [Feature] Implement rspamdgrep tool
* [Feature] Implement sane checksum for config file
* [Feature] Implement url tags concept
* [Feature] Improve detection of omographs using libicu
* [Feature] Improve url redirector module
* [Feature] Multimap: Received header processing
* [Feature] Multiple improvements in the maps
* [Feature] New URL filters in multimap
* [Feature] Plugin to force actions on selected symbols
* [Feature] RBL module: support hashing for emails and helo RBL
* [Feature] Reuse URL tags in SURBL module
* [Feature] Rework RRD ds count, add conversion path
* [Feature] Rework surbl module to avoid extra redirector calls
* [Feature] Send config id to the WebUI
* [Feature] Simplify HTTPCrypt client support
* [Feature] Skip processing for large images
* [Feature] Start collection only mode implementation for fuzzy storage
* [Feature] Start import of the optimized base64 decode
* [Feature] Store all received headers in lua
* [Feature] Store relational order of all headers in a message
* [Feature] Support DKIM signing in Lua plugins
* [Feature] Support HTTPCrypt client in lua_http
* [Feature] Support setting SMTP message in multimap
* [Feature] Support setting metric subject from Lua
* [Feature] Support setting subject in force actions module
* [Feature] Treat v6 mapped addresses as v4 addresses
* [Feature] URL reputation plugin
* [Feature] Use Redis instead of memcached in URLs redirector
* [Feature] Use Rspamd rfc2047 decoder instead of gmime one
* [Feature] Use a different normalization for fuzzy images
* [Feature] Use normalized images in fuzzy hashes
* [Feature] Use own code for parsing of date
* [Feature] Use shingles for images fuzzying
* [Feature] Use t1ha for hashes, allow inlining
* [Feature] Use t1ha instead of metrohash and xxhash32
* [Feature] Various new features in metadata exporter module
* [Feature] rmilter_headers: authentication-results (#78)
* [Fix] Add additional check to mark redis connection inactive
* [Fix] Add packed attribute for protocol structure
* [Fix] Adopt OMOGRAPH_URL rule
* [Fix] Allow static maps
* [Fix] Allow to disable classifiers checks using settings and conditions
* [Fix] Another try to fix 0 length maps
* [Fix] Another try to fix corruption during maps reload
* [Fix] Another try to fix descriptors leak
* [Fix] Another try to fix reload and logger
* [Fix] Antivirus module: register virtual symbols for patterns
* [Fix] Avoid extensive reallocs
* [Fix] Avoid mempool leak in SA plugin on reload
* [Fix] Avoid race condition on saving cache and reload
* [Fix] Avoid reusing g_error (Fixes #1262)
* [Fix] Break pool connection on fatal redis errors
* [Fix] Check for NaN properly
* [Fix] Couple of fixes for date parsing
* [Fix] Date header timezone adjustments (#1279)
* [Fix] Deal with EOF properly
* [Fix] Decode filename in content disposition
* [Fix] Disable fuzzy images by default
* [Fix] Disable zero-copy mode for text parts to avoid crashes
* [Fix] Do not destroy session when not all finish scripts are done
* [Fix] Do not greyscale images
* [Fix] Do not leave parent-less workers processes on fatal errors
* [Fix] Do not lowercase Content-Disposition to perform decoding
* [Fix] Do not penalize characters just after numeric prefix
* [Fix] Do not refork workers that are intended to die
* [Fix] Do not set pre-result and update records for no `Queue-ID` messages
* [Fix] Do not skip post-filters when pre-filters have set some results
* [Fix] Do not stop symbols planning if async events are pending
* [Fix] Do not try to set keys for unencrypted requests in proxy
* [Fix] Encode URLs according to rfc3986
* [Fix] Encode URLs before sending them to the protocol
* [Fix] Filter bad characters from message id
* [Fix] Fix CTE detection heuristic
* [Fix] Fix Content-Type in HTTP requests
* [Fix] Fix IDN eslds phishing checks
* [Fix] Fix adding maps from config in Lua
* [Fix] Fix another reload memory issue
* [Fix] Fix argument returned on redis backend errors
* [Fix] Fix assertion in graph handling
* [Fix] Fix body trie matching
* [Fix] Fix build
* [Fix] Fix byte array expansion during toutf8 conversion
* [Fix] Fix charset normalisation
* [Fix] Fix checking of DKIM bodies that needs just `\n` to be added
* [Fix] Fix couple of cornercases with email addresses
* [Fix] Fix couple of issues
* [Fix] Fix dependencies tracking for callback symbols
* [Fix] Fix detection of jpeg size
* [Fix] Fix errors handling in fuzzy backend initialization
* [Fix] Fix fuzzy hashes count
* [Fix] Fix globbing and convert lists to arrays in fuzzy_check
* [Fix] Fix heuristical CTE detection for QP encoding
* [Fix] Fix ignoring of bad text parts
* [Fix] Fix indexes in array access, interleave loop
* [Fix] Fix int64 -> double conversion
* [Fix] Fix invalid memory access on reload
* [Fix] Fix issues with empty updates
* [Fix] Fix issues with quoted-printable encoding
* [Fix] Fix keys names
* [Fix] Fix lots of issues in mime parser code
* [Fix] Fix lua maps load
* [Fix] Fix macro name
* [Fix] Fix mas group score calculations
* [Fix] Fix matching of the same patterns from different tries
* [Fix] Fix memory corruprtion and leak
* [Fix] Fix memory leak in HTTP maps
* [Fix] Fix memory leak in expression destroying
* [Fix] Fix memory leak in parsing of mime names
* [Fix] Fix memory leak in safe ucl iterators
* [Fix] Fix memory leak on reload in plugins
* [Fix] Fix modules reconfigure on reload
* [Fix] Fix monitored setup fro URLBLs with IP addresses
* [Fix] Fix name of var
* [Fix] Fix new rrd updates
* [Fix] Fix out of bounds access
* [Fix] Fix parsing messages with no body
* [Fix] Fix parsing of '=' character in headers
* [Fix] Fix parsing of messages with no content type
* [Fix] Fix plugins callbacks in webui
* [Fix] Fix possible memory corruption in redis pool
* [Fix] Fix probability calculations for fuzzy redis backend
* [Fix] Fix processing errors in lua_tcp
* [Fix] Fix processing of emails with name only
* [Fix] Fix processing of non-multipart messages
* [Fix] Fix processing of parts with no valid content type
* [Fix] Fix race condition in SIGUSR2 handler
* [Fix] Fix redis options parsing when no redis servers are defined
* [Fix] Fix reload and hyperscan ready event
* [Fix] Fix reload memory issue
* [Fix] Fix rra_ptr conversion
* [Fix] Fix rrd file conversion
* [Fix] Fix setting of content-type attributes
* [Fix] Fix signing headers creation in DKIM
* [Fix] Fix stddev calculations
* [Fix] Fix surbl plugin to work with composite maps
* [Fix] Fix timezones parsing
* [Fix] Fix tokens usage
* [Fix] Fix urls and emails hashes
* [Fix] Fix usage of unsafe ucl iterators
* [Fix] Fix work with broken utf8 tokens
* [Fix] Fix writing of user to roll history
* [Fix] Forgotten worker
* [Fix] Further memory leaks fixes
* [Fix] Ignore lua metatokens in bayes for now
* [Fix] Improve OMOGRAPH_URL rule
* [Fix] Lua IP from string should be invalid if parsing failed
* [Fix] Miltiple fixes to new lua_tcp, add debugging
* [Fix] More fixes for iterators cleanup
* [Fix] More fixes to logger initialization
* [Fix] More heuristic fixes for phishing detection
* [Fix] More leaks eliminated
* [Fix] More leaks...
* [Fix] More random fixes for reload...
* [Fix] Multimap: Fixes for email filters
* [Fix] Multiple fixes for fann module
* [Fix] Multiple memory corruption fixes
* [Fix] Normalize path in HTTP router
* [Fix] Plug memory leak
* [Fix] Plug memory leak in adding radix trees
* [Fix] Plug memory leak in configuration parser
* [Fix] Plug memory leak in expressions parsing during reload
* [Fix] Plug memory leak in learning fuzzy storage
* [Fix] Plug memory leak in lua_tcp
* [Fix] Plug reload leaks
* [Fix] Plug termination memory leaks
* [Fix] Really increase lock lifetime
* [Fix] Replies module: fix symbol weight
* [Fix] Restore content type params related functions
* [Fix] Set task's subject from mime subject
* [Fix] Sigh, one more reload leak
* [Fix] Simplify images shingles
* [Fix] Some more memory issues are fixed
* [Fix] Stop hardcoding of lua in C
* [Fix] Stop processing of bad parts as text parts
* [Fix] Strictly filter bad characters when emittin json
* [Fix] Strings returned from lua are ephemeral
* [Fix] Support unix sockets for lua redis
* [Fix] Try to fix issues with reloading config
* [Fix] Try to fix race condition in redis_pool
* [Fix] Use checksum to avoid intersection between different ANNs
* [Fix] Use rspamd hashes in embedded ucl
* [Fix] Use sane default rewrite subject (*** SPAM *** %s)
* [Fix] Various collection mode fixes
* [Fix] Various fixes to mime parser
* [Fix] Various reload leak fixing
* [Fix] Whitelist certain extensions from archive checks
* [Rework] Add preliminary implementation of the mime parser
* [Rework] Adopt code for the new options
* [Rework] Change logger setup interface
* [Rework] Composite configuration (#1270)
* [Rework] Finally remove gmime dependency from Rspamd
* [Rework] Further fixes to symbols frequencies
* [Rework] Implement content type parser for mime
* [Rework] Kill all InternetAddressList usages
* [Rework] Multiple fixes for symbols cache statistics
* [Rework] Refactor struct names
* [Rework] Rework images fuzzy hashes algorithm
* [Rework] Rework lua_tcp to allow TCP dialog
* [Rework] Start massive rework to get rid of gmime
* [Rework] Start new approach for multiparts parsing
* [Rework] Start rework of mime addresses
* [Rework] Start rework of symbols cache updates
* [Rework] Start switching to libicu
* [Rework] Use a special structure for stats tokens
* [Rework] Use hash tables for symbols options
* [Rework] Use libicu instead of iconv for conversions
* [Rework] Use new scheme to parse mime parts
* [WebUI] Add Access-Control-Allow-Origin for cluster management
* [WebUI] Add Throughput graph autorefreshing (#820)
* [WebUI] Add Visibility.js library
* [WebUI] Add basic cluster support to Throughput tab
* [WebUI] Add graph legend entries for new DSes
* [WebUI] Add graph tab
* [WebUI] Add neighbours RRD data consolidation
* [WebUI] Add preliminary save symbols clustering
* [WebUI] Add server selector to navbar
* [WebUI] Add soft reject to auth stats
* [WebUI] Add summary to the Throughput tab
* [WebUI] Allow to save maps on the cluster
* [WebUI] Avoid extra graph redraw and alerts glitching
* [WebUI] Be more generous with AJAX timeout
* [WebUI] Disable error ring loading in `read only` mode
* [WebUI] Enclose table header cells with `tr`s
* [WebUI] Finish interface rework
* [WebUI] Fix RRD summary pie chart position
* [WebUI] Fix `All SERVERS` graph fot just one available server
* [WebUI] Fix case when no cluster is defined
* [WebUI] Fix compatibility with non-ES6 compliant browsers
* [WebUI] Fix config ID
* [WebUI] Fix configuration page partially
* [WebUI] Fix disabled state
* [WebUI] Fix graph dataset selector initialization
* [WebUI] Fix graph selectors state resetting
* [WebUI] Fix mouse events on throughput summary table area
* [WebUI] Fix multiple JS issues
* [WebUI] Fix pie chart displaying
* [WebUI] Fix read only
* [WebUI] Fix read only2
* [WebUI] Fix retarded datatables
* [WebUI] Fix soft reject in pie chart
* [WebUI] Fix stat widgets timers multiplication on `Refresh` click
* [WebUI] Fix symbols config
* [WebUI] Fix various errors with login form
* [WebUI] Further fixes
* [WebUI] Hide learning tab in read-only mode
* [WebUI] Initial clusters support
* [WebUI] Make legend entry colours more contrast
* [WebUI] Move configuration tab to a separate module
* [WebUI] Move history tab
* [WebUI] Move symbols config as well
* [WebUI] New sec to time function
* [WebUI] Prevent multiple clicks on `Refresh`
* [WebUI] RRD summary: Hide inner labels of tiny pie sectors
* [WebUI] RRD summary: Respect undefined values
* [WebUI] Reduce font size of graph's legend
* [WebUI] Remove orphaned font duplicates
* [WebUI] Remove unused code
* [WebUI] Replace spinner with animated glyphicon
* [WebUI] Reset refresh timer on server switching
* [WebUI] Rework interface to use requirejs
* [WebUI] Rework neighbours query function
* [WebUI] Separate attributes by space
* [WebUI] Set focus to password field (#1230)
* [WebUI] Simplify neighbours table populating
* [WebUI] Start rework of modules
* [WebUI] Stop stats refreshing if the page is hidden
* [WebUI] Turn d3pie's stuff into a reusable function,
* [WebUI] Unify send data functions
* [WebUI] Update D3Evolution to 0.0.1
* [WebUI] Update d3.js
* [WebUI] Update datatables to work with the requirejs
* [WebUI] Use unified tab click event handler,
* [WebUI] clusters for the chart
* [WebUI] fix uptime
1.4.2:
* [CritFix] Deal with absent headers in DKIM
* [CritFix] Do not trust remote shingles count
* [CritFix] Fix headers selection in DKIM verification
* [Feature] Add EXT_CSS rule
* [Feature] Add toggle for disabling SURBLs
* [Feature] Extend redis lock when learning spawned
* [Feature] Parse <link> HTML tags
* [Fix] Avoid reusing g_error (Fixes #1262)
* [Fix] Do not reset loaded ANN when learning is requested
* [Fix] Fix another issue with external deps in SA
* [Fix] Fix body trie matching
* [Fix] Fix checking of DKIM bodies that needs just `\n` to be added
* [Fix] Fix fuzzy hashes count
* [Fix] Fix keys names
* [Fix] Fix length calculations for url encoded urls
* [Fix] Fix matching of the same patterns from different tries
* [Fix] Fix name of var
* [Fix] Fix parsing of URLs with spaces and other bad chars
* [Fix] Fix probability calculations for fuzzy redis backend
* [Fix] Fix signing headers creation in DKIM
* [Fix] Plug memory leak
* [Fix] Really fix chained SA dependencies
* [Fix] Really increase lock lifetime
* [Fix] Use checksum to avoid intersection between different ANNs
* [Fix] Use rspamd hashes in embedded ucl
* [Fix] Yet another change for testing external deps
1.4.1:
* [Feature] ASN support in Clickhouse module
* [Feature] Add clickhouse plugin
* [Feature] Add generic tool to add universal maps for lua modules
* [Feature] Add logger.debugm to debug lua modules
* [Feature] Allow to register metrics symbols using register_symbol
* [Feature] Allow to specify prefix for fann_redis
* [Feature] Clickhouse: support different masks for IPv4/IPv6
* [Feature] Support forcing action in antivirus plugin
* [Fix] Add handling of regexp maps
* [Fix] Allow backslashes in http urls
* [Fix] Avoid mapping of empty files
* [Fix] Do not load tld file to speed up rspamadm
* [Fix] Do not resolve numeric IP addresses due to ipv6 insanity
* [Fix] Filter incorrect training data
* [Fix] Fix Fuzzyconvert tool when password or DB is given
* [Fix] Fix build with custom glib/gmime
* [Fix] Fix converting of learn count from sqlite to redis
* [Fix] Fix crashes with invalid received and task:set_from_ip
* [Fix] Fix external dependencies for SA module
* [Fix] Fix fann_redis when number of scores has been changed
* [Fix] Fix hyperscan usage for non compatible platforms
* [Fix] Fix loading of maps from UCL objects
* [Fix] Fix memory leak for task-less redis requests
* [Fix] Fix mid module with new maps syntax
* [Fix] Fix parsing of URLs with username
* [Fix] Fix re cache initialisation
* [Fix] Fix replacements to sanitize '%' character
* [Fix] Fix set and regexp like static maps
* [Fix] Fix some issues in redis settings
* [Fix] Fix static IP maps
* [Fix] Fix total learns counter for redis stats
* [Fix] Fix usage of config during reload
* [Fix] Fix various warnings and issues
* [Fix] Invalidate ANN if training data is incorrect
* [Fix] Miltiple fixes to fann_redis module
* [Fix] More fixes for URLs with backslashes
* [Fix] Properly get options for ip_score module
* [Fix] Relax requirements for Received as gmail cannot RFC
* [Fix] Remove or fix hyperscan incompatible regexps
* [Fix] Settings: correctly read redis config
* [Rework] Rework lua logger interface slightly
* [Rework] Use new maps add function
1.4.0:
* [CritFix] Add guards for inactive redis connections
* [CritFix] Another fix for proxying files using rspamd_proxy
* [CritFix] Cleanup inactive redis connections
* [CritFix] Do not sometimes try to exec posfilters before classification
* [CritFix] Fix application of IPv6 mask
* [CritFix] Fix chunked encoding when reading messages
* [CritFix] Fix file mode for rspamd_proxy
* [CritFix] Fix hyperscan compilation on regexp change
* [CritFix] Fix issue with finding of end of lines pointers
* [CritFix] Fix iteration over headers array (introduced in 1.4)
* [CritFix] Fix processing of learned tokens count for redis backend
* [CritFix] Fix race condition in checking of cached maps
* [CritFix] Fix workers scripts by sharing workers configs
* [CritFix] Introduce raw content to text parts
* [CritFix] Plug memory leak and potential memory corruption
* [Feature] Adaptive ratelimits
* [Feature] Add ASN -> rbldnsd script for asn.rspamd.com
* [Feature] Add DMARC_NA symbol
* [Feature] Add F-Prot support to antivirus module
* [Feature] Add HTTP backend to metadata exporter
* [Feature] Add Lua API module for monitored objects
* [Feature] Add R_DKIM_NA / R_SPF_NA / AUTH_NA symbols
* [Feature] Add R_DKIM_PERMFAIL symbol
* [Feature] Add R_SPF_PERMFAIL symbol
* [Feature] Add Sophos antivirus support
* [Feature] Add ZSTD compression to Lua API
* [Feature] Add `mid` Lua module
* [Feature] Add `one_param` flag for metric symbols
* [Feature] Add a generic lua classifier
* [Feature] Add a very basic interface to access workers data from on_load
* [Feature] Add ability to delete a hash by its data to fuzzy_check plugin
* [Feature] Add ability to enable/disable symbols via dynamic_conf
* [Feature] Add ability to lookup settings by key
* [Feature] Add common way to disable Lua modules
* [Feature] Add compression support to rspamd client
* [Feature] Add condition to do antiviral check
* [Feature] Add configuration for lua classifiers
* [Feature] Add configuration knobs for the errors circular buffer
* [Feature] Add decompression support in rspamd client
* [Feature] Add errors exporter to the controller
* [Feature] Add expected value for monitored DNS resources
* [Feature] Add exporter from error ringbuf to ucl
* [Feature] Add extended version for fann creation function
* [Feature] Add ffi friendly version of process_regexp function
* [Feature] Add frequency and time display to webui
* [Feature] Add fuzzy_delhash command to rspamc client
* [Feature] Add implementation of redis connections pool
* [Feature] Add latency and offline time monitoring
* [Feature] Add learning support for lua classifiers
* [Feature] Add max-size and timeout options to CGP helper
* [Feature] Add method to enable/disable symbols in config
* [Feature] Add methods to get metric's actions and symbols from Lua
* [Feature] Add mmap support to lua_text
* [Feature] Add monitored object for surbl plugin
* [Feature] Add more exceptions to surbl whitelist
* [Feature] Add more meta-tokens to bayes
* [Feature] Add neural net classifier to fann_scores module
* [Feature] Add neural net serialization/deserialization
* [Feature] Add new dynamic conf module
* [Feature] Add periodic events support for lua_config
* [Feature] Add plugin to check MX'es for the sender's domain
* [Feature] Add preliminary monitored module
* [Feature] Add preliminary support of dynamic conf updates in Redis
* [Feature] Add preliminary version of clamav plugin
* [Feature] Add redis cache to asn module
* [Feature] Add replies compression
* [Feature] Add spamhaus DROP dnsbl
* [Feature] Add support for dictionary in client compression
* [Feature] Add support for fuzzy learn and unlearn from lua
* [Feature] Add support for input encryption
* [Feature] Add support of min_learns to neural net classifier
* [Feature] Add termination callbacks for workers
* [Feature] Add user-agent for rspamc
* [Feature] Add utility to perform classifier tests
* [Feature] Add zstd compression library
* [Feature] Allow HTTPS requests in lua_http
* [Feature] Allow conditions for pre and postfilters
* [Feature] Allow custom functions for ratelimits
* [Feature] Allow for excluding messages from AV scanning based on size
* [Feature] Allow for getting worker stats from Lua
* [Feature] Allow getting task UID from Lua
* [Feature] Allow parsing of mailbox messages from the commandline
* [Feature] Allow plugins to publish their lua API via rspamd_plugins
* [Feature] Allow to compare other systems with Rspamd
* [Feature] Allow to execute Lua scripts by controller
* [Feature] Allow to have a function to set custom greylist message
* [Feature] Allow to iterate over multiple tags
* [Feature] Allow to pass extra data from plugins to log helper
* [Feature] Allow to plan new periodics at different time
* [Feature] Allow to reset hashes
* [Feature] Allow to run rspamadm lua just as a lua interpreter
* [Feature] Allow to store settings in redis
* [Feature] Allow to update dynamic conf in Redis
* [Feature] Allow to use dictionaries for compression
* [Feature] Allow to use md5, sha1, sha256, sha384 and sha512 hashes in Lua
* [Feature] Allow whitelisting by IP for greylisting plugin
* [Feature] Antivirus: Support whitelists & pattern-matching sig names
* [Feature] Backport pack/unpack routines from Lua 5.3
* [Feature] Check settings with equal priopities in alphabetical order
* [Feature] Compress neural net in redis
* [Feature] Consider more tags when doing WHITE_ON_WHITE rule
* [Feature] Descriptive options for DMARC failure symbols
* [Feature] Descriptive options for RBL symbols
* [Feature] Enable configuration for monitored objects
* [Feature] Execute on_load scripts with ev_base ready
* [Feature] Fann scores now uses metadata from a message
* [Feature] Implement FANN threaded learning
* [Feature] Implement classifying for lua classifiers
* [Feature] Implement finish scripts for worker processes
* [Feature] Implement monitoring for DNS resources
* [Feature] Implement real priorities for pre and post filters
* [Feature] Insert two symbols: FANN_HAM and FANN_SPAM instead of one
* [Feature] Module to push metadata/messages to redis pubsub
* [Feature] Monitor RBL records
* [Feature] Move fann_classifier to a separate plugin
* [Feature] Normalize all ANN inputs
* [Feature] Preliminary version of metric exporter module
* [Feature] Preserve decompression context between tasks
* [Feature] Ratelimit: Support dynamic bucket size/leak rate
* [Feature] Relax FORGED_RECIPIENTS: allow senders to BCC themselves
* [Feature] Remove symbols weights on composites processing
* [Feature] Return symbol scores when getting resulting symbols
* [Feature] Rework lua tcp module
* [Feature] Rule to detect some obvious X-PHP-Originating-Script forgeries
* [Feature] Rule to identify some X-PHP-Script forgeries
* [Feature] Rules for scoring Google Message-ID fixes
* [Feature] Send hashes values to reply
* [Feature] Set expire for dmarc reports
* [Feature] Stop using cymru zone as it is unstable
* [Feature] Stop using of GLists for headers, improve performance
* [Feature] Store `for` in task:get_received_headers
* [Feature] Store `for` part in received headers
* [Feature] Store enabled flag for webui session
* [Feature] Store error messages in ring buffer
* [Feature] Support compressed maps
* [Feature] Support excluding selected users from ratelimits
* [Feature] Support looking up NS records in lua_dns
* [Feature] Support modern style SURBL configuration
* [Feature] Support multiple hashes in delhash path
* [Feature] Support new messages in rspamc
* [Feature] Support requests without reads in lua_tcp
* [Feature] Support setting task message from Lua
* [Feature] Track visibility of HTML elements
* [Feature] Try to add CRLF when checking DKIM
* [Feature] Try to guess line endings when folding headers
* [Feature] Try to improve normalization function for bayes
* [Feature] Use FFI to optimize SA module
* [Feature] Use length based arguments for redis, allow lua_text as arg
* [Feature] Use more layers for fann and another normalization
* [Feature] User-defined ratelimits
* [Feature] Utility to convert fuzzy storage from sqlite to redis
* [Feature] Yield DMARC_DNSFAIL on lookup failure
* [Fix] Adopt fuzzy storage for flexible backends
* [Fix] Allow plain IP addresses in Rspamd maps
* [Fix] Another fix for brain-damaged hiredis
* [Fix] Another fix for rdns write errors
* [Fix] Another fix for rdns_make_request_full invocation
* [Fix] Another fix in DKIM canonicalization
* [Fix] Another memory leak plugged
* [Fix] Another try to deal with posix idiotizm
* [Fix] Another try to fix RDNS events processing logic
* [Fix] Avoid double frees in HEAD requests
* [Fix] Avoid extra symbols for RBLs
* [Fix] Banish table.maxn from Lua parts
* [Fix] Check for socket error before connection in lua_tcp
* [Fix] Correctly propagate redis timeouts to Lua
* [Fix] Do not add extra newline in MIME mode
* [Fix] Do not be cheated by system hiredis
* [Fix] Do not classify when a message has not enough tokens
* [Fix] Do not crash on redis errors
* [Fix] Do not distinguish NXDOMAIN and NOREC for monitored
* [Fix] Do not replan retransmits if merely one server is defined
* [Fix] Do not use headers to calculate messages digests
* [Fix] Don't force action in replies module for authenticated users/local networks
* [Fix] Explicitly ban default passwords in webui
* [Fix] Finally fix ambiguity between parsed and resolved spf elts
* [Fix] Fix 'decoded' value in task:get_header_full()
* [Fix] Fix DKIM calculations
* [Fix] Fix DKIM signing for messages with no newline at the end
* [Fix] Fix DNS request in monitored
* [Fix] Fix DNS write errors processing
* [Fix] Fix HTTP methods other than GET and POST
* [Fix] Fix PERMFAIL for v6/v4 ambiguities
* [Fix] Fix absurdic scores for HFILTER_URL_ONLY
* [Fix] Fix actions in rolling history
* [Fix] Fix actrie patterns
* [Fix] Fix applying of lua dynamic confg
* [Fix] Fix autolearning errors and redis cache
* [Fix] Fix bayes learn_condition
* [Fix] Fix build with the recent OpenSSL
* [Fix] Fix caching and compressed maps
* [Fix] Fix check plain text part
* [Fix] Fix crash on OpenBSD in `url_email_start`
* [Fix] Fix double free in SPF
* [Fix] Fix extraction of shingles from redis fuzzy storage
* [Fix] Fix false sharing for symbols in the cache
* [Fix] Fix float usage in util:get_time
* [Fix] Fix folding algorithm to deal with empty tokens
* [Fix] Fix format string
* [Fix] Fix format string usage in controller errors handling
* [Fix] Fix handling of '\0' in lua_tcp
* [Fix] Fix handling of HTTP HEAD methods
* [Fix] Fix hash creation
* [Fix] Fix hiredis stupidity
* [Fix] Fix implicit settings module settingsup
* [Fix] Fix interaction with lua GC to avoid craches
* [Fix] Fix ip_score module registration
* [Fix] Fix issue with empty messages and dkim
* [Fix] Fix issues with CGP helper
* [Fix] Fix issues with the recent SPF changes
* [Fix] Fix key name to load ANN correctly
* [Fix] Fix lua tcp module by saving `do_read` in callback data
* [Fix] Fix memory leak in client when using compression
* [Fix] Fix min_learns option
* [Fix] Fix on_finish scripts and async handlers
* [Fix] Fix options for SPF dnsfail symbol
* [Fix] Fix parsing includes and redirects in SPF
* [Fix] Fix parsing of lua comments with empty lines
* [Fix] Fix parsing of unquoted HTML attributes
* [Fix] Fix periodic events and redis
* [Fix] Fix processing of fuzzy learns from Lua
* [Fix] Fix processing of redirect in SPF includes
* [Fix] Fix processing of symbols when reject limit is reached
* [Fix] Fix refcounts when map is specified by IP
* [Fix] Fix rspamd{session} class in Lua API
* [Fix] Fix setting ratelimit key for 'ip' bucket
* [Fix] Fix some cases of TLD urls detector
* [Fix] Fix statconvert tool
* [Fix] Fix stats for backend-less classifiers
* [Fix] Fix training script for fann_redis
* [Fix] Fix variable in ann module
* [Fix] Fix various errors in lua dynamic conf plugin
* [Fix] Forget old ANN when max_usages is reached to avoid overtrain
* [Fix] Further canonicalization fixes
* [Fix] Further fixes for fann_redis prefixes
* [Fix] Handle failures for inactive pooled connections
* [Fix] Improve multimap info message
* [Fix] More fixes in ANN loading
* [Fix] More fixes to fann_redis
* [Fix] More issues in fann_redis
* [Fix] More spaces fix in DKIM signature
* [Fix] Multiple fixes to asn script, add IPv6 support
* [Fix] Multiple issues in fann_redis
* [Fix] No greylist rejected messages
* [Fix] One more attempt to fix lua_redis
* [Fix] One more check for readdir...
* [Fix] Params should be treated as a hash
* [Fix] Plug memory leak in regexp desctructor
* [Fix] Process headers only once
* [Fix] Properly handle nil values in ratelimit plugin
* [Fix] Really fix redis shingles check
* [Fix] Remove fann with incorrect layers count
* [Fix] Remove mentions of deleted include
* [Fix] Remove some incompatible functions
* [Fix] Settings: fix `authenticated` parameter (#886)
* [Fix] Skip MX check for authenticated users and local networks
* [Fix] Slightly fix ANN routines
* [Fix] Stop caching records with DNS failures
* [Fix] Treat all errors in redis_pool as fatal errors for a connection
* [Fix] Try avoid false-positives in HEADER_FORGED_MDN rule
* [Fix] Try to avoid race condition when using rrd
* [Fix] Try to reload redis scripts if they are missing
* [Fix] Unbreak once_received skipping for local networks
* [Fix] Unlock ANN on error
* [Fix] Use memmove for overlapping regions
* [Fix] Use real size instead of displayed for core limits
* [Fix] Use the correct macro to get the size of control
* [Fix] Various fixes for errors ringbuffer
* [Fix] Yield R_SPF_DNSFAIL if lookup of included record fails
* [Fix] mid: fix map initialization
* [Fix] mid: handle incorrect rgexps in the map
* [Rework] Add extract training data function to fann_redis
* [Rework] Add preliminary train tests
* [Rework] Add redis storage feature to fann_redis
* [Rework] Adopt fuzzy storage for abstract backend
* [Rework] Adopt plugins
* [Rework] First reiteration on fann scores
* [Rework] Implement loading/invalidating
* [Rework] Make lua_redis task agnostic
* [Rework] Make rspamd protocol messages useful
* [Rework] Massive removal of legacy code
* [Rework] More cleanup actions
* [Rework] Remove legacy code never used for classifiers
* [Rework] Remove outdated and unused lua_session module
* [Rework] Reorganize fuzzy backend structure
* [Rework] Reorganize the internal backend structure
* [Rework] Restore old fann_scores, move common parts
* [Rework] Rework and simplify rbl plugin
* [Rework] Rework parsing of DMARC records
1.3.4:
* [Feature] ASN module; support matching ASN/country in multimap
* [Feature] Add SPF method in spf return result
* [Feature] Add Yandex and Mail.ru forwarding rules
* [Feature] Add mempool maps in multimap
* [Feature] Add rule for identifying mail sent by eval()'d PHP code
* [Feature] Add support of stub DNSSEC resolver to rdns
* [Feature] Add task:get_digest method
* [Feature] Allow for more fine-grained scoring for ip_score
* [Feature] Allow to get digest of a mime part from lua
* [Feature] Allow to print message digest in logs
* [Feature] Fold DKIM-Signature header
* [Feature] Implement encrypted logs
* [Feature] Log URLs encrypted if we have log encryption pubkey
* [Feature] Pass authenticated bit to lua
* [Feature] Read redis backend statistics configuration from global section
* [Feature] Show the exact value matched for multima symbols
* [Feature] Store task checksum
* [Fix] Avoid setting limits when required elements are missing
* [Fix] DMARC: Fix alignment checking for subdomains
* [Fix] DMARC: deal with missing and spurious spaces
* [Fix] Defer insertion of results in ip_score to avoid skewing stats
* [Fix] Disable DMARC for local/authorized mail
* [Fix] Fix handling of proxied headers in controller
* [Fix] Fix hex printing of strings
* [Fix] Fix issue with spaces in maps
* [Fix] Fix parsing of forwarded IP
* [Fix] Fix reload in some plugins and workers
* [Fix] Fix reloading on SIGHUP
* [Fix] Fix some border cases for DKIM canonicalization
* [Fix] Fix url maps
* [Fix] Make dnssec configurable option disabled by default for now
* [Fix] rspamadm statconvert: force db to be a string
* [Fix] rspamadm statconvert: use db/password for learn cache
* [Rework] Rework flags in rspamd logger
1.3.3:
* [CritFix] Check hyperscan cache sanity before loading
* [CritFix] Fix setting of fuzzy keys (completely breaks fuzzy storage)
* [Feature] Add SARBL (sarbl.org) uribl
* [Feature] Add `--search-pattern` option to rspamd_stats
* [Feature] Add some sanity check for very long from/to log elements
* [Feature] Allow to create hashes from string in a single step
* [Feature] Fix order of pre and postfilters
* [Feature] Improve lua URLs API
* [Feature] Improve message about fuzzy rules
* [Feature] Pre-calculate blake2 digest for all parts
* [Feature] Print radix duplicate keys as IP addresses
* [Feature] Simple mechanism for disabling RBLs in local.d/rbl.conf
* [Feature] Use faster hash function for fuzzy storage
* [Feature] rspamd_stats: support log directory reading
* [Fix] Add sanity check for url filters
* [Fix] Do not show rmilter section as a fake metric in rspamc
* [Fix] Fix URL filters
* [Fix] Fix a stupid mistake in util.strequal_caseless
* [Fix] Fix blake2b hash of the string "rspamd"
* [Fix] Fix filename maps filter
* [Fix] Fix finding tld in util.get_tld
* [Fix] Fix multimap content filters
* [Fix] Fix returning boolean from Lua
* [Fix] Fix returning of REDIS_NIL
* [Fix] Try to deal with multiple workers terminated
* [Fix] Use forced DNS request when calling for lua_http
* [Rework] Rework multimap filters, add redis maps
1.3.2:
* [Feature] Add a special symbol for SPF DNS errors: R_SPF_DNSFAIL
* [Feature] Add correlations report in fuzzy stats
* [Feature] Add experimental CGP integration
* [Feature] Add method to get urls length in a text part
* [Feature] Add new methods to lua_html to access HTML tags
* [Feature] Allow all types of symbols to be added via __newindex method
* [Feature] Allow to create settings for authenticated users
* [Feature] Allow to get block content for HTML tags
* [Feature] Improve DNS failures when dealing with SPF
* [Feature] Properly implement R_WHITE_ON_WHITE rule
* [Feature] Remove old ugly rules
* [Feature] Rspamc can now add dkim signature in mime mode
* [Feature] Store content length for HTML tags
* [Feature] Support reacher set of HTML colors
* [Feature] Try to avoid FP for low contrast fonts detection
* [Fix] Add missing HTML colors
* [Fix] Add spaces to dkim signature to allow folding
* [Fix] Avoid returning NaN as score on scan
* [Fix] Decode entitles in href parts
* [Fix] Do not cache SPF records with DNS errors
* [Fix] Do not crash on cyclic depends
* [Fix] Do not insert HELO/HOSTNAME unknown when they are not passed
* [Fix] Do not set absent hostname to "unknown"
* [Fix] Do not stress redis with KEYS command (#791)
* [Fix] Fix DMARC_BAD_POLICY symbol
* [Fix] Fix HFILTER_URL module
* [Fix] Fix HFILTER_URL_ONELINE rule
* [Fix] Fix buffering in CGP integration
* [Fix] Fix colors propagation from parent nodes
* [Fix] Fix confusing OpenSSL API usage of i2d_RSAPublicKey
* [Fix] Fix dependencies id sanity check
* [Fix] Fix folding for semicolon separated tokens
* [Fix] Fix largest possible TLD behaviour
* [Fix] Fix last token folding
* [Fix] Fix length calculations in white on white rule
* [Fix] Fix multiple request headers structure
* [Fix] Fix multiple values headers freeing
* [Fix] Fix parsing of background color
* [Fix] Fix printing from field in log_urls
* [Fix] Fix processing of last element of DMARC policies
* [Fix] Further fixes for HTML colors
* [Fix] Further fixes for multiple values headers
* [Fix] Further fixes for white on white rule
* [Fix] Further fixes in HTML tags parsing
* [Fix] Ignore content type/subtype case
* [Fix] Increase score of R_WHITE_ON_WHITE
* [Fix] Parse CGP envelope data
* [Fix] Propagate colors in HTML
* [Fix] Restore multiple values headers in protocol
* [Fix] Restore multiple values in headers processing
* [Fix] Some more changes to tag's content length calculations
* [Fix] Some more fixes for low contrast fonts detector
* [Fix] SpamAssassin plugin: support check_freemail_header('EnvelopeFrom', [..])
* [Fix] Trigger HTML_SHORT_LINK_IMG on any external image
* [Fix] rspamd_stats: remove deprecated defined(@array)
1.3.1:
* [CritFix] Fix catena passwords validation
* [CritFix] Fix crash when the first received is faked
* [Feature] Add DMARC_BAD_POLICY symbol when DMARC policy was invalid
* [Feature] Allow for matching hostnames in multimap (#773)
* [Feature] Allow for setting action based on DMARC disposition
* [Feature] Allow limiting of the inbound message size
* [Feature] Allow maps with multiple symbols and scores
* [Feature] Allow regexps in the emails maps
* [Feature] Allow to register metric symbols from multimap
* [Feature] Allow to reset redis tokens instead of appendig values
* [Feature] Allow to store strings in radix maps
* [Feature] Check UTF validity when there are utf regexps in a map
* [Feature] Correctly work when there is no hard reject action
* [Feature] Implement dependencies for maps
* [Fix] Another effort to unbreak sqlite locking
* [Fix] Avoid crash when closing mmapped file
* [Fix] Do not break history on NaN in required score
* [Fix] Ensure that hyperscan cache written is written properly
* [Fix] Filter NaN from scores in history
* [Fix] Fix DNSBL maps
* [Fix] Fix another locking issue in sqlite
* [Fix] Fix another locking issue with mapped files
* [Fix] Fix deadlock in mmaped file stats
* [Fix] Fix dependencies in multimap plugin
* [Fix] Fix emails module configuration
* [Fix] Fix greylist plugin (#755)
* [Fix] Fix greylisting plugin variable usage
* [Fix] Fix installed permissions for rspamd_stats
* [Fix] Fix locking in mmapped statistics
* [Fix] Fix paths in tests
* [Fix] Fix prefilter mode for multimap
* [Fix] Forgot to commit leftover changes
* [Fix] Really fix local.d includes
* [Fix] Restore selective greylisting behaviour
* [Fix] Set max size on per connection basis
* [Fix] Use temporary storage for hyperscan cache
* [Rework] Remove systemd socket activation
1.3.0:
* [CritFix] Fix SA rawbody processing - exclude top part
* [CritFix] Fix decoding of UTF HTML entitles
* [CritFix] Fix encrypted fuzzy requests
* [CritFix] Fix leak of shared memory fds and files
* [CritFix] Fix levenshtein distance calculations
* [CritFix] Fix mime headers processing
* [CritFix] Fix parsing of URLs in texts
* [CritFix] Fix parsing of missing classes
* [CritFix] Fix redis structure by adding {NULL, NULL} member
* [CritFix] Fix some more URL detector issues
* [CritFix] Fix systemd sockets activation
* [CritFix] Fix unencrypted passwords processing in the controller
* [CritFix] Fix writing CDPs to the database
* [CritFix] Fix writing of encrypted HTTP requests
* [CritFix] Plug memory leak in dkim module
* [CritFix] Plug memory leak in headers getting code
* [CritFix] Pre-filters and post-filters were completely broken
* [CritFix] Properly support SA body regexps
* [CritFix] Really skip filters in case of pre-result set
* [CritFix] Restore the intended pre-filters behaviour
* [Rework] Adopt new maps code
* [Rework] Compile ragel sources when building rspamd
* [Rework] Finish rework for the rest of places that use HTTP
* [Rework] Fix DKIM headers canonicalization
* [Rework] Fix lua maps API
* [Rework] Import linenoise for line editing
* [Rework] Include config structure to all rcl handlers
* [Rework] Make chartable module useful
* [Rework] Move http internal structures to a private header
* [Rework] Partly fix controller
* [Rework] Remove dedicated images list
* [Rework] Rename http proxy to rspamd proxy
* [Rework] Rename mime parts structures
* [Rework] Rework HTTP code
* [Rework] Rework exceptions and newlines processing
* [Rework] Rework pre and postfilters system
* [Rework] Separate method to close backend connections
* [Rework] Start the complete maps rework
* [Rework] Use dynamically generated ragel C sources
* [Feature] Add 'blacklist' and 'strict' modes for whitelists
* [Feature] Add 'symbols_enabled' and 'groups_enabled' to settings
* [Feature] Add ESMTPSA received type
* [Feature] Add a simple script to evaluate rspamd rules in the logs
* [Feature] Add a simple tool to generate DKIM keys
* [Feature] Add a trivial heuristic for codepages
* [Feature] Add and use mumhash for non-crypto hashing
* [Feature] Add better method to check lua userdata types
* [Feature] Add better zip files search algorithm
* [Feature] Add catena PBKDF function
* [Feature] Add configuration knobs for in and out parser scripts
* [Feature] Add content filtering support to multimap
* [Feature] Add different timeouts for proxy connections
* [Feature] Add dot commands for lua REPL:
* [Feature] Add execution of lua global functions script
* [Feature] Add function for pretty printing of inet addresses
* [Feature] Add function to convert fstring_t to c string
* [Feature] Add function to create temporary shared memory mapping
* [Feature] Add function to generate random hex string
* [Feature] Add generic fucnction to parse IP maps
* [Feature] Add initial version of HTTP lua repl
* [Feature] Add learn condition to the default configuration
* [Feature] Add learn conditions for classifiers
* [Feature] Add limit for dkim signatures to be checked
* [Feature] Add locking routines for lua_util
* [Feature] Add lua API for getting info from archives
* [Feature] Add lua utility to decode URL encoding
* [Feature] Add method to copy message from http connection
* [Feature] Add mirrors feature
* [Feature] Add more algorithms for shingles generation
* [Feature] Add more domains to redirectors list
* [Feature] Add more encodingsto cryptobox hash API
* [Feature] Add more file utilities to lua_util
* [Feature] Add more functions to extract data from text parts
* [Feature] Add more methods to get headers from a task
* [Feature] Add more methods to init http message body
* [Feature] Add more options for redis config parsing function
* [Feature] Add new representation of email address
* [Feature] Add new symbols to filter bad extensions in messages
* [Feature] Add new utility methods to mimepart object
* [Feature] Add openphish support to rspamd phishing module
* [Feature] Add parsers for SMTP address in ragel
* [Feature] Add parsing of mirror hosts for fuzzy storage
* [Feature] Add parsing scripts for master connection as well
* [Feature] Add preliminary greylist plugin
* [Feature] Add preliminary phishtank support
* [Feature] Add preliminary rarv5 support
* [Feature] Add preliminary version of ssl toolbox
* [Feature] Add protection against open files limit and accepting sockets
* [Feature] Add rar v4 support
* [Feature] Add reading scripts for master connection
* [Feature] Add replies plugin
* [Feature] Add results parsing code
* [Feature] Add routines to compare and check pubkeys
* [Feature] Add simpler versions of refcounts
* [Feature] Add some time manipulation functions for lua APi
* [Feature] Add support for non-standard BATV signatures
* [Feature] Add support of address with at-domain list
* [Feature] Add support to search archives by magic
* [Feature] Add task:get_rawbody method
* [Feature] Add test to check shared memory support sanity
* [Feature] Add the initial version of LUA repl to rspamadm
* [Feature] Add throughput graph for RRD backend to WebUI
* [Feature] Add universal function to make a proper redis request
* [Feature] Add universal function to parse redis servers for plugins
* [Feature] Add util.unlink function
* [Feature] Add utility function to return random number from 0 to 1
* [Feature] Add utility method to convert ftok to C string
* [Feature] Add utility to map shared memory segments
* [Feature] Add versions to fuzzy storage
* [Feature] Add workaround for legacy clients in rspamd proxy
* [Feature] Add workaround for systems without sane shmem support
* [Feature] Add xoroshiro+ fast rng for non-crypto purposes
* [Feature] Adopt plugins for new maps API
* [Feature] Allow SPF to be checked for empty tasks
* [Feature] Allow binary patterns in lua_trie
* [Feature] Allow catena encrypted passwords in controller
* [Feature] Allow client ip match in the settings
* [Feature] Allow easy adding and overriding of fuzzy rules
* [Feature] Allow empty tasks to be processed
* [Feature] Allow hostnames in IP maps
* [Feature] Allow https maps
* [Feature] Allow multiple PBKDF types in `rspamadm pw`
* [Feature] Allow named fuzzy rules
* [Feature] Allow non zero terminated patterns in multipattern
* [Feature] Allow partial hash updates
* [Feature] Allow pipelining for redis.make_request
* [Feature] Allow sending empty requests using client
* [Feature] Allow setting fuzzy flag by symbol not by value
* [Feature] Allow setting scores and actions from lua
* [Feature] Allow shared memory simple http client
* [Feature] Allow static lua files in any parts of rspamd sources
* [Feature] Allow to change flag from fuzzy learn condition
* [Feature] Allow to check rspamd_text using maps
* [Feature] Allow to disable composite rules from settings
* [Feature] Allow to disable some modules from common redis setup
* [Feature] Allow to extract ucl_object from lua using common API
* [Feature] Allow to get settings and settings id hash from lua_task
* [Feature] Allow to have specific settings for mirrored traffic
* [Feature] Allow to open message from a shared memory segment
* [Feature] Allow to parse pubkeys from the rcl config
* [Feature] Allow to pass extradata from rspamd to rmilter
* [Feature] Allow to query storage about number of fuzzy hashes stored
* [Feature] Allow to read logs without symbols scores
* [Feature] Allow to read password from console for rspamc
* [Feature] Allow to set ciphers and CA paths in config
* [Feature] Allow to skip some initialization phases to speed up rspamadm
* [Feature] Allow underscore separated names in settings
* [Feature] Allow versioning for sqlite databases
* [Feature] Always allow to terminate rspamd
* [Feature] Better deal with backend errors
* [Feature] Better lua_redis logging
* [Feature] Configure CA path and ciphers
* [Feature] Create a dedicated parser to strip newlines
* [Feature] Deduplicate the same urls in multimap module
* [Feature] Distinguish luajit from lua
* [Feature] Do not print garbadge in --compact output
* [Feature] Dynamically detect if a CPU is incompatible with hyperscan
* [Feature] Enable forced resolving for some lua plugins
* [Feature] Enable rrd by default
* [Feature] Enable workaround for exim
* [Feature] Fix task functions to work without rspamd_config
* [Feature] Further improvements to chartable module
* [Feature] Further micro-optimizations for hashing and shingles
* [Feature] Further relax parser
* [Feature] Humanize numbers in stats widgets
* [Feature] Implement HTTPS client
* [Feature] Implement SSL support in http client
* [Feature] Implement body rules for the trie plugin
* [Feature] Implement braced regexp quantifiers
* [Feature] Implement compare scripts for mirrors results
* [Feature] Implement compare scripts setup
* [Feature] Implement composites policies
* [Feature] Implement conditional learning for classifiers
* [Feature] Implement constructing of map from UCL
* [Feature] Implement dkim signing in dkim check plugin
* [Feature] Implement fuzzy storage updates
* [Feature] Implement fuzzy updates push protocol
* [Feature] Implement https maps
* [Feature] Implement inter-process maps caching
* [Feature] Implement mapping of remote flags to local flags
* [Feature] Implement mirroring in rspamd proxy
* [Feature] Implement multi-flags fuzzy replies
* [Feature] Implement multiple-sources fuzzy storage
* [Feature] Implement order of pre/post filters
* [Feature] Implement partial deleting for multi-flags
* [Feature] Implement pipelining for redis async interface
* [Feature] Implement ragel parser for received headers
* [Feature] Implement reading of messages to shared memory
* [Feature] Implement refcount for messages
* [Feature] Implement retransmits for master connection
* [Feature] Implement zero-copy mode for HTTP reading
* [Feature] Improve SPF domain detection logic
* [Feature] Improve config:register_symbol function
* [Feature] Improve error report for type mismatch in lua
* [Feature] Improve fstrings API
* [Feature] Improve getting SMTP data from lua_task
* [Feature] Improve levenshtein distance function
* [Feature] Improve logging in proxy and add refcounts
* [Feature] Improve logging lua types
* [Feature] Improve master/slave logging
* [Feature] Improve phishing plugin
* [Feature] Improve phishtank and openphish support
* [Feature] Improve ragel build target
* [Feature] Improve statistics script
* [Feature] Initialize ssl library to use SSL connections
* [Feature] Interpolate points sent to webui
* [Feature] Limit logging of elements that could have too many items
* [Feature] Lock ANN file when loading
* [Feature] New abstract hashing API in cryptobox
* [Feature] Normalize quoted addresses
* [Feature] Now cryptobox lua API accepts rspamd text as input
* [Feature] Optimize alignment to speed up hashing
* [Feature] Parse received date and ESMTPA proto
* [Feature] Parse received timestamp
* [Feature] Pass settings id to log helper
* [Feature] Pass settings id to lua script from log helper
* [Feature] Perform files expansion on proxying
* [Feature] Preliminary implementation of fuzzy master/slave updates
* [Feature] Print userdata using tostring if possible
* [Feature] Propagate HTTP result string when using proxy
* [Feature] Properly implement unweighted round-robin algorithm
* [Feature] Reduce number of timers queries
* [Feature] Rework and improve fuzzy storage
* [Feature] Rework dns resolving API for lua, add 'forced' option
* [Feature] Rework fann module to understand settings
* [Feature] Rework listening system to allow multiple socket types per worker
* [Feature] Rework ratelimit module to set expiration
* [Feature] Save bayes probability in memory pool var
* [Feature] Save settings id hash for convenience
* [Feature] Search for SSL_set_tlsext_host_name support
* [Feature] Send DKIM signature to protocol reply
* [Feature] Show DKIM signature in rspamc client
* [Feature] Show symbols description in scan output
* [Feature] Sign message merely after DKIM check
* [Feature] Simplify machines by assuming that headers are unfolded
* [Feature] Sort symbols when displaying them in log
* [Feature] Split main connection from mirrored connections
* [Feature] Start moving to the new email address structure
* [Feature] Store HTTP headers in a hash table
* [Feature] Store more information about compressed files
* [Feature] Store raw headers value to use them in DKIM
* [Feature] Store text parts content with newlines stripped
* [Feature] Support DKIM signing
* [Feature] Support EXIF jpeg images
* [Feature] Support archive files list extraction
* [Feature] Support archives when matching patterns in multimap
* [Feature] Support premium/academic feed for openphish
* [Feature] Support rspamd_updates via https
* [Feature] Supprort FQDNs in phishing module maps
* [Feature] Try to read on fuzzy timeout to avoid fake timeouts
* [Feature] Try to select the optimal possible function for input
* [Feature] Unescape and unquote smtp addresses
* [Feature] Update fuzzy timestamp when adding value
* [Feature] Update mumhash
* [Feature] Use -flto if possible when optimizations are enabled
* [Feature] Use extended map types in lua map, unify code
* [Feature] Use file lock in logger to avoid deadlocks
* [Feature] Use generic global string split function
* [Feature] Use metrohash as well
* [Feature] Use mumhash by default for incremental hashing
* [Feature] Use mumhash for words hashing
* [Feature] Use new ip parsing API
* [Feature] Use new maps API for local addrs
* [Feature] Use new ragel parser in message parsing code
* [Feature] Use new received parser instead of old one
* [Feature] Use new redis API in DMARC plugin
* [Feature] Use new redis API in greylist plugin
* [Feature] Use new redis API in ip_score plugin
* [Feature] Use new redis API in ratelimit plugin
* [Feature] Use new redis API in replies plugin
* [Feature] Use new version of register_symbol in rspamd plugins
* [Feature] Use offset when passing shm to deal with encrypted requests
* [Feature] Use one pass to remove newlines and store their positions
* [Feature] Use rspamd specific type checks for userdata
* [Feature] Use shared memory storage for http maps
* [Feature] Use universal redis definitions in rspamd plugins
* [Feature] Various improvements in greylist module
* [Feature] Wait for sqlite if locked when switching to WAL mode
* [Fix] Add filenames sanity filtering for mime types
* [Fix] Add guards for empty parts
* [Fix] Add missing types
* [Fix] Add more priority for config file symbols registered from UCL
* [Fix] Add sanity checks when compiling regexp maps
* [Fix] Add spaces instead of newlines to the normalized content
* [Fix] Add workaround for ancient openssl
* [Fix] Add workaround for gmime CTE stupidity
* [Fix] Add workaround for hex digits
* [Fix] Adjust MISSING_MIMEOLE score
* [Fix] Adjust body/body_buf when stealing encrypted message
* [Fix] Adopt lua task API for the new email addresses structure
* [Fix] Allow for disabling DMARC reporting when Redis is configured
* [Fix] Always register openphish and phishtank virtual symbols
* [Fix] Always use shmem on linux
* [Fix] Another change of newlines policy
* [Fix] Another d3evolution update
* [Fix] Another fix for exim workaround
* [Fix] Another fix for legacy clients
* [Fix] Another fix for maps scheduling
* [Fix] Another fix for marking upstreams inactive
* [Fix] Another fix for postfilters
* [Fix] Another fix for redis timeouts
* [Fix] Avoid `table.getn` method as it has been removed in lua 5.3
* [Fix] Avoid double hashing for images
* [Fix] Avoid linking with actrie if hyperscan is enabled
* [Fix] Check copy result when sending message to mirrors
* [Fix] Cleanup message when assiging body
* [Fix] Cleanup stack from global vars
* [Fix] Correctly parse query type
* [Fix] Disable all symbols if enable_groups is found in settings
* [Fix] Disable fts as it is completely broken in bloody linux
* [Fix] Disable multiple autolearn checks
* [Fix] Disallow updates by default
* [Fix] Do not abort when cannot load a map
* [Fix] Do not check recursion for non-DNS SPF record types
* [Fix] Do not delete uninitialized events
* [Fix] Do not die if shmem_mkstemp fails
* [Fix] Do not die when no metrics defined
* [Fix] Do not even try pcre in case of regexp maps
* [Fix] Do not greylist messages if redis has failed somehow
* [Fix] Do not greylist on rejection
* [Fix] Do not leave temporary maps cached
* [Fix] Do not output meaningless errors
* [Fix] Do not send NaN in json
* [Fix] Don't mix hyperscan and pcre processing within a same task
* [Fix] Finally rework and simplify redis backend for statistics
* [Fix] Fix Exim shutdown patch
* [Fix] Fix JIT compilation for PCRE2 expressions
* [Fix] Fix JIT usage for PCRE2
* [Fix] Fix REPL output
* [Fix] Fix SMTP address parsing machine
* [Fix] Fix UTF8 mode in PCRE2
* [Fix] Fix a stupid misprint in word 'phishing'
* [Fix] Fix accepting the first update when local idx is -1
* [Fix] Fix adding maps from ucl
* [Fix] Fix adding upstream to an active queue
* [Fix] Fix and rescore R_PARTS_DIFFER logic
* [Fix] Fix body rules in SA plugin
* [Fix] Fix body start position
* [Fix] Fix border case in urls detector
* [Fix] Fix border cases for incremental hashing
* [Fix] Fix caseless uthash application
* [Fix] Fix chartable issue with starting digits
* [Fix] Fix client_ip in users settings
* [Fix] Fix compilation issue
* [Fix] Fix conditional learning
* [Fix] Fix crash on empty maps
* [Fix] Fix creating of URLs from LUA
* [Fix] Fix creating of temporary shmem segment
* [Fix] Fix creation of mmapped statfiles
* [Fix] Fix descriptors leak on shmem detaching
* [Fix] Fix detaching of shared memory segments
* [Fix] Fix detection of URLs in text parts
* [Fix] Fix directories processing for rspamc
* [Fix] Fix displaying of rewrite subject in WebUI
* [Fix] Fix dkim private keys operations
* [Fix] Fix dkim signing
* [Fix] Fix dynamic scoring of subject symbols
* [Fix] Fix email address build
* [Fix] Fix encrypted proxy requests
* [Fix] Fix errors counting in upstreams
* [Fix] Fix errors handling in the proxy
* [Fix] Fix event bases for IO events
* [Fix] Fix events handling when scheduling map wacth
* [Fix] Fix fann rewrite
* [Fix] Fix files fallback for shmem transfer
* [Fix] Fix fuzzy adding in webui
* [Fix] Fix fuzzy storage encrypted mirroring
* [Fix] Fix fuzzy storage sync replies
* [Fix] Fix handling of the same words
* [Fix] Fix inserting values to the sources list
* [Fix] Fix ipv6 mask application
* [Fix] Fix issue with missing recipients
* [Fix] Fix issues with multiple returns from lua
* [Fix] Fix learning for non-existent backend
* [Fix] Fix legacy clients support in proxy
* [Fix] Fix length calculations for shared memory segments
* [Fix] Fix listening on UDP sockets
* [Fix] Fix loading of file maps
* [Fix] Fix long regexp flags (e.g. {sa_body})
* [Fix] Fix lua compare function init
* [Fix] Fix maps descriptions
* [Fix] Fix maps locking
* [Fix] Fix max_train setup in ANN module
* [Fix] Fix memory corruption
* [Fix] Fix memory leak in unsigned maps reading
* [Fix] Fix misprints for lto usage
* [Fix] Fix more issues with scripts processing
* [Fix] Fix next-to-last extension length check
* [Fix] Fix openssl initialization
* [Fix] Fix order of arguments
* [Fix] Fix order of initialization
* [Fix] Fix parser
* [Fix] Fix parsing of binary tries
* [Fix] Fix parsing of braced IPv6 addresses
* [Fix] Fix parsing of nested braces in SMTP comments
* [Fix] Fix parsing of rarv5 archives
* [Fix] Fix periodic scheduling when a map is not modified
* [Fix] Fix possible FP in TRACKER_ID rule
* [Fix] Fix post-filters processing
* [Fix] Fix potential NULL dereference
* [Fix] Fix processing of <br> and <hr> tags
* [Fix] Fix processing of addresses in protocol
* [Fix] Fix processing of messages without received headers
* [Fix] Fix proxying issue for unconnected sessions
* [Fix] Fix proxying of the encrypted messages
* [Fix] Fix race condition with shared memory by refcounts
* [Fix] Fix ratelimit initialization
* [Fix] Fix redis set request in replies plugin
* [Fix] Fix redis timeout events handling
* [Fix] Fix redis timeouts processing logic
* [Fix] Fix refcounts in lua_redis
* [Fix] Fix results checking if no master connection is active
* [Fix] Fix return value for couple of lua functions
* [Fix] Fix round-robin selection when upstreams have no weight
* [Fix] Fix rows calculation in graph
* [Fix] Fix rspamd_redis_make_request syntax in replies plugin
* [Fix] Fix scheduling of locked map events
* [Fix] Fix scores detection
* [Fix] Fix searching for newline positions
* [Fix] Fix secure_ip setting in controller
* [Fix] Fix sending data to graph command
* [Fix] Fix setting of score for parts differ
* [Fix] Fix setting of the lua top
* [Fix] Fix setting path for lua
* [Fix] Fix setting path for phishtank
* [Fix] Fix settings application
* [Fix] Fix shm_open call as described in POSIX
* [Fix] Fix size of length in fuzzy mirror wire protocol
* [Fix] Fix smtp grammar issues
* [Fix] Fix some issues with redis API
* [Fix] Fix some issues with retries in the proxy
* [Fix] Fix stack growing
* [Fix] Fix start of body detection in DKIM
* [Fix] Fix state on timeout
* [Fix] Fix stats script
* [Fix] Fix substring search when there are '\0' in strings
* [Fix] Fix symbol name for spf soft fail
* [Fix] Fix symbol type's check
* [Fix] Fix symbols registration and execution
* [Fix] Fix the case of multiple values keys
* [Fix] Fix the default symbol names according to metric
* [Fix] Fix timeout setup on learning
* [Fix] Fix timeouts in redis cache processing
* [Fix] Fix timeouts processing in lua_redis
* [Fix] Fix upstreams interaction for rspamd proxy
* [Fix] Fix usage of rdns reply structure
* [Fix] Fix varargs loop
* [Fix] Fix whitelists and blacklists in SA rules
* [Fix] Fix write servers setup for redis
* [Fix] Fix writing of HTTP messages
* [Fix] Force rspamd to upgrade fuzzy db on opening
* [Fix] Free the correct pointer
* [Fix] Further fixes for lto and static linking
* [Fix] Further fixes for surbl extensions map
* [Fix] Further fixes in maps code
* [Fix] Further improvements to error messages in fuzzy check
* [Fix] Further tweaks to redis garbadge collection
* [Fix] Groups are now case insensitive
* [Fix] Handle log pipe read errors
* [Fix] Handle nested dependencies in SpamAssassin plugin
* [Fix] Implement new automata to skip empty lines for dkim signing
* [Fix] Improve error messages on fuzzy add
* [Fix] Improve lua redis handling
* [Fix] Improve phishing module logging
* [Fix] Improve printing of fuzzy errors
* [Fix] Improve rrd diagnostic errors
* [Fix] Improve strcase hash used in uthash
* [Fix] Include fuzzy key to distinguish storages with different keys
* [Fix] Include slave cluster name into http request
* [Fix] Include some more information about archives
* [Fix] Indicate upstream error on timeout
* [Fix] Initialize hash tables array to avoid crashes
* [Fix] Initialize parser scripts properly
* [Fix] Initialize vars to avoid warnings
* [Fix] Inverse logic for saving ANN
* [Fix] Link lpeg to rspamd lua library
* [Fix] Make extension checks case-insensitive
* [Fix] Mark expired hashes as not found and not as zero flag
* [Fix] Match archive name as well
* [Fix] More and more fixes to redis states
* [Fix] More fixes about shared memory in proxy
* [Fix] More fixes for redis refcounts
* [Fix] More fixes to end of headers detection
* [Fix] More fixes to events logic
* [Fix] More fixes to multi-flag fuzzy storage
* [Fix] More fixes to parts distance calculations
* [Fix] More guards for redis free
* [Fix] One more fix in redis destructor
* [Fix] One more try to fix redis
* [Fix] PIE is required for static build
* [Fix] Partial fix for mmap'd statistics tests
* [Fix] Plug memory leak in proxy
* [Fix] Properly detect end of headers position
* [Fix] Properly init and free session structures
* [Fix] Reduce PRECEDENCE_BULK rule weight
* [Fix] Reduce the default thresholds for learning
* [Fix] Remove Type=forking from systemd unit file (#709)
* [Fix] Remove bad FANN file to save computational resources
* [Fix] Remove event before closing of fd to avoid race conditions
* [Fix] Remove parsing of 'from' variable in redis backend
* [Fix] Remove some bad domains from whitelists
* [Fix] Repair optional dependencies
* [Fix] Reset master connection when retransmitting scan request
* [Fix] Restore ONCE_RECEIVED symbol
* [Fix] Restore compatibility with old lua API behaviour
* [Fix] Restore redis runtime state
* [Fix] Reverse options when received
* [Fix] Send updates to mirrors only if we have some changes
* [Fix] Set host attribute properly when making HTTP request from lua
* [Fix] Set terminated state before calling of async free
* [Fix] Simplify MISSING_MIMEOLE rule
* [Fix] Simplify state machine by ignoring multiple spaces
* [Fix] Skip setting RPATH for static builds
* [Fix] Slightly reduce weights of rules with high FP rate
* [Fix] Some fixes to libmagic initialization
* [Fix] Some more fixes to ratelimit plugin
* [Fix] Strip '\r\n' properly
* [Fix] Switch hashes to mumhash
* [Fix] Treat NaN values properly in graph command
* [Fix] Try to avoid FP when checking for phished URLs
* [Fix] Try to avoid recursive events deletions
* [Fix] Try to fix false positive URL detections in text parts
* [Fix] Try to fix issue in redis stats backend when task is closed
* [Fix] Try to fix proxying of stupid spamc protocol to HTTP mirrors
* [Fix] Try to fix redis crashes
* [Fix] Try to fix upstreams with one element
* [Fix] Try to handle multiline history in a more sane way
* [Fix] Unbreak build on gcc < 4.9
* [Fix] Update RPM spec/sources (#700)
* [Fix] Update d3evolution version
* [Fix] Update mumhash implementation
* [Fix] Use custom error function for pre and post filters
* [Fix] Use new postfilters and prefilters API in the plugins
* [Fix] Use non-blocking mode for systemd sockets
* [Fix] Use shared memory merely for local backends in the proxy
* [Fix] Use watchers for spf plugin
* [Fix] Varioud fixes to the maps code
1.2.8:
* Another fix for exim workaround (#637)
* Fix unencrypted passwords processing in the controller
* Fix setting path for lua (#652)
* Fix usage of rdns reply structure (#654)
* Use file lock in logger to avoid deadlocks
* Add `application/octet-stream` mime type for `pdf` extension (by @moisseev)
* Implement new automata to skip empty lines for dkim signing (#651)
* Fix parsing of missing classes
* Clarify some rspamc arguments (by @fatalbanana)
* Correct suppress spelling
1.2.7:
* Slightly reduce weights of rules with high FP rate
* Add workround for rspamd-1.3
* Fix possible FP in TRACKER_ID rule
* Simplify MISSING_MIMEOLE rule
* Add workaround for gmime CTE stupidity
* Fix mime headers processing
* Fix false positive URL detections in text parts
* Fix Exim shutdown patch
* Enable workaround for exim mailbox format
* Backport shingles static test
* Fix levenshtein distance calculations
* Fix max_train setup in ANN module
* Fix redis structure by adding {NULL, NULL} member
* Fix build with unmodified LibreSSL opensslv.h
* Repair optional dependencies
* Really skip filters in case of pre-result set
* Restore the intended pre-filters behaviour
* Fix ipv6 mask application
1.2.6:
* Fix parsing of URLs in texts
* Fix creating of URLs from LUA
* Fix some more URL detector issues
* Fix unit tests
* Fix JIT compilation for PCRE2 expressions
* Fix JIT usage for PCRE2
* Fix UTF8 mode in PCRE2
* Add workaround for pre-historic compilers (#605)
* Fix and rescore R_PARTS_DIFFER logic
* Properly set lua paths for tests
* Fix SA rawbody processing - exclude top part
* Store text parts content with newlines stripped
* Properly support SA body regexps
* Fix body rules in SA plugin
* Fix setting of score for parts differ
* More fixes to parts distance calculations
- Use hashed words instead of full words for speed
- Improve levenstein distance calculations and penalise replaces
- Always return number from 0 to 1
- Use g_malloc instead of alloca
* Fix percents output in R_PARTS_DIFFER
* Plug memory leak in dkim module
* Plug minor memory leak in regexps creation
1.2.5:
* Plug an important memory leak in headers getting code
* Remove some bad domains from whitelists
1.2.4:
* Implement new multipattern matcher that uses hyperscan if possible
* Use mutlipattern for lua_trie code
* Add utility methods for multipattern
* Use multipattern in url matcher
* Add escape functions for hyperscan
* Allow to optimize lua -> C transition by flattening table args
* Optimize hot paths in SA plugin
* Optimize rspamd_re_cache_type_from_string
* Allow empty tries
* Fix extraction of URLs from Subject
* Allow to have different flags for different patterns in multipattern
* Add common directory for hyperscan cache to config
* Implement caching for hyperscan multipattern
* Attach domain part to `R_SUSPICIOUS_URL` (by @moisseev)
* Allow multipattern scans to be nested for the case of hyperscan
* Simplify SURBL redirector search code and avoid ac_trie
* Add two way substring search algorithm
* Avoid acism usage to find gtube pattern
* Fix processing of empty headers
* Allow to disable pthread mutexes on broken platforms
* Make web interface not send password in query strings (#585) by @fatalbanana
* Add maximum delay to ratelimit module
* Backport fix for empty files inclusion from libucl
* Fix settings id setup
* Add min_learns option to classifiers
* Use more clever to utf8 conversion strategy
* Fix disabling of virtual symbols in the settings
* Rework settings to work properly in metric-less configuration
* Set the default limit for classifier
* Fix ttl based expiration from LRU cache
* Rework DKIM module to use OpenSSL for digests
* Fix mailto urls parsing with hyperscan
* Do not set obscured flag for urls starting with spaces
* Fix crash on redis learn
* Fix ratelimit ctime setting
1.2.3:
* New DCC module (by @smfreegard)
* Rework whitelist module:
- Now we check different elements for different checks
- MIME from for DMARC
- DKIM signature domain for DKIM
- SMTP from or HELO for SPF
* Fix regexps results combination (*critical*)
* Fix issue with expressions processing (*critical*)
* Optimize strlcpy for aligned input
* Add support of half-closed connection in lua_tcp
* Allow to print compact json in client
* Save required score in history (#581)
* Allow to attach file descriptors to control commands
* Allow to send descriptors from workers to main
* Allow to attach fd when broadcasting to workers
* Implement log pipe feature for rspamd logs analysis
* Add `log_helper` worker
* Add `URIBL_SBL_CSS` (by @smfreegard)
* Add worker scripts functionality
* Add on load hooks for rspamd_config
* Add lua scripts for log_helper worker
* Add generic maillist detector (#584)
* Implement FANN autolearn using log_helper worker
* Rework metrics configuration to allow includes
* Change default value of forced removal in composite rules
* Allow to use assembly version of blake2b on x86 cpu
* Use less precise (but faster) clock if possible
* Insert redirected URL to the urls list
* Allow to get and set callback data for rspamd symbols
* Add binary heap implementation
* Use binary heap for expire algorithms in the hash
* Use `least frequent used` expiration strategy
* Allow to get mime headers from a task
* Add support for mime headers in `regexp` module
* Update Exim patches (by @fatalbanana)
* Allow building rspamd with jemalloc
* Save multipart boundaries
* SA plugin changes:
- Properly handle MIME headers
- Fix eval:check_for_missing_to_header rule
- Implement SA compatible body regexps
- Use sabody rules in SA plugin
* LUA API changes:
- Add util.get_ticks function
- Add util.stat function
- Add task:get_symbols_numeric method
- Add method to get number of symbols in the cache
- Add lua methods to get redirected urls
- Allow to get callbacks for lua symbols
- Add config:set_symbol_callback function
1.2.2:
* Use HTTP Content-Type on non mime input if possible
* Save log level when compressing log messages
* Further rework of composite rules (add '^' prefix)
* Add tracking for rspamd expressions
* Store actions limits in metric result
* Fix parsing of include/redirect with many records in SPF
* Add method to disable symbols execution in the cache
* Allow to disable checks from settings
* Allow to select settings by id in HTTP query
* Find URLs with '\r' and '\n' inside href attribute
* Implement vectored mode for hyperscan (experimental)
* Improve client connection errors diagnostics
* Allow to edit new files with signtool
* Improve hashes performance on 32 bit platforms
* Fix sorting of limits
* Remove slow and unused rules `INVALID_EXIM_RECEIVED*`
* Add expression:process_traced lua method
* Allow tables in task:insert_result
* Save trace for SA metas
* Do not parse broken TLD parts in URLs
* Investigate many border cases in URLs parser
1.2.1:
* Add list support to `mime types` module configuration (by @moisseev)
* Allow symbols params to be printed in logs
* Fix `MIME_BAD_ATTACHMENT` false positives for MDN/DSN
* Fix crashes on arm32
* Do not classify message if some class is missing
* Fix cryptobox cleanup
* Remove multipart/report from bad mime types (#569)
* Improve logging for fuzzy hashes
* Show map URLs in webui
* Sort symbols in webui
1.2.0:
* New dynamic updates plugin
* Regular expressions map support
* Faster radix trie algorithm
* Faster siphash for AVX2 supporing CPUs (used in fuzzy hashes)
* PCRE2 support
* Allow quoted and slashed keys in map
* Add proper support of DNS resolvers balancing (#552)
* Rework includes and configuration system for better local changes support
* New keypairs framework for signing and encryption
* Added support for dynamic modules and workers
* Allow to dump configuration with help comments
* Rework once_received module
- Fix priority for `good_hosts`
- If a good host has been found do not add once_received symbols
- Fix priorities for strict once_received
- Add ability to whitelist IP addresses
* Implement support of signed maps for HTTP and file maps
* Add command to sync fuzzy storage (#533)
* Rework system of symbols and actions registration
It is possible now to use priorities when adding symbols to metrics and
override scores for symbols with lower priority with the scores with
high priority.
* Add auth support and db selection for redis stats
* Improve composite rules application
* Add ignore_received option
* Fix critical issue with inconsistent resorting
* Fix `all` in spf redirects
* Add punycoded versions for IDN domains (#554)
* Improve sorting order for symbols cache
* Add lockless logging for processes management
* Allow to specify flags for metric symbols
* Load images height and width from style attribute (#538)
* Override DNS requests limits for SPF and DKIM
* Fix resetting symbols to their default values in WebUI
* Improve configuration agility for redis stats
* Allow to set db and password for redis in stat_convert
* Import the latest libucl
* LUA API changes:
- Add rspamd_version function to LUA API
- Add lua_cryptobox module
- Add lua_map module
- Add task:set_metric_action lua API method
- Fix race condition in lua_tcp module
- Fix a lot of issues in lua_redis module
- Rework and abstract lua maps API
- Add util.strlen_utf8 lua function
- Add lua functions for caseless comparison
- Allow optional symbols registration
- Add config:add_map table form method, add regexp maps
- Add task:has_urls method
- Add task:has_flag method
- Add html tags methods to lua_html
- Add task:get_dns_req
* Plugins changes:
- Add support for WLBLEval SA plugin
- Use caseless comparison in SA and DMARC plugins
- Allow SA plugin to set scores for rspamd symbols
- Add regexp maps support to multimap
- Allow filenames match in multimap
- Add more filters for the existing map types
- Fix html images rules to reduce FP rates
* New rules:
- LONG_SUBJ - too long subject
- MIME_BAD_ATTACHMENT - bad attachment type
- RDNS_NONE - no reverse DNS record for sender's IP
- Fix MISSING_MIMEOLE rule for modern OE
* Many other bugfixes, memory leaks plugs thanks to:
- Coverity scan
- New gcc-6 warnings
- valgrind manual iterations
* Documentation improvements:
- FAQ list: https://rspamd.com/doc/faq.html
- Reworked quick start guide
- Added documentation for all active modules
* Other changes:
- Dropped Ubuntu Vivid support
- Added Ubuntu Xenial support
- Rework build system for rspamd and rmilter
1.1.4:
* Print traceback on lua errors in lua config
* Fix leaks in lua error paths
* Improve 'R_EMPTY_IMAGE' rule
* Fix metas memoization in SA plugin
* Properly set `flag` in fuzzy replies
* Fix arguments order
* Fix issue with out-of-boundary reading
* Fix issues found by coverity
* Same result checking error found by coverity
* Fix varargs processing (found by coverity)
* Fix error in printing hex
* Reduce weights for some hfilter patterns
* Add aliases for task:get_from_ip:
- task:get_addr
- task:get_from_addr
- task:get_ip
* Rework once_received module
- Fix priority for `good_hosts`
- If a good host has been found do not add once_received symbols
- Fix priorities for strict once_received
- Add ability to whitelist IP addresses
* Fix `MISSING_MIMEOLE` rule for modern OE
* Treat meta tags as embedded tags (#501)
1.1.3:
* Fix DSN rules when SMTP from is unavailable
* Fix statconvert routine to avoid lua module usage
* Set a sane quark for configtest to avoid NULL to be printed in logs
* Support c11 if available
* Fix parsing of ip:port strings
* Add more diagnostic for lua subr errors
* Fix task:set_from_ip lua method
* Add basic routines for digital signatures
* Add tool for digital signatures
* Add plain open file API method for atomic open
* Fix parsing nested braces inside logger vars
* Pre filters now actually skip processing
* Add pre-filter mode for multimap
* Switch to apache 2 license
1.1.2:
* Fix stat_cache closing
* Add checkpoints to sqlite3 learn cache
* Do not recompile lua generated headers all the time
* Increase number of messages learned
* Fix issues with dual stack and hfilter
* Disable MID checks for hfilter by default
* Fix cache definitions in multiple classifier and no type
* Don't crash if learn cache failed to initialize
* Fix googlegroups support in maillist plugin
* Rework flags LUA API:
- Allow to check for a specific flag
- Add `learn_spam`, `learn_ham` and `broken_headers` flags
- Unify internal functions
* Add `BROKEN_HEADERS` rule
* Add support for forged confirmation headers (by @AdUser)
* Allow `any`, `mime` and `smtp` for get_from/get_recipients
* Add mime types checking plugin
* Add rule to detect spammers attempts to cheat mime parsing
* Rework parsing of IP addresses in configuration (better IPv6 support)
* Add `util.parse_mail_address` function to LUA API
* Add lua sqlite3 module
* Implement synchronous redis call
* Ratelimit: avoid possible indexing of nil value (Fixes #498) (by @fatalbanana)
* Add stat_convert command to convert stats tokens from sqlite3 to redis
* Implement redis advanced lua api with pipelining
* Fix memory leak on redis stat (#500)
* Fix user/language learn count in sqlite statistics (#496) (by @fatalbanana)
* Fix build with custom pcre
* Fix fuzzy relearning (#498)
* Improve planning of asynchronous tasks
* Show slow rules in log
* Add warning for slow regexps
* Add base32 decode/encode routines to lua util
* Allow converting of learn cache from sqlite to redis
* Add methods to check if a messages has from/rcpts
* Improve and fix multimap plugin:
- Restore 'header' maps
- Add filters for headers
- Add 'email:addr', 'email:user', 'email:domain' and 'email:name' filters
- Add generic regexp filters
* Disable reload command in rc scripts
* Improve runtime CPU dispatcher for libcryptobox
* Add preliminary support of digital signatures via ed25519
* Add detection for RDRAND support
* Print configuration of crypto on start
* A in SPF presumes AAAA lookup as well
1.1.1:
* Fix duplicated XBL symbol
* Reduce log severity for ratelimit missing servers
* Fix XBL composite to avoid duplicate symbols
* Reduce weight of URL_ONLY rule due to FP rate
* Disable fuzzy hashes from the metadata for now
* Fix processing of empty messages (#486)
* Always treat DNS timeouts as temporary fail for SPF
* Fix issue with SPF double IP stack (#483)
* Use X-Forwarded-For when checking secure_ip (#488)
* Fix hash calculation for sqlite stats
* Fix memory corruption on punycode
* Fix strings allocation in punycode
* Fix error message (#491)
1.1.0:
* Incompatible change: sqlite3 and per_user behaviour:
Now both redis and sqlite3 follows the common principles for per-user
statistics:
1) If per-user statistics is enabled check per-user tokens ONLY
2) If per-user statistics is not enabled then check common tokens ONLY
If you need old behaviour, then you'd need to use separate classifier
for per-user statistics.
* Implement redis statistics backend and cache
* Implement autolearning for statistics
* Reworked statistics architecture from scratch
* Add hyperscan (https://github.com/01org/hyperscan) engine for regular
expressions:
- add lazy loader for hyperscan databases
- rework regexp cache to have joint pcre/hyperscan scanning
- implement hyperscan pre-filter support
- add compilation guards for bad expressions
- implement `rspamadm control recompile` command
- implement hyperscan cache monitoring
- slides: <https://highsecure.ru/rspamd-hyperscan.pdf>
* Implement flexible task logging
* Rework fuzzy worker:
- it is now possible to run multiple fuzzy workers;
- implement lazy writing as sqlite3 is bad at concurrent writing;
- add retries for simple sql commands in fuzzy backend;
- use fine-grained transactions for fuzzy;
- implement new multi-pubkeys mode;
- allow encrypted only storages;
- rework statistics for fuzzy;
- add `rspamadm control fuzzystat` command for extended statistics;
- implement human readable output for the previous command;
- add condition script for learning fuzzy storage;
* Various fixes to SPF:
- fix `redirect` records;
- fix domains when parsing mx/ptr/a records in includes/redirects;
- fix issues with multiple addresses in SPF records;
- ignore SPF results in case of DNS failure;
- adjust TTL of records when resolving subelements of SPF records;
- always select `v=spf1` line if it is available
- do not cache records with DNS failure in subrequests;
- ignore records with temporary fails during subrequests resolving;
- fix `RDNS_RC_NOREC` support;
* Add clang plugin for static analysis:
- implement static checks for `rspamd_printf` format strings;
* Add 'allow_raw_input' option for non-mime messages
* Recognize types using libmagic
* Fix parsing of IPv6 received headers
* Add new interface of communication between workers in rspamd
* Add support for named socketpairs
* Don't write URLs by default as it is too verbose
* Set status for HTTP replies
* Try load `rspamd.conf.override`
* Implement words decaying for text parts to limit many checks
* Improve support of SA rules and plugins:
- add check_for_shifted_date and check_for_missing_to_header eval rules;
- add 'check_relays_unparseable' support;
- add `check_for_mime('mime_attachement')` function;
- use new re_cache interface for all SA rules;
- add support for `Mail::SpamAssassin::Plugin::MIMEHeader`;
- add support of 'special' SA headers to `exists` function;
- fix issue when SA metas contain other metas;
- fix freemail rules;
* Many fixes to the URL parser
* Match any newline character in regexps
* Fix resolving of upstreams and detection of poor IPv6 configurations
* Parse upstreams selection algorithm from the configuration line
* Add `reresolve` command to the control interface
* Generate fuzzy hashes from task metadata (URLs and headers)
* Add method to check if IP is local and `local_addrs` option
* Implement forced timeout for delayed filters
* Disable fast path of pcre-jit as it seems to be broken
* Bayes fixes:
- new normalizer function;
- really use weights of tokens from the OSB algorithm;
- restore multiple classifiers support;
* Rules changes:
- add `R_SUSPICIOUS_URL` rule that detects obfuscated URL's;
- improve empty image rule;
- rework `FORGED_RECIPIENTS` rule;
- reduce weight of `SUSPICIOUS_RECIPS`;
- fix `*_NORESOLVE_MX` symbols in hfilter;
- add `SUBJ_ALL_CAPS` rule with support of UTF8
- add spamhaus SBL to uribl
- fix `SUSPICIOUS_RECIPS` and `SORTED_RECIPS` rules
- remove `R_TO_SEEMS_AUTO` as it generates a lot of FP;
- add new Message-ID regexp for Thunderbird (by @moisseev);
* Plugins changes:
- allow ratelimit plugin to set symbol instead of pre-result
- support IP DNS black lists for URIBL (e.g spamhaus SBL);
- drop deprecated SURBL bits (by @fatalbanana)
- rename `JP_SURBL_MULTI` to `ABUSE_SURBL` (by @fatalbanana)
- add `SURBL_BLOCKED` (by @fatalbanana)
- add `CR_SURBL`
- SURBL: allow fallthrough to default symbol (by @fatalbanana)
- Settings: fix IP match (by @fatalbanana)
- SURBL: add missing symbols to metric (by @fatalbanana)
- allow processing images urls for SURBL
- unconditionally disable SPF for authenticated users and local networks
* Rework ratelimit plugin
- switch to `rates` instead of old and stupid strings to setup;
- check if a bucket is zero and disable the corresponding limits'
- turn off all buckets by default;
- check either `rcpt` or `user` buckets, not all together'
- document new `rates` and `symbol` options;
- inform user about what buckets are used in the configuration;
* Add neural network **experimental** plugin
* Add a sample script to learn neural network from rspamd logs
* Add documentation strings support to rspamd:
- add strings for the main configuration options;
- document workers options;
- add internal plugin options;
- create `rspamadm confighelp` routine;
- implement human readable output for the previous command;
- add subtree search support;
- add keyword search support;
* Documentation improvements, tutorials section, statistics description
* Many other minor and major bugfixes not noted here
1.0.11:
* Fix spf redirects
* Fix domains when parsing mx/ptr/a records in includes/redirects
* Fix unfolded base64 encoding
* Fix GError use-after-free
* Do not rewrite the original url when using redirector
* Fix parsing of fragment in urls
* Fix processing of HTML tags
* Improve empty image rule
* Avoid long double type
* Fix tokens weights in OSB algorithm
* Improve debugging for bayes
1.0.10:
* Fix settings application (#416)
* Fix another issue with fixed strings
* Fix hash function invocation
* Use the proper string for make_dns_request in lua_http
* Fix scan time output
* Update webui:
- fix labels for greylisting
- fix dimension of scan time
1.0.9:
* Emergency fix in keyed blake2 to fix fuzzy hashes and encrypted password
* Support passwords longer than 64 symbols
1.0.8:
* Add function to traverse AST atoms
* Allow dependencies on rspamd symbols for SA metas
* Fix memory corruption when timeout is removed in fuzzy check
* Fix encrypted fuzzy add processing
* Avoid use-after-free in controller session destructor
* Use session pool instead of task pool in fuzzy check
* Fix assembly in i386 mode (#413, #412)
1.0.7:
* Plugged memory leaks in internet address object & html parser
* Fixed static build
* Fixed multiple sigchld processing
* Fixed deletion of signal events after event processing loop
* Fixed build on ARM (#404 - reported by @Gottox)
* Fixed setting the default mask for SPF.
* Fixed sanitisation of HTTP query values
* Fixed parsing of the last header in encrypted HTTP messages
* Additions and fixes for test suite & benchmarks
* Added openssl aes-256-gcm support to libcryptobox & HTTP server
* Implemented support for starting multiple HTTP servers
* Implemented batch accept in HTTP server
* Added module to get data from HTTP headers (#285 - reported by @msimerson)
* Added `rspamadm control` command
* Added ability to sort counters output.
* Added ability to specify custom headers for rspamc client
* Fix architecture detection
* Converted history storage to the UCL format
* Allow flexible number of rows in history
* Fix action badges in WebUI
* Add universal cryptobox hash API
* Migrated to the optimized blake2b implementation adopted from Andrew Moon
* Allow explicit loading of specific modules
* Always load settings module
* Allow to add symbols from settings
* Fix double free in the controller fuzzy learn command
* Avoid endless loop when cannot open sqlite db
* Updated libucl
1.0.6:
* Fix build on i386
* Update CentOS7 service file patch (by @fatalbanana)
* Fix path to rspamadm in Debian init script (by @fatalbanana)
* Fix broken '_SC_GETPW_R_SIZE_MAX' on FreeBSD
* Fix portability issues
* Use cryptobox chacha for libottery
* Better support of 32 bit builds
* Fix header name tokens setup
* Fix levenstein distance method for words
* Add workaround for old libevent (#400)
* Fix microseconds in termination timer
* Fix some more issues with fixed strings
* Explicitly test CPU instructions even after CPUID call
* Do not check out of boundary memory
* Do not output broken emails
* Fix unknown symbols registration
* Handle SIGILL using longjmp
* Block signals when exiting event loop
* Fix incorrect allocation size
* Slightly optimize alignment
* Restore rspamd -t for compatibility
* Add more sanity checks for emails
1.0.5:
* Add rspamd control interface:
- support `stat` command to get runtime stats of rspamd workers
- support `reload` command to reload runtime elements (e.g. sqlite3 databases)
* Rework curve25519 library for modular design:
- add Sandy2x implementation by Tung Chou
- fix CPU detection for variables loading assembly
- add testing for curve25519 ECDH
* New fixed strings library
* Add `R_SUSPICIOUS_IMAGES` rule
* Enable mmap in sqlite3
* Use new strings in the HTTP code
* Improve google perftools invocation
* Improve performance profiling in http test
* Reorganize includes to reduce namespace pollution
* Allow specific sections printing in configdump command
* Rework workers signals handlers to be chained if needed
* Update socketpair utility function
* Add control_path option for rspamd control protocol
* Fix ownership when listening on UNIX sockets
* Rework signals processing in main
* Remove extra tools from rspamd (they live in rspamadm now)
* Remove global rspamd_main
* Add global timeout for the overall task processing (8 seconds by default)
* Sanitize NULL values for fuzzy backend
* Store NM between encrypt/decrypt
* Add textpart:get_words_count method
* Fix generic DNS request in lua
* Tune hfilter weights
* Add support of IPv6 in hfilter
* Fix parsing of HTTP headers with IP addresses
* Sync with the recent libucl
* Various minor bugfixes
1.0.4:
* Add configdump routine to rspamadm
* Implement retransmits for fuzzy_check plugin
* Fix events processing for learning anf checking fuzzy hashes
* Avoid dependency on unneeded and uncompatible glib include
* Add `historyreset` command to the controller
* Fix loading of tokenizer config from dump (#389)
* Add sorting hints for the history
* Allow custom lua scripts for users/languages extraction (#388)
* Do not add FORGED_RECIPIENTS when 'To' is missing (#387)
* Do not add R_UNDISC_RCPT when 'To' is missing (#387)
* Add encryption to fuzzy check plugin
* Add encryption for fuzzy storage
* Add new epoch for encrypted fuzzy request
* Add encryption for `rspamd.com` storage
* Remove gmime processing for LDA mode as it is deadly broken
* Add routine to find end of headers position in mime messages
* Fix LDA headers folding
* Init libraries in rspamc client as well to avoid locale issues
* Avoid collision with locally installed includes
* Allocate and free memory with the same allocator in rspamadm (#385)
* Preserve expired fuzzy hashes counter
* Improvements in webui:
- Add favicon.ico
- Rework history table
- Fix sorting for the history
- Migrate to bootstrap 3 and jquery 2
- Fix css bugs
- Add glyphicons
- Add reset history
- Improve history buttons
- Redraw graph to avoid display issues
- Webui is now MIT licensed to match licensing policy of rspamd
1.0.3:
* Fix piechart clean slice (#380)
* Fix controller crashes when GString is reallocated (#381)
* Correctly set locale before start
* Set C locale for numeric values
* Add rspamadm routine:
- add `pw` command to manage passwords
- add `help` command for displaying help
- add `configtest` command to check configuration files
- add `keypair` command for generating encryption keys
- add `fuzzy_merge` routine to merge fuzzy sqlite databases
- add a simple manual page for rspamadm
* Allow metric registration for composite expressions
* Add strict mode for configtest
* Add logger counters
* Save and show learned messages count (#383)
* Add `no_stat` flag
* Add `task:set_flag` and `task:get_flags` (#382)
* Enable foreign keys in sqlite3
* Remove orphaned shingles from fuzzy storage
* Optimize synchronization steps for fuzzy storage
* Allow delayed conditions registration
* Add lua API for conditions registering
1.0.2:
* Fix critical bug in webui that prevents password from being sent
* Rework webui view:
- Switch to d3.js for graphs
- Improve piechart look
- Rework colors for piechart
- Fix layout for symbols
- Fix refresh button
* Add descriptions for whitelist maps
* Fix build on arm (#379)
* Fix issue with the last element in the radix trie
* Add more tests for radix trie algorithm
* Allow to extract URLs from query strings of other URLs (#361)
* Initialize rrd fields before writing to file
* Fix double free if no password has been specified
1.0.1:
* Add writing to rrd from the controller
* Fixed lots of bugs in rrd code
* Adopt new DNS API in hfilter plugin (by @AlexeySa)
* Allow only one controller process to manage rrd file
* Set event base for fuzzy calls
* Improve fuzzy IO errors logging
* Add rra extraction function to rrd library
* Add graph handler to the controller
* Cache correct passwords to avoid too high CPU usage when working with webui
* Controller sockets are owned by router do not export them to task
* Optimize logging by skipping hash table search if it's empty
* Fix loading issue with broken statfiles
* Print assertions from glib to rspamd logger
* Load legacy `lua/rspamd.local.lua`
* Update webui with some fixes to learning and scanning
1.0.0:
* Rework symbols processing:
- Improve sorting logic for symbols
- Organize processing into multiple stages
- Added asynchronous watchers for symbols
- Added ability to organize dependencies between symbols
* Fixed URL redirector:
- Use optimized POE loop
- Organize dependencies
- Fix startup
* New sqlite3 backend:
- Allow to have per-languages and per-user statistics
- Allow sqlite3 to be used as statistics backend
* Store tokenizer configuration within statfiles
* Improve bayes statistics:
- Use headers and images metainformation in bayes
- Suggest using of pre-processed tokens for statistics
- Fix tokens normalization for OSB algorithm
* Rewrite url parsing:
- Fix numerous issues with url extraction and normalization
- Fix mailto urls
* Fix settings plugin to allow custom actions scores
* Improve rbl plugin
* Allow capturing patterns in rspamd lua regexp library
* Add GTUBE support
* Fix spamc legacy support
* Add DKIM support to RBL module
* Fix issues with multiple DKIM signatures
* Fix issue if rspamd cannot create statfiles (#331)
* Rework parts and task structure:
- Now text_parts, parts and received are arrays
- Pre-allocate arrays with some reasonable defaults
- Use arrays instead of lists in plugins and checks
- Remove unused fields from task structure
- Rework mime_foreach callback function
- Remove deprecated scan_milliseconds field
* Add ip_score plugin support (not enabled by default):
- Can check for asn/country and network using DNS lookups
- Can store and load reputation from redis server
* Improve PARTS_DIFFER rule to count merely different words
* New HTML parser:
- Parses HTML parts using a set of state machines
- Extracts useful data and exports it to lua functions:
+ Styles
+ Images
+ URLs
+ Colors
+ Structure elements
- Added HTML rules for some checks
* New version of LUA DNS API
* Table versions of many functions in LUA API
* Improve rspamc client:
- Print execution time
- Allow executing of external commands and passing output to them
- Allow mime output mode when rspamc alters message according to rspamd
checks and send it to an external command or stdout
* Allow scanning of local files using HTTP requests
* Rework configuration system:
- Rules are now moved from the $CONFDIR to $RULESDIR to avoid ambiguity
- All modules configurations are now split in $CONFDIR/modules.d/* to
simplify upgrades
- Move hfilter to plugins
- Allow plugins and rules to define default scores to simplify metrics
setup
- Include overrides for all modules to honor local/automatic parameters
- Tune scores for many modules
* Rework and enable DMARC plugin
* Add whitelist plugin for SPF/DKIM/DMARC based whitelisting
* Add some common domains to whitelists shipped with rspamd
* Rework logging:
- Now each log entry supports module name and a `tag`. Tag is used to
identify unique objects (such as tasks) when checking log files
- It is possible to turn on debugging for the specific modules
- Systemd logging is fixed
* Improve spamassassin plugin.
- Now headers are matched more like SA
- Improve support of Message-ID
- Add support of ToCc header type
- Fix :addr and :name in headers regexps
* Resurrect rrd support code
* Save controller stats between restarts
* Fixed tonns of bugs
* Added tonns of minor improvements and features
* Added more unit tests
* Create functional tests framework
* Added documentation for missing modules
* Added rpm/deb repositories and scripts
* Updated WebUI and libucl externals
0.9.10:
* Do not dereference null pointer on learning.
* Fix some extreme cases in BAYES.
* Add a workaround to avoid bad HTML messages breaking.
* Build with -O2 flags by default.
* Add constraints to limit DNS requests count per task.
* Add workaround for SURBL DNS flood.
* Set error if rspamd cannot learn anything.
0.9.9:
* Don't use RWL_SPAMHAUS_WL (unknown result) for whitelisting (by @fatalbanana)
* Import updated public suffix list (by @fatalbanana)
* Remove debug message
* Fix settings (by @fatalbanana)
* Remove duplicated symbol registration
* Use WAL for fuzzy storage
* RBL fixes (by @fatalbanana):
- silence errors;
- yield unknown results from RBLs;
- fix scoring for DNSWL;
- fix use of RBL name as symbol;
- ignore RBL names that would not be yielded;
* Support captures in regular expressions
* Add captures support to lua_regexp
* Support dist on FreeBSD and Darwin
* Add RCVD_IN_DNSWL_NONE as whitelisting exclusion (by @fatalbanana)
* Multiple fixes to URL detection:
- support port definition;
- fix query and path recognition;
- fix parsing of multiple slashes in URL;
- fix parsing query just after port;
- fix path field in `url:to_table` method;
- improve support of IP based URLs.
* Set ignore_whitelists = true for RECEIVED_SPAMHAUS_XBL (by @fatalbanana)
* Add GTUBE support
* Ignore User header in SA mode
0.9.8:
* Fix critical bug in bayes classifier (#305)
* Fix critical bug in RBL module (by @fatalbanana)
* Fix and rework settings plugin.
* Fix get_all_opts for a case of non-iterable options.
* Use tld for redirector's matching.
0.9.7:
* Add whitelist_exception setting to RBL module (by @fatalbanana)
* Don't use RWL_MAILSPIKE_POSSIBLE or DNSWL_BLOCKED for whitelisting (by
@fatalbanana)
* Fix extreme cases in bayes classifier.
* Fix parsing of urls with '?' at the end of hostname.
* Update interface.
* Fix number of issues with webui interaction.
* Fix saving maps.
* Allow user@ and @domain matches in multimap.
* Fix issues with bounces From processing.
* Fix abs/fabs misuse.
* Fix builds on suse and arch linux distributions.
0.9.6:
* Fix memory leak if mime cannot be parsed.
* Fix dkim cache expiration.
* Fix issues with redirector HTTP response.
* Fix abnormal connection closing with certains messages with a high score
(issue #296)
* Fix redirector installation.
* Use specific POE loop for some systems.
* Fix number of issues in URL redirector.
* Fix selecting URLs for sending to redirector.
0.9.5:
* Avoid double free when extending HTTP message.
* Fix double free if multiple classifiers are defined.
* Fix misprint in spamassassin plugin.
* Fix cpuid invocation on i386.
* Fix ownership issues for zero-copy decode.
* Allow __len metamethod on rspamd{text}.
* Add base64 decoding lua utility.
* Fix build on FreeBSD
* Skip spaces at the beginning of mime messages.
* DBL_ABUSE_REDIR should not have significant weight.
* Allow to split by lua_regexp rspamd{text} objects.
* Allow to specify custom stop pattern for lua_tcp.
0.9.4:
* Fix critical bugs in tokenization algorithm
* Write unit tests for tokenization
* Add documentation for lua_tcp
* Switch off legacy tokenization by default.
* Fix critical bugs in words normalization
* Add lua bindings to tokenizer.
* Implement storing of HTTP headers inside task
* Add lua API to accerss HTTP headers data
* Implemented base64 encoding suitable for MIME
* Use caseless hash and equal functions for HTTP request headers.
* Improve debian architectures support (by @dottedmag)
0.9.3:
* Revert incorrect regexp change that broke the default rules
* Fix lua_tcp module
0.9.2:
* Fix error on spawning unique workers.
* Add preliminary version of generic LUA TCP requests API.
* Use lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc)
* Fix fuzzy mime strings with only type.
* Improve thunderbird sanity checks.
* Fix critical bug on matching regular expressions.
* Make hiredis optional dependency.
* Fix multiple bugs in daemon reloading
0.9.1:
* Restore utf8 validation for regular expressions to avoid crashes
* Fix symbols displaying in the interface
* Add symbol groups to the interface
* Fix maps ID parsing in the controller
* Add multimap and regexp modules documentation
* Backport fixes from libucl
* Fix debian package (by @dottedmag)
* Rework XXH32 invocations
0.9.0:
* Add support of the fast and secure protocol level encryption:
- curve25519 is used for key exchange;
- chacha20/poly1305 cryptobox construction for bulk encryption;
- zero latency overhead;
- encrypting and balancing HTTP proxy worker
* Rework expressions and create new expressions library:
- aggressive optimizations based on the abstract syntax tree;
- abstract expressions support (regular expressions, functions, lua modules
composites and so on)
- New comparison and '+' operators support
- New greedy algorithm to minimize execution time of expressions and
all symbols
- Dynamic expressions benchmark and reoptimizations
* Many improvements to the LUA API:
- reworked logger module allowing to do pretty print of the most of lua
types (including tables and userdata classes)
- reworked lua redis and lua HTTP to support more features
- added opaque type for passing large text chunks without copying
- new regexp module with many auxiliary functions (e.g. `re:split`)
* LuaJIT is now the default requirement for rspamd allowing to speed up lua
execution by a large margin (however, plain lua is still supported)
* New plugins:
- spamassassin rules plugin that allows to load and re-use the most of
SA rules natively
- DMARC plugin that evaluates SPF and DKIM policies to the domain policies
- many old plugins has been reworked to implement new features and improve
stability
* New aho-corasic trie implementation from @mischasan that allows to load and
use hundreds of thousands of patterns with no influence on load
* Support of PCRE JIT and PCRE JIT fast path modes that significantly improves
the performance of regular expressions if supported by PCRE
* New URLs parser and extractor:
- removed legacy code that was useless for url finding
- reworked algorithms of URL parsing for more precise and accurate results
- added top-level-domains tree from http://publicsuffix.org
- improved emails parsing
- removed many phishing false positives due to TLD tree check
* New statistics infrastructure:
- created a separate layer of statistic library
- improved OSB-Bayes by re-weighting tokens according to the original
academic paper and `crm114` implementation, which reduced false positives
rate significantly
- created learn cache to avoid double learning of statistics and providing
an efficient way to re-learn class for a message
- created abstract layers for different statistics backends
- implemented new tokenization algorithms with fast or secure (siphash)
hashes to generate statistics features
* Reworked utf8 tokenization that previously corrupted all UTF8 words (minor
incompatibility with old fuzzy hashes with utf-8 symbols)
* SPF module has been completely rewritten to support complex cases of
`include` and `redirect` within SPF records
* DKIM module now supports multiple signatures
* Controller passwords can now be stored encrypted by `PBKDF2-HMAC` in the
configuration file
* Many hand-written HTTP clients has been replaced with the common rspamd
http module
* New test framework:
- import lua `telescope` test framework
- add unit tests for many rspamd modules and routines
- create a unit test for each possible bug found
- use luajit ffi for testing C code
- added preliminary support of functional testing by creating tasks from lua
* Randomize hash seed to avoid certain hash tables vulnerabilities
* Documentation improvements:
- added documentation for the vast majority of rspamd modules
- added documentation for rspamd protocol
- added documentation for the most of rspamd LUA extensions
* Fixed tonns of bugs and memory leaks
* Added tonns of minor features
0.8.3:
* Various critical fixes in distribution (by @dottedmag and @fatalbanana)
* Fixed bugs in url detector to parse certain patterns
* Add default host and helo for a client
* Some sanity checks for tokenizer and classifier
* Reiterate on systemd support
* Fix missing symbol registration
* Add support of spamc compatible output
* Filter double-dots in rbl.lua validate_dns (by @fatalbanana)
* Update ucl submodule due to critical bugfix
0.8.2:
* Create fuzzy db if it does not exist
* Fix: Centos init script: configtest() (by @AlexeySa)
* Enable one_shot for RECEIVED_SPAMHAUS_XBL - Fixes #102 (by @fatalbanana)
* Update Exim patch (by @fatalbanana)
* Fix processing of unix sockets.
* Allow applying settings to authenticated users (by @fatalbanana)
* Make settings priorities work as documented (by @fatalbanana)
* Fix race condition in symbols planner
* Add DNSWL_BLOCKED symbol (by @fatalbanana)
* Make Exim pass usernames to rspamd (by @fatalbanana)
* Update RBL module (by @fatalbanana):
- fix indentation;
- collapse loops;
- avoid calling for un-needed information;
- allow disabling RBLs for authenticated users
* once_received.lua: Fix indentation & add exclusion for authenticated users (by @fatalbanana)
* hfilter.lua: Add exclusion for authenticated users (by @AlexeySa)
* Updates to hfilter rules (by @AlexeySa)
* Set empty <> user or addr for msgs without FROM (by @eneq123)
* Fix: attempt to index field '?' (a nil value) (by @eneq123)
* Fix: if not exist Date-header (by @AlexeySa)
* Add task:get_content() method.
* rbl.lua: Ignore private IP space (by @fatalbanana)
* Allow to check radix maps from lua by rspamd{ip}
* Make local exclusions configurable per-RBL (by @fatalbanana)
* Add rspamd_config:radix_from_config() (by @fatalbanana)
* Support emails dnsbl in rbl (by @fatalbanana)
* Complete rework of url extraction logic
* Allow customizations for unix sockets. (fixes #182)
* Set lua path according to rspamd settings.
* Import lua-functional for plugins stuff.
* Completely rewrite multimap plugin in functional style.
* Fix FORGED_MUA_THUNDERBIRD_MSGID (fixes #186)
* Check IPv6 addresses at dnswl.org and Spamhaus whitelist (by @fatalbanana)
* Add lowercase utility for utf8 strings.
* Various fixes to build system
* Updated debian configuration infrastructure (by @dottedmag)
0.8.1:
* Add sqlite and perl as dependencies for RPM/Debian packages (by @fatalbanana)
* Remove whitelist.lua from RPM file list (by @fatalbanana)
* Make Exim pass hostnames to rspamd (by @fatalbanana)
* Fix building on Fedora (by @fatalbanana)
* Add toggle for disabling installation of systemd units on Linux (by @fatalbanana)
* Fix double format rounding that caused output corruption (reported by @fatalbanana)
* Revert broken change for destructors ordering that led to memory corruption
* Do not reset symbols case of settings if parsed from lua (reported by @andrejzverev)
* Fix build on SunOS (by @wiedi)
* Fix multiple crashes on broken DKIM DNS records
* Fix critical issue with composites weights removing
* Fix memory corruption in composites processing code
* Ignore non-SPF TXT records when parsing SPF includes
0.8.0:
* New fuzzy check logic:
- use shingles algorithm for fuzzy matching
- use blake2 instead of md5 for larger output space
- combine fuzzy and strict matching
- allow to organize private storages by means of keys
- preserve compatibility with previous versions
* New fuzzy storage:
- use sqlite instead of own memory based hash tables
- rework commands interface
- add conversion from the old format
- add fuzzy match by shignles
- support old rspamd versions
* Add lemmatizing for words used in fuzzy hashes that allows to improve match
quality by using of the first forms of all words
* Rework language detection
* Fix several critical bugs, memory leaks and deadlocks:
- memory leak in HTML nodes parsing
- deadlock in logger code
- deadlock in signals processing
- crashes in fuzzy_storage
- crashes in tokenizers if the input was empty
* Import new libucl with several bugfixes and improvements
* Support listening on ipv6 addresses only
* Fix macro expansion in SPF module
* Several bugfixes in DKIM module
* Add load headers support for mime parts to the lua API
* Add documentation for:
- workers in general
- fuzzy_storage worker
- fuzzy_check plugin
- mimepart and textpart lua API modules
0.7.6:
* Apply boundary fix for dkim simple canonization
* Fix ping command
* Return nil if header was not found in lua_task
* Fix hang in upstreams revive logic
* Decode entitles when normalizing HTML parts
* Fix logic of finding URLs in HTML parts
* Do not include \0 into length of text when performing conversion to utf8
* Fix raw vs parsed reperesentations
Raw parts are now:
- decoded b64/qp, but *NOT* converted to utf-8
Processed parts are now:
- converted to UTF-8
- normalized if needed (e.g. HTML tags are stripped)
* Rework DKIM canonization to line based
* Fix fuzzy hashes adding
* Use more specific hash function for fuzzy
* Fix leaking of iconv descriptors
* Fix PTR resolving in lua resolver
* Rework spf module.
- Copy data to memory pool as cached record might be destroyed causing
freed memory being passed to the protocol output (use after free)
- Allow SPF_NEUTRAL policy to be handled separately
- Add R_SPF_NEUTRAL to the default config
* Rework `register_symbols` function
* Allow to disable components of hfilter
0.7.5:
* Fix owner when creating folder /run/rspamd (by @sfirmery)
* Fix IP validity checks
* Decode URLs obtained from HTML tags
* Fix crash with unweighted upstreams
* Stop processing headers in parts
* Set sockaddr.sa_family properly when connectig to upstreams
* Fix reload issues in surbl and fuzzy_check (reported by @citrin)
* Fix timeouts in redirector
* Improve lua errors reporting
* Fix lua closures processing in libucl
* Rework calling of lua functions from regexp module
* Choose raw regexp for raw headers
* Rework conversion to utf since glib one is broken
* Ignore SGML style tags in html
* Fix old bug with non-capturing https urls
* Fix memory corruption on fuzzy reload (reported by @citrin)
* Fix percents display in rspamc
* Fix buffer update for DKIM
* Do not validate utf for raw headers
0.7.4:
* Fix build under *BSD
* Detect HAN unicode script
* Implement language detection heuristic for text parts
* Fix time output in history
* Improve piechart coloring
* Fix \r\n conversion in DKIM module (reported by @citrin)
* Try to detect systems with no IPv6 support
* Fix multiple/single values in use settings (reported by @citrin)
* Rework IP addresses in upstreams:
- Select ipv4/unix addresses if they exist and use ipv6 for ipv6 only
upstreams (since the support of ipv6 is poor in many OSes and
environments)
- Free IP list on upstream destruction
- Add test cases for addresses selection
- Allow adding of free form IP addresses to upstreams
* Fix endianness in lua_radix search (reported by @citrin)
* Soft shutdown should also set wanna_die flag (reported by @citrin)
* Stop use-after-free in event loop termination
* Fix processing of very short messages in DKIM (reported by @citrin)
* Detect systems without shared mutexes
* Fix issues with PTR and MX elements in SPF parser (reported by @citrin)
0.7.3:
* New upstreams code:
- simplify upstreams API;
- unify strings parsing in upstreams definition;
- add configuration options for the upstreams;
- for failed upstreams re-resolve their addresses;
- use all resolved addresses for an upstream (round-robin);
- implement stable hashing and use it by default for upstreams;
- add unit test for upstreams module.
* Rework signals processing in all rspamd workers:
- signals are now processed in the event loop;
- implement the most common signal handlers for all workers;
- add callbacks for workers specific signal handlers
* Fix critical issue with fuzzy storage:
Fuzzy stroage could not save any hashes on termination due to bugged
signals handling
* Fix roll history IP storage
* Rework ipv4/ipv6 handling in parsing addresses:
- turn off support of IPV6_V6ONLY socket option;
- create ipv6 socket prior to ipv4 one to handle systems with v6/v4
sockets enabled (Linux)
* Remove CBL as it's wholly included in Spamhaus XBL (by @fatalbanana)
* Remove nszones.com fake RBL (by @citrin)
* Fix upstreams interaction for fuzzy_check
* Verify spf PTR records (reported by @citrin)
* Fix spf MX records parsing
* Add compatibility for old libevent (by @yellowman)
* Sync bugfixes from libucl
0.7.2:
* Convert all maps to the compressed radix trie
* Allow IPv6 addresses in IP maps
* Remove dynamic items support from symbols cache
* Allow hex encoded output of strings
* Fix bug with control connections count
* Process fuzzy weight correctly (reported by @fatalbanana)
* Remove extra reference retain of http connection on error
* Remove deprecated options from the default config
* Add `one_shot` attr to metric's symbols
* Doc: add documentation for metrics
* Add Upstart job to debian packaging (by @CameronNemo)
* Config: improve SURBL symbols descriptions (by @citrin)
* Config: reflect SURBL changes (by @citrin):
- Outblaze removed, malware moved to separate list:
http://www.surbl.org/news/internal/MW-malware-sublist-added-to-multi
* Fix C modules initialization on restart
* Treat single IP as a single IP in radix lists (reported by @citrin)
* Do not touch file and core limits if not asked explicitly (reported by @citrin)
* Improve logging for fuzzy errors
* Block SIGPIPE for HTTP writing
* Doc: update manual pages
* Fix HTTP connection termination
* Reduce default number of parallel requests to 8
* Sync with libucl include features
0.7.1:
* Fix typo in stat output.
* Fix issues with includes crossing with the system includes
* Restore testing framework
* Add radix trie test suite
* Implement new path-compressed radix trie.
- The performance benefit over the old algorithm is about 1.5 times.
- Memory usage is significantly lower as well.
- Now radix trie can accept any IPv4/IPv6 values
* Various improvements to the memory pools code
* Fix writing reply to a client when no filters are defined
* Write base32 encoded fuzzy
* Fix 'soft reject' action
* Fix rspamd reload and modules reconfiguration
* Fix subject rewriting for the default subject
* Fix states for processing task and pre-filters
* Fix issues with connection closing
* Fix crashes in rdns
* Fix ratelimit pre-filter
* Update exim patch.
- Update to the recent exim version
- Strip extra leading src/ from the patch
- Remove sendfile since it was broken
- Fix rspamd spam report for exim
* Improve documentation
0.7.0:
* Use HTTP protocol for all operatiosn
* Webui worker is now removed and controller works as webui
* Allow to serve static files via controller by option `static_dir`
* Rspamd interface is now a part of rspamd
* Rspamc client has been rewritten to use HTTP and non-blocking mode
allowing to start multiple operations simultaneously (see `-n` option)
* Lua API was completely reworked to satisfy modern standards of LUA:
* Module `lua-message` was removed
* Reduced number of superglobals registered by rspamd
* Many functions has been redesigned
* Symbols registration is now more convenient
* Users settings has been rewritten as lua plugin
* Reworked headers system as gmime's based one misses many headers and is
very slow to get headers values
* Reorganized code and removed embedded jannsson by using UCL for all json
parsing
* Migrated to `librdns` for DNS resolving that improves concurrency for
DNS requests significantly
* Fixed tonns of bugs in MIME processing
* Improved metrcis and default symbol's weights
* Added new RBL's
* Fixed a number of issues in the modules
* Removed several memory leaks found
* Fix unicode processing
* Fix fuzzy checking for unicode parts
* Significantly improve documentation and especially LUA API docs
* See migration notes at https://rspamd.com/doc/migration.html
0.6.8:
* Controller now listen for localhost and not for 127.0.0.1 by default
* Allow FCrDNS-style RBL lookups (by @fatalbanana)
* Reduce threshold for parts_differ function.
* Fix hostname lookup for rdns rbl (by @AlexeySa)
* Fix HFILTER_URL_ONELINE to reduce false positive rate.
* Fix whitelist module.
* Allow override system predefined settings without touching system ones
by .try_include macro (by @andrejzverev)
* Check for [ip.address]-style HELO and suppress lookups. (by
@fatalbanana)
* Optimize hfilter (by @AlexeySa)
* Fix issue with random numbers generator in dns.
* Use more clever time values to setup entropy.
* Synced with the recent libucl.
0.6.7:
* Use ChaCha20 for DNS generator (more secure DNS id)
* Unknown symbols now has zero weight and not 1.0
* Fix fuzzy hashes expire time
* Fix critical issue in statfiles on FreeBSD 9 (and some other platforms)
* Add .include_map macro to ucl parser
* Update libucl
* Fix headers end detection for DKIM module
* Fix a bug in received headers parser
* Validate IP addresses before pushing them to lua
* Start new documentation project
* Fixed tonns of other minor bugs
* Start to prepare for 0.7 with HTTP protocol and new settings
0.6.6:
* Removed issue with BUFSIZ limitation in the controller output
* Simplify logging symbols escaping
* Adjusted weights for several rules
* Improve spamhaus rbl support
* Removed PBL for received headers checks
* Added hfilter module that performs various HELO and IP checks.
* Rspamd can now be reloaded by HUP signal
* Fuzzy storage should expire hashes properly
* Build system has been reworked for better supportof pkg-config
* Various minor bugfixes
0.6.5:
* Fixed critical bug in DNS resolver, introduced in 0.6.4
* Improved multimap and rbl plugins to skip
* Add dns_sockets option for tuning sockets per server in DNS resolver
* Improved packages for rspamd
0.6.4:
* Added io channels for DNS request to balance load and reduce id
collisions chance
* Fixed a bug in SPF filter that may cause core dump in specific
circumstances
* FIxed default config for rbl module
* It is possible to get a list of rspamc commands with their descriptions
* Added SORBS bl to the default config
* 2tld file for surbl module has been significantly extended
* Perl modules has been removed from the code.
* Fixed an issue in libucl when parsing macros
0.6.3:
* Fixed issues with DNS:
- labels decompression algorithm was fixed;
- added resolve_mx to lua API;
- fixed modules that use DNS.
* Lua modules once_received and emails reworked for new resolver API and UCL.
* Debian package was polished.
* Fixed a bug in fuzzy_check module that prevents correct processing messages
without valid parts.
0.6.2:
* Fuzzy check module has been reworked:
- now fuzzy_check operates with a group of rules, that define which
servers sre to be checked;
- it is possible to specify read_only groups to avoid learning errors;
- turn fuzzy_check to one_shot mode permanently;
- fuzzy_check module configuration is now incompatible with the previous
versions.
* Imported bugfixes from libucl.
* Fixed whitelist plugin.
* Fixed statfiles resizing.
* Improved logging initialization order.
* Fixed race condition in the controller worker.
0.6.1:
* Critical bugfixes:
- fixed build system;
- fixed in_class setting in bayes learning;
0.6.0:
* Use UCL instead xml for configuration (https://github.com/vstakhov/libucl)
* Fix statistics module normalization
* Rework the many modules for the new configuration:
- surbl has incompatible configuration;
- fuzzy_check has incompatible configuration;
- multimap has now new configuration;
- received_rbl is removed and replaced by rbl module.
* Removed deprecated options:
- statfile_pool_size;
- action and required/reject score for a metric.
* Simplify build system and unify configuration for all platforms.
* Improved URL detector (reduced false positive rate).
* Lua 5.2 is now the default and fully supported version.
* Tons of bugfixes and minor improvements.
0.5.6:
* Fix bug with counters incrementing in rolling history
* Detect expl and exp2l as some systems do not have it
* Support input streams without Content-Length
* Implement counters output via rspamc and controller interface
* Fix bug with udp sockets in fuzzy storage
0.5.5:
* New bayes normalizator based on inverse chi-square function
* Various fixes to fuzzy storage
* Allow update fuzzy storage only from specific IP addresses
* Better support of IPv6 and address selection algorithms
* Add CentOS spec file
0.5.4:
* Fixed issues with diff algorithm
* Added support of RRD statistics
* Add webui worker for interface interaction
* Fix a lot of issues with dynamic conf
* Fix critical memory leak in settings code
* Improve stability of the system
0.5.3:
* Added dynamic options
* Added advanced metaclassfication
* Added RESTfull API for controller
* Improved hashing algorithms
* Various fixes for rspamc client:
- allow interacting with unix sockets
- librspamdclient major cleanup
- bayes is now default classifier
0.5.2:
* Added lua bindings for:
- basic mime parts, that allows checkign attachements for example;
- DNS resolver;
* Existing lua bindings now works without task object allowing to use them
in custom code.
* Threads system was reworked to avoid global lua interpreter lock.
* DKIM module now converts all line endings to CRLF how opendkim does.
* URL detector is now more accurate for text parts.
* Several critical bugs and memory leaks were fixed.
0.5.1:
* Added lua worker type to handle network connections in lua.
* Added lua bindings for async_session, IO dispatcher, memory_pool, and
worker.
* Composites can now uses other composites in expressions.
* Fixes for debian package and for FreeBSD >= 9.1
* Add support of gmime-2.6 if gmime-2.4 is not found.
* Improve url detection and phishing detection.
* Add lua mime_part library to get an access to all message part
attributes (like filename, length, type).
0.5.0:
* Added SMTP lightweight balancing proxy with XCLIENT support.
* Added lua bindings for upstreams objects and API.
* New pre-filters are implemented to support initial checking for messages.
* Added ratelimit plugin that uses redis protocol to store data.
* Added ipv6 support to spf and some other modules.
* Unbreak spf plugin.
* Allow options with the same name be threated as list.
* DKIM plugin an parsing code was added.
* Separate build system to put logic in several shared libraries.
* Many bugfixes.
0.3.2:
* Add error handling for regexps
* Fix quit command in controller interface
* Write symbols weights to rspamc output
* Improve logic of selecting rspamc version
* Do not try to parse broken DNS replies
* Add 'raw' flag to FROM_EXCESS_BASE64 rule (requested by citrin)
* Output message id in rspamc reply
* Fix inserting composite symbol
* Fix output of log line
* Document composites
* Add logging for fuzzy checks
* Add logging for learning
* Improve logic of learning messages - do not learn more than specific threshold
* Fix inserting results for symbols that were incorrectly (for example more than 1 time) defined in config file
* Do not output control characters if output is not terminal
* Fix some logic errors in learning
* Consider lua plugins errors as fatal configuration errors
* Fix writing message id during fuzzy_add command
* Display weight of symbols correctly
* Fixes to winnow learning
* One more try to improve accuracy of winnow algorithm
* Add bayesian classifier (initial version)
* Remove normalizer as it is winnow specific thing, so all statistic algorithms now returns value from 0 to 1
* Some fixes to fuzzy hashes expiration:
* Fix assertion while look up value in NULL hash (found by cirtin)
* Fix normalization for systems that have not tanhl function
* Ignore rfc822 group addresses
* Move images library to core rspamd
* Add lua api to access images properties
* Add post filters to lua API - filters that would be called after all message's processing
* Add ability to check for specified symbol in task results from lua
* Add ability to check for metric's results from lua
* Add ability to learn specified statfile form lua
* Add ability to extract filename and size of images from lua
* Fix assertion while extracting internet address
* Fix races in fuzzy storage
* Make spf parser case insensitive
* Add ability to check hashes of selected mime types
* Add ability to set minimum size in bytes for mime types
* Add ability to set minimum dimensions for images
* Assume all text/* content types as text/plain
* Fix getting data wrapper for gmime24
* Many fixes to statfile synchronization system
* Fixed statfile pool initialization and synchronization with disk
* Prepare 0.3.2
* Fix Mail::Rspamd::Config for new rspamd features
* Use Mail::Rspamd::Config in rspamc client
* Write user's name to rspamd log
* Prepare rspamd build infrastructure for creating rpm and deb packages
* Fix depends
* Add start script for linux systems
* Fix shared usage of statfiles
* Add invalidation of statfiles in case of learning, so now statfiles
* This should fix shared usage of statfile pool by several processes
* Fix misprint (reported by az)
* Fix stupid error when all checks can be done in a single pass
* New trie based url scanner (based on libcamel)
* Small fixes to rspamd perl client
* Write fuzzy hashes info to log
* Add trie interface to lua api
* Explain sample config and cleanup it
0.3.1:
* Add modules documentation
* Continue implementing smtp proxy
* Implement new learning system, now rspamd should be much more intelligent while learning messages
* Convert statistic sums to use long double for counters
* Use hyperbolic tangent for internal normalizer
* In classify normalize result after comparing, not before
* New symbols sorter
* Fix strict aliasing while compiling with optimization
* Fix tanhl detection for platforms that have not implementation of it
* Remove several compile warnings
* Add experimental support of dynamic rules to regexp module
* Document views configuration
* Several fixes to documentation
* Add more logic for dynamic rules
* Add documentation for dynamic rules
* Add ability to make negations in networks in dynamic rules
* Clean up cache items correctly
* Implement basic SMTP dialog:
* Implement interaction with smtp upstream (with support of XCLIENT)
* Check messages received via smtp proxy
* Add support for sendfile in io dispatcher
* Fix issues with compatibility of worker_task and smtp proxy
* Proxy DATA command
* Fix SMTP
* Change metric logic
* Completely remove lex/yacc readers for config
* Make common sense of metric/action and symbols
* Sync changes with all plugins
* Incorrectly removed in previous commit
* Fix misprint (by Andrej Zverev)
* announce the "password" keyword in usage list
* Implement initial version of greylisting triplets storage
* Fix issues with smtp worker
* Fix QUIT command in SMTP worker
* Some fixes about new metrics system (may be incomplete)
* Get weights of symbol from default metric for symbols cache
* Fix setting task->from/task->rctp in smtp client
* Copy from and rcpt correctly
* Some performance improvements to IO dispatcher (do not drain the whole buffer after a single line readed)
* Fix smtp data input
* Fix misprint
* Add limit of maximum allowed smtp session errors
* New logic of SURBL module:
* Use system mkstemp(3) on systems where it is available as glib implementation
* Try to fix memmove issues in io dispatcher
* Remove debug from SURBL module
* Rewrite buffered input for line policy (again)
* Fix issue with links that are ip addresses in numeric form in surbl
* On Darwin use BSD style sendfile definition
* Reorganize platform specific knobs in CMakeLists
* Use gettimeofday on systems that have not clock_getres
* Use ftime for dns trans id generation on systems without clock_getres
* Darwin sendfile(2) support
* TIMEDB->TIMEB
* More to previous commit
* Pass env from main() arguments instead of platform specific global environ
* Fix compatibility issues
* Fix -lintl detection
* Init some variables to avoid problems
* Remove garbadge (gnome terminal sucks)
* Add more information about why we drop smtp connection
* Fix mkstemp call
* Send to upstream QUIT command at the end of session
* Check return value of each rspamd_dispatcher_write as in case of write errors sessions can be destroyed early
* Fix states in smtp dialog
* Use rspamd_snprintf instead of libc one
* Fix URLS command
* Fix reconfigure process of surbl module
* Fix destroying smtp session (unmap memory and do not delete pool early)
* Delete pool after using its variables
* Delay timer must be registered in async session to correctly handle connection termination
* Register dns requests in session too
* Make session before registering events
* Remove events in handlers
* Add ability to set filters for smtp worker for each smtp stage
* Add very initial version of DNS resolver (many things to be done)
* Announce weights and sync
* Fix few typo
* Understand short names of facility in logging config
* Add ability to make whitelist for spf checks
* Misprint != -> ==
* Handle lua tag in way that it is not required to write additional text:
* Strip all starting whitespace symbols from xml texts
* Fix stupid bug in calculating buffer length while reading file maps
* Add resolv.conf parsing into dns.c
* Fix microseconds<->milliseconds conversions
* Take callback argument in Mail::Rspamd::Client for processing files and directories
* Print results if rspamc is called for a directory
* Fix stupid error with surbl module reconfig (another one, blame me)
* Do not show duplicate urls in url header
* Fix detection of numeric urls (reported by citrin)
* Write real time of message's scan to log (not only virtual)
* Fix chartable module in utf mode
* Fix parsing of some broken urls
* Add ability to test regexp with 'T' flag
* Write more code for DNS resolver:
* Make DNS resolver working
* Many improvements to rspamd test suite: now it CAN be used for testing rspamd functionality
* Write DNS resolver tests
* Fix issues with memory_pool mutexes and with creating of statfiles
* Forgotten in previous commit
* Add support for parsing SPF and SRV records
* Fix PTR parsing
* Add tests
* Make SURBL module to use rspamd dns module
* Several fixes to DNS logic
* Remove evdns and use only rspamd resolver
* Very hard to detect problem with race among error in socket and destroying task while we are writing to socket and go through a hash table
* Fix resolving in smtp module
* Init events before configuring resolver in smtp worker
* Set resolver inside task
* Fix reload signal (reported by citrin)
* Some improvements to redirector
* Call has_forked method to inform POE about fork
* Fix lua DNS code
* Decompress labels in DNS packets more strictly
* Fix some problems with TXT records
* Try to fix removing of DNS events
* Do not insert unparsed RR's into reply
* Calling callbacks may cause destroying session from which we are calling callback so we MUST call callback as the latest action
* Fix check_smtp_data function
* Add ability to make views by recipient
* Add ability to set metric's action from config file
* Fix bug with writing garbadge if message has no urls or no messages
* Fix bug with incorrect behaviour of compare_parts_distance function
* Add ability to assign several actions to one metric
* Report action in rspamc protocol
* Mail::Rspamd::Client and rspamc can now understand Action header too
* Write action to log as well
* Make valgrind happy about comparing symbols
* Add more debug to comparing parts distance function
* Write action even if message has no symbols
* Make improvements to HTML entities decoder: now it replaces entities with common characters and
* Add -d option to force debug output
* Assume 7bit as default transfer encoding
* Do not overwrite lua plugins and configs if they already exists in target directory
* Improve logging
* Write queue id to log
* Remove test messages from cmake
* Reopen log file by USR1 signal
* Add reopenlog method to FreeBSD rc script
* Adopt foreach for cmake 2.6
* Fix to rc script
* Do not try to resolve names with several dots in a row
* Fix surbl request formatting for ip addresses
* Handle cases of broken requests
* Fix problems with parsing compressed names
* Fix TXT records parsing
* Fix expanding spf macros that may fail in rare cases
* Fix another error with early task destroying
* Handle empty from header
* Improve reopenlog command in rc script
* Strip trailing whitespace characters in Mail::Rspamd::Client
* Use ungreedy match to strip trailing whitespaces
* Stupid error in calculation compressed label length
* Some optimizations to client library
* Do not compare empty parts
* Empty and non-empty parts are always different
* Save in regexp cache the whole regexp with header name (if exists) and with flags
* Add rspamd_log variable to lua plugins to access logging functions
* Each part in rspamd task now can have parent part
* Check for parts distance only for multipart/alternative subparts
* Do not check attachements even if they are text (but attached as file)
* Do not die if write (2) returned ENOSPACE while doing logging, turn on throttling mode instead (1 write try in a second)
* Add ability to turn on debug for specific symbols
* Add ability to configure dns timeouts and dns retransmits in config file
* More debug
* Fix extracting arguments in lua logger interface
* Turn off debug_ip during reload if it was disabled by new config
* Improve lua logging
* Pre-init symbols cache when rereading config
* Fix lua representing of invalid ip (nil, not 255.255.255.255)
* Fix R_TO_SEEMS_AUTO rule (by citrin)
* Add multimap lua plugin
* Fix some multimap issues
* Try to save images hashes to fuzzy storage to stop some annoying spammers
* Allocate some more bytes for read buffer to avoid incorrect behavoiur
* Add ability to check dns black lists by multimap module
* Add multimap documentation
* Fix labels parsing
* Another try to save regexps in cache correctly
* Improve test logs for regexps
* Fix parsing txt records to avoid reading of uninitialized data
* Fix error with writing symbols cache file
* Fix error while working in utf mode when raw regexps was not created properly
* Do not add extra byte while converting text to utf
* Add error handling for regexps
|