You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

ChangeLog 309KB

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247124812491250125112521253125412551256125712581259126012611262126312641265126612671268126912701271127212731274127512761277127812791280128112821283128412851286128712881289129012911292129312941295129612971298129913001301130213031304130513061307130813091310131113121313131413151316131713181319132013211322132313241325132613271328132913301331133213331334133513361337133813391340134113421343134413451346134713481349135013511352135313541355135613571358135913601361136213631364136513661367136813691370137113721373137413751376137713781379138013811382138313841385138613871388138913901391139213931394139513961397139813991400140114021403140414051406140714081409141014111412141314141415141614171418141914201421142214231424142514261427142814291430143114321433143414351436143714381439144014411442144314441445144614471448144914501451145214531454145514561457145814591460146114621463146414651466146714681469147014711472147314741475147614771478147914801481148214831484148514861487148814891490149114921493149414951496149714981499150015011502150315041505150615071508150915101511151215131514151515161517151815191520152115221523152415251526152715281529153015311532153315341535153615371538153915401541154215431544154515461547154815491550155115521553155415551556155715581559156015611562156315641565156615671568156915701571157215731574157515761577157815791580158115821583158415851586158715881589159015911592159315941595159615971598159916001601160216031604160516061607160816091610161116121613161416151616161716181619162016211622162316241625162616271628162916301631163216331634163516361637163816391640164116421643164416451646164716481649165016511652165316541655165616571658165916601661166216631664166516661667166816691670167116721673167416751676167716781679168016811682168316841685168616871688168916901691169216931694169516961697169816991700170117021703170417051706170717081709171017111712171317141715171617171718171917201721172217231724172517261727172817291730173117321733173417351736173717381739174017411742174317441745174617471748174917501751175217531754175517561757175817591760176117621763176417651766176717681769177017711772177317741775177617771778177917801781178217831784178517861787178817891790179117921793179417951796179717981799180018011802180318041805180618071808180918101811181218131814181518161817181818191820182118221823182418251826182718281829183018311832183318341835183618371838183918401841184218431844184518461847184818491850185118521853185418551856185718581859186018611862186318641865186618671868186918701871187218731874187518761877187818791880188118821883188418851886188718881889189018911892189318941895189618971898189919001901190219031904190519061907190819091910191119121913191419151916191719181919192019211922192319241925192619271928192919301931193219331934193519361937193819391940194119421943194419451946194719481949195019511952195319541955195619571958195919601961196219631964196519661967196819691970197119721973197419751976197719781979198019811982198319841985198619871988198919901991199219931994199519961997199819992000200120022003200420052006200720082009201020112012201320142015201620172018201920202021202220232024202520262027202820292030203120322033203420352036203720382039204020412042204320442045204620472048204920502051205220532054205520562057205820592060206120622063206420652066206720682069207020712072207320742075207620772078207920802081208220832084208520862087208820892090209120922093209420952096209720982099210021012102210321042105210621072108210921102111211221132114211521162117211821192120212121222123212421252126212721282129213021312132213321342135213621372138213921402141214221432144214521462147214821492150215121522153215421552156215721582159216021612162216321642165216621672168216921702171217221732174217521762177217821792180218121822183218421852186218721882189219021912192219321942195219621972198219922002201220222032204220522062207220822092210221122122213221422152216221722182219222022212222222322242225222622272228222922302231223222332234223522362237223822392240224122422243224422452246224722482249225022512252225322542255225622572258225922602261226222632264226522662267226822692270227122722273227422752276227722782279228022812282228322842285228622872288228922902291229222932294229522962297229822992300230123022303230423052306230723082309231023112312231323142315231623172318231923202321232223232324232523262327232823292330233123322333233423352336233723382339234023412342234323442345234623472348234923502351235223532354235523562357235823592360236123622363236423652366236723682369237023712372237323742375237623772378237923802381238223832384238523862387238823892390239123922393239423952396239723982399240024012402240324042405240624072408240924102411241224132414241524162417241824192420242124222423242424252426242724282429243024312432243324342435243624372438243924402441244224432444244524462447244824492450245124522453245424552456245724582459246024612462246324642465246624672468246924702471247224732474247524762477247824792480248124822483248424852486248724882489249024912492249324942495249624972498249925002501250225032504250525062507250825092510251125122513251425152516251725182519252025212522252325242525252625272528252925302531253225332534253525362537253825392540254125422543254425452546254725482549255025512552255325542555255625572558255925602561256225632564256525662567256825692570257125722573257425752576257725782579258025812582258325842585258625872588258925902591259225932594259525962597259825992600260126022603260426052606260726082609261026112612261326142615261626172618261926202621262226232624262526262627262826292630263126322633263426352636263726382639264026412642264326442645264626472648264926502651265226532654265526562657265826592660266126622663266426652666266726682669267026712672267326742675267626772678267926802681268226832684268526862687268826892690269126922693269426952696269726982699270027012702270327042705270627072708270927102711271227132714271527162717271827192720272127222723272427252726272727282729273027312732273327342735273627372738273927402741274227432744274527462747274827492750275127522753275427552756275727582759276027612762276327642765276627672768276927702771277227732774277527762777277827792780278127822783278427852786278727882789279027912792279327942795279627972798279928002801280228032804280528062807280828092810281128122813281428152816281728182819282028212822282328242825282628272828282928302831283228332834283528362837283828392840284128422843284428452846284728482849285028512852285328542855285628572858285928602861286228632864286528662867286828692870287128722873287428752876287728782879288028812882288328842885288628872888288928902891289228932894289528962897289828992900290129022903290429052906290729082909291029112912291329142915291629172918291929202921292229232924292529262927292829292930293129322933293429352936293729382939294029412942294329442945294629472948294929502951295229532954295529562957295829592960296129622963296429652966296729682969297029712972297329742975297629772978297929802981298229832984298529862987298829892990299129922993299429952996299729982999300030013002300330043005300630073008300930103011301230133014301530163017301830193020302130223023302430253026302730283029303030313032303330343035303630373038303930403041304230433044304530463047304830493050305130523053305430553056305730583059306030613062306330643065306630673068306930703071307230733074307530763077307830793080308130823083308430853086308730883089309030913092309330943095309630973098309931003101310231033104310531063107310831093110311131123113311431153116311731183119312031213122312331243125312631273128312931303131313231333134313531363137313831393140314131423143314431453146314731483149315031513152315331543155315631573158315931603161316231633164316531663167316831693170317131723173317431753176317731783179318031813182318331843185318631873188318931903191319231933194319531963197319831993200320132023203320432053206320732083209321032113212321332143215321632173218321932203221322232233224322532263227322832293230323132323233323432353236323732383239324032413242324332443245324632473248324932503251325232533254325532563257325832593260326132623263326432653266326732683269327032713272327332743275327632773278327932803281328232833284328532863287328832893290329132923293329432953296329732983299330033013302330333043305330633073308330933103311331233133314331533163317331833193320332133223323332433253326332733283329333033313332333333343335333633373338333933403341334233433344334533463347334833493350335133523353335433553356335733583359336033613362336333643365336633673368336933703371337233733374337533763377337833793380338133823383338433853386338733883389339033913392339333943395339633973398339934003401340234033404340534063407340834093410341134123413341434153416341734183419342034213422342334243425342634273428342934303431343234333434343534363437343834393440344134423443344434453446344734483449345034513452345334543455345634573458345934603461346234633464346534663467346834693470347134723473347434753476347734783479348034813482348334843485348634873488348934903491349234933494349534963497349834993500350135023503350435053506350735083509351035113512351335143515351635173518351935203521352235233524352535263527352835293530353135323533353435353536353735383539354035413542354335443545354635473548354935503551355235533554355535563557355835593560356135623563356435653566356735683569357035713572357335743575357635773578357935803581358235833584358535863587358835893590359135923593359435953596359735983599360036013602360336043605360636073608360936103611361236133614361536163617361836193620362136223623362436253626362736283629363036313632363336343635363636373638363936403641364236433644364536463647364836493650365136523653365436553656365736583659366036613662366336643665366636673668366936703671367236733674367536763677367836793680368136823683368436853686368736883689369036913692369336943695369636973698369937003701370237033704370537063707370837093710371137123713371437153716371737183719372037213722372337243725372637273728372937303731373237333734373537363737373837393740374137423743374437453746374737483749375037513752375337543755375637573758375937603761376237633764376537663767376837693770377137723773377437753776377737783779378037813782378337843785378637873788378937903791379237933794379537963797379837993800380138023803380438053806380738083809381038113812381338143815381638173818381938203821382238233824382538263827382838293830383138323833383438353836383738383839384038413842384338443845384638473848384938503851385238533854385538563857385838593860386138623863386438653866386738683869387038713872387338743875387638773878387938803881388238833884388538863887388838893890389138923893389438953896389738983899390039013902390339043905390639073908390939103911391239133914391539163917391839193920392139223923392439253926392739283929393039313932393339343935393639373938393939403941394239433944394539463947394839493950395139523953395439553956395739583959396039613962396339643965396639673968396939703971397239733974397539763977397839793980398139823983398439853986398739883989399039913992399339943995399639973998399940004001400240034004400540064007400840094010401140124013401440154016401740184019402040214022402340244025402640274028402940304031403240334034403540364037403840394040404140424043404440454046404740484049405040514052405340544055405640574058405940604061406240634064406540664067406840694070407140724073407440754076407740784079408040814082408340844085408640874088408940904091409240934094409540964097409840994100410141024103410441054106410741084109411041114112411341144115411641174118411941204121412241234124412541264127412841294130413141324133413441354136413741384139414041414142414341444145414641474148414941504151415241534154415541564157415841594160416141624163416441654166416741684169417041714172417341744175417641774178417941804181418241834184418541864187418841894190419141924193419441954196419741984199420042014202420342044205420642074208420942104211421242134214421542164217421842194220422142224223422442254226422742284229423042314232423342344235423642374238423942404241424242434244424542464247424842494250425142524253425442554256425742584259426042614262426342644265426642674268426942704271427242734274427542764277427842794280428142824283428442854286428742884289429042914292429342944295429642974298429943004301430243034304430543064307430843094310431143124313431443154316431743184319432043214322432343244325432643274328432943304331433243334334433543364337433843394340434143424343434443454346434743484349435043514352435343544355435643574358435943604361436243634364436543664367436843694370437143724373437443754376437743784379438043814382438343844385438643874388438943904391439243934394439543964397439843994400440144024403440444054406440744084409441044114412441344144415441644174418441944204421442244234424442544264427442844294430443144324433443444354436443744384439444044414442444344444445444644474448444944504451445244534454445544564457445844594460446144624463446444654466446744684469447044714472447344744475447644774478447944804481448244834484448544864487448844894490449144924493449444954496449744984499450045014502450345044505450645074508450945104511451245134514451545164517451845194520452145224523452445254526452745284529453045314532453345344535453645374538453945404541454245434544454545464547454845494550455145524553455445554556455745584559456045614562456345644565456645674568456945704571457245734574457545764577457845794580458145824583458445854586458745884589459045914592459345944595459645974598459946004601460246034604460546064607460846094610461146124613461446154616461746184619462046214622462346244625462646274628462946304631463246334634463546364637463846394640464146424643464446454646464746484649465046514652465346544655465646574658465946604661466246634664466546664667466846694670467146724673467446754676467746784679468046814682468346844685468646874688468946904691469246934694469546964697469846994700470147024703470447054706470747084709471047114712471347144715471647174718471947204721472247234724472547264727472847294730473147324733473447354736473747384739474047414742474347444745474647474748474947504751475247534754475547564757475847594760476147624763476447654766476747684769477047714772477347744775477647774778477947804781478247834784478547864787478847894790479147924793479447954796479747984799480048014802480348044805480648074808480948104811481248134814481548164817481848194820482148224823482448254826482748284829483048314832483348344835483648374838483948404841484248434844484548464847484848494850485148524853485448554856485748584859486048614862486348644865486648674868486948704871487248734874487548764877487848794880488148824883488448854886488748884889489048914892489348944895489648974898489949004901490249034904490549064907490849094910491149124913491449154916491749184919492049214922492349244925492649274928492949304931493249334934493549364937493849394940494149424943494449454946494749484949495049514952495349544955495649574958495949604961496249634964496549664967496849694970497149724973497449754976497749784979498049814982498349844985498649874988498949904991499249934994499549964997499849995000500150025003500450055006500750085009501050115012501350145015501650175018501950205021502250235024502550265027502850295030503150325033503450355036503750385039504050415042504350445045504650475048504950505051505250535054505550565057505850595060506150625063506450655066506750685069507050715072507350745075507650775078507950805081508250835084508550865087508850895090509150925093509450955096509750985099510051015102510351045105510651075108510951105111511251135114511551165117511851195120512151225123512451255126512751285129513051315132513351345135513651375138513951405141514251435144514551465147514851495150515151525153515451555156515751585159516051615162516351645165516651675168516951705171517251735174517551765177517851795180518151825183518451855186518751885189519051915192519351945195519651975198519952005201520252035204520552065207520852095210521152125213521452155216521752185219522052215222522352245225522652275228522952305231523252335234523552365237523852395240524152425243524452455246524752485249525052515252525352545255525652575258525952605261526252635264526552665267526852695270527152725273527452755276527752785279528052815282528352845285528652875288528952905291529252935294529552965297529852995300530153025303530453055306530753085309531053115312531353145315531653175318531953205321532253235324532553265327532853295330533153325333533453355336533753385339534053415342534353445345534653475348534953505351535253535354535553565357535853595360536153625363536453655366536753685369537053715372537353745375537653775378537953805381538253835384538553865387538853895390539153925393539453955396539753985399540054015402540354045405540654075408540954105411541254135414541554165417541854195420542154225423542454255426542754285429543054315432543354345435543654375438543954405441544254435444544554465447544854495450545154525453545454555456545754585459546054615462546354645465546654675468546954705471547254735474547554765477547854795480548154825483548454855486548754885489549054915492549354945495549654975498549955005501550255035504550555065507550855095510551155125513551455155516551755185519552055215522552355245525552655275528552955305531553255335534553555365537553855395540554155425543554455455546554755485549555055515552555355545555555655575558555955605561556255635564556555665567556855695570557155725573557455755576557755785579558055815582558355845585558655875588558955905591559255935594559555965597559855995600560156025603560456055606560756085609561056115612561356145615561656175618561956205621562256235624562556265627562856295630563156325633563456355636563756385639564056415642564356445645564656475648564956505651565256535654565556565657565856595660566156625663566456655666566756685669567056715672567356745675567656775678567956805681568256835684568556865687568856895690569156925693569456955696569756985699570057015702570357045705570657075708570957105711571257135714571557165717571857195720572157225723572457255726572757285729573057315732573357345735573657375738573957405741574257435744574557465747574857495750575157525753575457555756575757585759576057615762576357645765576657675768576957705771577257735774577557765777577857795780578157825783578457855786578757885789579057915792579357945795579657975798579958005801580258035804580558065807580858095810581158125813581458155816581758185819582058215822582358245825582658275828582958305831583258335834583558365837583858395840584158425843584458455846584758485849585058515852585358545855585658575858585958605861586258635864586558665867586858695870587158725873587458755876587758785879588058815882588358845885588658875888588958905891589258935894589558965897589858995900590159025903590459055906590759085909591059115912591359145915591659175918591959205921592259235924592559265927592859295930593159325933593459355936593759385939594059415942594359445945594659475948594959505951595259535954595559565957595859595960596159625963596459655966596759685969597059715972597359745975597659775978597959805981598259835984598559865987598859895990599159925993599459955996599759985999600060016002600360046005600660076008600960106011601260136014601560166017601860196020602160226023602460256026602760286029603060316032603360346035603660376038603960406041604260436044604560466047604860496050605160526053605460556056605760586059606060616062606360646065606660676068606960706071607260736074607560766077607860796080608160826083608460856086608760886089609060916092609360946095609660976098609961006101610261036104610561066107610861096110611161126113611461156116611761186119612061216122612361246125612661276128612961306131613261336134613561366137613861396140614161426143614461456146614761486149615061516152615361546155615661576158615961606161616261636164616561666167616861696170617161726173617461756176617761786179618061816182618361846185618661876188618961906191619261936194619561966197619861996200620162026203620462056206620762086209621062116212621362146215621662176218621962206221622262236224622562266227622862296230623162326233623462356236623762386239624062416242624362446245624662476248624962506251625262536254625562566257625862596260626162626263626462656266626762686269627062716272627362746275627662776278627962806281628262836284628562866287628862896290629162926293629462956296629762986299630063016302630363046305630663076308630963106311631263136314631563166317631863196320632163226323632463256326632763286329633063316332633363346335633663376338633963406341634263436344634563466347634863496350635163526353635463556356635763586359636063616362636363646365636663676368636963706371637263736374637563766377637863796380638163826383638463856386638763886389639063916392639363946395639663976398639964006401640264036404640564066407640864096410641164126413641464156416641764186419642064216422642364246425642664276428642964306431643264336434643564366437643864396440644164426443644464456446644764486449645064516452645364546455645664576458645964606461646264636464646564666467646864696470647164726473647464756476647764786479648064816482648364846485648664876488648964906491649264936494649564966497
  1. 3.8.2: 20 Feb 2024
  2. * [Feature] Add extraction type for `from` maps
  3. * [Feature] Allow to add templates to redis history prefix
  4. * [Feature] Implement dynamic keys map in fuzzy storage
  5. * [Feature] Lua_url: Add `to_http` method
  6. * [Feature] Support JSON logging when in syslog mode
  7. * [Fix] Deal with `Connection` and `Host` headers on proxying
  8. * [Fix] Encode headers in metadata exporter
  9. * [Fix] Fix initial maps load
  10. * [Fix] Make stat tokens allocation consistent
  11. * [Fix] Resolve issue with bayes stat in `rspamadm` mode
  12. * [Fix] Try to fix url path issue
  13. * [Rework] Breaking: Do not report module as action
  14. * [Rework] Use khash instead of glib hashes for many reasons
  15. 3.8.1: 25 Jan 2024
  16. * [Fix] Fix headers insertion in the ordered list
  17. * [Fix] Fix learn error propagation
  18. * [Fix] Fix learning with long prefixes
  19. * [Fix] Fix potential double free by expclicitly zeroing pointers
  20. * [Fix] Fix storing of the bayes tokens
  21. * [Fix] Use short comments rather than long one
  22. 3.8.0: 19 Jan 2024
  23. * [Project] Rspamadm fuzzyping command
  24. * [Project] Use Redis scripts for Bayes statistics and cache
  25. * [Project] Support JSON logging
  26. * [CritFix] - Fix reported length of logging structure
  27. * [Feature] Allow to set `max_users` for Redis bayes backend
  28. * [Feature] Escape JSON when needed
  29. * [Feature] Proxy: Allow `encrypted_only` option
  30. * [Feature] Reiterate on gtube patterns
  31. * [Feature] Support ping command in fuzzy storage
  32. * [Feature] Support suppressing DMARC reporting for particular recipients
  33. * [Feature] rbl: support disabling or replacing url_whitelist per RBL
  34. * [Fix] Another try to fix setproctitle
  35. * [Fix] Cleanup session on exit
  36. * [Fix] Do not cleanup hyperscan files unless new ones are loaded
  37. * [Fix] Fix bad access when printing pending events on failure path
  38. * [Fix] Fix issues with the raw header on header insertion/modification
  39. * [Fix] Fix overflow in fuzzy_stats command
  40. * [Fix] Fix some corner cases when parsing maps with no newlines
  41. * [Fix] Fix various issues with canonicalisation of the paths
  42. * [Fix] Fix wrong raw_len usage
  43. * [Fix] Make words selection random deterministic upon content
  44. * [Fix] Properly set config field when creating tasks from Lua
  45. * [Fix] Really fix the language detector statistical heuristic
  46. * [Fix] Set loaded variable explicitly
  47. * [Fix] dkim_signing: siging_table: lowercase before lookup
  48. * [Fix] known_senders: fix config handling
  49. * [Fix] rbl: fix `exclude_local`
  50. 3.7.5: 15 Dec 2023
  51. * Add support for loongarch
  52. * [Fix] Fix rspamadm lua logger
  53. * [Fix] Fix systemd logging
  54. * [Fix] dkim_signing: siging_table: lowercase before lookup
  55. * [Minor] Fix build with no hyperscan
  56. * [Minor] Try improve test stability
  57. 3.7.4: 14 Nov 2023
  58. * [Enhancement] Add composite rule for suspicious URLs in suspicious messages
  59. * [Fix] Another try to fix setproctitle
  60. * [Fix] Do not cleanup hyperscan files unless new ones are loaded
  61. * [Fix] Fix various issues with canonicalisation of the paths
  62. * [Fix] Properly set config field when creating tasks from Lua
  63. * [Fix] Set loaded variable explicitly
  64. * [Fix] known_senders: fix config handling
  65. * [Fix] rbl: fix `exclude_local`
  66. * [Minor] Add missing <algorithm> include for gcc 14
  67. * [Minor] Add rule for messages missing both X-Mailer and User-Agent header
  68. * [Minor] Bundle fasttext as linux distributives are just useless
  69. * [Minor] Enable fasttext on RPM based linux
  70. * [Minor] Forgot it in one more place
  71. * [Minor] Improve FREEMAIL_AFF capture rates
  72. * [Minor] Limit fasttext to amd64 only
  73. * [Minor] Properly search for fasttext include
  74. * [Minor] Treat *.zpaq attachments as archives and harmful
  75. * [Minor] Update to 3.7.4
  76. * [Minor] force_actions: set a group for symbols
  77. * [WebUI] Update map editor
  78. 3.7.3: 27 Oct 2023
  79. * [Fix] Emergency fix for the hyperscan path error
  80. 3.7.2: 26 Oct 2023
  81. * [Feature] rbl: support checking returncodes by CIDR
  82. * [Feature] rbl: support checking returncodes by regex
  83. * [Feature] rbl: support globbed return codes
  84. * [Fix] DMARC reporting: fix reporting for subdomains
  85. * [Fix] Deal with fmtlib exceptions properly
  86. * [Fix] backport fix for dlfcn.h from backward-cpp
  87. * [Rules] Blank spam detection
  88. 3.7.1: 07 Oct 2023
  89. * [CritFix] Fix leak in `gzip` function
  90. * [Feature] Add ICAP Content-Type and Filename
  91. * [Feature] Add `logging`->`task_max_elts` option
  92. * [Feature] Add utility to split string like stuff for C++ code
  93. * [Feature] Allow to set HTTP auth parameters for the maps
  94. * [Feature] Check for plugin configuration errors on `configtest`
  95. * [Feature] `known_senders` plugin
  96. * [Feature] Use backward-cpp instead of manual libunwind stuff
  97. * [Feature] rbl: support checking numeric URLs in isolation
  98. * [Fix] CMakeLists.txt remove whitespace added by linter as it makes tests fail
  99. * [Fix] Change Date: header location to conform with RFC
  100. * [Fix] Correct format pattern for RE tree tempfile name
  101. * [Fix] Correct format string for unw_word_t
  102. * [Fix] Do not accept invalid ucl object types
  103. * [Fix] Do not pollute public headers with libev internals
  104. * [Fix] Do not set output type if list application failed
  105. * [Fix] Fix `url:set_redirected` method
  106. * [Fix] Fix format string and some length issues
  107. * [Fix] Fix grammar definition for content-disposition attributes
  108. * [Fix] Fix lua schema enrichment logic for Redis params
  109. * [Fix] Fix lua stack corruption when logging large tables
  110. * [Fix] Fix merge table utility
  111. * [Fix] Fix output of non-RSA DKIM keys
  112. * [Fix] Fix some corner cases of single-host urls parsing
  113. * [Fix] Fix various issues in the `url_redirector` plugin
  114. * [Fix] MISSING_MIMEOLE: avoid matching messages from Android GMail app (#4561)
  115. * [Fix] Prevent DNSWL sabotage
  116. * [Fix] Try to fix unzip function
  117. * [Fix] rbl: really fix dependency registration when symbols_prefixes is used
  118. * [Fix] rspamadm mime: arguments beginning with letter `t`
  119. * [Rework] Breaking: return back to semver
  120. * [Rework] Move rcl logic to C++
  121. 3.6: 03 Aug 2023
  122. * [Conf] Add `one_shot` to some specific multimap rules
  123. * [Conf] Add language detection configuration
  124. * [Conf] Add missing attributes for the language detection configuration
  125. * [Conf] Remove outdated composite rules
  126. * [Feature] Add `sentinel_password` option
  127. * [Feature] Add ability to deny specific fuzzy flags by default
  128. * [Feature] Add controller endpoint to get fuzzy hashes from messages
  129. * [Feature] Add extra symbol when URL redirector reaches nested limit
  130. * [Feature] Add function to transliterate utf8 to ascii with some normalisation
  131. * [Feature] Add html parsing limit
  132. * [Feature] Add order to urls structure
  133. * [Feature] Add some missing functions to `lua_rsa` library
  134. * [Feature] Allow fuzzy workers to exchange blocked information
  135. * [Feature] Allow to have weak flags in fuzzy storage
  136. * [Feature] Allow to read options from maps in the multimap plugin
  137. * [Feature] Allow to use other methods when fasttext detection is enabled
  138. * [Feature] Count stats per key per flag
  139. * [Feature] Finish all features of dkim_keygen in Lua
  140. * [Feature] Khash: Allow static initialisation
  141. * [Feature] Maps: Add on_load support
  142. * [Feature] Preliminary implementation of dynamic composites
  143. * [Feature] Process HTML parts before text ones
  144. * [Feature] Reorganise struct rspamd_url to be 64 bytes size
  145. * [Feature] Save fuzzy ratelimit buckets
  146. * [Feature] Use in-place deflation for strings
  147. * [Feature] external_relay: add ip_map strategy
  148. * [Fix] Avoid race between config new/free by using a counter
  149. * [Fix] Do not use `rspamadm.dkim_keygen`
  150. * [Fix] Feed fasttext language model with the pre-tokenized words
  151. * [Fix] Fix `rspamd_has_only_html_part`
  152. * [Fix] Fix an old issue with order of destruction race between redis pool and lua
  153. * [Fix] Fix format string usage
  154. * [Fix] Fix parsing due to old bug revealed
  155. * [Fix] Fix parsing of the mask values that are invalid
  156. * [Fix] Ignore non-unique stop words
  157. * [Fix] Include the last character when parsing the last header with no value
  158. * [Fix] More fixes to fuzzystat
  159. * [Fix] Set proper counter
  160. * [Fix] Try harder to clean pending bucket
  161. * [Fix] Try harder to remove bad hyperscan files
  162. * [Fix] Update stats before encryption...
  163. * [Fix] dmarc gramar - allow spaces before ";"
  164. * [Fix] rbl: fix dependency registration when symbols_prefixes is used
  165. * [Fix] remove obsolete rspamd-redirector files
  166. * [Fix] test external_relay: count should always be the last rule, as it have no matching condition
  167. * [Project] Allow to register multimap symbols dynamically
  168. * [Project] Implement fasttext language detection
  169. * [Rework] Default max shots must not influence options
  170. * [Rework] Write dkim keygen tool in lua
  171. * [Rules] Add thread hijacking composite rule
  172. 3.5: 17 Mar 2023
  173. * [Conf] Clarify `timeout` in rspamd_proxy
  174. * [Conf] Fix hashbl
  175. * [Conf] Add SURBL hashbl support (WIP)
  176. * [Conf] RBL: Fix selector
  177. * [Conf] Reduce the default timeout
  178. * [CritFix] Deserialise hyperscan to the page-aligned space to prevent alignment issues
  179. * [CritFix] Fill path field in hyperscan notice command
  180. * [Feature] Add `thresholds` field to the scan result
  181. * [Feature] Add possibility to execute lua scripts for blocked fuzzy clients
  182. * [Feature] Add preliminary support of the external maps in the multimap plugin
  183. * [Feature] Allow to build a map by combining tuples of selectors
  184. * [Feature] Allow to query external maps for settings
  185. * [Feature] Allow to specify `selector_alias` in the maps definition
  186. * [Feature] Enable Mime part filters on antivirus module
  187. * [Feature] Improve ratelimit redis scripts
  188. * [Feature] Selectors: Add specific_urls_filter_map extractor
  189. * [Feature] Some rework of the selectors framework
  190. * [Fix] Add O_CREAT flag when creating a file
  191. * [Fix] Add a database check function unless we have anything from Hyperscan
  192. * [Fix] Add hyperscan databases sanity check
  193. * [Fix] Add workaround for ENOBUFS error on sending
  194. * [Fix] Do not do `lstat` when we are creating file
  195. * [Fix] Finally get rid of RSPAMD_USE_47BIT_LIGHTUSERDATA_HACK
  196. * [Fix] Fix boundaries that contain only dashes
  197. * [Fix] Fix off-by-one error in css tokenizer
  198. * [Fix] Fix url reputation plugin
  199. * [Fix] Fix usage of the Redis config schema as `extra_fields`
  200. * [Fix] Further checks for the hs_scratch_alloc
  201. * [Fix] Honor group flag for one shot
  202. * [Fix] Normalize glob paths to avoid hash table misses
  203. * [Fix] Ratelimit: Use unpack for `HMGET` return value
  204. * [Fix] Rbl: Fix helo check pipeline
  205. * [Fix] Replace broken strict_domains with phishing_exceptions
  206. * [Fix] Restore `strict_domains` support
  207. * [Fix] Return true from has_urls(true) if only emails are present
  208. * [Fix] Rework lists applications
  209. * [Fix] Set symcache item in coroutine calls
  210. * [Fix] Treat `hs_allocate_scratch` errors as non-fatal
  211. * [Fix] Treat hostnames with no dots as eSLD of their own
  212. * [Fix] received: filtering of artificial header
  213. * [Project] Add external maps support
  214. * [Rework] Breaking: Do not report soft reject in history
  215. * [Rework] Convert chartable plugin to c++ for convenience
  216. * [Rework] Use a different approach for customization of the settings
  217. * [Rules] Mid: Add MID_END_EQ_FROM_USER_PART rule
  218. 3.4: 01 Nov 2022
  219. * [CritFix] Restore compatibility with the integrations and headers alterations
  220. * [Feature] Milter_headers: Add `x-rspamd-action` routine
  221. * [Feature] Share hyperscan database among processes
  222. * [Fix] Another corner case in url parsing
  223. * [Fix] Another fix for the enable password
  224. * [Fix] Another try to fix close method in lua_tcp
  225. * [Fix] Fix additional fields in the Redis schema
  226. * [Fix] Fix emoji joiner FP
  227. * [Fix] Fix favicon.ico Content-Type header
  228. * [Fix] Fix hang when close is used
  229. * [Fix] Lua_tcp: Sigh, another try to fix `close` invocation
  230. * [Fix] Mx_check: Cache the fact of a missing MX record
  231. * [Fix] Try to fix parsing of the unencoded `>` characters in html attributes
  232. * [Fix] Try to fix the case where password == enable_password
  233. * [Project] (Re)implement hyperscan caching
  234. * [Project] Rework cleanup
  235. * [Project] Synchronize hyperscan caches via the main process
  236. * [Rework] Convert multipattern to use hyperscan tools
  237. * [Rework] Make http normalize path function a generic function
  238. * [Rework] Split locked and unlocked files, as mmap does not need flock normally
  239. * [Rework] Start movement of the hyperscan related routines into a single unit
  240. * [Rework] Store the current worker, so other libraries could use this information
  241. * [Rework] Use blocking socket for IPC between main and workers
  242. * [Rework] Use more predictable size for commands buffers
  243. * [Rules] Do not insert ONCE_RECEIVED_STRICT on RDNS missing
  244. * [Rules] Reduce score of HTTP_TO_HTTPS - subject to remove completely
  245. 3.3: 01 Oct 2022
  246. * [Conf] Add missing groups for whitelist module symbols
  247. * [CritFix] Neural: Fix keys regression after #3968
  248. * [Feature] Accept upstream in lua_tcp
  249. * [Feature] Add ability to statically maintain disabled/enabled patterns
  250. * [Feature] Add function to store upstreams for HTTP urls
  251. * [Feature] Allow augmentations set in Lua API
  252. * [Feature] Allow lua_http module to accept upstreams
  253. * [Feature] Allow to limit write access to fuzzy storage by key
  254. * [Feature] Allow to sort symbols output
  255. * [Feature] Check content for binary stuff before dumping it to Lua
  256. * [Feature] Implement symbols augmentations
  257. * [Fix] Add missing flags
  258. * [Fix] Add more sanity checks for rua in dmarc_report
  259. * [Fix] Adjust length of the fuzzy checks for short text parts
  260. * [Fix] Another try to fix add headers compatibility logic
  261. * [Fix] Another try to fix race condition in the runtime destruction
  262. * [Fix] Avoid cyclic references in symcache and fix memory leaks
  263. * [Fix] Avoid overriding IP with Sender IP
  264. * [Fix] BAD_REP_POLICIES did not trigger when message was classified as spam by Bayes
  265. * [Fix] Bind AF_UNIX DGRAM client connection to annonymous address
  266. * [Fix] Disable IPv6 lookups for Blocklist.de RBL
  267. * [Fix] Distinguish dynamic and static items
  268. * [Fix] Dkim: Ignore unknown DKIM kv pairs as stated in RFC
  269. * [Fix] Dmarc report: Use local timezone instead of GMT
  270. * [Fix] Do not exclude authenticated users from URIBL lookups
  271. * [Fix] Empty envelopes should not be emitted as arrays (json+messagepack) when populated envelopes are objects. This greatly complicates decoding in strictly typed languages.
  272. * [Fix] External_relay: Restore the originating hostname check
  273. * [Fix] Fix DKIM keys with spaces still allowing errors on invalid base64
  274. * [Fix] Fix copying of sockaddr_un addresses
  275. * [Fix] Fix crash with cname replies
  276. * [Fix] Fix dependencies propagation
  277. * [Fix] Fix iteration over milter headers
  278. * [Fix] Fix ordering when sorting symcache
  279. * [Fix] Fix reading of the cached maps
  280. * [Fix] Fix several issues with the HTTP keepalive parsing
  281. * [Fix] Fix stack smashing
  282. * [Fix] Fix synchronous auth/select in lua_redis
  283. * [Fix] Fix various symcache issues
  284. * [Fix] Ignore all (I hope) unknown DKIM signature KV pairs
  285. * [Fix] Ignore directories in RarV5 archives
  286. * [Fix] Libucl: avoid memory leak on objects merging
  287. * [Fix] Lua_tcp: Another try to fix closing logic
  288. * [Fix] Mempool: Fix alloc_array function to actually multiply nmembers by size
  289. * [Fix] Only check allowed fuzzy worker update ips for non-unix sockets
  290. * [Fix] Plug memory leak in regexp destruction with pcre2
  291. * [Fix] Properly check the original email flag
  292. * [Fix] Properly deal with `get_symbol/get_metric_symbol` ambiguity
  293. * [Fix] Properly parse expressions atoms
  294. * [Fix] Properly set `Host` in rspamd_proxy
  295. * [Fix] Rbl: Fix received positioned checks
  296. * [Fix] Remove check for a score with no symbol being registered
  297. * [Fix] Same fix for lua_tcp
  298. * [Fix] Skip cname records when processing SPF records
  299. * [Fix] Skip sending dmarc reports in no-opt mode fixes https://github.com/rspamd/rspamd/issues/4241
  300. * [Fix] Stop slow timer on task destruction
  301. * [Fix] Symcache: Do not use C style comparators in C++ sorts
  302. * [Fix] Try to avoid a corner case for `@` pattern
  303. * [Fix] Try to fix dkim reputation adjustements
  304. * [Fix] Try to fix passthrough results processing logic
  305. * [Fix] Try to fix the mess with read only flag
  306. * [Fix] Upstreams: Don't ignore revive_time config option
  307. * [Fix] Use proper format string, sigh...
  308. * [Fix] Use space category in ragel automata to resolve space characters
  309. * [Fix] Zstd: Fix compression with the new Zstd API
  310. * [Fix] milter_headers: Header fields may be inserted at wrong position.
  311. * [Project] Add experimental HTTP statistics backend
  312. * [Project] Add more methods for symbols addition
  313. * [Project] Add raii_sink file helper
  314. * [Project] Add some more methods
  315. * [Project] Add symbols processing methods
  316. * [Project] Allow `=` separated augmentations to be treated as kv pairs
  317. * [Project] Allow to extract augmentation values
  318. * [Project] Few more methods
  319. * [Project] Fix on conditions
  320. * [Project] Further efforts to make a more consistent architecture
  321. * [Project] Further rework
  322. * [Project] Further rework tracking
  323. * [Project] Further split of the code
  324. * [Project] Get rid of C style ctors/dtors
  325. * [Project] Http_stat: Notice statfiles when creating runtime
  326. * [Project] Implement dynamic items lookup and processing
  327. * [Project] Implement item finalization
  328. * [Project] Implement more methods
  329. * [Project] Implement runtime creation
  330. * [Project] Implement settings processing + some neats
  331. * [Project] Implement some conditions checks
  332. * [Project] Implement validation logic
  333. * [Project] More methods
  334. * [Project] Move runtime cache part to a separate unit
  335. * [Project] Move some more methods
  336. * [Project] Re-implement counters method
  337. * [Project] Reimplement dependencies processing
  338. * [Project] Remove obsoleted methods
  339. * [Project] Remove old code (finally)
  340. * [Project] Rework symbols execution
  341. * [Project] Some more adjustments in symbols registration
  342. * [Project] Start rewrite symcache in c++
  343. * [Project] Support augmentations with values
  344. * [Project] Symcache: Use ordered filters to avoid extra lookups
  345. * [Rework] Another movement
  346. * [Rework] Augmentations can now imply flags
  347. * [Rework] Further steps
  348. * [Rework] Further work on deps processing
  349. * [Rework] Implement cache resorting
  350. * [Rework] Isolate disable/enable logic for the configuration ucl objects
  351. * [Rework] Move item implementation to a separate header
  352. * [Rework] Multimap: Avoid prefilters usage where augmentations can be used
  353. * [Rework] Pass upstream when sending TCP requests
  354. * [Rework] Re-implement cache sorting
  355. * [Rework] Reimplement saving/loading the cache items
  356. * [Rework] Reiterate on priorities
  357. * [Rework] Rework files structure
  358. * [Rework] Rewrite rspamc in C++
  359. * [Rework] Simplify scores check and extend it to pre/post filters
  360. * [Rework] Switch minimum C++ standard version to C++20
  361. * [Rework] Try to fix the mess with types & flags
  362. * [Rework] Use another version of hash table from the same author
  363. * [Rework] Use dynamic items for calling callbacks
  364. * [Rework] Use dynamic items in the callbacks
  365. * [Rework] Use hash map for id->symbol mappings
  366. * [Rework] Use khash instead of uthash in rdns compression logic
  367. 3.2: 26 Mar 2022
  368. * [Conf] Score MIME_OBFUSCATED_ARCHIVE to 8 points
  369. * [Conf] Set one_shot for URIBL rules by default
  370. * [CritFix] Fix upstreams name resolution when there is also a port
  371. * [Feature] Add ROC feature to neural network plugin
  372. * [Feature] Add public suffic compilation utility
  373. * [Feature] Add support of Cloudmark
  374. * [Feature] Allow hyperscan for ppc64, as vectorscan now suports it.
  375. * [Feature] Allow to skip DNS resolution for keep-alive connections
  376. * [Feature] Aws_s3: Allow to store large parts separately
  377. * [Feature] BIMI: Add preliminary version of the BIMI plugin
  378. * [Feature] JSON endpoint for querying maps
  379. * [Feature] Lua_magic: Add a sane CSV heuristic
  380. * [Feature] Lua_mime: Add schema for message transfer
  381. * [Feature] Output average scan time in /stat endpoint
  382. * [Feature] Show average scan time in `rspamc stat` output
  383. * [Fix] Add guards to avoid race condition on TCP connection
  384. * [Fix] Allow spaces in DKIM key records
  385. * [Fix] Apply the similar fix to the url_reputation
  386. * [Fix] Avoid overwriting whitelisted_signers_map
  387. * [Fix] Backport PR from libucl
  388. * [Fix] Clear SSL errors
  389. * [Fix] ClickHouse cleanup of old partitions
  390. * [Fix] Do not double call error handler on ssl errors in the timeout path
  391. * [Fix] Do not forget to clear pointers on IOC reset
  392. * [Fix] External_relay: Remove useless check of the map value
  393. * [Fix] Find suspicious url encodings that could break url extraction
  394. * [Fix] Fix HTTP(s) client timeout
  395. * [Fix] Fix exclude flags setting
  396. * [Fix] Fix expanding of the variables
  397. * [Fix] Fix host header usage in lua_http
  398. * [Fix] Fix http maps shared memory cache cleanup
  399. * [Fix] Fix logic in HTML processing FSM
  400. * [Fix] Fix parsing of the compound mailto urls
  401. * [Fix] Fix processing captures from pcre2
  402. * [Fix] Fix removing from khash
  403. * [Fix] Fix stuctured headers pushing
  404. * [Fix] Further fix for i386 compilation
  405. * [Fix] Improve duplicate settings error reporting
  406. * [Fix] Lua: task:remove_result didn't work in some cases
  407. * [Fix] Output service parts as well
  408. * [Fix] Phishing: Deal with phishing + redirected URL
  409. * [Fix] Phishing: Fix finding domains in the phishing map
  410. * [Fix] Plug memory leak by using mempool for a copied address
  411. * [Fix] Properly find the request and the number of requested entries
  412. * [Fix] Rbl: Fix inversed logic of the url_full_hostname
  413. * [Fix] Read file maps if they were not pre-read during preload
  414. * [Fix] Restrict x86_64 assembly to x86_64
  415. * [Fix] Return a real number of recipients when dealing with aliases
  416. * [Fix] Rework unshedule DNS request function
  417. * [Fix] Support definition of ungrouped symbol in conf file, use group info from lua or other conf file
  418. * [Fix] Unschedule DNS request when clearing IO channel
  419. * [Fix] When checking for phishing, we need to convert punicode -> UTF8, not vice versa
  420. * [Fix] lua_cfg_transform - actions without score (discard)
  421. * [Fix] lua_cfg_transform - silly break break actions
  422. * [Fix] ratelimit - symbol per bucket
  423. * [Project] BIMI: Fix helper integration issues
  424. * [Project] Further DNS over TCP architecturing
  425. * [Project] Rdns: Add more functions for TCP based requests
  426. * [Project] Rdns: Add preliminary reading logic for TCP channels
  427. * [Project] Rdns: Add reaper for inactive TCP connections
  428. * [Project] Rdns: Add timeout logic for TCP requests
  429. * [Project] Rdns: Do not treat TCP channels failure as fatal
  430. * [Project] Rdns: Fix TCP connection mess
  431. * [Project] Rdns: Fix TCP stuff cleanup
  432. * [Project] Rdns: Fix various ownership issues
  433. * [Project] Rdns: Implement TCP writing logic
  434. * [Project] Rdns: Initial support of TCP IO channels
  435. * [Project] Rdns: More fixes in TCP handling
  436. * [Project] Rdns: Restore the previous EDNS0 size
  437. * [Project] Rdns: Send truncated replies via TCP
  438. * [Project] Rdns: Unregister TCP requests
  439. * [Rework] Allow to restore SSL handlers after keepalive pooling
  440. * [Rework] Allow to set a different behaviour for actions from settings
  441. * [Rework] Include SSL flag into keepalive hash
  442. * [Rework] Make `rspamadm dmarc_report` default behaviour more sane
  443. * [Rework] Mempool: Use explicit alignment
  444. * [Rework] Rdns: Use faster and more compact hash table for DNS requests
  445. * [Rework] Rework SSL flag operations
  446. * [Rework] Take disabled flag into account
  447. * [Rework] Timeouts are now global per event and not reseted by IO activity
  448. * [Rework] Use xxh3 as a default hash and fix memory/alignment issues
  449. * [Rules] Fix old rules to stop global functions usage
  450. * [Rules] Fix symbol for DKIM temporary failure
  451. * [Rules] Remove ancient and inefficient rules
  452. * [Rules] Slightly reduce MULTIPLE_FROM score
  453. 3.1: 01 Nov 2021
  454. * [Feature] Add junk_threshold for autolearn
  455. * [Feature] Add neural test command
  456. * [Feature] Antivirus: Allow to set fake eicar patterns for testing AV engines
  457. * [Feature] Lua_cdb: Add cdb building interface
  458. * [Feature] Ratelimit: Add per bucket configurations
  459. * [Feature] S3: Allow to store structured data in messagepack
  460. * [Fix] Add concept of uncancellable events to prevent use-after-free
  461. * [Fix] Add temporary guard to prevent linked list exploitation
  462. * [Fix] Another rework of the ucl hashing
  463. * [Fix] Another try to fix references safety
  464. * [Fix] Another try to fix rspamd_text passing in the selectors
  465. * [Fix] Avoid copy for received structure as it has raw C pointers
  466. * [Fix] Avoid dangling reference
  467. * [Fix] Correctly check numeric URLs in URL DNS lists
  468. * [Fix] Delete the correct pointer type
  469. * [Fix] Dmarc: Always lowercase domain
  470. * [Fix] Fix compilation of the hyperscan databases with errors
  471. * [Fix] Fix hash table lookup
  472. * [Fix] Fix http message flag shift
  473. * [Fix] Fix parsing of the from_hostname when it is an IP address
  474. * [Fix] Fix parsing of the unquoted attributes in HTML
  475. * [Fix] Fix passing of rspamd_text in selectors pipelines
  476. * [Fix] Fix rubbish QP sequences decoding
  477. * [Fix] Fix some complicated case with the closing tags parsing
  478. * [Fix] Fix the case when l tag is too small
  479. * [Fix] Html: Fix the case where only bgcolor is explicitly set
  480. * [Fix] Libucl: Fix deletion from ucl objects
  481. * [Fix] Namespace and add metadata for OpenMetrics, fix interleaving
  482. * [Fix] Plug memory leak in http settings reload
  483. * [Fix] Preserve SPF top record in the mempool variable
  484. * [Fix] Remove aarch64 GC64 workaround
  485. * [Fix] Remove bogus G_LIKELY
  486. * [Fix] Spf: Do not parse non TXT DNS replies as TXT replies
  487. * [Fix] Try to use on_connect/on_disconnect callbacks to handle internal Redis failures
  488. * [Fix] buffer overflow in rspamc counters
  489. * [Fix] fix static building
  490. * [Fix] lua_scanners - message_min_words logic
  491. * [Fix] src/lua/lua_mimepart.c: fix null dereference
  492. * [Project] Add constant iterators
  493. * [Project] Add helper library to handle mime strings in a more safe matter
  494. * [Project] Add preliminary support of CDB bayes dump
  495. * [Project] Add trim operations
  496. * [Project] Allow mempool allocated mime strings
  497. * [Project] Cdb: Finish backend implementation
  498. * [Project] Cdb: Fix configuration load
  499. * [Project] Cdb: Use shared data between cdb statfiles
  500. * [Project] Cdb: continue statistics backend implementation
  501. * [Project] Finish received headers rework part
  502. * [Project] Move C++ specific declarations to C++ header
  503. * [Project] Rework received headers parsing to C++
  504. * [Project] Start using of the new received structure
  505. * [Project] Start work on cdb backend
  506. * [Rework] Further rework of the redis pool
  507. * [Rework] Redis_pool: fix issues found
  508. * [Rework] Rework learn and add classify condition
  509. * [Rework] Save invisible content to a separate buffer
  510. * [Rework] Start rewriting of the redis pool logic
  511. * [Rules] Improve zero font rule
  512. 3.0: 19 Aug 2021
  513. * [Conf] Align ARC scores with DKIM scores
  514. * [CritFix] Neural: Fix sorting application
  515. * [Feature] Add a simple dumper for bayes tokens
  516. * [Feature] Add lua_maps.fill_config_maps function
  517. * [Feature] Add preliminary exporter to AWS S3
  518. * [Feature] Add preliminary restore bayes support
  519. * [Feature] Add race condition protection against hs_helper restarts
  520. * [Feature] Add rspamd_utf8_strcmp utility
  521. * [Feature] Add zstd streaming API
  522. * [Feature] Allow to log severity level explicitly
  523. * [Feature] Allow to save and show attachment name when inserting AV scan results
  524. * [Feature] Allow to sort urls for Lua
  525. * [Feature] Allow to specify different timeouts/retransmits for fuzzy rules
  526. * [Feature] Aws_s3: Allow to compress data stored
  527. * [Feature] CMakeLists.txt: Change check and run-test to use rspamd-test-cxx * fixes #3807
  528. * [Feature] Dmarc_report: allow sending reports in batches
  529. * [Feature] Fuzzy_check: Allow to disable subject when making short text hash
  530. * [Feature] Lua_cryptobox: Add keyed ssl hash functions via HMAC
  531. * [Feature] Lua_task: Add get_urls_filtered method
  532. * [Feature] Make monitored checks less frequent
  533. * [Feature] Milter_headers: Add x-rspamd-pre-result header
  534. * [Feature] Neural: Allow to balance FP/FN for the network
  535. * [Feature] Ppopagate monitored errors from rbl module
  536. * [Feature] Pyzor calculate score dynamically Count - WL-Count of default_score in percent
  537. * [Feature] Rbl: Distinguish flattened and non-flattened selectors in RBL requests
  538. * [Feature] Re-add pyzor support
  539. * [Feature] Settings: add ip_map check and rework structure slightly
  540. * [Feature] Spamassassin: Allow to set the default priority for SA scores
  541. * [Feature] Strip smtp comments from message id
  542. * [Feature] add SYSTEM_ZSTD cmake option To use the system zstd instead on the bundled version
  543. * [Feature] external_relay plugin
  544. * [Feature] rspamadm clickhouse neural_train subcommand
  545. * [Fix] #3400 milter_headers: fix inverted logic for extended_headers_rcpt
  546. * [Fix] ASN: fix _FAIL symbol for when main symbol is disabled
  547. * [Fix] Add a special logic for text part with no text extraction
  548. * [Fix] Add diacritics flag for several eu languages
  549. * [Fix] Another FSM fix to accomodate possibility of multiple consequent ?
  550. * [Fix] Avoid curse of dynamic array referencing
  551. * [Fix] Avoid reinitialising neural settings
  552. * [Fix] Check remain before processing TXT records
  553. * [Fix] Enable error multiplier on http errors
  554. * [Fix] Finally rework parsing entities logic
  555. * [Fix] Fix '==' parsing in the content type attributes parser
  556. * [Fix] Fix IPv6 expansion for SPF macros
  557. * [Fix] Fix Mozilla Message-ID detection
  558. * [Fix] Fix an edge case in BITCOIN_ADDR rule
  559. * [Fix] Fix brain-damaged behaviour when http request has a custom Host header
  560. * [Fix] Fix check of limits in email address parsing
  561. * [Fix] Fix copy&paste error and rework
  562. * [Fix] Fix expressions logic for and/or and float values
  563. * [Fix] Fix fuzzy retransmits
  564. * [Fix] Fix http maps with no or invalid expires data
  565. * [Fix] Fix last quote character parsing in the content-type state machine
  566. * [Fix] Fix normalisation flags propagation
  567. * [Fix] Fix overflow when appending many broken tags
  568. * [Fix] Fix parsing of rfc2047 tokens with '?' inside
  569. * [Fix] Fix phishing flag set
  570. * [Fix] Fix rfc2047 embedded into rfc2231 pieces in special headers
  571. * [Fix] Fix round-robin rotation
  572. * [Fix] Fix searching for symbols
  573. * [Fix] Fix storing of the regexps inside variant
  574. * [Fix] Fix tokenization near exceptions
  575. * [Fix] Fix visibility calculations
  576. * [Fix] Html: Attach inline tags to the structure
  577. * [Fix] Html: Do not treat empty tags as block tags
  578. * [Fix] Ical: Do not extract urls from all flags using merely specific ones
  579. * [Fix] Initialise symcache even if it cannot be loaded properly
  580. * [Fix] Lua_fuzzy: Remove text parts check when checking image dimensions
  581. * [Fix] Lua_maps: Fix adjustments for the map type in the complex map definitions
  582. * [Fix] Lua_task: Fix deleted symbols in has_symbol/get_symbol
  583. * [Fix] Move metric and symcache link from validation to the init stage
  584. * [Fix] Oletools: Another try to fix table sorting
  585. * [Fix] One more default behaviour fix
  586. * [Fix] Phishing: Rework urls processing
  587. * [Fix] RBL: was missing some config schema
  588. * [Fix] Replies: Fix 'Reply-To' handling in task:get_reply_sender
  589. * [Fix] Rework metrics handling
  590. * [Fix] Save symcache on exit
  591. * [Fix] Selectors: Filter nil elements in lists
  592. * [Fix] Selectors: Properly fix implicit tostring for nils
  593. * [Fix] Try to fix some broken code in DMARC reporting plugin
  594. * [Fix] Urls: Fix processing of html urls when it comes to the flags
  595. * [Fix] Use proper buffer length
  596. * [Fix] Various visibility fixes
  597. * [Fix]: ASN: dns cb func should also return in case of an error
  598. * [Project] Add a simple css rule definition
  599. * [Project] Add css style skeleton
  600. * [Project] Add css syntax (adopted from ebnf)
  601. * [Project] Add css_selectors
  602. * [Project] Add doctest unit testing library
  603. * [Project] Add expected library
  604. * [Project] Add fmt library for simple string ops
  605. * [Project] Add fu2 library to better functions abstractions
  606. * [Project] Add hashing method
  607. * [Project] Add parsers skeleton
  608. * [Project] Add preliminary support of vcard parser
  609. * [Project] Add process exceptions for invisible text
  610. * [Project] Add some methods for css parser
  611. * [Project] Allow static libstdc++
  612. * [Project] Another whitespace hack
  613. * [Project] CSS: Various fixes in the declarations and values parsing
  614. * [Project] Cpp: Add robin-hood hash map library
  615. * [Project] Css: Add AST debug
  616. * [Project] Css: Add colors conversion functions
  617. * [Project] Css: Add dimensions handling
  618. * [Project] Css: Add display value support
  619. * [Project] Css: Add frozen library from https://github.com/serge-sans-paille/frozen/
  620. * [Project] Css: Add opacity support
  621. * [Project] Css: Add parser helpers to simplify debugging
  622. * [Project] Css: Add preliminary stylesheet support
  623. * [Project] Css: Add rules processing functions and tests
  624. * [Project] Css: Add simple selectors unit tests
  625. * [Project] Css: Add some c++ unit tests
  626. * [Project] Css: Add some debug methods
  627. * [Project] Css: Add some debug statements for the css parser
  628. * [Project] Css: Add some logical skeleton for declarations parser
  629. * [Project] Css: Add url/function tokens
  630. * [Project] Css: Allow at rules parsing
  631. * [Project] Css: Declarations parsing logic skeleton
  632. * [Project] Css: Enable conditional css parsing support from the HTML parser
  633. * [Project] Css: Finish generic lexer cases
  634. * [Project] Css: Fix HSL conversion
  635. * [Project] Css: Fix minus parsing
  636. * [Project] Css: Fix parser consumers nesting
  637. * [Project] Css: Fix parsing of the qualified rules
  638. * [Project] Css: Fix rules merging
  639. * [Project] Css: Further fixes to lexer
  640. * [Project] Css: Further steps to parse css colors + rework
  641. * [Project] Css: Further work on parser's methods
  642. * [Project] Css: Implement backlog of css tokens
  643. * [Project] Css: Implement numbers and ident parsers
  644. * [Project] Css: Implement simple css selectors lookup
  645. * [Project] Css: Implement styles merging
  646. * [Project] Css: Make debug strings json like to simplify tests
  647. * [Project] Css: Minor adjustments
  648. * [Project] Css: More meat to the lexer
  649. * [Project] Css: Move some of the tests to the doctest
  650. * [Project] Css: Projected a parser
  651. * [Project] Css: Properties attachment logic
  652. * [Project] Css: Remove ragel from build targets (maybe keep for reference)
  653. * [Project] Css: Rework css block structure
  654. * [Project] Css: Rework flags of css properties
  655. * [Project] Css: Rework tokens structure
  656. * [Project] Css: Several fixes + tests
  657. * [Project] Css: Simplify checks
  658. * [Project] Css: Simplify debug code
  659. * [Project] Css: Start css selectors parsing logic
  660. * [Project] Css: Start semantic parsing for rules
  661. * [Project] Css: Start stylesheet implementation
  662. * [Project] Css: Tidy up lambdas
  663. * [Project] Css: rework tokeniser
  664. * [Project] Dmarc: Add dmarc report tool (WIP)
  665. * [Project] Dmarc: Add munging configuration
  666. * [Project] Dmarc: Add preliminary munging logic
  667. * [Project] Dmarc: Fix header removal
  668. * [Project] Dmarc: Fix munging logic
  669. * [Project] Dmarc: Use full recipient address instead of a domain map
  670. * [Project] Dmarc: Use zlists for dmarc reports
  671. * [Project] Dmarc_report: Add message generation logic
  672. * [Project] Dmarc_report: Add preliminary sending support
  673. * [Project] Fix lua bindings
  674. * [Project] Fix xml/sgml tags processing
  675. * [Project] Handle new modification
  676. * [Project] Html/CSS: Add transform from a CSS rule to html block
  677. * [Project] Html/CSS: Link html and css styles
  678. * [Project] Html/CSS: Switch styles parsing to css parser
  679. * [Project] Html/Css: Fix some issues found
  680. * [Project] Html/Css: Implement visibility rules for a block
  681. * [Project] Html: Add more tests cases and fix some more corner issues
  682. * [Project] Html: Add rows display type support
  683. * [Project] Html: Allow decode entities function to normalise spaces + unit tests
  684. * [Project] Html: Another rework of the tags structure
  685. * [Project] Html: Another try to fix unbalanced cases
  686. * [Project] Html: Fix crossing spans
  687. * [Project] Html: Fix parent propagation
  688. * [Project] Html: Further rework of the html parsing stuff
  689. * [Project] Html: Implement logic for tags pairing
  690. * [Project] Html: Implement rawtext state machine
  691. * [Project] Html: Insert closing tags as well :(
  692. * [Project] Html: More fixes
  693. * [Project] Html: More fixes
  694. * [Project] Html: More spaces logic fixes
  695. * [Project] Html: One more attempt to write text content
  696. * [Project] Html: Replace \0 in html content
  697. * [Project] Html: Rework img/a tags handling
  698. * [Project] Html: Rework propagation method
  699. * [Project] Html: Rework tags placement
  700. * [Project] Html: Rework transparency logic
  701. * [Project] Html: Support 'hidden' attribute
  702. * [Project] Html: Try another approach to append tags content
  703. * [Project] Html: Try to deal with bad unknown tags properly
  704. * [Project] Lua_aws: Add canonicalisation utility
  705. * [Project] Lua_aws: Add function to produce AWS Authorisation header
  706. * [Project] Lua_aws: Implement request signing
  707. * [Project] Lua_mime: Add lua_mime.modify_headers routine
  708. * [Project] Lua_task: Add modify_header method
  709. * [Project] Lua_task: Allow to extract modified headers
  710. * [Project] Make unescape code public for unit testing
  711. * [Project] More fixes for closed tags
  712. * [Project] More fixes to calculations
  713. * [Project] Rework API for the modified headers
  714. * [Project] Rework html visibility rule
  715. * [Project] Skeleton of the css library
  716. * [Project] Start headers modification API structure
  717. * [Project] Start working on AWS Lua API
  718. * [Project] Use lua_mime to modify headers
  719. * [Project] Use modified headers on dkim signing
  720. * [Project] Use string_view to constexpr variant unpacking
  721. * [Rework] Add composites manager concept
  722. * [Rework] Add tags definitions
  723. * [Rework] Allow C code to be compiled with C++ compiler
  724. * [Rework] Clickhouse: Store url flags
  725. * [Rework] Composites: Rewrite the composites logic
  726. * [Rework] Composites: Start rework of the composites framework
  727. * [Rework] Dmarc: Move check policy function to the common utils
  728. * [Rework] Dmarc: Rework reports keys structure
  729. * [Rework] Further work to make html content private
  730. * [Rework] Html/CSS: Remove css C bindings as they are useless now
  731. * [Rework] Html/CSS: Rework Lua bindings
  732. * [Rework] Html/Css: Start rework of the html blocks
  733. * [Rework] Html: Add images processing logic
  734. * [Rework] Html: Add traverse function
  735. * [Rework] Html: Another steps to get rid of gnode
  736. * [Rework] Html: Convert to variant
  737. * [Rework] Html: Deal with the utf_content part
  738. * [Rework] Html: Final rework part for the html processing code
  739. * [Rework] Html: Fix Lua bindings
  740. * [Rework] Html: Forgot to add the internal include
  741. * [Rework] Html: Further html urls rework
  742. * [Rework] Html: Further rework of the tags content extraction
  743. * [Rework] Html: Make parameters as a vector again
  744. * [Rework] Html: Move blocks part
  745. * [Rework] Html: Move images processing stuff
  746. * [Rework] Html: Rework lua bindings
  747. * [Rework] Html: Start html text extraction rework
  748. * [Rework] Html: Start refactoring of the html tags handling
  749. * [Rework] Html: Start removing of GNode stuff
  750. * [Rework] Html: Start rework of the html content structure
  751. * [Rework] Lua_magic: Try to detect text parts with 8bit characters for non-utf8 encodings
  752. * [Rework] Move HTML url functions and rework them
  753. * [Rework] Move and adopt entities handling logic
  754. * [Rework] Move common and rarely used dmarc code to the library
  755. * [Rework] Move compression routines outside of rspamd_util library
  756. * [Rework] Move entities/tags handling
  757. * [Rework] Phishing: Split from redirectors usage
  758. * [Rework] Redesign html blocks propagation logic
  759. * [Rework] Remove tag name string
  760. * [Rework] Rename phished url to a linked url
  761. * [Rework] Reorganize dmarc plugin and remove unsupported reporting code
  762. * [Rework] Reputation: Use more flexible types in get/set functions
  763. * [Rework] Require proper C++ environment for Rspamd build
  764. * [Rework] Rework extended urls output
  765. * [Rework] Rework tags parsing machine
  766. * [Rework] Slightly improve old regexp API
  767. * [Rework] Start conversion of the redis pool code to c++
  768. * [Rework] Try to resolve failed upstreams more agressively
  769. * [Rework] Use C++ utf8 library with unit tests to trim whitespaces
  770. * [Rework] Use C++ version for unicode normalisation
  771. * [Rework] Use C++ version of the lua threads pool
  772. * [Rules] Add raw addresses to MULTIPLE_FROM options
  773. * [Rules] Another fix to HTTP_TO_HTTPS rule
  774. * [Rules] Do not trigger HTML_SHORT_LINK_IMG on external images
  775. * [Rules] Extend FORGED_X_MAILER
  776. * [Rules] Extend OLD_X_MAILER
  777. * [Rules] Fix CTYPE_MIXED_BOGUS for text attachments
  778. * [Rules] Fix FPs for CTYPE_MIXED_BOGUS
  779. * [Rules] Fix HTTP_TO_HTTPS rule
  780. * [Rules] Fix HTTP_TO_HTTPS rule
  781. * [Rules] Fix zerofont rule (partially)
  782. * [Rules] Micro-optimize X_PHP_EVAL
  783. * [Rules] Reduce default weight for R_MISSING_CHARSET
  784. 2.7: 08 Jan 2021
  785. * [Conf] Add R_DKIM_PERMFAIL to the metric
  786. * [CritFix] Dkim: Fix simple canonicalisation if multiple signatures are presented
  787. * [CritFix] Fix controller paths normalisation
  788. * [Feature] Add INVALID_DATE rule
  789. * [Feature] Add controller endpoint for training neural
  790. * [Feature] Add sanity checks for actions thresholds
  791. * [Feature] Add support of '==' and '!=' in Rspamd expressions
  792. * [Feature] Composites: Improve composite atoms parser
  793. * [Feature] Docker: use Debian slim variant
  794. * [Feature] Elastic: Add some missing fields
  795. * [Feature] Extract text from img alt attributes
  796. * [Feature] Improve charset detection logic
  797. * [Feature] Lua_clickhouse: Add optional row callback for large selections
  798. * [Feature] Lua_dns_resolver: Add idna_convert_utf8 method
  799. * [Feature] Lua_mime: Add ability to do multipattern replacement
  800. * [Feature] Lua_trie: Allow to report start of the match
  801. * [Feature] Multimap: support adding map values as extra options
  802. * [Feature] Neural: Move PCA learning to a subprocess
  803. * [Feature] RBL: support matching content/image URLs only
  804. * [Feature] RBL: support use of multiple selectors
  805. * [Feature] Reputation: Allow to specify ip masks
  806. * [Feature] Support SMIME signed messages container
  807. * [Feature] Support multiple conditions for symbols
  808. * [Feature] Support ping in milter mode
  809. * [Feature] Support rspamd_text in selector regexps
  810. * [Feature] Use own daemonization routine
  811. * [Feature] Vadesecure: Implement settings_outbound feature as recommended by Vade
  812. * [Feature] `rspamadm clickhouse` command
  813. * [Feature] allow hyperscan for aarch64
  814. * [Fix] Allow to set priorities between post init scripts
  815. * [Fix] Allow to use maps for strings that are not zero terminated
  816. * [Fix] Apply max_lua_urls limit for emails as well
  817. * [Fix] Arc: Fix CV check on signing
  818. * [Fix] Arc: Fix signing of the broken ARC chains
  819. * [Fix] Clickhouse: escape carriage return
  820. * [Fix] Composites: Allow partial match
  821. * [Fix] Deduct type of a table methods
  822. * [Fix] Do not load errored hyperscan database
  823. * [Fix] Do not process links in ignored html tags
  824. * [Fix] Fix ClamAV result for cached encrypted file (#3395)
  825. * [Fix] Fix canonicalisation when l= tag is presented
  826. * [Fix] Fix flag shift
  827. * [Fix] Fix handling of skip/skip_process http flags
  828. * [Fix] Fix html attachments checks
  829. * [Fix] Fix issue with pushing binary formats to Lua strings
  830. * [Fix] Fix logging for rspamadm
  831. * [Fix] Fix off-by-one with init check
  832. * [Fix] Fix parsing of escape characters in quoted pairs
  833. * [Fix] Fix pushing ucl strings with \0 inside
  834. * [Fix] Fix quoted-printable soft newlines bugged case
  835. * [Fix] Fix settings in case actions are set to null (#3415)
  836. * [Fix] Fix several issues with auth results producing
  837. * [Fix] Fix smtp comments exclusion
  838. * [Fix] Fix smtp date syntax definition
  839. * [Fix] Fix substring search in case if srchlen == inlen
  840. * [Fix] Fix text selectors
  841. * [Fix] Honour `systemd` setting when logging to console (#3514)
  842. * [Fix] Html: Add entities collisions prevention logic (e.g. for mathml entities)
  843. * [Fix] Lua_auth_results: Quote potentially bad values in AR header
  844. * [Fix] Multimap: Fix flags usage
  845. * [Fix] Multimap: Fix scoring for combined maps
  846. * [Fix] Plug GList * leak in redis pool
  847. * [Fix] RBL: allow for multiple matches of the same label if types are different
  848. * [Fix] Rely on libev checks for file maps
  849. * [Fix] Restore simple dkim canonicalisation mode
  850. * [Fix] Return MimeCharset as we work with emails...
  851. * [Fix] Spamassassin: Fix pcre_only flags
  852. * [Fix] Spamassassin: Preserve 'pcre_only' flag when dealing with regexp replacements
  853. * [Fix] Try to fix GError leak
  854. * [Fix] Try to fix a mess with settings loading by adding priorities
  855. * [Fix] Try to move setings initialisation to a later stage
  856. * [Fix] Use dup fd in milter handler to avoid races with the proxy
  857. * [Fix] Use message pointer to avoid obsolete data to be cached
  858. * [Project] Rbl: Migrate to `checks`
  859. * [Project] Rbl: Move config code outside of the plugin
  860. * [Project] Ressurect empty prefilters as connection filters
  861. * [Project] Support connection filters registration from Lua
  862. * [Rework] Add final cleanup logic
  863. * [Rework] Add preliminary support of hyperscan caching for re maps
  864. * [Rework] Add stale cache removal
  865. * [Rework] Clickhouse: Improve performance
  866. * [Rework] Distinguish between strict config test mode
  867. * [Rework] Furhter logging improvements
  868. * [Rework] Milter_headers: improve extended_headers_rcpt support
  869. * [Rework] Move parsers to a separate lua library
  870. * [Rework] Neural: Skip composite symbols
  871. * [Rework] Rbl: Rework defaults logic
  872. * [Rework] Some tunes to cache saving
  873. * [Rework] Track maps origins
  874. * [Rework] Use full crypto hash for regexp maps
  875. * [Rules] Remove broken rule
  876. 2.6: 30 Sep 2020
  877. * [Conf] Add missing symbols
  878. * [Conf] Add missing symbols
  879. * [Conf] Fix fat-fingers typo
  880. * [Conf] Fix wrong comment in options.inc
  881. * [Conf] Neural: Fix the default name for max_trains
  882. * [Conf] Register a known symbol
  883. * [Conf] Spf: Add R_SPF_PERMFAIL symbol
  884. * [CritFix] Arc: Fix ARC validation for chains of signatures
  885. * [CritFix] Distinguish socketpairs between different fuzzy workers
  886. * [CritFix] Fix IDNA dots parsing
  887. * [CritFix] Fix test assertion method
  888. * [CritFix] Fix usage of crypto_sign it should be crypto_sign_detached!
  889. * [Feature] Add BOUNCE rule
  890. * [Feature] Add controller plugins support and selectors plugin
  891. * [Feature] Add maps query method
  892. * [Feature] Add minimal delay to fuzzy storage
  893. * [Feature] Add multiple base32 alphabets for decoding
  894. * [Feature] Add preliminary support of BCH addresses
  895. * [Feature] Add query_specific endpoint
  896. * [Feature] Allow multiple base32 encodings in Lua API
  897. * [Feature] Allow to specify nonces manually
  898. * [Feature] Controller: Allow to pass query arguments to the lua webui plugins
  899. * [Feature] Fuzzy_check: Add gen_hashes command
  900. * [Feature] Fuzzy_check: Add weight_threshold option for fuzzy rules
  901. * [Feature] Implement address retry on connection failure
  902. * [Feature] Improve limits in pdf scanning
  903. * [Feature] Initial support of subscribe command in lua_redis
  904. * [Feature] Lua_cryptobox: Add secretbox API
  905. * [Feature] Lua_text: Add encoding methods
  906. * [Feature] Milter_headers: Allow to activate routines via users settings
  907. * [Feature] PDF: Add timeouts for expensive operations
  908. * [Feature] Preliminary maps addon for controller
  909. * [Feature] Split pdf processing object and output object to allow GC
  910. * [Feature] Support BLIS blas library
  911. * [Feature] Support input vectorisation by recvmmsg call
  912. * [Feature] Support multiple base32 alphabets
  913. * [Feature] add queueid, uid, messageid and specific symbols to selectors [Minor] use only selectors to fill vars in force_actions message
  914. * [Feature] allow variables in force_actions messages
  915. * [Feature] extend lua api
  916. * [Fix] #3249
  917. * [Fix] Allow to adjust neurons in the hidden layer
  918. * [Fix] Another try to fix email names parsing
  919. * [Fix] Arc: Allow to reuse authentication results when doing multi-stage signing
  920. * [Fix] Arc: Fix bug with arc chains verification where i>1
  921. * [Fix] Arc: Sort headers by their i= value
  922. * [Fix] Change neural plugin's loss function
  923. * [Fix] Deal with double eqsigns when decoding headers
  924. * [Fix] Default ANN names in clickhouse
  925. * [Fix] Disable reuseport for TCP sockets as it causes too many troubles
  926. * [Fix] Disable text detection heuristics for encrypted parts
  927. * [Fix] Distinguish DKIM keys by md5
  928. * [Fix] Distinguish type from flags in register_symbol
  929. * [Fix] Dmarc: Unbreak reporting after cf2ae3292ac93da8b6e0624b48a62828a51803c9
  930. * [Fix] Do not flag pre-result of virus scanners as least if action is reject
  931. * [Fix] Do not use GC64 workaround on 32bit platforms, omg
  932. * [Fix] Exclude damaged urls from html parser
  933. * [Fix] Fix FREEMAIL_REPLYTO_NEQ_FROM_DOM
  934. * [Fix] Fix FROM_NEQ_ENVFROM
  935. * [Fix] Fix FWD_GOOGLE rule (#1815)
  936. * [Fix] Fix adding of the empty archive file for gzip
  937. * [Fix] Fix aliases in forged recipients and limit number of iterations
  938. * [Fix] Fix authentication results insertion
  939. * [Fix] Fix calling of methods in selectors
  940. * [Fix] Fix clen length for hiredis...
  941. * [Fix] Fix endless loop if broken arc chain has been found
  942. * [Fix] Fix false - operation
  943. * [Fix] Fix get_urls table invocation
  944. * [Fix] Fix group based composites
  945. * [Fix] Fix headers passing in rspamd_proxy
  946. * [Fix] Fix incomplete utf8 sequences handling
  947. * [Fix] Fix lua_next invocation
  948. * [Fix] Fix lua_parse_symbol_type function logic
  949. * [Fix] Fix multiple listen configuration
  950. * [Fix] Fix occasional encryption of the cached data
  951. * [Fix] Fix parsing boundaries with spaces
  952. * [Fix] Fix passing of methods arguments
  953. * [Fix] Fix poor man allocator algorithm
  954. * [Fix] Fix regexp selector and add flattening
  955. * [Fix] Fix rfc base32 encode ordering (skip inverse bits)
  956. * [Fix] Fix rfc based base32 decoding
  957. * [Fix] Fix sockets leak in the client
  958. * [Fix] Fix storing of the original smtp from
  959. * [Fix] Fix types check and types usage in lua_cryptobox
  960. * [Fix] Fix unused results
  961. * [Fix] Fuzzy_check: Disable shingles for short texts (really)
  962. * [Fix] Ical: Fix identation grammar
  963. * [Fix] Improve part:is_attachment logic
  964. * [Fix] Mmap return value must be checked versus MAP_FAILED
  965. * [Fix] One more fix to skip images that are not urls
  966. * [Fix] Pdf: Support some weird objects with no newline before endobj
  967. * [Fix] Rbl: Fix ignore_defaults in conjunction with ignore_whitelists
  968. * [Fix] Restore support for `for` and `id` parts in received headers
  969. * [Fix] Segmentation fault in contrib/lua-lpeg/lpvm.c on ppc64el
  970. * [Fix] Skip spaces at the boundary end
  971. * [Fix] Slashing fix: fix captures matching API
  972. * [Fix] Spamassassin: Rework metas processing
  973. * [Fix] Store reference of upstream list in upstreams objects
  974. * [Fix] Understand utf8 in content-disposition parser
  975. * [Fix] Unify selectors digest functions
  976. * [Fix] Use `abs` value when checking composites
  977. * [Fix] Use strict IDNA for utf8 DNS names + add sanity checks for DNS names
  978. * [Fix] Use unsigned char and better support of utf8 in ragel parser
  979. * [Fix] add missing selector_cache declaration
  980. * [Project] Add `L` flag for regexps to save start of the match in Hyperscan
  981. * [Project] Add `lower` method to lua_text
  982. * [Project] Add a simple matrix Lua library
  983. * [Project] Add implicit bitcoincash prefix
  984. * [Project] Add linalg ffi library for prototyping
  985. * [Project] Add methods to append data to fuzzy requests
  986. * [Project] Add routine to call a generic lua function
  987. * [Project] Add ssyev method interface
  988. * [Project] Add tensors index method
  989. * [Project] Add text:sub method
  990. * [Project] Allow rspamd_text based selectors
  991. * [Project] Allow to specify re_conditions for regular expressions
  992. * [Project] Attach extensions to the binary fuzzy commands
  993. * [Project] Bitcoin: BTC cash addresses needs some checksum validation
  994. * [Project] Cleanup the redis script
  995. * [Project] Convert bitcoin rules to the new regexp conditions feature
  996. * [Project] Detect memrchr in systems that supports it
  997. * [Project] Do not listen sockets in the main process
  998. * [Project] Implement 'probabilistic' learn mode for ANN
  999. * [Project] Implement BTC polymod in C as it requires 64 bit ops
  1000. * [Project] Implement bitcoin cash validation in a proper way
  1001. * [Project] Implement extensions logic for fuzzy storage
  1002. * [Project] Implement symbols insertion in multiple results mode
  1003. * [Project] Lua_text: Add method memchr
  1004. * [Project] Neural: Add PCA loading logic
  1005. * [Project] Neural: Fix PCA based learning
  1006. * [Project] Neural: Fix matrix gemm
  1007. * [Project] Neural: Further PCA fixes
  1008. * [Project] Neural: Implement PCA in learning
  1009. * [Project] Neural: Implement PCA learning
  1010. * [Project] Neural: Implement PCA on ANN forward
  1011. * [Project] Neural: Implement PCA serialisation
  1012. * [Project] Neural: Start PCA implementation
  1013. * [Project] Neural: Use C version of scatter matrix producing
  1014. * [Project] Preliminary support of lua conditions for regexps
  1015. * [Project] Preliminary usage of the reuseport
  1016. * [Project] Process composites separately for each shadow result
  1017. * [Project] Remove old code
  1018. * [Project] Rework scan result functions to support shadow results
  1019. * [Project] Rework some more functions to work with shadow results
  1020. * [Project] Some more fixes
  1021. * [Project] Start results chain implementation
  1022. * [Project] Support fun iterators on rspamd_text objects
  1023. * [Project] Support multiply, minus and divide operators in expressions
  1024. * [Project] Tensor: Move scatter matrix calculation to C
  1025. * [Rework] Allow to specify exat metric result when adding a symbol
  1026. * [Rework] Change and improve openblas detection and usage
  1027. * [Rework] Close listen sockets in main after fork
  1028. * [Rework] Further rework of lua urls extraction API
  1029. * [Rework] Lua_cryptobox: Allow to store output of the hash function
  1030. * [Rework] Lua_task: Add more methods to deal with shadow results
  1031. * [Rework] Modernize logging for expressions
  1032. * [Rework] Remove empty prefilters feature - we are not prepared...
  1033. * [Rework] Remove old FindLua module, disable lua fallback when LuaJIT is enabled
  1034. * [Rework] Rework and refactor forged recipients plugin
  1035. * [Rework] Rework expressions processing
  1036. * [Rework] Rework fuzzy commands processing
  1037. * [Rework] Rework url flags handling API
  1038. * [Rework] Rework urls extraction
  1039. * [Rework] Split operations processing and add more debug logs
  1040. * [Rework] Update zstd to 1.4.5
  1041. * [Rework] Use google-ced instead of libicu chardet as the former sucks
  1042. * [Rework] add alias util:parse_addr for util:parse_mail_address
  1043. * [Rework] get rid of util:parse_addr duplicating the util:parse_mail_address, replace where used
  1044. * [Rules] Allow prefix for bitcoin cash addresses
  1045. * [Rules] More fixes for bitcoin cash addresses decoding
  1046. * [Rules] Refactor bleach32 addresses handling
  1047. 2.5: 01 Apr 2020
  1048. * [Conf] Mark Rspamd emailbl as ignore whitelist
  1049. * [Conf] RBL: Add missing emails = true option
  1050. * [Feature] Add support for scripts in fuzzy storage
  1051. * [Feature] Arc: Add whitelisted_signers_map option
  1052. * [Feature] Implement hosts file processing
  1053. * [Feature] Neural: Introduce classes bias that allows non-equal classes learning
  1054. * [Feature] Update libev to 4.33
  1055. * [Fix] Another brain damage html standard adoptions
  1056. * [Fix] Another fix for brain damaged obs-fws state
  1057. * [Fix] Fix flags that caused force_actions failure
  1058. * [Fix] Fix logging issue
  1059. * [Fix] Fix lua symbols scores registration when config does not define scores
  1060. * [Fix] Fix opaque maps logic
  1061. * [Fix] Fix parsing of the html tags with no spaces after attributes
  1062. * [Fix] Fix some corner cases in urls parsing, add limits
  1063. * [Fix] Fix tlds extraction if custom composition rules are used
  1064. * [Fix] Fix variables replacement in mempool
  1065. * [Fix] Improve base64 detection
  1066. * [Fix] Normalize dynamic scores in ANN correctly
  1067. * [Fix] Plug memory leak introduced by #3153
  1068. * [Fix] Stat_redis_backend: Fix memory leak and simplify learn path
  1069. * [Fix] Try hard to deal with ghost workers
  1070. * [Fix] metadata_exporter default formatter
  1071. * [Rework] Change the way to extract URLs when dealing with alternative parts
  1072. * [Rework] Fix various url extraction issues
  1073. * [Rework] Re cache: Load compiled hyperscan in the main process as well
  1074. * [Rework] Re cache: Load hyperscan early
  1075. * [Rework] Rework URL structure: adjust tld part
  1076. * [Rework] Rework URL structure: host field
  1077. * [Rework] Rework URL structure: more structure optimisations
  1078. * [Rework] Rework URL structure: user field
  1079. * [Rework] URL: Another update for urls extraction logic
  1080. * [Rework] Urls: Improve query urls handling
  1081. * [Rework] Urls: adopt html related stuff
  1082. * [Rework] Urls: more rework of the urls sets
  1083. * [Rework] Urls: process query urls in HTML urls correctly
  1084. * [Rework] Urls: rework urls hash structure
  1085. * [Rework] Urls: update lua libraries
  1086. * [Rework] Use multiple search tries for different url extraction types
  1087. 2.4: 26 Feb 2020
  1088. * [CritFix] Fix parsing of the content type attributes
  1089. * [Feature] Clickhouse: Add extra columns support
  1090. * [Feature] Rbl: Add url_compose_map option for RBL rules
  1091. * [Fix] 'R' flag is for all headers regexp
  1092. * [Fix] Allow to reset settings id from Lua (e.g. because of the priority)
  1093. * [Fix] Avoid collisions in mempool variables by changing fuzzy caching logic
  1094. * [Fix] Avoid strdup usage for symbols options
  1095. * [Fix] Do not trust stat(2) it lies
  1096. * [Fix] Filter all options for symbols to have sane characters
  1097. * [Fix] Fix all headers iteration
  1098. * [Fix] Fix allowed_settings for neural
  1099. * [Fix] Fix listen socket parsing
  1100. * [Fix] Fix maps expressions evaluation
  1101. * [Fix] Fix sentinel connections leak by using async connections
  1102. * [Fix] Fix smtp message on passthrough result
  1103. * [Fix] Fix tld compositon rules
  1104. * [Fix] Fuzzy_storage: Do not check for shingles if a direct hash has been found
  1105. * [Fix] Lua_mime: Do not perform QP encoding for 7bit parts
  1106. * [Fix] Neural: Distinguish missing symbols from symbols with low scores
  1107. * [Fix] Support listening on systemd sockets by name
  1108. * [Project] Add lua_urls_compose library
  1109. * [Project] Allow to set a custom log function to the logger
  1110. * [Project] CDB maps: Start making cdb a first class citizen
  1111. * [Project] Clickhouse: Add extra columns concept
  1112. * [Project] Fix urls composition rules, add unit tests
  1113. * [Project] Unify cdb maps
  1114. * [Rework] Logger infrastructure rework
  1115. * [Rework] Refactor libraries structure
  1116. * [Rework] Rework SSL caching
  1117. * [Rework] Update snowball stemmer to 2.0 and remove all crap aside of UTF8
  1118. 2.3: 04 Feb 2020
  1119. * [Conf] SPF is no longer a C module
  1120. * [Conf] Update spamtrap map path example
  1121. * [CritFix] Fix html entities decoding
  1122. * [CritFix] Fix re cache when mix of pcre and hyperscan is used
  1123. * [Feature] Allow milter code to deal with multiple headers
  1124. * [Feature] Antivirus: Add avast support
  1125. * [Feature] Dkim_signing: Allow to sign via milter_headers
  1126. * [Feature] Implement content hashes
  1127. * [Feature] Lua_text: Add regexp split iterator method
  1128. * [Feature] Lua_text: Implement flattening of the input tables
  1129. * [Feature] Send quit command to Redis
  1130. * [Feature] Speed up is_ascii function
  1131. * [Feature] Spf: Add external_relay option
  1132. * [Fix] Avoid double escaping
  1133. * [Fix] Fix O(N^2) algorithm
  1134. * [Fix] Fix arc seal validation
  1135. * [Fix] Fix base tag processing according to stupid HTML renderer behaviour
  1136. * [Fix] Fix dealing with `\0` in ucl strings and JSON
  1137. * [Fix] Fix gpg parts misdetection
  1138. * [Fix] Fix ignored symbols exporting
  1139. * [Fix] Fix processing of numeric url's
  1140. * [Fix] Fix processing of the closed tcp connections
  1141. * [Fix] Fix regexp type check for pcre2
  1142. * [Fix] Fix urls encode function
  1143. * [Fix] Fix urls shifting when doing decode to include separators
  1144. * [Fix] Fix white on white rule and add is_leaf flag
  1145. * [Fix] Further fixes in charset detection
  1146. * [Fix] Ignore diacritics in chartable module for specific languages
  1147. * [Fix] Limit size of symbols options by max_opts_len option
  1148. * [Fix] More fixes in html tag content calculations
  1149. * [Fix] Plug memory leak in fuzzy storage
  1150. * [Fix] Process high priority settings even if settings/id has been specified
  1151. * [Fix] Select a different upstream on last retransmit
  1152. * [Fix] Treat soft hyphen as zero width space
  1153. * [Fix] Try harder to watch the lifetime of the key_stat
  1154. * [Fix] Use ipv6-mapped-ipv4 addresses in radix trie
  1155. * [Project] Add logic to break execution when processing symbols*
  1156. * [Project] Add methods to set specific content for mime parts from Lua
  1157. * [Project] Lua_content: support PDF files
  1158. * [Project] Move dns_tool to using of the rspamd_spf from FFI module
  1159. * [Project] Preliminary SPF plugin in Lua
  1160. * [Project] Show debug stat for memory pool
  1161. * [Project] Some rework about specific data that is now tagged
  1162. * [Project] Start reworking of the mempool structure
  1163. * [Rework] Allow to add userdata as symbols options
  1164. * [Rework] Change mime part specifics handling
  1165. * [Rework] Move LRU SPF cache from spf plugin
  1166. * [Rework] Rework HTML tags content attachment
  1167. * [Rework] Rework options hash structure
  1168. * [Rework] Start lua_content library
  1169. * [Rework] Stop using of uthash for http headers
  1170. * [Rework] Use faster hashing approach for memory pools variables
  1171. * [Rules] Add PDF related rules
  1172. 2.2: 19 Nov 2019
  1173. * [Conf] Antivirus: Fix the default config
  1174. * [Feature] Add verdict library in lua
  1175. * [Feature] Allow exception when choosing upstream
  1176. * [Feature] Allow to disable symbols from the metric config
  1177. * [Feature] Allow to limit maps per specific worker
  1178. * [Feature] Always validate Rspamd protocol output
  1179. * [Feature] Antivirus: Add preliminary virustotal support
  1180. * [Feature] Clickhouse: Rework Clickhouse collection logic
  1181. * [Feature] Improve base64 usage
  1182. * [Feature] Shutdown timeout is now associated with task timeout
  1183. * [Fix] #3129 Multiple classifiers on redis working incorrectly
  1184. * [Fix] Allow real upstreams configuration
  1185. * [Fix] Another try to fix slow callbacks and timers
  1186. * [Fix] Check results of write message as SSL can bork them
  1187. * [Fix] Clickhouse: Avoid potential races in collection
  1188. * [Fix] Clickhouse: Fix periodic script
  1189. * [Fix] Fail DNS upstream on each retransmit attempt
  1190. * [Fix] Fix consistent hashing when upstreams are marked inactive
  1191. * [Fix] Fix issues found
  1192. * [Fix] Fix off-by-one in retries for the proxy
  1193. * [Fix] Fix termination
  1194. * [Fix] Fix upstreams exclusion logic
  1195. * [Fix] Fix utf8 validation for symbols options and empty strings
  1196. * [Fix] Oops, fix maps reload
  1197. * [Fix] Rbl: Allow utf8 lookups for IDN domains
  1198. * [Fix] Sigh, another try to fix brain-damaged openssl
  1199. * [Project] Add fast utf8 validation library
  1200. * [Project] Use own utf8 validation instead of glib
  1201. * [Rework] Another phase of finish actions rework
  1202. * [Rework] Further cmake system rework
  1203. * [Rework] Further isolation of the controller's functions
  1204. * [Rework] Make cmake structure more modular
  1205. * [Rework] Move cmake modules to a dedicated path
  1206. * [Rework] Replace controller functions by any scanner worker if needed
  1207. * [Rework] Rework final scripts logic
  1208. * [Rework] Rewrite rspamd_str_make_utf_valid function
  1209. 2.1: 28 Oct 2019
  1210. * [Conf] Update neural.conf
  1211. * [CritFix] Fix dkim verification for multiple headers listed
  1212. * [Feature] Add support of uudecode
  1213. * [Feature] Allow to explicitly set events backend
  1214. * [Feature] Implement configurable limits for SPF lookups
  1215. * [Feature] Lua_scanners: Use lua magic for inclusion/exclusion logic
  1216. * [Feature] Multimap: Do not check files in office archives
  1217. * [Feature] Neural: Add sampling when storing training vectors
  1218. * [Feature] SPF: Allow to disable AAAA checks in configuration
  1219. * [Feature] Spf: Add limits configuration support
  1220. * [Feature] Store etag in cached HTTP maps + better logging
  1221. * [Feature] Support segwit BTC addresses, fix LTC verification
  1222. * [Feature] Support uuencoding
  1223. * [Fix] Add configurable number of threads for OpenBLAS
  1224. * [Fix] Add workaround for ragel 7 in hyperscan related maps code
  1225. * [Fix] Another fix for numeric urls parsing
  1226. * [Fix] Correct EMA time calculations
  1227. * [Fix] Do not treat archives as text
  1228. * [Fix] Do not use strdup on data extracted from lua
  1229. * [Fix] Fix a failure calcuating URL reputation.
  1230. * [Fix] Fix crash due to constructors init order
  1231. * [Fix] Fix crash on parts with no cd
  1232. * [Fix] Fix empty prefilters that require mime structures
  1233. * [Fix] Fix event loop creation
  1234. * [Fix] Fix issues sending DMARC reports.
  1235. * [Fix] Fix misprint
  1236. * [Fix] Fix saving of the file maps
  1237. * [Fix] Fix size calculations when converting from utf16
  1238. * [Fix] Fix support of disable_monitoring in rbl
  1239. * [Fix] Fix use-after-free
  1240. * [Fix] Fix zip files check to relax requirements
  1241. * [Fix] Important hiredis fixes
  1242. * [Fix] Lot's of fixes in maps check logic
  1243. * [Fix] Lua_tcp: Deal with temporary fails on write
  1244. * [Fix] Lua_tcp: Make write errors fatal and rework error handlers
  1245. * [Fix] Meta: Filter some more values
  1246. * [Fix] Neural: Add protection agains infinities
  1247. * [Fix] Oops, fix math.huge invocation
  1248. * [Fix] Plug memory leak
  1249. * [Fix] Sigh, another email to string fix
  1250. * [Fix] Try to fix another ownership race in ssl connection
  1251. * [Fix] Uuencode: Fix parsing of corrupted uuencode
  1252. * [Fix] lua_scanners - razor rename need_check function
  1253. * [Rework] Require CMake 3.9 to work, remove manual lto crap
  1254. 2.0: 11 Oct 2019
  1255. * [Conf] Add BROKEN_HEADERS_MAILLIST composite
  1256. * [Conf] Add path to greylist-whitelist-domains.inc
  1257. * [Conf] Clarify documentation in the config files
  1258. * [Conf] Introduce maps.d directories
  1259. * [Conf] Log settings id by default
  1260. * [Conf] Make LEAKED_PASSWORD_SCAM a composite rule again
  1261. * [Conf] Move all surbl/emails rules to rbl
  1262. * [Conf] Register new Spamhaus codes
  1263. * [Conf] Remove configs for deleted modules
  1264. * [Conf] Remove surbl parts, fix hash_format attribute
  1265. * [Conf] Show autolearn sample
  1266. * [Conf] Slashing: Change default stats backend to Redis
  1267. * [Conf] Surbl: Utilise new `check_emails` option
  1268. * [Conf] Update header
  1269. * [Conf] Use multi-prefixes RBLs in the default config
  1270. * [CritFix] Deal with case-sensivity in Content-Disposition parser
  1271. * [CritFix] Eliminate old endpoint
  1272. * [CritFix] Fix case sensivity when parsing Content-Type
  1273. * [CritFix] Fix loading of DKIM public keys
  1274. * [CritFix] Fix procesing of urls
  1275. * [CritFix] Fix whitelisting when both spf and dkim are required to be valid
  1276. * [CritFix] Langdet: Fix language detection where no stop words found
  1277. * [Feature] Add description to the groups
  1278. * [Feature] Add limit for number of URLs in Lua
  1279. * [Feature] Add logging of groups to the log_format
  1280. * [Feature] Add lua_smtp library
  1281. * [Feature] Add maps cache and type refinement
  1282. * [Feature] Add p0f scanner
  1283. * [Feature] Adopt emails module to use lua_maps
  1284. * [Feature] Allow options matching in composites
  1285. * [Feature] Allow selectors in rbl module
  1286. * [Feature] Allow to output group results
  1287. * [Feature] Asn: Allow to use bgpdump when NET::MRT is broken
  1288. * [Feature] Calculate tokens occurrences distribution
  1289. * [Feature] Clickhouse: Add authenticated user and settings id columns
  1290. * [Feature] Clickhouse: Store groups data
  1291. * [Feature] Clickhouse: Utilise LowCardinality feature
  1292. * [Feature] Implement Redis prefixes registration logic
  1293. * [Feature] Implement settings id propagation between deps
  1294. * [Feature] Improve AV results caching
  1295. * [Feature] Improve autolearning
  1296. * [Feature] Improve logging locking logic (remove it actually)
  1297. * [Feature] Improve settings processing
  1298. * [Feature] Langdet: Limit number of stop words to be checked
  1299. * [Feature] Libucl: Allow to sort keys in ucl objects
  1300. * [Feature] Lua_config: Extend get symbols method
  1301. * [Feature] Lua_maps: Allow static maps for key-value pairs
  1302. * [Feature] Lua_mimepart: Add function filter_words
  1303. * [Feature] Lua_selectors: Add `words` selector
  1304. * [Feature] Lua_selectors: Add sort and uniq transform functions
  1305. * [Feature] Lua_selectors: Allow table arguments for selectors
  1306. * [Feature] Lua_tcp: Add preliminary support of SSL connections
  1307. * [Feature] Lua_trie: More flexible API
  1308. * [Feature] Lua_util: Add filter_specific_url function
  1309. * [Feature] Lua_util: table_digest can now recursively traverse tables
  1310. * [Feature] Maillist: Improve detection
  1311. * [Feature] Maps: Allow caching for complex maps
  1312. * [Feature] Monitored: Support random lookups
  1313. * [Feature] Multimap: Add combined maps prototype
  1314. * [Feature] Multimap: Add dependend maps via redis keys selectors
  1315. * [Feature] Multimap: Allow multiple email addresses matches
  1316. * [Feature] Multimap: Also check detected charset when do filename checks
  1317. * [Feature] Output number of messages processed to proctitle
  1318. * [Feature] Perform clean SSL shutdown
  1319. * [Feature] Performance: Do not use base64 SIMD version for bad inputs
  1320. * [Feature] RBL: Support bit results in replies
  1321. * [Feature] RBL: Support type specific prefixes
  1322. * [Feature] Ratelimit: Consider number of SMTP recipients
  1323. * [Feature] Rbl: Add ability to check urls
  1324. * [Feature] Rbl: Add resolve_ip based RBLs
  1325. * [Feature] Rbl: Make config checks much more strict
  1326. * [Feature] Rbl: Support per-rule whitelists
  1327. * [Feature] Rbl: Support process script
  1328. * [Feature] Rbl: Support replyto addresses
  1329. * [Feature] SURBL: Allow to check email domains
  1330. * [Feature] Selectors: Add `list` generator
  1331. * [Feature] Selectors: Add `specific_urls` extractor
  1332. * [Feature] Selectors: Add flatten function
  1333. * [Feature] Selectors: Support filter_map and apply_map functions
  1334. * [Feature] Store Clickhouse data outside of lua alloc
  1335. * [Feature] Support caching for encrypted files and macros
  1336. * [Feature] Support images when extracting urls
  1337. * [Feature] Support more hyperscan flags
  1338. * [Feature] Support protocol flags
  1339. * [Feature] URL: Apply stringprep to hostnames to filter garbage
  1340. * [Feature] Upstreams: Add lazy resolving logic to all upstreams
  1341. * [Feature] Upstreams: Set noresolve flag on numeric upstreams
  1342. * [Feature] Use `scores` in apply section
  1343. * [Feature] Use maps logic from lua_maps for multimap
  1344. * [Feature] Use random monitored in rbl module
  1345. * [Feature] lua_scanners - add Razor support
  1346. * [Fix] Add another safe-guard in urls processing
  1347. * [Fix] Add debug to ssl, fixed write hangs
  1348. * [Fix] Add missing groups to C callback symbols
  1349. * [Fix] Add more checks for ghosts symbols
  1350. * [Fix] Allow to enable or add new actions via settings
  1351. * [Fix] Allow to set 0 size for spf/dkim caches
  1352. * [Fix] Another bunch of fixes towards protocol mess
  1353. * [Fix] Another fix to deal with bad URLs
  1354. * [Fix] Arc: Another bunch of fixes for arc signing
  1355. * [Fix] Arc: More arc signing fixes
  1356. * [Fix] Avoid another overflow in fpconv
  1357. * [Fix] Clickhouse: Fix quoting
  1358. * [Fix] Clickhouse: Fix retention query quoting
  1359. * [Fix] Distinguish empty and non-empty prefilters
  1360. * [Fix] Distinguish remote and local addrs parsing
  1361. * [Fix] Do not assert if length of sig is bad, just fail verification
  1362. * [Fix] Do not assert if we have broken mime boundary in the headers
  1363. * [Fix] Do not call implicit strlen to avoid issues
  1364. * [Fix] Do not count images urls when checking url regexps for compatibility
  1365. * [Fix] Do not output rbl suffix in symbol option
  1366. * [Fix] Do not use config pool to avoid issues with double reload
  1367. * [Fix] Do not use ephemeral string
  1368. * [Fix] Do not use lightuserdata for traceback
  1369. * [Fix] Do not use priority in metric registration
  1370. * [Fix] Emails: Check email sanity before testing on BL
  1371. * [Fix] Emails: Fix misprint in key name
  1372. * [Fix] Escape utf in regexp to dodge ragel/hyperscan issue
  1373. * [Fix] Extend task_timeout to postfilters stage
  1374. * [Fix] Fix ARC signing after fixing another bug in it...
  1375. * [Fix] Fix AV scan logic
  1376. * [Fix] Fix DMARC_NA behaviour in case of no valid policies
  1377. * [Fix] Fix LRU hash iteration logic
  1378. * [Fix] Fix alignment mess
  1379. * [Fix] Fix configuring symbols without scores
  1380. * [Fix] Fix disabling of the actions
  1381. * [Fix] Fix dkim signing exceptions
  1382. * [Fix] Fix embedded images linking logic
  1383. * [Fix] Fix events leak
  1384. * [Fix] Fix eviction corner case
  1385. * [Fix] Fix fuzzy image score calculation #2962
  1386. * [Fix] Fix hang in fuzzy_learn when explicit rotation is set
  1387. * [Fix] Fix headers propagation logic
  1388. * [Fix] Fix hearbeats restart issue
  1389. * [Fix] Fix history reset
  1390. * [Fix] Fix log parameter
  1391. * [Fix] Fix lua_ip_equal logic
  1392. * [Fix] Fix more issues with nested messages + tests
  1393. * [Fix] Fix normalization of non-alphabet based languages
  1394. * [Fix] Fix offsets when parsing message/rfc822 in multipart
  1395. * [Fix] Fix options in rbl symbols
  1396. * [Fix] Fix out of bound access in lua logger
  1397. * [Fix] Fix out-of-bound read in qp decode
  1398. * [Fix] Fix parent CTE propagation
  1399. * [Fix] Fix parsing of the received headers with empty part
  1400. * [Fix] Fix pending checks for events
  1401. * [Fix] Fix printing of NULL pointer with fixed length
  1402. * [Fix] Fix race condition in watcher handler
  1403. * [Fix] Fix read-after-end in quoted printable decoding
  1404. * [Fix] Fix redis sentinel support
  1405. * [Fix] Fix registry leak in case of DNS errors
  1406. * [Fix] Fix reload logic
  1407. * [Fix] Fix sending of large entries via HTTPS
  1408. * [Fix] Fix settings reload
  1409. * [Fix] Fix some more corner cases for fpconv
  1410. * [Fix] Fix trie code when there are regexps and Hyperscan is absent
  1411. * [Fix] Further fixes to printing of the FP numbers
  1412. * [Fix] Fuzzy_check: Fix timeouts
  1413. * [Fix] Grrr, fix empty ip case
  1414. * [Fix] Html: Fix processing of fjlig entity
  1415. * [Fix] Lang_det: Try better to distinguish Chinese and Japanese
  1416. * [Fix] Lua_mime: Fix reversed extensions map
  1417. * [Fix] Lua_task: Fix message-less API
  1418. * [Fix] Lua_tcp: Report connection failures
  1419. * [Fix] Lua_tcp: Various fixes and debugging improvements
  1420. * [Fix] Metadata_exporter: This plugin is idempotent not a postfilter
  1421. * [Fix] More fixes to extract_specific_urls
  1422. * [Fix] More stages fixes
  1423. * [Fix] Neural: Another bunch of fixes
  1424. * [Fix] Neural: use version in ANN key profile
  1425. * [Fix] Postpone lua state destruction to allow lua dtors to be used
  1426. * [Fix] Prefer surbl/emails rule on rbl to preserve compatibility
  1427. * [Fix] RBL: Fix behaviour of emails_domainonly
  1428. * [Fix] Ratelimit: Fix dynamic score
  1429. * [Fix] Rbl: Fix emailbl functions
  1430. * [Fix] Really fix hyperscan workaround
  1431. * [Fix] Set sanity limits for pcre2
  1432. * [Fix] Settings: Fix settings check flags
  1433. * [Fix] Sort keys when getting data from Lua when filling rules
  1434. * [Fix] Statistics: Do not query Redis tokens when there are no learns
  1435. * [Fix] Stop IO event on write finished in http connection
  1436. * [Fix] Use heuristically detected text parts data
  1437. * [Fix] Various fixes to QP encoding algorithm
  1438. * [Fix] Various fixes to SSL state machine handler
  1439. * [Fix] Various fixes to asn module
  1440. * [Fix] Workaround for empty charset in rfc2231 encoding
  1441. * [Project] Switch from torch to KANN
  1442. * [Project] Add heartbeat events
  1443. * [Project] Add preliminary support of the Kaspersky Scan Engine
  1444. * [Project] Add preliminary version of maps expressions
  1445. * [Project] Add preprocessed settings to the config structure
  1446. * [Project] Add simple forward propagation function
  1447. * [Project] Add small helpers for migration simplifications
  1448. * [Project] Allow to replace body in milter
  1449. * [Project] Bundle libev
  1450. * [Project] First refactoring step libevent->libev
  1451. * [Project] Implement syntax highlighting for Lua
  1452. * [Project] Lua_magic: Adopt lua_magic stuff in mime_types
  1453. * [Project] Remove libfann, gd and other unsupported stuff
  1454. * [Project] Remove torch
  1455. * [Project] Rework upstreams
  1456. * [Rework] Allow execution of async events when hs compiles regexps
  1457. * [Rework] Bayes expiry: eliminate `default` expiration mode
  1458. * [Rework] Dkim: Remove signing code
  1459. * [Rework] Dkim_signing: Move sign condition to dkim_signing
  1460. * [Rework] Do not lowercase all data send to ClickHouse
  1461. * [Rework] Drop url tags
  1462. * [Rework] Eliminate lua_squeeze as it has shown no improvements
  1463. * [Rework] Eliminate virtual scan time as it is useless
  1464. * [Rework] Lua core: Use lightuserdata to index classes
  1465. * [Rework] Lua_util: Another rework for extract_specific_urls
  1466. * [Rework] Migrate from ip_score to reputation
  1467. * [Rework] Move mime modification functions to lua_mime library
  1468. * [Rework] Rbl: Major whitelisting logic rework
  1469. * [Rework] Remove deprecated plugins
  1470. * [Rework] Remove log helper worker
  1471. * [Rework] Remove rspamd.classifiers.lua
  1472. * [Rework] Rename filter.h to a more sane name
  1473. * [Rework] Reorganise selectors implementation
  1474. * [Rework] Replace linenoise with replxx
  1475. * [Rework] Reputation: Remove ipnet from the ip reputation
  1476. * [Rework] Reputation: Slashing - change name of symbols
  1477. * [Rework] Rework children operations
  1478. * [Rework] Rework config reload
  1479. * [Rework] Rework expression API
  1480. * [Rework] Rework image urls processing
  1481. * [Rework] Rework initialisation to reduce static leaks count
  1482. * [Rework] Rework request headers processing
  1483. * [Rework] Slashing: Change versioning schema - move to 2.0
  1484. * [Rework] Slashing: Turn off postfilters when passthrough result is set
  1485. * [Rework] Start moving to replxx
  1486. * [Rework] Stop support of signed HTTP maps to simplify code
  1487. * [Rework] Store ASN as UInt32 in ClickHouse
  1488. * [Rework] Url_redirector: Rewrite plugin
  1489. * [Rework] Use a dedicated library for autolearn
  1490. * [Rework] Use libsodium instead of hand crafted crypto implementations
  1491. * [Rework] Use opaque structure to store a table of mime headers
  1492. * [Rules] Add dedicated bitcoin addresses filter rule
  1493. * [Rules] Add more detection to LEAKED_PASSWORD_SCAM
  1494. * [Rules] Catch LTC addresses
  1495. * [Rules] Reduce weight of RSPAMD_EMAILBL
  1496. * [Rules] Rework LEAKED_PASSWORD_SCAM rule one more time
  1497. 1.9.4: 23 May 2019
  1498. * [CritFix] Fix case sensitivity when parsing Content-Type
  1499. * [Fix] Arc: Another bunch of fixes for arc signing
  1500. * [Fix] Arc: More arc signing fixes
  1501. * [Fix] Avoid another overflow in fpconv
  1502. * [Fix] Fix ARC signing after fixing another bug in it...
  1503. * [Fix] Fix dkim signing exceptions
  1504. * [Fix] Fix some more corner cases for fpconv
  1505. * [Fix] Further fixes to printing of the FP numbers
  1506. * [Fix] Ratelimit: Fix dynamic score
  1507. 1.9.3: 13 May 2019
  1508. * [Conf] Add IP_SCORE_FREEMAIL composite rule
  1509. * [Feature] Add cryptobox method to generate dkim keypairs
  1510. * [Feature] Add fast hashes to lua cryptobox hash
  1511. * [Feature] Add least passthrough results
  1512. * [Feature] Allow oversign if exists mode
  1513. * [Feature] Clickhouse: Modernise table initial schema
  1514. * [Feature] Implement IUF interface for specific fast hashes
  1515. * [Feature] Lua_util: Allow to obfuscate different fields
  1516. * [Feature] Tune memory management in Rspamd and Lua
  1517. * [Fix] Avoid buffer overflow when printing long lua strings
  1518. * [Fix] Change the default oversigning headers to a more sane list
  1519. * [Fix] Clickhouse: Do not store digest as it is not needed now
  1520. * [Fix] Clickhouse: Fix lots of storage issues
  1521. * [Fix] Clickhouse: Support custom actions
  1522. * [Fix] Deny URLs where hostname is bogus
  1523. * [Fix] Do not blacklist mail by SPF/DMARC for local/authed users
  1524. * [Fix] Fix DoS caused by bug in glib
  1525. * [Fix] Fix UCL parsing of the multiline strings
  1526. * [Fix] Fix buffer overflow when printing small floats
  1527. * [Fix] Fix init code for servers keypairs cache
  1528. * [Fix] Fix issue with urls with no tld (e.g. IP)
  1529. * [Fix] Fix memory in arc signing logic
  1530. * [Fix] Fix memory leak in language detector during reloads
  1531. * [Fix] Fix mixed case content type processing
  1532. * [Fix] Fix processing of the ip urls in file
  1533. * [Fix] Fix use after free
  1534. * [Fix] HTML: Fix `size` attribute processing
  1535. * [Fix] Hum, it seems that 99ff1c8 was not correct
  1536. * [Fix] Lua_task: Fix task:get_from method
  1537. * [Fix] Preserve fd when mapping file to scan
  1538. * [Fix] Re-use milter_headers settings when doing arc signing
  1539. * [Fix] Set dmarc force action as least action
  1540. * [Fix] Switch to GMT
  1541. * [Fix] allow PKCS7 signatures to be text/plain, too
  1542. * [Project] Add initial version of the vault management tool
  1543. * [Project] Add vault support for DKIM and ARC signing
  1544. * [Project] Implement keys rotation in the vault
  1545. * [Project] Improve dkim keys generation for vault
  1546. * [Project] Improve keys creation in rspamadm vault
  1547. * [Rework] Move lua_worker to a dedicated unit
  1548. * [WebUI] Add URL fragments (#) support
  1549. * [WebUI] Fix AJAX request URL
  1550. 1.9.2: 16 Apr 2019
  1551. * [Conf] Allow to load users plugins from plugins.d
  1552. * [Conf] oversign openpgp and autocrypt headers
  1553. * [Feature] Add SPF FFI library for Lua
  1554. * [Feature] Add more verbosity for SPF caching
  1555. * [Feature] Antivirus: Handle encrypted files specially
  1556. * [Feature] Clickhouse: Slashing - add new fields to CH
  1557. * [Feature] Dkim_signing: Add OpenDKIM like signing_table and key_table
  1558. * [Feature] Dkim_signing: Allow to use new options as maps
  1559. * [Feature] Import fpconv library
  1560. * [Feature] Lua_maps: Allow static regexp and glob maps
  1561. * [Feature] Parse ical files
  1562. * [Feature] Rspamadm: Add dns_tool utility
  1563. * [Feature] Store SPF records digests
  1564. * [Feature] Use fpconv girsu2 implementation for printing floats
  1565. * [Fix] Clickhouse: Use integer seconds when inserting rows
  1566. * [Fix] Fix floating point printing
  1567. * [Fix] Fix processing of embedded urls
  1568. * [Fix] Lua_clickhouse: Fix CH errors processing
  1569. * [Fix] Make spf digest stable
  1570. * [Fix] Properly detect encrypted files in zip archives
  1571. * [Fix] Slashing: Store times in GMT timezone in ClickHouse
  1572. * [Rules] Add additional conditions to perform BTC checks
  1573. * [Rules] Fix pay-to-hash addresses validation
  1574. 1.9.1: 5 Apr 2019
  1575. * [Conf] Add vendor groups for symbols
  1576. * [Feature] Add `rspamadm template` command
  1577. * [Feature] Allow to add messages from settings
  1578. * [Feature] Allow unconnected DNS servers operations
  1579. * [Feature] Check limits after being set, migrate to uint64
  1580. * [Feature] Greylist: Allow to disable greylisting depending on symbols
  1581. * [Feature] Improve lua binary strings output
  1582. * [Feature] Mime_types: Implement user configurable extension filters
  1583. * [Feature] Mime_types: When no extension defined, detect it by content
  1584. * [Feature] Preprocess config files using jinja templates
  1585. * [Feature] Replies: Filter replies sender to limit whitelisting to direct messages
  1586. * [Feature] Treat all tags with HREF as a potential hyperlinks
  1587. * [Feature] Validate BTC addresses in LEAKED_PASSWORD_SCAM
  1588. * [Fix] Add crash safety for HTTP async routines
  1589. * [Fix] Another fix for Redis sentinel
  1590. * [Fix] Clickhouse: Fix table schema upload
  1591. * [Fix] Core: Fix squeezed dependencies handling for virtual symbols
  1592. * [Fix] Finally fix default parameters parsing in actions section
  1593. * [Fix] Fix ES sending logic (restore from coroutines mess)
  1594. * [Fix] Fix finishing script for clickhouse collection
  1595. * [Fix] Fix priority for regexp symbols registriation
  1596. * [Fix] Fix various issues found by PVS Studio
  1597. * [Fix] Initialize lua debugging earlier
  1598. * [Fix] Neural: Fix training
  1599. * [Fix] Rework cached Redis logic to avoid sentinels breaking
  1600. * [Fix] SURBL: Fix regression in surbl module
  1601. * [Fix] Fix double signing in the milter
  1602. * [Project] Add support of HTTP proxy in requests
  1603. * [Rework] Change lua global variables registration
  1604. * [Rework] Rework HTML content urls extraction
  1605. * [Rework] Start rework of aliasing in Rspamd
  1606. * [WebUI] Combine Scan and Learning into one tab
  1607. * [WebUI] Fix symbol score input type
  1608. * [WebUI] Show grayed out pie
  1609. * [WebUI] Update Throughput summary values dynamically
  1610. 1.9.0: 12 Mar 2019
  1611. * [Conf] Add missing includes
  1612. * [Conf] Move to options
  1613. * [Conf] Rbl: DWL is actually special whitelist
  1614. * [Conf] Relax some uribl rules
  1615. * [Conf] Remove abuse.ch
  1616. * [CritFix] Html: Entities are not valid within tag params values
  1617. * [Feature] Add `rspamadm mime sign` tool
  1618. * [Feature] Add configgraph utility
  1619. * [Feature] Add dedicated ZW spaces detection for URLs
  1620. * [Feature] Add flag to url object when visible part is url_like
  1621. * [Feature] Add method task:lookup_words
  1622. * [Feature] Add pyzor support (by crosenberg)
  1623. * [Feature] Allow to add upstream watchers to Lua API
  1624. * [Feature] Allow to set rewrite subject pattern from settings
  1625. * [Feature] Better escaping of unicode
  1626. * [Feature] Clickhouse: Allow to store subject in Clickhouse
  1627. * [Feature] Core: Add QP encoding utility
  1628. * [Feature] Core: Add libmagic detection for all parts
  1629. * [Feature] Core: Add support for gzip archives
  1630. * [Feature] Core: Allow to construct scan tasks from raw data
  1631. * [Feature] Core: Detect charset in archived files
  1632. * [Feature] Core: Ignore and mark invisible spaces
  1633. * [Feature] Core: Normalise zero-width spaces in urls
  1634. * [Feature] Core: Process data urls for images
  1635. * [Feature] Core: Relax quoted-printable encoding
  1636. * [Feature] Core: Support RFC2231 encoding in headers
  1637. * [Feature] Core: Support telephone URLs
  1638. * [Feature] Core: allow to emit soft reject on task timeout
  1639. * [Feature] DCC: Add bulkness and reputation checks to dcc
  1640. * [Feature] Elastic: Modernize plugin
  1641. * [Feature] Export visible part of url to lua
  1642. * [Feature] Fuzzy_storage: add preliminary support of rate limits
  1643. * [Feature] HTML: Specially treat data urls in HTML
  1644. * [Feature] Implement event watchers for upstreams
  1645. * [Feature] Implement includes tracing in Lua
  1646. * [Feature] Improve dkim part in configwizard
  1647. * [Feature] Lua_scanners: Add VadeSecure engine support
  1648. * [Feature] Lua_task: Add flexible method to get specific urls
  1649. * [Feature] Mime_types: Add MIME_BAD_UNICODE rule
  1650. * [Feature] Mime_types: Use detected content type as well
  1651. * [Feature] Plugins: Add preliminary version of the external services plugin
  1652. * [Feature] Query sentinel on master errors
  1653. * [Feature] Regexp: Allow local lua functions in Rspamd regexp module
  1654. * [Feature] Rspamadm: Allow to append footers to plain messages
  1655. * [Feature] Rspamadm: Allow to rewrite headers in messages
  1656. * [Feature] Selectors: Add `ipmask` processor
  1657. * [Feature] Settings: Allow hostname match
  1658. * [Feature] Settings: Allow local when selecting settings
  1659. * [Feature] Settings: Allow multiple selectors
  1660. * [Feature] Settings: Allow to inverse conditions
  1661. * [Feature] Support User-Agent in HTTP requests
  1662. * [Feature] Support ed25519 dkim keys generation
  1663. * [Feature] Try to filter bad unicode types during normalisation
  1664. * [Feature] external_services - oletools (olefy) support
  1665. * [Feature] lua_scanners - icap protocol support
  1666. * [Feature] lua_scanners - spamassassin spam scanner
  1667. * [Fix] Add filter for absurdic URLs
  1668. * [Fix] Add some more cases for Received header
  1669. * [Fix] Allow to disable/enable composite symbols
  1670. * [Fix] Arc: Use a separated list of headers for arc signing
  1671. * [Fix] Archive: Final fixes for 7z archives
  1672. * [Fix] Clickhouse: Fix database usage
  1673. * [Fix] Controller: Make save stats timer persistent
  1674. * [Fix] Core: Detect encrypted rarv5 archives
  1675. * [Fix] Core: Don't detect language twice
  1676. * [Fix] Core: Fix address rotation bug
  1677. * [Fix] Core: Fix content calculations for message parts
  1678. * [Fix] Core: Fix emails comments parsing and other issues
  1679. * [Fix] Core: Fix etags support
  1680. * [Fix] Core: Fix headers folding on the last token
  1681. * [Fix] Core: Fix iso-8859-16 encoding
  1682. * [Fix] Core: Fix log_urls flag (and encrypted logging)
  1683. * [Fix] Core: Fix part length when dealing with boundaries
  1684. * [Fix] Core: Fix parts distance calculations
  1685. * [Fix] Core: Fix processing of NDNs of certain type
  1686. * [Fix] Core: Implement logic to find some bad characters in URLs
  1687. * [Fix] Core: treat nodes with ttl properly in lru cache
  1688. * [Fix] Fix Content-Type parsing
  1689. * [Fix] Fix HTTP headers signing case
  1690. * [Fix] Fix control interface
  1691. * [Fix] Fix deletion of the duplicate headers
  1692. * [Fix] Fix emails filtering in emails module
  1693. * [Fix] Fix greylisting log message and logic
  1694. * [Fix] Fix issues with storing of the accepted addr in rspamd control
  1695. * [Fix] Fix maps object update race condition
  1696. * [Fix] Fix memor leaks and whitespace processing
  1697. * [Fix] Fix processing of null bytes in headers
  1698. * [Fix] Fix rcpt_mime and from_mime in user settings
  1699. * [Fix] Fix rfc2047 decoding for CD headers
  1700. * [Fix] Fix rfc2231 for Content-Disposition header
  1701. * [Fix] Fix setting of the subject pattern in config
  1702. * [Fix] Greylist: fix records checking
  1703. * [Fix] HTML: Another HTML comments exception fix
  1704. * [Fix] HTML: Another entities decoding logic fix
  1705. * [Fix] HTML: Fix HTML comments with many dashes
  1706. * [Fix] HTML: Fix entities in HTML attributes
  1707. * [Fix] HTML: Fix some more SGML tags issues
  1708. * [Fix] Ignore whitespaces at the end of value in DKIM records
  1709. * [Fix] MID module: Fix DKIM domain matching
  1710. * [Fix] Milter_headers: Fix remove_upstream_spam_flag and modernise config
  1711. * [Fix] Mime_parser: Fix issue with parsing of the trailing garbadge
  1712. * [Fix] Mime_parser: Fix parsing of mime parts without closing boundary
  1713. * [Fix] Multimap: Fix operating with userdata
  1714. * [Fix] Process orphaned `symbols` section
  1715. * [Fix] Rdns: Fix multiple replies in fake replies
  1716. * [Fix] Rework groups scores definitions
  1717. * [Fix] Set proper element when reading data from Sentinel
  1718. * [Fix] Set rspamd user to initialise supplementary groups on reload
  1719. * [Fix] Settings: Fix selectors usage
  1720. * [Fix] Sort data received from Sentinel to avoid constant replacing
  1721. * [Fix] groups.conf - filename typo
  1722. * [Fix] lua_scanner - oletools typos, logging
  1723. * [Fix] lua_scanners - actions and symbol_fail
  1724. * [Fix] lua_scanners - fix luacheck
  1725. * [Fix] lua_scanners - kaspersky - response with fname
  1726. * [Fix] lua_scanners - savapi redis prefix
  1727. * [Fix] tests - antivirus - fprot symbols
  1728. * [Project] Add concept of flexible actions
  1729. * [Project] Add heuristical from parser to received parser
  1730. * [Project] Add new flags to clickhouse, redis and elastic exporters
  1731. * [Project] Attach new received parser
  1732. * [Project] Fallback to callbacks from coroutines
  1733. * [Project] Implement keep-alive support in lua_http
  1734. * [Project] Lua_udp: Implement fully functional client
  1735. * [Project] Plug keepalive knobs into http connection handling
  1736. * [Project] Rspamadm: Add `modify` tool
  1737. * [Rework] Convert rspamd-server to a shared library
  1738. * [Rework] Dcc: Rework DCC plugin
  1739. * [Rework] Enable explicit coroutines symbols
  1740. * [Rework] Rework telephone urls parsing logic
  1741. * [Rework] Rewrite RBL module
  1742. * [Rework] Settings: Rework settings check
  1743. * [Rework] Slashing: Distinguish lualibdir, pluginsdir and sharedir
  1744. * [Rework] Unify task_timeout
  1745. * [Rework] Use VEX instructions in assembly, relocate
  1746. * [WebUI] Notify user if uploaded data was not learned
  1747. * [WebUI] Remove redundant condition
  1748. 1.8.3: 03 Dec 2018
  1749. * [CritFix] Make flags mutually exclusive for mime parts
  1750. * [CritFix] Strictly deny unencoded bad utf8 sequences in headers
  1751. * [Feature] Add Kaspersky antivirus support
  1752. * [Feature] Add method to get dkim results
  1753. * [Feature] Add more words regexp classes
  1754. * [Feature] Allow to choose words format in `rspamadm mime`
  1755. * [Feature] Allow to get all types of words from Lua
  1756. * [Feature] Allow to get task flags in C expressions
  1757. * [Feature] Allow to require encryption when accepting connections
  1758. * [Feature] Ignore bogus whitespaces in the words
  1759. * [Feature] Implement more strict configuration tests
  1760. * [Feature] Improve SPF results in Authentication-Results
  1761. * [Feature] Support ClickHouse database
  1762. * [Fix] Add failsafety for utf8 regexps
  1763. * [Fix] Do not trigger BROKEN_CONTENT_TYPE on innocent text parts
  1764. * [Fix] Emit error if connection has been terminated with no stop pattern
  1765. * [Fix] Fix boundaries checks in embedded messages
  1766. * [Fix] Fix double free
  1767. * [Fix] Perform policy downgrade on sample out, add tests
  1768. * [Fix] Properly escape utf8 regexps in hyperscan mode
  1769. * [Fix] Selectors - attachments args condition
  1770. * [Fix] Some fixes for raw parts
  1771. * [Fix] Treat learning errors as non-fatal
  1772. * [Fix] Use tld when looking for DKIM domains
  1773. * [Project] Words unicode structure rework
  1774. * [Project] Add preliminary Redis Sentinel support
  1775. * [Project] Improve Authentication-Results header
  1776. * [Project] Rework DKIM checks results
  1777. * [Project] Use more generalised API to produce meta words
  1778. 1.8.2: 19 Nov 2018
  1779. * [Conf] Add DWL support in the default configuration
  1780. * [Conf] Disable rspamd_update by default (again)
  1781. * [Conf] Fix configuration sample for ratelimit
  1782. * [CritFix] Disable broken url tags by default
  1783. * [CritFix] Fix \0 processing when doing RSA sign
  1784. * [CritFix] Fix adding symbols to their primary groups
  1785. * [Feature] Add `rspamadm cookie` utility
  1786. * [Feature] Add specialised functions for generating encrypted cookies
  1787. * [Feature] Add support of cookies in replies module
  1788. * [Feature] Add support of words regexps
  1789. * [Feature] Allow to add 3rd party clang plugins
  1790. * [Feature] Allow to create lua regexps from glob or plain patterns
  1791. * [Feature] Allow to set custom limits for upstream lists
  1792. * [Feature] Detect orphaned parts and attach them to message
  1793. * [Feature] Filter tokens in bayes
  1794. * [Feature] Fold b= value when doing arc sealing
  1795. * [Feature] Ignore cookies in the future and too old in the past
  1796. * [Feature] Skip stop words in statistics
  1797. * [Feature] Store stop words and allow to query them
  1798. * [Feature] Support query arguments in controller's custom commands
  1799. * [Feature] Tune upstream limits in Rspamd proxy
  1800. * [Feature] Use different callback symbols for different uribls
  1801. * [Feature] Write DKIM selector in dkim allow/reject symbols
  1802. * [Fix] Add obs_fws state support to eoh state machine
  1803. * [Fix] Add sanity check when applying mime boundaries heuristic
  1804. * [Fix] Antivirus - virus names with 0 were recognized as tables
  1805. * [Fix] Disable headernames in bayes temporarily
  1806. * [Fix] Do not allow syntax errors in include files...
  1807. * [Fix] Do not allow to merge an object with an array (or vice versa)
  1808. * [Fix] Don't perform forged recipients check for missing recipients
  1809. * [Fix] Fix DKIM based RBLs
  1810. * [Fix] Fix actrie implementation (sync from upstream), fixed OOB read
  1811. * [Fix] Fix explicit methods call in selectors
  1812. * [Fix] Fix extraction of additional parts
  1813. * [Fix] Fix finalization for internal plugins
  1814. * [Fix] Fix override_defaults function
  1815. * [Fix] Fix squeezed symbols when using settings
  1816. * [Fix] Fix urls insertion in Clickhouse module
  1817. * [Fix] Furhter fixes to ratelimits logic
  1818. * [Fix] Ignore signatures when looking for boundaries
  1819. * [Fix] Properly set learned count
  1820. * [Fix] Really fix ratelimits configuration and work
  1821. * [Fix] Remove ambigious format flag from printf
  1822. * [Fix] Restore URLs exporting in ClickHouse plugin
  1823. * [Fix] Rework bayes calculations...
  1824. * [Fix] Switch from chi-square to naive for large Fisher value
  1825. * [Fix] Treat normal password as enable password if there is no enable password
  1826. * [Fix] Use proper syntax for making DNS requests
  1827. * [Fix] Various fixes in embedded plugins
  1828. * [Project] Change fuzzy check selection logic to lua_fuzzy library
  1829. * [Project] Rework async events and symbols
  1830. * [Project] Move all metatokens in Bayes to lua_stat from C
  1831. * [WebUI] Add history rows per page control
  1832. 1.8.1: 16 Oct 2018
  1833. * [CritFix] Fix options insertion
  1834. * [CritFix] Fix words decay one more time (affects long messages)
  1835. * [CritFix] Increase default words_decay
  1836. * [CritFix] Plug memory leak in redis pool
  1837. * [Feature] Add `check_violation` feature to DKIM/ARC signing
  1838. * [Feature] Add only unique elements to Clickhouse url arrays
  1839. * [Feature] Allow `g+:` and `g-:` composite atoms
  1840. * [Feature] Allow dkim domains check in surbl
  1841. * [Feature] Allow maps with HTTP auth
  1842. * [Feature] Allow to disable actions by users settings
  1843. * [Feature] Extend whitelisting options
  1844. * [Feature] Store url object in images
  1845. * [Feature] Use verdict instead of the plain action in plugins
  1846. * [Fix] Allow to call fstring append with NULL string
  1847. * [Fix] DCC - luacheck
  1848. * [Fix] Do not load torch on each rspamadm invocation
  1849. * [Fix] Fix boundaries detection and rework stop words algorithm
  1850. * [Fix] Fix dependencies for DNS_SIGNED symbol
  1851. * [Fix] Fix errors when dealing with dynamic rates/bursts in Ratelimit
  1852. * [Fix] Fix groups mess
  1853. * [Fix] Fix groups mess
  1854. * [Fix] Fix parsing address with comments
  1855. * [Fix] Fix resolving in DMARC reports
  1856. * [Fix] Fix various issues with parsing of the received headers
  1857. * [Fix] Fix watchers issue in lua_tcp when doing no resolving
  1858. * [Fix] Plug memory leak in language detector (affects reloads)
  1859. * [Fix] Remove one letter stop words
  1860. * [Fix] Slashing: backport chunk logic from libucl
  1861. * [Fix] Stop libevent from using cached time in rspamadm
  1862. * [Fix] Try to fix watchers chaining
  1863. * [Fix] Various fixes in redis sync interface
  1864. * [Fix] ip_score - respect check_authed and check_local settings from config
  1865. * [Project] Rework passthrough actions
  1866. * [Project] Clustering module
  1867. * [Rework] Always create result for a task
  1868. * [Rework] Completely rewrite DMARC checks logic
  1869. * [Rework] Rework and fix whitelist plugin
  1870. * [WebUI] Add symbols sorting buttons
  1871. * [WebUI] Change symbols order without updating history
  1872. * [WebUI] Colorize symbols
  1873. * [WebUI] Do not display password form when secure_ip is set
  1874. * [WebUI] Fix symbol description tooltips display
  1875. * [WebUI] History: add sorting by symbol score value
  1876. 1.8.0: 24 Sep 2018
  1877. * [Feature] Add arguments schemas to processors and extractors
  1878. * [Feature] Add functional selectors library
  1879. * [Feature] Add generic selector to reputation module
  1880. * [Feature] Add more ratelimits: by digest, by attachments data, by filenames
  1881. * [Feature] Add preliminary stop words detection support
  1882. * [Feature] Add pure Lua debugm function
  1883. * [Feature] Add schema validation for Redis settings
  1884. * [Feature] Add selectors combine function
  1885. * [Feature] Add some recursion protection to lua logger
  1886. * [Feature] Add support for Lua API tracing
  1887. * [Feature] Allow to apply schema to arguments
  1888. * [Feature] Allow to get dkim signing data directly from HTTP headers
  1889. * [Feature] Allow to reuse existing authentication results
  1890. * [Feature] Cache selectors results in re runtime
  1891. * [Feature] Implement new text tokenizer based on libicu
  1892. * [Feature] Integrate selectors framework to multimap
  1893. * [Feature] Relax FORGED_RECIPIENTS
  1894. * [Feature] Support (almost) all html entities
  1895. * [Feature] Support adding and deletion of recipients in the milter block
  1896. * [Feature] Support gathering HTTP body from fragments in lua_http
  1897. * [Feature] Support multi flag in regexp and glob maps
  1898. * [Feature] Support selectors in ratelimit module
  1899. * [Feature] Support selectors in settings
  1900. * [Feature] Use khash in HTML parser
  1901. * [Feature] Use pure Lua debugm function
  1902. * [Fix] Add fail-safety for destroying sessions
  1903. * [Fix] Allow to add result-less fake DNS records
  1904. * [Fix] Another try to fix race conditions on config unload
  1905. * [Fix] Call Lua callback on DNS timeouts
  1906. * [Fix] Deprecate task:inc_dns_req as it is redundant
  1907. * [Fix] Do not allow events deletions on cleanup
  1908. * [Fix] Do not try to process skipped messages
  1909. * [Fix] Fix HTTP requests with no body
  1910. * [Fix] Fix another cleanup race condition
  1911. * [Fix] Fix bug in processing of pcre regexps
  1912. * [Fix] Fix byte array allocation in the pool
  1913. * [Fix] Fix crashes on task cleanup
  1914. * [Fix] Fix dynamic buckets in ratelimits
  1915. * [Fix] Fix endless loop when waiting for Rspamd to stop
  1916. * [Fix] Fix lua_util.str_split in case of delimiters set
  1917. * [Fix] Fix more issues with watching of async events
  1918. * [Fix] Fix stop words detection and loading logic
  1919. * [Fix] Fix various corner cases for language detection
  1920. * [Fix] Fix watchers in lua_tcp
  1921. * [Fix] Fix words decay algorithm
  1922. * [Fix] Implement watchers replacement to handle nested calls
  1923. * [Fix] Save faked code into fake dns record
  1924. * [Fix] Show the proper frame when using lua_util.debugm
  1925. * [Fix] Use fake dns records in tests
  1926. * [Fix] Use unicode replacements for HTML entities
  1927. * [Fix] fixed "cannot find dependency on symbol 1" issue when using replaced symbols in spamassassin rules
  1928. * [Fix] partition_id is not available in old versions of CH
  1929. * [Project] Add implicit conversion logic to selectors
  1930. * [Project] Add initial support for selectors in regexps
  1931. * [Project] Add method concept
  1932. * [Project] Further changes in unicode operations
  1933. * [Project] Implement Clickhouse migrations
  1934. * [Project] Implement implicit conversions to userdata
  1935. * [Project] Implement insert method
  1936. * [Project] Implement selectors registration for regular expressions
  1937. * [Project] Implement selectors support in re_cache
  1938. * [Project] Improve language detector: cleanup unused files, categorize
  1939. * [Project] Migrate CH data to a fat table
  1940. * [Project] Rework selectors logic
  1941. * [Project] Start Clickhouse utilities library
  1942. * [Project] Start unicode rework
  1943. * [Project] coroutine threaded model for API calls: thread pool
  1944. * [Rework] Move phishtank to a DNS based service
  1945. * [Rework] Rework Clickhouse plugin to use the new API
  1946. * [Rework] Rework language detector
  1947. * [Rework] Rework utf content processing in text parts
  1948. * [WebUI] Add progress bar for AJAX requests
  1949. * [WebUI] Avoid errors table reinitialization
  1950. * [WebUI] Avoid history table reinitialization
  1951. * [WebUI] Avoid throughput summary table reinitialization
  1952. * [WebUI] Destroy summary table on disconnect
  1953. * [WebUI] Fix "auth" request URL
  1954. * [WebUI] Fix disabling and hiding controls on page reload
  1955. * [WebUI] Fix maps loading from neighbours
  1956. * [WebUI] Fix symbols sorting by score
  1957. * [WebUI] Fix tables destroying
  1958. * [WebUI] Fix throughput data consolidation
  1959. * [WebUI] Fix upload buttons disabling
  1960. * [WebUI] Notify user on module loading failure
  1961. * [WebUI] Update FooTable 3.1.4 -> 3.1.6
  1962. 1.7.9: 01 Aug 2018
  1963. * [CritFix] Fix caseless comparison of equal length strings
  1964. * [Feature] Add HTTP basic auth support to elastic and clickhouse plugins
  1965. * [Feature] Add SPF selector to reputation
  1966. * [Feature] Add support of the fallback backends for HTTP maps
  1967. * [Feature] Allow to print full mime structure when extracting mime data
  1968. * [Feature] Allow to split symbols in reputation plugin
  1969. * [Feature] Check attachments only on AV scanners in attachments_only mode
  1970. * [Feature] Disable all SSL checks if ssl_no_verify flag is set
  1971. * [Feature] Implement parsing of scoped IPv6 addresses
  1972. * [Feature] Improve `rspamc counters` output
  1973. * [Fix] Add sanity checks when expanding SPF macros
  1974. * [Fix] Allow to parse SA rules with no spaces around =~ (dirty hack)
  1975. * [Fix] Avoid one extra byte writing
  1976. * [Fix] Deal with direct hash table
  1977. * [Fix] Detect empty text part as text, not HTML
  1978. * [Fix] Do not reduce map watch timeout for mixed http/file maps
  1979. * [Fix] Fix HTML part detection heuristic
  1980. * [Fix] Fix double free in redirectors cleanup
  1981. * [Fix] Fix legacy history handling in the controller
  1982. * [Fix] Fix messages insertion
  1983. * [Fix] Fix sending string method
  1984. * [Fix] Fix statconver command line arguments
  1985. * [Fix] Fixed argument checking for being null
  1986. * [Fix] Fixed issues reported by luacheck
  1987. * [Fix] Freeze updates queue when do actual storage update
  1988. * [Fix] HTTP map hash is per-backend and not per-map
  1989. * [Fix] Plug memory leak in fuzzy updates
  1990. * [Fix] Prefer 'MTA-Name' when producing authentication results
  1991. * [Fix] Replace bad unicode sequences instead of stopping on them
  1992. * [Fix] Set classifier version on learning
  1993. * [Project] Reworked ratelimits
  1994. * [Project] Apply topological sorting for symbols in Rspamd
  1995. * [Project] Remove global contexts from C modules
  1996. * [Project] Move performance critical hash tables to khash
  1997. * [WebUI] Avoid unused indexes
  1998. * [WebUI] Do not execute `on_success` callback
  1999. * [WebUI] Fix history reset for "All SERVERS" (#2346)
  2000. * [WebUI] Fix query URL for selected server
  2001. * [WebUI] Fix symbols display in legacy history,
  2002. * [WebUI] Hide symbols order selector for legacy history
  2003. * [WebUI] Refactor query functions into one
  2004. * [WebUI] Remove previously-attached event handlers
  2005. * [WebUI] Save symbols to the selected server
  2006. * [WebUI] Unify arguments of query functions
  2007. * [WebUI] Use common query functions to get graph data
  2008. * [WebUI] Use common query functions to save symbols
  2009. 1.7.8: 12 Jul 2018
  2010. * [Feature] Add more extended statistics about fuzzy updates
  2011. * [Feature] Add more non-conformant Received headers support
  2012. * [Feature] Add preliminary function to get fuzzy hashes from text in Lua
  2013. * [Feature] Allow to configure AV module rejection message
  2014. * [Feature] Implement fuzzy hashes extraction in mime tool
  2015. * [Feature] Improve WHITE_ON_WHITE rule
  2016. * [Feature] Improve integer -> string conversion
  2017. * [Feature] Reuse maps in multimap module more aggressively
  2018. * [Fix] Avoid race condition in skip map as pool lifetime is not enough
  2019. * [Fix] Eliminate all specific C plugins pools
  2020. * [Fix] Fix DKIM check rule if DNS is unavailable
  2021. * [Fix] Fix build where ucontext is defined in ucontext.h
  2022. * [Fix] Fix crash in base url handling
  2023. * [Fix] Fix descriptors leak in sqlite3 locking code
  2024. * [Fix] Fix messages quarantine
  2025. * [Fix] Fix padded numbers printing
  2026. * [Fix] Fix race condition on maps reinit
  2027. * [Fix] Fix regexp functions when no data is passed
  2028. * [Fix] Fix specific urls extraction
  2029. * [Fix] Fix styles propagation
  2030. * [Fix] Improve resetting of the limit buckets
  2031. * [Fix] Initialize sqlite3 properly
  2032. * [Fix] Work with broken resolvers in resolv.conf
  2033. * [Project] Implement HTTP maps caching
  2034. * [Project] Refresh fuzzy hashes when matched
  2035. * [Project] Add logic to deduplicate fuzzy updates queue
  2036. * [WebUI] Add missed declarations
  2037. * [WebUI] Avoid using "undefined" property
  2038. * [WebUI] Do not accept passwords containing control characters
  2039. * [WebUI] Do not redeclare variables
  2040. * [WebUI] Enable strict mode,
  2041. * [WebUI] Fix variable assignment
  2042. * [WebUI] Initialize variables at declaration
  2043. * [WebUI] Remove duplicated path from RequireJS config
  2044. * [WebUI] Remove unused block
  2045. * [WebUI] Remove unused variable
  2046. * [WebUI] Remove unused variables
  2047. * [WebUI] Use self-explanatory notation
  2048. * [WebUI] Use type-safe equality operators
  2049. 1.7.7: 02 Jul 2018
  2050. * [CritFix] Check NM part of pubkey to match it with rotating keypairs
  2051. * [CritFix] Do not overwrite PID of the main process
  2052. * [CritFix] Fix maps after reload
  2053. * [CritFix] Fix maps race conditions on reload
  2054. * [CritFix] Fix shmem leak in encrypting proxy mode
  2055. * [Feature] Add a concept of ignored symbols to avoid race conditions
  2056. * [Feature] Add ability to print bayes tokens in rspamadm mime
  2057. * [Feature] Add method to get statistical tokens in Lua API
  2058. * [Feature] Add preliminary mime stat command
  2059. * [Feature] Add rspamadm mime tool
  2060. * [Feature] Add urls extraction tool
  2061. * [Feature] Address ZeroFont exploit
  2062. * [Feature] Allow rspamadm mime to process multiple files
  2063. * [Feature] Allow to extract words in `rspamadm mime`
  2064. * [Feature] Allow to print mime part data
  2065. * [Feature] Allow to show HTML structure on extraction
  2066. * [Feature] Distinguish IP failures from connection failures
  2067. * [Feature] Improve output for mime command
  2068. * [Feature] Improve styles propagation
  2069. * [Feature] Main process crash will now cleanup all children
  2070. * [Feature] Preload file and static maps in main process
  2071. * [Feature] Print stack trace on crash
  2072. * [Feature] Process font size in HTML parser
  2073. * [Feature] Propagate content length of invisible tags
  2074. * [Feature] Read ordinary file maps in chunks to be more safe on rewrites
  2075. * [Feature] Support base tag in HTML
  2076. * [Feature] Support more size suffixes when parsing HTML styles
  2077. * [Feature] Support opacity style
  2078. * [Fix] Another fix for nested composites
  2079. * [Fix] Fill nm id in keypairs cache code
  2080. * [Fix] Fix colors alpha channel handling
  2081. * [Fix] Fix destruction logic
  2082. * [Fix] Fix double free
  2083. * [Fix] Fix maps preload logic
  2084. * [Fix] Fix nested composites process
  2085. * [Fix] Fix proxying of Exim connections
  2086. * [Fix] Fix reload crash
  2087. * [Fix] Fix rspamadm -l command
  2088. * [Fix] Update ed25519 signing schema
  2089. * [WebUI] Stop using "const" declaration
  2090. * [WebUI] Update RequireJS to 2.3.5
  2091. 1.7.6: 15 Jun 2018
  2092. * [CritFix] Fix multiple neural networks support
  2093. * [Feature] Add decryption function to keypair command
  2094. * [Feature] Add gzip compression for HTTP requests in elastic module
  2095. * [Feature] Add gzip methods to lua util
  2096. * [Feature] Add maps based on Top Level Domains
  2097. * [Feature] Add pubkey checks for dkim_signing
  2098. * [Feature] Add support of fake DNS records
  2099. * [Feature] Add tool to encrypt files
  2100. * [Feature] Allow to add symbols using settings directly
  2101. * [Feature] Allow to match private and public keys for DKIM signatures
  2102. * [Feature] Allow to set task flags via settings
  2103. * [Feature] Allow to specify fake DNS address from the config
  2104. * [Feature] Implement signatures verification using rspamadm keypair
  2105. * [Feature] Implement signing using `rspamadm keypair`
  2106. * [Feature] Improve error reporting for DKIM key access issues
  2107. * [Feature] Provide $HOSTNAME variable in UCL
  2108. * [Feature] Rework levenshtein distance computation
  2109. * [Feature] Split message parsing and processing
  2110. * [Feature] Support ED25519 DKIM signatures
  2111. * [Feature] Support encrypted configs in UCL
  2112. * [Feature] Suppress duplicate warning on very large radix tries
  2113. * [Feature] Use OSB to combine header names
  2114. * [Fix] Cleanup maps data on shutdown
  2115. * [Fix] Fix '~' behaviour in composites
  2116. * [Fix] Fix HTTP maps updates
  2117. * [Fix] Fix NIST signatures
  2118. * [Fix] Fix RFC822 comments when processing a mime address
  2119. * [Fix] Fix double free
  2120. * [Fix] Fix dynamic settings application
  2121. * [Fix] Fix for CommuniGate Pro maillist
  2122. * [Fix] Fix keypair creation method to actually create keypair...
  2123. * [Fix] Fix matching patterns with no paths
  2124. * [Fix] Fix memory leak in parsing comments
  2125. * [Fix] Fix parsing of urls with numeric password
  2126. * [Fix] Fix plugins intialisation in configwizard
  2127. * [Fix] Fix potential crash on reload
  2128. * [Fix] Fix potential race condition for a finished HTTP connections
  2129. * [Fix] Fix race-condition leak on processes reload
  2130. * [Fix] Fix signing in openssl mode
  2131. * [Fix] Free language detector structures
  2132. * [Fix] Relax alignment requirements
  2133. * [Fix] Send DMARC reports compressed
  2134. * [Fix] Try to fix leak in dmarc module
  2135. * [Fix] Try to plug memory leak in metric exporter
  2136. * [Project] Convert rspamadm subcommands to Lua
  2137. * [WebUI] Display smtp sender/recipient in history
  2138. * [WebUI] Fix elements disabling in "Symbols" tab
  2139. * [WebUI] Limit recipients list in history column to 3
  2140. * [WebUI] Match envelope and mime addresses following in arbitrary order
  2141. * [WebUI] Update column header
  2142. * [WebUI] Wrap addresses in history
  2143. 1.7.5: 18 May 2018
  2144. * [Conf] Add MSBL proposed return codes
  2145. * [Conf] Add additional groups for policies
  2146. * [CritFix] Do not use volatile Lua strings as UCL keys
  2147. * [Feature] Add ability to add fuzzy hashes to headers
  2148. * [Feature] Add function to extract most meaningful urls
  2149. * [Feature] Add rule to block mixed text and encrypted parts
  2150. * [Feature] Allow multiple groups for symbols
  2151. * [Feature] Allow to disable lua squeezing logic
  2152. * [Feature] Allow to get multipart children in Lua
  2153. * [Feature] Allow to insert multiple headers from milter headers
  2154. * [Feature] Allow to print scores in subject and further extensions
  2155. * [Feature] Be more error-prone in squeezed rules
  2156. * [Feature] Support multiple return codes in emails module
  2157. * [Feature] Use EMA for calculating averages
  2158. * [Feature] Use common jit cache for all regexps
  2159. * [Feature] support for CommuniGate Pro self-generated messages
  2160. * [Fix] Allow to have multiple values for headers as arrays
  2161. * [Fix] Do not open sockets for disabled workers
  2162. * [Fix] Fix AuthservId
  2163. * [Fix] Fix base64 folding in Lua API
  2164. * [Fix] Fix build on non-x86 platforms
  2165. * [Fix] Fix cached maps logic
  2166. * [Fix] Fix compatibility with old maps query logic
  2167. * [Fix] Fix crash if skip_map is used
  2168. * [Fix] Fix importing static maps from UCL
  2169. * [Fix] Fix parsing of unix sockets
  2170. * [Fix] Fix raw_mime regexp on HTML part with no text content
  2171. * [Fix] Fix tables logging
  2172. * [Fix] Fix vertical tab handling in libucl
  2173. * [Fix] Try to fix frequency counters
  2174. * [Fix] Use better sharding for ip_score
  2175. * [Fix] Use multiple results from SURBL DNS reply
  2176. * [Fix] When doing AV scan select a different server for retransmit
  2177. 1.7.4: 01 May 2018
  2178. * [Conf] Major stock config updates:
  2179. - Workers are now specified in a new format worker "type" { ... }
  2180. - Enable fuzzy worker to simplify local fuzzy storages configuration
  2181. - Bind all workers to localhost by default to avoid security flaws
  2182. * [Conf] Make more sane fuzzy_check default settings
  2183. * [CritFix] Fix ucl escape for bad symbols
  2184. * [Feature] Add failure symbol for AV module
  2185. * [Feature] Add lazy expiration mode for new classifier schema
  2186. * [Feature] Add preliminary version of maps stats plugin
  2187. * [Feature] Allow to block fuzzy requests from specific networks
  2188. * [Feature] Allow to change `expire` of live statistics
  2189. * [Feature] Distinguish AV failure from clean result
  2190. * [Feature] Further improvements of language detector by using khash
  2191. * [Feature] Further optimization of the lang_detection
  2192. * [Feature] Implement cluster-aware bayes expiry
  2193. * [Feature] Implement exclude patterns in rspamc
  2194. * [Feature] Implement glob maps in addition to regexp maps
  2195. * [Feature] Implement map statistics function for lua API
  2196. * [Feature] Implement stop symbols for Clickhouse collection
  2197. * [Feature] Support recipients separated by commas
  2198. * [Feature] Try harder to upload scripts to the Redis server
  2199. * [Feature] Upgrade t1ha distribution
  2200. * [Feature] use_domain_sign_inbound
  2201. * [Feature] Use scores from maps if `symbols_set` is not defined
  2202. * [Fix] Add resolving version of radix map helper
  2203. * [Fix] Check URL before adding implicit prefix
  2204. * [Fix] Do not check pid/state when using PRNG
  2205. * [Fix] Fix CentOS logrotate script for systemd
  2206. * [Fix] Fix slash + dot in urls
  2207. * [Fix] Fix systemd version of the logrotate script
  2208. * [Fix] Propagate key when import implicit array from Lua
  2209. * [Fix] Strip spaces from map keys and values
  2210. * [Fix] Try to fix a specific case when processing milter protocol
  2211. * [Fix] Try to fix crash when a tcp connection cannot be set
  2212. * [Fix] Typo use_domain_local --> use_domain_sign_local
  2213. * [Fix] Various fixes to once_received module
  2214. * [Project] Store hits counters for map elements
  2215. 1.7.3: 10 Apr 2018
  2216. * [CritFix] Plug bad memory leak in protocol reply
  2217. * [Feature] Add avx2 codec for base64
  2218. * [Feature] Add method to receive all URL flags from Lua API
  2219. * [Feature] Allow to fold headers on stop characters
  2220. * [Feature] Allow to set lua_cpath from options
  2221. * [Feature] Allow to specify custom rejection message in milter
  2222. * [Feature] Deal with unnormalised Unicode obfuscation
  2223. * [Feature] Do not detect language twice for relative parts
  2224. * [Feature] Implement oversigning feature
  2225. * [Feature] Implement silent logging level to minimize noise in logs
  2226. * [Feature] Improve URL_IN_SUBJECT rule
  2227. * [Feature] Use hashing to reduce redis attack surface
  2228. * [Fix] Add oversigning for the most important headers
  2229. * [Fix] add 'rewrite subject' to History dropdown
  2230. * [Fix] Another fix in folding algorithm
  2231. * [Fix] Do not call multimap addr for parts of addr if filter is presented
  2232. * [Fix] Do not clean hostname on generic reset
  2233. * [Fix] Do not create pid file in no-fork mode
  2234. * [Fix] Fix fold_after case to preserve multiple spaces
  2235. * [Fix] Fix folding and folding tests
  2236. * [Fix] Fix hostname usage in milter mode
  2237. * [Fix] Fix lua RSA verify and its tests
  2238. * [Fix] Fix metadata exporter send_mail backend (#2124)
  2239. * [Fix] Fix processing of '\v' in libucl
  2240. * [Fix] Fix shemaless URLs detection
  2241. * [Fix] Fix support of multiple headers in sign_header
  2242. * [Fix] Fix usage of util.parse_mail_address
  2243. * [Fix] Fix weights of dynamic squeezed rules
  2244. * [Fix] Leak from bucket before checking the burst
  2245. * [Fix] Stop using own localtime as DST could be messy in many cases
  2246. * [Fix] Treat unnormalised URLs as obscured
  2247. * [Rework] Restore leaky bucket model in ratelimit plugin
  2248. * [WebUI] Add messages total to throughput summary
  2249. * [WebUI] Add symbols order selector to history
  2250. * [WebUI] Config: Load list on demand
  2251. * [WebUI] Fix modalBody for maps that appear more than once
  2252. * [WebUI] History: Fix Tooltips on paging, filtering and sorting
  2253. * [WebUI] Remove a previously-attached event handler
  2254. * [WebUI] Update D3 to v5.0.0 and jQuery to v3.3.1
  2255. 1.7.2: 23 Mar 2018
  2256. * [Feature] Store emails in Clickhouse
  2257. * [Feature] Support single quotes in config
  2258. * [Feature] Use templates when publishing CH schema
  2259. * [Feature] Improve Docker image
  2260. * [Fix] Add rounding when printing a lot of FP variables
  2261. * [Fix] Allow to disable certain actions by assigning null to them
  2262. * [Fix] Disable results caching
  2263. * [Fix] Fix disabling of squeezed symbols
  2264. * [Fix] Fix scan time set
  2265. * [Fix] Rework logic of actions setting
  2266. * [Fix] Try to fix various Lua stack issues
  2267. * [WebUI] Add link tag for favicon.ico
  2268. * [WebUI] Display hostname:port/path in the page title
  2269. 1.7.1: 20 Mar 2018
  2270. * [CritFix] Fix lowercase comparison
  2271. * [CritFix] Timezone defines seconds WEST UTC not East
  2272. * [Feature] Add filename to log format
  2273. * [Feature] Add lua rules squeezing
  2274. * [Feature] Add related symbols analysis to rspamd_stats
  2275. * [Feature] Remove upstream `X-Spam: Yes` header by default
  2276. * [Feature] rspamd_stats: Output progress info on STDERR
  2277. * [Feature] Whitelist for emails module
  2278. * [Fix] Do not allow dependencies on self
  2279. * [Fix] Do not cache metric result
  2280. * [Fix] Do not trust all issuers as a client certificate
  2281. * [Fix] Fix dependencies in lua squeeze
  2282. * [Fix] Fix enabling/disabling squeezed rules
  2283. * [Fix] Fix enabling/disabling symbols
  2284. * [Fix] Fix external dependencies
  2285. * [Fix] Fix processing of a single compressed file
  2286. * [Fix] Fix some typos
  2287. * [Fix] Fix various modules in case of empty message
  2288. * [Fix] Handle callbacks that returns table of options
  2289. * [Fix] Improve cached action interaction
  2290. * [Fix] Make dynamic conf more NaN aware
  2291. * [Fix] Never hide actions from WebUI `configuration` tab
  2292. * [Project] Implementation of Lua rules squeezing
  2293. 1.7.0: 12 Mar 2018
  2294. * [Conf] Add bayes_expiry as explicit module
  2295. * [Conf] Adjust names and weights for neural network plugin
  2296. * [Conf] Change updates url
  2297. * [Conf] Default statistics is stored in Redis now
  2298. * [Conf] Disable fann_redis module by default
  2299. * [Conf] Fix default elastic configuration
  2300. * [Conf] Fix double quote position
  2301. * [Conf] Massive config rework for new structure of symbols and scores
  2302. * [Conf] Rename Rambler BLs as they are now Rspamd's ones
  2303. * [Conf] Use dedicated rspamd.com subdomains
  2304. * [Conf] Use more data from rspamd.com fuzzy storage
  2305. * [CritFix] Add sanity guards for badly broken HTML
  2306. * [CritFix] Another errors path handling fix
  2307. * [CritFix] Another portion of tokenization fixes
  2308. * [CritFix] Do not send reject messages after set reply
  2309. * [CritFix] Fix ARC chain verification
  2310. * [CritFix] Fix crash in milter errors handler
  2311. * [CritFix] Fix memory leak in spf caching logic
  2312. * [CritFix] Fix milter commands pipelining
  2313. * [CritFix] Fix newlines detection
  2314. * [CritFix] Fix semicolons parsing in the content type
  2315. * [CritFix] Plug memory leak in zstd protocol compression
  2316. * [Feature] Add ability to match score in force_actions module
  2317. * [Feature] Add aes-rng PRF to libottery
  2318. * [Feature] Add 'composites' debug module
  2319. * [Feature] Add concept of experimental modules
  2320. * [Feature] Add DKIM trace symbol
  2321. * [Feature] Add EBL to the default config
  2322. * [Feature] Add expected ip check for emails plugin
  2323. * [Feature] Add framework to manage Redis scripts
  2324. * [Feature] Add framing for the new reputation generic plugin
  2325. * [Feature] Add function to show plugins stat
  2326. * [Feature] Add gzip compression support for clickhouse module
  2327. * [Feature] Add gzip compression support for rspamd controller
  2328. * [Feature] Add gzip support when sending lua http requests
  2329. * [Feature] Add json output for rspamd_stats
  2330. * [Feature] Add method to do a synchronous Redis connection
  2331. * [Feature] Add method to get all content-type attributes in Lua
  2332. * [Feature] Add `-m` flag to configdump to show modules states
  2333. * [Feature] Add mime types to extensions map
  2334. * [Feature] Add more features to rescore utility
  2335. * [Feature] Add more gtube like patterns to test other spam actions
  2336. * [Feature] Add more metafunctions, improve logging
  2337. * [Feature] Add more text attributes
  2338. * [Feature] Add new configwizard command to rspamadm
  2339. * [Feature] Add new tooling for stats conversation
  2340. * [Feature] Add old groups migration tool
  2341. * [Feature] Add plugins state variable
  2342. * [Feature] Add preliminary ecdsa keys support in DKIM
  2343. * [Feature] Add preliminary support of idempotent symbols
  2344. * [Feature] Add Redis server wizard
  2345. * [Feature] Add routine to convert old style stats to a new one
  2346. * [Feature] Add some sanity checks for actions and controller
  2347. * [Feature] Add statistic convertation module to configwizard
  2348. * [Feature] Add sugestions logic to mempool allocator
  2349. * [Feature] Add support of config transform in Lua
  2350. * [Feature] Add timeout to rspamc when doing corpus test
  2351. * [Feature] Add tooling to convert bayes schemas
  2352. * [Feature] Add torch conditional to configuration
  2353. * [Feature] Add torch-decisiontree package
  2354. * [Feature] Add torch-optim contrib package
  2355. * [Feature] Add TTL autodetection
  2356. * [Feature] Add urls reputation to the reputation framework
  2357. * [Feature] Allow floating and negative values in expressions limits
  2358. * [Feature] Allow multiple CTs in full extensions map
  2359. * [Feature] Allow multiple fann rules
  2360. * [Feature] Allow randomly select User-Agent from a list
  2361. * [Feature] Allow rspamadm commands to export methods in Lua
  2362. * [Feature] Allow rule specific min_bytes in fuzzy check
  2363. * [Feature] Allow to adjust symbols scores from Lua
  2364. * [Feature] Allow to attach stat signature to messages
  2365. * [Feature] Allow to change SMTP from via milter headers
  2366. * [Feature] Allow to configure monitored
  2367. * [Feature] Allow to create directories in Lua API
  2368. * [Feature] Allow to disable torch and skip train samples for ANN
  2369. * [Feature] Allow to discard messages dynamically
  2370. * [Feature] Allow to enable/disable languages from the detector
  2371. * [Feature] Allow to generate DKIM keys from rspamadm API
  2372. * [Feature] Allow to get CPU flags from Lua
  2373. * [Feature] Allow to have high precision timestamps in logs
  2374. * [Feature] Allow to insert headers into specific position
  2375. * [Feature] Allow to limit redirector requests per task
  2376. * [Feature] Allow to load and use dynamic ANNs with torch
  2377. * [Feature] Allow to quarantine rejected messages using milter interface
  2378. * [Feature] Allow to receive signing keys from mempool vars
  2379. * [Feature] Allow to reserve elements in libucl
  2380. * [Feature] Allow to reuse signal handlers chains
  2381. * [Feature] Allow to set custom mempool variables from settings
  2382. * [Feature] Allow to set headers from settings
  2383. * [Feature] Allow to set Settings-Id for all connections
  2384. * [Feature] Allow to skip real action and add a header instead
  2385. * [Feature] Allow to skip specific hashes in fuzzy storage
  2386. * [Feature] Allow to spawn asynchronous processes from Lua
  2387. * [Feature] Allow to specify number of threads for ANN learning
  2388. * [Feature] Allow to use global lua maps in settings
  2389. * [Feature] Allow to use postfilters in composites
  2390. * [Feature] Allow to verify signatures from HTTP headers in maps
  2391. * [Feature] Antivirus: ordered pattern matches
  2392. * [Feature] Authentication-Results: support hiding usernames
  2393. * [Feature] Automatically create tables in clickhouse
  2394. * [Feature] Catch next-to-last bad extension
  2395. * [Feature] Check cached maps more frequently
  2396. * [Feature] Check groups sanity
  2397. * [Feature] Deal with obscured URLs with @ symbols
  2398. * [Feature] Enhance task:store_in_file method
  2399. * [Feature] Export password encryption routines to Redis
  2400. * [Feature] Filter nan and inf when adding scores
  2401. * [Feature] Finalize 7zip files support
  2402. * [Feature] Further improvements in language detection
  2403. * [Feature] Further improvements in language detection algorithm
  2404. * [Feature] Generic key name expansion for Redis keys
  2405. * [Feature] Hash whitelist for fuzzy_check
  2406. * [Feature] Implement bayes signatures storage
  2407. * [Feature] Implement buckets for Redis backend
  2408. * [Feature] Implement DKIM reputation adjustments
  2409. * [Feature] Implement forked workers children monitoring
  2410. * [Feature] Implement headers flags in mime parser
  2411. * [Feature] Implement l1/l2 regularization against the current weights
  2412. * [Feature] Implement manual ANN train mode
  2413. * [Feature] Implement per-user ANN support
  2414. * [Feature] Implement torch based ANN learning
  2415. * [Feature] Implement upstreams logic for clickhouse exporter
  2416. * [Feature] Import torch to Rspamd...
  2417. * [Feature] Improve allocation policy when interacting with Lua
  2418. * [Feature] Improve Lua/C interaction in history_redis
  2419. * [Feature] Improve multiple fuzzy results combining
  2420. * [Feature] Improve parsing of DKIM keys: parse algorithm
  2421. * [Feature] Improve subprocesses termination handle
  2422. * [Feature] Improve symbol type parsing in Lua API
  2423. * [Feature] Metadata Exporter: e-Mail Alerts: support multiple recipients; alerting senders/recipients/users (#1600)
  2424. * [Feature] Milter headers: support adding/removing arbitrary headers from config
  2425. * [Feature] More metatokens
  2426. * [Feature] Multimap: checking of symbol options
  2427. * [Feature] Multimap: template URL filter
  2428. * [Feature] New bayes expiry plugin
  2429. * [Feature] Periodically save rspamd stats to disk
  2430. * [Feature] Preliminary import of the elasticsearch module
  2431. * [Feature] Ratelimit: allow full addresses in whitelisted_rcpts
  2432. * [Feature] Ratelimit: support fetching limits from Redis
  2433. * [Feature] RBL: received: filtering by position & flags
  2434. * [Feature] Read global maps for lua
  2435. * [Feature] Redis settings: support checking multiple keys
  2436. * [Feature] Rework fann plugin to be a normal post-filter
  2437. * [Feature] Rework logging configuration for rspamadm case
  2438. * [Feature] Rework short hashes generation to avoid FP
  2439. * [Feature] Save real ucl types when exporting to Lua
  2440. * [Feature] Set TCP_NODELAY for milter sockets
  2441. * [Feature] Setup DKIM signing from configwizard
  2442. * [Feature] Skip certain symbols from ANN classify
  2443. * [Feature] Store plugins state
  2444. * [Feature] Support etag for HTTP maps
  2445. * [Feature] Support Expires header when using HTTP maps
  2446. * [Feature] Support sending given header multiple times in lua_http
  2447. * [Feature] Support sha512 in DKIM signatures
  2448. * [Feature] Try to detect HTML messages better
  2449. * [Feature] Use array instead of queue to reduce memory fragmentation
  2450. * [Feature] Use controller port by default when connecting to local IP
  2451. * [Feature] Use rdtsc where possible
  2452. * [Fix] Actively load skip hashes map in fuzzy storage
  2453. * [Fix] Add another workaround to display history properly
  2454. * [Fix] Add definition for old glib compatibility method
  2455. * [Fix] Add missing rspamadm control options to help
  2456. * [Fix] Add workaround for IPv6 in sendmail
  2457. * [Fix] Add workaround for system with non-XSI compatible tzset
  2458. * [Fix] Allow oversigning in DKIM signatures
  2459. * [Fix] Allow to check negative scores in force_actions
  2460. * [Fix] Allow to have negative actions limits
  2461. * [Fix] Allow to set any layers number for fann rules
  2462. * [Fix] Another fix for rdtcs
  2463. * [Fix] Another fix to lua xmlrpc
  2464. * [Fix] Another try to deal with #1998
  2465. * [Fix] Another try to fix #1998
  2466. * [Fix] Another try to fix threading in torch
  2467. * [Fix] Apply language detection when adding fuzzy hashes
  2468. * [Fix] ARC: Fix Lua 5.3 compatibility; timestamp should be integer
  2469. * [Fix] Authentication Results: Fix SPF smtp.mail_from
  2470. * [Fix] Auth-Results: Multiple DKIM signatures
  2471. * [Fix] Avoid changing content-transfer-encoding header's value
  2472. * [Fix] Better handling of the legacy protocol
  2473. * [Fix] Check decoded headers sanity (e.g. by excluding \0)
  2474. * [Fix] Check for magic when checking for an archive
  2475. * [Fix] Cleanup mess with groups
  2476. * [Fix] Clickhouse: Insertion in the symbols table
  2477. * [Fix] Crash in URL processing
  2478. * [Fix] Deal with another case when processing exceptions
  2479. * [Fix] Deal with deeply nested messages more aggressively
  2480. * [Fix] Deal with nan and inf encoding in json/ucl
  2481. * [Fix] Deal with non-key arguments in lua_redis.exec_script
  2482. * [Fix] Deal with unknown weight
  2483. * [Fix] Deal with URLs with no slashes after protocol
  2484. * [Fix] Deal with URLs wrapped in [] in text parts
  2485. * [Fix] Deal with zero scores symbols
  2486. * [Fix] Default monitoring domain for surbl plugin
  2487. * [Fix] Delay upstream re-resolving when one upstream is defined
  2488. * [Fix] Detection of maillist optimized and fixed
  2489. * [Fix] DKIM signing: allow for auth_only to be false
  2490. * [Fix] DMARC: require report_settings for sending reports only
  2491. * [Fix] Do not allow garbadge when checking url domain
  2492. * [Fix] Do not cache SPF records with PTR elements
  2493. * [Fix] Do not constantly re-resolve failed upstreams with a single element
  2494. * [Fix] Do not crash if no words defined
  2495. * [Fix] Do not crash on empty subtype
  2496. * [Fix] Do not expose spamtrap messages to SMTP reply
  2497. * [Fix] Do not fail rbl plugin when there are no received or emails
  2498. * [Fix] Do not ignore short words
  2499. * [Fix] Do not include idempotent/nostat symbols to checksum
  2500. * [Fix] Do not override groups when converting metrics
  2501. * [Fix] Do not override unix socket group when group comes before owner
  2502. * [Fix] Do not skip the last character
  2503. * [Fix] Do not spawn too many workers by default
  2504. * [Fix] Do not stop monitored on dns errors
  2505. * [Fix] Do not stop parsing headers on bad IP header
  2506. * [Fix] Do not strip last character in the last word
  2507. * [Fix] Do not treat script content as text
  2508. * [Fix] Do not try to connect to non-supported addresses
  2509. * [Fix] Do not try to dereference last character
  2510. * [Fix] Do not try to sign unknown domains
  2511. * [Fix] Don't use whitelist/greylist maps as regexp, but as map
  2512. * [Fix] Erase unknown HTML entities
  2513. * [Fix] Exim Received header protocol parsing
  2514. * [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false
  2515. * [Fix] Fix a lot of FP in chartable in mixed languages
  2516. * [Fix] Fix ANN checks
  2517. * [Fix] Fix ANN loading logic
  2518. * [Fix] Fix another tokenization issue
  2519. * [Fix] Fix autolearn parameters reading
  2520. * [Fix] Fix bad archive characters stripping
  2521. * [Fix] Fix bad extension check
  2522. * [Fix] Fix bayes schema conversion
  2523. * [Fix] Fix blacklists and DMARC in whitelist
  2524. * [Fix] Fix brain-damaged torch build system
  2525. * [Fix] Fix build on FreeBSD
  2526. * [Fix] Fix clickhouse exporter
  2527. * [Fix] Fix clickhouse schema
  2528. * [Fix] Fix comparision
  2529. * [Fix] Fix composites processing
  2530. * [Fix] Fix connecting to a unix socket in rspamadm statconvert
  2531. * [Fix] Fix couple of warnings
  2532. * [Fix] Fix crashes in the rspamd_control path
  2533. * [Fix] Fix deletion from hash
  2534. * [Fix] Fix DKIM forgeries via multiple headers
  2535. * [Fix] FIx dynamic conf plugin
  2536. * [Fix] Fix emails detection
  2537. * [Fix] Fix empty headers simple canonicalization
  2538. * [Fix] Fix empty threshold check in greylisting module
  2539. * [Fix] Fix encrypted legacy reply in fuzzy storage
  2540. * [Fix] Fix enormous scores for R_WHITE_ON_WHITE
  2541. * [Fix] Fix exceptions list in surbl
  2542. * [Fix] Fix *_EXCESS_BASE64 rules
  2543. * [Fix] Fix expire rounding
  2544. * [Fix] Fix extra hits in PCRE mode for regular expressions
  2545. * [Fix] Fix format strings
  2546. * [Fix] Fix get_content method
  2547. * [Fix] Fix groups override when defining symbols
  2548. * [Fix] Fix learned count in new schema
  2549. * [Fix] Fix learn errors propagation
  2550. * [Fix] Fix loading of per-user redis backend for statistics
  2551. * [Fix] Fix logging buffer corruption in case of repeated messages
  2552. * [Fix] Fix lua cached elements invalidation
  2553. * [Fix] Fix merging of the implicit arrays
  2554. * [Fix] Fix mime_types scoring
  2555. * [Fix] Fix multiple headers in DKIM headers list
  2556. * [Fix] Fix null callee case in clang plugin
  2557. * [Fix] Fix obscured url in format user@@example.com
  2558. * [Fix] Fix parsing of the per-user script
  2559. * [Fix] Fix priorities in rspamd_update, disable rules execution
  2560. * [Fix] Fix processing of closed tags
  2561. * [Fix] Fix processing of idempotent rules when autolearn fails
  2562. * [Fix] Fix processing of multipart parts with no headers
  2563. * [Fix] Fix processing of skip-hashes in fuzzy storage
  2564. * [Fix] Fix PTR processing in SPF
  2565. * [Fix] Fix pushing country to clickhouse asn table
  2566. * [Fix] Fix random forests module
  2567. * [Fix] Fix real IP parsing for some strange Exim received
  2568. * [Fix] Fix Redis timeout setup
  2569. * [Fix] Fix reload crash when hyperscan is enabled
  2570. * [Fix] Fix reusing of redis connection after exec
  2571. * [Fix] Fix sanity checks on macro value
  2572. * [Fix] Fix setting of path and cpath for Lua
  2573. * [Fix] Fix setting of signals when spawning a thread
  2574. * [Fix] Fix text splitting: stack overflow (too many captures)
  2575. * [Fix] Fix ticks processing
  2576. * [Fix] Fix upstream addrs updating
  2577. * [Fix] Fix urls/emails distinguishing found in queries
  2578. * [Fix] Fix user settings check
  2579. * [Fix] Fix variable increment
  2580. * [Fix] Fix various issues in stat_convert
  2581. * [Fix] F-PROT Antivirus infection string for all known occurences
  2582. * [Fix] F-PROT Antivirus: only check return code to determine infection
  2583. * [Fix] Further fixes around floating point expressions
  2584. * [Fix] Further fixes to ANN module
  2585. * [Fix] Further fixes to rescore tool
  2586. * [Fix] Further fixes to support ES 6
  2587. * [Fix] Further tokenization fixes
  2588. * [Fix] Greylisting set phase is not idempotent
  2589. * [Fix] Handle proxy copy errors
  2590. * [Fix] Header checks: Fix get_raw_header method
  2591. * [Fix] Header checks: REPLYTO_UNPARSEABLE rule
  2592. * [Fix] Kill spawned processes on termination
  2593. * [Fix] Load skip map from all processes as shared cache is unavailable
  2594. * [Fix] Lowercase HTTP headers to make them searchable from Lua
  2595. * [Fix] Lowercase words
  2596. * [Fix] Lua_http: freeing
  2597. * [Fix] Lua: lpeg to be loaded with rspamd_lua_add_preload, to avoid "rspamd_config_read: rcl parse error: cannot init lua file […] module 'lpeg' not found"
  2598. * [Fix] Map absence is not an error
  2599. * [Fix] Metadata exporter: check IP sanity
  2600. * [Fix] Milter headers: custom headers: removing headers
  2601. * [Fix] Milter headers: skip_local / skip_authenticated settings
  2602. * [Fix] Milter headers: X-Spamd-Result header if X-Virus ran first
  2603. * [Fix] mime_types: fix next-to-last extension length check
  2604. * [Fix] More hacks to deal with old configs
  2605. * [Fix] Move composites second pass to the dedicated stage
  2606. * [Fix] Multimap: received: filtering of artificial header
  2607. * [Fix] Multiple fixes in torch based ANN plugins
  2608. * [Fix] Once more (#1879) fix bad extension check
  2609. * [Fix] Optimize rspamd_fstring_t reallocations
  2610. * [Fix] options.local_networks setting
  2611. * [Fix] Parse HREF urls without explicit prefix
  2612. * [Fix] Plan new event on HTTP errors
  2613. * [Fix] Plug another possible memory leak
  2614. * [Fix] Plug memory leak
  2615. * [Fix] Plug memory leak in lua_tcp
  2616. * [Fix] Plug memory leak when setting email addresses from Lua
  2617. * [Fix] Propagate learn/stat errors more precisely
  2618. * [Fix] Ratelimit: fix whitelisted_rcpts matching
  2619. * [Fix] Ratelimit: lowercase email addresses
  2620. * [Fix] RBL: received: deal with missing data (#1965)
  2621. * [Fix] Rebalance and slightly rework MX check plugin
  2622. * [Fix] Redis key expansion: EVAL: deal with strings
  2623. * [Fix] Redis script loading in DMARC; URL tags; URL reputation
  2624. * [Fix] Reject invalid bh for DKIM signatures earlier
  2625. * [Fix] Relax pem signature detection
  2626. * [Fix] Relax unicode properties requirements for chartable module
  2627. * [Fix] Remove extra noise from dkim and arc signing
  2628. * [Fix] Remove hop-by-hop headers in proxy
  2629. * [Fix] Remove incorrect method `task:set_metric_subject`
  2630. * [Fix] Replace space like characters in headers with plain space
  2631. * [Fix] Restore old style ratelimits support
  2632. * [Fix] Rework elasticsearch plugin
  2633. * [Fix] Rewriting subjects via force actions module
  2634. * [Fix] RPM postinstall
  2635. * [Fix] Sanitize IP in history redis
  2636. * [Fix] Select the correct signature when doing simple canon
  2637. * [Fix] Set CLOEXEC flag on files opened
  2638. * [Fix] Setting check_local / check_authed in plugins (#1954)
  2639. * [Fix] Settings: avoid checking invalid IP (#1981)
  2640. * [Fix] Settings: header: deal with multiple settings (#1988)
  2641. * [Fix] Skip checks if both extensions are not bad
  2642. * [Fix] Skip nostat tokens when get number of tokens
  2643. * [Fix] Some more fixes towards emails detection
  2644. * [Fix] SpamAssassin: Fail check_freemail_header if regexp didn't match
  2645. * [Fix] Stop using of g_slice...
  2646. * [Fix] Switch rspamadm logging to message level
  2647. * [Fix] Symbol 'FANNR_SPAM' has its score defined..
  2648. * [Fix] Table parameter for rspamd_config:add_doc()
  2649. * [Fix] Treat 'rewrite subject' as spam action
  2650. * [Fix] Try harder in passing IPv6 addresses
  2651. * [Fix] Try harder to find rfc822 notifications
  2652. * [Fix] Try harder to find urls
  2653. * [Fix] Use decoded values when parsing mime addresses
  2654. * [Fix] Use full URL when making an HTTP request
  2655. * [Fix] Use greylisting threshold in greylisting module
  2656. * [Fix] Use n_words attribute from ngramms
  2657. * [Fix] Use raw urls when sending requests to redirector
  2658. * [Fix] Use the right boolean operator on error check
  2659. * [Fix] Use weight from map for fuzzy scoring
  2660. * [Fix] Various fixes to elastic plugin
  2661. * [Fix] Various fixes to fann_redis instantiation
  2662. * [Fix] Various improvements in language detection
  2663. * [Fix] Virus infection string for F-PROT Antivirus
  2664. * [Fix] Virus infetction string for F-PROT Antivirus
  2665. * [Fix] WebUI: use relative path for savemap (#1943)
  2666. * [Fix] WHITE_ON_WHITE: Ensure score is matched to part that fired the rule
  2667. * [Fix] Write configuration changes as UCL config
  2668. * [Project] Add detection logic for words
  2669. * [Project] Add fast debug logging infrastructure
  2670. * [Project] Add more flags to languages
  2671. * [Project] Add n-gramms data files
  2672. * [Project] Add ngramms frequencies detector
  2673. * [Project] Add random words selection logic
  2674. * [Project] Add unigramms to language detection as well
  2675. * [Project] Convert all C modules to fast debug infrastructure
  2676. * [Project] Detect some languages based on unicode script
  2677. * [Project] Enable fast debug lookup for some modules
  2678. * [Project] Enable language detector init in scanner workers
  2679. * [Project] Further improvements to language detector
  2680. * [Project] Implement logic of ngramms application
  2681. * [Project] Improve weighting in lang_detection
  2682. * [Project] Initialize language detector
  2683. * [Project] Preliminary version of ngramms based language detector
  2684. * [Project] Preliminary version of the new stat_convert
  2685. * [Project] Remove old language detector
  2686. * [Project] Rework language detection ngramms structure
  2687. * [Project] Start language detection project
  2688. * [Project] Start rework of language detection to improve quality
  2689. * [Project] Use fast debug logging check
  2690. * [Rework] Add frame for new reputation based IP score module
  2691. * [Rework] Continue stat_convert rework task
  2692. * [Rework] Implement new version of fuzzy replies
  2693. * [Rework] Improve readability of xmlrpc API
  2694. * [Rework] Kill metrics!11
  2695. * [Rework] Ratelimit module
  2696. * [Rework] Rename fann_redis to neural plugin
  2697. * [Rework] Reorganize mime_types module
  2698. * [Rework] Rework rescore utility
  2699. * [Rework] Rewrite model and learning logic for rescore
  2700. * [Rework] Run post-loads when all initialization is completed
  2701. * [Rework] Simplify lua path initialization
  2702. * [Rework] Start major stat_convert rework
  2703. * [Rework] Start mempool fragmentation reduce project
  2704. * [Rework] Start moving of fann redis to torch
  2705. * [Rework] Stop embedding rspamadm scripts into C
  2706. * [Rework] Use floating point arithmetics in Rspamd expressions
  2707. * [Rework] Use frequencies distribution in language detector
  2708. * [Rules] Penalise R_BAD_CTE_7BIT for utf8 messages
  2709. * [WebUI] Compact graph selectors
  2710. * [WebUI] Escape strings inside HTML in history
  2711. * [WebUI] Fix message count in throughput summary (#1724)
  2712. * [WebUI] Fix NaNs display on Throughput graph
  2713. * [WebUI] Migrate widgets to D3 v4
  2714. * [WebUI] Restore passwordless login support (#2003)
  2715. * [WebUI] Show symbol descriptions as tooltips in history
  2716. * [WebUI] Stop using commas in pie chart tooltips
  2717. * [WebUI] Update D3 and jQuery
  2718. * [WebUI] Update D3Evolution 1.0.0 -> 1.1.0
  2719. 1.6.6: 16 Feb 2018
  2720. * [CritFix] Add sanity guards for badly broken HTML
  2721. * [CritFix] Another errors path handling fix
  2722. * [CritFix] Fix ARC chain verification
  2723. * [CritFix] Fix crash in milter errors handler
  2724. * [Feature] Allow to insert headers into specific position
  2725. * [Feature] Allow to receive signing keys from mempool vars
  2726. * [Feature] Authentication-Results: support hiding usernames
  2727. * [Fix] Another try to deal with #1998
  2728. * [Fix] Another try to fix #1998
  2729. * [Fix] Better handling of the legacy protocol
  2730. * [Fix] Check decoded headers sanity (e.g. by excluding \0)
  2731. * [Fix] Deal with nan and inf encoding in json/ucl
  2732. * [Fix] Deal with URLs wrapped in [] in text parts
  2733. * [Fix] DKIM signing: allow for auth_only to be false
  2734. * [Fix] Do not crash on empty subtype
  2735. * [Fix] Do not fail rbl plugin when there are no received or emails
  2736. * [Fix] Do not skip the last character
  2737. * [Fix] Do not try to dereference last character
  2738. * [Fix] Do not try to sign unknown domains
  2739. * [Fix] Exim Received header protocol parsing
  2740. * [Fix] First load selector_map and path_map. And only return false when domain not found if try_fallback is false
  2741. * [Fix] Fix bad archive characters stripping
  2742. * [Fix] Fix comparision
  2743. * [Fix] Fix connecting to a unix socket in rspamadm statconvert
  2744. * [Fix] Fix empty headers simple canonicalization
  2745. * [Fix] Fix extra hits in PCRE mode for regular expressions
  2746. * [Fix] Fix parsing of the per-user script
  2747. * [Fix] Fix processing of skip-hashes in fuzzy storage
  2748. * [Fix] Fix Redis timeout setup
  2749. * [Fix] Fix sanity checks on macro value
  2750. * [Fix] Fix text splitting: stack overflow (too many captures)
  2751. * [Fix] Fix urls/emails distinguishing found in queries
  2752. * [Fix] F-PROT Antivirus: only check return code to determine infection
  2753. * [Fix] Metadata exporter: check IP sanity
  2754. * [Fix] Multimap: received: filtering of artificial header
  2755. * [Fix] Plan new event on HTTP errors
  2756. * [Fix] Plug another possible memory leak
  2757. * [Fix] Remove hop-by-hop headers in proxy
  2758. * [Fix] Sanitize IP in history redis
  2759. * [Fix] Setting check_local / check_authed in plugins (#1954)
  2760. * [Fix] Settings: avoid checking invalid IP (#1981)
  2761. * [Fix] Try harder in passing IPv6 addresses
  2762. * [Fix] WebUI: use relative path for savemap (#1943)
  2763. * [WebUI] Fix message count in throughput summary (#1724)
  2764. * [WebUI] Fix NaNs display on Throughput graph
  2765. * [WebUI] Restore passwordless login support (#2003)
  2766. 1.6.5: 22 Oct 2017
  2767. * [CritFix] Another portion of tokenization fixes
  2768. * [CritFix] Fix memory leak in spf caching logic
  2769. * [CritFix] Fix milter commands pipelining
  2770. * [CritFix] Fix newlines detection
  2771. * [Feature] Filter nan and inf when adding scores
  2772. * [Feature] Implement headers flags in mime parser
  2773. * [Feature] Support Expires header when using HTTP maps
  2774. * [Fix] Actively load skip hashes map in fuzzy storage
  2775. * [Fix] Add workaround for IPv6 in sendmail
  2776. * [Fix] Authentication Results: Fix SPF smtp.mail_from
  2777. * [Fix] Check for magic when checking for an archive
  2778. * [Fix] Deal with another case when processing exceptions
  2779. * [Fix] Deal with URLs with no slashes after protocol
  2780. * [Fix] Do not allow garbadge when checking url domain
  2781. * [Fix] Do not ignore short words
  2782. * [Fix] Do not strip last character in the last word
  2783. * [Fix] Do not treat script content as text
  2784. * [Fix] Erase unknown HTML entities
  2785. * [Fix] Fix another tokenization issue
  2786. * [Fix] Fix DKIM forgeries via multiple headers
  2787. * [Fix] Fix emails detection
  2788. * [Fix] Fix empty threshold check in greylisting module
  2789. * [Fix] Fix enormous scores for R_WHITE_ON_WHITE
  2790. * [Fix] Fix loading of per-user redis backend for statistics
  2791. * [Fix] Fix multiple headers in DKIM headers list
  2792. * [Fix] Fix obscured url in format user@@example.com
  2793. * [Fix] Further tokenization fixes
  2794. * [Fix] Load skip map from all processes as shared cache is unavailable
  2795. * [Fix] Lowercase words
  2796. * [Fix] Milter headers: skip_local / skip_authenticated settings
  2797. * [Fix] Milter headers: X-Spamd-Result header if X-Virus ran first
  2798. * [Fix] Ratelimit: fix whitelisted_rcpts matching
  2799. * [Fix] Some more fixes towards emails detection
  2800. * [Fix] SpamAssassin: Fail check_freemail_header if regexp didn't match
  2801. * [Fix] Use greylisting threshold in greylisting module
  2802. 1.6.4: 10 Sep 2017
  2803. * [Feature] Add method to get all content-type attributes in Lua
  2804. * [Feature] Add some sanity checks for actions and controller
  2805. * [Feature] Allow randomly select User-Agent from a list
  2806. * [Feature] Deal with obscured URLs with @ symbols
  2807. * [Feature] Milter headers: support adding/removing arbitrary headers from config
  2808. * [Fix] Add another workaround to display history properly
  2809. * [Fix] Add missing rspamadm control options to help
  2810. * [Fix] Auth-Results: Multiple DKIM signatures
  2811. * [Fix] Crash in URL processing
  2812. * [Fix] Default monitoring domain for surbl plugin
  2813. * [Fix] Detection of maillist optimized and fixed
  2814. * [Fix] Do not cache SPF records with PTR elements
  2815. * [Fix] Fix blacklists and DMARC in whitelist
  2816. * [Fix] Fix exceptions list in surbl
  2817. * [Fix] Fix processing of closed tags
  2818. * [Fix] Fix PTR processing in SPF
  2819. * [Fix] Lowercase HTTP headers to make them searchable from Lua
  2820. * [Fix] options.local_networks setting
  2821. * [Fix] Ratelimit: lowercase email addresses
  2822. * [Fix] Rebalance and slightly rework MX check plugin
  2823. * [Fix] Redis script loading in DMARC; URL tags; URL reputation
  2824. * [Fix] Reject invalid bh for DKIM signatures earlier
  2825. * [Fix] Remove incorrect method `task:set_metric_subject`
  2826. * [Fix] Rewriting subjects via force actions module
  2827. * [Fix] RPM postinstall
  2828. * [Fix] Treat 'rewrite subject' as spam action
  2829. * [Fix] Try harder to find urls
  2830. * [Fix] Use full URL when making an HTTP request
  2831. * [Fix] Use raw urls when sending requests to redirector
  2832. * [Fix] Use weight from map for fuzzy scoring
  2833. * [Rules] Penalise R_BAD_CTE_7BIT for utf8 messages
  2834. 1.6.3: 26 Jul 2017
  2835. * [CritFix] Fix semicolons parsing in the content type
  2836. * [Feature] Add EBL to the default config
  2837. * [Feature] Allow to configure monitored
  2838. * [Feature] Allow to skip specific hashes in fuzzy storage
  2839. * [Feature] Multimap: checking of symbol options
  2840. * [Feature] Redis settings: support checking multiple keys
  2841. * [Fix] ARC: Fix Lua 5.3 compatibility; timestamp should be integer
  2842. * [Fix] Avoid changing content-transfer-encoding header's value
  2843. * [Fix] Don't use whitelist/greylist maps as regexp, but as map
  2844. * [Fix] Fix get_content method
  2845. * [Fix] Header checks: Fix get_raw_header method
  2846. * [Fix] Header checks: REPLYTO_UNPARSEABLE rule
  2847. * [Fix] Lua_http: freeing
  2848. * [Fix] Milter headers: custom headers: removing headers
  2849. * [Fix] Parse HREF urls without explicit prefix
  2850. * [Fix] WHITE_ON_WHITE: Ensure score is matched to part that fired the rule
  2851. * [WebUI] Escape strings inside HTML in history
  2852. 1.6.2: 08 Jul 2017
  2853. * [Conf] Remove Rambler email bl for now
  2854. * [Conf] Switch RAMBLER_URIBL to a locally managed source
  2855. * [CritFix] Switch from ragel to C for Content-Type parsing
  2856. * [Feature] Add `-e` option for lua_repl
  2857. * [Feature] Add per-domain emails normalisation rules
  2858. * [Feature] Add sessions cache to debug dangling sessions
  2859. * [Feature] Add short_text_direct_hash for fuzzy check module
  2860. * [Feature] Add text_part:get_stats function
  2861. * [Feature] Allow to add custom processing script for surbl
  2862. * [Feature] Allow to check reply-to email
  2863. * [Feature] Allow to customize spam header, remove existing spam headers
  2864. * [Feature] Allow to disable specific workers in the config
  2865. * [Feature] Allow to discard messages instead of rejection
  2866. * [Feature] Allow to specify custom delimiter in emails plugin
  2867. * [Feature] Allow to specify custom User-Agent for rspamc
  2868. * [Feature] Allow to store symbols data in Clickhouse
  2869. * [Feature] Allow to use HTTPS when connecting to Clickhouse
  2870. * [Feature] Enable sessions cache tracking for milter connections
  2871. * [Feature] Implement per-line mode in lua_repl (like `perl -p`)
  2872. * [Feature] Implement rdns-curve plugin based on rspamd cryptobox
  2873. * [Feature] Improve maps cached data lifetime
  2874. * [Feature] Improve maps checking frequency
  2875. * [Feature] Improve monitored timeouts logic
  2876. * [Feature] milter_headers: add `extended_headers_rcpt` option
  2877. * [Feature] Milter headers: Add X-Spam-Flag to rmilter-compatibility headers
  2878. * [Feature] Milter headers: remove-header routine
  2879. * [Feature] Multimap: received filters for extracting TLDs from hostnames
  2880. * [Feature] Normalize email aliases in emails module
  2881. * [Feature] Re-add rambler email bl (as hashed list)
  2882. * [Feature] Reload file maps more frequently
  2883. * [Feature] Rework newlines strip parser one more time
  2884. * [Feature] Skip updates for messages scanned via controller
  2885. * [Feature] Split long DKIM public keys
  2886. * [Feature] Store more data when stripping newlines
  2887. * [Feature] Support SPF macros transformations
  2888. * [Feature] Support suppressing DMARC reports for some domains
  2889. * [Fix] Add missing `break` statement
  2890. * [Fix] Allow modifiers in SPF macros
  2891. * [Fix] DKIM sign tools: edge-cases around use_esld
  2892. * [Fix] Do not cache SPF records with macros
  2893. * [Fix] Do not overwrite score when setting pre-action
  2894. * [Fix] Fix comparison logic
  2895. * [Fix] Fix DKIM base64 folding for milter flagged messages
  2896. * [Fix] Fix emails module configuration
  2897. * [Fix] Fix folding for arc headers when milter interface is used
  2898. * [Fix] Fix gmail dots removal
  2899. * [Fix] Fix rspamc detection in greylist module
  2900. * [Fix] Fix some more issues with HTTP maps
  2901. * [Fix] Milter sessions can live forever
  2902. * [Fix] Normalize fuzzy probability better
  2903. * [Fix] Plug memory leak
  2904. * [Fix] RBL: Fixed hashed email address lookups
  2905. * [Fix] Try to deal with brain-damaged milter behaviour
  2906. * [Fix] Use `\n` to fold headers for milter
  2907. * [Rework] Allow to use custom callback for monitored checks
  2908. * [Rework] Further steps towards one process monitoring
  2909. * [Rework] Send health checks from a single worker
  2910. * [WebUI] Round-up throughput summary values
  2911. 1.6.1: 14 Jun 2017
  2912. * [Fix] Allow to init resolver without rspamd_config
  2913. * [Fix] Do not crash when resolver failed to initialize
  2914. * [Fix] Fix abstract context layout
  2915. * [Fix] Fix CGP helper reply parsing
  2916. * [Fix] Fix crashes when socket write errors occur
  2917. * [Fix] Fix parsing IPv6 nameservers in resolv.conf
  2918. * [Fix] Milter: Don't defer on "greylist" action
  2919. 1.6.0: 12 Jun 2017
  2920. * [Conf] Add rspamd_proxy to the default configuration set
  2921. * [Conf] Add sample arc module config
  2922. * [Conf] Do away with systemd specifics completely
  2923. * [Conf] Increase min_bytes to avoid FP
  2924. * [Conf] Remove ratelimits from default configuration
  2925. * [CritFix] Fix accepting on IPv6 sockets
  2926. * [CritFix] Fix corruption when multiple fuzzy are defined
  2927. * [CritFix] Fix learn condition in fuzzy check
  2928. * [CritFix] Fix memory leak in fuzzy check
  2929. * [CritFix] Fix memory leak in maps scheduling
  2930. * [CritFix] Paese the last character in DKIM signature correctly
  2931. * [CritFix] Zero fill sockaddr_un
  2932. * [Feature] Add ability to add doc strings by example
  2933. * [Feature] Add API to verify DKIM (and ARC) signatures
  2934. * [Feature] Add compression/decompression to proxy
  2935. * [Feature] Add count to url structure
  2936. * [Feature] Add initial support of the new protocol reply
  2937. * [Feature] Add Lua plugin spamtrap
  2938. * [Feature] Add `monitored_address` for rbls
  2939. * [Feature] Add new schema for bayes tokens
  2940. * [Feature] Add preliminary ARC support to dkim code
  2941. * [Feature] Add preliminary support of ARC signing
  2942. * [Feature] Add rules to detect bad 8bit characters in From and To
  2943. * [Feature] Add scanning support for milter protocol
  2944. * [Feature] Add support for bidirectional symbols in rspamd_stats
  2945. * [Feature] Add support for static maps
  2946. * [Feature] Add support of maps with multiple regexps matches
  2947. * [Feature] Add `text_multiplier` param
  2948. * [Feature] Add the preliminary ARC plugin
  2949. * [Feature] Add top redirector targets rank
  2950. * [Feature] Allow async events to be registered from LUA rules
  2951. * [Feature] Allow storing bayes tokens in Redis
  2952. * [Feature] Allow to exclude specific domains from mx check
  2953. * [Feature] Allow to have a stack of watcher finalisers
  2954. * [Feature] Allow to pass hostname to `-i` flag in Rspamc
  2955. * [Feature] Allow to set custom user agent in url redirector
  2956. * [Feature] Allow to use custom callback when parsing resolv.conf
  2957. * [Feature] Allow to use domain from authenticated user
  2958. * [Feature] Bayes expiry plugin
  2959. * [Feature] Check dkim sign keys for modifications
  2960. * [Feature] DKIM signing: sign_networks/local address specific use_domain settings
  2961. * [Feature] DMARC: Support excluding domains from sampling
  2962. * [Feature] Expire processing items for URL redirector aggressively
  2963. * [Feature] Fix surbl monitored for IP lists, add `monitored_domain` option
  2964. * [Feature] Implement caching for dkim body hashes
  2965. * [Feature] Implement milter protocol scan reply
  2966. * [Feature] Improve omograph phishing detection
  2967. * [Feature] Initial support of self-scan in Rspamd proxy
  2968. * [Feature] Keep track of headers in milter interface
  2969. * [Feature] Milter headers: better controls for local/authenticated
  2970. * [Feature] Multimap: email:domain:tld filter
  2971. * [Feature] Preliminary DMARC reporting implementation
  2972. * [Feature] Reuse stemmers in the cache
  2973. * [Feature] Rework confighelp to load Lua plugins
  2974. * [Feature] Rework hfilter to use hyperscan if possible
  2975. * [Feature] Rework lua RSA API
  2976. * [Feature] Rmilter_headers: approximate rmilter's extended_spam_headers
  2977. * [Feature] Start integration of milter support in proxy
  2978. * [Feature] Store average words length and short words count
  2979. * [Feature] Store hash of headers order and names
  2980. * [Feature] Support MTA name header
  2981. * [Feature] Support multiple types of dkim signing in Lua
  2982. * [Feature] Support numeric arguments for Redis requests
  2983. * [Feature] Use headers hash in bayes metatokens
  2984. * [Feature] Use normal resolv.conf rules of rotation in Rspamd
  2985. * [Feature] Use version 2 proto for checking messages
  2986. * [Fix] Allow to follow symlinks when safe
  2987. * [Fix] Append MX name for authentication results as required
  2988. * [Fix] Change default text multiplier from 0.5 to 2.0
  2989. * [Fix] Check min_bytes for images as well
  2990. * [Fix] Deal with 7bit charsets properly
  2991. * [Fix] Deal with 8bit characters in email addresses
  2992. * [Fix] Deal with unpaired <a> tags
  2993. * [Fix] Detect confighelp in plugins initialisation
  2994. * [Fix] Disable certain checks for utf spoof detection
  2995. * [Fix] DKIM Signing: avoid nil index when From header is missing
  2996. * [Fix] Do not add exact hashes from different parts
  2997. * [Fix] Do not check DMARC if SPF or DKIM were not checked
  2998. * [Fix] Do not check URLs that are resolved to be redirected
  2999. * [Fix] Do not set bayes probability if we don't use it
  3000. * [Fix] Do not stop on illegal unicode points - replace them
  3001. * [Fix] Fix another race condition in arc checks
  3002. * [Fix] Fix arc count logic
  3003. * [Fix] Fix ARC signing
  3004. * [Fix] Fix brain-damaged spamc protocol for now
  3005. * [Fix] Fix calling for peak functions
  3006. * [Fix] Fix couple of issues in FORWARDED rule
  3007. * [Fix] Fix CTE propagation from parent containers to children parts
  3008. * [Fix] Fix errors processing in the controller
  3009. * [Fix] Fix format string in milter
  3010. * [Fix] Fix issues in SPF macros parsing
  3011. * [Fix] Fix logging format string
  3012. * [Fix] Fix logic of cached passwords check
  3013. * [Fix] Fix lowercasing of stemmed words
  3014. * [Fix] Fix LRU elements removal
  3015. * [Fix] Fix memory leak when accepting from unix sockets
  3016. * [Fix] Fix milter connections persistence
  3017. * [Fix] Fix objects merging in UCL
  3018. * [Fix] Fix order of operations to avoid race condition
  3019. * [Fix] Fix parsing of long regexp types
  3020. * [Fix] Fix passing data to log helper when many symbols defined
  3021. * [Fix] Fix pools management for milter session
  3022. * [Fix] Fix processing of the watchers
  3023. * [Fix] Fix queue id macro in milter
  3024. * [Fix] Fix R_BAD_CTE_7BIT rule
  3025. * [Fix] Fix Redis timeout set
  3026. * [Fix] Fix REPLYTO_UNPARSEABLE rule
  3027. * [Fix] Fix setting of email address
  3028. * [Fix] Fix some more issues about duplicated fuzzy requests
  3029. * [Fix] Fix spamc support in rspamd proxy
  3030. * [Fix] Fix syntax error in spamtrap plugin
  3031. * [Fix] Fix url counts for href urls
  3032. * [Fix] Fix url handling in the protocol
  3033. * [Fix] Multimap: Received IP filters with Redis
  3034. * [Fix] Oops, fix d9d0fa5e86db2f4470d34395a233b450478b2f60
  3035. * [Fix] Parse rgb[a](x,x,x[,x]) css colors
  3036. * [Fix] Phishing: strict_domains
  3037. * [Fix] Reduce maps aggressiveness
  3038. * [Fix] Reresolve upstreams even if there is a single server there
  3039. * [Fix] Rspamadm grep: Disable Lua patterns in string search by default
  3040. * [Fix] Skip text parts when checking binary parts in fuzzy check
  3041. * [Fix] Support v2 checks in controller
  3042. * [Fix] Treat empty address as valid
  3043. * [Fix] Try harder to detect CTE
  3044. * [Fix] Try to deal with v4 mapped to v6 addresses on accept
  3045. * [Fix] Use dkim signing callback properly
  3046. * [Fix] Use non-volatile memory for storing data
  3047. * [Fix] Use static maps instead of ugly hack for radix_from_config
  3048. * [Fix] Use the same pool for related sessions
  3049. * [Rework] Continue modularisation for lua library
  3050. * [Rework] Initial milter protocol support
  3051. * [Rework] Make log pipes worker agnostic, add scanners API
  3052. * [Rework] Move authentication results generation to a separate routine
  3053. * [Rework] Move common DKIM functions to a separate lua module
  3054. * [Rework] Move global functions to a separate directory
  3055. * [Rework] Prepare dkim module for ARC checks
  3056. * [Rework] Propagate ucl variables from the command line
  3057. * [Rework] Remove multiple metrics support from Rspamd
  3058. * [Rework] Stop using name 'rmilter' for the modern protocol
  3059. * [Rework] Use LFU algorithm in LRU cache
  3060. * [Rules] Fix received TLS rules
  3061. * [Rules] Improve URL_COUNT_ODD rule
  3062. * [WebUI] Fix add header filter in history
  3063. * [WebUI] Use modern protocol for checking messages
  3064. 1.5.9:
  3065. * [Conf] Increase min_bytes to avoid FP
  3066. * [Conf] Remove ratelimits from default configuration
  3067. * [CritFix] Fix accepting on IPv6 sockets
  3068. * [CritFix] Zero fill sockaddr_un
  3069. * [Feature] Add `text_multiplier` param
  3070. * [Fix] Check min_bytes for images as well
  3071. * [Fix] Do not add exact hashes from different parts
  3072. * [Fix] Fix memory leak when accepting from unix sockets
  3073. * [Fix] Fix some more issues about duplicated fuzzy requests
  3074. * [Fix] Phishing: strict_domains
  3075. * [Fix] Skip text parts when checking binary parts in fuzzy check
  3076. * [Minor] Add the same duplicates protection for all fuzzy hashes types
  3077. * [Minor] Fix braces
  3078. * [Minor] Fix test
  3079. * [Minor] SPOOF_DISPLAY_NAME: Use all SMTP/MIME recipients
  3080. * [Minor] Validate assumed spoofed display name domains to contain a dot
  3081. 1.5.8:
  3082. * [CritFix] Fix memory leak in fuzzy check
  3083. * [CritFix] Fix memory leak in maps scheduling
  3084. * [Feature] Multimap: email:domain:tld filter
  3085. * [Fix] DKIM Signing: avoid nil index when From header is missing
  3086. * [Fix] Do not set bayes probability if we don't use it
  3087. * [Fix] Do not stop on illegal unicode points - replace them
  3088. * [Fix] Fix brain-damaged spamc protocol for now
  3089. * [Fix] Fix Redis timeout set
  3090. * [Fix] Fix spamc support in rspamd proxy
  3091. * [Fix] Multimap: Received IP filters with Redis
  3092. * [Fix] Parse rgb[a](x,x,x[,x]) css colors
  3093. * [Fix] Reresolve upstreams even if there is a single server there
  3094. * [Fix] Treat empty address as valid
  3095. * [Fix] Try harder to detect CTE
  3096. * [Fix] Try to deal with v4 mapped to v6 addresses on accept
  3097. * [Minor] Add `wsf` and `hta` bad archive extensions
  3098. * [Minor] Fix configuration option
  3099. * [Minor] Fix result parsing for SAVAPI
  3100. * [Minor] Further logging improvements
  3101. * [Minor] Improve logging of errors
  3102. * [Minor] Prevent MID_CONTAINS_FROM from firing on empty address
  3103. * [Minor] Reduce digit->number transmission penalty
  3104. * [Minor] Relax CTYPE_MISSING_DISPOSITION rule
  3105. 1.5.7:
  3106. * [CritFix] Fix corruption when multiple fuzzy are defined
  3107. * [CritFix] Fix learn condition in fuzzy check
  3108. * [Feature] Add rules to detect bad 8bit characters in From and To
  3109. * [Feature] DKIM signing: sign_networks/local address specific use_domain settings
  3110. * [Feature] Support numeric arguments for Redis requests
  3111. * [Fix] Deal with 8bit characters in email addresses
  3112. * [Fix] Fix couple of issues in FORWARDED rule
  3113. * [Fix] Fix passing data to log helper when many symbols defined
  3114. * [Fix] Fix R_BAD_CTE_7BIT rule
  3115. * [Fix] Fix REPLYTO_UNPARSEABLE rule
  3116. * [Fix] Fix setting of email address
  3117. * [Fix] Rspamadm grep: Disable Lua patterns in string search by default
  3118. * [Minor] Add Lua 5.3 workaround
  3119. * [Minor] Add lua methods to detect if a part has 8bit characters
  3120. * [Minor] Allow session-less lua dns requests
  3121. * [Minor] Allow to append greylist end time to message reported
  3122. * [Minor] Avoid `nil` table
  3123. * [Minor] Disable dkim_signing if redis is specified but not configured
  3124. * [Minor] Fix build with pcre2
  3125. * [Minor] Fix rule
  3126. * [Minor] Fix warnings
  3127. * [Minor] Format floating point number
  3128. * [Minor] Push email flags to the lua API
  3129. * [Minor] Silence some warnings
  3130. * [Minor] Silence warning
  3131. * [Minor] Try all hostname regexps to find the most significant one
  3132. * [WebUI] Fix add header filter in history
  3133. 1.5.6:
  3134. * [Feature] Add unigramms support in bayes
  3135. * [Feature] Allow configurable sign headers for DKIM
  3136. * [Feature] Allow to add unigramm metatokens from Lua
  3137. * [Feature] DKIM Signing: envelope match exception for local IPs
  3138. * [Feature] UCL: register parser variables from Lua
  3139. * [Fix] Always try to adjust filename
  3140. * [Fix] Do extra copy to ensure that original content is never touched
  3141. * [Fix] Fix SPOOF_REPLYTO rule
  3142. * [Fix] Ignore Rmilter added Received
  3143. * [Fix] More fixes for hashed email dnsbls
  3144. * [Fix] Plug memory leak in chartable module
  3145. * [WebUI] Display multiple alerts at once
  3146. 1.5.5:
  3147. * [CritFix] Fix classifier learning with Redis backend
  3148. * [CritFix] Fix issue when parsing encoded rfc822/messages
  3149. * [Feature] Add escaped version of lua_ucl import
  3150. * [Feature] Add task:headers_foreach function
  3151. * [Feature] Allow to process filenames from content type
  3152. * [Feature] Allow to query hashed emails
  3153. * [Feature] Ignore bayes with mostly metatokens or with too few text
  3154. * [Feature] Probabilistically skip metatokens
  3155. * [Feature] Retrieve all virus names from SAVAPI
  3156. * [Feature] Rework classifiers lua metatokens
  3157. * [Feature] Store headers order
  3158. * [Feature] Store text tokens inside bayes tokens
  3159. * [Feature] Use cached shingles keys
  3160. * [Fix] Add missing score normalisation for HFILTER_URL_ONLY
  3161. * [Fix] Avoid lookup in absent hash
  3162. * [Fix] Check return values from Lua functions called from C
  3163. * [Fix] Do not count sending and loading time in rspamc
  3164. * [Fix] Escape json strings for controller rejplies from Lua
  3165. * [Fix] Fix archive scans for savapi
  3166. * [Fix] Fix domain_only emails RBL
  3167. * [Fix] Fix ip_score map configuration
  3168. * [Fix] Fix JSON output for history_redis
  3169. * [Fix] Fix one character length substrings search
  3170. * [Fix] Fix parsing of non-RFC compatible Exim received
  3171. * [Fix] Fix parsing of options for workers with the same type
  3172. * [Fix] Fix processing of small tokens vectors
  3173. * [Fix] Fix rfc2047 tokenization
  3174. * [Fix] Fix typo
  3175. * [Fix] More fixes for inplace decoding
  3176. * [Fix] Try to avoid modifications of the original data
  3177. * [Fix] URL redirector: Fix call to is_redirector
  3178. * [Rework] Set token data as uint64_t instead of chars array
  3179. * [WebUI] Check if neighbours' history backend versions match
  3180. * [WebUI] Disable phrase connectors replacement in history filtering
  3181. * [WebUI] Disable phrase connectors replacement in symbols filtering
  3182. * [WebUI] Do not hide messages with bad subject, just replace it with '???'
  3183. * [WebUI] Fix error message
  3184. * [WebUI] Fix history v2 display
  3185. * [WebUI] Fix legacy history
  3186. * [WebUI] history: break To address lists on commas
  3187. * [WebUI] Increase default timeout to 20 seconds
  3188. * [WebUI] Save some history table space
  3189. 1.5.4:
  3190. * [Conf] Add history_redis default configuration
  3191. * [Feature] Add spoofed rules
  3192. * [Feature] Add URL_IN_SUBJECT rule
  3193. * [Feature] Allow to get task's subject
  3194. * [Feature] Allow to specify maximum number of shots for symbols
  3195. * [Feature] Distinguish URLs found in Subject
  3196. * [Feature] Memoize LPEG grammars
  3197. * [Feature] Parse else parts in SA rules
  3198. * [Feature] Process subject for mixed characters
  3199. * [Feature] Resolve url chains in url_redirector module
  3200. * [Feature] Stat greylisted messages as greylisted not soft-rejected
  3201. * [Feature] Support checking for redirector in Lua SURBL
  3202. * [Feature] Support tag_exists SA function
  3203. * [Feature] Work with broken rfc2047 tokens
  3204. * [Fix] Check all watcher's dependencies
  3205. * [Fix] Do not compile hyperscan with no SSSE3 support
  3206. * [Fix] Do not crash if cannot decode qp encoded part
  3207. * [Fix] Fix dependencies of DKIM when multiple signatures are found
  3208. * [Fix] Fix lists in whitelist plugin
  3209. * [Fix] Fix one-shot symbols weight calculations
  3210. * [Fix] Fix options and shots match
  3211. * [Fix] Fix order of symbol options
  3212. * [Fix] Fix parsing of dot at the end of the address
  3213. * [Fix] Fix parsing of lua table arguments
  3214. * [Fix] Fix processing of subject words
  3215. * [Fix] Fix string split memoization
  3216. * [Fix] Fix templates grammar usage
  3217. * [Fix] Fix various issues related to Lua stack manipulation
  3218. * [Fix] Force actions: Use postfilter if we have honor_action / require_action
  3219. * [Fix] Further fixes to avoid PHISHING FP
  3220. * [Fix] Preserve order of options in symbols
  3221. * [Fix] Rspamadm grep: deal with unusually-formatted logs
  3222. * [Fix] Use hostname suffix when dealing with history
  3223. * [Rework] Remove outdated SA rules
  3224. * [WebUI] Add flexible columns
  3225. * [WebUI] Add footable
  3226. * [WebUI] Add sender, recipients and subject columns
  3227. * [WebUI] Allow message-id break
  3228. * [WebUI] Fix history clustering
  3229. * [WebUI] Fix history display
  3230. * [WebUI] Fix sorting
  3231. * [WebUI] Humanize sizes
  3232. * [WebUI] Initial move towards footable
  3233. * [WebUI] Remove datatables
  3234. * [WebUI] Replace `.values` method with `.map`
  3235. * [WebUI] Rework v2 symbols display
  3236. * [WebUI] Try to normalize frequencies
  3237. * [WebUI] Unbreak WebUI
  3238. * [WebUI] Use Footable to draw Throughput summary table
  3239. 1.5.3:
  3240. * [Conf] Add composite for hacked wordpress phishing
  3241. * [CritFix] Fix base64 decoding when there are unparseable characters
  3242. * [Feature] Additional symbol metadata in metadata exporter
  3243. * [Feature] Add method to get protocol reply from Lua
  3244. * [Feature] Add symbols when tagged rcpt/sender are normalised
  3245. * [Feature] Add task:get_symbols_all() function
  3246. * [Feature] Allow multiple formats of DKIM signing key
  3247. * [Feature] Allow to cache and use flexible protocol reply
  3248. * [Feature] Allow to set one_shot flag from register_symbol
  3249. * [Feature] Allow to skip certain types of hashes when learning fuzzy
  3250. * [Feature] Cache and insert scan time into the protocol
  3251. * [Feature] Detect newlines in rspamc --mime
  3252. * [Feature] DKIM signing: support use of maps
  3253. * [Feature] Greylist: Support excluding low-scoring messages from greylisting
  3254. * [Feature] Implement lua history in controller
  3255. * [Feature] Implement redis history querying
  3256. * [Feature] Preliminary implementation of redis history plugin
  3257. * [Feature] Support using request headers in settings
  3258. * [Fix] Change default template to deal with non-ASCII characters
  3259. * [Fix] Deal with lists of maps in whitelist module
  3260. * [Fix] DKIM signing: use domain-specific signing key
  3261. * [Fix] Do not reallocate completed zstd buffer
  3262. * [Fix] Do not use local_addrs in proxy
  3263. * [Fix] Fix crash when resolver is undefined
  3264. * [Fix] Fix double free when closing lua_tcp connections
  3265. * [Fix] Fix for lua 5.3
  3266. * [Fix] Fix freeing of arrays iterators
  3267. * [Fix] Fix issue with task:get_symbol and symbols with no metric
  3268. * [Fix] Fix log line duplication in `rspamadm grep`
  3269. * [Fix] Fix memory corruption on termination
  3270. * [Fix] Fix out-of-bound access in base64 decode
  3271. * [Fix] Fix ratelimit + greylisting
  3272. * [Fix] Fix subject rewriting
  3273. * [Fix] Fix task:set_recipients function
  3274. * [Fix] Fix URI_COUNT_ODD rule
  3275. * [Fix] Follow the traditional symbols conventions in RCPT_COUNT rule
  3276. * [Fix] Greylist: Suppress greylist action for whitelisted hosts too
  3277. * [Fix] Metadata exporter: use rule-specific settings for emails
  3278. * [Fix] Properly set missing fields in exporter
  3279. * [Fix] Proxy: max_retries option
  3280. * [Fix] RCPT_COUNT fixes
  3281. * [Fix] Rework HAS_X_PRIO rule to match symbols conventions
  3282. * [Fix] Update issues in ac-trie
  3283. * [Fix] Use optimised base64 decoding in DKIM
  3284. * [WebUI] Add preliminary v2 history parser
  3285. * [WebUI] Allow different history parsers
  3286. * [WebUI] Display symbols
  3287. * [WebUI] Rework history v2 function
  3288. 1.5.2:
  3289. * [Conf] Add default config for spamassasssin plugin
  3290. * [Conf] Add default configuration for antivirus module
  3291. * [Conf] Add dkim signing docs
  3292. * [Conf] Add mx_check default config
  3293. * [Conf] Add replies config
  3294. * [Conf] Add trie default config
  3295. * [Feature] Add heuristic to find text parts in files
  3296. * [Feature] Add rule to detect broken content type
  3297. * [Feature] Allow to extract CTE in Lua API
  3298. * [Feature] Allow to set from address for a lua_task
  3299. * [Feature] Allow to set recipients of a task from Lua
  3300. * [Feature] Enchance text_part:get_content method
  3301. * [Feature] Remove + aliases from emails
  3302. * [Feature] Support rmilter block and dkim signature in CGP helper
  3303. * [Feature] Support running event loop from Lua
  3304. * [Fix] Antivirus: use scanner-specific redis prefix
  3305. * [Fix] Couple of fixes for DKIM signing module
  3306. * [Fix] Distinguish missing and broken mandatory headers
  3307. * [Fix] Do more heuristical detection for missing CTE
  3308. * [Fix] Do not resort cache on each check
  3309. * [Fix] Fix CGP escaping
  3310. * [Fix] Fix MISSING_MIME_VERSION rule for plain messages
  3311. * [Fix] Fix parsing of cte in expressions
  3312. * [Fix] Fix partial matches in rspamadm grep
  3313. * [Fix] Fix setting class on style field
  3314. * [WebUI] Auto-switch Throughput units to `msg/min` for very low rate
  3315. * [WebUI] Update D3Evolution to 0.0.2
  3316. 1.5.1:
  3317. * [CritFix] Fix processing of stop_patterns with `\0` character
  3318. * [CritFix] Fix setting of raw key for signing
  3319. * [Fix] Fix lua exports from plugins during reload
  3320. * [Fix] Fix prefilters action scores
  3321. * [Fix] Fix symbols processing order
  3322. * [Minor] Help cmake find gthread
  3323. * [Minor] Some cmake fixes
  3324. 1.5.0:
  3325. * [Conf] Add configurations for asn, clickhouse and dcc
  3326. * [Conf] Add default config for url redirector plugin
  3327. * [Conf] Add the default config for greylist module
  3328. * [Conf] Allow to edit all local maps from WebUI by default
  3329. * [CritFix] Deal with absent headers in DKIM
  3330. * [CritFix] Do not trust remote shingles count
  3331. * [CritFix] Fix bad memory leak in TLS certificates validation
  3332. * [CritFix] Fix critical memory issues with radix maps
  3333. * [CritFix] Fix descriptors leak on reload
  3334. * [CritFix] Fix headers selection in DKIM verification
  3335. * [CritFix] Fix parsing of boundaries that end with `--`
  3336. * [CritFix] Repair PTR_ARRAY_FOREACH macro
  3337. * [Feature] Add CORS support to the controller
  3338. * [Feature] Add FROM_NAME_EXCESS_SPACE rule
  3339. * [Feature] Add REPLYTO_EMAIL_HAS_TITLE rule
  3340. * [Feature] Add `caseless_hash` method to `lua_util`
  3341. * [Feature] Add `rip` keyword to ratelimit module
  3342. * [Feature] Add a simple benchmark for content type parsing
  3343. * [Feature] Add boundaries parsing in content type
  3344. * [Feature] Add charset detection for text parts
  3345. * [Feature] Add content disposition parser
  3346. * [Feature] Add fallback if too many updates are failing
  3347. * [Feature] Add function to convert struct tm to time using timezone
  3348. * [Feature] Add function to normalize HTTP paths
  3349. * [Feature] Add fuzzy collection plugin
  3350. * [Feature] Add fuzzy logic for images
  3351. * [Feature] Add gmime parser to mime_tool
  3352. * [Feature] Add heuristic to detect broken messages
  3353. * [Feature] Add heuristic to find displayed URLs
  3354. * [Feature] Add heuristic to process broken email addresses
  3355. * [Feature] Add images normalization
  3356. * [Feature] Add mechanism for disabling composites (Fixes #1270)
  3357. * [Feature] Add method to create regexp from a glob pattern
  3358. * [Feature] Add mime encoding manipulation routines
  3359. * [Feature] Add mime tool to explore messages
  3360. * [Feature] Add more meta tokens from received headers
  3361. * [Feature] Add neighbours option to support Rspamd cluster in WebUI
  3362. * [Feature] Add new function to parse mime addresses
  3363. * [Feature] Add new methods for lua_tcp
  3364. * [Feature] Add own headers decoding routine
  3365. * [Feature] Add own routine to generate a message id
  3366. * [Feature] Add parser for SMTP date
  3367. * [Feature] Add per-task lua cache to reuse 'heavy' objects
  3368. * [Feature] Add plugins list path in WebUI
  3369. * [Feature] Add preliminary multipart support
  3370. * [Feature] Add preliminary version of DKIM signing module
  3371. * [Feature] Add profiling support in client output
  3372. * [Feature] Add rfc2047 grammar
  3373. * [Feature] Add rfc2047 variant for QP decoding
  3374. * [Feature] Add rmilter_headers module (Fixes #1227)
  3375. * [Feature] Add sse42 version of base64 decoding
  3376. * [Feature] Add ssse3 and avx2 base64 decoders
  3377. * [Feature] Add support of libgd
  3378. * [Feature] Add the preliminary version of redirects resolver in Lua
  3379. * [Feature] Add ucl_object_iterate_full function
  3380. * [Feature] Add url encoding function
  3381. * [Feature] Allow SOA requests in lua dns
  3382. * [Feature] Allow custom parse types in lua ucl
  3383. * [Feature] Allow plugins to register webui handlers
  3384. * [Feature] Allow to add options explicitly to symbols
  3385. * [Feature] Allow to call a callback when symbol frequency is on peak
  3386. * [Feature] Allow to call redirector script from SURBL
  3387. * [Feature] Allow to create variable length dkim keys
  3388. * [Feature] Allow to have module specific options for Redis in plugins
  3389. * [Feature] Allow to pass sign key directly from Lua
  3390. * [Feature] Allow to register configuration docs from Lua API
  3391. * [Feature] Allow to return options as a table
  3392. * [Feature] Allow to set peak callbacks from Lua
  3393. * [Feature] Allow to specify custom method for a message
  3394. * [Feature] Allow to store dkim keys in Redis
  3395. * [Feature] Allow to store messages in files
  3396. * [Feature] Apply DCT using AAN for fuzzy signature
  3397. * [Feature] Avira SAVAPI support
  3398. * [Feature] Cache and simplify DCT and jpeg decode
  3399. * [Feature] Cache libicu converters
  3400. * [Feature] Detect URLs with suspicious omographs
  3401. * [Feature] Do not increase score for duplicate options
  3402. * [Feature] Do not trust CTE, check base64 and qp strictly
  3403. * [Feature] Dynamic reputation in URL reputation plugin
  3404. * [Feature] Extend redis lock when learning spawned
  3405. * [Feature] Filter non-utf chars from all decoded headers
  3406. * [Feature] Fix phishing detection for IDNA urls
  3407. * [Feature] Ignore bad symbols on base64 decoding
  3408. * [Feature] Ignore too wide elements in SPF
  3409. * [Feature] Implement fuzzy collection mode
  3410. * [Feature] Implement helo maps in multimap
  3411. * [Feature] Implement human readable buckets configuration
  3412. * [Feature] Implement min-hash shingles for DCT data from images
  3413. * [Feature] Implement new algorithm for fuzzy hashes of images
  3414. * [Feature] Implement new unicode normalizer
  3415. * [Feature] Implement quoted printable decoding
  3416. * [Feature] Implement received headers flags
  3417. * [Feature] Implement rspamdgrep tool
  3418. * [Feature] Implement sane checksum for config file
  3419. * [Feature] Implement url tags concept
  3420. * [Feature] Improve detection of omographs using libicu
  3421. * [Feature] Improve url redirector module
  3422. * [Feature] Multimap: Received header processing
  3423. * [Feature] Multiple improvements in the maps
  3424. * [Feature] New URL filters in multimap
  3425. * [Feature] Plugin to force actions on selected symbols
  3426. * [Feature] RBL module: support hashing for emails and helo RBL
  3427. * [Feature] Reuse URL tags in SURBL module
  3428. * [Feature] Rework RRD ds count, add conversion path
  3429. * [Feature] Rework surbl module to avoid extra redirector calls
  3430. * [Feature] Send config id to the WebUI
  3431. * [Feature] Simplify HTTPCrypt client support
  3432. * [Feature] Skip processing for large images
  3433. * [Feature] Start collection only mode implementation for fuzzy storage
  3434. * [Feature] Start import of the optimized base64 decode
  3435. * [Feature] Store all received headers in lua
  3436. * [Feature] Store relational order of all headers in a message
  3437. * [Feature] Support DKIM signing in Lua plugins
  3438. * [Feature] Support HTTPCrypt client in lua_http
  3439. * [Feature] Support setting SMTP message in multimap
  3440. * [Feature] Support setting metric subject from Lua
  3441. * [Feature] Support setting subject in force actions module
  3442. * [Feature] Treat v6 mapped addresses as v4 addresses
  3443. * [Feature] URL reputation plugin
  3444. * [Feature] Use Redis instead of memcached in URLs redirector
  3445. * [Feature] Use Rspamd rfc2047 decoder instead of gmime one
  3446. * [Feature] Use a different normalization for fuzzy images
  3447. * [Feature] Use normalized images in fuzzy hashes
  3448. * [Feature] Use own code for parsing of date
  3449. * [Feature] Use shingles for images fuzzying
  3450. * [Feature] Use t1ha for hashes, allow inlining
  3451. * [Feature] Use t1ha instead of metrohash and xxhash32
  3452. * [Feature] Various new features in metadata exporter module
  3453. * [Feature] rmilter_headers: authentication-results (#78)
  3454. * [Fix] Add additional check to mark redis connection inactive
  3455. * [Fix] Add packed attribute for protocol structure
  3456. * [Fix] Adopt OMOGRAPH_URL rule
  3457. * [Fix] Allow static maps
  3458. * [Fix] Allow to disable classifiers checks using settings and conditions
  3459. * [Fix] Another try to fix 0 length maps
  3460. * [Fix] Another try to fix corruption during maps reload
  3461. * [Fix] Another try to fix descriptors leak
  3462. * [Fix] Another try to fix reload and logger
  3463. * [Fix] Antivirus module: register virtual symbols for patterns
  3464. * [Fix] Avoid extensive reallocs
  3465. * [Fix] Avoid mempool leak in SA plugin on reload
  3466. * [Fix] Avoid race condition on saving cache and reload
  3467. * [Fix] Avoid reusing g_error (Fixes #1262)
  3468. * [Fix] Break pool connection on fatal redis errors
  3469. * [Fix] Check for NaN properly
  3470. * [Fix] Couple of fixes for date parsing
  3471. * [Fix] Date header timezone adjustments (#1279)
  3472. * [Fix] Deal with EOF properly
  3473. * [Fix] Decode filename in content disposition
  3474. * [Fix] Disable fuzzy images by default
  3475. * [Fix] Disable zero-copy mode for text parts to avoid crashes
  3476. * [Fix] Do not destroy session when not all finish scripts are done
  3477. * [Fix] Do not greyscale images
  3478. * [Fix] Do not leave parent-less workers processes on fatal errors
  3479. * [Fix] Do not lowercase Content-Disposition to perform decoding
  3480. * [Fix] Do not penalize characters just after numeric prefix
  3481. * [Fix] Do not refork workers that are intended to die
  3482. * [Fix] Do not set pre-result and update records for no `Queue-ID` messages
  3483. * [Fix] Do not skip post-filters when pre-filters have set some results
  3484. * [Fix] Do not stop symbols planning if async events are pending
  3485. * [Fix] Do not try to set keys for unencrypted requests in proxy
  3486. * [Fix] Encode URLs according to rfc3986
  3487. * [Fix] Encode URLs before sending them to the protocol
  3488. * [Fix] Filter bad characters from message id
  3489. * [Fix] Fix CTE detection heuristic
  3490. * [Fix] Fix Content-Type in HTTP requests
  3491. * [Fix] Fix IDN eslds phishing checks
  3492. * [Fix] Fix adding maps from config in Lua
  3493. * [Fix] Fix another reload memory issue
  3494. * [Fix] Fix argument returned on redis backend errors
  3495. * [Fix] Fix assertion in graph handling
  3496. * [Fix] Fix body trie matching
  3497. * [Fix] Fix build
  3498. * [Fix] Fix byte array expansion during toutf8 conversion
  3499. * [Fix] Fix charset normalisation
  3500. * [Fix] Fix checking of DKIM bodies that needs just `\n` to be added
  3501. * [Fix] Fix couple of cornercases with email addresses
  3502. * [Fix] Fix couple of issues
  3503. * [Fix] Fix dependencies tracking for callback symbols
  3504. * [Fix] Fix detection of jpeg size
  3505. * [Fix] Fix errors handling in fuzzy backend initialization
  3506. * [Fix] Fix fuzzy hashes count
  3507. * [Fix] Fix globbing and convert lists to arrays in fuzzy_check
  3508. * [Fix] Fix heuristical CTE detection for QP encoding
  3509. * [Fix] Fix ignoring of bad text parts
  3510. * [Fix] Fix indexes in array access, interleave loop
  3511. * [Fix] Fix int64 -> double conversion
  3512. * [Fix] Fix invalid memory access on reload
  3513. * [Fix] Fix issues with empty updates
  3514. * [Fix] Fix issues with quoted-printable encoding
  3515. * [Fix] Fix keys names
  3516. * [Fix] Fix lots of issues in mime parser code
  3517. * [Fix] Fix lua maps load
  3518. * [Fix] Fix macro name
  3519. * [Fix] Fix mas group score calculations
  3520. * [Fix] Fix matching of the same patterns from different tries
  3521. * [Fix] Fix memory corruprtion and leak
  3522. * [Fix] Fix memory leak in HTTP maps
  3523. * [Fix] Fix memory leak in expression destroying
  3524. * [Fix] Fix memory leak in parsing of mime names
  3525. * [Fix] Fix memory leak in safe ucl iterators
  3526. * [Fix] Fix memory leak on reload in plugins
  3527. * [Fix] Fix modules reconfigure on reload
  3528. * [Fix] Fix monitored setup fro URLBLs with IP addresses
  3529. * [Fix] Fix name of var
  3530. * [Fix] Fix new rrd updates
  3531. * [Fix] Fix out of bounds access
  3532. * [Fix] Fix parsing messages with no body
  3533. * [Fix] Fix parsing of '=' character in headers
  3534. * [Fix] Fix parsing of messages with no content type
  3535. * [Fix] Fix plugins callbacks in webui
  3536. * [Fix] Fix possible memory corruption in redis pool
  3537. * [Fix] Fix probability calculations for fuzzy redis backend
  3538. * [Fix] Fix processing errors in lua_tcp
  3539. * [Fix] Fix processing of emails with name only
  3540. * [Fix] Fix processing of non-multipart messages
  3541. * [Fix] Fix processing of parts with no valid content type
  3542. * [Fix] Fix race condition in SIGUSR2 handler
  3543. * [Fix] Fix redis options parsing when no redis servers are defined
  3544. * [Fix] Fix reload and hyperscan ready event
  3545. * [Fix] Fix reload memory issue
  3546. * [Fix] Fix rra_ptr conversion
  3547. * [Fix] Fix rrd file conversion
  3548. * [Fix] Fix setting of content-type attributes
  3549. * [Fix] Fix signing headers creation in DKIM
  3550. * [Fix] Fix stddev calculations
  3551. * [Fix] Fix surbl plugin to work with composite maps
  3552. * [Fix] Fix timezones parsing
  3553. * [Fix] Fix tokens usage
  3554. * [Fix] Fix urls and emails hashes
  3555. * [Fix] Fix usage of unsafe ucl iterators
  3556. * [Fix] Fix work with broken utf8 tokens
  3557. * [Fix] Fix writing of user to roll history
  3558. * [Fix] Forgotten worker
  3559. * [Fix] Further memory leaks fixes
  3560. * [Fix] Ignore lua metatokens in bayes for now
  3561. * [Fix] Improve OMOGRAPH_URL rule
  3562. * [Fix] Lua IP from string should be invalid if parsing failed
  3563. * [Fix] Miltiple fixes to new lua_tcp, add debugging
  3564. * [Fix] More fixes for iterators cleanup
  3565. * [Fix] More fixes to logger initialization
  3566. * [Fix] More heuristic fixes for phishing detection
  3567. * [Fix] More leaks eliminated
  3568. * [Fix] More leaks...
  3569. * [Fix] More random fixes for reload...
  3570. * [Fix] Multimap: Fixes for email filters
  3571. * [Fix] Multiple fixes for fann module
  3572. * [Fix] Multiple memory corruption fixes
  3573. * [Fix] Normalize path in HTTP router
  3574. * [Fix] Plug memory leak
  3575. * [Fix] Plug memory leak in adding radix trees
  3576. * [Fix] Plug memory leak in configuration parser
  3577. * [Fix] Plug memory leak in expressions parsing during reload
  3578. * [Fix] Plug memory leak in learning fuzzy storage
  3579. * [Fix] Plug memory leak in lua_tcp
  3580. * [Fix] Plug reload leaks
  3581. * [Fix] Plug termination memory leaks
  3582. * [Fix] Really increase lock lifetime
  3583. * [Fix] Replies module: fix symbol weight
  3584. * [Fix] Restore content type params related functions
  3585. * [Fix] Set task's subject from mime subject
  3586. * [Fix] Sigh, one more reload leak
  3587. * [Fix] Simplify images shingles
  3588. * [Fix] Some more memory issues are fixed
  3589. * [Fix] Stop hardcoding of lua in C
  3590. * [Fix] Stop processing of bad parts as text parts
  3591. * [Fix] Strictly filter bad characters when emittin json
  3592. * [Fix] Strings returned from lua are ephemeral
  3593. * [Fix] Support unix sockets for lua redis
  3594. * [Fix] Try to fix issues with reloading config
  3595. * [Fix] Try to fix race condition in redis_pool
  3596. * [Fix] Use checksum to avoid intersection between different ANNs
  3597. * [Fix] Use rspamd hashes in embedded ucl
  3598. * [Fix] Use sane default rewrite subject (*** SPAM *** %s)
  3599. * [Fix] Various collection mode fixes
  3600. * [Fix] Various fixes to mime parser
  3601. * [Fix] Various reload leak fixing
  3602. * [Fix] Whitelist certain extensions from archive checks
  3603. * [Rework] Add preliminary implementation of the mime parser
  3604. * [Rework] Adopt code for the new options
  3605. * [Rework] Change logger setup interface
  3606. * [Rework] Composite configuration (#1270)
  3607. * [Rework] Finally remove gmime dependency from Rspamd
  3608. * [Rework] Further fixes to symbols frequencies
  3609. * [Rework] Implement content type parser for mime
  3610. * [Rework] Kill all InternetAddressList usages
  3611. * [Rework] Multiple fixes for symbols cache statistics
  3612. * [Rework] Refactor struct names
  3613. * [Rework] Rework images fuzzy hashes algorithm
  3614. * [Rework] Rework lua_tcp to allow TCP dialog
  3615. * [Rework] Start massive rework to get rid of gmime
  3616. * [Rework] Start new approach for multiparts parsing
  3617. * [Rework] Start rework of mime addresses
  3618. * [Rework] Start rework of symbols cache updates
  3619. * [Rework] Start switching to libicu
  3620. * [Rework] Use a special structure for stats tokens
  3621. * [Rework] Use hash tables for symbols options
  3622. * [Rework] Use libicu instead of iconv for conversions
  3623. * [Rework] Use new scheme to parse mime parts
  3624. * [WebUI] Add Access-Control-Allow-Origin for cluster management
  3625. * [WebUI] Add Throughput graph autorefreshing (#820)
  3626. * [WebUI] Add Visibility.js library
  3627. * [WebUI] Add basic cluster support to Throughput tab
  3628. * [WebUI] Add graph legend entries for new DSes
  3629. * [WebUI] Add graph tab
  3630. * [WebUI] Add neighbours RRD data consolidation
  3631. * [WebUI] Add preliminary save symbols clustering
  3632. * [WebUI] Add server selector to navbar
  3633. * [WebUI] Add soft reject to auth stats
  3634. * [WebUI] Add summary to the Throughput tab
  3635. * [WebUI] Allow to save maps on the cluster
  3636. * [WebUI] Avoid extra graph redraw and alerts glitching
  3637. * [WebUI] Be more generous with AJAX timeout
  3638. * [WebUI] Disable error ring loading in `read only` mode
  3639. * [WebUI] Enclose table header cells with `tr`s
  3640. * [WebUI] Finish interface rework
  3641. * [WebUI] Fix RRD summary pie chart position
  3642. * [WebUI] Fix `All SERVERS` graph fot just one available server
  3643. * [WebUI] Fix case when no cluster is defined
  3644. * [WebUI] Fix compatibility with non-ES6 compliant browsers
  3645. * [WebUI] Fix config ID
  3646. * [WebUI] Fix configuration page partially
  3647. * [WebUI] Fix disabled state
  3648. * [WebUI] Fix graph dataset selector initialization
  3649. * [WebUI] Fix graph selectors state resetting
  3650. * [WebUI] Fix mouse events on throughput summary table area
  3651. * [WebUI] Fix multiple JS issues
  3652. * [WebUI] Fix pie chart displaying
  3653. * [WebUI] Fix read only
  3654. * [WebUI] Fix read only2
  3655. * [WebUI] Fix retarded datatables
  3656. * [WebUI] Fix soft reject in pie chart
  3657. * [WebUI] Fix stat widgets timers multiplication on `Refresh` click
  3658. * [WebUI] Fix symbols config
  3659. * [WebUI] Fix various errors with login form
  3660. * [WebUI] Further fixes
  3661. * [WebUI] Hide learning tab in read-only mode
  3662. * [WebUI] Initial clusters support
  3663. * [WebUI] Make legend entry colours more contrast
  3664. * [WebUI] Move configuration tab to a separate module
  3665. * [WebUI] Move history tab
  3666. * [WebUI] Move symbols config as well
  3667. * [WebUI] New sec to time function
  3668. * [WebUI] Prevent multiple clicks on `Refresh`
  3669. * [WebUI] RRD summary: Hide inner labels of tiny pie sectors
  3670. * [WebUI] RRD summary: Respect undefined values
  3671. * [WebUI] Reduce font size of graph's legend
  3672. * [WebUI] Remove orphaned font duplicates
  3673. * [WebUI] Remove unused code
  3674. * [WebUI] Replace spinner with animated glyphicon
  3675. * [WebUI] Reset refresh timer on server switching
  3676. * [WebUI] Rework interface to use requirejs
  3677. * [WebUI] Rework neighbours query function
  3678. * [WebUI] Separate attributes by space
  3679. * [WebUI] Set focus to password field (#1230)
  3680. * [WebUI] Simplify neighbours table populating
  3681. * [WebUI] Start rework of modules
  3682. * [WebUI] Stop stats refreshing if the page is hidden
  3683. * [WebUI] Turn d3pie's stuff into a reusable function,
  3684. * [WebUI] Unify send data functions
  3685. * [WebUI] Update D3Evolution to 0.0.1
  3686. * [WebUI] Update d3.js
  3687. * [WebUI] Update datatables to work with the requirejs
  3688. * [WebUI] Use unified tab click event handler,
  3689. * [WebUI] clusters for the chart
  3690. * [WebUI] fix uptime
  3691. 1.4.2:
  3692. * [CritFix] Deal with absent headers in DKIM
  3693. * [CritFix] Do not trust remote shingles count
  3694. * [CritFix] Fix headers selection in DKIM verification
  3695. * [Feature] Add EXT_CSS rule
  3696. * [Feature] Add toggle for disabling SURBLs
  3697. * [Feature] Extend redis lock when learning spawned
  3698. * [Feature] Parse <link> HTML tags
  3699. * [Fix] Avoid reusing g_error (Fixes #1262)
  3700. * [Fix] Do not reset loaded ANN when learning is requested
  3701. * [Fix] Fix another issue with external deps in SA
  3702. * [Fix] Fix body trie matching
  3703. * [Fix] Fix checking of DKIM bodies that needs just `\n` to be added
  3704. * [Fix] Fix fuzzy hashes count
  3705. * [Fix] Fix keys names
  3706. * [Fix] Fix length calculations for url encoded urls
  3707. * [Fix] Fix matching of the same patterns from different tries
  3708. * [Fix] Fix name of var
  3709. * [Fix] Fix parsing of URLs with spaces and other bad chars
  3710. * [Fix] Fix probability calculations for fuzzy redis backend
  3711. * [Fix] Fix signing headers creation in DKIM
  3712. * [Fix] Plug memory leak
  3713. * [Fix] Really fix chained SA dependencies
  3714. * [Fix] Really increase lock lifetime
  3715. * [Fix] Use checksum to avoid intersection between different ANNs
  3716. * [Fix] Use rspamd hashes in embedded ucl
  3717. * [Fix] Yet another change for testing external deps
  3718. 1.4.1:
  3719. * [Feature] ASN support in Clickhouse module
  3720. * [Feature] Add clickhouse plugin
  3721. * [Feature] Add generic tool to add universal maps for lua modules
  3722. * [Feature] Add logger.debugm to debug lua modules
  3723. * [Feature] Allow to register metrics symbols using register_symbol
  3724. * [Feature] Allow to specify prefix for fann_redis
  3725. * [Feature] Clickhouse: support different masks for IPv4/IPv6
  3726. * [Feature] Support forcing action in antivirus plugin
  3727. * [Fix] Add handling of regexp maps
  3728. * [Fix] Allow backslashes in http urls
  3729. * [Fix] Avoid mapping of empty files
  3730. * [Fix] Do not load tld file to speed up rspamadm
  3731. * [Fix] Do not resolve numeric IP addresses due to ipv6 insanity
  3732. * [Fix] Filter incorrect training data
  3733. * [Fix] Fix Fuzzyconvert tool when password or DB is given
  3734. * [Fix] Fix build with custom glib/gmime
  3735. * [Fix] Fix converting of learn count from sqlite to redis
  3736. * [Fix] Fix crashes with invalid received and task:set_from_ip
  3737. * [Fix] Fix external dependencies for SA module
  3738. * [Fix] Fix fann_redis when number of scores has been changed
  3739. * [Fix] Fix hyperscan usage for non compatible platforms
  3740. * [Fix] Fix loading of maps from UCL objects
  3741. * [Fix] Fix memory leak for task-less redis requests
  3742. * [Fix] Fix mid module with new maps syntax
  3743. * [Fix] Fix parsing of URLs with username
  3744. * [Fix] Fix re cache initialisation
  3745. * [Fix] Fix replacements to sanitize '%' character
  3746. * [Fix] Fix set and regexp like static maps
  3747. * [Fix] Fix some issues in redis settings
  3748. * [Fix] Fix static IP maps
  3749. * [Fix] Fix total learns counter for redis stats
  3750. * [Fix] Fix usage of config during reload
  3751. * [Fix] Fix various warnings and issues
  3752. * [Fix] Invalidate ANN if training data is incorrect
  3753. * [Fix] Miltiple fixes to fann_redis module
  3754. * [Fix] More fixes for URLs with backslashes
  3755. * [Fix] Properly get options for ip_score module
  3756. * [Fix] Relax requirements for Received as gmail cannot RFC
  3757. * [Fix] Remove or fix hyperscan incompatible regexps
  3758. * [Fix] Settings: correctly read redis config
  3759. * [Rework] Rework lua logger interface slightly
  3760. * [Rework] Use new maps add function
  3761. 1.4.0:
  3762. * [CritFix] Add guards for inactive redis connections
  3763. * [CritFix] Another fix for proxying files using rspamd_proxy
  3764. * [CritFix] Cleanup inactive redis connections
  3765. * [CritFix] Do not sometimes try to exec posfilters before classification
  3766. * [CritFix] Fix application of IPv6 mask
  3767. * [CritFix] Fix chunked encoding when reading messages
  3768. * [CritFix] Fix file mode for rspamd_proxy
  3769. * [CritFix] Fix hyperscan compilation on regexp change
  3770. * [CritFix] Fix issue with finding of end of lines pointers
  3771. * [CritFix] Fix iteration over headers array (introduced in 1.4)
  3772. * [CritFix] Fix processing of learned tokens count for redis backend
  3773. * [CritFix] Fix race condition in checking of cached maps
  3774. * [CritFix] Fix workers scripts by sharing workers configs
  3775. * [CritFix] Introduce raw content to text parts
  3776. * [CritFix] Plug memory leak and potential memory corruption
  3777. * [Feature] Adaptive ratelimits
  3778. * [Feature] Add ASN -> rbldnsd script for asn.rspamd.com
  3779. * [Feature] Add DMARC_NA symbol
  3780. * [Feature] Add F-Prot support to antivirus module
  3781. * [Feature] Add HTTP backend to metadata exporter
  3782. * [Feature] Add Lua API module for monitored objects
  3783. * [Feature] Add R_DKIM_NA / R_SPF_NA / AUTH_NA symbols
  3784. * [Feature] Add R_DKIM_PERMFAIL symbol
  3785. * [Feature] Add R_SPF_PERMFAIL symbol
  3786. * [Feature] Add Sophos antivirus support
  3787. * [Feature] Add ZSTD compression to Lua API
  3788. * [Feature] Add `mid` Lua module
  3789. * [Feature] Add `one_param` flag for metric symbols
  3790. * [Feature] Add a generic lua classifier
  3791. * [Feature] Add a very basic interface to access workers data from on_load
  3792. * [Feature] Add ability to delete a hash by its data to fuzzy_check plugin
  3793. * [Feature] Add ability to enable/disable symbols via dynamic_conf
  3794. * [Feature] Add ability to lookup settings by key
  3795. * [Feature] Add common way to disable Lua modules
  3796. * [Feature] Add compression support to rspamd client
  3797. * [Feature] Add condition to do antiviral check
  3798. * [Feature] Add configuration for lua classifiers
  3799. * [Feature] Add configuration knobs for the errors circular buffer
  3800. * [Feature] Add decompression support in rspamd client
  3801. * [Feature] Add errors exporter to the controller
  3802. * [Feature] Add expected value for monitored DNS resources
  3803. * [Feature] Add exporter from error ringbuf to ucl
  3804. * [Feature] Add extended version for fann creation function
  3805. * [Feature] Add ffi friendly version of process_regexp function
  3806. * [Feature] Add frequency and time display to webui
  3807. * [Feature] Add fuzzy_delhash command to rspamc client
  3808. * [Feature] Add implementation of redis connections pool
  3809. * [Feature] Add latency and offline time monitoring
  3810. * [Feature] Add learning support for lua classifiers
  3811. * [Feature] Add max-size and timeout options to CGP helper
  3812. * [Feature] Add method to enable/disable symbols in config
  3813. * [Feature] Add methods to get metric's actions and symbols from Lua
  3814. * [Feature] Add mmap support to lua_text
  3815. * [Feature] Add monitored object for surbl plugin
  3816. * [Feature] Add more exceptions to surbl whitelist
  3817. * [Feature] Add more meta-tokens to bayes
  3818. * [Feature] Add neural net classifier to fann_scores module
  3819. * [Feature] Add neural net serialization/deserialization
  3820. * [Feature] Add new dynamic conf module
  3821. * [Feature] Add periodic events support for lua_config
  3822. * [Feature] Add plugin to check MX'es for the sender's domain
  3823. * [Feature] Add preliminary monitored module
  3824. * [Feature] Add preliminary support of dynamic conf updates in Redis
  3825. * [Feature] Add preliminary version of clamav plugin
  3826. * [Feature] Add redis cache to asn module
  3827. * [Feature] Add replies compression
  3828. * [Feature] Add spamhaus DROP dnsbl
  3829. * [Feature] Add support for dictionary in client compression
  3830. * [Feature] Add support for fuzzy learn and unlearn from lua
  3831. * [Feature] Add support for input encryption
  3832. * [Feature] Add support of min_learns to neural net classifier
  3833. * [Feature] Add termination callbacks for workers
  3834. * [Feature] Add user-agent for rspamc
  3835. * [Feature] Add utility to perform classifier tests
  3836. * [Feature] Add zstd compression library
  3837. * [Feature] Allow HTTPS requests in lua_http
  3838. * [Feature] Allow conditions for pre and postfilters
  3839. * [Feature] Allow custom functions for ratelimits
  3840. * [Feature] Allow for excluding messages from AV scanning based on size
  3841. * [Feature] Allow for getting worker stats from Lua
  3842. * [Feature] Allow getting task UID from Lua
  3843. * [Feature] Allow parsing of mailbox messages from the commandline
  3844. * [Feature] Allow plugins to publish their lua API via rspamd_plugins
  3845. * [Feature] Allow to compare other systems with Rspamd
  3846. * [Feature] Allow to execute Lua scripts by controller
  3847. * [Feature] Allow to have a function to set custom greylist message
  3848. * [Feature] Allow to iterate over multiple tags
  3849. * [Feature] Allow to pass extra data from plugins to log helper
  3850. * [Feature] Allow to plan new periodics at different time
  3851. * [Feature] Allow to reset hashes
  3852. * [Feature] Allow to run rspamadm lua just as a lua interpreter
  3853. * [Feature] Allow to store settings in redis
  3854. * [Feature] Allow to update dynamic conf in Redis
  3855. * [Feature] Allow to use dictionaries for compression
  3856. * [Feature] Allow to use md5, sha1, sha256, sha384 and sha512 hashes in Lua
  3857. * [Feature] Allow whitelisting by IP for greylisting plugin
  3858. * [Feature] Antivirus: Support whitelists & pattern-matching sig names
  3859. * [Feature] Backport pack/unpack routines from Lua 5.3
  3860. * [Feature] Check settings with equal priopities in alphabetical order
  3861. * [Feature] Compress neural net in redis
  3862. * [Feature] Consider more tags when doing WHITE_ON_WHITE rule
  3863. * [Feature] Descriptive options for DMARC failure symbols
  3864. * [Feature] Descriptive options for RBL symbols
  3865. * [Feature] Enable configuration for monitored objects
  3866. * [Feature] Execute on_load scripts with ev_base ready
  3867. * [Feature] Fann scores now uses metadata from a message
  3868. * [Feature] Implement FANN threaded learning
  3869. * [Feature] Implement classifying for lua classifiers
  3870. * [Feature] Implement finish scripts for worker processes
  3871. * [Feature] Implement monitoring for DNS resources
  3872. * [Feature] Implement real priorities for pre and post filters
  3873. * [Feature] Insert two symbols: FANN_HAM and FANN_SPAM instead of one
  3874. * [Feature] Module to push metadata/messages to redis pubsub
  3875. * [Feature] Monitor RBL records
  3876. * [Feature] Move fann_classifier to a separate plugin
  3877. * [Feature] Normalize all ANN inputs
  3878. * [Feature] Preliminary version of metric exporter module
  3879. * [Feature] Preserve decompression context between tasks
  3880. * [Feature] Ratelimit: Support dynamic bucket size/leak rate
  3881. * [Feature] Relax FORGED_RECIPIENTS: allow senders to BCC themselves
  3882. * [Feature] Remove symbols weights on composites processing
  3883. * [Feature] Return symbol scores when getting resulting symbols
  3884. * [Feature] Rework lua tcp module
  3885. * [Feature] Rule to detect some obvious X-PHP-Originating-Script forgeries
  3886. * [Feature] Rule to identify some X-PHP-Script forgeries
  3887. * [Feature] Rules for scoring Google Message-ID fixes
  3888. * [Feature] Send hashes values to reply
  3889. * [Feature] Set expire for dmarc reports
  3890. * [Feature] Stop using cymru zone as it is unstable
  3891. * [Feature] Stop using of GLists for headers, improve performance
  3892. * [Feature] Store `for` in task:get_received_headers
  3893. * [Feature] Store `for` part in received headers
  3894. * [Feature] Store enabled flag for webui session
  3895. * [Feature] Store error messages in ring buffer
  3896. * [Feature] Support compressed maps
  3897. * [Feature] Support excluding selected users from ratelimits
  3898. * [Feature] Support looking up NS records in lua_dns
  3899. * [Feature] Support modern style SURBL configuration
  3900. * [Feature] Support multiple hashes in delhash path
  3901. * [Feature] Support new messages in rspamc
  3902. * [Feature] Support requests without reads in lua_tcp
  3903. * [Feature] Support setting task message from Lua
  3904. * [Feature] Track visibility of HTML elements
  3905. * [Feature] Try to add CRLF when checking DKIM
  3906. * [Feature] Try to guess line endings when folding headers
  3907. * [Feature] Try to improve normalization function for bayes
  3908. * [Feature] Use FFI to optimize SA module
  3909. * [Feature] Use length based arguments for redis, allow lua_text as arg
  3910. * [Feature] Use more layers for fann and another normalization
  3911. * [Feature] User-defined ratelimits
  3912. * [Feature] Utility to convert fuzzy storage from sqlite to redis
  3913. * [Feature] Yield DMARC_DNSFAIL on lookup failure
  3914. * [Fix] Adopt fuzzy storage for flexible backends
  3915. * [Fix] Allow plain IP addresses in Rspamd maps
  3916. * [Fix] Another fix for brain-damaged hiredis
  3917. * [Fix] Another fix for rdns write errors
  3918. * [Fix] Another fix for rdns_make_request_full invocation
  3919. * [Fix] Another fix in DKIM canonicalization
  3920. * [Fix] Another memory leak plugged
  3921. * [Fix] Another try to deal with posix idiotizm
  3922. * [Fix] Another try to fix RDNS events processing logic
  3923. * [Fix] Avoid double frees in HEAD requests
  3924. * [Fix] Avoid extra symbols for RBLs
  3925. * [Fix] Banish table.maxn from Lua parts
  3926. * [Fix] Check for socket error before connection in lua_tcp
  3927. * [Fix] Correctly propagate redis timeouts to Lua
  3928. * [Fix] Do not add extra newline in MIME mode
  3929. * [Fix] Do not be cheated by system hiredis
  3930. * [Fix] Do not classify when a message has not enough tokens
  3931. * [Fix] Do not crash on redis errors
  3932. * [Fix] Do not distinguish NXDOMAIN and NOREC for monitored
  3933. * [Fix] Do not replan retransmits if merely one server is defined
  3934. * [Fix] Do not use headers to calculate messages digests
  3935. * [Fix] Don't force action in replies module for authenticated users/local networks
  3936. * [Fix] Explicitly ban default passwords in webui
  3937. * [Fix] Finally fix ambiguity between parsed and resolved spf elts
  3938. * [Fix] Fix 'decoded' value in task:get_header_full()
  3939. * [Fix] Fix DKIM calculations
  3940. * [Fix] Fix DKIM signing for messages with no newline at the end
  3941. * [Fix] Fix DNS request in monitored
  3942. * [Fix] Fix DNS write errors processing
  3943. * [Fix] Fix HTTP methods other than GET and POST
  3944. * [Fix] Fix PERMFAIL for v6/v4 ambiguities
  3945. * [Fix] Fix absurdic scores for HFILTER_URL_ONLY
  3946. * [Fix] Fix actions in rolling history
  3947. * [Fix] Fix actrie patterns
  3948. * [Fix] Fix applying of lua dynamic confg
  3949. * [Fix] Fix autolearning errors and redis cache
  3950. * [Fix] Fix bayes learn_condition
  3951. * [Fix] Fix build with the recent OpenSSL
  3952. * [Fix] Fix caching and compressed maps
  3953. * [Fix] Fix check plain text part
  3954. * [Fix] Fix crash on OpenBSD in `url_email_start`
  3955. * [Fix] Fix double free in SPF
  3956. * [Fix] Fix extraction of shingles from redis fuzzy storage
  3957. * [Fix] Fix false sharing for symbols in the cache
  3958. * [Fix] Fix float usage in util:get_time
  3959. * [Fix] Fix folding algorithm to deal with empty tokens
  3960. * [Fix] Fix format string
  3961. * [Fix] Fix format string usage in controller errors handling
  3962. * [Fix] Fix handling of '\0' in lua_tcp
  3963. * [Fix] Fix handling of HTTP HEAD methods
  3964. * [Fix] Fix hash creation
  3965. * [Fix] Fix hiredis stupidity
  3966. * [Fix] Fix implicit settings module settingsup
  3967. * [Fix] Fix interaction with lua GC to avoid craches
  3968. * [Fix] Fix ip_score module registration
  3969. * [Fix] Fix issue with empty messages and dkim
  3970. * [Fix] Fix issues with CGP helper
  3971. * [Fix] Fix issues with the recent SPF changes
  3972. * [Fix] Fix key name to load ANN correctly
  3973. * [Fix] Fix lua tcp module by saving `do_read` in callback data
  3974. * [Fix] Fix memory leak in client when using compression
  3975. * [Fix] Fix min_learns option
  3976. * [Fix] Fix on_finish scripts and async handlers
  3977. * [Fix] Fix options for SPF dnsfail symbol
  3978. * [Fix] Fix parsing includes and redirects in SPF
  3979. * [Fix] Fix parsing of lua comments with empty lines
  3980. * [Fix] Fix parsing of unquoted HTML attributes
  3981. * [Fix] Fix periodic events and redis
  3982. * [Fix] Fix processing of fuzzy learns from Lua
  3983. * [Fix] Fix processing of redirect in SPF includes
  3984. * [Fix] Fix processing of symbols when reject limit is reached
  3985. * [Fix] Fix refcounts when map is specified by IP
  3986. * [Fix] Fix rspamd{session} class in Lua API
  3987. * [Fix] Fix setting ratelimit key for 'ip' bucket
  3988. * [Fix] Fix some cases of TLD urls detector
  3989. * [Fix] Fix statconvert tool
  3990. * [Fix] Fix stats for backend-less classifiers
  3991. * [Fix] Fix training script for fann_redis
  3992. * [Fix] Fix variable in ann module
  3993. * [Fix] Fix various errors in lua dynamic conf plugin
  3994. * [Fix] Forget old ANN when max_usages is reached to avoid overtrain
  3995. * [Fix] Further canonicalization fixes
  3996. * [Fix] Further fixes for fann_redis prefixes
  3997. * [Fix] Handle failures for inactive pooled connections
  3998. * [Fix] Improve multimap info message
  3999. * [Fix] More fixes in ANN loading
  4000. * [Fix] More fixes to fann_redis
  4001. * [Fix] More issues in fann_redis
  4002. * [Fix] More spaces fix in DKIM signature
  4003. * [Fix] Multiple fixes to asn script, add IPv6 support
  4004. * [Fix] Multiple issues in fann_redis
  4005. * [Fix] No greylist rejected messages
  4006. * [Fix] One more attempt to fix lua_redis
  4007. * [Fix] One more check for readdir...
  4008. * [Fix] Params should be treated as a hash
  4009. * [Fix] Plug memory leak in regexp desctructor
  4010. * [Fix] Process headers only once
  4011. * [Fix] Properly handle nil values in ratelimit plugin
  4012. * [Fix] Really fix redis shingles check
  4013. * [Fix] Remove fann with incorrect layers count
  4014. * [Fix] Remove mentions of deleted include
  4015. * [Fix] Remove some incompatible functions
  4016. * [Fix] Settings: fix `authenticated` parameter (#886)
  4017. * [Fix] Skip MX check for authenticated users and local networks
  4018. * [Fix] Slightly fix ANN routines
  4019. * [Fix] Stop caching records with DNS failures
  4020. * [Fix] Treat all errors in redis_pool as fatal errors for a connection
  4021. * [Fix] Try avoid false-positives in HEADER_FORGED_MDN rule
  4022. * [Fix] Try to avoid race condition when using rrd
  4023. * [Fix] Try to reload redis scripts if they are missing
  4024. * [Fix] Unbreak once_received skipping for local networks
  4025. * [Fix] Unlock ANN on error
  4026. * [Fix] Use memmove for overlapping regions
  4027. * [Fix] Use real size instead of displayed for core limits
  4028. * [Fix] Use the correct macro to get the size of control
  4029. * [Fix] Various fixes for errors ringbuffer
  4030. * [Fix] Yield R_SPF_DNSFAIL if lookup of included record fails
  4031. * [Fix] mid: fix map initialization
  4032. * [Fix] mid: handle incorrect rgexps in the map
  4033. * [Rework] Add extract training data function to fann_redis
  4034. * [Rework] Add preliminary train tests
  4035. * [Rework] Add redis storage feature to fann_redis
  4036. * [Rework] Adopt fuzzy storage for abstract backend
  4037. * [Rework] Adopt plugins
  4038. * [Rework] First reiteration on fann scores
  4039. * [Rework] Implement loading/invalidating
  4040. * [Rework] Make lua_redis task agnostic
  4041. * [Rework] Make rspamd protocol messages useful
  4042. * [Rework] Massive removal of legacy code
  4043. * [Rework] More cleanup actions
  4044. * [Rework] Remove legacy code never used for classifiers
  4045. * [Rework] Remove outdated and unused lua_session module
  4046. * [Rework] Reorganize fuzzy backend structure
  4047. * [Rework] Reorganize the internal backend structure
  4048. * [Rework] Restore old fann_scores, move common parts
  4049. * [Rework] Rework and simplify rbl plugin
  4050. * [Rework] Rework parsing of DMARC records
  4051. 1.3.4:
  4052. * [Feature] ASN module; support matching ASN/country in multimap
  4053. * [Feature] Add SPF method in spf return result
  4054. * [Feature] Add Yandex and Mail.ru forwarding rules
  4055. * [Feature] Add mempool maps in multimap
  4056. * [Feature] Add rule for identifying mail sent by eval()'d PHP code
  4057. * [Feature] Add support of stub DNSSEC resolver to rdns
  4058. * [Feature] Add task:get_digest method
  4059. * [Feature] Allow for more fine-grained scoring for ip_score
  4060. * [Feature] Allow to get digest of a mime part from lua
  4061. * [Feature] Allow to print message digest in logs
  4062. * [Feature] Fold DKIM-Signature header
  4063. * [Feature] Implement encrypted logs
  4064. * [Feature] Log URLs encrypted if we have log encryption pubkey
  4065. * [Feature] Pass authenticated bit to lua
  4066. * [Feature] Read redis backend statistics configuration from global section
  4067. * [Feature] Show the exact value matched for multima symbols
  4068. * [Feature] Store task checksum
  4069. * [Fix] Avoid setting limits when required elements are missing
  4070. * [Fix] DMARC: Fix alignment checking for subdomains
  4071. * [Fix] DMARC: deal with missing and spurious spaces
  4072. * [Fix] Defer insertion of results in ip_score to avoid skewing stats
  4073. * [Fix] Disable DMARC for local/authorized mail
  4074. * [Fix] Fix handling of proxied headers in controller
  4075. * [Fix] Fix hex printing of strings
  4076. * [Fix] Fix issue with spaces in maps
  4077. * [Fix] Fix parsing of forwarded IP
  4078. * [Fix] Fix reload in some plugins and workers
  4079. * [Fix] Fix reloading on SIGHUP
  4080. * [Fix] Fix some border cases for DKIM canonicalization
  4081. * [Fix] Fix url maps
  4082. * [Fix] Make dnssec configurable option disabled by default for now
  4083. * [Fix] rspamadm statconvert: force db to be a string
  4084. * [Fix] rspamadm statconvert: use db/password for learn cache
  4085. * [Rework] Rework flags in rspamd logger
  4086. 1.3.3:
  4087. * [CritFix] Check hyperscan cache sanity before loading
  4088. * [CritFix] Fix setting of fuzzy keys (completely breaks fuzzy storage)
  4089. * [Feature] Add SARBL (sarbl.org) uribl
  4090. * [Feature] Add `--search-pattern` option to rspamd_stats
  4091. * [Feature] Add some sanity check for very long from/to log elements
  4092. * [Feature] Allow to create hashes from string in a single step
  4093. * [Feature] Fix order of pre and postfilters
  4094. * [Feature] Improve lua URLs API
  4095. * [Feature] Improve message about fuzzy rules
  4096. * [Feature] Pre-calculate blake2 digest for all parts
  4097. * [Feature] Print radix duplicate keys as IP addresses
  4098. * [Feature] Simple mechanism for disabling RBLs in local.d/rbl.conf
  4099. * [Feature] Use faster hash function for fuzzy storage
  4100. * [Feature] rspamd_stats: support log directory reading
  4101. * [Fix] Add sanity check for url filters
  4102. * [Fix] Do not show rmilter section as a fake metric in rspamc
  4103. * [Fix] Fix URL filters
  4104. * [Fix] Fix a stupid mistake in util.strequal_caseless
  4105. * [Fix] Fix blake2b hash of the string "rspamd"
  4106. * [Fix] Fix filename maps filter
  4107. * [Fix] Fix finding tld in util.get_tld
  4108. * [Fix] Fix multimap content filters
  4109. * [Fix] Fix returning boolean from Lua
  4110. * [Fix] Fix returning of REDIS_NIL
  4111. * [Fix] Try to deal with multiple workers terminated
  4112. * [Fix] Use forced DNS request when calling for lua_http
  4113. * [Rework] Rework multimap filters, add redis maps
  4114. 1.3.2:
  4115. * [Feature] Add a special symbol for SPF DNS errors: R_SPF_DNSFAIL
  4116. * [Feature] Add correlations report in fuzzy stats
  4117. * [Feature] Add experimental CGP integration
  4118. * [Feature] Add method to get urls length in a text part
  4119. * [Feature] Add new methods to lua_html to access HTML tags
  4120. * [Feature] Allow all types of symbols to be added via __newindex method
  4121. * [Feature] Allow to create settings for authenticated users
  4122. * [Feature] Allow to get block content for HTML tags
  4123. * [Feature] Improve DNS failures when dealing with SPF
  4124. * [Feature] Properly implement R_WHITE_ON_WHITE rule
  4125. * [Feature] Remove old ugly rules
  4126. * [Feature] Rspamc can now add dkim signature in mime mode
  4127. * [Feature] Store content length for HTML tags
  4128. * [Feature] Support reacher set of HTML colors
  4129. * [Feature] Try to avoid FP for low contrast fonts detection
  4130. * [Fix] Add missing HTML colors
  4131. * [Fix] Add spaces to dkim signature to allow folding
  4132. * [Fix] Avoid returning NaN as score on scan
  4133. * [Fix] Decode entitles in href parts
  4134. * [Fix] Do not cache SPF records with DNS errors
  4135. * [Fix] Do not crash on cyclic depends
  4136. * [Fix] Do not insert HELO/HOSTNAME unknown when they are not passed
  4137. * [Fix] Do not set absent hostname to "unknown"
  4138. * [Fix] Do not stress redis with KEYS command (#791)
  4139. * [Fix] Fix DMARC_BAD_POLICY symbol
  4140. * [Fix] Fix HFILTER_URL module
  4141. * [Fix] Fix HFILTER_URL_ONELINE rule
  4142. * [Fix] Fix buffering in CGP integration
  4143. * [Fix] Fix colors propagation from parent nodes
  4144. * [Fix] Fix confusing OpenSSL API usage of i2d_RSAPublicKey
  4145. * [Fix] Fix dependencies id sanity check
  4146. * [Fix] Fix folding for semicolon separated tokens
  4147. * [Fix] Fix largest possible TLD behaviour
  4148. * [Fix] Fix last token folding
  4149. * [Fix] Fix length calculations in white on white rule
  4150. * [Fix] Fix multiple request headers structure
  4151. * [Fix] Fix multiple values headers freeing
  4152. * [Fix] Fix parsing of background color
  4153. * [Fix] Fix printing from field in log_urls
  4154. * [Fix] Fix processing of last element of DMARC policies
  4155. * [Fix] Further fixes for HTML colors
  4156. * [Fix] Further fixes for multiple values headers
  4157. * [Fix] Further fixes for white on white rule
  4158. * [Fix] Further fixes in HTML tags parsing
  4159. * [Fix] Ignore content type/subtype case
  4160. * [Fix] Increase score of R_WHITE_ON_WHITE
  4161. * [Fix] Parse CGP envelope data
  4162. * [Fix] Propagate colors in HTML
  4163. * [Fix] Restore multiple values headers in protocol
  4164. * [Fix] Restore multiple values in headers processing
  4165. * [Fix] Some more changes to tag's content length calculations
  4166. * [Fix] Some more fixes for low contrast fonts detector
  4167. * [Fix] SpamAssassin plugin: support check_freemail_header('EnvelopeFrom', [..])
  4168. * [Fix] Trigger HTML_SHORT_LINK_IMG on any external image
  4169. * [Fix] rspamd_stats: remove deprecated defined(@array)
  4170. 1.3.1:
  4171. * [CritFix] Fix catena passwords validation
  4172. * [CritFix] Fix crash when the first received is faked
  4173. * [Feature] Add DMARC_BAD_POLICY symbol when DMARC policy was invalid
  4174. * [Feature] Allow for matching hostnames in multimap (#773)
  4175. * [Feature] Allow for setting action based on DMARC disposition
  4176. * [Feature] Allow limiting of the inbound message size
  4177. * [Feature] Allow maps with multiple symbols and scores
  4178. * [Feature] Allow regexps in the emails maps
  4179. * [Feature] Allow to register metric symbols from multimap
  4180. * [Feature] Allow to reset redis tokens instead of appendig values
  4181. * [Feature] Allow to store strings in radix maps
  4182. * [Feature] Check UTF validity when there are utf regexps in a map
  4183. * [Feature] Correctly work when there is no hard reject action
  4184. * [Feature] Implement dependencies for maps
  4185. * [Fix] Another effort to unbreak sqlite locking
  4186. * [Fix] Avoid crash when closing mmapped file
  4187. * [Fix] Do not break history on NaN in required score
  4188. * [Fix] Ensure that hyperscan cache written is written properly
  4189. * [Fix] Filter NaN from scores in history
  4190. * [Fix] Fix DNSBL maps
  4191. * [Fix] Fix another locking issue in sqlite
  4192. * [Fix] Fix another locking issue with mapped files
  4193. * [Fix] Fix deadlock in mmaped file stats
  4194. * [Fix] Fix dependencies in multimap plugin
  4195. * [Fix] Fix emails module configuration
  4196. * [Fix] Fix greylist plugin (#755)
  4197. * [Fix] Fix greylisting plugin variable usage
  4198. * [Fix] Fix installed permissions for rspamd_stats
  4199. * [Fix] Fix locking in mmapped statistics
  4200. * [Fix] Fix paths in tests
  4201. * [Fix] Fix prefilter mode for multimap
  4202. * [Fix] Forgot to commit leftover changes
  4203. * [Fix] Really fix local.d includes
  4204. * [Fix] Restore selective greylisting behaviour
  4205. * [Fix] Set max size on per connection basis
  4206. * [Fix] Use temporary storage for hyperscan cache
  4207. * [Rework] Remove systemd socket activation
  4208. 1.3.0:
  4209. * [CritFix] Fix SA rawbody processing - exclude top part
  4210. * [CritFix] Fix decoding of UTF HTML entitles
  4211. * [CritFix] Fix encrypted fuzzy requests
  4212. * [CritFix] Fix leak of shared memory fds and files
  4213. * [CritFix] Fix levenshtein distance calculations
  4214. * [CritFix] Fix mime headers processing
  4215. * [CritFix] Fix parsing of URLs in texts
  4216. * [CritFix] Fix parsing of missing classes
  4217. * [CritFix] Fix redis structure by adding {NULL, NULL} member
  4218. * [CritFix] Fix some more URL detector issues
  4219. * [CritFix] Fix systemd sockets activation
  4220. * [CritFix] Fix unencrypted passwords processing in the controller
  4221. * [CritFix] Fix writing CDPs to the database
  4222. * [CritFix] Fix writing of encrypted HTTP requests
  4223. * [CritFix] Plug memory leak in dkim module
  4224. * [CritFix] Plug memory leak in headers getting code
  4225. * [CritFix] Pre-filters and post-filters were completely broken
  4226. * [CritFix] Properly support SA body regexps
  4227. * [CritFix] Really skip filters in case of pre-result set
  4228. * [CritFix] Restore the intended pre-filters behaviour
  4229. * [Rework] Adopt new maps code
  4230. * [Rework] Compile ragel sources when building rspamd
  4231. * [Rework] Finish rework for the rest of places that use HTTP
  4232. * [Rework] Fix DKIM headers canonicalization
  4233. * [Rework] Fix lua maps API
  4234. * [Rework] Import linenoise for line editing
  4235. * [Rework] Include config structure to all rcl handlers
  4236. * [Rework] Make chartable module useful
  4237. * [Rework] Move http internal structures to a private header
  4238. * [Rework] Partly fix controller
  4239. * [Rework] Remove dedicated images list
  4240. * [Rework] Rename http proxy to rspamd proxy
  4241. * [Rework] Rename mime parts structures
  4242. * [Rework] Rework HTTP code
  4243. * [Rework] Rework exceptions and newlines processing
  4244. * [Rework] Rework pre and postfilters system
  4245. * [Rework] Separate method to close backend connections
  4246. * [Rework] Start the complete maps rework
  4247. * [Rework] Use dynamically generated ragel C sources
  4248. * [Feature] Add 'blacklist' and 'strict' modes for whitelists
  4249. * [Feature] Add 'symbols_enabled' and 'groups_enabled' to settings
  4250. * [Feature] Add ESMTPSA received type
  4251. * [Feature] Add a simple script to evaluate rspamd rules in the logs
  4252. * [Feature] Add a simple tool to generate DKIM keys
  4253. * [Feature] Add a trivial heuristic for codepages
  4254. * [Feature] Add and use mumhash for non-crypto hashing
  4255. * [Feature] Add better method to check lua userdata types
  4256. * [Feature] Add better zip files search algorithm
  4257. * [Feature] Add catena PBKDF function
  4258. * [Feature] Add configuration knobs for in and out parser scripts
  4259. * [Feature] Add content filtering support to multimap
  4260. * [Feature] Add different timeouts for proxy connections
  4261. * [Feature] Add dot commands for lua REPL:
  4262. * [Feature] Add execution of lua global functions script
  4263. * [Feature] Add function for pretty printing of inet addresses
  4264. * [Feature] Add function to convert fstring_t to c string
  4265. * [Feature] Add function to create temporary shared memory mapping
  4266. * [Feature] Add function to generate random hex string
  4267. * [Feature] Add generic fucnction to parse IP maps
  4268. * [Feature] Add initial version of HTTP lua repl
  4269. * [Feature] Add learn condition to the default configuration
  4270. * [Feature] Add learn conditions for classifiers
  4271. * [Feature] Add limit for dkim signatures to be checked
  4272. * [Feature] Add locking routines for lua_util
  4273. * [Feature] Add lua API for getting info from archives
  4274. * [Feature] Add lua utility to decode URL encoding
  4275. * [Feature] Add method to copy message from http connection
  4276. * [Feature] Add mirrors feature
  4277. * [Feature] Add more algorithms for shingles generation
  4278. * [Feature] Add more domains to redirectors list
  4279. * [Feature] Add more encodingsto cryptobox hash API
  4280. * [Feature] Add more file utilities to lua_util
  4281. * [Feature] Add more functions to extract data from text parts
  4282. * [Feature] Add more methods to get headers from a task
  4283. * [Feature] Add more methods to init http message body
  4284. * [Feature] Add more options for redis config parsing function
  4285. * [Feature] Add new representation of email address
  4286. * [Feature] Add new symbols to filter bad extensions in messages
  4287. * [Feature] Add new utility methods to mimepart object
  4288. * [Feature] Add openphish support to rspamd phishing module
  4289. * [Feature] Add parsers for SMTP address in ragel
  4290. * [Feature] Add parsing of mirror hosts for fuzzy storage
  4291. * [Feature] Add parsing scripts for master connection as well
  4292. * [Feature] Add preliminary greylist plugin
  4293. * [Feature] Add preliminary phishtank support
  4294. * [Feature] Add preliminary rarv5 support
  4295. * [Feature] Add preliminary version of ssl toolbox
  4296. * [Feature] Add protection against open files limit and accepting sockets
  4297. * [Feature] Add rar v4 support
  4298. * [Feature] Add reading scripts for master connection
  4299. * [Feature] Add replies plugin
  4300. * [Feature] Add results parsing code
  4301. * [Feature] Add routines to compare and check pubkeys
  4302. * [Feature] Add simpler versions of refcounts
  4303. * [Feature] Add some time manipulation functions for lua APi
  4304. * [Feature] Add support for non-standard BATV signatures
  4305. * [Feature] Add support of address with at-domain list
  4306. * [Feature] Add support to search archives by magic
  4307. * [Feature] Add task:get_rawbody method
  4308. * [Feature] Add test to check shared memory support sanity
  4309. * [Feature] Add the initial version of LUA repl to rspamadm
  4310. * [Feature] Add throughput graph for RRD backend to WebUI
  4311. * [Feature] Add universal function to make a proper redis request
  4312. * [Feature] Add universal function to parse redis servers for plugins
  4313. * [Feature] Add util.unlink function
  4314. * [Feature] Add utility function to return random number from 0 to 1
  4315. * [Feature] Add utility method to convert ftok to C string
  4316. * [Feature] Add utility to map shared memory segments
  4317. * [Feature] Add versions to fuzzy storage
  4318. * [Feature] Add workaround for legacy clients in rspamd proxy
  4319. * [Feature] Add workaround for systems without sane shmem support
  4320. * [Feature] Add xoroshiro+ fast rng for non-crypto purposes
  4321. * [Feature] Adopt plugins for new maps API
  4322. * [Feature] Allow SPF to be checked for empty tasks
  4323. * [Feature] Allow binary patterns in lua_trie
  4324. * [Feature] Allow catena encrypted passwords in controller
  4325. * [Feature] Allow client ip match in the settings
  4326. * [Feature] Allow easy adding and overriding of fuzzy rules
  4327. * [Feature] Allow empty tasks to be processed
  4328. * [Feature] Allow hostnames in IP maps
  4329. * [Feature] Allow https maps
  4330. * [Feature] Allow multiple PBKDF types in `rspamadm pw`
  4331. * [Feature] Allow named fuzzy rules
  4332. * [Feature] Allow non zero terminated patterns in multipattern
  4333. * [Feature] Allow partial hash updates
  4334. * [Feature] Allow pipelining for redis.make_request
  4335. * [Feature] Allow sending empty requests using client
  4336. * [Feature] Allow setting fuzzy flag by symbol not by value
  4337. * [Feature] Allow setting scores and actions from lua
  4338. * [Feature] Allow shared memory simple http client
  4339. * [Feature] Allow static lua files in any parts of rspamd sources
  4340. * [Feature] Allow to change flag from fuzzy learn condition
  4341. * [Feature] Allow to check rspamd_text using maps
  4342. * [Feature] Allow to disable composite rules from settings
  4343. * [Feature] Allow to disable some modules from common redis setup
  4344. * [Feature] Allow to extract ucl_object from lua using common API
  4345. * [Feature] Allow to get settings and settings id hash from lua_task
  4346. * [Feature] Allow to have specific settings for mirrored traffic
  4347. * [Feature] Allow to open message from a shared memory segment
  4348. * [Feature] Allow to parse pubkeys from the rcl config
  4349. * [Feature] Allow to pass extradata from rspamd to rmilter
  4350. * [Feature] Allow to query storage about number of fuzzy hashes stored
  4351. * [Feature] Allow to read logs without symbols scores
  4352. * [Feature] Allow to read password from console for rspamc
  4353. * [Feature] Allow to set ciphers and CA paths in config
  4354. * [Feature] Allow to skip some initialization phases to speed up rspamadm
  4355. * [Feature] Allow underscore separated names in settings
  4356. * [Feature] Allow versioning for sqlite databases
  4357. * [Feature] Always allow to terminate rspamd
  4358. * [Feature] Better deal with backend errors
  4359. * [Feature] Better lua_redis logging
  4360. * [Feature] Configure CA path and ciphers
  4361. * [Feature] Create a dedicated parser to strip newlines
  4362. * [Feature] Deduplicate the same urls in multimap module
  4363. * [Feature] Distinguish luajit from lua
  4364. * [Feature] Do not print garbadge in --compact output
  4365. * [Feature] Dynamically detect if a CPU is incompatible with hyperscan
  4366. * [Feature] Enable forced resolving for some lua plugins
  4367. * [Feature] Enable rrd by default
  4368. * [Feature] Enable workaround for exim
  4369. * [Feature] Fix task functions to work without rspamd_config
  4370. * [Feature] Further improvements to chartable module
  4371. * [Feature] Further micro-optimizations for hashing and shingles
  4372. * [Feature] Further relax parser
  4373. * [Feature] Humanize numbers in stats widgets
  4374. * [Feature] Implement HTTPS client
  4375. * [Feature] Implement SSL support in http client
  4376. * [Feature] Implement body rules for the trie plugin
  4377. * [Feature] Implement braced regexp quantifiers
  4378. * [Feature] Implement compare scripts for mirrors results
  4379. * [Feature] Implement compare scripts setup
  4380. * [Feature] Implement composites policies
  4381. * [Feature] Implement conditional learning for classifiers
  4382. * [Feature] Implement constructing of map from UCL
  4383. * [Feature] Implement dkim signing in dkim check plugin
  4384. * [Feature] Implement fuzzy storage updates
  4385. * [Feature] Implement fuzzy updates push protocol
  4386. * [Feature] Implement https maps
  4387. * [Feature] Implement inter-process maps caching
  4388. * [Feature] Implement mapping of remote flags to local flags
  4389. * [Feature] Implement mirroring in rspamd proxy
  4390. * [Feature] Implement multi-flags fuzzy replies
  4391. * [Feature] Implement multiple-sources fuzzy storage
  4392. * [Feature] Implement order of pre/post filters
  4393. * [Feature] Implement partial deleting for multi-flags
  4394. * [Feature] Implement pipelining for redis async interface
  4395. * [Feature] Implement ragel parser for received headers
  4396. * [Feature] Implement reading of messages to shared memory
  4397. * [Feature] Implement refcount for messages
  4398. * [Feature] Implement retransmits for master connection
  4399. * [Feature] Implement zero-copy mode for HTTP reading
  4400. * [Feature] Improve SPF domain detection logic
  4401. * [Feature] Improve config:register_symbol function
  4402. * [Feature] Improve error report for type mismatch in lua
  4403. * [Feature] Improve fstrings API
  4404. * [Feature] Improve getting SMTP data from lua_task
  4405. * [Feature] Improve levenshtein distance function
  4406. * [Feature] Improve logging in proxy and add refcounts
  4407. * [Feature] Improve logging lua types
  4408. * [Feature] Improve master/slave logging
  4409. * [Feature] Improve phishing plugin
  4410. * [Feature] Improve phishtank and openphish support
  4411. * [Feature] Improve ragel build target
  4412. * [Feature] Improve statistics script
  4413. * [Feature] Initialize ssl library to use SSL connections
  4414. * [Feature] Interpolate points sent to webui
  4415. * [Feature] Limit logging of elements that could have too many items
  4416. * [Feature] Lock ANN file when loading
  4417. * [Feature] New abstract hashing API in cryptobox
  4418. * [Feature] Normalize quoted addresses
  4419. * [Feature] Now cryptobox lua API accepts rspamd text as input
  4420. * [Feature] Optimize alignment to speed up hashing
  4421. * [Feature] Parse received date and ESMTPA proto
  4422. * [Feature] Parse received timestamp
  4423. * [Feature] Pass settings id to log helper
  4424. * [Feature] Pass settings id to lua script from log helper
  4425. * [Feature] Perform files expansion on proxying
  4426. * [Feature] Preliminary implementation of fuzzy master/slave updates
  4427. * [Feature] Print userdata using tostring if possible
  4428. * [Feature] Propagate HTTP result string when using proxy
  4429. * [Feature] Properly implement unweighted round-robin algorithm
  4430. * [Feature] Reduce number of timers queries
  4431. * [Feature] Rework and improve fuzzy storage
  4432. * [Feature] Rework dns resolving API for lua, add 'forced' option
  4433. * [Feature] Rework fann module to understand settings
  4434. * [Feature] Rework listening system to allow multiple socket types per worker
  4435. * [Feature] Rework ratelimit module to set expiration
  4436. * [Feature] Save bayes probability in memory pool var
  4437. * [Feature] Save settings id hash for convenience
  4438. * [Feature] Search for SSL_set_tlsext_host_name support
  4439. * [Feature] Send DKIM signature to protocol reply
  4440. * [Feature] Show DKIM signature in rspamc client
  4441. * [Feature] Show symbols description in scan output
  4442. * [Feature] Sign message merely after DKIM check
  4443. * [Feature] Simplify machines by assuming that headers are unfolded
  4444. * [Feature] Sort symbols when displaying them in log
  4445. * [Feature] Split main connection from mirrored connections
  4446. * [Feature] Start moving to the new email address structure
  4447. * [Feature] Store HTTP headers in a hash table
  4448. * [Feature] Store more information about compressed files
  4449. * [Feature] Store raw headers value to use them in DKIM
  4450. * [Feature] Store text parts content with newlines stripped
  4451. * [Feature] Support DKIM signing
  4452. * [Feature] Support EXIF jpeg images
  4453. * [Feature] Support archive files list extraction
  4454. * [Feature] Support archives when matching patterns in multimap
  4455. * [Feature] Support premium/academic feed for openphish
  4456. * [Feature] Support rspamd_updates via https
  4457. * [Feature] Supprort FQDNs in phishing module maps
  4458. * [Feature] Try to read on fuzzy timeout to avoid fake timeouts
  4459. * [Feature] Try to select the optimal possible function for input
  4460. * [Feature] Unescape and unquote smtp addresses
  4461. * [Feature] Update fuzzy timestamp when adding value
  4462. * [Feature] Update mumhash
  4463. * [Feature] Use -flto if possible when optimizations are enabled
  4464. * [Feature] Use extended map types in lua map, unify code
  4465. * [Feature] Use file lock in logger to avoid deadlocks
  4466. * [Feature] Use generic global string split function
  4467. * [Feature] Use metrohash as well
  4468. * [Feature] Use mumhash by default for incremental hashing
  4469. * [Feature] Use mumhash for words hashing
  4470. * [Feature] Use new ip parsing API
  4471. * [Feature] Use new maps API for local addrs
  4472. * [Feature] Use new ragel parser in message parsing code
  4473. * [Feature] Use new received parser instead of old one
  4474. * [Feature] Use new redis API in DMARC plugin
  4475. * [Feature] Use new redis API in greylist plugin
  4476. * [Feature] Use new redis API in ip_score plugin
  4477. * [Feature] Use new redis API in ratelimit plugin
  4478. * [Feature] Use new redis API in replies plugin
  4479. * [Feature] Use new version of register_symbol in rspamd plugins
  4480. * [Feature] Use offset when passing shm to deal with encrypted requests
  4481. * [Feature] Use one pass to remove newlines and store their positions
  4482. * [Feature] Use rspamd specific type checks for userdata
  4483. * [Feature] Use shared memory storage for http maps
  4484. * [Feature] Use universal redis definitions in rspamd plugins
  4485. * [Feature] Various improvements in greylist module
  4486. * [Feature] Wait for sqlite if locked when switching to WAL mode
  4487. * [Fix] Add filenames sanity filtering for mime types
  4488. * [Fix] Add guards for empty parts
  4489. * [Fix] Add missing types
  4490. * [Fix] Add more priority for config file symbols registered from UCL
  4491. * [Fix] Add sanity checks when compiling regexp maps
  4492. * [Fix] Add spaces instead of newlines to the normalized content
  4493. * [Fix] Add workaround for ancient openssl
  4494. * [Fix] Add workaround for gmime CTE stupidity
  4495. * [Fix] Add workaround for hex digits
  4496. * [Fix] Adjust MISSING_MIMEOLE score
  4497. * [Fix] Adjust body/body_buf when stealing encrypted message
  4498. * [Fix] Adopt lua task API for the new email addresses structure
  4499. * [Fix] Allow for disabling DMARC reporting when Redis is configured
  4500. * [Fix] Always register openphish and phishtank virtual symbols
  4501. * [Fix] Always use shmem on linux
  4502. * [Fix] Another change of newlines policy
  4503. * [Fix] Another d3evolution update
  4504. * [Fix] Another fix for exim workaround
  4505. * [Fix] Another fix for legacy clients
  4506. * [Fix] Another fix for maps scheduling
  4507. * [Fix] Another fix for marking upstreams inactive
  4508. * [Fix] Another fix for postfilters
  4509. * [Fix] Another fix for redis timeouts
  4510. * [Fix] Avoid `table.getn` method as it has been removed in lua 5.3
  4511. * [Fix] Avoid double hashing for images
  4512. * [Fix] Avoid linking with actrie if hyperscan is enabled
  4513. * [Fix] Check copy result when sending message to mirrors
  4514. * [Fix] Cleanup message when assiging body
  4515. * [Fix] Cleanup stack from global vars
  4516. * [Fix] Correctly parse query type
  4517. * [Fix] Disable all symbols if enable_groups is found in settings
  4518. * [Fix] Disable fts as it is completely broken in bloody linux
  4519. * [Fix] Disable multiple autolearn checks
  4520. * [Fix] Disallow updates by default
  4521. * [Fix] Do not abort when cannot load a map
  4522. * [Fix] Do not check recursion for non-DNS SPF record types
  4523. * [Fix] Do not delete uninitialized events
  4524. * [Fix] Do not die if shmem_mkstemp fails
  4525. * [Fix] Do not die when no metrics defined
  4526. * [Fix] Do not even try pcre in case of regexp maps
  4527. * [Fix] Do not greylist messages if redis has failed somehow
  4528. * [Fix] Do not greylist on rejection
  4529. * [Fix] Do not leave temporary maps cached
  4530. * [Fix] Do not output meaningless errors
  4531. * [Fix] Do not send NaN in json
  4532. * [Fix] Don't mix hyperscan and pcre processing within a same task
  4533. * [Fix] Finally rework and simplify redis backend for statistics
  4534. * [Fix] Fix Exim shutdown patch
  4535. * [Fix] Fix JIT compilation for PCRE2 expressions
  4536. * [Fix] Fix JIT usage for PCRE2
  4537. * [Fix] Fix REPL output
  4538. * [Fix] Fix SMTP address parsing machine
  4539. * [Fix] Fix UTF8 mode in PCRE2
  4540. * [Fix] Fix a stupid misprint in word 'phishing'
  4541. * [Fix] Fix accepting the first update when local idx is -1
  4542. * [Fix] Fix adding maps from ucl
  4543. * [Fix] Fix adding upstream to an active queue
  4544. * [Fix] Fix and rescore R_PARTS_DIFFER logic
  4545. * [Fix] Fix body rules in SA plugin
  4546. * [Fix] Fix body start position
  4547. * [Fix] Fix border case in urls detector
  4548. * [Fix] Fix border cases for incremental hashing
  4549. * [Fix] Fix caseless uthash application
  4550. * [Fix] Fix chartable issue with starting digits
  4551. * [Fix] Fix client_ip in users settings
  4552. * [Fix] Fix compilation issue
  4553. * [Fix] Fix conditional learning
  4554. * [Fix] Fix crash on empty maps
  4555. * [Fix] Fix creating of URLs from LUA
  4556. * [Fix] Fix creating of temporary shmem segment
  4557. * [Fix] Fix creation of mmapped statfiles
  4558. * [Fix] Fix descriptors leak on shmem detaching
  4559. * [Fix] Fix detaching of shared memory segments
  4560. * [Fix] Fix detection of URLs in text parts
  4561. * [Fix] Fix directories processing for rspamc
  4562. * [Fix] Fix displaying of rewrite subject in WebUI
  4563. * [Fix] Fix dkim private keys operations
  4564. * [Fix] Fix dkim signing
  4565. * [Fix] Fix dynamic scoring of subject symbols
  4566. * [Fix] Fix email address build
  4567. * [Fix] Fix encrypted proxy requests
  4568. * [Fix] Fix errors counting in upstreams
  4569. * [Fix] Fix errors handling in the proxy
  4570. * [Fix] Fix event bases for IO events
  4571. * [Fix] Fix events handling when scheduling map wacth
  4572. * [Fix] Fix fann rewrite
  4573. * [Fix] Fix files fallback for shmem transfer
  4574. * [Fix] Fix fuzzy adding in webui
  4575. * [Fix] Fix fuzzy storage encrypted mirroring
  4576. * [Fix] Fix fuzzy storage sync replies
  4577. * [Fix] Fix handling of the same words
  4578. * [Fix] Fix inserting values to the sources list
  4579. * [Fix] Fix ipv6 mask application
  4580. * [Fix] Fix issue with missing recipients
  4581. * [Fix] Fix issues with multiple returns from lua
  4582. * [Fix] Fix learning for non-existent backend
  4583. * [Fix] Fix legacy clients support in proxy
  4584. * [Fix] Fix length calculations for shared memory segments
  4585. * [Fix] Fix listening on UDP sockets
  4586. * [Fix] Fix loading of file maps
  4587. * [Fix] Fix long regexp flags (e.g. {sa_body})
  4588. * [Fix] Fix lua compare function init
  4589. * [Fix] Fix maps descriptions
  4590. * [Fix] Fix maps locking
  4591. * [Fix] Fix max_train setup in ANN module
  4592. * [Fix] Fix memory corruption
  4593. * [Fix] Fix memory leak in unsigned maps reading
  4594. * [Fix] Fix misprints for lto usage
  4595. * [Fix] Fix more issues with scripts processing
  4596. * [Fix] Fix next-to-last extension length check
  4597. * [Fix] Fix openssl initialization
  4598. * [Fix] Fix order of arguments
  4599. * [Fix] Fix order of initialization
  4600. * [Fix] Fix parser
  4601. * [Fix] Fix parsing of binary tries
  4602. * [Fix] Fix parsing of braced IPv6 addresses
  4603. * [Fix] Fix parsing of nested braces in SMTP comments
  4604. * [Fix] Fix parsing of rarv5 archives
  4605. * [Fix] Fix periodic scheduling when a map is not modified
  4606. * [Fix] Fix possible FP in TRACKER_ID rule
  4607. * [Fix] Fix post-filters processing
  4608. * [Fix] Fix potential NULL dereference
  4609. * [Fix] Fix processing of <br> and <hr> tags
  4610. * [Fix] Fix processing of addresses in protocol
  4611. * [Fix] Fix processing of messages without received headers
  4612. * [Fix] Fix proxying issue for unconnected sessions
  4613. * [Fix] Fix proxying of the encrypted messages
  4614. * [Fix] Fix race condition with shared memory by refcounts
  4615. * [Fix] Fix ratelimit initialization
  4616. * [Fix] Fix redis set request in replies plugin
  4617. * [Fix] Fix redis timeout events handling
  4618. * [Fix] Fix redis timeouts processing logic
  4619. * [Fix] Fix refcounts in lua_redis
  4620. * [Fix] Fix results checking if no master connection is active
  4621. * [Fix] Fix return value for couple of lua functions
  4622. * [Fix] Fix round-robin selection when upstreams have no weight
  4623. * [Fix] Fix rows calculation in graph
  4624. * [Fix] Fix rspamd_redis_make_request syntax in replies plugin
  4625. * [Fix] Fix scheduling of locked map events
  4626. * [Fix] Fix scores detection
  4627. * [Fix] Fix searching for newline positions
  4628. * [Fix] Fix secure_ip setting in controller
  4629. * [Fix] Fix sending data to graph command
  4630. * [Fix] Fix setting of score for parts differ
  4631. * [Fix] Fix setting of the lua top
  4632. * [Fix] Fix setting path for lua
  4633. * [Fix] Fix setting path for phishtank
  4634. * [Fix] Fix settings application
  4635. * [Fix] Fix shm_open call as described in POSIX
  4636. * [Fix] Fix size of length in fuzzy mirror wire protocol
  4637. * [Fix] Fix smtp grammar issues
  4638. * [Fix] Fix some issues with redis API
  4639. * [Fix] Fix some issues with retries in the proxy
  4640. * [Fix] Fix stack growing
  4641. * [Fix] Fix start of body detection in DKIM
  4642. * [Fix] Fix state on timeout
  4643. * [Fix] Fix stats script
  4644. * [Fix] Fix substring search when there are '\0' in strings
  4645. * [Fix] Fix symbol name for spf soft fail
  4646. * [Fix] Fix symbol type's check
  4647. * [Fix] Fix symbols registration and execution
  4648. * [Fix] Fix the case of multiple values keys
  4649. * [Fix] Fix the default symbol names according to metric
  4650. * [Fix] Fix timeout setup on learning
  4651. * [Fix] Fix timeouts in redis cache processing
  4652. * [Fix] Fix timeouts processing in lua_redis
  4653. * [Fix] Fix upstreams interaction for rspamd proxy
  4654. * [Fix] Fix usage of rdns reply structure
  4655. * [Fix] Fix varargs loop
  4656. * [Fix] Fix whitelists and blacklists in SA rules
  4657. * [Fix] Fix write servers setup for redis
  4658. * [Fix] Fix writing of HTTP messages
  4659. * [Fix] Force rspamd to upgrade fuzzy db on opening
  4660. * [Fix] Free the correct pointer
  4661. * [Fix] Further fixes for lto and static linking
  4662. * [Fix] Further fixes for surbl extensions map
  4663. * [Fix] Further fixes in maps code
  4664. * [Fix] Further improvements to error messages in fuzzy check
  4665. * [Fix] Further tweaks to redis garbadge collection
  4666. * [Fix] Groups are now case insensitive
  4667. * [Fix] Handle log pipe read errors
  4668. * [Fix] Handle nested dependencies in SpamAssassin plugin
  4669. * [Fix] Implement new automata to skip empty lines for dkim signing
  4670. * [Fix] Improve error messages on fuzzy add
  4671. * [Fix] Improve lua redis handling
  4672. * [Fix] Improve phishing module logging
  4673. * [Fix] Improve printing of fuzzy errors
  4674. * [Fix] Improve rrd diagnostic errors
  4675. * [Fix] Improve strcase hash used in uthash
  4676. * [Fix] Include fuzzy key to distinguish storages with different keys
  4677. * [Fix] Include slave cluster name into http request
  4678. * [Fix] Include some more information about archives
  4679. * [Fix] Indicate upstream error on timeout
  4680. * [Fix] Initialize hash tables array to avoid crashes
  4681. * [Fix] Initialize parser scripts properly
  4682. * [Fix] Initialize vars to avoid warnings
  4683. * [Fix] Inverse logic for saving ANN
  4684. * [Fix] Link lpeg to rspamd lua library
  4685. * [Fix] Make extension checks case-insensitive
  4686. * [Fix] Mark expired hashes as not found and not as zero flag
  4687. * [Fix] Match archive name as well
  4688. * [Fix] More and more fixes to redis states
  4689. * [Fix] More fixes about shared memory in proxy
  4690. * [Fix] More fixes for redis refcounts
  4691. * [Fix] More fixes to end of headers detection
  4692. * [Fix] More fixes to events logic
  4693. * [Fix] More fixes to multi-flag fuzzy storage
  4694. * [Fix] More fixes to parts distance calculations
  4695. * [Fix] More guards for redis free
  4696. * [Fix] One more fix in redis destructor
  4697. * [Fix] One more try to fix redis
  4698. * [Fix] PIE is required for static build
  4699. * [Fix] Partial fix for mmap'd statistics tests
  4700. * [Fix] Plug memory leak in proxy
  4701. * [Fix] Properly detect end of headers position
  4702. * [Fix] Properly init and free session structures
  4703. * [Fix] Reduce PRECEDENCE_BULK rule weight
  4704. * [Fix] Reduce the default thresholds for learning
  4705. * [Fix] Remove Type=forking from systemd unit file (#709)
  4706. * [Fix] Remove bad FANN file to save computational resources
  4707. * [Fix] Remove event before closing of fd to avoid race conditions
  4708. * [Fix] Remove parsing of 'from' variable in redis backend
  4709. * [Fix] Remove some bad domains from whitelists
  4710. * [Fix] Repair optional dependencies
  4711. * [Fix] Reset master connection when retransmitting scan request
  4712. * [Fix] Restore ONCE_RECEIVED symbol
  4713. * [Fix] Restore compatibility with old lua API behaviour
  4714. * [Fix] Restore redis runtime state
  4715. * [Fix] Reverse options when received
  4716. * [Fix] Send updates to mirrors only if we have some changes
  4717. * [Fix] Set host attribute properly when making HTTP request from lua
  4718. * [Fix] Set terminated state before calling of async free
  4719. * [Fix] Simplify MISSING_MIMEOLE rule
  4720. * [Fix] Simplify state machine by ignoring multiple spaces
  4721. * [Fix] Skip setting RPATH for static builds
  4722. * [Fix] Slightly reduce weights of rules with high FP rate
  4723. * [Fix] Some fixes to libmagic initialization
  4724. * [Fix] Some more fixes to ratelimit plugin
  4725. * [Fix] Strip '\r\n' properly
  4726. * [Fix] Switch hashes to mumhash
  4727. * [Fix] Treat NaN values properly in graph command
  4728. * [Fix] Try to avoid FP when checking for phished URLs
  4729. * [Fix] Try to avoid recursive events deletions
  4730. * [Fix] Try to fix false positive URL detections in text parts
  4731. * [Fix] Try to fix issue in redis stats backend when task is closed
  4732. * [Fix] Try to fix proxying of stupid spamc protocol to HTTP mirrors
  4733. * [Fix] Try to fix redis crashes
  4734. * [Fix] Try to fix upstreams with one element
  4735. * [Fix] Try to handle multiline history in a more sane way
  4736. * [Fix] Unbreak build on gcc < 4.9
  4737. * [Fix] Update RPM spec/sources (#700)
  4738. * [Fix] Update d3evolution version
  4739. * [Fix] Update mumhash implementation
  4740. * [Fix] Use custom error function for pre and post filters
  4741. * [Fix] Use new postfilters and prefilters API in the plugins
  4742. * [Fix] Use non-blocking mode for systemd sockets
  4743. * [Fix] Use shared memory merely for local backends in the proxy
  4744. * [Fix] Use watchers for spf plugin
  4745. * [Fix] Varioud fixes to the maps code
  4746. 1.2.8:
  4747. * Another fix for exim workaround (#637)
  4748. * Fix unencrypted passwords processing in the controller
  4749. * Fix setting path for lua (#652)
  4750. * Fix usage of rdns reply structure (#654)
  4751. * Use file lock in logger to avoid deadlocks
  4752. * Add `application/octet-stream` mime type for `pdf` extension (by @moisseev)
  4753. * Implement new automata to skip empty lines for dkim signing (#651)
  4754. * Fix parsing of missing classes
  4755. * Clarify some rspamc arguments (by @fatalbanana)
  4756. * Correct suppress spelling
  4757. 1.2.7:
  4758. * Slightly reduce weights of rules with high FP rate
  4759. * Add workround for rspamd-1.3
  4760. * Fix possible FP in TRACKER_ID rule
  4761. * Simplify MISSING_MIMEOLE rule
  4762. * Add workaround for gmime CTE stupidity
  4763. * Fix mime headers processing
  4764. * Fix false positive URL detections in text parts
  4765. * Fix Exim shutdown patch
  4766. * Enable workaround for exim mailbox format
  4767. * Backport shingles static test
  4768. * Fix levenshtein distance calculations
  4769. * Fix max_train setup in ANN module
  4770. * Fix redis structure by adding {NULL, NULL} member
  4771. * Fix build with unmodified LibreSSL opensslv.h
  4772. * Repair optional dependencies
  4773. * Really skip filters in case of pre-result set
  4774. * Restore the intended pre-filters behaviour
  4775. * Fix ipv6 mask application
  4776. 1.2.6:
  4777. * Fix parsing of URLs in texts
  4778. * Fix creating of URLs from LUA
  4779. * Fix some more URL detector issues
  4780. * Fix unit tests
  4781. * Fix JIT compilation for PCRE2 expressions
  4782. * Fix JIT usage for PCRE2
  4783. * Fix UTF8 mode in PCRE2
  4784. * Add workaround for pre-historic compilers (#605)
  4785. * Fix and rescore R_PARTS_DIFFER logic
  4786. * Properly set lua paths for tests
  4787. * Fix SA rawbody processing - exclude top part
  4788. * Store text parts content with newlines stripped
  4789. * Properly support SA body regexps
  4790. * Fix body rules in SA plugin
  4791. * Fix setting of score for parts differ
  4792. * More fixes to parts distance calculations
  4793. - Use hashed words instead of full words for speed
  4794. - Improve levenstein distance calculations and penalise replaces
  4795. - Always return number from 0 to 1
  4796. - Use g_malloc instead of alloca
  4797. * Fix percents output in R_PARTS_DIFFER
  4798. * Plug memory leak in dkim module
  4799. * Plug minor memory leak in regexps creation
  4800. 1.2.5:
  4801. * Plug an important memory leak in headers getting code
  4802. * Remove some bad domains from whitelists
  4803. 1.2.4:
  4804. * Implement new multipattern matcher that uses hyperscan if possible
  4805. * Use mutlipattern for lua_trie code
  4806. * Add utility methods for multipattern
  4807. * Use multipattern in url matcher
  4808. * Add escape functions for hyperscan
  4809. * Allow to optimize lua -> C transition by flattening table args
  4810. * Optimize hot paths in SA plugin
  4811. * Optimize rspamd_re_cache_type_from_string
  4812. * Allow empty tries
  4813. * Fix extraction of URLs from Subject
  4814. * Allow to have different flags for different patterns in multipattern
  4815. * Add common directory for hyperscan cache to config
  4816. * Implement caching for hyperscan multipattern
  4817. * Attach domain part to `R_SUSPICIOUS_URL` (by @moisseev)
  4818. * Allow multipattern scans to be nested for the case of hyperscan
  4819. * Simplify SURBL redirector search code and avoid ac_trie
  4820. * Add two way substring search algorithm
  4821. * Avoid acism usage to find gtube pattern
  4822. * Fix processing of empty headers
  4823. * Allow to disable pthread mutexes on broken platforms
  4824. * Make web interface not send password in query strings (#585) by @fatalbanana
  4825. * Add maximum delay to ratelimit module
  4826. * Backport fix for empty files inclusion from libucl
  4827. * Fix settings id setup
  4828. * Add min_learns option to classifiers
  4829. * Use more clever to utf8 conversion strategy
  4830. * Fix disabling of virtual symbols in the settings
  4831. * Rework settings to work properly in metric-less configuration
  4832. * Set the default limit for classifier
  4833. * Fix ttl based expiration from LRU cache
  4834. * Rework DKIM module to use OpenSSL for digests
  4835. * Fix mailto urls parsing with hyperscan
  4836. * Do not set obscured flag for urls starting with spaces
  4837. * Fix crash on redis learn
  4838. * Fix ratelimit ctime setting
  4839. 1.2.3:
  4840. * New DCC module (by @smfreegard)
  4841. * Rework whitelist module:
  4842. - Now we check different elements for different checks
  4843. - MIME from for DMARC
  4844. - DKIM signature domain for DKIM
  4845. - SMTP from or HELO for SPF
  4846. * Fix regexps results combination (*critical*)
  4847. * Fix issue with expressions processing (*critical*)
  4848. * Optimize strlcpy for aligned input
  4849. * Add support of half-closed connection in lua_tcp
  4850. * Allow to print compact json in client
  4851. * Save required score in history (#581)
  4852. * Allow to attach file descriptors to control commands
  4853. * Allow to send descriptors from workers to main
  4854. * Allow to attach fd when broadcasting to workers
  4855. * Implement log pipe feature for rspamd logs analysis
  4856. * Add `log_helper` worker
  4857. * Add `URIBL_SBL_CSS` (by @smfreegard)
  4858. * Add worker scripts functionality
  4859. * Add on load hooks for rspamd_config
  4860. * Add lua scripts for log_helper worker
  4861. * Add generic maillist detector (#584)
  4862. * Implement FANN autolearn using log_helper worker
  4863. * Rework metrics configuration to allow includes
  4864. * Change default value of forced removal in composite rules
  4865. * Allow to use assembly version of blake2b on x86 cpu
  4866. * Use less precise (but faster) clock if possible
  4867. * Insert redirected URL to the urls list
  4868. * Allow to get and set callback data for rspamd symbols
  4869. * Add binary heap implementation
  4870. * Use binary heap for expire algorithms in the hash
  4871. * Use `least frequent used` expiration strategy
  4872. * Allow to get mime headers from a task
  4873. * Add support for mime headers in `regexp` module
  4874. * Update Exim patches (by @fatalbanana)
  4875. * Allow building rspamd with jemalloc
  4876. * Save multipart boundaries
  4877. * SA plugin changes:
  4878. - Properly handle MIME headers
  4879. - Fix eval:check_for_missing_to_header rule
  4880. - Implement SA compatible body regexps
  4881. - Use sabody rules in SA plugin
  4882. * LUA API changes:
  4883. - Add util.get_ticks function
  4884. - Add util.stat function
  4885. - Add task:get_symbols_numeric method
  4886. - Add method to get number of symbols in the cache
  4887. - Add lua methods to get redirected urls
  4888. - Allow to get callbacks for lua symbols
  4889. - Add config:set_symbol_callback function
  4890. 1.2.2:
  4891. * Use HTTP Content-Type on non mime input if possible
  4892. * Save log level when compressing log messages
  4893. * Further rework of composite rules (add '^' prefix)
  4894. * Add tracking for rspamd expressions
  4895. * Store actions limits in metric result
  4896. * Fix parsing of include/redirect with many records in SPF
  4897. * Add method to disable symbols execution in the cache
  4898. * Allow to disable checks from settings
  4899. * Allow to select settings by id in HTTP query
  4900. * Find URLs with '\r' and '\n' inside href attribute
  4901. * Implement vectored mode for hyperscan (experimental)
  4902. * Improve client connection errors diagnostics
  4903. * Allow to edit new files with signtool
  4904. * Improve hashes performance on 32 bit platforms
  4905. * Fix sorting of limits
  4906. * Remove slow and unused rules `INVALID_EXIM_RECEIVED*`
  4907. * Add expression:process_traced lua method
  4908. * Allow tables in task:insert_result
  4909. * Save trace for SA metas
  4910. * Do not parse broken TLD parts in URLs
  4911. * Investigate many border cases in URLs parser
  4912. 1.2.1:
  4913. * Add list support to `mime types` module configuration (by @moisseev)
  4914. * Allow symbols params to be printed in logs
  4915. * Fix `MIME_BAD_ATTACHMENT` false positives for MDN/DSN
  4916. * Fix crashes on arm32
  4917. * Do not classify message if some class is missing
  4918. * Fix cryptobox cleanup
  4919. * Remove multipart/report from bad mime types (#569)
  4920. * Improve logging for fuzzy hashes
  4921. * Show map URLs in webui
  4922. * Sort symbols in webui
  4923. 1.2.0:
  4924. * New dynamic updates plugin
  4925. * Regular expressions map support
  4926. * Faster radix trie algorithm
  4927. * Faster siphash for AVX2 supporing CPUs (used in fuzzy hashes)
  4928. * PCRE2 support
  4929. * Allow quoted and slashed keys in map
  4930. * Add proper support of DNS resolvers balancing (#552)
  4931. * Rework includes and configuration system for better local changes support
  4932. * New keypairs framework for signing and encryption
  4933. * Added support for dynamic modules and workers
  4934. * Allow to dump configuration with help comments
  4935. * Rework once_received module
  4936. - Fix priority for `good_hosts`
  4937. - If a good host has been found do not add once_received symbols
  4938. - Fix priorities for strict once_received
  4939. - Add ability to whitelist IP addresses
  4940. * Implement support of signed maps for HTTP and file maps
  4941. * Add command to sync fuzzy storage (#533)
  4942. * Rework system of symbols and actions registration
  4943. It is possible now to use priorities when adding symbols to metrics and
  4944. override scores for symbols with lower priority with the scores with
  4945. high priority.
  4946. * Add auth support and db selection for redis stats
  4947. * Improve composite rules application
  4948. * Add ignore_received option
  4949. * Fix critical issue with inconsistent resorting
  4950. * Fix `all` in spf redirects
  4951. * Add punycoded versions for IDN domains (#554)
  4952. * Improve sorting order for symbols cache
  4953. * Add lockless logging for processes management
  4954. * Allow to specify flags for metric symbols
  4955. * Load images height and width from style attribute (#538)
  4956. * Override DNS requests limits for SPF and DKIM
  4957. * Fix resetting symbols to their default values in WebUI
  4958. * Improve configuration agility for redis stats
  4959. * Allow to set db and password for redis in stat_convert
  4960. * Import the latest libucl
  4961. * LUA API changes:
  4962. - Add rspamd_version function to LUA API
  4963. - Add lua_cryptobox module
  4964. - Add lua_map module
  4965. - Add task:set_metric_action lua API method
  4966. - Fix race condition in lua_tcp module
  4967. - Fix a lot of issues in lua_redis module
  4968. - Rework and abstract lua maps API
  4969. - Add util.strlen_utf8 lua function
  4970. - Add lua functions for caseless comparison
  4971. - Allow optional symbols registration
  4972. - Add config:add_map table form method, add regexp maps
  4973. - Add task:has_urls method
  4974. - Add task:has_flag method
  4975. - Add html tags methods to lua_html
  4976. - Add task:get_dns_req
  4977. * Plugins changes:
  4978. - Add support for WLBLEval SA plugin
  4979. - Use caseless comparison in SA and DMARC plugins
  4980. - Allow SA plugin to set scores for rspamd symbols
  4981. - Add regexp maps support to multimap
  4982. - Allow filenames match in multimap
  4983. - Add more filters for the existing map types
  4984. - Fix html images rules to reduce FP rates
  4985. * New rules:
  4986. - LONG_SUBJ - too long subject
  4987. - MIME_BAD_ATTACHMENT - bad attachment type
  4988. - RDNS_NONE - no reverse DNS record for sender's IP
  4989. - Fix MISSING_MIMEOLE rule for modern OE
  4990. * Many other bugfixes, memory leaks plugs thanks to:
  4991. - Coverity scan
  4992. - New gcc-6 warnings
  4993. - valgrind manual iterations
  4994. * Documentation improvements:
  4995. - FAQ list: https://rspamd.com/doc/faq.html
  4996. - Reworked quick start guide
  4997. - Added documentation for all active modules
  4998. * Other changes:
  4999. - Dropped Ubuntu Vivid support
  5000. - Added Ubuntu Xenial support
  5001. - Rework build system for rspamd and rmilter
  5002. 1.1.4:
  5003. * Print traceback on lua errors in lua config
  5004. * Fix leaks in lua error paths
  5005. * Improve 'R_EMPTY_IMAGE' rule
  5006. * Fix metas memoization in SA plugin
  5007. * Properly set `flag` in fuzzy replies
  5008. * Fix arguments order
  5009. * Fix issue with out-of-boundary reading
  5010. * Fix issues found by coverity
  5011. * Same result checking error found by coverity
  5012. * Fix varargs processing (found by coverity)
  5013. * Fix error in printing hex
  5014. * Reduce weights for some hfilter patterns
  5015. * Add aliases for task:get_from_ip:
  5016. - task:get_addr
  5017. - task:get_from_addr
  5018. - task:get_ip
  5019. * Rework once_received module
  5020. - Fix priority for `good_hosts`
  5021. - If a good host has been found do not add once_received symbols
  5022. - Fix priorities for strict once_received
  5023. - Add ability to whitelist IP addresses
  5024. * Fix `MISSING_MIMEOLE` rule for modern OE
  5025. * Treat meta tags as embedded tags (#501)
  5026. 1.1.3:
  5027. * Fix DSN rules when SMTP from is unavailable
  5028. * Fix statconvert routine to avoid lua module usage
  5029. * Set a sane quark for configtest to avoid NULL to be printed in logs
  5030. * Support c11 if available
  5031. * Fix parsing of ip:port strings
  5032. * Add more diagnostic for lua subr errors
  5033. * Fix task:set_from_ip lua method
  5034. * Add basic routines for digital signatures
  5035. * Add tool for digital signatures
  5036. * Add plain open file API method for atomic open
  5037. * Fix parsing nested braces inside logger vars
  5038. * Pre filters now actually skip processing
  5039. * Add pre-filter mode for multimap
  5040. * Switch to apache 2 license
  5041. 1.1.2:
  5042. * Fix stat_cache closing
  5043. * Add checkpoints to sqlite3 learn cache
  5044. * Do not recompile lua generated headers all the time
  5045. * Increase number of messages learned
  5046. * Fix issues with dual stack and hfilter
  5047. * Disable MID checks for hfilter by default
  5048. * Fix cache definitions in multiple classifier and no type
  5049. * Don't crash if learn cache failed to initialize
  5050. * Fix googlegroups support in maillist plugin
  5051. * Rework flags LUA API:
  5052. - Allow to check for a specific flag
  5053. - Add `learn_spam`, `learn_ham` and `broken_headers` flags
  5054. - Unify internal functions
  5055. * Add `BROKEN_HEADERS` rule
  5056. * Add support for forged confirmation headers (by @AdUser)
  5057. * Allow `any`, `mime` and `smtp` for get_from/get_recipients
  5058. * Add mime types checking plugin
  5059. * Add rule to detect spammers attempts to cheat mime parsing
  5060. * Rework parsing of IP addresses in configuration (better IPv6 support)
  5061. * Add `util.parse_mail_address` function to LUA API
  5062. * Add lua sqlite3 module
  5063. * Implement synchronous redis call
  5064. * Ratelimit: avoid possible indexing of nil value (Fixes #498) (by @fatalbanana)
  5065. * Add stat_convert command to convert stats tokens from sqlite3 to redis
  5066. * Implement redis advanced lua api with pipelining
  5067. * Fix memory leak on redis stat (#500)
  5068. * Fix user/language learn count in sqlite statistics (#496) (by @fatalbanana)
  5069. * Fix build with custom pcre
  5070. * Fix fuzzy relearning (#498)
  5071. * Improve planning of asynchronous tasks
  5072. * Show slow rules in log
  5073. * Add warning for slow regexps
  5074. * Add base32 decode/encode routines to lua util
  5075. * Allow converting of learn cache from sqlite to redis
  5076. * Add methods to check if a messages has from/rcpts
  5077. * Improve and fix multimap plugin:
  5078. - Restore 'header' maps
  5079. - Add filters for headers
  5080. - Add 'email:addr', 'email:user', 'email:domain' and 'email:name' filters
  5081. - Add generic regexp filters
  5082. * Disable reload command in rc scripts
  5083. * Improve runtime CPU dispatcher for libcryptobox
  5084. * Add preliminary support of digital signatures via ed25519
  5085. * Add detection for RDRAND support
  5086. * Print configuration of crypto on start
  5087. * A in SPF presumes AAAA lookup as well
  5088. 1.1.1:
  5089. * Fix duplicated XBL symbol
  5090. * Reduce log severity for ratelimit missing servers
  5091. * Fix XBL composite to avoid duplicate symbols
  5092. * Reduce weight of URL_ONLY rule due to FP rate
  5093. * Disable fuzzy hashes from the metadata for now
  5094. * Fix processing of empty messages (#486)
  5095. * Always treat DNS timeouts as temporary fail for SPF
  5096. * Fix issue with SPF double IP stack (#483)
  5097. * Use X-Forwarded-For when checking secure_ip (#488)
  5098. * Fix hash calculation for sqlite stats
  5099. * Fix memory corruption on punycode
  5100. * Fix strings allocation in punycode
  5101. * Fix error message (#491)
  5102. 1.1.0:
  5103. * Incompatible change: sqlite3 and per_user behaviour:
  5104. Now both redis and sqlite3 follows the common principles for per-user
  5105. statistics:
  5106. 1) If per-user statistics is enabled check per-user tokens ONLY
  5107. 2) If per-user statistics is not enabled then check common tokens ONLY
  5108. If you need old behaviour, then you'd need to use separate classifier
  5109. for per-user statistics.
  5110. * Implement redis statistics backend and cache
  5111. * Implement autolearning for statistics
  5112. * Reworked statistics architecture from scratch
  5113. * Add hyperscan (https://github.com/01org/hyperscan) engine for regular
  5114. expressions:
  5115. - add lazy loader for hyperscan databases
  5116. - rework regexp cache to have joint pcre/hyperscan scanning
  5117. - implement hyperscan pre-filter support
  5118. - add compilation guards for bad expressions
  5119. - implement `rspamadm control recompile` command
  5120. - implement hyperscan cache monitoring
  5121. - slides: <https://highsecure.ru/rspamd-hyperscan.pdf>
  5122. * Implement flexible task logging
  5123. * Rework fuzzy worker:
  5124. - it is now possible to run multiple fuzzy workers;
  5125. - implement lazy writing as sqlite3 is bad at concurrent writing;
  5126. - add retries for simple sql commands in fuzzy backend;
  5127. - use fine-grained transactions for fuzzy;
  5128. - implement new multi-pubkeys mode;
  5129. - allow encrypted only storages;
  5130. - rework statistics for fuzzy;
  5131. - add `rspamadm control fuzzystat` command for extended statistics;
  5132. - implement human readable output for the previous command;
  5133. - add condition script for learning fuzzy storage;
  5134. * Various fixes to SPF:
  5135. - fix `redirect` records;
  5136. - fix domains when parsing mx/ptr/a records in includes/redirects;
  5137. - fix issues with multiple addresses in SPF records;
  5138. - ignore SPF results in case of DNS failure;
  5139. - adjust TTL of records when resolving subelements of SPF records;
  5140. - always select `v=spf1` line if it is available
  5141. - do not cache records with DNS failure in subrequests;
  5142. - ignore records with temporary fails during subrequests resolving;
  5143. - fix `RDNS_RC_NOREC` support;
  5144. * Add clang plugin for static analysis:
  5145. - implement static checks for `rspamd_printf` format strings;
  5146. * Add 'allow_raw_input' option for non-mime messages
  5147. * Recognize types using libmagic
  5148. * Fix parsing of IPv6 received headers
  5149. * Add new interface of communication between workers in rspamd
  5150. * Add support for named socketpairs
  5151. * Don't write URLs by default as it is too verbose
  5152. * Set status for HTTP replies
  5153. * Try load `rspamd.conf.override`
  5154. * Implement words decaying for text parts to limit many checks
  5155. * Improve support of SA rules and plugins:
  5156. - add check_for_shifted_date and check_for_missing_to_header eval rules;
  5157. - add 'check_relays_unparseable' support;
  5158. - add `check_for_mime('mime_attachement')` function;
  5159. - use new re_cache interface for all SA rules;
  5160. - add support for `Mail::SpamAssassin::Plugin::MIMEHeader`;
  5161. - add support of 'special' SA headers to `exists` function;
  5162. - fix issue when SA metas contain other metas;
  5163. - fix freemail rules;
  5164. * Many fixes to the URL parser
  5165. * Match any newline character in regexps
  5166. * Fix resolving of upstreams and detection of poor IPv6 configurations
  5167. * Parse upstreams selection algorithm from the configuration line
  5168. * Add `reresolve` command to the control interface
  5169. * Generate fuzzy hashes from task metadata (URLs and headers)
  5170. * Add method to check if IP is local and `local_addrs` option
  5171. * Implement forced timeout for delayed filters
  5172. * Disable fast path of pcre-jit as it seems to be broken
  5173. * Bayes fixes:
  5174. - new normalizer function;
  5175. - really use weights of tokens from the OSB algorithm;
  5176. - restore multiple classifiers support;
  5177. * Rules changes:
  5178. - add `R_SUSPICIOUS_URL` rule that detects obfuscated URL's;
  5179. - improve empty image rule;
  5180. - rework `FORGED_RECIPIENTS` rule;
  5181. - reduce weight of `SUSPICIOUS_RECIPS`;
  5182. - fix `*_NORESOLVE_MX` symbols in hfilter;
  5183. - add `SUBJ_ALL_CAPS` rule with support of UTF8
  5184. - add spamhaus SBL to uribl
  5185. - fix `SUSPICIOUS_RECIPS` and `SORTED_RECIPS` rules
  5186. - remove `R_TO_SEEMS_AUTO` as it generates a lot of FP;
  5187. - add new Message-ID regexp for Thunderbird (by @moisseev);
  5188. * Plugins changes:
  5189. - allow ratelimit plugin to set symbol instead of pre-result
  5190. - support IP DNS black lists for URIBL (e.g spamhaus SBL);
  5191. - drop deprecated SURBL bits (by @fatalbanana)
  5192. - rename `JP_SURBL_MULTI` to `ABUSE_SURBL` (by @fatalbanana)
  5193. - add `SURBL_BLOCKED` (by @fatalbanana)
  5194. - add `CR_SURBL`
  5195. - SURBL: allow fallthrough to default symbol (by @fatalbanana)
  5196. - Settings: fix IP match (by @fatalbanana)
  5197. - SURBL: add missing symbols to metric (by @fatalbanana)
  5198. - allow processing images urls for SURBL
  5199. - unconditionally disable SPF for authenticated users and local networks
  5200. * Rework ratelimit plugin
  5201. - switch to `rates` instead of old and stupid strings to setup;
  5202. - check if a bucket is zero and disable the corresponding limits'
  5203. - turn off all buckets by default;
  5204. - check either `rcpt` or `user` buckets, not all together'
  5205. - document new `rates` and `symbol` options;
  5206. - inform user about what buckets are used in the configuration;
  5207. * Add neural network **experimental** plugin
  5208. * Add a sample script to learn neural network from rspamd logs
  5209. * Add documentation strings support to rspamd:
  5210. - add strings for the main configuration options;
  5211. - document workers options;
  5212. - add internal plugin options;
  5213. - create `rspamadm confighelp` routine;
  5214. - implement human readable output for the previous command;
  5215. - add subtree search support;
  5216. - add keyword search support;
  5217. * Documentation improvements, tutorials section, statistics description
  5218. * Many other minor and major bugfixes not noted here
  5219. 1.0.11:
  5220. * Fix spf redirects
  5221. * Fix domains when parsing mx/ptr/a records in includes/redirects
  5222. * Fix unfolded base64 encoding
  5223. * Fix GError use-after-free
  5224. * Do not rewrite the original url when using redirector
  5225. * Fix parsing of fragment in urls
  5226. * Fix processing of HTML tags
  5227. * Improve empty image rule
  5228. * Avoid long double type
  5229. * Fix tokens weights in OSB algorithm
  5230. * Improve debugging for bayes
  5231. 1.0.10:
  5232. * Fix settings application (#416)
  5233. * Fix another issue with fixed strings
  5234. * Fix hash function invocation
  5235. * Use the proper string for make_dns_request in lua_http
  5236. * Fix scan time output
  5237. * Update webui:
  5238. - fix labels for greylisting
  5239. - fix dimension of scan time
  5240. 1.0.9:
  5241. * Emergency fix in keyed blake2 to fix fuzzy hashes and encrypted password
  5242. * Support passwords longer than 64 symbols
  5243. 1.0.8:
  5244. * Add function to traverse AST atoms
  5245. * Allow dependencies on rspamd symbols for SA metas
  5246. * Fix memory corruption when timeout is removed in fuzzy check
  5247. * Fix encrypted fuzzy add processing
  5248. * Avoid use-after-free in controller session destructor
  5249. * Use session pool instead of task pool in fuzzy check
  5250. * Fix assembly in i386 mode (#413, #412)
  5251. 1.0.7:
  5252. * Plugged memory leaks in internet address object & html parser
  5253. * Fixed static build
  5254. * Fixed multiple sigchld processing
  5255. * Fixed deletion of signal events after event processing loop
  5256. * Fixed build on ARM (#404 - reported by @Gottox)
  5257. * Fixed setting the default mask for SPF.
  5258. * Fixed sanitisation of HTTP query values
  5259. * Fixed parsing of the last header in encrypted HTTP messages
  5260. * Additions and fixes for test suite & benchmarks
  5261. * Added openssl aes-256-gcm support to libcryptobox & HTTP server
  5262. * Implemented support for starting multiple HTTP servers
  5263. * Implemented batch accept in HTTP server
  5264. * Added module to get data from HTTP headers (#285 - reported by @msimerson)
  5265. * Added `rspamadm control` command
  5266. * Added ability to sort counters output.
  5267. * Added ability to specify custom headers for rspamc client
  5268. * Fix architecture detection
  5269. * Converted history storage to the UCL format
  5270. * Allow flexible number of rows in history
  5271. * Fix action badges in WebUI
  5272. * Add universal cryptobox hash API
  5273. * Migrated to the optimized blake2b implementation adopted from Andrew Moon
  5274. * Allow explicit loading of specific modules
  5275. * Always load settings module
  5276. * Allow to add symbols from settings
  5277. * Fix double free in the controller fuzzy learn command
  5278. * Avoid endless loop when cannot open sqlite db
  5279. * Updated libucl
  5280. 1.0.6:
  5281. * Fix build on i386
  5282. * Update CentOS7 service file patch (by @fatalbanana)
  5283. * Fix path to rspamadm in Debian init script (by @fatalbanana)
  5284. * Fix broken '_SC_GETPW_R_SIZE_MAX' on FreeBSD
  5285. * Fix portability issues
  5286. * Use cryptobox chacha for libottery
  5287. * Better support of 32 bit builds
  5288. * Fix header name tokens setup
  5289. * Fix levenstein distance method for words
  5290. * Add workaround for old libevent (#400)
  5291. * Fix microseconds in termination timer
  5292. * Fix some more issues with fixed strings
  5293. * Explicitly test CPU instructions even after CPUID call
  5294. * Do not check out of boundary memory
  5295. * Do not output broken emails
  5296. * Fix unknown symbols registration
  5297. * Handle SIGILL using longjmp
  5298. * Block signals when exiting event loop
  5299. * Fix incorrect allocation size
  5300. * Slightly optimize alignment
  5301. * Restore rspamd -t for compatibility
  5302. * Add more sanity checks for emails
  5303. 1.0.5:
  5304. * Add rspamd control interface:
  5305. - support `stat` command to get runtime stats of rspamd workers
  5306. - support `reload` command to reload runtime elements (e.g. sqlite3 databases)
  5307. * Rework curve25519 library for modular design:
  5308. - add Sandy2x implementation by Tung Chou
  5309. - fix CPU detection for variables loading assembly
  5310. - add testing for curve25519 ECDH
  5311. * New fixed strings library
  5312. * Add `R_SUSPICIOUS_IMAGES` rule
  5313. * Enable mmap in sqlite3
  5314. * Use new strings in the HTTP code
  5315. * Improve google perftools invocation
  5316. * Improve performance profiling in http test
  5317. * Reorganize includes to reduce namespace pollution
  5318. * Allow specific sections printing in configdump command
  5319. * Rework workers signals handlers to be chained if needed
  5320. * Update socketpair utility function
  5321. * Add control_path option for rspamd control protocol
  5322. * Fix ownership when listening on UNIX sockets
  5323. * Rework signals processing in main
  5324. * Remove extra tools from rspamd (they live in rspamadm now)
  5325. * Remove global rspamd_main
  5326. * Add global timeout for the overall task processing (8 seconds by default)
  5327. * Sanitize NULL values for fuzzy backend
  5328. * Store NM between encrypt/decrypt
  5329. * Add textpart:get_words_count method
  5330. * Fix generic DNS request in lua
  5331. * Tune hfilter weights
  5332. * Add support of IPv6 in hfilter
  5333. * Fix parsing of HTTP headers with IP addresses
  5334. * Sync with the recent libucl
  5335. * Various minor bugfixes
  5336. 1.0.4:
  5337. * Add configdump routine to rspamadm
  5338. * Implement retransmits for fuzzy_check plugin
  5339. * Fix events processing for learning anf checking fuzzy hashes
  5340. * Avoid dependency on unneeded and uncompatible glib include
  5341. * Add `historyreset` command to the controller
  5342. * Fix loading of tokenizer config from dump (#389)
  5343. * Add sorting hints for the history
  5344. * Allow custom lua scripts for users/languages extraction (#388)
  5345. * Do not add FORGED_RECIPIENTS when 'To' is missing (#387)
  5346. * Do not add R_UNDISC_RCPT when 'To' is missing (#387)
  5347. * Add encryption to fuzzy check plugin
  5348. * Add encryption for fuzzy storage
  5349. * Add new epoch for encrypted fuzzy request
  5350. * Add encryption for `rspamd.com` storage
  5351. * Remove gmime processing for LDA mode as it is deadly broken
  5352. * Add routine to find end of headers position in mime messages
  5353. * Fix LDA headers folding
  5354. * Init libraries in rspamc client as well to avoid locale issues
  5355. * Avoid collision with locally installed includes
  5356. * Allocate and free memory with the same allocator in rspamadm (#385)
  5357. * Preserve expired fuzzy hashes counter
  5358. * Improvements in webui:
  5359. - Add favicon.ico
  5360. - Rework history table
  5361. - Fix sorting for the history
  5362. - Migrate to bootstrap 3 and jquery 2
  5363. - Fix css bugs
  5364. - Add glyphicons
  5365. - Add reset history
  5366. - Improve history buttons
  5367. - Redraw graph to avoid display issues
  5368. - Webui is now MIT licensed to match licensing policy of rspamd
  5369. 1.0.3:
  5370. * Fix piechart clean slice (#380)
  5371. * Fix controller crashes when GString is reallocated (#381)
  5372. * Correctly set locale before start
  5373. * Set C locale for numeric values
  5374. * Add rspamadm routine:
  5375. - add `pw` command to manage passwords
  5376. - add `help` command for displaying help
  5377. - add `configtest` command to check configuration files
  5378. - add `keypair` command for generating encryption keys
  5379. - add `fuzzy_merge` routine to merge fuzzy sqlite databases
  5380. - add a simple manual page for rspamadm
  5381. * Allow metric registration for composite expressions
  5382. * Add strict mode for configtest
  5383. * Add logger counters
  5384. * Save and show learned messages count (#383)
  5385. * Add `no_stat` flag
  5386. * Add `task:set_flag` and `task:get_flags` (#382)
  5387. * Enable foreign keys in sqlite3
  5388. * Remove orphaned shingles from fuzzy storage
  5389. * Optimize synchronization steps for fuzzy storage
  5390. * Allow delayed conditions registration
  5391. * Add lua API for conditions registering
  5392. 1.0.2:
  5393. * Fix critical bug in webui that prevents password from being sent
  5394. * Rework webui view:
  5395. - Switch to d3.js for graphs
  5396. - Improve piechart look
  5397. - Rework colors for piechart
  5398. - Fix layout for symbols
  5399. - Fix refresh button
  5400. * Add descriptions for whitelist maps
  5401. * Fix build on arm (#379)
  5402. * Fix issue with the last element in the radix trie
  5403. * Add more tests for radix trie algorithm
  5404. * Allow to extract URLs from query strings of other URLs (#361)
  5405. * Initialize rrd fields before writing to file
  5406. * Fix double free if no password has been specified
  5407. 1.0.1:
  5408. * Add writing to rrd from the controller
  5409. * Fixed lots of bugs in rrd code
  5410. * Adopt new DNS API in hfilter plugin (by @AlexeySa)
  5411. * Allow only one controller process to manage rrd file
  5412. * Set event base for fuzzy calls
  5413. * Improve fuzzy IO errors logging
  5414. * Add rra extraction function to rrd library
  5415. * Add graph handler to the controller
  5416. * Cache correct passwords to avoid too high CPU usage when working with webui
  5417. * Controller sockets are owned by router do not export them to task
  5418. * Optimize logging by skipping hash table search if it's empty
  5419. * Fix loading issue with broken statfiles
  5420. * Print assertions from glib to rspamd logger
  5421. * Load legacy `lua/rspamd.local.lua`
  5422. * Update webui with some fixes to learning and scanning
  5423. 1.0.0:
  5424. * Rework symbols processing:
  5425. - Improve sorting logic for symbols
  5426. - Organize processing into multiple stages
  5427. - Added asynchronous watchers for symbols
  5428. - Added ability to organize dependencies between symbols
  5429. * Fixed URL redirector:
  5430. - Use optimized POE loop
  5431. - Organize dependencies
  5432. - Fix startup
  5433. * New sqlite3 backend:
  5434. - Allow to have per-languages and per-user statistics
  5435. - Allow sqlite3 to be used as statistics backend
  5436. * Store tokenizer configuration within statfiles
  5437. * Improve bayes statistics:
  5438. - Use headers and images metainformation in bayes
  5439. - Suggest using of pre-processed tokens for statistics
  5440. - Fix tokens normalization for OSB algorithm
  5441. * Rewrite url parsing:
  5442. - Fix numerous issues with url extraction and normalization
  5443. - Fix mailto urls
  5444. * Fix settings plugin to allow custom actions scores
  5445. * Improve rbl plugin
  5446. * Allow capturing patterns in rspamd lua regexp library
  5447. * Add GTUBE support
  5448. * Fix spamc legacy support
  5449. * Add DKIM support to RBL module
  5450. * Fix issues with multiple DKIM signatures
  5451. * Fix issue if rspamd cannot create statfiles (#331)
  5452. * Rework parts and task structure:
  5453. - Now text_parts, parts and received are arrays
  5454. - Pre-allocate arrays with some reasonable defaults
  5455. - Use arrays instead of lists in plugins and checks
  5456. - Remove unused fields from task structure
  5457. - Rework mime_foreach callback function
  5458. - Remove deprecated scan_milliseconds field
  5459. * Add ip_score plugin support (not enabled by default):
  5460. - Can check for asn/country and network using DNS lookups
  5461. - Can store and load reputation from redis server
  5462. * Improve PARTS_DIFFER rule to count merely different words
  5463. * New HTML parser:
  5464. - Parses HTML parts using a set of state machines
  5465. - Extracts useful data and exports it to lua functions:
  5466. + Styles
  5467. + Images
  5468. + URLs
  5469. + Colors
  5470. + Structure elements
  5471. - Added HTML rules for some checks
  5472. * New version of LUA DNS API
  5473. * Table versions of many functions in LUA API
  5474. * Improve rspamc client:
  5475. - Print execution time
  5476. - Allow executing of external commands and passing output to them
  5477. - Allow mime output mode when rspamc alters message according to rspamd
  5478. checks and send it to an external command or stdout
  5479. * Allow scanning of local files using HTTP requests
  5480. * Rework configuration system:
  5481. - Rules are now moved from the $CONFDIR to $RULESDIR to avoid ambiguity
  5482. - All modules configurations are now split in $CONFDIR/modules.d/* to
  5483. simplify upgrades
  5484. - Move hfilter to plugins
  5485. - Allow plugins and rules to define default scores to simplify metrics
  5486. setup
  5487. - Include overrides for all modules to honor local/automatic parameters
  5488. - Tune scores for many modules
  5489. * Rework and enable DMARC plugin
  5490. * Add whitelist plugin for SPF/DKIM/DMARC based whitelisting
  5491. * Add some common domains to whitelists shipped with rspamd
  5492. * Rework logging:
  5493. - Now each log entry supports module name and a `tag`. Tag is used to
  5494. identify unique objects (such as tasks) when checking log files
  5495. - It is possible to turn on debugging for the specific modules
  5496. - Systemd logging is fixed
  5497. * Improve spamassassin plugin.
  5498. - Now headers are matched more like SA
  5499. - Improve support of Message-ID
  5500. - Add support of ToCc header type
  5501. - Fix :addr and :name in headers regexps
  5502. * Resurrect rrd support code
  5503. * Save controller stats between restarts
  5504. * Fixed tonns of bugs
  5505. * Added tonns of minor improvements and features
  5506. * Added more unit tests
  5507. * Create functional tests framework
  5508. * Added documentation for missing modules
  5509. * Added rpm/deb repositories and scripts
  5510. * Updated WebUI and libucl externals
  5511. 0.9.10:
  5512. * Do not dereference null pointer on learning.
  5513. * Fix some extreme cases in BAYES.
  5514. * Add a workaround to avoid bad HTML messages breaking.
  5515. * Build with -O2 flags by default.
  5516. * Add constraints to limit DNS requests count per task.
  5517. * Add workaround for SURBL DNS flood.
  5518. * Set error if rspamd cannot learn anything.
  5519. 0.9.9:
  5520. * Don't use RWL_SPAMHAUS_WL (unknown result) for whitelisting (by @fatalbanana)
  5521. * Import updated public suffix list (by @fatalbanana)
  5522. * Remove debug message
  5523. * Fix settings (by @fatalbanana)
  5524. * Remove duplicated symbol registration
  5525. * Use WAL for fuzzy storage
  5526. * RBL fixes (by @fatalbanana):
  5527. - silence errors;
  5528. - yield unknown results from RBLs;
  5529. - fix scoring for DNSWL;
  5530. - fix use of RBL name as symbol;
  5531. - ignore RBL names that would not be yielded;
  5532. * Support captures in regular expressions
  5533. * Add captures support to lua_regexp
  5534. * Support dist on FreeBSD and Darwin
  5535. * Add RCVD_IN_DNSWL_NONE as whitelisting exclusion (by @fatalbanana)
  5536. * Multiple fixes to URL detection:
  5537. - support port definition;
  5538. - fix query and path recognition;
  5539. - fix parsing of multiple slashes in URL;
  5540. - fix parsing query just after port;
  5541. - fix path field in `url:to_table` method;
  5542. - improve support of IP based URLs.
  5543. * Set ignore_whitelists = true for RECEIVED_SPAMHAUS_XBL (by @fatalbanana)
  5544. * Add GTUBE support
  5545. * Ignore User header in SA mode
  5546. 0.9.8:
  5547. * Fix critical bug in bayes classifier (#305)
  5548. * Fix critical bug in RBL module (by @fatalbanana)
  5549. * Fix and rework settings plugin.
  5550. * Fix get_all_opts for a case of non-iterable options.
  5551. * Use tld for redirector's matching.
  5552. 0.9.7:
  5553. * Add whitelist_exception setting to RBL module (by @fatalbanana)
  5554. * Don't use RWL_MAILSPIKE_POSSIBLE or DNSWL_BLOCKED for whitelisting (by
  5555. @fatalbanana)
  5556. * Fix extreme cases in bayes classifier.
  5557. * Fix parsing of urls with '?' at the end of hostname.
  5558. * Update interface.
  5559. * Fix number of issues with webui interaction.
  5560. * Fix saving maps.
  5561. * Allow user@ and @domain matches in multimap.
  5562. * Fix issues with bounces From processing.
  5563. * Fix abs/fabs misuse.
  5564. * Fix builds on suse and arch linux distributions.
  5565. 0.9.6:
  5566. * Fix memory leak if mime cannot be parsed.
  5567. * Fix dkim cache expiration.
  5568. * Fix issues with redirector HTTP response.
  5569. * Fix abnormal connection closing with certains messages with a high score
  5570. (issue #296)
  5571. * Fix redirector installation.
  5572. * Use specific POE loop for some systems.
  5573. * Fix number of issues in URL redirector.
  5574. * Fix selecting URLs for sending to redirector.
  5575. 0.9.5:
  5576. * Avoid double free when extending HTTP message.
  5577. * Fix double free if multiple classifiers are defined.
  5578. * Fix misprint in spamassassin plugin.
  5579. * Fix cpuid invocation on i386.
  5580. * Fix ownership issues for zero-copy decode.
  5581. * Allow __len metamethod on rspamd{text}.
  5582. * Add base64 decoding lua utility.
  5583. * Fix build on FreeBSD
  5584. * Skip spaces at the beginning of mime messages.
  5585. * DBL_ABUSE_REDIR should not have significant weight.
  5586. * Allow to split by lua_regexp rspamd{text} objects.
  5587. * Allow to specify custom stop pattern for lua_tcp.
  5588. 0.9.4:
  5589. * Fix critical bugs in tokenization algorithm
  5590. * Write unit tests for tokenization
  5591. * Add documentation for lua_tcp
  5592. * Switch off legacy tokenization by default.
  5593. * Fix critical bugs in words normalization
  5594. * Add lua bindings to tokenizer.
  5595. * Implement storing of HTTP headers inside task
  5596. * Add lua API to accerss HTTP headers data
  5597. * Implemented base64 encoding suitable for MIME
  5598. * Use caseless hash and equal functions for HTTP request headers.
  5599. * Improve debian architectures support (by @dottedmag)
  5600. 0.9.3:
  5601. * Revert incorrect regexp change that broke the default rules
  5602. * Fix lua_tcp module
  5603. 0.9.2:
  5604. * Fix error on spawning unique workers.
  5605. * Add preliminary version of generic LUA TCP requests API.
  5606. * Use lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc)
  5607. * Fix fuzzy mime strings with only type.
  5608. * Improve thunderbird sanity checks.
  5609. * Fix critical bug on matching regular expressions.
  5610. * Make hiredis optional dependency.
  5611. * Fix multiple bugs in daemon reloading
  5612. 0.9.1:
  5613. * Restore utf8 validation for regular expressions to avoid crashes
  5614. * Fix symbols displaying in the interface
  5615. * Add symbol groups to the interface
  5616. * Fix maps ID parsing in the controller
  5617. * Add multimap and regexp modules documentation
  5618. * Backport fixes from libucl
  5619. * Fix debian package (by @dottedmag)
  5620. * Rework XXH32 invocations
  5621. 0.9.0:
  5622. * Add support of the fast and secure protocol level encryption:
  5623. - curve25519 is used for key exchange;
  5624. - chacha20/poly1305 cryptobox construction for bulk encryption;
  5625. - zero latency overhead;
  5626. - encrypting and balancing HTTP proxy worker
  5627. * Rework expressions and create new expressions library:
  5628. - aggressive optimizations based on the abstract syntax tree;
  5629. - abstract expressions support (regular expressions, functions, lua modules
  5630. composites and so on)
  5631. - New comparison and '+' operators support
  5632. - New greedy algorithm to minimize execution time of expressions and
  5633. all symbols
  5634. - Dynamic expressions benchmark and reoptimizations
  5635. * Many improvements to the LUA API:
  5636. - reworked logger module allowing to do pretty print of the most of lua
  5637. types (including tables and userdata classes)
  5638. - reworked lua redis and lua HTTP to support more features
  5639. - added opaque type for passing large text chunks without copying
  5640. - new regexp module with many auxiliary functions (e.g. `re:split`)
  5641. * LuaJIT is now the default requirement for rspamd allowing to speed up lua
  5642. execution by a large margin (however, plain lua is still supported)
  5643. * New plugins:
  5644. - spamassassin rules plugin that allows to load and re-use the most of
  5645. SA rules natively
  5646. - DMARC plugin that evaluates SPF and DKIM policies to the domain policies
  5647. - many old plugins has been reworked to implement new features and improve
  5648. stability
  5649. * New aho-corasic trie implementation from @mischasan that allows to load and
  5650. use hundreds of thousands of patterns with no influence on load
  5651. * Support of PCRE JIT and PCRE JIT fast path modes that significantly improves
  5652. the performance of regular expressions if supported by PCRE
  5653. * New URLs parser and extractor:
  5654. - removed legacy code that was useless for url finding
  5655. - reworked algorithms of URL parsing for more precise and accurate results
  5656. - added top-level-domains tree from http://publicsuffix.org
  5657. - improved emails parsing
  5658. - removed many phishing false positives due to TLD tree check
  5659. * New statistics infrastructure:
  5660. - created a separate layer of statistic library
  5661. - improved OSB-Bayes by re-weighting tokens according to the original
  5662. academic paper and `crm114` implementation, which reduced false positives
  5663. rate significantly
  5664. - created learn cache to avoid double learning of statistics and providing
  5665. an efficient way to re-learn class for a message
  5666. - created abstract layers for different statistics backends
  5667. - implemented new tokenization algorithms with fast or secure (siphash)
  5668. hashes to generate statistics features
  5669. * Reworked utf8 tokenization that previously corrupted all UTF8 words (minor
  5670. incompatibility with old fuzzy hashes with utf-8 symbols)
  5671. * SPF module has been completely rewritten to support complex cases of
  5672. `include` and `redirect` within SPF records
  5673. * DKIM module now supports multiple signatures
  5674. * Controller passwords can now be stored encrypted by `PBKDF2-HMAC` in the
  5675. configuration file
  5676. * Many hand-written HTTP clients has been replaced with the common rspamd
  5677. http module
  5678. * New test framework:
  5679. - import lua `telescope` test framework
  5680. - add unit tests for many rspamd modules and routines
  5681. - create a unit test for each possible bug found
  5682. - use luajit ffi for testing C code
  5683. - added preliminary support of functional testing by creating tasks from lua
  5684. * Randomize hash seed to avoid certain hash tables vulnerabilities
  5685. * Documentation improvements:
  5686. - added documentation for the vast majority of rspamd modules
  5687. - added documentation for rspamd protocol
  5688. - added documentation for the most of rspamd LUA extensions
  5689. * Fixed tonns of bugs and memory leaks
  5690. * Added tonns of minor features
  5691. 0.8.3:
  5692. * Various critical fixes in distribution (by @dottedmag and @fatalbanana)
  5693. * Fixed bugs in url detector to parse certain patterns
  5694. * Add default host and helo for a client
  5695. * Some sanity checks for tokenizer and classifier
  5696. * Reiterate on systemd support
  5697. * Fix missing symbol registration
  5698. * Add support of spamc compatible output
  5699. * Filter double-dots in rbl.lua validate_dns (by @fatalbanana)
  5700. * Update ucl submodule due to critical bugfix
  5701. 0.8.2:
  5702. * Create fuzzy db if it does not exist
  5703. * Fix: Centos init script: configtest() (by @AlexeySa)
  5704. * Enable one_shot for RECEIVED_SPAMHAUS_XBL - Fixes #102 (by @fatalbanana)
  5705. * Update Exim patch (by @fatalbanana)
  5706. * Fix processing of unix sockets.
  5707. * Allow applying settings to authenticated users (by @fatalbanana)
  5708. * Make settings priorities work as documented (by @fatalbanana)
  5709. * Fix race condition in symbols planner
  5710. * Add DNSWL_BLOCKED symbol (by @fatalbanana)
  5711. * Make Exim pass usernames to rspamd (by @fatalbanana)
  5712. * Update RBL module (by @fatalbanana):
  5713. - fix indentation;
  5714. - collapse loops;
  5715. - avoid calling for un-needed information;
  5716. - allow disabling RBLs for authenticated users
  5717. * once_received.lua: Fix indentation & add exclusion for authenticated users (by @fatalbanana)
  5718. * hfilter.lua: Add exclusion for authenticated users (by @AlexeySa)
  5719. * Updates to hfilter rules (by @AlexeySa)
  5720. * Set empty <> user or addr for msgs without FROM (by @eneq123)
  5721. * Fix: attempt to index field '?' (a nil value) (by @eneq123)
  5722. * Fix: if not exist Date-header (by @AlexeySa)
  5723. * Add task:get_content() method.
  5724. * rbl.lua: Ignore private IP space (by @fatalbanana)
  5725. * Allow to check radix maps from lua by rspamd{ip}
  5726. * Make local exclusions configurable per-RBL (by @fatalbanana)
  5727. * Add rspamd_config:radix_from_config() (by @fatalbanana)
  5728. * Support emails dnsbl in rbl (by @fatalbanana)
  5729. * Complete rework of url extraction logic
  5730. * Allow customizations for unix sockets. (fixes #182)
  5731. * Set lua path according to rspamd settings.
  5732. * Import lua-functional for plugins stuff.
  5733. * Completely rewrite multimap plugin in functional style.
  5734. * Fix FORGED_MUA_THUNDERBIRD_MSGID (fixes #186)
  5735. * Check IPv6 addresses at dnswl.org and Spamhaus whitelist (by @fatalbanana)
  5736. * Add lowercase utility for utf8 strings.
  5737. * Various fixes to build system
  5738. * Updated debian configuration infrastructure (by @dottedmag)
  5739. 0.8.1:
  5740. * Add sqlite and perl as dependencies for RPM/Debian packages (by @fatalbanana)
  5741. * Remove whitelist.lua from RPM file list (by @fatalbanana)
  5742. * Make Exim pass hostnames to rspamd (by @fatalbanana)
  5743. * Fix building on Fedora (by @fatalbanana)
  5744. * Add toggle for disabling installation of systemd units on Linux (by @fatalbanana)
  5745. * Fix double format rounding that caused output corruption (reported by @fatalbanana)
  5746. * Revert broken change for destructors ordering that led to memory corruption
  5747. * Do not reset symbols case of settings if parsed from lua (reported by @andrejzverev)
  5748. * Fix build on SunOS (by @wiedi)
  5749. * Fix multiple crashes on broken DKIM DNS records
  5750. * Fix critical issue with composites weights removing
  5751. * Fix memory corruption in composites processing code
  5752. * Ignore non-SPF TXT records when parsing SPF includes
  5753. 0.8.0:
  5754. * New fuzzy check logic:
  5755. - use shingles algorithm for fuzzy matching
  5756. - use blake2 instead of md5 for larger output space
  5757. - combine fuzzy and strict matching
  5758. - allow to organize private storages by means of keys
  5759. - preserve compatibility with previous versions
  5760. * New fuzzy storage:
  5761. - use sqlite instead of own memory based hash tables
  5762. - rework commands interface
  5763. - add conversion from the old format
  5764. - add fuzzy match by shignles
  5765. - support old rspamd versions
  5766. * Add lemmatizing for words used in fuzzy hashes that allows to improve match
  5767. quality by using of the first forms of all words
  5768. * Rework language detection
  5769. * Fix several critical bugs, memory leaks and deadlocks:
  5770. - memory leak in HTML nodes parsing
  5771. - deadlock in logger code
  5772. - deadlock in signals processing
  5773. - crashes in fuzzy_storage
  5774. - crashes in tokenizers if the input was empty
  5775. * Import new libucl with several bugfixes and improvements
  5776. * Support listening on ipv6 addresses only
  5777. * Fix macro expansion in SPF module
  5778. * Several bugfixes in DKIM module
  5779. * Add load headers support for mime parts to the lua API
  5780. * Add documentation for:
  5781. - workers in general
  5782. - fuzzy_storage worker
  5783. - fuzzy_check plugin
  5784. - mimepart and textpart lua API modules
  5785. 0.7.6:
  5786. * Apply boundary fix for dkim simple canonization
  5787. * Fix ping command
  5788. * Return nil if header was not found in lua_task
  5789. * Fix hang in upstreams revive logic
  5790. * Decode entitles when normalizing HTML parts
  5791. * Fix logic of finding URLs in HTML parts
  5792. * Do not include \0 into length of text when performing conversion to utf8
  5793. * Fix raw vs parsed reperesentations
  5794. Raw parts are now:
  5795. - decoded b64/qp, but *NOT* converted to utf-8
  5796. Processed parts are now:
  5797. - converted to UTF-8
  5798. - normalized if needed (e.g. HTML tags are stripped)
  5799. * Rework DKIM canonization to line based
  5800. * Fix fuzzy hashes adding
  5801. * Use more specific hash function for fuzzy
  5802. * Fix leaking of iconv descriptors
  5803. * Fix PTR resolving in lua resolver
  5804. * Rework spf module.
  5805. - Copy data to memory pool as cached record might be destroyed causing
  5806. freed memory being passed to the protocol output (use after free)
  5807. - Allow SPF_NEUTRAL policy to be handled separately
  5808. - Add R_SPF_NEUTRAL to the default config
  5809. * Rework `register_symbols` function
  5810. * Allow to disable components of hfilter
  5811. 0.7.5:
  5812. * Fix owner when creating folder /run/rspamd (by @sfirmery)
  5813. * Fix IP validity checks
  5814. * Decode URLs obtained from HTML tags
  5815. * Fix crash with unweighted upstreams
  5816. * Stop processing headers in parts
  5817. * Set sockaddr.sa_family properly when connectig to upstreams
  5818. * Fix reload issues in surbl and fuzzy_check (reported by @citrin)
  5819. * Fix timeouts in redirector
  5820. * Improve lua errors reporting
  5821. * Fix lua closures processing in libucl
  5822. * Rework calling of lua functions from regexp module
  5823. * Choose raw regexp for raw headers
  5824. * Rework conversion to utf since glib one is broken
  5825. * Ignore SGML style tags in html
  5826. * Fix old bug with non-capturing https urls
  5827. * Fix memory corruption on fuzzy reload (reported by @citrin)
  5828. * Fix percents display in rspamc
  5829. * Fix buffer update for DKIM
  5830. * Do not validate utf for raw headers
  5831. 0.7.4:
  5832. * Fix build under *BSD
  5833. * Detect HAN unicode script
  5834. * Implement language detection heuristic for text parts
  5835. * Fix time output in history
  5836. * Improve piechart coloring
  5837. * Fix \r\n conversion in DKIM module (reported by @citrin)
  5838. * Try to detect systems with no IPv6 support
  5839. * Fix multiple/single values in use settings (reported by @citrin)
  5840. * Rework IP addresses in upstreams:
  5841. - Select ipv4/unix addresses if they exist and use ipv6 for ipv6 only
  5842. upstreams (since the support of ipv6 is poor in many OSes and
  5843. environments)
  5844. - Free IP list on upstream destruction
  5845. - Add test cases for addresses selection
  5846. - Allow adding of free form IP addresses to upstreams
  5847. * Fix endianness in lua_radix search (reported by @citrin)
  5848. * Soft shutdown should also set wanna_die flag (reported by @citrin)
  5849. * Stop use-after-free in event loop termination
  5850. * Fix processing of very short messages in DKIM (reported by @citrin)
  5851. * Detect systems without shared mutexes
  5852. * Fix issues with PTR and MX elements in SPF parser (reported by @citrin)
  5853. 0.7.3:
  5854. * New upstreams code:
  5855. - simplify upstreams API;
  5856. - unify strings parsing in upstreams definition;
  5857. - add configuration options for the upstreams;
  5858. - for failed upstreams re-resolve their addresses;
  5859. - use all resolved addresses for an upstream (round-robin);
  5860. - implement stable hashing and use it by default for upstreams;
  5861. - add unit test for upstreams module.
  5862. * Rework signals processing in all rspamd workers:
  5863. - signals are now processed in the event loop;
  5864. - implement the most common signal handlers for all workers;
  5865. - add callbacks for workers specific signal handlers
  5866. * Fix critical issue with fuzzy storage:
  5867. Fuzzy stroage could not save any hashes on termination due to bugged
  5868. signals handling
  5869. * Fix roll history IP storage
  5870. * Rework ipv4/ipv6 handling in parsing addresses:
  5871. - turn off support of IPV6_V6ONLY socket option;
  5872. - create ipv6 socket prior to ipv4 one to handle systems with v6/v4
  5873. sockets enabled (Linux)
  5874. * Remove CBL as it's wholly included in Spamhaus XBL (by @fatalbanana)
  5875. * Remove nszones.com fake RBL (by @citrin)
  5876. * Fix upstreams interaction for fuzzy_check
  5877. * Verify spf PTR records (reported by @citrin)
  5878. * Fix spf MX records parsing
  5879. * Add compatibility for old libevent (by @yellowman)
  5880. * Sync bugfixes from libucl
  5881. 0.7.2:
  5882. * Convert all maps to the compressed radix trie
  5883. * Allow IPv6 addresses in IP maps
  5884. * Remove dynamic items support from symbols cache
  5885. * Allow hex encoded output of strings
  5886. * Fix bug with control connections count
  5887. * Process fuzzy weight correctly (reported by @fatalbanana)
  5888. * Remove extra reference retain of http connection on error
  5889. * Remove deprecated options from the default config
  5890. * Add `one_shot` attr to metric's symbols
  5891. * Doc: add documentation for metrics
  5892. * Add Upstart job to debian packaging (by @CameronNemo)
  5893. * Config: improve SURBL symbols descriptions (by @citrin)
  5894. * Config: reflect SURBL changes (by @citrin):
  5895. - Outblaze removed, malware moved to separate list:
  5896. http://www.surbl.org/news/internal/MW-malware-sublist-added-to-multi
  5897. * Fix C modules initialization on restart
  5898. * Treat single IP as a single IP in radix lists (reported by @citrin)
  5899. * Do not touch file and core limits if not asked explicitly (reported by @citrin)
  5900. * Improve logging for fuzzy errors
  5901. * Block SIGPIPE for HTTP writing
  5902. * Doc: update manual pages
  5903. * Fix HTTP connection termination
  5904. * Reduce default number of parallel requests to 8
  5905. * Sync with libucl include features
  5906. 0.7.1:
  5907. * Fix typo in stat output.
  5908. * Fix issues with includes crossing with the system includes
  5909. * Restore testing framework
  5910. * Add radix trie test suite
  5911. * Implement new path-compressed radix trie.
  5912. - The performance benefit over the old algorithm is about 1.5 times.
  5913. - Memory usage is significantly lower as well.
  5914. - Now radix trie can accept any IPv4/IPv6 values
  5915. * Various improvements to the memory pools code
  5916. * Fix writing reply to a client when no filters are defined
  5917. * Write base32 encoded fuzzy
  5918. * Fix 'soft reject' action
  5919. * Fix rspamd reload and modules reconfiguration
  5920. * Fix subject rewriting for the default subject
  5921. * Fix states for processing task and pre-filters
  5922. * Fix issues with connection closing
  5923. * Fix crashes in rdns
  5924. * Fix ratelimit pre-filter
  5925. * Update exim patch.
  5926. - Update to the recent exim version
  5927. - Strip extra leading src/ from the patch
  5928. - Remove sendfile since it was broken
  5929. - Fix rspamd spam report for exim
  5930. * Improve documentation
  5931. 0.7.0:
  5932. * Use HTTP protocol for all operatiosn
  5933. * Webui worker is now removed and controller works as webui
  5934. * Allow to serve static files via controller by option `static_dir`
  5935. * Rspamd interface is now a part of rspamd
  5936. * Rspamc client has been rewritten to use HTTP and non-blocking mode
  5937. allowing to start multiple operations simultaneously (see `-n` option)
  5938. * Lua API was completely reworked to satisfy modern standards of LUA:
  5939. * Module `lua-message` was removed
  5940. * Reduced number of superglobals registered by rspamd
  5941. * Many functions has been redesigned
  5942. * Symbols registration is now more convenient
  5943. * Users settings has been rewritten as lua plugin
  5944. * Reworked headers system as gmime's based one misses many headers and is
  5945. very slow to get headers values
  5946. * Reorganized code and removed embedded jannsson by using UCL for all json
  5947. parsing
  5948. * Migrated to `librdns` for DNS resolving that improves concurrency for
  5949. DNS requests significantly
  5950. * Fixed tonns of bugs in MIME processing
  5951. * Improved metrcis and default symbol's weights
  5952. * Added new RBL's
  5953. * Fixed a number of issues in the modules
  5954. * Removed several memory leaks found
  5955. * Fix unicode processing
  5956. * Fix fuzzy checking for unicode parts
  5957. * Significantly improve documentation and especially LUA API docs
  5958. * See migration notes at https://rspamd.com/doc/migration.html
  5959. 0.6.8:
  5960. * Controller now listen for localhost and not for 127.0.0.1 by default
  5961. * Allow FCrDNS-style RBL lookups (by @fatalbanana)
  5962. * Reduce threshold for parts_differ function.
  5963. * Fix hostname lookup for rdns rbl (by @AlexeySa)
  5964. * Fix HFILTER_URL_ONELINE to reduce false positive rate.
  5965. * Fix whitelist module.
  5966. * Allow override system predefined settings without touching system ones
  5967. by .try_include macro (by @andrejzverev)
  5968. * Check for [ip.address]-style HELO and suppress lookups. (by
  5969. @fatalbanana)
  5970. * Optimize hfilter (by @AlexeySa)
  5971. * Fix issue with random numbers generator in dns.
  5972. * Use more clever time values to setup entropy.
  5973. * Synced with the recent libucl.
  5974. 0.6.7:
  5975. * Use ChaCha20 for DNS generator (more secure DNS id)
  5976. * Unknown symbols now has zero weight and not 1.0
  5977. * Fix fuzzy hashes expire time
  5978. * Fix critical issue in statfiles on FreeBSD 9 (and some other platforms)
  5979. * Add .include_map macro to ucl parser
  5980. * Update libucl
  5981. * Fix headers end detection for DKIM module
  5982. * Fix a bug in received headers parser
  5983. * Validate IP addresses before pushing them to lua
  5984. * Start new documentation project
  5985. * Fixed tonns of other minor bugs
  5986. * Start to prepare for 0.7 with HTTP protocol and new settings
  5987. 0.6.6:
  5988. * Removed issue with BUFSIZ limitation in the controller output
  5989. * Simplify logging symbols escaping
  5990. * Adjusted weights for several rules
  5991. * Improve spamhaus rbl support
  5992. * Removed PBL for received headers checks
  5993. * Added hfilter module that performs various HELO and IP checks.
  5994. * Rspamd can now be reloaded by HUP signal
  5995. * Fuzzy storage should expire hashes properly
  5996. * Build system has been reworked for better supportof pkg-config
  5997. * Various minor bugfixes
  5998. 0.6.5:
  5999. * Fixed critical bug in DNS resolver, introduced in 0.6.4
  6000. * Improved multimap and rbl plugins to skip
  6001. * Add dns_sockets option for tuning sockets per server in DNS resolver
  6002. * Improved packages for rspamd
  6003. 0.6.4:
  6004. * Added io channels for DNS request to balance load and reduce id
  6005. collisions chance
  6006. * Fixed a bug in SPF filter that may cause core dump in specific
  6007. circumstances
  6008. * FIxed default config for rbl module
  6009. * It is possible to get a list of rspamc commands with their descriptions
  6010. * Added SORBS bl to the default config
  6011. * 2tld file for surbl module has been significantly extended
  6012. * Perl modules has been removed from the code.
  6013. * Fixed an issue in libucl when parsing macros
  6014. 0.6.3:
  6015. * Fixed issues with DNS:
  6016. - labels decompression algorithm was fixed;
  6017. - added resolve_mx to lua API;
  6018. - fixed modules that use DNS.
  6019. * Lua modules once_received and emails reworked for new resolver API and UCL.
  6020. * Debian package was polished.
  6021. * Fixed a bug in fuzzy_check module that prevents correct processing messages
  6022. without valid parts.
  6023. 0.6.2:
  6024. * Fuzzy check module has been reworked:
  6025. - now fuzzy_check operates with a group of rules, that define which
  6026. servers sre to be checked;
  6027. - it is possible to specify read_only groups to avoid learning errors;
  6028. - turn fuzzy_check to one_shot mode permanently;
  6029. - fuzzy_check module configuration is now incompatible with the previous
  6030. versions.
  6031. * Imported bugfixes from libucl.
  6032. * Fixed whitelist plugin.
  6033. * Fixed statfiles resizing.
  6034. * Improved logging initialization order.
  6035. * Fixed race condition in the controller worker.
  6036. 0.6.1:
  6037. * Critical bugfixes:
  6038. - fixed build system;
  6039. - fixed in_class setting in bayes learning;
  6040. 0.6.0:
  6041. * Use UCL instead xml for configuration (https://github.com/vstakhov/libucl)
  6042. * Fix statistics module normalization
  6043. * Rework the many modules for the new configuration:
  6044. - surbl has incompatible configuration;
  6045. - fuzzy_check has incompatible configuration;
  6046. - multimap has now new configuration;
  6047. - received_rbl is removed and replaced by rbl module.
  6048. * Removed deprecated options:
  6049. - statfile_pool_size;
  6050. - action and required/reject score for a metric.
  6051. * Simplify build system and unify configuration for all platforms.
  6052. * Improved URL detector (reduced false positive rate).
  6053. * Lua 5.2 is now the default and fully supported version.
  6054. * Tons of bugfixes and minor improvements.
  6055. 0.5.6:
  6056. * Fix bug with counters incrementing in rolling history
  6057. * Detect expl and exp2l as some systems do not have it
  6058. * Support input streams without Content-Length
  6059. * Implement counters output via rspamc and controller interface
  6060. * Fix bug with udp sockets in fuzzy storage
  6061. 0.5.5:
  6062. * New bayes normalizator based on inverse chi-square function
  6063. * Various fixes to fuzzy storage
  6064. * Allow update fuzzy storage only from specific IP addresses
  6065. * Better support of IPv6 and address selection algorithms
  6066. * Add CentOS spec file
  6067. 0.5.4:
  6068. * Fixed issues with diff algorithm
  6069. * Added support of RRD statistics
  6070. * Add webui worker for interface interaction
  6071. * Fix a lot of issues with dynamic conf
  6072. * Fix critical memory leak in settings code
  6073. * Improve stability of the system
  6074. 0.5.3:
  6075. * Added dynamic options
  6076. * Added advanced metaclassfication
  6077. * Added RESTfull API for controller
  6078. * Improved hashing algorithms
  6079. * Various fixes for rspamc client:
  6080. - allow interacting with unix sockets
  6081. - librspamdclient major cleanup
  6082. - bayes is now default classifier
  6083. 0.5.2:
  6084. * Added lua bindings for:
  6085. - basic mime parts, that allows checkign attachements for example;
  6086. - DNS resolver;
  6087. * Existing lua bindings now works without task object allowing to use them
  6088. in custom code.
  6089. * Threads system was reworked to avoid global lua interpreter lock.
  6090. * DKIM module now converts all line endings to CRLF how opendkim does.
  6091. * URL detector is now more accurate for text parts.
  6092. * Several critical bugs and memory leaks were fixed.
  6093. 0.5.1:
  6094. * Added lua worker type to handle network connections in lua.
  6095. * Added lua bindings for async_session, IO dispatcher, memory_pool, and
  6096. worker.
  6097. * Composites can now uses other composites in expressions.
  6098. * Fixes for debian package and for FreeBSD >= 9.1
  6099. * Add support of gmime-2.6 if gmime-2.4 is not found.
  6100. * Improve url detection and phishing detection.
  6101. * Add lua mime_part library to get an access to all message part
  6102. attributes (like filename, length, type).
  6103. 0.5.0:
  6104. * Added SMTP lightweight balancing proxy with XCLIENT support.
  6105. * Added lua bindings for upstreams objects and API.
  6106. * New pre-filters are implemented to support initial checking for messages.
  6107. * Added ratelimit plugin that uses redis protocol to store data.
  6108. * Added ipv6 support to spf and some other modules.
  6109. * Unbreak spf plugin.
  6110. * Allow options with the same name be threated as list.
  6111. * DKIM plugin an parsing code was added.
  6112. * Separate build system to put logic in several shared libraries.
  6113. * Many bugfixes.
  6114. 0.3.2:
  6115. * Add error handling for regexps
  6116. * Fix quit command in controller interface
  6117. * Write symbols weights to rspamc output
  6118. * Improve logic of selecting rspamc version
  6119. * Do not try to parse broken DNS replies
  6120. * Add 'raw' flag to FROM_EXCESS_BASE64 rule (requested by citrin)
  6121. * Output message id in rspamc reply
  6122. * Fix inserting composite symbol
  6123. * Fix output of log line
  6124. * Document composites
  6125. * Add logging for fuzzy checks
  6126. * Add logging for learning
  6127. * Improve logic of learning messages - do not learn more than specific threshold
  6128. * Fix inserting results for symbols that were incorrectly (for example more than 1 time) defined in config file
  6129. * Do not output control characters if output is not terminal
  6130. * Fix some logic errors in learning
  6131. * Consider lua plugins errors as fatal configuration errors
  6132. * Fix writing message id during fuzzy_add command
  6133. * Display weight of symbols correctly
  6134. * Fixes to winnow learning
  6135. * One more try to improve accuracy of winnow algorithm
  6136. * Add bayesian classifier (initial version)
  6137. * Remove normalizer as it is winnow specific thing, so all statistic algorithms now returns value from 0 to 1
  6138. * Some fixes to fuzzy hashes expiration:
  6139. * Fix assertion while look up value in NULL hash (found by cirtin)
  6140. * Fix normalization for systems that have not tanhl function
  6141. * Ignore rfc822 group addresses
  6142. * Move images library to core rspamd
  6143. * Add lua api to access images properties
  6144. * Add post filters to lua API - filters that would be called after all message's processing
  6145. * Add ability to check for specified symbol in task results from lua
  6146. * Add ability to check for metric's results from lua
  6147. * Add ability to learn specified statfile form lua
  6148. * Add ability to extract filename and size of images from lua
  6149. * Fix assertion while extracting internet address
  6150. * Fix races in fuzzy storage
  6151. * Make spf parser case insensitive
  6152. * Add ability to check hashes of selected mime types
  6153. * Add ability to set minimum size in bytes for mime types
  6154. * Add ability to set minimum dimensions for images
  6155. * Assume all text/* content types as text/plain
  6156. * Fix getting data wrapper for gmime24
  6157. * Many fixes to statfile synchronization system
  6158. * Fixed statfile pool initialization and synchronization with disk
  6159. * Prepare 0.3.2
  6160. * Fix Mail::Rspamd::Config for new rspamd features
  6161. * Use Mail::Rspamd::Config in rspamc client
  6162. * Write user's name to rspamd log
  6163. * Prepare rspamd build infrastructure for creating rpm and deb packages
  6164. * Fix depends
  6165. * Add start script for linux systems
  6166. * Fix shared usage of statfiles
  6167. * Add invalidation of statfiles in case of learning, so now statfiles
  6168. * This should fix shared usage of statfile pool by several processes
  6169. * Fix misprint (reported by az)
  6170. * Fix stupid error when all checks can be done in a single pass
  6171. * New trie based url scanner (based on libcamel)
  6172. * Small fixes to rspamd perl client
  6173. * Write fuzzy hashes info to log
  6174. * Add trie interface to lua api
  6175. * Explain sample config and cleanup it
  6176. 0.3.1:
  6177. * Add modules documentation
  6178. * Continue implementing smtp proxy
  6179. * Implement new learning system, now rspamd should be much more intelligent while learning messages
  6180. * Convert statistic sums to use long double for counters
  6181. * Use hyperbolic tangent for internal normalizer
  6182. * In classify normalize result after comparing, not before
  6183. * New symbols sorter
  6184. * Fix strict aliasing while compiling with optimization
  6185. * Fix tanhl detection for platforms that have not implementation of it
  6186. * Remove several compile warnings
  6187. * Add experimental support of dynamic rules to regexp module
  6188. * Document views configuration
  6189. * Several fixes to documentation
  6190. * Add more logic for dynamic rules
  6191. * Add documentation for dynamic rules
  6192. * Add ability to make negations in networks in dynamic rules
  6193. * Clean up cache items correctly
  6194. * Implement basic SMTP dialog:
  6195. * Implement interaction with smtp upstream (with support of XCLIENT)
  6196. * Check messages received via smtp proxy
  6197. * Add support for sendfile in io dispatcher
  6198. * Fix issues with compatibility of worker_task and smtp proxy
  6199. * Proxy DATA command
  6200. * Fix SMTP
  6201. * Change metric logic
  6202. * Completely remove lex/yacc readers for config
  6203. * Make common sense of metric/action and symbols
  6204. * Sync changes with all plugins
  6205. * Incorrectly removed in previous commit
  6206. * Fix misprint (by Andrej Zverev)
  6207. * announce the "password" keyword in usage list
  6208. * Implement initial version of greylisting triplets storage
  6209. * Fix issues with smtp worker
  6210. * Fix QUIT command in SMTP worker
  6211. * Some fixes about new metrics system (may be incomplete)
  6212. * Get weights of symbol from default metric for symbols cache
  6213. * Fix setting task->from/task->rctp in smtp client
  6214. * Copy from and rcpt correctly
  6215. * Some performance improvements to IO dispatcher (do not drain the whole buffer after a single line readed)
  6216. * Fix smtp data input
  6217. * Fix misprint
  6218. * Add limit of maximum allowed smtp session errors
  6219. * New logic of SURBL module:
  6220. * Use system mkstemp(3) on systems where it is available as glib implementation
  6221. * Try to fix memmove issues in io dispatcher
  6222. * Remove debug from SURBL module
  6223. * Rewrite buffered input for line policy (again)
  6224. * Fix issue with links that are ip addresses in numeric form in surbl
  6225. * On Darwin use BSD style sendfile definition
  6226. * Reorganize platform specific knobs in CMakeLists
  6227. * Use gettimeofday on systems that have not clock_getres
  6228. * Use ftime for dns trans id generation on systems without clock_getres
  6229. * Darwin sendfile(2) support
  6230. * TIMEDB->TIMEB
  6231. * More to previous commit
  6232. * Pass env from main() arguments instead of platform specific global environ
  6233. * Fix compatibility issues
  6234. * Fix -lintl detection
  6235. * Init some variables to avoid problems
  6236. * Remove garbadge (gnome terminal sucks)
  6237. * Add more information about why we drop smtp connection
  6238. * Fix mkstemp call
  6239. * Send to upstream QUIT command at the end of session
  6240. * Check return value of each rspamd_dispatcher_write as in case of write errors sessions can be destroyed early
  6241. * Fix states in smtp dialog
  6242. * Use rspamd_snprintf instead of libc one
  6243. * Fix URLS command
  6244. * Fix reconfigure process of surbl module
  6245. * Fix destroying smtp session (unmap memory and do not delete pool early)
  6246. * Delete pool after using its variables
  6247. * Delay timer must be registered in async session to correctly handle connection termination
  6248. * Register dns requests in session too
  6249. * Make session before registering events
  6250. * Remove events in handlers
  6251. * Add ability to set filters for smtp worker for each smtp stage
  6252. * Add very initial version of DNS resolver (many things to be done)
  6253. * Announce weights and sync
  6254. * Fix few typo
  6255. * Understand short names of facility in logging config
  6256. * Add ability to make whitelist for spf checks
  6257. * Misprint != -> ==
  6258. * Handle lua tag in way that it is not required to write additional text:
  6259. * Strip all starting whitespace symbols from xml texts
  6260. * Fix stupid bug in calculating buffer length while reading file maps
  6261. * Add resolv.conf parsing into dns.c
  6262. * Fix microseconds<->milliseconds conversions
  6263. * Take callback argument in Mail::Rspamd::Client for processing files and directories
  6264. * Print results if rspamc is called for a directory
  6265. * Fix stupid error with surbl module reconfig (another one, blame me)
  6266. * Do not show duplicate urls in url header
  6267. * Fix detection of numeric urls (reported by citrin)
  6268. * Write real time of message's scan to log (not only virtual)
  6269. * Fix chartable module in utf mode
  6270. * Fix parsing of some broken urls
  6271. * Add ability to test regexp with 'T' flag
  6272. * Write more code for DNS resolver:
  6273. * Make DNS resolver working
  6274. * Many improvements to rspamd test suite: now it CAN be used for testing rspamd functionality
  6275. * Write DNS resolver tests
  6276. * Fix issues with memory_pool mutexes and with creating of statfiles
  6277. * Forgotten in previous commit
  6278. * Add support for parsing SPF and SRV records
  6279. * Fix PTR parsing
  6280. * Add tests
  6281. * Make SURBL module to use rspamd dns module
  6282. * Several fixes to DNS logic
  6283. * Remove evdns and use only rspamd resolver
  6284. * Very hard to detect problem with race among error in socket and destroying task while we are writing to socket and go through a hash table
  6285. * Fix resolving in smtp module
  6286. * Init events before configuring resolver in smtp worker
  6287. * Set resolver inside task
  6288. * Fix reload signal (reported by citrin)
  6289. * Some improvements to redirector
  6290. * Call has_forked method to inform POE about fork
  6291. * Fix lua DNS code
  6292. * Decompress labels in DNS packets more strictly
  6293. * Fix some problems with TXT records
  6294. * Try to fix removing of DNS events
  6295. * Do not insert unparsed RR's into reply
  6296. * Calling callbacks may cause destroying session from which we are calling callback so we MUST call callback as the latest action
  6297. * Fix check_smtp_data function
  6298. * Add ability to make views by recipient
  6299. * Add ability to set metric's action from config file
  6300. * Fix bug with writing garbadge if message has no urls or no messages
  6301. * Fix bug with incorrect behaviour of compare_parts_distance function
  6302. * Add ability to assign several actions to one metric
  6303. * Report action in rspamc protocol
  6304. * Mail::Rspamd::Client and rspamc can now understand Action header too
  6305. * Write action to log as well
  6306. * Make valgrind happy about comparing symbols
  6307. * Add more debug to comparing parts distance function
  6308. * Write action even if message has no symbols
  6309. * Make improvements to HTML entities decoder: now it replaces entities with common characters and
  6310. * Add -d option to force debug output
  6311. * Assume 7bit as default transfer encoding
  6312. * Do not overwrite lua plugins and configs if they already exists in target directory
  6313. * Improve logging
  6314. * Write queue id to log
  6315. * Remove test messages from cmake
  6316. * Reopen log file by USR1 signal
  6317. * Add reopenlog method to FreeBSD rc script
  6318. * Adopt foreach for cmake 2.6
  6319. * Fix to rc script
  6320. * Do not try to resolve names with several dots in a row
  6321. * Fix surbl request formatting for ip addresses
  6322. * Handle cases of broken requests
  6323. * Fix problems with parsing compressed names
  6324. * Fix TXT records parsing
  6325. * Fix expanding spf macros that may fail in rare cases
  6326. * Fix another error with early task destroying
  6327. * Handle empty from header
  6328. * Improve reopenlog command in rc script
  6329. * Strip trailing whitespace characters in Mail::Rspamd::Client
  6330. * Use ungreedy match to strip trailing whitespaces
  6331. * Stupid error in calculation compressed label length
  6332. * Some optimizations to client library
  6333. * Do not compare empty parts
  6334. * Empty and non-empty parts are always different
  6335. * Save in regexp cache the whole regexp with header name (if exists) and with flags
  6336. * Add rspamd_log variable to lua plugins to access logging functions
  6337. * Each part in rspamd task now can have parent part
  6338. * Check for parts distance only for multipart/alternative subparts
  6339. * Do not check attachements even if they are text (but attached as file)
  6340. * Do not die if write (2) returned ENOSPACE while doing logging, turn on throttling mode instead (1 write try in a second)
  6341. * Add ability to turn on debug for specific symbols
  6342. * Add ability to configure dns timeouts and dns retransmits in config file
  6343. * More debug
  6344. * Fix extracting arguments in lua logger interface
  6345. * Turn off debug_ip during reload if it was disabled by new config
  6346. * Improve lua logging
  6347. * Pre-init symbols cache when rereading config
  6348. * Fix lua representing of invalid ip (nil, not 255.255.255.255)
  6349. * Fix R_TO_SEEMS_AUTO rule (by citrin)
  6350. * Add multimap lua plugin
  6351. * Fix some multimap issues
  6352. * Try to save images hashes to fuzzy storage to stop some annoying spammers
  6353. * Allocate some more bytes for read buffer to avoid incorrect behavoiur
  6354. * Add ability to check dns black lists by multimap module
  6355. * Add multimap documentation
  6356. * Fix labels parsing
  6357. * Another try to save regexps in cache correctly
  6358. * Improve test logs for regexps
  6359. * Fix parsing txt records to avoid reading of uninitialized data
  6360. * Fix error with writing symbols cache file
  6361. * Fix error while working in utf mode when raw regexps was not created properly
  6362. * Do not add extra byte while converting text to utf
  6363. * Add error handling for regexps