Anton Yuzhaninov [Tue, 23 Oct 2018 02:19:29 +0000 (22:19 -0400)]
Share robot log and report on https://ci.rspamd.com/testlogs/
Sending functional test log (about 5Mb) in a email attachmend was temporary
solution.
Upload them to https://ci.rspamd.com/testlogs/ instead.
Link to log/report will be printed in build log.
Anton Yuzhaninov [Sat, 20 Oct 2018 08:15:40 +0000 (04:15 -0400)]
Fix coverage (#2603)
* Add utility to prepare C coverage for upload to coveralls.io
It turned out that it is more easy to write own script then debug
and fix coveralls utility (https://github.com/eddyxu/cpp-coveralls).
gcov-coveralls.py can be used as a replacement for coveralls.
* Save coverage data from .gcda files only once
Coverage data in .gcda files is merged after each binary invocation,
so we can run all test and then gather coverage data.
If we dump them two times execution counts will be more then they be.
* Switch from coveralls (cpp-coveralls) to own script
Problem with coveralls was, that coverage for source files outside build
directory was not added to the report.
* Add tool to dump info from json for coveralls.io
* Add debug
* Fix: don't die if there is no service_job_id in json
* Debug
* Fix dump_coveralls.py
* Rename to gcov_coveralls.py (s/-/_/)
For most files in this repo '_' is used as separator.
* Don't add source code to coveralls JSON
According to https://docs.coveralls.io/api-introduction
Coverals don't need source code, only MD5 digest to tracks changes.
Anyway source code is already added by luacov-coveralls and source_digest
is added by cpp-coveralls and gcov_coveralls.py
Both options seems to work for now.
* Provide path to source directory to merge_coveralls.py
merge_coveralls.py has code to filter files and remove prefixes.
When --root points to source directory merge_coveralls.py can strip
prefix from absolute path in JSONs generated by luacov-coveralls.
* Style
Don't add parameters with default values.
* Make --output optional
It useful mainly for debugging. We can send report without saving it.
* Log CI_COMMIT_AUTHOR env var
It is not clear from drone.io source how CI_COMMIT_AUTHOR variable is
set. Log it to see what it means.
* Move merge_coveralls.py to test/tools
This script is used not only for funcional test coverage, but for
rspamd-test coverage too.
* Remove debug
* Style
Use more compact formatting.
* Write comment about parallel tests running [SKIP CI]
Document why running tests in parallel may be bad idea (but still do so).
Vsevolod Stakhov [Tue, 16 Oct 2018 17:30:16 +0000 (18:30 +0100)]
Release 1.8.1
* [CritFix] Fix options insertion
* [CritFix] Fix words decay one more time (affects long messages)
* [CritFix] Increase default words_decay
* [CritFix] Plug memory leak in redis pool
* [Feature] Add `check_violation` feature to DKIM/ARC signing
* [Feature] Add only unique elements to Clickhouse url arrays
* [Feature] Allow `g+:` and `g-:` composite atoms
* [Feature] Allow dkim domains check in surbl
* [Feature] Allow maps with HTTP auth
* [Feature] Allow to disable actions by users settings
* [Feature] Extend whitelisting options
* [Feature] Store url object in images
* [Feature] Use verdict instead of the plain action in plugins
* [Fix] Allow to call fstring append with NULL string
* [Fix] DCC - luacheck
* [Fix] Do not load torch on each rspamadm invocation
* [Fix] Fix boundaries detection and rework stop words algorithm
* [Fix] Fix dependencies for DNS_SIGNED symbol
* [Fix] Fix errors when dealing with dynamic rates/bursts in Ratelimit
* [Fix] Fix groups mess
* [Fix] Fix groups mess
* [Fix] Fix parsing address with comments
* [Fix] Fix resolving in DMARC reports
* [Fix] Fix various issues with parsing of the received headers
* [Fix] Fix watchers issue in lua_tcp when doing no resolving
* [Fix] Plug memory leak in language detector (affects reloads)
* [Fix] Remove one letter stop words
* [Fix] Slashing: backport chunk logic from libucl
* [Fix] Stop libevent from using cached time in rspamadm
* [Fix] Try to fix watchers chaining
* [Fix] Various fixes in redis sync interface
* [Fix] ip_score - respect check_authed and check_local settings from config
* [Project] Rework passthrough actions
* [Project] Clustering module
* [Rework] Always create result for a task
* [Rework] Completely rewrite DMARC checks logic
* [Rework] Rework and fix whitelist plugin
* [WebUI] Add symbols sorting buttons
* [WebUI] Change symbols order without updating history
* [WebUI] Colorize symbols
* [WebUI] Do not display password form when secure_ip is set
* [WebUI] Fix symbol description tooltips display
* [WebUI] History: add sorting by symbol score value
Anton Yuzhaninov [Sun, 14 Oct 2018 08:38:57 +0000 (04:38 -0400)]
Drone coverage fix (#2592)
* Build as nobody
This showld fix coverage files writing. At least some coverage files
(*.gcda, *.gcno) are written by rspamd running as nobody.
* Comment luacov-coveralls
* Run rspamd-test as nobody too
* Run coveralls from build root (/rspamd/build)
It need to find all `.gcda` files.
* Hack for coveralls was moved to Docker image
* Add test debug
* More debug
* Workaround to writing *.gcda from different users
*.gcda files written in parallel by root and by nobody
* Remove debug
* Use current build number instead of previous
* Remove unused variable
* Add more parameters
- If CI detected we don't need to check if service_job_id and other
fields in json already filled - this info anyway comes from environment
- coveralls.io API is not documented. Code from:
https://github.com/lemurheavy/coveralls-ruby/
and
https://github.com/coveralls-clients/coveralls-python/
was used as an example.
* In case of error there is no url in json respnse
Print message field anyway - it can contain useful error message.
* Add debug
* Simplify
* Fix: for pull request CI_BUILD_EVENT set to 'pull_request'