* Fix error in expression parser that causes bad errors with expressions that have regexp at the end

* Improve test for fuzzy hashes
* Add new object - TextPart to perl XS library that allows access to stripped parts and fuzzy hashes
* Add documentation for expressions parser and fot Mail::Rspamd::TextPart
* Allways calculate fuzzy hash for text parts
* Store text parts separately from other parts
* Add compare_parts_distance for expressions that calculates difference in 2 parts messages
* Do not try to substitute variables in empty strings

1 files changed, 39 insertions, 1 deletions

diff --git a/perl/Rspamd.pod b/perl/Rspamd.pod
index d40574820..2af9b4965 100644
--- a/perl/Rspamd.pod
+++ b/perl/Rspamd.pod
@@ -197,7 +197,7 @@ E.g.:
 
 Mail::Rspamd::Header is a private structure. This structure contains
 all the headers except special ones (Content-* MIME-Version).
-Look for L<Header tied hash> for easy maintaining for header.
+Look for Header tied hash for easy maintaining for header.
 Use also the Mail::Rspamd::Message::get_header() and set_header() methods.
 
 =back
@@ -476,6 +476,10 @@ Return Mail::Rspamd::Config object.
 
 Return message's urls as array of strings.
 
+=item I<get_text_parts> ()
+
+Return message's text parts as array of Mail::Rspamd::TextPart objects.
+
 =back
 
 =head2 Mail::Rspamd::Config
@@ -493,15 +497,23 @@ Gets and sets specified parameter in config.
 =item I<get_metric> (metric)
 
 Returns hash of parameters of specified metric:
+
+=begin text
+
 {
 'name'             => name of metric
 'func_name'        => consolidation function
 'required_score'   => score for metric
 }
 
+=end text
+
 =item I<get_statfile> (statfile)
 
 Returns parameters of specified statfile:
+
+=begin text
+
 {
 'alias'         => alias of statfile
 'pattern'       => fs pattern
@@ -510,12 +522,38 @@ Returns parameters of specified statfile:
 'size'          => size of statfile
 }
 
+=end text
+
 =item I<get_module_param> (modulename, paramname)
 
 Return parameter's value for specified module.
 
 =back
 
+=head2 Mail::Rspamd::TextPart
+
+Object that represent a single text part of message.
+
+=over 4
+
+=item I<get_content> ()
+
+Returns content of part.
+
+=item I<get_fuzzy> ()
+
+Returns fuzzy hash of part as string.
+
+=item I<compare_distance> (other)
+
+Calculate distance between two parts using their fuzzy hashes. Return value from 0 (identical) to 100 (totally different).
+
+=item I<is_html> ()
+
+Return 0 if part is plain text and not 0 otherwise.
+
+=back
+
 =head1 CONSTANT VARIABLES
 
     GMIME_LENGTH_ENCODED