revamping

This commit is contained in:
Christoph Haas 2025-10-11 22:09:53 +02:00
parent f2fd6f7043
commit e6abc3453d

View file

@ -20,73 +20,37 @@ the [milter](http://www.postfix.org/MILTER_README.html) (=**m**ail f**ilter**)
system, Postfix will send it to rspamd to have its content checked. rspamd runs a lot of checks on the email and system, Postfix will send it to rspamd to have its content checked. rspamd runs a lot of checks on the email and
computes a total score. The higher the score the more likely it it spam. computes a total score. The higher the score the more likely it it spam.
## How free is rspamd?
rspamd is quite effective against spam thanks to its many kinds of checks that incoming emails run through. Several of
these checks rely on services provided by the rspamd project. There are hosts like email.rspamd.com, uribl.rspamd.com
and maps.rspamd.com that provide information used by the default rspamd installation. That data is not freely available
and there are no public mirrors.
You are only allowed to use their data under certain conditions described in rspamds
[usage policy](https://docs.rspamd.com/other/usage_policy/). Spoiler: heavy or commercial usage will get you blocked.
Your friends-and-family mail server with low volumes of email will probably be fine.
rspamd is being packaged and shipped by the Debian project. However the rspamd principal developer builds and endorses
his own packages and has a history of getting angry and offensive against anyone using the packages shipped by Debian. I
find that attitude a bit suspicious and rather stay with the Debian packages that have been built by a process that I
trust.
## Make Postfix use rspamd ## Make Postfix use rspamd
Lets tell Postfix to send all incoming email through rspamd. Run these commands on the shell: Lets tell Postfix to send all incoming email through rspamd:
``` ```
postconf smtpd_milters=inet:127.0.0.1:11332 postconf smtpd_milters=inet:127.0.0.1:11332
postconf non_smtpd_milters=inet:127.0.0.1:11332
TODO: no… postconf milter_mail_macros="i {mail_addr} {client_addr} {client_name} {auth_authen}"
``` ```
Postfix will connect to rspamd that is listening to TCP port 11332 on localhost (127.0.0.1) and send the email through
that connection. The _smtpd_milters_ setting defines that connection for emails that came into the system from the
internet via the SMTP protocol. The *non_smtpd_milters* setting is optional it makes Postfix have all emails checked
that originate from the system itself. Finally the *milter_mail_macros* setting defines several variables that rspamd
expects for better spam detection. rspamd then runs its checks and tells Postfix whether the email should pass or get
rejected.
## Testing spam detection
For testing we can use a sample spam email that comes with SpamAssassin. It is called GTUBE (Generic Test For testing we can use a sample spam email that comes with SpamAssassin. It is called GTUBE (Generic Test
for Unsolicited Bulk Email). It contains a certain artificial pattern that is recognized as spam by SpamAssassin. Do you for Unsolicited Bulk Email). It contains a certain artificial pattern that is recognized as spam by SpamAssassin. Do you
know EICAR.COM to test virus scanners? This is the same thing for spam. know EICAR.COM to test virus scanners? This is the same thing for spam.
I suggest that you download the file on the server and send it to John: Get the GTUBE file, send it to John and let's see what happens:
```sh ```sh
wget http://spamassassin.apache.org/gtube/gtube.txt wget http://spamassassin.apache.org/gtube/gtube.txt
sendmail john@example.org < gtube.txt sendmail john@example.org < gtube.txt
journalctl -fu postfix
``` ```
Let's see what happened. Check your mail log: You will find something like this at the end of the log file:
```sh
journalctl -eu postfix
```
You will find something like this at the end:
``` ```
postfix/pickup[1384784]: 2B72320126: uid=0 from=<root>
postfix/cleanup[1386519]: 2B72320126: message-id=<GTUBE1.1010101@example.net> postfix/cleanup[1386519]: 2B72320126: message-id=<GTUBE1.1010101@example.net>
postfix/cleanup[1386519]: 2B72320126: milter-reject: END-OF-MESSAGE from localhost[127.0.0.1]: 5.7.1 Gtube pattern; from=<root@mailserver> to=<john@example.org> postfix/cleanup[1386519]: 2B72320126: milter-reject: END-OF-MESSAGE from localhost[127.0.0.1]: 5.7.1 Gtube pattern; from=<root@mailserver> to=<john@example.org>
postfix/cleanup[1386519]: 2B72320126: to=<john@example.org>, relay=none, delay=0.16, delays=0.16/0/0/0, dsn=5.7.1, status=bounced (Gtube pattern) postfix/cleanup[1386519]: 2B72320126: to=<john@example.org>, relay=none, delay=0.16, delays=0.16/0/0/0, dsn=5.7.1, status=bounced (Gtube pattern)
postfix/cleanup[1386525]: 4F0B720128: message-id=<20251009174553.4F0B720128@mailserver>
postfix/bounce[1386524]: 2B72320126: sender non-delivery notification: 4F0B720128
postfix/cleanup[1386519]: 2B72320126: removed (discarded)
``` ```
**milter-reject** tells that the milter (rspamd) recommended to Postfix to reject the email. It gave the reason **milter-reject** tells that the rspamd recommended to reject the email. It gave the reason `5.7.1 Gtube pattern`.
`5.7.1 Gtube pattern`. In mail communication you often find these three digit codes. They are defined in SMTP/LMTP always uses such three digit codes. They are defined in [RFC 3463](https://tools.ietf.org/html/rfc3463). The
[RFC 3463](https://tools.ietf.org/html/rfc3463). The first digit is most important: first digit is most important:
- 2 = Success - 2 = Success
- 4 = Temporary failure (come back later) - 4 = Temporary failure (come back later)
@ -97,20 +61,19 @@ that the 7 stands for an issue regarding the security policy. So its not a te
security-relevant component on the system has rejected the email. Thats what rspamd did. It even told us the reason: security-relevant component on the system has rejected the email. Thats what rspamd did. It even told us the reason:
`Gtube pattern`. So you see that rspamd knows about the Gtube spam test pattern and reacts as expected. `Gtube pattern`. So you see that rspamd knows about the Gtube spam test pattern and reacts as expected.
Accordingly you wont see this email in Johns inbox. This is a great advantage of using milters. Imagine Postfix That email will not be delivered to John. Instead Postfix will reject it at the doorstep while it is still in the SMTP
receiving a spam email and confirm its reception. What should it do when it finds out that its unwanted email? communication with the sending server. It just hung up the phone and now it's the sending server's problem to deal with.
According to the SMTP protocol it must not throw away any emails. Would you create a bounce message telling the sender
that you did not accept the email? That would be a bad idea because the sender address in spam emails is very likely It is important to either reject an email right away or to deliver it to a user. Never accept an email and later decide
forged. You would send the bounce to an innocent person thus creating so called *backscatter* and make it even worse. So to complain about it to the alleged sender. Trust me: the sender you see on a spam email is never the bad guy who sent
the better approach is to check the email while the sending server is still connected to your Postfix. This allows it. You would complain to the wrong person which is called
Postfix to reject the email with a 5.x.x error code and let the other side figure out what to do. [backscatter](<https://en.wikipedia.org/wiki/Backscatter_(email)>).
## Score metrics ## Score metrics
rspamd will however not reject all spam email. It computes a score that tells how likely a certain email is spam. You rspamd will however not reject all spam email. It computes a score that tells how likely a certain email is spam. You
can tell it which scores you would accept, flag as spam or make the incoming email get rejected. Rspamd has a large can tell it which scores you would accept, flag as spam or make the incoming email get rejected. Rspamd checks incoming
ruleset that checks incoming emails in various ways and adds to the score. Take a look at the `/etc/rspamd/actions.conf` emails in various ways. Take a look at the `/etc/rspamd/actions.conf` file:
file:
``` ```
actions { actions {
@ -123,13 +86,13 @@ actions {
These are the default actions. If rspamd computes a score of at least 15 then the email will get rejected at the These are the default actions. If rspamd computes a score of at least 15 then the email will get rejected at the
doorstep just like the _Gtube pattern_ in the previous test. Any other score above 6 will add a line "X-Spam: Yes" so doorstep just like the _Gtube pattern_ in the previous test. Any other score above 6 will add a line "X-Spam: Yes" so
that your mail software can detect them and maybe file the email to a different folder. And any other score above 4 will that your mail software can detect them and maybe file the email to a different folder. And any other score above 4 will
trigger _greylisting_ which is a mechanism that temporarily rejects the email with a 4.x.x code and waits if the sending trigger [greylisting](<https://en.wikipedia.org/wiki/Greylisting_(email)>) which is a mechanism that temporarily rejects
server will try again. After a waiting time of a few minutes greylisting will accept the email. The idea is to reject the email with a 4.x.x code and waits if the sending server will try again. After a waiting time of a few minutes
email from systems that do not have a sending queue. Malware like on infected Wind\*ws computers used to try sending an greylisting will accept the email. The idea is to reject email from systems that do not have a sending queue. Malware
email just once which triggered greylisting and successfully rejected the spammer. But even malware programmers have like on infected Wind\*ws computers used to try sending an email just once which triggered greylisting and successfully
learned and may try again after a few minutes thus circumventing greylisting. Your mileage may vary. The problem with rejected the spammer. But even malware programmers have learned and may try again after a few minutes thus circumventing
greylisting is that the recipient has to wait a couple of minutes for the email to be delivered which is often bothering greylisting. Your mileage may vary. The problem with greylisting is that the recipient has to wait a couple of minutes
the users. for the email to be delivered which may be bothering the users.
## X-Spam header ## X-Spam header
@ -203,17 +166,16 @@ the score but show you what rspamd has found. But if you consider certain criter
## Sending spam to the Junk folder ## Sending spam to the Junk folder
Your users will not realize that their spam emails have an added "X-Spam: Yes" header. It is not actively shown in their Your users will not realize that their spam emails have an added `X-Spam: Yes` header. It is not shown in their mail
mail client. Nor does it move the email out of the inbox into their spam folder. Such emails just appear like normal in client. Nor does it move the email out of the inbox into their spam folder. Such emails just appear in their inbox. So
their inboxes. So lets aid them by automatically moving spam to a separate _Junk_ folder beneath their inbox. Dovecot lets be nice and move spam emails to the user's _Junk_ folder automatically. Dovecot has support
has support for [Sieve](<https://en.wikipedia.org/wiki/Sieve_(mail_filtering_language)>) filtering rules that are simple for [Sieve](<https://en.wikipedia.org/wiki/Sieve_(mail_filtering_language)>) filtering rules that are if-then scripts
if-then scripts that run on the server automatically whenever an email comes in. that get run whenever an email arrives.
John could create a new Sieve filter (e.g. using the Roundcube webmail interface) for himself that would save any emails John could create such a Sieve script for himself (using the Roundcube webmail interface). But let's find a solution
to his "Junk" folder if the header line "X-Spam: Yes" was found. This rule would be useful for all your users though so that applies to all your users.
lets find a more general solution.
Dovecot supports _global_ Sieve filters that apply to all users. Let's do some preparation: Let's do some magic:
```sh ```sh
# Create a new config file for spam handling # Create a new config file for spam handling
@ -281,20 +243,13 @@ directly into Dovecot using the LMTP socket:
```sh ```sh
swaks --to john@example.org --header-X-Spam "yes" --socket /var/spool/postfix/private/dovecot-lmtp --protocol LMTP swaks --to john@example.org --header-X-Spam "yes" --socket /var/spool/postfix/private/dovecot-lmtp --protocol LMTP
``` journalctl -fu dovecot
Take a look at Dovecot's log:
```sh
journalctl -eu dovecot
``` ```
It should read: It should read:
``` ```
mailserver dovecot[1434406]: lmtp(1436598): Connect from local dovecot[1434406]: lmtp(john@example.org)<1436598><kh4lDNl26Wi26xUA5ANL0g>: sieve: msgid=<20251010211257.1436597@mailserver>: fileinto action: stored mail into mailbox 'Junk'
mailserver dovecot[1434406]: lmtp(john@example.org)<1436598><kh4lDNl26Wi26xUA5ANL0g>: sieve: msgid=<20251010211257.1436597@mailserver>: fileinto action: stored mail into mailbox 'Junk'
mailserver dovecot[1434406]: lmtp(1436598): Disconnect from local: Logged out (state=READY)
``` ```
<details class="collapsible"> <details class="collapsible">
@ -319,7 +274,7 @@ journalctl -eu dovecot
</details> </details>
The alleged spam email has been moved to the _Junk_ folder. Just like we wanted. The alleged spam email has been moved to the _Junk_ folder just like we wanted.
## About Redis ## About Redis