Migrated further pages from Wordpress

This commit is contained in:
Christoph Haas 2024-12-19 00:20:37 +01:00
parent 30341a60a4
commit 574bc01ea0
11 changed files with 459 additions and 17 deletions

View file

@ -156,22 +156,47 @@ password_query = SELECT password FROM virtual_users WHERE email='%u'
iterate_query = SELECT email AS user FROM virtual_users iterate_query = SELECT email AS user FROM virtual_users
</pre> </pre>
<Aside type="tip" title="Backslashes">
Ending a line with a backslash (\) means that it is continued on the next line. It keeps the configuration more readable when it is split over multiple lines.
</Aside>
What these lines mean:
- driver: the kind of database. MariaDB is the same kind as MySQL.
- connect: where to find the MySQL database and how to access it (username, password)
- user\_query: an SQL query that returns the user name (=the email address), the quota, the home directory, user ID and group ID.
- password\_query: this SQL query just gets the password hash from the database
- iterate\_query: doveadm uses this query to get a list of all users. That allows you to use the “doveadm user \*'” command later.
The _user\_query_ gets several pieces of information from the database. Lets look at it one by one:
- email AS user
It gets the the _email_ field from the database which corresponds to the user name. Dovecot expects it in the _user_ field so we set an alias to _“user”._
- userdb\_quota\_rule
This is the users quota in bytes. Think of it as the maximum possible space on disk that the user can occupy. As [documented](https://doc.dovecot.org/configuration_manual/quota/#per-user-quota) Dovecot expects the quota in a special format like “\*:bytes=10000” if the user should not be able to store more than 10,000 bytes. Thats why we begin the string with \*:bytes=.
- userdb\_home
This leads to the directory where all emails and various control files for this user are located. The placeholder %d replaces the domain and %n the user part. So for John that makes it “/var/vmail/example.org/john”.
- userdb\_uid and userdb\_gid
Those are the user ID and group ID of _vmail_ user 5000 for both. Dovecot uses it to set the permissions of files it creates. As all users share the same system user “vmail” this is just a static number.s
## Fix permissions
Make sure that only root can access the SQL configuration file so nobody else is reading your database access passwords:
``` ```
driver = mysql chown root:root /etc/dovecot/dovecot-sql.conf.ext
chmod go= /etc/dovecot/dovecot-sql.conf.ext
connect = \\
host=127.0.0.1 \\
dbname=mailserver \\
user=mailserver \\
password=**x893dNj4stkHy1MKQq0USWBaX4ZZdq**
user_query = SELECT email as user, \\
concat('*:bytes=', quota) AS quota_rule, \\
'/var/vmail/%d/%n' AS home, \\
5000 AS uid, 5000 AS gid \\
FROM virtual_users WHERE email='%u'
password_query = SELECT password FROM virtual_users WHERE email='%u'
iterate_query = SELECT email AS user FROM virtual_users
``` ```
Restart Dovecot from the shell:
```
systemctl restart dovecot
```
Look at your /var/log/mail.log logfile. You should see:
```
... Dovecot v2.3.13 (f79e8e7e4) starting up for imap, lmtp, sieve, pop3 (core dumps disabled)
```
If you get any error messages please double-check your configuration files.

View file

@ -0,0 +1,74 @@
---
title: Let Postfix send emails to Dovecot
lastUpdated: 2023-10-04
slug: ispmail-bookworm/let-postfix-send-emails-to-dovecot/
sidebar:
order: 130
---
import { Aside } from "@astrojs/starlight/components";
I hope you havent lost your mind yet. If you are unsure how Postfix and Dovecot work together take a moment and go back to the _big picture_ page.
In a previous chapter we made sure that Postfix knows which emails it is allowed to receive. Now what to do with the email? It has to be saved to disk into the mailbox of the mail user who is eagerly waiting for it. You could let Postfix handle that using its built-in mail delivery agent (MDA) called “_virtual_“. However compared to the capabilities that Dovecot provides like server-based sieve rules or quotas the Postfix delivery agent is pretty basic. We are using Dovecot anyway to provide the IMAP (and optionally POP3) service. So lets use its _delivery agent_.
How can we make Postfix hand over the email to Dovecot? There are generally two ways to establish that link.
1. Using the _dovecot-lda_ (local delivery agent) process. It can process one email at a time. And it starts up a new process for every email. This was for long the default way. But as you can imagine that it does not scale well.
2. The better option is to use [LMTP (local mail transport protocol)](https://en.wikipedia.org/wiki/Local_Mail_Transfer_Protocol) that was conceived for this purpose. It can handle multiple recipients at the same time and has a permanently running process which provides a better performance than using the LDA. In short, LMTP is a variant of SMTP with fewer features. It is meant for email communication between internal services that trust each other.
You guessed it already we will go for the second option. You installed the _dovecot-lmtpd_ package earlier. So lets configure it.
## Tell Dovecot where to listen for LMTP connections from Postfix
Edit Dovecots configuration file that deals with the LMTP daemon you can find it at `/etc/dovecot/conf.d/10-master.conf`. Look for the “service lmtp” section and edit it so that it looks like:
```
service lmtp {
unix\_listener /var/spool/postfix/private/dovecot-lmtp {
group = postfix
mode = 0600
user = postfix
}
}
```
This makes Dovecots _lmtp daemon_ create a UNIX socket at /var/spool/postfix/private/dovecot-lmtp. Just like in the section dealing with setting up Dovecot we make it put a socket into the /var/spool/postfix chroot directory because Postfix is restricted to that directory and cannot access anything outside of it. So from Postfixs point of view the socket is located at “/private/dovecot-lmtp”.
Restart Dovecot…
```
systemctl restart dovecot
```
Check if dovecot accepted that change:
```
systemctl status dovecot
```
The output should contain “Active: active (running)”.
## Tell Postfix to deliver emails to Dovecot using LMTP
This is even easier. The “_virtual\_transport_” in Postfix defines the service to use for delivering emails to the local system. Dovecot has created a socket file and is ready to listen to incoming LMTP connections. We just need to tell Postfix to send emails there:
```
postconf virtual_transport=lmtp:unix:private/dovecot-lmtp
```
The syntax looks crazy, but its actually simple. You just told Postfix to use the LMTP protocol. And that we want to use a UNIX socket on the same system (instead of a TCP connection). And the socket file is located at `/var/spool/postfix/private/dovecot-lmtp`.
(You will find further information on these steps in the [Dovecot configuration on Postfix integration](https://doc.dovecot.org/configuration_manual/howto/postfix_dovecot_lmtp/).)
## Enable server-side mail rules
One of my favorite features of Dovecot are automatic rules for incoming email that are processed on the server. You can sort away your mailing list emails into special folders. You can reject certain senders. Or you can set up vacation auto-responders. No need to have a mail client running it all happens automatically on the server even when your mail users are not connected.
The open standard (RFC 5228) for such rules is called Sieve. Basically, Sieve is a way to manage server-side email rules. A rule consists of conditions and actions. For example if the sender address matches `steve@example.org` you could tell Dovecot to move such emails to your “steve” folder automatically. These rules are stored on the Dovecot server and executed automatically. Whether you connect from your smartphone your laptop or use the webmail access the rules always work and require no configuration on the client side.
As we use LMTP thats where we need to tell the lmtp service that we want to use Dovecots “sieve” plugin. Edit the file `/etc/dovecot/conf.d/20-lmtp.conf` and within the “protocol lmtp” section change the “mail\_plugins” line to:
```
mail_plugins = $mail_plugins sieve
```
Restart Dovecot and you are done:
```
systemctl restart dovecot
```

View file

@ -0,0 +1,160 @@
---
title: Quotas
lastUpdated: 2023-10-04
slug: ispmail-bookworm/quotas/
sidebar:
order: 140
---
import { Aside } from "@astrojs/starlight/components";
<Aside type="tip" title="Optional feature">
This feature is completely optional. If you are eager to get finished then skip this page and maybe come back later.
</Aside>
Quotas are size limits for users. You can make sure that users do not waste arbitrary amounts of disk space but are forced to clean up old emails every now and then.
The magic happens in two places:
1. Postfix needs to reject new emails if the users mailbox is over quota.
2. Dovecot needs to keep track of the quota and how much the user has already used up of it.
### Dovecot quota policy service
Lets start with Dovecot. Find the file `/etc/dovecot/conf.d/90-quota.conf` and edit it. There are several `plugin {}` sections. Take one and make it look like:
```
plugin {
quota = count:User quota
quota_vsizes = yes
quota_status_success = DUNNO
quota_status_nouser = DUNNO
quota_status_overquota = "452 4.2.2 Mailbox is full and cannot receive any more emails"
}
```
The first line defines that you want to calculate the used space in a users _maildir_. There are several [backends](https://doc.dovecot.org/configuration_manual/quota_plugin/) like that but the _[count](https://doc.dovecot.org/configuration_manual/quota/quota_count/#quota-backend-count)_ is the best choice in this context. (Previous guides used _maildir_ here.) The string “User quota” is just an arbitrary string that may be queried from a mail user agent.
The lines starting with “`quota_status_…`” set return values for the service that you will set up in a minute. It will tell Postfix that it will not interfere (_DUNNO_ colloquial way to say “I dont know”). And it will return a string with a return code 452 if the user is over quota. Codes starting with “4” mean temporary errors. It will tell the sending party that it is worth retrying at a later time. However if the user does not resolve the issue it will lead to a _bounce_ error email after three days.
In the same file (_90-quota.conf_) add another section:
```
service quota-status {
executable = /usr/lib/dovecot/quota-status -p postfix
unix_listener /var/spool/postfix/private/quota-status {
user = postfix
}
}
```
This creates a new [Dovecot service](https://doc.dovecot.org/configuration_manual/service_configuration/) responding to requests from other processes. You surely recognize that we put it into the jail that Postfix runs in (_/var/spool/postfix_), so that Postfix can access it.
Time to restart Dovecot:
```
systemctl restart dovecot
```
Take a look at the /var/spool/postfix/private directory. If all went as intended you will find a socket file called `quota-status` there. Otherwise please check the `/var/log/mail.log` file for errors.
### Postfix recipient restrictions
If we stopped here, then Dovecot would reject emails for users who have no space left. However Postfix would still happily receive new emails and attempt to forward them to Dovecot via LMTP. Dovecot however will deny that. It will then keep the email in its queue and retry for a while. In the end it will send a _bounce_ back to the sender telling them about the problem. So why is this bad?
1. The sender will assume that the email was delivered while it is stuck in the queue for up to three days.
2. Spam emails use forged senders. So at the time that Postfix generates the _bounce email_ it will likely send it to an innocent person. This is called _backscatter_ and considered a mail server misconfiguration. Such a problem may get your mail server blacklisted. You dont want that.
So the next logical step is to make Postfix check whether a mailbox is over quota whenever a new email arrives. Lets hook up into the “RCPT TO” phase of the SMTP dialog when a new email comes in. Postfix checks its _smtpd\_recipient\_restrictions_ configuration at this stage. Run this command in the shell:
```
postconf smtpd_recipient_restrictions=reject_unauth_destination, \
"check_policy_service unix:private/quota-status"
```
This adds two checks:
1. `reject_unauth_destination` checks whether the mail server is the final destination for the recipients email address. This is pretty much the default behavior if you do not define any restrictions.
2. `check_policy_service` connects to the socket file at `/var/spool/postfix/private/quota-status` that was put there by Dovecot. It will use it to ask Dovecot whether the user is over quota in which case the email would get rejected.
### Test it
If you are curious to see this working, then set Johns mailbox quota to 5 KB:
```sql
# mysql mailserver
mysql> update virtual_users set quota=4000 where email='john@example.org';
```
Send him a few emails using the swaks tool:
```
swaks --server localhost --to john@example.org
```
After a few emails you will see the rejection message:
```
-> RCPT TO:john@example.org
<** 452 4.2.2 john@example.org: Recipient address rejected: Mailbox is full and cannot receive any more emails
```
### Troubleshooting
These are things you should consider if quotas do not seem to work properly:
- Check if you have enabled “quota” in the “mail\_plugins” in the 10-mail.conf file.
- Your users may complain that they have deleted many emails but are still over quota. Let them check if they actually emptied the _Trash_ folder. Of course emails in that folder also contribute to the disk space usage. Once the Trash folder is expunged the problem should be gone. You may also allow your users more space in the Trash folder. Thats explained in the [Dovecot documentation](https://doc.dovecot.org/configuration_manual/quota/#quota-rules).
- If you directly remove files from a users Maildir instead of properly accessing the mailbox using IMAP then you will screw up the quota calculation. In that case let Dovecot recalculate the quota:
`doveadm quota recalc -u john@example.org`
### Automatic warning emails
The last step is to inform the poor users if they accidentally went over quota. After all they do not necessarily recognize that on their own. Lets do that by sending them an email with a warning. Yes, we will make sure that the email gets through even if the quota is reached.
Edit the `90-quota.conf` file again. Add this section to the file (derived from the [Dovecot documentation](https://doc.dovecot.org/configuration_manual/quota/#quota-warning-scripts)):
```
plugin {
quota_warning = storage=95%% quota-warning 95 %u
quota_warning2 = storage=80%% quota-warning 80 %u
}
service quota-warning {
executable = script /usr/local/bin/quota-warning.sh
unix_listener quota-warning {
user = vmail
group = vmail
mode = 0660
}
}
```
This section defines two automatic quota warnings. The first (quota\_warning) is triggered if the user reaches 95% of the quota. The second (quota\_warning2) at 80%. These lines follow this schema:
- **Trigger** (e.g. “storage=95%”). The “%” sign needs to be used twice if you want to emit a literal percent sign. So this is not a typo.
- The **socket** you want to call in that case. Our socket is the “service quota-warning” that calls a shell script.
- Additional **parameters** that are passed to the shell script in our case. They tell the script the percentage that has been reached (e.g. 95) and the address of the user who should get the warning.
Apparently we need the script to run. So please create a new file at `/usr/local/bin/quota-warning.sh` and put these lines into it:
```
#!/bin/sh
PERCENT=$1
USER=$2
cat << EOF | /usr/lib/dovecot/dovecot-lda -d $USER -o "plugin/quota=maildir:User quota:noenforcing"
From: postmaster@webmail.example.org
Subject: Quota warning - $PERCENT% reached
Your mailbox can only store a limited amount of emails.
Currently it is $PERCENT% full. If you reach 100% then
new emails cannot be stored. Thanks for your understanding.
EOF
```
Make this file executable:
```
chmod +x /usr/local/bin/quota-warning.sh
```
Time to restart Dovecot again:
```
systemctl restart dovecot
```
Dovecots quota limits can be configured in many ways. If you have special needs then give [their documentation](https://doc.dovecot.org/configuration_manual/quota/) a look.

View file

@ -0,0 +1,27 @@
---
title: Testing IMAP
lastUpdated: 2023-10-04
slug: ispmail-bookworm/testing-imap/
sidebar:
order: 150
---
You have already completed the configuration of Dovecot. So fetching emails via IMAP should already work. Lets give it a try using a simple-looking but powerful IMAP client: _mutt_.
```
mutt -f imaps://john@example.org@**webmail.example.org**
```
The connection URL may look a little confusing because of the two “@” characters. Usually _mutt_ expects the format `imaps://user@server`. And as we use the email address as the “user” part you get this look.
You should get prompted for the password which we set to “summersun”. If you get any certificate warnings then check if you used the correct server name to connect to and if you completed the certificate/LetsEncrypt part earlier in this guide.
After logging in you will see an empty inbox:
![The mutt mail client showing an empty inbox](images/testing-imap-mutt-empty-inbox.png)
Or if you have played around with quotas in the previous section you will see a couple of emails plus the quota warnings:
![The mutt mail client showing quota warning emails](images/testing-imap-mutt-inbox-quota-mails.png)
Very good IMAP connections and authentication works. Thats all we wanted to test. Exit _mutt_ by pressing “q”.
Congratulations. At this point your server can already receive emails.

View file

@ -0,0 +1,156 @@
---
title: Webmail using Roundcube
lastUpdated: 2023-10-04
slug: ispmail-bookworm/webmail-using-roundcube/
sidebar:
order: 160
---
import { Aside } from "@astrojs/starlight/components";
<Aside type="tip" title="Optional feature">
This feature is completely optional. If you are eager to get finished then skip this page and maybe come back later. You can still access your mail server using a mail user agent like Thunderbird.
</Aside>
Power users may still want to use a mail client like Thunderbird. But most users nowadays seem to prefer reading their emails in the web browser. Let us install a web application for that purpose: [Roundcube](https://roundcube.net/). Roundcube is the software that was also used in the previous versions of this guide. So if your users are used to it… just stay with it.
## Installation
Start by installing the software packages:
```
apt install -y roundcube roundcube-plugins \
roundcube-plugins-extra roundcube-mysql
```
Roundcube stores user settings in the database. So you will get asked to set up database access:
![Debconf asking whether you want to have the database set up for you](images/webmail-roundcube-db-setup1.png)
Choose Yes.
When asked for a password just press _ENTER_.
![Debconf asking for the database password](images/webmail-roundcube-db-setup2.png)
## Configure Apache
Do you remember that earlier in this guide I asked you how want to name your mail server? Whether you want to use one common name like “webmail.example.org” for all your domains? Or if you prefer different host names for each domain like “webmail.domain1.com” and “webmail.domain2.com”? If you want to use just more then you will have to create one virtual host configuration per domain. The following instructions will just deal with one common host name.
To get Apache to serve the Roundcube application you need to edit the <tt>/etc/apache2/sites-available/**webmail.example.org**-https.conf</tt> file. I suggest you change the `DocumentRoot` line to:
```
DocumentRoot /var/lib/roundcube/public_html
```
All URLs are relative to that directory. So if you go to `https://webmail.example.com/` then files are looked up in that directory.
Also add this line within the same `VirtualHost` section to add a couple of prepared security settings:
```
Include /etc/roundcube/apache.conf
```
And as usual Apache needs to be restarted after the configuration change:
```
systemctl restart apache2
```
Check that Apache is running properly:
```
systemctl status apache2
```
In case of a problem run “`apache2ctl configtest`” to find the cause.
## Limit access to localhost
The main configuration file of Roundcube is located at `/etc/roundcube/config.inc.php`. Feel free to customize the file. Fortunately nowadays the basic settings are already as we need them. However these two settings need to be changed by you:
```
$config['imap_host'] = "tls://webmail.example.org:143";
$config['smtp_host'] = 'tls://webmail.example.org:587';
```
So now when your users enter `https://webmail.example.org/` in their browser they should get the Roundcube login form:
![Roundcube login dialog](images/roundcube-login-dialog.png)
Keep in mind that we are using the email address as the account name of the user. So when logging in please enter the email address as the user name. E.g. john@example.org and password summersun.
<Aside type="tip" title="Login failed? Storage server cant be reached?">
In that case please double check your Dovecot 10-ssl.conf file if you set the path to your Lets Encrypt certificate correctly. Also check the /var/lib/roundcube/logs/errors.log file for errors.
</Aside>
## Plugins
Roundcube comes with various plugins that you can offer your users. I recommend at least these two:
- password: Let the user change their access password.
- managesieve: Let the user manage rules that apply to incoming email. They can move mails to specific folders automatically for example.
Again edit the `/etc/roundcube/config.inc.php` file and look for the _plugins_ configuration. To enable the recommended plugins change it to:
```
$config['plugins'] = array(
'managesieve',
'password',
);
```
### password plugin
Plugins are configured through files located in the `/etc/roundcube/plugins` directory. Lets begin with the password plugin. Edit the `/etc/roundcube/plugins/password/config.inc.php` file.
Oops, that file looks pretty empty. But it refers us to an example file at `/usr/share/roundcube/plugins/password/config.inc.php.dist`. There are many different methods to let users change their passwords. As we store that information in the SQL database, that is the part we need to set up.
<Aside type="tip" title="No more doveadm">
In previous versions of this guide I used the “doveadm pw” command to generate passwords. This is no longer needed. Roundcube can now generate the passwords in the right format to be understood by Dovecot.
</Aside>
Remove the empty definition line of $config from your `config.inc.php` file. Lets go through the required settings one by one:
- `$config['password_driver'] = 'sql';`\
Simple. Use SQL as a backend.
- `$config['password_minimum_length'] = 12;`\
Allow no passwords shorter than 12 characters. I consider longer passwords more secure than short passwords with weird characters. You can even choose a larger minimum.
- `$config['password_force_save'] = true;`\
This will overwrite the password in the database even if it hasnt changed. It helps us improve the strength of the password hash by re-encoding it with a better algorithm even if the user chooses to keep his old password.
- `$config['password_algorithm'] = 'blowfish-crypt';`\
The cryptographic algorithm to encode the password. This one is considered very secure and supported by Dovecot.
- `$config['password_algorithm_prefix'] = '{CRYPT}';`\
Prepend every password with this string so that Dovecot knows how we encrypted the password.
- `$config['password_db_dsn'] = 'mysql://mailadmin:gefk6lA2brMOeb8eR5WYaMEdKDQfnF@localhost/mailserver';`\
Connection information for the local database. Use your own password for the _mailadmin_ (!) database user here. We cannot use the restricted _mailserver_ user because we have to write to the database if the user changes his password.
- `$config['password_query'] = "UPDATE virtual_users SET password=%P WHERE email=%u";`\
The SQL query that is run to write the new password hash into the database. %P is a placeholder for the new password hash. And %u is the logged-in user and conveniently matches the email address.
Make sure that this config file is not world-readable:
```
chown root:www-data /etc/roundcube/plugins/password/config.inc.php
chmod u=rw,g=r,o= /etc/roundcube/plugins/password/config.inc.php
```
Try it. Log into Roundcube as `john@example.org` with password summersun. Go to the _Settings_. Choose _Password_. Enter a new password twice. You should get a success message at the bottom right. Now logout and login with the new password. Does it work? Great.
### sieve plugin
[Sieve](https://en.wikipedia.org/wiki/Sieve_\(mail_filtering_language\)) is a simple programming language to be used for server-side rules. Dovecot executes these rules every time a new email comes in. There are global rules that are executed for every email. And of course every user/mailbox can have its own rules. To manage sieve rules Dovecot offers the _managesieve_ interface that you enabled earlier. So we just need to tell Roundcube how to access it.
The configuration file for Roundcubes _managesieve_ plugin is found at `/etc/roundcube/plugins/managesieve/config.inc.php`. Edit the file and again remove the empty or comment the `$config` line. You can again find all possible configuration options in the `/usr/share/roundcube/plugins/managesieve/config.inc.php.dist` file.
This time just one setting is required to tell Roundcube which server to talk to:
```
$config['managesieve_host'] = 'localhost';
```
Sieve rules are stored in a special syntax on the server. This is an example that moves all incoming emails to the _test_ folder that have “test” in the subject:
```
require ["fileinto"];
if header :contains "subject" "test"
{
fileinto "INBOX/test";
}
```
You do not need to learn this syntax though. Roundcubes sieve rule editor is way more user-friendly.
Try adding a sieve rule for `john@example.org` in Roundcube. That feature is located in Settings/Filters. You will find the machine-readable sieve code at `/var/vmail/example.org/john/sieve/roundcube.sieve`.
The rule editor looks like this:
![Roundcube's sieve rule editor](images/webmail-roundcube-sieve-editor.png)

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 68 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 41 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 51 KiB