More content migrated

This commit is contained in:
Christoph Haas 2024-11-04 01:28:15 +01:00
parent db13ca20d8
commit ec39ffed28
4 changed files with 330 additions and 2 deletions

View file

@ -0,0 +1,47 @@
---
title: Let Postfix access MariaDB
lastUpdated: 2023-10-04
slug: ispmail-bookworm/making-postfix-get-its-information-from-the-mariadb-database
sidebar:
order: 100
---
import { Aside } from "@astrojs/starlight/components";
In the previous chapter you have created the SQL database schema and inserted some data to play with. Lets start with the entry point for all email on your system: Postfix. So we need to tell Postfix how to get the information from the database. First lets tell it how to find out if a certain domain is a valid email domain.
## virtual\_mailbox\_domains
As described earlier a mapping in Postfix is just a table that contains a left-hand side (LHS) and a right-hand side (RHS). To make Postfix get information about virtual domains from the database we need to create a cf file (_configuration file_). Start by creating a file called `/etc/postfix/mysql-virtual-mailbox-domains.cf` for the virtual\_mailbox\_domains mapping. Make it contain:
<pre>
user = mailserver
password = **x893dNj4stkHy1MKQq0USWBaX4ZZdq**
hosts = 127.0.0.1
dbname = mailserver
query = SELECT 1 FROM virtual_domains WHERE name='%s'
</pre>
Please enter your own password for the _mailserver_ database user here. It is the first one you created before.
Imagine that Postfix receives an email for `somebody@example.org` and wants to find out if example.org is a virtual mailbox domain. It will run the above SQL query and replace %s by example.org. If it finds such a row in the virtual\_domains table it will return a 1. Actually it does not matter what exactly is returns as long as there is a result. Remember the puppies and kittens?
Now you need to make Postfix use this database mapping:
<pre class="wrap">
postconf virtual_mailbox_domains=mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
</pre>
The “postconf” command conveniently adds configuration lines to your `/etc/postfix/main.cf` file. It also activates the new setting instantly so you do not have to reload the Postfix process.
The test data you created earlier added the domain “example.org” as one of your mailbox domains. Lets ask Postfix if it recognizes that domain:
<pre class="wrap">
postmap -q example.org mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf
</pre>
<Aside type="tip" title="Access denied?">
If you get an error like “postmap: warning: connect to mysql server 127.0.0.1: Access denied for user mailserver@localhost'” then please double check the password of the mailserver database user. It is either wrong in the CF file or you did not create the database user correctly using the GRANT query on the previous page.
</Aside>
You should get 1 as a result. That means your first mapping is working. Feel free to try that with other domains after the `-q` in that line. You should not get a response.

View file

@ -194,3 +194,64 @@ SSLCertificateKeyFile /etc/letsencrypt/live/**webmail.example.org**/privkey.pem
</pre> </pre>
This virtual host configuration looks suspiciously similar to the HTTP virtual host above. It just listens on port 443 (standard port for HTTPS) instead of port 80. And it uses the “SSLEngine” that handles encryption and gets information about the certificate for your web server (that is shown to your users) and the private key (that the web servers uses to decrypt the users communication). This virtual host configuration looks suspiciously similar to the HTTP virtual host above. It just listens on port 443 (standard port for HTTPS) instead of port 80. And it uses the “SSLEngine” that handles encryption and gets information about the certificate for your web server (that is shown to your users) and the private key (that the web servers uses to decrypt the users communication).
Enable the SSL module in Apache:
<pre>
a2enmod ssl
</pre>
Then enable the virtual host for HTTPS:
<pre>
a2ensite **webmail.example.org**-https
</pre>
And _restart_ the web server. A _reload_ is not sufficient this time because you added a module.
systemctl restart apache2
Now when you point your web browser to
**webmail.example.org**, your browser should tell you that it trusts the web sites certificate:
![Browser bar showing successful encryption](https://workaround.org/wp-content/uploads/2018/01/Auswahl_063-1.png)
(Yes, sorry, this is not **webmail.example.org**. But I do not own the example.org domain and thus cannot get a valid certificate for it. This is my own site.)
So should you keep the HTTP virtual host? Yes. First for the HTTP-&gt;HTTPS redirection. And second to keep _certbot_ working.
## Redirect HTTP to HTTPS
Sometimes users forget to enter https://… when accessing your webmail service. So they access the HTTP web site. We obviously dont want them to send their password over HTTP. So we should redirect all HTTP connections to HTTPS.
One exception though. Lets Encrypt will use HTTP to verify your challenge token. So we need to serve files at http://**webmail.example.org**/.well-known/acme-challenge/… directly while redirecting all other requests to HTTPS. You can accomplish that by putting these lines inside the &lt;VirtualHost&gt; section of your `/etc/apache2/sites-available/webmail.example.org-http.conf` file:
```
RewriteEngine On
RewriteCond %{REQUEST_URI} !.well-known/acme-challenge
RewriteRule ^(.*)$ https://%{SERVER_NAME}$1 \[R=301,L\]
```
This requires the _rewrite_ module to be enabled in Apache. That is simple though:
```
a2enmod rewrite
systemctl restart apache2
```
So now entering http://**webmail.example.org** will redirect you to https://**webmail.example.org**.
## Automatic certificate renewal
The _certbot_ package automatically adds a timed job that runs twice a day at random times. The random part is important to avoid millions of server hammering the LetsEncrypt service at the same second.
<Aside type="tip" title="Systemd timer instead of a Cron job">
This job is not a classic Cron job but instead latches into systemd. You can find the _timer_ definition in the `/lib/systemd/system/certbot.timer` file. That timer triggers the renewal service defined in `/lib/systemd/system/certbot.service`.
To find out if the service has run and when the next occurence will be, run “systemctl status certbot.timer”.
There is also a Cron file at /etc/cron.d/certbot. Dont be confused. This job will not do anything due to the “`-d /run/systemd/system`” condition that checks if systemd is installed. And Debian nowadays uses systemd.
</Aside>
So the renewal already happens automatically. Should it fail then LetsEncrypt start sending you reminder emails that your certificate should be renewed. Thats a clear sign that something went wrong with the automatic renewal.
There is one puzzle piece missing though. Even if the renewal worked it will only update the certificate files. But the software components Postfix, Dovecot and Apache will not notice the change. So we need to add a so called _post-hook_ to certbot that triggers a restart of all processes thereafter.
For that purpose edit the /etc/letsencrypt/cli.ini file and add:
```
post-hook = systemctl restart postfix dovecot apache2
```
Well done. You have implemented Lets Encrypt for all your services now. Lets go on.

View file

@ -0,0 +1,219 @@
---
title: Preparing the database
lastUpdated: 2023-10-03
slug: ispmail-bookworm/prepare-the-database
sidebar:
order: 90
---
import { Aside } from "@astrojs/starlight/components";
import { Tabs, TabItem } from '@astrojs/starlight/components';
Now its time to prepare the MariaDB database that stores the information that controls your mail server. In the process you will have to enter [SQL](http://en.wikipedia.org/wiki/SQL) queries the language of relational database servers. You may enter them in a terminal window using the mysql command. But if you are less experienced with SQL you may prefer using a web interface. Thats what you installed _[Adminer](https://www.adminer.org/)_ for.
<Aside type="tip" title="Why not PostgreSQL?">
Several comments show that some system administrators prefer the PostgreSQL database engine over MariaDB. Count me in. So why is this guide still using MariaDB/MySQL? Mainly for historical reasons. I want to spare you any unneccessary changes when migrating to a new version. If you are familiar with PostgreSQL however you can easily adapt this guide to PostgreSQL. There are dovecot-pgsql and postfix-pgsql packages. And Adminer works with PostgreSQL as well.
It may be worth using SQLite instead. Because on small _family-and-friends_ servers that would be totally sufficient. Let us see if it will be added to this guide at a later time.
</Aside>
## Setting up Adminer
Basically Adminer is just a couple of PHP files served from your Apache web server. The setup is simple. Edit your /etc/apache2/sites-available/**webmail.example.org**-https.conf file and put this line anywhere between the &lt;VirtualHost&gt; and the &lt;/VirtualHost&gt; tags:
```
Alias /adminer /usr/share/adminer/adminer
```
Reload the Apache process:
```
systemctl reload apache2
```
<Aside type="danger" title="Security warning">
Having an SQL admin interface publicly available on your web site is an invitation for internet scoundrels to do bad things. Consider protecting the /adminer location by an additional password. The Apache documentation shows you how to do that. Or use a less obvious path than “/adminer” in the Alias.
</Aside>
You will not be able to login yet. The only available database user is root, but it is only usable from the shell by default not over a network.
## Generate two random passwords
In this section you will create the basic database “mailserver” and two users. One user (“mailadmin”) will be able to change the data in the database and is meant for you. The other user (“mailserver”) can only read from the database and is meant for the server processes.
Use the _pwgen_ tool to create two random passwords for these users:
```
pwgen -s1 30 2
```
Take a note of the passwords or store them somewhere safe.
## Create the mailserver database
This step is simple. Connect to the database using the mysql command:
```
mysql
```
You should see the MariaDB prompt that allows you to enter further SQL commands:
```
MariaDB [(none)]>
```
Now you are expected to speak SQL. To create a new database for our needs enter:
```
CREATE DATABASE mailserver;
```
You will be told that your query was OK and that one new row was added.
## Create the database users
Now you have an empty database. Let us give the “mailadmin” database user the required privileges to manage it.
You are still connected to the database, right? To create a user with full permissions enter this SQL command. Please use the **first** password you just generated instead of mine:
<pre class="wrap">
grant all on mailserver.* to 'mailadmin'@'localhost' identified by '**gefk6lA2brMOeb8eR5WYaMEdKDQfnF**';
</pre>
Also create the read-only user that will grant Postfix and Dovecot database access later (use your **second** random password here).
<pre class="wrap">
grant select on mailserver.* to 'mailserver'@'127.0.0.1' identified by '**x893dNj4stkHy1MKQq0USWBaX4ZZdq**';
</pre>
<Aside type="note" title="127.0.0.1 versus localhost">
Wait a minute. Why is there “127.0.0.1” instead of “localhost” in the second SQL command? Is that a typo? No, its not. Well, in network terminology those two are identical. But MariaDB (and Oracles MySQL) distinguishes between the two. If you initiate a database connection to “localhost” then you talk to the socket file which lives at /var/run/mysqld/mysqld.sock on your server. But if you connect to “127.0.0.1” it will create a network connection talking to the TCP socket on port 3306 on your server. The difference is that any process on your server can talk to 127.0.0.1. But the socket file has certain user/group/other permissions just like any other file on your file system. Postfix will be restricted to its /var/spool/postfix directory and cannot by default access that socket file. So by using 127.0.0.1 we circumvent that limitation.
When you use _Adminer_ you will have to use localhost as a database server when using the mailadmin user but 127.0.0.1 when using the mailserver user.
</Aside>
Now you can use _Adminer_ to log in using the _mailadmin_ account and the **first** password:
![](https://workaround.org/wp-content/uploads/grafik.png)
You should get logged in and see the “mailserver” database:
![](https://workaround.org/wp-content/uploads/image-6.png)
## Creating the database tables
Do you remember that I introduced three Postfix _mappings_ earlier? One for _virtual domains_, one for _virtual aliases_ and another for _virtual users_? Each of the mappings needs a database table that you will create now. Feel free to use _Adminer_. I will however also show the SQL statement to create the tables that you can enter on the mysql command-line tool. Below you can click on either \[Adminer\] or \[SQL\] to choose.
The first table to create is…
## virtual\_domains
This table just holds the list of domains that you will use as _virtual\_mailbox\_domains_ in Postfix.
<Tabs syncKey="sql">
<TabItem label="Adminer">
| **Column** | **Purpose** |
| ---------- | -------------------------------------------------------------------------------- |
| id | A unique number identifying each row. It is added by the database automatically. |
| name | The name of the domain you want to receive email for. |
</TabItem>
<TabItem label="SQL">
```sql
USE mailserver;
CREATE TABLE IF NOT EXISTS `virtual_domains` (
`id` int(11) NOT NULL auto_increment,
`name` varchar(50) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
```
</TabItem>
</Tabs>
### virtual\_users
The next table contains information about your users. Each mail account takes up one row.
<Tabs syncKey="sql">
<TabItem label="Adminer">
| **Column** | **Purpose**|
| ---------- | --- |
| domain\_id | Contains the number of the domains id in the _virtual\_domains_ table. This is called a _[foreign key](https://en.wikipedia.org/wiki/Foreign_key)_. A “delete cascade” makes sure that if a domain is deleted that all user accounts in that domain are also deleted to avoid orphaned rows. |
| email | The email address of the mail account.|
| password | The hashed password of the mail account. It is prepended by the [password scheme](https://doc.dovecot.org/configuration_manual/authentication/password_schemes/). By [default](https://doc.dovecot.org/configuration_manual/authentication/password_schemes/) it is `{BLF-CRYPT}` also known as _bcrypt_ which is considered very secure. Previous ISPmail guides used `{SHA256-CRYPT}` or even older crypt schemes. Prepending the password field the hashing algorithm in curly brackets allows you to have different kinds of hashes. So you can easily migrate your old passwords without locking out users. Users with older schemes should get a new password if possible to increase security. |
| quota | The number of bytes that this mailbox can store. You can use this value to limit how much space a mailbox can take up. The default value is 0 which means that there is no limit. |
</TabItem>
<TabItem label="SQL">
```sql
CREATE TABLE IF NOT EXISTS `virtual_users` (
`id` int(11) NOT NULL auto_increment,
`domain_id` int(11) NOT NULL,
`email` varchar(100) NOT NULL,
`password` varchar(150) NOT NULL,
`quota` bigint(11) NOT NULL DEFAULT 0,
PRIMARY KEY (`id`),
UNIQUE KEY `email` (`email`),
FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
```
</TabItem>
</Tabs>
### virtual\_aliases
The last table contains forwardings from an email address to other email addresses.
<Tabs syncKey="sql">
<TabItem label="Adminer">
| **Field** | **Purpose** |
| --- | --- |
| id | A unique number identifying each row. It is added by the database automatically. |
| domain\_id | Contains the number of the domains id in the virtual\_domains table again. |
| source | The email address that the email was actually sent to. In case of catch-all addresses (that accept any address in a domain) the source looks like “@example.org”. |
| destination | The email address that the email should instead be sent to. |
</TabItem>
<TabItem label="SQL">
```sql
CREATE TABLE IF NOT EXISTS `virtual_aliases` (
`id` int(11) NOT NULL auto_increment,
`domain_id` int(11) NOT NULL,
`source` varchar(100) NOT NULL,
`destination` varchar(100) NOT NULL,
PRIMARY KEY (`id`),
FOREIGN KEY (domain_id) REFERENCES virtual_domains(id) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
```
</TabItem>
</Tabs>
As described in the section about domain types there can be multiple targets for one source email address. You just would need to insert several rows with the same source address and different destination addresses that will get copies of an email. Postfix will consider all matching rows.
## Example data to play with
Too much theory so far? I can imagine. Lets populate the database with an `example.org` domain, a `john@example.org` email account and a forwarding of `jack@example.org` to `john@example.org`. We will use that information in the next chapter to play with.
To add that sample data just run these SQL queries:
```sql
REPLACE INTO mailserver.virtual\_domains (id,name) VALUES ('1','example.org');
REPLACE INTO mailserver.virtual\_users (id,domain\_id,password,email)
VALUES ('1', '1',
'{BLF-CRYPT}$2y$05$.WedBCNZiwxY1CG3aleIleu6lYjup2CIg0BP4M4YCZsO204Czz07W',
'john@example.org');
REPLACE INTO mailserver.virtual\_aliases (id,domain\_id,source,destination)
VALUES ('1', '1', 'jack@example.org', 'john@example.org');
```
Do you wonder how I got the long cryptic password? I ran…
```
doveadm pw -s BLF-CRYPT
```
…to create a secure hash of the simple password “summersun”. Once you have installed Dovecot you can try that yourself but you will get a different output. The reason is that the passwords are [salted](https://en.wikipedia.org/wiki/Salt_\(cryptography\)) to increase their security.
Remember to remove that sample data before you go live with your mail server. Thanks to the _delete cascade_ you just need to remove the virtual\_domain. The alias and the mailbox will be deleted automatically. This would be the SQL query you should run before taking your mail server into production:
```sql
DELETE FROM mailserver.virtual\_domains WHERE name='example.org';
```

View file

@ -19,6 +19,7 @@ li > br {
overflow: visible; overflow: visible;
padding-left: 3em !important; padding-left: 3em !important;
text-indent: -2em !important; text-indent: -2em !important;
background-color: var(--sl-color-gray-7);
} }
:root { :root {
@ -26,5 +27,5 @@ li > br {
} }
.starlight-aside { .starlight-aside {
margin-top: 1em !important; margin-top: 2em !important;
} }