autogenerating sidebar links

This commit is contained in:
Christoph Haas 2024-10-27 22:58:23 +01:00
parent 1546fa8199
commit 6f5cbd21aa
13 changed files with 522 additions and 34 deletions

View file

@ -1,100 +0,0 @@
---
title: The big picture
---
The mail server that you are about to set up uses several software components. Let me first explain briefly what the purpose of each software is:
- **[Debian](https://www.debian.org/)** “Bookworm” the operating system
- **[Postfix](http://www.postfix.org/)** receives incoming emails from the internet and sends out outgoing emails to other mail servers. It is the software that speaks SMTP.
- **[rspamd](https://rspamd.com/)** runs sanity checks on an incoming email to determine whether it is spam.
- **[Dovecot](https://www.dovecot.org/)** stores emails on your hard disk, applies filters and lets your users fetch their emails using the POP3 and IMAP protocols
- **[Roundcube](https://roundcube.net/)** is a webmail interface so users can read their emails using a web browser, manage their email rules and change their password
- **[MariaDB](https://mariadb.org/)** (formerly known as _MySQL_) is a database that stores information about your domains, email aliases and email accounts
## What happens when someone sends you an email?
Let us assume that you are responsible for the email domain _example.org_ and someone on the internet sends an email to `john@example.org`. This is what happens step by step:
1. **REMOTE:** Hey, DNS server. I have an email for someone in the _example.org_ domain. Can you tell me the name of the responsible mail server?
**DNS:** According to the DNS zone of _example.org_ it is _mx.example.org_.
2. **REMOTE**: Nice. Do you have an an IP address for _mx.example.org_?
**DNS:** I have an IPv4 address here. It is _85.25.72.76_.
3. _REMOTE connects to that IP address on TCP port 25 which is by definition used for SMTP the [simple mail transport protocol](https://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol)._
**POSTFIX:** Welcome. I am Postfix. Who is there? (“220 mx.example.org ESMTP Postfix”)
**REMOTE:** Hi, I am remote server. (“EHLO remoteserver”)
**POSTFIX:** Nice to meet you. I can offer a few features like pipelining and encryption by the way. (“STARTTLS, PIPELINING, SIZE 4000000, …”)
**REMOTE:** Lets switch to an encrypted connection. (“STARTTLS”)
_(The connection is now using TLS encryption.)_
**REMOTE:** I have an email from `donald@duck.com` here. (“MAIL FROM:<`donald@duck.com`>”)
**POSTFIX:** I see. (“Ok”)
**REMOTE:** The email is meant for `john@example.org`. (“RCPT TO:<`john@example.org`>”)
4. **POSTFIX:** Hey database. _(Connects to TCP port 3306 on the local host to talk to MariaDB.)_ Could you check if _example.org_ is one of our mail domains? (“SELECT … from virtual_domains …”)
**MariaDB:** Yes, I have a domain like that.
**POSTFIX:** Oh, good. And do you have a mailbox or forwarding for someone called `john@example.org`? (“SELECT … from virtual\_aliases/virtual\_users …”)
**MariaDB:** Yes, there is a mailbox for that address.
**POSTIFX:** Hey, remote server. The recipient looks good. (“Ok”)
**REMOTE:** Roger. Then heres the actual email. (“DATA”)
_(The remote server sends the email header and body.)_
5. _(POSTFIX connects to port 11332 on the local host to reach the rspamd.)_
**POSTFIX**: Hey, rspamd. I have a new email here. Could you give it a look for, you know, spam and stuff?
**RSPAMD:** Sure. Well, there are a few minor issues. But generally the mail looks good. I suggest you accept it.
**POSTFIX:** Hey, remote server. Your email is fine.
6. _(Postfix uses a socket file at /var/spool/postfix/private/dovecot-lmtp to talk to Dovecot.)_
**POSTFIX:** Hey, Dovecot. Here is a new email for `john@example.org`.
**DOVECOT:** Got it.
7. _(Dovecot checks for additional Sieve rules and then stores the email on disk at /var/vmail/example.org/john/Maildir/INBOX)_
Phew, that was a lot. I hope it helps to understand though how the different components are involved until an email gets delivered.
The next situation I would like to explain is…
## What happens if a user fetches their email using IMAP/POP3?
This process is way simpler. It looks like this:
![](https://workaround.org/wp-content/uploads/fetch-email.png)
![](https://workaround.org/wp-content/uploads/fetch-email.png)
User connects through IMAP
1. The user usually has a mail client (also called a _[mail user agent](https://en.wikipedia.org/wiki/Email_client)_) that can use the POP3 or IMAP protocol to fetch emails from the mail server. I prefer Thunderbird for example. That mail client connects to the POP3 (TCP 110) or IMAP (TCP 143) port, sends the STARTTLS command that initiates an encrypted connection and sends the users username (which equals the email address in our case) and their password. The client may as well use the secure TLS-encrypted ports directly 995 for POPs and 993 for IMAPs. If possible users should use IMAP. POP3 is pretty much deprecated nowadays and lacks support for multiple folders on the server.
2. Dovecot sends a query to the MySQL database and verifies that the username and password belong to a known user. The password is not stored in plain text in the database. Instead it computes the password [hash](https://en.wikipedia.org/wiki/Hash_function) and compares it to the hash in the database. If the password is wrong then Dovecot will refuse the login.
3. We have a fixed scheme how emails are stored on disk. So if the user is called `jane@example.org` then Dovecot will look for email files in _/var/vmail/example.org/jane/Maildir/…_ and send the user the requested emails.
Nowadays many users seem to prefer webmail to a mail client installed on their computers. As an email power user I honestly do not understand that trend. But who am I to judge. So lets take a look how webmail works technically:
## What happens if a user reads their email using web mail?
The Roundcube software that provides the web mail interface is basically a PHP software that is a gateway between HTML pages and a built-in IMAP client. So when a user uses their browser to connect to the web mail interface…
![](https://workaround.org/wp-content/uploads/webmail.png)
![](https://workaround.org/wp-content/uploads/webmail.png)
User connects through webmail
1. The user points the browser to the HTTPS URL of the webmail interface. Apache web server receives the connection and runs the PHP scripts of Roundcube. Roundcube shows a login form for the username and password. The user enters the username (that equals the email address) and the password and submits the login form.
2. Roundcube connects to Dovecot using IMAP and forwards the username and password to check if they are valid.
3. Dovecot treats this connection similar to a mail client connection. It queries the MariaDB database to verify the username (=email address) and password.
4. If the authentication was successful Dovecot fetches the mail files from disk and sends them through IMAP to Roundcube. Roundcube then renders the emails as HTML and the user can read it.
So you see that the web mail access also works through IMAP. The user does not realize that though.
Okay, the final scenario I would like to explain is…
## What happens if the user wants to send an email to the internet?
Of course your users also want to send emails to other internet users. But they cannot send the email directly to the destination mail server. First their mail client does not know which destination server is responsible for the recipient (hint: DNS) that functionality just is not built in. And second the user is likely assigned a _dynamic IP address_ which is blocked by most mail servers because they tend to get abused by infected Windows PCs that send out spam. So the correct way to send an email to the internet is through your mail server. This is called _relaying_ because your mail server acts as a relay. In this example your user wants to send an email to `fred@example.net`.
![](https://workaround.org/wp-content/uploads/relaying.png)
![](https://workaround.org/wp-content/uploads/relaying.png)
Relaying/sending email via the mail server
1. The user writes the email in their mail client and clicks on “Send”. The mail client establishes an SMTP connection to Postfix. To make sure that the user is allowed to send email through your system it requires a username and password along with the email. This information is sent in an encrypted way.
2. Postfix could now check the password in the database directly. But as Dovecot already knows how to handle authentication it is easier to ask Dovecot to verify the username and password. (SMTP authentication in Postfix is surprisingly ugly.)
3. Dovecot now sends a query to the MariaDB database to check if the username and password (hash) are correct and tells Postfix the result.
4. Postftix knows now that it is authorized to send the email on behalf of the user. It tells the user that it successfully accepted the email. The email is put into Postfixs mail queue for further processing. Postfix will now query a DNS (name server) to determine the responsible destination mail server. As the recipient has an “…@example.net” address it checks the MX record of the “example.net” domain and then gets the respective IP address.
5. Postfix now knows which mail server to send the email to. It opens an SMTP connection and delivers the email.

View file

@ -1,44 +0,0 @@
---
title: ISPmail guide for Debian 12 “Bookworm”
lastUpdated: 2023-09-24
---
Once upon a time there were many mail servers on the internet. If your organisation wanted to receive and send emails then you would have your system administrator set up a mail server. He would add a DNS record and create a cryptic Sendmail configuration file.
Fast forward a few decades and suddenly we have arrived in a monopolistic dystopy. The world of internet services is dominated by a few companies. Having worked in IT for 25 years I feel frustrated about the ignorance and laziness of decision makers. Cloud providers rip obscene profits and lock clueless customers in, while their IT staff is evolving into dumb victims instead of doing their actual job.
If you are like me then you want to stay as independent as possible. And that includes being in charge of your own email service. Maybe not for your employer but at least for yourself, your friends and family. Become your own internet service provider (ISP) for email. Hence the name: ISPmail.
This is a complete and free guide that teaches you how to set up and run your own mail server. At the end of the guide you will have your own fully featured mail server based on open-source software using a cheap virtual server for a few bucks per month. And you will have learned all about the various components, protocols and technologies. This guide has been battle-tested by thousands of other sysadmins and constantly evolved over the last 20 years. I am publishing this guide without any commercial motives, so all you would have to invest is your time.
## What your mail server can do
- **Receive** emails on your domains.
- **Filter** out spam. (We will not deal with detection of Wind*ws malware though.)
- **Send** emails out to any other servers/domains on the internet. Connections will be encrypted when possible.
- Add automatic cryptographic signatures (**[DKIM / Domain Keys](https://en.wikipedia.org/wiki/DomainKeys)**) to outgoing emails to prove that you are the owner of your domain.
- **Store** as many emails for as many email addresses as you have disk space. Set limits (“quotas”) per user. The only limit is the size of your disk.
- Let your users fetch email using **IMAP** or **POP3** and send email through your servers using **SMTP**.
- Allow users to manage server-based **filter rules**. Distribute incoming emails to different folders. Forward copies. Or send out-of-office notifications.
- Provide a **webmail** interface so users can access their emails securely from any location using a web browser.
- Mitigate **brute force** attacks.
## What you will need
- **Linux experience**. Preferably a Debian-derivative. No godlike skills required. But know your basics: navigating through the file system, editing files, watching log files. Have some basic understanding of DNS. Bonus points if you have played with an SQL database and know about SELECT, INSERT, rows, columns.
- **Time**. 2 hours to 2 days.
- A **server** that runs Debian Bookworm. 1 GB of RAM and a 20 GB disk/SSD is fine for your friends and family. Rent a cheap virtual server. Or use a decommissioned laptop. Other Linux distributions likely come with other versions of the software and have different configurations and paths on disk. It will work but you will need to deviate from this guide.
- The provider providing you with the server needs to allow **SMTP send and receive**. These providers are known to be troublesome:
- Hetzner (DE, FI, US): Will allow SMTP after a month of paying for their service and per a request issued at https://console.hetzner.cloud/limits
- DigitalOcean (US): Blocks SMTP and tells you that running your own mail server is a bad idea. Avoid.
- Your server needs to have a public **IP address** that does not belong to a range of typical ISP customers. You usually cant operate the mail server from a dialup IP address at home because those IP ranges are blacklisted by most other mail servers. Make sure that your IP address is not [blacklisted](https://multirbl.valli.org/) before you start. If you rent a virtual server from your favorite hosting company you probably wont have any problems.
- An **internet domain** (or several) to receive emails for. You need to be able to set A, MX and TXT records for that domain. You should also be able to set PTR records for your IP address because some mail servers on the internet require you to have matching forward and reverse DNS records.
- **Patience**. We will proceed slowly and after every step ensure that are still on track. Dont hurry and skip parts even if they appear confusing at first. If you get lost just submit your question at the bottom of any page throughout this guide and help is on the way. Or join the [chat channel](https://riot.im/app/#/room/#ispmail:matrix.org).
## What this is not about
If you just want to have a working mail server and do not care how it works then this guide is not for you. Check out ready solutions like [mailinabox](https://mailinabox.email/) or [iRedMail](http://www.iredmail.org/). Running a mail server requires technical understanding. And thats what the ISPmail guide is for. Experience from giving support to other sysadmins shows that most problems appear because some detail in a complex setup goes wrong and they have no idea how to fix it. Email has evolved a lot over the past 40 years. Go with ready solutions if you like. But I have a feeling that we meet again. And you will probably not save time either taking the supposedly easy route.
## Ready?
The entire tutorial is split into several pages. You can find the different chapters on the left. The navigation within a chapter can be found on the right. Lets go.

View file

@ -1,125 +0,0 @@
---
title: Migrating from your old (Bullseye) server
---
## Upgrade or fresh installation?
I recommend that you set up a new server and make sure that the new mail server is working correctly before you start migrating existing email users to it. You may argue that Debian can be upgraded easily using “apt-get dist-upgrade” but that is very dangerous on a live mail server. Automatic configuration changes may have evil side effects and you risk losing emails. At least you are causing a downtime for your users.
If you follow my advice then get a new server and install Debian Bookworm on it. Once the user mailboxes are migrated and all works well you can tear down the old server.
Once your new server is installed come back here.
## Read before you type
Follow this guide to the end. Only then start attempting a migration. Inform your users about the migration and set a time when you intend to move email accounts to the new server. The change will require a DNS change which takes a while to be visible worldwide so your users will have a period when incoming email is delayed. If you proceed careful though not a single email will be lost.
So you have your new server up and running and did everything to make it a working mail server? You really read all the pages in this guide and did what they told you? Okay, then lets start.
## Turn down the TTL of your MX record
The DNS “MX” record for your domain contains the hostname of your mail server. When switching to the new server you need to change the MX record. Every DNS record has a TTL (time-to-live) that defines the period of time that a record will stay valid even after you change it. Usually that TTL is rather high like 86400 seconds (=1 day). This information is used by caching name servers that they can use the cached values for a day. Turn that TTL down temporarily to 60 seconds so that the rest of the internet will pick up your change quicker. However it will take a day until everyone else on the internet picks up your TTL change.
## Migrate the mailserver database
You need to copy the database that contains the control data about your email domains and accounts. Log into the old (Bullseye) server as root and back up the _mailserver_ database. That is as easy as running…
```sh
mysqldump mailserver > mailserver.sql
```
Copy that file to the new server (using _scp_) and import it there:
```sh
mysql mailserver < mailserver.sql
```
Obviously any database changes on the old server from now on will have to be done on the new server as well until the migration is done.
## Roundcube contacts
If your users are using Roundcube as a webmail interface then you should migrate their data like their contact lists. Dump the SQL from the old server:
```sh
mysqldump roundcube > roundcube.sql
```
Copy that file to the new server and import it:
```sh
mysql roundcube < roundcube.sql
```
One caveat though. To distinguish multiple mail servers Roundcube stores the servers name in the mail_host column of the users table. So as a last step change that column if your new mail server has a new FQDN/hostname by running this SQL query on the new roundcube database:
```sql
UPDATE users SET mail_host='new.mail.server';
```
## Migrate rspamd spam training data
If you have been using rspamd with the Redis backend then copy over the Redis database from your previous server. Details are found in the rspamd chapter.
## Migrate the Maildirs hot
Fortunately Dovecot uses the maildir format that stores emails as plain files on disk. Login to the new (Bookworm) server and use _rsync_ to copy over the mails from the old (Bullseye) mail server:
```sh
rsync -va oldserver:/var/vmail/ /var/vmail/
```
(Note the trailing slashes. Type them exactly as shown above or your files will end up in wrong places.)
There is no need to shut down Dovecot on your production Bullseye server. Copying the files while Dovecot is running will not break anything. This is called a “hot copy”. It may not be consistent but it will save time during the final synchronization.
## Copy certificates
Copy over everything in /etc/letsencrypt and /var/lib/rspamd/dkim from your old to the new server.
```sh
rsync -va oldserver:/etc/letsencrypt/ /etc/letsencrypt/
rsync -va oldserver:/var/lib/rspamd/dkim/ /var/lib/rspamd/dkim/
```
## Downtime
You told your users about the downtime, right? The time has come? Okay. Shut down Dovecot on both servers.
## Migrate the emails cold
Lets synchronize again. _rsync_ will only copy those files that have changed which makes it much faster than the first sync. On your new server run:
```sh
rsync -va --delete oldserver:/var/vmail/ /var/vmail/
```
(The “`--delete`” option makes sure that files that have been removed from the old server will also be deleted from the new server. So if a user has deleted an email it will be deleted on the new server as well.)
The new Dovecot version uses a slightly different indexing mechanism. So force rebuilding the users indices:
```sh
doveadm force-resync -A '*'
```
## Switch the DNS records
For all your domains you will have to change the DNS “MX” or “A” record to point to your new server.
## Enable soft_bounce
Accidents happen. And you dont want to lose emails. So run this command to enable your safety net on the new server:
```sh
postconf soft_bounce=yes
```
This makes Postfix always keep emails in the queue that it would otherwise reject. So you can fix any errors and the queue will empty. Start Postfix and Dovecot on the new server. Watch your /var/log/mail.log and run “mailq” from time to time to see what emails get stuck in the queue. If you are certain that emails can be removed from the queue then use “postsuper -d QUEUE-ID” (as shown in the “mailq” output).
Once you are certain that emails are properly received and sent you can switch off the _soft_bounce_ mode again:
```sh
postconf soft_bounce=no
```
## Shut down the old server
If possible do a final backup of the old server. If users are not complaining then dismiss the old system after a week.

View file

@ -1,207 +0,0 @@
---
title: Types of email domains
---
import { Aside } from "@astrojs/starlight/components";
This is the pretty boring but at the same time very important part of the tutorial. Do not skip it. Most problems that
readers have with their mail servers are caused by a misunderstanding of the different types of email domains. There is
nothing you need to do on your server right now. Just lean back and relax and make sure that you read and fully
understand this page. It will spare you frustration.
As explained on the previous page Postfix is the software component that speaks SMTP. It receives emails from other
server and sends email out. Of course you will only want to receive emails for your domains only.
Postfix will accept emails for any of these [classes of domains](http://www.postfix.org/ADDRESS_CLASS_README.html):
1. Canonical domains (also called _local domains_)
2. Hosted domains
- Virtual mailbox domains
- Virtual alias domains
3. Relay domains
Let us look at the first two in detail. The third one relay domains is not needed in this context.
## Canonical domains
Once upon a time users logged into servers using [Telnet](https://en.wikipedia.org/wiki/Telnet) or
[SSH](https://en.wikipedia.org/wiki/Secure_Shell) to read their email in basic text-based mail clients. Their email
addresses were often just a combination of their login name and the server name. Something like
`mpauls@server17.biology.example.net`.
At least that made things pretty simple for the server administrator. He just accepted emails for any valid login user
on that server. The mail server knew what to do. It checked if “mpauls” is found in /etc/passwd and then stored emails
in “/var/mail/mpauls”.
If you wanted to do it that way…
1. You would need to create system accounts for all users who want to receive email. Every system user is a potential
security risk because they can login to the server using a shell. If the user chooses a weak password you are
essentially inviting attackers to harm your server.
2. If you want to host many email accounts this becomes impractical.
3. Postfix cannot distinguish the local domains. If you have three local domains “example.org”, “example.com” and
“example.net” then all these email addresses would lead to the same mailbox: `john@example.org`, `john@example.com` and
`john@example.net`. So you cannot use different domains for different purposes.
To tell Postfix which domain you consider local you list them in the “mydestination” configuration setting in your
main.cf configuration file. Example:
```
mydestination = example.org, example.com, example.net
```
<Aside type="caution">
A domain never belongs to more than one class. You would get a warning in your log file and Postfix will start
behaving strangely. One domain can only have one purpose. This is a frequent mistake.
</Aside>
It is common however to use the servers hostname as a local domain. If your server is called “scully.example.com” you
could set “mydestination = scully.example.com”. Or you just set it to “mydestination = localhost” if you rather want to
use “scully.example.com” as a virtual domain. Some parts of your system may send emails to root@localhost so this is a
sane setting.
## Virtual domains
This is the class of domains that will be our workhorse. Postfix allows us to add an unlimited number of domains that
can receive emails. Thats what an ISP does and so will we.
Where do we put the list of domains? Postfix is pretty flexible. We could put it in a huge text file. We might use LDAP
if we have some kind of user directory in our organization. But in this guide we will store that information in a simple
MariaDB database.
Postfix handles hosted email addresses by checking two _mappings_:
- Virtual Aliases (“redirect this email address to another address”)
- Virtual Mailboxes (“accept email for this email address”)
So much terminology. Time for an example, isnt it? Lets tell Postfix we want to have three email addresses:
- `john@example.org`
- `jack@example.org`
- `jack@example.com` _(note that this is another domain than example.org)_
Postfix only understands _mappings_. They are much easier than they sound. A mapping is just a table that has a left and
a right column. Postfix always looks for stuff in the left column (_key_) to find more information in the right column
(_value_). Like this:
| Virtual mailbox (key) | Virtual mailbox location on disk (value) |
| --------------------- | ---------------------------------------- |
| `john@example.org` | /var/vmail/example.org/john/Maildir |
| `jack@example.org` | /var/vmail/example.org/jack/Maildir |
| `jack@example.com` | /var/vmail/example.com/jack/Maildir |
So the left column lists the valid email addresses. And the right column is apparently some path on disk. Right, thats all the magic. If an email for `jack@example.org` is received then Postfix will find the entry in the left column and can figure out where Jacks mails are stored on disk.
Before checking if a specific email address is valid Postfix first checks if it is responsible for the domain at all. Thats done by this mapping:
| Virtualdomain (key) | Whateve (value) |
| ------------------- | --------------- |
| example.org | Kittens |
| example.com | Puppies |
So there are two domains in the left column. But what is that in the right column? Kittens? Really? Well, the truth is: it doesnt matter. It can be anything. So why is that column there at all? The reason is that _mappings_ always have two columns. Postfix wants a list of domains but it has no concepts for _lists_. So it uses this format and ignores the right column.
Essentially these two mappings are all we need. Add a little configuration and you can already receive emails. However Postfix provides another useful feature: _aliases_. An alias is a redirection (or _forwarding_) of one email address to one or more other addresses. Possible uses:
- forward postmaster@… for all your domains to one mailbox
- create an email distribution list for a team or department
- forward copies of all your emails to another address
- redirect emails meant for a coworker who has left the organisation
As usual Postfix expects a mapping for aliases. Time for another example:
| Step | Virtual email address (left) | Redirect to (right) |
| :--- | :--------------------------- | :---------------------------------------- |
| 1 | `postmaster@example.org` | `jack@example.com` |
| 2 | `abuse@example.org` | `jack@example.com` |
| 3 | `jack@example.com` | `jack@example.com`,`jacky@workaround.org` |
| 4 | `sophie@example.net` | `hr@example.net` |
| 5 | `hr@example.net` | `rick@example.net` |
| | `hr@example.net` | `tina@example.net` |
| 6 | `@example.org` | `jack@example.com` (DANGEROUS!) |
Quite a lot happens here. How does Postfix interpret this table?
1. Redirect emails for `postmaster@example.org` to `jack@example.org`
2. Redirect emails for `abuse@example.org` also to `jack@example.org`
3. Keep a copy of an incoming email for `jack@example.com` in his mailbox and send another copy to `jacky@workaround.org.` Yes, you can use multiple email addresses seperated by commas.
4. Sophie has left the organisation so her address is forwarded to HR.
5. The HR team consists of two members: `rick@example.net` and `tina@example.net.` Every email to `hr@example.net` will be forwarded to each of them. This works similar to (3) but this time the target addresses are not seperated by commas but stored in different rows.
6. Forward any email address of the _example.org_ domain to `jack@example.com.` We call this a _catch-all_ address. This is dangerous for two reasons. First it will send the poor Jack anything that is sent to the _example.org_ domain if there is no user account for that address. (Yes, spammers guess addresses.) And second it disables a security check in Postfix that the destination address `jack@example.com` actually exists. Postfix may first receive the email and then be confused if the recipient is not reachable. That leads to backscatter and harms your reputation. But dont worry. I will show you the proper way to use catch-all addresses later.
So basically this is the way that Postfix handles aliases:
1. Is the domain of the email address defined as a virtual domain?
2. Get all rows of the table where the email address is found in the left column. Send a copy of the email to everyone mentioned in the right column. If the right column contains multiple email addresses separated by commas then split them first.
3. If there was no such row then check again if there is an `_@domain_` row.
4. Still nothing found? Then reject the email.
<Aside type="tip">
Two email addresses are mandatory for every domain that you host. `postmaster@domain` and `abuse@domain.` These requirements are documented in RFC 521 and RFC 2142. Be sure to add aliases for them. If an email to those addresses will bounce your domain reputation will suffer.
</Aside>
## Database
Did you get the idea of mappings? Two columns? Keys and values? Good. Now how do those mappings work when we want to put the information into an SQL database?
Basically a relational database works in rows and columns, too. So we can take the format of the tables shown above and put them into database tables. Lets call these tables…
- virtual_domains (for the list of domains)
- virtual_aliases (for the aliases mapping)
- virtual_users (for the email users and their mailboxes respectively)
I wont bore you with the SQL stuff right now. Lets do that in a later chapter. Lets just briefly cover how Postfix can get data from the database. We provide Postfix with configuration files for that purpose. These files often have a “.cf” suffix (**c**onfiguration **f**ile).
Lets take the virtual_domains table for example. This is the contents of a file that is located at `/etc/postfix/mysql-virtual-mailbox-maps.cf`:
```
# Information on how to connect to your MySQL server
user = someone
password = some_password
hosts = 127.0.0.1
dbname = mailserver
# The SQL query string
query = SELECT mailbox_path FROM virtual_users WHERE email_address='%s'
```
The first four lines describe how the database can be connected to. You provide a database user and database password as well as the IP address or hostname of the SQL server and the name of the database.
A more interesting bit is the _query_ that is defined in the last line. Postfix still thinks in left and right columns. So it will run this query and replace the %s by the email address it is looking for thats what is expected in the left column. The database will then get all rows from the database table that match this criterion. If one row matches the query then the “SELECT mailbox_path” will return just the value of what Postfix would expect to be in the right column.
Thats all the magic that Postfix needs to talk to your SQL database. You tell Postfix how to connect to the database and how the data in the database table corresponds to the left and right columns.
Note that a lookup here must only return just one row from the database. Postfix must uniquely know where the mailbox path for a given user is. There are other mappings though where its allowed to have multiple right-hand side items for one left-hand side item for example in virtual aliases.
To use the above configuration file you have to configure it in Postfixs main.cf file:
```
virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual_mailbox_maps.cf
```
If you find that this mapping is not working as you expected then the “postmap -q” command is your friend. You can ask Postfix what the right-hand side value for a given left-side value is. Say that you are interested in the mailbox_path for the email_address “`john@example.org`”:
```
postmap -q `john@example.org` mysql:/etc/postfix/mysql-virtual_mailbox_maps.cf
```
Postfix will then run the above SQL query with your “`john@example.org`” argument:
```
SELECT mailbox_path FROM virtual_users WHERE email_address='`john@example.org`'
```
The result should be:
```
/var/vmail/example.org/john/Maildir
```
In this guide we will use a slight variation of that SQL query:
```
query = SELECT 1 FROM virtual_users WHERE email='%s'
```
Postfix does not need to know where the mailbox is located on disk. We will not use Postfix to store the email to disk. Instead we will make Postfix hand over the email to Dovecot which stores the email. Dovecot has a couple of nice extra features (like server-side scripts) that we want to use. So Postfix just needs to know whether an email address is valid or not. We just return “1” because actually the right column is not considered anyway. Postfix just needs to know whether there is a database row or not.

View file

@ -1,24 +0,0 @@
---
title: What's new
lastUpdated: 2023-09-24
---
I try to keep as many parts of the setup unchanged. Many of us are happy with their mail servers and want to change as little as possible. Email is a boring matter anyway and we do our jobs best if our users dont even realize that we are there.
A few things though that are new in this version of the ISPmail guide:
- **Newer software versions**  it is a new Debian release after all. There are no breaking changes. Just business as usual. If you want to check the changelogs here are the version changes:
- Postfix 3.5.18 -&gt; 3.7.6
- Dovecot 2.3.13 -&gt; 2.3.19
- PHP 7.4 -&gt; 8.2
- rspamd 2.7.1 -&gt; 3.4.1
- Apache 2.4.56 -&gt; 2.4.57
- Roundcube 1.4.13 -&gt; 1.6.1
- MariaDB 1.21 -&gt; 1.22
- Adminer 4.7.9 -&gt; 4.8.1
- An improved **Ansible** playbook to help you automate all the steps of this guide if you want to set up multiple servers without repeating all steps manually.
- Clearer explanations. I have reworded complex parts of the configuration.
- Fixed links. The Dovecot wiki has moved around quote a lot of pages and I didnt notice earlier because it did not lead to a 404 or a redirect.
- Dovecot now uses the “count” backend to compute a users quota. Previous guides still used “maildir”.
- Two chapters are currently not published: the part about Thunderbird auto-configuration because there seem to be problems with the newest versions of Thunderbird. And the part about firewalling and fail2ban because there seems to be a bug with firewalld and fail2ban. These pages deal with optional features and are not a showstopper.