The SMTP courier, which handles outgoing connections, uses the domain of
the envelope's from as the domain in the HELO/EHLO greeting.
This works fine in practice, but ideally the domain used in the greeting
should match the reverse DNS record. This used to be more relevant but
nowadays it is not really enforced; however, it sometimes comes up in
self checks, and might cause some confusion when troubleshooting.
So this patch makes it use the configured hostname instead, which is
under the users' control and more likely to be compliant. It also
simplifies the code.
The documentation of the hostname configuration option is also updated
to mention this behaviour.
Thanks to Jonas Seydel (thor77) for bringing this up.
The linter complains that we're not checking for errors, but on some
cases it's on code paths were it is reasonable to do so (e.g. we're
closing the connection and it's a best-effort write).
This patch adjusts the code to make those cases explicit.
The daemon attempts to change to the config directory on startup, for
security and convenience.
We currently don't check if this works, which is not a big deal since it
will just fail later on when it can't find the files. However, it makes
things more awkward to debug, so this patch adds an explicit check.
This patch implements two new hooks: alias-resolve and alias-exists.
They are called during the aliases resolution process, to allow for more
complex integration with other systems, such as storing the aliases in a
database.
See the included documentation for more details.
This commit brings back the experimental MTA-STS (Strict Transport
Security) implementation, removed in commit
7f5bedf4aa.
We will continue development in the "sts" branch, subject to rebase,
until it is ready to be integrated into "next" again.
This patch adds dovecot support to the chasquid daemon, using the
internal dovecot library added in previous patches.
Dovecot support is still considered EXPERIMENTAL and may be reverted, or
changed in backwards-incompatible ways.
The patch also adds the corresponding integration test, which brings up
a dovecot server with a custom configuration, and tests chasquid's
authentication against it. If dovecot is not installed, the test is
skipped.
Currently, chasquid exits if any mode (SMTP/submission/submission+tls)
has no addresses to listen on. This means that chasquid must be given
addresses for all three.
While that's generally the expected configuration, there are cases where
users may not want to have all three.
So this patch replaces that fatal error with a warning, and only makes
chasquid exit if there are no addresses to listen on at all.
This commit removes the experimental MTA-STS (Strict Transport Security)
implementation for now, as it's not up to date with the latest draft.
Development will continue on the "sts" branch, but this way it won't
block releases until it is ready.
Commits reverted:
- cb6500b993
- 0eeb964534
- e66288e4b4
- 216cf47ffa
- d66b06de51
- fe00750e39
- 933ab54cd8
This patch adds support for TLS-wrapped submission connections.
Instead of clients establishing a connection over plain text and then
using STARTTLS to switch over a TLS connection, this new mode allows the
clients to connect directly over TLS, like it's done in HTTPS.
This is not an official standard yet, but it's reasonably common in
practice, and provides some advantages over the traditional submission
port.
The default port is 465, commonly used for this; chasquid defaults to
systemd file descriptor passing as for the other protocols (for now).
This patch extends the SMTP courier to (optionally) do STS policy
checking when delivering mail.
As STS support is currently experimental, we gate this behind a flag and
is disabled by default.
We only care about directories within the certs/, but the code as-is
complains if there are files.
This patch makes the iteration skip non-directories entirely.
Thanks to Martin Ferrari for the bug report!
glog works fine and has great features, but it does not play along well
with systemd or standard log rotators (as it does the rotation itself).
So this patch replaces glog with a new logging module "log", which by
default logs to stderr, in a systemd-friendly manner.
Logging to files or syslog is still supported.
The default INFO logs are more oriented towards debugging and can be
a bit too verbose when looking for high-level information.
This patch introduces a new "maillog" package, used to log messages of
particular relevance to mail transmission at a higher level.
This patch implements a post-DATA hook, which is run after receiving the
data but before sending a reply.
It can be used to implement content filtering when receiving email, for
example for passing the email through an anti-spam or an anti-virus.
This patch moves chasquid's Server and Conn structures to their own
smtpsrv package, to make chasquid.go a bit more readable. It also helps
clarify the relation between Server and Conn.
There are no functional changes.
Note that git can still track the history across this commit (e.g. git
gui blame shows the right data).
This patch introduces a new "domaininfo" package, which implements a
database with information about domains. In particular, it tracks
incoming and outgoing security levels.
That information is used in incoming and outgoing SMTP to prevent
downgrades.
This patch is the result of running go vet, go fmt -s and the linter,
and fixing some of the things they noted/suggested.
There shouldn't be any significant logic changes, it's mostly
readability improvements.
Add some links to the monitoring HTML index, to reflect the new
additions.
Also reorder to make it more practical, and default to the expanded view
for the tracer links.
HELO and EHLO both take a mandatory parameter, which also should be used
in the Received header.
This patch tracks and enforces that parameter, and also updates the
Received header generation to use it.
https://tools.ietf.org/html/rfc5321#section-4.4
If a connection has accumulated 10 errors, it's very likely that
something has gone significantly wrong, or they're just probing/abusing
the service.
This patch makes chasquid break the connection after 10 errors.
The number is arbitrary, we may adjust it later.
We should ignore the domains' case, and treat them uniformly, specially when it
comes to local domains.
This patch extends the existing normalization (IDNA, keeping domains as
UTF8 internally) to include case conversion and NFC form for
consistency.
This patch implements local username normalization using PRECIS
(https://tools.ietf.org/html/rfc7564,
https://tools.ietf.org/html/rfc7613)
It makes chasquid accept local email and authentication regardless of
the case. It covers both userdb and aliases.
Note that non-local usernames remain untouched.
Currently, we do SPF checks for all connections.
However, authenticated users will be sending email from different
locations, applying SPF to them will result in false positives.
So this patch makes chasquid skip SPF checking if the connection is
authenticated.
This patch implements some measures against email loops, such as keeping
a limit on the lenght of an address, and rejecting email that has too
many Received headers.
It's not perfect (a server could be actively removing Received headers),
but it should cover the normal accidents and misconfigurations.
This patch introduces expvar counters to chasquid and the queue
packages.
For now there's only a handful of counters, but they will be expanded in
future patches.
The SMTP courier was not properly closing the connection, and chasquid's
closing of incoming connections was not ideal (it was closing the
underlying one, not necessarily the active one, like in the case of a jump
to TLS).
This patch fixes both by adding the missing calls to Close.
This patch tidies up the MAIL FROM and RCPT TO handling, in particular:
- Preserve the case on received email. It could be outgoing and we
should not change it.
- Accept (but ignore) RCPT TO options, instead of failing.
- Fix some error codes to make them follow the RFC.
This patch reviews various debug and informational messages, making more
uniform use of tracing, and extends the monitoring http server with
useful information like an index and a queue dump.
This patch adds initial support for SMTPUTF8, which for now consists of just
advertising it.
We support most of it, but sending emails over SMTP requires further work, as
the SMTP courier does not support this yet (it's not in Go's standard
library). That will come in subsequent patches, along with IDNA handling.
https://tools.ietf.org/html/rfc6531.html
This patch implements the first steps of support for IDNA (Internationalized
Domain Names for Applications).
Internally, we maintain the byte-agnostic representation, including
configuration.
We support receiving IDNA mail over SMTP, which we convert to unicode for
internal handling.
Local deliveries are still kept agnostic.
For sending over SMTP, we use IDNA for DNS resolution, but there are some
corner cases pending in the SMTP courier that are tied with SMTPUTF8 and will
be fixed in subsequent patches.
There are cases, like email bounces and forwarding, where a remote server may
use an address within our domain as "MAIL FROM".
The current test at MAIL FROM will block them, which can be quite an
inconvenience as those cases are not that rare.
It's a nice test but doesn't add much, as we don't really pass the validation
along, and we still do relay and user checks on RCPT TO.
So this patch removes that test.
Having the certificates inside the domain directory may cause some confusion,
as it's possible they're not for the same name (they should be for the MX we
serve as, not the domain itself).
So it's not a problem if we have domains with no certificates (we could be
their MX with another name), and we could have more than one certificate per
"domain" (if we act as MXs with different names).
So this patch moves the certificates out of the domains into a new certs/
directory, where we do a one-level deep lookup for the files.
While at it, change the names of the files to "fullchain.pem" and
"privkey.pem", which match the names generated by the letsencrypt client, to
make it easier to set up. There's no general convention for these names
anyway.
This patch performs some minor cleanups for things detected by "go vet":
- Remove one line of unreachable code.
- Don't leak contexts until their deadline expires, cancel them.
It's more convenient and in line with standard practice to fail RCPT TO if the
user does not exist.
This involves making the server and client aware of aliases, but it doesn't
end up being very convoluted, and simplifies other code.
Today, if the aliases file does not exist when chasquid starts up, the entire
domain will be skipped from aliases resolution.
That's a bug, as it means we don't perform character and suffix replacements
for known domains, and is also an inconvenience as it forces us to reload the
daemon when adding a file for a known domain.
This patch fixes this by adding them unconditionally, even if the file does
not exist.
When we permanently failed to deliver to one or more recipients, send delivery
status notifications back to the sender.
To do this, we need to extend a couple of internal structures, to keep track
of the original destinations (so we can include them in the message, for
reference), and the hostname we're identifying ourselves as (this is arguable
but we're going with it for now, may change later).
