This patch skips the resolution logic if the address is not local.
Today, the resolution logic handles that case transparently, and returns
the original email address, so this should be a no-op.
However, having an explicit early check makes the resolution logic more
robust, and will simplify future patches.
Note this also means that the `alias-resolve` hook is no longer run for
non-local aliases, which should also help simplify their implementation.
This patch simplifies the internal alias lookup logic, unifying it
across Resolve and Exists.
As part of this, the `alias-exists` hook is removed. It was redundant to
begin with, although it enabled a potential optimization, it isn't worth
the complexity. The timeout for execution of both was the same.
This change should be backwards-compatible because `alias-resolve` is
still used, and the semantics haven't changed.
If the `drop_characters` or `suffix_separators` options are set to "",
currently instead of the empty string, their default value is used instead.
This is a bug, and it also happens on other config options, but because
the others have to be set in order for chasquid to function, it's not a
problem in practice.
Thanks Björn Busse (bbusse@github) for finding and reporting this
problem, on irc and in https://github.com/albertito/chasquid/issues/25.
This patch fixes the problem by marking these fields explicitly
optional, which enables presence testing, as described in the protobuf
documentation:
https://github.com/protocolbuffers/protobuf/blob/master/docs/field_presence.md.
Document that only haproxy's PROXY protocol v1 is supported. This can
help users configure their instance and avoid trying to set up other
versions.
Thanks to Björn Busse (bbuse@github) for reporting this!
This patch adds tests for STS policy checks in combination with TLS
security levels.
This helps ensure we're detecting mis-matches of TLS status
(plain/insecure/secure) and STS policy enforcement.
The `which` command isn't guaranteed to be available, it is just
extremely common; `command -v` is the standard way to do find an
executable program. See https://lwn.net/Articles/874049/ for more
details.
This patch replaces the uses of `which` with `command -v`, which only
appears in a couple of tests.
When using STARTTLS, the SMTP courier needs to determine whether the
server certificates are valid or not.
Today, that's implemented via connecting once with full certificate
verification, and if that fails, reconnecting with verification
disabled.
This works okay in practice, but it is slower on insecure servers (due
to the reconnection), and some of them even complain because we connect
too frequently, causing delivery problems. The latter has only been
observed once, on the drv-berlin-brandenburg.de MX servers.
To improve on that situation, this patch makes the courier do the TLS
connection only once, and uses the verification results directly.
The behaviour of the server is otherwise unchanged. The only difference
is that when delivering mail to servers that have invalid certificates,
we now connect once instead of twice.
The tests are expanded to increase coverage for this particular case.
When resolving MX records, we need to distinguish between "no such
domain" and other kinds of errors. Before Go 1.13, this was not
possible, so we had a workaround that assumed any permanent error was a
"no such domain", which is not great, but functional.
Now that our minimum supported version is Go 1.15, we can remove the
workaround.
This patch replaces the workaround with proper logic using
DNSError.IsNotFound to identify NXDOMAIN results when resolving MX
records.
This requires to adjust a few tests, that used to work on environments
where resolving unknown domains (used for testing) returned a permanent
error, and now they no longer do so. Instead of relying on this
environmental property, we make the affected tests use our own DNS
server, which should make them more hermetic and reproducible.
tls.BuildNameToCertificate has been deprecated, and calling it is no
longer necessary since Go 1.14.
Now that our minimum supported Go version is 1.15, we can remove it.
In the Dovecot integration test, we can now simplify the configuration
as we assume Dovecot 2.3 is the minimum version supported for testing
(as that's the one from Debian stable at the moment).
Today, we use `golang.org/x/crypto/ssh/terminal` to read passwords. That
package is obsolete, replaced with `golang.org/x/term`.
We couldn't move them because term wasn't compatible with Go 1.11 which
was our oldest supported Go version.
Now that we moved to Go 1.15 as the oldest supported version, we can do
the update.
Travis hasn't worked in a while, is shutting down for most projects, and
we have already removed it from all public documentation.
Also all the functionality it provided is now provided by GitLab CI.
This patch removes the obsolete Travis configuration file.
The latest Debian stable images don't include the `setcap` binary by
default like they used to.
Our Docker build depends on it, so this patch makes the Dockerfile
install the libcap2-bin package (which contains the `setcap` binary).
When doing a `docker pull`, if a tag is not specified, it defaults to
`latest`. We currently don't push such a tag, so it can cause confusion
in some cases (e.g. https://github.com/albertito/chasquid/issues/21).
To help prevent this, make the docker automatic builds for `master` also
apply the `latest` tag.
As a part of this, update the obsolete `$CI_BUILD_REF_NAME` to
`$CI_COMMIT_REF_NAME`.
This patch adds some basic instructions to the documentation on how to
set up DKIM, using the tools supported by the example hook.
It's not meant to be a full DKIM how-to, but to help someone who already
knows enough, or who is complementing it with a more general purpose
DKIM guide.
This patch adds support in the default hook for using dkimpy for DKIM
signing.
Unfortunately, dkimpy binaries have the same name as driusan/dkim's, so
we need to use --help to disambiguate. It's not pretty but it should
work, and is quite self contained.
Also, for the integration tests, we still need driusan/dkim because
dkimpy lacks the features needed. Specifically, dkimpy's dkimverify
can't be made to use custom DNS, or override the TXT values in any way,
so we can't verify that the generated signature is reasonable.
Thanks to ne9z@github for suggesting this change and providing an
alternative patch in https://github.com/albertito/chasquid/pull/19.
Most integration tests depend on the $HOSTALIASES environment variable
being functional. That variable works on most systems, but not all. In
particular, systems with `systemd-resolved` can cause the variable to be
ignored.
This was reported by Alex Ellwein in
https://github.com/albertito/chasquid/issues/20.
This patch makes the affected tests to be skipped if $HOSTALIASES is not
working properly. It also removes unnecessary hosts files from tests
which don't need it, and documents this behaviour.
Thanks to Alex Ellwein and foxcpp@ for reporting and helping investigate
this issue!
The chasquid-rspamd utility (https://github.com/Thor77/chasquid-rspamd)
provides a better integration with rspamd, by taking envelope and
connection information from the environment variables, and communicating
with rspamd using its protocol.
So if it is available, use it instead of rspamc in the default hook.
Some LMTP servers (like dovecot) can't handle UTF8 addresses in the LMTP
commands. This can be problematic if we want to use them with UTF8
domains or usernames, which are well supported by chasquid.
To help workaround this issue, this patch adds a new -to_puny flag for
mda-lmtp, that makes it encode `from` and `recipient` in punycode.
That way, the server will get punycode-encoded (ASCII) strings in the
LTMP commands.
This can be particularly convenient when the recipients are ASCII
(because they're under the mail server control), but `from` may not be
(because it comes from the network).
By default, golang.org/x/net/trace currently only allows the tracing
pages to be seen from localhost.
This restriction can be confusing for people accessing the monitoring
server remotely, and adds no value in our environment.
The monitoring server already exports very sensitive information, and
must be enabled with care, and is not on by default. This is well
documented.
This patch removes the restriction, making all the monitoring pages
equally accessible.
Some deployments already have users that authenticate without a domain.
Today, we refuse to even consider those, and reject them at parsing time.
However, it is a use-case worth supporting, at least with some
restrictions that make the complexity manageable.
This patch changes the auth package to support authenticating users
without an "@domain" part.
Those requests will always be directly passed on to the fallback
authenticator, if available.
The dovecot fallback authenticator can already handle this case just fine.
The openmetrics proposed standard says we should use the
`application/openmetrics-text` content type when exporting the metrics.
Currently we use `text/plain` for backwards compatibility with
Prometheus, but the new content type is apparently supported since 2018,
so it should be safe to update to match the current proposed standard.
Today, we close the connection after 10 errors. While this is fine for
normal use, it is unnecessarily large.
Lowering it to 3 helps with defense-in-depth for cross-protocol attacks
(e.g. https://alpaca-attack.com/), while still being large enough for
useful troubleshooting and normal operation.
As part of this change, we also remove the AUTH-specific failures limit,
because they're covered by the connection limit.
When we receive unknown commands, we use the first 6 bytes for
troubleshooting (e.g. put them in traces and exported metrics).
While this is safe, since the different places know how to quote them
properly, it makes things more difficult to analyse, since it's not
uncommon to see be binary blobs.
This patch makes us use the ascii-quoted version instead, to make things
easier to analyze.
This patch extends the README to mention explicitly that reporting bugs
and sending patches on GitHub is welcome, and also adds a private email
where to report security issues.
The changes matches the common practice so far, but it's useful to have
it explicitly documented.
This patch adds tracing for the auth and domaininfo modules. In the
latter, we replace the long-running event with the short-term request
tracing, which is more practical and useful.
There are no logic changes, it only adds tracing instrumentation to help
troubleshooting.
The Linux tests under the Cirrus CI are currently brittle due to
environmental issues. They're also redundant, since Linux testing is
much better covered by the GitLab CI tests.
So this patch removes them, which removes the false positives and speeds
up the Cirrus CI runs.
This patch does a general pass updating Go modules to recent versions,
and regenerates the protobufs accordingly.
The main purpose is to make sure people building from source are using
relatively recent versions of our dependencies.
We also regenerate protobufs since the newer versions of the liberaries
have a much cleaner dependency tree, which speeds up fetches.
When we fail to check if a user exists, we currently return a permanent
error, which can be misleading and also make things more difficult to
troubleshoot.
This patch makes chasquid return a temporary error in that case.
Thanks to Thor77 (thor77@thor77.org) for suggesting this change.
Currently, chasquid attempts to auto-detect dovecot sockets when
starting up (if needed). If autodetection fails, chasquid emits an
error, continues serving, and never tries again.
This can be problematic if chasquid starts up before dovecot, and at the
time the dovecot sockets are not present (e.g. after a reboot). In that
case, chasquid will not use dovecot for authentication even after
dovecot has started.
This patch changes the autodetect logic, by doing autodetection at
startup and on each request, until we find a working pair of sockets.
Once we do, they're used consistently.
That way, if dovecot is not ready when chasquid starts, it's not a
problem and chasquid will start using dovecot once it becomes available.
Thanks to Thor77 (thor77@thor77.org) for reporting and helping
troubleshoot this issue.
There's a known issue in versions 0.07 to 1.5 where the post-data hook
invocation can fail if the dkimsign binary exists, due to a bug in the
post-data hook check.
This was fixed by commit b6248f3, but it is found on occasion since the
current Debian stable ships 0.07, and Ubuntu 20.04 LTS ships 1.2.
So this patch adds it to the known issues list.
It's common that people running old releases (for example, because of
their Linux distribution version) run into issues that have already been
fixed.
It can be convenient to have a list of the most common known issues and,
when available, their workarounds.
This patch creates the documentation page for them, currently empty. It
will be filled in subsequent patches.
This patch adds a new link to the RBL checking suggestion, since the
existing one doesn't work with IPv6, and it's important to get good
coverage.
While at it, it also fixes the path to mda-lmtp, which was wrong before.
In commit 5305d584 we fixed an issue with the way the Docker image
adds the "hostname" option to chasquid.conf.
Currently, the Docker entrypoint sets the "hostname" option in
chasquid.conf if it's missing.
That works fine, except when there is a configuration change and the
domain is removed. In that case, the hostname option will have a stale
value, forcing the user to re-create the container, which can be
cumbersome.
This patch fixes the issue by unconditionally setting the hostname
option to one of the available domains at the time of start up.
Thanks to Jaywann@github for finding and reporting this problem on
https://github.com/albertito/chasquid/issues/16, and suggesting an
alternative fix!
In Go 1.16, "go get" on non-module paths now require an explicit version
to point to. Without a specific version, the invocation fails.
See https://golang.org/doc/go1.16#go-command for more details on the
change.
The test Dockerfile uses "go get" to fetch driusan/dkim's binaries, used
for integration testing.
So this patch adjusts the Dockerfile to fetch the latest version.
When the chasquid docker container is restarted, entrypoint.sh will add
the hostname again, even if it is present.
This causes chasquid to fail to start due to the duplicated option
(`non-repeated field "hostname" is repeated`).
Thanks to Jaywann@github for finding and reporting this problem, on
https://github.com/albertito/chasquid/issues/16.
This patch fixes the issue by only adding the option if it isn't already
present.
The docopt-go library is quite convenient, but it has been abandoned for
a while :(
Since we only use it for chasquid-util, this patch removes it and
replaces it with a custom small parser, that is a reasonable fit for the
required use cases.
The patch also adds a couple of tests to increase coverage.
NOTE: docopt-go accepted some undocumented behaviour, in particular the
use of "-a b" instead of "-a=b". The new parser does not, so some
user scripts may require updating.
I think this should be rare enough not to be worth the complexity of
adjusting the parser to allow it.
This patch adds a minor test to dovecot-auth-cli to verify that the
check for invalid number of arguments is working as expected.
It's mostly for consistency, as the utility is only used for testing
purposes.
The image jobs should only run if there are valid credentials for
pushing the images to the respective registries, to avoid false
negatives in the test pipeline.
This can happen when the gitlab CI is run on projects that aren't set up
to push docker images, either because they're clones of the official
repo, or they are under a different gitlab instance (e.g. Debian's
salsa).
We do it by using a "rules:if" clause on specific variables:
- for Docker, $DOCKER_REGISTRY_USER which is set externally
- for GitLab, $CI_REGISTRY_IMAGE which has the address of the registry
tied to the project.
Note that for GitLab we can't use the credentials for conditional
execution directly, since they are "persisted variables" which are not
available in this context (see [1] for more details). The
$CI_REGISTRY_IMAGE should be good enough to determine whether image
registry is enabled for the repo.
[1]: https://docs.gitlab.com/ee/ci/variables/where_variables_can_be_used.html#persisted-variables
fexp is a testing utility, including it in the regular Go build confuses
some automation as it can think it's part of chasquid proper.
All other testing utilities are ignored via the "+build ignore"
annotation for this reason, so this patch adds it to fexp to fix this
issue.
The haproxy test config includes an obsolete "debug" entry, and is
missing some timeouts which, while harmless in this context, cause a
warning that can be confusing.
This patch fixes the debug entry by running haproxy -d as recommended,
and adds the essential timeouts to avoid the warning.
