This patch is the result of running Go 1.19's `gofmt` on the codebase,
which automatically updates all Go doc comments to the new format.
https://tip.golang.org/doc/go1.19#go-doc
When running a diff for dkimpy's output, we expect that diff to exit with
non-zero code.
Unfortunately, the way we set that expectation (by prefixing the diff
invocation with `!` is incorrect.
Running `! diff ...` will not cause the hook to fail if diff exits with
0, instead `!` will cause the exit code to be ignored.
This patch fixes the problem by running `diff ... && exit 1` instead.
This was caught by shellcheck, https://www.shellcheck.net/wiki/SC2251.
We've accumulated a few linter issues around comments and a couple of
variable names.
While none of them is major, this patch cleans them up so it's easier to
go through the linter output, and we can start being more strict about
it.
This patch does a general pass updating Go modules to recent versions, and
regenerates the protobufs accordingly.
The main purpose is to make sure people building from source are using
relatively recent versions of our dependencies.
The current generate_cert helper was originally taken from Go's source,
and is more complex than we need it to be.
This patch replaces it with our own version, rewritten from scratch
independently.
This patch moves the test helper binaries to a "one directory per
helper" layout, and also makes them to be ignored in the coverage build
instead of all builds.
With this change, "go build ./..." will build all binaries including the
test helpers, which helps make sure that module manage automation also
considers them. In particular, this makes "go mod tidy" work fine.
Today, we do setfacl unconditionally; this can be a problem for
user-provided certificates because they may be located somewhere else.
This patch fixes the problem by only doing setfacl after renewing the
certificates.
Externally provided certificates will be untouched, and the user is
responsible for ensuring that chasquid can read them.
Thanks to Alex Ellwein (aellwein@github) for reporting this in
https://github.com/albertito/chasquid/issues/29!
Dovecot's `state_dir` usually defaults to be at `/var/lib/dovecot`, or a
similar system-wide path.
Under some conditions, our test Dovecot instance can fail, because it's
wanting to write to state_dir, but it is not writeable by us in the test
environment.
This was reported by foxcpp in
https://github.com/albertito/chasquid/issues/28.
This patch fixes the problem by setting a custom state_dir to be within
our testing directory.
Thanks to foxcpp for reporting this problem and suggesting a fix.
This patch updates the dependency on blitiri.com.ar/go/spf from v1.3.0
to v1.4.0.
There are no code changes needed, but the new version contains some
important fixes around lookup limits counting.
Some dkimpy versions have a bug where it can't parse the keys generated
by its own key generator. That causes the dkimpy test to fail.
See https://bugs.launchpad.net/dkimpy/+bug/1978835 for more details.
This patch adds a workaround which detects the buggy version, and skip
the test if needed.
This patch implements support for catch-all aliases, where users can add
a `*: destination` alias. Mails sent to unknown users (or other aliases)
will not be rejected, but sent to the indicated destination instead.
Please see https://github.com/albertito/chasquid/issues/23 and
https://github.com/albertito/chasquid/pull/24 for more discussion and
background.
Thanks to Alex Ellwein (aellwein@github) for the alternative patch and
help with testing; and to ThinkChaos (ThinkChaos@github) for help with
testing.
This patch does a general pass updating Go modules to recent versions, and
regenerates the protobufs accordingly.
The main purpose is to make sure people building from source are using
relatively recent versions of our dependencies.
This patch updates the dependency on blitiri.com.ar/go/spf from v1.2.0
to v1.3.0, which includes a few bug fixes.
There are no code changes needed, just some minor adjustment to the
tests due to error strings changing.
The go.mod "go" keyword is also bumped up to 1.15 since it's the minimum
supported version since commit e444fe1f (2021-10-05).
The install guide mentions that the `certs/` directory can be symlinked,
but doesn't cover permissions much, so it's easy for users to be
confused (like it happened in
https://github.com/albertito/chasquid/issues/26).
This patch adds a bit more details on how to set up certificates, to
hopefully reduce the chances of confusion.
This patch skips the resolution logic if the address is not local.
Today, the resolution logic handles that case transparently, and returns
the original email address, so this should be a no-op.
However, having an explicit early check makes the resolution logic more
robust, and will simplify future patches.
Note this also means that the `alias-resolve` hook is no longer run for
non-local aliases, which should also help simplify their implementation.
This patch simplifies the internal alias lookup logic, unifying it
across Resolve and Exists.
As part of this, the `alias-exists` hook is removed. It was redundant to
begin with, although it enabled a potential optimization, it isn't worth
the complexity. The timeout for execution of both was the same.
This change should be backwards-compatible because `alias-resolve` is
still used, and the semantics haven't changed.
If the `drop_characters` or `suffix_separators` options are set to "",
currently instead of the empty string, their default value is used instead.
This is a bug, and it also happens on other config options, but because
the others have to be set in order for chasquid to function, it's not a
problem in practice.
Thanks Björn Busse (bbusse@github) for finding and reporting this
problem, on irc and in https://github.com/albertito/chasquid/issues/25.
This patch fixes the problem by marking these fields explicitly
optional, which enables presence testing, as described in the protobuf
documentation:
https://github.com/protocolbuffers/protobuf/blob/master/docs/field_presence.md.
Document that only haproxy's PROXY protocol v1 is supported. This can
help users configure their instance and avoid trying to set up other
versions.
Thanks to Björn Busse (bbuse@github) for reporting this!
This patch adds tests for STS policy checks in combination with TLS
security levels.
This helps ensure we're detecting mis-matches of TLS status
(plain/insecure/secure) and STS policy enforcement.
The `which` command isn't guaranteed to be available, it is just
extremely common; `command -v` is the standard way to do find an
executable program. See https://lwn.net/Articles/874049/ for more
details.
This patch replaces the uses of `which` with `command -v`, which only
appears in a couple of tests.
When using STARTTLS, the SMTP courier needs to determine whether the
server certificates are valid or not.
Today, that's implemented via connecting once with full certificate
verification, and if that fails, reconnecting with verification
disabled.
This works okay in practice, but it is slower on insecure servers (due
to the reconnection), and some of them even complain because we connect
too frequently, causing delivery problems. The latter has only been
observed once, on the drv-berlin-brandenburg.de MX servers.
To improve on that situation, this patch makes the courier do the TLS
connection only once, and uses the verification results directly.
The behaviour of the server is otherwise unchanged. The only difference
is that when delivering mail to servers that have invalid certificates,
we now connect once instead of twice.
The tests are expanded to increase coverage for this particular case.
When resolving MX records, we need to distinguish between "no such
domain" and other kinds of errors. Before Go 1.13, this was not
possible, so we had a workaround that assumed any permanent error was a
"no such domain", which is not great, but functional.
Now that our minimum supported version is Go 1.15, we can remove the
workaround.
This patch replaces the workaround with proper logic using
DNSError.IsNotFound to identify NXDOMAIN results when resolving MX
records.
This requires to adjust a few tests, that used to work on environments
where resolving unknown domains (used for testing) returned a permanent
error, and now they no longer do so. Instead of relying on this
environmental property, we make the affected tests use our own DNS
server, which should make them more hermetic and reproducible.
tls.BuildNameToCertificate has been deprecated, and calling it is no
longer necessary since Go 1.14.
Now that our minimum supported Go version is 1.15, we can remove it.
In the Dovecot integration test, we can now simplify the configuration
as we assume Dovecot 2.3 is the minimum version supported for testing
(as that's the one from Debian stable at the moment).
Today, we use `golang.org/x/crypto/ssh/terminal` to read passwords. That
package is obsolete, replaced with `golang.org/x/term`.
We couldn't move them because term wasn't compatible with Go 1.11 which
was our oldest supported Go version.
Now that we moved to Go 1.15 as the oldest supported version, we can do
the update.
Travis hasn't worked in a while, is shutting down for most projects, and
we have already removed it from all public documentation.
Also all the functionality it provided is now provided by GitLab CI.
This patch removes the obsolete Travis configuration file.
The latest Debian stable images don't include the `setcap` binary by
default like they used to.
Our Docker build depends on it, so this patch makes the Dockerfile
install the libcap2-bin package (which contains the `setcap` binary).
When doing a `docker pull`, if a tag is not specified, it defaults to
`latest`. We currently don't push such a tag, so it can cause confusion
in some cases (e.g. https://github.com/albertito/chasquid/issues/21).
To help prevent this, make the docker automatic builds for `master` also
apply the `latest` tag.
As a part of this, update the obsolete `$CI_BUILD_REF_NAME` to
`$CI_COMMIT_REF_NAME`.
This patch adds some basic instructions to the documentation on how to
set up DKIM, using the tools supported by the example hook.
It's not meant to be a full DKIM how-to, but to help someone who already
knows enough, or who is complementing it with a more general purpose
DKIM guide.
This patch adds support in the default hook for using dkimpy for DKIM
signing.
Unfortunately, dkimpy binaries have the same name as driusan/dkim's, so
we need to use --help to disambiguate. It's not pretty but it should
work, and is quite self contained.
Also, for the integration tests, we still need driusan/dkim because
dkimpy lacks the features needed. Specifically, dkimpy's dkimverify
can't be made to use custom DNS, or override the TXT values in any way,
so we can't verify that the generated signature is reasonable.
Thanks to ne9z@github for suggesting this change and providing an
alternative patch in https://github.com/albertito/chasquid/pull/19.
Most integration tests depend on the $HOSTALIASES environment variable
being functional. That variable works on most systems, but not all. In
particular, systems with `systemd-resolved` can cause the variable to be
ignored.
This was reported by Alex Ellwein in
https://github.com/albertito/chasquid/issues/20.
This patch makes the affected tests to be skipped if $HOSTALIASES is not
working properly. It also removes unnecessary hosts files from tests
which don't need it, and documents this behaviour.
Thanks to Alex Ellwein and foxcpp@ for reporting and helping investigate
this issue!
The chasquid-rspamd utility (https://github.com/Thor77/chasquid-rspamd)
provides a better integration with rspamd, by taking envelope and
connection information from the environment variables, and communicating
with rspamd using its protocol.
So if it is available, use it instead of rspamc in the default hook.
Some LMTP servers (like dovecot) can't handle UTF8 addresses in the LMTP
commands. This can be problematic if we want to use them with UTF8
domains or usernames, which are well supported by chasquid.
To help workaround this issue, this patch adds a new -to_puny flag for
mda-lmtp, that makes it encode `from` and `recipient` in punycode.
That way, the server will get punycode-encoded (ASCII) strings in the
LTMP commands.
This can be particularly convenient when the recipients are ASCII
(because they're under the mail server control), but `from` may not be
(because it comes from the network).
By default, golang.org/x/net/trace currently only allows the tracing
pages to be seen from localhost.
This restriction can be confusing for people accessing the monitoring
server remotely, and adds no value in our environment.
The monitoring server already exports very sensitive information, and
must be enabled with care, and is not on by default. This is well
documented.
This patch removes the restriction, making all the monitoring pages
equally accessible.
Some deployments already have users that authenticate without a domain.
Today, we refuse to even consider those, and reject them at parsing time.
However, it is a use-case worth supporting, at least with some
restrictions that make the complexity manageable.
This patch changes the auth package to support authenticating users
without an "@domain" part.
Those requests will always be directly passed on to the fallback
authenticator, if available.
The dovecot fallback authenticator can already handle this case just fine.
The openmetrics proposed standard says we should use the
`application/openmetrics-text` content type when exporting the metrics.
Currently we use `text/plain` for backwards compatibility with
Prometheus, but the new content type is apparently supported since 2018,
so it should be safe to update to match the current proposed standard.
Today, we close the connection after 10 errors. While this is fine for
normal use, it is unnecessarily large.
Lowering it to 3 helps with defense-in-depth for cross-protocol attacks
(e.g. https://alpaca-attack.com/), while still being large enough for
useful troubleshooting and normal operation.
As part of this change, we also remove the AUTH-specific failures limit,
because they're covered by the connection limit.
When we receive unknown commands, we use the first 6 bytes for
troubleshooting (e.g. put them in traces and exported metrics).
While this is safe, since the different places know how to quote them
properly, it makes things more difficult to analyse, since it's not
uncommon to see be binary blobs.
This patch makes us use the ascii-quoted version instead, to make things
easier to analyze.
