This patch adds a chasquid-util subcommand to remove a domain
information entry.
The main use case is to manually allow a security level downgrade, after
performing manual verification.
This patch makes chasquid reload domaininfo periodically, so it notices
any external changes made to it.
It is in line with what we do for aliases and authentication already,
and makes it possible for external removals an additions to the
domaininfo database to be picked up without a restart.
Some transient issues might take more than 12h to resolve, specially if
they happen overnight.
20h gives a bit more margin for retries, while still being short enough
so that users are notified early.
Rewrite some of the paragraph to make them (hopefully) clearer, reorder
the feature list in a 2-level structure for more legibility, reference
the IRC channel, and other minor wording changes.
SIGTERM can happen normally in our tests, and the current code has a
trap that makes it trigger a clean exit.
This causes some errors to be masked, as we end up calling "exit 0" when
they occur. The error message will still be displayed, but the caller
script will assume it worked.
This patch fixes the problem by adjusting the bash signal handlers, so
that we ignore SIGTERM (so bash does not get killed by the exit handler)
and exit with error on SIGINT (triggered by ctrl-c).
Note that under some conditions the SIGTERM trap is not necessary, but
this depends on the environment.
This patch adds man pages for chasquid's main commands and
configuration. They are generated using pod2man (commonly used for this,
and included with perl in most distributions).
The generated man pages are included to avoid introducing a dependency
for such simple task, similar to how we handle protocol buffer generated
files.
This patch adds a Docker configuration file, to build an image that can
run the integration and stress tests.
It might be used for configuring automated testing infrastructure in
subsequent patches.
Our chamuyero tests involve reading and writing utf8. This usually
works, but is dependent on the environment: on LC_ALL=POSIX environment,
for example, Python enforces ascii as the default encoding, and the
tests break.
So this patch makes chamuyero explicitly set utf8 encodings in stdout
and all the sockets.
This patch introduces stress tests, which put load on the daemon to help
sanity check its behaviour under stress.
They are separate from the existing integration tests, which focus on
correctness.
Two tests are included here: a load test, which sends emails repeatedly;
and a connection test which opens as many conections as possible.
This patch adds a missing docstrings for exported identifiers, and
adjust some of the existing ones to match the standard style.
In some cases, the identifiers were un-exported after noticing they had
no external users.
Besides improving documentation, it also reduces the linter noise
significantly.
This patch adds a brief how-to guide for setting up an email server with
Debian, dovecot and chasquid.
It is incomplete, but has enough information that it's ok to include it.
This patch adds HTTP fetching to the integration tests.
It checks that the URLs are properly exported and that the server
replies reasonably to them. The contents are saved as they might be
useful as a debugging aid.
They're added to t-09-loop as it already was doing other HTTP fetches,
but the changes are not particularly tied to it.
The content of the pages is not checked yet, that might come in
subsequent patches.
The test to check that chasquid fails on startup if there are no valid
TLS certificates is passing, but for the wrong reasons: it fails because
there is no logging directory, not because there are no certificates.
This patch fixes the problem by moving the logs directory creation
before the first test.
In the protoio tests, we were using unkeyed fields in some composite
literals. This can cause confusion and makes the code more brittle wrt.
future changes. go vet also complains about this.
This patch fixes the issue by adding the field names to the struct
initializations.
This patch adds a new integration test, which executes various small
dialogs, to cover corner cases that are not well covered (according to
our coverage report).
For example, "EHLO" without domain, or invalid DATA.
While we could do them via Go tests, this way is more realistic, and the
tests are easier to write.
This patch adds some tooling and scripts to generate test coverage
information.
Unfortunately, this involves some hacks as Go does not have support for
generating coverage-enabled binaries, or merging coverage reports; but
overall it's not very intrusive.
This patch extends various packages and integration tests, increasing
test coverage. They're small enough that it's not worth splitting them
up, as it would add a lot of noise to the history.
We don't expect the AddHeader function to be given an empty key;
however, if that were to happen, it currently crashes.
This patch fixes the bug, while also adding tests for that and other
similar cases.
Dovecot has options for changing the formatting of usernames; for
example, dropping the domain part, or replacing characters.
chasquid's implementation, however, fails to handle this well, as it
expects the reply to contain the username exactly as requested.
This patch fixes the problem by making chasquid ignoring the returned
username, which is unused anyway. The protocol is unambiguous enough.
Tests are also amended to always exercise this case.
This patch adds dovecot support to the chasquid daemon, using the
internal dovecot library added in previous patches.
Dovecot support is still considered EXPERIMENTAL and may be reverted, or
changed in backwards-incompatible ways.
The patch also adds the corresponding integration test, which brings up
a dovecot server with a custom configuration, and tests chasquid's
authentication against it. If dovecot is not installed, the test is
skipped.
This patch adds a new package which implements two basic primitives for
authenticating against dovecot ("user exists", and "check password").
It is still experimental/work in progress.
This patch implements an Authenticator type, which connections use to
do authentication and user existence checks.
It simplifies the abstractions (the server doesn't need to know about
userdb, or keep track of domain-userdb maps), and lays the foundation
for other types of authentication backends which will come in later
patches.
This patch updates the travis-ci.org configuration file to use generic Go
versions (instead of hard-coded), and Ubuntu Trusty in container mode, for
faster builds.
The mda-lmtp tests are missing a file in the repository, which causes the
test to fail. This went unnoticed because they are not included in the
automated test suite, which will be fixed in a follow-up patch.
This patch adds the mising test file, with a harder to miss name.
In Go 1.10 the TLS library will start to reject DNS SANs which are not
properly formed; and in particular, if they're not IDNA-encoded. See:
- https://github.com/golang/go/issues/15196
- 9e76ce7070
The generate_cert utility will write non-IDNA DNS SANs, which the TLS
library does not like, causing our idna tests to fail.
This patch fixes this incompatibility by making generate_cert IDNA-encode
the host names when adding them to the certificate.
There's a bug in wget where it logs to files if -q is used:
https://savannah.gnu.org/bugs/?51181
It is harmless to the test, but it clutters the directory and the test
output, so this patch works around the issue by forcing it to log to
/dev/null.
The test starts by removing the previous test certificates, which may
or may not exist.
If they don't, currently "rm" fails, causing the whole test to fail.
I am surprised I did not notice this before :(
This patch fixes the bug by using "rm -f" to remove the test certs.
For direct TLS connections, such as submission-over-TLS, we currently
don't get the TLS information so it appears in the headers as "plain
text", which is misleading.
This patch fixes the problem by getting the connection information
early. Note it explicitly triggers the handshake, which would otherwise
happen transparently on the first read/write, so we can use the hostname
(if any) in our hello message.
Either the recipient or from addresses can be "<>" to indicate the null
address. mda-lmtp does not handle that well, as it sends "<<>>" which is
invalid.
This patch fixes that by special-casing them, which is unfortunate but
reasonably common.
One of the SMTP tests was doing an external DNS lookup for a
non-existing host, which is reasonably harmless but makes the test less
hermetic.
This patch changes the non-existing host for an invalid address, which
has the same effect but avoids the network lookup.
This is a hack; ideally we would be able to override the resolver, but
Go does not implement that yet, and changing the code is not worth the
additional complexity.
We have many places in our tests where we create temporary directories,
which we later remove (most of the time). We have at least 3 helpers to
do this, and various places where it's done ad-hoc (and the cleanup is
not always present).
To try to reduce the clutter, and make the tests more uniform and
readable, this patch introduces two helpers in a new "testutil" package:
one for creating and one for removing temporary directories.
These new functions are safer, better tested, and make the tests more
consistent. All the tests are updated to use them.
The right-hand side addresses of an alias should be normalized, to
maintain the internal invariant that we always deal with normalized
addresses.
Otherwise, strange situations may arise, such as the same domain having
two different domaininfo structures depending on case.
The outgoing security level checks are not being performed, because of a
bug: the courier thinks the "to"'s domain is always empty.
This patch fixes the bug by simplifying the logic, as there's no need
for the conditional (there is always a domain in the "to" address if it
got to the SMTP courier).
The nc.py script is only used in a single test, and for waiting for a
TCP port to be opened for listening.
This patch replaces it entirely, by using chamuyero for the test, and
bash for waiting on a TCP port.
mda-lmtp is a very basic MDA that uses LMTP to do the mail delivery.
It takes command line arguments similar to maildrop or procmail, reads an
email via standard input, and sends it over the given LMTP server.
Supports connecting to LMTP servers over UNIX sockets and TCP.
Since chasquid does not support direct LMTP local delivery, this can be
used as a workaround instead.
Example of use:
$ mda-lmtp --addr localhost:1234 -f juan@casa -d jose < email
