mirror of
https://blitiri.com.ar/repos/chasquid
synced 2026-10-28 15:02:28 +00:00
Alberto Bertogli
6b64c9ccba
If chasquid runs out of file descriptors, that can cause problems accepting connections, writing to the queue, and even reloading configs. Some of those errors are fatal, as they should not happen under normal conditions. Since a remote client can open a lot of connections, this can end up consuming all file descriptors, and resulting in a denial of service. To reduce the impact of that kind of event, this patch makes chasquid limit the number of concurrent SMTP connections to half the maximum number of file descriptors. Connections over that limit will get closed immediately. The server will log (rate-limited) when such event happens, and new metrics make these events visible through monitoring too. This approach complements the previous patch, which made the server retry Accept errors, to reduce the impact of connection overloads. This mainly affects deployments with low file descriptor limits; most modern deployments have very high limits by default, so they are unlikely to experience this problem in the first place. Users who have to be very defensive can deploy OS-level safeguards (such as firewall rules to limit number of connections, etc.) which are more effective and tunable. Thanks to John JEAN <john@wargan.com> for finding and reporting this problem. The detailed report can be found in https://johnjean.io/advisories/chasquid-unauthenticated-remote-dos/.
65 lines
1.9 KiB
Bash
Executable File
65 lines
1.9 KiB
Bash
Executable File
#!/bin/bash
|
|
|
|
set -e
|
|
. "$(dirname "$0")/../util/lib.sh"
|
|
|
|
init
|
|
|
|
generate_certs_for testserver
|
|
add_user user@testserver secretpassword
|
|
|
|
# Build in advance, so the build doesn't run under the low fd limit below.
|
|
go-build-cached "${TBASE}/../../"
|
|
|
|
# Run chasquid with a low file descriptor limit, which is easy to exhaust.
|
|
# Reload very often: reloading needs to open files, so if we ran out of file
|
|
# descriptors, chasquid would fail to reload and exit.
|
|
mkdir -p .logs
|
|
(
|
|
ulimit -n 64
|
|
chasquid -v=2 --logfile=.logs/chasquid.log --config_dir=config \
|
|
--testing__reload_every=50ms
|
|
) &
|
|
CHASQUID_PID=$!
|
|
wait_until_ready 1025
|
|
|
|
# The connection limit should be half the file descriptor limit.
|
|
if ! grep -q "Incoming connection limit: 32$" .logs/chasquid.log; then
|
|
fail "unexpected connection limit"
|
|
fi
|
|
|
|
# Open and hold as many connections as the limit allows.
|
|
conngen -addr=localhost:1025 -count=32 -wait > .logs/conngen.log 2>&1 &
|
|
CONNGEN_PID=$!
|
|
wait_until "grep -q 'done, created' .logs/conngen.log"
|
|
|
|
# Open and hold more connections than there are file descriptors left. The
|
|
# server should close them right away (so we read EOF, not time out).
|
|
for _ in $(seq 1 40); do
|
|
exec {fd}<>/dev/tcp/localhost/1025
|
|
if read -r -t 5 line <&"$fd"; then
|
|
fail "connection over the limit got: $line"
|
|
elif [ $? -gt 128 ]; then
|
|
fail "connection over the limit was not closed"
|
|
fi
|
|
done
|
|
|
|
if ! grep -q "Incoming connection limit (32) reached" .logs/chasquid.log; then
|
|
fail "connection limit not logged"
|
|
fi
|
|
|
|
# Give it time to reload a few times, and check it's still running.
|
|
sleep 0.5
|
|
if ! kill -0 $CHASQUID_PID; then
|
|
fail "chasquid is not running"
|
|
fi
|
|
|
|
# Once the connections are closed, it should accept new ones, and work
|
|
# normally. The server notices the connections were closed asynchronously, so
|
|
# we retry until it works.
|
|
kill $CONNGEN_PID
|
|
wait_until "smtpc user@testserver < content 2>/dev/null"
|
|
wait_for_file .mail/user@testserver
|
|
|
|
success
|