smtpsrv: Strict CRLF enforcement in DATA contents

The RFCs are very clear that in DATA contents:

> CR and LF MUST only occur together as CRLF; they MUST NOT appear
> independently in the body.

https://www.rfc-editor.org/rfc/rfc5322#section-2.3
https://www.rfc-editor.org/rfc/rfc5321#section-2.3.8

Allowing "independent" CR and LF can cause a number of problems.

In particular, there is a new "SMTP smuggling attack" published recently
that involves the server incorrectly parsing the end of DATA marker
`\r\n.\r\n`, which an attacker can exploit to impersonate a server when
email is transmitted server-to-server.

https://www.postfix.org/smtp-smuggling.html
https://sec-consult.com/blog/detail/smtp-smuggling-spoofing-e-mails-worldwide/

Currently, chasquid is vulnerable to this attack, because Go's standard
libraries net/textproto and net/mail do not enforce CRLF strictly.

This patch fixes the problem by introducing a new "dot reader" function
that strictly enforces CRLF when reading dot-terminated data, used in
the DATA input processing.

When an invalid newline terminator is found, the connection is aborted
immediately because we cannot safely recover from that state.

We still keep the internal representation as LF-terminated for
convenience and simplicity.

However, the MDA courier is changed to pass CRLF-terminated lines, since
that is an external program which could be strict when receiving email
messages.

See https://github.com/albertito/chasquid/issues/47 for more details and
discussion.

test/t-12-minor_dialogs/bad_data_dot.cmy

@@ -0,0 +1,29 @@
+
+c tcp_connect localhost:1025
+
+c <~ 220
+c -> EHLO localhost
+c <... 250 HELP
+c -> MAIL FROM: <>
+c <~ 250
+c -> RCPT TO: user@testserver
+c <~ 250
+c -> DATA
+c <~ 354
+c -> From: Mailer daemon <somewhere@horns.com>
+c -> Subject: I've come to haunt you
+c -> 
+c -> Muahahahaha
+c -> 
+
+# An MTA must not accept isolated line breaks, otherwise it may fall victim to
+# an SMTP smuggling attack. See readUntilDot for more details.
+# This test triggers that condition with an invalid dot-ending, so we verify
+# the server returns an error in this case.
+c ~> '.\n'
+
+c -> That was a bad line ending, this is a good one.
+c ~> 'xxx\r\n.\r\n'
+
+c <- 521 5.5.2 Error reading DATA: invalid line ending
+

test/t-12-minor_dialogs/bad_data_dot_2.cmy

@@ -0,0 +1,29 @@
+
+c tcp_connect localhost:1025
+
+c <~ 220
+c -> EHLO localhost
+c <... 250 HELP
+c -> MAIL FROM: <>
+c <~ 250
+c -> RCPT TO: user@testserver
+c <~ 250
+c -> DATA
+c <~ 354
+c -> From: Mailer daemon <somewhere@horns.com>
+c -> Subject: I've come to haunt you
+c -> 
+c -> Muahahahaha
+c -> 
+
+# An MTA must not accept isolated line breaks, otherwise it may fall victim to
+# an SMTP smuggling attack. See readUntilDot for more details.
+# This test triggers that condition with an invalid dot-ending, so we verify
+# the server returns an error in this case.
+c ~> 'xxx\n.\n'
+
+c -> That was a bad line ending, this is a good one.
+c ~> '\r\n.\r\n'
+
+c <- 521 5.5.2 Error reading DATA: invalid line ending
+

test/t-12-minor_dialogs/bad_data_dot_on_message_too_big.cmy

@@ -0,0 +1,37 @@
+
+c tcp_connect localhost:1025
+
+c <~ 220
+c -> EHLO localhost
+c <... 250 HELP
+c -> MAIL FROM: <>
+c <~ 250
+c -> RCPT TO: user@testserver
+c <~ 250
+c -> DATA
+c <~ 354
+c -> Subject: Message too big
+c -> 
+
+# Max message size is 1 MiB. Note this includes line endings but converted to
+# \n (as per textproto.DotReader), and excluding the final ".".
+# We already sent (in the header) 26.
+# Send lines of len 900 to stay under the limit.
+# (1024 * 1024 - 26) - (900 * 1165) = 50
+c ~> ('a' * 899 + '\r\n') * 1165
+
+# We have 50 characters left before the message is too big.
+c ~> 'b' * 55 + '\r\n'
+
+# At this point the message is too big. The remainder data should be
+# discarded.
+# We use a "bad ." to try to do an SMTP smuggling attack.
+c ~> '.\n'
+c -> HELP
+c -> HELP
+
+# And now the "good .".
+c -> .
+
+c <- 521 5.5.2 Error reading DATA: invalid line ending
+

test/t-12-minor_dialogs/config/chasquid.conf

@@ -13,3 +13,6 @@ mail_log_path: "../.logs/mail_log"
 
 suffix_separators: "+-"
 drop_characters: "._"
+
+# Small max data size so we can reach it more easily in the tests.
+max_data_size_mb: 1

test/t-12-minor_dialogs/data_dot_stuffing.cmy

@@ -0,0 +1,28 @@
+
+c tcp_connect localhost:1025
+
+c <~ 220
+c -> EHLO localhost
+c <... 250 HELP
+c -> MAIL FROM: <>
+c <~ 250
+c -> RCPT TO: user@testserver
+c <~ 250
+c -> DATA
+c <~ 354
+c -> .From: Mailer daemon <somewhere@horns.com>
+c -> Subject: I've come to haunt you
+c -> 
+c -> .Muahahahaha
+c -> 
+c -> ..x
+c -> 
+c -> ..
+c ->
+c -> .This is stuffy.
+c -> 
+c -> .
+c <~ 250
+c -> QUIT
+c <~ 221
+

test/t-12-minor_dialogs/data_dot_stuffing.cmy.verify

@@ -0,0 +1,11 @@
+From: Mailer daemon <somewhere@horns.com>
+Subject: I've come to haunt you
+
+Muahahahaha
+
+.x
+
+.
+
+This is stuffy.
+

test/t-12-minor_dialogs/message_too_big.cmy

@@ -0,0 +1,28 @@
+
+c tcp_connect localhost:1025
+
+c <~ 220
+c -> EHLO localhost
+c <... 250 HELP
+c -> MAIL FROM: <>
+c <~ 250
+c -> RCPT TO: user@testserver
+c <~ 250
+c -> DATA
+c <~ 354
+c -> Subject: Message too big
+c -> 
+
+# Max message size is 1 MiB. Note this includes line endings but converted to
+# \n (as per textproto.DotReader), and excluding the final ".".
+# We already sent (in the header) 26.
+# Send lines of len 900 to stay under the limit.
+# (1024 * 1024 - 26) - (900 * 1166) = -850
+c ~> ('a' * 899 + '\r\n') * 1166
+
+c -> .
+
+c <~ 552 5.3.4 Message too big
+c -> QUIT
+c <~ 221
+

test/util/chamuyero

@@ -204,12 +204,15 @@ class Interpreter (object):
                 sock.connect()
                 self.procs[proc] = sock
 
-            # ->   Send to a process stdin, with a \n at the end.
-            # .>   Send to a process stdin, no \n at the end.
+            # ->   Send to a process stdin, with a \r\n at the end.
+            # .>   Send to a process stdin, no \r\n at the end.
+            # ~>   Send to a process stdin, string is python-evaluated.
             elif op == "->":
-                self.procs[proc].write(params + "\n")
+                self.procs[proc].write(params + "\r\n")
             elif op == ".>":
                 self.procs[proc].write(params)
+            elif op == "~>":
+                self.procs[proc].write(eval(params))
 
             # <-   Read from the process, expect matching input.
             # <~   Read from the process, match input using regexp.