Update: it has been brought to my attention that I may have been incorrect in identifying a potential Google policy around SPF mechanics as the root cause of the issue. If you were in a similar situation as me and this approach resolves your problem - great! But it's likely that this is a workaround rather than fundamentally addressing the issue.
If you're a selfhosting enjoyer like me, you may have been using Luke Smith's wonderful emailwiz to host your very own email server. It's a simple script that allows you to not think much: just run it on your Debian box, and it will install everything, and then tell you all the DNS records you need to set for your email server to work correctly. This includes records for the DMARC, DKIM, and SPF protocols, which increase security and help prevent spam. So just make sure to follow these instructions, and you can send mail anywhere.
Or can you?
It's a common complaint you hear from people selfhosting their email server: the server works fine, you can send and receive email for the most part, but there are several email hosts that just will never accept mail from you, replying with those dreaded "Undelivered Mail Returned to Sender" messages. The biggest culprit being gmail, which also happens to have the biggest market share of personal emails. Meaning you can't send anything to your friends, or even less established businesses and organizations that simply keep using their personal gmail accounts for communications.
The error message gmail will return contains something like this:
The IP address sending this 550-5.7.25 message does not have a PTR record setup,
or the corresponding 550-5.7.25 forward DNS entry does not point to the sending
IP. As a policy, 550-5.7.25 Gmail does not accept messages from IPs with missing
PTR records. 550-5.7.25 For more information, go to 550 5.7.25
My, and probably many others', conspiracy theory has always been that gmail is just malicious towards new email hosts, especially small selfhosting landchads. Turns out that it's something different: gmail just happens to be more strict about SPF records! Something that Luke missed.
SPF (the Sender Policy Framework) specifies several mechanisms. You can read
more about them here.
Luke's script only specifies a and mx mechanisms (plus
the general -all to reject anything not matching those). But you
can also explicitly specify the IP addresses allowed to send mail from your
domain with ip4 and ip6 mechanisms. And it looks like
gmail requires these to be specified to accept your email. I guess you could
look for some maliciousness in the fact that they're not clearer about what one
has to fix to comply with their policies. But in the end, the solution is...
If you're getting 550-5.7.25 errors from gmail, make sure you do
have your PTR record set up correctly, but also that your SPF record looks
something like this:
v=spf1 mx a:<your mail host> ip4:<your IPv4 address> ip6:<your IPv6 address> -all
I got to this solution after the owner of
storin.nl emailed me about
nocss.club. When I tried to reply to him, I
got an "Undelivered Mail Returned to Sender" reply, but with a different error
message than gmail's:
not allowed to send mail from 550 m-chrzan.xyz: Please see
http://www.open-spf.org/Why : Reason: mechanism (in reply to RCPT TO command)
Tom from tfaz.xyz is working on a PR to emailwiz to correct the SPF record.
And big thanks to Luke for emailwiz, without it I wouldn't even have my own mail server to begin with.