Raymond P. Burkholder - Things I Do - EMail

Anti-Spam Measures

nospam@example.com (Raymond P. Burkholder) — Sun, 27 Aug 2023 01:08:04 +0000

Request: Or people simple do not want to share own email, as it is not related for delivery, but required by the form. Thus they fill semi random email like string.

Response: That is what mailinator is for, or the temporary addresses offered by several browser plugins, or if you know they're going to use it to spam you, abuse@ their domain. nobody@example.com works as well, and 1-900-976-1212 for a "required" phone number.

General Guideline for Setting up a Mail Server

nospam@example.com (Raymond P. Burkholder) — Mon, 10 Jul 2023 22:52:53 +0000

make sure the MTA's forward and reverse DNS match
set up an SPF record, probably "v=spf1 mx ~all"
manage DNS MX, DKIM, DMARC and SPF records (SPF, DKIM, and DMARC are a recommended order of implementation)
set up DKIM signing for each domain you host, make the DKIM domain match the From: domain
start slow and look at any bounces
maybe collect DMARC stats but for a small volume MTA, not very interesting
instead of struggling with Postfix, OpenDKIM, Dovecot and friends (and losing out on quite a few features), try looking at maddy
arrange for a backup MTA
manage reverse lookup records, including managing the uncertain chain of authority between the instance and the nearest SOA
manage certificates associated with TLS for SMTP and IMAP
manage DKIM certificate
manage one's upstream to address PBL issues
keep the MTA secure and free from DOS attack
Use a static IPv4 address for your email system [counter point: If DNS is fully functional and IP addresses don't change too quickly and TTL is configured properly on DNS records, ... then why is a static IP address strictly required? ]
Do not share this IPv4 address with user machines
Do not host your email system 'in the cloud'
Make sure that your IP address is not listed in the PBL
Provide an MX record
Provide meaningful and consistent reverse DNS
Your system should say HELO (or EHLO) with its hostname
Keep your software completely up-to-date
Ensure that only authorised users can send email through your system.
Limit outgoing email volumes
Accept reports of problems with your systems
Review the mail system logs on a regular basis
Be reliable (viz have at least 4 9s availability)
Don't be an open relay
Don't create backscatter
Maintain a good reputation
users can probably have acceptable success with all but the large email providers if they simply have their MTA hello as the name that the ISP assigns to the connection presuming that the ISP has forward and reverse DNS configured therefor.

Mail Verifier Tools

nospam@example.com (Raymond P. Burkholder) — Wed, 24 May 2023 00:24:17 +0000

email-security-scans.org
Port25’s Authentication and Spam Assassin Tool
verifier.port25.com - send an email to this address to check headers
multirbl.valli.org - "The complete IP check for sending Mailservers"
postmastery - Check the DNA of your email against important best practices using our free email deliverability test.
email-security-scans - Public Email & DNS Testbed - Considering just DKIM tests, this one is the only tester which explicitly and clearly recognizes ed25519 signatures. For the rest, EmailAudit reported a pass, but then said "DKIM key size is 0 bits. More senders are starting to use 2048 bits."
internet.nl - test website, email, connectivity
Wander Science DKIM Test
uriports - Test and optimize SPF, DMARC, DKIM, and MTA-STS
Google Admin ToolBox - Check MX - maybe be broken in some aspects, take with grain of salt
Easy DMARC - verify dns data
dmarcian - DMARC resources

Note: seems ';' (semi-colon) in records may help with some records.

send an e-mail to spf@tester.realsender.com
check online the validation results (it will take a minute to appear) realsender

Definition of Email List Washing

nospam@example.com (Raymond P. Burkholder) — Tue, 12 Apr 2022 03:59:18 +0000

From an email list, the definition of email list-washing:

Trying to identify the addresses in a spam list that will get you in trouble, e.g., spam traps and people with a history of complaining, so that you can mail to what's left. And also, by implication, dealing with complaints by saying "we took you off the list" without looking at the overall list quality.
The technical methods for removing stale bouncing addresses from a legitimate list are somewhat similar but the motivation is not.

Email Services

nospam@example.com (Raymond P. Burkholder) — Wed, 23 Feb 2022 17:14:50 +0000

Extracts from a mailop email list:

little known control panel, for creating such email/web servers: Aetolos - secure open source control panel for virtual hosting
Roundcube is the default webmail client these days, I am not sure if its a "replacement" for gmail, but it is very efficient and works great on mobile devices
Postfix+dovecot is the most common combination for running your own services, once combined they offer pretty much all the features most people need (SMTP/IMAP/POP3/etc)
ClamAV anti-virus is said to require at least 4GB of ram to load its default data set.
For a full featured modern package with reasonable spam resistance I can vouch for Mailu which requires Docker as a basis. Under the hood, this is postfix as MTA, dovecot for IMAP/POP3, rspamd as anti-spam solution (I think the SPF/DKIM/DMARC stuff is located there, too), Roundcube or RainLoop as webmail (both pretty usable, probably not as feature-rich as GMail), PostgreSQL for account persistence, and REDIS as memory cache mostly for the rspamd engine. Supports Letsencrypt out of the box, of course. It does take some effort to set up the basics right (Docker and configuration) but then runs very reliably and is a breeze to manage. Sadly, it does not integrate Mailman, so I had to do that manually, which is kind of a pain.
If you have a reasonable insight into unix things, you could do worse than going with OpenBSD and an OpenSMTPD setup along the lines of what Aaron Poffenberger describes in his tutorial here: opensmtpd for the real world mail server tutorial (originally a BSDCan session I believe).
I’ve been running similar setups for years (but with exim as the MTA mainly for inertia reasons), documented mainly in articles you can find via the first link in my signature. The most comprehensive is perhaps Effective Spam and Malware Countermeasures - Network Noise Reduction Using Free Tools , others will be tagged with keywords like spam, mail, spamd, smtp and so forth.
Gmail's webmail interface is one of the worst possible. It is very inefficient to operate, counter-intuitive, hides many important information from the user etc., not mentioning that it is simply ugly. There are many much nicer and more user-friendly webmails, like for example already mentioned Roundcube.
Here I found an overview of various webmail apps, with screenshots of some. You can pick something that suits you: Top 15 Self-hosted open-source free web-based email clients
I've begun with OpenSMTP, but I've switched to Postfix as the primary developer has stopped actively working on it. I now run Postfix, Dovecot, Solr (for the search), and Rspamd, and it's working perfectly!
By the time you go through all the hassle of managing spam filters and getting your IP warmed up, Fastmail at $50/yr/mailbox looks pretty attractive.
If you can find someone who resells Tucows' white label e-mail, they have a pretty good product for about $10/mailbox/yr for 5GB, $20 for 10GB, $30 for 15GB.
take a look at mailcow dockerized. It's all you need (mail server, sogo groupware, antispam/-virus, 2fa,...). Put it on a VPS or your NAS and be happy.

Spam Fighting Resources

nospam@example.com (Raymond P. Burkholder) — Sat, 16 Oct 2021 18:04:22 +0000

Some recent suggestions I've encountered:

SpamAssassin KAM Rules - Just a note that the much better way to use the rules is with the KAM Channel, see KAM Channel
Sane Security Signatures

Rspamd is not a dead project and it can do all the stuff OpenDKIM+OpenDMARC+OpenARC can do (yes, even ARC signing and DMARC reporting). I have even added support of OpenDKIM style signing tables [1].
So far, Rspamd supports ed25519 and dual-signing out of the box. It also supports Hashicorp Vault to store (and rotate!) DKIM secrets [2]. In terms of DKIM signing there is also support of fast signing (with all content scan features disabled), so it is highly unlikely that it's performance can be beaten by any other signing tool in this mode [3].

Use of Signing Table
DKIM signing using Vault
Optimizing Signing Only Mode

Rspamd (embedded in a Mailu installation) for low-maintenance operation. That packages includes several mechanisms to score messages and handle them according to score intervals. This does a relatively good job but isn't really able to detect persistent spammers using their own IP ranges.

Are you aware of its reputation module? In conjunction with selector system you can build reputation on wide range of parameters without touching Lua. Beside its IP reputation i use sender's eSLD, IPnet & ASN reputation with score limited to not be enough for rejection.
If spammer uses not related IP, ASN and/or senders (snowshoe), you still can use its bayes and/or fuzzy checks. But basically the reputation module with RBL scoring does good job and in last year i do not remember any false negative mail and small amount of false positives, especially from gmail, these false positives are mostly not due reputation, but due SPF/DKIM/DMARC ignorance of our eshop systems.
I have small python script, which gets reputation data from rspam's redis DB and fill dynamic blocklist for exim, to block persistent spammers at MTA level (RCPT stage).

Anyway, I document what I do in a blog post: Zimbra AntiSpam Best Practices
The post describes how we rely on several layers of protection (including those provided by Postfix, which I know you are not using), as well as our adjustments to Amavis and SpamAssassin. I hope there are some techniques in the blog post that will be helpful to you.

For Let's Encrypt certificates, I use these awesome scripts, they are written as bash shell scripts and they are infinitely better than the official certbot tool, they can be used without a web server, by using DNS API integration. They are highly recommended:

acme.sh
dehydrated

For the first one, it no longer defaults to LE, it uses zerossl, because its rights are now owned by them. You can re-enable LE as default though by:
acme.sh --set-default-ca --server letsencrypt

I use Bruce Guenter's mailfront for my incoming mail so I had to add SNI support. It wasn't very hard. [when using SNI to differentiate between certificates]

if one abuses any of 25|465|587 they are blocked on all of 25/465/587 with a lengthy filter time
if one abuses any of 110|143|993|995 they are blocked on all of 110/143/993/995 with a slightly shorter filter time

Test SMTP / Email Server SSL / StartTLS

nospam@example.com (Raymond P. Burkholder) — Fri, 24 Jan 2020 18:08:27 +0000

There is a tool called testssl.sh - a free command line tool which checks a server's service on any port for the support of TLS/SSL ciphers, protocols as well as recent cryptographic flaws and more. Very extensive.

$ ./testssl.sh --starttls smtp myrelay.example.org:25
$ ./testssl.sh www.example.org

Or run some simpler tests:

$ openssl s_client -cipher 'DHE' -starttls smtp -crlf -connect myrelay.example.org:25

An older blog entry: SWAKS - Swiss Army Knife SMTP, the all-purpose SMTP transaction tester

2020/01/25 Send an email to

auth-results@verifier.port25.com

and receive back an informative reply about message handling: SPF, DKIM, iprev, SpamAssassin, DNS.

2020/01/25 Visit mail-tester.com for another flavour of email tester for spamminess.

2022/06/11 Authentication Checker

2022/06/11 mx toolbox

2024/01/26 about my email

swaks - Swiss Army Knife SMTP - the all-purpose smtp transaction tester

nospam@example.com (Raymond P. Burkholder) — Tue, 01 Aug 2017 23:51:41 +0000

swaks primary design goal is to be a flexible, scriptable, transaction-oriented SMTP test tool. It handles SMTP features and extensions such as TLS, authentication, and pipelining; multiple version of the SMTP protocol including SMTP, ESMTP, and LMTP; and multiple transport methods including unix-domain sockets, internet-domain sockets, and pipes to spawned processes. Options can be specified in environment variables, configuration files, and the command line allowing maximum configurability and ease of use for operators and scripters.

Thunderbird vs KMail

nospam@example.com (Raymond P. Burkholder) — Sat, 15 Apr 2017 13:58:01 +0000

For a while I was running KMail which is the default mail client with Debian KDE. It works quite nicely.

Many messages, at least the ones I see in my inbox, have an html component. By default, KMail tries to render all messages as text. At the bottom of the message window, it is possibly to show mail message body parts: typically there is a text component, possibly with one or more html parts. The html part can be clicked on to show the html rendering. If that becomes too unwieldly, it is possible to change a setting to use html view when ever possible. Remote content can be retrieved on an on-demand basis, but I don't think is 'remembered' on a by-sender basis.

KMail does have an offline/online mode where the content of all messages are saved locally. But it seems as though it retrieves a significant amount of content on each resync/refresh, and takes up considerable bandwidth when doing so. I didn't do any research into that.

Instead, I installed Thunderbird to see what it was like. I had two crashes while getting it started (during the initial sync processes), but now things seem to have stabilized. Once the initial syncs are complete, I'll see how much it takes to resync.

By default, Thunderbird will render html content as html content. And by default, it will not download remote content. When remote content is desired, it has an impressive mechanism for selecting which content to download. Better than even Outlook. It is possible to see from where the content is sourced. I have been turning on remote loads from sites which use similar names to the source of the email. Generic CDN (Content Distribution Networks) I keep turned off. These settings seem to be maintained on a per-site basis.

The nifty bit is that when replying to Outlook generated HTML messages, it knows how to 'break-in' to and perform inline text on the reply parts. Something Outlook can't or refuses to do.

There have been various rumblings that Thunderbird may become deprecated. Be that as it may, it still seems superior to some other mail readers, and is my new current email reader.

Any suggestions for other email readers?

Hacker News recently had an item on a Thunderbird Rewrite, but I don't think much consensus was reached. I was hoping to find suitable links to related solutions, but didn't find much of that either.

2018/05/22 - Mailspring might be worthy of evaluation.

Spam Fighting

nospam@example.com (Raymond P. Burkholder) — Sun, 02 Apr 2017 20:44:36 +0000

http://mainsleaze.spambouncer.org/: " documents the experiences of several long-time anti-spam professionals and activists with bulk email sent by legitimate companies to us, to our spamtraps, or to email addresses on mail servers that we manage."

The site has links to additional spam fighting resources.

Spammers Don't Like Us Mailing List.

If people want to try something *different* to SpamAssassin, then someone suggested the following: Rspamd. From the mailing list:

It's written in C, uses events internally and you can write plugins and rules for it in Lua or you can use regexps for more basic stuff. It also supports Intel Hyperscan to compile all of the regexp rules, which is extremely fast and less IO intensive, it's Bayes implementation is considerably better than SpamAssassin's and it has a SpamAssassin compatibility plugin to allow you to load all of your SA rules into it if you want (I prefer to rewrite mine however).
I've found rspamd considerably more easier to do complex stuff that would require considerably more complicated Perl plugins to do in SA, but is simple to write using a bit of Lua code. It's not perfect, the docs could be better and there's currently no corpus testing or automatic score generation (this is coming from GSoC this year though hopefully).

Another tool: > A new project was released by the guys at SpamExperts - OrangeAssassin. "In essence this seems to be a Python rewrite of Spamassassin".

2021/09/20 - DMARC in rspamd - generating and sending DMARC aggregate reports

Off-Line IMAP (referenced from NotMuch)

nospam@example.com (Raymond P. Burkholder) — Sun, 13 Nov 2016 20:55:42 +0000

Microsoft Outlook will typically replicate the contents of an IMAP mailbox locally to facilitate off-line access to email. I have been looking for a similar function for opensource based packages. I found it.

OfflineIMAP is a GPLv2 software to dispose your mailbox(es) as a local Maildir(s).

For example, this allows reading the mails while offline without the need for your mail reader (MUA) to support disconnected operations.

OfflineIMAP will synchronize both sides via IMAP.

Code is maintained at github.

I found this when seeing Notmuch -- Just an email system at LWN: A year with Notmuch mail.

2018/02/12 - astroid - a lightweight and fast Mail User Agent that provides a graphical interface to searching, displaying and composing email, organized in threads and tags. Astroid uses the notmuch backend for blazingly fast searches through tons of email. Astroid searches, displays and composes emails - and rely on other programs for fetching, syncing and sending email.

Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 6

nospam@example.com (Raymond P. Burkholder) — Fri, 24 Jul 2009 20:53:00 +0000

I've spent the last articles writing about getting an open source email server up and running. So far so good. My email logs show that a tremendous amount of spam is being blocked. One begins to wonder if there any real email remaining any more.

During the building of this server, a number of web sites provided useful information for troubleshooting and for configuration. I'm listing them here for reference before I close them out.

http://www.linuxweblog.com/blogs/sandip/20080206/sendmail-accessdb-example: provided useful explanations and examples of the interactions between the access database, the blacklist_recipients feature, the value part of the map, and how to use the delay_checks feature for negative or positive exception handling.
ZEN Return Codes: The 127.0.0.x return codes. Basically, 127.0.0.2 is for direct UBE sources, spam services, and ROKSO spammers; 127.0.0.4-8 are for illegal 3rd party exploits, including proxies, worms and trojan exploits; and 127.0.0.10-11 are for non-MTA IP address ranges set by outbound mail policy
: a good description of the sendmail.mc file, it's options, features, and ordering. It goes into some detail about special considerations of the VIRTUSER_DOMAIN_FILE. It also goes into uses and configuration examples of the access file.
SPF Setup Wizard: I'm not sure if the Sender Policy Framework (SPF) is much in use, but this web size provides a wizard for it's DNS record creation.
Sendmail Readme for Configuration: The original source for configuring Sendmail.
Linux Home Server HowTo: Sendmail: another article on how to build a full-fledged email server. One key command for ensuring you havn't configured an open relay through a series of 19 tests: 'telnet relay-test.mail-abuse.org'. When run from the mail server, the server at relay-test.mail-abuse.org will connect back to your server on port 25 and run the series of tests.
sendmail.mc: this is the best organized and best documented sample sendmail.mc file I've ever seen.
xabean's sendmail.mc: example sendmail.mc with native macros and a milter, with hotlinks to relevant sections in the Sendmail Readme file.
Hugo van der Kooij's sendmail.mc: looks like he no longer runs sendmail, but here is his sendmail.mc with some native macros.

In some follow-up, I came across MailWatch, which is a web-based front-end to MailScanner written in PHP, MySQL and JpGraph and is available for free under the terms of the GNU Public License.

Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 5

nospam@example.com (Raymond P. Burkholder) — Sun, 19 Jul 2009 02:00:00 +0000

A couple of articles ago, I started with a DoveCot Installation. I managed to download, build, and get a rough installation. I also prepared a userid for the service. It was at that point in the Dovecot installation instructions where they started talking about certificates, and I side-tracked into Certificate Authorities and certificate installation.

In /etc/dovecot, I copied dovecot-example.conf to dovecot.conf. In dovecot.conf, I updated the following lines to get things started:

protocols = imap imaps
disable_plaintext_auth = no
ssl = no
mail_location = maildir:~/Maildir
#mail_location = maildir:/%h/Maildir
auth_debug_passwords = yes

Dovecot Wiki does a good job of explaining the installation process. In fact, the non-ssl installation process is quite painless, and consists mostly of testing the connection.

Once the basic configuration is tested, then enable the configuration for ssl, and restart Dovecot.

disable_plaintext_auth = yes
ssl = yes
auth_debug_passwords = no
# Same keys from the sendmail installation
ssl_cert_file = /etc/ssl/private/mail.example.com.crt
ssl_key_file = /etc/ssl/private/mail.example.com.key

Startup an IMAP session with a Mail Client and try IMAP and IMAPS. Try sending email as well through the SMTP Sendmail connection with encryption. Tcpdump can be used to look at packets.

There is a Sample Dovecot init.d script which can be used to start, stop, and reload the service. The sample can be pasted verbatim into /etc/init.d/dovecot. Also do a 'chmod 755 /etc/init.d/dovecot'. Then '/etc/init.d/dovecot start'.

With a successful send and receive of email, that wraps up the rather lengthy configuration of a reasonably protected email solution encompassing Sendmail as an email transport mechanism, Dovecot as an IMAP/IMAPS service, and MailScanner with SpamAssassin/F-Prot for email scanning and protection.

Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 4

nospam@example.com (Raymond P. Burkholder) — Sun, 19 Jul 2009 01:26:00 +0000

It has taken a series of articles to get Sendmail installed and working with authentication, inline encryption, and some inline DNSBL capabilities. In this article, I'll see if I can get MailScanner, SpamAssassin and a virus scanner up and running with Sendmail.
Continue reading "Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 4"

Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 3

nospam@example.com (Raymond P. Burkholder) — Mon, 13 Jul 2009 08:00:00 +0000

In part two of this series, I started into the installation of the Dovecot IMAP service. The IMAP serivce can use validation and encryption through the use of SSL/TLS services. SSL/TLS services require the use of Certificates signed through a Certificate Authority. Many installation directions provide information for using the simple expedient of self-signed certificates. As some of these services I'm building are quasi-public, I wanted to go through the exercise of getting my certificates signed through a Certificate Authority. As such, I was side-tracked into doing some research to come up with two intermediate articles:
Continue reading "Debian Lenny with Sendmail, Dovecot, MailScanner, SpamAssassin: Part 3"