Author Topic: DomainKeys, SPF, Deliverability (Read 7743 times)

BoomPop · « **on:** June 15, 2007, 01:23:58 am »

Hello Dean (and everyone).

I've been running a cPanel box for years, but I just received confirmation that they (cPanel) will not be supporting DomainKeys for at least 6 more months. And considering that Yahoo & AOL already trash over 90% of the emails coming from my server (and I don't send newsletters or mass mailings of any kind), I'm looking to make some drastic changes.

I'm looking hard at Plesk. I know you recommend it Dean. But do you have any experience getting DomainKeys to work with it? I've read around some of the forums and it looks like it can (possibly) be done, but I'm not sure I'm up to the task.

REALLY not looking forward to learning a new control panel, but will suffer if it means some of my mission-critical emails will actually get delivered!

Thanks.

DW · « **Reply #1 on:** June 15, 2007, 02:04:16 am »

Greetings,

I've tried DomainKeys and cannot recommend it, nor am I able (after many, many efforts) to confidently add it to Plesk qmail alongside other important patches... When I did get it patched in (minus certain other conflicting patches) the main problem I had with it was the bounce email and the "send from" email for each message had to be exactly the same for the DomainKeys signature to be added. This means you would not be able to customize the "send from email" for each list for display purposes.

Furthermore, I do not believe DomainKeys guarantees you will get more messages delivered to Yahoo. They are doing some strict rate-limiting these days... I'm guesstimating but I doubt you can get more than about 20,000 messages to Yahoo per day, per server, on average.

You might have far better success increasing delivery to Yahoo by contacting them. Check out the following thread for more details, particularly the last post which contains a link with more information about solving rate-limiting-rejected email.

http://listmailpro.com/forum/index.php?topic=1579.0

You can phone AOL and have them run a check on your server IP and/or domain name for blocks. It may be possible to re-assure them of your professionalism and have your complaint thresholds increased. They are very friendly.

http://postmaster.aol.com/contact

You can also set up a script to automatically remove complainants, so they are less likely to complain more than once.

http://listmailpro.com/forum/index.php?topic=721.msg3592

First you would need to set up a "Feedback Loop" to a certain email address, then you would set up an alias on the same address, pointing to the script.

Regards

BoomPop · « **Reply #2 on:** June 15, 2007, 10:33:30 am »

Quote from: "DW"

This means you would not be able to customize the "send from email" for each list for display purposes.

Wow! Am I glad I came and asked you before I decided to do anything drastic. Never would have thought of this minor (but not so minor!) detail, It's very important to know.

Quote from: "DW"

I'm guesstimating but I doubt you can get more than about 20,000 messages to Yahoo per day, per server, on average.

Heck... I'd settle for 10 or 20! Much less 10 or 20 thousand. I hate Yahoo for what they're doing. Putting such high priority on a system that 95% (I guessed at that figure) of the webmaster population CAN'T implement. The only people who can afford to play exactly by their rules are the spammers...people who do nothing but figure out how to bypass/circumvent the spam safeguards. The rest of us small timers are just caught in the muck.

Thanks again, Dean. I appreciate your other tips as well. It's just that my main competitor has beautiful DomainKeys in the headers of every email he sends. And every email he sends is getting delivered (from the testing I've been able to do, he he he...) And my thinking has been, "if he can do it, then so can I!" But I guess maybe not...

So now I have to figure out what I'm gonna do. Perhaps have someone build a pure RH server for me with no control panel at all.

No that won't work either.

God I hate this.

Still love, love, love ListMail though.

BoomPop · « **Reply #3 on:** June 15, 2007, 11:26:17 am »

Ok... thinking some more DW. You said:

Quote

When I did get it patched in (minus certain other conflicting patches) the main problem I had with it was the bounce email and the "send from" email for each message had to be exactly the same for the DomainKeys signature to be added. This means you would not be able to customize the "send from email" for each list for display purposes.

Does this mean that you DID get it working happily, even though it crippled your ability to use ListMail to its fullest extent? If that is the case, I might consider getting a second server just specifically for this one mission-critical domain/email address. It would be a not-so-pretty work-around. And at this point I'll do just about anything. Fingers crossed. :wink:

BGSWebDesign · « **Reply #4 on:** June 16, 2007, 07:44:50 pm »

Hi BoomPop, DW,

Hmmm, yes this is a 'royal pain' - I'm going through it now myself. After setting up a dedicated mail server, I've spent the last week or so trying to get back (I was on previously) AOL's whitelist - it turns out the whitelisting only lasts for 6 months - at least that is what one of their techs told me on the phone.

AOL does seem very agreeable - and reachable, I'm not sure yet about Yahoo, but I planned on implementing their 'domain keys' just for the purpose you mention BoomPop - GETTING DELIVERED...

It's worth it, I bet you could hack the headers yourself in LMP - with DW's approval, but it sounds like you have to give up having a seperate 'send from' from each list being different than the 'bounce to' address. This wouldn't bother me in the bit, and I'd gladly give that up to get things delivered to Yahoo.

So, a few questions... perhaps we can collaborate on some ways to get Domain Keys working and provide some alternative to others using LMP? I'm really curious BoomPop, you say:

Quote

. And every email he sends is getting delivered (from the testing I've been able to do, he he he...) And

How in the world do you test your competitor getting their emails delivered - unless you're seeding their list with your own (dummy) email addresses and then watching them for deliverability?

Quote

So now I have to figure out what I'm gonna do. Perhaps have someone build a pure RH server for me with no control panel at all.

What's a 'RH server', and as you say, I don't think control panel has anything to do with it does it - I believe the DomainKeys are only header calls that can be implemented in the MTH (Mail Transfer Handler) on your server. For example my server runs Courier Dot Mail, there are specific calls that can be added to implement Domain Keys for this MTH, there must be similar ones for other mail server MTH's, just find out which MTH you are running and track down how to implement DomainKeys, or change the headers to build something similar. Yahoo itself has a page on it, here's a link to it:
http://domainkeys.sourceforge.net/

As it seems my mail handler isn't on there... well, I'll keep digging because as you mention, it's worth it... We need to be prepared for any of these type of 'header' validations that may take place - if you want to get your email delivered!

Let us know what you find, and what works for you - anyone else - please post your own solutions for implementing Yahoo DomainKeys with details - it will help others get it working...

DW · « **Reply #5 on:** June 18, 2007, 02:05:29 am »

BoomPop,

Quote

Does this mean that you DID get it working happily, even though it crippled your ability to use ListMail to its fullest extent?

Yes, and actually I could work on it again and -maybe- get it patched in with all my other favourite patches... it might be easier the second time around considering I've gained experience with the other patches since last time I tried DomainKeys.

Brett,

Quote

I bet you could hack the headers yourself in LMP

While creative, I think this would be too much work. Basically you'd need a full DomainKeys implementation in PHP... I'm not even sure this is possible considering the complexity (message length calculations, etc). There are easier ways, as you mention later on in your reply, where this is implemented at the MTA level.

Quote

What's a 'RH server'

It's the OS - RedHat linux - Fedora Core 1-5, RHEL (Red Hat Enterprise Linux), etc. Most of my servers run RedHat and I prefer it, while some others say it is not the most efficient OS.

Quote

I believe the DomainKeys are only header calls that can be implemented in the MTH (Mail Transfer Handler) on your server.

Yes, exactly - thanks for providing a link to more information on this at Yahoo!

In my experience, once it's set up in the MTA / MTH the DomainKeys signature/header should be automatically added, provided the Bounce and Send From emails match.

Regards

BGSWebDesign · « **Reply #6 on:** June 18, 2007, 06:45:03 am »

DW,

Thanks for looking at this, I wonder a few things about Domain Keys.

1.) After looking closer at the Sourceforge page I notice there is only support for a few MTH's/UA's: Sendmail, Qmail, Ironport, Mdaemon. So, I wonder which one do you use, which one is best to use? Is it best to go 'low level' if possible, and how low can you go - is Mdaemon the lowest level listed here?

I believe all must use Sendmail or Qmail, but I'm not sure, I thought my own server is using Courier Dot Mail as the MTH, are Sendmail/Qmail MTH's? If so, I don't see any way to install the Domain Keys in Courier Dot Mail, so do I still install for Sendmail/Qmail, and does Courier Dot Mail just pass the mail to Sendmail/Qmail? I believe it possibly does (I'm going to confirm this with my host).

2.) I find this in the documentation for Mdaemon:

Quote

*Exclusive mode will not work with mailing lists so use this with caution

, you can read this yourself on this page: http://www.altn.com/Support/KnowledgeBase/KnowledgeBaseResults/?Number=KBA-01744

so apparently you can't go into 'exclusive mode' at all using Domain Keys if you are using a mailing list, instead you must go into ' NEUTRAL or RELAXED (signature optional)' mode, and then what does this do for deliverability?

DW, do you have any way of testing a few of these different DK modes and possibly checking on how deliverability is affected? I figure with your clout as developer you might be able to contact Yahoo directly to get an answer on where to install DK for it to be most efficient - and how it affects deliverability if 'relaxed mode' is used?

mike2 · « **Reply #7 on:** June 18, 2007, 09:11:06 am »

Here's my 2 cents... Forget Domain Keys, you DO NOT need them.

Simply setup SPF records for your domain and be happy. It's VERY VERY VERY simple and it's what I use and get all ( well of course never all ) of my mail through to aol and yahoo...

BGSWebDesign · « **Reply #8 on:** June 18, 2007, 09:55:01 am »

Hi Mike,

Thanks for the feedback, I was just looking at setting up DomainKeys (DK) and DKIM.

You say:

Quote

Simply setup SPF records for your domain and be happy. It's VERY VERY VERY simple and it's what I use and get all ( well of course never all ) of my mail through to aol and yahoo...

Ok, it seems you have verified that SPF records are enough to allow Yahoo to accept email, is there some way to validate/check this from my own server?

One further thing, I did setup the SPF records, but I have a little different situation in that my server is referred to in to different ways:
1.) mydomain.com
2.) server.mydomain.com

I know this seems a little goofy, but I went round/round with the techs at my host and it is required for the CPanel to work correctly, I've seen it described at other large host services as well. Also, when the mail is delivered I notice it says the mail is coming from server.mydomain.com - so it does seem it is using that to deliver the email.

When I setup my SPF records I did it like this:

Code: [Select]

mydomain.com. IN TXT "v=spf1 a mx ~all"
server.mydomain.com. IN TXT "v=spf1 a ~all"

Does that look right to you? Can you post what your SPF records look like - just so I can make sure I've got it right? I also wondered one thing, maybe you know, when I look at the records it indicates there is a mail server at mydomain.com (because of the MX record), but it does NOT indicate there is a mail server at server.mydomain.com, I've wondered if I need to also add a MX record for server.mydomain.com also, and then indicate that it has a MX in the SPF record too, like this:

Code: [Select]

mydomain.com. IN TXT "v=spf1 a mx ~all"
server.mydomain.com. IN TXT "v=spf1 a mx ~all"

What do you think? Do you have any ideas/suggestions? If I use the 'SawMill' as DW suggested will I be able to see if my email is being delivered properly to Yahoo? How do you check it Mike - do you use software to verify delivery?

mike2 · « **Reply #9 on:** June 19, 2007, 06:40:03 am »

Well my spf record is very simple:

Code: [Select]

mydomain.com. IN TXT "v=spf1 ip4:68.79.xxx.0/24 -all"

It's simple because I own that 0/24 block of IP's. I used a tool on the web to help me but I can't remember where. And somewhere it said you should use "-all" instead of "~all" for some supposedly good reason.

Well for me verifying delivery to yahoo I simply use a yahoo account honestly. I am safelisted with yahoo so supposedly my emails goto the inbox, but I like to check for sure on occasion.

But for what you got in your SPF record, I'd say it should be "ok". You could add the mx for server.mydomain.com, it shouldn't hurt anything at all and might help depending on how your dns is setup I guess. Of course if you have a static IP then just use it. The only thing with doing that is if you change IP's you have to change the SPF record.

BGSWebDesign · « **Reply #10 on:** June 19, 2007, 08:11:00 am »

Hi Mike,

Thanks for the input...

Quote

I used a tool on the web to help me but I can't remember where. And somewhere it said you should use "-all" instead

Most likely you used the free tool at OpenSPF at this website:
http://www.openspf.org/Tools

I used it too, but I just noticed that they also have an SPF sender check. It worked great, in fact the one I used, sending an email to: check-auth@verifier.port25.com gave me a full report including DomainKeys and DKIM.

Of course my domain passed on the SPF and the SenderID check but came back 'neutral' for DK and DKIM because the message was not signed. I still think it might be an advantage to try to run DomainKeys signing, but then again, there is also an overhead using this method as every email that goes out must run a process to 'sign' the email...

How about using something like 'Habeas', have you ever thought of that, I know that Ralph Wilson over at WilsonWeb uses it, here's info on habeas:
http://www.habeas.com/

What do you think?

mike2 · « **Reply #11 on:** June 20, 2007, 07:38:00 am »

Quote from: "webshaman"

How about using something like 'Habeas', have you ever thought of that, I know that Ralph Wilson over at WilsonWeb uses it, here's info on habeas:
http://www.habeas.com/What do you think?

I have seen a few different variations of services similar to this and I believe they could possibly help with delivery ( I don't think to yahoo, aol or msn ) but probably to other places.

The other ones I saw were fairly costly for the amount of email I send, I couldn't find pricing on this one, but it is worth at least looking into.

BGSWebDesign · « **Reply #12 on:** June 20, 2007, 12:25:29 pm »

Hi Mike.

Quote

send, I couldn't find pricing on this one, but it is worth at least looking into.

I'm not sure the pricing either, I did get their free 'analysis' report which shows that I'm valid with SPF, reverse DNS, etc. I may consider the next step which is to request an audit, and then they probably give you a price, but you are correct - I believe many of these type of services are priced based on how many emails you send/month so it's probably not going to pay for me to even consider it at this point.

Actually, I think it might be some type of yearly fee, but I'm not sure.

I'll let you know if I hear anything else... I just got this from them, check out the links at the bottom of the page there is a free online seminar their doing on deliverability:
http://www.habeas.com/en-US/News/press-releases/Habeas-Wins-2007-American-Business-Award/

BoomPop · « **Reply #13 on:** June 24, 2007, 02:53:51 pm »

Sorry I missed this conversation guys. I didn't have reply notifications turned on.

I'm seriously about to blow my brains out on this whole spambox/undelivered/delayed email thing. I cannot BELIEVE how much time and money I've wasted(?) on it. Since I started this thread, I've set up a new server with a new host (who promised one thing and is now delivering another). So I'm going absolutely nowhere fast. :roll:

Quote from: "mike2"

Here's my 2 cents... Forget Domain Keys, you DO NOT need them.

Simply setup SPF records for your domain and be happy. It's VERY VERY VERY simple and it's what I use and get all ( well of course never all ) of my mail through to aol and yahoo...

mike2... you say that DomainKeys are not necessary. Well, I have SPF records. ReverseDNS. PTR. etc, etc. My IP is squeaky clean... never been on any spam lists anywhere. In fact, I've never sent a single mass mailing of any kind. Not once. Yet Yahoo and AIM never delivers my Listmail confirmation emails to the inbox. Well actually, when testing the other day, I *did* have 2 test messages make it to the yahoo inbox, but they were both delayed by nearly 72 hours (under Yahoo's fairly new delay procedure). These are confirmation emails I'm talking about, not newsletter mailings. As far as I can tell, this started in October or November, 2006.

So back to your quote above... if DomainKeys aren't the problem, what in the heck could be it?

My main competitor has it set up exactly right, and it works every time that I've tested it.

His setup is:
=======================
NS1.HISDOMAIN.COM (separate IP)
NS2.HISDOMAIN.COM (separate IP)
NS3.HIS-HOST.COM (separate IP)
NS4.HIS-HOST.COM (separate IP)
mail.hisdomain.com (separate IP)
www.hisdomain.com (same as NS1 IP)
Send & Bounce email is the same: support@hisdomain.com

Zero errors on DNSReport.com
=======================

This is the setup I'm trying to mimic. But I can't find a host that knows what the h3ll they're doing enough to make it happen. Actually, to be honest, I did find one that *could* set it up, but they won't because they're ultra-paranoid about security... so they only do things their way (meaning I cannot designate my own servername or dedicated mailserver IP).

And for anyone reading this... if you have a cPanel server, you can forget using DomainKeys... for at least 6 months...probably much longer. And it sounds like it will work on Plesk, but only with some significant caveats (as DW pointed out earlier). So as I mentioned earlier... I think the only way to do it is to setup a plain, stripped down Unix/Linux box and build the system from scratch. That way you can configure all the settings yourself and tell all the naysayers (and crappy, incompetent webhosts) what they can do with themselves.

Can you tell I'm furious and not thinking so clearly anymore? LOL...

BoomPop · « **Reply #14 on:** June 24, 2007, 03:52:38 pm »

Quote from: "webshaman"

I just got this from them, check out the links at the bottom of the page there is a free online seminar their doing on deliverability:
http://www.habeas.com/en-US/News/press-releases/Habeas-Wins-2007-American-Business-Award/

Thank you for all this info about Habeas, Brett. I wasn't familiar with them at all.