Sending out emails : SPF Framework

[lambertb]

Hi Dean

I've been working hard trying to get my emails to go through.  My servers didn't have an SPF record on them, but I added an SPF record.  That allowed me to get through a lot of ISPs but there is still one thing that's happening, I hope you can help.

See where it says in the first line "may be forged"?    On some email's this is not a problem (like this first example,), but there is a lot where it IS A problem (see below):

 EHLO localhost
250-upsa-intl.org Hello upsa-intl.org [66.29.144.103] (may be forged), pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-8BITMIME
250-SIZE
250-DSN
250-ETRN
250-AUTH GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN
250-DELIVERBY
250 HELP
LM: SMTP Pipelining Detected
> NOOP
250 2.0.0 OK
> MAIL FROM: <bounced@upsa-intl.org>
> RCPT TO: <someone@upsa-intl.org>
250 2.1.0 <bounced@upsa-intl.org>... Sender ok
250 2.1.5 <someone@upsa-intl.org>... Recipient ok
> DATA
354 Enter mail, end with "." on a line by itself
> SENT DATA
250 2.0.0 k3KGY3YE022177 Message accepted for delivery
> QUIT
221 2.0.0 upsa-intl.org closing connection

Even though the email goes through (for this 1 email), when I send out to 10,000 people I'm getting a lot that say: "

Relaying denied. IP name possibly forged [66.29.144.103]  It looks like this:


> MAIL FROM: <bounced@upsa-intl.org>
> RCPT TO: <someone@leros.net>
250 2.1.0 <bounced@upsa-intl.org>... Sender ok
550 5.7.1 <someone@leros.net>... Relaying denied. IP name possibly forged [66.29.144.103]
LM: Undeliverable. RCPT response: 550 5.7.1 <someone@leros.net>... Relaying denied. IP name possibly forged


I have updated my SPF record to include 66.29.144.103 as an "approved" sender of email.  I don't know why it's saying that "it's possibly forged" do you?  Is it because the "from" address is different then the bounced address?

[DW]

I don't have much experience with SPF but it's definitely something I'm interested in learning more about.  Anything that can help get email through is obviously very important to my business.

Is it because the "from" address is different then the bounced address?

At first glance this would be my first guess.  To test it, does the same thing happen when you set the List's "Send From Email" an address on the ListMail domain and/or from the bounce address (which should be on the ListMail domain)?

Regards

[lambertb]

Well, I changed the bounced email address to do_not_reply@upsa-intl.org  That now matches the "sender" header in the email

Someone told me that if you don't do this (match the "sender" and the "bounced/from" ) that it might get caught on the application side [like MS Outlook] and put into a "junk" folder.  So that's good,

however,  

I'm still getting the "may be forged" thing even when the "from" and "sender"  matched  :cry:

looks like this now:
> EHLO localhost
250-upsa-intl.org Hello upsa-intl.org [66.29.144.103] (may be forged), pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-8BITMIME
250-SIZE
250-DSN
250-ETRN
250-AUTH GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN
250-DELIVERBY
250 HELP
LM: SMTP Pipelining Detected
> NOOP
250 2.0.0 OK
> MAIL FROM: <do_not_reply@upsa-intl.org>
> RCPT TO: <someone@test.org>
250 2.1.0 <do_not_reply@upsa-intl.org>... Sender ok
250 2.1.5  <someone@test.org>... Recipient ok
> DATA
354 Enter mail, end with "." on a line by itself
> SENT DATA
250 2.0.0 k3KIHrU8011639 Message accepted for delivery
> QUIT
221 2.0.0 upsa-intl.org closing connection

[DW]

The error seems to be in response to our greeting "EHLO localhost".  This is hard-coded into ListMail but I wouldn't mind changing it if I knew another value worked better or was more standard-compliant.  I'll have to scan the SMTP RFCs again to find out more.

I wonder if the error is simply a warning and non-fatal?  Your message seems to have been delivered successfully this time.

Regards

[DW]

I've found a couple things on the web but nothing (so far) that explains the correct value for EHLO with SPF.

http://www.openspf.org/faq.html
http://en.wikipedia.org/wiki/Sender_Policy_Framework

[lambertb]

HERE'S WHAT MY WEBHOST TECH SUPPORT SAID.   I WILL POST THE RESOLUTION WHEN I GET TO IT... SOME OF THIS IS OVER MY HEAD
--------------------------------------------

 Usually, the "may be forged" tag comes from mismatched forward and reverse DNS. I checked this and at an initial glance, it appears to be fine.

I did notice that you were using localhost in the EHLO and sometimes that can cause interesting results due to IP addressing and DNS for "localhost".

From within your VPS, I was able to reproduce the problem:

CODE
bash-2.05b# telnet upsa-intl.org 25
Trying 127.0.0.1...
Connected to upsa-intl.org.
Escape character is '^]'.
220 upsa-intl.org ESMTP Sendmail 8.12.10/8.12.10; Thu, 20 Apr 2006 12:22:02 -0700
EHLO localhost
250-upsa-intl.org Hello upsa-intl.org [127.0.0.1], pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-8BITMIME
250-SIZE
250-DSN
250-ETRN
250-AUTH GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN
250-DELIVERBY
250 HELP
quit


221 2.0.0 upsa-intl.org closing connection
Connection closed by foreign host.
bash-2.05b# telnet upsa-intl.org 25
Trying 127.0.0.1...
Connected to upsa-intl.org.
Escape character is '^]'.
220 upsa-intl.org ESMTP Sendmail 8.12.10/8.12.10; Thu, 20 Apr 2006 12:22:24 -0700
EHLO upsa-intl.org
250-upsa-intl.org Hello upsa-intl.org [127.0.0.1], pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-8BITMIME
250-SIZE
250-DSN
250-ETRN
250-AUTH GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN
250-DELIVERBY
250 HELP
quit
221 2.0.0 upsa-intl.org closing connection
Connection closed by foreign host.
bash-2.05b# telnet powermail.upsa-intl.org 25
Trying 66.29.144.103...
Connected to powermail.upsa-intl.org.
Escape character is '^]'.
220 upsa-intl.org ESMTP Sendmail 8.12.10/8.12.10; Thu, 20 Apr 2006 12:22:53 -0700
EHLO localhost
250-upsa-intl.org Hello upsa-intl.org [66.29.144.103] (may be forged), pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-8BITMIME
250-SIZE
250-DSN
250-ETRN
250-AUTH GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN
250-DELIVERBY
250 HELP
quit
221 2.0.0 upsa-intl.org closing connection
Connection closed by foreign host.
bash-2.05b# telnet 66.29.144.103 25
Trying 66.29.144.103...
Connected to 66.29.144.103.
Escape character is '^]'.
220 upsa-intl.org ESMTP Sendmail 8.12.10/8.12.10; Thu, 20 Apr 2006 12:24:17 -0700
EHLO localhost
250-upsa-intl.org Hello upsa-intl.org [66.29.144.103] (may be forged), pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-8BITMIME
250-SIZE
250-DSN
250-ETRN
250-AUTH GSSAPI DIGEST-MD5 CRAM-MD5 LOGIN PLAIN
250-DELIVERBY
250 HELP
quit
221 2.0.0 upsa-intl.org closing connection
Connection closed by foreign host.
bash-2.05b#


Notice that the "may be forged" tag only shows up when you connect to something other than the default hostname. So, when you connect to powermail.upsa-intl.org or when you connect to just the IP address, the problem can be reproduced. This is because sendmail is configured to believe that it is upsa-intl.org and nothing else.

Simple solution: in the MX record for each domain pointing to the sendmail server on upsa-intl.org, be sure it points to upsa-intl.org.

Now, that is only for incoming mail, but you claim to be having trouble with outbound mail. I think the "may be forged" tag you see when connecting to your own mail server is a "red herring" to the problem you're trying to solve.

To solve the problem with outgoing mail, I need to know the following things:

1. What is the IP address of the system running the mail client. For instance, if it is webmail running on your VPS, it would be the default IP address of your VPS. If it is Outlook Express or equivalent running on your PC, it would be the public IP address your Internet access provider assigned to your network or PC.

2. What machine are you connecting to when sending the outgoing mail? If webmail, it would be your VPS. If a mail client on your PC, this would be the outgoing mail server you have configured in your mail client.

3. What is the "from address" used in the email you're sending?

Depending on the answers to those questions, I may need additional information.

I did find one other issue:

CODE
bash-2.05b# cat /etc/hosts
127.0.0.1  upsa-intl.org upsa-intl localhost localhost.localdomain
bash-2.05b# cat /etc/host.conf
order hosts,bind


This is the root of why sendmail within your VPS believes upsa-intl.org should be 127.0.0.1 and thus, when you connect to the sendmail server from within the VPS, it doesn't properly match forward and reverse DNS.

That being said, I can't think of a real world scenario in which that causes a problem. Very few people have reason to log into a VPS and connect to sendmail from within the VPS using anything other than the default hostname.

I'll still need the answers to the other questions to solve your outgoing mail issue.

----------------- MY RESPONSE BACK TO THEM WAS --------------------------

1. What is the IP address of the system running the mail client. For instance, if it is webmail running on your VPS, it would be the default IP address of your VPS. If it is Outlook Express or equivalent running on your PC, it would be the public IP address your Internet access provider assigned to your network or PC.

My "default domain" is upsa-intl.org  it's IP address is: 66.29.144.103  this is the same for all sub-domains including powermail.upsa-intl.org (66.29.144.103).  I'm not running a local system.

2. What machine are you connecting to when sending the outgoing mail? If webmail, it would be your VPS. If a mail client on your PC, this would be the outgoing mail server you have configured in your mail client.

 It's an email script via php.  It sends out autoresponses to people that subscribe and opt-in to a list.  It sends it at a predetermined time via a cron job (i.e, 30 days after subscribing etc).   I also use this LAMP type program to send outbound emails to lists for newsletters on a "push" basis whenever i need to.   I talked to their tech support and primary coding guru.  He said:

that the error seems to be in response to their  greeting "EHLO localhost". This is hard-coded into the script but they wouldn't mind changing it if they knew another value worked better or was more standard-compliant. They are scanning the SMTP RFCs again to find out more.  

Andy do you have any suggestions on what they should use for a value?  Do you think this is the problem?

3. What is the "from address" used in the email you're sending?

from is "do_not_reply@powermail.upsa-intl.org" currently.  But I have had it as several others before and it it didn't have an impact.

Do you see anything that it might be  -- I'm kind of clueless  :cry:

[lambertb]

This is teh RCPT response I get too.

As more and more people role out their SPF frameworks, I'm really taking a hit.  I wonder if anyone else is seeing this?   Over 1/2 my emails aren't going through.

LM: Undeliverable. RCPT response: 550 5.7.1 <someone@emailaddress.org>... Relaying denied. IP name possibly forged [66.29.144.103]
. Skipping.


I will let you know what my webhost comes back with.  I hope we can resolve it -- thanks for looking into new "ehlo local host" coding

Do you see anything that it might be from my webhost's comments -- I'm kind of clueless!

[lambertb]

here's what they're saying at my webhost.  

I would venture to guess that this is going to effect not just me, but anyone with SPF set up

But, this stuff's over my head!  I just hope you can fix it

They said ----------------------------------------------------

EHLO localhost isn't the problem. The problem comes before that.

You can see in my tests that I telnet'ed to port 25 at various hostnames and only got the "may be forged" tag when *connecting to* certain host names.

From your description, this script appears to connect to sendmail through the tcp/ip stack rather than calling it as a program like most scripts. Sendmail can be accessed as a mail sending system two ways: 1) through port 25 using tcp/ip or 2) by calling the sendmail program locally using its binary executable (/usr/sbin/sendmail, for instance).

Accessing sendmail through port 25 is best reserved for server to server communications. The binary executable is the best method when using a script within the same machine as sendmail.

That being said, the script is connecting to the wrong hostname, which is why it gets the "may be forged" tag. Change the hostname that the script connects to to upsa-intl.org rather than whatever it is configured to use right now.

[DW]

ListMail uses two methods of sending email, internal PHP mail() and a socket/port connection to SMTP.  Even wtih SMTP enabled PHP mail() is used for welcome, confirmation, and some notification messages (this prevents too many connections to the SMTP server in case of a large number of signups at the same time).

ListMail was designed to work with as many mailing systems as possible and sendmail isn't the only one, just the most talked about.  As of yet ListMail cannot call sendmail via the 'binary' method your host suggested.  You may have success with the SMTP hostname they recommend instead of "localhost" as I believe you had it before.

Regards

[lambertb]

Dean,

How do I do that?  

You may have success with the SMTP hostname they recommend instead of "localhost" as I believe you had it before.

How do I set the hostname to upsa-intl.org when LMPRo connects to send out?

brian

[DW]

On the ListMail Config page set the value for SMTP "Host".  This will not affect those messages sent with internal PHP mail() (ie. welcome messages),  however.  Do you have trouble with those, too?

Regards