bounce processing, List-Unsubscribe header, link redirects

[kyle]

Hi Dean and everyone,
      I have a feature request and a couple questions.  First of all, would it be possible to simply add a feature for the bounce processing CGI to also forward the emails that are sent to that address to another address?  For example, if someone actually replied to the return path address (in my case bouncedNo-Spam@newsletter.lessonplanspage.com) the email would also be delivered to a mailbox after being processed by the CGI (such as bouncemailNo-Spam@lessonplanspage.com)...  I'm afraid some abuse, etc. emails might actually go there...

      Second, have you implemented (or plan to implement) the List-Unsubscribe header I mentioned in this older post:
http://listmailpro.com/forum/index.php?topic=1092.0
I haven't upgraded to the latest version since there's still a test version out there...

      What's really giving me the most pain right now is the last item I mentioned in the forum post above - tracking newsletter clicks and then redirecting the user to another address (for ads in the newsletter).  Apparently that has caused all my email to stop going through to AOL subscribers!    I'm working with AOL right now to try to get the issue resolved.  What methods are you guys using to track ad clicks??  

       There were also a couple other items I'd mentioned in that post.  Have you / will you include the option of a the 'Precedence: bulk' header field?  I'm not too sure about that one though...  Argh, why does it have to be so difficult to get mail delivered properly?  I even had setup an AOL feedback loop and an SPF (sender policy framework).    Take care,
--Kyle

[kyle]

On the bounce processing, I have users set to be removed if they bounce 3 times (in 150 days).  Is there a way to make it remove all permanent failure "bounce" members after a single bounce, but handle other bounces by the method listed above?  Argh, the policies for being whitelisted by AOL are insane...    They even want us to include the date, URL, and IP address the user was subscribed with and the sending company's physical address and phone number - sheesh!  Not easy for one man operations like mine!  More info here:
http://postmaster.aol.com/tools/whitelist_guides.html
Take care,
--Kyle

[BGSWebDesign]

Hi Kyle,

A search of the Forums with "aol white" shows 3 good posts already talking about this topic.

As for sending ANOTHER email on the bounce - good luck, I'm not sure what you are trying to do, but tracking any of that through email addresses is pretty much 'unreliable', I've found that some emails bounce that shouldn't and there's not much to do about it, just hope it doesn't happen frequently enough to cause problems... maybe DW has more to say, but check out the other topics discussing these issues using a forum search.

Good luck...

[kyle]

A search of the Forums with "aol white" shows 3 good posts already talking about this topic.

Hey webshaman,
Thanks for the reply.  I didn't find anything in there pertaining to preventing AOL blocking due to URL redirects in newsletters though (counting clicks in ad serving software) or because your URLs are being blocked - here's the error I get:
http://postmaster.info.aol.com/errors/554hvub1.html
I'm already on AOL's whitelist, yet all my mail is being blocked including personal emails  

As for sending ANOTHER email on the bounce - good luck, I'm not sure what you are trying to do, but tracking any of that through email addresses is pretty much 'unreliable'...

All I mean is having the bounce also forwarded to a real address in addition to being processed with bounce.cgi.  I just want to be able to turn that on / off so that I can occasionally manually look at the bounce emails being generated (which might include abuse emails or something similar) to make sure nothing looks out of the ordinary.  

The reason I now requested an immediate removal of permanent failure bounces is that AOL requires it for being on the whitelist (although who knows if they actually enforce it).  I believe I was the one that had originally suggested the feature of removing a bounced user after X bounces in Y days simply because many bounces are errors or are due to full mailboxes, etc. and won't happen again.  Wouldn't permanent failure bounces be okay to remove after one bounce?  Thanks,
--Kyle

[BGSWebDesign]

Hi Kyle,

URL redirects in newsletters

Which software, or where are you doing the redirects from?  I've noticed that lately many autoresponders are full of blocking problems, for instance a popular one could also get blocked, what are you using to do the redirect, and WHERE is the link going to originally (before the redirect)?

All I mean is having the bounce also forwarded to a real address in addition to being processed with bounce.cgi.

I'm not sure you can do that with LMP (it would create a huge overhead), I'm  one version back from current and you don't have that option, you can either bounce directly to an email address (your bounce address), or process another way, I don't think you can do both, or add additional bounce addresses... if you want to look at the bounces - I suggest using something like MailWasher to view what's in your bounced mail account - MailWasher will not delete the mails - just shows you the subjects, etc... you can then preview, or delete the mails, or just leave them there for later bounce processing.

The reason I now requested an immediate removal of permanent failure bounces is that AOL requires it for being on the whitelist (although who knows if they actually enforce it).

Not sure about that, have you seen the script that lets you auto delete emails when they are bad from AOL?  It's posted here in the forums somewhere... DW wrote it...  in fact I believe it's part of being White Listed, you must have the script in place, do you have that setup?

[kyle]

Which software, or where are you doing the redirects from?

Well, first of all, emails that simply have my URL in them (http://www.LessonPlansPage.com) are even being blocked (presumably because some URLs within that domain would redirect to other URLs).  But to answer your question, I use PHPAdsNew to track clicks, so a typical URL would look like this:
http://www.lessonplanspage.com/phpads/adclick.php?bannerid=426&zoneid=79&source=&dest=http://www.drexel.com/lessons
...and redirect to the URL at the end of the link.  That would be in the HTML version.  For the text version I need to use shorter links, so I use a URL shrinker to make the same link above look like this:
http://www.LessonPlansPage.com/to/drexel06-06
That software is called CamoLink / CamoAlias - it's essentially a TinyURL type program for your own server:
http://www.wynia.org/wordpress/2005/10/25/camoalias-ditch-tinyurl-for-the-exact-urls-you-want/

All I mean is having the bounce also forwarded to a real address in addition to being processed with bounce.cgi.

I'm not sure you can do that with LMP (it would create a huge overhead), I'm  one version back from current and you don't have that option, you can either bounce directly to an email address (your bounce address), or process another way, I don't think you can do both, or add additional bounce addresses... if you want to look at the bounces - I suggest using something like MailWasher to view what's in your bounced mail account

I know it can't be done currently, but I *thought* it would be a very simple addition to LMP - after bounce.cgi processes the bounce, just forward it to another email address...  

I don't know anything about MailWasher, but it wouldn't be able to do anything with bounces that go to the bounce.cgi file anyway, would it?  

Not sure about that, have you seen the script that lets you auto delete emails when they are bad from AOL?

Yeah, I have the feedback loop setup, and I saw and plan to implement the script to auto delete those AOL emails (I've been doing it manually previously).  Unfortunately that involves changing the feedback loop email address...

[mike2]

Here's what I do for the feedback loop with aol.  

I have my regular email address setup with aol as the loop and I set another email address up with Deans script.

Anytime I get an email from aol, I simply do a quick "Forward" without changing the message to the email I setup with the script.  This way I can see them and know of any complaints.

If I had to guess, you are getting that 554 error from AOL because a large amount (over aol's limits) are complaining about your emails, basically exactly what the error suggests.  Even though they have you whitelisted, if so many people complain, you can still be blocked.

In General email is getting tougher and tougher, but I still believe it's one of the best promo methods around.  I'm whitelisted with AOL, Yahoo, (kind of with hotmail/MSN) and am joining MSN/HOTMAILs feedback loop program as well.

[kyle]

Anytime I get an email from aol, I simply do a quick "Forward" without changing the message to the email I setup with the script.  This way I can see them and know of any complaints.

Doesn't that defeat the purpose of Dean's script?  You have to manually forward the message to the bounce.cgi - it would be just as easy to click on the unsub link in your email...  Maybe I misunderstood you.  

That does give me an idea though for handling other bounces.  In cPanel I have:
bouncedNo-Spam@newsletter.lessonplanspage.com
forwarding to:
"|/home/newsltr/public_html/cgi-bin/bounce.cgi"
Couldn't I do something like this to make it forward to the CGI *and* to another email address?:
"|/home/newsltr/public_html/cgi-bin/bounce.cgi"; abuseNo-Spam@lessonplanspage.com

What would be the syntax for that?

If I had to guess, you are getting that 554 error from AOL because a large amount (over aol's limits) are complaining about your emails, basically exactly what the error suggests.  Even though they have you whitelisted, if so many people complain, you can still be blocked.

In General email is getting tougher and tougher

Yeah, I'm sure that's it (or because of the redirects).  Unfortunately my userbase is very low-tech-savvy (elementary school teachers), and rather than unsubscribe, they tend to just identify as spam (or at least that's my assumption)...  

Yeah, email delivery is getting to be more and more of a pain in the butt.    Thanks,
--Kyle

[BGSWebDesign]

Hi Kyle,

As you said:

Couldn't I do something like this to make it forward to the CGI *and* to another email address?:

Yes, that looks the easiest way to accomplish it - and not burden LMP with it since it is not necessary for most users.

I know it can't be done currently, but I *thought* it would be a very simple addition to LMP - after bounce.cgi processes the bounce, just forward it to another email address...

Sure, it's probably simple - but think about it, do you want to burden users with 1,000,000 subscribers handling bounces that way?  Probably not, it will overtax the server - I know I don't want a bounce forwarding on somewhere else every time it bounces....  UNLESS it can be turned Off in configuration.

If you really want this ask DW, maybe he'll do it for a charge, I don't see why you really need it, as I said Mail Washer is free....

I just want to be able to turn that on / off so that I can occasionally manually look at the bounce emails being generated (which might include abuse emails or something similar) to make sure nothing looks out of the ordinary.

This is what you asked, Mail Washer will do that with no problem, I just looked at 1700 messages in my Bounced Folder this morning... it works fine for what you want to do - get it at: http://www.mailwasher.net/

...and redirect to the URL at the end of the link. That would be in the HTML version. For the text version I need to use shorter links, so I use a URL shrinker to make the same link above look like this:
http://www.LessonPlansPage.com/to/drexel06-06
That software is called CamoLink / CamoAlias - it's essentially

Why do you want to go to ALL of that bother when LMP has Link Tracking built in?  Have you thought of using that?  Click on Link Tracking from the main menu....  if not, why not consider something that will let you shorten your links such as AdTrackz - http://adtrackz.com/ where you can cloak links, or create redirects with simple directories and make the URLS as short as you want....

Good luck and please do report back with what you find - yes Email delivery is getting NUTS - I've been doing it for almost 10 years and it was nothing like this back in 1997!  Now I'm lucky to get 1/10th of my email delivered/opened/read, then to get a click it's probably like 1/100!

[kyle]

Sure, it's probably simple - but think about it, do you want to burden users with 1,000,000 subscribers handling bounces that way?  Probably not, it will overtax the server

No, of course not - I only meant it as an option that could be disabled...  I wouldn't want it always on myself.  I just need to figure out how to forward to the bounce.cgi and an email address in cPanel and I should be set.

I'll also take a look at Mail Washer, but forwarding would be simpler for my purposes.  

Why do you want to go to ALL of that bother when LMP has Link Tracking built in?  Have you thought of using that?

Absolutely not    I need them tracked where my clients can login to see their stats.  I also already have PHPAdsNew setup with regular banner ad type campaigns, so integrating newsletter ad results with their banner ad campaigns makes much more sense.  It's not a hassle to set that up anyway.  It looks like AdTrackz does about the same thing I'm doing with Camo Alias and PAN (which are free).  Take care,
--Kyle

[DW]

Hi guys - Great thread!

I'll also take a look at Mail Washer, but forwarding would be simpler for my purposes.

I love MailWasher for keeping my Outlook Deleted Items free from spam and it does seem useful for this application.

If you would prefer forwarding, almost all hosts can set up aliases pointing to both a script and a mailbox.

Searching Google I came up with this for cPanel, which states that you can simply create a second forwarder for the same email address.

Plesk ".qmail" files might look like this:

Code: [Select]

|/usr/bin/perl /path/to/bounce.cgi
&you@example.com
Sendmail/Postfix /etc/aliases might look like this:

Code: [Select]

bounce@example.com: "|/path/to/bounce.cgi",you@example.com
If you are still in doubt, ask your host for help - I'm sure they can point you in the right direction.

Regards

[kyle]

Searching Google I came up with this for cPanel, which states that you can simply create a second forwarder for the same email address.

Thanks Dean, I think that will do the trick.  Oddly enough, my hosts didn't think it could be done    Take care,
--Kyle

BTW, any thoughts on the List-Unsubscribe header?

[DW]

Kyle,

Yes, in fact I was just thinking about it today as a client received a spam complaint where the headers were forwarded back as evidence.  A remove link right there might tempt the abuse department to simply click the link instead of having to send an email, perform an investigation, etc.  ISPs might get used to ListMail's link style and trust it to remove users without contacting the list owner unless a huge number of complaints come through...  Usually it's just one or two who take things to the top in a valiant effort to rid the world of the scourge of spam..

What does everyone else think of this? (RECAP: We're talking about adding a header "List-Unsubscribe: http://example.com/mail/rem.php?xxxx" header to LMP messages. Would you prefer it as an option as opposed to permanent?  How would we decide which lists to remove the user from.. ? I suppose the 'current' list would be fine...

Re: your AOL block - you need to call the AOL postmaster hotline to have your URL unblocked.  It should be a very quick process.  Be sure to have your server IP address handy as they will probably want to check for blocks on that, too.  In my experience they are very helpful and understanding.

Regards, DW

[BGSWebDesign]

Hi DW,

Would you prefer it as an option as opposed to permanent? How would we decide which lists to remove the user from.. ? I suppose the 'current' list would be fine...

Option - but I don't know enough about it, is it required?  I don't think it should be permanent, put it on the Config page?

Removal from Current List?  Not sure what the requirements are but I would prefer it work that way...

[kyle]

Option - but I don't know enough about it, is it required?  I don't think it should be permanent, put it on the Config page?

Removal from Current List?  Not sure what the requirements are but I would prefer it work that way...

Hey Guys,
I also think it would be best as an option, but I know that creates more work for you Dean...  Removal from the current list (although I only have one list currently) would probably be best...  A user might not want to unsubscribe from all newsletters someone provides.

Brett, the List-Unsubscribe header is something I brought up back in February.  It was something Google's GMail suggested to prevent email from being misidentified as spam:
"Provide a 'List-Unsubscribe' header which points to a web form where the user can unsubscribe easily from future mailings."
Here's my original post on the subject:
http://listmailpro.com/forum/index.php?topic=1092.0
I don't remember where on Google's site I found that though.  

I certainly don't mean to back off from my suggestion, but I do think the idea should be looked into a little more before being implemented.  It sounds like a great idea, but is it possible it could have negative consequences?  For example, ISPs see it, quickly determine it's a mass-mailing (although to opted in users), and identify it as spam...

I think it would much more likely be beneficial (oh, they provide an unsub link in the header - great, let's let this mail through!), but it might deserve a little more looking into.  Dean, do you know anything about how ISPs, mail clients, and even other newsletter hosting providers are treating this header?  

If it were implemented now, I would use it, but I would be more comfortable knowing it is going to be handled appropriately.  

Dean, I did get that AOL block removed earlier - they were helpful.  I'm just frustrated I even had to go through that.  ...and my email hadn't been getting through to AOL users for over a month b/c my URL was in my signature!  Take care,
--Kyle