At work on v1.85

[DW]

I'm going to put a few more features in and then will be releasing.  It should be ready in the next week :lol: or two or maybe three.

Regards

[BGSWebDesign]

Hi DW,

Ok, great, just a few questions on how this works... before I make changes, I have two questions:

1.) Subsequent Monthly CRONS will eventually not trigger
------------------------------------------------------------------

Quote:
Let's say that I want to hit the FIRST Weekday of the month, so I need Monday-Friday, then, I want the NEXT Followup to also hit the NEXT following FIRST Weekday (of the following month).... how would I do that, would the next followup Cron be the exact same as the Cron above?

Yes, that's right, but if the two crons are right next to each other you will need to put in a delay of a few days on the second one to get around the days following the first one.

Please explain this in detail, are you saying that 2 CRON schedulings that are in sequence MUST allow for a delay from the 1st to the 2nd?  Otherwise they will both trigger and run in the same month?   If the first CRON executes... you do not consider running the 2nd one until the next day... so... you're saying that FOLLOWUP2 in this scenario requires a delay of the number of days that may follow FOLLOWUP1 that could trigger FOLLOWUP2?   So, for example, using November/December as example... FOLLOWUP1 is as follows:

Code: [Select]

1-3 * 1-5


This would trigger FOLLOWUP1 on Tuesday, Nov 1, 2005, the next CRON then must allow for a delay so that it does NOT hit on Nov 2-4, which would require something like:

Code: [Select]

1-3 * 1-5  DELAY: 4


Which would mean that FOLLOWUP2 will go out on Dec. 5th, (4 days delay from Thursday December 1st - which basically throws FOLLOWUP2 into the second week in December, instead of the first, is that right?  

I think I see how this works, but am a bit confused by the logic (which can be fixed very easily), so it seems that you run the tasks as straight CRON jobs, and do not flag when one has been executed - and which month they run in?  

You could solve this problem if you added a flag to your Followup table for each followup, the flag would be a simple MONTH number - since it appears that the only time this will be a problem is one month to the next (I don't think this problem could occur with other scheduling, ie. daily, weekly, day of week, etc, at least I couldn't think of a scenario)... So with the flag you set it to the MONTH that the FOLLOWUP runs in: in the example above you set it to 11 after FOLLOWUP1 executes...  what this does is prevent FOLLOWUP2 from running in November, because you check the flag BEFORE running each followup, if it is the current month - and the CRON schedule for the followup is a MONTHLY (* in the 2nd CRON position), you ignore this followup... this would keep happening for each day in November that would trigger it, UNTIL you are in the month of December, since the monthly flag is less than the current month on December 1st, you can now trigger FOLLOWUP2.  

Do you see how this would prevent the problems you speak of, and would help those out who do Monthly scheduling... I think it's going to be necessary, because if you don't allow this to run this way, subsequent followups are ALWAYS going to be have to be delayed further and further ahead of the previous FOLLOWUP, for example think of a situation where you have 10 FOLLOWUPS, all of them Monthly, each one would be triggered later and later in the month under your current logic, that would never work, since after about 2 or 3 followups, the 4th followup would NEVER get triggered properly.

2.) CRON followups with negative delays
-----------------------------------------------
You did not answer my previous question about allowing for a followup that is 2 weeks PRIOR TO a yearly schedule, for example:

The help page says:

Cron-style followups can also use the Delay value. Example: You want to send an email out not the very next Sunday after the last followup, but the Sunday 2 weeks from then. To do this, set up a Delay of 13 and a cron set for "Any Sunday" which is, as per the example above, * * 0

I'm asking that you allow for a negative delay here ONLY if the CRON schedule is being used, that would allow for the example I posted:

is it possible to put in a -13 delay to specify that I need this sent out, for example 13 days BEFORE the CRON set runs out... here's an example... FOLLOWUP1 needs to go out 2 weeks BEFORE 1 year elapses, FOLLOWUP2, 3, 4, etc.. then proceed on DELAYS... after FOLLOWUP1.

These two issues are going to present a problem, most likely for all users, can you please modify to fix these issues?

[DW]

Which would mean that FOLLOWUP2 will go out on Dec. 5th, (4 days delay from Thursday December 1st - which basically throws FOLLOWUP2 into the second week in December, instead of the first, is that right?

I don't think so.  The value of Delay is processed BEFORE the cron task, not after.  The delay will decrease to 0 over the days and THEN the cron task will be processed when valid.

As for monthly scheduling, etc. I am thinking to provide a "repeat" option, either repeating it a few times or unlimited times.  One might have unexpected results if they set a scheduled message to run on "any day" of a given month - the message might be sent EVERY day of that month.  This can be avoided by specifying the particular day to send on.

I'm asking that you allow for a negative delay here ONLY if the CRON schedule is being used, that would allow for the example I posted:
Quote:
is it possible to put in a -13 delay to specify that I need this sent out, for example 13 days BEFORE the CRON set runs out... here's an example... FOLLOWUP1 needs to go out 2 weeks BEFORE 1 year elapses, FOLLOWUP2, 3, 4, etc.. then proceed on DELAYS... after FOLLOWUP1.

This might be difficult.  Could you simply set your first delay to 352?

Regards

[BGSWebDesign]

Hi DW,

I don't think so. The value of Delay is processed BEFORE the cron task, not after.

Got it, yeah, that makes a lot of difference, thanks  

I think you should UPDATE your Cron Help Page to make this clearer, I didn't know if the Delay ran BEFORE or AFTER the Cron schedule.

As for monthly scheduling, etc. I am thinking to provide a "repeat" option, either repeating it a few times or unlimited times.

This won't help me much as all of my monthly  mailings are different, I could see where it may help others though - nice feature...

This might be difficult. Could you simply set your first delay to 352?

Yes, sure this will work, I just thought it would be nicer to FORCE it into an exact yearly schedule and with leap years it will be off by one, but not bad, 1 day off every 4 years...  

Also, knowing that the delay runs BEFORE the CRON helps, I can set the delay to 352, and then a CRON for a weekday everything will be fine...

Thanks again, setting it up now, I'm going to add myself into some lists, and I'll let you know if I see any problems..

Update - Questions - Followup
----------------------------------
Ok, here's a quick question, I've been setting up the CRONs on my Followups, but here's what I wonder:  I have many lists already setup with DELAYED followups, I am changing those to CRON followups (with NO delay), can you tell me if the CountDOWN takes place as normal, since I see in the User Database that the Delay is currently reading 7, will that Count down and THEN start the CRONs?  If it will, I think it's best if I can just SET ALL THE Delays to Zero for these lists right now, and let the CRONs run as if the list just started... can you send me a bit of SQL to reset all of the Delays to 0 in specific lists?

Formatting
------------
It would be nice to SEE the CRON on the Followup Screen, instead of 'Y', and having to Edit to see what the CRON setting is.

[DW]

Brett,

I think you should UPDATE your Cron Help Page to make this clearer, I didn't know if the Delay ran BEFORE or AFTER the Cron schedule.

Sounds good - I have done this now.  I might also update it further to include examples of areas of confusion we have mentioned in this post.

Yes, sure this will work, I just thought it would be nicer to FORCE it into an exact yearly schedule and with leap years it will be off by one, but not bad, 1 day off every 4 years...

That would be better, and a 25% failure rate is pretty bad... This could be solved with a new feature "send followup X days/months/years after signup/last followup".  The feature could cause ListMail to simply process the # of days or years, taking leap years into account, and setting the delay #.

I have many lists already setup with DELAYED followups, I am changing those to CRON followups (with NO delay), can you tell me if the CountDOWN takes place as normal, since I see in the User Database that the Delay is currently reading 7, will that Count down and THEN start the CRONs? If it will, I think it's best if I can just SET ALL THE Delays to Zero for these lists right now, and let the CRONs run as if the list just started... can you send me a bit of SQL to reset all of the Delays to 0 in specific lists?

Countdown will take place as normal.  If your users have a delay it will be decreased over the days to 0 and then the cron will be processed or standard followup will be sent.

The MySQL query you are looking for is (example for list 1):

UPDATE lm_users SET cdel = '0' where list = '1'

It would be nice to SEE the CRON on the Followup Screen, instead of 'Y', and having to Edit to see what the CRON setting is.

I thought about this, too.  I would love to display it on that page.  The only thing is the cron-style string can be up to 255 characters long and  I can't fit that amount of information in the small space.  Make it a textbox with a restricted size.. yes, of course.  I'll do this but need to put the error checking in for the cron task there.  No problem.

Regards

[BGSWebDesign]

Hi DW,

Ok, I've found what seems to be a problem with the NEW Cron scheduling...  

I've added new CRON style scheduling to about 15-20 of my lists... everything seemed to be updating fine, DELAYs counting down, even AFTER I added the new CRONs... but.... here's the problem...

After I went into one of the LISTS and manually set the delay down to ZERO (using the MySQL query):

The MySQL query you are looking for is (example for list 1):
UPDATE lm_users SET cdel = '0' where list = '1'

... I just noticed this morning that the OTHER lists that I have setup with CRON style scheduling - the DELAY is NOT counting down... it has stayed at 5 (where it was when I used the MySQL query above) to set the other list down to zero...

Can you PLEASE HAVE A LOOK at this ASAP.... it's going to force me to do manual DELAY updates on ALL of my CRON style lists until you get this fixed!

This appears to be happening with ALL of my Lists, it seems like DailyMail is not even getting called?  

FOLLOWUP
-----------
Ok, had a quick check, and DailyMail has not executed since October 27th, about 4 days ago... so I tried running the 'wget' call that I have in my local CRON which is below:

Code: [Select]

15 2 * * * /usr/bin/wget -O - http://mydomain.com/mail/dailymail.php?pw=xxxxx

I ran that from the command line, and I get this error:

Code: [Select]


HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]

    [<=>                                                  ] 0             --.--K/s
Done!<br><br />
<b>Warning</b>:  mysql_fetch_row(): supplied argument is not a valid MySQL result resource in <b
>/home3/mydomain/mydomain-www/mail/admin.php</b> on line <b>2499</b><br />
admin-45-You have an error in your SQL syntax.  Check the manual that corresponds to your MySQL
    [  <=>                                                ] 369          319.48B/s

09:25:08 (319.46 B/s) - `-' saved [369]


There's an error on line 2499 in admin.php which is this line:

Code: [Select]

list($nbounce)=mysql_fetch_row(mysql_query("select nbounce from $ctable where 1",$link) or die('admin-44-'.mysql_error()));

I never had this problem previously?  Have you updated something in admin.php that is causing this problem?  Is there some other problem with MySQL?

FURTHER FOLLOWUP
-----------------------
Ok, I found/fixed the BUG at line 2499, it should read like this:

Code: [Select]

list($nbounce)=mysql_fetch_row(mysql_query("select nbounce from $ctable where 1",$link)) or die('admin-44-'.mysql_error());


But, when I fix that, I now get an error on the next mysql_query line, this one:

Code: [Select]

$urows = mysql_query("select id,list,bounces from $utable where email like '$email'") or die('admin-45-'.mysql_error());

I get this error:

Code: [Select]

Done!<br>admin-45-You have an error in your SQL syntax.  Check the manual that corresponds to your    [  <=>                                                  ] 183           87.23B/s

09:43:48 (87.22 B/s) - `-' saved [183]

So, it's bombing on this mysql query?  Why, I don't know because it seems that it is IDENTICAL to what you had in the last version that runs just fine - apparently it does not like to 'die' here, because the rest of DAILYMAIL does not get executed....   so, it seems there's some problem with this mysql query?   I'm going to switch it back to the old code WITHOUT the 'or die' and see if that works...

FURTHER FOLLOWUP
-----------------------
Ok, I removed all of the 'or dies' in the function bounce($email,$msg), now DAILYMAIL will run, BUT, I do NOT get my Delays updated, AND I do NOT get my Dailymail report, so something else has gone wrong.. it seems you have introduced a 'bug' DW in the code in admin.php that now prevents DAILYMAIL from running properly... this will have to be fixed... I cannot even RUN, or TEST ListMailPro until this is corrected.... I'm going to try to go BACK to the previous version and add in the CRON stuff... myself... I need this fixed ASAP as my CRONs will start firing off soon...

CORRECTION - FURTHER FOLLOWUP
------------------------------------------
On further checking, I noticed that DAILYMAIL was still processing in the send queue, though I did run it again (will that cause problems)... anyway, so it's been run twice now.. and it does appear that removing the 'or die' from the function bounce does FIX the problem.... so there's some problem with the MySQL in this line of code:

Code: [Select]

$urows = mysql_query("select id,list,bounces from $utable where email like '$email'") or die('admin-45-'.mysql_error());

ADDITIONAL FOLLOWUP - CRONs (followups with delays) being triggered when they should NOT BE!
---------------------------------------------------------------------------------
Here's another problem I just found, I noticed in the tracking database I'm getting hits to a list that I have setup with CRON followups, BUT, they should NOT be getting them, there are only 2 followups, as indicated below:

Code: [Select]

FOLLOWUP1 CRON: 15-19 * 2-4  Delay: 0
FOLLOWUP2 CRON: * * 1,5  Delay: 2

As you see, followup1 should ONLY go out on the 15-19th of the month, and only if the day is a Tue-Thursday...  followup2 ONLY goes out AFTER followup1, and ONLY if the day is a Monday or Friday, somehow, I notice now that in the list - FOLLOWUP2 has GONE OUT!  Apparently FOLLOWUP1 has NOT, as I do NOT see any clicks on Tracked Links that were included in FOLLOWUP1 (so I assume that NO Followup1's went out).   Anyway, this is troubling indeed, as I see that the SEQ # = 3 now for some of these users who have received FOLLOWUP2 - HELP - DW, is there some problem/reason why FOLLOWUP2 Triggered BEFORE FOLLOWUP1?  Is there a problem with setting a CRON, then a CRON/DELAY, without having a FIRST (PLAIN DELAY) for the Followups?

Any ideas on what is going on here, DW??

[DW]

Brett,

PHP doesn't seem to like the "or die()" statements nested.  I noticed this when adding the statements to better track errors. I thought I had fixed them all but must have missed this one (also, a bracket was misplaced...). I didn't notice because I wasn't processing any bounces in my testing.

Change the line:

Code: [Select]

list($nbounce)=mysql_fetch_row(mysql_query("select nbounce from $ctable where 1",$link) or die('admin-44-'.mysql_error()));
To:

Code: [Select]

$brow = mysql_query("select nbounce from $ctable where 1",$link) or die('admin-44-'.mysql_error());
 list($nbounce)=mysql_fetch_row($brow);

CRONs (followups with delays) being triggered when they should NOT BE!
...is there some problem/reason why FOLLOWUP2 Triggered BEFORE FOLLOWUP1?

I've done some testing but haven't recreated the error yet.  I've tried, for example, setting up followup 1 for "tuesday" (* * 2) and followup 2 for "monday" (* * 1) to see if #2 would go out before #1 (it being Monday now), but nothing is sent, as expected.  Also, with some debug output I see it only checking the cron entry for followup 1 to determine whether or not to send. I'll keep trying.  Is there any chance at all some of your users were already on Seq #2?  The MySQL statement I provided simply decreased delays to 0 - it was not intended to set all sequence numbers to 1.

Regards

[BGSWebDesign]

Hi DW,

PHP doesn't seem to like the "or die()" statements nested.

What is the proper way to handle the 'or die()' then?  Since this statement also fails:

Code: [Select]

$urows = mysql_query("select id,list,bounces from $utable where email like '$email'") or die('admin-45-'.mysql_error());
even though the one above worked just fine by fixing the parentheses?

Is there any chance at all some of your users were already on Seq #2? The MySQL statement I provided simply decreased delays to 0 - it was not intended to set all sequence numbers to 1.

It is possible.... but I'm not sure how, can you please provide the MySQL statement to set the SEQ # back to 1?

Thanks,

[DW]

What is the proper way to handle the 'or die()' then? Since this statement also fails:

Are you sure it's all on one line?  Do you get an error? I'm not having troubles with this one:

Code: [Select]

$urows = mysql_query("select id,list,bounces from $utable where email like '$email'") or die('admin-45-'.mysql_error());
To update all users to Seq #1 and Delay 0 this is the query (example for list 1):

update lm_users set cseq = '1', cdel = '0' where list = '1'

Is there any way you can test the new changes without using your live lists so-as to prevent undesired results with your active clients?

Regards

[BGSWebDesign]

Hi,

HEY - BIG HINT... here's a problem, ran your update query, this one:
----------------------------------------------------------------------------

Code: [Select]

update lm_users set cseq = '1', cdel = '0' where list = '32'
and I got this:

Code: [Select]

Affected rows: 4447 (Query took 0.2094 sec)

BUT, looking at LMP it says:

Code: [Select]

Active Users 1-20 of 14335

ANY IDEAS??  As you can see, LMP reports 14,335 Active Users in this List (list 32), but the query only affect 4,447?

So, 4,447 were the users that had the WRONG delay, I'm starting to wonder, maybe these users were the ones that were added - in the last 4 days (when DAILYMAIL was NOT running properly)... that MUST be the problem, look into it.... this is what I wonder about, ADDING NEW users, to this list, is causing SOME of those users to MISS the Cron Schedule... test it out and see DW, I believe that could be it...

Are you sure it's all on one line? Do you get an error? I'm not having troubles with this one:

Yes, it's all one line, I reported the problem above:

Done!<br>admin-45-You have an error in your SQL syntax.  Check the manual that corresponds to your    [  <=>                                                  ] 183           87.23B/s

09:43:48 (87.22 B/s) - `-' saved [183]

Any ideas?  This bombs, so I took out ALL of the 'or die' in the admin.php module after the first error (line 2499 and below).

Is there any way you can test the new changes without using your live lists so-as to prevent undesired results with your active clients?

No, it's the best way to do it, I've got email that must go out... so, 'let er rip'....    I'll keep watching to see if the sequences/delays get off...  I'll let you know what I find.

[DW]

Any ideas? This bombs, so I took out ALL of the 'or die' in the admin.php module after the first error (line 2499 and below).

Maybe something went wrong when removing the "or die" statements.  I'm going to send you my admin.php via email.  I have verified the fix for the "nbounce" line and successfully processed bounced messages via mailbox with it.

Regards

[BGSWebDesign]

Hi DW,

Unfortunately, nothing went wrong when I replaced the 'or die', I'm an experienced programmer (20 years), I know how to make those replacements...  

I received your update, but as I mentioned previously, this line does NOT execute (it bombs with a MySQL error), here's the line of code:

Code: [Select]

$urows = mysql_query("select id,list,bounces from $utable where email like '$email'") or die('admin-45-'.mysql_error());

Here's the MySQL error straight from the command line:

Done!<br>admin-45-You have an error in your SQL syntax.  Check the manual that corresponds to your    [  <=>                                                  ] 183           87.23B/s

09:43:48 (87.22 B/s) - `-' saved [183]

I have specifically listed this ERROR message above, DW, it seems that there is some problem with it executing, perhaps it has to do with the Version of MySQL (I'm running MySQL version 3.58 on this domain), maybe it's the older version of MySQL, anyway, they do NOT plan on upgrading on that server - so, perhaps you'll need to do away with the 'or die' calls for the next release.... since others may be running older versions of MySQL as well...

Let me know what you think?

Also, can you please answer my post at the end of this thread - I believe it's important for multi-list user Profile pages:
http://listmailpro.com/forum/index.php?topic=900.0

[DW]

Brett,

I am running MySQL "3.23.58-1.73" in my testing.

The next step would be changing the code to find the error.  It's possible that a malformed email address is coming through that needs to be addslashes()'d.  Take the query and place it in a variable so it can be echo'd in order to see if there is a problem with it.

Code: [Select]

$cmd = "select id,list,bounces from $utable where email like '$email'";
echo "CMD=$cmd<br>";
$urows = mysql_query($cmd) or die('admin-45-'.mysql_error());
If the $cmd line reads as follows there should never be the possibility of a syntax error, unless $utable is somehow undefined.

Code: [Select]

$cmd = "select id,list,bounces from $utable where email like '".addslashes($email)."'";
I don't have time to work on the user profile script right now as I have a couple paid things to take care of and need to get the above issue sorted out and get back to work on the update..  I hope to eventually get the feature into ListMail.

Regards

[BGSWebDesign]

Hi DW,

I'd like to test this, but I can't test it that way... I'm running in live mode, probably the best thing would be if you could, PLEASE provide a way to dump some of this info to a .log file (LOG FILE), it would help others too, who are trying to debug, that way you could just ask them for their log and you could determine what is going on...

should never be the possibility of a syntax error, unless $utable is somehow undefined.

I think this MAY be the problem, here's what I noticed...  1.) Bounced members from yesterday to today has NOT increased, even though Active members have, so I believe their should have been some bounces, 2.) I find in my bounced email box a group of addresses in there... I've saved that file, and tomorrow will look to see if they are removed... obviously if they are not removed tomorrow there IS A PROBLEM with the bounce tasks being handled properly...

so maybe $utable is not defined - as I mentioned above....  I can't keep running Dailymail to test it... please tell me the code to put in to write the info to a log file? OR - another option, when running DailyMail from LMP provide a checkbox (run in TEST MODE), which does NOT increment the Delay, and does NOT send messages?

That is the best way, right now, I can't just run dailymail over and over to test it, since it will BLOW my scheduling!

Further Update
----------------
Ok, I've confirmed that the BOUNCE handling seems to be working fine, so it seems that the $utable variable is fine...  what's next for testing?

[DW]

I apologize for the delay - I know these errors are hard on you.  Did you try adding "addslashes()" around the $email variable as I mentioned?  If you did, you may not have had a chance to test it. Before doing that I'd be curious to see the output of the query causing the error.

I have finished work on the new dailymail options.  You can now disable individual parts of dailymail when running it manually.  I have sent the new files to you by email.  Code note: I had foolishly mixed the report with followup processing and had to move a few things around and add an extra cycle through the list loop to get the report working.  I'm quite certain I've got it all sorted but please let me know if you notice any errors.

Let me know what you discover regarding the error during bounce processing - I'd love to get to the bottom of it.  I think it could be caused by invalid addresses being allowed through.  If that is the case the addslashes() suggestion should fix it and a call to the valid_email() function would guarantee input is as expected.

Have your cron-style followups been ok the last few days?

Regards