Queue stuck

[BGSWebDesign]

Hi,

I've researched this on the forum, yet, I still cannot find a good way to track these down?  My Send Queue keeps choking, and I need to find out WHICH email address is  causing it?  I'd like to find out for good, so I could prevent any email address like that from getting into the system, but for now if I could just find out which row is bad, and remove it (and then find a way to get at the email address of that row)?

Here's what my top row of the table sendq looks like:
--
ID:                                                           BAT         BATTYPE NTYPE UID  MID         XTRA
==============================================================
0dbae0e52d312ac7440f8a9aac542ff2    11c1e3    2    2    1138039    3421    fup
---

Can you give me some idea, of how I would track down the USER this row is trying to access?  Also, is THIS the row I should be concerned with (the TOP row of the table), or should I be looking at the BOTTOM row of this table?  Which looks like this:
--
ID:                                                           BAT         BATTYPE NTYPE UID  MID         XTRA
==============================================================
b8c90e8e37e935ffd0c29e5ce6f6e9cd    11c1e3    2    2    1138192    3518    fup
---

So, What does all this mean?  How can I check the UID, and run it back in the USER table to find that user? 

[DW]

Hi,

The SMTP log will likely indicate which email address is holding things up. It may end up eventually timing out and skipping the user. Such things sometimes happen when a server does a long or failing DNS lookup. The mail server may be able to be configured to delay lookups until sending time and not do them during queuing which could eliminate the symptoms.

Regards

[BGSWebDesign]

Hi DW,

The SMTP log will likely indicate which email address is holding things up.

I do not have SMTP logging on, but I just now turned it on.  When the queue resumes, will it write the log, or am I going to have to Cancel all of the queue entries, and let it re-start over, and wait for it to hit the bad address again?   It doesn't appear to be writing the log, now that I've turned it on... so seems I will have to delete all the Dailymail queue entries and watch for the next one to choke?

server does a long or failing DNS lookup. The mail server may be able to be configured to delay lookups until sending time and not do them during queuing which could eliminate the symptoms.

What is the preferred way to turn off the DNS lookup, is that in the code?  You say configure the mail server, what is the best way to do that?  I assume you mean changing the configuration of the Courier mail server?  Is that right?   Where would I look for those configurations? 

Is it only possible to make those changes in the mail server, or is it possible to add code (or update code) that is already in ListMailPRO now to limit DNS lookups when queuing is taking place?

[DW]

Hi Brett,

When the queue resumes, will it write the log

Yes, you should not have to cancel your queues. The logs should appear in the /attach folder, which must have the correct permissions (777 or read/write/execute for all).

What is the preferred way to turn off the DNS lookup ... Is it only possible to make those changes in the mail server

Yes, if this is the problem, and I've only heard of it being a problem on sendmail systems, the mail server config could be updated.

Regards

[BGSWebDesign]

Hi DW,

Thanks... unfortunately I cancelled all of my jammed queues!  I probably lost 50,000-80,000 emails that were scheduled to go out...   

After turning on SMTP logging, I see why I do not always have it on, it takes 2-3 times longer to send mail out when it is turned on.    This is why I would prefer it if there was a way to track down the bad emails that in the current Queue that is stuck?  I thought in the past I was able to do that, by converting the UID to HEX and looking it up that way in the Users table, is that correct, or will it not work that way? 

I believe I'm running on Sendmail, can you tell me what the suggested configuration updates are on a Sendmail system to turn off the DNS lookup, if I find that's the problem, I'm going to have to do it, I continue to get blocked up queues?

[BGSWebDesign]

Hi DW,

I once again have blocked up queue with 15,000 emails.  I do not have logging turned on, because it slows sending, and creates large log files, so I'm asking again, is there some way to interpret the queue database entries in lm_sendq and convert the uid field into the same uid field that appears in lm_users because they do not seem to match?

How do I make that conversion?  I think I tried getting the uid field from lm_sendq and converting it to HEX, but not sure that worked either? 

1.) Can you please tell me - what is the proper way to convert lm_sendq/uid field into the uid field that appears in lm_users?

2.) Are the queued entries that are stuck/jammed, coming from the TOP of the lm_sendq database, or the BOTTOM of the lm_sendq database?  In other words, when I am looking up 'stuck queue entries', do I assume that it is RECORD #1 that is stuck and cannot send, or is it the LAST RECORD in lm_sendq that is stuck?  Does it go sequentially after that?

[DW]

Hi Brett,

1.) Can you please tell me - what is the proper way to convert lm_sendq/uid field into the uid field that appears in lm_users?

The uid field in lm_sendq corresponds with the id field in lm_users, so you could do the following, where "123" is the UID from the lm_sendq table.

Code: [Select]

SELECT uid FROM lm_users WHERE id = '123';
There is no other way to convert, per se, as the lm_users uid is simply randomly generated.

2.) Are the queued entries that are stuck/jammed, coming from the TOP of the lm_sendq database, or the BOTTOM of the lm_sendq database?  In other words, when I am looking up 'stuck queue entries', do I assume that it is RECORD #1 that is stuck and cannot send, or is it the LAST RECORD in lm_sendq that is stuck?  Does it go sequentially after that?

The queue-reading process does not use an ORDER BY clause so the order cannot be guaranteed...

Messages may be "timing out", causing confusion in the sending process. You could try increasing the SMTP timeout per this and other posts.

The best way to see what's happening is with the SMTP log.

Regards

[BGSWebDesign]

Hey Dean,

The uid field in lm_sendq corresponds with the id field in lm_users, so you could do the following, where "123" is the UID from the lm_sendq table.

Are you certain about that?  I've tried that SQL query over and over, for multiple UID's here's a query I use:

Code: [Select]

select * from lm_users where uid='1063330';
AHHH - never mind that part,  I see the problem, I was looking for uid in lm_users, when it's the id field I am matching!

Not sure why though, you chose to name the field uid in lm_sendq, when it's matching the id field in lm_users?  Usually you want to keep the field names the same, especially when you have a id and uid field both in lm_users, and in lm_sendq!

Now, to resolve why I've got so many hung up in my queue, looking now at SMTP delay... just a min...

[BGSWebDesign]

Hi,

See my previous post, I was pulling the wrong uid field, anyway, all those email addresses (I looked up from the UID's) in the front & back of lm_sendq look fine, so I looked to update my smtp_timeout variable, what I found is that it was missing from my config.php, even though previously I had one set at 30:

Code: [Select]

$smtp_timeout = 30;
If it is missing and then I add it, does the queue continue on, as it appears to still be locked?  Do I need to cancel all those mailings that have failed, and re-start them (which is impossible as they are Dailymail sendings)?

[BGSWebDesign]

Hi,

My queue is stuck at 23075.  I have sent to large lists before without any problem, I have resume running every 2 minutes and this has worked for years, but now for some reason, my queue is stuck and does not move?

I've confirmed now that the email addresses are good in the lm_sendq database, I've tried removing 50 records at a time (25 from top, 25 from bottom), but the queue still does not move?  I've also changed my SMTP timeout to 45, as indicated.  Since this is a Dailymail sending, there's no way I know of to delete this and re-start it... so

How am I to get the queue moving again, and what else can I try?

[DW]

Hi Brett,

The best place for more clues is the SMTP log. What does the end of the log say when it won't send?

The $smtp_timeout adjustment is only effective if the problem is identified as individual emails timing out when queuing to the server due to DNS delays or other. You may see evidence of this in the SMTP log.

It is difficult and time consuming to make further guesses at a possible cause or solution.

If you still can't figure it out please create a support ticket of the type Other/Bug Investigation at https://listmailpro.com/support

Regards

[BGSWebDesign]

Hi DW,

How would I get the SMTP log to write, when this is a Dailymail send that is stuck right now?  If I turn on SMTP logging, that will not record anything for that (Dailymail) send, correct?  So, I would be stuck to looking at the SMTP log on the next one, right? 

When I did have SMTP logging on, I was getting 0 byte log files, I just deleted hundreds of them.  What would cause that?

[DW]

Hi Brett,

How would I get the SMTP log to write, when this is a Dailymail send that is stuck right now?  If I turn on SMTP logging, that will not record anything for that (Dailymail) send, correct?  So, I would be stuck to looking at the SMTP log on the next one, right?

When you Resume any queue, including a Dailymail queue, the SMTP log is written, if enabled.

When I did have SMTP logging on, I was getting 0 byte log files, I just deleted hundreds of them.  What would cause that?

Are you out of disk space?

Regards

[BGSWebDesign]

Hi DW,

When you Resume any queue, including a Dailymail queue, the SMTP log is written, if enabled.

It does not seem I can Resume a Dailymail queue, as I have auto-resume firing off every 2 minutes.  Even then, if I switch my configuration to Always write SMTP logfile, it doesn't make any difference, nothing gets written, no file at all.

Are you out of disk space?

It seems these zero byte files are written over and over, when I had resume running and had the: Always write SMTP log file set to

• On, before the Dailymail.php ran that stuck the queue.

Here's the one larger file that was written, but I don't think this queue got stuck, does this look like anything to worry about? I notice that before every message, there is that
SMTP message: '354 Enter message, ending with "." on a line by itself'?
--

Code: [Select]

250 OK

250 Accepted

> DATA

354 Enter message, ending with "." on a line by itself

> SENT DATA

250 OK id=1YQd6m-0000cf-NT
--

[BGSWebDesign]

Hi DW,

See my previous post... then what I found below...

It seems these zero byte files are written over and over, when I had resume running

Check that, it appears that every SMTP logfile written is zero bytes, even when it first starts Dailymail, and continues running it (with resume).  I've got a directory full of zero byte files, and I do not have a disk that is full. 

At some point though I did get a SMTP logfile written (see below), but that file may have been written from a Send an Email send (instead of a Dailymail send), is there some problem with Dailymail writing logfiles?