[Dev] Link Tracking Improvements

[DW]

This is waaay overdue.  I haven't touched this feature since ListMail's inception and it is becoming critical that it be improved.  It's primitive state is, no doubt, reflecting poorly upon ListMail and it's usefulness is well, just not that useful. I think improvements to Link Tracking will be good for everybody.

Ideas:

1. Tracked links will have their own "Edit" page with many more options.

a) The ability to set up image links for HTML messages could be incorporated, with a fancy "preview" of your images.  The width and height of images could be detected and be settable.  The title would be settable.  I could even incorporate a CSS option so you can set up margins and borders if it would be useful.

b) SPLIT testing could be incorporated, so that 50%, 33%, 25% of your list would receive a different link.  Each tracked link could have up to 4 different URLs.  "Timing" might be lost in the event of a resume, but since we're cycling quite quickly, every 2, 3 or 4, the results would be fairly reliable.  We should actually be able to avoid this by storing information in the new 'queue protect' table..

2) I can't say I'm overly attached to the current style of tracked links.  Links could have and optional format, similar to new Remove Links.  I would like to get some feedback on different link styles for this.  We need to include a variable for the unique textual userid and the link ref code.  Unfortunately, the auto-incremented numerical IDs for these are prone to abuse.

Currently the Unique UID is fixed to 7 characters, which is why the current implementation is possible:
http://example.com/mail/link.php?x=z1y2x3wmylinkref

More options could be provided, depending on your preference, ie:
http://example.com/mail/link.php?a=z1y2x3w&b=mylinkref
http://example.com/mail/link.php?x=z1y2x3w_mylinkref
http://example.com/mail/link.php?x=z1y2x3w-mylinkref
http://example.com/mail/link.php?x=mylinkref_z1y2x3w

I may be able to pull some tricks, so links might look like this:
http://example.com/mail/link/z1y2x3w/mylinkref

I think that would require a custom .htaccess file, though, which makes installation more complicated and would cause compatibility issues.  Not a bad idea to keep in mind as something optional, though..

Please let me know if you have any ideas for the Link Tracking "Edit" page.

3. Tracked user viewing needs a major overhaul.  The page system needs to be cleaned up and a direct interface between links, users, and lists needs to be established.  This could include:

- Options for copying or moving users who have clicked on certain links (in X amount of time?) to existing and new lists for further marketing and followup.

- Options for creating User Selections matching users who have clicked on certain links.

- Overall, Week, Month, or other range viewing and statistics

Yea or Nay?

[BGSWebDesign]

Hi DW,

Wow, once again, you've laid out a HUGE agenda here with link tracking, I like it, but I could see this taking you a long time to complete, and I believe that Split Testing should be completed first...  here's the Split Testing link:http://listmailpro.com/forum/index.php?topic=455.0

http://example.com/mail/link.php?a=z1y2x3w&b=mylinkref

I like this link format best... I don't think the advantage of doing it with .htaccess is worth it, the one above is fine.

The ability to set up image links for HTML messages could be incorporated, with a fancy "preview

I don't need a preview, so this is not necessary as far as I'm concerned...

SPLIT testing could be incorporated, so that 50%, 33%, 25%

You know my thougts on this - but I'll add updates to the Split Testing thread, this is HIGHLY needed, more than any other feature.

Tracked user viewing needs a major overhaul. The page system needs to be cleaned up and a direct interface between links, users, and lists needs to be established.

That would be INCREDIBLE... if you can pull it off... I'm saying just get the Split Testing working first, then come back to this...

This could include:
- Options for copying or moving users who have clicked on certain links (in X amount of time?) to existing and new lists for further marketing and followup.

Yes, brilliant, but tracking X amount of time - while useful - could be a TOUGH one to program...  not sure I need it, but just knowing that the user clicked is good enough for me.... more useful would be knowing that the user clicked on a GROUP of links - from many different messages - but I can do that now - by just moving the Tracked Users to their own 'Interested' list... and then building a user selection around multiple 'Interested' lists.

- Options for creating User Selections matching users who have clicked on certain links.

See above, it can be done now... just provide us a way to APPEND the clicked users to a current list.

- Overall, Week, Month, or other range viewing and statistics

That WOULD be very useful.... still I believe Split Testing is more important, then come back to this.... stats for clicking would be useful... but to do that, you're going to need to create your own 'Tracking Clicks' database - since I'll be moving many of the clicked users over to my own 'Interested' list....   ALSO - you'll need away to maintain the NUMBER of Clicks held in memory before they are purged when you create these reports - excellent idea - but still NOT as useful as Split Testing...

[DW]

I've been thinking about how to best implement some better stats into ListMail (Thanks for the great suggestions in this area, Brett - it won't be long now).  One idea I came up with is to set up click stats similar to most pay-per-click programs - that is, on a per-day per-link basis.

A database would be created as follows:

ID, DATE (YYYY-MM-DD), LINKID, SENDS, CLICKS

When a click is made the clicks for that day would simply be incremented by one.  I would then put in a 'date range' and/or preset system for viewing past clicks.  Similar to PPC engines we might offer to view stats for "Today", "Yesterday", "Last 7 Days", "This Month", "Last Month", etc.

As you can see by the table field names I would also like to track "sends" or "impressions".  This is possible but not easy considering there are currently around 8 different ways ListMail sends messages.  I'm sure I could make it work but would you prefer the additional MySQL UPDATE for each individual sent message be avoided?  You might say yes but remember MySQL is very efficient and you will probably hardly notice - there are already a number queries performed for each message, such as when processing message codes.  Updating at the exact time each message is sent would be the only way to track each and every impression reliably.

I would also like to track clicks on a per-message basis, but again this gets difficult with the way ListMail's messages are distributed throughout different tables.  (ie. welcome message vs. followup vs. standard email)  I should probably put all messages in a single table before I do this...  That, or I could simply track on a per-day, per-link, per-MESSAGE basis with an additional MSGID field in the new table suggested above. Each tracked link in each message would have it's own row for each day.  This would probably be the easiest and fastest way to implement per-link per-message tracking.  A maintenance option could allow you to restrict tracking to 1 or 2 years to avoid a huge database.  Of course, the problem with the various types messages still remains.  ie. what to do with Welcome messages, etc? I still haven't figured out how I might display per-message results.

What do you think?

[BGSWebDesign]

Hi DW,

ID, DATE (YYYY-MM-DD), LINKID, SENDS, CLICKS

When a click is made the clicks for that day would simply be incremented by one. I would then put in a 'date range' and/or preset system for viewing past clicks. Similar to PPC engines we might offer to view stats for "Today", "Yesterday", "Last 7 Days", "This Month", "Last Month", etc.

I do NOT need the fine level of detail you are discussing - knowing the exact times, etc, is not useful at all and will create a tremendous overhead as you already know.  Oops - I think I solved your overhead problem - read below for my idea on how this COULD work including the great detail of date/time/ID - you really need a time too probably you could just save it in Unix Time format to save on overhead?

I think it's better to think about this another way - I REALLY, really, Really like the script that moves Clicks (Hits) to another list.  Can you think about handling the click throughs this way - in other words ONLY track the Links that are listed in the Tracked Links list, but for each list provide a List Name (listid) to COPY that user info (id) to for further stats and other uses - that would solve the problem I have now - having to watch the Tracked Hits and every 2-3 weeks run the Copy Tracked Hits script to update (copy or append is what I use) my 'Interested Clicks' to this database.  

Here's an idea - you just provide a 'List ID to Append To' for each Link on the Link Tracking Page - when the user clicks you record the click on the Link Tracking page as you normally do (but you add a date/time clicked to this table), then later on you provide us a script that will run (via CRON) to Copy the User info to the appropriate List ID as a New User (starting them at Message 1 in Followups, etc) and adding them at the date/time stored in the Link Tracking table that you recorded when the user clicked. You can use this Date/Time for calculating Click/Hit stats now - since you know this user wasn't really ADDED to the list, but instead was PUT there by coming from a Click (link tracking), or from a Purchase - eventually you could Update the User if they make a Purchase - by included a Purchase Date/Time field in the Link Tracking table too - so that ROI and Click-Thrus to Purchase can also be tracked from this same List!

How does that sound?  

What this ALSO does is allows you to create ONE Statistics module that can be run on ANY List ID - to give anyone the Stats they need on users that were Added to that list (date/time), etc, BUT with these special 'CLICK List ID's, the Admin also gets more - they can view the Stats and get an actual Range of Stats (date/time) for when the users clicked through to get to this List.  The admin can Add useful information in the List Title (such as Prepending 'CLICK'), or better yet, think about adding a List Description field - the Admin can then add a long description here such as 'CLICKS from my Test List', etc...

I think this will make things easier for you - and you won't have to spend so much time thinking about building out another huge database to track clicks/links - just use your current format (for lists) and add users to new lists as needed when Clicks are done.

That could still be TOO much overhead trying to move the user when every new click is done -so instead just do what another user suggested - provide us with a script that will run as a CRON job to move the users over to the appropriate List ID's listed for each Link Tracking link Message Code some time (once/day) or whenever the Admin sets it up, that should prevent all of the overhead!

What do you think?