I've heard people suggest tracking which email the user unsubscribed from but I don't currently store the ID's of previously sent messages on a per user basis. I do, however, obviously have the current followup message stored so this is a possible addition to the unsubscribe report. Also, if you use the option "Keep removed users in database but don't mail" and choose to view "Removed" users on the User Database page, you may be able to see what followup the user is expecting Next - so you could assume that they unsubscribed after the previous followup.
