Talkback Data Loss Coming SoonTalkback is getting huge. Most people don’t realize that the size of Talkback (the database and the amount of processing needed) has grown immensely in the last couple of years as the userbase of Firefox has grown. Talkback wasn’t made to scale in the way we’ve needed it to. This is one of the (many) reasons it’s being replaced in Firefox 3 with Breakpad/Socorro. The Talkback database is now around 500 GB, which is way too large to be manageable. As a result, the scripts that removed old data are unable to complete and, thus, unable to lower the size of the database. There are several ways we can “fix” the issue:
There are advantages and disadvantages of all of these, but after some discussion, we’ve decided to do #3. Removing data will allow us to have a manageable database and will allow us to keep the database in shape for the long term by lowering the amount of data we keep. Sadly, as a result of this, we’re going to lose some (critical) data. Namely, remove this data will affect full stacks, topcrash reports, and smart analysis reports. Tomorrow we plan to take Talkback down for a few hours and do a full cold backup of the database. After the backup, next week we plan to remove full stacks from existing crash reports. If you have a stack you want to save, please put it in the relevant bug. New reports will, of course, generate full stacks, but all stacks from old crash reports (about 90 days worth, give or take) will be gone. Because full stacks will be gone, the main topcrash reports will not necessarily be correct and the smart analysis reports will be completely broken (they rely on full stacks to generate properly). This will last for a period of 10 days, then both reports will return to normal. Note: The removal of full stacks does not affect the stack signature. The signature will remain. We know this situation isn’t ideal, but it’s just yet another reason to convince your friends to upgrade to Firefox 3. If you have comments or questions, please email me. |
