I’ll have to consider the database meltdown as something of a blessing in disguise. First, in my digging around to find out what happened I discovered some previously unknown bugs. With those now fixed, I’ll be scraping in data much more reliably.
Second, in working to reconstruct as much data as I could I built some new code to back-fill a lot more data, so now we’ve got a lot more info than ever before!
Third, because we’ve got so much more data, I’m finding that I need to learn and discover more efficient techniques for searching through the database, which is bringing much improved performance as I rework the older code with the new techniques.
Last, Danny Ayers (wikipedia) at Talis (wikipedia), whose work and writing I’ve always much admired and has guided this project in many ways, popped right in to nudge me to sign up for an account with Talis. I’m on my way! (And notice, that’s another virtue of linking!)