Archiving tweets offline

Berkeley, California

Nothing about Twitter’s infrastructure (the web app, native app, or other tools) feels like it was designed with user experience in mind. Given that my colleagues and I use Twitter as a stand in replacement for (most) email type activities, having a search functionality is really important to me.

After much digging around, I didn’t see much development in the third party landscape. I had used Tweetnest a long time ago and it still seems to be the best way to archive and search one’s own Twitter archives. Since this site is now static on gh-pages, I am now running Tweetnest on a small Digital Ocean 1 instance with a cron job 2 updating my offsite tweet database every 30 minutes.

Features that could use improvement:

  • More granular search (e.g. search = "foo" before:10/10/2015 after:10/10/2014 has:links)
  • Other metrics that could show change in follower dynamics, perhaps overlaid on the monthly tweet count figure.

I could add these features but it will require dusting off my php skills which I haven’t used in over a decade.

1: A simple LAMP stack is easy to set up.

2: crontab guru is a nice interface to set up schedules.