Experiments with "The Quantified Self"

Ever since I read about "The Printable CEO", I've experimented with keeping track of some personal metrics. Whilst the PCEO allowed me to keep a rough score of how much work I was doing, I always felt that with a little tweaking it could answer much more detailed questions.

Specifically, I wanted to be able to answer questions like:

  • What am I really spending my time doing?
  • Where am I wasting the most time?
  • What parts can I remove or automate to make me more efficient?

Once I started thinking about what I'd like to know, my brain started throwing out way more questions:

  • Am I more effective in the morning, afternoon or night?
  • Does spending longer on the computer help me get more done?
  • How often should I take a break? For how long?
  • What websites waste my time the most? More importantly, does spending my time on these sites make me feel happy or miserable?
  • How much does exercise do I do? What has the most effect on my weight? What about on my mood?
  • What habits should I create to make me happier?

That's a lot of questions, but in essence what I really wanted to know was:

What can I do to be happier?

Perhaps that question is a little too big to be answered by a bunch of numbers and charts,

I tried to take an approach that took work and play into account, rather than just trying to wring another few minutes out of my day in the name of "productivity".

Answering questions

Most of my work is based on the computer, so I naturally went for a technical solution to all of this. To answer most questions, I needed to track the following:

  • What I'm doing on the computer
  • My todo items (scheduled + completed)
  • Exercise + diet
  • My mood

Tracking computer usage

I've used RescueTime in the past to track time spent on the computer. Although the Linux client doesn't get much love these days, it works well enough once it's setup. RescueTime automatically collects statistics silently in the background, so it's a pretty frictionless way of keeping tabs on things.

The RescueTime API provides plenty of ways to access information. The free version doesn't give access to document titles, so you can't track which websites you spend time on, but there's still enough information to build a good profile of where your time is spent.

If you use Google Chrome as your browser you can access the history database which is stored using SQLite. You still need to categorize sites into positive/negative productivity, but it's a start. Chrome History in R is a good place to start if you're interested in taking that route. I don't currently use it to build any stats, but I may integrate it at some point.

Tracking TODO items

Of all the to-do software I've used over the years, I've found Emacs + Org Mode suites my workflow the best. The learning curve is pretty steep, and it's certainly an acquired taste, but it's extremely powerful and configurable. It's probably the closest thing I've found to pen + paper, but with all the benefits of being digital (searchable, queryable etc).

Emacs itself is incredibly flexible, and Org Mode can generate summaries of time worked, along with a diary of work for each day. It can be extended with a little Lisp knowledge, although I use Perl for extracting detailed information as it has a few libraries for querying org files (and my Lisp knowledge is pretty poor).

Tracking exercise + diet

This one was a little trickier. There's no shortage of exercise tracking software, and wearable trackers are becoming even more prevalent. When I first started a regular fitness programme I used Traineo to track things. It's pretty nice to use, but there's no API which meant I couldn't access the data.

In the end I opted for the hacker approach and rolled my own solution. It didn't need to do anything particularly complex, just track exercise done, the times and calories burned. Org mode supports adding properties to any headline, so it can be used as an ad-hoc database.

With a little bit of tweaking it possible to just hit a few keystrokes and automatically insert your exercise.

Tracking my diet was the done in the same way. With a f keystrokes I can enter any data I need.

A key point was NOT to make things too complicated, but to give enough data so things can be charted.

Tracking mood

As with the exercise planning, Emacs was the winner here. The biggest problem with tracking mood is it can be quite difficult to quantify. It's also relative to how I've recently felt - much like temperature. 70f feels cold in summer but warm in winter.

To begin with I tracked mood as a simple 1 - 10 scale, with 1 being miserable and 10 being happy, but after a few days it became clear this wasn't going to be as useful as I thought.

There's a lot more to be said about this, but for now I use two measurements: positive/negative mood and relaxed/stressed.

Tools Used

I tried to keep things based around my current workflow. I use emacs org-mode for most of my project management, so most of my manually tracked data is done with org-mode.

Turning all the data into something human readable is a bit of a Rube Goldberg experience. I use a Perl script to extract info from my org files so they can be stored in an Sqlite database. A PHP script then generates the various HTML files for monthly & yearly reports. It's not as elegant as I'd like, but it only takes ~60 seconds to generate all the reports and deploy them, so for now it's good enough.

For more information on the build process, check out "how it's made".

Publishing my Stats

You can view all my statistics from January 2012 onwards in the "bits" section.

New Guide: Personal Development Blogs List

I've just added a new guide to the site: Personal Development Blogs List. It's a list of popular personal development blogs, ordered by their Alexa rank. I've been pretty liberal with what can go on the list, and any blog that focuses on helping other people to achieve more is a good fit.

At the moment there are just under 30 blogs on the list, but I hope to grow that over time.

If you have a blog you'd like added to the list, just leave a comment and I'll update things.

New "Guides" Section

I've recently updated the site and added a new section for guides. I'm not completely sure of what will end up there but at the moment the section contains two guides:

  • Emacs Extensions – A list of useful extensions for Emacs. Grouped into categories for writers, programmers and general usage.
  • Pomodoro Apps – A visual guide of pomodoro apps. Not much going on at the moment, but there's a hefty list of apps to add at the moment.

The resources section has also been given a minor facelift, and each printable now has its own page with a brief description and much larger preview image.

Beeminder Postmortem

I've been using Beeminder for about 3 weeks, and I finally fell off the road for my "writing" goal last night. I still have two other goals, so it's not all doom and gloom.

One thing I've learnt is that quantifying creative goals by time does not work well for me. I originally set up the goal because I wanted to write more, but it didn't really have the desired effect. I'd find myself rewriting things just to meet the time quota, which wasn't really productive.

I usually only write when I'm in the mood, and it can be a couple of days before inspiration hits. Forcing myself to write for 30 minutes every day seemed like a good idea at the time, but in the end just led to me scrambling for ideas and producing very little of value.

Still, Beeminder itself works really well. It's encouraged me to keep my email under control, and I've spent way more time working on personal projects than I used to. I'm still trying to come up with a good system for tracking personal & work goals, but I'll probably go with a points-based approach.

Lightning vs the Lottery

The 30th of March, 2012 saw the biggest lottery jackpot in World history. The pre-tax total was approximately 640 million dollars, and despite the odds of winning being rather narrow, nearly 1.4 billion dollars was spent on lottery tickets 1.

Whenever there is a large lottery jackpot, the same crop of articles always appear, such as:

  • What you could buy with $X million
  • How to pick your numbers
  • 10 things more likely to happen than winning

I'm going to focus on the last one, because the statistics are usually wrong (or rather, used incorrectly).

The odds of winning the Mega Millions jackpot are 1 in 179 million. That means that there are 179 million different number combinations that can occur. If you buy one ticket, there us a 1 in 179 million chance your numbers will come up. Buy two tickets (with different numbers), and the odds are halved. Nothing too complicated.

Normally there's something about how much more likely it is that you'll be struck by lightning than win the jackpot. The odds of being hit by a lightning strike in the US are 1 in 10,000 2. This makes it sound much more likely, but these odds are spread over an average lifetime (80 years) rather than a single event (a lottery drawing).

A slightly more realistic set of odds would be: if you buy a lottery ticket, what are the chances of you being struck by lightning before the draw compared to winning the jackpot?

Odds of being struck in a year 1/775,000
Time between lottery draws 3 4 days
Odds of being struck in 1 day 1/282,875,000
Odds of being struck in 4 days 1/70,718,750

Assuming you buy a ticket as soon as the previous draw ends, you have nearly a 1 in 71 million chance of being hit by lightning. Of course, this doesn't take into account the current season or state you live in (people in Florida are twice as likely to be killed or injured than people in Michigan 4).

So, in order to be more likely to win the jackpot than being zapped by Zeus, just buy two tickets.



According to ABC news (What $1.46B Spent on Mega Millions Could Buy), Americans spent nearly $1.5 billion on lottery tickets.


The Mega Millions lottery is drawn every Tuesday and Friday, which gives 4 days between the Friday and Tuesday draw.