Ever since I read about "The Printable CEO", I've experimented with keeping track of some personal metrics. Whilst the PCEO allowed me to keep a rough score of how much work I was doing, I always felt that with a little tweaking it could answer much more detailed questions.
Specifically, I wanted to be able to answer questions like:
- What am I really spending my time doing?
- Where am I wasting the most time?
- What parts can I remove or automate to make me more efficient?
Once I started thinking about what I'd like to know, my brain started throwing out way more questions:
- Am I more effective in the morning, afternoon or night?
- Does spending longer on the computer help me get more done?
- How often should I take a break? For how long?
- What websites waste my time the most? More importantly, does spending my time on these sites make me feel happy or miserable?
- How much does exercise do I do? What has the most effect on my weight? What about on my mood?
- What habits should I create to make me happier?
That's a lot of questions, but in essence what I really wanted to know was:
What can I do to be happier?
Perhaps that question is a little too big to be answered by a bunch of numbers and charts,
I tried to take an approach that took work and play into account, rather than just trying to wring another few minutes out of my day in the name of "productivity".
Answering questions
Most of my work is based on the computer, so I naturally went for a technical solution to all of this. To answer most questions, I needed to track the following:
- What I'm doing on the computer
- My todo items (scheduled + completed)
- Exercise + diet
- My mood
Tracking computer usage
I've used RescueTime in the past to track time spent on the computer. Although the Linux client doesn't get much love these days, it works well enough once it's setup. RescueTime automatically collects statistics silently in the background, so it's a pretty frictionless way of keeping tabs on things.
The RescueTime API provides plenty of ways to access information. The free version doesn't give access to document titles, so you can't track which websites you spend time on, but there's still enough information to build a good profile of where your time is spent.
If you use Google Chrome as your browser you can access the history database which is stored using SQLite. You still need to categorize sites into positive/negative productivity, but it's a start. Chrome History in R is a good place to start if you're interested in taking that route. I don't currently use it to build any stats, but I may integrate it at some point.
Tracking TODO items
Of all the to-do software I've used over the years, I've found Emacs + Org Mode suites my workflow the best. The learning curve is pretty steep, and it's certainly an acquired taste, but it's extremely powerful and configurable. It's probably the closest thing I've found to pen + paper, but with all the benefits of being digital (searchable, queryable etc).
Emacs itself is incredibly flexible, and Org Mode can generate summaries of time worked, along with a diary of work for each day. It can be extended with a little Lisp knowledge, although I use Perl for extracting detailed information as it has a few libraries for querying org files (and my Lisp knowledge is pretty poor).
Tracking exercise + diet
This one was a little trickier. There's no shortage of exercise tracking software, and wearable trackers are becoming even more prevalent. When I first started a regular fitness programme I used Traineo to track things. It's pretty nice to use, but there's no API which meant I couldn't access the data.
In the end I opted for the hacker approach and rolled my own solution. It didn't need to do anything particularly complex, just track exercise done, the times and calories burned. Org mode supports adding properties to any headline, so it can be used as an ad-hoc database.
With a little bit of tweaking it possible to just hit a few keystrokes and automatically insert your exercise.
Tracking my diet was the done in the same way. With a f keystrokes I can enter any data I need.
A key point was NOT to make things too complicated, but to give enough data so things can be charted.
Tracking mood
As with the exercise planning, Emacs was the winner here. The biggest problem with tracking mood is it can be quite difficult to quantify. It's also relative to how I've recently felt - much like temperature. 70f feels cold in summer but warm in winter.
To begin with I tracked mood as a simple 1 - 10 scale, with 1 being miserable and 10 being happy, but after a few days it became clear this wasn't going to be as useful as I thought.
There's a lot more to be said about this, but for now I use two measurements: positive/negative mood and relaxed/stressed.
Tools Used
I tried to keep things based around my current workflow. I use emacs org-mode for most of my project management, so most of my manually tracked data is done with org-mode.
Turning all the data into something human readable is a bit of a Rube Goldberg experience. I use a Perl script to extract info from my org files so they can be stored in an Sqlite database. A PHP script then generates the various HTML files for monthly & yearly reports. It's not as elegant as I'd like, but it only takes ~60 seconds to generate all the reports and deploy them, so for now it's good enough.
For more information on the build process, check out "how it's made".
Publishing my Stats
You can view all my statistics from January 2012 onwards in the "bits" section.