Lessons from 11 years of freelancing

I've been working as a freelance software engineer for just over 11 years now. My first contract came through Upwork (which was called oDesk back in 2009) and I charged $10 an hour. All of my early work was for individual bloggers or small businesses, and usually involved building or tweaking WordPress themes and plugins.

Since then I've been able to help dozens of different organizations and work on some really interesting projects. It's not quite the career I had in mind, but I enjoy the variety of work that comes my way.

Here's what I've learned.

It takes time

It took me about two years for my income to stabilize. The first year I could often go a week or two without any money coming in. This situation improved once I found a few steady clients, most of which came through referrals. After three years I was able to pay all of my bills (rent/food/etc) entirely through freelance work.

Use what you know

There is a time and a place to experiment with new technology. It's not when you're doing billed work for a client.

Charge more

This is a pretty common refrain in freelance circles, but it's still good advice. Raising prices does several things:

  • Signals that you're serious about what you do.
  • Weeds out clients that you may not want.
  • Allows you to be more selective about what you do.

It took me a long time to build the experience - and courage - to increase my rates. I could have been self-sustainable much earlier if I'd been brave enough to charge more.

Sometimes people won't pay

This has only happened a few times, but it's extremely frustrating when it does. These days I won't do any work without signing a contract first and being very clear about what is expected from both parties.

Not all clients are a good fit

This doesn't mean that they're bad people, just that their work styles don't mesh with mine. It's usually better to cut things early rather than trying to make things work.

Don't rely on a single client

After my first year I stopped looking for new projects and took all of my work from the same client. When they decided to switch to another company my income dried up overnight. There's no severance pay when you're a freelancer.

Communicate. Communicate more

Early on I developed a very bad habit where I wouldn't show what I'd done until it looked perfect. This sounds like a logical approach, but it causes all kinds of problems:

  1. I know that I've been working on something, but the client doesn't know that unless I tell them. They are paying for my services, and they want to see what they're getting.
  2. Often what they originally asked for isn't quite what they want. Showing things early gives time for feedback.
  3. It's not my place to decide what's perfect for the client.

I solved all of these by communicating clearly. I don't particularly like writing emails, but they're an important part of my work.

You need to learn a lot of non-work skills

Keeping accurate records, sending invoices, paying estimated taxes (and finding out who to pay), and filing taxes are all things I've had to learn since I started.

I keep a single spreadsheet for all of my financial information, and I store any printed receipts for the year in a manila folder. Once taxes are done, finances get printed and put in the filing cabinet. It's not glamorous, but it means I have an actual record if I need it.

Hint: scribbling "paid $xxx" on a post-it note is not good bookkeeping.

It can be expensive

My experience is based on living in the US, but I've found freelancing to have a lot of hidden costs. I have to pay health insurance, pay into social security (including the employer contribution), pay into a retirement account, and I don't get paid holidays.

Non-billable tasks take a lot of time

Hours of time can quickly get eaten by emails and administrative tasks. There are also non-work things like eating, drinking, and bathroom breaks that take time. Take these into account.

You're on your own

Unless you hire outside help, everything that needs doing rests on the shoulders of one person: you. This includes planning projects, scheduling work days, writing proposals, sending invoices (and following up on them), setting up meetings, and then actually doing the work you're being paid for on top of all that.

It's a lot.

Take all of this advice with a grain of salt; there's a hefty chunk of survivorship bias thrown in.


Post or Pay

Post or Pay

Aleix contacted me about their new service, "Post or Pay":

Post or Pay is a website that allows bloggers to put a price to not blogging. The user sets a deadline for her blog post and a price. If she doesn't publish it on time, she pays.

20% of the payment goes to charity (the World Literacy Foundation).

I use Beeminder for tracking my blog posting. As long as I post frequently enough, I'll stay on target and not get billed. It's working well for my 30 Days of Blogging trial, although it wasn't totally painless to set up - I had to hook up my RSS feed to Zapier in order to automate data submission.

Post or Pay works a little differently; you commit to writing a single post by a specific date, and then send proof once it's published. It's a much more focused approach.

If you're struggling to blog more, or you're looking to dip your toes into the world of commitment services, Post or Pay might be a good place to start.


Time confetti

A couple of days ago I read an article on the concept of "Time Confetti", about how our leisure time is slowly being fragmented into smaller pieces throughout the day:

I (and other researchers) call this phenomenon time confetti (a term coined by Brigid Schulte), which amounts to little bits of seconds and minutes lost to unproductive multitasking. Each bit alone seems not very bad. Collectively, though, all that confetti adds up to something more pernicious than you might expect.

Source: Time Confetti and the Broken Promise of Leisure

I'm certainly guilty of letting myself get distracted like this. I'll often take a short break after completing a work-related task; they don't give me enough time to really relax, but they add up over the course of the day. By the end of the day I'll have spent a couple of hours "relaxing", but I'll feel no better for it.

One solution to this is using an unschedule from "The NOW Habit". The idea is to create a schedule for the day, but to add fun activities first (rather than work tasks). I've tried it in the past with some success, but over time my schedule started skewing more heavily towards work. Eventually it ended up being entirely work-focused with tiny breaks scattered in ad-hoc.

I think now might be a good time to try it again.


The end of fun

I've been messing around with C++ in preparation for some of my 2021 goals. I haven't actually written anything in C++ since I was in university, and given the quality of what I wrote I'm surprised I graduated at all.

But what I've been trying for the last few days has been a lot of fun. Maybe it's because I'm writing what I want instead of code to pass an assignment, or maybe it's because there's a familiarity to it so I feel like I'm learning quicker.

I really enjoy trying to code up an idea and then seeing it come to life. Even after 25+ years of writing code, I still get enjoyment from seeing the ideas in my head becoming a reality.

But at some point I get a little carried away and try to create something outside of my abilities. And then the fun stops.

In my recent C++ adventure I managed to get a little pixel sprite moving around, all working on the Atari ST. That's something I never thought I'd be able to do.

But already my mind is thinking 200 steps ahead. Because the tool allows me to create something good, surely I must use it to create something good. I think this is similar to the problem I have with art supplies; part of my brain thinks better tools will improve my skills, rather than the other way around.

Even if I focus on something small, eventually I get to a point where working on it just feels bad. I'm not sure if that means I only enjoy the first 20% of a project, or if it's something deeper.

My hunch is that I enjoy the first part because it's all about discovery. I get to learn new things, and there's usually a quick turnaround between trying something and seeing it work. When I'm learning something new, I'm only concerned about the result, not how it's achieved. I allow myself to fail multiple times until it works.

Once I move past learning new things, everything changes. I start bumping up against areas where my skillset isn't so good. I have to make design decisions. Things start breaking and my motivation tanks. So I put it aside and learn something else, but all that gives me is a hard-drive full of half-baked prototypes. That's not what I want.

So where does that leave my little pixel sprite? My normal approach is to make a small prototype and then slowly grow it into the (un)finished article. I think this time I'll take a more practice-oriented approach, testing out new ideas and techniques separately, before building on that experience to create something new.


Filtering referral spam from my server logs

Note: I have updated and improved this system. See "Filtering referral spam from my server logs, part 2".

A few years ago I removed all third-party analytics software from my websites and went back to using plain server logs. I use goaccess for viewing server statistics; it's fast and flexible, and it can show information on the command line or generate HTML reports.

One problem is that some sites will send bogus requests with their url as the Referer value. They then show up in logs which makes it much harder to find useful information:

Referer spam ruins everything

There are a couple of solutions I considered:

Blocking the referrals directly from nginx
This would stop them for ever reaching the logs, but would require frequent updating of the server configuration. I was also a little concerned that adding hundreds of blocked urls could slow things down slightly.
Excluding the urls via .goaccessrc
goaccess can already exclude referrals, but there is a hard limit of 64 without recompiling. It also requires the list of urls to be regularly updated.
Filtering the logs before parsing
This adds an extra step, but gives me more flexibility in what is removed.

In the end I went for option #3, as I wanted to try and remove requests that matched a certain pattern instead of filtering individual domains or IP addresses.

Most of the spam referrals I see have a few things in common:

  • They only hit the homepage.
  • They're using an old url, so the request gets redirected.
  • They use HTTP/1.1.

This makes it a little easier to filter them out without using their domain.

GNU/Linux has a number of text processing tools, but grep was the simplest option as I'm ignoring a single string from each line.

The following snippet of bash code removes most spammy requests from the access log and then pipes it into goaccess:

#!/bin/bash
grep -v '\"GET / HTTP/1.1\" 301 194' access_log | goaccess -a

The finished result looks like this:

Referer spam is gone

It's a bit of a scattergun approach and there's a risk that I'll exclude legitimate traffic from the log. However, I'm more interested in referrals to individual posts (which I use to see what people like/don't like) so I'm not too worried.

There are still a few spam sites getting through which use HEAD to request the homepage. I made a few tweaks to exclude them as well, so the final version looks like this:

#!/bin/bash
grep -v -e '\"GET / HTTP/1.1\" 301 194' \
	-e '\"HEAD / HTTP/1.1\"' access_log \
    | goaccess -a

It's not perfect, but so far it's cut out 99% of the referral spam I was seeing.