How I get work done with Emacs and org-mode

I've been a freelance developer for 12 years, and Emacs has been at the center of my working environment for 10 of those. I use it for a lot of things - including writing this post - but today I'm concentrating on my workflow.

Here's how I use Emacs and org-mode during a regular day.

The packages

There are two of packages that are central to my daily routine: org-mode and projectile.

org itself is a text markup language with a lot of extras that make it great for organizing work. org-mode allows Emacs to extract this information and use it for scheduling tasks, estimating task effort, tracking time, and a whole bunch of other things. I remember reading that org is a gateway drug that leads to Emacs - that was certainly the case for me.

projectile adds functions for navigating between projects and files. A project in the world of projectile is a file directory, which is usually a git working directory in my case. In a typical day I can work on a dozen different projects, so being able to quickly jump between them is extremely helpful.

I use a lot of other packages when doing actual work, but these two keep me organized and allow me to quickly get to where I need to be.

The setup

All of my org files live in "~/Documents/org/" and are synchronized between my desktop and laptop.

Each client has their own .org file with headlines for each project that I'm working on. I also have inbox.org which I use for capturing new tasks, and goals.org which contains my monthly goals.

Major personal projects also have their own org files, although I'm slowly transitioning to per-project "TODO.org" files instead.

All of these files are added to the org-agenda-files list so that they show up in my org agenda.

;; For Emacs versions < 25
(setq org-agenda-files
      (find-lisp-find-files "~/Documents/org" "\.org$"))

;; For version >= 25
(setq org-agenda-files
      (directory-files-recursively "~/Documents/org/" "\\.org$"))

Update: thank you to kaiwen1 on reddit for letting me know about directory-files-recursively.

The agenda

The org agenda is a powerful way to display tasks from across multiple files. Agendas can be filtered to a time period, urgency, project, and much more.

My daily agenda looks something like this 1:

An example org-agenda

This view has 5 columns that display the following information:

  1. Category - For freelance clients I use the client name, followed by the project. This looks something like client:project. Personal projects use just the project slug.
  2. Estimate - I try to keep individual tasks on this list to under 2 hours. If it's longer than that, it probably means I haven't broken it down enough.
  3. Scheduled date or deadline - When this is task is due or if I've scheduled it for a particular day.
  4. Task name - This contains the task status, its priority, and its name.
  5. Tag - This is used to identify project and client tasks, and also to store my monthly goals. I used to use this for GTD-style contexts, but seeing as 99% of my work is at the computer I didn't get much use from it.

The configuration for these columns is fairly simple:

(use-package org-agenda
  :after org
  :custom
  (org-agenda-prefix-format '((agenda . " %i %-20:c%?-12t%-6e% s")
			      (todo   . " %i %-20:c %-6e")
			      (tags   . " %i %-20:c")
			      (search . " %i %-20:c"))))

And the actual agenda configuration (without the code to display my monthly goals) looks like this:

(setq org-agenda-custom-commands
      '(("d" "Today's Tasks"
	 ((agenda "" ((org-agenda-span 1)
		      (org-agenda-overriding-header "Today's Tasks")))))))

To view my daily agenda, I'll run M-x org-agenda (which I have bound to C-c a) and then select d for the "Today's Tasks" view. It shows me everything I have scheduled for the day, as well as due and upcoming deadlines.

The day plan

At the start of the day I'll open my agenda and choose which tasks to work on, based on due date and priority. I'll then block out time to work on them on my paper day plan. I'll always leave some extra room for tasks that will pop up during the day.

The day planner itself is really simple and looks a bit like this (shortened to save space):

  Monday Tuesday Wednesday Thursday Friday
9-10 Task A        
10-11 Task B        
11-12 Task C        
12-1 Lunch        
1-2          
2-3 Task C        

I split my work day into blocks of one hour to make things easier; I've tried slots of 30 minutes long, but that felt less flexible and quickly fell apart if something came up. I use red for billable tasks, green for personal projects, and orange for essentials like exercise and eating.

Although org-agenda can be used to create day plans that are similar to this, I prefer having a paper copy in front of me. It lets me get a quick overview of how busy the day is going to be, as well as how much is going to be spent on billable time vs personal projects.

Adding new tasks

I generally deal with two kinds of task: proactive, and reactive.

Proactive tasks are things I've already planned out; I can choose when to work on them, and I know when they're due and what urgency level they are.

Reactive tasks are when emails/calls/chat messages come in and I need to do something about them. They may be urgent and need doing ASAP, or they may end up on the proactive list.

We'll start with reactive tasks first.

An example non-urgent request would be something like "We're wanting to add a new report to <product> by next month. Design and documentation attached"2.

I'll open up the org file for this client, jump to the appropriate project node, and add a new sub-task for this request. In this example it would look a bit like this:

* TODO Create new "days without an accident" report [0/4]
  DEADLINE: <2021-04-30 Fri>
** TODO Email person re: new report
   SCHEDULED: <2021-04-15 Thu>
** TODO Create markup for new report
** TODO Add data backend to new report
** TODO Deploy new report to staging for tests

I'd also break down each sub-task into individual steps and add estimates, but that's a topic for another day.

Once this is done, the first task will show up on my daily agenda. My weekly agenda will also have a reminder that there's an upcoming deadline.

For scheduling proactive tasks I'll read through my list of tasks and assign dates as needed. org-agenda can search for top-level TODO items and filter them by category and context. It can also list "stuck" projects that don't have an assigned next action.

Working on a task

When it's time to work on a task, I'll use org's clock timer to track the time I spend on it. Starting work looks like this:

  • I highlight the task in the agenda.
  • Hit <enter> to jump to its entry in the appropriate file.
  • Switch the task to IN-PROGRESS.
  • Start clocking time with C-c C-x C-i.
  • Navigate to the project with C-c p p.
  • Choose the file I need to be working on and start.
  • Use C-c C-x C-o to stop the clock once I'm done or need to switch.

I know all the previous steps sound like a lot, but going from my agenda to working on a task only takes a few seconds.

Emacs also supports bookmarks, so if I want to start on something immediately the next day, I'll leave a bookmark in the right spot so I can jump straight to it.

Tasks in org have a keyword at the start to indicate their status. I use the following keywords:

  • TODO - A task that I need to do.
  • IN-PROGRESS - A task that I've started and is actively being worked on.
  • TESTING - A task that is being tested and reviewed by the client.
  • WAITING - A task where I'm waiting on something - could be more information about what needs doing, or it could need an external party to review it.
  • TO-DEPLOY - It's done and ready and waiting to deploy at a specific date & time.
  • DONE - It is done and nothing needs doing on it again.
  • CANCELLED - The task/project was cancelled for one reason or another.

Because org-agenda can search for keywords, it's easy for me to check for various tasks during my weekly review. For example, I can check through the WAITING list and see if I need to follow up on anything.

I also use org for detailed project planning; it's great for breaking down a task into chunks, adding estimates, and assigning to milestones. But this post is already boring long enough so I'll save that for another day.

~~~
If you found this helpful and would like to read more content like it, follow me on Twitter.

Footnotes:

1

I've been doing this job for 12 years but it's really hard to come up with imaginary tasks.

2

At least with made up examples they include all the info I need first time.


Moving text around in Emacs using move-text

move-text is a simple Emacs package that lets you move a line or region around using M + and M + . It's not limited to plain text buffers - you can move lines of source code around too.

In practice it looks like this:

Moving text around

I prefer to use Control + Shift with the up and down arrow keys, so my configuration looks like this:

(use-package move-text
  :bind
  (("C-S-<up>"   . move-text-up)
   ("C-S-<down>" . move-text-down)))

Cutting and pasting a region is usually quicker, but for shuffling a few lines around this method is quick enough.


Groundhog Day Resolutions - April 2021

March was a very busy week for freelance development, so it was hard to make time for a lot of these.

March's primary goals

1. Finish version 1.0 of Writing PHP with Emacs

I added the final chapters on setting up Emacs for WordPress, Symfony, Drupal, and Laravel development. There are some more details I want to add, but the basics are done.

2. Complete another session of deliberate practice

I created a small animation test using GodLib, a C library for creating Atari ST games and demos. It was much closer

3. Finish version 0.3 of Craft Roulette

The final part of the "add craft" form is complete and everything is ready for production.

4. Contribute to a free software project

I contributed some changes to the BlitzMax website and I'm working on some additions to Zetteldeft.

March's secondary goals

1. Run 120 miles

One run was canceled due to storms, but outside of that I stuck to my plan. I don't feel fitter, but my times have improved from last year so something must be working.

2. Read a book

I didn't even pick up a book, let alone read one.

3. Add detail pages to my secondary goals

I added pages for a handful of goals, but not enough.

Primary goals for April

1. Create a playable prototype of something

There are a couple of little demos I've been working on and want to finish up. I'd like to release something playable in April.

2. Release the new version of Craft Roulette

It's ready for release, but I'll probably polish a few more bits and pieces before deployment.

3. Contribute to another free software project

I was working on a couple of extra changes this month, and there are some other projects I've seen that I'd like to contribute to.

Secondary goals for April

1. Run 115 miles

April is the peak of my long runs, followed by tapering before race day. I am ready for training to be over.

2. Plan version 1.1 of Writing PHP with Emacs

There are plenty of parts of the book I want to improve, and there are some new chapters I want to write. By next month I'd like to have a roadmap of future improvements.


Configuring bug-reference mode from within org-mode

A few weeks ago I learnt about bug-reference-mode, a neat little Emacs minor-mode that automatically links text to a bug tracker. A few of my projects use bug trackers or GitHub issues, and different clients often have their own systems.

There are two variables that I usually want to set for bug-reference-mode:

bug-reference-url-format
The URL of an individual bug, with %s used as a placeholder for the bug ID.
bug-reference-bug-regexp
The regular expression that finds bug ID references. This is usually something like #1234.

I use directory local variables1 for a lot of projects, but Emacs also supports file local variables for more fine-grained settings. org-mode has its own syntax for setting file local variables, so I needed to adjust things slightly to get it all working.

Setting local variables for a bug tracker looks like this:

# Local Variables:
# mode: org
# bug-reference-url-format: "https://example.org/tasks/%s"
# bug-reference-bug-regexp: "\\(#\\)\\([0-9]\\{8\\}\\)"
# End:

On my first attempt I skipped the Local Variables: and End: keywords, but they are needed for org to pick these variables up.

Footnotes:

1

These are variables that are set for all files in a directory and its sub-directories.


Extracting zetteldeft notes

I've been using zetteldeft to manage my notes for about six months now. Recently I've been wanting to reorganize my notes as some of them don't follow the zettelkasten principal of one idea per note.

Okay, most of them don't follow that principal if I'm honest about it.

I wanted to slice up my existing notes into smaller files and then link them all together. In my mind the process would go something like:

  1. Highlight the region to extract.
  2. Enter a title for the new note.
  3. Emacs would automatically move the text to the new note and insert a link where it used to be.

Emacs macros are handy for this kind of repeatable behaviour, so I started recording with an example note.

However, I quickly ran into a roadblock due to zetteldeft's behaviour when creating a new note. zetteldeft will create a file, insert the title, and save the file name to the Emacs kill ring. This meant any text I'd copied would be overwritten by the note title.

I decided to abandon the macro method and use a dedicated function instead. I wrote a little function called sodaware/zd-extract-region-to-note that extracts a highlighted region to a new note:

(defun sodaware/zd-extract-region-to-note (title)
  "Extract the marked region to a new note with TITLE."
  (interactive (list (read-string "Note title: ")))
  (let* ((deft-use-filename-as-title t)
	 (note-id (zetteldeft-generate-id title))
	 (note-filename (concat note-id zetteldeft-id-filename-separator title))
	 (note-text     (kill-region (region-beginning) (region-end))))
    (save-excursion
      (deft-new-file-named note-filename)
      (zetteldeft--insert-title title)
      (insert "\n\n")
      (yank)
      (save-buffer)
      (when (featurep 'evil) (evil-insert-state)))
    (sodaware/zd-insert-link note-filename title)))

(defun sodaware/zd-insert-link (note title)
  "Insert a link to NOTE with TITLE as its link text."
  (interactive)
  (insert (format "[[zdlink:%s][%s]]" note title)))

I use the zdlink protocol for my links, but the sodaware/zd-insert-link function can be modified to use any kind of link. Eventually I'd like this to be controlled via a variable.

I'd also like a function that returns focus to the original note after extraction, but for now I'm pretty happy with how things work.