pn_net

Time confetti

Posted:

A couple of days ago I read an article on the concept of "Time Confetti", about how our leisure time is slowly being fragmented into smaller pieces throughout the day:

I (and other researchers) call this phenomenon time confetti (a term coined by Brigid Schulte), which amounts to little bits of seconds and minutes lost to unproductive multitasking. Each bit alone seems not very bad. Collectively, though, all that confetti adds up to something more pernicious than you might expect.

– Source: Time Confetti and the Broken Promise of Leisure

I'm certainly guilty of letting myself get distracted like this. I'll often take a short break after completing a work-related task; they don't give me enough time to really relax, but they add up over the course of the day. By the end of the day I'll have spent a couple of hours "relaxing", but I'll feel no better for it.

One solution to this is using an unschedule from "The NOW Habit". The idea is to create a schedule for the day, but to add fun activities first (rather than work tasks). I've tried it in the past with some success, but over time my schedule started skewing more heavily towards work. Eventually it ended up being entirely work-focused with tiny breaks scattered in ad-hoc.

I think now might be a good time to try it again.

Posted in: 30 Days of Blogging, Time Management | Comments (0)

The end of fun

Posted:

I've been messing around with C++ in preparation for some of my 2021 goals. I haven't actually written anything in C++ since I was in university, and given the quality of what I wrote I'm surprised I graduated at all.

But what I've been trying for the last few days has been a lot of fun. Maybe it's because I'm writing what I want instead of code to pass an assignment, or maybe it's because there's a familiarity to it so I feel like I'm learning quicker.

I really enjoy trying to code up an idea and then seeing it come to life. Even after 25+ years of writing code, I still get enjoyment from seeing the ideas in my head becoming a reality.

But at some point I get a little carried away and try to create something outside of my abilities. And then the fun stops.

In my recent C++ adventure I managed to get a little pixel sprite moving around, all working on the Atari ST. That's something I never thought I'd be able to do.

But already my mind is thinking 200 steps ahead. Because the tool allows me to create something good, surely I must use it to create something good. I think this is similar to the problem I have with art supplies; part of my brain thinks better tools will improve my skills, rather than the other way around.

Even if I focus on something small, eventually I get to a point where working on it just feels bad. I'm not sure if that means I only enjoy the first 20% of a project, or if it's something deeper.

My hunch is that I enjoy the first part because it's all about discovery. I get to learn new things, and there's usually a quick turnaround between trying something and seeing it work. When I'm learning something new, I'm only concerned about the result, not how it's achieved. I allow myself to fail multiple times until it works.

Once I move past learning new things, everything changes. I start bumping up against areas where my skillset isn't so good. I have to make design decisions. Things start breaking and my motivation tanks. So I put it aside and learn something else, but all that gives me is a hard-drive full of half-baked prototypes. That's not what I want.

So where does that leave my little pixel sprite? My normal approach is to make a small prototype and then slowly grow it into the (un)finished article. I think this time I'll take a more practice-oriented approach, testing out new ideas and techniques separately, before building on that experience to create something new.

Posted in: 30 Days of Blogging | Comments (0)

Filtering referral spam from my server logs

Posted:

A few years ago I removed all third-party analytics software from my websites and went back to using plain server logs. I use goaccess for viewing server statistics; it's fast and flexible, and it can show information on the command line or generate HTML reports.

One problem is that some sites will send bogus requests with their url as the Referer value. They then show up in logs which makes it much harder to find useful information:

Referer spam ruins everything

There are a couple of solutions I considered:

Blocking the referrals directly from nginx
This would stop them for ever reaching the logs, but would require frequent updating of the server configuration. I was also a little concerned that adding hundreds of blocked urls could slow things down slightly.
Excluding the urls via .goaccessrc
goaccess can already exclude referrals, but there is a hard limit of 64 without recompiling. It also requires the list of urls to be regularly updated.
Filtering the logs before parsing
This adds an extra step, but gives me more flexibility in what is removed.

In the end I went for option #3, as I wanted to try and remove requests that matched a certain pattern instead of filtering individual domains or IP addresses.

Most of the spam referrals I see have a few things in common:

  • They only hit the homepage.
  • They're using an old url, so the request gets redirected.
  • They use HTTP/1.1.

This makes it a little easier to filter them out without using their domain.

GNU/Linux has a number of text processing tools, but grep was the simplest option as I'm ignoring a single string from each line.

The following snippet of bash code removes most spammy requests from the access log and then pipes it into goaccess: 

#!/bin/bash
grep -v '\"GET / HTTP/1.1\" 301 194' access_log | goaccess -a

The finished result looks like this:

Referer spam is gone

It's a bit of a scattergun approach and there's a risk that I'll exclude legitimate traffic from the log. However, I'm more interested in referrals to individual posts (which I use to see what people like/don't like) so I'm not too worried.

There are still a few spam sites getting through which use HEAD to request the homepage. I made a few tweaks to exclude them as well, so the final version looks like this: 

#!/bin/bash
grep -v -e '\"GET / HTTP/1.1\" 301 194' \
	-e '\"HEAD / HTTP/1.1\"' access_log \
    | goaccess -a

It's not perfect, but so far it's cut out 99% of the referral spam I was seeing.

Posted in: 30 Days of Blogging, Tools | Comments (0)

Writing a Leanpub book with Emacs

Posted:

Leanpub has a robust web interface, but it seemed appropriate to create "Writing PHP with Emacs" using Emacs itself. This is how I do it, from managing ideas to publishing the content itself.

Managing ideas

Before I started writing the book I had a single text file that I collected ideas in. This worked okay in the beginning, but once things started moving along it became unwieldy to manage.

I switched to keeping my notes in zetteldeft which made life much easier. There is a single note titled "Writing PHP with Emacs" which I use as an entry point. All related notes are linked to this one, and I also tag them with #php-with-emacs so I can quickly search for them.

Not all of these notes are directly related to the book, but they might contain information that I want to include in the future.

Planning

The entire book plan is stored in a single TODO.org file in the project directory. I use it to keep a list of the overall contents, as well as planning out individual chapters.

The plan looks a little like this (with content removed for brevity): 

#+TITLE: PHP with Emacs - TODO List
#+SEQ_TODO: TODO STARTED TO-EDIT | COMPLETE POSTPONED CANCELLED 

* TODO Milestones [4/5]
** COMPLETE Version 0.2
   CLOSED: [2020-06-09 Tue 19:55] DEADLINE: <2020-05-31 Sun>
*** COMPLETE php-mode: Customization - Using TAGS files [3/3]
    CLOSED: [2020-05-30 Sat 10:26] SCHEDULED: <2020-05-28 Thu>
    :PROPERTIES:
    :EFFORT:   1:00
    :END:
    Prefer to use php-lsp, but this is an option too.
     - [X] What TAGS files can be used for
     - [X] Generating
     - [X] Navigating using a TAGS file
       Shortcuts and everything go here.
* TODO Backlog [1/22]
** TODO Configuring a project for WordPress
** TODO Configuring a project for Drupal

I use org-columns for quickly viewing the list of milestones. From this view I can set estimates and view time worked on each milestone and sub-task.

My book plan using column mode

The Backlog heading keeps a list of all future content that I want to work on. When it's time to move an item to the plan, I'll refile it to a milestone using a custom function called pn/org-refile-in-file: 

(defvar pn/org-last-refile-marker nil "Marker for last refile")

(defun pn/org-refile-in-file (&optional prefix)
  "Refile to a target within the current file."
  (interactive)
  (let ((helm-org-headings-actions
	 '(("Refile to this heading" . helm-org--refile-heading-to))))
    (save-excursion
      (helm-org-in-buffer-headings)
      (org-end-of-subtree t)
      (setq pn/org-last-refile-marker (point-marker)))))

I use this instead of org-refile as I have a lot of org files that I don't want to be used as targets. Once a headline has been moved to a milestone I'll add a time estimate and schedule it. I use org-projectile so any scheduled tasks in the book TODO show up in my agenda.

Organizing files

The project directory contains the manuscript directory, which is used by Leanpub to generate the book. It also contains a lightweight Emacs user directory which I use when generating screenshots.

I use binder to keep a list of manuscript files. binder also supports adding file-specific notes, although I prefer to keep things in my TODO file.

Using binder to view the book

For most navigation I'll use projectile, but binder is helpful when trying to get an overall look at the book or when I'm jumping between chapters.

Writing

Leanpub has its own offshoot of Markdown called Markua, so everything is written and edited using markdown-mode+. I would have preferred to write in org-mode - and there is an ox-leanpub exporter - but I didn't want to add an extra step to the write -> publish process. Maybe next time.

One of the things that caught me out at the start was line breaks. Markdown will normally ignore a single line break, so hard-wrapped lines will still be treated as a single line. Markua treats them as hard line breaks, so it altered the look of the book and made pages too narrow. I switched to visual-line-mode which wraps lines visually but doesn't alter the text.

I use a .dir-locals.el file to automatically enable specific modes and to increase the font size. 

((markdown-mode . ((eval . (progn (turn-off-auto-fill)
				  (text-scale-set 1)
				  (turn-on-olivetti-mode)))
		   (fill-column . 80)
		   (visual-fill-column-width . 80)
		   (mode . flyspell)
		   (mode . binder)
		   (mode . whitespace-cleanup)
		   (mode . visual-line)
		   (mode . visual-fill-column))))

The modes I use are:

  • flyspell – Enables real time spell-checking in the buffer.
  • binder – Enables binder navigation for moving forwards and backwards between chapters.
  • whitespace-cleanup – Strips trailing whitespace when saving a buffer.
  • visual-line – Enables soft word-wrapping.
  • visual-fill-column – Adds word-wrapping at a specific column, instead of the window's edge.

The code in the eval block turns off hard-wrapping, increases the font size, and enables olivetti-mode. olivetti centers the text in the current window which I prefer when writing words.

For taking screenshots, I have a separate Emacs configuration that contains just PHP and web-specific packages. It loads only the packages I want, and sizes the window/fonts to a width that fits on the page. Startup looks like this: 

emacs --no-init-file --load "emacs-env/init.el"

The --no-init-file= option tells Emacs to ignore my usual init.el file, and then I manually load the screenshot initializer.

Publishing changes

The "standard" Leanpub plan allows the use of git for managing book files. I use git every day, so the workflow is something I'm very accustomed to. The master branch contains content that will be published, and WIP content is stored in a named branch (usually something like add-XYZ-chapter).

Once content is ready I'll push it to the remote repository and then manually publish it from within Leanpub. There is an API, but that's reserved for the "PRO" plan.

I have two Beeminder goals set up for the book:

  • write-php-book – This goal is updated any time I make a commit to the book repository. This makes sure I'm writing often enough to get things done.
  • publish-php-book – Leanpub can notify a webhook when a new version is published, so I use this to add a data point to Beeminder. This makes sure I'm publishing changes I write instead of waiting until things are "perfect".

I originally started with a word count goal. This worked well when I was trying to write all of the initial content, but once I started editing it made more sense to change to a goal that didn't focus on raw words.

Posted in: 30 Days of Blogging, Emacs, Writing | Comments (0)

Reducing toil

Posted:

I recently read Google's advice on Eliminating Toil. Although the guidelines are aimed at Google's web services, I thought it was useful to apply in other places too.

But first things first, what is toil?

Toil is the kind of work tied to running a production service that tends to be manual, repetitive, automatable, tactical, devoid of enduring value, and that scales linearly as a service grows.

I think using the word "toil" for these tasks is a little bit of hyperbole, but the idea is sound. There are tasks that I perform on a regular basis that I can automate or eliminate with some thought.

Things I have done

Some of these are really small.

Created elisp functions for writing notes
There are a couple of projects where I keep update logs in a Markdown file. I wrote some small elisp functions to add timestamped headings, so that I can add entries with a shortkey instead of opening the file and typing it in.
Digitized my diet and exercise tracking
I used to keep these all on paper, but I brought things into Emacs which is much quicker.
Automated blog regeneration
The old version of this site needed building through a script, and then the built files had to be uploaded. I've changed this so that adding a post to the blog automatically rebuilds it, and the site is rebuilt every hour so that I can schedule posts. This is something I'll write about in the future.
Wrote some small scripts for fetching error logs
Super small, logging into a machine, cd'ing into the log directory, then running tail can all be done with a single script. Helpful when I get downtime emails.
Tracked how I work on a typical project and looked for rough edges
Some of these were incredibly minor. Things like:
  • Finding quicker ways to skip between text in an Emacs file.
  • Learning how to jump to a function's definition instead of searching for it.
  • Wrote auto-packaging for MaxCop. Manually creating a zip file and adding three files isn't a huge burden, but automating things makes creating releases much easier. It's also something I can reuse for other projects.

More to do

There are still plenty of places where I can make improvements, and I think I'll have to spend some time really looking at my workflows.

A few things I can think of right now:

  • Make use of snippets for inserting boilerplate code.
  • Automate adding tasks to FacileThings when they come in via email.
  • Create a repeatable checklist when processing each inbox item. Having a simple flowchart would reduce some of the mental overhead.
  • Automate downloading and parsing of data sources that I normally do by hand:
    • RunKeeper archives
    • Bank csv files
    • Calorie information for foods

Making these improvements takes some time, but sharpening the saw is an important habit to develop. I'm pretty confident that the effort I've put into to optimizing other areas has already paid off many times over.

Posted in: 30 Days of Blogging, Productivity | Comments (0)