Reflections on Software Engineering

One year of LLM usage with Clojure

2026-02-21T00:00:00Z

Introduction

This essay is a reflection on using LLM agents with the Clojure programming language. At Shortcut, we have spent the last year building Korey, an LLM agent focused on product management. During that time, we have attempted to use different LLM agents to work with our rather large Clojure code base, ~250,000–300,000 lines of Clojure code. In doing so, we discovered a lot. I hope this essay can help people in the Clojure community and in other unorthodox languages more generally take more advantage of LLM tools for their work. I would break our approach down into eras:

Early adoption and struggles
The doldrums of Claude Code
The clojure-mcp revolution
Skills, Prompts, and OpenCode
System prompt refinements
PI Agent and liberation

Early Adoption and Struggles

It is commonly understood that LLM models are more effective with languages that dominate the training dataset. Research has shown that model performance varies significantly based on the volume of training data for each programming language, with Python, JavaScript, and TypeScript being heavily represented in public code repositories. This fact is and was concerning to me. For one, at Shortcut we have a large Clojure code base that we have grown, nurtured, and maintained over the last 11 years. We don't have the time, money, or interest in rewriting this to Python or TypeScript simply because state-of-the-art models prefer these languages. Additionally, to me, it does not seem like a good business move to throw out working code.

With that in mind, we decided to try to teach Claude Code how to write Clojure code well. Quickly, we realized that certain aspects of the way Claude Code is structured by default make it very difficult to work with a large code base like ours. An example of this is that Claude will run the entire test suite after each thing it implements, and running the entire test suite locally on our machine takes several minutes at this point, unfortunately. As Clojure engineers, we prefer tight feedback loops on the REPL. This several-minute-long test suite run was unacceptable to us.

We began to tweak our AGENT.md file, teaching Claude Code about how Clojure and Clojure's data structures work. With that, we were able to narrow the static verification steps that Claude Code made after each point in its implementation process. We noticed large improvements in our performance and our iterative process at this point.

The doldrums of Claude Code

At this point we felt using Claude Code was mostly functional, and we were capable of achieving a certain level of development flow with Claude Code. Claude used our large code base itself as a model of how it should write Clojure code, although we often had to prompt it to do so. I began experimenting with the best ways of prompting the LLM agent. One of the things I discovered is that specifying in detail what the LLM agent should do is critical; you can't leave any ambiguity.

However, we still struggled with certain aspects of how Claude approached software engineering. For example, we noticed that Claude often leapt ahead before looking. Claude would write a bunch of code — potentially several thousand lines over a few minutes — and then attempt to verify it. The problem is that because of hallucinations or misunderstandings, the new code would often contain errors. At that time, roughly six months ago, Claude was poor at debugging these errors.

Additionally, we observed a pattern: when Claude ran into an error, it often solved the problem by adding more complexity, which, as software engineers, we know is rarely the right approach. Consequently, we would see Claude spend a lot of time and tokens debugging a problem it created, and it couldn't resolve it because the code didn't actually belong in our code base. This leap-before-looking severely limited Claude's effectiveness in our code base.

We also noticed some more fundamental flaws with the way Claude—this was Sonnet 3.5 at the time—was approaching Clojure code. One thing we commonly notice is that Sonnet defaults to an imperative programming approach, which we know does not work well in Clojure. For example, we discovered, embedded in a large code change, a doseq with an atom where a map, or a reduce would be preferable. These issues were problematic for us because Claude can generate a lot of code, which is very difficult to undo. Ultimately, we want the LLM to generate the correct code in the first place. So we faced the dilemma: how do we achieve better functional Clojure code from the LLM? The next step was to explore certain avenues to achieve that.

The clojure-mcp revolution

The first step I took was to explore the Clojure MCP tool. This tool exposed a set of editing and evaluation features that greatly reduce AI hallucinations, and with that we were able to ground the LLM better in our code base. The Clojure MCP's edit functionality was essential for preventing invalid parentheses, syntax errors, and other issues from entering our code base.

Clojure MCP fundamentally altered my belief in the ability of LLMs to write effective Clojure code. Previously, I was struggling and frustrated; working with Clojure and LLMs was a constant source of hallucinated functions, invalid syntax, and a generally unpleasant experience, with a lot of rework and misdirection. Clojure MCP really changed that. Thanks a ton to Bruce Hauman for building Clojure MCP and Clojure MCP Light and releasing them as open-source tools.

Skills, Prompts and OpenCode

The next step I took toward achieving better LLM output was to evaluate how Anthropic's skill system works. I also developed a tool I ended up calling Clojure Skills. Clojure Skills was envisioned as a SQLite database with a command-line interface that would allow the LLM agent and the human to search through a set of anthropic-style skills. These skills would inform the LLM agent about patterns, idioms, specific libraries, and whatnot, hoping that it would produce much better output and that I would spend a lot less time debugging. This was also the point when I started experimenting a little with tools like OpenCode.

What's interesting about OpenCode is that it lets you easily define your own system prompt. So at this point I defined a system prompt for building a skill that would dynamically load the library onto the Clojure REPL and build a skill for that library.

Here is an example of using OpenCode with a custom system prompt:

{
  "$schema": "https://opencode.ai/config.json",
  "agent": {
    "clojure": {
      "description": "Expert Clojure developer with REPL-driven workflow",
      "model": "anthropic/claude-sonnet-4",
      "prompt": "{file:./SYSTEM.md}"
    }
  },
  "default_agent": "clojure"
}

It was during the process of building my own skill system and using those skills day-to-day that I noticed a fundamental issue: over long LLM sessions with large context windows, the knowledge and the skill didn't stay very sticky. The LLM initially uses the correct skills and patterns, but eventually it starts to forget, ignore, and just do its own thing.

After some research I discovered that this problem is documented in the literature. In their 2023 paper "Lost in the Middle: How Language Models Use Long Contexts", Liu et al. demonstrated that LLMs effectively have a U-shaped memory curve: the most recent conversation turns and the system prompt are weighted more heavily than the middle of the conversation. Consequently, when skills are loaded mid-conversation, the LLM often fails to follow them.

I began experimenting within the clojure-skills system with a concept I call "prompt packing." I saw on Reddit that other people were putting all the knowledge of their code base directly into the system prompt. That made immediate sense given what we know about context windows. So I tried both inlining and referencing skills in the system prompt. OpenCode let me carefully curate the skills in my system prompt.

With this approach I achieved a new level of effectiveness: I was able to one-shot more and more tasks with the LLM than I ever could with Claude Code alone.

At this point in our development of Korey, it was time to tweak the system prompt for Korey itself. I was lucky enough to be assigned this task at work, and I spent a couple of weeks understanding how people evaluate system prompts and how they make them more effective for their task. I then drew the connection between my engineering system prompt and a prompt-evaluation system. This was key to our development for iterating and evaluating how we use LLMs and how our system prompt can become more effective. A helpful resource for this type of research is hamel.dev. Hamel Husain is a great communicator and writer. He was critical in my understanding of how to think about refining and defining system prompts, especially for working with code. Thanks, Hamel.

At this point, I started working on what eventually became my Clojure system prompt project. This was a prompt that we iterated a lot internally at Shortcut, and I have released it as an open-source project. I hope other Clojure developers will find this a helpful starting point to devise their own system prompts. One thing you can notice is I use the REPL extensively to ground truth, aka prevent hallucinations, before the LLM agent writes any code. This was key for transforming what was an incredibly frustrating experience into a much smoother and more graceful LLM interaction.

Another step I took was to deepen the ability of the LLM to use the Clojure platform itself. Clojure is designed to be interacted with from the REPL. This turns out to be a huge advantage when working with an LLM. For example, instead of defining a new skill for each library, I taught the LLMs about clojure.repl. This allows the LLM to dynamically explore any Clojure library on the REPL. Individual skills or lessons became less important, and the platform itself served as a dynamic feedback loop. Here is an example of that

<discovering-functions>
The REPL is your documentation browser. Explore before coding:

List all public functions in a namespace:
```shell
clj-nrepl-eval -p 7889 "(clojure.repl/dir clojure.string)"
# Output: blank? capitalize ends-with? escape includes? index-of join
#         lower-case replace reverse split split-lines starts-with? trim ...
```

Get detailed documentation for any function:
```shell
clj-nrepl-eval -p 7889 "(clojure.repl/doc map)"
# Output:
# -------------------------
# clojure.core/map
# ([f] [f coll] [f c1 c2] [f c1 c2 c3] [f c1 c2 c3 & colls])
#   Returns a lazy sequence consisting of the result of applying f to
#   the set of first items of each coll, followed by applying f to the
#   set of second items in each coll, until any one of the colls is
#   exhausted. Any remaining items in other colls are ignored...
```

Search for functions by name pattern:
```shell
clj-nrepl-eval -p 7889 "(clojure.repl/apropos \"split\")"
# Output: (clojure.string/split clojure.string/split-lines split-at split-with)
```

Search documentation text:
```shell
clj-nrepl-eval -p 7889 "(clojure.repl/find-doc \"regular expression\")"
# Shows all functions whose documentation mentions "regular expression"
```

Read function source code:
```shell
clj-nrepl-eval -p 7889 "(clojure.repl/source filter)"
# Shows the actual implementation - great for understanding edge cases
```

Despite these breakthroughs, the development flow was still not what I wanted it to be. First, OpenCode seems to consume a large amount of memory. It is a real bummer when you're working with a context window that you have crafted over thirty or forty minutes, and the LLM agent crashes because of the harness. It's a very frustrating experience.

Additionally, I've noticed that OpenCode tries to minimize the output of tool calls. This follows the general industry pattern we see in Claude Code, where certain information is hidden from the engineer. It's unclear to me what the exact design goals are, but I believe they're trying to minimize the information the developer sees so as not to overwhelm them.

However, when you're refining your system prompts and thinking carefully about your LLM interactions, this hiding of information becomes a real hindrance. I want to know that my tool calls are correct, that my edits are clear, and that my system functions as effectively as possible.

At this point I read a blog post about a new agent, pi-agent, and I was hooked.

PI Agent and liberation

Coming from an LLM agent harness like Claude Code that attempts to hide what the LLM is doing to a simple LLM harness that shows you everything felt like a liberating experience to me. Not only that, but pi-agent encourages you to solve your own problems, just like Emacs does. If there's a tool or functionality that you need that pi-agent doesn't provide, you simply build your own plugin. I have written a couple of plugins and used several more from the community.

One plugin I wrote is something to track my energy usage while working with LLMs. Like many, I am deeply concerned about LLMs' effects on our planet. My plugin tracks how much carbon my interactions are generating and compares that to standard car use.

What I value most about pi-agent is its reliability over long contexts. It also clearly indicates whether a tool call succeeded or failed with a green or red output. It's very minimal, does not use sub-agents, and allows me to focus on what my LLM is doing. As I follow along, I can tighten my iteration loop even more.

Current stack and future directions

My current stack is Clojure MCP Light, pi-agent, and my own system prompt. I plan to continue iterating on the Clojure system prompt and develop tools that make working with Clojure from LLMs more effective. I've also begun to experiment more and more with open-weight models, including the excellent Kimi 2.5 model from Moonshot AI, which is effectively my day-to-day driver now along with Sonnet 4.5.

There are a bunch of future directions that I would love to explore if given the time and opportunity. The rise of very effective open-weight models like Kimi 2.5 and MinMax 2.5 opens the possibility to post-train these models on Clojure and on our code base. Theoretically, this could allow us to not even have to use custom system prompts, skills, or specialized tools. We could effectively train our own Kimi Clojure or MinMax Clojure model that would have all of our best practices baked in. Of course, this is theoretical; I've done a little research, and there seems to be strong evidence that this would work. However, we won't really know until we try it out.

I also think it would be worthwhile to begin to curate all the tools that individual Clojure developers are building to work with Clojure and building an ecosystem and documentation around this.

Conclusion

I think there is a tendency among certain users of LLMs to limit the platforms and languages that we use with the LLMs. Out of the box, there are good reasons to consider this. If I were working on a greenfield project, I might consider using something like Python. However, I believe software decisions should serve the needs of the people who work on it and with it, not the needs of the LLM that might play a role creating it.

Ultimately, human beings are responsible for managing these software systems, and human knowledge still, in my view, trumps LLM statistical output. Taking that into consideration, the artifact produced by the LLM process is still a critical part of software development. I prefer to debug and deploy Clojure JVM artifacts, and I know many other organizations do as well.

Also, I think there are important reasons not to abandon all the lessons we've learned over the last twenty years of using Clojure. Clojure itself eliminates a whole set of common engineering problems like statefulness. I don't see the advantage of going back to a language where we have to use mutable state for iteration, for example, or a language where functional idioms are not the default.

My experiences with using an LLM for software engineering and developing an LLM have taught me that, rather than constraining the platforms we use and adopt, the LLM era could actually free us. The idea that LLMs are only good at something because it's in the training set ignores the tools, skills, and system prompts that we can apply post-training.

We can craft and refine LLMs for our specific platform. Not only that, there is something beautiful about LLMs: instead of narrowing how we work, they can allow people to work in very different ways.

I hope this blog post was insightful and informative, and I hope that my experience and our experience at Shortcut can help you craft and shape an LLM experience that solves your particular engineering problems. Thank you for reading.

References

Katzy, J., & Izadi, M. (2023). On the Impact of Language Selection for Training and Evaluating Programming Language Models. arXiv preprint arXiv:2308.13354. https://arxiv.org/abs/2308.13354

Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2023). Lost in the Middle: How Language Models Use Long Contexts. arXiv preprint arXiv:2307.03172. https://arxiv.org/abs/2307.03172

On Dyslexia, Programming and Lisp.

2026-02-10T00:00:00Z

Dyslexia is a difference in the way the brain processes language. It is often defined as the difference between intelligence (vaguely defined via the fake IQ test) and a person's ability to read. That is, a dyslexic with normal intelligence will underperform compared to their peers in learning how to read. There are many different theories on what the cause is, but most people believe that there are fundamental differences between the brains of people who have dyslexia and those that do not. In Maryanne Wolf's book, Dyslexia, Fluency, and the Brain, several essays point to a physical difference in the density of specialized brain cells that the brain uses to process letter shapes. It has become clear recently from fMRIs that dyslexics actually develop unique neural pathways for reading that are located in the right side of the brain compared to the normal left-side-dominated reading pathways. However, a root cause has not been identified, and the jury is still out. Dyslexia varies considerably, with up to 15% of the population experiencing some form of it.

The experience of dyslexia is very interesting. I am dyslexic, and I was diagnosed in the 3rd grade. My dyslexia was more "intense" than some, having both verbal and visual effects. I remember trying to learn to read and struggling with all the letter rotation on the page. PQ DB IL letters all looked the same to me.

The best way to describe it is that my brain was attempting to process these letters for their meaning and would automatically rotate the letter. When you think about it, it actually makes sense. When you pick up an object with your hand that you don't understand, what do most people do? They turn it to look at all sides of the object. That's what my brain does with letters.

Because English letters are not "fixed," there is no inherent difference between the bottom and top of the letter; my brain could not fix the position of the letter on the page. The rotation was intense for me, as I was experiencing rotation in "3D." For me, this made the letter move more on the page and also made the letters even more confusing for me.

It made reading very difficult until my brain was able to fix their position. I did this by adopting what I called the "Sky line" approach. Each word makes a unique shape, like the skyline of buildings in a city. By memorizing the shape of each word in the English language, I could fix the position of the word on the page and overcome this rotation problem. Interestingly enough, once I memorized the shape of most of the words in English, I found this method to be very fast. It may be because I can see the shape of the word quickly, not relying on letter shape.

Frankly, rotation was a nightmare for me, literally. As a child, I remember having dreams/nightmares where I was falling and spinning. I would fall forever, spinning and spinning. I could not orient myself. Once, in this nightmare, I realized my solution. There was no fixed position. My desire to fix the position of the letter, or myself, was a fool's errand. Rotation itself was the only constant; the rotation, in a way, was a fixed position. It's a little difficult to describe, and this "realization" did come to me as a dream, so take it for what you will. I guess I am still spinning.

Anyway, moving on from that, in college I developed an interest in software engineering. Taking a class in C and then quickly learning Python. C and its family of languages are interesting. They are frankly a nightmare of obscure scribbles that had little meaning to me. I think a classic example of this is the ternary conditional operator. This is an operator that I still struggle with. I rapidly found that Python's whitespace-based syntax was much easier on my spinning brain. Most of the obscure squiggles are removed, and I was able to focus on the semantic meaning of the program. I found that my Sky Line approach for visual memorization works very well for programming languages. Each function or class defines its own shape. Remembering each shape in a large codebase is easier than remembering how to spell my own name.

After college I learned about Lisp, specifically the Clojure programming language. After learning about Clojure, I dove headlong into Scheme and its family of S-expression languages. At last, I had found a language that was seemingly designed for my spinning brain.

Lisp-based languages use S-expressions. These S-expressions form a constant and regular pattern. All expressions are enclosed in parentheses (). The first symbol is always the function or the macro. The rest of the expressions are arguments to that function or macro. Lisp programming is layers and layers of S-expressions, each with their own shape, making them easy to memorize and locate. I no longer have to worry about the rotation of the ternary conditional operator.

Scheme fascinates me even further. Many of the coding conventions in Scheme encourage people to use full and descriptive function names. I believe this gave my mind a deeper ability to organize the skyline shapes. Maybe you can think of this like the difference between 32-bit and 64-bit pointers in garbage collectors. The longer the function name, the more functions I can store in my mind's working memory.

These experiences leave me wondering what other characteristics we could add to our programming languages that would help people like myself. In general, people with "normal," left-dominated reading pathways have constructed languages. In doing so, they have built language that works effectively for them. These languages are tangles of obscure squiggles, a spinning brain nightmare. This tangle allows for denser packing of meaning into a "smaller" space, easy for normies, or so they think. But I can't help but notice the popularity of languages like Ruby and Python. Both of these languages take approaches that reduce the visual complexity.

I wonder if by reducing the visual complexity, we can instead increase the amount of semantic complexity we can hold in our minds. Dyslexia is common, and many people experience some form of the issues I describe above. Perhaps, we are all suffering from the tyranny of obscure squiggles.

Use pandoc to create engineering-focused presentations.

2026-02-06T00:00:00Z

Introduction

The blog post describes how I use pandoc and Slidy WC3 to build engineering focused presentations.

Software engineering requires a lot of communication with stakeholders, your colleagues, and to LLM agents. There are many ways you can build a presentation these days, but I often find myself using Pandoc's presentation mode. Pandoc allows me to write a presentation as a simple markdown file. This allows me to commit the presentation to our code repo. It also makes it easier to work with an LLM agent in building or editing your presentation.

I have also used pandoc and its presentation system when working with agents. After I work with the LLM agent, I will often ask it to create a presentation for me using markdown and link to the code. This then allows me to review what the LLM Agent built. It also allows me to share the implementation with my colleagues. Okay, without further ado, let's build our presentation.

We are going to use pandoc, just, and pipenv to build our presentation with PlantUML diagram support.

Install

First, install the required tools:

brew install pandoc just pipenv

Next, create a Pipfile in your project directory to manage Python dependencies:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
pandoc-plantuml-filter = "*"

[dev-packages]

[requires]
python_version = "3"

Install the Python dependencies:

pipenv install

Create Your Presentation

Now let's create a basic presentation document. Save this as presentation.md:

% Engineering Presentation
% Your Name
% 2026-02-06

# Project Overview

## Goals

- Improve system reliability
- Reduce latency by 50%
- Better observability

## Architecture

# Implementation Details

## Key Changes

- Added caching layer with Redis
- Implemented circuit breaker pattern
- Enhanced logging with structured JSON

## Code Example

Here's how we implemented the circuit breaker:

```python
class CircuitBreaker:
    def __init__(self, threshold=5):
        self.failure_count = 0
        self.threshold = threshold
        self.state = "closed"

    def call(self, func):
        if self.state == "open":
            raise Exception("Circuit open")
        try:
            result = func()
            self.failure_count = 0
            return result
        except Exception as e:
            self.failure_count += 1
            if self.failure_count >= self.threshold:
                self.state = "open"
            raise e
```

## System Architecture

```plantuml
@startuml
!theme plain

package "Client Layer" {
  [Web App]
  [Mobile App]
}

package "API Layer" {
  [Load Balancer]
  [API Gateway]
  [Circuit Breaker]
}

package "Service Layer" {
  [Auth Service]
  [Cache Service]
  [Core Service]
}

database "Data Layer" {
  [Redis Cache]
  [PostgreSQL]
}

[Web App] --> [Load Balancer]
[Mobile App] --> [Load Balancer]
[Load Balancer] --> [API Gateway]
[API Gateway] --> [Circuit Breaker]
[Circuit Breaker] --> [Auth Service]
[Circuit Breaker] --> [Core Service]
[Core Service] --> [Cache Service]
[Cache Service] --> [Redis Cache]
[Core Service] --> [PostgreSQL]

@enduml
```

## Metrics

| Metric      | Before | After | Improvement |
|-------------|--------|-------|-------------|
| Latency p99 | 250ms  | 120ms | 52% faster  |
| Error rate  | 0.5%   | 0.05% | 10x better  |
| Cache hit   | 0%     | 85%   | New         |

## Next Steps

- Roll out to production
- Monitor for 1 week
- Document runbooks

Building Your Presentation

Now let's create a justfile to automate building our presentation with PlantUML support:

# Default recipe - build the presentation
default: build

# Build the HTML presentation using Slidy with PlantUML support
build:
    pipenv run pandoc -t slidy -s --filter pandoc-plantuml presentation.md -o presentation.html

# Build a self-contained single-file presentation
build-standalone:
    pipenv run pandoc -t slidy -s --embed-resources --standalone --filter pandoc-plantuml presentation.md -o presentation.html

# Clean generated files
clean:
    rm -f presentation.html
    rm -rf plantuml-images

# Watch for changes and rebuild
watch:
    just build && fswatch -o presentation.md | xargs -n1 -I{} just build

And now let's build our new presentation:

just build

This generates presentation.html which you can open in any browser. Use the arrow keys to navigate between slides. The Slidy format provides a clean, professional look that's perfect for engineering presentations. The PlantUML diagrams will be automatically rendered as SVG images inline in your presentation.

New Blog Setup

2026-02-02T00:00:00Z

I've migrated this blog to a new setup. Here's how it works now:

Stack

Lektor - Static site generator with a nice content model
Pandoc - Builds my resume from YAML to HTML
just - Command runner for build tasks
GitHub Actions - Automatic deployment on push to main
PureCSS - Lightweight CSS framework

How It Works

Content lives in YAML and Lektor's .lr files. The build process:

just resume - Pandoc converts src/resume.yml to HTML using a custom template
lektor build - Lektor builds the rest of the site
GitHub Actions deploys to GitHub Pages on every push

Why Lektor?

I really enjoyed using GNU Guix and Haunt. Guix is a fantastic package manager with reproducible builds, and Haunt is a lovely Scheme-based static site generator. However, I wanted something that runs easily on macOS without needing a Linux VM or container.

More importantly, I wanted to focus on writing blog posts, not building and maintaining my own blogging system. Lektor has a clean content model, and just works out of the box.

PlantUML Diagrams

I wrote a small Lektor plugin that renders PlantUML diagrams directly in markdown. Just use a fenced code block with plantuml as the language:

The plugin encodes the diagram text and fetches the rendered SVG from the public PlantUML server at build time. The SVG is embedded inline, so no external requests are made when viewing the page.

The full source is on GitHub.

Using Pandoc to Build a Technical Manual

2025-01-04T00:00:00Z

Pandoc is a markup transformation tool that converts documents between formats. It's an invaluable tool for any software engineer who works with documentation.

In this guide, we'll build a "technical manual" using Pandoc. Our manual will be a collection of Markdown documents with PlantUML diagrams. We'll use Pandoc to combine these documents into a single HTML page, rendering PlantUML diagrams as SVG images. Along the way, we'll explore Pandoc filters for rendering diagrams and including source code from your project.

By the end of this guide, you'll have enough understanding of Pandoc to build your own documentation system.

Installation

You can install Pandoc on macOS with Homebrew:

brew install pandoc

On Fedora:

sudo dnf install pandoc

Or with GNU Guix:

guix install pandoc

Introduction to Pandoc

Let's say you have a simple HTML document:

<!DOCTYPE html>
<head>
  <meta charset="utf-8" />
  <meta name="description" content="Software Wanderings" />
</head>
<body>

  <h1>Hello</h1>
  <ul>
    <li>one item</li>
    <li>two item</li>
    <li>three item</li>
  </ul>
  <a href="/">Relative reference</a>


  <table>
    <tr>
      <th>Company</th>
      <th>Contact</th>
      <th>Country</th>
    </tr>
    <tr>
      <td>Alfreds Futterkiste</td>
      <td>Maria Anders</td>
      <td>Germany</td>
    </tr>
    <tr>
      <td>Centro comercial Moctezuma</td>
      <td>Francisco Chang</td>
      <td>Mexico</td>
    </tr>
  </table>

  <svg viewBox="0 0 100 100" preserveAspectRatio="xMidYMid slice" role="img">
    <title>A gradient</title>
    <linearGradient id="gradient">
      <stop class="begin" offset="0%" stop-color="red" />
      <stop class="end" offset="100%" stop-color="black" />
    </linearGradient>
    <rect x="0" y="0" width="100" height="100" style="fill:url(#gradient)" />
    <circle cx="50" cy="50" r="30" style="fill:url(#gradient)" />
  </svg>


</body>

You can convert this from HTML to Markdown with the following command:

pandoc -f html -t markdown public/about.html
cat public/about.html | pandoc -f html -t markdown

Which produces the following output

# Hello

-   one item
-   two item
-   three item

[Relative reference](/)

  Company                      Contact           Country
  ---------------------------- ----------------- ---------
  Alfreds Futterkiste          Maria Anders      Germany
  Centro comercial Moctezuma   Francisco Chang   Mexico

![](data:image/svg+xml;base64,PHN2ZyB2aWV3Ym94PSIwIDAgMTAwIDEwMCIgcHJlc2VydmVhc3BlY3RyYXRpbz0ieE1pZFlNaWQgc2xpY2UiIHJvbGU9ImltZyI+CiAgICA8dGl0bGU+QSBncmFkaWVudDwvdGl0bGU+CiAgICA8bGluZWFyZ3JhZGllbnQgaWQ9ImdyYWRpZW50Ij4KICAgICAgPHN0b3AgY2xhc3M9ImJlZ2luIiBvZmZzZXQ9IjAlIiBzdG9wLWNvbG9yPSJyZWQiPjwvc3RvcD4KICAgICAgPHN0b3AgY2xhc3M9ImVuZCIgb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSJibGFjayI+PC9zdG9wPgogICAgPC9saW5lYXJncmFkaWVudD4KICAgIDxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSIxMDAiIGhlaWdodD0iMTAwIiBzdHlsZT0iZmlsbDp1cmwoI2dyYWRpZW50KSI+PC9yZWN0PgogICAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgcj0iMzAiIHN0eWxlPSJmaWxsOnVybCgjZ3JhZGllbnQpIj48L2NpcmNsZT4KICA8L3N2Zz4=)

Or Emacs's Org Mode

* Hello
  :PROPERTIES:
  :CUSTOM_ID: hello
  :END:

- one item
- two item
- three item

[[/][Relative reference]]

| Company                    | Contact         | Country |
|----------------------------+-----------------+---------|
| Alfreds Futterkiste        | Maria Anders    | Germany |
| Centro comercial Moctezuma | Francisco Chang | Mexico  |

[[data:image/svg+xml;base64,PHN2ZyB2aWV3Ym94PSIwIDAgMTAwIDEwMCIgcHJlc2VydmVhc3BlY3RyYXRpbz0ieE1pZFlNaWQgc2xpY2UiIHJvbGU9ImltZyI+CiAgICA8dGl0bGU+QSBncmFkaWVudDwvdGl0bGU+CiAgICA8bGluZWFyZ3JhZGllbnQgaWQ9ImdyYWRpZW50Ij4KICAgICAgPHN0b3AgY2xhc3M9ImJlZ2luIiBvZmZzZXQ9IjAlIiBzdG9wLWNvbG9yPSJyZWQiPjwvc3RvcD4KICAgICAgPHN0b3AgY2xhc3M9ImVuZCIgb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSJibGFjayI+PC9zdG9wPgogICAgPC9saW5lYXJncmFkaWVudD4KICAgIDxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSIxMDAiIGhlaWdodD0iMTAwIiBzdHlsZT0iZmlsbDp1cmwoI2dyYWRpZW50KSI+PC9yZWN0PgogICAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgcj0iMzAiIHN0eWxlPSJmaWxsOnVybCgjZ3JhZGllbnQpIj48L2NpcmNsZT4KICA8L3N2Zz4=]]

Pandoc also has native and JSON output formats. When you select JSON, the raw AST that Pandoc generates from your document is returned. Let's take a quick peek at this AST:

pandoc code-examples/sample.html -f html -t json | jq > code-examples/sample.json

        ]
      ]
    },
    {
      "t": "BulletList",
      "c": [
        [
          {
            "t": "Plain",
            "c": [
              {
                "t": "Str",
                "c": "one"
              },
              {
                "t": "Space"
              },
              {
                "t": "Str",
                "c": "item"
              }

Now we can clearly see that Pandoc is a transformation system. It parses a document into its native AST, then uses a set of writers to render that AST into the target format:

INPUT --reader--> AST --filter--> AST --writer--> OUTPUT

Taken from the Pandoc Website

Building Your Technical Document

Now that we understand the basics of how Pandoc works, let's use it to build a technical manual. We have three main goals:

PlantUML-based diagrams - Render architecture and sequence diagrams
Code inclusion - Include code fragments from your project
Markdown-based content - Write in Markdown for simplicity

Project Setup

We'll use a separate Markdown file for each section and a justfile to build our document into a single HTML file. Just is a modern command runner that's simpler and more intuitive than Make. Here's our project structure:

justfile
metadata.yml
sections/introduction.md
sections/architecture.md

First, let's create a Pandoc metadata file. Pandoc uses YAML for metadata, which you can include inline or in a separate file.

Create a file called metadata.yml with the following content:

title: "Software alchemist Docs"
author: "Ivan Willig"
email: "iwillig@gmail.com"
description: "Software Alchemist Docs"
lang: "en-US"
toc-title: "Table of Contents"
sections:
  - chapters/introduction.org
  - chapters/design.org

css:
  - css/pico.min.css
  - css/docs.css
  - css/highlighting.css

Now let's create a justfile to automate the build process. First, install Just:

# macOS
brew install just

# Or with cargo
cargo install just

Create a file called justfile with the following content:

# Build the documentation
default: docs.html

# Generate HTML documentation from markdown sources
docs.html: sections/*.md metadata.yml
    pandoc metadata.yml \
        --toc --toc-depth=5 \
        --template templates/template.html \
        --highlight-style zenburn \
        -F pandoc-plantuml \
        -f markdown -t html \
        sections/introduction.md \
        sections/architecture.md \
        -o docs.html

# Clean generated files
clean:
    rm -f docs.html
    rm -f plantuml-images/*.svg
    rm -f plantuml-images/*.uml

# Watch for changes and rebuild
watch:
    watchexec -e md,yml just

Now you can build your documentation by running:

just

Or clean up generated files with:

just clean

Using Pandoc Filters

Pandoc filters allow you to manipulate the AST between the reader and writer stages. This is where the real power of Pandoc comes in—you can extend it to handle custom markup or integrate with external tools.

For our technical manual, we'll use the pandoc-plantuml filter to render PlantUML diagrams. Install it with:

pip install pandoc-plantuml

You'll also need PlantUML itself:

# macOS
brew install plantuml

# Or download the JAR file
# https://plantuml.com/download

Now you can include PlantUML diagrams directly in your Markdown:

## Architecture Diagram

```plantuml
@startuml
actor User
participant "Web Server" as Web
participant "Database" as DB

User -> Web: HTTP Request
Web -> DB: Query
DB --> Web: Result
Web --> User: HTTP Response
@enduml
```

The filter automatically converts these code blocks into SVG images when you build your documentation.

You can also write custom filters in Python. Here's a simple example that converts emphasized text to uppercase:

#!/usr/bin/env python3
from pandocfilters import toJSONFilter, Emph, Str

def caps(key, value, format, meta):
    if key == 'Emph':
        return [Str(value[0]['c'].upper())]

if __name__ == "__main__":
    toJSONFilter(caps)

Pandoc Templates

Each output format has a default template. You can view a format's template using Pandoc itself:

pandoc -D html

This outputs the default HTML template. You can read more about Pandoc's template syntax in the official documentation.

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="$lang$" xml:lang="$lang$"$if(dir)$ dir="$dir$"$endif$>
<head>
  <meta charset="utf-8" />
  <meta name="generator" content="pandoc" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
$for(author-meta)$
  <meta name="author" content="$author-meta$" />
$endfor$
$if(date-meta)$
  <meta name="dcterms.date" content="$date-meta$" />
$endif$
<!-- ... rest of template ... -->

You can customize this template to add your own CSS, JavaScript, or HTML structure. Save your custom template and reference it with the --template flag as shown in the justfile above.

Getting Started Quickly

If you want to skip the manual setup, I've created a cookiecutter template with everything pre-configured:

pandoc-documentation-tool

This template includes:

Pre-configured justfile with common tasks
Sample metadata.yml
Example markdown sections
HTML template with responsive CSS
PlantUML filter setup
Code inclusion utilities
Example diagrams and structure

You can use it to bootstrap a new documentation project:

# Install cookiecutter if you haven't already
pip install cookiecutter

# Generate a new project from the template
cookiecutter gh:iwillig/pandoc-documenation-tool

The template will prompt you for a project name, author, and other details, then generate a complete documentation project. Customize it to fit your needs.

Conclusion

Pandoc is a powerful tool for building technical documentation. By combining Markdown, PlantUML, and Pandoc filters, you can create maintainable, version-controlled documentation that lives alongside your code. Using Just as a command runner keeps the build process simple and reproducible.

Key takeaways:

Pandoc converts between formats by parsing to an AST, then rendering
Filters extend Pandoc's capabilities
Templates customize the output format
PlantUML integration enables version-controlled diagrams
Just provides a simple, modern way to automate builds
The cookiecutter template gives you a head start

You now have everything you need to build your own technical manual. Start with the cookiecutter template, or build your setup from scratch using this guide. Happy documenting!

Hello, World

2024-12-28T00:00:00Z

Note: This post is a bit out of date. I no longer use GNU Guix and Haunt for this blog. Instead, I've migrated to Lektor as my static site generator, with Pandoc still handling resume generation.

Welcome to Software Wanderings, a technical blog where I'll be documenting my experiences, learnings, and observations about software engineering. I hope this blog will encourage me to write about my experiences and share them with a broader audience.

The blog will explore software design, functional programming, API design and more. While I primarily work in Clojure, I'll also cover JavaScript, TypeScript, Rust, Scheme, and SQL.

2025 Goals

I have five primary objectives for the year:

Game Development: Learn to build real-time strategy games inspired by Command and Conquer and StarCraft
Immutable Linux: Study NixOS, GNU Guix, and Fedora Silverblue distributions
Pandoc Mastery: Understand document transformation systems
API Design: Deepen my knowledge of REST and GraphQL architectures
Technical Writing: Publish insights throughout the year

Technical Architecture

This blog's infrastructure uses:

GitHub Pages for hosting
Haunt (a static site generator) for template processing
Pandoc for Markdown preprocessing with advanced diagram support
GNU Make for build orchestration
GNU Guix for dependency management

The setup requires Pandoc, PlantUML, and Haunt, with a custom GNU Guix manifest handling dependencies.

Reflections on Software Engineering

One year of LLM usage with Clojure

Introduction

Early Adoption and Struggles

The doldrums of Claude Code

The clojure-mcp revolution

Skills, Prompts and OpenCode

System Prompts Refinements and Prompt Evals

PI Agent and liberation

Current stack and future directions

Conclusion

References

On Dyslexia, Programming and Lisp.

Use pandoc to create engineering-focused presentations.

Introduction

Install

Create Your Presentation

Building Your Presentation

New Blog Setup

Stack

How It Works

Why Lektor?

PlantUML Diagrams

Using Pandoc to Build a Technical Manual

Installation

Introduction to Pandoc

Building Your Technical Document

Project Setup

Using Pandoc Filters

Pandoc Templates

Getting Started Quickly

Conclusion

Hello, World

2025 Goals

Technical Architecture