From launchd Plists to Jenkins to My Own CI: Building kipp, a 50 MB Scheduler

I keep a small always-on Mac with 16 GB of memory for the boring work: backups, log rotation, a DNS updater, a few report generators, and the CI of some private repos. This post is the story of how that work went from a pile of launchd plists, to Jenkins, to a scheduler I wrote myself and released as kipp. The point is not that you should write your own CI. It is where each step stopped fitting, and how small a tool can be once you know exactly what you need from it.

Stage one: launchd as cron

On macOS the native way to run something every night is a launchd plist with StartCalendarInterval. It works, and that is the trap. Each job becomes an XML file in ~/Library/LaunchAgents, with its own log path, its own environment and its own idea of what failure looks like:

<key>StartCalendarInterval</key>
<dict>
  <key>Hour</key><integer>3</integer>
  <key>Minute</key><integer>15</integer>
</dict>
<key>StandardErrorPath</key>
<string>/tmp/backup.err</string>

flowchart LR
  L((launchd)) --> A[backup.plist]
  L --> B[ddns.plist]
  L --> C[log-rotate.plist]
  L --> D[report.plist]
  A --> A1[/tmp/backup.err/]
  B --> B1[/var/log/ddns.log/]
  C --> C1[(nothing)]
  D --> D1[/~/report.log/]
  style C1 stroke-dasharray: 4 4

Seven of them, and the questions I could not answer quickly were always the same: did last night's run succeed, how long did it take, what did it print, and how do I run it again right now without retyping its command. launchd answers none of those. It is a process supervisor that happens to have a timer, not a scheduler with a history.

Stage two: Jenkins, with no plugins

Jenkins answers all four questions, so every job moved there: core freestyle projects with one shell step each, job definitions kept as XML in git and replaced wholesale on every start, so the UI was never the source of truth. It grew to 38 jobs within a week and it did the work.

It also cost what a JVM costs. The heap was capped at 256 MB and the live set was about 70 MB, yet the process sat at 3.6 GB resident and peaked at 8.8 GB after four and a half days. The growth was native memory from the end-of-build process scan, freed but never handed back to the OS. On a 16 GB machine that also runs databases and model servers, that is not a rounding error. The workaround was a Jenkins job whose only purpose was to force a garbage collection every ten minutes, which is the moment I knew the tool was the wrong size.

The other complaints were smaller but real: nobody reads freestyle XML, a lock between two jobs had to be improvised with a directory, and every plugin is one more thing to keep patched.

Why not something off the shelf

I looked hardest at Woodpecker, which is Go, small, and reads pipeline YAML from the repo. It misses on the shape of the work: login and permissions come from a forge, everything belongs to a repo and a branch, and half of my jobs (rotate a log, update a DNS record) have no repo at all. Its local backend is documented as having no isolation, and the primary one assumes Docker, which that machine did not run.

What I wanted was Jenkins' model, where a job is any job and not only a repo build, with a modern pipeline file and a footprint closer to a cron daemon.

Stage three: kipp replaces cron and Jenkins

kipp is one Go binary with the web UI embedded and SQLite (pure Go, no cgo) for state. A job is a YAML file in a directory. The directory is the source of truth: kipp reload reads it again, and a directory with one bad file is refused whole, so the server keeps the jobs it had.

flowchart TB
  a["Before October<br/>7 launchd plists, logs scattered"] --> b["2 Oct: Jenkins takes every job<br/>38 jobs, gigabytes resident"]
  b --> c["11 Oct: kipp replaces Jenkins<br/>the day after its design"]
  c --> d["23 Oct: kipp builds pushes<br/>GitHub Actions workflows removed"]
  d --> e["26 Oct: agents and containers"]
  e --> f["29-30 Oct: multibranch<br/>and organization folders"]
  f --> g["2 Nov: 1.0.0 released"]

The smallest useful job is three lines, and a real one is not much longer:

name: nightly-backup
group: ops
triggers:
  cron: "H 3 * * *"        # Jenkins' H: a stable minute per job, spread over the hour
concurrency: drop          # a fire that lands on a running build is dropped
timeout: 1h
secrets: [BACKUP_KEY]      # sealed in the database, masked as *** in the log
stages:
  dump:   {run: ./backup/dump.sh}
  upload: {needs: [dump], run: ./backup/upload.sh, retries: 2}
on_failure: {run: ./hooks/notify.sh}

Here is what is inside that one process:

flowchart TB
  subgraph bin["kipp (one binary)"]
    direction TB
    T["Triggers<br/>cron, webhook, after, by hand"] --> Q["Queue<br/>concurrency, locks"]
    Q --> R["Runner<br/>stage DAG, retries, timeout"]
    R --> P{"where?"}
    P -->|no runs_on| S["shell on the server"]
    P -->|runs_on| AG["standing agent"]
    P -->|runs_on + image| C["container on an engine"]
    UI["Web UI + JSON API"] --- DB
    R --> DB[("SQLite<br/>builds, users, secrets")]
    R --> LOG[("log files")]
  end
  J[/"jobs/*.yml"/] -->|reload| T
  GH(("GitHub")) -->|signed webhook| T
  CLI["kipp CLI"] --> UI

With 64 job files loaded and sixteen hours of uptime, the server process I measured on 3 November 2026 was about 50 MB resident. The binary is about 40 MB on disk, most of it two statically linked Linux agents it carries inside itself (more on that below).

Stages are a DAG, not a list

A job's stages declare needs, and anything whose needs are met runs in parallel in a shared workspace. when decides whether a stage runs at all: on a param, on the default branch, or on how the stages it needs ended. That last one is what makes teardown honest:

stages:
  up:       {run: ./e2e/up.sh}
  smoke:    {needs: [up], when: {params: {SUITE: [smoke]}}, run: ./e2e/run.sh smoke, junit: "reports/*.xml"}
  full:     {needs: [up], when: {params: {SUITE: [full]}},  run: ./e2e/run.sh full,  junit: "reports/*.xml"}
  report:   {needs: [smoke, full], when: {needs_status: failure}, run: ./e2e/report.sh}
  teardown: {needs: [up, report],  when: {needs_status: always},  run: ./e2e/down.sh}

flowchart LR
  up --> smoke & full
  smoke & full --> report
  up --> teardown
  report --> teardown
  classDef cond stroke-dasharray: 4 4
  class smoke,full,report cond

A green teardown never turns a failed build green, and a stage whose when does not hold is skipped along with what needs it, without failing anything. Exit codes can mean "unstable" rather than "failed" (unstable_exit: [4]), and a stage can hand variables to the next one. JUnit reports become a test view with history per test.

Running somewhere else: agents dial in

Some stages need a different machine: Linux tests from a macOS server, or a box with Docker. A stage says where it runs with labels, never a hostname, so a host can be replaced without editing a job:

stages:
  test:
    runs_on: linux          # every label must match
    image: golang:1.27      # an image means: a fresh container on a Docker engine
    run: go test ./...

The agent always opens the connection, over plain HTTP long-polling, the way GitHub's and Jenkins' inbound runners do. A laptop behind NAT or a container inside a VM can serve builds without the server reaching it.

sequenceDiagram
  participant S as kipp server
  participant E as Docker engine
  participant A as agent in container
  S->>E: create container from image
  S->>E: copy embedded agent binary to /kipp/agent
  E->>A: start /kipp/agent
  A->>S: long-poll: any stage for me?
  S-->>A: stage "test" + its secrets
  A->>S: stream log lines
  A->>S: done, exit 0
  S->>E: remove container (build is final)

The agent binary is embedded in the server, so any Linux image with a shell works, alpine included, and server and agent can never disagree on version. The container lives exactly as long as the build: no host mounts, no Docker socket, no published ports.

When GitHub Actions ran out of minutes

The second push came from outside. Private repositories on GitHub's free plan get a monthly allowance (at the time of writing GitHub's billing docs express it as $12 of compute, against $0.006 per Linux minute and $0.062 per macOS minute). A few busy repos used it up, deploys stopped, and the fix that already existed was the scheduler sitting idle on the Mac.

kipp gained webhook triggers (a signed push or tag), then two things lifted straight from Jenkins:

• A multibranch job builds every branch and pull request of one repo, each as a child job whose stages come from a .kipp.yml in the commit it builds. It posts commit statuses back, so a pull request shows a check.
• An organization folder does that for every repo of a GitHub user or organization that contains a .kipp.yml, and creates the webhooks itself.

flowchart TB
  O["acme (organization folder)"] --> S["acme.site"]
  O --> API["acme.api"]
  S --> S1["acme.site.main"]
  S --> S2["acme.site.pr-12"]
  API --> A1["acme.api.main"]
  API --> A2["acme.api.feature-x"]

The split of authority matters more than the discovery. The repo's file may only declare stages and plain variables. Secrets, locks, schedules and concurrency stay in the central job file, so a pull request cannot grant itself a deploy key:

# jobs/acme.yml, on the server
name: acme
owner:
  login: acme
  token: GITHUB_TOKEN_ACME
  include: ["*"]
  exclude: ["sandbox-*"]
  branches: [default]
  pull_requests: merge
secrets: [DEPLOY_KEY]

# .kipp.yml, in each repo
stages:
  test:   {run: make test, junit: "reports/*.xml"}
  deploy: {needs: [test], when: {default_branch: true}, run: ./deploy.sh}

Why not a self-hosted runner?

It is the obvious objection, and a fair one: GitHub's billing docs list self-hosted runners as free, outside the included allowance. If every job you have lives in a repo and github.com is where you want to watch them, a runner on your own machine is the simpler answer, and I would take it.

It did not fit here, for three reasons.

• A runner executes; it does not schedule. Workflows still start from GitHub events, and the schedule event is documented as best effort: it "can be delayed during periods of high loads", queued runs may be dropped, and a public repo's schedules switch off after 60 days without activity. A nightly backup should not depend on someone else's queue being quiet.
• A job with no repo still needs a repo. Rotating a log or updating a DNS record would each become a workflow in a placeholder repository, with its history on github.com instead of next to the machine it runs on. That is the visibility problem from stage one, moved rather than solved.
• Scope and credentials. On GitHub's free plan an organization has only the default runner group (custom groups need the Team plan), so a runner registered for the organization serves every repo in it. On macOS the runner's service script starts jobs in their own security session, which cannot read the login keychain, so gh and git over HTTPS fail until you either move credentials somewhere else or hand every job your keychain.

Pull requests from strangers are a risk either way: GitHub recommends self-hosted runners only for private repos, and the same caution applies to kipp. What kipp changes is where the boundary is written down: secrets and schedules live in job files on the server, and a repo's .kipp.yml can only declare stages.

What "enough" turned out to be

kipp is not trying to be Jenkins or GitHub Actions. There is no plugin system, no marketplace of actions, no matrix syntax, and it runs untrusted pull request code with the job's secrets on the server's machine, which is a fine trade for your own repos and a bad one for a public project that takes outside contributions.

Need	launchd	Jenkins	GitHub Actions	kipp
History and logs per run	no	yes	yes	yes
Jobs that belong to no repo	yes	yes	awkward	yes
Config	plist XML	XML or Groovy	YAML in repo	YAML, central plus in repo
Stage DAG and conditions	no	pipeline plugin	yes	yes
Branches and PRs	no	multibranch plugin	yes	yes
Cost on a private repo	free	free	metered minutes	free
Memory measured here	n/a	3.6 GB	n/a	50 MB

What I did need turned out to be a short list: cron with H and time zones, a run by hand with typed params, locks, a DAG with when, JUnit, secrets that never print, per-job grants for a token that may see one job, webhooks, multibranch, and a CLI and API that can do everything the UI can. Everything else, artifacts, a keyed cache, metrics with guards, came later and is optional.

A month ago those jobs were seven plists nobody could watch. Today they are 64 YAML files, one process, and one SQLite file to back up. kipp is MIT licensed and installs with brew install ndthuan/tap/kipp or as a container; the README has the quick start.