Composability explained

We built boxd on one belief: everyone should have access to a great runtime.

Most value is eventually created in the application layer. Therefore, we want every builder to work mostly on the application layer, and leave the infrastructure under it to the runtime.

That took two things:

1. Make it available to everyone, whatever their size.
2. Make it fit any environment, however they build.

Available to everyone

You can run boxd on boxd managed, our hosted solution, or on your own servers. We made both open to builders of every size.

boxd managed bills on use. You pay for active vCPU, the memory in use and the disk written, metered every second. You don't pay for what a machine reserves. On average a boxd machine keeps 10% of its reserved vCPU active, and that 10% is what you pay for. In standby you pay only for memory and disk, and in hibernation only for disk. The rates are on the pricing page.

Self-hosting is free for individuals and smaller teams. You can run boxd on your own servers for free, up to 16 vCPUs. The limit counts the logical CPUs of your worker hosts, so the control plane can run on a separate host and all 16 go to your machines. There is no licence key and no account to make. Above 16 vCPUs, larger companies pay for a yearly licence.

Self-hosting is easy. A self-hosted stack with many moving parts takes time to run, whether you are one person or a platform team. We took that into consideration when we designed the architecture. We ended up making boxd a single binary, so there is no external database, orchestrator or other exotic service to run. The binary runs on any host with KVM. The install generally takes 20 to 40 minutes. The installer is on the self-hosting page.

Available in any environment

Getting the runtime is a good first step. It also has to work well with the environment you run it in. For example, one team wants to store its machine disks in its own S3 bucket, while another wants to keep them on its own servers. That is one example, and in practice every company (and individual) has a different setup. What we want is a system that works for any use case, in any environment.

Software is full of bad abstractions. At Abstract, Convex's conference on the art of software, James Cowling opened with a talk on what makes a good abstraction. A good abstraction is simple and easy to use. That part is obvious. Then come three properties that are harder to get:

• Sound. Using the abstraction behaves the same as using what is under it, with no surprises in the corner cases.
• Instructive. Its shape tells you how to use it, so you rarely need the docs.
• Composable. You can change or use one part without affecting the others.

He called composability "the most important one and also the rarest one", and said that when a software product slows down, it is usually because it isn't composable. We agree.

In Billions of Machines we listed composition as one of the things an agent needs from a machine. The rest of this post is how we built it.

Everything is a setting

A boxd machine has many settings. This makes machines fully configurable. Below some examples of setting categories on a machine level:

You set them at three levels. The org sets the defaults every new machine gets, such as its size, region and backups, and the env vars and secrets its machines receive. A team can give its members one snapshot to start from. The machine itself can override any of them. A setting you leave out comes from the level above.

Access is role-based (RBAC). Org owners and admins set the org's defaults and manage members and teams. Team admins manage their own team. Members create and use machines.

You can change every setting from outside a machine, with the CLI, the SDKs, the console or the gRPC API. You also reach them from inside one, with its boxd CLI, signed in as the machine itself.

Define the machine

If you want to go beyond manual settings, you can also describe a whole machine, using a machine spec: one JSON value with 13 fields. It says what the machine boots from, its size, labels, networks, isolation, egress allowlist, env vars, secrets, the integrations and agent logins it needs, and the shell commands it runs on its first boot.

The builder was designed by Jonny Serban, one of our founding engineers.

You build a spec in code using a builder pattern. A builder makes a value step by step. You start from an empty spec and chain calls, and each call adds one part: where the machine starts, a network, an env var, an integration. The code reads as a list of what the machine has:

import {
  Boxd, Machine, team, github, slack, jira, claude, secrets,
} from "@boxd-sh/sdk";

const boxd = new Boxd({ apiKey: "bxd_..." });

// What every engineering machine has.
const engineering = Machine.spec()
  .from(team("eng"))   // the eng team's default snapshot
  .network("eng")
  .env("NODE_ENV", "development")
  .packages("apt-get update", "apt-get install -y nginx")
  .with(github({ shared: "eng-bot" }), claude());

// Jack's machine: the same, a size, and Jack's own extras.
const jack = engineering
  .size("2x8")
  .with(slack(), jira({ shared: "jira" }), secrets("NPM_TOKEN"));

await boxd.machines.create({ name: "jack-dev", org: "acme", spec: jack });

Every builder call returns a new builder, so engineering can grow in several directions without the branches affecting each other. A piece is plain data, so a team's own pieces are ordinary functions:

import type { MachineSpecPiece } from "@boxd-sh/sdk";

const service = (name: string): MachineSpecPiece => ({
  labels: { service: name },
  env: { SERVICE_NAME: name },
  packages: [`git clone https://github.com/acme/${name} /srv/${name}`],
});

const billing = engineering.with(service("billing"));

Building a spec vs. changing one

You work with a spec in two moments. Before a machine exists, you combine pieces, like engineering plus Jack's extras above. After it exists, you change its spec, for example to give it more memory or take it off a network. Combining can only add things while changing can also remove pieces.

Building adds up. When pieces merge, maps such as labels and env merge key by key, and the later piece overrules the earlier ones. Lists such as networks and packages add up, without duplicates. Two pieces that name different bases or sizes give an error, and so is a name that is both an env var and a secret. In this way, you cannot build a machine that contradicts itself.

Changing can remove things. updateSpec takes a JSON Merge Patch. A list in a patch replaces the old list, and null removes a field. That is how you take a machine off a network, which adding up never could.

Every change shows what it will change before anything is changed. A dry run lists each change and when it takes effect:

That is what we mean with being instructive. You don't need the docs to know what a change will do, because the change will tell it to you first.

An update applies completely or not at all. If someone else changes the spec while your update runs, nothing applies, and you get a conflict you can retry.

The server and both SDKs follow the same rules. If one of them drifts, a test fails. So a spec built on your laptop merges exactly as the server merges it.

The spec matches the machine

A spec is not a template that drifts away from its machine. The machine keeps its spec, and the spec follows the machine. Change its labels, networks, egress allowlist or size another way, from the CLI or the console, and the spec changes with it. The machine also keeps the spec it was created with, so you can always see what changed.

That is what sound means here: what the spec says is what the machine is.

Forks and snapshots keep the spec

A fork copies a running machine, memory and disk included, with a target of 100-200 ms. The fork gets the same spec, settings and egress allowlist as the machine it came from. If that machine has a delete-after timer, the fork gets its own timer, set the same way.

A snapshot keeps its machine's spec too. A machine restored from it gets that spec, with any spec you pass merged on top, by the same rules as pieces.

Three things work differently, on purpose. A machine created for another member doesn't pass that on to its copies. Secrets are looked up again for the new owner. A first-boot command that already ran never runs again.

From outside and from inside

Everything above also works from inside a machine. A private machine acts for its owner: it can create machines, fork itself, run commands on other machines, copy files to them and delete them. This automation runs the tests on a copy of the machine it runs on:

import { boxd } from "@boxd/run";

const me = process.env.BOXD_VM_NAME!;
const fork = await boxd.machines.fork(me, { name: `${me}-experiment` });
await boxd.machines.waitUntilReady(fork.id);
const r = await boxd.machines.exec(fork.id, {
  command: ["sh", "-c", "npm test"],
});
console.log(r.success ? "green on the fork" : r.stderr);
await boxd.machines.delete(fork.id);

So what stops a machine from giving itself more access? One rule: a machine can manage other machines, but it can't loosen its own limits.

From inside a machine, you can't change that machine's egress allowlist or secrets. You can't give any machine a secret that only reaches the machines a spec names, create a machine for someone else, create API keys, or change who is in the org. Those calls work only from outside, with an API key or in the console, and the error says so. A machine also can't stop, reboot or resize itself. It can delete or hibernate itself, but only with an explicit confirm.

What a call can manage follows the machine it runs on:

Machines that talk to each other

Machines reach each other in two ways. boxd machine exec runs a command on another machine, and boxd machine cp copies files to or from it. And inside any machine, <name>.boxd resolves to that machine's private IP. Only your own machines resolve.

By default, all your machines in an org share one private network. Using tag-based private networks, you can create networks with only one or more machines. You give a machine one or more network names, such as job-a. Two machines can then reach each other only if they share a name. A machine with a name leaves the default network, and machines without one stay in it together. The rule covers direct traffic, DNS, exec and cp alike, and a change applies at once.

So a coordinator can reach all of its workers, while the workers can't reach each other:

boxd machine new coordinator --networks=job-a,job-b
boxd machine new worker-a --networks=job-a
boxd machine new worker-b --networks=job-b

worker-a and worker-b share no network, so neither can reach the other. The coordinator shares one with each. An isolated machine goes further: it is in no default network, and it can never reach another isolated machine. Isolated machines are mostly used for sandbox use cases, such as running code you don't trust.

All of it together

If you combine all the above, you have a very composable system. To give you a small example, the automation below uses most of the above. When a pull request opens, it creates a machine from a snapshot, runs a review on it, posts the result and deletes the machine. A delete-after timer is the safety net in case something hangs.

import { boxd, githubApp } from "@boxd/run";

githubApp.on("pull_request.opened", { repo: "acme/api" }, async (pr) => {
  const machine = await boxd.machines.create({
    fromSnapshot: "api-main",
    name: `review-${pr.number}`,
    deleteAfter: 2 * 3600,   // gone in two hours, whatever happens
  });
  try {
    await boxd.machines.waitUntilReady(machine.id);
    const r = await boxd.machines.exec(machine.id, {
      command: ["sh", "-c", [
        `git fetch origin pull/${pr.number}/head`,
        "git checkout FETCH_HEAD",
        "claude -p 'Review the diff'",
      ].join(" && ")],
      timeout: 15 * 60_000,
    });
    const gh = await githubApp.client();
    await gh.rest.issues.createComment({
      owner: "acme", repo: "api", issue_number: pr.number, body: r.stdout,
    });
  } finally {
    await boxd.machines.delete(machine.id);
  }
});

The snapshot api-main was saved from a machine built from a spec. So every review machine starts with the repository and its dependencies in place, and with the spec the snapshot was saved with. The automation itself runs on a boxd machine, as its owner, with nothing to sign in to.

Next: a composable runtime

Everything above composes a machine. The next step is to make the runtime under the machines composable too.

A plugin can hook into the VMM that runs a machine, into what runs inside the machine, or into how boxd decides where a machine runs. Policies become plugins too. A policy plugin runs where boxd admits a new machine, and it can refuse any configuration you don't allow. Enterprise customers have plugins today, and they come to boxd managed soon.

The runtime under your application

By combining the two, we think we are moving toward our goal: give every builder access to the best runtime we can build, so they can focus on the application layer. A runtime anyone can get: on boxd managed, billed on what it uses, or on your own servers, free up to 16 vCPUs. And a runtime that fits any environment, because every part of a machine can be set on its own, described as data, changed from outside or inside, copied, and connected to other machines.

That leaves builders free to work on the application layer, on the best runtime we can make.

Michiel Voortman

PostShare⎘Copy link