Skip to main content
The exact rules for boxes, for looking things up. What a single command does is in the CLI reference, and timers and box sizes are in Configuration. The Boxes page explains the ideas.

Plan limits

  • The box cap counts boxes, awake or asleep. At the cap, make, exec, and dev fail with box_limit. tesser rm an idle box or upgrade.
  • Pool boxes belong to no repo. Any make, exec, or dev in the org claims one of its size.

Spend limits

  • A spend limit caps what boxes may spend in a month, including the usage the plan includes. Limits are available on every plan. Admins set them with tesser spend-limit or in the Spend limits table on the dashboard’s Billing tab. Members can see the org’s limit and their own.
  • A limit applies to the org or to one member. The org’s limit covers every box in the org. A member’s limit covers the boxes they own. Shared instances and pool boxes count only toward the org’s limit.
  • At a limit, the awake boxes it covers sleep. Launches and wakes of those boxes fail with credit_exhausted until an admin raises the limit or the month resets. The message names the limit and the command that raises it.
  • A limit is between $0 and $100,000 a month.
  • Limits on the Tesser bill reset with the org’s billing period. They count awake time and, for boxes in Tesser’s AWS, asleep storage.
  • An org whose boxes run in its own AWS account can also set AWS compute limits (--aws). These count awake time at on-demand EC2 prices, so they leave out disks, traffic, and discounts. They reset on the 1st of each month (UTC).

Credit

  • Usage is metered per second. The pricing page has the rates. tesser usage and the dashboard show what an org has used and how much credit is left.
  • The first org you create, on the page tesser login opens or on the dashboard, starts on Hobby with $5 of credit. An org you create while you are already an admin of another starts with none.
  • With no credit left, launches and wakes fail with credit_exhausted until an admin upgrades on the dashboard’s Billing tab. An org on a trial instead sleeps until more credit is added or the trial ends.
  • When the credit, or a spend limit that covers your boxes, has less than two hours left at the current rate, make, dev, status, and sync print one tesser: warning: line to stderr: roughly when the boxes will sleep, and how to add credit or which limit an admin raises. Out of credit or at a limit, the same line says so. Nothing goes to stdout, so --json output stays valid.

Snapshots

What a box gets from one

  • A snapshot is a disk image of a repo taken after every service’s setup ran. It holds /home/ubuntu, /var/lib/docker, and /var/lib/containerd.
  • A repo’s current snapshot is the newest ready one of a commit on the default branch. Every new box in the repo starts from it.
  • A claimed pool box mounts the snapshot at claim time and starts while the rest fills in. If the mount fails, the box starts without it.
  • The box reads the image as it needs it while the rest streams in, so it does not wait for the whole disk.
  • The first sync sends only what changed since the snapshot’s commit, and package stores and node_modules are already in place, so the first install is incremental. Without a snapshot, the first setup installs from scratch. Boxes never copy installs from each other.
  • Anything setup installed outside /home/ubuntu and Docker’s data is not on the new box, except the manifest’s apt packages, which the box reinstalls on every start.
  • If a box cannot read its snapshot, it starts on an empty home directory.
  • tesser make --snapshot <snapshot_id> and tesser dev --snapshot <snapshot_id> start a new box from a specific snapshot of the repo. --no-snapshot starts one without any. A shared instance from tesser make <service> --ensure-running <sha> always starts from the repo’s current snapshot.

Seeing which snapshot a box has

  • tesser ls --json shows the snapshot each box started from.
  • tesser status <box_id> prints it on a snapshot line as <snapshot_id>@<sha> (<age> old), with the first 12 characters of the commit, or - for a box without one. A claimed pool box shows - until its snapshot is mounted.

Building one

tesser snapshot create builds a snapshot of HEAD, or of --sha <commit>:
  1. A builder box launches at the largest size any of the repo’s services uses, up to large (standard on Hobby), starting from the repo’s current snapshot when there is one.
  2. It syncs the commit and runs each service’s setup with the values from tesser env set and the committed files in its env.load.
  3. It deletes ~/.tesser, ~/.ssh/authorized_keys, ~/.ssh/known_hosts, ~/.bash_history, ~/.gitconfig, ~/.git-credentials, ~/.netrc, ~/.config/tesser, ~/.config/gh, ~/.aws, ~/.claude, ~/.claude.json, and ~/.codex, so no one’s credentials reach other people’s boxes.
  4. It captures the disk.
  • A repo builds one snapshot at a time, and a build that runs past its time limit fails.
  • The builder needs room under the org’s box cap.
  • --wait streams the setup log from the builder over SSH until the snapshot is ready or fails. The log stays on the builder, so --wait is the only way to read it. Without --wait, tesser snapshot show <snapshot_id> shows the build’s state.

Automatic builds from tesser dev

When tesser dev runs for a service without --snapshot or --no-snapshot, it checks the repo’s snapshots once it has a box, against the default branch’s tip as your clone last fetched it (origin/HEAD, else origin/main, else origin/master).
  • If that commit needs a snapshot, it starts tesser snapshot create in the background and prints tesser: building a snapshot of <repo> at <sha> in the background on stderr, with the path of its log (snapshot-build.log in the state directory).
  • tesser dev does not wait for the build, and Ctrl-C does not stop it.
  • It builds nothing when:
    • no service has a setup at that commit
    • a snapshot of the repo is already building
    • the repo has a current snapshot that is not stale
    • the last build at that commit failed (it tries again once the tip moves)
    • the newest ready snapshot of the default branch is at a commit the tip does not contain
  • If a snapshot of the default branch is already building and tesser dev created a new box without a snapshot, it prints tesser: <snapshot_id> of <repo> at <sha> is still building on <builder_box_id>, so <box_id> sets up without it on stderr, followed by the tesser snapshot show <snapshot_id> command. That box runs setup from scratch, and boxes started once the snapshot is ready start from it. tesser dev a b prints one line for each new box. tesser make <service> --ensure-running <sha> prints the same line when it creates the shared box. A reused box, a box that started from an older snapshot, and a build of another branch get no line.
  • A background build leaves room for one more box under the box cap. It stops quietly when the org has no room, is out of credit, has reached a plan limit, or keeps no snapshots.
When tesser dev waits for the server and setup took 3 minutes or more, it prints a hint after the box id:
  • If the service has no shared instance, it names the tesser make <service> --ensure-running <sha> command that would share one box’s setup with everyone.
  • If the default branch’s tip needs a snapshot, it names the tesser snapshot create --sha <sha> command that builds one. If a snapshot of the default branch is already building, it names that snapshot instead. It skips this line when the same run already started a build or printed a still-building line.

Listing, staleness, and removal

  • Inside a repo, tesser snapshot ls lists that repo’s snapshots and prints this repo (<repo>): none when it has none. Outside a repo, it lists the org’s. --all lists every repo’s, and --removed includes removed and purged ones.
  • A snapshot of a commit not on the default branch is marked branch <name>, with the branch that was checked out when it was built. A snapshot built from a detached HEAD or with --sha of a commit other than HEAD has no branch name and is marked branch ?. New boxes start from a branch snapshot only with --snapshot.
  • tesser env set marks the repo’s snapshots stale, because their setup ran with the old values. A stale snapshot is still used until a new one is built.
  • Tesser keeps the newest ready snapshot of the default branch, and the newest ready snapshot of each other branch. Snapshots with no branch name share one slot, so only the newest of them is kept.
  • A branch snapshot stops being kept a fixed time after it was ready or after a box last started from it with --snapshot, whichever is later (see Timers).
  • Tesser deletes snapshots it no longer keeps once no box is reading from them. tesser snapshot rm removes one sooner, and boxes made from it keep running.
  • An org that runs boxes in its own AWS account keeps snapshots in its own S3 bucket and has none until it names one.

Where a box runs

In Tesser’s cloud, a new box is a microVM on a host that Tesser runs. A new box is its own EC2 instance instead when no host has room for it. Tesser also starts another host for the boxes made after it. A new box that goes to EC2 takes an unclaimed pool box of its size when one is free, and launches a new instance only when none is. The org’s tesser pool target keeps pool boxes warm only while none of its microVMs is awake or starting. When one of its microVMs wakes or starts, or a box moves onto a host from EC2, Tesser removes the unclaimed pool boxes the org no longer needs right away. A pool box that a new box is claiming at that moment stays. Otherwise an unclaimed pool box is removed on its timer (see Timers). An org’s microVMs and EC2 boxes reach each other on the mesh like any other boxes. Everything else on this page applies to both kinds unless it names one.

Sleep and wake

What counts as idle

A box is in use while any of these is true:
  • a visible browser tab is showing it
  • something other than a browser sends it requests (curl, a script, a test run)
  • a CLI command targets it
  • an SSH session is open where someone is typing, or something is running or printing
A background tab, or an open shell with nothing running, does not count.
  • Traffic between boxes does not count on its own, but a box in use keeps the boxes it depends on awake, so a server and its database idle together.
  • A kept box keeps the boxes it depends on awake while its server runs. Once its server stops, they go back to their own idle clock.
  • When an instance box starts or wakes, it wakes the boxes it depends on that you own or that are the org’s shared instances. A dependency wired to a teammate’s box is never woken this way.

What survives sleep

  • A sleeping box keeps its disk and its id. Any command that targets it wakes it first, and opening it on localhost shows a Wake button.
  • A request to a sleeping box’s localhost address from curl or a script does not wake it. It answers 503 with a line that names the command to run, such as web is asleep; run `tesser wake box_a1b2c3` or open it in a browser to wake it.
  • Sleep is hibernation: memory is written to disk, and on wake processes carry on where they stopped. A dev server keeps its state and does not run setup again. An EC2 box writes its memory to its own disk. A microVM’s memory is saved as a snapshot beside its disk on its host.
  • When the request to start a sleeping EC2 box fails, the box keeps trying until the wake gives up (see Timers). A wake that gives up leaves the box asleep with its disk, and its removal clock starts again. A wake refused because of credit or a plan limit ends at once.
  • When AWS has no capacity for a sleeping EC2 box’s size, the box keeps trying for a short time (see Timers). The wake then ends with fleet_full and the message <box_id> can't start: AWS has no <instance type> capacity in its zone right now; try again in a few minutes. The box stays asleep. A wake asked for right after that fails at once with the same message.
  • When AWS refuses to start a sleeping EC2 box because the account’s on-demand vCPU quota is used up, the wake also ends at once with vcpu_quota. The message names the quota and the on-demand vCPUs already in use, which include instances that are still stopping or shutting down. The box stays asleep. Sleep or remove boxes, or raise “Running On-Demand Standard instances” in Service Quotas, then wake it again.
  • After a wake, boxd waits for the health check (see Timers). A server that does not answer in time is started again from setup.
  • A wake that updates boxd to a newer version keeps the dev server running: the new boxd takes over the same process. If the new version fails and boxd goes back to the previous one, the dev server starts again.
  • A dev server that exited before the box went to sleep stays exited after the wake.
  • instance.startedAt in tesser status <box_id> --json is when the current dev process started, so you can tell whether a wake restarted it.
  • Connections between boxes close on wake, so your app has to reconnect to its dependencies.
  • These boxes get a plain power-off instead, and their dev server starts again on wake:
    • a box still streaming in its snapshot
    • a box put to sleep in its first minute
    • a box whose image or AWS settings do not support hibernation
    • a microVM that moved from its host to EC2 while awake, the first time it sleeps on EC2, when EC2 will not hibernate its instance or its disk has no room for its memory

Sleep and wake on a microVM

  • A sleeping microVM can wake on a different host. It keeps its id and its private address. A wake on another host takes longer than one on the same host.
  • A sleeping microVM keeps its memory for a limited time (see Timers). After that, Tesser drops its memory and keeps only its disk. The next wake starts from the disk: its processes do not resume, and the dev server starts again, as after a plain power-off.
  • An awake microVM can also be moved to another host while it runs. It pauses briefly and keeps its id, its address, and one continuous awake period. An interactive tesser ssh shell keeps running and reconnects when the move ends. Connections the box opened to the internet stall until the client reconnects, and connections between boxes close, as on any wake.
  • A microVM that moved from its host to EC2 while awake hibernates on EC2 and keeps its memory. It writes its memory to a file on its own disk, which needs half the box’s memory plus 2 GiB free.
  • EC2 cannot hibernate an instance until a few minutes after it launched. Until then, an idle box that moved to EC2 stays awake and tries again, and a tesser sleep gives it a plain power-off.
  • On a fleet in your own AWS account, when no running host has room for a wake, the box wakes on EC2 instead of waiting for a new host. It wakes from its disk and keeps its id and its private address, but its processes do not resume: the dev server starts again, as after a plain power-off. The box then runs on EC2.
  • When the box cannot wake on EC2, Tesser starts a host and the wake waits for it. When the fleet has no running host left at all, Tesser also starts one and the wake waits for it.
  • While every running host in your account is draining and no host is starting, a wake of a box on one of them goes to EC2 instead of stopping the drain or starting a host. When the box cannot wake on EC2, it wakes on its draining host if it fits there, and Tesser starts a host only when none is running.
  • On Tesser’s hosted fleet, a wake that finds no room starts a host and waits for it. The box wakes on EC2 only when no host can start.
  • When no host can start and the box cannot wake on EC2 either, the wake fails with fleet_full and the message the fleet has no room for <box_id> right now, and it can't wake on EC2 instead (<reason>); try again in a few minutes. The box stays asleep on the fleet. After a failed wake on EC2, Tesser does not try EC2 again for that box for a while (see Timers).
  • When a host stops answering while a box on it is awake, Tesser puts the box to sleep (see Timers), and the box wakes again once the host is back. A box that was awake on a host that never comes back cannot be woken.

Finding and inspecting boxes

tesser ls

  • Piped, one line per box: kind, service, size, power state, whether it is online, when it was made, its IP address, and the local worktree it was made from.
  • In a terminal, a live table with service, worktree, owner, size, state, created, idle, and box columns. Press a column’s number or click its header to sort, press / to fuzzy search, and click a box twice to use it on localhost.
  • A worktree shows as its folder name, followed by its repository in parentheses when the two differ, such as src (zulip). The repository name is the last part of the worktree’s origin URL, or the repository’s folder when there is no origin.
  • tesser status <box_id> adds the instance id, IP addresses, and when it was last active.

tesser ssh and tesser exec

  • When a command after -- exits 127 and its command word still has quotes or spaces in it, both print one extra line on stderr explaining why. The exit code is passed through unchanged.
  • tesser exec always runs without a terminal. tesser ssh <box_id> <command> gets one only from a terminal (see tesser ssh).

Without a terminal

  • The command runs in its own session on the box. Its output also goes to /tmp/tesser-exec/<id>/out and err, and its exit code to code.
  • When the ssh connection drops, the command keeps running. The CLI prints lost the ssh connection to <box_id>; reconnecting for up to 120s, retries until it gets back in (see Timers), then streams the rest of the output from where it stopped. Each byte arrives once, and the exit code is the command’s own.
  • Input piped into the command is still delivered after a drop only if it reached the box first. Input still in flight is lost, and the command gets end of file. When input had arrived and stdin was still open, a line says the input was cut short.
  • The next run without a terminal removes old run directories whose process is gone (see Timers).

With a terminal

  • The box records the exit code in /tmp/tesser-exec/<id>/code. The file is empty while the command runs.
  • When ssh exits 255 and you did not interrupt it, the CLI reconnects once, reads that file, and prints one line on stderr: When the command itself exited 255, nothing is printed.

tesser util

  • The box samples CPU and memory every second and reports a summary each minute.
  • Each row is one step: mean, 95th percentile, and peak CPU as a percentage of all the box’s vCPUs, plus mean, peak, and total memory in GiB.
  • --since takes values like 30m, 12h, or 90d, and defaults to an hour. Longer windows use wider steps so the output stays at 360 rows or fewer.
  • History is kept 90 days and is gone once the box is removed.
  • --json also includes CPU and memory pressure.
  • With no box id, it covers the whole org: one row per service, or the workbenches, per size, most spent first, with each size’s price per awake hour. tesser util --help explains each field and how to right-size an org from them.