Plan limits
- The box cap counts boxes, awake or asleep. At the cap,
make,exec, anddevfail withbox_limit.tesser rman idle box or upgrade. - Pool boxes belong to no repo. Any
make,exec, ordevin the org claims one of its size.
Spend limits
- A spend limit caps what boxes may spend in a month, including the usage
the plan includes. Limits are available on every plan. Admins set them with
tesser spend-limitor in the Spend limits table on the dashboard’s Billing tab. Members can see the org’s limit and their own. - A limit applies to the org or to one member. The org’s limit covers every box in the org. A member’s limit covers the boxes they own. Shared instances and pool boxes count only toward the org’s limit.
- At a limit, the awake boxes it covers sleep. Launches and wakes of those
boxes fail with
credit_exhausteduntil an admin raises the limit or the month resets. The message names the limit and the command that raises it. - A limit is between $0 and $100,000 a month.
- Limits on the Tesser bill reset with the org’s billing period. They count awake time and, for boxes in Tesser’s AWS, asleep storage.
- An org whose boxes run in its own AWS account can also set AWS compute
limits (
--aws). These count awake time at on-demand EC2 prices, so they leave out disks, traffic, and discounts. They reset on the 1st of each month (UTC).
Credit
- Usage is metered per second. The pricing page
has the rates.
tesser usageand the dashboard show what an org has used and how much credit is left. - The first org you create, on the page
tesser loginopens or on the dashboard, starts on Hobby with $5 of credit. An org you create while you are already an admin of another starts with none. - With no credit left, launches and wakes fail with
credit_exhausteduntil an admin upgrades on the dashboard’s Billing tab. An org on a trial instead sleeps until more credit is added or the trial ends. - When the credit, or a spend limit that covers your boxes, has less than
two hours left at the current rate,
make,dev,status, andsyncprint onetesser: warning:line to stderr: roughly when the boxes will sleep, and how to add credit or which limit an admin raises. Out of credit or at a limit, the same line says so. Nothing goes to stdout, so--jsonoutput stays valid.
Snapshots
What a box gets from one
- A snapshot is a disk image of a repo taken after every service’s
setupran. It holds/home/ubuntu,/var/lib/docker, and/var/lib/containerd. - A repo’s current snapshot is the newest ready one of a commit on the default branch. Every new box in the repo starts from it.
- A claimed pool box mounts the snapshot at claim time and starts while the rest fills in. If the mount fails, the box starts without it.
- The box reads the image as it needs it while the rest streams in, so it does not wait for the whole disk.
- The first sync sends only what changed since the snapshot’s commit, and
package stores and
node_modulesare already in place, so the first install is incremental. Without a snapshot, the firstsetupinstalls from scratch. Boxes never copy installs from each other. - Anything
setupinstalled outside/home/ubuntuand Docker’s data is not on the new box, except the manifest’saptpackages, which the box reinstalls on every start. - If a box cannot read its snapshot, it starts on an empty home directory.
tesser make --snapshot <snapshot_id>andtesser dev --snapshot <snapshot_id>start a new box from a specific snapshot of the repo.--no-snapshotstarts one without any. A shared instance fromtesser make <service> --ensure-running <sha>always starts from the repo’s current snapshot.
Seeing which snapshot a box has
tesser ls --jsonshows the snapshot each box started from.tesser status <box_id>prints it on asnapshotline as<snapshot_id>@<sha> (<age> old), with the first 12 characters of the commit, or-for a box without one. A claimed pool box shows-until its snapshot is mounted.
Building one
tesser snapshot create builds a snapshot of HEAD, or of --sha <commit>:
- A builder box launches at the largest size any of the repo’s services uses, up to large (standard on Hobby), starting from the repo’s current snapshot when there is one.
- It syncs the commit and runs each service’s
setupwith the values fromtesser env setand the committed files in itsenv.load. - It deletes
~/.tesser,~/.ssh/authorized_keys,~/.ssh/known_hosts,~/.bash_history,~/.gitconfig,~/.git-credentials,~/.netrc,~/.config/tesser,~/.config/gh,~/.aws,~/.claude,~/.claude.json, and~/.codex, so no one’s credentials reach other people’s boxes. - It captures the disk.
- A repo builds one snapshot at a time, and a build that runs past its time limit fails.
- The builder needs room under the org’s box cap.
--waitstreams the setup log from the builder over SSH until the snapshot is ready or fails. The log stays on the builder, so--waitis the only way to read it. Without--wait,tesser snapshot show <snapshot_id>shows the build’s state.
Automatic builds from tesser dev
When tesser dev runs for a service without --snapshot or
--no-snapshot, it checks the repo’s snapshots once it has a box, against the
default branch’s tip as your clone last fetched it (origin/HEAD, else
origin/main, else origin/master).
- If that commit needs a snapshot, it starts
tesser snapshot createin the background and printstesser: building a snapshot of <repo> at <sha> in the backgroundon stderr, with the path of its log (snapshot-build.login the state directory). tesser devdoes not wait for the build, and Ctrl-C does not stop it.- It builds nothing when:
- no service has a
setupat that commit - a snapshot of the repo is already building
- the repo has a current snapshot that is not stale
- the last build at that commit failed (it tries again once the tip moves)
- the newest ready snapshot of the default branch is at a commit the tip does not contain
- no service has a
- If a snapshot of the default branch is already building and
tesser devcreated a new box without a snapshot, it printstesser: <snapshot_id> of <repo> at <sha> is still building on <builder_box_id>, so <box_id> sets up without iton stderr, followed by thetesser snapshot show <snapshot_id>command. That box runssetupfrom scratch, and boxes started once the snapshot is ready start from it.tesser dev a bprints one line for each new box.tesser make <service> --ensure-running <sha>prints the same line when it creates the shared box. A reused box, a box that started from an older snapshot, and a build of another branch get no line. - A background build leaves room for one more box under the box cap. It stops quietly when the org has no room, is out of credit, has reached a plan limit, or keeps no snapshots.
tesser dev waits for the server and setup took 3 minutes or more,
it prints a hint after the box id:
- If the service has no shared instance, it names the
tesser make <service> --ensure-running <sha>command that would share one box’s setup with everyone. - If the default branch’s tip needs a snapshot, it names the
tesser snapshot create --sha <sha>command that builds one. If a snapshot of the default branch is already building, it names that snapshot instead. It skips this line when the same run already started a build or printed a still-building line.
Listing, staleness, and removal
- Inside a repo,
tesser snapshot lslists that repo’s snapshots and printsthis repo (<repo>): nonewhen it has none. Outside a repo, it lists the org’s.--alllists every repo’s, and--removedincludes removed and purged ones. - A snapshot of a commit not on the default branch is marked
branch <name>, with the branch that was checked out when it was built. A snapshot built from a detached HEAD or with--shaof a commit other than HEAD has no branch name and is markedbranch ?. New boxes start from a branch snapshot only with--snapshot. tesser env setmarks the repo’s snapshotsstale, because their setup ran with the old values. A stale snapshot is still used until a new one is built.- Tesser keeps the newest ready snapshot of the default branch, and the newest ready snapshot of each other branch. Snapshots with no branch name share one slot, so only the newest of them is kept.
- A branch snapshot stops being kept a fixed time after it was ready or after
a box last started from it with
--snapshot, whichever is later (see Timers). - Tesser deletes snapshots it no longer keeps once no box is reading from
them.
tesser snapshot rmremoves one sooner, and boxes made from it keep running. - An org that runs boxes in its own AWS account keeps snapshots in its own S3 bucket and has none until it names one.
Where a box runs
In Tesser’s cloud, a new box is a microVM on a host that Tesser runs. A new box is its own EC2 instance instead when no host has room for it. Tesser also starts another host for the boxes made after it. A new box that goes to EC2 takes an unclaimed pool box of its size when one is free, and launches a new instance only when none is. The org’stesser pool target keeps pool boxes warm only while none of its microVMs is
awake or starting. When one of its microVMs wakes or starts, or a box moves
onto a host from EC2, Tesser removes the unclaimed pool boxes the org no
longer needs right away. A pool box that a new box is claiming at that moment
stays. Otherwise an unclaimed pool box is removed on its timer (see
Timers).
An org’s microVMs and EC2 boxes reach each other on the mesh like any other
boxes.
Everything else on this page applies to both kinds unless it names one.
Sleep and wake
What counts as idle
A box is in use while any of these is true:- a visible browser tab is showing it
- something other than a browser sends it requests (curl, a script, a test run)
- a CLI command targets it
- an SSH session is open where someone is typing, or something is running or printing
- Traffic between boxes does not count on its own, but a box in use keeps the boxes it depends on awake, so a server and its database idle together.
- A kept box keeps the boxes it depends on awake while its server runs. Once its server stops, they go back to their own idle clock.
- When an instance box starts or wakes, it wakes the boxes it depends on that you own or that are the org’s shared instances. A dependency wired to a teammate’s box is never woken this way.
What survives sleep
- A sleeping box keeps its disk and its id. Any command that targets it wakes it first, and opening it on localhost shows a Wake button.
- A request to a sleeping box’s localhost address from curl or a script does
not wake it. It answers 503 with a line that names the command to run, such
as
web is asleep; run `tesser wake box_a1b2c3` or open it in a browser to wake it. - Sleep is hibernation: memory is written to disk, and on wake processes
carry on where they stopped. A dev server keeps its state and does not run
setupagain. An EC2 box writes its memory to its own disk. A microVM’s memory is saved as a snapshot beside its disk on its host. - When the request to start a sleeping EC2 box fails, the box keeps trying until the wake gives up (see Timers). A wake that gives up leaves the box asleep with its disk, and its removal clock starts again. A wake refused because of credit or a plan limit ends at once.
- When AWS has no capacity for a sleeping EC2 box’s size, the box keeps
trying for a short time (see Timers). The wake then ends
with
fleet_fulland the message<box_id> can't start: AWS has no <instance type> capacity in its zone right now; try again in a few minutes. The box stays asleep. A wake asked for right after that fails at once with the same message. - When AWS refuses to start a sleeping EC2 box because the account’s
on-demand vCPU quota is used up, the wake also ends at once with
vcpu_quota. The message names the quota and the on-demand vCPUs already in use, which include instances that are still stopping or shutting down. The box stays asleep. Sleep or remove boxes, or raise “Running On-Demand Standard instances” in Service Quotas, then wake it again. - After a wake, boxd waits for the health check (see Timers).
A server that does not answer in time is started again from
setup. - A wake that updates boxd to a newer version keeps the dev server running: the new boxd takes over the same process. If the new version fails and boxd goes back to the previous one, the dev server starts again.
- A dev server that exited before the box went to sleep stays exited after the wake.
instance.startedAtintesser status <box_id> --jsonis when the current dev process started, so you can tell whether a wake restarted it.- Connections between boxes close on wake, so your app has to reconnect to its dependencies.
- These boxes get a plain power-off instead, and their dev server starts
again on wake:
- a box still streaming in its snapshot
- a box put to sleep in its first minute
- a box whose image or AWS settings do not support hibernation
- a microVM that moved from its host to EC2 while awake, the first time it sleeps on EC2, when EC2 will not hibernate its instance or its disk has no room for its memory
Sleep and wake on a microVM
- A sleeping microVM can wake on a different host. It keeps its id and its private address. A wake on another host takes longer than one on the same host.
- A sleeping microVM keeps its memory for a limited time (see Timers). After that, Tesser drops its memory and keeps only its disk. The next wake starts from the disk: its processes do not resume, and the dev server starts again, as after a plain power-off.
- An awake microVM can also be moved to another host while it runs. It
pauses briefly and keeps its id, its address, and one continuous awake
period. An interactive
tesser sshshell keeps running and reconnects when the move ends. Connections the box opened to the internet stall until the client reconnects, and connections between boxes close, as on any wake. - A microVM that moved from its host to EC2 while awake hibernates on EC2 and keeps its memory. It writes its memory to a file on its own disk, which needs half the box’s memory plus 2 GiB free.
- EC2 cannot hibernate an instance until a few minutes after it launched.
Until then, an idle box that moved to EC2 stays awake and tries again,
and a
tesser sleepgives it a plain power-off. - On a fleet in your own AWS account, when no running host has room for a wake, the box wakes on EC2 instead of waiting for a new host. It wakes from its disk and keeps its id and its private address, but its processes do not resume: the dev server starts again, as after a plain power-off. The box then runs on EC2.
- When the box cannot wake on EC2, Tesser starts a host and the wake waits for it. When the fleet has no running host left at all, Tesser also starts one and the wake waits for it.
- While every running host in your account is draining and no host is starting, a wake of a box on one of them goes to EC2 instead of stopping the drain or starting a host. When the box cannot wake on EC2, it wakes on its draining host if it fits there, and Tesser starts a host only when none is running.
- On Tesser’s hosted fleet, a wake that finds no room starts a host and waits for it. The box wakes on EC2 only when no host can start.
- When no host can start and the box cannot wake on EC2 either, the wake
fails with
fleet_fulland the messagethe fleet has no room for <box_id> right now, and it can't wake on EC2 instead (<reason>); try again in a few minutes. The box stays asleep on the fleet. After a failed wake on EC2, Tesser does not try EC2 again for that box for a while (see Timers). - When a host stops answering while a box on it is awake, Tesser puts the box to sleep (see Timers), and the box wakes again once the host is back. A box that was awake on a host that never comes back cannot be woken.
Finding and inspecting boxes
tesser ls
- Piped, one line per box: kind, service, size, power state, whether it is online, when it was made, its IP address, and the local worktree it was made from.
- In a terminal, a live table with service, worktree, owner, size, state,
created, idle, and box columns. Press a column’s number or click its header
to sort, press
/to fuzzy search, and click a box twice to use it on localhost. - A worktree shows as its folder name, followed by its repository in
parentheses when the two differ, such as
src (zulip). The repository name is the last part of the worktree’soriginURL, or the repository’s folder when there is noorigin. tesser status <box_id>adds the instance id, IP addresses, and when it was last active.
tesser ssh and tesser exec
- When a command after
--exits 127 and its command word still has quotes or spaces in it, both print one extra line on stderr explaining why. The exit code is passed through unchanged. tesser execalways runs without a terminal.tesser ssh <box_id> <command>gets one only from a terminal (seetesser ssh).
Without a terminal
- The command runs in its own session on the box. Its output also goes to
/tmp/tesser-exec/<id>/outanderr, and its exit code tocode. - When the ssh connection drops, the command keeps running. The CLI prints
lost the ssh connection to <box_id>; reconnecting for up to 120s, retries until it gets back in (see Timers), then streams the rest of the output from where it stopped. Each byte arrives once, and the exit code is the command’s own.
- Input piped into the command is still delivered after a drop only if it reached the box first. Input still in flight is lost, and the command gets end of file. When input had arrived and stdin was still open, a line says the input was cut short.
- The next run without a terminal removes old run directories whose process is gone (see Timers).
With a terminal
-
The box records the exit code in
/tmp/tesser-exec/<id>/code. The file is empty while the command runs. -
When ssh exits 255 and you did not interrupt it, the CLI reconnects once,
reads that file, and prints one line on stderr:
When the command itself exited 255, nothing is printed.
tesser util
- The box samples CPU and memory every second and reports a summary each minute.
- Each row is one step: mean, 95th percentile, and peak CPU as a percentage of all the box’s vCPUs, plus mean, peak, and total memory in GiB.
--sincetakes values like30m,12h, or90d, and defaults to an hour. Longer windows use wider steps so the output stays at 360 rows or fewer.- History is kept 90 days and is gone once the box is removed.
--jsonalso includes CPU and memory pressure.- With no box id, it covers the whole org: one row per service, or the
workbenches, per size, most spent first, with each size’s price per awake
hour.
tesser util --helpexplains each field and how to right-size an org from them.