Live files protocol (Phase 3 contract)
Status: CONTRACT, built in Phase 3 (the "as built" notes under section 15 say where the code differs). It is the exact interface that several people build against in parallel for Phase 3 of Long-lived computer, file protocol and preview: the shared schemas (F0), the server (A1, A2), the container (C) and the web app (W). It builds on the merged and live Owner supervisor protocol (Phase 2a) and on the read-only snapshot file routes of Phase 1. Every claim about today's code carries a
file:lineor file reference intoorigin/developat the time of writing. Every statement about the proposal is the contract; a guess is labelled ASSUMPTION. The wire schemas in section 5 are code to be added inhost-protocol/src/live-files.tsandhost-protocol/src/live-file-rules.tsby the foundation step F0; until F0 merges, this page is the source. Once it merges, where this page and those files disagree, fix the page. The draft behavior is infeatures/computer_file_live_member.featureand the othercomputer_file_*.featurefiles.Revision 2 (owner decisions). The owner decided that a computer is needed only to run things, never to view or edit files: a person reads and edits files while the computer is off, an open Files panel neither starts nor keeps awake a computer, and the interactive pace is 1 second. That replaces the first revision's refusal of a write while the computer is off with an edit overlay in the snapshot bucket (sections 3 O1 to O4 and L30 to L47, 5.5, 7.7, 9.4, 11.7, 12.8, 19). Everything else in this page stands unless a section says otherwise.
Revision 3 (owner decisions). Delete, rename and new folders are required in Phase 3, including while the computer is off. The overlay therefore records four kinds of operation (put, delete, mkdir, move), and the owner applies them at hydrate in a defined order (section 7.8; decisions L48 to L66; schemas 5.2 and 5.5; helper operations 8.1). Also settled: non-current bucket versions expire after 30 days (part B, section 12.8), customer-account computers stay read-only for stated reasons (L46) with a later-phase ticket, any enabled member may discard a pending change, and
LocalWorkerRoleis left as it is.
1. What Phase 3 is
A member of the organization opens the Files panel and sees the files as they are now, and can change a text file. The computer is needed only to run things, never to view or edit files (owner decision O1). Which source answers depends on the computer:
| Computer | Reads | Edits |
|---|---|---|
| Running: an owner of protocol 2 that has finished hydrating and is serving | the owner's disk, through the poll channel | through the owner: an atomic write on the live disk, with a hash check |
| Off, or an owner that is still booting or extracting the snapshot | the saved snapshot with the edit overlay merged over it | into the overlay: one small object per edited file in the snapshot bucket, applied by the next owner after it extracts the snapshot and before any turn runs |
| An owner that has sealed the overlay but cannot serve yet (applying) or any more (stopping) | snapshot with the overlay merged, marked stale | refused for a few seconds with a retry time; the person's text is kept |
Viewing and editing never start a computer. An open panel does not keep one awake.
Browser (Files panel)
| GET/PUT/POST /orgs/T/computer/files/...
v
Front Door Lambda
| 1. member principal, rate limit, path rules
| 2. choose the source (7.2): live owner able to serve? else snapshot + overlay
| live: audit first, queue a request in the table (one transaction with the stop guard), wait up to 8 s
| overlay: audit first, put the blob, compare-and-swap the overlay index in S3 (If-Match)
v
Messaging table T#computer#fileops T#computer#audit Snapshot bucket tenants/T/computers/K/
^ snapshot.tar.gz index/ manifest.json
| poll response carries file_requests; overlay/index.json overlay/blobs/{sha256}
| POST /host/owner/file-results carries the answer ^ ^
| | reads | the Front Door alone writes and deletes
Supervisor (root, in the container) ---- hydrate: extract, seal, apply through the helper, publish, ask to clear
| spawns, with a timeout and a concurrency of 2
v
File helper (uid 2000, through chatticus-shell) list | stat | read(range) | write(atomic, precondition)
workspace
What does not change: no socket, no inbound port, no new AWS resource. What does change in
infrastructure (revision 2): the Front Door role gains PutObject and DeleteObject on the overlay
prefix, and the supervisor role gets an explicit deny of writes to that prefix (section 12.8).
1.1 Out of Phase 3 (reserved, nothing implemented)
| Reserved | Phase | Rule until then |
|---|---|---|
GET .../files/recent and watcher-based change events |
5 | The panel reacts to the turn's tool events and to a cheap stat probe (section 10). |
| Overwriting rename (rename onto an existing path), upload, download of large files, symlink creation, permission changes | later | Delete, rename and new folders are in Phase 3 (7.8); a rename never replaces an existing path and a link cannot be created. |
| Writing a binary file or a file over 256 KiB | later | Writes are UTF-8 text up to 256 KiB. |
| Waking a stopped computer from the Files panel | never (owner decision O2) | Viewing and editing never start a computer. |
| Customer-account computers | later phase (Kanbus ticket) | Read-only, for the reasons in L46: their host runs in the customer's AWS account through the older host-worker path and the customer-owned bucket, the Front Door has no write grant there and no owner applies an overlay. An edit is refused with 409 edit_unsupported. |
| Previews, background processes | 4 | |
| Reading the audit log from the web app | decided (O4) | Operators read the table. |
2. What is wrong or underspecified in the design, given the code now live
The design document is from before Phases 1 and 2a. These are the places where it is wrong,
silent or contradicted by origin/develop. Each one is settled by a decision in section 3
(L-numbers) unless it says "open".
| # | What the design says or omits | What the code says | Settled by |
|---|---|---|---|
| W1 | 7.1: "If the computer record says a healthy owner is live, use the owner." | liveOwnerOf (domain/computers.ts:119) is true for an owner that is booting, hydrating, serving, draining or stopping. A live read of a hydrating disk returns an empty or partial workspace; a write to a stopping owner is lost after the final publish. |
L2, L3 |
| W2 | 7.5: "the open file panel keeps it in the 0.5 second mode." No signal path is named. | The poll request is strict and carries nothing about a panel (ownerPollRequestSchema); the server learns of a panel only from requests it sees. At the 10 second idle tier, the first request after a quiet time is picked up 0 to 10 seconds later, so about one in five first reads times out against an 8 second wait. |
L6, L7 |
| W3 | 9 and 6.1: the 0.5 second mode costs 0.0054 USD/h. | OWNER_SUPERVISOR_PROTOCOL C13 revised the poll cost to one conditional write plus two or three reads: about 0.7 cents an hour at 1 s, so about 1.4 cents an hour at 0.5 s, which is the cost of the computer itself (about 1.5 cents an hour with its address). The design's number counted reads only. | L7, O3 |
| W4 | 7.5: items "TTL 120 seconds". | DynamoDB expires items by TTL lazily, typically within 48 hours (the table uses expires_at, infra/lib/thin-turn-stack.ts:50). A revealed secret, or the content of an edit, would sit in the table long after "expiry". |
L10 |
| W5 | 7.5: result route /host/owner/fileops/{opId}/result. |
authenticateOwnerHostRequest matches the owner routes by exact suffix (OWNER_COMPUTER_ROUTES, http/owner-host-auth.ts:27); a path parameter would not match and would fall through to the invoke key gate. |
L8 |
| W6 | 7.5: the Front Door "waits up to 8 seconds" and the response follows. Silent on what a timeout means for a write. | A timeout does not mean the owner did not run the request. A write the owner already took can still land. | L9 |
| W7 | 6.4 and 7.5 list activity that resets the idle clock "a file request". | The stop predicate (stopVeto, domain/owner-supervisor.ts) knows parked work and live turns only, and the supervisor's idle clock restarts only at a child exit (startIdleClock in onChildExit, supervisor.ts). A file edit would neither veto an idle stop nor postpone it, and a request queued the instant a stop is approved would be lost with its write. |
L11, L12 |
| W8 | 6.6: "Quiet publish 30 seconds after the last write ... a file edit landed." | The supervisor schedules a publish check only after hydration, after a child exit and after a publish (nextPublishCheckAtMilliseconds); it never publishes while a child runs (OWNER_SUPERVISOR_PROTOCOL C14). An edit alone would wait up to 10 minutes, and an edit during a long turn would wait for the turn. |
L13, O4 |
| W9 | 7.5 is silent on the dirty flag. | disk_dirty is set only when a disk-writing tool answers (recordComputerToolAnswered, domain/computers.ts:431), and a dead owner's record is settled only when the disk is dirty (settleComputerIfHostLost, :235). An edit by a person on a dead owner would be lost silently and unrecorded. |
L14 |
| W10 | 7.5: "the same helper started through the shell launcher". | The helper has read (whole file, base64), write (in place, mkdir -p, no precondition, not atomic), stat, realpath and no listing, no range, no hash, no size cap (pi/owner-file-helper.ts). A FIFO or a device would hang a read; a 2 GiB file would exhaust 512 MiB. A reader can see a half-written file. |
L15, L16 |
| W11 | 7.5: the supervisor runs the operation. | After every child exit and in the stop sequence the supervisor kills every process of uid 2000 (UnprivilegedProcessSweeper, supervisor.ts onChildExit, performStop). The helper runs as that uid, so a sweep would kill an operation in flight. |
L17 |
| W12 | 7.6: secret names .env*, *.pem, id_*, .aws/credentials, .git-credentials, .npmrc; "a reveal call". |
Phase 1 has a different list (snapshot/files.ts SECRET_NAME_PATTERNS: adds .key, .p12, .pfx, credentials, .netrc, the whole of .ssh and .aws; lacks .git-credentials) and no reveal. Two lists in two places is a defect. |
L18 |
| W13 | Silent on the record. | recordHostSnapshotPublished and recordHostHydrated read the whole computer record and putComputer it back (domain/computers.ts:343-391, dynamo-messaging-store.ts:525). Any field a conditional update wrote between the read and the put (a dispatch, a stop approval, and now the interactive window) is reverted. A quiet publish after every edit makes this race frequent. |
L19 |
| W14 | 7.5: "up to 256 KiB inline". | A DynamoDB item is limited to 400 KB. 256 KiB of binary is 349,528 bytes of base64 plus attributes: it fits with about 50 KB to spare, and a write of text up to 256 KiB fits easily, but the cap is a ceiling the item size test must hold. | L20 |
| W15 | 7.2: If-Match on the hash the UI last saw. |
A stale-fallback read answers from the snapshot; a hash from there is the snapshot's, not the disk's. Editing against it produces a spurious conflict or a silent overwrite of unpublished work. | L5 |
| W16 | 7.5: "The Front Door writes audit records ... hash before and after." | The Front Door knows the hash of the bytes it sent (after) and the hash the person expected (before). It does not know what the owner actually replaced unless the owner says so. The owner is not trusted for the trail. The design also leaves open whether the record exists when the Front Door gives up waiting (a late result). | L21 |
| W17 | 7.2 shows plain responses and no provenance fields. | Phase 1 responses carry source: "snapshot", as_of, masked, a { error, message } error body, and the web types source as the literal "snapshot" (web/lib/computer-files.ts:19, snapshot/files.ts SNAPSHOT_FILE_ERROR_STATUS). The live routes must extend these, not replace them. |
L1, L4 |
| W18 | 7.6: "A per-user limit ... so the poll channel cannot be flooded." No number, no store, no per-computer bound. | The only counter in the store is the organization-creation bucket (incrementOrganizationCreationAttempts, dynamo-messaging-store.ts:216). Each waiting request holds a Front Door invocation for up to 15 seconds. |
L22 |
| W19 | 6.8 and the contract C6: protocol skew is handled by draining owners below a minimum version. | The drain on version is Phase 2b: the server only logs a low version (pollOwner, owner_protocol_version_low). An owner up to 50 minutes old will not understand file_requests. |
L3 |
| W20 | Header: "Nothing on this page is built." Section 4 describes "today". | Phases 1 and 2a are merged and live. Sections 4.1 to 4.3 describe the system before Phase 2a. | corrected in the design document by this change |
| W21 | 7.2 and 7.3: symlinks are shown and followed only inside the index. | A live write through a symlink has two readings: replace the link, or write the file it points to. An atomic rename over the link would replace the link. | L16 |
| W22 | 7.6: "Writes under .git/ are refused (a hook would run as the shell user ...)". |
That is a defense for the person's click, not a privilege boundary: the shell user can write .git/hooks itself. The rule must also hold after links are resolved, or a link named notes to .git/config defeats it. |
L23 |
| W23 | 7.6 "Who": the existing resolveUserPrincipal enforces membership. |
The membership cache lives 30 seconds (MembershipCache.timeToLiveMilliseconds): a removed member can still write for up to 30 seconds. The principal can be an integration-test principal, which may write (L24). Only kind: "user" or kind: "integration" with a user id can be audited. |
L24 |
| W24 | 7.3 and 7.5: the viewer reads in ranges. | Each ranged read of a live file must hash the whole file (the ETag is the whole-file hash). At 256 KiB a page, a 50 MiB file is 200 pages of 50 MiB hashed each. | L15 (8 MiB cap, accepted: O4) |
| W25 | owner-role-policy.json MessagingWritable lists T#computer#record. |
The per-turn child may write the computer record directly. The interactive window and the file-operation token on that record are therefore advisory against a compromised child: it can make the owner poll fast or delay a stop, not forge a request, a result or an audit record, which live in other partitions. | L25 |
| W26 | The design never says how a file is edited while the computer is off (7.5 assumes an owner). | The pack is one snapshot.tar.gz at a fixed key rewritten in place and a manifest overwritten after it (snapshot/s3.ts putPack, putManifest). A Lambda cannot and must not rewrite a gzip tar; an edit needs a side store. |
L30 |
| W27 | "No infrastructure change." (first revision of this page) | The Front Door role has s3:GetObject on the computer prefix and s3:ListBucket on the bucket (conversation/src/gateway/front-door-snapshot-policy.json). Writing an overlay needs PutObject and DeleteObject on its prefix. |
L30, 12.8 |
| W28 | The supervisor role "may read" snapshot objects. | Its statement allows GetObject, PutObject and AbortMultipartUpload on the whole computer prefix (conversation/src/gateway/supervisor-role-policy.json), which includes any overlay key; and the test IAM evaluator knows only Allow (features-support/iam-policy-evaluator.ts:6). An explicit deny is needed and the evaluator must learn it. |
L37, 12.8 |
| W29 | Hydrate is a single step. | hydrateOnBoot returns early with no snapshot (reason: no_snapshot) and skips extraction when the local cache checksum matches (ComputerHostDisk.hydrate, cacheMatches). An overlay applied only after extraction would be skipped on a computer that never published, and on a cache hit. |
L42 |
| W30 | Publish makes a new snapshot. | publishChangedDisk calls publishComputerSnapshot after the upload; a crash between leaves the new pack and the old record. The overlay must be cleared only after a publish recorded by the control plane, and a clear can be lost, so applying must be idempotent. |
L36 |
| W31 | Snapshot answers are complete. | The pack excludes node_modules, .next, dist, build, target, .venv, __pycache__, .cache at any depth and files over 50 MiB (snapshot/exclusions.ts): an overlay edit under such a directory would be applied and then silently dropped by the next publish. |
L40 |
| W32 | The snapshot bucket is just storage. | It is versioned with no expiry of non-current versions (infra/lib/snapshot-stack.ts:19-29): an overlay blob deleted after it was applied (possibly a .env) stays recoverable as a non-current version. |
12.8, L63 |
| W33 | The file routes serve any computer. | snapshotFiles is one SnapshotFileService over the Anthus bucket (lambdas/front-door-composition.ts:216); customer-account computers use a bucket in the customer's account and a host worker that never applies an overlay. |
L46 |
| W34 | A computer with nothing published answers no_snapshot. |
With an overlay, a computer that has an edit but no snapshot must list /workspace as empty plus the edit; serve() returns 404 no_snapshot first (http/routes/computer-files.ts). |
7.7.2 |
3. Decisions made in the contract
3.0 Decisions of the owner (revision 2)
| # | Decision | Effect on this page |
|---|---|---|
| O1 | The computer is needed only to run things, never to view or edit files. A person reads and edits files while it is off. No refusal with 409. | The edit overlay (L30 to L47, 7.7). The first revision's question 1 is closed; computer_not_live is gone. |
| O2 | An open Files panel does not keep the computer awake and does not start it. Only genuine file operations that go through a live owner count as activity. | L12 restated; the watch ping on a computer that is off answers off and starts nothing; overlay edits touch no computer record and are not activity. |
| O3 | The interactive pace is 1 second. | L7 confirmed. |
| O5 | Delete, rename and new folders are required in Phase 3, also while the computer is off. Rename and delete take If-Match of the effective hash; creating a directory is idempotent. Overlay caps stay at 100 entries, 8 MiB, 256 KiB (O5). |
7.8, L48 to L66 |
| O6 | Non-current bucket versions expire after 30 days; explained plainly in 12.8; the rule is in part B. | 12.8, part B |
| O7 | A few seconds of 503 while an owner seals or stops is accepted. Any enabled member may discard a pending change, audited. LocalWorkerRole is left as it is (operator-trusted local-host role that already has read and write on the whole bucket). |
L33, 7.7.3, 12.8 |
| O4 | Accepted as recommended: audit read by operators only; every enabled member may reveal secrets, every reveal audited; 8 MiB live read limit; hashes, not text, in the audit; no channel notice when a person edits; no client IP in the audit; loss of an unpublished live edit on a crash is accepted and shown as "not yet in the saved copy". Customer-account computers read the snapshot only; the overlay is not cheap there (L46) so they cannot be edited. | Closes the first revision's questions 2 to 11; section 14 lists what is still open. |
The numbers are this page's own (L1, L2, ...). D8, D9 and D14 of the design are unchanged and used as stated: every enabled organization member reads and writes, the edit is direct with a hash check and an audit record, audit retention is 90 days.
| # | Decision | Why |
|---|---|---|
| L1 | The routes of Phase 1 stay and grow. GET /orgs/{t}/computer/files, /stat, /content keep their paths, their { error, message } error body and Cache-Control: private, no-store. Responses gain source: "live", stale, fallback_reason, modified_at; content responses gain revealed and editable. New: PUT .../files/content, POST .../files/reveal, POST .../files/watch. |
One API with a source field is the design's own rule (7.1). No second path. |
| L2 | Live is chosen by one predicate, liveFileSourceOf: liveOwnerOf is true, ownerProtocolVersion >= 2, the owner's last polled state is idle, serving, publishing or draining, ownerStopApprovedAt is unset and hydrateRequired is false. Otherwise the answer comes from the snapshot. |
W1, W19. pollState is already stored on the worker item (store/codecs/worker.ts). |
| L3 | A protocol version gate, not a drain. OWNER_FILE_REQUESTS_PROTOCOL_VERSION = 2. The supervisor bumps its OWNER_PROTOCOL_VERSION to 2 in the same change that handles file_requests. The server delivers file requests only to an owner whose stored ownerProtocolVersion is 2 or more; an older owner is answered from the snapshot with fallback_reason: "owner_too_old". OWNER_MINIMUM_PROTOCOL_VERSION stays 1. |
An owner lives at most 50 minutes, so skew is bounded; the gate is two comparisons, not a fork of behavior. |
| L4 | Provenance on every file response: source (live or snapshot), as_of (live: the time of the answer; snapshot: published_at), stale (a live computer exists whose disk may differ from this answer), fallback_reason (null unless a live attempt was skipped or failed), and for owner_warming a retry_after_ms. A computer that is off answers source: "snapshot", stale: false: the snapshot is the truth. |
W17. The UI needs exactly three states: live, saved copy, saved copy while the computer is on. |
| L5 | Editing has two paths and one rule. editable is true when the content is UTF-8 text, fully loaded (truncated: false), at most 256 KiB, not masked unless revealed, and a write path exists: the response is source: "live", or it is a snapshot answer for a computer whose overlay is writable (no valid seal, L33). The UI never offers an editor on a stale fallback of a running computer, or while the overlay is sealed. If-Match is judged against the effective hash: the owner's disk when live, otherwise the overlay entry over the snapshot (L32). |
W15, W26. |
| L6 | The panel signals with a watch ping, POST .../files/watch { open: true } every 15 seconds while the Files tab is visible, and { open: false } when it is hidden or closed. The server stores interactiveUntilAt on the computer record (now + 45 s for open: true, now for open: false). Every queued file request also sets it. |
W2. The poll request stays unchanged; the server answers the pace. |
| L7 | The interactive pace is 1000 ms, not the design's 500 ms: OWNER_POLL_INTERACTIVE_MILLISECONDS = 1000. While interactiveUntilAt is in the future, next_poll_ms = min(normal pace, 1000), in the states that can serve files (idle, serving, publishing, draining). |
W3: it is the same as the first-minute idle tier, costs about 0.7 cents an hour while a panel is open, and makes the average pickup 0.5 seconds. Decided by the owner (O3). |
| L8 | One fixed result route: POST /orgs/{t}/host/owner/file-results, the operation id in the body. It is added to OWNER_COMPUTER_ROUTES. |
W5. |
| L9 | A request has a delivery state. pending (queued) becomes delivered (the poll handed it to the owner, a conditional write), done (result recorded) or abandoned (the Front Door gave up while it was still pending, a conditional write). An abandoned request is never delivered. For a delivered write the Front Door waits a further 7 seconds and then answers 504 outcome_unknown; the late result is still recorded and still audited. A delivered read that times out falls back to the snapshot. |
W6. This is the dispatch lifecycle of OWNER_SUPERVISOR_PROTOCOL 6.3 applied to files. |
| L10 | Rows are deleted by the code that reads them; TTL is a backstop. The Front Door deletes a result item after it has read it. An abandoned item loses its request and result attributes in the same conditional update. expires_at (120 seconds after queueing) only bounds what a crash leaves behind, and every reader treats an item past its expires_at as absent. |
W4. |
| L11 | The idle stop is vetoed by a pending or delivered file request (veto file_operation_pending), judged and approved with the same compare-and-set trick as a dispatch: the computer record carries fileOperationId, the id of the last queued request; the approval is conditional on it being unchanged. Queueing a request is conditional on the owner being live and no stop being approved. An approval and a queued request cannot both win. lifetime and unhealthy stops are not vetoed (the supervisor finishes the operation in flight and answers the rest owner_stopping). |
W7. Mirrors OWNER_SUPERVISOR_PROTOCOL 5.4. |
| L12 | Only genuine operations are activity, and nothing here starts a computer (O2). A file operation through a live owner restarts an idle supervisor's idle clock when it finishes. An open panel, a watch ping, a read or a write answered from the snapshot, and an overlay edit are not activity and never queue a start job. | W7. |
| L13 | A successful write schedules the quiet publish: the supervisor sets its next publish check to at most 30 seconds from the write if it is idle or publishing, and at the next gap if a child runs. The response says backed_up: false so the UI can say the copy on the computer is not yet in the saved copy. |
W8. Accepted by the owner (O4): an unpublished live edit is lost on a crash and the panel says so. |
| L14 | The live path marks the disk dirty before it queues a write (markComputerDiskDirty) and refuses a write when hydrateRequired is true. An overlay write touches no disk and sets no flag. |
W9. A spurious dirty flag costs one index scan; a missing one loses an edit silently. |
| L15 | The helper gets four bounded operations: list (scans at most 20,000 names, returns at most 2,000 entries), stat, read (a range, at most 256 KiB, of a regular file of at most 8 MiB; hashes the whole file) and write (UTF-8 text up to 256 KiB, atomic, with a precondition). Opening is O_NONBLOCK and fstat must say regular file, so a FIFO or a device is not_a_file, not a hang. Each operation has a 6 second timeout enforced by the supervisor. |
W10, W24. The 8 MiB cap is below the snapshot's 50 MiB (accepted: O4). |
| L16 | Writes are atomic and resolve links first. The helper resolves the final location (following every link, as it does today), refuses it if outside the workspace, and writes a temporary file with a random name in the resolved directory, fsyncs, fchmods to the old mode (or 0664), compares the precondition once more and renames it into place. A create uses link then unlink, which fails if the name exists. A link is therefore written through and stays a link. After the rename the helper re-hashes the file it replaced; if that differs from the hash the caller expected (a concurrent in-place write won the sub-millisecond window), it keeps the replaced bytes as a sibling <name>.chatticus-replaced-<8 hex> and reports kept_as. The Pi write_workspace tool takes the same write path with no precondition: one path. |
W10, W21. The in-place writer of Pi and of the shell is why the post-check exists. |
| L17 | File operations and the sweep exclude each other. The supervisor's file runner has pause() (waits for running helpers, queues new requests) and stopAccepting() (waits for running helpers, answers new requests owner_stopping). The sweep after a child exit runs between pause() and resume(); the stop sequence calls stopAccepting() before it kills the child and sweeps. |
W11. |
| L18 | One list of secret names, in host-protocol/src/live-file-rules.ts, imported by the snapshot service, the helper and the Front Door. It is Phase 1's list plus .git-credentials. Masking is decided by the helper on the resolved path for a live read (so a link named config.txt to .env is masked) and by the snapshot service on the path and the link target, as today. A reveal is POST .../files/reveal (it has a side effect, so not a GET), for both sources, and is audited. |
W12. |
| L19 | The publish and hydrate records stop overwriting the record. recordHostSnapshotPublished and recordHostHydrated become targeted updates (SET snapshot_*, disk_dirty, hydrate_required). Part A1 does this before it adds the fields. |
W13. |
| L20 | A test holds the item size. A maximal read result (256 KiB of binary, base64) as a table item must be under 380,000 bytes. A write carries UTF-8 text only. | W14. |
| L21 | Audit: intent first, outcome by the result route. The Front Door writes the record (outcome: "pending") before it queues the request, and fails closed (503 audit_unavailable, nothing queued) if it cannot. The result route finalizes the record when a result arrives, even after the waiting request gave up. A record still pending after 120 seconds is read as unknown. The Front Door computes sha256_after from the bytes it sent; the owner's reported hashes are recorded as reported and a mismatch is flagged integrity_mismatch. The record holds no file content. |
W16. |
| L22 | Rate and pressure limits. Per user per minute: 240 reads (list, stat, content), 30 writes, 10 reveals, 12 watch pings. Per computer: at most 16 open requests (too_busy, 429). Per poll: at most 4 requests delivered. The counters are bucketed items with expires_at, the pattern of incrementOrganizationCreationAttempts, generalized into one store method. |
W18. |
| L23 | .git is refused twice. The Front Door refuses any lexical path with a .git segment (403 path_refused, audited); the helper refuses any resolved path with a .git segment when the request says so (refuseGitSegments, set by every live request, not set by the Pi tools: their behavior does not change). It is a guard against a click, not a privilege boundary. |
W22. |
| L24 | Writes and reveals need a user principal (kind: "user", or kind: "integration" for the acceptance-test session, which exists only outside production; in both cases with a user id) of an enabled organization. Worker, operator and anonymous callers are refused. The audit record carries the user id from the principal, never from the body. The 30 second membership cache is accepted and stated. |
W23. |
| L25 | The owner is not trusted for the trail or for policy. Masking is also applied by the Front Door before it hands content to the browser when the owner reports a secret-looking resolved_path for a request that was not a reveal. The audit key family and the file-operation key family are outside both owner role policies; the owner_role_policy.feature scenarios prove it. |
W25. |
| L26 | Status codes keep the design's 409. A failed If-Match is 409 precondition_failed with current_sha256, not RFC 9110's 412, because the design and the web app already speak 409 for conflicts and the client is first party. A missing precondition is 428. |
Consistency. |
| L27 | A Front Door wait is 8 seconds, and is skipped when it cannot succeed. expected pickup = max(0, lastHeartbeatAt + cadence - now) where cadence = ownerPollCadence(pollState, idleSeconds). If it is more than 7 seconds, the Front Door does not queue a request; it sets the interactive window, answers from the snapshot with fallback_reason: "owner_warming" and retry_after_ms = expected pickup, and the panel asks again then. For a write the Front Door answers 503 owner_warming with the same retry_after_ms and queues nothing. |
W2. A first look at a long-idle computer answers at once instead of after 8 seconds. |
| L28 | A live "not found" is an answer. Owner errors that describe the file (not_found, not_a_file, not_a_directory, file_too_large, outside_workspace) are returned as such and never replaced by the snapshot, which may still hold a file deleted since. Only failures of the channel (owner_timeout, helper_timeout, helper_failed, io_error, owner_busy, owner_stopping, expired) make a read fall back. |
A stale fallback must not resurrect a deleted file. |
| L29 | Reading is not audited, writing and revealing are. Refused writes (conflict, path_refused, size) are audited too; requests refused before authorization or by the rate limit are logged only. |
D14 names edits and reveals. |
| L30 | The edit overlay is in the snapshot bucket, under the computer's prefix: overlay/index.json and overlay/blobs/{sha256}. The pack is never rewritten. The Front Door alone writes and deletes overlay objects. |
W26. S3 because the owner already reads that prefix with scoped credentials (no new store or role, no size limit on the way down), edits cost S3 requests only, and the content is at most 256 KiB per file. DynamoDB was considered (native conditions, but the supervisor has no table access by design and would need the content through a Lambda response). |
| L31 | One object is the compare-and-swap target. Blobs are content-addressed and immutable, written first (If-None-Match: *; an existing blob is fine). The index is replaced with PutObject If-Match: <its ETag> (If-None-Match: * for the first one). A lost swap re-reads and re-judges, at most 3 times. A half-finished write leaves an unreferenced blob and nothing else. |
S3 conditional writes (IfMatch, IfNoneMatch; @aws-sdk/client-s3 3.1146 supports both). PROVEN by part A3 against moto 5.2.3 (package.json moto:start): IfNoneMatch: "*" and IfMatch are both honored (a failed condition is 412 PreconditionFailed; IfMatch on an absent key is 404 NoSuchKey; a conditional GET with a current tag is a 304). conversation/test/overlay-store.contract.test.ts holds the S3 store and the filesystem store (same semantics, for scenarios) to the same contract, including that exactly one of two concurrent swaps from one tag wins. |
| L32 | Effective hash of a path = the overlay entry's sha256 if there is a non-conflicting entry, else the snapshot index's sha256, else absent. If-Match and If-None-Match are judged against it. An entry carries base_sha256, the snapshot hash the first edit of the path started from (null for a create); later edits of the same path keep it. |
Lets the apply step tell "the disk still has what the person started from". |
| L33 | The overlay is writable if and only if it has no valid seal. A seal { owner_id, generation, sealed_at, snapshot_generation } is valid when the computer record's ownerId and ownerGeneration equal it and liveOwnerOf holds. Otherwise the seal is stale and the next writer replaces it. While a valid seal exists edits go to the owner (live path, 7.3); if the owner cannot serve yet the answer is 503 computer_starting, if it is stopping 503 computer_stopping, each with retry_after_ms, and the panel keeps the text. |
The seal is the single linearization point of every race in 7.7.6. |
| L34 | The owner seals, then applies the sealed set, before it reports idle. In hydrate(), after extraction: seal (Front Door compare-and-swap, returns the entries), apply each entry through the file helper, report. No turn is dispatched while the supervisor is hydrating (considerDispatch ignores a dispatch unless the state is idle), so no turn runs before the apply ends. The apply has a budget of 30 seconds; past it, or on an unreadable blob or index, the supervisor treats it as a hydrate failure (exit 8) rather than let the agent run on a disk missing edits the person saw. The agent waits for nothing else. |
Readiness per capability: the only wait is the owner's own file application. |
| L35 | Applying uses the helper, as uid 2000, with the precondition. Each entry is a helper write with if_match base_sha256 (or if_none_match for a create) and refuseGitSegments. ok means applied. precondition_failed whose current_sha256 equals the entry's sha256 means already applied (covered). Any other current hash is a conflict: the entry is not applied and is reported (L39). Root never opens a workspace path. |
The same containment as every other file operation (12.2). |
| L36 | An entry is cleared only after a publish that began after it was applied, and the control plane has recorded. The supervisor keeps the set it applied; when a publish starts it captures the set, and after publishComputerSnapshot succeeds it posts cleared for exactly that set. A lost post is retried every OVERLAY_CLEAR_RETRY_SECONDS and before the stopped report. If after the apply the disk equals the published index (nothing to publish), the set is cleared at once. After applying, the next publish check is scheduled OVERLAY_EARLY_PUBLISH_SECONDS (5) after the supervisor becomes idle instead of 600; while a child runs the publish still waits (C14) and the entries stay. |
W30. Until the clear, a crash loses nothing: the overlay still holds every edit. |
| L37 | The supervisor never writes or deletes overlay objects; the Front Door performs the clear. The supervisor reports cleared (path and revision) at POST /host/owner/overlay-applied; the Front Door removes exactly the entries that still match, while the seal is valid, and only if a publish is on record: the computer's snapshotGeneration is greater than the seal's, or the snapshot index already shows the entry's result (a put's hash, a deleted path gone, a directory present, a move's target holding the kept hash with its source gone: 7.8.5). The route checks it; it does not trust the owner's word. The supervisor role gets an explicit deny of PutObject, DeleteObject and DeleteObjectVersion on the overlay prefix. |
IAM cannot say "delete only applied entries"; a route can. One writer, no standing delete right on a long-lived credential. |
| L38 | The seal ends with the owner. The stopped report unseals (best effort, one compare-and-swap). A seal whose owner is no longer the live owner is stale, and the next overlay write, seal or discard replaces it. Entries are never lost by an unseal. | Crash and lease takeover (7.7.6). |
| L39 | A conflict is never resolved silently. A conflict exists when a path has an entry and the snapshot hash is neither the entry's base_sha256 nor its sha256 (derived at every read), or when the owner found a third hash on the disk while applying (stored: conflict on the entry). The effective content is the snapshot's (theirs); the person's text stays readable as variant=mine; the owner never applies an entry with a conflict. Resolution is explicit: a PUT with If-Match of theirs replaces the entry (the person overwrote on purpose; new base_sha256 is theirs), or POST .../files/overlay/discard. |
The owner's rule: refuse and surface, never overwrite. |
| L40 | The overlay accepts only what the next publish will keep. Refused: a path with a segment in PACK_EXCLUDED_DIRECTORY_NAMES (422 not_in_saved_copy), a path whose snapshot entry is a symlink or directory or one of whose ancestors is a symlink (403 path_refused / 422 not_a_file), a .git segment, a parent directory that is absent from the snapshot index (404 parent_missing; /workspace always exists), anything over the size caps. No directory is created. |
W31. |
| L41 | Caps: 100 entries of any kind (put, delete, mkdir, move), 8 MiB of blob content in total (only puts carry a blob), 256 KiB per file, an index of at most 256 KiB; a directory delete or rename covers at most 5000 descendants (413 too_many_entries). Beyond the first three 409 overlay_full. |
Bounds the apply time (L34) and the Lambda memory. A delete, move or mkdir carries no blob and costs no content bytes. |
| L42 | The apply runs after every hydrate whether or not the cache matched, and also when there is no snapshot (an empty workspace). It is independent of extraction. | W29. |
| L43 | Every snapshot answer merges the overlay (list, stat, content, reveal), including the stale fallbacks of a running computer. A computer with no snapshot answers an empty /workspace plus the overlay. Masking by name applies to overlay content. A live answer ignores the overlay. |
W34. |
| L44 | Audit for the overlay uses the same record (section 7.1) with source: "overlay"; operations write, create, discard, and overlay_apply (written by the Front Door when the owner reports a clear or a conflict; user_id is the entry's edited_by, plus owner_id). Intent first, fail closed, as L21. |
D14. |
| L45 | An overlay edit changes no computer record, queues no request, sets no flag and starts nothing. | O2. |
| L46 | Customer-account computers are read-only, for these reasons. They run in the customer's own AWS account, through the older host-worker path (customer-computers-stack.ts, the FargateHost of that stack), with the customer-owned bucket chatticus-snapshots-{OrganizationId} (docs/COMPUTER_SNAPSHOTS.md). The Front Door's write grant (part B) covers only the Anthus snapshot bucket, so it cannot write an overlay there, and no owner applies one: that path has a host worker, not a supervisor, so nothing seals, applies or clears. Reads are unchanged (the snapshot). An edit, delete, rename or new folder is 409 edit_unsupported. A Kanbus ticket (later phase) covers making them editable. |
W33, O4. |
| L47 | A running owner of protocol 1 blocks edits with 409 owner_too_old for the rest of its life (at most 50 minutes): it would never apply an overlay, and its publish would produce false conflicts. Reads are served as before from the snapshot with the overlay merged. |
The version gate of L3 applied to writes. |
| L48 | The overlay records four operations. put (create or replace a file; has a blob), delete (a file, a symlink, an empty directory, or a whole directory with recursive), mkdir and move (a file, a symlink or a whole directory). Each is one entry with a revision; one path has at most one entry. A move is one entry, not a copy: it keeps the content hash of the file it moves (sha256, or the tree_sha256 of a directory) and carries no blob, so a 40 MiB file can be renamed while the computer is off. |
Delete as a tombstone and rename as delete plus create with a rename record, as the owner asked, without ever copying content through a Lambda. |
| L49 | Effective hash of any path: a file, the SHA-256 of its content; a symlink, the SHA-256 of symlink, a NUL byte and its target; a directory, its tree hash: SHA-256 over one line for every descendant in the effective view, sorted by relative path bytes, kind NUL relative-path NUL value LF, where value is the size of a file, the target of a symlink, 0 for a directory (an empty directory hashes the empty input). The tree hash is the directory's ETag on stat. The same definition is computed from the snapshot index merged with the overlay and by the file helper from lstat, so every source agrees. |
A directory has no content; an If-Match for delete and rename still has to mean "the thing I saw". Sizes, not content, keep it cheap on the live disk. |
| L50 | Delete and rename need If-Match of the effective hash (a missing one is 428); creating a directory is idempotent (mkdir of a directory that exists is 200 created: false; of a file, 409 already_exists). A delete or rename of a path that does not exist is 404 not_found. |
O5. |
| L51 | Who may (supersedes the owner-role rule of design 7.6 and D8 for delete and move). Any enabled member may delete or rename a file or a symlink, rename a directory, create a directory, and delete an empty directory, each audited. Only the organization owner role (Principal.role === "owner") may delete a non-empty directory (recursive), 403 owner_role_required otherwise. |
Delete and move now matter to every member, as editing does, and the shared workplace already lets any member tell a bot to do the same; the audit is the safeguard. Only the irreversible bulk delete keeps the original restraint. |
| L52 | A rename never replaces an existing path. to must be absent in the effective view (409 already_exists), its parent must exist (404 parent_missing), it must not lie inside from (422 invalid_move), and neither path may be the workspace root. Overwrite is not offered; the person deletes first. |
Removes a class of silent data loss; Q-P1 asks if it is wanted later. |
| L53 | A pending directory rename or recursive delete is exclusive. While an entry that moves or deletes a directory exists, any other change at, under, or onto its source or target path is 409 path_busy (the person waits for the next owner to apply it or discards the change). A directory move is likewise refused while an overlay entry already exists at or under its source or target (path_busy). A recursive delete is not refused for the entries under it: the table of 7.8.4 wins, the entries below the folder are dropped and a move target or put with from that lay inside becomes a delete of its source. It is refused with path_busy when a pending change takes a file out of the folder (an entry elsewhere whose from lies under the folder), because the delete would then remove a file the person is moving. |
Avoids composing path translations and collapsing trees; one entry describes the whole subtree and the apply is a single rename or removal. |
| L54 | Entries collapse when they undo each other. Deleting a pending create (a put with no base and no from) removes it. Deleting a put that replaced a snapshot file becomes a delete with the snapshot hash as base. Renaming a pending create changes its path. Renaming a snapshot file back to its own path removes the move. Deleting the target of a move becomes a delete of its source. A put onto a path with a delete becomes a put whose base is the snapshot hash. Table in 7.8.4. |
Keeps the entry count and the apply minimal and makes undo natural. |
| L55 | Reads merge all four. A deleted path (and everything under a deleted directory) is absent from lists, stat and content; a mkdir or move target is a directory or file with overlay: "pending"; the content of a moved file is read from the snapshot pack at its source path. Rules in 7.8.3. |
|
| L56 | Apply order at hydrate is fixed: 1 mkdir (shallowest first), 2 directory moves (shallowest first), 3 file and symlink moves, 4 puts, 5 deletes of files and symlinks, 6 deletes of directories (deepest first; a recursive delete is one removal). Each step uses the file helper with its precondition, as uid 2000. The outcome of each operation is applied, already applied, or a conflict (table in 7.8.5). | Parents exist before children are created, moves happen before puts at their targets, and deletes come last so what moved out of a directory is gone before the directory is. |
| L57 | Collisions are conflicts, never overwrites. A move onto an existing path, a mkdir where a file exists, a delete of a file whose hash is no longer the one seen, a non-recursive delete of a directory that is not empty, a recursive delete whose tree hash differs: the operation is not applied, the entry stays flagged (conflict.reason) and is shown with both states. |
The owner's rule. |
| L58 | Already applied is detected, not assumed. A delete whose path is gone, a mkdir whose directory exists, a move whose source is gone and whose target holds the moved hash (or tree hash), a put whose disk hash equals the entry: covered, cleared with the rest after a recorded publish. | Idempotence across a lost clear (R5). |
| L59 | Path containment is unchanged and applies to every path an operation names. Parents are resolved and must stay inside the workspace; the final component is never followed (a delete or rename acts on the link itself); a path with an ancestor symlink in the snapshot index is path_refused; .git as a segment of the path, the source or the target is path_refused (a recursive delete may contain a .git directory below the named one); excluded directory names are not_in_saved_copy for the overlay; the workspace root cannot be deleted or renamed. |
L23, L40. |
| L60 | The live path uses four new helper operations under the same containment: mkdir, unlink (a file, a symlink or an empty directory), rmtree (recursive, tree-hash precondition, 5000 entries, one device, never following a link) and rename (exclusive target). Request kinds delete, move, mkdir are delivered like write (7.4) with the same states, audit, dirty flag and delivery rules. |
Section 8.1. |
| L61 | Audit: operations delete (recursive, removed), move (to_path) and mkdir, with source live or overlay, sha256_before or tree_sha256_before from the If-Match, intent first and fail closed, finalized by the Front Door (overlay) or the result route (live), and overlay_apply records when the owner applies or flags them. |
L44. |
| L62 | Rate and size. Delete, move and mkdir count in the write class (30 a minute). A request body is small (paths only). A tree-hash computation reads the snapshot index (cached) and the overlay once; more than 5000 descendants is 413 too_many_entries. |
L22. |
| L63 | Non-current bucket versions expire after 30 days (part B, snapshot-stack.ts). Explained in 12.8: versioning keeps every overwritten object; today every publish overwrites the whole pack, so old versions are whole-workspace snapshots, not per-file history, and are never expired; keeping them is a storage cost and a recovery window, not a product feature. |
O6, W32. |
| L64 | A pending change can be listed and discarded by any enabled member. GET .../files/overlay lists the entries (path, operation, from, who, when, state, revision); POST .../files/overlay/discard takes { path, revision }, works for every operation, and is audited. Discarding a move or a delete restores what the snapshot has. |
O7. |
| L65 | The seal and the stale-seal rules cover all four operations (L33, L38): a delete, rename or mkdir is a member write like a put and goes to the owner when it can serve, 503 while it seals or stops, the overlay otherwise. | |
| L66 | A conflicted entry of any kind is resolved only explicitly: a fresh operation with If-Match of the effective (theirs) hash replaces it, or the change is discarded. |
L39. |
4. Constants
All in host-protocol/src/live-files.ts unless stated. Tests inject clocks; nothing is read
from the environment.
| Constant | Value | Used by |
|---|---|---|
OWNER_FILE_REQUESTS_PROTOCOL_VERSION (supervisor.ts) |
2 | server gate (L3) |
OWNER_POLL_INTERACTIVE_MILLISECONDS (supervisor.ts) |
1000 | server cadence (L7) |
LIVE_FILE_WAIT_MILLISECONDS |
8000 | Front Door wait; also the largest expires_in_ms |
LIVE_FILE_RESULT_CHECK_MILLISECONDS |
250 | Front Door wait loop |
LIVE_FILE_OUTCOME_GRACE_MILLISECONDS |
7000 | extra wait for a delivered write |
LIVE_FILE_OPERATION_TTL_SECONDS |
120 | expires_at backstop (L10) |
LIVE_FILE_INTERACTIVE_WINDOW_SECONDS |
45 | interactiveUntilAt after a ping or a request |
LIVE_FILE_WATCH_INTERVAL_SECONDS |
15 | web ping interval |
LIVE_FILE_MAX_REQUESTS_PER_POLL |
4 | poll response |
LIVE_FILE_MAX_OPEN_OPERATIONS |
16 | per computer (L22) |
LIVE_FILE_MAX_CONCURRENT_HELPERS |
2 | supervisor |
LIVE_FILE_MAX_QUEUED_REQUESTS |
8 | supervisor, beyond it owner_busy |
LIVE_FILE_HELPER_TIMEOUT_MILLISECONDS |
6000 | supervisor (L15) |
LIVE_FILE_MEMORY_REFUSE_PERCENT |
90 | supervisor, beyond it owner_busy |
LIVE_FILE_CONTENT_MAX_BYTES |
262144 | read range, write body |
LIVE_FILE_READ_MAX_FILE_BYTES |
8388608 | helper read (L15) |
LIVE_FILE_LISTING_MAX_ENTRIES |
2000 | list result |
LIVE_FILE_LISTING_SCAN_MAX_NAMES |
20000 | helper list |
LIVE_FILE_STDOUT_MAX_BYTES |
1048576 | supervisor reading the helper |
LIVE_FILE_PATH_MAX_CHARACTERS (live-file-rules.ts) |
1024 | all paths (existing value) |
LIVE_FILE_RATE_WINDOW_SECONDS |
60 | counters |
LIVE_FILE_READ_RATE, _WRITE_RATE, _REVEAL_RATE, _WATCH_RATE |
240, 30, 10, 12 | per user per window |
LIVE_FILE_AUDIT_RETENTION_DAYS |
90 | audit expires_at (D14) |
LIVE_FILE_PENDING_AUDIT_UNKNOWN_SECONDS |
120 | reading a pending record |
LIVE_FILE_PI_CHANGED_WINDOW_SECONDS |
60 | web warning |
LIVE_FILE_EDIT_PROBE_SECONDS |
5 | web stat probe while editing |
LIVE_FILE_RETRY_AFTER_MIN/MAX_MILLISECONDS |
1000 / 10000 | retry_after_ms clamp |
LIVE_FILE_ERROR_STATUS |
code to status map (section 6.3) | server, tests |
LIVE_FILE_OWNER_ERROR_CODES |
owner code list (section 5.4) | supervisor, helper, server |
OVERLAY_MAX_ENTRIES, OVERLAY_MAX_TOTAL_BYTES, OVERLAY_INDEX_MAX_BYTES (edit-overlay.ts) |
100, 8388608, 262144 | overlay writes (L41) |
OVERLAY_CAS_ATTEMPTS |
3 | Front Door index swaps |
OVERLAY_APPLY_BUDGET_SECONDS |
30 | supervisor (L34) |
OVERLAY_SEAL_RETRY_AFTER_MILLISECONDS |
2000 | computer_starting answers |
OVERLAY_STOPPING_RETRY_AFTER_MILLISECONDS |
5000 | computer_stopping answers |
OVERLAY_CLEAR_RETRY_SECONDS |
10 | supervisor (L36) |
OVERLAY_EARLY_PUBLISH_SECONDS |
5 | supervisor (L36) |
OVERLAY_INDEX_OBJECT, overlayBlobObject(sha256) |
overlay/index.json, overlay/blobs/{sha256} |
all |
LIVE_FILE_TREE_MAX_ENTRIES |
5000 | directory delete, rename and tree hash (L49, L62) |
LIVE_FILE_MKDIR_MODE |
0775 (through the umask 002 of the launcher) |
helper mkdir |
Not changed: DEFAULT_HEARTBEAT_TIMEOUT_SECONDS = 30, OWNER_POLL_* tiers,
OWNER_IDLE_TIMEOUT_SECONDS, OWNER_POLL_SILENCE_FENCE_SECONDS. The 8 second wait fits
inside the 25 second fence only because the wait is in the Front Door, not the owner.
5. Wire schemas
Two new files in host-protocol/ and four small edits to host-protocol/src/supervisor.ts.
package.json exports ./live-files and ./live-file-rules. live-file-rules.ts imports
nothing at all (the file helper bundle must stay free of zod and of anything that could
carry a credential); live-files.ts imports zod and live-file-rules.ts;
supervisor.ts imports live-files.ts, never the other way round.
5.1 host-protocol/src/live-file-rules.ts (no imports)
/** The root of every path a person or the Front Door names. */
export const LIVE_WORKSPACE_ROOT = "/workspace";
/** The longest path, in characters, any file route accepts. */
export const LIVE_FILE_PATH_MAX_CHARACTERS = 1024;
export type LivePathProblem = "empty" | "too_long" | "nul" | "not_absolute" | "dot_dot" | "outside_workspace";
/**
* Why a path is not a workspace path, or null when it is one. Lexical only: it follows no link.
* The path must be absolute, at most 1024 characters, hold no NUL and no ".." segment, and after
* normalisation be "/workspace" or start with "/workspace/".
*/
export function livePathProblem(raw: string): LivePathProblem | null;
/** The normalised form ("/workspace/a/b", no trailing slash, no "." segment). Precondition: livePathProblem(raw) === null. */
export function normalizeLivePath(raw: string): string;
/** Whether any segment of a normalised path is exactly ".git" (a file or a directory). */
export function livePathHasGitSegment(normalized: string): boolean;
/**
* Whether a path looks like it holds a secret. Case-insensitive on the segments. True for a file named .env or .env.*,
* *.pem, *.key, *.p12, *.pfx, id_rsa*, id_ed25519*, credentials, .netrc, .npmrc, .git-credentials, and for any file under
* a directory named .ssh or .aws. It is the Phase 1 list plus .git-credentials.
*/
export function looksLikeSecretFile(path: string): boolean;
/** True when the first 8 KiB holds a NUL or is not valid UTF-8. */
export function looksBinary(bytes: Uint8Array): boolean;
/** Drop an incomplete UTF-8 sequence from the end of a byte range. */
export function trimToCharacterBoundary(bytes: Uint8Array): Uint8Array;
/** The number of bytes of the UTF-8 encoding of a string. */
export function utf8ByteLength(text: string): number;
conversation/src/snapshot/files.ts deletes its private copies (indexPathOf keeps its
index-path mapping on top of livePathProblem, looksLikeSecretFile, looksBinary,
trimToCharacterBoundary) and imports these. That is a behavior-preserving move plus
.git-credentials; the existing scenarios in computer_snapshot_index.feature stay green.
5.2 host-protocol/src/live-files.ts
import { z } from "zod";
import { LIVE_FILE_PATH_MAX_CHARACTERS, livePathProblem, utf8ByteLength } from "./live-file-rules.ts";
export const LIVE_FILE_WAIT_MILLISECONDS = 8000;
export const LIVE_FILE_RESULT_CHECK_MILLISECONDS = 250;
export const LIVE_FILE_OUTCOME_GRACE_MILLISECONDS = 7000;
export const LIVE_FILE_OPERATION_TTL_SECONDS = 120;
export const LIVE_FILE_INTERACTIVE_WINDOW_SECONDS = 45;
export const LIVE_FILE_WATCH_INTERVAL_SECONDS = 15;
export const LIVE_FILE_MAX_REQUESTS_PER_POLL = 4;
export const LIVE_FILE_MAX_OPEN_OPERATIONS = 16;
export const LIVE_FILE_MAX_CONCURRENT_HELPERS = 2;
export const LIVE_FILE_MAX_QUEUED_REQUESTS = 8;
export const LIVE_FILE_HELPER_TIMEOUT_MILLISECONDS = 6000;
export const LIVE_FILE_MEMORY_REFUSE_PERCENT = 90;
export const LIVE_FILE_CONTENT_MAX_BYTES = 262144;
export const LIVE_FILE_READ_MAX_FILE_BYTES = 8388608;
export const LIVE_FILE_LISTING_MAX_ENTRIES = 2000;
export const LIVE_FILE_LISTING_SCAN_MAX_NAMES = 20000;
export const LIVE_FILE_TREE_MAX_ENTRIES = 5000;
export const LIVE_FILE_STDOUT_MAX_BYTES = 1048576;
export const LIVE_FILE_RATE_WINDOW_SECONDS = 60;
export const LIVE_FILE_READ_RATE = 240;
export const LIVE_FILE_WRITE_RATE = 30;
export const LIVE_FILE_REVEAL_RATE = 10;
export const LIVE_FILE_WATCH_RATE = 12;
export const LIVE_FILE_AUDIT_RETENTION_DAYS = 90;
export const LIVE_FILE_PENDING_AUDIT_UNKNOWN_SECONDS = 120;
export const LIVE_FILE_RETRY_AFTER_MIN_MILLISECONDS = 1000;
export const LIVE_FILE_RETRY_AFTER_MAX_MILLISECONDS = 10000;
const identifier = z.string().min(1).max(256);
const timestamp = z.iso.datetime();
const generation = z.number().int().positive();
const sha256Hex = z.string().regex(/^[0-9a-f]{64}$/);
const mode = z.string().regex(/^[0-7]{3,4}$/);
/** A path under /workspace, lexically valid. The same rules as livePathProblem; the issue message names the problem. */
export const livePathSchema = z.string().max(LIVE_FILE_PATH_MAX_CHARACTERS).superRefine((value, context) => {
const problem = livePathProblem(value);
if (problem !== null) context.addIssue({ code: "custom", message: `not a workspace path: ${problem}` });
});
/** UTF-8 text a person may save: at most 256 KiB, no NUL. */
export const liveFileTextSchema = z.string().refine(
(text) => utf8ByteLength(text) <= LIVE_FILE_CONTENT_MAX_BYTES && !text.includes("\u0000"),
{ message: "content must be UTF-8 text of at most 262144 bytes with no NUL" },
);
/** One directory entry, the same shape for a live listing and a snapshot listing. */
export const liveFileEntrySchema = z
.object({
name: z.string().min(1).max(255),
path: livePathSchema,
kind: z.enum(["file", "directory", "symlink"]),
size: z.number().int().nonnegative().nullable(),
mode: mode.nullable(),
sha256: sha256Hex.nullable(),
link_target: z.string().max(4096).nullable(),
modified_at: timestamp.nullable(),
/** "pending": an overlay change is in effect; "conflict": the person's change conflicts with a newer snapshot (the entry shows theirs). */
overlay: z.enum(["none", "pending", "conflict"]),
/** Only on the stat of a directory: its tree hash (L49), the ETag of a delete or rename. */
tree_sha256: sha256Hex.nullable(),
})
.strict();
/** What a write must find on the disk. "if_match" replaces a file with this hash; "if_none_match" creates a file that must not exist. */
export const liveFileWritePreconditionSchema = z.discriminatedUnion("type", [
z.object({ type: z.literal("if_match"), sha256: sha256Hex }).strict(),
z.object({ type: z.literal("if_none_match") }).strict(),
]);
/**
* What a delete or a move must find: the effective hash of the path (L49), which is the content hash of a file, the link hash
* of a symlink or the tree hash of a directory. The owner decides which it is by what is really at the path.
*/
export const liveFileRemovePreconditionSchema = z.object({ type: z.literal("if_match"), sha256: sha256Hex }).strict();
const requestBase = {
operation_id: identifier,
path: livePathSchema,
expires_in_ms: z.number().int().min(0).max(LIVE_FILE_WAIT_MILLISECONDS),
};
/**
* One file request the poll hands to the owner. Not strict: a newer server may add a field an older owner ignores.
* `expires_in_ms` is relative to the moment the owner received the answer; the owner measures it with its own clock and
* answers `expired`, without touching the disk, if it cannot start the operation in that time.
*/
export const ownerFileRequestSchema = z.discriminatedUnion("kind", [
z.object({ ...requestBase, kind: z.literal("list"), max_entries: z.number().int().min(1).max(LIVE_FILE_LISTING_MAX_ENTRIES) }),
z.object({ ...requestBase, kind: z.literal("stat"), hash: z.boolean().default(false) }),
z.object({
...requestBase,
kind: z.literal("read"),
offset: z.number().int().min(0),
length: z.number().int().min(1).max(LIVE_FILE_CONTENT_MAX_BYTES),
mask: z.boolean(),
}),
z.object({ ...requestBase, kind: z.literal("write"), content: liveFileTextSchema, precondition: liveFileWritePreconditionSchema }),
z.object({ ...requestBase, kind: z.literal("delete"), recursive: z.boolean(), precondition: liveFileRemovePreconditionSchema }),
z.object({ ...requestBase, kind: z.literal("move"), to: livePathSchema, precondition: liveFileRemovePreconditionSchema }),
z.object({ ...requestBase, kind: z.literal("mkdir") }),
]);
/** Why an owner could not carry out a request. */
export const LIVE_FILE_OWNER_ERROR_CODES = [
"not_found",
"not_a_file",
"not_a_directory",
"outside_workspace",
"permission_denied",
"file_too_large",
"already_exists",
"precondition_failed",
"parent_missing",
"not_text",
"path_refused",
"expired",
"owner_stopping",
"owner_busy",
"helper_failed",
"helper_timeout",
"io_error",
"directory_not_empty",
"invalid_move",
"too_many_entries",
] as const;
export const liveFileOwnerErrorCodeSchema = z.enum(LIVE_FILE_OWNER_ERROR_CODES);
const listResult = z
.object({
kind: z.literal("list"),
resolved_path: livePathSchema,
entries: z.array(liveFileEntrySchema).max(LIVE_FILE_LISTING_MAX_ENTRIES),
truncated: z.boolean(),
})
.strict();
const statResult = z.object({ kind: z.literal("stat"), entry: liveFileEntrySchema }).strict();
const readResult = z
.object({
kind: z.literal("read"),
resolved_path: livePathSchema,
size: z.number().int().nonnegative(),
sha256: sha256Hex,
mode,
modified_at: timestamp,
encoding: z.enum(["utf8", "base64"]),
binary: z.boolean(),
offset: z.number().int().nonnegative(),
length: z.number().int().min(0).max(LIVE_FILE_CONTENT_MAX_BYTES),
content: z.string(),
truncated: z.boolean(),
masked: z.boolean(),
})
.strict()
.refine((result) => !result.masked || (result.content === "" && result.length === 0), { message: "a masked read carries no content" });
const writeResult = z
.object({
kind: z.literal("write"),
resolved_path: livePathSchema,
created: z.boolean(),
size: z.number().int().nonnegative(),
sha256_before: sha256Hex.nullable(),
sha256_after: sha256Hex,
modified_at: timestamp,
kept_as: livePathSchema.optional(),
})
.strict();
const deleteResult = z
.object({
kind: z.literal("delete"),
resolved_path: livePathSchema,
directory: z.boolean(),
removed: z.number().int().nonnegative().max(LIVE_FILE_TREE_MAX_ENTRIES),
sha256_before: sha256Hex.nullable(),
tree_sha256_before: sha256Hex.nullable(),
kept_as: livePathSchema.optional(),
})
.strict();
const moveResult = z
.object({
kind: z.literal("move"),
from: livePathSchema,
to: livePathSchema,
directory: z.boolean(),
sha256: sha256Hex.nullable(),
tree_sha256: sha256Hex.nullable(),
})
.strict();
const mkdirResult = z.object({ kind: z.literal("mkdir"), resolved_path: livePathSchema, created: z.boolean() }).strict();
/** The owner's answer to one request: the result, or the code of the error that stopped it. */
export const liveFileOutcomeSchema = z.discriminatedUnion("status", [
z.object({ status: z.literal("ok"), result: z.discriminatedUnion("kind", [listResult, statResult, readResult, writeResult, deleteResult, moveResult, mkdirResult]) }).strict(),
z
.object({
status: z.literal("error"),
code: liveFileOwnerErrorCodeSchema,
message: z.string().max(500),
current_sha256: sha256Hex.nullable().optional(),
})
.strict(),
]);
/** POST /orgs/{tenant}/host/owner/file-results: the supervisor posts the answer to one request. */
export const ownerFileResultRequestSchema = z
.object({ owner_id: identifier, generation, operation_id: identifier, outcome: liveFileOutcomeSchema })
.strict();
/** The answer: the result is recorded (also when it repeats one already recorded, byte for byte). */
export const ownerFileResultResponseSchema = z.object({ recorded: z.literal(true) });
/** Why a member-facing file response came from the snapshot although a live computer exists. */
export const liveFileFallbackReasonSchema = z.enum([
"owner_timeout",
"owner_error",
"owner_not_ready",
"owner_too_old",
"owner_warming",
"owner_stopping",
]);
/** The fields every member-facing list, stat and content response carries. */
export const liveFileProvenanceShape = {
source: z.enum(["live", "snapshot"]),
as_of: timestamp.nullable(),
stale: z.boolean(),
fallback_reason: liveFileFallbackReasonSchema.nullable(),
retry_after_ms: z.number().int().min(LIVE_FILE_RETRY_AFTER_MIN_MILLISECONDS).max(LIVE_FILE_RETRY_AFTER_MAX_MILLISECONDS).optional(),
};
export const liveFileListingResponseSchema = z.object({
...liveFileProvenanceShape,
path: livePathSchema,
truncated: z.boolean(),
entries: z.array(liveFileEntrySchema).max(LIVE_FILE_LISTING_MAX_ENTRIES),
});
export const liveFileStatResponseSchema = z.object({ ...liveFileProvenanceShape, entry: liveFileEntrySchema });
export const liveFileContentResponseSchema = z.object({
...liveFileProvenanceShape,
path: livePathSchema,
size: z.number().int().nonnegative(),
sha256: sha256Hex,
encoding: z.enum(["utf8", "base64"]),
binary: z.boolean(),
offset: z.number().int().nonnegative(),
length: z.number().int().nonnegative(),
content: z.string(),
truncated: z.boolean(),
masked: z.boolean(),
revealed: z.boolean(),
editable: z.boolean(),
modified_at: timestamp.nullable(),
overlay: z.enum(["none", "pending", "conflict"]),
mine_sha256: sha256Hex.nullable().optional(),
});
/** PUT /orgs/{tenant}/computer/files/content?path= with If-Match: "<sha256>" or If-None-Match: * */
export const liveFileWriteBodySchema = z.object({ content: liveFileTextSchema }).strict();
export const liveFileWriteResponseSchema = z.object({
source: z.enum(["live", "overlay"]),
as_of: timestamp,
path: livePathSchema,
created: z.boolean(),
size: z.number().int().nonnegative(),
sha256: sha256Hex,
previous_sha256: sha256Hex.nullable(),
modified_at: timestamp,
kept_as: livePathSchema.nullable(),
/** false for a live edit (on the computer's disk, not yet in the saved copy); true for an overlay edit (already part of the saved copy as the next read sees it). */
backed_up: z.boolean(),
});
/** POST /orgs/{tenant}/computer/files/reveal */
export const liveFileRevealBodySchema = z
.object({
path: livePathSchema,
offset: z.number().int().min(0).optional(),
length: z.number().int().min(1).max(LIVE_FILE_CONTENT_MAX_BYTES).optional(),
})
.strict();
/** POST /orgs/{tenant}/computer/files/overlay/discard: drop one pending overlay change of any kind (the revision is the entry's). */
export const liveFileDiscardBodySchema = z.object({ path: livePathSchema, revision: z.number().int().positive() }).strict();
/** POST /orgs/{tenant}/computer/files/move: rename a file, a symlink or a directory; header If-Match of the effective hash of `from` (L49). */
export const liveFileMoveBodySchema = z.object({ from: livePathSchema, to: livePathSchema }).strict();
/** POST /orgs/{tenant}/computer/files/directories: create a directory; idempotent. */
export const liveFileMkdirBodySchema = z.object({ path: livePathSchema }).strict();
/** DELETE /orgs/{tenant}/computer/files?path=&recursive= with If-Match of the effective hash (L49). */
export const liveFileDeleteResponseSchema = z.object({
source: z.enum(["live", "overlay"]),
as_of: timestamp,
path: livePathSchema,
directory: z.boolean(),
removed: z.number().int().nonnegative(),
kept_as: livePathSchema.nullable(),
backed_up: z.boolean(),
});
export const liveFileMoveResponseSchema = z.object({
source: z.enum(["live", "overlay"]),
as_of: timestamp,
from: livePathSchema,
to: livePathSchema,
directory: z.boolean(),
sha256: sha256Hex.nullable(),
tree_sha256: sha256Hex.nullable(),
backed_up: z.boolean(),
});
export const liveFileMkdirResponseSchema = z.object({
source: z.enum(["live", "overlay"]),
as_of: timestamp,
path: livePathSchema,
created: z.boolean(),
backed_up: z.boolean(),
});
/** POST /orgs/{tenant}/computer/files/watch: the Files panel is open (true) or closed (false). */
export const liveFileWatchBodySchema = z.object({ open: z.boolean() }).strict();
export const liveFileWatchResponseSchema = z.object({
state: z.enum(["off", "starting", "live"]),
interactive_until: timestamp.nullable(),
poll_interval_ms: z.number().int().min(250).max(60000).nullable(),
});
/** The body of every member-facing file refusal (Phase 1 shape, extended). */
export const liveFileErrorSchema = z.object({
error: z.string(),
message: z.string(),
current_sha256: sha256Hex.nullable().optional(),
retry_after_ms: z.number().int().positive().optional(),
kept_as: livePathSchema.optional(),
});
export type OwnerFileRequest = z.infer<typeof ownerFileRequestSchema>;
export type LiveFileOutcome = z.infer<typeof liveFileOutcomeSchema>;
export type OwnerFileResultRequest = z.infer<typeof ownerFileResultRequestSchema>;
export type LiveFileEntry = z.infer<typeof liveFileEntrySchema>;
export type LiveFileOwnerErrorCode = z.infer<typeof liveFileOwnerErrorCodeSchema>;
export type LiveFileFallbackReason = z.infer<typeof liveFileFallbackReasonSchema>;
export type LiveFileWriteResponse = z.infer<typeof liveFileWriteResponseSchema>;
A schema test (conversation/test/live-files-protocol-schema.test.ts) must prove at least:
the largest read result (a 262144 byte range of binary, base64) serialized as a table item
is under 380,000 bytes (L20); a write of 262144 bytes of multi-byte text passes and one
byte more fails; a path with .., a NUL, a relative path and a path outside /workspace
fails; a masked read result with content fails; a request with an unknown field fails and a
poll response with an unknown field passes.
5.3 Edits to host-protocol/src/supervisor.ts
/** The first protocol version whose owner understands file_requests in the poll response. */
export const OWNER_FILE_REQUESTS_PROTOCOL_VERSION = 2;
/** Milliseconds between polls while a Files panel is open, in the states that can serve a file request. */
export const OWNER_POLL_INTERACTIVE_MILLISECONDS = 1000;
// ownerErrorCodeSchema gains "operation_closed" and "overlay_not_sealed"; OWNER_ERROR_STATUS maps both to 409.
// ownerStopVetoReasonSchema gains "file_operation_pending".
// ownerPollResponseSchema gains, beside dispatches:
// file_requests: z.array(ownerFileRequestSchema).max(LIVE_FILE_MAX_REQUESTS_PER_POLL).default([]),
// The two refinements (drain_reason, no dispatch while draining) are unchanged; file_requests are allowed while draining.
OWNER_PROTOCOL_VERSION stays 1 in F0. Part C changes it to 2 in the same pull request that
makes the supervisor handle file_requests (L3). ownerPollRequestSchema is unchanged.
5.4 Owner error codes
overlay_not_sealed (409): the overlay is not sealed by this owner and generation (it was never sealed, or a newer
owner replaced the seal). operation_closed (409): the request was abandoned, expired, answered already with a
different result, never delivered to this owner, or belongs to another computer. The
supervisor drops the operation on it. The existing rules for 401, 403 and 409 stale_poll
of OWNER_SUPERVISOR_PROTOCOL 9.5 apply to the new route unchanged: a 401 or a superseding 403 ends the supervisor.
5.5 host-protocol/src/edit-overlay.ts (revision 2)
import { z } from "zod";
import { livePathSchema } from "./live-files.ts";
export const OVERLAY_INDEX_VERSION = 1;
export const OVERLAY_MAX_ENTRIES = 100;
export const OVERLAY_MAX_TOTAL_BYTES = 8388608;
export const OVERLAY_INDEX_MAX_BYTES = 262144;
export const OVERLAY_CAS_ATTEMPTS = 3;
export const OVERLAY_APPLY_BUDGET_SECONDS = 30;
export const OVERLAY_SEAL_RETRY_AFTER_MILLISECONDS = 2000;
export const OVERLAY_STOPPING_RETRY_AFTER_MILLISECONDS = 5000;
export const OVERLAY_CLEAR_RETRY_SECONDS = 10;
export const OVERLAY_EARLY_PUBLISH_SECONDS = 5;
export const OVERLAY_INDEX_OBJECT = "overlay/index.json";
export const overlayBlobObject = (sha256: string): string => `overlay/blobs/${sha256}`;
const identifier = z.string().min(1).max(256);
const timestamp = z.iso.datetime();
const sha256Hex = z.string().regex(/^[0-9a-f]{64}$/);
/** A conflict: the disk held something else than the change expected. Derived conflicts (the snapshot moved) are not stored. */
export const overlayConflictSchema = z
.object({
reason: z.enum(["changed", "collision", "not_empty", "kind_mismatch"]),
disk_sha256: sha256Hex.nullable(),
found_at: timestamp,
})
.strict();
const entryBase = {
path: livePathSchema,
edited_by: identifier,
edited_at: timestamp,
revision: z.number().int().positive(),
conflict: overlayConflictSchema.nullable(),
};
const mode = z.string().regex(/^[0-7]{3,4}$/);
/** One pending change. One path has at most one entry. The content of a `put` is the blob named by `sha256`; the other kinds carry no blob. */
export const overlayEntrySchema = z.discriminatedUnion("op", [
z
.object({
...entryBase,
op: z.literal("put"),
sha256: sha256Hex,
size: z.number().int().nonnegative().max(262144),
mode,
base_sha256: sha256Hex.nullable(),
/** The file this put was renamed from (hidden, and deleted at apply with `from_sha256`); null for a plain create or replace. */
from: livePathSchema.nullable(),
from_sha256: sha256Hex.nullable(),
})
.strict(),
z
.object({
...entryBase,
op: z.literal("delete"),
directory: z.boolean(),
recursive: z.boolean(),
/** The effective hash seen: a file's content hash or a symlink's hash (L49); null for a directory. */
base_sha256: sha256Hex.nullable(),
/** The tree hash seen, for a directory. */
tree_sha256: sha256Hex.nullable(),
})
.strict(),
z.object({ ...entryBase, op: z.literal("mkdir"), mode }).strict(),
z
.object({
...entryBase,
op: z.literal("move"),
from: livePathSchema,
directory: z.boolean(),
/** The kept content hash of a moved file or symlink; null for a directory. */
sha256: sha256Hex.nullable(),
/** The tree hash of a moved directory. */
tree_sha256: sha256Hex.nullable(),
size: z.number().int().nonnegative(),
mode,
})
.strict(),
]);
/** Who may write the index: nobody but the Front Door; and not at all while a valid seal exists. */
export const overlaySealSchema = z
.object({ owner_id: identifier, generation: z.number().int().positive(), sealed_at: timestamp, snapshot_generation: z.number().int().nonnegative() })
.strict();
/** The object `overlay/index.json`. Paths are unique; the contents add up to at most 8 MiB. */
export const overlayIndexSchema = z
.object({
version: z.literal(OVERLAY_INDEX_VERSION),
sealed: overlaySealSchema.nullable(),
entries: z.array(overlayEntrySchema).max(OVERLAY_MAX_ENTRIES),
})
.strict()
.refine((index) => new Set(index.entries.map((entry) => entry.path)).size === index.entries.length, { message: "paths are unique" })
.refine(
(index) => {
const sources = index.entries.flatMap((entry) => (("from" in entry && entry.from !== null) ? [entry.from] : []));
return new Set(sources).size === sources.length && sources.every((source) => !index.entries.some((entry) => entry.path === source));
},
{ message: "a source path is moved once and is not also an entry" },
)
.refine(
(index) => index.entries.reduce((total, entry) => total + (entry.op === "put" ? entry.size : 0), 0) <= OVERLAY_MAX_TOTAL_BYTES,
{ message: "content over 8 MiB" },
);
/** POST /orgs/{tenant}/host/owner/overlay-seal: after extraction, close the overlay to member writes and get the entries to apply. */
export const ownerOverlaySealRequestSchema = z.object({ owner_id: identifier, generation: z.number().int().positive() }).strict();
export const ownerOverlaySealResponseSchema = z.object({ entries: z.array(overlayEntrySchema).max(OVERLAY_MAX_ENTRIES) });
const entryReference = z.object({ path: livePathSchema, revision: z.number().int().positive() }).strict();
/**
* POST /orgs/{tenant}/host/owner/overlay-applied: after a publish that began after the apply, clear exactly these entries;
* and/or report entries that were not applied because the disk held a third version.
*/
export const ownerOverlayAppliedRequestSchema = z
.object({
owner_id: identifier,
generation: z.number().int().positive(),
cleared: z.array(entryReference).max(OVERLAY_MAX_ENTRIES).default([]),
conflicts: z.array(entryReference.extend({ reason: overlayConflictSchema.shape.reason, disk_sha256: sha256Hex.nullable() })).max(OVERLAY_MAX_ENTRIES).default([]),
})
.strict();
export const ownerOverlayAppliedResponseSchema = z.object({ remaining: z.number().int().nonnegative() });
export type OverlayIndex = z.infer<typeof overlayIndexSchema>;
export type OverlayEntry = z.infer<typeof overlayEntrySchema>;
A schema test proves: a 101st entry fails, duplicate paths fail, a source path moved twice or also an entry fails, 8 MiB of put content plus one byte fails (deletes, moves and
mkdirs add none), an entry with a path outside /workspace fails, each of the four kinds parses, cleared and conflicts default to empty, and an index with a seal parses.
6. Routes
6.1 Owner routes (audience computer-owner)
| Method and path | Request | Response (200) | Refusals |
|---|---|---|---|
POST /orgs/{t}/host/owner/poll (changed) |
ownerPollRequestSchema (unchanged) |
ownerPollResponseSchema with file_requests |
as before |
POST /orgs/{t}/host/owner/file-results (new) |
ownerFileResultRequestSchema |
ownerFileResultResponseSchema |
401, 403 as every owner route; 409 operation_closed; 422 invalid_request; 503 when the audit or request store write fails (the supervisor retries) |
POST /orgs/{t}/host/owner/overlay-seal (new) |
ownerOverlaySealRequestSchema |
ownerOverlaySealResponseSchema (the entries to apply) |
401, 403 as every owner route; 422; 503 when the overlay store cannot be written (the supervisor retries inside its budget) |
POST /orgs/{t}/host/owner/overlay-applied (new) |
ownerOverlayAppliedRequestSchema |
ownerOverlayAppliedResponseSchema |
401, 403 as every owner route; 409 overlay_not_sealed; 422; 503 (the supervisor retries) |
POST /orgs/{t}/host/owner/stop-request (changed) |
unchanged | ownerStopResponseSchema, a new veto file_operation_pending |
as before |
The three new routes (file-results, overlay-seal, overlay-applied) are added to OWNER_COMPUTER_ROUTES. Authentication is section 5.1 of
OWNER_SUPERVISOR_PROTOCOL, unchanged: signature, audience, tenant, computer, generation,
live owner. Additionally the handler requires that the request item names this owner id,
this computer and this generation (409 operation_closed otherwise).
6.2 Member routes (audience user)
All under /orgs/{tenant_id}/computer/files, resolved by resolveUserPrincipal
(http/user-principal.ts) exactly as the Phase 1 routes are. Every response carries
X-Content-Type-Options: nosniff and Cache-Control: private, no-store; content responses
add Vary: Authorization. A refusal of the principal is the Phase 1 { "detail": ... } 403.
| Method and path | Request | Response (200) | Notes |
|---|---|---|---|
GET ?path= |
path |
liveFileListingResponseSchema |
live or snapshot (7.2) |
GET /stat?path= |
path |
liveFileStatResponseSchema |
live: the entry as it is, without following a link and without a hash |
GET /content?path=&offset=&length=&variant= |
as Phase 1; variant=mine (only for an entry in conflict) returns the person's version |
liveFileContentResponseSchema, header ETag: "<sha256>" |
masked: masked: true, no content |
PUT /content?path= |
liveFileWriteBodySchema; header If-Match: "<effective sha256>" or If-None-Match: * |
liveFileWriteResponseSchema, header ETag |
7.3: to the owner when it is live, otherwise into the overlay (7.7.3) |
POST /reveal |
liveFileRevealBodySchema |
liveFileContentResponseSchema with masked: false, revealed: true; Cache-Control: no-store |
audited; both sources |
POST /watch |
liveFileWatchBodySchema |
liveFileWatchResponseSchema |
7.5; starts nothing |
DELETE ?path=&recursive= |
header If-Match: "<effective hash>" (L49) |
liveFileDeleteResponseSchema |
7.8; a non-empty directory needs recursive=true and the owner role |
POST /move |
liveFileMoveBodySchema; header If-Match: "<effective hash of from>" |
liveFileMoveResponseSchema |
7.8; never replaces an existing path |
POST /directories |
liveFileMkdirBodySchema |
liveFileMkdirResponseSchema |
idempotent |
GET /overlay |
none | { "entries": [overlay entries] } (5.5) |
the pending changes, for the panel |
POST /overlay/discard |
liveFileDiscardBodySchema ({ path, revision }) |
{ "discarded": true } |
any enabled member; audited; works for every operation; only while the overlay is writable |
Header rules for PUT: exactly one of the two preconditions (both or neither: 428
precondition_required); If-Match takes one strong quoted 64 hex digit value ("abc...";
*, a list, a weak W/ value or a malformed value: 400 invalid_body); If-None-Match takes
* only. CloudFront forwards both headers on /api* (OriginRequestPolicy.ALL_VIEWER_EXCEPT_HOST_HEADER,
allowedMethods: ALLOW_ALL, infra/lib/web-stack.ts:139-152) and the web app and the API share an origin, so there is no
CORS to configure. ASSUMPTION: CloudFront does not strip If-Match or alter ETag; Part A2 checks it on development.
6.3 Member error codes and statuses
The body is { "error": code, "message": text } plus current_sha256, retry_after_ms or
kept_as where the code says so. LIVE_FILE_ERROR_STATUS is this table.
| Code | Status | Meaning |
|---|---|---|
invalid_path |
400 | not a workspace path (lexical) |
invalid_range |
400 | offset or length not a non-negative integer (existing) |
invalid_body |
400 | body or precondition header malformed |
path_refused |
403 | a .git segment, lexical or resolved; audited |
not_found |
404 | no such path (live answer, never replaced by the snapshot) |
parent_missing |
404 | creating a file in a directory that does not exist |
no_snapshot |
404 | existing: nothing published and no live owner |
already_exists |
409 | If-None-Match: * on a path that exists |
precondition_failed |
409 | If-Match hash differs; carries current_sha256 |
computer_starting |
503 | the overlay is sealed and the owner cannot serve yet; carries retry_after_ms (2000); the text is kept |
computer_stopping |
503 | the owner is stopping; carries retry_after_ms (5000) |
owner_too_old |
409 | a running owner of protocol 1 would never apply an overlay (L47) |
edit_unsupported |
409 | a customer-account computer (L46) |
overlay_full |
409 | 100 entries of any kind or 8 MiB of content (L41) |
path_busy |
409 | an earlier pending directory rename or recursive delete covers this path, or this change would (L53) |
directory_not_empty |
409 | a delete of a directory that has children, without recursive |
owner_role_required |
403 | a recursive delete by a member who is not the organization owner (L51) |
invalid_move |
422 | a rename into the path's own subtree, or of the workspace root |
too_many_entries |
413 | a directory operation over 5000 descendants (L62) |
not_in_saved_copy |
422 | the path is under a directory the snapshot never keeps (L40) |
snapshot_not_indexed |
409 | existing |
overlay_unavailable |
503 | the overlay store cannot be read or written, or its index does not parse (added by part A3); the text is kept |
overlay_contended |
503 | three swaps of the overlay index were lost to other writers in a row (added by part A3); carries retry_after_ms (1000) |
content_too_large |
413 | body over 256 KiB, refused before the owner is asked |
file_too_large |
413 | live read over 8 MiB, or a write that would replace a file over 256 KiB |
range_too_large |
413 | existing |
not_a_file, not_a_directory, too_many_symlinks, not_text |
422 | existing, plus not_text for a write of text over a binary file |
precondition_required |
428 | no precondition header |
rate_limited |
429 | a user limit; carries Retry-After seconds |
too_busy |
429 | the computer has 16 open requests |
corrupt_snapshot |
500 | existing |
owner_error |
502 | the helper failed (helper_failed, io_error) on a write |
audit_unavailable |
503 | the audit record could not be written; nothing was queued |
snapshot_store_unavailable |
503 | existing |
owner_warming |
503 | a write when the owner will not poll within 7 seconds; nothing queued; carries retry_after_ms |
owner_timeout |
504 | a write that was never delivered (nothing happened) or expired |
outcome_unknown |
504 | a write was delivered and no result arrived; read the file again |
Owner error to member response, for the cases not in the table above: outside_workspace
and permission_denied become 403 path_refused (with the owner's message), owner_stopping
on a write becomes 503 computer_stopping, owner_busy becomes 429 too_busy, expired
becomes 504 owner_timeout, helper_timeout on a write becomes 504 outcome_unknown (the
rename may have happened), on a read it falls back. For reads, the channel failures of L28 fall back to the snapshot with
fallback_reason owner_timeout (no result in time or helper_timeout, expired),
owner_error (helper_failed, io_error, owner_busy) or owner_stopping.
7. Server behavior
7.1 Records
Computer record (store/codecs/computer.ts) gains two optional fields. Both are removed
with the other owner fields (OWNER_ATTRIBUTE_NAMES, withoutOwner) when an owner ends or a
generation is claimed, and both must be encoded and decoded or a putComputer drops them.
| Field (attribute) | Type | Written by | Meaning |
|---|---|---|---|
interactiveUntilAt (interactive_until_at, epoch seconds) |
date | watch ping, queueing a request | the owner polls at the interactive pace until then; the poll queries the request partition only until then |
fileOperationId (file_operation_id) |
string | queueing a request | the id of the last queued request; the idle stop approval is conditional on it (L11) |
Request item (new codec store/codecs/live-file-operation.ts): partition
{tenant}#computer#fileops, sort key op#{operation_id}; operation_id is fop- plus an id from deps.ids.
| Attribute | Type | Meaning |
|---|---|---|
operation_id, tenant_id, computer_id, user_id |
S | who asked, for which computer |
owner_id, generation |
S, N | the owner the request was queued for |
kind |
S | list, stat, read, write |
status |
S | pending, delivered, done, abandoned |
request |
S | the request as the owner receives it, JSON, without expires_in_ms; removed when abandoned or when the result is read |
queued_at, delivered_at |
S | ISO times; queued_at has second precision of the injected clock |
delivered_sequence |
N | the poll sequence that delivered it |
outcome |
S | liveFileOutcomeSchema JSON; set when done; removed when the Front Door reads it |
waiter_gone |
BOOL | the waiting handler gave up while the status was delivered; the result route then deletes the item |
expires_at |
N | queue time + 120 s; the TTL backstop (L10) |
The per-turn role has no access to either partition (owner-role-policy.json MessagingWritable lists exactly computer#actions and computer#record); the supervisor role has no DynamoDB access at all.
Audit item (new codec store/codecs/file-audit.ts): partition {tenant}#computer#audit,
sort key at#{epoch milliseconds, 13 digits}#{operation_id}.
| Attribute | Type | Meaning |
|---|---|---|
tenant_id, computer_id, user_id, operation_id |
S | user_id from the principal |
operation |
S | write (replace), create, reveal, discard, overlay_apply, delete, move, mkdir |
to_path, recursive, removed, tree_sha256_before |
S, BOOL, N, S | for move, delete and the directory preconditions (7.8.8) |
path |
S | the normalised path the person named |
resolved_path |
S | as reported by the owner, absent until the result arrives |
source |
S | live, snapshot (a reveal from the snapshot) or overlay (an edit or discard of a pending edit; a reveal of an overlay file) |
owner_id |
S | only for overlay_apply: the owner that applied or found a conflict |
outcome |
S | pending, ok, conflict, refused, failed, unknown |
error_code |
S | the owner or member code for a non-ok outcome |
sha256_before |
S | for a replace, the hash the person expected (If-Match); owner-reported value in sha256_before_reported |
sha256_after |
S | computed by the Front Door from the bytes it sent; the owner's in sha256_after_reported |
bytes |
N | size of the content sent |
secret_name |
BOOL | the path looked like a secret |
revealed |
BOOL | true for a reveal |
kept_as |
S | when the helper kept a concurrently changed copy |
flags |
SS | integrity_mismatch when reported hashes disagree with computed ones |
at, finished_at |
S | ISO times |
expires_at |
N | at + 90 days (D14) |
No file content, no body, no token is ever stored. A reader treats a record past
expires_at as absent and a pending record older than 120 seconds as unknown.
Stored answer. The outcome attribute of the request item is the JSON of the owner's answer deflated (a DynamoDB binary attribute), not text. A listing of 2000 entries is about
540 KB as text, past the 400 KB an item holds (W14 counted only the 256 KiB read), and about a tenth of that deflated; codecs/live-file-operation.ts encodeOutcome and its decode are the one place that knows.
Worker item (store/codecs/worker.ts): gains pollPaceMilliseconds (poll_pace_ms), the pace the control plane last told the owner (part A1). Expected pickup (L27) is
lastHeartbeatAt + pollPaceMilliseconds - now, and the watch answer reports the pace from it. It is cleared with the other poll fields when the owner exchanges its bootstrap token.
Rate counters: partition U#{user_id} (the user partition of dynamo-messaging-store.ts),
sort key file_rate#{class}#{bucket}, attributes attempt_count, expires_at; the new store
method incrementRateCounter(userId, name, now, windowSeconds) returns the count, and
incrementOrganizationCreationAttempts is re-expressed through it (one implementation).
7.2 Reading: choosing live or snapshot
liveFileSourceOf(computer, worker, now) -> "live" | { snapshot, reason }
computer is null or has no owner -> snapshot, no reason, stale=false (the computer is off)
not liveOwnerOf(computer, worker, now, 30) -> snapshot, no reason, stale=false (heartbeat lapsed: treated as off)
computer.ownerProtocolVersion < 2 -> snapshot, owner_too_old, stale=true
worker.pollState in {booting, hydrating} -> snapshot, owner_not_ready, stale=true
worker.pollState is stopping, or computer.ownerStopApprovedAt set -> snapshot, owner_stopping, stale=true
computer.hydrateRequired -> snapshot, owner_not_ready, stale=true
else -> live
Steps of a GET (list, stat, content), first failure wins:
resolveUserPrincipal(member of the path organization, organization enabled): else the Phase 1 refusal. 429rate_limitedfrom the read counter.livePathProblem(400).getComputer,getWorker(computer.ownerId),liveFileSourceOf.- Snapshot: the Phase 1 service with the provenance fields added and the overlay merged over it (7.7.2).
A computer with no snapshot answers an empty
/workspaceplus the overlay;no_snapshot(404) only when there is neither. A live-eligible computer never reaches here. - Live:
expected pickup(L27). More than 7 seconds: set the interactive window (a conditional update of the record, owner still live), answer from the snapshot withowner_warmingandretry_after_ms.- Fewer than 16 open requests (a
Query ... Select COUNTof the partition): else 429too_busy. - Queue the request in one transaction:
Updatethe computer record (SET interactive_until_at = :t, file_operation_id = :op, conditionowner_id = :owner AND owner_generation = :g AND attribute_not_exists(owner_stop_approved_at)) andPutthe request item (attribute_not_exists(pk), statuspending,expires_at). A canceled transaction is treated as "the owner stopped": snapshot,owner_stopping. - Wait: loop, strongly consistent
GetItemof the request item every 250 ms, untildoneor 8 seconds on the injected clock. done: delete the item, map the outcome (6.3). A secret-lookingresolved_pathon a read that was not a reveal and notmasked(an owner that did not mask) is masked here: content dropped,masked: true(L25).- Timeout (8 seconds after queueing): one conditional update
pending -> abandonedthat removesrequestandoutcome. Won: fall back to the snapshot withowner_timeout. Lost because the status isdelivered: the owner has the request. A read does not wait longer: it setswaiter_goneon the item (conditional ondelivered) and falls back to the snapshot. Lost because the status isdone: read the outcome as in step 5.
staleis true for any snapshot answer whosefallback_reasonis not null. A live answer hassource: "live",stale: false,as_of= the Front Door's clock at the answer.
editable (content only): source === "live" and !masked (or revealed) and !binary and !truncated and size <= 262144.
7.3 Writing: PUT .../files/content
One request, two paths: the owner when it can serve, the overlay otherwise. First failure wins.
- Principal:
kind === "user"or"integration",userId !== null, enabled organization (L24). 403 otherwise. - Write rate counter (429). Parse the precondition headers (428, 400). Parse the body with
liveFileWriteBodySchema(400invalid_body; a body over 256 KiB is 413content_too_large, checked fromContent-Lengthfirst and again after parsing). livePathProblem(400). The normalised path has a.gitsegment: write the audit record with outcomerefused,error_code path_refused, answer 403. (A refusal is audited, L29.)- Route. Read the computer record and the owner's worker item. First match wins:
- The organization is a customer-account organization: 409
edit_unsupported(L46). liveFileSourceOfislive(L2): the live path, steps 5 to 12 below.- A live owner (
liveOwnerOf) with protocol version below 2: 409owner_too_old(L47). - A live owner that is
stoppingor hasownerStopApprovedAt, and the overlay is sealed by it: 503computer_stopping. - A live owner that is booting or hydrating: read the overlay index; sealed by it: 503
computer_starting; not yet sealed: the overlay path (7.7.3). - No live owner (off, lapsed heartbeat,
hydrateRequired, any stale seal): the overlay path (7.7.3). None of these refusals is audited (nothing was attempted).
- The organization is a customer-account organization: 409
The live path:
- Fewer than 16 open requests (429
too_busy). expected pickup(L27): too slow: set the interactive window and answer 503owner_warmingwithretry_after_ms; nothing is queued and nothing is audited. The UI keeps the person's text and asks again at that time.- Write the audit record,
outcome: "pending"(L21). If the write fails: 503audit_unavailable, nothing queued. markComputerDiskDirty(L14).- Queue the request (transaction of 7.2 step 4.3). Canceled (the owner stopped in between): finalize the audit record
failed/owner_stopping, answer 503computer_stopping. - Wait (7.2 step 4.4). On
donedelete the item and map the outcome:okwrite: 200,liveFileWriteResponseSchemawithsource: "live",ETag: "<sha256_after>",backed_up: false.- error
precondition_failedoralready_exists: 409 withcurrent_sha256. - the other owner errors per 6.3.
- Timeout:
pending -> abandonedconditional update. Won: 504owner_timeout, nothing happened, the audit record is finalizedfailed. Lost because the status isdelivered: wait up to 7 more seconds (LIVE_FILE_OUTCOME_GRACE_MILLISECONDS); still nothing: setwaiter_goneon the item (conditional ondelivered) and answer 504outcome_unknown. The audit record stayspendingand becomesokorfailedwhenever the result route hears the owner.
The result route (7.4) finalizes the audit record of a live write. The waiting handler never writes the outcome, so
a late result is still recorded. A create (If-None-Match: *) is audited as create; a replace as write.
7.4 The result route and the poll
POST /host/owner/file-results handler, after authentication (6.1):
- Parse (422).
owner_idandgenerationequal the principal's (403credential_mismatch). - Read the item (consistent). Absent, past
expires_at, other owner, other computer, other generation, statusabandonedorpending: 409operation_closed. Statusdone: if the stored outcome equals this one, 200 (idempotent); else 409. - The result's
kindmust equal the request'skind(422invalid_request). - One conditional update
delivered -> donesettingoutcome. Lost: re-read and treat as step 2. If the item haswaiter_gone, the handler deletes the item after step 5 and does not keep the outcome (the content of a read nobody is waiting for is not stored). - Finalize the audit record (conditional on it existing): outcome from the result (
ok,conflictforprecondition_failedandalready_exists,refusedforpath_refused,failedotherwise),resolved_path,kept_as, the reported hashes, and theintegrity_mismatchflag whensha256_afterreported differs from computed. A failure here: 503 (the supervisor retries; the update in step 4 is idempotent). For operations that carry no audit record (reads) there is no step 5. - 200
{ "recorded": true }.
In pollOwner, after "settle lost actions" and before "drain" (the order of 5.3 of the supervisor contract; drain is computed after), when request.protocol_version >= 2, the request state is idle, serving, publishing or draining, computer.ownerStopApprovedAt is unset and computer.interactiveUntilAt > now:
Querythe partition for items withstatus = pendingfor this owner and generation, oldest first, limit 4, ignoring items pastexpires_at; plus the items withstatus = deliveredanddelivered_sequence = request.sequence(a repeated poll).- For each pending one, conditional update
pending -> delivered,delivered_at,delivered_sequence = request.sequence, owner id. A lost update skips it. - Return them in
file_requestswithexpires_in_ms = max(0, 8000 - (now - queued_at)). A request already past 8 seconds is not delivered (it is being abandoned).
Cadence: next_poll_ms = ownerPollCadence(state, idleSeconds); if interactiveUntilAt > now and the state can serve files, min(that, OWNER_POLL_INTERACTIVE_MILLISECONDS) (L7). ownerPollCadence keeps its signature; the interactive clamp is a second function ownerFilePanelPace.
Stop request. stopVeto for reason: "idle" gains, after the parked work check: any non-expired item with status pending or delivered for this computer: veto file_operation_pending. approveOwnerStop gains a second guard, attribute_not_exists(file_operation_id) OR file_operation_id = :seen, where :seen is the value read when the predicate was judged; the existing three-attempt re-judge loop covers a lost race (L11).
7.5 Reveal and watch
Reveal (POST /reveal): steps 1, 2 (reveal counter), 3 of a GET, then the audit record
(operation: "reveal", revealed: true, secret_name as the predicate says; fail closed with
503), then the same live-or-snapshot choice. Live: a read request with mask: false. Snapshot: the Phase 1
service with a reveal option that skips the masking. The response is a content response with
masked: false, revealed: true. The audit record is finalized ok by the Front Door for the snapshot source and by the
result route for the live source. A reveal of a path that does not look secret is still a read and still audited.
Watch (POST /watch): principal, watch counter. open: true: if there is a live owner able to serve files
(liveFileSourceOf is live), conditionally update the record (owner_id, generation, not stop-approved)
SET interactive_until_at = now + 45 s and answer state: "live" with poll_interval_ms = the pace the owner was last
given (ownerFilePanelPace). state: "starting" when an owner exists in booting or hydrating or the
computer has an unexpired start lease; state: "off" otherwise. open: false: if live,
SET interactive_until_at = now; answer as above. The ping does not queue a request, does not touch the idle state and
is not audited (so a forgotten panel does not keep the computer awake, L12, O2).
7.6 Retention and cleanup
| Item | Deleted by | Backstop |
|---|---|---|
Request item, status done |
the waiting Front Door handler after it reads the outcome | expires_at + 120 s |
Request item, delivered whose waiter gave up (waiter_gone) |
the result route when the late result arrives (it deletes after finalizing the audit record) | expires_at |
Request item, abandoned |
payload removed at once; the empty item expires | expires_at |
| Audit record | never by code | expires_at = 90 days |
| Rate counter | never | expires_at = window + 1 hour |
Readers ignore any item past expires_at. Part A1's contract test inserts an expired item and
proves the poll, the result route and the count do not see it.
7.7 The edit overlay (revision 2)
7.7.1 Objects
In the computer's snapshot prefix, tenants/{T}/computers/{K}/:
| Object | Content | Written by |
|---|---|---|
overlay/index.json |
overlayIndexSchema (5.5): sealed, and one entry per edited path with sha256, size, mode, base_sha256, edited_by, edited_at, revision, conflict |
the Front Door, only, by compare-and-swap (L31) |
overlay/blobs/{sha256} |
the UTF-8 text of one version, at most 256 KiB | the Front Door, only, If-None-Match: *; deleted by the Front Door when no entry names it |
The supervisor reads them (its role already allows GetObject on the prefix) and is denied every write (12.8). Nothing in the
overlay names a path in an S3 key: blobs are addressed by hash, so no member-chosen string becomes a key.
7.7.2 The effective view
This subsection describes put entries; the effective view with delete, mkdir and move entries is 7.8.3, and the effective hash of a symlink or a directory is 7.8.2.
Inputs: the snapshot index (an empty one holding only the directory /workspace when nothing was published) and the overlay
index (a conditional GET with the cached ETag; an absent object is an empty overlay). For each path with an entry E, and s the
snapshot hash of the path (absent is a value):
| Condition | State | Shown as |
|---|---|---|
s == E.sha256 |
applied (the snapshot already holds the edit; the entry waits to be cleared) | the snapshot, overlay: "none" |
E.conflict == null and s == E.base_sha256 (both absent counts) |
pending | the entry: overlay: "pending", size, sha256, mode, modified_at = edited_at |
anything else, or E.conflict != null |
conflict | the snapshot (absent if s is absent), overlay: "conflict", mine_sha256 = E.sha256 |
- List: the snapshot's children of the directory, plus the pending and applied-looking entries whose parent is the directory and which the snapshot lacks (new files); same order and the same 2000 cap.
- Stat, content, reveal: a pending entry is served from its blob; the bytes are hashed and must equal
E.sha256(else 500corrupt_snapshot, logged).variant=mineserves the blob of an entry in conflict. Masking and binary sniffing are the snapshot's. - Effective hash (L32) is the shown
sha256: the entry's when pending, otherwise the snapshot's. This is theETag. - A write whose content hash equals the effective hash is a no-op: 200, nothing written, no audit record.
7.7.3 Writing, discarding
Reached from 7.3 step 4.5 or 4.6. Steps, first failure wins:
- Path rules (L40) against the snapshot index: a segment in
PACK_EXCLUDED_DIRECTORY_NAMESis 422not_in_saved_copy; the path or an ancestor is a symlink in the index, 403path_refused; the path is a directory, 422not_a_file; the parent directory is not in the index (and is not/workspace), 404parent_missing. GET overlay/index.json(ETagE, or absent), parsed withoverlayIndexSchema. A valid seal (L33) was already handled by 7.3 step 4; a seal that is present but stale is carried into step 7 and cleared by the swap.- Judge the precondition against the effective hash (7.7.2).
If-Matchdiffers: 409precondition_failedwithcurrent_sha256(the snapshot's for a conflicted path).If-None-Match: *and the path exists in the effective view: 409already_exists. - Caps (L41): the entry count after the write and the sum of sizes: else 409
overlay_full. - Write the audit record (
source: "overlay",outcome: "pending"); 503audit_unavailableif it cannot, nothing written. PUT overlay/blobs/{sha256}withIf-None-Match: *; a 412 means it exists and is fine.- Build the new index: the entry
{ path, sha256, size, mode, base_sha256, edited_by, edited_at, revision, conflict: null }wheremodeis the existing entry's, else the snapshot entry's, else"664";base_sha256is the existing entry's, else the snapshot hash of the path (null for a create), and the snapshot hash when the existing entry was in conflict (the person overwrote on purpose);revisionis the old one plus 1, else 1;sealed: null.PUT overlay/index.jsonwithIf-Match: E(If-None-Match: *when absent). - A failed swap (412): re-read and go back to step 3, at most
OVERLAY_CAS_ATTEMPTS(3) times; a changed effective hash is the 409 of step 3. - Finalize the audit record
ok; delete the replaced blob if no entry names it (best effort); answer 200,liveFileWriteResponseSchemawithsource: "overlay",backed_up: true,ETag: "<sha256>".
Discard (POST .../overlay/discard): principal, write counter, path, 7.3 step 4 (so a sealed overlay is 503), read the index; the entry must exist with
this sha256 (409 precondition_failed otherwise); audit record (operation: "discard"); swap the entry out (same loop); delete its blob. Any
enabled member may discard (the workplace is shared and every discard is audited).
7.7.4 The owner: seal, apply, clear
In the supervisor's hydrate() (section 9.4), in this order and before the supervisor leaves hydrating:
- Extract the snapshot and share the workspace with the shell user (existing). Also when the local cache matched, and also when there is no snapshot (L42).
POST overlay-seal. The Front Door authenticates the owner (section 5.1 of the supervisor contract), reads the index, and writes it withsealed = { owner_id, generation, sealed_at, snapshot_generation }(the computer record'ssnapshotGenerationat that moment) by compare-and-swap (creating it withIf-None-Match: *if absent). A seal already held by this owner and generation is returned as is (a retry). A seal of anyone else is stale by construction (this owner is the live owner) and is replaced. A lost swap re-reads: a member edit that won the swap is in the set. The response carries every entry.- Apply every entry without a stored
conflictin the order and with the rules of 7.8.5 (mkdir, directory moves, file moves, puts, deletes of files, deletes of directories). For aput: read its blob from S3, check the bytes hash toentry.sha256, and call the file helperwritewithif_match base_sha256(if_none_matchwhenbase_sha256is null) andrefuseGitSegments.ok: applied.precondition_failedwhosecurrent_sha256equalsentry.sha256: already applied. Any other outcome ofprecondition_failedoralready_exists: conflict. The other operations use the helper operations of 8.1 and the outcomes of 7.8.5. A blob that cannot be read or does not hash, or a helper failure other than those, fails the hydrate (L34). - If there are conflicts,
POST overlay-applied { conflicts }, each with thereasonthe owner found (changed,collision,not_emptyorkind_mismatch, the column of 7.8.5) and thedisk_sha256it saw. The Front Door storesconflict: { reason, disk_sha256, found_at }on each entry whoserevisionstill matches, and writes anoverlay_applyaudit record with outcomeconflict. Failing this post is logged and retried; it does not fail the hydrate. - Remember the applied set. If the disk now equals the published index (nothing to publish) go to 7 at once.
- Return from
hydrate(); the supervisor is idle. The first publish check is 5 seconds later (L36). A turn that arrives first runs; the entries stay. - When a publish starts, capture the applied set; after
publishComputerSnapshotsucceeds,POST overlay-applied { cleared }for that set. The Front Door removes each entry whosepath,revisionandsha256match and for which a publish is on record (the rule of L37; an entry without one simply stays, andremainingsays so), deletes blobs no entry names, writesoverlay_applyrecords (outcomeok), and answersremaining. Entries not named stay. A failed post is retried every 10 seconds and before the stopped report.
The seal ends with the owner: the stopped report unseals (one swap). A seal left by a dead owner is stale (L38).
7.7.5 Transitions
| From | To | Overlay writes | Reads | What happens |
|---|---|---|---|---|
| Off | Starting (the starter ran, no bootstrap yet) | accepted into the overlay | snapshot + overlay | nothing about the overlay; nothing started by an edit |
| Starting | Hydrating (bootstrap done, extracting) | accepted (no seal yet) | snapshot + overlay, stale owner_not_ready |
edits made now are in the sealed set |
| Hydrating | Sealed and applying | 503 computer_starting, retry in 2 s |
snapshot + overlay | seal swap; apply; no turn can run |
| Applying | Live (idle) | through the owner | the owner | entries stay until a publish that began after the apply is recorded |
| Live | Live after publish and clear | through the owner | the owner | entries removed; blobs deleted |
| Live | Stopping (stop approved) | 503 computer_stopping, retry in 5 s |
snapshot + overlay (stale owner_stopping) |
final publish, final clear, stopped report unseals |
| Stopping | Off | accepted into the overlay | snapshot + overlay | normal |
| Live or hydrating | Crash (no stopped report) | for up to 30 s the owner still looks live; then accepted | for up to 30 s as live; then snapshot + overlay | the seal is stale once the heartbeat lapses; entries were not cleared, so nothing is lost; live-only edits since the last publish are lost (accepted, O4) |
| Live | Replaced (a newer generation while the old one lingers) | accepted until the new owner seals | snapshot + overlay | the new owner's seal replaces the stale one; the old owner's calls are refused (403 stale_generation) and its S3 writes are denied by IAM |
| Live, protocol 1 | any | 409 owner_too_old |
snapshot + overlay (stale owner_too_old) |
until it stops |
| Customer-account computer | any | 409 edit_unsupported |
snapshot | no overlay exists |
7.7.6 The races, exactly
- R1, an edit against the seal. Both are compare-and-swaps of
overlay/index.json. The one that swaps first wins. An edit that loses re-reads, sees the seal and answers 503computer_starting(an unreferenced blob remains); an edit that won is in the set the seal returns. There is no third outcome. - R2, an edit while the snapshot is being extracted. The seal is written after extraction, so the edit is in the sealed set and is applied before any turn.
- R3, two members editing while off. Each judges against the effective hash it read; the second swap fails if the first changed the index; it re-reads and re-judges.
The same path with a now-stale
If-Match: 409 with the first member's hash. Different paths: both stand. - R4, the apply against a live edit. No file operation is delivered before the supervisor leaves
hydrating(the poll delivers only inidle,serving,publishing,draining). The clear namespath,revisionandsha256that were applied, never what the disk holds now. - R5, a lost clear. Nothing is removed until the control plane records it. An applied entry that stays is idempotent at the next hydrate (the disk equals the entry: already applied). Only if the bot also changed the file in a later successful publish and the clear was lost until the next hydrate does the owner find a third hash and report a conflict; that false conflict needs two failures in a row and is surfaced, not resolved silently. The supervisor retries every 10 seconds and before the stopped report to make it rare.
- R6, a takeover in the middle of the apply. The new owner seals again (the old seal is stale, replacing it is one swap) and applies from the snapshot on its own disk. The old owner's disk is not the new one's.
- R7, a snapshot published by something that does not know the overlay (a customer host, a protocol 1 owner): the entry is a derived conflict at the next read and a stored conflict at the next apply. The person's text is kept and readable; nothing is overwritten.
- R8, a discard against the seal. The same compare-and-swap as R1.
- R9, the audit. The record is written before the blob and the swap, so every accepted edit has a record; one for an edit that then lost a swap is finalized
failed.
7.7.7 Size and cost
Caps in L41. Cost in section 19.
7.8 Delete, rename and new folders (revision 3)
7.8.1 Routes, roles, preconditions
Routes are in 6.2. Principal and rate rules are those of a write (7.3 steps 1 to 3, class write); the route decision is 7.3 step 4 (live path or overlay path) for every one of them.
Who may (L51). The original design (7.6, D8) said "Delete and move, when added, need the owner role". That no longer stands for the common case: delete, rename and new folders now matter to every member,
who can already change a file and can already tell a bot to delete anything. Recommendation, adopted in the contract: any enabled member may delete a file or a symlink, rename a file, a symlink or a directory, create a
directory, and delete an empty directory, each audited with who, what and the hash seen; only the organization owner role may delete a non-empty directory (recursive=true), the one bulk, hard-to-undo action.
A member who tries gets 403 owner_role_required, and the refusal is audited as refused (7.8.8 decides; an earlier draft said it was only logged). recursive=true needs the owner role whether or not the folder has children: the flag is the request for the bulk delete, so the rule is one rule on both paths and an empty folder is deleted by a member with recursive absent. Renaming a non-empty directory is not destructive and is open to members.
Preconditions (L49, L50). DELETE and move require If-Match of the effective hash of the path as the person last saw it; the ETag of stat and of content is that hash (a directory's is its tree hash). A missing
header is 428; a different hash is 409 precondition_failed with current_sha256 (the tree hash for a directory). POST /directories takes no precondition: it is idempotent.
7.8.2 The effective hash
File: SHA-256 of the content. Symlink: SHA-256 of symlink, a NUL byte, then the target. Directory (the tree hash): SHA-256 over, for every descendant of the directory in the effective view sorted by
relative path (bytes), the line kind NUL relative-path NUL value LF, with kind one of file, symlink, dir, and value the size of a file, the target of a symlink, or 0. The tree hash of an empty directory is the hash of
nothing. The Front Door computes it from the merged snapshot index and overlay; the helper computes the same from lstat and readlink (no file is read). A directory with more than 5000 descendants is 413 too_many_entries for a
delete, a rename and a tree hash.
7.8.3 Reads: how the four operations merge
Added to the rules of 7.7.2. s is the snapshot's state of a path.
| Entry at P | Effective view | Derived state (against the snapshot) |
|---|---|---|
delete of a file or symlink |
P and its hash are absent: not listed, stat and content 404 not_found |
s has P with the entry's base_sha256: pending. s has P with another hash: conflict (shown as the snapshot's file, overlay: "conflict"). s has no P: applied |
delete of a directory (empty, or recursive) |
P and everything below it are absent | s has the directory and its tree hash equals the entry's tree_sha256 (recursive) or it has no children (non-recursive): pending. Otherwise conflict. s has no P: applied |
mkdir |
P is a directory, overlay: "pending", with the children the snapshot has under it (none if it does not exist there) |
s has a directory at P: applied. s has a file or symlink at P: conflict |
move of a file or symlink from F |
P is the file with the kept hash, size and mode; its content is read from the snapshot pack at F's member; F is absent | s has F with the kept sha256 and no P: pending. s has P with the kept hash and no F: applied. Anything else: conflict (F changed, or P exists) |
move of a directory from D |
P lists what the snapshot lists under D (paths re-rooted: P/x is the snapshot's D/x); D is absent | s has D with the kept tree_sha256 and no P: pending. s has P with that tree hash and no D: applied. Anything else: conflict |
put with from F |
P is the file of the blob; F is absent | as a put (7.7.2), and F in s must still have from_sha256 |
- List. The children of a directory are the snapshot's children, minus every path hidden by a
deleteor by the source of amoveorputwithfrom, plus the entries whose parent is the directory (puts, mkdirs, move targets). A hidden directory hides its subtree. Order and the 2000 cap are unchanged. - Stat, content, reveal. A path under a moved directory is translated to the snapshot path before lookup (the exclusivity of L53 means no other entry is below it). A moved file's content is a ranged read of the snapshot member of its source, verified against the kept hash.
- Pending changes.
GET .../files/overlayreturns the entries so the panel can mark them and offer a discard.
7.8.4 Recording a change (the write algebra)
Steps 1 to 5 of 7.7.3 apply to every operation (path rules L40 and L59, judge the precondition against the effective hash, caps, audit intent first, fail closed), then the blob step only for a put, then the swap
of 7.7.3 steps 7 and 8 with the new set of entries, then the audit finalization. The new set is computed from the entry at the path and the operation:
| Request | State of the path | New set of entries |
|---|---|---|
put (create) |
absent; no delete |
add put, base_sha256 null |
put |
a delete of a file at P |
replace by a put whose base_sha256 is the snapshot hash |
put (replace) |
pending put, snapshot file, or move target |
as 7.7.3 step 7; a move target becomes a put with from and from_sha256 (its source stays hidden) |
delete a file or symlink |
snapshot file or symlink | add delete, base_sha256 = its effective hash |
pending put, no base, no from |
remove the entry | |
pending put with a base |
replace by a delete with that base |
|
pending put with from F |
replace by a delete of F with from_sha256 as base |
|
target of a move from F |
replace by a delete of F with the kept hash as base |
|
| a conflicted entry | replace by a delete with the snapshot (theirs) hash as base (the person chose) |
|
delete a directory |
empty in the effective view | add delete { directory, recursive: false }; a pending mkdir there is removed instead |
not empty, recursive, owner role, tree hash |
add delete { directory, recursive: true, tree_sha256 } and drop every entry below it; a dropped move target or put with from becomes a delete of its source |
|
not empty, not recursive |
409 directory_not_empty |
|
move a file or symlink F to T |
snapshot file | add move { path: T, from: F } with the kept hash, size, mode |
| pending create | the entry's path becomes T | |
pending put with a base |
replace by a put at T (same blob) with from: F, from_sha256 the base, base_sha256 null |
|
a put with from G, or a move from G |
the entry's path becomes T, keeping from G; if T equals G the entry is removed |
|
move a directory D to E |
snapshot directory, no entry at or under D or E | add move { directory, path: E, from: D, tree_sha256 } |
| any entry at or under D or E | 409 path_busy |
|
mkdir |
absent, parent exists | add mkdir |
| a directory (snapshot, mkdir, or move target) | 200 created: false, nothing written, no audit record |
|
| a file or symlink | 409 already_exists |
|
| any | a path under a directory with a pending move or recursive delete |
409 path_busy |
delete a directory, recursive |
a pending change elsewhere takes a file out of the directory (its from lies under it) |
409 path_busy; every other entry under the directory is dropped as in the row above |
A request that names a missing parent is 404 parent_missing; a target that exists is 409 already_exists; a source that does not exist is 404 not_found.
7.8.5 Applying at hydrate: order and collisions
The supervisor applies the sealed set (7.7.4 step 3) in this order, each operation through the file helper with its precondition (8.1), as uid 2000, containment and refuseGitSegments included:
- mkdir, shallowest path first.
- Directory moves, shallowest source first. Exclusivity (L53) means they do not interact.
- Moves of files and symlinks.
- Puts (creates and replaces), in path order. A
putwithfromis a create at its path; its source is removed in step 5. - Deletes of files and symlinks, including the sources of puts with
from. - Deletes of directories, deepest first. A recursive delete is one tree removal.
| Operation | Applied | Already applied | Conflict (not applied, flagged with reason) |
|---|---|---|---|
mkdir |
the directory is created | a directory exists | a file or symlink exists (kind_mismatch); the parent is missing (collision) |
move file |
from has the kept hash and to does not exist: renamed |
from is gone and to has the kept hash |
to exists (collision); from has another hash or is gone with to absent (changed) |
move directory |
from has the kept tree hash and to does not exist: renamed |
from is gone and to has the kept tree hash |
to exists (collision); the tree hash differs (changed) |
put |
as 7.7.4 step 3 | the disk equals the entry | the disk holds a third hash (changed), or the path is a directory (kind_mismatch) |
delete file or symlink |
the effective hash equals base_sha256: removed |
the path is gone | another hash (changed) |
delete empty directory |
removed | the directory is gone | it has children (not_empty) |
delete recursive |
the tree hash equals tree_sha256: removed |
the directory is gone | the tree hash differs (changed) |
- Rename onto an existing path: always a conflict (
collision), never an overwrite; the request side already refuses it (L52). - Non-empty directory delete: only the recursive entry removes children; a non-recursive one whose directory gained children meanwhile is a conflict (
not_empty). - Rename of a directory: one
rename(2)of the whole tree aftermkdir(target); it fails whole if anything about the target changed. - A conflict is reported with
POST overlay-applied { conflicts }, the entry is kept and flagged (conflict.reason), and the operations that depended on it (a put inside a directory whose mkdir conflicted) reportcollisionfor the parent that is missing. Nothing is retried in the hydrate. - Clear (7.7.4 step 7) after a recorded publish, for every applied or already-applied entry, using the rule of L37; for a non-put entry the Front Door additionally accepts the clear when the snapshot index already shows the result (path gone for a delete, directory present for a mkdir, target with the kept hash and source gone for a move).
7.8.6 The live path
When the owner can serve (7.3 step 4.2) the operation is a file request of kind delete, move or mkdir (5.2) with the same preconditions, queued in the same transaction, delivered by the same poll, answered by the same result route, with the same audit
(intent first, finalized by the result route), the same markComputerDiskDirty, the same abandon and outcome_unknown rules, and backed_up: false. The Front Door has already checked the owner role for a recursive delete and the path rules; the helper checks
containment and .git again. A directory_not_empty, invalid_move, too_many_entries, already_exists or precondition_failed from the helper maps to the member codes of 6.3.
7.8.7 Containment, links, .git
Every path in a request (path, from, to) is validated lexically by the Front Door (livePathProblem, no .., under /workspace) and again by the helper after resolving the parent directory. The final component is never followed: deleting or renaming a symlink acts on the link. An ancestor
that is a symlink is path_refused in the overlay (the snapshot index shows it) and is resolved-and-contained in the helper (a link that leads out of the workspace is refused as for any file operation). A .git segment in path, from or to, lexical or resolved, is
path_refused; a recursive delete of a directory that contains a .git directory below the named one is allowed (it removes, it does not inject). The workspace root and the home of the browser profile cannot be deleted or renamed. Excluded directory names (L40) are refused by the
overlay as not_in_saved_copy and are allowed on the live path. A recursive removal never crosses a mount (st_dev is checked) and never follows a link.
7.8.8 Audit
Records of section 7.1 with operation delete, move or mkdir; path is the source, to_path the target of a move; recursive and removed (the count) for a delete; sha256_before or tree_sha256_before from the If-Match; source live or overlay; no content.
Refusals (a .git path, owner_role_required after authentication, a failed precondition) are audited as for a write; the idempotent mkdir of an existing directory is not. The owner's overlay_apply records name the operation and the outcome (ok, already, conflict with its reason).
7.8.9 Size and cost
Caps in L41 and L62. A delete, a move and a mkdir are index-only: no blob, no content bytes. Cost in section 19.
8. The file helper
conversation/src/pi/owner-file-helper.ts gains operations and fields. Only Node built-ins and
@chatticus/host-protocol/live-file-rules are imported (the rules module imports nothing).
export type OwnerFileOperation = "read" | "write" | "stat" | "realpath" | "list";
export type OwnerFileRequest = {
readonly operation: OwnerFileOperation;
readonly path: string;
readonly allowedRoots: readonly string[];
readonly contentBase64?: string; // write (Pi tools)
readonly content?: string; // write (live): UTF-8 text
readonly precondition?: { type: "if_match"; sha256: string } | { type: "if_none_match" }; // write
readonly createParents?: boolean; // write: mkdir -p, the Pi tools only
readonly refuseGitSegments?: boolean; // every live request
readonly range?: { offset: number; length: number; maxFileBytes: number; mask: boolean }; // read: absent means the whole file, as today
readonly maxEntries?: number; // list
readonly maxScan?: number; // list
};
The response keeps the shape of today's (ok with fields, or ok: false, code, message) and
gains entries, resolvedPath (logical, /workspace/...), sha256, sha256Before, modifiedMs, mode,
created, keptAs, currentSha256, truncated, masked, binary, encoding, size,
offset, length, content. Error codes are LIVE_FILE_OWNER_ERROR_CODES; the existing
CHATTICUS_OUTSIDE_WORKSPACE is mapped to outside_workspace, ENOENT to not_found, EACCES
and EPERM to permission_denied, anything else to io_error. UnprivilegedExecutionEnv's CODES map
(pi/unprivileged-execution-env.ts) keeps working through the same table.
Every operation first resolves the final location (finalLocationOf, as today), asserts it
inside allowedRoots, and if refuseGitSegments asserts the resolved path has no .git
segment (path_refused). Every operation maps the real path back to /workspace/... for
resolvedPath.
list. The resolved path must be a directory (not_a_directory). opendir, read at most maxScan
names (20,000), lstat each, sort directories first then by name (the Phase 1 order), keep
maxEntries. A symlink entry carries link_target from readlink (4096 characters at most). A FIFO,
socket or device is left out. truncated is true if names beyond the cap exist. Entries carry modified_at,
mode, size (files), no hash.
stat. lstat of the final component without following it, after the containment check of its
parent (today's behavior), returned as an entry.
read. Resolve, containment. Open the resolved path with O_RDONLY | O_NONBLOCK; fstat must say regular
file (not_a_file) of at most maxFileBytes (file_too_large). Read it whole, hash it (SHA-256, hex), take
the range, sniff binary with looksBinary, trim a text range to a character boundary. If mask is true and the
resolved path or the requested path looks secret, return masked: true with size and sha256 and no content (the
same as the snapshot's masked answer). With no range the whole file is returned as base64 and no cap applies: that is the
Pi read_workspace path, unchanged.
write. In this order:
- Resolve and contain.
if_none_match:lstatthe final location must beENOENTelsealready_exists. Creating needs the parent directory (parent_missing) unlesscreateParents. if_match: open the final locationO_RDONLY | O_NONBLOCK,fstatregular (not_a_file) and at most 256 KiB (file_too_large), hash. Different:precondition_failedwithcurrentSha256. Keep this descriptor open.- Create
.<name>.chatticus-<32 random hex>.tmpin the resolved directory withO_WRONLY | O_CREAT | O_EXCL | O_NOFOLLOW, write the bytes,fsync,fchmodto the old file's permission bits (0664for a new file). - Compare again immediately before publishing:
if_match: hash the file at the final location again;if_none_match:lstatagain. A difference unlinks the temporary file and answersprecondition_failedoralready_exists. - Publish:
rename(temp, final)for a replace;link(temp, final)thenunlink(temp)for a create (EEXISTisalready_exists). - Post-check (replace only): re-hash through the descriptor of step 2, which still reads the replaced inode. If it
differs from the expected hash, someone wrote that inode in place between steps 4 and 5. Write its bytes to
<final>.chatticus-replaced-<8 hex>(the shell user's own file, in the same directory) and answer ok withkeptAs. - Answer
created,size,sha256Before,sha256(of the new bytes),modifiedMs.
On every failure after step 3 the temporary file is unlinked. A helper killed by its timeout can leave
.chatticus-*.tmp behind; it is visible in a listing and harmless (YAGNI: no sweeper). The window that remains is the time
between step 6's read and the end of the helper (a write into the replaced inode after the post-check): microseconds, and
the result is the same as a shell redirect landing a moment after a save.
The Pi tools' write (UnprivilegedExecutionEnv.writeFile) calls this operation with contentBase64, createParents: true,
no precondition and no refuseGitSegments: it now writes atomically (a reader never sees half a file). The existing
scenarios of computer_owner_file_isolation.feature stay green and are the proof that the link rules did not move.
8.1 Helper operations for delete, rename and new folders (revision 3)
OwnerFileOperation gains mkdir, unlink, rmtree and rename; the request gains to (rename), recursive, and one precondition field, sha256, the effective hash the entry must have (a file's content hash, a link's link hash or a directory's tree hash); the helper decides which by the entry's real kind, so the caller never has to know it. All of them: resolve the parent directory of each path they name
(finalLocationOf) and assert it inside allowedRoots; never follow the final component; refuse a resolved .git segment when refuseGitSegments; answer with the codes of LIVE_FILE_OWNER_ERROR_CODES.
mkdir: the parent must exist (parent_missing);mkdirwith mode0775through the launcher's umask;EEXISTon a directory isokwithcreated: false, on anything elsealready_exists.unlink:lstatthe final component. A file: openO_RDONLY | O_NONBLOCK,fstatregular, hash, compare with the precondition (precondition_failedwithcurrentSha256),unlink, then re-hash through the held descriptor; if it differs from the expected hash (an in-place writer in the window), write those bytes to<name>.chatticus-deleted-<8 hex>beside it and answer ok withkeptAs. A symlink: compare the symlink hash (L49),unlink. A directory:rmdir;directory_not_emptyif it has children.rmtree: only withrecursiveand asha256that is the tree hash. Walk withlstat(never following a link, staying on the starting device, at most 5000 entries, elsetoo_many_entries), compute the tree hash, compare (precondition_failedwith the current tree hash), then remove depth first withunlinkandrmdir. A tree that contains a.gitdirectory below the named path is removed (L59).rename:lstatthe source without following; compare the precondition (the content or symlink hash, or the tree hash of a directory); the target's parent must exist and the target must not (already_exists); the target must not lie inside the source (invalid_move); same device. A file or symlink:link(source, target)(it fails if the target exists) thenunlink(source); the inode is the same, so an in-place writer in the window loses nothing. A directory:mkdir(target)exclusively, thenrename(source, target), which replaces only the empty directory just made;ENOTEMPTYorEEXISTremoves the made directory and answersalready_exists.statwithhash: truereturns the effective hash:treeSha256for a directory (too_many_entriespast 5000) andsha256for a regular file of at mostmaxFileBytes(the live read limit, 8 MiB; a larger file isfile_too_large). A link'ssha256(its link hash) is always returned.
Pi's own file tools do not use these operations. The supervisor's file runner (9.1) maps request kinds delete, move and mkdir to them and passes refuseGitSegments: true.
9. The supervisor (container)
All in computer/host/src/. The bundle must not import Pi, UnprivilegedExecutionEnv, the starter
or the owner stores (computer-image-structure.test.ts). scrubbedShellEnvironment
(conversation/src/pi/local-computer-tools.ts) is moved, unchanged, to
conversation/src/pi/shell-environment.ts (no imports beyond Node) and imported by both.
9.1 The file runner
New live-file-operations.ts, class LiveFileOperations, with ports so a scenario supplies fakes:
export interface FileHelperLauncher {
/** Run the helper once with this request on standard input; kill it at the timeout; resolve with its parsed answer or a helper error. Never rejects. */
run(request: OwnerFileRequestToHelper, timeoutMilliseconds: number): Promise<OwnerFileResponse>;
}
export type LiveFileOperationsOptions = {
readonly launcher: FileHelperLauncher;
readonly postResult: (operationId: string, outcome: LiveFileOutcome) => Promise<"recorded" | "closed">;
readonly clock: SupervisorClock;
readonly log: LogEmitter;
readonly workspaceRoot: string;
readonly memoryUsedPercent: () => number | undefined;
readonly onFinished: (kind: OwnerFileRequest["kind"], outcome: LiveFileOutcome) => void;
};
ProcessFileHelperLauncher (live-file-helper-launcher.ts) starts
<launcher> -c 'exec "$@"' chatticus-file-helper <node> file-helper.mjs with cwd = workspaceRoot and the
scrubbed environment, writes the JSON request to standard input, reads at most 1 MiB of standard output, and at the timeout sends
SIGKILL to the process group. It does exactly what UnprivilegedExecutionEnv.ask does and nothing else; it never opens a
path itself.
LiveFileOperations.accept(requests), called by the supervisor for each poll answer:
- A request id already known (running, queued, or finished within the last 60 seconds) is ignored; a finished one whose post failed is posted again.
- Stopping: the answer is
error owner_stopping. Memory at or above 90 percent, or more than 8 queued:owner_busy. - Otherwise queue; at most 2 helpers run. A queued request whose
expires_in_ms(measured from receipt on the monotonic clock) lapses before it starts is answeredexpiredwithout touching the disk. - The request is turned into a helper request:
refuseGitSegments: true;readgetsrange: { offset, length, maxFileBytes: 8388608, mask };listgetsmaxEntriesandmaxScan: 20000;writegetscontent,precondition. - The helper's answer is validated against
liveFileOutcomeSchema; one that does not parse ishelper_failed. A timeout ishelper_timeout. - The result is posted with up to 3 attempts, 1 second apart, on a network error or a 5xx.
closed(409operation_closed) drops it. A 401 or a superseding 403 ends the supervisor as superseded, as for a poll. onFinishedlets the supervisor restart its idle clock (L12) and, for anokwrite, schedule the quiet publish (L13).
pause() and resume() and stopAccepting() are L17. A paused runner queues; it does not answer. stopAccepting() is
called first thing in performStop and in the SIGTERM path.
9.2 Supervisor changes
OWNER_PROTOCOL_VERSIONbecomes 2.onPollAnsweredhandsresponse.file_requeststoaccept. File requests are accepted in every state exceptbootingandhydrating: the supervisor answers thoseowner_busy(the server does not deliver in those states; this is the belt).onChildExit:await files.pause(); try { await sweep } finally { files.resume() }.performStop:await files.stopAccepting()beforeterminateChild.- On
onFinished: if the state isidle,startIdleClock()(anowner_idleline restarts the count); for a write,nextPublishCheckAtMilliseconds = min(existing, now + OWNER_QUIET_PUBLISH_SECONDS * 1000). While a child runs the publish stays deferred (C14). protocol-client.ts:OwnerSupervisorClient.postFileResult(request)withownerFileResultRequestSchemaandownerFileResultResponseSchema, the same headers and error handling aspoll.supervisor-main.tswiresProcessFileHelperLauncher,fileHelperCommand([process.execPath, join(dirname(import.meta.url), "file-helper.mjs")], asowner-main.ts:23does) andcontainerMemoryUsedPercent.
9.3 Logs
One line per operation, from the supervisor, never a path, a name or a byte of content:
file_operation: kind, bytes, path_hash (first 12 hex of SHA-256 of the path), source (live), outcome
(ok or the error code), duration_ms, queue_ms. Other lines: file_operation_refused
(reason: owner_stopping, owner_busy, expired), file_operation_post_failed (status or error_name,
attempt). Front Door lines (no content, no path): file_op_queued, file_op_abandoned, file_op_delivered,
file_result_recorded, file_result_refused code=<code>, file_audit_failed, file_rate_limited class=<class>.
9.4 The overlay at hydrate (revision 2)
Files: computer/host/src/overlay-applier.ts (new), host-disk-lifecycle.ts, supervisor.ts, supervisor-ports.ts, protocol-client.ts, supervisor-main.ts. The applier uses the
same FileHelperLauncher as the file runner (9.1); it never opens a workspace path itself.
export interface OverlayApplier {
/** Seal the overlay, apply its entries through the file helper, report conflicts. Resolves with the entries now on the disk. Throws OverlayApplyError (the hydrate fails, exit 8). */
applyAfterHydrate(): Promise<{ readonly applied: readonly OverlayEntryReference[]; readonly conflicts: readonly OverlayEntryReference[] }>;
/** After a recorded publish: ask the Front Door to clear exactly these entries; true when acknowledged. */
clearApplied(entries: readonly OverlayEntryReference[]): Promise<boolean>;
}
SupervisorPlanegainssealOverlay(request)andpostOverlayApplied(request)(OwnerSupervisorClient, same headers and error handling aspoll).HostDiskLifecycle.hydrate(): afterbootThroughWorkspace()andafterHydrate(the group sharing that lets uid 2000 write), callapplyAfterHydrate()inside the budget ofOVERLAY_APPLY_BUDGET_SECONDS; the entries it applied are returned to the supervisor.OwnerSupervisor: keepsappliedAwaitingClear. InbeginHydration, afterdisk.hydrate(), setsnextPublishCheckAtMillisecondsto now +OVERLAY_EARLY_PUBLISH_SECONDSwhen entries were applied (else the 600 second default of today).beginPublishCycleand the final publish ofperformStopcapture the set before the pack starts and callclearAppliedafterpublishChangedDisksucceeds;needsPublish() === falseright after the apply clears at once. A failed clear is retried on the 10 second tick (OVERLAY_CLEAR_RETRY_SECONDS) and up to 3 times before the stopped report; it never blocks the stop.- Failure rules: a 401 or a superseding 403 on either route ends the supervisor as superseded (as for a poll); a 5xx or a network error on the seal is retried inside the 30 second budget, then
snapshot_skipped reason=overlay_failedand exit 8; a blob that cannot be read or does not hash to its entry, or a helper failure other thanprecondition_failed, is the same. - Logs (no path, no content):
overlay_sealed(entries),overlay_applied(applied,already,conflicts,duration_ms),overlay_clear_requested(entries),overlay_cleared(remaining),overlay_apply_failed(reason).
10. The web app
Files: web/lib/computer-files.ts, web/lib/computer-files-api.ts, web/lib/touched-files.ts,
web/components/workspace/ComputerFilesPanel.tsx, web/components/workspace/ComputerFilesView.tsx, the fixtures and tests
beside them. Read docs/BRAND_GUIDELINES.md before touching a background, region or border: the new banners are flat
surfaces like the existing notices, with no new border. The web app does not import host-protocol (it is not a dependency of
web/); it mirrors the JSON of section 5.2 in its own types, and the @web scenarios are what keeps the two honest.
10.1 What the panel shows
| Situation | The response says | The panel shows |
|---|---|---|
| Live | source: "live", stale: false |
A source line Live from the computer with a status dot and aria-live="polite"; the listing and file may be edited |
| Computer off | source: "snapshot", stale: false |
Computer is off. Showing the copy saved <relative time> (today's SavedCopyNote wording with the time); the editor is offered (the edit goes into the overlay); a file with overlay: "pending" shows Edited, waiting for the computer |
| Fallback | source: "snapshot", stale: true |
Could not reach the running computer. Showing the saved copy from <time> with a Retry button; the panel asks again after retry_after_ms (default 2 seconds), three times, then waits for a click; no editor |
| Starting | watch state: "starting" |
The computer is starting. Showing the saved copy |
| Warming | fallback_reason: "owner_warming" |
Connecting to the running computer (not an error), asks again at retry_after_ms |
FileListing.source and FileContentRange.source widen to "snapshot" | "live"; both gain stale, fallback_reason,
retry_after_ms?; content gains revealed, editable, modified_at, overlay, mine_sha256?; an entry gains modified_at and overlay. The write response gains source: "overlay" and a boolean backed_up.
10.2 The watch ping
While the Files tab is the visible inspector tab and document.visibilityState === "visible", the panel posts
/computer/files/watch { open: true } on mount and then every 15 seconds. When the tab is hidden, switched away from or
unmounted it posts { open: false } once (best effort, fetch with keepalive; the 45 second window is the guarantee). The
answer drives the source line before any file is read. A 429 stops the ping for a minute. The ping carries no content and no path.
10.3 Reading while live
No timer re-reads a listing or a file (design non-requirement 6). The panel re-reads in these cases only: the member
navigates, opens or refreshes; a tool call of the running turn names the open directory or file (touched-files already
collects write_workspace, edit_workspace and read_workspace paths from the tool cards; the panel re-reads an open, unedited
file or listing when its entry changes); and the editor probe below.
10.4 Edit mode
Offered when the content response has editable: true, as a button Edit beside Copy. The text area is plain monospace text,
the whole file (a file over 256 KiB or truncated shows This file is too large to edit here and no button). State, a pure
function in web/lib/computer-files-edit.ts (vitest):
viewing -> editing(loadedSha256, loadedText, draft) -> saving -> saved -> viewing(new sha256)
| |
| +--409 precondition_failed--> conflict(currentSha256, theirText, draft)
+--changed on the computer--> editing + warning(piChanged)
- Save sends
PUTwithIf-Match: "<loadedSha256>"and{ content: draft }. While saving, the button is disabled and the status readsSaving. 200 withsource: "live":Saved on the computer, with the lineNot yet in the saved copy(backed_upis false); 200 withsource: "overlay":Saved. Your change will be applied when the computer starts, with no line about the saved copy (backed_upis true). The newsha256becomesloadedSha256and the file leaves edit mode.kept_aspresent: a noticeA change made at the same moment was kept as <name>. - New file: in a live listing a
New filebutton asks for a name (must not contain/), creates it withIf-None-Match: *and an empty draft, and opens the editor. 409already_exists:A file with that name exists. - Leaving (Back to folder, opening another file, navigating, the tab switching away) with a changed draft asks
Discard your changes?withKeep editingandDiscard. Reload with unsaved changes asks the browser's own prompt. - Errors keep the draft:
owner_warmingandowner_timeout(Could not reach the computer; your text is kept, Retry),outcome_unknown(Not sure the save happened. Reload the file to check, with a Reload button that reads the file and shows the conflict banner if it differs from the draft),rate_limited,computer_startingandcomputer_stopping(The computer is starting; your text is kept, tried again automatically afterretry_after_ms, three times, then a Retry button),owner_too_old(The computer is being updated; try again in a few minutes),overlay_full(Too many files were edited while the computer was off; start the computer or discard some),not_in_saved_copy(This folder is not kept in the saved copy),edit_unsupported(Files on this computer cannot be edited here),content_too_large,file_too_large,path_refused,precondition_required(a bug, shown as a generic error).
10.5 The conflict banner
An overlay conflict (the content response says overlay: "conflict") shows the same banner on open, with the text Your edit conflicts with a newer version of this file,
the saved version (theirs, the response) and the person's text (variant=mine), and the buttons Overwrite with mine (a PUT with If-Match of theirs), Use theirs (the discard route,
after the discard prompt) and Copy mine.
On 409 precondition_failed the panel reads the current file (GET /content, live) and shows a banner above the editor:
This file changed on the computer while you were editing. with the other version beside the member's draft (two read-only
panes on a wide viewport, stacked on a phone), current_sha256 as the new base, and three buttons: Overwrite with mine
(PUT with If-Match of current_sha256, an ordinary audited write), Use theirs (drops the draft after the discard
prompt), Copy mine. The draft is never lost without a click. There is no merge.
10.6 Pi changed this a moment ago
A non-blocking notice above the editor, with a Show their version button that opens the same comparison as the banner. It
appears when either:
- a tool card for
write_workspaceoredit_workspacewhose normalised path equals the open file appears or changes status, in the live turn or the last committed message, within the last 60 seconds of the browser's clock (ToolCardModelhas no timestamp; the panel stamps the time it first sees a card, andcollectTouchedFilesreturns alastCallIdper path so a second write to the same path is a change); or - the editor probe finds a change. While a file is in edit mode the panel calls
GET /stat?path=every 5 seconds (LIVE_FILE_EDIT_PROBE_SECONDS; paused while the tab is hidden) and comparessizeandmodified_atwith the values at load. The probe catches a shell command (git checkout,sed -i) that no tool card names. Each probe is a file request through the poll channel: about 12 a minute against a limit of 240.
Tool events carry the model's path as typed, up to 500 characters (conversation/src/turn/tool-event-fields.ts:12) and
argumentsTruncated when cut; a truncated or unnormalizable path is not matched (the probe is the safety net).
10.7 Masked files
A masked content response shows the existing This file looks like it holds a secret and is not shown and a Reveal button.
It opens a confirmation: Showing this file is recorded in the audit log, with your name and the time. with Reveal and
Cancel. Confirmed: POST /reveal, the text is held in component state only (never in storage, never in the URL), shown with a
Hide button, cleared on Hide, navigation, tab switch and unmount. A revealed file may be edited (the same If-Match of its
revealed sha256); every save is audited like any other.
10.8 Accessibility and phone width
Controls keep the 44 pixel minimum used by the existing buttons (min-h-11); the source line and the status messages are
aria-live="polite"; the conflict banner is role="alert"; focus moves to the banner when it appears and back to the Save
button after a choice; every state has text, not only colour. At 375 pixels the two-pane comparison stacks and the editor uses
the full width.
11. State machines
11.1 A file request item (server)
(none) --queue (transaction)--> pending --deliver (poll)--> delivered --result route--> done --Front Door reads--> (deleted)
| | |
+--abandon (Front Door)--> abandoned waiter_gone: result route deletes
+--expires_at--> (absent to every reader)
Events that do nothing: a result for abandoned, pending or absent (409 operation_closed); a second identical result for done (200);
a second different result (409). Only one owner id ever holds a request: the one named on the item at queue time.
11.2 One request inside the supervisor
received --(known id)--> ignored | re-post stored result
received --(stopping)--> answered owner_stopping
received --(busy / queue full / memory)--> answered owner_busy
received --> queued --(expires_in_ms lapsed)--> answered expired
queued --> running --> answered --> posted --> remembered 60 s
`-- helper timeout (6 s) --> answered helper_timeout
pause() holds queued requests back from running. stopAccepting() answers new ones owner_stopping and waits for running.
11.3 One member request (Front Door)
GET: authenticated -> limited -> source chosen -> [snapshot answer]
`-> queued -> waiting -> done -> answered live
|-> abandoned -> snapshot answer (stale)
`-> delivered, no result in 8 s -> snapshot answer (stale), waiter_gone
PUT: authenticated -> limited -> headers/body/path ok -> live? -> audit pending -> dirty -> queued -> waiting
-> done: ok -> 200 | conflict -> 409 | other error -> mapped
-> abandoned -> 504 owner_timeout (audit failed)
-> delivered + 7 s more -> 504 outcome_unknown (audit stays pending; the result route finishes it)
11.4 Audit outcome
pending -> ok | conflict | refused | failed; pending read after 120 seconds is unknown. A refused record is written
directly (path_refused) and never passes through pending.
11.5 Interactive window
interactiveUntilAt is set by a watch ping (+45 s), by queueing a request (+45 s) and cleared by open: false (= now). The poll
reads it: before it, the owner is told the interactive pace in states that can serve files; after it, the normal tier. It is removed with
the owner fields when the owner ends.
11.6 The panel
connecting (watch ping sent) -> live | saved-copy (off) | starting | warming -> (retry timer) -> live; live -> fallback on a stale answer
-> retry -> live. Editing exists only inside live; a transition out of live while editing keeps the draft and disables Save until live.
11.7 The overlay entry and the seal (revision 2)
(none) --member edit (swap)--> pending --member edit--> pending (revision + 1)
pending --snapshot hash becomes entry hash (a publish by the owner)--> applied --clear (route)--> (none)
pending --snapshot hash is neither base nor entry (a host that ignores the overlay)--> conflict
pending|conflict --discard (swap)--> (none)
conflict --member edit with If-Match of theirs (swap)--> pending (base = theirs)
pending --owner finds a third hash on the disk--> conflict (stored)
unsealed --owner seal (swap)--> sealed(owner, generation, snapshot_generation)
sealed --stopped report (swap)--> unsealed
sealed --owner no longer live (heartbeat lapsed, newer generation)--> stale (replaced by the next seal or member write)
An entry is cleared only from the sealed state and only when a publish is on record (L37). Member writes happen only from unsealed or stale.
12. Security
12.1 What is protected, and from whom
Members of an organization share one workplace; every enabled member may read and write (D8). What must hold: a non-member reaches nothing; the owner (root, holding the snapshot credentials) never opens a path a person or the model named; the trail of edits cannot be edited by the owner or by the shell; a request cannot be replayed to another computer or another owner; and the owner cannot be made to run out of memory or time by a member.
12.2 Confused deputy
| Deputy | Confusion tried | What stops it |
|---|---|---|
| The Front Door acting for a member | A member of organization B names organization A in the path | resolvePrincipal takes the tenant from the path and checks the membership row of that tenant (auth/principal.ts verifyOrgAccess); every partition key is built from principal.tenantId. Scenario "A member of another organization is refused". |
| The Front Door acting for the owner | An owner token used on the member routes, or a member token on the owner routes | The member routes call resolveUserPrincipal, which refuses worker credentials and any token that is not a Cognito user token; the owner routes accept only computer-owner audience tokens. Scenario "A supervisor token does not open the member file routes". |
| The supervisor (root) acting on the shell's behalf | A link the shell planted makes root read or write a file the shell could not | The supervisor never opens a path. The helper runs as uid 2000 through chatticus-shell with no capabilities (computer/chatticus-shell); whatever a link leads to, the operation reaches only what the shell reaches; the helper additionally refuses a final location outside the workspace. The supervisor's code does not call stat, realpath or open on any path in a request: a test greps the new modules for node:fs imports. |
| The helper acting on a path | notes is a link to .git/config or to .env |
The helper tests the resolved path for the .git segment (L23) and for the secret names (L18). |
| The result route | An owner posts a result for another computer's request, for a request it was not given, or twice with different content | Tenant partition, owner id, generation, status delivered and a compare-and-set (6.1, 7.4). |
| The audit | A member's edit credited to another member | user_id from the principal, never the body; the request item carries the same user id and the result route copies it. |
12.3 Symlinks and the time-of-check to time-of-use windows
- Path check to open (planted or swapped link). Closed by the privilege boundary, not by the check: the helper is uid 2000, so a
link swapped between the check and the open leads where the shell could already go (
owner-file-helper.tsheader). The containment test and the.gitand secret tests on the resolved path are a second line and the reason the behavior is testable on a machine with no second user. - Special files.
O_NONBLOCKandfstatregular-file check remove the hang of a FIFO or a device (L15); a file swapped for a FIFO after the check is opened withO_NONBLOCKand refused afterfstat. - Precondition to rename. The helper compares the hash, writes the temporary file, compares again, then renames. A concurrent
writer that writes the same inode in place (the shell
>, Pi's former in-placewriteFile) between the second comparison and the rename is detected by the post-check through the descriptor held since the first read, and the replaced bytes are kept beside the file (kept_as, L16). The residue is a concurrent writer that replaces the file by rename inside that same few-microsecond window: its file is a complete file and ours replaces it, which is what two people saving a moment apart without a lock produce. Node has no atomic exchange-if-unchanged, so this is stated, not closed. A change made after the post-check is indistinguishable from a change made a moment after the save. - Delivery to abandonment. A write the Front Door gave up on must not run later. The request is delivered only by a conditional update
pending -> delivered; the Front Door abandons only bypending -> abandoned; exactly one wins. A delivered write the Front Door stopped waiting for is reportedoutcome_unknown, never "not done". - Queue to stop. Closed by the token on the computer record and the transaction (L11); the stop sequence finishes the operation in flight (L17), so an accepted edit is in the final publish.
- Membership removed. The 30 second membership cache lets a removed member write for up to 30 seconds. Accepted and stated (L24).
- Temporary file names. Random 128 bit names and
O_EXCL | O_NOFOLLOW: the shell cannot pre-plant the name.
12.4 A compromised owner
An attacker who holds the supervisor token (a stolen token inside its hour, or code running in the container with the token) can:
delay, drop or fail any file request (availability, visible as fallbacks and owner_timeout); return invented file content to the member
(display only: text through React, images from a fixed list of types, SVG and HTML as source; nosniff); report wrong hashes
(the audit records the Front Door's computed sha256_after and flags a mismatch; a wrong current_sha256 only makes a conflict
confusing); read every request the poll gives it, which includes the content of a member's edit and, for a reveal, nothing more than the
owner could already read from the disk. It cannot: read or write the audit family or the request family in the table (the supervisor role has no
DynamoDB access; no owner route returns an audit record; the per-turn role does not list those partitions); obtain a request for another
organization, computer or owner (the poll queries one partition for one owner id and generation); forge a member's identity.
A compromised per-turn child (it holds turn credentials and may write T#computer#record, W25) can set interactive_until_at
and file_operation_id on the record: it can make the owner poll fast (cost) or delay an idle stop; it cannot queue a request, forge a
result or touch the audit. Scenarios "The owner cannot read the audit log" and "The owner cannot read or write the file operation records
directly" evaluate both policy files (owner-role-policy.json, supervisor-role-policy.json) against the key families with the existing IAM
evaluator (features-support/iam-policy-evaluator.ts).
12.5 Denial of service through file operations
| Vector | Bound |
|---|---|
| A member floods requests | 240 reads, 30 writes, 10 reveals, 12 watch pings per user per minute (429); at most 16 open requests per computer (429 too_busy); the counters are in the table, so the limit holds across Front Door instances |
| A request holds a Front Door invocation | at most 8 seconds (15 for a delivered write); the wait loop does a consistent read per 250 ms (about 32 consistent reads per request, about 0.00001 USD) |
| One request is huge | write body 256 KiB (413 before the owner is asked); read range 256 KiB; item size test; a listing at most 2000 entries; a read of a file over 8 MiB refused |
| The poll response grows | at most 4 requests, each at most 256 KiB, so about 1.4 MB base64 worst case in a Lambda response (limit 6 MB) |
| The owner runs out of memory | 2 helpers at a time, 8 queued, owner_busy at 90 percent memory, a helper reads at most 8 MiB, the supervisor reads at most 1 MiB of helper output, each helper dies at 6 seconds. ASSUMPTION: a Node helper costs 40 to 60 MiB; the container test must measure two helpers plus a publish at 512 MiB (risk R1) |
| A hostile directory | the scan stops at 20,000 names; special files are skipped; a deep tree costs one call per directory |
| A hanging helper (FIFO, slow disk) | O_NONBLOCK, fstat, the 6 second kill of the whole process group |
| An unbounded audit table | 30 writes a minute a user; 90 day TTL; records are about 600 bytes |
| Fast polling left on | the window is 45 seconds from the last ping; a forgotten tab pings every 15 seconds, so polling stays at 1 second while the computer lives (at most 50 minutes), about 0.7 cents an hour |
| Many sweeps and operations | the mutex of L17 bounds the wait to one helper timeout |
12.6 Secrets and data at rest
File content passes through the table for at most the length of a request and is deleted by the code that reads it (L10); the
audit holds hashes and sizes, never content; the browser holds a revealed secret in memory only. Logs carry a 12 hex digit hash of the
path, never the path or the content (the redaction rule of owner-main.ts:39-41 extended). Masked answers still carry the whole-file
sha256 (as Phase 1's do): a member can reveal anyway, so the hash gives no member a capability, but an offline guess of a weak
secret from its hash is possible; accepted (O4). Tokens never appear in file routes: the member's bearer is verified and dropped.
12.7 Findings, status
| Finding | Status |
|---|---|
| 1. Symlink escape in file tools | Holds. The new operations run in the same helper through the same launcher; the supervisor opens no path; a planted link scenario covers every new operation (@supervisor links, FIFO, .git). |
| 2. Invoke key reaches every owner | Holds. The new owner route takes only the supervisor token. |
| 4. Tenant-wide table reach | Holds. No role gains a statement; the two new key families are outside both. |
| Audit tampering (design 14) | Closed by the key families and the Front Door being the only writer. |
| 4, S3 reach (revision 2) | The per-turn role keeps no snapshot statement (OWNER_SUPERVISOR_PROTOCOL C4), so it cannot touch the overlay. The supervisor role gets an explicit deny of writes to the overlay prefix (12.8). The Front Door role gains write on that prefix only. |
12.8 The edit overlay (revision 2)
Who may write. The Front Door alone. Exact statements:
The Front Door role (CDK, infra/lib/control-plane-stack.ts, beside ReadComputerSnapshots; the bucket is the snapshot bucket):
{
"Sid": "WriteComputerEditOverlay",
"Effect": "Allow",
"Action": ["s3:PutObject", "s3:DeleteObject"],
"Resource": ["arn:aws:s3:::SNAPSHOT_BUCKET/tenants/*/computers/*/overlay/*"]
}
s3:GetObject on tenants/*/computers/*/* and s3:ListBucket on the bucket already exist. The statements are the file conversation/src/gateway/front-door-snapshot-policy.json, read as data by the CDK stack (frontDoorSnapshotPolicyStatements) and by the scenarios of features/front_door_snapshot_role_policy.feature.
Why the list statement has no s3:prefix condition. S3 answers a GetObject of a key that does not exist with 404 NoSuchKey only when the caller holds s3:ListBucket on the bucket; without it the answer is 403 AccessDenied. That permission is evaluated for the read with no s3:prefix key in the request, so a statement conditioned on s3:prefix never applies to it. The first deployment carried StringLike s3:prefix tenants/*/computers/*/, so reading overlay/index.json of a computer with no overlay yet answered 403, the Front Door reported overlay_unavailable (503), the owner seal route failed and every hydrate ended overlay_apply_failed with reason seal_failed. The least privilege that works is s3:ListBucket on the bucket ARN with no condition, on the Front Door role only: it lets that role see key names (never object content) of the bucket, and the same role can already read every object under tenants/*/computers/*/*. A prefix condition cannot narrow it, because a caller who omits the prefix is not constrained by it. The scenario The Front Door reads a missing object as not found fails when the list statement is conditioned or removed. IAM is not enforced by moto, so this is proved by the evaluator rule objectReadOutcome, not by the store tests. Conditional puts need only s3:PutObject. s3:DeleteObjectVersion is not granted, so a delete leaves a
delete marker and the earlier version (see "Secrets" below).
The supervisor role (conversation/src/gateway/supervisor-role-policy.json, read as data by infra/lib/owner-role-policy.ts and evaluated by the scenarios): the existing
SnapshotObjects allow stays (GetObject covers reading the overlay; PutObject and AbortMultipartUpload are what the pack upload needs), and a second statement is added:
{
"Sid": "OverlayIsWrittenByTheFrontDoorOnly",
"Effect": "Deny",
"Action": ["s3:PutObject", "s3:DeleteObject", "s3:DeleteObjectVersion", "s3:PutObjectTagging", "s3:AbortMultipartUpload"],
"Resource": ["{{snapshotBucketArn}}/tenants/${aws:PrincipalTag/tenant}/computers/${aws:PrincipalTag/computer}/overlay/*"]
}
An explicit deny wins over the allow whatever else is attached. The supervisor therefore reads the overlay and can neither write nor delete in it; it never holds a delete right at all. The
"delete only applied entries" of the brief is done by the Front Door at overlay-applied, which can check what IAM cannot: that the entry still matches by path and revision, that the seal
is valid, and that a publish is on record (L37). features-support/iam-policy-evaluator.ts (type Effect: "Allow" only today) learns Deny with the rule that a matching deny overrides any allow;
infra/lib/owner-role-policy.ts passes Effect through. The per-turn role has no snapshot statement; the scenario proves it against overlay/index.json and a blob key.
What a compromised supervisor (token and credentials) can do. Read the overlay of its own computer (it holds the files anyway). Claim cleared for entries: refused unless the seal is its
own and a publish is on record; the worst case is clearing an entry whose publish it faked, which needs the disk it already controls. Report false conflicts: an entry the person sees as a
conflict, and nothing is lost (the person's text stays readable). Refuse to apply: the person's edit waits (availability). It cannot write, delete or replace an overlay object, cannot seal
for another computer (the route authenticates the owner and the key is built from the token's tenant and computer), and cannot read another computer's overlay (the role's prefix is tag-bound).
Confused deputy. The Front Door writes the overlay for a member into the prefix built from principal.tenantId and the computer record, never from the request; blob keys are
hashes; no member-chosen string is an S3 key. The owner applies entries through the file helper as uid 2000 with the base-hash precondition, refuseGitSegments and the containment check, so an
entry for /workspace/leak where leak is a link out of the workspace is a conflict, not a write (scenario "The overlay is applied through the file helper and cannot reach outside the workspace").
Root never opens a workspace path.
Time of check to time of use. The seal swap is the linearization point (R1). Between the owner's blob read and the helper write, the disk cannot change under it: no turn runs and
no file operation is delivered before the supervisor leaves hydrating. A blob is verified against its hash before it is written.
Denial of service. Caps (L41); 30 writes a minute a user (the write counter); at most 3 swap attempts per request; an index of at most 256 KiB read per request; an apply bounded to 30 seconds with the failure meaning "the owner does not start" rather than "the agent runs without the edits" (a person who keeps an overlay unreadable can only hurt their own organization's computer, and the control plane's start lease and probes behave as for any hydrate failure).
Secrets. Overlay content is stored as plaintext objects under the bucket's managed encryption (S3_MANAGED, enforceSSL), deleted when cleared. Masking by name applies to overlay content; a reveal is audited (source overlay);
the audit never holds content.
Why a lifecycle rule on non-current versions (L63), in plain words. The snapshot bucket has versioning on (infra/lib/snapshot-stack.ts). Versioning means S3 keeps every object that is overwritten or deleted, as a hidden older
version, and keeps it until someone deletes it explicitly. Today every publish overwrites the single pack snapshot.tar.gz (and manifest.json), so each older version is a complete copy of the whole workspace as it was at that
publish; it is not a per-file history and the product never reads it. Nothing expires them, so they only accumulate: a storage cost that grows with every publish, and an administrator's recovery window (the way back from a bad publish).
The overlay adds small deleted blobs to the same pile, and an edited .env that was cleared would stay recoverable there. Neither a product feature nor a promise. The rule, in part B: expire non-current versions 30 days after they stop being current
(noncurrentVersionExpiration: Duration.days(30) on the bucket). Effects: storage stops growing without bound; an administrator can still restore a version for 30 days; after that a deleted overlay secret is gone. It does not touch the current pack, the
current manifest or live overlay objects.
Customer-account computers (L46). Exactly why they are read-only: they run in the customer's own AWS account through the older host-worker path, with the customer-owned bucket chatticus-snapshots-{OrganizationId}. The Front Door's overlay write grant
(WriteComputerEditOverlay) names only the Anthus snapshot bucket, so it cannot write there, and the host that would apply an overlay is a host worker with no supervisor: nothing seals, applies or clears. Making them editable needs three things the later-phase ticket
carries: a cross-account write grant for the Front Door through the existing customer role, the same overlay protocol in the host worker, and the lifecycle rule in the customer's stack.
The other writer to the prefix. LocalWorkerRole (snapshot-stack.ts, grantReadWrite on the whole bucket) can write any object, overlay objects included. By the owner's decision (O7) it is left as it is: it is the role the operator's local hosts assume,
it already has read and write on the whole bucket, and it is operator-trusted, outside the owner roles this section is about. It is noted here so that nobody reads the explicit deny on the supervisor role as a bucket-wide guarantee.
Audit. Every overlay edit and discard has a record before any object is written (L44); every owner-reported clear and conflict has an overlay_apply record. No owner role reads the audit.
13. Traceability
Scenario names are those of the registered computer_file_*.feature files and web_computer_files.feature: computer_file_live_member and computer_file_live_operations (server routes), computer_file_live_owner (the owner routes), computer_file_overlay and computer_file_overlay_authority (the overlay, store level and who may write it), computer_file_live_supervisor and computer_file_overlay_supervisor (the supervisor), web_computer_files (the panel) and computer_file_live_end_to_end (the whole path); the draft file that held them has been deleted. Tags: @server (A1, A2), @supervisor (C), @web (W),
@end-to-end (E). vitest is the unit-test exception of AGENTS.md: pure helpers only.
| # | Requirement | Section | Scenarios | Tag |
|---|---|---|---|---|
| LF-1 | A read of a running computer reflects what is on its disk now, not the last publish | 7.2 | A live read reflects a change that was never published; A live listing shows a file the saved copy does not have | server |
| LF-2 | A computer that is off is answered from the snapshot and is not stale | 7.2, L4 | A computer that is off is answered from the saved copy and is not stale | server |
| LF-3 | An owner that is hydrating, too old, or stopping is answered from the snapshot with the reason, stale | L2, L3 | An owner that is still hydrating is answered from the saved copy and marked stale; An owner that predates file operations is answered from the saved copy | server |
| LF-4 | A live request that times out falls back to the snapshot with stale |
7.2 step 4.6 | A live request that times out falls back to the saved copy and says stale; A request the owner took but did not answer in time falls back to the saved copy | server |
| LF-5 | A slow owner is not waited for; the panel is told when to ask again | L27 | An owner that is slow to poll is not waited for | server |
| LF-6 | A live not-found is an answer, never replaced by the snapshot | L28 | A file deleted on the computer is not found even though the saved copy has it | server |
| LF-7 | Reads are ranged, hashed as a whole, capped | L15 | A binary file is returned in pieces and every piece names the hash of the whole file; A file over the read limit is refused with its size; A listing is cut at 2000 entries | server |
| LF-8 | A member's edit reaches the disk and marks it dirty | 7.3, L14 | A member edits a file and the owner has it | server |
| LF-9 | A write with a stale hash is refused 409 and nothing changes | 7.3 | A write with a stale hash is refused and nothing changes; A concurrent edit by the bot is reported as a conflict with the bot's version | server |
| LF-10 | A write needs a precondition; a create only if absent | 6.2 | A write without a precondition is refused before it reaches the owner; A new file is created only if it does not exist | server |
| LF-11 | The size limits hold before the owner is asked and in the helper | L15, L20 | A write larger than 256 KiB is refused before it reaches the owner; A file larger than the edit limit cannot be replaced | server |
| LF-12 | A write into .git is refused, lexically and after links |
L23 | A write into a .git directory is refused; A link inside the workspace to a .git file is refused by the helper as well | server, supervisor |
| LF-13 | A live write needs a live owner able to serve it; otherwise it waits or goes to the overlay | L2, L5, L33 | A write to an owner that will not poll soon is refused with a retry time; An edit made while the owner is stopping waits for the owner; An edit made after a hydrating owner sealed the overlay waits for the owner | server |
| LF-14 | A write the Front Door gave up on never runs later; a delivered one is reported unknown and audited when it lands | L9 | A write the owner never took does not happen later; A write the owner took but did not answer is reported as unknown and audited when the answer comes; A write the owner answers within the extra seven seconds is a success | server |
| LF-15 | An idle stop is vetoed by a pending request, and a stop and a request cannot both win | L11 | An idle stop is vetoed while a file operation is pending; A stop and a file operation cannot both win; A request is not handed to an owner that is stopping; A member's read that loses the race with an idle stop is answered from the saved copy; A member's write that loses the race with an idle stop waits for the next owner | server |
| LF-16 | A stopping supervisor finishes the operation in flight and refuses the rest, so an edit is in the final publish | L17 | A stopping supervisor finishes the operation in flight and refuses the rest | supervisor |
| LF-17 | An edit lands in the audit log; refused edits too; no content | L21, L29 | An edit lands in the audit log; A refused write is audited as refused | server |
| LF-18 | The audit record exists before the owner is asked (fail closed) | L21 | The audit record is written before the owner is asked | server |
| LF-19 | Hashes the owner reports are checked against what the Front Door sent | L21 | An owner whose hashes disagree with what was sent is flagged in the audit record | server |
| LF-20 | Reading is not audited | L29 | Reading a file is not audited | server |
| LF-21 | The owner cannot read the audit family or the request family | 12.4 | The owner cannot read the audit log; The owner cannot read or write the file operation records directly | server |
| LF-22 | Secret-looking files are masked, also through a link and also if the owner forgot | L18, L25 | A masked secret file is listed but its content is not returned; A link with an ordinary name to a secret file is masked too; A secret the owner did not mask is masked by the Front Door | server |
| LF-23 | A reveal returns the content and is audited, from both sources, and fails closed | 7.5 | Revealing a masked file returns its content and is audited; A reveal is refused when the audit record cannot be written; A secret is revealed from the saved copy and audited as well | server |
| LF-24 | A revealed secret does not stay in the table | L10 | A revealed secret does not stay in the file operation record | server |
| LF-25 | Only an enabled member of the organization, with a user principal, may use the routes | L24 | A member of another organization is refused; A caller with no member token is refused; A supervisor token does not open the member file routes; A member of a suspended organization cannot edit | server |
| LF-26 | Paths outside the workspace are refused before the owner is asked | 5.1 | A path that is not under /workspace is refused before anything is asked of the owner | server |
| LF-27 | Per-user and per-computer limits | L22 | A member who asks too often is told to wait; Too many pending file operations are refused at once | server |
| LF-28 | The poll delivers a request once, and again only for a repeated poll | 7.4 | The poll delivers a file request once and again only when the poll is repeated | server |
| LF-29 | The result route accepts a result only from the owner that was given the request, once | 7.4 | A result from an owner that was not given the operation is refused; A result for an abandoned operation is refused; A result is recorded once; A result that does not parse is refused; A result for an abandoned read is refused; An answer the Front Door has read is not recorded again | server |
| LF-30 | An open panel makes the owner poll every second, and the pace returns after it closes | L6, L7 | An open panel makes the owner poll every second; The owner returns to its normal pace after the panel closes; A panel that says it is closed ends the fast pace at once; A busy owner polls every second while a panel is open | server |
| LF-31 | An open panel does not keep the computer awake and does not start it | L12 | A panel that is open does not keep the computer awake; A panel that is open when the computer is off learns it and starts nothing | server |
| LF-32 | The helper lists, with kinds, sizes and link targets | 8 | The helper lists a directory with kinds, sizes and link targets | supervisor |
| LF-33 | A planted link cannot make the owner read or overwrite what the shell cannot reach; a link inside the workspace is written through | L16 | A planted link to a file outside the workspace is not read; A planted link to a file outside the workspace is not overwritten; A link inside the workspace is written through and stays a link | supervisor |
| LF-34 | A special file or a hanging helper cannot wedge the supervisor | L15 | A pipe in the workspace does not hang a read; A helper that hangs is killed at its timeout | supervisor |
| LF-35 | A request that cannot start in time is answered expired without touching the disk | 9.1 | A request that waited too long in the supervisor is answered expired without touching the disk | supervisor |
| LF-36 | A file operation is not killed by the sweep | L17 | A file operation in flight is not killed by the sweep after a turn | supervisor |
| LF-37 | An edit counts as activity and schedules the quiet publish | L12, L13 | An edit schedules a quiet publish and counts as activity | supervisor |
| LF-38 | The supervisor logs no path and no content | 9.3 | The file operation line carries no path and no content | supervisor |
| LF-39 | A superseded owner stops when it posts a result | 9.1 | An owner that is told it is no longer the owner stops when it posts a result | supervisor |
| LF-40 | The panel says whether it is live, the saved copy, or a fallback, and retries | 10.1 | The panel says whether it shows the live computer or a saved copy; A fallback to the saved copy says so and tries again | web |
| LF-41 | A member edits, saves, and is asked before discarding | 10.4 | A member edits and saves a file; Leaving an edited file asks first; A file that is too large to edit offers no editor | web |
| LF-42 | A conflict shows the other version and never loses the draft | 10.5 | A conflict shows a banner with the other version | web |
| LF-43 | The panel warns when Pi changed the open file | 10.6 | The panel warns when Pi changed the open file a moment ago; The panel warns when the open file changed on the computer without a tool call | web |
| LF-44 | A reveal is confirmed first and the secret does not linger | 10.7 | A masked file asks before it is revealed | web |
| LF-45 | The panel signals it is open and stops when hidden | 10.2 | The panel tells the computer it is open and stops when it is hidden | web |
| LF-46 | The whole path works with a real supervisor, helper and a bot turn | 1 | A member edits a file while no turn runs and the bot reads it back; Both sides edit the same file at once and exactly one edit stands; An edit is in the saved copy after the quiet publish | end-to-end |
| LF-47 | A member edits and creates files while the computer is off; the next read shows the edit | L30, L32, 7.7.2, 7.7.3 | An edit made while the computer is off is visible in the next read; A new file made while the computer is off is listed and created only if it does not exist; A computer that never published can still be edited; The saved copy is answered with a modification time and an editable flag; An edit is refused while the computer is off when the audit record cannot be written | server |
| LF-48 | An edit with a stale hash is refused; two members cannot both win | L31, L32, R3 | A stale-hash edit made while the computer is off is refused; Two members editing while the computer is off cannot both win; Two members editing different files at the same instant both stand | server |
| LF-49 | Viewing and editing never start a computer | O2, L45 | Editing while the computer is off starts nothing | server |
| LF-50 | An edit made while off is audited with source overlay | L44 | An edit made while the computer is off is audited | server |
| LF-51 | The overlay accepts only what the next publish keeps | L40, L41 | A file in a directory the saved copy never keeps cannot be edited while the computer is off; A link in the saved copy cannot be edited while the computer is off; A file in a folder that does not exist cannot be created while the computer is off; A write into .git is refused while the computer is off; The overlay holds at most 100 edited files | server |
| LF-52 | A secret edited while off is masked on read and revealed with an audit record | L18, L43 | A secret edited while the computer is off is masked on read and revealed with an audit record | server |
| LF-53 | A member can discard a pending edit, audited | 7.7.3 | A member discards their own pending edit | server |
| LF-54 | The seal is the single linearization point between an edit and the owner | L33, R1, R2 | An edit made while the owner extracts the snapshot is in the set the owner seals; An edit made after the owner sealed the overlay waits for the owner; A seal and an edit that race leave exactly one of them in charge; An edit made through the route while the owner extracts the snapshot is in the set the owner seals | server |
| LF-55 | A stale seal is replaced; an owner of an older generation cannot seal or clear; a clear needs the caller's own seal | L38, 7.7.5 | A seal left by an owner that died is replaced by the next edit; An owner of an older generation cannot seal or clear; A clear that is not sealed by the caller is refused | server |
| LF-56 | Clearing removes exactly the applied entries, and only after a publish is on record | L36, L37 | Clearing removes only the entries that were applied; A clear that names an entry that changed since is ignored; A clear before any publish is recorded is refused | server |
| LF-57 | A change on both sides is a conflict that is shown, kept and never applied silently | L39, R7 | A snapshot that moved under an edit shows a conflict and keeps both versions; An edit made on both sides conflicts and is never applied silently; A conflict can be dropped explicitly; The owner finds a third version on the disk and does not overwrite it; A conflict is shown with both versions and is replaced only on purpose (the member routes); The reason the owner gives for a conflict is the one that is stored | server, supervisor |
| LF-58 | Computers that cannot apply an overlay cannot be edited | L46, L47 | A computer in a customer account cannot be edited while it is off; An owner of protocol 1 blocks edits while it runs | server |
| LF-59 | Only the Front Door writes the overlay; the owner roles cannot write or delete it | 12.8 | The owner roles cannot write or delete overlay objects; The Front Door writes the overlay with conditional puts | server |
| LF-60 | The owner applies the overlay after extraction and before any turn, in every hydrate, and the agent waits for nothing else | L34, L42 | The owner applies the overlay after extraction and before any turn; The owner applies the overlay when the local cache matched; The owner applies the overlay when no snapshot was ever published; A turn that arrives right after the hydrate runs while the entries wait to be cleared | supervisor |
| LF-61 | The owner asks to clear only after the publish that holds the edits; a lost clear is asked again | L36 | The owner asks to clear only after the publish that holds the edits is recorded; A clear that was lost is asked again and before the stopped report; An edit that equals what the snapshot already holds is cleared without a publish | supervisor |
| LF-62 | An overlay the owner cannot apply safely fails the hydrate or becomes a conflict, never a silent skip or an escape | L34, L35 | An overlay that cannot be read fails the hydrate; An overlay blob that does not hash to its entry fails the hydrate; The overlay is applied through the file helper and cannot reach outside the workspace | supervisor |
| LF-63 | The panel edits while the computer is off, marks waiting files, shows overlay conflicts and retries while the computer starts | 10 | A member edits a file while the computer is off; The panel marks a file that is waiting for the computer; A conflict made while the computer was off shows both versions; An edit made while the computer is starting is tried again | web |
| LF-64 | An edit made while off reaches the bot, empties the overlay after the publish, and survives a crash | 7.7.4, 7.7.5 | An edit made while the computer is off is applied on the next hydrate and the bot sees it; The overlay is empty after the publish that holds the edit; The overlay survives a crash of the owner before it publishes | end-to-end |
| LF-65 | Delete, rename and new folders work while the computer is off, with the effective-hash precondition; the next read shows them | L48 to L50, 7.8.3 | A file deleted while the computer is off is gone from the next read and from the listing; A file renamed while the computer is off keeps its content and is read from the saved pack; A folder created while the computer is off is listed and creating it again is harmless; A delete with a stale hash is refused; A rename without a precondition is refused | server |
| LF-66 | A rename never replaces an existing path; a folder cannot be moved into itself; the root is protected | L52 | A rename onto an existing path is refused; A folder cannot be renamed into its own subtree; The workspace root cannot be deleted or renamed | server |
| LF-67 | Any member deletes files and renames; only the owner role deletes a non-empty folder | L51 | A member deletes a file and an empty folder; Only the owner role deletes a folder that is not empty; A member renames a folder that is not empty | server |
| LF-68 | A pending folder rename or recursive delete is exclusive | L53 | A change under a folder that is waiting to be renamed is refused until it is applied | server |
| LF-69 | Pending changes collapse when they undo each other, and can be listed and discarded | L54, L64 | Deleting a file that was created while the computer was off leaves nothing to apply; Renaming a file back to its own path leaves nothing to apply; Pending changes of every kind are listed and any member can discard one | server |
| LF-70 | Containment, links, .git, excluded folders and caps hold for the new operations |
L59, L41 | A delete, rename or new folder under .git is refused; A link in the saved copy is deleted as a link and its target is untouched; A delete or rename under a folder the saved copy never keeps is refused while the computer is off; The overlay counts every kind of change toward its 100 | server |
| LF-71 | The live path does delete, rename and mkdir through the owner with audit and dirty flag | L60, L61 | A running computer deletes, renames and creates folders through the owner | server |
| LF-72 | The audit records delete, move and mkdir with the hashes seen | L61 | A delete, a rename and a new folder are audited | server |
| LF-73 | The owner applies the four operations in the fixed order and treats collisions as conflicts | L56, L57, L58 | The owner applies new folders, renames, edits and deletes in that order; The owner does not rename onto an existing path; The owner does not delete a file that changed since it was seen; The owner does not delete a folder that gained children; The owner treats a delete whose file is already gone as applied; The owner renames a whole folder in one step | supervisor |
| LF-74 | The helper's delete, rename and mkdir stay inside the workspace and never follow links | L59, L60 | The helper deletes a link and not its target; The helper refuses a rename that would leave the workspace; The helper removes a folder tree only when its tree hash is the one seen; The helper renames without replacing | supervisor |
| LF-75 | The panel offers delete, rename and new folder, marks and lists pending changes, and asks before a recursive delete | 10 | A member deletes a file while the computer is off; A member renames a file and sees it at once; A member creates a folder; The panel asks before deleting a folder that is not empty and offers it only to the owner role; The panel lists the changes waiting for the computer and discards one | web |
| LF-76 | Deleting and renaming while off reach the bot at the next start, and survive a crash | L56 | A file deleted and a file renamed while the computer is off are seen that way by the bot | end-to-end |
Unit tests (vitest), the exception of AGENTS.md: the overlay merge (overlay-merge: applied, pending and conflict states for the four operations, list ordering and cap, the tree hash and the symlink hash, path translation under a moved directory), the write algebra of 7.8.4 as a pure function of (entries, request) with a table-driven test of every row, the apply order of 7.8.5, the overlay index schema, the swap loop with a fake store that fails the condition, the seal and clear rules as pure functions of (index, computer record, request); live-file-rules (livePathProblem, normalizeLivePath,
livePathHasGitSegment, looksLikeSecretFile, looksBinary, trimToCharacterBoundary); the schemas of section 5.2 (including the item
size test of L20 and the unknown-field rules); ownerFilePanelPace; the expected-pickup arithmetic of L27; the helper's atomic write with an injected hook between the second
comparison and the rename (the post-check keeps the replaced bytes; the temporary file is removed on every failure; the create is exclusive); the runner's queue
and mutex with a fake launcher and clock; the audit and request codecs; the store contract test (conditional updates, transaction, expired items
ignored, waiter_gone); web/lib/computer-files-edit.ts and touched-files.ts. None of these duplicates a Gherkin scenario: each is a pure
function or a codec.
14. Open questions for the owner
The owner's answers close the earlier questions (section 3.0, O1 to O7). Two choices were made in revision 3 where the brief was silent; they are written down for confirmation, and nothing else is open.
- Q-P1. A rename never replaces an existing path (L52). The alternative is an explicit overwrite. Recommendation: keep it refused; the person deletes first. It removes a way to lose a file silently, and the overlay and the helper both rely on an exclusive target.
- Q-P2. A pending folder rename or recursive delete is exclusive (L53): other changes at or under it wait (409
path_busy) until the next owner applies it or the person discards it. The alternative is composing path translations across several renames. Recommendation: keep exclusive; it keeps the apply a single rename.
15. Work assignment
Everything is a pull request into develop with the Gherkin first (AGENTS.md). Steps for a draft scenario are written by whoever
owns its tag; a scenario enters features/cucumber-features.txt together with its steps. This contract, the draft feature and the design
document edits merge first. Do not read or write project/. Revision 2 adds infrastructure (part B), the overlay (parts A3 and C2) and splits the server and container work accordingly.
F0. Foundation (first, small; merged before the others integrate)
Files: host-protocol/src/live-file-rules.ts, live-files.ts (section 5), host-protocol/src/edit-overlay.ts (section 5.5); the exports and constants of
section 5.3 in host-protocol/src/supervisor.ts (including overlay_not_sealed and the other new error codes) and host-protocol/package.json (./live-files, ./live-file-rules, ./edit-overlay);
conversation/src/snapshot/files.ts (import the shared rules); tests conversation/test/live-file-rules.test.ts, live-files-protocol-schema.test.ts, edit-overlay-schema.test.ts, and the changes to
owner-supervisor-protocol-schema.test.ts. Acceptance: typecheck, npm test, npm run test:features green with no behavior change except the shared secret name. Does not change OWNER_PROTOCOL_VERSION.
B. Infrastructure and policy (independent; deploy before A2 or C2 reach development)
(Revision 3: the lifecycle rule is part of this part, not optional: noncurrentVersionExpiration of 30 days on the snapshot bucket in infra/lib/snapshot-stack.ts with a test in infra/test, explained in 12.8. The Front Door statement WriteComputerEditOverlay also covers the deletes the clear performs.)
Files: infra/lib/control-plane-stack.ts (statement WriteComputerEditOverlay, 12.8), conversation/src/gateway/supervisor-role-policy.json (the deny statement),
infra/lib/owner-role-policy.ts (pass Effect through), conversation/features-support/iam-policy-evaluator.ts (learn Deny), features/owner_role_policy.feature and
computer_owner_policy_coverage.feature (the overlay rows), infra/test/computer-owner-stack.test.ts (assertions: Front Door can write only overlay/*, supervisor role has the deny and no delete), and
infra/lib/snapshot-stack.ts (expire non-current versions after 30 days, L63; a CDK assertion in infra/test). Scenarios: LF-59. Acceptance: cd infra && npm test; cd conversation && npm run test:features. cdk deploy of the ThinTurn stack is by CI on develop.
A1. Server: store, domain, owner routes for file requests (needs F0)
Files: conversation/src/store/codecs/computer.ts, store/codecs/live-file-operation.ts (new), store/codecs/file-audit.ts (new), store/messaging-store.ts,
store/dynamo-messaging-store.ts (queue transaction, deliver, abandon, complete, count, delete, audit put and finalize, incrementRateCounter, approveOwnerStop
with the second guard, touchInteractiveWindow), features-support/in-memory-messaging-store.ts (the same), domain/computers.ts (recordHostSnapshotPublished and
recordHostHydrated as targeted updates, L19, first commit of the part), domain/live-files.ts (new: liveFileSourceOf, expected pickup, ownerFilePanelPace,
deliver, record result, finalize audit), domain/owner-supervisor.ts (poll delivery, cadence, stopVeto, approval token), http/owner-host-auth.ts
(OWNER_COMPUTER_ROUTES), http/owner-supervisor.ts (ownerFileResultHandler), http/app.ts (declare the route).
Tests: test/dynamo-messaging-store.contract.test.ts, test/owner-supervisor-domain.test.ts, test/route-audience.test.ts, test/owner-supervisor-client.test.ts if it lists routes.
Scenarios it makes pass (steps in features-support/steps/live-files-owner.steps.ts, a FakeLiveOwner in features-support/fakes/): the "poll", "result", "fast poll", "idle stop"
and "owner cannot read" scenarios (LF-15 server part, LF-21, LF-28 to LF-31). Acceptance: those scenarios and every existing owner scenario green.
The overlay (seal, clear, unseal in the stopped report) belongs to A3. A1 and A3 both add routes to OWNER_COMPUTER_ROUTES and http/app.ts; whichever merges second resolves a one-line conflict in http/owner-host-auth.ts and http/app.ts.
A3. Server: the overlay core and its owner routes (needs F0; independent of A1)
(Revision 3: the core covers all four operations. The write algebra of 7.8.4 as a pure function with a table-driven test, the merge of 7.8.3 including path translation under a moved directory, the tree hash and symlink hash of 7.8.2, the conflict derivation per operation, the clear shortcut of 7.8.5, and the GET .../files/overlay and discard-by-revision handlers.)
Files: conversation/src/snapshot/overlay-store.ts (new: the port with conditional get, put-if-absent and put-if-match; the S3 implementation with IfMatch and IfNoneMatch, the filesystem implementation with the same semantics for scenarios; first
task: prove S3 conditional writes against moto 5.2.3 and note the result, L31), snapshot/overlay-merge.ts (new: the pure merge of 7.7.2 over LoadedIndex), snapshot/store.ts (the interface extension),
domain/edit-overlay.ts (new: write with the swap loop, discard, seal, clear with the publish-on-record check, conflict marking, stale-seal rule, audit records), http/owner-host-auth.ts (OWNER_COMPUTER_ROUTES: overlay-seal,
overlay-applied), http/owner-supervisor.ts (two handlers), http/app.ts (declare), http/routes/computer-file-overlay.ts (new: POST /overlay/discard), domain/computers.ts and the stopped-report handler in http/host.ts (unseal). Tests:
test/overlay-merge.test.ts, test/overlay-store.contract.test.ts, test/edit-overlay-domain.test.ts. Steps: features-support/steps/edit-overlay.steps.ts (setup steps write through the domain function so A3 does not need A2). Scenarios: LF-53 to LF-56 and the
store-level halves of LF-57. Acceptance: those green; the overlay contract test passes for both stores.
A4. Server: delete, rename and new folder routes (needs F0, A2 for the shared handlers and A3; revision 3)
Files: conversation/src/http/routes/computer-file-operations.ts (new: DELETE /files, POST /files/move, POST /files/directories, GET /files/overlay), http/app.ts (declare), domain/live-files.ts (the request kinds delete, move, mkdir through the same queue, wait, audit and abandon code as write), domain/edit-overlay.ts use (the overlay path), the role check (Principal.role === "owner" for recursive; 403 owner_role_required), the 428/409/413/422 mapping of 6.3, rate class write, audit operations delete, move, mkdir (7.8.8). Steps: features-support/steps/computer-file-operations.steps.ts. Scenarios: LF-65 to LF-72. Acceptance: those green; every earlier scenario unchanged.
A2. Server: member routes (needs A1 and A3)
Files: conversation/src/http/routes/computer-files.ts (live or snapshot, PUT, reveal, watch), http/app.ts (declare), snapshot/files.ts (reveal option, provenance
fields, modified_at), domain/file-audit.ts (new), lambdas/front-door-composition.ts (wire the store methods and the clock/sleep), http/errors or the route file for the codes of 6.3.
Steps: features-support/steps/computer-file-live.steps.ts. Scenarios: LF-1 to LF-14, LF-17 to LF-20, LF-22 to LF-27. Existing scenarios that change: computer_snapshot_index.feature
(responses gain stale, fallback_reason, modified_at, source stays snapshot). Acceptance: those scenarios green; on development, curl shows If-Match reaches the handler and ETag returns.
Revision 2 adds the overlay integration: http/routes/computer-files.ts routes the write per 7.3 step 4 (customer, live, owner_too_old, stopping, starting, overlay), merges the overlay into every snapshot answer (7.2 step 3, L43), adds overlay, mine_sha256, variant,
source: "overlay"; snapshot/files.ts accepts the overlay for list, stat, content and reveal and answers an empty /workspace plus the overlay when nothing was published. Scenarios: LF-1 to LF-14, LF-17 to LF-20, LF-22 to LF-27, LF-47 to LF-52, LF-54, LF-57 (member side), LF-58.
A2 as built (features/computer_file_live_member.feature, steps in features-support/steps/computer-file-live.steps.ts, a FakeLiveDisk in features-support/fakes/). Where the code differs from the text above:
- The routes are
http/routes/computer-files.ts; the decisions are indomain/member-files.ts(routingContextOf,readThroughOwner,performMemberChange,refusalOfOwnerError),domain/file-audit.ts(memberAuditRecordOf,StoreOverlayAuditSink),domain/file-rate-limit.tsanddomain/member-file-refusal.ts. Part A4 adds routes that build aMemberFileChange(the audit intent, the owner request and the overlay request of one change) and callperformMemberChange; the live path, the audit, the wait, the abandon rule and the overlay path are not repeated. - The wait is
awaitLiveFileOutcomeindomain/live-files.ts, with the sleep behind aFileWaitPort(createApp({ fileWait })), so no scenario waits for real time. LIVE_FILE_ERROR_STATUS(section 6.3) lives inhost-protocol/src/live-files.ts;SnapshotFileErrorandEditOverlayRefusaltake their statuses from it.- An organization is a customer-account organization when its
awsAccountIdis set and is not the deployment account (createApp({ deploymentAwsAccountId })). - A
.gitpath is refused and audited by the Front Door on the live path (7.3 step 3); on the overlay path the write algebra refuses it and writes the same record. - A reveal writes its audit record just before it queues the request (after the pickup and busy checks, which are not audited), and with the source it is answered from:
live,overlayfor a pending edit,snapshototherwise. A live reveal that falls back is finishedokby the Front Door. statanswerstree_sha256: nullfor a directory in this part; the tree hash as anETagarrives with the delete and rename routes of part A4.- The production composition now passes
editOverlay(the S3 overlay store, the S3 snapshot store,StoreOverlayAuditSinkover the messaging store); before it the overlay owner routes answered 503. - The
ETagofvariant=mineis the hash of the person's text.
A4 as built (features/computer_file_live_operations.feature, steps in features-support/steps/computer-file-operations.steps.ts; FakeLiveDisk learned stat with a tree hash, delete, move and mkdir with the helper's precondition rules). Where the code differs from the text above:
- The routes are
http/routes/computer-file-operations.ts(DELETE /files,POST /files/move,POST /files/directories);GET /files/overlaystays with part A3. Each builds aMemberFileChangeand callsperformMemberChange. - The
ETagofstatis the effective hash: the tree hash of a folder (also inentry.tree_sha256), the link hash of a link (also inentry.sha256), the content hash of a file. A livestatasks the owner withhash: true: the owner hashes a regular file of at most 8 MiB, so a running computer gives a file the sameETagonstatas oncontent; a larger file answers 413file_too_largeonstat(the panel words it as for an over-limit file). A folder over 5000 descendants answers 413too_many_entriesonstat. - A person's
If-Matchdoes not say whether it is a content hash, a link hash or a tree hash, and it does not need to: the live request carries it as the oneif_matchprecondition and the owner's helper compares it with the effective hash of what is really at the path. A delete or a rename on the live path is therefore one round trip to the owner. A path that is not there is the owner's 404not_found, and the attempt is audited asfailedwith that code like any other answer. Because the Front Door does not know the kind on that path, the audit record of a live delete withoutrecursive, and of a live rename, holds the hash seen insha256_beforewhatever it is the hash of; a recursive delete, which is always a folder, recordstree_sha256_before. recursive=trueneeds the organization owner role whether or not the folder has children: one rule on both paths, checked before anything is queued or written. The refusal is audited asrefused(7.8.8), not merely logged (7.8.1).- On the live path the Front Door cannot know that a
mkdirfinds the folder already there, so that idempotent call is audited (the overlay path does not audit it).
C. Container: helper and the supervisor's file runner (needs F0)
(Revision 3 adds to this part: the helper operations of 8.1 (mkdir, unlink, rmtree, rename, stat with hash), the runner's mapping of request kinds delete, move, mkdir, and the tree hash and symlink hash in host-protocol/src/live-file-rules.ts shared with the Front Door. Scenarios: LF-74.)
Files: conversation/src/pi/owner-file-helper.ts (section 8), conversation/src/pi/unprivileged-execution-env.ts (Pi write through the same operation),
conversation/src/pi/shell-environment.ts (new, extracted), computer/host/src/live-file-operations.ts (new), computer/host/src/live-file-helper-launcher.ts (new),
computer/host/src/supervisor.ts, supervisor-ports.ts, supervisor-main.ts, protocol-client.ts, unprivileged-process-sweeper.ts only if the mutex needs a hook, owner-main.ts (a
live-file-probe subcommand beside file-probe), host-protocol/src/supervisor.ts (OWNER_PROTOCOL_VERSION = 2, one line, in this part), computer/test_owner_container.sh
(run live-file-probe for list, read, write and the .git refusal as the shell user), conversation/test/computer-image-structure.test.ts (the supervisor bundle does not gain node:fs
use on request paths, Pi or the starter).
Tests: test/owner-file-helper.test.ts (extend), test/live-file-operations.test.ts. Steps: features-support/steps/live-file-supervisor.steps.ts, a fake launcher and fake plane in
features-support/fakes/. Scenarios: LF-12 (helper), LF-16, LF-32 to LF-39. Acceptance: those green; computer_owner_file_isolation.feature and computer_owner_supervisor.feature green; the
image builds; test_owner_container.sh passes (it needs Docker; an owner of the image runs it, Kanbus chatticus-6c66c5 tracks the local image build).
C2. Container: the overlay at hydrate (needs F0 and C; the control plane side is faked)
(Revision 3: the applier implements the order and the outcome table of 7.8.5 for all four operations, with the detection of "already applied" through the helper's stat (tree hash) and read (hash). Scenarios: LF-73.)
Files: computer/host/src/overlay-applier.ts (new, 9.4), host-disk-lifecycle.ts, supervisor.ts (applied set, early publish check, capture-and-clear, retry), supervisor-ports.ts, protocol-client.ts (sealOverlay, postOverlayApplied), supervisor-main.ts
(wire the applier and the launcher), computer/test_owner_container.sh (an overlay entry applied by a real container), conversation/test/computer-image-structure.test.ts (the bundle still imports no Pi). Tests: test/overlay-applier.test.ts with a fake launcher, plane and S3.
Steps: features-support/steps/overlay-supervisor.steps.ts, fakes in features-support/fakes/. Scenarios: LF-57 (supervisor), LF-60 to LF-62. Acceptance: those green; computer_owner_supervisor.feature unchanged and green; the image builds.
W. Web (needs F0 for the shapes; develops against fixtures; integrates after A2)
Files: section 10. New web/lib/computer-files-edit.ts; changes to computer-files.ts, computer-files-api.ts, touched-files.ts, ComputerFilesPanel.tsx, ComputerFilesView.tsx, the
fixtures; vitest beside each; the @web steps in the web harness (features-support/web-harness-runner.ts, steps/web-*.steps.ts), which has no file-panel steps today and must learn a fake running
computer. Scenarios: LF-40 to LF-45. Acceptance: those green, npm test in web/ green, a manual pass at 375 pixels width, and docs/BRAND_GUIDELINES.md followed.
Revision 3 adds: delete, rename and new folder controls (any member; the recursive delete of a non-empty folder only for the owner role, behind a confirmation that names the folder and the count), pending-change marks and a Pending changes list with discard, tree-hash ETag of a folder from stat, wording for path_busy, directory_not_empty, owner_role_required, invalid_move, too_many_entries. Scenarios: LF-75.
Revision 2 adds: edit while the computer is off (source: "overlay"), the Edited, waiting for the computer mark, the overlay conflict banner, the retry on computer_starting and computer_stopping, the new error wording, backed_up messages (section 10). Scenarios: LF-40 to LF-45 and LF-63.
E. End to end and live proof (needs A2, A4 and C2)
The @end-to-end scenarios (LF-46, LF-64, LF-76) in one steps file, the live proof (section 17), and the milestone sub-agent for Kanbus and README.
E as built (features/computer_file_live_end_to_end.feature, steps in features-support/steps/live-files-end-to-end.steps.ts with features-support/live-files-end-to-end-support.ts; LF-59 is features/computer_file_overlay_authority.feature). Where the code differs from the text above:
- The scenarios run the production routes with the real
OwnerSupervisorloop (the long-lived harness ofowner-long-lived.steps.ts, now parameterised by a wiring), the realHostDiskLifecycleover the scenario's bucket, the realSealingOverlayApplierand the real file helper run in the test process on a real directory. A bot's turn works in the same directory, so one disk is seen by the member, the helper and the bot. TheFakeLiveDiskis not used here: it answers owner requests in memory and the bot's tools need a real workspace. Time, the child process of a turn, the model, the credential broker and the wait of the Front Door (ScenarioFileWait) are faked. - Time moves only by waking the supervisor's sleepers; work the supervisor does in real time (restore, publish, helper) is counted as in flight and finishes before time moves on. Each supervisor the scenario starts gets a fresh live disk, as each task of the computer does.
- "A task that crashes" is a supervisor that is never woken again. The next owner can be started only after the heartbeat has lapsed (30 seconds) and the start lease of the dead task has run out (
HOST_START_LEASE_SECONDS, 60), so the crash scenario lets 61 seconds go by, not 31. - LF-64 (empty after the publish): the scenario also lets the supervisor idle until it stops (reason
idle) and then asserts the overlay is empty and not sealed, which is the state 17 step 6a promises "after the stop". - LF-59 second scenario: the first write of the index is create-only (
If-None-Match: *); theIf-Matchof the entity tag that was read applies to every later write, so the scenario makes two edits and checks the chain of tags against moto's conditional writes. - The scenario that lists the workspace for the bot runs
lswithcwd/workspaceinstead ofls /workspace, because the test shell has no/workspacedirectory.
Order and dependencies
- This contract (with the feature draft and the design document edits) merges.
- F0 merges. B is independent and can merge at any time before the server parts deploy.
- After F0: A1, A3, C and W start in parallel. A2 needs A1 and A3. A4 needs A2 and A3. C2 needs C.
- E needs A2, A4 and C2 (and so B, A1, A3, C).
Deploy order (development; no compatibility path)
- B (the Front Door may write the overlay; the supervisor role is denied; the 30 day expiry of non-current versions) before anything writes an overlay object.
- A1, A3, A2 and A4: the routes exist; every running owner is protocol 1, so file requests are answered from the snapshot with
owner_too_old, edits go to the overlay when the computer is off, and are refused 409owner_too_oldwhile such an owner runs. - C and C2 (one image): the next owner started is protocol 2, seals, applies, serves file requests. An overlay entry written before it exists is applied by it.
- W when A2 is deployed.
- The live proof (section 17).
16. What changes in existing tests and files
conversation/test/owner-supervisor-protocol-schema.test.ts (poll response, veto and error enums, constants), conversation/test/owner-supervisor-domain.test.ts (approveOwnerStop
signature, poll with file requests), conversation/test/route-audience.test.ts (the new routes), conversation/test/dynamo-messaging-store.contract.test.ts,
features/computer_snapshot_index.feature and computer-snapshot-files.steps.ts (new response fields), features/owner_role_policy.feature and
computer_owner_policy_coverage.feature (the two key families are denied to both roles), features/computer_owner_file_isolation.feature (stays green; the Pi write is atomic),
conversation/test/owner-file-helper.test.ts, conversation/test/computer-image-structure.test.ts, computer/test_owner_container.sh,
web/lib/computer-files*.test.ts, web/lib/touched-files.test.ts, web/lib/computer-files-fixtures.ts. Revision 3 adds: conversation/test/overlay-merge.test.ts (four operations), owner-file-helper.test.ts (8.1), computer-image-structure.test.ts (no change in imports), infra/test (the lifecycle rule), web/lib/computer-files*.test.ts (new controls and pending list).
Revision 2 adds: features/owner_role_policy.feature and computer_owner_policy_coverage.feature (overlay rows, the deny), features-support/iam-policy-evaluator.ts (Deny), infra/test/computer-owner-stack.test.ts,
conversation/test/snapshot-files*.test.ts and features/computer_snapshot_index.feature (the overlay merged into snapshot answers, the empty-snapshot case), computer/host/src/host-disk-lifecycle.ts tests, computer/test_owner_container.sh.
The list is the author's reading of the code; the implementer finds the rest by running the gates.
17. Live proof (development)
- Send a message that makes a bot create two files. While the turn is done and the computer is idle, open the Files panel: the source line reads
Live from the computer; the new files are listed and openable beforesnapshot_publishedappears in the log. The log showsfile_operationlines withpath_hashand no paths. - Edit a file in the panel and save:
Saved on the computer; ask the bot to read it back and it answers with the new text; the owner log showsfile_operation kind=write outcome=ok, thensnapshot_publishedabout 30 seconds later. - Edit the same file from both sides at once (ask the bot to rewrite it while the panel is in edit mode): one edit stands and the panel shows the conflict banner with the bot's version.
- In the table, find the audit records of step 2 and 3 (
{tenant}#computer#audit): user id, path, hashes, outcome, no content. Reveal a.envfile: arevealrecord exists. - Close the panel and watch the poll pace return from 1 second to the idle tier within a minute (
owner_pollinginterval_ms). Leave the computer for 15 minutes:owner_stopping reason=idle, with the last edit in the snapshot. - With the computer off, open the panel:
Computer is off. Showing the copy saved ..., with the editor. Edit a file and save:Saved. Your change will be applied when the computer starts. Inaws ecs list-tasksno task appeared; the computer record is unchanged;overlay/index.jsonand one blob exist under the computer's prefix; the audit has a record with sourceoverlay. 6a. Ask the bot to read that file. The log showsowner_started,workspace_hydrated,overlay_sealed,overlay_appliedbeforeowner_dispatch_received; the bot answers with the edited text. About 5 seconds after the supervisor is idle,snapshot_publishedandoverlay_cleared remaining=0;overlay/index.jsonis empty and unsealed after the stop. 6b. Edit a second file while the computer is off, start it, andaws ecs stop-taskit right afteroverlay_appliedand beforesnapshot_published: the panel still shows the edit (snapshot plus overlay), and the next owner applies it. 6c. Make the snapshot move under an edit (publish a different content for the same file from another path, for example by a bot turn on a copy) and see the conflict banner; resolve it withUse theirs. 6d. With the computer off: delete a file, rename a file, rename a folder that is not empty, create a folder, as a member; try to delete a non-empty folder as a member (403) and as the owner role (accepted, with the confirmation). Start the computer by sending a message: the log showsoverlay_appliedwith the operations in the order mkdir, move, put, delete; the bot lists the workspace and sees the deleted, renamed and created paths; after the publish the overlay is empty. 6e. Rename a file onto an existing name (409), delete a file after another path changed it (409), and see a conflict for a rename whose source the snapshot changed. - Check on the CloudFront path:
If-Matcharrives,ETagreturns, thePUTbody of 256 KiB passes. - Prove S3 conditional writes on the real bucket (the
If-MatchandIf-None-Matchof the overlay index) and that the supervisor credentials are denied aPutObjecttooverlay/(aws s3api put-objectwith them). Measure on a running 512 MiB owner: memory with two helpers reading 8 MiB files during a quiet publish (risk R1).
18. Assumptions and risks
Assumptions (marked where they appear): CloudFront forwards If-Match and returns ETag unchanged on /api* (6.2); a Node helper costs 40 to 60 MiB (12.5);
SHA-256 of 8 MiB on 0.25 vCPU takes well under a second; the 342 KB worst-case item and result body fit the Lambda and table limits (L20); fsync on the task's ephemeral disk is
cheap; the membership cache bound of 30 seconds; DynamoDB TransactWriteItems on the computer record and a request item is available with the same client the store already uses
(dynamo-messaging-store.ts imports TransactWriteItemsCommand).
| # | Risk | Mitigation |
|---|---|---|
| R1 | Memory at 512 MiB: supervisor, one Pi turn, two helpers and a publish in one container | concurrency 2, owner_busy at 90 percent, 8 MiB cap, helper timeout; measured in the live proof before C14 is touched; the next lever is concurrency 1 |
| R2 | Poll cost of the interactive pace with a forgotten tab | 45 second window, no panel keep-alive of the computer, 15 minute idle stop |
| R3 | A lost edit if the owner dies before the next publish | the dirty flag settles the record; the UI says not yet in the saved copy; the quiet publish runs 30 seconds after a write at the next gap (accepted, O4) |
| R4 | The record race (L19) was an existing hazard for dispatch fields | fixed first in A1; contract test |
| R5 | Protocol skew between a new server and an old owner | the version gate; fallback owner_too_old; at most 50 minutes |
| R6 | Temporary file litter after a killed helper | recognisable name, listed, harmless; revisit with a sweeper if seen |
| R7 | Pi write changes to atomic and loses hard links |
stated; the isolation scenarios cover links; no hard links are expected in a workspace |
| R8 | Web harness has no file-panel steps | W owns extending it; the pure logic stays in vitest so the risk is the harness work, not the logic |
| R9 | Two sources of truth for masking (helper and Front Door) | one predicate in live-file-rules.ts; the Front Door applies it again (L25) |
| R10 | S3 conditional writes are the foundation of the overlay (L31) and moto 5.2.3 may not honor both | A3 proves it first; the filesystem store has the same semantics for scenarios; the real bucket is checked in the live proof; the fallback is a DynamoDB index with the blobs still in S3 |
| R11 | A false conflict after two lost clears in a row | the retry every 10 seconds and before the stopped report; the conflict is surfaced with both versions, never overwritten |
| R12 | Deleted secrets recoverable from non-current versions | the 30 day expiry (L63) bounds it |
| R13 | The apply fails the hydrate for a bad overlay, so the computer does not start | the budget is 30 seconds; a conflict is not a failure; only unreadable or tampered blobs fail it; the Front Door is the only writer and validates every blob it writes |
| R14 | A 503 window while an owner seals or stops | 2 to 5 seconds, automatic retry in the panel, text kept (accepted, O7) |
| R16 | Tree hashes cost on large directories and use sizes, not content | 5000-descendant cap; a delete confirms the folder the person saw by its listing; files inside cannot be edited while an exclusive rename or delete is pending (L53) |
| R17 | A pending exclusive change blocks further edits below it until applied | 409 path_busy with the reason; discard releases it; Q-P2 |
| R15 | Overlay edits are in the saved copy only as the next read sees it until a publish folds them in; a crash before the publish leaves them in the overlay | by design (L36); that is why the overlay survives a crash |
19. Cost
Prices (US East; ASSUMPTION: not re-verified in this session): S3 PUT 0.005 USD per 1000, GET 0.0004 USD per 1000, storage 0.023 USD per GB-month, DynamoDB on-demand write 0.625 USD per million
write units, Lambda ARM 0.0000133 USD per GB-second.
| Item | Cost |
|---|---|
| An edit made while the computer is off | one GET of the index, one PUT of the blob, one PUT of the index, two audit writes, about 100 ms of a 512 MB Lambda: about 0.000013 USD. A thousand edits: about 1.3 cents |
| A delete, a rename or a new folder made while the computer is off | one GET and one PUT of the index, two audit writes, a little Lambda time (a tree hash reads the cached snapshot index): about 0.000008 USD; no blob, no content bytes, so the 8 MiB content cap is not used by them |
| Applying them at the next start | up to 100 helper operations on the task, no extra request except the clear; a recursive delete or a folder rename is one operation however large the folder (at most 5000 descendants): no new cost line |
| Caps | 100 entries of any kind, 8 MiB of blob content, 256 KiB per file, 256 KiB of index, 5000 descendants per directory operation (L41, L62) |
| A read while the computer is off | the existing reads plus one conditional GET of the overlay index: 0.0000004 USD |
| Storage | at most 8 MiB per computer: 0.0002 USD a month at the cap, plus non-current versions for at most 30 days (L63) |
| An owner start | the seal (one GET, one PUT), up to 100 blob GETs, the clear (one GET, one PUT, deletes are free), audit writes: about 0.00001 USD typical, 0.0001 worst case |
| A computer to apply it | none beyond the start the person's next message causes anyway. Editing while off costs S3 requests only (O1); no Fargate time is spent for an edit and no task is started |
| The watch ping when off | two table reads per 15 seconds while the tab is visible: about 0.0000003 USD each |
| The live path | as section 12.5 and the cost table of the design: the 1 second pace costs about 0.7 cents an hour only while a running computer has a panel open (O3) |