--- title: "Self-hosted AI agents: what bring-your-own-key actually protects" slug: harness-self-hosted-ai-agents-byok-architecture date: 2026-09-17 excerpt: "A reference on running AI agents on your own API keys and your own machine: where the key actually sits, which of the four storage locations an attacker reaches first, and the three things local-first execution does not protect. Written from the inside of a harness that does this, with a declared methodology and a last-verified date." featured_image: "https://bbtxujdxvidaghmhxkqs.supabase.co/storage/v1/object/public/generated-images/blog-1789678982865-harness-self-hosted-ai-agents-byok-architecture.webp" featured_image_alt: "Abstract 3D render of a translucent navy and steel polygonal vault with gold filament lines flowing from a cloud of light particles toward the vault shell and stopping at its surface, on a soft off-white background." author: Kinan Hamwi author_url: https://www.linkedin.com/in/kinanhamwi/ canonical_url: https://cerevisor.com/blog/harness-self-hosted-ai-agents-byok-architecture updated_at: 2026-09-17T21:03:06.887867+00:00 --- # Self-hosted AI agents: what bring-your-own-key actually protects TLDR Running agents on your own API keys does not, by itself, keep those keys away from the model. The key has to sit somewhere, and there are only four somewheres: an environment variable, a plaintext config file, an operating-system keychain, or a remote vendor's account. This is a reference for picking one deliberately, written from inside a harness that made the choice, including the hole we shipped and then closed. On 10 September, OpenAI put its managed Codex harness into public beta as part of the Agents API. Two days earlier, a census of agent platforms published by Digital Applied had already described the shape of the problem it walked into. And on that same 10 September, Anthropic shipped a one-line fix in [Claude Code](/blog/harness-supervisory-engineer-org-chart-box) version 2.1.268 that is, to me, the most instructive release note of the month. The fix was for the tool’s own diagnostic output. Asking Claude Code to list or describe a connected tool server could print secrets that had been filled in from placeholders in the server’s configuration, and the same thing happened in its login errors. A second line in the same release fixed plugin and marketplace errors printing a token or password out of a repository address. The model was not the leak. The help text was. That is the whole subject of this page. Bring-your-own-key, usually shortened to BYOK, means the software you run calls a model provider using an API key you own rather than one the vendor owns. Local-first means the software keeps its files and its state on the machine in front of you rather than in someone else’s database. Both are real properties and both are worth having. Neither of them, on their own, decides who can read the key. I run a harness that is built on both of those properties, so I have had to answer the question concretely rather than in a positioning deck. What follows is the architecture, the numbers, the mistake, and the parts that stay uncomfortable. --- ## The two data contracts a coding agent signs, and why the second one is worse The census from Digital Applied looked at thirty-one combinations of agent feature and delivery surface across Anthropic, OpenAI, Google and AWS. Its core finding is that the terms attached to a model and the terms attached to the agent product wrapped around that model are two separate contracts. Zero data retention, usually written ZDR, is a contractual arrangement where a provider agrees not to store the text you send it after the request completes. A model endpoint can be under that arrangement while the files, sessions, connector configurations and memories the agent product keeps around it are not. The managed Codex harness that shipped on 10 September is the cleanest example. MarkTechPost’s write-up of the launch that day, and OpenAI’s own documentation quoted alongside it, both land on the same two constraints: the Agents API supports data residency only in the United States, and it does not support zero data retention in any configuration, including a self-hosted sandbox. > "OpenAI generates abuse monitoring logs for all API feature usage and retains them for up to 30 days." Security Boulevard / guptadeepak.com, September 2026 That retention floor survives a zero-data-retention agreement, because abuse monitoring is a separate obligation from content storage. It is not a scandal. It is a normal, defensible thing for a provider to do. It is just not what most engineering leaders think they bought when someone in the room said the words “zero retention”. Key Insight Owning the API key changes who gets billed and who can revoke access. It does not by itself change what the provider retains, where the request lands, or what the software around the model writes to disk. Those are three more decisions, and they are made in the harness, not in the contract. --- ## The four places an agent key can sit, ranked by how fast they leak Every agent tool holds credentials in one of four places. The ranking below is mine, from the reading and from building the thing, and it is the practical core of this page. **An environment variable.** The simplest option and the easiest to lose. Any child process the agent spawns inherits it, which means a single shell command that prints the environment sends the key straight back into the model’s context window, where it is now part of a transcript that may be stored, replayed or summarized. **A plaintext configuration file.** This is where most tool-server credentials live today, because the common configuration format for the Model Context Protocol, the open standard for connecting agents to external tools, takes an environment block per server and people paste real tokens into it. I wrote a separate [walkthrough of wiring a tool server into a real agent workflow](https://cerevisor.com/blog/harness-how-to-setup-mcp-server-agent-workflow), and the credential step is the one most integration guides skip. The file is readable by anything running as that user, and it gets committed to repositories more often than anyone would like to admit. **An operating-system keychain.** The platform key store: Keychain on macOS, the Data Protection API on Windows, libsecret or KWallet on Linux. The value on disk is ciphertext bound to the current operating-system user, so copying the file to another machine or another account yields nothing. **A remote vendor’s account.** Convenient, auditable by the vendor, and the option that gives up the property this whole page is about. The interesting failure is not choosing wrong between these. It is choosing the keychain and then handing the plaintext back to something that logs. --- ## How Cerevisor holds a secret: encrypted at rest, spliced in after the audit line Here is the concrete path a credential takes in the harness I build, so this is checkable rather than aspirational. A saved secret goes through the operating system’s own encryption and lands as ciphertext in a small file inside the application’s per-user data directory. When platform encryption is unavailable, which in practice means a headless Linux box with no keychain running, the code stores the value in plain form under a different scheme label, prints a one-time warning, and then silently upgrades that entry to the encrypted form the first time it reads it on a machine where encryption has become available. That upgrade path exists because the alternative, a secret that stays plaintext forever because the keychain happened to be locked at login, is a real and boring way to lose a token. An agent never receives the value. It receives a placeholder, and it can write that placeholder anywhere an argument goes: a shell command, a request header, the body of a file it is about to write. ``` # what the agent writes, and the only form it ever sees curl -H "Authorization: Bearer ${{vault:GITHUB_PAT}}" \ https://api.github.com/user/repos ``` The real value is spliced in by the application’s privileged process immediately before the tool runs, and after the call has already been written to the audit trail with the placeholder intact. So the audit record says which secret was used, by which agent, on which tool call, and never what it was. Three constraints make that more than a gesture: - **Resolution is scoped to the agent.** Only secrets explicitly assigned to that agent resolve. A reference to anything else is left on the page verbatim, so the failure is visible in the output rather than silently becoming an empty string that turns into a confusing authentication error twenty minutes later. - **Output is scrubbed on the way back.** Every result leaving the tool layer is passed through a redactor carrying the values of that agent’s assigned secrets, which covers the case where a command prints its own environment or an error message echoes the argument it choked on. Values shorter than four characters are deliberately not redacted, because blanking every occurrence of a two-digit number would mangle far more output than it would ever protect. - **The audit trail is local and value-free.** It writes one line of structured text per operation into a log directory in the user’s home folder, rotates at one megabyte, and its header carries a hard rule against logging a secret value or any metadata detailed enough to reconstruct one. If that sounds like a lot of machinery for one string, consider that the empirical work on this class of leak keeps landing in the same place. Credentials escape through standard output and logs far more often than through anything resembling a clever attack. --- ## The interface hole we shipped, and the reveal path that replaced it This is the named failure, because a reference page without one is marketing. The harness has a user interface process and a privileged process, the normal split for a desktop application. Early on, the interface could call a channel that returned a decrypted secret by its identifier, plus a second channel that listed every stored secret’s identifier. Individually each looked reasonable. Together they were a complete extraction primitive: enumerate the identifiers, request each value, and any JavaScript running in the interface, not only the intended screen, had the whole set in plaintext. The fix was not a permission check. It was removing the ability. Three channels that return decrypted values are no longer reachable from the interface at all. They still exist for the privileged process that legitimately needs them, and they are simply not exposed across the boundary. What replaced them is a reveal flow where the confirmation dialog is also the delivery surface. The interface can ask for a secret to be revealed. What comes back across the boundary is a status word: copied, shown, or cancelled. The value itself goes from the key store to the clipboard or to the body of a native dialog, entirely inside the privileged process, and is cleared from the clipboard after thirty seconds. A repeat reveal of the same entry is refused within three seconds, and any two confirmation dialogs are held at least one and a half seconds apart, which is invisible to a person and throttling to a script that is walking the list. > A compromised interface can trigger the flow. It cannot read the result. That distinction is the entire difference between a safe design and a re-skinned hole. The honest limit is written into the same file: this does not defend against social engineering. Something hostile can still put the prompt in front of a person and hope they click through. So the dialog text is plain language, states exactly what is about to happen, and defaults to cancel, which is the only real mitigation available for that particular problem. --- ## Why a model on the machine next to you is local for scheduling and remote for billing A detail I did not expect to matter, and now think is the single most useful thing on this page for anyone sizing a private setup. The harness asks two different questions about a model endpoint that speaks the common OpenAI-shaped request format, which is what Ollama, LM Studio and [vLLM](/blog/how-to-run-open-weight-llm-vllm-production) all expose. The first question is a billing question: is this call free. The answer is yes only for genuine loopback addresses, the ones that never leave the machine. The second question is a scheduling question: does this call compete for this machine’s processor and memory. That answer is yes for loopback and also for private network ranges and local hostnames, because a model being served from the box under the desk is still local inference from a capacity-planning point of view. The two answers are allowed to disagree, and a model served over the local network is exactly where they do. It is not billed as free, because the harness cannot know whether the box on the other end is metering somebody. The hosted side of that same question, what each provider charges and what it can actually do under a tool-heavy agent load, lives in a [separate maintained provider table](https://cerevisor.com/blog/harness-llm-pricing-provider-capability-matrix). It is scheduled as local compute, because it obviously is. How the local capacity budget is spent Kind of workLocal units drawnReal bottleneck Remote API call1Provider rate limit Delegated in-process agent runtime3Both Local model doing inference**12**This machine Offloaded remote worker0Neither, not metered here The local budget defaults to forty-eight units. The comment next to the twelve records why it is twelve rather than the thirty an earlier design called for: twelve reproduces the four-at-a-time concurrency the previous hard-coded batching gave a wave of agents pointed at a local model, and the governing rule for that release was that nothing gets slower. At thirty it would have been one at a time, which is a correct-looking number that would have made every private setup feel broken. That is the tax nobody quotes. Self-hosted inference is not free, it is prepaid, and the currency is the concurrency of everything else on the machine. The other half of that bill, the time [orchestration spends](/blog/harness-multi-agent-orchestration-overhead-benchmark) on its own scheduling with the models switched off, I measured separately in [a benchmark of multi-agent overhead](https://cerevisor.com/blog/harness-multi-agent-orchestration-overhead-benchmark). --- ## Three things local-first does not protect, stated plainly A reference page that only lists strengths is a brochure. **Prompt injection still reaches the tool call.** Prompt injection is when text an agent reads, a pull request comment, a web page, a tool server’s own description, contains instructions the agent then follows. Holding the key locally does not stop an injected instruction from asking the agent to use that key against a destination of the attacker’s choosing. The mitigation is not storage, it is egress control. In the harness, an agent that has read untrusted outside content is blocked from sending anything to a remote tool server the user has not marked trusted, and that block sits underneath the interactive approval rather than replacing it, so surfaces that bypass the interactive path still hit the wall. **Configuration files are a supply chain.** Tool servers arrive as configuration plus a command to run. A migration pass in the harness runs at every start, finds plaintext values in tool-server configuration that look like real credentials, moves them into operating-system encryption, and rewrites the file with a pointer instead. The detector matches known credential prefixes and then falls back to a length heuristic, erring toward treating an ambiguous value as a secret, because encrypting something that turned out not to be a secret costs nothing and missing a real one costs a rotation. **Local does not mean unobserved.** The harness emits internal snapshots of how many agents are running and waiting, roughly every two seconds while anything is actually running and never while idle. Those go to the application’s own windows so the operator can see them. They are not sent anywhere. That is the distinction worth insisting on: instrumentation that stays on the machine is a feature, and instrumentation that leaves it is a different product with a different promise. In Cerevisor every memory record is classified at write time and carries a consent scope that defaults to personal-only, and the [data inventory and consent page](https://cerevisor.com/docs/guides/memory-and-learning/data-inventory-and-consent) lists what that covers. The reason the consent model exists before there is anything to consent to is that retrofitting one later never goes well. 31 agent feature and delivery-surface combinations audited across four major vendors, where the model's retention terms and the surrounding product's retention terms differ (Digital Applied, September 2026) --- ## How this page was checked, and what to verify in any harness before trusting it Methodology, so this is reproducible rather than asserted. Every claim about Cerevisor’s own behaviour on this page was checked against the shipping source before publication: the secret storage wrapper, the placeholder substitution and redaction module, the vault resolver and its audit writer, the reveal flow and its three timing constants, the tool executor’s assignment gate and egress backstop, the tool-server configuration migration, the execution-class table with its two separate address heuristics, and the resource budget defaults. Constants are quoted directly from the files that define them. Nothing on this page is a figure obtained by multiplying two other figures together, with one exception that is explicitly a quotation of the source code’s own recorded reasoning about concurrency. External claims come from sources published between 8 and 17 September 2026 and were not extended past that window. Older incidents that shaped this design, including the credential-theft chain through pull request comments in April and the environment-redirection advisory in January, informed the architecture but are deliberately not cited as current evidence. **Last verified: 17 September 2026.** This page is maintained. When the storage path, the reveal timings or the execution classes change, the date moves and so do the numbers. Four questions to ask of any agent harness, including mine: - **Where does the credential sit when the application is closed** If the answer is a configuration file in a home directory, that is a plaintext secret with extra steps. Ask to see the file. - **Can the model ever hold the value** Ask whether the substitution happens before or after the request is assembled. Before means the secret is in the transcript. After means it is not. - **What does the tool output do on the way back** A harness that substitutes secrets but does not scrub results has covered the front door and left the diagnostics open. That is the exact shape of the Claude Code fix that shipped on 10 September. - **What leaves the machine, in a list** Not a policy paragraph. A list of destinations. If nobody can produce one, the answer is unknown, and unknown is the wrong answer for a tool holding production credentials. --- ## The architecture decision worth settling before the next agent rollout The comfortable story about self-hosted AI agents is that running things locally solves the trust problem. It does not. It relocates it, from a vendor’s [retention policy](/blog/harness-ai-agent-memory-retention-reviewer-recall) to a set of decisions inside the software on the desk, and those decisions are much more inspectable, which is the actual win. What surprised me most, building this, is how little of the work was cryptography and how much was plumbing discipline. The encryption was an afternoon. The rest was months of asking, at every boundary, whether a plaintext value had any business crossing it, and being wrong once in a way that had to be removed rather than patched. If one thing is worth doing this week, it is the fourth question above. Sit down with whoever owns the agent tooling and write the list of destinations that receive anything from a developer machine running agents. Model providers, tool servers, telemetry endpoints, update checks, crash reporters. Most teams find the list is longer than they expected and shorter than they feared, and either way they have then done the piece of work that every later conversation about [data sovereignty](/blog/local-open-weight-license-data-sovereignty-regulated) depends on. If Cerevisor is on the shortlist, the [provider connection guide](https://cerevisor.com/docs/getting-started/connecting-a-provider) is where the key-handling side of that list starts. #### Sources - [Zero Data Retention AI Providers: What It Actually Means](https://guptadeepak.com/zero-data-retention-ai-providers/) - Security Boulevard / guptadeepak.com, 2026-09-14 - [AI Agent Platform Data Retention Eligibility Census](https://www.digitalapplied.com/blog/ai-agent-platform-data-retention-eligibility-census) - Digital Applied, 2026-09-08 - [OpenAI introduces the Agents API with a managed Codex harness](https://www.marktechpost.com/2026/09/10/openai-agents-api-managed-codex-harness/) - MarkTechPost, 2026-09-10 - [Claude Code release notes, version 2.1.268](https://github.com/anthropics/claude-code/releases/tag/v2.1.268) - Anthropic, 2026-09-10