Private inference,
actually proven.

A user's prompt cannot be recovered by the nodes that serve their request — and we can prove it, per run, to a skeptic.

Prior proven work (Shard) established that a model too big for any single card can be served across untrusted machines over the open internet, fast and correct, with a receipt anyone can check. It also established the counter-fact that motivates us: a node running your layer decrypts to compute, and from the activations it sees it can reconstruct a large fraction of your tokens. Sealing the wire does not touch this.

Our model is one move applied rigorously: treat privacy as a measured property with a reproducible number. Shard proved sharded inference can be correct. Nuro's job is to turn the remaining leak into a number, drive that number down by construction, and put it in a receipt.

Privacy is meaningless without an attacker. We define a first-class reconstruction adversary:

Given exactly the activations a given node processes for a request — its block's inputs and outputs — how much of the user's input can it recover?

The load-bearing metric is the fraction of input tokens reconstructed, with a distributional variant (top-k recovery) for partial leaks. Prior work's baseline is roughly 35–59%. That is the number we exist to drive down.

We measure two threat tiers: honest-but-curious (logs what it legitimately sees) and malicious (runs the best available inversion). Both are measured. We never assume good faith. A defense is only real once it lowers this attacker's score.

Layered defenses, each independently measured against the adversary:

Boundary pinning. Embedding and final layers carry the most recoverable signal. Keep those blocks on trusted or staked nodes; let untrusted volunteers hold only middle blocks. This changes the shape of what any single untrusted node can see — it does not, by itself, bound a middle node. Residual-stream embeddings persist at every depth.

Activation obfuscation (proprietary weights only). A per-request invertible transform — orthogonal rotation or signed hidden-dimension permutation — is applied by the trusted head and inverted downstream. The untrusted node processes scrambled activations. Measurement shows this is lossless-correct and drops recovery to chance. Measurement also shows it fails on open weights: a node holding public W and conjugated W′ = W·R recovers the secret basis in closed form (R = W⁻¹·W′). Recovery returns to ~100%. This lever is real privacy only when the serving node does not have the base weights.

Secret-sharing MPC (open-weight privacy — the core mechanism). Split every activation additively across two non-colluding nodes x = s₀ + s₁. A single share is uniformly random, so its leakage is zero by construction — information-theoretic, not an assumption an attacker can undo. Public-weight linear layers are free under sharing; only non-linearities interact. One share-holder recovers the prompt at chance even holding the public weights and replaying the basis-recovery attack. The honest cost: compute is replicated, output is correct within tolerance, and communication rounds are slow over a wide-area network.

Edge containment. Raw tokens never leave a trusted boundary. Embedding and detokenization stay on the trusted head, so only hidden states — never token ids — cross the wire.

Trusted routing is a per-request option: trade some decentralization for a stronger bound on demand.

Ordering: pin the boundaries, contain the edge, disclose honestly. Offer trusted routing. Use obfuscation only for proprietary-weight serving. For open weights, secret-sharing MPC is the mechanism that earns the claim.

The artifact that makes “proven” literal. Every run can emit a privacy receipt recording the layer-to-node assignment, each node's trust tier, the transform class on each edge, and — the load-bearing field — the measured reconstruction score, obtained by running the adversary against the activations each node saw.

A skeptic re-runs the adversary on the recorded activations and confirms the score sits under the claimed threshold. Correctness receipts prove the output was right. Privacy receipts prove the input stayed hidden. Same discipline, different property.

{
  "run_id": "...",
  "utc": "...",
  "model": "...",
  "nodes": [
    {
      "role": "head | stage | tail",
      "layer_range": [a, b],
      "trust_tier": "operator | staked | volunteer"
    }
  ],
  "defenses": {
    "boundary_pinning": true,
    "edge_containment": true,
    "activation_transform": "orthogonal-rotation@per-request"
  },
  "privacy_eval": {
    "adversary": "gradient-inversion + token-classifier",
    "worst_untrusted_recovery": 0.06,
    "threshold": 0.10,
    "pass": true
  }
}

The following are measured on controlled transformers and, for the open-weight case, on a real pretrained model (Qwen2.5-0.5B). They prove the mechanism. They are not a blanket guarantee for every production request.

“Private” earns its word phase by phase, never on day one.

We publish the reconstruction number even while it is bad, and show it falling. A receipt proves a specific run was private to a measured degree. It is not a blanket privacy guarantee, and we never present it as one. Over-claiming privacy is the one failure mode that would make the whole thesis worthless.

A stage node is a worker holding a contiguous block of layers. The orchestrator assembles the pipeline, fits blocks to each worker's memory, and — new for this thesis — enforces trust-tier placement so boundary layers land on trusted nodes and the receipt's assumptions actually hold.

The live network at nuroai.xyz is the product built on this model: contributed GPUs, an OpenAI-compatible API, and a public data board. Privacy is not bolted on later. It is the open problem this paper exists to close.

Contribute compute · Read the docs · Live data