Cohere takes Model Vault confidential: verifiable, hardware-backed AI inference enters public beta

• Who: Cohere — Confidential Computing on Cohere Model Vault, its dedicated, fully managed model inference platform.
• What: Public beta of hardware-backed confidential inference: data in use is protected inside CPU+GPU enclaves, and remote attestation makes the protections independently verifiable.
• When: Announced October 22, 2026 (Cohere blog post published 2026-10-22).
• Hardware: NVIDIA H100 GPUs on Microsoft Azure. Azure supplies the confidential-computing infrastructure (confidential VMs); NVIDIA Confidential Computing extends it to the GPUs, with CPU↔GPU traffic encrypted during inference.
• Integration: Customers manage encrypted vaults through Model Vault and use the same Cohere APIs; payloads are encrypted end-to-end to the attested environment (via a Cohere OHTTP proxy). Support for NVIDIA Blackwell, including B200, is planned toward general availability.
• Open stack: Built on Kata Containers and Confidential Containers — Cohere intends to open-source the confidential inference stack so customers can inspect the implementation.

Cohere announced today the public beta of Confidential Computing on Cohere Model Vault, bringing hardware-backed protection — and, arguably more important, a way for customers to check that the protection is really there — to its managed enterprise AI inference platform.

The bet is straightforward: enterprises should get the simplicity of managed AI without handing the infrastructure that runs it access to their sensitive plaintext data. That unlocks workloads that contracts alone never quite covered: fraud investigations in financial services, privileged document search and due diligence in legal work, sensitive public-sector information, patent and research material, proprietary engineering design documents, and search across proprietary source code.

The third state of data

Encryption traditionally protects data in two states: at rest and in transit. Confidential computing targets the third: data in use.

Inference runs within a hardware-backed confidential environment spanning the CPU and GPU. Hardware-enforced isolation protects workload memory from everything outside the confidential boundary, including privileged infrastructure software and operators. With NVIDIA Confidential Computing, GPU memory itself is protected and traffic between the CPU and GPU is encrypted while the model runs.

Verify, don’t trust

Protection without proof is just another promise, so remote attestation sits at the center of the launch. Before any sensitive data is sent, customer-side software verifies cryptographically signed evidence about the environment that will process it — its hardware, firmware, and software configuration — against an approved security policy. If the environment doesn’t satisfy the policy, the client refuses to send the request.

That changes the trust model from “trust us” to “verify it yourself.”

Model Vault also protects request contents on the way in. Rather than relying on connection-level encryption alone, application payloads are encrypted end-to-end to the attested confidential environment — per Cohere’s documentation, through a Cohere OHTTP proxy that verifies attestation and encrypts only to a key bound to a genuine, attested environment. Responses get the same treatment on the way back. Customers keep using the Model Vault API and familiar request formats; the protection lives in the client and infrastructure layers, so applications don’t need to redesign their payloads.

Azure, H100, and what’s next

The public beta is powered by NVIDIA H100 GPUs on Microsoft Azure. Azure provides the underlying confidential computing infrastructure — the Cohere docs describe hardware-based confidential VMs (AMD SEV-SNP on Azure, or Intel TDX) paired with confidential-computing-capable NVIDIA GPUs — and NVIDIA Confidential Computing extends that protection onto the GPUs. Cohere stitches the pieces together with managed model serving, an encrypted request path, remote attestation, and customer-side verification.

The stack is built on open-source technology — Kata Containers and Confidential Containers — and Cohere says it intends to open-source the confidential inference stack itself, so customers and security researchers can inspect the implementation, understand the policies being enforced, and reproduce the measurements used during verification. Attestation that requires trusting another black box, the company argues, isn’t attestation.

What’s next: Cohere is working toward support for NVIDIA Blackwell infrastructure, including B200 GPUs, as the product moves toward general availability.

The industry is converging on this

Cohere isn’t moving alone. NVIDIA announced Confidential Bare Metal on the same day, framing confidential computing as the infrastructure layer under its OpenShell reference architecture — with Cohere planning to build it into Model Vault and Google Cloud extending support to its A4X Max instances. “With NVIDIA Confidential Computing in Cohere Model Vault, we’re extending hardware-rooted security directly to the GPU, protecting customer data in memory and across interconnects during execution,” said Cohere CTO Phil Blunsom.

Microsoft’s Vikas Bhatia, head of product for Azure Confidential Computing, called the work a demonstration of how confidential computing “combine[s] the simplicity of managed AI with independently verifiable technical controls.” And NVIDIA’s John Fanelli, vice president of enterprise software, said Confidential Computing on NVIDIA GPUs “provides hardware-backed protections that help platforms like Cohere Model Vault bring confidential AI inference to enterprise workloads.”

One honest caveat, from Cohere’s own announcement: confidential computing doesn’t eliminate all risk, and it doesn’t replace application security, compliance, contractual controls, or data-residency requirements. It adds a hardware-enforced, verifiable protection boundary around data and computation while inference is happening — which, for the workloads above, is the boundary that matters. Customers can get started at vault.cohere.com.