Summary: On July 2, 2026, NVIDIA announced Confidential Computing, a hardware-rooted layer that keeps AI inference data encrypted while it is in use. NVIDIA reports up to 98% of baseline performance, but its cited benchmark covers HGX B300, Qwen 3.5 397B-A17B at FP8, Intel TDX, and SGLang v0.5.12, with measured throughput and TPOT costs between 1.0% and 8.1%. Teams must confirm Blackwell and CPU TEE prerequisites, benchmark their own workloads, and account for attestation and framework maturity before adoption.
The Announcement: NVIDIA Brings Confidential Computing to AI Inference
The moment an AI project graduates from experiments to real production work, protecting data turns into a frontline concern — above all for organizations in healthcare, finance, and government that handle sensitive records. Conventional encryption covers data at rest and data in transit, but inference breaks that cover: the data has to sit decrypted in memory while the model works on it, within reach of anyone who can read memory — a malicious insider, a compromised operating system, or the hypervisor underneath it all. NVIDIA’s Confidential Computing (CC) closes exactly this hole with a security layer rooted in the hardware itself, keeping data encrypted even while it is in use.
NVIDIA made its case in a blog post dated July 2, 2026, describing CC as engineered for the era of agentic AI and built to scale any model securely without significant performance trade-offs. Benchmarks cited in the post put the cost of switching CC on remarkably low — the hardware still delivers up to 98% of baseline performance — which is the headline argument for product builders who cannot afford to sacrifice speed for security.
Under the Hood: Hardware Root of Trust plus Attestation
The protection begins in the silicon of NVIDIA’s Blackwell GPUs — the RTX PRO 6000, the HGX B200, and the HGX B300 are all covered. On the HGX B200 and B300, confidential computing stretches across up to 8 GPUs, with NVLink encryption guarding the traffic between them. Every GPU carries a private signing key fused in during manufacturing; software and the host system never see it. That key anchors an attestation chain, one that confirms the integrity of the hardware and software environment before any secrets are handed over.
Verification itself runs through the NVIDIA Remote Attestation Service (NRAS). The service takes a signed evidence bundle — the GPU’s hardware report joined with CPU TEE measurements (AMD SEV-SNP or Intel TDX) — and checks it against a known-good reference integrity manifest (RIM). Only once the Confidential VM (CVM) passes as unmodified are secrets such as model decryption keys released. Because the handshake normally happens just once at startup, individual inference requests pick up no extra latency.
Trimming the Overhead: Optimizations and Benchmark Numbers
According to the blog, CC mode hits performance from two directions: the latency of submitting work securely, and reduced host-to-device bandwidth. NVIDIA worked with open-source frameworks such as SGLang and FlashInfer to build mitigations:
- CC-safe autotuner timing: In CC mode, FlashInfer reads the GPU global timer register rather than relying on event timers, so kernel candidates can be compared accurately.
- Async D2H copy worker: SGLang shifts the per-step token readback off the scheduler’s critical path, bringing back the compute/copy overlap that CC would otherwise break up.
- Piecewise CUDA graph support: SGLang gains CUDA graph replay for prefill and mixed batches, cutting the kernel launch overhead that CC mode amplifies.
The measurements came from an HGX B300 built on Blackwell Ultra, running the Qwen 3.5 397B-A17B model at FP8 precision. The test bed combined a VM with GPU passthrough, Intel TDX, an Ubuntu 25.10 host, and SGLang v0.5.12 with specific PRs. Sweeping concurrency from 4 to 256 and testing input/output sequence lengths of 1024/1024 and 8192/1024, the cost of enabling CC landed between -1.0% and -8.1% for throughput and TPOT, with most configurations losing less than 5%. One caveat from the blog: smaller units of work feel the secure launch overhead more, so packing larger amounts of work into each GPU launch keeps the impact down.
What Product Builders Should Do Next
For teams shipping products that lean on AI inference while customers insist on strong data privacy, Confidential Computing belongs on the evaluation list. A sensible sequence:
- Confirm the hardware and software prerequisites: Turning on CC takes specific NVIDIA Blackwell GPUs plus a compatible software stack, including a CPU TEE (AMD SEV-SNP or Intel TDX). Check that your cloud provider or data center can supply these components.
- Benchmark with your own workloads: The hit differs from workload to workload. Run your models with CC enabled and measure throughput, TTFT, and TPOT against your service-level agreements (SLAs); the blog’s benchmarks are a reference point, not a promise about your results.
- Weave it into your existing security: Sit down with your security team and map where CC fits inside your overall defense-in-depth strategy, alongside encryption, identity management, and compliance requirements.
Keep in mind that the post stays at the overview level and never reaches exhaustive technical specifications. For detailed deployment guidance, NVIDIA’s official documentation is the place to dig, and their solution architects are worth a conversation.
Where the Limits Are
None of this makes CC a free lunch; even with its performance overhead kept minimal, a few constraints deserve attention:
- Locked to specific hardware: CC exists only on certain Blackwell GPUs, so an infrastructure upgrade may be required.
- A maturing software ecosystem: The optimizations are arriving in frameworks such as SGLang and FlashInfer, but not every inference framework has been fully tuned for CC so far.
- Extra attestation overhead: Wiring up remote attestation with NRAS and keeping reference integrity manifests in order adds operational complexity.
- Results vary by configuration: The published numbers cover one tested setup (Qwen 3.5, SGLang, Intel TDX); different models, batch sizes, or sequence lengths can move the outcome.
The Bottom Line: Security as Part of the Product
The message behind the announcement is that AI security belongs in the design phase, not bolted on afterwards. With Confidential Computing rooted in hardware, NVIDIA now offers a genuinely attractive way to shield data during inference without a punishing performance bill. Whether it fits still depends on your particular use case, the hardware you can get, and the performance you must hit. A sound order of operations: pin down your customers’ security needs, work out the technical feasibility, and only then decide whether to adopt CC. Do it that way, and sensitive data stays protected while your product remains responsive and competitive.