Accelerated Ai Inference With Confidential Computing Fortanix Solution Brief
Accelerated Ai Inference With Confidential Computing Fortanix Solution Brief
Accelerated Ai Inference With Confidential Computing Fortanix Solution Brief
Security Accelerated
Intel® Xeon® Scalable Processors
Accelerated AI Inference
with Confidential Computing
By optimizing workloads for Intel® Accelerator Engines, Fortanix uses 4th Gen Intel®
Xeon® Scalable processors to help secure and accelerate AI inference in the cloud
For Confidential AI, both security and performance matter. Intel 4th Gen Intel®
Xeon® Scalable processors are designed to help secure and accelerate AI
inference. Intel® Accelerator Engines are purpose-built integrated accelerators on
Intel® Xeon® Scalable processors that deliver performance and power efficiency
advantages across many of today’s fastest-growing workloads.
Workloads on the Fortanix Runtime Encryption® (RTE) platform using both Intel
Software Guard Extensions (Intel® SGX) and Intel® Advanced Matrix Extensions
(Intel® AMX) show up to a 7.57x increase in performance running TensorFlow
Resnet50,1 and up to 5.26x improvement running Bert-Large.2
Accelerators like Intel AMX help AI inference workloads achieve outstanding
performance, even when combined with hardware-backed security like Intel SGX.
Confidential Computing Helps Secure Cloud-Based AI
Fortanix RTE uses Confidential Computing, powered by Intel® SGX, to enable
general purpose computation on encrypted data without exposing plaintext
application code or data to the operating system or any other running process.
Even if the infrastructure is compromised, or malicious insiders have root
passwords, the application remains cryptographically protected.
Confidential Computing allows for the extraction of insights or training of AI
models using sensitive data without exposing that data to other software,
collaborators, or your cloud provider. This enables business transformation using
data that was previously too sensitive or regulated to activate for analytics and
other purposes.
ResNet-50v1.5 Inference performance on 4th Gen Xeon Bert-Large Inference performance on 4th Gen Xeon Scalable
Scalable Processors (8480+) Processors (8480+)
9 6 5.61
Normalized Throughput (Inferences per sec)
7.96 5.26
8 7.57
5
7
6 4
(Higher the better)
5
3
4
3 2
2 1.00 0.97
1.00 0.97 1
1
0 0
Without Intel SGX With Intel SGX accelerated by Int el AMX wi th Intel SGX and Without Intel SGX With Intel SGX accelerated by Int el AMX wi th Intel SGX and
accelerated by Intel AMX accelerated by Int el AMX
Batch size 64 Batch size 64
Figure 1. ResNet50 Inference Workload Performance using Intel Xeon Scalable 4th Gen Processors with Intel AMX & Intel SGX. 1
Figure 2. Bert-Large Inference Workload Performance using Intel Xeon Scalable 4th Gen Processors with Intel AMX & Intel SGX. 2
Solution Brief | Accelerated AI Inference with Confidential Computing
Accelerating AI Capabilities with Intel AMX When using Fortanix RTE with both Intel SGX and Intel
AMX, performance of Bert-Large inference workload at
Intel AMX is a new built-in accelerator that improves the INT8 precision on 4th gen Intel Xeon Scalable processors
performance of deep-learning training and inference on the vs 3rd gen Intel Xeon Scalable processors is up to 4.61x
CPU and is ideal for workloads like natural-language higher.3
processing, recommendation systems and image
recognition. Intel advances AI capabilities with 4th Gen Get the Most Built-In Accelerators Available
Intel Xeon Scalable processors and Intel AMX, delivering
higher inference and training performance compared to 4th Gen Intel® Xeon® Scalable processors have the most
previous generation Intel Xeon Scalable processors.3 built-in accelerators of any CPU on the market to help
improve performance efficiency for emerging workloads,
especially those powered by AI.
In addition to performance improvements, 4th Gen Intel®
Xeon® Scalable processors have advanced security
technologies to help protect data in an ever-changing
landscape of threats, while unlocking new opportunities for
business insights. Even when enabling hardware-enabled
features designed for security, like Intel SGX, Intel
Accelerator Engines like Intel AMX help AI inference
workloads achieve outstanding performance.
Performance varies by use, configuration and other factors. Learn more on the Performance Index site.
Performance results are based on testing as of dates shown in configurations and may not reflect all publicly available updates. See backup for configuration details.
No product or component can be absolutely secure.
Your costs and results may vary.
Intel technologies may require enabled hardware, software or service activation.
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.
0423/KHAN/PDF 355328-001 US 2