Computation offloading to hardware accelerators in Intel SGX and Gramine Library OS

The Intel Software Guard Extensions (SGX) technology enables applications to run in an isolated SGX enclave environment, with elevated confidentiality and integrity guarantees. Gramine Library OS facilitates execution of existing unmodified applications in SGX enclaves, requiring only an accompanying manifest file that describes the application's security posture and configuration. However, Intel SGX is a CPU-only technology, thus Gramine currently supports CPU-only workloads. To enable a broader class of applications that offload computations to hardware accelerators - GPU offload, NIC offload, FPGA offload, TPM communications - Gramine must be augmented with device-backed mmap support and generic ioctl support. In this paper, we describe the design and implementation of this newly added support, the corresponding changes to the manifest-file syntax and the requisite deep copy algorithm. We evaluate our implementation on Intel Media SDK workloads and discuss the encountered caveats and limitations. Finally, we outline a use case for the presented mmap/ioctl support beyond mere device communication, namely the mechanism to slice the application into the trusted enclave part (where the core application executes) and the untrusted shared-memory part (where insecure shared libraries execute).