2000 character limit reached
A Performance Evaluation of a Quantized Large Language Model on Various Smartphones (2312.12472v1)
Published 19 Dec 2023 in cs.LG, cs.AI, and cs.PF
Abstract: This paper explores the feasibility and performance of on-device LLM inference on various Apple iPhone models. Amidst the rapid evolution of generative AI, on-device LLMs offer solutions to privacy, security, and connectivity challenges inherent in cloud-based models. Leveraging existing literature on running multi-billion parameter LLMs on resource-limited devices, our study examines the thermal effects and interaction speeds of a high-performing LLM across different smartphone generations. We present real-world performance results, providing insights into on-device inference capabilities.
Collections
Sign up for free to add this paper to one or more collections.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
