Apple's AI Integration: Powerful NPUs and On-Device LLMs

Despite advancements, memory requirements and power consumption remain potential challenges.

Apple is expected to introduce AI functionality on its desktop and mobile platforms, with a focus on improving Siri's understanding and response capabilities.
Apple's Neural Processing Units (NPUs) are particularly powerful, with the A17 Pro found in the iPhone 15 Pro capable of pushing 35 TOPS, double that of its predecessor.
Apple has been quietly buying AI companies and developing its own large language models to ensure it can deliver generative AI capabilities.
Apple researchers have demonstrated the ability to run large language models (LLMs) on-device using flash memory, potentially overcoming memory limitations.
Despite the advancements, Apple's AI implementation may face challenges related to memory requirements and power consumption.