Apple has released several open-source large language models (LLMs) named OpenELM. These models are designed to run on-device rather than through cloud servers, prioritizing speed and privacy. The models are available on the Hugging Face Hub, a community for sharing AI code. OpenELM uses a layer-wise scaling strategy to efficiently allocate parameters within each layer of the transformer model, leading to enhanced accuracy. For example, with a parameter budget of approximately one billion parameters, OpenELM exhibits a 2.36% improvement in accuracy compared to OLMo while requiring 2x fewer pre-training tokens. Apple’s release includes the complete framework for training and evaluation of the language model on publicly available datasets, including training logs, multiple checkpoints, and pre-training configurations. Apple’s broader AI strategy is expected to be revealed alongside previews of its major software updates at WWDC in June. Apple is releasing the OpenELM models to empower and enrich the open research community with state-of-the-art language models. Sharing open-source models gives researchers a way to investigate risks and data and model biases. Developers and companies are able to use the models as-is or make modifications. The open sharing of information has become an important tool for Apple to recruit top engineers, scientists, and experts because it provides opportunities for research papers that would not normally have been able to be published under Apple’s secretive policies. Apple has not yet brought these kinds of AI capabilities to its devices, but iOS 18 is expected to include a number of new AI features, and rumors suggest that Apple is planning to run its large language models on-device for privacy purposes.
Read more at: www.macrumors.com