Red Hat Acquires AI Model Optimization Company NeuralMagic, Bolstering Open Source AI Strategy
Key Points
- 1Red Hat is acquiring Neural Magic, a startup specializing in optimizing AI models, particularly large language models (LLMs), to run efficiently on standard CPUs and GPUs.
- 2Neural Magic's technology allows LLMs to achieve speeds similar to specialized AI chips, addressing the increasing demand for cost-effective deployment on existing hardware.
- 3This acquisition significantly bolsters Red Hat's open-source AI strategy, complementing its recently launched RHEL AI platform that provides open-source LLMs and tools for custom model fine-tuning.
Red Hat, an open-source software company, announced its acquisition of Neural Magic, a machine learning startup specializing in optimizing AI models for deployment on diverse hardware, including CPUs and GPUs. This strategic acquisition addresses the escalating computational demands of large language models (LLMs) and the imperative for enterprises to deploy LLMs cost-efficiently on their existing hardware infrastructure.
Neural Magic, founded in 2018, offers technology that optimizes LLMs to operate at speeds comparable to specialized AI chips, even when running on commercial CPUs and GPUs. This capability is crucial for achieving high-performance inference on general-purpose hardware, circumventing the need for expensive, dedicated AI accelerators for all LLM deployments. The company has been a significant contributor to vLLM, an open-source project focused on efficient model deployment. vLLM provides libraries designed for deploying cloud-optimized AI models across a wide array of processor and GPU architectures, highlighting Neural Magic's expertise in hardware-agnostic, high-throughput inference solutions.
The acquisition of Neural Magic is a pivotal move within Red Hat's broader open-source AI strategy. Red Hat recently introduced Red Hat Enterprise Linux AI (RHEL AI), an enterprise-grade platform for developing and deploying AI solutions. RHEL AI integrates IBM's Granite LLM family with Red Hat InstructLab, an open-source community-driven tool for model tuning. Positioned as a dedicated foundation model (FM) platform, RHEL AI enables enterprises to build customized AI models, with a particular emphasis on supporting small language models (SLMs). It provides users with free access to Granite LLM libraries, while the InstructLab tool facilitates fine-tuning these models using an organization's proprietary knowledge and data. Neural Magic's optimization technology is expected to enhance the performance and efficiency of LLM inference within the RHEL AI ecosystem, making custom AI model deployment more accessible and economically viable for businesses.