"IBM has released a new generation of open-source language models, Granite 4. The series combines two neural network architectures and is designed to deliver better performance with less memory usage. At launch, the Granite 4 family consists of four models ranging in size from 3 to 32 billion parameters. According to IBM, they perform more efficiently than previous generations, thanks to a hybrid design that combines the Transformer architecture with Mamba, a new and hardware-efficient network structure."
"One of the smaller models, Granite-4.0-Micro, uses only the Transformer approach. This is known for its attention mechanism, which allows the model to select and prioritize the most important parts of a text. The three other models add elements of the Mamba architecture to this. Mamba offers similar capabilities, but utilizes a mathematical system known as a state space model, originally developed for space applications."
IBM released the Granite 4 family of open-source language models, offering four sizes from 3 to 32 billion parameters. The series uses a hybrid design combining Transformer attention with Mamba state-space networks to increase efficiency and lower memory usage. Granite-4.0-Micro uses only Transformer; three hybrid models incorporate Mamba 2 for reduced memory pressure on long prompts and improved hardware efficiency. The 32B Granite-4.0-H-Small employs mixture-of-experts to activate only portions of parameters, targeting automated customer support. The 7B and 3B hybrid models prioritize speed over maximum accuracy for lighter hardware and real-time applications.
Read at Techzine Global
Unable to calculate read time
Collection
[
|
...
]