Granite 3.0 Large Language Model Announced at IBM’s TechXchange

IBM announced the latest iterations of its flagship Granite large language model (LLM) at its TechXchange event Oct. 21. 

The release of the open-source Granite 3.0 comes with multiple variants, but stars Granite 3.0 8B Instruct. Trained on more than 12 trillion tokens of data across 12 natural languages and 116 programming languages, the general-purpose 8B Instruct and its 2B counterpart were designed to be “workhorse” models for enterprise AI, according to the announcement.

“These compact, versatile models are designed to be fine-tuned with enterprise data and seamlessly integrated across diverse business environments or workflows,” the announcement said.

Granite Instruct 8B Capabilities

Natural-language use cases for Granite Instruct 8B include: 

• Text generation
• Classification
• Summarization
• Entity extraction for customer service chatbots 

Programming use cases for Instruct 8B include: 

• Code generation
• Code explanation 
• Code editing
• Agentic use cases requiring tool calling 

The technical blog trumpets Granite 3.0 8B Instruct’s ability to pull information from external sources through effective retrieval augmented generation (RAG) abilities, leading evaluations on RAGBench. The 8B Instruct model matches similarly sized models on academic benchmarks and surpasses them on benchmarks for enterprise tasks and safety, the technical blog for Granite 3.0 noted.

In the latest release, joining the general purpose Instruct 8B and 2B LLMs are the following specialized models:

• Granite Guardian: A family of guardrail models for risk and harm detection used for monitoring and managing LLM inputs and outputs
• Mixture-of-Experts (MoE): Lightweight models designed for high inference efficiency, ideal for on-device applications, CPU servers and uses requiring extremely low latency

IBM Partners on Granite 3.0

The Granite 3.0 release marks IBM’s continued focus on partnerships in its AI endeavors, Kate Woolley, IBM’s general manager of ecosystem, noted in her blog accompanying the release. In addition to availability on the watsonx platform, Granite 3.0 is available on platform partners including Google Vertex AI, Hugging Face, NVIDIA, Ollama and Replicate. 

Woolley wrote, “The work the IBM Ecosystem is doing with AI reflects our evolution to become a partner-first company, a move we made years ago in recognition that the digital transformation needs of modern businesses cannot be met by one single company.”

---