Param2: what we built and how

This week at the India AI Impact Summit in New Delhi, BharatGen launched Param2, our new large language model. Param2 is available as open weights on Hugging Face and AIKosh. This post covers what it is, why we designed it the way we did, how it performs and how to try it.

What Param2 is

Param2 is a mixture-of-experts language model with 17 billion parameters in total, of which about 2.4 billion are active for any given token. We built it from scratch: it is not a fine-tune of someone else's model. It covers all 22 scheduled Indian languages and English, along with code, mathematics and tool calling.

The version released on Hugging Face, Param2-17B-A2.4B-Thinking, is a reasoning model: it works through a problem step by step before giving its answer.

Why mixture‑of‑experts

A conventional "dense" model uses every parameter for every token it reads or writes, so a bigger model is a slower and costlier one. A mixture-of-experts model divides much of its network into smaller sub-networks called experts. For each token, a small routing network decides which experts should handle it, and the others stay idle.

Think of a large hospital. Each patient sees a general physician and a few relevant specialists, not every doctor in the building. The hospital has a wide range of expertise, but each visit uses only a small part of it.

Param2 works the same way. It has 64 experts, and for each token the router picks 6 of them. Alongside these, 2 shared experts are always active. According to the model card, the shared experts keep a stable representation across languages, which helps with code-switching (moving between, say, Hindi and English in one sentence) and lets what the model learns in one language carry over to others. The result is a model that holds 17 billion parameters of knowledge but spends compute on only about 2.4 billion of them per token, so it runs at the cost of a much smaller model.

There is one trade-off. All 17 billion parameters still have to sit in memory, so loading the full model in bfloat16 takes about 34 GB of GPU memory.

Built from scratch for India

Param2 was pretrained from the ground up, with data chosen for linguistic diversity and cultural grounding. Indian-language data was filtered with quality classifiers for 22 Indian languages.

It also has its own tokeniser. Before reading any text, a language model breaks it into tokens. Tokenisers built mainly for English tend to break Indian-language words into many small fragments. More tokens mean slower and costlier processing, and less text fits in the model's context. Param2's tokeniser has a 128,000-token vocabulary, built with byte pair encoding and trained on all of its target languages, so words in Indian languages split into far fewer tokens than they do in global models.

A reasoning model

When you ask Param2 a question, it first writes out a reasoning trace, marked with <think> tags, and then gives its final answer. The model repository includes a small parser that separates the reasoning, any tool calls and the final answer. Examples on the model card include a multi-step averages problem, a Python binary search with all comments in Telugu, and an answer in Hindi on why languages, food and festivals differ across Indian states. The current checkpoint handles up to 32,768 tokens of context.

Languages

Param2 supports English and all 22 scheduled languages of India: Assamese, Bengali, Bodo, Dogri, Gujarati, Hindi, Kannada, Kashmiri, Konkani, Maithili, Malayalam, Manipuri, Marathi, Nepali, Odia, Punjabi, Sanskrit, Santali, Sindhi, Tamil, Telugu and Urdu. Hindi, Marathi, Gujarati and Tamil are among its strongest. As the model card notes, quality varies from language to language depending on how much training data was available.

How it performs

The model card compares Param2 with four other open models. All scores are zero-shot, measured with the LM Evaluation Harness, with thinking switched off for every model. The table shows Param2's score and its rank among the five models.

Selected results from the Param2 model card on Hugging Face. ARC Challenge (Indic) and TriviaQA (Indic MCQ) are averaged across 10 Indian languages.

Indian languages are where Param2 does best. On MMLU Hi, HellaSwag Hi, Indic BoolQ and ARC Challenge (Indic), it scores clearly higher than the other four models. On TriviaQA (Indic MCQ) it leads only narrowly, and on the Sanskriti cultural-knowledge test it comes third. On English, maths and code it is competitive: it scores highest on ARC, TriviaQA, LogiQA and HumanEval, while some of the other models do better on MMLU, GSM8K and instruction following. Because these numbers were measured with thinking turned off, they do not capture what the reasoning trace adds.

How to try it

• Hugging Face: bharatgenai/Param2-17B-A2.4B-Thinking. The card has a ready-to-run example using Transformers 4.52.3. Load the model with trust_remote_code=True and plan for about 34 GB of GPU memory. For repeatable outputs, the card recommends do_sample=False.
• AIKosh: Param2 on AIKosh, the IndiaAI Mission's platform for datasets and models.
• Licence: Param2 is released under the BharatGen Research License. You may use and modify it for non-commercial research and academic work. Commercial use, publicly releasing or deploying the model or a fine-tuned version, and offering it through a hosted service or API all need written permission from BharatGen.

The release is an early post-training checkpoint. It is a solid base for research and fine-tuning, but it needs further fine-tuning and safety alignment before it goes into production, as the model card points out. For the rest of our text models, see BharatGen's text AI models.