Contemporary large language models show clear limitations in traditional medicine, particularly in understanding the philosophical frameworks and reasoning patterns of Ayurvedic practice.
AyurParam is our 2.9 billion parameter transformer model built to close this gap. It is fine-tuned from the Param-1-2.9B-Instruct foundation specifically for Ayurvedic knowledge. AyurParam works in English and Hindi, and it is trained to handle Sanskrit terminology and the interpretation of classical texts.
Under the hood: AyurParam's architecture
AyurParam inherits the core transformer architecture of Param-1-2.9B-Instruct and is adapted for traditional medicine through fine-tuning. The model uses a decoder-only configuration with grouped-query attention for efficient inference.
Technical specifications:
- Parameters: 2.9 billion, with domain-specific fine-tuning
- Architecture: Decoder-only transformer with 32 layers
- Attention: 16 heads per layer with 8 key-value heads for efficiency
- Context length: 2,048 tokens, supporting multi-turn conversations
- Embeddings: Rotary positional embeddings (RoPE) with theta=10,000
- Activation: SiLU activation throughout the network
- Precision: bf16-mixed for computational efficiency
- Vocabulary: 256,000 tokens plus six special tokens for structured inference
At 2.9 billion parameters, the model is small enough for practical deployment while still handling detailed questions on traditional medicine concepts.
Data preparation: drawing on classical and contemporary texts
AyurParam's training dataset draws from classical and contemporary Ayurvedic texts, curated to reflect the depth of Ayurvedic knowledge. The process combined careful source selection, expert validation and structured knowledge extraction.
Source material:
- Books processed: About 1,000, including classical manuscripts and modern Ayurvedic literature
- Content volume: About 150,000 pages (around 54.5 million words)
- Classical texts: 600 manuscripts from openly available archives
- Contemporary sources: 400 specialised texts from the internet covering Ayurvedic domains
Domain coverage:
- Foundational texts: Charak Samhita, Sushrut Samhita, Ashtang Hruday
- Specialised branches: Kaaychikitsa, Panchakarma, Shalakya Tantra, Dravyaguna, Rasa Shastra, constitutional medicine, Balrog, Rog Nidan, Swasthvrutta
Data processing pipeline: The dataset went through a systematic process to ensure accuracy and coherence:
- Source gathering: Texts were digitised with Sanskrit terminology preserved, along with transliterations and contextual explanations.
- Q&A generation: Page-level Q&A pairs focused on Ayurveda-related, context-grounded questions, with domain experts validating accuracy and clarity.
- Taxonomic organisation: Knowledge was structured around core principles such as Dosha, Dhatu, Mala, Srotas, Nidana and Chikitsa, reflecting authentic reasoning patterns.
- Final dataset construction: Three types of Q&A were created:
- General Q&A: Knowledge-based questions
- Thinking Q&A: Reasoning and application-oriented scenarios
- Objective Q&A: MCQs, fact-checking and structured assessments
The dataset is bilingual (English and Hindi) and includes both single-turn and multi-turn conversations, totalling about 4.8 million training samples. This lets AyurParam learn both explicit knowledge and the reasoning patterns behind it.
Training methodology and optimisation
AyurParam was trained with supervised fine-tuning, using prompt templates designed for traditional medicine queries. Training balanced efficient convergence with preserving nuanced clinical reasoning.
Training configuration:
- Base model: Param-1-2.9B-Instruct
- Framework: Hugging Face Transformers with TRL supervised fine-tuning
- Distributed training: Multi-node torchrun implementation
- Training samples: 4.8 million examples
- Validation set: 800,000 held-out examples for convergence monitoring
- Training epochs: 3
Optimisation parameters:
- Learning rate schedule: Linear decay with warmup
- Base learning rate: 5e-6, decaying to 0
- Batch configuration: Global batch size of 1,024 with micro-batching at 4
- Gradient accumulation: 32 steps for memory-efficient training
- Special tokens: Vocabulary extended with tokens for structured inference
The custom prompt templates follow Ayurvedic consultation patterns, from classical text interpretation to contemporary clinical questions.
How AyurParam measures up: BhashaBench‑Ayur
Standard medical AI benchmarks do not test traditional medicine knowledge. AyurParam is evaluated on BhashaBench-Ayur (BBA), a benchmark designed specifically for Ayurvedic knowledge. BBA is built from questions in real government examinations and institutional assessments across India. You can read more in our BhashaBench-Ayur write-up.
Assessment scope:
- Question volume: 14,963 validated questions from 50+ official Ayurvedic examinations
- Source diversity: UPSC Medical Officer positions, State PSC AYUSH examinations, AIAPGET entrance assessments
- Domain coverage: 15+ specialised Ayurvedic disciplines, from foundational principles to advanced clinical practice
- Language distribution: 9,348 English and 5,615 Hindi questions, with Sanskrit terminology throughout
- Difficulty stratification: Easy (7,944), Medium (6,314) and Hard (705) questions
Assessment methods:
- Multiple choice questions for factual knowledge
- Fill-in-the-blank formats for terminological accuracy
- Match-the-column exercises for conceptual relationships
- Assertion-reasoning questions for clinical logic
Domain-specific evaluation criteria: BBA assesses competencies that generic medical benchmarks leave out:
- Classical text interpretation across major Ayurvedic literature
- Dravyaguna knowledge, including herb identification and therapeutic properties
- Constitutional analysis and personalised treatment protocols
- Panchakarma procedure selection and contraindication awareness
- Integration of traditional principles with contemporary clinical safety standards
Performance analysis
AyurParam scores higher on BhashaBench-Ayur than every similar-sized model we tested, and it is also ahead of several much larger general-purpose models.
Overall performance:
- Combined BBA accuracy: 39.97%
- English subset: 41.12%
- Hindi subset: 38.04%
- Bilingual gap: 3.08 points, compared with gaps of 5.6–6.8 points for Llama-3.2-3B-Instruct, Qwen2.5-3B-Instruct and granite-3.1-2b
Comparison with similar-sized models:
- Llama-3.2-3B-Instruct: 33.20% overall (AyurParam is 6.77 points higher)
- Qwen2.5-3B-Instruct: 32.68% overall (7.29 points higher)
- granite-3.1-2b: 31.10% overall (8.87 points higher)
Comparison with larger models: Despite having far fewer parameters, AyurParam scores higher than these larger models:
- Gemma-2-27B-IT: 37.99% (AyurParam is 1.98 points higher with roughly a tenth of the parameters)
- Pangea-7B: 37.41% (2.56 points higher with less than half the parameters)
- Llama-3.1-8B-Instruct: 34.76% (5.21 points higher with about a third of the parameters)
Performance by question difficulty:
- Easy questions: 43.93%
- Medium questions: 35.95%
- Hard questions: 31.21%
AyurParam scores highest among all the models we compared at the easy and medium levels.
Performance by question type:
- Multiple choice: 40.12%, the main assessment format
- Assertion-reasoning: 44.44%
Why this matters
Traditional medicine serves a significant portion of India's population, yet most medical language models are built around Western biomedical frameworks and English-speaking users. AyurParam is built for traditional knowledge and for the practitioners who work with it.
From AYUSH telemedicine and Panchakarma centres to Ayurvedic education and rural wellness programmes, AyurParam is our first step towards tradition-native language models that meet real needs. It is also a way to keep classical Ayurvedic knowledge accessible as healthcare becomes increasingly digital.
Resources:
- Model access: AyurParam on Hugging Face
- Benchmark dataset: BhashaBench-Ayur
- Paper: AyurParam on arXiv
- Technical documentation: Available in the model repository
About BharatGen
BharatGen is an AI company building open-weight AI models for India, across text, speech and vision. It is run by BharatGen Technology Foundation, a Section 8 not-for-profit anchored at IIT Bombay, and is backed by the Department of Science and Technology (DST) and the IndiaAI Mission (MeitY), Government of India. Its models are developed with a consortium of top Indian academic institutions.
