For most of India, speaking is easier than typing. Speech-to-text has to work for the way people really talk.
Many people are more comfortable speaking their own language than typing in any language. Speech-to-text lets them use helplines, apps and public services by talking.
Hindi with English, Marathi with Hindi, Tamil with English. Generic speech-to-text often breaks on this. Shrutam is trained for it.
Calls, clinics, markets and classrooms all have background noise. Shrutam pairs a speech encoder with a language model, which helps it use context to get the words right.
Hindi, Marathi, Gujarati, Tamil, Telugu, Kannada, Bengali, Malayalam, Odia, Urdu, Punjabi and Assamese, from a single model, so one deployment can serve callers across the country.
Information only helps if it reaches people in a form they can take in. For many Indians, that means hearing it in their own language.
Spoken information reaches people for whom reading is hard, from station announcements and helplines to health and farming advice.
Sooktam is trained to pronounce Indian names, places and terms the way people here say them, not the way a voice built abroad would.
With the speaker's consent, a short clip is enough for Sooktam to speak in that voice, in other languages too, so a teacher or a helpline keeps one voice.
Sooktam 2 is small enough to speak almost instantly, for live apps, phone calls and kiosks, where people should not have to wait for an answer.
Speech-to-text built on a language model, with mixture-of-experts projectors. It transcribes in real time and handles speakers who switch languages mid-sentence.
Natural, human-sounding speech, with zero-shot voice cloning: give it a short clip and it speaks in that voice, even across languages.
Our first speech recognition (ASR) model, for Hindi.
Our first text-to-speech model: 150 million parameters, trained from scratch on Indian speech, with a separate voice model for each language.
Two real recordings with background noise, and what Shrutam 2 wrote down. Press play to hear the input.
Recorded with background noise on our Shrutam demo.
திட்டத்திற்கு விண்வெளி அனுபவம் பெறுவதற்காக இந்திய விண்வெளி வீரர் சுபான்து சுக்லா சர்வதேச விண்வெளி நிலையத்திற்கு அனுப்பி வைக்கப்பட்டார். கடந்த மாதம் இருவத்தைந்தாம் தேதி
In English: Indian astronaut Shubhanshu Shukla was sent to the International Space Station to gain space experience for the mission. On the twenty-fifth of last month
Real output, not edited. Recorded on our Shrutam 2 demo. Shrutam 2 on Hugging Face ↗
A noisy lecture recording in Hindi, turned straight into English.
So, you also need to keep in mind that there are different notations by which we can refer this projection matrix as the projection matrix A
Real output, not edited. Recorded on our Shrutam 2 demo. Shrutam 2 on Hugging Face ↗
Sooktam 2 reading the same railway announcement: "Hello! Your train will arrive on platform number two in ten minutes."
नमस्ते! आपकी ट्रेन दस मिनट में प्लेटफ़ॉर्म नंबर दो पर आएगी।
வணக்கம்! உங்கள் ரயில் பத்து நிமிடங்களில் இரண்டாவது நடைமேடைக்கு வரும்.
নমস্কার! আপনার ট্রেন দশ মিনিটের মধ্যে দুই নম্বর প্ল্যাটফর্মে আসবে।
नमस्कार! तुमची गाडी दहा मिनिटांत फलाट क्रमांक दोनवर येईल.
Generated with Sooktam 2, unedited. Get Sooktam 2 on Hugging Face ↗
The weights are open: download them from Hugging Face and run them on your own servers, or talk to us about bringing speech into your product.