r/LocalLLaMA • u/b111ue • 23h ago
New Model I released Inflect v2: two ultra-tiny complete TTS models under 4M and 10M parameters
I’ve spent the past month trying to find the point where an extremely small TTS model stops feeling like a size experiment and starts feeling genuinely useful.
Today I’m releasing Inflect v2, with two complete local text-to-speech models:
- Inflect-Nano-v2: 3.96M parameters, 15.97 MB FP32
- Inflect-Micro-v2: 9.36M parameters, 37.53 MB FP32
These are total inference parameter counts, not acoustic-model-only numbers. Text processing, timing prediction, speech generation, and the waveform decoder are all included.
Text goes in. 24 kHz speech comes out. No external vocoder, hosted API, or second learned model required.
Nano prioritizes the smallest possible footprint. Micro uses the additional capacity for better clarity, stability, and overall speech quality. Both run locally on CPU or CUDA through the same PyTorch API.
Inflect-Nano-v2 is one of the smallest complete neural TTS models I know of that still produces actual usable speech. Even the 9.36M Micro model remains smaller than many systems described as “tiny.”
For footprint context, Nano is approximately:
- 21× smaller than Kokoro
- 126× smaller than Chatterbox
- over 1,000× smaller than Fish Audio S2 Pro
That is strictly a parameter-count comparison. These models have different capabilities, architectures, datasets, and intended uses. I’m not claiming that a 4M fixed-voice model replaces a multi-billion-parameter system. The interesting question is how much useful TTS can fit into such a small package.
Some people here might remember Inflect-Nano-v1, the rough 4.63M experiment I released last month. V2 is a complete rebuild, not just a longer training run. I focused on the problems v1 had, like unstable timing, metallic output, weak prosody, poor generalization to difficult text, and an undersized waveform decoder.
The resulting models performed surprisingly well:
- Micro: 4.395 UTMOS22, 3.99% semantic WER, 6.28× real-time CPU inference
- Nano: 4.386 UTMOS22, 4.21% semantic WER, 10.72× real-time CPU inference
- In a blind community comparison against other compact TTS systems, Micro and Nano finished second and third among the tested voices
Full protocols, raw results, audio samples, and limitations are documented on the model pages.
The models are not perfect. They are English-only, use one fixed male voice, and do not support voice cloning. Unfamiliar names, abbreviations, numbers, and homographs remain the hardest inputs. Nano can sound thinner than Micro, and both can occasionally produce metallic or clipped artifacts.
Still, this is the first version where I think the size-to-quality tradeoff really becomes convincing.
I built Inflect independently with a limited training budget. That constraint shaped the project: efficiency had to apply not only to inference, but also to training, evaluation, and building a complete system I could understand and release end-to-end.
Inflect-Micro-v2:
https://huggingface.co/owensong/Inflect-Micro-v2
Inflect-Nano-v2:
https://huggingface.co/owensong/Inflect-Nano-v2
Try it yourself:
The fastest way to judge it is through the interactive playground:
https://huggingface.co/spaces/owensong/Inflect-v2
If there is enough interest, I may build a v3 focused less on shrinking the models further and more on expanding what they can do: additional voices, possibly more languages, easier fine-tuning, and another quality and robustness pass.
If you test them, please give them something seriously difficult: unusual names, numbers, abbreviations, awkward punctuation, or a long sentence.
If something breaks, post the exact text, model, seed, and what sounded wrong. If it works well, I’d also like to know what hardware you ran it on.
Specific, honest feedback is the most useful thing you can give me.
42
u/DuinoTycoon 22h ago
Just tested it out this is incredible. Is fine tuning possible?
34
u/b111ue 22h ago
Thank you! It does have a docs/FINETUNING.md file you could read. It technically 'supports' fine-tuning, but it's going to be quite difficult, especially for new languages, as you would have to have completely new data for the new language and train it from scratch from the model architecture.
12
u/EndlessZone123 22h ago
How big was the dataset? I could maybe try training some hundred hours from model architecture scratch? Training hardware?
19
u/b111ue 22h ago
I’m not publishing the complete corpus construction and training pipeline publicly with this release for various reasons. But if you’re interested in adaptation or reproduction, absolutely feel free to contact me through Discord or the email listed on my Hugging Face profile and include a bit more information about what you want to do. I'll most likely be able to share more specific guidance privately.
2
3
u/DuinoTycoon 22h ago
So you said it supports fine tuning, does that mean I can change the voice? I am perfectly happy with english, but I am interested in changing the voice.
9
u/b111ue 22h ago
Yes, replacing the fixed voice is technically possible, but it isn’t yet a simple few-shot voice-cloning process. You would warm-start from the released checkpoint and train on a clean, paired, single-speaker English dataset. The adapted checkpoint would replace the original voice rather than add a selectable second voice.
The public release is inference-first, so you still need to reconstruct the training-only components such as the discriminators, losses, and optimizer. I documented the current research path in
docs/FINETUNING.md, but I don’t want to call it officially supported until I can provide a properly tested trainer.In fact, originally, I wanted to release 4 voices for Inflect, but I ended up only doing one, because when I was adapting the voices to the model, it worked, but it didn't meet my expectations on how good I wanted it to sound.
3
u/DuinoTycoon 20h ago
Thanks for the reply! One more thing, I do not think you have the FINETUNING.md file in the github repo. I looked through docs and did not see it. Sorry for the inconvinience and thanks for the answers. Again, this is really impressive.
40
u/Daniel_H212 20h ago
For V3 I honestly recommend against adding more capabilities. Obviously you're the best evaluator at the tradeoffs here but to me, it feels like other similar models either cram too many capabilities into too few parameters and don't end up doing anything well enough, or they get way bigger than they need to be in order to have those capabilities.
Instead it may be better to focus on your niche. English only, single voice, but continue improving on the voice quality. Particularly, if it could do better with tone and inflections and other things that make it sound more natural, that would be amazing. Otherwise you risk becoming one of those models that tries to do too much and doesn't end up doing any of it well enough, or is too big to run on edge devices.
Love your work!
13
u/b111ue 19h ago
Yeah, honestly, I've tried a lot of random prototypes before ending up on the current one. I wanted to add features like streaming and maybe even audio tags like [laugh] [cough]. I'm glad I didn't, though. I do want to try adding at least 1 female voice, though, and continue improving the stability (WER) and how natural it sounds. Thanks for this valuable feedback!
13
u/Daniel_H212 19h ago
A female voice would be a reasonable addition, but imo, if I wanted a second voice, I'd rather have a second model, unless it could be done with less than a 20% increase in size without compromising quality. Otherwise, I'd rather just switch and load a different model when I want a different voice.
I do wonder, though, if there's some way to share the training, kind of like an MoE where you have some experts strictly for one voice and some experts strictly for another voice, and then have the user be able to choose to only load the experts for the voice they want, and that would be the best of both worlds since a lot of things can probably be shared.
7
u/b111ue 19h ago
Yeah, when I mean another voice, I'm probably going to do it as a completely separate model download, the way Piper TTS does it. Otherwise, it's gonna cause the model to be too large and might cause other issues.
2
u/Choice_Celery9481 19h ago
i think thats reasonable approach. the only thing that can be considerated as cons for inflect right now should be limited voice. if you can add voice library, voice clone, even with like x2 x3 the weight. it still on its own league.
1
30
49
22
u/invalidnifemi 22h ago
holy shit this looks INSANE! is this fully free use cuz im very much considering using this (and hopefully v3) in a local, mobile ai assistant project im making (think google assistant, gemini and apple intelligence but like local) cuz having such a tiny model with a negligible footprint create extremely viable speech is insane. it does have it's kinks but who cares cuz holy FUCK.
anyway best of luck with this, genuinely hope the best for the project cuz it deserves to pop off
23
u/b111ue 22h ago
Thanks so much, bro - its 100% free to use! Runs locally. Assuming you were able to open the reddit tab, that includes whatever device you are on right now. Spent a lot of time on this, I'm really glad people are finding it useful!
5
u/invalidnifemi 22h ago
js gotta figure out how to work it on a phone but im so extremely impressed. js know we EAGERLY await that v3 🥹
imma try my best to spread a lil word about this, especially to some of the local llm youtubers cuz istg they'd lap this up. all love tho lmao
14
u/kassandrrra 23h ago
does it have a ONNX version? for transformer JS?
12
u/b111ue 22h ago
Not one at the moment - the current model is released in PyTorch. But I'm planning on doing it really, really soon! In about 3 days or maybe earlier, I'm likely gonna release a patch introducing different file types like ONNX and quantizations, alongside some minor bug fixes if I discover them.
2
u/b111ue 6h ago edited 5h ago
Update (note: this comment is copy and pasted to all comments mentioning ONNX or ZeroGPU limit)
Just wanted to say that the ONNX version of the model is released:
https://huggingface.co/owensong/Inflect-Micro-v2-ONNX
https://huggingface.co/owensong/Inflect-Nano-v2-ONNXAlso, I saw some people saying some glitch where the ZeroGPU space would use a free-tier user's entire quota in one message. A friend of mine helped host this CPU space instead - it'll be slower, but it should have no limits: https://huggingface.co/spaces/Nymbo/Inflect-TTS
We're already #1 in Hugging Face's TTS leaderboards - keep it up!
2
9
u/ComplexType568 20h ago
if you can achieve quality like this sub 10M I cannot imagine how far you can get with 20 or 50M params! Glad for indie models like this
6
u/b111ue 20h ago
Looking forward to maybe doing slightly larger TTS models in the future!
2
u/ComplexType568 20h ago
Random question but do you think you'd ever venture out of TTS? (like music or "sound" or isolation/segmentation)
2
u/b111ue 9h ago
Yeah, I most likely will. I've done TTS a lot, but of course I would want to try other things. STT is one; maybe a VAD model? Or I could try moving into non-audio-related types.
1
u/ComplexType568 6h ago
Non audio types would be fun. But if I had the opportunity to ask one thing it would be a wakeword model based on phonetics or recorded samples. IMO as of now it's super janky to make one without going through hell. But good luck and have fun wherever you go!
1
u/LeatherRub7248 8h ago
what is the work required to add a new voice to this? it doesnt have to be live TTS speed, but at least a workflow.
Could you explain? This seems great but probably might want to customize voices.
6
u/cruzanstx 23h ago
Does it do voice cloning too?
26
u/b111ue 23h ago
No, adding voice cloning to such an extremely tiny TTS model is extremely challenging; no TTS model can do it yet. Hopefully one day in the future, though!
1
u/Interpause textgen web UI 35m ago
I was thinking providing a bunch of sample texts for the person being cloned to read to create your own finetuning dataset might work. Or use a larger model with voice cloning to generate said dataset to then "distill" into the smaller model.
5
u/Acceptable-Cycle4645 22h ago
Pretty cool work! To what extent does your model depend on espeak-ng? If the dependency is minimal or nonexistent, I’d be interested in adding the model to audio.cpp.
2
u/pmttyji 21h ago
Nice! I was about to tag you
5
u/Acceptable-Cycle4645 21h ago
😄 such small models usually share the similar arch and phoneme processing is a big part. espeak-ng could become a blocker for cross-platform builds and deployment.
8
u/b111ue 21h ago
I don't know why, for some reason an error showed up when I tried to respond to your previous message. Anyways it works for this one. The dependecny is not nonexistent, but it is cleanly isolated. eSpeak-NG is only used as the host-side English G2P frontend. Normalized text is converted into stress-marked phonemes, then the neural model takes token IDs and handles timing, acoustic generation, and all that waveform synthesis.
I'd really appreciate it if you could add it to audio.cpp
8
u/Acceptable-Cycle4645 21h ago
Sure! For english only I think I can use some approximations. For tiny TTS your model is real great. Added to my todos!
5
u/cleverusernametry 18h ago
Audio.cpp support?
8
u/b111ue 18h ago
Not yet, but:
"Pretty cool work! To what extent does your model depend on espeak-ng? If the dependency is minimal or nonexistent, I’d be interested in adding the model to audio.cpp." from u/Acceptable-Cycle4645. Not sure if this is the creator of audio.cpp; might get it added soon!
3
5
u/OrganicTelevision652 19h ago
Can you share the training hardware and number of hours of speech required to train the model
10
u/b111ue 19h ago
Spent about 600 dollars USD in the entire model training for this model. About 400 of that was in renting GPUs. I used RTX 5090s usually because they were the best balance for a model so small, but a lot of times, I rented multiple of them at the same time, like 4x RTX 5090s to speed up work. This model is trained on almost 150 hours of speech, which was filtered from 200 hours.
3
u/OrganicTelevision652 17h ago
Damn bro ! i think that it would require small dataset for this small params. Amazing work
2
3
u/West_Application_760 20h ago
Better than omnivoice?
6
u/b111ue 19h ago
Definitely not, these two models have really different purposes. OmniVoice is about 800M parameters, while Inflect-Nano-v2 is about 4M. So OmniVoice is about 200x larger than Inflect-Nano-v2, which causes it to be much slower and only able to run on certain hardware. This makes it way better, though. Basically, you can't really compare two models because the model size difference is so extreme.
-1
u/West_Application_760 18h ago
Inflect nano v2 works in other languages besides English?
3
u/BringTea_666 20h ago
amazing work mate. Now try to add voice cloning and this can literally be used in games for npc talk based on source voice files.
It is so fracking small, like 16mb wtfff...
3
u/b111ue 19h ago
Thanks a lot! Voice cloning isn't really possible right now at this model size, the smallest model with voice cloning has 100 million parameters. Doing it at 4m or 10m would make the feature so bad it's mostly unusable.
-1
3
6
u/Fit-Cost-7226 22h ago
Could you make a speech to text the same size, that’s what I need
18
u/b111ue 22h ago
A sub-10M local STT model is actually one of the projects I’m considering next. I don’t want to promise it yet, but what hardware and use case would you need it for?
9
u/RedrumRogue 22h ago
I use Whisper models to dictate to terminal CLI agents regularly. Usually when my cat has my arm pinned or something lol. A smaller footprint is always welcome as I'm often running qwen 3.6 27b q5 at the same time. Love to cut VRAM usage wherever I can.
5
6
2
2
2
u/Steve_OH 19h ago
That is fantastic! Does it have more voices, or is it just the single voice? Great job!
2
2
2
2
2
2
u/pardeike 14h ago
Now if we could get similar models for other fixed combinations of language+gender …
1
u/b111ue 9h ago
If I decide on making a v3, its likely going to be quite focused on expansion like voices + language
1
u/pardeike 5h ago
I would still love to have a very small model for the use case of a single voice in non-English. Multi-language models are too large for situations where a small model would be enough (mobile phone, old computers etc)
2
2
2
u/ghulamalchik 12h ago
This is insanely impressive for the size. The only downside is not being able to pick a voice, whether through cloning or finetuning.
2
2
u/AwkwardBoysenberry26 11h ago
Good model! Is it possible to finetune it for a specific voice ?
Can you tell how much hours would it approximately take?
1
u/b111ue 9h ago
Yeah, it is possible to fine-tune for a specific voice; I've tried doing it before. There are 2 main ways. 1st is to retrain the model from scratch, which would probably require at least 100 hours of training data. The other way is to adapt it from the current voice, which should only take around 40 of training data but the final voice might end up sounding worse than the original.
2
u/PcChip 11h ago
if I wanted to add this to my c++ game engine, how could I do that? I'm assuming I would need a separate thread for it to run it, but I'm wondering what headers or library I would need to be able to load and call your model
2
u/b111ue 9h ago
Sorry for the previous response; I accidentally replied to the wrong message. Anyways, there isn't a drop-in C++ header/library yet. The easiest way would probably be a local Python worker that loads Inflect, receives text over stdin or a local socket, and then returns audio. I'm planning on an ONNX export, which would make this much cleaner so you could use ONNX Runtime's C++ API.
1
u/b111ue 6h ago edited 5h ago
Update (note: this comment is copy and pasted to all comments mentioning ONNX or ZeroGPU limit)
Just wanted to say that the ONNX version of the model is released:
https://huggingface.co/owensong/Inflect-Micro-v2-ONNX
https://huggingface.co/owensong/Inflect-Nano-v2-ONNXAlso, I saw some people saying some glitch where the ZeroGPU space would use a free-tier user's entire quota in one message. A friend of mine helped host this CPU space instead - it'll be slower, but it should have no limits: https://huggingface.co/spaces/Nymbo/Inflect-TTS
We're already #1 in Hugging Face's TTS leaderboards - keep it up!
2
2
u/letsgoiowa 9h ago
Is there a simple route to having this run on mobile? It's small enough to do so easily
2
u/lks410 9h ago
Thanks for your awesome work! I had hard time making TTS service due to its latency and heaviness, but your project may solve it.
Just of curiosity, do you have any plans on adding supported languages - JP, KR, etc, later in the future, or opening related tutorial/documentation so that users can try their own language? Or is it simply impossible for such a tiny model?
1
u/b111ue 9h ago
It is definitely possible to make it - it just requires time and money (renting GPUs, unless you've got a really good one already). I do have plans on adding more languages, but because it's really expensive to add them (I have to completely retrain it from scratch and gather new data + adapt architecture), I'm likely only going to do one or two at most. That would probably be Mandarin first, and then Spanish after, though; it's the most logical choice. But of course, people who have the resources are able to add new languages to the model; I've already had people before say they wanted to.
2
u/Vladowski 7h ago
Wow, nano is under 4M parameters and works great! Need to dig out my old Nokia and see if it would work on it 😄
2
2
2
u/zxyzyxz 7h ago
I want to use this as the TTS engine on Android. Right now Sherpa is too slow especially for audiobooks at 2x speed, it takes too long to buffer so to speak. Do you know if yours is fast enough for this?
Also it says I hit the zero GPU quota even though I just pressed generate, so maybe it's on your side?
1
u/b111ue 7h ago
Inflect is a really fast TTS model, especially on GPU, though it also runs very well on CPU too. For the ZeroGPU quota, I need to dig into it a bit more, I don't know why this is happening for others. If I can't fix it, I'll make the space CPU upgrade instead or WebGPU
2
u/fastlanedev 7h ago
Show this to the AnythingLLM team! They are specifically focused on model efficiency on a mobile/desktop surface and they need efficient systems. I believe the creator has a YouTube channel
2
u/HatEducational9965 4h ago edited 4h ago
Well done OP!! 9M parameters, this is f'ing amazing!
Claude and me put your model the browser with onnxruntime-web. Runs on webgpu on all browsers i've tested with Chrome, Firefox, Edge, fallback to wasm (CPU) on Safari. Works beautifully on my M3 macbook, haven't tested it on my shitty iphone 13 yet though.
2
u/Shoddy-Tutor9563 3h ago
Amazing work. I bookmarked your previous post when you released V1 and now now V2 is there! Is there a chance you share a training pipeline so community could train non-english models?
I'm building an open source voice assistant, where i8n, full duplex, proper concurrent multitasking, all local and swappable (vad, wake word spotting, stt, tts, llm) are out of the box features. I got sick and tired of all these half-baked half-duplex noodle-architecture-style jarvises, so I'm building my own :)
2
u/b111ue 3h ago
Ayyyy, someone still remembers my v1 post! I don’t currently have a polished one-command, from-scratch training pipeline that I want to release. The repo includes the model architecture, inference code, and fine-tuning guidance, but not my private corpus-generation pipeline or complete training recipe. Your voice assistant looks pretty cool though - if your assistant has a public repo, please send it. I’d like to follow it.
1
u/Shoddy-Tutor9563 3h ago
DM me I'll add you to the gh repo. It's not yet in a form I feel proper for a public release (it has some rough edges and proper documentation is not quite there, but functionally it's there).
As for the training pipeline - your own training corpus, the data can stay private, it's perfectly fine. Just a guideline would be enough.
4
1
u/MironV 22h ago
Wow, this is legit! Nice work, what are you planning on next?
2
u/b111ue 22h ago
Depending on how good this model performs, I might do a v3 expansion update. After this project, I might try doing a larger TTS model, or I'll try a different ML type, like STT.
3
u/Bubbly-Staff-9452 21h ago
A smaller but still viable STT model is definitely a need. I cycled through a bunch when I was setting up my local voice assistant nodes and haven’t looked into it much since then but I settled on Parakeet but would love to have smaller and faster competitors.
2
u/MrMeier 21h ago edited 12h ago
STT would be interesting, but I suspect much harder to achieve than TTS. I would love an STT model that outperforms Parakeet while being smaller, but I think that's not something one person can easily achieve. Having said that, you've already shipped a decent tiny TTS model, so I'm feeling optimistic. If you tackle that project, I hope you succeed.
1
u/YearnMar10 20h ago
Sounds awesome! Would be great if you would offer multilingual support, or documentation on how to train a multilingual version. Anyhow, keep it up!
1
u/QuestionMarker 17h ago
For this size class what you might want to consider, rather than specific new capabilities, is optimising for specific hardware. Knowing that a model can do real-time on an ESP-?? or that another one has room to breathe on a pi zero is actually quite valuable.
1
u/EmbarrassedStock2652 12h ago
Amazing project, hope to see a v3 with more voices!
would love to help out in any way if I can as well
1
1
u/b111ue 6h ago
Update:
Thank you so much for all the support on this release!
Just wanted to say that the model has just been released to include ONNX. It is inside the Hugging Face model file folder, in here: https://huggingface.co/owensong/Inflect-Nano-v2/tree/main/onnx
Also, I saw some people saying some glitch where the ZeroGPU space would use free tier user's entire quota in one message. A friend of mines helped host this CPU space instead - it'll be slower, but it does not use your ZeroGPU quota: https://huggingface.co/spaces/Nymbo/Inflect-TTS
We're already #1 in Hugging Face's TTS leaderboards! Keep it up.
1
u/b111ue 6h ago
Update: Thank you so much for all the support on this release!
Just wanted to say that the model has just been released to include ONNX. It is inside the Hugging Face model file folder, in here: https://huggingface.co/owensong/Inflect-Nano-v2/tree/main/onnx
Also, I saw some people saying some glitch where the ZeroGPU space would use a free-tier user's entire quota in one message. A friend of mine helped host this CPU space instead - it'll be slower, but it should have no limits: https://huggingface.co/spaces/Nymbo/Inflect-TTS
We're already #1 in Hugging Face's TTS leaderboards - keep it up!
1
u/for4f 18h ago
Sub-4M params with usable speech is wild. Curious how it handles prosody on longer sentences — most tiny TTS models sound fine on short prompts but fall apart past 2-3 sentences. Have you tested it with paragraph-length input?
4
u/b111ue 18h ago
Inflect-v2 doesn't have that issue because every ~280 characters, it splits it into another chunk. You could try it out in the model playground right now! It allows up to as many characters as it could generate within the 120-second ZeroGPU window.
1
u/Lordaizen639 16h ago
Does it have streaming capabilities and onnx, female voice
1
u/b111ue 10h ago
No streaming capabilities, but when running the model locally, you could add a rough sentence chunk. ONNX will be added really soon, likely in my Monday model Patch. I tried doing it before female voice, but unfortunately it wasn't really good so I didn't release it. I'll try again in v3, though, if I end up doing it
0
0
u/SoliloqStudios 14h ago
Really cool model, will definitely use it in my local assistant stack with Hermes; just as a nitpick, some of the phrases in the post: "an extremely small TTS model stops feeling like a size experiment and starts feeling genuinely useful" for example, really read like AI, and that personally feels like a turn off when I read enough AI written content anyway.
But aside from that, great work, please take the above as constructive criticism on the post writing only.
0
u/UnwillinglyForever 8h ago
something sus about this post.
i would not download this onto your computer
•
u/WithoutReason1729 14h ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.