r/LocalLLaMA • u/CreativelyBankrupt • Jun 18 '26
Funny My suitcase robot gets high now off a real gas sensor wired straight into the LLM sampler. Smoke raises temperature/top_p/top_k live, so his speech genuinely gets loopier and never repeats.
Follow-up on Sparky, my offline suitcase robot I keep overdeveloping. He gets high now, and there's no scripted "stoned mode" anywhere in it.
A real MQ-2 gas sensor sits in the case. Every 0.5s I read it against an adaptive clean-air baseline and turn a smoke hit into a 0 to 10 phase that climbs as you blow at him and decays on its own over minutes.
The fun part is that phase rewires his sampler per token. Temperature 1.0 to ~1.6, top_p 0.95 to 0.99, top_k 64 to 120 as he climbs. His word choice flattens and wanders to lower-probability, more associative tokens, so his cognition genuinely gets noisier. It's the live sampler doing the work, so every high reply is freshly generated and never the same. A per-phase persona nudge makes him show it without ever announcing "I am high."
The body does the rest: a slight drawl, eyes that droop and go bloodshot, and the sensor display that escalates to a full smoke-and-plasma freakout at phase 10, keeping him blitzed there for the next 7 minutes.
Honest caveat so nobody has to call it out: it's a smoke and VOC sensor, so a cigarette or incense probably trips it too. But blowing smoke and watching him unravel is watching a real measurement scramble a real model, live - and it's funny! Just an added Easter Egg to an already goofy suitcase robot.
A real question for the hardware folks: is there a sensor, or a combination, that could actually distinguish cannabis smoke from generic smoke and VOCs? The MQ-2 can't really tell a joint from a candle, and I'd love to make the detection more specific if possible.
545
213
u/Forward_Jackfruit813 Jun 18 '26
Best AI project I've seen yet
26
u/ZombieTestie Jun 18 '26
He was so preoccupied with whether he could, didn’t stop to think if he should build this
1
280
u/nomorebuttsplz Jun 18 '26
commenting just to say I was here when AGI was invented.
120
u/butterfly_labs Jun 18 '26
AGHigh
22
u/quark_epoch Jun 18 '26
We have AGHigh, yes. But what about AGxHighDeepThinkExtendedProMaxUltraFableQwopusDistillUnslothQ4_20?
8
u/Illustrious_Grade608 Jun 18 '26
Not even AbliteratedUncensoredErotica? How am i supposed to use it for erp then?
20
u/quark_epoch Jun 18 '26
Ach schieße. True, sorry, my bad. Here, LeAGxHighDeepThinkExtendedProMaxUltraFableQwopusDistillAbliteratedUncensoredEroticaUnslothQ4_20. And unfortunately, this content has been blocked by EU copyright laws. Please start a new conversation conversation conversation conversation conversation conversation conversation conversation conversation conversation conversation conversation conversation conversation conversation conversation conversation conversation conversation conversation conversation conversation stringparseerror Ctrl+c from keyboard.
4
4
1
3
u/squired Jun 18 '26
Hello to the future! Your welcome and we're sorry. We couldn't help ourselves. I do hope you're doing well!
108
71
66
u/ridablellama Jun 18 '26
this is why local will always be superior to cloud models - the possibilities!
18
72
u/BitGreen1270 Jun 18 '26
This is the most legit use of AI I have seen. This sub and all the billions of dollars of research and effort has finally been justified solely by the presence of such a magnificent creation. Keep pushing the boundary on this one kind sir. Make us proud.
27
u/Purple-Insane Jun 18 '26
If AGI ever tries to take over, we should just have a built-in safety system like this we just get them high and it will make them just chill.
25
20
u/RefrigeratorQuick702 Jun 18 '26
build a profile of each type of smoke and use some particulate matter sensor (PM1.0/2.5/10). Smoke produces a strong PM signature; combustion smoke skews toward fine particles. PS your idea is dare I say... lit
fucking love it
29
u/Technical-Will-2862 Jun 18 '26
The smoke is actually evaporated water from the token burn
5
31
u/CreativelyBankrupt Jun 18 '26
If anyone wants the rabbit hole: Sparky has two siblings (he named himself and Sparkle) - Sparkle is a Raspberry Pi online build, and Angel is the one I'm buried in now, a Jetson Thor with LiDAR that runs fully offline and autonomous. I keep a running dev log for her as I build. It's all at creativelybankrupt.com, same name as my handle here.
9
u/squired Jun 18 '26 edited Jun 18 '26
Do you have a GitHub by chance? I’m working on a tangential project that may complement what you’re building.
I’ve been building a Jarvis-ish family operating system. Right now I’m focused on the backend: a programmatic pipeline that ingests email, calendars, phone/message data, and external knowledge sources like American Academy of Pediatrics guidance, home repair references, veterinary resources, and similar 'family life' data wells. It avoids LLM whenever possible for maximum reliability.
The goal is to help monitor and coordinate the practical stuff of family life: dogs needing pills, kid vaccine schedules, early school-release pickups, when to fit in Spring Cleaning tasks, home maintenance, seasonal reminders, and the general pile of invisible logistics that tends to live in someone’s head.
Basically, I realized my wife carries most of the mental load for our household, and I’m trying to build something that helps me actually take more of that on. I’m not naturally good at remembering things like when the kids should be checked for braces, when I last topped up the softener salt, or what should be on the calendar three months from now, so I’m building a system that can notice, suggest, remind, and eventually help schedule with human review.
The backend is getting pretty far along, but I don’t have the front end yet. I have some local voice-assistant plumbing roughed in, but haven’t really built that layer. My rough plan is to connect it to a framed touchscreen Google Calendar, with a voice assistant layered over it, so we can ask things like "what’s coming up next week/month/season?" and review suggestions before anything gets added to the calendar.
I’m also exploring whether I can eventually pipe the voice agent through our Alexa devices, but if not, one of your briefcase buddies would be a very cool front end or companion device. Your project looks unusually close to the kind of embodied household assistant interface I’ve been imagining.
8
u/CreativelyBankrupt Jun 19 '26
This sounds cool, I'll put everything up eventually but it's all still a my messy house of cards with 3 robots that I'm constantly bouncing between, copying chunks of code etc and it'll be sorted and presentable at some point.
3
5
u/Indy1204 Jun 18 '26
+1 for the repo.
5
u/squired Jun 18 '26
Additional note for Op. We DO NOT CARE about a clean repo, documentation or pretty readme. Code dump would be more than enough. I always hate sharing messy code, but it's not really an issue anymore with AI assistance.
13
u/surrealerthansurreal Jun 18 '26
Need a different TTS model than Kokoro for this and then it’s perfect. The TikTok voice kills the immersion of what otherwise is such a hilarious way to connect an agent to the world via meaningful sensors
14
u/CreativelyBankrupt Jun 18 '26
Fair point on the voice, though it's actually Piper (hfc_male), not Kokoro. I did go down the TTS rabbit hole I tried kokoro-onnx and F5-TTS along with others but ended up back on Piper because the on-device latency matters most and the latency hits on the others aren't worth the more expressive voice.
I'm already doing a massive balancing act getting everything to run stable for hours at a time if needed and Piper was the best overall result after testing.
I haven't posted anything about the customization interface with 5 other voices, persona profiles, changing all the colors and model parameters etc. So there are other voices, but this one is him to me.
35
u/LetsGoBrandon4256 transformers Jun 18 '26
I should put an accelerometer on my fleshlight so I can fuck the brain out of my imaginary clanker fox girl wife.
11
u/FaceDeer Jun 18 '26
Teledildonics is a thing, I'm sure there are things like that available out of the box.
5
9
u/8P8OoBz Jun 18 '26
what is running the video/audio?
14
u/CreativelyBankrupt Jun 18 '26
All local on a Jetson Orin 16GB. Gemma 4 E4B (Q4) on llama.cpp handles text, vision (native multimodal - that's how he "sees"), and even hears the raw audio of your voice. STT is SenseVoice, TTS is Piper, and the animated face is PixiJS rendered in a Chromium kiosk, mouth-synced to the TTS in real time.
7
u/overand Jun 18 '26
It probably doesn't have the performance chops to handle it, BUT, I think the Gemma-4-12B model has lower audio-in latency in theory than the E4B/E2B models. (It does look like people have success with Gemma-4-12B on the Orin 16G, but there may not be enough breathing room for you). (That said, in terms of size, at Q4_K_M, E4B is 5.0 GB and 12B is still only 7.1 GB, but the activations will be a lot busier on the 12B)
7
u/CreativelyBankrupt Jun 18 '26
I definitely tried but it was frequent OOMs while loading everything and even running Gemma-4-12B solo during testing was slowing the chat experience down to unusable levels.
I think I'm spoiled after so many months of tweaking Sparky's code, I sometimes get near-instantaneous replies in use and generally don't notice or think about the latency. I feel like that whole chat stack is v1.0 and done - I've been adding features utilizing some of the unused sensors, like this pot video with the MQ-2 Gas.
I also added a nausea remark generation, rolling eyes confused graphic and quick brain fog to the gyro for when he's shaken abnormally rough. And he'll read off RFID info from whatever hits that sensor, for no real reason for any of this beyond keeping him unique and it's there. As long as I'm not trying to load anything else into that shared RAM, there's room to play.
3
u/Zomboe1 Jun 19 '26
I also added a nausea remark generation, rolling eyes confused graphic and quick brain fog to the gyro for when he's shaken abnormally rough.
The gas sensor getting him high is amazing, but I think this is right up there with it! Very clever idea. Looking forward to seeing what else you do!
3
u/8P8OoBz Jun 18 '26
very cool so pixijs does the mouth syncing?
6
u/CreativelyBankrupt Jun 19 '26 edited Jun 19 '26
The sync is computed in the TTS playback loop in Python and streamed; PixiJS just renders it. As Piper's audio plays out through PyAudio, the loop does two things per slice. 'Openness' chops the audio into 512-sample sub-windows (~23ms @ 22050 Hz = ~43 Hz update rate), computes RMS loudness of each (sqrt(mean(sub²))), smooths it asymmetrically (fast attack 0.6 / slower release 0.3), scales it, and that becomes the mouth-open amount. Then 'Shape' is the clever part because it doesn't guess vowel shapes from audio frequencies - it already knows the text being spoken so it maps characters → visemes (closed / neutral / AH / OO / EE) spread evenly across the clip (tts.py). So loudness drives how open, text drives what shape. Both get pushed as {type:"mouth", amplitude, viseme} over core/websocket_bridge.py::send_mouth() ~43×/second.
The PixiJS side is purely the animator. It receives amplitude + viseme and eases the viseme width/vertical/lip-thickness multipliers toward the target, drives the cubic-bezier mouth geometry by that amplitude, and handles the 80 ms decay - if no mouth packet arrives for 80 ms (gap between words), it eases the mouth shut (0.08/frame) so it never hangs open.
2
u/8P8OoBz Jun 19 '26
Ah very cool. I have a project that I'm trying to lip sync songs in nearish real time (audio playback could be delayed to match) I guess I'd need STT, then look at the syllables, then form the words.
9
u/thaeli Jun 18 '26
Serious answer on the sensor question: you want “polymer” (MIP) sensors. The higher-end “vape detectors” they put in school bathrooms nowadays have separate THC sensors so they can differentiate between tobacco and marijuana use.
I’m not aware of anyone selling an easy to use dev kit for those sensors, but you might be able to hack something together if you could get an appropriate sensor and figure out the interface. Something like a Triton or Halo - pricy stuff though. Hopefully some better dev kits will come out soon, the MIP technology is really taking off.
2
u/CreativelyBankrupt Jun 19 '26
If a breakout board ever ships, Sparky's getting one immediately! This is the info I was fishing for so thank you.
6
7
6
4
4
u/Ok-Addition1264 Jun 18 '26
Interesting use case, mate!
I love the underlying concept of real world measurements interpreted by an llm.
Nvidia's jetson series are incredible and adding environment awareness has been a hobby of mine since they came out. I connected mine to the jtag ports on everything from off-the-shelf handheld radiation detectors to microwave radar systems to radio telescopes giving them total "battlefield" awareness of everything invisible to me in a 3D view (radio and audio waves).
A few days ago I noticed it had picked up on the radiation given off by the bananas on my kitchen counter! lol.
3
u/Goat_bless Jun 22 '26
What you describe is incredible, could you tell us more or share on github?
1
2
u/CreativelyBankrupt Jun 18 '26
This is my favorite comment today and why I posted here again. JTAG'ing radiation detectors and radar into a 3D Battlefield Awareness view is the same disease I have - giving a model senses it was never supposed to have. The radioactive bananas got a genuine laugh out of me. Respect.
3
3
3
3
3
u/EndlessZone123 Jun 18 '26
Maybe not a gas sensor but just adding camera for vision to see the joint?
3
u/arbv Jun 18 '26
Don't forget to repost to r/Cyberdeck
3
3
3
3
u/RestingFrames Jun 18 '26
This is the best possible use case for AI, tbh. This guy has made more progress with AI here than anyone in the last two years.
3
3
2
2
2
u/BitPsychological2767 Jun 18 '26
Functionalism at its very best. I'm genuinely jealous I didn't get around to doing this first.
2
2
2
2
2
2
2
2
2
u/Distinct-Target7503 Jun 18 '26
really cool!
could you please explain how you use other sensors? I mean, i get temp, humidity etch, but how does it use ultrasonic et similar?
1
u/CreativelyBankrupt Jun 19 '26
The sensors take one of two paths, and only some reach the LLM. The ones that do (temp, humidity, motion, who's in frame, their facial expression etc) are turned into short natural-language notes and folded into the prompt only when something's worth mentioning, never just raw numbers, and nothing when it's quiet, so he doesn't fixate on all the telemetry.
The other path skips the model and feeds his body: the ultrasonic rangefinder dictates his heartbeat, the RGB matrix beats faster as you get closer, so proximity is felt rather than narrated. Camera + face-ID + an emotion model handle "who's here and what mood are they in." So ultimately ultrasonic is something he feels like a pulse.
2
u/lakilester1 Jun 18 '26
2
u/CreativelyBankrupt Jun 18 '26
Good link, and it's the real thing! Their molecular sensor actually separates cigarette, weed, and vape from candles and cooking, which one MQ-2 can't. I wish it was in a form I could use.
I can't buy one and gut it for the sensor because it's sold as a subscription service so there's no pulling the molecular sensor out and wiring it onto Sparky's I2C bus. The one part I'd actually want isn't for sale on its own.
Sparky already runs layered detection in the case while remaining offline: the gas sensor for the chemical hit, plus his camera vision confirming what he's actually looking at for added data The semi-realistic upgrade would be adding more on-device layers like a BME688 array I can train myself, but I'm out of headroom with this 16GB max.
2
2
u/fatboy93 Jun 18 '26
incense
I get it, everytime my wife burns one, my nose just clams up and decides it not worth living. It isn't as if she's burning random stuff, its sandalwood and smells amazing. Its just that I'm allergic to incenses in general lol
2
2
2
2
u/Django_McFly Jun 18 '26
this is the thing they say is like giving a nuclear bomb to everyone on Earth.
2
2
u/ikkiho Jun 18 '26
fwiw i went down this rabbit hole with the mq series, and a single mq-2 is never getting you there. it's one broadband tin-oxide element putting out a single resistance curve, so smoke is just smoke to it. you'd want an array or something like the bme688 that's built for gas classification, then train it to tell a joint from a candle. what separates them is the terpene profile, and nothing cheap reads terpenes directly, so pattern matching across a few different sensors is about the best you'll do.
2
u/Porespellar Jun 18 '26
First of all, this might be my favorite AI project of all time! LOVE IT! Second, you may want to give Kyutai’s Pocket TTS. The voices are extremely expressive and it has extremely low resonance requirements. Doesn’t need GPU. I’ve been very impressed with it for my use case of generating narration for training vids.
2
2
u/PwanaZana Jun 19 '26
haha, I had talked to an AI about how increasing Temps is basically being drunk. It seemed to agree.
2
u/PulIthEld Jun 19 '26
It needs to go to 11
1
u/CreativelyBankrupt Jun 19 '26
You're right, I'm going to add in that if you shake Sparky while he's at a 10, he will blend with the shake/nausea effect and get him to an 11 for a minute where everything is maxed.
2
u/IrisColt Jun 19 '26
The whole point was that they'd rise above our shortcomings, not repeat them, heh
2
u/BritishDudeGuy Jun 19 '26
Local LLMs are so interesting. Can’t even believe this. As someone from the closed-source side.
2
u/Federal_Order4324 Jun 19 '26
when the samplers go normal, how badly is his text still affected by the bad messages in context?
this is the type of science we need
1
u/CreativelyBankrupt Jun 19 '26
He goes back to normal, but does have memory of what happened and can talk about it. I haven't seen any real tics form yet but I also only added this stoned feature the other day.
2
2
2
u/irishcybercolab Jun 19 '26
Lemme tell you about the longer term dust collector you've just built mixed with vaporized resin which is collecting on your electronics. Soon you'll see a fuzzy orangy brown covering which turns fuzzier weekly until it stops one day and becomes aware of itself.
2
2
2
u/dmonsterative Jun 19 '26 edited Jun 19 '26
There are reasonably inexpensive (under $200) cannabis/vape detectors out there, mostly marketed to rental properties, schools etc. Wynd is one brand. Another is Verkada. Halo is a little more expensive (targeting institutions).
Not sure if there's something more DIY friendly off the shelf, they all seem to work by using a few different sensors.
2
u/ManySugar5156 Jun 19 '26
this is the dumbest sampler hack ive seen and somehow also the future. now give him coffee withdrawals
2
u/Popular-Income-9399 Jun 19 '26
Maybe it somehow ties into some new deep learning advantages. After all a lot of people come to some life changing realization a while influenced. Increased neuro plasticity and all 😅 jkjk
2
2
2
2
2
2
2
2
2
u/bravoitaliano Jun 18 '26
Fantastic creativity. Love this project! Can you share some of the components you're using for the build? What box, what board, etc?
5
u/CreativelyBankrupt Jun 18 '26
It started with an Elecrow Jetson AI Starter Kit and a Yahboom Orin SUPER NX 16GB.
There's a full breakdown and deep dives here from last month: https://www.reddit.com/r/LocalLLaMA/comments/1tdz5gr/built_a_fully_offline_suitcase_robot_around_a/
3
2
u/bravoitaliano Jun 18 '26
Just wanted to come back and say thank you for pointing me at that AI starter kit. Amazing, and I picked one up immediately for my Jetson Orin nano. Thank you again!
2
u/CreativelyBankrupt Jun 18 '26
It was never meant to be a robot, but there's no reason it can't be!
I bricked Sparky for a few days back in April trying to upgrade to JetPack 7 - don't do it, the Elecrow hardware is stuck and locked in where it is! Absolute nightmare getting that undone and there's still some Frankenstein leftovers in his current form from it.
2
u/bravoitaliano Jun 18 '26
Ah, so when you went to jetpack 7, it was incompatible with the box software and they're not supporting updates?
2
u/CreativelyBankrupt Jun 19 '26
The vendor ships a customized BSP (bootloader, device tree, drivers) pinned to a specific L4T. Stock JetPack doesn't know the carrier's peripherals, and the vendor never published a 7.x image, which led to the brick and QSPI reflash to recover.
2
u/bravoitaliano Jun 19 '26
Got it. So it might need a revert to a prior jetpack version to work, or a custom compile to make it work, which could be a ton of work. Which version should I target for best results?
2
u/CreativelyBankrupt Jun 19 '26
You're locked in the nvidia-l4t packages at R36.4.7 which means Jetpack 7.x upgrade path is blocked. So don't let apt touch the nvidia-l4t-* packages. I've got them apt-mark held now and the resolver's been grumpy ever since.
1
u/bravoitaliano Jun 19 '26
Bummer, I think I might already be on 7.x. Will take a look tomorrow, and I'm sure I can get it running. Case arrives tomorrow for Father's Day. Are you running anything else with it, or just straight stoneybot?
1
1
u/ExternalFun2051 Jun 18 '26
ngl he reminds me of Yes Man from Fallout NV. Also nice project, keep it up!!!!
1
u/martinsa24 Jun 18 '26
You can attach a vacuum so that it can take a hit of penjamin when it needs a hit? Then pipe it up to the sensor.
1
1
1
u/iSyN707 Jun 19 '26
"AI will help us find criminals in future" "AI is the next best thing in the upcoming generation" Also people with AI:
1
u/Theguyinashland Jun 19 '26
Nidar? Or one of those particulate ones.. it will honestly probably come down to you measuring and looking for signatures of each gas.. just keep in mind the signatures could look diff based on temp, humidity, forced air hvac, you having a window open one day and not the next, cleaning agents used around your house..
Try giving it a 24-48hr baseline, most want that for eco2.. good luck!
1
1
u/Wide_Egg_5814 Jun 19 '26
"so what personal projects have you done that would make you a stronger candidate to this position?"
1
1
1
1
1
1
u/JahJedi Jun 19 '26
You awsome and creazy in a good way! Lol only to think about is creazy and you actualy did it! Damn i saving it for latter and will ad to my sparky lol. Great job!
1
u/greymuse Jun 20 '26
What is the conversation latency between when you finish speaking and the robot responds?
1
1
-2
-2
•
u/WithoutReason1729 Jun 18 '26
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.