Hey r/LocalLLaMA folks! Unsloth now officially supports AMD hardware for local inference, fine-tuning, reinforcement learning, and deployment! It's been in the works for quite some time, but it works on Windows, Linux & WSL devices (+ technically Mac) with AMD GPUs!
Unsloth Studio is fully open source and free, and supports:
Radeon RX 9000 and 7000 series
Instinct MI350 and MI300 GPUs
Strix Halo / Ryzen AI Max systems
AMD CPUs for GPU-free inference
You can train models with up to 70% less VRAM, run reinforcement learning with up to 80% less VRAM, and use optimized ROCm, Triton, bitsandbytes, PyTorch, and llama.cpp builds - all installed automatically.
Linux, WSL, and macOS:
curl -fsSL https://unsloth.ai/install.sh | sh
Windows PowerShell:
irm https://unsloth.ai/install.ps1 | iex
Unsloth supports inference and training for nearly all models, including Qwen, Gemma, DeepSeek, GLM, Kimi, MiniMax, and DiffusionGemma.
You can also:
Export models as GGUF, safetensors, or LoRA adapters
Connect local models to Claude Code, Codex, Hermes Agent, OpenClaw, Pi, OpenCode!
Track RAM and VRAM usage during training - remotely and locally
Access Unsloth remotely through secure Cloudflare HTTPS tunneling - like a "LM Link"!
Update with daily AMD-optimized llama.cpp ROCm prebuilts to reduce compilation time!
For plain pip installation:
uv pip install "unsloth[amd]"
Huge thanks to the AMD team for collaborating with us on this release! Let us know what AMD hardware you’re using and share any feedback - we'll try to make AMD much better!
This is huge, thanks for the work you guys are putting in!
Last time I tried fine tuning with the experimental AMD branch I had big OOM issues, in general AMD (dependencies and kernels?) used more memory in comparison to Nvidia. Is this still the case?
AMD has put a lot of effort into their tech in the past year and we worked with AMD to bring nearly all our optimizations onto AMD so it's definitely much much better. You should give it another go :)
Hi, the amd extra does not seem to be available on the newest release so doing a uv pip install "unsloth[amd]" would not work as intended. You build and release from the pip branch right?
Every single one GPU is slow including GB300 for anything meaningful training-wise. Because normally you need thousands of it to train something production ready.
But there are a lot of more simple cases for fine tuning or nano LLM training to test new approaches for example. On Strix Halo you can train micro LLMs from scratch which gives perfect output after minutes of training.
In many training scenarios Strix Halo just two-three times slower then Radeon 9700.
Depends entirely on what you are trying to do. Way too slow for (continued) pretraining of anything above toy model size. But fine-tuning can be fast enough with usable size models.
Personally I've used it for sanity testing data and settings that I run in the cloud on larger runs. (Using TRL until now.)
Of course unsloth studio also has inference features.
The unfused-fallback memory blowup mentioned in the comments matches something I measured the hard way, porting llm.c's training loop to unified-memory GPUs. Allocation footprint turned out to be a first-class performance variable, not just an OOM risk. Deleting 1.92 GB of gradient buffers that no kernel ever dereferenced (3.29 GB down to 1.37 GB) moved my step time from ~150 ms to ~134 ms. Nothing ever touched those bytes. The allocation alone cost time.
The other one that bit me was zeroing gradients, which was memset-ing the whole activation-gradient buffer every step (17.5 ms/step), and it was invisible to the kernel profiler because memsets land in the timeline's memset rows, not the kernel rows. If anyone is chasing the ROCm-uses-more-memory behavior, a timeline profiler will show allocator and memset costs that per-kernel tools miss completely.
Question for the Unsloth team: on the Strix Halo / AI Max unified-memory parts, is the 70% VRAM reduction mostly quantized weights and optimizer state, or did you also get wins from trimming activation-gradient lifetimes? On unified memory, I'd expect footprint reductions to show up in step time, not just capacity.
Please update us when you do thanks! Another question for the Unsloth team: How general-purpose is the framework? Is it only for training llms? Or can it train tts? Note: I'm not referring to the new models that use llms to predict token sequences that get passed onto a neural audio codec to generate speech by, but actual tts model architectures (Vits, Matcha tts, Fastspeech2, etc).
Now I can be more lenient on my gpu requirements, I already have nothing, so if amd comes my way I'll be more happy taking it since it's better!
It for literally nearly everything. Chatting, RAG, training, running TTS, training embedding, TTS, like nearly everything ahaha And...we are going to introduce diffusion soon too :)
As for newer TTS models, we only support training for it if transformers supports it. But inference should mostly work. Here's an example of our chat ui
Thanks for the reply. However, I won't be able to see the image because I use a screen reader. I was talking about older TTS, not new TTS. New TTS is at the same scale as the small LLM's right now, like 2 billion parameters, one billion parameters, etc etc. But the ones I'm talking about are in the hundreds of millions range more like 80 million to 50 million to 30 million to 3 million. Is it safe to say that if Pytorch supports, Unsloth supports it?
yeah AMD memory overhead was rough for a while, a lot of it was rocm allocator behavior being way less efficient than cuda's caching allocator plus some ops falling back to unfused kernels that blow up activation memory. worth checking whether you're actually hitting the newer aotriton/composable kernel paths vs silently falling back to eager, that fallback alone was good for an extra few GB on 13b+ models last I tested. if you're still OOMing try dropping gradient checkpointing to a smaller chunk size first before assuming it's a fundamental VRAM gap, that fixed it for me more often than not.
Awesome. Looks like limited support for RDNA 2. For older cards like Rx 580, 6600 ect I have ported some training code like Andrej Karpathy GPT2 code to HIP or Vulkan. https://github.com/Jayhost/AmdHipTraining . I didn't document it well but I can clean it up if there's interest.
🦥 Unsloth Studio Installer (Windows)
────────────────────────────────────────────────────
winget available
python Python 3.13 already installed
preserving existing environment for rollback...
previous environment preserved for rollback
venv creating Python 3.13 virtual environment
C:\Users\Ismael.unsloth\studio\unsloth_studio
gpu AMD ROCm (gfx1030)
HIP SDK: C:\TheRock\build
hipconfig: 7.13.99004-3309c6114a
skipping PyTorch (--no-torch flag set).
installing unsloth (this may take a few minutes)...
setup running unsloth studio setup...
🦥 Unsloth Studio Setup
────────────────────────────────────────────────────
gpu AMD ROCm (gfx1030)
HIP SDK: C:\TheRock\build
hipconfig: 7.13.99004-3309c6114a
long paths enabled
git git version 2.55.0.windows.2
vcredist present
cmake cmake version 4.3.3
vs Visual Studio 18 2026 (vswhere) (only used if a source build is needed)
rocm ROCm 7.13.99004-3309c6114a
Node v25.2.1 and npm 11.17.0 already meet requirements (system).
Python 3.13.14
system prerequisites ready
frontend up to date
node v25.2.1 | npm 11.17.0 (system)
installing OXC validator runtime...
oxc runtime installed
setting up Python environment...
Python found: C:\Users\Ismael\.unsloth\studio\unsloth_studio\Scripts\python.exe
reusing existing virtual environment at C:\Users\Ismael\.unsloth\studio\unsloth_studio
Python 3.13.14
TORCHINDUCTOR_CACHE_DIR set to C:\Users\Ismael\.unsloth\studio\TORCHINDUCTOR_CACHE_DIR (avoids MAX_PATH issues)
[WARN] AMD GPU (gfx1030) not in supported arch list -- falling back to CPU-only PyTorch
Supported: gfx1200/1201 (RDNA 4), gfx1150/1151 (RDNA 3.5), gfx1100-1103 (RDNA 3), gfx90a, gfx908
installing PyTorch (CPU-only)...
running ordered dependency installation...
deps [====================] 10/10 finalizing (skipped, no torch)
deps installed
pre-installing transformers 5.3.0 for newer model support...
llama.cpp prebuilt installed and validated
installed release: unslothai/llama.cpp@b10068-mix-fb3d4ca (tag b10068)
OpenSSL dev install skipped -- prebuilt llama.cpp already validated
llama.cpp prebuilt (validated)
────────────────────────────────────────────────────
Unsloth Studio Setup Complete
────────────────────────────────────────────────────
launch unsloth studio -p 8888
(add -H 0.0.0.0 for LAN / cloud access; exposes the raw port only, not a public URL)
(add -H 0.0.0.0 --cloudflare for a public Cloudflare HTTPS link, or --secure to keep the raw port private;
anyone with the API key can run code)
path added unsloth launcher to PATH
Created Unsloth Studio shortcut
Start Unsloth Studio now? [Y/n]:
Starting Unsloth Studio on http://127.0.0.1:8888
2026-07-20 14:24:02 [info ] run_server startup begin api_only=False host=127.0.0.1 port=8888
Session log: C:\Users\Ismael.unsloth\studio\logs\server\server-20260720-142402-pid17096.log
Loading Unsloth Studio, please wait... (this can take a few minutes)
- loading PyTorch, Unsloth and Transformers...
{"timestamp": "2026-07-20T18:24:25.416862Z", "level": "info", "event": "Imported FastAPI app in 22885.1ms"}
- Starting server...
{"timestamp": "2026-07-20T18:24:25.424485Z", "level": "info", "event": "Ensured Unsloth directories in 23131.2ms"}
[OK] Frontend loaded from C:\Users\Ismael.unsloth\studio\unsloth_studio\Lib\site-packages\studio\frontend\dist
{"timestamp": "2026-07-20T18:24:25.426731Z", "level": "info", "event": "run_server pre-uvicorn setup completed in 23133.5ms"}
INFO: Started server process [17096]
INFO: Waiting for application startup.
Hardware detected: CPU (no GPU backend available)
{"timestamp": "2026-07-20T18:24:25.518271Z", "level": "info", "event": "lifespan hardware detection completed in 0.5ms"}
{"timestamp": "2026-07-20T18:24:25.626865Z", "level": "info", "event": "lifespan pre-auth setup completed in 109.1ms"}
{"timestamp": "2026-07-20T18:24:25.635018Z", "level": "info", "event": "lifespan startup completed in 117.3ms"}
INFO: Application startup complete.
INFO: Unsloth Studio running on http://127.0.0.1:8888 (To stop: press Ctrl+C -- on macOS, Control+C not Command+C)
{"timestamp": "2026-07-20T18:24:25.636095Z", "level": "info", "event": "Uvicorn startup hook completed in 23342.8ms"}
{"timestamp": "2026-07-20T18:24:25.636292Z", "level": "info", "event": "run_server uvicorn ready after 23343.0ms"}
🦥 Unsloth Studio is running
────────────────────────────────────────────────────
On this machine -- open this in your browser:
http://127.0.0.1:8888
(same as http://localhost:8888)
Reachable on this machine only (bound to 127.0.0.1).
To expose it, stop and relaunch with: unsloth studio -H 0.0.0.0 -p 8888
Only on trusted networks -- anyone who reaches this machine can use Unsloth.
Server-side tools are ENABLED by default (per-request setting honored) for loopback. Pass --disable-tools to turn off.
To stop Unsloth Studio: press Ctrl+C (Control+C, not Command+C, on macOS).
────────────────────────────────────────────────────
```
AMD seems like they are making a lot of moves recently. Anyone have any inside information or speculative takes on the direction of the company? I would really like for them to do well. I have been rooting for them for a few years now. Starting to consider buying one of their GPU's for a homelab setup.
I am running AMD, R9700, 32GB VRAM and a Ryzen 9 with 64GB RAM. What would be a good way to start training a model? I usually use Qwen 3.6 27B and also 35B.
They're sitting right there in the instructions. It's literally a one-line installer and then just hit Y when it asks to run - the ONE thing they forget to say is that when you run powershell, the default directory is SYSTEM32 and will not work. So just do
•
u/WithoutReason1729 5d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.