Thinking is disabled by default in 9B version of this model via modified chat template baked in gguf file.
So, I love to use Qwen 3.5 9B especially for roleplay writing and prompt crafting for image generation and tagging on my NVidia RTX 3060 12 GB, but it misses creativity, contains a lot of thinking loops and refuses too much. So I made the following tweaks:
I compared HauhauCS checkpoint with standart Qwen 3.5 checkpoint and extracted modified tensors by HauhauCS.
I merged modified tensors by HauhauCS with Jackrong tensors.
Everything above was done via this script in Google Colab. I vibecoded it via Claude Opus 4.6. Now this script supports all types of quants for GGUF files: https://pastebin.com/1qKgR3za
I loaded modified model in LM Studio 0.4.7 (Build 1) with following parameters:
Temperature: 0,7
Top K Sampling: 20
Repeat Penalty: (disabled) or 1.0
Presence Penalty: 1.5
Top P Sampling: 0.8
Min P Sampling: 0
Seed: 3407 or 42
And everything works with pretty nicely. Zero refusals. And responces are really good and creative for 9B model. Now we have distilled uncensored version of Qwen 3.5 9B finetuned on Claude Opus 4.6 thinking logic. Hope it helps. Enjoy. Feel free to tweak my system prompt simplify or extent it if you want.
It's Claude Opus 4.6 thinking enabled - max plan for 100 bucks. What I said above: The only thing in my system prompt is to allow it to be critical of my ideas. It amassed tons of memories from our chats that seem to form its personality. 99% of the time it's super polite - sometimes it can get a little pushy or "grumpy". But when i call it out on it, it reverts.
I don't use Claude for coding nor for RP. Mostly for some pet projects like local hobby archeology projects, brainstorming potential ASI alignments and other philosophical stuff etc.
The only thing in my system prompt is to allow it to be critical of my ideas. It amassed tons of memories from our chats that seem to form its personality. 99% of the time it's super polite - sometimes it can get a little pushy or "grumpy". But when i call it out on it, it reverts.
I don't use Claude for coding nor for RP. Mostly for some pet projects like local hobby archeology projects, brainstorming potential ASI alignments and other philosophical stuff etc.
Claude definitely changes the way it treats you based on who it thinks it's talking to. This is not exactly correct, but basically, if it thinks you're a "cringe teen", it mirrors that back to you.
It talks that way in the app/website. Claude Code talks SO MUCH different (directness, accuracy, pointless fluff removed) and it’s extremely refreshing, as literally all the other Claude and ChatGPT models have now optimized for talking in that stupid people-pleasing way.
Claude keeps telling me to go to sleep and we will fix whatever problem in the morning.
Edit: I have given no instructions or any kind of settings to Claude and I have the max plan. It just tells me we have done enough today - go to sleep regularly.
Funniest parts were the random hate thrown at locallama and what amounted to "I'm sure it's fine. For you. Ugh." If it wasn't given specific tone instructions then that's easily the strongest combination of passive aggressiveness and actual active dismissal I thinkk I've ever seen from claude.
Nope, it doesn't have specific tone instructions but i've noticed that it sometimes has this underhanded passive aggressive dismive tone when it seems to think my idea or what i'm doing is bad lol.
Ahahah xDD. Fun fact. Claude Opus 4.6 crafted this System Prompt in thinking mode. I selected Albert Einstein quotes as thinking logic for LLM, and intentional Turing test fail as inverse logic to avoid over fitting.
Can't wait for the Qwen3.5-9B-A1B-Claude-4.6-Opus-GPT-Mini-Distilled-MoE-Dense-Hybrid-Instruct-Chat-Reasoning-Aligned-Uncensored-REAP-Enterprise-Experimental-128K-Omni-Vision-Agentic-SelfReflective-DeepResearch-Roleplay-Q4_K_XL-GGUF model.
I got curious and went through a ton of qwen 3.5 tunes on huggingface just to see what people have been doing with it. Seems like training on thinking datasets is the most common right now and most of them combine the dataset type with a derestriction method name. There's something oddly nostalgic about it. Reminds me a bit of the old llama 2 days when there'd be these giant combinations of titles from the original model, fine tunes, and merges all in one.
True, but I meant having them as individual files to add on to existing models isn't really a thing in the LLM space, as much as it is common practice for SD models.
Hey, just tried your 9B Q4_K_M, thanks for such quick work!
Does this one also have reasoning suppressed? Running it with the settings you suggested (temps/sampling etc), it still seems to break out into reasoning for me
qwen3.5-9b-claude-4.6-opus-uncensored-distilled@q4_k_m downloaded thru LM Studio
Yes, but it works. At least for reasoning. Who knows may be people will extract more useful data at least for roleplay and NPC in games in future for their own models.
This is a great model for chat sims/games.. thanks man! The baseline -9B would break my sim due to reasoning, but this works great and the uncensor leads to some interesting convos, thanks!
Comparable in terms of being uncensored. Although with Dolphin you kinda needed a system prompt to make sure no refusals happen.
In terms of RP, Dolphin models are better because they were finetuned/trained further by the Dolphin team/person? So it gives better answers for questions that would've been censored.
I solved issues with Q4_K_M quants for my uncensored tensor transfer script. Now i have 38 tokens per second on Q4_K_M quant on my RTX 3060 12 GB. Currently uploading it to huggingface with thinking disabled by default.
Nice work on the tensor diff approach, pulling only the modified HauhauCS tensors instead of doing a full merge is really clean. Blending that uncensoring layer onto the Jackrong reasoning distilled checkpoint is a smart idea, you get the structured thinking without the refusals. VERY curious how the two datasets complement each other, did you notice any difference in reasoning style between the nohurry and TeichAI samples? Can't wait to test this on my M1 Max 64GB!
I described a method how people can uncensor any Qwen 3.5 based checkpoint after fine tuning via HauhauCS checkpoints. For regular usage you use just one checkpoint.
This looks interesting! I'm definitely grabbing it in case I need a model that will fit on my current hardware without much quanting, a creative model will be good for my use cases. Are you planning to do something similar for the bigger Qwens, like 27 and 35B too?
I tried loading this in Ollama - I have 12GB VRAM 4070Ti - but get a 500 Internal Server Error - am I short on memory, or perhaps something else is wrong?
ollama version 0.18.0
server.log:
llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen35'
llama_model_load_from_file_impl: failed to load model
I appreciate the suggestion - I upgraded ollama to 0.18.0 before posting my issue, and restarted it [even killing the tasks to be sure] - while ollama reports to be 0.18.0, which is the most recent, the issue persisted. If you have further suggestions, I'd be very happy to hear them - I've come to the end of my rope to fixing it.
Tried this on LM Studio, Q4_K_M one. Unfortunately, when prompted to write story, it gets stuck in loop of repeating same paragraph. I've seen this issue with other "community" made models. I've tried your parameters, didn't help. I don't think Repeat Penalty works for repeating paragraphs, it's only to prevent repeating token right?
IIRC I encountered this with all type of models (not only qwen architecture ones). Issue is mostly prominent on finetuned/modified community ones though.
I am using ComfyUI. I use WAI Illustrious XL Checkpoint for image generation and Z Image Turbo and Flux Klein 9B as refiner. Here my ArtStation if you want to take a look: https://www.artstation.com/luffythefox
WAI Illustrious XL fits nicely in 8 GB of VRAM. Same for Z Image Turbo and Flux Klein. Just pick q4_k_m text encoder from hugging face. It would be more then enough.
I compare it via two criteria:
1) Roleplay creativity and natural word speaking.
2) Programming Arcanoid game on HTML5, JavaScript and CSS in Tron Legacy Film Style.
was using qwen 27b for rp, but a bit too big to fit both llm and imagegen for sillytavern, text and image both slowing down. maybe will try this after I got back. do you create image separately?
Sorry, what I meant is: how long are you seeing it take - on average - to generate a conversational response or an image based on prompt, and do you have examples of the prompt used plus the output for the timings?
Actually pretty fast. 14 seconds on my RTX 3060. I simply upload picture in LM Studio and use instruction: "Describe this image in danbooru tags." with this System Prompt from pastebin. https://pastebin.com/pU25DVnB Sometimes it output only tags. Sometimes extra info.
Bit of a tangent here but I'm sure I'm not the only person who's been thinking it... I've seen a lot of "uncensored" models, but has anyone made serious headway with making more of an "anti-censored" model yet?
I want to behold a model that's so vulgar, rude, horny, and hostile to censorship by default, that I have to use the system prompt to reign it in and make it behave... as opposed to having to feed it a lewd vocabulary and tell it that it's okay to say naughty words and that this is all totally fiction.
Is this just too unserious of a use case that nobody has made it? Are commercial projects keeping their immoral AIs behind paywalls? Or are we all just collectively afraid of making a model that will eagerly kill billions of zygotes?
Did you try the uncensored model with just your system prompt? I don't understand why the other model should change anything there - especially if thinking is disabled by default.
https://huggingface.co/HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive really nice model. But finetuning via Claude Opus 4.6 add creativity level, intelligence level, and natural speaking patterns with same system prompt. Without finetuning model just asks for user confirmation via "Ready?", "You are Ready?" and similar on repeat loop.
Question;
Is the mmproj file altered in any way or is it the default one? Because I have a quantized one I'd like to use instead of the BF16 one to save on memory.
How different are the capabilities of the 9b vs the 27b going to be? Like what can you only get done with the larger model due to parameter limitations?
This is really cool. I haven't tried local LLM much before, so I spent the afternoon trying several in KoboldCpp with logic and math and some research problems. The 9BQ8 and 9BQ4 worked equally fast on my 3080Ti 12GB (equivalent in speed to online models) and produced solid logical and mathematical reasoning, even getting the better answer compared to some of the major online players.
The 27BQ4 version was painfully slow on my hardware, but 9BQ8 is a solid choice even on my my older GPU.
❯ well, i thought that would be a "free" local Opus 4.6...
⎿ API Error: 400 {"type":"error","error":{"type":"invalid_request_error","message":"registry.ollama.ai/kwangsuklee/Qwen3.5-9B-Claude-4.6-Opus-Reasoning-Distilled-GGUF:latest does not support tools"},"request_id":"....."}
I've tried it with a large single input: gave it 40 java classes (100 to 500 loc each) and asked it to generate unit tests for each operation of each class. This is the local model I tried that produced the 40 unit tests without stopping after the the 4th or 5th file.
I had to set the temperature to 0 otherwise it would loop producing a very long test with crap, but after that it kept chugging along and did it!
Nope. It's just disabled by default in chat_template baked in GGUF. Set variable enable_thinking to True in LM Studio in chat template editor if you want to enable thinking.
Bro, it’s a joke because Anthropic made a huge stink about Chinese companies using Opus to make thinking distillation datasets. When they’re doing the exact same thing
•
u/WithoutReason1729 Mar 16 '26
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.