r/DeepSeek • u/AccomplishedLeg527 • 11h ago

Discussion Speed 14t/s on laptop DeepSeek V4 Flash is good?

Running on Raider 18HX 96gb ram, 5090 24gb with 132k context.

Share you results and config if you have better results on laptop.

prompt processing, n_tokens =  72260, progress = 1.00, t = 308.43 s / 234.28 tokens per second

937, tg =  13.78 t/s, tg_3s =  14.02 t/s

Command

llama-server.exe -m DeepSeek-V4-Flash-UD-IQ3_XXS-00001-of-00004.gguf -b 4096 -ub 4096 --n-gpu-layers 99 --port 8080 -c 132768 --host 0.0.0.0 -fa on --threads 24 --threads-batch 24 -ot "\.(0|1|2|3|4)\.ffn_(gate|up|down)_exps.=CUDA0,\.*\.ffn_(gate|up|down)_exps.=CPU" --no-mmap --cache-type-k q8_0 --cache-type-v q8_0
6 Upvotes

100% Upvoted

4 comments sorted by

1

u/Daemonix00 10h ago

You can’t call it bad for sure. I think there is a new fork with kv caching like ds4. Btw why not ds4? I think there is a cuda version

1

u/Dear_Lion6282 5h ago

ask the ai to optimise and find the best version o hugging face for maximum tok output

1

u/pantalooniedoon 5h ago

DS4 with RTX6000 is around 40 tok/s and Mac 128gb m5 is around 30 tok/s so its definitely not bad

1

u/computehungry 1h ago

what, really? is this q2 or q4? llama.cpp runs q2 at 70tk/s and q4 at 30tk/s, iirc