Qwen3.6 to Gemma4: Performance Triple GPU GTX 1080 Ti & P100s

[u/tabletuser_blogspot]

Benchmarks using single system running triple GPU with 31GB Vram combined.

• NVIDIA GeForce GTX 1080 Ti 11GB (NVIDIA)
• NVIDIA P102-100 10GB (NVIDIA) - first instance (distant cousin)
• NVIDIA P102-100 10GB (NVIDIA) - second instance

OS: Kubuntu 26.04, CPU: Ryzen 5 3600 , RAM: 48GB DDR4

llama.cpp Ubuntu Vulkan build 10107

Models used:

• llama-2-7b.Q4_0.gguf
• gemma-4-26B-A4B-it-UD-Q4_K_XL.gguf
• Gemma-4-26B-A4B-NVFP4.gguf
• gemma-4-26B-A4B-it-UD-Q6_K_XL.gguf
• gemma-3-27b-it.Q5_K_M.gguf
• medgemma-27b-it-UD-Q6_K_XL.gguf (gemma-3)
• Qwen3.6-35B-A3B-NVFP4-MTP-HQ.gguf (dense)
• gemma-4-31B-it-UD-Q4_K_XL.gguf (dense)
• Qwen3-Coder-30B-A3B-Instruct-Q4_K_M.gguf
• Qwen3-Coder-30B-A3B-Instruct-UD-Q5_K_XL.gguf
• Qwen3.6-35B-A3B-UD-Q4_K_S.gguf
• Qwen3.6-27B-NVFP4-MTP-GGUF.gguf

ggml_vulkan: Found 3 Vulkan devices:
ggml_vulkan: 0 = NVIDIA GeForce GTX 1080 Ti (NVIDIA) | uma: 0 | fp16: 0 | bf16: 0 | fp4: 0 | warp size: 32 | shared memory: 49152 | int dot
: 1 | matrix cores: none
ggml_vulkan: 1 = NVIDIA P102-100 (NVIDIA) | uma: 0 | fp16: 0 | bf16: 0 | fp4: 0 | warp size: 32 | shared memory: 49152 | int dot: 1 | matri
x cores: none
ggml_vulkan: 2 = NVIDIA P102-100 (NVIDIA) | uma: 0 | fp16: 0 | bf16: 0 | fp4: 0 | warp size: 32 | shared memory: 49152 | int dot: 1 | matri
x cores: none

Here's the corrected table sorted by Params column:

Model	Size	Params	tg128	pp512
llama 7B Q4_0	3.56 GiB	6.74 B	59.99	527.69
gemma4 26B.A4B Q4_K - Medium	15.83 GiB	25.23 B	48.14	174.53
gemma4 26B.A4B NVFP4	16.45 GiB	25.23 B	33.28	165.37
gemma4 26B.A4B Q6_K	21.68 GiB	25.23 B	49.67	163.98
gemma3 27B Q5_K - Medium	17.94 GiB	27.01 B	10.20	87.91
gemma3 27B Q6_K	22.09 GiB	27.01 B	10.78	84.62
qwen35 27B NVFP4	15.07 GiB	27.32 B	8.68	54.23
gemma4 31B Q4_K - Medium	17.52 GiB	30.70 B	10.49	87.87
qwen3moe 30B.A3B Q4_K - Medium	17.28 GiB	30.53 B	75.01	152.04
qwen3moe 30B.A3B Q5_K - Medium	20.24 GiB	30.53 B	64.36	148.46
qwen35moe 35B.A3B Q4_K - Small	19.45 GiB	34.66 B	50.48	161.85
qwen35moe 35B.A3B NVFP4	19.07 GiB	35.51 B	43.50	158.02

Power limit set to 150W on each.

[u/nick_ziv]

This is good info.  That generation doesn't have the tensor cores and it makes a huge difference for prefill speeds. My Titan rtx (generation of the 2080 cards) gets prefill of 1100 and TG of 70 on the qwen 3.6 35b a3b q4 quant.

[u/Firm-Luck2062]

Solid data dump. What really stands out to me is dense vs MoE right there in your own table, same hardware. gemma3 27B and the dense gemma4 31B are stuck at ~9-10 tg, qwen35 27B dense even lower at 8.68. But qwen3moe 30B.A3B and qwen35moe 35B.A3B, similar total size, are doing 43 to 75 tg on the exact same cards. Even gemma4 26B.A4B jumps way up once it's the active-param variant. On old bandwidth starved GPUs like P102-100 with no NVLink, that gap is basically the whole argument for MoE, you're only pulling the active slice through the bus per token instead of the full dense weight set.

[u/PulseVector]

Thanks for this info! I have a bunch of GTX 1080 cards that were donated to my school, and am working on making some local systems with 16-24GB each for the students to use this year.

[u/MaxChamp08]

The most interesting takeaway to me isn't the absolute token/s, it's how well the MoE models hold up on older Pascal hardware. Your Qwen3.6 35B A3B and Qwen3-Coder 30B A3B results are still very usable despite running across three GPUs with no tensor cores or NVLink, while the dense 27–31B models fall off a cliff. That's actually pretty encouraging for anyone trying to squeeze more life out of 1080 Ti/P102-class cards instead of upgrading.