Ryzen AI Max+ 395 Local LLM Benchmark: Qwen3.5 27B vs 35B

Quick Answer

On the tested CWWK CW-A9 platform, Qwen3.5-35B-A3B-AWQ delivered much higher serving throughput than Qwen3.5-27B-AWQ under the same 64GB Ryzen AI Max+ 395 test environment. At 128 input tokens and concurrency 32, the 35B-A3B model reached 100.85 TPS, while the 27B model reached 20.88 TPS. At 4K input and concurrency 32, the same comparison was 7.25 TPS versus 0.73 TPS.

That does not mean a 35B model is automatically faster than a 27B model. The key difference is architecture: Qwen3.5-27B is a dense 27B model, while Qwen3.5-35B-A3B is a sparse mixture-of-experts model with 35B total parameters and about 3B activated per token. The benchmark is useful because it shows how architecture, context length and concurrency can matter more than the number in the model name.

Key Takeaways

· The best result in this test was 100.85 aggregate TPS for Qwen3.5-35B-A3B-AWQ at 128-token input and concurrency 32.

· Qwen3.5-27B-AWQ scaled much less efficiently in this vLLM/ROCm test and its latency rose sharply as context length increased.

· For both models, longer prompts reduced throughput and increased TTFT; short-context workloads were the strongest fit.

· Aggregate TPS is a server throughput metric, not the same thing as single-user response speed.

· The test used 64GB memory, Ubuntu 24.04.1 LTS, ROCm 7.2.1 and vLLM 0.20.0, so different stacks can produce different results.

Test Environment

The benchmark was run on an AMD Ryzen AI Max+ 395 platform with Radeon 8060S graphics and 64GB memory. The software stack was Ubuntu 24.04.1 LTS, ROCm 7.2.1, PyTorch 2.10.0 and vLLM 0.20.0. Output length was fixed at 32 tokens while input length and concurrency were varied. The current CWWK CW-A9 supports up to 128GB LPDDR5X-8000, but these benchmark numbers come from the 64GB test system and should be read as test results, not as guaranteed performance for every configuration.

What Happened at Short Context Lengths?

Short prompts were where the 35B-A3B model scaled most clearly. With a 128-token input, increasing concurrency from 1 to 32 raised aggregate throughput from 18.21 TPS to 100.85 TPS. TTFT also increased, but the system was processing many more simultaneous requests. Pushing concurrency to 64 reduced aggregate throughput to 79.67 TPS and increased ITL substantially, so the highest concurrency was not automatically the best operating point.

Context Length Changed the Result More Than Concurrency

The strongest pattern was the cost of long prompts. For Qwen3.5-35B-A3B-AWQ at concurrency 32, aggregate throughput fell from 100.85 TPS at 128 input tokens to 48.36 TPS at 512, 28.52 TPS at 1K, 15.24 TPS at 2K, 7.25 TPS at 4K and 2.86 TPS at 8K. TTFT moved in the opposite direction, rising from about 4 seconds to about 79 seconds over the same range.

The dense 27B model degraded more sharply in the tested stack. At 4K input and concurrency 32 it reached only 0.73 TPS, with TTFT near 396 seconds. This is why a local AI deployment should not be sized around the maximum context window printed on a model card. A supported context length is not the same thing as an interactive context length on a particular machine.

Which Model Is the Better Fit on CW-A9?

For this specific vLLM/ROCm test, Qwen3.5-35B-A3B-AWQ is the stronger serving choice. It provided materially better throughput and better token-to-token latency at the tested input sizes. Qwen3.5-27B remains a valid dense-model reference, but its performance here shows how a dense architecture can place a much heavier compute and memory-bandwidth burden on the platform.

A practical CW-A9 workload is therefore short- to medium-context local inference: interactive assistants, local code completion, short document processing, private APIs and batched generation. For long documents, chunking and retrieval are usually more efficient than sending the entire document as one prompt. The existing CW-A9 local AI overview explains the product architecture and why its high-capacity unified memory is useful for model residency.

How to Read These Numbers Correctly

· Use TTFT to judge how long a user waits before the first streamed token.

· Use ITL to judge how smooth generation feels after output begins.

· Use TPS to judge aggregate serving throughput, especially under concurrent requests.

· Compare results only when model, quantization, input/output length, concurrency and software stack are defined.

· Do not treat peak aggregate TPS as the expected experience for one interactive user.

FAQ

Did the benchmark use 128GB memory?

No. The reported comparison was run on a 64GB test system. The retail CW-A9 supports up to 128GB, which increases memory headroom but does not automatically double inference throughput.

Why can the 35B model be faster than the 27B model?

Qwen3.5-35B-A3B is a sparse MoE model with about 3B activated parameters per token, while Qwen3.5-27B is a dense 27B model. Total parameter count alone therefore does not predict inference cost.

Is 100.85 TPS a single-user generation speed?

No. It is aggregate throughput at concurrency 32 in the tested serving configuration. Interactive users should also examine TTFT and ITL.

Does CW-A9 make sense for very long context inference?

It can run long-context workloads, but the test shows a steep latency and throughput penalty as input length grows. Chunking, RAG or tighter prompt control is usually a better fit for responsive local use.

Related CWWK Resources

· Local LLM Hardware Requirements

· TTFT vs TPS vs ITL: Local LLM Benchmark Metrics

· CWWK AMD Series

External References

· AMD Ryzen AI Max+ 395 specifications

· Qwen3.5-27B model card

· Qwen3.5-35B-A3B model card

· vLLM Benchmark CLI and metric definitions