实际经验告诉你：4*5090 部署 Qwen3.6 27B NVFP4 够了，我们 vllm 部署并跑了两个月了。
我们部门 100 号人，实际在用这个模型的大概也就是三四十号人的规模，日常也就 4~6 个并发请求，缓存命中率最低 75%左右，有一段时间维持在 94%左右。
除了确实偶尔会卡顿一下（来了个大请求 prefill 占了资源），日常还是很流畅的，多数时候每个请求基本都能维持在 40~75 tok/s 。
当然，如果你们是要用来疯狂 vibe coding 那肯定就不行了，不过这个模型也达到可以随意 vibe coding 的能力。
放几条日志上来看看，不是瞎说（贴图太麻烦了直接贴文本）：
```
(APIServer pid=1) INFO 08-14 07:32:44 [
loggers.py:310] 2 Engines Aggregated: Avg prompt throughput: 6927.3 tokens/s, Avg generation throughput: 211.5 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 17.3%, Prefix cache hit rate: 75.5%, MM cache hit rate: 0.0%
(APIServer pid=1) INFO 08-14 07:32:44 [
metrics.py:120] SpecDecoding metrics: Mean acceptance length: 2.77, Accepted throughput: 135.20 tokens/s, Drafted throughput: 152.80 tokens/s, Accepted: 1352 tokens, Drafted: 1528 tokens, Per-position acceptance rate: 0.931, 0.839, Avg Draft acceptance rate: 88.5%
(APIServer pid=1) INFO: xxxxx - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=1) INFO 08-14 07:32:54 [
loggers.py:310] 2 Engines Aggregated: Avg prompt throughput: 3843.2 tokens/s, Avg generation throughput: 382.0 tokens/s, Running: 5 reqs, Waiting: 0 reqs, GPU KV cache usage: 8.8%, Prefix cache hit rate: 75.7%, MM cache hit rate: 0.0%
(APIServer pid=1) INFO 08-14 07:32:54 [
metrics.py:120] SpecDecoding metrics: Mean acceptance length: 2.73, Accepted throughput: 241.65 tokens/s, Drafted throughput: 280.15 tokens/s, Accepted: 2417 tokens, Drafted: 2802 tokens, Per-position acceptance rate: 0.916, 0.809, Avg Draft acceptance rate: 86.3%
(APIServer pid=1) INFO: xxxxx - "HEAD / HTTP/1.1" 404 Not Found
(APIServer pid=1) INFO: xxxxx - "HEAD / HTTP/1.1" 404 Not Found
(APIServer pid=1) INFO: xxxxx - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=1) INFO: xxxxx - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=1) INFO 08-14 07:33:04 [
loggers.py:310] 2 Engines Aggregated: Avg prompt throughput: 1300.6 tokens/s, Avg generation throughput: 332.5 tokens/s, Running: 2 reqs, Waiting: 0 reqs, GPU KV cache usage: 4.8%, Prefix cache hit rate: 75.7%, MM cache hit rate: 0.0%
(APIServer pid=1) INFO 08-14 07:33:04 [
metrics.py:120] SpecDecoding metrics: Mean acceptance length: 2.75, Accepted throughput: 211.70 tokens/s, Drafted throughput: 242.61 tokens/s, Accepted: 2117 tokens, Drafted: 2426 tokens, Per-position acceptance rate: 0.913, 0.832, Avg Draft acceptance rate: 87.3%
(APIServer pid=1) INFO: xxxxx - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=1) INFO: xxxxx - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=1) INFO: xxxxx - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=1) INFO 08-14 07:33:14 [
loggers.py:310] 2 Engines Aggregated: Avg prompt throughput: 4531.0 tokens/s, Avg generation throughput: 204.7 tokens/s, Running: 4 reqs, Waiting: 0 reqs, GPU KV cache usage: 11.4%, Prefix cache hit rate: 75.6%, MM cache hit rate: 0.0%
(APIServer pid=1) INFO 08-14 07:33:14 [
metrics.py:120] SpecDecoding metrics: Mean acceptance length: 2.90, Accepted throughput: 134.06 tokens/s, Drafted throughput: 141.16 tokens/s, Accepted: 1341 tokens, Drafted: 1412 tokens, Per-position acceptance rate: 0.970, 0.929, Avg Draft acceptance rate: 95.0%
(APIServer pid=1) INFO: xxxxx - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=1) INFO 08-14 07:33:24 [
loggers.py:310] 2 Engines Aggregated: Avg prompt throughput: 10786.8 tokens/s, Avg generation throughput: 173.7 tokens/s, Running: 7 reqs, Waiting: 0 reqs, GPU KV cache usage: 16.6%, Prefix cache hit rate: 75.6%, MM cache hit rate: 0.0%
(APIServer pid=1) INFO 08-14 07:33:24 [
metrics.py:120] SpecDecoding metrics: Mean acceptance length: 2.83, Accepted throughput: 111.99 tokens/s, Drafted throughput: 122.59 tokens/s, Accepted: 1120 tokens, Drafted: 1226 tokens, Per-position acceptance rate: 0.949, 0.878, Avg Draft acceptance rate: 91.4%
```
Select Language