You are viewing a single comment's thread:
RE: AI server improvements
The jump from 110 to 300 tokens a second after the vLLM tweaks is the part that really sells it, that is a serious leap for the same cards.
0.00000000 BEE
0 comments
You are viewing a single comment's thread:
The jump from 110 to 300 tokens a second after the vLLM tweaks is the part that really sells it, that is a serious leap for the same cards.