You are viewing a single comment's thread:

RE: AI server improvements

300 tokens a second decode and 700 with speculative decoding is wild for a home rig. Four RTX 6000 Pros on a Xeon board is serious hardware, and the jump from 110 to 300 just from engine tweaks says a lot about the software side. The clicky PSU would drive me mad though, that 240v power station sounds like the right fix.

0.00000000 BEE
0 comments