Wow! That's a beast of an AI server, and a huge capital outlay at current prices.
My AI Home server is called "Poe" and only has one 32 Gb AMD r9700 AI Pro GPU. That's enough…
300 tokens a second decode and 700 with speculative decoding is wild for a home rig. Four RTX 6000 Pros on a Xeon board is serious hardware, and the jump from 110 to 300 just…