You are viewing a single comment's thread:
RE: AI server improvements
Four cards on real server lanes, and 300 tokens a second on decode. What do you run on it most days, agents or just one big model?
0.00000000 BEE
You are viewing a single comment's thread:
Four cards on real server lanes, and 300 tokens a second on decode. What do you run on it most days, agents or just one big model?
large model
One big model is simpler than juggling agents, fair enough. Hope those four cards run it smooth.