You are viewing a single comment's thread:
RE: Run even larger AI models locally with LM Studio
Mac Studio and even Mac Minis are very popular option for LLM due to how unified memory works. Nowhere can you get ~188 VRAM for less than the cost of even a single A100 40G.
0.00000122 BEE
I'm getting 23 tokens per second using the 5 bit Mixtal 2.7 model.
View more