You are viewing a single comment's thread:
RE: The future of AI – self hosted open sourced models?
i think i could setup an ai at home very easy but as far as i know i have no use for it at the moment .
the comparing you make are hghard to judge without knowing the hardware specs of your old desktop , it could be an old 486-dx100 cpu with 8mb memory 😁 , but i guess it's something more modern and maybe no longer windows 11 compatible ?
What hardware would you need to run it at home without to many problems ? will it run on a VM server ? does it require a dedicated GPU ? i know i have played with Amuse 3.0 on my pc to make graphics and it works fine with 16gb Vram . But do all AI servers need Vram ?
0.00000848 BEE
You can actually run a very small gemma 4 model with about 8 GB vram on your computer but I tried that with my old one and it made a lot of noise and was very slow (I have only 16 GB ram and windows uses a big part of it and no dedicated GPU. So it didn't work. I researched a bit and the best solutions are the following to host real big models locally:
all three models are actually quite low in power consumption and can run pretty big models efficiently. Their build up is similar with shared RAM.
I think with 16gb Vram you could actually run gemma 4:e2b or even gemma 4:e4b. You would need to install ollama and download it over it. Then you could execute it directly in ollama and use it as local llm. Probably it wouldn't be enough to run agents but enough for a local llm. You would need to check if your computer can deal with the workload :-)
ahh it even seems to run on linux , let's make a ubuntu VM and play with it :)