Feature Post

Top

How my machine nearly died running ai and why im building a personal AI server instead

,

My machine encountered near-death experience when I tried running AI models locally, some time ago.

Since then I’ve been thinking about setting up my own AI rig.

Solving problem

Here’s the setup I’m planning:

  • CPU: AMD Ryzen 9 (5000 series or newer), enough cores + solid performance
  • GPU: 
    • RTX 4070 → 12–16GB VRAM. Fine for 7B–14B models
    • RTX 4090 → 24GB VRAM. Lets you run 30B+ models without hacks
  • RAM: 64GB. 32GB is too tight, 128GB is overkill
  • Storage: At least 1TB NVMe SSD. Models are big, and they load faster on NVMe.
  • OS: Ubuntu. Works better for AI than Windows
  • Stack: Ollama, VS Code, DeepSeek, Qwen

Btw, my machine: windows 11, Intel ~1690 Mhz, 12GB mem, +500 NVMe storage (yeah I know!)

 

Note for weak-hearted

Most GPUs are priced like they ($NVDA) are collecting kidneys on the side, and while Alibaba pretends to be merciful until the shipping cost shows up demanding whatever parts of your body are left.

 

So, why not just stick with OpenAI, Claude, etc, you ask?

-No usage fees once hardware is paid for
-Data stays private
-Faster responses (no API lag)
-Full control over models and updates
-No vendor lock-in or sudden policy shifts
-You can push context window limits as far as your hardware allows (default 16, push to 32)

 

Most importantly

Most (OpenAI, Anthoric, Gemini/etc) are propriety technologies - so no can do; and we are left with open source, like Qwen, Llama, Deepseek, etc.

Plus, this is just to playround (nothing serious). 


Why not a personal AI box?

After all, half of Dubai (imo) already have some sort of NAS at home (storing family photos, videos, or backups - forget iCloud), why not an AI box? A personal (+ private), always-on AI server that kids & family can also use.


Custom commercial AI boxes

Btw, @George Hotz (Tiny Corp) offers "TinyBox": a compact AI server specially designed for local model training and inference. The TinyBox comes in two versions:

  • Red (~USD 15000): 6 GPUs of AMD Radeon RX 7900 XTX (~$15,000)
  • Green (~$25,000): six NVIDIA RTX 4090 GPUs


Basically a pre-built plug/play AI server for people who want local power without building everything themselves. I have not tried, dyor please. (not sponsored)


Anyway

I want something portable, network-connected, and strong enough that I don’t think twice before running a large model. The 4070 setup is “good enough.” The 4090 setup is “don’t worry about it for a while.” - but lets see.

Either way my laptop deserves to live, a little while longer.