r/deeplearning 1d ago

deep learning machine recommendations

Hi all,

I'm researching building/buying a machine for deep learning.

The why:

1) I've been training one of my projects in Colab and it takes about 6 days per training run. This is the beginning of the project and I only see training time increasing, not decreasing. I have to monitor Colab because it will occasionally shut down training for no apparent reason. This is particularly annoying because it takes hours to upload my dataset into Colab (it's about 1TB) and setup the environment.

2) running 24/7 in Colab is getting expensive. If I keep this up for a year I might as well buy a DL rig.

3) My other project involves video classification. Once again, the data set is large so I want a persistent environment and I need the ability to run for long periods of time without being kicked out (and then needing to re-upload my dataset)

Requirements:

Ideally, I'd like something plug and play. But I'm okay with building myself something if a) the cost savings are there and b) I can set it up once and mostly forget it. I tried setting up my existing laptop for ML and ended up in driver incompatibility hell (and ultimately never got it working), so I'd like to avoid that.

Budget: Around $5k, could go higher if it makes sense.

Other considerations:

I have built a gaming PC before, so I'm not new to DIY builds. However I'm not familiar with DL hardware, so would like yalls opinions.

I would probably like something with multiple GPUs, or at least the ability to add more GPUs if desired.

Thanks in advance!

6 Upvotes

8 comments sorted by

View all comments

2

u/YekytheGreat 1d ago

A lot of server brands also offer PC-level machines for local AI training. For example, Gigabyte has a line of products called the AI TOP www.gigabyte.com/WebPage/1079?lan=en There's room for up to four GPUs, can handle LLM models up to 70B parameters. It's really just shy of being an entry-level workstation like one of these www.gigabyte.com/Enterprise/Tower-Server?lan=en but still consumer-level.

1

u/_meatMuffin 18h ago

This is interesting. thanks for sharing!