The Californer - eRacks Publishes a Private-AI Sizing Guide: 70B-Class Models Run On-Premise from ,995

Trending...

Heritage at South Brunswick Introduces New Ferndale Floorplan: The Largest Single-Family Home Design in the Community - 277
Attention Business Robots, Marketing Robots, News Robots (and humans too): This Marketing Math can help all kinds of businesses make a lot more money!
First Partner Siebel Newsom, Governor Newsom, and #HalfTheStory launch California's first Teen Tech Council

Plain-English guidance on how much GPU memory and system RAM each model size needs, when self-hosting beats cloud APIs, and how to keep AI data entirely in-house.

FREMONT, Calif. - Californer -- eRacks Open Source Systems today published "Private AI, Sized & Priced," a vendor-neutral buyer's guide for organizations that want to run large language models on their own hardware rather than through a cloud API, and announced that a 70-billion-parameter model now runs on-premise on its AILSA server starting at $5,995.

The guide answers the question eRacks hears most from AI buyers: how big a machine do I actually need? AI servers are priced largely by GPU memory (VRAM). The guide gives a model-size-to-VRAM table (a 70B model in 4-bit quantization needs roughly 42 GB, about 80 GB in 8-bit), and adds the part most buyers miss: system RAM should be 1.5 to 2 times the VRAM, and a multi-GPU server needs a server-class CPU for the PCIe lanes.

It also lays out the cost crossover where owning beats renting. A 30-person team paying $30 per user per month for a cloud AI service spends about $10,800 a year, every year, with all prompts transiting a third party's servers. An on-premise server covers the same everyday inference and pays for itself in under a year. For teams handling protected health information, attorney-client material, government data, or unpublished research, the calculus is simpler still: the data cannot leave their control, and a private server is the only option.

More on The Californer

eRacks AI servers run entirely on open-source software, with Ubuntu Linux plus Ollama, Open WebUI, vLLM, and PyTorch pre-installed and tested, so staff reach the AI from a browser on day one, with no per-seat fees and no data leaving the building.

The guide is published at https://eracks.com/guides/private-ai-sizing/. eRacks has built custom open-source servers since 1999 and will size a private AI server to a customer's exact models and user count at no charge.

About eRacks Open Source Systems

eRacks Open Source Systems is an open-source server and storage specialist founded in 1999, headquartered in Fremont, CA, building rackmount servers, NAS, and AI inference servers configured to customer requirements.

Media Contact

Joseph Wolff eRacks Open Source Systems joe@eracks.com https://eracks.com