From “Inference Box” to Dev Rig: What NVIDIA DGX Spark Actually Is | Ep 2 cover art

From “Inference Box” to Dev Rig: What NVIDIA DGX Spark Actually Is | Ep 2

From “Inference Box” to Dev Rig: What NVIDIA DGX Spark Actually Is | Ep 2

Listen for free

View show details

LIMITED TIME OFFER | Get 2 Months for ₹5/month

About this listen

 Everyone keeps calling NVIDIA DGX Spark an “inference box”… but in practice it behaves more like a dev rig.In Ep 2 of Domesticating AI, we break down what Spark is actually good for (AI development + fine-tuning) vs what it isn’t (a magical drop-in inference server). We also dig into why unified memory changes the local-AI experience, the “gateway stack” (Ollama + Open WebUI), when you outgrow turnkey UIs, and how homelab economics + networking decisions shape what you should run at home.In this episodeTraining vs inference (and why “inference server” gets misused)Unified memory: what it changes for model loading + workflowsOllama + Open WebUI as the fastest on-ramp for local AIFine-tuning workflows (QLoRA/Unsloth-style) and where Spark shinesHomelab reality: Docker “recipes,” troubleshooting, and collaborationSafer remote access: TailscaleCloud vs home economics (when cloud is cheaper… and when it explodes)NVIDIA / DGX SparkDGX Spark: https://www.nvidia.com/en-us/products/workstations/dgx-spark/Build hub / recipes: https://build.nvidia.com/sparkNIM on Spark playbook: https://build.nvidia.com/spark/nim-llmLocal AI runners + UIsOllama: https://ollama.com/Open WebUI (GitHub): https://github.com/open-webui/open-webuiOpen WebUI docs: https://docs.openwebui.com/llama.cpp: https://github.com/ggml-org/llama.cppLM Studio: https://lmstudio.ai/vLLM: https://github.com/vllm-project/vllmJan: https://jan.ai/Fine-tuning + workflowsUnsloth: https://github.com/unslothai/unslothImage generation tools (mentioned)ComfyUI: https://github.com/Comfy-Org/ComfyUIAUTOMATIC1111 SD WebUI: https://github.com/AUTOMATIC1111/stable-diffusion-webuiNetworking / Remote accessTailscale: https://tailscale.com/Cloud GPU alternatives (mentioned)Runpod pricing: https://www.runpod.io/pricingModal pricing: https://modal.com/pricingMiriah Peterson (Host): Miriah Peterson is a software engineer, Go educator, and community builder focused on production-first AI—treating LLM systems like real software with real users. She runs SoyPete Tech (streams + writing + open-source projects) and stays active in the Utah dev community through meetups and events, with a practical focus on shipping local and cloud AI systems.Connect:SoyPete Tech (YouTube): https://www.youtube.com/@SoyPete_TechSoyPete Tech (Substack): https://soypetetech.substack.com/LinkedIn: https://www.linkedin.com/in/miriah-peterson-35649b5b/Matt Sharp (Host): Matt Sharp is an AI Engineer and Strategist for a tech consulting firm and co-author of LLMs in Production. He’s a recovering data scientist and MLOps expert with 10+ years of experience operationalizing ML systems in production. Matt also teaches a graduate-level MLOps-in-production course at Utah State University as an adjunct professor. You can find him on Substack (Data Pioneer), LinkedIn, and on his other podcast, the Learning Curve.Connect:Data Pioneer (Substack): https://thedatapioneer.substack.com/Chris Brousseau (Host): Chris Brousseau is a linguist by training and an NLP practitioner by trade, with a career spanning linguistically informed NLP, modern LLM systems, and MLOps practices. He’s co-author of LLMs in Production and is currently VP of AI at VEOX. You can find him as IMJONEZZ (two Z’s) on YouTube, GitHub, and on LinkedIn.Connect:YouTube (IMJONEZZ): https://www.youtube.com/channel/UCPtkaw_x97yP4WevW7axk0gLinkedIn: https://www.linkedin.com/in/chris-brousseau/en📘 LLMs in Production (Matt Sharp & Chris Brousseau): https://www.manning.com/books/llms-in-productionLinks & ResourcesHosts
No reviews yet