ModelVault Senior Software Engineer

ModelVault is building the world’s first fully-integrated, on-prem AI appliance — a plug-and-play AI server preloaded with software, models, and tools that let businesses (and eventually consumers) run powerful AI models locally, privately, and securely.

We're looking for a Senior Software Engineer to help us build the core developer experience and control plane for our on-prem systems — everything from containerized runtime orchestration to local APIs, CLI tools, OTA update flows, and lightweight UIs. You'll be one of the first engineers on the team and play a foundational role in shaping our system architecture, tooling, and technical culture.

🚀 What You’ll Do

• Design and build the software layer that makes ModelVault “just work” — including:
  • Local APIs and system services
  • Developer-facing CLI tools
  • Simple admin web UI or dashboards
  • OTA update infrastructure
• Wrap system-level functionality into intuitive interfaces for local users (e.g., “start model,” “view logs,” “update software”)
• Integrate and manage containerized LLMs and vision models (e.g. Llama 3, Mistral, SDXL) for local inference
• Collaborate with our systems and hardware engineers to optimize performance across the full stack — from GPU runtimes to dev UX
• Participate in architecture decisions related to system orchestration, hardware/software coordination, and secure update delivery
• Interface directly with early users to gather feedback, debug edge cases, and iterate quickly
• Contribute to hiring and scaling the engineering team over the next 12 months.

🧠 You Might Be a Fit If You

• Have 5+ years of software engineering experience — ideally in devtools, infra, or platform teams
• Are comfortable working close to the system — building tools that wrap around Docker, shell scripts, or system services
• Enjoy working across the stack — from lightweight frontend dashboards to backend APIs to scripting hardware operations
• Have experience building interfaces for technical users (CLIs, admin panels, config systems)
• Have worked with containerized environments and understand Docker, Compose, systemd, etc.
• Like solving ambiguous technical problems with minimal abstraction — and high leverage

⭐️ Bonus Points

• Built tooling or platforms for AI workloads (e.g. inference runners, LLM wrappers, logging for local models)
• Experience with GPU systems, Nvidia drivers, CUDA, or model performance tuning
• Familiarity with update mechanisms (auto-updaters, patch delivery, rollback safety)
• Past experience with embedded systems, local-first products, or edge computing

⚙️ Our Tech Stack (Currently)

• Python, FastAPI, Shell scripting, Docker / Compose
• Click or Typer for CLI
• React / Next.js (if building dashboards)
• OS: Ubuntu
• LLM tooling: llama.cpp, vLLM, Ollama, GGUF, HuggingFace
• Remote: Tailscale, SSH, or light VPN-based access

🌍 Why Join ModelVault?

• Help define a new category: secure, local AI for everyone — starting with businesses, expanding to the world
• Work on real, shipped hardware + software that users will touch
• Be part of the founding engineering team with high autonomy and deep ownership
• Remote-first, async-friendly culture with deep focus and high velocity