GPU Cluster Administrator

The ideal candidate will be responsible for working cross-functionally with data science, AI, and product teams to understand their compute needs and ensure reliable, scalable access to NVIDIA H200 GPU resources. To be effective in this position, you must feel comfortable owning the entire GPU infrastructure workflow—from cluster design, deployment, and monitoring, to user management, scheduling, and optimization. The GPU H200 Cluster Administrator is responsible for developing and delivering robust infrastructure solutions that enable AI workloads to run efficiently and cost-effectively across the organization.

[Responsibilities]

• Design, deploy, and administer on-premise or cloud-hosted GPU clusters based on NVIDIA H200 (and related GPU families).
• Configure and manage Kubernetes (K8s) clusters for GPU workloads, including node pools, taints/tolerations, and GPU-aware schedulers.
• Implement and optimize load balancing, auto-scaling, and resource quotas to support multiple concurrent users and AI projects.
• Manage user accounts and access control (RBAC), namespaces, and multi-tenant isolation for data scientists and engineers.
• Maintain GPU software stack: CUDA, cuDNN, NCCL, drivers, container runtimes, and relevant libraries/frameworks.
• Develop standardized, reproducible environments using Docker/OCI images and Helm charts for ML and deep learning workloads.
• Monitor GPU utilization, pod performance, and system health using tools such as Prometheus, Grafana, or equivalent observability stacks.
• Troubleshoot performance bottlenecks, failed jobs, and scheduling issues across Kubernetes and GPU nodes.
• Collaborate with data science teams to support training, inference, and experimentation workflows (e.g., PyTorch, TensorFlow).
• Implement job scheduling policies (fair-share, priority queues, preemption) to balance throughput and responsiveness.
• Ensure cluster security and compliance, including network policies, image scanning, secrets management, and audit logging.
• Work with IT/DevOps and cloud teams to integrate GPU clusters with existing CI/CD, storage, and identity platforms.
• Document best practices, provide training and onboarding materials for users, and serve as the primary point of contact for GPU infrastructure questions.

[Requirements]

• B.S. with 5+ years of industry experience, M.S. with 3+ years, or Ph.D. in Computer Science, Computer Engineering, or a related field with a focus on infrastructure, distributed systems, or MLOps.
• Hands-on experience administering GPU clusters (NVIDIA H100/H200/A100 or similar) in production environments.
• Strong experience with
  Kubernetes
  for production workloads, including cluster setup, networking, storage, and security.
• Solid understanding of GPU scheduling and resource management in K8s (e.g., device plugins, GPU operators, custom schedulers).
• Proficiency with Linux system administration (networking, storage, services, security hardening).
• Experience managing multi-user environments: identity and access management (IAM), RBAC, quotas, and namespace isolation.
• Strong scripting and automation skills (Python, Bash, or similar) to build tools for cluster operations and user workflows.
• Familiarity with container technologies (Docker, containerd) and image lifecycle management.
• Experience with monitoring and logging stacks (Prometheus, Grafana, ELK/EFK, or similar) for infrastructure observability.
• Understanding of common ML/DL frameworks (PyTorch, TensorFlow) and their resource requirements on GPU clusters.
• Experience with cloud platforms (Azure, AWS, or GCP) for GPU-based workloads is a plus.
• Knowledge of CI/CD and infrastructure-as-code tools (e.g., Terraform, Ansible, ArgoCD, Jenkins/GitHub Actions) is a plus.
• Ability to work in a fast-paced, experimental environment where iteration and adaptation are key.
• A strong problem-solver who thrives when thrown into complex systems, can "figure things out as you go," and communicates clearly with both engineers and non-technical stakeholders.