Multi-Instance GPUs, with Kevin Klues and Pradeep Venkatachalam

NVIDIA and Google have teamed up to bring the new Multi-Instance GPU feature, launched with the NVIDIA A100, to GKE. We speak to Kevin Klues from NVIDIA and Pradeep Venkatachalam from Google Cloud on how and why people use GPUs, optimising instance shapes for machine learning, and why less is often more.

Do you have something cool to share? Some questions? Let us know:

• web: kubernetespodcast.com
• mail: [email protected]
• twitter: @kubernetespod

Chatter of the week

• Episode 64, with Sarah D'Angelo and Patrick Flynn
  • Catching up with Patrick in Episode 148
• Winthrop, Washington
• Blackdown Hills, Devon

News of the week

• Azure App Services now available for Azure Arc
  • Azure Arc and App Service blog posts
  • Other new AKS capbilities
  • Virtualization Review coverage
• ECS Anywhere made GA by press release
• AWS App Runner
• Integrating Google Cloud DNS with GKE
• Istio 1.10
• Terraform 1.0
• Grafana 8.0 and Tempo 1.0
• Argo Rollouts 1.0
• Kubesphere 3.1.0
• Cilium 1.10
• OpenSLO spec launched at SLOConf
  • Episode 147, with Brian Singer and Kit Merker
• Envoy GA on Windows
• Chaos Experimentation Framework for Envoy
• El Carro operator for Oracle Database from Google Cloud
• Moco operator for MySQL from Kintone
• PlanetScale GA
  • Episode 81, with Jiten Vaidya and Sugu Sougoumarane
• FoundationDB paper from ACM SIG MOD
• DockerCon announcements
• Coverage of Development Environments from The Register
• Deps: Open Source Insights project from Google
  • Graph for Kubernetes 1.0.0
  • Graph for Kubernetes 1.22.0-alpha.2
• Verifiable Supply Chain Metadata with Tekton Chains
• Kubernetes CVEs:
  • CVE-2021-25736
  • CVE-2021-25737
  • CVE-2021-25738
• runc CVE-2021-30465
• VS Code Plugin for Kubernetes CVE-2021-31938
• Steve Smith says "GitOps is a placebo" in a blog post and Twitter thread
  • Follow up from Vic Iglesias
  • GitOpsDays
• Styra raises $40m Series B round
  • Episode 101, with Tim Hinrichs and Torin Sandall
• Cloud Native community goes live with 10 shows on something called Twitch
• YouTube playlist for KubeCon EU 2021

Links from the interview

• Episode 92, with Pramod Ramarao
• Dogecoin
• Training and inference
• 12 things that prove Doom will run on literally anything
• "It runs Doom" subreddit
• CUDA
• vGPUs
• Multi-Instance GPUs
• GKE now supports multi-instance GPUs
• 7 core MacBook Air GPUs
• A100 GPU
• 16 A100 GPUs on a Google Cloud VM
• Running GPUs on GKE
  • Node taints for scheduling
• NVIDIA Container Toolkit
• GCP NVIDIA GPU device plugin
• Kubernetes NVIDIA device plugin
• GTC 2021 talks:
  • A Deep Dive on Supporting Multi-Instance GPUs in Containers and Kubernetes by Kevin and Pradeep
  • Gain Competitive Advantage using ML Ops: Kubeflow and NVIDIA Merlin and Google Cloud by Andrew Stein and Maulin Patel (Google) and Davide Onofrio (NVIDIA)
• Kevin's KubeCon talk and slides
• Kevin Klues on Twitter