21 GPU Development in Tercen Studio

Validated end-to-end 2026-08-26 (GCP T4, driver 580, toolkit 1.20): a gpu-capability operator installed through Tercen ran with --gpus=all added automatically by the worker, and saw the GPU from inside its container.

21.1 The modernized studio (2026-08+): the GPU overlay

Requirements on the host: an NVIDIA GPU, the NVIDIA driver, and nvidia-container-toolkit configured for docker (sudo nvidia-ctk runtime configure --runtime=docker).

Start the studio with the GPU overlay:

cd tercen_studio
docker compose -f docker-compose.yaml -f docker-compose.gpu.yaml up -d

That overlay (in the tercen_studio repo) reserves the GPU for the tercen-worker service and runs a boot-time setup script inside it which installs the NVIDIA container toolkit, generates the CDI spec, and registers the nvidia runtime with podman via a containers.conf.d drop-in. With the runtime registered, ExeRunner.hasGpuCapability() returns true and the worker adds --gpus=all to every operator whose operator.json declares:

"capabilities": ["gpu"]

To verify a setup, install and run tercen/gpu_smoke_operator — it prints the GPUs visible inside the operator container (GPU 0: Tesla T4 …) into its output and exits.

On production (GKE), GPU operators take a different, independent path: Kubernetes Jobs with nvidia.com/gpu limits (tercen.allow.k8s.job).

21.2 The pre-2026-08 studio (legacy)

The instructions below target the old studio architecture (dind-based). To follow them, check out revision 72e3bdc of tercen_studio.

GPU support is available in Tercen Studio. Follow the steps below to enable and test GPU access in your local setup.

21.3 Setup Instructions

Pull the latest version of the Tercen Studio repository:
https://github.com/tercen/tercen_studio

Configure Docker for GPU support. Modify the docker-compose.yaml file as described below:

Comment out the default runtime-docker image.

#image: docker:24.0.6-dind-alpine3.18

Uncomment GPU-related configuration.

Under the dind service:

image: tercen/nvidia-dind:12.1.0-runtime-ubuntu22.04
deploy:
  resources:
    reservations:
      devices:
        - driver: nvidia
          count: 1
          capabilities: [ gpu ]

Under the VS Code service code-server:

deploy:
  resources:
    reservations:
      devices:
        - driver: nvidia
          count: 1
          capabilities: [ gpu ]

Once the containers are up, you can access the GPU from Code Server.

21.4 Changes to operators

The Dockerfile uses a GPU-enabled runtime:

FROM tercen/runtime-tf:2.17.0-gpu

The operator.json declares GPU capability:

"capabilities": ["gpu"]

21.5 Verifying GPU Access

Once inside the container, verify GPU access using TensorFlow or PyTorch.

21.5.1 TensorFlow

import tensorflow as tf
print("Num GPUs Available:", len(tf.config.list_physical_devices('GPU')))

21.5.2 PyTorch

import torch
print("Is CUDA available:", torch.cuda.is_available())

If GPU is not detected, you might need to install some drivers and dependencies on your machine. Please refer to Docker’s GPU support documentation:
https://docs.docker.com/desktop/features/gpu/

# GPU Development in Tercen Studio **Validated end-to-end 2026-08-26** (GCP T4, driver 580, toolkit 1.20): a `gpu`-capability operator installed through Tercen ran with `--gpus=all` added automatically by the worker, and saw the GPU from inside its container. ## The modernized studio (2026-08+): the GPU overlay Requirements on the **host**: an NVIDIA GPU, the NVIDIA driver, and `nvidia-container-toolkit` configured for docker (`sudo nvidia-ctk runtime configure --runtime=docker`). Start the studio with the GPU overlay: ```bash cd tercen_studio docker compose -f docker-compose.yaml -f docker-compose.gpu.yaml up -d ``` That overlay (in the tercen_studio repo) reserves the GPU for the `tercen-worker` service and runs a boot-time setup script inside it which installs the NVIDIA container toolkit, generates the CDI spec, and registers the `nvidia` runtime with podman via a `containers.conf.d` drop-in. With the runtime registered, `ExeRunner.hasGpuCapability()` returns true and the worker adds **`--gpus=all`** to every operator whose `operator.json` declares: ```json "capabilities": ["gpu"] ``` To verify a setup, install and run [`tercen/gpu_smoke_operator`](https://github.com/tercen/gpu_smoke_operator) — it prints the GPUs visible inside the operator container (`GPU 0: Tesla T4 …`) into its output and exits. On production (GKE), GPU operators take a different, independent path: Kubernetes Jobs with `nvidia.com/gpu` limits (`tercen.allow.k8s.job`). ## The pre-2026-08 studio (legacy) The instructions below target the old studio architecture (dind-based). To follow them, check out revision `72e3bdc` of tercen_studio. GPU support is available in **Tercen Studio**. Follow the steps below to enable and test GPU access in your local setup. ## Setup Instructions 1. **Pull the latest version** of the Tercen Studio repository: [https://github.com/tercen/tercen_studio](https://github.com/tercen/tercen_studio) 2. **Configure Docker for GPU support**. Modify the `docker-compose.yaml` file as described below: a. **Comment out the default runtime-docker image**. ```yaml #image: docker:24.0.6-dind-alpine3.18 ``` b. **Uncomment GPU-related configuration**. Under the `dind` service: ```yaml image: tercen/nvidia-dind:12.1.0-runtime-ubuntu22.04 deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [ gpu ] ``` Under the VS Code service `code-server`: ```yaml deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [ gpu ] ``` Once the containers are up, you can access the GPU from **Code Server**. ## Changes to operators - The `Dockerfile` uses a GPU-enabled runtime: ```Dockerfile FROM tercen/runtime-tf:2.17.0-gpu ``` - The `operator.json` declares GPU capability: ```json "capabilities": ["gpu"] ``` ## Verifying GPU Access Once inside the container, verify GPU access using TensorFlow or PyTorch. ### TensorFlow ```python import tensorflow as tf print("Num GPUs Available:", len(tf.config.list_physical_devices('GPU'))) ``` ### PyTorch ```python import torch print("Is CUDA available:", torch.cuda.is_available()) ``` If GPU is not detected, you might need to install some drivers and dependencies on your machine. Please refer to Docker’s GPU support documentation: [https://docs.docker.com/desktop/features/gpu/](https://docs.docker.com/desktop/features/gpu/)