Zesty adds autonomous GPU fractioning for Kubernetes AI workloads

Key Takeaways

• Zesty’s Automated Fractional GPUs add an autonomous decision‑making layer on top of NVIDIA’s open‑source Run:ai GPU‑fractioning operator.
• The solution continuously reallocates GPU memory and compute slices in real time, matching workload demand without manual intervention.
• Early benchmarks show 30‑45 % higher GPU utilization and the ability to run 2‑3× more AI jobs on the same hardware footprint.
• For enterprises, this translates into up to 40 % reduction in GPU spend while preserving latency‑SLA guarantees.

Background: Why GPU Fractioning Matters on Kubernetes

Kubernetes has become the de‑facto operating system for AI workloads. The CNCF 2025 Cloud‑Native Survey reports that 66 % of organizations serving generative‑AI models run inference on Kubernetes【1】. GPU resources, however, remain a cost bottleneck: an NVIDIA A100 (40 GB) can cost $12 k–$14 k per year in cloud pricing, and many clusters operate at 30‑50 % average utilization.

Historically, teams have relied on three static sharing strategies:

These approaches either waste memory/compute cycles or require manual re‑partitioning whenever demand changes.

NVIDIA Run:ai – The First Step Toward Dynamic Fractioning

In early 2026, NVIDIA open‑sourced Run:ai, delivering a GPU Fractioning Operator that enforces per‑pod memory caps and integrates with the Kubernetes scheduler. While Run:ai defines how a GPU can be sliced (e.g., 4 GB or 8 GB chunks), it does not decide which workloads should occupy those slices or when to resize them.

Zesty’s Autonomous Layer: Continuous, Policy‑Driven Allocation

Zesty builds the missing decision engine:

1. Real‑time telemetry – The lightweight agent collects GPU memory usage, SM (streaming multiprocessor) utilization, and latency metrics every 5 seconds.
2. Policy engine – Teams pre‑define SLAs (e.g., ≤ 10 ms inference latency, max 70 % GPU utilization) and resource limits per namespace.
3. Dynamic placement – Using a proprietary heuristic (weighted‑score + reinforcement‑learning), the agent determines:
   • How many workloads can safely co‑locate on a single GPU.
   • The exact memory slice for each pod (e.g., 3.2 GB vs. 4 GB).
   • Whether to use MPS (for low‑latency inference) or MIG (for batch training) per workload.
4. Continuous adjustment – As demand rises, the agent can split a 12 GB slice into two 6 GB slices, or merge idle slices back into a larger block, all without pod restarts.

All decisions respect the performance policies set by administrators, guaranteeing that no workload exceeds its latency or availability budget.

Quantifiable Benefits

In a production test at a European fintech firm (Berlin), the cluster moved from 38 % to 81 % average GPU utilization, allowing the team to retire four A100 nodes and cut monthly GPU spend by ≈ €18 k.

Deployment Overview

• Agent footprint: < 30 MB container, runs as a DaemonSet.
• Compatibility: Works with any CNCF‑certified Kubernetes distribution (EKS, GKE, AKS, OpenShift).
• Integration points: Communicates with the Run:ai GPU Fractioning Operator via the Kubernetes API and with the scheduler through the Extender webhook.
• Security: Operates with read‑only access to node‑level metrics; all decisions are logged to an audit‑ready JSON stream.

Comparison with Competing Solutions

Bottom Line

Zesty’s Automated Fractional GPU capability transforms NVIDIA’s Run:ai from a static slicing tool into a **self‑optimizing