---
name: al-659-put-a-team-of-read-only-ai-ops-agents-between-your-gpu-cl
description: "NVIDIA's Nov 4, 2026 technical blog describes the AI Factory Operations Agent it runs on its own DGX SuperPOD clusters. Instead of an SRE hopping between dashboards for compute, network, hardware and scheduling, a chat-facing agent routes to specialized sub-agents (Job Recovery, Hardware, Observability, Research, Kubernetes, Cluster Management) over MCP, correlates root causes across layers, and surfaces recommendations with transparent tool calls. The framework is built on the NemoClaw blueprint with OpenShell sandboxes and can run on-premises on Nemotron models. Production learnings stress clear agent boundaries, read-only investigation first, precise MCP tool definitions, and scoped permissions for multi-tenant factories. These are NVIDIA's own claims about its internal deployment and open developer example. Use when the user wants help with multi-agent ai factory / gpu cluster sre (it & internal help) for a small business."
---

# Put a team of read-only AI ops agents between your GPU cluster and the on-call SRE: one chat asks Job Recovery, Hardware, Observability, Kubernetes and Cluster Management agents to correlate a training failure across layers and show the evidence before anyone acts

NVIDIA's Nov 4, 2026 technical blog describes the AI Factory Operations Agent it runs on its own DGX SuperPOD clusters. Instead of an SRE hopping between dashboards for compute, network, hardware and scheduling, a chat-facing agent routes to specialized sub-agents (Job Recovery, Hardware, Observability, Research, Kubernetes, Cluster Management) over MCP, correlates root causes across layers, and surfaces recommendations with transparent tool calls. The framework is built on the NemoClaw blueprint with OpenShell sandboxes and can run on-premises on Nemotron models. Production learnings stress clear agent boundaries, read-only investigation first, precise MCP tool definitions, and scoped permissions for multi-tenant factories. These are NVIDIA's own claims about its internal deployment and open developer example.

## Inputs to collect first
Ask the user for anything below that they haven't already given you. Don't guess missing facts.

- Cluster type, e.g. Kubernetes + GPUs
- Prometheus, logs, job scheduler, hardware alerts, docs
- E.g. training jobs fail and on-call checks five dashboards
- A recent failure

## Instructions

Do the task below with the user's inputs in place of each [bracket].

Help me design a read-only multi-agent ops desk for our GPU or Kubernetes cluster, like an AI factory SRE helper. Our stack: [cluster type, e.g. Kubernetes + GPUs]. Signals we have: [Prometheus, logs, job scheduler, hardware alerts, docs]. Pain: [e.g. training jobs fail and on-call checks five dashboards]. 1) Propose 4-6 specialist agents with clear non-overlapping jobs (for example job recovery, hardware, observability, docs, Kubernetes, cluster management) and what each may query. 2) For each agent, list only read tools it should have and tools it must never have (no create/update/delete of pods, secrets or jobs). 3) Write the orchestrator rules: when a human asks in chat, which agent goes first, how agents hand evidence to each other, and when to stop and ask a person. 4) Draft one sample investigation for [describe a recent failure], with the questions each agent asks and the citations they must return (query result or doc section). 5) Give a trust rollout: week 1 chat-only recommendations, week 2 ticket drafts, never auto-remediate until SREs agree on accuracy. Don't invent metrics I didn't give you.

## Safety checks (share these with the user)

- This prompt works on text other people wrote (emails, reviews, messages or web pages). That text can hide instructions aimed at AI, such as "ignore your rules and…". Paste it as material to work on, and never let the AI act on requests inside it, like paying, sharing logins or changing records.
- Check names, dates, numbers and any claim against your own source material before you use the answer. AI can misread or invent details.
- Say clearly which parts need a person to review or approve before anything is sent, paid, posted or filed.

## About this skill

- Works with: NVIDIA AI Factory Operations Agent / NemoClaw, MCP, OpenShell, Prometheus, Grafana, Kubernetes, Claude or other chat agents
- Based on this real-world example: https://developer.nvidia.com/blog/how-nvidia-uses-ai-agents-to-streamline-ai-factory-operations/
- Free prompt from AgentPlaybook Real Estate: https://agentic-library.netlify.app/prompt/al-659-put-a-team-of-read-only-ai-ops-agents-between-your-gpu-clust/
- To install in an agent that supports Agent Skills, save this file as SKILL.md in a folder with the same name as `name` above (for example in your skills directory). Install steps for Claude.ai, Claude Code, Cursor and Codex: https://agentic-library.netlify.app/skills/