How it works Features Production Scenarios Quickstart LLM Backends GitHub ↗
Open-source · MIT license

Your AI
On-Call SRE

Give AutoSRE a production alert. It pulls metrics, reads logs, diagnoses the root cause, runs the fix, and hands you back a written incident report. All autonomously.

Python 3.10+ LLM tool-use loop Groq · Gemini · Ollama MIT License
autosre -- agent.py -- zsh
0
LLM Providers
0
Demo Scenarios
Servers Required
0
Autonomous Steps

A closed-loop that runs itself

AutoSRE mimics exactly what an on-call SRE does: check dashboards, read logs, form a hypothesis, apply a fix, verify recovery. All in one autonomous loop.

Fault Alert

A production alert fires: latency spike, disk full, rising error rate.

Gather Signals

Query time-series metrics (Prometheus) and search error logs (ELK).

Diagnose

The LLM correlates all signals into a root-cause hypothesis.

Remediate

Pick and execute the matching Ansible playbook to fix the issue.

Report

Confirm recovery and emit a structured incident report in Markdown.

Why AutoSRE?

Designed as a reference implementation. Clean, hackable, and built to show exactly how autonomous AIOps agents work.

Backend-agnostic LLM loop

Uses a single OpenAI-compatible tool-use loop. Swap providers by changing one line in .env. Works with Groq, Gemini, Ollama, Anthropic, or any OpenAI-compatible endpoint.

Runs on your laptop, no infra needed

Prometheus, ELK, and Ansible are provided as lightweight Flask mocks. The entire stack starts in seconds with a single shell script and zero Docker required.

Verifiable reasoning

Each scenario ships with a known ground-truth root cause. You can check the agent's diagnosis against the expected answer to verify it reasoned correctly, not just got lucky.

Plug in real infrastructure

The mock services follow the same API contracts as real Prometheus, Elasticsearch, and Ansible. Point to your actual endpoints and the agent's reasoning loop works unchanged.

Hardened for real on-call

Beyond the laptop demo: webhooks, approval gates, provider fallback, incident history, and an independent rollback safety net.

Alertmanager webhook

Run python server.py and POST to /webhook/alertmanager. Incidents are processed one at a time through a serial queue.

Approval gate

Remediation can require operator approval via auto, interactive prompt, or an external webhook before any playbook runs.

LLM fallback + rollback

Chain providers with LLM_FALLBACK_CHAIN. If metrics stay unhealthy, an independent AUTOSRE_ROLLBACK_PLAYBOOK fires outside the LLM loop.

Incident history

Every run is stored in SQLite and listed at GET /incidents, alongside Markdown reports under reports/.

Three faults, one command each

Each scenario ships with a known ground-truth root cause so you can verify the agent's reasoning is correct, not just fast.

python agent.py db

Database Pool Exhaustion

Service order-service
Symptom API latency spikes from 200ms → 1.5s
Root cause DB connection pool misconfigured, max connections exhausted
Playbook restore_db_pool.yml
python agent.py disk

Disk Space Exhausted

Service file-service
Symptom /data partition at 98%, write failures
Root cause Log rotation disabled, disk filled with unrotated logs
Playbook clean_disk_space.yml
python agent.py network

Network Partition

Service payment-service
Symptom Rising payment failure rate, timeout errors
Root cause Network partition between payment-service and upstream gateway
Playbook restart_service.yml

Up and running in 4 steps

No Docker required. Everything runs on plain Python and your choice of LLM provider, including completely free options.

1

Clone and install dependencies

Clone the repo and install the Python requirements.

bash
git clone https://github.com/canyang25/AIOPS.git
cd AIOPS
pip install -r requirements.txt
2

Configure your LLM provider

Copy the example config and uncomment one provider. Groq is free with no credit card required — the fastest way to get started.

bash
cp .env.example .env
# Open .env and set your key, e.g.:
# GROQ_API_KEY=gsk_...    (free, no credit card)
# GEMINI_API_KEY=...       (free tier)
# LLM_PROVIDER=ollama      (fully local)
3

Start the mock backends

Spin up lightweight Flask mocks for Prometheus, ELK, and Ansible. Pure Python, no Docker needed.

bash
./start_services.sh

# Mock Prometheus  →  http://localhost:9091
# Mock Ansible     →  http://localhost:9092
# Mock ELK         →  http://localhost:9093
4

Run the agent

Pick any scenario and watch AutoSRE investigate, remediate, and write the report.

bash
python agent.py db        # DB pool exhaustion
python agent.py disk      # Disk full
python agent.py network   # Network partition
python agent.py --list    # List all scenarios

# No API key? Run the offline simulation:
python agent.py db --simulate

Works with any major provider

AutoSRE uses an OpenAI-compatible tool-use loop; swap the backend with a single line in .env. Three options run completely free.

Provider Cost Config Notes
Groq Free GROQ_API_KEY=gsk_... No credit card · fastest option
Gemini Free tier GEMINI_API_KEY=... Google's models · generous quota
Ollama Local / Free LLM_PROVIDER=ollama 100% local · fully private · no internet
Anthropic Paid ANTHROPIC_API_KEY=sk-ant-... Claude, strongest reasoning
OpenAI Paid OPENAI_API_KEY=sk-... GPT-4o · any OpenAI-compatible endpoint

Connect to real infrastructure

The mock services follow the same API contracts as the real tools. Swap the URLs and the agent's reasoning loop works unchanged.

Real Prometheus

Point to your actual Prometheus instance. The agent queries the same PromQL endpoints it uses against the mock.

PROMETHEUS_URL=http://your-prometheus:9090

Real Elasticsearch / ELK

Replace the mock ELK with your Elasticsearch cluster. Same search API shape; just update the URL and add auth headers if needed.

ELK_URL=http://your-elastic:9200

Real Ansible

Swap the mock Ansible API for your AWX/Tower instance or a custom webhook. The remediation step calls the same REST endpoint.

ANSIBLE_URL=http://your-awx:8080