GenAI and DevOps

Empowering our solutions with GenAI is becoming a must-have feature.

Trendwords such as RAG, Agents, and MCP are everywhere, filling our heads.

This Insights article explores how GenAI features can be used when designing and implementing CI/CD pipelines in our software.

Use Cases

• PR Analysis - we can build RAGs using diffs and results of the commit stage CI pipeline to review PRs automatically.
• Pipeline Debugging - we can have steps/tasks to debug pipeline failures using LLMs; the debugging step can find the root cause and output possible solutions.
• Steps Summary and Analysis - some pipelines create long outputs that sometimes are hard to interpret, as an example, a Terraform Plan output; we can add to our pipelines a step that uses LLMs to interpret a TF Plan as an example; with the right prompt LLMs can tells us what will update, the risk of destroying critical resources of our infra, and etc;
• Security Review
• Release Notes Generation
• Auto-approve PRs
• Auto test
• Auto fix

Even the inner structure of Applying GenAI to an application uses pipes, which is similar to what we do in pipelines.

There are numerous ways to integrate LLMs into our CI/CD pipelines and remember: integrate in different levels ...

The examples above demonstrate kind of passive or informative approaches, but we can dig deeper and implement steps that generate code to fix or remediate issues, or to test our application, and have the code be run by the pipeline in a more agentic way.

Tools You Can Use Today

by ChatGPT

• LangChain – LLM orchestration, agent tooling, and MCP support via LCEL (LangChain Expression Language)
• CrewAI – Multi-agent system framework, great for modeling CI/CD "task teams"
• OpenDevin – Open-source dev agent with CI/CD-style automation abilities
• LangGraph – Graph-based agent workflows, ideal for complex pipeline logic
• GitHub Copilot CLI – Adds AI-powered CLI suggestions and automation for developers
• AutoPR – GitHub App that automatically creates AI-generated pull requests
• Your own FastAPI + LLM server – Build custom endpoints to serve structured prompts using the MCP schema

Let's give some references to contextualize a bit ...

GenAI

definition by Claude

> Generative AI (GenAI) refers to artificial intelligence systems that can create new content such as text, images, audio, code, or video based on the patterns they've learned from training data. These systems use deep learning techniques, particularly large language models and diffusion models, to generate outputs that weren't explicitly programmed but instead emerge from statistical patterns in the training data.

Popular examples include text generators like GPT-4 and Claude, image creators like DALL-E and Midjourney, and code assistants like GitHub Copilot.

RAG (Retrieval-Augmented Generation)

definition by Claude

> RAG (Retrieval-Augmented Generation) is an AI framework that combines the strengths of large language models with external knowledge retrieval systems. It works by first retrieving relevant information from a knowledge base in response to a query, then using that information to generate more accurate, up-to-date, and factual responses.

The key benefits of RAG include:

Reducing hallucinations by grounding responses in retrieved facts

Enabling access to specialized or proprietary information

Allowing AI systems to reference current information beyond their training cutoff

RAG has become a fundamental approach for enhancing AI assistants with external knowledge sources

MCP (Model Context Protocol)

definition by Claude

> MCP (Model Context Protocol) is a framework for integrating context into AI systems through standardized interfaces. It provides a structured approach for AI models to request, receive, and incorporate external information during generation.

Key aspects of MCP include:

A unified protocol for passing context between different components of AI systems

Standardized methods for models to retrieve relevant information when needed

Mechanisms for maintaining coherence when incorporating external knowledge

Support for both static and dynamic context retrieval during inference

MCP helps improve AI responses by ensuring they have access to appropriate context while maintaining a consistent interface across different systems and use cases.

GenAI Agents

definition by Claude

> GenAI Agents are autonomous or semi-autonomous AI systems that combine generative AI capabilities with agency—the ability to take independent actions to achieve goals. Unlike basic generative models that simply respond to prompts, GenAI Agents can:

Make decisions and execute tasks through reasoning

Access and use tools (like web search, code execution, or APIs)

Plan multi-step processes to accomplish complex objectives

Maintain context and memory across interactions

Learn from feedback and adapt their approaches

Examples include AI assistants that can research topics, analyze data, book appointments, or create content through coordinated actions rather than single-step generation. These agents represent an evolution from passive text generators to more capable, goal-oriented AI systems.