<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Amit Bhatt on Medium]]></title>
        <description><![CDATA[Stories by Amit Bhatt on Medium]]></description>
        <link>https://medium.com/@baremetal-dev?source=rss-00f61ecfea5c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*WRu58l0qbYcaEwKXjY_y-Q.png</url>
            <title>Stories by Amit Bhatt on Medium</title>
            <link>https://medium.com/@baremetal-dev?source=rss-00f61ecfea5c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 05:08:16 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@baremetal-dev/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[I Turned My M1 MacBook Into an Offline AI Coding Agent — $0 API Cost, Zero Cloud]]></title>
            <link>https://pub.towardsai.net/i-turned-my-m1-macbook-into-an-offline-ai-coding-agent-0-api-cost-zero-cloud-8ca8b4b75ff1?source=rss-00f61ecfea5c------2</link>
            <guid isPermaLink="false">https://medium.com/p/8ca8b4b75ff1</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Amit Bhatt]]></dc:creator>
            <pubDate>Sat, 11 Apr 2026 08:41:06 GMT</pubDate>
            <atom:updated>2026-04-12T10:23:24.647Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>The step-by-step blueprint to building a fully offline, 26B parameter AI coding agent on Apple Silicon, leveraging llama.cpp, Unsloth, and OpenCode for zero-internet development.</strong></h4><figure><img alt="Building fully offline coding agent on M1 Macbook Pro" src="https://cdn-images-1.medium.com/max/1024/1*8IcH8ZtsT3j-HrBJmFz17A.png" /><figcaption>Source: Image by the author.</figcaption></figure><p>The cloud is great — until you hit a rate limit mid-refactor. Or you’re on a flight. Or you’re working on code that should never leave your machine.</p><p>I spent three weeks obsessing over one question: <strong>how close can you actually get to a GPT-level agentic coding experience running 100% locally, with zero internet?</strong></p><p>The answer surprised me. My M1 MacBook Pro — no discrete GPU, no cloud subscription, no API key — now runs a 26-billion parameter model that reads my codebase, writes code, applies diffs, and proposes Git changes. Autonomously. Offline.</p><p>This post is the exact, reproducible blueprint. Every command is copy-pasteable. Every decision is explained.</p><blockquote><strong><em>TL;DR</em></strong><em> — I compiled </em><em>llama.cpp with Metal GPU acceleration on an M1 Mac, loaded Google&#39;s Gemma-4 26B via Unsloth&#39;s quantization, and wired it to OpenCode for a fully agentic, offline coding workflow. Total API cost: </em><strong><em>$0</em></strong><em>. Data sent to the cloud: </em><strong><em>0 bytes</em></strong><em>.</em></blockquote><h3>Why This Matters Right Now</h3><p>Most conversations about “local AI” treat it as a hobbyist curiosity — small models, toy tasks, nothing you’d trust on real work. That was true 18 months ago. It isn’t anymore.</p><p>Three things converged to make this actually viable:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b1e8aeaa83c2e07eaeefe68208723a62/href">https://medium.com/media/b1e8aeaa83c2e07eaeefe68208723a62/href</a></iframe><p>Put those three together and a standard developer laptop becomes a credible inference machine.</p><h3>The Hardware and the Brain</h3><p>I’m running this on an <strong>M1 MacBook Pro with 32GB of unified memory</strong>.</p><p>For the model, I chose <strong>Unsloth’s Gemma 4 26B instruction tuned GGUF </strong>after reviewing the <a href="https://unsloth.ai/docs/models/gemma-4#hardware-requirements">hardware requirements for Gemma-4</a>. Here’s why each component of that model name matters:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fb8e2bb56ca31646dd792d7d925c4ea4/href">https://medium.com/media/fb8e2bb56ca31646dd792d7d925c4ea4/href</a></iframe><p>At a Q4 quantization, the 26B model requires roughly 15–16GB of memory. On 32GB unified memory, that leaves more than enough overhead for macOS, your IDE, and OpenCode running simultaneously.</p><h3>Prerequisites</h3><p>Everything below is available through Homebrew or pip. No manual compilation required except llama.cpp itself.</p><pre># Xcode Command Line Tools (required for cmake, git, and Metal framework headers)<br>xcode-select --install<br><br># Core build dependencies<br>brew install cmake libomp<br><br># Hugging Face CLI for model downloads<br>pip install huggingface_hub hf_transfer<br><br># Parallel download engine (optional but strongly recommended for large models)<br>brew install aria2<br><br># OpenCode - the agentic coding orchestrator<br>brew install anomalyco/tap/opencode</pre><p>With these in place, every step below works on a clean macOS install.</p><h3>Step 1: Compile llama.cpp from Scratch with Metal</h3><p>You <em>could</em> download a pre-built binary. But if you want every drop of performance from the M1’s Metal GPU framework, build from source.</p><pre>git clone https://github.com/ggml-org/llama.cpp.git<br>cd llama.cpp<br>cmake -B build -DGGML_METAL=ON<br>cmake --build build --config Release -j$(sysctl -n hw.ncpu)</pre><p>The key flag is -DGGML_METAL=ON — this compiles with Apple&#39;s Metal GPU framework. The -j$(sysctl -n hw.ncpu) parallelizes the build across all CPU cores.</p><p>When you compile directly on the M1, inference speeds jump dramatically. You aren’t just running code — you’re running code hyper-optimized for your specific silicon.</p><p>After the build, create symlinks to keep commands clean:</p><pre>ln -s ./llama.cpp/build/bin/llama-cli llama-cli<br>ln -s ./llama.cpp/build/bin/llama-server llama-server</pre><p>llama-cli handles interactive terminal prompts. llama-server is the HTTP inference server that exposes an OpenAI-compatible API — the piece that connects to OpenCode. Both are built from the same source tree.</p><h4>Validate the Build Before the Big Download</h4><p>Before downloading the massive 18GB Gemma-4 model, validate the entire pipeline with a smaller model: <a href="https://huggingface.co/unsloth/NVIDIA-Nemotron-3-Nano-4B-GGUF">NVIDIA Nemotron-3-Nano-4B</a> at Q8 quantization, just 3.9GB.</p><p><strong>This step is not optional.</strong> You don’t want to wait hours for an 18GB download only to discover your build is broken.</p><figure><img alt="Download progress of nemotron-3-nano-4B model" src="https://cdn-images-1.medium.com/max/800/0*NdTS8LHXhgQWGFAp.png" /><figcaption>Source: Image by the author.</figcaption></figure><p>Booting the server with the smaller model confirms what you need to see: Metal framework fully initialized, unified memory detected, all GPU families registered.</p><figure><img alt="Running nemotron-3-nano-4B locally via llama-server" src="https://cdn-images-1.medium.com/max/800/0*CjLO6aKpGhqLurcB.png" /><figcaption>Source: Image by the author.</figcaption></figure><p>The critical line in that output: has unified memory = true. The recommendedMaxWorkingSetSize of roughly 26,800 MB tells you exactly how much VRAM the Metal backend can access — and on the M1, it draws directly from system RAM.</p><p>Pipeline is solid. Now bring in the real model.</p><h3>Step 2: Download Gemma-4 26B Weights</h3><p>Because we’re building an offline environment, the model file needs to be local.</p><pre>hf download unsloth/gemma-4-26B-A4B-it-GGUF \<br>  --local-dir unsloth/gemma-4-26B-A4B-it-GGUF \<br>  --include &quot;*mmproj-BF16*&quot; \<br>  --include &quot;*UD-Q4_K_XL*&quot;</pre><p>The --include filters are important:</p><ul><li>*mmproj-BF16* — the multimodal vision projector, giving the model the ability to understand images alongside code</li><li>*UD-Q4_K_XL* — the sweet spot quantization for quality vs. memory on 32GB (~15.9GB on disk)</li></ul><h4>Fair Warning: 18.3GB Downloads Are Fragile</h4><figure><img alt="Failed download attempt for gemma-4 26B model using hf-download utility" src="https://cdn-images-1.medium.com/max/800/0*nA3FcpI6VDMNxBgI.png" /><figcaption>Source: Image by the author.</figcaption></figure><p>My download crawled to 519KB/s before failing entirely. The default hf download CLI supports resuming in theory, but in practice it&#39;s fragile on large files over unstable connections.</p><p>Switch to <a href="https://gist.github.com/yeahjack/31f542ee6cab3c3e2c30594b7693cb22#file-hfd-sh"><strong>hfd.sh</strong></a> with aria2c as the engine. Unlike the default CLI, aria2c tracks per-segment progress in .aria2 control files — a dropped connection picks up exactly where it left off instead of restarting the entire file.</p><pre>./hfd.sh unsloth/gemma-4-26B-A4B-it-GGUF \<br>  --local-dir unsloth/gemma-4-26B-A4B-it-GGUF \<br>  --include &quot;*mmproj-BF16*&quot; \<br>  --include &quot;*UD-Q4_K_XL*&quot; \<br>  --tool aria2c -x 16 -n 8</pre><p>-x 16 opens 16 connections per server. -n 8 splits each file into 8 parallel segments. On a decent connection, this is dramatically faster and more resilient than the default downloader.</p><h3>Step 3: Wire the Brain to OpenCode</h3><p>Having a powerful local LLM is interesting. Having it autonomously write, edit, and debug your code is a different thing entirely. That’s where <a href="https://opencode.ai/docs/">OpenCode</a> comes in.</p><p>OpenCode bridges the local LLM and your codebase. The key insight: llama-server exposes an OpenAI-compatible /v1/chat/completions endpoint out of the box. OpenCode&#39;s @ai-sdk/openai-compatible adapter speaks that protocol natively. No custom prompt templates, no manual token wrangling — the chat template baked into the GGUF handles everything at the server level.</p><p>Create opencode.json at your project root:</p><pre>{<br>  &quot;$schema&quot;: &quot;https://opencode.ai/config.json&quot;,<br>  &quot;provider&quot;: {<br>    &quot;llama.cpp&quot;: {<br>      &quot;npm&quot;: &quot;@ai-sdk/openai-compatible&quot;,<br>      &quot;name&quot;: &quot;llama-server (local)&quot;,<br>      &quot;options&quot;: {<br>        &quot;baseURL&quot;: &quot;http://127.0.0.1:8001&quot;<br>      },<br>      &quot;models&quot;: {<br>        &quot;gemma-4:26b-a4b-it&quot;: {<br>          &quot;name&quot;: &quot;Gemma-4-26B-A4B-it (local)&quot;,<br>          &quot;limit&quot;: {<br>            &quot;context&quot;: 32000,<br>            &quot;output&quot;: 65536<br>          }<br>        },<br>        &quot;nvidia-nemotron-3-nano:4b&quot;: {<br>          &quot;name&quot;: &quot;NVIDIA-Nemotron-3-Nano-4B (local)&quot;,<br>          &quot;limit&quot;: {<br>            &quot;context&quot;: 32000,<br>            &quot;output&quot;: 65536<br>          }<br>        }<br>      }<br>    }<br>  }<br>}</pre><p>A few things worth noting:</p><ul><li>The baseURL points to 127.0.0.1:8001 where llama-server will listen</li><li>Context is set to 32K tokens — the model supports up to 262K, but 32K is a practical ceiling for stable agentic sessions on 32GB RAM</li><li>The second model (Nemotron-3 Nano 4B) is configured as a lightweight alternative for fast, low-overhead tasks</li></ul><p>Verify the model is running correctly by checking the llama.cpp web interface at <a href="http://127.0.0.1:8001:">http://127.0.0.1:8001:</a></p><figure><img alt="model configuration of llama-cpp web interface" src="https://cdn-images-1.medium.com/max/800/0*PaVGO3hnh1rCnw7b.png" /><figcaption>Source: Image by the author.</figcaption></figure><blockquote>25.23 billion parameters. A 262,144-token context window. Vision capability. Running from a file on local disk, served over localhost. No cloud, no API key, no rate limit.</blockquote><h3>Step 4: Full Offline Agentic Coding</h3><p>With the model downloaded, llama.cpp compiled, and opencode.json locked in, I turned off Wi-Fi.</p><p><strong>Zero internet. Zero API calls. Zero data leaving the machine.</strong></p><p>Open two terminals:</p><pre># Terminal 1: Start the llama.cpp inference server with Gemma-4<br>./llama-server \<br>  --model unsloth/gemma-4-26B-A4B-it-GGUF/gemma-4-26B-A4B-it-UD-Q4_K_XL.gguf \<br>  --temp 0.6 \<br>  --top-p 0.95 \<br>  --alias &quot;gemma-4-26B&quot; \<br>  --port 8001 \<br>  -ngl 99 -t 8 -b 512 --mmap</pre><p>Still validating with Nemotron before the full download? Same flags work:</p><pre># Alternative: lighter Nemotron model for testing<br>./llama-server \<br>  --model unsloth/NVIDIA-Nemotron-3-Nano-4B-GGUF/NVIDIA-Nemotron-3-Nano-4B-Q8_0.gguf \<br>  --temp 0.6 \<br>  --top-p 0.95 \<br>  --alias &quot;nvidia/nemotron-3-nano-4B-GGUF&quot; \<br>  --port 8001 \<br>  --reasoning on \<br>  -ngl 99 -t 8 -b 512 --mmap</pre><p>Note the --reasoning on flag on Nemotron — it activates a built-in chain-of-thought mode that improves output quality on complex tasks. Useful for validating multi-step reasoning before scaling up to Gemma-4.</p><pre># Terminal 2: Launch OpenCode<br>opencode web</pre><p>Here’s what each server flag does:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7f2b9afddb35c3ebf876706301b0453e/href">https://medium.com/media/7f2b9afddb35c3ebf876706301b0453e/href</a></iframe><figure><img alt="Offline agentic coding with Opencode and Gemma-4 running via llama-server" src="https://cdn-images-1.medium.com/max/800/0*TsoGxSkqzQgMB0aU.png" /><figcaption>Source: Image by the author.</figcaption></figure><p><strong>The result:</strong> Gemma-4 26B analyzed my codebase, understood the architecture of local files, and began writing, diffing, and applying code. In the screenshot above, it analyzed an architect.py file, broke down Pydantic data models, explained the run_architect function flow, and proposed 5 Git changes across the project.</p><p>The M1 pushed out tokens fast enough for real-time development. The footer confirms it: gemma-4-26B, 45 seconds for a full architectural analysis and code generation pass.</p><h3>What This Actually Means</h3><p>We are crossing a threshold.</p><p>For two years, the industry assumed truly capable AI agents require data centers. This experiment proves otherwise.</p><p><strong>For engineering teams working on sensitive codebases</strong> — defense, healthcare, fintech — this means AI coding assistants without a single byte crossing a network boundary. No SOC 2 reviews for another SaaS vendor. No data processing agreements. No trust boundaries to negotiate.</p><p><strong>For engineering leaders</strong>, the math is compelling:</p><ul><li>Zero marginal API cost per developer</li><li>Zero vendor lock-in</li><li>Works identically on an airplane, in a SCIF, or behind an air-gapped network</li></ul><p><strong>For individual developers</strong>, the practical reality: you can now run a frontier-class coding agent on hardware you already own, using models that are openly licensed, with no subscription, no quota, no latency spikes on someone else’s overloaded GPU cluster.</p><p>Absolute privacy and top-tier AI capability are no longer mutually exclusive.</p><h3>What I’m Exploring Next</h3><p>This setup is a foundation, not a ceiling.</p><p><strong>Larger context windows.</strong> The 32K context in opencode.json is conservative. With careful memory management and llama.cpp&#39;s Flash Attention support, 128K+ is feasible on 32GB for longer agentic sessions.</p><p><strong>Multi-model routing.</strong> Running Nemotron for fast, lightweight tasks and Gemma-4 for heavy reasoning — switching between models based on task complexity, all locally. Think of it as a cheap/smart tier system without the cloud bill.</p><p><strong>Fine-tuning on proprietary code.</strong> Unsloth supports LoRA and QLoRA fine-tuning. Training a domain-specific adapter on your team’s codebase and merging it into the GGUF gives you a model that <em>thinks</em> in your architecture and naming conventions.</p><p><strong>Team-wide access.</strong> Embed llama-server in a container behind your internal network so the entire team gets local AI without each developer maintaining their own build.</p><h3>The Full Stack, in One Place</h3><p>For anyone who wants to reproduce this exactly:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/603c3f1af77387199b5eff5adf88f0ab/href">https://medium.com/media/603c3f1af77387199b5eff5adf88f0ab/href</a></iframe><p>The tools are here. The models are capable enough. The only question is what you build with them.</p><p>If you found this useful, the full write-up with additional context lives on <a href="https://sectumpsempra.github.io/"><em>my site</em></a>. Questions, improvements, or your own local AI stack? Drop them in the comments — I’d genuinely like to hear what you’re running.</p><p><em>Written by </em><a href="https://linkedin.com/in/amit-bhatt"><em>Amit Bhatt</em></a><em> — </em><a href="https://github.com/sectumpsempra"><em>GitHub</em></a></p><blockquote>This article was originally published on my <a href="https://dev.to/baremetal-dev/i-turned-my-m1-macbook-into-an-offline-ai-coding-agent-0-api-cost-zero-cloud-14hb">Dev.to</a> blog.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8ca8b4b75ff1" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/i-turned-my-m1-macbook-into-an-offline-ai-coding-agent-0-api-cost-zero-cloud-8ca8b4b75ff1">I Turned My M1 MacBook Into an Offline AI Coding Agent — $0 API Cost, Zero Cloud</a> was originally published in <a href="https://pub.towardsai.net">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beyond the Prompt: Building a Multi-Agent DevOps Squad with a Security Conscience]]></title>
            <link>https://pub.towardsai.net/we-let-ai-write-our-terraform-then-we-gave-it-a-security-conscience-8a6f37f59eab?source=rss-00f61ecfea5c------2</link>
            <guid isPermaLink="false">https://medium.com/p/8a6f37f59eab</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Amit Bhatt]]></dc:creator>
            <pubDate>Thu, 09 Apr 2026 19:06:20 GMT</pubDate>
            <atom:updated>2026-04-10T03:04:01.058Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>A deep dive into multi-agent collaboration, LangGraph, and automated Terraform auditing</strong></h4><p>Designing cloud infrastructure usually takes three meetings.</p><p>One with the architect to decide which services to use. One with the DevOps engineer to actually write the Terraform. One with the security team to explain, again, why 0.0.0.0/0 is not an acceptable production CIDR.</p><p>By the time all three conversations happen, the architecture diagram is already out of date.</p><p>So we asked a different question: what if all four roles ran as AI agents in a single automated pipeline?</p><p>You type your requirements in plain English. You get back deployable Terraform HCL, a security audit with specific remediation guidance, and a rendered architecture diagram. In one shot, without the meetings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lGgYMOJHeF3HKqmM4ofpJg.png" /><figcaption>Source: Image by the author.</figcaption></figure><p>That’s InfraSquad. This post is about what we learned building it, what broke badly, and what we would tell ourselves at the start.</p><blockquote><strong><em>TL;DR:</em></strong><em> InfraSquad is a multi-agent system built on LangGraph. Four agents collaborate in a cyclic state machine. Security findings loop back to the DevOps agent for fixes, capped at three cycles. Without that cap, the loop runs forever. We learned this during testing. The code is open source at </em><a href="https://github.com/Andela-AI-Engineering-Bootcamp/infrasquad"><em>Andela-AI-Engineering-Bootcamp/infrasquad</em></a><em>.</em></blockquote><h3>Meet the Squad</h3><p>Four agents. One shared pipeline. Here is what each one actually does:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4c112f6aacb14c8a67a9848a2cbdd635/href">https://medium.com/media/4c112f6aacb14c8a67a9848a2cbdd635/href</a></iframe><p>The critical word in that table is “sent back.” The Security Auditor does not just generate a report and hand it off. It can send the DevOps Engineer back to fix its own code. That feedback loop is the most interesting design decision in the system. It is also how we nearly created an infinite loop on the second day of integration testing.</p><h3>The Pipeline (and the Two Places It Can Loop)</h3><p>Here is the full state machine:</p><figure><img alt="InfraSquad pipeline diagram showing the LangGraph state machine-validate_input, architect, devops, validate_output, security, visualizer, with two loop-back arrows for HCL errors and security findings" src="https://cdn-images-1.medium.com/max/1024/1*m49iplyf9Szs94E9_VP07g.png" /><figcaption>Source: Image by the author.</figcaption></figure><p>The happy path is straightforward:</p><ol><li><strong>validate_input</strong> runs three checks before anything expensive happens</li><li><strong>architect</strong> produces a numbered AWS architecture plan</li><li><strong>devops</strong> writes Terraform HCL from that plan</li><li><strong>validate_output</strong> checks the HCL deterministically for forbidden patterns and structural validity</li><li><strong>security</strong> scans with tfsec or checkov via MCP</li><li><strong>visualizer</strong> renders the architecture as a Mermaid diagram</li></ol><p>Two of those six nodes can send the pipeline backwards. That is intentional. It is also dangerous if you do not cap the cycle count, which we did not do initially.</p><p>All six agents share a single typed state object:</p><pre>class AgentState(TypedDict, total=False):<br>    user_request: str<br>    architecture_plan: str<br>    terraform_code: str<br>    security_report: dict[str, Any]<br>    security_passed: bool<br>    remediation_count: int<br>    hcl_remediation_count: int<br>    hcl_validation_errors: list[str]<br>    current_phase: str</pre><p>The total=False matters. Without it, every agent would need to set every field, even fields it knows nothing about. With it, agents only write what they own. Silent downstream failures from unexpected None values were the most frustrating class of bug we hit early on.</p><h3>We Almost Created an Infinite Loop on Day Two</h3><p>During integration testing, we ran a request for an internet-facing Application Load Balancer.</p><p>The Security Auditor flagged it: AVD-AWS-0107, HIGH-security group allows unrestricted ingress from 0.0.0.0/0. The DevOps agent tried to fix it. The Security Auditor re-scanned. Same finding. The DevOps agent tried again. Same finding.</p><p>The problem: a public ALB is supposed to have unrestricted public ingress. That is what “internet-facing” means. The security finding was technically correct and permanently unfixable given the design intent. The LLM had no way to distinguish “security issue to remediate” from “accepted design constraint.”</p><p>Without an exit condition, this loop runs forever.</p><p>Here is what the routing logic looks like with the cap in place:</p><pre>def route_after_security(state: AgentState) -&gt; Literal[&quot;visualizer&quot;, &quot;devops&quot;]:<br>    if state.get(&quot;security_passed&quot;, False):<br>        return &quot;visualizer&quot;<br>    if state.get(&quot;remediation_count&quot;, 0) &gt;= settings.max_remediation_cycles:<br>        return &quot;visualizer&quot;  # move on regardless<br>    return &quot;devops&quot;</pre><p>After three cycles, the pipeline proceeds with whatever state it has. Unresolved findings appear as advisory warnings in the Security tab, not hard failures. The same cap exists on the HCL validation loop.</p><p><strong>Add your cycle caps before your first integration test.</strong> Not after. You will hit this case.</p><h3>The One Bad Habit We Couldn’t Engineer Away</h3><p>Every model we tested had the same behavior: for any internet-facing resource, it generated 0.0.0.0/0 as the security group ingress CIDR. Even with explicit instructions in the system prompt. Even with examples. Even with counter-examples.</p><p>We tried prompt engineering for weeks. The model would acknowledge the constraint, then generate 0.0.0.0/0 anyway on the next call.</p><p>So we stopped fighting it and added a deterministic sanitizer that runs on the DevOps agent’s output before validation even starts:</p><pre>_CIDR_SANITISATIONS: list[tuple[re.Pattern[str], str]] = [<br>    (re.compile(r&#39;&quot;0\.0\.0\.0/0&quot;&#39;), &#39;&quot;10.0.0.0/8&quot;&#39;),<br>    (re.compile(r&#39;&quot;::/0&quot;&#39;),         &#39;&quot;fc00::/7&quot;&#39;),<br>]<br><br>def _sanitize_hcl(hcl: str) -&gt; str:<br>    for pattern, replacement in _CIDR_SANITISATIONS:<br>        hcl = pattern.sub(replacement, hcl)<br>    return hcl<br><br>clean_hcl = _sanitize_hcl(output.terraform_hcl)</pre><p>10.0.0.0/8 is a broad internal placeholder. Operators narrow it before deploying to production.</p><p>This single function broke the HCL validation loop for the most common case. First-pass generations stopped triggering the guardrail on CIDR issues almost entirely. When the guardrail fires now, it catches a genuine structural problem.</p><p><strong>When a model reliably produces the same wrong output, fix it deterministically. Do not prompt your way out of a consistency problem.</strong></p><h3>Three Questions Before the LLM Sees Anything</h3><p>The most expensive mistake in an agentic pipeline is burning tokens on requests that should never reach the agents. InfraSquad catches these at three layers before anything expensive runs.</p><figure><img alt="Three layers of input validation-chitchat detection, keyword matching, and LLM classification as a narrowing funnel" src="https://cdn-images-1.medium.com/max/1024/1*i_EkxKKjL-sIoTEZwFB1ug.png" /><figcaption>Source: Image by the author.</figcaption></figure><p><strong>Layer 1: Chitchat detection (zero cost)</strong></p><p>A frozenset of 40+ conversational tokens returns immediately. “Thanks”, “ok cool”, “sounds good”, a thumbs-up emoji-none of these should reach the architect.</p><pre>_CHITCHAT_TOKENS: frozenset[str] = frozenset({<br>    &quot;cool&quot;, &quot;okay&quot;, &quot;ok&quot;, &quot;sure&quot;, &quot;thanks&quot;, &quot;thank you&quot;,<br>    &quot;hi&quot;, &quot;hello&quot;, &quot;great&quot;, &quot;awesome&quot;, &quot;got it&quot;, &quot;yep&quot;, ...<br>})</pre><pre>def _is_chitchat(text: str) -&gt; bool:<br>    normalized = text.strip().lower().rstrip(&quot;!.,?&quot;)<br>    tokens = [t.strip(&quot;!.,?&quot;) for t in normalized.split()]<br>    return bool(tokens) and len(tokens) &lt;= 4 and all(t in _CHITCHAT_TOKENS for t in tokens)</pre><p>No LLM call. No latency. Instant return.</p><p><strong>Layer 2: Keyword matching (zero cost)</strong></p><p>A compiled regex matches 45 AWS infrastructure keywords. Two or more matches skip the LLM check entirely. “VPC with RDS Postgres and ALB” is obviously a valid request. Spending 2 seconds and tokens to confirm this is wasteful.</p><pre>elif keyword_match_count(user_request) &gt;= 2:<br>    # High confidence-skip the LLM round-trip (~2s saved per clear request)<br>    is_valid = True</pre><p>About 70% of valid requests take this fast path.</p><p><strong>Layer 3: LLM plausibility (borderline cases only)</strong></p><p>Single-keyword matches are genuinely ambiguous. “Server” could be valid. “AWS tomato server” should not be. For these, a lightweight LLM call returns one of three outcomes:</p><pre>class _FirstMessageClassification(BaseModel):<br>    intent: Literal[&quot;proceed&quot;, &quot;clarify&quot;, &quot;reject&quot;]</pre><p>clarify triggers a helpful guidance message. reject returns a polite explanation. Both avoid running the full pipeline on nonsense input.</p><p>There is a catch. In active conversations, keyword matching stops working correctly. “Explain the Terraform code” contains the word “Terraform” but is clearly a follow-up question, not a new generation request. So in active sessions, we switch to a full intent classifier that distinguishes new_generation, follow_up, and off_topic.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*inFy6ctXVzQkqIGNeSUYFw.png" /><figcaption>Source: Image by the author.</figcaption></figure><h3>The Security Check That Does Not Trust the LLM</h3><p>Two patterns are blocked by hardcoded regex, independent of everything else:</p><pre>_ADMIN_ACCESS_PATTERN = re.compile(r&quot;AdministratorAccess&quot;, re.IGNORECASE)<br>_STAR_POLICY_PATTERN  = re.compile(r&#39;&quot;Action&quot;\s*:\s*&quot;\*&quot;&#39;)</pre><p>AdministratorAccess policies and wildcard IAM actions are blocked regardless of what the model thought it generated. Not by the HCL guardrail. Not by the Security Auditor. By a function that runs on every output, unconditionally.</p><p>The reason for running this separately from the guardrail: the HCL guardrail checks for AdministratorAccess in a string pattern that could miss an IAM policy embedded inside a heredoc JSON block. The standalone regex catches it regardless of context.</p><p>Two independent checks. Neither relying on the other being correct.</p><h3>The HCL Guardrail: Before Security Even Runs</h3><p>Before the Terraform reaches the Security Auditor, it passes through a deterministic validator. This runs on every generation-first pass and every remediation:</p><pre>_FORBIDDEN_PATTERNS = [<br>    (r&quot;AdministratorAccess&quot;, &quot;Uses AdministratorAccess IAM policy&quot;),<br>    (r&quot;0\.0\.0\.0/0&quot;,        &quot;Contains 0.0.0.0/0 CIDR-opens resource to the internet&quot;),<br>    (r&quot;public\s*=\s*true&quot;,   &quot;Sets public access to true&quot;),<br>]</pre><p>It also checks structural validity: provider and resource blocks must be present, resource signatures must be well-formed, and braces must balance. Any failure sends the code back to the DevOps agent with the specific error list attached to the next prompt.</p><p>The CIDR sanitizer runs before this check. That is intentional. Remove 0.0.0.0/0 before validation, so the guardrail only fires on real structural problems.</p><h3>What the Pipeline Actually Produces</h3><p>Here is a real run. Request: “VPC with an RDS Postgres instance, an Application Load Balancer, and a Redis caching layer.”</p><p>The DevOps agent follows a security baseline baked into its system prompt. S3 buckets get KMS encryption, versioning, and public access blocks by default. RDS gets storage_encrypted = true, deletion_protection = true, and 7-day backup retention. ElastiCache gets encryption at rest and in transit. VPCs get flow logs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vS0hYaRNjNEVVUClWawqcg.png" /><figcaption>Source: Image by the author.</figcaption></figure><p>After the security check passes, the Visualizer reads the finalized plan and code and generates a Mermaid diagram:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N3KAml74oIihoOKm2qEXKg.png" /><figcaption>Source: Image by the author.</figcaption></figure><p>If mmdc is not installed, the Mermaid source is saved as-is. It is still fully useful-paste it into any Mermaid viewer and you get the diagram.</p><h3>The Security Audit Loop: How It Actually Works</h3><p>When the Security Auditor finds issues, it does not just list them. It produces a structured prompt that becomes the DevOps agent’s next input:</p><pre>MANDATORY SECURITY REMEDIATION-3 finding(s)<br>Fix EVERY numbered item below. Do NOT skip any.<br><br>Finding 1. [HIGH] AVD-AWS-0107 - aws_security_group.app_sg<br>   Issue: Security group allows unrestricted ingress on port 443<br>   Fix:   Restrict ingress to specific CIDR ranges or security group references.<br>Finding 2. [HIGH] AVD-AWS-0132 - aws_s3_bucket.assets<br>   Issue: S3 bucket does not use KMS encryption with a customer-managed key<br>   Fix:   Add aws_kms_key + aws_s3_bucket_server_side_encryption_configuration</pre><p>The DevOps agent sees MANDATORY SECURITY REMEDIATION in its next prompt and treats every numbered item as a required fix.</p><p>security_passed = True only when there are zero CRITICAL and zero HIGH findings. MEDIUM and LOW findings get reported but do not block the pipeline. The visualization still renders.</p><h3>Why LangGraph Over CrewAI or AutoGen</h3><p>This came down to one question: does the framework support cycles with explicit state management and typed contracts?</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9e5761d897e114668be33d7c3b207ec3/href">https://medium.com/media/9e5761d897e114668be33d7c3b207ec3/href</a></iframe><p>The security remediation loop is cyclic by design. The Security Auditor sends the DevOps Engineer back; the DevOps Engineer generates new code; the new code gets re-scanned. Both CrewAI and AutoGen require workarounds for this pattern. LangGraph’s conditional edges handle it natively.</p><p>The typed state was also non-negotiable. Without a clear contract on what each agent receives and produces, integration failures are silent. An agent gets None where it expected a string and fails three nodes downstream with a cryptic error. TypedDict with total=False gives every agent a contract it cannot accidentally break.</p><h3>External Tools Through MCP</h3><p>tfsec and mmdc (Mermaid rendering) run as MCP tools, not direct imports. The agent calls a tool through a protocol.</p><p>This looks like over-engineering for a project at this scale. The argument for it: tfsec and mmdc are external processes that can timeout, crash, or produce unexpected output. Wrapping them in an MCP tool forces explicit failure handling at every call site.</p><pre>result = _try_tfsec(tmpdir) or _try_checkov(tmpdir)</pre><p>tfsec unavailable? Try checkov. checkov unavailable? LLM security review with the full security system prompt. mmdc unavailable? Save Mermaid source. Every external dependency ended up with a fallback path, which would not have happened if they were direct imports.</p><p>The MCP server also runs independently. It can be swapped or extended without touching any agent code.</p><h3>Five Things We’d Tell Ourselves at the Start</h3><p><strong>1. Hard-cap your cycles before the first integration test.</strong> You will hit the infinite loop case. Probably on a public-facing resource where the security finding is technically correct and architecturally intentional. Add the counter before you need it.</p><p><strong>2. Regex beats prompting for deterministic security invariants.</strong> If a property can be expressed as a pattern, enforce it with code. LLM compliance on security constraints is probabilistic. Code compliance is guaranteed. The CIDR sanitizer took 10 lines to write and immediately reduced first-pass HCL failures by the majority.</p><p><strong>3. Typed state is not optional in multi-agent systems.</strong> Silent failures are the worst kind. A TypedDict with total=False is a contract every agent signs. Without it, you are debugging None errors three nodes downstream and trying to reconstruct which agent set which field when.</p><p><strong>4. Pydantic schema retry saves more than you expect.</strong> Without invoke_with_schema_retry, the pipeline fails silently on every malformed JSON response. With it, about 80% of schema failures resolve on the first retry with an error correction prompt. Make this load-bearing from day one.</p><p><strong>5. Input validation saves more tokens than it looks like it will.</strong> Chitchat and off-topic requests are common in demo environments. Every one that reaches the architect burns tokens before returning an unhelpful or confusing response. The three-layer guardrail means only genuine infrastructure requests reach the expensive part of the pipeline.</p><h3>Run It Yourself</h3><pre>git clone https://github.com/Andela-AI-Engineering-Bootcamp/infrasquad.git<br>cd infrasquad<br>uv venv &amp;&amp; uv sync</pre><p>Add your OpenRouter API key to .env:</p><pre>OPENROUTER_API_KEY=sk-or-...</pre><p>Start the UI:</p><pre>python app.py              # localhost:7860<br>python app.py --share      # public Gradio URL<br>python app.py --port 8080  # custom port</pre><p>The default model is openai/gpt-4o-mini via OpenRouter. Swap to any model OpenRouter supports by changing two env vars:</p><pre>LLM_MODEL=anthropic/claude-3-5-sonnet<br>LLM_BASE_URL=https://openrouter.ai/api/v1</pre><p>Or point it at a local Ollama instance:</p><pre>LLM_MODEL=qwen2.5:72b<br>LLM_BASE_URL=http://localhost:11434/v1</pre><p>Optional tools for real scanner output and rendered diagrams:</p><pre>brew install tfsec<br>pip install checkov<br>npm install -g @mermaid-js/mermaid-cli</pre><p>If none of these are installed, the pipeline still completes. Security falls back to LLM review and diagrams save as Mermaid source.</p><p>Built with: <em>LangGraph, Python 3.12+, OpenRouter, FastMCP, Gradio, pydantic-settings, tfsec/checkov, Mermaid.js &amp; uv package manager</em></p><p>Full source - <a href="https://github.com/Andela-AI-Engineering-Bootcamp/infrasquad"><strong><em>infrasquad</em></strong></a></p><p>If you are building anything with <a href="https://github.com/langchain-ai/langgraph">LangGraph</a>, multi-agent pipelines, or IaC automation, drop a comment. Especially curious whether anyone else hit the public ALB infinite loop case-and how you handled it.</p><blockquote>This article was originally published on my <a href="https://dev.to/baremetal-dev/we-let-ai-write-our-terraform-then-we-gave-it-a-security-conscience-480e">Dev.to</a> blog.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8a6f37f59eab" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/we-let-ai-write-our-terraform-then-we-gave-it-a-security-conscience-8a6f37f59eab">Beyond the Prompt: Building a Multi-Agent DevOps Squad with a Security Conscience</a> was originally published in <a href="https://pub.towardsai.net">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>