agent-runtime is both an executable binary that can be run, and a library that can be used in Rust programs.

Installing `agent_with_tools_demo` `async_events_demo` `complex_viz` `config_demo` `hello_workflow` `llama_demo` `llm_demo` `mcp_tools_demo` `mermaid_viz` `multi_subscriber` `native_tools_demo` `nested_workflow` `production_features_demo` `step_types_demo` `workflow_demo` executables

Assuming you have Rust/Cargo installed, run this command in a terminal:

cargo install agent-runtime

It will make agent_with_tools_demo async_events_demo complex_viz config_demo hello_workflow llama_demo llm_demo mcp_tools_demo mermaid_viz multi_subscriber native_tools_demo nested_workflow production_features_demo step_types_demo workflow_demo commands available in your PATH if you've allowed the PATH to be modified when installing Rust. cargo uninstall agent-runtime uninstalls.

Adding `agent_runtime` library as a dependency

Run this command in a terminal, in your project's directory:

cargo add agent-runtime

To add it manually, edit your project's Cargo.toml file and add to the [dependencies] section:

agent-runtime = "0.3.0"

The agent_runtime library will be automatically available globally. Read the agent_runtime library documentation.

Back to the crate overview.

Readme

agent-runtime

A production-ready Rust framework for building AI agent workflows with native and external tool support, streaming LLM interactions, comprehensive event tracking, and intelligent loop prevention.

Features

🤖 Agent System

LLM-backed agents with configurable system prompts and context
Multi-provider LLM support - OpenAI and llama.cpp (LM Studio) included
Streaming responses - Real-time token-by-token LLM output
Tool loop prevention - Automatic detection and prevention of redundant tool calls
Execution history - Complete conversation and tool call tracking per agent

🔧 Tool System

Native tools - In-memory async functions with zero overhead
MCP tool integration - Connect to external MCP servers (filesystem, databases, web, etc.)
Tool registry - Organize and manage tools per agent
Automatic discovery - MCP tools auto-discovered from servers
Rich metadata - Full argument schemas and descriptions

🔄 Workflow Engine

Sequential workflows - Chain multiple agents with state passing
Transform steps - Data manipulation between agents
Conditional branching - Dynamic workflow paths
Nested workflows - SubWorkflows for complex orchestration
Mermaid export - Visualize workflows as diagrams

📡 Event System (v0.3.0 - Unified)

Unified event model - Consistent Scope × Type × Status pattern across all components
Complete lifecycle tracking - Started → Progress → Completed/Failed for workflows, agents, LLM requests, tools
Real-time streaming - Live LLM token streaming via Progress events
Multi-subscriber - Multiple event listeners per workflow
Type-safe component IDs - Enforced formats with validation

⚙️ Configuration

YAML and TOML support - Human-readable config files
Builder pattern - Type-safe programmatic configuration
Environment variables - Runtime configuration override
Per-agent settings - System prompts, tools, LLM clients, loop prevention

🔒 Production Ready

97 comprehensive tests - All core functionality tested
Tool loop prevention - Prevents LLM from calling same tool repeatedly with System::Progress events
Microsecond timing - Precise performance metrics via event data
Async event emission - Non-blocking event streaming with tokio::spawn
Error handling - Detailed error types with context and human-readable messages

Quick Start

Installation

[dependencies]
agent-runtime = { path = "." }
tokio = { version = "1", features = ["full"] }

Basic Agent

use agent_runtime::prelude::*;

#[tokio::main]
async fn main() {
    // Create LLM client
    let llm = OpenAiClient::new("https://api.openai.com/v1", "your-api-key");
    
    // Build agent with tools
    let agent = AgentConfig::new("assistant")
        .with_system_prompt("You are a helpful assistant.")
        .with_llm_client(Arc::new(llm))
        .with_tool(calculator_tool())
        .build();
    
    // Execute
    let input = AgentInput::from_text("What is 42 * 137?");
    let output = agent.execute(&input).await?;
    println!("Result: {}", output.data);
}

MCP External Tools

use agent_runtime::tools::{McpClient, McpTool};

// Connect to MCP server
let mcp = McpClient::new_stdio(
    "npx",
    vec!["-y", "@modelcontextprotocol/server-filesystem", "/tmp"]
).await?;

// Discover tools
let tools = mcp.list_tools().await?;
println!("Available: {:?}", tools.iter().map(|t| &t.name).collect::<Vec<_>>());

// Use in agent
let agent = AgentConfig::new("file-agent")
    .with_mcp_tools(Arc::new(mcp))
    .build();

Workflow

let workflow = Workflow::new("analysis")
    .add_step(AgentStep::new(researcher_agent))
    .add_step(TransformStep::new(|output| {
        // Transform data between agents
        AgentInput::from_text(format!("Summarize: {}", output.data))
    }))
    .add_step(AgentStep::new(summarizer_agent))
    .build();

let result = workflow.execute(initial_input, &mut event_rx).await?;

Event Streaming (v0.3.0)

use agent_runtime::{EventScope, EventType};

let (tx, mut rx) = mpsc::channel(100);

// Subscribe to events
tokio::spawn(async move {
    while let Some(event) = rx.recv().await {
        match (event.scope, event.event_type) {
            // Stream LLM responses in real-time
            (EventScope::LlmRequest, EventType::Progress) => {
                if let Some(chunk) = event.data["chunk"].as_str() {
                    print!("{}", chunk);
                }
            }
            // Track tool executions
            (EventScope::Tool, EventType::Completed) => {
                println!("✓ Tool {} returned: {}", 
                    event.component_id,
                    event.data["result"]
                );
            }
            // Handle failures
            (_, EventType::Failed) => {
                eprintln!("❌ {}: {}",
                    event.component_id,
                    event.message.unwrap_or_default()
                );
            }
            _ => {}
        }
    }
});

agent.execute_with_events(&input, &tx).await?;

Configuration Files

# agent-runtime.yaml
agents:
  - name: researcher
    system_prompt: "You are a research assistant."
    max_iterations: 10
    tool_loop_detection:
      enabled: true
      custom_message: "Previous {tool_name} call returned: {previous_result}"
    
  - name: analyzer
    system_prompt: "You analyze data."
    tool_loop_detection:
      enabled: false  # Disable if needed

let config = RuntimeConfig::from_file("agent-runtime.yaml")?;

Architecture

Core Modules

runtime - Workflow execution engine with event emission
workflow - Builder pattern for composing steps
agent - LLM-backed agents with tool execution loop
step - Trait for workflow steps (Agent, Transform, Conditional, SubWorkflow)
llm - Provider-agnostic chat client (OpenAI, llama.cpp)
tool - Native tool trait and registry
tools/mcp_client - MCP protocol client for external tools
event - Event types and streaming system
config - YAML/TOML configuration loading
tool_loop_detection - Intelligent duplicate tool call prevention

Event System (v0.3.0)

Unified Scope × Type × Status Pattern:

Scopes: Workflow, WorkflowStep, Agent, LlmRequest, Tool, System
Types: Started, Progress, Completed, Failed, Canceled
Status: Pending, Running, Completed, Failed, Canceled

Key Events:

Workflow::Started/Completed/Failed - Overall workflow execution
WorkflowStep::Started/Completed/Failed - Individual step tracking
Agent::Started/Completed/Failed - Agent processing lifecycle
LlmRequest::Started/Progress/Completed/Failed - Real-time LLM streaming
Tool::Started/Progress/Completed/Failed - Tool execution tracking
System::Progress - Runtime behaviors (e.g., tool loop detection)

Component ID Formats:

Workflow: workflow_name
WorkflowStep: workflow:step:N
Agent: agent_name
LlmRequest: agent:llm:N
Tool: tool_name or tool_name:N
System: system:subsystem

Tool Loop Prevention

Prevents LLMs from calling the same tool with identical arguments repeatedly:

Automatic detection - Tracks tool calls and arguments using MD5 hashing
System events - Emits System::Progress event with system:tool_loop_detection component ID
Configurable messages - Custom messages with {tool_name} and {previous_result} placeholders
Enabled by default - Can be disabled per-agent if needed

Examples

Run any demo:

# Event System
cargo run --bin async_events_demo    # NEW! Async event streaming demo with visible sequence

# Workflows
cargo run --bin workflow_demo          # 3-agent workflow with LLM
cargo run --bin hello_workflow         # Simple sequential workflow
cargo run --bin nested_workflow        # SubWorkflow example

# Agents & Tools
cargo run --bin agent_with_tools_demo  # Agent with calculator & weather
cargo run --bin native_tools_demo      # Standalone native tools
cargo run --bin mcp_tools_demo         # MCP external tools

# LLM Clients
cargo run --bin llm_demo               # OpenAI client
cargo run --bin llama_demo             # llama.cpp/LM Studio

# Configuration
cargo run --bin config_demo            # YAML/TOML loading

# Visualization
cargo run --bin mermaid_viz            # Generate workflow diagrams
cargo run --bin complex_viz            # Complex workflow diagram

Documentation

Event Streaming Guide - Complete event system documentation (v0.3.0)
Migration Guide - Upgrading from v0.2.x to v0.3.0
Changelog - Release notes for v0.3.0
Specification - Complete system design
Tool Calling - Native tool usage
MCP Integration - External MCP tools
LLM Module - LLM provider integration
Workflow Composition - Building workflows
Testing - Test suite documentation

Testing

cargo test              # All 97 tests
cargo test --lib        # Library tests only
cargo test agent        # Agent tests
cargo test tool         # Tool tests
cargo test event        # Event system tests
cargo clippy            # Linting
cargo fmt --all         # Format code

What's New in v0.3.0

🎉 Unified Event System - Complete rewrite for consistency and extensibility

Breaking Changes: New event structure with EventScope, ComponentStatus, unified EventType
Helper Methods: 19 ergonomic helper methods for common event patterns
Component IDs: Enforced formats with validation for type safety
Async Events: Non-blocking event emission via tokio::spawn()
Migration Guide: See docs/MIGRATION_0.2_TO_0.3.md

Upgrading from v0.2.x?

// Old (v0.2.x)
match event.event_type {
    EventType::AgentLlmStreamChunk => { ... }
}

// New (v0.3.0)
match (event.scope, event.event_type) {
    (EventScope::LlmRequest, EventType::Progress) => { ... }
}

See CHANGELOG.md for complete details.

License

Dual-licensed under MIT or Apache-2.0 at your option.

Installing agent_with_tools_demo async_events_demo complex_viz config_demo hello_workflow llama_demo llm_demo mcp_tools_demo mermaid_viz multi_subscriber native_tools_demo nested_workflow production_features_demo step_types_demo workflow_demo executables

Adding agent_runtime library as a dependency

Installing `agent_with_tools_demo` `async_events_demo` `complex_viz` `config_demo` `hello_workflow` `llama_demo` `llm_demo` `mcp_tools_demo` `mermaid_viz` `multi_subscriber` `native_tools_demo` `nested_workflow` `production_features_demo` `step_types_demo` `workflow_demo` executables

Adding `agent_runtime` library as a dependency