← Back to Technical Section

Tool Calling

How agents safely and reliably call external tools and functions: designing interfaces, validation, execution patterns and observability for LLM-driven systems.

Overview

"Tool calling" refers to an LLM or agent invoking external functionality (APIs, database queries, shell commands, or domain-specific tools) to extend its capabilities beyond text generation. Properly designed tool calling ensures correctness, safety, audibility and repeatability when language models take actions in the world.

Why it matters: Tool calling turns a static LLM into an agent that can fetch fresh data, perform transactions, or orchestrate other services — but it introduces risks (injection, unsafe side effects, and incorrect input) that must be mitigated.

Design Patterns

Function Spec + Schema

Define each tool with a strict schema (types, required fields, allowed values). Provide this spec to the model so it outputs structured arguments instead of free text.

Planner / Executor Separation

Keep planning (deciding which tool to call and with what arguments) separate from execution. Validators sit between planner and executor.

Dry-Run & Simulation

Support a simulation mode to show intended calls without side effects — useful for review and testing.

Least Privilege & Escaping

Limit tools' permissions and sanitize all inputs. Prefer typed parameters over string interpolation.

Tool Specification Example

A concise JSON schema describing a tool the agent can call. Provide this spec to the model as part of the prompt or via the API's function calling interface.

{
  "name": "get_customer_profile",
  "description": "Fetch customer profile by id",
  "parameters": {
    "type": "object",
    "properties": {
      "customer_id": { "type": "string", "description": "UUID of the customer" }
    },
    "required": ["customer_id"]
  }
}

Safety & Validation

Validate types and ranges server-side before executing any tool.
Enforce whitelists for allowed operations and endpoints; block high-risk actions.
Use intent confirmation for destructive actions (two-step confirmation or human-in-the-loop).
Keep an immutable audit log of requested calls, inputs, responses and the agent's decision rationale.

Execution Patterns

Synchronous Execution

Call the tool and return its result inline. Use for fast, idempotent operations.

Asynchronous / Background Jobs

Queue long-running or external workflows and return a tracking id; allow the agent or user to poll status.

Retries and Circuit Breakers

Apply exponential backoff, idempotency keys, and circuit breakers to avoid cascading failures.

Observability & Testing

Track metrics and enable tests specific to tool calls.

Metrics

success rate, latency, error codes, cost

Logging

structured logs with request/response payloads

Testing

unit tests for validators, integration tests with sandboxed tools

Minimal Example (Pseudo-code)

// 1) Model returns a structured call
{ "tool": "get_customer_profile", "args": { "customer_id": "1234-uuid" } }

// 2) Server-side validator checks schema and permissions
// 3) Executor calls the API and returns result
// 4) Audit log stored; result passed back to model or user