Tool Calling with Spring AI

Large language models are great at reasoning over text, but on their own they can't check a live order status, hit your database, or call an internal API. Tool calling closes that gap: you expose a few Java methods, and Claude decides when to call them to answer a question. Spring AI makes this remarkably clean — you annotate a method, hand it to the ChatClient, and the framework runs the whole request/response loop for you.

What "tool calling" actually does

The model never runs your code directly. Instead:

1. You send a prompt plus the list of available tools (name, description, parameters).
2. If the model needs one, it replies with a tool request instead of a final answer.
3. Your app executes the method and sends the result back.
4. The model uses that result to produce its final reply — repeating if it needs more.

Spring AI handles steps 2–4 automatically.

1. Add the Anthropic starter

<dependency>
  <groupId>org.springframework.ai</groupId>
  <artifactId>spring-ai-starter-model-anthropic</artifactId>
</dependency>

spring.ai.anthropic.api-key=${ANTHROPIC_API_KEY}
spring.ai.anthropic.chat.options.model=claude-sonnet-4-6

2. Define a tool

A tool is just a Spring bean method annotated with @Tool. The description is what the model reads to decide when to call it — treat it like an API contract, not an afterthought.

@Component
class OrderTools {

    private final OrderService orders;

    OrderTools(OrderService orders) {
        this.orders = orders;
    }

    @Tool(description = "Look up the current status and ETA of a customer order by its ID")
    OrderStatus orderStatus(
        @ToolParam(description = "The order ID, e.g. ORD-1234") String orderId) {
        return orders.status(orderId);
    }
}

record OrderStatus(String orderId, String state, String eta) {}

3. Wire it into the ChatClient

@Configuration
class ChatConfig {

    @Bean
    ChatClient chatClient(ChatClient.Builder builder, OrderTools orderTools) {
        return builder
            .defaultSystem("You are a concise customer-support assistant.")
            .defaultTools(orderTools)
            .build();
    }
}

4. Ask a question — the loop runs itself

String reply = chatClient.prompt()
    .user("Where is my order ORD-1234 and when will it arrive?")
    .call()
    .content();

Under the hood the conversation looks like this:

5. Bonus: structured output

You can map the reply straight onto a record — no manual JSON parsing:

record Resolution(String summary, boolean escalate) {}

Resolution r = chatClient.prompt()
    .user("Summarise order ORD-1234 and say whether it needs escalation.")
    .call()
    .entity(Resolution.class);

Production tips

• Descriptions are the routing signal. Vague descriptions cause the model to pick the wrong tool. Be specific about when to use each one and what it returns.
• Keep the toolset small per assistant — a handful, not dozens. Too many choices degrade selection reliability; split specialised work across separate assistants.
• Validate inputs and make writes idempotent. The model chooses the arguments, so treat them like untrusted input and guard money-moving actions with real checks rather than trusting the prompt.
• Return structured errors ("not found" vs "service down") so the model can recover or escalate gracefully instead of guessing.

Wrap-up

Tool calling is the bridge between a language model and your real systems — and with Spring AI it's barely more than a @Tool annotation and a builder call. Start with one well-described tool, watch the loop work, then layer in structured output and guardrails as you move toward production.

Related reading

• Spring AI Enterprise Integration Guide — the broader picture: chat clients, RAG pipelines, and production observability with Spring AI.