Calling an LLM from Spring Boot with Spring AI 2.0: chat, streaming and structured output
A tested Spring Boot 4.1 + Spring AI 2.0 example with a local Ollama model: a plain answer, a streamed answer and an answer mapped to a Java record, plus the configuration changes that break older tutorials.

Spring AI gives Spring Boot applications one API, the ChatClient, for talking to language models from OpenAI, Anthropic, Google, Mistral, Ollama and others. In this guide we build a small REST service with three endpoints:
/askreturns the model's whole answer as text;/ask/streamstreams the answer token by token, so the user sees it appear;/adviceasks for a structured answer and maps it onto a Javarecord.
We use Ollama, which runs open models on your own machine: no API key, no cost per request, and nothing leaves your laptop. Switching to a hosted provider later is a dependency and a few properties, as shown at the end.
Versions used (checked on Maven Central on 30 October 2026): Spring Boot 4.1.2, Spring AI 2.0.2, Java 21. Spring AI 2.1 and Spring Boot 4.2 were still release candidates on that date. The complete project is on GitHub and compiles in CI.
Prerequisites
- Java 21 and Maven.
- Ollama installed and running (it listens on
http://localhost:11434). - A model. We tested with
llama3.2:1b(about 1.3 GB) because it is fast on a laptop;llama3.2(3B) gives better answers.
ollama pull llama3.2
The Maven setup
Import the Spring AI BOM so every Spring AI module gets the same version, then add the starter for your model provider. The Ollama starter brings the auto-configuration and the ChatClient.
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>4.1.2</version>
</parent>
<properties>
<java.version>21</java.version>
<spring-ai.version>2.0.2</spring-ai.version>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-bom</artifactId>
<version>${spring-ai.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-starter-model-ollama</artifactId>
</dependency>
</dependencies>
Configuration: the property name changed in 2.0
This is where many older tutorials stop working. In Spring AI 2.0 the model and temperature moved one level up:
| Spring AI 1.x (still accepted, deprecated) | Spring AI 2.0 |
|---|---|
spring.ai.ollama.chat.options.model |
spring.ai.ollama.chat.model |
spring.ai.ollama.chat.options.temperature |
spring.ai.ollama.chat.temperature |
We checked this in the configuration metadata shipped inside spring-ai-autoconfigure-model-ollama-2.0.2.jar: the old names are marked deprecated with the new ones as replacement. Use the new names:
spring.application.name=spring-ai-chat
spring.ai.ollama.base-url=http://localhost:11434
spring.ai.ollama.chat.model=llama3.2
spring.ai.ollama.chat.temperature=0.2
# Download the model on first start if it is missing (default: never)
spring.ai.ollama.init.pull-model-strategy=when_missing
A low temperature (0.2) makes answers more repeatable, which is what you want for most application features. The default pull-model-strategy is never: without the last line, the app fails on the first request if the model has not been pulled.
Building the ChatClient
Spring Boot auto-configures a ChatClient.Builder for the provider on the classpath. Inject the builder, set the defaults you want for every request (here a system prompt), and build the client once:
@RestController
public class ChatController {
private final ChatClient chatClient;
public ChatController(ChatClient.Builder builder) {
this.chatClient = builder
.defaultSystem("You are a concise assistant for Java developers. Answer in plain text.")
.build();
}
}
The system prompt is sent with every request. Keep it short: you pay for it, in time or in money, on every call.
1. A plain answer
@GetMapping("/ask")
public String ask(@RequestParam String q) {
return chatClient.prompt()
.user(q)
.call()
.content();
}
call() waits for the complete answer; content() returns it as a String.
curl "localhost:8080/ask?q=What%20is%20a%20Java%20record%3F%20One%20sentence."
2. Streaming the answer
Language models produce text token by token. Waiting for the whole answer can take several seconds, so for anything a person reads, stream it:
@GetMapping(value = "/ask/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> askStream(@RequestParam String q) {
return chatClient.prompt()
.user(q)
.stream()
.content();
}
stream() returns a Reactor Flux<String>. Even in a classic Spring MVC application (we use spring-boot-starter-web, not WebFlux), returning a Flux with text/event-stream sends each piece to the browser as a Server-Sent Event:
curl -N "localhost:8080/ask/stream?q=Name%20three%20Java%20build%20tools."
# data:Here
# data: are
# data: three
# ...
In the browser, read it with EventSource or with fetch and a stream reader.
3. Structured output into a Java record
Often you don't want prose, you want data. entity() asks the model for JSON that matches your type and converts it:
public record LibraryAdvice(String library, String reason, List<String> alternatives) {}
@GetMapping("/advice")
public LibraryAdvice advice(@RequestParam String task) {
return chatClient.prompt()
.user(u -> u.text("Suggest one Java library for this task: {task}").param("task", task))
.call()
.entity(LibraryAdvice.class);
}
Spring AI adds format instructions with a JSON schema derived from the record to the prompt, then parses the reply. The {task} placeholder is filled by param(), which keeps user input separate from your template text.
Validate what comes back
This is the most important lesson from our test run. With the small llama3.2:1b model, the /advice endpoint returned perfectly valid JSON:
{"library":"org.apache.commons.csv","reason":"CSV file parser",
"alternatives":["org.apache.commons.csv.CSVParser","org.json.simple.CSVParser"]}
The structure is right, but org.json.simple.CSVParser does not exist. In the same session the model also said that records arrived in Java 14 (they were a preview there; they became final in Java 16). Correct format does not mean correct content. Before you act on a structured answer:
- validate fields with Bean Validation or your own checks (does this library exist in your allowed list? is this ID in your database?);
- keep the temperature low and prefer a larger model for factual tasks;
- treat the output as a suggestion for a human, or ground it with your own data (retrieval), when mistakes are expensive.
Retries: check the defaults
Spring AI retries failed model calls through spring-ai-retry. The defaults in 2.0.2 (from the configuration metadata) are spring.ai.retry.max-attempts=10, spring.ai.retry.backoff.initial-interval=2000ms and spring.ai.retry.backoff.multiplier=5. That is fine for a batch job, but for a user waiting on an HTTP request it can turn one failure into a very long hang. For interactive endpoints, lower them:
spring.ai.retry.max-attempts=3
spring.ai.retry.backoff.initial-interval=500ms
spring.ai.retry.backoff.multiplier=2
spring.ai.retry.backoff.max-interval=5s
If you call a hosted API directly, our guide on handling HTTP 429 with Retry-After explains what to retry and how long to wait, and the backoff simulator shows how different settings spread the retries over time.
Switching to a hosted model
The controller code does not change. Replace the starter and the properties, for example for Anthropic:
<dependency>
<groupId>org.springframework.ai</groupId>
<artifactId>spring-ai-starter-model-anthropic</artifactId>
</dependency>
The same pattern works with spring-ai-starter-model-openai and the other model starters (all published for 2.0.2). Each provider has its own spring.ai.<provider>.* properties for the API key and the model name: take the exact names from the provider's page in the Spring AI reference documentation, and keep the API key in an environment variable, never in application.properties committed to Git.
Before production
- Timeouts: a model can take tens of seconds. Set timeouts on the HTTP client and on your own endpoint.
- Costs and limits: hosted APIs bill per token and enforce rate limits. Log token usage and put limits on the size of user input.
- Logs: prompts and answers can contain personal data. Before sharing logs, remove secrets and personal data (our log and HAR redactor runs in the browser).
- Errors: map model failures to clear HTTP statuses for your own clients; see which status to return.
The complete example, with the three endpoints and the configuration above, is in our GitHub repository. The ChatClient chapter of the Spring AI reference covers advisors, memory and tool calling, the next steps after this guide.
The complete, tested example for this article is in our public repository: adamant-solutions/ai-dev-examples/spring-ai-chat