AI4JVM — Java & JVM AI Ecosystem Guide

12 min read Original article ↗

Latest Headlines

Agent Frameworks & Libraries

Open-source frameworks and SDKs for building AI-powered applications on the JVM — from full agent platforms to Model Context Protocol implementations.

Framework

Spring AI

The Spring ecosystem's official AI framework. Portable abstractions across 20+ model providers, tool calling, RAG, chat memory, vector stores, and MCP support. Built by the Spring team at Broadcom.

Framework

LangChain4j

The most popular Java LLM library. Unified API across 20+ LLM providers and 20+ embedding stores. Three levels of abstraction from low-level prompts to high-level AI Services. Supports RAG, tool calling, MCP, and agents.

Framework

Embabel

Created by Rod Johnson (Spring Framework creator). JVM agent framework using Goal-Oriented Action Planning (GOAP) for dynamic replanning. Strongly typed, Spring-integrated, MCP support. Written in Kotlin with full Java interop.

Framework

Google ADK for Java

Google's Agent Development Kit — code-first Java toolkit for building, evaluating, and deploying AI agents. Supports Gemini natively plus third-party models via LangChain4j integration. A2A protocol for agent-to-agent communication.

Framework

Quarkus LangChain4j

Enterprise-grade Quarkus extension for LangChain4j. Native compilation with GraalVM, built-in observability (metrics, tracing, auditing), and Dev UI tooling. Maintained by Red Hat & IBM.

Framework

Helidon LangChain4j

Oracle's Helidon framework integration with LangChain4j. Declarative AI Services via Helidon Inject, build-time code generation for GraalVM native images, streaming chat over Java Streams, guardrails, built-in metrics, and agentic support (workflows and dynamic agents). Runs on virtual threads.

Framework

Helidon MCP

Helidon's Model Context Protocol server and client implementation. Declarative and imperative APIs for building MCP servers with tools, resources, and prompts. Streamable HTTP and SSE transports, virtual threads, build-time processing. From Oracle's Helidon team.

Framework

LangChain4j-CDI

CDI extension for LangChain4j (part of the LangChain4j project) that brings AI services to Jakarta EE and MicroProfile applications. Inject AI services as CDI beans with @RegisterAIService, configure via MicroProfile Config, and add resilience with Fault Tolerance. Supports Quarkus, Helidon, WildFly, Payara, GlassFish, Liberty, and any CDI-capable runtime.

Framework

LangGraph4j

Build stateful, multi-agent applications with cyclical graphs. Inspired by Python's LangGraph, works with both LangChain4j and Spring AI. Persistent checkpoints, deep agent architectures, and a Studio web UI.

Framework

Akka Agents

Agentic AI platform built on Akka's actor model for distributed, resilient systems. Declarative Effects API for building goal-directed agents with durable memory, multi-agent orchestration, and automatic scaling. MCP and A2A protocol support, pluggable LLM providers, runtime prompt updates, and agents auto-exposed as HTTP, gRPC, or MCP endpoints. Java and Scala SDKs.

Framework

Koog (JetBrains)

Kotlin-native agent framework from JetBrains. Type-safe DSL, multiplatform (JVM, JS, WasmJS, Android, iOS), A2A protocol support, fault tolerance with persistence, and multi-LLM support.

Framework

Semantic Kernel (Java)

Microsoft's AI orchestration SDK with first-class Java support. Provides prompt chaining, planning, memory, and agent framework abstractions with deep Azure integration.

Framework

JamJet

Production-grade agent runtime with native Java SDK. Rust core (Tokio) for performance, graph-based durable workflow orchestration with event-sourced state, automatic crash recovery, audit trails, and first-class human-in-the-loop. Native MCP client/server and A2A protocol support. Java SDK uses records, virtual threads, and fluent builder API. Apache 2.0.

SDK

Spring AI AgentCore SDK

Spring Boot integrations for Amazon Bedrock AgentCore. Auto-configures /invocations and /ping endpoints, SSE streaming, short- and long-term memory, browser automation via Playwright, and a secure code interpreter. Deploy to AgentCore Runtime (managed, scales to zero) or standalone on EKS/ECS.

SDK

MCP Java SDK

The official Java SDK for Model Context Protocol servers and clients. Maintained by the Spring AI team. Sync/async, STDIO/SSE/Streamable HTTP transports, OAuth support via Spring integration.

SDK

Anthropic Java SDK

Official Java SDK for the Claude Messages API. Streaming, retries, structured outputs, extended thinking, code execution, and files API. Build Java apps powered by Claude.

SDK

GitHub Copilot SDK for Java

Official Java SDK for embedding the GitHub Copilot agentic engine directly into Java applications. Uses the same agentic harness that powers the Copilot CLI — exposes planning, tool calling, file editing, and MCP integration via a simple Java API. Currently in technical preview.

Library

Tracy (JetBrains)

AI tracing library for Kotlin and Java. Captures structured traces from LLM interactions — messages, cost, token usage, and execution time. Implements OpenTelemetry Generative AI Semantic Conventions with exports to Langfuse, Weights & Biases, and more.

Library

Docling Java

Official Java client for Docling Serve — invoke document conversion, table detection, formula recognition, reading order analysis, OCR, and more from Java via the Docling Serve backend.

Library

OmniHai

Unified Java AI utility library for Jakarta EE and MicroProfile. Single API across 10 providers with zero external runtime dependencies — just java.net.http.HttpClient. Chat, streaming, structured outputs, web search, translation, and moderation in a lightweight JAR.

Framework

WildFly AI Feature Pack

A feature pack for WildFly, providing seamless LangChain4j-CDI integration and exposing Jakarta EE code as MCP tools via MCP_JAVA Annotations.

Library

MCP_JAVA Annotations

A framework-agnostic Java library providing core annotations and APIs for implementing Model Context Protocol (MCP) servers and clients. Used by WildFly AI Feature Pack and LangChain4j-CDI. Compatible with OpenLiberty, Quarkus, and other Java frameworks.

Framework

Atmosphere

A portable layer across Java AI runtimes. Write @Agent once against a unified API (tool calling, memory, streaming, structured output); swap the runtime — Spring AI, LangChain4j, Google ADK, Embabel, Koog, or built-in OpenAI — by changing one dependency. @Coordinator orchestrates multi-agent fleets with parallel, sequential, and conditional routing. Served over transports (WebTransport/HTTP3, WebSocket, SSE, long-polling, gRPC) and protocols (MCP, A2A, AG-UI). Built by Async-IO.

Java with Code Assistants

Technologies that supercharge Java development when paired with AI code assistants — from MCP servers that give agents live Javadoc access, to reusable skill packages and IDE integrations.

MCP Server

Javadocs.dev MCP Server

Gives AI assistants live access to Java, Kotlin, and Scala library documentation from Maven Central. Six tools including latest-version lookup, Javadoc symbol browsing, and source file retrieval. Connect any MCP client via Streamable HTTP.

Assistant

JetBrains AI

AI-powered coding assistance built into IntelliJ IDEA and all JetBrains IDEs. Context-aware code completion, next-edit suggestions, and an agent-mode chat for refactoring, test generation, and complex tasks. Deep understanding of Java, Kotlin, and Scala project conventions. Supports cloud LLMs (Gemini, OpenAI, Anthropic) plus bring-your-own-key.

Skills

SkillsJars

A packaging format and registry for distributing reusable AI agent skills as Maven/Gradle JARs. Skills are Markdown files (SKILL.md) under META-INF/skills/ that teach AI agents domain-specific patterns. Discover and load skills on demand in Claude Code, Kiro, and Spring AI apps.

Skills

jvm-skills

Curated directory of AI coding skills from JVM ecosystem engineers. Opinionated best-practice guides that AI tools (Claude Code, Cursor, Copilot) use as context — covering Spring Boot, jOOQ, Testcontainers, Docker, and more. Only lists skills that teach AI something it wouldn't know on its own.

Skills

Awesome GitHub Copilot

Awesome Copilot Skills is a curated registry of reusable AI agent skills that developers can plug into agents, providing ready-made capabilities, prompts, and workflows. It helps Java AI developers quickly extend agent functionality without building everything from scratch.

MCP Server

jOOQ MCP Server

Gives AI assistants live access to jOOQ documentation and examples. Connect any MCP client via Streamable HTTP.

 MCP Server

Vaadin MCP Server

Gives AI assistants live access to Vaadin documentation and examples. Connect any MCP client via Streamable HTTP.

Inference & Training

Run models, train classifiers, and do ML inference directly on the JVM — no Python required.

Inference

Deliverance

Deliverance is a Java inference engine capable of generating text, tokenizing input, computing embeddings, and more. Can be used as embedded library inside your Java application or as an HTTP server /chat/completion). Deliverance also provides chat and Rag Chat through vibrant-maven-plugin allowing you to chat with your code!

Inference

Jlama

⚠️ No longer actively maintained. Modern LLM inference engine written in pure Java. Runs Llama, Gemma, Mistral, and more locally on CPU. Uses Java's Vector API (Project Panama) for SIMD-accelerated matrix math. Supports SafeTensors format, quantized models, and distributed inference.

Inference

Deep Java Library (DJL)

AWS's high-level, engine-agnostic deep learning framework. Supports PyTorch, TensorFlow, ONNX Runtime, and XGBoost backends. DJLServing provides high-performance model serving.

Inference

ONNX Runtime Java

Run transformer and classical ML models directly on the JVM. Hardware acceleration via CUDA, DirectML, CoreML, and more. Enables deploying scikit-learn, PyTorch, and HuggingFace models as ONNX in Java without Python at inference time.

Training

Tribuo

Oracle Labs' ML library for classification, regression, clustering, and anomaly detection. Strong typing, provenance tracking for reproducibility, and integrations with XGBoost, ONNX Runtime, TensorFlow, and LibSVM.

Inference

GPULlama3.java

Java-native LLM inference with automatic GPU acceleration via TornadoVM. Supports Llama 3, Mistral, Qwen, Phi-3, and IBM Granite models in GGUF format. TornadoVM translates Java bytecode to GPU kernels (OpenCL, PTX, SPIR-V). From the University of Manchester's Beehive Lab.

Training

TensorFlow Java

Java bindings for TensorFlow, maintained by the TensorFlow JVM SIG. Train and deploy TF models entirely in Java. Available as an optional Tribuo integration. Suitable for teams that want to stay within the JVM ecosystem while using TensorFlow's model formats.

People to Follow

Key voices at the intersection of Java and AI.

Bruno Borges

Bruno Borges

Java Champion

Principal Program Manager — Microsoft Java Engineering Group

Eric Deandrea

Eric Deandrea

Java Champion

Docling Java project lead, contributor to LangChain4j, Sr. Principal Software Engineer at IBM

Markus Eisele

Markus Eisele

Java Champion

Developer Advocate — IBM Research, JavaLand founder

Mario Fusco

Mario Fusco

Java Champion

LangChain4j core team, Sr. Principal Software Engineer at IBM

Antonio Goncalves

Antonio Goncalves

Java Champion

Principal Software Engineer at Microsoft CoreAI, ParisJUG, Devoxx France, Café IA, book author

Frank Greco

Frank Greco

Java Champion

NYJavaSIG founder, AI 4 Java educator, JSR 381 co-author

Rod Johnson

Rod Johnson

Java Champion

Creator of Spring Framework, CEO of Embabel

Kenneth Kousen

Kenneth Kousen

Java Champion

Author of six books including Kotlin Cookbook and Modern Java Recipes. O’Reilly instructor for AI + Java courses. Professor of Practice in Computer Science at Trinity College. President of Kousen IT, Inc.

Guillaume Laforge

Guillaume Laforge

Java Champion

Google Developer Advocate — Java, Groovy, AI

Dmytro Liubarskyi

Dmytro Liubarskyi

Creator of LangChain4j, Principal Architect — IBM

Josh Long

Josh Long

Java Champion

Spring Developer Advocate at Broadcom

T. Jake Luciani

T. Jake Luciani

Creator of Jlama — Java LLM inference

Simon Martinelli

Simon Martinelli

Creator of AI Unfied Process and the jOOQ MCP Server

Mark Pollack

Mark Pollack

Spring AI project lead

Lize Raes

Lize Raes

LangChain4j core team, Developer Advocate at Oracle

K. Siva Prasad Reddy

K. Siva Prasad Reddy

Developer Advocate at JetBrains, author of Beginning Spring Boot 3

Jennifer Reif

Jennifer Reif

Java Champion

Developer Advocate at Neo4j

Oleg Šelajev

Oleg Šelajev

Java Champion

Developer Relations Lead for AI — Docker

Bartosz Sorrentino

Bartosz Sorrentino

LangGraph4j creator, Principal Software Architect

Christian Tzolov

Christian Tzolov

Spring AI lead, MCP Java SDK founder, Spring team at Broadcom

Dan Vega

Dan Vega

Java Champion

Spring Developer Advocate, YouTube educator

Dmitry Vinnik

Dmitry Vinnik

Lead Developer Advocate at Meta

Craig Walls

Craig Walls

Java Champion

Author of Spring AI in Action

James Ward

James Ward

Java Champion

Developer Advocate — Java, Kotlin, Cloud, AI

FAQ

Frequently asked questions about AI development on the JVM.

What is the best Java framework for building AI agents?

The most popular choices are Spring AI and LangChain4j. Spring AI is ideal if you’re already in the Spring ecosystem, offering portable abstractions across 20+ model providers. LangChain4j provides a standalone library with three levels of abstraction, from low-level prompts to high-level AI Services. Other options include Google ADK for Java, Embabel, and Akka Agents — each with different strengths for specific use cases.

Can Java run LLMs locally?

Yes. Projects like Jlama and GPULlama3.java run Llama, Mistral, and other models directly on the JVM. Jlama uses Java’s Vector API for SIMD-accelerated inference on CPU, while GPULlama3.java leverages TornadoVM for GPU acceleration. For production deployments, ONNX Runtime Java supports hardware-accelerated inference across CUDA, DirectML, and CoreML.

What is MCP and how does it work with Java?

The Model Context Protocol (MCP) is an open standard that lets AI assistants interact with external tools and data sources. The official MCP Java SDK, maintained by the Spring AI team, provides both client and server implementations with sync/async support and multiple transports (STDIO, SSE, Streamable HTTP). Helidon MCP and several frameworks also offer MCP support.

Is Kotlin supported by Java AI frameworks?

Yes. Most Java AI frameworks run on any JVM language. Embabel is written in Kotlin with full Java interop, Koog from JetBrains is a Kotlin-native agent framework, and Tracy provides AI observability for Kotlin. LangChain4j and Spring AI work seamlessly from Kotlin code.

Recent & Noteworthy Content, Communities, and Resources

Talks, tutorials, books, and communities for learning AI development on the JVM.

Community

Java Conferences Tracker

Community-maintained calendar of all Java conferences worldwide

 Blog

Java Relevance in the AI Era

RedMonk analysis of Java's position as agent frameworks emerge

 Resource

Awesome Spring AI

Curated list of Spring AI resources, tools, and tutorials

 Book

Spring AI in Action (Manning)

Book by Craig Walls — comprehensive guide to building AI apps with Spring

 Book

Understanding LangChain4j

Book by Antonio Goncalves — explore the fundamentals of AI, learn the history and evolution of AI models, and understand the core concepts of LangChain4j

 Resource

Production LangChain4j — Inside.java

Advanced RAG, agentic workflows, and production tips from Devoxx Belgium

 Resource

Google ADK Java Codelab

Hands-on: build AI agents in Java with Google's ADK

 Videos

Devoxx YouTube

Thousands of conference talks on Java, AI, cloud, and architecture

 Videos

Coffee + Software

Spring ecosystem, AI integration, and Java community

 Resource

Foojay Podcast: Java AI Revolution

Agents, MCP, graph databases — developers navigate the AI revolution

 Workshop

Building Java AI Agents with Spring AI (AWS)

Hands-on AWS workshop for building intelligent AI agents with Spring AI and AWS services, including deployment to EKS

 Livestream

AI & Java on Serverless Office Hours

James Ward and Julian Wood explore building AI-powered Java apps — MCP integration, agent architectures with AgentCore, GraalVM optimization for AI workloads, and secure auth patterns for AI services on serverless