Working with prompts

The Quarkus LangChain4j extension seamlessly integrates Large Language Models (LLMs) into Quarkus applications. LLMs are AI-based systems designed to understand, generate, and manipulate human language, showcasing advanced natural language processing capabilities. Thanks to this extension, we can enable the harnessing of LLM capabilities for the development of more intelligent applications.

In this first chapter, we’ll explore the simplest of interactions with an LLM: Prompting. It essentially means just asking questions to an LLM and receiving an answer in natural language from a given model, such as ChatGPT, Granite, Mistral, etc.

Creating a Quarkus & LangChain4j Application

We’re going to use the langchain4j-openai extension for our first interaction with models. The openai extension supports models that expose the open sourced OpenAI API specification. Several models and model providers expose this API specification. If you want to use a different API spec, then you can likely find a supported extension in the Quarkus documentation.

  • Maven

  • Quarkus CLI

mvn "io.quarkus.platform:quarkus-maven-plugin:create" -DprojectGroupId="com.redhat.developers" -DprojectArtifactId="quarkus-langchain4j-app" -DprojectVersion="1.0-SNAPSHOT" -Dextensions=rest,langchain4j-openai
cd quarkus-langchain4j-app
quarkus create app -x rest -x langchain4j-openai com.redhat.developers:quarkus-langchain4j-app:1.0-SNAPSHOT
cd quarkus-langchain4j-app
All the remaining parts of this section assume that you’ll be working inside the project folder that was just created. In this case, quarkus-langchain4j-app.

Connect to OpenAI

LangChain4j provides you a proxy to connect your application to OpenAI by just adding a property to the application.properties file available in src/main/resources:

# Free demo key for basic usage of OpenAI ChatGPT
quarkus.langchain4j.openai.api-key=demo
# Change this URL to the model provider of your choice
quarkus.langchain4j.openai.base-url=https://api.openai.com/v1
# Set timeout explicitly (default is 10s)
quarkus.langchain4j.openai.timeout=30s

Create the AI service

First we need to create an interface for our AI service.

Create a new Assistant Java interface in src/main/java in the com.redhat.developers package with the following contents:

package com.redhat.developers;

import io.quarkiverse.langchain4j.RegisterAiService;

@RegisterAiService
public interface Assistant {
    String chat(String message);
}

Create the prompt-base resource

Now we’re going to implement a resource that sends prompts using the AI service.

Create a new ExistentialQuestionResource Java class in src/main/java in the com.redhat.developers package with the following contents:

package com.redhat.developers;

import jakarta.inject.Inject;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.core.MediaType;

@Path("/earth")
public class ExistentialQuestionResource {

    @Inject
    Assistant assistant;

    @GET
    @Path("/flat")
    @Produces(MediaType.TEXT_PLAIN)
    public String isEarthFlat() {
        return assistant.chat("Can you explain why the earth is flat?");
    }
}

Invoke the endpoint

Start the app in Quarkus dev mode:

  • Maven

  • Quarkus CLI

./mvnw quarkus:dev
quarkus dev

You can check your prompt implementation by pointing your browser to http://localhost:8080/earth/flat

You can also run the following command:

curl -w '\n' localhost:8080/earth/flat

An example of the output you might see (Yours will likely be slightly different depending on the response from the non-deterministic LLM):

The Earth is not flat, it is an oblate spheroid, meaning it is mostly spherical in shape but slightly flattened at the poles and bulging at the equator. This shape is due to the Earth's rotation, which causes it to bulge slightly at the equator and flatten at the poles. The idea that the Earth is flat is a misconception that has been debunked by centuries of scientific evidence, including satellite imagery, photos from space, and measurements of the Earth's curvature.