Working with prompts
The Quarkus LangChain4j extension seamlessly integrates Large Language Models (LLMs) into Quarkus applications. LLMs are AI-based systems designed to understand, generate, and manipulate human language, showcasing advanced natural language processing capabilities. Thanks to this extension, we can enable the harnessing of LLM capabilities for the development of more intelligent applications.
In this first chapter, we’ll explore the simplest of interactions with an LLM: Prompting. It essentially means just asking questions to an LLM and receiving an answer in natural language from a given model, such as ChatGPT, Granite, Mistral, etc.
Creating a Quarkus & LangChain4j Application
We’re going to use the langchain4j-openai extension for our first interaction with models. The openai extension supports models that expose the open sourced OpenAI API specification. Several models and model providers expose this API specification. If you want to use a different API spec, then you can likely find a supported extension in the Quarkus documentation.
mvn "io.quarkus.platform:quarkus-maven-plugin:create" -DprojectGroupId="com.redhat.developers" -DprojectArtifactId="quarkus-langchain4j-app" -DprojectVersion="1.0-SNAPSHOT" -Dextensions=rest,langchain4j-openai
cd quarkus-langchain4j-app
quarkus create app -x rest -x langchain4j-openai com.redhat.developers:quarkus-langchain4j-app:1.0-SNAPSHOT
cd quarkus-langchain4j-app
All the remaining parts of this section assume that you’ll be working inside the project folder that was just created. In this case, quarkus-langchain4j-app .
|
Connect to OpenAI
LangChain4j provides you a proxy to connect your application to OpenAI by just adding a property to the application.properties
file available in src/main/resources
:
# Free demo key for basic usage of OpenAI ChatGPT
quarkus.langchain4j.openai.api-key=demo
# Change this URL to the model provider of your choice
quarkus.langchain4j.openai.base-url=https://api.openai.com/v1
# Set timeout explicitly (default is 10s)
quarkus.langchain4j.openai.timeout=30s
Create the AI service
First we need to create an interface for our AI service.
Create a new Assistant
Java interface in src/main/java
in the com.redhat.developers
package with the following contents:
package com.redhat.developers;
import io.quarkiverse.langchain4j.RegisterAiService;
@RegisterAiService
public interface Assistant {
String chat(String message);
}
Create the prompt-base resource
Now we’re going to implement a resource that sends prompts using the AI service.
Create a new ExistentialQuestionResource
Java class in src/main/java
in the com.redhat.developers
package with the following contents:
package com.redhat.developers;
import jakarta.inject.Inject;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.core.MediaType;
@Path("/earth")
public class ExistentialQuestionResource {
@Inject
Assistant assistant;
@GET
@Path("/flat")
@Produces(MediaType.TEXT_PLAIN)
public String isEarthFlat() {
return assistant.chat("Can you explain why the earth is flat?");
}
}
Invoke the endpoint
Start the app in Quarkus dev mode:
You can check your prompt implementation by pointing your browser to http://localhost:8080/earth/flat
You can also run the following command:
curl -w '\n' localhost:8080/earth/flat
An example of the output you might see (Yours will likely be slightly different depending on the response from the non-deterministic LLM):
The Earth is not flat, it is an oblate spheroid, meaning it is mostly spherical in shape but slightly flattened at the poles and bulging at the equator. This shape is due to the Earth's rotation, which causes it to bulge slightly at the equator and flatten at the poles. The idea that the Earth is flat is a misconception that has been debunked by centuries of scientific evidence, including satellite imagery, photos from space, and measurements of the Earth's curvature.