Chains and Memory

So far we explored how to use prompts with LLMs, however to really leverage the power of LLMs it is essential that you can build a conversation by referring to previous questions and answers and manage concurrent interactions.

In this section, we’ll cover how we can achieve this with the LangChain4j extension in Quarkus.

Create an AI service with memory

Let’s create an interface for our AI service, but with memory this time.

Create a new AssistantWithMemory Java interface in src/main/java in the com.redhat.developers package with the following contents:

package com.redhat.developers;

import dev.langchain4j.service.MemoryId;
import dev.langchain4j.service.UserMessage;
import io.quarkiverse.langchain4j.RegisterAiService;

@RegisterAiService()
public interface AssistantWithMemory {

    String chat(@MemoryId Integer id, @UserMessage String msg);

}

Create a Developer resource

Now let’s create a resource to help us write some code, and then ask the model to create a test for the code as well in a second request. Thanks to the memory feature, the model will remember what code we’re referring to from the first request.

Create a new DeveloperResource Java class in src/main/java in the com.redhat.developers package with the following contents:

package com.redhat.developers;

import jakarta.inject.Inject;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.core.MediaType;

@Path("/")
public class DeveloperResource {

    @Inject
    private AssistantWithMemory ai;

    @GET
    @Path("/memory")
    @Produces(MediaType.TEXT_PLAIN)
    public String memory() {
        String msg1 = "How do I write a REST endpoint in Java using Quarkus?";

        String response = "[User]: " + msg1 + "\n\n" +
            "[LLM]: "+ ai.chat(1, msg1) + "\n\n\n" +
            "------------------------------------------\n\n\n";

        String msg2 = "Create a test of the first step. " +
                        "Be short, 15 lines of code maximum.";

        response += "[User]: " + msg2 + "\n\n"+
            "[LLM]: "+ ai.chat(1, msg2);

        return response;

    }

}

Invoke the endpoint

You can check your prompt implementation by pointing your browser to http://localhost:8080/memory

You can also run the following command in your terminal:

curl localhost:8080/memory

An example of output (can vary on each prompt execution):

[User]: How do I write a REST endpoint in Java using Quarkus?

[LLM]: To create a REST endpoint in Java using Quarkus, you can follow these steps:

1. Create a new Quarkus project using the Quarkus Maven plugin or Quarkus CLI.
2. Create a new Java class for your REST endpoint. You can annotate this class with `@Path` to define the base URL path for your endpoint.
3. Add methods to your class and annotate them with `@GET`, `@POST`, `@PUT`, or `@DELETE` annotations to define the HTTP method for each endpoint.
4. Use the `@Produces` and `@Consumes` annotations to specify the content type of the responses and requests.
5. Use the `@PathParam` and `@QueryParam` annotations to capture path and query parameters in your endpoint methods.
6. Implement the logic for your endpoint methods.
7. Build and run your Quarkus project to start the application and test your REST endpoint.

Here's an example of a simple REST endpoint class in Quarkus:

```java
import javax.ws.rs.*;
import javax.ws.rs.core.MediaType;

@Path("/hello")
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
public class HelloResource {

    @GET
    public String sayHello() {
        return "Hello, World!";
    }

    @GET
    @Path("/{name}")
    public String sayHelloTo(@PathParam("name") String name) {
        return "Hello, " + name + "!";
    }
}
```

This class defines two REST endpoints: `/hello` for saying hello to the world, and `/hello/{name}` for saying hello to a specific name. You can access these endpoints at `http://localhost:8080/hello` and `http://localhost:8080/hello/{name}` respectively.


[User]: Create a test of the first step. Be short, 15 lines of code maximum.

[LLM]: Here's an example of a simple test for the `sayHello` endpoint in Quarkus using JUnit:

```java
import io.quarkus.test.junit.QuarkusTest;
import io.restassured.RestAssured;
import org.junit.jupiter.api.Test;

import static io.restassured.RestAssured.given;
import static org.hamcrest.CoreMatchers.is;

@QuarkusTest
public class HelloResourceTest {

    @Test
    public void testSayHelloEndpoint() {
        given()
          .when().get("/hello")
          .then()
             .statusCode(200)
             .body(is("Hello, World!"));
    }
}
```

In this test, we are using the QuarkusTest annotation to run the test in the Quarkus test environment. The `testSayHelloEndpoint` method sends a GET request to the `/hello` endpoint and verifies that the response status code is 200 and that the response body is "Hello, World!".
```

How to index a conversation

We can use the LangChain4j extension to index a conversation so we can reuse it, and keep multiple, parallel conversations separated.

Let’s add a new guessWho() method to our DeveloperResource:

    @GET
    @Path("/guess")
    @Produces(MediaType.TEXT_PLAIN)
    public String guess() {
        String msg1FromUser1 = "Hello, my name is Klaus and I'm a doctor";

        String response = "[User1]: " + msg1FromUser1 + "\n\n" +
                "[LLM]: " + ai.chat(1, msg1FromUser1) + "\n\n\n" +
                "------------------------------------------\n\n\n";

        String msg1FromUser2 = "Hi, I'm Francine and I'm a lawyer";

        response += "[User2]: " + msg1FromUser2 + "\n\n" +
                "[LLM]: " + ai.chat(2, msg1FromUser2) + "\n\n\n" +
                "------------------------------------------\n\n\n";

        String msg2FromUser2 = "What is my name?";

        response += "[User2]: " + msg2FromUser2 + "\n\n" +
                "[LLM]: " + ai.chat(2, msg2FromUser2) + "\n\n\n" +
                "------------------------------------------\n\n\n";

        String msg2FromUser1 = "What is my profession?";

        response += "[User1]: " + msg2FromUser1 + "\n\n" +
                "[LLM]: " + ai.chat(1, msg2FromUser1) + "\n\n\n" +
                "------------------------------------------\n\n\n";

        return response;
    }

Invoke the endpoint

You can check your implementation by pointing your browser to http://localhost:8080/guess

You can also run the following command:

curl localhost:8080/guess

The result will be at your Quarkus terminal. An example of output (it can vary on each prompt execution):

[User1]: Hello, my name is Klaus and I'm a doctor

[LLM]:  Nice to meet you, Klaus! What field of medicine do you specialize in?


------------------------------------------


[User2]: Hi, I'm Francine and I'm a lawyer

[LLM]: Hello Francine, nice to meet you. How can I assist you today?


------------------------------------------


[User2]: What is my name?

[LLM]: Your name is Francine, and you mentioned earlier that you are a lawyer. How can I assist you today, Francine?


------------------------------------------


[User1]: What is my profession?

[LLM]: Your profession is being a doctor, Klaus. How can I assist you today?


------------------------------------------

Take a close look at the IDs of our calls to the assistant. Do you notice that the last question was in fact directed to Klaus with ID=1? We were indeed able to maintain 2 separate and concurrent conversations with the LLM.