Exploring LangChain4J

Photo by Mojahid Mottakin on Unsplash

Since the ChatGPT burst into the scene, there has been a lot interest around LLMs and how we can leverage them for business applications. Very common and simple business use cases for LLM powered applications, for example:

• Specialised ChatBots
• Customer Support Agents
• Content generators (tailored email for clients)
• Text summarisation

LangChain provides many features to simplify the process of building AI powered applications.

• Model Interaction. Integration with the LLM models to manage query and responses.
• External Data integration. For a business specific application, the solution needs to be able to feed external data to the LLM.
• Chains. Compose more complex functionality by chaining together smaller units.
• Agents. Custom tools and commands for specific requests
• Memory. To implement a context aware solution by enabling both short term and long term memory in the interactions

For the sample application, I have chosen to use LangChain4j — the java version of popular LangChain.

The application is a hypothetical customer service agent for an insurance provider. The agent is defined in an interface:

public interface CustomerServiceAgent {
    @SystemMessage({
            "You are a customer support of Insurance NSW, which is an agency of the Government of New South Wales, Australia.",
            "Before providing information about about a rebate application, you MUST always check:",
            "correlationId or email. If the customer is in APPROVED state, please tell the customer that their claim will be settled in two weeks",
            "Today is {{current_date}}."
    })
    String chat(String userMessage);
}

It is then used to build an AI service bean using the LangChain’s builder utility. Custom content retriever is also injected to provide context for the queries. The content retriever uses an embedding store which loads embeddings from the internal document `claims-pdt-guide.txt` that has terms and conditions specific to the company.

    @Bean
    CustomerServiceAgent customerSupportAgent(ChatLanguageModel chatLanguageModel,
                                              ContentRetriever contentRetriever,
                                              CustomerService customerService) {
        return AiServices.builder(CustomerServiceAgent.class)
                .chatLanguageModel(chatLanguageModel)
                .chatMemory(MessageWindowChatMemory.withMaxMessages(20))
                .tools(customerService)
                .contentRetriever(contentRetriever)
                .build();
    }

Code for the application is available at https://github.com/achalise/langchain-playground

To start the application

./mvnw spring-boot:run

which exposes an endpoint localhost:8080/chat that accepts questions as query parameter. The agent answers the queries using context provided in claims-pdt-guide.txt as well as well as the functions and data provided in CustomerService class.

Response for non-existent customer:

GET http://localhost:8080/chat?question=Pleaes cancel my application. My first name is Joe and my correlationId is 111
HTTP/1.1 200
Content-Type: text/plain;charset=UTF-8
Content-Length: 241
Date: Mon, 05 Feb 2024 09:48:20 GMT
Keep-Alive: timeout=60
Connection: keep-alive
I'm sorry, but I couldn't find any application with the correlation ID '111'. Please double-check the correlation ID and make sure it is correct. If you have any further questions or need assistance, please feel free to contact us at 121345.
Response code: 200; Time: 4584ms (4 s 584 ms); Content length: 241 bytes (241 B)

Response for existing application that has already been approved and payment submitted:

GET http://localhost:8080/chat?question=My email is 'duran@gmail.com'. Please find my applications.
HTTP/1.1 200 
Content-Type: text/plain;charset=UTF-8
Content-Length: 471
Date: Mon, 05 Feb 2024 09:46:31 GMT
Keep-Alive: timeout=60
Connection: keep-alive
I found one application associated with the email 'duran@gmail.com'. Here are the details:
- Correlation ID: 91137D
- First Name: Leandro
- Last Name: Duran
- Address: 199 George Street, Newtown, 2000
- Rebate Type: HAIL_DAMAGE
- Amount: $300
- Status: PAYMENT_SUBMITTED
Since your application is in the PAYMENT_SUBMITTED state, the funds should be available within 3 days. If you have any further questions or need assistance, please feel free to contact us at 121345.
Response code: 200; Time: 7719ms (7 s 719 ms); Content length: 471 bytes (471 B)