Chomp Sci: The Computer Science Chatbot

Introduction

Chomp Sci is a retrieval based chat bot with a web-based user interface that was developed as part of my senior project for my BS in Computer Science from the University of Florida. The chatbot uses RASA to infer the intent of user queries and respond with an appropriate pre-programmed response.

The figure below details the system architecture used by Chomp Sci. I built a React based chat widget which is embedded on a web page. The user interacts with the chat widget, and messages are exchanged between the front-end and back-end via Socket.IO. RASA Core and its Natural Language Understanding unit infers the intent of user input and either responds with a matching response or passes the query via HTTP to the RASA Action Server. The RASA Action Server allows programmatic actions to be taken in response to user input such as running a Python function to query a database.

My Contributions

Frontend Design

I worked on the frontend design of the Chomp Sci project page and chat widget in Figma. I created the Chomp Sci hero concept and high fidelity prototypes and assets for the chat widget.

React Chat Widget Implementation

I implemented the React based chat widget for Chomp Sci. This consisted of writing code to:

• integrate the Socket.IO client with the chat widget
• send messages via the Socket.IO client to the RASA Core server
• listen for responses from the RASA Core server via side effects
• manage the state of the chat widget:
  • chat window open/ closed status
  • chat window message log
• process and present chat window message log entries:
  • support for different message data formats sent by RASA, including simple strings, images, and quick link buttons
  • support for markdown in responses via Remark
• automate the testing of the chat widget's sub-components using React Testing Library

Chatbot Content and Training Data

I contributed training input for the RASA Natural Language Understanding Unit and pre-programmed responses for the following topics:

• the campus vs. online experience at UF
• computer science overview and history
• miscellaneous chit-chat content
• support for course name and prefix synonyms

Project Context and Relevance

The initial proposal for Chomp Sci was submitted in the Fall of 2022, before OpenAI had granted public access to ChatGPT. Large Language Models went viral during development of Chomp Sci and it was interesting to see how chat bot technology changed while working on the project.

While RASA's Natural Language Understanding unit utilizes AI to determine the intent of user input, that intent is matched to pre-programmed responses. This is in contrast to generative AI which not only processes user input but also generates a predictive response. The benefit of generative AI is that chatbots for the first time are able to convincingly simulate conversation. The drawback of this approach is that while LLMs generally produce feasible responses, they are not always appropriate ones. Microsoft Tay is an infamous example of how generative AI can go wrong.

RASA provides a framework for a hybrid approach to chatbot development, where recent advances in AI are leveraged to make the processing of input easier while leaving the authoring of responses to humans. Such a system is not good at simulating open-ended conversation, however it is a good tool for transactional conversations and avoiding inappropriate or inaccurate responses. Approaches to leveraging generative AI are currently changing at a rapid pace. Better curation of training data to improve alignment or multi-agent systems to incorporate AI powered checks and balances into products are likely to make Large Language Models suitable for use in a wider variety of contexts.