Training Your Own AI Chatbot with Open-Source Tools

Choosing the Right Open-Source Framework

Preparing Your Data

1. Define the Scope and Intents
List out user intents (e.g., greeting, FAQ, booking, etc.) and sample utterances for each.

Example:

intents:
  - greet
  - book_flight
  - faq_hours

2. Collect and Structure Training Data
For frameworks like Rasa, use YAML or Markdown formats.

Example (Rasa NLU format):

nlu:
- intent: greet
  examples: |
    - hello
    - hi there
    - good morning
- intent: book_flight
  examples: |
    - I want to book a flight
    - Book a flight to Paris
    - Can you help me book a ticket?

3. Annotate Entities
Mark important entities such as dates, locations, or names.

Example:

- intent: book_flight
  examples: |
    - Book a flight to [Paris](destination)
    - I need a ticket to [New York](destination)

Setting Up the Environment

1. Install Prerequisites
– Python 3.8+
– pip
– (Optional) GPU drivers for deep learning models

2. Install Framework (Example: Rasa)

python3 -m venv chatbot_env
source chatbot_env/bin/activate
pip install rasa

Training the Model

Rasa Example:

rasa init
# or if you have your own data
rasa train

Botpress Example:
– Download Botpress from https://botpress.com/download
– Run ./bp or bp.exe in the extracted folder
– Access the admin panel at http://localhost:3000

Building Dialogue Logic

Rasa (Stories):

stories:
- story: book flight path
  steps:
    - intent: greet
    - action: utter_greet
    - intent: book_flight
    - action: flight_form
    - action: utter_confirm_booking

Botpress (Visual Flow):
– Use the GUI to create nodes and transitions
– Connect user intents to actions and responses

Integrating with External APIs

Example: Fetching Flight Data in Rasa

# actions/actions.py
from rasa_sdk import Action, Tracker
from rasa_sdk.executor import CollectingDispatcher
import requests

class ActionSearchFlights(Action):
    def name(self):
        return "action_search_flights"

    def run(self, dispatcher, tracker, domain):
        destination = tracker.get_slot("destination")
        # Call external API
        response = requests.get(f"https://api.example.com/flights?dest={destination}")
        flights = response.json()
        dispatcher.utter_message(text=f"Found {len(flights)} flights to {destination}")
        return []

• Add this action to domain.yml and update stories.yml accordingly.

Testing and Improving Your Chatbot

1. Interactive Testing (Rasa)

rasa shell

• Engage with the bot and review responses.

2. Evaluation

rasa test nlu

• Examine precision, recall, and F1-score.

3. Iterative Improvement
– Expand training data with real user conversations.
– Refine intents and entity definitions.
– Retrain and redeploy as needed.

Deploying the Chatbot

Docker Example (Rasa):

FROM rasa/rasa:latest
COPY . /app
WORKDIR /app
RUN rasa train
CMD ["rasa", "run", "--enable-api", "--cors", "*", "--debug"]

Exposing Endpoints:
– RESTful endpoint: /webhooks/rest/webhook
– Integrate with platforms: Slack, Facebook Messenger, Telegram (via connectors)

Security and Privacy Considerations

• Sanitize and anonymize user data in logs.
• Implement authentication for admin endpoints.
• Use HTTPS in production deployments.
• Regularly update dependencies to patch vulnerabilities.

Summary Table: Key Steps and Tools

Further Resources

• Rasa Documentation
• Botpress Docs
• DeepPavlov Tutorials
• Haystack Docs