Skip to main content

Chatbot & AI Agent Creation - Quick Reference

🎯 TL;DR​

We're building a production-grade chatbot platform that leverages our existing LangGraph agent infrastructure and integrates carefully selected open-source libraries to avoid reinventing the wheel.

✅ What We Already Have (USE THESE!)​

🤖 Agent Framework - LangGraph Based​

Location: packages/agents/

  • ✅ Complete RAG workflow (retrieve → rerank → plan → act → answer)
  • ✅ Tool registry with 4 built-in tools (retrieval, reranking, web search, escalation)
  • ✅ Safety policies (input/output filtering, PII detection, rate limiting)
  • ✅ Middleware (guardrails, cost tracking, latency tracking, auth)
  • ✅ Callback handlers (metrics, LangSmith integration)
  • ✅ 4 specialized agents (medical, compliance, manufacturing, research lab)

💡 Use These: Don't rebuild agent orchestration! Just enhance with intent recognition and dialogue management.

💬 Memory & Conversation Management​

Location: recoagent/memory/

  • ✅ SQLite-based persistence with connection pooling
  • ✅ Thread-based session isolation
  • ✅ Multi-type search (exact, fuzzy, semantic)
  • ✅ Memory optimization and cleanup strategies
  • ✅ LangGraph state compatibility
  • ✅ Multi-user support

💡 Use These: Don't rebuild conversation storage! Just add analytics UI on top.

🔍 RAG System​

Location: packages/rag/

  • ✅ Hybrid retrieval (BM25 + vector search)
  • ✅ Cross-encoder reranking
  • ✅ Query expansion
  • ✅ Document search and summarization
  • ✅ Faceted search

💡 Use These: Don't rebuild RAG! Just enhance with Haystack readers if needed.

📊 Observability​

Location: packages/observability/

  • ✅ Structured logging (structlog)
  • ✅ Metrics collection (Prometheus)
  • ✅ Distributed tracing
  • ✅ LangSmith integration
  • ✅ Cost and performance tracking

💡 Use These: Don't rebuild monitoring! Just add conversation analytics dashboard.

🌐 API Infrastructure​

Location: apps/api/

  • ✅ FastAPI-based REST API
  • ✅ JWT authentication
  • ✅ Rate limiting (Redis-based)
  • ✅ CORS and security headers
  • ✅ Health checks
  • ✅ PostgreSQL for persistence

💡 Use These: Don't rebuild API! Just add chatbot-specific endpoints.

🚀 What We Need to Add (BUILD THESE!)​

1. Intent Recognition & Entity Extraction​

Library: Rasa NLU + spaCy
Why: Understand user intent before routing to agent
Effort: 1-2 weeks

# What to build
packages/conversational/intent_recognition.py
packages/conversational/entity_extraction.py

2. Dialogue Management​

Library: Rasa Core
Why: Multi-turn conversations, context tracking, slot filling
Effort: 1-2 weeks

# What to build
packages/conversational/dialogue_manager.py
packages/conversational/context_manager.py

3. Chatbot UI Components​

Libraries: Chainlit (production) + Streamlit (demos) + React (custom)
Why: User-facing chat interfaces
Effort: 2-3 weeks

# What to build
apps/chatbot_ui/              # Chainlit production UI
examples/chatbot/streamlit_demo.py  # Quick demos
apps/web_ui/                  # React components

4. Multi-Channel Adapters​

Libraries: Slack SDK, python-telegram-bot, botbuilder-core
Why: Deploy to Slack, Teams, Telegram, etc.
Effort: 1 week

# What to build
packages/channels/slack_adapter.py
packages/channels/teams_adapter.py
packages/channels/telegram_adapter.py
packages/channels/webhook_adapter.py

5. Voice Capabilities​

Libraries: OpenAI Whisper (STT) + OpenAI TTS / Piper TTS
Why: Voice-enabled chatbot
Effort: 1 week

# What to build
packages/voice/stt_service.py
packages/voice/tts_service.py
apps/api/voice_api.py

6. Agent Builder UI​

Library: Streamlit or custom React
Why: Let users create custom agents without code
Effort: 2 weeks

# What to build
apps/agent_builder/           # Visual agent creator
packages/agents/agent_registry.py  # Agent management

7. Conversation Analytics​

Library: Plotly Dash
Why: Track engagement, performance, costs
Effort: 1 week

# What to build
packages/analytics/conversation_analytics.py
apps/analytics_dashboard/

📦 Libraries to Install​

Core Conversational AI​

pip install rasa>=3.6.0                    # Intent & dialogue
pip install farm-haystack>=1.22.0         # NLP pipeline
pip install spacy>=3.7.0                  # Entity extraction
python -m spacy download en_core_web_lg

UI Frameworks​

pip install chainlit>=1.0.0               # Production chatbot UI
pip install streamlit>=1.28.0             # Demo UIs
pip install streamlit-chat>=0.1.1
pip install gradio>=4.0.0                 # Testing UIs

Multi-Channel​

pip install slack-sdk>=3.26.0             # Slack
pip install slack-bolt>=1.18.0
pip install python-telegram-bot>=20.7     # Telegram
pip install botbuilder-core>=4.16.0       # Teams

Voice​

pip install openai-whisper>=20231117      # Speech-to-text
pip install piper-tts>=1.2.0              # Text-to-speech (offline)

Analytics​

pip install dash>=2.14.0                  # Analytics dashboard
pip install plotly>=5.18.0

Advanced​

pip install langchain-community>=0.0.10   # Extended tools
pip install pyautogen>=0.2.0              # Multi-agent collab

🏗️ Architecture Overview​

┌─────────────────────────────────────────────────────────┐
│              USER INTERFACES (NEW)                       │
│  Web │ Mobile │ Slack │ Teams │ Voice │ Custom         │
└─────────────┬───────────────────────────────────────────┘
              │
              ▼
┌─────────────────────────────────────────────────────────┐
│         CONVERSATIONAL LAYER (NEW)                       │
│  Intent Recognition (Rasa) → Dialogue Manager (Rasa)    │
│  Entity Extraction (spaCy) → Context Management         │
└─────────────┬───────────────────────────────────────────┘
              │
              ▼
┌─────────────────────────────────────────────────────────┐
│         AGENT LAYER (EXISTING - USE!)                    │
│  LangGraph Agents + Tool Registry + Safety Policies     │
└─────────────┬───────────────────────────────────────────┘
              │
              ▼
┌─────────────────────────────────────────────────────────┐
│         DATA LAYER (EXISTING - USE!)                     │
│  RAG Pipeline + Memory System + Vector Store            │
└─────────────┬───────────────────────────────────────────┘
              │
              ▼
┌─────────────────────────────────────────────────────────┐
│         MONITORING (EXISTING + NEW)                      │
│  LangSmith + Prometheus + NEW: Conversation Analytics   │
└─────────────────────────────────────────────────────────┘

🎯 Integration Strategy​

✅ DO THIS​

  1. Keep existing LangGraph agent orchestration
  2. Add Rasa for intent recognition as pre-processing layer
  3. Keep existing memory system
  4. Add conversation analytics dashboard
  5. Keep existing RAG pipeline
  6. Add Haystack readers for extractive QA (optional)
  7. Keep existing API infrastructure
  8. Add new chatbot endpoints

❌ DON'T DO THIS​

  1. ❌ Don't rebuild agent orchestration (we have LangGraph!)
  2. ❌ Don't rebuild memory system (we have it!)
  3. ❌ Don't rebuild RAG (we have it!)
  4. ❌ Don't rebuild API (we have FastAPI!)
  5. ❌ Don't rebuild monitoring (we have observability stack!)

📋 Implementation Checklist​

Week 1-2: Core Infrastructure​

  • Install all dependencies
  • Create conversational layer (intent + dialogue)
  • Build basic chatbot API endpoints
  • Create Streamlit demo UI
  • Test end-to-end flow

Week 3-4: UI Components​

  • Build Chainlit production interface
  • Create Gradio testing interface
  • Start React component library
  • Add streaming support
  • Add file upload

Week 5: Multi-Channel​

  • Slack bot integration
  • Telegram bot integration
  • Teams bot integration
  • Webhook adapter

Week 6: Voice​

  • Speech-to-text service
  • Text-to-speech service
  • Voice API endpoints
  • Voice UI components

Week 7-8: Agent Builder​

  • Agent configuration schema
  • Visual agent builder UI
  • Agent registry
  • Testing playground

Week 9: Analytics​

  • Conversation analytics
  • Agent performance metrics
  • Analytics dashboard
  • Export reports

Week 10: Advanced​

  • A/B testing framework
  • Multi-agent collaboration
  • Production hardening
  • Documentation

💡 Key Design Decisions​

1. Why Rasa?​

  • ✅ Industry standard for intent recognition
  • ✅ Excellent dialogue management
  • ✅ Production-tested
  • ✅ Integrates well with LangGraph

2. Why Chainlit?​

  • ✅ Built specifically for LangChain/LangGraph
  • ✅ Production-ready
  • ✅ Beautiful UI out of the box
  • ✅ Easy deployment

3. Why Keep LangGraph?​

  • ✅ Already implemented and working
  • ✅ Excellent for agent orchestration
  • ✅ Tool integration is perfect
  • ✅ Production-tested

4. Why Keep Our Memory System?​

  • ✅ Already production-ready
  • ✅ SQLite is fast and reliable
  • ✅ Excellent search capabilities
  • ✅ No need to rebuild

5. Why Add Rasa on Top?​

  • ✅ Better intent understanding
  • ✅ Multi-turn conversations
  • ✅ Context management
  • ✅ Complements LangGraph perfectly

📊 Expected Outcomes​

Technical Metrics​

  • Response Time: < 2s for text
  • Intent Accuracy: > 90%
  • Uptime: 99.9%
  • Concurrent Users: 100+

Business Metrics​

  • User Satisfaction: > 4.0/5.0
  • Conversation Completion: > 80%
  • Escalation Rate: < 10%
  • Cost per Conversation: < $0.10

Features​

  • ✅ Multi-channel deployment (Web, Slack, Teams, Telegram)
  • ✅ Voice-enabled conversations
  • ✅ Custom agent creation without code
  • ✅ Real-time analytics
  • ✅ A/B testing capabilities

🚀 Getting Started​

Step 1: Review Full Plan​

Read: docs/docs/features/CHATBOT_AI_AGENT_CREATION_PLAN.md

Step 2: Install Dependencies​

# See full list in plan document
pip install rasa chainlit streamlit gradio spacy

Step 3: Build Core Layer​

# Create conversational layer
mkdir -p packages/conversational
touch packages/conversational/intent_recognition.py
touch packages/conversational/dialogue_manager.py

Step 4: Create Demo​

# Build Streamlit demo
mkdir -p examples/chatbot
touch examples/chatbot/streamlit_demo.py

Step 5: Test Integration​

# Test with existing agents
pytest tests/test_chatbot/

📚 Resources​

Full Documentation​

  • Complete Plan: docs/docs/features/CHATBOT_AI_AGENT_CREATION_PLAN.md
  • Architecture: See plan document section 5
  • Integration Strategy: See plan document section 7

External Resources​

❓ FAQ​

Q: Why not use only Rasa?​

A: Rasa is great for dialogue, but our LangGraph agents are already production-ready with advanced RAG, reranking, and tool integration. We'll use Rasa for what it's best at (intent/dialogue) and LangGraph for agent orchestration.

Q: Why not build everything from scratch?​

A: We'd spend 3-4x more time and money reinventing wheels. Open-source libraries are production-tested, community-supported, and integrate well.

Q: Can we use a different UI framework?​

A: Yes! The plan includes Chainlit, Streamlit, Gradio, and React. Choose based on your needs. Chainlit for production, Streamlit for demos, React for full customization.

Q: What about costs?​

A: Infrastructure ~$900/month, LLM costs ~$2250/month for 1000 users. See full cost breakdown in plan document.

Q: How long will this take?​

A: 10 weeks for full implementation. But you can have a working prototype in 2 weeks!

📖 For complete details, see: CHATBOT_AI_AGENT_CREATION_PLAN.md