Hybrid AI System for SMEs: Self-Hosted LLM, Orchestrator, and Agents

Introduction

SMEs face the challenge of automating customer service, internal operations, and knowledge generation without relying solely on public clouds. This practical tutorial demonstrates how to set up a hybrid system: internal forums with AI capabilities, agents that perform tasks, and a multi-channel orchestrator that connects a local self-hosted LLM with Telegram and WhatsApp. The goal is to automate real workflows, protect data, and deploy incrementally with measurable results.

Proposed Architecture (Overview)

Key Components

• Local self-hosted LLM: Large language model deployed on your own servers (e.g., Llama2/Alpaca-style, Mistral if the license permits) for offline inference.
• Multi-channel orchestrator: Service that receives messages from Telegram/WhatsApp/Forum and decides which agent to invoke.
• AI agents: Microservices with specific roles (support, ticketing, RAG/retrieval, task execution).
• AI forums: Internal community platform (existing platform or Discourse) integrated with RAG for automated responses and moderation.
• Connectors: Webhooks/bridges for Telegram Bot API and WhatsApp Business API (or open-source bridge).
• Storage: Vector DB for embeddings, ticket database, logs, and audit trail.

Basic Flow

1. User writes on WhatsApp/Telegram or posts on the forum.
2. Connector sends payload to the orchestrator.
3. Orchestrator evaluates intention (light classifier) and calls the corresponding agent.
4. Agent queries RAG/LLM local and/or executes an action (create ticket, return FAQ).
5. Response is sent back to the original channel and logged in the forum or CRM.

Step-by-Step: Incremental Deployment for an SME

1) Discovery and Priorities (1 week)

• Identify 3 top workflows: example: product inquiries, order status, technical support.
• Define KPIs: first response time, first-contact resolution rate, escalation rate.

2) Minimum Infrastructure (2 weeks)

• LLM server: GPU compatible (e.g., NVIDIA A10/T4) or small cluster.
• Vector DB: Milvus/Weaviate/PGVector.
• Orchestrator: Simple REST API (can start with FastAPI + Celery).
• Backups and private internal network; enable TLS and VPN for remote access.

3) Local LLM Deployment (1–2 weeks)

• Select a self-contained model with a compatible license.
• Optimize with quantization (INT8/4) if needed.
• Expose internal endpoint: /v1/generate with rate limits.
• Example prompt template (for RAG): "You are the assistant for [Company]. Use the knowledge base to respond briefly and ask for more information if needed."

4) Orchestrator and Agents (2 weeks)

• Implement orchestrator with routes:
• /incoming/telegram
• /incoming/whatsapp
• /incoming/forum
• Create agents:
• Agent-Support: Uses RAG for FAQ, generates responses, and suggests forum articles.
• Agent-Ticket: Creates and updates tickets in the CRM.
• Agent-Action: Executes tasks (e.g., cancel orders) after human verification.
• Example rule: if intention == "order_status" -> Agent-Ticket; if "product_question" -> Agent-Support.

5) Telegram and WhatsApp Integration (1 week)

• Telegram: Create a bot, configure webhook to the orchestrator.
• WhatsApp: Use WhatsApp Business API or bridge (e.g., Twilio/Meta Cloud) to receive messages at the orchestrator.
• Map fields: user_id, message_text, attachments, channel.

6) AI Forums: RAG and Automated Moderation (1–2 weeks)

• Index articles and Q&A in the vector DB.
• Automate suggested responses in draft (human moderator approves).
• Moderation: agent-moderator that detects spam and suggests closing/editing.

Concrete Examples / Use Cases

Use Case A: Travel Agency ("Change Reservation" Flow)

1. Customer writes on WhatsApp: "I need to change my flight from the 12th to the 14th."
2. Orchestrator detects intention "change_reservation" -> Agent-Ticket.
3. Agent-Ticket:

• Queries CRM via API to locate the reservation by phone/email.
• Calls the local LLM to generate a confirmation message with change policies.
• If there are fare differences, Agent-Action creates a pre-bill and marks it for human approval.

4. Result: Message on WhatsApp with options (accept/cancel), internal ticket created, and forum entry with updated FAQ.

Prompt template for LLM: "Context: reservation {id}, change policy: {policy_text}. Generate a clear message for the customer with 2 options and request confirmation."

Use Case B: SME E-commerce ("Post-Sale Technical Support" Flow)

1. User on Telegram shares a photo of a faulty product.
2. Orchestrator sends the image to Agent-Support with OCR/vision tools.
3. Agent-Support:

• Classifies the issue (defect / misuse / warranty).
• Queries RAG for quick solution steps.
• If it's a warranty, calls Agent-Ticket to initiate a return and sends a label to the customer.

4. Automatic forum registration: Creates an internal thread with the detected pattern (e.g., affected batch) to notify production.

Example step-by-step in the orchestrator:

• Reception -> intention -> enrich (fetch order metadata) -> decide agent -> agent executes -> notify channel -> log.

Security, Privacy, and Governance

Practical Recommendations

• Keep LLM on a private network; expose only to the orchestrator with mTLS.
• Anonymize personal data before indexing in the vector DB.
• Implement prompt guardrails: lists of prohibited content and human verification checklist for critical actions (payments, cancellations).
• Immutable logs and audit trail for traceability.

Backups and Compliance

• Daily backup of vectors and relational DB.
• Retention policies: retain sensitive chat logs only as needed according to GDPR/local regulations.
• Monthly review of prompts and automated responses by the compliance team.

Actionable Conclusion

Initial checklist (prioritize and execute in 8–10 weeks):

• Identify 3 critical workflows and KPIs.
• Prepare server for local LLM and vector DB.
• Implement basic orchestrator and 2 agents (Support, Ticket).
• Connect Telegram + WhatsApp with webhooks.
• Index knowledge and deploy RAG for the forum.
• Establish security rules and human verification processes.

First measurable goal for the SME: Reduce first response time in messaging channels by 50% in 2 months through automated responses + 30% reduction in repeated tickets thanks to RAG in forums. Start with a small use case (e.g., order status) and scale agents and functions based on results.