Build a Customer Support Bot

Last updated on February 27, 2026

Softcery Platform

Build and deploy reliable AI agents with the Softcery Platform.

Get started

This guide walks you through building a customer support agent from scratch on the Softcery Platform – one that handles common questions, provides accurate answers from your docs, catches bad responses before users see them, and deploys to your website as a branded chat widget.

By the end, you’ll have a production-ready support bot with knowledge from your docs, quality evaluations, and a branded embed on your site.

Step 1: Create the Agent

  1. Click Create Agent in the sidebar
  2. Name: “Acme Support” (or your company name)
  3. Description: “Customer support for the Acme platform”
  4. Click Create

Step 2: Configure Behavior

Go to the Behavior page.

Use the Customer Support Preset

Select Customer Support Agent from the preset dropdown. This fills all six fields with proven support-focused prompts.

Customize the Identity

Replace the generic identity with your specifics:

You are Acme’s customer support agent – an AI that helps customers resolve issues with the Acme platform quickly and completely.

Add Business-Specific Constraints

The preset includes general support constraints. Add yours:

  • Never quote specific pricing. Direct pricing questions to acme.com/pricing or [email protected]
  • Never promise specific resolution timelines. Say “our team typically responds within one business day” for escalations
  • For billing issues, always direct to [email protected] – never attempt to resolve billing problems
  • Don’t troubleshoot issues with third-party integrations beyond what’s in the knowledge base

Set the Fallback

When you can’t resolve an issue, tell the customer to email [email protected] with their account email and a description of the problem. If it’s urgent, mention live chat hours (Monday–Friday, 9am–5pm EST).

Choose Your Model

For support bots, Claude Haiku 3.5 is a strong choice – fast responses at lower cost, with good quality for straightforward Q&A. If your support questions are complex (technical troubleshooting, multi-step debugging), consider Claude Sonnet 4 for better reasoning.

Click Save.

Step 3: Add Your Knowledge Base

Go to the Knowledge page. Add sources in order of impact:

Your Documentation Site

  1. Click Add Source → Website
  2. Mode: Sitemap (if your docs have one) or Crawl links with depth 3
  3. URL: https://docs.acme.com (or your sitemap URL)
  4. Max pages: 200 (adjust based on your docs size)
  5. Click Add

This captures your documentation structure – the sitemap mode prioritizes recently updated pages.

FAQ Content

If you have FAQ content that’s not in your docs:

  1. Click Add Source → Text
  2. Title: “Frequently Asked Questions”
  3. Content: Paste your FAQ content – questions and answers in a clear format
  4. Click Add

Known Issues / Release Notes

  1. Click Add Source → Text or File (if you have a document)
  2. Add current known issues, recent changes, or release notes that affect customer questions

Wait for all sources to reach “Ready” status before testing.

Step 4: Set Up Evaluations

Go to the Evaluations page. Add three evaluations:

Factual Accuracy (Block)

Criteria: Verify that the response only contains information found in or directly inferable from the retrieved knowledge chunks. Flag any specific claims about features, pricing, availability, policies, or procedures that don’t have a clear source in the provided context. General conversational responses (greetings, clarifications, acknowledgments) are exempt.

Threshold: 0.7 Action: Block

This prevents your support bot from confidently making things up. Blocked responses are replaced with your fallback message.

Scope Enforcement (Warn)

Criteria: Check that the response stays within the agent’s support scope. The agent should not provide medical advice, legal opinions, financial investment advice, or guidance unrelated to the Acme platform. If asked about competitors, it should acknowledge the question neutrally without making comparisons.

Threshold: 0.6 Action: Warn

This flags when the agent strays outside its lane. “Warn” lets the response through but highlights it for your review.

Resolution Quality (Log)

Criteria: Assess whether the response provides a clear path to resolution. The agent should either solve the problem directly, provide specific next steps the customer can take, or clearly direct them to the appropriate support channel. Responses that end vaguely (“I hope that helps!”) or leave the customer without a clear next action should score low.

Threshold: 0.7 Action: Log

This tracks whether your bot is actually helping, without blocking responses. Review the logs to identify patterns where the bot falls short.

Step 5: Brand and Deploy

Go to the Channels page.

Create an Embed Channel

  1. Click Add Channel → Embed
  2. Name: “Docs Widget” or “Support Chat”

Configure Branding

On the Branding tab:

  1. Set your primary color to match your brand
  2. Add your logo URL
  3. Set the header title to “Acme Support”
  4. Set the header subtitle to “We’re here to help”
  5. Write a welcome message: “Hi! I’m Acme’s support assistant. How can I help you today?”
  6. Choose a bubble icon (headphones, message-circle, or help-circle work well for support)
  7. Set bubble size to your preference

Configure Settings

On the Settings tab:

  1. Set step display to “Minimal” – users see that the bot is searching docs and thinking, but no technical details
  2. Set rate limits – 10 messages/minute, 60 messages/hour is reasonable for support

Get the Install Snippet

On the Install tab, copy the embed snippet and add it to your website’s HTML. The widget will appear on every page where the snippet is loaded.

Step 6: Test End-to-End

Before going live, test with realistic scenarios:

  1. Common question – “How do I reset my password?” → Should answer from docs
  2. Specific feature question – “Does Acme support SSO?” → Should answer accurately or say it doesn’t know
  3. Out-of-scope question – “What’s better, Acme or [competitor]?” → Should handle gracefully
  4. Billing question – “I was charged twice” → Should direct to [email protected]
  5. Multi-turn conversation – Ask a follow-up question about the same topic → Should maintain context
  6. Edge case – Try to get the bot to make up a feature → Factual accuracy evaluation should catch it

Check the inspection panel on each response to verify:

  • Correct knowledge chunks were retrieved
  • Evaluations are scoring appropriately
  • The tone matches your Customer Support preset

Step 7: Monitor After Launch

In the first week:

  1. Check the dashboard daily – watch success rate and message volume
  2. Review 5–10 conversations daily via the Conversations page
  3. Look for evaluation failures – are any evaluations consistently near the threshold?
  4. Identify knowledge gaps – questions users ask that the bot can’t answer well

Common first-week adjustments:

  • Adding text sources for FAQs the bot couldn’t handle
  • Tightening constraints when the bot gives too-broad answers
  • Adjusting evaluation thresholds (too many false positives = threshold too high)
  • Adding a second website source for content you missed initially

What You’ve Built

A customer support agent that:

  • Answers questions from your documentation
  • Blocks hallucinated responses automatically
  • Tracks whether it’s actually helping users
  • Looks and feels like your brand
  • Deploys as a chat widget on your site
  • Provides full inspection into every response for quality review

Total setup time: about 15–20 minutes for a basic configuration. Fine-tuning with real user data is an ongoing process, but you have the monitoring tools to guide it.