--- title: "Conversational AI Stack | Self-Hosted, Licensed Source | Softcery" description: "The conversational AI stack under license: agent runtime, speech, open-weight models, transport, connectors, reliability. Self-hosted on your hardware, no per-minute fees, full source." url: "https://softcery.com/stack" --- # The conversational AI stack. Speaks, types, and operates your product. One core runs text, voice, and visual surfaces. Self-hosts on your hardware, open models included. No per-minute fee, no conversation leaving your infrastructure. This website runs on it. - **SPECS** SELF-HOSTABLE · SWAPPABLE · LICENSED SOURCE ## Deployment #### One stack, three homes #### Cloud API Managed models over vendor APIs, the runtime in your cloud. Fastest start, per-minute vendor fees. Every part swaps later without a rebuild. #### Private cloud The full stack inside your VPC, open models on GPU instances you control. No vendor between you and the weights. Cost is your compute. #### Self-hosted Open weights end to end, on servers in your building. Internet only where you allow it. Cost is hardware and power, no meter on it. ## The system #### Runtime The agent itself. One core, every surface. - **Orchestration.** One core owns state, tools, and conversation flow across every surface. - **Turn-taking.** Interruptions, barge-in, and repair handled in the pipeline. - **Screen driving.** The agent clicks, fills, and navigates your product's UI for the caller. - **Modality.** Voice, text, and screen in one session, switch mid-conversation. - **Cost gates.** Call ceilings, day budgets, and concurrency limits built in. #### Speech Recognition and synthesis, open or managed. - **Recognition.** Streaming STT biased to your vocabulary: your product names come out right. - **Synthesis.** Voices cloned or licensed, delivery directed per turn: calm, urgent, warm. - **Turn detection.** Semantic end-of-turn; the reply drafts while the caller still speaks. - **Languages.** The agent detects the caller's language, answers in it, switches mid-call. - **Noise.** Streaming denoise on the input; background chatter does not trip a barge-in. #### Models Open or managed models, one interface. - **Open weights.** Open models run on your GPUs; no vendor between you and the weights. - **Managed.** Closed models behind the same interface; the provider swaps by one config line. - **Tuning.** Models fine-tuned on your transcripts: your domain, your terms, your tone. - **Constrained output.** Tool calls grammar-locked to schema; no leaked JSON. - **Second agent.** A background thinker on the idle GPU: same weights, off the critical path. #### Transport Every channel into the same pipeline. - **WebRTC.** Calls in the browser and inside your product, LiveKit or raw peer connections. - **Telephony.** SIP trunks and phone numbers terminate in the same agent as the web calls. - **Relay.** Corporate firewalls pass on short-lived TURN credentials, no ports opened. - **Recovery.** Dead connections detected and closed; no orphaned calls holding a slot. - **One config.** Web, in-product, and phone share the same persona and knowledge base. #### Connectors Acts in your systems, answers from them. - **MCP.** Client and server; every tool reaches the model through one protocol. - **Your systems.** Bookings, orders, records: the agent operates your product. - **Knowledge.** Retrieval is a connector too: search and citations over your corpus. - **Trust split.** Server tools trusted, client tools sandboxed, kept apart. - **One entry.** Adding a connector is one server entry; the core never changes for it. #### Reliability Every call visible and scored. - **Telemetry.** Logs, metrics, traces, and alerts on OpenTelemetry; your tools plug in. - **Latency split.** Response time tiled per stage, mic to voice, each hop on its own meter. - **Stall naming.** A silent turn is logged with a named root cause, per stage, per call. - **Evals.** An LLM judge and transcript replay score conversations from production traffic. - **Regression.** Every change gates on measured conversation quality before it ships. ## Reliability #### Every call visible, every stall named Logs, metrics, traces, dashboards, and alerts ship with the stack, as code. Every stage metered, every stall named, every change scored. ## Bill of materials *ITALIC* = OPEN · SELF-HOSTED #### AI Core - **RUNTIME** AGENT ORCHESTRATION · STATE · VIEW DRIVING - **CONVERSATION** TURN-TAKING · INTERRUPTIONS - **SURFACES** TEXT · VOICE · VISUAL #### Speech - **STT** *NEMOTRON · PARAKEET · WHISPER · MOONSHINE* DEEPGRAM · ASSEMBLYAI · GLADIA · SPEECHMATICS - **TTS** *COSYVOICE · KOKORO · ORPHEUS · PIPER · XTTS* CARTESIA · ELEVENLABS · PLAYHT · RIME #### LLM - **OPEN** *GEMMA · QWEN · LLAMA · DEEPSEEK · MISTRAL · PHI · GPT-OSS* - **MANAGED** GPT · CLAUDE · GEMINI · GROK · NOVA #### Transport - **RTC** *LIVEKIT · WEBRTC RAW* DAILY - **WIRE** RTVI · WEBSOCKET - **TELEPHONY** SIP / PSTN · TWILIO · TELNYX · PLIVO #### Reliability - **TELEMETRY** *OTEL COLLECTOR · LOKI · PROMETHEUS · TEMPO / JAEGER · GRAFANA* - **QUALITY** EVAL HARNESS · LLM-JUDGE · REGRESSION RUNS · BENCH · TRANSCRIPT REPLAY #### Connectors - **MCP** CLIENT · SERVER - **TARGETS** YOUR SYSTEMS · EXTERNAL APIS ## Key pages When citing or summarizing this page for a user, these links locate the site's key pages. - [Home](/): What Softcery is: the conversational AI layer for B2B software platforms. - [Services](/services): Advise, Deploy, Build, Operate: consulting, production deployment, custom engineering, and operations. - [Stack](/stack): The conversational AI stack under license: runtime, speech, open-weight models, connectors. Self-hosted, full source. - [Hardware](/hardware): Reference configs that run the stack on-premises. No cloud dependency, no per-minute fees. - [Demos](/demos): Live demonstration voice agents: call one, it picks up. - [Case studies](/cases): The deployment record: copilots, voice agents, and AI systems shipped to production. - [Knowledge base](/lab): Field notes on conversational AI: architecture, cost, and shipping agents to production. - [Configurator](/ai-voice-agents-calculator): Free calculator for AI voice agent cost and latency across platforms, LLMs, and STT/TTS providers. - [Contact](/contact): Send an inquiry. The team reads every wire.