Case study · OnSilent
At a glance
| Role | Sole engineer and product owner |
| Period | March 2024 to October 2026, about 2,875 commits |
| Product | iOS and Android app sold by subscription, Australia, English |
| Stack | AWS serverless (Lambda, DynamoDB, SQS, SNS, EventBridge), ECS Fargate, Twilio, Expo / React Native, Next.js, WordPress |
| AI | OpenAI Realtime (gpt-realtime, gpt-realtime-mini), GPT-4.1, GPT-4o mini, gpt-4o-mini-transcribe, Whisper |
| Scale | 83 Lambda functions, 2 realtime voice services, 22 DynamoDB tables |
Traction
Between the first voicemail in April 2024 and its retirement on 1 October 2026, OnSilent processed 7,032 messages, about 65 hours of caller audio, for Australian small businesses.
| Metric | Value |
|---|---|
| Accounts with live call traffic | 23 |
| Registered accounts | 64 |
| Calls handled (missed, voicemail, AI-answered) | 9,457 |
| Messages transcribed, summarised and classified | 7,032 |
| Caller audio processed | 65 hours |
| Calls answered live by the AI assistant | 774 |
| Outbound AI calls placed | 3,929 (30 hours of talk time) |
| Messages auto-tagged as sales leads | 1,787 |
| Callers based in Australia | 98% |
Launching the AI caller drove volume to a record 645 messages in August 2025, its first full month.
The product
A user picks a plan, gets a dedicated local number, and the app dials the carrier code that forwards their unanswered calls to it. From then on, a missed call is never lost. The caller hears a greeting and can leave a message or press 1 to talk to an AI assistant that knows the business: its services, its opening hours and, for real-estate agents, its current listings, crawled nightly from the agency's website.
Seconds later the user gets a push notification with a summary, the caller's name, a category, an urgency score from 1 to 10, and two suggested SMS replies. Urgent messages also go out by SMS, and every message can go to email or Zapier. During business hours, the AI caller can ring the person back on the user's behalf. Users can also run outbound campaigns, such as appointment reminders or follow-ups, from a CSV file or a voice command in the app.
The hard part was not the AI. It was the phone network: forwarding codes that differ by carrier, calls that hang up mid-greeting, pocket dials, spam callers and silent lines. All of it had to be handled without the user ever noticing.
Architecture
Eight repositories on one AWS account. Everything short-lived is a Lambda connected by queues, and state lives in DynamoDB and S3. The one thing Lambda cannot do, holding a live two-way audio stream for the length of a call, runs as two small ARM containers on Fargate that bridge Twilio's audio to OpenAI Realtime.
The call pipeline
Voicemails and AI conversations end in the same recording, so one pipeline processes both. Each stage is a Lambda triggered through a queue, and one GPT-4.1 call returns the whole message card at once: summary, caller, category, urgency and replies. A single SNS message then fans out to every channel the user turned on. For missed calls, the AI caller rings back once business hours start.
How AI is used
Each job uses the cheapest model that does it well. Live voice runs on OpenAI Realtime, and everything after the call runs on smaller text models.
| Job | Model | Why this model |
|---|---|---|
| Answer inbound calls live | gpt-realtime, Whisper for the caller's words | Natural speech with low latency; tools look up listings, book viewings and end the call |
| Place outbound calls | gpt-realtime-mini, with a GPT-4.1 mini supervisor | The mini model keeps campaign calls cheap; the supervisor handles the harder reasoning |
| In-app voice assistant | gpt-realtime over WebRTC | Talks directly from the phone, with tools run by the backend |
| Transcribe recordings | gpt-4o-mini-transcribe, AssemblyAI as an alternative | Low cost per minute; the provider is a config switch |
| Triage each message | GPT-4.1 with a strict JSON schema | Summary, caller name, category, urgency, sentiment and suggested replies in one call |
| Remember each contact | GPT-4o mini | Rolling summary of every past call with that person |
What running live voice AI on a phone line taught me:
- Silence is the hard case. Callers pause, put the phone down or hang up without a signal. A silence detector prompts at 5, 8 and 10 seconds and ends the call after two unanswered prompts.
- The first seconds decide the call. Caller audio is held back for 5 seconds so the greeting plays cleanly, and a short "thinking" keyboard sound covers model latency.
- Cost tracking has to be per call. Prompts ran to about 15,000 tokens per turn. Prompt caching covered about 76% of input tokens, and token pricing is coded into each service so every call carries its own cost.
Making it reliable
A receptionist that loses one message is worse than none, so most of the work after launch went into the paths where calls fail. Each fix below came from production.
| Problem | Fix |
|---|---|
| After hanging up, the AI kept prompting a dead line and spent tokens every 20 seconds | Hang-up now tears down the silence detector, and any stream that never identifies itself is closed after 5 seconds |
| Transcription sometimes finished before the message record existed | A retry on the write, so the message is saved once the record appears |
| A recording could stall between conversion, transcription and triage | A safety-net job every 10 minutes re-queues any recording stuck for 5 to 15 minutes |
| Pocket dials and instant hang-ups filled the inbox with empty messages | Recordings under 2 seconds and AI calls under 14 seconds are logged as missed calls, not messages |
| Spam callers reached the AI and cost money | A spam score from Twilio Lookup sends high-risk callers straight to voicemail |
| A scheduler running every minute could pick up the same outbound call twice | Atomic status changes, a FIFO queue with deduplication, and a second read before dialling |
The pattern: every stage hands off through a queue, every hand-off is retried or swept up by a scheduled job, and nothing depends on a single webhook arriving.
Monetization and growth
Revenue. OnSilent sold monthly or yearly subscriptions through Apple in-app purchase, priced in AUD including GST, with a free trial. Each plan included AI minutes, billed per second: half rate for voicemail and outbound calls, 5% for a missed call. A single Lambda verified Apple's signed server notifications and drove a 13-state account lifecycle, from trial through grace period to archive. Archiving released the phone number and deleted the login.
Acquisition. The WordPress site let visitors record a message in the browser and watch it transcribed and triaged live. For real-estate agents, OnSilent sent appraisal leads into the agency's CRM (LeadPlus / Reapit) and read calendars and contacts through Google and Microsoft sign-in.
Outcomes
Working alone for two and a half years, I took OnSilent from a Twilio voicemail script to a production AI receptionist. It handled 9,457 calls and placed 3,929 AI calls before I retired it on 1 October 2026.
- Shipped: a native iOS and Android app, live AI voice for inbound calls, outbound calls and in-app use, a message pipeline from recording to push notification in seconds, Apple subscription billing, CRM and calendar integrations, an admin backoffice and a marketing site with a live demo.
- Reliable: every stage hands off through a queue and has a scheduled sweep behind it, so a dropped webhook or a slow model delays a message instead of losing it.
- Retired cleanly: before anything was deleted, every table, all 7,171 recordings, both user pools and the deployed code of all 83 functions were archived (1.4 GB) with a written restart guide. Customers were told in advance, and services were disabled first, with deletion held back until costs and the services still running had been checked.
My lessons for teams building voice AI products:
- Telephony is the product. Carrier forwarding, short calls, spam and silence took more engineering than the models did.
- Put a queue and a sweeper behind every step that depends on a third party, and treat a webhook as a hint, not a guarantee.
- Record cost, duration and outcome on every call. With that data on each record, prompts, silence thresholds and model choices could be tuned against real traffic instead of guesses.