Meetly Auto Engine, Part 3: Cloud Summaries and Benchmarks
How Meetly schedules models on an iPhone, what the optional text-only summary step sends and refuses, where the time goes, and what we measured but dropped.
Meetly transcribes and tags speakers on the iPhone, one heavy model at a time. The only cloud step is the optional summary, which sends transcript text, never audio, with zero data retention requested. For a median 11.4-minute conversation the stages add up to about 92 seconds, and the transcript is readable after 26.

Introduction
In Part 1, Meetly Auto Engine picked a language and a speech model on the iPhone. In Part 2 it transcribed in resumable chunks, filtered recordings not worth more work, tagged speakers and matched saved voices, all on the device. This last part covers how the models share memory, the one optional cloud step, where the time goes and what did not ship.
It closes the series with the lessons that held across all three parts. For the median 11.4-minute conversation, the stages add up to about 92 seconds, and the transcript is readable after the first 26.
Background: the limits a phone sets
Two limits shape this last part. iOS ends an app without warning when it holds more memory than the system allows, and older phones allow less. And the small models a phone can hold are not yet reliable enough at summaries, so the summary is the one step that can leave the device, as text only, and only with your consent.
The problem: memory, privacy and time
Each remaining question pulls against another. Running models side by side saves time but risks memory. Sending text to a larger model gives a better summary, but it must send as little as possible and accept only what the transcript supports. And without measuring each stage in real use, we cannot tell which of these trades is worth making.
- Two heavy models can exceed the memory limit. A speech model and a speaker model together can exceed what iOS lets an app hold on an older phone.
- A summary needs a larger model than a phone can run well today. The cloud step must send no audio, ask providers to keep nothing, and reject anything the transcript does not support.
- Time is spread across stages. We need to know where it goes before deciding what to run in parallel.
One heavy model at a time
Because two heavy models can exceed the memory limit, every model stage asks a scheduler first. The scheduler has one owner slot, and the speech model is unloaded before the speaker model loads. That is safe, and it makes the pipeline strictly sequential, which is the cost the rest of this section tries to recover without giving up the safety.
private func admits(family: String, partners: Set<String>?) -> Bool {
if let partners { return companion == nil && (owner.map { partners.contains($0.family) } ?? true) }
return owner == nil && (companion.map { $0.partners.contains(family) } ?? true)
}
private func grant(_ id: UUID, family: String, partners: Set<String>?) {
if let partners { companion = Companion(id: id, family: family, partners: partners) }
else { owner = Holder(id: id, family: family) }
}A companion slot lets the speaker pass run beside transcription
The scheduler also has one companion slot for a model allowed to run beside one named partner. The speaker model asks to be a companion to Parakeet and nothing else, and when granted, the whole speaker pass runs at the same time as transcription. Figure 1 shows the effect on two long recordings.

Opt-in until we measure it on a phone
We have not turned this on for everyone. It runs only when you opt in, on a phone with at least 7.5 GB of memory that is cool, idle and not recording, and it can be switched off remotely. Until we measure memory and heat on a phone with both models loaded, the 38% stays a simulator result.
The optional cloud step sends text, never audio
Everything so far ran on the device. The summary is the one step that uses the cloud, and it needs your consent first. What is sent: transcript text with speaker labels and timestamps, notes you typed, and photos you attached to notes. What is never sent: audio and voiceprints. The consent is versioned, and it is checked before every request and again at the moment of sending.
Every request asks providers to keep nothing
var provider: [String: JSONValue] = [:]
if request.schema != nil { provider["require_parameters"] = true }
if requireZDR { provider["zdr"] = true }
if denyDataCollection { provider["data_collection"] = "deny" }
if let order = providerOrder(for: [request.model] + request.fallbackModels) {
provider["order"] = .array(order.map { .string($0) })
provider["only"] = .array(order.map { .string($0) })
provider["allow_fallbacks"] = true
}
if !provider.isEmpty { body["provider"] = .object(provider) }Each request asks for zero data retention, denies data collection and is restricted to a named list of providers in a fixed order.
Three passes instead of one prompt
The work is split into three passes rather than one prompt, because a single prompt that cleans, classifies and summarises does all three poorly.
- Pass A, cleanup and names. A small, fast model reads the transcript and returns a list of edits, never a rewrite.
- Pass B, the kind of conversation. At the same time, it classifies the recording as one of eight types, such as a one-to-one, a team meeting, an interview or a voice note, and proposes up to three tags.
- Pass C, the summary. A larger model writes the summary for that type from the cleaned transcript and the names from Pass A.
What the app refuses to accept matters most
An edit is rejected if the text it replaces is not in the transcript, if it inserts words nobody said, or if it changes a spoken amount; the validator understands lakh and crore. A person or date is dropped unless it appears in a line it cites, and a summary bullet is dropped if its citations do not resolve. The stored transcript is never rewritten.
Summary length follows duration
Every type shares a core of recap, key points, decisions, commitments and open questions, and adds its own sections. Length follows duration, from about 250 words to about 700, in the language you chose. In real use the passes took a median of 13.8, 3.0 and 26.7 seconds, and the whole step 40.6 seconds at the median and 74.8 at the 90th percentile (Telemetry).
Results: where the time goes
Put together, this is the median path from tapping Stop to reading a summary for the median conversation of 11.4 minutes. The stages add up to about 92 seconds, a sum of medians rather than a measured end-to-end figure, and the transcript is readable after the first 26. Voice matching takes 30 milliseconds, which is why it compares against every saved voice.

What we measured and did not ship
The useful part of keeping run records is the list of things that looked right and were not. Each of these was built or measured, and each lost to a simpler option or waits for a better model. Listing them is how we stop ourselves from rebuilding them every few months under a new name.
- Steering the speech model toward known names. Names written correctly rose from 49.7% to 57.3% with the word error rate unchanged at 8.2%, but it cost 130% more transcription compute and a 97.5 MB model, and about one replacement in ten was wrong (Mac host).
- Relaxing the clean-window rule (see Part 2). Letting a voiceprint use windows with up to one second of someone else's speech should have helped quick talkers. It lost prints instead: 63 people down to 58.
- Whisper with retries. On a one-hour recording, five retries on difficult segments took 454 seconds and covered 31% of the speech; no retries took 272 seconds and covered 98% (Mac host).
What comes next
Two features are deliberately not on the phone today, and both are the next iterations as phone models get better. Summaries and the extraction of people, dates and topics run in the optional cloud step, because the small models a phone can hold are not yet reliable enough at either. When they are, both move back on device.
- On-device summaries. We built them with Apple's on-device model and a local open model. In our telemetry the cloud path finished a summary for 86% of recordings and the on-device path for 21%, so we paused it; the code stays for the next generation of phone models.
- On-device people, dates and topics. Apple's language tagger found 62% to 65% of people on our earnings-call set, and we judged it too inaccurate to keep. Names come from the validated cloud pass until a small on-device extractor beats it.
- Switching models inside one recording. The decision logic exists and is tested, with a 3-second minimum to judge a region and an 8-second minimum run. It is not wired in yet; today a recording gets one model.
- Phone numbers for the parallel speaker pass, and long-meeting accuracy scored on a physical phone rather than the simulator.
- Cantonese. The detector cannot separate it from Mandarin; SenseVoice identified it on every test clip and is behind a switch while we finish checking it.
- The Mac. It starts every recording on Parakeet with no language check and does not yet call the same engine as the iPhone.
Lessons learned across the series
Three parts, one engine, and a handful of lessons that held at every stage. None of them is specific to speech. They apply to any system that has to make decisions on a phone with small models and cannot stop to ask a person for help each time.
- Ask a small model a smaller question. Whisper Tiny was right 78.2% of the time among all its languages; deciding between English and one other language, recording-level detection reached 94.0% with no extra model (Part 1).
- Prefer specialists you download one at a time. The 198 MB Indian language packs beat a 631 MB general model on both accuracy and speed (Part 1).
- Make every stage resumable. Saved chunks turn an iOS interruption into seconds of repeated work instead of a restart (Part 2).
- Abstain rather than guess, and count the wrong answers. Voice memory refuses whenever two voices could fit; it named 81 of 81 enrolled people correctly and accepted none of 1,725 impostors (Part 2).
- Check constants against real data. A speaker ceiling of 8 blended a 19-person call into 8 people until we raised it to 32 (Part 2).
- Keep the list of what did not ship. It stops us rebuilding the same ideas under new names (Part 3).
Conclusion
None of this is exotic research. It is small models chosen carefully, thresholds that were measured instead of assumed, and a strong preference for saying nothing over saying something wrong. That preference is the part we would keep if we had to rebuild the rest of the engine from scratch.
That is the whole engine. To read the series from the start, go back to Part 1, language detection and model routing.
Record something in any language you speak. The language step, the model it picked and the speakers it found are shown on the conversation page while it works.
Get Meetly for iPhoneQuestions about Meetly Auto Engine
Does Meetly send my audio to the cloud?
No. Meetly Auto Engine transcribes, tags speakers and matches saved voices on your iPhone, and audio is never sent to Meetly or to a model provider. The only step that can use the cloud is the summary, and it is optional and needs your consent. When you allow it, the app sends the transcript text with speaker labels and timestamps, plus any notes or photos you attached, to providers that are asked for zero data retention. Voiceprints are 256 numbers that stay on your devices; your library of saved voices syncs only through your own private iCloud, encrypted, and only if you turn sync on.
Does Meetly write summaries on the iPhone itself?
Not today. We built on-device summaries with Apple's model and a local open model, but in our telemetry the cloud path finished a summary for 86% of recordings and the on-device path for only 21%, so the iPhone now uses the optional cloud step for summaries, names, dates and topics. Transcription, speaker tagging and voice memory stay on the phone. On-device summaries and entity extraction are planned for a future version, once the models a phone can hold are reliable enough, and the code is kept for that.
How long does Meetly take from Stop to a summary?
For the median conversation of 11.4 minutes on the shipped iPhone app, the stages add up to about 92 seconds: transcription 26.3 seconds, speaker tagging 25 seconds, voice matching 0.03 seconds, cleanup and quality gate 0.78 seconds and the optional cloud summary 39.8 seconds. That is a sum of medians across 59 conversations over 30 days, not a measured end-to-end figure. The transcript is readable after the first 26 seconds, and the summary step itself took 40.6 seconds at the median and 74.8 seconds at the 90th percentile.
Why does Meetly not run the speech and speaker models at the same time?
Together they can exceed what iOS lets an app hold on an older phone, and then the system ends the app without warning. So every model stage asks a scheduler first, and the speech model is unloaded before the speaker model loads. A companion slot lets the speaker model run beside Parakeet, which cut import-to-finished time by 38% in the simulator with byte-identical output. It runs only when you opt in, on a phone with at least 7.5 GB of memory that is cool, idle and not recording, until memory and heat are measured on a physical phone.