How to interrupt an AI voice assistant on iPhone, and what happens next

Since version 2.4, Orvena listens while it talks. You can cut an answer short, change your mind halfway through it, or say "mm-hm" without stopping it, and a small model on the iPhone's Neural Engine works out which of those you meant. This is what it does with each, the short replies it gives without the language model, and how well it does.

What happens when you talk over Orvena

During a voice session the microphone stays open while Orvena thinks and speaks. The iPhone's echo cancellation removes Orvena's own voice from what the microphone hears, so it only picks up you. As soon as Apple's speech recognizer reports words, Orvena lowers its voice to about a third of its volume, and Cue, its turn model, reads what you said next to the sentence Orvena was in the middle of. There are three outcomes:

For a request, Cue also marks the word where the request starts, so "ok got it, now check Lisbon" reaches the model as "now check Lisbon". When it is unsure, it keeps a word too many: an extra "okay" costs the model nothing, while losing "Lisbon" would.

A clear stop takes effect while you are still talking, as soon as Cue is at least 90% sure. Everything else waits for a pause: 0.7 seconds without new words, or 0.4 seconds once the recognizer has marked your sentence as finished. A "huh", a word to someone else in the room or the TV in the background should count as keep going, so the answer isn't cut off every time someone speaks.

The same works while Orvena is still preparing an answer. If you add "actually, make it 7:30" before it starts speaking, it stops working on the first request and answers the new one.

Replies that skip the language model

Some things you say in a voice session don't need a language model at all. When a new turn starts, Cue first checks whether it is one of nine short phrases Orvena can handle itself, and if it is, Orvena answers right away with one of a few fixed lines:

Cue reads the end of what Orvena last said, so the same words can mean different things. After "Should I add it to your calendar?", an "okay" or a "thanks" is your answer to the question, and it goes to the model. Anything with a request in it goes to the model too: "thanks, now add milk to the shopping list" is a request, not a thank-you. Cue only answers on its own when it is at least 90% sure, and 97% for a goodbye, because that one ends the session.

When a request does go to the model and nothing has been said for 0.8 seconds, Orvena says something short like "Okay, let me check." If the answer still isn't ready after 10 seconds, it says "Still working on it." Neither plays once the answer has started.

Where the decision runs

Cue is a small model of Orvena's own, with 376 million parameters, trained for exactly these two decisions. It runs on the iPhone's Neural Engine, where one decision took about 8 milliseconds with the phone idle and about 10 while the GPU was busy generating, measured on an iPhone 17 Pro Max. That leaves the GPU to the language model that writes the answers, which matters most when that model is Orvena's own, generating on the same phone.

Cue only reads text: the transcript from Apple's on-device recognizer and the words Orvena was saying. Nothing about an interruption leaves the phone. It is a 385 MB download, fetched with the voices after you unlock voice, and the first voice session prepares it for the Neural Engine once, behind the same progress as the voices. If it couldn't download, for example because you were offline, voice still starts and tells you: "Interrupting is off for now. Orvena will finish setting it up next time you start voice."

The model and reference code are on Hugging Face under the Apache 2.0 license.

How often it gets it right

On test sets written separately from its training data, in all 30 of its languages, Cue made the right call on 90.4% of 900 interruptions and 95.2% of 1,800 new turns. It only takes the shortcut when it is confident: of 990 requests meant for the language model, 1 got a short reply instead.

Good to know

• Languages. Cue knows 30 languages. Voice listens in the languages Apple's on-device recognizer supports, and the short replies work in all of them except Norwegian.
• Speaker or headphones. With AirPods or other headphones, Orvena listens throughout. On the iPhone's own speaker it relies on echo cancellation, and without it Orvena listens between answers instead.

Common questions

Which models does interrupting work with?

Orvena's own model, Apple Intelligence, Private Cloud Compute and cloud models you add with your own key. Cue runs on the phone whichever model writes the answer. In a live voice session with your own OpenAI key, OpenAI's realtime model handles interruptions itself.

Does it work offline?

Yes, once Cue and the voices are on the phone and the answer comes from Orvena's model or Apple Intelligence. Recognition, the decision and the voice all run on the iPhone.

Does Orvena keep what I say while it is talking?

It transcribes it with Apple's on-device recognizer, as it does when it listens for your turn, and Cue reads that transcript on the phone. Words it decides were not meant for it are dropped and do not become part of the conversation.

Why does Orvena say "let me check" before answering?

When a request goes to the language model and nothing has been said for 0.8 seconds, Orvena says a short line so you know it heard you. If the answer takes more than 10 seconds, it tells you it is still working.

Is Cue open source?

The weights, a Core ML version for the Neural Engine and reference code are on Hugging Face under Apache 2.0, the same license as the Qwen3 model it is built on. The training data is published there too, under CC BY 4.0.