On-device voice assistant for iPhone: how it works and what to expect

A voice assistant is three programs working in turn: one that hears you, one that decides what to say and do, and one that speaks. Most assistants run the middle one, and often the other two, on a server. Orvena runs all three on the iPhone. This is what that involves, how to set it up, and where the limits are.

What runs on the phone

Hearing. Orvena uses Apple's on-device speech recognizer from the Speech framework in iOS 26. The first time you speak a language, iOS downloads that language's recognition model; you see it as a short "Getting ready" the first time, and never again for that language.

Deciding. The same model that answers when you type, Qwen 3.5 4B, a 3.1 GB download that runs on the phone's GPU. It can use the phone's tools while you talk: move a meeting, set an alarm, check the weather, find a photo.

Speaking. Two neural speech engines generate the voice on the phone. One covers English, Spanish, French, Italian and Portuguese and is a 331 MB download; the other covers 26 more languages and is 400 MB. Both are fetched after you unlock voice and finish in the background.

Nothing in that loop crosses the network. Airplane mode is the honest test: turn it on, start a voice session, ask what is on your calendar tomorrow, and listen to the answer.

Starting a conversation

• In the app. Tap the waveform button in the ask bar. A full-screen voice view opens, and a keyboard button switches you back to typing at any point without losing the conversation.
• From the Action Button. Open Settings, choose Action Button, swipe to Shortcut, and pick "Talk to Orvena". Press and speak. The same "Talk to Orvena" action works from Shortcuts and from Siri.
• From the Dynamic Island. While a session is running, a Live Activity shows Listening, Working or Approval, and a Stop button ends listening even if you have left the app.

How a turn works

Orvena listens and shows the transcript as you speak. It treats your turn as finished after 1.5 seconds of quiet, or 0.6 seconds once the recognizer has already marked your sentence final, which is what makes the exchange feel quick without cutting you off mid-thought. It then thinks, uses a tool if the request needs one and shows the result as a card, and speaks the answer. When the answer ends it listens again, so a conversation carries on hands-free. A session ends when you tap End, or after 90 seconds with nothing said.

The microphone is off while Orvena thinks and speaks. That is deliberate: two recognizers listening at once is how assistants end up answering themselves. It also means you cannot talk over an answer on the on-device path. To cut one short, tap End, or Stop in the Dynamic Island.

What works offline

Everything above, and every tool that lives on the phone: calendar, reminders, alarms and timers, photos, Apple Health summaries, contacts, files, music. What needs a connection is what needs the outside world: web search, reading a web page, weather, place search and travel times, connected services such as Notion or GitHub, and any cloud model you add with your own key. Each of those is listed, with the condition under which it reaches the network, on the homepage ledger.

The limits

There is no wake word. iOS reserves always-on listening for Siri, so a third-party assistant has to be opened or triggered, which is what the Action Button is for. The model is smaller than the ones behind cloud assistants, and for long research or a specialized field you will notice; for what people ask a phone out loud, the schedule, the weather, a reminder, a photo, an alarm that accounts for travel time, it is the right size. Voice needs an iPhone 15 Pro or newer with 8 GB of memory, because the language model does. And which languages Orvena can listen to depends on Apple's on-device recognizer; the language picker only offers ones it can also speak.

The optional cloud voice

If you have an OpenAI key you can run a live voice session on OpenAI's realtime model instead. It is quicker in the back-and-forth and you can interrupt it mid-sentence, and in exchange your audio streams to OpenAI. Orvena asks for consent before the session starts, asks again if the conversation already contains health or financial data, and labels the voice screen "cloud" in amber rather than "on this iPhone" in green, so you always know which mode you are in.

Common questions

Does Orvena's voice need an internet connection?

No, once the model and the voice packs are downloaded. Recognition, the answer and the speech all run on the phone. Only requests that need the outside world, a web search or the weather, use the network, and only when you ask.

Can I use AirPods or a Bluetooth headset?

Yes. Voice sessions allow both Bluetooth hands-free and stereo profiles, and the built-in speaker is used otherwise.

Is voice free?

Typing is free without limits. Voice conversations are part of Premium Unlock, a one-time purchase of $14.99 with no subscription.

Why is there no wake word?

iOS allows only Siri to listen in the background. Orvena can be started from the Action Button, from Shortcuts, or from the waveform button in the app.