voice recognition in safari

Apple Safari builds speech recognition into the web with MacOS 11.3

Google Chrome and Microsoft Edge already can turn your words into text.

• Shankland covered the tech industry for more than 25 years and was a science writer for five years before that. He has deep expertise in microprocessors, digital photography, computer hardware and software, internet standards, web technology, and more.

Apple has added support for speech recognition technology into a version of its Safari web browser the company is testing with the release of MacOS 11.3 Big Sur for developers. The speech recognition interface lets websites and web apps listen to spoken words and use the resulting text.

Apple released the developer beta version of MacOS 11.3 on Tuesday. The speech recognition interface is still experimental , but browsers including Google's Chrome and Microsoft's Edge support it . It's the kind of technology useful for tasks like dictating messages into a chat app or online word processor.

Speech recognition is one of the triumphs of modern neural network technology, which processes data in a way inspired by the human brain. Neural networks are trained on real-world data -- in this case countless hours of spoken words -- until an artificial intelligence model can reliably turn speech into text. Related AI technology can turn text into speech.

Together, it's profoundly transformed how we use smartphones , made technology more accessible to people with vision problems, opened up an entirely new market for smart speakers, and surmounted some language barriers.

Browser boost

• Browser privacy boost: Here are the settings to change in Chrome, Firefox, Safari, Edge and Brave
• Google Chrome’s biggest challenge at age 10 might just be its own success
• Google's web app plans collide with Apple's iPhone, Safari rules
• MacOS Big Sur: How to use all the new features in Messages, Maps, Safari and more

Another change in the upcoming version of Safari is an ability to let extensions programmers control the new tab page -- the screen you see when you open a blank new tab. That should bring Safari a step closer to Chrome, which dominates usage of the web today. Safari is embracing Chrome's style of extensions programming with Big Sur, a move that should make life easier for extension developers and for Safari users who need those extensions.

The new Safari version also lets you customize the new tab page by rearranging what the browser shows there -- frequently visited websites, Siri suggestions, browser tabs from Safari running on other devices, and Apple's privacy report.

If you want a taste of what's to come with Safari , you can try the Safari Technology Preview designed to help developers test new versions of the browser with their websites.

Computing Guides

• Best Laptop
• Best Chromebook
• Best Budget Laptop
• Best Cheap Gaming Laptop
• Best 2-in-1 Laptop
• Best Windows Laptop
• Best Macbook
• Best Gaming Laptop
• Best Macbook Deals
• Best Desktop PC
• Best Gaming PC
• Best Monitor Under 200
• Best Desktop Deals
• Best Monitors
• M2 Mac Mini Review
• Best PC Speakers
• Best Printer
• Best External Hard Drive SSD
• Best USB C Hub Docking Station
• Best Keyboard
• Best Webcams
• Best Laptop Backpack
• Best Camera to Buy
• Best Vlogging Camera
• Best Tripod
• Best Waterproof Camera
• Best Action Camera
• Best Camera Bag and Backpack
• Best E-Ink Tablets
• Best iPad Deals
• Best E-Reader
• Best Tablet
• Best Android Tablet
• Best 3D Printer
• Best Budget 3D Printer
• Best 3D Printing Filament
• Best 3D Printer Deals

Looks like no one’s replied in a while. To start the conversation again, simply ask a new question.

SpeechRecognition having issues in Safari (v17.1)

I was trying to implement a speech recognition using the web speech API (see the doc for webkit API) .

Below is the code : (Angular -Typescript -v14)

Now this piece of code works perfectly in Chrome. But when I run this is Safari I found the below issues:

• If Siri / Listen to 'Hey Siri' option is enabled in Mac settings, then I am not getting any response in .onresult method.
• If Siri is disabled, then after the Microphone permission is given, we will need to wait for 2-3 seconds and then speak, otherwise voice is not getting captured.
• Voice is not properly getting captured in Safari like chrome. Sometimes some words are skipped.
• in Chrome, once we speak and stops, then onResult is triggered automatically. But in Safari, I had to write another condition for vSearch.stop() after 5 seconds. Otherwise microphone will be listening continuously.

It is said in the documentation that webkitspeech is supproted in Safari v14 or above.

Kindly share any inputs that will help solving this.

Thanks in advance.

MacBook Pro (2021)

Posted on Feb 23, 2024 1:36 AM

Similar questions

• Why doesn't VoiceControl work with Firefox? I see that VoiceControl works in Safari (in fact I'm using it right now for this). But it doesn't work in Firefox. Why is this? Is there a Firefox setting that can fix it? Or does it require a revised setting, or other workaround, in the macOS? 335 1
• Voice Control - Dictate does not work on websites in Chrome browser Voice Control can be used to type in most MacOS apps, as well as the Chrome address bar, but it does not work on websites. It works in Safari, but not Chrome. Note that this is not "Keyboard - Dictate" which gets disabled when Voice Control is enabled. 3689 1
• Accessibility and VoiceOver and on and off Are use VoiceOver not a problem on a MacBook Pro all the sudden there is a second voice in the background reading when I’m going around my Mac know the difference between this and the VoiceOver they both go together is when I go over the apps on the home screen or anything like that it do not read what they are saying whereas VoiceOver would and is when it on iTunes VoiceOver off to see if that was causing the problem and it’s still switched on with a woman speaking in the background when I’m reading in Safari the two reading alongside each other what is the problem how can I solve it 306 3

Loading page content

Page content loaded

There are no replies.

Safari Enables ‘SpeechRecognition’ by Default in Tech Preview Release

Safari Speech

The release notes for the Safari tech preview fit the speech recognition updates among changes in scrolling, media, and other facets of the browser. At the top of the list is setting SpeechRecognition on by default. The SpeechRecognition interface is what allows the browser to discern someone speaking from other audio, understand what is being said, and formulate a response. It’s a crucial step toward adding voice controls and interactions for any voice assistant. The update also puts the prefix ‘webkit’ in front of SpeechRecognition and changes speech recognition in Safari to adjust when it responds, turning it off in instances when a page’s audio capture is muted or if the page becomes invisible.

The technical update suggests Safari is laying the groundwork to supporting the Web Speech API created by Mozilla, which allows web apps to process voice data and make voice controls feasible. The Web Speech API uses speech recognition to detect and integrate the voice data, while its speech synthesis aspect handles text-to-speech, which lets programs read text on websites and talk back to the user. Web Speech is already supported by Google Chrome on Android and desktop, as well as Microsoft Edge and Samsung’s browser. Safari does not yet support it in either desktop or mobile forms. If Safari does make a move to adding more voice options, it would presumably give that access to Siri, making the voice assistant more useful for web browsing, especially on mobile devices. That may not come about until the next big update with iOS 15.

Mozilla Silence

Apple’s test of speech recognition for Safari and the possible inclusion of Mozilla’s Web Speech API comes just as Mozilla has officially set the end date for Firefox Voice the voice control browser extension it has been beta testing for a year. The extension operated like a voice assistant within the browser awakened by clicking on a microphone icon. It could answer questions via a search engine and open specific web pages if it understood the name of the website. Like Voice Fill, it managed browser tabs and media playback on videos, including YouTube. The extension used the Google Cloud Speech Service, routing voice commands through Google’s servers. Now, the code will be open-source, but Mozilla won’t be supporting it. Whatever voice control Safari adds, it will still be behind Google, which has been introducing Google Assistant as a way to do searches by voice on Android devices.

Mozilla Officially Shuts Down Firefox Voice Browser Extension

Mozilla Updates Massive Open Source Voice Data Collection

Google Assistant Voice Search on Chrome is Expanding to Android

Subscribe to Voicebot Weekly

Latest posts.

McDonald’s Abandons Drive Through AI for Order Taking

Apple Debuts ‘Apple Intelligence’ Generative AI Features Across All Devices

Stability AI Shares Open-Source Generative AI Audio Model for Creative Sound Design

Fable Studio Launches Generative AI TV Show Production Platform for Custom Streaming Content

• Skip to main content
• Skip to search
• Skip to select language
• Sign up for free

SpeechRecognition

The SpeechRecognition interface of the Web Speech API is the controller interface for the recognition service; this also handles the SpeechRecognitionEvent sent from the recognition service.

Note: On some browsers, like Chrome, using Speech Recognition on a web page involves a server-based recognition engine. Your audio is sent to a web service for recognition processing, so it won't work offline.

Constructor

Creates a new SpeechRecognition object.

Instance properties

SpeechRecognition also inherits properties from its parent interface, EventTarget .

Returns and sets a collection of SpeechGrammar objects that represent the grammars that will be understood by the current SpeechRecognition .

Returns and sets the language of the current SpeechRecognition . If not specified, this defaults to the HTML lang attribute value, or the user agent's language setting if that isn't set either.

Controls whether continuous results are returned for each recognition, or only a single result. Defaults to single ( false .)

Controls whether interim results should be returned ( true ) or not ( false .) Interim results are results that are not yet final (e.g. the SpeechRecognitionResult.isFinal property is false .)

Sets the maximum number of SpeechRecognitionAlternative s provided per result. The default value is 1.

Instance methods

SpeechRecognition also inherits methods from its parent interface, EventTarget .

Stops the speech recognition service from listening to incoming audio, and doesn't attempt to return a SpeechRecognitionResult .

Starts the speech recognition service listening to incoming audio with intent to recognize grammars associated with the current SpeechRecognition .

Stops the speech recognition service from listening to incoming audio, and attempts to return a SpeechRecognitionResult using the audio captured so far.

Listen to these events using addEventListener() or by assigning an event listener to the oneventname property of this interface.

Fired when the user agent has started to capture audio. Also available via the onaudiostart property.

Fired when the user agent has finished capturing audio. Also available via the onaudioend property.

Fired when the speech recognition service has disconnected. Also available via the onend property.

Fired when a speech recognition error occurs. Also available via the onerror property.

Fired when the speech recognition service returns a final result with no significant recognition. This may involve some degree of recognition, which doesn't meet or exceed the confidence threshold. Also available via the onnomatch property.

Fired when the speech recognition service returns a result — a word or phrase has been positively recognized and this has been communicated back to the app. Also available via the onresult property.

Fired when any sound — recognizable speech or not — has been detected. Also available via the onsoundstart property.

Fired when any sound — recognizable speech or not — has stopped being detected. Also available via the onsoundend property.

Fired when sound that is recognized by the speech recognition service as speech has been detected. Also available via the onspeechstart property.

Fired when speech recognized by the speech recognition service has stopped being detected. Also available via the onspeechend property.

Fired when the speech recognition service has begun listening to incoming audio with intent to recognize grammars associated with the current SpeechRecognition . Also available via the onstart property.

In our simple Speech color changer example, we create a new SpeechRecognition object instance using the SpeechRecognition() constructor, create a new SpeechGrammarList , and set it to be the grammar that will be recognized by the SpeechRecognition instance using the SpeechRecognition.grammars property.

After some other values have been defined, we then set it so that the recognition service starts when a click event occurs (see SpeechRecognition.start() .) When a result has been successfully recognized, the result event fires, we extract the color that was spoken from the event object, and then set the background color of the <html> element to that color.

Specifications

Browser compatibility.

BCD tables only load in the browser with JavaScript enabled. Enable JavaScript to view data.

• Web Speech API

New WebKit Features in Safari 14.1

Apr 29, 2021

by Jon Davis

Safari 14.1 for macOS Big Sur, iPadOS 14.5, and iOS 14.5 brings new WebKit features, APIs, performance improvements, and improved compatibility for web developers. Take a look.

Flexbox Gap Support

Safari 14.1 now supports the gap property inside Flexbox containers, along with row-gap and column-gap . Gaps in Flexbox make it possible for web developers to create space between Flex items without resorting to annoying margin hacks.

The gap property, of course, has worked inside Grid containers since Safari 12. Because gap is supported for Grid, testing support for the property in a Flexbox formatting context wasn’t possible by using feature queries ( @supports ). By adding support for gap in Flexbox, Safari brings the web closer to widespread compatibility.

For more information, see the “flex containers” definition in the “Row and Column Gutters” section of the CSS Box Alignment specification .

Date & Time Inputs on macOS

In HTML, the date , time , datetime-local attributes for the input element prompt the browser to create date and/or time controls — an interface that’s designed to let the user easily enter a time or a date, usually with a calendar. Safari has supported these input fields on iOS since 2012. Now with Safari 14.1, these fields are supported on macOS as well.

CSS Individual Transform Properties

With WebKit support of Individual Transform Properties, web developers can write CSS rules and keyframe animations in a more straightforward way.

For years , the transform property has provided the ability to scale, rotate, and translate. You could access this power through code like this:

Now, if you wish, you can instead write:

This is a syntactical and conceptual difference — the result is the same. You might find it easier to understand and remember.

This syntax also avoids unintentional overrides of other transform-related properties and eliminates pre-computing intermediate values when using keyframe animations.

You can learn more by reading “ CSS Individual Transform Properties ” on the WebKit blog.

Paint Timing API

A valuable metric to improving the performance of web content is the time it takes for the user-agent to show something to the user. WebKit added the Paint Timing API to its suite of performance APIs to provide this measurement. Developers can measure:

• first-paint for the time it takes to show pixels of anything that is not the user-agent’s default background
• first-contentful-paint to get the time for the user to see content such as text or an image

To learn more about the API, see the Paint Timing specification .

Web Speech API

The Web Speech API in WebKit has supported speech synthesis for some time. Now, Safari supports speech recognition powered by the same speech engine as Siri. That means web developers can enjoy the benefits of high-quality transcription for over 50 languages and dialects. Note that users will need Siri enabled in System Preferences on macOS or Settings in iOS or iPadOS for the API to be available to be used.

For more information on speech recognition and speech synthesis on the web, see the Web Speech API specification .

Web Audio API

As a continuing area of focus, Safari compatibility improvements are ongoing. Updates to the Web Audio API bring it to standards compliance. It is now available unprefixed with support for advanced audio processing via Audio Worklets.

You can learn more about web audio from the Web Audio API specification .

Interoperability Improvements

There are several new interoperability improvements in WebKit:

• Web Animations now work on 122 more properties
• Animation of pseudo-elements beyond ::before and ::after .
• Improved mouse support on iPadOS and in Catalyst apps, including wheel events and hover/pointer media queries

Updated wheel Event handling improves performance and interoperability with other browsers. Wheel handlers registered on root objects (window/document/body) with default arguments will be treated as passive. Pages that want to prevent the default handling of Wheel Events which result from gestures like trackpad swipes on macOS, must now call preventDefault() on the first Wheel Event in the sequence.

MediaRecorder API

WebKit added support for MediaStream Recording, also known as the MediaRecorder API. It allows websites to record audio and video, then encode them using the platform’s available set of default encodings.

Learn more in the MediaStream Recording specification .

WebM Support

WebKit added improved support for WebM media. With Safari 14, WebKit added support for WebM via MSE on iPadOS and macOS. Now, WebKit on macOS supports WebM files containing VP8 or VP9 video tracks and Vorbis audio tracks. Developers can now offer WebM content to users, though users will enjoy the best quality and power efficiency with h.264 or HEVC.

See the WebM Project for details.

JavaScript Improvements

Class fields.

Updates to the JavaScript engine in WebKit adds new support for private class fields to enforce restrictions for static and instance fields in ES6 classes. Developers that used conventions before can switch to built-in support to manage access to properties. Public static class fields are also available, adding to the previously supported public instance class fields.

To learn more, see the public and private instance field proposal .

Internationalization API

New Internationalization API features include Intl.DisplayNames , Intl.ListFormat , and Intl.Segmenter . Intl.DateTimeFormat was updated to support dateStyle and timeStyle options. The Intl.NumberFormat method was updated with support to display measurement units, notation formats, sign display, and narrow symbol currency formatting.

For more information on these formatting methods, see the proposals for Intl.DisplayNames , Intl.ListFormat , Intl.Segmenter , Intl.DateTimeFormat , and Intl.NumberFormat .

WeakRef and FinalizationRegistry

WeakRef supports holding an object that can be garbage collected when there are no strong references to it. The FinalizationRegistry object compliments WeakRef to manage cleanup tasks when a target object is garbage collected.

Read more details in the WeakRefs proposal .

WebAssembly

WebAssembly support, introduced with Safari 11 , is a low-level binary format used as a compilation target for existing languages.

WebAssembly support for the atomic instructions in the Threading specification, are enabled in Safari 14.1. Note, that until Safari supports the COEP/COOP headers, shared memory is not enabled as it could expose users to cross-origin Specter data leaks.

For more information, see the WebAssembly Specification for WASM Threads .

WebAssembly Sign Extension Operator

New sign-extension operator support preserves the number’s sign while extending the number of bits of an integer.

Learn more in the Sign-extension Ops proposal.

JavaScript BigInt Integration

Support for a new JavaScript API allows bidirectional conversion of a JavaScript BigInt value to a WASM 64-bit integer.

See the WebAssembly Specification for toJSValue .

Private Click Measurement

This release features Private Click Measurement – a proposed web standard that enables advertisers to measure the effectiveness of click-through ad campaigns in a privacy-preserving way. This new technology is part of a larger effort to remove cross-site tracking from the web and provide privacy-preserving alternatives where needed.

See “ Introducing Private Click Measurement, PCM ” on the WebKit blog.

Storage Access API Updates

WebKit has improved the Storage Access API to allow per-page storage access and allow nested iframes to request storage access. These interoperability changes are from the ongoing standardization of the Storage Access API together with Mozilla, Microsoft, and the web community. This API has shipped in Safari since 2018 and is part of a larger effort to remove cross-site tracking from the web and provide privacy-preserving alternatives where needed.

For details, see “ Updates to the Storage Access API ” on the WebKit blog.

Web Inspector Updates

The updates to Web Inspector available in these releases include:

• A new three-panel layout in the Elements Tab brings the Styles sidebar into an independent panel alongside the existing details sidebar.
• The new Font panel gives content authors visibility into details of the fonts used on the page.
• Breakpoints in the Sources Tab can now be configured with conditions or actions, reducing the need for stray console.log statements left in production code.

To learn more about Web Inspector features, see the Web Inspector Reference documentation.

Availability

These improvements are available to users running Safari on iPadOS 14.5, iOS 14.5, or Safari 14.1 on macOS Big Sur (11.3), macOS Catalina, or macOS Mojave. These features were also available to web developers in Safari Technology Preview releases. Changes in this release of Safari were included in the following Safari Technology Preview releases: 110 , 111 , 112 , 113 , 114 , 115 , 116 , 117 , 118 , 119 , 120 , 121 , 122 .

Download the latest Safari Technology Preview release to stay at the forefront of future web platform and Web Inspector features. You can also use the WebKit Feature Status page to watch for changes to web platform features you’re interested in.

If you run into any issues, we welcome your bug reports for Safari or WebKit bugs for web content issues. Send us a tweet @webkit to share your thoughts on this release.

How iOS and macOS Dictation Can Learn from Voice Control’s Dictation

Speech recognition has long been the holy grail of computer data input. Or, rather, we have mostly wanted to control our computers via voice—see episodes of Star Trek from the 1960s. The problem has always been that what we want to do with our computers doesn’t necessarily lend itself to voice interaction. That’s not to say it can’t be done. The Mac has long had voice control, and the current incarnation in macOS 10.15 Catalina is pretty good for those who rely on it. However, the simple fact is that modern-day computer interfaces are designed to be navigated and manipulated with a pointing device and a keyboard.

More interesting is dictation, where you craft text by speaking to your device rather than by typing on a keyboard. (And yes, I dictated the first draft of this article.) Dictation is a skill, but it’s one that many lawyers and executives of yesteryear managed to pick up. More recently, we’ve become used to dictating short text messages using the dictation capabilities in iOS.

Dictation in iOS is far from perfect, but when the alternative is typing on a tiny virtual keyboard, even imperfect voice input is welcome. Most frustrating is that you cannot fix mistakes with your voice while dictating, so you end up either having to put up with mistakes in your text or use clumsy iOS editing techniques. By the time you’ve edited your text onscreen, you may as well have typed it from scratch.

macOS has also had dictation features for years, but it has been even less successful and less commonly used than iOS’s feature, in part because it requires so much more setup than just tapping a button on a virtual keyboard.

With iOS 13 and Catalina, Apple significantly beefed up its voice control capabilities and simultaneously introduced what seems to be an entirely different dictation technology—call it “Voice Control dictation,” which I’ll abbreviate to VCD here. In many ways, VCD is better than the dictation built into iOS and macOS. An amalgamation of the two technologies would be ideal.

What’s Wrong and Right with iOS and macOS Dictation

The big problem with dictation in iOS and macOS is that, when it makes mistakes, there’s no way to fix them. But there are other issues. To start, you have to tap a microphone button on the keyboard (iOS) or press a key on the keyboard twice (Mac, set in System Preferences > Keyboard > Dictation) to initiate dictation. That’s sensible, of course, but it does mean that you have to touch your keyboard every time you want to dictate a new message. And that, in turn, means that you cannot just carry on a conversation in Messages, say, without constant finger interaction, which defeats the purpose.

Another problem with dictation in iOS and macOS is that it works for only a certain amount of time—about 60 seconds (iOS) or 40 seconds (macOS) in my testing. As a result, you cannot dictate a document, or even more than a paragraph or two, without having to restart dictation by tapping that microphone button.

But the inability to edit spoken text is the real problem. There is little more frustrating than seeing a mistake being made in front of your eyes and knowing that there is no way to fix it until you stop dictating. And once you have stopped, fixing a mistake is tedious at best, even now that you can drag the insertion point directly in iOS. iOS just isn’t built for text editing. Editing after the fact is much easier on the Mac, of course, but you can’t so much as click the mouse while dictating without stopping the dictation.

On the plus side, dictation in iOS and macOS seems to be able to adjust its recognition based on subsequent words that you speak. You can even see it doing this sometimes, changing a word back-and-forth between two possibilities as you continue to speak. Other times, changes won’t be made until you tap the microphone button to start or your dictation time runs out. Regardless, it’s good—if a little weird—to see Apple adjusting words based on context rather than brute force recognition.

What’s Right and Wrong with Voice Control Dictation

The dictation capabilities built into Apple’s new Voice Control system are quite different. First, instead of navigating to Settings > Accessibility > Voice Control (iOS) or System Preferences > Accessibility > Voice Control (macOS), you can enable Voice Control via Siri—just say “Hey Siri, turn on Voice Control.” Once it’s on, whenever a text field or text area has an insertion point, you can simply speak to dictate text into that spot. You can, of course, also speak commands, but that takes more getting used to.

Unlike the standard dictation, however, VCD stays on indefinitely. You just keep talking, and it will keep typing out whatever you say into your document.

The most significant win, however, is that you can edit the mistakes that VCD makes. For instance, in the previous sentence, it initially capitalized the word “However.” (It has a bad habit of capitalizing words that follow commas.) By merely saying the words “lowercase however,” I was able to fix the problem. Those who are paying attention will note that the word “however” has appeared several times in this article. How does Voice Control know what to fix? It prompts you by displaying numbers next to each instance of the word; you then speak the number of the one you want to change. It’s slow but effective.

There is another approach, too, although it works best on the Mac. If you select some text, which you might do with a finger or a keyboard on an iPhone or iPad, or with a mouse or trackpad on a Mac, you can then direct Voice Control to act on that particular text. For instance, in the previous sentence, VCD didn’t initially capitalize the words “voice control.” That wasn’t a mistake; I’m capitalizing those words because I’m talking about a particular feature, but they would not generally be capitalized. Nevertheless, I can select those two words with the mouse and say, “capitalize that,” to achieve the desired effect. This is a surprisingly effective way to edit. It’s easy and intuitive to select with the mouse and then make a change with your voice without having to move your hands back to the keyboard.

Some mistakes are easily fixed. When I said above, “it prompts you,” VCD gave me the word “impromptu.” All I had to do was say, “change impromptu to it prompts you,” and Voice Control immediately fixed its mistake. When that works, it feels like magic, particularly in iOS. Whenever I’m using a Mac, I prefer to select with the mouse and replace using my voice.

Of course, there are situations where voice editing falls down completely. Several times while dictating this article, I used the word “by.” VCD interpreted that as the word “I” most of the time, and no matter how I tried to edit it with my voice, the best I could do was the word “bye” and the command “delete previous character.” Or, when I wanted the word “effect” above, I ended up with “affect.” It was likely my fault for not pronouncing the word clearly enough. But when I tried “change affect to effect,” Voice Control treated me to “eat fact” the first time and “ethernet fact” the second time. Maddening! It’s strange, because if I just say the word “effect” on its own while emphasizing the “ee” sound at the start, it works fine.

There are other annoyances. With all dictation, you must, of course, speak punctuation out loud, which is awkward and requires retraining your brain slightly. If VCD interprets a word as plural instead of possessive, you can move the insertion point in front of the “s” and say, “apostrophe,” but will put a space in front of the apostrophe, requiring yet more commands to fix the word. And just try getting VCD to write out the word “apostrophe” or “colon” or “period” instead of the punctuation mark.

Another issue that afflicts all dictation systems is the problem with homonyms. Without context, there is simply no way to distinguish between “would” and “wood,” or “its” and “it’s,” or “there” and “their” and “they’re,” by sound alone. VCD has no advantage here; standard dictation may do better.

Careful elocution is essential for recognition success when working with VCD (not that it ever recognizes the word “elocution” correctly). It is probably a good habit to get into. Many of us—myself included—slur our words together while speaking. It’s amazing that speech recognition works at all, given how sloppily we speak.

Unfortunately, VCD doesn’t work everywhere. On the Mac, I can’t get it to work in BBEdit or in Google Docs in a Web browser. In iOS, it has fewer problems, although I’m sure I’ve hit some in the past. I haven’t attempted to produce a comprehensive overview of where it works and where it doesn’t, so suffice it to note that it may not always work when you want.

Another problem, primarily in iOS, is that leaving VCD on all the time is a recipe for confusion because it will pick up other people speaking as well, or even music or other audio playing in the background. Luckily, you can always ask Siri to “turn off voice control” to disable it. Also, if you leave VCD on all the time, it will negatively impact your battery life.

Why Can’t We Have the Best of Both Worlds?

It doesn’t seem as though Apple would have that much work to do to bring the best of VCD’s features to the standard dictation capabilities in iOS and macOS. All that’s necessary is for the company to stop seeing VCD as purely an accessibility feature, instead of something that could be of use to everyone.

The most important change would be to enable dictation to be invoked easily and stay on indefinitely. In iOS, I could imagine tapping the microphone button twice, much like tapping the Shift key twice turns on Caps Lock. On the Mac, perhaps tapping the dictation hotkey three times could lock it on until you turn it off again. That would let you dictate longer bits of text without having to leave Voice Control on at all times or rely on Siri to turn it on and off.

Next, all of VCD’s voice editing capabilities need to migrate to the standard dictation feature. I see no reason why Apple has made VCD so much more capable in this way, and it shouldn’t be hard to reuse the same code.

Finally, you should be able to move the insertion point around and select words while dictating. It’s ridiculous that any such action stops dictation in iOS and macOS now.

If it sounds like I’m suggesting that Apple replace standard dictation with a form of VCD that’s more easily turned on and off, that’s correct. Apart from occasionally improved recognition of words by context as you continue to speak, standard dictation simply doesn’t match up to VCD in nearly any way.

Unfortunately, as far as I can tell in the current betas of iOS 14 and macOS 11 Big Sur, Apple has made no significant changes to either standard dictation or VCD. So we’ll probably have to wait another year or more before such improvement could see the light of day.

Subscribe today so you don’t miss any TidBITS articles!

Every week you’ll get tech tips, in-depth reviews, and insightful news analysis for discerning Apple users. For over 33 years, we’ve published professional, member-supported tech journalism that makes you smarter.

Registration confirmation will be emailed to you.

This site is protected by reCAPTCHA. The Google Privacy Policy and Terms of Service apply.

Comments About How iOS and macOS Dictation Can Learn from Voice Control’s Dictation

Notable replies.

macOS Big Sur beta 1 was released at WWDC on Monday, June 29, 2020. I coughed up the money for a developer account, and downloaded and installed it on Tuesday, because I was keenly interested in what new functionality/features Apple added to Voice Control. I went through the command list to see if Voice Control had any new commands. Specifically looking for the spelling command/mode I’ve been asking for. Remember I have no use of my limbs. So if Voice Control misrecognizes a word, and does not have an appropriate alternative in its correction list, I can’t just grab the keyboard and type in the appropriate word. I need to make the correction by voice and if Voice Control has spelling functionality I could make the correction by voice. Unfortunately no spelling commands/mode yet.

There are a few new commands in macOS Big Sur beta 1. The new commands by category are as follows:

Basic Navigation:

• <item name>
• Find next text <phrase>
• Go to sleep Mac
• Wake up Mac

Overlays & Mouse:

• Show numbers continuously
• Show grid continuously
• Press Space key

Text Selection:

• Select <phrase> emoji

Text Editing:

• Insert <phrase> after

Accessibility:

• VoiceOver activate
• VoiceOver interact
• VoiceOver read all
• VoiceOver stop interacting
• VoiceOver select last item
• VoiceOver select next item
• VoiceOver select previous item
• VoiceOver actions
• VoiceOver applications
• VoiceOver commands
• VoiceOver contextual menu
• VoiceOver item Chooser
• VoiceOver notification menu
• VoiceOver rotor
• VoiceOver next heading
• VoiceOver previous heading
• VoiceOver next link
• VoiceOver previous link
• VoiceOver find
• VoiceOver find backward
• VoiceOver find forward
• VoiceOver verbosity
• VoiceOver help
• VoiceOver more help
• VoiceOver hint
• VoiceOver describe image
• VoiceOver where am i
• VoiceOver speak summary
• VoiceOver stop speaking

Note: Accessibility category is brand new in Big Sur. It was not there in Catalina.

In addition to the new commands, Big Sur Voice Control seems to be quicker. Show numbers in Safari now numbers links on web pages. This makes it much easier surf the web completely hands-free.

If you think that it is frustrating to use the voice options, you try to do it without sight. My father’s macular degeneration has finally reached the point where is both legally and practically blind. While he can see enough to navigate the house, he can no longer interact with the computer or phone. The OS is a real disappointment when it comes to solving these problems. You should try to read some of his email or text messages, Siri really butchers things quite often. You have to develop a bit of skill at deciphering cryptic messages to be successful.

The ability to do mouse actions by voice: click and double-click.

I work a lot in Blackboard, which requires a great deal of clicking to do most things a teacher needs to.

Extra Scripts used to be able to do that, but then a system upgrade disabled it.

If anyone has any ideas how to click by voice, I would love to hear about them!

Thanks so much for the comparison list, @tscheresky !

I’m glad you noted that there is no spelling mode, since that’s a capability I’ve wanted as well, though I didn’t know enough to know what to look for or how to tell it wasn’t there.

I’d encourage everyone interested in Voice Control to submit the lack of spelling mode as feedback to Apple.

@pellerbe , it’s built in! With Voice Control turned on, move the pointer to the right spot and say “Click” or “Double-click.”

Well, I’m glad to hear that! [so are my arms].

Which is the earliest system that has voice control with that included?

Thanks so much!

PS I now have a reason to buy a new Mac, never a bad thing to have.

I would assume, though I don’t know for sure, that it was part of the major Voice Control revamp in Catalina.

Voice Control , the Accessibility feature, was first introduced with Catalina.

Because I could not find the complete list of Voice Control commands online, and I wanted a complete list I could review to come up to speed on Voice Control more quickly, I created the following documents and shared them:

2020 iPadOS 14 Voice Control Commands https://drive.google.com/file/d/1qD_V3YlZmSJ5UOJJlP47-PYNnk1OTDsr/view?usp=sharing

2020 macOS Big Sur Voice Control Commands https://drive.google.com/file/d/1P4dh1H9pzEedCv2-1xXyE37Ej0QW_7-U/view?usp=sharing

Please note : the tabs (a.k.a. sheets) at the bottom of the spreadsheet (a.k.a. workbook) represents the categories for the voice commands. Each tab has the commands for the particular category.

Check out the 2020 macOS Big Sur Voice Control Commands link I just shared. It contains all the voice commands by category. Including mouse commands.

Dictation is something of great interest to those of us with manual dexterity problems, like me. Having used dictation in both macOS and iOS since its inception for this reason, it’s been interesting to track the ups and downs of its usability over this time period, and developing workarounds for its most annoying foibles. (I’m using “interesting” in its most diplomatic sense here.)

This article is greatly appreciated — while I am unwilling to upgrade my Mac to Catalina yet, I’ll see how Voice Control behaves on my iOS devices using the latest system.

By the way I used to use Dragon Dictation and I’m a little surprised the article did not mention it at all. It had pretty good voice editing capabilities.

Editing after the fact is much easier on the Mac, of course, but you can’t so much as click the mouse while dictating without stopping the dictation.

I haven’t experienced any time limitation with dictation for MacOS–through Sierra anyway. I do download ‘enhanced dictation’ for offline use (a GB or two depending on system). Editing by voice was easy on El Cap, though you need to turn on “Enable advanced commands” in Accessibility/Dictation/Dictation Commands… There’s a list of the editing commands there, and you can keep a list open while dictating for reference. In principle it should work the same way on Sierra, but for for some reason instead of editing it just parrots back the editing commands on my system. I may have installed a conflict of some sort.

Clicking the mouse and otherwise editing via mouse and keyboard doesn’t dismiss dictation on either el cap or sierra. It might time out after awhile if you wander off or spend a lot of time just thinking, but mine has just been open and idle for about 5 minutes and is still there.

I haven’t tried dictation in Mojave or Catalina yet since my modern mini has no microphone, but all of the preferences look the same. Has dictation really regressed so much? If so is it because Apple severed relations with Nuance a while back?

Enhanced dictation, as you point out, does behave slightly differently. I’ve tried it many times over the years, especially when my power goes out (as it does frequently) and I lose Internet connectivity, but I always end up going back to the Internet-based dictation because it’s slightly more satisfactory for my use. I do appreciate the reminder, though, because I recently upgraded to high sierra and I don’t think I’ve tried it yet.

I would say dictation has definitely regressed, although of course it’s impossible to say why. For example, lately on my Mac I have not been able to get it to capitalize anything to save my life. Dictation on my phone tends to work a lot better.

Perhaps @tscheresky is more familiar with what was possible in the past, though he would have been relying on Dragon Naturally Speaking then.

Adam, you are right. The only part of macOS built voice recognition I’ve used, going back to Snow Leopard, was its ability to turn Dragon Dictate for Mac’s microphone on after I have turned it off, and restarting Dragon Dictate for Mac after it has crashed.

I don’t see a need for macOS Dictation because Voice Control has everything Dictation has and more. macOS Voice Control is only missing a few features that would make it a complete replacement for Dragon Dictate for Mac.

I still use Dragon dictate for the Mac to this very day and it has a very high accuracy rate. The accuracy rate is the key thing you want in voice recognition because as you noted, it gets frustrating to have non-sensical words show up in your text. One thing that you did not mention in the article is the use of a high quality microphone to improve accuracy on the Mac. A high quality microphone can greatly increase the accuracy rate. When errors do occur, I find it much easier/intuitive and far more efficient to edit errors with the keyboard and mouse.

I do too. I’m using it on my primary computer under Catalina. I even have it working on my secondary computer under macOS Big Sur beta. However those days are numbered. Not only has Dragon for Mac not been supported since October 2018, Dragon speech engine is x86 based. Therefore I’m guessing anyone wanting to move to Apple Silicone (me) won’t be able to do so unless Apple’s Voice Control comes up to speed and adds the missing features needed to become the alternative for Dragon for Mac.

As I see it, here is the path for running Dragon on the Mac. You tested it on Big Sur and it worked and that was something I was personally wondering. Next on Apple silicon it should work just fine with Rosetta 2. If Apple removes Rosetta 2 in the future, then we head for virtualization. Will Parallels and Fusion have a solution for Apple Silicon? My guess is yes. So I think Dragon for the Mac will be sustainable for the foreseeable future.

Apple engineers are some of the best in the business. I’m sure they have done an excellent job on Rosetta 2 emulation. However the constraints for emulating an x86 speech engine are a lot more challenging than emulating say a word processor. I’m sure it can be done but what’s the performance going to be like. Is it going to be quick enough to be usable. Will it have the hooks necessary to control the non-emulated environment.

Here I’m not sure if you’re talking about virtualization to run an older version of macOS, or Windows to run the Windows version of Dragon. Regardless of which, neither of them would work for me. I’m part of the mobility impaired group. I need native Voice Recognition (VR) to command-and-control my Mac in addition to being able to reliably dictate text. If you’re not interested in command-and-control of your Mac, then you should really try Voice Control right now (assuming you’re on Catalina or newer). The majority of the things I’m talking about Voice Control not having are primarily for those individuals, like myself, that need to operate their Mac completely hands-free. If you do not need to command-and-control your Mac by voice, then the current dictation capabilities and editing by voice with Voice Control should meet your needs.

Rossetta 2 does translation and not emulation. It translates the x86 instructions into arm just once at installation time. This results in excellent speed perfomance. So the constraints for running a speech engine and a word processor are no different on Rossetta 2. Apple demonstrated this capability at WWDC using Maya which is far more CPU intensive than a word processor or Dragon.

Hands down, Dragon is the best dictation software. Accuracy in dragon is really good and that alone is the deciding factor for me. So it really comes down to price and since for the Mac you can’t buy Dragon anymore, it is essentially free for those who already have it.

Yup, I have to agree and continue to do medical dictation in Dragon successfully even to Big Sur. Fortunately there will likely be better products that are cloud based and am pleasantly surprised by Fluency by m.modal (though still difficult to correct). If there is a case for machine learning, this is it.

Is there some kind of speech recognition users group? I find these conversations cropping up here and there, but I wonder if there is a more central place.

Join the discussion in the TidBITS Discourse forum

Participants.

How to use voice search in Safari on iPhone and iPad

In Apple's iOS 15 and iPadOS 15 , you can use your voice to search in mobile Safari using Siri. It's available on the best iPhones and best iPads .

Here's more about the feature and how it works. (Hint: You use your voice!)

Using your voice with Safari on iPhone and iPad

Instead of your fingers, you can now do searches on Safari using your voice. To do so:

• Open the Safari app on your device's Home screen.
• Choose the Tab Bar at the bottom of the screen.
• Tap the microphone icon at the far right of the text field.
• Voice your search .

A Safari search acts differently depending on the results. If Siri can identify a specific website based on the search (for example, iMore, it will open it immediately (iMore.com). If, however, your search is more generic ("spinach salad"), you'll see different results as you can see below:

Big changes

Safari has seen significant changes come it's way on iOS/iPad 15. And these changes are one of the seven features you might have missed on the latest update.

Besides voice search, there's Share with You , making it easier to find content sent to you through Messages and new privacy protections. Mobile Safari also includes a bottom tab bar, customizable start page, Tab Groups with syncing, web extensions , and more.

Also explore ...

There are other new features that arrived with iOS/iPad 15, including the latest Focus tool , significant FaceTime changes , new Memoji, and many others.

Master your iPhone in minutes

iMore offers spot-on advice and guidance from our team of experts, with decades of Apple device experience to lean on. Learn more with iMore!

Do you have any questions about voice search on Safari on iPhone and iPad? How about a question concerning iOS 15 or iPadOS 15? If so, let us know in the comments below.

Bryan M. Wolfe has written about technology for over a decade on various websites, including TechRadar, AppAdvice, and many more. Before this, he worked in the technology field across different industries, including healthcare and education. He’s currently iMore’s lead on all things Mac and macOS, although he also loves covering iPhone, iPad, and Apple Watch. Bryan enjoys watching his favorite sports teams, traveling, and driving around his teenage daughter to her latest stage show, audition, or school event in his spare time. He also keeps busy walking his black and white cocker spaniel, Izzy, and trying new coffees and liquid grapes.

Hate the new iOS 18 Photos redesign? Apple explains how it's more "personal" than ever

Apple exec explains why you should pick Apple Music over Spotify: "We've innovated in quality of the music"

Apple Intelligence's biggest Siri features expected next year, with a beta in January

Most Popular

• 2 Apple exec explains why you should pick Apple Music over Spotify: "We've innovated in quality of the music"
• 3 Apple Intelligence's biggest Siri features expected next year, with a beta in January
• 4 Here's what's coming to the Apple Watch Series 10 and Ultra 3 - and what's not
• 5 Google reportedly wants you to ditch Safari on your iPhone to increase its search revenue - putting its deal with Apple at risk

Exclusive: Speech recognition AI learns industry jargon with aiOla’s novel approach

• Share on Facebook
• Share on LinkedIn

We want to hear from you! Take our quick AI survey and share your insights on the current state of AI, how you’re implementing it, and what you expect to see in the future. Learn More

Speech recognition is a critical part of multimodal AI systems. Most enterprises are racing to implement the technology, but even after all the advancements to date, many speech recognition models out there can fail to understand what a person is saying. Today, aiOla , an Israeli startup specializing in this field, took a major step towards solving this problem by announcing an approach that teaches these models to understand industry-specific jargon and vocabulary.

The development enhances the accuracy and responsiveness of speech recognition systems, making them more suitable for complex enterprise settings –  even in challenging acoustic environments. As an initial case study, the startup adapted OpenAI’s famous Whisper model with its technique, reducing its word error rate and improving overall detection accuracy.

However, it says it can work with any speech rec model, including Meta’s MMS model and proprietary models, unlocking the potential to elevate even the highest-performing speech-to-text models.

The problem of jargon in speech recognition

Over the last few years, deep learning on hundreds of thousands of hours of audio has enabled the rise of high-performing automatic speech recognition (ASR) and transcription systems. OpenAI’s Whisper , one such breakthrough model, made particular headlines in the field with its ability to match human-level robustness and accuracy in English speech recognition. 

Countdown to VB Transform 2024

Join enterprise leaders in San Francisco from July 9 to 11 for our flagship AI event. Connect with peers, explore the opportunities and challenges of Generative AI, and learn how to integrate AI applications into your industry. Register Now

However, since its launch in 2022, many have noted that despite being as good as a human listener, Whisper’s recognition performance could decline when applied to audio from complex, real-world environmental conditions. Imagine safety alerts from workers with continuous noise of heavy machinery in the background, activation prompts from people in public spaces or commands with specific utterances and terminology such as those commonly used in medical or legal domains. 

Most organizations using state-of-the-art ASR models (Whisper and others) have tried solving this problem with training tailored to their industry’s unique requirements. The approach does the job but can easily end up taking a toll on the company’s financial and human resources.

“Fine-tuning ASR models takes days and thousands of dollars — and that’s only if you already have the data. If you don’t, then it’s a whole other ballgame. Collecting and labeling audio data could take months and cost many tens of thousands of dollars. For example, if you want to fine-tune your ASR model to recognize a vocabulary of 100 industry-specific terms and jargon, you’d need thousands of audio examples in various settings that would all need to be manually transcribed. If afterward, you wanted to add to your model just one new keyword, then you’d have to retrain on new examples,” Gil Hetz, VP of research at aiOla, told VentureBeat.

To solve this, the startup came up with a two-step “contextual biasing” approach. First, the company’s AdaKWS keyword spotting model identifies domain-specific and personalized jargon (pre-defined in a list of jargon) from a given speech sample. Then, these identified keywords are utilized to prompt the ASR decoder, guiding it to incorporate them into the final transcribed text. This augments the model’s overall speech recognition capability, adapting it to correctly detect the jargon or terms in question.

In the initial tests for keyword-based contextual biasing, aiOla used Whisper – the best model in the category – and tried two techniques to improve its performance. The first, termed KG-Whisper or keyword-guided Whisper, finetuned the entire set of decoder parameters, while the second, termed KG-Whisper-PT or prompt tuning, used only some 15K trainable parameters — thereby being more efficient. In both cases, the adapted models were found to be performing better than the original Whisper baselines on various datasets, even in challenging acoustic environments.

“Our new model (KG-Whisper-PT) significantly improves on the Word Error Rate (WER) and overall accuracy (F1 score) compared to Whisper. When tested on a medical dataset highlighted in our research, it achieved a higher F1 score of 96.58 versus Whisper’s 80.50, and a lower word error rate of 6.15 compared to Whisper’s 7.33,” Hertz said. 

Most importantly, the approach works with different models. aiOla used it with Whisper but enterprises can use it with any other ASR model they have – from Meta’s MMS and proprietary speech-to-text models – to enable a bespoke recognition system, with zero retraining overhead. All they have to do is provide the list of their industry-specific words to the keyword spotter and keep updating it from time to time.

“The combination of these models gives full ASR capabilities that can accurately identify jargon. It allows us to instantly adapt to different industries by swapping out jargon vocabularies without retraining the entire system. This is essentially a zero-shot model, capable of making predictions without having seen any specific examples during training,” Hertz explained.

Saving time for Fortune 500 enterprises

With its adaptability, the approach can come in handy across a range of industries involving technical jargon, right from aviation, transportation and manufacturing to supply chain and logistics. AiOla, on its part, has already started deploying its adaptive model with Fortune 500 enterprises, increasing their efficiency at handling jargon-heavy processes.

“One of our customers, a Fortune 50 global shipping and logistics leader, needed to conduct daily truck inspections before deliveries. Previously, each inspection took around 15 minutes per vehicle. With an automated workflow powered by our new model, this time went down to under 60 seconds per vehicle. Similarly, one of Canada’s leading grocers used our models to inspect product and meat temperatures as required by health departments. This led to time savings that are projected to reach 110,000 hours saved annually, more than $2.5 million in expected savings, and a 5X ROI,” Hertz noted.

aiOla has published the research for its novel approach with the hope that other AI research teams will build on its work. However, as of now, the company is not providing API access to the adapted model or releasing the weights. The only way enterprises can use it is through the company’s product suite, which operates on a subscription-based pricing structure.

Stay in the know! Get the latest news in your inbox daily

By subscribing, you agree to VentureBeat's Terms of Service.

Thanks for subscribing. Check out more VB newsletters here .

An error occured.

China’s robot dog aids blind people with 90% voice recognition accuracy

A team of researchers in China has developed a robotic dog aimed at helping visually impaired people with navigation.

The six-legged robot, developed by Shanghai Jiao Tong University’s School of Mechanical Engineering, uses cameras and sensors to function and identify traffic signals.

The robot’s current voice recognition accuracy rate is over 90 percent, and its response time is less than one second. This helps it react to and understand voice instructions from visually impaired people quickly and correctly.

According to researchers, through clever two-way communication, the robot can simultaneously give vocal orders and offer real-time feedback on its surroundings and gait.

Affordable guide dog tech

In China, there are just over 400 guide dogs for almost 20 million blind people. Pet ownership and service animals are relatively new concepts in the country, meaning many workplaces, restaurants, and other public areas are not yet welcoming to traditional helpers like Labradors.

According to researchers, unlike these dogs, which are limited in supply due to natural breeding limitations and the intense training required, the production of robot guide dogs could be scaled, especially in a major manufacturing hub like China.

“It’s a bit like cars. I can mass-produce them in the same way as cars, so it will become more affordable. I think this could be a very large market, because there might be tens of millions of people in the world who need guide dogs,” Professor Gao Feng, the head of the research team at the institute’s School of Mechanical Engineering, told Reuters .

According to experts, the use of real guide dogs is limited primarily due to their high cost, around $50,000, and the extensive two to three-year training required.

Additionally, only about half of the trained dogs actually go on to serve visually impaired individuals. Seeing-eye robot dogs have the potential to significantly reduce costs, enhance efficiency, and increase accessibility for those in need.

Enhanced accessibility robots

The team’s robot dog is about the size of an English bulldog but slightly wider than a real dog. It has six legs instead of four, which researchers say enhances its stability and results in smoother movements.

The robot boasts a maximum speed of 3 meters per second, catering to needs ranging from slow walking to running. Its unique six-legged design ensures stable, low-noise movement.

Currently, in the field-testing phase, the guide robot is being evaluated through offline demonstrations and functionality tests with visually impaired participants. The development team plans to refine and enhance the robot based on real-time feedback from these users.

According to researchers, implementing the guide robot extends beyond the device itself; it relies on backend big data support, a robust operations and maintenance team, and comprehensive promotional tests.

In a similar project, engineers at Binghamton University’s Computer Science Department in New York have been developing a robotic guide dog to improve accessibility for the visually impaired. Last year, they tested the quadruped robot with a trick-or-treating exercise.

After nearly a year of development, the team created a novel leash-tugging interface using reinforcement learning. With about 10 hours of training, these robots can navigate indoor environments, guide people, avoid obstacles, and detect tugs.

Engineers see great potential in the pulling interface, which allows users to steer the robot by pulling it in a certain direction at hallway intersections. However, they believe that further research and development are needed before the technology can be effectively used in specific contexts.

macOS User Guide

• What’s in the menu bar?
• Work on the desktop
• Search with Spotlight
• Quickly change settings
• Get notifications
• Open apps from the Dock
• Organize your files in the Finder
• Connect to the internet
• Browse the web
• Preview a file
• Take a screenshot
• Change your display’s brightness
• Adjust the volume
• Use trackpad and mouse gestures
• Use Touch ID
• Print documents
• Keyboard shortcuts
• Apps on your Mac
• Work with app windows
• Use apps in full screen
• Use apps in Split View
• Use Stage Manager
• Get apps from the App Store
• Install and reinstall apps from the App Store
• Install and uninstall other apps
• Create and work with documents
• Open documents
• Mark up files
• Combine files into a PDF
• Organize files on your desktop
• Organize files with folders
• Tag files and folders
• Back up files
• Restore files
• Change System Settings
• Choose your desktop wallpaper
• Add and customize widgets
• Use a screen saver
• Add a user or group
• Add your email and other accounts
• Automate tasks with Shortcuts
• Create Memoji
• Change your login picture
• Change the system language
• Make text and other items on the screen bigger
• Set up a Focus to stay on task
• Set up Screen Time for yourself
• Use Dictation
• Send emails
• Send text messages
• Make a FaceTime video call
• Edit photos and videos
• Use Live Text to interact with text in a photo
• Start a Quick Note
• Get directions
• Work across devices using Continuity
• Use iPhone as a webcam
• Use iPhone with Desk View
• Stream audio and video with AirPlay
• Use one keyboard and mouse to control Mac and iPad
• Hand off between devices
• Unlock your Mac with Apple Watch
• Make and receive phone calls on your Mac
• Sync music, books, and more between devices
• Manage Apple ID settings
• Set your Apple ID picture
• What is iCloud?
• What is iCloud+?
• Store files in iCloud Drive
• Share and collaborate on files and folders
• Manage iCloud storage
• Use iCloud Photos
• What is Family Sharing?
• Set up Family Sharing
• Set up Screen Time for a child
• Share purchases with your family
• Watch and listen together with SharePlay
• Share a Photo Library
• Collaborate on projects
• Find content shared with you
• Find your family and friends
• Play games with your friends
• Listen to podcasts
• Watch TV shows and movies
• Read and listen to books
• Read the news
• Track stocks and the market
• Apple Music
• Apple Arcade
• Apple News+
• Podcast shows and channels
• Manage subscriptions in the App Store
• View Apple family subscriptions
• Guard your privacy
• Use Mail Privacy Protection
• Control access to your camera
• Use Sign in with Apple for apps and websites
• Set up your Mac to be secure
• Keep your data safe
• Create a passkey
• Understand passwords
• Keep your Apple ID secure
• Find a missing device
• Get started with accessibility features
• Connect an external display
• Use the built-in camera
• Connect a Bluetooth device
• Use AirPods with your Mac
• Optimize your Mac battery life
• Optimize storage space
• Burn CDs and DVDs
• Control accessories in your home
• Use Windows on your Mac
• Resources for your Mac
• Resources for your Apple devices

Dictate messages and documents on Mac

With Dictation, you can enter text just by speaking, anywhere that you can type it.

On a Mac with Apple silicon , Dictation requests are processed on your device for supported languages—no internet connection is required. When you dictate in a search box, dictated text may be sent to the search provider in order to process the search. Additionally, you can dictate text of any length without a timeout. You can turn off Dictation manually, or it stops automatically when no speech is detected for 30 seconds.

When you dictate on an Intel-based Mac or in a language that doesn’t support on-device dictation, your dictated utterances are sent to Apple to process your requests.

Note: Dictation may not be available in all languages or in all countries or regions, and features may vary. See the macOS Feature Availability webpage to see Dictation languages and on-device processing support. To learn more about how Apple protects your information and lets you choose what you share, click About Ask Siri, Dictation & Privacy at the bottom of Keyboard settings, or see the Apple Privacy website .

If you need to dictate text and control your Mac using your voice instead of a keyboard and trackpad, use Voice Control. See Use Voice Control commands . When Voice Control is on, you can’t use Dictation.

Turn on Dictation

Open Keyboard settings for me

Go to Dictation on the right, then turn it on. If a prompt appears, click Enable.

If you’re asked if you want to improve Siri and Dictation, do one of the following:

Share audio recordings: Click Share Audio Recordings to allow Apple to store audio of your Siri and Dictation interactions from your Mac. Apple may review a sample of stored audio.

Don’t share audio recordings: Click Not Now.

Note: You can delete the audio interactions (which are associated with a random identifier and less than six months old) whenever you like—see Delete Siri and Dictation history .

To dictate using another language, click the Edit button next to Languages, then select a language and dialect. (To remove a language, deselect it.)

To learn more about how Apple protects your information and lets you choose what you share, click About Ask Siri, Dictation & Privacy at the bottom of Keyboard settings, or see the Apple Privacy website .

Dictate text

In an app on your Mac, place the insertion point where you want the dictated text to appear.

On a Mac with Apple silicon , you can type text even while dictating; there’s no need to stop dictation. The microphone icon disappears while you type, and then reappears after you stop typing, so you can continue dictating.

To insert an emoji or a punctuation mark, or perform simple formatting tasks, do any of the following:

Say the name of an emoji, like “heart emoji” or “car emoji.”

Say the name of the punctuation mark, such as “exclamation mark.”

Say “new line” (equivalent to pressing the Return key once) or “new paragraph” (equivalent to pressing the Return key twice). The new line or new paragraph appear when you’re done dictating.

For a list of the commands you can use while dictating, see Commands for dictating text .

For information about setting up Dictation for multiple languages, see Turn on Dictation .

When you’re done, press the Dictation keyboard shortcut or the Escape key. Dictation stops automatically when no speech is detected for 30 seconds.

Ambiguous text is underlined in blue. For example, you may get the result “flour” when you intended the word “flower.” If this is the case, click the underlined word and select an alternative. You can also type or dictate the correct text.

Set the Dictation keyboard shortcut

You can choose a specific Dictation keyboard shortcut or create one of your own.

Go to Dictation on the right, click the pop-up menu next to Shortcut, then choose a shortcut to start Dictation.

To create a shortcut that’s not in the list, choose Customize, then press the keys you want to use. For example, you could press Option-Z.

Change the microphone used for Dictation

The microphone source in Keyboard settings shows which device your Mac is currently using to listen for Dictation.

Go to Dictation on the right, click the pop-up menu next to “Microphone source,” then choose the microphone you want to use for Dictation.

If you choose Automatic, your Mac listens to the device you’re most likely to use for Dictation.

Turn off Dictation

Go to Dictation on the right, then turn it off.

Speech recognition API in Safari is slow on iPhone 14

We have recently noticed that the speech recognition API in Safari is extremely slow and inaccurate, specifically on iPhone 14 with iOS 16.1.1 … but works fine on iPhone 12 with the same iOS 16.1.1.

Does anybody else run into the same issue or have any suggestions?

Hello, we have the same experience like you. For example, this demopage:

https://www.google.com/intl/en/chrome/demos/speech.html

We have tested it on iPhone 12,13 (even iPhone 7s) - it works very well... On iPhone 14 the recognition is not usable at all. It recognizes several words, but it works way more worse than older iPhones.

Our privacy statement has changed. Changes effective July 1, 2024.

Civil Rights Advocates Achieve the Nation’s Strongest Police Department Policy on Facial Recognition Technology

DETROIT, Mich. — Civil rights advocates announced today a settlement in the lawsuit brought on behalf of Robert Williams, who was wrongfully arrested by the Detroit Police Department in 2020 after the department relied on incorrect results from facial recognition technology. The groundbreaking settlement agreement achieves the nation’s strongest police department policies and practices constraining law enforcement’s use of this dangerous technology. The agreement will also lower the likelihood of wrongful arrests, especially for people of color and women, who are substantially more likely to be misidentified by facial recognition technology.

Mr. Williams is a Black man who was wrongfully arrested at his Farmington Hills home in front of his wife and two children for allegedly stealing watches from a Detroit store. His case is one of three known wrongful arrests where Detroit police relied on facial recognition technology. All three who were wrongfully arrested were Black.

Key components of the settlement include:

• Police will be prohibited from arresting people based solely on facial recognition results, or on the results of photo lineups directly following a facial recognition search.
• Police will also be prohibited from conducting a lineup based solely on a facial recognition investigative lead without independent and reliable evidence linking a suspect to a crime.
• Police training on facial recognition technology, including its risks and dangers and that it misidentifies people of color at higher rates.
• An audit will be conducted of all cases since 2017 in which facial recognition technology was used to obtain an arrest warrant.

The court will retain jurisdiction to enforce the agreement for four years. Under the terms of the settlement, Detroit will also pay monetary damages to Mr. Williams and attorneys’ fees.

“The Detroit Police Department’s abuses of facial recognition technology completely upended my life,” said plaintiff Robert Williams . “My wife and young daughters had to watch helplessly as I was arrested for a crime I didn’t commit and by the time I got home from jail, I had already missed my youngest losing her first tooth and my eldest couldn’t even bear to look at my picture. Even now, years later, it still brings them to tears when they think about it.

“The scariest part is that what happened to me could have happened to anyone,” continued Williams . “But, at least with this settlement, it will be far less likely to happen again to another person in Detroit. With this painful chapter of our lives closing, my wife and I will continue raising awareness about the dangers of this technology.”

“This settlement finally brings justice to Detroit, and the Williams family, after years of fighting to expose the flaws of this dangerous technology,” said Phil Mayor, senior staff attorney at the ACLU of Michigan . “Police reliance on shoddy technology merely creates shoddy investigations. Under this settlement, the Detroit Police Department should transform from being a nationwide leader in wrongful arrests driven by facial recognition technology into being a leader in implementing meaningful guardrails to constrain and limit their use of the technology.”

“The multiple wrongful arrests by police in Detroit and other American cities show that face recognition technology is fundamentally dangerous in the hands of law enforcement,” said Nathan Freed Wessler, deputy director of the ACLU Speech, Privacy, and Technology Project . “The most effective way to avoid abuses is for lawmakers to ban police use of the technology, as city councils from Boston to Minneapolis to San Francisco have done. But in jurisdictions where lawmakers have yet to act, police departments should look to Detroit’s new policies, which will seriously mitigate the risk of further false arrests and related harms.”

“We hope this groundbreaking settlement will not only prevent future wrongful arrests of Black people in Detroit, but that it will serve as a model for other police departments that insist on using facial recognition technology,” said Michael J. Steinberg, director of the Civil Rights Litigation Initiative at the University of Michigan Law School . “We are also thrilled that Mr. Williams, who has become a face of movement to stop the misuse of facial recognition, will receive some measure of relief.”

In addition to Mayor, Wessler, and Steinberg, Mr. Williams is represented by ACLU attorneys Dan Korobkin and Ramis Wadood, and CRLI student attorneys Julia Kahn, Collin Christner, Ewurama Appiagyei-Dankah, and Nethra Raman.

The settlement agreement can be found here.

A detailed summary of the settlement agreement can be found here.

Other related court documents can be found here.

Williams v. City of Detroit

This case seeks to hold Detroit police accountable for the wrongful arrest of our client due to officers’ reliance on a false match from face recognition technology.

Source: American Civil Liberties Union

Stay Informed

Sign up to be the first to hear about how to take action.

By completing this form, I agree to receive occasional emails per the terms of the ACLU’s privacy statement.

Learn More About the Issues in This Press Release

• Face Recognition Technology
• Privacy & Technology
• Surveillance Technologies

Related Content

The Supreme Court Ruled, What Now?

Commonwealth v. Foster

Supreme Court Ruling Underscores Importance of Free Speech Online