What just happened? Apple’s Worldwide Developers Conference (WWDC) kicked off on Monday, starting with a preview of what’s coming to iOS 15. The latest version of the iPhone operating system features revamped notifications, improvements to Messages, support for keys and corporate badges in Wallet, and more.
One of the more remarkable new features coming to the iPhone’s native Photos app is called “Live Text.” Using the iPhone’s onboard AI processor, Live Text can recognize writing that appears in photos or screenshots you take. It can visually pinpoint words and phrases in various styles, including script. It also understands several languages. But Live Text doesn’t just recognize text. It makes writing in photos useful by allowing you to highlight and copy any of the words right within the image.
For example, say your teacher or boss has written notes or meeting highlights on the whiteboard (demo below). Instead of manually writing them down, take a picture or point your iPhone’s camera at the text and hit the Live Text button. Select the text by tap-dragging with your finger. Once highlighted hit copy, and then you have a couple of options.
If you have your MacBook handy, Apple’s Handoff feature allows you to paste (command+V) whatever is on your iOS clipboard right into whatever document you are using in macOS. If you don’t have your MacBook, simply paste the text into Notes or the iOS version of Pages. Later, you can open Notes or Pages on your Mac, and the copied text will be right there, thanks to iCloud.
Another nifty feature of Live Text, is that it can recognize phone numbers. Say you have a picture of a business that listed its phone number on its facade. Live Text will identify this as a phone number and create a link for it. When you tap the number in the photo, it will automatically pull up the phone app and dial the number.
I can think of any number of other scenarios where Live Text would come in handy; copying a physical document into a word processor for editing, photographing a sign or menu written in another language, and translating it. The possibilities are left only to the imagination.
Apple did not mention a release date for iOS 15, but it typically launches major iterations in September, right before or alongside the newest iPhone.