macOS Sequoia introduces iPhone Mirroring, Tiled interface, and AI features

Apple on Monday revealed macOS Sequoia, the next version of the Mac operating system. Version 15 of macOS is loaded with AI features, many of which are also in iOS 18 and iPadOS 18.

Apple highlighted the following macOS 15 features during the keynote. (Updated on 6/11/24 with information on video playback with iPhone Mirroring and TIles on large displays.)

iPhone Mirroring

Mac users are iPhone users, and the two have often felt like separate platforms. iPhone Mirroring closes that gap, allowing users to directly access their iPhones on their Mac desktop. The Mac and iPhone connect wirelessly and users see the iPhone Home screen on the Mac. Users can click and use iPhone apps, iPhone app notifications can be activated, and the appropriate iPhone app launches. Users can also drag and drop photos and videos from the iPhone to the Mac.

The iPhone audio will come through the Mac’s sound output, and iPhone notifications will integrate with the macOS notifications list.

iPhone Mirroring could be a workaround for playing an iPhone game on a bigger Mac screen–the iPhone Mirroring window will be resizable (it currently is not in the beta). It could also be used as a way to watch a video streaming app that’s on the iPhone but not on the Mac, however, the rule of AirPlay comes into play. If an app has AirPlay support, then the video will play. No AirPlay support? No video playback through iPhone Mirroring,

Tiled UI for organizing windows

Mac users work in multiple apps and files at once, which can result in window organization being a chore unto itself. In macOS Sequoia, users can now drag a window to the edge of the screen and macOS will suggest a tiled view of your open windows. Tiles can be arranged side by side or in a corner arrangement. New keyboard shortcuts can be used to manage the tiles.

If you use multiple displays, then the Tiles organization is specific to each display. Also, on larger displays such as the Apple Studio Display, Tiles offers more organizational layouts for your windows.

New Password app

Apple is finally providing a better way to access passwords and other information stored in Keychain by introducing a new Passwords app. Data in the app is encrypted and can sync with other devices through iCloud, and can even sync with Windows via the iCloud for Windows app.

Safari

The new Highlights feature offers a way to get information related to the webpage you are visiting. For example, if you are booking a hotel room, Safari offers a way to quickly get directions on Maps. The Reader feature now offers a summary and table of contents to quickly access the webpage’s content. The new Viewer kicks in when a video is detected on a webpage and includes a full set of controls and the ability to set the video in Picture-in-Picture mode.

Video conferencing improvements

With macOS Sequoia, Apple introduces new features that can be used in video conferencing apps such as FaceTime and Zoom. Presenters can now see a preview of their slides so they can make sure their presentation is going as planned. Participants can now customize their background with a macOS wallpaper or a personal photo.

Apple Intelligence features

Apple has implemented a ton of AI-based features that work across macOS Sequoia, iOS 18, and iPadOS 18. Here’s brief summary of what will be available:

• Writing tools: in apps such as Mail, Pages, Notes, and others, written content will have several features available, including suggestions for revisions, proofreading and copy editing, and more. Mail will offer email summarizations and smart replies.
• Image Playground: apps will be able to create images in one of three styles: Animation, Illustration, or Sketch. In Messages, Image Playground can be used to create personalized images; in Notes, Image Playground can be used to transform sketches into polished pictures.
• Photos: new tools in the Photos app allow users to find photos based on a set of characteristics–show me pictures with a basketball,” for example. A new Clean Up tool can be used to remove items from a picture.
• Siri: many Apple Intelligence features are implemented through Siri, which will have a better understanding of contextual commands, and will be able to perform functions based on the context of its use.

While Apple Intelligence mostly relies on Apple’s own AI models, the Writing Tools and Siri will have access to ChatGPT, a large-language model (LLM) created by OpenAI. In a media session after the WWDC keynote, Apple VP Craig Federighi said that Apple partnered with OpenAI because ChatGPT creates models that Apple doesn’t specialize in. He also stated that Apple is open to implementing other LLMs, and mentioned Google Gemini as an example–but was clear that there is nothing to announce there. “We want to be able to bring users the models of their choice,” said Federighi.

Apple has posted a research paper about how their on-device and server foundation models for Apple Intelligence work.

There’s one big caveat to the macOS Sequoia compatibility list: to use Apple Intelligence features, the Mac requires an M1 chip or later. These features will not work on Intel Macs. Also, the beta of Apple Intelligence features won’t be available until the fall–they’re not included in the current beta releases.

To access the beta, sign in to an Apple ID that is registered as a developer, and then, on your Mac, open System Settings > General > Software Update > Beta Updates and choose the macOS Sequoia Developer beta.