Apple Introduces 'Apple Intelligence,' ChatGPT Integration

• By John K. Waters
• 06/11/24

Apple has unveiled "Apple Intelligence," a comprehensive software update across its devices, including iOS 18, iPadOS 18, and macOS Sequoia, aiming to provide users with a personalized version of generative AI (GenAI).

Craig Federighi, Apple's Senior Vice President of Software Engineering, introduced Apple's big jump into the GenAI race, emphasizing its ability to understand and assist users through deep integration with their "personal context" while maintaining robust privacy protections.

Apple Intelligence leverages generative models to deliver highly personalized experiences on iPhone, iPad, and Mac, Federighi said. Unlike existing chat tools that rely on "world knowledge," Apple Intelligence draws on the user's personal context to provide more relevant and helpful intelligence, while safeguarding privacy at every step.

Capabilities of Apple Intelligence

Apple Intelligence will enable devices to understand and generate language and images, simplifying interactions across apps. The large language models integrated into Apple Intelligence enhance natural language understanding, making daily tasks quicker and easier. For example, the system can prioritize notifications, offer advanced writing tools, and create personalized images.

Users can access system-wide writing tools to rewrite, proofread, and summarize text to improve their writing across various applications, including Mail, Notes, Safari, Pages, and Keynote. Additionally, the new image capabilities allow users to create original images to enhance conversations, with the ability to personalize images based on the photo library.

Advanced Actions and Personal Context

Apple Intelligence will also have the ability to automate tasks across apps. Users can give commands such as "show me all the photos of mom, Olivia, and me," or "play the podcast that my wife sent the other day." The system's deep understanding of personal context enhances its ability to deliver relevant assistance, processing data from emails, calendar events, and other apps.

Privacy and Architecture

Apple Intelligence provides privacy through on-device processing. The integration of advanced Apple silicon, such as the A17 Pro and M family of chips, ensures the computational power needed for these sophisticated models. Apple has developed private cloud compute to handle more complex requests, ensuring data remains secure and private.

Transforming Siri and Other System Features

Apple Intelligence will also enhance the company's Siri digital assistant to make it more natural and contextually relevant. It will feature a new design, deep integration with the system experience, and support for both voice and text interactions. Users will be able to rely on Siri for a broad range of tasks, including managing personal context and taking actions across apps.

Apple Intelligence also introduces new tools for writing and communication, such as system-wide writing aids and Smart Reply in Mail. The system helps prioritize urgent notifications and enhances focus through features like Reduce Interruptions.

Fun Visual Expressions

For visual expression, Apple Intelligence brings innovative features like GenMoji and Image Playground. These tools allow users to create personalized and playful images, enriching their messaging experience. The Image Playground app provides an intuitive way to experiment with image creation, offering styles like animation, sketch, and illustration.

ChatGPT Integration

Apple Intelligence includes support for ChatGPT, powered by GPT-4o, allowing Siri to tap into its extensive knowledge base. Users can ask Siri complex questions that may require broader world knowledge or specialized domain expertise, such as menu ideas for elaborate meals or decorating advice. Siri will seek permission to share the user's question with ChatGPT and present the answer directly.

Users will also be able to include photos with their questions for more detailed advice. For example, by taking a picture of a space and asking for plant suggestions, Siri can provide tailored recommendations through ChatGPT. This integration also extends to documents, presentations, and PDFs, making it easier for users to get comprehensive assistance without switching between different tools.

ChatGPT will also be integrated into system-wide writing tools. With the new Compose feature, users can generate content, such as custom bedtime stories, and utilize ChatGPT's image capabilities to create illustrations in various styles. Users can access ChatGPT for free without creating an account, while subscribers can connect their accounts for additional features. Apple emphasizes user control over when ChatGPT is used and requires permission before sharing any information.

"We're excited to partner with Apple to bring ChatGPT to their users in a new way," said OpenAI CEO Sam Altman, in a statement. "Apple shares our commitment to safety and innovation, and this partnership aligns with OpenAI's mission to make advanced AI accessible to everyone. Together with Apple, we're making it easier for people to benefit from what AI can offer."

Developer Integration and Enhanced Tools

For developers, Apple has updated its SDKs with new APIs and frameworks, enabling the integration of Apple Intelligence into their apps. The Image Playground experience, for example, can be added to apps like Craft, allowing users to create images to enhance their documents. Writing tools are automatically available within apps using the standard editable text view, providing features like rewrite, proofread, and summarize with minimal development effort.

Apple is also enhancing Siri's capabilities through the App Intents framework, which connects apps with Apple Intelligence. New intents are being defined across operating systems, making it easier for developers to adopt and implement intelligent features within their apps.

Availability

Apple Intelligence "harnesses the power of Apple's advanced silicon," Federighi said, including the A17 Pro and M1 chips. It will be available to try in beta this summer for users in the U.S., with a full rollout as part of iOS 18, iPadOS 18, and macOS Sequoia this fall. Additional features and language support will be introduced over the next year.

Apple CEO Tim Cook called the integration of Apple Intelligence into his company's six platforms "a game changer." He added, "Built in a uniquely Apple way we think Apple intelligence is going to be indispensable to the products that already play such an integral role in our lives."