Microsoft Unveils 'Computer Use' Automation in Copilot Studio

Microsoft has announced a new AI-powered feature called "Computer Use" for its Copilot Studio platform that allows agents to directly interact with websites and desktop applications using simulated mouse clicks, menu selections, and text inputs.

According to Charles Lamanna, corporate vice president of business and industry for Microsoft Copilot, the feature marks a major leap forward in Microsoft's vision for intelligent automation by allowing Copilot Studio agents to treat graphical user interfaces similarly to how a human user would.

"Computer Use adapts to changes in apps and websites automatically," said Lamanna. "It adjusts in real time using built-in reasoning to fix issues on its own, so work continues without interruption. It is also built on Copilot Studio's robust security measures and governance frameworks, to help ensure compliance with organizational and industry standards."

Key use cases for the new feature include:

  • Automated data entry across systems that lack direct integrations;
  • Market research automation by collecting and organizing information from online sources; and
  • Invoice processing, streamlining finance operations by extracting and transferring data into accounting systems.

Computer Use is hosted on Microsoft infrastructure. The company said this eliminates the need for organizations to provision or manage dedicated RPA environments, reducing both overhead and maintenance requirements.

Microsoft says the new feature also modernizes traditional robotic process automation (RPA) by making it more resilient and accessible. Rather than requiring extensive coding or scripting, users can describe automation tasks using natural language and refine workflows with side-by-side video previews showing how the agent interprets and executes UI actions.

Additional enhancements include full transparency, with the ability to audit every action the agent performs, including screenshots and reasoning steps.

Microsoft plans to showcase Computer Use in greater detail next month at its Build conference.

Microsoft Begins Rolling Out Controversial Recall Feature

In other Microsoft Copilot news, the company recently announced it has begun rolling out a preview of its Recall feature in Windows in the latest Windows 11 preview build.

Microsoft's Recall feature for Copilot+ PCs offers an AI-driven way to search and retrieve past activity across apps, websites, documents, and images by describing what you remember. The feature works by taking opt-in snapshots of your screen throughout the day and requires Windows Hello authentication to ensure only the device owner can access them.

The feature was initially supposed to start rolling out in preview last June. However, after security concerns from the community arose, the feature was yanked. It was then scheduled for release in October, but was yet again pulled when the feature was found to be taking screengrabs of users' credit card information.

Microsoft has made it clear that Recall is an optional feature and can be turned off at any time. It also said the new version now rolling out adheres to the company's commitment to AI and privacy.

"Recall does not share snapshots or associated data with Microsoft or third parties, nor is it shared between different Windows users on the same device. Windows will ask for your permission before saving snapshots. You are always in control, and you can delete snapshots, pause or turn them off at any time. Any future options for the user to share data will require fully informed explicit action by the user," said Microsoft.

About the Author

Chris Paoli (@ChrisPaoli5) is the associate editor for Converge360.

Featured

  • semi-transparent AI brain with circuit elements under a microscope

    AI 'Microscope' Reveals the Hidden Mechanics of LLM Thought

    Anthropic has introduced new research tools designed to provide a rare glimpse into the hidden reasoning processes of advanced language models — like a "microscope" for AI.

  • digital illustration of Estonia with glowing neural network-like connections spreading across the map

    Estonia to Roll Out ChatGPT Edu for all Secondary Schools

    In a nationwide artificial intelligence program dubbed "AI Leap 2025," the country of Estonia plans to provide free access to leading AI applications for all secondary school students and teachers. The initiative will launch with a rollout of ChatGPT Edu to 20,000 high school students in grades 10-11 and their 3,000 teachers, beginning Sept. 1.

  • DreamBox Math

    Discovery Education Announces Accessibility Enhancements for DreamBox Math

    Discovery Education has updated DreamBox Math, an online math program for K–8 students to supplement core instruction, to improve accessibility for K–5 students, according to a news release. DreamBox Math provides personalized instruction by adapting to individual learners’ responses and providing an engaging, dynamic learning environment.

  • teenager interacts with a chatbot on a computer screen

    Character.AI Rolls Out New Parental Insights Feature Amid Safety Concerns

    Chatbot platform Character.AI has introduced a new Parental Insights feature aimed at giving parents a window into their children's activity on the platform. The feature allows users under 18 to share a weekly report of their chatbot interactions directly with a parent's e-mail address.