Anthropic’s latest development gives its Claude AI assistant the ability to control a PC, reportedly just like a person would. The feature, dubbed ‘computer use,’ entered public beta today. With computer use, Claude can be directed to execute tasks such as “looking at a screen, moving a cursor, clicking buttons, and typing text,” according to the company’s .
We’ve built an API that allows Claude to perceive and interact with computer interfaces.
This API enables Claude to translate prompts into computer commands. Developers can use it to automate repetitive tasks, conduct testing and QA, and perform open-ended research. pic.twitter.com/eK0UCGEozm
— Anthropic (@AnthropicAI) October 22, 2024
In theory, this could make the AI even more useful in automating repetitive computer tasks. However, a second focused on computer use acknowledged that this application of Anthropic’s AI models is still early in development and, to paraphrase, buggy as heck. The company said that in internal testing, Claude stopped in the middle of an assigned coding task and began opening images of Yellowstone National Park. While that is uncannily human behavior (who doesn’t want to take a break to stare at natural beauty during the work day?), it’s also a reminder that even the best AI models can have errors.
In addition to unveiling computer use, Anthropic also released an upgraded version of its model alongside a brand new model called Claude 3.5 Haiku that will be released later in October. In August, Anthropic joined OpenAI in to share its work with the US AI Safety Institute.