"Now that AI can control your web browser, the next frontier might be to take over your entire computer. At least that's what Seattle-based startup Vercept is trying to do with Vy, a currently free Windows and Mac app that can manipulate your mouse and keyboard to automate tedious or repetitive tasks. You just tell it what you're trying to do, and then it takes control."
"My experiments with Vy have yielded mixed results. If you've ever yelled at ChatGPT for failing to follow instructions, that frustration becomes magnified when AI is piloting your entire computer - tasks you might want to automate might just be done faster manually. Still, I can see some areas where an AI computer agent could be useful, which is why other companies (including Microsoft) are pursuing the same goal."
"This ends up taking a while, though, as each individual action requires Vy to take a screenshot and upload it to its servers for analysis. Everything from opening an app to clicking a menu button requires another screenshot and more time waiting for a response - so a routine that takes 10 seconds for a human might take Vy five minutes."
Vy is a currently free Windows and Mac app that manipulates the mouse and keyboard to automate repetitive tasks by taking control of the computer. The app analyzes the screen through frequent screenshots which are uploaded to servers for processing, then executes mouse and keyboard commands to mimic human actions. Common uses include automating Excel tasks, extracting web data for sharing into apps like Slack, and learning how to use new software. Performance can be slow because each action requires a screenshot round-trip, turning short human routines into minutes of waiting. Early user experiences are mixed, with limited practical speed benefits so far.
Read at Fast Company
Unable to calculate read time
Collection
[
|
...
]