Computer Use

Use the computer like human-mouse interaction.

Computer Use A specialized structure that enables LLMs to understand any point on the screen.

Currently, Anthropic and OpenAI are actively developing this capability. When agents need to perform actions that don’t have an API or direct access, Computer Use allows them to interact with the screen by clicking, scrolling, or typing—just like a human user.

Within BeaconLabs, we support Computer Use both locally and in the Secure Runtime environment. This allows you to integrate and utilize Computer Use effortlessly.

At present, Computer Use is only supported in the claude-3-sonnet models.

---

Example Usage

pythonCopyEditfrom beaconlabs import Task, Agent, ObjectResponse
from beaconlabs.client.tools import ComputerUse  # Importing Computer Use

task = Task(
  "Open Chrome and search inside the companies section for the latest health companies in YC", 
  tools=[ComputerUse]  # Enabling Computer Use
)

agent = Agent("Browser Use Agent", model="claude/claude-3-5-sonnet")

agent.print_do(task)