Recently, the OSWorld team unveiled OSWorld-MCP, the first-ever benchmarking tool specifically designed for a thorough evaluation of computer-using agent products. This innovative tool is geared towards offering developers and users a reliable means to assess product capabilities within real-world settings. It offers a comprehensive evaluation of a model's abilities across various dimensions, encompassing 158 rigorously validated MCP tools. Moreover, it supports seven widely utilized applications and incorporates 25 tools dedicated to robustness testing. Furthermore, it has established 250 tasks to evaluate tool applicability, significantly boosting the model's precision and effectiveness through multi-round invocation configurations. As an open-source project, comprehensive details are accessible on its official website and GitHub repository. OSWorld-MCP not only serves as a potent instrument for product assessment but also establishes a robust groundwork for the progression of future technologies.
