CRAB (Cross-environment Agent Benchmark)

About

CRAB is an open-source benchmarking framework developed by the CAMEL-AI community. It enables multimodal language model agents to operate on multiple devices simultaneously, offering fine-grained metrics for agent performance evaluation in various environments including GUIs, virtual machines, and physical machines.

Key Features

• Cross-platform and Multi-environment support
• Easy-to-use Configuration
• Novel Benchmarking Suite

Use Cases

• Multimodal Agent Benchmarking
• GUI Agent Evaluation
• Research and Development