PolymathSimulated worlds for training & evaluating AI agents on long-horizon, multi-tool tasks in any domain.
- Applications & tools: the tools agents interact with (e.g. slack, email, web, excel, github, linear, etc)
- Data: information seeded into the environment which represents an initial state
- Tasks: descriptions for what the agent should accomplish
- Verifiers: criteria that evaluate how well agents perform on tasks in the environment
- Agent(s): AI actor(s) that navigate the environment and complete tasks using the available tools