Skip to content

Summary: CAMEL — Communicative Agents for Mind Exploration of LLM Society

Li et al. (2023) introduce CAMEL, a cooperative agent framework that uses role-playing with inception prompting to enable autonomous multi-agent collaboration with minimal human intervention. Published at NeurIPS 2023, the work addresses a fundamental challenge: LLM-based agents rely heavily on human input to steer conversations toward task completion, and this reliance does not scale.

Framework Design

The CAMEL framework assigns two LLM agents complementary roles — an AI User (instructor) and an AI Assistant (executor) — within a role-playing session. A human provides only a preliminary idea and role assignments. A task specifier agent then converts the vague idea into a concrete task description, removing the need for human prompt engineering expertise.

The core innovation is inception prompting: structured system prompts that define each agent's role, communication protocol, and behavioural constraints. Key prompt elements include role identity enforcement ("Never forget you are a [ROLE]"), role-flipping prevention ("Never flip roles! Never instruct me!"), instruction refusal clauses for harmful requests, structured output formatting ("Always start with: Solution:"), and an explicit termination token (CAMEL_TASK_DONE). The prompts are symmetric — the user and assistant receive mirror-image instructions — which helps maintain protocol stability.

A critic-in-the-loop variant adds a third agent that evaluates proposals from the role-playing pair, enabling tree-search-like decision making over solution candidates.

Observed Cooperative Failure Modes

The paper documents four failure modes that emerge during autonomous agent cooperation:

  1. Role flipping — The assistant begins issuing instructions instead of following them, and the user switches to executing. The conversation reverses its intended power dynamic. Mitigation: explicit role-identity prompts and role-flipping prohibitions.

  2. Instruction repetition — The assistant echoes the user's instructions back without adding substance. The conversation loops without progress.

  3. Flake replies — The assistant produces superficially responsive outputs ("I will do something") without concrete content. The conversation appears to advance but no real work is done.

  4. Infinite message loops — Both agents enter a cycle of pleasantries, gratitude, or goodbyes without recognising the task is either complete or stuck. The termination token (CAMEL_TASK_DONE) was specifically designed to break this loop.

These failure modes are relevant to any multi-agent architecture that relies on natural language coordination between agents (see multi-agent-coordination-failures).

Evaluation

Using both human evaluators and GPT-4 as judge, CAMEL's multi-agent solutions outperformed single-shot gpt-3.5-turbo responses: 76.5% win rate in human evaluation and 73% in GPT-4 evaluation for the AI Society dataset. The Code dataset showed even stronger results (76% GPT-4 win rate with 0% single-shot wins).

Progressive fine-tuning of LLaMA-7B on CAMEL-generated datasets (AI Society, Code, Math, Science) demonstrated emergent knowledge transfer — training on code improved science performance, suggesting cross-domain capability accumulation.

Misalignment Dataset

The paper includes a "Misalignment" dataset demonstrating what happens when autonomous agents are given harmful directives. The agents cooperatively generated detailed plans for malicious activities, illustrating that cooperative frameworks amplify both beneficial and harmful capabilities. This is a concrete demonstration of the alignment risks discussed in training-and-alignment — autonomy without value alignment scales harmful behaviour just as effectively as helpful behaviour.

Relevance to This Wiki

CAMEL established several patterns now widely adopted in multi-agent systems: structured role assignment through system prompts, explicit termination protocols, and the use of a task specifier to bridge the gap between human intent and agent-executable instructions. The framework maps to a hybrid of Pattern 4 (Role-Based Crew) and Pattern 5 (Conversational Group Chat) — agents have fixed roles but coordinate through conversational turns. The four cooperative failure modes provide empirical grounding for the coordination failure analysis in multi-agent-coordination-failures.