π The First Self-Improving agents with RL / Prompting Optimization
Python agentaigptmulti-agentpythonreinforcement-learningself-improvementsociety-of-mindswarm-intelligence
Run the following commands to reproduce our experiments in the paper
MMLU
Run the baseline:
PYTHONPATH=. python experiments/run_mmlu.py --mode=DirectAnswer
Run fully-connected swarm ablation:
PYTHONPATH=. python experiments/run_mmlu.py --num-truthful-agents=3 --mode=FullConnectedSwarm
Run randomly-connected swarm ablation:
PYTHONPATH=. python experiments/run_mmlu.py --num-truthful-agents=3 --mode=RandomSwarm
Run the main experiment with optimization and eventual evaluation:
PYTHONPATH=. python experiments/run_mmlu.py --num-truthful-agents=3 --mode=OptimizedSwarm
Mini Crosswords
Run the REINFORCE algorithm for edge optimization with three agents as described in the paper.
PYTHONPATH=. python experiments/run_crosswords.py
HumanEval
Run node optimization that improves the demonstration examples of each node.
PYTHONPATH=. python experiments/run_humaneval.py --learn_demonstration True
GAIA
Run the general assistant tasks.
PYTHONPATH=. python experiments/run_gaia.py