Skip to content

🌟 The Multi-Agent Framework: First AI Software Company, Towards Natural Language Programming

Update readme.md

didi5d6fa7a68f57
diff --git a/examples/aflow/readme.md b/examples/aflow/readme.md
index 3012a4517..62b92548d 100644
--- a/examples/aflow/readme.md
+++ b/examples/aflow/readme.md
@@ -2,17 +2,17 @@
AFlow is a framework for automatically generating and optimizing Agentic Workflows. It uses Monte Carlo tree search in a code-represented workflow space to find effective workflows, replacing manual development with machine effort. Our approach shows potential to outperform handcrafted workflows on various tasks.
-[Read our paper on arXiv](arxiv_link_here)
+[Read our paper on arXiv](https://arxiv.org/abs/2410.10762)
[Insert performance graph/image here]
## Framework Components
- **Node**: Basic unit of LLM invocation. See `action_node.py` for a flexible interface to control LLM, temperature, format, and prompt.
-- **Operator**: Predefined combinations of Nodes to enhance search efficiency.
-- **Workflow**: [Brief description needed]
-- **Optimizer**: [Brief description needed]
-- **Evaluator**: [Brief description needed]
+- **Operator**: Predefined combinations of Nodes to enhance search efficiency. Encapsulates common operations like Generate, Format, Review, Revise, Ensemble, Test, and Programmer.
+- **Workflow**: A sequence of LLM-invoking nodes connected by edges. Can be represented as graphs, neural networks, or code to express various execution structures.
+- **Optimizer**: Uses LLMs within a Monte Carlo Tree Search variant to explore and refine workflows. Iteratively selects, expands, evaluates, and updates workflows based on performance.
+- **Evaluator**: Assesses workflow performance on given tasks. Provides feedback to guide the optimization process towards more effective workflows.
## Datasets
@@ -24,14 +24,33 @@ For custom tasks, [brief instructions or link to documentation].
## Quick Start
-1. Configure your search in `optimize.py`
+1. Configure your search in `optimize.py`:
+ - Open `examples/aflow/scripts/optimize.py`
+ - Set the following parameters:
+ ```python
+ dataset = "HumanEval" # Choose from: "HumanEval", "MBPP", "GSM8K", "MATH", "HotpotQA", "DROP" or your custom dataset name
+ question_type = "code" # Choose from: "math", "code", "qa"
+ sample = 5 # Number of samples to use for optimization
+ check_convergence = True # Whether to check for convergence
+ optimized_path = "path/to/optimized/workflows" # Path to save optimized workflows
+ initial_round = 1 # Starting round number
+ max_rounds = 20 # Maximum number of optimization rounds
+ ```
+ - Adjust these parameters according to your specific requirements and dataset
2. Set up parameters in `config/config2.yaml` (see `examples/aflow/config2.example.yaml` for reference)
+3. Set the operator you want to use in `optimize.py` and in `xxxx`
+4. Download the init round of six datasets and put them in `xxxxxx`
+5. Add your custom dataset and corresponding evaluation function:
-[Add any additional setup or running instructions]
+- Create a new Python file in the `examples/aflow/benchmark/` directory, named `{custom_dataset_name}.py`
+- Implement the following key functions in this new file:
+ - `load_data`: for loading the dataset
+ - `evaluate_problem`: for evaluating a single problem solution
+ - `evaluate_all_problems`: for evaluating all problems
+ - `save_results_to_csv`: for saving evaluation results
+ - `optimize_{custom_dataset_name}_evaluation`: main evaluation function that integrates the above functionalities
+- Add your custom dataset name and config val_list in `examples/aflow/scripts/evaluator.py`
-## Contributing
-
-[Instructions for contributing, if applicable]
## License
@@ -42,7 +61,10 @@ For custom tasks, [brief instructions or link to documentation].
If you use AFlow in your research, please cite our paper:
```
-[Citation details]
-```
-
-For more information, visit our [project website/documentation].
\ No newline at end of file
+@article{zhang2024aflow,
+ title={AFlow: Automating Agentic Workflow Generation},
+ author={Zhang, Jiayi and Xiang, Jinyu and Yu, Zhaoyang and Teng, Fengwei and Chen, Xionghui and Chen, Jiaqi and Zhuge, Mingchen and Cheng, Xin and Hong, Sirui and Wang, Jinlin and others},
+ journal={arXiv preprint arXiv:2410.10762},
+ year={2024}
+}
+```
\ No newline at end of file