diff --git a/README.md b/README.md index 790f379..45b94c2 100644 --- a/README.md +++ b/README.md @@ -75,6 +75,7 @@ We're launching a Global AI Agent Hackathon in collaboration with AI Agent ecosy * [đŸ—žī¸ AI Journalist Agent](advanced_ai_agents/single_agent_apps/ai_journalist_agent/) * [🧠 AI Mental Wellbeing Agent](advanced_ai_agents/multi_agent_apps/ai_mental_wellbeing_agent/) * [📑 AI Meeting Agent](advanced_ai_agents/single_agent_apps/ai_meeting_agent/) +* [đŸ§Ŧ AI Self-Evolving Agent](advanced_ai_agents/multi_agent_apps/ai_Self-Evolving_agent/) ### 🎮 Autonomous Game Playing Agents diff --git a/advanced_ai_agents/multi_agent_apps/ai_Self-Evolving_agent/README.md b/advanced_ai_agents/multi_agent_apps/ai_Self-Evolving_agent/README.md new file mode 100644 index 0000000..735f601 --- /dev/null +++ b/advanced_ai_agents/multi_agent_apps/ai_Self-Evolving_agent/README.md @@ -0,0 +1,344 @@ + +
+ + EvoAgentX + +
+ +

+ Building a Self-Evolving Ecosystem of AI Agents +

+ +
+ +[![EvoAgentX Homepage](https://img.shields.io/badge/EvoAgentX-Homepage-blue?logo=homebridge)](https://evoagentx.org/) +[![Docs](https://img.shields.io/badge/-Documentation-0A66C2?logo=readthedocs&logoColor=white&color=7289DA&labelColor=grey)](https://EvoAgentX.github.io/EvoAgentX/) +[![Discord](https://img.shields.io/badge/Chat-Discord-5865F2?&logo=discord&logoColor=white)](https://discord.gg/SUEkfTYn) +[![Twitter](https://img.shields.io/badge/Follow-@EvoAgentX-e3dee5?&logo=x&logoColor=white)](https://x.com/EvoAgentX) +[![Wechat](https://img.shields.io/badge/WeChat-EvoAgentX-brightgreen?logo=wechat&logoColor=white)](./assets/wechat_info.md) +[![GitHub star chart](https://img.shields.io/github/stars/EvoAgentX/EvoAgentX?style=social)](https://star-history.com/#EvoAgentX/EvoAgentX) +[![GitHub fork](https://img.shields.io/github/forks/EvoAgentX/EvoAgentX?style=social)](https://github.com/EvoAgentX/EvoAgentX/fork) +[![License](https://img.shields.io/badge/License-MIT-blue.svg?)](https://github.com/EvoAgentX/EvoAgentX/blob/main/LICENSE) + + +
+ +
+ +

+ +English | įŽ€äŊ“中文 + +

+ +
+ +

+ An automated framework for evaluating and evolving agentic workflows. +

+ +

+ +

+ + +## đŸ”Ĩ Latest News +- **[May 2025]** 🎉 **EvoAgentX** has been officially released! + +## ⚡ Get Started +- [đŸ”Ĩ Latest News](#-latest-news) +- [⚡ Get Started](#-get-started) +- [Installation](#installation) +- [LLM Configuration](#llm-configuration) + - [API Key Configuration](#api-key-configuration) + - [Configure and Use the LLM](#configure-and-use-the-llm) +- [Automatic WorkFlow Generation](#automatic-workflow-generation) +- [Demo Video](#demo-video) + - [✨ Final Results](#-final-results) +- [Evolution Algorithms](#evolution-algorithms) + - [📊 Results](#-results) +- [Applications](#applications) +- [Tutorial and Use Cases](#tutorial-and-use-cases) +- [đŸŽ¯ Roadmap](#-roadmap) +- [🙋 Support](#-support) + - [Join the Community](#join-the-community) + - [Contact Information](#contact-information) +- [🙌 Contributing to EvoAgentX](#-contributing-to-evoagentx) +- [📚 Acknowledgements](#-acknowledgements) +- [📄 License](#-license) + +## Installation + +We recommend installing EvoAgentX using `pip`: + +```bash +pip install git+https://github.com/EvoAgentX/EvoAgentX.git +``` + +For local development or detailed setup (e.g., using conda), refer to the [Installation Guide for EvoAgentX](./docs/installation.md). + +
+Example (optional, for local development): + +```bash +git clone https://github.com/EvoAgentX/EvoAgentX.git +cd EvoAgentX +# Create a new conda environment +conda create -n evoagentx python=3.10 + +# Activate the environment +conda activate evoagentx + +# Install the package +pip install -r requirements.txt +# OR install in development mode +pip install -e . +``` +
+ +## LLM Configuration + +### API Key Configuration + +To use LLMs with EvoAgentX (e.g., OpenAI), you must set up your API key. + +
+Option 1: Set API Key via Environment Variable + +- Linux/macOS: +```bash +export OPENAI_API_KEY= +``` + +- Windows Command Prompt: +```cmd +set OPENAI_API_KEY= +``` + +- Windows PowerShell: +```powershell +$env:OPENAI_API_KEY="" # " is required +``` + +Once set, you can access the key in your Python code with: +```python +import os +OPENAI_API_KEY = os.getenv("OPENAI_API_KEY") +``` +
+ +
+Option 2: Use .env File + +- Create a .env file in your project root and add the following: +```bash +OPENAI_API_KEY= +``` + +Then load it in Python: +```python +from dotenv import load_dotenv +import os + +load_dotenv() # Loads environment variables from .env file +OPENAI_API_KEY = os.getenv("OPENAI_API_KEY") +``` +
+ + +### Configure and Use the LLM +Once the API key is set, initialise the LLM with: + +```python +from evoagentx.models import OpenAILLMConfig, OpenAILLM + +# Load the API key from environment +OPENAI_API_KEY = os.getenv("OPENAI_API_KEY") + +# Define LLM configuration +openai_config = OpenAILLMConfig( + model="gpt-4o-mini", # Specify the model name + openai_key=OPENAI_API_KEY, # Pass the key directly + stream=True, # Enable streaming response + output_response=True # Print response to stdout +) + +# Initialize the language model +llm = OpenAILLM(config=openai_config) + +# Generate a response from the LLM +response = llm.generate(prompt="What is Agentic Workflow?") +``` +> 📖 More details on supported models and config options: [LLM module guide](./docs/modules/llm.md). + + +## Automatic WorkFlow Generation +Once your API key and language model are configured, you can automatically generate and execute multi-agent workflows in EvoAgentX. + +🧩 Core Steps: +1. Define a natural language goal +2. Generate the workflow with `WorkFlowGenerator` +3. Instantiate agents using `AgentManager` +4. Execute the workflow via `WorkFlow` + +💡 Minimal Example: +```python +from evoagentx.workflow import WorkFlowGenerator, WorkFlowGraph, WorkFlow +from evoagentx.agents import AgentManager + +goal = "Generate html code for the Tetris game" +workflow_graph = WorkFlowGenerator(llm=llm).generate_workflow(goal) + +agent_manager = AgentManager() +agent_manager.add_agents_from_workflow(workflow_graph, llm_config=openai_config) + +workflow = WorkFlow(graph=workflow_graph, agent_manager=agent_manager, llm=llm) +output = workflow.execute() +print(output) +``` + +You can also: +- 📊 Visualise the workflow: `workflow_graph.display()` +- 💾 Save/load workflows: `save_module()` / `from_file()` + +> 📂 For a complete working example, check out the [`workflow_demo.py`](https://github.com/EvoAgentX/EvoAgentX/blob/main/examples/workflow_demo.py) + + +## Demo Video + + +[![Watch on YouTube](https://img.shields.io/badge/-Watch%20on%20YouTube-red?logo=youtube&labelColor=grey)](https://www.youtube.com/watch?v=Wu0ZydYDqgg) + +
+ +
+ +In this demo, we showcase the workflow generation and execution capabilities of EvoAgentX through two examples: + +- Application 1: Intelligent Job Recommendation from Resume +- Application 2: Visual Analysis of A-Share Stocks + + +### ✨ Final Results + + + + + + +
+
+ Application 1:
Job Recommendation +
+
+ Application 2:
Stock Visual Analysis +
+ +## Evolution Algorithms + +We have integrated some existing agent/workflow evolution algorithms into EvoAgentX, including [TextGrad](https://www.nature.com/articles/s41586-025-08661-4), [MIPRO](https://arxiv.org/abs/2406.11695) and [AFlow](https://arxiv.org/abs/2410.10762). + +To evaluate the performance, we use them to optimize the same agent system on three different tasks: multi-hop QA (HotPotQA), code generation (MBPP) and reasoning (MATH). We randomly sample 50 examples for validation and other 100 examples for testing. + +> Tip: We have integrated these benchmark and evaluation code in EvoAgentX. Please refer to the [benchmark and evaluation tutorial](https://github.com/EvoAgentX/EvoAgentX/blob/main/docs/tutorial/benchmark_and_evaluation.md) for more details. + +### 📊 Results + +| Method | HotPotQA
(F1%) | MBPP
(Pass@1 %) | MATH
(Solve Rate %) | +|----------|--------------------|---------------------|--------------------------| +| Original | 63.58 | 69.00 | 66.00 | +| TextGrad | 71.02 | 71.00 | 76.00 | +| AFlow | 65.09 | 79.00 | 71.00 | +| MIPRO | 69.16 | 68.00 | 72.30 + +Please refer to the `examples/optimization` folder for more details. + +## Applications + +We use our framework to optimize existing multi-agent systems on the [GAIA](https://huggingface.co/spaces/gaia-benchmark/leaderboard) benchmark. We select [Open Deep Research](https://github.com/huggingface/smolagents/tree/main/examples/open_deep_research) and [OWL](https://github.com/camel-ai/owl), two representative multi-agent framework from the GAIA leaderboard that is open-source and runnable. + +We apply EvoAgentX to optimize their prompts. The performance of the optimized agents on the GAIA benchmark validation set is shown in the figure below. + + + + + + +
+ Open Deep Research Optimization
+ Open Deep Research +
+ OWL Optimization
+ OWL Agent +
+ +> Full Optimization Reports: [Open Deep Research](https://github.com/eax6/smolagents) and [OWL](https://github.com/TedSIWEILIU/owl). + +## Tutorial and Use Cases + +> 💡 **New to EvoAgentX?** Start with the [Quickstart Guide](./docs/quickstart.md) for a step-by-step introduction. + + +Explore how to effectively use EvoAgentX with the following resources: + +| Cookbook | Description | +|:---|:---| +| **[Build Your First Agent](./docs/tutorial/first_agent.md)** | Quickly create and manage agents with multi-action capabilities. | +| **[Build Your First Workflow](./docs/tutorial/first_workflow.md)** | Learn to build collaborative workflows with multiple agents. | +| **[Automatic Workflow Generation](./docs/quickstart.md#automatic-workflow-generation-and-execution)** | Automatically generate workflows from natural language goals. | +| **[Benchmark and Evaluation Tutorial](./docs/tutorial/benchmark_and_evaluation.md)** | Evaluate agent performance using benchmark datasets. | +| **[TextGrad Optimizer Tutorial](./docs/tutorial/textgrad_optimizer.md)** | Automatically optimise the prompts within multi-agent workflow with TextGrad. | +| **[AFlow Optimizer Tutorial](./docs/tutorial/aflow_optimizer.md)** | Automatically optimise both the prompts and structure of multi-agent workflow with AFlow. | + + +đŸ› ī¸ Follow the tutorials to build and optimize your EvoAgentX workflows. + +🚀 We're actively working on expanding our library of use cases and optimization strategies. **More coming soon — stay tuned!** + +## đŸŽ¯ Roadmap +- [ ] **Modularize Evolution Algorithms**: Abstract optimization algorithms into plug-and-play modules that can be easily integrated into custom workflows. +- [ ] **Develop Task Templates and Agent Modules**: Build reusable templates for typical tasks and standardized agent components to streamline application development. +- [ ] **Integrate Self-Evolving Agent Algorithms**: Incorporate more recent and advanced agent self-evolution across multiple dimensions, including prompt tuning, workflow structures, and memory modules. +- [ ] **Enable Visual Workflow Editing Interface**: Provide a visual interface for workflow structure display and editing to improve usability and debugging. + + + +## 🙋 Support + +### Join the Community + +đŸ“ĸ Stay connected and be part of the **EvoAgentX** journey! +🚩 Join our community to get the latest updates, share your ideas, and collaborate with AI enthusiasts worldwide. + +- [Discord](https://discord.gg/SUEkfTYn) — Chat, discuss, and collaborate in real-time. +- [X (formerly Twitter)](https://x.com/EvoAgentX) — Follow us for news, updates, and insights. +- [WeChat](https://github.com/EvoAgentX/EvoAgentX/blob/main/assets/wechat_info.md) — Connect with our Chinese community. + +### Contact Information + +If you have any questions or feedback about this project, please feel free to contact us. We highly appreciate your suggestions! + +- **Email:** evoagentx.ai@gmail.com + +We will respond to all questions within 2-3 business days. + +## 🙌 Contributing to EvoAgentX +Thanks go to these awesome contributors + + + + + +We appreciate your interest in contributing to our open-source initiative. We provide a document of [contributing guidelines](https://github.com/EvoAgentX/EvoAgentX/blob/main/CONTRIBUTING.md) which outlines the steps for contributing to EvoAgentX. Please refer to this guide to ensure smooth collaboration and successful contributions. 🤝🚀 + +[![Star History Chart](https://api.star-history.com/svg?repos=EvoAgentX/EvoAgentX&type=Date)](https://www.star-history.com/#EvoAgentX/EvoAgentX&Date) + + +## 📚 Acknowledgements +This project builds upon several outstanding open-source projects: [AFlow](https://github.com/FoundationAgents/MetaGPT/tree/main/metagpt/ext/aflow), [TextGrad](https://github.com/zou-group/textgrad), [DSPy](https://github.com/stanfordnlp/dspy), [LiveCodeBench](https://github.com/LiveCodeBench/LiveCodeBench), and more. We would like to thank the developers and maintainers of these frameworks for their valuable contributions to the open-source community. + +## 📄 License + +Source code in this repository is made available under the [MIT License](./LICENSE). diff --git a/advanced_ai_agents/multi_agent_apps/ai_Self-Evolving_agent/ai_Self-Evolving_agent.py b/advanced_ai_agents/multi_agent_apps/ai_Self-Evolving_agent/ai_Self-Evolving_agent.py new file mode 100644 index 0000000..a010a46 --- /dev/null +++ b/advanced_ai_agents/multi_agent_apps/ai_Self-Evolving_agent/ai_Self-Evolving_agent.py @@ -0,0 +1,86 @@ +import os +from dotenv import load_dotenv +from evoagentx.models import OpenAILLMConfig, OpenAILLM, LiteLLMConfig, LiteLLM +from evoagentx.workflow import WorkFlowGenerator, WorkFlowGraph, WorkFlow +from evoagentx.agents import AgentManager +from evoagentx.actions.code_extraction import CodeExtraction +from evoagentx.actions.code_verification import CodeVerification +from evoagentx.core.module_utils import extract_code_blocks + +load_dotenv() # Loads environment variables from .env file +OPENAI_API_KEY = os.getenv("OPENAI_API_KEY") +ANTHROPIC_API_KEY = os.getenv("ANTHROPIC_API_KEY") + +def main(): + + # LLM configuration + openai_config = OpenAILLMConfig(model="gpt-4o-mini", openai_key=OPENAI_API_KEY, stream=True, output_response=True, max_tokens=16000) + # Initialize the language model + llm = OpenAILLM(config=openai_config) + + goal = "Generate html code for the Tetris game that can be played in the browser." + target_directory = "examples/output/tetris_game" + + wf_generator = WorkFlowGenerator(llm=llm) + workflow_graph: WorkFlowGraph = wf_generator.generate_workflow(goal=goal) + + # [optional] display workflow + workflow_graph.display() + # [optional] save workflow + # workflow_graph.save_module(f"{target_directory}/workflow_demo_4o_mini.json") + #[optional] load saved workflow + # workflow_graph: WorkFlowGraph = WorkFlowGraph.from_file(f"{target_directory}/workflow_demo_4o_mini.json") + + agent_manager = AgentManager() + agent_manager.add_agents_from_workflow(workflow_graph, llm_config=openai_config) + + workflow = WorkFlow(graph=workflow_graph, agent_manager=agent_manager, llm=llm) + output = workflow.execute() + + # verfiy the code + verification_llm_config = LiteLLMConfig(model="anthropic/claude-3-7-sonnet-20250219", anthropic_key=ANTHROPIC_API_KEY, stream=True, output_response=True, max_tokens=20000) + verification_llm = LiteLLM(config=verification_llm_config) + + code_verifier = CodeVerification() + output = code_verifier.execute( + llm = verification_llm, + inputs={ + "requirements": goal, + "code": output + } + ).verified_code + + # extract the code + os.makedirs(target_directory, exist_ok=True) + code_blocks = extract_code_blocks(output) + if len(code_blocks) == 1: + file_path = os.path.join(target_directory, "index.html") + with open(file_path, "w") as f: + f.write(code_blocks[0]) + print(f"You can open this HTML file in a browser to play the Tetris game: {file_path}") + return + + code_extractor = CodeExtraction() + results = code_extractor.execute( + llm=llm, + inputs={ + "code_string": output, + "target_directory": target_directory, + } + ) + + print(f"Extracted {len(results.extracted_files)} files:") + for filename, path in results.extracted_files.items(): + print(f" - {filename}: {path}") + + if results.main_file: + print(f"\nMain file: {results.main_file}") + file_type = os.path.splitext(results.main_file)[1].lower() + if file_type == '.html': + print(f"You can open this HTML file in a browser to play the Tetris game") + else: + print(f"This is the main entry point for your application") + + +if __name__ == "__main__": + main() diff --git a/advanced_ai_agents/multi_agent_apps/ai_Self-Evolving_agent/requirements.txt b/advanced_ai_agents/multi_agent_apps/ai_Self-Evolving_agent/requirements.txt new file mode 100644 index 0000000..5af3cd7 --- /dev/null +++ b/advanced_ai_agents/multi_agent_apps/ai_Self-Evolving_agent/requirements.txt @@ -0,0 +1,52 @@ +pytest +pytest-cov +pytest-mock +pytest-asyncio +pytest-subtests +pytest-json-report +ruff + +sympy +stopit +scipy +setuptools +tree_sitter +tree_sitter_python +antlr4-python3-runtime==4.11 +tenacity +networkx>=3.3 +nltk>=3.9.1 +numpy>=1.26.4 +openai>=1.55.3 +litellm>=1.55.6 +# pydantic>=2.9.0 +pydantic>=2.9.0,<=2.10.6 +pydantic-settings==2.8.1 +pydantic_core>=2.23.2,<=2.27.2 +loguru>=0.7.3 +pandas>=2.2.3 +matplotlib>=3.10.0 +# --extra-index-url https://download.pytorch.org/whl/cu118 +# torch==2.2.1 +# torchvision==0.17.1 +# torchaudio==2.2.1 +transformers>=4.47.1 +datasets>=3.4.0 +faiss-cpu==1.8.0.post1 +textgrad>=0.1.8 + +# fast api dependencies +fastapi>=0.115.11 +motor>=3.7.0 +uvicorn>=0.34.0 +sqlalchemy>=2.0.38 +python-jose>=3.3.0 +passlib>=1.7.4 +python-multipart>=0.0.6 +bcrypt>=4.0.1 +celery>=5.3.4 +redis>=5.0.0 +httpx>=0.24.1 +asgi-lifespan>=1.0.1 +python-dotenv>=1.0.0 +jwt>=1.3.1