AI Agents Now Learn and Improve Themselves
A significant leap forward is happening in artificial intelligence. New methods are allowing AI agents to learn from their experiences and get smarter over time. This means AI tools could become more helpful and efficient as you use them.
This progress builds on earlier ideas like Andrew Cupsy’s Auto-GPT. The goal is to create AI systems that can improve themselves. They learn from conversations and tasks to become better at what they do.
Understanding Self-Evolving Agents
There are two main types of self-evolving agents. One type, like Auto-GPT, focuses on improving the AI system itself. Its goal is to create a better tool for specific tasks.
The other type, which includes agents like Auto-Dream and Hermès Agent, focuses on learning within conversations. These agents remember what they learn and become more capable over time. This is the area seeing the most practical use right now.
How Agents Learn: Memory and Skills
To make an agent learn continuously, three main parts are key. First is memory, which stores important information. This includes facts about the user, project details, or past feedback.
Second are skills, which give the agent the knowledge to perform specific tasks. Third is the conversation history, which logs interactions so the agent can look back. Different agent systems use these parts in unique ways.
Cloud-Code’s Evolving Memory System
Cloud-Code started with a simple system. It used a single file, `cloud.md`, to store instructions and preferences. But this file quickly became too large and unmanageable.
Later, Cloud-Code added a three-layer memory system. This includes a ‘hot memory’ that’s always in the agent’s main instructions.
It also has ‘warm memory’ that can be loaded when needed. This setup helps the agent remember more details.
The Auto-Memory feature in Cloud-Code instructs the agent to save important information. It creates separate memory files and an index file, `memory.md`. This helps organize what the agent learns.
However, this system relies on the agent remembering to save and update memories. Large language models can sometimes forget steps, leading to outdated information. This can hurt the agent’s performance.
Auto-Dream: Consolidating Knowledge
To address the memory update issue, Cloud-Code introduced Auto-Dream. This feature works in the background after a session ends. It reviews existing memories and conversation history.
Auto-Dream consolidates this information and updates the memory index. This process helps ensure the agent’s knowledge stays current and accurate. It’s a way to automatically keep the agent’s memory in good shape.
Open-Source Agents: Prioritizing Memory
Open-source agents like Open-Claw put memory at the forefront. They use multiple memory files, each for a different aspect of knowledge. A `bootstrap.md` file guides the agent to gather information from the user.
A key feature is the memory search tool. This tool can search across all memory files and conversation history. This makes Open-Claw feel like it remembers things from different sessions.
Open-Claw also focuses on skills. It has instructions for the agent to search for and use relevant skills. This integration of memory and skills makes the agent more capable.
Hermès Agent: Autonomous Skill Creation
Hermès Agent takes self-learning a step further. It introduces two main concepts: autonomous skill generation and memory review.
The agent monitors its own actions. If it performs many steps without creating a new skill, it starts a background process. A separate agent reviews the recent work to see if a new skill can be created to make the process smoother.
This skill creation process looks for useful approaches that involved trial and error. The agent can then create, update, or delete skills using a special tool. It’s designed to ensure skills are maintained and updated.
Hermès Agent also has a safety scan for new skills. It checks for patterns that could be harmful before saving a new skill. This helps maintain the agent’s safety and reliability.
Hermès Agent’s Multi-Layered Memory
Hermès Agent uses a four-tiered memory system. `user.md` stores user preferences and habits. `memory.md` holds project facts and conventions. These are always loaded into the system prompt.
Skills are loaded on demand for specific knowledge. Raw conversation history is saved in a searchable database. For longer-term memory, it can connect to semantic memory systems.
The agent is designed to use skills for most task knowledge. It has an automatic process to extract new memories. After about ten turns, a memory reviewer agent checks for new facts about the user or expectations.
The State-of-the-Art in Agent Learning
The current best approach involves several key elements. Use skills to capture specific knowledge.
Use memory for facts and searchable conversation history. Ideally, have background processes that automatically update knowledge.
You don’t necessarily need to switch to a new agent. Tools are available to enhance existing agents like Open-Claw or Cloud-Code. These tools can improve their memory and self-learning capabilities.
Looking Ahead
The development of self-evolving AI agents is progressing rapidly. These advancements promise more intelligent and adaptive AI tools.
Companies are creating platforms like Koolit, which aim to autonomously monitor business data and prioritize growth actions. Early access to such platforms is becoming available to members of AI builder communities.
Source: This Agent Self-Evolves (Fully explained) (YouTube)
