AI Learns Smarter Tricks with ‘Pantry’ Storage
Imagine asking a world-class chef to make a simple peanut butter sandwich. Instead of just grabbing the ingredients, the chef decides to grow peanuts from scratch, wait six months, make the peanut butter, and then start on the sandwich. This sounds silly, but it’s similar to how many advanced AI systems, like ChatGPT and Gemini, currently work.
When these AIs need to recall a basic fact, such as who Alexander the Great was, they go through complex calculations from the very beginning each time. This uses a lot of computer power and time, like our chef planting peanuts for a sandwich. The problem lies in the standard AI design, called transformers. They lack a simple way to quickly look up information they already know.
DeepSeek AI Introduces ‘Engram’ for Smarter AI
Researchers at DeepSeek AI have developed a new technology called ‘Engram’ that could change how future AI systems operate. Think of Engram as giving our busy chef a pantry. Instead of making everything from scratch, the AI can now quickly grab stored information, much like a chef grabbing pre-made ingredients.
This makes the AI much more efficient. But the surprise is that Engram doesn’t just save time; it can also make the AI smarter. When researchers removed some of the complex reasoning parts of an AI and replaced them with Engram, the AI made fewer mistakes. It achieved a better balance between doing complex work and quickly accessing stored facts.
Checking Ingredients for Better Results
DeepSeek also added a way for the AI to check if stored information is still useful, preventing errors. They call this a ‘context-aware gating mechanism.’ It’s like making sure a chef doesn’t use spoiled fish in a dish. If the stored information doesn’t fit the current task, the AI discards it.
When tested, the Engram technique improved the AI’s performance across the board. Unlike many new technologies that are better at some things and worse at others, Engram made the AI better at everything it was tested on. This is a significant leap forward.
How Engram Works and Its Impact
Engram uses a method called ‘n-gram embeddings combined with multi-head hashing.’ In simpler terms, it’s like the AI looking at a short phrase in an order ticket and instantly knowing which shelf in the pantry holds the right ingredient. This approach simplifies AI by automating the easy parts, allowing the AI to focus on more difficult tasks.
Researchers found that when the Engram memory was turned off, the AI’s ability to answer trivia questions dropped by 70%. However, its reading comprehension remained high. This suggests the AI effectively splits its tasks, using Engram mainly for storing facts while its other parts handle understanding and complex reasoning.
This efficiency could lead to cheaper and smarter AI systems. It might also mean more AI tools that people can own and run locally, rather than relying on expensive subscription services. These pocket-sized AIs could run very fast and potentially for free.
Limitations and Future Potential
There are some limitations. If the Engram module is placed too deep within the AI’s processing layers, its accuracy can decrease. This is because the AI might have already spent time calculating information that it could have simply looked up earlier. The pantry needs to be checked at the start of the cooking process, not after the meal is served.
DeepSeek has made this research publicly available, aiming for widespread adoption. This open approach contrasts with proprietary systems that can be very costly to use. The researchers believe Engram could become a key component in many major AI systems, making advanced AI more accessible.
For those interested in running DeepSeek models privately, tools like Lambda are available. The developers emphasize that this kind of innovation, shared openly, is crucial for the future of AI development and accessibility.
Source: DeepSeek Just Fixed One Of The Biggest Problems With AI (YouTube)
