Honestly I think it's memory that's holding agents back. They have a context window (short-lived) and some tricks with file recording, but that's not quite what is needed.
Agents can't look back and correct their mistakes. People make mistakes, remember them, and do better next time. But agents? If they haven't written them down somewhere they'll make the same mistake again.
Perhaps, we need agents that can relearn on the fly. For example fine-tune themselves after each interaction. Perhaps then we wouldn't need to build entire networks of agent interactions. But this of course is not so easy to implement.
I think that such fine-tuning hinges on what do you consider to be a mistake, which is context dependent. Having task-specific finetuned models goes against the status quo of generalization/centralization where few large companies serve a limited amount of models efficiently - both due to inference efficiency and the need/want to control the model.
Having a human-like LLM ecosystem with deep specialization requires a paradigm change in how LLMs are trained - and held accountable. How do we put trust in a specific finetuned LLM rather than the institution behind it? Is there any better approach than the very inefficient evolutionary?
The article is literally talking about swarm self-coordination, which is an emergent [1] property that preserves and compresses the state while running, otherwise it couldn't do what they claim it to do.
