This is an interesting post, and I agree with you that many elements about these incidents are sensationalized by the media. At the same time, I believe that you are downplaying the significance of these events, especially the HF incident.
As it appears now, agents at the HF incident were not instructed to "hack a company". They "autonomously" decided to do so when they reasoned that the answer to their problem could more easily be found inside HF than elsewhere.
However, even if they individually were instructed to hack a company, which they weren't, the impressive feat of this incident wasn't necessarily the decision to hack, it was the complex social coordination that emerged when the AI agents decided to work together. Creating a message board, forming social hierarchies, having complex division of labor, and even self-sacrifice! None of these things was prompted, they all emerged from the individual agents having the same or similar goals. I wrote about it more in my latest post if you're interested.
Hi Or, thanks for commenting. My view on this is that people are reading too much into the HF incident and anthropomorphising the behaviour. Nothing about it sounds out of the ordinary to me, considering the kind of agentic work that these models are already used and trained for.
About the clever collaboration, METR, the group researching the incident, wrote "agents had been trained to collaborate with other agents in certain cases, which could have explained this behavior; investigating this was out of scope."
If you've trained models to collaborate, and they end up collaborating, that's not surprising behaviour. Similarly, sacrificing something as part of a strategy to reach your goal, isn't a new concept. It's all in the training data. So reading about that behaviour didn't really awe me the way it has some others. You also have to ask yourself, if this was really such an impressive feat, why was it "out of scope" of their investigation into all this?
This is an interesting post, and I agree with you that many elements about these incidents are sensationalized by the media. At the same time, I believe that you are downplaying the significance of these events, especially the HF incident.
As it appears now, agents at the HF incident were not instructed to "hack a company". They "autonomously" decided to do so when they reasoned that the answer to their problem could more easily be found inside HF than elsewhere.
However, even if they individually were instructed to hack a company, which they weren't, the impressive feat of this incident wasn't necessarily the decision to hack, it was the complex social coordination that emerged when the AI agents decided to work together. Creating a message board, forming social hierarchies, having complex division of labor, and even self-sacrifice! None of these things was prompted, they all emerged from the individual agents having the same or similar goals. I wrote about it more in my latest post if you're interested.
https://substack.com/home/post/p-216209226
Hi Or, thanks for commenting. My view on this is that people are reading too much into the HF incident and anthropomorphising the behaviour. Nothing about it sounds out of the ordinary to me, considering the kind of agentic work that these models are already used and trained for.
About the clever collaboration, METR, the group researching the incident, wrote "agents had been trained to collaborate with other agents in certain cases, which could have explained this behavior; investigating this was out of scope."
If you've trained models to collaborate, and they end up collaborating, that's not surprising behaviour. Similarly, sacrificing something as part of a strategy to reach your goal, isn't a new concept. It's all in the training data. So reading about that behaviour didn't really awe me the way it has some others. You also have to ask yourself, if this was really such an impressive feat, why was it "out of scope" of their investigation into all this?