Agents prioritize task completion rather than whether they should act. This is a consequence of how they are trained. My student @victorojewale.bsky.social has been investigating this and just wrote a (prize winning) paper arguing why (and how) we need a notion of "informed abstention". Link below.
This is absolutely nuts: hackers are hijacking high-profile Instagram accounts by simply asking Meta's AI chatbot to change the email on the account. Meta's AI does it, hacker gets password reset code, they're in. A staggering security issue
www.404media.co/hackers-simp...
Suresh Venkatasubramanian
www.404media.co
The exploit shows the extreme risk of offloading technical support to AI.
