Person typing on laptop showing identity hijacking of AI workflows

“Identity Hijacking” of AI Workflows Can Expose Sensitive Internal Information to Attackers With a Simple Request

As documented by researchers with Noma Labs, a new attack on organizational AI workflows can provide attackers with sensitive internal information via nothing more than a polite request to their public-facing AI agent.

The researchers call the approach “Workflow Identity Hijacking.” It really is as simple as asking a company chatbot or automated email management system for some sort of internal business information, such as sales figures or meeting notes. Provided the AI sees nothing malicious in the request itself, it will happily provide an unauthorized user with the requested information. The researchers stress that this is not a type of “prompt injection” attack, because it does not involve tricking the AI into deviating from its internal instructions; it is simply a matter of those instructions not adequately addressing authorization and privileges.

Attack on AI workflows exploits failure to draw internal boundaries

The difficulty with identifying such an attack is that it barely qualifies as one, at least in the technical sense. It demonstrates that AI workflows can be trivially exploited if the AI does not recognize what internal information is supposed to require privileged access.

In this case, the researchers give the example of an email sent to an AI-managed support account asking for quarterly sales numbers from the Finance Director’s most recent email. The only potential bit of trickery is that the researcher frames the question as if they already are a recipient of this person’s emails, and that something expected is missing. However, they are not emailing from any specially authorized account recognized by the AI (which then happily returns the requested information).

As demonstrated, the weakness lies entirely in some company AI workflows never recognizing the need for authentication for this sort of information. The attacker merely needs to access some unauthenticated endpoint managed by the AI that accepts general requests of this type. In addition to a support email or chatbot, this could be something like a shared document, web form or access to a Github Issue.

This is simply a common design oversight. For many modern enterprise AI workflows, the identity and permissions needed to trigger an action (if even present) are totally decoupled from the identity and permission status of the requesting user. The AI can retrieve information on behalf of any unauthorized user that comes along by using its own advanced privileges. This will generally not even trigger any data exfiltration alarms, as the attacker is not breaking any rules or violating any policy.

As the researchers explain, this also does not really fall under the heading of a prompt injection attack because there is not even any manipulation of the internal rules of AI workflows. The AI is simply not checking the requester’s ID and level of access before fetching information for them using its own privileged access.

Morey Haber, Chief Security Advisor at BeyondTrust, summarizes the issue as follows: “In my opinion, this is the asset, privilege, identity triad breaking down in a new place. If you consider multiple assets are a part of the workflow, the privilege is owned by the AI identity doing the executing, the threat actor simply triggered the task for execution, and nothing was verified in the entire process for entitlements, privileges, or behavior.”

How common are vulnerabilities in AI workflows?

The researchers did not provide a comprehensive list of AI workflows that can potentially be compromised by these requests. However, they do point to prior work published in July involving GitHub Agentic Workflows; they also say that the issue was found to be present in Google Workflows and was responsibly disclosed to Google prior to the publication of this report. These incidents join more direct exploitation of prompt injection vulnerabilities in workflow platforms such as n8n and Microsoft Semantic Kernel in recent months.

The hijacking issue is potentially much more widespread, however, as it is typical of standard AI workflows. While the AI models almost certainly have all manner of safety and security guardrails, these measures must specifically address authentication at transitions between workflow steps. It seems that a considerable amount do not.

As with many AI security issues, this is one that developers must ultimately address. The researchers do have some advice for organizations looking to mitigate this possibility on their own, however. This begins with auditing the lowest possible level of trust available to access an AI capable of acting on privileged information (for example, a support email address or web form being available to anyone on the internet).

In general, security controls need to be moved from the model layer to application and infrastructure layers to head this off. The researchers advise identity-aware token delegation to replace static administrative API keys, contextual authorization checkpoints that treat all LLM-generated outputs as untrusted inputs, and asymmetric output separation that walls off AI workflows handling potentially sensitive information from automated response systems available to a broad range of people.

Jeremy London, Director of Engineering, AI & Threat Analytics, at Keeper Security, adds: “Workflow identity hijacking works because authentication and authorization operate on different questions. An AI agent can authenticate itself properly by using valid credentials, following its instructions exactly and generating accurate outputs, but still disclose sensitive data to someone who should never see it. The workflow’s identity is verified, but the requester’s access rights are never checked.”

“Security teams need to treat every AI workflow as a privileged identity with explicit authorization boundaries,” London notes. “That means inventorying which non-human identities each workflow can access, enforcing least-privilege permissions, isolating data retrieval from external communication and requiring identity-aware authorization checks before any agent fulfills requests from untrusted channels. Audit logging matters too, but logging prompts and model outputs alone misses the full picture. You need to record who initiated the request, which identity executed it, what data was accessed and why the authorization succeeded. When you implement proper workflow identity governance, you eliminate the gap between what an agent is permitted to do and who is actually requesting it. That gap is where breaches live, and by closing it, you close a major attack surface.”

 

Senior Correspondent at CPO Magazine