LLM Jacking : brand new attack sequence against AI model

• Facebook
• Twitter
• はてブ
• LINE
• Pocket

Researchers discovered evidence of a reverse proxy for LLMs being used to provide access to the compromised accounts, suggesting a financial motivation.  Every platformer now a days, has host large language model (LLM) services. These platforms provide developers with easy access to various popular models used in LLM-based AI.Cloud vendors have simplified the process of interacting with hosted cloud-based language models by using straightforward CLI commands. The success factor of the attack that verifies if credentials are able to use targeted LLMs via a reverse proxy( e.x. OAI Reverse Proxy) for LLM services. Using this software such as this would allow an attacker to centrally manage access to multiple LLM accounts while not exposing the underlying credentials, or in this case, the underlying pool of compromised credentials. During the attack using the compromised cloud credentials, a user-agent that matches OAI Reverse Proxy was seen attempting to use LLM models.

Financial motivation means If the attackers gather an inventory of useful credentials and sell access to the available LLM models, a reverse proxy like this could allow them to monetize their efforts.

The InvokeModel call is to be sent a legitimate request which causes the “ValidationException” error, but it is useful information for the attacker to have because it tells them the credentials have access to the LLMs and they have been enabled. Otherwise, they would have received an “AccessDenied” error.

For prevention measurement, Monitoring your organization’s use of language model (LLM) services is crucial, and various cloud vendors provide facilities to streamline this process. This typically involves setting up mechanisms to log and store data about model invocations. This attack could have been prevented in a number of ways, including:

• Vulnerability management as Hardening system from scratch.
• Monitoring System prevention by using CSPM/CIEM

• Facebook
• Twitter
• はてブ
• LINE
• Pocket