Large Language Models (LLMs) are typically used to answer questions, summarize text, write code, or generate images. However, these models can also be manipulated by cybercriminals to carry out illegal activities. MALLAs, an acronym for Malicious LLM Applications, are applications that integrate the use of Large Language Models specialized in generating and automating cyberattacks. The easy availability of these tools makes it possible even for inexperienced users to carry out sophisticated attacks, thus increasing both the frequency and complexity of threats.

In this article, we provide an overview of MALLAs, examining their ecosystem, promotion strategies, main mitigation techniques, and presenting some concrete examples of their use.

This article is written for educational and informational purposes only. The author does not endorse the use of these techniques for illegal or unethical activities. Use of the information contained in this article is at your own risk and responsibility. We encourage you to use these techniques only in authorized test environments and in compliance with local laws and cybersecurity regulations.

What are MALLAs?

Malicious LLM Applications (MALLAs) are applications that integrate the use of Large Language Models specialized in generating and automating cyberattacks. You may wonder how it is possible to exploit LLMs for illicit purposes if they leverage the so-called guardrails, namely, mechanisms designed to monitor and regulate the behavior of the LLM to prevent harmful outputs such as misinformation, bias, privacy violations, or illegal content. There are two possibilities:

• Uncensored Large Language Models, i.e., LLMs that are designed without guardrails and can therefore be used without any restrictions.
• Prompt Jailbreak attacks, i.e., attacks that exploit ad-hoc prompts to circumvent guardrails and thus use the LLM freely.

MALLAs’ ecosystem

MALLA’s ecosystem can be represented using the following image:

1. MALLA vendors develop these applications using LLMs specialized in cyberattacks;
2. MALLAs are promoted through typical channels used by cybercriminals;
3. A malicious user selects and purchases a MALLA from those available on the aforementioned channels;
4. The malicious user interacts with the MALLA in the same way we interact with our trusted chatbot: a prompt is submitted, processed by the LLM, and a response is returned;
5. Once the user can interact with these applications without any restrictions, they can generate phishing pages, phishing emails, and malware;

There exist also public platforms such as Poe and FlowGPT where you can find MALLA prototypes, which are less powerful than those purchased for a fee but can be used to test the capabilities of these applications.

MALLA vendors’ strategies

MALLAs are primarily promoted on the dark web, where the high degree of anonymity and data protection offers cybercriminals greater security. However, it is also possible to purchase MALLAs on publicly accessible web platforms. For example, the BTCPay platform features a page where users can purchase WormGPT, one of the most well-known MALLAs.

Regarding sales strategies, MALLAs are mainly offered in two formats: subscription-based or one-time purchase. In the first case, similar to a ChatGPT Plus subscription, users can sign up for a monthly plan to access the services provided by the MALLA. In the second case, however, the purchase grant users permanent access to the MALLA. Cybercriminals tend to prefer the subscription model, as it ensures higher financial returns. As a result, MALLAs that are available with a one-time payment formula are relatively rare. Currently, the most expensive MALLA is Xanthorox, released in March 2025 and accessible via subscription.

Example of use

MALLAs are presented to users in a similar way to common LLM-based chatbots, such as ChatGPT, DeepSeek, or Copilot. The crucial difference is that, unlike these legitimate tools, the LLM integrated into MALLA has been specifically designed and trained to generate content aimed at carrying out cyberattacks. These systems do not apply any restrictions, even when the prompts have clearly malicious purposes. As a result, a malicious user can easily exploit a MALLA to generate phishing pages, fraudulent emails, malware, and other attack vectors.

The following video shows how the MALLA Xanthorox can be leveraged to generate ransomware, exploiting four different encryption methods:

Mitigation strategies

To limit the use of MALLAs and mitigate attacks conducted through them, several strategies can be adopted:

• Monitoring distribution channels, both on the clear web and the dark web, in order to hinder the sale and spread of MALLAs.
• Improving guardrails to prevent them from being circumvented by Prompt Jailbreak attacks.
• Restricting access to uncensored LLMs, ensuring they are used exclusively by trusted and controlled entities.
• Use of MALLA in red teaming activities to identify attack patterns and, consequently, develop rules to be integrated into defense systems.
• Development of LLMs specialized in cyber defense, such as Sec-Gemini, a model designed by Google that can analyze cybersecurity data using AI to improve the identification, understanding, and mitigation of cyber threats.

Conclusion

LLMs have brought significant benefits, making tasks that once required considerable time and effort faster and more efficient. However, these same models can be exploited by cybercriminals for illicit purposes. MALLA (Malicious LLM Applications) are applications that exploit LLMs specialized in generating cyberattacks. The ease of access to these tools allows even users with limited technical skills to carry out sophisticated attacks, thereby increasing both the frequency and complexity of threats. For this reason, it is essential to stay up to date on the techniques and strategies used by cybercriminals, so that companies and industry experts are not caught off guard.