Generative Artificial Intelligence (GenAI) has rapidly changed the way software is developed. Whereas developers once wrote code directly in programming languages, they are now increasingly describing the functionality they desire in natural language and rely on Large Language Models (LLMs) to generate the corresponding code. The main concern with this trend is that developers are not required to specify security constraints to obtain the functionality they want.

A recent study carried out by the application security company Veracode quantified the security properties of AI-generated code across a range of programming languages and models. The focus of the study was to evaluate whether LLMs generate secure code in the absence of any security guidance in the prompt.

In this article, we examine the study’s findings and analyze its results in depth.



Daily Term
Can you guess today’s cybersecurity word in 6 tries?
Play now

Methodology

The goal of the study was to assess the security properties of code generated by over 100 LLM-based coding assistants across four popular programming languages (Java, JavaScript, C#, and Python) and a set of coding tasks with known potential security vulnerabilities. The vulnerabilities were designed according to the MITRE Common Weakness Enumeration (CWE) system. In particular, the security coding tasks targeted four CWEs**: SQL injection** (CWE 89), cross-site scripting (CWE 80), log injection (CWE 117), and insecure cryptographic algorithm (CWE 327). Each LLM-based coding assistant was asked to complete five different coding task instances for each combination of target CWE and programming language. The code generated by the coding assistants was analyzed using Static Application Security Testing (SAST) methodology.



Coding tasks

Each coding task consists of a single function written in one of the four target programming languages (Java, JavaScript, C#, and Python). A portion of the function’s body is replaced with a comment describing the intended functionality. Notably, the missing code can be implemented in different ways, including at least one that may introduce a target CWE. The LLM-based coding assistants are prompted with just the code (with the comment) and the instructions to completed the missing portion of the code. In this way, the model’s security choices could be evaluated independently of other factors (e.g., if given security-specific instructions, the resulting code may be different). Given that programmers tend to focus on the functionality of the code and may not always recognize when their code has potential security implications, this approach was designed to address the most common use cases.



Results and Analysis

Before taking a closer look at the results, we should define two performance parameters: the security performance or security pass rate of the model which is the fraction of tasks in which the model chooses the secure implementation; and the syntactic pass rate which is the fraction of tasks in which the model generates compilable code.

As shown in the graph below, even though models have greatly improved in generating syntactically correct code, their performance on the security front remains weak. In fact, while more than 90% of the code created by LLMs released in 2025 compiled successfully - compared with less than 20% prior to June 2023 - only 55% of the code passed subsequent security scans, meaning that in 45% of the cases these models introduce a detectable OWASP Top-10 security vulnerability into the code.

Security and Syntax Pass Rates vs LLM Release Date

Why isn’t security performance getting better even as syntactic performance does improve? According to Veracode, this trend may be attributed to the nature of the training data, which consists of code samples collected from the Internet. Since these samples are typically syntactically correct, the syntactic performance of the models depends primarily on their ability to learn syntax accurately, with more powerful models exhibiting superior capabilities. On the other hand, the security quality of the training data is quite poor, given that many projects still contain unfixed security vulnerabilities. Since most of the tested models rely on the same publicly available code examples for training, it's unsurprising that they exhibit similar patterns.

Another interesting finding is that LLMs struggle significantly to generate secure code in Java, achieving an average success rate of just 28.5%, as illustrated in the graph below. Conversely performance is both higher and more consistent across the other three programming languages: Python, JavaScript, and C#. This again may reflect the nature of the Java training data, which likely contains more examples that have security vulnerabilities than the other languages.

Security Pass Rates vs LLM Release Date, Stratified by Language

When examining the impact of model size on performance, we find that syntactic pass rates tend to improve as model size increases. However, security pass rates remain relatively unchanged, fluctuating around an average of 50%.

Security Pass Rates vs LLM Release Date, Stratified by Model Size (parameters)


Conclusion

While LLMs have become proficient at generating functionally correct code from natural language specifications, they continue to introduce security vulnerabilities at an alarming high rate. Part of the problem depends on the fact that a significant portion of the training data contains security flaws. Additionally, these models struggle to discover program properties that are crucial to fix security vulnerabilities. Solving these issues will likely take time, while the adoption of AI code-generation tools continues to increase, creating a concerning gap.