AI Coding Assistants: Productivity & Quality Research

AI Coding Assistants: What the Research Says About Productivity and Code Quality (2026)

The research on AI coding assistants tells a more complicated story than the marketing: they help most people write more code faster, but controlled studies show they can slow experienced developers down (METR), and large-scale data shows AI-generated code carries more security vulnerabilities (Veracode) and more maintainability debt than human-written code (GitClear). The consistent conclusion across every serious 2025 to 2026 study is the same: AI is a fast junior that needs a human reviewer, so the teams that win treat AI output as untrusted input to inspect, not finished work to ship. This guide summarizes the actual findings on productivity, code quality, and security, and what they mean for how you build.

Quick answer: The evidence is mixed and specific. On productivity, most developers self-report large gains, but a randomized controlled trial found AI slowed experienced open-source developers by 19% even though they expected a 24% speedup. On quality, one analysis of 45% of AI-generated code samples introduced a security vulnerability, and multi-year data shows refactoring and code reuse collapsing while duplication rises. The takeaway is not "AI bad," it is "review it": the biggest gains go to teams that keep humans on architecture, security, and review.

Productivity: the gap between what developers feel and what studies measure

Adoption is nearly universal. Stack Overflow's 2024 Developer Survey found 63% of professional developers already use AI in their process, with another 14% planning to (GitClear). And AI unquestionably makes more code get written, faster.

But "more code, faster" is not the same as "more productive," and the most rigorous study to date complicates the story. In a randomized controlled trial run at the early-2025 AI frontier, 16 experienced open-source developers completed 246 real tasks on mature projects they averaged five years on. They forecast AI would cut their completion time by 24%, and even after finishing estimated a 20% speedup. The measured result: "allowing AI actually increases completion time by 19%, AI tooling slowed developers down" (METR RCT). That slowdown also contradicted expert forecasts (economists predicted 39% shorter, ML experts 38% shorter).

The reconciling data comes from GitClear: heavy AI users out-produce non-users by 4 to 10x, but "most of that gap pre-dated AI," and compared to their own past selves heavy AI users saw "a more modest 25% velocity gain" (GitClear). So the honest read is: AI helps a lot on greenfield, boilerplate, and unfamiliar tasks, and can hurt on mature codebases where an expert already knows the terrain and now spends time reviewing and correcting AI output.

Code quality: the maintainability signals are sliding

This is where the largest datasets are most consistent, and most concerning. GitClear analyzed 211 million changed lines (2020 to 2024) and later 623 million changes (2023 to 2026), tracking the structural habits that keep code maintainable (GitClear). The trend lines all point the same way:

• Refactoring is collapsing. The share of "moved" (refactored) lines fell from 25% in 2021 to under 10% in 2024, and refactoring line moves are "down 70% vs 2022 levels" (GitClear).
• Duplication is climbing. Copy/paste rose from 8.3% to 12.3% of changed lines, and 2024 was "the first year on record where within-commit copy/paste exceeded moved (refactored) code"; block duplication is "up 81% since 2023" (GitClear).
• Code reuse is falling. Cross-file function calls, a signal of reuse, are "down 35%," and long-term legacy maintenance is "down 74% vs 2022," meaning old code is left to calcify while new v1 features pile up (GitClear).

The summary from the research: "the throughput is real, but so is the debt it accrues," and the bill "arrives when you can least afford it" (GitClear).

Security: AI writes working code that is not safe by default

The security picture is the most quantified, thanks to Veracode's 2025 benchmark across 100+ large language models in Java, JavaScript, C#, and Python. The headline: "in 45% of the tasks the model introduces a known security flaw," so only 55% of AI-generated code was secure (Veracode). More findings that matter:

• Bigger and newer models did not fix it. "Security performance remained flat, regardless of model size or training sophistication" (Veracode).
• Some languages are far riskier. Java had a 72% security failure rate, and models "failed to defend against cross-site scripting in 86% of relevant code samples" (Veracode).
• Reasoning models help, but not enough. By October 2025, OpenAI's GPT-5 reasoning models reached a 72% security pass rate, the highest recorded, "but it still means more than one in four coding tasks introduce a known vulnerability," while most other providers stayed in the 50 to 59% range (Veracode).

A large-scale study of AI code "in the wild" adds the ecosystem view: AI concentrates in "glue code, tests, refactoring, documentation, and other boilerplate, while core logic and security-critical configurations remain mostly human-written," and in human-AI edit chains "humans act as security gatekeepers; when review is shallow, AI-introduced defects persist longer" and are disproportionately exposed to remote attack (86.8% of AI-introduced vulnerabilities sat on network attack surfaces) (arXiv).

The research at a glance

Every figure below is from a named study.

What the research says to actually do

The studies converge on a practical playbook, not a verdict against AI:

• Treat AI code as untrusted input. "Validate it. Scan it. Review it. Just as you would any other code contribution" (Veracode).
• Keep humans on the parts AI is worst at. Core logic, security-critical config, and architecture stay human; let AI handle boilerplate, tests, and glue (arXiv).
• Prefer reasoning models for code, since they "function like an internal code review" and score meaningfully higher on security (Veracode).
• Budget for refactoring and put a tripwire on duplicate blocks, because the maintainability debt is measurable and manageable if you watch for it (GitClear).
• Own and inspect the output. The whole review-and-harden playbook only works if you can actually see and keep the code, which is an argument for tools that hand you real, exportable code rather than an opaque black box.

The harness is the point, not the raw model

The most useful signal in all of this research is buried in the security data: the thing that actually improved code quality was not a bigger model, it was a better process around the model. Veracode found that reasoning models scored higher precisely because the reasoning step "functions like an internal code review," evaluating options and filtering out insecure patterns before committing, while a standard model "generates code in a single forward pass, selecting implementations based on probabilistic patterns in training data, which often includes insecure code" (Veracode). Model size barely moved the numbers; the review-like process did.

That is the whole game. The quality of AI-generated code is set less by which base model you use and more by the harness around it: the context the tool feeds the model (the framework, the conventions, the constraints), the guardrails and review loops it runs, and the domain-specific rules its engineers encode. A raw prompt to a general model has none of that context and produces the 45%-vulnerable, duplication-heavy output the studies measured. A purpose-built harness supplies the missing context and a review pass, which is exactly what the research says raises quality. So the honest conclusion is: if the harness is built by top engineers around a top model, tuned to one domain, the output can land far above what the same model produces from a cold prompt.

Superapp is a concrete example of this for native iOS. Its harness is purpose-built for the Apple ecosystem by a senior engineering team with backgrounds at companies like Bolt, Superhuman, Grammarly, MacPaw, and Wix, including a top-ranked white-hat security engineer, and it is tuned to real native Swift and SwiftUI, Apple's Human Interface Guidelines, and App Store requirements (Superapp, Silicon Gardens). Because the harness carries that iOS context and review discipline, the generated Swift aims for the level a senior iOS engineer would ship rather than the generic, unreviewed output the research warns about, which is also why a testing team could open its export in Xcode and have it "compile without any syntax errors or missing dependencies" (AI Founder Kit). This does not exempt the output from review, the research's rules still apply, but it is the difference between a cold prompt and a domain-tuned harness, and that difference is where code quality actually lives.

The trade-off for iOS and Swift builders

This research is exactly what makes the "AI tools versus traditional Swift development" question answerable rather than ideological. The trade-off is not speed versus quality in the abstract; it is: does the AI hand you code you can review, own, and ship through Apple's review, or an opaque output you cannot inspect and Apple may reject.

For iOS specifically, that points to two things the research implies. First, prefer output you can review and keep, because every study says AI code needs human review, which is impossible if the tool locks the code away. An AI-native builder like Superapp is aligned with this: it generates real native Swift and hands you a standard Xcode project you own and can export, so a developer can inspect and harden it, or continue it in Claude Code or Codex, exactly the review step the research demands. Second, native output clears App Store review far more easily than web wrappers, which sidesteps a whole category of rejection risk that AI-generated web-wrapper apps run into. None of this exempts AI-generated Swift from the findings above, it still needs review like any AI code, which is why serious builders pair generation with a human pass. The point is that "AI versus traditional Swift" is a false binary: the durable approach is AI generation on a foundation you can review, own, and ship, with a human owning architecture and security.

Frequently asked questions

Do AI coding assistants actually make developers more productive?

It depends who and what. Most developers self-report large gains and more code does get written, but a 2025 randomized controlled trial found AI slowed experienced developers on mature codebases by 19%, even though they expected a 24% speedup (METR). GitClear data suggests a real but more modest roughly 25% velocity gain for heavy users versus their own past output (GitClear). AI helps most on greenfield and boilerplate, and least where an expert already knows the code.

Is AI-generated code less secure than human-written code?

The data says yes by default. Veracode's 2025 benchmark of 100+ models found 45% of AI-generated code samples introduced a known security vulnerability, and that security performance stayed flat across model sizes and over time (Veracode). Even the best model tested (GPT-5, October 2025) still failed on 28% of tasks (Veracode). AI code is functional but not safe by default, so it must be scanned and reviewed.

Does using a bigger or newer AI model fix the code quality problem?

No, not for security. Veracode found "larger, newer AI models didn't improve security," with performance clustering in the 50 to 59% range across most providers. The exception is reasoning models, which review their own output internally and scored higher (GPT-5 reached a 72% pass rate), but even those still introduced a vulnerability in more than one in four tasks (Veracode).

What does the research say about AI and code maintainability?

It shows the structural habits that keep code maintainable are eroding as AI authorship rises. Across hundreds of millions of changes analyzed by GitClear, refactoring line moves are down 70% versus 2022, code duplication blocks are up 81% since 2023, and long-term legacy maintenance is down 74% (GitClear). More code ships faster, but with more duplication and less reuse, which accrues technical debt.

Should you stop using AI coding assistants?

No. The research recommends using them well, not avoiding them: treat AI output as untrusted input to validate and scan, keep humans on core logic, security, and architecture, prefer reasoning models, and budget for refactoring. Used that way, AI is a strong accelerator for boilerplate and prototyping; used unreviewed, it ships vulnerabilities and debt.

What are the trade-offs of AI tools versus traditional Swift development?

The real trade-off is reviewability and shipping, not just speed. AI generation is far faster, but every study says the code needs human review, and for iOS the output also has to pass App Store review. So the durable approach is AI generation on native Swift you can inspect and own, with a human on architecture and security, rather than either hand-writing everything or trusting unreviewed AI output. Tools like Superapp generate real native Swift you export as an Xcode project, which keeps that review step possible.

Where does AI-generated code introduce the most vulnerabilities?

In input handling and unsafe API usage, not complex logic. Veracode found cross-site scripting defenses failed in 86% of samples and log injection was similarly poor, because models cannot reliably tell which variables carry untrusted data (Veracode). A large in-the-wild study found AI-introduced vulnerabilities concentrated in input validation, encoding, and risky APIs, and disproportionately exposed to remote attackers (arXiv).

What kinds of work is AI best and worst at in software development?

Best at high-volume, patterned work: boilerplate, tests, glue code, documentation, and scaffolding, where studies show AI concentrates and adds the most value. Worst at core business logic, security-critical configuration, and long-term maintainability decisions, which is why the consistent finding is that humans should own architecture and review while AI handles the repetitive layer.

How should teams safely adopt AI coding assistants?

Layer controls the way you already secure any code. Use reasoning-enabled models, run static analysis and dependency scanning on AI output, require human code review (especially for auth, payments, and data handling), separate development and production data, and track maintainability signals like duplication and churn. The research is explicit that AI coding assistants are tools, not replacements for a security and review process.

Does the AI tool's harness matter more than the base model?

For code quality, largely yes. Veracode found that model size barely changed security outcomes, while reasoning models improved because their extra step acts like an internal code review. That means the harness around the model, the context it is fed, the guardrails, and the review loops, drives quality more than which base model is used. A domain-tuned harness built by strong engineers can produce far better code for its domain than a cold prompt to a general model. Superapp is an example for native iOS: a harness built by senior engineers (backgrounds at Bolt, Superhuman, Grammarly, MacPaw, Wix) tuned to Swift, SwiftUI, and Apple's guidelines, so the output aims for senior-iOS-engineer quality, though it still needs review like any AI code.

Is vibe coding safe for production apps?

Only with review. Pure vibe coding means shipping code you did not read, which inherits the 45% vulnerability rate (Veracode) and the maintainability debt the research documents (GitClear). For a prototype or internal tool the risk is low; for anything touching money, auth, or personal data, the studies are unanimous that a human must review and harden the code before it ships.

References

• Productivity: METR randomized controlled trial (2025), GitClear: AI code quality 2025, GitClear: the maintainability gap (2026)
• Security: Veracode 2025 GenAI Code Security Report, Veracode October 2025 update (GPT-5 data), Veracode report hub
• In the wild: AI Code in the Wild: security risks and ecosystem shifts (arXiv)
• iOS trade-off context: Superapp, Superapp features