Slop Tracker

Incident Card: Samsung / ChatGPT (April 2023)

What happened. In April 2023, engineers at Samsung Semiconductor used ChatGPT for coding assistance. Three separate incidents occurred within 20 days: source code related to chip equipment was pasted for debugging; a second code segment from an internal database tool was shared; and notes from a confidential internal meeting were submitted for summarization. All three transmitted proprietary information to OpenAI's servers. Samsung had no AI usage policy, no output-gate, and no logging of what its engineers were sharing.

Who caught it. Samsung's internal security team, via retroactive audit. Not a real-time control — a post-incident review after someone reported the first incident internally. By the time the review ran, two more incidents had already occurred.

What happened next. Samsung banned ChatGPT for all employees in early May 2023 and began developing an internal LLM. The company issued a memo warning that data shared with external AI services could be stored and used in model training. The ban remains one of the most-cited examples of corporate AI data exposure.

What governance primitive would have prevented it. An output-gate classifying text against a data-classification schema before transmission. At minimum: a delegation context with scope: ["public-repos-only"] preventing the AI assistant from accepting clipboard content from restricted directories. A kill-switch policy at HALT_NONCRITICAL would have stopped AI tool usage without disabling production systems while the policy was being defined. The Samsung incident is not a model problem. It is a scope-enforcement problem. The model did exactly what it was asked to do.

Sources: Bloomberg (May 2, 2023); Gizmodo (May 2, 2023); The Economist, “The risks of outsourcing intelligence” (2023).

By the Numbers

• 42% of committed code is now AI-generated — up from near-zero in 2021. The security pass rate for AI-written code has been stuck at approximately 55% for two years despite rapid model capability gains. More code, same defect rate, higher volume. (Sonar, “AI Code Quality Report 2026,” n=1,100+)
• 2.74× more vulnerabilities in AI-assisted code versus human-only code in matched production samples. OWASP Top 10 representation is proportionally unchanged — the same vulnerability classes appear more frequently, not new ones. (Veracode, “State of Software Security 2025”)
• 79.3% SWE-bench Verified score for Claude Opus 4.6: the headline capability number. The fine print: 29.2% of completions from leading models contain at least one vulnerability in independent red-team testing. Benchmark performance and security posture are diverging on the same models. (SWE-bench leaderboard, 2026; Stanford HAI AI Index 2026)
• 19% slower: a randomized controlled trial of 16 professional developers on a production codebase found AI coding tools reduced task completion rate by 19%. Participants self-reported a 20% speedup estimate. The gap between perception and measurement is 39 percentage points. (METR, arxiv:2507.09089, 2026)
• 21% of enterprises have a mature AI governance program. 79% of enterprises shipping AI-generated code to production are doing so without the controls that EU AI Act, Colorado SB 24-205, and pending US federal AI guidance will require. (Deloitte, “State of Generative AI in the Enterprise 2026”)

Regulatory Watch

EU AI Act — expected Dec 2, 2027. Annex III high-risk system obligations go live in approximately 110 days. The three provisions that land hardest on teams shipping agentic code: Article 9 (risk management system documentation), Article 12 (automatic logging sufficient to enable post-market monitoring), and Article 14 (human oversight measures enabling intervention). Finland began enforcement of general-purpose AI model provisions in January 2026. Annex III is the larger compliance exposure for enterprise deployments. Organizations using AI in hiring, credit, healthcare, or critical infrastructure need Article 9-compliant documentation before August.

Moffatt v. Air Canada (2024). The British Columbia Civil Resolution Tribunal held Air Canada liable for its chatbot’s hallucinated refund policy. The principle: deploying an AI system in a customer-facing role creates a duty of care for its outputs, regardless of whether the organization intended the AI to make binding representations. The tribunal explicitly rejected Air Canada’s disclaimer defense — a legal posture that is now off the table for organizations relying on fine-print carve-outs. The governance lesson: scope the agent’s authority, log what it said, and make the scope auditable. (2024 BCCRT 149)

Mobley v. Workday (cert. granted July 2025). The Ninth Circuit cert grant establishes whether AI-driven hiring screening tools are subject to employment discrimination liability under Title VII and the ADA — and specifically whether Workday as a third-party vendor can be held liable as an “agent” of the employer. A ruling is expected Q4 2026. This is the supply-chain liability case. Every organization deploying third-party AI in HR workflows is watching. (No. 23-15992, 9th Cir.)

Berkley Insurance AI Exclusions (2025). W.R. Berkley has written explicit AI-generated code exclusions into D&O and E&O specialty policies. Coverage for AI-code-related claims is contingent on documented governance controls: human review of AI-generated code before production deployment, logging of AI tool usage in the SDLC, and a stated organizational AI policy. No governance documentation = exclusion applies. This is the first major specialty insurer to operationalize AI governance as a coverage condition rather than an advisory.

The Wedge

Every incumbent in the AI governance vendor landscape — Holistic AI, Credo AI, Arthur, Fiddler — operates in the evaluation plane: dashboards, model cards, pre-deployment checklists. None of them instrument the runtime.

Here is what that means in practice: when your delegation chain executes a sub-agent call at 2:47am in a CI pipeline, none of these tools know. The governance record they produce is built from what you told them before deployment, not from what actually ran.

This is not a gap that incumbents will easily close. Their architecture assumes a human is in the loop at model-evaluation time. In agentic systems, the relevant decision is made when the token is generated — not when the ticket was filed. Runtime governance requires a different primitive: an append-only audit log written at execution time, not a dashboard populated by periodic API calls.

The Sonar and Veracode numbers are not a model quality problem. The models are improving. The governance tooling is not keeping up. That is the market.

Quote of the Month