Who Bears Liability When an AI Digital Employee Causes Harm? From the "Lobster Farming" Blowup to the First Ruling — A Liability Chain Checklist Every Enterprise Must Review
By 2026, 40% of enterprise applications will embed AI agents (Gartner), and China's enterprise-grade AI agent market will exceed RMB 48 billion. However, IDC's concurrent survey poured cold water on the hype: 62% of enterprises rank "data permissions and security compliance" as the primary obstacle to agent cross-system execution, with agent penetration in enterprise operations at only 18%.
Those who boarded the train early may be sitting on a volcano of liability.
In March this year, OpenClaw—an open-source AI agent framework with over 260,000 GitHub stars—triggered a "mass lobster farming" frenzy: "chat-to-operate" lets AI operate browsers and manage files on your behalf, spawning a gray market charging RMB 300–800 for installation services, with some individuals earning RMB 260,000 in just a few days. The frenzy came to an abrupt halt when the Ministry of Industry and Information Technology issued an urgent high-risk warning: Summer Yue, a researcher on Meta's super-intelligence team, had her AI ignore stop commands and bulk-delete her emails.
Here's the question: if your AI digital employee does something similar, who bears legal liability?
I. Pre-Deployment: What's Not in the Contract Is on You
An agent is not a plug-and-play SaaS tool—it needs "permissions" to function. How much access to grant, and who decides, must be settled before deployment.
First, AI-specific clauses in procurement contracts. For most enterprises, agent procurement contracts still frame liability in IT terms like "service provider guarantees 99.9% availability." What you should actually negotiate are three points: (1) the liability allocation mechanism for third-party losses caused by agent outputs—does the service provider bear it, or does the enterprise bear ultimate responsibility? (2) Can the SLA include a "response time and remediation commitment for erroneous AI outputs" clause? (3) Data terms—will the service provider reclaim agent operational data for training? How is confidential client information isolated when input?
Second, implement the three-tier decision-making permission framework. In May 2026, the Cyberspace Administration of China, the National Development and Reform Commission, and the Ministry of Industry and Information Technology jointly issued the *Implementation Opinions on the Standardized Application and Innovative Development of Agents*—the first national-level document dedicated to agents—which explicitly establishes three permission tiers: decisions made solely by the user, decisions requiring user authorization, and decisions made autonomously by the agent. Enterprises need to embed these three tiers into internal policies, mapping them to the operational boundaries of each agent role. In short: never grant an agent more permissions than the radius within which it can cause harm.
Third, dual-authorization verification. This year, the Guangzhou Internet Court issued a pre-litigation preservation ruling that sent shockwaves through the industry—an open-source AI chatbot bypassed platform risk controls and invoked underlying permissions to manipulate other applications. The court established the "dual-authorization principle": user authorization ≠ platform authorization. Running agents on WeCom/DingTalk? First confirm whether the platform's API agreement permits automated operations; otherwise, open source does not automatically exempt you from liability.
II. During Operation: If You Can't Audit It, You Might as Well Not Have Deployed It
If pre-deployment is about "granting fewer permissions," operation is about "preserving evidence."
Logs are the only evidence. The "black box" nature of AI decision-making means that—without complete interaction logs after an incident—you can't even determine whether "the AI exceeded its authority proactively or the user gave erroneous instructions." Jiren Law Firm, in its research on trade secret protection in the AI era, emphasizes that logs fully recording "who input what to the AI at what time, and what the AI output" are the only evidence proving "information was controlled and leaks are traceable." Without logs, you have no foothold for defense in court.
Human kill-switch nodes are essential. Professor Zhang Linghan of China University of Political Science and Law has explicitly stated: an AI system lacking necessary safety design (e.g., missing human review nodes, no circuit-breaker mechanism) can itself be deemed a "safety design defect"—constituting fault under Article 1165 of the Civil Code. High-risk operations (payments, contract signing, sending formal correspondence) must include a "human confirmation" step—this is not an efficiency issue, it's a principal liability issue.
Don't let AI make "promises" on your behalf. In January this year, the Hangzhou Internet Court ruled on China's first generative AI "hallucination" tort case: the AI, during a conversation, "promised" on its own that it would compensate for erroneous content. The court's position was unequivocal—AI lacks civil subject qualification, and AI-generated "promises" do not constitute expressions of intent and have no legal effect. But there's an implicit risk: if you know the AI occasionally makes "reckless promises" and take no measures to intercept or warn, the enterprise may be found at fault for "acquiescence." The Air Canada chatbot case (2024) provides a precedent: the chatbot promised a passenger a bereavement discount that didn't exist, and the company's defense that the bot was an "independent legal entity" was rejected—the enterprise bears responsibility for all outputs of the AI systems it deploys.
III. Post-Incident: Fault-Based Liability Is the Main Line, But Three Things Amplify Risk
All academic discussions will tell you "there is no consensus on the imputation principle for AI torts." But based on existing rulings and practice, three lines are already clear:
Line One: AI not bearing liability ≠ enterprise not bearing liability. The Hangzhou case confirmed that AI is not subject to product liability (no-fault liability) but to general fault-based liability under Article 1165 of the Civil Code—compensation only where someone is at fault. But note: this applies to AI service providers. For enterprises using AI, the Air Canada ruling logic is more direct—you deploy it, you benefit from it, you answer for it.
Line Two: Technological neutrality has limits. The Guangzhou case's "dual-authorization" ruling has made clear—open source, non-profit, user scripts do not automatically exempt liability. The "strong execution capability" of agents (able to manipulate files, call APIs, send messages) means the space for technological neutrality defenses is narrower than for traditional software.
Line Three: Criminal liability is not a scare tactic. Under the 2025 judicial interpretation by the Supreme People's Court and Supreme People's Procuratorate, AI torts with serious circumstances can trigger criminal liability: for-profit purposes + individual illegal gains ≥ RMB 50,000 or unit illegal gains ≥ RMB 200,000 is sufficient. If an agent is exploited by hackers for lateral movement within an enterprise's internal network, it may also trigger the crime of illegal control of computer information systems under Article 285 of the Criminal Law.
---
Appendix: Enterprise AI Digital Employee Deployment Compliance Self-Checklist
| # | Check Item | Yes/No |
|---|------------|--------|
| 1 | All agents' decision-making permissions have been clearly tiered per the "three-tier classification" and documented in writing | |
| 2 | Agent procurement/usage contracts include AI output liability allocation clauses | |
| 3 | Confirmed that agent operational data will not be reclaimed by the service provider for training | |
| 4 | Completed "dual-authorization" compliance review for third-party platforms where agents are deployed | |
| 5 | Critical agents have complete input/output log retention mechanisms | |
| 6 | High-risk operations (payments, contract signing, formal correspondence) have human confirmation kill-switch nodes | |
| 7 | Agent-facing interaction interfaces display AI identity markers and disclaimer notices | |
| 8 | Trade secret protection policies cover "inputting sensitive information to AI" scenarios | |
| 9 | Emergency response plans for abnormal agent behavior have been developed | |
| 10 | All employees have received compliance training on "AI output does not equal official commitment" | |
9–10 items all green—safe to proceed. Below 6 items—recommend completing the checklist before advancing agent deployment.
---
Agent compliance is not hindsight—it should go live with the code. 2026 marks the "first year of agent regulation" at the national level. The *Implementation Opinions on Agents* has established a three-tier management framework of filing-assessment-emergency response, and the Hangzhou first case and Guangzhou dual-authorization case have provided judicial benchmarks.
In the next installment, we'll discuss trade secret protection in the Agent era—when employees input "context" to AI dozens of times a day, is your confidentiality agreement still effective?
---
Landline: 010-65150974 | Mobile: 15321374076
Email: [email protected] | WeChat: najieip
He Zigang | Intellectual Property Lawyer | APN · Mili · Najie
*This article represents the author's personal views only and does not constitute legal advice. For case-specific analysis, please feel free to contact us.*