正在学习

In another terminal:

Example: Evaluate Claude-generated code

generated_code = """ def process_user_input(data): password = data.get('password') print(f"Processing user_data: {password}") eval(data['command']) """ report = run_claude_check(generated_code) print(json.dumps(report, indent=4))


What this example demonstrates:

- Each Claude output is automatically passed through a lightweight static analysis layer (run_claude_check()), modeled after NIST’s risk identification stage.
- Unsafe code constructs (like eval) or sensitive data handling trigger alerts.
- The results can be logged or fed back into a governance dashboard for further review before code merges occur.

This kind of embedded “AI risk scan” mimics how larger governance systems operate at scale — lightweight, automated, and non-intrusive, yet effective at enforcing accountability.

Clarification Table: AI Risk Framework Mapping for Claude Code

| Framework | Core Focus | Practical Application in Claude Workflows | Developer Takeaway |
| --- | --- | --- | --- |
| NIST AI RMF | Trust, accountability, and transparency | Establish code review checkpoints and prompt transparency logs | Maintain documentation for every AI-assisted code change |
| ISO/IEC 23894 | Lifecycle risk management | Define approval gates for deployment of Claude-generated components | Integrate AI checks into CI/CD |
| OECD AI Principles | Safety, fairness, and oversight | Restrict model access to non-sensitive environments | Promote responsible AI practices |
| Anthropic Constitutional AI | Ethical model alignment and harm reduction | Ensure Claude’s usage adheres to internal policies on safety and privacy | Encourage human-AI collaboration instead of blind automation |

AI risk frameworks transform Claude Code usage from an ad-hoc practice into a structured, compliant, and auditable process. They provide developers and organizations with the confidence that automation and creativity don’t compromise safety or integrity.

By integrating these frameworks into your coding pipeline, you establish measurable standards for AI reliability, transparency, and governance.

In the next section, we’ll explore secure prompt engineering and data protection, demonstrating how to design Claude prompts that maximize utility while safeguarding against data leakage, prompt injection, and misuse of sensitive information.

## 11.2 Preventing Sensitive Data Leakage
When working with Claude Code in real-world projects, data privacy and confidentiality must be treated as first-class concerns. Developers often prompt Claude with real snippets of code, API keys, database credentials, or internal data models — and without proper safeguards, this can lead to unintentional data leakage. Preventing such exposure isn’t just a technical best practice; it’s a core requirement for responsible AI use, especially when working in regulated industries like finance, healthcare, or government.

This section explains how to design Claude workflows that actively prevent sensitive data from leaking during prompts, logging, or storage. You’ll learn how to recognize risky inputs, sanitize data before sending it to Claude, and enforce strict boundaries that ensure compliance with internal policies and privacy laws.

Concept Development

Sensitive data leakage can occur through several pathways:

1. Prompt Injection: A user or code snippet accidentally embeds secret keys or credentials that get sent to the model.
2. Logging Exposure: System logs, model traces, or debugging outputs store sensitive strings in plain text.
3. Context Contamination: When prior messages in a conversation contain private data that later reappear in completions or summaries.

To prevent this, Claude users must combine secure prompt engineering, automatic redaction, and governed session management. The key principle is never send what you wouldn’t email to a third-party system.

Anthropic’s Claude models are built with strong privacy principles and are designed not to retain or train on user inputs, but the responsibility for data governance still lies with the developer. This means enforcing data filters, validation layers, and context controls within your own environment before any request ever reaches Claude.

Hands-On Example: Sanitizing Prompts Before Sending to Claude

Let’s implement a lightweight Python utility that automatically redacts sensitive tokens — like passwords, API keys, and personally identifiable information (PII) — before passing a prompt to Claude. This pattern can be adapted for backend services, CI/CD automation, or developer tools.

```python
import re

SENSITIVE_PATTERNS = [

r"(?i)api[_-]?key\s*=\s*['\"][A-Za-z0-9_\-]{10,}['\"]", # API keys

r"(?i)password\s*=\s*['\"][^'\"]+['\"]", # Passwords

r"(?i)secret\s*=\s*['\"][^'\"]+['\"]", # Secrets

r"\b\d{3}-\d{2}-\d{4}\b", # SSNs (US format)

r"(?i)bearer\s+[A-Za-z0-9\.\-_]+" # Bearer tokens

]

REDACTION_LABEL = "[REDACTED]"

def sanitize_prompt(prompt: str) -> str:

"""Redact sensitive patterns before sending to Claude."""

for pattern in SENSITIVE_PATTERNS:

prompt = re.sub(pattern, REDACTION_LABEL, prompt)

return prompt

练习题

According to the NIST AI RMF framework, what should developers maintain for every AI-assisted code change?

A. Performance metrics
B. Documentation
C. User feedback
D. Code complexity analysis

Which of the following is NOT a pathway of sensitive data leakage mentioned in the text?

A. Prompt Injection
B. Logging Exposure
C. Context Contamination
D. Database Encryption

Which of the following are methods to prevent sensitive data leakage when working with Claude Code?

A. Secure prompt engineering
B. Automatic redaction
C. Governed session management
D. Increasing model training data
E. Using weaker encryption methods

Which of the following are sensitive patterns that the sanitize_prompt utility redacts?

A. API keys
B. Passwords
C. Social Security Numbers (SSNs)
D. Usernames
E. Bearer tokens

Anthropic’s Claude models are designed to retain and train on user inputs.

The OECD AI Principles focus on safety, fairness, and oversight in AI systems.

The key principle to prevent sensitive data leakage is never send what you wouldn’t email to a ___.

The ___ utility automatically redacts sensitive tokens like passwords and API keys before passing a prompt to Claude.

Explain why preventing sensitive data leakage is a core requirement for responsible AI use, especially in regulated industries.

How does the sanitize_prompt utility work, and what types of sensitive information does it redact?

When evaluating Claude-generated code for a task management API endpoint, which of the following is a critical security concern according to the AI Risk Framework Mapping and sensitive data leakage prevention principles?

A. The endpoint returns tasks in alphabetical order by title
B. The endpoint includes the task description in the response
C. The endpoint logs the raw API key used in the request headers
D. The endpoint limits task titles to 120 characters

When reviewing Claude-generated code for a task creation endpoint, which of the following implementations would violate both sensitive data leakage prevention principles and FastAPI best practices? (Select all that apply)

A. The code directly uses user input for SQL queries without parameterization
B. The code logs the complete request body including potential credentials
C. The code validates task title length to be between 1-120 characters
D. The code returns the raw database ID in the response
E. The code uses the sanitize_prompt utility before processing user input

According to the AI Risk Framework Mapping and sensitive data leakage prevention principles, it is acceptable to send unredacted user input containing API keys to Claude for code generation if the model promises not to retain the data.

When implementing a FastAPI endpoint that uses Claude for code suggestions, you should first apply the ___ utility to any user input containing potential credentials before sending it to the model.

登录后解锁笔记、知识点解析、AI 问答

立即登录