AIF-C01 ยท Securing and governing AI solutions
Security and privacy threats to AI systems
Prompt injection, data leakage, toxicity, encryption at rest and in transit, output filtering and validation, and audit logging of AI interactions.
Key points
- Prompt injection: tricks a model into ignoring its instructions, through user input or retrieved content.
- Data leakage: outputs expose training data, system prompts or other users' information.
- Toxic output: harms users and brands.
- Layered defense: encryption at rest and in transit, input and output filtering and validation (for example Bedrock Guardrails), least-privilege access, audit logs of AI interactions.
Source: https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html
Practice questions
What is prompt injection?
- Crafted input that makes a model ignore or override its intended instructions
- Compressing a prompt to reduce token costs
- Adding examples to a prompt to improve accuracy
- Encrypting the prompt before it is sent
Show answer
Answer: A. Crafted input that makes a model ignore or override its intended instructions
Prompt injection manipulates the model through its input. Compression, few-shot examples and encryption are legitimate techniques, not attacks.
Which controls help prevent leaking sensitive data through a generative AI app?
- Masking PII in model responses with sensitive information filters
- Giving the retrieval layer access only to documents the user may see
- Encrypting traffic to the model endpoint in transit
- Raising the model temperature
- Turning off all logging to hide the data
Show answer
Answer: A. Masking PII in model responses with sensitive information filters, B. Giving the retrieval layer access only to documents the user may see, C. Encrypting traffic to the model endpoint in transit
Output masking, scoped retrieval access and encryption in transit each close a leak path. Temperature has no security effect and removing logs only removes visibility.
True or false: encrypting data at rest is enough to stop prompt injection attacks.
Show answer
Answer: False
False. Encryption protects stored data from unauthorized reading. Prompt injection arrives through legitimate input, so it needs input and output filtering and least privilege.
Practice 8 questions on this topic in Bitzu
Short lessons, instant explanations and spaced review of what you get wrong. Free on iOS and Android.
