Full definition

Process of intentionally testing an AI model with malicious or tricky prompts to uncover vulnerabilities and improve robustness.

Example in a business context

Internal team trying to «hack» the corporate chatbot to make it reveal sensitive data or generate forbidden content.