A Proposal for Evaluating the Operational Risk for Chatbots Based on Large Language Models

Researchers proposed a new risk metric for evaluating security threats in Large Language Model (LLM) chatbots, considering system, user, and third‑party risks. An empirical study using three chatbot models found that while prompt protection helps, it's not enough to prevent high‑impact threats like misinformation and scams. Risk levels varied across industries and user age groups, highlighting the need for context‑aware evaluation. The study contributes a structured risk assessment methodology to the field of AI security, offering a practical tool for improving LLM‑powered chatbot safety and informing future research and regulatory frameworks.