Lessons Learned: Security-Focused AI Chatbots

Organizations face a common challenge – how can they improve their security posture with limited budget and staff?  One approach is leveraging tools which detect and remediate vulnerabilities in technology, but let’s consider other opportunities. People and processes, along with technology, comprise the three “pillars” of information security. Automation can help strengthen the people and process security pillars for example by using an artificial intelligence (AI) tool to actively guides personnel through security-related tasks and activities. With such an approach, an organization can:

• Increase employee awareness of security policies, standards and other security requirements
• Help personnel properly complete tasks such as configuring authentication credentials (changing passwords), reporting a lost laptop, or asking policy questions
• Motivate employee action by proactively sending messages in collaboration tools
• Guide personnel to resources they’re permitted to share with your customers

My role in Oracle’s Corporate Security involves providing internal guidance. While doing this, I noticed that many of the same requests and questions among customer-facing teams were occurring frequently. I got curious about using AI to address these common questions and began experimenting. Here’s what I learned from the automation project to answer common security-related questions using Oracle Digital Assistant (ODA). The project’s objective:

We found that ODA’s integration with various “channels” and collaboration tools enabled automated delivery of the accurate information to customer-facing staff in real time. Personnel didn’t need to wait for responses or hunt for guidance – the AI provided correct answers to their questions.

Benefits of security-focused chatbots

What are Oracle Digital Assistant (ODA) chatbots?

Digital assistants are artificial intelligence (AI) user interface tools (commonly known as chatbots) that help people accomplish tasks through natural language conversations, without having to search and wade through various applications and web sites. Each digital assistant contains a collection of specialized skills.

You can make digital assistants available to targeted users through a variety of applications, such as chat and collaboration tools. Each digital assistant can use one or more task-specific or topic-specific skills such as answering specific questions and initiating a process. As an example, you can try the Sales chatbot on oracle.com:

Use cases for security-focused chatbots  

Chatbots can summarize requirements and provide guidance with links to web pages, instructions, and applications, such as these examples:

• Policy questions such as:
  • Can I use my personal mobile phone for work email? 
  • May I use a password vault?
  • Must I use XYZ tool for collaboration, or may I use another tool instead?
• Process questions such as:
  • How do I report a risk? 
  • How do I replace a lost badge?
  • How do I get approval for special contract terms?

Chatbot-initiated employee engagement

Where to deploy chatbots

Make it easy for your personnel by embedding or integrating chatbot into tools they already use to seek help or complete relevant tasks. Delivering relevant guidance when and where it is needed helps people comply with internal policies. To determine the best location, ask your target audience where they seek help now. Likely candidate locations include:

• Collaboration, messaging and chat tools where your target audience interacts with peers or seeks help
• Applications for executing relevant processes such as replacing a lost laptop or requesting access

No single tool is right for every need

As large language models may “hallucinate”, person-to-person communications are advised when discussions require a high degree of sensitivity, people may have emotional reactions to the message, significant nuanced professional judgment is needed or when there are potential legal or safety consequences.

There are several options for limiting risk by controlling how chatbots respond. For example, you can use “retrieval augmented generation” (RAG) in the AI is configured to solely quote from your knowledge base of documents. You can use a structured ODA dialog flow so that users select from a menu of options, with limited or no natural language processing. A third option is configuring a limited set of chatbot responses using ODA “intents”. As with any technology, it should be monitored to detect any “drift” from intended operation.

Oracle Digital Assistant (ODA) configuration

Intents allow your skill to understand what the user wants it to do. An intent categorizes typical user requests by the tasks and actions that your skill performs.

Chatbot training

Skills can have multiple “intents” – these are the chatbot responses which offer guidance or solutions. Intents can include bulleted lists as well as links to documents, html pages or applications. Since the artificial intelligence powering ODA chatbots is trained on language in general, configuring “entities” educates ODA about terminology and sets of synonyms specific to your organization so it selects the right intent of a skill. For example, a chatbot about Oracle cloud services might define these entities:

• OCI: Oracle Cloud Infrastructure, IaaS, IaaS cloud, OCI cloud services, PaaS, cloud infrastructure, IaaS services
• ERP: Oracle Fusion Cloud Applications, Oracle Fusion Cloud ERP, Oracle Cloud ERP, Oracle ERP, enterprise resource planning, financial management, software as a service, SaaS

Test the chatbot with utterances you configured in the skill’s intents, but also try variations. Solicit test questions from your intended audience and peers, so that you can effectively train the chatbot to select the correct intent (response) for each skill. Chatbot training is an iterative process.

Manage artificial intelligence risk

Limit how chatbot responds:

• Use retrieval-augmented generative (RAG) AI trained on your knowledge base
• Leverage structured menu-driven dialog
• Configure intents with defined chatbot responses, not generative capabilities

Monitor and measure chatbot performance

• Repeat pre-deployment tests to detect “drift” from intended operation

Get started in 10 steps

1. Select a specific problem to solve – focus on a common task. What challenges does your target audience face?
2. Assign responsibilities for defining requirements, OCI tenancy administration and ODA chatbot training.
3. Identify your content, the common questions and desired chatbot answers.
4. Complete OCI AI Foundations course & certification to get the basic concepts.
5. Get OCI tenancy with Digital Assistant (ODA)
6. Configure security, authentication (MFA): OCI Security Guide and  OCI Developer Guide
7. Add user accounts
8. Grant access to tenancy and resources to relevant accounts
9. Apply ODA upgrades if needed
10. Create an ODA Skill

Before building your own AI chatbot, you may wish to experiment with this hands-on ODA chatbot lab.

Oracle recommends that organizations consider relevant use cases in addition to applicable regulations and guidance for using artificial intelligence as well as select a methodology for managing AI risk.