Setting up Answer Bot

How to set up and start using Answer Bot

  • Understanding how Answer Bot works

    How does Answer Bot process natural language?

    Answer Bot is powered by Artificial Intelligence which means that it is able to mimic human behavior. Answer Bot uses natural language processing (NLP) to read every article in your help center and to understand the main concept behind each article. Answer Bot then takes all the concepts from all the articles and places them onto a map. Each concept gets its very own “address” on the map so that it lives near other, similar concepts. However, instead of just city, street, and zip code, this address has 500 parts. Whenever a new question comes in, Answer Bot does its best to understand the concept that the question is asking about and use the map to determine the closest existing article.

    For example, here are some concepts that Answer Bot might extract from a few questions:

    Question Possible concept
    How do I dump my tickets to a file? Exporting Data
    I’m locked out of my account Account Access / Password Reset
    How do I create a crane? Folding Origami Birds

    How does Answer Bot decide which articles to recommend?

    When an incoming question closely matches with an existing article, they become “neighbors” on the map (as described above) and it’s clear that Answer Bot should recommend the article. However, when the closest match is a few streets over, or in a nearby neighborhood, it becomes less certain that the concepts are related.

    The data science team at Zendesk carefully monitors Answer Bot’s performance and has finely tuned this over time by adjusting a “threshold knob”. This threshold is not adjustable by admin or agents, it’s only accessible to the Zendesk development teams. The threshold knob is a global control, meaning it affects all Answer Bot accounts, and is used to determine how closely two concepts must be on the concept map to be considered similar concepts. If the threshold knob is turned up, Answer Bot becomes more conservative and will recommend fewer articles that are more likely to be relevant to the question. However, this means there will also be more questions where Answer Bot does not make any recommendations at all. If the threshold knob is turned down, Answer Bot will recommend more articles, but there’s a higher chance that some of the articles will appear irrelevant to the end user.

    Common misconceptions: What Answer Bot doesn’t do

    There are some common misconceptions about Answer Bot, and machine learning in general, that can lead to confusion over how they work. In this section, we’ll address these misconceptions and hopefully give you a clearer understanding about what Answer Bot does – and doesn’t do – with your data.

    Does Answer Bot learn based on end user feedback? Isn’t that where the machine learning comes in?

    Although Answer Bot is powered by a machine learning model, this does not mean that Answer Bot is constantly learning. Answer Bot’s model does not incorporate feedback in real-time from end users or agents. Therefore, the feedback has no influence on which articles Answer Bot will recommend.

    The end user feedback is captured and used in a number of ways:

    • It is displayed to agents to provide additional context on what articles were viewed, marked as “not helpful,” or used to resolve a case
    • It is exposed in reporting for admin to track Answer Bot’s performance
    • It is evaluated by the data science team at Zendesk

    If you see that Answer Bot is repeatedly recommending incorrect articles, the best thing to do is modify the title and the first 75 words of the articles to make the main concept more clear.

    You can also create a “whitelist” of articles for Answer Bot by using labels so that Answer Bot’s suggestions will only draw from a sub-set of articles.

    Overall, we’ve found that Answer Bot’s AI-powered recommendations are more accurate and relevant than a keyword search, especially when the question is asked as a full sentence (instead of one to three words).

    However, there are times when a keyword search may work better. For example, when a user asks a single-word question via Web Widget, Answer Bot defaults to using a keyword search, as this is generally more accurate for single-word queries. The exception to this is languages, like Chinese, that do not have explicit word boundaries like spaces.

    Can I “train” Answer Bot by asking the same question and answer over and over again, and responding with “Yes” or “No” to mark an article as relevant or irrelevant?

    No. Answer Bot will consistently recommend the same articles regardless of any feedback from agents or end users. Answer Bot is specifically built so it doesn’t require any training to get started. It’s already pre-trained to understand natural language. If you test out a phrase/question and Answer Bot is making incorrect recommendations, the best thing to do is modify the title and the first 75 words of the articles to make the main concept more clear.

    If I add labels to my articles, is that like adding a keyword to the article? Can this be done to boost how often an article is suggested?

    Labels are a great way to create a “whitelist” of approved articles that Answer Bot can pull from. However, labels do not have an influence on the weighting that Answer Bot gives to each article.

    If I can’t train Answer Bot, how can I improve Answer Bot’s performance?

    The best way to improve Answer Bot’s performance is to consider the following:

    • Analyze your Answer Bot Activity - Use Explore to see which articles are you best and worst-performing.
    • The Structure of Existing Articles - Look at your help center articles and make sure that the content is concise and well organized. Each title should be phrased as a short sentence or a question.
    • Content Cues - Use machine learning technology and Guide article usage data to help you discover opportunities and tasks that will improve the health of your knowledge base.
  • Understanding where you can use Answer Bot

    Answer Bot functionality is available in a number of Zendesk products and integrations. This article is a guide to all the ways you can use Answer Bot, and where to find more information on adding it to your toolbox.

    Answer Bot in the Knowledge Capture app and Slack integration is included free of charge. An add-on subscription is required to use Answer Bot with the other features discussed in this article.

    Answer Bot in Support emails

    The most basic Answer Bot functionality is the automated email response. When an end user submits a support request via email, they receive an auto-response acknowledging their request. When Answer Bot is enabled, that email includes a list of articles from your Help Center that can help the requester solve their own issue:

    Answer Bot email

    If a suggested article answers their question, the end user can close their support request; if no article addresses the problem, the support request remains in the ticket queue to be answered by an agent.

    Answer Bot in web forms

    You can add Answer Bot to your web-based ticket forms as well. When enabled, if an end user submits a support request through your Help Center’s submission form, a pop-up modal offers them suggestions for articles that they may find useful:

    Answer Bot webform

    As with the emailed suggestions, if any article in the modal answers the end user’s question, they can close the support request with no input from an agent; if it doesn’t, their request remains in the ticket queue.

    Answer Bot in the Knowledge Capture app

    If you’re on Guide Professional or Enterprise, and using the Knowledge Capture app, by default you have some internal Answer Bot functionality, without needing a separate Answer Bot subscription.

    There are two features in the Knowledge Capture app that give your agents more access to Answer Bot functionality:

    • Answer bot for agents. When a requester adds a comment to a ticket, Answer Bot looks for relevant articles, and suggests them within the app, allowing agents to view them. Agents can then decide to insert them into their response to the requester, or use the linked articles to help them craft their reply.
    • Rapid Resolve. When an agent sends a link to an end user directly from the Knowledge Capture app (using Answer Bot for agents), end users can then self-solve their tickets directly from the article they’re viewing.

    You can disable the self-solving option in the Knowledge Capture app, if needed. This prevents end users from closing their own requests based on articles offered by an agent, or by Answer Bot.

    Answer Bot for Slack

    The Answer Bot for Slack is a free feature enabled in the Slack for Zendesk Support integration if you are on the Guide Enterprise or Professional plan.

    This feature allows Answer Bot to “listen in” on questions posed in any Slack channel configured to use the Support integration, and offer relevant article suggestions:

    Answer Bot Slack

    Users can indicate whether the offered article is useful to them by clicking the Yes or No buttons and, if more than one article is found, they can click the More suggestions button to view additional articles.

    If their question is not answered by any of the articles offered, they can submit a Support ticket:

    Answer Bot Slack response

    Answer Bot in the Web Widget

    If you have the Web Widget installed on your Help Center or web site, your customers can engage with Answer Bot whenever they need help across your site, in a conversational way.

    Answer Bot Web Widget

    Escalation options to a human agent can always be made available, such as requesting a callback, live chat or leaving a message.

    Answer Bot in the SDK

    Answer Bot can be a part of your mobile support offering. Our mobile SDKs make it easy to integrate into any mobile app, answering customer questions in-context without them ever having to leave the app or disrupt their experience. Users can mark the articles as “solving” their issues, or indicate they still need help - and escalate their issue to a Support ticket.

    Answer Bot APIs

    The Answer Bot API enables businesses to extend AI-powered self-service help to any channel. Developers can build their own own self-service automation experiences wherever they’d like. When implemented correctly users can quickly and easily mark their questions as resolved / not-resolved giving you more data for reporting, and improving the Answer Bot model over time.

  • Using Answer Bot with web forms

    When a user submits a support request through a web form on your Help Center, Answer Bot can immediately suggest links to potentially relevant knowledge base articles.

    Understanding the end user experience

    With Answer Bot enabled on your web forms, the end user receives a list of suggested articles when they make a help request through your Help Center. As soon as they submit their request, an automated pop-up window appears on screen:

    After submit request

    From this window, the end user can click any of the article titles, or the View full article button, to open the article in a new tab.

    View full article

    While viewing the full article, a modal appears allowing them to perform a number of related actions, including:

    • Clicking the request number to view to their help request in a new tab.
    • Clicking Yes, close my request to close the help request. If the end user clicks this button, they should not expect to receive any more communication about the request.
    • Clicking No to open an optional feedback window, asking for more information about why the article didn’t help. If the end user clicks this button, the ticket will be handled as usual.

    Enabling and configuring Answer Bot for web forms

    Just as email allows you to continue to manage the triggers that Answer Bot uses to respond with the suggested articles via email, web forms allow you to select and configure each web form experience which will then render the Answer Bot pop up modal on all Help Centers and Brands within your account.

    To enable web form functionality in Answer Bot

    1. Click the Admin icon (Manage icon) in the sidebar, then select Business Rules > Answer Bot.

    2. Click the Web form tab.

    3. Toggle on Web form channel enabled.

      Webform toggle

      This displays an expandable list of your brands and their related webforms.

    4. Use the toggles to determine which brands, and web forms, will use Answer Bot.

      Webform brands

    Testing Answer Bot results

    You can use labels (as with email and triggers) to segment and refine the overall articles that Answer Bot uses when searching for the correct articles, for each brand and form. Read about the best practice for using labels with Answer Bot.

    To configure and test Answer Bot labels

    1. After enabling Answer Bot for a web form, hover over the form name to display the Configure and test link.

    2. Click Configure and test to open the testing modal.

    3. Enter sample subject and description text to view the possible Answer Bot results for those terms.

  • Using labels in Answer Bot triggers

    You can use labels to help with targeting for your Answer Bot triggers. Adding labels to your Answer Bot triggers is optional. Labels enable you to specify a limited subset of articles that you want to search within.

    Labels

    Without labels, Answer Bot searches all article titles and content to identify suggested articles. By adding labels in triggers, you can restrict the search to articles containing those labels.

    This article covers three scenarios for using labels in triggers to better target articles with your Answer Bot triggers.

    Scenario 1: Targeting customer segments

    The most common scenario for when to use labels in triggers is when you have different customer segments and you want to show each segment only the relevant articles. For example, suppose you are a mobile game developer and you support both Android and iOS platforms. When you get a request from a customer who’s using Android, you want to show only Android articles.

    To accomplish this, create an “Android” Answer Bot trigger with the condition based on your custom field “Platform = Android.” Then, configure Answer Bot using labels to include only articles that contain the “android” label.

    Likewise, set up an additional trigger for the iOS platform and label.

    Scenario 2: Reducing the “noise” in your Help Center

    Your Help Center might contain a lot of articles, most of which you never want to be used as Answer Bot recommended articles.

    In this case, review your articles and add a use_for_answer_bot label to ~200-300 of the best articles. This will allow Answer Bot to focus and only suggest articles that make sense.

    Scenario 3: Conducting a limited trial for a specific type of inquiry

    While not recommended (it’s a slippery slope), some customers have proven the value of Answer Bot by focussing it on a specific type of inquiry, such as “password reset” requests.

    By creating an Answer Bot trigger that looks for specific words in the subject / description, and then using labels to restrict the articles suggested, you can limit the pilot and get some quantitative data to help support a broader rollout.

  • Setting up Answer Bot triggers, views and workflows

    Figuring out the best way to set up Answer Bot triggers, automations, views, and tags can be confusing for a lot of users.

    Basic tagging

    Basic tagging is important for almost all configurations and best practices for Answer Bot. To allow us to optimise how triggers are fired, set up new triggers to take new actions, or change automations, we have to start with some basic tag manipulation:

    1. Click the Admin icon (Manage icon) in the sidebar, then select Business Rules > Answer Bot.
    2. For every trigger listed in the Answer Bot trigger section, select Edit.
    3. Scroll down to the bottom of the trigger, to the Actions section and select Add Action.
    4. Select Add tags from the drop-down list and then insert the tag answer_bot_fired.
    5. Save the trigger.

    Now all tickets Answer Bot has fired on will have the answer_bot_fired tag and we can easily create a view to see them:

    1. Click the Admin icon (Manage icon) in the sidebar, then select Manage > Views.
    2. Create a new view - call it Answer Bot Tickets
    3. Set the conditions:
      • Ticket Status | Less than | Closed
      • Ticket Tags | Contains at least one of the following | answer_bot_fired

    Tagging solved tickets

    In this step, you’ll create a trigger that determines whether an end user has resolved their ticket based on an Answer Bot suggestion, and tags the ticket as solved by Answer Bot.

    1. Click the Admin icon (Manage icon) in the sidebar, then select Business Rules > Triggers.
    2. Create a new trigger - call it Answer Bot: Tag as solved
    3. Set the following Conditions:
      • Ticket | Is | Updated
      • Requester Role | Is | (end user)
      • Current user | Is | (end user)
      • Ticket Status | Changed to | Solved
      • Ticket Channel | Is | Email
      • Ticket Tags | Contains at least one of the following | answer_bot_fired
    4. Add the following Actions:
      • Select Add tags from the drop-down list and then insert the tag answer_bot_solved.
    5. Save the trigger.

    Now all tickets Answer Bot has solved will also have the answer_bot_solved tag, and we can easily create a view to see those as well:

    1. Click the Admin icon (Manage icon) in the sidebar, then select Manage > Views.
    2. Create a new view - call it Answer Bot Solved Tickets
    3. Set the conditions:
      • Ticket Status | Greater than | On-hold (or Pending, if on-hold status is not available for your account)
      • Ticket Tags | Contains at least one of the following | answer_bot_solved

    You should examine your ticketing workflows and make adjustments to the suggested trigger configuration to take into account non-standard ticketing workflows. For instance, if you have agents self-assigning and solving tickets without adding a public comment, the tag answer_bot_solved would be added, even though the ticket was not solved by Answer Bot. Adjusting the workflow, or adding conditions to the trigger to specify a ticket assignee, for example, can help avoid these conflicts.

    Removing tags from reopened Answer Bot tickets

    You could even take this one step further and add another trigger to remove the answer_bot_solved tag, if a ticket is reopened:

    1. Click the Admin icon (Manage icon) in the sidebar, then select Business Rules > Triggers.
    2. Create a new trigger - call it Answer Bot: Tag as Reopened
    3. Set the conditions:
      • Agent replies greater than 0
      • Ticket tags: contain answer_bot_solved
    4. Add the following Actions:
      • Select Add tags from the drop-down list and then insert the tag answer_bot_reopen.
      • Select Remove tags from the drop-down list and then insert the tag answer_bot_solved - this will remove that tag
    5. Save the trigger.

    Following up when customers self-solve

    Extending on the previous steps, you can also add another action to send the requester a follow-up email to confirm that their request has been marked as solved.

    1. Click the Admin icon (manage_icon) in the sidebar, then select Business Rules > Triggers.
    2. Create the trigger in the previous step, or edit the Answer Bot: Tag as solved trigger
    3. Add a new action:
      • Email > Requester
      • Enter an email subject and body
    4. Save the trigger.

    The final trigger should look like this:

    Tag as solved

    Creating an Answer Bot trigger for follow-up tickets

    In some situations, you may want to check in on a closed ticket. Closed tickets cannot be reopened, so to continue the conversation (rather than starting a new one) you need to create a follow-up ticket.

    When you create a follow-up ticket, all of the closed ticket’s information, including tags, is carried over into the new ticket. That means that the answer_bot_solved tag is applied to the follow-up ticket, which prevents Answer Bot from firing on the new ticket. This is fine if you do not need Answer Bot to work on the new ticket; however, if you want to include Answer Bot suggestions in the ticket notifications, you’ll need to remove the answer_bot_solved tag.

    To create a trigger removing the answer_bot_solved tag from a follow-up ticket

    1. Click the Admin icon (Manage icon) in the sidebar, then select Business Rules > Triggers.
    2. Create a new trigger - call it Answer Bot: Follow_up
    3. Set the conditions:
      • Ticket is: Created
      • Channel is: Closed ticket
      • Ticket tags: contain answer_bot_solved
    4. Add the following Actions:
      • Remove tags: answer_bot_solved
    5. Save the trigger.

    Suppressing Customer Satisfaction surveys on Answer Bot tickets

    Customer Satisfaction surveys were designed primarily for when human agents have been involved in solving the ticket. Many customers choose to disable satisfaction surveys for Answer Bot tickets. This step assumes that you have tagged tickets solved by Answer Bot with the answer_bot_solved tag.

    1. Click the Admin icon (Manage icon) in the sidebar, then select Business Rules > Automations.
    2. Open the automation that’s been set up to send Satisfaction surveys, by default it’s called Request customer satisfaction rating (System Automation).
    3. Add a new condition:
      • Ticket tags: contains none of the following: answer_bot_solved
    4. Save the automation.
  • Analyzing your Answer Bot activity

    Zendesk Explore features a pre-built dashboard to help you monitor your Answer Bot activity and article effectiveness. The dashboard can help you identify if Answer Bot is solving your support requests, how quickly users are opening suggested articles, and how your individual articles are performing.

    You can edit and customize the Answer Bot dashboard by cloning it (see Duplicating pre-built dashboards). If you need something more complex, you can write your own reports using a wide range of metrics and attributes. For details, see Getting started creating queries.

    The information on the dashboard updates on the following schedule:

    • Explore Lite: Each day at midnight in the timezone of the account.
    • Explore Professional: Once every hour. The update time is randomized within the hour.

    Accessing the Answer Bot dashboard

    Use the following procedure to access the Answer Bot dashboard.

    To access the Answer Bot dashboard

    1. In the Zendesk product tray, click the Explore icon (Explore icon).

    2. From the list of dashboards, select the Zendesk Guide dashboard.

    3. In the Guide dashboard, click the Answer Bot tab.

    Knowledge Capture and Answer Bot are the Guide components that Explore currently reports. If they are not configured then the Guide dashboard won’t be displayed.

    Understanding the Answer Bot dashboard reports

    The Answer Bot dashboard shows information about Answer Bot activities, ticket resolutions, and activity by articles. You can filter the reports on the dashboard by Time, Answer channel, Answer brand, Article language, Ticket group, and Ticket form.

    Answer Bot dashboard headline metrics

    The dashboard displays the following headline metrics (KPIs):

    • Suggestion rate: Displays the percentage of customer enquiries where Answer Bot offered a suggestion. The KPI also displays the number of answers, unsuccessful attempts, and attempts that Answer Bot made. Click the Improve link to get tips about how to improve the suggestion rate.
    • Click-through rate: Displays the percentage of responses clicked by end users from the total responses offered by Answer Bot. The KPI also displays the number of clicks, clicked articles, and the median click time. Click the Improve link to get tips about how to improve the click-through rate.
    • Resolution rate: Displays the percentage of enquiries that are resolved with no agent involvement. The KPI also displays the number of resolutions, indirect resolution answers, and the median resolution time. Click the Improve link to get tips about how to improve the resolution rate.
    • Rejection rate: Displays the percentage of suggested article marked as unhelpful by end users from the total number of suggestion offered by Answer Bot. The KPI also displays the number of articles marked unhelpful. Click the Decrease link to get tips about how to improve the rejection rate.

    Dashboard

    Answer Bot dashboard reports

    The dashboard displays the following reports:

    • Answer Bot activity volumes by date: Shows the number of offered, clicked, and resolved bot answers and resolutions over the chosen time period.

    Volumes by date

    • Answer Bot activity rates by date: The percentage of Answer Bot answers clicked, rejected, or resolved over the selected time period.

    Rates by date

    • Resolutions by month (12 month): The number of resolutions and the percentage resolution rate over a 12 month period.

    Resoltuions by month

    • The following reports can be filtered by Suggested article language, Answer brand, or Answer channel:

    Answers by attribute

    • Answers by selected attribute (top 10): Displays the top ten offered answers by language, brand, or channel.

    Top 10 answers

    • Clicks and resolutions by selected attribute (top 10): Displays the top ten clicks and resolutions by language, brand, or channel.

    Clicks and resolutions

    • Resolutions by selected attribute (12 months): Displays the resolutions over the last 12 months sorted by language, brand, or channel.

    Resolutions by attribute

    • Answer Bot activity by article: A detailed report about Answer Bot activity for all of your Guide articles. You can restrict the range of results shown by using the Top and Bottom filters.

    Activity by article