I joined a leading Australian telecommunications company as a contractor to help with their digital transformation journey to provide customers with more innovative and dynamic services. I was responsible for crafting experiences between a chatbot and a human in the app and a website. I also scoped the analytics tool to automate manual processes and help with the planned scaling from 10,000 to 100,000 conversations per month.


Conversation Designer


Aug - Dec 2019


iOS, Web

Background and my role

When I joined the team, some intents were already built and rolled onto a small segment of customers. As a part of a cross-functional team, I worked in an Optimisation squad. I analysed how intents were performing in production, identified gaps and new scenarios, and highlighted API improvements needed.

Chatbots and design process

One remark. Most chatbots aren't that intelligent at the moment. Our chatbot wasn't entirely rule-based and utilised natural language processing, which allows for increased personalisation and being more human. It accepts and processes queries in a written conversational form and can understand the context.

But still, "AI" is setting up high expectations, and Virtual Assistants are relatively basic now. Generally, they follow a predefined script based on a user's utterance. At the beginning of a conversation, they try to match a user's question to a list of predefined intents and trigger one of the intents if the confidence level is high enough.

On a high level, the chatbot design process is not much different from any other design process — the same design thinking methodology, adjusted to conversational design nuances. In short, it is information architecture (on steroids) + interaction design.

Creating a chatbot analytics tool

The problem

Soon after I joined, it became clear that the manual process of the design analytics wouldn't scale, and, as we roll the virtual assistant to a bigger audience, the manual approach might leave blind spots on critical design flaws. After some market research, we decided to build an analytics tool ourselves - there was nothing good enough on the market back then. Creating the tool ourselves, among other things, helped us ensure data privacy.

My role was to ideate and design the requirements and then polish the tool, and my colleague built the scripts to match these requirements.

Tool's objective

Find, quantify, and analyse the most significant problem areas on an intent level.

Target analysis on the main problem areas instead of blindly and manually trying to find and quantify patterns in thousands of rows of data.

Defining user stories

As a conversation designer,

  • I want to be able to detect "bad" signs, not on a conversation level, but on a decision points level, so that I can target my analysis for the main problem areas.
  • I want to be able to see at each decision point if customers deviate from the designed flow so that I can find the problem areas for further analysis.
  • I want these deviations from the designed flow to be grouped by type and quantified so that I can prioritise the most important ones.
  • I want to be able to see and download filtered data so that I can analyse that segment in more detail.
  • I want deviations from the designed flow to be grouped, highlighted, and quantified so that I know where customers break the flow and what's the volume.
  • Tool's performance

    As a result, we had a tool that reduced approximately one-fifth of the designers' time spent on performance analytics, which improved efficiency and team morale (sticking to manual processes that everyone knows the algorithm can do is never fun).

    Key takeaways

    Before I start, I must say there're many great articles and guidelines around the process and core principles of conversational design, so I won't cover these. Instead, I want to talk about some insights that I had while working on the Virtual Assistant that isn't that mainstream and can be niche but worked perfectly well for us.

  • Setting expectations early and having the right personality is everything. Don't overlook the right welcome message - it is vital. Customers should know they're talking to a bot, not a human, so introduce yourself and your tone of voice.
  • The whole flow should feel conversational, users should be able to type at any decision point, and the bot should understand an entry. But sometimes, force-click buttons are a must. e.g. if a transaction is about to happen. A typed response can trigger a wrong transaction.
  • Small talks feel human, but don't go overboard with them. These intents can confuse natural language understanding (NLU). It's better to say that we don't understand than trigger small talk inappropriately.
  • Understand the underlying infrastructure thoroughly first. When designing new intents, you should spend most of your time understanding the business processes and tech infrastructure behind that particular problem you're trying to solve. The business side is essential for defining your personas and possible scenarios and understanding how we could serve these customers (what rules apply, what would live agents do).
  • Never take product structure and customer-facing documentation for granted; question all flows and decision points when building conversational design. Infrastructure might work entirely differently from what you expect by looking at a user-friendly product description. There might be technical nuances you have to cater for.
  • Improving the design is not always about making changes to the converation flow - you may find that a better customer experience requires API improvements. The designer's goal is to highlight all gaps, including technical.
  • If a user enquires about anything related to money, bring more trust by either providing exact amounts or directing customers to agents immediately. Approximate and indicative amounts seldom satisfy customers.
  • This doesn't have to
    end yet, you know!

    Email me at
    and let's talk about anything

    See more work

    Designing an app to help 20+ million practice well-being daily

    Rebuilding an app for the largest bank in Estonia