Customer service agents at Agoda have long been burdened with a disjointed set of tools and resources (examples below) with which to resolve customer requests. We're working on a new case management system called Athena that consolidates these legacy tools, improving efficiency and data collection so we can continue to optimize our operating procedures.
A crucial aspect of this project has been change management and juggling various stakeholders—not just our agents, but the teams responsible for standard operating procedures, agent learning and development, and other products to be integrated into our system.
Team & role
The Athena case management system was my first direct exposure to Agoda's Customer Experience Group (CEG)—the business unit responsible for our customer service channels, tools, and operations. I began providing limited design support while working on several other products, but eventually became their dedicated designer.
Although I've been involved in Athena since 2019, my role has evolved over time. This write-up focuses on a re-design effort that took place when I the sole CEG designer, and Athena was my primary product. At the time, the Athena core team comprised a product manager (Lei Lei Wu), a project manager (Pimmada Jetsadakraisorn), and myself, with support from Sophie Auzanneau and many other collaborators.
I've since onboarded a second designer to the CEG product area and passed on ownership of the Athena interface, but continue to help strategize and support integration of other projects and products.
Problem & scope
A first version of Athena was launched at the end of 2019, where agents could search customer bookings, create cases for customer issues, and track information related to each case. Intended as a proof of concept, it was unfortunately created with little engagement from our agents, and missing functionality deemed "critical" cropped up left and right. It quickly became a disorganized collection of features, and after several problematic feature launches, we acknowledged the need to step back and re-examine both the product and our process.
An agent's journey from receiving a customer contact to closing a case can loosely be broken down into the steps below, and we decided to separate the re-design of Athena into two phases. This first phase spanned June to September of 2020, and our goal was to integrate into Athena everything an agent needs to confidently identify a contact reason and open a case, eliminating the need for other tools in this part of the agent journey.
Success metrics
We had two main measurements of success.
The first was Athena engagement, measured in percentage of what we call handle time—the time agents spend on a customer request, from receiving the initial contact to resolving the issue at hand. As we migrate information and functions into Athena, agents should be spending more time in Athena and less time in legacy tools.
The second was a decrease in visits (before opening a case) to the legacy system used to access customer bookings. If we were successful in including all necessary information, agents would not need to refer elsewhere.
We would also look at global handle time to ensure there was no increase, and measure agent satisfaction by means of a survey.
Process
With the first version of Athena having fallen short, we gathered our learnings and piloted a more structured, collaborative process with special attention paid to change management. The previous approach was best described as a waterfall, but here we continuously engaged stakeholders throughout, with a recurring meeting of representatives from all teams involved.
Understand and define requirements
First, we set out to understand and compile all use cases for our product. This included going through documentation of standard operating procedures (SOPs) our agents are asked to follow, and discussions with our SOP team. It also included rounds of interviews with different agent teams, which helped us develop a more nuanced understanding of Agoda's complex procedures, and distinguish between SOP and agent habits born of inefficient tools. In addition, we engaged special project teams to understand what data should be tracked to inform future optimizations.
The Athena team worked closely to define the requirements for our MVP. A compilation of use cases allowed us to prioritize and be deliberate about what we were leaving out, as well as determine potential fast and slow follows. Requirements were then shared with a wider audience of stakeholders to reach a consensus on what we would be building.
Iterative design and feedback
After a consensus was reached regarding product scope and definition, I started working with the Athena core team to draft agent flows, wireframes, and content. Although I owned the design files and (most of!) the sketches, we held working sessions several times a week, and this was really a collaborative effort.
We continuously presented iterations of increasing fidelity to agents, special project teams, and our learning and development team (LnD) for feedback. Although my final deliverable would be high-fidelity UI for our development team, one thing I loved about this project was the opportunity to challenge and inform our processes with my proposals along the way.
Because agents are experts who go through training, there is sometimes the temptation to sacrifice intuitive features to a "clean" UI. Engaging LnD in the design process and learning about the training methods at their disposal helped us hypothesize what changes might be easier versus more challenging for agents to adopt. This also gave the team more insight into our reasonings behind the product, allowing them to then communicate this to our agents as well.
Design review
The last step in our process before handing designs to our development team (and working with them on implementation) was to gather all stakeholders for one final review. This review included the CEG senior leadership team, as well as representatives from each team we had been collaborating with.
Having engaged everyone throughout the process, I'm happy to report that this went smoothly as hoped for, with no surprises 🙂
Final design
Below are just a few snapshots of the final design, but I'm happy to show a demo and walk through some specific design decisions in a deep dive.
Outcomes
Launched re-design
Athena engagement increased by 5% of handle time.
Instances where an agent received a customer contact and visited our legacy system before opening a case dropped by over 60%.
According to survey results, a majority of agents were satisfied with our re-design. We had taken the time to communicate what we were trying to achieve in the long term, and found they were much more receptive to change, even when it inconvenienced them in the interim.
We did unfortunately see an increase in global handle time, but this was attributed to shortcomings in the rollout, to be addressed with changes in agent training.
Collaboration and process
Our adoption of this iterative, collaborative process resulted in a much smoother launch than we had previously seen, and our team has approached the second phase of the Athena re-design in a similar manner.
Of course, there was room for improvement. For instance, our sessions with agents had been quite scrappy the first time around, but we've now worked with a researcher on moderation technique, as well as to introduce methodologies such as the cognitive walkthrough.
Although the integration of this process into other project timelines is something our team continues to advocate for, other teams have learned from our experience and begun to change the way they engage their users and other stakeholders.
Design's role within CEG
CEG only recently partnered with our product team, and this re-design was the first project where design was truly involved from the beginning.
My contributions to this case management system helped me build trust and make allies within CEG—especially within the teams working on our ecosystem of internal tools. I went from working only with the Athena core team, to having other projects referred to me, to being included in more strategic crossover conversations, which has helped me identify additional opportunities for design.
Appendix
Design principles
I've encountered many challenges and learned a lot from working on our agent tools. To prevent repeats of previous mistakes, I worked with the Athena team (before this re-design) to introduce a list of principles that could be used as shared language for design decisions. These—along with some notes on how we apply each principle—now serve as design guidelines for not just Athena, but all agent-facing products.
Always understand the needs being addressed.
Build trust and empower. Information should inform action.
Design for scalability. Don't design single-use features in response to ad hoc requests.
Design for efficiency. Don't increase handle time.
Strive for a cohesive agent experience. Build frameworks.
Involve relevant stakeholders along the way.
Use training for tool functionality. Do not rely on training for process recall.
Agent shadowing and immersion
In addition to project-based research, I shadow agents on an ongoing basis. This helped me develop context when I began working on agent-facing products, and continues to provide insight into how agents handle requests and use their ecosystem of tools.
As part of an initiative by senior leadership, I also went through a condensed version of training for new agents, and spend a few hours each month handling customer requests.