Methodology

The Research Process

Client
Allstate

Year
2024

My Roll

Lead UX Research

How many users:

7 Users

Interviews

I recently chatted with senior developers at Allstate.

They’re quite skilled and prefer to handle things themselves. They don’t trust the value of certain tools provided, even if they work well. If they try a tool and it doesn’t meet their expectations, they’re done with it!

Key Research Decisions and Outcomes

• Target Participant Selection

• Through preliminary discussions and alignment with stakeholder and business objectives, the target demographic was precisely defined.

• Goal: Recruit mid-level developers with 3 to 5 years of experience as they are the primary users responsible for code development and tool integration.

• Interview Protocol Design

• The semi-structured interview guide was crafted to investigate critical pain points within the daily workflow.

• Key Questions Focused On:

• The step-by-step process required for accessing internal development tools.

• Identifying the primary time sinks during code production.

• Understanding the approach and resources utilized for code issue resolution.

• Inventorying the most frequently used and desired development tools.

Core Research Insights

• High Friction in Issue Resolution: Developers often encounter multiple, sequential steps to fix code, leading them to outsource solutions (using Stack Overflow) instead of relying solely on internal resources.

• Significant Time Loss: Participants reported spending hours, sometimes days, attempting to locate necessary internal documentation and resources, often resulting in a reliance on external communities or other workarounds.

Actionable Next Steps

• The discovered insights revealed substantial systemic friction impacting the developers' daily flow. Crucially, the exact location and context of these issues were not yet defined.

• Proposal: I recommended and initiated a subsequent Usability Testing phase across the system. This approach was chosen as the fastest and most intuitive method to observe, in real-time, the specific friction points and mandatory steps that mid-level developers must navigate.

Competition Analysis

Here is the 3 of the resources that developers turn to, when they have questions that require immediate assistance:

1. Service Now

2. Ms Teams Channel,

3. Right Anwers (internal tool at Allstate)

Insights

Service Now:

Formal Ticketing System:Developers submit a ticket for code-related issues and must wait for a response, a process that can take days or weeks.

High Friction & Inefficiency: Low resolution rate due to abandoned or unresolved tickets (e.g., wrong team assignment, insufficient initial information).

The system is not searchable for peer solutions, lacks community engagement, and offers no trackable solution-saving functionality.

Right Answers Documentation Library:

Designed to simplify code-fixing steps by allowing developers to browse and reference existing documentation.

Trust & Maintenance Issues: The library contains an unreliable mixture of outdated and new documentation; there is a critical lack of ownership and maintenance, making documentation accuracy questionable. The tool suffers from poor searchability, low data metrics, and is not discoverable or community-driven.

MS Teams Channel Real-Time Peer Community:

Functions as an informal community channel where developers can chat directly with peers to find quick, accurate results in minutes.

Scalability & Reliability Challenge: While fast and accurate, resolution is dependent on a peer being available and willing to chat, making it difficult to consistently get hold of immediate assistance during peak demand. The valuable knowledge shared in chats is not systematically captured or indexed.

Usability Testing

By systematically observing developers interaction with internal tools.

I identified and documented nuanced points of frictions thought target video sessions. These artifacts enabled the research team to translate, observed paint points into actionable opportunities, driving the tool’s evolution towards greater intuitiveness and usability for developers.

Integrating key stakeholder in to live session viewings not only fostered interdisciplinary collaboration but also ignite a wave of technical ideation, ensuring that propose solutions were grounded in real world developer experience and prioritized their success.

This approach transformed users feedback into strategic advantage, elevating the quality and relevance of our internal product for its core audience.

What task I choose to focus in Usability Testing?

• Onboarding to products ( what steps they take) trouble shooting.

• Utilize internal tools for code production trouble shooting.

  Insights:

• facilitating Usability testing we find out developers stock for days until they found solutions in Stack overflow or community channels.

• Access to security tools took months. (I have interview Security groups at Allstate) I didn’t want to look for resolving issues since this was a test as a researcher. we wanted to track

• Documentation with 5 years updated mixed with current documentation (without ability to learn who is the author)

• Users recruited were able to move on and documented) to show us how the fix those issues.

Survey

Was conducted to track usages and feedback from our users after Iterating.

This survey data provided crucial success metrics, enabling the team to validate design hypotheses and prioritize the roadmap for future enhancements based on empirical user sentiment and experienced efficiency gains.

Survey Highlights and Strategic Value

• Progress Tracking: Implemented specifically to measure performance and sentiment shifts immediately following the deployment of the redesigned system features.

• User Acceptance Metrics: Quantitatively assessed how developers felt about the new system, focusing on overall satisfaction and perceived utility.

• Friction Identification: Enabled the large-scale capture of specific, remaining workflow pain points that surfaced after the initial redesign, ensuring no friction was overlooked.