Experimentation Platform
This application centralizes A/B test documentation, communications, launch capabilities, and results.
My Role
Team Lead and
Manager
Stakeholders
Data Scientists
Developers
Project Managers
Leadership
Design Tools
Figma
Google Form
Survey Monkey
My Team
2 designers
5 engineers
Timeline
3 years
Dev Tools
React
Node
Java
O V E R V I E W
Background
Role
Disclaimer
The Experimentation Platform began as a simple tool for users to document their test ideas. However, after numerous feature requests and discussions with users, we identified a significant business need for a centralized testing tool across games and domains. Consequently, the platform evolved into a comprehensive product, continually increasing in functionality and purpose.
I led the multidisciplinary design and development team for the Experimentation Platform. My key responsibilities included shaping the platform's long-term vision and managing the day-to-day development of new features from conceptualization to final delivery. This involved leading user research, conducting design critiques, coordinating between design and development, and overseeing the development process.
This project is an internal business tool, so I can only share limited information and cannot provide images of the final product.
I N I T I A L D I S C O V E R Y
Defining user problems and business objectives
Goals
Develop a complex Experimentation Platform UI from the ground up.
Support multiple test environments and numerous types of tests, such as web, machine learning, and in-game.
Create a comprehensive user experience tailored to a diverse range of stakeholders, including data scientists, developers, product managers, and non-technical leadership.
Launch with a Minimum Viable Product (MVP), identify the minimal functionality required to start, then iteratively build upon it to enhance the platform.
Outcome
After leading the team in conducting user interviews, collecting surveys, and performing a competitive analysis of other experimentation platforms, we compiled a comprehensive list of desired functionalities. I prioritized this list, developed our team roadmap, and initiated the design phase.
C H A L L E N G E S
Replace an Existing UI
Problem
The functionality to launch an in-game test was confined to an outdated, legacy application that was no longer maintained. This forced users to switch between two applications for test information and launch capabilities, resulting in a cumbersome and disjointed user experience.
Process
Audit: Conducted a comprehensive audit of the old UI.
Interviews: Identified current pain points through user interviews.
Brainstorm: Developed solutions to enhance existing features and create new.
Outcome
Streamlined user flows for quicker experiment creation and cloning.
Added functionality for non-technical users to configure tests.
Increased test volume and frequency across multiple games.
Integrate with API Services Managed by Cross-Functional Teams
Problem
Integrating with both new and legacy API services managed by different teams presented significant challenges. Some APIs had fixed data schemas that could not be modified, imposing constraints on our integration efforts. Conversely, other APIs offered flexibility, allowing us to define the data structures that worked best for our needs.
Process
Investigation: Documented the capabilities and data of various APIs.
Collaboration: Developed and maintained cross-team relationships.
UI Development: Created UIs to showcase addiontal test data.
Outcome
Expanded functionality and broadened team support for Experimentation Platform features.
Streamlined the user experience by reducing the need for users to visit multiple applications for test information.
Meet Tight Deadlines
Problem
Ensuring all features were complete and integrated before the game launch, while coordinating with multiple teams each facing tight deadlines, was a significant challenge. This was particularly critical when our team was working on developing the test-launch capability and integrating it with a segmentation service managed by another team.
Process
Planning: Set early delivery dates to allow sufficient buffer time.
Segmentation: Broke work into manageable chunks, prioritizing critical features first.
Communication: Consistently provided updates to directors on progress and adjustments.
Unblocking: Frequently unblocked team members to ensure continuous progress and maintain feature priority order.
Outcome
The team frequently met deadlines ahead of schedule, which reduced stress and minimized the risk of quick fixes and errors.
I maintained open communication with directors throughout the process to ensure everyone was informed and to avoid any surprises.
Onboard Summer Interns and Junior Designers and Developers
Problem
Onboarding new team members in a fast-paced environment for a complex application required careful coordination and adaptability.
Process
Overview: Provided a comprehensive project overview.
Small Tasks: Started with smaller tasks to familiarize them with the project.
Leadership: Allowed them to lead specific features and communicate with stakeholders.
Presentation: Encouraged them to present their work at the end of their internship.
Outcome
Fresh perspectives introduced.
Enhanced skills of your teams designers and developers.
Interns gained real-world work experience.
T A K E A W A Y S
Insights Gained Through Experience
Decisive Decision Making
The power lies in continuously moving forward. I didn’t always have an answer at every turn, but it was crucial to make a choice, learn from it, and keep progressing.
User Insights
I believe engaging with users early is essential. Truly understanding the pain points we are addressing is critical for creating meaningful solutions. This approach saves our company time and money by ensuring we build the right feature from the start.
Communication
Constant communication with external and internal teams, and leadership, ensures alignment on goals, fosters collaboration, and enables quick problem-solving, keeping projects on track and building trust.
Technical Feasibility
Assessing feasibility before designing prevents wasted effort. Recognizing when to update the tech stack is crucial to meeting design goals and driving teams to deliver the best user experience.