Yogev Raved | UX & Product Designer

Configuration templates

Designing a Big-data applications management tool for data engineers
Ocean for Apache Spark

Role

Product Designer
and Researcher

Teammates

Product Manager,
full stack engineer

Deliverables

UX research, wireframes,
prototypes, user testing

Overview

Spot by NetApp is a CloudOps company helping data engineers automate and optimize their big-data applications. My role was to improve the experience and process of managing configurations of big data applications.
As the Lead UX Designer for the Big-data product, I managed this project's user research and user experience design from start to finish with full collaboration between the design, engineering, and product teams.

What the #*% is it?

What are Config-templates?

Data engineers use Apache Spark to run big-data applications. Each type of application needs its own configuration (in a JSON format).
The Config-templates page is a workspace within the Spot console for storing and managing them.

How do users use them?

Select a template and copy its code.
Paste it in Apache Spark environment and run the app.
Come back to Spot's console to view the app's progress, stats and issues.

Motivation for the redesign

Improving usability issues of the current design:

Template ID is visible only for the selected.
Viewing few amount of templates - only 3 on small screens! makes it hard to browse between templates.
Adding more info to each template such as last editor, number of versions and tags will make each card much more overloaded.

Original config-templates screen

What do we want to achieve?

Reduce time spent on troubleshooting

Increase user satisfaction

Reduce support calls

Save time and money for the customers and for our organization

More users per team

Increase product value

Help sell the product to more clients

More revenue to our organization

Initial Discovery

We began with a solid foundation:

Existing user personas from active customers.
Usage data from current config-template features.
Internal subject matter experts.

Who are the users?

Data engineer

Designs, builds, and optimizes data pipelines on Apache Spark, ensuring efficient data ingestion, transformation, and processing for analytics and machine learning applications.

"I need things to be very organized so I can find what I need easily and focus on my tasks"

Data Platform Engineer (DevOps)

Develops and maintains the infrastructure for running Apache Spark applications, managing scalability, performance, and reliability to support seamless data operations.

"I need an option to view what changes were made in the templates over time and the reasons they were made"

Methods

Surveys

Open-ended and closed questions.
Features rating and ideas ranking.
Gained qualitative and quantitative insight.

Survey results

Customer & Stakeholder interviews

Explored their daily workflow for creating and maintaining config-templates.
Examined integration with existing tools and processes.
Assessed work aspects both within and outside our product.
Analyzed current usage patterns and desired capabilities.
Studied user interactions and workflows.
Uncovered key pain points and frictions.
Identified short-term and long-term objectives.

User Journey Mapping

Collaborative workshops with cross-functional teams.
Identified key interaction points and friction areas.
Prioritized problem spaces.
Created alignment on solution requirements.

Key Research Findings

"As a data engineer I want to:"

Filter and sort templates based on various criteria so that I can focus on relevant items.
Organize and filter config-templates more effectively.
Group templates by different attributes so that I can better organize my view.
Perform bulk actions (like editing tags or deleting) on multiple templates at once so that I can manage them more efficiently.
Compare templates side by side so that I can easily identify differences for debugging.

Insights

Users strongly desired version control and change history.
Template organization was a major pain point: Need for logical grouping and categorization and no way to track template purposes, and uses per application.
Search and filtering capabilities were crucial for large template libraries.

Design solution

So, we had to do some prioritization based on development capabilities

Add version history and changelog option.
Tagging system - This feature did not exist in the Spot console, and deciding on its behavior was challenging.
Advanced search and filtering.
Template comparison tools.
Applications running on config-templates list.
Presets for templates with common configurations.
Bulk actions for template management.

The one page improvement task evolved into an enhanced design and development project...

Design process

How should we approach each feature's design

Analyzed existing design system components.
Researched applicable design patterns.
Should we add a draft mechanism for unfinished editing.
Should commenting on a changelog be mandatory.
Mechanism of the tagging system.
Low fidelity wireframes on the page layout.
Original Cards layout vs a new table design.
Working with tags.
History view - compare and changelog.
Presets - custom and built in use cases.

Editing

Tagging Process

Exploring layout directions

The new features were added to the cards design. Several attempts and iterations of various layouts were made, such as wider cards to hold more data, an expanding panel and a table.
Several user' interviews and testing were conducted with internal users and with customers.
The testers were presented with the various layouts, and were asked to perform tasks such as filtering, sorting and performing bulk actions.

Expanded panel

By default: The template’s content is collapsed.
Clicking the expand icon expands the panel over the config-templates list.

Pros:

More space for browsing templates

Cons:

Can view the template content at first look
Not a usual design pattern

Cards design - Actions popup

Issue:
The placement of bulk action buttons outside this area could be confusing.

Solution:
Actions popup - Triggered by checking an item

Selecting a template opens a modal with options.
Selecting 2 items enables comparison.

Pros:

Saves space on the page

Cons:

No indication for the actions without checking an item

User testing insights

Card design

Strength:

Intuitive grouping by attribute - Tags, Author, Cluster
Template view is wide from the start

Weaknesses:

Sorting is limited
Only a few templates are on screen
Added more info to each template makes the cards much more overloaded and harder to browse

Table design

Strength:

Sortable by multiple parameters (ID, name, cluster, author, date)
No need to hide config-template info
Easier template browsing

Weaknesses:

Less initial space for templates (solved with full-screen option)
Grouping is limited to column headers, missing tag-based grouping (design system)

Instead of the design system’s grouping option I switched to a simple “Group by” selection

Main page

I have created high-fidelity mockups and ready-for-dev Figma files, considering all edge cases, empty states and placeholders. I also had the opportunity to use Spot's newly developed design system.

Behavior of the template panel for multi select.
The compare button is enabled when two items are selected.

New template popup
The user can select to start from a preset, duplicate a template or start from scratch.

Presets page

In order to aid selecting the right preset, the user can:

Select from some common starter use-cases
Select from a user created preset specific to their needs

Tagging system

So, how did we do?

Time saving

Reduced troubleshooting and config-template creation time

Monitored via Pendo

Efficiency

Reduced the amount of unused config-template per customer

Monitored via BI tools

User satisfaction

More users per team will start working with the config-template

Monitored via Solution Architects

Selling point

Helping tool for the sales team to recruit new clients over competitors

Monitored via sales reports

Key takeaways

The config-templates redesign project presented both challenges and learning opportunities, which are the reasons I find UX work so fulfilling. Through this experience, I developed a deeper understanding of the pain points faced by this product's users. The project also enhanced my ability to effectively communicate with colleagues and stakeholders - a valuable skill for all projects.