Paul Henry Smith
AI Product Designer

UI and Interaction

Range-setting for Music Practicing

The users’ mindset: focused and strapped for time

One of the goals for improving the iOS app, Cadenza, was to help users be more effective with their practice sessions. The app accompanies the musician with audio and displays the musical notation for their part, synced in real time to their playing.

I approached this rather broad goal with a specific, and often-requested new feature: being able to stop playing and define a short bit of a piece of music to practice right then and there.

Musicians need to be able to start at a specific spot within a piece of music while they are practicing, but their mindset during practice is very important to understand when designing a solution.

User concerns while practicing include:

  • hands are occupied holding a musical instrument

  • in the midst of a practice session (in the zone)

  • don’t hinder the train of thought

  • want to practice a small bit slowly and then faster

  • want to easily repeat that bit

  • want to save bits to come back to again

Tasks for the user

  • define a range as quickly as possible

  • start, stop, and restart quickly

  • save a newly defined range

  • more precisely define a range

  • change speed while playing a range

Design actions

  • Assess current solutions

  • Define flow

  • Build and test prototypes

  • Refine flow and interaction

  • Integrate solution with current product capabilities

Company: Sonation
App: Cadenza
Year: 2015
Platform: iOS
Roles: UI/UX, Interaction design

Predictive AI powered this Star Wars Experience


The scope of this journey starts with the practice session, however, the area of the journey I worked on is the point at which the user stops playing to work on a problematic point in the music.

Predictive AI powered this Star Wars Experience

Details for the optional steps.

Predictive AI powered this Star Wars Experience

low-fi mockups tested with users

Concept 1: Horizontal picker

  • Shows measure numbers for currently selected line

  • Swipe horizontally to extend range

  • Tap to set starting measure number

Problems with this approach

  • Minimum actions to complete:
    Long Press + Drag + Tap + Tap 

  • Horizontal list limited to few visible options

  • No way to specify ending of range

  • Large number of list items can be unwieldy (horizontal scroll)

  • Tap target for “OK” too small for use case (holding an instrument)

Predictive AI powered this Star Wars Experience

Low-fi mockups tested with users.

Concept 2: Vertical picker

  • Starts with left-most measure on currently selected line

  • Swipe up/down to see available choices

  • Tap to set starting measure number

  • Could use iOS picker

Problems with this approach

  • Minimum actions to complete:
    Long Press + Drag + Tap + Tap 

  • No way to specify ending number

  • Large number of list items can be unwieldy

Predictive AI powered this Star Wars Experience

Final design: Popover

  • Minimum actions to complete:
    Long Press + Tap (outside)

  • “Plus” icon/button appears if selected range is not already in preset list.

  • Optional “End” measure number (defaults to end of piece of music

  • Drag to rotate circle to change number in Start or End box. Clockwise = higher, counterclockwise = lower.

  • Uses velocity of rotation to enable fast and slow changing of numbers

Problems with this solution

  • Unclear to “tap outside” to set. 

  • No “dismiss” or cancel option.

  • Defaults to first measure in line.

Thinking about scores and grades in a college study app
Predictive AI powered this Star Wars Experience

Prototype version 1
Made with Principle

A study app prototype for user testing

This prototype was used to test whether and how well students understood the scores they earn in Pearson’s study app, Revel. In the app they can take quizzes, do reading, and answer questions to earn points. The problem was that scores are often weighted, so it’s hard to know what impact individual types of work would have on total performance. For example, essays might count less than quizzes, so doing well on essays might not matter as much as doing well on quizzes.

Observations showed that students try to figure out their “real” total score based on individual item scores, but often miscalculate, due to opaque weighting factors. If that error leads them to think they're doing okay in their class when they’re not, there can be a bad outcome.

The bottom line is that students simply want to know if they are doing okay, or if they need to worry. They are not usually looking for a number for its own sake.

Company: Pearson
App: Revel
Year: 2019
Platforms: iOS, Android
Role: UX/UI/interaction design, data visualization

Predictive AI powered this Star Wars Experience

Prototype version 2

This second version tested better with students. Although few of them swiped to see the details behind the main score, they were more directly drawn to seeing what types of assignments needed improvement. That was their primary goal: to understand if they had a problem in this class, and if so, what to focus on to fix it.

Predictive AI powered this Star Wars Experience

Final design for Revel “scores” view.

- Scroll -

Final design

To calm down the over-use of numbers we used circular graphs to show relative performance. Also, the complexity of the numbers was confusing. Some assignments might be weighted more than others, and using numbers there could be ambiguous. That was causing confusion and anxiety, too.

Including a way to understand how different types of work are weighted in a particular course by using relative size helped clarify the scores. At the end of the day, the primary job of this view was to answer the question, “am I doing okay in this class?”

Organizing iPad gestures

Gestures and behaviors

These mockups show the main gestures and resulting behavior of Pearson’s eText app for reading, navigating, annotating, and marking up with Apple Pencil. Color overlays are explained by the legend on the first screen.

Company: Pearson
App: eText
Year: 2018
Platform: iOS (iPad)
Role: Interaction design

See “Tablet Gestures” in InVision

An app for trumpet-playing Star Wars fans!?

Created as a promotion for Sonation, the “Millennium Trumpet” app for iPad was launched one week before the release of Star Wars, The Force Awakens. The purpose was to take advantage of the hype around the movie opening to connect musicians who love Star Wars with Sonation’s apps for expressive music-making.

A new hope for reading music? The Music Crawl
Like the opening prologue in every Star Wars film, the music notation in the app scrolls into the distance while AI controls scroll speed to ensure only the upcoming notes are on the largest and most legible line.

Company: Sonation
App: Millennium Trumpet
Year: 2015
Role: UI/UX, Interaction, Visual design, Mini Cooper driver

Predictive AI powered this Star Wars Experience

See it in action.