User Researcher
Mobile Strategy Research
Our mobile app is largely focused on requesting signatures
Context
"What if we could rethink the mobile app?" It wasn't clear why certain personas and industries did so well with the app whereas others had less engagement. There was an opportunity to challenge the status quo of our mobile app strategy and see how we could increase engagement in certain verticals. We had a hypothesis about making mobile a companion or compliment experience to desktop but it was just a hunch. It made no sense to create and then test only for it to fail. If it was wrong, we needed proof it was wrong and positive direction towards what's right and then start creating.
This wasn't an easy task since changing a well established strategy would take more than a few user tests. We needed a multi-faceted approach to understand whether to keep or challenge the mobile status quo.
Persona
The target personas for mobile was all possible eSign users. Given that the direction was unclear, we wanted to keep all possible personas in mind before ruling any one of them out.
This includes:
-
Senders: People requesting signatures.
-
Signers: People signing a document.
-
Admins: Managing permission sets, settings, and administering licenses.
-
Developers: People using code to customize DocuSign for their organization.
Method & Goals
A big question in product and engineering leadership had was why certain people were not using mobile. Knowing what worked wasn't important but rather what was not working. Their big question was: "What's missing?"
I decided that we should do in-depth interviews (IDIs) with customers who had high and low engagement. I asked our recruiters to target customers in specific verticals based on the usage data.
Given that the mobile app had a high rating in the Apple App Store, I knew any insights found from the interviews wasn't going to be enough to drive major changes in the app. We needed to prove that the qualitative insights were backed by statistically significant data to lower the risk of hurting our App Store rating. So, I requested funding for a vendor to do a MaxDiff survey to to compliment the IDIs. This is different from Kano Method or a stack ranking survey question.
In a MaxDiff, the respondent can only choose the most and least important forcing a trade off for one feature over others
But why MaxDiff instead of a Kano Method or a simple stack rank? Why spend money on a vendor? The reason was because I've never done a MaxDiff and didn't have the expert knowledge to get it right. We did Kano in the past and had high attrition for our respondents. Kano required people to respond to 3 different variations of the same questions about a feature, which people found monotonous. Although the results were helpful, they lacked the clear prioritization the team desired.
Kano puts features into categories that have prioritization but feature prioritization within them isn't as clear
Stack ranking was a possibility and didn't require a lot of technical knowledge to analyze but with some limitations. It works well when the number of items is ten or less. More than that and people tend to rank the first few and either ignore ranking the rest or choose at random to get through the question. With 20 features to prioritize, a stack rank wouldn't work. We also liked that MaxDiff is better for prioritization.
Insights
From the IDIs we came away with insights on why people chose not to use the mobile app. Some of the reasons were due to the difficulty of doing certain tasks on mobile, such as adding text and signatures fields on the document.
Others felt more confident doing work on the desktop web experience because of the larger screen and easier reading experience. People use e-signature for important agreement so they're afraid of making mistakes. People use mobile for quick and easy tasks that were low risk.
People do use mobile but usually for monitoring updates, edits and check-ins
Why did certain verticals use mobile more than others? Time at their desk and investment in digital tools. The traveling salesperson with a high volume of agreements to send and sign who relies on technology all day had no problem using our product on mobile. But for a desk prone, low volume user in a largely paper reliant industry was less likely to use.
From this qualitative data we had strong insights about what the mobile app should be but as mentioned prior, I wanted more data to put little doubt in product and engineering's mind about the proposed changes.
The MaxDiff survey (n=800+), launched and analyzed by a research vendor to North American eSign customers, found that features that were for creating, such as create a new document for signing or a new template were less desired than monitoring and editing features (e.g. completion status, editing signers email address), which supported the insights from the IDIs. It also gave a list of prioritized features to include to satisfy the majority of eSign paid users.
Actions Taken
The mobile app is still being designed and follow up research is still being planned. Due to confidentiality, all I can say is that new and exciting things await the mobile team and the new vision we have for our customers' mobile experience.