“…the greatest growth happens when we allow ourselves to be groundless and lead from a place of uncertainty.”

In early September, I took part in the Seattle Virtual Reality Hackathon. A hackathon is typically a weekend event where teams of people come together working in a specific technology type, to develop a prototype product in 48 hours.

For the Seattle event, I put forward a vision to build a prototype of a virtual reality (VR) immersive language learning experience using the Rosetta Stone Speech Recognition Engine (SRE). The Rosetta SRE is a remarkable piece of speech technology capable of grading a learner’s speech inclusive of pronunciation and grammar. Full disclosure: I’m part of the Rosetta Stone Product Team, so though I may be a little biased, but there are plenty of customers who will attest that it is a powerful tool in the proper acquisition of a target language by a learner.

Further disclosure: I had no previous experience with VR products, technologies, or development methods before choosing to enter the hackathon, but feel that augmented and virtual reality are inevitable, obvious and will subsume a shocking array of technology platforms over the next ten or more years. As a technology product person, it felt irresponsible not to dive in and start to figure it out. I’ve tended to find that the greatest growth happens when we allow ourselves to be groundless and lead from a place of uncertainty. This project forced me to do exactly that.

In a blog post on building a divergent creative mind, I assert that participation in performing arts projects is an important vehicle for practicing divergent thinking, hackathons are another. The Seattle Virtual Reality Hackathon proved an invaluable sandbox to practice the art of delivering a product on a new platform, under pressure, with new people amidst an unfamiliar culture. It was a safe place to wrestle with my problem-solving habits and fail forward into important learnings in product experience development.

“The Seattle Virtual Reality Hackathon proved an invaluable sandbox to practice the art of delivering a product, under pressure, with new people amidst a familiar culture.”

What we delivered

Rosetta Stone granted the YammeringEarl team access the Rosetta Stone Speech Recognition Engine (SRE) for the project. David Espe from the Rosetta Stone SRE team volunteered to support the team through the hackathon weekend. Vectorform, a company providing a platform to help organizations invent digital products and experiences, also supported our team through recruitment help in advance of the event and general project guidance from James Brunk.

“In 48 hours, we produced a working prototype of a Spanish language training experience…”

In 48 hours, we produced a working prototype of a Spanish language training experience intended for medical triage professionals for the HTC Vive, a premium VR platform. Our delivery was a significant achievement, and every member of the team gave their all to get there. I remain honored and humbled by their commitment to the project we put forward.

Will Blaschko does a brilliant job of describing our team process and what the development side of the event was like in his blog post on the hackathon.

The prototype worked!  But what could have worked better?  The interaction design.

Demos

Rosetta Stone/Vectorform: First Responder VR Training from Vectorform on Vimeo.

The “Facepalm” Moment

If there is one lesson I took away from this project, it is that when you are building out an immersive experience, interaction design and experiential design are at the center of delivering a great experience. You can keep a ‘lean scope,’ and have a ‘clear sense of your user’, but if you don’t storyboard and/or consider the experiential design first, then when you knit the team’s work together and put a user inside, there will be gaps in the experience.

“…if you don’t storyboard and/or consider the experiential design first, then when you… put a user inside, there will be gaps in the experience.”

As a person learning about a whole new way of delivering a product using VR technologies, I need to follow Yoda’s advice, “You must unlearn what you have learned.” Specifically, your user’s “happy path” is not a flat interaction flow anymore. You must build a multidimensional story with sight, sound, and emotion while designing a path through it.  Welcome to wonderland kiddies!

In retrospect, this was a “facepalm” moment for me. We think about these issues in Product Management all the time in 2D-software. It is the multidimensionality of immersive experiences that you can’t come to appreciate until you climb inside and play. Rather than a flat experience that may involve multi-media presentations of content, you are inside. This means you add sight, sound, touch, and emotion to the interaction flow.

As a sound designer for immersive theater and podcasts, I felt like I should have gotten this right away, but the Einstellung Effect often prevails when we are out of our comfort zones. The fact that we landed as positively as we did, from the perspective of interaction design, was a testament to what is available when you attract, listen to, and collaborate with talented team members.

The vision doesn’t end…

For three weeks after the hackathon, Will Blaschko continued working after hours, refining the team’s work.  He and I kept the conversation going over Slack and discussed what could be removed from the hackathon experience to focus on the core vision that proved the concept we were after immersive language learning for a specific context.  He simplified the interaction design, got a little extra help from a 3D modeling colleague on our character “Cecilia,” and simplified the conversational structure.

Where do we go from here?

Anywhere we want, imagination and creativity are the limits! 

“I look forward to the day when… the only suit I put on for work in is haptic.”

For my part, there will be more hackathons in my future, as well as a study of interaction design for VR and 360 degree sound design. I am in love with this space as a product category and look forward to a day when I no longer carry a smartphone, working in software means going to work inside the software, and the only suit I put on for work is haptic.

Hackathon Rumors

Please note that there are many kinds of hackathons, including those that are for big prizes and recognition that may demand a high bar of the team members. I don’t recommend starting your hackathon experiences attempting a super competitive hackathon. Do start small, local and fun.

“I don’t recommend starting your hackathon experiences attempting a super competitive hackathon.”

Now, let’s dispel the nasty rumor about hackathons being unfriendly to non-engineering types. It turns out; there are plenty of hackathons that welcome Product Managers, Program Managers, and Designers. Additionally, if you have some ability to code, or other skills, you will be surprised by the ways you can support a successful hackathon.

Next, you don’t need to be an expert in the technology focus of the hackathon. However, you should have a passionate interest in the technology of the hackathon to help drive you to tackle the fierce learning curves involved. Also, be willing to surrender your ego and all the skills you think you know well. Otherwise, you are likely wasting your time and making your hackathon team unhappy.

Special Thanks

To the team members of YammeringEarl, who fueled the best first hackathon experience I could’ve asked for…Thank you for your commitment, hard work, and passion in this project. You are all world class.

Team YammeringEarl was formed of combined talents of Will Blaschko (SRE/Unity), David Espe (Rosetta Stone SRE Dev), Xandon Frogget (Unity), Gustavo Kuhn Andriotti (Spanish language voice acting and Dev support), Scott Sizemore (Unity and 3D modeling), Rob Rood (3D Modeling), and Shimone Alkon (Narrative Design) and myself (Product Management, Rosetta Stone Liaison, and Sound Design.)

I would like to personally thank the Product Development Team at Rosetta Stone for their openness and support of Rosetta Stone’s involvement in the Seattle Virtual Reality Hackathon. Without their forward-looking vision and willingness to experiment, this project wouldn’t have happened.

Special thanks, Woody Floyd of Vectorform for encouraging his team to support and enrich the Seattle VR Community and projects like ours.

Extra special thanks to Trond Nilsen and Eva Hoerth for their herculean strength as leaders and builders for the Seattle Virtual Reality Community.  Thank you for an excellent hackathon!

And last, but certainly the most pivotal, a big thank you to Joshua Du Chene of Tomorrow Today Labs for the random conversation in Ballard and the following invitations to the Seattle VR Community meetups. Thank you for your warm welcome into the community and opening doors for me.