When we talk to developers, one of the top points of appeal around building for Misty is the fact she is a platform with personality. The majority of platforms we are all familiar with — tablets, phones, AR/VR, etc. — don’t have this creative element, so it’s no wonder the construct piques the imagination. CP, our Prototype Engineer, agrees with this and had a lot of fun programming Misty’s personality in our recent Game of Thrones and Misty video. Now we’ll explain how he did it.
So Happy (…Or Not) to See You: Facial Recognition
Misty’s ability to recognize faces can trigger the personality she exhibits. However, lighting conditions in a room and/or on a TV can make facial recognition tricky; in the interest of being fully transparent, we didn’t use it in this video. Had CP used Misty’s facial recognition capabilities, the process would have been simple. He would have trained Misty to recognize Daenerys’ face and the Night King’s face by showing her their pictures. After a minute or so of training, all that would be needed is an if else condition to make Misty emote when she saw them.
Rather than using Misty’s facial recognition capabilities, CP coded Misty’s first display of personality in the video as a skill. The process requires a simple combination of facial expression, chest LED, and playing a sound. Just three lines of code later and we have the result we want: Misty’s clear approval of Daenerys and distaste for the Night King.
All Together Now: Scripting Behavior
In the second half of the video, when Misty encounters the “Misty Walkers”, CP took advantage of Misty’s external REST API endpoints and used the MistyPy Python library to write a Python script. This script multithreaded each action of each robot so they could all be controlled at the same time. There are four definition blocks (i.e. functions in Python):
- Make Misty look somewhere random.
- Make Misty look straight.
- Change screen image to full black.
- Change screen image to Blue Eyes.
CP can run any combination of these blocks with any amount of delay he wants between each robot. According to CP:
“It was pretty cool to watch so many Misty’s turn their head to look straight at the same time.”
Personality serves multiple roles in the world of robots as platforms. It brings down potential walls in engaging with the technology. The platform becomes more approachable, especially for end users not as adept with technology. It can also be a key factor in driving involvement and engagement with the platform and related skills. As noted by CP:
“The added value as a developer is that one can go beyond meeting functionality. You can create a bond with the user which is always very important for any product, app, or skill.”
Consider the manifestations of personality on a robot as a continuum: From providing developers with the ability to program small elements of personality relative to an individual skill, all the way to robots responding to outside stimuli and human interaction with representations of human emotion. In the latter case, of course, this emotional effect will be tied to a modeled way of responding relative to inputs vs. being actual emotions.
As a platform for developers, Misty starts with the approach of enabling developers to program her personality. When you consider the early stage of the market for an autonomous robot to be used in a business or home, there is practicality in developers having control. We’re excited to explore how robots can represent human emotion as we make progress on Misty’s roadmap. In the meantime, we can’t wait to see how you program Misty’s personality in your future skills.