One Part Hardware, One Part Software, Packed with Technology
She may look cute and approachable from the outside, but on the inside, Misty is all business.
Capabilities
Detect and Recognize People
With her high-resolution camera, Misty has built-in, on-the-edge face detection and face recognition capabilities. You’ll also get size (rough distance) and positional information that you can use to code Misty to follow the faces she sees. We’ve seen this used to create a variety of greeter skills, conversational skills, and security skills, and there are many more possibilities.
Hear, Act, and Communicate via Voice
Misty has a built-in array of three far-field microphones that she can use to record audio and get positional information about the sounds in her envioronment. With beam-forming, you can filter out sounds (like music) that Misty detects at an unspecified angle. What’s more, you can code Misty to respond to the “Hey, Misty” wake word to trigger speech capture. Misty uses her native end-of-speech detection engine to automatically record until the user stops speaking. From there, it’s up to you! For example, you can code Misty to send the audio file to your favorite natural language processing (NLP) service—such as Google’s Dialogflow—and use Misty’s text-to-speech engine or a third-party TTS service to create deep voice interactions between Misty and your users.
Convey Personality
Misty provides extensive eye and sound packs, so it’s easy to add personality to your skills. These assets let Misty express everything from admiration to fear, love to goofiness, surprise to exhaustion, and tons of emotions in-between. Mechanically, you can use Misty’s 3 degree-of-freedom neck to move her head to a curious tilt, or you can write a line of code to have her wave, point, dance, or throw her arms up in excitement. Our team learned a lot about personality when we worked on products like the app-controlled BB-8 droid at Sphero, and we knew programmable personality would be a must-have capability when we designed Misty. You’ll be amazed at the scope of personality and expression you can add to bring your skills to life.
Be Physically Expanded & Personalized
Whether you’re adding greater manipulation to Misty with a custom 3D-printed gripping arm, mounting a specific type of sensor for your business needs, or (as an educator) teaching your students to design creative attachments, Misty has you you covered. We provide CAD models for Misty’s shell, as well as for several “blank” parts you can easily build off for many of her attachment points (like her arm sockets, head panel, and trailer hitch). We can’t wait to see what you create!
Utilize Third-Party Hardware
We couldn’t possibly pack everything for everyone into one Misty-sized robot. That’s why we designed her to be easily expanded, from both a mechanical and an electrical perspective. Electrically, you can expand her capabilities via the USB and UART serial connections underneath Misty’s Backpack. Use these ports to interface with thousands of existing devices, like the Raspberry Pi. Or, even easier, use the Misty Backpack for Arduino (sold separately) to open Misty up to the entire Arduino ecosystem and take advantage of the hundreds of third-party Arduino Shields on the market.
Respond to Touch
Misty has six capacitive touch panels embedded around her head—four in quadrants on the top, one in her chin, and one in the handle at the back. You can use these panels to code Misty to move and express personality in response to human touch, or use them as an input device for users to interface with your skill (for example, tap Misty’s head to change modes, and touch her chin to select). There are dozens of creative uses for capacitive touch in Misty’s skills. We can’t wait to see yours.
Capture Images, Collect Data, and Share
Centered in Misty’s visor is a high-resolution camera that she can use to take pictures and record videos. These might be pictures of intruders that Misty takes while running a security skill, or image data to analyze in search of something specific. In truth, how Misty uses this data is up to you. We’ve experimented with sending image data to Microsoft’s Cognitive Services APIs for in-depth analysis that returns context about Misty’s surroundings, which she she can then use to make decisions elsewhere in the skill. We’ve also built integrations with Twilio that let you send a text message to Misty, and she sends back a photo of what she sees. These are just two simple examples of what you can do with Misty’s image capture, collection, and sharing capabilities.
Play and Record Audio Clips
We put a great deal of effort into designing Misty’s speaker system, and consulted with an audio engineering professional to make sure she would have premium sound with great bass—especially for a robot her size. This means two speakers, a large, airtight chamber, and a bass tube that exits below Misty’s chest. Whether you’re playing voice from a text-to-speech engine, listening to music, or playing a file from her personality sound pack, Misty sounds great. In addition to playing sounds, you can program Misty to record audio using her far-field microphone array.
Stream Video and Audio
In addition to capturing data, Misty can use her microphones and camera to stream audio and video to a remote device. She can broadcast this stream to clients on your local network and beyond, so you can see the world from Misty’s point of view no matter where you are. There are countless applications for AV streaming with Misty — you might feed the video stream into object recognition software, use the robot for telepresence, or set up a remote robot control station to navigate Misty around unfamiliar environments (just to name a few).
Integrate with Third-Party APIs
Integrating the skills you build for Misty with third-party APIs opens up a tremendous amount of potential. Misty’s hardware and software have tons of capabilities built right in, and when you pair these capabilities with all of the services available in the cloud, you end up with an extremely powerful robot. You can do something simple with alerts or weather data. Or you can build a home monitoring skill that connects the text messaging features of Twilio, to Google’s text-to-speech APIs, to the APIs for your home automation system, to Misty’s driving and camera APIs. Data streams. Natural language processing. Computer vision APIs. There’s no end to what’s possible!
Move Around a Space
A robot that’s stuck in one spot is very limiting, which is why we gave Misty robust, smooth locomotion to move around her environment. Beneath her body panels sit two motor/gearbox driven treads that let Misty get around without getting stuck. And because movement isn’t useful unless Misty can sense the world around her, she uses three time-of-flight range-finding distance sensors in front, one in rear, and four facing downward for obstacle and edge detection. In the event these sensors miss a thin obstacle, Misty’s got four mechanical bump sensors that trigger when she collides with an object.
Misty II Comes in Three Versions
| Enhanced | Standard | Basic | ||
|---|---|---|---|---|
| Detect and Recognize People | ||||
| Hear, Act, and Communicate via Voice | ||||
| Convey Personality | ||||
| Physically Expanded and Personalized | ||||
| Utilize Third-Party Hardware | ||||
| Respond to Touch | ||||
| Capture Photos, Collect Data and Share | ||||
| Play and Record Audio Clips | ||||
| Stream Video | ||||
| Integrate with Third-Party API’s | ||||
| Move Around a Space | ||||
| SLAM Mapping & Navigation | ENHANCED SLAM: Produce a map for a 1600–2000 sqft space** | Produce a map for an 800–1000 sqft space** | ||
| Autonomous Charging | ||||
| RGB+Depth for AI and CAD | ||||
| ** Maps can be stitched together to cover a larger space |
Misty II Basic Edition
The Misty II Basic Edition provides a professional-grade platform robot at a highly affordable price, especially relative to the sophisticated technology packed inside. This Misty II is designed for assignments (use cases) where a fully autonomous, self-charging robot, sophisticated mapping/SLAM (Simultaneous Location and Mapping), and 3D images for AI and/or CAD are not required.
Misty II Basic Edition includes all of the capabilities listed in the capabilities section above.
Misty II Standard Edition
The Misty II Standard Edition is the original Misty II model. It provides developers and their organizations with a professional-grade platform robot that can take on a wide variety of assignments, including use cases that benefit from SLAM, 3D images for AI and/or CAD, and full robot autonomy made possible through auto charging. The Standard Edition produces maps for an ~800-1000 sq ft space.
Misty II Standard Edition includes all of the capabilities of the Misty II Basic Edition PLUS…
SLAM Mapping & Navigation
The Standard Edition of Misty adds an Occipital Depth Sensor and complex simultaneous localization and mapping (SLAM) algorithms inside the Qualcomm 820 SOM to allow Misty to generate and navigate within a map of her space. This is important for many use cases and skills that you create. The depth sensor itself has a (safe to the eye) laser projector, a wide angle IR camera, and two more narrow IR cameras to provide depth data. On the skill side, once you’ve created a map by driving Misty around your space, you can specify X, Y coordinates in your skill to have Misty drive to those points.
Autonomous Charging
For Misty to be truly autonomous, we have added the ability for Misty to return to her wireless charger. Coupled with the SLAM mapping and navigation capability in the Misty II Standard Edition and Misty II Enhanced Edition, you can create skills to monitor a space, provide security, add safety, and satisfy many more assignments.
RGB+Depth for AI and CAD
The data from Misty’s depth sensor is also available to you, the developer, to use as you wish. This additional data on top of the RGB image data can be used in machine learning models or to generate 3D representations.
Misty II Enhanced Edition
The Misty II Enhanced Edition has all the capabilities of the Misty II Standard Edition PLUS provides an enhanced mapping capability. It provides developers and their organizations with a professional-grade platform robot that can take on assignments requiring SLAM and would benefit from maps covering a larger space of ~1600-2000 sq ft. The Enhanced Edition’s greater mapping capability is thanks to its Open-Q™ 820Pro µSOM.
Misty II Enhanced Edition includes all of the capabilities of the Misty II Standard Edition PLUS…
Enhanced SLAM Mapping & Navigation
Enhanced SLAM in the Misty II Enhanced Edition allows you to map and navigate a much larger space than is possible with the Misty II Standard Edition. With Enhanced SLAM—made possible with the 820 Pro µSOM—you are able to map and navigate within a roughly 1600-2000 sq ft space. If your space is larger then you can start to tile maps similarly to what is done in some video games.
Make Misty More
Misty is not just an amazing piece of tech on her own—she’s designed for easy hardware modification with her many attachment points.
Backpack
Misty’s basic “backpack” allows expandability via USB and serial. The Misty (Arduino-Compatible) Backpack turns Misty into a robotic base for your hardware imagination (sold separately).
Customizable Arms
Misty’s emotive arms can be easily removed and replaced with creations you 3D print or build. In our lab we’ve created a drink holder arm, laser pointer arm, an arm with an integrated fingerprint sensor, and some students at the University of Denver even created a gripper arm that can reach out and grasp small objects. What will you create?
Headpiece Attachment
A magnetized connection point on the side of Misty’s head allows you to easily attach additional accessories. This could be anything from a thermal camera for doing temperature monitoring to a fun project to create an LED mohawk for Misty.
Trailer Hitch
Into the trailer lifestyle? Misty is too. Just hook her up via her built-in hitch port, and she’ll tow the (reasonably weighted) trailer of your choice.
Take Misty II to New Heights
Misty II has her own concept vehicle, the RoboChariot. Built as a rough proof of concept in the Misty Labs, it gives the benefit of increased robot height with her still being able to roam. She can take on a host of new assignments where roaming and increased visibility of her are desired. She could be your next retail store greeter, an office escort, a museum guide and more. You can build the Misty II RoboChariot* yourself, with our blueprints, or we can give you a hand.
*While this locomotion rig elevates Misty from her 14” stature to 54”, other locomotive options can be explored.
Features
Misty is packed with sophisticated hardware and software features that contribute to her ruggedness and extensibility as a platform.
Processors
- Qualcomm® SDA 820™ SOM (Basic, Standard)
- Qualcomm® SDA 820™ PRO SOM (Enhanced)
- Qualcomm® SDA 410™ SOM
Computer Vision
- Occipital Structure Core depth sensors for 3D maps (Standard, Enhanced)
- 166° diagonal field of view wide-angle Structure Core camera – 106° horizontal x 60° vertical (Standard, Enhanced)
- 4K camera
- Facial recognition
- Deep-learning AI using Qualcomm® Snapdragon™ Neural Processing Engine
Sound
- 3 far-field microphones using Qualcomm® Fluence™ PRO
- 2 high-fidelity speakers with engineered sound box and bass port
Touch
- 6 capacitive touch sensors on head and chin
Distance and Obstacle Detection
- 8 IR-based time-of-flight sensors (3 forward, 1 rear, 4 edge/downward)
- 10 bump sensors (3 tied in parallel on each front corner, 2 tied in parallel on each rear corner)
Movement
- Patent-pending 3-degree of freedom neck
- Easily customizable moving arms
- Sturdy track-driving tread system
- Trailer hitch to pull a payload
Display & Light
- 4” LCD image display/screen
- Bright LED flashlight
- Multi-color LED chest light
Connectivity
- 2.4 and 5 Ghz WiFi connection
- Bluetooth and Bluetooth Low Energy capabilities
Extensibility
- USB/Serial-connected backpack for hardware extension
- The Misty Backpack for Arduino (sold separately)
- Magnetic helmet connection point
Operating Systems
- Windows IoT Core™ (Main)
- Android™ 8 (Navigation/Computer Vision)
Development Tools
- JavaScript and C# SDKs
- REST, JavaScript, and Python remote command interfaces
- Perception Engine
- Documentation, sample code, and tutorials
Dimensions
