After spending two decades as the CEO and CTO for various companies developing consumer robotics products, including the Roomba vacuum cleaner, Paolo Pirjanian decided he wanted to elevate robotic technology to a more impactful level for society. He left his job to explore how to fulfill his lifelong dream of building robots that can care for and enhance the lives for people in need.
Originally, he was interested in helping “aging in place” for the elderly population that can at times be socially isolated and who may need companionship, motivation to stay active and stay socially contented with family members. But then he spoke with a couple of families he knew that have children who are on the spectrum. “I started to get a better sense for how hard and devastating it is for these families to provide the necessary care and developmental skills for a child that has been diagnosed with a disorder,” said Pirjanian. After deeper research, he found the recent studies that showed many children, including neurotypical ones, are falling behind on their social, emotional, and cognitive developmental targets. According to the CDC, anxiety, depression, and other mental disorders among children are on the rise which is partially attributed to screen time and social media.
Pirjanian founded Embodied in 2016 and started to assemble a world-class team of experts in child development, engineering, technology, game design, and entertainment. The result is Moxie – a robot that helps promote social, emotional, and cognitive learning for children.
Research shows strong scientific evidence that robots can help children improve their learning abilities and social skills. When looking at task performance and user experience across 65 studies comparing embodied agents (robots) vs. non-embodied agents, robots significantly outperformed non-embodied agents in 78.5% of the studies.
“As we started to pull all the pieces together, we realized this project was a moonshot,” said Pirjanian. “In terms of developing an animate product that could be described as an imaginary friend, we had to push the frontier of technology in so many areas. While daunting and a bit crazy, we went for it because we believed it could make a huge impact on the most vulnerable members of our society.”
“It not only helps children diagnosed with a disorder. Every child can benefit from Moxie. There are a lot of solutions for STEM education and IQ-related products, but not many that help children with social, emotional, and cognitive development. Studies indicate EQ is as important, if not more important, than IQ in terms of happiness and success in life.”
– Paolo Pirjanian, CEO and Founder, Embodied
The biggest technical challenge Embodied had to tackle was developing the interaction component between human and robot. “To create meaningful interactions and engaging conversations with a robotic device, the sense and react loop is the major factor,” said Stefan Scherer, CTO of Embodied. “We needed to make this loop as fast and snappy as possible to support rapport-building between the child and the robot.”
After conducting extensive research and development, Scherer and his team developed a new platform, SocialX™, that provides a way to use natural human interaction to engage with a robot by combining technology that lives on the edge (the robot) with technology based in the Cloud. This allows Moxie to perceive, process, and respond to natural conversation, eye contact, facial expressions as well as recall people, places, and things with incredible speed and accuracy.
Craig Allen, Embodied’s Chief Creative Officer, incorporated his previous experience in video gaming to create a character that portrays the illusion of life. “We needed Moxie to be emotionally real to a child so that it could successfully serve as a springboard to developing life skills beyond academics,” said Allen. “Moxie is really in the unique position to basically be a direct eyewitness into a child’s abilities, needs, and behaviors. These interactions can inform parents, or perhaps help therapists, understand how a child is progressing and make informed assessments by utilizing data analytics.”
“We have an opportunity to start a different kind of relationship between technology and people,” said Allen. “Most of our technology revolves around it telling us what we ask for. At Embodied, we’ve flipped that to explore and come to the right conclusion as to what our actual needs are and then provide the necessary tools to become better people. To me, that’s really what Moxie’s all about.”
Partnership & solutions
Embodied evaluated several different technology companies to achieve their goal of creating a robot with the ability to have fluent conversation back and forth at an affordable price point. “We selected Google Cloud Platform and it has played a very important enabling part in building this system,” said Pirjanian. “We leverage our entire cloud backend on GCP.”
The robot’s operating system is Android. “The significance of this cannot be underestimated,” said Pirjanian. “We are doing a ton of processing embedded on the robot. Thanks to Google’s ecosystem, enabled by Android and TensorFlow, we are able to run super sophisticated AI algorithms locally on a computer inside the robot itself – volumes you could previously only see in robots that literally cost $30,000 – $40,000.”
The main technology offering that has been industry-defining and a game-changer for Moxie is GCP’s Speech-to-Text API. “It is incredibly accurate and allows us to do the fast turnarounds with speech in and text back. With incremental speech recognition functionality, you can upload data and get results back while you’re waiting for the speech to end. That has been phenomenally successful.”Stefan Scherer, CTO, Embodied
Embodied has taken careful steps to ensure that information provided by children and families is handled with high standards of privacy and security. Some information, such as video data from the robot’s forehead camera, is processed on the robot and never leaves it. The audio communication that enters and exits the robot is completely secured and encrypted by default using Google Cloud’s 256-bit encryption. In addition, the child’s personally identifiable information (PII) is encrypted and can be decrypted by a unique key that only the parent can access.
Leveraging Google Cloud’s experimental and cutting-edge automatic speech recognition (ASR) models, Embodied provides customizable input with predetermined context words that are likely to be used during specific themed interactions with Moxie. During post-processing while the robot “sleeps” it crunches the data from the day’s interactions to create child development insights for the parents. To serve parents with such insights, Embodied leverages Google Cloud Storage & Computing signal processing backend services and its robust machine learning algorithms. The fully encrypted and anonymized data is sent to an app that parents use to monitor their child’s progress on various tasks and review metrics such as reading comprehension and language use.
To help ensure that their team is using GCP to its fullest potential, Embodied partnered with SADA, a Google Cloud Premier Partner and 2019 and 2018 Google Cloud Global Partner of the Year. A leader in providing business and technology consulting services that transform organizations by leveraging the entire Google Cloud portfolio, SADA is supporting Embodied in fulfilling their ongoing technical backend requirements as they continue to optimize their GCP solution. By partnering with SADA, Embodied has access to:
- Solution Architects that provide guidance, consultation, and best practices for GCP
- Customized billing dashboards that give insight into usage that’s not available from the standard console
- 24/7 personalized support
- Technical Account Management (TAM) services to assist in scale and cost optimization
“We have been super excited about our interactions with SADA,” said Pirjanian. “I know this relationship will prove to be very beneficial for us as we grow our company.”
GCP allowed Embodied to overcome several technology challenges while developing Moxie:
- Cost Savings – AI algorithm processing embedded on the robot increases speed and reduces expense
- Realistic Interactions – ASR powered by deep learning neural networking produces accurate and engaging conversations
- EQ Assessment – Machine learning and data analytics provide key metrics to measure cognitive, social, and emotional developmental goals
- Security & Privacy – Google Cloud’s industry-leading security capabilities protect information, identities, applications, and devices
Moxie launched in the fall of 2020 and can be ordered at embodied.com. Included in the purchase, customers will have access to over-the-air software updates for new features and content, monthly Moxie Mission Pack mailings, special access to the Global Robotics Laboratory (G.R.L.) child portal site, and behavioral analytics via the Embodied Moxie parent app.
With Moxie, children can engage in meaningful play, every day, with content informed by the best practices in child development and early childhood education. Every week explores a different theme such as kindness, friendship, empathy, or respect, and children are tasked to help Moxie with missions that explore human experiences, ideas, and life skills.
“The robot encourages children to go out and practice things in the real world and report back, because that’s where we want them to succeed.”
– Paolo Pirjanian, CEO and Founder, Embodied
These missions include creative unstructured play like drawing, meditation, mindfulness practice through breathing exercises, reading with Moxie, and exploring ways to be kind to others. All of these activities help children learn and safely practice essential life skills such as turn-taking, eye contact, active listening, emotion regulation, empathy, relationship management, and problem solving. Moxie encourages curiosity so children discover the world and people around them.