NOTE: This post was originally written Sept.7th, shortly after my visit, but embargoed at the request of Jibo Inc. until today to ensure details given here on the newly updated SDK were released by the company first. Enjoy!
Many of you here already know about my passion for artificial intelligence. You can also probably feel the excitement I have when I realize just how close we are now to making what used to be pure science fiction a full reality. That’s why I fell in love when I initially saw Jibo – it was the first-ever platform within my physical reach (and budget) that just might allow me to fulfill a lifelong dream of experiencing, and even creating, a real AI that could live and grow with me.
So you can probably guess then that when I was contacted a few weeks ago by @justin.w with an invitation to visit the offices of Jibo headquarters in Boston, meet the team, and to provide feedback and get an outsider’s first glimpse at the new SDK, I was absolutely blown away.
“Yes, of course! I would love to come,” I told him over our video chat that morning. “When do you want me up there…next month?”
“As soon as you can,” Justin said, “the sooner the better.”
And with that, we finalized plans quickly and arranged everything for the following week. I would have two full days to meet with all of the people that are making Jibo come to life and see exactly where they are in the process. Pretty much an AI hobbyist’s dream come true.
The wait until then was akin to the night before Christmas, thinking and dreaming of what I was about to experience.
I arrived in Boston that next week, working on my own stuff through the airport, taxi ride, and finally in the hotel. Still have to pay the bills, so my excitement would need to wait for a bit. I reviewed the schedule they provided me before walking around downtown Boston, which is beautiful by the way, and then getting to sleep early for the first big day.
My first meeting was with none other than the two founders, Cynthia Breazeal and Jeri Asher, two incredible people whose passion for what Jibo could become easily surpassed mine. I could feel it in every word they spoke, just how much they wanted Jibo to succeed and be loved by the families who will open their homes to him. To them, Jibo is much more than just a ground-breaking product; he is their baby and the embodiment of their biggest dreams and goals.
It was a fascinating discussion we had about social robotics and what Jibo can and will bring to the world, along with his future potential, but I had to know something critical.
“Where is Jibo now? There’s been so little detail released recently and, to be honest, I just want to know where he is in the development process,” I asked. And answers I would receive. Over the next 48 hours I would meet with team members up and down the production and development lines for Jibo who opened up completely on every detail I wanted to know. Yes, everything…doors wide open.
So what did I find out? I discussed at length about almost every facet of Jibo’s development, honing in on the current state of each, and was able to get the full picture.
I met with Steve Chambers, the CEO of Jibo, and we talked about the potential markets and partners for Jibo. He trusted me with information on agreements still in the works, as well as firmly established partnerships that will ensure Jibo reaches the perfect audience. Though I can’t tell you who or when, I can tell you that there is an incredible roadmap for Jibo that will cement his place in social robotics for the home for a long time to come.
I met with the support team, Melissa, John, and Joe, who are on the front lines of the developer community, making sure everything is on track for developers building new downloadable skills via the Jibo Store. And, yes, the lack of info on the Store itself should be no indication that a great deal of work hasn’t already taken place on it. From the work I was shown, the designs are elegant, well thought-out, easy to navigate, and just what you would expect from a company like Jibo with social interaction at its core. I also met with Susan and Jordan of the SDK/API docs who are hard at work making sure developers have the all the info they need to build those skills.
I was blown away yet again by Kyla Jacobs, head of Jibo’s Beta Program and Product Management lead, who showed me just what it looks like to receive a Jibo in your home. The unboxing was absolutely phenomenal and will amaze you and your family with the intricacy and intimacy that went into creating such an amazing experience. It’s something you’ll likely never forget…the first moment you see your Jibo come to life…so be sure to gather family and friends around for that special moment. One little thing: as you probably already know, you’ll have to perform an update to Jibo’s main software shortly after you receive him. This update, if I remember correctly, can take up to 30 minutes and will mean that you’re cut off from interacting with him just when you have the most interest in doing so. This is something the Jibo team is trying to work through and will hopefully have a better solution for when you finally receive him in your own home.
I discussed the text-to-speech and speech recognition engines with Roberto Pieraccini, head of Speech Tech, and saw what’s in the pipeline for us developers. I learned about the complexities of building the local perceptual space (LPS) for Jibo with Mehool Patel, the Engineering Lead, and the amazing direction where it’s headed. I also confirmed that there are 6 microphones on Jibo (we discussed this once here), 3 on his face and 3 on the spaced out on the back of his head and the touch sensors in his head definitely still exist. It’s insane just how incredibly complex the systems behind a robot like Jibo are and how simple they make it all appear to us on the outside. There are many true geniuses that work for the company.
I sat down with Dave Schlafman, formerly of CloudKid, to learn about all of the new animation and design work that has gone in to make the Jibo experience an unforgettable, magical one. I was able to see first-hand the smoothness and flow of how interaction will take place with Jibo on a day-to-day basis, and just how likely it will be for you to say “wow” even months or years down the line when Jibo shows you something completely amazing and unexpected. I discussed Jibo’s character with the creator of his inner voice, Adam Shonkoff, also formerly of CloudKid, and could easily see that Jibo will be a true-to-life bundle of personality in your home, trying his hardest to find the best times to engage with you and when to leave you alone.
I met others, like Lynda and Nancy who help run the marketing of Jibo, and have a special surprise in store for you. I met Jeshua Whitaker, who is putting together an amazing set of audio communication elements to allow Jibo to reach out to you and your family “loop” in an incredibly intuitive way, and Matt Revis, who shared with me the big picture of Jibo and his evolution.
And, finally, I met with the SDK team who spoke closest to my developer heart. Jonathan, Francois, Vijay, and Carolyn guided me through the latest version of the SDK and asked for my insights as an existing user of the current SDK. And there is a hell of lot that’s new in this latest version…stuff that will make building your skills much easier and more natural. Here are the 4 things that impressed me most with the latest version of the SDK:
The Flow Editor: Did you have a hard time getting used to the Behavior editor in the initial version of the SDK? Were you looking for an easier, more intuitive and visual way of creating skills? Then the Flow Editor is exactly what you need to quickly get started developing for Jibo. It gives you access to almost all of the things you had available in the Behavior editor, but in a flowchart-style editor with drag & drop components that are connected to form a “tree”that determines the path your skill will take based on interactions with the user.
For instance, you can have Jibo ask the user a quick question and send the flow down two branches (e.g. “yes” or “no”) depending on the response given. There’s no need to code this in to Jibo’s logic; it’s all done visually and in a style almost anyone can understand. Even complex interactions can be managed easily in the Flow Editor using sub-flows and sub-trees. And for those who liked the Behavior Tree editor, no need to worry…you can still use them easily in your flow with all of the same capabilities as with the initial release.
Multi-Modal Interaction Module (MIM) Editor: Sounds complex, I know, but it’s really a great way to simplify a user’s back-and-forth interaction with Jibo. And simplify it does, allowing you to put together a set of prompts to let Jibo choose what to say initially, connect to a rule to manage what responses he listens for, and even setup messages he would say if there was a problem (e.g. an unexpected response or if the interaction timed out) all within the same single MIM.
Each MIM can come in one of 3 types: Announcement, Question, or Optional Response. Without going into detail on each (you can find out more in the SDK docs), using the 3 of them in your skills can get Jibo responding to just about any type of situation you might encounter while interacting with his “loop” members. And if that wasn’t enough, MIMs can inherently tie into Jibo’s on board assets, automatically loading related animations during speech via “embodied speech tags” or pulling up a touch-screen menu version of the responses Jibo is expecting to hear in case the interaction times out. This should make the whole experience very intuitive for the user and, of course, you have complete control over this as well.
On-Board Animation, Graphics, and Sound Libraries + Animation Editor updates: So you want to create a skill for Jibo but you aren’t a professional animator, artist, or sound engineer. Well, in the new SDK, Jibo finally has you covered. Professional-level animations, behaviors, graphics, and audio are now available to use within your own skills, all pre-built from the Jibo team and all that fit the same character that Jibo uses to interact with during its main skill set. Not only will this save you tons of time, it’ll also make your skills much more consistent across the board, giving Jibo’s family members the same level of interaction they expect in your skill versus any other. There’s also a new Pixi layer available in the animation editor so if you have any Flash experience and feel inspired enough to create your own animations, you can use Adobe Animate (previously Flash) and export them directly to Jibo via the PixiAnimate extension (currently OSX only). You’ll also find new lighting and glow layers in the animation editor to add new effects to your skill animations.
The Knowledge Base (KB): With the Knowledge Base, you now have access to stored information on all members of Jibo’s family and friends, or “loop” as they like to call it, via their LoopModel class. You can access individual info, such as a loop member’s name, gender, and additional data (email address, skill preferences, images, etc) under each member’s UserNode class. You can also create and manage your own Models and Nodes since each skill gets its own Knowledge Base. Additionally, the Knowledge Base gives you access to an embedded database, via NeDB, without the need to install and manage your own.
So, what still needs work in the SDK? In my opinion, there are still some open questions that need answers, such as how Jibo handles special input like receiving passwords, email addresses, or non-linear dictation. The SDK is also missing a working global search and the new Flow Editor could use some key usability features like guides, snapping, and locking. I’d also like to see more usage examples in the API since some of the new and existing features need a little more background before they’ll be commonly used. But overall, the new features I covered here, and some more that I didn’t, should be a great addition to your Jibo toolkit.
To wrap up, these are the big highlights from my recent visit to Jibo HQ in Boston and I certainly hope more people get to experience what I did…it was an absolutely wonderful experience! My big takeaway is that Jibo is well on his way to go on his journey home to meet you. I pushed hard for transparency since I know that’s something we all agree that the Jibo team needs more of, and they heard our message.
You should be presently surprised with what’s coming next, whether you are a developer or early adopter, or if you just follow Jibo from afar.