Connectivity: skill with server features (websockets, udp, http) or WebRTC

####1. Describe Your Issue

While the forum was down, I asked some connectivity questions by email. I want to follow-up with more specifics and get the topic and answers for others to see. From this perhaps documentation updates and feature requests could follow.

My interest is that clients can connect to a skill that provides server capabilities e.g. websockets, udp, http. Obviously provides another way for users to interact with Jibo. I’m particularly interested in devices in the local network.

In the simulator, I can run a simple chat demo using npm socket.io and http (via express for now). The Jibo skill listens on port 3000, my browser connects to the simulator skill and everything works fine. I have game and story feature ideas that include chat.

Question 1: Now on the real robot, would this chat skill work where the skill provides a websocket server?
If not, then it would be useful if these restrictions were documented and the simulator changed to reflect these connectivity restrictions.

I’m also looking at WebRTC - same issue.
Question 2: can we use webrtc on the actual robot? The chromium browser is likely ok, not sure about the restrictions.

This might be one way that another device captures audio/video without breaking any Jibo privacy policy - that’s handled by the other device. However, right now I’m more interested in the data channel. Again, I’m only considering the local network, no ice server needed for signaling.

best, Bob

2 Likes

Hello Bob,

Thank you for your patience over the holiday weekend!

We chatted briefly about this via email and I am glad to see that you posted it here so the community can see it as well. I did some further talking with our team and I wanted to provide some feedback on these questions.

Ultimately we expect that developers will be able to access Jibo’s services through dedicated behaviors in the SDK as opposed to accessing directly via something like webRTC.

We do intend to allow outside services to make a connection with a skill but we are further defining the finer points of this and we will provide further information in this thread as it becomes available. In the meantime it would probably be best not to assume further functionality.

All my best,
JohnW

1 Like

Hi, I have a same idea, but I run a simple http server (via express) and create websocket server using socket.io, but not running in jibo.
In Jibo I using socket.io connect my server, and tip connected. But Jibo can’t be received any message from server.
I don’t understand why?

Hello Madong,

Thank you for your input on this topic!

Since Bob’s original post we have had some internal developers do some prototyping that explored managing web sockets/web services in this way. The main component that they found usefl was an npm package called ws.js (https://www.npmjs.com/package/ws.js) that is a WS-* client stack for node.

As I mentioned to Bob, we have still not defined a specific method for accomplishing this that we guarantee will be supported in Jibo’s skills on release. With that in mind, using a node package or database similar to ws.js would likely be your best option if you would like to explore that further in the meantime.

Thank you!
-John W

1 Like

thank you , John.w
I had fix it using socket.io.:grin:

Hi @madong, Thank you for bringing this up, creating a simple webserver and connecting to Jibo using socket.io is really really cool! Congratulations! Do you happen to have the code on github? I’d love to tweet it out or feature this on a blogpost.

I’d also love to hear more about what you are building and tips and tricks you’ve learnt along the way. As always don’t hesitate to contact us if you have any questions.