Request: musical feature data in lieu of audio capture

Following on from Save speech/voice into a local audio file

My understanding is that Jibo won’t deliver raw audio and video to third-party devs due to privacy concerns, and will instead make feature data available from the raw video and audio.

If this is the case, here’s my plug for those of us hoping to do music related skills that the feature data extracted from the raw audio feed include some features useful to music skills. Offhand, I can think of beat detection, spectrum data, and pitch/timing info as being useful. With a bit of research or other folks chiming in, I’m sure we could flesh this out a bit.

From what I’ve seen in various responses from @Admin.JiboSupport, likes on a feature request help the SDK prioritize feature development, so if this would be useful to you, please slap a like on it. And if you have other music-related features that would be handy to detect, add them to the discussion.

1 Like

I ran into this issue as well and was forced to use pre-recorded audio files to send back to the other STT system I was communicating with.

1 Like

Being able to extract features from audio would be a useful capability, but I don’t think it can replace capturing raw audio/video. What are the privacy concerns, out of curiosity?

That the always listening audio will somehow be monitored by unauthorized people; The Big Brother fear. AVS has this same restriction on audio stream access.

Curious, to Jibo Team; what about captured video? Do we have access to video that has audio inside it? How is that differentiated between just the raw audio? If I was writing a Karaoke skill, can I use Jibo’s camera’s to follow the signer and record video to store locally or offline?

1 Like

Seems like ASR suffers from the same privacy issue.

Hi everyone,

Thanks as always on the input on this. To answer @alfarmer’s question video does fall under similar concerns. We are looking into what we can do to give developers more freedom around visual/audio without impacting privacy concerns. As with all posts in this category I will be sure to communicate this feedback to the team so they can take it into direct consideration going forward!

2 Likes