Speech recognition: non-verbal utterances (laughter,yawning,and other social contagions)

I’d like Jibo to react to various non-verbal utterances or social contagions… even to be affected by social contagions (laughter, yawning) or even become the source of the social contagion.

Reacting to laughter or creating laughter would support reciprocity with users and enable our skills to react to this aspect …Jibo skill might even try to elicit laughter (behaviour success if laugher detected;) ) or even yawn at bedtime :wink:

++ rules: we can detect laughter, and perhaps the quality and degree. Quality in this case relates to a genuine laugh, a groan (see Chris’s knock knock joke - Jibo groans sometime in response to a joke), or a bit of sarcasm when the joke or response can be taken seriously - “ha ha ha”.
++rules: we can detect Yawning, “shhhh” and so on…

Jibo as the source of a social contagion
++ get people laughing - we need some laughter asset options :wink: Jibo with a laughing fit (laugh, animate,etc) might light up a whole room of people (social contagion)
++ Jibo starts yawning - a skill might use that around the kids near bedtime :wink:

Other
++detect “shhhh” or shhh Jibo - a way to pause Jibo - wait unit
++detect “pssst” or psst Jibo - get Jibo’s attention (or Jibo says “pssst Sally” to get Sally’s attention in a fun way!
well, endless really, … I’m sure the speech team is having a grand time :wink:

I’m looking forward to learning about the support of laughter, yawning, shhh …for starters, and other non-verbal utterances that you all are considering.
best, Bob

3 Likes

If Jibo could detect laughs and groans, I could use it as a cue to which jokes were actually funny, and rate the jokes accordingly. That’s my joke case, but it could be extended to other things. Generally reading the response of the content and possibly adjusting accordingly.

Detecting yawns might also be a way of telling that something is not interesting to the user, and maybe offer to quit the skill early.

Startling might be interesting, too. Like if there’s a sudden loud noise somewhere, and Jibo does a little “jump”. OR if a user does a little bit of a start, maybe they were just reminded of something, or thought of something new, and Jibo could offer to pause and take a note or reminder. Maybe they just found that particular item to be particularly interesting and mark it as such.

And of course, on top of it all, just seeming more of a personality to interact with, more of a companion.