I expect that skills will listen for a common set of words/phrases and one idea is to process these short responses locally, rather than on the cloud. Already we have "hey jibo" processed locally.
The most obvious are the factory "word rules" such as the affirmation factory yes_no and variations we've seen discussed in the forum.
Likewise there are many possible candidate words such as commands and navigation related (north, south, east, west, left, right up down, next back, go, stop and all that..). Overall, these "words/short phrases" could be defined in special factories so that the usage is clear in the rule files.
As I design skills that expect the user to say navigation commands, afirmations (yes, no,... ), etc. , I realize that I have a higher expectation of responsiveness when using particular words.
Anyways, I assume it's not so simple, but just suggesting...