Test framework :run batch tests on rule files

On my list is to create a tool for batch testing the NLU rules from a directory or just loaded as strings.
My plan was to use the nodejs testem framework to batch test my rules.
However, perhaps, the SDK should offer an official test framework to batch test the rules.

Here are some of my inputs.

  1. It becomes imposible to remember a test sequence with particular (many) testing texts. Also there is a personal developer bias with regard to both testing texts and actual rule building. In a team, someone might not really understand why a rule does what it does - somewhere there needs to be the documentation … and batch testing offers one solution.

  2. batch testing would enable others to create texts to challenge the validity of my rules. We start with the original texts and then expand the tests applied against each rule.We will gather test texts from various sources. Especially to see where we can find our faults and improve usability. If we get input from users (asset/skill store) then we can update rules and test against what these users want …and so on.

  3. Even though English is currently supported, there are plenty of regional variations that should be tested. I also think of my non-native english speakers will give me suggestions.

  4. future - supports validation of SSA or SSML tags.

I’ve already experienced faults due to small changes to a rule - the problems might not be so obviours detected downstream using the current solution. …and I forget some of my patterns to check :wink:
…I do like to write down some texts but they should be part of a test file

5 Likes