Case Study: Saving Remote Test Results in SDK
We needed a way to save test results for future analysis + we wanted to explore our own API library while away on a recent team PBH (product building holiday, of course...)
In the past, we've been able to see 'real time' test outcomes via terminal, but there is no way to go back in time to see test results and determine: which ones passed? which ones failed? when was the last time a test passed? how have test results changed over time? etc.
PBH Solution -
Using a generic dataset, we were able to successfully run a feature test and confirm the data saves to Secure Data Kit (SDK), therefore creating a 'test outcome log'.
Here's how it works:
And here's why it's awesome:
1. With this tool, we took advantage of the SDK API by adapting a separate test suite for a completely different project and saving its results to an SDK dataset. By doing so, it shows how any app using any language can do the same thing - essentially, we can remotely execute tests in whatever framework we want and remotely add the results to an SDK dataset!
2. Another advantage to this is we can report on 'pass/fail rates' or 'date last passed' for all of our projects, plugins, clients, etc. in a single dataset, which allows us to gather insights over a period of time (no matter the framework). Because code bases change over time and tests can start failing, if unmaintained, a log of test results is incredibly valuable for analysis + understanding the clear picture.
Sep. 14 2018