Stratified mix of automated and functional testing
Our approach to improving QA is a stratified mix of automated and functional testing. Having close to 100% code coverage by automated tests is hard to achieve and harder to manage, it is important to note that automated tests themselves do not guarantee any sort of regression mitigation, they simply test for predefined conditions based on a given data set. In order for those tests to be accurate, they need to cover a sufficient number of conditions and have an adequate dataset, which is a project in itself.
We always start with a functional test approach by creating a sanity check document, where we document all features of the application as well as provide notes as to how to perform each test, what the expected result would be, last known status of the feature (whether the feature is passing or failing the test), and the last date the feature was tested. This allows me to have a base testing strategy of the entire application and usually can be done in a matter of hours.
After having the sanity check document, we identify key areas that benefit from automated testing such as:
- Simple features
- Areas susceptible to regression faults
- Areas that have been problematic in the past
- Areas with tricky logic.
We’d write integration or unit tests around those depending on circumstances.
Next, we implement third party UI testing tools such as LambdaTest that perform testing and collect data on UI elements and views that I can later review for any responsiveness or other UX issues.
Additionally , the deployment steps where we aim to ensure that database schema migrates correctly, that dependencies are installable, as well as all other deployment steps behave as expected. This helps avoid deployment issues in production.
Finally, we use code quality tools to be sure that the code is standard compliant. Coding standards exist for a reason, and complying with them has numerous benefits such as improved maintainability, improved security, improved stability and last but not least, it helps other tools understand the source code better and assist engineers. An example of this would be the IDE, having standard compliant code would allow the IDE to perform real time checks and assist with code hinting and other suggestions.
