Software development, as a field, has seen rapid growth in the last few decades. Different paradigms have come and gone and changed the way developers build their software to be more robust and secure.
A notable example of a recent paradigm that has made a significant difference is the micro-services paradigm. In the past, companies built their project as one huge application; the monolith architecture.
They bundled the user interface and data logic in a single code-base to function as one system. While this approach has its benefits, micro-services came along and has long been identified as a better way of building flexible, robust systems.
Among other things, the micro-services paradigm greatly affects how tests are written as its usage introduces new layers of risk to the application as a whole; the business logic is no longer tightly coupled with the human interfaces (GUI) and is now nested deeply within the Application Programming Interface (API).
In this article, we will take a look at the repercussions of ineffective testing, we’ll see how API driven software development calls for the rebuilding of the test pyramid to cater for the changes introduced to the services layer by micro-services.
The Test Pyramid
A famous concept in software testing is the test pyramid. We are specifically interested in the agile test pyramid, it is a visual metaphor that hints on the different layers of automated tests to incorporate in your test suite:
The concept of a test pyramid was introduced by Mike Cohn in his book — Succeeding with Agile. As the image above shows, it consists of three layers that are usually followed from bottom to top:
Unit tests
Service tests
User interface tests
Most teams find it easy to structure their test suite against the test pyramid due to its simplicity. While this is a fair point, its overly simplistic nature can (especially with modern stacks) be misleading and unideal for testing applications that follow today’s software development patterns and paradigms.
The names of the layers in the test pyramid can also be somewhat misleading, for instance, the service tests layer is a term that’s hard to grasp, and for this reason, a lot of developers ignore it, introducing a potential gap in the suite.
The general idea of the pyramid can be summarized in the following points:
You should integrate only a few high-level tests.
Your test suite should comprise tests of different granularity.
Write lots of small and fast unit tests.
Teams still use the pyramid shape to contrive their test suites, but it should be noted that the resulting suite might not meet the effectiveness that teams aim for. Completely going by the original test suite might not maximize the full potential of your tests; costing your team a significant amount of time and creating an ineffective test suite that can be improved upon.
Maximize the potential of your test suite
The goal of software testing is to identify and eliminate bugs before they get to the production environment. Different layers of tests will scrutinize the different layers of an application, and it is possible to catch the same bug using different tests. However, tests must be written to be effective so that bugs are caught as soon as they appear because remediating a bug late in the test cycle will be a lot more expensive than remediating it at an early stage.
The optimizing of your test suite to effectively shift-left bug detection ensures that remediation is cheaper and easier to achieve. The bugs caught in the earliest parts of the test cycle may not cause any real harm and can be removed without major repercussions.
The chart below shows how the cost of remediating a bug grows exponentially along the cycle:
Our goal is to write modern, effective tests by reorganizing the need to cater for the modern paradigm and architecture that weren’t necessarily included in the original test pyramid.
Effectively rebuilding the Test Pyramid
It is common knowledge that UI tests can be brittle (and flaky) due to the limitations of the human interface. API tests, on the other hand, are more stable as they effectively test the business logic of the application, independent of the UI.
API tests give room for overall comprehensive testing of the application stack, rather than limiting tests to just the user interface.
Note: APIs are how applications communicate with each other using a common interface managed by a defined contract.
A good approach to writing more effective tests is to rebuild the test pyramid by reserving a space for automated API testing. That sounded easy, didn’t it? Well, some factors can make this modification a tad difficult to achieve:
Developers work from the bottom to the top of the pyramid and find it convenient to write unit tests since it’s their code. Creating meaningful API tests might be a harder task to accomplish, it might require knowledge about how other services interact and a more robust test environment
Test automation engineers (working from the top of the pyramid) might know which scenarios to test on the API level, but lack some of the knowledge about the internal component of the system, and might find it difficult to translate their ideas into code.
Like almost every other problem in the software world, this problem can easily be tackled by including the appropriate tools into your test suite. There are tools that can capture real-time user behaviour in a production environment and convert them to comprehensive API tests (GoReplay, Loadmill and Istio, just to name a few)
The benefits of such tools are endless as they play a vital role in ensuring the improvement of your test suite to effectively shift-left bug detection with API testing.
Conclusion
Although software teams make an effort to base their test suite on the diversity of various layers of tests, it can be observed that they still rely heavily on unit testing and convoluted last minute UI tests. This prevents them from building proper left-shifted test suites that are capable of catching bugs early in the cycle and saving cost.
API testing is gaining a lot of traction as modern applications run on the exchange of data over the REST interface. However, developers find it difficult to integrate API testing into their test suite because they may not possess the knowledge required to decide on what to test. Test engineers, on the other hand, might know what to test but be unfamiliar with API testing.
This article suggests the rebuilding of the test pyramid following the modern development paradigm and suggests tools that help with the integration process.