An Analysis of the Different “Test Shapes”

When entering the testing area, one of the first pieces of knowledge one should acquire is about the different test levels and types that exist.

One of the most recognized representations in the industry was created by the International Software Testing Qualifications Board (ISTQB), which defines the different test levels as component-level testing, integration-level testing, system-level testing, and acceptance-level testing.

However, the ISTQB is not the only entity to develop a model for separating these test levels.

One of the most recognized graphical models for visualizing these different test levels is the schematic model created by Mike Cohn, described in his book "Succeeding with Agile" as the "Test Pyramid."

Test Pyramid

Mike Cohn's "Test Pyramid" is divided into three layers, as visualized in the diagram above, which are:

Unit Tests: Tests applied to the smallest testable part of code, regardless of the integration they will undergo;

Service Tests (some authors refer to these as integration tests): Tests applied to verify the integration between unit tests;

UI Tests (some authors refer to these as E2E tests): Tests applied to check the behavior of the entire system or product.

This figure has the shape of a pyramid because, according to the established concept, the greatest effort from the test analyst should be placed at the lowest level of the pyramid—unit testing level, then service testing level, and finally interface testing level.

But why in this order?

According to this characteristic, the more tests established at the lowest test layer (unit tests), the more precise, simpler, and faster the tests will be.

Later, another author, Martin Fowler, deduced that the closer you are to the base of the pyramid, the faster and cheaper your tests will be (similar to the "shift-left" deduction, where the test analyst should focus efforts at the beginning of the software development cycle rather than at the end).

The test pyramid is one of the "geometric shapes" of tests established.

Inverted Test Pyramid

Another shape that emerged is the "Inverted Test Pyramid," also called the "Test Cone" or "Test Ice Cream Cone"

This inverted pyramid is essentially the opposite format proposed by the test pyramid. That is, when using the "Ice Cream Cone of Tests" within a testing team, the greatest effort is spent on the layer closest to production release, rather than being spent on the most basic layers.

Using this inverted pyramid within a team demonstrates its immaturity in agile concepts, resulting in higher costs and slower execution of tasks.

Test Trophy

Another geometric shape (or set of shapes) developed was the "Test Trophy," by Kent C. Dodds, whose intention is to demonstrate that, depending on the software being analyzed, testing efforts should not primarily be on unit tests, but rather on integration tests.

Where "Static" could be a static analysis tool, such as ESLint, used for code standardization, for instance.

In the words of Kent C. Dodds:

“The test pyramid is based on the assumption that integration tests are expensive, slow, and fragile compared to more focused tests like unit tests. While this is generally true, there are exceptions. If my high-level tests are fast, reliable, and inexpensive to modify, then low-level tests aren't necessary.”

In other words, the author claims that there are exceptions regarding which test layer should be prioritized most: unit or integration.

Testing Honeycomb

Finally, the last geometric shape worth mentioning is the "Testing Honeycomb," which can be translated as "Colmeia de Testes" or inferred as "Diamond Testing."

The "Testing Honeycomb" is a graphical representation created by the quality engineering team at Spotify.

It can be observed that the "Testing Honeycomb" brings a similar proposal to the "Test Trophy," placing more effort on the integration layer than the unit testing layer.

This occurs because the product Spotify's team works on has strong associations with various other microservices—a set of independent services integrated within a scope.

In Spotify's own quality team's perspective:

Microservice has become our new "unit," and that's why we avoid the term "unit tests" for microservices in favor of "Implementation Detail Tests." Microservices take the old idea of isolated components and show us what the abstractions should be.

Conclusion

Although there are different authors and teams creating various geometric shapes to graphically visualize testing, a similarity can be identified among them—they diverge on where effort and focus should primarily be (unit or services), and always agree that E2E tests should require less effort.

Basically, the largest layer of your tests (either unit or services) will depend on the product's focus, particularly if it involves many third-party microservices (like Spotify) or not.

However, E2E tests should always have the least effort within your test suite; otherwise, the testing process will always be slower and more costly to maintain.