Effective Testing Part II - Strategy

So in Part I, I mentioned that a good test suite has these three properties:

1. They should stop bugs.
2. They should execute quickly.
3. They should be easy to read and understand.

In the next part I want to show you how you can achieve the above with DSL-based functional testing. Part II, this part, is admittedly the least interesting of the series, because in some sense it only exists to justify deviating from the collective wisdom that is the Test Pyramid. If you are completely open to the idea of focusing engineering effort towards something closer to functional testing, and only selectively using unit tests, then you, my friend, can probably skip straight to Part III. It will be 20 years since I got my first paid developer job next month, and in that time I’ve encountered teams suffering from ineffective test strategies probably around half a dozen times. Often when I’ve suggested focusing test effort up the pyramid, following a ‘make them good, make them fast’ strategy, I encounter pushback. Even when developers admit that the tests are not great at stopping bugs, are hard to understand, and they have no idea how to improve the situation, the idea of deviating from the Test Pyramid seems fundamentally wrong, and that’s the end of the discussion. It is said that engineering involves trade-offs, and I certainly think better testing outcomes are worth the downside of deviating from established folk wisdom.

And again, as mentioned in Part I, this series is not meant to be exclusionary to other approaches, but is meant only to provide a vision or path forward for those who feel unhappy with their testing strategy and are exploring other paths forward.

The Test Pentagon

It’s not so much that I think the Test Pyramid is wrong, or that its proponents have missed something. It’s that for the most part, your average developer (even great ones¹) has probably read zero books on testing, watched zero videos on testing, and has about zero interest in testing. Most developers’ passions lie elsewhere, and so testing is often really only an afterthought and done in a perfunctory way. In my experience, most developers’ familiarity with testing strategy is just the following: write more unit tests than functional tests and write more functional tests than end-to-end tests. So they write tests going function-by-function and don’t invest in good test infrastructure, because whatever mocking library is available is the solution to every test problem, and they primarily measure success by code coverage, rather than defect reduction or understandability.

In 2016, when I first started going down the rabbit hole of testing, these perfunctory test suites were hard to understand, didn’t stop bugs and significantly slowed down software delivery. In 2026, even though development itself has changed, with agentic coding, the result is still perfunctory test suites that are hard to understand and don’t stop bugs. What agentic coding has changed is that instead of meaningfully slowing down software delivery (because of a need to refactor dozens of tests), you just spend more tokens to fix things. Another thing that has changed, critically, is that now it’s not just the tests – the system itself that is poorly understood. In my experience, even top-tier agents² seem to follow this pathological testing strategy, writing tests function-by-function and not investing in good test infrastructure, because whatever mocking library is available is the solution to every test problem, and they primarily measure success by code coverage, rather than defect reduction or understandability.

The traditional Test Pyramid (left) versus the Test Pentagon (right).

The testing strategy that I have had repeated success with is when tests are shaped not like a pyramid, but like a pentagon, with more effort on functional tests. I’m not the first person to propose this. For instance, some sleuthing online points to the Test Trophy, and the article has some nice quotes:

Write tests. Not too many. Mostly integration.

Guillermo Rauch

People love debating what percentage of which type of tests to write, but it’s a distraction. Nearly zero teams write expressive tests that establish clear boundaries, run quickly & reliably, and only fail for useful reasons. Focus on that instead.

Justin Sears

The more your tests resemble the way your software is used, the more confidence they can give you.

Kent C. Dodds

But to be fair both to other developers and to the LLMs, it’s not immediately obvious how one writes a great test suite. I mentioned that after reading xUnit Test Patterns, we very quickly hit a wall in being able to apply it and make our code base testable. Part of the reason I was motivated to write this series is actually because of my ongoing worries about software quality, and my desire to give guidance and communicate a vision to both other developers and AI agents, in both my personal and professional projects.

Functional Tests?

To borrow a quote from The Practical Test Pyramid, “If you ask three different people [what a unit test is], you’ll … receive four different … answers”. Different sources define unit tests differently, but common themes include being very fast, not having any dependencies (e.g., other classes, database, file system), being the lowest level of testing. Functional tests can be even more nebulous to define, but for some the term can mean integration tests, component tests, etc. I wouldn’t just call or recommend any assemblage or collection of non-mocked classes a (good) functional test. The term component test does a good job capturing the idea, and is a term I’ve used at work, but for this article I want to define the functional tests I’m advocating for as a shorthand for testing against an architectural boundary. Some examples of architectural boundaries:

• In a microservice architecture this could be tests written at the HTTP controller level, or even API tests actually executing HTTP calls.
• In a database, executing raw SQL.
• In pluggable architectures³, it’s testing against the plugin boundary.

Architectural boundaries are often the most stable pieces in the code, and are unlikely to change (e.g., it’s unlikely your HTTP-based microservice is going to get replaced with an OpenGL real-time strategy game), and consequently this makes tests written against it particularly stable and minimizes the need for changes unrelated to the feature. Architectural boundaries are often very well understood across your company or domain or perhaps even outside of it. If I wrote a test for Postgres that inserted 3 rows and then did a SELECT COUNT(*) FROM table, and verified the result was 3, that test would likely be understood by several orders of magnitude more people than individual unit tests validating the functionality involved in the call, function by function. This is important because if you are reviewing an MR or trying to understand why several tests have failed, jumping between different contexts or semantic layers is a challenge.

I was chatting recently with a very experienced architect about this topic and the Test Pentagon, and while they were skeptical of the idea, they did admit that for them now unit tests are just for the AI Agent, and they don’t even look at them, which doesn’t seem great. You should be able to understand the tests, and therefore get a lot of confidence from the level of tests where most of the effort is spent. On top of aiding understanding, testing along an architectural boundary also makes refactoring much safer, as the tests need far fewer changes as you evolve the system. In 2016, testing along an architectural boundary was what prevented your tests from slowing down software delivery, and in 2026 it means fewer tokens spent. My best experiences with agentic coding have been when the tests targeted an architectural boundary, the agent made significant rewrites to the code, and yet the change required near-zero changes to existing tests, and the new tests it wrote were easily understandable.

Drawbacks

Shifting towards more functional tests is not without its drawbacks, so let’s be transparent about them:

Defect Localization

If you are testing at an architectural boundary, and a test fails, you are now on your own to figure out why. Unit tests are, in theory, better suited for defect localization. However, in my experience this issue comes up rarely in practice, for a few reasons. First, the tests are breaking because of a change you made to the system, so you aren’t really starting from scratch to find the issue. Secondly, this argument assumes that for the most part your tests catch bugs, which I would say I’m very skeptical of in a mock-heavy test suite (the next example is equally applicable here).

Exceptional Edge Cases

Functional tests can make it much harder to test various edge cases, for instance trying to trigger specific database edge cases or thread interruption can be very difficult and involve a lot of heavy lifting. Occasionally you can architect your functional tests to support this, or alternatively at times I’ve relied on manual testing, the odd unit test for these, but often just decided that they couldn’t be tested. So while functional tests aren’t great for these kinds of things, I think a lot of unit tests for these kinds of things have far less bug-stopping power than you would expect. A situation I’ve run into somewhat regularly is ensuring that certain edge cases in the database are handled properly. Often this occurs when some unexpected edge case (say a transaction aborts earlier than expected) occurs in production causing issues, and you want to have a test in place that prevents this from happening now and in the future. But with unit tests, as defined above, you mock what you believe to be the behaviour, and if/when that is no longer true, the test doesn’t tell you. MySQL 8.0 changed the format of the error message that your code would need to scrape to determine which unique constraint was violated. The code that depended on that behaviour was broken by the upgrade, and the mocked test doesn’t catch the incompatibility.

Engineering Effort

As I alluded to in my previous post, testing is often not given enough attention by architects/senior engineers. You shouldn’t read this post as saying just get everyone to start writing functional tests, because they will be slow, brittle, and hard to read. Indeed, after having read xUnit Test Patterns we had a good grounding on what good tests looked like, but it wasn’t clear how to write either unit or functional tests for most of the code base that were consistent with the principles I outlined above. In particular, functional tests were painful to write and scale because the approach taken within each test was ad hoc, just ensuring enough of whatever system was in place to get the test and its true dependencies working. The solution ultimately came via another book, Domain Specific Languages by Fowler. By expending engineering effort to build a domain-specific language, you can ensure that tests remain simple to write, easy to read, while also providing a central and uniform way to bootstrap your system and make necessary performance improvements to keep the tests fast.

In Part III, I want to show exactly what these functional tests with domain-specific languages can look like, and hopefully provide you with a direction if you are willing to invest the engineering effort necessary. But I should stress that this will involve effort. A bad functional test suite can have worse outcomes than a bad unit test one. Finally, this article represents the vision behind the execution shown in the next part, an approach that has resulted in significantly better testing outcomes for me on several occasions. It’s certainly not the only way to do testing well.

1. I’ve worked with lots of great developers. I suspect I could count on one hand how many of them had read a book, or otherwise done a deep dive on testing :). ↩

2. A few of us on a team were curious about this on a greenfield code base, where we had let various LLMs loose to write tests over the past 6 months. We then had Claude Opus evaluate 50 meaningful mutations to the code (e.g., test bugs that we likely wanted or thought would be prevented by the tests), to see how many of these possible bugs were caught by tests. The answer was only 2 out of 50 bugs :(. ↩

3. By this I mean systems that define a plugin architecture: the platform outlines interfaces that plugins can implement, and then most actual user functionality is implemented in composable and swappable plugins. ↩