My Test-Driven Design (TDD) Experiment

Test Driven Design

Photo by Helloquence on Unsplash

Is a Test-Driven Design approach effective, even when working on a medium scale personal project? Yes, and here’s why.

I’ve long been an advocate of Unit Testing in software development, having found that even though the work is often tedious, the investment in good tests significantly reduces overall development time. In my experience, the time invested in unit testing pays back by a factor of ten or more.

For my latest as-yet-unreleased project, I decided to try a Test-Driven Design approach. In short, the TDD method requires that you write tests for your code before writing the code. This has some significant benefits from a design/architecture perspective:

  • It forces the developer to think more about the external interface for a package and less about the implementation details.
  • It requires a more precise definition of what the inputs and outputs of a functional block are. This can expose flaws in the overall architecture. In the case of my project, I was working on the principle that the module would take one input file and generate an output. While creating the third test case, I realized that I was duplicating content from one input file to the next, that a far better approach would be to break that information into a second, common file that could be used multiple times.
  • It assures near 100% code coverage for tests. In theory it guarantees 100% code coverage. In practice that’s more difficult. More on that later.
  • Writing a test for everything forces you to to become a user of your own code. This serves to highlight problems early, before they become deeply embedded in code. When designing something, it’s easy to oversimplify. The only easy part of the “that part will be easy” trap is falling into it. In this project my “easy” trap required no less than three refactorings and a partial re-write before it was actually easy for the user. If I was coding first and testing later, backing out of the initial design would have been much more difficult and required throwing out a significant amount of code. Code is time.
  • It reduces useless code. I have a distaste for missing functionality. This means that when I write a method to do something, I’m inclined to generate similar methods that do parallel things at the same time. TDD puts an end to that.
  • It highlights common functionality that may not have been evident in the requirements phase. This makes it easier to spin out that functionality into independent classes early in the implementation.

Software Engineering Purity and Test Structure

Although I’m a strong advocate of automated testing, it turns out I’m far from a purist. Even though there are some clear benefits, I’m not likely to build a suite of mock classes just to isolate a subject class. 100% coverage is always a good thing, but I’ll take 80% coverage, knowing that some abstract class I don’t have an explicit test for got a pretty thorough run through by other tests, metrics be damned.

If tests require some expensive operation to complete, such as accessing a database, then creating a data provider is worth the effort, but doing so purely for the sake of correctness? Not so much.

It’s a little too easy to write code for something obvious, like checking for a null value, without writing a corresponding test. Sticking to Test-Driven Design requires a level of discipline that’s difficult to maintain, particularly when you’re the only developer on a project. Because of this my test suites tend to be a mix of unit tests, integrated tests, and some kind of ugly hybrid between the two, and I’ve decided that I don’t really care. Generating sample output in a bastardized test that ends with assertTrue(true) is still useful. Even though that test always passes, every once in a while the “real world example” test exercises an unexpected pathway and throws an error that would otherwise sneak by. I’ll take that find over purism ten times out of ten.

TDD and Over-engineering

I’m also more relaxed about software engineering principles when it comes to testing. I’m more likely to copy and paste test code than I am to carefully craft a hierarchy of test classes. I may be more relaxed but I’m not lax… as it turns out this project has a bunch of test cases that are common across multiple classes. I initially just copied the first class, with all the cases, which are fairly elaborate. Then I needed to copy them again for a third class. Three is the magic number when you realize you just screwed up. It took a fair bit of effort to go back and decouple the test case generation from the expected results, but there is an immediate payback: tests that exercise more sophisticated features in the latter classes are now automatically passed to their predecessors. If a simpler class can’t handle the new case in a reasonable way, it’s evident immediately.

While I have clearly strayed form a pure Test-Driven Design methodology, starting out with TDD gave my project an obvious lift.

Reducing gold plating and improving the design

On more than one occasion I found that I was inclined to embellish code with things like getters and setters that looked useful but actually had no use case. All the “I should add this” moments were converted to “I need to write a test for this first” and it’s not long before you realize that you don’t need a test because nothing will ever need to use the method in this way. Better yet, it makes you think about how that task would ever be used. The end result of this was twofold.

First, entirely removing functionality that seemed like a good idea early in the design process but in reality was just useless baggage. Second, if the functionality was useful, it was usually in the wrong place. This led to a series of code refactors that extracted that functionality into a conceptually clean solution, either a trait or a stand-alone class, in either case useful in many places. Less code or better code. Both excellent benefits of starting with TDD.

TDD Offers Measurable Progress

When working on a project of significant complexity, particularly when working alone, it’s easy to lose track of where you are. That’s not a problem if you have a client deadline looming in the not too distant future, but when it’s a personal project, and particularly if you’re blessed with a healthy dose of ADD, it’s easy to lose momentum.

For me, loss of momentum is the death of a project. I’ve got a long list of unfinished projects that I thought would take a few weeks when I started, but in fact they needed many months. Nearly all of them died from a momentum deficit.

Test-Driven design, with it’s focus on top level functionality, really helps with that. Even though my current project is perhaps 50% complete, it’s generating useful results. The implementation is partial, but it’s functional, and needless to say it’s tested, well structured, and robust. Instead of substantially complete code that winds up with significant problems when put to actual use, I have working but functionally incomplete code that I expect will be a joy to keep working on. All of these things are giving me enthusiasm for implementing the next level of features.

Conclusions

Even though I have strayed form the TDD methodology as the project progressed, starting with TDD was the best thing I’ve ever done when working on something of significant size.

If I was working in a team, not only would I start with TDD, I’d be far more strict about sticking to it throughout the development cycle. It highlights architectural issues very early in the development process, when it’s far easier to adjust and fix the problem. A dozen minor restructurings when the code base is small is a thousand times easier than rewriting thousands of lines of code after the mistakes have been baked into the project.

It’s hardly an objective measure, but I think this code represents some of the best work I’ve done. My three week project has extended to five months so far, but I’m still excited about it. Best of all it’s still fun. It also has the unexpected benefit of spinning off Configurable, which has proven itself to be very useful in other projects (not to mention that it’s just cool, IMO).

Fix: Single File Tests Break in Netbeans 11.0 / PHPUnit 8.2

UPDATED 2019-07-22: Netbeans 11.1, released today, incorporates a robust fix for this problem that should survive future changes in PHPUnit.

TL/DR: There’s a one line patch to PHPUnit below that will kludge the kludge and get you running again.

Anyone who has worked with PHPUnit for some time knows that backwards compatibility isn’t exactly a prime consideration. Meanwhile, although Netbeans currently has very good support for PHP, you have to figure that the intersection set between the Java developers working on the project and the people who figure that PHP is anything but a toy language for building simple projects is, well… small. [We’ll just ignore the fact that Facebook, the largest application on the planet, is written in PHP].

So when Netbeans says it offers support for PHPUnit 3.4.0 or higher, it’s okay to expect the integration to be out of date. Rather surprisingly, it’s actually worked right up to PHPUnit 8.1.

But now we have 8.2. Command line parsing has been made more robust, and the extra parameter Netbeans passes in to a kludged custom test when running a single file doesn’t work anymore. [BTW I don’t blame the Netbeans developers for this kludge, it was probably the only solution that worked back in 3.4.0.]

This makes the current Netbeans approach outdated and unworkable. Like many open source projects, this means someone who cares has to go in and do some significant re-work. Don’t hold your breath. I’d give it a go but my Java foo is about 20 years old now. I probably don’t know enough any more to even be dangerous. I think I know a good architectural approach but attempting to implement it would be a recipe for failure.

So what to do? Patch PHPUnit! This is a pain since the patch will have to be reapplied every time PHPUnit is updated, which is frequently. But at least it works.

So here’s the kludge: in the file TextUI/Command.php in the handleArguments array, just change the line that reads

if (isset($this->options[1][1])) {

to

if (isset($this->options[1][1]) && substr($this->options[1][1], 0, 6) != '--run=') {

This ignores the Netbeans-generated argument and everything works as before. Not pretty but it works.

For more information (and a possible fix), follow the Netbeans Issue.

The Blog is Ported (again). Hello WordPress.

Welcome to the third major iteration of It’s Fixed in the Next Release. From Blogger (yuck) to Serendipity (which truly deserves more attention than it gets) to WordPress (market share wins, even in the open source world) here we are.

Speaking of open source, what you see here is only possible because of the FOSS (Free Open Source Software) model. Not only is the core of WordPress completely open, most of the themes, plugins, and tools are as well.

Independent of the cost, FOSS made this blog because the theme is a hybrid. The base theme is the free version of “Blogolife”. I’ve made some changes to it, both in terms of some invisible code rework, and by hacking in the tag display from “AskIt”, a commercial template I purchased for a project that died. At first I thought AskIt was going to do the trick, but in the end I just didn’t like the main page layout. Blogolife had the clean traditional blog feel but the display of tags and categories wasn’t all that great (I haven’t done anything to the categories yet).

I also removed all of Blogolife’s admin links and promotions for the “PRO” version. Needless to say I wasn’t likely to be buying that anyway.

It’s interesting hacking on WordPress code, and it’s a classic comparison between simplicity and power. I think it’s possible to keep most of the WordPress API in your head, and its possible to be a true expert on it in a year or so. Meanwhile, I’ve been working with Joomla for almost six years now, and it seems I keep on discovering new tricks, not including the outside-the-core stuff like template frameworks and the like. So there’s a freedom in WordPress that makes it possible to hack bits of two themes together in a few days and get something that works. But at the same time I feel like I’m working in a small shop with hand tools. By comparison Joomla feels like a manufacturing line full of robots — configure each one to do a job then watch the whole facility in action.

Each environment has its place. It’s actually more personally satisfying to feel you’ve built something by hand. WordPress certainly rocks the blogging / simple site space, but it has its limitations. Over in the Joomla development world we wrestle with issues like how to make it so those “robots” of ours plug into each other most effectively and with a minimum of code duplication. While we’re making it easier to plug things together I think we’d also be wise to keep an eye on offering freedom to site builders who need to produce a custom solution but only know the basics of PHP development. We never want to be in the position where someone says “Joomla is a great platform but you need a team of developers to build and run a site.”

After all, market share wins, even in the open source world.

WYSIWYG? WYSIKSWYGBNQ!

I Twittered this but it deserves more permanence. I’ve been using it for years and unfortunately it still summarizes my frustration with browser based Javascript “WYSIWYG” text editors:

WYSIKSWYGBNQ = What You See Is Kinda Sorta What You Get But Not Quite.

On Development Teams

Earlier today Amy Stephen and I had an interesting discussion on development teams, I thought it worth organizing and preserving. Because it’s a bit of a threaded conversation it takes a little work to follow the flow, but there’s no easy way to sort it out.

One of the most interesting things here is that even though I tend to take a bit of a hard-ass “no prisoners” approach to the problem, and Amy is fairly close to my polar opposite, we actually look at the fundamental problem in nearly the same way.

AmyStephen Wondering about the market potential of the promise of a 1 day workshop that can turn a dysfunctional group into a high performance team. It’s puzzling to watch a dysfunctional group where members keep barriers to entry high and engage in intense, private battles within.
FxNxRl Lets see… the seminar starts with a real-time attitude and teamwork survey, then lists who should be retained and who fired.
AmyStephen That might do it – if management would so empower. Typically, you’ll find spineless leadership behind dysfunction.
FxNxRl Indeed. I have done consulting assignments where I had to tell the person who hired me that they were the main problem. Awkward.
AmyStephen So, how would you sort out the understandably frustrated’s from those who might be, shall we say, squelching innovation?
FxNxRl Most of the time it’s pretty easy to tell one from the other by simply talking to them face to face.
AmyStephen I used to think that I could tell quickly but now I am doubting my early instincts. It’s a bit unsettling, in fact. I can’t tell.
FxNxRl What’s difficult is to unmask the passive-aggressive ones. Usually high-value team members who kill things by “contributing”.
FxNxRl It is impossible to tell when there’s distance involved, and it can be difficult in person too.
FxNxRl I once managed a team where I had to use source code control metrics to track project progress. Every time a project was really doing well, the lead developer would step in to “help” and the project would nearly stop. There was no way to fire him…
AmyStephen Yes, it’s tricky when the one causing the problem has a great deal of authority. You have to control the process to figure it out.
AmyStephen Source code control can be a great tool for lots of challenges simply because people who are traumatized can see justice/hope
AmyStephen That is, provided they can commit to it, of course. Otherwise, it’s just another situation where they have to face this barrier.
FxNxRl Yup, and it’s hard data in an environment where everyone has some emotional involvement no matter how hard they try not to.
FxNxRl Development without revision control is art, not engineering.
FxNxRl The very first thing is to put proven development methodologies in place. If there’s strong resistance, fire the dissidents.
AmyStephen lol – I definitely do not agree, but I understand. Many times, your skeptics are simply people who think for themselves.
AmyStephen In fact, you can bag anyone who’s onboard with change too quickly. Everyone should resist process changes if they are moving forward.
FxNxRl I realized, at about age 45, that I had become the “asshole manager” I loathed at 25. Team results trump individual brilliance.
AmyStephen In general, I think it’s a bad idea for mgmt to plunk a set of “productivity tools” on an engineer’s desk. Best to let them pick.
FxNxRl Well yeah, you can’t say “effective Monday everyone is using SVN and methodology X”. The *method* of change is critical.
AmyStephen I’m a HUGE believer in team. I think, though, the strongest teams are built from a complementary collection of individual brilliance.
FxNxRl But the guys who say “never did it that way, ain’t gonna start now” and then dig in? Well they can do it “their way” elsewhere.
FxNxRl So am I… and I have seen effective teams operate in mayhem. It’s all about getting complimentary characters.
AmyStephen That’s why it’s important to involve them. Good engineers know what productivity aids are needed and welcome it. They resist “silly”
FxNxRl I have seen highly effective teams with vastly different cultures. You could NEVER take a person from one of those teams and put them in the other. It’s all about getting people who work well together.
AmyStephen Absolutely! I’ve worked on a team like that 2 times in my life. When you have tasted it, you know what it is & why it’s worth having
FxNxRl I’ve also become friends with people who I’ve let go as a “parachute in manager/jerk”. They found teams where they fit & r happy
AmyStephen Yes – there’s a chemistry that cannot be ignored. In MBTI, it’s the middle 2 indicators that can sometimes predict compatibility
AmyStephen Yes. That’s best. A self-directed work team w clear boundaries and focus and empowerment. It’s how “grown-ups” should be treated.
FxNxRl Dated though it may be I’m a huge MBO fan. Here’s what we need to do, tell me what you need to do it, then get it done.
FxNxRl as long as it’s legal and ethical I don’t care how it gets done, just that I can measure progress and that there is progress.
AmyStephen I don’t think common sense is ever dated. Keeping a focus on “what this is really about” is always good.