• Verified no changes with Laravel 8.
• Added some details on running with Chromium.

There’s a mouthful. There’s two parts to this post. The first part is the story of how I arrived at using Behat, a tool that facilitates Behavioural Driven Design (BDD). The last part is the TL;DR configuration I used to get it working. If you just want to get this configuration going and don’t care about the background, skip down to it. I won’t be hurt.

First an admission: I’m really, really late to the BDD camp, and it kind of pisses me off. If I’d been using this approach for the past 15 years, there’s no question I would have gotten more done in less time.

I’ve written before about how unit tests pay back 10:1 in development time. At the same time I’ve also written a lot of not-so-pure “unit” tests that validate interactions between multiple classes. Laravel acknowledges this paradigm intrinsically by creating two folders under the test folder, “unit” and “feature”, although my preferred naming for feature is “integrated” since testing an actual feature is more in the domain of tools like Selenium. Now my Laravel “feature” tests have two areas of focus: transactional tests (stuff like add a user, verify login, have the user edit a profile, verify that the profile is in the database), and controller tests using Mockery to validate that a controller method fetches the expected things and hands the expected arguments to the correct view. These tests suck at isolation and I don’t really care as long as all the dependencies have adequate unit tests of their own.

After a few years of seeing “Behat” mentioned here and there, it has been on my “gotta check that out sometime” list. Unfortunately that list is rather extensive. “Sometime” could be years out, if ever. Having just spent months getting NextForm and Nextform-Laravel to the point where they are useful, far longer than I’d care to admit getting up to the point where I feel I have some competence in Laravel (an unexpectedly steep learning process), and with getting more into Vue.js next in the queue, the last thing I needed was a Beryllium hat! [no, that’s not the origin of the name, it’s just what it makes me think of, BeHat]

No, instead it was time to focus on the Next New Thing, which is a multi-headed beast featuring multiple subdomains, complex configurations, multiple user roles, configurable payment methods, and more. Too much more.

It doesn’t take long when working on a project of this size before you lose two things: the ability to test quickly and more importantly a clear understanding of what the hell the darned thing is supposed to be doing in the first place. All those wire-frame sketches, state diagrams, and hand-waving descriptions start to become cumbersome and either difficult to edit or difficult to revision control.

So I figure it’s time to find a better tool to keep track of the design objectives. I’ve been keeping some of this stuff in Markdown files, basically a series of bullet points that describe what’s expected in a specific scenario. Maybe there’s something I can use to formalize that a bit. At the same time I figure I’m going to have to fire up Selenium for some proper integration testing. Off I go to DuckDuckGo to see if I can find some solutions to this problem, and darn if Behat doesn’t start popping up for both. Okay, time to figure out WTF this Behat thing is, at least. If it doesn’t fit, fine. If it does, well it is on the list of things to check out anyway.

Off to the site. “Behat: a PHP framework for autotesting your business expectations.” What the heck is that? My “business expectation” is that revenue will exceed expenses! If I could auto-test that life would be a lot simpler! Next paragraph: “Behat is an open source Behavior-Driven Development framework for PHP.” Right. Got it. What the hell is “Behavior-Driven Development”? I just started implementing a Test-Driven Development methodology! Now this? Time to go down the rabbit hole… again.

You can just Plug Behat into Laravel’s test system… but

The first resource I found was this presentation by Marcus Moore at Laracon AU in 2019: Behaviour Driven Development with Laravel. Sadly, the slides link offers a 404. However I did find his helpful gist with references. As it turns out, Moore’s presentation is a great introduction to BDD, but the Laravel integration is relatively basic. BDD itself may not be about testing, but the Gherkin language describes features that are perfectly suited to acceptance testing. I felt that applying BDD at the unit/integration test layer is missing an opportunity.

Wait, there’s browser emulation?

One of Moore’s references is to Integrating Behat With Laravel by Matthew Daly. This provided a useful introduction to Mink, a browser controller/emulator for PHP. Now we’re beginning to get somewhere! Daly mentions that Laracast’s Jeffery Way has a Behat-Laravel Extension but that it looks unmaintained (It only works with Laravel 5) and was too complicated for his needs. But for my more complicated use case it seemed to have some attractive features, like direct access to your application’s models. Maybe outdated, complex and poorly maintained isn’t as scary as it sounds.

Making Behat-Laravel work

Needless to say there’s an introductory Laracasts video tied to this extension. The video offers some guidance to setup and promises much more in the subscription based video, but I figure I’m going to either see stuff I’ve already discovered or it’s never going to work in Laravel 7 anyway. So, we get the configuration in, initialize Behat give it a run, and as expected it throws exceptions. Fortunately, the changes aren’t extensive. In the extension’s KernelDriver.php all I had to do was replace a reference to Symfony\Component\HttpKernel\Client with Symfony\Component\HttpKernel\HttpKernelBrowser and we’re in business!

The best part of this is we’ve got “browser based” tests that don’t need a browser. Everything gets emulated under the hood. No processes to launch, no heavyweight browsers to wait for page loads, decently fast acceptance tests? Yes please! (caveat: Mink has a series of drivers, each with trade-offs some of your scenarios might need a different emulator or even a real browser, and it’s relatively easy to achieve that).

Not long after I’ve got a feature with a scenario that says, in short, “Visit the home page, see the pre-launch message” and it passes! Right, time to dig in and get to real work.

Making Subdomains work

First problem: the base test URL is set up in your behat.yml file. Good feature descriptions use explicit URLs rarely, and when they do they’re relative. All good design. Meanwhile all the heavy lifting for this thing is done in subdomains, which would require absolute URLs. That needs to be fixed. Only question is how.

My personal take is that the Behat documentation is fairly comprehensive, but there’s a decent learning curve for all this stuff, and the documentation strives to streamline and simplify the process to make it accessible to newcomers. Unfortunately that makes it relatively disjointed and opaque when I want a point by point reference for everything that can be stuck into a configuration file. After a fair bit of digging, some of it in the Behat code, I discover the @BeforeScenario hook and the way to set parameters at runtime:

    /**
     *  @BeforeScenario
     */
    public function before(BeforeScenarioScope $scope)
    {
        $this->setMinkParameter(
            'base_url', 'http://mysub.domain.test/'
        );
    }

It’s important to note here that it’s the @BeforeScenario doc tag that makes this method special, you can name the method anything you want.

Now we’re in business! Until the second subdomain, at least. The good news is that my test subdomains were already established to exercise different test scenarios, so all we need to do is set up a separate FeatureContext and we’re good to go.

While this is true, it would result in a lot of duplication in steps. But that won’t work anyway because Behat merges feature context steps, and the duplication isn’t permitted. As it turns out, the solution I went with was to put all the common steps into a trait, and then bring the traits in to the feature contexts that need them. Right, time to dig in and get to real work, again!

Countdown is… at 15 seconds and holding

This application has some moderately complicated forms. That’s one of the reasons why NextForm exists, to assist with their generation and processing. Loading them with data on each test is tedious and the main driver that took me on this journey in the first place, so that’s my first focus. I get a scenario running that logs a user in, fills in a form with incomplete data, and expects to see a validation error. I get nothing. It takes a little debugging, but not long after I discover that my application is returning a 500. But I know this part of the code works. In fact it works so well that I’m willing to spend a few days exploring automated tools so I don’t have to keep re-testing it on the way to the next step.

Check the logs and the 500 is happening because… NextForm is throwing a RuntimeException. WTF? The Laravel implementation of NextForm uses a singleton. Implicitly it expects to start in a pristine state on each request. But the Mink browser emulation isn’t doing real requests, it’s just passing request objects into my application, and NextForm is complaining that I’m trying to load my data schemas twice.

I’m relatively new to Laravel, but I can’t recall a “start of request” event. There’s already a significant infrastructure for initialization and boot, and I can’t recall a “soft boot” or anything like it. I can switch to using Mint as a Chromium controller but that’s going to be a lot slower. I’ll use Chromium for the scenarios that need Javascript but I want to draw the line there.

So maybe there’s more than one use case for resetting NextForm. I add a reset() method, update my controllers to call that before loading schemas, and carry on.

Fireworks! The form comes back with the expected validation error, my emulated user supplies it, resubmits the form, validation passes, and the app moves to the next step. This is victory.

Netbeans

While it’s not a huge deal to keep a console window open to run Behat, I have a bad habit of doing that before saving my changes in Apache Netbeans, which is counterproductive. Since there is as yet no native support for Behat in Netbeans. This is inconvenient, especially when there are times when running Xdebug in a scenario is helpful. Fortunately there is a kludge.

The trick is set up a runtime configuration in Netbeans that just runs Behat as a script file. Rather than mess with paths, I just took the lazy approach by copying /vendor/behat/behat/bin/behat to behat.php in my project root, dropped the shell script directive, and added the new file to git ignore. Then Netbeans starts everything with the right path and we’re off to the races: just select the new runtime configuration, press run, and watch the output window. Breakpoint your code and debug like any other script.

Running Chrome/Chromium

I found that running tests strictly inside Laravel was a little flaky when it came to sessions. I’d run an individual test just fine but if I ran the same test along with several others in the same feature, some tests would return 419 “Page Expired” response codes. Since the CSRF token was present in the submitted data, this could only mean that Laravel somehow managed to lose the session. After spending far too long trying to get this sorted out, I decided to bring in a real browser.

There are a few tricks to this. Browser based tests are by necessity slower, so you don’t want to go this route unless you need to. The first thing to know is that you can tag the scenarios you need to run in a browser with @mink:with_chrome.

Next up is to run your browser in a new profile. I launch with a command line along these lines:

path\to\chrome.exe --remote-debugging-address=127.0.0.1 --disable-gpu --remote-debugging-port=9222 --user-data-dir=c:\temp\chrome-remote-profile

If you don’t want to see what’s going on as your scenarios run add --headless to that command.

A good way to test that things are working is to open your regular browser and visit http://127.0.0.1:9222/json/version. If you get Chrome’s version information back, you’re good to go.

While I’m on the subject, the easiest way to install Chromium and update on Windows is with Chocolatey, a Windows based package manager that tries to be similar to yum or apt.

The TL;DR

Here are details of the setup I arrived at. This supports both a browserkit emulation and a full fledged Chromium connector.

Composer require –dev:

        "behat/behat": "^3.7",
        "behat/mink": "^1.8",
        "behat/mink-goutte-driver": "^1.2",
        "dmore/behat-chrome-extension": "^1.3",
        "dmore/chrome-mink-driver": "^2.7",
        "friends-of-behat/mink-extension": "^2.4",
        "laracasts/behat-laravel-extension": "^1.1",

Remember: the behat-laravel-extension needs a minor change in two places to make it work as described earlier.

In your project root run /vendor/bin/behat --init. This will create a /features folder and within that a bootstrap folder with a skeleton FeatureContext.php.

Here’s my behat.yml. My WebContext does little more than extend MinkContext, since it can only be extended once without causing duplicate name errors. FeatureContext does main domain stuff, GroupContext works on a subdomain. Both use a trait that load my common steps. I also override the default .env.behat that the Laravel extension uses with the same environment as my unit tests.

default:
  extensions:
    DMore\ChromeExtension\Behat\ServiceContainer\ChromeExtension: ~
    Behat\MinkExtension:
      base_url: http://domain.test/
      default_session: laravel
      laravel: ~
      sessions:
        chrome:
          chrome:
            api_url: http://localhost:9222
            download_behavior: allow
            download_path: /download
            validate_certificate: false
    Laracasts\Behat:
      env_path: .env.testing

  suites:
    default:
      contexts:
        - WebContext
        - FeatureContext
      paths:
        - "%paths.base%/features/bootstrap"
    group:
      contexts:
        - WebContext
        - GroupContext
      paths:
        - "%paths.base%/features/group"

That’s all

I hope people find this useful. While it’s pretty straightforward overall, it took some time to cobble all the details together from many sources. I hope this saves someone from that effort. Comments are welcome, let me know if you have issues and I’ll try to update the article.

I might also write a follow up to the to describe some steps I found useful, particularly when working with a page that uses Bootstrap 4, but don’t hold your proverbial breath.