End-to-end testing w/Rails 5+, Angular 5+, Capybara, and Selenium: Part 2 of 4

By Zach Dennis on 13 Jun 2018

A few days ago, we looked at the rationale behind how one of our project teams tackled the question: How would we set up end-to-end testing for a product that had a Angular 5 frontend and Rails 5 API backend? This was from Part 1 of this series. If you're wondering which tools to use – should we use Protractor, Capybara, or something else? – then that first post will give you some good things to think about.

Today, we'll be walking through setting up a fresh Angular 6 and Rails 5 API project with end-to-end testing using Capybara, RSpec, Selenium WebDriver, and Chrome.

The walkthrough will start from nothing, we'll work our way up from there, and we won't skip over the errors. Heck, half the time, knowing what causes a specific kind of error is exactly what you need to know what to do about it. I will not rob you of that opportunity, it's too valuable. One downside of exposing the errors for our mutual benefit though is that this entire walkthrough gets a lot longer. Don't let the length fool you though, the value you get out of this walkthrough surely outweighs the thirty minutes you spend doing it.

One assumption I have is this: Your goal is to take what you learn here and apply it to your own application. Assuming that assumption is spot on – assumptions on top of assumptions, what could go wrong? – we'll keep "example" features to an absolute minimum. So basically: We – as in you and me compadre! – are interested in setting up end-to-end testing, not building the world's 957th to-do app. With this shared goal in mind, I will only let two more sentences stand between us and the walkthrough.

Sentence one: If you're the sort of person who benefits from seeing the final product and poking around – You want to skip the walkthrough? Gasp. I'm not judging. You be you. – then you can find this up on Github in this repository: https://github.com/mhs/angular-rails-e2e-sample-app

Sentence two: Now, if you're still with me, then you're someone who sees benefit in learning by doing so let's get to it, starting with the Rails 5 API backend.

Setting up our Rails 5 API

We're going to be running a lot of commands so keep your terminal open.

Installing the latest version of Rails

First, let's make sure you have Rails 5.2.0 gem – the latest version of Rails at the time of this writing – installed:

Creating a new Rails project

Next, let’s create a Rails API project by running the following command:

Two things worth noting at this point:

  1. There’s a whole lot of output from the above command so I’ve excluded the output. You'll likely see the [output suppressed] a few more times throughout the post where the command output is excessive.
  2. We’re skipping tests with --skip-test for now and will add in rspec-rails later so we can use RSpec for testing.

At this point you should have a my-great-new-app/ directory. Let’s cd into there:

Using Git

A short meander off the beaten path: By default, Rails will assume that the project will use git for version control. It will generate a .gitignore file, but it does not initialize the repository or make the first commit.

Now that we’re in the project directory (and before doing anything else work) let's initialize the Git repository for the project and make our first commit.

Run the following commands:

If the commit message above looks funny it's likely because it's a multiline message. Personally, I like to put commands in the commit message for future reference. My future-selves and teammates sometimes find this helpful.

That's enough meandering for now, let's set up our testing framework: rspec-rails.

Adding rspec-rails

The first thing we need to do is add rspec-rails to the Gemfile.

Open up Gemfile and add/append it to the file. If you want to do it all from the command line here's one way to do that:

The last thing to do here is a sanity check that RSpec is happy. Even though we don’t have any specs (yet) let’s run the rspec command and make sure it executes successfully:

We now have a Rails application ready for development. Let's set this aside for a moment and get our Angular frontend up and running.

Setting up the Angular 6 app

Install @angular/cli

First, we need to install @angular/cli which is at version 6.0.8 at the time of this writing:

Create the Angular app

From within the Rails project directory generate an Angular app named frontend:

Quick note on directory placement

Even though the frontend/ is inside of our Rails project directory it doesn't have to go there. We could put this somewhere else entirely. We're doing it in this post for the benefit of the walkthrough. It's not advocating one way or another how to organize frontend and backend projects.

If you see: Unknown option --silent error

If you see the below error then you need to upgrade your version of yarn and re-run the ng new frontend. If you don't see the below error skip ahead to the next section.

Upgrading yarn is as easy as the below command:

This particular issue has been reported as @angular-cli/issues/10428 on Github. Thanks to @clydin for the tip that yarn was out of date.

Installing dependencies

Next, install all of the dependencies for this Angular project:

This makes sure the Angular has all of its NPM dependencies installed, then it will launch a browser with the stock “Welcome to app!” page.

Welcome to app!

You can Ctrl-C to kill the server in the terminal.

Angular and Git

Similar to Rails, Angular assumes the project uses Git. All we need to do is add the directory and commit:

Huzzah!

We now have a Rails 5 API backend and an Angular 6 frontend ready for integrating. Yes, I know we have no functionality. That's not necessary at this point. We'll get there soon enough, trust me!

Before moving to the next step make sure you're back in the Rails project root:

Now, we're ready to set up our first end-to-end feature spec.

Setting up the first end-to-end feature spec

I like taking lots of small steps until I have the confidence to take larger ones. So, let's start small and build our confidence together.

How can we take this approach in an app that has literally no functionality? We-ell, we can do this by writing a few simple end-to-end tests that tell us when we have achieved a working end-to-end testing setup. Once that is working we can then easily start writing end-to-end tests that exercises other product-specific functionality.

So, let's start there and make the simplest possible spec to show that we're on the right path. As a quick aside, this post will use the "terms" test and "spec" interchangably.

Creating the spec

Make a spec/features/ directory if one doesn’t already exist:

Now, we need to create a file for our first end-to-end test, spec/features/welcome_spec.rb, with the following contents:

It's hard to get any simpler than this.

Running the spec

Next, run the spec from the terminal:

No database schema warning

If we read the output from the rspec command from top to bottom we'll notice that there's a warning in there:

This is not causing our test to fail since we do not have any functionality that relies on the database, but let’s take the necessary action to remove the warning:

Running the spec: Capybara not loaded

Now, run the spec again:

The same failure exists as before but without the database schema warning. Now, let's shift focus to to the actual RuntimeError:

This error is telling us that we need to set up Capybara. Let's do that next.

Setting up Capybara and friends

If you’re not familiar with Capybara

Capybara is a framework for testing web apps. From its project README:

Capybara helps you test web applications by simulating how a real user would interact with your app. It is agnostic about the driver running your tests and comes with Rack::Test and Selenium support built in. WebKit is supported through an external gem.

We’ll need to use a driver that is capable of executing JavaScript since we are testing a frontend application that is pure JavaScript. Selenium WebDriver is a cross-language tool for doing just that and it can work with multiple browsers.

We'll also use Chromedriver and rely on Google Chrome for this end-to-end testing setup. You can configure Capybara and Selenium WebDriver to use different browsers, but that's outside the scope of this post.

In addition, we’re going to add CapybaraSpaShameless plug, I know. It's a small RubyGem we wrote at Mutually Human, but it will save us time and headache, so we're going with it! – to manage the HTTP server process running the Angular app in our tests.

Adding Capybara and friends to the Gemfile

It's time to update the Gemfile with the following test group of gems:

Next, let's install those gems with Bundler:

Now we're ready to write a capybara configuration.

Configuring Capybara

Make a spec/support/ if one doesn’t already exist and create the file spec/support/capybara.rb:

Now, edit spec/support/capybara.rb file and give it the following contents:

Let's defer walking thru the contents. There will be a follow-up post that walks thru this file and introduces some additional helpers for end-to-end testing. For now, I ask that you extend a small amount of trust as we soldier on.

Running the spec: RoutingError

Run the welcome_spec.rb again:

This will produce a lot of output, but that output should have an error message in red that looks like this:

The error is telling us that the Rails application is trying to serve the request. We can tell because it's an ActionController::RouteringError, and ActionController is part of Rails. This isn’t what we want. We want the Angular app to handle the request.

In our spec/support/capybara.rb file we added configuration for a FrontendServer to run, but it doesn't seem to be.

What did we forget?

Whoops! Forgot to load support files

You may have caught this or had a hunch, but we forgot to update our spec/rails_helper.rb file to ensure that we are loading the files in our support/ directory.

Personally, I like to recursively load all ruby files in spec/support/ rather than one-by-one. Let's go with that approach by adding the following line to spec/rails_helper.rb:

This should go right under the line (usually around line 8) that says:

This will automatically load any files in support at the start of our specs running. This way whenever we add new support files we can rest assured that they will be loaded.

Running the spec: angular-http-server not found!

Now, let's run the welcome_spec.rb again.

This is exactly the error message we want to see at this point. We need an HTTP server capable of serving our Angular app.

Why?

Well, our backend is an API server so it's not serving up JavaScript (or any assets for that matter) for us. The above error actually coems from the capybara_spa and it's trying to help us out by giving us a hint that it can work with angular-http-server to serve our Angular app.

Installing angular-http-server

Change your working directory to frontend/ and let's install angular-http-server from NPM and save it as a development dependency:

Now, run the spec again:

Progress, but we've got a new failure to deal with.

Missing an Angular build

The capybara_spa gem is telling us that it cannot find a static build of the Angular app. It’s suggesting that we need to build it with an integration configuration. It's also suggesting a command used to build Angular 2/4/5 apps, not Angular 6 apps which uses a slightly different command, but we can address that our own soon, you'll see.

For now, we can get past this by using a building our Angular app with the stock ng build command:

This builds the Angular app to the frontend/dist/frontend directory. This matches up with the creation of the FrontendServer in the spec/support/capybara.rb file which specifies that we’ll be using the frontend/dist/frontend directory for the static app.

Running the spec: Success!

Now that we've got the Angular app built, let's run the specs again:

Hooray! Granted, there's a lot of noisy output, but let's not take away from the feeling of a small victory, which, we'll address next.

Redirecting angular-http-server logging

All of the noisy output we just saw is from the angular-http-server process. The output can be helpful when things aren't working but we don't need it to go to STDOUT.

Let's redirect the angular-http-server process output to a log file.

Open up the spec/support/capybara.rb file and uncomment line 17, the one that looks like this:

Now, run the spec again:

There we go, a lot less noise! To confirm that it got redirected to the right spot go open up the log/angular-process.log file.

If you haven't been continuing to commit along the way now is a great place to do it. We've proven that we have a basic working configuration for end-to-end testing.

Well, almost.

Connecting the frontend and the backend applications

At this point, we have the most basic end-to-end testing set up in place. Our frontend app doesn't actually talk to our backend API though. In any real application it would and we would run into an obstacle: CORS and the Angular build configuration/environment.

Let's implement a single trivial feature that forces our frontend app to talk to the backend so we can see what these problem looks like. Then, we'll go ahead and fix them.

Displaying a list of welcome messages

We're going to display a list of welcome messages in the Angular app that comes from the the Rails API. We'll work outside in.

Before we get started start up your angular server and your Rails server in two different terminal shells:

Updating the frontend starting with the component HTML template

Let's add the welcome messages to the current AppComponent that was generated by the ng command.

Add the following somewhere after the div that contains the Welcome to {{ title }}! message in frontend/src/app/app.component.html:

At this point the Angular app should compile successfully and the "Welcome to app!" page should render successfully in your browser when viewing http://localhost:4200.

Updating our component source file

Next, we need to update frontend/src/app/app.component.ts to supply the welcome messages.

Here's what that file look like:

Here's what we've done to the AppComponent:

  • import the environment configuration
  • inject an HttpClient into the component
  • implement OnInit so we can fetch the welcome messages from the server
  • store the messages on the component's public property messages

Updating our environment configuration

If you look at the terminal window that is running the Angular server you should see that we have a compilation error:

We're attempting to call environment.apiUrl, but our environment doesn't have an apiUrl property. Edit the development environment configuration (e.g. frontend/src/environments/environment.ts) and add an apiUrl property:

This points to our local Rails development server that is running on port 3000. If you take a peek back at the terminal window running our Angular server our compilation error should now have gone away:

Importing the HttpClientModule

If you look in the browser you'll see nothing but a blank white page. Something is amiss.

Go ahead and open your browser's developer console. We should both be seeing the following error:

Since we are injecting HttpClient into AppComponent we need to make sure it's available. Right now, it isn't.

We need to update our parent module frontend/src/app/app.module.ts to import the HttpClientModule.

Add the below import to the top of the file:

Then, add the HttpClientModule to the list of imports in the @NgModule declaration. Your frontend/src/app/app.module.ts file should now look like this:

Alright, we've got the Angular app compiling successfully and it's rendering once again in the browser. But – Always a but isn't there? – if you peek again at your browser's developer console, what do you see?

We should both be seeing a few more errors.

Issues with Routing and CORS

The Angular app now compiles and render successfully although we have a few errors to deal with:

Browser's developer console errors: Routing and CORS

In case the errors are too hard to read in the above they've been pulled out below:

There are two errors spread amongst three messages:

  • a 404 Not Found for the /welcome_messages path on the server
  • a Cross Origin Resource Sharing (CORS) issue

If you look at the terminal window where your Rails server is running you'll see evidence of the 404 printed there:

Let's address this 404 Not Found issue first. Then, we'll get back to dealing with CORS.

Add the missing route; fixing the 404

Go ahead and stop your Rails server process running in terminal. Then, add a welcome message resource to the Rails app:

This command does the following:

  • Update config/routes.rb file to add the /welcome_messages end-point
  • Add a WelcomeMessagesController for handling incoming requests
  • Add a WelcomeMessage model
  • Add a database migration for adding a welcome_messages table along with a message column.

Next, let's migrate:

Now, start your Rails server back up:

Refresh your browser. Bummer, same errors!

But, look at the terminal running your Rails server. You no longer see the the following error:

Instead, we get a new error:

We can get address this error by implementing an index action on our WelcomeMessagesController.

Let's add one that returns a simple array of strings by updating the app/controllers/welcome_messages_controller.rb:

Next, we's add a WelcomeMessage to our database. Launch rails console and let's add one directly:

If everything is wired up correctly we can expect to see "Hallo there!" in the browser.

Refresh your brower.

Shucks!

The message isn't showing up. If we check the browser's developer console and refresh again we can see that we have indeed addressed the 404 Not Found issue. Although we've still got the CORS error showing up:

Why?

We-ell, the Angular application is running on a different port than our Rails server. The browser sees this as a potential security threat. We need to update our Rails server to allow Cross Origin Resource Sharing with our Angular app.

Angular + Rails Communication with CORS

We're going to use the rack-cors gem. Stop your Rails server, then open up the Gemfile and add it to the development and test groups:

Then, bundle again:

CORS Configuration

Next, let's add an CORS initializer. Create config/initializers/cors.rb with the following contents:

Quick note: Only allowing CORS for development

We're wrapping the server-side CORS configuration in a Rails.env.development? check because we only want to allow CORS to happen in specific environments. Right now, the only environment we need it for is the development environment.

Quick note: Allowing any origin (domain) to make a CORS request

This configuration also allows any origin (aka domain) to make these requests. During development this makes life easy. For any production or production like environment that needed to use CORS we would not want to allow this to be wide open.

Okay, did you see that note on the first line of the cors.rb file we just added? The one about restarting the server.

It's time to boot our Rails erver back up:

Once it's back up, refresh your browser. We should both be seeing the welcome message "Hallo there!" in the browser.

And, if you check the browser's developer console:

Routing and CORS success

No more errors. Hot dog!

Updating our end-to-end test

Now, that we've got things working in the app let's update our spec/features/welcome_spec.rb by adding a new example to it. This will help us work through similar issues that our test environment may have.

Add the below example right under the it "loads the Angular app" example:

This new example is still really simple. We create two WelcomeMessage records in the database, then we visit the site, and we expect to see those welcome messages show up.

Let's run this specific example only. We can do this in terminal run with the below command (where :9 at the end is the line number that the new example starts on):

It couldn't find the "Top of the morning to you" welcome message on the page. This seems fishy, doesn't it?

Let's debug this.

Debugging the test run

Let's add a sleep 60 to our "shows a list of welcome messages" example right after the visit '/'. This will give us a minute to poke around in the Chrome browser that pops up.

With that in place, re-run the spec:

At this point, we're looking to make sure everything matches our expectations. First we'll check the console.

When Chrome pops up you'll see the "Welcome to app!" message then:

  • Right-click and inspect the page to open developer console
  • Click over on the Console tab
  • Hit Cmd-R or Ctrl-R to refresh the page

There are no errors at this point. Let's check out the network requests.

In the developer tools panel click over on the Network tab. Then, hit Cmd-R or Ctrl-R to refresh the page.

Missing /welcome_messages network request

We've got the standard requests, but wait a minute! there's no request to the /welcome_messages end-point. Aha!

Here's what this means: The Angular app that our test suite is using is an old build. If you remember from earlier, we had to build a static version of the frontend app that the angular-http-server can use for end-to-end tests.

We need to rebuild our app.

Rebuilding the Angular app

Let's go into the frontend/ directory, rebuild the app, and re-run the welcome_spec.rb:

Don't forget to take the sleep 60 out of the welcome_spec.rb. Then, run the test again:

This time it fails in a subtle, but important way (if you still have your Rails server running). Looking closely at the error message (or if we watch the Chrome browser when the test runs) we can see that is displaying "Hallo there!" on the page:

Hallo there! showing up in our test run

But, we don't have a "Hallo there!" welcome message in our test environment. We only created two messages: 'Top of the morning to you' and 'Good day'.

What's going on?!

Giving our Angular app an e2e configuration

We-ell, one place we did put that 'Hallo there!' message was in our development environment. Could the frontend app be hitting our development server? Let's find out.

Run the test again, then switch over to the terminal window where your Rails server is running.

If another request come in for /welcome_messages then we'll know that our tests are hitting our development environment.

That's definitely what I saw. How about you? Did you see a request like this come thru?

This confirms that our frontend app used in our tests is currently configured to hit the development environment. This kind of makes sense though. We haven't configured any additional environments in our frontend app. We need to make one for our end-to-end tests.

Let's do that now.

Adding an e2e build configuration to our Angular app

First, we'll need to add a configuration in the frontend/angular.json file. if you open that file up you'll see a "configurations" section around line 30. The only configuration listed is "production".

Let's add one named e2e for our end-to-end testing. Copy the "production" configuration and paste in another configuration changing the production to e2e. Be sure to the .prod. in the fileReplacements to .e2e as well.

Now, the configurations should look like this:

You can tweak the settings of your e2e configuration to your heart's content.

This configuration references a new environment file, src/environments/environment.e2e.ts, that doesn't yet exist.

Kill your Angular server and let's create this file by copying the existing development env configuration:

We're going to make one edit to that file to start. Open it up and change the apiUrl to http://localhost:3000 to http://localhost:3001. Why port 3001?

If you recall from earlier in the post when we configured Capybara in spec/support/capybara.rb we specified the server_port to be on 3001. This is the port that the Rails API will be listening for requests on when running feature specs, and it's why we're updating the apiUrl so the Angular e2e environment is in sync with the test environment of our Rails app.

Once the environment.e2e.ts file is updated and saved we need to re-build the Angular app:

Now, go back to the terminal window where you were running specs, and run the spec again:

We can put the sleep 60 back in the test, re-run it, and poke around the Console and Network tabs in the developer tools panel.

What we should both see is an error that looks familiar to from earlier:

CORS issue in test run

It's the CORS issue we ran into earlier.

Our test environment needs to allow CORS, but we currently only have it configured to allow development requests.

We need to update config/initializers/cors.rb to allow for that. Edit that file and add to the Rails.env.development? conditional so it looks like this:

Now, re-run the test:

Whoopee! We did it. We've got a working end-to-end testing setup.

Before we get to crazy in celebration let's run both examples in this spec (removing the :9):

Here's a video of the successful test run itself. It slows down when the Chrome browser pops up so you can indeed see both welcome messages being listed:

Alright, now we can celebrate. 🎉

Nicely done

Congrats! We now have a working end-to-end testing set up between an Angular frontend and a Rails API backend using Capybara, Chrome, RSpec, and Selenium WebDriver.

Going step-by-step (and there were a lot of steps) we encountered several errors. Sure, we could have skipped most of them and this post could have just plopped down the end result. That certainly would have made this post much quicker to read (and to have written). But, we would have missed out on the experience of encountering those challenges, getting to know what they look like, debugging them, and more importantly getting beyond them.

We can apply what we've done here to any Angular and Rails project. Heck, even if we work on a non-Angular or non-Rails project we can apply the same thinking and steps for getting an end-to-end test suite up and running. Sure, the syntax, specific library, or configuration may be different, but the steps are largely the same.

Earlier in this post we deferred going through the contents of the spec/support/capybara.rb file. In the next post, we're going to take a closer look at that file and introduce a few more helper support files that can enhance our end-to-end test suite.

If you have any questions in the meantime feel free to hit us up on twitter. Otherwise, I'll see you in Part 3 soon!

About Mutually Human

Mutually Human is a custom software design and development consultancy specializing in mobile and web-based products and services. We help our clients design, develop and bring to market innovative products and services based on insightful research and strategy aligned with business objectives. We’ve helped Fortune 500 companies, state governments, and startups.