ODI Head of Robots Sam Pikesley uncovers some new features in Rspec in his search for alternative tools for building Ruby command-line interface apps
‘Doing it all with Rspec.’ Photo: Sam Pikesley
Once upon a time, I was a UNIX SysAdmin, which means I’ve spent a lot of my working life deep in the command-line. As a consequence, many of the Ruby tools I’ve built in recent years have been CLI ones.
My weapon-of-choice for building Ruby command-line interface (CLI) apps has long been the mighty Thor, and up until now I’ve always used Aruba to test my Thor apps (while sticking with Rspec to TDD the actual workings of my gems). This has mostly worked OK, but I’m also (for better or worse) a big fan of VCR, and these things really do not play nicely together.
Because Aruba spawns a separate Ruby process to run its tests, it’s all invisible to VCR. There are a number of (now deprecated) hacks to get around this problem, but I was finding that I had to write my features in very contrived ways (which definitely defeats the purpose of Cucumber), and it still behaved unexpectedly. And when Aruba also started to interfere with some of my other favourite tools, I decided it was Different Solution time.
I came across this blogpost which mentions this capture method in Thor’s spec_helper. Turns out this is kinda generic, and we can paste it right into our Gem’s own spec_helper.
But wait, don’t call yet, because then I was pointed towards this Stack Overflow post about validating exits in Rspec, and that led me to Rspec’s output
matcher which appears to make all of the foregoing redundant.
So how does this all fit together?
module Banjaxer
describe CLI do
let :subject do
described_class.new
end
it 'has a version' do
expect { subject.version }.to output(/^banjaxer version #{VERSION}$/).to_stdout
end
end
end
module Banjaxer
class CLI < Thor
desc 'version', 'Print banjaxer version'
def version
puts "banjaxer version #{VERSION}"
end
map %w(-v --version) => :version
end
end
In the spec, we set up a instance of our class, which is a Thor, then we call its #version
method and inspect whatever lands on STDOUT.
There is a certain amount of sleight-of-hand going on in this: note that our argument to the output matcher is a regex, even though we really want to match a string. That’s because the actual output string will have a "\n"
on the end of it, so we’d have to match that explicitly.
module Banjaxer
describe CLI do
let :subject do
described_class.new
end
it 'gets the url', :vcr do
expect { subject.get_url 'http://uncleclive.herokuapp.com/banjax' }.to output(/^Content-Length is 808$/).to_stdout
end
end
end
module Banjaxer
class CLI < Thor
desc 'get url', 'GET a url and tell us the Content-Length'
def get_url url
h = HTTParty.get url, headers: { 'Accept' => 'application/json' }
puts "Content-Length is #{h.headers['Content-Length']}"
end
end
end
We might notice some more prestidigitation here, when we consider how Thor works: it takes something like banjaxer get_url http://uncleclive.herokuapp.com/banjax
from STDIN, and turns that (via the ./exe/banjaxer
executable) into a call to #version('http://uncleclive.herokuapp.com/banjax')
- we’re bypassing that step and making the method call directly. The corresponding Aruba:
Scenario: Get url
When I successfully run `banjaxer get_url http://uncleclive.herokuapp.com/banjax`
Then the output should contain "Content-Length is 808"
… will do exactly what it says, which may be a more accurate test, but notice that we’ve dropped a :vcr
into the Rspec version and it worked as expected, which simply would not happen with Aruba.
module Banjaxer
describe CLI do
let :subject do
described_class.new
end
context 'with options' do
it 'can handle an option' do
subject.options = {json: true}
expect { subject.embiggen 'the smallest man' }.to output(/^{"embiggening":"the smallest man"}/).to_stdout
end
end
end
end
module Banjaxer
class CLI < Thor
desc 'embiggen', 'Embiggen something'
method_option :json,
type: :boolean,
aliases: '-j',
description: 'Return JSON on the console'
def embiggen value
if options[:json]
puts({ embiggening: value }.to_json)
else
puts "embiggening #{value}"
end
end
end
end
Some more trickery here, which took me a little while to figure out: when we pass options on the command-line, Thor shoves them into the options hash on the instance. So in our spec, we set up that hash ourselves with subject.options = {json: true}
and then call the method.
module Banjaxer
describe CLI do
let :subject do
described_class.new
end
context 'deal with exit codes' do
it 'exits with a zero by default' do
expect { subject.cromulise }.to exit_with_status 0
end
end
end
end
module Banjaxer
class CLI < Thor
desc 'cromulise', 'Exit with the supplied status'
def cromulise status = 'zero'
lookups = {
'zero' => 0,
'one' => 1
}
code = lookups.fetch(status, 99)
puts "Exiting with a #{code}"
exit code
end
end
end
Checking the exit status is supported out-of-the-box in Aruba:
Scenario: Get version
When I run `banjaxer -v`
Then the exit status should be 0
… but for Rspec, we have to cook up our own custom matcher:
RSpec::Matchers.define :exit_with_status do |expected|
match do |actual|
expect { actual.call }.to raise_error(SystemExit)
begin
actual.call
rescue SystemExit => e
expect(e.status).to eq expected
end
end
supports_block_expectations
end
This is surprisingly simple: we just #call
the method passed in as actual
, trap the exception it raises, and check its #status
against the expectation. That supports_block_expectations
is apparently required because this matcher actually calls a block (but this is a bit magical and I don’t fully understand it, I just know that it didn’t work without it).
module Banjaxer
describe CLI do
let :subject do
described_class.new
end
context 'read output files' do
it 'writes the expected output file' do
subject.say 'monorail'
expect('said').to have_content (
"""
The word was:
monorail
"""
)
end
end
end
end
module Banjaxer
class CLI < Thor
desc 'say', 'Say the word'
def say word, outfile = 'said'
File.open outfile, 'w' do |f|
f.write "The word was:\n#{word}"
end
end
end
end
Replicating this (very useful) feature of Aruba:
Scenario: Write file
When I run `banjaxer say monorail`
Then a file named "said" should exist
And the file named "said" should contain:
"""
The word was:
monorail
"""
… required considerably more work. The full code for the have_content
custom matcher (and its supporting bits and pieces) can be seen here. There’s quite a bit going on in this, so let’s dig in:
Presumably our CLI app would generate any output files in its Present Working Directory, but we can get Rspec to make us a temporary directory and switch to that before each test (and then bounce back out of it afterwards). Notice that it deletes the tmp/ directory before it starts, not at the end of the run. I stole this idea from Aruba and it means that in the event of a spec failure, we can run just the failing test and then debug by having a look at exactly the output it produced.
This matcher takes the expected string from the spec and reads the actual file, then splits them both into lines and compares them – if it finds a mismatch, then pass
becomes false
and we get a failure. The clever stuff is in the next section, though:
String
I originally wrote these as normal static methods, but it occurred to me that everything would be a lot more elegant if they were String
instance methods. The interesting (and possibly brittle) thing here is the #is_regex
stuff: if the string looks like a Regular Expression (ie with leading and trailing slashes) then we take the body of it and turn it into an actual regular expression and then do our comparison against that. I think this may bite me somewhere down the road.
This matcher is significantly more sophisticated than the exit_with_status
one - so much so that it became necessary to generate Rspec with Rspec.
Any self-respecting CLI app is likely to be generating feedback on the command line, but this is going to pollute our Rspec output. We can suppress it with something like this in the spec_helper:
RSpec.configure do |config|
# Suppress CLI output. This *will* break Pry
original_stderr = $stderr
original_stdout = $stdout
config.before(:all) do
# Redirect stderr and stdout
$stderr = File.new '/dev/null', 'w'
$stdout = File.new '/dev/null', 'w'
end
config.after(:all) do
$stderr = original_stderr
$stdout = original_stdout
end
end
Before each test, we redirect STDOUT and STDERR to /dev/null
, then bring them back afterwards. Note that this is not platform-independent, you need to something different on Windows, but I don’t know what. Also note that this causes pry to do really odd things - disable this if you want to reliably pry into your code (maybe this should be wrapped in an unless ENV['PRY']
guard of some sort).
The code is all here, please feel free to have a look at it and run it:
git clone https://github.com/theodi/banjaxer
cd banjaxer
bundle
bundle exec rake
As always, feedback, PRs etc are welcome.
I seem to have replicated quite a lot of the functionality of Aruba, but with the added benefit of not using Aruba. I think the thing to do now might be to package this up into a Gem and use it on a real project.
I sincerely hope somebody else finds this useful – I certainly did.
Sam Pikesley is Head of Robots at the ODI. Follow @pikesley on Twitter.
If you have ideas or experience in open data that you’d like to share, pitch us a blog or tweet us at @ODIHQ.