At GitLab, we believe that everyone can contribute. We also use automated
testing extensively to make contributing to GitLab easier. Using automated
testing is a great way to improve confidence when someone needs to change
the code, which actually is the case in the majority of contributions to
software projects.
But how do we ensure that our test suite covers enough to aid the confidence
in changing behavior of the software, and what can we do to keep on improving
it?
What is code coverage?
Using the code coverage metric is a
technique that helps to improve the test suite, development process, and the software itself.
Tools used to measure the code coverage percentage usually extend the test harness
environment and make it possible to map the application execution process
back to the source code while automated tests are being executed. With that
approach, you can not only learn how much of your code is covered by tests,
but it is also possible to find out what exact parts of the codebase are not
covered well enough.
Some coverage analysis tools also make it possible to generate code coverage reports in HTML
format that you can then view in your browser. It makes it much easier to
inspect the areas of code that are missing tests and are likely to need some
improvements as well.
You can take a look at the Ruby code coverage report for GitLab
that is hosted on GitLab Pages.
How to generate a code coverage report
There are a lot of code coverage tools available for many different languages,
and you will need to find the most appropriate option for your particular needs. At GitLab, with
projects using Ruby, we often use SimpleCov.
You will need to check the documentation for your tool of choice to learn how to
generate the code coverage report. Once you are able to do this locally,
check out the rest of this tutorial to learn how to publish the report with
GitLab Pages!
For the sake of this example, we will assume that you are using Ruby with RSpec
and SimpleCov.
How to configure your tools
Configuring SimpleCov can be as simple as extending your spec_helper.rb with:
require 'simplecov'
SimpleCov.start
When you run the rspec command, you will notice the code coverage report being
generated when tests are completed. The RSpec example below comes from a very simple
code that contains a single test for the single class that is there:
spec/dog_spec.rb
describe Dog do
it 'barks' do
expect(subject.bark).to eq 'Woof, woof!'
end
end
dog.rb
class Dog
def bark
'Woof, woof!'
end
end
And the RSpec test harness output is:
Dog
barks
Finished in 0.00058 seconds (files took 0.08804 seconds to load)
1 example, 0 failures
Coverage report generated for RSpec to /tmp/coverage_example/coverage. 6 / 6 LOC (100.0%) covered.
At the end of the output, you can see that code coverage report was generated
to the coverage/ directory whose contents look like:
$ ls coverage/
assets/ index.html
Yes! This is an HTML code coverage report that we can publish with GitLab Pages!
GitLab CI configuration
Take a look at our documentation
to learn more about how to use .gitlab-ci.yml.
The GitLab CI configuration can be defined in .gitlab-ci.yml file. Let's go
through the configuration that is necessary to publish coverage report with
GitLab Pages.
