summaryrefslogtreecommitdiff
path: root/docs/running-tests.md
blob: c3ede265113e33398fc48b202b39cf06cf0169da (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
# Usage

Buttercup is primarily meant to be used non-interactively, to
automatically test a project independent of a user’s setup, before a
commit and on a continuous integration platform. Because of this, the
recommended workflow for buttercup does not use interactive commands
but instead the command line.

## Cask

[Cask](https://github.com/cask/cask) is a project to create an
environment separate from your usual interactive Emacs environment.
This allows you to install the packages your project depends on, and
only those, and test your project in a well-defined environment.

Buttercup works best in such environments, so the following best
practices rely on Cask to be installed.

## Project Directory Layout

A basic project layout requires a project file, called `feature.el`
here, a `Cask` file to define dependencies, and a `tests/` directory
for tests. It should look roughly like this:

```
feature/feature.el
        Cask
        tests/test-feature.el
```

**feature.el**

```Lisp
(defun featurize (bug feature)
  (format "It's not a %s, it's a %s" bug feature))

(provide 'feature)
```

**Cask**

```
(source gnu)
(source melpa-stable)

(development
 (depends-on "buttercup"))
```

**tests/test-feature.el**

```Lisp
(require 'feature)

(describe "The feature"
  (it "can use bug and feature"
    (expect (featurize "bug" "feature")
            :to-equal
            "It's not a bug, it's a feature")))
```

## Running Tests

You can now use Cask to run your tests.

First, you have to install the dependencies. You only have to do this
once, or when the dependencies change:

```
$ cask
Extracting buttercup-1.1/
Extracting buttercup-1.1/buttercup-compat.el
Extracting buttercup-1.1/buttercup.el
Extracting buttercup-1.1/bin/
Extracting buttercup-1.1/bin/buttercup
Extracting buttercup-1.1/buttercup-pkg.el
Generating autoloads for buttercup-compat.el...
Generating autoloads for buttercup-compat.el...done
Generating autoloads for buttercup-pkg.el...
Generating autoloads for buttercup-pkg.el...done
Generating autoloads for buttercup.el...
Generating autoloads for buttercup.el...done
```

Now, you can run your tests:

```
$ cask exec buttercup -L .
Running 1 specs.

The feature
  can use bug and feature

Ran 1 specs, 0 failed, in 0.0 seconds.
```

That’s it. Buttercup’s built-in discover test runner looks for files
named `test-*.el`, `*-test.el` or `*-tests.el`. You can specify a
different pattern using the `--pattern` command line argument to the
`buttercup` program.

You can run this command whichever way you like. Common choices
include a makefile or shell scripts.

## Projectile

If you use [Projectile](https://github.com/bbatsov/projectile) for interacting with your projects you can set the "default" project test command to be available when you invoke `projectile-test-project`.  Create a `.dir-locals.el` file in the the root of your project tree (next to your Cask file).  An example:

**.dir-locals.el**

```
((nil . ((eval . (progn
                   (require 'projectile)
                   (puthash (projectile-project-root)
                            "cask exec buttercup -L ."
                            projectile-test-cmd-map))))))
```

## Travis

If your project is hosted on github, you can use
[Travis CI](https://travis-ci.org/) as your continuous integration
environment. Buttercup can easily be used in such a setup. Simply add
the following `.travis.yml` file:

```
language: emacs-lisp
sudo: false
cache: apt
env:
  - EVM_EMACS=emacs-24.5-travis
  - EVM_EMACS=emacs-25.1-travis
before_install:
  - curl -fsSkL https://gist.github.com/rejeep/ebcd57c3af83b049833b/raw > travis.sh && source ./travis.sh
  - evm install $EVM_EMACS --use --skip
  - cask
install:
  - cask install
script:
  - emacs --version
  - cask exec buttercup -L .
```

Most of the complexity here is from installing
[EVM](https://github.com/rejeep/evm) and Cask to be able to test your
project using different Emacs versions.