dh_elpa_test: add --autopkgtest for autodep8

Squashed commit of barely-readable autodep8 branch history, below.

commit 8d2315062dca79183d04fa944966d984620301c2
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Aug 1 18:43:07 2016 -0700

    talk about "configuration keys" for consistency

commit db0156d9e28c2aaf8b233d729172ed5cf92f01a1
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Aug 1 18:41:31 2016 -0700

    s/environment variable/configuration key/

commit 8399f5090d1fbcb058f4b3f03224832c87699a5d
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Aug 1 17:48:58 2016 -0700

    remove superfluour env var check

commit 0f5ce047796c87beaa3008910cfa62f54c9c9292
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Aug 1 17:48:15 2016 -0700

    rm TODO

commit 445b94126d4f28706b39334562fe0f84b1ac1808
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Aug 1 17:42:40 2016 -0700

    DH_ELPA_TEST_AUTOPKGTEST_KEEP never existed

commit 3cd551acf8f85da78a04d2cf75fe880063af6ab6
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Aug 1 17:34:01 2016 -0700

    yet another env var -> config key update

commit 868d3b44b4c10826e0135b1c9cd4abae23428b02
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Aug 1 17:32:00 2016 -0700

    another env var -> config key update

commit 133d23355d05115cf18774cf853fa6e4714e0b7c
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Aug 1 17:31:28 2016 -0700

    bold dh_elpa_test in manpage

commit 5db2773be5631eae6c27940217375a6fec32465b
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Aug 1 17:30:29 2016 -0700

    drop pointless changelog entry

commit dd2dad1301f89cb85514dc418023c586df935976
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Aug 1 17:29:39 2016 -0700

    drop Makefile env var reading hacks

commit bc5a46c69460d43b20f8380cf785149816a9b63c
Merge: 17cbb6a c863c90
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Aug 1 17:24:23 2016 -0700

    Merge branch 'master' into autodep8

commit 17cbb6af2b05895a9b5908ebbe8de4589dca95ec
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Tue Jun 14 16:59:07 2016 +0900

    tidy up explanatory comments

commit 1a3930d549160775f1e55e587528d52510e2fb8d
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Tue Jun 14 16:46:42 2016 +0900

    read cmdline opts once at beginning of script

commit 97083171123ba64cf09012617804113891b11f42
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Tue Jun 14 16:43:26 2016 +0900

    save & restore stderr handle

    emacs --batch and ERT both produce a lot of output on stderr even when
    there has been no error, and adt-run interprets this as a test failure.

commit 37825508c539f92b2535eabeac2b9911da0a371f
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Tue Jun 14 15:39:19 2016 +0900

    Revert "try to apply quilt patches"

    This reverts commit 987359a296d9e9e1f796507f40a22028f76fc6da.

    adt-run does this for us if we invoke it correctly.

commit dbe3d3ccdf617cf026b78b9778cf246a1f0855cf
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Tue Jun 14 15:19:51 2016 +0900

    fix clean-up renaming

commit 4d91d1543e5435b27c3d9b94a8277c623c91c94c
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Tue Jun 14 15:18:29 2016 +0900

    wrap clean-up code in an END block

    This ensures that if the test suite fails, the files get renamed back
    anyway.

commit 987359a296d9e9e1f796507f40a22028f76fc6da
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Jun 13 22:48:25 2016 +0900

    try to apply quilt patches

commit 84208f9f7671e35a842d1c81c2597ee2d9e10094
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Jun 13 22:37:15 2016 +0900

    TODO

commit 10512ca632def4f17b4c522e9ec04d8f83a8a599
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Jun 13 22:35:15 2016 +0900

    tidy imports

commit 5bf58ea014b08b9ce040ff3ca8502a0475c22ea5
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Jun 13 22:30:28 2016 +0900

    document current glob behaviour

    Not sure if this is the most sensible behaviour to have.

commit ab04676c8611b1c7e8cfc379b641c660ea0d06cf
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Jun 13 22:30:19 2016 +0900

    finding & renaming code

commit f3ea2bceca52ba486eeb580bc6977d1681e78571
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Jun 13 22:20:39 2016 +0900

    changelog tweak

commit 4c2c048ed70980a8c4bb108ff4a3fcd6df61a7d8
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Jun 13 22:18:30 2016 +0900

    fix syntax error from rebase

commit bb9adfce53121f6dd030f4af881a86e794980d8d
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Jun 13 16:51:29 2016 +0900

    we will rename the files, not move them

commit 2e9f74eeef89f5c2804e85f5f601a310ec3378af
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Jun 13 16:46:45 2016 +0900

    finish makefile_set_env_var

commit c37c56d486c4f1254e019f2328b652f58b5b7210
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Jun 13 16:43:07 2016 +0900

    new dependency on Capture::Tiny

commit 8c678862a3aeaef3033e49f76a0f689b46147420
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Jun 13 15:31:08 2016 +0900

    link to dh_auto_test.1 in dh_elpa_test.1 intro

commit 8fbc9aea8a023826e8dacf75a4ceb04c32b8aab6
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Jun 13 15:27:26 2016 +0900

    new env var: DH_ELPA_TEST_AUTOPKGTEST_KEEP

commit 449968375ba6cd73b0fd6205e3179a6a2174571f
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Mon Jun 13 15:22:04 2016 +0900

    script structure for --autopkgtest

commit 984315e243523b1f207e4bde4b5c11fb9657b518
Author: Sean Whitton <spwhitton@spwhitton.name>
Date:   Tue Jun 7 13:50:32 2016 +0900

    changelog & POD for --autopkgtest

1 files changed, 118 insertions, 12 deletions

diff --git a/dh_elpa_test b/dh_elpa_test
index 170932e..09fe39d 100755
--- a/dh_elpa_test
+++ b/dh_elpa_test
@@ -8,16 +8,17 @@ dh_elpa_test - run ELPA package testsuites
 
 use strict;
 use warnings;
+no warnings "experimental::smartmatch";
 
 =head1 SYNOPSIS
 
-B<dh_elpa_test> [S<I<debhelper options>>]  [S<I<pkg-file>>]
+B<dh_elpa_test> [S<I<debhelper options>>] [S<I<--autopkgtest>>] [S<I<pkg-file>>]
 
 =head1 DESCRIPTION
 
 B<dh_elpa_test> is a debhelper program that is responsible for running
 the testsuites of ELPA packages, when those test suites use ERT or
-buttercup(1).  dh_auto_test is rarely suitable.
+buttercup(1).  dh_auto_test(1) is rarely suitable.
 
 Testing with buttercup(1) will be activated if the package
 build-depends on elpa-buttercup.  Testing with ERT will be activated
@@ -50,7 +51,7 @@ can figure out how to run the test suite itself.
 
 =item B<disable>
 
-If this variable is set to any value, dh_elpa(1) will not invoke
+If this key is set to any value, dh_elpa(1) will not invoke
 B<dh_elpa_test>.
 
 =item B<buttercup_load_path>
@@ -63,7 +64,7 @@ B<-L> command line argument.
 
 A comma-separated list of Emacs regular expressions jointly matching
 all and only the files containing Buttercup tests that you wish to
-run.  If this variable is not defined, all tests that can be found
+run.  If this key is not defined, all tests that can be found
 will be run.  Will be passed to buttercup(1) with its B<-p> command
 line argument.
 
@@ -72,10 +73,13 @@ line argument.
 A comma-separated list of file globs matching files containing ERT
 tests that should not be run.
 
+The '*' character in globs in this configuration key does NOT match
+'/' directory separators.
+
 =item B<ert_helper>
 
 The name of a *.el file containing Emacs Lisp code that will run the
-ERT test suite.  When this variable is not defined, B<dh_elpa_test>
+ERT test suite.  When this key is not defined, B<dh_elpa_test>
 calls the function B<(ert-run-tests-batch-and-exit)>.
 
 Note that this is not the way to load test helpers that do needed work
@@ -97,15 +101,40 @@ instance spawned by B<dh_elpa_test> to run those tests.
 A comma-separated list of directories to add to the load-path when
 B<dh_elpa_test> invokes Emacs to run ERT tests.
 
+=item B<autopkgtest_keep>
+
+A comma-separated list of file globs matching files that should not be
+moved out of the source tree before running DEP8 tests (see the
+B<--autopkgtest> option below).
+
+The '*' character in globs in this configuration key matches '/'
+directory separators.
+
+=back
+
+=head1 OPTIONS
+
+=over 4
+
+=item B<--autopkgtest>
+
+Operate in autopkgtest mode.  B<dh_elpa_test> will rename non-test *.el
+files so that Emacs will not load them.  This ensures that the test
+test the installed binary package rather than the unpacked source
+package.
+
+To exclude files from this removal, specify them in the
+B<autopkgtest_keep> configuration option (see above).
+
 =back
 
 =head1 ENVIRONMENT VARIABLES
 
 Older versions of B<dh_elpa_test> were configured using environment
-variables corresponding to the above configuration keys.  For example,
-the B<ert_eval> configuration key replaced environment variable
-B<DH_ELPA_TEST_ERT_EVAL>.  Their use is now deprecated.  If the
-debian/elpa-test file exists, they will be ignored.
+variables corresponding to some of the above configuration keys.  For
+example, the B<ert_eval> configuration key replaced environment
+variable B<DH_ELPA_TEST_ERT_EVAL>.  Their use is now deprecated.  If
+the debian/elpa-test file exists, they will be ignored.
 
 =cut
 
@@ -139,6 +168,9 @@ sub long_name {
 
 # ---- Script setup
 
+# check command line opts for autopkgtest mode
+my $autopkgtest = ( "--autopkgtest" ~~ @ARGV );
+
 # This checks whether the debhelper compat is 10 or above.  With
 # debhelper compat below 9, dh_elpa_test will get run more in more
 # than one sequence, including inside fakeroot, which can cause tests
@@ -147,9 +179,6 @@ if ( get_buildoption("nocheck") || compat(9) ) {
 	exit 0;
 }
 
-my $control = Debian::Control->new();
-$control->read("debian/control");
-
 # note that we don't need package.elpa-test since we are testing all
 # lisp we can find in the *source* package
 my $options = Config::Tiny->new;
@@ -174,11 +203,72 @@ if ( -f "debian/elpa-test") {
     }
 }
 
+my @non_test_files;
+if ($autopkgtest) {
+    # Since we were not invoked by the dh sequencer, we need to check
+    # here whether tests have been disabled.  Note that we can't
+    # replace the test in elpa.pm with this one because we can't
+    # (cleanly) disable dh_auto_test from within dh_elpa_test
+    if ( defined $options->{_}->{ "disable" } ) {
+        exit 0;
+    }
+
+    my $rule = File::Find::Rule->new;
+    $rule
+      ->or(File::Find::Rule
+           ->name('.pc', 'debian')
+           ->directory->prune->discard,
+           File::Find::Rule->new);
+    $rule
+      ->file()
+      ->name( '*.el' )
+      ->none(File::Find::Rule->grep( "ert-deftest" ), # ERT
+             File::Find::Rule->grep( "\\(describe \"" ) # Buttercup
+            );
+
+    if ( defined $options->{_}->{'autopkgtest_keep'} ) {
+        foreach my $glob (split(',', $options->{_}->{'autopkgtest_keep'} )) {
+            # We want the user to be able to say
+            #     autopkgtest_keep = test/resources/*
+            # and have that affect all subdirs of resources
+            $Text::Glob::strict_wildcard_slash = 0;
+            # TODO should we do this for ert_exclude too?
+            $rule->not_exec( long_name($glob) );
+        }
+    }
+
+    @non_test_files = $rule->in('.');
+
+    foreach my $old ( @non_test_files ) {
+        my $new = $old =~ s|\.el$|.disabled|r;
+        rename $old, $new or die "Failed to rename: $!\n";
+    }
+}
+
 # these are from dh_elpa
 # TODO have dh_elpa export them so we can import them, instead of copy/paste
 my $dhelpadir="/usr/share/emacs/site-lisp/elpa";
 my $elpadir="/usr/share/emacs/site-lisp/elpa-src";
 
+my $control = Debian::Control->new();
+$control->read("debian/control");
+
+if ($autopkgtest) {
+    # save stderr handle for restoration
+    {
+        no warnings 'once';
+        open(OLDERR, ">&", \*STDERR);
+    }
+    # redirect stderr to stdout while we run our tests
+    # emacs --batch and ERT itself both produce a lot of output on
+    # stderr even when there has been no error, and adt-run interprets
+    # this as a test failure
+    open(STDERR, ">&STDOUT");
+    # unbuffer them to ensure lines appear in the right order
+    select STDERR; $| = 1;
+    select STDOUT; $| = 1;
+}
+
 # ---- Buttercup
 
 if ($control->source->Build_Depends->has( "elpa-buttercup" )) {
@@ -256,3 +346,19 @@ if (@ert_files) {
 
     print_and_doit(@args);
 }
+
+# ---- Cleanup
+
+END {
+    if ($autopkgtest) {
+        # restore stderr now we've finished running tests
+        open(STDERR, ">&OLDERR");
+
+        # Restore files that we renamed so that we comply with DEP8
+        # rw-build-tree
+        foreach my $new ( @non_test_files ) {
+            my $old = $new =~ s|\.el$|.disabled|r;
+            rename $old, $new or die "Failed to rename: $!\n";
+        }
+    }
+}