Import Debian changes 7.1-1

dtrx (7.1-1) unstable; urgency=medium

  * QA upload.
  * New upstream release.
  * Bump to Standards-Version 4.0.1.
  * Update compatibility level to 9.
  * Migrate debian/rules to dh.
  * Set Debian QA Group as maintainer.
  * Update debian/copyright to DEP-5.

16 files changed, 347 insertions, 181 deletions

diff --git a/INSTALL b/INSTALL
index abe9d07..c3bb66b 100644
--- a/INSTALL
+++ b/INSTALL
@@ -36,6 +36,9 @@ gem archives
 7z archives
   7z
 
+lzh archives
+  lha
+
 Microsoft Cabinet archives
   cabextract
 
diff --git a/NEWS b/NEWS
index 24448ea..92c81c0 100644
--- a/NEWS
+++ b/NEWS
@@ -1,6 +1,51 @@
 Changes in dtrx
 ===============
 
+Version 7.1
+-----------
+
+New features
+~~~~~~~~~~~~
+
+ * LZH archives are now supported.
+
+Bug fixes
+~~~~~~~~~
+
+ * dtrx will no longer offer to extract the zero archive files found in a
+   zero-file archive.
+
+ * Temporary directories will be cleaned up after extracting an empty
+   archive.
+
+Version 7.0
+-----------
+
+At this point, I consider dtrx to be mature software.  It's maybe a little
+too interactive, but otherwise it does everything I want, and it does it
+very well.  Expect new releases to be few and far between going forward.
+
+New features
+~~~~~~~~~~~~
+
+ * If any of dtrx's command line arguments are URLs, it will automatically
+   download them with `wget -c` in the current directory before extracting
+   them.  See the documentation for more information about this feature.
+   Note that there might be trouble if there's already a file in the
+   directory where wget would normally save the download.
+
+Enhancements
+~~~~~~~~~~~~
+
+ * dtrx will try to extract ZIP files with 7z if unzip is not successful.
+   Thanks to Edward H for reporting this bug.
+
+ * dtrx will be smarter about removing extensions from filenames when
+   extracting to a new directory or file.
+
+ * dtrx will not ask you if you want to recurse through an archive if
+   the number of archives inside the original file is small.
+
 Version 6.6
 -----------
 
diff --git a/PKG-INFO b/PKG-INFO
index 7c6e219..685b86d 100644
--- a/PKG-INFO
+++ b/PKG-INFO
@@ -1,6 +1,6 @@
 Metadata-Version: 1.0
 Name: dtrx
-Version: 6.6
+Version: 7.1
 Summary: Script to intelligently extract multiple archive types
 Home-page: http://www.brettcsmith.org/2007/dtrx/
 Author: Brett Smith
@@ -8,17 +8,17 @@ Author-email: brettcsmith@brettcsmith.org
 License: GNU General Public License, version 3 or later
 Download-URL: http://www.brettcsmith.org/2007/dtrx/
 Description: dtrx extracts archives in a number of different
-        formats; it currently supports tar, zip (including self-extracting
-        .exe files), cpio, rpm, deb, gem, 7z, cab, rar, and InstallShield
-        files.  It can also decompress files compressed with gzip, bzip2,
-        lzma, xz, or compress.
+              formats; it currently supports tar, zip (including self-extracting
+              .exe files), cpio, rpm, deb, gem, 7z, cab, rar, and InstallShield
+              files.  It can also decompress files compressed with gzip, bzip2,
+              lzma, xz, or compress.
         
-        In addition to providing one command to handle many different archive
-        types, dtrx also aids the user by extracting contents consistently.
-        By default, everything will be written to a dedicated directory
-        that's named after the archive.  dtrx will also change the
-        permissions to ensure that the owner can read and write all those
-        files.
+              In addition to providing one command to handle many different archive
+              types, dtrx also aids the user by extracting contents consistently.
+              By default, everything will be written to a dedicated directory
+              that's named after the archive.  dtrx will also change the
+              permissions to ensure that the owner can read and write all those
+              files.
 Platform: UNKNOWN
 Classifier: Development Status :: 5 - Production/Stable
 Classifier: Environment :: Console
diff --git a/README b/README
index 204bd2f..1cc5097 100644
--- a/README
+++ b/README
@@ -7,10 +7,10 @@ cleanly extract many archive types
 ----------------------------------
 
 :Author: Brett Smith <brettcsmith@brettcsmith.org>
-:Date:   2009-07-04
+:Date:   2011-11-19
 :Copyright:
 
-  dtrx 6.5 is copyright © 2006-2009 Brett Smith and others.  Feel free to
+  dtrx 7.1 is copyright © 2006-2011 Brett Smith and others.  Feel free to
   send comments, bug reports, patches, and so on.  You can find the latest
   version of dtrx on its home page at
   <http://www.brettcsmith.org/2007/dtrx/>.
@@ -28,7 +28,7 @@ cleanly extract many archive types
   You should have received a copy of the GNU General Public License along
   with this program; if not, see <http://www.gnu.org/licenses/>.
 
-:Version: 6.5
+:Version: 7.1
 :Manual section: 1
 
 SYNOPSIS
@@ -41,8 +41,8 @@ DESCRIPTION
 
 dtrx extracts archives in a number of different formats; it currently
 supports tar, zip (including self-extracting .exe files), cpio, rpm, deb,
-gem, 7z, cab, rar, and InstallShield files.  It can also decompress files
-compressed with gzip, bzip2, lzma, or compress.
+gem, 7z, cab, rar, lzh, and InstallShield files.  It can also decompress
+files compressed with gzip, bzip2, lzma, xz, or compress.
 
 In addition to providing one command to handle many different archive
 types, dtrx also aids the user by extracting contents consistently.  By
@@ -55,6 +55,11 @@ arguments.  For example::
 
    $ dtrx coreutils-5.*.tar.gz
 
+You may specify URLs as arguments as well.  If you do, dtrx will use `wget
+-c` to download the URL to the current directory and then extract what it
+downloads.  This may fail if you already have a file in the current
+directory with the same name as the file you're trying to download.
+
 OPTIONS
 =======
 
@@ -108,12 +113,11 @@ dtrx supports a number of options to mandate specific behavior:
    contents.
 
 -q, --quiet
-   Suppress warning messages.  Listing this option twice will cause dtrx to
-   be silent.
+   Suppress warning messages.  List this option twice to make dtrx silent.
 
 -v, --verbose
-   Show the files that are being extracted.  Listing this option twice will
-   cause dtrx to print debugging information.
+   Show the files that are being extracted.  List this option twice to
+   print debugging information.
 
 --help
    Display basic help.
diff --git a/debian/changelog b/debian/changelog
index ba9089e..ee93982 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,15 @@
+dtrx (7.1-1) unstable; urgency=medium
+
+  * QA upload.
+  * New upstream release.
+  * Bump to Standards-Version 4.0.1.
+  * Update compatibility level to 9.
+  * Migrate debian/rules to dh.
+  * Set Debian QA Group as maintainer.
+  * Update debian/copyright to DEP-5.
+
+ -- Mònica Ramírez Arceda <monica@debian.org>  Sat, 12 Aug 2017 12:31:15 -0400
+
 dtrx (6.6-1.2) unstable; urgency=low
 
   * Non-maintainer upload.
diff --git a/debian/compat b/debian/compat
index 7f8f011..ec63514 100644
--- a/debian/compat
+++ b/debian/compat
@@ -1 +1 @@
-7
+9
diff --git a/debian/control b/debian/control
index 9f7381c..7e41e15 100644
--- a/debian/control
+++ b/debian/control
@@ -1,10 +1,10 @@
 Source: dtrx
 Section: utils
 Priority: optional
-Maintainer: Miguelangel Jose Freitas Loreto <miguelangel.freitas@gmail.com>
-Build-Depends: debhelper (>= 7), python (>= 2.6.6-3~)
+Maintainer: Debian QA Group <packages@qa.debian.org> 
+Build-Depends: debhelper (>= 9), python (>= 2.6.6-3~)
 Build-Depends-Indep: python-yaml, bzip2, unzip, cpio, rpm, binutils, p7zip-full, cabextract, unshield, lzma, python-docutils (>= 0.6), xz-utils
-Standards-Version: 3.8.4
+Standards-Version: 4.0.1
 Homepage: http://brettcsmith.org/2007/dtrx/
 X-Python-Version: >= 2.4
 
diff --git a/debian/copyright b/debian/copyright
index 2cf153f..1d528f4 100644
--- a/debian/copyright
+++ b/debian/copyright
@@ -1,7 +1,7 @@
-Format-Specification: http://wiki.debian.org/Proposals/CopyrightFormat?action=recall&rev=196
+Format: http://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
 Upstream-Name: dtrx: Intelligent archive extraction
-Upstream-Maintainer: Brett Smith <brettcsmith@brettcsmith.org>
-Upstream-Source: http://brettcsmith.org/2007/dtrx/
+Upstream-Contact: Brett Smith <brettcsmith@brettcsmith.org>
+Source: http://brettcsmith.org/2007/dtrx/
 
 Files: *
 Copyright: Copyright 2006, 2007, 2008 Brett Smith <brettcsmith@brettcsmith.org>
diff --git a/debian/rules b/debian/rules
index de9c241..851c7d0 100755
--- a/debian/rules
+++ b/debian/rules
@@ -1,35 +1,10 @@
-#!/usr/bin/make -f
-# -*- makefile -*-
-#export DH_VERBOSE=1
+#! /usr/bin/make -f
 
--include /usr/share/python/python.mk
+#export DH_VERBOSE = 1
 
-build: build-indep
+%:
+	dh $@ --with python2
 
-build-indep:
-	dh build --before auto_build
-	/usr/bin/python setup.py build
-	/usr/bin/python tests/compare.py
-	dh build --after auto_build
+override_dh_auto_build:
+	dh_auto_build
 	rst2man -q README debian/dtrx.1
-	touch $@
-
-clean: 
-	dh clean --with python2
-	rm -rf $(CURDIR)/tests/testscript.sh $(CURDIR)/tests/unreadable-file.*
-	/usr/bin/python setup.py clean
-	rm -f debian/dtrx.1 build-indep
-
-install: build
-	dh install --with python2 --before auto_install
-	/usr/bin/python setup.py install --root $(CURDIR)/debian/dtrx $(py_setup_install_args)
-	dh install --with python2 --after auto_install
-
-binary: binary-arch binary-indep
-
-binary-arch:
-
-binary-indep: build install
-	dh binary-indep --with python2
-
-.PHONY: build clean binary-arch binary-indep binary install
diff --git a/scripts/dtrx b/scripts/dtrx
index 7a98ba5..2fc99e3 100755
--- a/scripts/dtrx
+++ b/scripts/dtrx
@@ -2,7 +2,7 @@
 # -*- coding: utf-8 -*-
 #
 # dtrx -- Intelligently extract various archive types.
-# Copyright © 2006-2009 Brett Smith <brettcsmith@brettcsmith.org>
+# Copyright © 2006-2011 Brett Smith <brettcsmith@brettcsmith.org>
 # Copyright © 2008 Peter Kelemen <Peter.Kelemen@gmail.com>
 #
 # This program is free software; you can redistribute it and/or modify it
@@ -38,15 +38,16 @@ import tempfile
 import termios
 import textwrap
 import traceback
+import urlparse
 
 try:
     set
 except NameError:
     from sets import Set as set
 
-VERSION = "6.6"
+VERSION = "7.1"
 VERSION_BANNER = """dtrx version %s
-Copyright © 2006-2009 Brett Smith <brettcsmith@brettcsmith.org>
+Copyright © 2006-2011 Brett Smith <brettcsmith@brettcsmith.org>
 Copyright © 2008 Peter Kelemen <Peter.Kelemen@gmail.com>
 
 This program is free software; you can redistribute it and/or modify it
@@ -81,6 +82,7 @@ RECURSE_LIST = 5
 mimetypes.encodings_map.setdefault('.bz2', 'bzip2')
 mimetypes.encodings_map.setdefault('.lzma', 'lzma')
 mimetypes.encodings_map.setdefault('.xz', 'xz')
+mimetypes.encodings_map.setdefault('.lz', 'lzip')
 mimetypes.types_map.setdefault('.gem', 'application/x-ruby-gem')
 
 logger = logging.getLogger('dtrx-log')
@@ -138,8 +140,8 @@ class ExtractorUnusable(Exception):
 EXTRACTION_ERRORS = (ExtractorError, ExtractorUnusable, OSError, IOError)
 
 class BaseExtractor(object):
-    decoders = {'bzip2': 'bzcat', 'gzip': 'zcat', 'compress': 'zcat',
-                'lzma': 'lzcat', 'xz': 'xzcat'}
+    decoders = {'bzip2': ['bzcat'], 'gzip': ['zcat'], 'compress': ['zcat'],
+                'lzma': ['lzcat'], 'xz': ['xzcat'], 'lzip': ['lzip', '-cd']}
     name_checker = DirectoryChecker
 
     def __init__(self, filename, encoding):
@@ -161,18 +163,12 @@ class BaseExtractor(object):
             raise ExtractorError("could not open %s: %s" %
                                  (filename, error.strerror))
         if encoding:
-            self.pipe([self.decoders[encoding]], "decoding")
+            self.pipe(self.decoders[encoding], "decoding")
         self.prepare()
 
     def pipe(self, command, description="extraction"):
         self.pipes.append((command, description))
 
-    def first_bad_exit_code(self):
-        for index, code in enumerate(self.exit_codes):
-            if code != 0:
-                return index
-        return None
-
     def add_process(self, processes, command, stdin, stdout):
         try:
             processes.append(subprocess.Popen(command, stdin=stdin,
@@ -243,7 +239,16 @@ class BaseExtractor(object):
 
     def basename(self):
         pieces = os.path.basename(self.filename).split('.')
+        orig_len = len(pieces)
         extension = '.' + pieces[-1]
+        # This is maybe a little more clever than it ought to be.
+        # We're trying to be conservative about what remove, but also DTRT
+        # in cases like .tar.gz, and also do something reasonable if we
+        # encounter some completely off-the-wall extension.  So that means:
+        # 1. First remove any compression extension.
+        # 2. Then remove any commonly known extension that remains.
+        # 3. If neither of those did anything, remove anything that looks
+        #    like it's almost certainly an extension (less than 5 chars).
         if mimetypes.encodings_map.has_key(extension):
             pieces.pop()
             extension = '.' + pieces[-1]
@@ -251,6 +256,9 @@ class BaseExtractor(object):
             mimetypes.common_types.has_key(extension) or
             mimetypes.suffix_map.has_key(extension)):
             pieces.pop()
+        if ((orig_len == len(pieces)) and
+            (orig_len > 1) and (len(pieces[-1]) < 5)):
+            pieces.pop()
         return '.'.join(pieces)
 
     def get_stderr(self):
@@ -259,13 +267,25 @@ class BaseExtractor(object):
         self.stderr.close()
         return errors
 
-    def check_success(self, got_output):
-        error_index = self.first_bad_exit_code()
-        if (not got_output) and (error_index is not None):
+    def is_fatal_error(self, status):
+        return False
+
+    def first_bad_exit_code(self):
+        for index, code in enumerate(self.exit_codes):
+            if code > 0:
+                return index, code
+        return None, None
+
+    def check_success(self, got_files):
+        error_index, error_code = self.first_bad_exit_code()
+        logger.debug("success results: %s %s %s" % (got_files, error_index,
+                                                    self.exit_codes))
+        if (self.is_fatal_error(error_code) or
+            ((not got_files) and (error_code is not None))):
             command = ' '.join(self.pipes[error_index][0])
             raise ExtractorError("%s error: '%s' returned status code %s" %
                                  (self.pipes[error_index][1], command,
-                                  self.exit_codes[error_index]))
+                                  error_code))
         
     def extract_archive(self):
         self.pipe(self.extract_pipe)
@@ -340,6 +360,7 @@ class CompressionExtractor(BaseExtractor):
         self.content_type = ONE_ENTRY_KNOWN
         self.content_name = self.basename()
         self.contents = None
+        self.file_count = 1
         self.included_root = './'
         try:
             output_fd, self.target = tempfile.mkstemp(prefix='.dtrx-', dir='.')
@@ -352,6 +373,7 @@ class CompressionExtractor(BaseExtractor):
         except EXTRACTION_ERRORS:
             os.unlink(self.target)
             raise
+
             
 class TarExtractor(BaseExtractor):
     file_type = 'tar file'
@@ -410,7 +432,7 @@ class DebExtractor(TarExtractor):
             raise ExtractorError("data.tar file has unrecognized encoding")
         self.pipe(['ar', 'p', self.filename, data_filename],
                   "extracting data.tar from .deb")
-        self.pipe([self.decoders[encoding]], "decoding data.tar")
+        self.pipe(self.decoders[encoding], "decoding data.tar")
 
     def basename(self):
         pieces = os.path.basename(self.filename).split('_')
@@ -486,6 +508,39 @@ class ZipExtractor(NoPipeExtractor):
     extract_command = ['unzip', '-q']
     list_command = ['zipinfo', '-1']
 
+    def is_fatal_error(self, status):
+        return status > 1
+
+
+class LZHExtractor(ZipExtractor):
+    file_type = 'LZH file'
+    extract_command = ['lha', 'xq']
+    list_command = ['lha', 'l']
+
+    def border_line_file_index(self, line):
+        last_space_index = None
+        for index, char in enumerate(line):
+            if char == ' ':
+                last_space_index = index
+            elif char != '-':
+                return None
+        if last_space_index is None:
+            return None
+        return last_space_index + 1
+
+    def get_filenames(self):
+        filenames = NoPipeExtractor.get_filenames(self)
+        for line in filenames:
+            fn_index = self.border_line_file_index(line)
+            if fn_index is not None:
+                break
+        for line in filenames:
+            if self.border_line_file_index(line):
+                break
+            else:
+                yield line[fn_index:]
+        self.archive.close()
+
 
 class SevenExtractor(NoPipeExtractor):
     file_type = '7z file'
@@ -673,7 +728,9 @@ class EmptyHandler(object):
         return contents == EMPTY
     can_handle = staticmethod(can_handle)
 
-    def __init__(self, extractor, options): pass
+    def __init__(self, extractor, options):
+        os.rmdir(extractor.target)
+
     def handle(self): pass
 
 
@@ -799,7 +856,8 @@ class RecursionPolicy(BasePolicy):
 
     def prep(self, current_filename, target, extractor):
         archive_count = len(extractor.included_archives)
-        if (self.permanent_policy is not None) or (archive_count == 0):
+        if ((self.permanent_policy is not None) or
+            ((archive_count * 10) <= extractor.file_count)):
             self.current_policy = self.permanent_policy or RECURSE_NOT_NOW
             return
         question = self.wrap(
@@ -825,48 +883,56 @@ class RecursionPolicy(BasePolicy):
             
 
 class ExtractorBuilder(object):
-    extractor_map = {'tar': {'extractor': TarExtractor,
+    extractor_map = {'tar': {'extractors': (TarExtractor,),
                              'mimetypes': ('x-tar',),
                              'extensions': ('tar',),
                              'magic': ('POSIX tar archive',)},
-                     'zip': {'extractor': ZipExtractor,
+                     'zip': {'extractors': (ZipExtractor, SevenExtractor),
                              'mimetypes': ('zip',),
                              'extensions': ('zip',),
                              'magic': ('(Zip|ZIP self-extracting) archive',)},
-                     'rpm': {'extractor': RPMExtractor,
+                     'lzh': {'extractors': (LZHExtractor,),
+                             'mimetypes': ('x-lzh', 'x-lzh-compressed'),
+                             'extensions': ('lzh', 'lha'),
+                             'magic': ('LHa [\d\.\?]+ archive',)},
+                     'rpm': {'extractors': (RPMExtractor,),
                              'mimetypes': ('x-redhat-package-manager', 'x-rpm'),
                              'extensions': ('rpm',),
                              'magic': ('RPM',)},
-                     'deb': {'extractor': DebExtractor,
-                             'metadata': DebMetadataExtractor,
+                     'deb': {'extractors': (DebExtractor,),
+                             'metadata': (DebMetadataExtractor,),
                              'mimetypes': ('x-debian-package',),
                              'extensions': ('deb',),
                              'magic': ('Debian binary package',)},
-                     'cpio': {'extractor': CpioExtractor,
+                     'cpio': {'extractors': (CpioExtractor,),
                               'mimetypes': ('x-cpio',),
                               'extensions': ('cpio',),
                               'magic': ('cpio archive',)},
-                     'gem': {'extractor': GemExtractor,
-                             'metadata': GemMetadataExtractor,
+                     'gem': {'extractors': (GemExtractor,),
+                             'metadata': (GemMetadataExtractor,),
                              'mimetypes': ('x-ruby-gem',),
                              'extensions': ('gem',)},
-                     '7z': {'extractor': SevenExtractor,
+                     '7z': {'extractors': (SevenExtractor,),
                              'mimetypes': ('x-7z-compressed',),
                              'extensions': ('7z',),
                              'magic': ('7-zip archive',)},
-                     'cab': {'extractor': CABExtractor,
+                     'cab': {'extractors': (CABExtractor,),
                              'mimetypes': ('x-cab',),
                              'extensions': ('cab',),
                              'magic': ('Microsoft Cabinet Archive',)},
-                     'rar': {'extractor': RarExtractor,
+                     'rar': {'extractors': (RarExtractor,),
                              'mimetypes': ('rar',),
                              'extensions': ('rar',),
                              'magic': ('RAR archive',)},
-                     'shield': {'extractor': ShieldExtractor,
+                     'shield': {'extractors': (ShieldExtractor,),
                                 'mimetypes': ('x-cab',),
                                 'extensions': ('cab', 'hdr'),
                                 'magic': ('InstallShield CAB',)},
-                     'compress': {'extractor': CompressionExtractor}
+                     'msi': {'extractors': (SevenExtractor,),
+                             'mimetypes': ('x-msi', 'x-ole-storage'),
+                             'extensions': ('msi',),
+                             'magic': ('Application: Windows Installer',)},
+                     'compress': {'extractors': (CompressionExtractor,)}
                      }
 
     mimetype_map = {}
@@ -886,6 +952,7 @@ class ExtractorBuilder(object):
                     ('tar', 'gzip', 'tar.gz', 'tgz'),
                     ('tar', 'lzma', 'tar.lzma', 'tlz'),
                     ('tar', 'xz', 'tar.xz'),
+                    ('tar', 'lz', 'tar.lz'),
                     ('tar', 'compress', 'tar.Z', 'taz'),
                     ('compress', 'gzip', 'Z', 'gz'),
                     ('compress', 'bzip2', 'bz2'),
@@ -898,6 +965,7 @@ class ExtractorBuilder(object):
     for mapping in (('bzip2', 'bzip2 compressed'),
                     ('gzip', 'gzip compressed'),
                     ('lzma', 'LZMA compressed'),
+                    ('lzip', 'lzip compressed'),
                     ('xz', 'xz compressed')):
         for pattern in mapping[1:]:
             magic_encoding_map[re.compile(pattern)] = mapping[0]
@@ -907,12 +975,13 @@ class ExtractorBuilder(object):
         self.options = options
 
     def build_extractor(self, archive_type, encoding):
-        extractors = self.extractor_map[archive_type]
-        if self.options.metadata and extractors.has_key('metadata'):
-            extractor = extractors['metadata']
+        type_info = self.extractor_map[archive_type]
+        if self.options.metadata and type_info.has_key('metadata'):
+            extractors = type_info['metadata']
         else:
-            extractor = extractors['extractor']
-        return extractor(self.filename, encoding)
+            extractors = type_info['extractors']
+        for extractor in extractors:
+            yield extractor(self.filename, encoding)
 
     def get_extractor(self):
         tried_types = set()
@@ -932,7 +1001,8 @@ class ExtractorBuilder(object):
                 tried_types.add(ext_args)
                 logger.debug("trying %s extractor from %s" %
                              (ext_args, func_name))
-                yield self.build_extractor(*ext_args)
+                for extractor in self.build_extractor(*ext_args):
+                    yield extractor
 
     def try_by_mimetype(cls, filename):
         mimetype, encoding = mimetypes.guess_type(filename)
@@ -1228,6 +1298,21 @@ class ExtractorApplication(object):
             self.show_stderr(logger.error, stderr)
         return True
         
+    def download(self, filename):
+        url = filename.lower()
+        for protocol in 'http', 'https', 'ftp':
+            if url.startswith(protocol + '://'):
+                break
+        else:
+            return filename, None
+        # FIXME: This can fail if there's already a file in the directory
+        # that matches the basename of the URL.
+        status = subprocess.call(['wget', '-c', filename],
+                                 stdin=subprocess.PIPE)
+        if status != 0:
+            return None, "wget returned status code %s" % (status,)
+        return os.path.basename(urlparse.urlparse(filename)[2]), None
+
     def run(self):
         if self.options.show_list:
             action = ListAction
@@ -1238,9 +1323,12 @@ class ExtractorApplication(object):
             self.current_directory, self.filenames = self.archives.popitem()
             os.chdir(self.current_directory)
             for filename in self.filenames:
-                builder = ExtractorBuilder(filename, self.options)
-                error = (self.check_file(filename) or
-                         self.try_extractors(filename, builder.get_extractor()))
+                filename, error = self.download(filename)
+                if not error:
+                    builder = ExtractorBuilder(filename, self.options)
+                    error = (self.check_file(filename) or
+                             self.try_extractors(filename,
+                                                 builder.get_extractor()))
                 if error:
                     if error != True:
                         logger.error("%s: %s" % (filename, error))
diff --git a/setup.py b/setup.py
index b8ab4e7..bda1d70 100644
--- a/setup.py
+++ b/setup.py
@@ -3,7 +3,7 @@
 from distutils.core import setup
 
 setup(name="dtrx",
-      version = "6.6",
+      version = "7.1",
       description = "Script to intelligently extract multiple archive types",
       author = "Brett Smith",
       author_email = "brettcsmith@brettcsmith.org",
diff --git a/tests/compare.py b/tests/compare.py
index 56122aa..6a92fd1 100644
--- a/tests/compare.py
+++ b/tests/compare.py
@@ -24,7 +24,10 @@ import yaml
 import sys
 import tempfile
 
-from sets import Set as set
+try:
+    set
+except NameError:
+    from sets import Set as set
 
 if os.path.exists('scripts/dtrx') and os.path.exists('tests'):
     os.chdir('tests')
@@ -34,16 +37,10 @@ else:
     print "ERROR: Can't run tests in this directory!"
     sys.exit(2)
 
-X_SCRIPT = os.path.realpath('../scripts/dtrx')
+DTRX_SCRIPT = os.path.realpath('../scripts/dtrx')
+SHELL_CMD = ['sh', '-se']
 ROOT_DIR = os.path.realpath(os.curdir)
 OUTCOMES = ['error', 'failed', 'passed']
-TESTSCRIPT_NAME = 'testscript.sh'
-SCRIPT_PROLOGUE = """#!/bin/sh
-set -e
-"""
-
-input_buffer = tempfile.TemporaryFile()
-output_buffer = tempfile.TemporaryFile()
 
 class ExtractorTestError(Exception):
     pass
@@ -62,25 +59,27 @@ class ExtractorTest(object):
             if isinstance(value, str):
                 value = [value]
             setattr(self, key, value)
-        
-    def get_results(self, commands, stdin=None):
-        print >>output_buffer, "Output from %s:" % (' '.join(commands),)
-        output_buffer.flush()
-        status = subprocess.call(commands, stdout=output_buffer,
-                                 stderr=output_buffer, stdin=stdin)
-        process = subprocess.Popen(['find', '!', '-name', TESTSCRIPT_NAME],
-                                   stdout=subprocess.PIPE)
-        process.wait()
+        if self.input and (not self.input.endswith('\n')):
+            self.input = self.input + '\n'
+
+    def start_proc(self, command, stdin=None, output=None):
+        process = subprocess.Popen(command, stdin=subprocess.PIPE,
+                                   stdout=output, stderr=output)
+        if stdin:
+            process.stdin.write(stdin)
+        process.stdin.close()
+        return process
+
+    def get_results(self, command, stdin=None):
+        print >>self.outbuffer, "Output from %s:" % (' '.join(command),)
+        self.outbuffer.flush()
+        status = self.start_proc(command, stdin, self.outbuffer).wait()
+        process = subprocess.Popen(['find'], stdout=subprocess.PIPE)
         output = process.stdout.read(-1)
         process.stdout.close()
+        process.wait()
         return status, set(output.split('\n'))
         
-    def write_script(self, commands):
-        script = open(TESTSCRIPT_NAME, 'w')
-        script.write("%s%s\n" % (SCRIPT_PROLOGUE, commands))
-        script.close()
-        subprocess.call(['chmod', 'u+w', TESTSCRIPT_NAME])
-
     def run_script(self, key):
         commands = getattr(self, key)
         if commands is not None:
@@ -88,38 +87,27 @@ class ExtractorTest(object):
                 directory_hint = '../'
             else:
                 directory_hint = ''
-            self.write_script(commands)
-            subprocess.call(['sh', TESTSCRIPT_NAME, directory_hint])
+            self.start_proc(SHELL_CMD + [directory_hint], commands)
 
     def get_shell_results(self):
         self.run_script('prerun')
-        self.write_script(self.baseline)
-        return self.get_results(['sh', TESTSCRIPT_NAME] + self.filenames)
+        return self.get_results(SHELL_CMD + self.filenames, self.baseline)
 
     def get_extractor_results(self):
         self.run_script('prerun')
-        input_buffer.seek(0, 0)
-        input_buffer.truncate()
-        if self.input:
-            input_buffer.write(self.input)
-            if not self.input.endswith('\n'):
-                input_buffer.write('\n')
-            input_buffer.seek(0, 0)
-        input_buffer.flush()
-        return self.get_results([X_SCRIPT] + self.options + self.filenames,
-                                input_buffer)
+        return self.get_results([DTRX_SCRIPT] + self.options + self.filenames,
+                                self.input)
         
     def get_posttest_result(self):
         if not self.posttest:
             return 0
-        self.write_script(self.posttest)
-        return subprocess.call(['sh', TESTSCRIPT_NAME])
+        return self.start_proc(SHELL_CMD, self.posttest).wait()
 
     def clean(self):
         self.run_script('cleanup')
         if self.directory:
             target = os.path.join(ROOT_DIR, self.directory)
-            extra_options = ['!', '-name', TESTSCRIPT_NAME]
+            extra_options = []
         else:
             target = ROOT_DIR
             extra_options = ['(', '(', '-type', 'd',
@@ -138,8 +126,8 @@ class ExtractorTest(object):
     def show_status(self, status, message=None):
         raw_status = status.lower()
         if raw_status != 'passed':
-            output_buffer.seek(0, 0)
-            sys.stdout.write(output_buffer.read(-1))
+            self.outbuffer.seek(0, 0)
+            sys.stdout.write(self.outbuffer.read(-1))
         if message is None:
             last_part = ''
         else:
@@ -153,13 +141,13 @@ class ExtractorTest(object):
         status, expected = self.get_shell_results()
         self.clean()
         if expected != actual:
-            print >>output_buffer, "Only in baseline results:"
-            print >>output_buffer, '\n'.join(expected.difference(actual))
-            print >>output_buffer, "Only in actual results:"
-            print >>output_buffer, '\n'.join(actual.difference(expected))
+            print >>self.outbuffer, "Only in baseline results:"
+            print >>self.outbuffer, '\n'.join(expected.difference(actual))
+            print >>self.outbuffer, "Only in actual results:"
+            print >>self.outbuffer, '\n'.join(actual.difference(expected))
             return self.show_status('FAILED')
         elif posttest_result != 0:
-            print >>output_buffer, "Posttest gave status code", posttest_result
+            print >>self.outbuffer, "Posttest gave status code", posttest_result
             return self.show_status('FAILED')
         return self.show_status('Passed')
     
@@ -187,24 +175,23 @@ class ExtractorTest(object):
         return None
 
     def check_results(self):
-        output_buffer.seek(0, 0)
-        output_buffer.truncate()
         self.clean()
         status, actual = self.get_extractor_results()
-        output_buffer.seek(0, 0)
-        output_buffer.readline()
-        output = output_buffer.read(-1)
+        self.outbuffer.seek(0, 0)
+        self.outbuffer.readline()
+        output = self.outbuffer.read(-1)
         problem = (self.have_error_mismatch(status) or
                    self.check_output(output) or self.grep_output(output))
         if problem:
             return self.show_status('FAILED', problem)
-        if self.baseline:
+        if self.baseline is not None:
             return self.compare_results(actual)
         else:
             self.clean()
             return self.show_status('Passed')
 
     def run(self):
+        self.outbuffer = tempfile.TemporaryFile()
         if self.directory:
             os.mkdir(self.directory)
             os.chdir(self.directory)
@@ -212,6 +199,7 @@ class ExtractorTest(object):
             result = self.check_results()
         except ExtractorTestError, error:
             result = self.show_status('ERROR', error)
+        self.outbuffer.close()
         if self.directory:
             os.chdir(ROOT_DIR)
             subprocess.call(['chmod', '-R', '700', self.directory])
@@ -240,5 +228,3 @@ for outcome in OUTCOMES:
 for result in results:
     counts[result] += 1
 print " Totals:", ', '.join(["%s %s" % (counts[key], key) for key in OUTCOMES])
-input_buffer.close()
-output_buffer.close()
diff --git a/tests/test-1.23.lzh b/tests/test-1.23.lzh
new file mode 100644
index 0000000..942e2cb
--- /dev/null
+++ b/tests/test-1.23.lzh
diff --git a/tests/test-recursive-no-prompt.tar.bz2 b/tests/test-recursive-no-prompt.tar.bz2
new file mode 100644
index 0000000..ff439e5
--- /dev/null
+++ b/tests/test-recursive-no-prompt.tar.bz2
diff --git a/tests/test-text.lz b/tests/test-text.lz
new file mode 100644
index 0000000..b1d0f2f
--- /dev/null
+++ b/tests/test-text.lz
diff --git a/tests/tests.yml b/tests/tests.yml
index 95f0aef..462baf3 100644
--- a/tests/tests.yml
+++ b/tests/tests.yml
@@ -22,6 +22,13 @@
     cd test-1.23
     unzip -q ../$1
 
+- name: basic .lzh
+  filenames: test-1.23.lzh
+  baseline: |
+    mkdir test-1.23
+    cd test-1.23
+    lha xq ../$1
+
 - name: basic .deb
   filenames: test-1.23_all.deb
   baseline: |
@@ -74,46 +81,63 @@
     tar -xOf $1 metadata.gz | zcat > test-1.23.gem-metadata.txt
   cleanup: rm -f test-1.23.gem-metadata.txt
   posttest: |
-    if [ "x`cat test-1.23.gem-metadata.txt`" != "xhi" ]; then exit 1; fi
+    exec [ "$(cat test-1.23.gem-metadata.txt)" = "hi" ]
 
 - name: recursion and permissions
   filenames: test-recursive-badperms.tar.bz2
   options: -n -r
   baseline: |
-    mkdir test-recursive-badperms
-    cd test-recursive-badperms
-    tar -jxf ../$1
-    mkdir test-badperms
-    cd test-badperms
-    tar -xf ../test-badperms.tar
+    extract() {
+      mkdir "$1"
+      cd "$1"
+      tar "-${3}xf" "../$2"
+    }
+    extract test-recursive-badperms "$1" j
+    extract test-badperms test-badperms.tar
     chmod 700 testdir
   posttest: |
-    if [ "x`cat test-recursive-badperms/test-badperms/testdir/testfile`" != \
-         "xhey" ]; then exit 1; fi
+    exec [ "$(cat test-recursive-badperms/test-badperms/testdir/testfile)" = \
+           "hey" ]
 
-- name: decompressing gz
+- name: decompressing gz, not interactive
   directory: inside-dir
   filenames: ../test-text.gz
+  options: ""
+  antigrep: "."
   baseline: |
     zcat $1 >test-text
   posttest: |
-    if [ "x`cat test-text`" != "xhi" ]; then exit 1; fi
+    exec [ "$(cat test-text)" = "hi" ]
 
-- name: decompressing bz2
+- name: decompressing bz2, not interactive
   directory: inside-dir
   filenames: ../test-text.bz2
+  options: ""
+  antigrep: "."
   baseline: |
     bzcat $1 >test-text
   posttest: |
-    if [ "x`cat test-text`" != "xhi" ]; then exit 1; fi
+    exec [ "$(cat test-text)" = "hi" ]
 
-- name: decompressing xz
+- name: decompressing xz, not interactive
   directory: inside-dir
   filenames: ../test-text.xz
+  options: ""
+  antigrep: "."
   baseline: |
     xzcat $1 >test-text
   posttest: |
-    if [ "x`cat test-text`" != "xhi" ]; then exit 1; fi
+    exec [ "$(cat test-text)" = "hi" ]
+
+- name: decompressing lzip, not interactive
+  directory: inside-dir
+  filenames: ../test-text.lz
+  options: ""
+  antigrep: "."
+  baseline: |
+    lzip -cd <$1 >test-text
+  posttest: |
+    exec [ "$(cat test-text)" = "hi" ]
 
 - name: decompression with -r
   directory: inside-dir
@@ -163,7 +187,7 @@
     tar -xf test-badperms.tar
     chmod 700 testdir
   posttest: |
-    if [ "x`cat testdir/testfile`" != "xhey" ]; then exit 1; fi
+    exec [ "$(cat testdir/testfile)" = "hey" ]
 
 - name: no files
   error: true
@@ -222,6 +246,17 @@
     test-1.23/a/b
     test-1.23/foobar
 
+- name: list contents of LZH
+  options: -n -l
+  filenames: test-1.23.lzh
+  output: |
+    1/
+    1/2/
+    1/2/3
+    a/
+    a/b
+    foobar
+
 - name: list contents of .cpio
   options: -n -l
   filenames: test-1.23.cpio
@@ -434,17 +469,16 @@
     i
     n
   baseline: |
-    mkdir test-recursive-badperms
-    cd test-recursive-badperms
-    tar -jxf ../$1
-    mkdir test-badperms
-    cd test-badperms
-    tar -xf ../test-badperms.tar
+    extract() {
+      mkdir "$1"
+      cd "$1"
+      tar "-${3}xf" "../$2"
+    }
+    extract test-recursive-badperms "$1" j
+    extract test-badperms test-badperms.tar
     chmod 700 testdir
     cd ../..
-    mkdir test-recursive-badperms.1
-    cd test-recursive-badperms.1
-    tar -jxf ../$1
+    extract test-recursive-badperms.1 "$1" j
 
 - name: interactive recursion (never)
   filenames: test-recursive-badperms.tar.bz2 test-recursive-badperms.tar.bz2
@@ -685,9 +719,28 @@
 
 - name: extracting empty archive
   filenames: test-empty.tar.bz2
+  options: ""
   baseline: ""
+  antigrep: '.'
 
 - name: listing empty archive
   filenames: test-empty.tar.bz2
   options: -l
   antigrep: '.'
+
+- name: download and extract
+  filenames: http://brettcsmith.org/2007/dtrx/test-download.gz
+  directory: inside-dir
+  baseline: |
+    wget "$1"
+    zcat test-download.gz >test-download
+  cleanup: rm -f test-download.gz test-download
+
+- name: recursive archive without prompt
+  filenames: test-recursive-no-prompt.tar.bz2
+  options: ""
+  baseline: |
+    mkdir test-recursive-no-prompt
+    cd test-recursive-no-prompt
+    tar -jxf ../$1
+  antigrep: '.'