From cabeb85dd040bd7363ccf36ccf65e54cccde067d Mon Sep 17 00:00:00 2001 From: Dmitry Bogatov Date: Sun, 2 Dec 2018 05:36:58 +0000 Subject: Import Upstream version 7.1 --- INSTALL | 3 + NEWS | 45 +++++++++ PKG-INFO | 22 ++--- README | 22 +++-- scripts/dtrx | 170 +++++++++++++++++++++++++-------- setup.py | 2 +- tests/compare.py | 98 ++++++++----------- tests/test-1.23.lzh | Bin 0 -> 314 bytes tests/test-recursive-no-prompt.tar.bz2 | Bin 0 -> 295 bytes tests/test-text.lz | Bin 0 -> 39 bytes tests/tests.yml | 103 +++++++++++++++----- 11 files changed, 322 insertions(+), 143 deletions(-) create mode 100644 tests/test-1.23.lzh create mode 100644 tests/test-recursive-no-prompt.tar.bz2 create mode 100644 tests/test-text.lz diff --git a/INSTALL b/INSTALL index abe9d07..c3bb66b 100644 --- a/INSTALL +++ b/INSTALL @@ -36,6 +36,9 @@ gem archives 7z archives 7z +lzh archives + lha + Microsoft Cabinet archives cabextract diff --git a/NEWS b/NEWS index 24448ea..92c81c0 100644 --- a/NEWS +++ b/NEWS @@ -1,6 +1,51 @@ Changes in dtrx =============== +Version 7.1 +----------- + +New features +~~~~~~~~~~~~ + + * LZH archives are now supported. + +Bug fixes +~~~~~~~~~ + + * dtrx will no longer offer to extract the zero archive files found in a + zero-file archive. + + * Temporary directories will be cleaned up after extracting an empty + archive. + +Version 7.0 +----------- + +At this point, I consider dtrx to be mature software. It's maybe a little +too interactive, but otherwise it does everything I want, and it does it +very well. Expect new releases to be few and far between going forward. + +New features +~~~~~~~~~~~~ + + * If any of dtrx's command line arguments are URLs, it will automatically + download them with `wget -c` in the current directory before extracting + them. See the documentation for more information about this feature. + Note that there might be trouble if there's already a file in the + directory where wget would normally save the download. + +Enhancements +~~~~~~~~~~~~ + + * dtrx will try to extract ZIP files with 7z if unzip is not successful. + Thanks to Edward H for reporting this bug. + + * dtrx will be smarter about removing extensions from filenames when + extracting to a new directory or file. + + * dtrx will not ask you if you want to recurse through an archive if + the number of archives inside the original file is small. + Version 6.6 ----------- diff --git a/PKG-INFO b/PKG-INFO index 7c6e219..685b86d 100644 --- a/PKG-INFO +++ b/PKG-INFO @@ -1,6 +1,6 @@ Metadata-Version: 1.0 Name: dtrx -Version: 6.6 +Version: 7.1 Summary: Script to intelligently extract multiple archive types Home-page: http://www.brettcsmith.org/2007/dtrx/ Author: Brett Smith @@ -8,17 +8,17 @@ Author-email: brettcsmith@brettcsmith.org License: GNU General Public License, version 3 or later Download-URL: http://www.brettcsmith.org/2007/dtrx/ Description: dtrx extracts archives in a number of different - formats; it currently supports tar, zip (including self-extracting - .exe files), cpio, rpm, deb, gem, 7z, cab, rar, and InstallShield - files. It can also decompress files compressed with gzip, bzip2, - lzma, xz, or compress. + formats; it currently supports tar, zip (including self-extracting + .exe files), cpio, rpm, deb, gem, 7z, cab, rar, and InstallShield + files. It can also decompress files compressed with gzip, bzip2, + lzma, xz, or compress. - In addition to providing one command to handle many different archive - types, dtrx also aids the user by extracting contents consistently. - By default, everything will be written to a dedicated directory - that's named after the archive. dtrx will also change the - permissions to ensure that the owner can read and write all those - files. + In addition to providing one command to handle many different archive + types, dtrx also aids the user by extracting contents consistently. + By default, everything will be written to a dedicated directory + that's named after the archive. dtrx will also change the + permissions to ensure that the owner can read and write all those + files. Platform: UNKNOWN Classifier: Development Status :: 5 - Production/Stable Classifier: Environment :: Console diff --git a/README b/README index 204bd2f..1cc5097 100644 --- a/README +++ b/README @@ -7,10 +7,10 @@ cleanly extract many archive types ---------------------------------- :Author: Brett Smith -:Date: 2009-07-04 +:Date: 2011-11-19 :Copyright: - dtrx 6.5 is copyright © 2006-2009 Brett Smith and others. Feel free to + dtrx 7.1 is copyright © 2006-2011 Brett Smith and others. Feel free to send comments, bug reports, patches, and so on. You can find the latest version of dtrx on its home page at . @@ -28,7 +28,7 @@ cleanly extract many archive types You should have received a copy of the GNU General Public License along with this program; if not, see . -:Version: 6.5 +:Version: 7.1 :Manual section: 1 SYNOPSIS @@ -41,8 +41,8 @@ DESCRIPTION dtrx extracts archives in a number of different formats; it currently supports tar, zip (including self-extracting .exe files), cpio, rpm, deb, -gem, 7z, cab, rar, and InstallShield files. It can also decompress files -compressed with gzip, bzip2, lzma, or compress. +gem, 7z, cab, rar, lzh, and InstallShield files. It can also decompress +files compressed with gzip, bzip2, lzma, xz, or compress. In addition to providing one command to handle many different archive types, dtrx also aids the user by extracting contents consistently. By @@ -55,6 +55,11 @@ arguments. For example:: $ dtrx coreutils-5.*.tar.gz +You may specify URLs as arguments as well. If you do, dtrx will use `wget +-c` to download the URL to the current directory and then extract what it +downloads. This may fail if you already have a file in the current +directory with the same name as the file you're trying to download. + OPTIONS ======= @@ -108,12 +113,11 @@ dtrx supports a number of options to mandate specific behavior: contents. -q, --quiet - Suppress warning messages. Listing this option twice will cause dtrx to - be silent. + Suppress warning messages. List this option twice to make dtrx silent. -v, --verbose - Show the files that are being extracted. Listing this option twice will - cause dtrx to print debugging information. + Show the files that are being extracted. List this option twice to + print debugging information. --help Display basic help. diff --git a/scripts/dtrx b/scripts/dtrx index 7a98ba5..2fc99e3 100755 --- a/scripts/dtrx +++ b/scripts/dtrx @@ -2,7 +2,7 @@ # -*- coding: utf-8 -*- # # dtrx -- Intelligently extract various archive types. -# Copyright © 2006-2009 Brett Smith +# Copyright © 2006-2011 Brett Smith # Copyright © 2008 Peter Kelemen # # This program is free software; you can redistribute it and/or modify it @@ -38,15 +38,16 @@ import tempfile import termios import textwrap import traceback +import urlparse try: set except NameError: from sets import Set as set -VERSION = "6.6" +VERSION = "7.1" VERSION_BANNER = """dtrx version %s -Copyright © 2006-2009 Brett Smith +Copyright © 2006-2011 Brett Smith Copyright © 2008 Peter Kelemen This program is free software; you can redistribute it and/or modify it @@ -81,6 +82,7 @@ RECURSE_LIST = 5 mimetypes.encodings_map.setdefault('.bz2', 'bzip2') mimetypes.encodings_map.setdefault('.lzma', 'lzma') mimetypes.encodings_map.setdefault('.xz', 'xz') +mimetypes.encodings_map.setdefault('.lz', 'lzip') mimetypes.types_map.setdefault('.gem', 'application/x-ruby-gem') logger = logging.getLogger('dtrx-log') @@ -138,8 +140,8 @@ class ExtractorUnusable(Exception): EXTRACTION_ERRORS = (ExtractorError, ExtractorUnusable, OSError, IOError) class BaseExtractor(object): - decoders = {'bzip2': 'bzcat', 'gzip': 'zcat', 'compress': 'zcat', - 'lzma': 'lzcat', 'xz': 'xzcat'} + decoders = {'bzip2': ['bzcat'], 'gzip': ['zcat'], 'compress': ['zcat'], + 'lzma': ['lzcat'], 'xz': ['xzcat'], 'lzip': ['lzip', '-cd']} name_checker = DirectoryChecker def __init__(self, filename, encoding): @@ -161,18 +163,12 @@ class BaseExtractor(object): raise ExtractorError("could not open %s: %s" % (filename, error.strerror)) if encoding: - self.pipe([self.decoders[encoding]], "decoding") + self.pipe(self.decoders[encoding], "decoding") self.prepare() def pipe(self, command, description="extraction"): self.pipes.append((command, description)) - def first_bad_exit_code(self): - for index, code in enumerate(self.exit_codes): - if code != 0: - return index - return None - def add_process(self, processes, command, stdin, stdout): try: processes.append(subprocess.Popen(command, stdin=stdin, @@ -243,7 +239,16 @@ class BaseExtractor(object): def basename(self): pieces = os.path.basename(self.filename).split('.') + orig_len = len(pieces) extension = '.' + pieces[-1] + # This is maybe a little more clever than it ought to be. + # We're trying to be conservative about what remove, but also DTRT + # in cases like .tar.gz, and also do something reasonable if we + # encounter some completely off-the-wall extension. So that means: + # 1. First remove any compression extension. + # 2. Then remove any commonly known extension that remains. + # 3. If neither of those did anything, remove anything that looks + # like it's almost certainly an extension (less than 5 chars). if mimetypes.encodings_map.has_key(extension): pieces.pop() extension = '.' + pieces[-1] @@ -251,6 +256,9 @@ class BaseExtractor(object): mimetypes.common_types.has_key(extension) or mimetypes.suffix_map.has_key(extension)): pieces.pop() + if ((orig_len == len(pieces)) and + (orig_len > 1) and (len(pieces[-1]) < 5)): + pieces.pop() return '.'.join(pieces) def get_stderr(self): @@ -259,13 +267,25 @@ class BaseExtractor(object): self.stderr.close() return errors - def check_success(self, got_output): - error_index = self.first_bad_exit_code() - if (not got_output) and (error_index is not None): + def is_fatal_error(self, status): + return False + + def first_bad_exit_code(self): + for index, code in enumerate(self.exit_codes): + if code > 0: + return index, code + return None, None + + def check_success(self, got_files): + error_index, error_code = self.first_bad_exit_code() + logger.debug("success results: %s %s %s" % (got_files, error_index, + self.exit_codes)) + if (self.is_fatal_error(error_code) or + ((not got_files) and (error_code is not None))): command = ' '.join(self.pipes[error_index][0]) raise ExtractorError("%s error: '%s' returned status code %s" % (self.pipes[error_index][1], command, - self.exit_codes[error_index])) + error_code)) def extract_archive(self): self.pipe(self.extract_pipe) @@ -340,6 +360,7 @@ class CompressionExtractor(BaseExtractor): self.content_type = ONE_ENTRY_KNOWN self.content_name = self.basename() self.contents = None + self.file_count = 1 self.included_root = './' try: output_fd, self.target = tempfile.mkstemp(prefix='.dtrx-', dir='.') @@ -352,6 +373,7 @@ class CompressionExtractor(BaseExtractor): except EXTRACTION_ERRORS: os.unlink(self.target) raise + class TarExtractor(BaseExtractor): file_type = 'tar file' @@ -410,7 +432,7 @@ class DebExtractor(TarExtractor): raise ExtractorError("data.tar file has unrecognized encoding") self.pipe(['ar', 'p', self.filename, data_filename], "extracting data.tar from .deb") - self.pipe([self.decoders[encoding]], "decoding data.tar") + self.pipe(self.decoders[encoding], "decoding data.tar") def basename(self): pieces = os.path.basename(self.filename).split('_') @@ -486,6 +508,39 @@ class ZipExtractor(NoPipeExtractor): extract_command = ['unzip', '-q'] list_command = ['zipinfo', '-1'] + def is_fatal_error(self, status): + return status > 1 + + +class LZHExtractor(ZipExtractor): + file_type = 'LZH file' + extract_command = ['lha', 'xq'] + list_command = ['lha', 'l'] + + def border_line_file_index(self, line): + last_space_index = None + for index, char in enumerate(line): + if char == ' ': + last_space_index = index + elif char != '-': + return None + if last_space_index is None: + return None + return last_space_index + 1 + + def get_filenames(self): + filenames = NoPipeExtractor.get_filenames(self) + for line in filenames: + fn_index = self.border_line_file_index(line) + if fn_index is not None: + break + for line in filenames: + if self.border_line_file_index(line): + break + else: + yield line[fn_index:] + self.archive.close() + class SevenExtractor(NoPipeExtractor): file_type = '7z file' @@ -673,7 +728,9 @@ class EmptyHandler(object): return contents == EMPTY can_handle = staticmethod(can_handle) - def __init__(self, extractor, options): pass + def __init__(self, extractor, options): + os.rmdir(extractor.target) + def handle(self): pass @@ -799,7 +856,8 @@ class RecursionPolicy(BasePolicy): def prep(self, current_filename, target, extractor): archive_count = len(extractor.included_archives) - if (self.permanent_policy is not None) or (archive_count == 0): + if ((self.permanent_policy is not None) or + ((archive_count * 10) <= extractor.file_count)): self.current_policy = self.permanent_policy or RECURSE_NOT_NOW return question = self.wrap( @@ -825,48 +883,56 @@ class RecursionPolicy(BasePolicy): class ExtractorBuilder(object): - extractor_map = {'tar': {'extractor': TarExtractor, + extractor_map = {'tar': {'extractors': (TarExtractor,), 'mimetypes': ('x-tar',), 'extensions': ('tar',), 'magic': ('POSIX tar archive',)}, - 'zip': {'extractor': ZipExtractor, + 'zip': {'extractors': (ZipExtractor, SevenExtractor), 'mimetypes': ('zip',), 'extensions': ('zip',), 'magic': ('(Zip|ZIP self-extracting) archive',)}, - 'rpm': {'extractor': RPMExtractor, + 'lzh': {'extractors': (LZHExtractor,), + 'mimetypes': ('x-lzh', 'x-lzh-compressed'), + 'extensions': ('lzh', 'lha'), + 'magic': ('LHa [\d\.\?]+ archive',)}, + 'rpm': {'extractors': (RPMExtractor,), 'mimetypes': ('x-redhat-package-manager', 'x-rpm'), 'extensions': ('rpm',), 'magic': ('RPM',)}, - 'deb': {'extractor': DebExtractor, - 'metadata': DebMetadataExtractor, + 'deb': {'extractors': (DebExtractor,), + 'metadata': (DebMetadataExtractor,), 'mimetypes': ('x-debian-package',), 'extensions': ('deb',), 'magic': ('Debian binary package',)}, - 'cpio': {'extractor': CpioExtractor, + 'cpio': {'extractors': (CpioExtractor,), 'mimetypes': ('x-cpio',), 'extensions': ('cpio',), 'magic': ('cpio archive',)}, - 'gem': {'extractor': GemExtractor, - 'metadata': GemMetadataExtractor, + 'gem': {'extractors': (GemExtractor,), + 'metadata': (GemMetadataExtractor,), 'mimetypes': ('x-ruby-gem',), 'extensions': ('gem',)}, - '7z': {'extractor': SevenExtractor, + '7z': {'extractors': (SevenExtractor,), 'mimetypes': ('x-7z-compressed',), 'extensions': ('7z',), 'magic': ('7-zip archive',)}, - 'cab': {'extractor': CABExtractor, + 'cab': {'extractors': (CABExtractor,), 'mimetypes': ('x-cab',), 'extensions': ('cab',), 'magic': ('Microsoft Cabinet Archive',)}, - 'rar': {'extractor': RarExtractor, + 'rar': {'extractors': (RarExtractor,), 'mimetypes': ('rar',), 'extensions': ('rar',), 'magic': ('RAR archive',)}, - 'shield': {'extractor': ShieldExtractor, + 'shield': {'extractors': (ShieldExtractor,), 'mimetypes': ('x-cab',), 'extensions': ('cab', 'hdr'), 'magic': ('InstallShield CAB',)}, - 'compress': {'extractor': CompressionExtractor} + 'msi': {'extractors': (SevenExtractor,), + 'mimetypes': ('x-msi', 'x-ole-storage'), + 'extensions': ('msi',), + 'magic': ('Application: Windows Installer',)}, + 'compress': {'extractors': (CompressionExtractor,)} } mimetype_map = {} @@ -886,6 +952,7 @@ class ExtractorBuilder(object): ('tar', 'gzip', 'tar.gz', 'tgz'), ('tar', 'lzma', 'tar.lzma', 'tlz'), ('tar', 'xz', 'tar.xz'), + ('tar', 'lz', 'tar.lz'), ('tar', 'compress', 'tar.Z', 'taz'), ('compress', 'gzip', 'Z', 'gz'), ('compress', 'bzip2', 'bz2'), @@ -898,6 +965,7 @@ class ExtractorBuilder(object): for mapping in (('bzip2', 'bzip2 compressed'), ('gzip', 'gzip compressed'), ('lzma', 'LZMA compressed'), + ('lzip', 'lzip compressed'), ('xz', 'xz compressed')): for pattern in mapping[1:]: magic_encoding_map[re.compile(pattern)] = mapping[0] @@ -907,12 +975,13 @@ class ExtractorBuilder(object): self.options = options def build_extractor(self, archive_type, encoding): - extractors = self.extractor_map[archive_type] - if self.options.metadata and extractors.has_key('metadata'): - extractor = extractors['metadata'] + type_info = self.extractor_map[archive_type] + if self.options.metadata and type_info.has_key('metadata'): + extractors = type_info['metadata'] else: - extractor = extractors['extractor'] - return extractor(self.filename, encoding) + extractors = type_info['extractors'] + for extractor in extractors: + yield extractor(self.filename, encoding) def get_extractor(self): tried_types = set() @@ -932,7 +1001,8 @@ class ExtractorBuilder(object): tried_types.add(ext_args) logger.debug("trying %s extractor from %s" % (ext_args, func_name)) - yield self.build_extractor(*ext_args) + for extractor in self.build_extractor(*ext_args): + yield extractor def try_by_mimetype(cls, filename): mimetype, encoding = mimetypes.guess_type(filename) @@ -1228,6 +1298,21 @@ class ExtractorApplication(object): self.show_stderr(logger.error, stderr) return True + def download(self, filename): + url = filename.lower() + for protocol in 'http', 'https', 'ftp': + if url.startswith(protocol + '://'): + break + else: + return filename, None + # FIXME: This can fail if there's already a file in the directory + # that matches the basename of the URL. + status = subprocess.call(['wget', '-c', filename], + stdin=subprocess.PIPE) + if status != 0: + return None, "wget returned status code %s" % (status,) + return os.path.basename(urlparse.urlparse(filename)[2]), None + def run(self): if self.options.show_list: action = ListAction @@ -1238,9 +1323,12 @@ class ExtractorApplication(object): self.current_directory, self.filenames = self.archives.popitem() os.chdir(self.current_directory) for filename in self.filenames: - builder = ExtractorBuilder(filename, self.options) - error = (self.check_file(filename) or - self.try_extractors(filename, builder.get_extractor())) + filename, error = self.download(filename) + if not error: + builder = ExtractorBuilder(filename, self.options) + error = (self.check_file(filename) or + self.try_extractors(filename, + builder.get_extractor())) if error: if error != True: logger.error("%s: %s" % (filename, error)) diff --git a/setup.py b/setup.py index b8ab4e7..bda1d70 100644 --- a/setup.py +++ b/setup.py @@ -3,7 +3,7 @@ from distutils.core import setup setup(name="dtrx", - version = "6.6", + version = "7.1", description = "Script to intelligently extract multiple archive types", author = "Brett Smith", author_email = "brettcsmith@brettcsmith.org", diff --git a/tests/compare.py b/tests/compare.py index 56122aa..6a92fd1 100644 --- a/tests/compare.py +++ b/tests/compare.py @@ -24,7 +24,10 @@ import yaml import sys import tempfile -from sets import Set as set +try: + set +except NameError: + from sets import Set as set if os.path.exists('scripts/dtrx') and os.path.exists('tests'): os.chdir('tests') @@ -34,16 +37,10 @@ else: print "ERROR: Can't run tests in this directory!" sys.exit(2) -X_SCRIPT = os.path.realpath('../scripts/dtrx') +DTRX_SCRIPT = os.path.realpath('../scripts/dtrx') +SHELL_CMD = ['sh', '-se'] ROOT_DIR = os.path.realpath(os.curdir) OUTCOMES = ['error', 'failed', 'passed'] -TESTSCRIPT_NAME = 'testscript.sh' -SCRIPT_PROLOGUE = """#!/bin/sh -set -e -""" - -input_buffer = tempfile.TemporaryFile() -output_buffer = tempfile.TemporaryFile() class ExtractorTestError(Exception): pass @@ -62,25 +59,27 @@ class ExtractorTest(object): if isinstance(value, str): value = [value] setattr(self, key, value) - - def get_results(self, commands, stdin=None): - print >>output_buffer, "Output from %s:" % (' '.join(commands),) - output_buffer.flush() - status = subprocess.call(commands, stdout=output_buffer, - stderr=output_buffer, stdin=stdin) - process = subprocess.Popen(['find', '!', '-name', TESTSCRIPT_NAME], - stdout=subprocess.PIPE) - process.wait() + if self.input and (not self.input.endswith('\n')): + self.input = self.input + '\n' + + def start_proc(self, command, stdin=None, output=None): + process = subprocess.Popen(command, stdin=subprocess.PIPE, + stdout=output, stderr=output) + if stdin: + process.stdin.write(stdin) + process.stdin.close() + return process + + def get_results(self, command, stdin=None): + print >>self.outbuffer, "Output from %s:" % (' '.join(command),) + self.outbuffer.flush() + status = self.start_proc(command, stdin, self.outbuffer).wait() + process = subprocess.Popen(['find'], stdout=subprocess.PIPE) output = process.stdout.read(-1) process.stdout.close() + process.wait() return status, set(output.split('\n')) - def write_script(self, commands): - script = open(TESTSCRIPT_NAME, 'w') - script.write("%s%s\n" % (SCRIPT_PROLOGUE, commands)) - script.close() - subprocess.call(['chmod', 'u+w', TESTSCRIPT_NAME]) - def run_script(self, key): commands = getattr(self, key) if commands is not None: @@ -88,38 +87,27 @@ class ExtractorTest(object): directory_hint = '../' else: directory_hint = '' - self.write_script(commands) - subprocess.call(['sh', TESTSCRIPT_NAME, directory_hint]) + self.start_proc(SHELL_CMD + [directory_hint], commands) def get_shell_results(self): self.run_script('prerun') - self.write_script(self.baseline) - return self.get_results(['sh', TESTSCRIPT_NAME] + self.filenames) + return self.get_results(SHELL_CMD + self.filenames, self.baseline) def get_extractor_results(self): self.run_script('prerun') - input_buffer.seek(0, 0) - input_buffer.truncate() - if self.input: - input_buffer.write(self.input) - if not self.input.endswith('\n'): - input_buffer.write('\n') - input_buffer.seek(0, 0) - input_buffer.flush() - return self.get_results([X_SCRIPT] + self.options + self.filenames, - input_buffer) + return self.get_results([DTRX_SCRIPT] + self.options + self.filenames, + self.input) def get_posttest_result(self): if not self.posttest: return 0 - self.write_script(self.posttest) - return subprocess.call(['sh', TESTSCRIPT_NAME]) + return self.start_proc(SHELL_CMD, self.posttest).wait() def clean(self): self.run_script('cleanup') if self.directory: target = os.path.join(ROOT_DIR, self.directory) - extra_options = ['!', '-name', TESTSCRIPT_NAME] + extra_options = [] else: target = ROOT_DIR extra_options = ['(', '(', '-type', 'd', @@ -138,8 +126,8 @@ class ExtractorTest(object): def show_status(self, status, message=None): raw_status = status.lower() if raw_status != 'passed': - output_buffer.seek(0, 0) - sys.stdout.write(output_buffer.read(-1)) + self.outbuffer.seek(0, 0) + sys.stdout.write(self.outbuffer.read(-1)) if message is None: last_part = '' else: @@ -153,13 +141,13 @@ class ExtractorTest(object): status, expected = self.get_shell_results() self.clean() if expected != actual: - print >>output_buffer, "Only in baseline results:" - print >>output_buffer, '\n'.join(expected.difference(actual)) - print >>output_buffer, "Only in actual results:" - print >>output_buffer, '\n'.join(actual.difference(expected)) + print >>self.outbuffer, "Only in baseline results:" + print >>self.outbuffer, '\n'.join(expected.difference(actual)) + print >>self.outbuffer, "Only in actual results:" + print >>self.outbuffer, '\n'.join(actual.difference(expected)) return self.show_status('FAILED') elif posttest_result != 0: - print >>output_buffer, "Posttest gave status code", posttest_result + print >>self.outbuffer, "Posttest gave status code", posttest_result return self.show_status('FAILED') return self.show_status('Passed') @@ -187,24 +175,23 @@ class ExtractorTest(object): return None def check_results(self): - output_buffer.seek(0, 0) - output_buffer.truncate() self.clean() status, actual = self.get_extractor_results() - output_buffer.seek(0, 0) - output_buffer.readline() - output = output_buffer.read(-1) + self.outbuffer.seek(0, 0) + self.outbuffer.readline() + output = self.outbuffer.read(-1) problem = (self.have_error_mismatch(status) or self.check_output(output) or self.grep_output(output)) if problem: return self.show_status('FAILED', problem) - if self.baseline: + if self.baseline is not None: return self.compare_results(actual) else: self.clean() return self.show_status('Passed') def run(self): + self.outbuffer = tempfile.TemporaryFile() if self.directory: os.mkdir(self.directory) os.chdir(self.directory) @@ -212,6 +199,7 @@ class ExtractorTest(object): result = self.check_results() except ExtractorTestError, error: result = self.show_status('ERROR', error) + self.outbuffer.close() if self.directory: os.chdir(ROOT_DIR) subprocess.call(['chmod', '-R', '700', self.directory]) @@ -240,5 +228,3 @@ for outcome in OUTCOMES: for result in results: counts[result] += 1 print " Totals:", ', '.join(["%s %s" % (counts[key], key) for key in OUTCOMES]) -input_buffer.close() -output_buffer.close() diff --git a/tests/test-1.23.lzh b/tests/test-1.23.lzh new file mode 100644 index 0000000..942e2cb Binary files /dev/null and b/tests/test-1.23.lzh differ diff --git a/tests/test-recursive-no-prompt.tar.bz2 b/tests/test-recursive-no-prompt.tar.bz2 new file mode 100644 index 0000000..ff439e5 Binary files /dev/null and b/tests/test-recursive-no-prompt.tar.bz2 differ diff --git a/tests/test-text.lz b/tests/test-text.lz new file mode 100644 index 0000000..b1d0f2f Binary files /dev/null and b/tests/test-text.lz differ diff --git a/tests/tests.yml b/tests/tests.yml index 95f0aef..462baf3 100644 --- a/tests/tests.yml +++ b/tests/tests.yml @@ -22,6 +22,13 @@ cd test-1.23 unzip -q ../$1 +- name: basic .lzh + filenames: test-1.23.lzh + baseline: | + mkdir test-1.23 + cd test-1.23 + lha xq ../$1 + - name: basic .deb filenames: test-1.23_all.deb baseline: | @@ -74,46 +81,63 @@ tar -xOf $1 metadata.gz | zcat > test-1.23.gem-metadata.txt cleanup: rm -f test-1.23.gem-metadata.txt posttest: | - if [ "x`cat test-1.23.gem-metadata.txt`" != "xhi" ]; then exit 1; fi + exec [ "$(cat test-1.23.gem-metadata.txt)" = "hi" ] - name: recursion and permissions filenames: test-recursive-badperms.tar.bz2 options: -n -r baseline: | - mkdir test-recursive-badperms - cd test-recursive-badperms - tar -jxf ../$1 - mkdir test-badperms - cd test-badperms - tar -xf ../test-badperms.tar + extract() { + mkdir "$1" + cd "$1" + tar "-${3}xf" "../$2" + } + extract test-recursive-badperms "$1" j + extract test-badperms test-badperms.tar chmod 700 testdir posttest: | - if [ "x`cat test-recursive-badperms/test-badperms/testdir/testfile`" != \ - "xhey" ]; then exit 1; fi + exec [ "$(cat test-recursive-badperms/test-badperms/testdir/testfile)" = \ + "hey" ] -- name: decompressing gz +- name: decompressing gz, not interactive directory: inside-dir filenames: ../test-text.gz + options: "" + antigrep: "." baseline: | zcat $1 >test-text posttest: | - if [ "x`cat test-text`" != "xhi" ]; then exit 1; fi + exec [ "$(cat test-text)" = "hi" ] -- name: decompressing bz2 +- name: decompressing bz2, not interactive directory: inside-dir filenames: ../test-text.bz2 + options: "" + antigrep: "." baseline: | bzcat $1 >test-text posttest: | - if [ "x`cat test-text`" != "xhi" ]; then exit 1; fi + exec [ "$(cat test-text)" = "hi" ] -- name: decompressing xz +- name: decompressing xz, not interactive directory: inside-dir filenames: ../test-text.xz + options: "" + antigrep: "." baseline: | xzcat $1 >test-text posttest: | - if [ "x`cat test-text`" != "xhi" ]; then exit 1; fi + exec [ "$(cat test-text)" = "hi" ] + +- name: decompressing lzip, not interactive + directory: inside-dir + filenames: ../test-text.lz + options: "" + antigrep: "." + baseline: | + lzip -cd <$1 >test-text + posttest: | + exec [ "$(cat test-text)" = "hi" ] - name: decompression with -r directory: inside-dir @@ -163,7 +187,7 @@ tar -xf test-badperms.tar chmod 700 testdir posttest: | - if [ "x`cat testdir/testfile`" != "xhey" ]; then exit 1; fi + exec [ "$(cat testdir/testfile)" = "hey" ] - name: no files error: true @@ -222,6 +246,17 @@ test-1.23/a/b test-1.23/foobar +- name: list contents of LZH + options: -n -l + filenames: test-1.23.lzh + output: | + 1/ + 1/2/ + 1/2/3 + a/ + a/b + foobar + - name: list contents of .cpio options: -n -l filenames: test-1.23.cpio @@ -434,17 +469,16 @@ i n baseline: | - mkdir test-recursive-badperms - cd test-recursive-badperms - tar -jxf ../$1 - mkdir test-badperms - cd test-badperms - tar -xf ../test-badperms.tar + extract() { + mkdir "$1" + cd "$1" + tar "-${3}xf" "../$2" + } + extract test-recursive-badperms "$1" j + extract test-badperms test-badperms.tar chmod 700 testdir cd ../.. - mkdir test-recursive-badperms.1 - cd test-recursive-badperms.1 - tar -jxf ../$1 + extract test-recursive-badperms.1 "$1" j - name: interactive recursion (never) filenames: test-recursive-badperms.tar.bz2 test-recursive-badperms.tar.bz2 @@ -685,9 +719,28 @@ - name: extracting empty archive filenames: test-empty.tar.bz2 + options: "" baseline: "" + antigrep: '.' - name: listing empty archive filenames: test-empty.tar.bz2 options: -l antigrep: '.' + +- name: download and extract + filenames: http://brettcsmith.org/2007/dtrx/test-download.gz + directory: inside-dir + baseline: | + wget "$1" + zcat test-download.gz >test-download + cleanup: rm -f test-download.gz test-download + +- name: recursive archive without prompt + filenames: test-recursive-no-prompt.tar.bz2 + options: "" + baseline: | + mkdir test-recursive-no-prompt + cd test-recursive-no-prompt + tar -jxf ../$1 + antigrep: '.' -- cgit v1.2.3