2016-12-30 Jim Meyering version 4.3 * NEWS: Record release date. gnulib: update to latest, to fix parallel getopt test failure 2016-12-28 Assaf Gordon doc: new annotated example for N/P/D/b commands Illustrates how to restructure line breaks over multiple lines. * doc/sed.texi (Line length adjustment): New section. 2016-12-28 Assaf Gordon doc: new annotated example section for N/D commands Illustrate how to use N/D to search for doubled-word spanning multiple lines. * doc/sed.texi (Text search across mutliple lines): New section. 2016-12-28 Assaf Gordon doc: add 'multiline techniques' section Expand on using D,G,H,N,P commands to process multiple lines. * doc/sed.texi (Multiline techniques): New section. 2016-12-22 Assaf Gordon maint: update m4/.gitignore * m4/.gitignore: Add 'dirfd.m4', 'hard-locale.m4'. 2016-12-20 Jim Meyering maint: RE_ICASE, RE_NO_SUB: remove useless '#ifdef's * sed/regexp.c (compile_regex_1): Remove #ifdef RE_ICASE guard, since dfa.c uses it unconditionally. Suggested by Norihiro Tanaka. Also remove the nearby "#ifdef RE_NO_SUB". Those macros are guaranteed to be defined by virtue of configure-time tests that ensure we're using either a new-enough native glibc (RE_NO_SUB was added in 2004), or the included replacement. 2016-12-19 Jim Meyering gnulib: update to latest; dfa improvement and getopt-posix-tests build fix 2016-12-18 Jim Meyering gnulib: update to latest and adapt to modified dfa API * sed/regexp.c (compile_regex_1): Do away with use of DFA_CASE_FOLD. This gnulib update pulls in a dfa module in which that symbol no longer exists. Instead, it uses RE_ICASE in syntax bits. * m4/.gitignore: Add files created by running bootstrap. * lib/.gitignore: Likewise. tests: use just-built sed in more places * cfg.mk (PATH): Prepend $(srcdir)/src, so that we use the just- built sed also when running commands like those of "make distcheck". Tested by running this in a just-built directory: f=sed/sed; printf '%s\n' '#!/bin/sh' 'sleep 9h' > $f; chmod a+x $f and then verifying that nearly every "make syntax-check" rule hangs. 2016-12-14 Jim Meyering doc: fix sed.1-building rule to work on mingw * doc/local.mk (doc/sed.1): Add $(EXEEXT) suffix on "sed/sed" dependent. Patch by Miodrag Milanovic via Stephan T. Lavavej, in https://lists.gnu.org/archive/html/sed-devel/2016-12/msg00026.html 2016-12-12 Jim Meyering tests: port tests to avoid tr NUL-eliding behavior on Solaris * testsuite/Makefile.tests (elide_cr): Define this to use sed. Replace every prior use of "$(TR) -d \\r" with it. Our just-built sed can do that job (elide CR bytes) even when the input contains NUL bytes. Solaris 11.3's /bin/tr would remove any NUL byte from the result. Reported by Nelson H.F. Beebe in https://lists.gnu.org/archive/html/sed-devel/2016-12/msg00006.html gnulib: update to latest, for solaris-10-vs-configure fix doc: add a README-hacking file * README-hacking: New file: instructions for building from a git clone. Copied from grep's repo, but with s/grep/sed/. 2016-12-11 Jim Meyering build: avoid "make dist" failure due to missing dependencies * sed/local.mk ($(sed_sed_OBJECTS)): Depend on $(BUILT_SOURCES). Reported by Assaf Gordon in https://lists.gnu.org/archive/html/sed-devel/2016-12/msg00016.html 2016-12-10 Assaf Gordon doc: simplify documentation building Do not bother autogenerating sed.texi from sed-in.texi. All it did was to add a few '@group' commands. Since these are rarely modified, add them manually. This simplifies the documentation-building rules and lets us distribute only one copy of the texinfo manual. See http://lists.gnu.org/archive/html/sed-devel/2016-12/msg00013.html * doc/groupify.sed, doc/sed-in.texi: Remove files. * doc/sed.texi: Remove the 'auto-generated' warning, as this file is no longer auto-generated. * doc/local.mk (doc_sed_TEXINFOS): Rename from sed_TEXINFOS. (dist_noinst_DATA): The above renaming enables us to remove doc/config.texi and doc/fdl.texi from this list. (dist_noinst_SCRIPTS): Remove. (doc/sed.texi): Remove rule. (doc/s-texi): Likewise. 2016-12-06 Assaf Gordon maint: add missing fdl.texi to distribution Neglected to add it in commit v4.2.2-175-gc6a55aa. * doc/local.mk (sed_TEXINFOS, dist_noinst_DATA): Add doc/fdl.texi. 2016-12-06 Assaf Gordon doc: follow-up changes for manual improvements Fixes few missing issues from commit v4.2.2-176-g801a2c8, discussed in http://lists.gnu.org/archive/html/sed-devel/2016-12/msg00005.html * doc/config.texi: Add stub texinfo macros '@codequotebacktick' and '@codequoteundirected' (needed if using older versions of texinfo). * doc/sed-in.texi: Fix syntax that's accepted by newer texinfo but rejected by texinfo-4.13; Disable directed quote characters in examples. * doc/sed.texi: Auto-regenerated. 2016-12-05 Jim Meyering gnulib: update to latest 2016-12-05 Assaf Gordon doc: reorganize and expand manual Discussed in http://lists.gnu.org/archive/html/sed-devel/2016-11/msg00005.html * doc/sed-in.texi: Reorganize and expand. * doc/sed.texi: Auto-regenerated. 2016-12-02 Assaf Gordon maint: upgrade documentation license to GFDL-1.3 (or later) * bootstrap.conf: Add gnulib's fdl module. * doc/sed-in.texi: Update license blurb to latest recommended text. Remove duplicated copyright information from title page (copied from coreutil's texinfo style). Include fdl.texi in a new appendix. 2016-11-29 Jim Meyering tests: use "returns_ N env VAR=val ..." Use "returns_ N env VAR=val ..." rather than "VAR=val returns_ N ...". Some shells do not propagate envvar settings through our use of the "returns_" function, so set any envvar via use of "env". This was an issue at least on Ubuntu, Debian and *BSD-based systems. * testsuite/mb-bad-delim.sh: As above. * testsuite/mb-y-translate.sh: Likewise. gnulib: update to latest 2016-11-27 Jim Meyering tests: use "returns_ 1" rather than "&& fail=1" * testsuite/colon-with-no-label.sh: Rather than failing only upon success (exit 0), fail upon exit with any value other than 1. build: stop suppressing many -W... warnings This code in configure.ac added many -W options to the list of warnings that we do *not* enable. When I copied most of this code into this file from coreutils, I did not try to remove the options that happened not to be needed for sed. Now, compilers have improved and I have done that triage. Many -W options can be left enabled. * configure.ac (GNULIB_WARN_CFLAGS): There is no longer any need to use a different set of warnings for lib/ files, so remove this variable, along with many -W-related exemptions. build: avoid "make distcheck" failure due to leftover .Po files The following definitions caused trouble for no gain. They caused some .o file names to have a long additional prefix and even resulted in some .Po files not being removed by "make distclean" when building with recent automake. The only reason to use these definitions was -- long ago -- to require different compiler options for files in lib/ than elsewhere. That is no longer necessary. * lib/local.mk (lib_libsed_a_CPPFLAGS): Remove definition. (lib_libsed_a_CFLAGS, lib_libsed_a_LIBADD): Likewise. (lib_libsed_a_DEPENDENCIES): Likewise. maint: fix -Wformat-exposed errors in uses of "panic" * sed/utils.h (panic): Add __printf__ attribute, so that compilers can diagnose format/arg-type errors. This exposed the following: * sed/utils.c (ck_fwrite): Don't try to print size_t via %u. Instead, use %llu and a cast to unsigned long long. * sed/mbcs.c (is_mb_char): Cast an "int" to unsigned int, to avoid mismatch with %x. maint: remove unused parameters * sed/compile.c (finish_program): This function's sole argument was unused. Remove it and update callers. * sed/sed.h: Update prototype. * sed/utils.c (register_open_file): This functions's third argument, "temp", was unused. Remove it and update callers. * sed/sed.c: Update use. maint: fix two "make syntax-check" failures * cfg.mk (exclude_file_name_regexp--sc_prohibit_empty_lines_at_EOF): Also exempt zero-anchor.{inp,good}. * sed/regexp.c (match_regex): s/can not/cannot/ 2016-11-16 Assaf Gordon build: remove explicit 'html' makefile target The 'html' target was a left-over from the recursive Makefiles, which were converted to non-recursive in sed v4.2.2-129-g3b29bec. With this removed, the 'html' target in 'Makefile' is correctly genererated by automake (from doc/local.mk), and 'make html' works again. * Makefile.am: Remove explicit 'html' target. 2016-11-12 Jim Meyering tests: use "returns_ 1" rather than testing $? = 1 * testsuite/temp-file-cleanup.sh: As above. 2016-11-04 Assaf Gordon sed: standardize exit code values Use exit-code constants, and document them: 1=syntax error, 2=bad input files, 4=I/O error or panic. Add "Exit Status" section in manual. * sed/utils.h: Define new constants. * sed/utils.c, sed/compile.c, sed/execute.c, sed/sed.c: Use constants instead of hard-coded values. * doc/sed-in.texi: Add "Exit Status" section. * doc/sed.texi: Automatically re-generated. 2016-11-04 Assaf Gordon maint: regenerate sed.texi after doc change Regenerate sed.texi from sed-in.texi after doc change in v4.2.2-160-10549b7 ( https://bugs.gnu.org/24799 ). * doc/sed.texi: Regenerated with 'make info'. 2016-11-04 Norihiro Tanaka sed: fix multi-line regex matching with -z Sed incorrectly matches the anchor '^' to a newline character (ASCII 10) despite using NUL line terminators (-z). See https://bugs.gnu.org/24615 . Previous (incorrect) behavior: $ printf "a\0b\0" | sed -z 'N;s/^/X/mg;' | od -An -a X a nul b nul With this fix: $ printf "a\0b\0" | sed -z 'N;s/^/X/mg;' | od -An -a X a nul X b nul Current regex engine can not handle NUL multiline anchors. Implement such search manually (by breaking the pattern into separate NUL-terminated lines) and searching each one individually. * NEWS: Mention it. * sed/regexp.c (compile_regex_1): Don't use newline_anchor of regex, if the buffer delimiter is not newline character. (match_regex): Do above case line-by-line. * testsuite/zero-anchor.good, testsuite/zero-anchor.inp, testsuite/zero-anchor.sed: New test. * testsuite/local.mk: Add the test. * testsuite/Makefile.tests: Add the test. * bootstrap.conf: Add memrchr. * lib/.gitignore, m4/.gitignore: Ignore memrchr files. 2016-11-04 Norihiro Tanaka sed: optimize regex processing of lone ^ or $ Optimize /^/ and /$/ patterns and avoid invoking the regular expression engine if possible. See https://bugs.gnu.org/24615 . * sed/regex.c (compile_regex_1): Mark the patterns which consist of ^ or $. (match_regex): Handle the patterns which consist of ^ or $ manually. * sed/sed.h (struct regex): New members 'begline' and 'endline'. * testsuite/newline-anchor.good, testsuite/newline-anchor.sed, testsuite/newline-anchor.sed: New test. * testsuite/Makefile.tests: Add the test. * testsuite/local.mk: Add the test. 2016-10-26 Vincenzo Romano doc: make an example's description more precise * doc/sed-in.texi (Regular Expressions): Mention that the match must be at the end of a line. 2016-10-02 Jim Meyering maint: address syntax-check warnings about @pxref * doc/sed-in.texi: Improve cross-references. * doc/sed.texi: Regenerate. build: address warnings from GCC 7 * sed/sed.c (usage): Declare with the _Noreturn attribute to suppress this warning/error: sed/sed.c:359:11: error: this statement may fall through \ [-Werror=implicit-fallthrough] usage(0); ^~~~~~~~ * sed/execute.c (execute_program) [case 'q']: Add FALLTHROUGH comment to avoid warning from GCC 7's -Wimplicit-fallthrough. * sed/compile.c (compile_program): Likewise. gnulib: update to latest 2016-09-25 Norihiro Tanaka sed: fix incorrect match for title case * sed/regexp.c (compile_regex_1): Avoid building fastmap for case- insensitive matching. * testsuite/title-case.sh: New file, to test for this fix. * testsuite/local.mk (T): Add the new file name and sort alphabetically. * init.cfg (require_el_iso88597_locale_): Define. 2016-09-11 Jim Meyering dfa: reflect move of dfa code to new gnulib module * bootstrap.conf (gnulib_modules): Add dfa. * sed/dfa.c: Remove file. * sed/dfa.h: Likewise. * sed/local.mk (sed_sed_SOURCES): Remove dfa.c. (NOINST_HEADERS): Remove dfa.h. * sed/regexp.c (compile_regex_1): Use new dfasyntax API. * sed/sed.c (localeinfo): New global. (main): Call init_localeinfo to initialize it. * sed/sed.h: Include localeinfo.h and declare the new global. * lib/.gitignore: Ignore new gnulib-imported files. * m4/.gitignore: Likewise. * po/POTFILES.in: s,sed/dfa.c,lib/dfa.c, gnulib: update to latest, for new dfa module 2016-09-04 Assaf Gordon sed: new --sandbox option In sandbox mode, r/w/e commands are not allowed. This ensures sed operates only on files designated on the command line, and cannot execute external programs. * sed/sed.c (sandbox): New option variable. (usage): List new option. (main): Accept new option in getopt. * sed/sed.h (sandbox): New extern option variable. * sed/compile.c (DISALLOWED_CMD): New error message. (read_filename): Abort in sandbox mode (r/w commands). (compile_program): Abort on 'e' in sandbox mode. * testsuite/sandbox.sh: Test new option. * testsuite/local.mk: Add new test. * NEWS: Mention new option. * doc/sed-in.texi, doc/sed.texi: Document new option. 2016-08-27 Jim Meyering maint: fix distclean-vs-excess-.Po files more cleanly * doc/local.mk (distclean-local): Remove rule. Moved to ... * Makefile.am (distclean-local): ...here, and add a command to remove all .deps directories. It could have stayed in that .mk file, but since with this change, it is no longer specific to doc/, the top level makes more sense. * lib/local.mk (MOSTLYCLEANFILES): Remove this. Covered by the above. Reported by Assaf Gordon in https://lists.gnu.org/archive/html/sed-devel/2016-08/msg00019.html 2016-08-26 Assaf Gordon maint: update .mailmap Consolidate author's upper/lower case email addresses. Fixes the following problem: $ make THANKS GEN THANKS ./thanks-gen: THANKS.in: duplicate name: Ralf Wildenhues Due to: $ git log --pretty=format:'%aN %aE' | grep -i ralf | sort -u Ralf Wildenhues ralf.wildenhues@gmx.de Ralf Wildenhues Ralf.Wildenhues@gmx.de see http://lists.gnu.org/archive/html/sed-devel/2016-08/msg00023.html http://lists.gnu.org/archive/html/sed-devel/2016-08/msg00025.html * .mailmap: Add email mapping. 2016-08-24 Assaf Gordon doc: update README * README: Update information: remove mention of README.boot, mention AUTHORS, THANKS, website and help/usage information. maint: update program name in THANKS.in * THANKS.in: Replace 'grep' with 'sed'. maint: remove superfluous name from THANKS.in * THANKS.in: Remove Pádraig Brady: He is the author of two commits (v4.2.1-1-g737ca5e and v4.2.1-37-g629ef76) and will be automatically included in the auto-generated THANKS file. maint: remove README-alpha * README-alpha: Removed. 2016-08-22 Assaf Gordon maint: update AUTHORS * AUTHORS: Update information. 2016-08-18 Jim Meyering doc: work with read-only sed.1 * doc/local.mk (doc/sed.1): Also remove $@, so that mv does not prompt about read-only target. 2016-08-14 Jim Meyering build: fix sed.1 dependency; work with latest automake * doc/local.mk (doc/sed.1): Depend not on sed.c, but on the binary that help2man must run. Also depend on .version, to be sure we pick up a version change. Also, be careful not to write directly to $@: instead write to $@-t, and make read-only to mark as generated, and then, only if all is well, rename atomically to $@. Also, prefix commands with $(AM_V_GEN) and $(AM_V_at) and ensure that doc/ exists via $(MKDIR_P). (distclean-local): New target. Use this instead of ... (dist-hook-man-page): ... this. Remove rule. (SEDBIN): Remove unnecessary $(top_builddir)/ prefix. (doc/sed.texi, doc/s-texi): Remove some uses of $(top_builddir). Replace others with $(srcdir). * Makefile (dist-hook): remove dependency on dist-hook-man-page. * lib/local.mk (MOSTLYCLEANFILES): Arrange for lib/.deps/*.Po to be removed as part of "make distclean", so that make distcheck no longer fails when using latest automake. sed: remove obsolete advice from --help output * sed/sed.c (contact): Now that there is a sed-specific mailing list, there is no longer any need to include "sed" in the subject. Remove that advice. 2016-08-11 Assaf Gordon doc: mention \s,\S,\<,\> regex gnu-extensions * doc/sed-in.texi: Mention the extensions. * doc/sed.texi: Automatically re-generated. doc: expand on N command * doc/sed-in.texi: Expand on gnu-extension of 'N' command. * doc/sed.texi: Automatically re-generated. 2016-08-09 Jim Meyering sed: avoid one-byte heap buffer overrun * sed/execute.c (DFA_SLOP): Define. (resize_line): Use it in each line-related allocation. (line_init): Likewise. (line_copy): Likewise. 2016-08-08 Assaf Gordon sed: adjust line-terminator of F/l/= commands when -z is used Change F/l/= commands to use NUL line terminator if '-z' option is used. Previously F/l/= used '\n' regardless of -z option. See: http://lists.gnu.org/archive/html/sed-devel/2016-08/msg00000.html http://lists.gnu.org/archive/html/sed-devel/2016-08/msg00002.html . * sed/execute.c: (do_list,execute_program): Use 'buffer_delimiter' as line delimiter instead of hard-coded '\n'. * testsuite/nulldata.sh: Adjust tests accordingly. 2016-08-08 Norihiro Tanaka sed: use grep's DFA matcher to speed up regular expression matching When possible, sed now uses grep's DFA matcher to match regular expression, often resulting in a 10x speed-up. * NEWS (Improvements): Mention it. Move this section to the top. * sed/local.mk (sed_sed_SOURCES): Add dfa.c, and sort file names. (noinst_HEADERS): Add dfa.h, and sort file names. * sed/regexp.c (dfaerror, dfawarn): New functions. (compile_regex_1): Compile pattern with dfa. (match_regex): Use dfa. * sed/sed.h: Include dfa.h. (struct regex): New member dfa. * sed/dfa.c: New file, copied from grep. * sed/dfa.h: New file, copied from grep. * po/POTFILES.in: Add sed/dfa.c. 2016-08-08 Assaf Gordon sed: reject 's///e' in --posix mode 's///e' modifier is a gnu extension - reject it in --posix mode. * sed/compile.c: (mark_subst_opt): Exit upon 's///e' in posix mode. * testsuite/posix-mode-s.sh: Adjust tests accordingly. 2016-08-08 Assaf Gordon sed: allow multiple (non-conflicting) -E/-r parameters Accept multiple -E/-r parameters as they enable the same ERE operation mode. Previously, sed would reject multiple -E/-r/-R parameters (as preparation to supporting PERL syntax and wanting to avoid -E/-r/-R conflicts). This enables 'sed -E -E -r =' which previously would exit with a failure (and print the help screen). * sed/sed.c: (main): Don't exit on multiple -E/-r parameters. 2016-07-26 Assaf Gordon maint: add static-analysis target to cfg.mk Run 'make static-anslysis' to perform static code analysis using clang's 'scan-build' tool. * cfk.mk: (static-analysis,static-analysis-init,static-analysis-config, static-analysis-make): New targets. 2016-07-26 Assaf Gordon maint: add copyright/license information to configure.ac Starting year chosen as 1993 based on earliest 'configure.in' found in sed-1.18.tar.gz from that year. * configure.ac: Add standard copyright and license information. 2016-07-26 Assaf Gordon maint: update .gitignore after bootstrap Added by running ./bootstrap on a clean sed repository. * build-aux/.gitignore: Add config.rpath. * lib/.gitignore: Add additional gnulib C files. * m4/.gitignore: Add additional gnulib m4 files. * po/.gitignore: Add additional PO-related files. 2016-07-26 Assaf Gordon maint: update .gitignore * .gitignore: Add coverage files, .dirstamp files, testsuite files. 2016-07-25 Jim Meyering maint: skip a check when en_US.UTF-8 collation rules are broken * cfg.mk (sc_THANKS_in_sorted): This check would fail on systems for which "." is not ignored. Add a quick sort-based check for that error, and skip the check on any broken system. This also corrects the command to use $(srcdir)/THANKS.in, rather than just THANKS.in. 2016-07-24 Jim Meyering maint: sort THANKS.in * THANKS.in: Move J.T. Conklin's name "down" so it is in en_US.UTF-8-sorted order. 2016-07-24 Assaf Gordon build: switch to non-recursive makefile * .gitignore: Ignore '.dirstamp' files * bootstrap.conf: Add 'non-recursive-gnulib-prefix-hack' gnulib module. Add missing 'stdalign' module. (bootstrap_post_import_hook): Use module to adjust gnulib.mk file. * configure.ac: Add 'subdir-objects' automake option, don't generate makefiles in subdirectories. Adjust path of XFAIL tests. * Makefile.am: Include 'local.mk' files, adjust for non-recursive make. * doc/Makefile.am: Rename to ... * doc/local.mk: Adjust paths. Rename SED to SEDBIN variable, to avoid overriding the default system-wide SED value which is automatically set by autotools. * lib/Makefile.am: Rename to ... * lib/local.mk: Adjust paths. * lib/.gitignore: Ignore 'stdalign.h' from gnulib module. * sed/Makefile.am: Rename to ... * sed/local.mk: Adjust paths. * testsuite/Makefile.am: Rename to ... * testsuite/local.mk: Adjust paths. * testsuite/init.cfg: Move to ... * init.cfg: ... here. Moved to avoid changing the path in init.sh, and to keep the same directory structure as coreutils. (require_en_utf8_locale_): Adjust path for get-mb-cur-max executable. (require_ja_shiftjis_locale_,require_valid_ja_shiftjis_locale_): Adjust path for test-mbrtowc executable. * m4/.gitignore: Add gnulib's non-recusive-make module. * testsuite/runtest: Adjust paths of Makefile.tests and dir variables, Skip utility program 'get-mb-cur-max' which is not a standalone test. * testsuite/Makefile.tests: Adjust path of sed exeutable. * testsuite/eval.sed, testsuite/eval.good: Adjust path of sed executable. * testsuite/cmd-R.sh, testsuite/cmd-l.sh, testsuite/colon-with-no-label.sh, testsuite/comment-n.sh, testsuite/compile-errors.sh, testsuite/compile-tests.sh, testsuite/convert-number.sh, testsuite/execute-tests.sh, testsuite/follow-symlinks-stdin.sh, testsuite/follow-symlinks.sh, testsuite/help-version.sh, testsuite/in-place-hyphen.sh, testsuite/in-place-suffix-backup.sh, testsuite/invalid-mb-seq-UMR.sh, testsuite/mb-bad-delim.sh, testsuite/mb-charclass-non-utf8.sh, testsuite/mb-match-slash.sh, testsuite/mb-y-translate.sh, testsuite/normalize-text.sh, testsuite/nulldata.sh, testsuite/panic-tests.sh, testsuite/posix-char-class.sh, testsuite/posix-mode-N.sh, testsuite/posix-mode-addr.sh, testsuite/posix-mode-bad-ref.sh, testsuite/posix-mode-s.sh, testsuite/range-overlap.sh, testsuite/recursive-escape-c.sh, testsuite/regex-errors.sh, testsuite/stdin-prog.sh, testsuite/subst-mb-incomplete.sh, testsuite/subst-options.sh, testsuite/subst-replacement.sh, testsuite/temp-file-cleanup.sh, testsuite/unbuffered.sh: Adjust paths of init.sh, sed executable. 2016-07-17 Assaf Gordon tests: test --follow-symlinks option * configure.ac: (TEST_SYMLINKS): New AM_CONDITIONAL. * testsuite/follow-symlinks.sh: New tests. * testsuite/Makefile.am: (T): Conditionally add new test (and existing test 'follow-symlink-stdin.sh') based on TEST_SYMLINKS. tests: test y command in multibyte locales * testsuite/mb-y-translate.sh: New test. * testsuite/Makefile.am: Add new test. 2016-07-16 Assaf Gordon tests: test y,s non-slash multibyte delimiters Test various scenarios of using a multibyte delimiters (instead of slash) for s,y commands under UTF-8 and C locales. * testsuite/mb-bad-delim.sh: New test. * testsuite/Makefile.am: Add new test. 2016-07-06 Assaf Gordon tests: skip tests upon buggy mbrtowc implementations Sed tests purposefully use invalid non-utf8 multibyte sequences to test sed's handling of invalid input/program. Some implementations wrongly accept invalid sequences, causing false alarms. Add a test program to detect such buggy implementations and skip the tests if needed. See discussion in: http://lists.gnu.org/archive/html/sed-devel/2016-07/msg00005.html http://lists.gnu.org/archive/html/sed-devel/2016-07/msg00000.html http://lists.gnu.org/archive/html/sed-devel/2016-06/msg00031.html http://lists.gnu.org/archive/html/sed-devel/2016-06/msg00014.html * bootstrap.conf: Add gnulib's closeout module. * m4/.gitignore, lib/.gitignore, po/POTFILES.in: Adjust after adding closeout module. * testsuite/Makefile.am (check_PROGRAMS): Add new program. * testsuite/test-mbrtowc.c: New program. * testsuite/init.cfg (require_valid_ja_eucjp_locale_) (require_valid_ja_shiftjis_locale_): New functions. Call test-mbrtowc with known invalid multibyte sequences to ensure implementation is not buggy and rejects them; Skip test otherwise. * testsuite/invalid-mb-seq-UMR.sh: Use require_valid_ja_eucjp_locale_. * testsuite/mb-charclass-non-utf8.sh: Use require_valid_ja_shiftjis_locale_. 2016-07-04 Assaf Gordon sed: fix minor multibyte parsing bug Previously sed would parse multibyte characters incorrectly in two scenarios: 1. Slash following an incomplete-yet-valid multibyte sequence (match_slash): $ LC_ALL=en_US.UTF-8 sed $'s/\316/X/' sed: -e expression #1, char 6: unterminated `s' command 2. Open/close brackets as part of a valid mutilbyte string inside a character class (snarf_char_class). In the example below, '\203]' is a valid multibyte character in SHIFT-JIS locale: $ LC_ALL=ja_JP.shiftjis sed $'/[\203]/]/p' sed: -e expression #1, char #5: Unmatched [ or [^ Both cases stem from mbcs.c:brlen() being non-intuitive: It returned 1 for valid single-byte character, invalid multibyte-character, and a for the last byte of a valid multibyte sequence - making it non-trivial to use correctly. This commit replaces brlen() with a simpler is_mb_char() function: returns non-zero for multibyte sequences, zero for single/invalid sequences. * sed/sed.h: (BRLEN, brlen): Remove delaration. (IS_MB_CHAR,is_mb_char): Add macro and function declaration. * sed/mbcs.c: (brlen): Remove function. (is_mb_char): New function. * sed/compile.c: (snarf_char_class, match_slash): Use IS_MB_CHAR instead of BRLEN; Adjust local variables accordingly. * testsuite/mb-match-slash.sh: New test for scenario 1. * testsuite/mb-charclass-non-utf8.sh: New test for scenario 2, requires SHIFT-JIS locale. * testsuite/Makefile.am: Add new tests * testsuite/init.cfg: (require_ja_shiftjis_locale_): New function. * NEWS: Mention bug fix. 2016-06-26 Tristan Verniquet sed: use unlocked-io Sed already imports gnulib's unlocked-io module, but it was not included in 'utils.c'. Suggested in http://bugs.gnu.org/23848 . * sed/utils.c: Include 'unlocked-io.h' header. * NEWS: Mention improvement. 2016-06-05 Assaf Gordon tests: test text normalization (\X sequences) Sed understands escape-sequences such as \r \t \n \cX. Test various scenarios of text normalization. * testsuite/normalize-text.sh: new test. * testsuite/Makefile.am: add new test. 2016-06-02 Assaf Gordon tests: test \dNNN \oNNN \xNN escape sequences Test different variations of \dNNN \oNNN \xNN escape sequences, exercising compile.c:convert_number(). * testsuite/convert-number.sh: new test. * testsuite/Makefile.am: add new test 2016-06-02 Assaf Gordon tests: test less-common compilation cases Test various odds-and-ends stemming from GNU sed implementation of the program compilation module. * testsuite/compile-tests.sh: new test. * testsuite/Makefile.am: add new test. 2016-06-01 Assaf Gordon tests: test less-common execution cases * testsuite/execute-tests.sh: new test. * testsuite/Makefile.am: add new test. tests: test 'R' command * testsuite/cmd-R.sh: new test. * testsuite/Makefile.am: add new test. tests: test program file from STDIN * testsuite/stdin-prog.sh: new test. * testsuite/Makefile.am: add new test. tests: test auto-silent mode with '#n' comments * testsuite/comment-n.sh: new test. * testsuite/Makefile.am: add new test. 2016-06-01 Assaf Gordon tests: test character-classes escaping in POSIX mode No \X escaping inside character classes in POSIX mode. Compare: $ printf "t\t\n" | sed 's/[\t]/X/' | od -a 0000000 t X nl $ printf "t\t\n" | sed --posix 's/[\t]/X/' | od -a 0000000 X ht nl * testsuite/posix-char-class.sh: new test. * testsuite/Makefile.am: add new test. 2016-06-01 Assaf Gordon tests: test substitue replacements (s//\N/) Test standard POSIX (e.g. \1 \2 \3) and gnu extension (\U \u \L \l \E) replacements in s/// command. * testsuite/subst-replacement.st: new test. * testsuite/Makefile.am: add new test. 2016-06-01 Assaf Gordon tests: test s/// modifiers Test standard posix and gnu extension modifiers to s/// command. * testsuite/subst-options.sh: new test. * testsuite/Makefile.am: add new test. 2016-06-01 Assaf Gordon tests: test compilation-related errors Cover code paths in copmile.c leading to errors. * testsuite/compile-errors.sh: new test. * testsuite/Makefile.am: add new test. 2016-06-01 Assaf Gordon tests: test regex compilation errors Test scenarios leading to regex compilation errors. * testsuite/regex-errors.sh: new test. * testsuite/Makefile.am: add new test. 2016-06-01 Assaf Gordon tests: exercise 'panic' calls Test scenarios in which GNU sed terminates with a panic error message (and verify the error message text). * testsuite/panic-tests.sh: new test. * testsuite/Makefile.am: add new test. 2016-06-01 Assaf Gordon tests: test -i/--inplace with backup suffix option GNU Sed accepts optional backup filename/suffix with -i option. If the name contains '*', sed replaces it with the input filename. Test various scenarios of suffix names. Example: sed -i'*-*-*.txt' = a will generate backup file 'a-a-a.txt'. * testsuite/in-place-suffix-backup.sh: new test. * testsuite/Makefile.am: add new test. 2016-06-01 Assaf Gordon tests: test -u/--unbuffered mode Test GNU Sed behaviour in --unbuffered mode (reading only the minimum needed characters from STDIN, leaving output for next program when sed terminates). Example: seq 10 | (sed -u 1q ; sort -nr ) * testsuite/unbuffered.sh: new test. * testsuite/Makefile.am: add new test. 2016-06-01 Assaf Gordon tests: test -z/--null-data option Test input/output line-terminators with -z option. * testsuite/nulldata.sh: new test. * testsuite/Makefile.am: add new test. 2016-06-01 Assaf Gordon tests: test 'l' command Test output width of 'l' command, based on COLS envvar, lN (GNU Extension), '-l N' command-line parameter, and default output width. * testsuite/cmd-l.sh: new test. * testsuite/Makefile.am: add new test. 2016-06-01 Assaf Gordon tests: test gnu address extension in gnu/posix modes The following addresses are gnu extensions: 0,/regexp/ addr1,+N addr2,~N * testsuite/posix-mode-addr.sh: new test. * testsuite/Makefile.am: add new test. 2016-06-01 Assaf Gordon tests: test 's///' gnu extensions in gnu/posix modes s///[iImMe] should be rejected in --posix mode. s//\[lLuUN]// should have no special meaning in --posix mode. * testsuite/posix-mode-s.sh: new test. * testsuite/Makefile.am: add new test. 2016-06-01 Assaf Gordon tests: test invalid backreferences in gnu/posix modes 's/foo/\1/' is rejected by default (gnu extension), silently accepted in POSIX. * testsuite/posix-mode-bad-ref.sh: new test. * testsuite/Makefile.am: add new test. 2016-06-01 Assaf Gordon tests: test N command in gnu/posix modes 'N' with insufficient output prints nothing in POSIX mode, prints the last read line in GNU Extension mode. Compare: echo a | sed N echo a | sed --posix N echo a | POSIXLY_CORRECT=y sed N * testsuite/posix-mode-N.sh: test output of N command under gnu and posix modes. * testsuite/Makefile.am: add new test. 2016-06-01 Assaf Gordon tests: invalid-mb-seq-UMR: avoid false-failure * testsuite/Makefile.am (TESTS_ENVIRONMENT): Propagate the LOCALE_JA envvar setting determined at configure time into the test environment. Without this, the test would fail on some systems: at least CentOS6 and 7, but not Fedora 24. 2016-06-01 Jim Meyering tests: don't let any envvar setting perturb tests * testsuite/envvar-check: New file. Identical to the one from grep, but with the addition of sed-specific "COLS". * testsuite/Makefile.am (EXTRA_DIST): Add it here. (TESTS_ENVIRONMENT): New prologue, mostly copied from grep. 2016-05-29 Assaf Gordon sed: reject recursive escaping after \c previously, sed 's/./\c\\/' or 's/./\c\d/' would produce incorrect results. Require two backslashes after \c to denote control sequence ^\ (ASCII 0x34), and reject recursive escaping (e.g. \c\x61). * sed/compile.c: (RECURSIVE_ESCAPE_C): New error message. (normalize_text): Check for \c-backslash, reject recursive escaping. * testsuite/recursive-escape-c.sh: New file. Test new behaviour. * testsuite/Makefile.am (T): Add new test. * NEWS (Bug fixes): Mention it. 2016-01-02 Jim Meyering revert v4.2.2-87-gc033bde, to make sed -i treat "-" as a file name Stephane Chazelas made a fine case for why sed's --in-place (-i) should treat "-" as a file name, and not as standard input in http://debbugs.gnu.org/21249 There was no specific test for the prior behavior, so rather than letting the revert remove the sole test, adapt and retain it: * testsuite/in-place-hyphen.sh: However, retain and adapt this file to test for the restored behavior. * testsuite/Makefile.am: Retain this change, too. 2016-01-01 Jim Meyering maint: update copyright year, bootstrap, init.sh Run "make update-copyright" and then... * gnulib: Update to latest. * tests/init.sh: Update from gnulib. * bootstrap: Likewise. 2015-12-27 Jim Meyering sed: do not elide an invalid byte in a substitution RHS Now, sed copies such bytes into the output. * sed/execute.c (str_append_modified): Copy each invalid byte into the result string, as the comment suggests. * testsuite/invalid-mb-seq-UMR.sh: Adjust this test to conform. Perl does the same thing. Also, remove an unnecessary "-e" option. * NEWS (Bug fixes): Mention it. Bug introduced prior to the switch to git in 2004, between the releases of sed-4.0.9 and sed-4.1. This addresses http://debbugs.gnu.org/22254. sed: fix a heap-clobbering buffer overrun * sed/execute.c (str_append_modified): Upon encountering an incomplete multi-byte sequence in a substitution replacement string, do not treat mbrtowc's return value of -2 as a large, positive number. * testsuite/subst-mb-incomplete.sh: New file/test. * testsuite/Makefile.am (T): Add it. Reported by Hanno Böck in http://debbugs.gnu.org/22127. Hanno used the AFL fuzzer to find the segfault-inducing input from which I derived the test's input. Introduced by v4.2.2-76-g78e8e58, this bug was never in a release. 2015-12-12 Jim Meyering tests: add helper prog, get-mb-cur-max and re_en_utf8_locale_ function 2015-08-23 Jim Meyering sed: reject a ":" command without a label Before, sed would accept it and treat it as a label whose name had length 0, and that could be referenced via a "b" or "t" command with no label. * sed/compile.c (errors): Add the new diagnostic. (COLON_LACKS_LABEL): Define. (compile_program): Reject ":" command with no label. * testsuite/colon-with-no-label.sh: New file. * testsuite/Makefile.am (T): Add it. * NEWS (Bug fixes): Mention it. Reported by Stephane Chazelas in http://bugs.gnu.org/21250 2015-08-01 Assaf Gordon build: comment out definitions of unused macros * sed/compile.c (OPEN_BRACE): Comment out definition, to prevent gcc-5.2 from failing with -Werror -Wunused-macros. (END_ERRORS): Likewise. 2015-07-30 Jim Meyering tests: avoid false-failure on system with no ja_JP.eucJP locale * testsuite/invalid-mb-seq-UMR.sh: Use the configure-detected locale name, $LOCALE_JA, rather than hard-coding ja_JP.eucJP. If the required type of locale was not detected, then skip this test. Reported by Assaf Gordon in http://bugs.gnu.org/21124 2015-07-24 Jim Meyering maint: remove dead code * sed/utils.c (ck_mkstemp): Remove $TMP/$TMPDIR-envvar-using code that would have been run only if this function's second parameter were to be NULL, but that could never happen, given the current and sole caller. Also, use mode_t as the type for the saved umask value, not "int". Also, move each declaration down to point of definition. * sed/utils.h (ck_mkstemp): Mark each of the four parameters as nonnull. 2015-07-15 Stanislav Brabec sed: fix --follow-symlinks to work when reading stdin When reading from stdin, use of --follow-symlinks would cause failure: $ echo abc | sed --follow-symlinks s/a/d/ sed: cannot stat -: No such file or directory * sed/execute.c (open_next_file): Set input->in_file_name earlier, so we can rearrange logic to avoid calling follow_symlink("-"). * testsuite/follow-symlinks-stdin.sh: New file. * testsuite/Makefile.am (T): Add it. * NEWS (Bug fixes): Mention it. Bug introduced by commit v4.2.1-13-g84066bf. http://bugs.gnu.org/20795 2015-06-23 Jim Meyering typo fix: s/singlebyte/single-byte/ * sed/compile.c (compile_program, normalize_text): Fix typo. * sed/mbcs.c (brlen): Likewise. 2015-06-20 Jim Meyering sed -i: do not treat "-" as a file name Most GNU utilities treat "-" as standard input, sed itself does that -- in most contexts. However, since the addition of support for --in-place (-i) in sed-4.0, sed -i has treated a "-" argument as a file name, i.e., like ./-. Now, that usage evokes a diagnostic: $ sed -i s/a/b/ - sed: couldn't edit -: is a terminal If you require the old behavior, specify the file name as "./-". Prompted by the report/patch from Stanislav Brabec in http://bugs.gnu.org/20796 to document the strangely inconsistent legacy behavior. * sed/execute.c (open_next_file): Call panic before even attempting to operate on the file descriptor. * testsuite/in-place-hyphen.sh: New file. Test for this. * testsuite/Makefile.am (T): Add it. * NEWS (Feature removal): Mention it. Admittedly, the old behavior feels more like a misfeature. 2015-06-20 Jim Meyering tests: initialize $fail to 0 * testsuite/range-overlap.sh: Otherwise, with $fail set to nonzero in the environment, this test would report failure, even when it should pass. 2015-05-10 Jim Meyering build: avoid warning about implicit declaration of unlink * sed/sed.c: Include , for declaration of unlink. Without this change, commit makes sed v4.2.2-84-g7689015 fail to compile when configured with --enable-gcc-warnings. sed -i: don't leave behind a temporary sedXXXXXX file For example, running a command like "sed -i s//b/ F" would fail to remove a temporary file named sedXXXXXX (for random XXXXXX) in the directory alongside F. * sed/sed.c (G_file_to_unlink): New global. (register_cleanup_file, cancel_cleanup, cleanup): New functions. (main): Call atexit. * sed/execute.c (open_next_file): Register for unlink. (closedown): Call cancel_cleanup right after the rename. * sed/sed.h: Declare two of the new functions. * NEWS (Bug fixes): Mention it. * testsuite/temp-file-cleanup.sh: New file. Test for this. * testsuite/Makefile.am (T): Add it. Reported by David Jones in http://bugs.gnu.org/20002. 2015-05-07 Jim Meyering maint: remove stray blank line in the middle of a comment * sed/utils.c (ck_mkstemp): As above. tests: skip the new test in presence of buggy valgrind * testsuite/invalid-mb-seq-UMR.sh: Upon failure due to a valgrind-internal assertion, skip this test. Reported by Norihiro Tanaka in http://debbugs.gnu.org/20490 2015-05-06 Norihiro Tanaka sed: fix mishandling of overlapping address ranges When the line number ranges of two or more editing commands overlap, sed applies all commands but the first to a line that is just beyond the union of all ranges. E.g., with this command, seq 9|sed '2,7d;3,6d', sed would mistakenly delete line 8. * sed/execute.c (match_an_address_p) [ADDR_IS_NUM]: Make this function work also in this case. (match_address_p): Move the ADDR_IS_NUM + ...->line_number == ...->addr_number comparison "up" into match_an_address_p, so we can hoist two similar if/return blocks out of "if" and "else" branches. Change so that this function returns false when the current line number is outside the specified range. * testsuite/range-overlap.sh: New file, to test for this. * testsuite/Makefile.am (T): Add it to the list. * NEWS (Bug fixes): Mention it. Reported as http://bugs.gnu.org/19899 2015-05-04 Jim Meyering build: do not require autoconf.git * configure.ac: Use AC_CONFIG_MACRO_DIR, not the new (as yet unreleased) AC_CONFIG_MACRO_DIRS. Reported by Hongyi Zhao in http://debbugs.gnu.org/20424 2015-05-03 Jim Meyering maint: housekeeping: sync a few rules from coreutils * cfg.mk (sc_preprocessor_indentation): New rule, from coreutils. (local-checks-to-skip): Exempt it, for now. (sc_THANKS_in_sorted): Another rule from coreutils. Enabled. (announcement_Cc_): Define, so the auto-generated announcement message template includes sed-devel@ as a recipient. (update-copyright-env) [UPDATE_COPYRIGHT_USE_INTERVALS]: Update to use =2, rather than=1, so that the next time we update copyright dates, it will collapse a few more ranges. maint: tweak comments and rearrange code slightly * sed/execute.c (str_append): Trivial NSC code transformation. (str_append_modified): Tweak another comment. tests: test for just-fixed UMR bug * testsuite/invalid-mb-seq-UMR.sh: New file. Test for just-fixed bug. * testsuite/Makefile.am (T): New variable. Add the two init.sh-using test names here, rather than in two separate lists. (SEDTESTS, EXTRA_DIST): Use $(T). (EXTRA_DIST): Add init.cfg. * testsuite/init.cfg: New file, to add functions from coreutils. (require_valgrind_, print_ver_): New functions. 2015-05-03 Norihiro Tanaka sed: avoid a UMR bug when processing an invalid multibyte sequence * sed/execute.c (str_append_modified): When attempting to append an invalid multibyte sequence, e.g., from the RHS of a substitution expression, we would read uninitialized memory. Don't capitalize first word of diagnostic; remove exclamation point. Wrap new translatable strings in _(...). * NEWS (Bug fixes): Mention it. This addresses http://debbugs.gnu.org/20490 2015-05-02 Jim Meyering maint: remove SED_FEATURE_VERSION; use PACKAGE_VERSION instead * configure.ac (SED_FEATURE_VERSION): Remove definition et al. * sed/compile.c (compile_program): Remove sole use. Use PACKAGE_VERSION instead. maint: add "/*~" to build-aux/ and m4/.gitignore files Otherwise, bootstrap would add entries for individual file names. * build-aux/.gitignore: Add /*~ * m4/.gitignore: Likewise. 2015-05-01 Jim Meyering maint: enable sc_file_system * cfg.mk (local-checks-to-skip): Remove its exemption. * BUGS (characters): s/filesystem/file system/ * ChangeLog-2014 (setup_jump): Likewise. * doc/sed-in.texi: Likewise. * doc/sed.texi: Likewise. build: require gettext 0.19.2, perl-5.5 * configure.ac: Require gettext-0.19.2. * bootstrap.conf: List perl and gettext version requirements. 2015-04-27 Jim Meyering tests: set sed's expected failure status to 4 * testsuite/help-version.sh (expected_failure_status_sed): As above. 2015-04-26 Jim Meyering maint: invoke make via $(MAKE), not via "make" * Makefile.am (html): Invoke make via $(MAKE). (full-distcheck): Likewise. use generated version.[ch] and add init.sh and help-version.sh * Makefile.am (DISTCLEANFILES): Initialize. (EXTRA_DIST): Add .version. (BUILT_SOURCES): Set to .version. (.version): New rule. (THANKS): Depend on .version. * sed/Makefile.am: Add rules to build version.[ch]. * sed/sed.c: Include version.h, and use the new variable, Version. * testsuite/Makefile.am: Arrange to distribute and run the new help-version.sh test. Also distribute the new init.sh. * testsuite/help-version.sh: New file. * testsuite/init.sh: Likewise. * .gitignore: Ignore the generated files, .version and src/version.[ch]. 2015-03-01 Jim Meyering maint: COPYRIGHT_YEAR: remove, along with tests that use it This is covered by other tests, and the general --help/--version checks provided by help-version.sh (about to be added) are more thorough. * configure.ac (COPYRIGHT_YEAR): Remove definition and AC_SUBST. * Makefile.am (dist-hook): Remove test. * testsuite/Makefile.am (SEDTESTS): Remove "version". * testsuite/version.gin: Remove file. * .gitignore: Remove mention of generated file, version.good. 2015-01-01 Jim Meyering maint: initialize an mbstate_t with "= { 0, };", rather than with memset I.e., use this: mbstate_t mbstate = { 0, }; rather than this: mbstate_t mbstate; memset (&cur_stat, 0, sizeof (mbstate_t)); Also, prefer "sizeof VAR" over sizeof(TYPE). * sed/execute.c (execute_program, translate_mb): As above. * sed/compile.c (match_slash, compile_program, normalize_text): Likewise. maint: enable sc_prohibit_long_lines * cfg.mk (local-checks-to-skip): Remove the overall exemption, but continue to exempt some files in testsuite/. * configure.ac: Split a long line. * doc/sed-in.texi: Split long lines. * doc/sed.texi: Likewise. * sed/compile.c: Likewise. * sed/execute.c: Split long lines. (execute_program): Reorder "#ifdef HAVE_POPEN" and "if (pipe_fp != NULL)" if/else blocks to make the code clearer and (incidentally) the lines shorter, due to a decrease in nesting level. * sed/sed.c: Split long lines. * sed/utils.c: Likewise. * sed/utils.h: Likewise. maint: translate_mb: new function factored out of 'y'-handling case * sed/execute.c (translate_mb): New function, factored out of the "case 'y':" part of... (execute_program): ...here. maint: remove more unnecessary declarations * sed/execute.c: Remove now-duplicate forward declarations. maint: update copyright year ranges to include 2015; update gnulib 2014-12-20 Jim Meyering maint: enable sc_prohibit_strncpy check * cfg.mk (exclude_file_name_regexp--sc_prohibit_strncpy): Remove its exemption. * sed/compile.c (compile_program): Remove two misuses of strncpy; use memcpy instead. * sed/execute.c (do_list): Likewise, for one instance. maint: begin to use "size_t" more, where appropriate * sed/execute.c (do_list): Update some "int" locals to "size_t", and combine a few declarations with needlessly separate initializations. maint: update .gitignore * .gitignore: Ignore build artifacts. maint: remove po/sed.pot from version control * po/sed.pot: Remove file. It is unnecessary and undesirable to version-control a file like this (mechanically-derived). build: update build tool version requirements * bootstrap.conf (buildreq): Blindly update tool-version number requirements from those required by coreutils. maint: fix "make distcheck" failure * Makefile.am (EXTRA_DIST): List THANKS.in, not THANKS. Add .mailmap, too. build: build-aux/texi2dvi: remove file * build-aux/texi2dvi: Remove file. It was out of date and could no longer parse our .texi files. * Makefile.am (EXTRA_DIST): Don't list it here. * cfg.mk: Remove an exemption for that file. * doc/Makefile.am (TEXI2DVI): Remove definition. 2014-12-16 Jim Meyering remove support for the 'L' (fmt/flow-paragraph) command * sed/Makefile.am (sed_SOURCES): Remove fmt.c. * sed/execute.c (execute_program): * sed/fmt.c: Remove file. * sed/sed.h (fmt): Remove declaration. * doc/sed.texi: Remove documentation for 'L' command. * doc/sed-in.texi: Likewise. * NEWS (Feature removal): Document it. Prompted by the report from Jodie Cunningham that using this command with a large number could cause sed to segfault: https://bugs.launchpad.net/ubuntu/+source/sed/+bug/1400575 2014-12-01 Jim Meyering doc: NEWS: mention the gzip->xz release tarball switch * NEWS (Build-related): Mention that we are now distributing only .tar.xz archives; no more gzip-compressed tarballs. build: avoid warning about unused fchown result * sed/execute.c: Include "ignore-value.h". (closedown): Explicitly ignore fchown return value. * bootstrap.conf (gnulib_modules): Add ignore-value. * lib/.gitignore: Ignore the new .h file. * .gitignore: Ignore all **~ backup files. 2014-11-30 Jim Meyering build: use gnulib's manywarnings module bootstrap.conf: Add the module name. configure.ac: Copy boilerplate and exclusions from coreutils. sed/Makefile.am (AM_CFLAGS): Define in terms of the two new variables, $(WARN_CFLAGS) and $(WERROR_CFLAGS). m4/.gitignore: Add the two new .m4 files. maint: add three casts-in-initialization to avoid warnings * sed/compile.c (special_files): When initializing, cast string literals to (char *) to avoid warnings. maint: avoid a gcc "const discard" warning * sed/compile.c (compile_program): Make a local "const" to avoid discarding "const" attribute in assignment. maint: avoid more gcc warnings: apply _GL_ATTRIBUTE_PURE * sed/utils.h (get_buffer, size_buffer): Make each functions sole parameter "const" and declare the function itself with the "pure" attribute. * sed/utils.c (get_buffer, size_buffer): Adjust definitions to match. maint: add _Noreturn attribute to two functions This avoids two gcc may-be-noreturn warnings: * sed/sed.h (bad_prog): Add _Noreturn attribute. * sed/utils.h (panic): Likewise. maint: manually convert K&R to ANSI style decls; and add "static" Convert many function definitions from K&R to ANSI style. When possible, make a function static and remove the immediately- preceding declaration of that same function. Also, add the const to a few declarations. * sed/compile.c: As above. (special_files): Likewise. * sed/execute.c: Likewise. * sed/fmt.c: Likewise. * sed/mbcs.c: Likewise. * sed/regexp.c: Likewise. * sed/sed.c: Likewise. * sed/sed.h: Likewise. * sed/utils.c: Likewise. maint: declare several "int" locals to be of type size_t * sed/compile.c (compile_program): Using "int" was wrong: not only too narrow, but would also evoke warnings from gcc when comparing with variables of unsigned type. maint: avoid gcc warning about unused macro * sed/regexp.c (END_ERRORS): Remove definition of unused macro. maint: avoid false-positive used-uninit. warning from gcc * sed/sed.h (IF_LINT): Define. * sed/compile.c (snarf_char_class) [lint]: Use it to initialize DELIM, so that gcc doesn't report it is used uninitialized. maint: enable sc_prohibit_empty_lines_at_EOF check * cfg.mk (exclude_file_name_regexp--sc_prohibit_empty_lines_at_EOF): * cfg.mk (local-checks-to-skip): Remove its exemption. Adjust many files to end in a single newline, but exempt a few test suite input and output files. * testsuite/fasts.sed: Remove three empty lines at EOF. * BUGS: Remove empty line at EOF. * sed/mbcs.c: Likewise. * sed/sed.h: Likewise. * testsuite/BOOST.tests: Likewise. * testsuite/dc.inp: Add a newline at EOF; there was none. * testsuite/flipcase.sed: Likewise. * testsuite/utf8-1.good: Likewise. * testsuite/utf8-1.inp: Likewise. * testsuite/utf8-2.good: Likewise. * testsuite/utf8-2.inp: Likewise. * testsuite/utf8-3.good: Likewise. * testsuite/utf8-3.inp: Likewise. * testsuite/utf8-4.good: Likewise. * testsuite/utf8-4.inp: Likewise. * testsuite/xbxcx.good: Likewise. * testsuite/xbxcx.inp: Likewise. * testsuite/xemacs.good: Likewise. * testsuite/xemacs.inp: Likewise. maint: enable sc_prohibit_tab_based_indentation syntax check * cfg.mk (local-checks-to-skip): Remove its exemption. Exempt all files under testsuite/, for now, and a few others. * .gitmodules: Change each leading TAB to 8 spaces spaces. * BUGS: Likewise. * NEWS: Likewise. * configure.ac: Likewise. * sed/compile.c: Likewise. * sed/execute.c: Likewise. * sed/fmt.c: Likewise. * sed/regexp.c: Likewise. * sed/sed.c: Likewise. * sed/sed.h: Likewise. * sed/utils.c: Likewise. * sed/utils.h: Likewise. * COPYING: Update from coreutils. maint: enable sc_m4_quote_check syntax check * cfg.mk (local-checks-to-skip): Remove its exemption. * configure.ac: Fix the under-quoting problems thus exposed. maint: enable sc_cast_of_argument_to_free syntax check * cfg.mk (local-checks-to-skip): Remove its exemption. * testsuite/tst-rxspencer.c (mb_test): Remove a cast. maint: enable sc_cast_of_alloca_return_value syntax check * cfg.mk (local-checks-to-skip): Remove its exemption. * sed/regexp.c (match_regex): Do not cast return value of alloca. maint: enable sc_po_check syntax check * cfg.mk (local-checks-to-skip): Remove its exemption. * po/POTFILES.in: Adjust to reflect added/removed file names. maint: enable sc_prohibit_always_true_header_tests syntax check * cfg.mk (local-checks-to-skip): Remove its exemption. Exempt configure.ac. maint: enable sc_makefile_at_at_check syntax check * cfg.mk (local-checks-to-skip): Remove its exemption. * testsuite/Makefile.am (LDADD, TESTS_ENVIRONMENT): Use $(...) rather than obsolescent @...@ notation. maint: enable sc_space_tab syntax check * cfg.mk (local-checks-to-skip): Remove its exemption. * doc/groupify.sed: Reverse SP-TAB sequences. * sed/compile.c (compile_program): Remove spaces before TAB. * sed/execute.c (append_replacement): Likewise. maint: enable sc_program_name syntax check * cfg.mk (local-checks-to-skip): Remove its exemption. * sed/sed.c: Include progname.h". (program_name): Remove now-unnecessary declaration. (main): Call set_program_name. * bootstrap.conf (gnulib_modules): Add progname. * lib/.gitignore: Append the two new names. maint: enable more checks * cfg.mk (local-checks-to-skip): Remove exemptions for immutable NEWS as well as the two config.h checks. Also accept "sed.h", since the first thing it does is to include . * sed/sed.h: Change "config.h" to , for consistency with other GNU tools. * sed/utils.c: Likewise, here and for each of the following. * testsuite/bug-regex10.c: * testsuite/bug-regex11.c: * testsuite/bug-regex12.c: * testsuite/bug-regex13.c: * testsuite/bug-regex14.c: * testsuite/bug-regex15.c: * testsuite/bug-regex16.c: * testsuite/bug-regex21.c: * testsuite/bug-regex7.c: * testsuite/bug-regex8.c: * testsuite/bug-regex9.c: * testsuite/runptests.c: * testsuite/runtests.c: * testsuite/tst-boost.c: * testsuite/tst-pcre.c: * testsuite/tst-regex.c: * testsuite/tst-regex2.c: * testsuite/tst-rxspencer.c: maint: enable sc_const_long_option syntax check * sed/sed.c (main): Add "const" to decl of longopts. * cfg.mk (local-checks-to-skip): Remove its exemption. maint: enable double-word prohibition * cfg.mk (local-checks-to-skip): Remove exemption for sc_prohibit_doubled_word. Exempt 4 files under testsuite/. doc: remove doubled word * doc/sed-in.texi: Remove doubled "the". * doc/sed.texi: Likewise. maint: add .prev-version * .prev-version: New file. maint: turn on the test that prohibits use of HAVE_CONFIG_H * cfg.mk (local-checks-to-skip): Remove the exemption here. * testsuite/bug-regex10.c: Remove the use of the offending macro. * testsuite/bug-regex11.c: Likewise. * testsuite/bug-regex12.c: Likewise. * testsuite/bug-regex13.c: Likewise. * testsuite/bug-regex14.c: Likewise. * testsuite/bug-regex15.c: Likewise. * testsuite/bug-regex16.c: Likewise. * testsuite/bug-regex21.c: Likewise. * testsuite/bug-regex7.c: Likewise. * testsuite/bug-regex8.c: Likewise. * testsuite/bug-regex9.c: Likewise. * testsuite/runptests.c: Likewise. * testsuite/runtests.c: Likewise. * testsuite/tst-boost.c: Likewise. * testsuite/tst-pcre.c: Likewise. * testsuite/tst-regex.c: Likewise. * testsuite/tst-regex2.c: Likewise. * testsuite/tst-rxspencer.c: Likewise. maint: generate THANKS from THANKS.in * THANKS.in: Renamed from THANKS, adjusted formatting and added header. Also remove from this list of names those that are now generated automatically. * THANKS: Removed, now that it's generated. * thanks-gen: New file. * Makefile.am (EXTRA_DIST): Add thanks-gen (THANKS): New rule. * .mailmap: New file. maint: add cfg.mk, derived from grep's * cfg.mk (local-checks-to-skip): Exempt many failing tests. (old_NEWS_hash): Define. * NEWS: Tweak wording to avoid semblance of doubled word: in in-place. tests: hook up gnulib-tests * Makefile.am (SUBDIRS): Add gnulib-tests. * gnulib-tests/Makefile.am: New file. * configure.ac (AC_CONFIG_FILES): Add gnulib-tests/Makefile. * .gitignore: Ignore all but Makefile.am. maint: .gitignore: ignore more maint: remove ACLOCAL_AMFLAGS in favor of AC_CONFIG_MACRO_DIRS * Makefile.am (ACLOCAL_AMFLAGS): Remove this assignment. * configure.ac: Use this macro instead. maint: do not version-control generated files. * doc/sed.1: Remove. * po/sed.pot: Likewise. * po/Makevars: Likewise. * .gitignore: Add their names. maint: begin to modernize configure.ac * configure.ac: Turn on silent rules, by default. Use git-derived versions all the time. Required a new, signed v4.2.2 tag, which I've pushed. Distribute .tar.xz tarballs, not .bzip2 or .gz. * Makefile.am (AUTOMAKE_OPTIONS): Remove. * bootstrap.conf (gnulib_modules): Add git-version-gen. maint: convert obsolescent @VAR@ notation to $(VAR) * sed/Makefile.am (sed_LDADD): Use $(VAR), no @VAR@. maint: update copyright dates; use gnulib's update-copyright module * bootstrap.conf (gnulib_modules): Add update-copyright. Then, run "make update-copyright". * Makefile.am: Add copyright header. maint: remove all trailing white space First, run this command: git grep -l ' $'|xargs perl -pi -e 's/[ \t]+$//' Then some minor fix-up to make the two newly-failing tests pass once again: * testsuite/mac-mf.sed: Append this, s/ $//, to eliminate trailing spaces in the actual output. * testsuite/y-newline.good: Manually remove a trailing space between two concatenated prompts. maint: update autogenerated .gitignore files maint: update build/gnulib infrastructure; generate ChangeLog * ChangeLog-2014: Renamed from ChangeLog. * po/ChangeLog-2014: Renamed from po/ChangeLog. * Makefile.am (EXTRA_DIST): Add the two new file names. (dist-hook): Generate ChangeLog at tarball-creation time. (gen-ChangeLog): New rule. Just like the one in coreutils and grep. * bootstrap.conf (gnulib_modules): Add the modules, readme-release and gitlog-to-changelog, and revamp to be much more like grep and coreutils. * lib/Makefile.am: Update. maint: update bootstrap from gnulib 2014-09-06 Jim Meyering fixup: add the properly-named new test files * testsuite/Makefile.am (EXTRA_DIST): Add y-zero.*, not y-NUL.* here. maint: correct failing "version" test * configure.ac (COPYRIGHT_YEAR): Update to 2014, so the "version" test passes once again. * doc/sed.1: Also commit this auto-generated-yet-VC'd file. fix "y" to work with NUL in the RHS * sed/execute.c (do_list) [case 'y']: Handle NUL bytes in the RHS of a y/LHS/RHS/ transliteration. * testsuite/y-zero.good: New test-related files. * testsuite/y-zero.inp: * testsuite/y-zero.sed: * testsuite/Makefile.am (SEDTESTS): Add y-zero here. * testsuite/Makefile.tests: And here. * NEWS (Bug fixes): Describe it. The bug was reported by table@inventati.org, with the execute.c change mostly by Paolo Bonzini.