====================================================================== Version 1.0.1, by Aleksey Cheusov, Thu, 14 Aug 2014 01:44:39 +0300 Unbuffered (broken) stdout seen on Darwin was fixes. Who knows, this may happen on other systems too. ====================================================================== Version 1.0.0, by Aleksey Cheusov, Sun, 13 Jul 2014 19:10:48 +0300 mk-configure>=0.27.0 is required for build. Option -X was introduced to paexec(1) for ignoring calculator's stdout. PAEXEC_ENV environment variable sets a list of variables passed to the calculator. PAEXEC_TRANSPORT environment variable sets the transport unless option -t was applied. PAEXEC_NODES environment variable sets the nodes unless option -n was applied. pareorder(1) is a synonym for paexec_reorder(1) ====================================================================== Version 0.19.1, by Aleksey Cheusov, Mon, 26 Aug 2013 19:05:53 +0300 paexec: Fix for segfault seen on Linux Version 0.19.0, by Aleksey Cheusov, Sun, 25 Aug 2013 17:21:03 +0300 This release of paexec was successfully tested on the following platforms: NetBSD-6.1/amd64, OpenBSD-5.3/i386, FreeBSD-8.3/i386, Solaris-10/sparc, Solaris-11/amd64 and diverse Linux/{i386,amd64}. paexec is now selfcontained, libmaa is not needed anymore. Presentation paexec.pdf was added to presentation/. I hope it will help easier understand how "paexec" works. It is installed to ${DOCDIR}. paexec: - POSIX-2008 getline(3) is used for reading lines instead of home-made function. - t '' is equivalent to "no transport", spaces are trimmed. - "-n +NNN" has higher priority than -t, i.e. if they both are specified, transport is ignored. - Fix for -W1. - Environment variable PAEXEC_EOT was introduced. - Option -y was added to paexec(1) and paexec_reorder(1). - Option -C was added to paexec(1). paexec_reorder: - Option -x was added to paexec_reorder(1). A number of fixes and improvements in regression tests. ====================================================================== Version 0.18.0, by Aleksey Cheusov, Thu, 7 Mar 2013 15:17:33 +0300 paexec: - fixed: NULL dereference when the first line given on input is empty. Thanks to Sergey Revyako for bug report! - fixed: entire command passed tp ssh-like transport should be shquoted. In particular this fixes -x that didn't work with -t. Thanks to Sergey Revyako for bug report! - 'paexec -g' accepts empty strings as tasks. - More regression tests were added paexec_reorder.1: - Mistype fix ====================================================================== Version 0.17.0, by Aleksey Cheusov, Sun, 9 Sep 2012 02:07:34 +0300 paexec: - Option -x was added. With its help paexec can run one command per task. If -g is also specified, command's exit status is analysed. Appropriate task and dependants are marked as "failed" if it is non-zero. - First character of -n argument must be alphanumeric, `+', `_', `:' or `/'. Other symbols are reserved for future extentions. - With '-n :filename' paexec reads a list of nodes from the specified file. - With a help of new option '-m t=' end of task string may be specified, which is an empty line by default. - Option -md= was added that overrides the default delimiter (space character) between tasks in graph mode (-g). - Output line that contains failed dependants no longer ends with unnecessary space. - Long options were completely removed. paexec_reorder: - Fix. "paexec_reorder -g" now handles correctly failed tasks' output. One extra line after "fatal" is expected. - Options -m was added. It does the same things as paexec's -m. More examples of use and regression tests. Documentation update, clean-ups and improvements. Regression tests: - Signals handling was fixed in. - LC_ALL is always set to C in regression tests, this fixes some problems in internationalized environment. mk-configure>=0.23.0 is required at build time ====================================================================== Version 0.16.1, by Aleksey Cheusov, Wed, 23 Mar 2011 00:14:15 +0200 paexec.1 and paexec_reorder.1 are included to paexec-0.16.1.tgz. Just like in paexec<=0.15.1 pod2man is not needed for building paexec. ====================================================================== Version 0.16, by Aleksey Cheusov, Fri, 11 Mar 2011 11:58:31 +0200 Project's structure has been reorganized. Now the top-level Makefile uses mkc.subprj.mk. This adds a lot of flexibility in building the project and development. See doc/INSTALL for updated installation instructions. New modes for reordering tasks were added: -W0 and -W2 See the manual page for details about option -W. Long options are considered deprecated. They are still supported but will be removed in the future. Please use POSIX short options. At the moment use of them produces warning message on stderr. Fix the compilation bug on old versions of OpenBSD (at least <=3.8) and probably other systems where intptr_t is declared in inttypes.h but not in stdint.h Fix for 'mkcmake test' failure on Solaris and HP-UX because their /usr/bin/awk sucks. On these platforms now it is possible to run the following command. env AWK=/full/path/to/gawk mkcmake all test ====================================================================== Version 0.15.0, by Aleksey Cheusov, Sun, 10 Oct 2010 18:06:31 +0300 After some thoughts I decided to switch from plain mk-files to mk-configure build system. It provides very cool features and makes development drammatically easier. It also makes Makefiles much cleaner and easier. Installation instructions has been updated in README file. PAEXEC reads the tasks from stdin and distributes them to all available hosts or CPU. If some tasks are easy and can be made quickly while others require much more CPU time it makes sense to reorder tasks calcultion in order to reduce total calculation time and reduce computers/CPUs idle time. For this purpose -W option is added to paexec(1) using weights assigned to each tasks. At the moment only two values are allowed for : 0 -- do not use weights at all, 1 -- run heavier tasks first as soon as possible. New tool paexec_reorder(1) for reordering sliced output of paexec(1). It is written in runawk. So, you'll need it at run time. FIXED: 1 byte buffer overflow if -d option is applied. Documentation update. Tons of new regression tests. Regression tests framework has been significantly reworked. Clean-ups. README: notes about my Debian/Lenny/x86 repository. Minor fixes (warning messages) for different compilers. ====================================================================== Version 0.14.0, by Aleksey Cheusov, Sun, 3 Jan 2010 12:38:54 +0200 fixed: "paexec -z" without -s never worked. Now it is fine. New options -g|--graph are synonyms for -s|--pos. New option -w for waiting for failed nodes if they ALL failed. By default (with -Z) if ALL nodes fail paexec exits with error. New option -m for setting alternative strings for success/failure/fatal. Target for installing directories has been renamed from install-dirs to installdirs. Other minor fixes and code clean-ups. ====================================================================== Version 0.13.0, by Aleksey Cheusov, Sat, 7 Mar 2009 19:17:03 +0200 FIXED: When 'paexec -s' retreives 10000 tasks, it allocates 10000*10000*sizeof(int) bytes for detecting cycles, i.e. ~400Mb on 32-bit system or 800Mb on 64-bit system. This is absolutely inacceptable. This bad algorithm is replaced with new one which doesn't need quadratic matrix and works much faster. ADDED: -Z option When I<-z> applied, if a I fails, appropriate node is marked as broken and is excluded from the following task distribution. But if B<-Z> applied, every I seconds an attempt to rerun a comand on a failed node is made. I<-Z> implies I<-z>. This option makes possible to organize clusters over unreliable networks/hardware. No EINTR wrappers anymore (iread, xread etc.). SIGCHLD and SIGALRM are blocked most of the time. They are unblocked before select(2) and blocked just after it. SA_RESTART is not used anymore. Minor clean-ups in Makefile. ====================================================================== Version 0.12.1, by Aleksey Cheusov, Tue, 23 Dec 2008 20:33:20 +0200 FIX: support for -z appeared in paexec-0.12.0 was actually incomplete :-/ Now everything should be fixed. More regression tests. Makefile: adapted for FreeBSD make which doesn't support .PARSEDIR ====================================================================== Version 0.12.0, by Aleksey Cheusov, Mon, 22 Dec 2008 21:48:01 +0200 ADDED: -z option. If applied, read/write(2) operations from/to nodes becomes not critical. In case paexec has lost connection to the node, it will reassign failed task to another node and, if -s option applied, will output string "fatal" to stdout. This makes paexec resistant to the I/O errors, as a result you can create paexec clusters even over network consisting of unreliable hosts (Internet?). Failed hosts are marked as such and will not be used during the current run of paexec. NOTE: "success", "failure" and "fatal" strings should be used to process the output of 'paexec -s'. select(2) do not listen stdin (fd=0) anymore. Blocking read(2) is used to read tasks from stdin. Makefile: CPPFLAGS -> CFLAGS, FreeBSD make Mk scripts don't use CPPFLAGS. ====================================================================== Version 0.11.0, by Aleksey Cheusov, Sat, 25 Oct 2008 17:49:47 +0300 paexec -s: before beginning actual work an input tasks graph is checked for cycles. If they are detected, paexec exits with error. minor fix in man page ====================================================================== Version 0.10.1, by Aleksey Cheusov, Sat, 25 Oct 2008 02:00:55 +0300 This version of paexec was sucessfully tested on the following platforms: NetBSD/x86 NetBSD/Alpha(64bit) Linux/x86 Linux/x86-64 Interix-3.5/x86 FreeBSD/x86 Solaris-10/x86 Minor fix for rhomb-like dependencies (paexec -s). Suppose A depends on (B1 and B2), (B1 and B2) depends on C. If C fails, earlier paexec versions produced the following output failure C B1 A B2 A That is, A was output twice. Now it is fixed and A is printed only once. New regression test for this. By default(!) getopt_long(3) is enabled on the following platforms (macro): __NetBSD__, __FreeBSD__, __OpenBSD__, __DragonFly__, __linux__ and __APPLE__. On others only short options provided by getopt(3) are used. This means that long option may to not work on your platform. fix in manual page: accept(2) -> select(2) code clean-ups .sinclude removed from Makefile tests/test.sh: diff -U ---> standard diff -C10 gawk appeared by mistake removed ====================================================================== Version 0.10.0, by Aleksey Cheusov, Sun, 31 Aug 2008 21:37:59 +0300 Lots of new regression tests README file: 'make test' is documented ADDED: -s option Partially ordered set of tasks are read from stdin. Instead of autonomous tasks, graph of the tasks is read from stdin. In this mode every task can either FAIL or SUCCEED. As always an empty line output by command means end of task. The line before it shows an EXIT STATUS of the task. The word "failure" means failure, "success" - success. See examples/1_div_x/1_div_X_cmd for the sample. An input line (paexec's stdin) should contain either single task without spaces inside or two tasks separated by single space character, e.g. task1task2. task1task2 line means that task1 must be done before task2 and it is mandatory, that is if task1 fail all dependent tasks (including task2) are also failed recursively. Tasks having dependencies are started only after all dependencies are succeeded. When a task succeeds paexec outputs "success" word just before end_of_task marker (see -e or -E), otherwise "failure" word is output followed by a list of tasks failed because of it. Samples: tasks (examples/make_package/make_package_tasks file) textproc/dictem devel/autoconf wip/libmaa devel/gmake wip/libmaa wip/libmaa wip/dict-server wip/libmaa wip/dict-client devel/m4 wip/dict-server devel/byacc wip/dict-server devel/byacc wip/dict-client devel/flex wip/dict-server devel/flex wip/dict-client devel/glib2 devel/libjudy command (examples/make_package/make_package_cmd__flex) #!/usr/bin/awk -f { print $0 # print a package name if ($0 == "devel/flex") print "failure" # cannot build flex ;-) else print "success" # all other packages are ok print "" # end of task marker fflush() } output of "paexec -s -l -c make_package_cmd__flex -n +10 \ < make_package_tasks" 3 devel/autoconf 3 success 4 devel/gmake 4 success 7 devel/m4 7 success 8 devel/byacc 8 success 9 devel/flex 9 failure 9 devel/flex wip/dict-server wip/dict-client 10 devel/glib2 10 success 11 devel/libjudy 11 success 1 textproc/dictem 1 success 2 wip/libmaa 2 success ====================================================================== Version 0.9.0, by Aleksey Cheusov, Sun, 15 Jun 2008 10:23:52 +0300 -t '' means "no transport". This significantly simplifies writing shell scripts with paexec. Added: tests for this case. paexec has no limited internal buffers anymore. All they are resized automatically as it is needed. PAEXEC_BUFSIZE environment variable sets an *initial* buffer size, not *maximum* one. README: note about non-standard function getopt_long, and advice how to build paexec on platforms with no getopt_long support (HP-UX, Solaris etc.). More regressions tests paexec.1: minor corrections. 'make test' fix: In case regression test fails, 'make test' exits with non-zero exit status. paexec can be built with ancient version pmake-1.45 (found in som Linux distributions). paexec -h|--help outputs messages to stderr - my new religion :-) ====================================================================== Version 0.8.0, by Aleksey Cheusov, Tue, 4 Mar 2008 00:32:56 +0200 New options implemented: -E, -i and -I See manual page for details. Fixes and minor improvements in the documentation ====================================================================== Version 0.7.0, by Aleksey Cheusov, Tue, 26 Feb 2008 23:57:15 +0200 new -e|--eot option implemented. It prints the end-of-task marker (an empty line) to stdout. See manual page. WARNS=4 to see compilation warnings. several gcc warnings fixed. fixed: '-n +0' and '-n ""' now fails with error message. minor fix in man page minor clean-ups ====================================================================== Version 0.6.0, by Aleksey Cheusov, Thu, 24 Jan 2008 21:31:26 +0200 First publicly available release ======================================================================