diff options
author | Fun <just.the.real.fun@gmail.com> | 2018-02-24 17:49:54 +0200 |
---|---|---|
committer | Fredrik Rinnestam <fredrik@crux.nu> | 2018-03-25 03:06:43 +0200 |
commit | ece7f90bf8b1fb747e4607cdfb054ef5d22a13e5 (patch) | |
tree | 91375adc071a86a964133d56958d8ead02ce01f0 | |
parent | 9acb4ea98b0f97f822c75aa710b664827cbe6b17 (diff) | |
download | pkgutils-ece7f90bf8b1fb747e4607cdfb054ef5d22a13e5.tar.gz pkgutils-ece7f90bf8b1fb747e4607cdfb054ef5d22a13e5.tar.xz |
faster strip_files() function
Tested with a port with 5500 output files, from which 53 candidates
for stripping. The timing was:
- 30 secs (old code)
- 0.2 secs (new code - current patch)
- 1.4 secs (new code with proper quoting - not commited)
Most of the time is spent in getting the output from the file program.
The old code started the file program for every file.
The new/present code starts N=$(nproc) processes in parallel with 10 input
files for each 'file' process. The output of the file program is feed
to an awk process which filters-out only the candidates for stripping.
This process runs in parallel too (but with one file per strip process).
The --no-buffer options is used because it sounds good (the strip should
start as soon as one of the file processes has a verdict for one of
their 10 files), but I didn't measure it.
The "xargs -r -L10 -P$N" command will miss the files with spaces.
For a file named "a b" it will spawn:
file "a" "b"
A slower version, with proper quoting, "xargs -r -L1 -P$N -I{} file ... '{}'",
will spawn:
file "a b"
* xargs will force -L1 if -I{} is used
Given that the file process doesn't return error codes for non-existing
files, and that there is a very low probability that we have ports with
filenames constaining spaces that are worth stripping them, I choose to
keep the faster (non perfect) version.
-rwxr-xr-x | pkgmk.in | 27 |
1 files changed, 14 insertions, 13 deletions
@@ -409,7 +409,8 @@ refresh_signature() { } strip_files() { - local FILE FILTER + local FILTER + local N=$(nproc) cd $PKG @@ -419,18 +420,18 @@ strip_files() { FILTER="cat" fi - find . -type f -printf "%P\n" | $FILTER | while read FILE; do - case $(file -b "$FILE") in - *ELF*executable*not\ stripped*) - strip --strip-all "$FILE" - ;; - *ELF*shared\ object*not\ stripped*) - strip --strip-unneeded "$FILE" - ;; - current\ ar\ archive) - strip --strip-debug "$FILE" - esac - done + find . -type f -printf "%P\n" | $FILTER \ + | xargs -r -L10 -P$N \ + file --no-buffer --separator '>' \ + -e apptype -e ascii -e encoding -e tokens \ + -e cdf -e compress -e tar -e text \ + | awk ' + BEGIN { FS = ">[ ]+" } + $0 ~ /ELF.*executable.*not stripped/ { print "--strip-all \"" $1 "\"" } + $0 ~ /ELF.*shared object.*not stripped/ { print "--strip-unneeded \"" $1 "\"" } + $2 == "current ar archive" { print "--strip-debug \"" $1 "\"" } + ' \ + | xargs -r -L1 -P$N strip } compress_manpages() { |