[PATCH] Tweaks to build process for embedded scripts

[Ron Yorston]

- Force a rebuild if a script in applets_sh is changed.

- Move the dummy usage messages for custom applets to usage.h and
change the name from 'dummy' to 'scripted'.

- Hide an error from gen_build_files.sh if an embed directory exists
but is empty.

- Strip leading comments and blank lines from embedded scripts before
compressing them. Removing all comments would be nice but is hard.

Signed-off-by: Ron Yorston <***@pobox.com>
---
Makefile | 2 +-
include/applets.src.h | 6 ------
include/usage.src.h | 3 +++
scripts/embedded_scripts | 12 ++++++++++--
scripts/gen_build_files.sh | 4 ++--
5 files changed, 16 insertions(+), 11 deletions(-)

diff --git a/Makefile b/Makefile
index 4b5a01df9..f0b4da234 100644
--- a/Makefile
+++ b/Makefile
@@ -853,7 +853,7 @@ quiet_cmd_split_autoconf = SPLIT include/autoconf.h -> include/config/*
quiet_cmd_gen_embedded_scripts = GEN include/embedded_scripts.h
cmd_gen_embedded_scripts = $(srctree)/scripts/embedded_scripts include/embedded_scripts.h $(srctree)/embed $(srctree)/applets_sh
#bbox# piggybacked generation of few .h files
-include/config/MARKER: scripts/basic/split-include include/autoconf.h $(wildcard $(srctree)/embed/*) $(srctree)/scripts/embedded_scripts
+include/config/MARKER: scripts/basic/split-include include/autoconf.h $(wildcard $(srctree)/embed/*) $(wildcard $(srctree)/applets_sh/*) $(srctree)/scripts/embedded_scripts
$(call cmd,split_autoconf)
$(call cmd,gen_bbconfigopts)
$(call cmd,gen_common_bufsiz)
diff --git a/include/applets.src.h b/include/applets.src.h
index a9db5d160..60968cec7 100644
--- a/include/applets.src.h
+++ b/include/applets.src.h
@@ -22,12 +22,6 @@ s - suid type:
BB_SUID_REQUIRE or BB_SUID_MAYBE applet.
*/

-#define NOUSAGE_STR "\b"
-
-#define dummy_trivial_usage NOUSAGE_STR \
-
-#define dummy_full_usage "" \
-
#if defined(PROTOTYPES)
# define APPLET(name,l,s) int name##_main(int argc, char **argv) MAIN_EXTERNALLY_VISIBLE;
# define APPLET_ODDNAME(name,main,l,s,help) int main##_main(int argc, char **argv) MAIN_EXTERNALLY_VISIBLE;
diff --git a/include/usage.src.h b/include/usage.src.h
index 00369dfb3..d22efd3ba 100644
--- a/include/usage.src.h
+++ b/include/usage.src.h
@@ -14,6 +14,9 @@

#define NOUSAGE_STR "\b"

+#define scripted_trivial_usage NOUSAGE_STR
+#define scripted_full_usage ""
+
#if !ENABLE_USE_BB_CRYPT || ENABLE_USE_BB_CRYPT_SHA
# define CRYPT_METHODS_HELP_STR "des,md5,sha256/512" \
" (default "CONFIG_FEATURE_DEFAULT_PASSWD_ALGO")"
diff --git a/scripts/embedded_scripts b/scripts/embedded_scripts
index c2e7c6961..b419bcb1f 100755
--- a/scripts/embedded_scripts
+++ b/scripts/embedded_scripts
@@ -63,15 +63,23 @@ then
exit 1
fi

+# remove leading comments and blank lines
+strip_header() {
+ awk '
+ /^#/ { if (!found) next; }
+ /^$/ { if (!found) next; }
+ { found=1; print; }' $1
+}
+
concatenate_scripts() {
for i in $custom_scripts
do
- cat $custom_loc/$i
+ strip_header $custom_loc/$i
printf '\000'
done
for i in $applet_scripts
do
- cat $applet_loc/$i
+ strip_header $applet_loc/$i
printf '\000'
done
}
diff --git a/scripts/gen_build_files.sh b/scripts/gen_build_files.sh
index 64e4bffa9..362632df3 100755
--- a/scripts/gen_build_files.sh
+++ b/scripts/gen_build_files.sh
@@ -23,9 +23,9 @@ custom_scripts()
custom_loc="$1"
if [ -d "$custom_loc" ]
then
- for i in $(cd "$custom_loc"; ls *)
+ for i in $(cd "$custom_loc"; ls * 2>/dev/null)
do
- printf "APPLET_SCRIPTED(%s, scripted, BB_DIR_USR_BIN, BB_SUID_DROP, dummy)\n" $i;
+ printf "APPLET_SCRIPTED(%s, scripted, BB_DIR_USR_BIN, BB_SUID_DROP, scripted)\n" $i;
done
fi
}

[Denys Vlasenko]

Post by Ron Yorston
- Force a rebuild if a script in applets_sh is changed.
- Move the dummy usage messages for custom applets to usage.h and
change the name from 'dummy' to 'scripted'.
- Hide an error from gen_build_files.sh if an embed directory exists
but is empty.
- Strip leading comments and blank lines from embedded scripts before
compressing them. Removing all comments would be nice but is hard.

Well, this last item can mangle scripts. Let's not do this.
Please resend with first three items only.

[Ron Yorston]

Post by Denys Vlasenko

Post by Ron Yorston
- Strip leading comments and blank lines from embedded scripts before
compressing them. Removing all comments would be nice but is hard.

Well, this last item can mangle scripts. Let's not do this.

What's the concern?

The awk script that removes the lines is:

/^#/ { if (!found) next; }
/^$/ { if (!found) next; }
{ found=1; print; }

It's pretty conservative in what it deletes. As soon as it finds a
line that doesn't match its definition of a comment it copies the
rest of the script verbatim.

It would break scripts that use a shebang line to run a different
interpreter but they aren't supported in this context anyway.

Ron

[Kang-Che Sung]

Post by Ron Yorston

Post by Denys Vlasenko

Post by Ron Yorston
- Strip leading comments and blank lines from embedded scripts before
compressing them. Removing all comments would be nice but is hard.

Well, this last item can mangle scripts. Let's not do this.

What's the concern?
/^#/ { if (!found) next; }
/^$/ { if (!found) next; }
{ found=1; print; }
It's pretty conservative in what it deletes. As soon as it finds a
line that doesn't match its definition of a comment it copies the
rest of the script verbatim.
It would break scripts that use a shebang line to run a different
interpreter but they aren't supported in this context anyway.

Script stripping should be optional, for at least two reasons:
1. It's beyond the scope of the script embedding feature, and it would better
be implemented and maintained as a separate tool.
2. Vendor may sign the scripts or publish their hashes or do something with
them so that every bit of the script must remain intact.

[Ron Yorston]

Post by Kang-Che Sung
1. It's beyond the scope of the script embedding feature, and it would better
be implemented and maintained as a separate tool.

I don't think it's out of scope. If you're handing your scripts over
to the tender mercies of the BusyBox build process you should expect
stuff to happen to them.

Letting authors indulge their literary aspirations without bloating the
binary seems a nice feature to have.

Post by Kang-Che Sung
2. Vendor may sign the scripts or publish their hashes or do something with
them so that every bit of the script must remain intact.

I suspect that most authors won't care. Those who do need their scripts
to be untouched should just ensure the first line doesn't match either
of the regular expressions. The old csh hack of putting ': /bin/sh'
on the first line would do, for example. And has a nice retro feel.

Ron

[Kang-Che Sung]

Post by Ron Yorston

Post by Kang-Che Sung
1. It's beyond the scope of the script embedding feature, and it would better
be implemented and maintained as a separate tool.

I don't think it's out of scope. If you're handing your scripts over
to the tender mercies of the BusyBox build process you should expect
stuff to happen to them.
Letting authors indulge their literary aspirations without bloating the
binary seems a nice feature to have.

Post by Kang-Che Sung
2. Vendor may sign the scripts or publish their hashes or do something with
them so that every bit of the script must remain intact.

I suspect that most authors won't care. Those who do need their scripts
to be untouched should just ensure the first line doesn't match either
of the regular expressions. The old csh hack of putting ': /bin/sh'
on the first line would do, for example. And has a nice retro feel.

Dammit. Why should I workaround my script just for an ugly
"feature" you employ if I were building it?
This isn't funny. Just take it off. Or make it _optional_.