Bash:Lesser Known Bits
======================
:author: Aaron Ball
:email: nullspoon@iohq.net
I won't lie, bash is my shell of choice (as if that's not obvious). Sure, the
ability to handle arrow keys, a command history, colors, and escape sequences
for terminal formatting are all great pieces, but most other shells can do
those things as well. What really makes bash stand out? There's a pretty good
list of things that are lesser known but are super useful, albeit not always
often though. All of these are well documented in the bash man page, but that
one is not exactly easy to find stuff in unless you know what you're looking
for. Running it through the wc command, the bash man page apparently has 41,452
words. All that aside though, this is a list of some lesser known things I use
occasionally (about once a week-ish) from our friend bash.
[[one-liner-loops]]
== One-liner Loops
This is one that is supported by most if not all of the other shells out there,
but it is still super useful and I don't see it used often. A one-liner loop is
effectively a very short (one line in fact) script used to perform a small
number of operations (it gets confusing if you do too many) in bulk. A good
example here is with a server environment of any size greater than I'd say two.
I frequently need to check lots of servers for something, be it the existence
of a file, the status of a file in comparison with a local copy (diffs), bulk
modifying remote files using sed, etc.
Recently though, I needed to verify the installed version of sudo
specifically on a list of about 50 servers. I sent the list of servers
to a text file, one server per line, and did the following and had my
answer within about 30 seconds (it takes a few hundred milliseconds for
ssh connections to establish on our atrociou...er...awesome network).
----
for i in $(cat ./servers.list); do echo $i; ssh user@$i 'sudo -V | grep "I/O plugin version"'; done
----
Presto! A big list of sudo versions across the entire environment.
[[process-substitution]]
== Process Substitution
This one is really great. Some commands require one or more file paths
to do what they need to do. A good example is diff. The diff command
requires two file path parameters: file a and file b. What if you want
to diff the outputs of two remote files though? Using process
substitution, we can cat out a remote file using the typical command,
+ssh user@server 'cat /etc/something'+, and have the output
go to a local temp file for the life of the command calling it so we
have something to work on. For example...
----
diff /etc/something <(ssh user@server 'cat /etc/something')
----
What we have here is a diff of the local /etc/something file and the remote
/etc/something. The ssh connection string is encapsulated in a +<()+. This is
the process substitution. This doesn't just work with remote files though. Say
for instance you wanted to diff the contents of a directory on a local system
and a remote system. Here's how you'd do that.
... or comparing the output of two remote commands...
----
diff <(ls -1 /var/log/) <(ssh user@server 'ls -1 /var/log/')
----
Here we used process substitution to write the output of +ls -l /var/log+ to a
temp file, then write the output of the same command run on another system over
ssh to yet another temp file, then we use diff as usual to show us what is
different. If you really wanted to get crazy, you could throw this into a bash
one-liner loop and run the diff on multiple systems.
[[brace-expansion]]
== Brace Expansion
Brace expansion is really neat and I think super handy. This is the one I don't
have a lot of use for though. It gets used about once every few scripts or
about once or twice a month. Brace expansion is effectively on-the-fly array
loops for commands. For a simple example, say you wanted to create three
directories: dev, test, and prod. To create these without brace expansion,
you'd have to run _mkdir_ three times. With brace expansion, you can do this
----
mkdir {dev,test,prod}
----
That's cool, but what's REALLY cool is that you can use this with nested
directories. Say for isntance we are creating a small (and poorly designed) dev
environment. Inside of each we want the directories bin, etc, lib, var (we're
just making 'em up now). Here's how you'd do that in one command
----
mkdir {dev,test,prod}/{bin,etc,lib,var}
----
That is the equivelant of mkdir dev/bin
mkdir dev/etc mkdir dev/lib mkdir dev/var mkdir test/bin mkdir test/etc
mkdir test/lib mkdir test/var mkdir prod/bin mkdir prod/etc mkdir
prod/lib mkdir prod/var
Another application for this is if you want to cat out a big list of
specific files without catting out the entire directory (I did this one
earlier this morning actually). Say you have 20 files called
*list.* (0-19) and you want to cat out numbers 1-9. Now, there are
a lot of ways to do this of course, but this is how you can do it with
brace expansion.
----
cat list.{1,2,3,4,5,6,7,8,9}
----
...or even shorter...
----
cat list.{1..9}
----
Those are the equivelant of
----
cat list.1 list.2 list.3 list.4 list.5 list.6 list.7 list.8 list.9
----
How's that for time saving.
Category:Bash
Category:Shells
Category:Linux
// vim: set syntax=asciidoc: