Practical bash command grouping

Page contents

Command grouping aren't subshells

The basic syntax of command grouping in bash are curly braces with commands inbetween. This can be written in two ways, either with newlines:

{
  echo "text" 
  cat file.txt 
}

or in a single line, in which case commands MUST end with semicolons:

{ echo "text"; cat file.txt; }

This latter syntax may seem familiar to you if you have seen subshells before, they look like this:

( echo "text"; cat file.txt; )

The main difference between command grouping and subshells is scope: a subshell receives a copy of the environment and variables of the current shell, which is lost when the command(s) complete. Command grouping instead stays in the context of the current shell, so it can manipulate variables of the current session that remain usable after it completes.

Grouping commands into single units

The first use of command grouping is to group commands (surprise!). This new group now acts as a single unit and can be treated as a single command, even though it consists of multiple commands internally.

A sample use case found in a fair few scripts is checking if a program is available. If it exists, everything is fine - but if it isn't, then you will likely want to display an error message and stop with exit code 1, indicating the error.

Now you could write this in a verbose form:

if ! command -v myprog &>/dev/null; then
 echo "Error: 'myprog' command not found." >&2
 exit 1
fi

But using command grouping, you can turn this into a single line of code:

command -v myprog &>/dev/null || { echo "Error: 'myprog' command not found." >&2; exit 1; }

The short-form condition syntax works here, because it executes the entire command group if the first command fails, not just the first command in it.

You can also use this trick to time an entire workflow:

time { cat input.txt | tr 'a-z' 'A-Z' | tee output1.txt > output2.txt }

The main advantage of grouping these commands together for timing is that it shows the complete time the workflow takes to complete, including reading the file input.txt and writing both output files to disk. Getting a full picture when measuring performance is often more important than benchmarking single commands or functions in isolation.

Communicating with background jobs

The bash shell is very capable of running jobs in the background, and that job can even be a command group consisting of multiple statements. Using a command group instead of a subshell here allows it to access and change the main shell's variables, to hand output back to it if needed. Yes a subshell's output could also be caught in a variable, but what if your program needs to return multiple strings, like a host address, username and password? This is where command grouping shines:

{ HOST=$(get-secret "host"); USERNAME=$(get-secret "user"); PASSWORD=$(get-secret "pass"); echo "Credentials available!"; } &

Running this job in the background will sequentially fetch all three variables, then print a success message to the main terminal that launched it. The variables remain available to the parent terminal even after the job completes, making it perfect to fetch information from potentially slow servers in the background.

Merging output

When working with log messages, administrators frequently need to check more than one log file for specific entries. This is especially true when tracing information (for example the same ip address) across multiple services (a load balancer, the web application firewall, and the actual web application). Viewing multiple files at once can be simplified with command groups:

{ fetch_load_balancer_logs; echo "===== start of waf.log ====="; cat waf.log; echo "===== start of waf.log ====="; webapp_logs; } | less

By combining them as a single stdout stream, they can all be viewed and searched from a single less viewer, reducing friction of jumping between files and outputs. Of course, if all of these are files on the local system, a single cat command could have done the same, but assuming some of those logs need to be fetched over the network makes the grouping benefit obvious.

Merging output like this can also be used to write file atomically:

{ echo "Prepended contents"; cat file.txt; echo "Appended contents"; } > new.txt && mv new.txt important.txt

Without the command group, an error in the cat command would leave new.txt with partially written contents. Using the command group ensures that all output stream data is written successfully before new.txt is renamed to important.txt.

Mixing file contents and terminal inputs

One fairly unknown use case for command grouping is to execute some scripting language commands from a file, then drop to an interactive session while maintaining the script state.

Here is an example python script:

x = 10
y = 20
#DEBUG_HERE
z = x + y
print(z)

You may notice the #DEBUG_HERE comment. We can use it as a marker to execute all the commands above it, then dropping to an interactive python interpreter session for investigation:

{ sed '/#DEBUG_HERE/q' script.py; cat /dev/tty; } | python3 -i

Normally, the python3 command would stop once it reached EOF on standard input, but since we merge the file contents with the contents of /dev/tty, which simply outputs anything typed on the terminal, the python3 command never receives an EOF, so the session remains after the file's commands are executed, allowing us to inspect the current state of variables or call functions at will.

This form of debugging is quite powerful, as it allows inspecting any point within a script file by adding a simple comment to it.

The same mechanism can also be used for other text-based languages, for example php, perl or node.js, or even mysql / postgresql (especially useful to debug session-scoped temporary tables).

More articles

Introduction to ZeroMQ networking patterns

Networking software simplified

Finding common bugs with valgrind

From memory leaks to race conditions

How attackers hide backdoors in web servers

Understand and detect post-exploitation changes