Executing shell commands in Groovy
Posted in groovy, java, tools on September 30th, 2010 by Joerg – 10 CommentsGroovy is a good alternative to classic shell scripting for the Java experienced developer. I already wrote about this here. A common scenario is to call other shell commands from your scripts. Groovy makes this easy, but during a recent project I learned about some pitfalls. I will explain them here and also how to avoid them.
The simple way
Executing a simple command is just a matter of putting an .execute() after a string like that:
"mkdir foo".execute()
(Please ignore the fact for now, that you could do this using the Java standard functionality. This would be more effective in this example. Believe me, there are more complex requirements where you definitely need to call a shell command. I just keep the examples simple.)
Evaluating the exit value
What if you want to know whether this action was successful? In a shell script you usually check the exit value. Thats still straightforward:
def exitValue = "mkdir foo".execute().exitValue() if(!exitValue) //do your error-handling
“execute()” just creates a java.lang.Process object. This simply returns an int with the exit value when calling exitValue() which you can use in your script.
Printing the output
Let’s say you want to see the result of the shell command during the runtime of your script. The easy approach would be:
println "ls".execute().text
If you want to combine text output and evaluation of the exit value it starts getting less elegant:
def process = "ls".execute()
process.text.eachLine {println it}
if(!process.exitValue())
...
This works, but not very well. It has two major issues:
- The output will not happen until the shell command is finished. This is ok for a simple “ls” but will get ugly, when you have a long running command.
- process.text contains only standard out by default, but not err out.
To solve this you can’t use the simple “.execute()” anymore. You need to fall back to java.lang.ProcessBuilder. This allows you to “redirectErrorStream()” which joins both streams together as it would look like when executing the command on a shell. Here is the example code:
def process=new ProcessBuilder("ls").redirectErrorStream(true).start()
process.inputStream.eachLine {println it}
The working directory
Another pitfall is the current working directory. In a simple shell script you would just “cd” into the new working directory and then continue with your script. This would not work in a groovy script. As groovy is using the underlying Java mechanisms the “cd” command has no effect to the following instances of Process. Instead you need to tell the ProcessBuilder where your working directory has to be. There are several ways to do this. The first would be to add a “directory(File)” call to the ProcessBuilder from the example above. This would look like that:
def processBuilder=new ProcessBuilder("ls")
processBuilder.redirectErrorStream(true)
processBuilder.directory(new File("Your Working dir")) // <--
def process = processBuilder.start()
...
Fortunately there is a more convenient way when you use the execute method. You can use an overloaded version with two parameters. The first is a String array or a List containing environment variables or null if you don’t need them. The second is the working directory in form of a File object. Here is a code example:
"your command".execute(null, new File("your working dir"))
As a side note, if you need to set specific environment variables you also need to set them for each call.
Using wildcard characters
This pitfall is described in many Groovy books but this post would not be complete without it. Using Wildcards like in the following example would not work.
"ls *.java".execute()
Instead of listing all Java source files in the current directory as you might expect, it will try to list a file called “*.java”. It is very unlikely that this file is found. Wildcards like * are interpreted by the shell. The execute command is not using one. You need to call the shell first and let it interpret the wildcard-characters. On OSX and Linux this is simply done by adding a “sh -c” in front of the command. Something slightly different applies on Windows which I leave you to look up. The following command will do the expected.
"sh -c ls *.java".execute()
It doesn’t do any harm to add this prefix to all shell commands you like to start. You only need to take care of the operating system your script is running on. I am in the fortunate position to expect a UNIX-like shell on every computer in my project so I simply add “sh -c”.
Putting it all together
Because of all the issues mentioned above I ended up adding two methods to my Groovy scripts called executeOnShell. These methods solves the pitfalls mentioned and still result in a concise syntax when calling shell commands from the script.
def executeOnShell(String command) {
return executeOnShell(command, new File(System.properties.'user.dir'))
}
private def executeOnShell(String command, File workingDir) {
println command
def process = new ProcessBuilder(addShellPrefix(command))
.directory(workingDir)
.redirectErrorStream(true)
.start()
process.inputStream.eachLine {println it}
process.waitFor();
return process.exitValue()
}
private def addShellPrefix(String command) {
commandArray = new String[3]
commandArray[0] = "sh"
commandArray[1] = "-c"
commandArray[2] = command
return commandArray
}
The method executeOnShell is overloaded, so that you can either call it with a working directory or without. It also returns the exitValue, so you can easily evaluate it. This is of course not complete, as it does not include the ability to set Environment variables and it works only on a Unix-like shell. It is sufficient for my current use cases.
Please feel free to use it and to add your own requirements. If you know of any other pitfalls when executing shell commands from Groovy please leave a comment.
