You can use shell commands in a shell script. This makes it possible to create powerful scripts that are very short, and you can use a lot of built-in commands instead of having to make your own stuff. And if you install additional command line programs, you can run them from shell scripts too.
A shell script is not a suitable replacement for a general-purpose programming language. Instead, it can be useful for automating tasks (especially system-related ones), writing short and simple programs, and manipulating command line programs. You can even make a shell script run at a certain time, such as with cron or systemd in Linux. Some examples of shell scripts could include ones for renaming photos in a directory, updating software, running a clamav scan, checking network connectivity, or running a program with certain parameters at set intervals. There are plenty of things you can do in shell scripts that you can also do in something like Python, but using shell commands is only possible in Python through the subprocess module and isn’t its primary purpose. Think of a shell script as gluing multiple CLI things together.
There are some definite limitations of shell scripts, such as no multidimensional arrays or classes/objects. But it’s not intended for advanced programming stuff though.
You might think all the software we use is new these days, but bash was originally made in 1989. It’s still widely used. It’s older than I am!
Shell scripts require a shebang (such as #!/bin/bash or #!/usr/bin/env bash) and they need to have the executable bit set (such as chmod +x somescript.sh). Many people give them .sh file extensions, but it’s not necessary, just as long as it has the shebang at the first line in the script. However, for the sake of organization, .sh makes it easier for you to remember that it’s a shell script, even if your computer can tell by looking at the shebang. /bin/sh links to the system shell, so it’s not advised to use, as it might have varying shells. /bin/bash means use bash in the /bin directory, which is the directory in Unix for binaries (compiled executables). #!/usr/bin/env bash means to use the environment variable for where bash is, wherever it may be. Another shell some people use is zsh, but it’s not standard (except on newer Macs), so it’s best to stick with bash for the most part.
While I was writing this book, Apple announced that they would be switching macOS from bash to zsh as the default shell. So, if you’re using macOS, you might as well learn zsh (and an additional framework for it called oh-my-zsh). Think of zsh as like bash but with more bells and whistles. But keep in mind that there’s a lot of overlap between zsh and bash, and more people use bash. So even if you have your preferred shell, bash is going to come up a lot, especially for server things. I know someone who prefers Dvorak keyboards over QWERTY, but not very many people use Dvorak, so that’s a disadvantage, even if he says there are benefits based on the layout. Shells (and technology in general) are the same way.
Your best bet for Windows is Git Bash with MinGW and possibly ConEmu. You can also use PowerShell on Windows, but I’m honestly not a huge fan of it. Your best shell for Linux is bash.
If you’re getting into information security and pen testing, when you “pop a box” (slang for hacking a computer or server), you’ll most likely be using a reverse bash shell. A regular shell is a shell you connect to, whether local or remote. But based on how firewall rules work, hackers use reverse shells, which are shells that connect to them rather than the other way around. This is because unsolicited ingress (inbound) connections are usually not allowed, but there are more lax firewall ACLs (Access Control Lists) for egress (outbound) traffic. You need to set up a netcat listener in order to listen for a reverse shell before you spawn it. You might get a reverse shell after uploading a web shell because of a file inclusion or file upload vulnerability on a site, or perhaps even with shell_exec() or a remote code execution CVE.
Long story short, bash is important for many different things, whether it’s for legitimate development or system administration, or even hacking.
In addition to all the previous shell commands, which you can use in various places in a shell script, you can use the following stuff in a shell script:
Variables – a variable in bash doesn’t require a specified data type. Declaration is quite simple:
someVar=”hi”
Please note that there can’t be any spaces.
Also, you can’t reference it without a $, like so:
echo $someVar
Command line arguments are special variables in bash. They accessible using $1, $2, $3, and so on.
You can get the output of a command line program in bash by using $() with the command inside it, such as $(uptime)
coolThing=$(uptime)
echo $coolThing
If you want to use a variable in a string, such as when echoing something, use something like this:
echo “the value of myCoolVariable is ${myCoolVariable}”
Comments – like other languages, you can use comments in a shell script, like so:
#this is a comment in bash
Numeric comparison operators – there are many ways to compare two numbers. Some comparison operators to use inside of square brackets include the following:
-ge greater than or equal to
-gt greater than
-le less than or equal to
-lt less than
-eq equal
-ne not equal
If/else – to perform if/then logic in bash, you need to use [ ] and spaces around conditions. Unlike other languages, bash uses if/then/elif/else/fi. then and fi are not common in other languages. However, just think of them as a substitute for opening and closing curly braces, which are more common in other programming languages.
Booleans don’t really exist in bash (it’s complicated because control structures still rely on Boolean logic, but not Boolean data types per se), but you can use strings, like “true” and “false”.
#!/bin/bash
# testing if/else
myVar=”false”
blah=”5″
if [ $myVar == “true” ]; then
echo “it is true”
elif [ $blah -eq 5 ]; then
echo “blah is 5 and myVar is false”
else
echo “false and not 5”
fi
&& – and. You can use it in something like this:
if [[ $thingOne == “true” && thingTwo == “true” ]]; then
echo “both thingOne and thingTwo are true”
else
echo “one of them is not true”
fi
Please note that is requires either double brackets, [[ ]], or alternatively, you can do something like this:
[ $thingOne == “true” ] && [ thingTwo == “true” ]
|| – one or the other. It will be true if just one of the two conditions is true. Use it in places just like the above && example.
Arithmetic – arithmetic should happen surrounded by $[]. In the $[], you don’t need to use $ to designate variable names. Numbers should be surrounded by quotes to distinguish them from command line arguments, which are $1, $2, etc.
Here’s an example:
blah=”10″
echo $[blah + “20”]
While loop – a while loop in bash looks like this:
x=”0″
while [ $x -lt “10” ]; do
echo -n “${x} “
x=$[x+”1″]
done
echo “”
For loop – a for loop in bash looks like this:
for i in {1..10}; do
echo -n “${i} “
done
echo “”
Functions – instead of repeating code in your script, you might want to create functions. However, you will notice that they are slightly different from functions in other languages covered in this book.
You do not specify function arguments in the (). They are like command line arguments in the sense that the caller’s provided arguments are accessible with $1, $2, and so on. Keep in mind that these are different from the command line args, even if the script is run with args.
function myCoolFunction(){
echo “”
uptime
date
ping -c 4 google.com
echo “The 1st arg passed to this func is ${1}”
echo “The 2nd arg passed to this func is ${2}”
uname -a
who
}
Here is an example invocation of the above function:
myCoolFunction “hey” “how are you”
Notice how there are spaces and no parentheses. Very weird compared to other languages, but it is what it is. Notice how you can run command line commands directly from a shell script. However, if you want to run them with arguments, you should use echo $(some_command arg1 arg2) because otherwise you will have issues.
User input – to get input from a user and store it to a variable, while also giving them a prompt about what they’re entering, do this:
echo -n “Enter your name: “
read userInput
echo “Hello, ${userInput}!”
File IO (input and output) – you can read from and write to files easily in shell scripts.
For the following demonstrations, assume there is a file called somefile.txt in your home directory, and its contents are as follows:
Here
Are
Some
Cool
Lines
To read from a file, use cat. For example:
loadedFromFile=$(cat ~/somefile.txt)
To get a specific line in a file, use pipes and a combination of the head and tail commands, like so:
specificLine=$(cat ~/somefile.txt | tail -n 2 | head -n 1)
That means, get the whole file (cat), then pipe the output of cat to tail to get the last 2 lines in the file, and then pipe the output of tail to head, and get just one line from the top of that list of lines.
The end result is that it gets the second from the last line in the file.
To write to a file (and delete any of its previous contents), use >, such as echo “hello” > somefile.txt.
To append to a file, use >> instead of >.
To edit a file, you can use a combination of cat to store the contents of a file to a variable, followed by changing the string somehow, and then using > or >> to finally write to it again.
Editing is a little more complicated than just reading from or writing to a file though. You might also want to look into sed, or awk for doing pattern matching. grep is good for searching too. There is another command line editor, ed, but it’s so minimal that it’s not worth using over something more capable, like vim or even nano.
For more complicated file IO, such as binary IO instead of text IO, or when working with massive files that should be buffered for performance, you might want to consider fully-fledged programming languages instead of a shell. Trying to do really complicated programming or working with large amounts of data using a shell script is like trying to dig a ditch with a spoon.