Grep for System Admins: Using Grep to Automate Daily Tasks
Photo by Lucas Sankey on Unsplash
If you work with computers as a programmer or system administrator, you probably spend a lot of time staring into the command-line interface! And if you’re used to the command line, you have probably come across the
So what exactly is grep? And how do I use it, and use it better? In this post, we’ll cover some of the most useful grep options and techniques to help you look for stuff more efficiently!
Grep stands for “Global Regular Expression Print”, and is a command-line utility included by default in Unix-based operating systems. You can use it to perform searches in text, files, and command outputs. A simple grep command looks like this:
grep abc file.txt
This tells grep to search for the string “abc” in the file “file.txt”, then print the matching lines in standard output.
You can also use grep to filter the output of other Unix utilities via command-line piping:
who | grep vickie
This will output all the lines of the output from the
who command (which displays who is logged in on the machine) with the string “vickie” in it.
curl www.google.com | grep "href="
And this will find all the links embedded in the Google homepage.
Grepping For More
But grep is much more than a simple string searching tool! With its advanced options, grep is a powerful tool for automating and simplifying tasks. Here are a few of the most useful options that grep offers.
Adding Context to Your Search
What if you want to see the context of each search, instead of just the matching line itself? Grep has a few surround search options for this purpose:
-A2 Prints two lines of trailing context after each match.
-B2 Prints two lines of leading context before each match.
-C2 Prints two lines of leading and trailing context surrounding each match. This is equivalent to
Searching From Multiple Inputs
What if you are not sure where the information is among a few files, or if you want to find all matches across several files? You can simply supply multiple input files to grep like so:
grep abc file1.txt file2.txt file3.txt
You can even specify filenames using the wildcard character. This searches in all
grep abc file*.txt
Grep will show the matching lines in each file along with the filenames. If you want to just see the filenames that have matches, you can use the
— files-with-matches flag.
grep abc file*.txt --files-with-matches
You can also search entire directories at once using the
-r (recursive) flag:
grep abc -r directory1
You can also utilize the inverse search functionality (using the
-v flag), which will make grep print lines that don’t match the provided string.
grep abc file1.txt -v
This can be useful when you want to rule out certain IP addresses and application names when doing log analysis, for example.
An extremely useful option is the ability to make grep ignore letter cases with the flag
-i (ignore). For example, the following command will match with both “abc” and “Abc”.
grep abc file1.txt -i
Count the Matches!
If you supply grep with the
-c flag, it will only output the number of lines that matched, instead of the content of the lines.
grep abc file1.txt -c
Grep with regex
Finally, you can make your search more flexible by using regular expressions in your search string. You can accomplish this by using the
grep -E "^abc" file1.txt
For example, this command will only search for lines that start with the string “abc”.
Grep + Linux
But your grep journey does not have to end there! When combined with a few other Linux utilities, grep can become one of the most useful and flexible tools in your arsenal.
Lightweight Text Editing Using grep
If you use grep in conjunction with a few other command-line utilities, you can actually use it as a lightweight text processing tool!
For example, you can use grep to remove the empty lines of a file. This command will find all the non-empty lines in
logfile.txt and rewrite all the non-empty lines back to
grep -E "^$" -v logfile.txt > logfile.txt
You can also use grep to fill entirely new documents with data. For example, you can use this command to search for the subdomains of google.com embedded on Google’s homepage, and paste all the URLs to
-o flag tells grep to print only the matching text, not the entire matching line.)
curl www.google.com | grep -Eo "http?.*google\.com" > subdomains.txt
You can even filter out certain subdomains by chaining grep:
curl www.google.com | grep -Eo "http?.*google\.com" | grep -Eov "http?.*www.\google\.com" > subdomains.txt
Monitor the System With Grep
You can also use grep to process the output of system-monitoring utilities. For example, you can use grep to filter out the process statuses you want to examine:
ps aux | grep mysql
Similarly, you can use grep to quickly read specific pieces of information from the output of commands like
Make Your Life Easier With grep
There are many, many more ways grep can make your life easier. For example, when you are using
nslookup, you can quickly navigate to the lines you want using grep:
dig google.com | grep "\sA\s" nslookup google.com | grep "Address"
Or, use grep to quickly find that tar command that you can’t remember the flags for!
history | grep tar
Recent and Relevant
You can also use grep in conjunction with other filtering utilities like
tail to find information that is both recent and relevant.
For example, to find the most recent times
sudo was run in a particular log file, you can run:
tail logfile.txt | grep sudo
If you enjoyed this article, please tweet us and let us know what you think! Or, if you’d like to see some of our videos, you can subscribe to our YouTube channel.
Anddddd, be sure to check out some of our other great devops-esque posts!
Okta Developer Blog Comment Policy
We welcome relevant and respectful comments. Off-topic comments may be removed.