grep sed awk

Linux Basics: Working with Text Files Using grep, sed, and awk

Introduction

In our journey through the world of Linux, we’ve encountered various tools and techniques to manage and manipulate our files. Today, we’re delving into the fascinating realm of text processing. Three of the most potent tools in a Linux user’s arsenal are grep, sed, and awk. These command-line utilities, born in the era of Unix, are ideal for searching, editing, and manipulating text data.

Understanding grep

What is grep?

grep is a utility used for searching plain-text data sets for lines that match a regular expression. It’s an invaluable tool for finding patterns and filtering out information from files and command outputs.

Using grep

The basic syntax of grep is:

grep [options] pattern [file…]

For instance, to search for the word “example” in a file named file.txt, you would use:

grep ‘example’ file.txt

This will display all lines in file.txt that contain the word “example”.

More Examples of grep

1. Searching recursively in all files in a directory:

The -r or -R option can be used with grep to search for a pattern recursively in the current directory and its subdirectories:

grep -r 'example' 

2. Case-insensitive search:

By default, grep is case-sensitive. Use the -i option for case-insensitive search:

grep -i 'example' file.txt
grep -i

For this test as you can see below when using the grep command in our example it’s important to note that it is case sensitive as almost everything in Linux is so using the -i option above is good depending on your search.

grep ‘purevoltage’ greptest.txt failed to output anything. However when using proper capitalization for PureVoltage it shows up from this file.

Editing the file and putting in more lines with PureVoltage in it shows up more details.

grep 'example'

3. Displaying line numbers with search results:

The -n option displays the line numbers with search results:

grep -n 'example' file.txt
grep -n

This example can be quite useful when you need to know the exact line a word or command is on. This is extra helpful when working or calling on specific scripts or documents.

Exploring sed

What is sed?

sed (stream editor) is a tool used to perform basic text transformations on an input stream (a file or input from a pipeline).

Using sed

The basic syntax of sed is:

sed options… [script] [input-file…]

For example, to replace all instances of the word “apple” with “banana” in file.txt, you would use:

sed 's/apple/banana/g' file.txt

In this example below you will see we replace purevoltage with PureVoltage this can come in handy if you need to make multiple adjustments to a script or document.

If you have migrated a customer with a script that uses an IP address of their old provider you can quickly edit and adjust their old IP address to your new servers IP address from PureVoltage.

Note that this will output the modified content to the standard output (usually your terminal) but it won’t modify the original file. If you want to save the changes back to the file, you can use the -i option (in-place):

sed -i ‘s/apple/banana/g’ file.txt

With this command, every instance of “apple” in file.txt will be replaced with “banana” and the changes will be saved directly back to file.txt.

More Examples of sed

1. Deleting lines in a file:

You can use sed to delete lines in a file. For example, to delete the 3rd line in file.txt:

sed '3d' file.txt

2. Deleting all blank lines in a file:

sed '/^$/d' file.txt

3. Replacing only on a specific line:

To replace only on the 2nd line:

sed ‘2s/apple/banana/’ file.txt

Unveiling awk

What is awk?

awk is a versatile programming language designed for pattern scanning and processing language. It’s highly useful for data extraction and reporting.

Using awk

The basic syntax of awk is:

awk ‘pattern {action}’ file.txt

For example, to print the third column of a text file, you can use:

awk ‘{print $3}’ file.txt

This will print the third column of each line in file.txt.

More Examples of awk

1. Printing specific lines:

To print only the lines where the third column is equal to “example”:

awk ‘3==”example” {print0}’ file.txt

As you can see from this example as PureVoltage is the third word it shows up for this example.

2. Summing up the values of a column:

If the file has numerical values, you can use awk to sum them up. For instance, to sum up the values of the third column:

awk ‘{sum+=$3} END {print sum}’ file.txt

3. Counting lines in a file:

awk ‘END {print NR}’ file.txt

These are just a few of the many things you can do with grep, sed, and awk. As you spend more time using them, you’ll discover that they are incredibly powerful tools for text processing in Linux.

Conclusion

With the power of grep, sed, and awk, you can efficiently process text data in Linux. They are especially useful for scripting, log file management, and data analysis. As always, this is just the tip of the iceberg when it comes to the capabilities of these tools. We encourage you to explore their man pages and online resources to deepen your understanding.


Thank you for joining us in this edition of the Linux Basics series. Interested in exploring more topics? Check out the full series.


Posted

in

,

by

Tags: