14. Unix commands

14.1. Introduction

In this chapter, we will implement some of the Unix commands using ‘Python’.

14.2. ‘argparse’

The ‘argparse’ module is a command line interface. We will use ‘argparse’ module to read the values from the terminal. For more details, please see the online documentation of the module.

Listing 14.1 ‘argpase’ module
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
# argparse_ex.py

import argparse

def mulNum(a):
    p = 1
    for item in a:
        p *= item
    return p

parser = argparse.ArgumentParser(description="First argparse code")

# read  list of integers
parser.add_argument(
        type=int, # type int
        nargs='+', # read multiple values in a list
        dest='integers', # name of the attribute returned by argparse
        help='read list of integer: 1 2 3')

parser.add_argument(
        '-p', '--product', # name -p or --product
        dest='multiply', # by default 'dest=product'
        action='store_const', # store values
        const=mulNum, # call function 'mulNum'
        help='product of integers'
    )

parser.add_argument(
        '--sum',
        dest='accumulate',
        action='store_const',
        const=sum, # inbuilt method 'sum' to add the values of list
        help='sum of integers'
    )

args = parser.parse_args() # save arguments in args

# if --sum is in command
if args.accumulate:
    sum = args.accumulate(args.integers)
    print("sum =", sum)

# if '-p or --product' is in command
if args.multiply != None:
# if args.multiply is not None:
    # pass arg.integers to arg.multiply, which calls function 'mulNum'
    product = args.multiply(args.integers)
    print("product = ", product)


####### Execution ####################


######## Help ############
# $ python argparse_ex.py -h  (or $ python argparse_ex.py --help)
# usage: argparse_ex.py [-h] [-p] [--sum] integers [integers ...]

# First argparse code

# positional arguments:
  # integers       read list of integer: 1 2 3

# optional arguments:
  # -h, --help     show this help message and exit
  # -p, --product  product of integers
  # --sum          sum of integers

############ Results #########
# $ python argparse_ex.py 2 5 1 --sum
# sum = 8

# $ python argparse_ex.py 2 5 1 -p
# product =  10

# $ python argparse_ex.py 2 5 1 --product
# product =  10

# $ python argparse_ex.py 2 5 1 -p --sum
# sum = 8
# product =  10

# $ python argparse_ex.py -p 2 5 1 --sum
# sum = 8
# product =  10

14.3. find

14.3.1. Command details

Create some files and folders inside a directory. Next go to the directory and run following commands. We have following files and folder in the current directory,

$ tree .
.
├── box.py
├── contributor.py
├── csv_format.csv
├── datamine.py
├── data.txt
├── expansion.py
├── expansion.txt
├── mathematician.py
├── methodEx.py
├── price2.csv
├── price.csv
├── price_missing.csv
├── price.py
├── pythonic.py
├── ring.py
├── text_format.txt
├── unix_commands
│   ├── argparse_ex.py
│   ├── cat.py
│   ├── file1.txt
│   └── file2.txt
└── wide_text_format.txt
(show all items which ends with .py)
$ find -name "*.py"
./box.py
./contributor.py
./datamine.py
./expansion.py
./mathematician.py
./methodEx.py
./price.py
./pythonic.py
./ring.py
./unix_commands/argparse_ex.py
./unix_commands/cat.py


(show all items which contains 'x' in it)
$ find -name "*x*"
./box.py
./data.txt
./expansion.py
./expansion.txt
./methodEx.py
./text_format.txt
./unix_commands
./unix_commands/argparse_ex.py
./unix_commands/file1.txt
./unix_commands/file2.txt
./wide_text_format.txt


(show all directories which contains 'x' in it)
$ find -name "*x*" -type d
./unix_commands


(show all files which contains 'x' in it)
$ find -name "*x*" -type f
./box.py
./data.txt
./expansion.py
./expansion.txt
./methodEx.py
./text_format.txt
./unix_commands/argparse_ex.py
./unix_commands/file1.txt
./unix_commands/file2.txt
./wide_text_format.txt


(for more options, see man page)
$ man find

14.3.2. Python implementation

  • First, we need to write the code, which can traverse through the directories. Following is the current directory structure,
$ tree .
.
├── argparse_ex.py
├── cat.py
├── f2
│   └── tiger.txt
├── file1.txt
├── file2.txt
├── find_ex.py
└── folder1
    └── dog.txt
  • Python code to traverse through each directory is shown below,
Listing 14.2 traverse directories using ‘pathlib’
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# find_ex.py

from pathlib import Path

# location of directory
p = Path('.') # current directory

print("All files and folders in current directory:")
all_item = [x for x in p.iterdir()]
for a in all_item:
    print(a)

# files in current folder
print("\n")
print("Files in current folder:")
files = [x for x in p.iterdir() if x.is_file()]
for f in files:
    print(f)

# directories in current folder
print("\n")
directory = [x for x in p.iterdir() if x.is_dir()]
print("Directories in current folder:")
for d in directory:
    print(d)

Below is the output for above code,

$ python find_ex.py

All files and folders in current directory:
argparse_ex.py
cat.py
f2
file1.txt
file2.txt
find_ex.py
folder1


Files in current folder:
argparse_ex.py
cat.py
file1.txt
file2.txt
find_ex.py


Directories in current folder:
f2
folder1
  • Below is code to implement the ‘find’ command,
Listing 14.3 ‘find’ command using Python
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# find_ex.py

import argparse
import sys

from pathlib import Path

parser = argparse.ArgumentParser(description="Find command")

# positional argument when defined without --
# nargs='*' or '?' is required for default-positional-arg values
# * : read all command line arguments;  ?: read one command line arg
parser.add_argument('location', type=str, nargs="*", default='.')

# optional argument when defined with --
# no 'nargs' i.e. it's str (not list of str)
parser.add_argument('--name', type=str, default="*")

# possible values are "d", "f" and "all"
parser.add_argument('--type',type=str,default="all",choices=["d", "f", "all"])

args = parser.parse_args() # save arguments in args
loc = Path(args.location[0])

items=[]
for l in loc.rglob(args.name):
    if args.type == "d" and l.is_dir():
        items.append(l)
    elif args.type == "f" and l.is_file():
        items.append(l)
    elif args.type == "all":
        items.append(l)

# print output
for i in items:
    print(i)
(show files which starts with 'f')
$ find -name "f*" -type "f"
./file1.txt
./file2.txt
./find_ex.py

$ python find_ex.py --name "f*"  --type "f"
file1.txt
file2.txt
find_ex.py



(show directories which starts with 'f')
$ python find_ex.py --name "f*" --type "d"
f2
folder1

$ python find_ex.py --name "f*" --type "d"
f2
folder1


(show everything which starts with 'f')
$ find -name "f*"
./f2
./file1.txt
./file2.txt
./find_ex.py
./folder1

$ python find_ex.py --name "f*"
f2
file1.txt
file2.txt
find_ex.py
folder1


(show all directories)
$ find -type "d"
.
./f2
./folder1

$ python find_ex.py --type "d"
f2
folder1

(read t* from different location)
$ python find_ex.py ./folder1 --name "t*"
$ python find_ex.py ./f2 --name "t*"
f2/tiger.txt
$ python find_ex.py .. --name "t*"
../text_format.txt
../unix_commands/f2/tiger.txt

14.4. grep

  • “grep” command is used to find the pattern in the file, e.g. in below code ‘Dog’ is search in the file ‘file2.txt’.
$ grep "Dog" file2.txt
Dog nudged cat.
  • Below is the implementation of ‘grep’ command using Python
# grep_ex.py

import argparse
import re

from pathlib import Path


parser = argparse.ArgumentParser(description="grep command")

# positional argument when defined without --
# nargs='?' is required for default-positional-arg values
parser.add_argument('pattern', type=str, nargs=1)
parser.add_argument('location', type=str, nargs='*')


args = parser.parse_args() # save arguments in args
loc = Path(args.location[0]) # save location
ptrn = args.pattern[0] # save pattern

lines = open(loc).readlines();

for line in lines:
    if re.compile(ptrn).search(line):
        print(line, end="")
  • The output of ‘grep’ command are shown below,
$ grep "nud" file2.txt
Dog nudged cat.
$ python grep_ex.py "nud" file2.txt
Dog nudged cat.

14.5. cat

In this section, we will implement the command ‘cat’ of unix. We will write a code to read the name of the files from the command line.

  • First create few files e.g. ‘file1.txt’ and ‘file2.txt’ etc. and add some contents to it.
  • Now read these files as shown below, which emulates the functionality of ‘unix’s cat command’,
Listing 14.4 Read files from command line
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
# cat.py

import argparse

parser = argparse.ArgumentParser(description="First argparse code")

# read arguments
parser.add_argument(
        nargs='+', # read multiple values in a list
        dest='files', # name of the attribute returned by argparse
        help='unix "cat" operation')

# line number
parser.add_argument(
        '-n', '--numbers', # default 'dest=numbers'
        action='store_true', help='print line numbers')

args = parser.parse_args() # save arguments in args
# print(f"args: {args.files}") # print argument read by 'dest=files'
# print(f"first element: {args.files[0]}") # print first element of arg.files


# open and read each files
line_no = 0
for file_name in args.files:
    with open(file_name, 'r') as w:
        if args.numbers:  # print data with line numbers
            for data in w.readlines(): # start from 1
                line_no += 1
                print("{0:5d}\t {1}".format(line_no, data), end='')
        else: # print data without line numbers
            data = w.read()
            print(data)

Below are the outputs of above code,

  • actual ‘cat’ command output

    $ cat file1.txt file2.txt -n
         1  This is cat. Cat is pet.
         2  This is cat. Cat is pet.
         [...]
        15  This is cat. Cat is pet.
        16  Dog nudged cat.
        17  Duck is sleeping.
    
  • Python code output

    $ python cat.py file1.txt file2.txt -n
         1   This is cat. Cat is pet.
         2   This is cat. Cat is pet.
         [...]
        15   This is cat. Cat is pet.
        16   Dog nudged cat.
        17   Duck is sleeping.
    

Note

  • Try both commands without ‘-n’ as well.
  • Run below command to see more functionalities of ‘cat’ command; and try to add some of those to ‘cat.py’
$ man cat