Working with Python

Command-line Arguments

I use Python as my go-to tool for command-line scripts, these often requiring passing command-line arguments. Since I use various programming languages I don't remember anything, so I create reference docs for myself and hopefully others.

What Library to Use

I don't know the history, but there are a couple standard libraries you can use to parse command-line arguments. The one you want to use is argparse module. Similar in name to optparse but that is the older deprecated module.

Also confusingly there is a getopt module that handles parsing of command-line arguments but is more complicated and requires writing more code.

Just use argparse module, it works great for both Python2 and Python3.

Basic Example

First, you may not need a module. If all you want to do is grab a single argument and no flags or other parameters passed in, you can just use sys.argv array that contains all of the command-line parameters.

The first element in sys.argv is the script itself. So a parameter passed in will be in the second element: sys.argv[1]

import sys
if len(sys.argv) > 1:
    print( "~ Script: " + sys.argv[0] )
    print( "~ Arg : " + sys.argv[1] )
else:
    print(" No arguments ")

Saving as test.py and running gives:

$ python test.py Foo
~ Script: test.py
~ Arg   : Foo

Multiple Arguments with sys.argv

Since sys.argv is simply a list, you can grab blocks of arguments together or slice around as you would any other list.

Last argument: sys.argv[-1]

All args after first: " ".join(sys.argv[2:])

Flag Parameters

You need to start using a module when you want to start including flags such as --help or want to have optional arguments, or varying length parameters. As mentioned, the best standard module to use is argparse.

Help and Verbose Examples

import argparse
 
parser = argparse.ArgumentParser(description='Demo')
parser.add_argument('--verbose',
    action='store_true',
    help='verbose flag' )
 
args = parser.parse_args()
 
if args.verbose:
    print("~ Verbose!")
else:
    print("~ Not so verbose")

Here's how to run the above example:

$ python test.py
~ Not so verbose
 
$ python test.py --verbose
~ Verbose!

The action parameter tells argparse to store true if the flag is found, otherwise it stores false. Also a great thing about using argparse is you get built-in help. You can try it out by passing in an unknown parameter, -h or --help

$ python test.py --help
usage: test.py [-h] [--verbose]
 
Demo
 
optional arguments:
  -h, --help  show this help message and exit
    --verbose   verbose output

Extended help

If you want to add to the automatically created help use the epilog parameter when creating the object.

parser = argparse.ArgumentParser(
    description='Demo',
    epilog="My extended help text"
)

By default, regardless of your formatting argparse will strip all whitespace in epilog and display as a single long string. If you want it to keep the whitespace, use the RawDescriptionHelpFormatter like so:

parser = argparse.ArgumentParser(
    description='Demo',
    formatter_class=argparse.RawDescriptionHelpFormatter,
    epilog="""
My longer text
    includes fancy
             whitespace formatting
    """
    )

Parameter error checking

A side effect of using argparse, you will get an error if a user passes in a command-line argument not expected, this includes flags or just an extra argument.

$ python test.py filename
usage: test.py [-h] [--verbose]
test.py: error: unrecognized arguments: filename

Multiple, Short or Long Flags

You can specify multiple flags for one argument, typically this is down with short and long flags, such as --verbose and -v

import argparse
 
parser = argparse.ArgumentParser()
parser.add_argument('--verbose', '-v',
    action='store_true',
    help='verbose flag' )
 
args = parser.parse_args()
 
if args.verbose:
    print("~ Verbose!")
else:
    print("~ Not so verbose")

Required Flags

You can make a flag required by setting, required=True this will cause an error if the flag is not specified.

parser = argparse.ArgumentParser()
parser.add_argument('--limit', required=True, type=int)
args = parser.parse_args()

Positional Arguments

The examples so far have been about flags, parameters starting with --, argparse also handles the positional args which are just specified without the flag. Here's an example to illustrate.

parser = argparse.ArgumentParser()
parser.add_argument('filename')
args = parser.parse_args()
 
print("~ Filename: {}".format(args.filename))

Output:

$ python test.py filename.txt
~ Filename: filename.txt

Number of Arguments

Argparse determines the number of arguments based on the action specified, for our verbose example, the store_true action takes no argument. By default, argparse will look for a single argument, shown above in the filename example.

If you want your parameters to accept a list of items you can specify nargs=n for how many arguments to accept. Note, if you set nargs=1, it will return as a list not a single value.

import argparse
 
parser = argparse.ArgumentParser()
parser.add_argument('nums', nargs=2)
args = parser.parse_args()
 
print("~ Nums: {}".format(args.nums))

Output:

$ python test.py 5 2
~ Nums: ['5', '2']

Variable Number of Parameters

The nargs argument accepts a couple of extra special parameters. If you want the argument to accept all of the parameters, you can use * which will return all parameters if present, or empty list if none.

parser = argparse.ArgumentParser()
parser.add_argument('nums', nargs='*')
args = parser.parse_args()
 
print("~ Nums: {}".format(args.nums))

Output:

$ python test.py 5 2 4
~ Nums: ['5', '2', '4']

If you want to require, 1 or more parameters, use nargs='+'

Positional arguments are determined by the position specified. This can be combined with the nargs='*' for example if you want to define a filename and a list of values to store.

parser = argparse.ArgumentParser()
parser.add_argument('filename')
parser.add_argument('nums', nargs='*')
args = parser.parse_args()
 
print("~ Filename: {}".format(args.filename))
print("~ Nums: {}".format(args.nums))

Output:

$ python test.py file.txt 5 2 4
~ Fileanme: file.txt
~ Nums: ['5', '2', '4']

You can also specify nargs='?' if you want to make a positional argument optional, but you need to be careful how you combine ? and * parameters, especially if you put an optional positional parameter before another one.

This makes sense, not requiring the last args:

parser = argparse.ArgumentParser()
parser.add_argument('filename')
parser.add_argument('nums', nargs='?')
args = parser.parse_args()

Output:

$ python test.py test.txt 3
~ Filename: test.txt
~ Nums: 3
 
$ python test.py test.txt
~ Filename: test.txt
~ Nums: None

However, using the nargs='?' first will give unexpected results when arguments are missing, for example:

parser = argparse.ArgumentParser()
parser.add_argument('filename', nargs='?')
parser.add_argument('nums', nargs='*')
args = parser.parse_args()

Output:

$ python test.py 3 2 1
~ Filename: 3
~ Nums: ['2', '1']

You can use nargs with flag arguments as well.

parser = argparse.ArgumentParser()
parser.add_argument('--geo', nargs=2)
parser.add_argument('--pos', nargs=2)
parser.add_argument('type')
args = parser.parse_args()

Output:

$ python test.py --geo 5 10 --pos 100 50 square
~ Geo: ['5', '10']
~ Pos: ['100', '50']
~ Type: square

Variable Type

You might notice that the parameters passed in are being treated like strings and not numbers, you can specify the variable type by specifying type=int. By specifying the type, argparse will also fail if an invalid type is passed in.

parser = argparse.ArgumentParser()
parser.add_argument('nums', nargs=2, type=int)
args = parser.parse_args()
 
print("~ Nums: {}".format(args.nums))

Output:

$ python test.py 5 2
~ Nums: [5, 2]

File Types

Argparse has built-in filetypes that make it easier to open files specified on the command line. Here's an example of reading a file, you can do the same writing a file.

parser = argparse.ArgumentParser()
parser.add_argument('f', type=argparse.FileType('r'))
args = parser.parse_args()
 
for line in args.f:
    print( line.strip() )

Default Value

You may specify a default value if the user does not pass one in. Here's an example using a flag.

parser = argparse.ArgumentParser()
parser.add_argument('--limit', default=5, type=int)
args = parser.parse_args()
 
print("~ Limit: {}".format(args.limit))

Output:

$ python test.py
~ Limit: 5

Remainder

If you want to gather the extra arguments passed in, you can use remainder which gathers up all arguments not specified into a list.

import argparse
 
parser = argparse.ArgumentParser()
parser.add_argument('--verbose',
    action='store_true',
    help='verbose flag' )
parser.add_argument('args', nargs=argparse.REMAINDER)
args = parser.parse_args()
 
print(args.args)

Specifying remainder will create a list of all remaining arguments:

$ python test.py --verbose foo bar
['foo', 'bar']

Actions

The default action is to assign the variable specified, but there are a couple of other actions that can be specified.

Booleans

We have already seen the boolean flag action which is action='store_true' which also has a counter action for action='store_false'

Count

You can use the count action, which will return how many times a flag was called, this can be useful for verbosity or silent flags.

parser = argparse.ArgumentParser()
parser.add_argument('--verbose', '-v', action='count')
args = parser.parse_args()
 
print("~ Verbose: {}".format(args.verbose))

Output:

$ python test.py
~ Verbose: None
 
$ python test.py --verbose
~ Verbose: 1
 
$ python test.py --verbose -v --verbose
~ Verbose: 3

Append

You can also use the append action to create a list if multiple flags are passed in.

parser = argparse.ArgumentParser()
parser.add_argument('-c', action='append')
args = parser.parse_args()
 
print("~ C: {}".format(args.c))

Output:

$ python test.py
~ C: None
 
$ python test.py -c hi
~ C: ['hi']
 
$ python test.py -c hi -c hello -c hey
~ C: ['hi', 'hello', 'hey']

Choices

If you only want a set of allowed values to be used, you can set the choices list, which will display an error if invalid entry.

parser = argparse.ArgumentParser(prog='roshambo.py')
parser.add_argument('throw', choices=['rock', 'paper', 'scissors'])
args = parser.parse_args()
 
print("~ Throw: {}".format(args.throw))

Examples

I'll end with two complete examples; many of the examples above are not as complete, they were kept short to focus on the idea being illustrated.

Copy Script Example

import argparse
import sys
 
parser = argparse.ArgumentParser(description='script to copy one file to another')
 
parser.add_argument('-v', '--verbose',
    action="store_true",
    help="verbose output" )
 
parser.add_argument('-R',
    action="store_false",
    help="Copy all files and directories recursively")
 
parser.add_argument('infile',
    type=argparse.FileType('r'),
    help="file to be copied")
 
parser.add_argument('outfile',
    type=argparse.FileType('w'),
    help="file to be created")
 
args = parser.parse_args()

Bug Script Example

Here is an example of a script that closes a bug

import argparse
import sys
 
parser = argparse.ArgumentParser(description='close bug')
 
parser.add_argument('-v', '--verbose',
    action="store_true",
    help="verbose output" )
 
parser.add_argument('-s',
    default="closed",
    choices=['closed', 'wontfix', 'notabug'],
    help="bug status")
 
parser.add_argument('bugnum',
    type=int,
    help="Bug number to be closed")
 
parser.add_argument('message',
    nargs='*',
    help="optional message")
 
args = parser.parse_args()
 
print("~ Bug Num: {}".format(args.bugnum))
print("~ Verbose: {}".format(args.verbose))
print("~ Status : {}".format(arg.s))
print("~ Message: {}".format(" ".join(args.message)))

Resources

• Official Argparse Documentation - the official documentation includes all available options and an example tutorial using argparse.