Automation Using Python

I previously wrote about automation using Makefiles to bundle together running multiple scripts into a single command. Using the subprocess library we can perform the same tasks entirely in Python.

Why Python?

Make is a quick and easy to use tool for automation, and may be a good option. But Python offers certain advantages:

  • Make isn't necessarily installed on Windows machines, but if the scripts you are automating are in Python it may be easier to automate with Python.
  • Likewise, if you are already working in Python there is not new Make syntax to learn.

Writing a Makefile Equivalent in Python

To start, our entire Python equivalent of a Makefile is below:

import sys
import subprocess

cwds = ['../CommercialData/', '../MarketView/', '../FleetView/']

def help():
    print('\nMakefile to update websites on MAG GitHub pages                                   ')
    print('                                                                                    ')
    print('Usage:                                                                              ')
    print('   python html                                                              ')
    print('   <verify you like the changes that were made>                                     ')
    print('   python publish                                                           ')
    print('                                                                                    ')
    print('Commands:                                                                           ')
    print('   python data                MILE_file_copy and update_utilization         ')
    print('   python html                make data, then regenerate the websites       ')
    print('   python html-only           regenerate the websites (no data update)      ')
    print('   python publish             publish regenerated websites to GitHub      \n')

def data():'python', cwd='../MILE_Data/', shell=True)'python', cwd='../MILE_Data/', shell=True)'python', cwd='../MILE_Data/', shell=True)

def html():
    for cwd in cwds:'python', cwd=cwd, shell=True)

def publish():
    for cwd in cwds:
        print('\n')'git pull origin master', cwd=cwd, shell=True)'git add -A', cwd=cwd, shell=True)'git commit -m "auto-regenerate from python"', cwd=cwd, shell=True)'git push origin master', cwd=cwd, shell=True)

if __name__=='__main__':

    if sys.argv[1]=='help':
    elif sys.argv[1]=='data':
    elif sys.argv[1]=='html':
    elif sys.argv[1]=='html-only':
    elif sys.argv[1]=='publish':

We need the sys library to parse arguments for the commands, and the subprocess library to execute the commands. We want to use this by calling python <command>, so we structure a function for each of the primary commands of help, data, html, and publish. Each command is assigned the appropriate function(s) to perform the required operations.

The functions themselves then use the subprocess library to issue the needed shell commands. If these commands are to be performed in another directory, it is easy to specify the correct directory with the cwd argument.

Executing the File

Back to our example, the previous process required us to:

  • cd into each repo
  • run python update files
  • commit the changes
  • repeat for each

Using this automated file, the process now is simply:

  • cd in to the directory containing Makefile
  • python publish


Whenever you find yourself doing the same rote task often it may be a good time to ask yourself if some simple automation is appropriate. For anything you perform at the command line, a simple file like this Makefiles can be a lightweight automation solution.

© 2005 Matthew Kudija | Source