Python with Multiple Threads

I have a need to have a script to execute the same task, among many devices, as close to the same time as possible. As a non-programmer, whom happens to write code in an effort to make my job easier, I thought the task would be easier than it actually is. Spawning multiple threads is pretty easy. However, hitting resource limits is a limiting factor - as is how you output your data.

Here is an example of how I’m using BoundedSemaphores, within Python to limit the number of threads that I spawn.

#!/usr/bin/env python

from threading import Thread
from threading import BoundedSemaphore
from os import system
import time

threads = []
max_threads = 1
sema = BoundedSemaphore(value=max_threads)
hosts = ['host1', 'host2', 'host3', 'host4', 'host5', 'host6']

def ping_pong(host, sema):
	ping_pong_host = system('ping -c 100 -t 1 -m 1 %s > /dev/null 2>&1' % host)
	sema.release()
	
	return ping_pong_host
	
if __name__ == '__main__':
	print "Start: %s" % time.time()
	print "Max Threads: %s" % max_threads
	tr = 1
	for host in hosts:
		print "Thread: %s - %s" % (tr, time.time())
		sema.acquire()
		t = Thread(target=ping_pong, args=[host, sema])
		t.start()
		threads.append(t)
		tr += 1
	for t in threads:
		t.join()
	print "End: %s" % time.time()

With the ‘max_threads’ variable set to ‘1’, here is the output:

Start: 1416558663.05
Max Threads: 1
Thread: 1 - 1416558663.05
Thread: 2 - 1416558663.05
Thread: 3 - 1416558664.07
Thread: 4 - 1416558665.08
Thread: 5 - 1416558666.1
Thread: 6 - 1416558667.12
End: 1416558669.15

With the ‘max_threads’ variable set to ‘6’, here is the output:

Start: 1416558829.13
Max Threads: 6
Thread: 1 - 1416558829.13
Thread: 2 - 1416558829.13
Thread: 3 - 1416558829.13
Thread: 4 - 1416558829.13
Thread: 5 - 1416558829.13
Thread: 6 - 1416558829.13
End: 1416558835.23

With the max_threads set to one, it essentially disable’s threading - in that it doesn’t spawn multiple threads to complete a task. You can see that it took approximately five seconds to spawn all threads and another two seconds to complete the tasks.

With the max_threads set to six, it spawned all threads in less than a millisecond and took approximately six seconds to complete the tasks.