python - Print multithread subprocess -

good day!

i have python script, creates file list , processes in multiprocess.pool.map , thread function. thread function uses outside executable , calls via subprocess.check_call. outter executable prints information stdout.

so have problem reading output - it's messed , can't useful information it. i've read printing , multithreading in python think it's not problem, because don't explicitly call print function in script.

how can solve problem? thank you.

also, have noticed if redirect output script file, output isn't messed @ all.

[update]:

this works fine if run script: python mp.py > mp.log

import time, argparse, threading, sys os import getenv multiprocessing import pool  def f(x):     cube = x*x*x     print '|lorem ipsum dolor sit amet, consectetur adipisicing elit, sed eiusmod tempor incididunt ut %d|'%(cube)     return cube  if __name__ == '__main__':      #file = open('log.txt', 'w+')     parser = argparse.argumentparser(description='cube', usage='%(prog)s [options] -n')     parser.add_argument('-n', action='store', help='number', dest='n', default='10000', metavar = '')     args = parser.parse_args()      pool = pool()      start = time.time()     result = pool.map(f, range(int(args.n)))     end = time.time()     print (end - start)     #file.close() 

to avoid mixed output multiple concurrent subprocesses, redirect output of each subprocess different file:

from multiprocessing.dummy import pool # use threads subprocess import call  def run(i):     open('log%d.txt' % i, 'wb') file:         return call(["cmd", str(i)], stdout=file)  return_codes = pool(4).map(run, range(10)) # run 10 subprocesses, 4 @ time 

or collect output , print single thread in code:

from functools import partial multiprocessing.dummy import pool, queue, process # use threads subprocess import popen, pipe  def run(i, output):     p = popen(["cmd", str(i)], stdout=pipe, bufsize=1)     line in iter(p.stdout.readline, b''):         output((p.pid, line)) # collect output      p.stdout.close()     return p.wait()  def print_output(q):     pid, line in iter(q.get, none):         print pid, line.rstrip()  q = queue() process(target=print_output, args=[q]).start() # start printing thread return_codes = pool(4).map(partial(run, output=q.put_nowait),                            range(10)) # run 10 subprocesses, 4 @ time q.put(none) # exit printing thread 

or use lock:

from __future__ import print_function multiprocessing.dummy import pool, lock # use threads subprocess import popen, pipe  def run(i, lock=lock()):     p = popen(["cmd", str(i)], stdout=pipe, bufsize=1)     line in iter(p.stdout.readline, b''):         lock:             print(p.pid, line.rstrip())     p.stdout.close()     return p.wait()  return_codes = pool(4).map(run, range(10)) # run 10 subprocesses, 4 @ time 

note: print() function used workaround issue question: why script uses threads prints lines occasionally?

to avoid mixing lines different subprocesses, collect units larger single line @ time depending on actual output is.