CodeBlanks Logo Icon CodeBlanks
Login Register

Summary - Multiprocessing

Combine Process, Pool and Queue to write a parallel worker.

Recap: Multiprocessing

In this group you learned process-based parallelism. You started new processes with Process and managed their lifecycle with start and join.

You passed data between processes with Queue. You did parallel mapping with Pool. You used a shared list with Manager. Now you will combine these into a parallel worker: a Process, a Queue and a computation together.

Practice Write a parallel worker

Import with from multiprocessing import Process, Queue. Define a worker(q, val) function that does q.put(val * 2). Create a queue with q = Queue(). Create a process with Process(target=worker, args=(q, 21)), call .start() and .join(). Print q.get().

Starter code

main.py
# write the worker function and queue
# start a process and get the result

Sample solution

main.py
from multiprocessing import Process, Queue

def worker(q, val):
    q.put(val * 2)

q = Queue()
p = Process(target=worker, args=(q, 21))
p.start()
p.join()
print(q.get())

Expected output

42

What you learned

In this group you learned process-based parallelism in Python. You saw new processes with Process and lifecycle management with start and join.

You used Queue for inter-process data, Pool for parallel mapping and Manager for shared objects. In the summary you combined these into a parallel worker. This knowledge is critical for speeding up CPU-bound work.

Questions and answers

Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.

1 Which tool for which job?

  • Process for parallel mapping
  • Queue for inter-process data Correct answer
  • Manager for function calls
  • Pool to start a single process

Explanation: answer: Queue for inter-process data. Pool does parallel mapping, Process starts a single process.

2 Match all multiprocessing concepts

Term Meaning
Process New process
Queue Data queue
Pool Process pool
Manager Shared object

3 Final blank

Question 
from multiprocessing import Process, ____
q = Queue()
Answer 
from multiprocessing import Process, Queue
q = Queue()

Explanation: Queue is the inter-process queue.

All lessons in Expert

1 Metaprogramming

  1. type()
  2. Metaclass
  3. __init_subclass__
  4. __class_getitem__
  5. Decorate class
  6. Summary - Metaprogramming

2 Descriptors

  1. Descriptor protocol
  2. Data descriptor
  3. Non-data descriptor
  4. @property
  5. __set_name__
  6. Summary - Descriptors

3 AST

  1. ast.parse
  2. ast.walk
  3. NodeVisitor
  4. NodeTransformer
  5. compile
  6. Summary - AST

4 ctypes and CFFI

  1. ctypes.CDLL
  2. ctypes types
  3. argtypes and restype
  4. cffi
  5. Structure
  6. Summary - ctypes and CFFI

5 Multiprocessing

  1. Process
  2. start and join
  3. Queue
  4. Pool
  5. Manager
  6. Summary - Multiprocessing

6 Threading and asyncio

  1. threading.Thread
  2. Lock
  3. GIL
  4. concurrent.futures
  5. asyncio vs threading
  6. Summary - Threading and asyncio

7 Performance

  1. timeit
  2. cProfile
  3. sys.getsizeof
  4. slots and tuple
  5. numpy vectorization
  6. Summary - Performance

8 Expert final

  1. Project design
  2. Metaprogramming
  3. Multiprocessing
  4. ctypes
  5. Putting it together