Multiprocessing
- Python course
- Expert
- Lesson 45/47
- 6 min
Write parallel processing with multiprocessing.Pool.
The parallel processing layer
In this lesson you will add parallel square computation with multiprocessing.Pool. You define a square function and apply it to all items in a list with Pool.map.
Pool.map preserves input order, so the output is deterministic. The previous lessons' Tool, Field and ModelMeta code is already in the starting code. You only add the square function and the Pool.map call. This layer will be combined with ctypes in the next lesson.
Practice Compute squares in parallel with Pool
The starting code already has Tool, Field and ModelMeta. Now import with from multiprocessing import Pool. Write a square(x) function returning x * x. Inside a with Pool() as p: block assign p.map(square, [1, 2, 3]) to result. Print result.
Starter code
# Build the Tool class with type()
Tool = type("Tool", (), {
"name": "arac",
"run": lambda self: f"{self.name} calisti",
})
# Add Field descriptor and ModelMeta
class Field:
def __set_name__(self, owner, name):
self.name = name
if not hasattr(owner, "fields"):
owner.fields = []
owner.fields.append(name)
class ModelMeta(type):
def __new__(mcs, name, bases, ns):
return super().__new__(mcs, name, bases, ns)
class Model(metaclass=ModelMeta):
ad = Field()
# Add parallel square computation
# write square and use Pool
# print the resultSample solution
# Build the Tool class with type()
Tool = type("Tool", (), {
"name": "arac",
"run": lambda self: f"{self.name} calisti",
})
# Add Field descriptor and ModelMeta
class Field:
def __set_name__(self, owner, name):
self.name = name
if not hasattr(owner, "fields"):
owner.fields = []
owner.fields.append(name)
class ModelMeta(type):
def __new__(mcs, name, bases, ns):
return super().__new__(mcs, name, bases, ns)
class Model(metaclass=ModelMeta):
ad = Field()
# Add parallel square computation
from multiprocessing import Pool
def square(x):
return x * x
with Pool() as p:
result = p.map(square, [1, 2, 3])
print(result)Expected output
[1, 4, 9]
Next step: ctypes
Parallel square computation is ready. In the next lesson you will add a native code layer: calling abs with ctypes.CDLL.
This layer will add the ability to call functions from a C library. So far the Tool, Field, ModelMeta and Pool layers are ready. The final lesson will combine everything.
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Complete the Pool call
with ____() as p:
result = p.map(square, [1, 2, 3])with Pool() as p:
result = p.map(square, [1, 2, 3])Explanation: with Pool() as p creates a process pool.
2 Match multiprocessing concepts
| Term | Meaning |
|---|---|
Pool |
Process pool |
square |
Square function |
map |
Applies in parallel |
result |
Result list |
3 What does Pool.map provide?
- Randomly ordered result
- Parallel result preserving input order Correct answer
- A single value
- An error
Explanation: Pool.map runs in parallel but preserves input order; the output is deterministic.