Summary - Asyncio
- Python course
- Advanced
- Lesson 36/83
- 5 min
Combine gather, Queue and Semaphore to write a web scraper.
Summary - Asyncio
This group covered asyncio's concurrent tools. gather awaits all; wait gives done/pending sets; Queue for producer-consumer; Lock protects a critical section; Semaphore controls the concurrency limit.
The final task combines gather and Semaphore: you'll await several coroutines limited by Semaphore.
Practice gather + Semaphore combined
Start with import asyncio. Define sem = asyncio.Semaphore(2). The function async def fetch(n) returns n * 2 inside async with sem:. Inside async def main() call return await asyncio.gather(fetch(1), fetch(2), fetch(3)). Add print(asyncio.run(main())).
Starter code
# import the asyncio module
# define a two-permit semaphore (Semaphore with value 2)
# fetch(n): inside async with sem, return n doubled
# main: gather three fetch calls, print resultSample solution
import asyncio
sem = asyncio.Semaphore(2)
async def fetch(n):
async with sem:
return n * 2
async def main():
return await asyncio.gather(fetch(1), fetch(2), fetch(3))
print(asyncio.run(main()))Expected output
[2, 4, 6]
Questions and answers
Review the lesson questions together with their answers. To test yourself, solve the lesson interactively.
1 Which function awaits all completions?
- asyncio.wait
- asyncio.Queue
- asyncio.gather Correct answer
- asyncio.Lock
Explanation: asyncio.gather awaits all coroutines.
2 Summary: Semaphore
sem = asyncio.____(2)sem = asyncio.Semaphore(2)Explanation: Semaphore(2) creates a two-permit semaphore.
3 Match the group tools
| Term | Meaning |
|---|---|
gather |
Awaits all |
wait |
Gives done/pending |
Queue |
Passes data |
Semaphore |
Limits permits |