Concurrency in real systems question 127 of 180 senior
Your service has 200 HTTP threads and a pool of 10 database connections. Where is the bottleneck?
- The HTTP threads, add more
- Correct: The connection pool, and adding HTTP threads only moves the queue to a worse place
- The garbage collector
- The database, always
The scarcest resource sets the throughput. With 10 connections, at most 10 requests are doing database work at any moment; the other 190 threads are either idle or queued inside the pool's borrow call, holding memory and a request each. Raising the HTTP pool to 400 doubles the queue and the latency and changes nothing about throughput.
The right moves, in order: put a timeout on connection acquisition so a saturated pool sheds load rather than piling up, size the connection pool from the database's capacity and the query time (a small number, often smaller than people expect), and make the HTTP pool a similar order of magnitude so backpressure reaches the client instead of being absorbed into a queue.
This is also the answer to "will virtual threads fix it". They will not: a million virtual threads still queue on 10 connections. What changes is that the queue is cheap and visible, instead of being an invisible limit imposed by a thread pool.
Quiz yourself on Concurrency in real systems Suggest a correction