Java Concurrency Interview

Concurrency in real systems question 127 of 180 senior

Your service has 200 HTTP threads and a pool of 10 database connections. Where is the bottleneck?

The scarcest resource sets the throughput. With 10 connections, at most 10 requests are doing database work at any moment; the other 190 threads are either idle or queued inside the pool's borrow call, holding memory and a request each. Raising the HTTP pool to 400 doubles the queue and the latency and changes nothing about throughput.

The right moves, in order: put a timeout on connection acquisition so a saturated pool sheds load rather than piling up, size the connection pool from the database's capacity and the query time (a small number, often smaller than people expect), and make the HTTP pool a similar order of magnitude so backpressure reaches the client instead of being absorbed into a queue.

This is also the answer to "will virtual threads fix it". They will not: a million virtual threads still queue on 10 connections. What changes is that the queue is cheap and visible, instead of being an invisible limit imposed by a thread pool.

scenario pools backpressure

Quiz yourself on Concurrency in real systems Suggest a correction