Loading…
While this loads — quick one
Every subscriber must get each event. A queue, or pub/sub?
Message queueorPub/Sub
Loading…
While this loads — quick one
Every subscriber must get each event. A queue, or pub/sub?
Loading the challenge…
While this loads — quick one
Every subscriber must get each event. A queue, or pub/sub?
A search company needs to download a billion web pages a month and keep them fresh, starting from a list of known sites and following the links it finds. The same page is often reachable through many different links, and fetching it twice wastes a great deal of work. Sites must be treated politely: never more than one request every few seconds to the same site, or the crawler gets blocked. Fetching is slow and unpredictable, since some sites answer in milliseconds and others time out, so hundreds of machines work in parallel, and a machine that crashes must not lose the pages it was about to fetch. The downloaded pages are large and kept for later processing.
Every system design question opens with an account. Your design is scored the same way every time, with the reason behind each requirement, and your attempts are kept so you can see what you found hard.