Optimized Web Crawling

Published: Oct. 28, 2018, 11:56 p.m.

b'Got a fun optimization problem for you this week! It\\u2019s a two-for-one: how do you optimize the web crawling logic of an operation like Google search so that the results are, on average, as up-to-date as possible, and how do you optimize your solution of choice so that it\\u2019s maintainable by software engineers in a huge distributed system? We\\u2019re following an excellent post from the Unofficial Google Data Science blog going through this problem.\\n\\nRelevant links: http://www.unofficialgoogledatascience.com/2018/07/by-bill-richoux-critical-decisions-are.html'