Podcasts Technology The Backend Engineering Show with Hussein Nasser GoogleBot Crawler Now Uses HTTP/2 to Index the Web, Let us discuss how this affects our Back-end?

GoogleBot Crawler Now Uses HTTP/2 to Index the Web, Let us discuss how this affects our Back-end?

Published: Sept. 18, 2020, 8:47 p.m.

Google Bot is now attempting to use HTTP/2 to crawl the web

Ever since mainstream browsers started supporting the next major revision of HTTP, HTTP/2 or h2 for short, web professionals asked us whether Googlebot can crawl over the upgraded, more modern version of the protocol.

Today we're announcing that starting mid November 2020, Googlebot will support crawling over HTTP/2 for select sites.

Article

https://webmasters.googleblog.com/2020/09/googlebot-will-soon-speak-http2.html

* Intro 0:00

* What is a Crawler 1:06

* Current Crawler uses h1 2:00

* Crawler now uses h2 4:40

* Less connections, slightly high CPU usage 6:30

* Opting out of h2 crawling 9:00

* FAQ 11:40

Http/2 playlist 0;50 https://www.youtube.com/playlist?list=PLQnljOFTspQWbBegaU790WhH7gNKcMAl-

http/2 not cheap 7;00 https://www.youtube.com/watch?v=GriONb4EfPY

lucid chart 9;50 https://www.youtube.com/watch?v=gejfT1h6LBo

h2c smuggling 18:30 https://www.youtube.com/watch?v=B2VEQ3jFq6Q

--- Send in a voice message: https://anchor.fm/hnasr/message