About this episode
In this episode of Search Off the Record, Gary Illyes and Lizzi Sassman take a deep dive into crawling the web: what is a web crawler, and how does it really work? Listen along as the Search team is joined by an expert web developer in the SEO community, Dave Smart, for an in-depth and technical discussion of all things crawling, and maybe dispel some myths along the way.
Resources:
Episode transcript → https://goo.gle/sotr070-transcript
Managing your crawl budget → https://goo.gle/3IzRZxl
Dave Smart on LinkedIn → https://goo.gle/3wPSuRA
Tame the Bots → https://goo.gle/4cfCQ1P
Search Central Help Forum → https://goo.gle/sc-forum
Indexing API docs → https://goo.gle/3v8yVU0
Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team.
#SOTRpodcast
Speaker: Gary Illyes, Lizzi Sassman
Products Mentioned: Search Off The Record
Get every episode summarized
Each time Search Off the Record publishes, we email you a written briefing from the transcript — the topics, who appeared, and any specific claims, with the ad reads skipped.
Email me new episodesFree for 3 shows. No card needed.
Hosts & guests
No transcript yet
This episode has not been transcribed. Request it and it moves to the front of the queue.
More episodes
More from Search Off the Record
Are websites getting "fat"? Page weight, HTML size & Googlebot limits explained...
Search Off the Record
Are websites getting "fat"? Page weight, HTML size & Googlebot limits explained
Search Off the Record
Google crawlers behind the scenes - transcript
Search Off the Record
Google crawlers behind the scenes
Search Off the Record