.Google's Gary Illyes discloses Googlebot gathers web links as opposed to "complying with" all of them, contradicting official information.Googlebot gathers hyperlinks as opposed to following them in real-time.Gary Illyes discussed this info in Google's 'Explore Off The Document' podcast.Google.com's information contradicts the genuine spider habits.