Bing Discovers 12s Of Billions Of URLs It’s Never Seen Before Every Day

Fabrice Canel from Microsoft said that each day Bing discovers 12s of billions of normalized URLs never seen before. That is a lot of new URLs for BingBot to find in a single day, don’t you think?

But the web is big and content is constantly being produced, not just quality content but a lot of junk, gibberish, machine-generated content, and so forth.

Fabrice explained on Twitter that most of the content is “mostly useless content,” he listed examples such as duplicate content, scraped content, automatically generated content, spam content, junk content, and more.

So while Bing may discover billions and billions of new URLs per day, I doubt it indexes much of it.

Here are those tweets:

Forum discussion at Twitter.

Leave a Comment

Your email address will not be published.

YouTube
YouTube