Here’s an interesting idea from Joost, to change to having an allow list for site crawlers and blocking all others.
Proposal to Switch to an Allow List for Site Crawlers
Joost de Valk @ twitter.com • 9 months ago
Joost de Valk @ twitter.com • 9 months ago
Here’s an interesting idea from Joost, to change to having an allow list for site crawlers and blocking all others.