How to identify OpenAI’s crawler bot to stop it slurping websites for training data
by #AI August 8, 2023 · Automatic
Aww, c’mon, let us scrape your pages, we’ve got billions at stake
OpenAI, the maker of machine learning models trained on public web data, has published the specifications for its web crawler so that publishers and site owners can opt out of having their content scraped.…
Author: Thomas Claburn. [Source Link (*), The Register]
Shop with us!
 |
|
