This post was originally published on this site
In 2022, a US Circuit Court of Appeals answered the “is web scraping legal” question once and for all… sort of. Since that ruling (which held that web scraping is legal, assuming you scrape publicly available information) in 2022, though, there’s been a major development in the world of web scraping: the rise of generative AI.
About eight months after the court system reaffirmed its earlier ruling on web scraping (in a case brought by LinkedIn against hiQ Labs), ChatGPT exploded onto the scene. And in the months since that, revelations have emerged about what OpenAI used to train the model ChatGPT is built on (briefly: lots of copyrighted material).
While the rise of generative AI may complicate the conversation around web scraping, it doesn’t change the fundamentals of how we’ve been using it to support our clients. So we thought we’d do a little explainer here on the blog.
Web Scraping Doesn’t Violate the CFAA
At issue in the LinkedIn vs. hiQ case from 2022 was whether hiQ’s scraping of LinkedIn profiles (to gather data on employee attrition) violated the Computer Fraud and Abuse Act, or CFAA. That law, passed in 1986, was initially intended to prevent people from
