php|architect's Guide to Web Scraping with PHP - Wind Business ...

A p p e n d i x A
Legality of W e Scraping b
The legality of web scraping is a rather complicated question, mainly due to copyright
and intellectual property laws. U n f o r t u n a t e l y,there is no easy and completely
cut-and-dry answer, particularly because these laws can vary between countries.
There are, however, a few common points for examination when reviewing a
prospective web scraping target.
First, web sites often have documents known as T e r of Service m s (TOS), T e or m s
Conditions of U s e , or U s e r Agreements (hereafter simply known as TOS documents
for the sake of reference). These are generally located in an out-of-the-way location
like a link in the site footer or in a Legal Documents or H e l p section. These types
of documents are more common on larger and more well-known web sites. Below
are segments of several such documents from web sites that explicitly prohibit web
scraping of their content.
• “You specifically agree not to access (or attempt to access) any of the Services
through any automated means (including use of scripts or web crawlers)...” –
Google T e r of Service, m s section 5.3 as of 2/14/10
• “You will not collect users’ content or information, or otherwise access F a c e -
book, using automated means (such as harvesting bots, robots, spiders, or
scrapers) without our permission.” – F a c e b o o k St a t e m e n t of Rights and Responsibilities,
Safety section as of 2/14/10