php|architect's Guide to Web Scraping with PHP - Wind Business ...

cURL Extension ” 41
that file and use it as the value for the Cookie request header when the request
is constructed.
• If $cookiejar is set to an empty string, cookie data will persist in memory rather
than a local file. This impro ves performance (memory access is faster than
disk) and security (file storage may be more open to access by other users and
processes than memory depending on the server environment).
In some instances it may be desirable for the CURLOPT_COOKIEJAR value to have
a different value per request, such as for debugging. In most cases, ho wever,
CURLOPT_COOKIEJAR will be set for the first request to receive the initial cookie data
and its value will persist for subsequent requests. In most cases, CURLOPT_COOKIEFILE
will be assigned the same value as CURLOPT_COOKIEJAR after the first request. This will
result in cookie data being read to include in the request, follo wed by cookie data
from the response being written back (and o verwriting any existing data at that location)
for use in subsequent requests. On a related note, if you want cURL to begin a
new session in order to have it discard data for session cookies (i.e. cookies without
an expiration date), you can set the CURLOPT_COOKIESESSION setting to true.
If you want to handle cookie data manually for any reason, you can set the value
of the Cookie request header via the CURLOPT_COOKIE setting. To get access to the response
headers, set the CURLOPT_HEADER and CURLOPT_RETURNTRANSFER settings to true.
This will cause the curl_exec call to return the entire response including the headers
and the body. Recall that there is a single blank line between the headers and
the body and that a colon separates each header name from its corresponding value.
This information combined with the basic string handling functions in PHP should
be all you need. Also , you ’ll need to set CURLOPT_FOLLOWLOCATION to false in order
to prevent cURL from processing redirections automatically. N ot doing this would
cause any cookies set by requests resulting in redirections to be lost.
HTTP A uthentication
cURL supports both Basic and Digest HTTP authentication methods, among others.
The CURLOPT_HTTPAUTH setting controls the method to use and is set using constants
such as CURLAUTH_BASIC or CURLAUTH_DIGEST. The CURLOPT_USERPWD setting is a string