php|architect's Guide to Web Scraping with PHP - Wind Business ...

84 ” Rolling Y o u Own r
Logic to separate individual headers must account for the ability of header values to
span multiple lines as per RFC 2616 Section 2.2. As such, preg_match_all is used here
to separate individual headers. See the later chapter on PCRE for more information
on regular expressions. If a situation necessitates parsing data contained in URLs
and query strings, check out the parse_url and parse_str functions. As with the request,
it is generally desirable to parse response data into a data structure for ease of
reference.
T r a Encoding
s f e r
Before parsing the body, the headers should be checked for a few things. If a
Transfer-Encoding header is present and has a value of chunked, it means that the
server is sending the response back in chunks rather than all at once. The advantage
to this is that the server does not have to wait until the entire response is composed
before starting to return it (in order to determine and include its length in the
Content-Length header), which can increase o v e r a l l server throughput.
When each chunk is sent, it is preceded by a hexadecimal number to indicate the
size of the chunk followed by a CRLF sequence. The end of each chunk is also denoted
by a CRLF sequence. The end of the body is denoted with a chunk size of 0,
which is particularly important when using a persistent connection since the client
must know where one response ends and the next begins.
The strstr function can be used to obtain characters in a string prior to a newline.
T oconvert strings containing hexadecimal numbers to their decimal equivalents,
see the hexdec function. An example of what these two might look like in action is
included below. The example assumes that a request body has been written to a
string.