Beginning Web Development With Perl : From Novice to ... - Nabo

98CHAPTER 5 ■ LWP MODULESThis example uses the get() function from LWP::Simple, which enables you to quickly andeasily retrieve a web page using the GET method, as explained earlier in the chapter. The programwill perform a GET against the web page at http://www.braingia.org/, and then searchfor some text within the page, including any HTML, scripts, or other material returned. If thattext is found, the program will print a simple message to STDOUT indicating that it found thetext, something like this:I found the textThe choices for working with the resulting text from the get() function are limited only bywhat you would like to do with the results.Setting Additional ParametersThe get() function works well for simple GET method requests. However, some sites requireyou to set additional parameters, such as authentication, user agent, and other values. Whenyou need to set these additional parameters, use the LWP::UserAgent class.Consider the example in Listing 5-5 (Getua.pl), which performs a GET on a URL and alsosets the agent parameter.Listing 5-5. Setting a User Agent and Retrieving a Web Page#!/usr/bin/perl -wuse LWP;use strict;my $browser = LWP::UserAgent->new(agent => 'Perly v1');my $result = $browser->get("http://www.braingia.org/ewfojwefoj");die "An error occurred: ", $result->status_line( ) unless$result->is_success;#Do something more meaningful with the content than this!print $result->content;You may recognize this as the example I showed you at the beginning of this chapter.The program will report itself as “Perly v1” to the web server. You can use this to mimic anyweb browser or make up your own, as shown in the example. The output from this programis raw HTML and JavaScript, as shown previously.■Note For more information about user agent strings, see the appropriately titled “User-Agent Strings”document at http://www.mozilla.org/build/revised-user-agent-strings.html.Setting TimeoutsSometimes, the web server is slow to respond, or other network-type issues cause the browserto time out. You can set the timeout of the browser to a value appropriate for your application.