9.2.3 Automatically Filling a GET Form Using HTTP::Request

9.2.3  Automatically Filling a GET Form Using HTTP::Request

In section 9.1.2, we see how a form that uses the GET method can be automatically filled using the

LWP::Simple module. We see in that section that filling a GET form and obtaining response is very simple. Of course, the same can be done with the more complex LWP modules such as LWP::UserAgent, HTTP::Request, HTTP::Response and others.

A GET form request is sent to the Web server in a single transmission as a URL. That is why processing GET forms is easy. The following program is similar in objective and functionality to the program described in 9.1.2.

 Program 9.8

#!/usr/bin/perl
#file bordersISBN2.pl

use strict;
use HTTP::Request;
use LWP::UserAgent; 

my ($url, $content);
my ($ua, $contentRequest, $contentResponse, $price);
my $ISBN = "1565922433";
print "ISBN = $ISBN\n";

#Make up the URL to search for the book's ISBN
$url = "http://search.borders.com/fcgi-bin/db2www/search/search.d2w/Details?";
$url .= "code=$ISBN&mediaType=Book&searchType=ISBNUPC";

$ua = LWP::UserAgent->new();
$contentRequest = HTTP::Request->new(GET=>$url);
$contentResponse = $ua->request ($contentRequest);
if ($contentResponse->is_success){
    $content = $contentResponse->content;
}
else{
    print $contentResponse->error_as_HTML;
    exit 0;
}
($price) =  ($content =~ m#Our Price:.+?\$(.+?)<#si);
print "price = $price\n";

The program is almost exactly like the programs we see in Section 9.2. The program uses the

LWP::UserAgent module to create a new user agent called $ua. The request is formed in terms of a URL where the form parameters follow the ? mark following the URL. Although we have a statement

use HTTP::Request;

on the top of the program, it is not necessary to use the HTTP::Request module. The form is automatically submitted by creating an HTTP::Request object with the URL as the argument. Note that, in general, it is not necessary to convert the URL to a URI before sending it to a Web server, as we have done in examples in previous sections.

The response that comes back from the server is captured by the user agent $ua. The response is automatically an HTTP::Response object. If the response has a successful status code, the HTML content of the content is saved in the variable $content. This variable is later parsed to obtain the price of the book with the ISBN number given in the variable $ISBN.

There is another module called HTTP::Request::Common that makes it slightly more convenient to write common HTTP requests. For GET forms, we need to simply set up a user agent and then issue a GET request as given below.

$contentRequest = GET $url;

Thus, it is as simple as the LWP::Simple module to use. The GET $url call automatically sets up the appropriate HTTP::Request object. It also captures the response coming back from the Web server automatically. A program given earlier in this section can be rewritten as shown below.

 Program 9.9

#!/usr/bin/perl
#file bordersISBN4.pl

use strict;
use HTTP::Request::Common;
use LWP::UserAgent; 

my ($url, $content);
my ($ua, $contentRequest, $contentResponse, $price);
my $ISBN = "1565922433";
print "ISBN = $ISBN\n";

#Make up the URL to search for the book's ISBN
$url = "http://search.borders.com/fcgi-bin/db2www/search/search.d2w/Details?";
$url .= "code=$ISBN&mediaType=Book&searchType=ISBNUPC";

$ua = LWP::UserAgent->new();
$contentRequest = GET $url;
$contentResponse = $ua->request ($contentRequest);
if ($contentResponse->is_success){
    $content = $contentResponse->content;
}
else{
    print $contentResponse->error_as_HTML;
    exit 0;
}
($price) =  ($content =~ m#Our Price:.+?\$(.+?)<#si);
print "price = $price\n";